Devloop: random notes on the development and architecture

Like everything that happens nowadays in tech, this article starts from AI :)

Lately, I've been generating a lot of code using LLMs and agents. The raw output gets the job done, but during the maintenance phase, I've found that the resulting code and overall codebase understandability are... let's say, not ideal.

This led me to search for ways to structure code that can act as a "cage" for AI - constraining it to reduce bugs and minimize the time I spend manually correcting its output.

LLMs can output tremendous amounts of code per iteration, but when we need quality and workability that meets our development goals, the only "tool" we can rely on is human review - which is inherently slow. The more noise there is in the AI's output, the more mental detours our brains must take, and the more issues slip through unnoticed.

I built custom linters and additional codebase checks, which act as automated feedback for AI agents during the generation loop.

Which partially improved the situation, but still, they could not cover issues in logic, especially in projects that are logic-heavy.

Here's what I mean by logical errors: In one code review, I missed that AI had confused entity IDs deep within the business logic, leading to very annoying and hard-to-debug issues. In another case, AI treated admin users and public users as interchangeable, attempting to fetch public user profiles for admin users 🤦

These are exactly the kinds of problems that linters can't catch - they're logical, not syntactic. This got me thinking about applying Domain-Driven Design (DDD) principles to make the code more "sound" and restrictive, ideally improving the LLM's ability to self-correct.

Recently I was lucky enough to run an experiment of developing a project using purely AI tools.

Instead of a team of developers it was only me, with AI tools of my choice.

At the moment the experiment is 4 months old :) and continues, with pure effort spent on this project I'd say at least 3 months and 1 week.

There were complications involving me needing to touch other services within the company codebase, owned by other teams, some of them critical like payments, which took a good piece of overall timeline, but the core is a single service with front pages, auth hidden pages, admin, some internal APIs, workers and cron jobs.

So pretty much typical project.

At some point in time I started to doubt whether I output results at the full possible speed.

A lot of people saying that they did with AI something x10 faster than it would be done without before, so reasonably this project shouldn't take 40 developer/months without using AI agents. I changed approaches and AI tools during this experiment, and at the moment I have a setup where I'm fully dedicated to review of that gigantic AI output and I cannot find where I can find x2..3...4...etc speedup.

And while measuring of the development speed is something that's not solved well in the industry at the moment I still want to try to compare a process before heavily relying on AI agents and current approaches in terms of how much time it saves.

Typical approaches taken in the organization to measure velocity wouldn't work here as I do not have story points, estimations in hours, team, etc.. So I'll try to measure a one task purely relying on my understanding of my process of development with AI agents and without.

Recently, I got tasked with integrating another internal API into an LLM-based agent that I was working on. The application already had several APIs connected, but this new addition was crucial, and I was looking for a way to automate the integration process. The challenge wasn’t just about hooking up the API—it was about transforming the complex internal structures into something more digestible for the LLM. Basically, the API needed a makeover: stripping out internal entities, identifiers, and making the data coherent and contextually relevant for the AI.

Now, generally speaking, adding a new API is a pretty clear-cut process—parse the endpoints, connect the dots, and fit it into the application. But it's also a time-consuming routine. It’s not just about writing the code but ensuring everything aligns perfectly, which eats up a lot of time, especially when scaling this up to integrate more APIs quickly. We wanted to simplify this routine, making it more efficient and repeatable.

I’ve been using GitHub Copilot and ChatGPT extensively in my coding routine, and they’ve been great. But for this task, I wanted to see if there was a better, more streamlined way. That’s when I decided to experiment with Cursor Editor and a custom script approach, hoping they could give me that extra edge.

If you missed OpenAI's Dev Days presentation, you missed out on Sam Altman, the CEO, killed many startups has shown updates to OpenAI and ChatGPT.

Haven't seen it? Do yourself a favor and watch it here. It's not just a presentation; it's a glimpse into the future, at least until the next big thing rolls around.

Post-presentation, like many others, I had a lightbulb moment. Why not feed all our internal organization docs to a GPT and let it do the heavy lifting? Then just kick back, relax, and sip coconuts on an island somewhere.

Turns out, it's not as easy as Sam made it look. If your company's processes and org structure resemble a labyrinth, you'll need to roll up your sleeves. It's not all AI magic and coconut sipping, unfortunately. But at the same time, it is much-much-much-.....-much easier than it was before. We just need a little bit of wizarding around crafting precise instructions to GPT.

As of writing this, internet does not have a lot of takes on how to make it work. So, I decided to put my two cents in and draft one.

Asynchronous data loading is a pervasive challenge in modern web development.

When using React and MobX, the complexity can quickly escalate, giving rise to questions about type safety, error handling, and boilerplate code.

In this article, we won't just talk about these challenges; we'll solve them.

Through a targeted design session, we will dissect the problem and build a type-safe and efficient solution for asynchronous data loading in React and MobX applications.

Additionally, this article aims to offer more than just a solution; it aims to walk you through the thinking process involved in tackling a problem that lies at the intersection of technology and application design.

Sooner or later, companies reach a point where architects are needed, but it can be difficult to hire them on the market, so they have to be trained in-house. Training takes place when someone within the company wants to move towards architecture or when promising people are found in the labor market, if, for example, a person falls slightly short of the required position.

Teaching architects is, in my opinion, a rather difficult task because architecture is poorly formalized and heavily dependent on the organization's context. Since I do not consider myself a specialist in training, this post will change from time to time as I make mistakes and find better approaches.

Over the years, I've seen many different models for organizing work on infrastructure. In this article, I would like to systematize what I've seen and briefly describe when and why different options should be preferred to others.

It is important to note that infrastructure refers to work with hardware, cloud, hosting, etc., which is usually referred to as DevOps.

Each section describes one model, its pros and cons, risks, and when to prefer other models.

So, we came to the point where at the beginning we discuss the requirements that need to be taken into account in the decision before going into any technological specifics.

The team agreed on the goals and the points that need to be achieved. Now we finally need to move on to the solution and discuss one or another idea.

Suppose we have several solutions and all of them already meet the requirements. Solutions that do not meet the requirements are not considered by default.

More precisely, if we imagine collective work - then we gently remind that some requirement is not considered in the solution and it needs to be either expanded or not considered.

I constantly hear that in order to write tests, you need to increase the time estimate for the task by 1.2, 1.5, 2, 3 times.

Usually, I don't argue with this, since the team knows best, but inside me there is often a simmering feeling, or at least a bubbling one.

I would like to explain my view on automated tests and why I believe that such an increase is not entirely correct.

Very often, organizations experience local conflicts between developers and infrastructure teams that need to be resolved afterwards.

Since I am often brought in to resolve such conflicts, I would like to share my view on why these conflicts arise and what needs to be done,

in my opinion, to significantly reduce their number.

How did DevOps come about?

We all know why such a marketing name as DevOps was invented. In case someone doesn't know...

I think many of you have worked in a team and have faced heated discussions when making a decision. Usually, there is a person in the team who has the final say, for example, the tech lead. But sometimes there are disagreements that are quite difficult because the team consists of senior people, each of whom has many arguments why their opinion should be heard.

If there is only one architect on the project, then this problem is less pronounced in their work because there is an option, in case of equivalent solutions, to keep the last word to themselves. Not always their option, but it is important that there is an opportunity to stop the argument if it exists.

However, if there is a team of architects on the project, then an interesting game begins here. Even if there is a lead or a CTO among them, the decision is still an exhausting activity. Everyone understands that architects are specialists who know their job, and it is necessary to try to listen to everyone, but since everyone has a bunch of arguments for their options and in opposition to the opponent, a lot of energy is spent.

Once, I was in such a situation for a very long time and at one point, I got tired of it and started thinking about how to get out of it.