Hello everyone and welcome to this video / lecture in which I will try to understand the big question that we have here on the screen and that is is clean code dead in the age of AI. And this is a question that I've been fighting and grappling with for about 2-3 weeks now and it's interesting because really the game has changed. We are writing code in a new way. We are using mostly agents to actually write our code at least my workplace.

And also agents have a different way of understanding code than humans do and a lot of the clean code practices and the clean architecture practices have actually been built around to make code more understandable for humans. But that does not necessarily make it the case that it's more understandable for agents. And with the experiments that I run which I'll walk you through throughout this video lecture, let's call it.

The idea is to understand the data and to be able to see if some of these practices are actually hurting us or if they're costing more and to get to come up with a framework or an opinion on how we should be adjusting these practices and not just blindly following rituals that haven't been designed for the age that we are living in. So generally what does architecture buy, right? It's always has been an investment of some sorts.

You invest more up front but it should make your code generally more maintainable, easier to maintain and easier to add new features as time goes on, right? Because if you do not have any good if you do not have good architecture and structure to your project, then it will probably become a jumbled mess and you won't really be able to write in it. You won't be able to understand it. You won't be able to debug it. And this has been the thing that architecture has bought us all this time.

However, as I said previously, uh agents are different than humans. Uh they have superhuman capabilities in terms of let's say they can load up a class that is 5,000 lines of code and they can understand that in an instant. Something that humans could not do previously. So, really I started with belief I want to test because actually I was one of the biggest proponents at my workplace at least for clean code because I uh let's say I understood the benefits that uh came with it.

However, let's see uh the data and actually let's see what the experiment shows and maybe it's time that I change my opinion and stance on on these things. As as I said, the environment truly has changed. So, uh let's begin and I'll walk you through the experiment and what I did. Uh essentially, I needed uh some example business domain of an application and I decided uh called it charger ledger. The idea is that it's just like uh charging EVs, uh monitoring like how much uh uh what the cost is, uh calculating bills, recording sessions, issuing invoices, etc.

It's just some sample domain. We don't really need to get too deep into it. However, the idea is that I would create uh using this domain two projects, one with structure, one without, and I'll start adding feature. And here I've wrote them like some sample user stories, right? That we will get throughout uh the the journey and I have started with a scope of nine features that we want to implement. So, we have two starting points as I alluded to previously.

We have one project where I did not prescribe any architecture to it. I just let the agent run wild. And I have another project which we call hexagonal where I set deliberately set out to create like some basic boundaries uh for the project to uh define ports and adapters. Pretty much that's the architecture. And uh enforce it actually through ArchUnit tests that you can imagine are like checking if all the boundaries and dependency directions are uh compliant with the architecture so that it gives the agent structure and a clear direction of where it needs to go.

So, here just as a very short and compressed uh reminder on what hexagonal architecture is. As I said, ports and adapters. Essentially, you have your core and your business domain as a first-class citizen. It sits in the middle of your application and this is where you have your business logic and everything. And you define these ports which are pretty much contracts. They're You can think about them like they're just interfaces that are implemented by adapters.

And these adapters can be like an HTTP adapter so you can receive HTTP request. It could be again in this picture an example would be an SQLite adapter so that you have persistence using SQLite. You do not change any of your business logic and that is a very clear separation of concerns and responsibilities. And it's overall a good architecture, let's call it, for long-term projects that would have a lot of business logic and also need to be very, let's say, flexible and requirements would be changing all the time on what technologies you'll be using around them.

And also just to have a full uh coverage of the experiment, how I'm running it. So, I'm using ChatGPT-6, uh the new Astra model, as my orchestrator. So, it has been my planner. It pretty much generates a all of the business requirements that we need, all of the features, helps me record everything that we've run. Then I have an open code instance running that is connected to quant 3.8 flash next that is running on my DJ Spark.

It's a local model for coding and I'm hosting it myself, so I don't need to pay for tokens. And what we're doing is that we're generating specifications for each of the new features as we discussed. We use the same model and harness for both of the projects. And then we have created an external acceptance suite that essentially accepts or declines if a feature has been implemented properly. So this is like our arbiter.

So in order for a feature to be accepted, it needs to pass the external acceptance which is one and it tests both projects. Here is pretty much the acceptance loop. We give the specification to an agent. The agent is going to do the code and when it says I'm done, we're going to run the external suite and if it's green, we just accept it and we mark the feature as finished and correct. And if it does fail, then we send it for repair.

Essentially, we re-prompt the agent so that it tries to fix the mistake. And we're measuring mostly time here, but we'll also be measuring a lot of things like tool costs and token counts. However, what the the clock includes is the agent work plus the the time it runs it takes to run the validation, the external suite and also any extra repair. So if we send the agent back to fix something broken from the external suite, this is also going to be included in the wall in the clock timer.

We have these two projects. We have our baseline, we have the hexagonal one and uh at what point do you think that uh structure actually starts to pay off, right? This has been the idea that over time the project will become a mess if we do not add any deliberate structure to it. So it would be harder and harder to add features. At least this has been the way with human workers, right? Because when the project becomes too big, we just cannot comprehend it, we cannot work on it well.

So it takes us more time to add a new feature. However, in our case, so we had 18 accepted runs. So all features were written. The payoff that I expected did not appear actually. So here is a line graph of the times it took for each individual feature from one to nine. And as you can see, we have a few features where flat took a little more time. However, overall hexagonal was slower than our flat project is. And actually it was quite noticeably slower.

It took 38% longer time to complete the nine features. And here we can see in terms of work that we used more tokens, we outputted more tokens, we had a lot more model steps and tool calls. Actually both had two repairs by our orchestrator, which means that that front is equal. However, really take a look at these numbers. Around 50% more input tokens and around 30% more output tokens. However, there are challenges to this experiment and this wasn't the only thing that I've done, right?

Because maybe just the work was too easy, so it actually didn't matter how well you structured your code. So that makes sense. And for that reason, I we generated with Astro, we generated a few let's let's call them challenges, which with our harder features that had to be implemented, we took a checkpoint point from feature nine. So, we already have some more bulky code base. And here you can see the features. Actually, it doesn't really matter what the challenges are.

It's just that they are let's say more tough to figure out features with more requirements on them. And yes, here as I said, we took the checkpoint at feature nine and we run each of the challenges in parallel and these are the numbers. So, here we got a little bit of a mixed result. So, really with the highest um complexity tasks, we are seeing that Hexagonal was faster two out of six times and the other four were for flat.

And you can see that the times are very similar. Here what I expected again was a big deterioration in the flat project. So, that implementing a hard feature it just wouldn't allow the agent to either finish it or it would take like double the time. But we can see that it actually didn't. And here are the the whole numbers and we can see a similar picture here as with the first nine features. We are spending more input tokens, output tokens are more, so And another challenge here.

So, it First challenge was complexity, right? We didn't have enough complex tasks. So, maybe the repositories were too small. Maybe we're not at that production scale where it actually matters. So, for that reason, what we did is we did another campaign and we have these S features. It's just features that just create scale. They give scale to the repository. So, we're going to take a checkpoint from our second challenge.

I picked that one because it was just the biggest in terms of Java code. As we're looking for scale, we're measuring like lines of production Java code. And we're going to run 15 of these feature requests. That That point is to make the project more complex and bigger and bulkier. And then we have a special 16th one, which I'll get to in a minute. So, here I again have it in the story format. I don't really think it matters what the features are.

It's more about how much time it took to implement them or if we finally saw this degradation that we've been looking for so that the repo becomes too big, too messy, and the agent cannot reason its way throughout it. However, then another line graph, we see that actually it looks like the flat implementation is perfectly fine for the agent to reason with, and hexagonal is taking more time again. And it's again significant amount of time.

It took 30% longer to finish the hexagonal project than it did the flat one. And as you can imagine, the token counts, it's the same thing, really. You can see input tokens about 50% higher from 80 to 126 million. Uh output tokens are like 40% uh roughly, just calculating it in my head. Again, this is something you pay for. Uh we see a lot more model steps, and we have like 50% more tool calls. Again, it makes sense.

The hexagonal project is structured, but has more indirection. It has more abstractions. So, an agent has to go, it has to read the file, then it has to follow what this file references to read another file, and so on and so on. It needs to collect all the context so it's able to work on it. However, in the flat project, it just reads one file that has all the logic garble inside of it, and it actually can understand what's going on, and it can prepare and to do the change.

And here we can see like the scale we are talking at here. I would say that So, at C 2, obviously were small, like 4 to 5K lines of production code. It's What I'm saying production code is because I'm just excluding test suites. Uh for for this statistic, we can see that after 15 of these scale features, we've actually raised the number of lines to quite a lot, 12 and 15K. And I would say 12 and 15K are becoming representative of a production service.

Like, if you're running microservices, probably you would even have smaller microservices than 15K lines of of production code. And you can also see that the number of files, like the difference is enormous. So, at S15 checkpoint, the flat one has 84 files, again production Java files, and the hexagonal one has 239 files. And that's why it's taking more time for the agent because again, it needs to read all those files.

It needs to gain the context, and it probably needs to touch more files to implement the change. If we recap, we've actually seen that we cannot uh get to a point where uh the agent cannot work with the code. It can still deliver features without introducing regressions because our external suit validates that all previous features are working plus the new one. So, you're introducing no regressions, and it's been outperforming uh the hexagonal project both by uh time and actually cost, the input and output tokens that we're paying for.

And uh I wanted to challenge that more because uh yeah, I was starting to get convinced. However, I'm like, "Okay, we made this investment, right? In uh using ports and adapters architecture to give it the structure. Let's give it something that is specifically designed for this." And that would be a switch of a port uh of an adapter, apologies. Essentially, what we wanted to do now is we were using SQLite as a persistence layer, so we had the SQLite adapter in the hexagonal project.

We want to switch our database to a Postgres database. It's a big change usually, and ports and adapters again is designed for those changes. That's why you have the port and the adapter uh abstractions. And really this time, the boundary did help. We can see now that the flat implementation took much longer time. Actually, hexagonal was 45% faster for this specific change. And we can see where this saving comes from.

It's just agent work in total here. You can see the numbers in minutes. And if we take a look into the tokens, we can also see that the hexagonal structure did pay off here. really got uh fewer tokens being used. And uh fewer output tokens again, less model steps, less tool calls. Both again managed to finish the the change. And I'm actually impressed that the flat one didn't have any repair rounds. That means that it was able to make the change even though it took longer, make the change, do not reduce regression, introduce a regression, and uh still pass our like the full suite.

So, that means that still we're able to operate the flat project even with the 12K production line scale, the agent is able to reason its way through it. And yes, here we did uh we did save some time. However, I'm thinking is did we actually again get a return on our investment? And here really uh hexagonal was faster, but it also generated a lot more production lines in order to achieve that change. And now, the the big question, are we actually getting a good return on our investment?

Because we did change the database, and we did do it faster than we would have have in the flat project. And the really the question here is, did we get uh paid? Like, did we recover the gap that we spent the extra time on each of our previous features? Well, actually no. So, we accumulated such a big difference. We accumulated about a 90-minute difference from S1 to S16. And we recovered only 30 minutes from those. So, we still have like over 2/3 of the extra time spent that we did not win back uh with the uh ports and adapter.

And here is like a summary of the of the trade-offs. So, we can see that first nine features, the six challenges, and the 15 scale features all were faster and cheaper on the flat project. The only thing that paid back was the adapter change. And it was a meaningful payback. Let's Let's not overlook that. And here we come to the opinion piece. At this point, so I've showed you all the data that I've seen and based on my in this data and my professional experience, this is what I have to say.

I still think that this investment has value. I still don't think we're at that point where we can completely like a 100% rely on agents to only operate the code. We will still have time the times that uh human needs to look at the code base. So, it's still valuable to think about architecture, obviously. And still valuable to structure the ways and to keep the practices. What I would say is we need to be more deliberate and more careful about which practices we follow.

It's not about following a religious ritual at this point so that we have to do everything this way. We have to be thinking on where will we will get a payback and how will our return on the investment because it is really an investment. You're spending more tokens and more time. And so, that means you're spending more money on this. And uh yeah, because really yes, we could change the database of the project, but also you have to ask yourself how often do you change your database when you have a production system running.

And you have to think that uh also everyday work has a cost, too. So, even now like so 15 features outweigh two to two to three times uh the saving that we got from the database change. And uh yeah, it's also you have to think about the frequency and the types of changes that you would have to be doing on this project. Really, it's about like these fundamental questions about architecture that we had before, right? You have to you invest in architecture if you have a long time horizon uh for the project.

Uh again, you keep options open so you can switch technologies as we did with uh SQLite and Post- and PostgreSQL. However, I'm just saying that we just need to recalculate that uh that trade-off in this DNA age and really decide case by case on each of our on each of our projects. I mean, if a project isn't is going to be like a simple POC, right? You probably don't need to invest that much in this very structured architecture.

You can just use the agents to generate it. You do that MVP and see how it goes. Where just here I have to add a little bit of a note of caution is that how often does the POC become production? Because I'm saying that we need to be able to decide based on the timeline and the life lifetime of a project. We usually don't know that and we usually don't know how often we'll be changing technologies. So, really yes, that it's it's not rare that the POC actually becomes a production system that you're supporting much more longer than you anticipated in the beginning when you were building it.

So, it's a really a tightrope balance that you need to be walking. And again, another point I want to raise is if this is a critical system and you have a sev one at night, some human is going to be able to wake up at two and be able to decipher the code and understand what's going on. At this point, such an investment into structure and into clean code is absolutely going to be repaid. Because if you're able to so I would say for critical systems, we really need to think twice or thrice before we get away from architecture and tell them we just let our coding agents do whatever they want.

One point I would like to make is that automated checks for business rules are a must. I actually believe that they're becoming even more valuable because you need a way to have to check the work that your coding agent has done. So, there is uh no point in my suggesting that we skip CI and that we skip uh external testing suits. No, these are even more and more uh important. I guess we can wrap up with saying that agents uh now write the code.

However, we, the human engineers behind them, still own the consequences. So, we have this responsibility to manage our projects appropriately and we know the business context of our projects, so we should know what standards and what practices we should follow. And with that, I would say I will wrap this experiment. If you're interested in more of the data and the details that came up uh from this experiment, I have a blog post that I very thoroughly describe every step that we went through on the presentation.

So, I do encourage you to check that. And also, I would love to hear some community feedback on this. What are your thoughts about the data? Because here we've spent pretty much the last 10 minutes me discussing what I think uh this data means. Maybe you have a different view on it. Maybe you can uh challenge the experiment in some ways. I'm not saying that I have proved uh that code to be obsolete. No, it's uh it's just interesting data that uh I got and I did not expect this result.

So, really please do write in the comments. Tell me uh tell me what you think about these results. Tell me if this changed some of your beliefs or if or if it didn't, why. And yeah, I really uh enjoy discussing this stuff, so uh yeah. Uh I'll be happy to to answer comments and uh to keep this discussion going. And with that said, I want to thank you for your attention uh and watching this and um I hope to see you in a in another video.