Ralph Loops: Build Dumb AI Loops That Ship — Chris Parsons, Cherrypick transcript

Welcome. So, this workshop is on Ralph loops. Hands up here who knows what a Ralph loop is? That's almost everyone. I'm guessing that the other folks who came in were just here because they thought that sounded weird or maybe looking for a quiet place to work, I don't know. But I'm you're very welcome. So, what we're going to do today, this is a 2-hour workshop. We're going to if you could just kind of make a little bit of space if you need to as people are coming in, that'd be really helpful.

Thank you. This is 2-hour workshop. We're actually going to build Ralph loops together. We're going to do this together on our own laptops in order to to make some stuff happen and get some things done. So, so it's not just about theory. This is a very practical thing. So, if you've got a laptop, you're welcome to get it out in a second. We're going to actually going to try and do this ourselves. So, I don't I have a few slides, but not many.

Most of this is going to be live demos and kind of interaction points as well. And the idea is that at the end of this, you should be able to leave with something that works. That will will apply it to a kind of toy code base just for fun to create a Pomodoro timer. But but hopefully the idea is that you'll be able to use this on your real work when we get when we get done. So, another show of hands, it's okay. Who is using Claude Code or Codex specifically to write code?

Hands up. Oh, quite a lot of people. Specifically to write code, okay. Who is using it to write all their code? Who is no longer writing any code? That's quite a lot of people. Look around for a minute. That that is a huge change. If I'd asked a bunch of programmers 6 months ago who was not writing any more code, you'd be get a very different answer. So, next question. Who is using either Claude Code or Codex and I'll include cursor here as well, in their non-coding work.

Okay, quite a lot of you. What about for all your normal non-coding work? Okay, interesting. Interesting. So, can you just see the future in the room, right? There's a few people who are starting, but we're still on that journey for sure. And who has built Ralph loops before? Last show of hands. One or two people. Okay, great. I'm going to be looking to you for all the answers. That's great. So, just a little bit about me.

My name is Chris Parsons. These days I spend most of my time trying to help teams like the team I used to run figure out what on earth to do with AI mostly. So, I'm a CTO by background. I've done a couple of VC-backed startups and scale-ups. And this is they've taken me and my friends by storm rather, and we are trying to still all figure it out professionally together in terms of how to help our teams adopt and use AI.

So, that's what I do for a living these days. I have about 30 years or so of building software professionally, being CEO of an agency. I've done a lot of agile consulting, remember that? And that kind of training as well back in the day. And and funnily enough, all of those, it's a whole 'nother talk, all of those principles and practices that we taught for years and no one really listened, were are still very much applicable to AI.

So, so there we go. So, these days I'm actually running Ralph loops all the time, 24 hours a day, to get my work done. So, I'm I'm using them to write my emails. I'm using them to check my calendar. I'm using them to write content and newsletters. I'm using them to help me do my client work. So, I'm using them in absolutely everything. I also use them for code, which is what we're focusing on today, but they are very much applicable to every part of our lives.

So, by the end of the day, the idea is that you will be in the position where you can do that, too. So, this is how I used to work with AI until quite recently. You probably can't see that very well. This is an N8N workflow that I used in order to create my weekly newsletter. It took me probably a week to write, let alone actually test and debug. It's got a huge number of different things in here. This This is like a a featured article flow, which would read various different articles from my blog and figure out whether I posted it before, summarize it using AI, and put it there.

And then there's another one for grabbing links that I'd posted into a particular list, and it kind of did a bit of commentary on that. It was really quite complicated and and difficult to run and maintain. And it kind of worked okay, except that 2:00 p.m. on a Monday, pretty much every Monday, I would get the dreaded notification from N8N that my workflow had failed, and I was just like, "Oh, no." And then I'd have to go in and figure out in here what whatever had broken and try and run it and fix it.

Now, this is nothing against N8N. N8N's a really cool tool, and it can do some really cool things, and I'd never have been able to orchestrate AI in the way that I was doing before without a tool like N8N. Des- Despite being a coder, it's just so much easier to manage in here, and you can manage all the API keys really easily, and it's it's a nice tool to to stick things together. But it was so brittle to use at this kind of level of complexity, and I didn't get a huge amount of value out of it.

And then every time I fixed it, I would do something else. And so, honestly, it was probably easier for me to just write the newsletter than it would have been to maintain the thing that wrote the newsletter, and I'd probably had a slightly better newsletter. So, this wasn't great, but but this was to my mind a few months ago the really the only way to use AI. You had to kind of orchestrate it and manage it, give it the right data, and kind of handle all the context.

And I thought that this was the future of automation, but it isn't really the future of automation. The future of automation is a lot more like something like this, running in Claude code. So, this is obviously not the actual skill, but I have now in Claude code a skill that writes newsletters for me, and it has all of those instructions. In fact, I copied and pasted the NA10 JSON code from there into Claude and said, "Write a skill based on this flow." And it did a great job.

Um and then what it does is it goes through and does all all of the things. But what's interesting about that is how does Claude code work? Well, it reads the first thing, it decides on the next step, and then it reads the next step, and then it decides on the next step, and it kind of works through over a some minutes to actually write and produce the newsletter that I was writing. And it's the same for code. It's the same for anything that we want to build using Claude code.

Claude kind of just takes care of it. You describe the kinds of things you want, and it does it. Now, what's interesting is that Claude code fundamentally is running on a loop, isn't it? It just reads the skill, calls a tool, goes back to the beginning, reads the skill again, calls a tool, calls a tool, and then at some point it figures out that it's done, and it stops, and it gives you your newsletter in whatever form you want it in.

Um so, what's interesting is that this ships much better, more coherent newsletters than the previous uh the previous workflow. I still have to change them, write them, screw around with them, but but they are uh they are a much better first draft than they ever were. And I haven't really touched the skill. All I really do with the skill is I say at the end of a newsletter writing process, "Please just update the skill with anything you can figure out from the session you should have done differently." And it makes the odd tweak here and there.

Um so, that's a loop. That is this kind of form of working in loops with AI. So, so agents that originally start in workflows where you have quite complicated orchestration that looks a bit like something hellish like this end up in a quite a simple loop, perhaps with a bit better context. Now, this didn't work for the longest time, but it's now beginning to work with the latest models. And by latest model, I really mean GPT-5.x, really GPT-5.1 to onwards, uh and Claude Opus 4.6 or Sonnet 4.6 upwards.

So, those models started emerging around about the end of November. I've no idea about Methos, by the way. I I've spoken to people who've used it, and they say it's it's it's good, but it's mostly marketing, but we'll see. Um but um but yes, we'll see what that where that takes us. Maybe we won't even need skills. Maybe we'll just say, "Write a newsletter." And it'll do it. Who knows? Um but what what I'm trying to make my point is is that rather than using complicated workflows, we're actually using skills and loops much more in context and loops.

And any kind of agent that we run is in some way a loop already. Okay? And this kind of powerful looping construct is something that you can more generally apply. So, what happens when we take loops a little bit further? So, um the first stage um is this idea or the first idea came from Jeffrey Hinton Lee uh a little while ago in the ancient times in AI, which means probably about last June. And he said basically, what we should do is whenever we finish using an AI to do anything, we should just try the thing again in some way.

We should just just give it exactly the same prompt and see what happens. See what it does again. And it sounds it sounds a bit stupid, and it's based on Does anyone know where this story comes from? Uh who knows why it's called a Ralph loop? Like two people. It's called a Ralph loop because of Ralph Wiggum, which is a Simpsons character, who basically says um the uh he just tries the same thing over and over and over again, and eventually it works.

And and it's all it is, really. All all that a Ralph loop is is uh build this thing or do this thing inside a prompt. Then the AI goes away and does the thing. And then it finishes and says, "Okay, I've done the thing." And then it says, "Okay, great. Go and build this thing." And you know, and do all the things I said to build this thing. And it goes, "Oh, okay. I'll do it again." And and the the groundbreaking nature of what that meant was that the AI would often review its code and realize it had missed something in some way, right?

So it figured out that there wasn't it wasn't quite finished. And this is quite a common problem with AI coding tools last year. It wasn't quite done. It didn't quite get to the end. And then they go, "Oh yeah, I should have fixed that." But then it does it again. And then and then when it stops it says, "Right, I'm definitely finished now, 100%. It's It's done. It's finished." And then what do you do? You give it another problem to solve.

Say, "Go away and build the feature." It's like, "I've built the feature." And then it tries and looks and go, "Oh yeah, there was actually this tiny thing that I should have done. I really am now finished." And so on and so on. So you can kind of see the utility of kind of going through that loop where you just build the feature and just then ask it to build the feature. And then you ask it to build the feature. So So that's kind of the first stage of rough loops.

And what I'd like us to do is I'd like us to try that. So firstly, I'm going to do a bit of live coding. Hold on to your hats. Uh we'll see how that goes. And we're going to try and do that process using Claude code to see where that takes us. So um let me start change what I share. Sorry, just takes a moment. Okay. Great. Uh this one. Can everybody see that okay? Can people see that at the back okay? Do you want me to increase the size?

Got thumbs up. Great. Okay. So this is a piece of code that I live coded in about 3 minutes last night, so it's not good. But well, that's the whole point. We're going to fix it. Um so it is literally a Pomodoro timer. And you can see how it works. If I go to Python and type Pomodoro start, woohoo, we've got a Pomodoro timer. Uh that's all it does. It literally just does start. There is no way of finding out whether it's finished or complete or anything like that, but that's what we're going to change.

Um the other cool thing, which is very important for any self-respecting vibe coded AI project, is that it has tests. So look, it's got a test. There's one test and the check to see whether it starts. So that's great. So if we just have a quick look and you have to forgive me if you're not a Vim fan because I am. Um we, although I hardly use it now, it's quite sad. You know, 20 years of muscle memory just gone. But um but yeah, so all it does it literally just runs a start command and then it saves in your Pomodoro dot Pomodoro in your home directory, it saves the time in which you started.

Really, really simple. So this is a very simple, quite straightforward project. The difference is that it has a new folder with different things in it and these are tickets. So there are a whole bunch of ways in which we could improve this Pomodoro timer. Um and the first ticket is it would be really nice to know how long is left on our Pomodoro rather than just starting it. Um so I've what I've done is I've created a very simple ticket system to allow us to just kind of capture some changes.

These are not Um this is one shot it, so I've no idea if these tickets are actually good. In fact, I haven't looked at some of them. So we'll see how that goes. But um but the idea is that uh we can use these in order to start building a loop of work in order to get something done. So Um what I'm going to do to start with I'm going to start Claude and I'm literally going to say write the first ticket. Um so bear with me while Claude fires up.

In some ways I'm quite glad they didn't actually release Mythos yesterday because I think I don't think it would be working today if they did. Uh that is really not working, is it? That's frustrating. Wow. Let's try again. >> I think there's a problem with Wi-Fi. Uh okay. I mean, that could cause some problems to my talk, but we'll have to see how that goes. I don't have one of those fancy new um Macs that allow you to run.

No, this is actually locked up my computer. Can you believe that? It was working literally 10 minutes ago. Uh let me just have a look. Uh bear with me while I debug my machine. Um There is Um I've got I think I'm on a different Wi-Fi, so it should Uh no, yeah, the Wi-Fi has gone down. Fun. Tethering time. Might have to be. Okay, hang on a second while I tether to my phone, which I think has decent 5G, so we should be good.

Okay, let's see if that's any better. Hooray! Okay, let's try again. Uh not that one. Um cool. So, uh code pomodoro workshop. Okay, is that big enough? Okay, good. Let's try this. Claude, via the power of 5G. Look at that, it works. Uh fantastic. Okay, so um what I'm going to do is we have, as I said, a very, very simple, stupid pomodoro timer. And we're going to implement a ticket. So, what I'm going to do is I'm going to say implement this ticket.

So, it's in doc tickets 001. Great. And I'm just going to say that. See what happens. So, what's going to do is going to read the ticket, uh which I showed you briefly earlier. It's It's um very straightforward, and all it does is it implements a status to see how far we've got. Um and then uh what I would like it to do What I'm going to do after this is I'm then going to say when it's done, cuz it will literally be two files.

It's not going to be difficult for it to do. And then I'm going to say, you know, implement it again, and see what happens. So, there's a few different ways that you can do this. Um and there is no kind of one set way of doing a Ralph loop. It's really about the concept, not about anything else. Great. So it's done the the ticket. If I just quickly do a quick get diff, you can see that what it did is it added a status command.

I think when I had a show of hands earlier, most of you were coders, so hopefully this is not tricky to follow. And then we've got a new test. Look at that. We added a test. It didn't even ask it to and added a test. Oh my gosh. What is the world coming to? So um now what I'm going to do is I'm literally going to say the same thing. Now, a year ago, this would have been really important step because it would have definitely missed something.

Um whereas now it's like I've you've already done it. It's fine, right? So Opus is now much better at noticing when things are done. Now a traditional Ralph loop would just keep doing this, right? And it would keep going with this kind of implement this ticket, implement this ticket, implement this ticket. And um And this is kind of boring and it's not really going to do very much else. Um and at some point, actually when I tried this earlier, uh it's interesting, it's done something different.

It actually noticed that what it should have done is it actually should have updated the status to done. So the process worked. That didn't work earlier. Uh so that's great. So it's actually noticed something it didn't do. So there you can see the fundamental early principle of early Ralph loops, right? The idea that you can just zoom through and and do something and it will find things eventually that it missed. Because it missed that, I'm just going to try it once more, but I I don't think it will come up with anything else.

Um like I said, latest models really don't need this step in quite the same way. Um they they tend to just kind of get it done. In fact, this time it's just like um "Oh, if you're running a Ralph loop, that picks up the next ticket." Oh, that's hilarious. It's It's literally giving away my presentation. Um That's fantastic. Okay. Um what I'd like us to do is I think as a starting point, the other thing you can do is you can just kill the context and then you can do the same thing again and you can say implement doc ticket 001 and I can't bother to spell it.

It's It'll find it. Um and then um so now uh what I'm doing is basically doing the same thing but without it knowing about the previous context. So it'll be quite interesting to see what it does with this. Um I'm assuming it'll find assuming it found the ticket. Yeah, it did find the ticket. That was easy enough for it to do. It's just running the test to make sure that they work and it all passes. Okay, so it's happy.

So some people when we when first started using Ralph loops is that they weren't doing it within the same status and and uh there was an early Claude code plugin in that just on the stop hook which is which is what runs right now when it stops running, it would just do the same command again. Uh so rather like me just typing the same thing in each time. But that didn't really work very well cuz it didn't get very far.

Whereas uh now what's uh more useful or or what people started doing was just running Claude code in a kind of loop. So they would do something like while true um and then do Claude um implement ticket 001. Right, and then done. And then that would just go through uh not quite actually cuz I didn't do Claude P but effectively that's that's what they were doing. Um oh no, now I've really screwed it up. I really shouldn't have pressed into hit enter on that, should I?

Okay, there we go. So that's but that's effectively what people were doing. The dumbest Ralph loop is literally that just a while loop and it just goes through and implements stuff. Super super easy. Um so um what is the next step in Ralph loop? Well, in fact, what we're going to do now is I'm going to get you to get to that point and then I'm going to take questions from other folks. So um let me just switch to back to here.

I'm hoping. There we go. Yeah, great. Um so what I'd like you to do if you could crack open your laptops and um grab the code from here. So um it's just on my GitHub uh as pomodoro workshop. You should be able to find that quite easily. Um and you saw how I ran it. It's very simple. Uh you might need to set up Python in your machine. So there's hopefully that won't be too hard. Uh or you just run that Python uh uh pomodoro.py will give you the command you can type and then it's um just a unit test thing to run the test pomodoro, super easy.

And then in step four, I want you to fire up Claude Code or Codex and I want you to try and build that ticket and make sure that it's working. Um and that will be a great starting point. Um but don't build any more tickets yet. Don't don't let it take you too far. And then uh if you are really used to that and that is just like a literal no-brainer for you, try in Codex. Try in something else. Try in maybe try settings up something similar in one of your own projects.

Um so, something different. So, I'm I'm going to take questions now while people are typing away on that. I'm going to give you maybe a few minutes just to get that set up and then we'll kind of move on to the next step. Does anyone have any questions or comments or thoughts? I have a microphone here. Uh if people would like to ask anything, yeah. There we go. Um It should come on in a second. Hopefully. The guys at the back are It might not be on.

Is it on? You can shout it and you repeat it. Yeah, you could shout it and I'll repeat. Yeah, that could work. Is it Can I just check it's on first? Yeah, it looks on. That's weird. Shout and I'll repeat anyway. Oh, there we go. There we go. Great. Um I've I've played a bit around with the B mad method, which I don't know whether you've seen that. Um it's got a guy a guy who's basically written a whole load of skills and commands for following a full Agile process from you know, and he's got >> I think I have.

He's got an agent for build it, test it, you know, everything. And and I I guess so um have have you done anything where sort of using this kind of rough loop process you go through that cycle, you go through the full software development life cycle of each stage and then Yes. Um I I might get as far as that at the end. Um but yes, I have tried some of that stuff. Um it's really interesting and it ans- it asks some really very good questions both around context and and actually the value of the work.

It's really interesting. So yeah, we'll talk about maybe that a bit more at the end. Um so ask again, if I haven't got to it, just ask the same question again and we'll get that in a rough loop. Okay, thank you. >> Um anybody else got any questions? How people getting on with setting that up? If has anyone managed to set it up? You know, kind of wave at me if you've managed to get it running. Great. Great stuff. Has anyone managed to implement the first ticket?

Hooray! A few people. You got a question? No, no, no. Okay, go, go. You've done it. Great. Fab. Yeah, I probably should have given different directions for asking a question versus having finished. That's great. So a few people have got started. Fantastic. Great. So you can probably tell where this is going and if you were paying attention to the live demo, you'll already know the answer. I'll grab the mic from you so you don't have to keep holding that.

Thank you. Um but yes, you don't have to just stop at one ticket. Now, is Matt Pocock happen to be in the room? I know that he's doing the workshop after this one. He is the person I got this from. So if he's watching the video, thank you, Matt. Uh this was a this was a a revelation to me back in sort of September last year. Posted a brilliant YouTube video just about exactly how to kind of take rough loops to the next level.

Because when they first came out, I spotted on the internet, I played with it and I was like, "Yeah, this is fine. It's kind of cool. It kind of spots things that AI can do where where it's missed things and it can maybe do a slightly better job of things, but it's not going to change everything that I do." Um and then the answer is actually does change the entire way that I work and approach code now. So um I guess the the really interesting thing is not how do I make sure Claude has finished this one thing, it's what happens if I point this kind of loop at a whole pile of things to do.

Right? What happens when we point it at a whole list of things. Now, I tried this. I I wrote a blog post about this, which was a bit depressing cuz it just showed abject failure in the entire post, to be honest, but it was more about I tried What I tried to do is I tried to get um Claude, I think it was Claude at the time, to break up a big project into a lot of different tickets. And then I got it to break down all of those tickets into smaller tickets.

And then I got it to to figure out what all the dependencies were between those tickets and write them all down really carefully. And then I got it to figure out how I it could use like a ton of different agents. Sorry, did you have a question? I was saying I had a problem with Wi-Fi. That's fine. You've got a someone there with a mic if you wanted to say it. Oh, one two one two. Okay. I have a problem with I had a problem with Wi-Fi and I didn't do the clone.

Oh, I'm sorry. Let's go to this one. Over minutes. Yeah, thank you. >> fine. I'll leave it on there. That's fine. Um has everyone um appreciated the slide? Okay, good. Okay, there we go. Um so um these slides, by the way, are created using a slide skill using Nana Banana Pros. Absolutely incredible at making slides. Um I didn't I haven't apart from like at the tiniest thing, uh like adding a QR code, I haven't added any text to these slides.

They're just flat images in Google Slides. Um they're actually it's incredible at making slides. Um what was I saying? Yeah. So uh So the idea of just creating uh a a Ralph loop to just do one thing seemed a bit pointless and and it was just working around a few limitations. If you point this loop on a whole pile of work, then it becomes incredibly uh powerful. And as I was saying before, I created this huge complex dependency graph with a whole ton of different tickets about how I was going to build this really complex system for me.

And then I got I fired up like six or seven parallel agents and I was like, "Right, you do one stream and you do that stream and you pick up this ticket." And it just failed horribly because the the system just couldn't figure out what had been done and what hadn't been done. Picked up There was lots of contention between tickets like, "Well, I can't do anything until you until I get that shared ticket done, so I'm going to do it." And another Claude was like, "Well, I can't do anything until that shared ticket is done, so I'm going to do that, too." And then they both implemented the same thing and it was a huge mess.

So, that was really very depressing. And I wrote a whole thing about it and I was basically like, "It's impossible to orchestrate large numbers of agents, you know, you just can't do it." Which was obviously nonsense, but um that's how I felt at the time. And what was interesting was that what I'd done effectively was recreate the waterfall processes that were seen in some of the worst companies back when I was starting to code for myself in the '90s, where people would write requirements documents that you had to stagger to carry into them the requirements meetings, uh where the the entire project was specified up front with all the intricate dependencies handed to the development team and then given 2 years to build.

I can see some perhaps um uh slightly more seasoned people in the room sort of nodding and smiling at me when they when they hear me talk about this. But yeah, I thankfully managed to avoid working on any of those teams, but I some of my friends did and it was absolutely awful. And what I had done is basically I had given that to Claude to do. I'd given that waterfall process to Claude to to um to organize and figure out as as it went, which was really bad.

So, no wonder it didn't work. If humans can't do that, how was how was AI supposed to do any better? Um however, if instead of saying with all of your tickets, right, the first one is the most important, then this one, and then you should do this one, but don't think about this one until you've done that one. Instead of doing that, I'm going to go back for just a minute. Are we all good with this slide, by the way? Does anyone still need the slide?

Okay. Okay, we're good. Right. Uh instead of doing that, you can just run a round loop where you say something like, "Hey, just pick the next most important ticket." It's as simple as that. Just figure out here are all the tickets, just figure out what is the most important next one to run. Okay, you don't have to worry about the dependencies. You don't have to figure it out yourself. The AI is quite capable of looking at all of them, figuring out the dependencies on the fly based on what's just been done, and figuring out what the next most important thing to do is.

That's actually quite easy for an AI to do. The one thing it cannot do so easily is manage that process in parallel, but to be honest, when we're running these kind of loops, if you're running them continually, the bottleneck is usually not the number of agents. It's usually you just keeping up with the AI just doing things on it over and over again. So, let's forget parallelism just for a minute and just start with a loop.

See if you can keep up with an AI, just one AI that's running continuously, you're fine. Don't worry about parallelism just yet. Don't worry about gas, time, any of that stuff just yet. Um as impressive as those projects are, you can just start with a simple loop. It is okay. So, um what I'd like you to do again at this point is I'm going to quickly show how this works for those of you um who don't have your laptops, but then I'd like you to just try it on your computer.

So, again, let me find my mouse. And then move to sharing my screen again. Okay. Great. So, if I go back to Claude, I'll first I'll go back to Vim first and look at the tickets folder. So, I've got a whole bunch of tickets here. I've got a status command. I've got a stop command. I've got custom durations. Never used that anyway. I use um other things like, you know, labels and all of that. Um I could try and figure out the dependencies myself, but I really just don't need to.

I can just simply go into Claude and say, "Implement the next most important ticket using uh TDD principles from doc tickets. Um commit when done. Something like that. Okay. So, let's see what it does. So, it's now reading a whole bunch of tickets. As you can see, it's read number one, two, and three. And it's decided the next one is number two. It's just going to do it. That's great. So, it's going to work on it. Um Now, the interesting thing now is that once this is finishes, now it's using TDD, so it wrote the test first.

Uh when it finishes, hopefully Yeah, it's marked as done. Very good. Remember that time. And then should commit. Let's see if it does. Sorry, is this in the same session No, this is a brand new session. Although, I think it probably had the working directory from the previous one still. So, um In fact, I think it has. So, it What it hasn't done is committed those atomically, which is definitely something I could improve in my prompt, but we can cover that in a minute.

Um so, then hopefully it's just going to do that. And then it's going to finish. Yep. Great. It's done it. Fantastic. Now, what I can do is I can either do that again uh as a route loop, or I can just restart a new session. Just do the same thing. And this time it will pick something else. And then it will keep working. Now, you can imagine that if I put this inside a while loop, then it should, in theory, work through all of the tickets in some way.

Now, whether what I get at the end is actually what I want is a whole different question. Um but, it will definitely get a lot of work done in a row. So, I'd love you to try it. So, if you're if you've got the um app working on your computers, um see if you can get it to work through just as many tickets as you want to within that amount of time. Um so, it should be able to just carry on. See if you can get it to to actually um maybe write a little bash script um just like I've done where you do a while true and then Claude.

I'll show you how to do that briefly. In fact, if I just quickly get reset so I can start just from the beginning. Um and in fact, I'm actually going to go up one more. hard head. There we go. Great. Yeah, that's the right place to start. Um so, uh if you can get it to do that, then that's great. The other thing that you can do is instead of using Claude like this, you can do Claude {dash} P. Uh can you all see that okay by the way?

I'm not sure how I can make that um Yeah, yeah, clear is a good one. Yeah. Clear. Claude. There we go. So, what we can do is actually use Claude {dot} P like this. And you can get it to output by just doing stream JSON or something like that. Uh what's it called? Something like stream JSON. Um Hang on a second. Let's see. Uh I think they've removed it. That's annoying. Never mind. You should have seen the output. So, there's nothing to stop you setting it up like this.

And then you can just do that. But you you got to set up Claude to have full permissions so it doesn't prompt you. Yes. So, the only way that this works is if you want to run this properly and for it to not stop, you have to be quite selective about permissions that you give it. Um so, the question was presumably you have to run Claude with full permissions for this to work. Yes. Yes, you do. Um it depends on what you're doing.

Yeah, if you're working in a little sandbox project like this, the chances of it going elsewhere to find stuff out is is very small. Um I have a project called lockbox and the sole purpose of that is to try and stop it doing stupid stuff um by by when it reads untrusted tokens, which would potentially send it off track, it um it basically just prevents any kind of file system access or anything after that. So, there are ways of of kind of managing it.

Um so, what this is doing in the background is you can't actually see it doing anything um because I don't have that that output mode, but you can figure out basically that to to run this in in some kind of script and if I in fact if I quit that um hopefully it will stop. There we go. No. Let's just keep going. Sorry. Clearly clearly a a more production-ready Ralph loop would not look like this, but you can see it's done a bunch of work.

So, if I go to here, you can see it's already started on the status command um and it's just working through that at the moment. So, it started at the beginning again and it was just working. So, um there's a few things to be aware of here. One is that feedback is really really important with Ralph loops. Um you know, you need to be able to have it run uh um in a uh in a way that you can tell what it's doing and how it's doing it.

So, this kind of super basic one that I've given you there isn't very good. That's not one that I would recommend running in production. Um equally, you need to um figure out uh exactly what the prompt is for Ralph and that's a really really important point. And I think what I'd like you to do when you're trying this is yes, it's going to be building a bunch of tickets in a row, but equally it's going to be um uh it's going to be doing them in a way that you don't like.

Um so, for example, if we go if I'm if I'm running this test, which is literally just implement the next most important thing, uh let's start with this one. Um I would probably do something like um run simplify, which is a really useful skill from Claude, uh from the Anthropic team, uh when finished, and ensure you refactor to reduce duplication. You can imagine that you could create quite a complicated skill for this.

I'll show you I'll show you my kind of actual skill. I'll put this at the end. But as you're kind of working through this, uh do try and figure out um if there's ways that you can improve what it's doing. So, give it a go, let it make a decision, and then let it write some code, and then read the code and think, "Okay, what could I have actually improved about the process?" And then reset everything and then improve the prompt after that.

Um have a go at that and see how that works. Whilst people are kind of working through that on their machines, I'm happy to take questions. Yeah. Have you used the skills like superpowers? Uh which one, sorry? Superpowers one. Uh the Superpowers one, what about the process? And then reset everything and then improve the prompt after that. Um have a go at that and see how that works. Whilst people are kind of working through that on their machines, I'm happy to take questions.

Yeah. Have you used the skills like superpowers? Uh which one, sorry? Superpowers one. Uh the Superpowers one, what I did is I pointed Claude at the entire repository and said, "Figure out anything that isn't currently in my skill set and implement them for me with my context and that worked well. So, I haven't used those ones particularly but basically ripped them off. That's great. Um then because I use Superpowers a lot and then I just give tasks like this Mhm. and then ask it to like run multiple agents in the background.

Yeah. How you done that is what I'm Yeah, yeah. So, what you can do is there is an agent teams version, which I think I've got turned off in this particular instance of Claude cable, what can happen is you can get Claude to um use uh sub agents within T marks. So, in fact, I think I might be able to turn that on if I can find the agent teams. There it is. Claude code experimental agent teams. So, if I grab that and then run Claude with that tickets in this repo.

Or something like this. And I don't This isn't actually running within tmux. So, um So, actually maybe this way works. So, I might just try this again. Bear with me a second. So, if I grab that and then paste that there and then grab that and then run tmux and then run that. In theory, in theory um this should uh start pulling up other agents. And it And because like I said, trying to orchestrate uh raft loops my orchestrate agents myself to try and organize all of the dependencies and complexities is actually really, really difficult to do.

But, um what you can do is just give the job to Claude to do and it does a much better job of managing that for it. So, as you can see, it's already decided to print out the entire thing, the file name, and the the ticket for each. And it's got a whole bunch there. So, it's actually decided that they're all sequential. So, therefore, it should run It should run none of them in parallel, which is kind of interesting. Um and then it should in theory start an implementation agent.

Let's see if it's going to Uh I think it's just running as a sub agent. Never mind. Not sure that's going to work. Never mind. If you can get it to do it, then let me know. But, but but basically, it's an experimental feature that only came out a few weeks ago that allow um it to kind of start sub agents in Claude as well um in order to do things. Any other questions while people are working through that? Yeah. You said you built a raft loop with the internal automation collaborate.

Mhm. So, what was the feedback criteria on that? So, like what decides if it's a good newspaper article? Like I That's great. send it into a website consensus engine framework. But, what good looks like, if that's dynamic, if that's static, do you put that in the Claude MD file? Or like how does that how does that work? >> so um the question is just to to repeat the first half of that is when I use the NA10 workflow in order to build a Ralph loop for a newsletter creator, uh how did I know how did how did the agent know what good was when it How did you define good as well? >> Okay, great question.

So so in terms of newsletters, I had already been writing my newsletter manually. So I knew roughly what I wanted it to read like and sound like. I also did a bunch of research um using a research skill, which was something like which I built, which is something like um great newsletters. And I've also did things like that. I also said things like um this is a fantastic written new In fact, I'm just going to This is not what I do.

I actually do This is a fantastic newsletter that I've written or that I've read somewhere. Could you please figure out why this is so good? And what are the kind of editorial principles that go went into this newsletter for it to to work really, really well? And then I would just paste that into paste the newsletter in, get it to figure out what what was good about the newsletter, and then and then I would check it, and then I would say yes.

There still is an element of human taste here. You can't entirely get away with that. Having said that, I do also have a simulate audience skill, which um basically uh uses a whole bunch of different personas uh for different clients or or prospective clients that I work with, and then I would run run the finished newsletter through that and say run all of these in parallel. Um and then once you have finished that, figure out ways that I can improve this newsletter or newsletter skill in order to do that.

So there's a number of different ways you can do that. They're kind of audience simulation, super experimental, but it's actually really effective. And and often will will surface insights I just hadn't thought of. Um I'm uh my my personality as as I'm a bit slightly all over the place, slightly kind of the way that I talk and and communicate, um often my my are not like that. So um I tend to barrage people with information and sometimes my skill will say, "Okay, there's a lot of ideas in this, Chris.

You just need to focus on one main point that makes sense." And I'm like, "That's so helpful." So, um, so yes, what's quite helpful and interesting is that you can use AI to give feedback on AI like that. The great thing about this particular um, project that we're writing, this little Pomodoro thing that people are writing, is that it's a command line tool and it's really simple to know whether it works. So, it's perfect for a rough loop.

In fact, these little tools that we build for ourselves, like for example, the newsletter skill, perfect for this kind of loop. I I will often say, "I want to improve this skill. Could you please back and forth uh, and and write the content then use another agent to read the content, decide if it's any good, come up with things things to improve, then send that back in." And just run that as a loop. Um, there's a really cool um, skill.

I wasn't going to tell you about this until the end, but I'll tell you now. Um, there's a really cool uh, feature inside Clojure code called loop where is instead of um, creating, in fact, I'll start this in a new session. Instead of just doing this thing where you have to create your own while loop, you can say loop every minute um, build the next ticket from doc tickets. Basically, and then what will happen is um, the loop will set up a um, a kind of almost like a a repeat timer and it's you see it's it's got a cron creator tool which for the uninitiated just means do something every minute.

This is what those five stars mean. And um, what it's going to do is it will literally just build the next ticket. It when it finishes, it will then check the cron again, build the next ticket. When it finishes, it will check the cron again and keep going. Um, so that's great for working through a bunch of tickets, but it isn't just applied to a a set of things that you've got from before. If you think about it, I'll just leave that running up there, you could have a loop that does something like this.

Loop every 1 hour a linear for new bug reports from test. Um and then I just leave that running. Um oh, yeah. Can't spell. Um just leave that running. And that time you're going to get you're going to annoy your testing team. But But anyway, the point is is that you can run these kinds of loops in order to get work done in a in a in a quite a interesting I guess dynamic way even though it's quite a simple loop. It's just find the next thing, do the next thing.

If you think about it, heck of a lot of our work is just loops. If we're software developers, what do we do? We look at the backlog. We pick the top thing from the backlog. We pull it over to you know, in progress. We assign it to ourselves. We check on the architecture. We figure out whether there's other contexts we need. We uh look at the change. We We make the change. We submit a PR. We um uh you know, wait for reviews.

We uh comment on the reviews. We reject the reviews. We implement the changes occasionally. We submit the PR. We merge the PR. We then go through the release process. Then we start again. We pick up the next ticket and so on. That is a loop. It's quite a complicated one um like we talked about just a minute ago. But But it is still a loop. It is possible to get an AI to run that entire loop. There's no reason not to.

Um and that's effectively what's happening here. When When um you can set up In fact, you would never actually write this. You would much more likely to write something like this where you'd say every 1 hour uh linear bug finding. And you'd have a skill that encoded all of those um those chunks of information that I just gave it in a way that would work for you and your particular team. Does everyone Does anyone not know what skills are before I go any further?

No, I think pretty much almost everyone knows what skills are. If you haven't figured out what a skill is yet, then then this is your homework. Go and understand how skills work. They are the best way that we have at the moment of packaging up useful little parcels of context and scripts and moving them to different places or or creating different things. So for example, I mean I have about 50 of them that I've written and then they just do lots of different things.

The great thing about skills is that you can pull them into your context whenever you need them. So for example, and I'll just do this, I could say, "Do you know how to create images using uh Nano Banana?" And I can ask the AI the question and the answer is, "Well, I could kind of look this up, but it actually knows that I have an images skill for this funnily enough." But if you hadn't got one, it wouldn't know. But if I then do images and say, "How do you create images?

Give me the step-by-step." Then what it's going to do is it's going to pull in that images skill and then it tells me exactly how it does it. And I've actually written, in fact, I will make that bigger so you can see, I've actually written a script within that skill that actually does the generation for me. So it's codified the process of doing that and it just picks whichever model it wants to and it gives it content.

And I have these specific templates that I use in order to create specific Nano Banana skills. Nano Banana is brilliant. This is how I created the presentation that you're looking at. I have a slide skill and an images skill that work in tandem in order to create these presentations. Cool. So let's see what the other thing has done. As you can see, it's already on ticket six. The great thing about Ralph Lips is you just keep working, keep talking about something else and it's done a whole ton of stuff here.

And it's just stopped at this point, but in a second, hopefully, if we just wait, it will start the whole process again. There we go. Uh, it's got the schedule task to run and it's going again. Um, so you can just you you can just leave Claude code sessions uh running with these kind of loops in them. They last about 3 days, so you do have to keep refreshing them. Um, but you can you can just do that and keep it running even before you get to more complicated write a script that wraps Claude to do a thing and all of those kinds of things.

Um, any other questions? Anyone got anything interesting or surprising out of their Ralph loop? Has anyone tried this on their real work yet? This would be the interesting thing. Yeah. How what was your experience? Have you still got the mic? Yeah. Yeah, yeah. I just made a screenshot Ralph loop for a website context entering framework. So Claude just takes the screenshots and then looks at the layout cuz it has problems with geometric like spacing. >> Mhm.

It works well. Nice, cool. So you're actually using Claude Claude screenshotting to to get feedback. >> Yeah. Yeah, that's pretty advanced. Not many people are doing that. The um, people are trying to use um, like Playwright and things like that as well to take screenshots and the Claude in Chrome plugin that comes with Claude as well, you can use that in order to get it to drive Chrome and then take screenshots of what's going on.

I've had mixed success with that because it it's quite a complex thing for it to manage, but but for just basic screenshots it works really well. For my um, images and content that I write, um, I when it runs those images skills, it will always look at the images first to see whether there's any kind of a weird AI gobble text or whatever and it will reject them without even showing me if there's a problem with an image.

Um, was there another question or comment? Yeah, there's a question just back here. Can you just pass the mic? Is that okay? Thank you so much. Um, I think it's it's close to a question that has been already asked. Um, cuz I'm I'm not quite familiar with Ralph loops. If I ask the agent to implement task one that has already been implemented, would it actually check the quality of what was implemented or only check if it was done or not or marked or not? >> Yeah, it very much depends on what you set it up for.

So So there's no kind of magic to a Ralph loop, it's just a loop. So this loop that I'm running at the moment, in fact, I probably should just say loop stop, otherwise it's going to keep going. Use my quota. I think I think you can just stop like that. We've got a quite a fully featured Pomodoro set up now. Come on, time to stop. Um so it depends on it entirely depends on what you write. So if you um if you go through to What was the loop that I set up?

I think it was this one. I just said build the next ticket. That's very ambiguous and not very helpful. So it might not actually finish it. It may just decide to build it and not ship it. It may not actually be very helpful. So what's more interesting is if you go to um let me just probably the easiest thing to do. If I load my my Ralph skill, this is my actual skill that I use um for Ralph loops. So um and what I'm doing actually, this is slightly out of date, um but the one that I've got here is actually using a doc changes folder.

Uh you can see that there, but um I'm using a doc tickets in this example, but I've changed it on the the latest one. Um but um but ultimately, you don't have to use a ticketing system like a flat file in a in a um in the GitHub repository. You could use Beads, which is Steve Yegge's version of of this kind of approach, which is quite cool. I've used it. Um you could use Linear, you could use Jira, you could you know, as long as you can get access to it from the AI.

You can use whatever uh ticketing system you want. Uh for this for the purposes of this exercise that you're working through, I tend just to use uh flat files because they just work, you know? It It's not You don't really need anything sophisticated. Um in the same way, um the uh the Ralph loop is entirely what you make it in terms of its effectiveness. So, So, for example, um in this particular one, um I've given it a proper kind of role in the sense that you are one energy day in a relay team, do exactly one change, then drop the context, and stop it start again.

That's the idea. So, with this one, it's designed to be run in a shell script where it has an entirely fresh context each time, because I didn't want the context to pollute each time. These days, I care much less about that, because context is so much larger than it used to be. But, um but when I wrote this, that was very important. As you can see, it's specifically for code that doesn't need human review before shipping.

Um and then basically um it tells you about when work should go in there, uh what the right time for the tool is, read the claw.md, change the format. This is the format of a ticket. Um this is all of the rationale. These are the different status values. Um I ask it to check uh Git state um to make sure that it hasn't got a working directory. Um it's also got, you know, recovery states. So, if it crashed, it knows that if there's a uh dirty working tree, but the tests are passing, but you're probably done, um but you might not be, so just double-check.

If the tests are failing, um then it's probably just mid-flight and broken, so you should probably just throw it away or or just treat it differently. So, So, you can you can imagine that this was built up over time of trying to trying to get this working, trying to understand what the user wants, make sure test passing is not enough, verify the actual behavior works, run things in parallel, mark it done, blah blah blah blah blah.

There's an awful lot going on. Um so, with a real Ralph loop, you want to be building up over time for your specific project exactly how to check something is working, exactly how to run the tests in your particular framework and dialect, how you submit things to the test team, how you want to uh comment on particular changes, and what styling you want to use, whether you want to um pull off the thing that feels most obvious to you or the thing that's highest priority or a mixture of both depending on how you're feeling that day.

Whatever you Whatever you want needs to be coded in it. So, when you're writing a route loop, um I'll show you a a link to grab this one at the end, but there's no need to use just mine or or something else. Just start with mine and then say, "Fix this for my project." and and and allow it to change and morph and evolve. Uh couple of questions. So, um just come forward. Thank you. Hi. Um thanks for the talk. Um could you expand a bit more on the topic of sandboxing cuz that would be the thing uh stopping me from running this uh work. >> sense.

Yeah. Yeah, absolutely. So, there's a number of different ways to sandbox this. For this particular small project, I'm not doing that. Um most of my work happens on a VPS which is away from my main machine. It has a few keys on it that are specific to what I want it to do and um it can access developer tools. Uh a lot of them it can only access them read only. Um it can also access my email, but again it has quite strict fine-grained Claude permissions for not sending emails cuz that's quite important.

It I don't let it ever send an email. I only ever let it draft them. So, um so I use a combination of uh positioning the code physically not physically, but like away from the machine on a different machine on a VPS. Uh I use Claude permissions for that as well. The permission system's a bit broken, but it mostly works. Um I'm trying to you to build lockbox to make it even better. Um I use um What else do I do? Uh so, the keys that I use are separate keys.

So, um the AI has its access to its own keys which I don't use for my other stuff. So, I can I can see the kind of audit trail of what it's done. I said there's a number of different ways of doing it. Um if you want to just run things simply on your own machine, there's Docker sandbox which is quite cool. It's a new feature in Docker which just allows you to do Docker sandbox code and it'll run code within that sandbox.

Um so you can kind of isolate it within a specific container. That's quite powerful because it allows you to only change things within that specific place in the file system. The challenge with that is that it can still leak data from one of your systems to another of your systems. Uh there's a thing called the lethal trifecta. I don't know if you've heard of that. A Simon Willison uh coined it. It's um an idea that if you have untrusted tokens, uh internet access, and access to secret important data you don't want to lose, you're going to lose that data, basically.

That's That's the the um the bottom line of it. Uh so you have to kind of minimize the amount of times that those things collide in the same context. Um so yeah, lots to say about security and sandboxing specifically. Um I tend to run I don't run with dangerously skip permissions, uh but I do run with with um a number of things turned on by default, but not everything, basically. And you kind of have to go through and figure out what what your risk profile is and how much you care about these things.

And certainly as you're giving the main the main things to to read up if you're interested is to read up about the lethal trifecta if you didn't already when already aware of it. And um kind of be thoughtful about how much power and permission you're giving to your agents, especially if you're using something like Open Claw, which is uh unfortunately insecure by default. Um I know that they've been doing a huge amount of work um on Open Claw to make it more secure, but it is still a a challenge for those kinds of agents.

They do have access to a lot of things. Any other questions? Yeah. Uh yeah, you had a validation step in the loop. >> Mhm. Uh this might be anecdotal evidence, but as soon as I changed mine to use sub agents here now for the validation step, it started finding things. Ah, interesting. >> Whereas as as as you're doing the validation in the same step with the same context, it just pats itself on the back and like, "Yeah, you did good." >> really that's a really good point.

Um there's definitely uh confirmation bias going on with agents where they're like, "Oh, yeah, of course I wrote it fine. It was fine. I checked it a minute ago." Uh yeah, using sub-agents is really powerful because a sub-agent starts with only a small chunk of context. It doesn't start with the full context, right? So So you can get much more uh power from it. So as a good example from this particular project, um a really useful um skill which I mentioned earlier is simplify.

Simplify is a Claude coding bundled skill. Um what it does is it will look at the most recent changes and it will run three sub-agents to try and figure out what whether your code should improve. So you You can see here what it's doing. So hopefully, this will run. These will load and it'll probably find a bunch of problems. Yeah, great point. Good presentation. Thank you. Uh did you try Open Spec or combine with Open Spec already other Spec driven No, um I I if I'm honest, I'm not a huge fan of Spec driven development.

I know that's that's um controversial and I'll qualify that. I'm I worry that Spec driven development is taking us at the extreme is taking us back to the bad old days of waterfall where we would specify the entire or try and over-specify project. Even these little set of tickets, I'm not that comfortable with. I feel like Spec should be much more iterative um and uh things that we can I see it's already fixing a bunch of things.

That's quite cool. So it found just to finish off that point, it found a bunch of issues there um and it's got some fixes. Uh yeah, so Specs um I like just-in-time Specs. I like the idea of building or thinking through what you're trying to build, creating some kind of plan and Claude code and then executing it. That's fine. I'm happy about that and I think that's a useful step. What I worry about is A, I I worry about things like here where they've codified that into the to the tool.

I worry that that will almost fossilize that that one approach with with AI that works today, but may not work again when Mythos eventually comes out, right? So, I worry that the tools are being too quick to jump to a specific structure of work that may not be the right thing in the future. So, I'm I'm cautious about that. Um I think it is obvious it's a it's a truism that AI needs more context in order to do well. So, we should try to give it more context.

But, I think the idea of overdoing that and and over specking a project is one to be careful of, as well as over structuring our process based on what we know about agents today, cuz then we'll end up with working on with a a new kind of AI-driven process that worked best with agents that came out in 2025 or 2026, you know, when we're still be using that in 2030, and that'll be a pointless waste of time. So, those are the kind of concerns I have with it.

Any other questions? Yeah, great talk. So, uh you mentioned that you don't like spec-driven, and you use Ralph. So, basically, there is no human loop. So, the question arises, does Claude actually need you there? Where Where is your input there? Great question. Um so, I've been thinking about this quite a lot recently and having a bit of an existential crisis. I don't know about anyone else. Um but, um but yes, what what what value am I adding here to this thing?

Um certainly not with writing a Pomodoro timer. I'm not sure I'm adding much value at all. I mean, I literally said I one-shotted those specs, and and there was no point there at all. I'm not saying that I don't I don't like planning out a system. What I'm What I'm interested in at this point, and I don't have the answers, is the fact that AI often will pick better specs and and write better specs that I can write, and will often have a better idea of the kinds of direction my software should go in than I necessarily will have.

So, I like the idea of actually having rough loops that create other rough loops and potentially or having rough loops that track whole customer engagements or even whole startups. So, I have a a skill that I'm working on. Should I show this? I'm going to show it. It'll be fine. What could go wrong? Um, which is called startup. It's pretty ambitious, but the idea is that it should um basically guide a product through an entire startup framework.

Um, so it is it is meant to be run as a loop. The idea is that it Oh, I see you're all taking pictures. Now now I'm owning this thing. Tell it. Um, but with great thanks to Ash Maurya who wrote some brilliant stuff on this. I should say that for the for the for the tape. Um, so uh you know, really really helpful um to me. So, I built this out of um basically all of the cool books I I read about stuff. So, I I I'm I'm a startup founder co-founder CTO.

So, so this is a near and dear to my heart. And what I'm trying to do here is I'm trying to give the AI enough context such that it could run my startup for me and potentially figure out what the next most important thing to work on is and then do that in a loop. You know, and then it then there's a big outer loop that runs that says, "Okay, well what's the next most important thing to do? Let's do that." Um, so it doesn't work.

But it it's it's interesting and it's getting somewhere and um it will often the first thing it does I don't think I've got it to show. Um, but um oh yeah, no, I will show it cuz it is hilarious. Hang on just a second. Um, there's a it I asked it how it was doing uh on one of its loops and it produced a a startup update deck as an investor memo which was I didn't even ask it to do this. I'll show you the the demo. Hang on a second.

Um, cuz it's absolutely brilliant. Uh, let's see if I can just show this window. There we go. Air skills startup update. Um and so yeah, it said yeah, I need to give him an update. So what I did is it said basically, this is how far we've got. These are the problems nobody has solved. This is what we know that's real. These are the number of This This is a skills management tool that I'm working on the background. Um this is These are all the kind of issues and it came up with all of this cool stuff that could go into an investor deck.

Um to be honest, yeah, it's not not bad. It's It's not a bad I think it's actually the GitHub for AI skills, but there we go. And um you know, who's going to pay for this thing? How much do they pay? Those numbers are definitely not right. But um but what's interesting is that it decided that it wanted to do this uh and figure out all of these numbers based on based on this um which is I I think was hilarious and it was quite proud of this deck to be honest and I had to kind of be like, "Hang on a minute.

We haven't You There's some serious thinking you need to do before you kind of go to that um uh will will orgs pay for skills governance? Would your org pay for skills governance? Great question. Um not sure yet. So anyway, um the reason for showing that is more to kind of point out that that AI can do a heck of a lot and and it doesn't do startups well yet, but that's probably down to my skill file, not down to the um agent itself.

I I have a feeling that there are an awful lot of things that potentially will be um will be loops in the future. Um I only got that far on my slides. Oh my gosh. Um hang on a second. Um so we've done that. We've done that. If you are still If you're not just listening to me and are still working on this demo, I've got a couple of challenges for you if you'd like to do this. One is is um you could try upgrading your ticket format.

Um if you like the raw markdown file, the doc tickets is fine. If you wanted to just type BD install or install beads, it's super easy to do that. And you wanted to kind of get Ralph to work with your beads, try that out. See if that works, you know, no There's no pressure on you to achieve anything. And this little folder uh beads is great cuz it only works within your folder. So um and it just installs a little tool.

So it's quite a useful thing to to to try it on. So, if you wanted to try a different ticket format or you wanted to kind of move this into your main project and connect your your Ralph loop to your ticketing system to see how that feels. Yeah, maybe not submit tickets yet, but you know, you potentially could you could try that and see where that takes you. So, that's an option. The other is is the scale. You you're going to need to keep upgrading and working on your loop.

The loop basically contains all of the know-how about how you as a person will go through that. You can take it all the way through from do the next ticket and you can take it all the way up to do the next step in the world dominating startup you're trying to build or whatever it is. You know, it works for all of those kind of things. What's super interesting about this is that I'm I'm more and more convinced that everything in fact is a loop.

Maybe maybe as an engineer I'm definitely on a loop on on a lot of the work that I do. Maybe as a project manager or or project manager I'm on a loop. Maybe as a CEO I'm on a loop. Who knows? Um maybe uh certainly a lot of the kind of cadences that I work on running loops, too. So, I have a skill and if you're running an open clawbot you're doing a similar thing. They they just runs a heartbeat every 15 minutes. It just on my VPS it just fires up Claude, checks a few things, checks my calendar see if I've got anything happening, and then sends me Telegram messages.

Maybe that's on a loop. And that is that's definitely on a loop it's every 15 minutes. I have a worker loop which I'll show you in a minute, and I have a morning loop where every morning at at 6:00 a.m. it comes up with a full briefing of my day, figures out exactly what I should be doing, and just gives me all the information that I need that's happened overnight. All the emails that come in, all of that stuff. The the worker loop is particularly interesting because it basically I'm not sure I can actually show this.

Uh let's see if I can find find us find something that I can show. Um, let's see. No, the reason I can't is cuz it's got a bunch of client information in it, so I can't show you that. But, um, what I can show you is, for example, this screen I can show you. So, if I quickly switch to this, um, So, this is an app, but I'll make that slightly bigger. This is now how I run my worker loop. So, this is an app that I wrote, um, uh, to manage projects.

So, I don't have tickets inside my my work folder, I have project files, and each of the projects is a is a set of work that I need to do, and then every so often I have a I basically vibe coded a Kanban system, and a worker will pick up and do the next step on the project. So, if the next step on the project is writing an email, cuz it has a an overview, or it's checking things, or uh, it's producing the slides for my project, it will do the next step.

So, this for example is the uh, workshop uh, prep spec uh, project that it's working on. And it's it's got a bunch of front matter that's just looking like that. Um, and it's got some questions for me. I haven't updated this, it needs to be updated. It's got the contacts, it's got a decision trail of things that it's done and why it's done them. Um, and so basically for every different thing that's happening, um, it it just is figuring out the next one.

Um, it's also got um, notes on um, other talks that I might be giving, um, that didn't happen in the end. Um, it's got feedback from a previous workshop I did on a similar topic, um, which you can click on. I shouldn't I can't click on that, there's a bug. But, um, it will basically show the notes from a feedback session. So, so this project pulls everything together from all of the contexts it can find, and then does the next step in a loop.

So, you can run everything in a loop. Uh, you can run all of your work in a loop. When I wake up in the morning, normally I have about 15 or 16 draft emails where people have got back to me and it's had a go at replying them, replying to them. I always have to edit them. They're always okay. But, um, but it definitely has a go at at getting getting on with trying to schedule some of my work. I have very specific rules about what it can and can't do.

My basic rule is, is this reversible without embarrassment to me? And, um, if the answer is no, don't do it. Um, and but just make a little note in the project and hand it back to me. Um, so sending emails it's not allowed to do. Creating a slide deck, like for example, this one, that's reversible. It doesn't cause me any embarrassment. So, it just gone on and did it. Um, and gave it to me, uh, for example. Uh, it doesn't post on LinkedIn for me.

It doesn't, um, it doesn't send emails. It doesn't send messages. It, uh, but it does it does get everything ready for me to review. To your point earlier, which is a very long answer to your question, um, it's, um, it has caused me to genuinely question what I'm good at and what I'm here for. Quite a lot of the time, I've got to a point where I'm just the email person who just checks emails and send, check email, send, check it.

That doesn't sound like a a proper job. That doesn't feel good. So, so therefore, what does that mean for my work? And I've had to make a conscious decision which bits of my work do I want to do and which bits of my work don't I want to do. I don't want to be the email reviewer, but I do want to be the strategist. I do want to be helping organizations think through what on earth is going on with AI and how to kind of fix it for their organization.

Now, I could get AI to do a bad first draft, but I don't want to be reviewing AI's draft. I actually want to be doing that thinking myself. So, therefore, I basically said, don't do any of that work. I want to do that work. Just give me all the information I need and I'll do the work cuz I enjoy that work and I'm good at it. So, um, AI can do all of the rubbish work, but it can't and it shouldn't do the work that I'm uniquely good at.

But, because everything is a loop and Ralph loop, this is getting so existential. Because Ralph loops are are so um, are really everything and can be used for everything. We have to start asking hard questions about which of the bits of work we actually want to do. What do we want to do out of this work? It's not just about what AI can do or can't do anymore. Um yes, there's a question there's like there's a question, but let's go to the back.

I think your hand was up first. I think the chap's coming with the mic. Well, we just for the recording, it's really helpful. Thank you. So, with the open-ended tasks, Yeah. >> how would you think about sort of when to stop? So, do you set KPIs up at the beginning? Or how do how do you how do you know when it's done? Yeah, great question. Um I ask it to So, again, this is this comes down to if I just go back to Sorry, different window.

Uh this one. It comes down to upgrading your loop and you have to basically tell it when to stop. So, I don't just have one Ralph loop file that works for all of those different loops that I showed you earlier. Um I have different ones for each. And for example, the worker says, "When you get to a point where you've either running out of context or you've got to a point where there's an irreversible thing to do, then I want you to stop and and report." And what report means is in this case, update the project file, which is just a file in a repository, with what where you've got to and in a way that where you present it to me for review.

So, I'm working quite hard at the moment about about that kind of presentation step because I definitely don't want to be reviewing a diff and just reading text I find really difficult just because there's often a huge wall of it and it's hard to pass. So, I'm getting it to start giving me things step by step in slide format. I'm trying to get it to that interface I showed you before, you can kind of see a little bit how I'm trying to get it to show things in different places in different ways to for it to um, uh, to get to uh, what I need to uniquely do next, I suppose.

So, it So, to the answer your question, it depends on what you're doing. I think the most important bit is that you figure out what the edges are for yourself and and and have that real, you know, moment of, you know, what do I actually want to be doing? How do I want to be involved in this work here? Not just uh, you know, AI's helping me do my work, a companion to me. It's much more now, which bits of work I might not even need to know about, you know?

Uh, next question. There's one here. There's a mic just at the front somewhere, I think. Yeah, great. Thank you. Uh, hi. So, since you brought up this topic of our involvement, right? So, at what stage we really get involved? I mean, you mentioned you don't really review the diff. Um, I guess the most important part of our work now is creating these tickets, right? I mean, first identifying what the most useful feature to implement is and then describing it in a way that you foresee all the different edge cases or um, just explain it the best way possible so that the outcome is what you desired in the first place.

What's your process of creating these tickets? And So, like like my concern is sometimes you don't really know yourself until you start implementing. I mean, the way we used to do it, right? So, during the development process you encounter certain um, different cases where you need like a custom logic, you need to It's just difficult to foresee these things from the get-go. And do you like iteratively improve the tickets and you re-implement them or what's your process? >> Yeah, great question.

So, um, in terms of how I work, there are two modes of work that is done on my behalf. One is the fully automatic work that we're talking about here where the AI will just get something done. I don't need, when I've got a a decent spec that I trust and and there's a way of feeding back so that the AI knows that it's good, I don't need to be involved in that work. That can just happen. For every other piece of work I do, I have a I I work on it with and in Claude Code.

So, um I with that system I showed you earlier, in fact, I'll go back to it so I can show you. Um Let me just change back to this window. Where is it? There it is. With this one, at the very bottom of any of these kind of projects that it's running for me, there's a little thing here which I This is just a Vi Coded app that I've written for me. Nobody else has access to this. Um it has a little VCP command, which if I take that and I type this into a terminal window.

Um so, if I go back to this one, for example, I think this is okay. Um Yeah, I can't I can't easily show that just cuz my internet is not going to be able to connect to my VPS. But, the point is is that I'm able to go back to um Yeah, I'll go back to that. The point is is that I'm able to um type that in and just paste that into my VPS. What that does is it starts a new Claude Code session. That session grabs knows where to find that project and it pulls in all of the project context.

So, rather like loading a skillet, loads it as a It loads the project. It reads the entire project file and it knows where everything else is. At that point, it's loaded in everything it needs in order to supercharge that session with me. And then we work together on it. So, if I'm, for example, to your point about specking tickets, I'd have a ticket I don't know, probably a project for that particular feature, and then I would say, "Okay, load in everything you know about that." And it would load them all in and then we would work back and forth on specking out those tickets.

And then the output would be whatever I needed to get done in order to get that done. So, if I'm working on something that where I I want to usefully and uniquely do that myself, that's when I would jump into a project with Claude code. And when I say do it myself, I don't mean typing it myself or I don't mean doing all the thinking. I normally mean I get Claude to interview me to ask questions so that I can give it the information it needs to formulate to do the writing cuz I don't like doing the typing.

But I get it to pull the information out of me in order to to get that work done. So, those are the two modes. It's the the back and forth iterating and then it's the it's just the automatic stuff. And I should also point out that I don't I don't like reading diffs, but ultimately that's the only way that you can review code. Um when I'm when I'm reading a newsletter item, I don't want to read the diff. I want to read the newsletter.

Whereas if I'm reviewing code that's important um that is for other people, then yes, I read the diffs. I don't like doing it. Nobody likes reading diffs, but I I check to make sure it's working and I will I can't see myself not doing that for a while, especially not with security um like conscious code. Maybe with Mythril saw. Just delegate it. That'd be nice. Any other questions? Uh yes, one here. Um how do you deal with context rot?

So, for example, uh your example where you have a loop and it takes one task after the other. Is it the same Claude code session that takes all those tasks or >> We'll have to experiment with that. With the with the slash loop command, yes, it is. Um it's the same session. Um I when you run it as a kind of while loop outside Claude, then it's a different session. Um you have different trade-offs uh with that. With the same session, you have all the context of the previous tickets and the previous changes.

That might be useful. In practice, I've not found that so useful cuz it can just pull the files as it goes. Um if you're not typing anything into to the session, you're not really adding anything to that, so there's nothing really in there that's useful. So, I tended I've tended in the past to prefer starting a fresh context for each new session. Um but the loop is very the slash loop is very easy to run and it just works.

And especially Opus is very very good at long context retrieval. So, it's less much much less of an issue. Okay. Uh, sorry. Yeah, there's a there's one at the back as well. Is there a microphone as well? Okay, great. >> Um, are you reviewing sessions that are done by by your loops or or are you just reviewing diffs on on the GitHub? Great question. Because >> allow any of my workers to close a project. Um, so uh, I I would always say "If you think you're done, tell me what's finished and I will close.

I will I will close that off." Um, so it could it could be that there's a a big list of completed things that I need to take off check off myself, but I I want to be that kind of final step of verification. The reason that I've added that is because I worry that I'll miss something. There's a thing uh, that someone coined recently called cognitive debt, uh, which is the idea of just not being up to speed with everything that your code base can do or or all of the code in your code base and that that worries me.

So, so I tend to want to to at least understand how the code fits together and and how or how the piece of work that I'm working on fits together. So, I don't let AI get away with just putting something, you know, out of my sight without me having a chance to look at it. Otherwise, I feel like I'd lose track of what's happening. Yeah, because I I mean, uh, for example, I um, I I'm using sessions to to track uh, tickets.

Yeah. So, so instead of reviewing the code or diffs in the code, I'm just reviewing what the particular session was doing. >> Yeah. And I even have a like a marking system which which session is on which status. Yeah. Is there any way how how you do it similarly? Um, similar. Yeah, I think the sessions and the status I I I think that can work. Um, I I haven't tended to use sessions like that. What I tend to do with sessions is I get Claude to every night go through all of the previous sessions that I've run that day across all the machines I run Claude in.

It saves them all into a JSON file for me. And then I get it to both figure out how my system could improve and also just what I did so that I haven't I don't forget what happened. So it writes a little paragraph for how much I did. And and I use that in order to to kind of track work. the quite the same as one ticket per session. I quite like the idea of having like one context per session. I think that's quite a nice idea.

And one sorry, one um, like by per unit of work. I I just haven't made that work, but that's something I >> I I found it really useful because then then I can go back to the particular session when the particular thinking was happening. Yes, and I do do that for sometimes when I've got a project that's running over multiple sessions, I can go back to the previous session. Um, instead of the VCP command I showed you, I could type VCR and it'll do the same thing.

But um, in practice though, I I like the discipline of it having to pick up again because it means if it has to pick up again from a fresh context, it means that all of the information that was in that session has actually been codified into other places that any Claude code session or human could find, which means that you end up with a a much more I guess richer kind of repository of knowledge that you're working in.

So So there's a question mark around if if session if you if sessions are truly not ephemeral and you've got them as a store, are they accessible as future context? If you treat them as ephemeral and make sure you capture everything within them into your repository anyway or into document quotation files or whatever, I think that could be more powerful. So worth thinking through. For sure. Any other questions? Feels like we've come a long way from just write this ticket, but there we go.

It's good It's all good. Yeah. Thank you so much for the talk. I have a question. It seems like in a loop self test might not be necessary. Like it might go to the code and then find nothing there. Yeah. And would you consider to optimize it somehow or you just let the token burn? No, just burn the tokens. They're not that expensive. Depends what you're doing. I think we're at the point I should This is a whole other thing.

We're basically in the era of free tokens right now. You know, I have a Max 20 subscription I definitely use more than the average person probably who is is paying for one of those. So I think at this point I would optimize for for freeing your own time up as opposed to optimizing for for burning a few more tokens. I don't think tokens will ever get that expensive. I think that the frontier models potentially will be very expensive, but we have really good cheaper or free alternatives just around the corner.

Not quite as good for the latest kind of work that we're trying to do, but they're really really good. So you know, I think um There was at least one that just came out the GLM one that just came out looks really promising. That's the ZAI one I think that just came out this week. Really really interesting. I'm still running Claude, but that won't necessarily always be the case. So I think I I would just burn them. I would like I said at the very beginning, you know, I spent a long time doing the whole optimization thing where I was doing this if you weren't here at the beginning this thing.

You know, I spent a lot time trying to to screw around with with all of this, but but ultimately I just now let it run. It's much simpler. I do get quite close to the end of my Max subscription sometimes though. I'm slightly slightly nervous about what that means. I have to figure out how to get another account. Yeah, Max the $200 a month one. Yeah, yeah. I get pretty close to that every week. I'm quite all right 80% now.

Getting the jitters. Yeah, you had a question. Do you want to bring the mic back down? Is that okay? Thank you. I'm not looking at that anymore. Hello. Is this what I mean? I wanted to ask you about Thank you so much for the presentation. Yes, sure. I was fine-tuning for for the prompt. You version it. Do you have data sets that you use to fine-tune your entire loop? So, in terms of versioning the Ralph loop specifically, like the prompt for the Ralph loop.

Yeah, just the same. >> I use skills for that. So, as I pointed out before, everything like that goes into the skill, and I get Claude to write the skill for me. And that saves in either your your .Claude skills folder within your project, or it goes into your home directory under .Claude skills. I use GitHub to version all of those for myself. I don't think Git is the right skills format for this long-term. I think we need a new thing.

Hence, trying to build S- skills, in fact. Which is this idea of trying to make skills much more portable and shareable within teams, which I'm trying to figure out. So, yes, I do I do version control them, and I treat them as quite important code. And I don't actually I do share some of them, but I don't share all of them routinely because they There's a lot of my own IP in there, and actually a lot of my customers' IPs in there, too.

The question is more with regards to the performance of the prompts. So, you were saying that in the beginning you you go along, you improve the prompts as you go along. And but are you versioning the that going along, and are you versioning the the performance of the prompt overall? So, when you say prompt, do you mean the the skill itself that I'm using? Yes, yes, yes, yes. Yes, so yes. So, the skill So, the prompt lives within the skill.

So, when I type {slash} bug tracking or {slash} Ralph, Uh, that that is the prompt. That that that gets written by Claude and managed by Claude, which means that the um uh, that whole file is is is the prompt and therefore that is version controlled. So, I I always I have a get running within that setup and then every time I change it, I update um I update the >> but it but it remains subjective. How much you mention that you have improved.

Let's say that you have your data set will be an issue. I say that. And the expected output would be the new feature added to the repo. >> Yeah. So, you could >> how do I evaluate whether it's any good or how >> Yeah, exactly. How do you know you're actually improving? I see. So, how do you know if you're improving? That's a really good question. Um, I do stress test my skills so with other skills and I say, you know, is this skill any good?

Could you improve it? Could you write it? Um, I do I I spend quite a lot of my time tinkering with my system and my skills probably more than I should. Um, I think um it's a bit subjective at the moment. What I haven't done and it would be a really good exercise is to try running um blind testing where you would run a set of tickets with one skill and a set of tickets with another. Ultimately, because Claude is non-deterministic anyway, I think there's a high level of variability with any of those kinds of tests.

So, it's it's really difficult to think about how to to construct a a useful test in that way to know whether you're actually improving or not. Um, in general, the more context you give um into your prompt, the better it will do up until a point which isn't very easy and obvious to think figure out where it becomes worse. So, um it's about kind of balancing that ultimately. But yeah, I haven't done a kind of objective improvement process.

A great question though. There's a question just behind you. How are you version controlling the skills? I'm using GitHub at the moment. Um, I do have a product that I'm trying to build, which is I mean, this thing here. So, if you want my skill, by the way, that's what how you get it. Um, it's a project called air skills, which you saw a brief preview of earlier from my my slide deck that my agent put together for me.

But, the idea is that you can package and manage those skills as a unit. So, you can create skills for your organization, you can create skill bundles, you can create a skill set for your org that works for different teams within your org. Um, and then that all gets versions and updated for you without everyone having to learn how to use Git and GitHub. That's the idea. It is a real pain at the moment. I found it really really difficult to manage just for myself, even just putting a skill on GitHub.

You know, I can't imagine anyone from That's quite a lot of friction for for a coder like me. It's I can't imagine non-coders using that. So, so yeah, trying to trying to build this. Um, so yeah, run that command on your machine, you'll have my skill. What is the future? Um, sorry, there's a question just here first, and then go next. Um, how do you You sort of touched on this a little bit, but sort of around the edges.

How do you do knowledge management? So, I guess you know, I use Claude for why is my VPN not working? And then I learn something, and I want to record that, and then I'm like, I've got a some a meeting with somebody with a transcription, and I have that somewhere else, and I've got a bit of code that I'm writing, and all all I've got all of these different contexts, but they're sort of very disorganized. Do Do you have a way of thinking about how you organize all of that? >> Yeah, so I have a code directory, and I have a vault directory, and those are the two directories I work in.

So, the code directory contains a few different projects that I work in a more classic way. The vault directory is where I do all of my other work, and and frankly, I I mostly start working in there even if I'm working on code, and just tell it where the code is, because the vault contains several thousand files with all of of different stuff that I have picked up, learned, um, worked on with Claude over the last several years.

Well, not with Claude for that long, but you know what I mean. Um, I started with, uh, Obsidian a long time ago and I've been working on that vault for for a long time. Um, and and with Claude now, it just works on that for me. So, when I do some research on how to fix my VPN or whatever it is, it just saves a a file in there. I have some specific rules for how to kind of structure and manage that. Um, if you're interested more, I haven't written a lot about this, but I know Andre Karpathy has just written about it using LLMs as a wiki.

That's a great article if you haven't seen it already. Um, I know there's a a Mina Jovovich, actually, funnily enough, has done a thing on Mem Palace yesterday. That's another version of this. Um, uh, you can do you can use that, too. There's lots of different systems for that that out there. The best way to get started is it's markdown files in a file system and use it like a wiki. So, I run Obsidian in one window and, um, Claude in the other and just kind of work with it and and save things as you go.

So, do you have an agent that then structures and puts those vault into folders or something like that? >> Yeah, so it depends on your method. I use the Zettelkasten approach, which is the one where you have one note per thought. So, any thought of all just goes into a flat folder. Then I have a {slash} projects uh, thing which has all of the projects that you saw, um, including one for this presentation, um, which which is my kind of unit of work for an agent that we work on together.

It does a lot and then I do some and then it does some. Uh, I have, um, transcripts in there. All of the the calls that I've ever recorded go in there. Um, and I use a a tool called Lianne, uh, which is a command line embeddings tool. So, it basically just runs embeddings across the entire all of the text in the repository, all of the transcripts, all of the links I've ever saved, including all of the content. It's huge.

Um, and, um, then it can find things usefully and easily in there. Um, so, you you the best time to start that is today because it just takes years to put together. I need to write more about that. Any other questions? Yes, one here. You said you had friction while versioning your skills. I've been using skills only for last month, so I'm not aware of this friction. Can you explain what the friction is? >> Um I can. I'm sure there are Has anyone here had had any kind of friction with my kind of managing and using skills yet?

Is anybody else? Yeah, quite a few different people. So yeah, it's it's a it's an emerging thing. It's not surprising you haven't experienced it yet if you're not using it for very long. What I found is that if you're just using them on your own, creating a file of skills and and managing them is is quite straightforward. Putting them in GitHub is quite straightforward. It's simlinks and and GitHub repository. It's fine.

What Where it becomes difficult is how you share that. So how would you share a skill? Okay, well, if you want to use MPX skills, you have to then put it in its own GitHub repository. That feels quite heavyweight just for one skill. I'd have to have 50 of them in order to share all my skills, so that doesn't really work. Then it's more like, okay, if I don't want to do that, I just Do I just send them the skill file?

Do I send them a zip file? I mean, I can't think of a better way of doing it. Um do I have to have a submodule in my skills folder for every single Git repository I share a skill with? It just doesn't make any sense. So I think I think the idea of Claude has got some stuff in there around plugin marketplaces where you can have a plugin which has a bunch of skills. That's the best way, but then you're versioning the plugin, not the skills.

So that's probably the most seamless way. It does It just doesn't work that well. Also, there's a challenge around if you If somebody contributes to your skill, do you want their changes or not? It will depend on what the contribution is. Are they local to just them or are they um changes that could be generally incorporated? And that depends on the skill and depends on them. So you have to then manage that. So do you run a backlog for each skill where you have tickets to improve the skills?

Do you see Do you see the kind of I think these are all unsolved problems. I'm trying to My contribution is trying to solve some of those, but these these are big problems we haven't figured out yet. Um other questions? There's one right at the back. Um if there's a mic, that'd be amazing. Thank you. 1 2 1 2. Okay, it's working. Uh my question is about how we can scale up this approach with Ralph loop, but like for the actual production team, like I I don't know, three engineers.

How to coordinate, how to cooperate. Do you have any idea how we can organize it? Do you have any experience? That's a great question. How so just to make sure I've understood it, how do you scale this up so that you can coordinate whole teams using this kind of looping approach? Is that Yeah. With all the tickets and the skills, yeah. 100% Yeah, it's it's difficult. I think the teams that I where I've seen this work well is where they are proactive about updating tickets.

The great thing is if you connect your ticketing system to the AI, it's really good at updating it, so you should definitely do that. Um make sure that you claim the ticket and move it into the doing column before it starts work. Um and make sure that somebody else hasn't just done that before you start work. Do you see what I'm saying? That's really important to avoid contention. Those have always been issues with with bigger teams.

Um just in the same way, a couple of controversial things, just in the same way that Ralph loops work really well by just doing one thing in a loop and quite sequentially, um you know, it could well be that the coordination overheads in our teams is caused by the fact we've got too many people in our teams, and maybe we should have smaller teams and just more of them, right? So, maybe maybe if you're trying to get 10 people to coordinate in using AI and Ralph loops and all of that, that's just not going to work.

Maybe you need three. And maybe that's the way to to kind of run that project, and then you split it down, and then you have another the other seven people doing something else or or whatever. Does that make sense? So, so making the problem go away is the first step, and making sure that you're already using your coordination mechanisms is the second step. Um and then just try it and and figure out what what the bottlenecks are.

Be really No, be really good at retrospectives with this stuff. I think retrospectives and teams are often pretty anemic. It's like, "What should we do less of? What should we do more of?" That's just a recipe for the same and more of the same. Um and just changing tiny increments, which can be good, but ultimately this requires a radical rethink. So, be really conscious in making sure that your retrospectives are changing actual things about how you actually work or or have space to try let's just try using a Ralph loop on all of our work for a week and see what happens.

You know, and if it doesn't work after 2 days, that's fine, you know. And then if you are someone who's in a leadership capacity and is able to sponsor that kind of work, this is what it means to try and move to AI. If you want to transform your team, you are going to have to sponsor these kind of experiments and be okay with failure because So, I'm speaking to the leaders in here for a minute. Because it's going to be messy and and it's going to it's going to fail a lot, but if you want real transformation, that's the only way to get it.

You've got to give your team space to try a whole bunch of different things. So, give them air cover. Um so, yeah, it's it that's a big and complicated question. Um I think if you're able to and have the agency to just try it and see where you it gets to. There's a whole uh separate thing called uh theory of constraints, which I haven't talked about at all, which is the idea that within any team in any system, there is always a bottleneck.

There's always one bottleneck that's the big bottleneck. If you don't work on that one bottleneck, all of the other work that you might do to optimize and improve the system is pointless. And actually probably counterproductive. So, this is why some teams when using AI tools and using advanced AI tools like Ralph loops, which is just you know AI or you know, what we're doing now, but just on steroids. Some teams when they implement it actually go slower.

Some teams go amazingly fast. Some teams go slow. Why is that? It's because they're not working on the constraint. The constraint in those teams might be the review process. If you or the release process, if you release your code once a month, um and you're shipping 200 PRs not 20 in that release, how do you think that's going to go, right? You know, it's not going to go well. So, that's why teams go slower because what they need to do is fix their release process, not their coding speed.

Um so, always fix the thing that is the biggest bottleneck first, then figure out where the bottleneck moves in the system, and that's not predictable, it's random, so you have to figure that out, then move and fix the next thing in the system. Uh for more on that, read The Goal by Eliyahu Goldratt from 1984 no less. It's an amazing It's amazing book. Um one more Is there another question down here? Is there another mic?

Where's the mic? >> I have the mic. Oh, you got the mic. Great. Keep going. Um S- since you were asking about like talking about the constraints, but this reminded me like I'm part of the AI team, and we have an AI team and they write a lot of microservices, and it's in different repositories. Okay. How do you deal with uh like coding now since is it like one big mono repo or is it like small small repos? Um you you have to try it different ways and see.

Um I don't think that the the GitHub or sorry, Git architecture, whether it's many repos or one repo, really matters. You can always start your AI in um a folder above all of your other repos and just get it to work. It does a great job of that. So, that's okay. I think the bigger question is what are the coordination patterns within your teams and your services, who's responsible for what, and how does that change with with AI?

I think that's a more interesting challenge. The main reason I'm asking this was because like some of the microservices depend on others, and then you have to release one of them and you release a tag, and then updating other That's just >> Yeah, I think what AI will do is it will expose all of the places in which that process is inefficient because it will do everything faster, which means that you if you're seeing those bottlenecks where you are getting dependencies between your microservices, guess what?

That's your biggest bottleneck. Um therefore, you fix it. So, how do you fix that bottleneck? Well, you might try atomic release system or you might build something that um using cloud that are using a raft loop that um figures out a way of um coordinating releases across multiple repos more successfully. I don't know. But, that's what you do. That's where you work. Don't work on anything else until that's fixed if that's the bottleneck.

Yes, a question here. Do you want to pass the mic? >> Yes. I should say I'm kind of at the end of the content. There's if you which you probably was was clear half an hour ago. Um the only other thing I I mean, I had a Q&A slide. If you are leaving, you're welcome to leave, but you're welcome to stay for more questions. I would really appreciate some feedback though. So, this this QR code is the only thing that I manually added to these slides.

Um if you could just um fill that in, that would be lovely and amazing. Thank you. It's literally only 3 minutes, four questions. Um it just helps me to improve and make sure that I do a good job of these workshops going forward. Um that's also my LinkedIn. I post a lot of content on there. Do um do connect with me. Do um I encourage you to meet some of this stuff. Disagree with me. I love disagreement. I love it when people say, "Surely, Chris, that's nuts.

You shouldn't be doing that." Love those kind of comments uh because it really helps me to think and improve, which is what I love to do and cuz all cuz the raft loops is doing all my other work, so. Got no got nothing else to do. Uh great. Thank you. Um so, I just wanted to put that up there. Very happy to continue answering questions though. Um but, if people wanted to drift away, then that might be a good time. Go for it.

Um my question is regarding multi-agent orchestration tools. My I'm curious if you've tried things like CVA G's Gas Town or Yes. >> There's another guy who does like MCP agent mail. Yeah, there's some really cool and interesting stuff. I still think we're in the Wild West, literally, with Gas Town things like that. But we we don't really know how how that's going to go. I have tried Gas Town. I couldn't really get it to work, uh but it was pretty early on.

Uh for me, I feel like the the agent orchestration side of things is I I think that we overcomplicate things by assuming that they need to be in parallel. I quite like the idea of just starting with a loop to start with. Uh I don't feel the need to um have my AI um do lots of things at once before I can get just get it to to to do one thing well. It kind of goes back to the theory of constraints thing again. Um I don't think speed of, you know, number of tokens per second is the bottleneck.

I think that it's our ability to specify what we want and review what the AI has done. So So if that's my bottleneck, I don't want to introduce more agents. So I haven't spent lots of time with those tools kind of for that reason. I feel like they're solving a problem that not many people have yet. So So Hello. I go, yeah. Not just for speed, but for example, I don't know if you've experimented with MCP agent mail. So agents can uh lock files and speak to each other so they don't step on each other's toes, and you can use different like Claude, Opus, and Codex work on the same project.

So you get different brains working on the same project. >> Nice. Yeah, no, I haven't I think I've heard of it, but I haven't tried it. Sounds like a super interesting idea, rather like uh someone mentioned earlier about sub-agents trying to look at things from a different perspective. I've had a lot of value for with doing that. I mentioned earlier my simulate audience um approach, which takes um ultimately the way the way that it works, by the way, is it takes like um uh transcripts and also survey responses on my website, and it it creates personas and has those personas think differently in parallel sub-agents to take fresh looks at content from different perspectives.

Um so that whole idea of having it um having two different things in two different models as well in that instance is a super interesting one. I think we'll see a lot more of that. I can see a lot of value in it. We definitely know that a they're pretty good at agreeing with themselves and you often get better a better contrarian take if you throw away the context and look at it again. So, great principle. I think that the tooling is still super early, which we all know, but they're interesting ideas for sure.

Uh there's a question just behind you. Yeah, just keep going. They they'll turn on. So, great talk by the way, thank you. >> Um how much importance do you put on So, in terms of phases of how you develop your spending a lot of time building out the context, creating the tickets, and you have a system to run them in sequence, gets pushed up, how much do you focus or emphasize on CICD, running automated tests, linting?

Um does that give you the confidence to reduce the amount of code you're reviewing? >> Absolutely. Uh well, yes and no. It depends what the code is. I think Firstly, I think CICD, good testing is absolutely essential. Linting and all of those things. If you want an AI to do a good job for you, why wouldn't you give it those tools to to help it do a good job for you, right? Just the same way that humans do much better when they have linting and CICD and good tests.

It's exactly the same. It's the same with clean code bases, you know? Um you know, it's worth doing all of that work