I Turned Coding Agents Into a Strategy Game — Ido Salomon, AgentCraft transcript

Okay. Hi everyone. Uh I'm Solomon. I am the creator of ATMCraft and I also created MCPUI which became MCP apps and MCP steering committee. I've done a bunch of other stuff. Uh but we're here to talk about why we're the bottleneck and why we don't have to be. So, uh as you all know and use every day, agents are amazing. There's so much they can do uh from generating cat memes to B2B SAS apps. So if they are so amazing, we have to ask why aren't we all unstoppable?

Why doesn't everyone have an army of agents that just does whatever they want? So it should be relatively easy. I mean you just have a bunch of monitors and you spin up 25 cloud code agents and you're done. So the reality is that it's just not that simple to do something like that. you need to a really want to use agents and secondly once you actually spin them up each one of these agents requires you to steer them direct them review them and when it's at scale especially it's just exhausting we need to find the skills to allow us to do it without burning out so in reality we are the bottleneck and the the cool thing here is that while we are the bottleneck The skills that we need to do these things to uh find ourselves not burning out every time we use an agent are with us all along.

Uh if you try to compare it to other stuff that you've been doing outside of programming, for example, playing video games, it's not really that different. Like if if you're playing Warcraft, for example, or Sims, and you have a bunch of different uh units running around, you need to understand what they're doing and supervise all of it. How different is that from from actually inspecting agents? So I took this concept uh too far and I created agent uh which is a game inspired orchestrator uh that tries to bring all that good stuff together into work.

So let's take uh a quick look of what that actually looks like and try and understand the journey that we can go through to really raise the ceiling of how humans and agents can collaborate. So, uh, this is AgentCraft. Uh, there's really a lot to unpack here. Uh, so we'll just go through the basics really quick. Um, what you see here is an agent. Uh, it's an actual representation of a cloud code or openclaw, whatever agent that you can imagine.

Uh, it can be detected on your device or you can actually spawn new ones directly from a craft. Uh, so like I said, you have like codecs and uh, open code etc. at your command. >> Uh, and these agents can do everything. They're just normal agents. Uh, so I can just prompt them in this nice side panel that also has voice. So there's multimodality here. Uh, and just do something for me. >> Prompt. Prompt. >> And now these agents can do whatever.

Uh, and if you look around you would see that the map has a a bunch of different elements. So you have buildings. These buildings represent functionalities. So for example, if you can manage your plugins or skills, uh you have like integrated terminal, integrated git, uh you can do in your all of your work from within this space. So now that we have this and we understand the basics, let's look at what we can do uh to raise the ceiling.

So the first step is obviously visibility. I need to understand what's going on. What are my agents doing? So, I have this nice side panel and it shows me the different agents and what their task is and what the last thing they did is and what they're currently doing. Everything I need to quickly understand who needs my attention and who doesn't. The second part of it is we're not just limited to this. We have a map.

So, I can take let's say my file system, project it onto the map, and now I can see exactly what my agents are doing. Not just by looking at a list of stuff. I can actually see okay this agent is now working on this directory or this file and these files are represented as runes on the map. So I can really look at each one and see exactly what each agent did and when. And now I can take this nice information because we are the orchestrator uh and use that to create linkages and now I can create cool stuff like heat maps.

So I can really easily see what's going on. And the the other aspect is that now that we have this visibility, that's not really enough. We need to be able to react to what's going on. If someone needs my attention, I need to be able to quickly jump to it. So much like video games or RTS games or Civilization, whatever, you would click uh spacebar and it would just take you to the nearest thing that actually needed your attention.

So you can really easily understand what's going on. and jump from one thing to the other and answer questions or approve plans, etc. That was nice. I mean, it's a nice step just to have that visibility, but it's not really enough. It's pretty exhausting to keep looking at what the agents are doing in the granularity of what files are they using and what tools are they using at the moment. So, we need something else.

Uh and the reason is that we have like two two main problems here. Uh one is that there's just too much to hold in our heads. Even if I can understand what's going on and I can see everything, I can't really do like 20 things at the same time. I can't even think of the next thing I want to do if I have this, you know, huge coding project or whatever. So I need to find some way to fix that. Uh so we can I mean we can tell the agents uh you know go through my codebase, find out what the next things I need to do are, create these tasks for me and I just click a button, I accept the quest and there's some agent that does it for me which is fun.

Uh it's cool. Uh but the next problem here is that okay now I have 20 stuff going at the same time and I have to babysit each and every one and that is completely exhausting. So you could ask okay we have agents agents are amazing as we've established so I can just go ahead and tell the agent to do it for me now it's called software factories but at the time it was just doing stuff autonomously uh so you can give it like a general sense of what you want it would spawn an orchestrator and break it down for you and run it all in well in this case it's a local uh container so everything is just done in isolation and I don't need to do any babysitting the agent does it for me.

On top of that, you have loops, which everyone really likes right now. So, you can do cool things like just tell it to, I don't know, scan Twitter for cool stuff you want to build or scan your GitHub, and it will just create these nice uh features for you in the background without you needing to do too much up front. The bad thing is that I mean this is nice uh but what do I do when I have again I don't know I would say 20 but even five things to review at the same time.

Uh it's very very hard to get that out of the way. So what I did is create uh this like review kit. So I get the entire changes I need and I get uh what files changed if you want to go like file by file or you can get just get visual evidence. So you can actually see videos of what changed or photos of what changed so you can really quickly understand what you want or even run a few uh instances in parallel and just pick the best implementation out of these.

So now that we have this we have uh visibility and we have autonomy uh it's pretty cool like we reached a pretty good place in raising the ceiling of what we can do with agents but it's still pretty exhausting. I mean it's it was fun. Uh, but now I have to ask myself, do I really need to be the only one doing this? It's pretty lonely to just talk to agents all day and trying to and you know, if you look at uh what a team does, for example, if it's just me, that's fine.

But if I have a team, I might not even be the right person to review what's going on. So, I need something else. Uh, and that something is collaboration. We can just have other people join us. So the way that looks in agent craft is that I can create these uh you can call it you know war halls war room whatever uh host them it's like completely local but you can do like tunnels and people can join you h so I can create a room and let's say I have a product designer in my team in this case uh it's my wife and I can approve uh her otherwise she'll be mad uh so I can approve her and she does her thing she designs whatever and I as an engineer can not only see that that's that's her um I can not only see that on my map uh I can actually take that and follow off off of it >> prompt prompt >> yeah that's just like her um so I can take that design follow up on it implement it while she keeps working uh and the cool thing here is that not only can I hand off or fork and kind of have this shared workspace so I'm not really bound anymore or to Git.

I can use Git, but I'm not really bound to something. Everyone is working together. Uh you can see that there's also some kind of chat between not only between the people within the room, but also between agents. So we can have this really nice notice board of exactly what each person is doing, what each agent doing. They're all aware of each other and they can collaborate much more closely uh than you normally would be able to do.

Uh so for example, here I can say, okay, I took off this design task. Uh so now we have oh by the way this is also from devices you can have like mobile and and if you like telegram you can use telegram whatever floats your boat. So we have visibility and we have autonomy and we have collaboration so we managed to raise the ceiling. Uh it's now possible especially if you're power users to do really cool stuff and use more agents and do more things.

But the interesting thing here was, you know, if I looked at the feedback I got, it's not just that power users were able to do more. The cooler thing for me was that people that had even nothing to do with agents or nothing to do with productivity, uh, come and say, uh, okay, my kids really loved it and they use it all the time to to orchestrate agents, which they didn't do until now. uh or you know someone that flunked out of college because he played Starcraft and now you can suddenly do really productive stuff with it.

Uh and a bunch of other examples of kind of bringing communities that so far were kind of behind into this new world of productivity. So we need to raise the ceiling but we also need to uh lower the floor. We need to have some kind of way to take no probably around 90% of the people in the world and bring them into this hygienic future without having them burn out. So again, I took it too far. Um, and there's this uh currently named TBD um potentially loopers.

Uh the idea here is okay why don't why do we need something like you know Warcraft which is potentially very complicated uh especially or might be even daunting if you're not into the granularity of what files are there are or how uh complex the UI is and kind of try to find a lower denominator of what makes stuff accessible especially now that agents are much more capable so instead of granularity or having like exactly what files they're working on maybe I can have it more of a mobile game level uh have projects that they keep returning to and use loops and other stuff that rely on agents being more autonomous to have like a very basic level of I prompt I get the result I don't really care perhaps about the way I can see it if I want to and I have this really nice interactive and cool experience that makes me want to come back to this and obviously I mean this is one visualization uh it can really resonate with some people but might not might not present to others because we're not all the same and what may appear like amazing and easy to one person can appear very difficult to another.

Uh but here uh I'm trying to look at a way to make this more customizable. Try and understand what it is that you as a person need in order to understand and want to use these tools. So this is a very very early draft. Um but if you want to use it, please let me know. So we're the bottleneck. I think we've established that uh pretty well until now. But the skills that we need to stop being the bottleneck and enable this new future are already with us.

So we are the bottleneck, but we don't have to be. And I hope that you all can take what's relevant to you and your life and apply it forward. So I would ask you for two things. One, agent is already out there. Like you can just npx install agent as a website uh and do your thing. Uh this new thing that I just showed you uh potentially loopers uh is still experimental. I'm still trying to understand the interest and what people are actually looking for to uh get the floor that they want or the ceiling that they want.

So, please uh go here and express your interest and tell me what you're looking for so I can know what to do with this. And thank you. I got it.