How I Code With AI Agents (Spec-Driven Development) transcript

If you write code with AI, you've probably heard about the term specd driven development. In this video, I'm going to break down exactly what it is and how you can use it in your workflow to improve the quality of your code that you generate when working with AI agents. One of the big challenges I found when I was approaching this topic is there are so many different frameworks. There's a lot of different terminology and no one really agrees on how to implement spectrum in development.

Everyone is doing it slightly differently. So in this video, I'm going to give you an opinionated and much simpler breakdown of how I use spectrum in development [music] in my own workflow. Okay, let's start with a quick definition. So spectrum development is the idea that instead of just prompting first and figuring things out as you go, you define a specification up front. And this is just a document that describes your intent about what you want to build.

It's a short document that defines what you're building, the constraints, so what you're not building and what you're not doing, and the key decisions, and it's like a contract that you can use for how your code should behave. So, when you tell an AI agent to go and add authentication to your application, there are so many things you need to consider when building a feature like this. Claude is pretty good about prompting you and asking you some clarifying questions, but ultimately, there are so many decisions you need to make about how to implement this feature.

And if you just allow the agent to work on its own, it's going to guess about what you actually want, it's going to make decisions that maybe don't align with how you should actually build the feature. So the way you fix this is to use specd driven development. And so what we do instead of just telling the AI to add authentication, what we're going to do is we're going to work with the agent to write a specification, which is a plan for what we want, we're going to define what we're building, the constraints.

So things like what libraries do we not want to add? How should the feature be implemented? Should we add tests? How should we add tests? What library should we use? Then the AI doesn't have to guess about what you want. So before you might have a vague prompt like add authentication to the app. But with a spec we can be really really clear. So we can say we want JWT tokens. We don't want password reset. We don't want oorthth.

We don't want to install new dependencies. What we're doing now is we're being much more clear to the AI agent about what we want and what's in scope. So, one thing I would say is that there's a lot of confusion about the terminology. I hear words like PRD, um, technical design, doc, specs, all of these words can be a bit confusing and I think people are using these terminologies very differently. So, in my mind, the way software engineering has always worked is you typically have a product requirements document.

This document is intended for human beings to read. So this is for product managers, stakeholders to decide about what are we building, why are we building it, what are the what's the business value for this feature. Then in most tech companies, you'll have some kind of architecture or design dock for any significant feature. And this is intended for engineers. And this is all about how we're building something. thinking about the architecture, thinking about the constraints, thinking about scalability and security and all of the important things we need to consider as software engineers when we're building anything.

And then the other document you can think about is the AI spec, which is what we're talking about today. And in my mind, this is a document designed specifically for AI agents. So what we put in this document is not designed necessarily for humans. It's more designed as an execution document for agents. So, a spec for an agent should probably include some why and how, some context about what we're building and why because that will help the AI agent if it needs to fill in any guesswork or make decisions.

Um, but it's not a debate. It's kind of a plan that the agent is going to execute for you. This is an example of a typical spec you might see. So, this has a a what and a why. So, this is a bit like the PRD um elements about, you know, why are we building this thing? What are we trying to solve? But then more importantly, we also want to define the constraints about what we're building. So AI agents or coding agents in general are quite eager about solving problems.

So they're going to try and implement the thing. They're going to try and get it working, but there may be certain requirements we want to enforce like don't use external libraries or don't use certain design patterns. Some things may be out of scope. So if you go back to the ad authentication example, there are so many possible things you could add that would be in scope there. You want to really shrink down the work to specifically what you want.

And then ultimately we want to break down the work into a number of discrete tasks. So task one, what to build, which files to touch, and then how are we going to verify that that task is complete. So this is the same way we would break down tasks in an engineering team. We would break down the work into a number of tasks and then we would give those tasks to engineers to work on. After the engineers are complete with a task, they would typically test the work. they would put it up for a code review and it's kind of the same cycle that we're doing here.

So here's a highle view of how you might work with a specd driven workflow. You generate a spec. If you've got a large enough bit of work that needs to be broken down over multiple tasks, maybe you're going to work on this over multiple coding sessions. You generate a spec using a metaprompt. So we describe to the AI agent what we want to build. The agent will write an initial spec and then we're going to review that spec.

We're going to iterate on it. Make sure that all of the details in that spec are correct. Make sure the agent isn't having to guess about the implementation details. Then we break the work down into a number of tasks within the spec. The agent can help you do that as well. And then we want to run each of those tasks independently. Then we're going to review the work. We're going to iterate and we're going to commit. And this is nice because we can gradually work through a task step by step.

And this is the way we've always built software. You don't start by giving a software engineer an entire feature dock to go and build. You would typically give them a single task to work on and they would incrementally work through a problem step by step. So let's take a quick look at a specification example. So this is a hypothetical uh specification for adding JWT authentication to your app. So we start with Y. So users currently share a single account.

We need individual accounts to track usage per user for billing and to enable personal settings. Then we're defining what we want to do. So we want to make sure that users can register, login, and refresh tokens. We want JWT-based orth with 1hour access tokens and 7-day refresh tokens. And you can see here already we're adding quite a lot of detail that is really important to the implementation. So 1 hour access tokens.

We want to have 7-day refresh tokens. This is the kind of detail that would get missed if you were just allowing an AI agent to go and execute a slightly less defined task for you. We're also being clear about how we want to structure the API. So when an AI agent is now given this specification, it's completely clear and non-ambiguous about what they need to build. So they're not having to guess about what you want. We're also defining some constraints.

So using specific libraries to store the user data in Postgress via Prisma and then we're also declaring about how to handle errors and then we're also saying that you must not add new dependencies and you you must not store tokens in the database and then we're also talking about things that are out of scope which an AI agent may otherwise go and implement because again they're very eager to to build and then what we do is we break the work down into a number of tasks.

So you can see this is how a senior engineer might break down any bit of work. We start by adding a user model and we can implement that task and then we can verify it with some tests and we can review the code. We can commit that change and then we can move on to the next task. So then we want to add the endpoints, we want to create the login endpoints, we create the refresh endpoint and so on and so on. We go through the whole feature until we're done.

And then afterwards, once we've finished all of the the individual features, we may want to add some additional tests just to test the end-to-end flow and ensure everything works correctly. So, as you can see, there's a lot of detail here for an AI agent to take. And so, when the agent is going to implement the work, it's very clear about what it needs to do. It's worth mentioning that as you go through the specification, you may find that you implement task one and when you review the code, you may see that something is wrong with the code.

You may want to change something. You can always go back and change the spec. This isn't like a waterfall design where you have to know everything up front. You just want to give enough detail to the agent about what it needs to do. You can refine the spec. You can go and execute with an agent, see that there's something wrong, and then you can come back and correct it. That's totally fine. Here's another example where we're adding hybrid search with PG Vector.

This is a complicated feature. There's a lot of details here. And if you were to give this to an AI agent or if you were to vibe code this kind of thing, you know, you just kind of allow the agent to build it without you giving any input, there are so many things that the agent would probably do wrong. So there are so many things to think about. So what dimensions we want to use for the text embeddings? We're going to use fast API.

We want to use OpenAI library for embeddings. So again, we break this down into a number of tasks. So we start by adding the PG vector extension. This is just what you would do as a software engineer, right? You would take a design, a senior engineer or a tech lead would break the work down into a number of tasks and you would do those tasks individually step by step and you would review the work along the way. And this is exactly what we're doing with agents.

So let's take a look at this in action. So I'm building an application called YouTube IQ and this is a software application I built for myself. And the idea of this application is that you can basically track particular channels and it's going to give you guidance. It's going to coach you and give you insights into different videos and how to improve your video content in general. So you can see here we've got a bunch of videos that we're tracking.

There's also the ability to get coaching on your channel. It will give you feedback and then it also gives you the ability to ideulate. you can, you know, research a particular topic, find videos that are that are trending on that topic and give you suggestions for titles and things like that. And so I want to add a new feature to this application. So jumping over to the terminal. So what we want to build is a new feature that's going to analyze a video.

So the idea of this feature is we're going to paste in a YouTube URL and then the AI is going to analyze that video. We're going to pull down the transcript and we're going to generate a bunch of insights about the video about the introduction, the key points, the pacing. We're going to learn all about that video so that we can understand it. So, let's go ahead and implement this. The way I would do that is I have a single slash command called spec and then I can really briefly describe what I want to build.

Help me design a feature that allows a user to paste in a YouTube URL. And the idea is that AI will analyze that video. It'll pull down the transcript and tell us all about the video, what made it work. We're going to analyze the hook, the pacing, and the key points of the video. And then you're going to give suggestions about how to improve the video. I want you to do this in a new tab. This is going to be a new uh UI feature.

We're going to have it under an ID8 tab. Help me design the spec. All right. So, we're going to allow AI to help us build this specification. So, what this is going to do, it's going to use a template. Um, you can see here I already have a uh existing spec. We're going to overwrite that. So, let's do a new analyze page. So, where should the video analysis feature live in the UI? We're going to say a new analyze page and we've generated a spec.

So, you can see here we've broken it down to a number of tasks. So, task one is we're building a transcription service. We're going to fetch captions via the YouTube transcript. We're going to um prompt for video analysis. And then we're going to have these different routes. We've got UI components we need to build and then we need to implement that into the sidebar. And it's generated a spec here as a markdown file.

And then all we need to do now to start working on this feature is to go and execute task one. Before we go and start executing the tasks, what we want to do is review the spec and make sure everything looks good. So here's the spec we generated. This is the video analysis feature. We've defined the why. So creators want to learn from successful videos. We defined the what. So we want to analyze any YouTube video by URL returning structured feedback on the introduction, key points, pacing, and improvements.

We've defined the constraints. So we're following existing patterns and use the YouTube transcript package for transcript fetching. This is where we can use specdriven development. So the the LLM has come up with this approach of using the YouTube transcript package. I want to use a different approach because I found some of these third party libraries unreliable. So what I want to do is I want to use a third party library and I want to use the super data API.

I found this pretty good. You get 100 free requests and it's pretty good about pulling out transcripts from YouTube videos. It's a really simple API and I found this generally more reliable than some of the other packages where I kept hitting rate limiting issues. So let's go ahead and make a change to the spec. Okay. So what we can do is we can say use the use the super data API. Super data API to fetch transcripts.

And so really we're making changes to the spec as we go along. You can tweak anything here. You might want to review any of these tasks. You can also, you know, regenerate this spec. You can iterate with AI. You can improve it. And you can just make sure that everything here is exactly what you want. The great thing about this approach is you can incrementally work through this feature. You can implement task one and then you can commit this task.

You can go away. You can do something else. You can load up a fresh clawed code session and then you can implement task two. So you're not stuck in a single session. You can come back to this later on. Okay. So now we're going to start up clawed code in a fresh session. And what we're going to do is we're going to ask it to implement the first task. So we're going to say uh read the spec and implement T1. So we're going to go through implement this step by step.

We're going to build the first task and then we're going to work through the rest of the spec a bit at a time. So as you can see the agent is working through the task. It said task one is complete. We've generated the transcript service and we've implemented everything wanted. And we can see here that the agent also verified the build which is one of the requirements we defined in the task that after you finish the task just make sure that everything passes.

We could also say to run some tests etc. So now that we've implemented the change what we can do is just quickly review the code that the agent has generated. And so this is an important step just to sort of make sure that everything looks good. I don't really like this extract video ID logic. This looks really suspect to me. We're using regular expressions. So, this is the kind of thing I might want to dig into. Um, the agent has implemented likely a working solution, but it might not be the optimal solution.

It may have made decisions that you maybe don't agree with as an engineer. So, you can you can take a look at this code. You could obviously ask the agent to improve this particular code. You can make changes yourself. You can just basically verify that everything works. Okay. So, before we move on to step two, we have the implementation. Like I said, there's probably some changes we might want to make to the code, but I'm also just going to verify this works.

So, can you test that this works by fetching this URL? So, let's see if Claude can actually execute the service and just make sure it actually does what we need it to do. All right, so you can see here that Claude was able to invoke that new service. It seems to be working. We extracted the video ID from this longer URL. We fetched the transcript with 18 segments and we converted it into plain text for AI processing.

It correctly identified the uh the title of the video. It's actually Claudebot, not Cloudbot, but it did seem to pull the transcript for analysis. So, this is good. So, one of the things you might be wondering is should I use a framework to implement Spectrum and development? There are a lot of tools that are popular. Openspec is one of them. This is a relatively lightweight framework. you install a CLI, it will generate some of these files for you.

Another popular tool is a GitHub spec kit. I found this pretty heavyweight. It generates tons of files. So to take you through the workflow with Kira, you start with a requirements phase. You define user stories and acceptance criteria. Then you go to a technical design. So again, this is mapping to the product requirements doc, technical design, and then the implementation tasks. You break this down into a number of discrete trackable tasks and then you do the execution phase where you iterate through all of the tasks.

You hand those over to an agent to implement. I found a lot of this stuff was overkill. I don't think you need to define user stories in markdown. I think those things should be done in the traditional way where you define a a product requirements doc which is somewhat separate to the agent execution. I felt like these were just adding a ton of complexity. I don't think product managers should necessarily be editing markdown files.

I don't think that makes it much sense. And uh similar with a design phase, I feel like there's a lot more detail that needs to go into an architectural document that doesn't really make sense to in my opinion in a in a markdown file. So if you're building anything non-trivial with AI agents, this workflow prevents agents from going and implementing things you don't want. It gives you more control over what you're building and it's going to lead to higher quality code.

Working in a more incremental way is definitely preferable to allowing an AI agent to go and implement 10,000 lines of code. Then you having to review that at the end is very difficult to unpick things once you get further down the line. So working in a more incremental manner makes just a ton of sense. And if you think about specri development, there's nothing new here. In software engineering teams, we've always worked in the same way.

For any non-trivial bit of work, a product manager or someone will come up with the requirements for what we need to build. Typically, this would be in a PRD. This document is for humans to agree on what to build. Engineers would typically break the work down into a technical design dock, which includes architecture breakdowns, details about how the features going to work. This would typically be reviewed by a number of engineers.

Then you break the work down into a number of discrete tasks and you hand those tasks off to individual developers. The only real difference here is we're handing the tasks off to agents. So if you find yourself working on anything non-trivial, writing a specification first, storing it on local disk as a persistent file, reviewing the file, editing it, making sure all of the tasks are correct, making sure the agent has enough information to implement correctly.

It's just a really good approach. and working in a more incremental manner will just raise the quality of the code when you're working with these AI coding agents.