The End of Understanding: Why Good Abstractions Matter transcript

I'm gonna be introducing James Cowing who is the co-founder and CTO of Convex. He has spent years building infrastructure at serious scale notably at Dropbox where he actually led the project to migrate the company off of Amazon S3 onto their own I have to read this multi-exabyte geodistributed storage system. James did his PhD at MIT studying systems design, distributed systems in the lab of Barbara Liskoff, who has won computer science's highest honor, the touring award.

And that strong research foundation is something that James has been paying forward by acting as a mentor and adviser to a large number of Silicon Valley companies on how to design their infrastructure. So he's here today to kick us off with a talk about why we should still care about the principles, the first principles of responsible software design in a moment when software and systems can feel so complex, too complex maybe even to fully understand.

So please welcome James Cowling. >> Hello everyone. Hello and welcome to Abstract. Welcome to Convex's inaugural conference. Um, when we came up with the idea of a conference, we had to think what the conference is going to be about. And the typical thing normally is to make it a a shilling conference where we talk about how good the Convex product is. And uh, we decided that wasn't really on brand for us. We wanted to have this to be a conference about ideas and we wanted this to be a conf conference about ideas that we really genuinely care about are at the heart of our product and that ideas we want to kind of proliferate through the software industry uh during a period of really um large change right so so abstract is a conference about the art of software and this might seem kind of strange right now it might kind of seem kind of strange in an era where all you're being compelled to do is just to feed the machine you know, feed the token machine.

Turn your brain off and everything will be okay. And I'm not here to tell you to not feed the machine, right? This is not a anti-agentic development conference. This is actually a very pro-agentic development conference. But what we want to focus on is what is the differentiator when software becomes commoditized like when coding is you know a commodity what are any of us doing right and I think what we are doing is exhibiting taste.

Um, I know it's a bit of a cliche right now, but you know, senior engineers are in tremendous demand right now in the software industry. And this is because, you know, of the design and the accountability they bring to a to a to a project and this is what this conference is about. So the conference is called abstract. It's a pretty vague name by definition and it gives us a bit of flexibility in the future if we want to change what it's about.

Uh, but why is it called abstract? Specifically, this is a conference celebrating good abstractions, right? And an abstraction is the process of taking a complicated system, extracting detail away from it and ending up with a simple interface that allows you to represent that system. So abstractions are a tool for managing complexity. And abstractions really are a tool for democratizing power, for democratizing ability to to do good things, right?

Because they take something really complicated and hard and make it easy to use, right? And abstractions actually have an interesting history in computer science. So we're going to go back 50 years now. This is older than Postgress if you can even imagine right uh to something called the software crisis right don't panic it was 50 years ago u but 50 years ago there was a genuine belief not just the belief but an observation that the software industry had stagnated right there was all this tremendous progress and then there was some stagnation where we just couldn't innovate anymore briefly and so there's this guy Edyra Edyra was a famously smart person you might know him from the Dystra algorithm but did a lot of uh a lot of innovation in computer science and his quote was when we had a few weak computers programming became a mild problem and now we have gigantic computers programming has become an equally gigantic problem.

So the observation was it had nothing to do with the computers right the observation was that they had built systems that were too complex for a human being to manage right this was a time if you can imagine when there were go-to statements and and shared memory and global variables and the lack of good interfaces and collection classes all these kinds of things and very very smart people like Dystra were kind of panicking a little bit like wait a second we don't know how to manage these systems anymore because to manage a system back then required understanding ing all of the parts within it and how all those parts interacted with each other.

We just maxed out the you know the context window in our minds. Right now of course you might have noticed we achieved a couple of things in the software industry since 19 what was that something right um 1972 right um because of folks like Barbara and and many others by the way who really pioneered this idea of abstraction in computer science. So Barbara has this quote and again this is going to seem obvious now it was not obvious then right the main idea is to separate what an abstraction is from how it's implemented right so if you want to use a file system or say you want to use Amazon S3 don't have to think about all the millions of hard drives out there you just have to use a very simple interface with put and get right and so abstraction really allowed us to manage cognitive complexity Um because abstractions brought about structural simplicity.

So if you can imagine a system without good abstractions, it resembles kind of a tightly coupled mesh. You know, every part relates to every other part in the system. I'm sure you've all worked in code bases like this, right? If you've used decorators in Python or Ruby, you've probably dealt with a codebase like this, right? Where you can change parts of the codebase and they affect other parts around, right? Poorly abstracted, right?

But abstraction, good abstraction allows us to represent that instructions that look more like trees, right? Where you don't have to know how something's implemented. You just get to use it. Abstractions are about minimizing the occupancy of an idea in our little human context window. Right? Uh this is the one educational slide. I'll get through it really fast. Right? These are properties of abstractions. They should be simple.

That's obvious. They should be easy to use, right? Otherwise, what's the point? They should be sound. Using an abstraction should be the same as using the underlying implementation. There shouldn't be any weird gotchas or corner cases. This is terrible in databases. Most databases are not very sound, right? You can use the interface and oh no surprise, you shouldn't have done select rand materialized an entire table into memory, right?

Um they should be instructive. You shouldn't have to use documentation maybe is one way of saying it, right? It should be obvious from the shape of the abstraction how to use it. And the most important one and also the rarest one is they should be composable. Right? You should be able to change or use one abstraction over here without affecting all the others. This seems like an obvious thing, but most systems are not composable, right?

Uh whenever software products slow down, it's generally because of lack of composability. It's why building systems are always so hard to change, right? because you can't add a plan over here without affecting every other freaking part of the damn system where you have to change how your users are represented and all this kind of stuff, right? Good abstractions. Now, if you do this well, the abstraction becomes invisible because if you do this well, people just assume an abstraction is obvious.

Everyone's like, "Oh, isn't that the obvious way to build stuff?" That's the best feedback we ever get at Convex is, "Isn't this just the obvious way to to build a platform?" Great. Great. We succeeded in our mission. Um, and I know uh Jamie and I always laugh because oftentimes, you know, someone relatively new to the software industry be like, "Hey, you shouldn't use new stuff like Comvex. You should use like traditional technologies like the cloud, right?" And, you know, I I remember the cloud didn't exist.

It wasn't that long ago. It was pretty new, right? And I remember certainly when people didn't trust the cloud. And now, you know what? S3 and EC2, all this stuff, they're just reliable enough that no one thinks that they exist. That's just the bottom of the stack as far as many people are concerned. So good abstractions replace complexity in your mind. Now you might say, James, okay, great. Um, software crisis like abstractions help manage context.

This would have been a total banger in 1987, right? But why now? Why now? Why do we care about this now in an era where frankly it's never been easy to write software, right? Is there a software crisis right now? I can't with a straight face tell a 12-year-old who's building a amazing web app, right? that is not it's hard to write software right now because you obviously it's it's easy to generate code at least right very easy to generate code but um it's not easy to build high quality software still it's but um I think it's relevant right now because of a very rapidly changing relationship we have with the work that we do and the software that we write right so if your job have you seen office space it's very relevant today office is very relevant today if your job is to take the specs from the designer or the PM and give them to the agent, what are you doing?

Right? You're not contributing value to the economy, right? And when people are worried about like the role of software engineers, yeah, that that's something to worry about, right? If your jobs just taking someone's specs and handing them to an agent for implementation, that is not engineering, right? That's just being the middleman, right? And so, you are you going to spend all your money, all your effort just making marketing videos?

Like, you know, that's that's important, right? Um, but I don't think software engineers have to give up on that, right? I think we have to start approaching engineering through a new lens. Right? So the biggest rapid change happening right now is something I'm calling the comprehension gap. Right? Everyone has this problem. I have this problem. Right? If you use a coding agent, it will get away from you. Right? Now many people are not looking at the code at all.

Even if you are reading the code, you do not understand that code as well as if you wrote it yourself. You just don't. Right? And the genie is not going back in the bottle, right? It's not like next year is going to be the year of handwriting code again and we're all going to be, you know, doing stuff. It's not happening, right? Those we've we've moved on, right? So now we have to think about how to manage software in a world where people have declared bankruptcy on understanding.

So there's a crisis of understanding right now. How do we manage that? Does it matter? Well, yeah, it's a pretty awkward position to be in if you care about the security of your software or if you care about innovation and you want to be an innovator and you want to be relevant um but you don't understand what you've built. But I think there's no time to fix this to be honest because of a more insidious, more powerful, more understandable problem called the caring gap.

Right? People just don't care. Right? I I I'm not going to sit here and tell you how to review code really carefully. That matters by the way. Um and you know, especially a place like Convex, we have to we do have to review code very carefully. Um but a lot of people just do not care anymore. And that's okay, right? I remember this this really struck me when um claw uh clawhub uh launched this clawbot you know the um the personal agent launched and the skills repository was on convex and it was a very very popular application and all of a sudden we had this massive load spike on convex out of nowhere no warning right and it was you know driving the databases pretty hot and I remember speaking to Pete from openclaw and saying hey um if you just like rewrite your queries you know and don't subscribe to the date then then that won't update your client unnecessarily blah blah blah blah blah and Pete's response was really stunning at the time which was I don't care just so you know I haven't read that code and I haven't written that code right and I'm not really going to change it so if you want to change it you guys can put up a PR and I was struck with this feeling oh no this is my problem now like crap um and this is not this is not shade on Pete shade Pete was just giving us a window into future, right?

That was just a early window of a new way of development which is just status quo right now, right? So I've now stopped trying to kind of educate our customers on how to use convex. Instead trying to design for a lack of understanding and this I don't think is a problem. This is a good thing. This might be kind of counterintuitive. I think not understanding stuff is at the heart of progress. I don't It's very tempting to think that we're smarter than our ancestors cuz we take the self-driving car and they were eating sticks or something, right?

But we're not like biochemically we're not that much smarter than the people who came before. We just have abstractions all around us that we use every day, right? And when you turn the switch on and the light the light switch, you're probably not consciously thinking that like someone's shoveling more coal into a fire somewhere else and it's making the voltage go up a little bit and the wires, right? We benefit from abstractions every day, right?

This is great. This is how we can get more things done. So, it's a good thing I think that MPM exists and the cloud exists and I can call the OpenAI API without knowing, frankly, how inference works, right? But what's different now is that you may not understand your own stuff, right? That's a new thing, right? That's a pretty new thing we have to grapple with, right? Um, now we have people building stuff that they don't even understand themselves, right?

And if you have produced something that you don't understand how it works, you damn well better hope it's a good abstraction. You better hope you have a good functional understanding of how to use it. Right? And so you need to be designing around good abstractions, right? So one argument against this is do abstractions matter if software is free? Can't I just use Fable for this? Right? Um well firstly software is not currently free, right?

We're paying a lot of money for these tokens. And also, there's a lot of things that LLM agents aren't currently good at. Not particularly good at API design, certainly not very good at simplicity. They love adding extra layers instead of reducing things. Um, very bad at non-local reasoning. What happens if you change one part of the codebase and how it interacts, you know, non-transactionally with something else. But the agents are getting better.

Um, but I think this is beside the point because the point of software is not the code. It's about owning responsibility for the quality of something we're producing. Right? So I hear this one of the joys of being a startup founder is once a week someone who's 23 years old will tell me, James, you're an idiot. In the future, we're all just going to build our own databases with Fable. Why? Why does anything exist? You know, why does anything exist?

Right? And what I want to do is be snarky and say, look, here's some sand. Go build your own CPU. Right? But but but that's uh that's very reductive. It's reductive. And by the way, someone is almost definitely building their own CPU right now, right? Um so I think the point is that uh supply chains exist and there's value in abstractions consolidating effort, right? So when you go down the supply chain, your differentiated value goes down, right?

If you run an ice cream truck, you probably don't need to be formulating rubber compounds for the tires on the truck, right? You can pay Dunlop who has a, you know, company full of people with wisdom on how to build tires. Um, similarly, abstractions and platforms allow us to consolidate effort around people who have wisdom and have the tokens to spend on innovating in that space. Right? But interestingly when you go down the supply chain your differentiated value increases but the value of being able to outsource that responsibility to someone else goes up right and the magical thing about capitalism right it's why any of the companies exist any of the B2B companies exist certainly is you can take your money and you can give it to someone else and they make your problems go away it's great it's supply chains right and so I'm not here selling you a database or I told you we're not selling you anything today Ordinarily, I'm not trying to sell you a database.

I'm selling you the opportunity to not care about something, right? I'm selling you the opportunity to make that my problem and make scaling my problem. Right? That's why at Convex, we use S3 for our storage even though we've built storage systems before because I just want to pay someone else to make it their problem, right? It's why infra is so hot right now, right? Or the infra startup is so hot right now. Um, but if you're an an engineer or a company, typically your differentiated value prop is making problems go away for someone else and having confidence in in the work you've done, which requires understanding your abstractions.

Now, fortunately, there's no difficult trade-off here, right? Because agents work better with good abstractions, right? If you are able to give an agent something simple to use rather than something complicated to use, doesn't it seem kind of obvious that they will perform better, right? Um, less context needed means the agent can focus on the differentiated part of your codebase, which is, you know, your product that you sell, right?

Narrowing the state space, making it impossible to express invalid states because your abstraction doesn't allow for invalid states means the agent you have to worry about going down bad paths, right? And in particular, good abstractions allow for local reasoning, right? Good abstractions like transactions. this amazing thing that we the world invented and then we kind of gave up on briefly, right? It's coming back in transactions, right?

A transaction means you can you can make a change to adding a new team to the system at the same time as renaming a user and not have to worry about what happens when those things run at the same time. Right? So, if you want your code, your platforms to work well with agents, designing them around good, simple abstractions uh is a good idea. Now, what's the catch? The catch is it's really really freaking hard, right?

The catch is it's really hard to to design good abstractions because it involves combining two quite disjoint skill sets, right? You have to have the wisdom to know what is po what is possible and um this is why I think by the way it's still very important to know computer science fundamentals, right? is I think it's really really important to know how systems work to have some kind of understanding of of of how your code does its things because that gives you the wisdom to know what is possible what you can ask the agent to do right but it also requires marrying this with design thinking and design thinking is is actually really hard it's really hard especially if you're an infraineer who spends so much time in the implementation at the bottom of the stack right and doesn't have a lot of empathy for for maybe how the user uses it on top of that Right?

And there's an important principle of abstraction that form should follow function, not form follows implementation. So an abstraction should look like the shape of the problem that you're solving, not the shape of how it was built. And that might seem obvious, but almost every infra company gets this wrong, you know, and you see this in Kubernetes and all this kind of stuff, right? You know, hey, I built this really cool big system and good luck using it, right? uh but Google did something like this.

So yeah uh but it's it's really really really prevalent in databases storage systems they normally have quite terrible abstractions frankly if you have used database and uh like a my SQL postgress SQL database and you don't know what repeatable read is that's the default isolation level I don't know how to describe this as an abstraction I can only describe it as an implementation detail and if you don't know what that means and you don't know that you have to call select for update sometimes to lock moves in the table to prevent them getting read.

Um, your code is probably incorrect. Your data is probably corrupt, right? This is not a abstraction. This is an implementation detail that's leaking out of the database and you have to know, frankly, if you want to use a SQL database, well, you do have to know how databases are built currently. Um, S3 is an interesting one. So, you might not know this. When S3 first launched, its guarantees were eventual consistency.

So you would write data to S3 and then you'd read it again and it wouldn't be there and then you read it again later and it would be there. Cool. Right? And I can stand here for an hour and tell you why that's a good idea. Right? Because if you understand async replication and partition tolerance blah blah blah blah that's the right idea. In practice what it meant was if you wanted to use S3 properly back then and we we used it back then you had to have a second storage system just to record whether the data was written to S3 or not.

Right? So, it really didn't solve your problems uh easily, right? Um and by the way, now now S3 doesn't have that guarantee anymore. They just have a much more convenient API. You write it there and then it's there, right? Great. That maps to users requirements. Um so, it's a tricky time for engineers and you got to get used to it because my hot take is engineering is not getting easier. Um I don't think capitalism really affords for that.

I don't think it's going to be like all of a sudden these tools get great and our job gets easy because then someone else will come along and just do it better, right? And so we have to get used to how we can be productive and valuable at this time where we have to think like designers because we're designing interfaces largely, right? But we also have to think like a tech lead. Um, and if you've never been a tech lead before, it's a hard job.

But we're all tech leads now. We all have a tech lead of a bunch of agents, right? And the first thing you go through as a tech lead is you work on a project, you know every line in that project. You know what every engineer in that team is doing. You're micromanaging the hell out of that thing, right? And then you run out of brain capacity. You run out of the ability to know what's going on in that team, right? And the wrong response is to say, "Well, whatever.

I'll step back and get out of the way. Let people do that thing." The right response is to figure out how to build structures to understand whether you can trust parts of that system to know how to use parts of the system without having to know how it was built. This is a skill that's hard to develop as a is a skill we have to all develop really really fast right now. But the good thing is that it's very gratifying to be operating conceptually once you've figured that out.

So we have a lot of work ahead of us, right? um we have a lot of work ahead of navigating this very rapid change and my interest my passion is not just taking the systems and software we already have and making it more efficient right what I'm interested in is can we achieve great new things right and great new things really hedge on our ability to be able to make manageable software managing building blocks and that's what it's all about right so welcome to abstract you're going to hear a lot of talk about design today unsurprisingly a lot of talk about how to how to be a how to be a manager and managing complexity, managing this immense productivity.

And hopefully you come out of here maybe a maybe a better engineer. Thanks everybody.