Tech Truth: Agile Evolution & the Future of SW Engineering • Martin Fowler & Kent Beck • GOTO 2025 transcript

[applause] I'm Kent Dak and this is my dear friend Martin Fowler. We've known each other when did we meet? [cough] >> We met at uh the workshop on objectoriented design. >> Oh, this is the pre- snowbird snowbird. >> Yeah, pre- snowbird snowbird. I'm thinking it was just after I moved to America. So that would place it to be in 1994, possibly 95. >> And I instantly fell in love with Martin who introduced himself as being the only person in the room he he had never heard of, >> which was true. >> And I thought, okay, okay, we can we can get along.

I also remember you socially engineered that meeting. >> It was a bunch of big egos. I mean, Kent for starters. >> Whoa. [laughter] Oh, easy. Um, and uh, and so everybody was talking over each other, talking over each other. I noticed in the the morning of the first day, Martin just raised one finger and everybody's still talking over each other, talking over each other, talking and then his finger went down. Then a little bit later, the finger goes up again and talking and somebody said, "Um, Martin, do you have something to say?" And Martin said his thing and it was really wise and interesting and oh, okay.

And then somebody talked over him and then away we went. In the afternoon, the finger goes up and pretty quickly somebody says, "Oh, Martin, what what is it that you have to say?" By the next morning, other people's fingers were going up and the interruptions were stopping. And by the end of the con the that workshop, people were talking politely to each other, which I realize doesn't sound like a high bar if if you're say a parent to teach people how to talk politely, but apparently it is a really high bar among some people.

So, um, >> and I use that trick ever since. [laughter] >> Did you invent it for that workshop? >> I have no idea. I mean I I mean I mean for me it was so natural. I don't like jumping over into a conversation even though I have a loud booming voice which would allow me to do so. Um and so it was I just feel like I like to put the hand up. Um and I mean it's got very institutionalized in when we I was just last week we were at uh for works radar meet the meeting where we come up with Fort Works radar and that has a very firm but essentially that system we have a system of three colored cards red green and yellow because we take votes a lot and the votes are always done green it's on the radar red it's off the radar and yellow means I wish to speak and if you want to speak uh during the thing cuz there's 20 of us in the room, right?

There's quite a few people. Yellow card goes up. Somebody's making a note and got a queue going and the chair follows from the queue. Um I don't Rebecca or who used to chair it. She would m maintain the queue and be the chair, but that's too hard for the current crew. They can't manage that. So have one person manages >> the younger generation. >> The younger generation just not up to have it. Uh >> exactly. But I mean the thing is it and it was interesting. we we were doing this and two for it was new for a couple of people that just joined the group and they were saying wow this is really good that I can actually speak and and get heard and we're not jumping on top of each other and it's a bit weird because you want to speak and there's a delay because if you know there's two or three people in front of you in queue so if you want to answer somebody else it it gets a bit stilted but you do get to speak and it makes for a much more civilized meeting.

Um, but I mean if I'm not in that circumstance and I want to speak and there's a lot of cross chat, yep, it's hand up or finger up or whatever. Um, I've not never have had quite that effect that you describe. Um, to quite such a devastating degree, but it is a generally useful hack. If you're in a meeting and everybody's talking over each other, just put your hand up and you'll be ignored perhaps the first couple of times, but gradually people will cotton on.

And if you're really successful, people will realize, "Oh, this is a good idea." And it will spread. >> Yeah. The powerful moment is when you put your hand down. >> Yeah. >> And there there's a sense of loss like, "Oh, >> yes." Yeah. >> What was Martin going to say? And I'll never catch that ever again. >> You realize, okay, my point's irrelevant now. You put your hand back down again. And you'll notice in fact it infects online systems.

It took a while before Zoom got the handup capability, but it it came into Zoom. Um, and before it did that, what did we do on our Zoom calls? We were >> over Yeah. Yeah. Yeah. Yeah. [laughter] Yeah. On on camera, they finally figured out, oh, let me let me add that. >> And I actually added red green voting as well to Zoom, which was cuz when we first did the radar, had to do the radar remotely. Um, which was a shock because it was we the radar meeting was scheduled in Singapore for March 2020. >> Oopsie. >> Oops.

[laughter] Yeah. So, um, at that point we suddenly had to panic. We couldn't do a meeting and we'd never tried to do it remotely even though we we were pretty comfortable with remote. We had a lot of remote meetings because we're a global company. So, we we spent a lot of time doing remote meetings, but we never attempted this. And it's I mean, if you think 20 people arguing about this stuff, it's it's very difficult to do remotely.

Fortunately, we had Superwoman Ni, who I may have been, hopefully is in the audience, um, who was the TA at the time, and she figured out how to implement a complete voting system and hand raise and keep track of all of what we were doing all on Google Sheets because Google Sheets will update quite nicely um, across the the world in in real [snorts] time. and she just put this stuff together over the weekend, I guess, or something like that.

And we relied on it completely for the next few radars that we had to do remotely. And we still use the sheet now without the voting when we're working live. Um, so one of these definitely necessity requires you to do the thing. Um, but as I said, we kept with the the the hands up notion um even when even when tech wasn't helpful. >> So you can do it too. [snorts] >> First tip. first tip. So, you just finished a radar. >> Mhm. >> Uh I don't know what the secrecy rules are about radar. >> Absolutely.

Total. Yeah. I mean, I can mention that it's a radar. >> I'm prepared to say that there's a lot of AI in the blips. [laughter] >> That's more speculative. >> No, no, no, no. I can say with safety that there is a lot more radar, lot more AI in the blips. That's not unpredictable. I mean the main the main main secrecy point is I can't remember. [laughter] >> Yeah, there'sund and something blips because there's alwaysund and something blips and I can't remember them. >> Good, good, good, good.

So, we have a microphone somewhere, there's a microphone. Uh, if you'd like to ask a question, uh, please feel free to do whatever process she runs. But while but I'm going to ask you a question. So I'm I'm interested in your latest experiments with the genie. >> The genie >> where are where where is your what is your relationship with the genie these days? >> It's uh it's a a bit tense. [laughter] >> Um so in April I saw Jean Kim and Steve Yaggi give a demo of vibe coding, what they call vibe coding.

And I finally got over my reluctance. You know, I'd used autocomplete and thought, well, that's helps me not >> have to remember a bunch of stuff, but I wasn't it hadn't affected me emotionally. So, I saw this demo. I thought, okay, maybe I can try some projects. Let me let me see. Let me pick something ridiculously uh ambitious. Something I've wanted to try for years and years. I've tried manually a few times and never gotten much traction with which is uh >> to-do list.

No, that came later. [laughter] Uh no uh the graphics primitive in small talk is called bit blip and it's a way of moving a rectangular block of bits from one area to another with various combining rules and uh because it was so important it was highly optimized in small talk uh in including the the most sophisticated implementations uh treated the parameters ers to a bit blit operation as a programming language that it would then compile to machine code and execute the machine code.

So we called it selfm modifying code but it's really a compiler. It takes the the parameters to an API call, compiles them, unrolls the loops, strength reduction, all that cool stuff, and then runs it blazingly fast. So this is exactly the kind of mechanism that tickles my funny bone. So I thought, well, let me let me see if I can if I can do this and I got into it and it I implemented the reference implementation of BitBllet and then I implemented a little compiler just compiled to JavaScript just to try it and uh I kept running into these magic moments.

So the genie is is a slot machine. It's a variable reinforcement. You pull the handle and you get either something wonderful or I guess this, you know, in the slot machine, you only lose the dollar. You don't lose all the progress you made getting to the dollar. [laughter] So, I was just hooked though. You you you either get this wonderful outcome or this horrible No, that's not what I wanted at all. And um you can see on my on my GitHub activity page the day I started because it's just suddenly there's all this activity and I was waking up in the middle of the night going oh that's how I should have prompted it but it it it did magic stuff like I said write a testing framework that'll show me the difference between the reference implementation and the optimized implementation. and I came up with this really clever testing framework.

Uh, this is great except pretty soon it got to where it couldn't make any more changes to the code. So, I started over. You'll see on GitHub project two, project three, project 4, new project, [laughter] new project two. You notice I never number the first project cuz I always assume that one's going to go. [laughter] Project three, new project three, new project 4. So that's how I that's how I got into it. And what I love about it is that I can exercise the higher level programming skills. choosing what to implement, choosing what not to implement, figuring out a distant vision, figuring out way stations on the path to that, figuring out optimizing feedback loops.

Those are things that decisions that I enjoy making that now have more leverage because I don't have to worry about the details uh of the I mean, that's not true. I can pretend that I don't have to worry about the details of the dis smaller scale decisions that it makes. Now I do have to worry about the smaller scale details of the decisions it makes because it can just do some horrifically bad stuff. And if I'm not paying attention, there's always this temptation.

Prompt. Okay, boss. I got it. What do you mean the tests run? Well, 10 of the 12 tests pass. Wait a minute. That's not running. It has to be 100%. Okay. Okay. I'll fix it. Okay. The two tests that were failing are now running. [laughter] If you're a parent, you recognize this language. [laughter] If it's suddenly, if a child suddenly goes vague, what are you doing? Nothing. Are you definitely doing something at that point? >> Yeah, it always gets me the way it confidently says yes, all the tests pass and I run the tests myself and several of them fail and I think you know they talk about you know the AI being like a pair programmer.

It wouldn't it wouldn't last a day before I'm calling HR saying >> this person's lying to me and I catch them and they still lie to me. I mean it's just come on. [laughter] Does that answer your question? >> Yeah. Yeah. I mean, well, I mean, I'm fascinated by I mean, I haven't had the enough time to be able to do what you're doing with this workflow stuff, but I'm very interested. Fortunately, I've got lots of other people, but I'm kind of got my intellectual claws out to try and grab what they're learning and pull it back.

So, I'm I'm kind of doing it a different way, but it I'm definitely uh more of a spectator than a participant at the moment, but uh I'm enjoying watching you part u actually doing the thing. The [snorts] thing I love about it is the when you're a junior, you don't know what would be cool. And then you're a senior programmer and you say, "Oh, it would be really cool if I had a system that did this and a system that did that and system that." Then you're a principal programmer and you realize this would be too expensive to implement and that would be too expensive to implement and that would be too expensive.

And so your your universe expands, all these cool projects that you could do, and then it shrinks because you're realistic and you've been through this again. And and the longer it's been since that shrinkage, the the the smaller that your universe of what all projects could I maybe profitably tackle gets smaller and smaller. All of a sudden, all bets are off. Now most of the projects that you imagine would be too much work are still too much work because there's just a lot of essential complexity and you can't shortcut that.

But some of those projects just m like the to-do list the voice activated uh genie enabled to-do list on my phone that has a beautifully designed icon by the way. Uh, that's within range and until it erases all of your to-do list items because >> no. >> So, >> oh, the another thing I love about it is I don't care about programming languages anymore. >> Ah, it's the best. Let me implemented this in Rust. Okay. Chug, chug, chug, tug, tug.

Oh, I can't do this. I can't do that. Oh well, let me implement it in Python. So I'm implementing stuff in languages that I've never used before or used rarely. >> I even implemented Haskell sometimes now. >> Yeah. Oh, me and I'm doing formal verification too for the first time in 40 years, five, 45 years. Yeah. >> Which is it turns out is fun in the right context. with the right tools. >> So, >> okay, give somebody else a chance to >> question in the back. >> Oh, question. >> You run it.

You run it. You're going to decide who gets to answer the [laughter] questions. Okay. >> So, I have a question about uh it's a totally different topic about working in an agility in an agile mode. So, we both worked on the agile manifesto. It works perfect. But I want to know your stance about doing agile in a scaled manner. Like we know there's what they call scaled agile uh frameworks whatever. >> Uh a lot of companies they do call it that. >> Yeah.

I know I know a lot of companies >> they do use those words. >> They do use those words. >> I'm not sure those words mean what they think they mean. [laughter] >> Exactly. uh a lot of companies go uh to such solutions because they have the word agile in them but they come with a lot of big costs a lot of risks dependencies etc etc etc. So what do you do you think first that we can do agile as in what you consider uh working in an agile mode but at a bigger state scale if not what would be the other solution?

Well, I mean certainly I have very little time for things like scaled agile framework. Um we uh we got we've been annoyed about it several times with the footworks radar putting it in the hold ring. The holder ring officially means proceed with caution but can also mean don't touch this with a barge pole. Um and uh when it comes to safe it's definitely in the latter but with a very very long barge pole. Um, and you know, we we were so annoyed about it, we actually put it back on again the previous issue cuz we somebody had said something and we were extra ticked off for I can't remember what we got ticked off and so we put it back on again.

Um, I mean the again I I struggle with this because I it's something where I really ought to put more attention onto some of the work that we do as a thoughtworks with larger teams because we do tend to work in an agile way. I I I mean there's no official body that says do we do extreme programming or not thank goodness but I would say we're more influenced by extreme programming than we are anything else. So we do things like pair programming and testing and this kind of stuff.

I mean it would be reasonably recognizable I think as as an XP approach and we do do it on larger teams but figuring out the details of how to make it work in larger teams is it the thing is as as you get to larger organizations you end up having to do very specific things for that organization's culture and that organization's nature and and so it's very hard to come up with some kind of generalization and also So I've been kind of lazy and I haven't tried to go out and talk to the people who are doing the larger projects to say okay what have you been doing here partly because when I hear things it it's hard to pick out the patterns because of the fact that they are very particular to the organization's concern you know oh there's this particular body that meets once a month but we were able to hack it in this way so that they were able to work in a more frequent manner.

I think there's some broad things I would say are important. Um the first is the most important communication path is between development teams and their users/c customers. The most important thing is to work on that communication path and improve its flow because time and time I mean this is one of I found that the the opening part of the opening keynote very depressing because he spent he did this extremely polished keynote saying that and we've been saying this for 30 odd years [laughter] and it still needs to be said clearly um and I find that intensely disappointing um because you know how of how we got to We go spend another 30 years saying this again and again and again again.

Sort out the communication between developers and your users. That's >> um Martin I I don't think we have another 30 years. >> No, we we I think we could do 30. [laughter] With a bit of luck with a bit of luck, we can do 30. And and so that is that is the most focus on that. and what makes I mean I my touch pole for that is cafier um with her you know figure out what your how to make your us what your users need to succeed to do whatever they want to do to be badasses at what they do in Cathy's words and then look to help them be that.

So that's the most important communication path. The other thing and there are there are lots of really good reasons to slow that path down to make it longer to have more approvals and more filters and more aggregation and more and organizations are designed to slow that down because that's going to introduce change. >> Mhm. And organizations are successful having done what they did in the past. And so anything they can do to slow down the course of change is organizationally attractive.

Is it possible for large organizations to be more agile? Absolutely. Do people want to pay the price for that? For living in the forest as I would say no. >> So you had to you had your next >> Yeah. So the next thing is look at the flow of work in the organization. Figure out where the cues are. Where are bits where stuff is being stuck or you know is it oh we can only integrate once a month or something like that. Find those cues and have them.

I mean you can't go from integrating once a month to continuous integration in one shop unless you're really good and you've got Kent kind of whipping you on way like you did at Chrysler. Um yeah. >> [laughter] >> Um but you know you can say oh well if we can do it once a month let's figure out how to do it every two weeks and that will cause no end of problems. You'll say well we can't do it because because because because you have to just knock those down and over time you'll get it down to two weeks. then go out and celebrate and then say, "Okay, now get it down to one week and just continue that cycle until you really are continuous and look for other things that you can do about what is what else is what what other cues are in the system that are slowing down the flow of work." And so those are the two things I would most focus on communication with customers and users and finding the cues and having them. too many cross. >> Consciously or not, um at least the both of you are amongst those p people who introduced principles and ethics and and and ways to deal with problems like that and change perspective in the past and you still do.

So what I saw at this conference was um many bright people and many happy people and and [snorts] many tools um and obviously one one or two genies who will do a big leap but nothing of the same quality. So if you can't make it another 30 years, who will um how how do we come back to that kind of more substantial and and more basic quality that that helps us sustain that path because it's going to be more risky and even faster and even worse the next years. >> Well, we we know for sure that new leaders are going to emerge.

Uh we don't know for sure the quality of those leaders. We do know more about the lower bounds of those quality of those leaders than we used to know. Um but uh leaders will emerge. Um, am I annoyed that I still have to be saying this Yeah, absolutely. I I thought I said that 25 years ago. And I'm ready to move on. I mean, I don't talk a lot about process in the abstract now. I don't talk about agile. I don't talk about large organizations generally see speaking because my particular skill set doesn't lend itself well to having influence in large organizations.

The particularly the one about pissing people off on a regular basis. So that said, there are people out there who are figuring this out for a new generation. I mean, we're not going to be the ones who come up with the brilliant new idea because we're too steeped in in the old stuff. Imagine though, you're 15 years old right now and just learning to program and you've never not had an LLM. 10 years from now, that person is going to be so unbelievably good that I won't even recognize what they're doing as programming.

And I think that's going to be progress. Did that address your question? >> One of the nice things about what I've been doing over the last couple of decades is because of the the weird nature of my job at at Thoughtworks, um I've put a lot of effort into de helping the next generation as it were go up. And this is this is partly because of the guy who mentored me. I think you may have met him once or twice, Jim Odell. >> Um, who was this lovely guy who kind of took me under his wing when I was straight out of college.

Um, he, you know, showed me a lot about object-oriented development and about thinking conceptually. Um, he's the one who got me into conceptual modeling, the analysis patterns work, that kind of thing. Um, he also showed me what to be, how to be an independent consultant. um sponsored my H1 into United States. I and and it more than once I said I can never repay him for what he did for me. But his answer was well you you pay it by doing the same thing for other folks.

So I've constantly had that line and you know and I've focused a lot on trying to help other people communicate. So, and that's what what I do with the website. It may say martinfowler.com. You may have noticed most of the articles aren't written by me anymore. >> Um, that's because >> trick >> and that's exactly and that's I would have named it something else, you know, had I thought about it at the time and realized, oh, it's something I mean, but there we are.

And I'm uh I I enjoy developing that and uh we're seeing quite a few really good people um some of which are speaking at this conference and doing things at this conference and I think that will continue and they'll build on you know I I've spent a career basically cribbing off Kent's work. Um and we'll see a whole bunch of people not just cribbing off us but building on us and that's that's good. So I I feel optimistic in a way for the future when because my expectations were very low for the agile movement.

[laughter] I mean when when we came up with a manifesto I thought no one's going to read this, [laughter] no one's going to care. Um and >> and it's even worse. [laughter] Well, it well it was Alistister I think has this glorious quote that I'm going to slightly mang that says that whenever you come up with a brilliant idea either it'll get ignored or it'll get perverted into something completely opposite to what you originally intended and you don't get to choose which of the two it is. >> That's true. and uh you know as it turned out you know the latter but on but having said that my hope was that we could act that looking at it from a footworks perspective when we were operating in 2000 and we wanted to work the way we wanted to work which was you know XP style it was really really difficult to get have clients to prepare to at all permit us to do so um the battles are now slightly weird because it's now people saying oh to work in an agile way.

You shouldn't do as much testing as you're doing or something. But we actually have made prog. It is easier than it used to be. You know, I'm no longer banned from using the word agile if I give a talk. Well, I don't give talks anymore, but if I were to give a talk, so I think we're making progress. It's just horribly slower than would like it to be. >> I have a question about the radar. Two times a year, we get a a good radar about new trends and something like that.

Um have you thought about um bringing out a radar of uh trends um with were not um um yeah addressed in the past and which have to be addressed in the companies something like um the most misunderstood trends with are not um addressed in the companies. No, no, we haven't thought about that. Might be interesting to do. Mind you, the effort required to build the radar is considerable. I mean, it's a reason a lot most most companies are so stoked to attempt to do something like that because it's a lot of work and a lot of expense.

Um, but uh yeah, it's an interesting thought. >> Yeah. Right. So about the forest and the desert versus the uh the cathedral and bazaar, are there any intersections? Would you care to elaborate more on that? The Smiths? >> Glad to elaborate on it. I haven't thought about it at all. [laughter] >> Oh, you asked Chad GPT. Well, that's the definitive answer then. >> Yeah. >> So, you should tell me what I should say. So, Cathedral in the Bazaar is uh is making this distinction between I I would say between of when you make design decisions.

Do you make design decisions as early as possible or as late as possible? In the in the bizaar, we make design decisions as late as possible, but just before we need their benefits. Also, that's tidy first. So, um, forest in the desert. In the desert, you don't dare do that because you need predictability. Doesn't matter. The predictability is impossible, but you need a date when this is going to be finished even though you don't know what this is.

And uh, in the forest, so in the desert, you're just always going to be biased towards the cathedral. In the forest, you can accept the fact that it's a bizaar and lots of people are making decisions. Uh the addition in the forest is that you have a shared distant goal. The bizaar to me is just I mean in terms of the metaphor is just people buying and selling on their individual hook without some larger shared purpose.

So that's my off-the cuff response. How How'd I do? >> 80%. So, I'm bet I'm still better than an LLM. [laughter] >> How much of that of the LLM stuff did I make up or what I said just now? [laughter] >> Oh, 100%. I mean, that's what the Buddhist said, right? This is all an illusion. Yeah. Oh, we're getting deep. Sorry. Yeah. >> Hello. Hi. So, what are your thoughts on the latest Dora research which was published few weeks ago if you uh already had the opportunity to read?

So, I was waking up at 8:00 to have breakfast to start a meeting at 9:00 that finished at 6:00 and then we went to dinner and we didn't get back till 10:00 and then James Lewis took us to the bar and we didn't manage to get bed until 12:00 for every day last week. I did not have time to read the door. [laughter] Uh I heard some summaries of it and uh nothing about it surprised me terribly. Um I am as I think I've expressed very skeptical about gathering these kind of metrics and then because they [snorts] the the temptation to turn them into goals seems to be overwhelming.

So, um I mean I can read them and every once in a while there's a Huh. Well, that's seems strange. Was there something about it that caught your eye? >> Well, Jez and Nicole are no longer involved with Dora. >> Yeah, they started and I really liked the work they did and the Accelerate book. big fan of that. Um I mean most metric stuff is badly warped for all the kinds of reasons Kent indicates some more. Um but they I really do think made an attempt to do some good work with the with the original Dora stuff and with the accelerate book, but I have not been tracking much so much since.

I I do intend to read the Dora report, but uh I don't actually get back to my desk at home for another couple of weeks and and then I've got a ridiculous amount of email in my inbox. It's going to take me a while to get through. So, it's going to be a while before I uh get through it. On the other hand, you might ask some of my colleagues because uh some of them may have taken a look at it and see what they have to say.

That's that's what I usually do is I wait for them to tell me what what the good story is and then I'll figure out a way to repeat it and make it sound like it was mine. [laughter] >> Hello. Thank you. Well, so recently I've started to work a lot with the co-pilot. So it does a lot of of magic stuff and it's very great. But it sort of made me think that maybe I no longer need to understand the code. I mean, why would I want to waste time doing that when it just produces stuff?

And that's made me think that why would we even want to waste the step of generating code in the coding languages that we know? Why not just do it directly to machine code? And do you see that happening, you know? Nope. That's a quick answer. Thank you. [laughter] >> Next question, please. No, no. I so so the the uh I don't need to understand uh I definitely push back on if if the genie made all good decisions if the genie's decisions were consistently as good as its best decisions then yeah I don't need to understand the code but there's some horrifically short-sighted decisions in there and I have to sort between the two so that that's one response uh if it's pure vibe coding.

So I talk about augmented development or augmented coding because I'm making the I'm responsible. I'm making the decisions. I'm driving the bus and I'm being assisted like an exoskeleton by the genie. Um vibe coding is when you really just don't care. You know, when when somebody who with no programming who couldn't make the distinctions with no programming skills particularly, but they want an application, they type in a prompt and well, looks like an application to me.

If it works, it's great. And if it doesn't, you're completely stuck. So, um, and I had another point. >> I'll go then. >> Okay. >> So, [clears throat] yeah. So, I mean, I I've been listening to Kent. Um, I've been listening to folks like Simon Willis, who I really strongly recommend um reading on this stuff. Um, I've been listening to my colleagues, in particular, Bigita Berkeler, who um has been writing a whole bunch of emo memos, and also some other of my colleagues, Matteo Vicari, and and Mashi.

So, a lot of pe even though I haven't been doing much experimentation myself, other people have been. And the constant thing I'm hearing back is you can't trust what the genie produces. I mean, sometimes it will get it right, but maybe one in 10, one in 20, one in 50. I It's not that often really. You've always you got to watch it constantly um and often in some kind of interesting way. So, a little story that Bea um told me for instance, she was validating.

So, someone had said, "Oh yeah, we got this spec driven code thing working. It's doing really great stuff." So, Bita gets the spec, runs the yellow lm on it. It generates some code. Then she gets the spec, runs the yellow lemon on it, generates code in a different directory, diffs the two. Had she had to do a little bit of work to make it line up because of naming, but basically diff the two. Same same spec, same model, just different runs.

Have a look at the difference. What does the difference? Oh, I'm doing this API to submit orders. It's very simple. Um both of gen both generations put some validation on the order number not mentioned at all in the spec. The spec did not say anything about any validation rules for the valid for the order but it just kind of introduced it and they were completely incompatible. That's the kind of error that the genie produces all the time.

That's probably a mild case, right? Um, but that's the thing and it's often difficult to see, difficult to spot, which is why you have to take very small steps because you've got to carefully read each line of code as if it was written by not just a junior programmer, not just an overconfident, overenthusiastic junior programmer. Not just an overconfident, overenthusiastic, really obsequious junior programmer, but also an overconfident, overenthusiastic, obsequious and actually kind of vindictively malicious [laughter] junior programmer who's determined to screw your life up in some really subtle way. >> They're having a bad day.

[laughter] >> You're telling me. >> So, the second answer is try it always. We should all be shifting to explore mode. And in explore mode, the answer to everything is try it. So try it. And if it goes great, it goes great. A thought experiment I've run is what if we invented a program? Programming languages pay a lot of attention to human ergonomics. At least they should. [snorts] And people try and with varying varying degrees of success. and the genie is more or less capable of manipulating that language with all the human optimized ergonomics.

What if we invented a programming language that was built around genie ergonomics that the genie could manipulate easily even though people couldn't read it? Would we then switch? I don't know what such a programming language would look like. I don't know why people can't read it. You know, it's done with hieroglyphics or APL. Turns out that APL is the perfect target language for generated code. Are we gonna use it? >> So, so I like throwing out useful tips from time to time.

And this one, um, really whenever you're using an LLM, not just for programming, but for anything else, ask it the same question more than once, two or three times, preferably slightly varied a question. when you ask it the question, you know, change a few words and compare the results. And that's often a very useful technique kind of can be handy for, you know, write me a few paragraphs about X and just do so because the variation between them will probably be quite interesting just like Bit's example of generate the same code twice and diff the comparisons.

That's I think a generally useful thing to do. Um, I did run into uh one case where somebody was talking about they they actually asked the LLM to tell them how many lines of code in this commit there were that had changed and they asked more than once and got different answers. And I'm going [laughter] a command to just tell you this. It's deterministic. Come on. But yeah, I mean especially if it involves numbers, ask more than once to figure out whether it can give you a consistent answer.

Now in computer science, we've been studying how to build reliable systems out of unreliable components since vonoyman since the 50s. And the answer is always double-checking. Y >> some form of some form of inhibition. The system's going to want to get into invalid states and we have to we have to prevent that. So we do extra work to avoid getting into those invalid states. That principle is still going to hold. I'm in the middle of of reviewing kind of the history of the papers talking about Byzantine voting and and all back to vonoman.

How do we build reliable systems out of unreliable components? This isn't a new problem. This is a problem we've been thinking about for a long time. We wish that we didn't have this problem to solve. We wish that you could just ask the genie, get an answer, and it's just not that way. Hi uh young aspiring developer. Uh I think you both are brilliant and you compliment each other very well. Um how do you guys get to work together and how do you find like new colleagues to work with?

Well, it's easy for me to find new colleagues to work with because I work at a company that's influenced by, you know, a very similar mindset and ethos. So, I I keep just running into them all the time. Um, and uh it's a lot of fun. That's why I why I'm there, right? Because it's saves a bit of trouble of finding like-minded people because there's a whole bunch of them around. Um, and so that's that's how I find similar people.

You have a more struggle because you're more on your own. Yeah. So, I keep a u like my little black book. If if someone young says something surprising, I make a note of it and I keep in contact with those people and sometimes they turn into collaborators or people who work on workshops with me or things like that. So, I'm constantly on the lookout for new talent. um in particular people who are going to say things that are surprising or unusual.

Unusual and not stupid. I just wanted to clarify that. So it it is the job of us seniors to find and encourage the next generation of talent. For sure. >> Yeah. And I the way I talk about it is that uh I'm old gray and past it and I'm surviving by drinking the intellectual blood of my younger colleagues. [laughter] >> That started out so hopeful, >> but their intellectual blood is very tasty. [laughter] >> Um I have a question also about the future. um generally about software engineering.

Do you think are you optimistic about the future of software engineering? Because to me it seems like we're doing the same mistakes again now with VIP coding that already happened in the 80s in the 70s and so on. Same security issues. Are you optimistic that we're going to learn our lesson and make progress? >> I'm optimistic that we're not going to learn our lesson. So I'll I'll still be employed saying the same stupid 20 years from now. >> Uh I I mean I think we have made some progress over the years.

It's I said it's been much slower than we would like I would like it to be. Um but I think we can we have we have learned a lot particularly forest dwellers. I think the forest dwellers have made some really nice progress and we can achieve quite a lot of things. It's just unfortunately the forest is still relatively small. A and [clears throat] I I'm definitely optimistic about the future. I think we're going to get better at this.

This is a this is a phase change for sure. Um we have new capabilities. Things are cheap now that weren't cheap before. That's the nature of all technological innovations is that uh that the you know I was thinking about this on the way walking over here. Uh, instructions used to be very expensive. I tried to build a model in my head of how much an instruction would cost in the 70s and in 1970 1980 and I didn't make much progress and I I didn't ask the genie.

So, oops. Um, but when instructions suddenly got cheaper because you had a microprocessor, things that had would have been too expensive to implement suddenly became not too expensive and you can't predict what those are because you're you're stuck in that old mindset. And if you want to get unstuck, the only or the most efficient way to get unstuck is to try a little bit of everything. >> Yeah. Thinking slightly meta on your question, it's kind of ironic that you're asking us to predict the future since I think the one of the core principles of our thinking about software development is that it's impossible to predict the future, >> right?

And and the whole thing about extreme programming is it is it does not attempt to predict the future and says the predict the future is unpredictable. So you've got to therefore assume a posture that can cope and survive and even thrive on that unpredictability. And I think that's why it appealed to me so much because I'm very definitely of the I can't predict the future stuff. So the question is how do you how do you live how do you live in that kind of circumstance?

And part of it is you've got to find ways to be flexible and and uh so that you can take advantage of opportunities when they they come up. >> Maybe this is just playing with words, but it seems like you're talking to a couple of guys who predicted the past. [laughter] Yeah. Hello. Um, so in my opinion, you created a really big masterpiece uh 25 years ago. For me, it's really like a compass, the whole manifesto. Uh, did you thought about to make a class reunion all of you guys? >> Well, we can't sadly because we're not all still here. >> True.

Um I don't I don't think there's any particular plan for us to get back together. I mean we're all a bunch of all old past it guys now. Um we need to get a a different group of people who will come up with stuff for the future and we can be part of it in the sense of sort of encouraging and supporting and um you know adding a few words of old outdated wisdom that they might take seriously. But it it is it is for future generations to hold the torch and carry on.

Um and in fact we kind of said that right at the beginning. Um after the we came up with a manifesto and it caught on way more than we expected it to. Um there was a meeting at the about 6 months later at the oopsler conference in Florida where the question was you know what are we going to do as the creators of the manifesto and we quite deliberately said let it go. We wrote the manifesto. Hopefully, it was helpful, but it's for others to continue on.

Now, that continue on wasn't always in the direction we would have liked, but there we are. Um, but that's always the way. You know, you want new blood, new ideas. It's coming up to 25 years, so there's probably going to be attempts to try and do something to say 25 years since the manifesto. But and I've been asked about it as well and my attitude is you know we've got to look to the future and it's got to be people of the future doctor don't don't look to me cuz I'm too old.

You might not be able to look to Ken because he's still programming and is youthful of youthful brilliance but you know me less so. >> If if this is youthful brilliance I feel sorry for the youth. [laughter] Uh, no. No, nobody's offering offering me offering to pay me enough for me to just talk about the dusty past. I'm much more interested in what's going to come next. >> Our time's almost up, but we had one final question. >> Thank you very much.

Um, I just wanted to say thank you very much. Um, inspiring talk. I have a question. I'm a fresh junior and I'm looking for guidance on how to path my way into the future and become well good in everything [laughter] or how to get better in everything and what can you give me for tips on my way um especially yeah what there'll be pressure on you once you finish a feature to start implementing the next feature instantaneously faster faster faster, faster, faster.

Don't do that. The gaps between features is what's going to make you good and what you do with those gaps. And the thing I love about the Genie is this is endlessly patient tutor if I think to ask the questions. So you finish a feature, there's pressure to start the next one and you say, "What's another way I could have implemented that? Explain this language feature. What is amperand amperand single quote single quote do?

Oh well, is that useful right now? I don't know. Maybe, maybe not. You know, help me help me refactor this. What are some tests that I'm missing? Run coverage testing on this and and uh propose some tests. Find duplication in what I've just done and propose how a refactoring might help it. You have the opportunity to learn between the features. And if you apply that, the features will come more quickly. Even though you're not pressing to have that happen, that habit of learning is what separates people who really master programming versus people who just do programming.

And I would go back to one of the points I made earlier on which is hey talking to the computer is its own skill and is fun and is interesting and important but talking to the people who will benefit from the computer who are you know the users customers whatever that's important because that way you know what to build and more you understand of the domain you're working in and the people who are working in that domain and what they need to do in order to succeed um the more you get that sense of that broader context that um I think is vital to being a really good programmer.

Um and it's you know talking to the computer is only part of the problem and it's actually in many ways the easiest. Um but uh understanding a domain and being able to think to the point where you can actually bounce ideas off somebody as to what how to make progress in terms of what features to build can be really exciting. Um but it's the the key thing about computer programming is the communication between people and I don't think a genie actually alters that.

It changes the way it works but it's still that inter people are still the most important part of this. >> Thank you very much. [applause] [bell]