Opus 4.5 changed everything

Well friends, the end is here. Not the end of the show, but the end of a fantastic partnership. I started the change law back in 2009 with Win Netherlands, and the show has been a beloved breakout ever since. In 2012, Win stepped away to lead GitHub's API, so I had to reinvent what the change law was.

That same year, Jarrod joined as a contributor and guest on the podcast. Eventually, it made sense for him to join as a full-time host with me. I grew into a business partnership where Jarrod joined as a part of the business and managing editor, and the rest is kind of history. Without going into too much detail, late last year, it made the most sense to part ways.

No drama, no issues. We remain friends. It's just time for change, and this week is a week of that change. Today, I am solo once again on this podcast, and of course, I'm reinventing what it is we do here.

For now, it'll be more the same, but in the near future, expect change, good change. In the meantime, if you join the community at changelaw.com slash community, it is for your join. Hang with us, ends your look chat. It's a ton of fun.

And to my dear friend, Jarrod, you will be missed. Thank you so much for the many laughs and all the good times and just everything. Friends forever. Alright, let's do the show.

Welcome back. This is the change log. I'm Alex DeKoviak, Editor-in-Chief here at ChangeAll.com, and today, I'm joined by Burke Holland from the GitHub Copot team, and we're talking about how Opus 4.5 changed everything. This is coming straight from a post, Burke shared in January, just after the AI Holiday High we were all on, from the Generous 2X usage bump with Clod, and the major step function Opus 4.5 offered.

Burke and I get into all the details, Opus 4.5 may have started the fire, but GPT 5.3 Codex is certainly living up to the hype. A massive thank you to our friends and our partners over at fly.io, that is the home of ChangeAll.com. Learn more at fly.io. Alright, let's do this.

Well friends, I'm here with my good friend, Chris Kelly, over at Augment Code. Chris, I'm a fan. I use Augie on the daily. It's one of my daily drivers.

I use Cloud Code. I use Augment. Augie. And I also use Antcode and others, but Augie, I keep going back to it.

And here's where I'm at. I feel like not enough of our audience knows about Augment Code, not enough of Augie the CLI. It's amazing. I love it.

What can you share? Yeah, we often say Augment is the best coding assistant you've never heard of, and that's both frustrating as someone that works there, and it's like very proud of the work we've done. But also inspiring, we want to go and punch above our weight, because we aren't anthropic and we aren't open AI. And so the quality of the product itself, with our context engine, once you do touch it, people are blown away by that.

And so that keeps me going every day. So not to be the lead here, but this is a paid spot. You are sponsoring this show to get this awareness. Now, at the same time, we're selective, and I love to use it as your tool.

But there is in the world, so a lot of developers look at the space and say, okay, well, how long can this work? How long is this sustainable in the case of Cursor or WinServe? Or you pick the name and you think this kind of tokens help me shape a lens for audience? I think it's a lot of awareness, right?

Like, Cursor got a lot of publicity early on for like, fast revenue growth, which well deserved, I think, you know, frankly, some of the media got this, gets the story wrong, and that like, if I gave you $1.50 for every dollar you sent me, I'd be the fastest growing startup in the Valley. And so when you're selling discounted tokens, yes, of course, you're going to go very fast, but all that money plus more goes to the model providers. So I think the real story is the story of anthropic and, you know, being an API provider. I think the market just moves so fast, and there's so many pieces of competition out there that it's just hard to get noticed.

So friends, I love Augment Code, and I love using Augie, and I highly recommend you use it. I love using Augie. I can hand Augie a well-defined specification, a well-defined PIP, as I call them in my world, an agent flow, and it executes falsely. So the cool thing about Augie that I love most really is that context engine, and I can hand it a task, and it can just churn away on my well-defined plan and just never bother me and accomplish the mission.

It is so cool leveraging the latest models, the context engine, and all the fun things behind the scenes in that awesome CLI. So yes, go try it out, augmentcode.com, right in the top there is a CLI icon, a terminal icon, click that, install it, and change your world. It's going to be awesome. Augmentcode.com.

Well, friends, we're here with Burke Holland, a dear friend, finally on the pod. We can't tell if you've been on the podcast before, but you know what, I'm glad you're here. You were from Microsoft, working on the GitHub team, running co-pilot, and, you know, I just don't know. This is a crazy word we're in.

It was part of this idea. It was actually Jared who has since retired from the podcast, but you shared a post, one of the most recent posts, but not, you're not an avid blogger, other than that, January 5th, Opus 4.5 is going to change everything. And that was January 5th, and obviously, you know, today is February 25th, a lot's changed in those two months. What do you think?

Yeah. So just a quick rejoin around the intro there, I work on GitHub co-pilot, but that's a massive product surface area. There's the cloud coding agent, there's the CLI, and there's visual studio code, and there's code review, and there's the SDK, and there's like, there's so many different teams working on it. I work in developer relations and I work across those different groups, but so, yeah, I'm not an avid blogger, am I?

I think I do more video now. It's a game out here. You're like, I haven't posted in a while, and here comes the major. That's how I want to frame.

You hadn't posted in about a year. You just post quite frequently other than that, but it was like a year gap, and then you're like, listen, Opus 4.5 is changing everything. Well, bam. Yeah, I saw that.

And then it, well, I mean, yeah, that's what happened. I saw that it went to, I was on the front of Hacker News, I was like, that's, whenever you blog, this happened to me maybe two or three times in my career, and something like that happens, not the post you think. Right? Yeah.

You just sort of throw something up. Which you've written differently given the attention. Of course. Yeah, of course.

Like if you go in, there's this big disclaimer at the top that was added after the fact, it's just like, as essentially everything I wish I had said, coming out of the gate. But yeah, let's talk about that. I mean, I, so there's this inflection point around around December, and everybody knows that this is true. Like, I mean, intrinsically people can just, they just- I'm not sure what he knows.

I'm not sure what he knows. Honestly, I think people who are the inner circles know this, but I'm having conversations are like, you use AI to do that? Why? No one wants to use AI, AI is, you know, this or that.

And you and I and a lot of people in this audience are in the know, and some of them are actually like, I don't know, you know, so maybe not everybody knows, not everybody knows something. Yeah. So let's, let's add some context there. So I've been working on GitHub co-pilot for years, for literally for years.

So I've been with it since GPT-4-1, which is kind of like the first model we got, and actually created a custom chat mode called Beast Mode, which we just get 4-1 to like do things, like just to get the model to do something instead of telling you it was going to do something and then not doing anything at all. And, and then we, the model started to get better, and then we got Sonnet 3-5, and Sonnet 3-5 was like, I mean, it's crazy, right? Yeah. It would actually do the thing and it would build you a nice purple gradient website every single time.

But the point is that like it could do a lot, but it was a really sloppy model, like it just would generate a lot of spaghetti code, Sonnet models are super eager, so they want to please you. And so they end up just doing just running kind of really nearly through your code base, but they will build things agentically. When we say agentically, we mean like the agent is like calling tools, it's running terminal commands, it's looking stuff up on the internet, it's acting like a developer. But the problem was, you know, even for us from where I was sitting, I could see that like it would get you to like a demo or a prototype, but then it's like, you couldn't, then it would get stuck on an error, and then you had a problem because it doesn't understand the error and neither do you because you didn't write that code.

And that's, that's, that's a problem that you can't really sit in both spaces. You can't like have an agent that is half right half the time. You need it to be like, all right, all the time, ideally. And so in December, when it was November or December, I forget, um, Enthrobic releases this model called Opus, which they've had Opus for a while, but they released Opus 4-5.

And it's because my job is to like test these things out and like, what does this tool do? How does this model work? I'm testing out Opus 4-5 over the Christmas break. And the first thing that I did was I tried to, I was building like native windows tools.

I tried to build a tool that would allow me to resize windows on windows to a specific width and height. Like I'm just testing models inside of the studio code with code pilot and it just one shots it. Just one shots it. And I was like, what the heck?

And not only did it one shot it, but it used native when UI libraries and it just did it right, right? And then I'm going through and I'm looking at the code and I'm like, this is really well structured, right? It's not perfect, but it looks really good. And the model's thinking is clear and I was like, that's nuts.

And so then I built another tool that was like, let's capture a part of the screen and then turn it into a gift. So like, you know, for sharing on Twitter and things like that. So I did that and it one shot at that. And I was like, well, shoot, I wonder if I could just like turn this into a complete screen video editing tool, like Snagit or something like that.

And I did. And of course, I don't know. A couple hours, right? I was able to build something that functioned approximately, approximately.

Not approximately. It worked. Exactly. That's just, that's just nuts.

Like when you think about, and all I'm doing is prompting the model, I don't, there's no special workflows happening here. It's just me telling the model what I want, brainstorming with it a little bit and it goes off and does the thing. So when we went into the Christmas break, I was like, I was pretty confident that I could actually build useful things. And my wife owns a small business.

People who are listening to this pod may have seen this before, but like if you're in the US, you may have driven by houses and there's like a big happy birthday sign in the front yard. It's like different letters. Have you seen this? You may have seen this.

Yeah. We're in Texas. So we do things like that. Okay.

So that's a franchise for our city in Tennessee. And so she has a lot of, it's called card my yard is the name of the company. Yeah. Yeah.

Cardmyyard.com. Go order a sign for your, do it up for your kids birthday or your dog's birthday doesn't matter. Okay. So she has, I build all these little apps for her and these little like there's her entire business runs on a Raspberry Pi.

No way. Yeah. And it does things like it. She gets orders in her email and it prints them out for her and stuff like that.

But one of the things that they wanted people to do that the franchise wants people to do is to like get on Facebook and post pictures of the signs that people have put out in yards along with a caption. And like running a page for a small business is inherently marketing slop as it were. But it's like it's a pain. People don't want to do it.

And she doesn't want to do it. Right. So she has no Facebook presence at all. And I was like, this is so automatable because of the nature of what it's doing.

It's posting a picture of a sign describing what's in the sign and then saying like happy birthday or something like that. So I built in an afternoon, I built an iOS app for her that does exactly that. It just allows her to upload as many pictures as she wants. And then it uses the Gemini, uses Gemini to look at the images and generate captions.

And then she can tweak the captions. And then all of that is saved. And so each time the model is called, it's like, oh, by the way, these are the last 10 captions that the user accepted. So they like it when it looks like this.

Right. And so it's it's mixing it up. And so I did that inside of an afternoon, what was crazy about that is I had tried to build that app like a year ago at night, I was spending like two or three hours a night using AI. And I got like 80% of the way there and could not get it over the finish line.

And I did it in an afternoon with opus. And then I built another app for her that just replaced an app that she used for for driving where it would route to the different stops that she has when she goes out to set up signs. She's just paying for that. Yeah.

And so I'm in place. We talked about the SAS killer run here before. This is an example of were you paying for an application before that driving one where she paid for something and then you replaced it with something that's exactly what we did. Yeah.

Yeah. And of course, because it's personal software, it's not distributed to the app store, it's just loaded straight to her phone because I can do that. So this was when I just had this moment of I'm just sitting here like, well, I can do everything right now. Like, what does this mean for the world, right?

Totally. And I thought that I was discovering something that nobody else knew about when I was listening to me. Only I know. But no, that's not at all.

Over December, like cloud code usage is just skyrocketed because everybody's trying to figure this out. You used to believe it was a good day as well. So that helped out a ton just to be like, if you've got the $20 plan at the low plan, then you got like a effectively $100 plan, you got max limits in a way. And if you had the $200 plan, the max, the true max plan, I believe like the sky was literally the limit.

You could not reach the upper pound of that. The context when it was still a deafening factor, of course, but usage limits overall across the model daily, weekly was quite high. Yeah. And they still are, right?

Like if you're paying, so for co-pilot, you can pay, so co-pilot, regardless of what people think about it, is the best deal on going by an enormous margin. So it's like 40 bucks a month and you get, it's premium request based, it's request based billing right now. So if you, if you do one request and that request does 6,000 things, that's just one request. You just pay for one request and you have 1,500 of those a month for cloud code, even at $1.00 a month, right, you can beat that thing up to the tune of like a billion tokens a month.

I mean, that's like, I don't know. They're cost on that. It's like $25,000. And your cost is $200.

It's just the subsidization that is happening right now is wild and it won't be around forever. It can't be. Or we're going to have to figure out how to make clean cheap energy in a hurry, either that or it's going to get really expensive to build with models at some point in the future. I don't know.

I have no idea. Yeah. This is a really interesting thing because we have a lot of angles. We have, you know, you and me and everyone else in the holiday break, literally being flabbergasted what we can do weeks prior to that hitting walls, hitting errors, hitting hurdles, not being able to really shift something of high quality with AI to then using opus for five and just literally opening our eyes to be like, that thing just did everything I asked and more and not being able to believe that when you got energy concerns, obviously isn't there.

But now the status quo is this level of AI assistant development that high performance teams are now like super high performance teams. The front, the lowest run of developer is now high performance. And so the status quo now is high performance, but it comes at a cost. You have to pay to be a developer, maybe or to build software.

You're going to want to use this kind of tooling because it's going to keep getting better and better. And I think the concern I have is this toll booth. I'm enjoying the benefits of what we're all using right now, but the concern I have is the toll booth that to be a developer, you have to pay enormous amounts of money or have maybe even several team members that are high performance team members to do the things that are now the benchmark of the marketplace and industry because the bar has just risen so high. What do you think about that?

I mean, I think that you still need to be a really high performing developer to be able to actually ship. And the reason for that is that you need to understand a lot about architecture and security and how an app actually gets to production. Coding something is easy. Code has never been the hard part.

That's always been the easy part, honestly, for being honest. It's getting that thing into production that's hard and that's still the case. And so this is why you see so many people talking about what they've I've coded and yet none of that stuff is in production because that's that's hard. So I think that you you still need to know a substantial amount about what you're doing.

What I don't have an answer to is like, so people who are junior, then the question is like, well, do you still need to learn how to code? I would think that you do because you can't these concepts build on each other, right? You can't. How do you understand what a REST API is?

If you don't understand how to call that REST API or how it's consumed underneath and the only way to really understand all these levels of abstraction, but but the second that comes out of my mouth, I'm like, yeah, but at some point you had to understand what the compiler was doing. And you don't anymore. I don't know what the compiler does. No, where do I care?

So as you said, like we don't know where all this is going, but I don't think that like, you couldn't bring a non-technical person in and just drop them inside of an AI to be like, do it unless it was like V zero or something for resellers really abstracting. But to actually ship a piece of software, no, you still need to be a dev. I think that for your high performing devs, I actually wonder about this because the thing is like in the past, you're really high performing devs are people that could do things like build very complex algorithms to solve speed problems, performance issues, et cetera, AI is really good at that, right? Like it knows all the algorithms, it can just destroy leak code.

So I'm unsure at this point, if that's really the case, other than that you need to know what to ask for a lot of times, like you may need to know that you need a binary search at this point, right? And to be able to direct the AI and say, why don't we do a binary search here and see if that's faster. You don't have to implement, you don't have to know what that is, how that's implemented, that's not your problem, but it is your problem to understand the conceptual nature there. Nothing concepts is a bigger thing to learn.

I'm not saying nay against learning the code as we've known it, like bootcamp level, kind of thing, or even a hackathon, you go and go hackathon because somebody says it's cool and then you get hooked on software development, I'm not saying nay to that. But I think if you can conceptually understand, even things like ETL pipelines, like even that, I'm using ETL pipelines, never before, before December, November last year, never knew what they were and knew how they were utilized, but my ETL pipeline is not exactly the definition of extract transform load, but it's a version of takes something from here, have a silver layer or heavy bronze layer, then transform into a silver layer and then have a final production to go, which is the medallion method, and I didn't even know that until AI taught me it. It's like, hey, what we're doing here? Oh, you learned by the brainstorm?

Yeah. Yeah. Well, because I knew about ETL as a concept, and then when they explained it to me, I'm like, you know, there's got to be a better way. And like, yeah, we should build a pipeline.

I'm like, okay, let's talk about that. And then I learned about medallion method, which is the bronze silver gold kind of process to an ETL pipeline. And as a practitioner, I began to learn more. Now I'm really well versed in it.

But before that, I'd only understood as a concept. And so I was using a concept to drive the direction, which then taught me through iteration. And now it's, you know, table stakes, it's too easy. Yeah.

That's a really good point. Somebody had made this point. I think I made this point before that, like, my fear was, we're all going to stop coding and just prompt AI's and we're all going to get dumber because no one is going to know what they're doing anymore. Because if you don't use it, you lose it kind of thing.

But that's not what's happening. What's actually happening is that, like you said, I'm learning all sorts of concepts that I never would have learned before. The world has expanded versus contract. Correct.

It's exactly right. It's really focused on, like, how do I compose this function? It's like, who cares? Do you know what a unique socket is and how cool that is and how that works in Go?

I didn't last week, but I do now. All right. That's a wild. And so I think that, like, as the people who are using AI are getting this knock-on-effect benefit that no one really realizes, where they're just learning at an accelerated rate.

Because they let go of whether or not this needs to be an if or a ternary or whether I should extract this out to a different file. Who cares? Yeah. Take us into, I suppose, more of your world.

And for one, you work for Microsoft. You're playing with other people's models, that's a thing. But tell me how your practices to build something has changed. What specifically has changed, step by step?

I mean, part of what I do is try to figure out the workflows that developers actually need to be successful. And there's sort of two different ways to do this. When you're teaching someone, you really want to start, like, Greenfield and Simple. But in the real world, it isn't like that.

It's more complex. So let's talk about the first one first, and we'll talk about the second. So for my Greenfield projects, I've experimented a lot with composing workflows that allow me to get from, OK, I have nothing. And I want to get to this end point, what are the things that I need to do along the way?

And how involved do I need to be? Because you have all these different, you have the Agentic tool itself, whether that's VS Code or the CoPilot CLI. And then you have, or others, right, cursor, Cloud Code, there's many others. And then you have like MCP servers, which is how your agent can talk to GitHub or how it can talk to Figma or, and then you have skills, which have serious overlap with MCP servers.

And then you have instruction files and you have custom agents, which is you basically just overriding the agent behavior into finding workflows. And it's all very confusing for people because they don't know what they're supposed to do with these things. And the honest truth is that neither do the people who are building them. Like, we don't know what the future looks like.

All we can do is look at what building blocks we could create, create them, and then see how people put them together and then iterate. That's the honest truth. And so that's what I try to do is figure out what does a workflow look like for me. So currently, my workflow is I'm now fully into the CoPilot CLI using it within VS Code, but I start off with a plan where I basically talk to the CLI and I tell it kind of what I want.

And I'm in plan mode. And these plan modes have gotten really, really good. And people are like, I don't need a plan, but it's not about the plan. That's not what you need.

What you need is the agent to think through with you all of the things that you just forgot to tell it that it needed to include in this initial prompt, because prompting is super hard within this idea of like prompt engineering, which is just prompt engineering is just giving the model the answer that you want. If you give it enough context, it will get the answer right and give you a right. If you were to spell out exactly what to do, it could literally build Minecraft on its own. But you can't do that.

That's not, you can't put the prompt to be too big, you can't do it. So in plan mode, you get this iteration where the agent's like, okay, cool, here's, here's, did you think about this? Here's what I recommend, but here's three other options or you can do a fourth one. And you go through about like four or five, six loops of this and I'm using opus four, six for this.

It's incredibly good. And then once you get to a point where you're like, yeah, that's pretty good. Of course, you're not going to one shot an operating system, but for a lot of us, we can get to sort of like an MVP in a one shot. So you can like one shot to MVP and then iterate from there.

So in your MVP, see, then in co-pilot, we have something called autopilot and autopilot is like a Ralph Loop. You're familiar with Ralph Loops? Yes. It's an orchestrator.

It's a rust project. Okay. Cool. Yeah.

So similar, similar concept where it will basically feed the agents output back to itself. It's just running the agent in a loop. And it does this until the agent has some amount of confidence. There's this idea of like, if you tell agents like, do this until it's done, they're like, okay, I did it.

It's done. You're like, it's not done. It isn't done. But if you tell them, do this until you have 95% confidence that it's done, then they will run and run and run and run and run building up the confidence level.

So a lot of the stuff's abstracted into the tool, but then I'm also like using a custom mode, a custom agent in that that I call anvil and what anvil does is it, it does two major things. The first thing that it does is it decides like, is this a, this is a easy task, a medium task or a hard task because you want different workflows from the AI, depending on what those are. Those are nice. Not the same thing.

If you ask the agent a simple question, that's an easy task. It should just answer that. But if you give it a hard task where you need it to build something, then it really needs to plan. It needs to use subagents.

It needs to farm work out and get a co-pilot. You can use different models. So like you can use Gemini three, five, three codecs and opus four, six all in the same run and you can use as many of them as you want to. Okay.

So now we're in an orchestration scenario. What four, six, if we're going to do a master refactor, it might farm out the design work to Gemini. It might farm out the refactoring to five, three, you might have 26 subagents running. You need to review that code.

And so this is my workflow now is to try to automate all of that. So I'm planning and then I'm autopilot with my custom agent, which out the other side needs to give me something that's verifiable. Like how do I know that you think that this worked? What is the evidence that you have that this worked?

And if it's in a browser, then you can use skills like the agent browser skill to have the thing check its own work. That's like the easiest scenario. Like for native apps, this gets really tough. Like if you're building a native Mac app, how the heck is the agent going to do that?

It can't. Right? It could write unit tests, but unit tests as we're learning are like woefully inadequate for telling you whether or not your code actually works. So that's a long way of answering your question is that like I don't have all the answers.

These are just the things that I think about on a daily basis. And then how do I further get myself out of the loop? So if I'm sitting here babysitting the thing constantly, how can I automate what I'm doing right now? Is it possible?

Could Adam, for instance, is it possible for me to give co-pilot CLI a large job and then walk away and have it message me on telegram and say, Hey, listen, like I did this much. I think this is what we should do next. Let me know what you think. And then I can respond and say, yes, do or don't do this.

And then it goes off and continues to do it. But can it operate more as like a team lead and less as I think the answer is yes, but you have to have the workflows and the checks and balances set up to actually make that happen. Well, friends today's podcast is brought to you by friends over at Squarespace. Now use Squarespace personally.

I love it. We use it on one of our small businesses. It's amazing. But here's one that hits closer to home for some of us.

Maybe you got friends, family, yourself, maybe even maybe some content. Maybe do consulting. Maybe you have the desire to do content and put a paywall in front of it and make some money from different ideas you want to share. Well, that's the gap that Squarespace fills.

It's the all in one platform for turning what you know into a real business, not just a website, but a way to offer services, book clients, send professional invoices and even get paid. It's all in one place. Two things worth knowing. First, offering services.

If you do consulting, workshops, coaching, freelance, speaking, dev work, Squarespace gives you scheduling on brand invoicing, online payments, email marketing, all of it built right in. No station together, five cents tools, just one dashboard, one way, no chasing payments in your inbox, you set up your offerings, clients book and pay, and you get back to actual work. Back to that AI project you're doing. Second, selling content.

Got a course idea, maybe. Video tutorials, probably. Second, make your selling content. Maybe you have a course idea, video tutorials, premium posts, Squarespace lets you gate content behind a paywall, add value, deliver on that value, get a one-time fee or subscription, your call, and even recurring revenue from what you already know.

The platform handles the access control for you, the payments for you, the presentation for you. And all you got to do is bring the expertise and ideas. Here's the next step. It's Squarespace.com slash changelog for a free trial and when you're ready to launch, use our offer code changelog to save 10% off your first purchase of a website or a domain.

Again, squarespace.com slash changelog, get that free trial, use it, enjoy it, and have fun. That's an interesting one because recently I did a version of that. I've been, I'm trying to, to decide if I should say the brand name. I similar to what you sask killed, something for your wife.

I sask killed them for myself because I use an invoicing tool that is largely glorified email. It's really just an email through a service that is a paying service and they largely pay us via wire or ACH, which is not at all through the conduit of the invoicing service. And the invoicing service is like 500, 800 bucks a year because the kind of clients and brands we work with, there's a lot of them. And I just don't feel like I need to delete brands to kind of get back to a lower tier service.

And it's kind of like paying extra for a database just for a couple of rows of brands that work with us a couple of times a year, but don't need to get deleted. All I have to say is it's kind of thought about, I was like, could I just build this in Rails? Because I love Rails. And I actually miss Ruby a lot.

Could I build this in Rails and could I one-shot this away? And I was like, here's all the things I want and I wrote my own list and handed that to, uh, to Cloud Web. I call it Cloud Web, my Cloud Code. Cloud Web, Opus 4.6, extended thinking, hey, this is my thinking, I'm going to go to bed.

Give me the best prompt to get a good V1 in place. I'm going to hand this, and in this case, I'm going to name drop to Augie because Augie is from our friends over to Ahmed Code. They're one of our sponsors, not one of our sponsors, and I love them. But Augie is also good at doing things on a loop with a great prompt and not have to do it in that dash, dash, dangerously way, if you know what I mean.

And so I can walk away happily knowing this, not going to arm anything or do any negative to my system. And somebody, I'm going to hand this, I'm literally telling Cloud this. I'm going to hand this, play by play, I'm taking what you give me, I'm going to copy and paste it directly into Augie, let it go, and I'm going to walk away, okay? The next morning I wake up, Ledger is born, okay?

The next morning I have an app that works, invoices, emails, PDFs, logs in, uh, all the things. I mean, there's a couple things I could probably do to make it more production ready, but like it's the V1. And so that's one thing. The other thing is, is on the telegram front, the challenge I wonder is that there are times when I'm in those kind of loops, like you are.

And as a great teammate, it may have a more complex problem that can solve in telegram. I may actually need to be there with it. And I guess in that case, it might be great to know that, but it might bug me on that walker, it might bug me on that restful step away. So I'm kind of torn on that bug me on telegram front, but it'd be kind of nice because maybe I'm at lunch with a colleague and I'm still working, I want to be in more context, but sometimes the thing that I have to that sort of, you know, human loop scenario with the agent to take that next step requires deeper thinking than I can do in a telegram message, or I may need to access more tools or whatever.

And so then I might just be like, oh, man, I got this agent anxiety again, I got to get back to my agent. Yeah. So that's my long-term response to you is like, I one shot is on the recently out of frustration, but it wasn't a true one shot. It was a thoughtful deep V1 one shot.

It wasn't like build me a rails out to replicate X. It was more like, here's what I use. Here's what I use about it. Here's what I really care about it.

Here's what I don't want you to do. There's things I do want you to do. And I'm literally going to bed. Come back the next morning, a ledger's born.

Yeah. I've thought about this a lot myself, where it's like what you really want is for the agent to just work all nights, like you said, and then build the thing that I want. But the problem is it doesn't know what you want, and you cannot get out everything because you don't know. And that's not how software is built anyway.

In no world are all the requirements and constraints to find out front. What happens instead is that you think you know and you discover it as you go. That's right. And it changes.

And you make trade-offs and the agent has to do the same thing. And so I don't know what the answer to this is other than whatever product you and I are replacing currently probably has a team of devs and an engineering lead and every Monday morning they get together in a stand-up and maybe they have a stand-up every day where they go over all of the work that's in the backlog, what the status is on each one of those things. And I almost wonder if that isn't the future, right? Because you can't just hand this work off and then just walk away and expect that the agent is just magically going to know exactly what you want.

It just doesn't. And so in order to really build anything of size, you don't have to have a daily stand-up with your agent where you go over all of the things that it's working on and review. And that's tedious. That's actually the part no one likes, right?

If you're listening to this pod, if you're in a daily stand-up, you're like, that's my least favorite part of the day. I'm with you. But there are reasons why we have processes and that's so that we can actually build things that work. I did something similar where I have a route flute.

I have copilot CLI running in a loop. And it's building a multiplayer game where you just larp as a baby bird and you're hatched from an egg and you fly around and I'm letting it's deciding everything. It decides what features it should add. It decides what's engaging.

It decides what bird behavior is. And I check on it every morning, but it's just running 24-7 and it does a pass and it adds something. And it comes back and looks to see what it could add and it just keeps going. Do I think that this will end in a playable viable piece of software?

I do not. Right? Probably not. I think it will be some monster of a thing that is interesting to look at, but it's not something you would ship.

And from what I can tell, this is currently where we're at. So I won't name names, but they're companies that have built, they sent agents off to build an entire browser or they sent agents off to build an entire compiler. And it's phenomenal that the agent can even do that. Instead of throwing stones when this happens, people should just step back and acknowledge the incredible feat that that is, that you have 1,000 agents that built a browser.

That's amazing. But to actually ship that browser, that is a whole other story. To ship that compiler, that is a whole other story. I don't have answers here other than acknowledge that these are the gaps that we're currently trying to think through.

I have some first up an answer. Okay. I think this is the lens we need to take. This is the lens I'm taking and it's helpful to me is that not all software is pretty equal.

Right? Just like your game is not the same kind of production level, shippable, sellable, SLA-able software. Those are not, they're still software, but they're not the same software. The benchmarks, requirements, the desires are totally different.

And so I think the lens we have to take is that there's a lot of naysayers like, oh, but I can't do this. Or I can't do that. Well, let me just tell you. You even called it out in your post, Sam Altman and others talking about where we're going and engineers being replaced.

We're like, now they ain't going to happen. And this year we're like, oh my gosh, this is going to happen soon. It might happen like three months from now. And so I think not all software is pretty equal.

I think versions of software will change in terms of your toy game and it's probably that for you versus somebody else's browser they want to figure out if they can actually ship. Getting at the production, getting at the ship is way different, but it's a different kind of beast. Even the most recent example with CloudFlare and open next. I saw that.

I saw that. Yeah, shout out to Steve. I used to work with Steve here at Microsoft. That guy that wrote that blog post.

That was a great post. Yeah. I mean, like even just like the way they wrote it, that was just phenomenal. I'm excited about that because the uptick of software getting created, the uptick of repost being created on GitHub, the uptick of all the things essentially apps on the app store, there's a lot.

And I think what's happening is people are learning how to build more software. They're learning like you are about Go and WebSockets or Go and how you can use a unique socket, for example, and what that does for the way you can connect to stuff like that. I mean, RPC versus a REST API, you've got all sorts of things that are happening where you're now learning different things. And that's the explosion happening is that your world, your world, and my world as well, and hopefully your audience as well, is now flattened in terms of, well, I can touch pretty much any piece of language or protocol or software as I'd like to because the AI is getting enough better, getting better enough to take me there and I'm learning with it.

And hopefully you're not just simply one shotting and praying, but you're more like iterating with it because I find the most, most fun I'm having building software is iterating with it, learning with it. And not like it's teaching me on teaching, but more like I'm just learning through the process of actually, actually being a developer again, touching things I would have not touched a year ago, building things I wouldn't have even dreamed about, and learning along the way and iterating along the way, using, iterating, using, iterating that feedback loop for me is really just a one person shop in that regard, because a lot of my software is just built for the most part. Yeah. Yeah.

Yeah. Somebody said the age of personal software is here and they're 100% right. And I think that's what most of us are doing right now. And so for people who are listening to this, they're like skeptical.

Like I can't stress enough. You can build anything. And if you're listening, thinking yourself, no, you can't. I've seen what AI can do.

The problem is that we've been trying, we've been on this sort of like hype grind for so long that we sold people like six months ago, like a year ago, we were like, you can build software with AI and you could, but it was tedious. And then a year later, we're telling people like, no, no, no, no, no, you can build software with AI and they're like, for me once, right, you're not going to fool me again. But I don't know what to tell you besides just go, y'all, just go do it. Like everyone had to build a AAA game, just go do it, just go do it, see if I'm wrong.

Test me and see if you can't get to something that's functional inside of an hour. Yeah, and you can. You really can. Now these, and really thanks to open source.

I mean, we were, and maybe you could speak to this given GitHub and co-pilot, like there was a lot of outrage on the initial release of GitHub co-pilot in that, oh, GitHub released this thing and it trained on all of our code, damn, you get up and all this stuff. But then open source is still here and still, I would say open source is driving more than ever and there's this massive swing back to just simply open source and free and open tooling. I mean, open next to the example, we just live with there. I mean, that's a good thing.

Yeah, because it's so cheap to create software now that it's nothing for you to just create something and then make it available to everyone. Yeah, I think until it's not worth sharing, you know, where you want to protect your mode or build a mode. 100%. But I do think that there are emerging problems that we don't really have answers for.

So like the VS code is probably one of the biggest open source projects. And I think last year, every year when we get to December, they do like a triage of the open issues. They spend December just trying to do spring cleaning, essentially get the repo down to a manageable number and do clean up the backlog. And it was like a year ago was 8,000 issues.

This year when we went to December, they were like, okay, how many issues are we working through? It was 15,000 issues. Like this is not, we have to figure out the process and the team is actually doing this, right? So the VS code team doesn't just build VS code.

They're also trying to figure out the AI processes for their own workflows that you use to create a tool which has to be reliable. It has to be fast. It has to work. It cannot go down.

We can't just ship a release where people are like the editor doesn't work anymore. We can't do that. Right. Exactly.

And so it's like you said, there's different kinds of software. But when we're talking about what we build for ourselves, that's one thing. But these teams, I was just in Redmond talking with one of the engineers Michael Lively on the team who's just, that's what he's doing is trying to figure out what are the workflows and putting them together so that the team can actually ship software with AI and have it remain at the quality bar that they are going to insist that it is. It just has to be a non-negotiable for people.

Yeah. We should talk about, like your point about our developer is going to be replaced. We should talk about that. Yeah.

Because I have thought to him. I know you do. Let's do it. So what I see right now is this sentiment just keeps getting thrown out, but it isn't actually happening.

AI is taking jobs but not in the sense that it's doing them. It's taking them in the sense that if you're in tech, that is money that's being diverted to GPUs. Everybody's investing in GPUs and not headcount. And if you're in the enterprise, I think a lot of places are using it as an excuse to downsize.

Right? It's just being, it's a convenient scapegoat. But it isn't taking the job in the sense that we don't need you because now AI does what you do. We're not there.

Models are extremely capable. And so I'm pessimistic that that's actually going to be the case. And I actually wonder if we don't end up with more developers. But then you start to ask, well, what is a developer exactly?

And we even have looked at this at Microsoft, like, am I an advocate? Am I a PM? Am I an engineer? Or am I all of those things?

Because I can now, if there's a feature that's missing a VS code, I can open an issue or I can open a PR with three different implementations of that issue done with the AI. And then we can either approve and say, yes, we're going to do this or we can just be like, no, we're not actually going to add this at this time. Close it out. What do I care?

It's a 15 minute job for me to create three different iterations. So it's like this age of abundance where I can contribute in a place where I before didn't have the time to. So I'm going to do this at town and learn the LLM API in the tool. I just don't.

I have other things to do. But now I can actually, I can I contribute. So am I an engineer? Am I an advocate?

Am I a PM? And I think we're going to see more of this where like, what is it exactly that the job title is, right? Is there such thing as just a developer anymore? If I'm in marketing and I need to build a, I need a quick page or something, can I just vibe that up?

If I'm in sales, right? And I need a system that makes it easier for me to answer questions the customers have. Can I just vibe that up? I mean, I don't know the answer here other than to say that it seems to me that like the AI is going to replace developers would be better stated is like AI, everybody is going to be a developer at some level, maybe going forward.

I don't know. Yeah. Yeah. Yeah.

I'm with you on that. I feel like calling everyone a developer is is a slight challenge. Calling everyone a builder is more plausible. I think I'm down to why is let me push back on that a bit though, because I think that for us as developers, one of the things that we've always had over other people is that we have the knowledge and the skills that they don't have and therefore we're worth a lot of money.

It's a lucrative field and we don't want to give that up. We don't want to give that up if we're just being honest and I don't blame people for not wanting to give that up. This will make you a bad person. But I think like we're still trying to gate keep some concepts, we don't want these things to be available to everyone because then if they do, then we're in an existential crisis.

I'm kind of for everyone being a builder, honestly, I'd actually toy with creating a new podcast focused on the idea of builder. I was going to call it future builder. It was a double entendre. It was like, you know, you're literally building the future and it's the future builders of tomorrow is innocence.

And it's the developers today, but tomorrow it's everyone's a builder. And I'm not sure the world's ready for that yet. I'm not sure I'm ready to produce that kind of show yet, but I like the idea that everyone is a builder because, and the reason I feel that way, I talked to a good friend of ours, the maintainer, he runs layer code and we talked recently the podcast. And one of the things we talked about was that he just in time built into himself and then his sales team and sales folks just in time built dashboards for themselves.

And so now you have production level software that's good for them because they're production of one or they're the user of one, you know, they're not the builder, but they're also the user of it. So it works. That's all that matters. I think that's super cool.

I want that world to exist because the more people who can build software for themselves is going to be, there's a there's a long tail upside for that. Whether or not, and the reason why T run it, they're a developer or not is for the reason she said before, which was they have, they may miss this deep knowledge of, say, the Unix socket with go, for example, or ETLs for me, or software about life cycle. They don't understand this larger, truly developer world kind of thing and maybe that will erode over time. And I'm fine with that too.

I'm mostly fine with that. I think that should be more software and more people building software. And if you want to call them software developers, I'm cool with that. I'm zero about gatekeeping.

In fact, I invite them all. It's like saying it's no different than being anti-nuby into our world. We've been so pro-nuby for years and years and decades, basically, or at least this last decade been pro-nuby and kind of anti-colon people, juniors, even like there's a lot of, it's a majority of I'm for everyone coming in. I'm like a, you know, like the one song, arms wide open gun, you know, I'm with arms wide open.

Yeah, man. Creed, all the way. I love Creed. I'm a, I'm a metal James kind of guy.

Yeah, with arms wide open. Come on in, come build. Have fun. More laughing.

More software is better for the world. And I think that as this, as this world evolves, as this thing evolves, we're going to need more people to build good software, but spoke software for personal use, bespoke software for just in time, business level functional personal software, and then truly platforms and production systems, like our friends at cloud, just did with, you know, open next, as an example of them using, you know, 1,000, 1,200 bucks to take next from what it was with, you know, for sale only essentially, and enabling Vte in all the process behind that, like that's a different kind of thing than you are in our building, but they got a team and some smarts and some true talent that not so much you don't have, but they've got some skills behind that. Yeah, you have to know a bit about what you're doing to pull that off. If I was going to take the other side of the argument on what I just said, it would be that the reason why we recoil at this is because to us, to those of me writing code for decades, it's a craft, right?

It is. Remember when you're post? I did. And when, when we had this big influx of people and even my kids when they were in high school, they're in college now, but it was this big push, you know, that everybody should learn to code.

And like literally everybody was in these coding classes because they wanted to make a ton of money. That's not a bad reason to want to be to want to learn to code, but they zero interest in the craft. Right? And that's why so many so few of them just made it out the end of the pipeline is because it's hard.

It's hard and it isn't fun and it's tedious and you really have to enjoy the craft. And it's like AI is never going to write code that's up to our standards because it's not a craftsman. We are craftsman. AI is not.

And so I do worry that we're building essentially like strip malls and track homes, right? Whereas like if you go back in history in my town here, they were built the homes that they built were just beautiful. And if you fast forward to today, they're nothing, right? Even my house is just a brick square.

And that's fine. I don't want to be ungrateful. But at the same time, can we even build? Can we build cathedrals?

Adam, any more? Does anyone even know how? I know that we do. And I worry that we're not.

We're going to forget how to build cathedrals. This should be the next blog post. Oh, yes. You know, I actually, I agree with that sentiment because I was recently in Pittsburgh, Pennsylvania.

And if you haven't been to Pittsburgh, it is a beautiful city. There's a lot of history there because it's one of the original 13 colonies. There's a lot of history there, whereas, you know, other places are sort of new. Everything's new.

As an example, nothing against California. It just doesn't have the same kind of history that Pennsylvania or other states might have. And I'm in downtown Pittsburgh, and there's this courthouse where they literally conduct court and it's just breathtaking. It's massive.

A massive, you know, craftsman level stone building that when my wife and I and our kids were walking through Pittsburgh, we're like, we got to go toward this thing. No one, it's just we got to go inside and see it. Yeah. No one cares.

Right? Everyone who lives there, it's like, yeah. Been there, done that. Seen it.

Nobody's going there. And when my family's like, what'd you do today? We're like, we toured the courthouse. Like the, the level of quality of work it took to make that.

And like it was just breathtaking. I'll put a phone on the show notes because it's just, it really is just breathtaking. And do you think, could we do that today? Right?

Exactly. Because that's the skills? I think we do. There's a small margin of folks who do still have those skills.

And because it's a, a, um, a rare skill, they can charge high dollars. And because they can charge high dollars because it's a rare skill, it happens less. I don't think we can't do it. I think it's become such a, uh, smaller minority that does that kind of craft.

And for those reasons, it's elevated in cost. And for those reasons, it's just done less. Yeah. You never see it because it doesn't make any sense to do it.

Only the hyper-rich can afford it. And so only the, in the unique places where the hyper-rich can afford it. Does it take place? For sure.

It happens. It just doesn't. But it used to be everywhere. That used to be, you know, how things got built.

And it took a long time. Right. And it took time. It took years, decades for things to get built.

And now we can throw these things up in a matter of months. Right. Like we can throw them a whole house in like less than three months. It used to take years.

But the thing is like the difference in the quality is unmistakable. Right. These, these, and so anyway, the analogy being is that are we moving into a place where like we look at the point where people are like post stuff they built on X and I'm like, I'm not impressed because I know what you did. I know that.

Because I can do it too. And how sad is that? I remember people used to post stuff on like Hacker News about like these CSS transformations that they pulled off. You're like, that's unbelievable.

How did you do that? And now it's nothing. I can write a whole JavaScript framework by the time this podcast is over. I could literally enter that prompt and we'd have one.

I'm not saying it would be good. But you see where I'm going with this. Right. It's like it's devaluing.

The thing. Let's talk about that though. Okay. So there's things that are incumbents in the marketplace.

I'm trying to think of like particular ones. Like NPM, for example, they say that NPM can't be unseated because of its ubiquity and because of its just tentacles out there. I think we're two or three. I mean, I guess this might actually hit closer at home because I just said NPM and I've forgotten where you work.

I forget that GitHub owns that too all the time. Okay. Good. I'm not throwing shots.

It's just my example. But I kind of am in a way because we talked about this recently was that securing GitHub or sorry. Securing NPM is table sakes. And I do believe that.

That's still the show we did recently with Nicholas. Oh, he's awesome. Yeah. Yeah.

A lot of great things to say. We talked about the details of that. People major security issues away, which you've already had a couple with NPM away from and the incumbent being threatened by a newcomer. If a newcomer came with the abilities of NPM and maybe a couple of people with the seat that have the clout to be behind X, whatever X may be, not literally X dot com, but whatever X as a variable may be, if we're a couple of security issues away from NPM, having the possibility of people saying, you know what, if there was an alternative, I'd choose it.

We're there. We're so close to that. There's a lot behind that. Don't get you wrong.

There's a lot behind that. But then the two things pop into my head. First of all, right now, people's appetite for risk seems to be ludicrous. It's just like, right?

They are open for risk. Well, they're not. Yes. They're just open to risk, right?