Skip to content
Building Pi, and what makes self-modifying software so fascinating episode artwork

Apr 29, 2026 1 hr 33 min

Building Pi, and what makes self-modifying software so fascinating

from The Pragmatic Engineer Hosted by Gergely Orosz

Brought to You By: • Statsig — ⁠ The unified platform for flags, analytics, experiments, and more. • Sonar – The makers of SonarQube, the industry standard for automated code review • WorkOS – Everything you need to make your app enterprise ready. — Mario Zechner is the creator of Pi, a minimalist, self-modifying AI coding agent, that is the foundation upon which OpenClaw (created by Peter Steinberger) is built. Meanwhile, Armin Ronacher is the creator of Flask, and a longtime user of Pi. The pair are also friends. I sat down with Mario and Armin for the latest episode of the Pragmatic Engineer Podcast for an interesting conversation about AI and their reservations about it – even though both are heavily invested in building AI-powered tools. Mario explains why he built Pi, and gives his take on why it has become so popular. Armin walks us through how he uses AI tools, including building a game with Pi, and why he always puts human judgment firmly at the heart of his approach. We cover the risks of over-automation, the limits of agentic workflows, and why strong engineers with informed judgment still matter. We also get into the challenges of working with code written by non-engineers, and whether open source can withstand a tidal wave of agent-generated code. — Timestamps (00:00) Intro (07:30) How Mario, Armin, and Peter Steinberger met(15:15) How 30 dev teams use AI agents: learnings (21:50) The importance of judgment (24:26) Challenges when non-engineers write code (28:30) Downsides of over-automation (32:18) Pi (48:09) OpenClaw + Pi (50:54) “Clankers” (57:32) Open source and AI (1:00:22) Complexity as the enemy (1:02:50) Building an AI-native startup (1:11:52) “Slow the F down” (1:16:40) MCPs vs. CLI (1:25:03) Predictions and staying up to date — The Pragmatic Engineer deepdives relevant for this episode: • The impact of AI on software engineers in 2026: key trends • Cycles of disruption in the tech industry • The AI engineering stack • The creator of OpenClaw: "I ship code that I don't read" • What is inference engineering? Deepdive — Production and marketing by ⁠⁠⁠⁠⁠⁠⁠⁠https://penname.co/⁠⁠⁠⁠⁠⁠⁠⁠. For inquiries about sponsoring the podcast, email [email protected]. Get full access to The Pragmatic Engineer at newsletter.pragmaticengineer.com/subscribe

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

Building Pi, and what makes self-modifying software so fascinating

0:00 1:33:07
of matches

Transcript Auto-generated, may contain errors

What if I told you that one of the most influential AI coding agents of 2026 was built by single developer in Austria who got frustrated with existing AI coding agents? This is Pi, minimalist self-motifiable coding agent which has quietly become the engine behind the wildly popular personal AI assistant, OpenClaw. Mario's echner is the creator of Pi and joining him today's Armin Ronecher, the creator of Flask, and now an early adopter and contributor to Pi. In today's episode we cover, the backstory of Pi and why self-motifying software is much easier to do with AI agents. Both Armin learned interviewing theory plus engineering teams about how AI agents are changing how they work and why software quality feels like it's trending down, the case against MCP and why CLIs are becoming so popular and many more. If you want to hear from two

very grounded voices in the industry honestly talk about what's working and what isn't and why we need to slow down as an industry, this episode is for you. This episode was presented by Statsig, the unified platform for flags, analytics, experiments and more. This episode was brought to you by WorkOS. Engineers love to build. Today's episode will be a great example of this. We'll get into why and

how Pi was built from the ground up. But when you're shipping a product some problems are better to solve with trusted infrastructure, built for scale. Enterprise features like SAML, Directory Sync and Auditlogs are some of those. WorkOS gives you APIs to add them in days not in months. Shift faster without reinventing the wheel. Now let's get into the episode.

Mario and Armin, it's so good to have you here on the podcast. Thanks for having us. Thank you. As a kickoff, Mario, how did you get into tech and eventually into building AI stuff? Oh, well that's a long story. How much time do we have?

So I'm at the mid 90s actually. I got my first PC at 96 and the trigger for that was that I love computer games. We were kind of working for so we couldn't afford any of the Game Boy and any of the guys who have been here and stuff. But I had an uncle who had an Amiga 500 and I would go to his place every second day and just take games there. Eventually my parents told me if you work, you can save up and buy yourself a computer. And the reality my dad would do what

he called shots of it. Well, you're not necessarily paying us access on it. Yeah, so he would do his normal job and after he's known, he would go fix cars and work at construction sites. Yeah, it's a disarker on the neurofic, I know I've run about. Yeah, after two or three years or so, they just said it's time and took me to a computer shop in the nearby big city and bought me a 486 and that's how it started basically. And in 486. Yeah, until 486, DX, 4E, megahertz, with a robot and that's where I started.

And I've always been into games a lot, which also led to graphics programming. And through Sherlock, I got a job while I was studying at university at the Applied Science Organization, who was doing LPs, machine learning, applied machine learning, basically taking research results and trying to stuff them into industry applications. And that's where I learned the ropes of machine learning. That was all before deep learning became a thing. And I actually quit that kind of domain in 2010, 11-ish because I joined the start of in San Francisco. And then later came back and

joined another start with two friends in Sweden where we did an un-head of some compiler for our job by code to iOS that got sold. And since then I have a little bit more time. And I've always kept up with machine learning stuff because obviously it's super interesting. And yeah, and GPT happens. That's the story. Yeah, and here we are. And then Arman, where are your roots?

My roots are definitely not working poor, but because my parents run an architectural office where they kind of adopted computers for kept drawing. My first computer was like old computers that they recycled. So my first computer, even though I'm younger, was in three. Sorry for you. And so basically none of the computers that I ever had were capable of playing computer games properly. Because one, they used Windows NT and which at the time didn't do anything. So you had

to sort of like build your way through it. And the only way you could actually get to the run was because before it didn't know yet how to get the Windows 95 or Windows 3.11. That was like before it booted into either one of those you could put into DOS, like the old DOS games at a time when you could already get better stuff. But because it was this kind of thing, I started going around with Quick Basic a lot with tool Pascal, I bought a bunch of books on that. And that was my roots of learning how these things work. And I wasn't ever really good at this,

but I found it really interesting. This idea of like, no, for sure. We call it a teafestob in German. No, I swear to you, when I started downloading of this, I just really sucked. But over time, we could do it better. And then in 2002 or 3,

I used to use Delphi a lot, which was like a visual version of tool Pascal. And it wasn't 2003, someone also showed me because I've got this idea, I want to use Linux and then Delphi didn't work on Linux and then I found Python. And through that, I started doing some Python programming and there was a Ubuntu just came up with four. And that was a venture-packed vehicle, but they created all these local communities. So that was like Ubuntu association. So I

together with a bunch of friends, we started the German Ubuntu Foundation, another foundation association. And we ran this online community called Ubuntu users for five years. I mean, because Ubuntu was popular, the community grew and the games came, so that's how it got into web development. And then for building this, I just want to build a templating engine, a web library, all of this. And then eventually I bundled that together and made this flash framework, which could be popular. Even nowadays still is, I think that Clankers like to

spit out. But I left that and then in 2013, 14, I worked on computing for a couple of years in London, but then afterwards I went back to home source and I worked on sentry for 10 years, and then left in April last year to try something. So both of you are virtually from Austria. In fact, you right now live in Austria as well, right? You were doing games, you were working on sentry, you also did games before. And then the third person who was on the room, but was on

this podcast just before as Peter Schaimberger also from Austria, where did the two of you meet, where did the three of you meet? Because I've recently seen a bunch of photos, especially before Open Claw and Pi started you hanging out the three of you experimenting playing with AI. I think two of us met on the internet, right? It depends because I definitely met you once when I was at the university. But you didn't recognize me the time when I was at the university.

I was already famous. But yeah, we started by structuring met on the internet. But eventually we met up in Vienna. We were screaming a lot at each other, but on the internet, but in a very cute kind of way, in a very non-confrontational kind of way. And even though we might not think like in all areas of our lives, it was a culture exchange, I would say. So that was nice.

And Peter, I, like six degrees of Peter Schaimberger, basically, I was working at an office in my town. And the company that gave me free office space in exchange for being like an enter to the CEO had some kind of business dealings with Peter's company, PSB. Yes, really, yes. And eventually came to the office in Gratz. And I think that's where we met the first time.

And then also the same year we met at the conference in Istanbul, which was turned out for an entire night. And that's basically where it all started. Nice. And then how did both of you go from being skeptical about AI when these tools came out? And again, both of you have at that point, and by 2022, you've been doing a decade plus of building complex software in different domains.

What was your first reaction to it? And then eventually how did you kind of come across the side of like, well, this thing is actually really interesting. So for me, it was 2020. I think co-pilot, GitHub co-pilot came out before TPT. Yes, in 2021. And through my previous startups,

I was working with Net Treatment and Miguel Likazza from Xamarin, which Xamarin. Yeah, they acquired the company. I talked about it. I knew Net, Freeman from our early startup stuff, and eventually moved to GitHub. And then it wasn't my DM's in 2022, I think, and ask if I want to have access to the co-pilot, the TPTIP, all the complete thing.

And I was like, I don't really care. I don't think this is going anywhere. And it's like, no, man, it's the future. I got to try it. It's the future. So I tried it. It was absolutely horrible.

But yeah, after when GPT came out, and especially when you started providing API access, I did a lot of projects, just figuring out what works and what doesn't work necessarily in the coding space. Eventually, once they had tool calling, that's when they became very interesting or function calling as a way I called it back then. But it took until 2000 and I would say 24 and 24, a clover, so for that to actually be useful. And that's where the coding agents also became kind of interesting. And then 2025, the Cloud Code team came out with Cloud Code, and that introduced the GENTI search. So basically just give the agent a way to plow through your file system and read all

your files, and then make the whole difference, especially now. Like all the things that came before, the cursor with indexing and any AST based stuff, and all of that, it just went away. And then the CEO of Chroma is probably mad at me for saying this, but that was the difference. It wasn't like a dense and sparse search thing that the agent could go through. It was just give it access to your files. That was it for me. That's where it clicked for me.

I think my path was kind of similar, because I think Copa came out quite a bit earlier. But I know that there was a program at GitHub that gave me early access to Copa at the time. I think it was like this maintainers group or something, whereas there was an... I got the feeling for Copa that this will actually be really interesting, but not in any way in which it is now, because I felt like, oh, I'm in open source for such a long time and not doing like training in open source data. It's like something at the very least, this will be controversial. I didn't think about

being productive. I felt like, oh, this is going to be a controversial thing. I was trying to probe it really. What is this flask in there? No, I was trying to probe it really adversarial.

So one of the things that I probed on is like I probed on a really detailed trivial code. And I remember at one point, I got it to spit out the Chroma's Inverse Hybrid function, which is very specific names. It was very easy to get to recall. But I was finally like, you can sort of have it in a certain way, then it would then continue putting like license text on top of it. It was completely wrong. So it came from an open source, Chibi L drop of

of normal, which I think. And so it was like, it would have been Chibi L code if it would have done it. But it actually attributed like MIT license for a random dude. And it was like, oh, like Mr. Copa that's the wrong thing. And that's really the time that really, really popular. And I sort of

people started like sharing with me. Because I was at a time not really exposed to how much actual AI progress was being made in those labs. I didn't come from this AI space or ML space. So I learned about university and AI winter and nothing happened. But through this tweet, some other things, I was like, I recognize that there was something there. Like there's actually

CEOs in certain companies are convinced this will get off. And that's how it started like paying attention to us essentially was trying all kinds of stuff with the API. Like, can you do like bug fixing things that I got really interested in it. But it didn't at all feel like the world is going to change until close quote. And you also change your stance on whole, oh my god, this is spitting out the source code. It memorized. So because like my like my

shtick for many years now has been that I really, I'm like a I want people to share stuff. Like I think like human progress comes from like building on top of each other. And I'm a huge supporter of the fact that in the US, you basically take knowledge from one company to another company to then no compete. Like I like this pirate kind of approach to sharing. Yeah, it's by knowledge. Yeah. And so like I was like my optimal version is like operates

on exist in a way like very, very limited kind of version of this. I was like, I really didn't care that it's been so GPL code doesn't actually be like, oh, maybe this will just completely destroy copyright. And like for me, it was like, oh, this is like, if that's the outcome, I'm fine with this. But it was it was an interesting kind of thing in the beginning that it sort of like it sort of creates this license balance. I'm like, I want to see like what chaos will emerge from it.

And so far, I think that most of what has emerged from it is like a strong belief now that like the system in place for copyright has some assumptions assumptions in the US about how it's supposed to work. And we're all kind of like ignoring that right now, because we want to create a mess first and then re-regulated probably because like at least in theory, a lot of the things that we're producing right now are probably by historic readings of the copyright interpretation, actually not copyrightable. Yeah, that's an interesting one. But speaking of jumping to today, so an interesting thing that you did recently, we talked about it just before is as part of your new startup is built things on top of agents. And you talked about 30 different engineering teams saying, hey, how are you using agents inside of your company inside of your team? What did you learn from large

companies to startups? I think that a bunch of learnings, I'm surprised is that whenever people had location, there was more time spent on trying these tools. And just to be clear, like you talk with folks at the likes of like meta startups, yeah, like a bunch of different people, right? So a bunch of different people from like different European dinosaurs, like, are you pointing at me? Well, I mean, like the European dinosaur would be some like Siemens, or I also talked to you two companies, which are sort of in a critical space. And what I mean, like,

adoption happens when people have a case is that like when you're CEO or the tech lead comes and says like, you've got to use curse on our you've got to use clock code now is actually you don't get it in a way, because you need to actually spend some time like this. It's like a two to three week kind of thing until it really clicks on you. And so I always felt like with the people that I knew, I had a lot of free time like I left the company in April until October, like I can dive into this. And I was like, this is like, how does nobody get this? It was a catnip for all of this. It was crazy

catnip. I didn't sleep much all of this. But what happened within the company seemingly is that when there was like Thanksgiving, there was in for Europeans, a lot of it was over summer, and then at Christmas, a lot of people sort of, and they also get free credits during those times. And so like more and more people go, you mean the companies often give you generous credits? Yeah. And so like more and

more people went into this and especially after Christmas, I would guess like in more than half the companies I talked to after Christmas, it really exploded. And it exploded in all the ways and we'd expect it where like all of a sudden the quality drops. And it doesn't necessarily drop because like people want to make worse codes, but because it actually takes some effort to stay within this. And we have seen this in the startup ecosystem already in the summer last year. Like if you if you pay attention to like the YC startups, a lot of them, some of them have the stuff on the top of some period of time and get them and look at it at the time because like plan and defiles checked in and like everything attribute to cloud. So like that vibe coding kind of thing was like prototype

or whatever, like that built it out. It was already out there to see. But gradually a small version of this is like been code basis with a little bit of vibe slop on top. And an interesting sort of part of this was like how engineering teams and companies are now responding to that. With all kinds of like different findings, but a lot of it has been challenged to review PRs. They're getting larger

and larger and becoming like more like more practical things. So like an engineer specifically are having a hard time keeping up with the longer PRs that they're more frequent. Yeah. And they're also a lot of the code in those PRs is how an engineer wouldn't do it. Because as an engineer sort of get a really bad feeling committing certain code because you're think of your future self. And the agent really

does not care. This is I will retell the story over and over. But I worked for an Xbox one game at the time right around the Xbox one launch. So that was like a fixed date has a really summit date. So I worked on the Halo Master's collection. And there was a game where I had like a metronic component

and you had to like start this thing and whatever. And it was like it was an all hands on that kind of situation where people had to go in and unslop the human made slop that was the match maker. And it was like it was it was a system with like way too many states we call it an emergency machine because it was like 16 pools and one magic thing and like a theory about 26 valid states. But in reality it was a dramatic explosion of possible states. And that's how a chanting code feels where it really should only be like a very clearly defined system but in all reality they're like oh okay config doesn't load let's catch it down and load the default config. So instead of actually

failing it now recovers. But now your code is way more complex than it should be because instead of failing properly it is now recovering and entering these many more failure states. And that makes it much harder to work with this code because you can also not really ask the agent to refactify the case. It's like oh yeah this could be possible so we need to maintain this variant. I think it's kind of universal what you described about your human made complex system because there are moments of brilliance in agents where they spit out perfectly fine simple code. Exactly the amount

on type of code you didn't need for the specific thing. And the USD steering engineer looking at that like wow this can just sit back and not care because it's obviously doing the thing like two minutes later you have another agent running this way now and it speeds out the worst horrible garbage it goes. But you might not notice because now you have fallen into automation bias and think your agent is doing the job well. Do you think this might be a bit of a human bias because because you know like typically like onboarding a new engineer. When you join a new grad you review their code and if it's terrible code you will review the next one thoroughly until they get to the point that oh it writes the code that I do and then it typically takes you on six months or a year or something like that. But then you know I can trust this person. Yes but you don't

have anything like that with agents. Like agents don't learn you can put as much stuff in the agent's empty you'll build a memory system but that's not the same type of learning than a human does. Obviously humans are available as well no matter but they have some capability of learning. They're retaining that learning graphs and they also feel pain. I think that's one of the defining things about humans. It's kind of ties back to what you said eventually if the pain gets to

be a human or incentivized to fix the cause of your pain. And in the code base the cause is usually terrible interfaces, terrible complexity that you want to get rid of because you cannot longer maintain that system. Isn't this why just calling on to the you know like senior engineers are always in demand because the CEO sees a senior engineer as like they just get it done. But in reality as a senior engineer or most senior engineers who are effective they've had battle scars they've been burned they felt the pain they saw what happened when they left tech def spiral. So they now make all these decisions that they know they will help avoid and of course do this progress goes faster.

I personally think in your mileage may vary but a good engineer's an engineer that's know a lot and I don't need this a lot because that keeps complexity done. If you're using agents the exact opposite happens you say yes I want this and that. Listen I want this because I don't have to type it myself. I don't have to think about it. I just give the little machine a prompt and it will spit out something that kind of looks like the thing I want and good enough.

And that's where all the problems start. And one thing that I also think is like good engineering is all about knowing the trade-offs that you have to make. And there's sometimes the right solution is actually if you were to sort of like sit at university and learn about it. You kind of learn that you shouldn't be doing this in a way. I think Cal Henderson had this one way he said like you do the dumbest solution first until it doesn't work anymore. Because

the actual problem is there's so much stuff that you need to do that if you actually do the right solution the correct solution all of this it is creating a kind of complexity that kills you at scale. And the engineer learns that but also like if you if you don't have that battle scar it's actually very hard for you to argue correctly because it is this learning process that gives you the authority to then convince other engineers in engineering work that you should be doing it this way. That is part of it that you learn that but the other thing is also that the agents give you now world knowledge access. And one of the other things that I learned through interviewing engineering teams now is that the senior person says no knowing something and then 48 hours later the junior comes by and says like I talked to the agent and I already had this inkling but now I have all the evidence of why we should be doing it this way. Because like previously you really didn't have that ready-made access to someone who can tell you're senior off. Yeah and this

this creates other stresses now that we're previously like not every team has that because they get a really good engineering. So people going to the doctor with a chat chippy tea print out and saying this is what the machine said you better do that. Is it fair to say that we are based on what you're seeing and talking we might face a thing where it's very hard for experienced engineers to speak harder just for them to say no in spite of the product manager or a junior engineer saying this much worse because the product management comes in and sends pull requests and all the mergers shoots them. Yeah that's another thing cause like non-engineer participating in engineering processes is a thing now. Ask him how does it work? How does it work?

How does it work? How does it work? Well it's hard because if because on one hand it's well intended right if someone who's like what is your experience is this your company talking with other people. So first of all we have a little bit of this and we're small and so like my co-founder sometimes sometimes like a board has on the website. I talk to people that have it at scale. I like the marketing team all of a sudden does stuff on the website and the sales team

creates ever more elaborate like sales demos that sort of land up on the GitHub board. And partially one of the most funny ones was like what a sales demo built a feature that didn't exist but nobody noticed right. So this is all like this is new right because I previously met up that happened. And I think it's empowering like if you're empowering it's like there's a good thing to it. If you're entire or give everybody in your or can participate in the creation of software in some form right. Previously people couldn't do that like you had a designer who could

figure something out in Figma but they might not be able to kind of put it into a clickable dummy demo whatever we might have a PM who wants to try out a feature without kind of wasting time of an engineer. Now you can do that. The problem is that people are now so focused on everybody can do everything now that they forget that you still need a process to kind of guardrail of all that. And the integration part is the hard thing it's like the Peter gives this idea of like the prompt request but I'm actually really warming up to this idea like once you've demonstrated it I no longer need to quote and just to have the prompt request was him saying that he doesn't like to get pull requests and said he would rather see the prompt because he will run the prompt or he will tweak it and it will generate in the style that for me it's less about like I want to see the prompt as like what is it supposed to be doing and now that we understand because like actually in many ways I think like the interesting part is like often you don't really fully know what you want to do in the first place and so like the act of creating clarifies what you really want to do. And so like that part is highly valuable often the approach and the code that comes out of it is not what an engineer with sufficient seniority would have done. So it's not like I want your prompt

so that I can re-clank my clanker so that it does it slightly better but more like not I don't know what I wanted to build probably faster for me to start. Yeah and I also got a disagree with Peter on that just in your prompt I actually value seeing a terrible implementation of something. Like if I got a pull request and most of the pull requests we get on the pie because it's already are made by agents without a lot of human touch let's say then I immediately know okay this is not the garbage but it's valuable garbage because someone has put in at least a minimum amount of thought instructing their agent to create this pull request and I get to see how actually the implementation of what they wanted I don't need to waste my own time on trying that out so somebody else tried it out already that the naive dumb agent do the same do the mistakes version and that saves me time.

I'm not saying I like pull requests by agents because they're terrible and they're autoclose them now but they have value it's not just a prompt it's on the next adventure right? It's been a big moment eventually always because they're in dynamics but I think we got to find out way earlier than in previous cycles that this is a bad one. But I think it's going to be interesting and I don't know the answer to this but I write this fascinating retelling of the British Industrial Revolution and how it changed the textile industry. The industrial yeah and so the general thesis on that article was like every time something at the head of the pipeline got optimized it created an incentive downstream of the whole thing to create something right? Like you know beginning like if you can weave the thing faster then eventually you need to have a garden that can be weaved at faster speeds then eventually you need to everything sort of turns the bottom like all the way down and like ultimately the biggest bottleneck in the entire thing it turned out to be what I think like is actually the next bottleneck we're hitting in engineering which is like at one point you made a shirt and if you didn't like the shirt you went back to the person that made it and they fixed it up for you and so the actual thing was like if the shirt is bad nobody cares about any matter who destroyed the shirt in the process this is just going to be the new one right like the responsibility actually went from

anyone in this chain to the entire factory as a whole doesn't have to care responsibility anymore because we have we've commoditized the whole thing so much that you don't you don't have to do this and if you take the engineering approach it's like a pretty significant part of running a company and running a service is like running it reliably and so you have these post-mortems on incidents to figure out like what went wrong in a process. You go back and fix the shirt. Yeah and the thing is like we we are running all on this idea that every engineer that sort of is in this creation process that ultimate let up is carry some responsibility and that we are going to that person and not like to blame the person but like to figure out like why why did you do wrong here and so like if you do if you like the machine produces stuff at like 10 times the speed the responsibility thing does not scale in the same way because the machine cannot be yet to be responsible and I don't actually know if there is a future where you can abstract the way human failure so much in how we run engineering that now the entire company now no longer cares about who signed off on a podcast or something like that they be automated in the same way I think as we are sort of automating different creation I just don't yet see that but so here's the thing I think one thing we software engineers or IT people

underestimate is just how freaking complex the world is and how much human squishiness is in each little nook and then granny in the corner right so we were thinking oh we were not able to automate that thing now we can automate everything like every bit of knowledge work but but we are soft engineers are so bad at becoming the main experts that we don't see all the non-machine parts that go into a workflow and we are running through the same fallacy here again we are seeing models doing incredible things I'm not just doing that like this is for me this is like war basically all my research in the 2000s is now null and void because transformers can do all the things but we are overextending that to everything like we always do in software like we did in edtech yeah we have tablets in classrooms now sure now it's soft education is soft because we have now computers well in fact I've heard I don't know which country it was but they're now rolling back yes we as we are there taking the tablets out from the classroom it turns out if you do some scientific investigations into the tactics and effects on pupils if you do just throw a bunch of tablets into a classroom close it and hope for the best turns out the best is terrible so yeah I'm that for me I think the biggest takeaway in the past two to three years is the hype is terrible because it's

dehumanizes everything and I want to not be part of that circus well speaking of not wanting to be part of the circus let's talk about pie which is a which is a very popular in my clown nose and also minimalist coding agent can we start with the backstory of why you decided to build pie at a time where there were already ancient harnesses around right because they were sub-optimal that's not anymore yeah sure and so I was a belief in cloud code just because they kind of created that whole genre truly a mention of a gigantic search I mean you mentioned there were that and shows of shines and so on but they were to first package it up in a really compelling package and at the time that fit my work well it was simply it was predictive song the LMM heuristic nature or the stochastic nature of being kind of unpredictable but everything around the other level is kind of nice and tidy and easy to know where you were happy with your clock out right it's super happy I was proselytizing it but eventually the team started dog fooding and getting more and more tokens I guess and kind of increased velocity and team size and with that

came more features and much much much more bugs and I personally like simple tools that are stable that I can rely on even if they have non-deterministic parts but all the deterministic parts should be as stable as possible that was just not the experience we've called around summer 2025 so I kind of saw it on that real hard it was it was it bugs was it unexpected like so they take away your control of the context they would inject stuff behind your back which is bad and then your work closed that used to work stop working because there's no system reminded that you don't even see the UI that will modify the behavior of the model they would also do the system prompt I reverse engineering I mean I wouldn't call opening an obfuscated JavaScript file and unafusgating it reverse engineering coming from a more low-level background but I reverse engineered clock code 2025 and build a little service where I can track the progression or evolution of the system from two definitions in clock code and it's like every release it was like messing with stuff cchistory.mariosection.at if you want to see that and yeah that just messed with my workflows and I don't appreciate that if I commit to a development tool I wanted to be a stable reliable thing like a hammer I don't want my hammer to break a different spot every day yeah the

server so that's what happened with clock but again I this is not like I'm not roasting I think they're some of them are really nice people I got to know on the internet they're just dog fooding and that's perfectly fine we need somebody who like goes to the full velocity kind of but I don't want to work with a tool like that because I can't get worked on it sounds like the move fast and break things break things was not for you no and then I look into alternatives and amp and droids came out around that time I think but early in 2025 I don't remember I was very early I think they sort of spun off from the same experience of taking because I think I was around when told came out that time yeah in any case I looked into those harnesses and they were super good they were just super expensive as well because none of them could basically use what made called enticing on top of it being a cool tool on the subscription and that works in an enterprise setting where you are paying by token anyways but it doesn't work for the small tinker and the garage while I'm not a small tinker and the garage in the financial sense anymore I can still relate to that community and I would like to use my subscription with something so I looked into open source alternatives and found open code

but while it kind of wipes with my OS roots it too did stuff to the context that didn't appreciate behind my back pruning two results after a certain amount of uh two result token output or asking let's vserver after every single edit the model makes if there's an error yes there will be an error because the model isn't done yet with its work so the code doesn't compile so the other species or whatever you're sharing so else we know the language server protocol server yes so when you go into these codes and you type some type script you have like in the bottom some error diagnostics and that comes from an LSP server four type script and open code runs an LSP server on your behalf in the background and feeds the model with diagnostics from the server on every edit we just program and so do we work right we go into one more more files we edit line after line and only then look at the errors the result from that in open code's case or another harnesses cases that also support LSP the model calls an edit tool to change lines and they would inject the acoustics after every edit call and that's just not smart because now you're confusing the model with you have an error you have an error and the one's like yeah I know I'm not done yet oh

it's not yeah it's not great anyway still the R open code wasn't for me either it was also I had to forget to modify it which I don't think should be necessary so then it's just how hard can it be a big little thing and then your own little thing is pretty minimalistic what does it use what's the basics of Pi? The basics of Pi are my own abstraction over all the lm provider APIs because I didn't like to burst LSTK, I never sell the ISTK for various reasons Armin kind of wrote a blog post eventually about that as well it's obviously good to use a lot of people use it it just didn't fit my old man sense of abstraction. The advanced resistance is the beauty of software and opens up especially open source you can build your own always yeah and now with agent you can even do it faster and produce terrible complex software now as I build an abstraction with that and I build an abstraction for a generalized agent with tool calling and streaming I love that I built a bespoke little tool that doesn't flicker or not a lot and then I tied it all to get into a coding agent that looks like cloud code or codec so where we have that's it and the accessibility comes from the fact that this minimal core has so many hook points that you can basically hook into with a simple types of module that gets loaded into the same node process and

that allows you to do things like provide the lm with custom tools to your own compaction implementation fully revamped the 2e itself you can modify everything in the 2e so if you have a special version of UI exactly if you want the 2e to behave differently for a specific workflow you have like you say you're non-techy you can change the 2e to become whatever you need as a non-techy and have a couple of non-techy friends that did that because they don't need to know how to build this they can just ask pi to build it and pi will modify itself oh so this is the thing right so you can ask pi to modify itself because of the extension points and it can write code that extends itself and it's trivial but it's a big unlock is this what you meant when you said that or open code you needed to fork it to modify it it doesn't have this it does have a plugin system but there's not a lot of extension points it was very rigid I think they changed it recently I think it's much more open now I haven't kept up with it but I do better now so as pi stars as a very minimalistic thing as I understand the tools it has is read right you read it best it's only it's that's it and then you can actually like start to make it your own like okay like what are examples of

people without why doesn't have mcp people just ask pi to build mcp support into pi pi doesn't have a plan mode so arming goes and my plan mode must be fantastically spoken I don't have a plan mode yeah but he has like five implementations of a plan mode until he realized plan mode is entirely useless other people just like messing with UI making it their own like a different visual style of the editor box where you enter your prompts stuff like trivial stuff more cosmetic stuff other people have reachiggered it for a full blown RL environment for open weights models where they use pi as the agent that does that's part of the RL execution environment so it's you can do anything really but roomy to it beyond like actually using the library abstraction was was in fact the custom tools part because one moment for me was oh christmas again like many people at some time and I tried to build other things and Peter was talking to me in November that he's like vibing for looking at code more or less I don't know exactly how he said like like he said like he said he can do this now like okay I want to build a thing where I don't look at the code I wanted it to not look like stuff like I wanted a version of it but like afterwards even though I don't really look at the code it should look like whatever and so I want to make

a game and so then I basically started the whole experience of like just basic pie I was like I want to build a game but actually before we build the game I want you to set up the codebase in a way that you can validate the change that you're making but also I can see them because like like a like a two-prong kind of approach like I want to be in the loop but also have the agent be able to validate itself and and what sort of emerged out of that was well first of all like it built itself some debugging tools into the game so I can make screenshots and like run a simulation and sort of dump out state and read it again but also pie can control images in a toy and and I added so a bunch of like I talked with the client to figure out like what would be interesting things to do but we ended up having like a all these screenshots I can tap through quickly in the UI or I can piles that this great feature can reverse to an earlier state in the conversation and then can branch within the conversation to build a bunch of stuff around that and because like these sessions they switch with screenshots and it became very token inefficient very quickly it was actually one of the other things that pie was rather quickly rather good at was having a little screenshot set because open claw people had a lot of screenshots in the chat and open closing pie yeah but but having this like it felt really magical for me to actually treat the

problem as I don't know what the right way of engineering here is but very clearly part of it is like I should be in a loop so we can figure out like how to specifically for the problem attend do that and and it turned like for that project and computer games and some of the other things that tried they're kind of different but very many of them are sort of come down to similar thing where like the agent interacts now with my program and should do the most optimal way and I want to interact with it in conjunction with it interacting with the program and the entire experience should be as little confusing as possible to both me as a human and to the agent and I found it very very fascinating just to see how that emerges what I get to all of a sudden when you launch it in this program looks and feels different than if you launch it in the other program I realized this point army made just a few seconds ago that ai works best when the engineer stays in the loop and the system can actually validate what changed and this is a great time to mention our season sponsor sonar AI can now generate code faster than you can verify it sonar the makers of sonar cube sees this leading to seer's gap in verification with the rise of coding agents autonomous writing code verification is no longer nice to have while the latest coding models are extremely intelligent

they also are error prone and they don't fully understand your code base and your context or your objectives this is why verification must be mandatory and agent workflows sonar cube provides a zero trust multi-layered approach to code verification that is consistent and repeatable it analyzes semantics and attacks data flows and architectural boundaries at agent speed acting as a critical trust and verification layer before any code reaches production covering 40 best languages and 7500 issue types sonar cube is the most comprehensive code verification platform available and with easy integration vmcp cli and hooks if it's writing to your existing ai tool chain let agents move fast and have sonar cube as independent multi-layered verification for safe reliable and auditable agent development head to sonar source calm slash pragmatic to start verifying your agent work flow today i'd also like to talk about a presenting sponsor statzig statzig build unified platform that enables bold experimentation and continuous shipping built an experimentation means that every role that automatically becomes a learning opportunity which proper statistical analysis show you exactly how features impact your metrics feature flags let you show continuously with confidence and because it's all in one platform with the same product data teams across your

organization can collaborate and make data driven decisions to learn more head to statzig calm slash pragmatic with this let's get back to the episode and to the topic of general versus purpose mate tools i mean i spend a lot of my youth on construction sites to earn money and you don't use a hammer for all your problems at the construction site you have a screwdriver you have your hammer you have your drill you have whatever and i think in engineering it's kind of the same i'm not using the same tool for every task i do as an engineer so now if i use an agent i don't want a general agent for every task per se i want to specialize thing where no the performance will be talked not short that's a specific task because we build the harness in the way that the agent can be most effective at this this style because of the construction of the way the harness is and that's what i wanted to enable if i that said i'm probably the person that has the least amount of modifications in fire i have like two extensions that are using a trivial they're basically just if you see a url that looks like a github issue or pull request thing pull down the details where the github api and display me a small little widget on top of the editor that gives me the issue titled the author account and a link to the issue that's basically all i do well but it might

work for you as a minimalist yeah i mean that's how it works on the on the pi-month request because i'm at a two or three of sessions open which i process an issue of pull request that way i remember what session what the session was about but sounds like you also made your pi for that for working on the pi-mona-reap way specific one and if you if you were working on up if you went back to building games you probably have a i never thought of the fact that you might want a different harness for a different task i guess we just kind of assume that most developers you work on your main thing at work you might have a side project and just experiment experiment with whatever but this one this i wonder if this is a new thing that we could never have we could never have custom tools for a project that that just sounds crazy you know here's his that like my intuition is this i think where we are going is software that modifies itself on behalf of the user's wishes and needs and the agents can do it now if you give them enough rope to modify themselves and i think you've pied this my first foray into this kind of self-modifiable maleable thing i'm just for the coding agent sector but i think this this actually can be extended to all kind of knowledge work to agree for specific tasks within the broader set of knowledge work obviously the

humanization so you know but yeah i'm the next plan here is actually to have an alternative using the face to the two because the two is obviously limited and the best alternative stack is obviously the web because it works everywhere and can do anything so once i have that built out that then it really becomes interesting because then you're not limited anymore to the line-based rendering of a terminal now you can do really really interesting stuff and so yeah we'll see how that and one reason that i learned about pi before i knew that it was this minimalist interface is how open claw is using pi how did that come from hanging out and reviewing each other's blog posts and just throwing ideas at each other and in October i started building up pi and Peter started building up our relay his little what's up assistant so this oh that's how it started yeah news in search of a agenti core he could reuse or copy i think it started out by him taking pi and cloning it and calling it towel and then modifying it but eventually he got tired of having to maintain that so just i'm gonna use your stuff that's how they end up big i wouldn't have compaction if it weren't for call i specifically built that because Peter was crying in chat and then he'd have compaction

okay you get compaction but i'm gonna tell all my users don't use compaction it's bad for you yeah but that's i guess the beauty of building on top of open software one another right i mean it does pros and cons yes i now get to enjoy all the open claw instances that think boxing open claw are actually pike bucks so the autonomously send me i guess alien issues and pull requests without their users probably even knowing and i get to deal with that in my open source so that's not that's not a matter of effect well so you're you're really only receiving enough of this i guess i mean just like a good self is which is much more exposed to this problem i mean there's tens of thousands of issues now and there's no way they can get a good creep on that but but how are you dealing with the fact that you now have open claw just AI autonomously opening things on your repo as a maintainer and do you build tools to battle this and try to close them out or build a tool for open claw ones which embeds issue and pull requests into a 3d space so i can see the clusters of similar things that agents would have sent to a repository and then i can bulk select things and close them out in oh really so you actually have to really like visualization yeah open travel context that i think is less crazy now but end of december so i think mid-February i mean it was exploding obviously but like this

explosion almost like directly translated to i was on this repo refreshing for requests and the number went up yeah we actually tried to contribute and help out here a little bit but i mean i gave up i didn't know how to do anything useful i was looking at this and i was like this is a type of soft-engineering i'm just not used to i would fix two things and spend an hour on them and then five minutes after i committed and pushed it some clanger comes along and just reverse my fixes and this is not how i can we talk about the name of the name clanger so clonvers star wars i actually never watched it but kids of friends of mine watched it a lot while we were visiting them so i kind of vruzmosis got the lower and there is an army of robots and each others would call them clankers or people would call them clankers because when you moved it clank clank yeah that's the origin of that yeah so in the ai a droid of yeah exactly yeah but coming back to the how do you deal with the influx of agentic pull requests and issues i just autocloss every pull request human agent doesn't matter um what i do is if i haven't had contact with you previously my bit of workflow knows about

this because if you have your in a file in my kit repository your account name so if you're not in there in your sample request your pull request gets autoclossed and then my little workflow post a comment on your pull request that says hey thanks so much for coming really appreciate it could you please open an issue in a human voice uh no longer than on a screen's worth of text and if i like it i type look good to me and then that account name gets put into the file and the next time there's in the pull request they pass and it turns out agents don't see the comment might get the workflow posted underneath the pull request so there's a great filter for filtering out agents and keeping the humans safe more from it's this interesting i wonder if this might be the like an unavoidable future where like we just need we need a way to separate is this coming from a human with an intent or an ai i don't necessarily care if if it were actually a good PR then if it came from a machine it's it's actually bienish i think what's interesting applies like an open plugin more so it's like it accumulates pull requests well actually there was no intentionality behind it at all and so the the person that dispatched the machine didn't actually care that much about it

or didn't even know about it or didn't even know about it and i've done open source for many years and that was also there was a big difference between someone that's any pull request up or like an issue and it's like hey please fix this but actually didn't care enough to even reply to questions anymore like this is not uncommon and then you don't actually have to fix that but you have to close it out because like maybe it's still useful input but like clearly that person wasn't caring enough and if the pull request is even worse now because they come in so quickly the many of them cannot be merged anyways without manual resolution of the conflict and they said there's a lack of back pressure mechanism because i even i as a human if i see there's like five hundred pull requests open i like i probably will not contribute to this thing now because at worst i won't make it worse yeah and i think previously in open source you had the people who would just send issues and be very entitled and say you're the worst person on the planet if you don't fix my little issue but that's fine that can be handled and pull requests were kind of special because it needed a human to invest quite a bit of time to reduce them and you don't have that anymore you just have people oh this should be easy uh agent please do a thing make a mistake send it to this repository and that's just not

going to happen so basically what we need are bottlenecks i'm not necessarily i don't necessarily need human verification or verification that you're human i just need a bottleneck that allows me to process the amount of incoming things as a human because in order for pi to not deteriorate into a pile of garbage i still believe that it needs me and other capable people reviewing at least the important code and for that any poll next because otherwise i can't deal with it's a second law of thermodynamics right it's like everything degrades towards chaos and you have to put extra energy into to keep it away from this uh on this outcome and we don't see and feel like the pain of the code is anymore if it's not looking at it and people don't feel the pain or like they feel no restraint anymore and it's the issues are also interesting because on the one hand it is something great about someone doing investigation and sending you a description of that that can be good and can be bad but they look very similar like it takes quite a bit of energy to tell apart a good and a bad AI-generated issue request and unfortunately like most of them are not great but some of them are actually good and

that's also kind of it's weird like all of it is weird i really don't know what the feature of open source is in many ways because like the a lot of open source really worked because people piled out on hard problems and so they congregated around it and said like now we need to have a good database so we're going to put all this energy on building new data ways and so the value of open source came from there's some hard problems and we're going to grow our energy together and we're trying to figure out how to solve it and now it feels like open source is all about like growing stuff up what really grinded me is so mad was people particularly like a lot of energetic engineering and i was like building more stuff for genetic engineering so it's like it's uboros or uboros or what i call it and i see this tweet and i was like oh i solved problem x where i see and here's my solution for it and you click on this thing as like it's 48 hours old that person probably never used the thing that they built i would like to suggest to the viewership to look at arman's github account over the last year and what happened there yeah i built a lot of this stuff but i don't then go on twitter so like hey i solved the problem right it's like i have a shit on a vipelop on my github account and i wish i could market differently because like maybe there is some utility in it but unless you're going to actually have that codebase still be there a year

a year and a half from now and someone is still using it the utility of that is actually not validated in a way and there's so many markers and metrics you can look at now for github that really demonstrate this is explosive growth of it but if you were to then maybe find some other number to see like how many of the things that are being created actually turning into like really fundamental pieces it can sustain open source communities that can actually deliver this value that scales amazingly we haven't actually created many vip engineered projects that have become that but i like how you mentioned energy and how open source always worked if we just think pre-ai and let's let's say linux the most successful or widely used open source project it has both an energy and a structuring of people come in with intent that they want to add something they have a process where it goes through there's human trust at every level there's a little pyramid and in the end it all goes back this change request goes up on level and in the end line is does the cut but there's a lot of energy there's a lot of intent there's a lot of humans there's a lot of humans and it was always about human energy and now we suddenly have this AI which

it's just tokens right now they're who knows how much they're subsidized or not or it's just machines doing and then suddenly you'll be able to create plausible things that look like human energy and it's hard to differentiate and suddenly just like throws this wrench actually disagree i don't think a lot is changed through open source okay um the volume was changed no yes but that's just a number uh the amount of as you said the amount of actually useful and maintain projects probably not changed a lot so you're saying that the ones that were there they're so useful and even the ones that were there might i mean there's a specific rate of new open source project that survived longer than two weeks that's always been the case right so now we just have more projects that die after two days and before but we still have the same amount of projects that will have long-term viability just because there are humans that actually care to maintain the thing over a long time build a community of humans that support the entire thing build an ecosystem around the entire open source project that's not the same you're not delivering to multiple no i mean good job matter putting that up it's pretty useful um no i i think at the end of the day we were kind of freaking out when we don't actually need to because apart from the fact that i personally

cannot generate code faster it must be the flight for me building an open source project and then it's not just the code but the community around it the spirit around it the ecosystem around it did nothing changed um what change is mechanical parts i need the bottom next to deal with the influx of exponentially growing uh agent full request whatever um github itself is not a mens pressure because now it's not just humans hammering their infra it's now billions of millions of open call instances hammering their infra everybody complains about github going on i actually think they're doing a pretty good job like that's a lot of traffic that's coming their way since basically christmas so yeah i i i would be a little bit more optimistic we're just indeed messing around and finding out stage at the moment and everybody wants talking to be a kpi just like lines of code using kpi we've seen this speaking around of things that don't change and messing around and finding out you you wrote a uh a tweet or you wrote somewhere that your biggest enemy is complexity it's also your agent's biggest enemy we're talking about that very simple if i have a 600 lines of code code base and my agent can at best be effective effective up to a context minuscule around 200 000 tokens

how much of a code can the agency a third right right um if you manage to get all the relevant code for a task into that context minum probably okay although that is a separate project and information retrieval problem which is not solved which agent search also doesn't solve that is that's are you sure that the agent finds all the relevant code it needs to find to fulfill a thing that's also where all the garbage code comes from because it doesn't see all the things you need to see in this case let's assume the best case information tree will solve everything fits into a context agent does a good job okay that's not the reality we're living in because now the agents we had so much code that they themselves cannot possibly read into their context on a new task anymore you know what i mean yeah but they fill their own context window yeah exactly the complexity they add is their own worst enemy because eventually the copies will be so big and so complicated and so interconnected um that the agent has absolutely no way on a technical level to ingest all the context it needs to do the new task and i would like to point out that the agent has learned all of this garbage from the internet and from us because on the internet there's all our old code while

there are some pearls uh there's also a lot of swine um because we have a gazillion guitar projects from the olden days where we just tried out things and because instances like Linux or any other really well-entained and well-written open source projects are minuscule in compared to all the rest of the garbage and a machine learning model will kind of converge towards not well simplified to the mean right and what is the mean then it's not the handful comparatively of excellently engineered projects all the garbage on the internet all the cargo culting all the trend type of the day kind of stuff and that's what we get when we let the agents do all the things for us yeah so we have this problem of things are getting more complex which still is agents down which will in fact impact quality which we were just talking about but Armin now that you're you're building your own startup you too if you're building your startup now how are you and you're working with agents right and they will have these things how are you dealing with generating code building products balancing quality typed up complex so dealing with that badly like i think that the coping we're not dealing i don't know if i wrote this in the blog i definitely have it on

my slides for for for conference here was um i enjoyed the time from from april to about october immensely because it felt like i can do so much but also like there was no heightened expectation like the world has not yet gotten used to this idea that everything has to now also move at 10 times the speed and that was that there was a moment of time where i felt like like we worked in this wipe tunnel thing at the beginning and i was like it felt so much fun because like i i have time now to play with the kids and i just prompted a little bit of my phone and like it felt that that was where you could set up with your phone talking with your yeah it was just like remote terminal basically yeah and it's not that we did much with it but like it it had this like heavy vibe and like i know that i spent too much time on the computer but it didn't i didn't feel any pressure but now it's like this like we're collectively feeling like everything has to ship faster it has to like iterate faster like the the the baseline that we want to achieve in terms of fidelity and everything has to be higher and so now it feels very stressful even in your own startup yeah because like to some degree you cannot like you can be

the most stoic person in the world and it's still going to get at you in a way that i'm slowly learning to work with my own emotions in a way on on like on dealing with this but it's i find it very very hard in a way to because like i was i was used to things working a certain way and i knew how i do some stuff and and then i felt a little too much in the trap of like giving into the machine and actually doing things in a way that i normally wouldn't have done things that you regret it's definitely really gigantic regret and so like quite frankly the answer is like i i feel like now with a little bit of power fine side um learn some things that i wish i would have learned probably in November tell us well i mean like a lot of it is like really the recognition that if you there's no back channel 2d to me or to any other engineer when an enormous circumstances there was a back channel that was this this this feeling of like things are not quite right in the code base like there was this now it changes harder and like the complexity like you sort of see it in the complex with the pull request getting higher but like if you rub a stamp it then like what's the back channel there and so like this mechanism this back pressure this friction in the code base you don't feel when you work with the agent i think there's a way to kind of measure it and um like if i scan

from my sessions on the projects from start to current date i think the frequency of course works increases because the agent starts messing up more because it itself cannot deal with the complexity of the project and i would be actually really interested in whether this is measurable because i feel it in most of my projects now that occurs a lot more but you mentioned friction in the software you didn't say tagged up you didn't say complexity what what is this friction because i i don't remember us talking about this pre-ai at all so i found this ironically kind of funny and it's kind of sad but i won't have any names but there was a what i what i assumed was an incident related to this in part so it's engineering on a company where it shipped out a configuration change that ultimately resulted in a security issue and look things happen but the link that i saw on this at the social preview of the company's tagline and the tagline was ship with friction and that can really give me pause because like i know as an engineer like we used to talk about like you got to get rid of like all the things in a way so that you feel happy shipping stuff but there they're always where changed that's why you really wanted to think is like do you want to drop the database like do you want to merge this migration which might take a table lock that could potentially take you down

right it's like this this this moment every once in a while what you really you are really supposed to think and and you and people created checklists or people created like like mechanical gates that would where you would have to confirm something like there's certain things that we used to put particularly if you run a sass company did you put stuff in so to slow things down or in some of the best engineering teams in order to mature service you have to define slo you have to define like expectations and like if your service is supposed to be critical but like there's some other stuff that unlocks on this sort of tree of requirements and a lot of engineers be like oh this is also bureaucracy but like the reality is like if you do this correctly then it saves you time and it like it makes you happier you're not waking up with three o'clock in the morning like all of this is useful it's like friction injected to deliver this still things down like as the easiest example in any decent size company you have services based on tier based on criticality the highest tier software now needs to have let's say two or three code reviews or an approval from a director to do a configuration change which again also is and but it's kind of like we know that this is on purpose

like by adding this friction we want you to think do I want to push through this friction in terms of time invested or effort or having to justify things it makes you think about do I really want to add this to the code base if I know that the end effect will be the test to go through this entire chain of order is so we can be coming back to saying no to yourself don't wait pain going through that process and then taking on the pain when you know that you have the company you have the backing you have the confidence as well right like so typically when it's a high friction thing let's say a tier one service or highest your service where they're gonna have to sign off when you're new joiner on the first day and you don't know the context you probably know that's a pretty large ass and you'll probably socialize get behind from a from an experience and say like oh this is right thing you'll go with that right back to human dynamics a little bit the thing is like it's a there's a there's a very delicate balance in the whole thing because like you don't want the friction to be just an accident of having created bad develop experience right but some things look the same and but they were deliberate but they may be not sufficiently documented but there's this feeling now like you get rid of all the friction so that the agent can be very

autonomous so that he can run many of them simultaneously a lot of it comes from that it's like I like these things are actually rather slow and the only real time saving that you get from it is parallelism and so somewhere there is this trap I feel like a little bit more experienced now in managing the trap but I don't have this motion for the leader and I will not like say that he's an example code base where I felt like really really great about the stuff that I built except for pre-existing libraries from before I try to take days where I still feel like we strong emotional attachment and I'm much more careful about doing them than then I have the code that we other empire but which I don't have access to there's still a right access there's a lot of sloppy pie but I try to avoid it in the bits and pieces where I know that's important code like we have an HTML export functionality where it takes the current session and just fits out an HTML file that you can then also get to open whatever I have not looked at a single line of code for that function I don't care if it's broken if it looks right when it comes out

but then there's the the agent loop itself or the the extension loading mechanism and all of that stuff and that's important and the way I deal with enduring that that has or at least trying to ensure that has high quality as I refactor mercy Leslie because that pulls me into a code base I need to understand what I want to change structurally not just line per line but I need to understand what's going on to do a good refactor and I'm doing that every now and then I can do it now at the moment prompted by wanting to add a new feature that's currently not possible with the current architecture being in the code is the one thing that keeps the code base quality high and complexity low but that's against the industry wisdom of burning as many tok maxing basically yeah that's interesting one happening but you just recently wrote on the same thing blog post called we only to slow the f down can we rehash some of the thing and what triggered you to just put it out there okay so basically this okay your agent can almost be done 10 times more code than you can but it also means it's been so 10 times more boogos errors even if it has half your error rate then okay it's not 10 times more it's five times more and still more than you

would speed up so the rate of deterioration in your code base is not increased and now go dart factory now take a hundred agents they do this to your code base what's the entry sort of that that's the first problem right and you need some way to review all of the code it now gets generated to fix all the boogos but you can't as a human because as a human you're used to spitting out 1.5k lock a day and that's about the limit that you can actually review well right the agent speeds up 10 times that which is you can review that and not all of the code variation might be important like the HTML exporting right but even if the agent speeds up 3 to 5k a day you have no way of reviewing that in any meaningful sense and then if you do the armies yeah I mean and then the armies is interesting so you call it the dark factory the idea being that tens or hundreds or thousands of agents you give them a spec they go and they break it up they organize themselves they like their mirror and all that jazz they have the quality QA agent they have you give them roles you give them context and then you give them enormous amounts of tokens and spend and the idea is that oh there's not for what we've done in oh there will be something will be

done I definitely something's gonna be done for your person and you know yeah sure more part of the people that make that work I can make it work and the reason I think I can make it work is because I still care about the quality of my product and I don't care if it's built by hand or by agent I just want the quality to be good both in terms of how easy this to maintain it and add new stuff to it on a developer side and on the user side all the companies claiming that all the code is not written by agents yes we know quality is garbage we feel you know bones when we use your products it's garbage so I don't want that and yeah basically I think people to turn around and say hey what what are we doing here we have these wonderful machines now they can take away so much pain from us by doing stuff we hate doing and doing that really well why do we start by giving up some more free time to work on the interesting bits and delegating the stuff we know they can do to them on large like like across the entire organization find all the things that annoy the out of you and have the agents automate that for you and then you suddenly have time to think about what do we actually want to build what do our users need and if we decide to build

thing then we can pull in the agents again and say and we're going to polish the out of that because now we have the time and the means and tools to do an excellent job but that's not how we're working we we build an army of agents and install beats and make a big spec that hopefully will be solved in something perfect but here's the thing we talked about where did the agents learn the knowledge from right the internet so garbage to mediocre now if you write a spec what's the best possible spec you can have the best possible spec is what you define exactly how it should work you give it test cases as well as the spec is the software itself oh I see what you mean so yes okay you're right as back that's not the software itself so that means there's a lot of planks that need filling in yes what do you think is the agent got to fill those planks in not most likely from stuff it from its training data and we already identified what the quality of the training that is right garbage to me okay well and even even before I don't forget like stack overflow had a really bit criticism because there was a thing of like well you control c control v from stack overflow and oftentimes there will be some answers where the first answer was either not correct or not correct in many cases red jacks for email was a good one you emailed red jacks for email first

page was stack overflow everyone just copied the first solution and I think underneath numbers we've said it missed a bunch of cases but but but here's the thing though I'm not saying agents or humans are better they're clearly not but agents also don't solve that problem and if you then don't let just one agent that's already 10 times more productive as you do the thing that is better than that you as a human are better but a hundred of those what do you think is the outcome it's just very simple math let's talk about another controversial topic mcp versus cli oh my god it's coming up and you know right now i'm hearing a lot of people really going for cli is the future and I think i'm saying with two of them but also mcp are also really popular inside of large companies especially when you talk with a bunch of people working at large companies as seems mcp's have found a real product market that inside of larger enterprises despite what people might think i don't actually hate mcp quite as much oh wait we have it on recording yeah no we don't deal in so my fundamental challenge with mcp is that i think professor but the spec is very complex i think for it but i think this is just generally how spec's happened to be so it's a bit like the

the core bar of its time so there's an inherent complexity but if you wait to some of these what is it really doing at the end of the day it's its its authentication and it's sort of invoking some stuff and mcp even theoretically the structure responses but mcp for the most part is run some stuff put stuff back in the context and then work with it so it fills your concept very quickly and there's a cloudflay has a scotmoth mcp which i think in principle i really like i have an mcp testing which is a javascript interpreter that gives me access to google api and between an mcp like this and a scale that's not a huge difference because the scale also needs to be in a system prompt so that it defines it but the the agents are just very very very very good at running code and mcp is not quite running code it's basically rag it's like input in and do some stuff and maybe some station at the model also doesn't see but it is in that sense just as a hard problem to solve but it does solve off it solves a whole bunch of things i want it to work i just still don't get it to work like i wish it could work and my suspicion is still the glue is it has to be code

execution but because mcp servers are largely not defined in a way that the model actually understands them i haven't found ways to compose mcp tools reliably i found ways to make the mcp itself be composable by having the mcp one tool run code but i haven't found ways to then orchestrate larger ones i wanted to work and i think it has found this niche and i don't think it's going to go away i think it's just a victim of its own success really when the whole thing started i think it was in october 2024 if it was more or less a solution to get external services into consumer facing chat apps yeah connect to emails connect your one drive connect your one drive really much and then id also took it over because it was convenient the cursor is the one serves yeah but i think the origin was basically the consumer side not the developer side and i think that's a totally great use case i don't want my mom to having mess around with code generation or whatever to invoke some api or calls them api so it's perfectly fine use case and then developer side also picked it up and thought it was a great way to provide tools to my other tools as in in the system from some of this if you want to call this tool provide this

chasing payload and you get this thing back right and that kind of felt right at the time because if you read the tropics documentation they would say our models can deal with about 30 to 40 tools in the context and even that wasn't case like 12 20 they would just break down but doesn't matter but but there was still like yeah this can work if you kind of keep it small and contains and very specific to use case and then people started building mcp service that would just basically map an entire open api spec into a gazillion tools that's very all fell apart that's the first problem very bad mcp servers from big corporations that thought we need this now what's the fastest thing we can build i just push the open api spec for api through the thing and make it an mcp that's garbage the second problem is that it's inherently non-composable if you want to combine the tool out with the mcp tool outputs of two different servers they need to go through the context the the model itself needs to do the data transformation the the the the the composition of multiple pieces of data fetch through and compare to the cli it's a pipe right exactly did the model only see the end result and it is super free in how it massages that data

and that's also the idea behind code mode basically it's a heck it's basically okay we now have mcp we know it doesn't work for this specific use because we have multiple sources of data and you want to combine them but don't kind of pull that through the context so let's build code mode and code mode is basically we take all the mcp servers we expose the functions and types with and then the model can actually just write some code to cause the mcp service and then does the composition in the code that sets like how many interactions do we want here we can just let them all write the code we don't need the mcp server and then the third part is david from century it's a big program of mcp because it's awfully off thing and honestly that's again for me super valid but the model itself kind of doesn't make sense anymore i think that there's a there's a world for mcp 2 which is ironically maybe based more on there's a company called stainless which basically generates as the case out of open i specs and i'm really warming up to the idea of like maybe there's an mcp is entirely based on off plus like libraries or like directly like hcp request against off specs because if you compose it together there and i think like one of the things that's also like

kind of underappreciated and sort of acc but they give you some pie with stuff because it's kind of transparent of the tool cause that it does it's kind of magical at times like how creative agents get at large outputs like for instance pie when it runs a program in bash and it proves too many lines of code actually only reads i don't know what the cut off is but it reads the first couple and it's like oh if you want the rest of the file it's 20 megabytes large and it's in this file and then the agent is like oh 20 megabytes that's too much i'm going to grab on the file right and they get really ingenious in how they're interacting with it and like and mcp takes that away the question is like how would you define mcp in a way where it wouldn't take that away where it still has all of that magic and and capability and and and i don't really know the answer because i think it's hard but often it's solving and and composability in solving and i think that is like the bright future of of that kind of stuff and also like what maria said if coding agents wouldn't have become so popular then the idea of code generation code running for like non-code related problems probably wouldn't have taken off quite as much too but like the most capable personal agents open-flopping a good example of it that's just coding agents hidden from you and then that just

naturally some random person who's not a programmer is going to say how am i going to do this and the model doesn't say like install this mcp the model says like okay i can write the pipe descriptor and so you naturally have this in the sort of the crazy space you have the adoption of of more code execution and the compliant enterprise space you don't have that that's a different path to it and i personally don't think that models are going anywhere else other than code generation going forward for any kind of a genetic task i think that's that's mostly a function of there being a lot of training data for code generation and code generation being a very easy means to control computers so i don't see a different paradigm there coming out of the model labs anytime soon so i think taking that as the assumption where the future is going you just need to figure out how to make code generation kind of work within an enterprise setting with off and all of the other enterprises things that entails so so let's do a fun trying to predict a year out which is hard but in 2027 knowing some of these basics just again from first principles where do you think these coding agents might be and the software engineering workflow might be you know basically this is just like in speculation we know we can not predict the future but where do you think

that there'll be a lot of focus in the coming year and we might in an optimistic case see some results in in tools and how we work and what's working with not working i have no idea but i honestly could make up something that's probably not going to happen i think the self-mariability thing is obviously something i believe in i think we will see more of that and see self-muteable software yeah including the tools themselves with which we built the software and i think that will expand not only to the tech sector but also to non-tech applications of agentic tools my is it dog years which is time seven so that works so that's that's basically the model i have right now of like how this stuff works it's like when you ask me like what's going to be in in a year it's like seven years right and to me that makes it incredibly hard to have any sort of predictions about the future because like it's still not one year like maybe now it's one year from like people starting to using code but it feels like it's much much longer much more time behind and more time has passed and i think like right now that the closest that i can imagine is going to be like we know the code execution and code generation and like this harnessing around

it this is this is going to be it because reinforcement learning gets more of that data and my my strong hypothesis is that as more and more people are starting to wake up to this you can do interesting things with agents there will be a societal recognition also of how much more dependant you are on basically two companies and i think we'll have a conversation about that part we should have a conversation about that part but it's because we don't really have these labs over here and so i hope we have a conversation but like my best guess is that we'll wake up to the fact that we are now i mean engineering teams are already not telling me that they have code bases that they think they could maintain anymore with automation my guess is that one of those companies will be public and and it will be expensive and i think that might actually dominate or at least become a conversation that's much bigger than the question of are using py or using code code or or something like this i also see we've seen this with was it missus the new cloud one oh no spot the new gpt model they will only give this to select partners so now we are seeing a split in who can get the best intelligence yep or the perceived best intelligence yeah that'll be

interesting dynamics so both of you are working on AI on popular AI tools you're building a startup that of course you're using AI and it's also around agents how do you both keep up to date and just seem things and it's not as easy to get in a high train as it used to be but that comes with age it's definitely not being in service because i think that's just driving crazy i hear so many things from my peers over there that's just like yeah i'm not gonna go to the services because thank you so i have a peaceful environment around you where it's not all about tech it might be helpful in terms having a kid it helps just going outside climbing trees going ice skating and then looking back at what you could just have an hour ago and be like why would i do that it's just stupid i mean to the depth that's a tremendous maybe people that are trying to stay in contact with me i i got very good at not muting notifications not reading emails and that has in part become necessary i think over the last years or so but this like it actually turns out that passage of time sometimes carifies stuff a lot because i give it really necessary it's going to to reach you again like i have an unhealthy twiddediction which i'm not particularly proud of but in in terms of source of like interesting

things that is still a thing but i i try to now sort of consume it in a form of if it's really really important it will stay in the discourse for quite a while and i just wait it out and if it's if it's there until like three weeks after the region happened then probably something to it and and i don't need this three week i'd start necessarily but it is honestly it's really hard it is really hard to to deal with this because there's a genuine excitement in it and i feel like my my 20 years of experience in that space of software engineering doesn't it helps me a lot of stuff but at the same time it hits you in certain ways where you felt like there will be grounding and there will be something to build on in a strong foundation and that feels like well seemingly everybody else doesn't care about that foundation anymore so maybe you don't need the foundation and for quite a while it sort of it works and and that is sort of weird and i don't know i kind of feel like since we've been fun employed in 2025 and all this started that we had like a head start like i see on the excitement the two of us and peter had in april last year as well as wait nobody else nobody else at the time has kind of shared that excitement uh that much and then the christmas

break game and now everybody else has that excitement that we had in april right so now they are learning ropes now they are captaining themselves to immeasurable amounts of lost sleep uh and at terrible copases um i think it will self correct because it's not sustainable yeah we did see this as well i did the pragmatic engineer is early march when a lot of people who are very excited and january about all and they start to use the new models what they can do they went all in at work or on side projects in about two months time a lot of them were like hang on it introduced all this complexity it has these things i'm not going as fast as i thought i would be a sector so i guess there's just a natural thing where you you have a time anything you write a job anything you have a honeymoon period where you've got the blinders on which you should by the way and then you start to realize and maybe over correct but there's a natural thing where it in general like it just takes time to see the outcome of your decisions yeah so i'm not worried about all the dark factory and all the software is there then sassy's there not that i terribly believe this is just part of the high machine that was self correct you know as closing what's a book that you would recommend and why code by petzold

classic i just love it it's just such a great read it's also for non-techy so it's the first thing i recommend if anybody ask me what's your job i'm warning about that it's like it has much less to do with computers than you think and i read recently breakneck which i unfortunately forgot the author of um it sort of goes a little bit into an exploration of like how china works and or maybe europe and the u.s. and i found it at least um both were welcome well very good arban thanks a lot for this conversation it was great to have it in person so having us thank you this was a really fun conversation thanks to mario and arban the idea of self-motify will software really grill me mario set help pie doesn't have emcee support plan mode and then the other features that depth would want from it but you can build it into some code so far it's working pie is popular because it motivates itself i wonder if and when this concept of self-motifying software thanks to ai will spread outside of just the stuff too i also like how we talked about the observation that agents don't feel pain but humans do when a code base gets too complex the human engineering feels the issues this creates and this tech depth is what pushes refactures and rewrites but agents simply do not do this they just keep

adding to the complexity and at a code base where deaths regularly feel the pain of the code base and do something about it the quality will probably be also better and finally the mcp versus the cli discussion this was a good one mcp is more about offering tools for ai through context and cli's allow piping one to laughter the other with mario and armin are more of the fans of the cli but in all fairness mcp has its new cases for example inside larger companies the right tool for the right job new check out the shown as below for related the panic engine deep dives that goes in deeper into related topics if you enjoyed the podcast please do subscribe on your favorite podcast platform and on youtube a special thank you if you also be ready for the show thanks and see in the next one

Frequently asked questions

How long is this episode of The Pragmatic Engineer?

This episode is 1 hour and 33 minutes long.

When was this The Pragmatic Engineer episode published?

This episode was published on April 29, 2026.

Is there a transcript available for this episode?

Yes, a full transcript is available for this episode. You can read the complete transcript on the episode page.

Can I download this The Pragmatic Engineer episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.