Codex is one of the most popular AI coding harnesses today, but how did it all start? Many of you will know today's guest, Tebow, from his generous and pretty-fruquent Codex usage resets. He was also there when Codex started and has let the broader Codex team since. Today we cover how Codex started and why it was built in Rust and made open source. How codaries are changing inside the Codex team and OpenAI.
What it means when maintenance and re-architecting are getting ridiculously cheap. What the merge of Codex into challenging PC look like and the many under-appreciated engineering challenges of this project. If you want to understand how teams inside of OpenAI plan review and ship software, this episode is for you. This episode is presented by TurboBuffer, a ridiculously scalable, fast-in-chief hybrid search engine built on top of object storage by an engineering team that are really going to like after spending time with them. TurboBuffer is the tool that companies like Intropic, Notion, Cognition and Harvey all use to connect their AI products to massive amounts of unstructured data.
When I've talked with engineers to use TurboBuffer, the theme that always comes up is reliability and performance at scale. There are reasons for this to have everything to do with TurboBuffer's architecture. TurboBuffer uses only object storage for state and MVM, SSEs with memory cache for compute. Data in TurboBuffer is organized into namespaces. You can think of a namespace as a database table or search index or an S3 prefix depending on the world you come from.
When a namespace is not being queried, it stays on cheap object storage with no associated compute cost. When a namespace is active, TurboBuffer pulls it up into hot-catching tiers so queries are very fast. This design fundamentally makes it effortless to scale to hundreds of millions of namespaces. If you're building a multi-tenant AI product, every user and their agent can have their own dedicated search index without overhead. And each namespace can hold hundreds of millions of documents without any special configuration.
You can scale TurboBuffer virtually without limit and the performance, reliability, and operating model all stay the same. If you need to connect AI to lots of data, TurboBuffer should be your first choice. Check it out at www.tribbuffer.com.com. Thank you for having me. It's good to see you again.
It's good to do this last time with it in person. Now we're doing a video first. I wanted to ask you, how did you get into tech? When did you first know that you want to work with computers? I had a good question.
It was a long, long time ago. My parents actually decided to move out of Brussels where I was born. And I just thought it was great to just buy a small house and refurbish it. But it was in the middle of a village with not much going on. I think there was roughly 200 people living there.
Many that I felt I wanted to talk to or could make friends of it. And so I kind of got stuck. This is like very eight years old. I got stuck as computers and early days of the internet. That was my way to learn about things.
And so the rest of it just came from that. I sort of like I owe it to my parents to have moved into the middle of nowhere. And then I had no choice but to get interested in computers. Once you finish high school, you went on and went to university, right? Actually studying it properly.
Yes. I studied mathematics, applied mathematics at university. I went there quite early. And so I graduated earlier as well. I thought for a long time that I would actually not make it and then I would drop out.
I had just small companies and small consulting business like the while I was studying. I was working for banks. I was working for, I was very interested in supply chain and applied mathematics problems. And I started selling that and learning a lot through that. Eventually I ended up in the startup world in Belgium.
I did that for a little while. And then moved to London to work initially at Google and in D-Mind. And it now moved to be here at OpenAI. It's like California, I love the California weather. We can talk about that.
It's been very good. After university, you found the startup. You had the startup bug in you or the entrepreneur bug. Yeah. So this started with all about pharmaceutical supply chain.
Looking at the supply chain for clinical trials and trying to optimize and decide, okay, should you produce more medicine? Where should you send it? Where should you dispatch it? How do you avoid waste? And through that, making clinical trials more efficient.
And this was using traditional non-ML techniques, more optimization, solving Monte Carlo simulations, these kinds of things. Stochastic multi-stage optimization problem, really. And we also applied it on steel industry and we applied it to electrical grid as well in Europe. It was like anything that had the shape of an optimization problem, we sort of get interested in. And to this data, this company still exists and I think they do some of the most interesting work still, but it's changing a lot with modern AI for sure.
But it's interesting because you kind of said, oh, yeah, that wasn't ML. It was just a traditional stuff. And then you go into Monte Carlo simulation and optimization and this algorithm. I get a sense that you kind of just went deep, right? It was like, okay, here's a problem space.
How can I use mathematics stuff that I learned? Something I didn't learn to just go deeper and deeper? Do I sense that correctly? Yeah. That's why I was obsessed with applied mathematics is just really this idea.
So you have theoretical mathematics or you have theoretical science and physics and like there, you just you do it because there's something to be discovered and something beautiful about it. And so all about patterns and pushing different here, but you don't necessarily always know like how you're going to apply it. And then there's like a real world, right? This is like, you know, there's all these cool problems that just lie around. And I was like, very interested in seeing like, you know, how can I make a world better?
And so like how do I apply like, you know, sophisticated mathematics, you know, to just optimize the world around me. And that was like a lot of the thesis behind that started. Yeah. And then after a startup, you ended up at Google and first at Google London, it was in 2015. And I remember if 2015, Google was really, really competitive place to get into like, maybe as competitive as opening eyes today in terms of the industry or in terms of prestige, you worked on maps initially and then you moved over to DeepMind.
Can you talk a little bit about what you worked on and then why did you move on from a already really interesting space that you securely loved, you know, like optimization logistics and all these things? Yes. I did start on Google Maps. I started on a project that was meant to make the web faster and make to make websites faster, especially on mobile at the time, you know, Google was like, so like seeing the transition from desktop to mobile and like more and more traffic going to like mobile phones. And so like, when I got ahead of that, so funded like a number of initiatives and projects, I was working on one of them.
This was like really, really fun because it was a small group within actually the ads organization. It was meant to sort of like, you know, offset the loss for the ad revenue loss because of this shift of traffic to mobile and worked on it for roughly two years. And then it was canceled. And although it was like the most fun I've had on, you know, solving hard technical challenges, I learned a lot from not having product market fit, not having the right users, not having the right feedback loop, not trusting your product manager when they say the project is going well, when in fact it's not going well at all. And then one day it's just like this VP flew in from California and then it was just like, oh yeah, we're canceling this project.
You know, unfortunately you only have hundreds of users and this is clearly not Google scale. And then it's unbelievable, but people were surprised. And I think there's a lesson there that I carried with me, of course, is just always question, always go to like always, you know, deeply think about the impact that you're having, but also like the importance of the overall project that you're contributing. And then I moved into Google Maps, Google Maps was super fun, worked on reviews. And then after roughly a year, I couldn't ignore like, it was just, it was this special place headquartered in London.
So many great things were happening. So it's like really early days, you know, with rumblings of things like AlphaGo. And they just seemed to be doing extraordinary things and, you know, just really tackling the very, very hardest problems that you can tackle. And like, with my background, I was obviously drawn to that. I started there.
Like I worked on a lot of the research infrastructure, research tooling. This is a theme that I carried on for almost a decade. And so this is very much also the con way approach things is how can I build tooling and products that help make others more efficient and bring a lot of utility to them. Initially, I was doing this for research. And then like over time, you know, I got like into thinking about things in a much more like more general and general way, you know, eventually like, you know, ending up where I'm now.
And a fun story that you recently shared on X as well is how you were part of the team that built this internal Google about that was, you know, if you want to say similar to Chad GPT, but a year before Chad GPT. Can you talk about that? That's a new story I haven't heard it before. This was part of the mind. There were like multiple efforts as well.
There was like great as well that was separated at the time. They had their own efforts on large language models, but it was definitely something that was being explored. It was not the main thrust of mind. Mine was like very much worried and busy like thinking about grand challenges and games and you know, thinking about RL, not in the language sense. So there was like this group that was pushing on large language models and you know, thinking about, you know, what if what if large text corpus is everything?
What if you just pushed language to its maximum and you just scaled language models? Like, you know, would that be enough to get to general intelligence? That was like a hard debate at the time. And then one group decided to just really push on that. And then it felt really natural, like, you know, as I was building tooling, you know, with others for research is like, you know, obviously, you're like, you know, what can we do with this model?
Like, how do we present it, you know, to the researcher, like how can they sort of like debug the inputs outputs and eventually you sort of like end up with, you know, like a chat system. So we built that internally. We had a lot of fun. Initially, the models were like, you know, kind of like almost like a little bit absurd, like, you know, not very coherent, not super useful, but it was a lot of fun to sort of like think here with them that caught up like, you know, like wildfires, you know, this application is just sort of like, you know, everyone was kind of like sharing little conversations with in mind, it felt more than like a research project or like a research project for researchers. And so then there was this desire over time to launch it as an external product, but the plan was just like not set up, you know, there was like the, the right way to launch products at Google, there was like, you know, the whole machinery of like, you know, how you do that, you know, the whole like, the last production stock, obviously very, very optimized over the years to do things well, but also very, very hard as an environment to truly innovate.
And then I want to ask what made you, you know, look around or maybe in case they're opening I but I feel you partially answered this question. Just putting myself back into your shoes. Like you're, you know, it's 2024 or 2023. You're inside of Google who are publishing amazing papers, doing really good research. You're doing super fun stuff, right?
That's pushing the limits of what's been done before. Yeah, it's an inside a company where you already moved, you know, for people who are feeling kind of comfortable or good about where they are right now, which I imagine you must have been like, what made you still explore? All right, like what else might be there? Yeah, I was very comfortable. Uh, it's a good place.
But really I had a desire to meet great people, but also join a mission that I truly believe in and that, you know, I felt like the people were through to the mission and cared deeply about impacting the world in a very, in a deeply positive way, but also in a direct way. Not being like, oh, yeah, it's just like, you know, we just do this work over here. And then it's like, it's the job of someone else to figure out, you know, how to make this useful is like, I wanted to join a group where, you know, like all the parameters were sort of like, consider together where, you know, research and products were like really co-designing. Okay, I was just like crushing it. Uh, I thought Chachibiti was like, you know, taking off.
It was like, I was, I met a couple of people from OpenAI and I was like, wait, what, you know, you only have like 20 people working on Chachibiti? Like that is an insanely small number. That must be like extremely empowering. Like, you know, how does that work? How do you manage to maintain, you know, a product with that level of scale and with that level of autonomy with, you know, only 20 engineers?
And then, you know, I was like kind of a dog and a dog and a dog and it was just an amazing group of people, amazing mission, you know, super talented, super driven. And it was, it was like, drew me in. And then I joined pre-re reasoning efforts immediately, like, typically opening eye fashions, like I joined, it was like, oh, yeah, you know, like, there's this thing going on, like, you know, we're going to launch reasoning models, like, you know, it's like some new paradigm and then you know, start sprinting on that. And like, you know, like a month later, like a company launched O1, uh, O1 preview. And I was exhilarating to be part of it.
I wanted to be part of like a place that moves fast, cares about impact, would be in tune with the world and, you know, just really listen. Um, and sort of like, that's also to me, like, you know, what I've carried with me, like when, when building code eggs, when building products, it's like having a community, listen to the community, just really focus on like a really intense feedback loop. Uh, and then building something that is just like, you know, you just really want to care about it and like, you know, care about the utility of it that it provides the world. And then of course you start to work pretty quickly on code X. So you join in 2024.
Can you take us back? What the thinking back there when you joined was about AI or LMS and code. I know there was this ASWE effort back then, uh, we talked about it in the deep dive as well that we didn't private engineer, the autonomous software engineer. A3. Yeah.
That's what it was pronounced internally. Um, we don't, we don't have an A3 effort anymore. Like, it's, it's, it's pretty. Um, but really for me, it was, I, I joined, I started building infrastructure for research, like my, a lot of what I did before was large scale, um, data storage analysis and then tools to understand training runs. I, I did a lot of different things over my years.
Um, but it was always about building for others and making them faster and just really caring about, you know, fundamentally doing that well. And then through tooling and infrastructure making new things possible. And so when I joined open AI, it was like, with the same idea. And then with the, with, with one preview and like, you know, some, some of the later models, it was very clear that we had to use the models themselves to help us go faster. And so I just really got obsessed with the idea of what were the limitations, how were we going to use those models for research itself?
So we got together with, um, other folks in research, we started training models. We started building little agents that were truly the precursor to, to, to connect. And this was like, we were training internal models to be very proficient on the Python code base of opening AI and then very proficient with, you know, having like good, good taste in architecture, good taste in, you know, like code style. It was a Python only. And then the idea was like, you know, we would sort of like use that to build infrastructure very quickly and, you know, help researchers go faster as well.
And then, you know, and then we would move faster. And then over time when you just kind of push that and simplify it to score, you're making a lot of, you know, we found that we could make a lot of progress very quickly and then learn very quickly and then drag and Sam are, you know, people with, with immensely supportive and also, uh, Craig was very adamant that, you know, we would, we would not just focus on ourselves, but we would also focus on benefiting the world. And so he just sort of encouraged that we would be thinking about this, not just as a tool for opening itself, but also as something that we would actually make it to a product. And this is when we merged this research effort with this ACI effort, um, and we started building one thing and then that led to a sprint, which was like the initial cloud code X that we launched, which didn't really have P enough because it was like a little bit too high friction.
And we also launched the codec CLI and we did push, but it was always this idea of, Hey, how do we get models to really help here? You mentioned the first, you started to build this model to train on the Python code and actually help me build in for better, but then you made this interesting decision where for code X, you built it in Rust. And at the time the model was not on distribution for Rust, right? It wasn't as good as Rust and it wasn't Python or TypeScript. Why did you make that kind of a decision?
Was it kind of like, did you expect that it'll catch up or, or you figure that performance is more important, right? It was very counterintuitive. Most of the other harnesses built were actually not built in Rust. They were built on distribution on TypeScript or Python or something else. Yes.
From first principles, like we very early on, we were thinking about the product interface and the agent as different things. So it was very important to build the core of the agent in a way that was robust, that was secure as well, that was, um, you know, engineered for efficiency and scale. And having worked through projects over the years that go from, Hey, this is a fun thing to like, Hey, we need to scale this to the scale of like the largest data center. Um, is the early decisions early on are like really turned out to be quite important.
As long as you don't sacrifice too much of the velocity. And so it's like, it's a, it's a trade off, but we had very prolific and amazing Rust developers are internal models were not bad at Rust. Um, and then you get a lot of validation as well, like compile time is like, you know, statically verified and all these things. And that is great for agents too. So it turns out, you know, it was quite clear that, you know, Rust as a language would actually be quite good for agents fairly quickly if we decided to put some effort into it.
But primarily we were focused on correctness and we were focused on efficiency as well. Interesting. So you're saying, you know, it's worth in your case, it was worth thinking ahead of where you want this thing to be. And for example, thing like a language choice, obviously with agents, you can rewrite a bunch of stuff and easier than in the past, but it's still like, you can save, save yourself reworking by putting in the right, I guess scaffolding or, or, or, or, or, you know, the, the baseline of what you're building on, right? I think we could have been successful if we had been in a type script or, you know, maybe even Python and then it would be fine.
And then, you know, we would have written it at some point, but having a very clean separation between the agent itself, which can exist irrespective of the product. Um, it was a very important principle. And if you write everything in the same code base in the same language, it's like, inevitably you're going to be a little bit sloppy and, um, you're going to intertwine things more than you should. And then it's going to prevent further innovation after that. And so that was, that was very important.
Like the rest of the under reignist, that's like, was very useful for that. One interesting decision that you made, which is what unique across all of the major labs is having this built in open source, right? The CLI is open source, the SDK on the app server, all open source. When and why did you decide that it's not a given, especially, you know, there used to be jokes about open AI having things closed, but this, this is actually the opposite where like this is open, whereas like some competitors would, which have closed source harnesses, which again, I think it's very easy to understand why you want something cool source. Why did you want it open source?
There was something really cool about the idea of having the code open source because fundamentally what you're building is you're building a coding agent. And so we were sort of like thinking about, well, if you have that, you know, you're obviously going to point it at itself and, you know, maybe, you know, you can build a community of contributors that use it to improve it. And then, you know, you can learn a lot from that. Also, it felt at the time, it's like, you know, very clear to us that if we were going to be successful, open source itself would change. And the role of code itself would change.
And so being part of that community seems important instead of divorced from it. I think, you know, it's hard to solve problems if you don't sort of like witness them yourself. And then the other thing was just it still feels like early, but it was very early at the time. It felt like we would have some ideas for how to solve things well. And we were go designing these, you know, with the training and the research.
And it's all about expressing like the capabilities of the model and like the most flexible in the best way. And then also we didn't have all the answers. And sort of being very open about, hey, this is what a good harness looks like. This is how we think about it. We did like a couple of like very technical like deep dives and block posts.
And we talked about it a lot. And we thought, you know, hey, it's just like the world is fast out there. Just like crazy smart people is like, you know, we're going to get inspired by other open source project as well. And so let's just make this a level playing field and sort of like encourage a lot of tinkering and exploration at the stage. Now this has been now, you know, like a year later, a year and a half later, which is a very long time in right now, this AI side, but looking back or taking the experience, what are the benefits you've seen that kind of entering benefits, the entering team's benefits from being open source and just honestly, what are things that are kind of hard about being open source, right?
Like there must be downsides. Like just trying to get on the stake on both sides. Yeah, they're definitely downsides. It comes at a cost, right? The benefits are it's almost something to build in the open.
It's awesome to have like a small repo as well. Like whenever we hire someone and they join the codec team is like, they've seen the repo before. They've looked at PR. They're like, boarding is done. Yeah, it's done.
Yeah, it's like an onboarding is just like, you use codecs to look at the repo, you know, with you and you ask some questions, but it's like, it's not, it's not a secret issue that you can get productive right away. We get a lot of good contributions, although we get like, you know, a tsunami of like random stuff as well. Obviously you and everyone else, right, open source is changing. I think this is one of the examples. That's right.
And then to me, it's just, and to a lot of the team, it just brings a lot of energy to just be part of the community and like the directly contributing, not just saying that we care about the community, but actually doing things that, you know, you can see it's, it's costing us effort, right? We don't have to do it. And the downsides are, you know, it's separate from the rest of our code. So, you know, sometimes we have to draw like artificial boundaries and, you know, work across multiple repos when we're working on something particularly exciting. And, you know, we're building it in the open then, you know, at times we find that, you know, others copy it, you know, before we have the time to release it.
And it's like, it's just a little bit sad. Um, but also it's like, it's like, you know, it's like, you're building into open and it's like, you know, that's, that's sort of like a contract that you signed. It's like, you know, you can copy it. Uh, we have a very permissive license as well, but it does thing a little bit when you're working on something and you're like, you know, and then, uh, the third thing is just like everyone else is like, you know, we are overwhelmed with, you know, random contributions and you know, we have to deal with that additional tax, but then that pushes us to, you know, also like try and solve for it, right? Which I think is good.
And on top of the open source, one thing that surprised me about Codex, and I didn't even know about it until recently, it's not tied to the open AI models. You can use other models with Codex, you know, like putting myself in a vendor shoe, it might not be very obvious because again, all the other brands that I look at when they do a CLI, it's kind of use it with our models. Again, what made you decide to be this permissive about, you know, using or allowing to use your harness with other models? It felt quite natural if you are part of this community and building an excellent coding harness is like, why would you couple it to your model? That felt like so like quite disappointing to make that decision.
So it didn't feel right. And in general, it's like, I think, you know, I kind of tried to make decisions that I'm like, yes, you know, just like I can just like explain it, you know, it is correct. It's the same reasoning with, you know, it is open source in the first place. It would have been trivial for anyone to fork it and then add support for another thing. But then you're just encouraging people to just like, you know, go and use that fork and then constantly you have overhead.
And the only reason you have a fork is because, you know, you wanted to change like 10 lines of code to add support for like another mobile provider. That feels very silly. So like, you know, why not just order in the first place? The other thing is we benefit a lot from like being able to just give optionality. So it was like, maybe today, you know, you love using open AI models and, you know, you're super productive with them.
But like tomorrow there's a new model that comes up. You want to try that. Why force you to go and completely change your setup just to try a new model? And then we benefit from the feedback that we didn't get, which is like, you know, maybe there's something that you liked about them or maybe it actually didn't work well. But it's sort of like being nice to our users and to the community.
It's like, you know, feels like the right thing to do here. And then, you know, we also, we will also try like other models, right? So, you know, we tried them in the same harness and, you know, it's just all good. And then this is also often like this optionality is very important to companies that we work with. No, yeah.
And this is something that, you know, we absolutely lean into this last point. I think, you know, as any serious company, you want to have optionality and you want to use a tool that gives you that optionality. But I kind of appreciate it because I feel to me like it's kind of honest, like, look, like it forces the whole company to be to compete the best in everywhere in the model layer and the harness layer with open source, which is what models and it kind of like doesn't, doesn't allow you to like kick back and say like, all right, we're done. We can hang back for a little bit for now. Yeah.
I want us to win users by having, you know, the best models, the most efficient models, the best product. And then, you know, if we do all of these things, it's like, we're going to have a good time if we sort of like force you to use the product because, you know, this one thing, it's just like, then I don't think that will attract, you know, the very best people to work on this product either. And so, you know, we're doing our best work here. We care a lot about the experience. It should feel delightful, you know, like it doesn't.
In respect of the model that powers it, it should feel delightful. I love the idea of winning based on merit, not based on lock-in. And this is a perfect time to mention our season sponsor entire also play by the same rules. Like it or not, get is becoming a bottleneck for modern agent heavy software development. That's our creating more code with agents.
These agents are pushing more code, many devs are running more parallel agents. These are pushing even more code. GitHub is clearly struggling to keep up and has frequent outages. So what's the solution? Entire was founded by GitHub last CEO, Toma Stumpka, and he rebuilt get hosting for the agent to get up from scratch.
Entire was built to be very fast and to have your repos regionally close to you, to reduce latency, allowing for fleets of agents to push in parallel. So numbers they published. Entire can handle 418 pushes per second. That's up to 89 times faster than every competitor on the market. When GitHub is down, you can still keep working and you don't even need to migrate away from GitHub.
You just sign up to Entire and the platform mirrors your repo. And one more neat thing. Have you ever wondered what prompt resulted in this specific code being generated? I find that the prompt and conversation with the agent carries more information than the peer itself, at least for me. Entire captures all the prompt history with your agent right in the repo, easy to check back.
And has a pretty innovative UI to show all of this. If you're looking for a git hosting that works, even when GitHub is down, head to Entire.io slash pragmatic, install the CLI and mirror your repo with a click. I've already done it. Oh, and did I mention that it works with any agent and its open source? I'd also like to mention our season sponsor, Anticists.
Teebo talks about how the experience of the software you use should feel delightful. The life full includes no annoying bugs. But when you're using agents to write your code, how do you watch your shipping bugs? Reviewing every line of code is becoming a challenge with the amount of code that agents generate, which is why Anticists goes well beyond code review. Anticists runs your whole system in a hostile simulation.
This simulation includes both targeted testing and fast testing. By running this simulation, it finds every bug before your users do. And because the simulation is fully deterministic, it doesn't only find bugs, it gives you a perfect reproduction of every issue, which makes it much easier to fix issues. The first thing I thought when I heard about Anticists is that automated bug discovery and fully deterministic testing sounds like science fiction, but it's actually hardcore engineering under the hood. Jane Street, fly.io on the Etsy community shift agent written code with full confidence because they know it's been verified by Anticists.
To see more case studies and details, head to Anticists.com slash pragmatic. And with this, let's get back to Tebow and why competition between tools is great. Yeah. And I think as an engineer, like I always see that whenever there's competition, as someone who's using tools, it's always amazing. Like I remember like when Microsoft had with JetBrains with ID Wars, and then there's a clouds battling with each other with all the features.
And now, of course, we have the harnesses, we have the models. And as a user is great because now we have more choice. They just develop faster, I guess our voice gets a bit better. So it's great to hear. Speaking of the harness, can you tell me how it works today in the sense of like when I start a codec's task, does it run always on my machine?
Does it choose the cloud? Does it use a sandbox? And how do I control this or know this or how much do I know about this as an engineer? Yes. So by default, it runs in, it runs sandboxed.
It everything that if there is like a command that should run with additional permission inside of the sandbox, it will ask you as a user for permission. But everything, every tool execution happens within the sandbox by default. And it runs entirely on the machine, your local machine. And this has been the case for a more than a year now, but it is something that is evolving and shifting where you can select to run this in the cloud, which then runs in like a managed VM where it's the same VM that you get through Chatshope. Your work.
And you can sort of like inspect it, but like it runs in a cate container. It's like a secure environment. And so everything runs inside of that VM and it doesn't run on your machine. And then the only thing that happens in your machine is like the, your input and then the streaming back of the output. And so that obviously then is like much nicer on your CPU and your machine.
You can scale much, much, much more. And this is like just a step. It's going to be much more seamless in the future to like, you know, use cloud machines and then you know, maybe have a combination of like partial execution on your laptop, partial execution on cloud machines. And really the thing that we're thinking about that is very natural is as models just get better and more capable, they can leverage so much more compute and many more resources than are available on your local machine. And so it would be a constraint at some point to just limit execution on your local machine.
But one thing that is great about it's not running locally. And I think the reason I love it when it runs locally, of course, is the pain because if I'm doing some work, it's like, you know, I have several agents is eating CPU. If I want to close my laptop, I cannot leave it like half open. Right. When I was in one of the offices of an AI company, I had to hop open and they're like, are you running agents?
I'm like, yeah, I have one right. I get it. But the reason I do it because I have my local tools. I have my local Postgres database. I have my, my this and that.
How are you thinking about the cloud is amazing, but it doesn't have this setup or it's just a pain to set it up. Are you thinking or are you experimenting with making the setups? And I'm kind of reminded of a topic that we talked about pre which is cloud development environments in like 2022, 23 of their hot. And then we talked about AI more, but yes, I think outside of large tech companies, like cloud Dev boxes really never took off because there's a very big upfront cost. And then you need to pay like a maintenance cost as well.
And you just, you know, don't benefit from it as like a solo developer or like a small team with the level of capabilities that we have in agents now is like the setup almost is free, right? So like this, the setup cost and this maintenance cost is like, if your agent is capable of doing it, you know, you should just do it for you. So for example, if you're saying like, Hey, you know, I have like, I have my local deep light or I have a local server and MCPs and whatnot, it's like, how hard is it to actually configure exactly the same setup and keep it in sync on a cloud that box? Well, maybe it's not that hard if the model just does it for you. And so I think we're going to see a resurgence of, you know, fully cloud orchestrated machines, which then frees you from your laptop.
Right. It's like one thing that you know, ton of success with your chat with your work is like, it's just available on your mobile. I started my agent dictating a bunch of tasks into it next to the coffee and it just does it has access to my calendar. It has access to my email. It has that access to Slack.
And it's just so awesome to be able to walk around and, you know, get stuff done without having to, you know, carry my laptop everywhere. And I think it's the same thing. You know, we ship like, critics remote where, you know, execution is like still happening on your laptop, but it would be wonderful if, you know, you didn't have to keep your laptop open. Can you tell me, but on how in the past, how did you improve codecs? Because I remember when I first used codecs, this was one of the early versions, you know, like you could talk to it.
It did stuff, but for example, I said like, all right, make this change and it did that change and I had unit tests and it didn't run it. And then later, a few months later, I don't know exactly when it just started to run it automatically. Were these things, did you improve the, you know, the script that runs, you know, the instructions, I'm not sure how exactly you call the bootstrapping script or whatever that is, is it improving the model? Like as a dev, how can I imagine you making each version better between the harness and then between the model and like, what's the connection between the two? Yeah, this is a good question.
So the harness in a sense is always a little bit ahead of the model. Oh, really? How so? Oh, what I mean by that is that you have the model, it's capable of certain things, but then you set it up with like a couple of crutches so that you can actually do the thing to a level of reliability and in a way that is like efficient and also with the behavior that you expect as a user and sort of like, that's the role of the harness, right? Like, you know, provide guardrails, like safety, make it more efficient, make it more like, steerable, controllable.
And then the harness usually is also responsible for, you know, what we call like the developer message, which is sort of like infected in the context at the start of each turn. And so that affects obviously, like the purpose of that is to affect the behavior of the agent throughout, throughout the turn. A lot of what you have is like the result of the harness and the model. Initially, like, maybe you're like, Oh, it doesn't run tests. So, you know, you have to remind it to run tests.
And then, you know, we train a battle model that is just, you know, capable of like better reflecting on what is it that you really want when you ask for something. And then, you know, you don't actually have to tell it anymore. So over time, what we see is like the system, the developer message drinks and then the harness also shrinks. Inside of the codecs, seem to have specific goals, you say, like, all right, right now the codecs as a harness and model combined is not very good at this. Or it's kind of doing silly mistakes here.
How can I imagine how, as the engineering team, how you're working on the next, you know, version of, of COVID-19? Because the thing that I don't really get as a, as a dev is like, okay, there's a model which to me is a magical thing, which will get better. Of course, I'm sure you have some feedback channels, but also the hardness, which is the tools that you're building. Like that's probably what the team is responsible for. How do you set even your goals, right?
Like in traditional software, you'll be like, we will be able to feature and you build that feature because you know how to do it. But it feels a bit more fuzzy to me, this, this whole process. Yeah, and it's why we co-design most things. And it's a process where it's a collaboration between research and the engineering team, like primarily building the core, the core agent harness. It's always a, it's always a question of like, okay, we see today that, you know, we are very good at this, but we're not very good at this.
And you know, we have a desire to do like another thing because it would be a very cool product feature. And then, you know, we always look at it as like, okay, this should just be like a harness change or should just be a model change. And if it's a model change, like how soon can we have it? And we have it in a month, can we have it in, you know, three months, six months, and we sort of like work through that. And then depending on, you know, how soon we can fix it in the model at which level of training, then we might decide to not even do something in the harness at all and not just wait for the model to solve it.
You know, it's agents all the way, right? So we use agents to analyze like a lot of the feedback, to like, you know, come up with themes, you know, to just help us have these conversations and decide on priorities, but we analyze it across all of coding, we analyze it across like all of like, on the other domains, like finance, comms, marketing, you know, all the things where our users are using these agents nowadays. And there's like, you know, sub categories within those. And then we roughly know, like, you know, how well we perform and then we're always pushing different here. And there's a thing that is interesting is like, as we make, you know, as our pre-training model gets better, as we make the overall model better, like the whole thing lifts up, but then there are sometimes things that we pay a little bit more attention to.
You mentioned, you know, analyze agents all the way. Can we talk about the software development life cycle on codecs in the sense of whenever a new engineer joins a team, any team, it's like, okay, how are things done here? And you know, back pre AI, it would have been you join the company like Uber or Google, and they would tell you like, cool, the way it works is we have an idea or the PM has an idea, we make a plan, we get together, we do some estimations, we break up the work, we code the work, we do test, we do code reviews, we release, we do feature flags, and then, you know, we were on call that, you know, that's how it used to be when someone joins the codecs team, you know, they, they've been contributing to the open source part. But what do you tell them? How do things get done here?
If they're like a brand of total newbie. I introduced them to great people. Um, and then the thing that they're hearing the most about like when they have a question is like, have you asked good X? Um, and good X is like, it's just by default, I'd open eyes, like plugged into everything. So it has access to Slack is access to all the documents, access to all the code.
And it still surprises new starters that you can basically ask it anything. And it will very often just like come up with like a really good response. Um, and so the easiest way to understand the state of a project or who's working on something or why decision was made is like critics know about it all internally. Um, so you just, you just use all of that. We do a lot of work in, you know, for that reason, we do a lot of work in public channels.
Um, we open up documents with like, you know, fairly broad permissions and so that, you know, everyone has access to this information as well. And so that, you know, your agent can go through things and like, you know, reasons, three things. And then, you know, we have a couple of other things that are just really, uh, very helpful for team productivity and team collaboration that we haven't released yet, but are going to comment like some of it that day. All of that just sort of like makes you very grounded and in tune with the rest of the team. Um, and allows you to like, you know, just very, very quickly, like understand the state of things and, and, and produce things yourself.
The general recommendation is just like, Hey, care about the user, care about the coherence of the product, care about the models and where they're going. Uh, if you're doing something and, you know, you're building like this pentasant lines of code, crutch to work around the model flaws, you know, you're probably doing the wrong thing. So we have a set of principles, um, but it's just really, um, so like a team culture and ethos at this point. And you know, it's just very much like, carries on, you know, when people join this, like through the rest of the team, just like teaching their roots. And then when I have an idea, I think it's a good idea.
I talk to a, I talk to a, maybe I talk with you as my colleagues, like here's a cool new feature I'm going to build is my first, first contribution or first major contribution to codex. How do I go about that? Obviously I go to them with codex. I obviously tested and make sure that it works from there on what's the process. Do you still have the concept of code review or AI code review, a verification of rolling out, a verifying of stage rolled out, you know, the things because code itself, it goes out to millions of people, like I just crossed a big 20 million, uh, active user mark.
But if it's chat GPC, then it also goes out to like, you know, a lot of bigger number of people. Yes. Um, but it's, it's surprisingly like a similar process, whether you ship on codex or chat to be, even though it goes out to like a billion, a billion, you know, active users and growing, you can ship a PR, um, you know, you can make a change and you don't get to chip like the next day or like even the same day. Um, and it just goes out to a billion users and it's fine. We just really instill a sense of ownership and care.
So you're like, people are very empowered to make changes, even large changes. The general thing that is being asked is like, evidence that it's going to be well received, evidence that is like a worthy addition, evidence that, you know, it's like it is worth maintaining over time. But also like the cost of maintenance is like just really, as, you know, gone and done significantly as well. So we, we think about these things slightly differently than, you know, say like two years ago or three years ago, the other thing is also, you know, we automate as much as possible. So like a lot of like a process of like code review and deploys and, you know, catching regressions is like, you know, all of that is like pretty much automate.
Uh, and so, you know, you get to just focus on really the idea and, you know, how it's going to help our users and you care about, you know, the coherence of it all is like the overall power of the agent, um, and making things better. And we don't, we have a long, long list of things that, you know, we sort of like a aspire to do and haven't gotten to yet. And then there's like the sort of like the north star direction, um, which is a delightful, simple to use personal AGI that, you know, knows everything about you, like, you know, that it needs to know has access to the right resources, can take, like, you know, sometimes risky actions on your behalf, but then, you know, you get like a push notification and then, you know, you can verify that. And it's like a thing that, you know, you deeply understand as a user, but also, you know, it knows about your schedule, it knows about your goals, it can be proactive.
And it should be like extremely natural. It should be something that you can control through like natural language, your voice, you know, like maybe it should understand, you know, your emotions, like if it has like a camera, it should be the most natural thing on earth. It's like, it should not be like, I think with 10, you know, different buttons and configurations, it's like AGI should be simple to use. You kind of mentioned just briefly the review, the code review, but I wanted to go back to it. You worked at Google on a product used by, you know, like hundreds of millions, which is Google Maps and Google is very well known for their culture of very strict code reviews.
They have like two layers of code reviews. There's a language correctness review and they've taken, I think they've really perfected across the industry for a long time and they do believe that it works and they use it. How do you think that part is changing specifically the human review? Because for a very long time, until maybe a year or two ago, I would have said your code review has all these benefits, knowledge sharing, the second pair of eyes, removing the bus factor because now someone else understands them with that person is out that they can jump in. Conversations are happening about architecture, not just a code, but now there's, you know, there's a lot more code.
And what was the value of code review? What, in what cases, and so on your team, because you guys are so ahead of this, where do you see humans still being or developers being involved in the review stage of valuable and where is it fine? Did you find it fine to hand it off to an agent? Yeah, the role of code review is changing. One of the early projects that I did on Connect was like working with research on developing a code review model that was going to be to a level where it can spot mistakes in logic and reasoning to a degree where it would require humans, like, you know, multiple, potentially multiple hours to catch the same level of mistake, because it requires like really digging, like, you know, three, four levels deep into like the dependencies and you don't understand that.
Maybe the documentation actually was wrong and like the implementation of like the third party depends on what you expected. And so therefore your invariants are not upheld. And these things, just like, you know, unless you're an expert in that library, you would know, and therefore you have a bug. And so we developed like these code review models and, you know, we released them and now they're like the same level of like capability and like ability to spot these mistakes by doing like deep verification or like just part of the main line models. Like when we benchmark them, it's like, they're like superhuman in code review.
And this is not just true for correctness. This is also true for security, for example, where they're capable of like reasoning across, you know, very, very complex things. And then, you know, coming up with like, hey, you know, you have a critical security vulnerability here, which is now mandatory across like all of OpenAI pull requests, like we block full requests from merging if, you know, we flag them with like a security issue. And this is like all automatic. And the role of code review now is like, I think it was always about correctness.
It was always about, you know, ensuring that things work. But it was also sort of like a little ritual for information exchange and, you know, bringing people on the same page and like, you know, encouraging like a discussion, which ideally would have happened before, but sometimes it just only happens like around the code because once it merges, it's just actually runs in production and it's doing stuff. And then you have to maintain it. So there's like this social aspect to it as well. And so I think all of it is changing, like the correctness, the cyber, the security is like, I think that will be automated.
Really what we see and I see is there's a sort of a really discussion around the intent that takes place around the pull requests. It's like, what are you even trying to do? And is that a right thing to attempt to do? I think you can have that discussion outside of the pull requests. It doesn't have to be around good.
So maybe this helps crystallize the, you know, like where a discussion needs to happen versus where we did it because maybe we didn't have the type of tool that we have right now. Yeah, I think this is going to change. And it was like a forcing function because you have to have that discussion where it's good to have that discussion before you merge it and it becomes production code. But I think there are other ways to have these discussions and, you know, design things together and make sure that the intent is good. And then the code doesn't matter as much.
And it's interesting because when I think back of all my code reviews, like, of course, I have like memories where like it was great. We had a good discussion where I learned something really interesting, but a bunch of times, honestly, it was such a pain in the past. Like, I was trying to get my stuff. You're paying. Hey, could you read my code and like, no, right now I'm busy.
No, I really need this to unblock me and then you context switch. And then I feel it's always been like good and bad, right? So I feel whatever we do, there will be always upsides and downsides. But they're now they're just moving. So I guess one upside is as an engineer, you might have to not give your attention to just kind of basic stuff that doesn't need your input per se.
Yes, it's a time. And progressively what we're going to see is also that you have an agreement on, you know, the box and the overall contract of what it's supposed to do. And then, you know, what is inside the box, as long as you have like strict guarantees in terms of resource utilization, data access, security, these kinds of things, it's like what happens inside the box is, you know, it could be literally anything. You just like don't really need to care. And like really what you need to agree on is like, what does the box actually do and what are the invariants that must be satisfied?
And I think that is then worth, you know, having like a really good conversation on, you know, maybe assisted by your favorite agent. But then once you have that and you have that understanding, it's just like changing anything within the box is like, you know, doesn't require for this discussion. And it's like, you know, just really preserves your attention. The cost of maintenance has gone down. You know, maintenance is always such a topic whenever we build something inside of all these companies, like Google, Uber, even startups, like building was the fun part, but then maintenance was painful.
And that's when we learned like, okay, it was not we're building it, et cetera. Inside of codecs and open AI, what do you see maintenance becoming cheaper changing in terms of instead of what you're building, what the ambition is, the, I guess custom tooling, those kinds of things. Maintenance is really like, like a tax that you pay over time just to keep exciting and it's always been necessary. Well, it can't be necessary, but where I think it changes, it's like a lot of it is just going to be automated. So, you know, it's like, okay, you have to have this third party dependencies, like you need to upgrade the version.
I'm always like, oh, yeah, you can fully automate this, you know, if you have good change log and, you know, and the code is well documented and like, you know, in the market, just like reasons where you can just like blast through your quick days do it in a couple of hours and, you know, previously you would have like sort of funded on it because it's not the most fun thing to do, but it's actually really important for your business. So it's like really important for your project, you know, especially for security vulnerabilities, you want to stay up to date, right? You want to apply, you know, all these patches. And I think that's just going to be fully automated. So a large part of like maintenance, it just kind of comes for free, right?
And then I think it's awesome to also think about before, you know, when you want to just completely react, you have to do like a new architecture because you're trying to make space for like a new, you know, different kind of trade offs or you have a new understanding of like the workload or you're trying to fit a new feature and like, I don't really realize like your current system is just very limiting and you need to completely rearchitecture it. That was like a really, really tough thing to ever write. So, you know, sometimes like multiple years. And I think this is also like super, super accelerated now. So like the cost of mistakes, you know, I would say like always going down, but then at the same time, the good old rules, I would say of stuff, you know, having good abstractions that really help, like, you know, going back to this, like having the box within variance, like, you know, if you sort of like draw the right shape, you're going to be able to change things much more quickly within the box and like not affect the rest of the services or the rest of your infrastructure.
And I think it's important. It's important to design for a very quick iteration and change. I remember when I talked with Peter Schoenberger, that was before he joined OpenAI, but about OpenClaw and how he thinks about it. Like, you know, he told me that he doesn't read the code, but he kept thinking about like, I could see that he's holding the architecture in his head and he was telling me how he rearchitects a lot and he thinks about how to make it modular, how to allow a hundred contributors to each build their thing without stepping on each other's toes. So I'm hearing what you're saying that this care, this planning, this structuring has become maybe just a lot more important to like, which was something back in the day, you know, it was like the architecture to staff, engineer or experience folks were doing this thing and other engineers around the world kind of building those smaller parts.
But it sounds like now all engineers need to be aware of when you're building yourself or right in plan for it. Yeah. And the two models are getting better and better at this as well, like, you know, thinking about long-term maintenance and like good architecture. And like, this is like a natural sort of like next step, right? It's like not just about code quality in the sense of like, oh, is this code clean within this file?
But you know, is this architecture actually correct to reduce maintenance burden over time and make space for like future product or future extensions or changes and just really does active like, you know, engineering over time. That's kind of like something that models are starting to become capable of thinking about. Well, I think it's just kind of fascinating to understand that the software that we're building is just going through the lifecycle much, much faster. You know, before you had, you know, you were scaling it, you were starting it, you know, maybe as like a small team of, you know, yourself, maybe a couple of engineers and then you would add engineers like slowly, and then, you know, maybe after a year, you know, like if it's very, very successful, you would have 50 engineers on it or like a hundred engineers on it, you would have time to see it coming, you would have time to see, you know, the humans on board and you know, you can think about the documentation of that stuff.
But now it's just sort of like that explosion of like, you know, suddenly you have like a hundred agents contributing to this thing is like, you know, that can happen like in a weekend. And so, you know, you're just going into it at, you know, major, major speed compared to before. Okay, but how do you and the folks that are opening, I like deal with it, does not mess with your mind. Like, you know what I mean? In the sense of like you've been in this business for quite some time now, like decades or well over.
And there was a pace that we kind of got used to and obviously it's now a lot faster, but how do you get your head around the fact that a it's faster B, the stuff that you've been doing a year ago, right now, we're not doing because now the model is good at it. And you know, like how do you kind of reconcile that? Because I'm sure there's stuff that you've been really good at related to software that now you can hand off to the agent. Do not get a little bit of sting. You know, we talk about it's singing for your features to be in open source, but it can also say that I've been really good at like refactoring or right now, maybe architecture, but maybe the model will be really good at that.
And now I'm like, okay, time. Like I'm glad, but also like, it would have been nice for me to do that. Yeah, I think there's like a craft aspect to it, which occasionally I still, you know, pull up an editor and like write some code and it's just like, it feels nice and it's sort of like iPhone memories of late nights sitting in them and just like cranking it out, you know, drinking. Go zero and yeah, just not have to think about anything else other than like a problem in front of me. But really, I think it's all about being in the flow and solving problems.
And what I find is like, you know, folks here and also like everyone I talk to is just like adapting very quickly. And I think if you have a mindset where it's all about code is a tool to solve problems and you can solve so many more problems. It's like before you want to benchmark something and you weren't quite sure where you were going to net out. It's like, you can just do it. It's going to take you like no more than 30 seconds, you know, to launch something in the background and you know, get proper number and be able to do like a better trade off.
It makes it, it should make you a better engineer if you just really care about, you know, the outcome and the system working well. And so what it allows us to do it, it allows us to run, you know, our inference much more efficiently. It allows us to, you know, get like much more like effective compute and, you know, deploy that to the world. And so like everyone's just like very focused on that and solving important problems at the speed that was not possible before. And like I haven't yet, you know, encountered someone who's like, oh, that's not, that's not good.
That's not fun. Do I understand correctly that it sounds like if you have ambitious problems, if you have way more problems than what you can solve today or tomorrow or the next week, sounds like this is not really a problem. Cause when, you know, you get more efficient somewhere, you keep going, which is a lot of startups, right? Like startups always weigh more ambitious than what they're able to do. I, we're not, we're not out of problems for sure.
Right. So, and I don't think we will before it while we have a long, long road ahead of us in terms of like, mathematical breakthroughs, scientific breakthroughs, you know, making the world a better place, like just really billing for humans and solving the most important problems that everyone is facing and just doing it in a deeply human way. That's what we're here for also just going back to coding and like, you know, these late nights is like, I think there's like, it's also like maybe like a glammers version of it. Just like, I also had very, a lot of the nights where I was trying to refactor something. Um, and, you know, it's like, it would be like three hours deep into the refactor and then realize like, actually this is a dead end.
Uh, and I must restart from scratch. And it was like very frustrating. Um, and so it's like, there was like, there are like these very, very fun times that there's also a time where it's like, it doesn't compile and yours was like, why, it's not compiling yet. Like, I'm sure you have the time where you go, you go later is now super late. You need to go to bed because you need to get some sleep and then you can't really sleep.
And you have this thing where like you, you have some, some tasks that is halfway and it upsets you. Sometimes I remember dreaming about the code as well. And I guess one thing I don't really have these days when I'm working on my, uh, software for my business is I don't really have something that is halfway because I can just tell it, do this and I can leave it at the state where it's kind of like, you know, done either finished, it's either working or it's, I have proved that it's failed, but it's interesting because you know, everything's sped up, right? Yeah. I, maybe like I do have, like what a lot of people doing, I do myself is like, you know, I have like sometimes like bigger questions that I'm asking myself, like, and I, you know, from conversations I've had during the day or like, I haven't yet, you know, just like, had the time to just look into it.
And so I, you know, I will send off put X to just like look at it overnight. And then I'm very excited to then wake up and look at the results. And so, you know, it's always like an exciting morning. Well, I feel there's an arch to doing long running tasks. And of course you can use the slash goal, which will go and run.
You know, that's also something that was recently added like a few months ago, right? The slash goal commands code X. Yeah. And back to, you know, maybe like the harness is a crutch, right? Is a, uh, uh, that goal was like necessary to allow, like, you know, to keep the model like on track on like a singular goal for a very long period of time.
And it's like, it allows the model to literally run for days or weeks. So it's like a really, really hard problem. But with the new generation of models, like what we're seeing is like, you know, you don't need slash goal anymore. You don't need a harness around it. You can just tell the model like, you know, hey, go and work for a week.
And you know, I will actually do it. Speaking of hard problems and the fact that you're not out of them, one of the interesting things that you shipped from the outside, it, I would say it was, you know, as an engineer, it was moderately interesting. Uh, is the, what you call the merge, which is code X appeared inside of chat GPC. And that really, I say that's, as engineers, it's kind of moderately interesting because we've been using code X, like, yeah, it's there. You can now open it in the chat GPC upgrade.
Like I just went there and I just immediately went to code X because I don't, I don't really use chat GPT in the app per se. But I talk with folks at OpenAI and people on your team and you're telling me, like there was a lot of preparation going on, a lot of engineering challenges. Can you give a sense of how big this project was, what you needed to do? And why was it difficult to pull off and how, you know, how did code X under tools help you get it done in ways that would have been hard before? Because since you've launched the merge, the numbers that you keep sharing of how many people use code X, it's like, it's going up way faster than before.
So I assume there's a big scale problem you've solved here. A lot of things were challenging with the merges. First of all, completely different stacks. Chat GPT is like fully managed cloud based, like, you know, you run everything on our, on our systems, we restore things like traditional, traditional way of like building things, built for scale, built for efficiency, connects fully local. And so the merges is really high.
How do you get the same, the same benefits and the same capabilities from this local coding agent and then build the product around it and build it in a way where it can benefit like a much, much broader pool of people, which is just also why, you know, all of this joined open EIs like to benefit like this very, very broad population across the world. And so it was a very exciting journey of like figuring out like how do we build a cloud version of this that in essence is capable of like very, very much the same things, but it's also a building way where, you know, we can serve it through like tens and hundreds and millions of millions of millions of users in a way that is still like efficient so that we can include it all the way into the plus plan.
The chance to work is essentially like running the full codecs harness in a cloud together with like a cloud computer. It's a very powerful machine actually, like people have sort of picked up on it and showed that, you know, what you can do, if you are creative with the prompt is that, you know, you can get, you can get a chat to be seen to like train another model in there, you know, there's some pretty well things, you know, you can get it to install blender and you know, do like 3D modeling is like very permissive. It has like internet access. It's like a powerful machine and then codecs just works on it. And this is like what we shift through casually work.
So a lot of system challenges, the team did it very quickly. Obviously, like, codecs helped, you know, to make it more efficient, look at and a lot of the infrastructure and then, you know, help resolve a lot of the little differences as well that, you know, had been occurring between, between codecs and chance to be the emerging plugins, architecture, you know, emerging library and like really, really working towards like unified system, which is really the goal is that you shouldn't feel like, you know, you can do something in products that you can't do in charge of your vice versa. Like what we're trying to build is like one unified product that gives you access to the same intelligence, but in the way that you want to use it. And so it was very fun as well because critics throughout the whole journey also acted as a journalist to sort of like document all the steps and the debates and the discussions that the teams were having.
And it was very animated debate, you know, how we should do it. And how we should name the thing and, you know, when to introduce it in what way and like what to merge into what there were like many different permutations considered. And so there's like a very fun journalistic element to it where we think we have a full recounting that good exit over time. And yeah, it's just it's kind of become known as well as like the toggle arc of OpenAI where, you know, we introduced like the work toggle, which there was also a lot of debate around of like, no, whether this was like the right thing. And then, you know, it's just like we kind of grew to just really like it.
But over time, we're going to merge things further. So it's like we're really headed into the direction of like full unification. And you know, we kind of do this as like a temporary state. Where, you know, you have like you have better, stronger capabilities when you're in work mode. But over time, we're bringing this, you know, all, all the way to like, you know, everyone that uses chance.
And how do you personally use codecs? Like what's your, what's your working setup in terms of agents, in terms of what you, what you manage with it. And related to this, I asked Peter Stainberg or what I should ask about you. And he said like you need to ask him, how do you deal with the fact that you involved with all these projects, your calendar is like Tetris, but usually you show up pretty cheerful. My daughter is fine.
Um, um, and it's just, I am capable of doing so many more things nowadays because I have the technology like codecs and I actually shifted a lot of like my, my, my work on, uh, on mobile using Tensority to work where whenever I have something that I want to take note of, I just like fired it off. I use dictation a lot. Um, whenever I have a question, instead of like writing it down to look into later or delegating to someone and just like fire it off in Tensority to work and I get like a report, it has like a whole bunch of like custom skills and custom instructions where it's not like very, very tailored to do like, you know, produce a kind of reports and slide decks and code explorations, you know, in the style that I can consume effectively.
And so every time I'm like between meetings or like, you know, you'll kind of like, see me like, I was like dictating to my phone. As I said before, it's just like, we do a lot of work in public channels. We have like a lot in Slack. We have a lot in, in, in Notion and Google Docs as well. And so there's pretty much like, there's no question really that I feel like I cannot ask that, you know, critics will be able to sort of like do at least a first pass of thinking through whether it is like public sentiment on a feature, um, looking at production logs for, you know, how much usage we have on a certain thing, making a list of things that we should deprecate because they're not getting traction, understanding what a certain team is up to.
It's like, any question I have, I can get an answer to it, you know, within 30 minutes. And so that's how you use it. I use it for everything. It's like my personal agent in like all the ways. And then oftentimes on, on weekends as well, I do some like code explorations or like I built some prototypes and I have fun, like sort of like imagining the future of the product in some ways.
And I do that with others on, on, on the teams. It's not always the same team. And it's just like, in one day, I can build things that I sort of like, I had it in my system, like it's like, I woke up on the, I'm just like, we should explore what it means to build this. And then I can just sort of express all of that and get like something in front of people in a day so that they can think through it and criticize it and hopefully get inspired by it. It's like, by no means, you know, we need to ship it, but it's more like, okay, I flush it out of my system and then, you know, I go on and like, no, the other thing.
So it's just like so, I know it's such a magical time and it's like so empowering. And as closing, what would your advice be for a software engineer, such an engineer, somebody built software who would want to get the skill set and the experience to have the opportunity to work at a place like the Codex team, like OpenAI or like an AI startup. So like, you know, just become this really great builder with these tools. Because the question that comes up with often like, should I start with the theory? How important are the basics?
Do I just get really good at using the tools? Yeah, I think there are two things that are important is deep, deep curiosity for how things work and an ability to like, you know, train yourself to understand things very quickly. And so it's, it is the case that things will continue to change, but people that do externally well, I'd open the eye or like, you know, people that just sort of like are able to like, grok a system quickly and like, you know, also dive into like a new code base and so like, no, it makes sense of it. But obviously like all of that is helped with agents nowadays, right? So there's like, there's so much information that you need to absorb and you know, being able to understand and reason through it.
A lot of that is asking good questions, really, about, you know, how do you think so work and just like going into like the five wise, which I think, you know, you can just kind of keep digging and digging and digging and you know, you're learning very, very fast through that. The other thing is being in tune with the community or, you know, the people that you're trying to solve a problem for is like, not everything is like solving a direct problem. Sometimes you're solving a problem that will be useful, you know, to like another group of people in the pursuit of like solving a problem for humans, but just being crisp about the taste or the needs or the requirements and being able to think clearly and like, you know, exercising through this clarity of thought feels really important to me. Like if you can't explain what you're trying to achieve, you can't explain your intent. If you don't have a tie to a community, if you don't have the taste, it's, it's, um, it's going to be much harder to do great work.
Awesome. T but well, thanks so much for this conversation. This was awesome. Thanks for having me. I've always wanted to get together with T but I'm glad that we finally made it happen.
I appreciate how T but not just the upsides open source, but also the downsides, most notably how competitors can coffee features you are just working on in the open right now and then ship it right before release and just how much this things plus you get a lot of low quality contributions that you still need to somehow deal with. Another interesting one was T but saying how the harness is always a step ahead of the model from the inside. The codecs team see their job as building clutches for the model with the harness, the tools and the set of instruction. And then the next version of the model will be trained to need fewer of these clutches. I'll be honest, as a dev, this sounds a little demotivating that the stuff I build in the next version of the model, it'll just know and we can get rid of it.
Plus I do suspect that it's not just about building these clutches, but also building tools that model will use and it's not like the next version of the model will reamend MCV protocol or scales or plugins. At least I hope not. I also enjoyed hearing what the merge merging chat GPC and codecs look like from the inside. It was merging a previously fully local coding agent in codecs into a managed cloud-based stack and doing it efficient enough so that it can be included in an opening ice $20 per month's plan when $20 is not all that much in terms of compute purchase. It was pretty amusing to hear how codec is self-acted as a journalist at the whole project as it was present in all the slack conversations and all the documents and so it could capture all the important debates and decisions.
I'm not going to lie, this part felt a little bit of a big better feel to it where the AI is always watching, but it could well become the new normal in startups in the future. I've not yet decided how I feel about this. And finally, I appreciated Tibo's advice for engineers to succeed, be curious, under-sadsense quickly and be in tune with the group you are building for. It's reassuring to hear from Tibo as well how much the fundamentals don't matter. Do check out the show notes below for deep dives on how codecs, clock code, and cursor were built and other related topics.
If you like what you heard, please hit a rating on a podcast player that you're using. It means a lot to me and to the show. Thanks and I'll see you in the next one.