A new direction for AI developer tooling (Friends)

It's Changelog and friends, a weekly talk show about MCP hot takes. Thanks as always to our partners at fly.io, the public cloud built for developers who ship. We love fly. You might too. Learn more about it at fly.io. Okay, let's talk. What's up friends? I'm here with Kyle Galbraith, co-founder and CEO of Depot. Depot is the only build platform looking to make your builds as fast as possible. But Kyle, this is an issue because GitHub Actions is the number one CI provider out there, but not everyone's a fan. Explain that.

I think when you're thinking about GitHub Actions, it's really quite jarring how you can have such a wildly popular CI provider, and yet it's lacking some of the basic functionality or tools that you need to actually be able to debug your builds or deployments. And so back in June, we essentially took a stab at that problem in particular with Depot's GitHub Action Runners. What we've observed over time is effectively GitHub Actions, when it comes to actually debugging a build, is pretty much useless. The job logs in GitHub Actions UI is pretty much where your dreams go to die. They're collapsed by default. They have no resource metrics. When jobs fail, you're essentially left playing detective.

working each little dropdown on each step in your job to figure out like, okay, where did this actually go wrong? And so what we set out to do with our own GitHub Actions of Observability is essentially we built a real observability solution around GitHub Actions. Okay, so how does it work? All of the logs by default for a job that runs on a Depot GitHub Action Runner, they're uncollapsed. You can search them. You can detect if there's been out-of-memory errors. You can see all of the resource contention that was happening on the runner. So you can see your CPU metrics, your memory metrics, not just at the top-level runner level, but all the way down to the individual processes running on the machine. And so for us, this is our take on the first step forward of actually building a real observability solution around GitHub Actions so that developers have real debugging tools to figure out what's going on in their builds. Okay, friends, you can learn more at depot.dev. Get a free trial, test it out, instantly make your builds faster. So cool.

Again, depot.dev. I got like the news, like I got the update in the podcast that there was like an Oxide event and you were there. Was that a thing? How do you do that where you go like to the place? Yeah, this was a first for us, I guess, because that's like an internal conference for their company. And obviously we are not internal to their company. So that's a first for us. But, you know, we've hit off with Brian Cantrell and with Steve Tuck, the co-founders, and they wanted us to experience the team and meet everybody. And we had been kind of ogling and gushing over how cool their server racks are for years.

I don't have enough money to buy one of their racks and neither does Adam. They're not going to make a homeland version despite Adam's incessant cries for them to have an affordable, maybe a half rack. And so we've always wanted to like see their hardware in real life. And so this opportunity presented itself. So we went, hung out, met a bunch of cool people and saw kind of inside their company what they are up to, which was, it was cool. It was different. Nice. Very nice. First time to Emeryville. You ever been to Emeryville, Oakland area in the Bay, Jose? I don't think so. You don't think so? I don't think so. Right across the street from Pixar.

like the pixar headquarters i actually didn't even know like pixar was around that area so i didn't either until i looked across the street and there was pixar you know isn't isn't like that area in auckland where you're had like some startups starting to move there is that the thing that you're saying or i'm getting things i don't know honestly i think so but we're outsiders you know we get invited to the valley uh from time to time yeah and eventually we say yes you know uh it's a cool experience i think i think it depends on the the different places we get invited to but i think it's a chance to explore the world and meet some cool people, put back the layers, tell some cool stories. So I favor the IRL. I think it's cool to do it a few times a year or as often as it makes sense. Some version of that. We usually go to all things open, but our schedule conflicted this year. What about you, Jose? You get out and see the people over? Yes. And I had kind of like, I was used to do a lot of that, especially at Elixir, like at the beginning, go to a bunch of different conferences and just talk about Elixir.

And then, of course, at some point it gets very exhausting. And then I ended up just kind of like, okay, I'm going to focus on the Elixir community from now on. And even the Elixir community, there's the adjacent Erlang community as well, that it's enough to keep a person busy. But now we've Tidewave and we are supporting, now we support Phoenix and Rails. We're working on Django and other frameworks. I have started to...

to like kind of go back, for example, to Ruby conferences. So last week I was at Uruko, which is one of my favorite conferences. I don't know if you are, you were Ruby folks, weren't you? Yeah. Yeah. So I'm familiar with that. I haven't been to it, but I'm familiar with most of the Ruby confs. Yeah. So just, I don't know if you're live already, but for the listeners, I talked about this in another place. What I really like about Uruko is that every year they, people say, look, I want to host on my CD. They do like a three minutes, five minutes presentation and people attending the event vote.

where it's going to be next year. And then it's usually somebody with like no experience organizing an event now has to like organize an event for like 500 people, right? 600 people. And it's probably very daunting, but I think it keeps like, it keeps it fresh and keeps it always like community centric, right? Because it's always moving around. So yeah, so I was at Uruku and then the Elixir events I'm going to, in two days, I'm going to the GoTo conferences in Copenhagen. So yeah, I'm kind of back on traveling mode for now.

You like that or do you just do it because you have to do it? I'm enjoying it right now because I think one of the things with everything that's happening around AI, right? And coding agents that they are like, nobody kind of knows where it's going. Some people tell you that it's going to go there, but nobody really knows, right? Like I think the CEO of Entropic made a prediction about like 90% in six months, six months have passed. It has not been 90% of code being written by coding agents. So I think like I'm enjoying a lot like this opportunity with like talking to different people and you are getting like a bunch of different takes and different ideas and things to explore.

It has been really fun just going out and talking to people. But I'm sure that I'm going to do it enough that at some point, like maybe six months, I'm like, okay, I had my fail. It's time to hibernate again and go back to the Elixir conferences. But right now it has been really fun. I agree. I think it's fun to step back for a while and become a recluse and enjoy your local world. And then to come out, you know, peek your head out from underneath the rock and see the people again. There's something invigorating and exciting about it. But when you're just constantly on that track of just like travel, travel, travel, conference, conference, conference.

it can tend to burn out. So I think everyone needs to step back, but then also step out and see some people because that's where the magic happens, isn't it, Adam? I mean, that's where the real relationships actually form. I think so. You know, the IRL is really where it's at. I heard that somewhere and I liked it. Then I experienced it and I was like, you know, just give me more, please. Non-stop. Well, trying to figure out where the AI thing is going.

Cloud 4.5 dropped today. I don't think anybody played with it yet. I have not. But, you know, better, stronger, faster. Still not writing all the code for us. I think I did use it today. But they said something like it can go 30 hours on a coding bender. I just thought, well, that's really good marketing because I have no idea if it's true or not. But I was like, that's a great way to describe what your thing can do. Which is more than I can last. I don't know, Adam, how long you can bend or Jose. I'm sure you've been on some benders in your life, but 30 hours. Holy cow, man. Yeah.

So that would be stronger. I'm trying to think of where it fits in the category of better, stronger, faster. Stronger, man. 30 hours straight. I think it doesn't lose context or something. I don't know. Did you read this? Maybe they are doing more things. So right now, like they do the auto-compactation of the chat. And context engineering is all the rage right now, right? But like they do the automatic compactation of the conversation, which is summarizing it. But something that they do is also that when the context is getting filled.

They just prune the tools output from the beginning. So there is like some files or some searches or commands you ran at the very beginning of the conversation. They prune that. And that also allows them to go for long without having to summarize stuff. Because when you summarize, there's always a chance that you are losing some data.

It's really something that I do a lot is that you can actually have conversations with the agents about this stuff. So a lot of people say like, oh, the agent doesn't tell me like, oh, I'm using this agent and this agent doesn't tell me which tools you have available. But you can always just ask the agent like which tools you have available. And then you can just say, look, invoke. So something that I do is like invoke like echo with A 100,000 times.

three times so i can force it to fill in the context name like what do you see now on that tool that was invoked and then it's it's like oh the two output disappeared it's like a surprise that stuff like itself like it now says this thing and you can You can trick it to crash very easily as well. So you're like, oh, why do you have this tool? Wouldn't you think this tool would be better? And then you just say, would this tool be better? It imagines that the tool exists. And it's like, yeah, it would be better. Let me try calling it. But it obviously doesn't exist. And then the agent crashes. So I like having those meta conversations and they get surprised or tripped up. Yeah, it's fun.

It's almost like talking to a kid. You know, it's just very easy to pull the wool over a kid's eyes and they'll just, they're just gullible because they don't have life's experience that we do. And you can have a lot of fun as long as you're, you know, keep it in good natured fun and not trying to actually trick a kid. But with your AI, who cares? It's a robot. Trick it all you want, Jose. You know, you did all kinds of stuff. Have you heard, we just learned this from Feroz, that people are actually using prompts in their malware now. So if you can get arbitrary code execution on someone's computer, for instance, this was in the case of NX, which is like a monorepo command line tool, and they hacked the NX.

npm package distributed some malware and if you're running the nx command and you're and you're infected in there was an actual prompt to ask cloud code to do stuff for it instead of like you know instead of coding it out yeah and it was really kind of smart because what they asked us to do was the things that are kind of fuzzy finding for humans which is like for programs i should say which is like find all the interesting files on this computer which of course you could have a list of where the interesting files are and you could like search certain things but you can but cloud code can just go do tool calls and read stuff and Just hand back a list of interesting hackable files, you know, like secrets and whatnot. Anyways, I just found that to be amusing. So even attackers are getting lazy to write their own code. Exactly. Like why? This is the promised land, isn't it? You don't have to code anymore when you're hacking someone's computer. Yeah, I wonder why that was the best route. Was it because of their laziness or their lack of desire to write that script or just because they were just trying to leverage, you know, a cloud code enabled developer's machine? Like, what do you think the true psychology of that choice was? I think they're just thinking this is the fastest way to the best result.

You know, just like most programmers, like what's the fastest way to the best result? Well, I could write a program. And besides, I only have so much stuff I can shove in. I'm assuming like the more stuff you put in, the more likely to be found. So maybe some compression is in there. But it's like if I just prompt something to scour your computer for interesting files, that's a lot. That's pretty good at it. That's a lot faster than me having to write a program that scours your computer for interesting files. That's my guess. I don't know. Jose, why do you think somebody might do that? Maybe they're just showing off. Yeah, I don't know. They just want to trick a computer, you know, they want to trick an AI. Yeah, they're nefarious deeds.

So, okay, you like to mess with them. How much value are you getting? Because a survey says that we're getting tons of value, but quantified research says that we're not. I don't know if you've read any of the research, but a lot of recent papers, a lot being at least more than one, have come out and said developers think that they're more productive with AI coding tools, but it's actually slowing them down. What are your thoughts, Jose? Well, I think I have many thoughts on this. The first one is that to nobody's surprise, we're really...

useless at estimating stuff. So of course we are. We've been proving that for years, haven't we? Yeah. So of course we are estimating things wrong. And I feel like the fact like people come with like this exaggerated, like, oh my God, like three times more productive or even twice. For me, it's just kind of pointless because if actually, if you're even like a third more productive, 33%, that's like kind of like massive. That's huge. Yeah. Right. And then I think people fail to consider like there are other studies where like developers, I don't remember the exact number. It's like we spend like

50% of the time coding, let's say, right? And then if you're using agents for coding, of course, how more productive you're going to be is going to be where you're using agents. And if you're only using agents for coding, you can only optimize that 50% and not all the other things. So all that said, yeah. And then the other thing is that we, a lot of people, they don't estimate, they don't consider the time that they lose.

when something doesn't work, right? Everybody's happy. Like, oh, I use the agent. It worked. I was super productive. But there are a lot of times where it's just, it's not productive. And then you ended up like trying to coerce it to do the right thing and then it doesn't do it. And then you quit, right? And then you try it again and then it works and you completely forgot about that bad experience. The bad experience is actually one of the reasons why I never liked the completion, like the AI completion suggestion, because I would read it. And if it's not what I want, it will always throw me out of my loop.

And like that time where I'm like, I read it and then I'm like, oh, damn, I lost my flow. Like, how do you measure that? Right. If you're only measuring like, oh, it was accepted, you know, two thirds of the time. But like the one third was so disruptive to me that, you know, like. So with all that said, I think that I get a benefit from it. You do. I think. I think. Everybody says caveats, but I still think I do. Citation needed, right? I joke that I would love like we could use the.

the wikipedia citation need it should like be like an html feature we should just be able to put that everywhere you know like in conversation after every sentence that i say yes yeah because the other thing is it's rigby is that samsung's thing silicon valley oh gosh sorry oh yeah no i'm gonna get that you're hoping jose is gonna get it weren't you no yeah yeah we can talk about silicon valley later but yeah so the Yeah, you just did the AI completion. In the reels. He lost his limit. Okay, he's back. So there are a couple of things that I do that I think, like, you have to find where it works and where it doesn't work. And of course, it's going to change as those things improve. So for example, I tried it a couple of times to help me work with Elixir-type system stuff, and it doesn't work. It's going to be useless. I'm not going to try again. Maybe six months, maybe in a year, things change enough that it can help me with that kind of work. But I don't feel it's there. But for example, when working with TideWave, because we support...

other web frameworks. I often implement the feature in Elixir. Tell the people what Tidewave is real quick so that the three of us know. Yes. So Tidewave is a coding agent for full stack web applications. So I'm going to summarize it. We can jump into it later. But the idea is to have a coding agent that is tightly integrated with our web framework. So we understand like what is on the DOM and how that maps to a template. It can coordinate the browser. So it gives it a really strong verification loop.

So as you ask it to build features, it can verify that features work. You can interact with the actual web page and ask changes on the page instead of asking for changes on the code. I have this whole idea that I think we should run coding agents on top of what we produce. So if I'm working on a library, what I produce is like API docs, fine, run that in an editor. But if I'm building a web application, I want to run the coding agent in the actual browser because I want it to understand what I produce, right? And I want to be able to interact with what I produce because if I can't do that, we are doing boring translation work.

all the time, like looking what happens in the screen, go to the editor, ask it to change things, right? And then the agent says, I'm done. You reload the page, there's an exception. You have to copy and paste the exception back to the agent. Like, you don't want to do this boring stuff, right? And I say, like, the data science folks, they were the first ones to notice that because they were the first ones to put the code agents inside notebooks, right? They're like, okay, that's run this thing inside a notebook because if it understands my variables, if it understands my cells, they're going to be more productive. But nobody caught up to that trend, right? We kind of regressed. We first put it in the editor and then we put it in the command line, right? Like, you know, so we should be going up, right? So...

That's TideWave. And I do think TideWave can help you like be more productive with AI because being able to allowing the agent to verify what it builds is going to make it so it builds better things, things that are guaranteed to work and are going to spend less time on that loop, right? So when I'm working on TideWave, like we support Phoenix, we support Rails, we are working on Django, Next.js and a couple others. I usually implement the feature in Elixir, tell the agent like, I implemented this feature in Elixir. Now do, then I go to the Rails project, implement the same thing. And I'm like, there are a couple of things that I tell. It's like, don't add tests, don't use mocks, right? So there are some facts in there, but- You say don't add tests and then you say don't use mocks. I mean, if it's not writing any tests, why is it using mocks? Sorry, don't add additional tests, sorry, than the ones I wrote. Because the Elixir PR is like, it's good. I wrote it, right? Right, it has a test in there. So it's copying those over. It has like the proper tests. Yeah, because it tends, I think my experience with CodingAgent for coding is way better than testing because testing stands.

Not in Elixir, but because I'm doing a lot of Ruby and Python, it tends to use mocks a lot and just writes a bunch of redundant tasks. So it's a whole separate discussion. But it's really good. When I ask it, get this PR here, translate to this repository, a lot of the time it's just perfect. It's done, it runs the task, it runs the linter, and I can just push it. I send a PR, people review. So that's really good.

So I think that that's one of the things you have to figure out where it works and where it doesn't. Take notes of that, right? And find the loops and tools that make it work for you. And then you can be, it's like any other tool. And I think AI has this particular problem that some people say like, oh, it's just magical. It's kind of like a lottery, a lottery in some sense. Like some people go try AI and because it's, you know, it's probabilistic, they get a bad experience and they're like, oh, this sucks. I want you to try it again because people come with the expectation that it's just going to work. And then some people, again, like you're trying for the first time.

by just the randomness of it, they have a good first experience and then they start investing on it and refining it, right? And that's the process you do have like to figure out what is there and what isn't. And then the other thing that I like, I tell people to do, which works really well for me is to, I don't correct the agent. I just like, if it does something wrong or if it's like 70, 80% good, I just go and finish it. That's fine. You code it for it. It depends. So I do two things. So imagine that I ask it, does this thing? And then I leave, I come back like, oh, this sucks.

I'm not asking like, oh, you were supposed to do this instead. Because often when it does something wrong, it's there in the context. It has a really. It's going to keep getting wrong. Yeah. And then when it fixes, it doesn't fix everything. So when it does something wrong, I usually go and like, I start a new chat. I just discard everything, right? Nobody's going to be upset. I just discard everything. I was like, okay, start again. But do this, this, and don't do that. I add a little bit more of context. And then if necessary, I start again. So you start fresh with additional little warnings or instructions. Yeah. Adam, you do the opposite, don't you? You never write the code. You just keep telling it to do stuff. Yeah, I don't leave.

Yeah, I think by and large, it's writing code. I can't really write myself anyway. So it's like, it's going to do a better job. But Jose is a better programmer than both of us. So he can just fix things. We're just different people, you know? Yeah. So what are you using it for? CLI tools, really. I'm having fun with like a Proxmox CLI where it instantiates like a virtual machine with a given cloud init image. And it's like a command line away, basically. It's cool. So I can spin up a new server.

immediately, essentially. I can package it as a server. You can share it with me as a Git repo. It's kind of cool. That, and I would say 7zarch, which is a, you know, 7z is the compression algorithm. So I was just working on a version of that as a CLI that's just cooler, basically, because 7z's, its existing command structure is just kind of like not a lot of fun. It's hard to remember. I can always forget it. It's highly configurable. And so I wrote something that was just more fun. So does it wrap it and then call it underneath the hood with specific flags? Yeah, essentially. It did that for a while. And so then it was like,

Uh, we essentially just like, we rebuilt, uh, something called lib7z, which is a wraparound. I think it is like rust7z2 is a crate out there. Okay. And so it's like, it actually acts as a library around 7z essentially. And then you can write a CLL on top of that because it's a library. So that's where it's currently at right now.

That's cool. So you don't have to actually shell out. You're actually re-implementing the functionality. Yeah, precisely. And you get a lot more data in that API as well. Like, you get a lot more granularity around, like, files and process and progress. And you can control a lot of the UX around the CLI that way. We deal with a lot of large files and folders. Yeah. So I'm just sort of a neighbor by archiving them very well. Archiving it to the best of his ability. Yeah.

Well, friends, it is time to let go of the old way of exploring your data. It's holding you back. But what exactly is the old way? Well, I'm here with Mark DePuey, co-founder and CEO of Fabi, a collaborative analytics platform designed to help you explore yourself. So Mark, tell me about this old way.

So the old way, Adam, if you're a product manager or a founder and you're trying to get insights from your data, you're wrestling with your Postgres instance or Snowflake or your spreadsheets. Or if you are, and you don't maybe even have the support of a data analyst or data scientist to help you with that work. Or if you are, for example, a data scientist or engineer or analyst, you're wrestling with a bunch of different tools, local Jupyter notebooks, Google CoLab, or even your legacy BI to try to build these dashboards that someone may or may not go and look at. And in this new way that we're building at Babi, We are creating this all-in-one environment where product managers and founders can very quickly go and explore data regardless of where it is, right? So it can be a spreadsheet, it can be an Airtable, it can be a Postgres, it's really easy to do everything from an ad hoc analysis to much more advanced state analysis if, again, you're more experienced. So with Python built-in right there and our AI assistant, you can move very quickly through advanced state analysis. And a really cool part is that you can go from ad hoc analysis and data science to publishing these as interactive data apps and dashboards, or better yet, at delivering insights as automated workflows to meet your stakeholders, whether they are in, say, Slack or email or spreadsheet.

If this is something that you're experiencing, if you're a founder or product manager trying to get more from your data or for your data team today, you're just underwater and feel like you're wrestling with your legacy BI tools and notebooks, come check out the new way and come try out Fabi. There you go. Well, friends, if you're trying to get more insights from your data, stop wrestling with it, start exploring it the new way with Fabi. Learn more and get started for free at fabi.ai. That's F-A-B-I dot A-I. Again, fabi.ai.

I also use coding agents for things that I'm not reviewing, particularly for like prototypes. And that part has been really fun because if you're working on a product, you have ideas of, wait, what could this product, which directions it could go in the future? But usually before you would like think about it, put on some notes, right? And then maybe if you're lucky in two, three months, like somebody from the team can take a look at it, give feedback, right? And now with agents, you can just say, okay, go for it, implement this thing, right? And so as I was saying, like I have this idea that coding agents should.

run on top of the thing that we produce, right? And we talked about TideWave Web that works for our applications. I talk about notebooks. Well, but if I'm working on a game, right, I want to have TideWave running in the game engine. If I'm building a mobile app, I need to know about the mobile device, simulators, right, and all this kind of stuff. So I was able to, like, I think for...

Four or five weekends straight, what I would do during the weekend is to come to the computer from time to time, see if the agent was working, and just have it build a different proof of concept of embedding TideWave somewhere completely different. Like, oh, what would a TideWave browser extension look like? Which capabilities we got from this? And doing that when I had to do these kind of things for other products, like we were doing that for Livebook. it would take a really long time to validate all those things. And I could very quickly explore something different, get the lessons learned and provide a way better blueprint for the team to work on. Do you run it in Yolo mode or whatever is the equivalent where it's just doing whatever it wants to do and you come back every once in a while?

Yeah. Yeah, totally. So do you have it on it? Have you considered like a notifier and text me when you're finished kind of thing? Otherwise, you keep coming back. Are you done yet? No, it's not finished. Oh, it's been done for two and a half hours, but I was watching TV. Yeah. So in this case, because it's the weekend, I don't care in the sense that I don't want to be also interrupted, right? Like it's not my priority. Gotcha. So when you feel like it, you go over and check it. Yeah. Otherwise, it's like, yeah. Otherwise, I'm using the notifications, right? Like I use that a lot and Tidewave and they all have notifications. And then I'm kind of like listening. I'm waiting for them.

Oh, they don't like push to your phone or anything? I don't think that. Tidewave doesn't. I don't think that. Then you don't have to wait and listen. You can be out on a walk or whatever and be like, oh, it's done.

Maybe you might. Yeah, but if I'm on a walk. Give it his next task, you know? Yeah, right. So it's funny because I talked to Chris McCord about this, right? And it's like, oh, maybe I am on a coffee. I'm out to get a coffee. And then I'm like, look, if I'm out to get a coffee, I'm out to get a coffee. You know, it's like, if it's done, I don't care. I'm out, you know? Right. Yeah, same. But I do. Maybe you lose three hours of productivity, man. I mean, all you gotta do is tell it to keep going. You know, trade-offs. I get it. It's the weekend. I like to unplug as well, but I don't do any of the stuff that you're talking about. I don't have anything coding for me over the weekend. So if I was, I'd at least want to like, you know.

Be a good babysitter, not a neglected, not a neglecting. Babysitter, but to each their own, I guess. So you're talking to Chris. It sounds like you and Chris, are you guys competitors now? I mean, doesn't Chris have Phoenix.new? And isn't this like, there can be only one, Jose. Right. So I, and that's why he's not calling me on the show anymore. No, I'm kidding. So no, we do talk, talk a lot about those things. And like, we are still balancing many ideas of each other. So the way I think about this is that I think there's a very easy way to separate those things is that Phoenix.new is remote. And I did not, so maybe we should go deeper in Tideway because there is a bunch of.

additional context here. So as I was saying, Tidewave is a coding agent for full stack applications, but the thing is that it runs on your machine. So one of my ideas is that we're looking at Bolt.new, Lovable.dev, and they have all those things where you can click around, ask it to do changes. But it's like they want to own your code. They want to be responsible for your code. And most of the time, it's front-end or for React apps. And then I'm like, I want that for my Phoenix app.

that I run on my machine. A lot of people are pushing, oh, AI and those app builders that are running on the cloud. TideWave is you are accessing localhost. So the way you would install it is that you would add the TideWave package. So today it's for Phoenix or Rails or in the future for next Django. So just install the package. After you install the package, you go to your application, localhost, whatever, 4,000, and you do a slash TideWave. And then the agent is running there.

like in the browser and your web app running on the side. And now you can do all those things. You can go to the inspector, click it and say, hey, I want you to, on top of this element, I want you to add a chart of the most listened podcasts in the last month, right? So you can be very UI driven and everything's running locally. So for the people who are fine, look, I want to have Phoenix.new be responsible for my code, for my deployments. I don't care about that. And I want that thing to do everything for me. Then go use Phoenix.new. I still think.

It also like owns the getting started experience. It is the best way of getting started with a Phoenix app, right? Just go put things in the prompt. It's going to build something for you that you can throw it out, right? And for me, I'm like, look, okay, I have my own thing. I already have my own infrastructure, my own development, you know, like cycle. And I want to incorporate all those tools into what I do every day. For me, it's like, you know, when there was like a trend, everybody was saying like, oh, you're all going to be developing on remote machines, you know, like, and then there was like those developer containers and that never really happened, right?

I remember we did the show, didn't we, Adam? We did the show with whoever it was at GitHub Codespaces. Cloud development environments, essentially. Yes, yes. Yeah, and also people use it. Yeah, people do do it, but they're on the fringe. But it's not like everybody, because people would say that everybody would use it, right? Like, why would you have a local machine, right? So I say it the same way. I want those tools for my framework and running on my machine. Okay, I am with you. I actually...

When I heard TideWave runs in the browser, I was like, another browser thing, Jose? Like they're all running in the browser, but actually it's different than that, right? It's in the browser because that's what your output of your web app goes. But it's in your local browser running against your local web server with your local environment and helping you build cool stuff right there, which is kind of how I developed now anyways. Whereas Phoenix.News was making me go into the browser and have a remote browser session, which I've always got excited about for the hour that we do the show. And then when I go back to my real life, I just don't want to do that. I want to be on my local machine. I always have. Maybe I always will. I'm getting old, so I'm getting stuck in my ways.

So that makes me like Tidewave a little bit more than when I first thought, oh, it's, it's, because one of my questions for you is going to be like, why the browser, you know? But it's because I didn't understand. Yeah. And, and, and the thing is, so we actually went through many possible designs. So over the last month you had.

We already had like, let's talk a little bit about the browser design. So like we already had for some time the Playwright MCP. So somebody may be listening to this and say, well, I can use VS Code with Copilot and install the Playwright MCP. Recently. Chrome DevTools MCP. Chrome, yeah, they released there. And I think yesterday, Courser Browse came out. What's that? It's just controlling Chrome. It is like the Playwright Puppeteer MCP. It's just building, right? And the issue with those tools is that it is a...

separate browser session is not the one that you are developing. So imagine, for example, that you are working on a project manager, and what you need to do is implement a feature for transferring a project between two organizations. So in order to implement this feature, you need to create a user, create two organizations, probably make sure that the user is admin on both organizations, create the project, and then you can transfer it. And then a lot of the times the MCP is going to get stuck just in this process. Like all the times MCP cannot create an account because create an account requires sending something to an email that the MCP doesn't have. So now we start writing like those back doors for tests. Like, so there's such, there's a big amount of work, right? And then the fact that we run in the browser, we're literally running in your browser session. So, because when you're going to develop the feature, right? You open up the, you already logged in your development version. You go to the page already. And when you're going to validate that the feature works, you already have all that set up.

So because it's running there in your session, everything that you do for development, the agent can do. And the agent is going to verify the things in front of you, not in a separate session. And then you can actually have a back and forth. Like if you're using the MCP, imagine like the agent, like, okay, let me test if it works. And then the MCP with the separate browser is running. And then you see a bug right when the thing is testing. How you're going to...

debug that because it's a separate browser. How are we going to click things and say, hey, you would have to go around, say, around this page, maybe there is a bug here. With TideWave, it is your browser. I think that's the most important thing. You can stop the testing, go to the TideWave inspector, there is a bug here, fix it. And we also go the next step, which is that we integrate with the web framework. We understand, like when you inspect like a DOM element, we know the DOM element and send to the agent, but we also know where in the template or which React component.

that thing came from. And we send that to the agent. So you don't have to do the manual working of figuring that out and do it to the agent. When there's an error page, we detect the error page of all the web frameworks we support, automatically feed that to the agent. So it's really meant to be like, look, it's like you, the agent, the browser, the web framework, like in a shared context, kind of everybody can see what the other is doing because otherwise it becomes our responsibility. You are the ones who are like getting information from all those places and pass it around. Sounds pretty cool, man. It does sound pretty cool. Bypassing a lot of that stuff. I mean, the fact that it has, you know, that, I mean, that's something I just,

i've always wanted i guess i mean you're going back and forth like that it's better to do right there real time i haven't played with it to know the ux really of that like when you're filling with let's say is it a button maybe it's not working properly what is the back and forth with the the experience can you speak to it can you type to it like what are some of the interfaces you can think of right so i think they are three ways that you are interacting with it. So one is the usual chat prompt, right? And with the difference that we know what is the page that you're currently looking at. So you can talk to the page in the sense that, so for example, imagine that you just boot up your dev instance, your database is empty. You can go to a page that is listing all the podcasts, like for changelogging to say, oh, this page is empty. Add some podcasts. It knows which page it is at. So it can find information from the controller or from the live view and then say, okay, that's the data I need to, like, it gives you an entry point.

So that's the chat. It has the contents of the page. The other one is the inspector. It's like the browser inspector, but so you can click it and then you can mouse over elements. We show the DOM element. We also show like which templates or which Phoenix template it came from. And then you can like click it to open your editor or you can click it to ask the agent to do something. And the other way that we interact with it is when we detect that something goes wrong, we just show up a pop-up like, oh, you want to fix it, right? And then you can just click a button and have it fixed for you.

I think as a human, those are the, I may be missing some, those are like the three. So it's a very classic like chat experience with a few like things on top, like inspector, the error. But I think a lot of the part that we shine is in giving more tools to the agent, right? So the agent can do everything that the coding agent can do, but it can also run JavaScript on the page. And that's how the agent can test that it implements something. So for example, one of the coolest features that we use TideWave to implement, like if you go to tidewave.ai today, we have videos.

in the homepage. And so I added like the YouTube URLs, or not YouTube, like the URLs. I added the video tags, right? And then I wanted to make it so as I was scrolling through the page, the video started to autoplay. So I asked about Tidewaves to implement this, which it can do. It's a straightforward feature. I can't do it, but I assume it's a straightforward feature for me. I didn't look at the code, but I'm sure it was pretty easy. Yeah, so it implemented the thing, right? It implemented the thing.

And then in order to test that it worked, it actually like reloaded the page. So this side wave, it wrote JavaScript code to reload the page and scroll to the first video. And then it runs on JavaScript to validate that the first video was playing, but not the other two. Then it automatically scrolled a little bit more. So the second video started playing, and then it ran some JavaScript to make sure that the second video was playing and not the other two, right? And I think that's the important part, because if you can see the agent doing that, because if the agent doesn't do that, there's a chance they get it wrong, right? And then if they get it wrong, who is paying the price to fix it? It's you, because you are going to be the one who tests it, and then you have to go and tell it. Well, they're going to say your users, because you're going to push it out live. And then your users have to tell you if it's broken or not.

How do you limit it to the viewport? I assume the scrolling is either simulated or it's real, or it's simulating it so you think it's only scanning with you. No, it just runs JavaScript on the page. So what's in the viewport? So it's looking at what you're seeing, essentially. Yes, yes. It's running in your browser. There are a lot of complexities in there, but this part is as straightforward as it can be. It can control the page. Because that's the thing. People are coming up with all those different APIs to have the agent. There's an MCP with 30 different commands to control the page.

It knows JavaScript. It knows the DOM API. Just have it run things on the DOM. It knows what is, I don't know what is the command, but it knows what is the command to say, hey, scroll a little bit, right? It knows. So the only things like we have to intervene like very little, like so there's one of the things that it can't do, like resize the browser window. I think it's because browsers don't allow you to do that because of security concerns or something like that. So there are some things that we have to intervene and add like extra capabilities, but it's just running things on the page.

Well, friends, you don't have to be an AI expert to build something great with it. The reality is AI is here. And for a lot of teams, that brings uncertainty. And our friends at Miro recently surveyed over 8,000 knowledge workers. And while 76% believe AI can improve their role, most, more than half, still aren't sure when to use it. That is the exact gap that Miro is filling. And I've been using Miro from mapping out episode ideas to building out an entire new thesis. It's become one of the things I use to build out a creative engine. And now with Miro AI built in, it's even faster. We've turned brainstorms into structured plans, screenshots into wireframes.

and sticky notes, chaos into clarity, all on the same canvas. Now, you don't have to master prompts or add one more tool to your stack. The work you're already doing is the prompt. You can help your teams get great done with Miro. Check out Miro.com and find out how. That is Miro.com, M-I-R-O.com.

Can it take a, I wouldn't say a fixed width, but a desktop designed website and implement it? So it looks the way you want to look on a desktop. And can you say, make this a progressively, what's it called? Not progressive web enhancement. Responsive web design. There we go. Can you make this responsive for these six viewports or something? Right. Not yet. I knew it. I knew it. I got you.

Not because it's Jose's fault, because I don't think these things can do that anymore. It's hilarious, though. I love it. This pursuit of rightness. I'll try. I'll actually try it. Because I've been doing a lot of front end lately. I'm not good at it anymore. I'm learning. All the new tools are fancy and they're hard to use. I can't figure Clamp out. I mean, I've been using Clamp wrong for weeks now. Finally starting to get it to work. And none of these tools can do it either.

I play what I call LLM Russian roulette. So I take the same prompt and I'm usually like, hey, can you do this thing in SVG or whatever? I can try to accomplish stuff that I don't think is possible. And I thought it should be possible. It's the modern web, you know? And so I ask ChachiBT, I ask Claude, I ask, I'll even ask Grok if I get too angry. And then I'll ask Gemini and they all give me different responses that are all wrong. None of them can do it. I want one that just tells me, actually, Jared, that's not a thing that you can do. You know, like you can't do that with web technology. They're not going to do that because they want to make me happy. But I know that like that kind of stuff.

We're not there yet, man. I'm just doing way too much work in the browser as a human, I know. Here's how I would try to implement it. Okay. Okay. And let me know if that's an approach that you tried, because if that's an approach that you tried, then my solution obviously is not going to work. I'll let you know, trust me. Yeah. So I was talking about the resize. That's something that we identified recently. So we haven't implemented it. So it doesn't have currently the ability to resize, which means that it cannot validate responsive designs. Right. As simple as that. But what I would try doing would be.

is add the feature to resize and the feature to take a screenshot of the page, which there are some other complications because the browser's not allowed to do it for secret reasons as well. I know how to solve it. I'm just explaining why you're not going to have this feature tomorrow. There's some work. And then have it look at the screenshots and see if you can see things are good or bad. How do you think about this approach? In my experience, their ability to look at screenshots and decipher things is really bad. Like it's not there. Like they have vision, but it's not precise enough, you know? And so I haven't tried that specifically.

But I also don't think it's going to work. I'd love you to try and prove me wrong. I would love to be wrong. But in my experience, when you pass a screenshot or you say take a screenshot and then inspect the visual, nine times out of ten they're wrong. All of them. I wonder if we could use accessibility APIs.

Wait, what? This is Jose. I love the exchanges on you. He's such a problem solver that he can't help right now. He's like, let's debug this thing. So you already saw me getting off track with the AI suggestion with SightLive. So this is also a real-life nerd sniping happening right here. So yeah, what were you going to say? Accessibility APIs. If we could use accessibility APIs somehow to measure size of elements and what is visible, what is not. But maybe not. Right. Yeah, I don't know about that. I just get angry and I just do it myself.

Because it goes back to what I was saying in the sense that the way for us to eliminate the AI guessing is adding more verification tools. So if the browser had a way, if the browsers could tell me like, oh, the fonts here are too small, these things are clipping, that's why I was thinking about accessibility APIs. Because if the browser tells me that, then I can get that thing, which is going to be better than a screenshot, and send it to the agent. That might actually work. Right, but I don't know if this accessibility API exists, right? So that's why I'm... I'll tell you that it does exist. I'll give you the code.

I love when I have it, they produce SVG and I'm trying to get like a tapered border and all this kind of stuff. And they're like, here you go. And then they like tell me all the reasons why it's going to look good. And I put it in there and I put it in there and I'm like, dude, it looks like a bow tie. Like you just drew a bow tie. And it's, it's so far off that I have to laugh because I'm, otherwise I'm just going to cry and just be like, why am I even wasting my time with you guys? So there's certain things where they just have these inadequacies and they're all inadequate at this point in my experience. I haven't done 4.5 yet. So maybe after this call, I'll go see if Claude can do this. But, um, I don't know. I don't feel like I'm pushing the envelope. I feel like I'm a.

kind of just an intrepid person trying to get something done and thinking that you can do things that maybe you just can't even do in the browser right now. But I think being able to develop out a simple, I'm not gonna say fixed width, desktop styled website and say, make this responsive, like that should just be a thing. Don't you think? If you build that inside wave, Jose, I mean, people are going to line up with their money. I think so. I mean, because that's the thing. Like, so I don't want to do that work, right? So I hope that I can do it. Yeah, totally. That's the thing. Like, I can also do it, but it's just slower and tedious. And that's what the promise is. We don't have to do this stuff anymore. And I'm not a good at it anymore. It's just, it's guess and check. I got a guess and check. Oh, it's still too big. Now it's too little.

All right, I'm done complaining. Adam, take us somewhere else. I'm just airing my grievances. Well, you know, one thing I was going to go back to was, Josie, I think one thing you were mentioning was how when you scroll TideWave.ai, as you see these movies come in, I'm actually, you know, back I think maybe 15 minutes potentially, but you were describing this page here. And now that I've actually caught up and I've scrolled it, maybe that's where we can go. It's like, what was the aha moment here when you did this? Because you said you were kind of going back and forth. Did you not do any of this design yourself? Did you just sort of prompt it? What was the experience like for getting this page to be like this? Yeah, no, for this page in...

So for this page in particular, we were just doing the design of the page and then we knew we wanted to add all the scrolling and then we just asked it to do it and then it did it, right? And I think what was surprising about that is because, I mean, it's obvious, but that's exactly how- The autoplay of the videos was key, right? Autoplay video, but it wasn't the autoplay, it was how it tested itself to know that it got the autoplay right. Yes, and that's exactly how we would test it. I mean, it's obvious that the way you test the autoplay scrolling is by scrolling. You scroll and you watch it autoplay and you make sure the other ones aren't, but it's just running JavaScript. But it's really nice to see it happening by itself.

Right. And then it goes back to, to other stuff. Like, like Tidewave has access to everything. Another way that I like to phrase this is we imagine like you're working with somebody and somebody sends up a request and then you open up like the work they did in the browser. And then they're like, wait, this looks bad. And then you go back to the person like, did you look at it in the browser? Just try it out. And then like the person says like, no. And then we'll be like, what? Like you have to test things in the browser. Right. And that, or like I use the repo all the time.

as well, right? It helps me develop a lot. But we are asking coding agents to develop without the proper browser, without a repo. So TideWave gives all those things as well, right? So, oh yeah, I was talking about like the, you asked about what are the user tools and I started talking about the agentic tools. So one is coordinating the browser, but the other one is that we also give access to a repo running inside the web application because we use the repo for development. Why are we not giving one to the agent? Like I would be a worse developer if I didn't have a repo, right? And then we have MCPs for like, oh, you can install an MCP to talk to Postgres.

But I'm like, my web application already knows how to talk to the database. It already has all the credentials in there. Why are you asking me to configure a separate thing? So a lot of the times, it builds a feature, and then it tests the feature in the browser, and then it does a database query to make sure that the change also happened in the database. So that's kind of, yeah, so we're going back 15 minutes, but that's closing the loop of like, what are the tools that the agent have? And the whole purpose is to make sure they are producing something that is...

That is really good. And I'm not going to waste my time telling it obvious things like, oh, the video actually doesn't play. Oh, the change was not actually saved to the database. You mentioned, I think one thing you mentioned there was MCP service. Have you, Jared, messed with MCP service at all? I really haven't.

Personally, just figmas. Yeah. And I told you my experience with that, which was right. Nobody's fault except for the state of the art is not quite needed. But I imagine you're playing with them heavily. How exactly does that fit into your flow? Because from understand, it just adds more tooling to the context window, which is already kind of small. And so we're always battling that, you know, that auto compression or just having to refresh the entire chat whenever you feel like, I suppose. How do you work in those kind of tools into your workflows without, I guess, bloating the context? I actually have a hot taking here.

It's not a unique hot take. So to answer your question, which is going to kind of reveal the hot take, is that... He's just teasing it. He's not going to reveal this. He's just teasing the hot take. Come on, man. Come set it up, Jose, and give us the hot take.

Almost all of our APIs is write code. Oh, you can execute code in the context of the web application. You can execute code in the context of the web page. That's it. Because we are doing all this dance. Like, oh, I said about the database, right? Like, oh, I'm going to have an MCP for the database. No, my web application already knows how to talk to the database. Just use that. Oh, I want to have an MCP to talk to GitHub. And I'm like, well, I already have, like, I already logged in on GitHub in the browser. I already have the GitHub command line. Use that, right? Like for coding agents, we are even going as far as adding MCPs for documentation.

And then I'm like, why am I going to a separate website to get documentation? You are a coding agent. The code is in your machine. And usually with the code, you have documentation. Why don't you use the documentation that is there already on your machine with the exact version that you're using? Because sometimes you go to the remote server and then we get the documentation for Phoenix 1.8, but we are still on 1.7, right? So for me, like the answer for, oh, there are like too many, like the context thing is like, I'm going to have just a small amount of tools. And what those tools are going to do is that they can run code, right? And I'll let them...

do whatever they need to do. So trying to keep the amount of tools minimal and powerful. And like this, this take that, you know, like, oh, MCPs are too much. You probably just need code, right? That's kind of, I'm not the first one to say it, but I also think like MCPs, the user experience, like the developer experience around MCPs for coding agents is really poor.

I mean, to be fair, it's new. It's still evolving, right? It's probably six months old at this point. But like, so we have issue where, so one of the MCP2 that we're using, it works, it was working for GPT-5, but not for Gemini. And then we fixed it for Gemini and it broke for GPT-5. Like it, if the server disconnects, they can not reconnect again. Like there are all those sort of like annoying issues there. And then do you know about the Figma dev mode thing? So like, so there's an MCP in Figma dev mode. So you can run like Figma on your machine, like there's a desktop client.

And then I can go to Figma, inspect an element, click like a component that I wanted to implement, right? And then you know what the workflow is today? It's like I have to go to Figma, click on the component, and then I have to go to the agent and say, I have selected a component. Please implement it. And then when it's done, you have to redo it yourself.

Yeah. That's my experience. Oh, good job. Not good. Not good. It doesn't look like it's what it looks like. I already clicked the thing. Why do I have to go back and tell you that I already clicked? Because they're separate tools, right? They're distinctly different tools. You're meant to have a protocol for those things to communicate. I know you know the answer. I'm just saying it out loud. Whereas TideWave, it's all integrated. It's all integrated. And actually, I want to hold as much as possible with actually adding MCP support to TideWave because I think we will have better integrations.

if we do it by hand. So, for example, when I do Figma for TideWave, when you click on the Figma thing, we will know. And we just tell it, oh, you want to implement this? Oh, yeah, just click the button, right? Like, you don't have to type anything. Just click it. Now, is it going to do it right? Well, that's up to your model, right? Yeah, maybe we can do more. TideWave is just using whatever model you bring to it, basically? Yes. What we can do is that we can help it, like, you'll be able to click something on Figma and then click something on TideWave.

And then we'll be able to say, oh, you should implement this, and this is exactly where it is. So we can improve the experience there, right? But when we send all the information to the agent, if that's going to be ultimately better, and the agent will be able to validate that some things look good, like it did with the video auto-scrolling. So we are giving it more tools to verify that it did a better job than it just working blind. But ultimately, yeah, right? And I think that's going to be true for a lot of things. So there is a tool called Conductor that, for example, they added GitHub integration.

And one of the things they do is that they know which GitHub, they know which GitHub branch you're using. So in their GitHub integration, they know the comments dropping a PR for that branch and they automatically surface that in the UI. So you can ask the agent to solve a comment as somebody's commenting on GitHub. So like those sort of experiences doing through MCP, it's just, oh, you know, it's like, oh, get all the comments for me. Right. It's like, so I really think that.

For coding agents, I don't want to generalize this too much, but I think for coding agents, for a lot of the things you can build, like MCPs doesn't allow you to push information, right? So that's what I'm complaining about. GitHub should be able to push information for the MCP. Like, oh, there's this comment. Oh, I clicked this on Figma. It doesn't support that. And we're not talking about even the security issues. So I feel like...

Yeah, I want to give you a good package with everything. At the point you're telling users to go like, oh, just go and install those different MCPs, you kind of gave up on the developer experience, right? Because it's not there yet. I would tend to agree, I think, with that. How hot was that? Was way too much teasing for not too spicy, or? It's a lot of spice in there. A lot of spice. It's a variety of spices. There's some hedging around, you know, I just feel like you could have dropped it a little hotter, and then... I could, yeah. Yeah. Kanye could have gone ghost, you know, gone ghost.

And also I just tend to agree. I think MCP servers seems to be like a builder, like builder driven technology right now versus user driven. Like I feel like it's, it was like so quickly adopted by all the builders and as users were kind of like, were we asking for this necessarily? And can you, could you do it so that it was, I would say more transparent perhaps, or like maybe just user friendly for us as end users. But man, I've never seen an API or specification or a protocol.

get built out across the entire tech industry so fast. And we're talking like less than a year. Well, I mean, from their first announcement back in November, I believe, was when MCP was announced by the Anthropoc team like less than a year ago and nobody paid much attention to it for three months. And then all of a sudden in the spring-ish, it's like everybody just started building MCP servers. Like everybody. And then as end users, we're kind of like, did we ask for this? Or I don't know. I'm not sure what.

it's like you want to be first, I don't know, you want to be left out. I'm not sure why everybody just thought immediately we got to do this, but it was pretty interesting to behold that. You know, I don't know. I don't have a lot of context around MCP servers, but I don't really use any of them. But when I think of them is more like a CLI tool that's on the system already, rather than like, you know, pollute my context window with a tool that's an MCP server. Why not just have a tool?

on the system that you can use, not have to be an MCP server. Instead of using the GitHub MCP server, you might use the GitHub CLI to access data from GitHub? Is that what you're saying? Right, like if I already have GH installed and started authenticated, why not just use the tool versus some sort of MCP server that just is like in my context? Like why does that have to be in my session and configured? It's also instrumentation of tooling. It's a lot of ceremony, you know? It's a lot. But even if you say like, look, what if you don't have the GitHub command line tool? Yeah, have it, right? Like you can write Elixir, you can write JavaScript, you can write Python to talk to the API.

Then, of course, I think the authentication part of the MCP is interesting because if it had to write a tool, it would have to ask for your credentials somehow. So that part is good, but it feels like in certain ways, that's probably all we needed, a way for the agents to ask, which we have all of and other things, and a way for the agents to ask for your permission to talk to some API on your behalf. Because if it has the code, it could also do things like, oh, it can ask information, it can get the raw data from GitHub. than use whatever library to compute the information that you want, right? And give you a better result than trying to do with the MCP, getting plain text, and then maybe doing something interesting with that, right? Like we have those things that are really good at coding and we are sometimes like dumping things down to a text interface while they could write code. I like to think, one of the questions that I ask myself, like people are talking a lot about like personal devices, right? You're going to have our personal devices that are AI augmented and these kinds of things. And I say like, that thing needs to know how to run code, right? It's like...

Because how can you have some generic personal assistant that can do everything and everything cannot run code? Any assistant of mine has got to be able to run code. That's right. You better run code. I'm telling you, get out of here. It's like the first thing on the resume. Can this person run code? Well, yeah, I tend to agree. I think MCP is an interesting phenomenon and most widely adopted by builders technology that I can think of in history. So there it is. It's there now, but not necessarily. It didn't necessarily have to be there. And there you have it. Spicy. Spicy, Jose.

What else? I mean, Tidewave, you're trying to make a business out of this thing? You're trying to make a living? What are you trying to do? Yes. So it is a paid product. We consider a little bit, but then realize, well, everybody, this is an AI thing. It's a very rapidly changing landscape. So if you want to be able to keep up and to invest in on this and continue improving it and also support different kinds of frameworks, we need to find a sustainable way of doing that. So yeah, we'll see. The launch was pretty good.

We got a lot of people excited, but it also pointed out like, so today's like bring your own API key. And the feature that people ask the most for is cloud code support. So being able to bring like codecs and cloud code. And yeah, let's see. So right now, I like to, I think in the email, I sent you folks like my product history, my history of building products has been like cataloged by the changelog. Yeah, pretty much. We're doing our job there, like the changelog of Jose's products, you know? Yes, yes. It really is. Yeah, so there's Livebook, which is also running, right? And now Tidewave. And what about Elixir, man? Is it done or are you done with it or still working on it? Still working on it. I still, it changes.

So around now that there are a good amount of Tidewave things happening, it's fresh. I think we are about five weeks since we launched. So we just launched it. And you know, when you launch something like that, there's a lot of work, feedback and prioritizing. I think it's kind of like about half, half of my time on Elixir, half on Tidewave. But otherwise, like most of my work is still going to the Elixir type system and Elixir work. The other thing that I want to do, one of the things that going outside of Tidewave is an example, but going outside of Tidewave, one of the things that...

It makes me excited about AI because we can look at the tools and find ways to improve and build new developer tools. And I've been exploring some ideas around those areas. So I was saying tests. The tests that the coding agents write, I usually don't like them. They are redundant. And I think a lot of people don't pay attention to, or they use too many mocks. A lot of people don't pay attention to code quality in tests. Test is test, right? So I'm trying to figure out ways of improving that. So for example, when the agent's writing tests, can we measure coverage and guide the agent to write tests based on coverage, but also give information about, oh, those tests, they are redundant. They are pretty much checking the same lines of code. You can try and define them. And the cool thing is that we are thinking about those things because we want to automate the agent, but a lot of it translates to...

better developer tools, right? We release this for the agent, but developers can also use it. So I think a lot of the work that we are doing right now will go back, will fit back into like better tools. Even when I'm working on TideWave, a good amount of the work will eventually fit back into better tools for like Elixir and the community too. Right on, man. Well, keep fighting the good fight. I always love talking to you. I always love hearing what you are working on. I am going to give TideWave a ride in earnest. I have a Rails app now. You know why we have an Elixir Phoenix app, so I can use it in both contexts.

Let you know what I think. Give me some feedback. Let me know. Yeah. And then right now is bring, it's either bring your own key or you can use your GitHub, your GitHub Copilot integration. And then hopefully in about a month. What are the tools that you use today? So I use, I use Cloud Code. I have ChatGPT Pro, but I don't actually use Codex. I'm not sure if I get Codex with Pro. I have Gemini CLI. Okay. I don't know. It's very confusing. Like when you get Cloud Code, do you also get.

tokens for the API? I don't think so, right? So you buy those separately. I'd rather not buy more of those. I'd rather, is this why people want to bring their cloud code subscription? Exactly. Yeah, I would love to do that and not have another toll bridge, toll road, as Adam calls them? Toll booth. Toll booths. So that's my current setup. Adam, what are you doing? You got some AMP subscription maybe? About five bucks left in AMP.

I still can't hold it right. It's always expensive for me. It's really great though. It's so cool how it works. It's really one of the best, but I haven't found a way to hold it in a way that isn't expensive. Oh, okay. So cloud code primarily. Same. I have an anthropic key, but only because I think one thing had to have it. And I think I got like the trial balance they give you. I'm still on that. So nothing past that. But cloud code, automate code I like as well. They're cool.

I still like AMP. It's just, I haven't found a way to make it expensive for me. I just, I don't know, but it is really, really, really good when it does. Is there anything in particular that you like about it? It seems to be just, it's got this Oracle. So speaking to AMP code, it's got an Oracle where it can go back and consult. It's kind of like ultra think. Now that I think about it, Jared, it's not quite that, but it's a bit more where it'll go into a deeper understanding of coding patterns and, and like a learned behavior across, you know, let's just say like a rust CLI ecosystem. Like how do those work generally? We're good patterns. And it will come back and tell you stuff like that. So I find that it's research and it's ability to execute in a,

you know, hands-free YOLO environment is just, it's really good at that. Like you wind it up on the right thing with the right research, the right context, the right everything. It just plows through it for hours and just does amazing work. So, but it gets expensive if you don't like work with it and babysit it. Do you prompt for the, for the Oracle or it automatically figures out that it's like the plan mode or the, or it automatically figures out, Oh, now's the time for like some oracle. And yeah, you know, I think it does it on its own desire, but you can also say, Hey, in this exercise,

Go ahead and prompt the Oracle as well. Tap them, get them involved. It feels cool. It does it. And good results come, I guess. But you can either prompt it yourself or it just kind of does what it needs to.

I am not an AMP code expert by any means, but that's how I experienced it. Yeah, so wrapping it up, yeah, right now it's being your own API key. And you cannot, yeah, and Cloud Code, like your OpenAI subscription or Cloud subscription does not give an API token. And we cannot actually, because, you know, Cloud Code is just using a Cloud API, right? But we cannot use that API. It's actually not according, it's not legal according to Cloud, to entropic terms. So we decided to not do that. That's why we're working on the whole Cloud Code codex integration kind of things.

And so either bring your own key, but I really, at the end here, I really would recommend giving the GitHub Copilot a try because it's confusing because Microsoft calls everything Copilot, right? But you can, there is a GitHub Copilot plan that gives you access to kind of like a bunch of different models. And it's actually like, it's a predictable plan in the sense that, because the thing with paying for tokens is that it's very hard for you to predict how much it's going to be and paying and GitHub, the GitHub Copilot subscription is for messages, which at least improves the visibility a little bit and it has like a basic plan quite affordable so that's a good way to try it out for now to to get some feedback and yeah we are hopefully uh launching uh cloud code we are uh released something called acp i don't know if you saw those news like with the agent protocol so you can talk to codex cloud code gemini cli right so we are building on top of that but it's a bunch it's it's it's work because they're running on the browser right and acp is an io protocol so you can figure out all the the hopes that we have to jump

to make those things talk to each other. But yeah, hopefully we'll be launching that soon alongside Django, Next.js, and so on. I wish Anthropic would just give you, when you get some sort of subscription, they'll just give you a token or a key that you can use against that subscription at the same pace that you use Cloud Code, right? Like, I guess they're just subsidizing that to death and then they want to subsidize their API because I can pay 20 bucks a month or whatever it is and use the dog do out of Cloud Code, but I got to pay 200 bucks or 500 bucks equivalent to use this API the same amount. I just made those numbers up, but you can see the discrepancy is there. It doesn't make sense to me. I guess they just want to use their CLI a lot. I would say it's not even about how much it costs. It's just how predictable it is, right? Yeah, because you don't want to get dinged for making the bad prompt. You want to set, I'm fine with $20, $40, $50 a month, but just because I use it a lot, don't give me a... I tell it to UltraThink and it's like, well, that's $17 for that UltraThink. I'm still wrong.

Can I get my money back too? Are there returns? I know, right? Service degradation is a real problem for me. But here's the thing. I actually think pushing people towards using cloud code more or codecs and building on top of those tools, like with ACP, is not actually a bad idea. Because...

Because here's... Okay, let me tell you a story. Quick story. I know you're going. So when we first implemented TideWave, we focused on Anthropic and the cloud models. And if you go to Cloud Code Prompt, it has things like, you should be concise. Don't use too many words. Use four words. I think it even said at some point, one answer word is best. And of course, it doesn't listen to that. It finishes the feature. It just dumps four pages of text about the thing that it implemented that nobody ever reads.

Right. And so when we did our prompt, we tested with those things as well. It does improve a little bit. Right. But it also say things like don't write a code comment. It always writes a code comment. So anyway, so we wrote the prompt.

And then when GPT-5 came out, we decided to give it a try and start supporting OpenAI. And it was very curious because it would say, like, hey, implement this feature. It would do all those things. And then at the end, done. And then we would ask something, and it would say, good. And then we realized that the prompt we had for Entropy that was saying, be concise, GPT-5 was actually listening to that prompt, and it was being concise. That's why we were just saying, done, good. It was not doing any fluff or anything. And that's when...

You realize that you actually have to come with a, if you're building a coding agent, like I am, you have to actually build a prompt per model, right? And now GPT-5 Codex came up with its own prompt that it's different from the GPT-5. And then not only, so just doing that, like fine tuning the prompt per model, that's a pain. That's like, that's boring work. That's not something I want to do, right? And then it gets the other thing, which is, then they have the tools.

And at this point, those coding models, they are becoming so important for those companies that they're actually fine-tuning how it should send edits to a file. They are fine-tuning the models for that. So when the GPT-5 Codex model came out, they also said, look, this model is best at sending these kind of diffs and edits over the wire. So now I have to implement the specific editing tools per model that I support. And then each of those models come with their own context engineering techniques. So at that point,

So if you're like me, you're building a coding agent, you want to be able to get that infrastructure and build on top, right? And then it comes with a nice thing. So going back to the hot take, if you're building your agentic tooling for coding, instead of doing the MCP, don't do the MCP. Build on top of ACP and have control of the agent and use all those things and extend that instead. And with the announcement with Claude Sonnet...

4.5 today, yesterday, they actually recognized that. They renamed the Cloud Code SDK to Cloud Agents SDK. They moved a couple things around for it to be a better SDK for people to build on top, right? Because I think that there is a lot to gain for leveraging everything. They are tightening, right? The model, those tools, and we want to be able to leverage that. That makes sense. They're putting a lot of work in to take that model and make it an agent. There's no reason why everybody else needs to do that work as well. How would you go about building on that right now? Where would you go? What's the starting point to building that right now? I don't know. What's the website?

Oh, I don't know. Zen.dev slash ACP or something? No, I would just search for agent client protocol on Google. Agentclientprotocol.com. All right. All together, right? All spelled out. Yep. All right. Of course, if you Google that, it will be your first hit. And I believe there is an SDK type script. I don't know other languages right now, but the protocol is... That's the only language that you know?

There's Ante, you heard it here first. Jose only knows TypeScript. That's when you know it's getting late. I just autocompleted you. So, yeah, but I can see it becoming more and more important and we are going to see more SDKs. But the protocol is also relatively straightforward as well. So, yeah, I'm hoping that I can really see a lot of value in there and I hope it's going to catch up to the point where, because a lot of the, I think Gemini CLI supported built-in in the CLI, but when Zad released support for...

cloud code for example is because they have a wrapper so i hope it grows to the point where more uh like the the clis they are coming with building support for it right and then i hope it grows to the point that like is it who which of the big providers have their cli's version right now it's gemini it's open ai and and it's on traffic right like grox they don't have theirs i think No. ZAI, they don't have theirs. So I actually hope like those other companies, they start providing those CLIs as well with all those things we have been talking about in the sense like, look, here are the optimized diffs. Those are the things we improved for. So we can move to the point where we are all building on top and not particularly inventing that wheel. So I really hope it grows. More CLIs. Give them to me. There you go. Thanks for hanging with us, Jose. It's always a pleasure, man. My pleasure. Yeah. All right. Bye, friends. Ooh, synchronized.

All right, that is your changelog for this week. We hope you enjoyed Monday's news episode about exiting the feed for Cell vs. Cloudflare and why overengineering happens. We hope you enjoyed Wednesday's interview with Evan Yu, and we hope you enjoyed this episode with Jose. Because after all, we're here for your enjoyment. We also want you to learn and to keep up the easy way, and we want you to level up your own work and to feel connected to this worldwide community of hackers. But we'd love for you to do all those things while enjoying the process. Because after all, the process, that's our life, isn't it? If you do enjoy our work, please tell a friend, or three, or send us an email, editors at changelog.com. We absolutely love hearing from you all.

Thanks once again to our partners at Fly.io and to our sponsors of this episode, depot.dev, fabi.ai, and miro.com. Thanks also to the one, the only, the mysterious Breakmaster Cylinder. Have yourself a great weekend. Let someone else praise you and not your own mouth. And let's talk again real soon.