Interactive notebooks were popularized by the Jupyter Project and have since become a core tool for data science, research, and data exploration. However, traditional imperative notebooks often break down as projects grow more complex. Hidden state, non-reproducible execution, poor version control ergonomics, and difficulty reusing notebook code in real software systems make it hard to move from exploration to production. At the same time, sharing results often requires collaborators to recreate entire environments, limiting interactivity and slowing feedback.
Marimo is an open source, next-generation Python notebook designed to address these problems directly. Akshay Agrawal is the creator of Marimo, and he previously worked at Google Brain. He joins the show with Kevin Ball to discuss the limitations of traditional notebooks, the design of a reactive notebooks in Python, how Marimo bridges research and production, and where notebooks fit in an increasingly agentic AI-assisted development world. Kevin Ball, or K-Ball, is the vice president of engineering at MENTO and an independent coach for engineers and engineering leaders.
He co-founded and served as CTO for two companies, founded the San Diego JavaScript meetup, and organizes the AI and action discussion group through Latinspace. Check out the show notes to follow K-Ball on Twitter or LinkedIn, or visit his website, kball.llc. Akshay, welcome to the show. Thanks, Kevin.
It's great to be here. Yeah, I'm excited to get to talk to you. Let's start out with a little bit about you. So can you give a quick background of who you are and how you got to our topic today, where you got to with Marima?
Sure, happy to. So my name's Akshay. I've got a background in computer systems, but also machine learning research. I spent a little bit of time at Google Brain.
This was a while ago, back when Google Brain existed before I was taken over by DeepMind. So this was 2017 to 2018. I worked on a TensorFlow team. And then after that, I went back to Stanford to do a PhD in machine learning research.
And through that experience, in my time at Google, I realized what I really enjoyed doing is building open source developer tools for people who work with data. So after my PhD in 2022, I started working on Marima, and Marima is, it feels like an next generation open source Python notebook. And it is a notebook, but as we'll talk about in this episode, it's different than traditional notebooks in many ways. It's reproducible, it's stored as pure Python, so you can version it with the XK as a Python script, and share it as an interactive Web App.
So let's maybe look into that, right? I feel like notebooks have been a part of the Python ecosystem for a while. And when they came out, it was this big breakthrough of like, oh my gosh, I can do this kind of interactive exploration of data and then share it and do all these different pieces. What's wrong with the state of notebooks?
Why Marima? Yeah, it's a great question. And traditional notebooks, like Jupyter notebooks and things like them, I think have been extremely useful in research and education. And I use Jupyter a lot during my own PhD.
So my thesis was on vector embeddings. Now, I would make a lot of low dimensional plots of high dimensional data after doing some dimensionality reduction. So lots of scatter charts and stuff like that. And it was really, really useful to do that in like a Jupyter notebook because I needed to run code and then see what the results of my algorithm were.
And like there was like a back and forth between you as the algorithm developer and your data, right? And so that's great. There's nothing wrong with that. The issues that I ran into and that others have run into with like these traditional notebooks are a few, there's a few issues.
And I think it comes down to like maybe two or three. So one, like the one that sort of really trips up many people myself included is this idea of hidden state. So in a Jupyter notebook, I'm using that shorthand, but just like a default experience of a Jupyter notebook using the iPipon kernel. It's an imperative paradigm, right?
Like you run a cell and mutate memory and then you run another cell and then mutate memory. And what you're really doing though is oftentimes once you get past the just exploratory phase, you're kind of writing a program like even in a notebook, right? But because it's imperative and because Jupyter doesn't know how your cells are related, you may run one cell and forget to run other cells that depended on that cell. And then all of a sudden the code on your page doesn't match the variables in memory.
And this can lead to like a lot of these in myself. Like you know, I would do things like I would delete a cell and then they would delete some variable that I forgot was defined in that cell. And it was still a memory and like other code referred to that variable. And then four hours later, I would realize like I just had like a bunch of inconsistent state and I was a restart my notebook, restart my analysis.
And so this broad idea of hidden state is like one thing that I think is really challenging that I wanted to solve with my remote. And then other things include like the ergonomics of using notebooks like as part of like modern software projects. The default file format for Jupyter notebooks is great for communication because it like stores not just the code, but also your blocks and things like that in a single JSON file. But it makes it difficult for any kind of software engineering like task, right?
Because you don't want to version basically for data with Git like it's just not gonna work, right? And then also you might want to reuse some code you wrote in a notebook in like another notebook or a Python module, right? But you can't really those aren't Python files. So then you have the issue where like, I've been guilty of this, you duplicate your Jupyter notebook like 40 times and then just a total mess.
So that was another thing I wanted to solve. And then finally the third thing has to do with like shareability and like sort of interactivity. So on the one hand, Jupyter notebooks are really great because you have plots, you have code, you have visuals, this document you can share out. Like if you share out some analysis or some like research investigation with a collaborator, they need to also have Python installed in Jupyter installed in order to like interrogate your results.
Like they might want to say like if I change some parameters, how might this plot change? And that feels like kind of unfortunate. I mean as a matter of practice, like my PhD advisor couldn't do that, right? And even have questions and then I would have to go back and change things.
And so with Merimo, we wanted to make it so that any notebook could also really easily double as like an interactive web app where like it can promote any variable, anything to like this UI component, like a slide or something. So that people could for themselves just interact with the document and see how results would change. So if I'm hearing you properly, I'm gonna play back. So two of these things sounded like essentially saying traditional notebooks or Jupiter notebooks or whatever, where in a lot of ways just kind of a fancy rebel.
They're a rebel environment that has a UI baked in and is able to be a little bit more approachable in interweave plots and interweave descriptive text and things like that. But they have all the drawbacks of a rebel in the sense that it's not really meant to be something that's packaged and duplicated. It's your exploring as you said, having a dialogue with your data. And what I'm hearing is you want to keep that vibe of dialogue with your data, but kind of bring this up into first world software development.
Like, hey, this is reproducible. It's data driven. It's state. It like keeps track of all the things.
It's not me hacking away in a rebel somewhere. Yeah, that's exactly right. And like, I think one of my teammates, I'm really much, Trevor. We were just talking about this the other day.
And the model of just a rebel works well as long as just you sort of working on that. No, but like, no one else. But as soon as you bring in a second person and like, or even the second person can be you a day from now. Right.
Like, yeah, you kind of want some guarantees, some reproducibility, some. Yeah, that's exactly right. All right. So let's maybe talk then about how you solve some of these problems and I'm particularly interested in kind of understanding what is the execution model that you're adopting that is not just this sort of imperative rebel.
Definitely. So in designing Merimo, we took inspiration from other projects as well and the two biggest ones are these other really cool notebook systems for other languages. So Pluto, JL, for Julia, which in terms inspired by observable for the JavaScript ecosystem. And what those two notebooks are and what Merimo is at the core, they're what are called reactive notebooks.
So reactivity is sort of the alternative to this imperative style of notebooks. And so what this means is say in a Merimo notebook, if you run a cell, say that cell, divine some variable X when you run that cell, Merimo, then by default will automatically run all other cells that read the variable X and so it reacts to your. Your code execution of one cell to keep the outputs of the rest of the notebook in sync with the action you just took. The way that this works is that there's no runtime tracing involved and instead Merimo basically just statically reads the code of every cell that you have and determines where the variables it defines and where the variables it references.
And from there, it just builds this dependency graph. And it's kind of like Excel in some sense, right? It's not magic. And we have like configuration.
So like automatic execution may not be even desirable for all notebooks, especially if like some downstream cells are going to take a long time to run. So you can make the executor like call it lazy so that you run a cell and then Merimo will just mark the effect, sell the stale, but it won't run them automatically. But I'll give you one button that you can push to bring all your code and outputs back into sync. So that's the core thing.
And so that can minimize hidden state. And also as you play with it, you also find that it actually enables you to do data exploration a lot faster because like you just change a variable value, hit enter and then you see everything change and et cetera. Well, and this is a paradigm that I think web user interface frameworks have very much gone towards this kind of reactive model. I think you did it first and you see swelt and reacting all these folks like taking this very data driven reactive model and it does allow very.
Fast and easy keeping things in state. So looking at that dependency graph, then how much overhead does that end up creating or like, are there any things in Python that are hard to statically analyze? I'm not super deep on Python in particular. I know it's some languages that can actually be hard to trace the dependencies.
So on the question of overhead, it's like totally negligible. Python has a built in AST module that you can use to do static and semantic analysis of code and like most heavily used libraries and Python that's implemented in C. So it's really fast. So they overhead and that was one, especially because it's static analysis.
So it's like we parse and analyze once and then every other execution. There's no overhead. So you don't notice that. And in terms of what is hard, like I guess the scope is the static analysis itself is the semantic analysis that difficult.
And I think what we chose to do is to like there's two approaches you could take to making at least two approaches to making a reactive notebook and Python. Right. What we did is our data flow graph is based only on variable definitions and references. Right.
So like I mentioned, the cell defines a variable X. You run that cell. We run all ourselves that read the variable X. That's easy to implement faithfully for the Python language.
You could take another approach which would not be based on definition references, but just on memory access. And for example, you could say, if my cell mutates or touches some verify, if I'm at this cell, it pens this on list, I'm going to run all ourselves that read that list. That's extremely hard to do reliably. And so we explicitly don't try to do that at all because you can't.
It's impossible to do is just static analysis and Python. So then you're going to have to start tracing user code and you're invariably going to miss like it's impossible to implement that 100% correctly. And then you get into an uncanny value when the user runs a cell and they won't know what else is going to run. So we're just really upfront.
We only track variables, definitions and references. So if you mutate things, go for it, but be aware that we are not there. Yeah. That's an escape hatch.
Right. Exactly. If you're mutating, make sure you do an assignment afterwards. Exactly.
That's exactly right. And you know, the benefit of this is that like the rules exceedingly clear to the user, right? Like they can understand it. It's like a sentence long.
And then also it encourages the users to write like functional code, right? Which especially for like data driven stuff, machine learning and it's kind of what you want to do anyway. So it's good. Like I think like one of our users described it as gentle parenting or something like we know data, the scientist, machine learning engineers, et cetera.
They're right. Just generally good code. There's a lot of value in that. I actually I was slight aside, but I was hearing somebody describe.
They said, if LMS generate no other value, they have dramatically upgraded the quality of code. Coming out of graduate schools. That's fair actually. Yeah.
I like it. Okay. So let's let's keep going down this road. So now you have instead of immutable, you have a well defined dependency graph and you mentioned the next thing was around sort of reproducibility and not checking in these massive binaries or things like that.
So how does Remo handle treating the stuff as code? Yeah. So that's a great question. So the data flow graph gives you some amount of reproducibility.
And so far as that you can't like run a cell and forget to run some other cell like remo, just running for you or market is still and like really, really loud. If you've done something sort of, I guess it just won't let you step out of its reactive execution model. So that handles like reproducibility and execution in some sense. And then I'll get to the file format.
I guess it is related actually, but Merimo does have sort of an optional built in package management system that's like powered by the UV package manager. So if you opt into it, when you import a package, remote detect that import a module, resolve it to a package prompt if you want to install it. And if you do, it'll add it in a comment block at the top of the Python file. Python has a standard called pep seven two three for this.
So basically we'll document all the packages your notebook has used. And then the next time you run that notebook, we'll use the UV package manager to create an isolated virtual environment, install just those packages. And then you're off to the races in this sort of reproducible package environment. So we handle that as well.
But the reason that was an easy feature for us to add is that we actually decided to store our notebooks as Python files instead of as these JSON files that sort of Jupiter has historically used. And the way that this works is each cell is represented as like a function. There's like some decorator to like demarcate like this is like a cell that's going to be going into the notebook. And you can think of each cell as a function mapping the variable references it uses to the definitions it creates.
And at the bottom of the notebook, there's a Python you can do an if name equals main guard, which is like when you run the script, that's what's going to run. And so there's an if name equals main guard that will then say app dot like it will run the marimba notebook in the cells in a topologically sorted order. So basically all that to say you can go to the command line and say Python might not book that P Y and it'll run it as a script. You can even parameterize with C L I R.
You're already though getting to a place where now now this stuff is pluggable because it's just Python code. You can import the functions to wherever you can do what have you. So before we dive down that road, which I am interested to go down the implications there, what if anything is lost by storing this Python rather than this sort of proprietary format? Yeah, so there's definitely something lost.
So the main thing that's lost is by default when you're using Jupiter, not only is your code, but like your plots, for example, like are stored as basic before encoded data in the file so that you can just like put it on GitHub and like you can immediately see a record of your analysis. So I actually think the IPython notebook file is a really valuable artifact. I just don't think it should be the artifact that you're developing centered around. And so what we do to sort of bridge the gap is that there's a configuration setting that you can turn on, which will basically automatically snapshot your notebook as an IPython notebook alongside the Python file.
So there's a little underscore and a sort of memory, but directory and you'll save like my notebook that I pi and B in there alongside your Python file so that you can try and get the best of both worlds. There's other things that we ended up having to sort of implement our own versions of because, for example, one thing that's nice about a Jupiter notebook file format or like just something that stores the outputs is that when you load up the notebook, where you can see the previous runs execution without running the whole thing. If that makes sense, right? Whereas we start from just the Python file and that file doesn't have those outputs.
So we implemented our own sort of like session cache, which is stored in some sort of directory that is hidden from the user and so to replicate some of these nice features that flat file format did provide. That makes sense. Well, and since you have the dependency, you have all the variables already labeled, you know what you need to save. Exactly.
Yeah. Okay. So I want to come back to the UI, which is because that was a thing you talked about as well. And I think that's interesting, but this has like brought me into this question or discussion topic around how notebooks fit into the broader software development lifecycle because I think one of the things I have seen in places where I Python notebooks were tended to be used before is they were heavily used by, for example, a data scientist or data science team.
They were used for data exploration. And then if you wanted to then take something that was there and package it for reuse or embed it in a product or whatever it was like a whole effort porting new code, all these different pieces. But to me, it sounds like this Merimo file is literally just Python once you have something that works, you could use it. Yeah.
Yeah. That's correct. So you can and we do and many of our own sort of internal utilities that we write for our team, just like internal tools that happen to be in remono books that are reusable as Python files. You can even say like from my notebook import my function from my notebook import my class like that syntax kind of just works.
There's like some details. The function needs to be pure so that serializes correctly, which by the way ends up you in the writing better code. Right. So this is and so yeah, you totally can.
And so it really does blur the boundaries of what you can use a notebook for, which I think is really exciting because like we see like all kinds of use cases from the traditional data science and research use cases to like back end engineers like emailing us telling us like, yeah, we're doing our data pipelines with Merimo books just can and so I want to hear more about that because yeah, my experience is all been notebooks. Often research ML data science communities and then that's its own thing. So how are you seeing the integration happening? When do you choose if you have one of these back end engineers?
When are they using a notebook? Why would they choose to do that over something else? And like, what is the process there? Yeah.
So I think there's at least two reasons. One that we'll touch on with like the interactive components that you alluded to earlier, but even without that. So I think often when making like simple data pipelines, like, you know, not super complicated ones, but simple ones, it can be helpful to prototype a data pipeline in a notebook because similar to like what I was mentioning, having a back and forth with your data, right? Like you write some code, you see if the data is put into the shape you want it to be in.
Sometimes it's easier to do that with visual inspection. So yeah, because these are doing visual inspection, it can be nice to do it in a notebook. And notebooks also a good choice because with data pipelines, the job runs and it can be nice to just have a report alongside it to see like what was the shape of the data that they had, et cetera. Right.
So it's nice to prototype as a notebook. But now with Merimo not only as a prototype as a notebook, it's also really easy to just run it as a grand job or as a script. Like you don't need to like reach for sort of other tools that orchestrate IP and B files. You just say Python my job dot py, whatever, it's just a Python script.
So that's one area where we do see sort of natural usage. So this already is getting me to a place that I'm curious, right? So often if I'm doing a big data analysis job, I will want to do something interactive on a subset of my data. And then I'm going to run something async when I do the full data because it's going to be big, slow, expensive, what have you.
But maybe I want that same visualization, right? I want that report right in there. So like, is there an easy way within Merimo and maybe I'm just missing something obvious to be like, OK, right now we're using this local subset of data for this one. You're going to do a remote call fetch it from here.
You're going to do what have you. Can you plug into those like I want to run my big data off in the cloud somewhere, facing fast or slow, but get it back into my notebook? Yeah, you totally can. So it's not necessarily productized, but we have a number of primitives.
And so one primitive you can use some remotes, a notebook, but it's also a library that you typically only use in the Merimo notebook. So you can import Merimo as Merimo into Merimo notebook and you get some primitives. One of those primitives is am I running inside in an interactive session or am I running as a script? And so you can just use mode out running in notebook to parameterize where the data is being fetched from.
And I think that would do what you're asking, essentially, or you could use that to do what you're asking. I'm also wondering about, yeah, can I, well, maybe this comes back to the publishing side of publishing things as well. So maybe we'll come back to this, but I'm thinking like, okay, I've run this on my temporary thing. I still want my interactive view, even though I'm running this asynchronous lake and I run it async and still publish the web version of Merimo so I can see the report at the end or something like that.
Yes. Yeah, you can do that too. That is from the CLI. We have like a Merimo export and then choose your file format choice such as HTML.
So Merimo is where HTML that will run it as a script, but also pull up and generating the HTML report at the end. Got it. Oh, that's super cool. Right.
So what I'm hearing then just thinking about life cycles here, right, is like, okay, I'm taking with it. I'm exploring it. I'm running it locally as a notebook subset of data. I have this flag.
I say, okay, I think this is ready. I put in my flag saying when you run a subscription, run against this source instead of that source. And then I run it generating an HTML report that looks the same as my notebook. Let's make a look at it.
Do it. Yeah. Yeah. That's great.
That's really cool. Digging into that shareable web side of it. So what is interactive? What does that web generation look like?
Can I still go and tinker with cells or change things like what is the output starting to look like there? Yeah. So interactivity in Merimo starts, let's say in an interactive edit session. So you're working on your notebook, your browser, VS, or using your notebook.
We do have a VS code extension as well. So I guess I just think one step back, rebels are like, everyone thinks of a repolos interactive, right? Because it is, right? Like you run a cell and then you see what happens.
You run something else in like notebook. Traditionally, like when you want to see what happens when you change a value of some variable, you have x equals five, then you hit backspace and you change the value of x to six. Then you hit like shift enter and then you shift enter a bunch more times, right? Then you see what the new thing looks like.
And then you go back to the right, then you hit backspace, x equals seven, and then you do it again. And it's a great TV. And like that's the kind of thing like any normal person like that. There should be some UI element right to control the value of that variable.
So in Merimo, you can import Merimo as Mo into your notebook. And then the Mo.ui module gives you access to a bunch of different UI elements, ranging from the very simple to like sliders and text inputs and and drop downs to sort of more complicated ones like interactive scatter charts, selectable, scatter charts and things like this. And the way that it works in Merimo is that you can assign a UI element to a variable. And so like x equals Mo.ui.slider.
Then when you output that variable in the notebook, so if you may x the last expression of the cell, Merimo will display the slider. Then if you scrub the slider, then Merimo will then do is then automatically run all other cells that refer to the variable x. So it hooks into the reactive execution system. And then every UI element has a value attribute that gives you the value that was assigned or like associated with it on the front end gives it back to you in Python.
And so like just like that with no callbacks required, now you have like user interface or interactivity in your notebook. And you can use that to say speed up data exploration. But as you can imagine, you can also use that to make like really simple interactive data apps or web apps or whatever you want to call them or like any kind of internal tool. And so that's another big use case of Merimo.
For different folks, different sort of, I guess, pathways. Some people will just use the UI elements to speed up exploration. But this will just like, you know, I was just on the phone with like a, I guess I have to speak about them anonymously right now. They're a big Merimo user.
They're like a very well known sports team. And they're using like Merimo for a bunch of like analytics and stuff. And they make like a bunch of Merimo apps that like you type in a player's name and you see like a big table of like a bunch of rec stats about that player, et cetera. And it's just like, fancy interactive web apps that they deploy on an internal site for the rest of their team to use.
And the way that works in Merimo is that any notebook from our CLI, you can type at the CLI Merimo run my notebook.py. And it'll serve it as a read only web app code cells are hidden. And then now some non technical stakeholder can like interact with your data. And so you were mentioning backend engineers and like surprisingly to me, like there was we talked to this one company.
We have a case that he published in our blog about them. Their name is Tax Wire. They're like, yeah, all our backend engineers, we all use Merimo on a weekly basis because we make all these internal web apps about this tax software we're building and it's use cases and then we embedded inside our internal next JS app and it just makes our lives easier. I'm like, okay, cool.
That's awesome. And like honestly, that's not something I anticipated as a use case when I first started making Merimo right after my PhD, but it was really cool to hear hear people being empowered about. That's awesome. So I think what I'm hearing too with this is like you can decide which variables you are exposing the UI versus which are loaded from somewhere versus how you're managing all of that.
So if you had this example of loading up a whole bunch of data, I mean, I guess your your source example is that it's got a whole bunch of data in the backend, but let's you configure which player you're looking at, maybe which slice of data you want to analyze based on. Exactly. Yeah, that's exactly it. And I guess the value of the notebook here in this case, because it may not necessarily be obvious.
It's just like it's it's like this like progression from like I just have some data here and I don't really know what it is and I'm just like kind of playing with it and then like, oh, actually, there's something useful here and like, okay, now I want to expose this to other people. And it's just like you can just like stay in the same tool and just incrementally like abstract did a little bit and promote things to you I am in and then had some explanatory text. It's just like a really seamless path, whereas like if you had never opened an open up to look at your data in first place, it might be hard to then, you know, think about, okay, what is a react act app? I'm going to write to like empower the rest of my team.
I don't even know what's in the data. So I don't even know what to make. And so I think that's what the value in the notebook is just making it really easy for you to first get a feel for your data and then from there making it really easy to make like a tool that's good enough for you and your colleagues and yourself. Yeah, that is super valuable.
I mean, I have an example where I've just whipped up some internal scripts to analyze things for me, but it outputs in text because I think in text, if I had thought ahead and use the notebook, suddenly I can share it much more easily with different folks. Another question that I have in this is I think one of the things that's going on in the software development world right now is things are changing incredibly rapidly. We've got L&M's, co-pilots, Agentic tools, all these different things. Is there a similar transformation going on in the sort of data and data programming worlds or how are you seeing notebooks fitting in with this kind of new industrialized coding era we're getting into?
Yeah, I think there is a similar transformation. It's a few different ways. So one of our users is like a very large republic company and they told us they have hundreds of like merimo apps deployed internally. And they said what made it really easy for them to adopt merimo and the reason they adopted it so quickly was because it turns out cloud is like really good at writing them because it is a pure Python file format.
It's like all these things you can get around, but like, well, I guess your notebooks aren't interactive by default anyway, so you can't make web apps with them. But with yeah, merimo being a pure Python notebook, I can write it easily. You can also run it up because the script to check if it's doing the right thing. We also have like a linear CLI tool called merimo check, which will like report in the errors it finds like with this syntax of how the notebook is stored.
And honestly, like I use cloud also to like make really quick internal like things where I would have made a merimo notebook by hand. Now I have cloud help me make a merimo notebook and it does a good job. Like, especially like compared to like earlier in projects like merimo is now, I think popular enough that it's like in distribution. So it works pretty well.
I guess that is more similar to like how cloud is like just speeding up software development in general, right? In terms of for data specifically or for ML research specifically, I think that people are still figuring out. I was just talking to someone on my team and he told me that he's seen a bunch of VC things out early this year saying 2026 is going to be the year that AI and agents revolutionize how we work with data. So I'm like, I don't understand that to me because I haven't seen one of those yet, but I guess people are talking about it.
I just I'm not too familiar. I think like Databricks says that 80% of something something databases are created with agents, but I think that's like neon or something. So I mean, it's it's sexy right now to say, okay, we're going to use this is going to be revolutionized with AI. I feel like software development is the one place I am actually seeing that play out.
And so yeah, kind of interesting to see how that happens. Where I think is I was just going to say, I think the basic things I think many people have been trying, maybe just being quietly integrated in a bunch of products, but like text to SQL type of things like schema. It's got like, it seems natural that some form that will benefit from code generation and sort of injective workflows. But I guess we'll see.
Yeah, we'll see if this prediction is in a thing true. On the sort of forward looking prediction side, what do you see as sort of the frontier in terms of use of notebooks, data access, data exploration, that sort of world? And what kinds of stuff are you working on internally to address it? The frontier.
And that's a good question. The frontiers are always hard to apply not because it's the frontier. So I can talk about what we're working on and see if I'll back out and see if any of that is going to be getting us to the frontier. So let's see.
So some amount of work we have like a good amount of work is like sort of we're close to parity with the Jupiter and colab ecosystems, but we're not at full parity. So we do actually have a good amount of parity that we're doing, like getting really my working seamlessly inside of Jupiter hub, which is like a multi user hosted sort of Jupiter deployment that many universities use. We also have a free hosted notebook that's similar to Google collab called a tongue and cheek. It's called MoLab Mo from Marine Mom.
Nice. And so we're working on that quite a bit this year. I guess one thing in terms of frontier. So we have a speculative project, which I don't exactly even know what exactly it is and where the part of projects forgot what it is, but it's driving Marine Mom headlessly, like potentially with agents.
There was this paper that came out recently that someone on my team sent me that I've so far only skinned the headline and intro, but it's something to do with like instead of using, I think the claim of the paper was that Python repels or repels in general can be very valuable ways to like dynamically create context for LMS or agents. And in that paper, I think they use a Jupiter kernel as like the thing that the LMS access to. And so we actually did some one of our engineers did a lot of work to like sort of modularize Marima towards the end of last year. So we're getting closer to a place where like you can use the kernel endlessly with that or UI.
And one project that he's really interested in exploring is like, well, what if you gave an agent access to that kernel? Could that somehow like speed up whether it's like data exploration workflows or research workflows? We're not exactly sure, but it seems like that could just be like a really valuable primitive, like a sandbox in some sense for agents. Like, so I'll give you like guys one example and this is not related to like necessarily headless, but like enabling agents to work more effectively with data.
So like in Marima, we have a built-in like AI assistant sort of system. And one thing that you can do is when you write a prompt like for generating some code, you can tag a variable, say like a data frame. And when you do that, we inspect the data frame, see it's schema, get like sample values, et cetera, and like dynamically generate context to give the LMS. And now you have like code that's like specialized to the data hand, which is like more useful than like say, if you're not using Cloud or cursor and it doesn't yet.
So they won't know what's in the data frame unless you explicitly tell them. That's just like one way that you can make like empower agents with runtime information. And I guess that's one thing that we want to explore more this quarter is to be. No, it's super interesting to kind of think about that because there's a couple of different pieces that stand out to me.
So one is the fact that you have kind of the dependency graph already mapped becomes quite interesting in terms of showing just the relevant context to the agent. Right. So you might say, Hey, you're changing something down in this one cell, but you don't actually care about that instead of forcing the agent to absorb the context of all the different steps. You could say like register which step in the graph you're interested in.
We'll do all the computation. Here you go. Here's the output. I do wonder.
I feel like there is some sort of interesting opportunity here in terms of just like showing it the right things at the right time. Do you have a concept of lint or correctness checking on cells? So like you've got this dependency graph you're going through and maybe a step three or four ways down. It's like, Oh, this is outside the bounds of what could be valid.
So something must be broken upstream. Yeah, yeah, we do. We have a checker or a linter that will check your entire program for like semantic correctness as well as like syntactic correctness. So there's a couple of rules that we might enforce is to make sure that your graph remains like a DAG basically one of which is you can't have cycles across cells.
Which I think is sensible. Although I recently learned that Excel has a feature that you can turn on in settings to enable cyclic calculations and then you choose like the number of iterations you wanted to go until convergence. I got a spreadsheet that was all refarers. I'm like, Why did you send me this spreadsheet with all refers?
No, no, no, you have to enable fixed point iterations. Anyway, sorry. There's a lot of value in keeping it to a DAG outside. Yeah.
So I'm already my husband DAG and we enforce that. That's one rule that we check. And the other is actually you can't redefine the same variable across multiple cells. And the reason is like we actually allow you to reorder cells for like presentation purposes.
That's like column view and stuff. So those are the two main semantic things that we check for. I was also wondering though, in terms of like data range validations, right? So for example, you have a dependency of a center, different computations and you might know something about the shape of the data and say like, Okay, this data needs to actually be in this shape.
If not flag something and don't keep going or what have you. Oh, that's super interesting. And we haven't explored that. The reason I think about that is like with a gentic coding, which has been diving down it like the more you can programmatically deterministically limit the sort of scope of possibility and then give that feedback to the agent, the more it's able to independently iterate.
Yeah, that's very interesting. There's a lot of things we could play with, but that doesn't interesting. So you mentioned in Merimo, you have these things as functions with decorators around it and you're already building in this kind of some amount of static analysis, some amount of links and things like that. What hooks do you expose to your end users in order to kind of plug into that?
So right now, to be honest, we don't have the biggest extension API surface area in terms of extension points, not in the file format, but we have standardized on this protocol called any widget for building like third party interactive widgets, the developer of any widget Trevor Manz actually works at Merimo now. And so that's one way that you can plug into Merimo. You can also hook into our display protocol for objects that we support the iPad on display protocol, but we also have some additional hooks. And then in terms of the file format itself, you can actually write Merimo notebooks by hand in VIN or whatever your text choices.
I don't know why you say that. If you're holding software engineering daily, that's my guess. But so you can do that. It's not an extension point, but it is designed well.
So that you actually still, the file format will guarantee you still get code completion and tell us all these things. But we haven't opened up an actual, like put another way, Jupiter is famously, I think, well designed in terms of like the internal protocol, like this is the kernel protocol. They have a wire protocol. They've got a bunch of things that like their party developers can't look into.
We haven't done that yet just because it was too early for us to do. I think things are really quickly starting to change. We now have our own still internal, but semi-public APIs for ourselves because we're not consuming Merimo in many different ways. And I think eventually over time this will evolve into like some of these will be open up to the public.
But right now, I guess in terms of the day, I guess like our file format specification is public in six. And so people can target that with code generation tools, but that's about the extent. That makes sense. Cool.
Let's look a little bit actually at any widget. You mentioned that as one of the places that you plug in. It's an open source. You hired the developer or whatever the sequence.
I'm a big fan of that type of thing. What is it? How does it interact with the reactive data model that you have? And are there any constraints or things to know going into it?
Yeah. So I'm going to caveat this by saying that Trevor is a far better spokesman for any widget than I am, but I will try to channel. And he was just on, I think, Talk Python with me and had a great hour long conversation about any widget. But it is both a spec and a tool set for making like reusable widgets.
I think really focused on for using them in interactive, like local environments. And so what any widget was born from, my understanding is from talking to Trevor is so Trevor also has a PhD and sort of like the bio computation space was originally his focus. And he was found himself having to make widgets for like all kinds of sort of domain specific tests in the Jupyter ecosystem. And they were kind of really difficult to build and maintain and test.
Like web programming has advanced a lot in recent years and like Python widgets had not kept up. And so I think sort of add up some of those difficulties in frustration like Trevor sort of built any widget to like make it a lot easier to implement and maintain these widgets and also to make it a lot easier for like different frontends to consume them. Like so before any widget, if you made some kind of domain specific widget that worked in Jupyter lab, then you would have to go and then customize it to work in CoLab and then also make sure it worked in the VS code extension. And now by like making the spec, like you can make it any widget, just make it once.
And because people have agreed to support it, it'll kind of work anywhere. And so that's been really valuable. I know it's really good for us. My co-founder Miles discovered any widget pretty early in his life cycle.
And he was like, oh, we should use this. And I'm like, I never heard of this. Because my original, he's like, no, no, no, it's really good. It was a really good bet because I think it has emerged as the standard for interactive notebooks.
And in terms of hooking into a reactivity model, it's such a quite nice. So basically any widget, you can wrap it in a mo.ui.any widget wrapper. And then it basically binds it to our reactivity model and makes it into just like any other UI element. That's first part of the demo.
Yeah, so it hooks into the data flow graph in the same way. And I think that the value of it is, I mean, people make all kinds of really, really cool widgets. And like, we just like, there's no way that like us as like a team of seven would be able to satisfy everyone. But like, which is that we're originally developed for Jupiter is a scatter plot, which is called Jupiter Scatter, which now lets you see like 10 million points on a scatter plot, like really efficiently and like zoom in and zoom out, et cetera.
That now works in Meri Month today. And it's also reactive, which is sort of like gives it superpowers that you might not have had in traditional hooking environment. Got it. So yeah, it looks to me like it's essentially a vanilla JavaScript spec that if you meet that, then you can wrap it up.
And it'll just plug in. You have a wrapper. Jupiter has a wrapper. Other folks who support this have a wrapper and it'll just kind of work anywhere.
Yeah. And we're going to be focusing a lot on any widget this quarter as well. One of our employees Vincent, he runs our YouTube channel and does a bunch of things. He never sees his teammates me like how far he can get with vibe coding is like really cool.
And he widgets like, and he like, vibe code is any widget for like robotics, anything like this, even though he's a human way person. Like, I don't know, it was like, it was really cool. So like, it really allows you to like expand your imagination and get creative. Nice.
Well, we're getting kind of close to the end of our time here. Is there anything we haven't talked about yet that we should talk about before we wrap? I think we've covered all the basics. So I guess we didn't really talk about Remo's origins.
And then I could talk a little bit about his origins and a little bit about where we're going, like at least in the next few months. So I started writing out after my PhD, like I mentioned, out of both appreciation for notebooks and frustration with them. And I actually originally got funding from a national lab at Stanford. It's a lab called Slack, which is a particle accelerator lab.
And there are a bunch of scientists who use Python and like had basically the same gripes as I did. And I say, we're really excited to sort of partner with us to bring like a new open source programming environment into the world. So we have our roots in academia in that sense. And this quarter, one thing that we're really interested in doing is like engaging a lot more with universities to help them try out Remo for education, help support them, like incorporate it into their classes.
Because I really do feel like that a combination of reactivity and interactivity can really just make concepts just like a lot more intuitive. Like it's just somehow like, if you learn like some numerical algorithm, it's just so much easier to just change a parameter and see what happens as opposed to just like a strictly through it. And in fact, Remo's main inspiration to do for the Julia language. It was originally designed exclusively for education.
It still actually is advertised in that way. It was at MIT where it was designed for a computational thinking class. And I don't know, just something I care a lot about and something that we really want to support. So to the extent anyone in your audience in the intersection of software engineering and education, and if I'm really interested in like, please try it out or reach out to me in my team, we would be happy to chat with you guys and support you.
All right. Let's call that a wrap.