Skip to content
477: Change Management episode artwork

Oct 7, 2025 44 min

477: Change Management

from The Bike Shed Hosted by thoughtbot

Time to plan an upgrade as Joël and Aji talk about the hurdles involved with various change management in their projects. The pair lay out some different approaches to protecting your data when planning a migration, the risks of code and data changes, the elements that will and won’t be affect in the process, and Joël gives his experience on a tough migration project and what he learnt from it. — If you’ve not used Merge before you can learn more about it here. Thanks to our sponsors for this episode Judoscale - Autoscale the Right Way (check the link for your free gift!), and Scout Monitoring. Your hosts for this episode have been thoughtbot’s own Joël Quenneville and Aji Slater If you would like to support the show, head over to our GitHub page, or check out our website. Got a question or comment about the show? Why not write to our hosts: [email protected] This has been a thoughtbot podcast. Stay up to date by following us on social media - YouTube - LinkedIn - Mastodon - BlueSky © 2025 thoughtbot, inc.

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

477: Change Management

0:00 44:25
of matches

Transcript Auto-generated, may contain errors

Autoscaling is a simple concept. Automatically add resources when needed, and automatically shut them down to avoid paying for excess capacity. How hard could that be? If you've set up Autoscaling yourself, you know it's not that easy, especially using the native Autoscalers on platforms like AWS and Heroku. That's why you need JudoScale.

JudoScale is Autoscaling as a service, and they make Autoscaling simple and easy, as it should be. You can use JudoScale on AWS, Heroku, Render, Fly.io, and more. It's free for low traffic apps, and unlimited plans start at $25 per month. Autoscale on Easy mode at judoscale.com. Hello, and welcome to another episode of The Bike Shed, a weekly podcast from your friends at Thoughtbot about developing great software.

I'm Joel Kinville. And I'm Anjou Slater, and together we're here to share a bit of what we've learned along the way. So Anjou, what's new in your world? I had one of those active record queries that is just so complicated that I honestly enjoy those so much. They're the thing that gets closest to an individual single puzzle in programming.

There's a lot of things that are like puzzles, but I find those just so crunchy and fun a situation to work on. And I think my two strategies are my two things that finally worked helped here in this situation. The first one being to step away for the night and come back fresh the next day. Fair. I spent so much time throwing different things at the problem, but never getting exactly what I needed.

But at the same time, learning so much about what wasn't working and the domain and everything of this particular challenge, and stepping away, coming in the next morning, it got it right away. There's something about this type of problem that really lends itself to diving in really deep and then stepping away, letting your subconscious chew on it for a little while, and then the next morning comes together. In this particular instance, it's a healthcare app. And so what's the most commonly complicated thing about healthcare? It's scheduling.

I needed a set of records where the associated patient had an upcoming appointment of a specific type. The records have the foreign key patient ID and the patients are related to appointments through another table functioning basically as a joint table. And you still need to be able to sort an query on the original record. There was a query in place that had a lot going on. I think probably requirements had changed or been focused or updated a few times over the course of developing this method and query in question.

So there was a lot to dig through as to what was kind of the intent and what was going on there in the first place. But at the end of the day, our good friend, Murch, was really what saved the day. Just added a new association on my first record and merged in the complicated query about the appointment and everything fell into place. I feel like I don't reach for Murch as often as I should. Me too.

This is the stepping away part. I had not thought of Murch everything I was drawing before and stepping away and coming back in and realizing that there was a path that I hadn't gone down because I just wasn't thinking about that path. I wasn't thinking about that tool, but having some time away allowed me to assess my tool blocks a little bit more fully. And for listeners who are not familiar with Murch, we'll link to the documentation in the show notes, but it basically takes an active record relation and takes all of the wear conditions and combines them into whatever sort of other active record relation that you're merging in with. Crucially, though, it does not pull over joins or selects or just the tables that you're pulling from.

And so it's not doing a full merging of everything from the two relations. It's just pulling the wear conditions from one relation and tacking them on to the other one. For those times when you need posts that have comments and all of those overlapping concerns that need to knock out some of your results if they don't fit into another set. It's often a really nice way to compose scopes where you might have a scope for verified or approved comments, and then you need to pull that into a more complicated query on an article's model. And you don't want to duplicate that condition.

You can just pull them in from the scope on the comment model. Absolutely. I think it was something that was going on in the code that already existed at this end point. There was a lot of duplicating of conditions and scopes. I think as my assumption is as requirements were changing and getting layered on one at a time that existed as a scope already in the appointment model.

So clearing away a lot of the duplication of those scopes and just using merge with that appointment scope got it all solved and a lot easier to read to. How about yourself? What's up in your world? I've been playing around a little bit with my routine. So I spend a lot of my day just sitting down at a computer and writing code.

Turns out that sitting down for a long time is not good for you. So something I've been trying to incorporate into my routine is a little bit of stretching. Particularly targeting what they call the posture of your chain. So sort of your glutes and then sort of everything going down the back of your legs. The hamstrings are particularly for me are often really tight.

So finding some stretches that target that in particular has been really helpful for me. What's spurred on this change? Just feeling really tight, realizing that I can't really bend over that well and wanting to feel a little bit more comfortable when I'm sitting as well. So did a little bit of research. You know what I should probably incorporate some more stretches?

Turns out that a lot of people have a lot of opinions about stretching. I'll bet. Yeah. Figuring out from the fire hose of the internet what you actually want to do is non-trivial. Yeah, absolutely.

Do you have any stretches or exercises that you like to build into your day? I think that I would call a routine. I probably should. I think we could all benefit from that. As you were mentioning sitting for an entire day is not the best thing for your body.

I find I have a lot of tightness around the kind of across the front of my chest from shoulder to shoulder and that contributes to slouching over a lot with my posture. And so I try to open that up. I'll put a hand on a door frame or something and then turn as much as I can to open that up. And that can be really, I find it kind of invigorated too. I'm sure if you're in the middle of a problem with something you stand up and you move around a little bit and you get a little bit of blood flow.

It can be really beneficial just to your mood and everything as well. Yeah. I have a pull up bar set up in my home. So I can just, it's a quick thing that I can go and do in between calls if I've got five minutes or something. It's also I find really nice just to hang because that really sort of stretches out the a lot of the upper body.

It's also the sort of thing where you can use it to sort of lean a little bit as well, sort of do some stretches where you're holding overhead but then you can sort of like almost fall forward and sort of stretch various parts of your legs and back. So I've been really appreciating that. There used to be a slack bot, a company that I worked that would once an hour, I think, just fire off a random exercise to do for 30 seconds to a minute or something to just remind us to get up and move around a little bit. So it became a bit of a, yeah, a bit of a routine for everybody who's working on the second floor of the office to just drop into a minute or as close as we could get plank just once a day.

And yeah, it became a fun group activity too. So it really kept us doing it, kept us honest. Did you find that that had an impact on how you felt? It probably did. I don't know that there was much that I could pointed to or noticed.

It wasn't certainly enough time to be considered like a workout. So I don't know if there was enough of a benefit to be as noticeable as something like that. I was also much younger. So that's also a thing, right? You're like, you can do whatever with your body and you won't notice it.

Exactly. Not so much anymore. Have you ever tried to debug an issue in production while switching between your air tracker, log viewer and APM tool? Maybe you know how painful it is when you can see the error but can't find the relevant logs or when you see slow performance but can't connect to the root cause. Scout monitoring is all in one platform, built to make your life easier.

With log management, performance tracking and error monitoring coming soon, all of your data will be together at last. It will go from bug to fix in one place. The result? Less context switching, faster debugging and monitoring that actually helps you shipcode instead of distracting from it. No DevOps team?

No problem. Scout works with your existing stack and gives you insights in five minutes, not hours. And bug to fix in one place at ScoutEPM.com. I see a topic here in our backlog that I think is something that we have to deal with pretty frequently as developers and got a couple of different things that are going on current project and I wonder if there's something that is in front of mind for you as well is change management around like large changes. What are some of the ways that like what comes up to you first when you see that a project coming up is going to be something like a framework upgrade or switching to a new third-party vendor or something?

What are some of the things that pop in your mind first? First, I want to step back and think a little bit about how we want the upgrade or the change over to happen and a question you have to answer is does this need to be sort of a zero downtime switch over? And if you're using some sort of third-party system that's pretty core to your functionality, let's say you were using some kind of third-party authentication system and now you're switching over to a new provider, that's the kind of thing where you either got to shut down your site for a little while you switch over or you have to have a plan so that the switch over happens seamlessly. And thinking through all of that is going to be really important.

Absolutely. Yeah. And if you're doing something important in a lot of cases then the actual technical work to do it because a lot of times that can be pretty straightforward, right? You're replacing one system with another but it's that transition between the two that is going to take the most planning, the most kind of focus from the team, focus on alignment and things like that. It's also tricky if you're doing sort of a change over like that and you want to think about all the sort of in between like transition states.

So again, we're talking about let's say you're switching over a third-party authentication service. What happens in world where someone kicks off an authentication request before you do the change over? It doesn't complete. You start the change over and then you're how do you handle like that request coming back? Right.

And for something like authentication, maybe you say, you know what? If they happen to try to sign in during that 30 second window when things are switching over automatically, they just get a hey, your authentication failed, try again and that's fine. Right. Yeah. If you're writing data to a third-party vendor or if you're reading data from a third-party vendor, then you might have to have some more better mitigation strategies.

That does kind of come in handy with authentication or authorization, things like that you can just sort of default to deny. And it's like, oh, if something went wrong, just you can't and try again. But when it's more subtle, you know, I'm going to break out a fancy word here. I dip it in C. Mm-hmm.

Things that are I dip it in are usually easier to do this sort of change over with because you can do the multiple times, you can retry them. And it's usually fine. Yeah. First of all, I like very much that you pronounce that word the same way that I do. There are several pronunciations all are valid going around, but this makes me very happy.

That's something I think about a lot with things like even just database migrations and like data changes, things like that, if you aren't sure that it's going to go to complete the first time, then making sure that you can rerun it and not have things exploding in your face is always going to be super important. And yeah, that's the same sort of idea whether you're making a change to a database or a larger scale change like we're talking about. Just to be clear here, I don't want to see for a data change over our migration or something, doesn't necessarily have to mean transactions. You could allow a partial rate where half the data set gets written and then you crash halfway.

And then you rerun the task. The important thing for it to be independent is that that first half of the data set doesn't get like then duplicated when you rerun it the second time. So you're doing a lot of things like find or create for your migration. So you can run the migration 50 times and each run it gets another 50 of the percent or 50 of the task done. And at the end, you get the same result as if you'd run it once and it had gone to completion.

And I think a lot of us will encounter the idea of a dependency most often, but how can we take that and apply it to not just data changes, but pieces of the application like we're talking about. It's a fancy term, but I like to think of it as sort of a shorthand for safely retryable. And again, that doesn't mean transactions. So it doesn't necessarily mean that things will roll back, but it doesn't mean that you can rerun it multiple times. And that's important property for things like background jobs, because queueing system might retry a failed job that may have partially succeeded.

But I think having a sense of that term and being able to label things as a non-ident or non-ident, now you know as part of your migration strategy, which parts of your app can sort of safely fail and retry as you're in that transition period and which parts you have to be really careful about with your scaffolding. Yeah, absolutely. I think we saw that log in effectively, I depended on that you can log in. And if we say, no, just try again. And okay, you haven't hit the new service and you're good.

And it's maybe not quite the perfect experience for a user, but it doesn't result in a bad state. Right. It's not an experience for the user that's completely out of left field. Like that's something that they may have run into before. It's just an unauthorized or like, hey, try again, instead of having data on the screen that doesn't make any sense or half of the app loaded or something like that, that's something I think about a lot to from a user experience point of view is, is this something that users going to be familiar with or is this something that's going to confuse or make them worried or make them do something that they probably don't want to or don't need to do, but based on what they're seeing on the screen, we'll take an action that is different.

Another thing I think it's worth thinking about is the change you're working with a code change or a data change. Code changes are often generally safe to refactor and instructor around. Like there's a limit to how bad you can mess up. You might crash the site if you don't do it quite right, or users might get an error page or maybe they even get a garbled page. But then after that, you're going to see the bug, you're going to fix it and it'll be fine moving forward.

You mess up data and then you fix the bug. Great. Data moving forward will not be corrupted, but the data created in that window, you might be stuck with forever and it might not be possible to clean it or to recover it. And that might be a headache you have to live with for the rest of the apps lifetime. And so the stakes are much higher when you're working with a, anything that migrates data structure, so like a database migration, but then also sort of data storage if you're doing any kind of like reading and writing from a third party.

Yeah, that's really true. And those kinds of bugs that are caused by that corrupted or misaligned or whatever data can be some of the hardest to figure out because they're an impossible state, right? You're knowing and your models of the system and everything that you can look at in the code will tell you that that should have never happened. But you've got this ticking bomb sitting there until somebody accesses that particular part of the system or that piece of data. Yeah, you violated some invariance of your system.

Yeah, 100%. Do you have some strategies? Let's say you're working with just a database for your app, no third parties, and you're trying to do a migration that will like significantly restructure the way tables are handled. So what's your sort of favorite approach to handle that kind of change? I think being as incremental and stepwise as possible is what I'm going to be trying to form that around, making sure that each step is moving us forward but is also not creating those states where if it ever gets to a place where this one step can write data that is just not going to work at all or something like that, or is not considering which dependencies or what pages and different actions are dependent on data looking a certain way.

If you can do it stepwise and there's going to be an overlap in the middle that maybe the data is in both states for a little while before you know for sure that everything cut over and worked just fine and then you can start to deprecate or move away from the old system. Yeah, doing everything really piecemeal and thoughtfully I think is. So would you effectively sort of make certain columns, let's say read only in ActiveRecord as a like temporary state as you're doing this changeover? So you still have the old data but that you're now writing to a new column? Yeah, exactly.

Writing the new column, having the new, I don't know what other attribute that you might have like some validations or something like that have changed, having two columns that exist at the same time in both states as you're ensuring each step goes smoothly before you make that potentially a rather cool change. I feel like that's a sort of a general pattern where you sort of, if you have a stream of like data that's coming in and you're like writing it to a store and then you keep the old store because you just redirect the write stream and then you can read from one or both places and maybe eventually you get to the point where you're only reading from the new place and then at some point as a background process you can go and take the old column and like convert it or move it or convert it over to the new column.

And then once everything is both writing and reading and historical data has been treated that the old column is like totally untouched then you can safely drop it. I think that works as well for even parts of your application, right? You've got the example of the authorization, third party system. You might incrementally move over maybe an endpoint and start to run everything there off of the new authentication system and as you're like moving over one at a time when you finally got everything all set to the new application, you've hopefully got your logging, your testing and everything is coming back green and what you want to see, then you can sunset that old way of doing it.

I, while back, did a really interesting migration with a third party system where we were doing effectively a search through a bunch of the entities that they were offering. It's like a private database thing and they decided that they were going to change over their primary key scheme. So all of their primary IDs were going to change on a certain day and that was a problem for us because we were using these IDs to be able to reference things. So that effectively invalidated all the data in our system, but they did not want to give us a mapping file between old IDs and new IDs because they felt like that was going to give us all their proprietary data or whatever.

So now you're stuck up. On this day, all of your primary keys that we can fetch them from a search API when we're figuring out based on user interaction to build associations in our system, but then we want to load these associations based on these primary keys. We can't do anything because they're all bad. Turns out that they also on their end built user profiles for every user that did searches and things in their system and they had some IDs associated to that. So on a certain day, they were going to go and migrate all of the IDs in their profile on the third party and then we could query that on a per user basis to get their updated IDs.

The whole thing was a bit wild. But they weren't going to migrate it until right before the migration happens, so we couldn't go and do this in the background. What we ended up doing was a bit of a live changeover where effectively we said detect whether we're using old versus new IDs and we had a system that was trying to detect whether the third party had changed over or not. And we said, if the third party has changed over and we can tell that this user's account has not migrated, when that user's account tries to do anything, we will in the background do a conversion sort of on the spot.

So instead of like pre-migrating everything because we can't do that, we are sort of doing it live as users use their accounts and so it becomes more of a pull versus a push model in terms of the migration. That is incredible. Don't do that to your customers, everybody. So it was definitely a really interesting like as this problem was evolving, initially we had a much simpler migration strategy and we're like, all right, we're going to do this and this. And then as we're sort of talking more with this vendor and they're like, oh, but here's another limitation.

Oh, but actually we can't do that. And then we kept having to adjust and then eventually we landed on this kind of complicated thing, but that honestly it worked. Changeover day came and their sort of IDs turned over and our system kept going. So it was seamless on our end. It took a lot of very careful planning, a lot of testing, but it worked.

Hopefully soon after that, you found another vendor for that part of the system, right? Absolutely no. Yeah, that's not usually possible. Wow, that's really incredible that their change like that required so much work from, I'm sure, most of their customers, most of the people using this service. And yeah, that's just wild.

So if put yourself in their position, what do you think you would have liked to have seen or what do you think that you would have done if you wanted to make that change and had other people accessing your data that way? And I think the way you do it is you probably offer a mapping of all your old IDs for your new IDs. Maybe you're uncomfortable doing that. Although, again, we sort of did it ad hoc for only the things that we had. I could imagine a world where maybe you don't want to expose the whole list, but you're comfortable offering a like maybe rate limited API endpoint where you can exchange an old ID for a new ID.

So again, you're not offering the entire list all the way to affect. If you knew all the old IDs, you could go through this endpoint and generate all the new ones. So from a security perspective, you're not totally preventing that, but you know, people do need to be able to convert from old to new in some way. And so I think a nice compromise would have been probably offering an endpoint where you can just do an exchange. Yeah, that's a nice compromise there.

I wonder if there could have been some kind of deprecation cycle vision of that, not exactly sure what their needs were, but having the new ID for a little while on those requests. Yeah, so going back to what you were saying at the beginning, you have a transition period where you give both in your responses. Yeah, but that's of course not going to catch all of the ones that a user doesn't log in for six months or something like that. And the whole cycle has happened. You're still going to need some kind of mitigation strategy there.

I mean, that would allow us to run a background job that just like finds all the entities in our system and sort of re-sinks them against the third party. True, true. I think that's one thing that's really interesting about this sort of migration strategy is sometimes you want to sort of do all the work ahead of time before the cutover happens. And sometimes you want to do after and sort of let user action trigger it so that things only let's say, re-sync themselves in response to a user trying to access that entity. So it's a bit of like an eager versus a lazy approach.

So of course, like always the approach depends on the context. Yeah, but knowing what tools you have available that you can use a lazy strategy versus an eager one, I think helps you to plan for that kind of migration more easily. Yeah, helps you make those decisions in the first place if you know what all the eruptions are. I feel like something we've been dancing around a little bit is the Strangler Fig pattern. And I know what that is.

So basically what you do is you for slowly build a wrapper around your old system that pretends to be the old system, then becomes the new system. It's still working with the old system APIs and then eventually you sort of you cut over. So it's a code strategy that allows you to keep the same interface and migrate that while still sort of keeping things underneath sort of working either with the old one or the new one and you can't tell from the calling code. Right. And I think you can do that a little bit with some of the, like we're saying that switching from your writing stream from an old column to a new column in the database just by changing around some of your like readers and writers on active record model so that anybody interacting with that model from the controller doesn't know that you're doing a big structure change at the database level.

And it just sort of keeps working as before, but it is slowly moving from the old one to the new one. I think this is a very good argument for keeping your separation of the concerns always in mind, right? Like not knowing how a different system is performing their action, but just sending messages and relying on that interface instead of the internals of your collaborators. Yeah. And I think there's arguments for sort of how much you want to do that for every collaborator or every third party system in your app.

If you know you're going to be doing a big change, it might be worth as a first step just wrapping an adapter around that part of the system because that then gives you almost like a wrapper or like a shield that you can work behind and nobody else needs to know that changes are happening. Do you have any kind of heuristic or how often would you start with something like an adapter like that? I've gone back and forth. I've heard advice as extreme as like every gem you use should be wrapped in an adapter just in case you want to swap it. That feels a bit much.

Again, I think a little bit of that is a sort of an eager versus a lazy approach. Sometimes what you can do is only when there is an need for change. And so then that adapter becomes less of a core part of your architecture. It's more of the scaffolding for migration. And so step one of migrating is introducing a wrapper around the section that you want to change in your code that perfectly mimics the existing API.

You are going to use that to basically just be a pass through it, perfectly mimics the existing API. And then behind that API, you can start changing all sorts of things. And that allows you to make those changes in a way that are like safely encapsulated with other the app knowing. So you don't always need to preemptively encapsulate in your architecture. You can do that around only the components that you care about at the time of wanting to do a big migration or change.

I think something else that also kind of factors in there sometimes is how much you trust the other thing to keep its interface or that you think you will be with this, that pretty vendor, that sort of thing. So if there's a lower level of trust and you think that you might have to hit the eject button sometime in the future, if that looks imminent for some reason, then you might kind of bake that in from the beginning. There's no plan for a migration, but you just know that it's likely. In general, some of the encapsulation is just a good thing to have among components in your app. So we've been talking about data migrations and integration services.

And I think we focused on that because those are the more dangerous ones. But we did talk a little about code. And I'm curious to hear a little bit about your thoughts on how you like to proceed code migrations. I think a really common one is a framework upgrade. What are your favorite strategies for a framework upgrade?

Well, I think again, working as incrementally as possible. It's a little bit of a different kind of concept, but a different implementation of the same concept where if you can move some things over before the framework upgrade to the way that it, you know, it functions later, or if there are little things you can put shims in, but make the change and then kind of take them out. But doing small changes, I really am terrified of the hard cut over sort of thing that maybe makes sense if you have no users over the weekend or something like that. That's not going to be a fun weekend for anybody either. How can it be a fun PR to review either?

I guess that framework upgrade was so minimal. I guess if you can give it a try first and it's only a few lines that have to be changed, then maybe it's not so scary. But you mentioned sort of shims or sort of tools that you can use to make your current app compatible with sort of a future way of running things. And one of my favorite examples is the transition from Rails 3 to Rails 4, which I think maybe shows my age a little bit. That's before my time.

But that is when strong parameters were introduced prior to strong parameters, which came in Rails 4. All the access controls were handled at the model level, or not at all. I think at least someone hacked GitHub to demonstrate that this could be done. They hacked GitHub by giving themselves commit access to Rails to prove there was a vulnerability in Rails. And it was a big deal back in the day.

That led to strong parameters being introduced in Rails 4. And what the Rails team did is that they kind of backported it for Rails 3 as an optional gem that you could install. So if you install the strong parameters gem in a Rails 3 app, all of a sudden you can use strong parameters in your Rails 3 app. And that means that when you do the upgrade to Rails 4, that's not a whole subsystem you need to change out. You just drop the gem because it's now built in and that's it, your app works.

And that's a really beautiful way because it meant that you didn't have to sort of do a whole cut over a long running branch or a dual boot or anything like that. You could just include the gem and over the course of month, sort of one by one, you can control or start adding strong parameters. And it worked incrementally and made it much easier to upgrade. Yeah, and that's also allowing people access to the stronger security before maybe cutting over the rest of their application. Because I know in those earlier days of Rails, those upgrades were much bigger than the they have been in the last couple of major versions.

So yeah, having that split out was a really great way to handle that from way to go, Rails team. Yeah, I'm really impressed by that. It is my go to example. And I love that approach of saying, hey, let's take our Rails 3 code base and make it as like four compatible before we do the changeover as possible. So that ideally then when we do the changeover, it's just very minimal amounts of code.

That's an argument too for staying on top of what new changes are coming down the road or putting, you know, good deprecation cycles into libraries and different things like that so that people can be making those proactive changes as it goes along. Not quite as helpful if that version is already old, but the version you're on is before that one. So you're less able to follow along. You can't run off of Rails main for a few months to slowly build it over when you're sort of putting a position to have to make a more sudden or more complete change, all the ones. Yeah, I tend to think of applications that are sort of on a type of framework as being sort of a backwards looking, neutral or forwards looking position.

Again, my go to example is working with a Rails 3 application. Rails 2 made a lot of big changes to the active record query interface. It did allow a lot of sort of older style queries to remain just deprecated in Rails 3 and then got dropped in Rails 4. A lot of Rails 3 apps out there are backwards looking because there's still all the queries written in that Rails 2 style, which is still allowed in Rails 3 but will be dropped in Rails 4. And so not only do you need to do the Rails 3 to 4 stuff to get your upgrade, you also have to fix some of that baggage you still have left over from Rails 2.

Contrast that with some of you who just generated their app at Rails 3, they don't have any of of that baggage, but they're maybe not super, they're still going to have some work to upgrade to Rails 4. And then someone who's really put in some intention, maybe in preparation for upgrading to Rails 4. And then some of the parameters came. They've done other things that mean that when they cut over to 4, there's just a much smaller different thing that needs to happen. That app is now forwards looking.

And so that's the kind of mental model that I have in general for where you are in terms of an upgrade, are you backwards looking, are you neutral, are you forward looking, and before you get to the really hard painful stuff of doing that cut over is trying to move you along from wherever you are, maybe you're backwards looking, maybe you're neutral to get to that forward looking position. And on the idea of forwards looking, have you very often worked on an application that is pulling from Rails main and following along? I have not. Most projects that you are locked to just a release version. Sure.

I've been able to do that just a small, small handful of times working with more early stage applications. The Rails core team puts in a lot of work to make sure that main is stable and is usable. I know a lot of large companies will run on main and they're part of the feedback loop for the Rails team of a little bit of a canary in a coal mine or something like that when something does go a little bit wrong that they know they have something to address that they might not have known about before. Yeah, so it's high priority for them to make sure that that is usable in production. And so it's I think it's something that more people could be doing than they probably think.

So I would encourage folks out there to give it a try. Maybe not in production right away, but see if you could move on to that train because you'd be providing really valuable service to the Rails community by making sure that the new stuff is working. I'm curious for you. Are there any sort of techniques that you're aware of that you're not yet tried that you would like to get a chance to put in a practice the next time you have to do a tricky sort of migration problem? Well, there is sort of a one concept that I find super interesting that would be fun to have a chance to implement at some point.

And that's like a canary deploy having something like a load balancer or something in front of your application where you have a lot of different boxes running your app and you might spin up the new version of it and the load balancer is going to point I don't know 5% of your traffic to that new version something like that kind of a b testing at the hardware level almost or closer to it. And that just seems like an interesting thing to get to implement but not always applicable. So I'm waiting for the right moment for that to be the right solution. Yeah, so more of like an ops solution to the problem of that. Yeah, yeah.

Especially for rolling out like high risk changes. Absolutely. And you can always just shut off that 5% if something's not working right and you still got your application right there ready and waiting. Something that I've been thinking about and I've never quite gotten into because the investment is a little bit high but is digging into more of a formal methods where you can sort of describe a process and some variants and then there's like a logical solver that will find whether or not your process is consistent and keeps those invariance. So I can imagine something like that or weird process where I had to have that in my third party API that was cutting over with its IDs and I was going to have to sort of live sync things and you can, a formal methods systems do allow you to encode like temporal elements.

And so I would have been really interested if I'd known some formal methods things, these are tools like Aloe or TLA+, I could have sort of described my, actually migration strategy, run it through this and be like, hey, find are there any ways in which user data gets corrupted or a user loses their data in the cut over or somehow we miss our opportunity to sync these things. Now we just end up with some orphan bad IDs with no way to map them over. Is there going to be any downtime because of this? And then having the solvers sort of say, hey, given this approach you're taking, sort of modeling of the solution, this particular sequence of events could trigger one of the bad things you don't want to happen. So it'll tell you like, oh, well, if they migrate their data at this point, but you have a request that comes in that creates new old fashioned or like old data in the old style before they're cut over after they run their migration. And then they cut over and then you do this

thing and then you end up with a race condition or something like that. It's really good at finding those things that you wouldn't necessarily think of yourself. So finding all of the different like edge conditions or leaks in your welds or whatever between the two. Yeah. Wow. Okay. You're giving me a give me some homework for the weekend here.

That sounds really interesting. I'm a big fan of a Hillel-Wainz blog. He talks a lot about formal methods. And I think it's some of the work that he's done that's shown me some of the capabilities of these systems. Again, it's kind of like learning a whole new language. And for me, the application

would be rather niche. But when I do have situations like that where I only get to run this migration once, like it's the night that the third party is doing their cut over. And I don't get to try it again. It's got to go right. Otherwise I corrupt data. I would

like that through D. Yes, I can do some testing ahead of time. But I would love that extra layer. And so that's a time where it's like, I wish I knew that system. Well, well, all right. I've got some reading to do. So I think it's probably done to wrap

up about there, huh? Let's wrap up.

Frequently asked questions

How long is this episode of The Bike Shed?

This episode is 44 minutes long.

When was this The Bike Shed episode published?

This episode was published on October 7, 2025.

Is there a transcript available for this episode?

Yes, a full transcript is available for this episode. You can read the complete transcript on the episode page.

Can I download this The Bike Shed episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.