Imagine the last time you had to deal with request timeouts or backed up job queues in your job environment. Take yourself there. Can you feel it? The stress, the frustration? Here's what JudoScales knows. You're either not autoscaling or your autoscaler is not doing its job. Because autoscaling is supposed to prevent scenarios like this. In comes JudoScale. JudoScale helps your app smoothly handle traffic spikes while dramatically lowering your hosting costs.
You can use JudoScale's free plan forever, so you've got nothing to lose. If you want to say goodbye to those painful production incidents, visit judoscale.com slash Bikeshed for your special gift as a Bikeshed listener. That's judoscale.com slash Bikeshed. Hello, and welcome to another episode of The Bikeshed, a weekly podcast from your friends at ThoughtBot about developing great software. I'm Joel Kinville, and today I'm joined by fellow ThoughtBotter, Valerie Brzezinski. Hi, good to be on the show. And together we're here to share a bit of what we've learned along the way.
So Valerie, what's new in your world? Client project I'm working on, I've been digging into how I'm working on Sidekick. So there's a lot of concurrency and asynchronously involved in custom internal gems that I'm working on. But they're triggering certain lifecycle events in the Sidekick client middleware and the server side and client side middleware. With client side runs when you're queuing up a job, you're pushing it onto the queue. So that's when the client middleware and you can affect how the job gets queued or tell it not to queue at that point. And then the server middleware, that's when it's actually being picked off the job, potentially being executed.
The job's being executed at that point. So you can run additional middleware at that point. So they have some odd interactions between what they're doing with this middleware and the Sidekick batches, which is part of Sidekick Pro, I think it is. Otherwise, it's Enterprise. But I think it's Sidekick Pro. They're using both. But under certain circumstances, some of the callbacks and death handlers for Sidekick won't fire. Or under certain circumstances, they were firing. Other ones weren't firing. And yeah, it was fun digging into the whole middleware and the whole method chain of their processing of jobs to see, like, OK, how are all these two different pieces interacting in odd ways?
the concurrency and how do I actually fix it? So that's one thing that's new in my world. That's really interesting. So there's middleware sort of on the in-queuing side, but then also middleware on the dequeuing side. Correct. Yeah. So you can kind of like, there's a method chaining happening on the in-queuing side so that if you want to reroute or change certain aspects of the job or do anything on the, it's a hook to be able to do something when you're in-queuing sidekick jobs onto, you're sending it to the sidekick client and before it gets sent to one of the queues, you can run something in the middleware on the client side. And then on the opposite end of things, when it's getting picked off of a queue,
Before the job's executed, there's server-side middleware, and you can do additional things there. I don't think I've ever had to deal with Sidekick middleware. And it's interesting to realize that there's actually two different types. Is that something that you had encountered before? Most of my encounters with Sidekick have been all just job-based. So, like, implementing the jobs, making sure it runs, and how job executions interact between each other. But actually digging into the implementation side of, like, internals to Sidekick and enhancing some of the functionality there has been really interesting. I've encountered batches before, but I've never had to drop to, like, the middleware level. Yeah. It's part of their Sidekick Pro, and you can create...
A parent job can trigger a whole bunch of child jobs and enqueue those. And the middleware, the way that they implement the psychic batches, it gets run through middleware. If a job dies, it gets the psychic death handlers get called and queue up some functionality there so that it can be notified whether or not it's succeeded, if it's failed, when the entire batch is completed and run additional logic and business logic when those events occur. How do you like to approach situations like that where you're sort of assigned maybe a bug that you have to fix and it's in a whole new sort of system that you're not familiar with that can feel like a very daunting task? I guess a couple of approaches that I often take. Well, one, trying to replicate it in like either local development, staging server, development server, if I have to, production, but trying to find a way to like, okay, how do I get this actually to trigger and reproduce the behavior that they're seeing and see how I understand it from that perspective. Another way, actually digging through and just pretty much becoming a human code compiler and like walking through the code and how it's going to progress through the whole call stack and see, okay, what is it doing at these points and kind of just understanding the general environment and context of what's happening so that it...
I can confirm whether or not I truly understand what's happening. Like with this, I've been digging through the middleware on the Sidekick client side in their actual code base and the open source code and seeing how their Sidekick processor works, how their Sidekick retry job works, and how all these things interact with queuing and dequeuing of jobs. So it's just kind of taking notes and actually getting a understanding of like, okay, what is actually happening here so that I have a good mental model of what's working, how it works. And then I guess finally trying to write a test that replicates the failure so that I can work from it from that aspect and try to fix the behavior. or writing a test that of what I expect to happen in the breaking of things so that then I can get it to actually do the correct behavior. I love all those. I appreciate that you're so methodical with your approach to problem solving.
I feel like that's more than we see oftentimes with a lot of developers. We tend to, I don't know, try random things or click around or dig into things. But it sounds like you've got a very sort of step-by-step process. And I really appreciate that. Yeah. Sometimes I do the clicking around too. But yeah, when I'm really getting stuck or it's really complicated, then I definitely turn to the more methodical approach and make sure that I'm getting a full understanding, especially when it's a tricky production issue where it's happening intermittently. And it's like, OK, there's something that we've missed this a couple times. Let's be a little bit more methodical and capture what's actually happening underneath code.
So you mentioned that you sometimes like to take notes to help you along there. And I know that you're a big fan of note-taking in general. Have you always leaned on note-taking as a tool for debugging? No, that's more of a recent development. I've been getting into Obsidian, Second Brains, Doublecasting, Digital Gardens, the whole ecosystem personal knowledge management. That's been something that I've been getting into and starting to do since maybe 2022, something or other. And so starting to find different ways of working on notes when I'm learning something.
Like the today I learned kind of events that you often write blog posts about, like making sure I'm capturing those while I'm working and then just taking, logging what I'm doing while I'm developing. And there's one article, I forget the title of it, but it presented the concept of like document your confusion. And I really liked that because like when you're reading a readme for the first time or going through a setup guide and you just, it's not working, like documenting that confusion and figuring out like, okay, where am I getting stuck? Because somebody else is probably gonna get stuck here as well. And having that log, that trail of what you've tried can help you later remember like, how did I actually fix this? So I've also applied that to
digging through, like when I'm spelunking through code and digging through things, like kind of just taking notes inside of little observance, things I observe or notice about the code that'll be relevant. So you mentioned a bunch of terms earlier when you're talking about notes, the idea of second brain, Zettelkasten, personal knowledge management. Can you maybe briefly summarize what these ideas are about for those in our audience who've not heard of those before? Sure. Second brain, I've...
I forget who wrote the book. I've not read the book yet, but there's how to create or how to write a second brain. That's kind of using notes as a repository of your own personal knowledge and offloading some of that memory so that you can reference things and learn things in a lot of new ways. Digital Gardens are kind of, I'd say they're a little bit more web-based and kind of the, what always comes to mind is Mark Bernstein. I think I'm pronouncing that correctly, but he has Hypertext Gardens. It's an old website where he's just exploring how Hypertext and Hyperlinks interact when...
going through a website and it can be more personable than something that's strictly organized and more corporate and business-like. So it gives you a different experience. That's kind of been applied to personal knowledge management to create a digital garden of your notes to where you're interlinking your notes and creating hyperlinks to connect ideas and make associations between different concepts. Zettelkasten, that's literally like a slip box. Nicholas Lumen was a prolific user of this. That one originally started on note cards and you'd create the identifier in the corner.
Write down your atomic idea onto the note and then just keep on connecting those to different notes. And he used this to write, I don't even know, I forget how many books, but he popularized the idea of a civil casting. So it sounds like all of these are sort of similar, but slightly different approaches to organizing notes and thoughts. Yeah. You mentioned that some of them do sort of hyperlinking between notes. Is that common across all of them? The idea of linking between ideas?
Yeah, bi-directional linking and just connecting your thoughts together so that you can kind of keep track of how your ideas link together and then also discover with Obsidian. You can graph out your notes in a visual graph form to see how either the links interconnect or you can create tags on your notes and see how those tags interlink and form clusters or concepts that you might not have realized are interrelated and connected previously. I know. So I keep a set of nodes in Obsidian and I do interlink them. And I've used that graph view to help me find clusters where I might want to write a blog post or maybe give a conference talk. You know, it's CFP season and I'm sort of like, oh, what do I have to talk about? Sometimes I'm good and just keep a running document of potential topic ideas throughout the year. But something that I'll do sometimes is just open up Obsidian, go to the graph view, look at clusters and say, are there any interesting clusters in here that I haven't talked about? And that can be a great place to start for me.
Are there ways that you like to sort of get insights out of, out of your notes? One thing that I often do is create maps of content. And it's an idea that I've seen introduced from two different aspects. Like one, I believe Tiago Forti, I think it was, introduced it to the world of personal knowledge management. But there's also an old document from Lion Kimbrough that talks about it in more of a analog written format and creating maps of content in that. And I think that predates the PKM, personal knowledge management usage of it. But it's kind of creating a map, an index, more personalized index of all the notes surrounding a concept or an object or some.
area that you want to collect information and notes about. And so if you create these, then you can start seeing, have entry points into your notes that are on a certain topic, and then you can map out and see how they're actually interacting. So this can be in an outline form, or just a pure list form as well, or all sorts of different ways of organizing the actual page. But one way they have to do is I have one on patterns I'm thinking of. So just on the idea concept of patterns. And so I have all my other indexes listed of like, okay, here's patterns that I've seen in programming. So like architectural patterns or design patterns and principles and heuristics of things that you can use in programming. But then I also have ones over in...
art of like different art styles and different patterns that you see between those or note-taking the different patterns and models that you see in note-taking. So it's kind of like that one's more abstracted and they don't always have to be abstract like that, but they can consider it as navigation points and then also just collecting ideas on a certain topic. So it sort of allows you to see a summary of like maybe where you're thinking of on a sort of broader, almost like meta level. Yeah, exactly. What am I thinking about patterns in general? Instead of looking up every note you have on particular design patterns, you might have a high level paragraph on patterns and then a link to 20 different notes you've written on specific patterns or something like that. Yeah, exactly.
It's a great tool to have. I imagine it's probably helpful in sort of browsing and navigation as well. If you're not just wanting to search pattern in your code, or not your code, it's your note system and find every note on patterns, it might be useful to start sort of actually navigating through links by going to the meta pattern note and then working down from there. Definitely. The maps can also be in a different format than the way that you actually organize the physical files in within your note system as well. So they can reach across any sort of file system that you may or may not set up. Like some people organize their notes into just one flat file folder of here's all my notes. And then there's
indexes and maps and everything, the connection notes. And that's the organizational structure. And then other people go into really deep, like Johnny Decimal system kind of things. Johnny Decimal is kind of a cataloging classification system where you can create 10 categories and then number them one through 10, or maybe it's zero through nine, depending on how you want to start your counting, one of those two, but there's 10 categories. And then each of those major categories, you can then break down and subdivide into subcategories. You might have like...
work, and then you have different areas of work. Or you might have your programming could be one of the top-level categories, and then you break it down into, here's my notes on JavaScript or Ruby or web development versus other software development concerns. So that's more structured and very folder-based. So regardless of how you actually organize those files, whether it's flat or more structured message folders, indexes can serve as an alternate navigation point, an alternate structure to rise above those constraints of the file system. Yeah, and those would be necessary depending on how you like to access or organize your data. I think that probably becomes more important.
as your number of notes in your note-taking system grows. I'm curious how you decide what goes in your note-taking system and what doesn't. Because I know a common pitfall for note-takers is what's known as the collector's fallacy of just sort of wanting to just put data into your system. And at some point, we're not out there trying to replicate Wikipedia. It already exists. It's out there. So what is your heuristic for what you decide is worth putting into your note-taking system versus what doesn't need to go in? I actually have about three different vaults of notes, vaults or collection of notes. One is my Obsidian notes, where I take most of my personal knowledge management kind of things where I'm learning about programming, or I have a cookbook kind of vaults there where I have my recipes and how they interconnect with ingredients and notes on ingredients or techniques that I might be using.
Yeah, so I take source notes and then those often get either converted into fleeting notes of some sort in an inbox where any concepts of them... digging into, I collect notes there and that's just kind of like a ephemeral fleeting kind of temporary notes where I'm just collecting ideas and that I'll either discard that later or I'll disseminate that or clean it up to actually go into my actual vaults of knowledge management kind of topics. I keep a lot of my like temporal calendar type of things more in a bullet journal, which for that more of rapid logging and interstitial journaling on that end where throughout the day, like when I'm context switching between one task to another, I'll write about like, okay, this is what I've been doing so far since the last time I did a log in my...
bullet journal and kind of like a brain dump of like, okay, these are the things I need to work on throughout the day so that I have to write them out and then kind of like set an intention for what I'm working on next. And so it's just kind of like a very raw, here's my thoughts, here's my task, here's what I'm working on. It's just really digging into the rapid logging aspect of bullet journals. If just this is raw, it's only going to make sense in the context of like that day or the period of like a week or so. But after that, it's like they'll start to lose meaning. And if I do happen to capture anything in those, if like I need to write this in actual notes.
in my obsidian notes i'll capture those later but that's just basically i'll throw away just in the moment scratch notes of like what i what i need to be doing and kind of like task management then i have a secondary obsidian vault where that's pretty much like my diary so brain dumping if i'm want to emotionally write about something or something's happening on my day it's very personalized writing of just not necessarily having to do with anything but just raw brain dump of here's what i'm thinking about and just playing out with ideas and stuff like that so that's kind of like my obsidian diary of sorts so those are pretty much like three places where I might write notes. And each one has a slightly different distinct usage. So one for knowledge, one for personal things, and one for just a log of what I'm doing throughout the day. It sounds like the one for knowledge sort of gets filtered through a few different steps. So you're not just, anytime you come across a fact, writing it down into this knowledge system. But instead, it sort of like goes through this sort of filtering of, OK, well, I've got a bunch of random bits and pieces of things I've encountered during the day. And then I'll sort of maybe either put it together into something more compact or maybe find the ones that are most interesting and turn those into some more cohesive note. And so you end up with a more
kind of curated set in your sort of permanent notes. Does that sound about right? Yeah, like a pipeline system of like, how do I actually take notes from like inputs coming into my system? What are the activities and work and processes that I use to work with these notes and then create the... permanent or like settles of like more permanent notes that have more long lasting value. And then eventually another process is like having output, having a purpose for your notes and creating something with the notes, whether that be a blog post or just throughout my day, like I'm constantly referencing back to things that I've learned or taken notes about before just throughout the course of my workday of like, oh, I've worked on this before. What did I write about it? Or I've worked on this before. Like, especially if there's a code snippet or something like that, where I'm like, if I figure something out, I'll usually capture a note so that I can reference it again. Especially if it's an interesting problem that I've...
dealt with or dove into previously. Yeah. So there's multiple different levels of working with those kind of like in systems theory, there's the concept of flows and stocks and flows. So flows being like the feedback loops, the systems and processes in it. And then the stocks being in this case, like your information stocks, those permanent or more permanent places where information.
collects or gets stacked and then gets processed. So the inbox is less of a permanent store, but more of a stack where things are constantly inputting into it. You're processing it, either figuring out, OK, I can discard this. It's not important. Or I need to disseminate this into multiple notes. Or I need to polish this up. It's a different process and feedback there. Revisiting notes that way. And permanent notes, more of taking it from all those inputs and putting them into a more structured format, connecting it within all of your existing notes and system of knowledge. I like what you said about having a purpose for your notes.
I know for mine, I guess the heuristic that I tend to use is, is this something that I can turn into a small thesis statement about something I believe to be true about software? So I don't tend to just put random facts as much. But if I can find an angle where this is a sort of truism or a thing that I think is true about software, then it can get turned into a note. And so all of my notes are titled as thesis statements. But I also am pretty aggressive at keeping them small. So one paragraph, maybe a code sample, maybe a diagram to support that thesis that I have.
And then clusters of theses can then turn into artifacts. So again, going back to your idea of having a purpose, part of my purpose here is to enable me to create more in terms of writing blog posts, giving conference talks, that sort of thing. A lot of them have inspired topic discussions here on the bike shed. I've done that, but I also have a lot of things where it's just kind of more like a wiki as well, more of a reference. I don't capture everything about each subject, but like key points of having a reference and like a quick guide to sort of cheat sheets and just links to things that I've found on subjects like...
React or when I was learning Angular or different things that I've done in Ruby on Rails, having references so that I can more easily get to the bookmarks and links and resources that I know that have helped me in the past of like working on those subjects or relearning contexts when I'm jumping back into like, I've not worked in Ember for years. Like maybe, maybe I need to look up my notes and like what, how did I work on that? And how did, how might those, what lessons learned relate to working in Ember, the modern version of Ember? So yeah, so some of them are more referential about specific topics or objects in that sense, kind of like the nouns. And then other ones are more of like the claims and assertions of like writing a thesis and capturing a atomic note of like an idea.
So I have a little bit of both of those in my notes. I do think that that was something that I could get more value out of, is keeping track of more sort of cheat sheet style notes. A thing I appreciate about My Vault is that it is very clearly defined in its purpose. But that does mean that I say no to a lot of things. And sometimes I'm like, oh, I probably should have tracked that information somewhere. Sometimes I do probably get caught by that collector's fallacy of having more fun with the learning and input and capturing of notes, which is the journey of itself. That can be a purpose of if you want to learn and capture everything that you're learning about.
I didn't have Obsidian around when I was in college, but thinking back, I'm like, this would have been really useful for me if I was going through classes and capturing things on each of the subjects and connecting notes. It's a great tool for learning about things. So sometimes that is my purpose of just, I love learning about different topics and subjects. So sometimes that's the purpose, but then there's other times where it's like, no, I need to produce something. I need to be creative with this. So it's kind of like I do multiple purposes within one system. I think the self-awareness is really valuable there. Something I knew about myself getting started is that I could get really wrapped up in the process of note-taking.
And especially when I was getting started, when I've got like only two notes or whatever, my fear was that all of my time would go into like optimizing my organizational structure and that I wouldn't write much. So I put some pretty hard limits on myself. I was like, I don't get to do folders. I don't get to do tags. I don't like any of these things. They won't matter once I have more notes. But the first couple of months, I've got one flat folder. And the only thing I'm allowed to do is add new notes. And it can't be notes about note taking. I have to actually take notes about software.
because I was trying to introduce that as like a rhythm in my professional life was, you know, I want to be the kind of person that takes more notes. And I knew that all these other things could just become a black hole for me. So those are some things I set up pretty early on. And then as the note system grew, I was like, all right, now I can like, I've got a little bit of a novelty budget and I can allow myself to maybe start thinking a little bit in terms of tags or play around the structure a little bit. Maybe I have a side folder here where I put some different things, but I've had to put some pretty strict boundaries on myself around that.
Yeah, there's definitely been times where I didn't have those boundaries. Like I've taken, because I've already mentioned patterns, but that's like the patterns of different models and systems that people have, like building second brain, para, like projects, areas, resources, and archive, or other ones that are escaping my mind at the moment. But like I have a whole list of like different people's processes and like how they've approached either Zevilcaston or Obsidian or taking notes in different senses of the word, even analyzing different people's online digital gardens and taking notes of like, okay, what patterns do I see in some of their structures? How do they organize some of their... both the structure in the notes and the structure that's like if they have a sidebar, actual like hierarchy list of files or not. And so I've kind of like gleamed patterns from that and dug into like the details of like, okay, how can I organize this? And like, should I use tags? Should I not use tags? And so I've dug really deeply at times into the note-taking, like where do I even start? And it's, there's so many options that I think that's not where anyone should actually start when they start their notes. Like it's better to just kind of like let it build up its own structure and kind of guide it that way as well. Like give it enough structure so that you can get started, but don't overthink.
how do I need to structure this? Because you're going to change it multiple times. I've changed the structure of my notes a couple times, not too drastically, but enough to where I used to do more folders, and now I've kept it more flat. Not completely flat, because I still have some things organized into themes. So all development notes go into one folder, and there's a few additional folders in there, and that's pretty much my deepest nested level of folders. Everything else is no more than two folders. Here's my notes directory of all my permanent slash.
polished up notes and then the themes and then notes. And that's about the deepest level of structure that I have and everything else is just maps or indexes of content. Whereas previously I had played around with like Johnny Decimal and trying to come up with more of like an actual catalog system of like, how do I number these systems? And like, it was just, I was spending too much time on that. It was like, no, I just need to capture the notes and the structure can, it can just arise from how the connections are made. So I've over time, I've gone from
I've not abandoned folders altogether because I still find some value in having folders, but going a bit more flat has definitely helped over time. So all of this, I think, can be really inspiring to some of our listeners who want to get into note-taking, but it can also be really intimidating. What might be some concrete recommendations for someone who's getting into note-taking for maybe the first time? The basic thing is just get started. Start capturing notes. Don't worry about how it's organized. Have a bare minimum number of folders to have some sort of structure that makes sense to you.
If you need a model, there's a few mental models out there of how you can structure your system. But most of the core folders that I use, I have basically an inbox or anything that's just a scratch fleeting note of like, this is a work in progress. And then I have a notes folder where these are more polished, interconnected notes that I've worked on and distilled and curated into more of a structured format. And then an outbox where these are the blog posts I'm writing, the book that I'm writing, the projects that I'm working on that I'm applying to this to. Some other folders that I have are like...
context-based. So I have Thoughtbot folder in my notes where I just capture things throughout the day that relate to my work on client projects or Thoughtbot itself, or I have a home folder. So anything happening at home that relates to my notes, I'll capture it there. Whatever makes sense to you, what you're working on, create some folders if they're more distinct patterns, but don't overthink it. It's hard not to overthink when you're just getting started, especially because you need a little bit of like FOMO or like, oh, am I doing it right? And I think I'd like to say it's okay to have one single folder, just dump it all in one place and start from there. I've seen some Obsidian faults where everything's just, there are no folders. Everything's just dumped right there. They just started creating notes and started interconnecting them. And that's how their system works. Yeah, I'd agree that that's perfectly fine. If you're more of a systematic, like you need to know how things work. Like that's something that I love digging into. Definitely have fun and dig into it if that speaks to you.
Try not to get carried away by digging it to so far that it's preventing you from actually writing the notes that you want, working towards the purpose that you have for your notes. I think that's important, the purpose you have for your notes, having that self-awareness and knowing what do you want to get out of this. And maybe you just enjoy sort of nerding out over knowledge information structures. And in that case, maybe playing around with different systems and not writing a lot is the purpose. And you're fulfilling that purpose. Good on you. Enjoy that.
If the goal is you're trying to write a book, then maybe don't do more than you need to for what you're trying to write for your book. Maybe you're just trying to be a little bit more mindful about your day-to-day, trying to grow in that way. Bring in enough where you're fulfilling that goal. starting from a position where you know what you want out of this note-taking system will enable you to make the right set of trade-offs. I think another thing is if you're using Obsidian or an application like that, and there's a lot of plugins out there that can help you, but it's really tempting to start tweaking and working on your just setting up and configuring Obsidian to do all these interesting and intricate things. They don't necessarily contribute to writing actual notes and producing any sort of output. It's just...
It can turn into almost busy work of like, I'm going to tweak my system just a little bit more and get just a little bit more perfect. And then it'll all just fall into place. And so it's like, it's fun to dig into those things, but it's also... it might not actually be providing much benefit if you're just building the system rather than actually putting those tools to work and producing some sort of artifact or notes from your efforts. Yeah, I know for me, what I wanted was to write more. I needed something that would allow me to write more and more consistently and everything else is sort of downstream of that goal. Part of my goal is writing more, but I've also not done as much of that with my notes as I originally set out to do. So that's one area that I'm improving on recently is I try to actually write more blog posts or produce more output from that. But the other purpose I've also mentioned is that...
the learning process. So digging into different areas of programming or design or cooking, different special interests I have, and using the notes as that learning tool to learn subjects that I'm interested in. So the output in that case is more of just feeling more confident and proficient in an area that I'm interested in. Yeah, and that's a great thing to be sort of aware of. And you can sort of see that and see that your note-taking system actually has an improvement on your life in that way.
I'm a huge proponent of note-taking. I think it's improved my life. It's always fun to chat with other people who are excited about note-taking systems and sort of compare notes. So yeah, thank you so much for coming on the show. Yeah, thanks for having me. This was awesome. All right, on that note, let's wrap up.