Designing Data-Intensive Applications – Maintainability

We dig into what it takes to make a maintainable application as we continue to learn from Designing Data-Intensive Applications, as Allen is a big fan of baby Yoda, Michael's index isn't corrupt, and Joe has some latency issues. In case you're reading this via your podcast player, this episode's full show notes can be found at https://www.codingblocks.net/episode122 where you can join in on the conversation. Sponsors Educative.io – Level up your coding skills, quickly and efficiently. Visit educative.io/codingblocks to get 20% off any course or, for a limited time, get 50% off an annual subscription. ABOUT YOU – One of the fastest growing e-commerce companies headquartered in Hamburg, Germany that is growing fast and looking for motivated team members like you. Apply now at aboutyou.com/job. Survey Says Which sci-fi series is best? Take the survey at: https://www.codingblocks.net/episode122. News Thank you for taking a moment out of your day to leave us a review: iTunes: Kodestar, CrouchingProbeHiddenCannon Stitcher: programticalpragrammerer, Patricio Page, TheOtherOtherJZ, Luke Garrigan Be careful about sharing/using code to/from Stack Overflow! You need to be aware of the licensing and what it might mean for your application. What is the license for the content I post? (Stack Overflow) Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) (Creative Commons) Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) (tldrlegal.com) We Still Don't Understand Open Source Licensing (episode 5) Get your tickets now for NDC { London } for your chance to kick Allen in the shins where he will be giving his talk Big Data Analytics in Near-Real-Time with Apache Kafka Streams. (ndc-london.com) Maintainability A majority of cost in software is maintaining it, not creating it in the first place. Coincidentally most people dislike working on these legacy systems possibly due to one or more of the following: Bad code, Outdated platforms, and/or Made to do things the system wasn't designed to do. We SHOULD build applications to minimize the pain during the maintenance phase which involves using the following design principals: Operability – make it easy to keep the system running smoothly. Simplicity – make it easy for new developers to pick up and understand what was created, remove complexity from the system. Evolvability – make it easy for developers to extend, modify, and enhance the system. Operability Good operations can overcome bad software, but great software cannot overcome bad operations. Operations teams are vital for making software run properly. Responsibilities include: Monitoring and restoring service if the system goes into a bad state. Tracking down the problems. Keeping the system updated and patched. Keeping track of how systems impact each other. Anticipating and planning for future problems, such as scale and/or capacity. Establish good practices for deployments, configuration management, etc. Doing complicated maintenance tasks, such as migrating from one platform to another. Maintaining security. Making processes and operations predictable for stability. Keeping knowledge of systems in the business even as people come and go. No tribal knowledge. The whole point of this is to make mundane tasks easy allowing the operations teams to focus on higher value activities. This is where data systems come in. Get visibility into running systems, i.e. monitoring. Supporting automation and integration. Avoiding dependencies on specific machines. Documenting easy to understand operational models, for example, if you do this action, then this will happen. Giving good default behavior with the ability to override settings. Self-healing when possible but also manually controllable. Predictable behavior. Simplicity As projects grow, they tend to get much more complicated over time. This slows down development as it takes longer to make changes. If you're not careful, it can become a big ball of mud. Indicators of complexity: Explosion of state space, Tight coupling, Spaghetti of dependencies, Inconsistent coding standards such as naming and terminology, Hacking in performance improvements, Code to handle one off edge cases sprinkled throughout. Greater risk of introducing bugs. Reducing complexity improves maintainability of software. For this reason alone, we should strive to make our systems simpler to understand. Reducing complexity DOES NOT mean removing or reducing functionality. Accidental complexity – Complexity that is accidental if it is not inherent in the problem that the software solves, but arises only from the implementation. Ben Moseley and Peter Marks, Out of the Tar Pit How to remove accidental complexity? Abstraction Allows you to hide implementation details behind a facade. Also allows you to reduce duplication as abstractions can allow for reuse among many implementations. Examples of good abstractions: Programming languages abstract system level architectures such as CPU, RAM registers, etc. SQL is an abstraction over complex memory and data structures. Finding good abstractions is VERY HARD. This has become apparent with distributed systems as these abstractions are still being explored. Evolvability It's VERY likely that your system will need to undergo changes as time moves on. Agile methods allow you to adapt to change. Some methods include: Test Driven Development Refactoring How easy you can modify a data system is closely linked to how simple the system is. Rather than calling it agility, the book refers to it as evolvability. Resources We Like Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems by Martin Kleppmann (Amazon) Grokking the System Design Interview (Educative.io) All permutations of English letters with a handy search feature (libraryofbabel.info) What is the license for the content I post? (Stack Overflow) Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) (Creative Commons) Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) (tldrlegal.com) We Still Don't Understand Open Source Licensing (episode 5) Streaming process output to a browser, with SignalR and C# (Coding Blocks, dev.to) Googlewhack – A Google query using two words, without quotes, that returns a single search result. (Wikipedia) The Website is Down – Sales Guy vs. Web Dude (YouTube – Explicit) Software Design Anti-patterns (episode 65) Clean Architecture – How to Quantify Component Coupling (episode 72) Static Analysis w/ NDepends – How good is your code? (episode 15) Static Analysis of Open Source .NET Projects (Coding Blocks) Does Facebook still use PHP? (Quora) Tip of the Week Need to move data from here to there? Check out Fluentd. (fluentd.org) Use NUnit's Assert.Multiple to assert that multiple conditions are all met. (GitHub) Turns out C#'s where generic type constraints are even cooler than you might have realized. (docs.microsoft.com) Best Practices for Response Times and Latency (GitHub)