What up gophers Jared here. This episode is different than what you're used to hearing from go time We've been clipping highlights of the show for a while now to share on Twitter and YouTube and a side effect of that effort Is I have a bunch of awesome clips just sitting on my hard drive collecting digital dust So here's a beta test of a best of style clips show covering the summer months I hope you enjoyed it You might if you missed a few episodes or if you listen distracted sometimes like I do Please let me know what you think in the comments if people dig it will probably do this more often if not Well, I'll just pipe the whole thing to them all. Oh, yeah And since it's a special kind of episode we have a very special intro song for you here we go Being with her changelog is provided by fastly learn more at fasta calm We move fast and fix things here at changelog because of roll bar check them out at roll bar calm And we're hosted on linocloud servers head to linocom slash changelog This episode is brought to you by digital ocean droplets managed Kubernetes managed databases spaces object storage volume block storage Advanced networking like virtual private clouds and cloud firewalls developer tooling the robust API and CLI to make sure you can interact With your infrastructure the way you want to do the ocean is designed for developers and built for businesses join over 150,000 businesses that develop manage and scale their applications with digital ocean head to the Dio.co slash changelog and start with a 100 credit again Dio.co slash First up we have a panel only show Yana Matt and John discuss how to effectively work with databases in episode 132 The trouble with databases. It just so happens.
This was the most popular episode of the summer The way the database works or the way it models thinks there's a lot of things in common with the storage engine So the way you store the way you shard the way you know Really partitioned the data is a lot to do with the type of like capabilities. It provides the query So from a you know high level perspective, it's always important I think for a user to understand how it like some sort of like a lower layer things are stored So you can like estimate what is feasible? What kind of like use cases are actually a good fit for that type of database even though it sounds like a bit of work I really like such as people to take a look at like, you know What type of like usage use cases make sense and in the end of the day like at the storage level? What did I do before evaluating anything?
The classic example of that that I've heard is that I've been told that it's stripe one of the common things They've done is that they have a no-SQL database that they're using for all the you know really high-speed transactions But then on the back end when they run analytics and do all these other things It's really hard to do that and a lot of times people want SQL You know they want to be able to use some tools that you sequel for that So they actually take a lot of that data and translate it into a SQL database and while it's delayed It's only used internally, so that's okay So like they're taking that trade off and deciding it's useful to have this data in both formats and you know It's like you said they didn't switch from one to the other It's more of a this makes sense for this use case and we poured it over to this for another use case Yeah, in my experience, I'm seeing always like two or three databases, you know in a system like you can't really fight the trade-offs You know you get benefit from them differently There's usually a relational database and other database for their housing reasons like in all things and so on And then there's usually a database like or something like elastic you know for search reasons So you know you can at least like list three core data resources Yeah, and of course backup could even be a different one where you're taking taking backup in some kind of cold storage or just less active Place it's common I think for developers to want to get the perfect solution from the beginning and just build back But probably a better strategy is to just start with something one thing simple do what you're gonna do with it There are like three things you can have the you know in distributed systems You can't have three of these things you have to pick two and those three things are represented by cap Which is CAP cap means consistency. Sorry a means availability p means partition tolerance Like you know network partitioning tolerance and what he says is if you want a hundred percent consistency and a hundred percent Network failure partitioning tolerance. You can't have a hundred percent availability You always have to have make that sacrifice You know we're talking about relational databases as well as like stimulus no single type of databases actually Relational databases are more like CP systems. They have higher consistency.
They are more to learn the partitioning on the other hand No, SQL databases are you know compromising from consistency their eventual consistent, but they provide higher availability So they are AP systems. So if you have this like mental model I think databases are becoming easier to understand because you know that like there's a limit like there's like physical limits To the world and you can't have it all I've worked for some project managers that really just don't agree with this. They want all three Yes, because you know, it's just hard to explain. It's almost like spender is actually a typical CP system It has a hundred percent consistency and it's very tolerant to partitioning But it's availability is significantly higher than any other relational database Which is it provides five nines of availability which means like five minutes downtime a year You know that's like amazing like most of the relation database require 10 minutes or whatever a month for maintenance and so on Or if you want to upgrade the you know, the the schema it requires downtime or the failover It requires downtime.
So how did this happen? Like you know the spender team kind of says that they're beating the cap theorem because they provide us like high availability And it has a lot to do the way how the internals of this distributed system is working plus our good networking infrastructure So we're just kind of like you know improving the availability not to a hundred percent We're still talking about five nines, but five nines is actually a lot in practice. So, you know, our goal is like Maybe you shouldn't make as many as compromises We will try to you know provide you a higher availability But you will still have the like transactional relational database But at the same time we have a lot of limitations around like, you know the type of the schema limitations for example some SQL limitations I actually was reading Martin Kleppmann's book Maybe like two months ago and then in my dream I saw myself right in that blog post and as soon as I woke up I took notes like I think I dropped the 10 items on the cover of that book So that block was actually like probably came from some of the you know the ideas that I got from his book and so on So that's really funny that like, you know, I saw that like the article was being very useful in my dream And it turned out to be true. It's so funny.
Well, if people didn't feel stupid before the fact that Jana's coming up with this stuff in a sleep He's certainly gonna do that. That's how Paul McCartney wrote yesterday by the way. He just woke up and had the song I dreamed it was way less productive. Yeah, I just dreamed my legs were made of jelly That's not healthy I can't say no to blog post probably could I might on episode one thirty five John was joined by Chris Brando Ben Johnson and Aaron Some of their mistakes and how they learn from them One of the mistakes I made when I first was jumping into go was that I just felt like I over planned or I tried to like Overoptimize for getting things perfect So you'd read about how you shouldn't use MVC and you shouldn't do all these different things and I sat down on the guard I'm gonna write this side project.
It was a side project wasn't like my main work stuff So I was like I'm gonna build this thing and it's gonna be a good go code And I think I spent so much time rewriting some stuff because I was like Oh, this is a bad way and I realized why and then I'd go back and rewrite it and in the end I'm pretty sure what I ended up doing was just using a simple MVC model and just got it done and then later I was able to come back and tweak things and adjust but it just I wasted so much time trying to like meet the expectations of everybody I guess is how I put it and to me that was a big mistake because I feel like you learn more by just kind of jumping in and Doing stuff rather than like trying to find the optimal path I guess kind of long ongoing mistake in my career is just not understanding the underlying technologies that I use Especially early on like using like an HTTP framework like web framework You know, just assume that you do but like understanding that framework and maybe even like layers on top of that framework is a lot of times more complicated than just understanding HTTP or whatever the underlying technology is So I feel like I've done that for a long time in my career But more recently just trying to step back and just understand like, you know What are those frameworks actually give me what do they add and just a lot of times the net HTTP is you know enough with a router So I guess first off are you talking about going all the way down to actually understanding how like TCP works and going even below that or I think it's a trade-off of like what do you get for the abstraction that you're working with like I'm I'm not pushing bits across an ethernet Court, but if I can you know understand like, you know Just basic headers and what they do rather than having some other library on top of that actually, you know Set certain parameters. I feel like just understanding that kind of underlying HTTP RFC or some aspects about it It helps me just to write more direct code I feel like if you go on GitHub and you look at some popular open source projects I feel like that's sort of the the Instagram of programming and that you see the perfect result after everyone's You know put on the makeup and a rolled camera and everything and you know that we put that on ourselves Whereas you know like Ben you just said you can just go in and write for loop and it'll be good for a while Just you got to start somewhere and there's always the telltale sign of like initial commit has like Twenty source files and four thousand lines of code and you're like that was your initial commit It's like that clearly was not your initial commit. I like to just kill the get report or the folder and just restart after probably A couple hundred commits in on the same way. It's like I've got the initial version and then that eventually gets to something I'm like, okay I'm okay with sharing this and then delete you know and just go from scratch on one hand I get why you do it because you don't necessarily want that bad history there But on the other hand it is kind of hard for somebody jumping in who actually thinks that might be the initial commit Next up one thirty six This is when Matt joined the show as a guest with his pace co-founder David Hernandez to talk about the technical decisions They made while building pace with go Yeah, absolutely and I tell teams this when I talk to different teams as well about that when they're choosing their technology That's a big thing You know gRPC for example might be the perfect choice from a purely technical perspective But if nobody on the team has experience with gRPC, then there's a learning curve there And some people talk about them in terms of innovation tokens and things, you know You're not allowed they say to just all the technology can't be new and unfamiliar You can do some of that But there's effort and there's kind of a cost to being productive in any of those and we have that already on the front end because we haven't done Much front-end work for a while and we knew we wanted to use a it was going to be a rich front end So we knew it had to be somewhat of a model So you weren't trying to emulate gRPC basically you wanted to you get some ideas from how sort of it works And you stole some and I use that very generous So you stole some ideas some some commutation details rather than showing the sort of issue the whole you know binary Formatting just went to playing JSON basically you saw your own problem in a sense and rather than sort of bringing in something for the State of because it's cool.
Yeah, it's stealing from open source. It's not really stealing It's just just kind of rubbing hood Yeah, different sizes comes with different problems Speed is different. That's why people try to put things like microservices not because the microservices are better than only because it's easier to control The size of the thing or the responsibility things like that in this case is everything is easy in that sense because we are two people But we became from Full stack developers to full company developers. We do support we do marketing we do Accountancy, so it's not only the take a stack is quite wide in that you have to worry about a lot of more things Yeah, I remember that release manager wanted to do like two releases a month and then being synced with everyone And they asked how many times we were releasing and it was that day was something between 10 and 20 or something You know, it was a very different mindset of rapid as soon as it's a bit better than it was, you know We want to get it out on Episode one thirty seven databases were back on the table This time you on Ron Horst joined Johnny Matt and John to focus in on Postgres Big thing that's a postgres has a very super light in go specifically is really good library support Unfortunately, the sequel like the driver that everyone uses is a sego driver and as mostly you probably know sego means building with sea It means longer build times it means less portable binaries and stuff like that So in go with postgres we actually have several different pure go libraries to speak with postgres Which is really great.
So that's just one reason to use postgres But other things are such as the stability of the software for example as you say it's over 20 years old It's been used by thousands of companies worldwide, you know, it's not gonna just you know corrupt your files because Those parts have been ironed out by now. So within databases I think you often say you don't want to use something that's less than 10 years old because like the state that needs to live for a Long time you want to make sure it doesn't you know corrupt on the disk or whatever and postgres is one such stable mature solution It's also very fast because it's a redundancy and sees fast You know, I was sitting here knowing along to what you were saying there that start with postgres Because I think a lot of users come into the space knowing what to use and those kind of hope with my talk Just giving you an instruction to something you can use some opinionated tips on what libraries to use and so I'm giving you Like a head start just getting started with actually like moving away from the problem of choosing a technology and actually using a technology to solve your problems I really agree with that sentiment that a lot of people will try and you know Google What should I use to store my data or even like read some popular programmers blog Where they tested out the latest technology and found it to be really good for their very specialized use case And then apply that to like all of their problems because that's all they know and you know starting with something well tested Well established like you say good development share like postgres will take you very far before you need to change How do you actually use postgres and I'm even thinking like in the context of say a website that is hosting a blog At what point would you make a connection to postgres? Do you tend to make one connection per instance of your code running and then create sessions off that or does each handler Would each handler make its own connection? How does it actually work?
You know from a Go developers point of view. Yeah, so if we take a step back and look at the standard library database SQL package That actually has a connection pool built in so if you come from another language like iPhone or Ruby You might be familiar with putting something I think there's a postgres band or something like that which just connection pool in between your database and your client can go You don't need to do that sort of thing because it's already built into the standard library So what you would normally do when you connect to the database is just create a single SQL DB handle And then that's safe for current use so you can use that in all of your handlers even though they're being called from different Gourgings from different clients and you can also configure things such as a max connections on the SQL connection But normally that's all handled by the rest of the libraries. You don't really have to worry about it Which is really nice Actually, I have never had the pleasure of using an ORM. So what am I doing on the start talking about databases?
I I would very early my career I was kind of persuaded against using ORM because naturally as a beginner program I was like this is cool But I never really had to use it because I was told by someone who knew better than me that that was a bad idea So here I am saying to other beginners like myself once was that you should try not to use the ORM even though It looks really appealing at first. You should probably just learn to use SQL and it's really not that bad and you'll love it I think like I do I would agree with that having company other side because everything specific to rails that I learned is Useless to me now and everything I learned about SQL along the way when I couldn't get that to work is much more useful to me And it'll carry over to any language. That's right. Well, that would have been an unpopular opinion section, but everyone agrees It's not that unpopular.
It's not happened before we all agree any to ask. Have you met Bobby tables? So the reference that Johnny's making there is to an xkcd comic which has a very illustrative way of showing just exactly what SQL Injection means where there's a school principal who's making a call to a concerned parent I suppose asking about their son Bobby tables that dash drop table students or something like that the parents says yes I'm going to be tables and then the principal says well, I hope you're happy We've lost this year's student records. It's really funny because of the implication there is that they have to enter their Kid's name somewhere in some sort of form or something and they thought hey It would fun to see if this is vulnerable to SQL injection And then they put in their command that would if it was one of those people injection drop the table called students And of course the joke Then is that it actually did and principal is fears about it and the kind of lesson I guess is that you shouldn't have been vulnerable to SQL injection No, he's got no job though.
That's no good telling him that is like his life's devastating But yeah Episode 138 was quite a departure and inspiring to many listeners So we hear Jackie Grindrod and a Sean Carter join John to talk about their first week with the go programming language You know if somebody's trying to learn go what do you suggest? I think Jackie you said exorcism is that where you would suggest people start so I found it worked really well for me because You put your solution in you could see how other people solved it But you'd also have mentors come and comment and the really nice thing for me about the mentors was that they're really good at giving like iterative feedback So give me like some feedback and be like here's how you can solve this a little bit more elegantly And then I'll do that and then they come back and be like okay So now you've done that here's like another step you can do until we got to like that final iteration And I do have all those commits like tracks So I can kind of watch the way that I grew as I was learning and the feedback I got so I would actually recommend that because it was a very nice experience for me You got to do it and you also got like professional feedback from people who do this which you might not always get if you're Just learning a language on your own the front of learning things an advantage that I didn't really realize like I knew I had it But I didn't really consider it until a conversation I had I think last week or this week was about concepts like garbage collection I would have learned that in my computer science classes and there are things that I just kind of knew about going into this That I didn't really think about as in terms of like when I'm learning it on the job in an obstacle Maybe that's not something that just gets presented to you It might be like those kind of concepts that you need to learn outside of just the code and that's kind of been interesting Just figure out like how do you review either like best practices across different languages or different architecture patterns? Which is not my strength but try to figure that out while learning the language can be pretty overwhelming and just figuring out where to start But yeah, there was people that were consistently there and it was nice It was kind of like having this little support circle that I really didn't expect this like I expected streaming to be more like people Coming in and being like hey, you're doing that wrong like hey, why didn't you do this? So it was it was really nice.
I'm gonna paint a picture This is very similar like I said every rep counts up into the gym You guys been to the gym maybe not lately, but it's kind of the same experience Like you definitely went out and you kind of said hey, I'm gonna go to the gym I put on a schedule and I'm going and maybe I have a gym buddy. I think everything goes better You know, you're pairing you had some accountability and I basically said I'm gonna buy some weights And I'm gonna put it on here and this was my path I didn't put any like deadline or I didn't have any severe accountability as kind of digging it in But it was my number one focus, but still not only is it gonna be sticky for you I think that accountability. Yeah, I think you probably went further faster How much time does your team spend building and maintaining internal tooling? I'm talking about those behind the scenes apps the ones know what else sees the S3 uploader You built last year for the marketing team that quick firebase admin panel I let you monitor key KPI's maybe even the tool your data science team act together So they could provide custom ad spend analytics now These are tools you need to build them and that makes sense But the question is could you build them in less time with less effort and less overhead and maintenance required and the answer that question is yes That's where retool comes in Rohan Chopra engineering director at door dash has this to say about retool quote The tools we've been able to quickly build with retool have a lot of manpower and scale our local operators all are reducing the dependency on engineering and quote now The internal tooling process at door dash was bogged down with manual data entry Missed handoffs and long turnaround times and after integrating retool door dash was able to cut the engineering time required to build tools by a factor of 10x And eliminate the error prone manual processes that play their workflows They were able to empower back at engineers who wouldn't otherwise be able to build front-end some scratch and these engineers were able to build fully functional In retool in hours not days or weeks.
Your next step is to try it free at retool dot com slash change log again retool dot com slash cheat log Next up testifies maintainer muy on superchove joins matt and mark baits to talk about testing frameworks and go Because you literally put those things as strings into the test code Yeah And then from that you can generate some quite nice looking failures if if something fails You can you know it reads quite nicely But found it to be too verbose actually and just saying you know not equal and showing you the two values or something was just Easier to see the thing that I like about BDD versus unit testing actually has to do with those kind of the names of the tests Hmm, you know when you're writing a simple test right you know test that create does something right? That's a pretty simple test name But when you start having all those weird variants right then the string based text names become really really useful You know when you can when you just need a little bit more description as to what it is You're trying to test and that's harder to do in a unit type of a test where you have a function name Have you seen a property based testing as well? This is a another kind of Style yeah, it's an almost like fuzzing for your functions where you specify not what values to test with but what types of values your function takes a simple and Obviously also what kind of output you expect and then the property based testing framework of which one example is a copter Well, then just generate You know whether it's random or in some specific sequence will generate plenty of values to the integer function and try and find H cases for you that do not conform to a specification That's a really cool isn't it? Yeah, the first exposure I had to that was with hypothesis in Python and it was pretty awesome I mean an obvious problem there is you now have especially if your functions are a bit more complex that it can take a while for Test to run because you're now running the same function 10,000 times rather than one sort of five times But you know for pure functions where you know where your code doesn't have side effects Yeah, it's very good at finding H cases and and little behaviors that you didn't anticipate in Ruby We had a great thing called time cop when it was such a random Ruby thing where it would override basically time not now to be whatever Because you could just override anything so you can say like I want to be three weeks into the future and time not now would return three weeks in the future Great Scott.
Oh It was spectacularly awful and good and fun. Yeah, it was one of these you can only do in a dynamic language. Yeah, it's also reason I'm not using a dynamic language No, the way I look at it at least is if it's something that's How likely is it to change basically how likely is your database state to change how likely is your time to change if it's anything other than low Take it out mock it out make it as a dependency. That's injected But that's a rule of thumb that I follow anyways Episode 140 was our most anticipated show of the summer the latest draft proposal for generics was posted in late June and it took us a few weeks But we managed to get Robert Grissommer and Ian Lance Taylor on the show to answer many of the community's nagging questions about the latest proposal Robert and I released the updated design draft or moving forward with generic the biggest change was that we dropped the idea of contract and Just decided that instead of having a separate thing tactic construct Which was a contract that we could just use interface types to describe the contract between the type argument and the type parameter A lot of people looking at contracts had seen that they seemed a lot like interfaces and people had trouble sort of separating out exactly when you Were using contract and when you were using interface.
So we simplified this and this was just add almost entirely to Robert We simplified this to just use interface types And then the second big step we made was we've released a translation tool and a type checker So we have a type checker that works for the design draft the description of generics in the design draft So that gives us you know some confidence that what we have written about can actually work And we have a translation tool which translates code into where they are go The translation tool is not by any means a final thing. There's cases. It doesn't handle It's just an experimental tool But let's people actually write code that can actually run using generics So we can get a feel for whether generics actually works for people and whether it actually addresses the issues that they have So more practically for the Go community When do you think that you're going to get enough feedback to move forward with moving from a draft proposal to actually putting it forth as a proposal To change in the language. Yeah, we don't have any timelines in mind I'd say as we mentioned earlier We're still trying to pin down some of the precise semantics which I don't think is going to affect any existing But I'm sure it's not going to affect any existing code We want to make sure that we understand it We want to make sure that you know the multiple Go compilers will implement the same thing We're gonna have to have some sense of how to add the language spec So those are the steps we're looking at now I mean we're certainly gonna move forward as fast as we can for making a formal proposal or at that time none of it will be a surprise People have seen all of the ideas already and we'll just have to see how it flies So far I feel like the reactions are largely positive, which is encouraging But I don't know exactly what the time I'm going to be.
My experience with generics was maybe C++ with templates And probably the highest point there was when I was able to as you alluded to before it's doing complete I was able to write a program using C++ templates that we decided whether a constant was a prime number or not And the compiler we decided at compile time So that's not the kind of thing we would like to support with respect to you know What would I like to see or not see? Honestly, I'm worried about the kind of code that people are gonna write There's no question about that and we see some of the examples that people send us they cause crashes and a prototype And they are just unbelievably convoluted and really really hard to decipher But as other people have pointed out those people are really pushing the envelope They're trying to just see what can I do with the same and I hope this is not gonna be you know The kind of code that people are gonna write down the road I think one of the first things we need to do if we have this for real We need to come up with a kind of best practices guide that guides everybody a little bit as to how you should use generics And when you should use them and when you should not use them very similar to what we Developed for you know go teens and channels in the early days of go Everybody was using go teens and jobs for everything and it took a little while for us to learn where it was appropriate That very wasn't the experimental tool has no similarity whatsoever to any real implementation So I we know it's low and it's kind of it's so and that's just inevitable if this does move forward to become a proposal And it gets accepted then we most likely the implementation will be to start with a branch of the main go-to chain And we'll start adding generic support on that branch which will involve changing compiler mainly And any other changes to other tools that are required and so that'll be the time to start giving feedback about changes to build speed We've talked about it with some of the compiler developers I like keep Randall especially and we think we can do it without a significant increase in build speed I mean there will be some increase in those we don't think it's going to be on huge increase But you know this is really speculative at this point So the time to give that feedback is when when we're able to start doing development And hopefully probably people will also be able to contribute work when we start doing that work on the public branch Programming language evolution is really a social process Okay, it doesn't actually matter if you have if you have seen the light and you know exactly the perfect language You know and you would just put it out there And you know really it's 20 years ahead nobody would even buy it because people would not see the reasoning while you got to that point And so you really have to get everybody along and some people may already be where you are and some people may not But you have to get everybody along in little steps And that's how we eventually end up where we want to be and we can see this with all kinds of things like a coverage collection A coverage collection was invented, you know, in 1950 something with lists, you know, the first list by coverage collection 1958 I believe and It's taken forever before it became accepted as something that the programming language should you know, a mainstream program I should have maybe Java was the first one that really made it mainstream and now this is not something that is I mean Still disputed or debated I should say but it's not as outrageous anymore until I think that's true for other things On episode 141 guest, Daniel Marti helped Matt and Johnny explore goes encoding JSON package Yeah, and it's JavaScript object notation, so it comes out of JavaScript But it turns out to be really kind of useful across any every language really has now some kind of JSON support Hmm, it's practically everywhere practically every language out there That's modern today has to have JSON support because you just do and your computer might not see it But it definitely is running JSON. That's a level. Hmm.
Yeah And so there's like it's an object and it has fields and those fields have some types And it's the types that we're used to as well in go like strings and numbers and ballons and you're this other objects arrays Those kinds of things I think that's that might be the whole list and why did it get such popular use on the web? I mean, it kind of is kind of perfect, isn't it for web technologies I would say it came from the from all the success that browsers had, you know the modern web had and you know suddenly HTTP HTML CSS and JavaScript and JSON all these technologies kind of took everybody by surprise initially everybody thought they were Just toys, but now suddenly people are building real companies on top of them and JSON is just you know has too much momentum I don't think anything is ever going to replace it at this point Honestly, and I have mixed opinions and feelings about all the third party JSON implementations out there I think some of them do make sense for example one use case is you do absolutely want the most performance you can get because maybe This is a bottleneck for you and you don't mind go generating some code to them, you know, right? Generate automatically a decoder for you for JSON So you can use packages like easy JSON for that Which is pretty popular and the trade off there is you have to run go generate and your binary is gonna Wait quite a little bit more because it has quite a lot of extra code, but that extra code It just encodes all the logic directly in binary code in machine code. So there's no reflect.
There's no references There's no extra work involved. So I think that's clearly one of the cases where it might make sense for use case I like how you framed that while you're saying maybe it's a bottleneck in your case and that's the thing It's like once you've seen that this is a place where an improvement is gonna make a difference for you Then it's worth taking on the extra pain whether it's complexity or learning a new API Whatever it is. I like that approach because well I think it's what we should always be doing, you know as you alluded to Johnny We kind of can get a bit obsessed with why wouldn't we want the fastest possible thing? And the answer is it might be good enough just using the standard library There's some bugs for example There's one that I would say affects most code bases out there Which is the standard, you know, you have an HTTP endpoint and the body is JSON so you want to decode it So what you do is you take the R to body and you do JSON type new decor New decoder dot decode with the body and then into some structure and if you do that It's buggy if you just do that.
I've just got to go What do you mean? It's buggy. Tell me why please so this was found by Joe one of the maintainers I want to say about a year ago and the bug is the decoder is meant to be useful for streams of JSON values And that is for example when you do go test with a JSON flag It's gonna give you a new line separated stream of JSON of JSON values of JSON objects Yeah, that's kind of how I was using it in those tools I was talking about Exactly in a way it is kind of streaming in a way like takes the reader for each object It buffers it I guess but it discards that previous object, isn't it? Yeah, yeah next time Right, so in a sense it's streaming it appears to you as a streaming but internally that's not it's not what it's still doing It only one object at a time which you could say is a stream.
It's just if it's a great big fat object, but exactly in trouble Episode 142 was all about that infrastructure with special guest Shabak Shajalan Maybe we should bust some of this jargon because you know, you hear infrared I hear systems engineering and I hear DevOps Do we agree at least on what these terms mean? No? It's kind of funny. I mean let's take for example one of the hot new titles under SRE, right?
And you'd think that like that would care some sort of consistency right from organizations organization. That is entirely not the case Right, that's on so I'm sorry. I do go is gonna be very different from SRE at Salesforce Just be very different from SRE as Microsoft. Yes, there is a through a lot of these things and that goes with the other titles as well System engineering, you know DevOps engineer.
I crunched a little bit about say that Yeah, basically these things are gonna mean different things and different organizations even a different sort of an over the lifespan of an engineering team Right that the definition of that role may also change, right? So it's it's not I don't think only there's one sort of solid definition right of what you know software engineer database engineer infrastructure in your ops Whatever it is. I think it's gonna be different everywhere you go. Yeah, I thought SRE was tech speak for sorry.
That's how I That's exactly what I ran into as well. It's a chicken and egg problem You need experience to get a job but to get a job you need experience So like where do you actually start because that is a very hard limit to like how much you can do and learn on your own When it comes to a job of this type, especially if you want to work at scale If you simply cannot replicate the kind of things you'd be doing every day at home So why is that and it's such a big missed opportunity? Like I'm seeing a lot of like people without a lot of experience coming into this field and like looking things from a very like new perspective Yeah, in my experience They've been really much more effective in terms of like pointing out the you know the core friction points than the experience engineers I think experience people have a lot of like you know Like they're accepting the current status and they always assume that like there's all this like there's all this Like there's all things that you know you have to satisfy in order to kind of like provide but is a new in experience engineer just coming in my questioning some of these like things more carefully and That's sort of like the perspective that we are missing an infrastructure in general because you know There's no good connectivity and we're just doing a bad job in terms of hiring people into this area Yeah, I completely Last but not least on episode 143 Francesca Campoy and Isabella Rettlemeyer join Matt and Yana for a deep dive on Go's context package Like there's a very straightforward way of cutting down the latency like the tail latency in your requests So if you have a request Let's say that you have a request that you send into a server and it takes 5 milliseconds 99% of the time But then there's 1% of the time that it takes one minute, which is not good So what you could do is make that call multiple times and you're gonna do it with cancellation So you're gonna do context with cancel of your regional context And then what you're gonna do is you're gonna use the same context for all of those calls and have defer Cancel at the top of your function and then as soon as any of those values returns and your return from that function the rest will be canceled And that's gonna take down your 99% out from like one minute down to five seconds So that's like small things that you can get a lot of performance Especially when you're using a server that is not something you manage so you cannot go and complain I'm complaining to the end of like hey your 99% the latency is awful You can still fix up like doing this little hack. Why is there context ought to do?
Mmm. Great question. Yeah, I don't know So Context ought to do and context of background do exactly the same thing they return an empty context in an empty context That doesn't have values doesn't have timeouts. It doesn't it never get canceled So it's like literally an empty struct.
That's what it is So the interesting thing is that when you return a background what you're saying is that oh this is something that I'm starting from scratch So you're basically saying there's no previous context is something that I'm creating right? So for instance in the example of the CLI you're running your your CLI and at the beginning there's not previous context or anything Like maybe at one point we'll have actually a context coming from signal that will be an interesting thing But otherwise we don't have anything. So so you would call background context ought to do is actually was added just so as Different functions like you need to start like you're creating a tree of functions that are calling and passing context around So how do you do it if you want to add it to all of them? But little by little if you start from the top it's gonna be you can not pass functions until they're accepted But if you do the other way around like you build a function that starts by saying Oh, I accept a context now and you can pass a context to me then the caller could say Oh, okay, so I should have a context, but I do not have it yet So instead of calling context of background which implicitly says I do not have a context and I will never will So to do you just hey, I don't have it yet, but let's fix it later So it's literally just so when you grab to do you can find where you need to still do more work Anything that's gonna cool like the fact that they thought about these Otherwise, I mean you could have done the same calling context of background and then having on top like I comment going to do Is that real context?
Yeah, they did it this way. So it's more explicit and you actually could do code analysis and like hey This is not done yet What's up, gophers? Are you looking for a way to instantly debug and troubleshoot your applications and services running in production on Kubernetes? That's a mouthful Well, pixie gives you a magical API to get instant debug data and the best part is this doesn't involve changing code There are no manual you eyes and all this lives inside Kubernetes pixie is an API which lives inside your platform Part is all of your data that you need and exposes a bunch of interfaces that you can ping to get data you need pixie is essentially like a decentralized Splunk it's a programmable edge intelligence platform Which captures metrics, traces, logs and events without any code changes And the team behind pixie is working hard to bring it to market for broad use by the end of 2020 But I'm here to say I can get your hands on the beta today links are in the show notes So check them out so you can click through to the beta and there's like community Once again links from the show notes check them out and look forward to pixie day coming soon Of course, no go time hits collection would be complete without some unpopular opinions in the mix Let's finish up with a whole package of them logbot roll that unpop jingle Do you want to hear the breakmaster say under mix or the rocker-peller version?
Let's rock that rocker-peller version. Why not? I'll find selection if I do say so myself rolling on god rocker-peller enjoy I don't know. I feel like most new technologies are just not necessary.
I guess with my unpopular opinion Docker honestly like Kubernetes a lot of those I feel like you can run You know a stack but you can run basically like where we used to run back in the 90s or whatnot Or early 2000s where it's just like here's a web server running you can run a business off of that Probably be fine. I mean obviously back up your date, but that'd be my unpopular opinion I think for me an unpopular opinion I have is I guess what your organization or your I guess the engineering organization reach a certain level That you shouldn't really just take software from other companies Like I think gRPC is like a big one for me Where's just like once you have a group of people that understands how like build things with DCP and HTTP You could probably just build it yourself and you should do that because your organizations need to be very different than like What Google needed for when they built your PC So I think it's like I fall closer to that what used to be called not invented here syndrome And I think like that's probably where we should be edging back to but I realize that that is wildly unpopable A lot of people and they usually say I'll just use whatever's out there. That's better Well, I mentioned REST APIs earlier. I don't think they're good.
That's my unpopular opinion I think they cause more confusion than problems they solve actually an unpopular opinion I have is you know should try and work in tiny teams a lot of the problems when it comes to Software engineering come at scale and that's not just code scale But people scale so if you can have tiny little teams working on a problem And you can do this within bigger teams like just literally as two or three people you are now a new little team You can be so effective in such a small group Because you cut out a lot of the work needed really to marshall the team You can't always do it and it sounds a little bit antisocial, but that would be my own popular opinion tiny teams I have this library that I like to use which is called a squirrel and it's a Query builder and it uses the builder pattern Ha everyone hates the build button and go right and for good reason because the build button doesn't work well And typing that provides because we don't have generics and squirrel suffers from this problem as well But it's also provides a lot of power at the same time So it's like the one exception to the rule of don't use builder The builder pattern anywhere is for query building use the squirrel package because it's really easy to use and constructing queries with So the builder package then just to just for anyone not familiar This is where you get these fluent apis where every method returns the main object itself and then change your problems Yeah, right and I should say I hate these because you can't define interfaces that work with them like at all It's just a nightmare. So I do have one question Johan Could you reconstruct the swirl package using like functional options? Potentially So do either of you have something you'd like to share I have one I think that pair programming is an unpopular opinion I did it early in my career took a lot of time off like companies a lot of companies are afraid to adopt it I work for coming that I think does it really well But I still think it's an unpopular opinion that pair programming my programming I think that's the way things are to be done if you want to be successful teacher I think the closest I have is that uh, hopefully it's not controversial here exciting I said but the documentation contribution is contribution that is as important and sometimes more important than just the actual code And I see a lot of that where people always kind of drop the ducks That's close thing I have right now, but you know, it's important it builds up that How do we bring people into our circle that's building this tech and expand and then be able to build better things together? So hopefully not too much about take here, but And yeah, let's fight the fight like keep preaching it.
I don't know if it's not popular opinion But you know, I like short identifiers I do and uh, I feel like The closer they are to where you use them the shorter they can be And the further away they are from where you use them the longer they should be And then there's some exceptions like when an identifier is really really really important in your package and prevalent Then it can be one letter even if it's a global and you know, the most prominent Example for that is perhaps testing that tea. I'm not sure if you're allowed to be a teacher now Why don't every teacher like expects you to write really long like explain it like self-explanatory variable names regardless of where you use them or when you're at least That was my experience. I felt like every teacher wanted long variable names So I will comment on your changeless if it uses, you know in a simple for loop if the iteration variable is called index I will probably comment on that. You know called I Okay, I don't know if this opinion is unpopular, but I feel like I write it a lot So there's certainly people who don't seem to grasp it And that's that the language is not perfect, but every change to the language carries a heavy cost So when you want to come and argue for why the language should be changed and you know, we see that a lot I'd say that there's probably one a day suggestion for some way to change the goal language Don't just talk about how it makes the language better But also spend some time to talk about how it makes the language worse Because there's no such thing as a 100% good change in language.
I shouldn't say there's no such thing Maybe it's out there Maybe no one has thought of it yet But probably it's a good bet that all the 100% good changes in language have already been made And so when you want to change the language spend some time to think about how it makes things worse As well as how it makes things better. I think I just saw you in dropper metaphorical mic We're gonna go back now and think a lot about how generics makes things worse So my own popular opinion is that encoding JSON is fast enough Oh, come on. Wow. This is the guy responsible for making it faster Well, I'm going to say generally where it generally means it most most likely applies to you But it might not apply to the 1% that's doing something completely esoteric such as handling 20 divides of JSON But most people don't do that And kind of my point goes back to the trade-offs, right?
Yes, if you pick another package you can get maybe a 2x 3x, maybe even 4x improvement But is it really worth sticking with JSON at the point? The overlap between the people that are stuck with JSON because they are And people that have to deal with a lot of data is very small because people that have to deal with a lot of data They generally pick better formats that that are faster to decode I think that is a pretty solid argument actually. Yeah, that's not unpopular with me that one I think you've nailed that. Yeah, yeah You would think that the amount of people yelling about encoding JSON being too slow would disagree Well, that's because we gave them the tools to benchmark things.
I don't know what you expect Take them back JSON isn't always as bad as people make it out to be Hmm tell me more. Who is he? Who is this? Jason you talk about?
Yeah, that the basin who gets so much flack. Why are you defending him all the time? Yeah, well, I've seen a lot of people switch in my opinion prematurely to protobufs in particular Sometimes to thrift where you just change from one problem to another and especially I think for anything that is used externally to your company So for example open source code protobufs can get very complicated Especially if you're exposing something that is going to be used across multiple languages So nice to use and go not necessarily as nice to use in Ruby for example or in PHP Yeah, or indeed the web browser actually yeah We did an episode on this very recently We called it encoding JSON and we actually spelled the episode title using JSON to see if any podcasting technology is vulnerable to uh, JSON injection attacks So far everything's just been fine. Which is a shame Yana, do you have an unpopular opinion?
I have a controversial one. Oh, let's do it. Let me just rerecord the theme tune then Got a controversial opinion. I do think that like I would really like go as a language You know like the simplicity and you know where it was otherwise.
It's just one of the best options that you have But uh all the proto-generated artifacts is just making everything just kind of like messed up Like each time I have to touch you know some proto-generated protos It just doesn't look like go anymore. It's like so cryptic like there's all these like types on top of the standard library I have to learn about you know protest its own like struck like all of the like, you know mess and Even like a timestamp for example type is like a completely different representation So you basically have to adopt into that like verbose um alternative universe And it's just like my main pain point And I've been like trying to collect all these like gachas and tips and everything about protos for a long time And I can tell you there's like at least 20 pages of me putting some tips like here and there And I still need to go back to that document in reference to be able to kind of like take a look like hey This is what i'm supposed to do if I you know see a type like this but proto-generated type like this And that's just like a big big struggle to me. They've been trying to improve, you know the generated artifacts, but it's just too late I don't know if it's popular or popular, but I think that generics and go are a good idea I would say that's unpopular with many people that I know But I do think that like I gave this talk long time ago around functional programming and go And and basically why not to do it and one of the biggest reasons like there were two reasons The first one is because there's no tail percussion optimization Which means that your program is actually 10 times lower just because of it So that's you know like that's a small thing that makes you fix But the biggest thing was the fact that if you want to do any kind of like interesting compositional types without generics You're out of luck. You cannot really do it.
You need to do anti-interfaces everywhere, right? So generics I'm very excited about seeing them like I've been trying them and how to look and now that you know contracts are kind of like Gone or at least like they make much more sense. They're not as complicated as they used to be I'm pretty excited about getting to use it. So I don't know when it's gonna be released for real but looking forward to that Okay, those were your summer hits I'd love to know if you find value in this style episode Please leave a comment one way or the other open your show notes click discuss on changelog news and let your voice be heard You can also tweet at me if that's easier.
I'm at Jared Santo. That's j-e-r-o-d As an to special thanks to bmc for all the music and sounds you hear on our shows, especially that johnny borsersko izmars remix We play to the top Pretty dope if you ask me. Thanks. Also to our longtime sponsors shout out to fasli.
I would have to denote shout out to robar That's all for now back to our regularly scheduled programming next week The panel did a reddit ama so we're answering all of the community's questions with a couple special guests Stay tuned for that. We appreciate you listening. Talk to you again next time