Mesos and Mesosphere DCOS

Welcome back everyone. This is the change log on your host, Adam Stikoviak. This is episode 167 and on today's show We're talking to Toby Canal of the CTO of mesosphere Toby was the tech lead at Airbnb and all three left back in 2013 Just our mesosphere a company that is building a data center operating system for the next generation of internet scale applications We talked about mesosphere mesosphere DC OS and all the open source around it Apache mesos dog or containerization Linux Cooper nadys core West all the in between Kronos great show today with Toby We have three awesome sponsors for the show co-chip top towel and digital ocean Our first sponsor is co-chip. They've launched a brand new feature called organizations And now you can create teams separate missions for specific team members and improve collaboration in your continuous delivery workflows Maintain centralized control over your organization's projects and teams with co-chips new organizations plans You can save 20% off any plan you choose for three months by using this code the change log podcast again That code is the change log podcast and you'll save 20% off any premium plan for three months by using that code Head to co-chip.com slash the change will be restarted and now on to the show Alright, we're back and this is actually take number two with Toby now He is the CTO of mesosphere Often mesosphere and mesos and Apache mesos These are all sort of like mixed and intertwined.

So hopefully during the show will talk to Toby a bit about all that and get settled out But so this is this is take to know what you think? Well, let's let's let's try this again. That's what we have we have better like this thing So just a chair is also on the call to but just explain what happened here We had a little glitch and we had to reschedule in today's that reschedule today and Toby's back So if we didn't tell you you wouldn't know so I let the cow the bag It's a nice object lesson though because as I said last week when we had our glitches that if Toby had had a Recording machine a day or something operating system If he had if his recording machine was spread out across a cluster of thousands that little hardware failure would have been no big deal, right? No big deal at all.

Yeah, it's all about high availability high availability. Well, let's um Let's go back in the past a bit. So maybe let's set the tone for what the calls about So obviously you're the CTO at mesosphere You got a lot of cool stuff for doing there a lot of operating around open source a lot of stuff out It's really picked up in the last year. That's just got all sorts of things happening But you were also the tech lead at Airbnb You've done lots of cool stuff in your past So let's get to know you a bit and learn a bit about what your past is and kind of who you are.

Sure. Yeah, so I grew up in in beautiful Bavaria in Germany and moved to Silicon Valley about six years ago and You know I went to school in Germany You did some some work in you know machine learning and sentiment analysis and then you know when you grow up in Germany And you work in tech you always have these like these ideas these romantic ideas about what Silicon Valley is like And you know, and it's everybody lives in the future there and stuff and so I always you know wanted to want to check it out And so you know did an internship at a small startup in Silicon Valley You know a couple years ago and then you know through a friend actually got connected to the Airbnb founders a little later And so I joined those guys pretty early on I was engineer for it Airbnb and So yeah, you know when you join that early where many hats so it did a lot of different things I helped them scale the infrastructure You know hold higher a lot of the engineering team build some of the back-end services They like search and the fraud detection service Build some features in the product too. So lots of lots of different things and You know a couple of years into Airbnb brought my best friend on board flow who's a co-founder of Mesa Sphere 2 So brought him into Airbnb to build out the data infrastructure team and so there, you know We worked with Apache mesos, which you know ultimately was the reason for starting mesos here Cuz we you know, we were pretty successful with mesos that Airbnb flow I'd use it at Twitter before as well And so you know that that led us to start the company So maybe since you mentioned Apache mesos I mentioned as well and mesos here can we knock down some hurdles in terms of terms and terminology Apache mesos mesos sphere mesos DCOS Let's look up. Let's help us out and explain the audience the differences between all these names.

Totally. Yeah, so let's So the first thing that was out there was was mesos I was actually called mexos before it was called mesos, but there was another product called mexos so they changed the name So it was it started as a project at UC Berkeley at the Amplethap and in fact Ben Heintman, who's the third founder of mesos sphere You know was one of the co-curators of the project. So it started there The idea was to build a cluster management system So sort of this layer that manages all the machines in a data center in a large cluster and that provides APIs for building large-scale systems on top and making that process really easy It's it became an Apache project a little later. So it's called Apache mesos then and you know Twitter was one of the largest backers initially Now so that's Apache mesos, you know been in the patchy top-level patchy project for a couple years now mesos sphere is the company that Flow and Ben and I created to commercialize the patchy mesos and to build a product around it You know the way to think about it patchy mesos or the way we like to think about it is It's kind of like the Linux kernel.

So it's fairly it sits fairly low in the stack. It does a lot of cool stuff It's very you know very sophisticated piece of technology It's it's very high-performance a lot of really smart people working on it But if you look at the Linux kernel, you know the Linux kernel is not Linux right There's a lot of things around the kernel that you need to to run your applications and you know to make the whole thing useful So that's basically what what's DC OS is the datacenter operating system, which is the main product that we're building at mesos sphere So it has Apache mesos at its core But it has all the pieces around it to that that make it, you know a full operating system experience So we got lots of different names there mesos sphere Apache mesos the kernel itself basically Let's come back to Airbnb where I don't want to derail it was too far off the conversation I did want to set some tone in terms of the terms and things like that people sort of stumble over like and part of this conversation So to demystify a bit of what's happening in the cloud And so hopefully you can help us do that, but take us back to when you guys were originally starting Mesa sphere what what inspirations were happening? What was happening at Airbnb in terms of the technology that made you guys eventually leave and start this new company? Yeah, so it was really our experiences from both Airbnb and Twitter that lecture it because we you know Ben and flow at Twitter and then flow at Airbnb we use the use mesos for completely different use cases actually and The environment was completely different to you know Twitter runs their own datacenter's Airbnb is entirely on Amazon web services So cloud and on premise and Twitter was running you know They're running pretty much all of their production services on top of mesos So, you know, it's things like search and the ad server and a lot of user-facing kind of requests response type things and Airbnb we were running big data stuff.

So we ran to do on top of it Cassandra Spark so big data analytics and You know that was kind of where the idea for calling it a data center operating system came from Because we looked at this and we're like you know This can really run all the workloads you can run in the data center the whole range of applications kind of the same way that you know Your desktop operating system is general purpose No, there's not one operating system that's great for doing development And then there's another one that's great for doing graphic design and the third one for doing like Word and Excel You know operating systems are mostly general purpose. So that's kind of what drove this And you know pre-mace those at those two companies There were really a bunch of big challenges that we were able to solve with mesos One thing that was that both Airbnb and Twitter started with was kind of scaling scaling up and being able to handle the user growth So Twitter if you remember the fail-wale Yes, like 2009 right you saw that lot and you know and I think there's even like a Twitter down dot com and like all that stuff It was a hacker it was all the time the whole internet got angry and took other pitch works and Twitter was down So, you know what happened behind the scenes is they you know they started Twitter started as a Ruby on Rails application and They had to you know millions of people showed up hundreds of millions tweeted a lot the infrastructure couldn't scale with this So they really needed to rethink stuff And one thing they did is they they'd circuit this model They could be in Rails application and broke it down in pieces and do that microservices micro banking services So you know there's like a different service for you know your timeline maybe your search She adds those are all different different code bases different services And so one thing they needed is really a platform to run all these things because that's you know a lot of stuff to manage and And that's what they use mesos for and you know at Airbnb it was a slight difference in area We we wanted to use Hadoop and we wanted to use it in sort of a self-serve way where we could start to do clusters very quickly And then shut them down again, and we also wanted to be able to try out new data analytics tools as they come out You know data analytics that is never just a doop It's always a combination of things and you know we were using Kafka also at the time So it's just a bunch of different tools, and we really wanted a platform to run all this stuff on to make it really easy to install these things Instead of you know spending a month or even multiple months to you know trying to figure out how to install Hadoop or Kafka So it was one use case that would be the other one which was pretty interesting and probably the most advanced one So what we were doing at the time is we we had one machine that had a cron top on it and that basically ran the whole ETL and analytics pipeline So it would do things like you know step one dump the SQL database to a text file and then maybe merge it with the web server logs And pull some other data from a key value store and you know build a data set from all that and then another step would be It'll take that and count the revenue account. You know other things number of visitors would have you So there's always these multiple steps that depend on each other and we were doing that at the time with with cron So you have to be you know be like okay the first step should probably take like 30 minutes So you know let's give it an hour and then run the next step And so obviously like if that first step would take longer than an hour for every reason everything would fall over and you had to like Manually debug things and you know folks were unhappy because the reports weren't there and so it was kind of a struggle And the other thing to you know the business was very fast And so these jobs would would take longer to run over time and this one single box that we had just get overloaded And so what we want to do to solve that problem is we wanted to sort of build a system that could dynamically scale with The workload with the ETL workload that's coming in. That's what became chronos, which we open source at Airbnb So you were you a part of that then for us?

Yes, it was mostly a flows team I contributed a little bit to it on but the flow is running the day infrastructure team and they built that And we built it so we looked at this you know we looked at the requirements that we had you know being able to scale dynamically Elastically being able to populate new machines as needed and we're like, you know, this is a lot of this is really hard You know, those are not trivial problems But but then we looked at mesos and we're like, hey, wait a minute You may just solve a lot of these things already like it has it has those things built in this primitive So we can just build on top of that and you know spend a lot of that a lot less time to to build this thing And in fact, it took only three months to build the whole thing and it had it had just like 2009's code of think somewhere around there, which you know given that it's a distributed default distributed system That is fault all around and can scale it lastly can survive machine crashes and so on That's that's pretty nice. That's not a lot cool for that. So you guys got excited about mesos and so excited that you decided to start mesosphere a company Kind of built on top of mesos So I'm interested a little bit in the the social and economic kind of background with the project because you have it started You see Berkeley all of a sudden huge players such as Airbnb Twitter more recently Apple and many others popping and saying This is something that we want so that we need and we'd like to build upon And then it became a patchy foundation project So maybe just kind of explain that whole milieu a little bit and the corporate interest the open source interest and break that Sure, yeah, so it became in the patchy project pretty early on basically when when Twitter decided to really invest in it And and make it their production the platform for running production before that You know because it came out of the amp lab in Berkeley Berkeley had the rights and had the IP and so Twitter You know because it the plan was to make it such a simple piece of their stack of the infrastructure They wanted the IP be owned by the Apache Foundation Just so they could contribute to it as well and to have sources neutral entity. So it's not the lab It's not Twitter or any other company that owns it, but it's owned by the foundation So that was you know the decision that they made pretty early on and then you know using Berkeley donated all the IP to the Apache Foundation Ben Heintman became the chair for the project And and then it sort of went the way that that most of Patrick projects go so You know every Patrick project actually has a lot of freedom over how they want to manage it, but Every every Apache project has this idea of Committers and you know the way works is when you set up an Apache project and it gets accepted There's an initial set of committer So at the time it was you know the folks from Berkeley that have worked on it in the past and then the project set up a process for you know How do we how do we accept new committers and the goal there was really because it is such a you know Central such an important piece of the stack It had the project has a really high bar for people becoming committers So this it typically takes at least six months for someone to you know Write enough code fix enough bugs get enough credibility into the community to be accepted as a committer And so usually what happens is there is a vote that gets done among among the existing committers So someone will propose a new person as a new committer and then the existing committers make a vote and if there's enough votes Then that goes through and that person becomes a committer.

So being a patchy project. I assume it's the Apache license, right? That's right. Yeah, it's a patchy license Which is one of the you know more free as far as you are free to build proprietary systems around it that being said you're building a company a VC funded company around mace house and Curious your thoughts on you know building a product or a service around software that ultimately is out of your control Yeah, so this actually works, you know the model works really great for us And you know we do have some control over the software because we are you know We are an active participant in the project in fact We have the most committers of any company on the Apache missile project and so you know even though we don't own the IP or we don't own the project We can we have you know a big seat at the table and you know what we really wanted to do is Build build substantial product around the open-source project as well so we really think that you know there's there's a lot we can add in terms of management tools and You know applications around mace house and and sort of you know the whole How do you operationalize this thing and really make it work in enterprise data centers?

That's really where we think we add a lot of value as a company And so you know if a lot of things open source projects, you know, they're built by you know They built by hackers and we kind of built these things for for ourselves And that was great, you know that those things those tools work they do the job But they're not really go for enterprise environments and so in mace This case for example when you set up a cluster and you use open source mace house out of the box is kind of open You know anybody can do anything it doesn't have some controls But not enough were to satisfy, you know, enterprise requirements where folks have you know really strict policies and auditing requirements You know, especially when it's a bank they have all sorts of regulations that they need to make sure they meet So so what we're really doing there is building all these tools and APIs around mace house to make it work in those environments, too Yeah, so it sounds like because mace house is kind of the kernel of a distributed clustering system There's a you know, there's a lot of other pieces to the operating system puzzle and everybody At least the large players appear to be building their own so Apple has something I think proprietary called Jarvis not sure if that's open source proprietary There's marathon you mentioned chronos that was built at Airbnb somebody's their open source some of these are not Apache Aurora Can you kind of explain all you mentioned chronos and you mentioned there's other services and things that need to be built around it? Is there a comprehensive list of missing things that you need to have a data center operating system if all you start with is a patching Yeah, so you're the best way to do this is is really you know DCS the days and operating system has a free community edition that You know, we can just go to our website Launch a cluster on AWS and other clouds and just you'll get started with it. It doesn't cost anything besides, you know, painful the machines In the cloud so that's really the best way to get started and you know, you get all the pieces you get the CLI to interact with your cluster You get the GUI to see what's going on you get a package manager So it's really, you know, really all the things you know from Linux or other operating systems There's an equivalent of that in the DCS So our package repository so you can really easily say install Hadoop Kafka Cassandra all these systems with one single command The same way, you know, in Linux you would do app get install, you know, we have DCS package install And you know in terms of the applications that run on top, you know, you mentioned a bunch of them I think I think the operating system analogy works really well there So if you think of you know, let's pick macOS You when you install macOS it comes with a few applications pre-installed, right? So you fire it up for the first time You already have finder on there and you have a browser, right?

So the basics are there the color apps are there, right? But there's many other browsers you can run on macOS, right? You can use Chrome, you can use Firefox or you can use Opera And I think it works kind of the same way in the DCS, you know, when we ship DCS it comes with Marathon Which is another open source project that we maintain at Nisys here Which is kind of the equivalent often in its system that you know from Linux So it starts, you know, long-lived processes in a data center So for example your Ruby and Rails application or Node.js app, so anything you want to keep running forever It does that but you know, that's just that's just a safari equivalent, right? That's the one that we ship and we believe it's awesome But if you if you want to use a different one you can do that and you know, like you said app will build their own It's Jarvis hop spot build one called Singularity Twitter build one called Aurora Netflix is working on one So it's you know I think this really shows that the data center operating system model works right because you get this foundation and it allows developers out There are all these companies to build their own applications on top to use the API and build something that's custom for the environment That works well for their needs that works well with their workflow And in fact, I'd argue, you know, if those things are kind of like platform as a service equivalence passes and we've seen a lot of passes in the past And I would argue that none of them have really been that successful and I think it's because they're generally pretty opinionated They have one specific workflow and that you know, usually just works for a handful of people It's that same workflow does not work for every company And so one thing that the DCS allows you to do is really, you know Either take one of those existing things and modify them or just build your own completely if you want to have your own workflow And in fact, there's I think there's more than a dozen in total that are you know, has like systems that run On top of the DCS.

Another example is actually Dr. Swarm, which which they're also building on top of mesos So as developers, we always try to point out patterns and what's the same and what's different and it seems like the mesos makes a lot of sense That have that cluster management and scheduler shared and but everybody seems to be green at the platform the marathon the Jarvis This is where the concerns break out and you can't actually that's not sharing for structure, huh? Could it be could those all be shared like with one pass and we all you know, just like Apache mesos I know it's Aurora what you said with Twitter started isn't a patchy project Well, how come how come it's not everybody's working on a patchy Aurora and then you guys are adding value at any higher level? It's just because there's different needs at a level right?

Yeah, it's for the reason that I mentioned I think they all these things take slightly different approaches and you know Aurora and Jarvis and singularity they're all they all have something that is, you know, feels a specific need Inside the company that built it and and that makes it less, you know, generalize it. And so, you know, I don't think that's a bad thing I think it's I think it's actually awesome that there's that there's choice and And you know that if you're new to the space you can you can look at the patterns that each one of those systems Use and just pick the one that works best for you Humanities is another example, which you know came out of Google and has sort of their workflow and their abstractions building You can run that on the DCS as well. Yeah, I was just gonna ask about Google because I seem to be the missing entity in the large Players here Amazon as well. I remember we mentioned them a little bit But Google has a thing called board.

Could you explain how board fits into this or doesn't fit into this? It absolutely fits in. Yeah, so actually board was probably the first system ever in this space and You know Google uses it internally. It's not open source.

They don't sell it. They wrote a paper about it and in fact May so take some inspiration from board Google is a sponsor of the lab where it came from. So there was you know a good exchange of ideals back then It also does a few things differently than board, but I'm definitely you know took a lot inspiration So yeah, Bork is the cluster manager and that that that Google uses internally for pretty much everything So if you're using Gmail that runs on board if you use in Google search and runs on Bork They run all the databases. I think even Google file system runs on top of Bork So it's really their one stack that the user's internally to run all the things Awesome.

Well, I want to ask a few more questions about Kubernetes and Clear up exactly how that fits in everything. It seems like it does play nice in this ecosystem But we'll take a quick break here from a sponsor and we get back. We will ask Toby about Kubernetes You've heard me talk about top-tow several times in this podcast, but today is different I've got a special treat for you I went out and spoke with a listener who a year ago had never heard of top-tow He listened to the show just like you're doing right here right now today and heard us talk about top-tow What they're all about and he decided to get in touch and now He's lived in the dream as a freelance software developer with top-tow his name is Dean He loves on and I sat down on top of them. I said hey What is it that you love most about top-tow?

I listen well for me the thing about top-tow, which I thought would be very hard for me personally as I transitioned to a more consulting role Was the way I would have access to new clients and what quality of those would be so I found that I've had access to awesome clients Through top-tow and it hasn't been that hard to find because they have a lot of choice and even more than that There's enough choice and I can actually be a little selective about what kinds of things I want to be working on so I use that as a way to sort of hone my skills and you know Go towards the technology that I think are worth investing in for the future So whether it's you know including new front-end frameworks or doing a little DevOps work on the side I usually I'm able to find clients who are have the needs of the things I want to get better at that's been that's been Truly useful. All right That was being a lasagna on a listener of the change log and also a freelance software developer with top-tow If you want to follow in Daniel's footsteps go to top-town.com slash developers. That's T. O.

P T. A. L.com slash developers to learn more about what top-tow is all about and tell them the cheese log cinch All right, we're back talking about Apache mesos mace osphere the cloud digital digital distributed systems Curious about Kubernetes you mentioned it previous to the break, but I'd kind of like you just explain it in more detail for us Yeah, so Kubernetes is an open source project that Google kickoff last year 2014 They announced it I think in around June last year after working on it for a couple months, so it's really it's It's a container manager container orchestrator That is that uses a lot of the same abstractions and learnings from Google internally You know the things that Google learned over the years building board and you know It's multiple iterations And so they took all those learnings and you know put it into a new open source project that they built from scratch And that's what Kubernetes is so it's you know, it's a really nice tool. It's really Simple and easy to use They have it has kind of two main abstractions that you know, Google found very useful for managing large numbers of containers One of them is the idea of a pot which is basically a group of containers that get launched together on the same physical machine that No, share the same network address and share the same Volumes so you case would be for example If you're running a web application in one container and you want to run some monitoring system right next to it or some logging system logging agent Right next to that web application you can run that in another container But they get launched together so and they share the volume share this so so they have access to each other So that's the idea of a pot It's one of the you know things that are unique about Kubernetes The other one is this idea of labels and using labels to model dependencies in the system and discover other pieces in the system So you know when you're running lots and lots of containers at scale one problem is really how do you discover things?

You know, how do you figure out where things are running and how do you say you know this web application depends on that database and traditionally How we did that on you know in the past is with DNS, right? We would just say you know my Rails application You know go talk to database.company.com and you know that doesn't really work in very dynamic and lasting environments We're containers move around and you don't really have to say model of you know painting an application to a specific machine And and you're making sure it lives there forever So say how Kubernetes does instead is it gives you labels so you can basically to say you know My web app depends on the thing that is labeled, you know type database and environment production something like that So those are kind of two main abstractions and communities parts and labels and you know We like those a lot and Kubernetes, you know is a really popular source project getting a lot of traction It's really great for running containers and you know sort of a microservices environment and that's why we decided to become part of the project too and And I think you know that where we really I value there is we can make it run really well In you know any environment where the DCS runs So you don't have to know the same way you don't have to go through the setup of the other distributed systems that we support We make that really easy for Kubernetes as well So you can literally say you know DCS package install Kubernetes and you're up and running you have a Kubernetes cluster configured And you can start running containers. Hmm. So because it's operating at the level of orchestrating Linux containers It can actually sit on top of mesos and on top of the DCOS that makes your ads to mesos.

All right, that's right Yeah, so it's this sort of at the same level where all of our other services sit So right next to you know Spark could do marathon how these other things? So I'm just gonna try to lay out my understanding of this whole stack And I just want you to tell me where it falls down or if I'm tracking you because I feel like I am But then I turn around and I can't realize I have no idea what's going on So you have and maybe Adams will help you as well. That's definitely okay So you have your hardware, right? Yeah, and then on that you have an operating system like Linux And then on top of that you have mesos which turns many Linux's, you know thousands are scalable up and down into one Closer thing and then on top of that now you add on top of your your app It's not really application layer, but now you can start adding your this is where your Kubernetes your services So maybe you have a Hadoop service So you have Kubernetes which allows you then to manage Linux containers, so now you have a second layer of Linux But I'm sure acted away from the hardware now.

Yeah, which then inside those containers you could run your a dupe, right? Exactly, so I think you described it really well. So you have the hard work you the next layer up well Linux is there I'm right next layer up is Mesos that's the layer that abstracts not really abstracts but manages the resources not just hardware resources So it knows how big your cluster is it knows how many cores are available how much memory is available and it uses those resources from that one big pool Which is your whole data center or able cloud and and offers them to the services that run on top So the services on top are kind of your building blocks They're kind of your legos that you use to build your business application, right? So if you're building a web app you need a database So you know launch a database on the DCS if you're if you're building a web app You need a way to run containers So use you know one of the container Orchestrator service like Kubernetes like marathon like caucus warm and so on So the building blocks and then you use those building blocks to manage your application So your application code goes into a Linux container you get that Linux container to one of those orchestrators They run it in the cluster you get the tools You know the services for example to that your application talk to the database that you launched earlier That's how it all fits together.

Gotcha. I think I follow that Adam Yeah, I'm definitely tracking on that make it's what it's definitely still complicated, but I'm tracking for sure It's a whole new world everything's different. Yeah, it is And I think one of Kubernetes pitch was like Google's infrastructure for everybody And you know a data center operating system is kind of the same idea It's like you could have access to this kind of scale without having to manage all those you know tricky pieces below where you care about and You know, we've been talking about big players Apple Google Twitter Airbnb And so the question that pops up because I'm just a little guy, you know And a lot of developers out there a lot of our listeners are developers wondering like is this something I don't even need to be Caring about somebody who maybe runs a couple servers that maybe I have a web server and a database server Should we be paying attention to the stuff or is it really the world of Twitter's and Airbnb's? I think everybody should be paying attention to this and here's the reason so I think when we built things today We sort of have to we always have to choose between building things quickly or building things for scale I've definitely been in that situation if you look at you know every game Twitter in the early days They were just you know a simple Ruby and Rails application that talked to Davis, right?

That's that's how you both started And they sort of had to choose to you know build build things quickly for build for a time to market And then when they when they started growing You know it became really hard to scale those things So I think you know for all the developers out there that are working on something that you know They hope that one day will be big I would say build it on top of the DCOS from from day one because you know when when it comes time to scale I think you'll have a lot less things to worry about and you know I think even if you just have two servers and the DCS can already add value You know if one of those two servers fails it can move your applications to the other one So I think that's already pretty awesome I think you know the reason why we're seeing mostly big companies using this stuff right now is because you know for them There's no alternative like they're paying and managing those many many servers that they have is so big There's just no alternative to automating the whole thing And so you know they're they're bleeding is bigger. That's why we're seeing a lot more of those guys using it But you know if I were to start a company today And you know build some say build a mobile app with it with the back end I would definitely build it on the DCOS from day one You don't think things are moving too quickly for those who don't touch it quite often is say daily like a large Dot scene light. It's not moving so quickly though The swimmer's their time kind of playing catch up with this new tech No, I think so and you know really that's our our mission and mesosphere is to really make this kind of an easy to use Product so you don't have to be a cluster management expert or distributed systems PhD to run that thing We want to make that really really easy You know as easiest as Linux, you know and get to get to that same level of Sort of turnkey experience. Maybe should take some time not a breakdown mesosphere Then so now we've talked about mesos marathon chronos and a whole slew of things Kubernetes board even Let's talk about what this does to to bring it to a DCOS.

So DCOS is Mesa's firstly company DCOS or mesosphere DCOS is a product Where is this a shipping is it the I guess you downloaded you is it a cloud service? How does this work? Right, so it's there's basically you know two ways to run it Which is on one of the public clouds like it'll be asked in Google Cloud and Azure Or you can run it on your own machines if you have you know a bunch of machines in a data center somewhere where you or you own a whole data center You can go to the mesosphere website today Basically click a button and launch a fully configured cluster in one of the clouds So all the you know It just uses the standard provisioning tools that the cloud providers have like cloud formation on AWS and you'll bring up all the machines It's fully configured, you know takes about 10 minutes and you're up and running and so it's not you know hosted by mesosphere But you know you kind of we just give you a template we redirect you to AWS you log in with your own account And you know you use that template to bring everything up, but you know It's your machines and your AWS account. So you manage the whole thing If you want to run it in in a data center and your own data center on your own machines we have sort of an early access version of that product and We're giving that to you know a handful of design partners right now and early customers that are helping us, you know Sort of polish it up So it's kind of you know call us and and we'll get back to you and help you install it at the moment So we go to the product page we see community edition free and we see enterprise edition Let's talk that the dividing line of the community edition is what you can go and launch today and the enterprise versions What you can take your end at a center?

Exactly. Yep. Okay, so I guess since there's so much underlying tech on this And we all know what open sources the most easily asked is is a wise and community edition free area I guess is free But why isn't it open source is there a reason why you went the way you went with it? Or do you plan on having that as a paid version at some point?

So we're you know we're evaluating our options there and you know we love open source We're a big contributor to mesos in fact and to marathon and kronos and other open source projects And so you know today the majority of code that we write is open source and we've definitely have you know We're having the conversation right now about DCS. You know what should we do there? Do we make entirely open source to be and parts of it are already open source? So yeah, we'll make we'll probably have some news there sometimes which which parts are open source right now So kind of the parts that I mentioned so mesos marathon and kronos and all the other frameworks that we're running like a sandra You know the integration between Cassandra and DCS for example with Kafka and DCS all that stuff is open source So the so early we talked about terms and try to divide the lines a bit so patchy mesos is different from mesos No, that seems to me okay patchy mesos mesos same thing trying to make sure cuz I see it's a fork I wasn't sure if it was is it your own flavor of the fork or is it the real thing itself?

Yeah, so we work we work with the patching is that's what we contribute to and and that's what that's the version that goes into DCS Also, so what's uh, do you have any thoughts on the future this then in terms of how it plays back and open source on your components in there? But is it something where you know for example The one comes of mine right now is just because of naming so similar to is get lab right get lab It's an enterprise version it has community version and community is the open source reversion and then enterprise is something you can buy and install or They even have hosted somebody to get up right? Yeah, you know for us we we really think that this is you know This makes operating infrastructure so much easier and we really want to give it to everybody You know, we don't want to we don't want to hold those things back And so that's why we have we already have a free community vision that there's no charge for it You know and we're thinking hard about open sourcing it as well You know has some implications of course, but you know, we're thinking through that process right now I'm just curious cuz it seems like that would be the place to start if you were going to I figure that was the Question on every listeners mind is a wise and community free then or a wise and open source if it's if it's free Why not make it open source to right? Do you guys feel any pressure from because you are VC funded investors on the open source front like do they push you away from it?

Do they push you toward it is a complete non-factor? So our investors are really awesome, you know, it's someone told me at some point It's like well besides besides looking for money. You don't really look for a business partner and you know our two biggest investors are close adventures and and reason targets and they've been really great to work with and You know, they really want to see this they want to see this be successful in the in the long run and and they give us a lot of freedom to run the company So it's you know, it's really in a large part It's it's our decision, you know, how much we want to do open source and and how much we want to close You know, not really getting getting pressure from from the VCs on that and they see the value of the open source to and you know They've invested in other source companies before so, you know, they understand the model. They see the benefits Cool, switching gears a little bit here I'm thinking about languages and Apache Masons itself is a C++ project.

It seems like a lot of the projects built on top of it So just marathon are scala or is it scalar? I call it scala. Thank you scala if I'm right or at least according to a couple of us I'm right According to me, which you are yeah, we'll let you have the final say so scala Yeah, I'm just you know I'm a big fan of right tool for the right job and learning why tools are the right for particular jobs I think like scala is well fitted for this space. I'm wondering if you could speak to that Yeah, so actually we have we're working in a lot of different languages in that in that layer and sort of the DCS services layer So they have kernels in marathon aren't scala.

There's a bunch that are in Java Just you know because the project started that way for example Cassandra and HFS and they're all they're all Java And then there's go also preventing system to go We're really language agnostic there. So you can Masons has an API for a lot of languages Python in addition to those that we just mentioned I think someone wrote Haskell bindings too So it's you know, we can really use pretty much anything. It's it's pretty easy to write your own language findings Mises is right now getting a new HTTP based API, which will ship fairly soon. So it'll be even easier to build language bindings We picked scala originally Trying to think why we picked that You know at the time So this was three years ago.

I think at this point when we started building kronos You know Java was still very it was a very popular language for systems engineering and Scala is also a JVM language and we found it to be more expressive than Java You know, you have to write fewer lines of code It allowed us to do functional programming, which was really interesting And so you know, that's why we went with it It just seemed you know more modern more more effective and more efficient than in terms of you know the time it takes to write software And so that's where we went with it at the time But you know, I completely agree with you I think it's all about you know finding the right tool for the job And I think there's right now, you know new exciting languages and systems engineering go is definitely getting a lot of You know picking up a lot of steam scala is getting more popular too. There's a few new ones like rust So we're really we're really language diagnosing there So if you want to develop for the DCS, you know tied to a specific language Yeah, a few years of experience looking back now on that decision Do you feel like it was a good decision? Are you are you still bullish on scala or you personally even because I know she's been writing some as well Just curious your thoughts on it And then if you are personally looking at go or looking at these other things rust or if that's more as a company Right. So personally, I think You know, it looks like The go community is really growing and a lot of the newer tools that we're seeing in the systems engineering space are written in go So, you know today, I would probably build the thing and go Um, you know for that for that reason, um, I personally still like scala a lot and I prefer it over over go actually It's um, you know, it it has go as a very simple language, which is great.

You know, it has a low barrier of entry It's um, you know, the code that you write is very consistent because there's usually only, you know, only one way to do things Or a few ways to do things. Um, so those are huge benefits Um scala, you know, it's let's you do functional programming really well Um, and so, you know, they both have their strengths and weaknesses. Um, for a type of for a project like, um, you know, a systems uh, systems engineering project cluster manager, I would probably go with that with go today I probably used to use scala for um, say if I was doing something in data analytics, you know, spark based That's using scala or, um, you know, wet back ends and things like that. I would probably still go with scala So all the uh, all the folks who we've gotten recently As listeners of the show jared are pretty excited because we just came back from gopher con and most of those guys are Either writing go or interested in writing go so they're in their hands in the air except when you said scala Yeah, anyways, well, uh, we're gonna take a quick break Uh, we'll come back and ask you some really awesome closing questions, but uh, we're gonna take a break, we're right back I have yet to meet a single person who doesn't love digital ocean If you've tried digital ocean, you know how awesome it is And here at the change log everything we have runs on blazing fast ssd cloud servers from digital ocean And I want you to use the code change log when you sign up today to get a free month Run a server with 1g of ram and 30 gigs of ssd drive space totally for free on digital ocean use the code change log Again, that code is change log use that when you sign up for our new account head to digital ocean.com to sign up And tell them the change logs that you All right, we're back Uh, great break there and we got some awesome closing questions that many many listeners are always just like I love when they ask those questions and the first question I think is is maybe a call to arms though So of the projects you have out there from the cli to um to marathon to all the different projects You mentioned, uh, what are some call to arms some ways that the open source community can help rally around what you're working on Or what you're doing to amplify what you're doing or to just step in and help out Right, yeah, so, you know, I think the goal of things like marathon and or you know, the whole DC s is really to automate everything Um, so, you know, it's it's making making ops people's lives better sre's Um, so if you're working in that field and uh, you know, you hate doing the same thing over and over again And um, you hate, you know getting working up in the middle of the night Because some random machine your data center fail or any cloud.

Um, you know, may want to check out marathon and the DC s and um, you know See if it works for you use case um and you know try it out and you know, help us Help us fix some bugs in there and help us make it work for even more use cases And the the github or that you guys operate off of is just slash mesos here, right? That's right. Yeah, get up slash mesos here tons of stuff on there, you know, from little weekend projects that some of our guys are working on There's some really cool stuff there actually so, uh, you know, I don't know if you guys have any, uh, HPC high performance computing listeners, but um, we have a couple guys on the team that are playing with stuff there like MPI Um, so so yeah, some some really cool things there Also, you know, if you want to get started, um, developing for the DC s with building a distributed system if you've never done that Um, there's a really cool example project on our github called rendler. Uh, it's a rendering web crawler So, you know, it crawls the web renders all the pages shows them in a big graph Um, and it's basically example code for developing a distributed system on on top of the DC s Um, it's in a lot of different languages.

I think we have scala, java, python, you know, a bunch of others So it's a really good place to get started. It's called wrangler. Is that right? I'm wrangler.

I just found it. Yeah, it's a couple pages back in the holding to some of the show notes, but it's It's uh, r-e-n-d-l-e-r rendler And it seems like it's uh, you know, based on the The guy here on the read me. It looks like joker is it a joker? It's the um, what is that, you know?

Riddler, okay. Yeah, riddler. I know there's something like that. I don't think joker for a second, but it was a riddler, but anyway, yeah Well, you're not kidding when you say you have a lot of open source stuff out there because six pages on there, uh, slash mesosphere I think there's a lot.

I think it's one of those are forks, obviously Um, but still lots of lots of cool repos out there for those who want to go digging. Yep, next question one. We cannot skip Everyone's favorite is who is your programming hero? My programming here.

I think it's mark and reason because um, you know, the guy did so much he wrote, um, really the first usable web browser Back in the day and um, you know, he did units so much for making the internet what it is, you know, went on to start an escape Which, um, you know, at the time they came up with javascript, um, the netscape browser, of course The netscape application server, which, you know, back then was basically the way to build web applications You know the equivalent of no JS and rootin rails and all those tools that we have today Um, so yeah, he's my hero very cool and uh, a question we don't ask every single show But I love asking this question, which is what's on your open source radar? So if you've got a weekend or even a week where you can just like take a vacation It's not really like traditional vacation. We just go and travel and have fun You actually like me, you know, go travel have fun and hack too. It's a vacation or something I don't know but if you had some time where you weren't forced to work on what you work on daily Either on your own passions or your own commitments if you just could take a weekend or a week What would you play with what would you work on?

Yeah, um, so yeah, you know, obviously our open source projects There's a ton of events. I would definitely work in those but you know things things that we aren't working on as a company. Um, I can think of two. Um, there's a really cool deep learning framework.

Um, also managed by using Berkeley. It's called cafe Um, so let's you do deep learning neural networks. Um, that's kind of a machine learning is kind of a passion of mine Um, so I probably check that out. Um cafe and the other one I take a look at it's it's a monitoring tool built by some clubs called prometheus Um, that looks really cool too.

Oh, wow. I don't probably check those out. That's a good see up because that's our next episode. Isn't that an exception?

It's definitely in the close pipeline. I can't remember it's next or second to next, but yeah, we're having a team on, uh, I think it is next week actually. Yeah, it is. It's after the show right now It's like Julia's vaults and uh, is it just Julia's come on the show or close?

It's probably Julia's but possibly Bjorn as well. You're awesome. Those guys were so awesome Well, yeah, we hired um, we have a few something about people here and they're all decorating about it Yeah, well, we'll be talking to prometheus soon. So you can tune into that episode so so you got to cafe Which is a deep learning, uh, don't want to describe that as deep learning framework.

Yeah, we're a toolkit for deep learning So why prometheus? So I think um, you know, I'm I did a lot of ops in my career that a lot of, you know, a lot of sre and so monitoring tools is always um, It's always a hot topic. Um, there's just so many shitty tools out there. Um, so you know prometheus really looks like like something fresh, something something different.

Um, I'm really, you know, taking a, uh, try it out yet. So, um, that would be, that would be my first thing that was just getting up and running and Firing some data at it and see what else just played with it. Very cool. Well, uh, Toby, it was really awesome having on the show for those who don't know how to reach out to you.

What's the best way to get in fetch twitter and get hub? Twitter works. Um, my twitter handle is, uh, it's super ginger. So that's how I can find it.

We'll link to that one. That's awesome. Uh, cool, man. So any, any of the closing thoughts before we close out the show for yourself?

This was fun. This was fun. All right. That's good.

Well, I want to say a huge thanks to everyone who listens to the show and specifically, uh, Those members out there that support the show where members support it. We're also sponsored. So the sponsor we have for the show are coach ship, top towel and digital ocean for the show. Uh, we love those guys to make this show possible.

Jared, you're awesome. And Toby, you're awesome for joining us as well today. Uh, until next week, We talk about Prometheus. Let's say goodbye guys.

See ya. See ya. Goodbye.