Cloud computing changed the economics of running a software company. A cloud is a network of data centers that offers compute resources to developers. In the 1990s, software companies purchased servers. This was an upfront capital expense that required tens of thousands of dollars.
In the early 2000s, cloud computing started and it turned the capital expense into an operational expense. Instead of a huge bulk purchase of servers, a software developer can now pay as they go, and this is so much easier on the budget than the bulk purchase. This transformation of capital expense into operational expense significantly lowered the barrier to entry to starting software companies. Cloud computing created an economic boom that continues to today and shows no signs of slowing down.
Of course, the upfront capital expense did not disappear altogether. It got aggregated into the cloud providers. Instead of individual software companies buying servers like they did in the 90s, cloud providers buy thousands of servers and build data centers around the world. Because of the aggregation, cloud providers get economies of scale.
It's much cheaper for a single cloud provider to buy 1000 servers than for 1000 software companies to buy one server. The first wave of cloud computing was all about deploying our applications to servers that run in cloud data centers. The next wave of cloud computing is moving more and more of our application logic into managed services. In the first wave of cloud computing, we had to spin up a server to run our database.
We needed a server for our search engine and our load balancer. The logical unit of a server comes with a risk that that that server will crash or will be hacked or will develop a bug that is difficult to solve and managed services abstract away the notion of a server. Databases of service searches of service load balancing as a service. These services are reliable Lego blocks that we can build entire applications on top of developers pay a premium for a managed service.
But they're happy to pay that premium because it represents a promise that this service will not crash. It will not be hacked due to a software vulnerability that has been patched by everybody but you and it's not going to develop bugs that are your responsibility because it's a managed service. Developers are very happy to pay higher unit prices for these managed services than they pay for raw servers. And cloud providers are very happy to develop and offer these managed services because it turns the infrastructure as a service business into the software as a service business.
So software as a service has better margins, better retention and better competitive differentiation than this infrastructure as a service business. And of course, the cloud providers actually are offering infrastructure and software as a service, but the managed services are often an additive higher margin business that they can build. And as a developer, I'm actually looking forward to building applications completely out of specialized managed services. But we're still pretty far from that time.
Developers today still need to write their own custom application logic and their own backend services. But there are a set of tools that allow developers to write these services, their own services, while getting some of the resiliency and the scalability of the managed services. And these tools for getting the resiliency of a managed service while getting to write your own code are functions as a service and Kubernetes. Functions as a service let developers deploy stateless application logic that is cheap and scalable.
Functions as a service still have some problems to overcome in the areas of state management and function composition and usability and developer education. But the future of functions as a service is quite bright. Kubernetes on the other hand is a tool for managing containerized infrastructure that is fully ready for adoption by lots of people. Developers put their apps into containers on Kubernetes.
And Kubernetes provides a control plane for deployment and scalability and load balancing and monitoring. So it's basically all of the things that you would want out of a managed service, but it's your own code. So things have become a lot easier when they're deployed into Kubernetes. And this is why Kubernetes has become so popular.
And it's also why Kubernetes itself is being offered as a managed service by many cloud providers, including IBM. For the last decade, IBM has been building out its cloud offerings. And for two of those years, Jason McGee has been CTO of IBM Cloud Platform. In this episode, Jason discusses what it is like to build and manage a cloud from operations to economics to engineering.
It was a real treat to have the CTO of an entire cloud platform on the show. And we've done some other shows about engineering cloud services at Microsoft and Google and Amazon and digital ocean, basically all of the cloud providers. You can find these old episodes by downloading the software engineering daily app for iOS and for Android. You can, of course, find many of our episodes in podcast players, but in those podcast players, you can actually only access the most recent 100 episodes.
And with the apps, you can access all of our old episodes, you can search over our content, you can find content that is related to each other. And we're building a new way to consume content about software engineering. These apps are also open sourced at github.com slash software engineering daily. So if you're looking for an open source project to get involved with, we would love to get your help.
And with all that said, let's get on to this episode with Jason McGee. For years, when I started building a new app, I would use MongoDB. Now I use MongoDB Atlas. MongoDB Atlas is the easiest way to use MongoDB in the cloud.
It's never been easier to hit the ground running. MongoDB Atlas is the only database as a service from the engineers who built MongoDB. The dashboard is simple and intuitive, but it provides all the functionality that you need. The customer service is staffed by people who can respond to your technical questions about Mongo.
With continuous backup, VPC peering, monitoring, and security features, MongoDB Atlas gives you everything you need from MongoDB in an easy to use service. And you can forget about needing to patch your Mongo instances and keep it up to date, because Atlas automatically updates its version. Check out MongoDB.com slash S.E. daily to get started with MongoDB Atlas and get $10 credit for free.
And even if you're already running MongoDB in the cloud, Atlas makes migrating your deployment from another cloud service provider trivial. With its live import feature, get started with a free three node replica set, no credit card is required. As an exclusive offer for software engineering daily listeners, use code S.E. daily for $10 credit when you're ready to scale up.
Go to MongoDB.com slash S.E. daily to check it out. And thanks to MongoDB for being a repeat sponsor of software engineering daily. It means a whole lot to us.
Jason McGee is the VP and CTO of the IBM cloud platform. Jason, welcome to software engineering daily. I want to start by talking about cloud in a broad sense. We've had the cloud for more than a decade at this point that just the word cloud computing, we've had things like cloud computing before that.
But it really started getting formalized and started getting a lot of traction, I think about a decade ago. How have cloud workloads changed over that period of time in the last 10 or 12 years? I think they've changed substantially. I think in the early years of cloud, well, in the very early years of cloud, I think people mostly were trying to get their head around, you know, what the model was all about and used it for some very specific scenarios like burst capacity or, you know, public facing applications within their companies.
I think as cloud started to get adopted in earnest, people viewed it initially as a cost saving spot for it. You know, it's all about efficiency and cost savings and being able to run their core infrastructure at a lower price point. What I think has happened in the last few years is that mindset has completely flipped and most enterprises today are looking at cloud computing as a vehicle to help them move faster, to help them innovate, to help them deliver capabilities, to help them compete in the markets that they're in by having more agile access to capability and by allowing them a platform where they can try things out and experiment at a relatively low, you know, starting cost. Many of the shows that I've done have been with newer companies that started on the cloud.
And for these companies, I think the contemporary term might be cloud native, but there are lots of older companies that were around before the cloud was a thing. And some of the ones I've talked to are hybrid cloud. So they have their own on-prem infrastructure. They haven't moved off of it.
And they use the cloud as an augmentative tool. These hybrid cloud companies, what do they do for the cloud and what do they do with their on-prem infrastructure? How does that division of labor between their resources manifest in the hybrid cloud model? Yeah, I mean, I think those older companies, which many of them those kind of established successful companies, the history of IT in general for them has been one of evolution and new technologies being blended with existing infrastructure investments that they've made.
So to some extent, hybrid for them is no different than what they've always done in IT, which is look at new developments, look at new technologies and approaches and figure out how best to adopt them and use them for leverage within their companies. And I kind of see two predominant patterns emerging within those hybrid enterprises. One is they use cloud as a platform for building new things to adopt the latest technical approaches and architectures and use it as an innovation platform for building new solutions. And in those scenarios, they of course use things like public cloud and it's really important for them to be able to connect back to the investments that they had.
So they're hybrid in the sense of connecting cloud-hosted applications with existing systems. The other scenario which I see pretty commonly with them is what I kind of call modernize and extend. They look at cloud, both on-prem and public cloud as a way to re-host existing applications without rewriting them, to get some efficiencies, some improved velocity around DevOps for those existing apps, and then extend those apps with new experiences. A simple example is the revolution that's happening now around AI and cognitive.
And there's lots of companies who want to add cognitive capabilities to those existing apps, but they're not going to start over and write them for scratch. So they use hybrid as a kind of modernization platform. I've done a bunch of shows with companies about how infrastructure has changed from the point of engineers who are utilizing the cloud. And for these engineers, work has gotten a lot easier over the past decade, largely due to managed services.
Even over the last two to five years, I just had a conversation with somebody from Thumbtack, which is a marketplace company that's really taken off in the last couple of years, and talking to them about how their rapid growth, their hockey stick growth in the last couple of years has manifested in the level of operational difficulties that they've had. It's much easier than the operational difficulties of a company like Uber that hockey-sticked maybe five years ago. So even in the last three or four years, the managed infrastructure, the tooling, perhaps, we could have containers. I'm sure we'll talk about that.
And even containers that in the ways that many people use them are require a little bit more operational burden than something like a managed database service or functions as a service. But so this is all talking about cloud infrastructure from the point of view of an engineer. And you're the CTO of a cloud itself. So I have a unique opportunity to ask you some questions about managing a cloud platform.
And the first thing I'm curious about is, does that translate to managing a cloud platform itself? Because obviously, if I'm an engineer building an application level thing, like a marketplace, things have gotten a lot easier. But if I'm writing the code that is managing resources across a data center, and is the underlying guts of a managed service, has my life gotten easier as well? Yeah, I think it has, actually.
I mean, I think the macro trend that's been going on for a number of years that cloud has certainly accelerated is this shift from when you're going to solve a problem having to think about the infrastructure to support that application. That was the original model. I want to solve some problem. The first third of the project is like, well, how do I build up all the infrastructure I need to get started?
And that has been flipping to the inverse, which is, I can just focus on my application. And all the resources underneath my app are kind of at my fingertips, either as managed services, or as API provisioned infrastructure resources. And that has enabled people to start with lower risk and to be much more agile about how they solve problems. And I think that transition applies to me as well as a cloud service provider.
It depends on what layer of the stack we're talking about. Obviously, the closer we get to the bottom, maybe the less that managed approach that API driven approach helps because you're the guy providing the APIs and providing the management. But if you think about a comprehensive platform like IBM Cloud, at different layers of the stack, absolutely, it helps us. I have teams like the Watson team, who's building AI capability.
They're riding on top of our container platform in cloud. And so for those engineers building those cloud services, they don't have to think about infrastructure and they don't have to think about how to do availability and scale in the same way. They can simply ask for a managed container environment and focus on building cognitive APIs that we expose as part of our cloud. So I think that same trend that's helping end user application developers is also helping me and my team build the cloud.
You've got these data centers all around the world that are managing the software that provides these managed services and provides the ability for developers to provision VMs or provision containers. Give me an overview of the software that is used to manage and provision these clusters themselves. Yeah, sure. So the exact architecture and software that's used to manage some portion of the cloud is pretty specific to the problem domain, right?
Whether we're doing VM management or storage or containers or databases, a lot of that is frankly, a combination of open source technology, which as IBM is a big believer in contributor to and custom software that we've written. Because frankly, the problem that I have as a cloud provider is very different than the typical application developer problem. I have to provide services that are managed at scale with isolation and multi-tenancy that isn't really an off-the-shelf software problem. And so we've built stacks that are appropriate for the different services that we host.
In many cases, we've reused the things that we expose for developers. So for example, my team runs our Kubernetes-based container service on IBM Cloud and we actually use Kubernetes to manage Kubernetes because the core problem of how do I provide a scalable, highly available platform to host application logic is the same. I have that problem in my control plane, just like a customer might have the problem in their application space. And so we tend to reuse a lot of the same services that we're building for customers to manage around stuff.
Now we have to do that globally and we have to operate that. And so we also have adopted SRE operations principles. We build a lot of automation. We've adopted things like chat ops and AI technologies as part of our operations stack to help us do this at scale.
Dice helps you accelerate your tech career. Whether you're actively looking for a job or you need insights to grow in your current role, Dice has the resources that you need. Dice's mobile app is the fastest and easiest way to get ahead. Search thousands of tech jobs from software engineering to UI to UX to product management.
Discover your worth with Dice's salary predictor based on your unique skill set. Uncover new opportunities with Dice's new career pathing tool, which can give you insights about the best types of roles to transition to and the skills that you'll need to get there. Manage your tech career and download the Dice Careers app on Android or iOS today. You can check out dice.com slash se daily and support software engineering daily.
That way you can find out more about dice and their products and their services by going to dice.com slash se daily. Thanks to Dice for being a continued sponsor. And let's get back to this episode. Is there anything like OpenStack or Mesos that does the job of managing the cluster at a lower level than Kubernetes or is Kubernetes kind of the good enough as a system for managing a global data center?
I guess I'm wondering if there's some piece of software that has the God's Eye view of all the different data centers. My understanding was that Kubernetes, you might use Kubernetes to manage perhaps one data center. I'm not sure about a network of data centers or maybe I'm just thinking about the abstractions wrong. Yeah, I mean, so there's no one single God's Eye view of the whole planet.
Below containers at the infrastructure layer, we obviously have an infrastructure control plane that's managing the data centers that's managing bare metal servers and virtual machines and networks within those data centers. That is a largely custom control plane that's custom built for the prominent hand of running a global scale public cloud platform. Kubernetes, though, is actually one of the fundamental building blocks that we use and is the base layer that we build most of the I-beam cloud on. And so not only is it a service that we make available to developers who use our platform, but it's actually the underpinnings of our identity and access management systems.
It's the underpinning of our cognitive data services. It is the common layer that everyone builds on and has many of the characteristics you would need to run the cluster, if you will, underneath a whole collection of workloads. When Kubernetes came out, was there some significant refactoring to be done to replace the underlying infrastructure that was before Kubernetes? Or do you just leave that stuff running in its previous manifestation and you spin up a new stuff on top of Kubernetes?
That's one of the great challenges and advantages of cloud is this pace of innovation that underlies it. We update our environments continuously and when new technologies emerge or we help them emerge like Kubernetes, we absolutely go and change our architecture. Change our implementation as appropriate to take advantage of and to expose those capabilities to our users. None of those underpinnings are static, if you will, are left around.
Now, the mechanics you're leaning a little bit on, well, how do you actually do it? Absolutely, we use eB testing and Canary deployment models to be able to roll in new capabilities and new implementations of existing capabilities in a non-structed way. One of the challenges of being a co-provisor is you have to do everything while the bus is moving, while everything is up without disruption. As we bring in something significant, like Kubernetes, we'll do that side by side for a period of time as we transition over to some new architecture that we're going to run.
You mentioned the adoption of SRE-style operations. Talk about the operations model of the IBM cloud. What's the division of labor look like? Are these SRE people sitting in the data center itself or are they teams that are remote from the data center?
Yeah, I guess talk about just the operations framework and the management structure. Sure. Generally speaking, the way we describe it is you build it, you run it. Meaning, if you think about a cloud platform, like I think cloud is comprised of a collection of services and each of those services is owned by a team and that team owns the lifecycle of that service end to end.
Meaning they build it and they run it. They operate it. R SREs are generally speaking aligned with each of the services. They sit conceptually with the service, not in the data center, but in the development team.
SRE is a model where you're applying software engineering practices and principles to the domain of operations. Logically, those SREs are aligned with development organizations and sit in this part of those teams. Their job is to run that service and to design it for availability and to continuously improve how we deploy and operate and update and recover that service through automation and through software technology. We of course have some centralized functionality around SOC and Knock and central alerting and other things to help wire the cloud together.
But on the whole, we run each service as its own end and lifecycle. That's really powerful. It's actually a model that a lot of customers are trying to adopt. It's a model that I think is aligned with this transition that I described that we're becoming more application-oriented.
It makes sense that you run things from the perspective of that application or service, whatever you want to use. Therefore, that operations function tends to align itself with the capability itself. How rapidly do you have to grow the size of an operations team relative to the size of your customer base? Because I can imagine being able to scale your employee operations base at a linear rate and being able to scale customers at an exponential rate.
I think it seems like one of the great economies of scale of being a cloud operator, is my intuition correct there or am I off? No, I think your intuition is correct. I mean, obviously, the great trick of running these kinds of systems is making your operations growth and your customer growth not be parallel with each other. Or in the worst case, your operations cost goes faster than your customer base, which is death to a cloud service.
We spend a lot of time. Part of the reason that we use an SRE methodology is to ensure that we're always looking towards how do we run these services, how do we deal with their growth as efficiently as possible, so that as consumption scales, the cost of operation around that service doesn't go up linearly with it. How that's accomplished varies a lot by service, but fundamentally, it's rooted in deep automation, in deep instrumentation of the environment. So, we actually have a sensor that's going on at all levels, not only operational status, but performance and user behavior, and using that data to drive the next round of development changes and how we automate the operations of that part.
And that's something that we work on every day. So, on the developer side, containers have become the deployment unit of choice. How does the growth curve of containerized workloads look over the last five years? Do you have any numbers or any insights on just how much of the industry has adopted containers and what they're using those containers for?
Yeah, I mean, I don't know that I have great numbers off the top of my head. Certainly, customer interest in containers has been off the charts, I think the highest of any new technology, certainly in my 20-year career in IT, it is used in essentially every customer I talk to, they have containers, and they're using containers. Now, where they are in their journey has varied, and I think one of the big transitions we've been seeing in the last 12 months, maybe, even last maybe nine months, is really the transition of people using containers early in the life cycle for development. So, they're using it on their development teams, they're using it for packaging and testing, but then they were deploying using traditional models.
Maybe they take those containers and put them in virtual machines and they deploy them with a traditional virtual machine model. And then the last nine months or so, they've started to make the transition to actually using containers as their production operations methodology, meaning adopting things like Kubernetes, that the unit of deployment is a container or a collection of containers not a VM. And it means that they're starting to take advantage of the network architectures and operational monitoring that containers bring. And the pace of that has been just really tremendous.
Like, you know, every year that goes by, you know, it's probably double where I thought it would be the previous year. How does the growth of adoption containers change the economics of being a cloud provider? It's interesting. I think containers are one of those technologies.
The reason that I think their growth has been so tremendously high is that there's a value proper containers for kind of all constituencies. You know, it's it helps developers build software and deliver software faster, it helps operations teams on the customer side be more efficient and achieve higher availability. And it's a cloud service provider. It helps economics for us as well, because fundamentally a container-based model allows the platform to be more dynamic.
So, you can create and delete things more quickly and allows the environment to be consumed at a higher density. And it's hard. You know, part of the cloud economics model is, you know, density and sharing of resources, right? And containers allow the resources we deployed in the cloud to be, you know, to be utilized at a higher rate and has enabled new models like you mentioned earlier, things like serverless or functions.
You know, their hard functions or serverless platforms are essentially container platforms. If you look at how they're actually built internally, they're container platforms. They have a different execution model and API, but they're fundamentally container platforms. And because of the advantages of containers, you actually can get to an economic model, which is literally paying for CPU cycles with no idle time, right?
And that isn't really possible with other technical approaches, right? And so I think those technologies that really enable new economic models to emerge in the cloud. And this is basically because if I provision a virtual machine, it's going to take up this large block of space. And I'm probably not going to utilize that space to capacity.
Whereas a container is typically a slice up a slice of a virtual machine, you're going to take a virtual machine, slice it up into a bunch of containers. And those containers are provisioned with a better, more accurate fraction of the overall resource and more closely aligns with what you actually need so that more efficient provisioning is a more efficient usage of resources. Did IBM have expertise, like internal expertise and containers prior to the popularity of Docker and the mass adoption of containers? Were you using them internally?
Yeah, absolutely. And in fact, you know, one of the interesting things about the history of containers is that, you know, containers are not new, right? Containers are 12 plus years old, and their origin goes back to work in Linux and in Unix. IBM was one of the lead contributors with early work that we did in Linux in an REIX or Power Platforms.
Some of the ideas like namespaces that are popular in the container space today are ideas that IBM helped create. So, you know, our history goes back to actually helping be one of the people in the industry, along with, you know, Google and Sun and others who actually created all the underpinnings that Docker then put together. You know, Docker's hat trick was taking all this powerful technology that built into Linux kernel and making it accessible for the average developer. And adding a logo.
Yeah, making it consumable, right? I mean, they made it so that someone who was not a Linux kernel expert could figure out how to leverage this technology to advantage for them. But the core capability is stuff that we helped build and stuff that we had used in various capabilities that we built over time, both the work we've been doing before, you know, Docker and Kubernetes in the Cloud Foundry space or back further into our mainframe and Unix platforms, you know, we have a long history of kind of taking advantage of these capabilities and how we ran things. And what's happened, I think, in the last few years is just the democratization of this technology in a way that means that it's now the de facto way you build it like that.
There's no project, I think, that starts today that doesn't just assume there needs something like containers as the platform to build that up. And the big thing about Kubernetes was it gave people this single tool to manage their different containers. It lets you, you know, for example, look at your collection of containers together as the abstraction of a service. And you can just start to, you know, create rules around how that service scales up and down, you know, the underlying containers are scaling up and down.
Of course, to deploy and manage Kubernetes, for some people has been a burden and they don't want to do it. So they decide to use a container platform and all the cloud providers are building their container platforms to give a fully managed version of the container platform. And these things are typically built on Kubernetes. Can you help explain what are these operational burdens that people have from self-managing Kubernetes that tend to get remedied when they move to a container platform that's fully managed by the cloud provider?
Yeah, great. Yeah. It's your question. I think it goes back to your comments at the beginning about, you know, the value of kind of managed services in cloud.
And you know, I think the problems that a managed container service solves for the user, they range, right? So there's the obvious, like, the mechanics of installing connecting and configuring a collection of resources into a functioning container cluster, you know, that's not easy or free. That takes work. It takes knowledge and managed services like the Ag and Cloud container service do that for you, right?
They literally give you single API call, wait a minute or two, you have a cluster up and running. That's a day one experience, right? Once you have a cluster, you know, if you always keep in mind at the end of the day, what people want to do is build and run applications on that cluster. Anything you can take away from them that they don't have to worry about anymore is an advantage, right?
And so if you think about Beyond Day One, you have things like how do I add and remove capacity from that environment, how do I recover when failures happen? And maybe the thing that people really don't recognize is keeping up with the pace of change. You know, if you look at containers with Docker, you look at Kubernetes, you look at the surrounding technologies, they're moving at just a blistering pace. You know, the Kubernetes has releases every few months, all the underlying technologies are constantly updating, keeping up with that pace in the industry is really hard if you do it yourself.
Like, you need like full time people who are connected to the industry and in the communities to kind of know what's going on to keep up with those changes to make the associated configuration changes. And so one of the advantages of a managed services, all of that is just done for you, right? And you can just focus on your apps. You know, me personally, my career has kind of always centered around what we used to think of as a middleware space, you know, as a Java WebSphere guy for a long time.
And, you know, the mission statement is kind of the same. You know, it's like, we're going to provide, in the cloud now, all of the building blocks you need to build your apps so that you don't have to worry about all the plumbing, right? And a managed container service at the end of the day is plumbing to help you host your applications. The thing that's cool about them is they solve some really hard problems like availability and scale and networking in a way that's common across different programming languages and different application patterns.
You know, so to me, that's the advantage is like, I can just focus on my apps. Do you have a product that is sold to software engineers? Are you looking to hire software engineers? Become a sponsor of software engineering daily and support the show while getting your company into the ears of 24,000 developers around the world.
Developers listen to software engineering daily to find out about the latest strategies and tools for building software. Send me an email to find out more, jeff at softwareengineeringdaily.com. The sponsors of software engineering daily make this show possible. And I have enjoyed advertising for some of the brands that I personally love using in my software projects.
If you're curious about becoming a sponsor, send me an email or email your marketing director and tell them that they should send me an email. Jeff at softwareengineeringdaily.com. Thanks as always for listening and supporting the show. And let's get on with the show.
The managed container platform that you built at IBM is called IBM Container Service. Can you describe the design process for this project? Yeah, sure. I mean, so that would be a container service.
You know, as we've been alluding to is a Kubernetes-based service. And you know, the design process was really honed in around that basic storyline. I just gave you know, how can we provide the easiest user experience around creating and managing a container environment that a customer can use to host their apps? And then what hard problems can we solve for them beyond just getting them the environment?
So an example could be security. Maybe the unspoken reality is that most developers know that security is important and most of them don't want to think about it. And so if we can do things in the platform to make the environment secure out of the box, to give people visibility into the security posture and vulnerability of software that they're developing, that will help them be more successful at deploying secure software without having to become security experts. And so on the basis of this kind of core Kubernetes environment we can stand up, we thought about in that design process, what are the additional problems that are hard that we can solve?
Security. Another one is if you look at Cloud Native apps, one of the design principles of Cloud Native apps is you deploy your application across multiple availability zones in multiple regions around the world. And that's how you achieve higher availability in the face of a failure of one zone or one region in the cloud. On paper, that's easy to say, but in practice, that's hard.
You know, it's multiple deployment pipelines. It's, you know, global traffic managers and DNS tracks to distribute traffic. And so we said, can we automate that? Can we have the service help you automatically set up highly available, globally distributed clusters?
So that as an app developer, you can just deploy your app into these environments in those problems or software. So that was our design process was give them the core capability they need to host their apps with Kubernetes and then around that environment solve the hard problems that all developers have to solve to build production worthy scalable cloud native software. Okay, so you mentioned security. You had a project recently partnered with Google on called GraphFace, which does this is like audit and compliance of a containerized software platform.
Explain what that means. Yeah, so one of the things that's kind of emergent in the container space is this idea of kind of a secure software delivery pipeline or supply chain. I think it's a manifestation of the fact that because containers have kind of normalized things like packaging and because Kubernetes is normalized how you launch applications and make them available, you can actually start to define the steps that you go through to get a change, checked into a source control system and deliver all the way to production. And along that pipeline, there's a lot of metadata that's generated and you want to use that metadata to make decisions.
A simple example in iDIM Cloud as part of our container service, we have a tool called vulnerability advisor and what that does is scan all of your container images and it looks for things like vulnerable software packages that have known CD's against them and it looks for poor configuration like you have weak password rules or you don't have password expiration or you have SSH enabled and it generates a report of the vulnerability status of that software and if you think about a delivery pipeline, you might want to be able to say things like in the production environment, no applications to get deployed that has a vulnerable image or whose image has not passed the vulnerability scan. Now we can do that in a point-wise way. We can integrate all these tools but if you look at the container space, there's just a tremendous wealth of point solutions that look at different parts of these problems and they don't know how to talk to each other. So the Graphiys project that we've been doing with Google and others is really about solving that.
It's like can we all agree on a common way to share all of this metadata that's being generated in the delivery pipeline of software and use it to make policy decisions about whether we're going to deploy something or not. And so Graphiys actually has two pieces. It has this metadata database essentially where we can all share the information and then has an enforcement component that runs inside Kubernetes for example that looks at that data and applies policy to it to decide whether this is something we want to allow. And so to me this is the exciting part of containers is like some people think of containers as kind of more efficient virtualization but in reality it's completely changing how development life cycle happens, how development process works, how we build applications and I think that's pretty exciting.
Again the other project that comes to mind that you worked with Google on is the Istio project and this is another fairly new project we've done a show about this we're doing another one soon and Istio as I understand is it's a service mesh and it's built on top of what is called a service proxy. So Envoy is a service proxy that was originally built by Lyft because Lyft had this problem where they were afraid to deploy their software on any individual container because they were afraid of all of the interrelations between different services that were dependent on an upstream or a downstream container. And so basically what they said is like okay we've got all these containers that are running what if we try to standardize some things around how these containers interact with each other so that there's less sensitivity when we deploy a container. And then Envoy had such success that this Istio project came up around it and Istio is sort of like okay now that we've got some standardization among these different services that have a proxying layer we can now create this mesh that I think if I recall correctly is kind of gives you a control plane it gives you a way of leveraging the fact that now you've got this side car running alongside all of your services that are doing proxying and doing all kinds of other things and that the Istio service mesh gives you a centralized area where you can do things that leverage having a side car and all those different services.
Am I articulating that correctly? Yeah you are and maybe I'm able to context them. First off I think Istio is one of the most exciting projects in the industry today. In fact I was looking at some early kind of the you know adoption data on the on the GitHub project and I think it's it's actually trending ahead of where Kubernetes was at the same point as lifecycle.
Istio I think you know so I think you got some of the history pretty spot on there actually is interesting you know both my team at IBM had been doing work around this idea of a service mesh and kind of programmable routing control plane between different microservices. Google had been doing some work in parallel they've been really focused on like security and telemetry like visibility into what's happening between microservices and then Lyft had this really awesome proxy component that had all the smarts of how to handle the traffic and the real problem that Istio is trying to solve is as you adopt microservices and you go from having one thing to having you know hundreds or thousands of components you need some way to kind of both understanding control what's happening between them and what we had seen happening for early adopters of microservice architectures was they were building some of the things that Istio does into each microservice you know they they would build monitoring agents in they would put configuration into control who they routed to they would solve problems in a kind of custom way per service well that of course puts a burden on each microservice team it means every service solves the problems in a different way it makes it really hard to change tools or get global visibility so what we decided was there's a need and a value in having a smart interconnect layer between the services that abstracted all of those kind of common communication concerns from out you know out of the microservice in into the smart mesh and to do that in a way that was totally like programming model and language neutral right so you can use whatever program language you want you don't have to write to an Istio API when you build your apps you just use normal TCP and HTTP communication and we can kind of insert ourselves and with that I think Istio really gives people a lot of power in controlling you know how their services deploy and interact you know even simple things like one of the most common things a development team has to do when they're doing microservices is deliver an update to production and even in Kubernetes by default when you deliver a new version of an application a pod into the cluster as soon as it's running it starts receiving traffic and you're having control over that right it just starts receiving the traffic well maybe what you want to do is decouple when you did the deploy from when people actually started using it so you could deploy it validate that everything looks good and then maybe gradually roll people onto it right that's actually a pretty common scenario and that's the kind of thing that Istio makes super easy because which version of a service somebody routes to is not a function of the deployment operation it's a function of a policy statement in the SQL control plane so you can deploy version three of a service everyone still uses version two and then you can go into Istio and say hey send 50% of the traffic to version three or send you know all of Jeff's traffic to version three and let him make sure that it looks good before I start to roll that out to others and so that that kind of architecture that service mesh architecture I think is solving some really practical problems that people have been grappling with as they thought about you know how do I dot microservices for some people this might be a challenging to understand because you know they might be thinking okay wait I thought Kubernetes was the thing that like manages all these containers and now you're giving me this other thing that manages all my containers why aren't we just bundling all you know okay so we've got these notes in front of me I was at Qcon a couple days ago and I saw a presentation from Lewis Ryan who started the Istio project and he's like okay so yeah Istio gives us let's see observability resilience traffic control security policies and it's like well why doesn't I thought Kubernetes gave me this this God's idea of like a chance why do you need another project why is this a good abstraction to throw into the mix yeah it's a great question and I think if you look at it in detail they're actually very complimentary and one of the interesting decisions that Lewis and my team made together was you know while the Istio stated objective is to work across different environments not just Kubernetes but you know Docker and Maesos and VMs and bare metal servers in a microservice architecture you might have all those things what we decided on day one was to deeply integrate with Kubernetes and if you look at the problems that are being solved they actually don't overlap they're actually complimentary and a lot of the core underpinnings of Kubernetes are used by Istio so Kubernetes handles the basics of service discovery and handles the basics of routing of traffic between containers inside the cluster Istio doesn't really override that what it does is builds on top of that and adds in things that Kubernetes doesn't do like Kubernetes will handle if my container fails restart it Kubernetes will not handle what happens when service B fails and service A depends on it what happens in service A when its dependency starts to misbehave right you might want to fail back to an alternate source or you might only want to wait 10 seconds or you know for that failure to occur before going on that kind of cascading failure problem which sometimes people describe as circuit breaking like those concepts are not built into Kubernetes version routing is not built in to Kubernetes and you can argue we could just add all those to Kubernetes and that's kind of what Istio did and if you actually look at the architecture of istio it actually sits inside of Kubernetes as API extensions to Kubernetes it doesn't actually feel like you've installed an alternate orchestrator it just feels like you extended Kubernetes with some new capabilities right to solve this specific domain of you know interaction between microservices so I think they're actually quite complementary. I want to jump forward to talking about serverless a little bit the functions as a service I interviewed one of your colleagues Roderick Araba who helped architect the open whisk project because we had a pretty detailed discussion about serverless you know one thing I'm curious about since you're you know in this CTO position you are kind of like looking several steps ahead of where we stand today as people that are using clouds and deploying and managing clouds themselves do you think functions as a service is this an abstraction that's going to be around for a long time or do you feel like the functionality the functions of service idea I can imagine a world where this just gets absorbed into other managed services like if you need to have your blob your object storage respond to an event it feels kind of weird to to spin up a function as a service to respond to that event rather than having that code tightly coupled with the object storage itself maybe I'm not sure if that question makes sense but I guess the bigger question is do you think functions of service is actually going to be something that's around for a long time I think the idea of you know event driven function execution absolutely will be around for a long time in fact it's been around for a long time like the base paradigm of the kind of serverless movement has existed for a long time like it's basically how all uis work right they're all event driven function handlers right I think what will change over time is like my view has always been that the serverless platforms the reason they grew up or I shouldn't say grew up but are growing up in the context of cloud is because the value of those platforms is largely derived from the integration and events that flow between all the other services and clouds used object storage as an example that's a really common use case for event systems or functions as a service which is like something changed in my object store bucket do something about it right and the user experiences you're right are kind of divorced at the moment like you go to a function service to define a function that responds to an object storage event you could just as easily go to object storage and say when this event happens to us right like that's just a user experience thing like under the covers that's kind of the same thing but I agree with you I think it will tend to get embedded in the platforms like in the overall part you can imagine that being more efficient to provision and manage if you've well I don't actually I know nothing about that so I could be wrong about that yeah I don't think it's provisioning efficiency I think it's like how do people think about the world and I think in the use case where it's like when x happens here I want to do something it logically makes sense for you to be able to define the function there in that context and there's other cases where maybe you're doing something more complex or you're doing a composition of a sequence of events in which case you don't think about any one source you kind of think about the sequence of events so going to a functions platform and defining an application which is a sequence of functions that respond to a chain of events makes sense as a developer right and so I think you need both under the covers they're really driving the same infrastructure but you want to expose them and we're in the early stages of this movement and so in the early stages things more natural to just build a discrete thing you know you build a function service and over time it will get more embedded I also think you see that same split with functions and containers like today functions like obonwesk and containers like with Kubernetes feel like two parallel things that sometimes are not positioned with each other and if you and you talk to true believers on both platforms they'll tell you that all problems can be solved with containers or all problems can be solved with functions right I think the reality is all problems will be solved with a combination of functions and containers because they're good at different things and so like one of the things I've been really focused on in my role is how do we think these things evolve and how do we bring them together so they feel like a single cloud native platform where I can use an event oriented functions paradigm for some things and I can use containers for others like as an example we have a client who's a bank and you know they have an online banking application and you know the core of that online banking system is a you know web application that's naturally fits into the containers and they're running into containers but they have a certain function of that app which is to do digital check deposits right and if you think about a digital check deposit it's a very event oriented activity like new check arrives like image of new check arrives in object storage do image correction do ocr make a deposit to the back end system send a confirmation email go away right and it's very spiky in nature like you get a lot of check deposits on Fridays when people cache their checks and a lot less other times like for that slice of the application serverless makes a lot of sense like it's a really natural paradigm but that doesn't mean you want to build the entire online banking application serverless right and so using them together I think is the next round of this maturity is how do things like Kubernetes and and functions come together okay I know we're slowly running up against time I want to ask a few more questions about sure you know being CTO of a cloud platform because I don't get asked these every day there's so much blue ocean as a cloud provider this is a very new computing paradigm relative to how big of a shift it is and there are so many managed services that you could potentially offer how do you identify the managed services that are worth pursuing it's like a trick of business isn't it like how do you build the right thing at the right time you know some of that I think is I think it's kind of two sources that we use right I mean one is we're deeply involved in the technology space and you know have a long heritage in IT and from that comes our own knowledge and perspective about you know where people are in their journey and what technology is to adopt and we use that to inform you know inform us you know we deal with containers which is something I spend a lot of time on you know it's something I've been working on for three or four years but it's really only been in the last 12 months or so that you know it's starting to kind of cross the line into widespread adoption and so timing that is really important you know the other thing is like you know we run a lot of applications for a lot of companies and we have deep relationships and we use that conversation I personally spend you know pretty high percentage of my time you know 25 30% of my time talking directly to clients and you know helping them with the evolution and adoption of technology and that of course informs me about where are they really you know it's it's dangerous as a CTO you can get caught up in the technology and think people are way farther ahead than they really are right in their adoption and so that there's a grounding in kind of talking to clients and understanding you know where they are that we use to try to inform when we work out things right interesting okay well you know we've done shows about managing data centers and site reliability engineering and lots of operational challenges as the CTO of a cloud platform all of the difficult problems that can't be solved by other people who are maybe closer to the problems themselves those problems eventually bubble up to you that's kind of what a CTO does yeah what's the hardest challenge that you've had in your time managing a cloud platform I think the hardest challenge especially in a comprehensive platform like ours is actually not a discrete technical challenge it's getting people on the same page about how to move forward together architecturally you know getting a hundred services to to agree on simple things like we're all going to build on Kubernetes right and maturing a platform like that to meet the workload needs of a diverse set of services right you know so I think my role as a CTO is twofold one is setting technical direction helping navigate the forest of you know all of these projects and activities and and solutions that are out there but a big part of it too is building consensus on the right way for us to move forward together collectively and that has a big impact on our customers that's not it sounds like an internally focusing but it directly results in the user experience that customers have when they adopt a platform like IBM Cloud right if our stuff all works on a common singular architecture you know if we solve problems in a consistent way like we're all going to do logging and monitoring in the same way then that manifests itself in user experience right you know to me that's actually a big challenge like one of the lessons of my personal career in software is that the software parts the easy part the people parts the hard part and as a CTO you have to do both right you have to solve the technology and help the people understand why that's the path they should follow Jason McGee thanks for coming out software engineering daily thank you Jeff when your application is failing on a user's device how do you find out about that failure ray gun lets you see every problem in your software and how to fix it ray gun brings together crash reporting real user monitoring user tracking and deployment tracking see every error and crash affecting your users right now monitor your deployments to make sure that a release is not impacting users in new unexpected ways and track your users through your application to identify the bad experiences that they are having go to softwareengineeringdaily.com slash ray gun and get a free 14 day trial to try out ray gun and find the errors that are occurring in your applications today ray gun is used by Microsoft slack and unity to monitor their customer facing software go to software engineering daily dot com slash ray gun and try it out for yourself wow