Episode #67: The Story of the Serverless Framework with Austen Collins (PART 2)

About Austen CollinsAusten Collins is the founder and CEO of Serverless, Inc. Austin is an entrepreneur and software engineer located in Oakland, CA. His specific focus is on building cheap, scalable Node.js applications while minimizing DevOps requirements as much as possible. An enthusiastic AWS Lambda user from day one, Austen founded the Serverless Framework (formerly JAWS), an open source project and module ecosystem to help everyone build applications exclusively on Lambda, without the hassle and costs required by servers. Serverless, Inc.: https://www.serverless.com/Twitter: https://twitter.com/austencollinsTranscriptJeremy: Yeah. And I mean, I think that was one of the things that I thought was amazing about how you took the approach to the community, because it was very much so a community driven project. And I think that helped to have AWS who is also very much so like what do they call it? The leisure paths or whatever it is when you basically create paths in a college campus by letting people walk and then you pave over the dirt or whatever it is. I can't remember the name of it, but anyways, that idea of letting people sort of decide where it's going to go and how it's going to develop. And I always remember there being a delay, because there had to be between the framework supporting something new that AWS just came out with.And whether that be and again, I'll use a bad example, but like maybe it's a new event that it supports. And I remember every time that came out that there was always it seemed like there was a lot of debate, a lot of pull requests and conversations about how to abstract that piece of it and fit it into the existing framework. Right. And so you developed this entire plugins community around the framework too, which was really great because that was the other thing. And that has been one of the things that I've had challenges when I work with SAM is that you don't have the ability to build plugins. So you essentially have to write a separate Lambda function that just does a module or whatever they call it there that allows you to do something, a custom resource.But with the framework, you just had that ability to write that, to do things that you to do, to extend it, to fill gaps that you may have had during a certain amount of time until the framework supported it. But I always appreciated that more so after the fact, sometimes I was a little frustrated that there wasn't support right away. But after the fact to say, I liked the fact that you took the time to think it through because abstractions are the hardest thing to build and when you do it wrong and then you're married to it forever you know what I mean? It's really hard to change that. So that was one of the things I always liked was that it was the framework doesn't support this yet, but build a plugin and it can, and then you can always deprecate the plugin and then accept whatever the valid way is or I guess the established way that the framework does it.So I just thought that was a really good approach. And I think really helped with all these people contributing because then also you saw plugins that came up and you're like, we should support that in the framework.Austen: Yeah, absolutely. I'm glad that you brought that up. That was something that I think on one end the hallmark of like a great developer tool often is how extensible it is. Right. But a lot of this just came out of the fact that I started as one person and I couldn't take on this massive project, just being one person in the early days. And those people who've had, like things go viral on Hacker News. I think they have probably been through a similar experience where one day this project is just blowing up, but the next day, the hangover sets in, when you look at the issues in GitHub. And people say this project does not support my use case, this does not support my workflow.This does not support my organization's policies. And then on top of that, the ambitious goal of trying to abstract almost essentially it's just so many AWS services. It was like, okay, how is this ever going to work? And that's where plugins came from largely, and you're right, you honed right into it. People can actually just overwrite any part of the framework, extend it, and it's up to them to figure out what the right patterns are. And once we see those emerge, then we'll just merge that into the framework. So it was a lot of outsourcing product development to some extent, but all goes back to just the fact that we didn't have the resources to do all that in the early days and it's great. That's where creativity comes from a lot of the time. It's just limitations.Jeremy: Actually. It's funny you mentioned limitations because that's the other thing... This has been an ongoing theme with serverless too, is that you do have to work around some of these limitations, but sometimes that's not a bad thing, right? Sometimes when you have constraints, you can build something better because you can't just go and do something crazy, like say, well, we're going to set up a K8 cluster because we want to do some type of processing over there. If you ask yourself, how can I do it within these constraints? Not only is it usually faster, it's a lot cheaper. And oftentimes I think you'd get a better product out of it.Austen: Yeah, absolutely. I totally agree with that. Constraints are super important. And also, I got to say the amount of stuff, use cases, that people were trying to use serverless for, especially in the early days when it really wasn't meant for it, was just so amazing to me. People would try and duct tape together crazy serverless architectures to accommodate a use case it clearly wasn't ready for. There's so much creativity in that area. And it seemed pretty crazy, but it's just people love those serverless qualities. Auto-scale, only pay for use and massive scalability. People want that so bad. The fact that so many people were doing this and they still do it a lot today because there's still not even great serverless database options, right.The stuff that people are doing to get around that, the DynamoDB movement right now, the popularity and all that I think is a lot of it is due to the pent up demand that is just waiting for the cloud to become more and more serverless and offer more and more serverless options for all use cases. And that's really I saw that early on and I'm just thinking like, everybody wants this, this is going to be table stakes for the cloud. Again, this is not a fad. This is like the next... This is cloud 2.0, that's cliche to say, I hate to saying that, but this is like the evolution of where-Jeremy: Cloud 3.0 now I think we're already getting towards 3.0. So 2016, 2017, 2018 you're working on building a community which you did an amazing job of. And I know you hired a few people. I know Alex DeBrie, for example, a ton of growth hacking, which is just brilliant by the way. So if you just take your serverless hat off for a second and go back and look at how you built this company and how you built this community, it is like a masterclass in how that worked. Now, I know I'm sure you made a lot of mistakes along the way, like you said, for every one success, there's 100 failures behind you. But again, it was the persistence. It was the approach to it. It was hiring a bunch of great people. And like you said, having some really, really great people working for you and working on the project that I think got it to the success. But now we get into 2018, maybe towards the end of 2018, 2019, you'v...