Episode #4: Serverless Development Workflows with Chase Douglas

About Chase Douglas Chase Douglas is the co-founder and CTO of Stackery.io, the leading serverless acceleration software solution. His experience spans the gamut of technical and managerial, specifically focused on how teams of developers build products collaboratively. In prior roles he has been a VP of engineering at a web application security company, technical architect of the New Relic Browser product, and an architect of the multitouch implementation for the Linux desktop. Twitter: @txase Email: [email protected] Stackery: stackery.io Stackery Blog: stackery.io/blog Stackery Changelog: https://docs.stackery.io/en/changelog/ Live Stream Series: https://app.livestorm.co/stackery/ Portland Serverless Meetup: https://www.meetup.com/Portland-Serverless-Architecture-Meetup/ Transcript Jeremy: Hi, everybody. I'm Jeremy Daly and you're listening to Serverless Chats. This week, I'm chatting with Chase Douglas. Hi, Chase. Thanks for joining me. Chase: Hey, glad to be here. Jeremy: So you are the CTO at Stackery, which is in Portland. Why don't you tell the listeners a little about yourself and what Stackery does? Chase: Yeah. So I'm the CTO and co-founder of Stackery and I've spent my career figuring out how to manage complex systems. And as serverless as an architectural pattern started to take off I was really interested in finding how do we help people adopt that architectural pattern more easily? Stackery is a product, a tool set that makes it easier for anyone from individual developers on up to teams and organizations, but especially at larger sizes, manage to design, manage environments, deploy and at the other side, help monitor their serverless applications. Jeremy: I wanted to have you on the podcast today because I want to talk about serverless development workflows. And I think when people start moving into the serverless paradigm, we gotta sort of change the way that we think about developing applications and that you know everything from whether you're developing locally or developing remotely, or whether you're trying to do something like offline emulation or trying to do the remote testing things like that. Just what are some of your thoughts on sort of this idea of these serverless development workflows? How does it sort of change things? Chase: Yeah, serverless itself is a different way of building, developing and including testing applications. And one of the things that we have to step back and recognize is that at the end of the day, we're still developing software, we're still testing software, but we need to find the right ways to be efficient at how we do those. It's slightly different in a serverless world, and so we once we find the right patterns. And once we start to use those as an individual or in the team, things actually speed up once again. So there is an interesting play here. Uh, but it's all about just finding the right mix and match of how to do the things we're familiar with when it comes to the development and testing. Jeremy: Yes, so that makes a ton of sense. So what I think I'd like to do is sort of dive down into a number of these different topics, you know, break it down a little bit and get into I mean, because again, you're an expert. Stackery obviously, is all about building out these workflows or helping developers build these workflows. So I want to get into these these details here and let's start with sort of just really maybe 30,000 foot view. How has cloud sort of changed the way that we develop software? Chase: Yeah. So the way that we've always developed software up until very recently was it would, in the end, be running on servers, whether it's in a data center or in the cloud. But these servers were monolithic, compute resource. That meant that typical architectures might be a LAMP style stack. You've got a Linux server, and you've got a MySQL database off to the side somewhere, maybe on the same machine, maybe on a different machine. But mostly as a developer, you're focused on that one server, and that means that you can run that same application on your laptop. So were we become very comfortable. We built up tooling around the idea of being able to run an entire application on our laptop, on our desktop in the past, that faithfully replicated what happens when that gets shipped into production in a data center or in the cloud. With serverless, everything is kind of a little works differently. You don't have a monolithic architecture with a single server somewhere or a cluster of servers, all running the same application code. You start to break everything down into architectural components. So you have an API proxy layer. You have a compute layer that oftentimes is made up of Lambda, though it can include other things like AWS Fargate, which is a docker-based, serverless, in the sense that you don't manage the underlying servers approach. So you've got some compute resource, if you need to do queuing instead of spinning up your own cluster of Kafka machines, you might take something off the shelf, whether it's SQS from AWS or their own Kafka service or Kinesis streams. There's a whole host of services that are available to be used off the shelf. And so your style of building applications is around how to piece those pieces together rather than figuring out how to put those and merge those all into a single monolithic application. Jeremy: So how then do developers need to think differently? I mean, again, I'm super familiar with the LAMP stack. That was probably where I started. Well, I started with Perl and static text files, but we won't talk about that. But as we got a little bit more advanced and we started using things like the LAMP stack, obviously, it was very easy for us to just either test it locally or to even building in the cloud. It was, or not the cloud on our hosting provider, but we could just easily upload a new file and things would magically work for us. But as you mentioned, things get more distributed. Right? Once we go into this cloud environment, you've got multiple services working together. You don't own those services necessarily. If something breaks with those service is you kind of have to deal with that. So maybe what are some of the limitations that a developer might have to deal with when they're starting to move, you know, their production workloads to the cloud? Chase: Yeah, for all the benefits you get from serverless, with its auto scaling and its capabilities of scaling down to zero, which reduces developer cost, you do have some things that you have to manage that are a little different than before. One of the key things is, if I've got, like, a compute resource like a Lambda function in the cloud that has a set of permissions that it's granted and it has some mechanism for locating, the external service is like SQS queue or an SNS topic or an S3 bucket. So it has these two things that it needs to be able to function the permissions and locations. So the challenge that people often hit very early on in serverless development is if I'm writing software on my laptop and I want to test it without having to go through a full deployment cycle, which may take a few minutes to ah to de...