Episode #1: Serverless Purity vs. Practicality with Alex DeBrie

About Alex DeBrie: Alex is an Engineering Manager at Serverless, Inc., a blogger, and a big fan of serverless. He's held a variety of roles at Serverless, from Data Engineer to Head of Growth to Product Manager. He's also an important voice in the serverless community, creating in-depth guides (like DynamoDBGuide.com) to help users build the future with Serverless. Blog: alexdebrie.com Twitter: @alexbdebrie DynamoDB Guide: dynamodbguide.com Serverless, Inc.: serverless.com Transcript: Jeremy: Hi, everybody. I'm Jeremy Daly, and you are listening to Serverless Chats. This week, I'm chatting with Alex DeBrie. Hey, Alex. Thanks for joining me. Alex: Hey Jeremy. Thanks for having me on. Jeremy: So you are an engineering manager at Serverless Inc. ⁠— that's Serverless with a capital "S," not to get confused. They're out of San Francisco, but you actually work out of Omaha, Nebraska. So why don't you tell the listeners a little bit more about yourself and what Serverless Inc. is up to. Alex: Yeah, sure. I've been at Serverless, Inc. for two years now. I started originally on the growth team, and now I'm working on the engineering team. But, you know, Serverless, Inc. we're the creators of the serverless framework, which is a tool for developing and managing serverless applications. It really reduces the tedium around setting up API gateway and IAM and all that stuff, and really helps you write your business logic and use AWS Lambda and serverless technologies effectively and quickly. There's a huge community of advocates, plug-ins, and best practices around the Serverless framework. I think we just crossed 30,000 stars on GitHub. So I'm really loving what we're doing here. Jeremy: That's awesome. Yeah, I think if somebody doesn't know what the Serverless framework is yet, then they haven't been paying attention for the last couple of years. So you also write a blog, and you have a really, really good resource for people who are interested in learning DynamoDB, and people who are using DynamoDB and want to learn how to use it better. That's DynamoDBguide.com. That and your blog ⁠— what's going on with that stuff? Alex: Let's start with DynamoDB Guide first. This was when I was still on the growth team at Serverless. I was doing a fair bit of content writing, and we were using DynamoDB a lot. I watched the 2017 re:Invent talk from Rick Houlihan, who's this wizard that works on DynamoDB at AWS. He did a talk on some best practices and I just loved it. I think I watched it four times in two or three weeks. This was Christmas break 2017, and I'm like, "I've just got to get some of this stuff out here." I wrote the resource that I wish I had when I started with Dynamo, because I thought I knew it well, and then I saw Rick teach it, and I did not. So DynamoDBGuide.com ⁠— it has a walk-through of all the different API stuff around DynamoDB, secondary indexes, all that stuff, as well as some data modeling examples too. Jeremy: And your blog is mostly serverless and S3 batch stuff, all kinds of stuff like that, right? Alex: Yeah, my blog I would say is a lot of, again, sort of like DynamoDB Guide, just the guides I wish I had when I started. I think both with DynamoDB Guide - and then a lot of the content on my blog - it's stuff I was familiar with, and then I want to teach it to people. And then when I teach it, I find I learned a lot of stuff that I actually didn't know. So it helps me, and I hope it helps other people as well. Jeremy: Great blog, and the DynamoDB Guide is awesome. And yes, Rick Houlihan is a wizard and I don't know how he does some of things he does. But I have watched his 2018 podcast or the 2018 [re:Invent] has a podcast version that I think I listened to maybe 50 times on like .75 speed, so that you can maybe understand it. So anyways, I wanted to have you on because I want to talk about this idea of serverless purity versus practicality, right? I think that we see a lot of debate on Twitter - and forget about what serverless is and what serverless isn't - but more so, what's the right way? How should we build a serverless app versus how we can practically build a serverless app? I think there's a lot of things around that, whether it's developer experience and that sort of stuff, but what are your thoughts on this sort of debate? Alex: I think it's pretty fascinating to see. Like you say, if you're on Twitter and you're following a lot of the big time people doing serverless architectures in this space, they have a lot of great tips around best practices, and this is what you should be doing, all that stuff. But I find, as I'm building serverless applications or as I'm talking to customers and users that are building serverless applications, there are times when there's tension between what the best practices are and what their circumstances are. This could be because maybe they're not coming in with a green field application, or maybe they have a data model that doesn't fit DynamoDB or something like that. It's difficult on how you sort of square that with recommending something that you know isn't the best practice or the most pure serverless application, but you also gotta help people ship products, right? I think balancing that tension can be tough at times. Jeremy: Yeah. I want to dive into a couple of these discussions that we've been seeing on Twitter, and I think, like you said, there are a few champions who sort of lead the effort for each one of these. But let's talk about the API Gateway service integrations. So we know that the typical serverless model would be API gateway to Lambda function, and then access something else. But it's possible to do that without using Lambda, right? Alex: Correct. Like you're saying, you can do what's called an API Gateway service integration, where maybe you take that incoming HTP event, maybe you validate, authenticate it, maybe twist up the shape a little bit, and then you can put it directly into a different AWS service, like SNS, SQS, Kinesis, something like that, rather than going through a Lambda function first. Jeremy: If you're just sending the data straight in, and you've got maybe a Lambda authorizer, that's one thing, but what about if you're transforming the data? That's seems like a different beast than writing some Node or some Python. Alex: Yeah, absolutely. API Gateway allows you to write what are called VTL templates, so it's in Velocity Template Language, which I believe is an Apache project. It's a semi-declarative templating language where you can take some input, like a JSON payload body, the headers, all that stuff, and create a different shape that you want that satisfies the API format of whatever service you're integrating with. It's doable, but I would say not a lot of people have experience with VTL, so it's definitely a learning curve there. It has some quirks and unexpected stuff for people that are new to VTL. Jeremy: You mentioned the quirks. I'm thinking to myself, here I am writing an application. I've got all my tooling in place. I've got my testing frameworks and I can test all this stuff. Then I say, okay, well, I'm gonna I'm gonna go pure - API Gateway to DynamoDB - and I'm going to write some transformations in VTL. I do that, and ...