Episode #16: Serverless Workflows using Step Functions with Rowan Udell

About Rowan Udell: Rowan Udell is Cloud Practice Director at Versent, an AWS Premier Consulting Partner in the Asia Pacific region. Working with customer and internal Versent teams, he helps them deliver change at scale and speed using serverless and AWS native services. He co-authored the AWS Administration Cookbook and has published video courses on AWS. Twitter: @elrowan Blog: blog.rowanudell.com AWS APN Ambassador: https://aws.amazon.com/partners/ambassadors/ambassador-apac/ AWS Administration Cookbook: https://www.packtpub.com/virtualization-and-cloud/aws-administration-cookbook (2nd edition coming out soon) Transcript: Jeremy: Hi, everyone. I'm Jeremy Daly and you're listening to Serverless Chats. This week, I'm chatting with Rowan Udell. Hi, Rowan. Thanks for joining me. Rowan: Hey, Jeremy. Thanks for having me. Jeremy: So you are the technical director at Versent which is in Sydney, Australia. And you are also an AWS APN Ambassador. So why don't you tell the listeners a little bit about yourself, what Versent does, and actually, I'm kind of interested in this AWS APN ambassador thing, if you could tell us about that as well. Rowan: Yeah, sure. So Versent is a premier consulting partner here in Australia and, you know, we work with a lot of enterprise customers, really helping them do cloud the right way. You know, if I was to describe us to a wider audience, we kind of want to see ourselves as the heart specialists for AWS. You know, when you have a heart problem, you go to see a specialist. You don't just go to any old doctor and we want to be that for AWS. In my role as technical director, I work with a lot of teams that are using Step Functions and other serverless technologies, building out applications on AWS. I'm part of the APN Ambassador Network, which is a new program that AWS has started up for consulting partners. APN Stands for the AWS Partner Network, and what they've done is get together a group of like-minded partners in one room so that we can kind of give feedback to AWS but also help them get new features, services, technologies out into a wider audience, you know. And so a big part of what we do is things like coming on this podcast, but also doing blogs, doing meet ups, giving speaking events and things like that. And so they really kind of encourage us and enable us to get the word out there and try and make AWS easier to use for everybody, not just consulting partners. Jeremy: Great. And what about your background? Where do you come from? Rowan: Yeah, So, I mean, I've been working in IT for longer than I'd like to admit now, you know, mainly with AWS, especially over my last four years here at Versent, just working purely with AWS. Before that, I was leading developer teams, working at some startups, things like that. And, you know, when the cloud came along, I really kind of jumped on board because I was sick of administering servers in the first place. And that's probably another reason why you see me online talking a lot about serverless stuff. Jeremy: Awesome. Alright, so I have had a number of episodes where we've gone down sort of the technical path of some subjects. Some of them we've talked more about the business things, but I know that you do a lot of work with Step Functions, both as obviously as your role as a consultant, but also just I think just what you're doing out there working with teams and what you're seeing. So I want to talk about Step Functions today, AWS Step Functions. I think this is a really fascinating service that they offer. I think it's under utilized by a lot of people. I mean, it certainly has good use cases and use cases that may not be the best for it. But maybe so if people don't know, what are AWS Step Functions? Rowan: Yes, so AWS Step Functions is the name of Amazon's state machine as a service offering that they have. State machines are sometimes called finite state machines, and this just makes them much more easily consumed and also integrated with your serverless applications. Jeremy: All right, so let's let's start high level here. What are state machines? Rowan: Yeah, so a lot of people get turned off by some of the terminology. But I mean, really, it's just a lingo or, you know, some vocabulary that's actually been around for quite a while. State machines are nothing new. State machine’s really just a mathematical way of modeling an application. Most people kind of understand what that means conceptually, but what it means in reality is you can describe your application as a mixture of your inputs, the states that your application can be in, and the transitions between those states. And what this forces an application designer or really a developer to do is to think really concisely and clearly about what their application does and what it could do next, and it kind of forces you to do that upfront, which I think is something that's really valuable and often overlooked. Jeremy: Yeah, and one of things that I really like about state machines and specifically Step Functions in the serverless world is that ability to do function composition. Because I think that's one of the things that many people are confused about. Like how do I have function X talk to function Y, right? So state machines are the glue in between those, right? Rowan: Yeah, definitely. And you bring up a really good point because we see a lot of people out there discussing on forums and in Slacks about like, “Should I have one function call another function directly?” and usually someone will jump in and say, “Oh no, you should never do that.” But obviously, then that's going to complicate things a little bit. And in some ways, if you're using Step Functions that problem goes away because you're able to link two functions together, you know, which allows each one of those to do one thing well, and you don't have to worry about calling them directly and coupling those two functions to each other. At the same time, you don't have to introduce things like SNS topics or SQS queues in between those functions. You know, in my opinion, it's kind of the best of both worlds. Jeremy: Yeah, I mean, and that's where we're talking about orchestration versus choreography, right? So and that's one of the things that if you're using SNS or you’re using EventBridge or using some other communication channel or messaging bus, you can decouple the applications — state functions are a way of sort of creating coupling. But it's a different type of coupling, right? Because a function can run on its own outside of Step Functions or it can be part of several Step Functions, right? Different steps within that could reuse that same piece of logic. It's just a way of kind of gluing all that stuff together, like you said. Rowan: Yeah, it is a form of coupling, but it's a loose coupling. You know, the function that is calling or being called doesn't know that it's being called by another function, to your point where it might be part of a complicated workflow or it might not. It really doesn't matter to the implementation function. Jeremy: So what about the visualization of these things, too? That's another sort of important piece, right? Rowan:...