Episode #5: Event-Driven Applications using Amazon EventBridge with Mike Deck

About Mike Deck Mike Deck is focused on building a community of partners around the AWS Serverless Platform. He spent the first half of his career as a full-stack software developer building enterprise applications and coaching teams on agile delivery methodologies. For the last four years he’s been working as a solutions architect for AWS on the partner team helping both ISVs and consulting partners accelerate their pace of innovation using the cloud. Twitter: @mikedeck EventBridge Product Page: https://aws.amazon.com/eventbridge/ EventBridge Documentation: https://docs.aws.amazon.com/eventbridge/index.html EventBridge Partner Information: https://aws.amazon.com/eventbridge/partners/ Transcript Jeremy: Hi, everyone. I'm Jeremy Daly, and you're listening to Serverless Chats. This week, I'm chatting with Mike Deck from AWS. Hi, Mike. Thanks for joining me. Mike: Hey, Jeremy, thanks a lot for having me. Jeremy: So you're a Solutions Architect at AWS, and I'm pretty sure most people are probably familiar with what Amazon Web Services does. But why don't you tell the listeners a little bit about yourself and maybe what a Solution Architect does at AWS? Mike: Yeah, sure thing. So I'm actually on our Partner team, so I work as a Partner Solutions Architect, which means that I work with both our ISV and consulting partners to help them with kind of any technical questions they have, work with our ISV partners on their product roadmaps and how they're integrating with our services, helping them with architectural questions, and things like that. Been at AWS for about four years at this point. Been on the Partner team the whole time, and most recently, I've been kind of specializing in the serverless space. So working with a lot of our great ISV and consulting partners that are doing things around Lambda and API Gateway, as well as with the new service that we just recently launched. Jeremy: Cool. All right, so speaking of services recently launched at AWS Summit New York, AWS launched this new product called EventBridge, which is sort of this, and you can correct me if I'm wrong, but sort of this cool extension to CloudWatch Events and since you're on the Partner Team or you're the Partner Team Solutions Architect for Partner Integrations with EventBridge, you obviously know probably more about this than anybody else. So why don't you tell us a little bit about EventBridge and sort of what it does? Mike: Yeah, absolutely. So I think you know, it's definitely accurate to compare it to CloudWatch Events. So really kind of the genesis of this service was that, you know, we saw customers building more and more with this kind of event-driven model, and CloudWatch Events is really a fantastic tool for for doing these kinds of things. I think Forrest Brazeal had a blog post a little while ago about using CloudWatch Events to do awesome event-driven things. We see a lot of customers that are interested in this space. The event for pipelines projects came out, got a lot of traction, and so we realized that, you know, there's really this kind of need to build additional services that make it easier to build these kind of things. So we took the existing CloudWatch Events infrastructure and APIs and kind of extended them to add some additional features around integrating with SaaS providers to create more native event sources that you can use within your AWS applications, and then, yeah, extended those APIs to make it easier for customers to do things like creating custom event buses and patching to SaaS event sources, etc. Jeremy: So what are some of those use cases then that customers might build with this? Mike: Yeah, so, you know, I think that one of the obvious ones is: Hey, I want to trigger a Lambda function every time someone creates a new ticket in my CRM, for instance. Right? I want to go and kick off some sort of workflow. Or maybe I'm going to start a step functions or do something like that. Um, we also see a lot of customers interested in doing kind of audit-and-analytics type workloads. So I just want to ingest kind of the full event stream of all of the things that have changed out, you know, maybe in my identity management tool that I'm using. So every time a failed login happens, or every time a new user is registered, I just want to ingest that, throw it into a Kinesis Firehose Data Stream, and put it out into an S3 data lake so I can go and create with Athena or something like that. And then, obviously, you know, ML and and doing kind of AI inference and things like that on all of these various data streams is becoming super popular as well, so this gives you a great opportunity to, yo u know, every time a new email is opened in your kind of customer engagement platform, you can ingest that and add it to your modeler or do some sort of inference on it in order to drive additional kind of business decisions. Jeremy: Yes. So those are some really cool sort of things that you can do with it. And I remember Forrest's article about sort of using CloudWatch Events and using custom events as basically almost like an SNS topic, in a sense. So maybe let's get into the nuts and bolts of EventBridge, kind of how it works. So with existing CloudWatch Events, you can use a, put events API or put custom events API or something like that, where, and then Forrest explains that in his article where you can send an event and then you can subscribe consumers to it. But why this extension? What's this idea of having separate event buses? Mike: Sure. So, um, yes. So I guess there's a few different reasons that you might want to have different event buses. So like you mentioned with CloudWatch Events, there's really just sort of the default event bus in each region of your account. We publish all of these sort of native AWS events to that default event bus. And so those just kind of appear in your account automatically without your having to do anything. You know, the ability to create additional custom event buses now lets you kind of segregate these different application domains or different context, I guess, for the various different types of events that you need to ingest across your different surfaces and applications. It also gives you a good way to create separate channels for different SaaS applications that you may have that you're trying to build event-driven architectures on the back of. So, yeah, we can we can talk a little bit more about kind of the specifics of how it all works, but basically, every individual SaaS provider that you're integrated with is going to have its own event bus so that you can write rules against that, and keep all of those different events separated from the different sources. Jeremy: All right, so I definitely want to get into the SaaS side of things because I think that's probably one of the coolest features of this EventBridge. But let's go back, maybe, and start at a lower level, talk about events, sources and targets. So what events were you, because you mentioned that you can get all of these events that are happening within your AWS account. So anytime a new account's created or new resource is created or something changes, there's these events that are sent to that default event bus. But now with this separate event bus, how would we subscribe sources to that or set targets to trigger Lambda function or other other s...