Ask a Cloud Economist: Cost Attribution in AWS

LinksFollow Pete + Jesse on TwitterTranscriptCorey: This episode is sponsored in part by our friends at Linode. You might be familiar with Linode; they’ve been around for almost 20 years. They offer Cloud in a way that makes sense rather than a way that is actively ridiculous by trying to throw everything at a wall and see what sticks. Their pricing winds up being a lot more transparent—not to mention lower—their performance kicks the crap out of most other things in this space, and—my personal favorite—whenever you call them for support, you’ll get a human who’s empowered to fix whatever it is that’s giving you trouble. Visit linode.com/screaminginthecloud to learn more, and get $100 in credit to kick the tires. That’s linode.com/screaminginthecloud.Pete: Hello, and welcome to AWS Morning Brief. I am Pete Cheslock.Jesse: And I'm Jesse DeRose.Pete: We're back again, and we're here to answer an audience question. So, every once in a while people tweet at us—you can tweet me @petecheslock. Jesse, what is your Twitter handle?Jesse: @Jessie_DeRose.Pete: Yeah, mine is just petecheslock. I do feel bad for the other Pete Cheslock, who actually does live in Boston as well because taking all of his profile names.Jesse: You should change yours to @therealpetecheslock, or he should change his to @therealpetecheslock, and then it'll just be an ongoing escalating battle.Pete: That's very true. So, occasionally on the Twitters, we get questions asked of whatever around Amazon cost management, things like that. And we wanted to actually take this opportunity to answer one of the more interesting questions that we received. Because granted, sometimes we get questions and they're pretty boring, so we don't answer them. We just focus on the fun ones, [laugh]—Jesse: [laugh].Pete: —selfishly, but we got this question that was really interesting. It had to do with someone who is essentially starting over within Amazon Web Services, meaning they were going to be redeploying their application into a series of new AWS accounts. And they asked us, “What are the most recent best practices—” I hate that term, but the important things you should do and consider when you're deploying into Amazon, into AWS. And we kind of sat back, we thought to ourselves, “Wow, how often does someone have that opportunity?” Right, Jesse?Jesse: Yeah. Not in any of my experience has that happened for me. I'm very, very envious of these people.Pete: Yeah, I had that opportunity one time, where we were essentially doing that, like, net-new, starting over. But this was years ago, where there wasn't a lot of insight into this, and we didn't have the features like we have today where Amazon organizations—AWS Organizations—allows such an easy way to create accounts and get started with multiple accounts. So, anyway, we want to take this opportunity to talk about what we believe and what we see as the things that you should focus on, what you should optimize for when getting started, when creating, kind of, net-new in AWS.Jesse: Yeah, there's a lot of different things that you can optimize for in AWS, and it really depends on what your business goals are; what do you ultimately want to accomplish when you are deploying your application into the cloud? But one of the big ones that we see, selfishly, here at Duckbill Group is cost optimization. And so we wanted to talk a little bit more about cost allocation and cost attribution—which are essentially the same thing, we may use the terms interchangeably in this conversation—to talk about how you can think about cost attribution, why you should think about cost attribution and some of the best ways to go about implementing that in AWS as you're building these new accounts, this new space.Pete: Yeah, and that being said, I really like people to really think when they create these things. Again, what are you optimizing for? Some people might say, “Oh, well, we want to optimize for security.” And that's great. You absolutely should do that.Jesse: Sure.Pete: Security is a first principle, something to absolutely focus on. But what if I told you that the other, probably, most important thing in AWS is—and something if you're not doing it today, you're going to be asked to do it in the future—is accurate cost attribution. And what if you could do both highly secure accounts, and segment based on security, but also get this cost attribution? That is, I think, what we're going to dive into today.Jesse: Yeah, I think that there's a lot of big conversations around engineers, and multiple other teams when you start talking about the DevOps movements, the DevSecOps movements, all these movements of the software engineers who are actually writing the code and the engineers or the operations folks who are—maybe—managing the infrastructure, maybe deploying the code, maybe the software engineers are deploying the code, it really depends on your team setup. But there's this, kind of, idea that the engineering teams that are working with this code, and then there's all these other teams in the company that have other things that are their top priority, and starting to bridge that gap to have conversations with finance to better understand what do they need to know from you about how you're spending money in AWS, and security who wants to better understand are we patched for the upcoming audit? Are we compliant based on these terms? It's really important to start thinking about how you optimize in AWS based on those ideas, those conversations with other teams. So, that's kind of ultimately what I'm thinking about, specifically, today, specifically about the conversation between finance and engineering and talking about cost attribution.Pete: But Jesse, aren't tags supposed to solve all of my problems when it comes to cost allocation?Jesse: [laugh]. Oh, I wish. They are supposed to. There's that whole idea of ‘set it and forget it,’ there's a big movement of ‘tag it and forget it,’ and as much as I want to believe in that, it’s unfortunately just not true. Like, tagging is definitely a first step, but it goes so much further than tagging and I think that's one of the big things that a lot of folks miss or don't think about when they're talking about tagging and cost attribution.Pete: If you loved it, you would have put a tag on it.Jesse: [laugh].Pete: But really, while tagging is an important thing to do, and we've seen some of our clients, their tagging percentages can be upwards of 90 percent, which is herculean in ability and effort to reach that level of coverage, but even then, getting that last 5 to 10 percent in many cases could be actually impossible to do because there can be a series of spend within Amazon which is just untaggable, or at least untaggable in a realistic way. And that's where multiple accounts can really help your busine...