Welcome back to Invisible Machines. I'm Josh Tyson, a contributing editor here at UX Magazine, and I'm joined again by resident visionary Rob Wilson, CEO and co-founder of OneReach.ai. Rob and I are co-authors of the best-selling book Age of Invisible Machines, and this podcast is where we continue our explorations of agentic AI. Today, we're excited to welcome back returning guest Cassie Kozarkoff for a deeper exploration of decision science and how it relates to an AI-first approach. to technology adoption. Cassie was the first chief decision scientist at Google, and she currently advises world leaders and executives on how to optimize their most critical decisions. As you'll hear, Cassie makes the point that with AI, we now have a memory prosthesis, one that opens up headspace for all sorts of new learning, and she shares some great advice with us on how to become an AI-first individual. In terms of becoming an AI-first organization, Cassie says to look in the attic for all that shit that seemed impossible to automate, and she cringes at the low-hanging fruit.
of using AI to automate processes you already have clear instructions for. That's really just a taste of everything we explore in this episode. So settle in for another fascinating conversation with the brilliant Cassie Kozarkoff. It's so interesting. Like, you know, we've talked a while ago. We've talked to a lot of folks. I've talked to you. A lot of really, really smart people. And I have to tell you, and I'm not trying to toot your own ear, but it made me realize how smart you are.
Of all the people we've talked to, in the last conversation we had, it's like etched in my brain. It's like, that person's brain is incredible. But then I sat down and I'm like, what do I want to talk about? And I have like two pages. And I kind of kept going. Let's do it. Let's keep going. I've got nothing more planned today except theater in the evening. Oh, we were going to the theater last time too.
I'm always going to the theater when I can. It's my drug of choice. It's my therapy. I love the theater. The theater's amazing. It is.
I do love the essentialness of what theater stands for, which is that we come together into a room to tell stories. That's essential in that way of probably pre-language humans would sit around campfires and, you know, make gestures of the hunt and the scary animals and act it all out with their bodies telling stories. And we've just gotten better and better and better at that storytelling. And no matter how digital we get, I'm still such a big believer of actually coming into that room together, of resonating as an audience and of exchanging those stories. We've talked about that a lot on this podcast.
that especially as people get more and more inundated with generated content that truly marvelous human content might become more of a premium like going to the theater might be more important than ever and people might be more interested in seeing live music because they're hearing so much generated music and I like that idea and I think theater is really special too because like when you go see a movie it's good to see I like seeing movies with in a theater full of people because that's a unique experience but the theater is like an even deeper shared experience because it's being created in real time and whatever's going on with people on stage is coming through like it's it's so much more pure in that way Yeah, and I think we don't spend enough time really owning how good that is for us to interact with other people, to resonate with other people, to be in a group experiencing something. And the one prediction could be that theater and such things become more important in the digital age. But we must be careful because you can see how...
Junk foods and terrible made-of-chemical snacks have become a prevalent form of food, right? Though you might say, look at all these things that we clearly need for good, healthy bodies and good, healthy minds. And yet so much is available that isn't a substitute, pretends to be a substitute, but really isn't. And so we could see digital experiences failing to fulfill some of those in-person needs, masquerading as fulfillment. Empty calories of socialization. Empty calories of socialization, empty calories of human interaction.
Yeah, it probably all stems, like the kind of core of it is like the shame of loneliness and that, you know, not becoming something shameful, but just like, I'm hungry, I'm lonely, not I'm embarrassed to say I'm lonely. And then like, what do you do when you're hungry? You go get some good real food. Sure. I mean, that's what you hopefully do. None of us is perfect. I have absolutely walked into a 7-Eleven and went, I will have food here. I don't think that there is much food in a 7-Eleven. Wait, I can put nacho cheese into the Doritos bag? That's amazing. I'm going to do that and go home and talk to my AI. Diet, proteins, fats, carbs, you've got everything you need. AI and machine learning, those types of inquiries. What's special about them? Well, if it's based on data.
then all the considerations involving data and data not being truth and data not being perfect and data not necessarily being representative or useful, all those concerns come into play. But more than that, why do we automate with data? Because AI and machine learning, it's all about automating with data. And all these other chat GPTs, et cetera, are built on top of automations with data. So why would we do that? And so what I like to ask leaders point blank, I want them to think about this. I tell them, let's forget these fancy words, data and code.
Let's instead use humble words like examples and instructions. Data is just examples in electronic form. Code is just instructions in electronic form. And then I tell them, I want you to imagine that you are going to teach a human how to do something that is very, very important, mission critical. No messing around is wanted there. They have to get the task right. Tell me why you might prefer instructions over examples. All right, you're teaching a human. Why would you give them instructions instead of saying, hey, watch me do it a few times, you figure it out. What is it that's attractive about instructions? And so what might you say? What comes to mind?
Control. Like sense of... Control. You got it. You got it immediately. Sometimes I have to, with my audiences when I'm giving a talk, oh, I have to pull it out like a medium. Like someone is saying reproducibility and someone is saying deterministic and I have to kind of, you know, sculpt this all together into that word, which you got immediately, which is control. Right. Instructions are great for control. So then I ask, for a task that's so, so important, what would possess you to give up control?
Right. Don't go, don't be doing this AI machine learning data thing because the Joneses are doing it. Why are you giving up control? Control is a good thing on a mission critical task or an important task, a valuable task, a task with this ROI. Why are you giving that up? So what would, what would possess you to do that really? Oh, to give it up? Like what's been censored? Yeah. Why? Why would you give up control? We're imagining this is a super duper important. It has to be done right. And you're teaching it to another human. What would possess you to, instead of giving instructions, give examples, which gives up that control when you give examples, because then you have to hope, you have to hope that the person sees in those examples what you see in those examples.
The only reason that it would not be completely nuts for you to do this is if you do not know the instructions. If you know them, you're confident that they're the right ones. You have no doubt. You have no reason to doubt your instructions. In that case, you teach with examples. Yeah, you're totally right. Now that we see this, we're like, okay, the only reason that I would use examples instead of instructions on something important, because when it's not important, then I use examples to be lazy, right? I can't be bothered to write out the instructions. I'm like, here, watch me order coffee a few times, you figure it out, and it's okay if you mess up. But where it's important, you're only giving up control, the control of the instructions, because you do not know them. Right. Over on a case scale, it's that you believe someone else's instructions will be better than yours. Exactly. So it is your fundamental humility relative to the task.
Yes. That possesses you to go to data. You either say, I'm not confident that the instructions that I've made are the best possible instructions. Right. You might say this task is so complicated that the instructions don't fit in my head. You might say this task has to operate with so much nuance and on so much scale that again, I can't fit everything that has to happen in that decision tree in my head. And so it is for that reason that I moved to humanity's memory prosthesis, otherwise known as data. So it is the complexity of the task relative to me.
Because if I'm smart and it's an easy task, I can quickly get the instructions, write them down, and off we go. This episode is brought to you by OneReach.ai, the runtime for AI agents. Their GSX platform provides turnkey private architecture and tools for building enterprise-grade agentic solutions. Recently, I worked with a team at OneReach to create an agentic companion for the second edition of Age of Invisible Machines. So now when you scan any of the QR codes found in either edition of the book, an agent is there to provide additional insight using a custom knowledge base. There's also a newsfeed feature with updates on agentic AI as well as this podcast.
and a book resources skill that provides clickable galleries of footnotes from each chapter. All of it built using the revolutionary Generative Studio X platform. Learn more at onereach.ai. So what is that illusion that I have better instructions than others, but I don't? The illusion that I... Right. Of confidence bias. Yeah, what is that? Yeah, let's talk about that. Because I think that's in the way, right? So if you don't ever question yourself, and, you know, the statistician and the philosopher, the epistemologist, which is the intersection between both of them, lives in a dark, dark world in which they're constantly questioning their own perspective. And they're forced to seek...
it attempts to poke holes in their own arguments. So let me hold it up to actual data and see if it holds water. But if you don't naturally have that way of navigating your life, you may find something that seems to fit a pattern, that seems to be a good explanation, or maybe it doesn't even fit a pattern. Maybe your grandparent told you something when you were five years old at a time when your brain was not mature enough to question anything at all. That fact lodged itself in there. And from now on, this is the right way to do things. So who knows? It can come from anywhere. But my point is that whether if you have the overconfidence, the foolish overconfidence to think that you have the solution,
you're going to teach by instruction. You're not going to go over the data. So the whole paradigm of using data for automation, using examples for teaching, the same thing as using data for automation, that whole paradigm is, I might not know the instructions. And so for any situation where that is true, data is a boon. Because it might be that when we go to this memory prosthesis, incredible memory prosthesis that humanity has built, and I really want to emphasize that data is a memory prosthesis and an attention prosthesis in the sense that, you know, I might show you...
photograph. And if beforehand I told you, pay attention to the color of the shoes of the person in the photograph. If you get it right, then I will give you a million dollars. I can be sure that you will remember that color. But if instead I tell you to pay attention to the haircut of the person in the photograph, and then I ask you later about the shoes, you might not remember the color. And yet, if I store that photograph as data...
Even afterwards, I can go back. And if my attention was on the wrong thing earlier, I can go back and pay a different kind of attention. Or I could pay attention to many photographs at once by collating them. We're not able to see all of Monet's paintings all in one go. Unless, of course, we stack them together with data. So it's a memory and attention prosthesis and it's quite big and powerful now that we've got so much data and of reasonable quality too and all accessible and moldable by our machine codes. Fantastic. Now we have the opportunity to use this memory prosthesis for making an attempt at automatically pulling instructions out of patterns in examples. And we only do this
When there's some reason to suspect that we cannot come up with the instructions ourselves, typically because we're trying to solve problems where the solution is so enormous, the problem is so difficult that we could not possibly write out every line of instruction. That's why we're here. So another way to say all this is that there is something complex about what we're dealing with. If it were simple. We'd have done it. We'd just written out those instructions off we got with a human, with a machine, because that's traditional programming. Going to machine learning, going to AI, invoking teaching by example at an enormous scale that overwhelms the human mind, that is invoking complexity. And so I'm not against AI, never have been. That's why you spend more than 20 years working in it, not because you hate it. I mean, some ethicists maybe, who knows. But for me, I love the possibility. I love the fact that at some point, programmers, designers, whoever, they try to come up with instructions for things like...
Let's come up with the instructions for identifying whether there is a cat or not a cat in a photograph. Or even before that, for the purpose of postal service, optical digit recognition, let's see what this zip code is automatically. Is this a 9? Is this a 2? So you take a picture, and then what do we do with each pixel in this picture to figure out if it's a 9 or a 2? To write that instruction is quite hard. Or you could show millions of 9s, 2s, etc. And then...
Look how well it works. So we've gone from all these tasks where we just hit a wall over and over. Oh, no, this is too complicated. This is too difficult. This is too big. And we cannot do it. We put it in this attic of automation wishes where we say, oh, we tried it. It's too hard. And now with data, we're just busting open that attic and saying, well, what impossible things might be on the table? Because suddenly we have enough examples and we've got the machinery to process them and we could take a shot at it. And what we're going to do is we're going to go after these things that are way too complex. So where I'm rolling around to this, and again, if leaders don't understand this, and I think most of them don't.
this trouble. Because we are not talking about AI adoption, or we shouldn't be. Let's forget that word AI when we say adoption. Because otherwise that sounds like we're adopting a baby robot and no good can come of this. So instead, I want people to think of this as complexity adoption. If simple solutions are not solving your complex problems, you're stuck. Traditional engineering won't do it. But if you adopt complexity, maybe, just maybe, you'll make that breakthrough. And that's powerful. But at the same time, as a leader...
Complexity adoption? Excuse me? That sounds dangerous. That sounds risky. It's part complexity and part just a mental model shift that may not be on a depth of complexity as much as it's a completely different lens. That's just as cognitively difficult.
but not necessarily on the complexity spectrum. Like just, again, like looking at examples versus instructions, it may not be more complicated to provide examples. In fact, it might even be easier, but it's a different lens and we're just not cognitively there, especially on the computer side. Again, let's not lose the nuances. I'm sorry to call you out on this, but we really want the viewers to be with me on this one.
we're not talking about the complexity of building the system, right? It might be as trivial as finding a big old database of cat photographs and running some algorithm on them and, you know, hey, Presto, it's all finished. I'm talking about the complexity of the solution, the complexity of the transformations of information that take you from where you started to where you wanted to go. Because that's what's happening every time, right? Some input comes in, we want something to happen as a result of it. And what takes us from the input to the output, if that is a simple set of things, a really simple set of instructions,
Please, I implore everybody, just write those instructions. Robust, simple, debuggable, everything you want. Beautiful. If you have some doubt that your instructions are right because you worry that you may not actually understand your domain well enough, you could go to data. Why would you doubt? Maybe you live in an epistemological void like I do, but mostly you doubt because you have some sense that there's something that you're missing because there's complexity in play. Or maybe you just can't even begin writing instructions. Correct me if I'm wrong. I think what you're saying is the lens is like...
Input, output. Complexity, simplicity of input. Complexity, simplicity of output. Nope, nope, not at all. The input and the output, whatever they may be, they may be simple, they may be sophisticated. That part doesn't matter so much. What matters is what has to happen to them in the digital pipes. What I'm talking about is...
imagining a big black box, because that's what we imagine. And some input comes in, whether it's text or a picture or, you know, sensors, temperature reading, whatever it is. And then something comes out on the other side and that's something, it could be all kinds of things. It could be an action. It could be a whole workflow kicked off. It could be a reading, whatever it is. Now, if the steps that take us from the input to the output are simple, they're like simple instructions that an engineer should just write. And if we can know them, we should just write them. That's my point. The reason that black box gets filled with...
AI and machine learning models is that the recipe is complicated and complex because the task itself has a solution that is complex. So anybody who is upset that, for example, you can't read what the model is doing, not really, you can read what the model is self-reporting in, let's say, reasoning models, reasoning LLMs, but you can't really see what the LLM is doing inside. The reason you can't do that partly is that... You're in this whole game because of complexity. Complexity that overwhelms the human mind. So when you go and open that up and you see a whole, you know, maybe it's not secrets. Maybe you just see the math. A whole bunch of billions of parameters, billions of coefficients. That makes no sense to you. It's like saying, I'm looking at a book and I want to tweet.
Okay, but the book's too long for a tweet. There's a lot here. And it overwhelms you. And so that's the game that we're playing. We are using very sophisticated, in terms of their complexity, in terms of their moving parts, solutions wrapped in a black box to get from our inputs to our outputs. And so what businesses and business leaders need to understand is that, again, that is...
a wonderful quantum leap in progress for situations where you just couldn't go from the input to the output. You didn't come up with that recipe. You were stuck. And there was no way you were bridging that gap. And now suddenly you have a way. But when you open that thing up, complicated stuff inside because your complicated problem needed a complicated solution. And so when you talk about AI adoption, don't talk about AI. Instead, say what we're bringing into our business is complexity.
Why do we need it? Because what we want, our input to output movement, does not happen any other way. We require complexity. But when we bring complexity, we bring rights. You can say you can look at most companies have automated most of the simple things, right? That's exactly right. And what's left is the complex things. Yeah, that's exactly right. And that's why you want to be excited by AI. But that's also why you want to be so frustrated every time that you see that people have an AI adoption mandate. Because that's not the right thing. The right thing is to say, hey, surprise. With all of these...
recent technological innovations, particularly the automation of language, which is what the rise of the LLMs has been about, you now have a shot at automating lots of things that you thought were off the table. What does that mean for you as a business? It doesn't mean adopt AI. It means, remember that attic full of wishes that somebody made where our business would be better if only we could whatever it is. you go back to that attic, you open those doors and you reconsider your business now because maybe, just maybe, you can get that thing to happen. So when you look at all these POCs coming grassroots, POCs going nowhere, having no tie-in to what the business actually needs or wants, of course they're failing. The whole game here is to stop talking about AI and to start saying, this is how I would like my business to be. I'm going to make a clear vision of what that actually means.
And some of the things that are technologically required here that were previously thought impossible might now be possible. So that's what we have to examine. That means the leader has to kick the process off. It has to start with the leader. The leader has to say, this is what our business needs to do and be in order for good things to happen. This is what success looks like. Now, if it turns out that you can actually get that through some way that doesn't involve AI, like, hey, you just need to change some legal contracts or actually your engineers can code it up for you. Well, then more power to you. Good. Do it that way.
Because you don't want to just adopt complexity just because you can. You adopt complexity because it's worth it and you sign up for the risks. So when you talk about saving time, you know, maybe as content creators, maybe there was just no way within reasonable budget without hiring a small army that you could do things like process and rewrite news articles on the internet automatically and then bring them into your own inbox or into, let's say, a speech-to-text system that can pipe right into your headphones when you wake up. That's how I get my news. You could not do that before.
Now you would say, hey, is that still impossible? And there will be complexity there. There will be mistakes. Things won't quite work as intended. But here you have a use case that is safe because you're there watching it and you're responsible for it. And you're also educated enough to understand that because it doesn't always work perfectly, you're going to write disclaimers or you're going to check it before you ship it or whatever else. But you're not saying I want AI because AI. You're saying my business would be better if I could do this thing. Now, can I do this thing? Yeah, we were talking earlier today with...
uh andy sack and adam brotman who wrote a book called ai first and and they were talking about how important it is to have leaders have that aha moment sort of where they can sort of envision the possibilities with ai and in their experience it was often like just personal use you know they had it summarize something very complex um but yeah i like what you're pointing to because there's the aha moment and then there's the then there's the follow-up uh sort of realization that's more important which is what you're talking about which is recognizing that just because there's all this cool stuff you can do that's not a reason to press the go button. You need to also understand that you're bringing in all this complexity. That's such a good point. Just start there. Adopt complexity. Why are we stuck? Because we've been trying to use simple solutions for complex problems. The simple solutions don't work. Cool. We need to adopt complexity. But don't just adopt complexity because complexity also brings risk. What I'll say about AI firstness. So the OG of AI first is, of course, Google in 2016. And I was there at the time at Google. And I was in the research and machine intelligence group.
And what I was so proud of, and still admire Google so much for, is just how big of a density of AI researchers and data professionals they had. Because here they are, organizing the world's information, making it universally useful and accessible, organize the world's information. So of course, Google is hiring all these people, myself included. I was feeling very lucky to be in such a wonderfully well-versed group of AI researchers. And then the company announces AI First 2016. And most of the company, outside of research,
does not know what AI is, right? To them, it's still Asimov, robots, all these kinds of things. So how can you reasonably be an AI-first company if, sure, you've got the researchers, but AI-first company means the company. It means the barista should know what AI is as well. Everybody should know. Everybody should understand how they can contribute. And I believe that everybody can contribute because what are we talking about? We're talking about automation with data. We're talking about automation. We're talking about what can we make better.
by adopting complexity, by using examples? And how does all that stack together? And so my story within Google was I was dead set on training everybody else, not the researchers, when we went to AI first. So I created this course, and some people were joking with me over beers and said, can you make a course that CEOs would love, that our external customers would love, that would train Google internally, that AI researchers would enjoy, right, CEOs and current AI researchers, engineers would enjoy, sales and marketing would enjoy, and high schoolers with no math background.
And I went, hold my beer. I will make you this course. And not only that, I will make it so that everyone will want to come and enjoy it and we won't have to make it mandatory. In fact, it will all be their idea to show up. So then we ended up training 20,000 Googlers. I trained 20,000 in person through this program of how to set this common culture and language. Who does what? What parts do you need to know? How should we all collaborate?
And it's out of that that you can actually go AI first. Once you've got the language, once you've got the culture, once you've got the expectations, once even the barista knows and can point out, hey, are you guys collecting data on what we do with our coffee orders? Yes, you are. Okay, let's talk about this. Let's talk about what we could do with that in theory. And so I want to see that form of AI first when we're talking about automating with data, right, with our data. There's another kind of AI first, which is the individual's AI first. And the individual's AI first has to do with taking advice. This is my...
guide for individuals who want to be AI first. So what every professional who's further along in their career understands, hopefully, and if you're far in your career and you're listening and you haven't gotten this one, then forget everything else I've said. This is my big career gift to you. So when you are faced with something important, expensive, valuable, The thing you have to do is ask for advice. Don't suffer in silence. Ask for advice, right? So if you're far in your career and you haven't gotten this one, please. This is the breakthrough. This is how you do well. This is how you tap into humanity's store of knowledge. You ask for advice. And you should be asking for advice from humans. You're used to doing that. You call a mentor. You call a friend. You ask a colleague. You ask somebody. You don't suffer in silence. And you get more done this way. And that's fantastic. So you're asking a human. Keep asking a human. But here's the fantastic economics of advice.
advice has just become cheaper than ever. Personalized advice has become cheaper than ever. That's the language revolution with LLMs. So to be an AI-first individual, you simply have to recognize that asking advice is a good thing, and the economics now work out so that before you would only go and bother somebody to help you when something was that important. Now, for much less important things, because it's so cheap to do it, you could ask for advice. Slightly stuck? Ask for advice. Think you might not be on the right track? Ask for advice.
Curious. Ask for advice. Very, very cheap. And then about all this poor private information that we're putting into the LLMs. And hey, you all know how to not violate your NDAs and still ask for advice. Right? You just don't put details in. Don't put people's names. You might say, help me out with a draft of how I might write the performance review of a middling performer whose feelings I don't want to hurt. Right? And then you get a draft. You get some advice. And you don't have to say that that person's name is Sam. Right? You're just still private. Pretty useful.
And then you, of course, there are nuances. You use it skillfully. But still, around getting advice, the three skills you need. Knowing what you should ask advice about. That's priority. That's knowing what's important. And that's a leadership thing.
right? The greener you are, the more scattered you are, the more everything pulls your attention. And as you become a leader, you begin to see what's more important and what's less important. And you have a nose for these things. If this were better, my business would be better. This is where I should focus my effort, energy, and attention. So knowing what to ask about, that's skill number one. Skill number two is knowing how to ask. If you're faced with the most intelligent, wisest advisor in the world, there they are sitting right in front of you. And you ask your question poorly. You give the wrong context. You don't give enough context. You're ambiguous.
Why would you expect that what comes out of them is any good? They could fluently start saying a whole lot of things. Will it help you? Might be useless, might hurt you. So you have to know how to give them what they need to help you. Give the right amount of context, give the right amount of information, how to phrase it, how to remove the ambiguity. And then number three is don't assume that it is the smartest, best and wisest advisor that you're talking to.
it might be completely full of nonsense. So skill number three is how not to take bad advice. So the questions you begin to ask yourself are, how important is it? Because if it's not important, we're just having some beers and just talking smack. No need to fact check it, right? Don't worry about it. Not important. If it is important...
how would I check whether this is good advice? Should I act on it? How do I get a second opinion? How do I evaluate it? How do I think about what I'm looking at? Don't just act on the advice. Think about it. Don't take bad advice. So those three skills, and those three skills, they work at the individual level. They do scale up to the leadership level too, but knowing what's important, knowing how to ask and not taking bad advice, those are our big LLM-based skills and those are our AI-first individual skills. That this doesn't, it doesn't translate up to the organization because
When you start getting the hang of, oh, I can get all this personalized advice and I can use these tools for all kinds of cool things. And hey, I want to learn a new tool that I've never tried before. And I want to learn it in a me-friendly way. And now I've got this really patient tutor advisor that I can even air my frustrations onto and will just go gently until I learn it. How cool is that? You get so excited, so fun. You get so excited, you're enjoying yourself so much. And you feel empowered. And you might forget to realize that at the organization, any decision that's made.
it has much more gravity because lots of other things are going to start to barnacle around it. So it might have been a no-brainer in some way. Maybe it was the best technological solution at the time you pop it into a large organization. But you forget to consider that it might be really hard to take that out and replace it with another solution later when it's no longer the best solution for you. Right? So all that...
That ease of motion, all that low consequences to asking for advice, all that economics where it's really cheap, it doesn't translate over to scale. At scale, decisions have gravity. At scale, things demand to be measured. At scale, there are a lot of incentives and desires pulling in different directions, and so things work differently at scale. And so to lead systems at scale then, and to lead probabilistic systems at scale, is also a tricky thing. Sorry, I'm going to keep ranting because I need to get some new things out. No, no, get on that. Hear me, hear me, leaders. A lot of your stuff.
Think of tools as deterministic. So let's say that you have a mistake in a spreadsheet. Something doesn't compute. Your first instinct is to look for problems where? In instructions. In the inputs, in the stuff that humans put in, right? Yep. You're unlikely to go and question whether there's a hiccup in the summation function itself, right, inside the spreadsheet logic. You assume that that works that same way every time. But with LLMs, it's like there's a little, you know, it's Minesweeper with little landmines of randomness.
Not often, but every now and then, there's a random number generator participating in your summation function, if you will. And so now, if you trust your tools to be deterministic and to work perfectly and consistently reliably the same way every time, you're going to make a big mess. And at the organizational level, another thing happens is things start to be combined together. And as you combine things, and as you combine probabilistic things, whether they involve humans, machines, agentic AI, statistical programming, whatever it is, the uncertainty bounds on one step might be here. And then on the next step, wider and wider and wider. And there is a lot more chaos, if you will, in what could happen downstream. So this feeling of contained LLMs that are fun to use on a personal level, it just doesn't translate to what happens when you put in probabilistic systems at scale. Right.
The statisticians are back, folks. I keep saying I'm a recovering statistician, but I guess you never recover. Figuring all that probabilistic automation out is easier said than done. Again, when we go to scale and leaders are going to need training in that as well. So to wind it all back to what I'm about, which is decision intelligence. Decision intelligence is the leadership discipline that looks at turning information to better action in any scale, any setting. Let me say that again. Turning information to better action, any scale, any setting. We can use information, data to automate all kinds of things, potentially at enormous scale.
And we, the leaders, are responsible for setting the intent, but also making sure that what we're creating is safe, effective, and not having all kinds of surprising consequences. One of the skills that is demanded of leaders then is clarity of thought and clarity of vision. Think of this as a proliferation of magic lamps with genies in them. AI systems are like the genies. The world is racing to build better and better and better automations based on data. Genies that, because they're based on the automation of language, and language is the interface for human collaboration, you can ask the genie in your mother tongue to do something for you. And in agentic AI,
that genie is connected to the world and can actually act upon the world. I always say the difference between a chatbot and an agent is that a chatbot only hurts you if you let it because it presents information and you can just say, I don't believe you. I'm not going to do anything with this. But an agent takes it further and acts. And when you then look at the genie, you remember the genie stories. It's not the genie who's dangerous.
you should actually be concerned with the wisher. That's the leader. Do they know how to specify their wish? Do they know how to not be ambiguous? Do they know how to simulate and foresee the consequences of what they're asking for? Do they understand how this fits in with not just their technological organization, but their human organization? Have they prepared their humans? Have they understood all the moving parts? Have they delegated decision-making throughout their organization so that they're not trying to do everything themselves? And have they delegated it well? And have they thought about whom they trust? Because trust is the grand challenge of our agentic AI world. So, wisher is so important, but then with agentic AI also, the lamp. We never talk about the lamp.
But the lamp, powerful lamp that never lets the genie out under any circumstances, well, it doesn't matter how stupid the wisher is. So the lamp's pretty important too. And the lamp is our access controls that we give to Identic AI. It's our security. It's our guardrails. It's all the stuff that keeps those AI systems under control. And so decision intelligence then is about the wisher and the components of their system, preparing wishers to be responsible. I want to talk about the lamp a little bit. I want to, one aspect of the lamp and all of this.
All of this makes so much sense. Where do you think that synthetic data or let's say simulation fits into this? So stimulated outcomes. So now we're not talking about like one outcome or one example, but we're talking about stimulated outcomes. Oh, I love this one. I love this one because I am, as I said, a recovering statistician. Let me tell you, statistician folks are for the most part not impressed. We are not impressed by synthetic data.
It is just simulation and we're quite used to it. And so for those who are not sure what synthetic data is, allow me a short definition. It is fake data. So that's what it is. That's the long and short of it. So why might fake data be useful to you? For the same reason that a sketch might be useful to you before you begin fleshing it out. For the same reason that...
In, let's say, physics, you might imagine a spherical cow for that kind of reason that you are willing to work with not quite the right thing that doesn't quite represent the world. It still gets us some of the way. Gets us some of the way to where we're going. One of the great uses of simulated data, which is fake and so probably different from the real world. You should always suspect it if it is fake. One of the great uses is if you're going to train a model.
When you train neural networks, you begin with some weights already throughout your network. And I have an episode that's short and sweet. It uses blenders on YouTube. If you guys could link it in the podcast to explain neural networks with blenders. For those who are not familiar. But for those who are familiar, I'm just going to say it quite quickly. To train that network, you have to start with some weight settings already. And then you pass data through and you update the weights. And the question is, where do the initial weights come from? If they come completely at random, then you know you've started in a bad starting point. And it's probably going to take you quite a lot of data to improve over that.
If instead you start not at random, but having trained on not perfect data, okay, not really from the real world, but possible. You might train a cat-not-cat classifier based on, I don't know, paintings I painted of cats. Not real photographs, but paintings of cats. That's not perfect. We're still going to need some actual cats to get it all the way to trained. But we might skip some of those initial steps, and we might require less data from the real world to get our model all the way there. But we also are playing a dangerous game when we use fake data. Because the more we rely on it, the more we are essentially relying on the assumption that our fake data is exactly like our real-world data. And what I've always told Google engineers to pretty much tattoo on their foreheads, if they want to go anywhere near machine learning and AI, is this sentence.
In mirror scripts, every morning you wake up, you see in the mirror, reverse script. The world represented by your data is the only world you can expect to succeed in. So if you have failed to represent your data by using all the synthetic data, and by the way, I also have on my blog a series of guides about synthetic data. I have lots to say. There's five of them. So if you just search for CASI AI synthetic data, they will show up, I'm sure. And maybe you guys can put a link in the bottom there. But if you hope too much or someone sold you this idea that synthetic data allows you not to need real data.
They've told you fake data allows you not to need real data. For some definitions of need and some applications, that might be true. But you're still playing a dangerous game to rely on data that's not like the data where you need to operate. So you have no guarantees. So at the very least, if something's been trained on synthetic data, at the very least, test it on real data. If you train it on synthetic data, and if you test it on synthetic data, it only works in the synthetic world. You bring it to the real world, expect disaster. But it can still get us some of the way.
some of the ways so we don't have to do all the work with real data. It can save us some effort. The other thing about data classes to realize in this AI revolution is you've got the kind of data that only you have as a business, as a person, only you know it. That's your proprietary secret data. And that data was the target usually for data science endeavors.
And that was where most of the machine learning efforts, say, last decade were happening. A perfect example would be Netflix has data that nobody else has about how people watch the shows available on Netflix, right? I don't know that. You don't know that. Unless they give us that data set, that's theirs. They know. And then they can use that to automate recommendations that they can show it to folks. Meanwhile, there's a different kind of data set, which is what every garden variety human can do.
The garden variety human can pick a cat out of a photograph. The garden variety human, what else do we do, can ramble on and on the way that I'm rambling on and on. The garden variety human can write an email. Those types of things. And for those everybody type tasks, that's where the AI renaissance is really working right now. Let's get a bunch of data from the internet.
garden variety humans doing garden variety things. Let's augment it with some, you know, workers in Kenya or somewhere who are going to do some of those tasks on demand, right? I'm basically expressing to you how OpenAI got itself to GPT. And then, then we might be able to help out with every human type of task. Now, don't expect a system like that to suddenly know things about you where it has never experienced that data.
Those are two different endeavors. And we also see a lot of career confusion for data scientists as it seems to them sometimes that their work in managing all this proprietary data is no longer in vogue because everybody's talking about going after the garden variety human attributes and that this is AI, it's language, it's vision, it's the creation of music. We still will have proprietary data. Knowledge is still power. It's still important. And you will not be able to use synthetic data or somebody else's data to reveal your secrets if they're your secrets.
So there we go. That's my take on data and synthetic data. There's something in, I don't know why this is popping in my head, but it's the idea that we want simple, as you pointed out, and we're heading to a world too complex. And one way to make something simple isn't to simplify it, but just to ignore the data. Yeah. To pretend it's not there. Yep. That is what leaders are doing and it breaks my heart. Yeah. So that's what? Protecting cognitive load, protecting energy.
Or it is maybe one more in a series of broken promises. We are seeing a lot of broken promises that were told to people early in their careers about striving. Hey, if you just go to med school and you work really hard and you memorize all these texts and you do all these years and years of work, there will absolutely be a job for you for life, I promise you. And that job's not going to change. You're going to be good. You're going to be golden. You're going to have a little update of newest research once every two weeks. And mostly you're going to practice in peace.
You know, you can become a very senior leader by really skillfully managing organizations of humans. And it's fine that you're non-technical. You don't need to know anything about technology. It's fine. Right. That's a promise. Become really skillful at this and you get to coast and chill out. And we are seeing these promises broken over and over and over. We're seeing the biggest breaking of these promises in the memory heavy profession. Any memory that any profession that relied on memory is going to have an upset because we have a memory prosthesis now. We also see this upset in. A memory stressful. Remembering stressful.
Yeah, good riddance, right? Why did I have so much memorization in elementary school? I'm so happy that the next generation hopefully will not. And we'll move on to bigger things like vision and creativity and problem solving. Right, right. And personalized education, right? Why must everybody memorize the same facts from the same chemistry textbook? Maybe one student doesn't take chemistry, they take the history of butterflies. I don't know, whatever it is. We can have so much diversity in what we can all be interested in and study. And I'm also excited about this because if people play to their strengths, if young people pursue their passions.
This is smarter, I think, than many parents realize. Because the worst possible thing you could do in college right now is... study something that makes you hate learning because we're on a treadmill. Technology is going to be changing. The world's going to be changing. And those of us who love learning and who love adapting and who love going like, whoa, there's this new tool. How do I use it? How do I do something with it? Those are the people that are going to win. Whereas somebody who says, oh, I hated these four years in college. I can't wait to be done. I'm never touching a book ever again. I'm never learning anything again. It's going to be rough for them. They will be forced to keep learning things. So the skill of learning, which you get by allowing your, yeah, exactly. By allowing your passion to shine.
So it's going to get you through. But back to the broken promises. There are broken promises happening very much at the senior levels around what can be expected. What do you need to interact with? What do you need to learn? What types of people do you need to collaborate? How is the future going to look for you? And anybody who said it's stable, it's the way it always was, nothing to learn, nothing to do, nothing to see here. You've got all the skills you're ever going to need. I promise you, you're set. Well, whoever said that lied.
We all have to upskill. And I'm here to try to help the leaders upskill because I am really worried about the genie and wisher paradigm where I see wishers being put in command without even realizing it of enormous decisions. They think setting a KPI is a small thing, but in an agentic world, you may get exactly what you wished for. Did you know how to wish? That's going to be a problem. And that's decision intelligence. That's why I do what I do. Yeah. Yeah. Interesting. The decision of what to ask versus the decision of what to do.
It's like a new way of thinking about decisions. Before you get to the decision of what to do, you've got to slow down and talk about the decision of what to ask first before you make the second decision, which is what to do. People quote Warren Buffett as sort of saying that you don't really make money for complexity in business. Simple companies make as much as bigger companies do. And we're sort of headed towards...
Again, as we were talking, this like spectrum of complexity and leaning into complexity. And every time I hear that, I always think like NVIDIA where like complexity though does, if you can handle complexity, you increase your odds of success in a complex business. In other words, well, I don't have to, in other words, you get what I'm saying. I get what you're saying. And I have a response to this too. So here's what I would like to imagine Warren Buffett is saying.
I would like to imagine that what he's saying is that inviting complexity, just because it sounds fun, isn't a smart thing to do. And that is also what I mean when people are like, we need to adopt AI, we need AI, we need AI everywhere. That without a purpose is a problem. So as somebody who loves progress and I really want to see humanity do all that it's capable of, I want us to solve our hardest problems. I want us to make really amazing art. I want us to cure diseases. I want us to start using the environmental resources that we have in efficient ways that help the world instead of creating all these negative externalities and climate impacts, right? I want these things. And for us to achieve these things, we're going to have to be smarter and we're going to have to solve harder problems, harder problems than individuals can hold in their heads.
I'm excited about automation with data. I'm excited about what this memory prosthesis means because I see all these unsolved problems and I want them solved. And I know that we won't be able to solve them with short things that fit in a tweet. We're going to have to do a little more than that. And this is our opportunity to step up and solve bigger problems. But it is, that's what I want it used for. I want it used for all the things that you wish you could achieve, but you couldn't do any other way. But like Warren Buffett, when I see you then apply this power tool to a situation that would have required you to press with one finger.
I don't know what you think you're doing. Why would you adopt complexity just to adopt complexity? So when there is the simpler way, I want to see you doing that. But I'm sure that Warren Buffett would go back on his quote if someone presented him with a problem. that requires a complex solution and requires a complex business. And they would ask him, well, should this business then do this complex thing? Or should they just try to be simple? I'm sure that he would say it's an important mission. The world needs it. Solve it however, whatever way it takes to solve it. And among those ways, the most simplest one, please. Right? He's not going to say, let's say that the organization of Google that has to process a lot of data and has to automate based on that data. I'm intimately familiar with them needing, absolutely needing AI for their operations. I wouldn't think that Warren Buffett would turn to Google and be like, you guys are too complicated. Right on.
This was great. We're going to do this again, hopefully, if you're up for it. Love it. Cassie, we heard you have an agentic leadership course that's available out there. Can you tell us more about it? Absolutely. When it comes to agentic AI.
The person who most needs to be trained is not the engine here. Definitely not the sales and marketing person, though. Maybe we should try to reduce some of that hubris and enthusiasm. But who needs to be trained is the leader. Because I would say the difference between a chatbot and an agent, an agentic AI system, is that a chatbot can only hurt you if you let it. So if you refuse to be gullible, anything that the chatbot puts on screen, you don't have to do anything with it, right? You don't have to let it hurt you.
an agentic system that's connected to the real world could take bad output and take bad actions that could actually have quite large consequences and all of this starts out with the leader intending having a vision hopefully a clear vision of how we're going to automate something complex at scale with logic that is probabilistic and based on language models that then plans its own steps and touches the real world. That is a lot of responsibility. And there are a lot of non-technical and technical things that you need to know. So the course contains everything from AI basics to agentic security, to systems thinking, to thinking about how to find and design good use cases and how to put together teams. So it is the leader's crash course in how to wield that power responsibly.
I know that a lot of companies are flirting with Agentic AI. They're getting into it. And I really, really urge them to consider making sure that the leaders have been trained and prepared. This isn't one where you just leave it for the engineers. It is an act of great responsibility, especially with great scale comes great responsibility. And so my course, and it's my course with Brian Evergreen and Pascal Bournet, the authors of the book, Agentic AI. So I hope that the three of us can help leaders find their way in the dark there and take that responsibility knowingly.
Yeah, that's awesome. I wanted to mention, I'll share with you that surprised us when we released the book. You know, when we released our book, we really tried to make it more practical, distant power. You know, that was the idea. And then we've had a number of situations, but one in particular, I'm thinking of where somebody would read the book and then they quit their job. And it came down to like, once they sort of had more of a granular view of what it was going to take to kind of succeed in this world, they looked at their leadership team. They looked around them and said, like, this company is not going to make it. I'm going to go somewhere else. And it dawned on me that leadership.
You know, if you're not in a power position or a position of control, then you just feel helpless if people don't get in. So you may want to just like to understand this technology in a granular basis doesn't mean you need to do it to your point earlier to provide instruction to the rest of your team, but just to instill confidence that this matters to you so that they don't quit. Yeah, absolutely. It's so much more than that, though, because. When you have machine systems that act on your intent, combining with human systems that act on your intent, and that this works differently for the human and the machine systems, but it's all essentially serving the leader's intent. And the leader can, of course, delegate that. They can delegate that down if they don't want to deal with it. But if they haven't delegated it and it all bubbles up to them and they have blinders on that only let them see the human part of this very complex system and not all the rest of it, which is still their responsibility, then yes, in terms of their leadership effectiveness, I would say that they are missing out. They're going to be poorly skilled for the coming wave.
But from the point of view of everybody else in the organization, you owe your humans more than that. Be better. You owe your humans not the nitty gritty. Nobody wants you going in there and learning C++ or whatever it is. What we want is for you to actually understand the resources at your disposal and more or less how they fit together and what your role in them is. And the simplest way to put this is any leader who thinks they have no role in agentic AI.
is going to be a problem. So come to the course and understand what your role is. There's a role here. If it's a key ingredient and becoming more and more a key ingredient, then the absence of it is a sense. Essentially, you're not a leader, right? Because that's a key ingredient. If you're still doubting this, you've gotten used to technology, right? Leaders know what email is. Leaders know what to expect of a workforce that has email, that has the internet. Now, imagine you take somebody from a different time, a brilliant leader, someone smart. Pick who you like from history.
who's not a modern person. There will be things that they would need to catch up on, right? They might be pretty good at the human side, but there would be a vacuum when it comes to some of that technological side. And we would want to upskill them. We wouldn't want them to take a programming course, but we would want to upskill them. There would be things we would want to tell them about how the world works now. And we would want to tell them a little bit about how the world is about to work as well. Otherwise, they will have those blinders on and things will take them by surprise. And they won't even know that they were making far-reaching decisions. They won't even know that they are the problem. So that's why we designed this course, not to be a...
for engineers course, but to get leaders oriented in what their part in the whole process is. Awesome. Good luck on the class. I think it's going to be very successful. It's a good team. Thank you so much. And it's good perspective. Awesome. We've already had two strong cohorts, so we're doing well. We've got two more this summer, so I hope people will. All right. Well, thanks for hanging out with us again here on Invisible Machines. Don't forget to like and subscribe wherever you get your podcasts. Always remember that you can watch this podcast, too, on our Invisible Machines YouTube channel. Head over to uxmag.com. We have a podcast page there where you can explore our back catalog and you can look at episodes centered around popular topics. Many, many thanks to our incredible production team here in the U.S. and in Ukraine.
Elias Parker, Michael Lipinov, Allison Harshberger, Ola Slessarenko, Vera Pakulkov, and Darina Moskovchuk. Till next week. Owaki.