Skip to content
S4E9 Digital Twins in an Agentic World episode artwork

Sep 20, 2024 1 hr 8 min

S4E9 Digital Twins in an Agentic World

from Invisible Machines podcast by UX Magazine Hosted by Invisible Machines

Digital twins are critical to the orchestration of AI agents, providing the context they need to create meaningful experiences quickly and efficiently. Robb and Josh welcome Dr. Michael Grieves for a conversation about the origins of the concept, which he developed while working with NASA in the 2010s. The architecture required for orchestrating AI agents relies on different types of digital twins that may emerge within an organization, touching on physical elements, temporal data, and collections of unstructured data. Dr. Grieves joins the podcast to explore these connections, drawing from his book Product Lifecycle Management as well as his numerous scientific publications. The trio also discusses how something Michael calls “retirementitus” prevents organizations from embracing the sweeping technologies surrounding AI and digital twins. In this episode, we mention: Dr. Michael Grieves’ book “Product Lifecycle Management: Driving the Next Generation of Lean Thinking”: https://www.amazon.com/Product-Lifecycle-Management-Generation-Thinking/dp/0071452303 Dr. Michael Grieves’ book “Digital Twins, Simulation, and the Metaverse: Driving Efficiency and Effectiveness in the Physical World through Simulation in the Virtual Worlds (Simulation Foundations, Methods and Applications): https://www.amazon.com/Digital-Twins-Simulation-Metaverse-Effectiveness/dp/3031691067 The Invisible Machines episode “Building an Advanced Digital Assistant” with Jeff McMillan (Head of AI) and David Wu (Head of Knowledge Management & Generative AI) at Morgan Stanley: https://www.youtube.com/watch?v=3iQbufK1luE&t=1567s Charles Perrow’s book “Normal Accidents: Living with High Risk Technologies: https://www.perlego.com/book/735258/normal-accidents-living-with-high-risk-technologies-updated-edition-pdf The Invisible Machines episode with Dr. Nathan Price, Chief Scientific Officer of Thorn HealthTech, and Dr. Lee Hood, a world-renowned scientist, a recipient of the National Medal of Science, and a key contributor to the human genome project: https://www.youtube.com/watch?v=rrrEu5TNqFg The Invisible Machines episode “Lean Knowledge Management” with Roger Forsgren, former Chief Knowledge Officer at NASA: https://www.youtube.com/watch?v=e4A_JiBRCr8 You can learn more about orchestrating AI agents for your team: https://onereach.ai/ai-agents/?utm_source=soundcloud&utm_medium=social&utm_campaign=digital_twins_episode&utm_content=1 #AIPodcast #TechPodcast #InvisibleMachines #AI #ArtificialIntelligence #ConversationalAI #AIAgents #TechInsights #DigitalTwins #AIOrchestration #ProductManagement #TechInnovation #DataArchitecture #MichaelGrieves #FutureOfAI #TechTalk

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

S4E9 Digital Twins in an Agentic World

0:00 1:08:53
of matches

Transcript Auto-generated, may contain errors

Welcome to Invisible Machines, a podcast where we explore the expanding world of AI agents and conversational technologies. I'm Josh Tyson, co-author of Age of Invisible Machines, a bestselling book all about the agentic approach to automation and the pursuit of organizational AGI. I'm joined every week by Rob Wilson, CEO and co-founder of OneReach.ai. Rob and I wrote Age of Invisible Machines over the course of many, many conversations, but we're a lot like the ones that we have on this podcast. Rob has spent decades researching and deploying conversational technologies, and Invisible Machines is where we share big ideas with fellow AI insiders. We are continuing these conversations in partnership with UX Magazine because proper attention to experience design is totally critical to the proper deployment of AI agents, or as we sometimes call them, invisible machines. So digital twins are pivotal to the orchestration of AI agents.

They provide the necessary context that machines need to efficiently create meaningful and rewarding experiences for users. Digital twins can take all kinds of forms, from a virtual rendering of the physical layout of a factory floor to a map of the processes that machines and humans engage in within that space. Digital twins can use temporal data to run countless detailed simulations of events. They can also make use of unstructured data, like emails and documents and recorded calls, to create virtual representations of an organization's inner workings. On this episode, we discuss all of this and more with Dr. Michael Greaves.

executive director of the Digital Twin Institute, and someone who helped originate the digital twin concept while working with NASA back in 2010. On this episode, Michael sets the record straight on the fallacy that digital twins began with the Apollo program. He also declares the fading relevance of structured data and emphasizes the importance of starting your digital twin journey small with reasonable use cases attached to real value propositions. We also explore the reasons why information is really just a means to reduce physical exertion and how an overemphasis on intelligence and fidelity where AI and digital twins are concerned might slow companies down. Interestingly enough, Rob and Michael have both led projects with the Office of the Director of National Intelligence, and in this conversation, they compare notes on piloting a Boeing 787 simulator, something they've both done, and together we discuss the general lack of high-value simulations and simulators in the broader business world, and why something Michael calls retirement-itis keeps organizations from embracing necessary change. With that, let's jump into this exciting and jam-packed conversation with Dr. Michael Greaves.

Invisible Machines is produced for UX Magazine in partnership with OneReach.ai. Named a leader by every major analyst group, OneReach.ai's Generative Studio X platform was built specifically to facilitate the no-code orchestration of AI agents. GSX is trusted by global giants like PepsiCo, PwC, Deloitte, and DHL, but anyone can experience the level of agent orchestration we talk about on this podcast through interactive demos at OneReach.ai. Now, on with the show. Michael, we are super excited to have you here on the podcast. Thank you so much for joining us today. Thank you for inviting me. Yeah, Rob, a pleasure is always seeing you.

As always. Yes. So Rob and I are pretty fascinated by the digital twin concept, the scope of it, the potential that's wrapped up in there. And I think our listeners are kind of on the same page there as well. So we thought it might be interesting to kind of kick us off if you could describe how the concepts kind of first came to you and then also kind of how it's evolved, particularly with the arrival, I think, of generative tools and the way that we've seen LLMs capable of summarizing unstructured data and kind of creating almost a new facet of digital twin hood, you might say. So I got into the computer business.

early on in the 60s when I was a teenager. And that was pretty unusual because at that point in time, all they had was mainframes. But I sort of lucked into it with the National Science Foundation, a program out of Oakland University. And in my early 20s, I'm actually running a large-scale mainframe operation. And it was approached to basically be able to, in fact, if you've ever had to put some wires into your house, you have to get the wires and someone come out and find all of them. I thought, well, that'd be great if we could basically create what was not called a digital twin, but a representation.

and computers that would basically look at every square yard and indicate whether it was alert or not. And this was in the Detroit area, so it was Oakland McColl and Wayne County. And when I did a back-of-the-envelope calculation, it wasn't anywhere near possible. We had megabytes of hard drive. In fact, the mainframe computer that I was dealing with had 192K of internal main reverend. So there wasn't a whole lot of capability. I wound up, in fact, being in charge of the first supercomputer, the Eliac 4, and even that thing didn't even have enough to really know what your interest in terms of.

of creating a digital twin. I mean, it had 25 megaflops worth of capability. Today, your iPhone has about two teraflops. And so that's kind of where it started. So I kind of put it on the shelf and I started a computer company and then got tired of dealing with the accountants and attorneys and kicked myself upstairs. I got my doctorate from an executive doctorate program at Case Western University and pulled that stuff back out to really look at doing that. And I've always been interested in the idea of data and information and how that can come together. And Moore's Law has given us tremendous capabilities.

that we couldn't even conceive of early on. And so in looking at my dissertation, I talked to my advisor about it and quickly decided that was a stupid idea because of the fact that it wasn't well known and the whole idea of getting your doctorate is to do a dissertation that you can get signed off on. So I kind of, there's two paths during my doctoral program, one working on the dissertation I actually did, which was kind of interesting. It was exploring internet companies in terms of language called Business is War. But I hadn't let go of this idea of the fact that information was embedded in physical things that we could create.

in essence a representation or a mirror of it in the digital world and so um once i got my doctorate in fact i had to quasi retire which i failed miserably at um i then started working on this idea and it was part of product life cycle management at the time which was also in essence a brand new concept hence the book that you have and uh if you look at that book you'll see the first model i didn't i didn't have a name for it right um right it was basically called the underlying premise repeal but it was a digital twin model with the the three main components that i think exist to this day you know, the physical environment with physical objects, the digital virtual environment with digital objects, and then the connection between the two of them. I get data to populate the digital virtual world, and then I use that to send data and information back so that I can actually do some useful work to the physical world. So that was kind of the long version of that. And so the idea came fairly early, but I had to wait till Moore's Law kicked in to give me enough capability. And even then, when I introduced it, there wasn't that. It wasn't until about 2015 when I wrote the article for The Economist about digital twins and what their capability was.

that we're really getting enough computing capability to do what I think we need to do in terms of represent the physical world over in the digital one. Got it. And that's kind of the area, I guess, maybe to start is because I know there's different sort of versions of twins. There's like the physical twin, digital twin versus the systems, digital twin. And I guess the boundaries, like what makes something a digital twin?

you know, versus not, you know, to me, I started thinking about this as like data with relationships being like a key component, you know, data without relationships is just data. Once you have those relationships, even if they're just where they exist in space, then you begin to like, it formulates into something. But help me understand like how you guys, or how you think about like, what is a digital twin and what is a bunch of data? So again, and so I get asked about, you know, give me your definition of digital twin and I sort of left, I don't even like it. of digital twin. I think that, and I have a pretty broad view of digital twins. First of all, they're built, but they're basically driven by use case. If I can't create value with it, I don't care how much data you got. It's, yeah, why are we doing this? Okay. But it goes back to the three things that it's going to have. I mean, it's going to have a physical object. And I ought to add, before we get wrapped up into it, I started off with physical things, tangible, but I always thought that if you could visualize it, you could have. So intangible things like logistics or supply chains or even economic systems, you know, are legitimate digital twins because I can visualize them.

so it has to have the physical component it then has to have the ability of having the digital version of that okay and then i have to have the connection between the two i have to have the ability of getting data from the physical world into my digital one and then i have to be able to send that back into the physical world to be able to to do some work okay because um got it sort of a side discussion but my definition of information is very different everybody else information is a replacement for wasted physical resource i have any task and we are co-oriented task people you know as humans i can divide it into two pieces one is the most efficient use of resources and then everything above that is waste. Information doesn't replace the resources I physically need to do something, but it can replace the waste. Won't ever replace it all given we're humans, but that's sort of the value of information. So the data that I bring over into my digital twin is to allow me to create information or actions, if you will, to replace waste of resources. So if you can basically, you know, add the model and then tell me what value it's creating, I'm good with that being a digital twin.

Got it. And so let's, one of the areas we kind of try to focus on is like a digital twin for your company. And the idea as practical use, which is in the world of agentic AI, where if you want an AI agent to reason and think and do work, they need a context, right? Like a vacuuming robot needs the map of the room and the house to operate in. These things need context. And so digital twin of a company, yes, they're physical things. They're people, customers. And then there's the whole temporal aspect, which is...

What other conversation that I want to have about, like, how do you separate the, you know, the temporal data from the, like, I guess the parameters or, you know, the static data. I guess starting with just digital twin for a company. That qualified. Sure. Again, you've got all three of the components and you've got a use case. I assume that you're trying to reduce wasting resources by having this, right? I mean, you're not saying, gee, I'm just going to create it and it's going to be a nice picture and so on. Okay. So obviously why that, you know, effective, be more effective and efficient. Okay. So, so you got to use it. I mean, and then the idea that says I've got, you know, physical stuff in the physical world, I've got digital stuff in my digital environment. And generally we want to kind of do two things. We want to do replication.

Right. So I'd like to know what exactly is happening. Okay. So who's employed there? Who's not employed there? You know, what are they working on kind of thing? And then I'd like to predict, you know, if I've got a project that's supposed to be done in a month, it'd be good if I could predict the fact that given this employee and their track record, they could actually do this project in a month. If it's an employee that always says, hey, I can get this done in a month and it takes a four months, I know I got a problem going into it. So I want to do the replication and prediction. So it meets all my requirements for a digital plan. Yeah. I took that approach and then I realized, um,

To me, it just came down to how high the fidelity was around the relationships of data. To your point, once it's useful enough to create context for human intelligence or artificial intelligence, it forms an environment around them, then it starts to become a digital twin. If it's just random data and the relationships between them don't create an environment for a human. or an AI agent to operate. And even if that's a small environment, even if that's like inside a tiny box, like you said, as long as it's useful, as long as it's needed. One of the areas I started thinking about, and this might be...

you know kind of getting too far into it too early but i was thinking about how like a digital twin versus like multiple digital twins like in the idea that a self-driving car or a vacuuming robot in essence has like two kind of digital twins it operates from one super high fidelity which is its cameras and its sensors and its immediate surroundings and it's almost ephemeral right it's it's just it's it knows like within a certain distance and range around itself at a high fidelity level and then you have like the gps or the room map that's low fidelity and you have these Two twins in a sense, or the same twin, but at different fidelity levels. And then this idea of like.

where it gets powerful is like where the high fidelity one is updating the low fidelity. It's not always high fidelity is better, I guess, as though. Back to value, okay? You know, fidelity is driven by value. So what am I getting out of this high fidelity? So for example, I mean, I get asked, gee, I'd like a photorealistic version of this particular plant. So what are you going to do with it? You know, why does that have to be photorealistic? I mean, quite frankly, you could have stick figures if that's all you kind of needed to know where the people were at a certain point in time and if that's what your usage is. So fidelity is always driven.

by by what value is it going to create um right so so i think you could have different vitalities uh depending on what it is you were trying to do i mean let's take a you know an automobile okay um probably needs high fidelity if you're dealing with pedestrians around it right you don't want to say oh within a couple uh within a yard i'm good as you run over a pedestrian you know if again you're you're simply looking at where the your uh your delivery truck is you may not you know may not care within a couple yards of where the thing gets parked so again You know, I'm driven by value. Tell me what you're going to get value from, and I'll tell you what you need for fidelity, data, the image, anything else. So my perspective about data, in fact, I'm involved in a chapter right now, the data hand.

You know, data always has context. Data without context is just noise as far as I'm concerned. So, you know, where is it? In terms of the data, what data is it? How do you categorize it? When is it? Where is it? And how did you go get it? Are things all metadata that have to be associated with data, but you really don't have data. You just got stuff.

There is not such thing as bad data. We just use language badly. You know, bad data is noise. No, I mean, you can't have bad data. Because data is basically, data is the realistic view of, basically, it's a fact about reality or the realistic state of reality is, right? It's like misunderstood data, not bad data. If it isn't representing reality, it ain't data. It's noise. I think sometimes when we think about digital twins in this space, we end up talking a lot about like LLMs and the way people are using them.

to suddenly kind of summarize and categorize these piles of unstructured data that they've had around that haven't necessarily provided a lot of value in the past. We had a conversation with some folks, Jeff McMillan at at Morgan Stanley, and they were able to create kind of a digital assistant. And part of that was feeding it a whole lot of company data. But then another equally big part of that was then figuring out how to map that data and how to keep it fresh and how to make sure it was relevant. And that, you know, when people were querying the LLM, not only were they getting reliable responses, but they were getting the information that they were looking for. But, you know, as these systems communicate with each other more, as AI agents are kind of talking with other AI agents within an organization, even, it's almost like this, yeah, this other type of digital twin starts to emerge where there's this very large picture or landscape that can be accessed that kind of shows.

you know, customer history, employee history, it has like this amazing topography, but then you do need to extract value. You need to be able to serve up almost low res pieces of that to individual agents. Like if they're going to automate the task, they don't need to know the entire history of that, of interactions with that customer. They just need to know about three or four different things that they can then kind of create a focused response with. Yeah, it's an interesting vector, this idea that noise, I like it's there's data and then there's noise, but there's how many organizations are storing, paying for, preserving noise?

you know, thinking it's valuable, thinking it has value. I think it's pretty common. So it may be data, it may be a fact about reality, but it's not useful. I mean, you've got sort of three categories. You've got a fact about reality that I can't do anything with. You've got basically facts, and they're really not about reality. They're basically noise. And then the third thing is you have facts about reality that I can use. And so the...

The second one, facts about reality that I don't do anything with are fairly common. Generally, we can figure out noise pretty quickly. I mean, that's our bad data. And once we try to make a decision and find out where we basically screwed up horribly because this wasn't a fact about reality, we generally get rid of that pretty quickly. But I would contend that the second version is what I call lots of data but no information. That's a lot of data, no information. And that category is fairly large. I'd have to say, though, I tend to...

err on the side if it doesn't cost me a lot to collect that if I can kind of look into the future and say you know I could use that somewhere along the line because not having data having to recreate it is magnitudes of cost more than if I just capture it at the beginning so if I'm going to err a bit on that I'm going to collect it and keep testing hey can I use this for something can I grade value with it I was thinking about the temporal side because I think you said you know data that's not temporal is not useful a twin that's not based on time or doesn't consider time um is less useful and and i was also thinking about how that's it's not an on off switch it's not like um time or no time but there's like real time where it's kept up to date in real time and then there's sort of a you know near real time where it's it's asynchronous it's kept up to date but it's not real time and then there's static no time element to it um and uh i just wanted to hear like more with

what you meant about the value of the temporal side. And I think, you know, from my work, I can imagine it's a lot about running prediction. You need that sort of history, but I want to hear from you, your thoughts around the temporal side. So all of it is temporal, okay? I mean, but some of it, you know, doesn't change. So for example, if I'm interested in having a building that I'm going to operate in, once I have that building, unless there's any revisions to it, you know, that thing doesn't change, right? But it's still useful. I mean, if I'm going to, in essence, look at, you know, how people move in the building or how much I need air conditioning or electricity, that building itself is important, even though it hasn't changed, okay? The things that do change, I have concerns about.

And again, how often I need to basically refresh that is going to be dependent on what the use case is. So in one chapter or paper, you know, I basically say the cadence of that is dependent on use case. So airbags going off, I need to basically have that in, you know, instantaneous communications of that. My oil level usage in the car, you know, that could be a monthly thing and I may not be able to update that. So again, I'm back to tell me where your use case is and what you're getting value from. And I'll tell you what the cadence of time needs to be.

you know the interesting thing is that well i kind of like to point out here is that when we get into to analysis is we are time unconstrained in digital twins you know um in fact i've got a book coming out next month where i talk about digital twin simulation in the metaverse and if you remember and maybe you have read it but if you have read snow crash where uh the first metaverse was it was you know synchronous with our time and it was a social thing okay i mean i'm time unconstrained i can go forward in time i can go backward in time uh and for forensic purposes uh kind of two ways one is if i've i've captured you know the state changes i can just simply walk backwards or i may even be able to simulate backwards but for example if i'm looking to see you know i've got an airplane that i'm trying to do i'd like it to fly you know 20 years with emissions and see how it performs so so the advantage we have in the digital world that we actually don't have in the physical world is we're time unconstrained so we can pick our own cadence of time and assuming in simulation we get our either physics or correlations right you know we can go predict it so you know one of the things that i

keep saying it is an opportunity is this idea of what I call a front running simulation, that every product in the future will at every time zero collect its state in the physical world and then predict what's going to happen, even either with causality or Bayesian predictions, what's going to happen. And so it gives me a little crystal ball. Everybody will have a little crystal ball and be able to look into the future to see, at least probabilistically, what's going to happen to their vehicle. I mean, the example I kind of always use is, I never wanted an airplane. engine to fail at 30,000 feet, okay? I don't want it to fail, you know, when I'm sitting in the boarding area because it's going to be a long delay. You know, what I really like to do is predict that there's going to be a component failure five weeks before it fails and the next time you do the hub, have a replace. And so this idea of predictive maintenance, the ability of, in essence, you know, being time unconstrained is a huge opportunity here if we kind of do three things. One, the centering to collect the data of what the world is. The second thing is the ability of running simulation predictions. And the third is having, you know, the probabilities.

of the various thing happening. So, so, but it's like, it's a huge opportunity, you know, all we need is more computing power, but, you know, given the story I told at the beginning where I only had 192K worked with in the, in the 70s, um, you know, I'm, I'm pretty confident to, you know, wait a lot long enough and have enough computing capability. Yeah. That's interesting too, because you, you mentioned earlier, you know, data is sort of this, um, force, this existence of reality, this by-factor or whatever, a by-product of reality that we have to deal with. Time is, is that way too. And so.

as these digital twins become more sophisticated and that crystal ball analogy certainly plays out because it feels like this incredible tool for an organization, really, to be able to kind of future cast what might happen based on a wide number of variables. Yeah, yeah. You can see why, of course, something like that would be of huge value to NASA. Were there things that you kind of saw or learned there that proved valuable to smaller organizations as well? Well, in fact, NASA's a little high bound, so we were sort of pushing the envelope. I mean, the actual digital twin name comes from John Vickers, who was my colleague.

But when I got there, they were basically fitting things in their new rocket in a plywood version of it. So we're really pushing them out. Look at what SpaceX is doing. And I think that nobody has to push SpaceX off. I think they've shown what they can do if you move into the digital realm. So it wasn't so much that as the ability of the fact that that.

You know, the idea of NASA is you had no ability to go get your stuff once it's locked, right? And so you better be right the first time. And the ability of being able to model and simulate in order to work out the cake. So, you know, what I'm always saying is, you know, what I really want to do with new product development is I want to create the product virtually. I want to test the product virtually. I want to manufacture the product virtually. I want to operate the product virtually. And only when I get it all right do I want to basically go out and move around expensive apps. And especially when you're talking about space aspects, you know, you have no ability to say, oops, you know.

Let's go try that one again. I mean, if $500 million launch, you know, it gets damn expensive to say, oops. So you really want to, in essence, get as much of that done, you know, digital virtual arena that you could possibly do it. But I think it's applicable in all areas. In fact, you know, as I said, I listened to Dr. Hood. You know, what he was really talking about is a crystal ball into the future for individual patients, right? I mean, that was his perspective. I mean, I'm data-driven. I mean, I think a lot of times that we basically go by common wisdom rather than data. But if I'm data-driven and I get my, you know, probabilities right, okay?

um and you know you know absent some black swore and stuff uh i should have a pretty good handle on what's going to happen i mean i prefer causality over correlation but i'll take correlation every day if the correlations are high enough you know they um that at least gives me ability to put probabilities out of things to uh to decide and my best probability is to do this rather than that right And I think that's like this idea that digital twins can help both AI agents, but mostly humans make better decisions. Like you said, it's probabilistic more than it's black and white. It's not going to tell you what to do. It's just going to tell you the likelihood of success or failure or whatever it is the use case is. I just make you better informed and make your decisions better informed. And I always just imagine even a low to medium fidelity point of your company and how useful that would be for running prediction. Like you said, that was a great example. Like, will this project run over budget or be late?

it we're not talking about it high fidelity twin necessary to to determine something like that it's it's it's not high compute it's it but but it's use case driven you know it's it's you know and it's relationships of data to understand like okay this project is connected to this person and this person was connected to these five projects that were late um uh the the one thing you've talked about a lot and i really connect with is this idea of the effort and cost and efficiency of keeping it updated and how important that is to the whole system itself. And I often think about the vacuuming robot, just imagining if they tried to release that robot, but you had to draw to scale your house. And then every time you moved a piece of furniture, you had to go in and move it to scale. The vacuum just would have not taken off. It just would have gone nowhere. But the fact that it maps your house and self-updates its digital twin of the...

of the floor plan um is the difference between being useful and not at all so i i that's my example but you you talk a lot about that being a critical component that it's that the maintenance you know getting a twin that that's out of date um you know instantly is not useful and maintaining that what goes into maintaining that It all depends on the use case. I mean, some digital twins don't have to update a lot because, as I said, basically, you know, the oil life of my car, you know, I can look at that once a month. But, for example, let's take your example. If I had, you know, a showpiece house that I moved nothing in, once I had a map, it would be fine. The issue of it is that if there is change, and change is always the thing that kills us, right? It's the reason people don't like change. I mean, now I have to do something with that. So tell me how much change you got, and I'll tell you, and tell me what the use case is, and I'll tell you how often you need to update.

your uh um your your map um your digital twin i mean one of the examples that i've used i can use i think is with odni is you know a digital twin of a hostage situation okay i mean knowing where being able to have a digital twin of of the building where the bad guys were where the hostages were where they you know the uh infill and x-fill rocks were and being able to basically keep it up to date so you know when i actually went in to do that um i i knew where the bad guys were that that and where the hot guys were and how many of them were and so so so the digital point of that required rate fidelity, but, but at the, at, you know, a continual basis and especially at the point of time when I'm going to go out and rescue them. So, so that, you know, so that was sort of people said, well, that's reality capture. No, that's a digital twin. I mean, it's not a, it's not a static thing. It's something that, that it's, it's critical to know exactly where all these pieces are, you know, at a particular point in time. I love that example because it's, it's, it demonstrates that it's not a physical object always like it could include that, but it's, it's far more. And one of the.

things i'm most excited about in this realm is the idea is the human component as you said there are hostages and then there are you know kidnappers or whatever it was um terrorists whatever they were defined as and and now with with ai and llms and ai agents uh that can reason we can run simulations on the human beings. Like it's, what would they do? Where would they go? So as you affect the environment in a simulation, it's very difficult to, in the past, to simulate what the humans would do, right? It's easy when you have mechanics and this sort of rigid, very deterministic type of environment, you know, like Apollo or whatever, where you have a digital twin of something.

mechanical that has these limitations but now you're like trying to predict what people will do that's a whole nother story but now now you can have humans in your digital twin model and you can simulate what they might do given changes to the environment which is kind of fun and interesting So let me clarify that because I don't think you can. I mean, I think that's the, that is the big problem with humans is they will do stuff that is unexpected. And so that was where my pushback to ODNI was. I don't think you can predict what people will do. They will not do the logical thing, especially under stress, but you can predict where they are and you can predict, you know, how you might defeat them depending on what they would do. So you could maybe cover the solution space, if you will, but to say, we do this, they're going to do that. I don't see that happening.

So I think you got to be a little bit careful when you got humans involved in things. If you look at all, if you're familiar with Perot's work, the normal action stuff, generally all the normal accidents were caused by people doing something they shouldn't do. They left the doors open, they shouldn't have been open, or they decided they just weren't going to watch the instruments on their ship and run into other ships. So I think that my, and in fact, I have criticized Alan Turing with his imitation game was, I think we will be able to, in essence, simulate. every part of our universe except for him so we can basically uh we can basically um be able to to understand what the universe is going to do but it wins i think yeah and maybe maybe some uber intelligence does that and there's some secret sauce that basically that says we're going to do that but i haven't seen it you know in all my years of running a business people do strange things you know i mean one time i had the most mild banner employee come into my office with a gun and try to shoot me you know i would have never suspected the guy yeah to do that um so so so people people are the wild card

yeah and i think i should have clarified to your point depends on the use case if it's something simple you know it's it might be a little less predictable if it's not a hostage situation where guessing correct has high consequence or guessing incorrectly has high consequence you know you're just you're right you know where there's obvious like there's there's you know two doorways to go down and one one is glass and there's fire behind it like these are But you're right. It's less about predicting what the humans will do and more about running simulations that at least kind of allow you to anticipate some obvious human choices. Again, not the edge cases. When I talk to folks and when I talk about digital twin, a lot of them think of it as like a single model, like an AI model. And I see it as like the data itself and the system for the twin, it's usually a system, like a series of products. It's not a single...

GPU and a single model. I don't know if you want to talk a little bit about like, from a technology standpoint, what makes up a system? It's more of like an operating system than a single type of database. I have this evolution of digital twins and kind of where I perceive ourselves in is the ad hoc phase of digital twins, which means pretty much we decide we have a digital twin and we got to go build it. I think the next sort of pieces are platforms where we can take in different digital twins and have them interoperate. So there's a call for, okay, well, we're going to need standard. And I am of the opinion, and it's my opinion, I get to have one, is that that's going to be awfully difficult because of the fact that exhaustive standards for things like digital twins.

where their entire complex systems is problematic. It's where I think AI can sit in. So my perspective is that, and as I say, I've got this chapter coming out where I kind of show it, is that if we have AI sit in the middle of different digital twins, they can explore the solution spaces and be able to do the mapping of... So the example I use is, okay, I have an airplane. So the airplane has a fuel tank, it has wings, it has a fuselage, and it has an instrument panel, okay? When I put fuel into the fuel tank, okay, not only does the fuel tank have to fill in my digital twin, but I basically have to increase the fuel gauge, I have to deform the wings of the plane because they change a little bit, and I have to increase the weight in the entire airplane. So I have to have all those sorts of things happen, even though all I really care about, for example, for my digital twin in the fuel tank is how much fuel is in there.

But it impacts the entire system. And so my perspective was where I kind of use this as my sort of touchstone is that, you know, in nature, nature tries all possible combinations and lets the environment sort it out, right? I mean, that's how we do it. Humans can't do that because we don't have the resources. And in fact, nature doesn't care about individual units. We particularly care about individual units, especially us, you know? So we're not prepared to try all possible things and see if it works.

when i move into digital space that's ai's natural habitat i mean ai can potentially subject to compute bounds is try all possible combinations and see what the fit is and so so i i view ai as this you know as this nature of a digital environments that will basically allow me to to do a number of things one is certainly have interoperability between different digital twins but also explore the solution spaces and go back to what you said is be able to give me advice as you know what's my best probability um you look at the book that you showed originally i have a characteristic of digital twins called cute availability and it wasn't my best phrase but but what that actually is is this unit that that's the book um is basically i always thought that there would be an ability to have a little agent sitting on your shoulder whispering in your ear about hey you want to think about this or you want to think about that At that point in time, we're talking 2005, 2006, AI was, you know, in fact, was just refuted at the time. It's like, yeah, never going to happen. You know, today, that is really AI. I mean, AI, so I knew, and I keep saying this, is I don't view AI as a replacement for humans. We have plenty of humans. We don't need to replace them. But it really could be an assistant to humans. And again, if you think about AI's natural habitat being a digital environment and its ability, in essence, run multiple simulations, you know, and make big mistakes that cost you nothing other than a little computer time.

you know i come back and tell you as the human hey don't do that because you think that that's going to work but it ain't well there's this fatal flaw you're listening here um you know that's going to be a big deal and remember my idea of information is it's a replacement for waste of resources and so so i don't if i don't have to waste physical resources i'm a happy camper um dr england and um you i think you referred to that episode sorry i just want to jump in here this is like uh i got excited um you mentioned dr nathan price yeah the the ai on your shoulder and we kind of got into it and realized like Actually, it's not just an AI. It's probably one AI on each shoulder arguing back and forth about whether you should or you shouldn't. And then us observing the argument, learning from watching them argue. But what you're bringing up that just is registering with me is it only makes sense if they're referring to the same digital twin. Like they need the same context to make that argument useful. If they're arguing, but they're both looking at different contexts, it's a useless argument. But if they're looking at the twin and the same data and they're arguing, then it becomes useful. And so it's this idea of like, yes, we can have multiple AIs.

arguing, but they need to be operating off of similar data for it to be useful. So what's interesting is if you look at my digital twin model today, the one in fact NASA, you know, had much better graphic artists than I was, it basically shows, you know, the two spaces and connected. You look at my original model, which is in that book, is basically I have these sub-virtual spaces that are sitting below the space. I got rid of that because people say, oh, it's too complicated. Nobody's going to understand that. But that's exactly what you're talking about, is that the big space, you know, is the replication. That's what's happening in the physical world, you know, right now. If I feed that down into sub-virtual spaces and turn AI loose,

know to run different simulations they're all running off the same data but they can run different simulations and change the brand and then go back and say here's life out here's what i found here so it turns out that my original model is a little bit more uh appropriate for how ai has come into play than the simplified model that everybody told me i needed to simplify because it was too So, it's sort of ironic that, and in point of fact, in this book I've got coming out, you know, like I go back to that and say, you know what, this is a better model, quite frankly. I don't think it'll ever, I don't think it'll ever recover, but, you know, when you're thinking about it, talking about the role of AI.

I like that model much better because it does exactly what you said it was going to do. And my assumption in terms of when I talk about metaverses is that you can't have different metaverses, but they all better have the same starting condition that I'm operating. And I think that's a critical point that we need to keep emphasizing. If they basically have different assumptions about where we are in the world, what reality is, we're going to screw ourselves up. This is interesting. I was thinking about the work I did at Boeing.

where there's the Dreamliner cockpit project. And a huge part of that project was the simulator. And obviously the simulator represents almost like the quintessential example of digital twin, right? You're running simulation and crashing in virtual space instead of in real space. And you're creating a twin. And even some of that is like physical, right? Because you've got the cockpit recreated, but digitally it's a digital twin behind it because it's not a real airplane.

it seems like that's been around for so long. And it's so obvious its value, like being able to like sit in there and run simulations and credit. We've known this for so long. And what's driving me nuts is it seems like this concept is so obvious and yet only implies like in these very niche areas. Why do you think this hasn't, like it's so obvious its value. Why hasn't this taken off? Why are there not more stimulators out there?

first of all I landed that single your Boeing field a number of times so um so I'm well aware of that and point of fact when I maybe you know I set it up for a windy day and I'm having problems I said oh my god what do I do when they put the switch and land itself I said exactly but I think I think you're seeing more and more and more simulators come into play I mean I've seen a number of them the OD has got a whole bunch of them working out I mean Lockheed Barton north of Grumman all the big guys are in essence, you know, buying into simulations, you know, on air vehicles, ground vehicles, things like that. I think the thing is, is that, you know, the physical world is a fairly complex place and getting it right in the digital side is sometimes a little tough and it forces, you know, organizations to figure out exactly what is the causality here and build that in. And a lot of times, you know, you have to have not only the programmer can do it, but basically the engineers, you know, how this stuff works.

So I think that it's, I think it's one of those things, you know, they call punctuated equilibrium. You know, there is a whole lot of it and all of a sudden, bang, you know, you see, you know, tons on the, you know, out in use. I think we're seeing more and more and more simulated scenarios as, you know, you need the tools, you need the environment, you need the people who understand what that is, and then you just gotta go out and go get them done. Back to simulations, you know, you're seeing tons of those come about now because, you know, I mean, I kind of laugh, this tutorial production system was, you know, the touted thing. But they had to move around physical equipment on the plant floor, and it was expensive. Today, you know, I basically have it on my system, and I want to see if I can basically get the stamping machine to produce more parts because I'm going to feed into two different lines. I just move it around and then run it. So, but you're right. I mean, I think part of it is, you know, is what I call retirement-itis. I mean, I think that there are executives today that don't want to invest in this. And I say two things. One is that I always use the Arthur C. Clarke.

uh phrase that any sufficiently advanced technology is indistinguishable from magic and you know having sat on the board of companies you know they don't like magic you know crystal ball's scary it is and so so they're not going to bust the magic because they don't understand it i think that as digital natives you know get older and start to take those positions but the other thing is these people growing up in the physical world I mean you know contrast you know one of the things drive me crazy is this idea that that the digital swing starts with Apollo you know there's 192k where the digital stuff at that point I actually when I went out to to NASA the first time in the early sub like after the Apollo program you know I asked to go to I needed there was never problem with their apron which was a gb435 which you have to look up but and I had the the time-sharing version which was a gb465 so out there they're in operation asked for the computer room. They said, oh yeah, we're expecting you, Mr. Grease. I was in the doctor at that time. And they took me this room. It's a room full of women with calculator. I said, what's this? They said, this is our computer room. These were computers. I mean, I said, no, no, no, I'm looking for the actual computer itself. They said, oh, you want the mainframe room? And so that was the state of NASA at that particular point in time. So there wasn't any digital printing of anything at that point. But I think that we're starting to see this kind of capability.

because of our ability to do it. And it's not only a computer capability, we need the software and the software tools are coming online to do fabulous stuff. So I have high hopes that what you described is rapidly diminishing as people understand that I can do simulations and I can run 100 of them where I could only run one physical test. I mean, physical tests are only a proxy, right? I mean, let's take car crashes because I'll use that one, is that that car is never going to be driven again, right? You can't sell very many cars that have been crash tested. And I can only afford to crash test a physical car at certain orientation.

I can crash test my digital car at all kinds of orientations and find a crumple zone, for example, that didn't show up in, you know, a front end crash or even a side crash, but an oblique crash at an odd angle, you know, creates a crumple zone and it's like, ooh, didn't know that. So I think the opportunity of replacing physical testing with digital testing is rapidly advancing. I was thinking about a conversation we were having earlier today about Tony Stark of the Marvel comic series, and it kind of correlates to the cute sort of assistant on your shoulder in that he has Jarvis, and it's a conversational interface that basically allows him access to hundreds of digital twins, right? He's got digital twins of all his suits, of his building, of all aspects of his business operations. And so it's almost like we have sort of a mental model for how big this could get, but that makes it also kind of difficult maybe for an organization, right? Because the point shouldn't be, I think to Rob's point about like focusing only on the intelligence of the system, the point doesn't necessarily need to.

B, that I have an assistant that can tell me everything I want to know about everything. But in a business context, if I can find some use cases where I can start thinking about the way different digital twins might share information through a conversational interface, then it starts to get feasible that I could create, you know, a lower resolution version of something like Jarvis that could over time evolve into something far more sophisticated. Yeah, again, it goes back to my, you know, saying about do you really need a photorealistic factory if all you're interested in is, you know, the materials moving around. and where the people are actually. So again, you know, in the physical world, it's all another proposition. We have to have the whole physical thing where we don't have any of it, right? In the digital world, we can have pieces of it. I mean, information granularity is something that really allows us to determine, you know, all I want is, for example, the outer mode line of my rocket ship. I don't need, you know, engines or anything else because of what I'm looking to do, what my use case is. We were talking to...

Remind me, Josh. He headed up knowledge management at NASA. Yeah, Roger Forsgren. He was the former chief knowledge management officer at NASA. Yeah, and I see a huge kind of relationship between like knowledge. You know, knowledge is data, right? I mean, you know, documents and it's just unstructured data, structured data. So if information is a replacement for wasted resource as driven by data, what knowledge is is a collection of that.

and what wisdom is is you know figuring basically picking the right elements out of my knowledge database so I don't screw up so so the knowledge is like is this idea if no it costs those repository of information I mean not only do I need data but I also need the fact that that if I have this data I need to do this in order to be able to not waste resource So I can create information on the fly, but I also have, I mean, the beauty of information is what they call a non-rival resource. I mean, I can use it and I don't use it up. In the physical world, if we use something, it deterrents, right? Information doesn't. And so I can basically have this information that I collect over time in my knowledge database and it's a repository. So knowledge is a repository of potential things that I can use to replace wasted resources. And it basically is driven by data because that's what you need for information.

But I basically, when I have a task that I want to not waste resources on, which should be everything, even if it's getting three kids to practices, you know, at different times, at different fields, you know, I basically don't want to waste resources. And so, so I want to pick out the information that allows me to do that. Actually, I'll determine my data, but it's in my knowledge repository. Got it. That's how I do this. That makes sense. That makes sense. Yeah. Like now with AI and AI's capability to mine unstructured data, using unstructured data.

uh now or unstructured data becomes more valuable or more usable with ai in its ability to update um and keep updated a digital twin in other words if people are having conversations or writing documents and the information is you know to update the twin is within those documents um the ai can potentially utilize that now to to keep the digital twin more in that sort of automated sense of keeping it up to date, right? It's now able to mine information like a human might. And granted, it still needs that logic of what matters, what doesn't. You wouldn't want it, you know, just mining everything. So I have a pretty simple view of that. Structured data is because the capabilities that we had, especially early on in the digital area, were pretty permanent, okay? So we had to structure data if we were going to be able to do anything with it with the poor computing resources that we have. Quite frankly, you know, we live in a world of unstructured data, you know? I mean...

Right. Do we need to structure it if we have the ability of just basically pouring it into a lake and being able to, you know, go through that lake to, in essence, categorize the things that we want when we want it? And the answer is no. So, I mean, in fact, I'm sort of dealing with it in this chapter right now is, you know, when things are advanced as well, structured versus unstructured. I don't care, you know, and I'm going to care less and less over time.

So you're absolutely right. I mean, we live in a world of unstructured data. I mean, it's the state of the physical world. It's changing all the time, and there's a whole bunch of it going around us, and we figure out what's important or what's not, in essence, to decide what to do. We're going to do the same thing, and AI is going to get better and better at, in essence, helping us do that. Yeah, if you really compare, in my mind, if you look at the structured data and systems for structuring it, like XML and JSON and things like that, they're very, very primitive in comparison to just unstructured language that creates much better context.

Super primitive, you know, it's cryptic. You know, I don't know what this, I think it comes back to your, and maybe if you don't mind, we can end on this explaining, because you kind of called me out on that and I agree, this idea that knowledge is up the chain, data is down the chain. Can you explain that chain, you know, as we go from data up the wisdom? so so so so I don't view it as a chain I view it as different component of let's call it okay for one of a better term you know a system all right and so so I've got facts about reality right I mean that's that if I'm gonna be data-driven as a doctor hood said which I need to be or anybody else um I need to have facts about reality or the state of reality at a particular point I mean it has metadata around you know who what when where why okay if it doesn't all I have is noise I have information, which is then, so I take this data in and I want to basically do a goal-oriented task that basically allows me to replace my wasted resources with this information that I'm creating. So information's got data and it's got, what do I do, right?

I collect this in a knowledge repository. So it's not a one and done kind of thing. I mean, it isn't like I'm going to take a sheet of metal and I can only use that sheet of metal for the thing I'm doing. I can use this information over and over again. And then finally, they show it in the DITW pyramid wisdom. So I need to select out the right information that gives me, you know, we can say optimal, but in essence, one of the best solutions to not wasting resources. Because the whole point of this is I don't want to waste physical resources, right? I mean, that's what humans are all about. I mean, if we do things for leisure and things like that.

But when we're trying to, you know, the early prehistoric man, he was all about survival. So that's what he used information for. If those reeds are moving, you know, it's a saber-toothed tiger. I'm in the hell out of there. So it was a pretty simple lifestyle. Now we have a lot more complications and we have a lot more tasks and things we're trying to do, but we're pretty much goal-oriented creatures that are trying to minimize physical resources.

And that's the framework that I sort of existed. And it really comes from an information systems framework as opposed to philosophical or anything else. It's about, you know, what can I do with it? So I'm all about use cases. I'm all about, you know, where do I get value? And I should apologize for that, but that's the way I help. I get it.

No, it makes sense. Practical magic, in a sense. Yeah. It is practical magic. When I throw you in, it is absolutely magic as far as I'm concerned. I just understand how that works. I think if people understood they could start small, that they'd probably embark on this journey a little more readily, especially companies. And that was really, you know, I think the most exciting part about it, you don't have to build out a high-fidelity digital twin of your entire company. You just pick a use case and what is just enough and just learn and iterate. Yeah. Yeah, this concept can be approachable if you, like Michael Singh, take it one use case at a time, in a sense.

and go out to the low-hanging fruit. It's always what I advise is pick a project you know is going to work. I mean, you go in and say, I'm going to boil the ocean and for $500 million, I'll give you a wonderful digital plan. Executives are going to throw you out faster than you can say hi. So I always say, go pick something where the use case creates value and build up. Well, great. Thanks for joining us. This was amazing. Well, Michael, we really appreciate your time. Yeah.

Thanks for taking the time with us today. I had fun. Thank you. All right. Well, thanks for hanging out with us again here on Invisible Machines. Don't forget to like and subscribe wherever you get your podcasts. Always remember that you can watch this podcast, too, on our Invisible Machines YouTube channel. Head over to uxmag.com. We have a podcast page there where you can explore our back catalog and you can look at episodes centered around popular topics. Many, many thanks to our incredible production team here in the U.S. and in Ukraine. Elias Parker, Michael Lipinov, Allison Harshberger, Ola Slessarenko, Vera Pakhodkov, and Darina Moskovchuk. Till next week. Owaki.

Frequently asked questions

How long is this episode of Invisible Machines podcast by UX Magazine?

This episode is 1 hour and 8 minutes long.

When was this Invisible Machines podcast by UX Magazine episode published?

This episode was published on September 20, 2024.

Is there a transcript available for this episode?

Yes, a full transcript is available for this episode. You can read the complete transcript on the episode page.

Can I download this Invisible Machines podcast by UX Magazine episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.