Skip to content
S4E6 How Human Should AI Agents Really Be? episode artwork

Aug 29, 2024 58 min

S4E6 How Human Should AI Agents Really Be?

from Invisible Machines podcast by UX Magazine Hosted by Invisible Machines

Sophie Bushwick is a regular contributor to Science Friday on NPR and has previously worked for Scientific American and Popular Science. As the Senior News Editor at New Scientist, she joined Robb and Josh in exploring anthropomorphism and AI. Sophie’s work reminds us that our introduction to pocket computers came with the heavily anthropomorphized Tamagotchi pets of the late '90s. As we head into the era of conversational machines, Sophie brings her insights to a conversation about the pros and cons of making them human-like across a whole range of scenarios, including those geared toward productivity and entertainment. The correlations and similarities between corporations and AI pose both ethical considerations as well as design challenges, and this discussion draws on Sophie’s extensive background in technology reporting to look for answers. Learn about orchestrating conversational AI agents for your team: https://onereach.ai/ai-agents/?utm_source=soundcloud&utm_medium=social&utm_campaign=sophie_bushwick_episode&utm_content=1 See more of Sophie’s work at sophiebushwick.com #AIPodcast #InvisibleMachines #AI #ArtificialIntelligence #ConversationalAI #AIAgents #TechInnovation #DigitalExperience #TechJournalism #NewScientist #TechEthics #FutureOfAI

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

S4E6 How Human Should AI Agents Really Be?

0:00 58:58
of matches

Transcript Auto-generated, may contain errors

Welcome to Invisible Machines, an ongoing conversation about the expanding world of AI agents and conversational technologies. I'm Josh Tyson, co-author of Age of Invisible Machines, a best-selling book all about the agentic approach to automation and the pursuit of organizational AGI. I'm joined each week by Rob Wilson, CEO and co-founder of OneReach.ai. Rob and I wrote Age of Invisible Machines over the course of many conversations, not unlike the ones we have on this podcast. We were always exploring the many technologies, strategies, and ideologies rolled up inside the term AI, and what Rob has learned about them over decades of intensive research and deployment. We are continuing these conversations in partnership with UX Magazine because attention to experience design is paramount to the proper deployment of AI agents, or as we've come to call them, invisible machines. Today, we're going to talk some more about anthropomorphization, a critical design element for AI agents.

We are already prone as humans to assigning human-like qualities to things like cars and pets, and with AI agents primed to become frequent partners in our daily activities, it's important to consider just how human we should make them seem. Our guest in this exploration is Sophie Bushwick, the senior news editor at New Scientist. Sophie has more than a decade of experience covering technology online and in print, with work appearing in places like Discover Magazine and Gizmodo. She's also produced numerous podcasts and videos and has made regular TV appearances on CBS This Morning and MSNBC. She is also a frequent guest on NPR's Science Fridays.

Prior to working for New Scientist, she was an editor at Scientific American and Popular Science. And as you'll hear us talk about in a minute, Sophie's work reminded us that our introduction to pocket computers came in the late 90s with the heavily anthropomorphized Tamagotchi pets that you had to feed and take care of. So as we head into the era of conversational machines, Sophie helps us weigh some of the pros and cons of making AI agents human-like across a whole range of scenarios, including those geared toward productivity, as well as those looking into entertainment. The correlations and similarities between corporations and AI also come up. That gray area poses both ethical considerations as well as design challenges. And this conversation, we're lucky enough to draw on Sophie's extensive background in technology reporting to look for answers. So enjoy this rousing conversation with Sophie Bushwick.

Invisible Machines is produced for UX Magazine in partnership with OneReach.ai. Named a leader by every major analyst group, OneReach.ai's generative Studio X platform was built specifically to facilitate the no-code orchestration of AI agents. GSX is trusted by global giants like PepsiCo, PwC, Deloitte, and DHL, but anyone can experience the level of agent orchestration we talk about on this podcast through interactive demos at OneReach.ai. Now, on with the show. Sophie, thanks so much for joining us on the podcast. We're delighted to have you with us today. Yeah, thanks for having me. Rob, a pleasure as always seeing you. Yeah, yeah, yeah. We were jamming.

Yeah, so Silky, I was actually, yeah, I can't sit here and pretend to be an aficionado of sizzle reels, but I was checking out the sizzle reel on your site and really very impressive. I liked it a lot. Oh, that's great. Thank you. Oh, yeah, it was so cool. And there was actually one comment that was made that you made that really I kind of latched onto. You were talking about Tamagotchis and how they sort of predicted the rise of smartphones and kind of our attachment to carrying around technology. We talk a lot. Yeah, we talk a lot about anthropomorphization on this podcast. And I realized I hadn't really thought about Tamagotchi as sort of setting a precedent for this.

This thing that, I mean, sometimes we feel like we're saddled with it. Sometimes it feels like it's useful. But I was wondering if you thought about that connection. That's the whole like feeding. What's that? It was a little like egg-shaped plastic toy. And inside on this, it had a little screen with a couple of very simple buttons. And a little creature, the Tamagotchi would like. Right. On the screen and you needed to give it food. Oh my God, that's right. And if you didn't sort of constantly tend to it and keep it with you all the time. Yeah, I killed it every day. I could never keep that thing alive.

Yeah, but like, it was very upsetting when it would die. So people would like, there's all these things like, oh, stop, you can't take your Tamagotchi to school. We'll confiscate it. But like, then the students are pleading like, no, it'll die if you take it away. And they'll be like, grab it back at the end of the day. Too late. You guys aren't going to believe me, but I remember being upset about the fact that I cared about feeding it. Like, what is wrong? I was like, what is wrong with my brain? Why do I, why am I wasting my time on this? Why do I care? We get so attached. I was reading about, there's these robot dogs that the Mako stopped manufacturing or stopped supporting so that when the dog...

ran out of batteries or it had a problem, you couldn't, like, get it fixed at a store. So it was essentially dead. And people had funerals for their robot dogs. Like, they felt that strongly, that much of an attachment. One of my favorite, favorite anthropomorphization things that people have done is these researchers did an experiment where they basically put a bunch of trash cans on wheels, and then they had these remote controls so they could steer the trash cans and recycling bins around in, like, a park in New York. Yeah, but then what was really cool is they had all these cameras set up so they could record how people responded to the trash cans, which, again, they're not even robotic. They're being controlled by humans. But the people who sat down just thought...

that they basically projected want onto the can. They'd be like, oh, the can's coming up to me. He wants my trash. Are you waiting for me to finish my food? Oh, I better eat faster. They were just having this whole conversation with these, not only with trash. Yeah, the trash can has an objective. They must have an objective, yes. Right, we're just like, as humans, I think, just so good at projecting and focusing at what another human wants, and we sort of start applying that to machines and to pets too, you know? Oh, yeah.

There's so many animal facial expressions that don't mean the same thing that human facial ones do. Like a dog making a smiling face isn't necessarily smiling, but we feel that it is. Yes. Yeah, we're always playing. Yeah, I've been reading that part of the reason maybe humans were gravitated towards cats and dogs is because their facial arrangements are close enough to humans that we can do that, that we can easily kind of project our emotions on them. It's either amazing that we anthropomorphize as adults, or it's amazing that we think we're adults and we ever grow up.

I don't know which one. Yeah, it's like a very small step from, like, heckling games with your stuffed animal and doing the name and cuddling it and then doing the same with your, you're talking about you, your robot dog. Yes, yes. Like, I think we just don't grow up. I don't think this is surprising. It's kind of beautiful, though, isn't it? Isn't it? I like that version. Yeah. Yeah, it's an imagination. Yes. But, I mean, it also does make people pretty vulnerable to doing this with chatbots because the chatbots talk back so fluently now that people just, it's not very... It draws you in. It draws you in. Exactly, yeah.

Yeah, it's a sucker for it. And they've been primed by Tamagoshi to care what these conversational things have to say to us. Yeah. Right? Yeah. Well, there was something, a character AI company, I think a couple things were being reported on them either this week or last week, but basically they sort of, some of the people who use it say that they had changed the programming somehow, that the bots they were used to communicating with and these characters they made were no longer as loose and sort of...

as unpredictable as they have been. Oh, temperature. Yeah. They call that temperature. Probably. They turn that up more randomness. So less randomness. Less randomness. Yeah. Yeah. People get very upset if you build this.

Even if it's one-sided, I think people do feel a rapport with chatbots and you build this rapport. You have what feels to you like a friendship or a relationship and then your partner changes overnight. And then it gets boring. And what's funny about that is why does it get boring? Because most of the text that's fed into this LLM comes from books and professional writing. So we're all like, oh, you mean boring. Yeah. Yeah. I mean, it's the same thing with the companies that make their bots like very, very polite, right? You've got limitations on it, but it also means it's going to be left worth it to surprise you or say something weird and unexpected and interesting.

Now we could talk about real animals and ability to communicate with real ones and then say, are they Tamaguchis? What's the truth here? Or is it both? I mean, I think that there's been a long history of humans trying to talk to animals or to train animals to talk back to us. And sort of best practices and most ethical behavior has kind of shifted over time. I know I think researchers are trying to, I think we're really aware of that animals are capable of communicating in really complex ways. We might not necessarily put that at the level of human speech in terms of complexity and in terms of the range of thoughts that we can express, but they are doing much more complex communication than we used to think possible. And I think that they're sort of, when you talk about animals, they're sort of, on the one hand, we have a tendency to attribute too much to them, to be like, attribute human-like feelings to them.

But on the other hand, there's a tendency to say, oh, they're not as complex as humans, so they must not feel anything. They must not feel pain or emotions or communicate by name the way we can. But, you know, we are finding that there's a middle ground of sort of like appreciating animal communication for what it is without trying to make it the same thing as human speech. Right, right. Yeah, anthropomorphizing it. I think there's a...

Hey, you kind of get to the practical nature of communication, the survival portion, you know, and you say, well, it's about how well we coordinate and work together. So without understanding it, we could probably infer how complex the language is by just how well they work as a team, how coordinated they are and how coordinated they're not. And if they're not, it seems hard to believe that the speech could be so sophisticated because then what's the point? Right, right. Yeah, because a lot of social animals, you know, they're not necessarily like, let's make a plan to hunt this prey animal, but they might be.

you know, having a social interaction that enforces bonding and that makes them feel a connection with each other or wanting to support each other so that when they do go out hunting prey together, they'll work better as a team, even if they're not, they're not like, you know, laying out. Directly saying, you go to the right, I'll go to the left. But yeah, it's like mycelium, right? Where it's just trying to right size themselves and their environment. And it's not, but the way we, our purpose of language is to coordinate and hunt. And maybe theirs isn't, and therefore that's why it's complicated. And we're trying to bake it into the way we do it to coordinate and hunt. But that's not, we're doing something else to survive. We're trying to make sure we don't overuse the resources around us or.

you know or we're avoiding a fight that's our point like to not even get into one you guys are looking for one we're avoiding it you know depending on you know on what creature you are like a whale is not looking for a fight um So, yeah, we don't understand it as long as we think it's human. Right. I mean, in a lot of cases, it's hard to understand, not just because of all the baggage that we're bringing to it, but just because, like, they're communicating with a combination of dancing and of audible communication and of baromones. And smelling, yeah. Yeah, so researchers, I think, are getting better at, like, developing these sensors that can pick up a lot of different aspects of animal communication. And sensors are sort of small and transportable and sturdy enough that they can be placed all over the place. Like, they can be, you can set up, you can sort of...

bug a cave of bats in order to make these like very careful observations of them in the wild as opposed to in laboratory conditions when you know they might behave really differently um there was one study that used a i wanted to study bats again and they wanted to study bats as they were flying and echolocating and in order to get a sensor into this swarm of bats they used a trained hawk They trained this hawk to carry a little backpack with these sensors on it. And it blew these passes through this swarm of thousands and thousands of bats, just enabling the researchers to record all these signals. And I thought that was incredible. Yeah. I'm, I think I tend to be more fascinated, not with the interspecies communication, but the sort of, uh, or sorry, the interspecies communication, not this communication with any species, not the.

whales talking to other whales but but whales talking to other species and that because it seems more like uh more about the ecosystem right i mean it does feel like the more we learn about animal intelligence it feels like we're discovering that they're they're smarter than we give them credit for and then it doesn't seem like we're discovering like oh they're dumber than we thought yeah it seems like the trend is towards like animals are smarter but it also feels like some of that so that intelligence is sort of hidden from sight because of the way we perceive communication and intelligence like some of it might be Like it's, there's intelligence in the way bats swarm because they've been evolving these patterns in their activity over hundreds of years. And so that's a part of us to like pick out. The interspecies, like if you think about that with AI, right? Like we don't, we don't wag our tails when we get home to our dogs. I mean, most of us don't, right? We don't just wag our butts. Like we, we expect them to speak in our way. Like, you know, they sort of mimic us versus we mimic them.

Like we expect other, but that, but it's funny with the same with machines. Like, you know, we've spent so much time mimicking machines. Like we, we use interfaces. Like we talk in their language, right? We've been trying to, to learn how to speak to machines. And now we have this, you know, this moment in time where, where machines will now communicate with us in our, in our way. And that definitely confuses us because we start to think they're other people. And where Josh and I were going, where we're getting into, I think like an interesting conversation right before this was.

Like if we're going to anthropomorphize machines, which we can discuss as good or bad, and it has this ups and downs, it's the same way animals. But setting that aside, like what type, like there's this whole Pac-Man, there's alpha, there's the beta. And we kind of got into this, like, I think everyone's afraid of alpha AI, but I think they're all cool with beta AI, you know? Or the front AI? AI as the servant as opposed to AI, our alpha leader. Yeah. And we've been worried about that from the very beginning. The word robot comes from a play about these machines that were the laborers and they were enslaved and then they had an uprising against the people who were exploiting them. So that's the classic. It's like we've never been able to get away from that idea of machines. We see it again and again in fiction from like, you've got that, you've got HAL 9000, you've got Terminator, you've got the Matrix. We keep telling these stories about...

AI that starts out as a tamed, you know, helper and then rebels against us and destroys humanity. It's like, is that actually a likely scenario or have we just sort of, or is it just like a good story? Because that's based on humans, right? Science fiction about enslaved machines is really science fiction about like oppressed workers, right? If you're talking about the era that this is dating from. And so we're using the machines as metaphors. And I think the trick is to not let that skew how we see.

the modern machines and mechanisms that we're calling AI now. The AI here is not the same as, like, the AI in fiction where it's played by a human actor and, like, maybe they put some makeup or prosthetics on, but it's like, you're like, oh, there's Jude Law playing a robot or, like, oh, that's Scarlett Johansson's voice sounds so warm and lovely as the robot. Yes, but why was she so submissive? Why not, like, the dominant, like, female that was, like, telling him what to do all the time, you know? Like, you know? Our voice assistants, Alexa, Siri, I'm dominant. Exactly. Yeah, our toys. Like, do we make our toys dominant? Are we the dominant one in our play?

in our minds, right? That's where Josh and I went. We ended up like, well, maybe anthropomorphizing keeps us safe because we're not going to make any alpha AIs as long as we're anthropomorphizing because we'll always want them to be best. It could be the other problem, right? Like if you anthropomorphize your AI and you get used to like scolding your AI assistant, well, are you then going to treat other humans as poorly as we treat your AI, you know? Like some people say please and thank you to their assistants, but most of us don't because they're not people. Like it would be bad, I think, to get into the habit of always treating a human service worker, you know, someone in a restaurant the same way that you might treat, you know, an AI chatbot.

Yeah, I mean, there are people, too, who probably spend greater portions of their day interacting with machines than people. And so, like what you were saying, if they start talking down to their machines on a regular basis, that is definitely going to affect the way that they interact with other people, especially if those interactions are happening less frequently. Yeah. Yeah, so I think we want the beta AI. I think we all can agree that's the one we want. We like the beta AI. Or even better, let's not make it a person at all. Let's just make it a tool, you know? Why do we have to, just because it sucks.

But I mean, we can't help ourselves in some ways. It's a very instinctual response that we have to anthropomorphize these things. Like when I cover tech and I'm writing about it, I need to check myself all the time from saying things like that the AI wants something. It doesn't want anything. It doesn't have wants or feelings, but there's that instinct to sort of describe what it's doing in these very human terms. And I sort of have to catch myself from doing that because, yeah, even knowing that it doesn't feel emotions, I sort of continue to think of it, you know, a bot in terms that I'm more familiar with. Right. Yeah.

yeah i wonder too if like as these systems get more sophisticated if it might be harder to kind of assign that that single person feel to them because like right now they do things that are pretty similar to a lot of the interactions we have with other people they can answer questions for us things like that but if they're part of this like bigger ecosystem where they have access to so much information are able to isolate patterns and find connections that people just aren't capable of seeing and then bring us these choices that we can make that are just kind of like astounding in a way it might be hard to like like we wanted to stay in that beta mode but we're also like completely wowed by its ability to outthink us Yeah, I think there's a designer responsibility here. Sorry, go ahead. I was just going to say, I think there's a designer responsibility here to say like...

designers have a bigger responsibility to not anthropomorphize you know to make the most out of it that says look you know and i see one great example the the anthropomorphic nature of these systems is like i ask a question i get an answer um we don't most people don't even stop to think that that's anthropomorphic like why don't i ask a question and get four answers that are different and have different perspectives right like so so a designer needs to go like but that's a better answer four answers is a better answer than one answer um one answer is is limited that's what you get out of it mere human right like why don't we get four answers um and then you know sort of all have valid perspectives and then we can either merge two into one or we can choose the one we want because that makes more sense and then the interface needs to not look like a chat session it needs to look like something where you plug plug in a question and get four answers you know something that looks more like maybe a web searcher I mean, not just that, but why are these answers being given to us written in first person? Why does the LLM use the term I? You know, that's a design choice. We don't think about it because we're so used to it now, but like that was a design choice and that absolutely adds to the anthropomorphization, hard word to say, but it absolutely adds to the anthropomorphization of this thing. Yeah, and we see OpenAI like feeding into it. Like, let's get, you know, Scarlett Johansson's voice and let's make it flirty. Like, let's make it really giggly and flirty. And I'm just, to me,

you know, I'm just like, this is wrong. This doesn't feel right. This feels like we're, it's deception. It's not wrong in the sense that I can tell you that it's clearly wrong and here's the damage, but I can tell you that I generally have just learned over time that deception in general leads to not good things. And that truth, even no matter how hard it is to tell it, tends to lead to better things. And it's just better, an easier life is a truthful one, no matter how embarrassing it might be. And so that's kind of where I go.

I think that the idea is that the people who are making these bots, the goal is not necessarily to deliver information to you in as clear a way as possible while making it very obvious that it is a algorithm spitting out the answer. The goal is to engage you. The goal is to make you use this product and to make you use it for as long as possible, right? So in that sense, the deliberate blurring of the lines, the deliberate use of this chat interface of this first person, it's engagement. It's to keep you engaged. It's a deliberate choice. It's not like...

a happy coincidence because they could they could change at any time right yeah they don't because that's not the point one of the things we talked about earlier was this idea of like all these ai assistants or even us smart assistants they often have female voices by default right they'll come a lot of different voice options but the default that they ship with is the female voice and there's been there was one case i heard about a company was developing a um a chatbot and they had a uh they used it had a it had a female voice and a little like cartoon face and they found that people who were using it were hitting on it and flirting with it and also being really rude to it in some cases and they were like this isn't very good They changed it to a neutral orb and people stopped having those, you know, corporate conversations with the bot, but they also stopped using it as much. They weren't as engaged. And so the company switched back because the goal wasn't to just deliver information neutrally. The goal was to play with it. Yes, exactly. So that brings up like a past episode we had with James Bridle, who talks a lot about the machine not being the AI, but the corporation is the AI. And we already have AIs. They're called corporations. They have legal entities.

They're held responsible for their actions. They possess things. They have assets. And we actually treat them like they're entities. And it's that.

that these machines at service to this AI, which is a corporation, this artificial intelligence, that's a corporation. And it's the corporation's objectives that's scary. Like the AI is a tool to meet corporate objectives. And what I think you're kind of saying is like, the goal is to make money, not to make the world a better place. And so this sort of anthropomorphic, you know, these attributes are just a window into a company that's gone from one.

that maybe is trying to move the world forward to one that's trying to make money. And you're like, oh, well, maybe this is advancing it, but it's advancing in a direction that's deceptive. if I'm to be negative or more fun if I'm to be positive. Yeah, I mean, one of the examples people that has been given about the threat of super intelligent AI is the paperclip optimization problem, right? Oh, I thought that you brought that up. Yeah, so the job of AI in this hypothetical is to make as many paperclips as possible, but then AI is like, you know what you could do is you could harvest a lot more raw materials just with paperclips and it ends up processing all... Right.

our life and objects on earth wanted to turn them into paperclips and i have to say like for me the scary scenario is not necessarily that it's the scenario where you have a ceo who's being advised by ai and the ai is like you know how you can optimize these paperclips turn the workforce into paperclip or wait what if it's what if it's a paperclip company what if it's just a paperclip company or wait what about an oil company what happens an oil company and its objective is to make profits and if they just pursue that without conscience that they'll destroy the planet and us wait it's happening to their shareholders you know let's see i can picture someone justifying a pretty negative application of ai in order to do that, you know? Yeah, any corporation, right? I mean, why wouldn't oil companies use AI to perpetuate profits, right? I mean, they are. AI is a useful tool if you're looking to harvest fossil fuels. Yeah, yeah. So I guess it just kind of comes down to our problems with the corporation, maybe. And our corporate objective is not as much the tech. Like, the tech furthers that objective, absolutely. But maybe we all have...

a real issue with the alpha AI, which is the company, the one we all are subservient to that tells us what to do. That isn't a real person, you know, that has this objective of profit. Yeah. I mean, I'm kind of channeling James's book at this point, but, you know, but there's something to that, isn't there? I mean, that these AIs are, they are just tools and given the right cause, they'll further those given the wrong cause.

They won't. Yeah, absolutely. That's why it's very, one tricky thing is like, there's been a lot of coverage about AI and energy usage and AI and climate change. But so when you've got people who are like, well, you can use AI to help develop greener solutions and help hasten the transition to renewables and that it can, you know, it can do all these great things that will help mitigate climate change. But at the same time, AI is already being used in the fossil fuel industry in a way that contributes to climate change. And AI is also enormously energy hungry. It requires a ton of water. It requires a ton of energy. And, you know, it's just...

It's very hard to predict how exactly it will impact climate right now because so much depends not on any inherent property of the technology, but on the way that people are and will apply it. Yeah. Yeah. I keep going back and forth on this. It's hard. I mean, in the industry, I've been in it for a long time, so I work really hard to not get sucked into the pro-AI mindset that services me, of course. And that's hard. But I think a big part of it is to say, you know, AI...

you know, will service, you know, bad actors, it will service good actors. And, and the question is, like, is AI... forcing us as a society to start looking at these corporate objectives because now we can see the danger. Like before it was like, eh, it's dangerous, but it's, but now it's, now like, do we have to face the fact that capitalism and as it's, you know, as it's exercised today and corporate objectives that we all complain about, but don't do anything about, like, does this, does this finally make us have to address this? Because it's not AI, it's global warming. Like there's a lot of things happening that are bringing society together to question the objectives of.

of the corporate AI, the Alva AI, right? Well, I think a lot of the problems that AI is, and the worries about AI are really about AI's ability to take an existing problem and make it worse, right? So like climate change, it's not because of this technology that it's happening, but we see its impact and it's sort of like a reminder of what the actual situation is. And the same thing with misinformation and disinformation, right? People talk a lot about bad information online and AI can absolutely contribute to that, but it's not necessary for it to happen, right? I see plenty of people sharing.

You know, there's sharing a still from a video game and claiming it's an actual attack in a conflict zone somewhere on the globe, you know? And then that'll get, you know, shared a ton on social media with people just buying into it. So like AI can make and generate an image showing an explosion, but is that going to be that much more effective than what's already being used? Right. And the same thing with talking about the problem with corporate focus on profits and all else. Right. AI can contribute to that, but it's not the only thing. This is a problem that already exists and the worry is just AI will make it worse. Yeah. And it's a James Brattle's point, but if you include the corporation.

into the definition of AI. If you're, if you say, you know, just like you study life and you realize life's a lot broader than we think it is. You know, a lot more things are living. If you tick the rules of what alive is, there's a lot more things alive than we've, we've noticed, right? We think things are inanimate and they're like, ah, they're actually communicating like plants, right? Right. So if you extend and you say, no, the corporation is an AI. We anthropomorphize it. We give it a name. We give it a face. Yeah, we call it a logo. We give it a personality. We, there's things that it does and it doesn't do. We all refer to it. The company is making me work late tonight. The company is like, you know, we.

Maybe if it is AI, if it's part of it and the machines that do machine learning is just, you know, an extension of this AI, just like we all are extension of this AI. I know this gets really deep. But then when we talk about the AI getting out of control, we're including companies in that. We're including companies in that definition. Then we're saying, well, it's types of AI and maybe it's just alpha AI. Maybe it's just... It's undefining the form of intelligence that a corporation is. I wouldn't call it AI. I would call it...

Yes. Yes. Yes. Yes. Yes.

Yes, I agree. Let me play devil's advocate just for fun. But if all of the data that the AI has ingested is us collectively talking, then it is us as a collective. It is a group of people. It's not a thing. It is just channeling an amalgamation of a lot of people and their ideas and their thought, just like a company is aggregating the ideas of the group, like you said. And so now we're just...

It's just the definition of what form of aggregation it's taking. But if you think of the machine learning as just a channel, like a telephone, right? There's two people on each side. The telephone doesn't have ideas. The telephone line isn't mediating or doing anything. It's just translating from one person to another. And then you go, well, what if there's a million people on one side of the telephone and one person on the other side? And you try to synthesize everyone's ideas into one sentence.

That's just the telephone line. That's just a smart telephone line, right? But it's still a bunch of people, their ideas getting amalgamated and simplified into one to the other, which is just like a company, like you said. And so it's just an efficient way of taking all of these people's general. ideas and communicating them in like one simple sentence, losing a ton of detail, as you pointed out. Right. Okay. So if we're talking specifically about generative AI systems, like, so, cause sometimes when AI comes up in science, it's not necessarily, we're not necessarily. True. True. So if we're talking about like large language models or about image generators, these things that are, or music generators, right? These things that, like you say, they're trained on all this human output on all this human conversation and art and music. And then it sort of synthesizes that into a collective, you know, a summary in some ways to get its output. So if you're talking about it from that perspective, I think the.

That's an interesting comparison to make with the corporation, but I do think that a corporation has sort of external limitations that focus its behavior in that, you know, you've set a top-down mission for the company to make a lot of money, right? Objectives, yeah. Objectives, right? And I don't know that the output of an AI, like image generator or... uh music generator it has those objectives but it also has um it has the objectives of whatever you type in with the prompt right so that way you are uh focusing its behavior yes but i would say it's more flexible because you can take the same thing and have it do all these different outputs all these different kinds of outputs i mean a part of that is just like there's a lot of variety in what humans make and write and and create so like there's a lot of different options for it whereas i feel like there's a like a smaller number of um of restrictions that you put on a company yeah the channel its output yeah it's like what's interesting is uh like in our book and i feel like the more and more we kind of get into this topic like for companies, especially big companies to really maximize these technologies, they have to almost stop behaving like big companies. They have to reorganize their objectives. Like they can't, at least by our estimation, like you can't really put shareholders first. It makes more sense to put employees first and to have them take an active part in sort of crafting the way these technologies take form and organize information within an organization. So it feels like, you know, we could, if corporations just kind of run with this technology on their current trajectory, we might end up with like a really stilted version of artificial intelligence or whatever we want to call it kind of floating out in the world where

There's also this flip side to it where if companies actually do it the most efficient way, they'll sort of maybe stop behaving like such jerks. I don't know, because I feel like the people who are making those decisions aren't necessarily the ones who want to necessarily empower their employees. They're the shareholders. I think that would require a shift in strategy that, yeah, I'm not sure they would be willing to make. But I agree. I think that the people who are doing the day-to-day work are the ones who might be best positioned to say, well, here's...

an application for an independent AI agent to help me with, you know, as opposed to someone from top saying like, AI is really buzzy, incorporate it some way. And if you don't, you're fine or whatever, you know, like that's not a very efficient application. Yeah. Yeah. We, we've, we've likened the corporation. I'm not anti-corporation, by the way. This is just more like, you know, logical thinking leads you to the fact that once you have to create definition around AI, it doesn't have to be a machine, right? It can, you know, it's anything we anthropomorphize that acts like a human, but isn't right. And, um, and so that we, you know, we can kind of carve our way from that to like, where we think of a

almost like a company as a terrarium or an aquarium, right? And the problem with an aquarium is there's like, there's being in, there's being an animal in the aquarium, there's being a fish or a plant that has to live in the aquarium. And then there's us who the aquarium exists to serve us, but we don't have to live in this aquarium. So we want the water clear so we can see, you know, but maybe they don't, but that doesn't matter. We don't care what they want. And so I think like the shareholder is that like outside party that benefits, you know?

But inside are the customers and the employees that have to live in this terraria or aquaria, right? And they're like, you're making decisions that's comfortable for you guys that don't even have to be here, right? Yes. I am intrigued by this idea that the corporation is an AI and that when we talk about AI, why not throw it into the mix? And I am interested in this idea of being against alpha AI as a thing and saying anytime we position the AI, if we're going to anthropomorphize it, maybe...

Maybe we don't at all. But if we're going to, then at least making it alpha in any way, that's where we have a real design problem because people aren't going to want the AI to be their boss and start telling them what to do. Right. So, I mean, here's the question though. I mean, I see the point of not wanting an AI to be a CEO necessarily because that's absolutely putting it in that position of power. But what about like an independent AI agent, you know? Yeah, that's not a shareholder. Is that getting too much power? I think if it's, that's sort of, if it, so, you know, and any CEO is going to get up and say, I'm here for you guys. Any politician is going to be like, oh, I just serve the people. If they, an AI has a chance to genuinely actually not have self-interest, right? So I think if, if it's in check, if self-interest is in check.

then, yeah, it's going to be better than a CEO that is incentivized by stock and says, you know, in six months, I'm going to get my package. I don't care if this company lives long enough to provide all you guys with your retirement benefits because I'm going to have mine and I just care about the stock over the next six months and then I'll hand it off to the next guy who cares about the next six months and, you know, eventually you guys will get it. But we know that that short-term thinking, you know, isn't good. Josh did a bunch of research recently on longevity of companies as we talk about... How can AI affect this downward trajectory of the longevity of companies? Because they're really, they're getting shorter. And Josh, you can speak to that. I'll let you. It was interesting to read that like, you know, a lot of the longest living companies are in Japan. They have companies there that are like a thousand years old. And, you know, they mentioned Nokia as being one of them. And Nokia was this very adaptable company. They started out as a pulp manufacturer and then they kind of segued into electricity production, I think, and then eventually mobile phones. But they also pointed out that, you know, a lot of these really longstanding companies in Japan are small and don't really have profit as their core motivator.

And they're also often like family run, which kind of ties back into what we were talking about with like really wanting to focus on employee experience first for the health of the company and also to be able to properly and potentially fully kind of integrate AI technology. You know, if you're asking people to help build something that they might even think is sort of meant to replace aspects of what they do. if they're inside of a company that feels more like they have a vested interest in it, like they're not just going to do the work and be forgotten, like that they are part of it and that the company is actually kind of changing the way they exist in order to benefit the employees as much as benefiting other people, then it's easy to see how a company could sort of exist over the long term, even through something as tumultuous as sort of this AI revolution or whatever we might call it. We've constantly kind of been advocating for AI, start with AI to make employee experience better. Don't start with customer, don't start with, I mean...

you know, I don't think too many, well, I guess, yeah, shareholder, like how do I save money and get rid of employees? But what if you, what if you took the contrarian view to that and said, I'm going to start investing in AI to make employee experience better? The evidence shows that if longevity is your objective, then why not look at those companies and see, well, a family-run company, it's clear employees first. There's just no doubt that that's what it is. And so, yeah, it kind of comes down to changing the corporation seems too daunting. No one's going to do that overnight. But if we just focus on employees and AI to make employee experience better, not alpha AI, but beta AI, then maybe that evolves into better corporate objectives because now the fish in the aquarium get to decide.

the way they want the temperature of the water and the clarity. I really like the aquarium metaphor. It's very easy to read a picture. It doesn't make me feel a little bit like a fish, but yes. i think is interesting though is the you know you talked about sort of setting the objective of the ai to become selfless to be sort of helpful as opposed to domineering but sometimes you set an objective for an ai and then it ends up doing like having a really roundabout way of doing it that ends up still being weird so like there's a paperclip example right but then there's also something like i think i'm not gonna remember the details here but researchers were training an ai agent to play like a racing video game and they found the objective was just to get back to the finish line as quickly as possible. And so it would immediately crash the car because that would reset it back to the beginning and then it would have completed the objective. Like the fastest way for it to complete the objective was to crash the car. And so that was a case of them having set what seemed like a reasonable objective and then having the AI inadvertently show a potential way to sidestep that. Yeah. We had a conversation on this podcast once we were kind of exploring the idea of like, what if AI only spoke in the third person? Like if it just referred to itself that way. Because when people do that, it's kind of alienating and odd. Like when people speak to you in the third person, you kind of...

you know they don't they feel less human in a weird way so it could be as simple as that uh you also mentioned like the the difference between people communicating with kind of the orb avatar versus like an anthropomorphized one and it kind of reminded me of another thing we've talked about a lot which is that like skewmorphism has this interesting connection with anthropomorphism in this realm because part of the reason some of this anthropomorphism might exist is to kind of like keep people engaged so they the way that drop shadow on a calculator app let people know that like hey this is a calculator kind of like hey this is a thing that you can talk to but then you know you know there are opportunities to use it constructively in design to point people in the right direction but then is there a point where it sort of outlives its usefulness and where everyone's now like okay we get it we're talking to machines they don't really need names they don't need faces as much anymore yes so yeah so interesting to see yeah because at the same time i thought that might happen but then i was wrong because a lot of companies have come out with these like ringing of ai characters you know oh my god i had all these bots that were like supposed to be inspired by real people and now there's uh discussions of you know social media influencers making bots that their followers can chat with that will sort of

simulate what it's like to talk to that influencer and new companies like Character AI and Replica that are building these customizable bots for people to have conversations with. And it does seem like there is an appetite for folks to have these conversations with the anthropomorphized version. And so I think as long as there are people, as long as there's demand for that kind of thing, then we'll definitely still see it. But I mean, maybe the personality will flow in the direction of that application and then the AI apps for use in the office will hopefully skew more towards the person. Yeah, the UX world. It's easy to get kind of focused on productivity and forget about.

The fact that a lot of people are lonely and want to be entertained and like have a conversation. AI for entertainment. In the UX world, you have to know what mode you're in, right? You have to know because if you're trying to entertain somebody with a video game, them throwing the controller across the room is a win. If you're trying to help somebody get their job done, them throwing their keyboard across the room is not a win. And I think, yeah, a lot of these tools, you know, they're just entertainment. And I wonder how entertaining and engaging they really are. Like I've tried to talk, I'd like rather play Tetris than talk to one of these. I don't know about you guys, but I just, I find it like not quite hacking my brain in that fully entertained way. So I see like, maybe it's a piece, maybe it's a small piece of, of an entertaining experience, but it's definitely to me, not the centerpiece, you know, I'm struggling to buy into the full.

entertainment aspect of it. You know what I found really helpful was I agree with you because to me the chatting with a bot is not very appealing. I don't want to spend that much time on it. But then I listened to a podcast miniseries called Bot Love and a couple of journalists interviewed a lot of people who were heavy frequent users of Chatbot Companions. Okay. And it was really enlightening about the way people felt about this and the addictive behavior it engendered for some people. So I think this might be one of those things where it's like it's a niche application. It's not going to be for everyone. Right. But the people who like it, they are intensely. Totally great. Because entertainment fills something, some void, right? Like we get to be the king of our own.

that that's what we always wanted to be and so yeah for those that maybe socially just you know are starved for companionship then then we then yeah we call it games but maybe it's not games you know it's just a way to get companionship and if that's convenient and it works and that's how you get it that's how you get it and and so yes it's probably a minority of folks that crave companionship to the degree that a bot is satisfying and they don't have better real life alternatives um but yeah of course like if you're in solitary confound confinement any human We'd probably start talking to it every day. Well, and another application people talk about is for the elderly, right? To have these places like... Speaking of solitary confinement. Right. They're designed for people who aren't able to interact with fellow humans and to try to alleviate some of that loneliness. Yeah. Yeah. So again, right, there's a place as a tool. And then maybe, yeah, then there's a question, is it entertainment at that point? Like, yeah. I do wonder about the long-term effects, right? Like you've seen recently, like there's been a lot of rise in sports gambling and the accessibility of that.

And so I think that we're just starting to see how that's affecting people who have like gambling addictions. And I do wonder if there's going to be people who are addicted to using these chatbots and sort of let their human, human interactions atrophy a little bit in response to it. Like that's an unintended potential consequence. I agree. I agree. We've, we've thought about it as like kind of food. Like all of a sudden.

the whole world gets like potato chips and processed food. Like, oh, you can put foods in cans and it can last a long time. And now the world's like, we're solving the world, world hunger problems. And it is valuable. Like, wow, cans, like this is cool. And processed food, like this is amazing. It's lasting longer. We're not starving. And then the world gets to a place where you can have real food. And now the canned processed food is not so good anymore. And so maybe this is canned processed socializing. It's empty calories of socializing. Like, yeah, it's better than none, but.

But instead of potato chips, maybe carrots and salad, like maybe real people. Perishables. Just got to pass this phase where it is better than nothing. Yeah, yeah. Like, I think that's the thing. I think ideally we would have...

you know people talk about the epidemic of loneliness and ideally we would have that we know that there are other solutions to it but it does feel like chatbots are sort of like the shitty band-aid you slap on top of a bad situation they're like this is what we've got for now yeah but what about the ai that facilitates human interaction with each other like isn't that like instead of imitating it right or sodifying a human like get out of the anthem like i'm not i'm not your friend but i'm gonna but i did set up a meeting a lunch meeting with you and another human That would be a good thing. And I took care of all these tedious tasks, so now you have time to meet with this person. Yeah, I need the reservation for you. Here's the travel instructions for how to get there as quickly as possible. Right. All the barriers that keep you isolated, I've removed so that you have no excuse now. You have to go eat your vegetables. So it's, yeah, and then you're like, well, who's going to make that? Well, the company that makes money from that. And, you know, in the early days, you stock the shelves with junk food. That's how you make money. But look, it's changing, right? It is.

Like the Coca-Colas of the world had to start buying Snapple. Switch to bottled waters and filters. So we as a society might just get tired of empty. So I just think it's a passing escomorphic phase. And I think that AI is facilitating real human connection. I think that's real. I think that would probably be what has staying power. But, you know, again, I'm that optimist. Yeah, they can do things that are vastly more interesting than just talking to us. Like if they can optimize our daily lives in ways that are astounding to us.

kind of change the way we feel about ourselves and our relationships with others, then maybe we're not as interested in whether or not they yeah we built an ai that could like like we're happy that they're not we built an ai that could that could coordinate like 30 people to attend a single meeting and accommodate like the best schedules and the mandatory people that had to be there and the people that were the nice to haves and and and communicate with them each individually and so all you had to do was hey i need to meet with these 30 people and off it would go that's awesome and you're like okay that was a huge obstacle to that human interact like oh the coordinating and the picking a time in a place and a um and and And if it was a lunch, the dietary, like, ah, forget it. I'll just stay home and talk to my bot. Um, but if it takes that out and makes that easy, like it doesn't have to necessarily have an objective to, to make sure you socialize, although that's interesting in itself, but that maybe becomes more like alpha AI, but maybe it's job is just, just that it makes it, it lowers the barrier as you were saying, Josh, to like getting 30 people together, being so effortless.

that we just do it more often. You know, I think comfort, we often mistake for happiness, you know? But this has been a really interesting conversation. This is great. Yeah, absolutely. All right. Well, thanks for hanging out with us again here on Invisible Machines. Don't forget to like and subscribe wherever you get your podcasts. Always remember that you can watch this podcast too on our Invisible Machines YouTube channel. Head over to uxmag.com. We have a podcast page there where you can explore our back catalog and you can look at episodes centered around popular topics. Many, many thanks to our incredible production team here in the U.S. and in Ukraine. Elias Parker, Michael Lipinoff, Alison Harshberger,

Ola Slessarenko, Vera Pakhodkov, and Darina Moskovchuk. Till next week, Owaqi.

Frequently asked questions

How long is this episode of Invisible Machines podcast by UX Magazine?

This episode is 58 minutes long.

When was this Invisible Machines podcast by UX Magazine episode published?

This episode was published on August 29, 2024.

Is there a transcript available for this episode?

Yes, a full transcript is available for this episode. You can read the complete transcript on the episode page.

Can I download this Invisible Machines podcast by UX Magazine episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.