Skip to content
Ethics and the Future of Artificial Intelligence episode artwork

Apr 11, 2018 30 min

Ethics and the Future of Artificial Intelligence

from CNAS Podcasts Hosted by Center for a New American Security

Join Paul Scharre, Senior Fellow and Director of the Technology and National Security program at CNAS, in a discussion with Amir Husain about the ethical implications of artificial intelligence on human existentialism, the future of work and war, and technological risk.

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

Ethics and the Future of Artificial Intelligence

0:00 30:31
of matches

Transcript Auto-generated, may contain errors

Hi, I'm Paul Scharring, Senior Fellow and Director of the Technology and National Security Program here at CNAS. I'm joined today by Amir Messain, founder and CEO of Spark Cognition, an AI company and author of the Sentee Machine, the coming age of artificial intelligence by Scribner. Amir, thanks for joining us. It's a really pleasure to be here, Paul.

Thank you very much. Amir, you've been on our podcast earlier talking about AI. Today, I really want to talk about your book, The Sentee Machine. This has been out for a couple of months now, out in the market.

I want to get your thoughts. Why did you write this book about artificial intelligence? I've been thinking about a lot of the stuff that's in the book for a long, long time. As I mentioned in the book, my first exposure to computers was at the age of four.

For the first time, I was exposed to this idea of programmability, that through this computer, we could make things happen the way we wanted to and those manifestations would appear on screen. This fungible world that the computer introduced me to and the idea of programmability then led me to think about whether everything around us was a consequence of computational processes. Something led to another in my early teens is when I became obsessed with artificial intelligence and started reading everything I could possibly find. I was in college at a young age, and when I was 15, I was working on my first AI paper that got published in the Nitro-E, AI publication by the time I was 16.

I did a couple of those for systems, man, and cybernetics. Then I just got the bug, if you will, because in solving those sorts of problems, which were now I look back, basic problems, obviously, many, many years ago, in comparison to the work that we do now. But they planted the seed of this new way of thinking, this way of thinking about how you can offload truly cognitive tasks to the machine and expect results that you as an individual perhaps would not have been able to come up with. That is always magical for me to witness, the machine doing something that I hadn't even thought of.

That's magical. And listeners may hear me correctly through the pages of the book here in the background. One of the things that you do and I think is really so great in this book is not just talk about the technology, but also your personal story and experience with this. Why AI?

What do you do with your life? What really drove your interest in this area? Well, there's two things. One is a very earthly mundane reason.

And the other one is perhaps just my cognitive makeup and just my natural tendency towards certain types of things. I've tried to allude to that. There's a very grand story, a grand tradition in all the Abrahamic religions, in the Bible, for example, in Genesis, in the Quran, also the story of how Adam came to be, the first sentient man. The story basically says that God cast Adam of clay and then breathed into him.

And as Adam was animated, God then asked the angels to bow down unto Adam and also Lucifer. And Lucifer refused. And God said, why do you refuse? And Lucifer said, well, because I'm made of the finest blue flame and Adam is but made of clay.

So God responded, yes, but to him, I have taught the names and the nature of all things, alluding to the fact that what made Adam great was not what he was made of the clay, but rather his cognitive capability, his capability to learn and go beyond the fixed form creation that is abundant in the universe. So even if you look at that as a parable, the interesting thing to take away from it is that in our view, in our religions or called them our fictions, whatever makes you comfortable, even the conception of God is one, the ultimate creator produces his ultimate creation. And what is that creation? That is creation that can create intern that can know intern that can acquire knowledge intern.

So in that very real sense, to me, artificial intelligence is the ultimate pursuit. If I am to be, if I'm to come anywhere near to my qualities per that allegorical story, then I must look to produce the way that that example sets for us and things that are thinking things, things that are perhaps synthetic in our language, but able to perceive and able to chart their own course and able to perhaps produce yet more things. That to me is the ultimate form of creation. So it's a noble pursuit in my mind, a very noble pursuit.

And the other aspect is more of an, like I said, an earthly and a tangible aspect, which is that artificial intelligence applies to so many different things. I love solving problems, real problems, and almost every problem I come up against, I find that there's an elegant artificial intelligence application that could address that. So that for me is actually very personally satisfying. So I suppose you could say at both levels, I'm a huge fan.

We were talking earlier and you mentioned AI as a mirror to understand ourselves. Tell me more about that. Well, the point that I was trying to make there was that, you know, in this age of AI, and I call this the AI century, AI will be the greatest conversation that we will have in the century. At some point in time, we will tire referring to it as AI, but the underlying automation that it will enable will become the driver for the rest of the century.

And, you know, as that happens, the question is, it will start to do more and more of what we do. And if it does more and more of what we do, does that somehow diminish us? You know, for the longest time, we have become so caught up in the economic labors that we perform that we've even named ourselves by last names that signify those labors, you know, Goldsmith and Farmer and so on and so forth. And to me, that's so interesting because certainly we are more than those economic labors.

But the way the mind works, you start to become one with what you do. I suppose it's a higher form of muscle memory. And so when you're told that you are no longer that you have a crisis of identity. And when you have that crisis of identity, you can do two things.

One, you can of course be in disarray. Two, you can sort of wonder whether the conclusion that caused you to go into this crisis is actually the correct conclusion that you're drawing. And in my case, I am not the economic labors that I perform. And this I saw in the mirror that I held up to myself in contemplating AI.

AI as a mirror showed this to me. So then what am I? What is unique about me? Because one could say, well, thinking is unique about a human being.

But maybe the other camp will tell you that computers will think too and they'll think faster. So in contemplating that, I came to the conclusion that it doesn't matter actually. There is value in thought of all types. There is value in perception of all types.

And I tried to explain this both from a scientific perspective as well as from the perspective of even Sufi philosophy, which essentially says the same thing. The idea is that if the landscape of ideas before you is infinite, if we can agree that what we have to discover yet is infinite, then the speed at which we traverse that landscape of ideas, that idea of escape is immaterial. Because you can have the world's fastest artificial intelligence roaring through that landscape and you can have the world's most innocent child slowly picking up one concept after another familiarizing himself with each and then moving on gently to the next. The percentage of the idea escape that both these entities will grasp, will view, will know is zero because no speed can deal with the idea of infinity.

So what matters now is not speed, but what matters in this kind of discussion is perspective. The value of a sentient human being is the ability to perceive from a unique perspective. That actually is fundamental real value. That is what we are.

And that value does not get robbed from us, taken from us, diminished in any way in the age of artificial intelligence. The question of course is that we haven't yet come to the point in our collective evolution as the human species. To think about these things and to say these economic systems that we had established prior to our technology progressing to where it has, these economic systems are now a parce. They are old.

They are decrepit. And in this new age, the new door to this new age that's about to open requires us to adjust these a little bit, to clean them up a little bit and perhaps to tear a few down and completely innovate in certain areas. And that becomes the scaffolding that guarantees fairness, that guarantees equality at least to a much greater degree than exists today while we all collectively benefit from the bounties of automation. We've been in this interesting space in AI for the past couple decades where the set of things that only humans can uniquely do, but not machines, keeps getting smaller.

And we've seen this particularly in the past few years with some advances in machine learning. And so it seems like every couple months, there's some new thing that we used to say well only humans can pipe poker or do some other tasks there. Machines keep knocking these down. Where's this going?

Are we on a path to someplace where machines can do everything that humans can do? Or do you think there's some kernel of human uniqueness that we're likely to remain? There's two examples that I'd like to give you here. Well, first of all, let's take diamonds.

Diamonds humans agree are valuable. For whatever reason, there's collective agreement that diamonds are valuable. And it is in the collective agreement of humanity and the scarcity of naturally sourced diamonds. These two things when they exist together, you can actually have an economy and you can have some economic rules apply to diamonds.

So diamonds are valuable because they're rare and because we all agree that they have value. The reality is that diamonds can also be produced in factories now and industrial diamonds perform all of the work that a diamond would be valuable for. Diamond is very hard. It's used in cutting tools and so on.

But industrial diamonds can do those things. Industrial diamonds are not as valuable as real diamonds. So even though we came up with the process of creating industrial diamonds that can do all the work of natural diamonds, natural diamonds still retain their place. Why?

Because we collectively continue to agree that they retain their place. Number one, and number two, if you think about our social structures, you think about what we consider to be a job. The idea of a job has evolved so drastically. Four hundred years ago, if we told somebody that the richest man in the land is one who is a bard, he sings songs.

People would look at that and say, what land have you come from, foreigner? Nobody would have believed it. But today, that is possible. And control of agricultural land, for example, is no longer the asset that it once used to be.

So those things shift and evolve some because of natural human need and evolution in our social systems and our technology, but others simply because of agreement. The fact that we all agree now that computer games are sports and they're called esports and tons of people get together and they play these games in what look like stadiums and they give away a million dollar rewards. It's a collective agreement. So the person that can play that game is suddenly valuable.

So are those existential needs to play that game at that advanced level in that stadium in South Korea? No. So what does that tell you? What that tells you is that we are at a point where we can use technology to basically fulfill our needs.

The question is one of distribution. And the rest of our time, we can adapt and use to pursue things that we simply like and many of us collectively agree have value. This is where we are today, but it's a very disorganized place to be because some of us understand this, others don't. National governments certainly don't.

And the underlying way to distribute the bounty of AI and autonomy doesn't yet exist. Even though in my view, and I feel it's very strongly and I'm not the first one, Buckminster Fuller said the same thing. And Stephen Hawking, God rest his soul. He said the same thing.

The idea is that we have enough to where everybody can be comfortable. The issue is one of intelligent distribution. And I am not arguing for socialism or communism here. I'm arguing for the running of an efficient machine.

I'm arguing for it in that context. There's a lot of fear and speculation about what automation might do to employment and wages. Having looked at the technology, what are your thoughts on the range of likely possible outcomes in the next couple of decades for how automation will affect the future of work? My view is that as time goes by almost every job that we think of and recognize today will go the way of the machines.

There will always be certain activities that we will want humans to perform. I will always want to know that this painting was made by a human being. And in looking at that artifact, I'm attempting to connect mentally with someone that I can understand at a subjective level. And that increases its value.

The story of someone died up in that artifact, even though it is an imprecisely drawn artifact. Because a photograph is a far more real rendition than an oil painting. But yet we have photographs. We have 10 ATP and 4K and now 8K imagery, which is much more accurate than Renoir could ever have achieved.

Renoir is in our museums and 8K TVs and 4K TVs sell a dime a dozen. I also want to talk about AI in the military context. You've written about this a lot. You talk about military applications of AI in your book.

You and former General John Allen have written on a concept of hyperwar, but you also talk about in your book. In your mind, what is hyperwar? And what does it mean for the future of war? So the application of artificial intelligence in war has many implications.

And one of them, which is the most major implication, is really a collapse in the Uda loop. And just for your listeners who might not be familiar with this term, the Observe Orient Decide Act loop. In military parlance, when you take an action, you have to go through a series of steps to observe, validate, direct yourself, and then disposition yourself towards the target and then maybe take an action. So that Uda loop takes time.

There are many steps that intervene from the observation to the action. With artificial intelligence, the latencies, the delays associated with human beings making those decisions in the Uda loop can be shrunk. And the question is, if they can be shrunk down to a level where the presence of a human in that loop in any way creates a disadvantage for the party that allows humans to participate in that loop, then fundamentally the more rapid Uda loop is creating an advantage where you have somewhat of a David on Goliath-like effect, where you have a small force, but you have precision and you can take a decision and get a strike in before the other party has any time to respond or to even know what the heck's going on. So this is very fundamentally dangerous because what this means is that smaller forces can overpower larger forces.

And in fact, another quality of artificial intelligence in the battlefield that we refer to as distributed concurrency and control essentially allows a large number of assets to coordinate in ways that human activated or controlled assets cannot do. So precise control to be able to apply force exactly at the point where it has the highest probability of achieving maximum effect. And then the principle of application of force, what you're attempting to do, if you cannot match the enemy's entire outlay of resource, you find the one place where you can best the enemy and you go and apply your force there. Conserve your force, concentrate your force, apply your force and then get through.

So artificial intelligence also provides the kind of mobility and coordination that would make this a very repeatable and almost guaranteed process. And then of course, the fact that with artificial intelligence, the more autonomy we bring to these systems, what's referred to again in military power nonsense, the teeth-to-tail ratio. Today, when you have human beings, you also have human needs. You have medics.

You have food. You have resources. You have extra clothes. You have all of these things that come with a human army.

If you can reduce that, not eliminate it, but even if you can reduce that, what you've done is automatically you've increased the teeth-to-tail ratio. The operative part, the combat-focused part of your force, as opposed to the logistics-dependent part of your force, has increased. And this, of course, also has many implications. I won't go through all of them, but fundamentally, our view is that hyperwar is not just about a revolution in military affairs.

It's actually a revolution in human affairs. And this goes back to our view that war is an expression of human character. There are many definitions of war, but fundamentally, if you look at, again, our allegorical history, war existed when the first four people showed up on the planet, and one of the four murdered the other. So it predates armies, it predates organizations, it is an expression of human character.

Our goal, of course, as human beings, should be to elevate the human character. But until we do, not recognizing the reality of who we are, would be foolish. And therefore, deterrence is incredibly important. Deterrence is what guarantees the stability that we've seen in the recent past.

So as artificial intelligence is applied to warfare, we must ensure that that deterrence continues to hold, that it is not just one party that has access to this greater technology, while the others assume the best and simply go off on their merry way. Deterrence in this new paradigm is of incredible importance, which is why we focus on this. We focus on this not to beat the drums of war, but to point attention to the fact that a new technology which will be applied, which is being applied to war, is coming, let us understand this structurally, deeply, and then let us not fall behind in creating the deterrent that would ensure peace in the future. And this is the history of man.

Somebody builds a bigger sword and then the other guy has to find a stronger shield. I want to get your thoughts, Amir, on AI risk. We've seen a number of really bright intellectual leaders argue that we should be concerned in a long line about risk associated with AI. There's of course a cartoonish sci-fi version of this, a machine's turning on us.

But you have folks like Bill Gates has warned that in your term, AI could have many benefits, but in the long term could have some risks. Stephen Hawking, who recently passed away, warned that machines might surpass humanity. These are not easy folks to dismiss. What are your thoughts about AI risk in the long term?

Let me latch onto one statement, machines surpassing humanity. Machines have already surpassed humanity in so many ways. Where they have not surpassed humanity is that their general purposeability in a single package to do all the things we do better than the way we do them has not yet been achieved. But there are many risks that will come our way even before that kind of AGI is achieved, because with artificial narrow intelligence, with AI that doesn't have self-awareness or doesn't have general purpose, solving capability, problem solving capability, but is better than a human being in a specific area, such as maintaining pumps or predicting utility grid failure or driving a car or driving a piloting an aircraft.

These are all jobs that are associated with vast amounts of salaries and wages and so on So that is happening. ANI is going to get there and that job displacement is coming. So one of the questions of course is that's a near-term risk, that's something that is quite likely. What are we doing about that?

Because I'd rather focus on problems that are happening sooner and then focus on problems that are happening later. Focusing on those later problems is also very important. However, in computer science, there is this concept of laziness. The way some programming languages work is that it turns out that you can write certain expressions in a way where certain parts of those expressions don't have to be evaluated till the very end.

If you're incredibly aggressive at evaluating those expressions, you might evaluate them too soon and then have to re-evaluate them over and over again. But some of these lazy languages actually end up evaluating only what is needed at the very last moment it's needed. Because by that time they have the most amount of information about what actually needs to be evaluated and they can speed up and conserve resources and so on. So this is actually a way of thinking.

In the case of risks, I think we should focus our attention on all types of risks, but the really horrendous types of risks that we don't know too much about and Nick Bostrom's excellent book, Super Intelligence covers a lot of these. Frankly, we don't have enough underlying information yet to really even validate some of those scenarios within the realm of reality. I'll give you one example. There is the famous Paperclip Maximizer scenario in Super Intelligence.

Can you explain it for a minute in case some of our listeners don't. You're not familiar with this particular horrifying AI scenario. Absolutely. So the idea is that we build an AI and we give it the role of optimizing the production of Paperclips in a factory.

And over time, the AI becomes more and more and more intelligent. And it says, well, my goal is to maximize the production of Paperclips. So it jumps up all the humans and turns them into molecules. And then it starts to consume the Earth and then it shifts its factories to the other planets.

And then it consumes them and suddenly the universe is nothing but Paperclips. Because the AI thought that that is what it was supposed to do. Now, the issue of course here is that there are to me, there is a dichotomy. On the one hand, the AI is so clever that the collective efforts of humanity could not foil its plans and its designs.

And on the other hand, it is. And so obviously, this AI would have read human literature and known about human behavior, read human history, read philosophy, read all these sorts of things. So it would be generally aware of who we are and what we do and what we pursue. And yet it would not realize having that level of intellect to foil all of humanity, it would not realize that the goal of building Paperclips of in that volume is entirely redundant and stupid.

So to me, that's not a likely scenario. Is it a nice thought experiment to get a point across about a fixed objective function? Sure. But should we start having nightmares about literally that scenario?

No, it's to me, there's contradictions in that scenario. So I think as we go along and as we solve the near term problems, we will learn a lot more useful specific stuff about the long term problems. And frankly, nobody will tell you, nobody that's responsible will tell you that AGI is three years away or two years away. AGI still requires quite a few problems to be solved.

And it is many years away. And in that time, yes, we should focus our attention, we should solve these problems. But with the greater knowledge that work on this problem during that course of time will arm us with. Last question, Amir.

How do you do it all? You've written this fabulous book on artificial intelligence. You are also running an AI company. Have you secretly already uploaded your mind to a machine and there are multiple amiures?

How do you make it happen? That's exactly right. You've got it. No, the book, the book took three years and I had lots of support from my wonderful publisher, Scribner.

They've been absolutely fantastic. My team at Spark cognition is wonderful. And in my life, I've found that when I really, really, really want to do something, it was never a good time. By the way, the book and my company, Spark cognition, weren't the only two things.

My wife had started a robotics company at the same time which she's running. And we decided that two startups weren't enough. So we went for the third one. My four year old boy, who's four now, also arrived at roughly the same time and the book.

So it's been a busy past four years. But like I said, you only have two options. You can keep going forward or you can turn around. And all I can tell you is so far I haven't turned around.

That's a lot going on. You're living life to us fullest. That's great. Thank you for coming in today, Amir.

Thank you very much for having me. And let me just close with Amir saying the sentient machine, the coming age of artificial intelligence by Scribner. Check it out. Excellent book.

Thanks.

Frequently asked questions

How long is this episode of CNAS Podcasts?

This episode is 30 minutes long.

When was this CNAS Podcasts episode published?

This episode was published on April 11, 2018.

Is there a transcript available for this episode?

Yes, a full transcript is available for this episode. You can read the complete transcript on the episode page.

Can I download this CNAS Podcasts episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.