This is a great article that highlights some things people have been thinking about for a while (starting with Isaac Asimov), but haven't been much of an immediate concern until now: namely the nature of artificial intelligence and our relationship with it.
We could consider the emergence of a friendship like this a milestone, a bit like when chess AI got good enough to beat a typical club player, but wasn't quite ready to beat Kasparov. We're probably only a few years away (OK, maybe 10+, but it's not 50+) from the Kasparov point, where an AI like Siri can beat the turing test for any living human.
There are all sorts of questions like whether, to be considered alive and imbued with the inalienable rights that should come with sentience, is self-awareness necessary, or is the anthropomorphic perception of nearby humans more important? Should an AI have similar rights by proxy as a pet dog?
One especially important question is what does it mean for an AI to die? If Siri developed a fault, and fixing that fault would cause a change in personality so that it was no longer recognisable to Gus as his BFF, should that act be called "roboticide"? Such questions are particularly relevant when AI/ML systems (deep recurrent nets, for example) are so complex that we don't really understand them fully, so we have no way to surgically correct specific faults; all we can do is revert to an earlier state and re-train. That may be loss of life, for certain definitions of "life".
As an occasional game developer, I tend to think about these issues in other contexts too, such as our relationship with virtual characters in games. It's already very easy to get highly immersed in single-player virtual worlds, like any of the Elder Scrolls games. Most would not be fooled into thinking that an NPC is "alive", but it's certainly possible to develop emotional reactions to certain characters that we perhaps like because they say nice things about us, or dislike because we find then annoying, etc.
There are two kinds of character interaction a person can have in a game: NPCs and human avatars. As we start to build virtual worlds (partly spurred by the Oculus Rift and partly just because of the Internet), this could affect not only our relationship with NPCs, but with other humans too. I don't know whether this will a net positive or negative thing, but we're certainly going to learn a lot about human psychology as we head towards the point where, in VR, nobody knows you're a human.
Your comment doesn't seem to relate to the article.
where an AI like Siri can beat the turing test for any living human.
A key aspect of Siri in this article is that the responses make sense. There is no trickery or deception. When she says "Marriage is not included in my end-user agreement," there is no way someone could mistake her a human, and that's a good thing. Here, the more relevant philosophical questions are ones like "How much can machines or machine intelligences be a part of our lives and our society as machines?" and "Are there more beneficial and interesting goals for our machine intelligences than the Turing test (given that their value may be predicated on the fact that they are not humans)?"
This issue is commonly considered to be far-off or irrelevant due to the difficulty of developing strong AI, but I think that the reasoning behind that judgement is faulty.
We are unlikely to develop strong AI any time soon, but it is far easier to create the convincing impression of sentience. Humans are naturally prone to pareidolia and the pathetic fallacy - we see faces in potato chips, we convince ourselves that a faulty photocopier hates us, we attribute personality traits to automobiles.
Looking back to the 1960s, many people experienced strong emotions when interacting with ELIZA, a program that is laughably simple by modern standards. Today, there are numerous accounts of soldiers in Iraq and Afghanistan holding funerals for bomb disposal robots and decorating them with honorary medals.
Machines don't need to be as intelligent as humans for us to begin forming deep attachments to them, they don't need to pass the Turing test, they don't even need to match the intelligence of a cat or dog - they only need to appear plausibly sentient, which I think is a far lower bar than most of us realise.
Personally, I think we're on the cusp of a profound change in how we interact with computers. We'll see it first in cases like these, people with limited capacity for reasoning - children, the learning disabled, elderly people suffering from cognitive decline. Machines that aren't smart enough to fool a skeptical observer, but that represent a perfectly satisfactory stand-in for Rex or Tiddles. The natural human tendency to see patterns and project ourselves onto the world will meet the machines in the middle, glossing over their deficiencies.
We'll dismiss it at first, framing it as the slightly pitiable self-delusions of the vulnerable and lonely. Change will come imperceptibly gradually, creeping in from the edges as the technology improves and filters out into the world. We won't really notice that society is being reshaped, just as there was no shocking moment of realisation that a whole generation had grown up with ubiquitous access to communication technology. We'll just have a vague "huh?" moment, like the first time you were sat on a train and realised that nearly everyone on board was staring at a small glowing rectangle. Huh, my elderly mother spends most of her time talking to a bot. Huh, my child's best friend is a bot. Huh, these students are passing around a petition for bot rights. Huh.
I don't know if you've played Shadow of Mordor, but even as an occasional game developer like yourself I was able to forget about the technology running the Nemesis System at times and treat the Orc Chiefs as though they had intelligence and history. You can easily imagine how that system works when you take a step back, but if something even at that stage can trick us I have high hopes for the next few years of AI / NPC programming in the next few years.
On a related topic, a lot of money has gone into graphics programming in the last few years as studios developed their own engines in anticipation of the now-current generation of consoles. I am hopeful that with virtual reality on the horizon we'll see improvements in AI and Physics, among other tech necessary to create a convincing virtual world.
I wonder if governments will try to mass manipulate us by forcing AIs to be somewhat political, and be that "friend" that tells us we should vote a certain way.
Considering that every single of these personal assistants comes from private companies, ads are much more likely. I also like that political is a dirty word now.
Not to mention what you "fear" is already true, it's called Rush Limbaugh and Fox News.
I think the word is acquiring dirt from that acts; the slimier and more partisan the political conversation is perceived to be by people, the less they want to associate with it in any context.
When phones' personal assistants (every company seems to want to have theirs these days) become orders of magnitude more advanced, we may be able to tell them to "check on our vacation booking" and they will know to search through our emails and calendars and connect the dots, so in those tasks they will be more advanced. There will probably be more services tied more deeply into them- for example, you'll be able to say "get me an Uber" and they will reply with "A blue civic will be here in 3 minutes", or "fill up the fridge on Sunday morning" and an Instacart-like company will show up at your door Sunday morning with eggs, vegetables (but no milk because you didn't finish this week's).
However, we won't be able to have those deep meaningful conversations with our voice assistants, because they won't have the necessary life experience to follow meaningful conversations. The current products on the market have no parents, no age, no experiences in school, no previous job, no former lovers, etc (they will jump around those questions playfully, but the canned answers get old quite fast). Those trite details that we recount when we bond with people do not exist in current personal assistants and likely never will.
There are several reasons for this: the first being that being able to model a structure of "life experiences" that can be queried based on what the user is saying is an incredibly complex problem on which we have pretty much no angle of attack on.
12 year old child: "I got picked on at school today."
Digital personal assistant: "You know, it happened to me too when I was your age. Let's talk to your mom about it."
Or even more complex:
24 year old student: "Hey, remember that problem on the Trigonometry 402 final from my senior year that I asked you to solve the equations for back in college?"
Digital assistant: "Oh yes! Here it is."
This, happening for millions and millions of possible interactions, life experiences, contexts? That's the algorithmic equivalent of light speed travel. Who knows maybe we'll get here one day. But 10 years from now? 50 years from now? 100 years from now? Not a chance. Remember that 40 years ago, Minsky thought it'd take a bunch of grad students a summer to write a program that recognize objects in pictures.
The second reason that such systems are extremely unlikely to emerge is that even if it would be doable technically, it would be extremely expensive to a company. And there would be literally no demand for it, because the overwhelming majority of people don't care about talking to robots. They already don't have enough time to spend with their children, partners, friends, parents... why would anyone waste time with a fake person? No company would invest the billions, if not hundreds of billions of dollars, to solve this problem in the next few hundred years.
Those questions of sentience and whether powering off a digital voice is "killing it" are appealing to ask to those of us who grew up reading Isaac Asimov, because we want this future to exist so bad. But they are red herrings- those questions have no meaningful answers, because our society is not configured in a way in which those questions could actually arise. When you turn your phone off today, the personal assistant definitely doesn't "die"; and in 50 years, even if it can carry out tasks way more efficiently and give somewhat more "human sounding" answers to certain categories of questions, people will still have no problem turning it off.
The second reason that such systems are extremely unlikely to emerge is that even if it would be doable technically, it would be extremely expensive to a company. And there would be literally no demand for it, because the overwhelming majority of people don't care about talking to robots. They already don't have enough time to spend with their children, partners, friends, parents... why would anyone waste time with a fake person? No company would invest the billions, if not hundreds of billions of dollars, to solve this problem in the next few hundred years.
You've missed the point. Why would people watch a TV show and grow to love/hate characters, where there's no interaction between you and them, and all the lines, actions and scenes are scripted and rehearsed. They certainly don't have time to do it, and have children, partners, friends, parents. They would surely not binge watch, schedule time to watch new episodes come rain hail or shine or spend big bucks to visit filming locations etc.
Companies will invest stupid amounts of money in AI that has human qualities. Take the TV Show example: the AI is the "show" and each day they get into entertaining or emotional circumstances that you can "catch up" about, joke about etc. Their charisma is highly engineered to be incredibly engaging and fun to talk to, so you keep coming back for more. Over time they become a "good friend" that you love talking to. They never rebuff you, snub you, have no time for you, make fun of you (too much, just enough to joke around and have some banter). But why do they keep telling you how great Pepsi's new flavours are??
Because the TV show has no bugs, no "I'm sorry, I didn't understand why you said, rephrase" in the middle of your rant against whatever, and creates the same experience for everyone who watches it, so that people can discuss amongst themselves.
Not in the middle of a rant, or while you are professing your love for someone. In neutral discourse, sure that's expected.
You can even not speak the same language, humans can understand each other by other means like body language and cultural cues. It's not something you can replicate in a machine unless you really understand how it works. Unless you are saying that human behaviour is completely understood...
Not in the middle of a rant, or while you are professing your love for someone.
It's possible to detect mode of speech even without visual cues. Factors such as pitch dynamics, speed of talk, etc. can be accounted for. Software can be trained to be even more sensitive to these signals than we are.
Of course, the problem is that this varies somewhat among different people. Therefore, part of AI training would need to happen with actual customer after purchase. There're a lot of unknowns in this process from manufacturer's point of view, so I can understand why it's not happening yet.
Once a “special” mode of speech is detected, though, it's simple to avoid canned “Please rephrase” response in case of unclear voice input. Instead the software would change its own mode of speech appropriately, and then evade the direct reply—humans do this all the time in real-life communication.
And there would be literally no demand for it, because the overwhelming majority of people don't care about talking to robots. They already don't have enough time to spend with their children, partners, friends, parents... why would anyone waste time with a fake person?
People don't care about talking with robots because robots aren't able to hold conversations today.
A "fake person" is an advantage - it doesn't have needs. It won't manipulate you, it won't think less of you, it won't get angry at you, it won't feel bad because you did something, it has perfect memory and unquestioning loyalty.
It's always available. Best of all - if you want it to have the reverse of the above properties, you can always switch it on. Want an arrogant robotic friend? Just go to the settings.
I'm pretty sure most people would prefer robots to humans. Obviously, almost nobody would admit this today.
>24 year old student: "Hey, remember that problem on the Trigonometry 402 final from my senior year that I asked you to solve the equations for back in college?" Digital assistant: "Oh yes! Here it is."
Maybe I'm just being naive from my layman's perspective, but that did not seem that far ahead. It's fairly objective to pinpoint "Trigonometry 402 final from my senior year" in time, and, if the same AI was already there, there's no reason for it not to remember, unless we're systematically deleting stuff to save space.
I think the point is that differentiating between the first and the second is very difficult to do. And communicating to someone not technically inclined why it can respond to question A and not question B would be very frustrating.
Maybe they will be similar like the replicants in blade runner and each have a scripted past instilled into them. As my sibling comment states, its not terribly unlike TV shows.
Comments
This is a great article that highlights some things people have been thinking about for a while (starting with Isaac Asimov), but haven't been much of an immediate concern until now: namely the nature of artificial intelligence and our relationship with it.
We could consider the emergence of a friendship like this a milestone, a bit like when chess AI got good enough to beat a typical club player, but wasn't quite ready to beat Kasparov. We're probably only a few years away (OK, maybe 10+, but it's not 50+) from the Kasparov point, where an AI like Siri can beat the turing test for any living human.
There are all sorts of questions like whether, to be considered alive and imbued with the inalienable rights that should come with sentience, is self-awareness necessary, or is the anthropomorphic perception of nearby humans more important? Should an AI have similar rights by proxy as a pet dog?
One especially important question is what does it mean for an AI to die? If Siri developed a fault, and fixing that fault would cause a change in personality so that it was no longer recognisable to Gus as his BFF, should that act be called "roboticide"? Such questions are particularly relevant when AI/ML systems (deep recurrent nets, for example) are so complex that we don't really understand them fully, so we have no way to surgically correct specific faults; all we can do is revert to an earlier state and re-train. That may be loss of life, for certain definitions of "life".
As an occasional game developer, I tend to think about these issues in other contexts too, such as our relationship with virtual characters in games. It's already very easy to get highly immersed in single-player virtual worlds, like any of the Elder Scrolls games. Most would not be fooled into thinking that an NPC is "alive", but it's certainly possible to develop emotional reactions to certain characters that we perhaps like because they say nice things about us, or dislike because we find then annoying, etc.
There are two kinds of character interaction a person can have in a game: NPCs and human avatars. As we start to build virtual worlds (partly spurred by the Oculus Rift and partly just because of the Internet), this could affect not only our relationship with NPCs, but with other humans too. I don't know whether this will a net positive or negative thing, but we're certainly going to learn a lot about human psychology as we head towards the point where, in VR, nobody knows you're a human.
Your comment doesn't seem to relate to the article.
A key aspect of Siri in this article is that the responses make sense. There is no trickery or deception. When she says "Marriage is not included in my end-user agreement," there is no way someone could mistake her a human, and that's a good thing. Here, the more relevant philosophical questions are ones like "How much can machines or machine intelligences be a part of our lives and our society as machines?" and "Are there more beneficial and interesting goals for our machine intelligences than the Turing test (given that their value may be predicated on the fact that they are not humans)?"
That's one of the issue explored in the Swedish TV series "real humans" http://en.wikipedia.org/wiki/Real_Humans
I highly recommend this series. It explores the issues of AI in a deeper and smarter way than the usual hollywood movies, and it's very funny too.
Wow, Siri is Robbie from 'I, Robot'; now in the story the robot could not talk, now it's a phone instead, but it does talk.
i found the story online: http://www.angelfire.com/blog2/endovelico/IsaakAsimov-IRobot...
This issue is commonly considered to be far-off or irrelevant due to the difficulty of developing strong AI, but I think that the reasoning behind that judgement is faulty.
We are unlikely to develop strong AI any time soon, but it is far easier to create the convincing impression of sentience. Humans are naturally prone to pareidolia and the pathetic fallacy - we see faces in potato chips, we convince ourselves that a faulty photocopier hates us, we attribute personality traits to automobiles.
Looking back to the 1960s, many people experienced strong emotions when interacting with ELIZA, a program that is laughably simple by modern standards. Today, there are numerous accounts of soldiers in Iraq and Afghanistan holding funerals for bomb disposal robots and decorating them with honorary medals.
Machines don't need to be as intelligent as humans for us to begin forming deep attachments to them, they don't need to pass the Turing test, they don't even need to match the intelligence of a cat or dog - they only need to appear plausibly sentient, which I think is a far lower bar than most of us realise.
Personally, I think we're on the cusp of a profound change in how we interact with computers. We'll see it first in cases like these, people with limited capacity for reasoning - children, the learning disabled, elderly people suffering from cognitive decline. Machines that aren't smart enough to fool a skeptical observer, but that represent a perfectly satisfactory stand-in for Rex or Tiddles. The natural human tendency to see patterns and project ourselves onto the world will meet the machines in the middle, glossing over their deficiencies.
We'll dismiss it at first, framing it as the slightly pitiable self-delusions of the vulnerable and lonely. Change will come imperceptibly gradually, creeping in from the edges as the technology improves and filters out into the world. We won't really notice that society is being reshaped, just as there was no shocking moment of realisation that a whole generation had grown up with ubiquitous access to communication technology. We'll just have a vague "huh?" moment, like the first time you were sat on a train and realised that nearly everyone on board was staring at a small glowing rectangle. Huh, my elderly mother spends most of her time talking to a bot. Huh, my child's best friend is a bot. Huh, these students are passing around a petition for bot rights. Huh.
I don't know if you've played Shadow of Mordor, but even as an occasional game developer like yourself I was able to forget about the technology running the Nemesis System at times and treat the Orc Chiefs as though they had intelligence and history. You can easily imagine how that system works when you take a step back, but if something even at that stage can trick us I have high hopes for the next few years of AI / NPC programming in the next few years.
On a related topic, a lot of money has gone into graphics programming in the last few years as studios developed their own engines in anticipation of the now-current generation of consoles. I am hopeful that with virtual reality on the horizon we'll see improvements in AI and Physics, among other tech necessary to create a convincing virtual world.
"Should an AI have similar rights by proxy as a pet dog?"
I think so, when valued in the same way by a human. This is no different to https://news.ycombinator.com/item?id=8465088
I wonder if governments will try to mass manipulate us by forcing AIs to be somewhat political, and be that "friend" that tells us we should vote a certain way.
Considering that every single of these personal assistants comes from private companies, ads are much more likely. I also like that political is a dirty word now.
Not to mention what you "fear" is already true, it's called Rush Limbaugh and Fox News.
"I also like that political is a dirty word now"
I think the word is acquiring dirt from that acts; the slimier and more partisan the political conversation is perceived to be by people, the less they want to associate with it in any context.
When phones' personal assistants (every company seems to want to have theirs these days) become orders of magnitude more advanced, we may be able to tell them to "check on our vacation booking" and they will know to search through our emails and calendars and connect the dots, so in those tasks they will be more advanced. There will probably be more services tied more deeply into them- for example, you'll be able to say "get me an Uber" and they will reply with "A blue civic will be here in 3 minutes", or "fill up the fridge on Sunday morning" and an Instacart-like company will show up at your door Sunday morning with eggs, vegetables (but no milk because you didn't finish this week's).
However, we won't be able to have those deep meaningful conversations with our voice assistants, because they won't have the necessary life experience to follow meaningful conversations. The current products on the market have no parents, no age, no experiences in school, no previous job, no former lovers, etc (they will jump around those questions playfully, but the canned answers get old quite fast). Those trite details that we recount when we bond with people do not exist in current personal assistants and likely never will.
There are several reasons for this: the first being that being able to model a structure of "life experiences" that can be queried based on what the user is saying is an incredibly complex problem on which we have pretty much no angle of attack on.
12 year old child: "I got picked on at school today." Digital personal assistant: "You know, it happened to me too when I was your age. Let's talk to your mom about it."
Or even more complex:
24 year old student: "Hey, remember that problem on the Trigonometry 402 final from my senior year that I asked you to solve the equations for back in college?" Digital assistant: "Oh yes! Here it is."
This, happening for millions and millions of possible interactions, life experiences, contexts? That's the algorithmic equivalent of light speed travel. Who knows maybe we'll get here one day. But 10 years from now? 50 years from now? 100 years from now? Not a chance. Remember that 40 years ago, Minsky thought it'd take a bunch of grad students a summer to write a program that recognize objects in pictures.
The second reason that such systems are extremely unlikely to emerge is that even if it would be doable technically, it would be extremely expensive to a company. And there would be literally no demand for it, because the overwhelming majority of people don't care about talking to robots. They already don't have enough time to spend with their children, partners, friends, parents... why would anyone waste time with a fake person? No company would invest the billions, if not hundreds of billions of dollars, to solve this problem in the next few hundred years.
Those questions of sentience and whether powering off a digital voice is "killing it" are appealing to ask to those of us who grew up reading Isaac Asimov, because we want this future to exist so bad. But they are red herrings- those questions have no meaningful answers, because our society is not configured in a way in which those questions could actually arise. When you turn your phone off today, the personal assistant definitely doesn't "die"; and in 50 years, even if it can carry out tasks way more efficiently and give somewhat more "human sounding" answers to certain categories of questions, people will still have no problem turning it off.
You've missed the point. Why would people watch a TV show and grow to love/hate characters, where there's no interaction between you and them, and all the lines, actions and scenes are scripted and rehearsed. They certainly don't have time to do it, and have children, partners, friends, parents. They would surely not binge watch, schedule time to watch new episodes come rain hail or shine or spend big bucks to visit filming locations etc.
Companies will invest stupid amounts of money in AI that has human qualities. Take the TV Show example: the AI is the "show" and each day they get into entertaining or emotional circumstances that you can "catch up" about, joke about etc. Their charisma is highly engineered to be incredibly engaging and fun to talk to, so you keep coming back for more. Over time they become a "good friend" that you love talking to. They never rebuff you, snub you, have no time for you, make fun of you (too much, just enough to joke around and have some banter). But why do they keep telling you how great Pepsi's new flavours are??
Because the TV show has no bugs, no "I'm sorry, I didn't understand why you said, rephrase" in the middle of your rant against whatever, and creates the same experience for everyone who watches it, so that people can discuss amongst themselves.
Slightly OT, but I'd be tempted to argue TV Shows can and do have bugs; plot holes, bad acting, continuity errors et al. They can be quite jarring.
That's a good point. At least they have less bugs than Siri.
Nevermind that for much of the history of TV, and still today, it randomly cuts out?
Heck, I sometimes say "I don't understand, can you explain?" Why would people be less tolerant of that from a robot than from me?
Not in the middle of a rant, or while you are professing your love for someone. In neutral discourse, sure that's expected.
You can even not speak the same language, humans can understand each other by other means like body language and cultural cues. It's not something you can replicate in a machine unless you really understand how it works. Unless you are saying that human behaviour is completely understood...
It's possible to detect mode of speech even without visual cues. Factors such as pitch dynamics, speed of talk, etc. can be accounted for. Software can be trained to be even more sensitive to these signals than we are.
Of course, the problem is that this varies somewhat among different people. Therefore, part of AI training would need to happen with actual customer after purchase. There're a lot of unknowns in this process from manufacturer's point of view, so I can understand why it's not happening yet.
Once a “special” mode of speech is detected, though, it's simple to avoid canned “Please rephrase” response in case of unclear voice input. Instead the software would change its own mode of speech appropriately, and then evade the direct reply—humans do this all the time in real-life communication.
People don't care about talking with robots because robots aren't able to hold conversations today. A "fake person" is an advantage - it doesn't have needs. It won't manipulate you, it won't think less of you, it won't get angry at you, it won't feel bad because you did something, it has perfect memory and unquestioning loyalty. It's always available. Best of all - if you want it to have the reverse of the above properties, you can always switch it on. Want an arrogant robotic friend? Just go to the settings.
I'm pretty sure most people would prefer robots to humans. Obviously, almost nobody would admit this today.
Maybe I'm just being naive from my layman's perspective, but that did not seem that far ahead. It's fairly objective to pinpoint "Trigonometry 402 final from my senior year" in time, and, if the same AI was already there, there's no reason for it not to remember, unless we're systematically deleting stuff to save space.
I think the point is that differentiating between the first and the second is very difficult to do. And communicating to someone not technically inclined why it can respond to question A and not question B would be very frustrating.
Maybe they will be similar like the replicants in blade runner and each have a scripted past instilled into them. As my sibling comment states, its not terribly unlike TV shows.