The ethics of creating AI characters
Richard Bartle is a professor of computer science at Essex University in the United Kingdom, and he’s also a well-known online gaming expert. He believes that game developers should think about how to ethically create AI characters who are “sapient,” or have their own free will. He said game developers should view themselves as gods who are creating people, and that the ethical consequences of what they do matters.
For myself, I feel like how well we treat those virtual characters depends on how well they are done.
“For us, we want them to appear to have their own destiny,” he said. “Just as in Westworld, they program in a core motivator. Or core trauma. For realism’s sake, you need to have that. It will, of itself, I hope, change the way we treat the character.”
Saatchi said the ethical challenge reminds him of The Truman Show, the Jim Carey movie where Truman’s life is engineered by people who are watching him, and he doesn’t know that he’s being manipulated. Saatchi believes the virtual being should have some way to step out of a relationship, if necessary. He also believes the creators should use their authorial intent, rather than just make them mirrors of us, so that these are clearly works of art, rather than feats of technology.
“The memories form the character’s destiny,” Saatchi said. “It doesn’t so much mean free will and what are we forcing [a character] to do. It’s more about what memories have we implanted and whether you have their destiny already laid out for them.”
Saatchi believes that for characters to feel real, you need to creates memories for the characters. To design those memories, you need storytellers. You have to then design a story that takes into account what users can do with that character. You could have a character driven by traumas of the past that are inescapable. The character designer, then, is the one that limits the virtual being’s free will.
Saatchi said that it’s interesting to see discussions between the AI experts and the game designers, as Fable Studio has both of those kinds of people under one roof. Saatchi believes the AI needs to be subordinate to the storytelling, as in some ways AI isn’t advanced enough to be the complete star of the experience.
Peter Molyneux’s Black and White from 2001 is a good example of thinking about how players should or should not be allowed to treat virtual characters, Saatchi said. In that game, the player is a god, with subjects who are characters in the game. It raises a lot of questions. Are they property? Or are they people?
In this case, the developers hold sway on whether the players can allow themselves to distance themselves from the virtual being, like in Detroit: Become Human. Players may have the impulse to behave badly toward the virtual beings, just because they know it’s a game and being bad is an option they can pursue to unravel a whole different side of the game.
Some users might want to get to know game characters, the same way they want to get to know music celebrities through both their work at massive concerts as well as their day-to-day personalities as they appear on Instagram.
“You would figure out what your role is, just as I would figure out at a concert that I shouldn’t feel bad that Travis Scott isn’t talking to me,” Saatchi said. “You understand the context. The user needs to be trained in each circumstance which role to pretend to take on. That’s what gets us back to what we pretend to be. The mutual cosplay of it quickly takes over. You will know what to do.”
The next operating system
Over time, Saatchi believes we will go down the path of Her, the Joaquin Phoenix movie where a man falls in love with an operating system that takes the form of a female character, or Blade Runner 2049, with its own virtual character, Joi.
“When I look at Miquela, she is real to me,” Saatchi said.
The community has discussed issues such as trying to do a fake bank robbery, as once suggested by philosopher Jean Baudrillard, Saatchi said.
“You’ll quickly find out you are doing a real bank robbery,” he said, because of the way that people would react to your actions. “You’ll be arrested.”
“What that means to people in the community is that if you are able to take enough of the symbols of what you are pretending to be, like with Lil Miquela … it’s so similar to how a real person would do it that she is what she pretends to be,” Saatchi said. “She would pass through. I would encourage any game developer to study what Miquela has done. She is a great example of being what she pretends to be.”
Asked where he wants to see this go, Saatchi said, “My No. 1 vision and goal is to create the next operating system. It’s so boring. iOS 14 now. Mac and Windows updates. Alexa and Siri. It feels like they are the same as they were five years ago. I used to think the paradigm was going to be spatial computing, or AR and VR. Now I think the next paradigm will be a virtual being, a character who is our friend who helps us through life and helps us organize our digital world. That’s where I want to go.”
He added, “I think everyone is in very different ways unlocking a piece of the puzzle to for this to be the next paradigm of computing. A virtual being is a representative of all computers. We saw that in Her. We saw that in lots of movies. Ethically, it’s obvious that level of power is terrifying. If you think of Blade Runner 2049, the company that created Joi, the name of the virtual being, would be the Microsoft, Google, Amazon of that day. All of those combined. We would have to hope that consumers would reward companies that have invested their virtual beings with a moral sense, and that care about what their virtual beings are doing to people. You could do so much with addicting people to virtual beings. Ultimately, I have to have faith in the market forces that consumers will be able to detect the intentions of the people creating these virtual beings. Just as you get a vibe from a person, you also get a vibe from a virtual being.”
That brought us back to our beginning. We are what we pretend to be, so we must be careful about what we pretend to be.