Sony are experimenting with AI-powered characters who’ll speak to the player directly using OpenAI and Chat-GPT to hold a conversation, according to leaked footage seen by the Verge. The video shows familiar robot whacker Aloy from the Horizon games “responding to queries with an AI-powered synthesized voice and facial movements”. Sony’s copyright patrolmen showed up with a “‘ello ‘ello what have we ‘ere” and got the video taken down from YouTube. But, shhh, there are other ways to see it.
The footage is still posted on Streamable, for example. Skip to the three minute mark if you just want to see the Aloybot in action. It shows her answering some questions about herself and her origins, then entering the game world and responding to other questions as she moves around, like whether or not she can see enemies nearby, what time it is, what weapons she’s using, and so on. After that, they return to an “interview” setting and she summarises the past few moments.
The presentation is given by Sharwin Raghoebardajal, a software engineer who works on AI and face technology for an internal Sony body called the PlayStation Studios Advanced Technology Group. Raghoebardajal points out that this is just a prototype but confirms it is also operating within a specialised build of Horizon Forbidden West. Guerilla Games worked with the group to develop the demonstration, he says.
As for the technology behind it all, it’s a melange of different AI, sound, and animation tools. OpenAI’s Whisper is used to generate the speech-to-text, then language models GPT-4 and Llama 3 do their wibbly wobbly thing, before Sony’s own voice synthesis and face animation technology wraps it all up. The result is a number of robotic if somewhat accurate face movements, and an unsettling voice from the depths of tech bro hell. I don’t want it near me or my pets.
“We’ll not go into whether this is fun for games,” says Raghoebardajal, “whether it even makes sense to talk to Aloy when you’re her while playing the game… or where we’re going to get the memory from to run this all on a consumer console. This is just a glimpse of what is possible.”
After the Verge published their story on AI-loy, the video they posted was taken down via a claim from Muso, a copyright enforcement company who have Sony as a client. Hey, Sony, relax. It could have been worse. Remember when the Vatican’s AI chatbot was defrocked for telling people you could baptise children in Gatorade? That was fun.
If you haven’t played the Horizon games, they are open world action adventures about human tribes trying to survive in a world where humanity was almost wiped out. The collapse of civilisation was caused by artificial intelligence.