SpiritAI: Talking Heads

For many years, when artificial intelligence came up in conversations about game development, people might have been talking about the likes of path-finding, navigation and combat.

But something exciting is going on that could bring the AI of science fiction into contemporary game design. NPCs could be granted the power to speak their artificial minds, and offer community managers a powerful weapon in their anti-harassment arsenal.

Making game characters communicate more convincingly might not seem directly related to seeing off unpleasant player behaviour in online worlds, but that is where the SpiritAI team excel in their craft; artificial intelligence and its ability to communicate naturally.

“Really I see our mission as making games a more engaging and also more accessible, through AI,” says Mitu Khandaker-Kokoris, SpiritAI’s chief creative officer and renowned game designer. “That lies at the core of both the products we have underway at the moment. And then broader than that, and at a top level, we’re trying to make AI more humanistic.”

Those products are Character Engine, which hopes to provide game designers and others with ways to write characters that will converse naturally and off script, and Ally, which delivers a fascinating means to face off the challenge of online abuse. Both technologies, shown at GDC this year, are certainly distinct. And they could have a profound impact on how games in the near future are played and managed.

One of the things I feel games have never done is social physics. There are so many gameplay mechanics there

Mitu Khandaker-Kokoris 

CHARACTER BUILDING WORK

Character Engine offers a combination of an SDK and authoring tool, the latter of which lets designers and writers sketch out the personality, preferences and knowledge of NPCs and other in-game beings, which can then converse freely, and convincingly, with players via voice input, text chat, controller inputs, or other means.

As Khandaker-Kokoris fires up a demo, created in partnership with UK studio Bossa, it’s immediately obvious that the technology has potential. The demo title, Interrogation, lets its player cross-question a robot, which sits in a dimly lit room. Using natural, typed English, questions must be asked to extract information from the robot, which replies with a spritely, generally smooth Scottish accent.

This is not just a robot that replies relevantly. Depending on the player tone and line of questioning, this is an NPC that can be helpful, direct, evasive, irritated or considerate.

“We want to let game NPCs respond in all these wonderful, dynamic ways, using AI,” Khandaker-Kokoris reveals of the effort with the Character Engine. “Really the power there is game designers’ ability to let the player express themselves, and have expressive input that results in expressive output. That means a system that really understands what you are saying, and having a character that can decide what to do as a result of that, where that character can even steer you in one direction.”

The result are NPCs that not only know what to say, but equally know when ‘she’ is appropriate, or if a first name is best needed for conversational clarity. They’ll certainly know exactly when they’re being less than crystal clear and you’ll only have your own social skills to blame when an NPC outwits you in conversation.

EXPRESS YOURSELF

It won’t just be what players say that impacts the artificial thinking of Character Engine powered entities. “We want more than that,” Khandaker- Kokoris asserts. “We want to have what players are doing and how they stand relative to NPCs influence things in the scene, whether in a traditional game environment or something like room-scale VR.”

Player body language, eye-line and attention may soon be factors to consider when pulling narrative from an NPC. The impact on games could be vast.

“What this means for the player is being able to stop thinking of NPCs as these very static things that dispense set stories,” Khandaker-Kokoris suggests. “This could see interesting, nuanced interactions with NPCs, and interactions that will never be the same for different players. And in this age of streaming, the lack of dynamism – or real dynamism – is strikingly apparent.

“One of the things I feel games have never really done is this idea of social physics, where players have to really think about communicating with other beings, and where or how they converse or interact – maybe using shorter sentences – will have a real impact, just as it does in real life,” Khandaker-Kokoris continues. “There are so many gameplay mechanics there.”

WRITING THE UNSCRIPTED

All that technology and potential, however, would mean very little if it wasn’t accessible to those that shape characters and stories in game worlds. Those creatives aren’t always coders, and rarely have they been expected to master the engineering of artificial intelligence. Fortunately, that consideration is a foundation stone of the development of Character Engine, and in particular its authoring tool.

“What we’re really interested in doing is adding more depth to characters,” Khandaker-Kokoris explains. “So this idea of ‘cognitive AI’ is actually giving characters motivations, their own agendas, and things they want to talk about, or want to convey to you. But we don’t want to take away control from the creators and the writers. This is still a creative tool. It’s not about instantly automated characters that are just released to do whatever they want.

“You still have to architect them according to your creative vision. That’s why with Character Engine what we effectively sell is an SDK and an authoring tool.” That tool looks similar to standard scripting software, but never demands users to write long, complex branching narratives. 

“Writers can set social practices within their characters. Very specific ways in which we’d expect them to behave, or expect them not to behave,” says Khandaker-Kokoris. “In the real world, for example, if I say hello to you, I’d likely expect you to say ‘hello’ back. If you didn’t I might be a bit miffed. So game writers can set what an NPC expects from conversation, or how it responds, and play with that. You can, using our authoring tool, sketch out an example of how a character might expect a certain behaviour from players, and behave in a certain way if you don’t meet their expectation of certain social practices.”

So if you’re writing a character that has a thing about manners, it is entirely possible to set that they find words like ‘please’ and ‘thank you’ important. But you won’t have to write their every response to every possible etiquette slip-up.

“The authoring tool gathers lots of conversational snippets, using them as examples,” Khandaker-Kokoris details. “So we can write questions the player might ask ‘did you witness the murder?’, for example. What the system will then do is automatically register that as an example of a clarifying question to do with particular keywords.

“What that means is players don’t have to ask questions exactly as scripted in the examples. They ask something else around the same area, and depending on their question, they’ll get some or all of the information as set out in the example. There’s lots of variation in how questions and dialogue are handled, because we’ve been doing this natural language classification, using all the classifiers that we’ve trained up to recognise what a clarifying question looks like, or what a question about identity is, or how farewells typically look. That’s thanks to the power of machine learning.”

THE WRITE APPROACH

This isn’t just a tool conceived to compliment the existing experience of writers. More than that, the wider approach deliberately echoes the way people have been shaping stories and characters for centuries. “The way you build up a personality in Character Engine is really the same way any writer would build up a character and conceive that character’s personality,” says Khandaker-Kokoris. “In many ways, a big part of the personality of a human is how that react and respond in any given situation. How do people respond to conversation? It’s something we all think about, because it’s how we present ourselves to the world. It’s a big part of how people traditionally build up characters and write for them. And it’s how you can approach Character Engine.”

The tool offers ways to set and confirm what NPCs know, what their allegiances are, and how they relate to various entities – from individuals to entire institutions or companies. There are various ways to help writers keep track of the characters and world they are establishing. 

“We’ve got tonnes of visualisations for writers, which we’re adding,” Khandaker-Kokoris enthuses. “At our GDC demo, in a room full of narrative designers, people really nerded out over it.

“And those kinds of visualisations are super useful for writers, especially if they’re writing in this new way, building up who characters are and how they might react to a given situation. Whether you’re a big triple-A team with a number of dedicated writers, those writers can use the authoring tool, while the programmers can integrate our SDK into their game engine.

“But if you’re a one-person team or a small indie, you can use the Character Engine tool, and easily integrate it into Unity. We wanted to be as flexible as possible and as extendable as possible, and build a very accessible and engaging to use tool that lets people bring this kind of AI into their games. Hopefully that will result in accessible and engaging games.”

A FRIEND OF COMMUNITY

Of course, Character Engine’s close relative is a rather different beast in terms of its aims. But Ally’s community management powers pull on exactly the same ability to harness and focus social physics and cognitive AI, as detailed ing the Community Matters panel.

Currently, SpiritAI is working with several game developer partners, while working on serving other industries and creative specialties also in need of natural, meaningful conversation between human and computer. Robotics makes for an obvious market, but there’s also a space for Character Engine’s abilities in the worlds of training, medical care and education.

“The idea of assistance bots has become a very real idea now, and using our tech there could be very powerful,” Khandaker-Kokoris offers as an example. “Now we have all these things like Alexa and Siri, which are great for short question and answer interactions, but they have no personality behind them. That’s not what the future of assistance AI should be. We want – again, going back to our original mission of building much more humanistic AI – to help people building these other technologies or working in other fields to build characters that actually have personality, and actually can work according to a particular agenda.”

Exactly what that means for video games is as uncertain as it is exciting. There’s much talk of new game genres as Khandaker-Kokoris packs away the demo, and even a little musing on what this might mean for the role of voice acting in games.

“Bossa were keen to answer the question – as Imre Jelle [co-founder and creator-in-chief] there puts it – ‘what is the walking simulator of conversations?’.” It’s a striking question, as are most of those that centre around Character Engine’s potential to reinvent the conventions of gameplay.