Aural nirvana: Bringing outstanding sound to games

Game audio has come a long way since the early days of Super Mario, where much of the same music was condensed to provide extra sound effects.

Enormous musical scores are now provided on the scale of film, and sound is fast becoming a near central part of all gaming.

Take our Develop Awards finalists. Simogo’s text-based adventure title Device 6 heavily relies on its immersive audio to engage the player with the story and ratchet up the tension during key parts of the narrative.

And the bigger titles like Battlefield 4 and Total War: Rome II rely on realistic sound effects to make the player actually feel like they’re in a warzone, and not just playing a game.

“To me sound is as an important factor for feedback as anything visual,” says Simogo founder Simon Flesser.

“I can’t for my life understand people that say they play games with sound turned off. To me, that’d be just as absurd as turning off the screen and playing with sound only.”

Develop Award winner, audio veteran and the man behind Monument Valley’s sounds, Stafford Bawler, sums up the importance of such effects by offering an example from a large scale project he’d previously worked on.

“I’d just delivered the first proper audio build to the game, replacing all the early tests/placeholder with first pass final audio that represented where the game was going in terms of audio,” he reminisces. “On hearing this for the first time the game’s lead designer was speaking with the audio coder and said ‘It feels like a proper game now, we’re no longer just making builds’.”

Sharpening sounds

New hardware for both mobile and console, as well as a number of constantly refreshed audio technologies such as FMOD and wwise – which recently opened up new and free indie licences – are also helping studios record and implement new styles of audio.

“In previous generations, a limited set of additional tools was available, and it was more restrictive to set up multiple instances due to technological shackles,” says Creative Assembly and Total War senior audio designer Matt McCamley. “As games tech matures, so does in-game audio, chiefly in the way we can deliver an experience to the player.”

One of Creative Assembly’s hotly anticipated titles, Alien: Isolation, is a prime example of how key audio can be in games. The team has had to adopt various new and unique techniques to evoke the original film’s haunting atmosphere and turn it up to ten to keep players on the edge of their seats for its 15-hour or so duration. And from early footage, the game just wouldn’t be the same without sound.

The team’s sound designer Sam Cooper says the game’s audio has been crafted to subtly conjure up dark and scary imagery on even seemingly mundane objects, tapping into the player’s subconscious, in-built fight-or flight responses. He explains this creates an unpredictable soundscape that will keep users second-guessing what their ears are telling them, moving the experience beyond just twitch-play and how good they are at games.

And much of this has been made possible thanks to new consoles and tools.

“With additional processing power and frequently updated middleware, we’re now able to use rich convolution reverbs on new-gen consoles and experiment with emerging DSP such as in-game HRTF/binaural processing,” says Cooper. “We’re far less limited by the hardware with new consoles.”

Another CA sound designer, Total War audio manager Richard Beddow, says Rome II marked the first time the studio had used audio engine wwise, as it offered tools for sound designers in terms of how to play back assets assigned to game events, asset containers and shared resources such as DSP settings. This meant that designers could set up new items, previously the realm of coders, thanks to its visual GUI, creating what Beddow says was a more efficient workflow.

Bawler recommends other tools, such as Unity and Fabric, which he says lent themselves to the way he likes to work, creating ordered hierarchies and the careful management of data trees, allowing him to spend more time on the creative elements of audio creation.

Mobile beats

Much like Creative Assembly is experimenting with sound implementations in its big budget console and PC outings, mobile is also home to some interesting techniques.

Simogo’s Flesser had the tough task of integrating audio, along with composer Daniel Olsén, in interactive story Device 6. The audio had to match the reader’s interpretation, while also not losing their attention by playing sounds at the wrong time.

“We wanted to keep a lot of the things up to the players’ imagination, so it was a tough balance to decide what things that are mentioned in the text should be audible or not,” he explains.

“Creating a sense of space was our biggest priority. So working with footsteps to communicate different materials, the right amount of reverb on sounds to communicate size of rooms, mixing things properly, fiddling a lot with EQ on every sound to make everything feel just right within the space that the player was in at the moment.”

Audio constraints

Time and budget constraints can of course play a big role in the quality of audio and its variety, particularly if designers are required to go to the source, but mobile in particular throws up a unique set of challenges.

The major limitations presented by smartphones, Bawler says, is the speakers themselves, which often use mono and face away from the listener or are placed just under the player’s hands.

“The Nintendo 3DS has amazing HRTF playback going on with its speakers. It would be awesome if more phones featured this kind of thing,” he states.

“You also have to deal with the fact games on mobile are played all over the place, often crowded or noisy environments, or places where you don’t want to annoy other people. When I was traveling on the tube in London whilst working on Monument Valley, there were about ten people in the carriage all playing their phones. On the one hand I thought it was a shame they were all silent, but I was also thankful that we didn’t have an amusement arcade cacophony going on.”

Flesser adds that getting people to play mobile games with the sound turned on is actually one of the biggest challenges for developers, and suggested stating the requirement early on if it’s central to the experience. He also says designers need to make sure audio can run on older devices, which can limit effects on newew mobiles despite their more advanced capabilities.

“For some reason or another, people tend to play mobile games without sound, and quite a few puzzles in Device 6 requires the player to listen to audial clues,” he says.

“We used low pass filters here and there, and we actually had to turn that off on older devices because it was too demanding and we want to keep a steady framerate and performance to keep the experience smooth.”

In the field

While many games keep to a small size, it could be argued many more are becoming bigger and more ambitious, particularly at the top end. As such, the use of field-recording is becoming more widely used and new libraries are being created as the old sounds from past games become overused.

Bawler suggests sound designers now need to up their game if they’re to create the soundscapes consumers clamour for.

“Going out and recording your own material or investing in boutique libraries specially constructed for games is a must,” he states. “The older libraries have been our bread and butter for years, but they were all recorded for TV, film, radio etcetera – a linear environment where you’re making a ‘photograph’ of sound rather than the dynamic, living, breathing soundscapes we have to create for games.”

Beddow says the audio process isn’t necessarily becoming significantly more difficult as games get bigger, but says such scale has sparked a greater management challenge of the process.

“You have to be smart,” he says. “Largely it’s about the best way to manage the asset-volume and deploy the right resources, both internal and external, to deliver the kind of deeply immersive, cinema-grade soundscape that the modern player, quite rightly, has come to expect.”

Looking forward, many more techniques, both old and new, could become more central to the audio process, such as binaural audio in virtual reality games [See boxout Virtual hearing], for example.

Bawler says he’s excited by the prospect of working with procedural audio in future, particularly given the prominence of various uses of the procedural generation technique evident in the industry at the moment.

“Since very early days, my way of working has been built upon building complex sounds and audio systems from simple components combined in interesting ways,” says Bawler.

“Procedural audio seems like a natural fit for this way of working, and from what I can tell, you still need a sound designer to make sure it’s all sounding as it should.”

You can hear more from Stafford Bawler at this month’sAn Audience With… Audio Game-Changers. The free event is strictly invite-only and takes place in London on Tuesday, July 29th – email audiencewith@intentmedia.co.uk to register your interest. More details here.