‘Review scores climb 10%â?? from usability testing

Usability testing, when done correctly, can push a game’s review score up by ten per cent.

That was the view of Black Rock game director Jason Avent, who spoke at a midday session at the fourth annual Develop conference.

Usability testing is the practice of placing target audiences in front of games – in a controlled environment – and seeking their feedback on the play experience itself.

Unlike play testing, it is ideally done by people with no expertise in game development, but certainly an interest.

Black Rock, based in Brighton, had applied usability tests to its previous title, Pure, and has pledged to use the process more extensively for the studio’s upcoming title, Split Second.

“I believe that our use of usability testing on Pure had lifted the game’s score by five to ten per cent," Avent said, adding that games which employ the service should expect the same boost in critical reception.

Avent said he regretted not implementing usability testing on Pure before it hit alpha, seeing as some elements of the game, by that stage, could not be adjusted. He added that the process can aid game production both through spoken feedback and usage statistics.

“We can map out play results on a graph and determine where the difficulty spikes in our game lie,” he said.

It also allows players to talk through a game’s bugs so that the developer can prioritise which ones are the most important.

Usability testing feedback can lead to new concepts being added into games, Avent said, as well as be used for downloadable content, patches and updates.

Black Rock has recruited around 200 usability testers without any help from an intermediary, making the process inexpensive. People, such as neighbors of the Black Rock team, had signed up to test Black Rock projects and were given modest pay for their day’s work. Many of these testers were teenagers. All had to sign an NDA.

Avent concluded that it’s crucial for developers to give usability testers as little guidance as possible, as that would defeat the object of finding out if play becomes unclear or confusing.

“But it’s the hardest thing for developers to not step in and help them,” he confessed, “it’s tremendously difficult to not shout at them ‘why can you not see what’s happening here!?’”