MCV/DEVELOP News, events, research and jobs from the games industry
You’ve got to feel a bit sorry for bug testing and localisation companies.
Every time an exec takes to the stage at E3 and reveals their company’s latest ‘awesome innovation’, you can picture teams at QA firms around the world holding their heads in their hands, knowing that this new ‘exciting announcement’ is potentially going to cause them a world of pain.
In recent memory, we’ve had the dawn of mobile gaming and motion controls. Then there’s been the trend of publishers patching their games on launch day and beyond. And of course, there’s the road-map of DLC and add-on content that needs to be tested for a considerable length of time after a title’s release.
And now it’s VR – a new technology that has the potential to change the industry as we know it. But it once again forces QA and localisation firms to adapt.
Just like when Kinect first came out, we need our testers to be comfortable using the equipment, and avoid any repercussions from over use,” Universally Speaking director Loreto Sanz Fueyo says.
This means taking frequent breaks during testing, which may affect how long it takes to complete a round. For instance, Oculus suggests a 10 to 15 minute break every 30 minutes. In addition, should any discomfort be experienced by the tester then they would be asked to take a break. The cause would then need to be checked by the team, to ensure it’s not a game or device issue, and stems from the user.
In many ways, VR lends itself to an array of issues that would not necessarily be considered with non-VR testing, like motion sickness. In a non-VR game, unnatural player movement would just be an annoyance to the player, which would then be vigorously tested to ensure that the movement was as easy and intuitive as possible. Compare the same type of testing to VR and all of a sudden unnatural movement leads to testers becoming disorientated, which in turn leads to the testers needing to take more regular breaks. Issues such as these would be rigorously tested on non-VR games, but within VR these kinds of issues need to be handled more carefully.”
Pole To Win Europe’s director of localisation Chris Rowley adds: You can’t sit with a VR headset on for eight hours a day and test. You have to have better planning in some ways for your testers and they would work in shorter cycles before you give them breaks. We had to do something similar when Kinect came out on the Xbox 360 because people can’t go running around for eight hours a day. It’s the same thing with VR headsets.”
(Above, left to right): Pole To Win’s Rowley, Universally Speaking’sSanz Fueyo, VMC’s Whittle and Testology’s Robson)
Testology MD Andy Robson also says that the mere fact testers are wearing goggles has slowed down the process.
Testing for VR has slowed things down a bit,” he explains. Because you have a headset, you have an entirely different environment to test in. And when you come to reporting a bug, you have to take the headset off and write it down, go back and try to reproduce it. It is a slow process.
You also need to have a lot of space around you a bit like you had to do with the Kinect. It’s slowed it down a bit. And it also takes time to adjust to new technology and how you test and develop on it. Over a period of time it will get easier. But ultimately you are still testing a piece of software. The testing process never really changes, it’s just the technology and the games.”
And VMC says that it has rebuilt its VR testing spaces in the last year.
"The VR testing area that we use is very different from the area we started in a year ago," VMC’s Kirstin Whittle says.
"The growth and changes that have happened within the VR space have changed the way that testing is done time and time again. The VR testing space has been redesigned and rebuilt multiple times throughout the past 12 months and we expect that there will be more redesigning and reconfiguration throughout the next year. Having the VR testing space within the same office as more traditional QA testing, rather than a separate office, means that a redesign or reconfiguration just means moving other QA teams into different spaces so that the VR QA team can have the exact space that they need. It is this flexibility that means that VMC can continually adapt to the changes within the growing and changing VR environment."
Leading QA firms are making sizeable investments in VR. Testronic recently announced it was opening a dedicated lab for the new tech – and it’s not the only one.
We have recently acquired new facilities to dedicate to VR testing,” Universally Speaking’s Sanz Fueyo says.
This allows us to offer our clients unprecedented VR testing capabilities. Our test teams have access to a huge range of VR devices and a highly protected working environment, which allows them to work efficiently and safely. We have also invested heavily in training for our testers so that they can learn and become comfortable with all of the VR devices currently on offer, especially with Oculus, HTC and Sony devices.”
VMC’s Whittle adds: "We have been offering production support services for almost a year now and have developed a dedicated in-house team in our Montreal lab.
"VMC’s investment in VR is also reflected through its recent expansion, not only has the company increased its VR footprint but ensured it has sufficient bandwidth to support additional VR QA. Currently VMC is testing across over a hundred VR devices."
What’s more – one QA firm is testing VR experiences outside of the game space.
There is a saying that suggests that any new tech will get big attention from two industries in particular – the military and the porn sector.
The latter has taken a keen interest in VR (the former might, too, but we’re less likely to hear about that). And Testology’s Robson says that his firm is speaking to leaders in porn to help test their upcoming releases.
VR is going to be massive in the porn industry,” Robson says.
We don’t stand still when it comes to VR in games – we are looking at it in the digital media space. VR is going to be big in other entertainment sectors. If you are going to be at the forefront of testing, you need to be doing everything.”
VR is not the only issue facing QA and localisation. Increasingly publishers are doing betas to try out the online functionality of their upcoming releases. And leading QA firms suggest that these publishers are trying to crowdsource their bug testing for the online components of their games.
Crowdsourced testing has its pros and cons, just as professional QA does, and developers need to know what they want to get out of QA before making an investment,” Sanz Fueyo says.
Crowdsourcing can be cheaper, more flexible, and sometimes more diverse. But while it may seem like the safer option you have to consider the testing process and the end result. QA firms like us establish a relationship with our clients, which makes communication a lot easier, and makes the entire experience more personable by being able to interact with testers directly.
This proximity allows for communication to be a lot more effective as well, and can save a lot of time allowing for more testing to be performed. We also pride ourselves on being able to inte
