As an important part of the games industry, the QA and localisation sphere is subject to the same changes that the entire sector is experiencing.
For example, the shift in the ways that games are distributed these days has been a challenge for the way QA firms operate. Where before once the game has shipped their jobs were over. Today, that's simply not the case.
The main challenge we face is still how to operate ‘games as a service' as all releases are now live. Even the triple-A PC and console IP require ongoing support,” says Ben Wibberley, VMC's director of business development for games.
QA and localisation firms have had to change how they operate, specifically to exist in an embedded, agile development environment and adjust to the constant updates of content whether that is game updates or DLC.”
But QA and localisation firms aren't just stretched into supporting games post-launch, they are also working on more platforms now than ever before, and this is testing their resources.
The main challenge we've observed is platform diversity. Not only do the product types differ between platforms, various hardware has different functionality features that need to be considered during every test phase,” says Harrison Baker, executive QA manager at Testology.
We've got console, handheld, PC, browser, Steam, Android, iOS, Amazon, VR, TV Games, toy integration, and all sorts of other platforms. Our business goes beyond video games and delves into the digital sector, generally, so we have to tackle multiple different product types within these evolving platforms. As our business grows, our biggest challenge is reacting to platform diversity while training, developing, and promoting diverse tester skillsets at the very highest industry level.”
Mathieu Lachance, functionality QA manager at Babel Games is positive about these changes:
The challenges QA and localisation are facing are the same are all other sectors working with new technology.
We have to be flexible and adapting at a pace that may seem quite aggressive to other older and well established industries. It's a double-edged sword that ends up being positive for the organisations though. It allows them to offer new and better products and services every year.”
"QA and localisation, publishers and developers are under more strain to deliver games cheaper and in shorter time frames."
Mathieu Lachance, Babel
Included in the high number of platforms are countless mobile devices. One way that QA firms are getting round these issues is by using simulated devices – tech that emulates the functionality of a given platform. But this hardware is somewhat divisive in the sphere.
The point of view on the quality outcome of a product regarding the use of emulation is divided,” says Lachance.
Some argue that the simulations are not an exact representation of an actual device, which could affect the final product unexpectedly. On the other hand, having the flexibility of emulating multiple devices rapidly instead of a few in a similar timeframe could actually mean testing the potential outcome of products much more efficiently, allowing us to get through higher volumes of work than what would have been possible otherwise.”
Testronic's operations director Chris Rowley is positive on emulation: It allows for more hardware to be used for testing, which can be cheaper in the long run due to device costs, plus it allows QA to do more testing using few staff when it is done effectively. Good automation will provide a good return for QA on bug yields, stress and stability testing. Bad automation, however, becomes a huge waste of money.”
Baker says it's just not worth the risk. Emulation provides a couple of benefits: cost and time. But it does have a negative impact on the quality of testing.
"Hardware emulation has cost and time benefits. But it
has a negative impact on the quality of testing."
Harrison Baker, Testology
We don't emulate anything. We have over 180 devices and are committed to be market representative so our clients can rest assured. There's way more that goes into a device compatibility phase that can only be achieved with real-device testing. We've had clients unable to reproduce issues we've found when emulating devices – critical issues are missed.”
Univerally Speaking's director Loreto Sanz Fueyo believes that actually the best solution is to use both. Simulated devices allow games to be checked across a higher number of devices, without having to spend money on the actual devices,” she says.
This helps, but doesn't replace testing on devices, as issues can arise from the emulator itself. In general any issues found on an emulator will need to be confirmed on an actual device. This is increasing the quality by enabling test teams to cover a large number of devices, and thereby providing broader testing to ensure a good end user experience.”
HEALTH AND SAFETY
As well as the plethora of devices available now, there is a number of new technologies such as VR on the horizon. But before they get into stores, they need to be tested, and this new tech comes with potential health problems such as giving users motion sickness.
New technology that can effect the health and safety of users, employees included, is not new at all,” says Lachance.
It wasn't that long ago that dancing games came out and, even if most people didn't seem to realise it, some health risks needed to be taken in account for those. Having employees working multiple hours, days, weeks on these can be a risk, and every risk needs to be managed.
Appropriate candidates are found, for which health risks are minimised. Mandatory breaks at predetermined intervals are applied. And assignment flexibility ensures employees are not strained, like rotating projects.”
Fueyo adds: It is certainly an exciting time with these new technologies. We take the protection of our teams seriously. We adhere to all manufacture guidelines in the use of equipment but we take this one step further and have established regular team rotations. In addition, we have dedicated space with enough room to avoid collisions with people or equipment.
She concludes: The technology is moving in giant leaps and we are confident manufacturers are addressing any concerns that have been raised in order to ensure the maximum enjoyment while adhering to health and safety regulations.”