Quality time: Beating the industry’s broken games

The controversy surrounding games unfit for release continues to rage, as some of 2015’s biggest hitters repeat the woes of last year’s buggy and broken titles.

MCV asks QA and localisation experts at Universally Speaking, Testology, La Marque Rose, Testronic and Keywords about what needs to be done

The new Consumer Rights Act allows players to obtain a refund for games up to 30 days after their purchase if they are deemed ‘broken’ or ‘faulty’. What impact could this have on the QA and localisation sectors?

James Cubitt, QA manager, Universally Speaking: We hope this new legislation will mean that QA will have additional focus, as it has historically been a part of the development cycle that is often cut short. This should mean that testing is more extensive to ensure that all major issues are fixed before release, rather than fixing issues in a day one patch. However, it is entirely possible that an increase of paid Early Access or open betas will become a more common occurrence. This will depend on the public’s reaction to being utilised as pseudo QA testers. The industry as a whole needs to be aware of these changes, as a premature release of a title may cause a backlash from users against the IP, publisher and developer.

Chris Bewick, operations director, Testronic: QA has always been at the bottom end of the development process, so the introduction of the new Act is good for us and our contemporaries in the business. It highlights how important quality is in the whole game development process. My only concern is how this will be policed; who will decide whether a game is not up to standard?

Erik Hittenhausen, head of QA for games, Testronic: From an industry perspective, this is a very good development, because it means more of an emphasis on quality. Not only do publishers and developers have to think about their reputation and the reputation of their brands, especially in the days when word can quickly spread via social media, there are also serious financial implications of a game being released that does not meet the quality bar. There isn’t much you can get away with if your game isn’t up to scratch.

Andy Robson, MD, Testology: This is something that should have happened a long time ago. I’m hoping that developers and publishers will give more time to QA and add more quality to games being released. It’s good for QA service providers, as there could be more work, but this is also a consideration for internal QA. We all want to release and work on high quality products, and having something in place to stop games getting released in an essentially incomplete state is going to help a lot. The industry, as a whole, will have to learn that QA is an integral part of development and more time and money is needed for QA to add this quality to games.

"The new Consumer Rights Act highlights how important quality is in the whole game development process. But who will decide whether a game is up to standard?"

Chris Bewick, Testronic

An increasing number of firms are releasing their titles to the public in ‘Early Access’ or alpha/beta states. What are your feelings on this growing trend?

Christopher Kennedy, Asia GM, Keywords Studio: There is no substitute for a real-world check on your game with hundreds or thousands of users in just as many play environments. This is where alpha and beta testing can deliver meaningful results. However, there still remains a need for professional checks within the boundaries of test cases. If anything, there is a huge potential for harmony here and the chance for developers to test their games in continuously rigorous environments.

Robson: It has helped with the big releases, as it allows developers to get lots of players from all over the world playing at the same time. For QA service providers and internal QA, we can’t do this, as you need thousands or millions of users. The downside is that consumers don’t all understand the process of games and simply expect the early releases to work and be functional. This maybe gives them a false impression and negative thoughts about the games.

Cubitt: The majority of developers and publishers understand the importance of a good QA service, and use it to complement Early Access or beta testing. It has had a generally positive impact on the perception of development. Users are now seeing the process first hand, how slow it can move, and what can go wrong. There is a big worry that publishers will try to get around bug-testing by increasing the amount of sold beta releases: that way, there is no finished product and those bugs won’t be a big issue. That avoids delays in release, keeps the audience interested and cheapens QA by allowing users to do it.

Hittenhausen: Early Access works well for publishers and developers, but it depends how you use it. Some people misuse it and charge gamers too much – and then still expect to use those people as beta testers. With Early Access, the feedback you get on bugs is just what’s been found. You have no idea what has been checked and what hasn’t. This is where a QA vendor comes in, which can provide a more comprehensive and consistent solution from a technical perspective. Early Access is incredibly useful for developers and publishers in terms of gauging feedback from the people who are most interested in a particular game, brand or genre.

Ccile Irlinger, localisation department manager, La Marque Rose: It provides an opportunity for the game’s localisation team to be hands-on. However, it’s not always manageable to have earlier access to the game as an outsourcing company. As far as workflow is concerned, we deal with bugs and anomalies at an earlier stage than before, while the localisation process is still ongoing, so changes are more easily integrated into the localisation workflow.

Kennedy: Methodic QA and real-world environment testing are two separate yet potentially harmonic processes. Too much or too little of either could tip the balance, but working in tandem can create great results across the board. If anything, there is an opportunity here for developers and professional QA firms to work together to create thorough test cases for a wide array of real world scenarios and assess the outcome side by side.

With more and more titles being released as on-going ‘services’, how has the approach to QA and localisation changed?

Kennedy: Games are living, breathing creatures. Processes need to be kept elastic and we need to make sure that common resources – or at least a common knowledge base – remain in place over time to feed and nurture the product. Translation, QA, and customer experience management are more intertwined than ever, and it falls on us to take advantage of the synergies possible. For example, keeping good records of the issues faced during QA can help the customer support team when they jump into position.

Cubitt: In the past, when a game was released it usually signalled the end of QA support. However, now the ‘games as a service’ ethos is on the rise, with titles constantly being updated with content and fixes, QA and localisation support continues far beyond the original release.

Irlinger: We used to deliver a project and move on to something different. Today, one doesn’t just perform localisation and deliver: we are now committed to escort the game post-launch as well. Efforts are focused on keeping references and teams up to date, and we must stay as reactive as hell for post-launch requests.

Bewick: Working with a developer or publisher on a ‘game as a service’ means a great on-going relationship with the company, the title, the brand and the audience. We can keep the same team on a game, as opposed to moving them on once a traditional major title has been released – this benefits the developer and the publisher, as the QA team build up a knowledge base on the game that improves the quality of the title and by extension, the player’s experience.

Robson: With the internet, games can be updated all the time, so there is a constant flow of work even after initial release. QA isn’t seasonal anymore – we aren’t just testing games for the Christmas period.

"Today, one doesn’t just perform localisation and deliver: we are now committed to escort the game post-launch as well."

Ccile Irlinger, La Marque Rose

What challenges could virtual reality experiences present for QA?

Kennedy: Kinect and PlayStation Move testing gave us a strong tasting of this. Testers had to come to work in tracksuits and keep towels and water bottles near their workspace. Sore wrists and thumbs suddenly turned into body aches when testers had to perform jumping jacks or jog for long stints of time. VR and augmented reality might take this further but, at the same time, we are confident that gamers and testers alike will accept the challenge. In the end, it will be up to publishers and QA professionals alike to make sure there is an adequate environment and play space ready to make this all happen.

Cubitt: As when Kinect and 3DS came out, there are some concerns when it comes to tester safety. Although we are unsure how much of an effect nausea, and other potential issues, could have, we will have precautions in place to ensure tester safety. The obvious ones are making sure they have plenty of breaks and a clear working area, but we will also make sure that testers are rotated often; ensuring a single person is not testing with a VR unit on their head for extensive periods of time. As the teams become more familiar with the new devices, and their limits, the processes will be revised.

Hittenhausen: There are no standards at present, but this is what we are working towards and are investing heavily in R&D to create guidelines along with the publishers and developers in this sector.

What are the trends you expect to shape the QA and localisation sectors over the next five years?

Kennedy: Flexibility is the key word here. Games are showing up on more and more platforms and in more languages than ever. This means that we need to be ready to tackle new situations when they happen: multiplayer scenarios across borders and territories, in-app purchases in new countries, and stylistic text and audio performances in new languages. We need to keep our fingers on the pulse of the burgeoning territories without losing grip of the methods that have taken us this far.

Cubitt: It seems the majority of publishers have learned from last year’s technical issues. Already a number of titles that were originally billed for release over the next month have been pushed to next year, which we see as a positive thing as developers will have more time to deliver a polished experience.

There will always be a few issues in larger games but, as long as these do not affect the gameplay and the majority of users, things should be a lot better.

Irlinger: There is an increasing amount of data to localise, such as influencers’ input, web marketing, eSports and more. Working with large-scale triple-A titles offers an opportunity for rock-solid localisation companies to stand out. The race between firms is not about to end: the games industry is flourishing, attracting non-specialised agencies eager to offer lower prices to increase their business. Their focus is not on quality: they don’t run the same race.