BUILD: Development testing

From the very start of my career in games testing QA fifteen years ago, it became quickly apparent to me that strict processes and methodologies needed to be in effect to ensure maximum productivity.

Some people often remark that QA testers simply play games all day; alas, this is not our reality. In fact, it is quite the contrary.

A successful QA department is crucial when delivering milestones, stabilising builds and ultimately releasing products that live up to (or surpass) expectations. From the structure of test plans to the creation of bug templates, Testology’s processes mirror those established in my early years as a QA manager. In fact, most processes we employ have withstood the incredible growth of the industry – although, as development teams and projects grew in size, so did our test plans, testing hours and the volume of bugs in the databases.
The only other variation on the projects we work on is content.

We have to mould and adapt our approach when being presented with completely varying tasks, as the two case studies cited later on demonstrate. Fortunately, we don’t just rely on checklists and structured tests to conduct our work. Testing relies on an eye for detail and an educated awareness of how things should work within a game: a QA tester is not only expected to run through checklists but also use unstructured investigation to find obvious bugs and also the issues that require intricate reproduction steps.

TESTING TIMES
There are numerous key qualities necessary for a successful QA team. Communication is a vital feature of our work ethics at Testology. We maintain a high level of communication through various forms: throughout the working day we would expect multiple examples of contact between our test team and production, other developmental departments and the client’s points of contact. QA would be a useless tool if important information was not relayed immediately and with a high level of professionalism.

Our team is extremely passionate about the industry and this is reflected in the way they are always ready to pass on key messages regarding the testing work. This is imperative, as the daily reports we send at the end of every working day – although a great method of communication – may raise incidences that perhaps should have been flagged as soon as they occurred.

We recently experienced development testing in two very different forms. Lionhead Studios already operates a QA department which utilises its own processes and methods for conducting QA work. We were asked to provide a number of testers late on in the project – typically the most stressful period for QA – which meant that our testers were required to immediately understand and implement Lionhead’s current procedures.

In contrast to the way we worked with Lionhead, we recently had to assemble and manage a test team within Media Molecule’s development offices in Guildford, Surrey. This posed a different challenge, as we were entering the developer’s environment and introducing our own testing methods. As always, our main priority was the client. We wanted to help LittleBigPlanet become the best title it possibly could and do everything within our testing power to achieve this. The main focus was ensuring that all builds were as stable as possible and milestones were met with the strongest candidates possible.

Obviously, LittleBigPlanet has a lot of depth within its content. We put in many hours but our team was highly motivated and passionate about the project. Staff worked tirelessly to ensure that all areas of the game and all modes, such as single- and multiplayer, were stable.

In addition to the functionality testing, experienced QA testers bring a wealth of knowledge and ideas regarding how games should play and function. We had the opportunity to deliver constructive game play feedback to design and production regarding our thoughts on all areas of the game. Media Molecule was always very willing to listen to our points, whether negative or positive, and changes were often made as a result of our concerns or suggestions.

Testing has the advantage of expandability at any given time: numbers can be increased quickly, but this often comes at a cost of quality. It is a common misconception that sheer numbers in a test team will ensure that the quality of a product is sufficient. It is also a common misconception that sheer numbers alone will stabilise builds and ensure developers and publishers reach deadlines.

It is the combination of numbers and the quality of testers that will ultimately determine the service being provided. A nonchalant attitude and approach to QA can be dangerous for developers, and if the importance of QA is overlooked, products may be released without full, attentive testing coverage.

PLAYING THE GAME
A tester does not only have the ability to stabilise a product – after playing a title every day for months at a time, a tester will have a very clear impression of all aspects of the game. I think the industry undervalues the worth of a test department and misses a great opportunity to improve what they are creating. Peter Molyneux and I both shared the same mindset that QA should start well before the normal alpha to beta stages. Dev testing is vital, as major concerns can be exposed before it’s too late.

Testology offers the service of gameplay consultancy, which was born from our experiences in these types of testing environments. We utilise our wealth of gameplay testing, consultancy and dev testing experience to improve our clients’ games. This is an area of testing and quality that many outsourcing companies, developer and publisher test teams cannot offer. A test team should be much more than simply a QA service. At Testology we sincerely believe that our consultancy and feedback service can dramatically improve our clients’ projects.

Testers can pick up on major design flaws and faults as well as assist in maintaining stable builds and helping with milestone submissions. Test teams are often called into action towards the very end of the development cycle, but this is a waste: they can offer so much more than stabilisation towards the end of projects. I believe experienced QA professionals can provide honest, passionate and constructive feedback suggestions all the way through development.

For this reason Testology encourages our staff to think outside the box, taking into consideration how a game could be improved not just fixed.

www.testology.co.uk