Making games by the numbers

It is no longer enough to know how many people are playing your game. Now developers need to know how long they’re playing for, why they stop playing, whether they spend in-game and how they might be tempted to spend more.

The world of analytics might not seem particularly exciting, but as DeltaDNA CEO Mark Robinson observes, paying attention to key metrics will “shine a light on problems that would be impossible to discover any other way”.

“The ultimate goal of using analytics is two-fold: monetisation and player happiness,” he explains. “Contrary to popular belief, monetising your game does not lose you players; what does is a failure to understand player behaviours and deliver the right experience to the right user. This is where analytics comes in. 

“Know your players and you will build successful games. Instinct only gets you so far. Data and analytics gives insight into different playing experiences. Essentially, there are only two questions, both of which can only be answered with analytics: Why are payers leaving, and how do I improve my revenues?”

Ninja Metrics CEO Dmitri Williams agrees, quoting a long-running business adage: “If you can’t measure it you can’t manage it. Great decisions require both smart decision makers and good data. You won’t get far with only one of those. Decisions without data are as dumb as data without context.”

Some devs remain sceptical, believing that making decisions based on analytics stifles creativity. But John Cheng, general manager for Unity Analytics, says this is not the case.

“Analytics can tell you if your best players are quitting at level five because the boss is too hard, or if your new players are going crazy over the latest treasure hunts you’ve added,” he says. “These are insights into your game design that you wouldn’t be able to unearth without analytics.”

So which metrics should you be tracking? There are so many values to monitor, it can be hard to know what to focus on.

The obvious one is your daily and monthly active users – DAU and MAU. After all, your game needs an audience, and you need to know how big that audience is. 

Retention is also important, but Robinson points out this needs to extend beyond day one. day seven and day 30 can be just as vital, as you need to understand how long players stick around before leaving your game.

That’s not to say day one isn’t important. Tracking the first time user experience is instrumental in understanding how your title can retain players for the long run.

“A typical free-to-play game might lose 20 to 30 per cent of players within the first two minutes,” Robinson explains. “So the FTUE is the place where small improvements can make a big difference to game performance.”

Bring a friend

While it’s also vital to monitor your players’ spender rate after install, number of purchases made, and the all-important ARPDAU – average revenue per daily active user – Williams suggests another, less discussed metric that can be a central pillar to your game’s long-term success: Social value, the amount of spending or play that a user drives among others.

“Anyone can spot a whale, but without knowing who the ‘social whales’ are, you don’t know who’s causing the spending,” he says. “They usually don’t look important because they are rarely big spenders themselves, but if a social whale is at risk of churning, you’re going to not just lose them, but all of their friends. 

“Causing a social whale to act has a positive ripple effect out. Treat them right, and help them enjoy their friends, and you always get a spending lift from the larger group. Watch game mechanics and other player experience events and how they lead to social value. 

“It’s crucial to know what part of the game is growing the community, and what part is killing it.”

Spenser Skates, CEO of Amplitude, says simply tracking metrics is not enough – developers need to act on them, particularly those concerned with player behaviour.

“Knowing that you have a 15 per cent day seven retention is helpful, but how do you actually improve that number?” he says. 

“The best way is to compare the behaviours of the 15 per cent that stuck around compared to the 85 per cent that churned. Identifying the differences will help you tailor your early product experience to drive better retention and grow your game.”

Analytics in action

Cheng reminds devs they should only monitor metrics that are actionable – if only to avoid drowning in data. 

“Focus only on the key drivers for your game’s success,” he says. “In free-to-play, your emphasis is on engagement metrics. Are your players staying in your game, and do they keep on coming back? 

“Don’t get bogged down worrying about the number of total players or downloads as they are often just vanity metrics.”

It’s crucial to know what part of the game is growing the community, and what part is killing it.

Dmitri Williams, Ninja Metrics

Allison Bilas, GameAnalytics’ VP of product, suggests studios think beyond the usual stats and use analytics to evaluate game design. 

“Track the concepts that lead to players being more engaged or more likely to spend money, such as player progression, virtual economies or in-game resources,” she says. “These give you a deeper understanding of how players interact with your game, and why your overall KPIs [Key Performance Indicators] trend the way they do. It’s important to know the high-level KPIs, but also the underlying context to understand how to act on the data.”

Once you’ve established whether changes need to be made, Upsight’s director of services and analytics Nathan Triplett stresses that it is imperative to test those alterations first – and that they should only be made if absolutely vital to the long-term success of your game.

“The more changes you make, the higher the chance of doing something awesome or terrible,” he warns. “The first step should be to have good hypothesis, supported by data. Making level ten twice as hard is a risky proposition. Doing so because the success rate of users passing level ten is 40 per cent below levels nine and 11, and the rate of users spending on power-ups during level ten is also low, is how decisions should be made.

“Making level ten harder may cause people to spend more, but it may also cause others to spend less. Is the trade-off going to be worthwhile? Reduce that risk by only rolling your changes out to a small fraction of your users first – say, one per cent. Then compare the churn, retention, and spending rates of those to your baseline users. Once you’ve acquired enough data, you can be confident in your decision to roll out the change to everyone.”

Statistical support

Not all developers have the know-how to access and track this data, so multiple companies have formed businesses around providing analytics and the tools to better understand them. 

You can see our comparison of leading providers below, but what should studios expect from these firms?

“These days, detailed analytics is only part of the picture,” says Robinson. “Successful third-party tools combine dashboards with flexible and powerful analytics and multi-channel marketing tools, allowing users to quickly diagnose and action active player management.

“The dashboards should tell you how the game is performing and how it benchmarks. Data capture and querying tools give you the insight you need to make changes. You also need the tools to interact with players, make offers, make gifts, balance the game, serve ads and test changes made.”

Triplett adds: “You want an analytics provider to make you more efficient, but you also need flexibility. Your CEO should be able to glance at a dashboard and glean the high-level picture in seconds. Your marketing team should be able to know if the new acquisition campaign they launched is a return on investment. These use cases are often available with many analytics tools.”

Skates says studios should ensure their analytics package can scale as they and their game grows: “As you continue to build out your game you’ll want to track new features, and having to calculate trade-offs because of event-based pricing will waste time that’s better spent on building product.”

But above all, an analytics provider should make your data as accessible as possible. Bilas says that, fortunately, there are plenty of gaming-specific packages available that already know the key metrics you’ll need to track.

“They also often have features that allow you to compare your game performance against other games, which is invaluable in bringing context to your data,” she says. 

“That said, there is no analytics solution that is going to be able to do everything for you, so having a tool that gives you complete access to your raw data is important so that you can do custom analysis when needed.”