MCV/DEVELOP News, events, research and jobs from the games industry
This article was created in collaboration with Facebook Gaming.
Compelling digital creative isn’t only more enjoyable for audiences, it’s better for businesses. Great ads help drive engagement and boost efficiency in online auctions by lowering KPIs such as cost per ad recall and cost per action.
But what makes for great creative? It can be difficult to understand what makes ads resonate and how to replicate their successes given the many components to consider.
In working with advertisers to develop effective strategies, we’ve found the key to unlocking this value is a combination of systematic measurement and creative expertise. In this article we’ll look at how to design and perform simple creative tests on Facebook.
HOW TO APPROACH CREATIVE TESTING
Like all good tests, scientific methodology should be used. This minimizes the influence of bias or prejudice while providing a standardized approach to conducting experiments and, in doing so, improving their results.
In Marketing Science, the method is as follows:
- Determine what the business goal or objective the creative is solving for. For example: what style of creative drives more installs, gameplay or CGI?
- Define a hypothesis to be tested. For example: Gameplay creatives outperform live-action creatives in terms of cost per install.
- Design a test with metrics that can prove/disprove your hypothesis. For example: A/B test to determine which creative drives more installs, purchases, and so on.
- Plan your next steps based on possible outcomes. For example: If gameplay wins, use gameplay in the main campaign.
DESIGNING A TEST
Typically, when designing a test, you’re aiming to represent your normal environment in a cost effective manner so you will have to make decisions about the scope and execution which either increases the reliability of the results or reduces the cost or time required to run the test. You can see some examples in the chart below.
It’s important to note the cost and benefits associated with the decisions you make during the design phase as this can influence the validity and scalability of the test results.
GET STARTED WITH TWO BASIC CREATIVE TESTS
It’s important to note the cost and benefits associated with the decisions you make during the design phase as this can influence the validity and scalability of the test results. Tests can be broadly separated out into standard or ‘always-on’ (A/B testing) and advanced.
In general, standard tests are quicker to run and more simplistic in their nature. Advanced tests have more complex setup and design but tend to give more detailed and accurate insights. Below are two types of standard creative test methods you can try as a starting point.
Test one: standard A/B Test
When running A/B tests, our first recommendation is to use Facebook’s A/B Test tool [formerly Split Test tool] in the Experiments [formerly Test & Learn] section of Ads Manager. Standard creative tests can seem simple and easy to run, but to set yourself up for success and ensure that your learnings are robust, follow the below guidelines.
With the A/B Test tool, it’s possible to design a test containing from two to five test groups. A test group can contain individual creatives, ad sets [for non CBO campaigns], campaigns or groups of campaigns.
As a part of the test set-up, you’ll need to select the most relevant success metric(s) for the test (e.g. cost per result) most relevant to your campaign objective (installs, purchases, clickthrough rate etc).
The tool ensures that each cell receives an even spend and selects a winner based on the lowest cost per chosen metric using your account attribution window in Ads Manager.
When to use: Testing creative variations, video vs. static, video duration, including audio vs. silent, different concepts for the same audience etc.
Advantages: Fast, simple to run, can be incorporated into a wider testing framework.
Test two: ad ranking
As the Facebook auction algorithm optimizes delivery towards creatives which show the strongest performance. Some advertisers utilize this to test creative performance and choose top-performing assets. An advertiser can put creatives in an adset, and the creative which achieves the most delivery and subsequent best performance is determined the winner.
However, it must be noted that the auction is heavily biased by early performance and, unlike in a test, fair exploration isn’t forced for all creative candidates. As a result, we do not consider this to be a robust method to test creatives. It can be used in cases where it’s unfeasible to run a standard test on every creative, but it should not be the sole method of testing creatives.
Where to perform: Ads Manager
When to use: As a part of a testing framework where it’s prohibitively costly to run standard tests during all phases.
More information
While standard creative tests are useful tools in isolation, they deliver the most value when they’re utilized as a part of a testing framework embedded in your team’s ongoing operations. For more information on testing frameworks, best practices and FAQs, catch up on our Creative Testing Webinar at www.facebook.com/fbgaminghome/marketers.
