As the advertising industry rapidly adopts “people-based marketing” to target actual people rather than cookies or devices, 15% of marketers are already practicing “people-based testing,” a new ad testing method for determining and optimizing ad effectiveness.
What is people-based testing?
People-based testing is the practice of comparing creative assets to understand which performs best for a particular audience made up of real people.
No matter which ad testing method you use, there is always the case that one ad might do better than another just by chance. What you really want to know is if you can confidently predict which ad would do the best the most amount of times.
To do that, you need to distinguish whether one ad really did do better than the other for a large enough group of people such that, if even more people saw both ads, they’d continue to respond better to the winner.
This is where people-based testing methods shine.
By using people, instead of advertising cookies or devices, as the basis for your audience, you’re able to split test ad delivery with confidence and consistency. It ensures test groups don’t see each others’ creative versions which would contaminate the test by making it difficult to say what creative led to what result.
This ad testing method leads to more accurate and faster results when determining what is working in your marketing strategy, and the resulting winner is reliable as a predictor of future performance.
Wait—am I doing this already?
You’re probably already testing your marketing, but if you’re not tying your results back to real people, then you’re not doing people-based testing. Marketers use a lot of proxies for people that get them a piece of the picture, but never the full view. Let’s look at some of these ad testing methods:
Impression-based ad testing
When the internet first started, there wasn’t an easy way to identify the same person over and over again. The default method of ad serving and testing became “impression-based testing” (also known as creative rotation testing). In this approach, you would not factor who the user is and instead just run multiple creative versions or media placement that evenly rotate. The results are random and you will get a different predicted creative each time you run this, making this approach unreliable.
Cookie-based ad testing
Slightly better than impression-based, cookie-based ad testing methods use “cookies,” which are temporary IDs assigned to a web browser, to keep track of a person. You would split test by dividing cookies randomly into different groups. The problem is that a user has multiple cookies on different browsers and devices they own. In addition, a user deletes cookies every 30-days on average so for experiments that run weeks or months, a user is likely to have multiple cookies and therefore get put into multiple test groups and subsequently be contaminated from a data standpoint. Cookie-based split testing historically worked until the rise of mobile devices and faster cookie-deletion by browsers.
Device-based ad testing
Using device fingerprinting and probabilistic modeling can be more accurate than cookies because people are more likely to be tracked continually and accounted for by device. However, people do change their devices and update their technical settings (operating system version, app version, etc.) which forms the basis of device-based identification.
People-based ad testing
People-based is a deterministic method where verified user logins provide a method to identify the same person across time and devices. As a result, people-based testing can divide individuals into test groups that are stable, measurable, and predictive.
Got it. So what does this testing look like?
In an ideal scientific experiment, you would run what’s known as a “randomized controlled trial” where you randomly divide people into two or more groups and give each group a different treatment so that at the end of the trial you can compare the results and know that there could only be one variable that explains the differences in results.
For example, when drug companies are figuring out if a new breakthrough drug works, they will have a group that receives a placebo (like a sugar pill that doesn’t have any of the new medicine) and a group that receives the new drug. After a year, they will compare the two groups to see if the group with the new drug did better materially than the group that didn’t get the drug (or did the same or worse!).
This is also how people-based testing works. Over the course of a test campaign, you’re consistently giving creative A to one group of people and creative B to the other group of people. At the conclusion of the campaign you’re able to connect the version of copy to better (or worse) performance.
Impression-, cookie-, and device-based testing, however, work like giving the drug or placebo at random each week. At the end of 52 weeks, it will be hard to figure out the impact of the drug because most of the test subjects will be “contaminated” by having tried both the drug and placebo, therefore offering no clear indication of what caused the result.
For more ideas on how to leverage people-based marketing for measurement, take a look at our Journey to People-Based Measurement guide (and turn to page 14 for more on how to get started with incrementality testing).