Measurement

Incrementality is a type of measurement that breaks down specific key performance indicators (KPIs) to uncover the direct impact certain actions are having on other metrics. It allows marketers to measure what was previously thought to be unmeasurable—did this specific creative cause an uptick in sales, or would people have bought regardless of the ad? Did your media investment have a real payoff? 

Whether you’re trying to dial in ad spend or get a better grip on the true efficacy of your paid media strategy, learning more about incrementality can help you boost conversion rates and make the most of your marketing campaigns.

What Is Incrementality Testing?

Incrementality in marketing measures the impact of not only your entire marketing campaign, but also the individual components of the campaign that help drive traffic. This could manifest as the measurement of an ad’s effectiveness or a landing page’s conversion rate.

With incrementality testing and assessments, you can better attribute and understand the incremental contributions made by each paid tactic or distribution channel. This can be particularly helpful when evaluating campaigns that are otherwise difficult to assess, such as streaming TV.

What’s the formula for incrementality in marketing?

With every ad, you can assume there are three outcomes: someone sees your ad and doesn’t convert, they see it and do convert, or they convert without seeing your ad. With incrementality testing, you choose a test group and a control group and compare their outcomes. Most simply, you can quantify incrementality with this equation:

Incrementality = (Test Conversion Rate – Control Conversion Rate) / (Control Conversion Rate)

Imagine that you’re in charge of managing ad campaigns for a beauty brand. You’re already running display ads on key retail media networks, but your team would like to run an awareness campaign using streaming advertisements. However, they’re concerned that they won’t be able to articulate the value of the top-of-funnel streaming ad once the experiment has concluded.

Incrementality can solve these problems. Start by randomly splitting your target audience into two groups for the same time period:

• Your Test Group: This group sees your existing display ads and the new streaming advertisements. After the same test period, you measure this group and find they have a 2% conversion rate.
• A Control Group, which only sees the existing display ads. After the test period, you measure this group’s engagement and find they have a 1.75% conversion rate. This is your baseline.

(0.02 – 0.0175) / (0.0175) = .143

So, in this case, your streaming ads have a 14.3% incremental impact on your conversion rate. Once you have the conversion rate, you can start to determine your incremental impact on revenue – and by further layering in ad costs, you can better understand the profitability of running that ad. In short, the equation will help you isolate the marketing impact of each touchpoint.

However, marketing strategy is rarely as simple as the scenario above. The basic version of this framework doesn’t explore external variables that influence conversions for example, and not all marketing channels are easy to measure. Most marketers will need to account for a wide range of tactics in their campaigns. And for that, you’ll need a more involved way to test. 

Why Is Incrementality Important for Marketers?

Marketers typically develop campaigns and measure efficacy using key data points. When those data points are generated in-house, there tends to be more control. But when you’re relying on data provided by a third party, like a social media platform, there’s a significant loss in visibility. Instead of understanding the entirety of a consumer’s journey, reports often rely on “last-touch attribution.”

Last-touch attribution basically gives credit for an action (such as a sale, click-through, or inquiry) to whatever consumer-facing asset was used last. For instance, one could easily assume that a consumer signing up for a new streaming service did so because they saw the banner ad on the streaming company’s home page.

But that assumption ignores the influence of all the other marketing collateral that brought the customer to the site, such as:

• The Facebook ad that first raised awareness about the service
• The in-feed videos that explained the benefits of streaming
• The emails sent to foster engagement
• The landing page outlining multiple types of streaming packages

Incrementality helps identify which of these touchpoints are actually making solid contributions, and even how much they’re contributing. Once you have that data, you can make educated guesses about the individual impact of each marketing touchpoint. In other words, incrementality is how marketers test things like vendor accountability, the viability of your media buying strategy, and which ads encourage your audience’s continued engagement.

These metrics help marketers inform better decisions and keep up with trends. Incrementality allows for more flexible spending because it highlights what’s actually moving the needle, and what’s dead weight. It also gives marketing teams the confidence to try new tactics because they can immediately track results and more easily get stakeholder buy in. 

Use Case for Incrementality in Marketing: NOBULL & Streaming Ads

Let’s explore a case study with NOBULL, a sportswear brand that targets highly motivated athletes with bold, in-your-face messaging. As part of developing a mature full funnel measurement approach, they took a close look at their existing streaming ads strategy.

Previously, they used television as an awareness play and had limited visibility into how those ads influenced purchase behavior. Sure, an ad can raise awareness in a new audience – but it can also stoke excitement in existing audiences, bolstering the effectiveness of their existing low-funnel strategies.

With this situation in mind, NOBULL started thinking about incrementality testing.

First, they needed to measure how a streaming ad impacted performance across a variety of paid and organic search channels. They also needed to see how small budget shifts could impact performance, where people were buying their products, and which seasonal factors drove sales. Given the complexity of their existing strategy, their measurement platform of choice was Bliss Point by Tinuiti.

Rather than using a simple formula, platforms like Bliss Point by Tinuiti ingest large volumes of an advertiser’s first-party data, then utilized partnerships with platforms like NBCUniversal to get rich second-party data. From there, the platform used an advanced form of ghost bidding to compare how the streaming ad influenced lower-funnel purchase behavior. The results were a significant increase in site visits, purchases, and revenue.

Outside of this example, incrementality testing can also answer questions like:

• What would have happened if you hadn’t increased your budget for a certain channel?
• What if you hadn’t run your most recent campaign?
• Which cities were most impacted by a specific marketing campaign?
• Which ad formats drive the most revenue per dollar invested?
• Are any of my ads cannibalizing each other?
• Where can I reallocate spend to maximize my revenue and minimize costs?

Incrementality testing lets you probe deeper into outcomes with every additional factor. By comparing different channels or demographics, you can fully understand how each dollar behind your media spend will drive growth, and which parts of your strategy are wasting your spend.

What Are the Best Methods to Measure the Incrementality of Media?

Experts all have their own opinions on which methods are best when measuring incrementality, but they all involve a combination of testing and experimentation. It’s kind of like classic A/B testing with a twist. You’re measuring an experimental option against a control group to see how the outcomes differ.

User-level holdout testing is the most simple option, where you split your audience into two groups, show your ad to only one of the groups, and then compare outcomes between the groups. This is effective, but not very detailed, and doesn’t work for some channels, like linear TV.

Causal interference is another option, using statistical modeling to distinguish correlation and causation between marketing activities and outcomes. Or budget holdout tests, where you withhold spending on specific audiences or channels to reveal the true performance impact your budget has. Additionally, geo-lift studies can be useful, where you segment audiences by location to build control and test groups.  

All of these options can help you track incremental lift, however there are two that can give you deeper insights—Marketing Mix Modeling and Multi-Touch Attribution.

Marketing Mix Modeling (MMM)

Marketing mix modeling helps quantify the impact of individual inputs, gauging how each one contributes to a final sale. MMM is a very inclusive approach, as it takes into account not only years’ worth of historical data, but also outside influences like competitor activity and the economy.

“Media Mix Modeling is a top-down approach that evaluates how historical media activity, promotions, pricing, seasonality, and uncontrollable factors such as economic activity impact sales. Additionally, it provides a measured marketing ROI which accounts for external factors such as weather, unemployment, and others.”

— Annica Nesty, Group Director of Econometrics at Tinuiti

MMM is considered “high touch,” meaning it requires lots of hands-on input with little to no automation, and it can take a minimum of 6 months to get the modeling up and running. It’s also a more top-down view of multichannel marketing, delivering that bigger picture.

Because MMM is so labor-intensive and macro, it’s more often used for long-term planning and major portfolio moves.

Multi-Touch Attribution (MTA)

Multi-touch attribution (MTA) is an “always on” attribution approach that relies on automation. By leveraging existing systems, MTA can pull tons of data on a regular basis, resulting in usable reports in as little as 4-6 weeks.

“Multi-touch or multi-channel attribution modeling can be used to improve ROI by showing which channels or campaigns are most effective at driving conversions, allowing marketers to strategically manage their marketing spend across channels/campaigns with deep channel performance insights.”

— Annica Nesty, Group Director of Econometrics at Tinuiti

Whereas last-touch attribution gives all the credit for a sale to whatever touchpoint came immediately before the customer converted, MTA breaks up the credit, assigning a fraction of the total to every touchpoint that impacted the buyer’s journey. Basically, it focuses on those individual user interactions to paint a picture of what’s going into the decision-making process.

The agility, granular assessments and rapid reporting offered by MTA make it a good choice for situations that demand daily data generation. For example, you might use it to rapidly determine how to reallocate your budget as you’re ramping up an intense, short-term campaign.

Which Channels Are Best Suited for Incrementality Testing?

You should test the incremental impact of all channels, although some are more complicated than others, especially if you can’t measure at the direct response level. To measure incrementality for online display, video and YouTube, Facebook, TV, and direct mail, there are a few other options.

Channels like display advertising can be measured through PSA testing. This is where you serve an audience a piece of content that has nothing to do with your business so you can compare their outcomes to the audience that received your ad. Sometimes, you can use “ghost bidding” to identify the PSA audience group without actually serving them anything. That’s not always available, but if it is, it’s a great way to test without paying for impressions that don’t feature your brand. 

For channels like Facebook and Meta, incrementality can be measured through on-platform studies, where you run conversion lift studies or PSA testing on the specific platform. MMM and MTA are also viable options for Meta testing.

How to Calculate Incremental Impact

Imagine you’re in charge of a marketing campaign surrounding the launch of a new streaming project. You feel in your gut that your streaming ads are responsible for most of the sign-ups to date, but your boss is convinced that those sign-ups are driven by social media ads.

Understanding how to calculate incremental impact can help you prove the efficacy of your streaming ads and show your boss that your efforts are indeed generating awesome results.

1. Define Your Goal

If marketing is an experiment, then this step is where you offer your hypothesis. Look across your media mix and identify where you’re making decisions without a clear cause-and-effect—these are the perfect areas to test.

Think about what you’re looking to identify, then pinpoint the key performance indicators (KPIs) you’ll use to figure out whether you’re making progress toward your goal. This will help you objectively determine if your campaign(s) caused the desired outcome, and which touchpoints in your campaign are contributing to that end. Similarly, goal setting will help you determine if your touchpoints aren’t accomplishing the outcomes you anticipated, helping your course-correct your campaign before it’s too late.

For example, if your goal is to surpass 100,000 total paid customers for your subscription box by the end of the year, you may want to monitor KPIs that quantify:

• Clickthroughs on paid ads
• Total number of free trial sign-ups
• Customer attrition rate

Or, your goals might be something much smaller, such as measuring the impact of email retargeting in terms of getting old customers to sign up for your subscription box again.

2. Segment a Test Group and Control Group

Breaking a segment of your audience into two groups allows you to create your own apples-to-apples comparison between those who saw your ad and those who didn’t. It’s crucial to compare two audiences that have similar characteristics for this reason — comparing two similar customers will isolate the impact of your marketing activities, rather than highlighting pre-existing differences between two distinct groups.

Bear in mind that not every channel supports incrementality testing the same way. Focus first on platforms where user or geo-level holdouts are possible and where results could influence broader strategy, like streaming or social media platforms. Then, with that data-driven backbone, you can layer in less precise channels like linear TV or traditional out-of-home ads.

3. Choose a Test Design and Commit to Rigor

Decide which approach (user- or geo-level holdouts) makes the most sense for what you’re trying to uncover. User-level holdout testing is best for mediums that allow individual-level tracking, like social media platforms. Geo-level holdout testing is designed for channels with little data granularity, like linear TV. Either option will help you understand the impact of your campaigns on exposed and unexposed audiences.

Next, let stakeholders know what to expect based on the approach you chose. This way, they’ll understand the value of testing before results start rolling in. 

You’ll want to run the experiment for at least a week, but ultimately, the time frame depends on factors like the complexity of the campaign, how long it takes to get traction using the channels you’ve chosen, and how quickly you need to analyze and act on results to achieve your larger goals.

4. Analyze the Outcome

The data that comes in from your experiment is just raw info until you sit down and analyze it. Remember to look at the KPIs in comparison mode — you should be examining data from the test and control groups side by side to highlight differences.

Ideally, you want to see an incremental lift in the test group, otherwise known as an increase in desirable outcomes versus the control group. That’s the proof of concept you’ll need to fine-tune your campaign as you get ready to push to a wider audience.

Keep in mind, though, that major gaps aren’t necessarily a sign that you’ve knocked your experiment out of the park. Sometimes those surprisingly large gaps are red flags, signaling a configuration error. For instance, if you have zero conversions from the control group, you’d want to make sure you designed the ad campaign correctly and pushed it out to the right audience, as planned. When in doubt, double-check the settings and run the experiment again.

5. Apply the Results

Finally, take everything you learned and use that info to get your test campaign as close to perfect as possible. This part is where your data turns into tangible financial outcomes, allowing you to focus on what’s growing your business and cutting out the waste.

Dial in your messaging, decide whether you want to reallocate funding and concentrate on some channels more than others, and scrutinize your chosen audience. Once you’re satisfied, it’s time to turn your experiment into a real campaign.

6. Build Testing into the Rhythm of Planning

Incrementality shouldn’t be a one-off experiment. Treat it as part of how your team allocates, evaluates, and iterates not just something you do after the fact. Not only will your strategies be even stronger, but you’ll have a better understanding of why they work. 

If this feels complex—that’s because it is. Getting it right requires clean design, clear data, and stakeholder alignment. Find a partner who’s been through it before and can guide the process from setup through analysis.

Conclusion

Incrementality is a hugely useful tool that can help marketers identify what’s working and what isn’t, creating stronger, more focused campaigns that get the job done without sacrificing the bottom line.