How to Measure Incremental Revenue & Prove Your A/B Testing’s Worth

A/B testing holds the power of catapulting any business to new heights. It’s the gift that keeps on giving and supports all digital marketing areas. If you’re not running A/B tests, you’re leaving money on the table. Even worse, you’re not fully capitalizing on your other investments in SEO, Advertising, or Content Marketing.

After all, how can you expect a positive ROI if your website adds unnecessary friction, fails to stimulate interest, doesn’t speak your prospect’s language, or offers little to no trust?

Sure, you may have an offer that’s too good to resist or a sensational product or service – but even so, without split-testing, you’re not fully seizing your opportunities.   

For today’s article, we’d like to bring to your attention the incremental revenue – or, in plain language, exactly how much extra money your split testing efforts bring into the company. 

If you’re a marketer looking for new ways to convince top management to invest in website optimization or an agency unsure about assessing the value of your CRO services, figuring out how to determine incremental revenue could be a game-changer. 

There are three different methods we use for determining the incremental revenue of your A/B tests. We’ll walk you through all of them and tell you which one we deem the best. Even better – with your permission – we’ll send you an email with a plug and play formula that automatically measures your incremental revenue. All you have to do is to insert the variables. Make sure to add your email in the box at the end of this post, and we’ll get it to you in a heartbeat. 

Key Notes

Before we get to the actual how-to part, it’s important to emphasize a few key aspects. 

We do not recommend calculating the incremental revenue for experiments where you test discounts (unless you plan on running that discount on a long-time basis). Tests ran during holidays, or special e-commerce occasions, such as Black Friday or Cyber Monday, are also off the table. 

Our suggestion is to use the formulas from this article only when testing new copy, design, and usability strategies. 

Last but not least, if you’re investing more than $100K/month in A/B testing (congrats, by the way), make sure to calculate your incremental revenue using only the Forward Method. But don’t stop there! Analyze it and consider building a CRO revenue statistics model. 

Method #1. Computing the value brought by an A/B test while the test is running

This is the easiest method among the three. We need a hypothetical example before we show you how to calculate your incremental revenue. 

Let’s say you ran a website A/B test for two weeks on the homepage. Here’s what the data looks like:

• 100,000 homepage sessions per month
• Initial conversion rate 1%
• Conversion rate after A/B test 1.4%
• Statistical significance of 99% 
• Average order value: $100
• Number of variations: 2
• The AB test ran for 2 weeks

Computing the $ generated by A/B test uplift: 

Sessions * (new conversion rate – initial conversion rate) * Average Order Value

When determining the number of sessions, make sure to take into account how many ways your traffic splits into. Since this is a two-way test, your pages’ traffic over one month is evenly split. Moreover, because the test only ran for two weeks, we’ll need to subtract another 50% off – leading us to a final number of 25,000 user sessions for each page.  

Incremental Revenue for 2 Weeks 

= 25,000 sessions * (1.4% – 1%)  *$100 = $10, 000

During the two weeks your A/B test was up, your 40% conversion uplift translated into an extra $10,000. 

Method #2. The Forward Method – Estimating the amount your uplift will generate long-term immediately after turning off the A/B test and implementing the winner.

This method creates a long-term forecast for the extra revenue your A/B test will generate.  Use when asked to estimate how much more the business stands to make without having any historical data to look at. 

We call it The Forward Method because it has zero concern for unpredictability. And based on our experience, unpredictable things do happen. Seasonality hits. Traffic is never consistent. Traffic quality is even less so. Google’s algorithm can throw your SEO efforts into the ditch. 

On the other hand, you may hit the jackpot with your Google Ads adjustments. Those new audiences you began targeting turn out to be gold mines. Our point here is anything can happen. Just remember this method will not take any of it into account. 

Nonetheless, let’s see how to compute the incremental revenue using The Forward Method. 

This method is as straightforward as it gets. To demonstrate, we’ll be using the same set of data previously mentioned. 

We’ll start off with the extra amount our variation generated during the split-test, $10,000. The following steps imply some basic math: 

1.Multiply $10K by the number of pages the experiment had (2 pages in our case, initial page and variation 1). That’s because if you are to implement the winner, then your uplift will also be added to the initial page.

$10,000 * 2 = $20,000 if you were to implement the winner

$20,000 * 2 = $40,000 per month

2.Then multiply the above result by the time period you are interested in.

$40,000 * 12 months = $480,000 per year

Not too shabby, right? Maybe it’s time you got a raise! No one can say you don’t deserve it. 

Now let’s move on to the third method – the one we believe to be the most reliable.

Method #3. The Backward Method – Estimating the amount generated by the A/B test uplift two months after you’ve implemented the winner

Since we’re not playing a guessing game, but counting on real historical data, this is the most accurate estimation we can make. 

However, based on our experience, the difference between this method’s results and the Forward Method is rarely substantial.

As you go through the computation, please don’t forget we have baseline data to consider (the conversion rate before the uplift). 

Let’s assume that after stopping the A/B test and implementing the winner, your homepage performed, as you can see below.

1st Month after implementing the winner – Assume the winner’s conversion rate was lower than it was during the A/B test

As we can see above, the website homepage generated $108K during the first month after the winning variation replaced the control. Taking into account the 40% uplift the winner brought in, we can assume that without it, the total revenue generated would have been 40% lower.  

Based on this assumption, we can go ahead and identify the amount our A/B test raked in.

2nd Month after implementing the winner – Assume the winner’s conversion rate was higher than what we got during the AB test

Unfortunately, you can’t attribute this bump in conversion rate to your A/B test alone. 

Sometimes, your PPC team does a better job. Or maybe you’re riding a positive seasonality wave, launched new campaigns that perform better, and so on. 

Going to the 1st Month example, all we know is you can attribute 40% of your 2nd Month revenue to your A/B testing effort. 

Thus we apply the same formula:

= $187,000 – $187,000/(1+40%)

= $187,000 – $133,571= $53,429

Two months after implementing your A/B test winner, the company made $84,286 more! All thanks to your split-testing efforts.

Final Thoughts

Now you know exactly how much your A/B testing efforts are worth. This gives you leverage to pull your boss’s sleeve and demand an increase in the CRO budget. Or maybe you think it’s time for a raise.

If you’re a marketing manager, you can find out precisely whether your CRO team is bringing in the goods or not. 

Either way, the advantage is in your hands. How you use it is entirely up to you. 

Don’t forget to drop us your email address to forward you our plug and play formula for calculating your incremental revenue. 

Data analytics, CRO, and advertising are what we do best at GrowthSavvy. If you feel your business or marketing department could use our help with any of these, book a free strategy call or email us.