How To Optimize Your Shopify Store With A/B Tests – And 4 Ways Privy Customers Are Using Them To Improve Their Online Conversions


We’ve all been stuck before.        

Even now as I write this, I’m sitting at my kitchen table thinking about all the different hooks I could start with.

Well it’s no different with your store. You spend time designing a new popup, or editing a cross-sell campaign.

But let’s face it. Until you test it, you have no idea what’s actually going to work.

Luckily for you, the days of guess and check are over.

With Privy’s A/B test functionality, you don’t have to worry about writing the best headline every time.

We’re launching The Ecommerce Marketing Handbook on 10/6. Join the 1,000+ other ecommerce founders to be the first to know when it’s available.

What is an A/B test?

An A/B test allows you to create two variations of a campaign and run them simultaneously.

Here are two I just created on Respoke Collection:

Privy A/B Test PopupPrivy A/B Test Popup

It’s the same offer (10% off) with different copy.

So now if 100 people visit the site, 50 will see one version, and 50 will see the other.

Now you can take the guesswork out of which headline is better and make your decisions based on real data.

Privy’s A/B tests let you even test things like how long someone was on your site, or which URL they are on.

Should You Run An A/B Test On Your Shopify Store?

You should think of your store like a house.

You need to first build your foundation. These are the welcome popups, cart savers, cross-sells, etc.

These are the campaigns you need to make money.

Without these campaigns in place, you are missing out on the core part of capturing customer information.

If you haven’t set up your fundamental campaigns, I’d recommend doing that before diving into the rest of this article.

Still here? Good. Next comes the 2.0 stuff.

That’s where the A/B tests come in. These experiments are the best way to take all of the campaigns on your site and iterate on them to make them even better.

One thing to note is that you should be getting at least 1,000 monthly page views on your site before implementing an A/B test. This isn’t required, but it’s an easy benchmark to make sure you are getting enough data to make an informed decision.

What type of A/B Test should you run?

There are thousands of variations you can create.

And with Privy, you can completely customize your campaign down to the button colors and font faces.

But my favorite part is how you can test the behavioral parts of a campaign.

Factors like time on page or scroll percentage.

Here are 4 plays Privy customers are already running today:

1. Customer Info

Fun and Function has done an amazing job running A/B tests throughout their store.

They even test what information a pop-up captures.

At first, Fun and Function was asking for 3 pieces of info from their site visitors:

1. Role

2. Birthday

3. Email

When they removed the role field, their conversion rate jumped 74%!

Fun and Function Conversion Rate Privy

And what’s best is that there was hardly a difference between email only and asking for email + birthday. So they can still send birthday coupon codes!

Use A/B tests to help you capture as much customer information as possible, without sacrificing your conversion rate.

2. The Offer

If I offered you $100 for your email address, I bet you’d probably give it to me.

Well, while most brands can’t offer $100, they can offer something.

The question you need to ask yourself is – how much is that extra email worth to me?

We found each additional email is worth around $15.98.

Having a good offer is one of the most important parts of any good popup, and A/B testing the difference in conversion is a great way to see what works best for your business.

Pete and Pedro had a great exit intent popup.

They tested the difference between 10% and 15% off as people with items in their carts are trying to bounce.

Privy A/B Test PopupPrivy A/B Test Popup

The difference was over 50% in conversion rate!

Pete and Pedro Conversion Rate Privy

If you’re deciding “Can I really afford the extra 5% discount?” ask yourself “How many more customers would you need to make it worth it?”

3. Behavior Test

We’ve heard before that pop-ups can be intrusive.

So once you nail the way your campaigns look, it’s time to optimize the way they appear on your site.

One Privy customer, True Citrus, wanted to test out when a popup should display on their site.

Turns out, showing the popup after 10 seconds on the site boosted conversion by 13.2%! While the 15 second version actually performed worse.

True Citrus Conversion Rate Privy

You can even do this for other indicators, too. Like scroll percentages, number of visits, or number of pages viewed.

These subtle differences can improve the conversion and customer experience on your site.

4. The Headline

Well, forget all of the other components if your headline isn’t good.

This is the most important part of any good popup.

Most people pick a plain headline – something like “Subscribe to our newsletter,” and turn it on.

You can do so much better!

You need to give customers a reason to engage with your content.

Here’s an example that we love from Marketing Examples.

Marketing Examples AB Testing

If you’re not sure what the best headline would be, that’s where A/B tests can help. Write a bunch and see what converts the best.

And one last tip: If you are using photos, we recommend including a person! It’s an easy way to get people to stop and see what you have to say.

Setting up an A/B test in Privy

Getting started with these tests in Privy is easy.

Head to the A/B test navigation bar. Click on “New A/B test

Name your experiment and select an existing campaign that you want to test.

A/B Test Privy

From there, you can create your second campaign.

Name it something simple and we recommend keeping the traffic ratio 1:1 for best results.

A/B Test Privy

From there, simply edit the feature you want to change!

A/B Test Privy

We recommend starting with just one change at a time, that way you know exactly what it was that caused a spike or decline in conversion.

Last step – decide how long you want to run the test for.

I’d recommend getting enough traffic to make an educated decision. For some that might be 1,000 visitors. For others, 100,000 visitors will do the job.

A/B Test Privy

And finally, turn it on and watch the results in real time.

A/B Test Privy

Want to learn more about setting up A/B tests in Privy Convert? Check out everything from this post and more here.

This article originally appeared in the Privy blog and has been published here with permission.

Leave a Reply

Your email address will not be published. Required fields are marked *

What Are KOLs In Marketing, And How Can You Use Them In Your Campaigns?

What Are KOLs In Marketing, And How Can You Use Them In Your Campaigns?

We’ve Expanded Our Return Bar Network To Nearly 9,000 Locations

We’ve Expanded Our Return Bar Network To Nearly 9,000 Locations


Steve has entrepreneurship in his DNA. Starting in the early 2000s, Steve achieved eBay Power Seller status which propelled him to become a founding partner of, a contact lens and eyewear retailer. Four years later through a successful exit from that startup, he embarked on his next journey into digital strategy for direct-to-consumer brands.

Currently, Steve is a Senior Merchant Success Manager at Shopify, where he helps brands to identify, navigate and accelerate growth online and in-store.

To maintain his competitive edge, Steve also hosts the top-rated twice-weekly podcast eCommerce Fastlane. He interviews Shopify Partners and subject matter experts who share the latest marketing strategy, tactics, platforms, and must-have apps, that assist Shopify-powered brands to improve efficiencies, profitably grow revenue and to build lifetime customer loyalty.

You May Also Like
payday loans loans for bad credit
where can i buy clomid buy clomid