A/B testing, also known as “split testing”, is a strategy used to optimize your ecommerce store. Individuals, teams, and companies use it to make informed changes to their site’s user experience in support of a larger goal, like increasing conversions.
An A/B test presents 50% of users with an alternate version of a webpage in order to test the effectiveness of a particular variable. For example, a variable to test could be a type of font, or a promotion, or a form. A/B tests are then evaluated based on whether the alternate page achieves a higher conversion rate. If the findings show a notable difference in conversion rate, then a store can move forward with the better performing version of the page, resulting in a higher conversion for the site.
Why is A/B testing important?
Building a website that looks beautiful and feels functional to you is easy. And while you can start measuring the success of your website using tools like Google Analytics, analyzing existing metrics doesn’t allow you to see if you’re operating in the most optimal way.
A/B testing is important because it offers a data-oriented and comparative approach to determining whether your page elements are performing most effectively. An A/B test gives you the opportunity to optimize your site in an objective way, rather than going with what you think works best. And the great thing is, A/B testing doesn’t fully replace your knowledge or intuition around what elements will perform best on your site. Instead, A/B testing allows you to prove or disprove your hypothesis.
This is exactly why A/B testing is such a popular conversion rate optimization method. It is the second most used CRO optimization method after customer journey analysis. In fact, 71% of companies are running at least two A/B tests per month.
How to get started with A/B testing
The options are endless when it comes to A/B testing, so it can be hard to figure out where to start. Before you start A/B testing, the key is to know what page you’d like to test, because knowing what you’d like to improve will guide your testing strategy.
There are four areas that companies tend to focus their A/B testing efforts on:
- Website
- Specific landing pages
- Email marketing
- Paid media
To decide which of these areas is the best place to start, spend some time analyzing pages that have low conversion rates or high drop-off rates. Using an analytics tool like Google Analytics will show you the metrics needed to identify where your conversions are falling short.
Once you’ve identified what page could benefit from optimization, you can look at the various elements on that page and brainstorm what variants could be created to test for optimization. Here are some elements that companies typically test:
- Calls to action
- Headlines
- Product pricing
- Product pictures
- Social proof (reviews, testimonials)
Deciding what you will test and formulating a hypothesis for how that element can be improved is the most crucial step for implementing an A/B test that will provide you with valuable results.
The most popular elements to test are call to action (CTA) buttons, which are used in your website and on your landing pages to guide users towards your goal conversion. Testing the CTAs on your site could entail creating variants in the copy, design, color or even size of a CTA button. It makes sense that CTA buttons are the most tested element because these buttons are what entice a customer to make a purchase, sign up for a newsletter, or book an appointment with your business.
A/B testing to improve your Shopify conversion
A/B testing is a great way to improve the conversion on your Shopify store, but before you get started it’s important to understand the process of testing from start to finish. Your A/B test will be most successful if there is a strategy behind it because the strategy will ensure that regardless of the results of the testing, you’ll be able to derive useful information from those results.
Here are the 5 stages of A/B testing from start to finish:
Identify what needs optimizing – Spend some time analyzing your website’s analytics to identify areas that could benefit from optimization. For example, pages that have low conversion rates or high drop-off rates are a good starting point.
Decide on goals – Before A/B testing, it’s important to identify what metric constitutes a conversion. For example, if your goal is to earn newsletter sign-ups, you’ll structure your A/B test to support that goal.
Create your variants – Using A/B testing software, make the desired change to the element that you’d like to test. The original version of the element is called the “control,” this is what you’ll be testing against. The variant to a control could entail changing the call to action on a button, or subbing in an alternative background image. When creating your variants, you should have a hypothesis that you are aiming to either prove or disprove.
Run the A/B test – Once the test is set to live, visitors to your site will randomly be shown one of the variants (A or B). You will want to run your test long enough to gather data to substantiate the results.
Analyze the findings – When your experiment comes to a close you’ll be given a report that shows how the two versions of your page performed. From this report you can determine whether there is a statistically significant difference between the variables that you tested.
Analyzing the results of your A/B test
There are 2 key things to consider when analyzing the results of your A/B test.
1. Consider how long your test ran.
If it does not run until there’s a 95% chance to beat the control, then the resulting data may not be statistically relevant. The other thing to note is that if the improvement is low (for example, 2 – 15%), then it is necessary to run the test longer. But, if the results of a short test show high improvement (above 40%) then a longer testing period is not necessary.
2. Run the A/B test for at least 7 days.
Regardless of the improvement that you see in the first few days of an A/B test, it is recommended to run the test for a minimum of 7 days. This ensures that your test spans a weekend when traffic can vary. If the results of the test after a full 7 day period show improvement, then you can be confident that the results are sound.
With these considerations in mind, you’ll then analyze the results of your A/B test based on a specific metric. For example, if you are testing a CTA button, then you will want to see if your variant button earned more clicks (engagement) than your control – if it did, then you have a “winner” from your test.
In many cases, you’ll find that you don’t have a winner. This isn’t a bad thing. In the case of a “failed” A/B test, you’ve still gathered useful information. If the control CTA button, for example, proves most successful in an A/B test, then you’ve learned that that CTA is relatively strong, and you can begin to test other elements of that page for optimization.
A/B testing software for Shopify
If you’re looking for a tool to optimize your ecommerce store, consider implementing A/B testing. A strong A/B strategy can help you make informed improvements to your Shopify or ecommerce store in support of your business goals.
There are various tools available for A/B testing your Shopify Plus store, but Dynamic Yield is one of diff’s favorites. Dynamic Yield combines A/B testing and personalization with machine-led optimization, and their software allows you to test anywhere in your tech stack. Dynamic Yield can be implemented on your Shopify Plus store using the app, and integration can take just a few minutes.
This article originally appeared in the Diff Agency blog and has been published here with permission.
