A/B Testing: How Retailers Can Optimize Their Sales With Experimentation

A/B testing, sometimes known as split testing, is the process of comparing two or more variables under the same conditions. In its most basic form, you have two versions of a similar product, and you see how people react to whichever sample they’re given. This type of experimentation has been around for a while. In fact, some argue that work done by English statistician William Sealy Gosset in the early 1900s was a form of A/B testing. Unquestionably, it has grown in popularity in the past decade, with online companies like Google, Buzzfeed, and Electronic Arts utilizing split testing regularly to optimize for and measure success. While it might not occur to retailers, A/B testing can be a useful tool to help boost their sales as well. In addition to helping business owners figure out the best course of action for just about anything, there are multiple advantages to split testing.

What is A/B testing?

A/B testing is a method of comparing two versions (A and B) of a variable, such as a landing page, to determine which performs better. This process helps identify the most effective option based on metrics like user engagement or conversion rates.

This type of experimentation has been around for a while. In fact, some argue that work done by English statistician William Sealy Gosset in the early 1900s was a form of A/B testing. Unquestionably, it has grown in popularity in the past decade, with online companies like Google, Buzzfeed, and Electronic Arts utilizing split testing regularly to optimize for and measure success.

While it might not occur to retailers, A/B testing can be a useful tool to help boost their sales as well. In addition to helping business owners figure out the best course of action for just about anything, there are multiple advantages to split testing.

Here, we’ll look at ways retailers can use split testing in their businesses, discuss ways to implement it, and provide step-by-step instructions on how to get started.

Image Credit: Crowd Sourced Testing

How To Use A/B Testing For Retail

Whether you realize it or not, A/B testing is all around us. News websites like Buzzfeed and CNN regularly do it by altering headlines and preview images in articles to see which words and photographs get more pageviews. Politicians use it, especially during election campaigns, to see what issues, words, and photos garner more support. An article on Medium, for example, looked at ways in which U.S. senator Bernie Sanders used A/B testing during the presidential election to drive donations to his website.

So how do retailers use it, and more importantly, why should they?

A/B testing for brick-and-mortar retailers looks different than it does for ecommerce businesses, so we’ll discuss them separately.

For brick-and-mortar stores, retail data management platform Askuity provides some examples of A/B testing. They include:

• Testing two variations of a promotion in order to maximize gross margin dollars. For example, comparing flyers or in-store signage that reads “$10 off when you spend $50” versus “15% off when you spend $50.” While the discount amount is the same, which phrasing draws in more customers? Which creates an uptick in foot traffic and/or sales?
• Testing in-store merchandise teams to justify the investment. For example, calculating the return-on-investment from in-store visits compared to nearby competitors’ stores.
• Testing two variations of merchandise assortment. For example, uncovering the best ways to display your products and drive the most sales.
• Testing two types of packaging. For example, launching a new type of packaging to see which is more effective.

Image Credit: Shopify

When it comes to online retailers, A/B testing might actually seem more obvious. Most people who run a website do some form of split testing just to check how various sections of their site perform in order to increase traffic and conversions into sales. An article from Econsultancy offers some good examples of split testing for ecommerce businesses, including:

• Tweaking copy and images of products to see which one performs better.
• Changing the placement of the call-to-action button, such as “Add to basket,” to see which one performs better.
• Comparing sales strategies, such as “Free shipping when you spend $50” versus a discounted amount comparable to shipping costs when shoppers spend $50.
• A/B testing subject lines in your email marketing campaigns to test which versions gets the best open and clickthrough rates.

There are so many ways to start A/B testing for retail that once you start doing it for your business, you’ll likely think of other applications for it, too. Constantly testing aspects of your business is the best and only real way to find out what works for your store or website and for your customers.

How To Start A/B Testing

Now that we’ve discussed why you should be using A/B testing to improve your retail business and some basic examples of what it entails for both brick-and-mortar and ecommerce retailers, let’s look at how to implement it.

There isn’t really all that much to setting up a split test.

If you’re curious about how something is working — whether that’s a product, sale, advertisement, or navigation — A/B testing can likely be used to help find an answer.

Before you start A/B testing, however, there are a few things you’ll want to keep in mind. According to Askuity, it’s imperative that you have a steady stream of data that’s collected and easy to analyze. What does that mean exactly? Think back to your days of high school chemistry. Before starting any laboratory experiment, your teacher likely asked you to inspect whatever equipment you were using to ensure that everything was in proper, working form. They probably also required that students confirm the materials they were testing, whether that was the purity of ingredients or amount used in the experiment. If you don’t have data that’s constant, reliable, and typical to use, then your A/B testing results will not make sense.

Another thing to consider before performing A/B testing is finding comparable environments to run tests. This might mean finding two groups of stores that achieve similar annual sales, or narrowing down a split test to only include customers from similar socioeconomic or regional backgrounds. External factors and extraneous circumstances can cause detrimental impacts and influences on A/B tests. Just think of how poor weather, like a snowstorm, affects foot traffic in a brick-and-mortar store, or how online events like Cyber Monday impact ecommerce sales. Finding comparable environments to conduct you’re A/B test in is crucial to achieving meaningful results.

The third thing to consider before you start running a split test is what your goals are. Without clear target metrics, such as gross margin dollars or spikes in foot traffic, it’s impossible to know whether the results you found were successful or not. Write down what outcomes you would like to see from A/B testing, and what goal you’re hoping to achieve by doing it. Would you like to increase the number of customers that come into your store during a sale? Or perhaps you’re interested in knowing how certain visual displays affect the ways customers navigate your store? Maybe you’d like to validate the effectiveness of promotional signage, or you want buyers to add more items to their online basket? Whatever the reason, make sure it’s clear, so you know exactly why you’re doing A/B testing and what you’re trying to achieve.

Not sure what other metrics matter for your retail store? Get an in-depth view of the analytics needed to measure your shop’s success with our Insider’s Guide to Running a Successful Retail Store.

Things To Consider In A/B Testing

Now that you know the basics of how to start A/B testing, let’s discuss a few things to consider that might be less obvious than the examples and tips provided above.

For Brick-And-Mortar Retailers

When it comes to split testing for brick-and-mortar shops, comparing how shoppers react to sales messaging might be the most obvious experiment to run. After all, the wording around discounts directly affects how much shoppers decide to spend in store, which impacts overall sales.

One other way to use split testing that might not be as obvious is figuring out how customers use stores. In an article featured in the Outsourced Client Solutions website, Laura Shuh of Chicago’s Bucketfeet used security footage to help improve the navigation in her store. By examining how customers acted on security cameras, Shuh could see which items should be placed where and which displays attracted customers more than others.

Business owners who don’t have access to security footage, or time to comb through hundreds of hours of tape, could garner similar results by trying it another way: by changing their store layout one month to the next.

Other ways to include A/B testing in brick and mortar stores are to change up background music, opening hours, and staff-customer interactions. There are plenty of ways split testing can be used offline, and by performing these tests, brick-and-mortar retailers have the potential to create a retail store that’s just as optimized as ecommerce businesses.

Once you target what variables you want to test, high-tech tracking tools can help entrepreneurs measure what’s working (and what isn’t). Retailers can keep customers roaming their aisles for longer using relevant metrics like:

• In-store dwell time;
• Areas of a store shoppers visit most; and
• Comparisons across store locations.

Entrepreneurs can employ tools like Aislelabs and Measurence to capture these metrics, then analyze that invaluable data to see which option garnered the best result in your split test.

For Ecommerce Retailers

For online retailers, in the simplest terms, A/B testing means you take a page on your website and create a slightly different version of it, then run both pages simultaneously to see which version yields better results. This might include changing elements like layout, design, style, pricing, promotions, headlines, copywriting, and photos.

Online and physical retailers can also use A/B testing when it comes to their digital marketing efforts. Using tools like MailChimp to split test email subject lines, Unbounce to optimize and test landing pages, or the In Cart Upsell app to track what works when upselling to targeted customers.

Another Econsultancy article offers a few more tips that online retailers should keep in mind:

• Know exactly what you’re trying measure and decide what you want to improve. For example, change one thing on a product page with a low conversion rate to see if it changes the number of customers adding the product to their basket.
• Start with just one variable and keep things simple. For example, change only one element of a page, like an image or the call-to-action button.
• Limit the length of the A/B test. Results from tests lasting longer than a couple of months online start to lose their validity.
• Never stop testing. The beauty of a website is that changes can be made often, instantaneously, and with very little effort. Online retailers should tweak and tinker with their websites all the time.

Other Benefits Of A/B Testing

In addition to helping business owners find the best course of action to improve sales, split testing offers a number of other benefits. One of them is that experimenting forces you to pay closer attention to how your business is run. Perhaps, you’ll notice other elements that need to be revamped. You may see trends that you might otherwise miss.

Another benefit is team building. A/B testing only works if everyone in your company is honest. The results don’t mean anything if someone isn’t doing their fair share, such as providing consistent customer service. It also allows every member of your company to feel like they can contribute. Split testing requires input from other team members for alternatives and engages employees in decision-making.

Perhaps, most importantly, split testing helps put ideas into action. As a business owner, making even the smallest changes can sometimes feel overwhelming. You spend more time deliberating than taking action, and there’s no clear reason for positive results. A/B testing is a proven method for finding the truth. You’ll learn to rely less on your gut, and know exactly what works when it comes to running your retail business.

A/B testing in retail FAQ

What is AB testing in retail?

AB testing in retail is a technique used to compare two different versions of a retail website or app to determine which one performs better. The purpose of AB testing is to identify which version of the page or app performs better in terms of user engagement, conversion rate, and other metrics. The test is conducted by randomly assigning each visitor to either version A or version B. Once the test is completed, the results are used to inform decisions on how to optimize the website or app.

What is AB in testing?

AB testing, also called split testing or bucket testing, is a method of comparing two versions of a web page or app against each other to determine which one performs better. It involves showing the two variants (let's call them A and B) to similar visitors at the same time. The one that gives a better conversion rate, click-through rate, or other desired outcome is the winner.

What is AB testing in pricing?

AB testing in pricing is the practice of testing different prices of a product or service to determine which one is more effective in driving sales or other desired outcomes. It works by randomly assigning different prices to different customers and then measuring the results to determine which price is more successful. This can be used to optimize pricing to maximize profits, increase customer engagement, or to test the impact of specific pricing strategies.