Amazon A/B Testing: The Full Guide to Split Testing for Amazon Sellers

The Impact of Successful Amazon Split Tests

Improving a single variable can have a huge impact on your overall sales. Let’s look at a split test example to show what’s possible.

First, let’s say you test your product title. You test two titles against each other, with the goal of seeing what attracts more clicks from the search results.

The first version, your original title, gets 500 page views over the course of your test. It converts at 10%, at a price of $25 per item.

– 500 x 10% = 50 purchases.
– 50 x $25 = $1250 in sales.

The second version attracts more clicks – 750, over the same time period, with the same rankings. Everything else (conversion rate, price) is the same.

– 750 x 10% = 75 purchases.
– 75 x $25 = $1875 in sales.

The result is an increase in sales of $625, by testing this one variable and seeing what works better.

Your results may be more or less dramatic than this, but it’s an example of how small variables can impact your sales.

Other things may influence your conversion rate, or other metrics. Whatever it is, making multiple small gains will add up to a huge increase in sales, revenue and profit over time.

How to Do Split Testing (Amazon AB Testing Process)

The key to split testing is to create a controlled environment, where the only change between A version and B version is the variable you want to test.

This is the basic scientific method. When scientists run a study to see whether vitamin C supplementation has an effect on resting heart rate, they’ll get two groups, have them as similar as possible in terms of age, lifestyle factors etc, while giving one group vitamin C supplements.

At the end of the test, they should be able to conclude whether or not the supplements had any effect.

You do the same with Amazon A/B testing. Create an environment where, if there’s any difference in results, you can point to a single variable that you changed.

You’ll want to test many different variables, of course. But it’s important to do only one test at a time, in order to maintain clean data. By all means run multiple tests, but only one at a time.

The other key to A/B testing is time. You need a long enough time frame to produce statistically significant results.

If you ran a split test for a day, for example, you might get a big difference in results (percentage-wise) from one variable to another. However, this could easily be a normal fluctuation, or due to an unforeseen variable (like the weather, or another event causing people to shop more or less).

Running your test for a longer time allows you to be more sure that any difference in results is due to the variable you changed, not outside conditions or random fluctuations. You won’t be 100% sure, but you can be much more certain.

How Long to Run Split Tests

There’s no magic time frame that’s best for split tests.

The longer you run a test, the more statistically significant your results will be. If you really wanted to know which image works better, you could run your test over a number of months.

But that’s not really optimal. Ideally you want to get results sooner than this, so you can make changes, start benefiting from the increase in sales, conversions, clicks etc from the winning version, and move on to testing something else.

A rough rule of thumb is to test for at least two weeks. But this can vary. It generally depends on how much traffic/sales you get. Faster-moving products can test for a shorter time, as you’ll have a larger data set to work with. But slower products, which only get a few sales per day, would need to test for a longer time to get statistically significant results.

What Can Amazon Sellers Split Test?

Now let’s move on to what different variables you can run A/B tests for.

You can essentially test anything. UK vs American English in your product listings, for example. But the more granular the variable, the less worthwhile the test.

You want to test more important things first, which are likely to have a bigger impact. Get the benefit from improving on these factors first, before testing little things that could have a minor gain.

Also, be aware that you don’t need to limit your testing to one test per variable. You could test your title, for example, get a winner, then test that winner against another option. There’s no reason you can’t continually test the same variable to see if a change gives any improvement.

Here are some things you can test:

Title

Your product title is one of the first things potential customers see from your product. When you show up in the search results or on sponsored results, your title tells the shopper what your product is, and if it’s relevant to what they’re looking for.

A great title convinces people to click through to your product listing. A bad title has them look past you.

When you test this, consider how clear your title is, what kind of information you convey in the title (such as size/material/functionality), and which keywords you choose to put in the title.

Images

Product images are even more important than your title. A/B testing your images is essential, and really easy.

The primary test you want to do is with your main image. This is the image that shows in the search results, and improving this can have a big impact on your clickthrough rate.

It’s very simple to rotate between different images every two weeks, observing your results and sticking with the one that produces more clicks.

You can also go deeper and test different secondary images, different order, adding video… but leave testing these until after you’ve run other tests, as they’re unlikely to have as big an impact (outside of your main image).

Price

Finding the optimal price point is only possible with extensive testing.

You might think that you need to battle your competitors for price, and fight to provide the lowest price. But that’s not always true.

To be as profitable as possible, you want to sell your product for the highest price you can without massively affecting clickthroughs, conversion rate and rankings.

You may find that a higher price results in fewer units sold, yet higher revenue overall due to the higher price per sale.

E.g.:

• Version A sells 50 units at $10 each, for a total of $500 sales
• Version B sells 40 units at $15 each, for a total of $600 sales

This is one area you probably don’t want to do manually. You’re best to use a repricing tool to continually test your pricing and come out with the best price to optimize revenue.

Bullet Points

After title, images and pricing, test different ways to write copy in your bullet points.

This is generally where you put important information about your product, your product’s features, benefits, and long-tail keywords.

You could test longer vs shorter copy in your bullet points, putting different benefits at the top, capitalizing key parts of your bullet points, and more.

When you test this, try to have a clear concept behind what you’re testing. Don’t just create two, random variations and test against one another. Do it with a theory, as we talked about above – long vs short bullets, X benefit vs Y benefit at the top, or something else like that.

This way, you have more of a conceptual idea as to why the winning variation was more successful, which you can use to create better listing copy in the future.

Product Description/A+ Content

Your product description is a little lower on the page, so it’s not likely to have as big an impact as other variables, but you can still test this.

What’s going to have more of an impact is if you’re a brand registered seller with A+ Content in this area. This will probably affect conversions more, so you might see a bigger difference between different iterations of your A+ Content.

Like with your bullets, have a clear idea behind what you’re testing. Make sure there’s a conceptual difference between your two variations, or else you’re not going to learn too much from your split test.

PPC

Testing is going to be a big part of your PPC management strategy. You’ll want to continually test and optimize your PPC campaigns, to figure out the best, most profitable keywords and campaign strategies.

Testing with PPC is perhaps not exactly the same as a traditional split test, as you don’t really need to control and isolate variables as much with this. But it’s still worth pointing out as an area where you should test and make decisions based on data.

External Traffic Variables

Perhaps even more importantly than split testing elements on your product detail page – if you’re doing external traffic, you have to run A/B tests.

There are so many variables involved with this that you can, and should, test. Split testing is the difference between an external traffic campaign that loses money, and a profitable campaign you can run constantly, on autopilot.

Take a Facebook Ads campaign, for example. Here are all the things you should test:

• Ad headline
• Ad copy
• Type of ad (video vs image)
• Ad creative (different videos or images)
• Carousel vs single-image ad
• Offer (e.g. 30% off vs 50% off, discount vs buy one get one)
• Audience (you can test so many different variables within this – interests, demographics, location, etc…)
• Different variations of copy, image, headline, email optin on landing page
• Landing page vs no landing page

We talk to a lot of people who tried external traffic ads (Facebook, Google, TikTok etc), and said they didn’t work, only to find that they ran one campaign and didn’t even test anything.

You should factor the cost of testing into your budget for external traffic. There’s no way you should expect your ads to turn a profit immediately. It usually takes a lot of testing to uncover the ideal formula, and eventually figure out how to run profitable ads.

The best part is that, when you do, you’ll have found a secret weapon to help you beat your competitors who aren’t running external traffic ads, or who are doing so at a loss.

Learn More: Split Testing Creatives for Facebook Ads