In this article, we’ll explain all you need to know about Amazon split testing (aka A/B testing), and why you should be doing this in your business.
Successful businesses make decisions based on data. They’re always testing, gathering data, and using that data to shape key decisions.
This process is called split testing, or A/B testing. Keep reading and we’ll explain more, and how Amazon brand owners can start utilizing A/B testing correctly.
What is Split Testing or A/B Testing?
Split testing, or A/B testing, is a way to conduct experiments and see which variables produce the best results.
These are two terms for the same thing. They come from the idea that you “split” your experiment into two versions – an A version and a B version. In each version, you only change one variable, keeping everything else the same.
At the end of a set period of time (generally at least two weeks), you compare the results of the A version and B version. You can assume that any difference in the results is down to the variable that you changed.
What’s the Benefit of A/B Testing?
In any marketing campaign, product page, website, landing page, or anything similar, there are a huge number of variables.
Each of these variables has an impact on your success – sales, conversion rate, etc. Yet it’s difficult to pinpoint the right way to approach each of these variables.
If you don’t have any data to work with, any decisions you make are just guesswork. You may think that a title, or an image, or a certain ordering of the features and benefits works well. But at the end of the day, you’re just guessing.
Split testing gives you actual data, letting you make a much more reliable guess on what works and what doesn’t.
You might find that you get more clicks to your product listing with one image over another, or a higher conversion rate when you put a specific benefit at the top of your bullet points.
The Impact of Successful Amazon Split Tests
Improving a single variable can have a huge impact on your overall sales. Let’s look at a split test example to show what’s possible.
First, let’s say you test your product title. You test two titles against each other, with the goal of seeing what attracts more clicks from the search results.
The first version, your original title, gets 500 page views over the course of your test. It converts at 10%, at a price of $25 per item.
– 500 x 10% = 50 purchases.
– 50 x $25 = $1250 in sales.
The second version attracts more clicks – 750, over the same time period, with the same rankings. Everything else (conversion rate, price) is the same.
– 750 x 10% = 75 purchases.
– 75 x $25 = $1875 in sales.
The result is an increase in sales of $625, by testing this one variable and seeing what works better.
Your results may be more or less dramatic than this, but it’s an example of how small variables can impact your sales.
Other things may influence your conversion rate, or other metrics. Whatever it is, making multiple small gains will add up to a huge increase in sales, revenue and profit over time.
How to Do Split Testing (Amazon AB Testing Process)
The key to split testing is to create a controlled environment, where the only change between A version and B version is the variable you want to test.
This is the basic scientific method. When scientists run a study to see whether vitamin C supplementation has an effect on resting heart rate, they’ll get two groups, have them as similar as possible in terms of age, lifestyle factors etc, while giving one group vitamin C supplements.
At the end of the test, they should be able to conclude whether or not the supplements had any effect.
You do the same with Amazon A/B testing. Create an environment where, if there’s any difference in results, you can point to a single variable that you changed.
You’ll want to test many different variables, of course. But it’s important to do only one test at a time, in order to maintain clean data. By all means run multiple tests, but only one at a time.
The other key to A/B testing is time. You need a long enough time frame to produce statistically significant results.
If you ran a split test for a day, for example, you might get a big difference in results (percentage-wise) from one variable to another. However, this could easily be a normal fluctuation, or due to an unforeseen variable (like the weather, or another event causing people to shop more or less).
Running your test for a longer time allows you to be more sure that any difference in results is due to the variable you changed, not outside conditions or random fluctuations. You won’t be 100% sure, but you can be much more certain.
How Long to Run Split Tests
There’s no magic time frame that’s best for split tests.
The longer you run a test, the more statistically significant your results will be. If you really wanted to know which image works better, you could run your test over a number of months.
But that’s not really optimal. Ideally you want to get results sooner than this, so you can make changes, start benefiting from the increase in sales, conversions, clicks etc from the winning version, and move on to testing something else.
A rough rule of thumb is to test for at least two weeks. But this can vary. It generally depends on how much traffic/sales you get. Faster-moving products can test for a shorter time, as you’ll have a larger data set to work with. But slower products, which only get a few sales per day, would need to test for a longer time to get statistically significant results.
What Can Amazon Sellers Split Test?
Now let’s move on to what different variables you can run A/B tests for.
You can essentially test anything. UK vs American English in your product listings, for example. But the more granular the variable, the less worthwhile the test.
You want to test more important things first, which are likely to have a bigger impact. Get the benefit from improving on these factors first, before testing little things that could have a minor gain.
Also, be aware that you don’t need to limit your testing to one test per variable. You could test your title, for example, get a winner, then test that winner against another option. There’s no reason you can’t continually test the same variable to see if a change gives any improvement.
Here are some things you can test:
Your product title is one of the first things potential customers see from your product. When you show up in the search results or on sponsored results, your title tells the shopper what your product is, and if it’s relevant to what they’re looking for.
A great title convinces people to click through to your product listing. A bad title has them look past you.
When you test this, consider how clear your title is, what kind of information you convey in the title (such as size/material/functionality), and which keywords you choose to put in the title.
Product images are even more important than your title. A/B testing your images is essential, and really easy.
The primary test you want to do is with your main image. This is the image that shows in the search results, and improving this can have a big impact on your clickthrough rate.
It’s very simple to rotate between different images every two weeks, observing your results and sticking with the one that produces more clicks.
You can also go deeper and test different secondary images, different order, adding video… but leave testing these until after you’ve run other tests, as they’re unlikely to have as big an impact (outside of your main image).
Finding the optimal price point is only possible with extensive testing.
You might think that you need to battle your competitors for price, and fight to provide the lowest price. But that’s not always true.
To be as profitable as possible, you want to sell your product for the highest price you can without massively affecting clickthroughs, conversion rate and rankings.
You may find that a higher price results in fewer units sold, yet higher revenue overall due to the higher price per sale.
- Version A sells 50 units at $10 each, for a total of $500 sales
- Version B sells 40 units at $15 each, for a total of $600 sales
This is one area you probably don’t want to do manually. You’re best to use a repricing tool to continually test your pricing and come out with the best price to optimize revenue.
After title, images and pricing, test different ways to write copy in your bullet points.
This is generally where you put important information about your product, your product’s features, benefits, and long-tail keywords.
You could test longer vs shorter copy in your bullet points, putting different benefits at the top, capitalizing key parts of your bullet points, and more.
When you test this, try to have a clear concept behind what you’re testing. Don’t just create two, random variations and test against one another. Do it with a theory, as we talked about above – long vs short bullets, X benefit vs Y benefit at the top, or something else like that.
This way, you have more of a conceptual idea as to why the winning variation was more successful, which you can use to create better listing copy in the future.
Product Description/A+ Content
Your product description is a little lower on the page, so it’s not likely to have as big an impact as other variables, but you can still test this.
What’s going to have more of an impact is if you’re a brand registered seller with A+ Content in this area. This will probably affect conversions more, so you might see a bigger difference between different iterations of your A+ Content.
Like with your bullets, have a clear idea behind what you’re testing. Make sure there’s a conceptual difference between your two variations, or else you’re not going to learn too much from your split test.
Testing is going to be a big part of your PPC management strategy. You’ll want to continually test and optimize your PPC campaigns, to figure out the best, most profitable keywords and campaign strategies.
Testing with PPC is perhaps not exactly the same as a traditional split test, as you don’t really need to control and isolate variables as much with this. But it’s still worth pointing out as an area where you should test and make decisions based on data.
External Traffic Variables
Perhaps even more importantly than split testing elements on your product detail page – if you’re doing external traffic, you have to run A/B tests.
There are so many variables involved with this that you can, and should, test. Split testing is the difference between an external traffic campaign that loses money, and a profitable campaign you can run constantly, on autopilot.
Take a Facebook Ads campaign, for example. Here are all the things you should test:
- Ad headline
- Ad copy
- Type of ad (video vs image)
- Ad creative (different videos or images)
- Carousel vs single-image ad
- Offer (e.g. 30% off vs 50% off, discount vs buy one get one)
- Audience (you can test so many different variables within this – interests, demographics, location, etc…)
- Different variations of copy, image, headline, email optin on landing page
- Landing page vs no landing page
You should factor the cost of testing into your budget for external traffic. There’s no way you should expect your ads to turn a profit immediately. It usually takes a lot of testing to uncover the ideal formula, and eventually figure out how to run profitable ads.
The best part is that, when you do, you’ll have found a secret weapon to help you beat your competitors who aren’t running external traffic ads, or who are doing so at a loss.
Amazon Split Testing Tools
You can do split testing manually, but it’s going to take a lot of time and effort. We’ve spoken at length about the benefits of automation for your Amazon store, and A/B testing is one area where software tools can help you a lot.
Here are some of the best Amazon AB testing tools:
For A/B testing Amazon listings, the first thing you need to know is the right place to get data. Luckily, all of this is readily available in your Seller Central account.
Amazon’s business reports have everything you need to analyze your listing split tests.
- Go to the Reports dropdown in your Seller Central account
- Choose Business Reports
- Click Detail Page Sales and Traffic By Child Item
This will give you an exceptionally powerful reports, showing Sessions, Buy Box Percentage, Units Ordered, Product Sales, and Unit Session Percentage. These are all the listing metrics you need to adequately measure the success of your tests.
Just generate reports by the date range you’ve run your test, and assess from there.
Manage Your Experiments
Amazon provides a split testing tool called Manage Your Experiments to brand-registered sellers, which you can use to run split tests on your product listing.
Amazon states that the Manage Your Experiments tool can increase sales as much as 25%. That’s not to mention the value it provides in terms of saving time managing your split tests.
You can use this to test most aspects of your product detail page, including images, title, bullet points, product description and A+ Content.
It comes with customizable settings for how long you want to test for, and the option to automatically go with the winning result after the test is complete.
Manage Your Experiments is free for sellers enrolled in Amazon Brand Registry. If you’re a brand owner, you have to take advantage of this free tool.
To test your price, as we mentioned earlier, you’ll want to use a repricing tool.
Most of these tools will automatically test different price points, and keep whichever price provides you with the highest overall sales.
You might want to check in manually from time to time to ensure nothing goes wrong, but most of these tools are battle-tested and easily automate this process.
There are many quality Amazon repricing tools out there, including:
An Amazon PPC software tool is a must to test and optimize your PPC.
Something like SellerMetrics is perfect for this. It’s built on proprietary technology, made by PPC experts who know Amazon advertising inside and out.
PPC is one of the few areas of Amazon selling where you really need to use software. Managing all of this manually is too much, when software tools can provide you with so much data, and even automatically analyze this data and optimize your campaigns to fit what the data shows.
You can get some of the info for split testing within Facebook Ads manager, or the ad console for whichever platform you’re running ads on.
But you’ll also want to create separate landing pages or tracking links for each variable you test. This will let you see how many people from each version clicked through to your listing, opted into your email list, etc.
This means creating a lot of landing pages or tracking links. LandingCube is the best tool for this, letting you set up landing pages in just minutes. It’s the easiest option to build high-converting landing pages at scale.
You can organize the landing pages and view the data together in your LandingCube dashboard, and get the information you need to choose the winner of your split tests.
It’s also the easiest way to create Amazon Attribution links at scale, which will give you more info to judge your A/B tests on.
Split testing and making decisions based on data is essential for any online business – especially Amazon businesses. The only way to truly know your customer behavior, and what results in the most sales, is to test and find out.
Even small improvements in clickthrough rate, conversion rate and sales due to improving things like your images and product title can have a huge long-term effect on revenue.
The higher your sales right now, the more you have to benefit from split testing.
And when we get to Facebook Ads and other forms of external traffic, split testing is the only way to build profitable campaigns.
If you’re not split testing yet, start now. You need time to get enough statistically relevant data, so start as soon as possible to enjoy the benefits sooner.
RANK LIKE THE PROS
Discover a better way to rank products from external traffic, like Facebook, Google and TikTok, with LandingCube promo pages.
Try it free for 21 days.