Amazon A/B split testing using Manage Your Experiments (MYE) can be used by brands and manufacturer selling on Amazon to test assumptions about how to position and communicate your product's value and merchandising on Amazon. A/B testing is a powerful tool that can be used to validate or discredit these assumptions and improve the performance of your Amazon listings. By testing different versions you can identify what resonates with your customers and confirm your listings are set up for success to drive more sales. If you don’t run A/B tests, you may miss out on valuable insights that would improve your product listings. This post will cover more about the Manage Your Experiments tool, how to set up A/B tests, and tips for using the MYE tool.
A/B Testing on Amazon Product Listings
Amazon's Manage Your Experiments (MYE) tool makes it easy to run A/B tests on your listings. With MYE, you can create two versions of your content and show them to different customer groups of the same size at random. After a set period of time, you can compare the results to see which version performed better. You can currently allows experiments on the following product listing merchandising options:
A+ Content
Bullet Points
Main Image
Product Description
Product Title
How to use Amazon A/B testing and MYE
To use A/B testing and MYE, you'll need to first have a professional selling account and be enrolled in Amazon Brand Registry. When you are ready to run your experiment, follow these steps:
Go to the Manage Your Experiments page in Seller Central. This can be found under Brands > Manage Experiments.\
Click the "Create a New Experiment" button.
Select the type of content you want to test and the ASIN that you want to run the test on. Note that not all ASINs will be eligible for A/B testing. Eligible ASINs need to be correctly branded and have enough page traffic in recent weeks.
Create and submit two versions of the content.
Edit the Settings including adding an experiment name, defining a hypothesis (changing X to Y will Z), and setting the duration.
Click the "Schedule Experiment" to start it
Analyzing A/B Manage Your Experiment Results
After your experiment is complete, you'll be able to see which version of the content performed better. Amazon simplifies the analysis of A/B Test Results with a probability % regarding which version performed better. The following metrics are used to define the, which version is better:
Units Per Unique Visitor - Total units ordered divided by total unique visitors in the experiment. Only visitors who see the new content are counted in the experiment. Not all visitors to the detail page were counted in the experiment.
Conversion - Customers who purchased at least one item divided by total customers in the experiment.
Units Sold - Total units ordered by customers in the experiment.
Units Sold From Search - Total units ordered by customers in the experiment who first saw the product on the Search page.
Sales - Includes sales across all product offers to customers in the experiment.
Sales From Search - Includes sales across all product offers to customers in the experiment who first saw the product on the Search page.
Sample Size - Number of unique shoppers in the experiment. Only logged-in shoppers who see the Image are counted. Not all shoppers are necessarily part of the experiment. Don't worry about small differences in sample size; we automatically adjust for that when calculating results.
An evidence confidence measure that ranges from weak to strong helps you determine how definitive the test data is. This information is useful for determining how clear the results of the experiment are and what action should be taken. See below for an example of the display of this information.
The potential one-year impact estimates outline what impact implementing the recommendations discovered in the experiment might be for incremental sales and units from a best case, most likely, and worse case scenario. These estimates are not available for all experiments and Amazon is transparent that actual revenue impact may vary.
Tips for Using Amazon A/B Testing Effectively
Here are different best practices to use to use the MYE tool most effectively:
Test different elements of your listings. You can test different product titles, images, descriptions, and A+ content.
Test for a long enough period of time. It can take several weeks or even months to get statistically significant results. By selecting “To significance” for duration you can increase the odds of collecting actionable results.
Don’t be afraid to test and challenge content that you are confident in.
Make sure you have a clear assumption or hypothesis in mind when you start designing your experiment. This will help ensure that your experiment is worthwhile.
For A+ Content try testing image-based content vs. text and information-based content. Depending on what you are selling, customers may prefer more visual image content or text information-focused content.
Conclusion
A/B testing and Manage Your Experiments are powerful tools that can be used to improve the performance of your Amazon listings. By testing different elements of your listings and targeting different audiences, you can identify what resonates with your customers and drive more sales. If you're not already using A/B testing and MYE, we encourage you to give them a try. They can be a valuable tool for improving the performance of your Amazon listings and driving more sales. Goat Consulting conducts A/B experiments on behalf of our clients to challenge assumptions and continually improve listing performance. If you have any questions about how to design an A/B test or what action to take based on an experiment's results, start a conversation with us today by clicking the Contact Us link below.
FAQ - Frequently Asked Questions about Amazon A/B Listing Testing from Manage Your Experiments MYE
Here are some frequently asked questions about Amazon A/B Listing Testing from Manage Your Experiments MYE:
What is Amazon MYE A/B Testing?
Amazon MYE A/B Testing is a feature that allows sellers to test different versions of their product listings to see which one performs better. This can be done by changing the title, description, images, or other elements of the listing.
What are the benefits of using Amazon MYE A/B Testing?
Improved conversion rates: A/B testing can also help you improve your conversion rates by identifying which elements of your listing are most effective at convincing customers to buy your product.
Increased customer satisfaction: By testing different versions of your listing, you can identify which one is more likely to satisfy customers. This can lead to increased positive reviews and repeat business and improve brand perception.
Gain a better understanding of your customers: By engaging with customers and showing them varying content and communication you’ll gain a better understanding of your customers mind frame and decision process.
How long does it take for Amazon MYE A/B Testing to run?
The amount of time it takes for Amazon MYE A/B Testing to run depends on the number of customers who see your listing. In general, it can take a few weeks to get statistically significant results. We recommend utilizing the to significance setting.
How do I know which version of my listing is performing better?
At the end of your experiment, Amazon MYE will show you which version of your listing performed better and provide a measure of confidence in the results of the experiment. You can use this information to make changes to your listing to improve your sales and conversion rates.
Why can’t I view the Amazon MYE in Seller Central?
If you are unable to view the Amazon MYE in Seller Central check to make sure that your brand is enrolled in Amazon Brand Registry and that Brand Registry is correctly connected to your Seller Central account. Also, check user permissions to confirm that the administrator on the Seller Central account has provided you access to view the page. If you are still having issues accessing the page, contact Amazon Seller Support for further investigation.
About the Author - Eric Sutton
This post was written by Eric Sutton. Eric is the Operations Manager of Goat Consulting and is thankful for the opportunity to work with and learn from retail brands of varying size and categories. Eric recognizes that the Amazon marketplace can be a complex environment where merchandising and distribution can be impacted by a variety of factors and parties. He enjoys helping our clients gain a better understanding of their brands current position in the marketplace and finding creative ways to implement the client’s ideal brand image. If you have a question about updating the content displayed on your product detail page, Eric can help identify what factors may be impacting the publication of the content and provide instruction on how to improve rate of correct display. If you have any questions or need assistance with improving your brands image on Amazon, please reach out using the Contact Us form.