What is A/B Testing?

If you run a website or are involved in conversion rate optimization then you are certain to have heard about A/B testing, but what is it?

What is A/B testing?

In its simplest form A/B testing is the process of testing a new variation of a webpage against the current version in an attempt to increase the conversion rate of a defined goal.

An example would be an e-commerce website who may want to test a different ‘Add to Basket’ button on their website. They create a new version of their product page, this is usually called ‘the challenger’ or ‘the variant’. When visitors arrive at the website they are split with one set seeing the original page and the other set seeing the new variant page.

The split is usually 50/50, however it is no uncommon to see 75/25 or 90/10 for riskier tests. Your split percentage will largely depend on the amount of traffic to your website as you need to ensure that there is enough volume going through each split in order to reach a significant result. I would advise anyone new to A/B testing to stick with 50/50 until they are more confident with calculating the number of people required through their split.

Once the test has received a sufficient amount of traffic, analysis can then be undertaken to calculate the winning split. The winning split is generally the split with the highest conversion rate  as this split will convert more visitors into buyers. One key consideration is the significance of your test results, you need to be sure that enough data has been collected to provide a statistically significant result. Tools such as this A/B Test Significance Calculator are great for checking if you are in a position to end your test and select a winner.

Once you have a significant response you can select your winner and put that variation live across your website. You are then ready to start your next test.

If you are new to A/B testing then it is good to know that you should generally only be running one test on your website at any one time as running more than one test may produce faulty figures as the tests may conflict with each other.

Hopefully this helps to answer the question of what is A/B testing and you are ready to start testing on your website.

If you have any questions regarding A/B testing, feel free to leave them below.