you are here: home > > > > >

A-B Split Testing

Summary

This is the simplest form of conversion testing. You compare your original version against one or more alternatives. Once you collect enough data to know which is better, you keep the best performing one.

As an we use the Google Website Optimizer for A-B Split Testing.

Description

A-B split testing is named for the two website versions that you are considering. "A" refers to the original or baseline version of your site. "B" refers to the alternative or challenger version. Often A/B splits are done on an ongoing basis. This format is commonly referred to as "champion/challenger".

A-B splits can also be done with more than two versions. Depending on your traffic velocity you can test more than two alternatives (e.g. three alternatives would become "a-b-c-d split testing").

A-B split tests can be conducted on individual page elements such as headlines, calls-to-action, offers, or sales copy. They can also be used to compare completely unrelated versions of your whole page. There are pros and cons to both approaches. If you only test individual elements within their current context, you will not be able to capture the advantages of a comprehensive "clean sheet" page redesign. But you will be able to tell the exact impact of changing the element in question. If you choose to test completely redesigned pages, you will not know which elements contributed to the improved performance. You will also incur higher design costs without knowing if you will have a favorable outcome.

The math of a-b split testing is relatively straightforward. You simply pick a confidence level (how sure you want to be that one version is really better than another), and then wait to collect enough data. The length of the data collection depends on two factors: the traffic data rate to your pages, and the size of the conversion rate differences found in your test. Clearly superior versions of your site will "separate out" much more quickly than two versions with almost identical performance. Tracking your a-b test can be done with simple spreasheet formulas or through reports in most a-b split testing tools.

Test Size & Data Rate

Typically 1-10 unique site versions or "recipes". Minimum of 10 conversion actions per day.

Advantages

• Simple to implement.
• Requires little knowledge of statistics.
• The only approach available to low data rate sites (if you are patient).