Split Testing For Dummies

We started split testing for some of our products last week. Only after a week of testing we found out that we were making only 25% of what we could have make from our One Time Offer promotion. I was total newbie to split testing before we performed this to our website. So, today in this post I’m going to cover few thing what a newbie to split testing should know.

What is Split Testing?
Split testing is simply testing different element of a page for better result. The result can directly be related to money or other actions in the website like increasing subscriber to your website.

You can test different version of all of the component for your website. For example, if you’re split testing on a sales page then you might want to test two headlines. You might want to try different buy buttons. Or you can try two different sales pages. One with lots of images and another with a lot of texts or may be with videos.

What is most important while testing is your visitors may have totally different choice. They may like totally different thing than you like in a certain web page. So do not judge your visitors yourself. They them tell you what they prefer. And let their activity tell you what sells from your website.

Types of Split Testing
There are three types of split testing you can do:
A/B/C testing: This testing is the most simple one. In this testing method you test two or more different version of a same page. Sometimes your designer can come up with with two version of page redesign. Or you may conflict with your colleagues on which page sells better. This testing is perfect for such situation.

Multi Variate Testing: This one is about testing different components with in a single website. For example headline, image, screenshot, buy button etc. You can easily set up this type of testing using Google Website Optimizer if you have some HTML programming knowledge.

Taguchi Testing Method: This testing is based on statistical methods developed by Genichi Taguchi. This method has very complex algorithms to understand. What it exactly does it reduces the number of the actions needed to find our the winner among your versions of the page element.

Which is the best Split Testing Software?
Google has almost everything. When you’re newbie Google website optimizer is the first thing you want to try for mainly two reason. First, its free. Second, its dead simple. And works perfectly for the newbies. The tool itself explains how to set up your split test campaign. You only have to decide what component you want to test.

But when you have a lot of things to test you need something more than Google Website Optimizer. The one I’m using is “Split Test Accelerator”. Its not easy to set up but not to hard too. Its quite expensive also but its worth the money.

The main advantage of Split Test Accelerator over Google Webmaster Optimizer is that, it has the Taguchi split testing facility.

The next advantage is that, you can get more detailed report of your test with Split Test Accelerator. Lets say you’re trying to sell a product with two different price. Lets say page A is priced 70 and page B is priced 25. Even 2 sales from page A is better than 5 sales from page B. In such cases you need to figure out the winner on the basis of amount your pages are making not on the basis of action performed on your site. This type of complex calculations can not be performed with Google Website Optimizer.

How to choose the winner?
Two different version of anything always have chance to perform different. So when do you know which one is the winner? Well, there’s a formula for determining winner.

Lets say you are testing two version of a sales page, page A and page B. Total the sales generated by all your versions. And then calculate the square root of that. Now, calculate the difference between two sales. If the difference is less than the square root of total then the current leader is good to be declare winner.

For example, after 5000 visitors to your page page A generated 12 sales and letter be generated 18 sales. so here, total sales is 30. Square root of 30 is 5.4. The difference in sales in two page is 6. Now, the difference is greater than the square root. That means page B is performing better than page A.

Be patient:
Lets say you ran a split testing campaign. You waited for 3 days for the result. One performed better than second. But you should not hurry yet in choosing the winner because particular result might have been just a chance. Let enough actions be performed before you declare something a winner.