- A/B Testing (Split Testing)
- Multivariate Testing (MVT)
A/B Testing is a fairly basic test where a single aspect of your web page is tested in two different formats to figure out which is more appealing to your users, be it the button color to ‘purchase’ your product or the email subject line asking for donations. This is easily measured by which version generated more clicks. The reason it is called A/B testing is because you usually have a version ‘A’ and a version ‘B’ that you are comparing. In the example below “Acme Widgets” is A/B testing their “Buy Now” button by offering a red version as well as the original gray. The test results showed that with this change 3.5% more of their visitors bought from their site. Bottom line: sales increased when switching to the red button.
MVT takes A/B testing a little bit further letting you change more than one aspect of your web page or email. Say you wanted to change the images, copy and color of your website all at one time and test multiple variations of each? You would use MVT. The idea is to identify the different aspects of your page that affect users’ conversion, target different variations of those aspects, and then put them together to test. To illustrate this lets use the “Acme Widgets” example. With MVT you might not only change the button color but also the text and size of the button. Along with that you might want to change the image of the cog to a different image and have a different background. All those changes might result in a lot of different variations that you will end up testing.
Which test to run
Both tests have a purpose in answering the questions at the beginning of this post, however how do you know which one to use? That will depend on how much time you have? A/B Testing is appealing because it lets you create a test quickly and get it out there in front of your users. If you have an email that you are sending to 100 of your users with a call to action, you could send one subject line to 50 and another subject line to the other 50. This takes almost no time to put together and get return results. As you begin desiring to test more variations your resources need to be greater also. To get a better sample per variation you need to make sure you have enough customers to email or enough visitors to your website. But MVT might give you a better insight into which direction you should be taking your design in order to generate more clicks.
Using Analytics to help your Bottom Line
Google once A/B Tested 41 different shades of blue to see which one would generate more clicks. They changed nothing on the page but the color they used in the header of the page. If you have the time and resources to do a A/B/C/D/E… Test then it can be to your advantage. Remember, Google’s bottom line is directly related to clicks. A test like this might seem superfluous but it affected Google’s bottom line directly.
When you are performing your own tests make sure the same can be said for your changes. You want to be able to increase your revenue and not waste time or money figuring out your customer’s favorite color. Make sure you also have the time to create more than one version of your page. If you are going to use MVT, figure out if you have the resources and time to create the different variations that you want to test. When using either one of these tests make sure that you are properly tracking the outcome of these tests. Make sure that you can tell which version the click or donation came from in order to have a successful test.
Before jumping in to these tests, make sure you know where to start. Find your “call to action” page or button and start there. Try to attract clicks to a shopping cart before the checkout button. Hopefully you already use site analytics and can view what pages your customer are jumping ship from and start there. Also try to use pages that have more traffic, the more results you get the more accurate your tests will be.