How to Create a Successful Test Campaign

How to Create a Successful Test Campaign

1.  Start with A/B Testing

A/B Testing involves taking a current web page and comparing it to one or more updated interfaces.  Because a large number of changes are being tested at one time, this method of analysis typically produces a variety of conversion rates and in most cases, a clear winner is easy to establish.

"The Conversion rate is defined by how a prospect moves in stages through a company's sales process, and how effectively the route they take matches this process [1]" - Brian Eisenberg, CIO of Future Now, Inc.

The diagram below provided by WiderFunnel Marketing Optimization demonstrates the testing process. Users are randomly divided between each page flow and analyzed until either a fallout or a conversion occurs.  

*Image by WiderFunnel Marketing Optimization

A/B Testing can be done in a condensed duration and still achieves accurate results.  Once a desired sample size has been reached based on a preferred confidence level, testing can be immediately deactivated.

Greg Linden, a developer at Amazon, recommended offering customers a personalized, impulse buy while in the check out, but his idea was turned down.  “I was told I was forbidden to work on this any further,” Linden stated in a blog post[2].

Linden didn't give up, but instead ran an A/B Test which confirmed his idea generated higher revenue.  “I do know that in some organizations, challenging an SVP would be a fatal mistake, right or wrong,” Linden continued to write. But once his idea proved to be successful, Amazon’s executives couldn't refuse.  Amazon is now known for offering personalized features to customers thanks to Linden's determination to test an idea before letting it go[2].

Data will always speak louder than status. 

A/B Testing Tools on the Market Today:

Testing Tools	Advantage	Disadvantage
Adobe Test & Target	A/B Testing and Multivariate Testing Segmentation on All Features Expanded Customer Profile Connects with SiteCatalyst and Google Analytics	Expensive Longer development time
Google Optimizer	Free A/B Testing Easy to setup	Limited Multivariate Testing Limited Analytic Tools
Optimizely	Minimal Cost Simple Javascript Snippets Easy to setup	No Connection to SiteCatalyst Limited Multivariate Testing
Vanity	Free A/B Testing Sidebar Connects with Google Analytics	Difficult to Develop Rails and Ruby coding No Multivariate Testing

2.  Mix in Multivariate Refinement

After the initial A/B Test is complete, the campaign will still require fine-tuning. Multivariate Refinement inspects a variety of features at once. The tests are explored individually as well as combined to determine the greatest outcome in the shortest amount of time.

*Image by WiderFunnel Marketing Optimization

Segmenting can also be applied to Multivariate tests to determine if certain browsers or device types prefer a specific feature. If necessary, segmenting can continue to be used permanently on the website to accommodate these conditions.

3.  Repeat to maintain Conversion Optimization Strategy 

The last step is to continually examine the website’s conversion strategies by lifting up value and removing pitfalls. The following questions will guide analysts to make wise strategic decisions[3]:

• Relevance – Does the content of the page appeal to customer’s expectations?
• Clarity – How clear are your solutions, products, and content?
• Anxiety – Are there features visible or missing that cause users confusion or frustration?
• Distraction – Are there unnecessary components that are distracting your customers from a successful conversion?
• Urgency – Are there features motivating your customers to act immediately? Buy now?

Use these questions to continually test  and optimize your website. Optimization is an on-going process. When one test is over, a website should be exploring additional solutions that need to be taken.  Regular testing will ensure your website is producing its highest level of revenue and keep your brilliant engineers employed. Since very few engineers will make it in stand-up comedy, the more work you give them the better it is for everyone. 

Learn More About: Sherrie Cowley

Blog analysis:

Research Time – 4 hours

Blog Creation – 2 hours

Blog Channels:
Facebook
Linkedin

Reference:

[1] Eisenberg, Bryan (2002). “TheMarketer’s Common Sense Guide to E-Metrics”. Future Now. Fairfield, CT.

[2] Christian, Brian (2012). “The A/B Test:Inside the Technology That’s Changing the Rules of Business”. Wired.

[3] WiderFunnel (2013). “OurProcess for Strategic Conversion Optimization”. WiderFunnel Marketing Inc. Vancouver, Canada.