Friday, September 21, 2007

Measuring Usability: Split (A/B) Testing

How and when to use A/B testing ...

"In part two of this series, I wrote about conversion rates, what they are and how to measure them. So, now that you've got something to measure, what’s the next step? You could sit around, just measuring things, hoping for the best. More likely, though, you'll want to make changes to your website, presumably changes that will improve conversion and help motivated visitors become buyers.

Most of the time, people make these changes and never look back. They take their best guess and hope for positive results. Ideally, though, you'd like to know whether a change was for the better or worse, in a real, quantifiable sense. Enter the split test, sometimes called an "A/B" test. The name is pretty self-explanatory; you have two versions of something (copy, a graphic, a layout element, etc.) and want to split them to your visitors and find out whether A or B performs better.

Why Split Test?
So, why not just run version A for a while, then run version B for a while, and see what works better (sometimes called "sequential" testing)? To simplify a whole lot of statistical jargon, it comes down to this: you never know what might have changed over time. People may go on vacation, the market may slump, you (or your competitor) may launch a major marketing campaign, the Fed might raise rates, Starbucks might introduce a size bigger than Venti, etc. In the end, you want to have some confidence that any difference you measure between groups A and B is because A and B are actually different, not because of some external factors muddying up your results.

What Should You Test?
Split-testing originated in the advertising world and is often used on the web to test subtle differences in advertisements and landing pages (the pages people arrive on when they click on an ad). You might test changes in copy, layout, colors, button shapes, or just about any page element that potentially has an impact on your visitors."    (Continued via User Effect)    [Usability Resources]


Post a Comment

<< Home

<< Home