Saturday, July 15, 2006

Granular Bucket Testing

Using website metrics to determine best design ...

"A recent discussion on the Interaction Design list about the utility of product design research got me thinking about how the bucket testing of Web sites has seemed to change over time.

Bucket testing, otherwise known as A/B testing, is a methodology for gauging the impact of different product designs on a Web site’s metrics. For those unfamiliar, the basic premise is to run two simultaneous versions of a single or set of Web pages in order to measure the difference in clicks, traffic, transactions, and more between the two.

I first began using bucket testing when designing Web sites with an already established and often quite profitable user base. Because we were experimenting with very different interactions or visual presentations, we needed a way to see the impact of our changes without migrating all our existing users to a new design. Bucket testing provided a great way to send a small amount of traffic (usually less than 5%) to a different user interface without negatively impacting the bottom line if our new design had unintended negative consequences.

In this context, the point of bucket testing was to confirm we were making the right decisions when we made big changes. Since then, the technology for running bucket tests has grown, and as a result, it is easier than ever before to pit two iterations of a design against each other. This has led to bucket testing of not only of pages but also individual features, UI elements, and even details such as the text color of a set of words."   continued ...   (Via Functioning Form)


Post a Comment

<< Home

<< Home