Thursday, October 25, 2007

In Appreciation of Measures That Tell Stories

New measures give better measure of usability ...

"Not long ago, usability measures and Web analytics were few and far between. The usual standards amounted to little more than task completion, error rates, and click streams. Yet, they served us well.

Some years ago, when relaying one telling measure—how many clicks it took to find a book—to clients at a large metropolitan library group the room fell silent. Finding a book on a library web site should have been, as my father was fond of saying, “as easy as shooting fish in a barrel.” In our test sessions, however, it took eight of 12 participants an average of 6.25 clicks to find John Grisham’s book, A Painted House. The benchmark for the task was one click.

All but a couple of the participants meandered through pages looking for the best-selling book without feeling they were progressing toward their goal. Some participants clicked 18 or 20 times before giving up. Of all the performance data in our 147-page report, this one piece of information, the number of clicks it took to find the Grisham book, moved the client to take action.

“The Three Cs”

This was back in 2002. Now, of course, we have more measures in our toolbox. Or, at least, we should. While the old standards are still useful, but the digital spaces we try to improve have become much more complex and so too, have clients’ expectations for functionality and a return on their investment.

Whether you call this a Web 2.0 era or not, there is no disputing that most clients these days care more than they ever did before about the “Three Cs”: Customers, Competitors, and Conversion. Click streams have made room for bounce rates, search analytics, and so much more. If we play our cards right, we can reduce and synthesize the raw data and give our clients more meaningful information that foments action.

Emblematic Measures Have Teeth

Of all the data we report, there are certain measures that are more meaningful than others. I call the more meaningful data emblematic measures. In dictionary terms, emblematic is a “visible symbol of something abstract,” which is “truly representative.”

In our presentation to the library group, the rate for the Grisham task was emblematic. That is, the measure was representative of the library website’s greater inadequacies: its failure to fulfill the basics of its fundamental purpose and meet its customers’ needs. In turn, the measure was understandable to the client on a visceral level because it was firmly planted in their business objectives.

“Emblematic measures ensure that the data are always in the service of the business,” writes Avinash Kaushik, author of Web Analytics: An Hour a Day analyzing data with the mindset that the sole reason for the business’s existence is so that it can produce data (for us to analyze!).”

However, not all of the measures we deliver to clients are emblematic, nor should they be. Emblematic measures need to epitomize the entire study’s findings eloquently and elegantly. In layman’s terms, emblematic measures are a lot like the best line from a classic movie: It’s not the only line, but it’s the one that is remarkable, memorable, and eminently quotable.

Emblematic measures are far from prescriptive, static, or context-free, too. With every bit of user experience research we conduct and on each and every site, the measures will surely vary, given the context of testing, the sample, the tasks assigned, the business objectives for the site, the functionality being studied, and so on.

Therein lies one challenge of our daily work."    (Continued via Boxes and Arrows)    [Usability Resources]


Post a Comment

<< Home

<< Home