Monday, September 04, 2006

Giving usability methods the critical eye

A good usability testing discussion ...

"Last week, UPA-DC hosted a talk by Dr. Patricia Chalmers, a usability researcher for the US Air Force. The subject, ostensibly, was usability metrics — the kinds of data collected during testing. Dr. Chalmers provided an overview of testing and discussed at a high level different kinds of metrics to collect. For the second hour, she opened the floor to discussion.

The usual themes emerged: usability professionals are unappreciated, developers are nothing but obstacles, and there’s never enough money to go around. For me, what emerged was a picture of “old school” usability — the notion that usability professionals conduct experiments within the confines of a lab and are judged based on the quality of their statistical analysis.

One attendee, for example, reminded us to distinguish between formative and summative testing. These are big words to describe the purpose of a test — as a tool during the design process, or as a means for predicting the product’s success. I asked the group whether this distinction still holds meaning within the context of the web. The assembled crowd wanted to believe that there was still a place for summative testing, that clients would pay for a big test at the end of the process to confirm the feasibilty of the design, but they didn’t convince me, much less themselves.

Though confined to the largest of the commercial sites these days, there is technology that allows us to monitor what works and what doesn’t. Amazon gradually introduces new screen designs to a small percentage of users and tracks their performance. The scale may scare you away from such an approach, but the cost of performing these “real-time studies” will diminish quickly. The contribution of a usability professional changes when data pours in constantly. Instead of designing and running tests, perhaps organizations can squeeze more value out of usability engineers by asking them what data they should be tracking and how to interpret the results."   (Continued via Greenonions)   [Usability Resources]


Post a Comment

<< Home

<< Home