"In this installment of Ask UXmatters, our experts discuss how to write effective usability requirements and metrics for the redesign of a legacy public sector system.
... The Difficulty of Writing Usability Requirements
“Your classic examples are just examples, so I won’t pick on them too much,” says Whitney. “But you are right that they are not very helpful. For one thing, they raise more questions than they answer—for example, What is a novice user? What is a critical problem?”
Mike offers this opinion: “I think we overemphasize metrics when it comes to usability. They often introduce problems in terms of the validity and reliability of the data. First, ask whether a metric is a valid measure of usability. For example, the amount of time it takes to perform a task is often a usability goal—as in your example Six out of ten novice users shall perform task X in Y minutes. What if all of the participants took more than Y minutes to complete the task, but everyone said, ‘This is the best application for doing this task I have ever experienced. I’m recommending this product to all my friends.’ Does the product fail to meet the requirements regarding that task? What if all participants perform the task in less than Y minutes, but everyone says, ‘That was a horrible experience. I would never buy this product.’ Does the product satisfy requirements?
“The problem with these metrics is exactly the one you have articulated: What is a good number? Such measures are invalid unless
* you can derive a metric from prior data—For example, you might have Web metrics that show, if users cannot complete a purchase in Y minutes, the abandonment rate goes up.
* a metric is intrinsic to the product—For example, if a medical team cannot complete a triage procedure in less than Y minutes, the patient dies." (Continued via UXmatters, Janet M. Six) [Usability Resources]