Saturday, September 01, 2007

Process or Outcome? Measuring the Success of Usability

The criteria for measuring usability ...

"A friend has been working with a client for 9 months. She’s sort of been ‘usability midwife’ to the birth of their redesigned web site: a major development, loads of nasty back-end integration problems, the effort of many teams. Her role was to conduct a series of usability tests on different iterations. And she told me it had finally launched.

“So, are you pleased with it?” I asked.

“No, it’s horrible” she said. “But it’s much better than it might have been”.

Which led me to brooding on how we measure our successes.


Measuring the usability of something is quite easy: define your product, users, goals and context. Run a measurement evaluation (also known as ‘summative evaluation’). Publish your report. You’re done. There’s even an ISO standard for reporting these results: the "Common Industry Format". And you’ll find discussion of the details of how to do it in any of the standard textbooks: my own, for example.

But that’s a different matter from measuring the success of “usability”, meaning a single or series of user-centred design activities. I’m going to call this a ‘usability intervention’.

To measure the success of a usability intervention, we need to know:
- what the starting point was, ideally by measuring the usability of the product before the usability intervention
- how the usability intervention affected the product
- what the finishing point was, by measuring the usability of the product again.

Do some sums, and bingo: you’ve got a return-on-investment success story."    (Continued via Usability News - Caroline's Corner)    [Usability Resources]


Blogger Marcel Kornblum said...

This is all well and good if your budget stretches to an extra evaluation at the beginning of the process to act as a benchmark.

In my experience getting the budget agreed in the first place is hard enough; many sites that go this far have a far easier metric you can use: conversions.

1:19 PM  

Post a Comment

<< Home

<< Home