Thursday, January 04, 2007

Thumbnail: Rolf Molich

An interview with the inventor of CUE tests ...

"Rolf Molich is “curious.” He cites that curiosity as the impetus behind his CUE, or Comparative Usability Evaluation, studies ( These now-world-famous studies look at how usability tests and expert reviews are actually carried out in practice – and about how reproducible they really are. There’s a lot more to Rolf, though, than his CUE studies.

... CUE
“Lots of people were talking about how wonderful usability studies were” Rolf points out. “I tended to agree with them because I had seen myself the enormous political impact a usability study can have. But then I was curious and said, ‘Are they reproducible?’”

He shared his thoughts on a usability list serv, got four volunteers, then launched CUE-1. This study looked at a Windows application, having the participants test it in any way they saw fit to try and identify usability problems.

The results were presented at UPA 1998. Though “everyone in the crowd thought it was interesting,” there was also some resistance, something that Rolf would become used to over the years. He notes, “People said, ‘This just can’t be true. The results just can’t be this diverse.”

But they were. In CUE-2, Rolf found that 75% of the problems generated by nine teams running simultaneous, independent tests of Hotmail were unique, something very similar to what he had found in CUE-1.

The next study, CUE-3, took a different approach. Instead of doing tests, he had 12 teams do an expert review on a car reservation site ( Though this study “has not been published in any great detail,” it did serve as a springboard for CUE-4.

That study, the largest to date, was part of a workshop at CHI 2003. It divided up 17 teams, with half doing tests and half doing expert reviews. Rolf notes, “One of the things we wanted to find out was, ‘Is there a difference?’ Tests are more expensive, but do they give better results – more quality, more reliability, more reproducible results? We found that not to be the case."    (Continued via The UPA Voice)    [Usability Resources]


Post a Comment

<< Home

<< Home