Wednesday, November 01, 2006

How many users should you test with in usability testing? (Latest Research)

An argument for 15 users for user testing...

"Question: How many users do you need to test with for a usability test?

Answer 1: = 5 users (Jakob Nielsen and Thomas Landauer, 1993).
Answer 2: = 15 users (Laurie Faulkner, 2004), PDF file.

So, which is it, 5 or 15? And why are we arguing about an extra 10 users, doesn't one need to test with at least 100 or more users for statistical significance, accuracy and validity?

Statistical Validity in Usability Testing

Usability research is largely qual-itative, or driven by insight (why users don't understand or why they are confused). Qual-itative research follows different research rules to quant-itative research and it is typical that sample size is low (i.e. 15 or 20 participants).

The end result of usability testing is not statistical validity per say (the outcome of quant-itative research) but verification of insights and assumptions based on behavioral observation (the outcome of qual-itative research).

Why don't we do large numbers in usability testing?

• We are looking for behavioral based insight (what they do).
• Statistics tell half the story and often are devoid of context (e.g. Why did they fail?). Also one of the major problems with gaining insight from web analytics (website traffic statistics).
• Our objective is to apply findings to fix design problems in a corporate setting (not academic analysis).
• Research shows that even with low numbers, you can gain valid data.
• Usability testing is being used industry-wide and has been for past 25 years. Experts, authors and academics put their reputations and credentials behind the methodology."    (Continued via UsabilityLabRenta)    [Usability Resources]


Post a Comment

<< Home

<< Home