Monday, May 15, 2006

Variability in User Performance

Looking at differences in users...

When doing website tasks, the slowest 25% of users take 2.4 times as long as the fastest 25% of users. This difference is much higher than for other types of computer use; only programming shows a greater disparity.

Anyone who's done user testing knows that there are tremendous individual differences among users. Some people sail through user interfaces, while others get bogged down. Even if you've never performed a formal measurement study, you've probably noticed that the fastest users are much faster than the slowest ones.

To better grasp this variability, we can look at the ratio between the top and bottom quartiles of a given study's measured task times:
Q3 is the highest number in the third quartile (the slowest 25% of users).
Q1 is the highest number in the first quartile (the fastest 25% of the users).
More precisely, in Q1, 25% of users are faster and 75% are slower; in Q3, 75% of users are faster and 25% are slower. So, half of the users lie between Q1 and Q3 and the other half are evenly distributed outside this interval. We divide Q3 by Q1 to compute the Q3/Q1 ratio as a measure of individual differences between users at the low and high ends of performance.
Example Study Results
The following figure shows an example of Q1 and Q3 from one of our eyetracking studies, in which seventy-six users tried to find the location of the Agere Systems corporate headquarters using the company's website. The data plotted is for the forty-eight users who identified the correct city. (While we don't use the times from the twenty-eight users who failed the task, understanding what caused their failure is obviously important as well; see separate article on how to improve the "about us" info on a website.)"   continued ...   (Via useit)


Post a Comment

<< Home

<< Home