"Usability scores for 51 websites shows some correlation between navigation, content, and feature quality, but no connections to other usability areas.
Some user interfaces are good, some are bad; we all know that. But why do designs differ in usability?
The easy answer is that some design teams have good designers, listen to their usability specialists, and comply with documented guidelines; other teams have bad designers, either do no usability or ignore the findings, and prefer their own pet theories to established best practices.
But this easy answer just begs a second question: Why are some teams more focused on the quality of the user experience than others? Thanks to a new dataset I just collected, we can now analyze this question statistically.
It's extremely rare to have data about the usability outcomes of numerous design projects that are all focused on the same problem. Even when we do competitive usability research for our consulting clients, we test only 3–4 competing sites in the same industry because that's all we need to derive strategic usability recommendations. Testing additional competing sites would result in severely diminishing returns; it's better to spend that money testing additional iterations of the company's own design.
But we now have usability scores for 51 similar websites, thanks to a study sponsored by the Pew Charitable Trusts in which we evaluated the usability of the voter information websites from all 50 states and the District of Columbia.
Because voting laws differ between states, the sites are not literally identical. However, they are similar enough that it's fair to compare them. For example, states have different deadlines for requesting absentee ballots, but all states must inform their residents about absentee ballot rules — including the relevant deadlines — and offer voters a way to request the ballots.
Distribution of Usability Scores
The following histogram shows the distribution of usability scores for the 51 voter sites. The possible score range was from 0% to 100%, with higher numbers being better.
Note that a perfect score wouldn't indicate a site with perfect usability. A score of 100% would simply indicate that the site got full marks with respect to the current state of the art in all the usability aspects we evaluated. In actuality, the highest-scoring site in our study got a rating of only 77%, showing how far voter sites have to go relative to the best commercial websites. (E-commerce sites tend to have particularly good usability because they go out of business if people can't shop there. Although government sites also benefit from usability, it's rarely a matter of organizational survival.)
The histogram shows a fairly normal distribution of usability: most states have middling usability. There are a few states with decent usability, with 3 sites scoring above 70%. Sadly, there are more states with low usability, with 8 sites scoring below 40%. But at least there are no sites with horrendous usability. The lowest-scoring site came in at 29%, which — while truly bad — would still allow the most determined and skilled users to complete tasks on the site." (Continued via Jakob Nielsen's Alertbox) [Usability Resources]