"When I presented my suggestion for how to compare multiple design alternatives, one step involved creating what I called a Weighted Differences Matrix. In the matrix, each row represents a difference between the design alternative, which, in turn, we interpret to be a factor to help us decide which design is better. How do we know what the list of differences should be?
The process we’d use is to compare the designs side-by-side and list the differences. A method I’m fond of is to do the comparisons with two sites at a time. In this case, we’d probably start with the current design and alternative #1. We’d put them side-by-side and ask, “What makes these designs different?” Once we’d exhausted our thinking, we’d replace alternative #1 with alternative #2 and repeat the questioning, looking for new differences to add to the list.
To be complete, after comparing all the alternatives to the current design, we might try comparing them to each other. However, we can probably just eyeball the different designs to catch the last few differences that we may have missed on the first pass of comparisons. In my experience, 95% of the differences are discovered on the first pass. For most projects, that’s good enough.
Given five alternatives, we’d allocate an hour to generate this list." (Continued via UIE Brain Sparks, Jared Spool) [Usability Resources]