Monday, April 21, 2008

'The Ultimate Lab Test'?

Comparing apples and oranges? ...

"PopularMechanics has performed three types of benchmarks on Apple and 'ordinary' PC machines, with the former running Mac OS X Leopard, and the latter Windows Vista. The first type of benchmark consisted of users giving ratings to things like design, ergonomics, web browsing experience, and so on. The second benchmark focused on real-world performance (launching applications, boot/shutdown times, and so on), while the third and final benchmark consisted of Geekbench and Cinebench runs. PM concludes: "The results gave us a clear winner in the performance categories, but the big surprise was how little difference we found in user preferences. Turns out, both platforms are capable and easy to use, but only one was the victor." Even before I got to the results, I noticed a whole set of problems with the benchmarks performed in this article, that would seriously skew the results.

The biggest problem is that the machines are not at all similar in specifications. They used desktops and laptops, but in both cases, the Apple machines had far less memory to go around, but did have slightly faster processors with better bus speeds. The Gateway all-in-one machine had 2GB of RAM more than the iMac, but a 400Mhz slower processor. In the laptop category, the ASUS notebook had 1GB of RAM more than the MacBook (on page 1, that is - the results page say the MacBook had 1GB more), but a 2.2Ghz processor compared to the 2.4Ghz in the MacBook. In addition, the MacBook had an integrated Intel graphics chipset, where the ASUS notebook had an Ati one. This means that the ability to draw conclusions on the performance-related benchmarks is limited, at best.

The hardware differences will also skew the battery results in the laptop comparison. The ASUS has a more powerful graphics chipset, which also drains more battery power. The MacBook had a Penryn processor, which, according to AnandTech, has a significant increase in battery life compared to its predecessor in the ASUS. In addition, the ASUS has a much larger screen (15.4" compared to 13.3") which also affects battery life.

Another big issue is that the Vista computers did not seem to have Service Pack 1 installed, which is another thing that could have seriously skewed the results - this time in favour of the Macintosh. Service Pack 1 is said to fix a set of performance-related problems, and it has been released quite a while back already - it should have been installed. When you do these comparisons, you always use the latest versions, fully patched. Common practice."    (Continued via OSnews)    [Usability Resources]


Post a Comment

<< Home

<< Home