Thursday, October 25, 2007

The Limitations of Server Log Files for Usability Analysis

A detailed analysis about using log files for usability analyses ...

"One of the challenges faced most often by those of us in the field of usability is finding good data about user behavior quickly, accurately, and, in most cases, cheaply. In an environment where many stakeholders question the return on investment in usability, some in the industry have developed interesting ideas aimed at gathering user data. One such idea is the analysis of server log files to gather information about user behavior. On the surface, it is easy to understand the gravitation towards server logs: They’re supposedly a data source which portrays what people are doing on a site. Server logs supposedly show what people click on, which pages they view, and how they get from page to page.

Unfortunately, practitioners who espouse such methods seem to lack important technical knowledge regarding the nature of the web, the Hypertext Transfer Protocol (HTTP) and the process of caching within networks, proxies, ISPs, and browsers. These technical details greatly limit the types and quality of information that can be retrieved from server logs.

In addition to the technical limitations of server log file analysis, without information regarding exactly what the user expects to find and why he makes the choices he makes, there’s no way for us to know whether he was successful in his quest and whether that quest was satisfying. Ultimately that is the usability information we seek.

Server log files are inappropriate for gathering usability data. They are meant to provide server administrators with data about the behavior of the server, not the behavior of the user. The log file is a flat file containing technical information about requests for files on the server. Log file analysis tools merely assemble them in a conjecture-based format aimed at providing insight into user behavior. In the commentary below, I will explain why the nature of the web, the HTTP Protocol, the browser, and human behavior make it impossible to derive meaningful usability data from server logs.

First, some technical background information is needed.

... Conclusion – Server Log Analysis Is an Unreliable Tool for Usability

It is recommended that an organization not spend extensive amounts of time and money to gain usability data from server logs. An organization would be better served by hiring an experienced human factors engineer to perform an expert review or conduct a formal study with users. The results would be much quicker, more accurate, and more informative."    (Continued via Boxes and Arrows)    [Usability Resources]

AWStats File - Usability, User Interface Design

AWStats File


Post a Comment

<< Home

<< Home