Thursday, November 13, 2008

Grounded versus Speculative Reasoning in HCI

Science vs. philosophical ...

"As readers of this blog are well aware, HCI is at an interesting cross-roads. The history of the discipline is fundamentally scientific, with primary inputs from psychology and computer science. The future of the discipline appears minimally to include cultural, with the rise of affective, entertainment, domestic, social, and other culturally dense forms of computing. In its main venues, from CHI to Interactions magazine, it is crying out for approaches that will help interaction designers and HCI researchers work seriously on problems like the experiential qualities of interaction, interaction aesthetics, and so on.

I consider myself one of many voices trying to respond to that call. And perhaps one of the greatest challenges this agenda faces is the unspoken but omnipresent expectation that whatever solutions are offered will meet similar standards of scientific “rigor” that have been in place for decades. One problem, of course, is that culturally relevant knowledge is not necessarily the same kind of knowledge as scientific understandings of problem spaces.

Another problem, perhaps even worse, is the normalizing notion that traditional science has a monopoly on rigor and “serious” practices of knowledge production. This is not asserted explicitly, as a form of intellectual bigotry, but rather it comes out unconsciously, through habits of mind. And the goal of this post is to expose and subject to critique that habit of mind.

I came up against this in the course of doing a review for CHI2009. I was asked to review a paper that I immediately recognized as a work of philosophy in HCI, the kind of work that to me is responsive to the call for a more humanistic HCI. The paper explicated several widely accepted knowledge constructs in HCI, each of which were presented as a collection of principles. The paper offered an abstract, yet reasoned, analysis of these principles, suggested the existence of a gap exposed by said analysis, and offered to fill that gap with a new set of principles.

I quite liked the paper, but not all of the reviewers agreed with me. One of the reviewers in particular repeatedly claimed that the argument was “not grounded.” (The reviewer had other — and legitimate — objections, which I am not talking about here; I just want to take on this single objection.) Initially, I inferred that the reviewer was criticizing the speculative nature of the entire argument; there was no data, no study to support it. Later, that reviewer clarified and suggested that the argument would have been better if the author’s definitions of key vocabulary had been derived from scholarly literature. Instead, in one key instance it was derived from a “dictionary” (which was implied to be lazy), and in another key instance no source was offered (which was considered to be opinion-based and arbitrary).

Now, I should make very clear that although the author was roundly criticized for using insufficiently serious sources, or no sources at all, neither the reviewer nor the meta-reviewer made any suggestion whatsoever that there was anything wrong with the author’s actual use of this terminology; they did not suggest that there was any limitation, distortion, incompleteness, conflation or any other legitimate problem with the terminology. The only thing wrong with it is that its provenance was not deemed “serious” enough. As I learned in a 100-level logic class in 1993, this argument is a logical fallacy (it depends on an “appeal to authority“). Put simply, the argument was deemed problematic not because anyone could find any fault with it anywhere, but simply because it failed to showcase the apparatus of intellectual seriousness.

As someone trained in philosophy (it was my Ph.D. minor and my dissertation, recently published by Routledge, is about the philosophy of language), I have a real problem with these objections. In logic, one is allowed to stipulate premises as a part of an argument, even far more outlandish ones than were found in this paper. And building on this stipulation, one can develop a rational argument. Obviously, anyone can reasonably question the stipulation, e.g., by saying that premise P stipulated as true is in fact not true, and therefore any subsequent reasoning that depends upon its truth is irrevocably damaged. All of that is fair game. But to say that premise P can’t be taken seriously because it came from source S, without even bothering to engage in whether premise P has any value in its own right, is just a poor philosophical response."    (Continued via Interaction Culture)    [Usability Resources]


Post a Comment

<< Home

<< Home