Monday, July 16, 2007

Embracing the Un-Science of Qualitative Research Part One - Small Sample Sizes are Super

Qualitative usability testing (Part 2 below) ...

"If you’re into qualitative research at all, it wouldn’t have taken long before you had someone ask you about the statistical significance of your research and how you could back your findings with such a small sample size, or to find others out there trying to make qualitative research look more scientific by trying to extract hard data from it.

There are three main ways that you can try to make qualitative research look more scientific, being:

1. Use a relatively large sample size
2. Ensure that your test environment doesn’t change
3. Ensure that your test approach doesn’t change (don’t change the script, and stick to it)

Now, there are some times when one or more of these tactics is appropriate, but conversely, in many instances it has been my experience that by breaking these rules, you are able to get much greater insight into the research question(s) you have set yourself.

There are many different kinds of qualitative research study, so in the interests of clarity, let’s pick one just like I’ve been working on this week - a lab based combination of interview & a wee bit of usability which is intended to ensure that my client’s proposition is sound, that it is being well communicated, that the users understand what the service is and how it works, and to weed out any critical usability issues.

In the interest of not making you read an enormous post, I’ve divided this into three parts. So, let’s start with part one - a large sample size. Now… to the best of my knowledge there is no scientific way to determine the correct number of participants in a qualitative research study. Now, I’m no statistician (if you are, please feel free to weigh in here), but it is my understanding that the likelihood of reaching a statistically significant result using the methodology I’ve described above, is pretty much nil. Not that it’s impossible, but you’d have to do a heck of a lot of interviewing."    (Continued via disambiguity)    [Usability Resources]


Post a Comment

<< Home

<< Home