Conducting a Guerrilla Usability Test, Part I
Jens Meiert, September 25, 2007 / March 7, 2008.
This entry is filed under Usability.
It’s time for usability testing again. I plan to retest several projects, including this site as well as UITest.com, and I want to share and discuss the process of this “guerrilla testing” with you. On hand, the protocol of part I.
What I intend to do is clear, but I’m in the more or less popular situation that I don’t want to spend much money on the tests. Thus, I bank on “low budget” testing, similar to the process Roger described some time ago. But while I made good experiences with former guerrilla tests (and still suggest them as the minimum everybody should do), it will be a kind of experiment where I need to be aware of and avoid certain methodology slips anyway.
In part I, it’s all about preparation. The tests will be conducted at my home on a notebook with Windows XP unless somebody asks for using a Mac. So for now, I need to specify certain things I want to test, and I need to recruit test persons.
Preparing the Tests
The tests will target the German sections of the aforementioned sites. For meiert.com, I need to test the impact that all changes over the last months really had on the overall usability; concerning UITest.com, I’m just curious whether certain problems I suspect there to be continue to hold true, like e.g. the questionable footer navigation – the site will be redesigned and retested soon after.
For the tests, I quickly wrote down a few tasks and questions I intend to ask the probands. For example, concerning this site:
- What’s this site about?
- Where and how can you reach the author?
- Please figure out something about “Web Design”.
- Does the site offer information on “Marketing”?
- How up-to-date is the website?
- What is your overall impression, of the site; what do you like or dislike?
I still fine-tune this and UITest.com’s test, but that’s the rough direction for both of them. The “overall impression” question, by the way, will be the last question of each site’s evaluation – I’m interested in each proband’s opinions and feelings, but at first, I want to observe them performing tasks, not to listen to them.
Another problem’s not far away – how will probands be influenced when I test my site, them certainly noticing that? There’s quite a “probability” that this test gets falsified, so I need to test a “neutralized” version. Depending of how many people are interested in test participation as well as the efforts necessary, I might even consider testing my unchanged website as well as a really “alienated” version. That’s something for part II, though, as these questions being noted in a “guerrilla testing” post are great.
Recruiting Test Persons
That’s most “experimental”, and I’m really curious about the outcome: I just prepared a “poster” that I’ll hang up in certain shops and bus stops close to my home (today; I’ll take photos). Sure, I chose this method mostly due to the low costs, but also since it appears to me that it might attract more people just by the proximity.
Figure: Looking for probands.
The posting (quick and dirty PDF, 27 KB) just asks for testers, announces a “small reward”, requires “some internet experience” as well as “20-30 minutes” and allows for easy phone number pull-off. So both “how” and “what” are actually experimental, at least for me.
The reward, by the way, would mean one to two bottles of quite good red wine, and I’ve got quite a few at home. I stole this idea from BBDO anyway, they once also performed tests with wine thankyous. Unfortunately, I cannot write this on the posters – who knows how many people will be strangely motivated by that 
That’s it for now. I’m looking forward to your remarks (you may even leave suggestions for other German sites to be quickly tested as I want to include other sites as well), and I’ll keep you posted on the next steps over the next days or weeks.
Read More
Enjoy the most popular posts, probably including:
Comments
What’s your point of view? Comment.
