QA: On Errors, and Why Paying for Errors Pays Off

Jens Meiert, January 20, 2007 / June 17, 2008.

This entry is filed under Web Development, Accessibility, Usability, Design.

A pseudo-scientific approach that is intended to improve websites and webservices, and which is applicable almost anywhere:

Problems with Errors

They worsen any user experience.

Simple. When a product stops working from time to time, when you notice an author not being able to handle her mother-tongue, when there is no clue in which format a date must be entered and error messages show up without being helpful, or anything else goes wrong or works differently than expected, you certainly won’t feel any better.
Consequentially, errors undermine your (perceived) professionalism.

While there doesn’t seem to be much research on this issue, there is some. Yet years ago, the “Stanford Persuasive Technology Lab” proved that typographical errors and broken links hurt a site’s credibility ~~[…]~~, and recommended: Avoid errors of all types, no matter how small they seem. (This is just one out of ten Guidelines for Web Credibility.)
They’re a pain in the *** for every perfectionist.

Okay, that’s rather strong motivation than a real problem. Errors just suck.

Strategies

Dealing with errors is simple:

Ignore them. You did your best.
Fix them. Invest in your own quality assurance measures (testing, counter-checking, proof-reading, and stuff) and probably think about other ways how to get rid of mistakes, mistakes that will always occur and will always be there.

Beside other measures, I try to solve this problem by paying for errors people find on this website. To be more precise, I pay for any (monthly “Top 3”) errors, mistakes, and uncomelinesses since November, 2005, and I also throw in a little SEO incentive by pointing to all “quality angels’” websites. Because almost nobody seems to reward people that point out mistakes, here is why you should:

Benefits of Recompensing for Error Reports

It increases the probability that you get any hints at all, and it increases the likelihood that error reports are more specific.

That is a nice two-in-one that becomes all the more important when you think about it again. Allow me to give sources later, but as far as I know, only every tenth customer complains when something goes wrong. (Alternatively, let’s just agree on “a small percentage”.) Offering some benefits does make it far more likely that you receive information on problems. From my experience with meiert.com, the resonance is almost tenfold, despite of additional QA activities.

By the way, this positive effect has already been described as an HCI factor anyway: B.J. Fogg writes about “one hand washes the other” in human-computer context in his highly commendable book Persuasive Technology.
Service in return compensates for user experience degradation.

While this is just an “educated guess”, it certainly holds true. Errors you find and that annoy you, but that get rewarded once you drop a line while the errors get fixed, too, have an entirely different, more positive impact on the overall experience with a service. It’s clear that errors, from spelling mistakes to service breakdowns, always occur, but yet it’s not self-evident that those who point them out are taken seriously. Remember that your website or service needs your users and customers, not necessarily vice-versa.
It makes your product better – more quickly.

Getting users aboard accelerates the process of shipping an even better site, an even better product. But it’s not only that: At the same time, you will start to think and act differently. Promise, you will take a closer look at what you do, because once you know that errors and mistakes and whatever will more or less hurt you and your users, you already climbed a step towards more focus on users. And finally, that’s what we strive for, a win/win for us all.

Overall, error handling is just one aspect when it comes to better services. It’s an important one though. So please, email me when something’s wrong here. Or add a comment.

Update (January 22, 2007)

I feel very honored to be allowed to publish Donald Norman’s feedback on this post:

I think your scheme works fine on small sites, but runs into complexity issues on large ones. ~~[…]~~ this might not work on a newspaper site where there are hundreds of thousands of pages, so where it isn’t even clear to whom the error report should go, and, moreover, where the people maintaining the site often have a list of far more urgent things to fix (e.g., the next story, or pages that crash, or videos that don’t spool, or …).

And when it comes to physical products, errors might require months or even years to fix because of the complexity of the product and manufacturing process.

In my experience, errors are indeed taken seriously. But spelling and typography errors on small websites are one thing: complex errors are quite another. ~~[…]~~

Your scheme is innovative. Whether it is needed will depend upon the site. Thus, on my personal website (www.jnd.org) readers continually write to tell me about spelling, typographical, and factual answers. I try to respond immediately, both in fixing the pages they are talking about and in sending an email of thanks. For this site, the rapid response plus the thank-you seems to be enough – as subsequent emails confirm. Other sites might need monetary encouragement. But if my site were an order of magnitude or two larger (that is with 10 or 100 times more pages), I might be unable to keep up with the suggestions. At that point, monetary rewards wouldn’t help either: delimitation would be on the site side, not on the reader side.

So, although the idea is clever, I suspect the real problems lie elsewhere: in the ability of the site owners to maintain the pages.

Enjoy the most popular posts, probably including:

Comments

On January 20, 20:59 CET, Robert Nyman said:

I agree that end users should be encouraged to point out your errors . When I, for instance, see web sites or presentations full of typos, I automatically dismiss the one behind it.

Not that perfect spelling skills is necessary, per se, but that I think they should have done proper proof-reading and there are tools out there to help them out.
On January 21, 15:52 CET, Roger Johansson said:

Interesting approach! I’m not aware of anyone else doing this.

I agree with Robert that an article containing typos or bad grammar makes me trust the actual message slightly less, so I always appreciate it when readers make me aware of mistakes in my articles.
On January 21, 20:09 CET, Jens Meiert said:

Robert, Roger – thanks for stopping by!

Give it a try though as there’s much more resonance, and as rewards definitely help building better sites.
On August 3, 22:10 CEST, Robert Rother said:

Actually as I read your reward for errors I read your site more carefully and spend more time on it. Your content is great and this is a very smart marketing idea. The users want to get the reward or at least attention from your side if they find an error.

And of course your site will improve…

I will think about something similar.

Best regards,

Robert
On September 19, 16:07 CEST, Roman Schechtel said:

Your approach reminds me of Donald Knuth, the creator of TeX.

In the 80s, he claimed TeX to be bug-free and offered money for every bug to be found. This bold claim plus the moneraty incentive resulted in a) so many found bugs that he had later to lower the reward and b) a very much improved code base.

It certainly worked out for TeX, so Jens, why not claim meiert.com error-free in the next article?
On September 19, 17:10 CEST, Jens Meiert said:

I didn’t know that Donald Knuth did this, but I definitely appreciate that. However, claiming anything to be “error-free” on this site’s certainly too early

Jens Meiert