Jens Meiert

QA: On Errors, and Why Paying for Errors Pays Off

Jens O. Meiert, January 20, 2007 (↻ August 15, 2014).

This entry has been written by Jens the .

A pseudo-scientific approach that is intended to improve websites and services, and which is applicable almost anywhere:

Problems with Errors

  1. They worsen any user experience.

    Simple. When a product stops working from time to time, when you notice an author not being able to handle her mother-tongue, when there is no clue in which format a date must be entered and error messages show up without being helpful, or anything else goes wrong or works differently than expected, you certainly won’t feel any better.

  2. Consequentially, errors undermine your (perceived) professionalism.

    While there doesn’t seem to be much research on this issue, there is some. Yet years ago, the “Stanford Persuasive Technology Lab” proved that “typographical errors and broken links hurt a site’s credibility […],” and recommended: “Avoid errors of all types, no matter how small they seem.” (This is just one out of ten Guidelines for Web Credibility.)

  3. They’re a pain in the *** for every perfectionist.

    Okay, that’s rather strong motivation than a real problem. Errors just suck.

Strategies

Dealing with errors is simple:

Beside other measures, I try to solve this problem by paying for errors people find on this website. To be more precise, I pay for any (monthly “Top 3”) errors, mistakes, and uncomelinesses since November, 2005, and I also throw in a little SEO incentive by pointing to all “quality angels’” websites. Because almost nobody seems to reward people that point out mistakes, here is why you should:

Benefits of Recompensing for Error Reports

  1. It increases the probability that you get any hints at all, and it increases the likelihood that error reports are more specific.

    That is a nice two-in-one that becomes all the more important when you think about it again. Allow me to give sources later, but as far as I know, only every tenth customer complains when something goes wrong. (Alternatively, let’s just agree on “a small percentage.”) Offering some benefits does make it far more likely that you receive information on problems. From my experience with meiert.com, the resonance is almost tenfold, despite of additional QA activities.

    By the way, this positive effect has already been described as an HCI factor anyway: B.J. Fogg writes about “one hand washes the other” in human-computer context in his highly commendable book Persuasive Technology.

  2. Service in return compensates for user experience degradation.

    While this is just an “educated guess,” it certainly holds true. Errors you find and that annoy you, but that get rewarded once you drop a line while the errors get fixed, too, have an entirely different, more positive impact on the overall experience with a service. It’s clear that errors, from spelling mistakes to service breakdowns, always occur, but yet it’s not self-evident that those who point them out are taken seriously. Remember that your website or service needs your users and customers, not necessarily vice-versa.

  3. It makes your product better—more quickly.

    Getting users aboard accelerates the process of shipping an even better site, an even better product. But it’s not only that: At the same time, you will start to think and act differently. Promise, you will take a closer look at what you do, because once you know that errors and mistakes and whatever will more or less hurt you and your users, you already climbed a step towards more focus on users. And finally, that’s what we strive for, a win/win for us all.

Overall, error handling is just one aspect when it comes to better services. It’s an important one though. So please, email me when something’s wrong here. Or add a comment.

Update (January 22, 2007)

I feel honored to be allowed to publish Donald Norman’s feedback on this topic:

I think your scheme works fine on small sites, but runs into complexity issues on large ones. […] this might not work on a newspaper site where there are hundreds of thousands of pages, so where it isn’t even clear to whom the error report should go, and, moreover, where the people maintaining the site often have a list of far more urgent things to fix (e.g., the next story, or pages that crash, or videos that don’t spool, or…).

And when it comes to physical products, errors might require months or even years to fix because of the complexity of the product and manufacturing process.

In my experience, errors are indeed taken seriously. But spelling and typography errors on small websites are one thing: complex errors are quite another. […]

Your scheme is innovative. Whether it is needed will depend upon the site. Thus, on my personal website (www.jnd.org) readers continually write to tell me about spelling, typographical, and factual answers. I try to respond immediately, both in fixing the pages they are talking about and in sending an email of thanks. For this site, the rapid response plus the thank-you seems to be enough—as subsequent emails confirm. Other sites might need monetary encouragement. But if my site were an order of magnitude or two larger (that is with 10 or 100 times more pages), I might be unable to keep up with the suggestions. At that point, monetary rewards wouldn’t help either: delimitation would be on the site side, not on the reader side.

So, although the idea is clever, I suspect the real problems lie elsewhere: in the ability of the site owners to maintain the pages.

Comments (Closed)

  1. On January 20, 2007, 20:59 CET, Robert Nyman said:

    I agree that end users should be encouraged to point out your errors . When I, for instance, see web sites or presentations full of typos, I automatically dismiss the one behind it.

    Not that perfect spelling skills is necessary, per se, but that I think they should have done proper proof-reading and there are tools out there to help them out.

  2. On January 21, 2007, 15:52 CET, Roger Johansson said:

    Interesting approach! I’m not aware of anyone else doing this.

    I agree with Robert that an article containing typos or bad grammar makes me trust the actual message slightly less, so I always appreciate it when readers make me aware of mistakes in my articles.

  3. On January 21, 2007, 20:09 CET, Jens O. Meiert said:

    Robert, Roger—thanks for stopping by!

    Give it a try though as there’s much more resonance, and as rewards definitely help building better sites.

  4. On August 3, 2007, 22:10 CEST, Robert Rother said:

    Actually as I read your reward for errors I read your site more carefully and spend more time on it. Your content is great and this is a very smart marketing idea. The users want to get the reward or at least attention from your side if they find an error.

    And of course your site will improve…

    I will think about something similar.

    Best regards,

    Robert

  5. On September 19, 2007, 16:07 CEST, Roman Schechtel said:

    Your approach reminds me of Donald Knuth, the creator of TeX.

    In the 80s, he claimed TeX to be bug-free and offered money for every bug to be found. This bold claim plus the moneraty incentive resulted in a) so many found bugs that he had later to lower the reward and b) a very much improved code base.

    It certainly worked out for TeX, so Jens, why not claim meiert.com error-free in the next article? ;)

  6. On September 19, 2007, 17:10 CEST, Jens O. Meiert said:

    I didn’t know that Donald Knuth did this, but I definitely appreciate that. However, claiming anything to be “error-free” on this site’s certainly too early ;)

  7. On February 25, 2009, 15:52 CET, Italie said:

    How did you manage to have the great Donald Norman’s feedback? He seems such a distant guy…

  8. On February 26, 2009, 17:25 CET, Fliesen said:

    Jens, I like your article. But I have to agree to Donald, that it will work only well on smaller sites. I will test it on my own site and maybe report later!

  9. On March 3, 2009, 20:16 CET, Josh said:

    Really interesting idea, how do you handle payments of the “rewards”? I wonder if tthis make users assume their are mistakes and your website is relatively new?

  10. On March 14, 2009, 22:30 CET, Jay Morris said:

    Great post. I cant believe im just now seeing it. Great idea and approach. I am going to to try this out on my next site/blog i put up and see cross compare against another.

    Thanks again!

  11. On March 16, 2009, 14:14 CET, Jason Thorton said:

    Wonderful idea thanks for this post going to try it on one of my slacking websites. Is paying with paypal good. or you recommend something else

  12. On April 1, 2009, 4:36 CEST, Eric said:

    Wonderful idea thanks for this post going to try it on one of my slacking websites. Is paying with paypal good. or you recommend something else
    Thanks.

  13. On April 18, 2009, 18:38 CEST, France Flights said:

    Proper content is the key to the success of any websites. It is really annoying to see a site with full of typos and broken links. I agree to err is human but at this level you need to maintain your professionalism and have a good team to fix these silly errors.

  14. On May 20, 2009, 10:30 CEST, armyshop said:

    It is nearly impssible to create websites - which are fastidious - without some code-mistakes. To have a perfect result you need a code specialist. A normal user cant creat a perfekt code - in my opinion.

  15. On October 8, 2009, 20:52 CEST, Florian said:

    Great idea to reward people for pointin out mistakes!! only a few pages ask for errors to be reported to them ! This IS a great thing to do, because errors do undermine a perfect appearance. Greets

  16. On December 3, 2009, 9:32 CET, John said:

    Imagine you are working on some crucial work project and an error occurs abruptly which might lead to data loss and as a result this can disorganize you and disrupt all your plans. Errors can make you lose concentration since the messages can really get annoying each time they occur. Therefore, the remedy you are offering is set to be very helpful mostly for individuals who experience this problem regularly. From your website, I can find the possible techniques I will require to fix any errors that might occur on my system. I will surely point out any mistakes I observe with your website and remedies that you can offer, you can bet on that.

Read More

Have a look at the most popular posts, eventually including:

Found a mistake? Reward! Email me, jens@meiert.com.

You are here: meiert.comArchive for 2007 → QA: On Errors, and Why Paying for Errors Pays Off

Last update: August 15, 2014.