QA: On Errors, and Why Paying for Errors Pays Off
Published on January 20, 2007 (⻠April 7, 2024), filed under Development and Everything Else (RSS feed for all categories).
This and many other posts are also available as a pretty, well-behaved ebook: On Web Development. And speaking of which, hereâs a short treatise just about managing the quality of websites: The Little Book of Website Quality Control (updated).
A pseudo-scientific approach to improve websites and services, and that is applicable almost anywhere:
Problems With Errors
-
They worsen the user experience.
When a product stops working from time to time, when you notice an author not mastering their mother-tongue, when there is no clue in which format a date must be entered and error messages show up that arenât helpful, or anything else goes wrong or works differently than expected, itâs not a great experience.
-
Consequentially, errors undermine your reputation.
While there doesnât seem to be much research on this issue, there is some. A few years ago, the Stanford Persuasive Technology Lab proved that âtypographical errors and broken links hurt a siteâs credibility
[âŠ],â and recommended: âAvoid errors of all types, no matter how small they seem.â (This is one of ten Guidelines for Web Credibility.) -
Theyâre a pain in the *** for a perfectionist.
Okay, thatâs motivation rather than a problem. Errors suck.
Strategies
Dealing with errors may be simple:
-
Ignore them. Or:
-
Fix them. Invest in your own quality assurance measures (proof-reading, testing, &c.) and think about other ways how to get rid of mistakes, mistakes that will always occur and will always be there.
Beside these measures, I try to solve this problem by paying for errors people find on this website. To be more precise, I pay for errors, mistakes, and imperfections since November 2005, and I also throw in a little SEO incentive by pointing to all contributorsâ websites. Because very few reward people that point out mistakes, here is why you should:
Benefits of Paying for Error Reports
-
It increases the probability that you get reports at all, and it increases the likelihood that these are specific.
This is a nice two-in-one. Allow me to give sources later, but as far as I know, only every tenth customer complains when something goes wrong. (Alternatively, letâs agree on âa small percentage.â) Offering some benefits does make it more likely that you receive information on problems. From my experience with meiert.com, the resonance is almost tenfold, despite additional QA activities.
By the way, this positive effect has already been described as an HCI factor: B.J. Fogg writes about âone hand washes the otherâ in human-computer context in his highly commendable book Persuasive Technology.
-
Service in return compensates for the user experience degradation caused by the error.
This is a guess I place confidence in. Errors you find and that annoy you, but that get rewarded once you drop a line while the errors get fixed, have a different, more positive impact on the overall experience. Itâs clear that errors, from spelling mistakes to service breakdowns, always occur, but itâs not clear that those who point them out are appreciated like this. Remember that your website or service needs your users and customers, not necessarily vice-versa.
-
It makes your product better, more quickly.
Getting users aboard accelerates the process of shipping an even better site, an even better product. But itâs not only that: At the same time, you will start to think and act differently. You will take a closer look at what you do, because when you realize that errors and mistakes hurt you and your users, you take another step towards more focus on users. And finally, thatâs what we strive for, a win/win for everyone.
Overall, error handling is just one aspect when it comes to better services. Itâs an important one though. (So please, email me when somethingâs wrong here. Or add a comment.)
Update (January 22, 2007)
I feel honored to publish Donald Normanâs feedback on this:
I think your scheme works fine on small sites, but runs into complexity issues on large ones.
[âŠ]this might not work on a newspaper site where there are hundreds of thousands of pages, so where it isnât even clear to whom the error report should go, and, moreover, where the people maintaining the site often have a list of far more urgent things to fix (e.g., the next story, or pages that crash, or videos that donât spool, orâŠ).And when it comes to physical products, errors might require months or even years to fix because of the complexity of the product and manufacturing process.
In my experience, errors are indeed taken seriously. But spelling and typography errors on small websites are one thing: complex errors are another.
[âŠ]Your scheme is innovative. Whether it is needed will depend upon the site. Thus, on my personal website (www.jnd.org) readers continually write to tell me about spelling, typographical, and factual answers. I try to respond immediately, both in fixing the pages they are talking about and in sending an email of thanks. For this site, the rapid response plus the thank-you seems to be enoughâas subsequent emails confirm. Other sites might need monetary encouragement. But if my site were an order of magnitude or two larger (that is with 10 or 100 times more pages), I might be unable to keep up with the suggestions. At that point, monetary rewards wouldnât help either: the limitation would be on the site side, not on the reader side.
So, although the idea is clever, I suspect the real problems lie elsewhere: in the ability of the site owners to maintain the pages.
About Me
Iâm Jens (long: Jens Oliver Meiert), and Iâm a frontend engineering leader and tech author/publisher. Iâve worked as a technical lead for companies like Google and as an engineering manager for companies like Miro, Iâm a contributor to several web standards, and I write and review books for OâReilly and Frontend Dogma.
I love trying things, not only in web development (and engineering management), but also in other areas like philosophy. Here on meiert.com I share some of my views and experiences. (Be critical, interpret charitably, and send feedback.)
Comments (Closed)
-
On January 20, 2007, 20:59 CET, Robert Nyman said:
I agree that end users should be encouraged to point out your errors . When I, for instance, see web sites or presentations full of typos, I automatically dismiss the one behind it.
Not that perfect spelling skills is necessary, per se, but that I think they should have done proper proof-reading and there are tools out there to help them out.
-
On January 21, 2007, 15:52 CET, Roger Johansson said:
Interesting approach! Iâm not aware of anyone else doing this.
I agree with Robert that an article containing typos or bad grammar makes me trust the actual message slightly less, so I always appreciate it when readers make me aware of mistakes in my articles.
-
On January 21, 2007, 20:09 CET, Jens Oliver Meiert said:
Robert, Rogerâthanks for stopping by!
I believe such incentives to help, so give it a try!
-
On August 3, 2007, 22:10 CEST, Robert Rother said:
Actually as I read your reward for errors I read your site more carefully and spend more time on it. Your content is great and this is a very smart marketing idea. The users want to get the reward or at least attention from your side if they find an error.
And of course your site will improveâŠ
I will think about something similar.
Best regards,
Robert
-
On September 19, 2007, 16:07 CEST, Roman Schechtel said:
Your approach reminds me of Donald Knuth, the creator of TeX.
In the 80s, he claimed TeX to be bug-free and offered money for every bug to be found. This bold claim plus the moneraty incentive resulted in a) so many found bugs that he had later to lower the reward and b) a very much improved code base.
It certainly worked out for TeX, so Jens, why not claim meiert.com error-free in the next article? đ
-
On September 19, 2007, 17:10 CEST, Jens Oliver Meiert said:
I didnât know that Donald Knuth did this!
Claiming anything to be âerror-freeâ on this site is certainly too early đ
-
On February 25, 2009, 15:52 CET, Italie said:
How did you manage to have the great Donald Normanâs feedback? He seems such a distant guyâŠ
-
On February 26, 2009, 17:25 CET, Fliesen said:
Jens, I like your article. But I have to agree to Donald, that it will work only well on smaller sites. I will test it on my own site and maybe report later!
-
On March 3, 2009, 20:16 CET, Josh said:
Really interesting idea, how do you handle payments of the ârewardsâ? I wonder if tthis make users assume their are mistakes and your website is relatively new?
-
On March 14, 2009, 22:30 CET, Jay Morris said:
Great post. I cant believe im just now seeing it. Great idea and approach. I am going to to try this out on my next site/blog i put up and see cross compare against another.
Thanks again!
-
On March 16, 2009, 14:14 CET, Jason Thorton said:
Wonderful idea thanks for this post going to try it on one of my slacking websites. Is paying with paypal good. or you recommend something else
-
On April 1, 2009, 4:36 CEST, Eric said:
Wonderful idea thanks for this post going to try it on one of my slacking websites. Is paying with paypal good. or you recommend something else
Thanks. -
On April 18, 2009, 18:38 CEST, France Flights said:
Proper content is the key to the success of any websites. It is really annoying to see a site with full of typos and broken links. I agree to err is human but at this level you need to maintain your professionalism and have a good team to fix these silly errors.
-
On May 20, 2009, 10:30 CEST, armyshop said:
It is nearly impssible to create websites - which are fastidious - without some code-mistakes. To have a perfect result you need a code specialist. A normal user cant creat a perfekt code - in my opinion.
-
On October 8, 2009, 20:52 CEST, Florian said:
Great idea to reward people for pointin out mistakes!! only a few pages ask for errors to be reported to them ! This IS a great thing to do, because errors do undermine a perfect appearance. Greets
-
On December 3, 2009, 9:32 CET, John said:
Imagine you are working on some crucial work project and an error occurs abruptly which might lead to data loss and as a result this can disorganize you and disrupt all your plans. Errors can make you lose concentration since the messages can really get annoying each time they occur. Therefore, the remedy you are offering is set to be very helpful mostly for individuals who experience this problem regularly. From your website, I can find the possible techniques I will require to fix any errors that might occur on my system. I will surely point out any mistakes I observe with your website and remedies that you can offer, you can bet on that.
Read More
Maybe of interest to you, too:
- Next: Why âConditional Commentsâ Are Bad, Repeat: Bad
- Previous: 1 + 1 = 3: Explaining Busyness and Background Noise on Websites
- More under Development or Everything Else
- More from 2007
- Most popular posts
Looking for a way to comment? Comments have been disabled, unfortunately.
Get a good look at web development? Try WebGlossary.infoâand The Web Development Glossary 3K (2023). With explanations and definitions for thousands of terms of web development, web design, and related fields, building on Wikipedia as well as MDN Web Docs. Available at Apple Books, Kobo, Google Play Books, and Leanpub.