Revitalizing SUS, the System Usability Scale
Post from April 23, 2007 (↻ November 25, 2016), reflecting Jens the Developer.
This and many other posts are also available as a pretty, well-behaved e-book: On Web Development.
About 20 years ago, John Brooke published the concept of a “System Usability Scale,” a “reliable, low-cost usability scale that can be used for global assessments of systems usability.” SUS is based on a Likert scale questionnaire with standardized content that gives an overall usability and user satisfaction index (ranging from 0 to 100).
Figure: Sample statement from SUS.
While there’s no need to repeat everything written in that PDF, I may cite some important paragraphs from it:
In response to these requirements, a simple usability scale was developed. The System Usability Scale (SUS) is a simple, ten-item scale giving a global view of subjective assessments of usability.
The technique used for selecting items for a Likert scale is to identify examples of things which lead to extreme expressions of the attitude being captured.
It can be seen that the selected statements actually cover a variety of aspects of system usability, such as the need for support, training, and complexity, and thus have a high level of face validity for measuring usability of a system.
The SU scale is generally used after the respondent has had an opportunity to use the system being evaluated, but before any debriefing or discussion takes place. Respondents should be asked to record their immediate response to each item, rather than thinking about items for a long time.
To calculate the SUS score, first sum the score contributions from each item. Each item’s score contribution will range from 0 to 4. For items 1,3,5,7,and 9 the score contribution is the scale position minus 1. For items 2,4,6,8 and 10, the contribution is 5 minus the scale position. Multiply the sum of the scores by 2.5 to obtain the overall value of SU.
Please note that feedback from John Brooke, Hewlett-Packard (which once indirectly acquired the original copyright holder Digital Equipment Corporation), and Serco Usability Services is still missing. However, “SUS has been made freely available for use in usability assessment.” And wishes to be used more often.
About the Author
Jens Oliver Meiert is a German philosopher and developer (Google, W3C, O’Reilly). He experiments with arts and adventure. Here on meiert.com he shares and generalizes and exaggerates some of his thoughts and experiences.
I came across “SUS” a while ago and like to apply it in my diploma thesis.
Do have any experience using it?
How would you rate the results?
Jan, unfortunately, I can’t publish any results of former SUS cases of application, yet I still hope to use this method in large-scale projects (this “missing large-scale implementations” problem appears to be popular, as far as I remember Tom Tullis). So far SUS is quite insightful though.
By the way, please also see UPA’s “A Comparison of Questionnaires for Assessing Website Usability” (PDF, 273 KB).
On August 18, 2007, 15:45 CEST, John Brooke said:
What feedback were you hoping for? Anybody who wants to get in touch with me about SUS is welcome to do so (mail john.brooke at contingent-solutions.com). SUS seems to have taken on a life of its own, and as your comment notes, it’s 20 years old. I think it still meets its original goal of being system-independent and giving a simple, reliable global assessment of subjective usability (despite various attempts to rewrite it to make it more “relevant”, which I would have to point out effectively invalidates it).
John, great to get your feedback. I wasn’t a 100% sure if it’s really okay to publish the Serco paper here, so I tried to contact you and others involved… without success, just until now.
On October 26, 2007, 13:22 CEST, John Brooke said:
I wrote a chapter on SUS in a book published in 1996 (Brooke, J. (1996). “SUS: a ‘quick and dirty’ usability scale”. In P. W. Jordan, B. Thomas, B. A.
Weerdmeester & I. L. McClelland, (Eds.) “Usability Evaluation in Industry”, pp. 189-194. London: Taylor & Francis.).
There are a number of interesting comparative studies that have been done by Thomas Tullis and others that show that SUS tends to give a reliable result even when used with small samples. Jurek Kirakowski in Cork also found that there’s a very high correlation between SUS scores and general usability scores on the SUMI scale that was developed as part of the MUSiC project, and there is a high level of reliability for both questionnaires.
There are various electronic copies knocking about the Internet, and it continues to be used - Google Scholar has about 150 citations of its use, many in last couple of years. And if anybody can’t find one out there, I can let them have a copy of the chapter if they want to drop me an email.
Anyway, I appreciate the fact that you’re encouraging its use, and hope that people find it useful for many years to come. It seems to have stood the test of time….
On March 3, 2009, 17:30 CET, R. Martin said:
I was wanting to quote Brooke from that article in my thesis. May I use the citation he gave below? Also, are we allowed to duplicate the scale in full?
On August 27, 2009, 14:46 CEST, Kathryn Sapnas said:
Thank you for free access to the SUS. I am teaching a class to doctoral students in Nursing at the University of Miami School of Nursing and Health Sciences and will be introducing the tool to my Fall 2009 class. The students will be given an assignment where they will use the tool to evaluate the usability of a Nursing website.
Katrhyn G. Sapnas, PhD, RN, CNOR
University of Miami School of Nursing and Health Sciences
Responding quite formally:
Mr Martin, apologies for the delay—my understanding is that quoting should be fine. Maybe ping Mr Brooke to err on the safe side.
Ms Sapnas, maybe the follow-up post, which includes a template, is useful for your plans too. Thanks for your comment!
On September 24, 2009, 2:04 CEST, Carolin said:
There’s also some interesting follow-up work on the SUS which adds adjective ratings to the SUS scores:
A Bangor, P Kortum, J Miller: “Determining what individual SUS scores mean: Adding an adjective rating scale” (2009), Journal of Usability Studies Vol 4, Issue 3, pp114-123
On December 4, 2009, 19:03 CET, Thierry Koblentz said:
“SUS – A quick and dirty usability scale.”
That’s a great read, thanks!
Have a look at the most popular posts, possibly including:
Looking for a way to comment? Comments have been disabled, unfortunately.