Compared to What?
Published on Jul 3, 2008 (updated Nov 19, 2022), filed under design (feed). (Share this on Mastodon or Bluesky?)
…is probably one of the most important questions there is. “Compared to what?” is the question that should be answered every time it is about data, be it through charts, in newspapers, on websites, or in conversations. Yet it is rarely asked, rarely answered, and people end up with less or even false information.
The first time I heard about “compared to what?” was when reading Tufte who writes about it in the context of information design. Ever since I am reminded of it when reading news. I wanted to write about it when coming across Cory Doctorow’s article on rare events—it may be based on a misconception as people probably aren’t that bad in understanding rare incidents, but don’t usually get an answer to the question: “Compared to what?”
“Compared to What?” in Action
Let’s go through a couple of examples.
For instance, you read about 100 people dying in car accidents each month in some city. Horrible. You probably don’t want to enter a car anymore—unless you ask for additional data. And asking how many people actually live in the city in question might be a good start. What does it mean when the answer is 2,000, talking a small village, or 4,000,000 residents, talking a large city? How do you judge these 100 car accidents in each case? “Compared to what” is the difference between staying in your apartment all day, or suing the journalist because of freaking everybody out for no reason.
Another example is your company’s performance. Let’s say that last year, your sales dropped by 10%. Bad. “Compared to what?” might make you have a closer look at your competitors’ performance. Does your 10% decrease change meaning when all your competitors lost 25% on average, or when sales generally went up 10%? Of course; your company might perform very well despite the observed development, or really bad. So again, panicking because of a single number that lacks a link to other metrics is not a good choice.
Another example involves, dare I say it, terrorist acts. We hear about suicide attacks and casualties, but we never ask how rarely they occur, in what context they happen, and how unlikely it is to be affected by an attack. Yet governments cut civil rights at an alarming rate, replacing the illusion of a threat by an illusion of safety.
Conclusions
Two more things:
“Compared to what?” should always be answered. One might, by default, consider newspaper articles unprofessional that don’t provide data for comparison. It’s not acceptable that news sources fail to provide what is necessary to assess the data being presented.
There is no obligation to ask “compared to what?” only once. You can and sometimes should go for more information in order to be able to learn and act on it.
The simple conclusion: Ask “compared to what?” when being presented with data, and make sure to answer “compared to what?” when displaying data.
About Me
I’m Jens (long: Jens Oliver Meiert), and I’m a web developer, manager, and author. I’ve been working as a technical lead and engineering manager for companies you’ve never heard of and companies you use every day, I’m an occasional contributor to web standards (like HTML, CSS, WCAG), and I write and review books for O’Reilly and Frontend Dogma.
I love trying things, not only in web development and engineering management, but also in other areas like philosophy. Here on meiert.com I share some of my experiences and views. (I value you being critical, interpreting charitably, and giving feedback.)