Website Issues: On the Relevance of Audience Size and Impact

Published on May 19, 2023 (↻ July 4, 2023), filed under and (RSS feed).

Did you ever feel funny about the fuss about conformance issues?

Have you wondered why security issues would cause fire drills?

Or, did you at any time suspect a discrepancy between the severity of accessibility issues and the (lack of) urgency with which they’re being resolved?

Observations and questions of this type go back to everyone’s understanding of issues—in some cases indicating a fine sense for them, and in others an incomplete understanding, or an unbalanced treatment, as when issue ends up equaling issue.

While everyone knows that issues aren’t all the same—that’s why we work with different priority and severity levels in our issue management systems—, somehow, conformance, security, accessibility, performance, content, any issues may at some point all feel kind of the same, and be handled accordingly.

Issues in Relation to Audience Size and Impact on the Audience

When we keep it simple by lumping together said groups of issues—all accessibility issues in one bucket, security issues in another, &c.—, estimate the share of users they roughly affect on average, and the impact it has on them, we obtain another perspective:

Chart with an audience size x-axis, an audience impact y-axis, and different issues mapped according to what percentage of the audience they affect, and what impact they have on that audience. While it’s possible and perhaps desirable to map this out yourself, content issues (large audience, low impact), security issues (large audience, high impact), accessibility issues (smaller audience, large impact), and conformance, SEO, and typography issues (small audience, low impact) make for the corners of the chart.

Figure: A model to map website issues.

Or maybe:

The same chart but with general areas drawn for each of the issue groups, to help counter the idea that there was one specific spot on the chart where issues would need to be located.

First, what do these issues refer to?

Largely what you may connect with them, but here are some examples:

The chart could map design, internationalization and localization, legal, and other groups of issues, too, which I may include in future iterations. Ideally, the model enables you to allocate these on the chart, too.

A Model to Enable Better Prioritization

How is the chart to be read?

First and foremost, this is a model. There are to be exceptions—some performance issues, for example, may ruin it for an entire audience; and conformance issues (here, validation issues) could also be so bad that an entire service becomes inaccessible, meaning a dramatic user impact. It’s also possible that there are factors other than audience size and the impact issues have on them that would allow for an even clearer look at website issues.

The general approach to this, however, is to work with a group of issues, estimate the share of users they roughly affect—generous but non-cumulative—, and how severe the impact of the issues can be on them.

We can observe, then, why conformance may justly get little attention: In most cases, from a user perspective, no one is affected, and even if they are, the impact likely is negligible. (Yet!—this doesn’t make shipping invalid code more professional.)

Here we notice why security is such a priority: Typically, we can assume a large audience to be affected, and that the effects are major. If a health app gets powned and finds their users’ data open on the Web, then that may—and probably should—affect the business.

And we have more to take away from this: Accessibility, for example, usually affects fewer people directly than security, performance, or content issues. But—these issues tend to be much more impactful, i.e., they can ruin the user experience, and prevent access altogether.

This insight alone may make the classification (or the exploration of similar classifications) worthwhile: We can customize our responses to issues better if we have clarity about how many users are how affected.

Audience Size and Impact in Your Work

What does this mean for your work?

If you find yourself treating issues largely the same way, this may and probably should mean adjustments to your prioritization.

It can mean to de-prioritize conformance, typography, and even SEO issues. From a user perspective, few of them may warrant a priority higher than P4.

It can mean to push on concerted efforts and initiatives to resolve security, accessibility, and content issues. “Fix-its”—group-wide bug-fixing and refactoring events—for these issues may make a real difference.

From there it may depend on whether you find it more important to fix issues for more people (as with content defects), or for those who are more impacted (as with accessibility problems). But you may need to make a call, and would benefit from it.

(Generally, however, you still want to stay away from one-dimensional prioritization and optimization.)

Personally, I’ll be excited to review responses and alterations to this model. I’m likely to invite this to more scrutiny, perhaps at Miro, try and test it, especially when it comes to the prioritization of accessibility issues. And I may report back.

Many thanks to Thomas Steiner and Jad Joubran for reviewing and sharing feedback on both chart and post, as well as members of the Miro Accessibility team for spontaneous suggestions for the charts!

Toot about this?

About Me

Jens Oliver Meiert, on September 30, 2021.

I’m Jens, and I’m an engineering lead and author. I’ve worked as a technical lead for companies like Google, I’m close to W3C and WHATWG, and I write and review books for O’Reilly and Frontend Dogma. I love trying things, not only in web development, but also in other areas like philosophy. Here on meiert.com I share some of my views and experiences.

If you have a question or suggestion about what I write, please leave a comment (if available) or a message. Thank you!