Website Issues: On the Relevance of Audience Size and Impact

Published on May 19, 2023 (↻ July 4, 2023), filed under Development and Management (RSS feed for all categories).

Did you ever feel funny about the fuss about conformance issues?

Have you wondered why security issues would cause fire drills?

Or, did you at any time suspect a discrepancy between the severity of accessibility issues and the (lack of) urgency with which they’re being resolved?

Observations and questions of this type go back to everyone’s understanding of issues—in some cases indicating a fine sense for them, and in others an incomplete understanding, or an unbalanced treatment, as when issue ends up equaling issue.

While everyone knows that issues aren’t all the same—that’s why we work with different priority and severity levels in our issue management systems—, somehow, conformance, security, accessibility, performance, content, any issues may at some point all feel kind of the same, and be handled accordingly.

Issues in Relation to Audience Size and Impact on the Audience

When we keep it simple by lumping together said groups of issues—all accessibility issues in one bucket, security issues in another, &c.—, estimate the share of users they roughly affect on average, and the impact it has on them, we obtain another perspective:

Or maybe:

First, what do these issues refer to?

Largely what you may connect with them, but here are some examples:

• Accessibility issues: WCAG violations, any problems impairing access to content or services
• Conformance issues: invalid code (i.e., here not including legal or other compliance)
• Content issues: typos or factual errors
• Performance issues: anything that slows down actual or perceived performance, that can reasonably be avoided
• Security issues: any security-related defect
• SEO issues: code- or content-related barriers or opportunities for higher search engine rankings
• Typography issues: poor legibility, poorly crafted type

The chart could map design, internationalization and localization, legal, and other groups of issues, too, which I may include in future iterations. Ideally, the model enables you to allocate these on the chart, too.

A Model to Enable Better Prioritization

How is the chart to be read?

First and foremost, this is a model. There are to be exceptions—some performance issues, for example, may ruin it for an entire audience; and conformance issues (here, validation issues) could also be so bad that an entire service becomes inaccessible, meaning a dramatic user impact. It’s also possible that there are factors other than audience size and the impact issues have on them that would allow for an even clearer look at website issues.

The general approach to this, however, is to work with a group of issues, estimate the share of users they roughly affect—generous but non-cumulative—, and how severe the impact of the issues can be on them.

We can observe, then, why conformance may justly get little attention: In most cases, from a user perspective, no one is affected, and even if they are, the impact likely is negligible. (Yet!—this doesn’t make shipping invalid code more professional.)

Here we notice why security is such a priority: Typically, we can assume a large audience to be affected, and that the effects are major. If a health app gets powned and finds their users’ data open on the Web, then that may—and probably should—affect the business.

And we have more to take away from this: Accessibility, for example, usually affects fewer people directly than security, performance, or content issues. But—these issues tend to be much more impactful, i.e., they can ruin the user experience, and prevent access altogether.

This insight alone may make the classification (or the exploration of similar classifications) worthwhile: We can customize our responses to issues better if we have clarity about how many users are how affected.

Audience Size and Impact in Your Work

What does this mean for your work?

If you find yourself treating issues largely the same way, this may and probably should mean adjustments to your prioritization.

It can mean to de-prioritize conformance, typography, and even SEO issues. From a user perspective, few of them may warrant a priority higher than P4.

It can mean to push on concerted efforts and initiatives to resolve security, accessibility, and content issues. “Fix-its”—group-wide bug-fixing and refactoring events—for these issues may make a real difference.

From there it may depend on whether you find it more important to fix issues for more people (as with content defects), or for those who are more impacted (as with accessibility problems). But you may need to make a call, and would benefit from it.

(Generally, however, you still want to stay away from one-dimensional prioritization and optimization.)

Personally, I’ll be excited to review responses and alterations to this model. I’m likely to invite this to more scrutiny, perhaps at Miro, try and test it, especially when it comes to the prioritization of accessibility issues. And I may report back.

Many thanks to Thomas Steiner and Jad Joubran for reviewing and sharing feedback on both chart and post, as well as members of the Miro Accessibility team for spontaneous suggestions for the charts!

About Me

I’m Jens (long: Jens Oliver Meiert), and I’m a frontend engineering leader and tech author/publisher. I’ve worked as a technical lead for companies like Google and as an engineering manager for companies like Miro, I’m a contributor to several web standards, and I write and review books for O’Reilly and Frontend Dogma.

I love trying things, not only in web development (and engineering management), but also in other areas like philosophy. Here on meiert.com I share some of my experiences and views. (Please be critical, interpret charitably, and give feedback.)