Google Lighthouse and PWA

Published on Jan 17, 2019 (updated Jun 16, 2024), filed under development (feed). (Share this on Mastodon or Bluesky?)

This [now partially outdated] article has been in the making for a while and as it happens, it just got hit by a major update of Lighthouse, Lighthouse release 4.0.0. And yet, I believe all is not lost and it will be very interesting to compare this review of Lighthouse 3 with version 4, because some issues persist. In the meantime, I’ll wonder whether dropping this post and writing a new one would have been the more favorable option. Swearword.

Google Lighthouse is a useful analysis tool but the PWA category’s audits appear questionable and should in many cases be disabled. Why? From my point of view Google has so far done an incomplete job normalizing the category and making it really useful. Let’s have a look.

The PWA Audits

When we start with the assumption that the test object is, indeed, a web app, about half of the PWA audits are reasonable. If we understand manifests and service workers as implicit parts of a PWA, it’s legitimate to check on them.

However, a good number of audits suffers of one of the following problems: They’re either weak or they belong to a different category (if they’re not a duplicate).

Address Bar Matches Brand Colors: This is one of the weakest checks in Lighthouse for two reasons. One, it tests for something arguably unimportant in an engineering tool, for there’s no drawback in not setting an address bar color. Two, and worse, what’s tested for endorses an awful solution, because it permanently pollutes markup with non-information. Theme colors should not be set as meta information, especially not given that this information can probably be obtained through other means.
Configured for a Custom Splash Screen: This audit is reminiscent of “splash screens must die,” something I noted at sum.cumo, too. Yet even as a loading screen, is this truly a defining, critical part of an app? To me, it’s a weak audit that mustn’t be a compulsory part of a PWA.
Contains Some Content When JavaScript Is Not Available: As it’s long impractical and perhaps impossible to use the Web in any meaningful way without having JavaScript enabled, and as all technology (web browsers, assistive technology, bots) are by now JavaScript-capable, it looks dated to require something as oddly defined as “some content“ without JavaScript.
Content Sized Correctly for Viewport: This audit seems to belong into the “Best Practices” category.
Has a Viewport Meta Tag with width or initial-scale: Likewise, this audit seems also better suited in the “Best Practices” category.
Manifest Exists: There are six audits that check contents of the manifest. This implies that a manifest exists. It’s not clear what good reasons there are to check for the mere existence of a manifest and that it has “some data.” Unless documentation is just poorly phrased and there is something I missed in my work with Lighthouse, rewarding existence despite complementary checks that imply that existence makes for a questionable approach to validation.
Page Load Is Fast Enough on Mobile: This audit appears to fit better under “Performance.”
Redirects HTTP Traffic to HTTPS: This audit seems more fitting under “Best Practices.”
User Can Be Prompted to Install The Web App: Prompting for installation may be as undesirable a pattern as is “make start page“ or “add to favorites.” It did look as if we had put these dark patterns behind. Users know by now how to bookmark or save web content. As what this prompt brings with it—when it shows—also occupies screen real estate, this audit is one of the least advisable tests in Lighthouse, one that should be disabled.
Uses HTTPS: This is a duplicate already present in the “Best Practices” category.

The PWA Category

Now that we checked the individual PWA audits—of which the rest is sound–, we get a better idea of the usefulness of the category.

Apparently there was a strong urge to have a category just for PWA. Well. But what then seems to have happened is a problem all of Lighthouse now suffers from: Some checks have been moved to “PWA” that better fit elsewhere (viewport, load time, encryption), and then the category looks like it got filled up for good measure (colors, scripting, splash screens).

This seems to have happened. But even if the story is different, the outcome, at this point, means a weak architecture that undermines Lighthouse’s usefulness as a whole:

For websites it’s already necessary to exclude the PWA check. (It would have been great if Lighthouse could employ some detection to make that step unnecessary.)
For PWAs, now, it’s then necessary to exclude some audits. At least, my recommendation, Address Bar Matches Brand Colors and User Can Be Prompted to Install The Web App, and, perhaps, Configured for a Custom Splash Screen, Contains Some Content When JavaScript Is Not Available, and Manifest Exists.
For the other categories, then, it’s actually still useful to also review some of the PWA checks because they do include some useful Best Practices.

If Lighthouse would make out the difference between sites and apps, would focus on important criteria (not colors and modals and such), and had its information architecture normalized to allow for some sort of General checks, it could up the ante to be even more useful, in a more usable fashion—by default.

I will have erred in some aspect or other, but that’s my current take on Lighthouse. If you’re interested in some of the tests we consensually disabled at sum.cumo, or the wrapper we use to simplify Lighthouse configuration, stay tuned over at sumcumo.com and check out Lighthouse Keeper. And then let’s see what I can do to merge this review with Lighthouse 4.

Val strides swiftly eastward, using the forest paths. He only sees his men when a shadowy figure steps silently from among the trees and gives him directions. — Lighthouse would sometimes suggest a different route. (Copyright King Features Syndicate, Inc., distr. Bulls.)

About Me

I’m Jens (long: Jens Oliver Meiert), and I’m a web developer, manager, and author. I’ve been working as a technical lead and engineering manager for companies you’ve never heard of and companies you use every day, I’m an occasional contributor to web standards (like HTML, CSS, WCAG), and I write and review books for O’Reilly and Frontend Dogma.

I love trying things, not only in web development and engineering management, but also in other areas like philosophy. Here on meiert.com I share some of my experiences and views. (I value you being critical, interpreting charitably, and giving feedback.)