What gets measured gets done. You are what you measure. Measurement eliminates argument. If you work in an environment that puts store in these oft-quoted business adages then I urge you to take a moment to challenge your calculations. Let’s review our metrics to ensure they can stand up and be counted.
Tuesday, April 12th, 2022
Monday, January 31st, 2022
A thoroughly researched look at what our baseline criteria should be for making websites today:
The baseline for web development in 2022 is:
- low-spec Android devices in terms of performance,
- Safari from two years before in terms of Web Standards,
- and 4G in terms of networks.
It’s somewhat damning to Safari to see it as a baseline browser, but with Internet Explorer out of the picture, something has to be the lowest common denominator. In this case, Safari is quite literally the new IE.
Saturday, December 4th, 2021
We’ve got click rates, impressions, conversion rates, open rates, ROAS, pageviews, bounces rates, ROI, CPM, CPC, impression share, average position, sessions, channels, landing pages, KPI after never ending KPI.
That’d be fine if all this shit meant something and we knew how to interpret it. But it doesn’t and we don’t.
The reality is much simpler, and therefore much more complex. Most of us don’t understand how data is collected, how these mechanisms work and most importantly where and how they don’t work.
Tuesday, November 23rd, 2021
Even if you can somehow justify using tracking technologies (which don’t work reliably) to make general, statistical decisions (“fewer people open our emails when the subject contains the word ‘overdraft’!”), you can’t make individual decisions based on them. That’s just wrong.
Prompted by my post on tracking, Chris does some soul searching about his own use of tracking.
I’m interested not just in the ethical concerns and my long-time complacency with industry norms, but also as someone who very literally sells advertising.
He brings up the point that advertisers expect to know how many people opened a particular email and how many people clicked on a particular link. I’m sure that’s right, but it’s also beside the point: what matters is how the receiver of the email feels about having that information tracked. If they haven’t given you permission to do it, you can’t just assume they’re okay with it.
Tuesday, November 16th, 2021
The idea that it’s alright to do whatever unethical thing is currently the industry norm is widespread in tech, and dangerous.
It stood out to me because I had been thinking about certain practices that are widespread, accepted, and yet strike me as deeply problematic. These practices involve tracking users.
The first problem is that even the terminology I’m using would be rejected. When you track users on your website, it’s called analytics. Or maybe it’s stats. If you track users on a large enough scale, I guess you get to just call it data.
Those words—“analytics”, “stats”, and “data”—are often used when the more accurate word would be “tracking.”
Or to put it another way; analytics, stats, data, numbers …these are all outputs. But what produced these outputs? Tracking.
Here’s a concrete example: email newsletters.
Do you have numbers on how many people opened a particular newsletter? Do you have numbers on how many people clicked a particular link?
You can call it data, or stats, or analytics, but make no mistake, that’s tracking.
Follow-on question: do you honestly think that everyone who opens a newsletter or clicks on a link in a newsletter has given their informed constent to be tracked by you?
You may well answer that this is a widespread—nay, universal—practice. Well yes, but a) that’s not what I asked, and b) see the above quote from Design For Safety.
You could quite correctly point out that this tracking is out of your hands. Your newsletter provider—probably Mailchimp—does this by default. So if the tracking is happening anyway, why not take a look at those numbers?
But that’s like saying it’s okay to eat battery-farmed chicken as long as you’re not breeding the chickens yourself.
When I try to argue against this kind of tracking from an ethical standpoint, I get a frosty reception. I might have better luck battling numbers with numbers. Increasing numbers of users are taking steps to prevent tracking. I had a plug-in installed in my mail client—Apple Mail—to prevent tracking. Now I don’t even need the plug-in. Apple have built it into the app. That should tell you something. It reminds me of when browsers had to introduce pop-up blocking.
If the outputs generated by tracking turn out to be inaccurate, then shouldn’t they lose their status?
But that line of reasoning shouldn’t even by necessary. We shouldn’t stop tracking users because it’s inaccurate. We should stop stop tracking users because it’s wrong.
Well, this is just wonderful! Jim has written copious notes after listening to my favourite episode of season three of the Clearleft podcast, measuring design:
I’m going to have to try really, really hard to not just copy/paste the entire transcript of this podcast. It‘s that good. Don’t miss it.
Tuesday, October 26th, 2021
A good post by Andy on “the language of business,” which is most cases turns out to be numbers, numbers, numbers.
While it seems reasonable and fair to expect a modicum of self-awareness of why you’re employed and what business value you drive in the the context of the work you do, sometimes the incessant self-flagellation required to justify and explain this to those who hired you may be a clue to a much deeper and more troubling question at the heart of the organisation you work for.
This pairs nicely with the Clearleft podcast episode on measuring design.
Thursday, October 7th, 2021
I’ve noticed a trend in recent years—a trend that I’ve admittedly been part of myself—where performance-minded developers will rebuild a site and then post a screenshot of their Lighthouse score on social media to show off how fast it is.
But I’m going to respectfully decline Phil’s advice to use any of the RUM analytics providers he recommends that require me to put another
script element on my site. One third-party script is one third-party script too many.
Wednesday, October 6th, 2021
Measuring design on the Clearleft podcast
There was a bit of a theme running through UX Fest earlier this year. On the one hand, there was all the talk of designers learning to speak the language of business (to get that coveted seat at the table), which means talking in numbers. But on the other hand, isn’t there a real danger in reducing user experience to numbers in a spreadsheet?
For this episode I put the narrative together using lots of snippets from different talks, not just from UX Fest but from previous Clearleft events too. I also got some good hot takes from my colleagues Chris, Andy, and Maite. Oh, and it opens with former US Secretary of Defense, Robert McNamara. If you know, you know.
This episode comes in at 22 and a half minutes and I think it’s well worth your time. Have a listen.
This is the penultimate episode of season three. Just one more to go!
Saturday, September 4th, 2021
An oldie but a goodie. If you think you’re getting statistically significant results from A/B testing, you should probably consider doing some A/A testing.
In an A/A test, you run a test using the exact same options for both “variants” in your test.
Sunday, June 6th, 2021
The typography of horology.
Tuesday, April 20th, 2021
Core web vitals from Google are the ingredients for an alphabet soup of exlusionary intialisms. But once you get past the unnecessary jargon, there’s a sensible approach underpinning the measurements.
From May—no, June—these measurements will be a ranking signal for Google search so performance will become more of an SEO issue. This is good news. This is what Google should’ve done years ago instead of pissing up the wall with their dreadful and damaging AMP project that blackmailed publishers into using a proprietary format in exchange for preferential search treatment. It was all done supposedly in the name of performance, but in reality all it did was antagonise users and publishers alike.
A new and unusual phenomenon: clients reluctant (even refusing) to fix performance issues unless they directly improve Vitals.
Once you put a measurement on something, there’s a danger of focusing too much on the measurement. Chris is worried that we’re going to see tips’n’tricks for gaming core web vitals:
This feels like the start of a weird new era of web performance where the metrics of web performance have shifted to user-centric measurements, but people are implementing tricky strategies to game those numbers with methods that, if anything, slightly harm user experience.
The map is not the territory. The numbers are a proxy for user experience, but it’s notoriously difficult to measure intangible ideas like pain and frustration. As Laurie says:
This is 100% the downside of automatic tools that give you a “score”. It’s like gameification. It’s about hitting that perfect score instead of the holistic experience.
And Ethan has written about the power imbalance that exists when Google holds all the cards, whether it’s AMP or core web vitals:
Google used its dominant position in the marketplace to force widespread adoption of a largely proprietary technology for creating websites. By switching to Core Web Vitals, those power dynamics haven’t materially changed.
We would do well to remember:
When you measure, include the measurer.
(If you prefer using initialisms, remember that CFP is Certified Financial Planner, CLS is Community Legal Services, and FID is Flame Ionization Detector. Together they form CWV, Catholic War Veterans.)
I hadn’t come across this before—run Lighthouse tests on your pages from six different locations around the world at once.
Monday, March 22nd, 2021
And, no, you don’t need to
npm install any of these. Try “vendoring” them instead (that’s copying and pasting to you and me).
Monday, March 8th, 2021
The Right Number is a gentle, noncommercial space where your only job is to be yourself. Upon dialing you’ll be connected to a voicemail box and given a brief prompt. You have three minutes to answer however you’d like.
Monday, December 14th, 2020
I spent most of the weekend reading through this and I’ve still barely scratched the surface—a lot of work has gone to the analyses and write-ups!
The sections on accessibility and performance get grimmer each year but the raw numbers on framework adaption are refreshingly perspective-setting.
Monday, November 16th, 2020
Goodhart’s Law applied to Google’s core web vitals:
If developers start to focus solely on Core Web Vitals because it is important for SEO, then some folks will undoubtedly try to game the system.
Personally, my beef with core web vitals is that they introduce even more uneccessary initialisms (see, for example, Harry’s recent post where he uses CWV metrics like LCP, FID, and CLS—alongside TTFB and SI—to look at PLPs, PDPs, and SRPs. I mean, WTF?).
Saturday, May 23rd, 2020
I must admit I’ve been wincing a little every time I see a graph with a logarithmic scale in a news article about COVID-19. It takes quite a bit of cognitive work to translate to a linear scale and get the real story.
Wednesday, March 11th, 2020
A curl in every port
A few years back, Zach Bloom wrote The History of the URL: Path, Fragment, Query, and Auth. He recently expanded on it and republished it on the Cloudflare blog as The History of the URL. It’s well worth the time to read the whole thing. It’s packed full of fascinating tidbits.
In the section on ports, Zach says:
The timeline of Gopher and HTTP can be evidenced by their default port numbers. Gopher is 70, HTTP 80. The HTTP port was assigned (likely by Jon Postel at the IANA) at the request of Tim Berners-Lee sometime between 1990 and 1992.
Kimberly was spelunking down the original source code, when she came across this line in the
#define TCP_PORT 80 /* Allocated to http by Jon Postel/ISI 24-Jan-92 */
We showed this to Jean-François Groff, who worked on the original web technologies like
libwww, the forerunner to
libcurl. He remembers that day. It felt like they had “made it”, receiving the official blessing of Jon Postel (in the same RFC, incidentally, that gave port 70 to Gopher).
Then he told us something interesting about the next line of code:
#define OLD_TCP_PORT 2784 /* Try the old one if no answer on 80 */
Port 2784? That seems like an odd choice. Most of us would choose something easy to remember.
Well, it turns out that 2784 is easy to remember if you’re Tim Berners-Lee.
Those were the last four digits of his parents’ phone number.