Tags: test

We do quite a bit of prototyping at Clearleft. There’s no better way to reduce risk than to get something in front of users as quickly as possible to test whether you’re on the right track or not.

As Benjamin said in the podcast episode on prototyping:

It’s something to look at, something to prod. And ideally you’re trying to work out what works and what doesn’t.

Sometimes the prototype is mocked up in Figma. Preferably it’s built in code—HTML, CSS, and JavaScript. Having a prototype built in the materials of the medium helps establish a plausible suspension of disbelief during testing.

Also, as Trys said in that same podcast episode:

Prototypical code isn’t production code. It’s quick and it’s often a little bit dirty and it’s not really fit for purpose in that final deliverable. But it’s also there to be inspiring and to gather a team and show that something is possible.

I can’t reiterate that enough: prototype code isn’t production code.

I’ve written about the two different mindsets before:

So these two kinds of work require very different attitudes. For production work, quality is key. For prototyping, making something quickly is what matters.

Addy recently wrote an excellent blog post on the topic of prototyping. The value of a prototype is in the insight it imparts, not the code.

It’s crucial to remember that in a prototype, the code serves merely as a medium—a way to facilitate understanding. It’s a means to an end, not the end itself. The code of a prototype is disposable and mutable. In contrast, the lessons learned from a prototype, the insights gained from user interaction and feedback, are far more durable and impactful.

This!

It can be tempting to re-use code from a prototype. I get it. It seems like a waste to throw away code and build something from scratch. But trust me—and I speak from experience here—it will take more time to wrangle prototype code into something that’s production-ready.

The problem is that quality is often invisible. Think about semantics, performance, security, privacy, and accessibility. Those matter—for production code—but they’re under the surface. For someone who doesn’t understand the importance of those hidden qualities a prototype that looks like it works seems ready to ship. It’s understandable that they’d balk at the idea of just throwing that code away and writing new code. Sometimes the suspension of disbelief that a prototype is aiming for works too well.

As is so often the case, this isn’t a technical problem. It’s a communication issue.

It’s often said that it’s easier to make a fast website than it is to keep a website fast. Things slip through. If you’re not vigilant, performance can erode without you noticing.

It’s a similar story for other invisible but important facets of your website: privacy, security, accessibility. Because they’re hidden from view, you won’t be able to see if there’s a regression.

That’s why it’s a good idea to have regular audits for performance, privacy, security, and accessibility.

I wrote about accessibility testing a while back, and how there’s quite a bit that you can do for yourself before calling in an expert to look at the really gnarly stuff:

When you commission an accessibility audit, you want to make sure you’re getting the most out of it. Don’t squander it on issues that you can catch and fix yourself. Make sure that the bulk of the audit is being spent on the specific issues that are unique to your site.

I recently did an internal audit of the Clearleft website. After writing up the report, I also did a lunch’n’learn to share my methodology. I wanted to show that there’s some low-hanging fruit that pretty much anyone can catch.

To start with, there’s keyboard navigation. Put your mouse and trackpad to one side and use the tab key to navigate around.

Caveat: depending on what browser you’re using, you might need to update some preferences for keyboard navigation to work on links. If you’re using Safari, go to “Preferences”, then “Advanced”, and tick “Press Tab to highlight each item on a web page.”

Tab around and find out. You should see some nice chunky :focus-visible styles on links and form fields.

Here’s something else that anyone can do: zoom in. Increase the magnification to 200%. Everything should scale proportionally. How about 500%? You’ll probably see a mobile-friendly layout. That’s fine. As long as nothing is broken or overlapping, you’re good.

At this point, I reach for some tools. I’ve got some bookmarklets that do similar things: tota11y and ANDI. They both examine the source HTML and CSS to generate reports on structure, headings, images, forms, and so on.

These tools are really useful, but you need to be able to interpret the results. For example, a tool can tell you if an image has no alt text. But it can’t tell you if an image has good or bad alt text.

Likewise, these tools are great for catching colour-contrast issues. But there’s a big difference between a colour-contrast issue on the body copy compared to a colour-contrast issue on one unimportant page element.

I think that demonstrates the most important aspect of any audit: prioritisation.

Finding out that you have accessibility issues isn’t that useful if they’re all presented as an undifferentiated list. What you really need to know are which issues are the most important to fix.

By the way, I really like the way that the Gov.uk team prioritises accessibility concerns:

The team puts accessibility concerns in 2 categories:

1. Theoretical: A question or statement regarding the accessibility of an implementation within the Design System without evidence of real-world impact.
2. Evidenced: Sharing new research, data or evidence showing that an implementation within the Design System could cause barriers for disabled people.

The team will usually prioritise evidenced issues and queries over theoretical ones.

When I wrote up my audit for the Clearleft website, I structured it in order of priority. The most important things to fix are at the start of the audit. I also used a simple scale for classifying the severity of issues: low, medium, and high priority.

Thankfully there were no high-priority issues. There were a couple of medium-priority issues. There were plenty of low-priority issues. That’s okay. That’s a pretty good distribution.

If you’re interested, here’s the report I delivered…

Accessibility audit on clearleft.com

Colour contrast

There are a few issues with the pink colour. When it’s used on a grey background, or when it’s used as a background colour for white text, the colour contrast isn’t high enough.

The SVG arrow icon could be improved too.

Recommendations

Medium priority

• Change the pink colour universally to be darker. The custom property --red is currently rgb(234, 33, 90). Change it to rgb(210, 20, 73) (thanks, James!)

Low priority

• The SVG arrow icon currently uses currentColor. Consider hardcoding solid black (or a very, very dark grey) instead.

Images

Alt text is improving on the site. There’s reasonable alt text at the top level pages and the first screen’s worth of case studies and blog posts. I made a sweep through these pages a while back to improve the alt text but I haven’t done older blog posts and case studies.

Recommendations

Medium priority

• Make a sweep of older blog posts and case studies and fix alt text.

Low priority

• Images on the contact page have alt text that starts with “A photo of…” — this is redundant and can be removed.

Headings

The site is using headings sensibly. Sometimes the nesting of headings isn’t perfect, but this is a low priority issue. For example, on the contact page there’s an h1 followed by two h3s. In theory this isn’t correct. In practice (for screen reader users) it’s not an issue.

Recommendations

Low priority

• On the home page, “UX London 2023” should probably be h3 instead of h1.
• On the case studies index page we’re currently using h3 headings for the industry sector (“Charities”, “Education” etc.) but these should probably not be headings at all. On the blog index page we use a class “Tags” for a similar purpose. Consider reusing that pattern on the case studies index page.
• On the about index page, “We’re driven to be” is an h3 and the subsequent three headings are h2s. Ideally this would be reversed: a single h2 followed by three h3s.

Link text

Sometimes the same text is used for different links.

Recommendations

Low priority

• On the home page the text “Read the case study” is re-used for multiple links. It would be better if each link were different e.g. “Read about The Natural History Museum.”

Forms

The only form on the site is the newsletter sign-up form. It’s marked up pretty well: the input has an associated label, although a visible (clickable) label would be better.

Tabbing order

The site doesn’t use JavaScript to mess with tabbing order for keyboard users. The source order of elements in the markup generally makes sense so all is good.

The focus styles are nice and clear too!

Structure

The site is using HTML landmark elements sensibly (header, nav, main, footer, etc.).

I was doing some accessibility work with a client a little while back. It was mostly giving their site the once-over, highlighting any issues that we could then discuss. It was an audit of sorts.

While I was doing this I started to realise that not all accessibility issues are created equal. I don’t just mean in their severity. I mean that some issues can—and should—be caught early on, while other issues can only be found later.

Take colour contrast. This is something that should be checked before a line of code is written. When designs are being sketched out and then refined in a graphical editor like Figma, that’s the time to check the ratio between background and foreground colours to make sure there’s enough contrast between them. You can catch this kind of thing later on, but by then it’s likely to come with a higher cost—you might have to literally go back to the drawing board. It’s better to find the issue when you’re at the drawing board the first time.

Then there’s the HTML. Most accessibility issues here can be caught before the site goes live. Usually they’re issues of ommission: form fields that don’t have an explicitly associated label element (using the for and id attributes); images that don’t have alt text; pages that don’t have sensible heading levels or landmark regions like main and nav. None of these are particularly onerous to fix and they come with the biggest bang for your buck. If you’ve got sensible forms, sensible headings, alt text on images, and a solid document structure, you’ve already covered the vast majority of accessibility issues with very little overhead. Some of these checks can also be automated: alt text for images; labels for inputs.

Then there’s interactive stuff. If you only use native HTML elements you’re probably in the clear, but chances are you’ve got some bespoke interactivity on your site: a carousel; a mega dropdown for navigation; a tabbed interface. HTML doesn’t give you any of those out of the box so you’d need to make your own using a combination of HTML, CSS, JavaScript and ARIA. There’s plenty of testing you can do before launching—I always ask myself “What would Heydon do?”—but these components really benefit from being tested by real screen reader users.

So if you commission an accessibility audit, you should hope to get feedback that’s mostly in that third category—interactive widgets.

If you get feedback on document structure and other semantic issues with the HTML, you should fix those issues, sure, but you should also see what you can do to stop those issues going live again in the future. Perhaps you can add some steps in the build process. Or maybe it’s more about making sure the devs are aware of these low-hanging fruit. Or perhaps there’s a framework or content management system that’s stopping you from improving your HTML. Then you need to execute a plan for ditching that software.

If you get feedback about colour contrast issues, just fixing the immediate problem isn’t going to address the underlying issue. There’s a process problem, or perhaps a communication issue. In that case, don’t look for a technical solution. A design system, for example, will not magically fix a workflow issue or route around the problem of designers and developers not talking to each other.

When you commission an accessibility audit, you want to make sure you’re getting the most out of it. Don’t squander it on issues that you can catch and fix yourself. Make sure that the bulk of the audit is being spent on the specific issues that are unique to your site.

You can listen to an audio version of Weighing up UX.

This is the month of UX Fest 2021—this year’s online version of UX London. The festival continues with masterclasses every Tuesday in June and a festival day of talks every Thursday (tickets for both are still available). But it all kicked off with the conference part last week: three back-to-back days of talks.

I have the great pleasure of hosting the event so not only do I get to see a whole lot of great talks, I also get to quiz the speakers afterwards.

Right from day one, a theme emerged that continued throughout the conference and I suspect will continue for the rest of the festival too. That topic was metrics. Kind of.

See, metrics come up when we’re talking about A/B testing, growth design, and all of the practices that help designers get their seat at the table (to use the well-worn cliché). But while metrics are very useful for measuring design’s benefit to the business, they’re not really cut out for measuring user experience.

People have tried to quantify user experience benefits using measurements like NetPromoter Score, which is about as useful as reading tea leaves or chicken entrails.

So we tend to equate user experience gains with business gains. That makes sense. Happy users should be good for business. That’s a reasonable hypothesis. But it gets tricky when you need to make the case for improving the user experience if you can’t tie it directly to some business metric. That’s when we run into the McNamara fallacy:

Making a decision based solely on quantitative observations (or metrics) and ignoring all others.

The way out of this quantitative blind spot is to use qualitative research. But another theme of UX Fest was just how woefully under-represented researchers are in most organisations. And even when you’ve gone and talked to users and you’ve got their stories, you still need to play that back in a way that makes sense to the business folks. These are stories. They don’t lend themselves to being converted into charts’n’graphs.

And so we tend to fall back on more traditional metrics, based on that assumption that what’s good for user experience is good for business. But it’s a short step from making that equivalency to flipping the equation: what’s good for the business must, by definition, be good user experience. That’s where things get dicey.

Broadly speaking, the talks at UX Fest could be put into two categories. You’ve got talks covering practical subjects like product design, content design, research, growth design, and so on. Then you’ve got the higher-level, almost philosophical talks looking at the big picture and questioning the industry’s direction of travel.

The tension between these two categories was the highlight of the conference for me. It worked particularly well when there were back-to-back talks (and joint Q&A) featuring a hands-on case study that successfully pushed the needle on business metrics followed by a more cautionary talk asking whether our priorities are out of whack.

For example, there was a case study on growth design, which emphasised the importance of A/B testing for validation, immediately followed by a talk on deceptive dark patterns. Now, I suspect that if you were to A/B test a deceptive dark pattern, the test would validate its use (at least in the short term). It’s no coincidence that a company like Booking.com, which lives by the A/B sword, is also one of the companies sued for using distressing design patterns.

Using A/B tests alone is like using a loaded weapon without supervision. They only tell you what people do. And again, the solution is to make sure you’re also doing qualitative research—that’s how you find out why people are doing what they do.

But as I’ve pondered the lessons from last week’s conference, I’ve come to realise that there’s also a danger of focusing purely on the user experience. Hear me out…

At one point, the question came up as to whether deceptive dark patterns were ever justified. What if it’s for a good cause? What if the deceptive dark pattern is being used by an organisation actively campaigning to do good in the world?

In my mind, there was no question. A deceptive dark pattern is wrong, no matter who’s doing it.

(There’s also the problem of organisations that think they’re doing good in the world: I’m sure that every talented engineer that worked on Google AMP honestly believed they were acting in the best interests of the open web even as they worked to destroy it.)

Where it gets interesting is when you flip the question around.

Suppose you’re a designer working at an organisation that is decidedly not a force for good in the world. Say you’re working at Facebook, a company that prioritises data-gathering and engagement so much that they’ll tolerate insurrectionists and even genocidal movements. Now let’s say there’s talk in your department of implementing a deceptive dark pattern that will drive user engagement. But you, being a good designer who fights for the user, take a stand against this and you successfully find a way to ensure that Facebook doesn’t deploy that deceptive dark pattern.

Yay?

Does that count as being a good user experience designer? Yes, you’ve done good work at the coalface. But the overall business goal is like a deceptive dark pattern that’s so big you can’t take it in. Is it even possible to do “good” design when you’re inside the belly of that beast?

Facebook is a relatively straightforward case. Anyone who’s still working at Facebook can’t claim ignorance. They know full well where that company’s priorities lie. No doubt they sleep at night by convincing themselves they can accomplish more from the inside than without. But what about companies that exist in the grey area of being imperfect? Frankly, what about any company that relies on surveillance capitalism for its success? Is it still possible to do “good” design there?

There are no easy answers and that’s why it so often comes down to individual choice. I know many designers who wouldn’t work at certain companies …but they also wouldn’t judge anyone else who chooses to work at those companies.

At Clearleft, every staff member has two levels of veto on client work. You can say “I’m not comfortable working on this”, in which case, the work may still happen but we’ll make sure the resourcing works out so you don’t have anything to do with that project. Or you can say “I’m not comfortable with Clearleft working on this”, in which case the work won’t go ahead (this usually happens before we even get to the pitching stage although there have been one or two examples over the years where we’ve pulled out of the running for certain projects).

Going back to the question of whether it’s ever okay to use a deceptive dark pattern, here’s what I think…

It makes no difference whether it’s implemented by ProPublica or Breitbart; using a deceptive dark pattern is wrong.

But there is a world of difference in being a designer who works at ProPublica and being a designer who works at Breitbart.

That’s what I’m getting at when I say there’s a danger to focusing purely on user experience. That focus can be used as a way of avoiding responsibility for the larger business goals. Then designers are like the soldiers on the eve of battle in Henry V:

For we know enough, if we know we are the kings subjects: if his cause be wrong, our obedience to the king wipes the crime of it out of us.