Tags: testing

170

sparkline

Monday, June 7th, 2021

Weighing up UX

You can listen to an audio version of Weighing up UX.

This is the month of UX Fest 2021—this year’s online version of UX London. The festival continues with masterclasses every Tuesday in June and a festival day of talks every Thursday (tickets for both are still available). But it all kicked off with the conference part last week: three back-to-back days of talks.

I have the great pleasure of hosting the event so not only do I get to see a whole lot of great talks, I also get to quiz the speakers afterwards.

Right from day one, a theme emerged that continued throughout the conference and I suspect will continue for the rest of the festival too. That topic was metrics. Kind of.

See, metrics come up when we’re talking about A/B testing, growth design, and all of the practices that help designers get their seat at the table (to use the well-worn cliché). But while metrics are very useful for measuring design’s benefit to the business, they’re not really cut out for measuring user experience.

People have tried to quantify user experience benefits using measurements like NetPromoter Score, which is about as useful as reading tea leaves or chicken entrails.

So we tend to equate user experience gains with business gains. That makes sense. Happy users should be good for business. That’s a reasonable hypothesis. But it gets tricky when you need to make the case for improving the user experience if you can’t tie it directly to some business metric. That’s when we run into the McNamara fallacy:

Making a decision based solely on quantitative observations (or metrics) and ignoring all others.

The way out of this quantitative blind spot is to use qualitative research. But another theme of UX Fest was just how woefully under-represented researchers are in most organisations. And even when you’ve gone and talked to users and you’ve got their stories, you still need to play that back in a way that makes sense to the business folks. These are stories. They don’t lend themselves to being converted into charts’n’graphs.

And so we tend to fall back on more traditional metrics, based on that assumption that what’s good for user experience is good for business. But it’s a short step from making that equivalency to flipping the equation: what’s good for the business must, by definition, be good user experience. That’s where things get dicey.

Broadly speaking, the talks at UX Fest could be put into two categories. You’ve got talks covering practical subjects like product design, content design, research, growth design, and so on. Then you’ve got the higher-level, almost philosophical talks looking at the big picture and questioning the industry’s direction of travel.

The tension between these two categories was the highlight of the conference for me. It worked particularly well when there were back-to-back talks (and joint Q&A) featuring a hands-on case study that successfully pushed the needle on business metrics followed by a more cautionary talk asking whether our priorities are out of whack.

For example, there was a case study on growth design, which emphasised the importance of A/B testing for validation, immediately followed by a talk on deceptive dark patterns. Now, I suspect that if you were to A/B test a deceptive dark pattern, the test would validate its use (at least in the short term). It’s no coincidence that a company like Booking.com, which lives by the A/B sword, is also one of the companies sued for using distressing design patterns.

Using A/B tests alone is like using a loaded weapon without supervision. They only tell you what people do. And again, the solution is to make sure you’re also doing qualitative research—that’s how you find out why people are doing what they do.

But as I’ve pondered the lessons from last week’s conference, I’ve come to realise that there’s also a danger of focusing purely on the user experience. Hear me out…

At one point, the question came up as to whether deceptive dark patterns were ever justified. What if it’s for a good cause? What if the deceptive dark pattern is being used by an organisation actively campaigning to do good in the world?

In my mind, there was no question. A deceptive dark pattern is wrong, no matter who’s doing it.

(There’s also the problem of organisations that think they’re doing good in the world: I’m sure that every talented engineer that worked on Google AMP honestly believed they were acting in the best interests of the open web even as they worked to destroy it.)

Where it gets interesting is when you flip the question around.

Suppose you’re a designer working at an organisation that is decidedly not a force for good in the world. Say you’re working at Facebook, a company that prioritises data-gathering and engagement so much that they’ll tolerate insurrectionists and even genocidal movements. Now let’s say there’s talk in your department of implementing a deceptive dark pattern that will drive user engagement. But you, being a good designer who fights for the user, take a stand against this and you successfully find a way to ensure that Facebook doesn’t deploy that deceptive dark pattern.

Yay?

Does that count as being a good user experience designer? Yes, you’ve done good work at the coalface. But the overall business goal is like a deceptive dark pattern that’s so big you can’t take it in. Is it even possible to do “good” design when you’re inside the belly of that beast?

Facebook is a relatively straightforward case. Anyone who’s still working at Facebook can’t claim ignorance. They know full well where that company’s priorities lie. No doubt they sleep at night by convincing themselves they can accomplish more from the inside than without. But what about companies that exist in the grey area of being imperfect? Frankly, what about any company that relies on surveillance capitalism for its success? Is it still possible to do “good” design there?

There are no easy answers and that’s why it so often comes down to individual choice. I know many designers who wouldn’t work at certain companies …but they also wouldn’t judge anyone else who chooses to work at those companies.

At Clearleft, every staff member has two levels of veto on client work. You can say “I’m not comfortable working on this”, in which case, the work may still happen but we’ll make sure the resourcing works out so you don’t have anything to do with that project. Or you can say “I’m not comfortable with Clearleft working on this”, in which case the work won’t go ahead (this usually happens before we even get to the pitching stage although there have been one or two examples over the years where we’ve pulled out of the running for certain projects).

Going back to the question of whether it’s ever okay to use a deceptive dark pattern, here’s what I think…

It makes no difference whether it’s implemented by ProPublica or Breitbart; using a deceptive dark pattern is wrong.

But there is a world of difference in being a designer who works at ProPublica and being a designer who works at Breitbart.

That’s what I’m getting at when I say there’s a danger to focusing purely on user experience. That focus can be used as a way of avoiding responsibility for the larger business goals. Then designers are like the soldiers on the eve of battle in Henry V:

For we know enough, if we know we are the kings subjects: if his cause be wrong, our obedience to the king wipes the crime of it out of us.

Monday, May 10th, 2021

Building a resilient frontend using progressive enhancement - Service Manual - GOV.UK

Using progressive enhancement means your users will be able to do what they need to do if any part of the stack fails.

What a terrific short guide to sensible web development!

  • Start with HTML
    • Using interactive elements
    • Adding the extras
    • Building more complex services
    • Testing your service
    • Do not assume users turn off CSS or JavaScript
    • Case studies and related guides

Sunday, May 2nd, 2021

Performance-testing the Google I/O site - JakeArchibald.com

Modern web development:

Imagine you went to a restaurant, took a seat, and 20 minutes later you still haven’t been given a menu. You ask where it is, and you’re told “oh, we’re currently cooking you everything you might possibly ask for. Then we’ll give you the menu, you’ll pick something, and we’ll be able to give you it instantly, because it’ll all be ready”.

Single page apps, ladies and gentlemen.

Tuesday, April 20th, 2021

Global performance insights for your site | Lighthouse Metrics

I hadn’t come across this before—run Lighthouse tests on your pages from six different locations around the world at once.

Monday, February 1st, 2021

Thursday, November 12th, 2020

An opinionated guide to accessibility testing /// Iain Bean

  1. First impressions
  2. The Tab key
  3. Automated testing tools
  4. Screen reader testing
  5. Next steps

Thursday, October 8th, 2020

Top 5 things to review in an Accessible Design Review - Hassell Inclusion

Considering how much accessibility work happens “under the hood”, it’s interesting that all five of these considerations are visibly testable.

  1. Think about accessible copy
  2. Don’t forget about the focus indicator
  3. Check your colour contrast
  4. Don’t just use colour to convey meaning
  5. Design in anticipation of text resizing

Monday, September 14th, 2020

Tolerance | Trys Mudford

Trys ponders home repair projects and Postel’s Law.

As we build our pages, components, and business logic, establish where tolerance should be granted. Consider how flexible each entity should be, and on what axis. Determine which items need to be fixed and less tolerant. There will be areas where the data or presentation being accurate is more important than being flexible - document these decisions.

Tuesday, July 7th, 2020

We need more inclusive web performance metrics | Filament Group, Inc.

Good point. When we talk about perceived performance, the perception in question is almost always visual. We should think more inclusively than that.

Thursday, June 25th, 2020

Accessible to some - Manuel Matuzović

A score of 100 in Lighthouse or 0 errors in axe doesn’t mean that you’re done, it means that you’re ready to start manual testing and testing with real users, if possible.

Tuesday, June 23rd, 2020

Introduction to Screen Readers Using Voiceover | Gymnasium

This is a great short introduction to using VoiceOver with Safari by the one and only Ethan Marcotte.

Thursday, April 30th, 2020

Prioritizing users in a crisis: Building the California COVID-19 response site

This is a great case study of the excellent California COVID-19 response site. Accessibility and performance are the watchwords here.

Want to know their secret weapon?

A $20 device running Android 9, with no contract commitment has been one of the most useful and effective tools in our effort to be accessible.

Leaner, faster sites benefit everybody, but making sure your applications run smoothly on low-end hardware makes a massive difference for those users.

Tuesday, March 10th, 2020

Lighthouse bookmarklet

I use Firefox. You should too. It’s fast, secure, and more privacy-focused than the leading browser from the big G.

When it comes to web development, the CSS developer tooling in Firefox is second-to-none. But when it comes to JavaScript and network-related debugging (like service workers), Chrome’s tools are currently better than Firefox’s (for now). For example, Chrome has a tab in its developer tools that lets you run Lighthouse on the currently open tab.

Yesterday, I got the Calibre newsletter, which always has handy performance-related links from Karolina. She pointed to a Lighthouse extension for Firefox. “Excellent!”, I thought, and I immediately installed it. But I had some qualms about installing a plug-in from Google into a browser from Mozilla, particularly as the plug-in page says:

This is not a Recommended Extension. Make sure you trust it before installing

Well, I gave it a go. It turns out that all it actually does is redirect to the online version of Lighthouse. “Hang on”, I thought. “This could just be a bookmarklet!”

So I immediately uninstalled the browser extension and made this bookmarklet:

Lighthouse

Drag that up to your desktop browser’s bookmarks toolbar. Press it whenever you’re on a site that you want to test.

Tuesday, March 3rd, 2020

HTML: The Inaccessible Parts - daverupert.com

Well, this is a grim collection from Dave:

There are some cases where even using plain ol’ HTML causes accessibility problems. I get frustrated and want to quit web development whenever I read about these types of issues. Because if browsers can’t get this right, what hope is there for the rest of us.

It’s worth clicking through each link he lists—the situation is often much more nuanced than simply “Don’t use X.”

Friday, January 10th, 2020

Listen To Me And Not Google: HeydonWorks

We have to stop confusing the excesses of capitalism with the hallmarks of quality. Sometimes Google aren’t better, they’re just more pervasive.

cough AMP cough

Thursday, November 7th, 2019

What I’ve learned about accessibility in SPAs

Nolan writes up what he learned making accessibiity improvements to a single page app. The two big takeways involve letting the browser do the work for you:

Here’s the best piece of accessibility advice for newbies: if something is a button, make it a <button>. If something is an input, make it an <input>. Don’t try to reinvent everything from scratch using <div>s and <span>s.

And then there are all the issues that crop up when you take over the task of handling navigations:

  • You need to manage focus yourself.
  • You need to manage scroll position yourself.

For classic server-rendered pages, most browser engines give you this functionality for free. You don’t have to code anything. But in an SPA, since you’re overriding the normal navigation behavior, you have to handle the focus yourself.

Friday, November 1st, 2019

Location, Privilege and Performant Websites

Testing on a <$100 Android device on a 3G network should be an integral part of testing your website. Not everyone is on a brand-new device or upgrades often, especially with the price point of a high-end phones these days.

When we design and build our websites with the outliers in mind, whether it’s for performance or even user experience, we build an experience that can be easy for all to access and use — and that’s what the web is about, access and information for all.

Wednesday, October 16th, 2019

Beyond automatic accessibility testing: 6 things I check on every website I build - Manuel Matuzović

Six steps that everyone can do to catch accessibility gotchas:

  1. Check image descriptions
  2. Disable all styles
  3. Validate HTML
  4. Check the document outline
  5. Grayscale mode
  6. Use the keyboard

Thursday, September 19th, 2019

An HTML attribute potentially worth $4.4M to Chipotle - Cloud Four

When I liveblogged Jason’s talk at An Event Apart in Chicago, I included this bit of reporting:

Jason proceeds to relate a long and involved story about buying burritos online from Chipotle.

Well, here is that story. It’s a good one, with some practical takeaways (if you’ll pardon the pun):

  1. Use HTML5 input features
  2. Support autofill
  3. Make autofill part of your test plans

Friday, August 23rd, 2019

Is client side A/B testing always a bad idea in your experience? · Issue #53 · csswizardry/ama

Harry enumerates the reasons why client-side A/B testing is terrible:

  • It typically blocks rendering.
  • Providers are almost always off-site.
  • It happens on every page load.
  • No user-benefitting reuse.
  • They likely skip any governance process.

While your engineers are subject to linting, code-reviews, tests, auditors, and more, your marketing team have free rein of the front-end.

Note that the problem here is not A/B testing per se, it’s client-side A/B testing. For some reason, we seem to have collectively decided that A/B testing—like analytics—is something we should offload to the JavaScript parser in the user’s browser.