Sunday, December 3rd, 2017
Saturday, November 18th, 2017
Hooked and booked
I think there might be a connection.
A/B testing is a great way of finding out what happens when you introduce a change. But it can’t tell you why.
I used the site earlier in the year, and actually felt my heart rate increase. Never again.— Paul Robert Lloyd (@paulrobertlloyd) October 24, 2017
The problem is that, in a data-driven environment, decisions ultimately come down to whether something works or not. But just because something works, doesn’t mean it’s a good thing.
If I were trying to convince you to buy a product, or use a service, one way I could accomplish that would be to literally put a gun to your head. It would work. Except it’s not exactly a good solution, is it? But if we were to judge by the numbers (100% of people threatened with a gun did what we wanted), it would appear to be the right solution.
When speaking about A/B testing at Booking.com, Stuart Frisby emphasised why it’s so central to their way of working:
One of the core principles of our organisation is that we want to be very customer-focused. And A/B testing is really a way for us to institutionalise that customer focus.
I’m not so sure. I think A/B testing is a way to institutionalise a focus on business goals—increasing sales, growth, conversion, and all of that. Now, ideally, those goals would align completely with the customer’s goals; happy customers should mean more sales …but more sales doesn’t necessarily mean happy customers. Using business metrics (sales, growth, conversion) as a proxy for customer satisfaction might not always work …and is clearly not the case with many of these kinds of sites. Whatever the company values might say, a company’s true focus is on whatever they’re measuring as success criteria. If that’s customer satisfaction, then the company is indeed customer-focused. But if the measurements are entirely about what works for sales and conversions, then that’s the real focus of the company.
I’m not saying A/B testing is bad—far from it! (although it can sometimes be taken to the extreme). I feel it’s best wielded in combination with usability testing with real users—seeing their faces, feeling their frustration, sharing their joy.
In short, I think that A/B testing needs to be counterbalanced. There should be some kind of mechanism for getting the answer to “why?” whenever A/B testing provides to the answer to “what?” In-person testing could be one way of providing that balance. Or it could be somebody’s job to always ask “why?” and determine if a solution is qualitatively—and not just quantitatively—good. (And if you look around at your company and don’t see anyone doing that, maybe that’s a role for you.)
Curious: How many large companies have an ethics board? Or some kind of moral advisory role employed?— Erin Weigel (@endesignonline) November 2, 2017
If there really is a connection between having a data-driven culture of A/B testing, and a product that’s filled with dark patterns, then the disturbing conclusion is that dark patterns work …at least in the short term.
Monday, November 6th, 2017
Installing Progressive Web Apps
It used to literally say “add to home screen.”
Now it simply says “add.”
I vaguely remember there being some talk of changing the labelling, but I could’ve sworn it was going to change to “install”. I’ve got to be honest, just having the word “add” doesn’t seem to provide much context. Based on the quick’n’dirty usability testing I did with some co-workers, it just made things confusing. “Add what?” “What am I adding?”
Additionally, the prompt appeared immediately on the first visit to the site. I thought there was supposed to be an added “engagement” metric in order for the prompt to appear; that the user needs to visit the site more than once.
You’d think I’d be happy that users will be presented with the home-screen prompt immediately, but based on the behaviour I saw, I’m not sure it’s a good thing. Here’s what I observed:
- The user types the URL
archive.dconstruct.orginto the address bar.
- The site loads.
- The home-screen prompt slides up from the bottom of the screen.
- The user immediately moves to dismiss the prompt (cue me interjecting “Don’t close that!”).
This behaviour is entirely unsurprising for three reasons:
- We web designers and web developers have trained users to dismiss overlays and pop-ups if they actually want to get to the content. Nobody’s going to bother to actually read the prompt if there’s a 99% chance it’s going to say “Sign up to our newsletter!” or “Take our survey!”.
- Because the prompt now appears on the first visit, no trust has been established between the user and the site. If the prompt only appeared on later visits (or later navigations during the first visit) perhaps it would stand a greater chance of survival.
It’s still possible to add a Progressive Web App to the home screen, but the option to do that is hidden behind the mysterious three-dots-vertically-stacked icon (I propose we call this the shish kebab icon to distinguish it from the equally impenetrable hamburger icon).
I was chatting with Andreas from Mozilla at the View Source conference last week, and he was filling me in on how Firefox on Android does the add-to-homescreen flow. Instead of a one-time prompt, they’ve added a persistent icon above the “line of death” (the icon is a combination of a house and a plus symbol).
When a Firefox 58 user arrives on a website that is served over HTTPS and has a valid manifest, a subtle badge will appear in the address bar: when tapped, an “Add to Home screen” confirmation dialog will slide in, through which the web app can be added to the Android home screen.
This kind of badging also has issues (without the explicit text “add to home screen”, the user doesn’t know what the icon does), but I think a more persistently visible option like this works better than the a one-time prompt.
Firefox is following the lead of the badging approach pioneered by the Samsung Internet browser. It provides a plus symbol that, when pressed, reveals the options to add to home screen or simply bookmark.
I don’t think Chrome for Android has any plans for this kind of badging, but they are working on letting the site authors provide their own prompts. I’m not sure this is such a good idea, given our history of abusing pop-ups and overlays.
Sadly, I feel that any solution that relies on an unrequested overlay is doomed. That’s on us. The way we’ve turned browsing the web—especially on mobile—into a frustrating chore of dismissing unwanted overlays is a classic tragedy of the commons. We blew it. Users don’t trust unrequested overlays, and I can’t blame them.
For what it’s worth, my opinion is that ambient badging is a better user experience than one-time prompts. That opinion is informed by a meagre amount of testing though. I’d love to hear from anyone who’s been doing more detailed usability testing of both approaches. I assume that Google, Mozilla, and Samsung are doing this kind of testing, and it would be really great to see the data from that (hint, hint).
But it might well be that ambient badging is just too subtle to even be noticed by the user.
On one end of the scale you’ve got the intrusiveness of an add-to-home-screen prompt, but on the other end of the scale you’ve got the discoverability problem of a subtle badge icon. I wonder if there might be a compromise solution—maybe a badge icon that pulses or glows on the first or second visit?
Of course that would also need to be thoroughly tested.
Wednesday, September 27th, 2017
Are you the creator, programmer, or quality-tester of a podcasting application? This page provides a range of podcasts that exemplify a range of atypical use case from merely uncommon to exceedingly fringe. If your app can handle all these, you’re doing well.
The title is pure clickbait, and the moral panic early in this article repeats the Toyota myth, but then it settles down into a fascinating examination of abstractions in programming. On the one hand, there’s the problem of the not enough abstraction: having to write in code is such a computer-centric way of building things. On the other hand, our world is filled with dangerously abstracted systems:
When your tires are flat, you look at your tires, they are flat. When your software is broken, you look at your software, you see nothing.
So that’s a big problem.
Bret Victor, John Resig and Margaret Hamilton are featured. Doug Engelbart and J.C.R. Licklider aren’t mentioned but their spirits loom large.
Thursday, September 21st, 2017
Well, I guess it’s time to change all my locally-hosted sites from
.dev domains to
.test. Thanks, Google.
Sunday, August 20th, 2017
Thursday, July 27th, 2017
Riffing on Rachel’s talk at Patterns Day:
At the Patterns Day conference last month, Rachel Andrew mentioned something interesting about patterns. She said that working with reusable interface components, where each one has its own page, made her realise that those work quite well as isolated test cases. I feel this also goes for some accessibility tests: there is a number of criteria where isolation aids testing.
Hidde specifically singles out these patterns:
- Collapsible (“Show/hide”)
- Form field
- Video player
Sunday, July 23rd, 2017
James gives—if you’ll pardon the pun— hands-on advice on making sites that consider motor impairment:
- Don’t assume keyboard access is all you need
- Auto complete/Autofill
- Show me my password
- Allow for fine motor control issues
- Don’t autoplay videos
- Avoid hover-only controls
- Infinite scrolling considerations
- Be mindful of touch
- Avoid small hit targets
- Provide alternate controls for touch gestures
Far from being a niche concern, visitors with some form of motor impairment likely make up a significant percentage of your users. I would encourage you to test your website or application with your less dominant hand. Is it still easy to use?
Tuesday, June 6th, 2017
Your website’s only as strong as the weakest device you’ve tested it on.
Wednesday, May 31st, 2017
It’s no substitute for testing with real devices, but the “device wall” view in this Chrome plug-in is a nifty way of getting an overview of a site’s responsiveness at a glance.
Sunday, May 14th, 2017
This looks like a useful tool, not just for testing locally-hosted sites (say, at a device lab), but also for making locally-hosted sites run on HTTPS so you can test service workers.
Thursday, April 27th, 2017
This is a great free service for doing a bit of performance monitoring on your site. It uses WebPageTest and you do all the set up via a Github repo that then displays the results using Github Pages.
Saturday, March 18th, 2017
Hope is a belief that what we do might matter, an understanding that the future is not yet written.
Rebecca Solnit’s piece reminded me of something I mentioned a couple of year’s back when I referred to Margaret Atwood’s phrase “judicious hope”:
Hope sounds like such a wishy-washy word, like “faith” or “belief”, but it carries with it a seed of resistance. Hope, faith, and belief all carry connotations of optimism, but where faith and belief sound passive, even downright complacent, hope carries the promise of action.
Sunday, March 12th, 2017
Excellent and practical advice for before, during, and after research sessions and usability tests.
Tuesday, March 7th, 2017
Smart use of attribute selectors in CSS to catch mistakes in HTML.
Sunday, March 5th, 2017
This is really good fun! And thanks to service workers, it works offline too.
The rounds are:
- Dead or Not Dead,
- Number 1 or Not Number 1, and
- Oscar or Not Oscar.
Thursday, January 19th, 2017
Some interesting insights from usability and accessibility testing at the Co-op.
We used ‘nesting’ to reduce the amount of information on the page when the user first reaches it. When the user chooses an option, we ask for any other details at that point rather than having all the questions on the page at once.
Monday, November 21st, 2016
A little tool for testing common form issues.
- Did we remember to give every input a label? (No, placeholders are not an adequate replacement)?
- Do our labels’ for attributes match our inputs’ ids?
- Did we take advantage of the url, email, and password input types, or did we forget and just use text?
- Are our required fields marked as such?
Sunday, November 6th, 2016
Ire rounds up a bunch of tools you can use to test accessibility, from dev tools to Tenon.