Tags: testing

136

sparkline

Thursday, September 13th, 2018

The Importance Of Manual Accessibility Testing — Smashing Magazine

This is very timely. I’ve been doing some consulting at a company where they are perhaps a little over-reliant on automated accessibility tests.

Automated accessibility tests are a great resource to have, but they can’t automatically make your site accessible. Use them as one step of a larger testing process.

Saturday, September 1st, 2018

Accessibility is not a feature. — Ethan Marcotte

Just last week I came across an example of what Ethan describes here: accessibility (in a pattern library) left to automatic checks rather than human experience.

Friday, July 6th, 2018

Fonty: the new way for testing web fonts

This is very neat! Test out how Google Fonts will look on your website: type in your URL and away you go. Works well on mobile too.

Tuesday, June 12th, 2018

Password Tips From a Pen Tester: Common Patterns Exposed

I’ve been wondering about this for quite a while: surely demanding specific patterns in a password (e.g. can’t be all lowercase, must include at least one number, etc.) makes it easier to crack them, right? I mean, you’re basically providing a ruleset for brute-forcing.

Turns out, yes. That’s exactly right.

When employees are faced with this requirement, they tend to:

  • Choose a dictionary word or a name
  • Make the first character uppercase
  • Add a number at the end, and/or an exclamation point

If we know that is a common pattern, then we know where to start…

Wednesday, May 23rd, 2018

Monday, April 30th, 2018

Apart From Code

A good developer…

  • debugs
  • follows the KISS principle (and respects YAGNI)
  • knows how to research
  • works well with others
  • finds good developer tools
  • tests code

Wednesday, March 7th, 2018

How we’ve made GOV.UK Elements even more accessible

A nice run-down of incremental accessibility improvements made to Gov.uk (I particularly like the technique of updating the title element to use the word “error” if the page is displaying a form that has issues).

Crucially, if any of the problems turned out to be with the browser or screen reader, they submitted bug reports—that’s the way to do it!

Sunday, March 4th, 2018

Accessible By Design | Sparkbox | Web Design and Development

A primer on accessible colour contrast with links to some handy tools for testing.

Sunday, December 3rd, 2017

Saturday, November 18th, 2017

Hooked and booked

At Booking.com, they do a lot of A/B testing.

At Booking.com, they’ve got a lot of dark patterns.

I think there might be a connection.

A/B testing is a great way of finding out what happens when you introduce a change. But it can’t tell you why.

The problem is that, in a data-driven environment, decisions ultimately come down to whether something works or not. But just because something works, doesn’t mean it’s a good thing.

If I were trying to convince you to buy a product, or use a service, one way I could accomplish that would be to literally put a gun to your head. It would work. Except it’s not exactly a good solution, is it? But if we were to judge by the numbers (100% of people threatened with a gun did what we wanted), it would appear to be the right solution.

When speaking about A/B testing at Booking.com, Stuart Frisby emphasised why it’s so central to their way of working:

One of the core principles of our organisation is that we want to be very customer-focused. And A/B testing is really a way for us to institutionalise that customer focus.

I’m not so sure. I think A/B testing is a way to institutionalise a focus on business goals—increasing sales, growth, conversion, and all of that. Now, ideally, those goals would align completely with the customer’s goals; happy customers should mean more sales …but more sales doesn’t necessarily mean happy customers. Using business metrics (sales, growth, conversion) as a proxy for customer satisfaction might not always work …and is clearly not the case with many of these kinds of sites. Whatever the company values might say, a company’s true focus is on whatever they’re measuring as success criteria. If that’s customer satisfaction, then the company is indeed customer-focused. But if the measurements are entirely about what works for sales and conversions, then that’s the real focus of the company.

I’m not saying A/B testing is bad—far from it! (although it can sometimes be taken to the extreme). I feel it’s best wielded in combination with usability testing with real users—seeing their faces, feeling their frustration, sharing their joy.

In short, I think that A/B testing needs to be counterbalanced. There should be some kind of mechanism for getting the answer to “why?” whenever A/B testing provides to the answer to “what?” In-person testing could be one way of providing that balance. Or it could be somebody’s job to always ask “why?” and determine if a solution is qualitatively—and not just quantitatively—good. (And if you look around at your company and don’t see anyone doing that, maybe that’s a role for you.)

If there really is a connection between having a data-driven culture of A/B testing, and a product that’s filled with dark patterns, then the disturbing conclusion is that dark patterns work …at least in the short term.

Monday, November 6th, 2017

Installing Progressive Web Apps

When I was testing the dConstruct Audio Archive—which is now a Progressive Web App—I noticed some interesting changes in how Chrome on Android offers the “add to home screen” prompt.

It used to literally say “add to home screen.”

Getting the “add to home screen” prompt for https://huffduffer.com/ on Android Chrome. And there’s the “add to home screen” prompt for https://html5forwebdesigners.com/ HTTPS + manifest.json + Service Worker = “Add to Home Screen” prompt. Add to home screen.

Now it simply says “add.”

The dConstruct Audio Archive is now a Progressive Web App

I vaguely remember there being some talk of changing the labelling, but I could’ve sworn it was going to change to “install”. I’ve got to be honest, just having the word “add” doesn’t seem to provide much context. Based on the quick’n’dirty usability testing I did with some co-workers, it just made things confusing. “Add what?” “What am I adding?”

Additionally, the prompt appeared immediately on the first visit to the site. I thought there was supposed to be an added “engagement” metric in order for the prompt to appear; that the user needs to visit the site more than once.

You’d think I’d be happy that users will be presented with the home-screen prompt immediately, but based on the behaviour I saw, I’m not sure it’s a good thing. Here’s what I observed:

  1. The user types the URL archive.dconstruct.org into the address bar.
  2. The site loads.
  3. The home-screen prompt slides up from the bottom of the screen.
  4. The user immediately moves to dismiss the prompt (cue me interjecting “Don’t close that!”).

This behaviour is entirely unsurprising for three reasons:

  1. We web designers and web developers have trained users to dismiss overlays and pop-ups if they actually want to get to the content. Nobody’s going to bother to actually read the prompt if there’s a 99% chance it’s going to say “Sign up to our newsletter!” or “Take our survey!”.
  2. The prompt appears below the “line of death” so there’s no way to tell it’s a browser or OS-level dialogue rather than a JavaScript-driven pop-up from the site.
  3. Because the prompt now appears on the first visit, no trust has been established between the user and the site. If the prompt only appeared on later visits (or later navigations during the first visit) perhaps it would stand a greater chance of survival.

It’s still possible to add a Progressive Web App to the home screen, but the option to do that is hidden behind the mysterious three-dots-vertically-stacked icon (I propose we call this the shish kebab icon to distinguish it from the equally impenetrable hamburger icon).

I was chatting with Andreas from Mozilla at the View Source conference last week, and he was filling me in on how Firefox on Android does the add-to-homescreen flow. Instead of a one-time prompt, they’ve added a persistent icon above the “line of death” (the icon is a combination of a house and a plus symbol).

When a Firefox 58 user arrives on a website that is served over HTTPS and has a valid manifest, a subtle badge will appear in the address bar: when tapped, an “Add to Home screen” confirmation dialog will slide in, through which the web app can be added to the Android home screen.

This kind of badging also has issues (without the explicit text “add to home screen”, the user doesn’t know what the icon does), but I think a more persistently visible option like this works better than the a one-time prompt.

Firefox is following the lead of the badging approach pioneered by the Samsung Internet browser. It provides a plus symbol that, when pressed, reveals the options to add to home screen or simply bookmark.

What does it mean to be an App?

I don’t think Chrome for Android has any plans for this kind of badging, but they are working on letting the site authors provide their own prompts. I’m not sure this is such a good idea, given our history of abusing pop-ups and overlays.

Sadly, I feel that any solution that relies on an unrequested overlay is doomed. That’s on us. The way we’ve turned browsing the web—especially on mobile—into a frustrating chore of dismissing unwanted overlays is a classic tragedy of the commons. We blew it. Users don’t trust unrequested overlays, and I can’t blame them.

For what it’s worth, my opinion is that ambient badging is a better user experience than one-time prompts. That opinion is informed by a meagre amount of testing though. I’d love to hear from anyone who’s been doing more detailed usability testing of both approaches. I assume that Google, Mozilla, and Samsung are doing this kind of testing, and it would be really great to see the data from that (hint, hint).

But it might well be that ambient badging is just too subtle to even be noticed by the user.

On one end of the scale you’ve got the intrusiveness of an add-to-home-screen prompt, but on the other end of the scale you’ve got the discoverability problem of a subtle badge icon. I wonder if there might be a compromise solution—maybe a badge icon that pulses or glows on the first or second visit?

Of course that would also need to be thoroughly tested.

Wednesday, September 27th, 2017

Canonical test podcasts (Joe Clark)

Are you the creator, programmer, or quality-tester of a podcasting application? This page provides a range of podcasts that exemplify a range of atypical use case from merely uncommon to exceedingly fringe. If your app can handle all these, you’re doing well.

The Coming Software Apocalypse - The Atlantic

The title is pure clickbait, and the moral panic early in this article repeats the Toyota myth, but then it settles down into a fascinating examination of abstractions in programming. On the one hand, there’s the problem of the not enough abstraction: having to write in code is such a computer-centric way of building things. On the other hand, our world is filled with dangerously abstracted systems:

When your tires are flat, you look at your tires, they are flat. When your software is broken, you look at your software, you see nothing.

So that’s a big problem.

Bret Victor, John Resig and Margaret Hamilton are featured. Doug Engelbart and J.C.R. Licklider aren’t mentioned but their spirits loom large.

Thursday, September 21st, 2017

Chrome to force .dev domains to HTTPS via preloaded HSTS

Well, I guess it’s time to change all my locally-hosted sites from .dev domains to .test. Thanks, Google.

Thursday, July 27th, 2017

Testing the accessibility of pattern libraries

Riffing on Rachel’s talk at Patterns Day:

At the Patterns Day conference last month, Rachel Andrew mentioned something interesting about patterns. She said that working with reusable interface components, where each one has its own page, made her realise that those work quite well as isolated test cases. I feel this also goes for some accessibility tests: there is a number of criteria where isolation aids testing.

Hidde specifically singles out these patterns:

  • Collapsible (“Show/hide”)
  • Form field
  • Video player

Sunday, July 23rd, 2017

What I’ve learned about motor impairment

James gives—if you’ll pardon the pun— hands-on advice on making sites that consider motor impairment:

  • Don’t assume keyboard access is all you need
    • Auto complete/Autofill
    • Show me my password
  • Allow for fine motor control issues
    • Don’t autoplay videos
    • Avoid hover-only controls
    • Infinite scrolling considerations
  • Be mindful of touch
    • Avoid small hit targets
    • Provide alternate controls for touch gestures

Far from being a niche concern, visitors with some form of motor impairment likely make up a significant percentage of your users. I would encourage you to test your website or application with your less dominant hand. Is it still easy to use?

Tuesday, June 6th, 2017

Left to our own devices. — Ethan Marcotte

Your website’s only as strong as the weakest device you’ve tested it on.

Wednesday, May 31st, 2017

Emmet Re:view — fast and easy way to test responsive design in multiple viewports

It’s no substitute for testing with real devices, but the “device wall” view in this Chrome plug-in is a nifty way of getting an overview of a site’s responsiveness at a glance.

Sunday, May 14th, 2017

ngrok - secure introspectable tunnels to localhost

This looks like a useful tool, not just for testing locally-hosted sites (say, at a device lab), but also for making locally-hosted sites run on HTTPS so you can test service workers.

Sunday, March 12th, 2017

Design Ethics in Practice – The Interconnected

Excellent and practical advice for before, during, and after research sessions and usability tests.