Tags: testing

145

sparkline

Wednesday, April 10th, 2019

Improving accessibility with accessibility acceptance criteria — Paul Hayes

Wouldn’t it be great if every component in your design system had accessibility acceptance criteria? Paul has some good advice for putting those together:

  • Start with accessibility needs
  • Don’t be too generic
  • Don’t define the solution
  • Iterate criteria

Wednesday, March 13th, 2019

Accessibility Insights

A handy accessibility tool. The browser plug-in is only for Chrome right now (I use Firefox as my main browser) but it’s pretty nifty—the tool for visualising tabbing is really useful.

Tuesday, March 5th, 2019

From Ideation to Iteration: Design Thinking for Work and for Life by Una Kravets

The unstoppable engine of An Event Apart in Seattle rolls onward. The second talk of the second day is from the indominatable Una Kravets. Her talk is called From Ideation to Iteration: Design Thinking for Work and for Life. Here’s the description:

Have you ever had a looming deadline and no idea where to start? Do you have a big task to face but are having trouble figuring out how to get there? Have you ever wanted to learn a technology, or build a side project but didn’t know what to build? In this talk, Una will go over an actionable approach and several techniques for applying design thinking to our work and every aspect of our lives. This includes ideating product features, blog post ideas, or even what general direction we want to move toward in our businesses. We’ll go over traditional approaches and breakout techniques that will leave you feeling more in control and ready to reach your goals.

Let’s see if I can keep up with this…

Una’s going to talk about design thinking. Una does a variety of different work outside her day job—a podcast, dev doodles, etc. Sometimes at work she’s given big, big tasks like “build a design system.” Her reaction is “whut?” How do you even start with a task like that.

Also, we make big goals sometimes. Who makes new year’s resolutions? But what does “get more fit” or “earn more income” even mean?

In this talk, Una will break things down and show how design thinking can be applied to anything.

Design thinking

A stategic, solution-based approach to solving problems.

It’s a process. It’s iterative. It’s used by IBM, Apple, GE, and it’s taught to students at a lot of different universities.

Tim Brown of Ideo points out that there’s a Venn diagram of feasability, desirability, and viability. In the middle is the point of innovation.

The steps are:

  1. Empathise — develop a deep understanding of the challenge
  2. Define — clearly articulate the problem.
  3. Ideate — brainstorm potential solutions.
  4. Prototype — design a protoype to
  5. Test — and iterate.

Una feels that the feedback part is potentially missing there. IBM uses a loop diagram to include feedback. Ideo uses these steps:

  1. Frame a question
  2. Gather inspiration
  3. Generate ideas
  4. Make ideas tangible
  5. Test to learn
  6. Share the story

Another way to think about this is how the teams interact. There’s divergence and convergence throughout. Then there’s the double diamond: design, deliver, discover.

Ideo wrote a book called Design Thinking for Libraries. It has some useful tools and diagrams. Una found this fascinating because it wasn’t specifically about products. In healthcare, GE Health used design thinking for their Adventure Series MRI scanners—it resulted in 80% less need to use sedatives. The solution might seem obvious to us in hindsight, but it wouldn’t have been obvious to medical professionals in their everyday busy lives.

Design thinking is bullshit, says Natasha Jen. She describes how it’s become an over-used term that has lost its value. Una can relate—she gets annoyed when there’s too much talking and not enough doing. Design thinking is not diagrams and sticky notes. It’s a process. It’s very much about doing something to shift perspective. It’s another tool in our toolkit, even if it has become an overused term like “synergy.”

Back in 2014, when Una was working at IBM, she thought design thinking was stupid. It seemed to be all talk, talk, talk. It felt tedious. It was 75% talking and 25% development. The balance wasn’t right.

But it’s also true that solutioning too early leads to cruft. If you end up going back to the drawing board, maybe the time could’ve been better spent doing some design thinking up front. Focus on the problem, not the solution.

Now some developers might be thinking that this is outside their area. But it can really help you in your career. It can help you choose technologies. Also, everyone on the team, regardless of role, is responsible for the product.

1. Empathise

Understand your users and the challenge. This could be a task that a user is trying to accomplish, or it could be you trying to get a raise.

Sometimes we forget who our user is. The techniques in this first step helps us solve their needs, not our needs.

You might have many users that you’re trying to help, but try focusing in on a few. You can create personas. When Una was working at Digital Ocean, the users were developers. The personas reflected this. Do the research to get to know your users.

Next, you can create an empathy map for your users. What are their goals? What are their hopes? What will they gain from your product?

Connect the empathy map to a specific context—a goal and or a scenario that the user is going through.

Bear in mind that there are many layers to your user. There are conscious rational thoughts, but also subconscious emotional thoughts. Empathy mapping helps you understand how to best communicate with your user.

Una shares a real-life scenario of hers: create a new shop-able product that increases time on site. That’s a pretty big goal. She creates a persona for a college-educated woman working in the medical field who commutes on the subway, keeps a skin-caring routing, and is getting into cake-making. Next, Una creates an empathy map for this persona. What she says, thinks, feels, and does. All of this is within the context of browsing your fashion media website casually at work.

2: Define

The problem statement should be:

  • Human-centred,
  • Specific, but not too technical (don’t solutionise too soon),
  • Narrow in scope.

How can we best create a product-highlighting web experience that Rosalyn will enjoy to increase her time on site?

You can use a tool with two columns: As-Is and to-Be. The first column is what users currently do. The second column is what you want to achieve.

3. Ideate

This is the fun part. Good old-fashioned brainstorming is good here. Go for quantity here. Get loads of ideas out.

There’s also a “worst possible ideas” game you can play at this stage. It can be a good ice-breaker.

Have a second round of brainstorming where you play the “yes, and…” game to build on the first round.

When Una was working on The Zoe Report, she found that moodboards were really useful. The iteration cycle was very fast. A moodboard allowed them to skip a lot of the back and forth between design and development. They built the website without any visual design mock-ups. They prototyped quickly, tested quickly, and shipped quickly.

Journey-mapping is the next tool you can use in this ideation phase. Map out the steps between the start and end of a user journey. Keep it simple. This is a great time to refine your product and reduce complexity.

Next, start sketching out ideas. Again, this is a great time to uncover issues and solve problems before things get too defined. But remember, when you’re showcasing your ideas in sketches, too many ideas can lead to analyis paralysis.

Oh dear. Jam. Jam. Jam. Jam. Jam. Yes, Una is using the paradox-of-choice jam example …the study who’s findings could not be reproduced.

4. Prototype

Go forth and build. A prototype can exist on a number of different axes:

  • Representation—the form it takes.
  • Precision—the detail it contains.
  • Interactivity—the extent a user can interact with it.
  • Evolution—the life stage it is at.

There are lot of prototyping tools out there. Prototypr.io helps you find the right tool for you. It breaks things down by fidelity and life cycle.

But not all prototyping has to be digital. Paper prototyping only needs pen, paper, and scissors. Some tips:

  • Use a transparency sheet for forms.
  • Use well-visible and mid-tip pens.
  • Draw up your prototype in black and white—people can get caught up in colour.

But on the web, Una recommends getting to digital as quickly as possible because interaction is such a big part of the experience. That’s why Una likes to prototype in code. But this is still a rapid prototyping phase so don’t get too caught up in the details.

5. Test

Testing with internal teams is fine during the ideation phase, but to understand how users will relate to your product, you need to test with representative people. We are not our users.

As well as the user, have a facilitator, a computer, and a scriber. As a facilitator, it’s a good idea to reduce the amount of input you give a user. Don’t hand-hold too much or you will give away your pre-existing knowledge. Encourage your user to be verbal.

Sessioncam is a tool for creating a heatmap of where people are interacting. There are also tools for tracking clicks or mouse hovers. These all feel so utterly icky to me.

The metrics you might be looking to gather could be click-through rate, time-on-site, etc. But, Una cautions, be very wary of adding all these third-party scripts to your site and slowing it down. Who’s testing the A/B tests?

On Bustle, Una wanted to measure interactions on mobile. They tested different UI elements for interactions. They ended up updating the product with a horizontal swiping component. They were able to improve the product and ship a more refined experience.

6. Review and iterate

Una feels that this step is the most important. Analyse your successes and failures, and plan to improve.

Technology changes over time so what’s feasible and viable also changes.

Design thinking on the daily

You can use design thinking in your everyday life. Maybe you want to learn JavaScript, or write blog posts, or get more fit. Una used design thinking brainstorming to break down her goals, categorise and organise them.

“Get better at JavaScript” is a goal that Una has every year.

Empathise. In this situation, the user is you. You can still create a persona of yourself. Define. Why do you want to get better at JavaScript? Is it about making better use of your time? Ideate. There are so many different ways to learn. There are books and video courses. Or you could have a project to focus on. Break. It. Down! Create actionable steps and define how you will measure progress. Match the list of things you want to learn with the list of possible side projects. Prototype. Don’t take it literally. Just build something. Test. You’re testing on yourself in this case. Review. Una does an annual review. It’s a nice therapeutic exercise and helps her move forward into the next year with actionable goals.

Another goal might be “Write a blog post.”

Empathise. Your users are your potential readers. Who do you have in mind? Make personas. Define. What’s the topic? Ideate. If you don’t know what to write about, brainstorm. What are you working on at work that you’re learning from? Select one to try. Prototype. Write. Test. Maybe show it to co-workers. Review. How did it go? Good? Bad? Refine your process for the next blog post.

Here’s a goal: “Buy a gift for someone.”

Empathise. What does this person like? What have they enjoyed receiving in the past? Define. Is the gift something they’ll enjoy for a long time? Something they can share? Ideate. Bounce ideas off friends and relatives. Prototype. In this case, this means getting the gift. Review. Did they like it?

“Get Fit.”

In this case, the review part is probably the most important part.

Marie Kondo asks “Does this spark joy?” Ask the same question of your goals.

Remember, design thinking is not just about talking, and sticky notes. It’s about getting in the right headspace for your users.

Design thinking matters—because everything we do, we do for people. Having the tools to see through the lens of those people will help you be a more well-rounded person.

Wednesday, December 19th, 2018

ANDI - Accessibility Testing Tool - Install

Another bookmarklet for checking accessibility—kind of like tota11y—that allows to preview how screen readers will handle images, focusable elements, and more.

Friday, December 7th, 2018

Browser diversity starts with us. | Zeldman on Web & Interaction Design

Hear, hear!

When one company decides which ideas are worth supporting and which aren’t, which access problems matter and which don’t, it stifles innovation, crushes competition, and opens the door to excluding people from digital experiences.

So how do we fight this? We, who are not powerful? We do it by doubling down on cross-browser testing. By baking it into the requirements on every project, large or small. By making sure our colleagues, bosses, and clients know what we’re doing and why.

Friday, November 23rd, 2018

Responsive Images on the Apple Watch — ericportis.com

Some tips for getting responsive images to work well on the Apple Watch:

  • test your layouts down to 136-px wide
  • include 300w-ish resources in your full-width img’s srcsets
  • art direct to keep image subjects legible
  • say the magic meta words

Monday, November 12th, 2018

Home  |  web.dev

I guess this domain name is why our local developmemnt environments stopped working.

Anyway, it’s a web interface onto Lighthouse (note that it has the same bugs as the version of Lighthouse in Chrome). Kind of like webhint.io.

Sunday, October 28th, 2018

The Three Types of Performance Testing – CSS Wizardry

Harry divides his web performance work into three categories:

  1. Proactive
  2. Reactive
  3. Passive

I feel like a lot of businesses are still unsure where to even start when it comes to performance monitoring, and as such, they never do. By demystifying it and breaking it down into three clear categories, each with their own distinct time, place, and purpose, it immediately takes a lot of the effort away from them: rather than worrying what their strategy should be, they now simply need to ask ‘Do we have one?’

Monday, October 8th, 2018

Notes on prototyping – Ben Frain

Good tips on prototyping using the very materials that the final product will be built in—HTML, CSS, and JavaScript.

The only thing I would add is that, in my experience, it’s vital that the prototype does not morph into the final product …no matter how tempting it sometimes seems.

Prototypes are made to be discarded (having validated or invalidated an idea). Making a prototype and making something for production require very different mindsets: with prototyping it’s all about speed of creation; with production work, it’s all about quality of execution.

Thursday, September 13th, 2018

The Importance Of Manual Accessibility Testing — Smashing Magazine

This is very timely. I’ve been doing some consulting at a company where they are perhaps a little over-reliant on automated accessibility tests.

Automated accessibility tests are a great resource to have, but they can’t automatically make your site accessible. Use them as one step of a larger testing process.

Saturday, September 1st, 2018

Accessibility is not a feature. — Ethan Marcotte

Just last week I came across an example of what Ethan describes here: accessibility (in a pattern library) left to automatic checks rather than human experience.

Friday, July 6th, 2018

Fonty: the new way for testing web fonts

This is very neat! Test out how Google Fonts will look on your website: type in your URL and away you go. Works well on mobile too.

Tuesday, June 12th, 2018

Password Tips From a Pen Tester: Common Patterns Exposed

I’ve been wondering about this for quite a while: surely demanding specific patterns in a password (e.g. can’t be all lowercase, must include at least one number, etc.) makes it easier to crack them, right? I mean, you’re basically providing a ruleset for brute-forcing.

Turns out, yes. That’s exactly right.

When employees are faced with this requirement, they tend to:

  • Choose a dictionary word or a name
  • Make the first character uppercase
  • Add a number at the end, and/or an exclamation point

If we know that is a common pattern, then we know where to start…

Wednesday, May 23rd, 2018

Monday, April 30th, 2018

Apart From Code

A good developer…

  • debugs
  • follows the KISS principle (and respects YAGNI)
  • knows how to research
  • works well with others
  • finds good developer tools
  • tests code

Wednesday, March 7th, 2018

How we’ve made GOV.UK Elements even more accessible

A nice run-down of incremental accessibility improvements made to Gov.uk (I particularly like the technique of updating the title element to use the word “error” if the page is displaying a form that has issues).

Crucially, if any of the problems turned out to be with the browser or screen reader, they submitted bug reports—that’s the way to do it!

Sunday, March 4th, 2018

Accessible By Design | Sparkbox | Web Design and Development

A primer on accessible colour contrast with links to some handy tools for testing.

Sunday, December 3rd, 2017

Saturday, November 18th, 2017

Hooked and booked

At Booking.com, they do a lot of A/B testing.

At Booking.com, they’ve got a lot of dark patterns.

I think there might be a connection.

A/B testing is a great way of finding out what happens when you introduce a change. But it can’t tell you why.

The problem is that, in a data-driven environment, decisions ultimately come down to whether something works or not. But just because something works, doesn’t mean it’s a good thing.

If I were trying to convince you to buy a product, or use a service, one way I could accomplish that would be to literally put a gun to your head. It would work. Except it’s not exactly a good solution, is it? But if we were to judge by the numbers (100% of people threatened with a gun did what we wanted), it would appear to be the right solution.

When speaking about A/B testing at Booking.com, Stuart Frisby emphasised why it’s so central to their way of working:

One of the core principles of our organisation is that we want to be very customer-focused. And A/B testing is really a way for us to institutionalise that customer focus.

I’m not so sure. I think A/B testing is a way to institutionalise a focus on business goals—increasing sales, growth, conversion, and all of that. Now, ideally, those goals would align completely with the customer’s goals; happy customers should mean more sales …but more sales doesn’t necessarily mean happy customers. Using business metrics (sales, growth, conversion) as a proxy for customer satisfaction might not always work …and is clearly not the case with many of these kinds of sites. Whatever the company values might say, a company’s true focus is on whatever they’re measuring as success criteria. If that’s customer satisfaction, then the company is indeed customer-focused. But if the measurements are entirely about what works for sales and conversions, then that’s the real focus of the company.

I’m not saying A/B testing is bad—far from it! (although it can sometimes be taken to the extreme). I feel it’s best wielded in combination with usability testing with real users—seeing their faces, feeling their frustration, sharing their joy.

In short, I think that A/B testing needs to be counterbalanced. There should be some kind of mechanism for getting the answer to “why?” whenever A/B testing provides to the answer to “what?” In-person testing could be one way of providing that balance. Or it could be somebody’s job to always ask “why?” and determine if a solution is qualitatively—and not just quantitatively—good. (And if you look around at your company and don’t see anyone doing that, maybe that’s a role for you.)

If there really is a connection between having a data-driven culture of A/B testing, and a product that’s filled with dark patterns, then the disturbing conclusion is that dark patterns work …at least in the short term.

Monday, November 6th, 2017

Installing Progressive Web Apps

When I was testing the dConstruct Audio Archive—which is now a Progressive Web App—I noticed some interesting changes in how Chrome on Android offers the “add to home screen” prompt.

It used to literally say “add to home screen.”

Getting the “add to home screen” prompt for https://huffduffer.com/ on Android Chrome. And there’s the “add to home screen” prompt for https://html5forwebdesigners.com/ HTTPS + manifest.json + Service Worker = “Add to Home Screen” prompt. Add to home screen.

Now it simply says “add.”

The dConstruct Audio Archive is now a Progressive Web App

I vaguely remember there being some talk of changing the labelling, but I could’ve sworn it was going to change to “install”. I’ve got to be honest, just having the word “add” doesn’t seem to provide much context. Based on the quick’n’dirty usability testing I did with some co-workers, it just made things confusing. “Add what?” “What am I adding?”

Additionally, the prompt appeared immediately on the first visit to the site. I thought there was supposed to be an added “engagement” metric in order for the prompt to appear; that the user needs to visit the site more than once.

You’d think I’d be happy that users will be presented with the home-screen prompt immediately, but based on the behaviour I saw, I’m not sure it’s a good thing. Here’s what I observed:

  1. The user types the URL archive.dconstruct.org into the address bar.
  2. The site loads.
  3. The home-screen prompt slides up from the bottom of the screen.
  4. The user immediately moves to dismiss the prompt (cue me interjecting “Don’t close that!”).

This behaviour is entirely unsurprising for three reasons:

  1. We web designers and web developers have trained users to dismiss overlays and pop-ups if they actually want to get to the content. Nobody’s going to bother to actually read the prompt if there’s a 99% chance it’s going to say “Sign up to our newsletter!” or “Take our survey!”.
  2. The prompt appears below the “line of death” so there’s no way to tell it’s a browser or OS-level dialogue rather than a JavaScript-driven pop-up from the site.
  3. Because the prompt now appears on the first visit, no trust has been established between the user and the site. If the prompt only appeared on later visits (or later navigations during the first visit) perhaps it would stand a greater chance of survival.

It’s still possible to add a Progressive Web App to the home screen, but the option to do that is hidden behind the mysterious three-dots-vertically-stacked icon (I propose we call this the shish kebab icon to distinguish it from the equally impenetrable hamburger icon).

I was chatting with Andreas from Mozilla at the View Source conference last week, and he was filling me in on how Firefox on Android does the add-to-homescreen flow. Instead of a one-time prompt, they’ve added a persistent icon above the “line of death” (the icon is a combination of a house and a plus symbol).

When a Firefox 58 user arrives on a website that is served over HTTPS and has a valid manifest, a subtle badge will appear in the address bar: when tapped, an “Add to Home screen” confirmation dialog will slide in, through which the web app can be added to the Android home screen.

This kind of badging also has issues (without the explicit text “add to home screen”, the user doesn’t know what the icon does), but I think a more persistently visible option like this works better than the a one-time prompt.

Firefox is following the lead of the badging approach pioneered by the Samsung Internet browser. It provides a plus symbol that, when pressed, reveals the options to add to home screen or simply bookmark.

What does it mean to be an App?

I don’t think Chrome for Android has any plans for this kind of badging, but they are working on letting the site authors provide their own prompts. I’m not sure this is such a good idea, given our history of abusing pop-ups and overlays.

Sadly, I feel that any solution that relies on an unrequested overlay is doomed. That’s on us. The way we’ve turned browsing the web—especially on mobile—into a frustrating chore of dismissing unwanted overlays is a classic tragedy of the commons. We blew it. Users don’t trust unrequested overlays, and I can’t blame them.

For what it’s worth, my opinion is that ambient badging is a better user experience than one-time prompts. That opinion is informed by a meagre amount of testing though. I’d love to hear from anyone who’s been doing more detailed usability testing of both approaches. I assume that Google, Mozilla, and Samsung are doing this kind of testing, and it would be really great to see the data from that (hint, hint).

But it might well be that ambient badging is just too subtle to even be noticed by the user.

On one end of the scale you’ve got the intrusiveness of an add-to-home-screen prompt, but on the other end of the scale you’ve got the discoverability problem of a subtle badge icon. I wonder if there might be a compromise solution—maybe a badge icon that pulses or glows on the first or second visit?

Of course that would also need to be thoroughly tested.