Journal tags: research



Priority of design inputs

As you may already know, I’m a nerd for design principles. I collect them. I did a podcast episode on them. I even have a favourite design principle. It’s from the HTML design principles. The priority of constituencies:

In case of conflict, consider users over authors over implementors over specifiers over theoretical purity.

It’s all about priorities, see?

Prioritisation isn’t easy, and it gets harder the more factors come into play: user needs, business needs, technical constraints. But it’s worth investing the time to get agreement on the priority of your constituencies. And then formulate that agreement into design principles.

Jason is also a fan of the priority of constituencies. He recently wrote about applying it to design systems and came up with this:

User needs come before the needs of component consumers, which come before the needs of component developers, which come before the needs of the design system team, which come before theoretical purity.

That got me thinking about how this framing could be applied to other areas, like design.

Designers are used to juggling different needs (or constituencies); user needs, business needs, and so on. But what I’m interested in is how designers weigh up different inputs into the design process.

The obvious inputs are the insights you get from research. But even that can be divided into different categories. There’s qualitative research (talking to people) and qualitative research (sifting through numbers). Which gets higher priority?

There are other inputs too. Take best practices. If there’s a tried and tested solution to a problem, should that take priority over something new and untested? Maybe another way of phrasing it is to call it experience (whether that’s the designer’s own experience or the collective experience of the industry).

And though we might not like to acknowledge it because it doesn’t sound very scientific, gut instinct is another input into the design process. Although maybe that’s also related to experience.

Finally, how do you prioritise stakeholder wishes? What do you do if the client or the boss wants something that conflicts with user needs?

I could imagine a priority of design inputs that looks like this:

Qualitative research over quantitative research over stakeholder wishes over best practices over gut instinct.

But that could change over time. Maybe an experienced designer can put their gut instinct higher in the list, overruling best practices and stakeholder wishes …and maybe even some research insights? I don’t know.

I’ve talked before about how design principles should be reversible in a different context. The original priority of constituencies, for example, applies to HTML. But if you were to invert it, it would work for XML. Different projects have different priorities.

I could certainly imagine company cultures where stakeholder wishes take top billing. There are definitely companies that value qualitative research (data and analytics) above qualitative research (user interviews), and vice-versa.

Is a priority of design inputs something that should change from project to project? If so, maybe it would be good to hammer it out in the discovery phase so everyone’s on the same page.

Anyway, I’m just thinking out loud here. This is something I should chat more about with my colleagues to get their take.

Design research on the Clearleft podcast

We’re halfway through the third season of the Clearleft podcast already!

Episode three is all about design research. I like the narrative structure of this. It’s a bit like a whodunnit, but it’s more like a whydunnit. The “why” question is “why aren’t companies hiring more researchers?”

The scene of the crime is this year’s UX Fest, where talks by both Teresa Torres and Gregg Bernstein uncovered the shocking lack of researchers. From there, I take up the investigation with Maite Otondo and Stephanie Troeth.

I won’t spoil it but by the end there’s an answer to the mystery.

I learned a lot along the way too. I realised how many axes of research there are. There’s qualitative research (stories, emotion, and context) and then there’s quantitative research (volume and data). But there’s also evualative research (testing a hyphothesis) and generative research (exploring a problem space before creating a solution). By my count that gives four possible combos: qualitative evaluative research, quantitative evaluative research, qualitative generative research, and quantitative generative research. Phew!

Steph was a terrific guest. Only a fraction of our conversation made it into the episode, but we chatted for ages.

And Maite kind of blew my mind too, especially when she was talking about the relationship between research and design and she said:

Research is about the present and design is about the future.


I’m going to use that quote again in a future episode. In fact, this episode on design research leads directly into the next two episodes. You won’t want to miss them. So if you’re not already subscribed to the Clearleft podcast, you should get on that, whether it’s via the RSS feed, Apple, Google, Spotify, Overcast, or wherever you get your podcasts from.

Have a listen to this episode on design research and if you’re a researcher yourself, remember that unlike most companies we value research at Clearleft and that’s why we’re hiring another researcher right now. Come and work with us!

Making Research Count by Cyd Harrell

The brilliant Cyd Harrell is opening up day two of An Event Apart in Chicago. I’m going to attempt to liveblog her talk on making research count…

Research gets done …and then sits in a report, gathering dust.

Research matters. But how do we make it count? We need allies. Maybe we need more money. Perhaps we need more participation from people not on the product team.

If you’re doing real research on a schedule, sharing it on a regular basis, making people’s eyes light up …then you’ve won!

Research counts when it answers questions that people care about. But you probably don’t want to directly ask “Hey, what questions do you want answered?”

Research can explain oddities in analytics weird feedback from customers, unexpected uses of products, and strange hunches (not just your own).

Curious people with power are the most useful ones to influence. Not just hierarchical power. Engineers often have a lot of power. So ask, “Who is the most curious engineer, and how can I drag them out on a research session with me?”

At 18F, Cyd found that a lot of the nodes of power were in the mid level of the organisation who had been there a while—they know a lot of people up and down the chain. If you can get one of those people excited about research, they can spread it.

Open up your practice. Demystify it. Put as much effort into communicating as into practicing. Create opportunities for people to ask questions and learn.

You can think about communities of practice in the obvious way: people who do similar things to us, and other people who make design decisions. But really, everyone in the organisation is affected by design decisions.

Cyd likes to do office hours. People can come by and ask questions. You could open a Slack channel. You can run brown bag lunches to train people in basic user research techniques. In more conventional organisations, a newsletter is a surprisingly effective tool for sharing the latest findings from research. And use your walls to show work in progress.

Research counts when people can see it for themselves—not just when it’s reported from afar. Ask yourself: who in your organisation is disconnected from their user? It’s difficult for people to maintain their motivation in that position.

When someone has been in the field with you, the data doesn’t have to be explained.

Whoever’s curious. Whoever’s disconnected. Invite them along. Show them what you’re doing.

Think about the qualities of a good invitation (for a party, say). Make the rules clear. Make sure they want to come back. Design the experience of observing research. Make sure everyone has tools. Give everyone a responsibility. Be like Willy Wonka—he gave clear rules to the invitied guests. And sure, things didn’t go great when people broke the rules, but at the end, everyone still went home with the truckload of chocolate they were promised.

People who get to ask a question buy in to the results. Those people feel a sense of ownership for the research.

Research counts when methods fit the question. Think about what the right question is and how you might go about answering it.

You can mix your methods. Interviews. Diary studies. Card sorting. Shadowing. You can ground the user research in competitor analysis.

Back in 2008, Cyd was contacted by a company who wanted to know: how do people really use phones in their cars? Cyd’s team would ride along with people, interviewing them, observing them, taking pictures and video.

Later at the federal government, Cyd was asked: what are the best practices for government digital transformation? How to answer that? It’s so broad! Interviews? Who knows what?

They refined the question: what makes modern digital practices stick within a government entity? They looked at what worked when companies were going online, so see if there was anything that government could learn from. Then they created a set of really focused interview questions. What does digital transformation mean? How do you know when you’re done? What are the biggest obstacles to this work? How do you make changes last?

They used atechnique called cluster recruiting to figure out who else to talk to (by asking participants who else they should be talking to).

There is no one research method that will always work for you. Cutting the right corners at the right time lets you be fast and cheap. Cyd’s bare-bones research kit costs about $20: a notebook, a pen, a consent form, and the price of a cup of coffee. She also created a quick score sheet for when she’s not in a position to have research transcribed.

Always label your assumptions before beginning your research. Maybe you’re assuming that something is a frustrating experience that needs fixing, but it might emerge that it doesn’t need fixing—great! You’ve just saved a whole lotta money.

Research counts when researchers tell the story well. Synthesis works best as a conversational practice. It’s hard to do by yourself. You start telling stories when you come back from the field (sometimes it starts when you’re still out in the field, talking about the most interesting observations).

Miller’s Law is a great conceptual framework:

To understand what another person is saying, you must assume that it is true and try to imagine what it could be true of.

You’re probably familiar with the “five whys”. What about the “five ways”? If people talk about something five different ways, it’s virtually certain that one of them will be an apt metaphor. So ask “Can you say that in a different way?” five time.

Spend as much time on communicating outcomes as you did on executing the work.

After research, play back how many people you spoke to, the most valuable insight you gained, the themes that are emerging. Describe the question you wanted to answer, what answers you got, and what you’re going to do next. If you’re in an organisation that values memos, write a memo. Or you could make a video. Or you could write directly into backlog tickets. And don’t forget the wall work! GDS have wonderfully full walls in their research department.

In the end, the best tool for research is an illuminating story.

Cyd was doing research at the Bakersfield courthouse. The hypothesis was that a lot of people weren’t engaging with technology in the court system. She approached a man named Manuel who was positively quaking. He was going through a custody battle. He said, “I don’t know technology but it doesn’t scare me. I’m shaking because this paperwork just gets to me—it’s terrifying.” He said who would gladly pay for someone to help him with the paperwork. Cyd wrote a report on this story. Months later, they heard people in the organisation asking questions like “How would this help Manuel?”

Sometimes you do have to fight (nicely).

People will push back on the time spent on research—they’ll say it doesn’t fit the sprint plan. You can have a three day research plan. Day 1: write scripts. Day 2: go to the users and talk to them. Day 3: play it back. People on a project spend more time than that in Slack.

People will say you can’t talk to the customers. In that situation, you could talk to people who are in the same sector as your company’s customers.

People will question the return on investment for research. Do it cheaply and show the very low costs. Then people stop talking about the money and start talking about the results.

People will claim that qualitative user research is not statistically significant. That’s true. But research is something else. It answers different question.

People will question whether a senior person needs to be involved. It is not fair to ask the intern to do all the work involved in research.

People will say you can’t always do research. But Cyd firmly believes that there’s always room for some research.

  • Make allies in customer research.
  • Find the most curious engineer on the team, go to lunch with them, and feed them the most interesting research insights.
  • Record a pain point and a send a video to executives.
  • If there’s really no budget, maybe you can get away with not paying incentives, but perhaps you can provide some other swag instead.

One of the best things you can do is be there, non-judgementally, making friends. It takes time, but it works. Research is like a dandelion in flight. Once it’s out and about, taking root, the more that research counts.


There’s a veritable smörgåsbord of great workshops on the horizon…

Clearleft presents a workshop with Jan Chipchase on field research in London on May 29th, and again on May 30th. The first day is sold out, but there are still tickets available for the second workshop (tickets are £654). If you’ve read Jan’s beautiful Field Study Handbook, then you’ll know what a great opportunity it is to spend a day in his company. But don’t dilly-dally—that second day is likely to sell out too.

This event is for product teams, designers, researchers, insights teams, in agencies, in-house, local and central government. People who are curious about human interaction, and their place in the world.

I’m really excited that Sarah and Val are finally bringing their web animation workshop to Brighton (I’ve been not-so-subtly suggesting that they do this for a while now). It’s a two day workshop on July 9th and 10th. There are still some tickets available, but probably not for much longer (tickets are £639). The workshop is happening at 68 Middle Street, the home of Clearleft.

This workshop will get you up and running with web animation in less time than it would take to read all the tutorials you have bookmarked. Over two days, you’ll go from beginner or novice web animator to having expert level knowledge of the current web animation landscape. You’ll get an in-depth look at animating with CSS, JavaScript, and SVG through hands-on exercises and learn the most efficient workflows for each.

A bit before that, though, there’s a one-off workshop on responsive web typography from Rich on Thursday, June 29th, also at 68 Middle Street. You can expect the same kind of brilliance that he demonstrated in his insta-classic Web Typography book, but delivered by the man himself.

You will learn how to combine centuries-old craft with cutting edge technology, including variable fonts, to design and develop for screens of all shapes and sizes, and provide the best reading experiences for your modern readers.

Whether you’re a designer or a developer, just starting out or seasoned pro, there will be plenty in this workshop to get your teeth stuck into.

Tickets are just £435, and best of all, that includes a ticket to the Ampersand conference the next day (standalone conference tickets are £235 so the workshop/conference combo is a real bargain). This year’s Ampersand is shaping up to be an unmissable event (isn’t it always?), so the workshop is like an added bonus.

See you there!

Links from a talk

In two weeks time, I’ll be in Seattle for An Event Apart. I’ll be giving a brand new talk. The title is The Way Of The Web (although perhaps a more accurate title would be The Layers Of The Web).

Here’s the description:

Do you ever get overwhelmed by the ever-changing nature of web design and development? Exhausting, isn’t it? How are you supposed to know which technologies and tools you should invest your time in? Will they stick around or will you just have to relearn everything in another few months? Join Jeremy as he takes a tour of the past, present, and future of working on the web. From the building blocks of HTML, CSS, and JavaScript through to frameworks and libraries right up to the latest and greatest Progressive Web Apps, this talk will examine our collective assumptions with a critical eye. By learning from the past, we can make sensible design decisions today to build the web of tomorrow.

There’s a direct evolution line from my previous talks—Resilence and Evaluating Technology—to this new one. (Spoiler: everything I talk about is in some way related to progressive enhancement …even if I never use the words “progressive" or “enhancement" in the talks.)

I’ve been preparing this new talk for months. It started with a mind map—an A3 sheet of paper with disconnected thoughts, like something from the scene in the crime movie where they enter the lair of the serial killer and find a crazy wall.

Then I set it aside and began procrastinating. But it was the good kind of procrastinating, right? I mean, I had made a start and all those thoughts were now bubbling around in my head.

Eventually I forced myself to put things in some sort of order and started creating slides. That’s the beginning of the horrible process bouncing between thinking “this is pretty good!” and “this is absolute crap!” To be honest, I never actually know if a talk is any good until I give it in front of an audience (practice runs at work are great for getting feedback but they’re not the same as doing the talk for real).

Anyway, I think the talk is ready to roll. If you see me giving this talk and you’re interested in diving deeper into the topics raised, I’ve gathered together some of sources I used.

Further Reading

Related posts on

Progressive Web Apps




This year’s Render conference just wrapped up in Oxford. It was a well-run, well-curated event, right up my alley: two days of a single track of design and development talks (see also: An Event Apart and Smashing Conference for other events in this mold that get it right).

One of my favourite talks was from Frances Ng. She gave a thoroughly entertaining account of her journey from aerospace engineer to front-end engineer, filled with ideas about how to get started, and keep from getting overwhelmed in the world of the web.

She recommended taking the time to occasionally dive deep into a foundational topic, pointing to another talk as a perfect example; Ana Balica gave a great presentation all about HTTP. The second half of the talk was about HTTP 2 and was filled with practical advice, but the first part was a thoroughly geeky history of the Hypertext Transfer Protocol, which I really loved.

While I’m mentoring Amber, we’ve been trying to find a good balance between those deep dives into the foundational topics and the hands-on day-to-day skills needed for web development. So far, I think we’ve found a good balance.

When Amber is ‘round at the Clearleft office, we sit down together and work on the practical aspects of HTML, CSS, and (soon) JavaScript. Last week, for example, we had a really great day diving into CSS selectors and specificity—I watched Amber’s knowledge skyrocket over the course of the day.

But between those visits—which happen every one or two weeks—I’ve been giving Amber homework of sorts. That’s where the foundational building blocks come in. Here are the questions I’ve asked so far:

  • What is the difference between the internet and the web?
  • What is the difference between GET and POST?
  • What are cookies?

The first question is a way of understanding the primacy of URLs on the web. Amber wrote about her research. The second question was getting at an understanding of HTTP. Amber wrote about that too. The third and current question is about state on the web. I’m looking forward to reading a write-up of that soon.

We’re still figuring out this whole mentorship thing but I think this balance of research and exercises is working out well.

Research on evaluating technology

I’ve spent the past few months preparing a new talk for An Event Apart San Francisco (and hopefully some more AEAs after that). As always happens, I spent the whole time vacillating between thinking “this is good!” and thinking “this is awful!” I’m still bouncing between those poles. I won’t really know whether the talk is up to snuff until I actually give it to a live audience.

Over the past few years, my presentations have built upon one another. Two years ago, my talk was called Enhance! and it set the groundwork for using a layered approach to web design and development. My 2016 talk, Resilience, follows on with a process and examples for that approach (I also set myself the challenge of delivering a talk about progressive enhancement without ever using the phrase “progressive enhancement”).

My new talk goes a bit meta, but in my mind, it’s very much building on the previous talks. The talk is all about evaluating technology. I haven’t settled on a final title, but I was thinking about something obtuse, like …Evaluating Technology.

Here’s my hastily scribbled description:

We work with technology every day. And every day it seems like there’s more and more technology to understand: graphic design tools, build tools, frameworks and libraries, not to mention new HTML, CSS and JavaScript features landing in browsers. How should we best choose which technologies to invest our time in? When we decide to weigh up the technology choices that confront us, what are the best criteria for doing that? This talk will help you evaluate tools and technologies in a way that best benefits the people who use the websites that we are designing and developing. Let’s take a look at some of the hottest new web technologies like service workers and web components. Together we will dig beneath the hype to find out whether they will really change life on the web for the better.

As ever, I’ll begin and end with a long-zoom pretentious arc of history, but I’ll dive into practical stuff in the middle. That’s become a bit of a cliché for my presentations, but the formula works as a sort of microcosm of a good conference—a mixture of the inspirational and the practical, trying to keep a good balance of both.

For this new talk, the practical focus will be on some web technologies that are riding high on the hype cycle right now: service workers, web components, progressive web apps. I’ll use them as a lens for applying broader questions about how we make decisions about the technologies we embrace, and the technologies we reject.

Technology. Now there’s a big subject. It’s literally the entirety of human history. I had to be careful not to go down too many rabbit holes. I’m still not sure if I’ve succeeded, but I’ve already had to ruthlessly cull some darlings.

One of the nice things that the An Event Apart crew started doing was to provide link lists for each talk to attendees. That gives me an opportunity to touch briefly on a topic in the talk itself, but allow any interested attendees to dive deeper at their leisure.

For this talk on evaluating technology, I’ve put together this list of hyperlinks for further reading, watching, listening, and researching…





Wibbly-wobbly timey-wimey stuff

I met up with Remy a few months back to try to help him finalise the line-up for this year’s Full Frontal conference. Remy puts a lot of thought into crafting a really solid line-up. He was in a good position too: the conference was already sold out so he didn’t have to worry about having a big-name speaker to put bums on seats—he could concentrate entirely on finding just the right speaker for the final talk.

He described the kind of “big picture” talk he was looking for, and I started naming some names and giving him some ideas of people to contact.

Imagine my surprise then, when—while we were both in New York for Brooklyn Beta—I received a lengthy email from Remy (pecked out on his phone), saying that he had decided who wanted to do the closing talk at Full Frontal. He wanted me to do it.

Now, this was just a couple of weeks ago so my first thought was “No way! I don’t have enough time to prepare a talk.” It takes me quite a while to prepare a new presentation.

But then he described—in quite some detail—what he wanted me to talk about …and it’s exactly the kind of stuff that I really enjoy geeking out about: long-term thinking, digital preservation, and all that jazz. So I said yes.

That’s why I’ve spent the last couple of weeks quietly freaking out, attempting to marshall my thoughts and squeeze them into Keynote. The title of my talk is Time. Pretentious? Moi?

I’m trying to pack a lot into this presentation. I’ve already had to kill some of my darlings and drop some of the more esoteric stuff, but damn it, it’s hard to still squeeze everything in.

I’ve been immersed in research and link-making, reading and huffduffing all things time-related. In the course of my hypertravels, I discovered that there’s an entire event devoted to “the origins, evolution, and future of public time.” It’s called Time For Everyone and it’s taking place in California …at exactly the same time as Full Frontal.

Here’s the funny thing: the description for the event is exactly the same as the description I gave Remy for my talk:

This thing all things devours:
Birds, beasts, trees, flowers;
Gnaws iron, bites steel;
Grinds hard stones to meal;
Slays king, ruins town,
And beats high mountain down.

If you’re coming along to Full Frontal next Friday, I hope you’ll be in a receptive mood. I also hope that Remy won’t mind that what I’m going to present isn’t exactly what he asked for …but I think it’s interesting stuff.

I just wish I had more time.

Testing Huffduffer’s sign-up

Ever since I launched Huffduffer, one of the features that really caught people’s attention was the sign up form.

I have to admit, I didn’t really think it was that revolutionary an idea. All I was trying to do was make the sign-up process a little friendlier and if web standards have taught us anything, it’s that there’s nothing inherent in the presentation of any element, much less forms. So I made the form more conversational and less blocky and rigid.

Well, it turns out that people love it. I’ve received bucketloads of Twitter messages and emails from people telling me how much they enjoyed the sign-up process.

But amongst all the positive comments I saw about the sign-up form when Huffduffer launched, I saw some armchair UX practitioners wondering about the usability of this somewhat unorthodox approach to forms. Fair point. Without user testing, how can I really know if the mad-libs approach is really working?

Now, it happens that Luke W. likes the Huffduffer sign-up form, as evidenced by a recent chat he had with Jared.

SpoolCast: Moving Beyond Static Forms with Luke Wroblewski on Huffduffer

If anyone knows anything about the usability of web forms, it’s Luke. He literally wrote the book on it.

Not content with simply expressing a liking for the Huffduffer-style of human-friendly form presentation, he decided to put it to the test with

After seeing the Huffduffer form in action, I was curious how it would perform against a traditional form. Would people be more inclined to complete it because of the narrative format? Or would the unfamiliar presentation format confuse people? Thanks to Ron Kurti and the team at, I now have some early answers.

Ron and his team ran some A/B testing online that compared a traditional Web form layout with a narrative “Mad Libs” format. In’s testing, Mad Libs style forms increased conversion across the board by 25-40%.

That seems to be a statistically-significant result, even accounting for Cennydd’s reality-check on A/B testing.

It’ll be interesting to see if this is the start of a trend. If nothing else, it’s a way of getting designers to think about the presentation of common human-computer interactions, such as signing up to a new website.