Browser testing

On just about every client project that I work on, the subject of browser support comes up. Rightly so. It’s an important issue on which to get mutual understanding and agreement. But all too often, this important question is framed in a binary, true/false, go/no-go way: “Which browsers do we/don’t we support?”

Really, the first thing to get agreement on is not a list of browsers, but what we mean by the word “support”. In my mind, that word implies that a user of a particular browser should be able to accomplish the primary tasks on the website, whether that’s reading an article, booking a ticket, or buying a product. That doesn’t mean that the task must be experienced in pixel-perfect fidelity to an ideal visual design.

But to others, that’s exactly what “support” means. Personally, I’d call that optimisation. As Brad puts it:

There is a difference between support and optimization.

So to put it in glib terms, I support every browser …but I optimise for none.

Alright, fine. But I still need to get to some mutual understanding with a client about which browsers will get the optimised experience and which browsers will simply be supported.

Personally, I like the Filament Group’s approach of discussing this in terms of features rather than browsers. It makes sense to me to say the browsers that support geolocation will get the geolocation features, or the browsers that support offline caching will get the offline caching features. There’s no need to produce a list of what those browsers are for each feature, and in any case, the list would be constantly changing and updating with each new browser release.

But—and this is a big but—nine times out of ten, when the issue of browser support comes up, it isn’t about functionality; it’s about branding. What clients generally want to know is which browsers will get the ideal visual design. Obviously the newer versions of Chrome and Firefox are going to get all the lovely layouts, rounded corners, gradients, transparencies, and animations …but what about older versions of Internet Explorer? Even if users of IE8 and IE7 can accomplish their tasks, will the “degraded” visual presentation hurt their experience?

My hypothesis is that it won’t. Users of older versions of Internet Explorer aren’t doing a side-by-side comparison of the same website opened up in the latest Chrome nightly. Considering what their daily usage must be like—unable to use Facebook, unable to use Google services—I suspect that they are happy just to be able to complete their task, regardless of the site’s visual fidelity.

There’s another viewpoint—one that I’ve heard expressed by clients—that even users of older browsers should still get the ideal, pixel-perfect visual design. The hypothesis here is that, by allowing someone to experience anything less than the perfect presentation, the client’s brand will be damaged in the mind of that person.

Like I said, this is something that comes up on most client projects, and this is the point at which we’d have to come to an agreement about which hypothesis we’re going to go with. Of course I’m going to argue in favour of the first hypothesis, but I’ve come to realise that arguing in favour of either hypothesis is the wrong approach. We shouldn’t be debating this …we should be testing it.

We have two competing hypotheses about a group of users. Instead of trying to read their minds, why not test with that group of users to find out which hypothesis is correct? No matter what the results of the test, they will be valuable either way.

Think about the amount of work that’s going to go in to optimising for older browser versions—it’s going to take quite a bit of time and money. It makes sense to ensure that this time and money isn’t being spent on little more than a hunch that pixel-perfection is important to those users. On the other hand, if the test reveals that actually those users really will have a lesser opinion of a brand unless they get pixel-perfect parity with newer browsers, then you’ll know that the time and money spent making that happen isn’t wasted.

Josh wrote recently that 1 hour of research saves 10 hours of development time:

Or, in longer terms if more people appreciated how one day of user research can save weeks of coding I think they would do it more. It is remarkable what you decide to not build after talking to a few people closely.

When it comes to decisions around browser support/optimisation, I think that even a little bit of up-front research and testing could potentially save a lot of time, money, and heartache. I’m not sure exactly what form the testing should take, but I’m interested in figuring it out.

Have you published a response to this? :

Responses

tommorris.org

Jeremy Keith has smart things to say about browser support.

I’ve been screwed over in the past by saying “we’re going to support IE9, IE10 and all versions of Firefox, Safari and Chrome that people actually use”. I’ve done this based on solid data—basically, going to Google Analytics, dumping the data out and applying the Pareto Principle and a craptastic Python script I have that extracts the information I need out of the morass of Google Analytics mess.

The problem occurs because then (a) we don’t build the back-end in a sane, reasonable, agile1 way (like, say, having good programmers building it in Django, instead management decides that it needs to be built by enterprise Java devs in some monstrosity of a CMS that refers to itself as an “enterprise portal engine” or some other bullshit) and (b) the front-end gets built in some crack-addled, buzzword-compliant JavaScript framework picked because it is on Hacker News and is sexy. If we built websites in a sane and rational way, this shit would be so much less complicated.

  1. By ‘agile’ in this context, I mean simply that it is built with technologies that make it easy to adapt to change based on feedback from design and front-end developers. Like, say, Django or Rails rather than Spring, and deployed on Heroku (at least during development) rather than some half-baked ops process.

# Saturday, May 23rd, 2015 at 10:41am

Jeremy Keith

RT @jgarber: Really, really smart words from @adactio on browser “support” and user testing. adactio.com/journal/8982

Alex Hardy

RT @jgarber: Really, really smart words from @adactio on browser “support” and user testing. adactio.com/journal/8982

# Posted by Alex Hardy on Sunday, May 24th, 2015 at 11:27am

Jeff Jones

RT @jgarber: Really, really smart words from @adactio on browser “support” and user testing. adactio.com/journal/8982

# Posted by Jeff Jones on Sunday, May 24th, 2015 at 11:29am

html and bacon

RT @jgarber: Really, really smart words from @adactio on browser “support” and user testing. adactio.com/journal/8982

Jaron Ghani

RT @jgarber: Really, really smart words from @adactio on browser “support” and user testing. adactio.com/journal/8982

sll

RT @jgarber: Really, really smart words from @adactio on browser “support” and user testing. adactio.com/journal/8982

# Posted by sll on Sunday, May 24th, 2015 at 11:49am

Julia Beach

RT @jgarber: Really, really smart words from @adactio on browser “support” and user testing. adactio.com/journal/8982

Michael Jovel

RT @jgarber: Really, really smart words from @adactio on browser “support” and user testing. adactio.com/journal/8982

Maz Ameli

RT @jgarber: Really, really smart words from @adactio on browser “support” and user testing. adactio.com/journal/8982

# Posted by Maz Ameli on Sunday, May 24th, 2015 at 1:22pm

Javier Rios

RT @jgarber: Really, really smart words from @adactio on browser “support” and user testing. adactio.com/journal/8982

matthias haak

RT @jgarber: Really, really smart words from @adactio on browser “support” and user testing. adactio.com/journal/8982

James Bell

RT @jgarber: Really, really smart words from @adactio on browser “support” and user testing. adactio.com/journal/8982

# Posted by James Bell on Sunday, May 24th, 2015 at 8:12pm

22 Likes

# Liked by html and bacon on Sunday, May 24th, 2015 at 12:35pm

# Liked by Marc Drummond on Sunday, May 24th, 2015 at 12:35pm

# Liked by Leroy Kirby on Sunday, May 24th, 2015 at 12:36pm

# Liked by James Barley on Sunday, May 24th, 2015 at 12:56pm

# Liked by Jaron Ghani on Sunday, May 24th, 2015 at 12:56pm

# Liked by Pascal Birchler on Sunday, May 24th, 2015 at 12:57pm

# Liked by Alex Nemeroff on Sunday, May 24th, 2015 at 1:32pm

# Liked by Marc Friederich on Sunday, May 24th, 2015 at 1:32pm

# Liked by Arunan Skanthan on Sunday, May 24th, 2015 at 1:58pm

# Liked by John Morrison on Sunday, May 24th, 2015 at 1:59pm

# Liked by Logan Scavo on Sunday, May 24th, 2015 at 2:11pm

# Liked by darkroom7 on Sunday, May 24th, 2015 at 2:24pm

# Liked by Ricardo Gladwell on Sunday, May 24th, 2015 at 2:24pm

# Liked by Mark Huot on Sunday, May 24th, 2015 at 2:39pm

# Liked by geertmelotte on Sunday, May 24th, 2015 at 2:57pm

# Liked by Michael Wiik on Sunday, May 24th, 2015 at 3:30pm

# Liked by hans verhaegen on Sunday, May 24th, 2015 at 6:22pm

# Liked by Tom Kraak on Sunday, May 24th, 2015 at 7:58pm

# Liked by Clément Hardoüin on Sunday, May 24th, 2015 at 9:22pm

# Liked by vollepeer on Monday, May 25th, 2015 at 12:01am

# Liked by Charla Pate on Monday, May 25th, 2015 at 4:31am

# Liked by Giovanni Morelli on Wednesday, May 27th, 2015 at 6:57pm