Archive: May, 2012

Phil Gyford was down in Brighton visiting the Clearleft HQ today. We’re working with him on Matter, which I’m very excited about.

Today wasn’t just any ol’ day for Phil. Today marks the end of a project of his that has been running for nine years and five months: Pepys’ Diary:

This site is a presentation of the diaries of Samuel Pepys, the renowned 17th century diarist who lived in London, England. A new entry written by Pepys will be published each day over the course of several years; 1 January 1660 was published on 1 January 2003.

We invited Phil down to Brighton last year to talk about Pepys’ Diary at a Skillswap event. You can listen to the audio on Huffduffer.

I’m a big fan of long-term thinking and—in web terms—this project is as old as Methuselah. It’s refreshing. In an industry so caught up in the churn and grind of the new and the shiny, I think it’s wonderful that Phil dedicated himself to a project that he knew would require a long-term investment of his time. Russell wrote about it in Wired recently:

In some worlds ten years isn’t very long: it’s not if you’re digging an undersea tunnel or discovering a cure for disease. But in the busy, silly world of early 21st-century media, making a ten-year assertion was a big deal — something akin to the Clock of the Long Now.

I’ll be sorry to see you go, Mister time-shifted Pepys. But I understand that it’s hard for you to keep writing a diary when your eyesight is failing.

The ill condition of my eyes, and my neglect for a year or two, hath kept me behindhand in my accounts, so as to render it difficult now.

— Samuel Pepys (@samuelpepys) May 31, 2012

It’s been an exciting day at Clearleft Towers. Tickets for this year’s dConstruct went on sale today.

Sales are going briskly. At the end of the day, two thirds of the tickets were gone.

Needless to say, I think you should grab yourself a ticket. But then, I would say that, wouldn’t I? I think the line-up is bloody brilliant: James Burke! Lauren Beukes! Jason Scott! Unnecessary exclamation points!

But before you slap your virtual money down via the Herculean challenge of Google Checkout, let me reiterate what I wrote on the dConstruct website: dConstruct is not a conference of practical web design and development tutorials.

The presentations may not contain any practical tips you can take back to work on Monday morning but you will gain insights into the direction our digital technology is taking.

If you’re looking for practical know-how, I highly recommend the workshops. The actual conference day, however, is all about challenging your brainbits and making you think.

In this interview with .net magazine Andy does a good job of explaining where dConstruct is coming from, man, and what you can expect on the day:

While other conferences try to educate and inform, dConstruct aims to enlighten and inspire – to challenge the way you think about design and technology as it applies to our industry. It’s basically brain candy for a curious mind.

If that sounds like something you’d enjoy, get a ticket. I predict they’ll probably be gone by the end of the week.

There’s been quite a brouhaha over the past couple of days around the subject of standardising responsive images. There are two different matters here: the process and the technical details. I’d like to address both of them.

Ill communication

First of all, there’s a number of very smart developers who feel that they’ve been sidelined by the WHATWG. Tim has put together a timeline of what happened:

1. Developers got involved in trying to standardize a solution to a common and important problem.
2. The WHATWG told them to move the discussion to a community group.
3. The discussion was moved (back in February), a general consenus (not unanimous, but a majority) was reached about the picture element.
4. Another (partial) solution was proposed directly on the WHATWG list by an Apple employee.
5. A discussion ensued regarding the two methods, where they overlapped, and how the general opinions of each. The majority of developers favored the picture element and the majority of implementors favored the srcset attribute.
6. While the discussion was still taking place, and only 5 days after it was originally proposed, the srcset attribute (but not the picture element) was added to the draft.

A few points in that timeline have since been clarified. That second step—“The WHATWG told them to move the discussion to a community group”—turns out to be untrue. Some random person on the WHATWG mailing list (which is open to everyone) suggested forming a Community Group at the W3C. Alas, nobody else on the WHATWG mailing list corrected that suggestion.

Then there’s apparent causality between step 4 and 6. Initially, I also assumed that this was what happened: that Tess had proposed the srcset solution without even being aware of the picture solution that the Community Group had independently come up with it. It turns out that’s not the case. Tess had another email about the picture proposal but she never ended up sending it. In fact, her email about srcset had been sitting in draft for quite a while and she only sent it out when she saw that Hixie was finally collating feedback on responsive images.

So from the outside it looked like there was preferential treatment being given to Tess’s proposal because it came from within the WHATWG. That’s not the case, but it must be said: the fact that srcset was so quickly added to the spec (albeit in a different form) doesn’t look good. It’s easy to understand why the smart folks in the Responsive Images Community Group felt miffed.

But let’s be clear: this is exactly how the WHATWG is supposed to work. Use-cases are evaluated and whatever Hixie thinks is best solution gets put in the spec, regardless of how popular or unpopular it is.

Now, if that sounds abhorrent to you, I completely understand. A dictatorship should cause us to recoil.

That’s where the W3C come in. Their model is completely different. Everything is done by committee there.

Steve Faulkner chimed in on Tim’s post with his take on the two groups:

It seems like the development of HTML has turned full circle, the WHATWG was formed to overthrow the hegemony of the W3C, now the W3C acts as a counter to the hegemony of the WHATWG.

I think he’s right. The W3C keeps the rapid, sometimes anarchic approach of the WHATWG in check. But the opposite is also true. Without the impetus provided by the WHATWG, I’m not sure that the W3C HTML Working Group would ever get anything done. There’s a balance that actually works quite well in practice.

Back to the situation with responsive images…

Unfortunately, it appears to people within the Responsive Images Community Group that all their effort was wasted because their proposed solution was summarily rejected. In actuality all the use-cases they gathered were immensely valuable. But it’s certainly true that the WHATWG didn’t make it clearer how and where developers could best contribute.

Community Groups are a W3C creation. They don’t have anything to do with the WHATWG, who do all their work on their own mailing list, their own wiki and their own IRC channel.

I do think that the W3C Community Groups offer a good place to go bike-shedding on problems. That’s a term that’s usually used derisively but sometimes it’s good to have a good ol’ bike-shedding without clogging up the mailing list for everyone. But it needs to be clear that there’s a big difference between a Community Group and a Working Group.

I wish the WHATWG had done a better job of communicating to newcomers how best to contribute. It would have avoided a lot of the frustrations articulated by Wilto:

Unfortunately, we were laboring under the impression that Community Groups shared a deeper inherent connection with the standards bodies than it actually does.

But in any case, as Doctor Bruce writes at least now there’s a proposed solution for responsive images in HTML: The Living Standard:

I don’t really care which syntax makes the spec, as long as it addresses the majority of use cases and it is usable by authors. I’m just glad we’re discussing the adaptive image problem at all.

So let’s take a look at the technical details.

src code

The Responsive Images Community Group came up with a proposal based off the idea of minting a new element, called say picture, that mimics the behaviour of video

<picture alt="image description">
  <source src="/path/to/image.png" media="(min-width: 600px)">
  <source src="/path/to/otherimage.png" media="(min-width: 800px)">
  <img src="/path/to/image.png" alt="image description">
</picture>

One of the reasons why a new element was chosen rather than extending the existing img element was due to a misunderstanding. The WHATWG had explained that the parsing of img couldn’t be easily altered. That means that img must remain a self-closing element—any solution that requires a closing /img tag wouldn’t work. Alas, that was taken to mean that extending the img element in any way was off the cards.

The picture proposal has a number of things going for it. Its syntax is easily understandable for authors: if you know media queries, then you know how to use picture. It also has a good fallback for older browsers : a regular img element. This fallback mechanism (and the idea of multiple source elements with media queries) is exactly how the video element is specced.

Unfortunately using media queries on the sources of videos has proven to be very tricky for implementors, so they don’t want to see that pattern repeated.

Another issue with multiple source elements is that parsers must wait until the closing /picture tag before they can even begin to evaluate which image to show. That’s not good for performance.

So the alternate solution, based on Ted’s proposal, extends the img element using a new srcset attribute that takes a comma-separated list of values:

Not nearly as pretty, I think you’ll agree. But it is actually nice and compact for the “retina display” use-case:

Just to be clear, that does not mean that otherimage.png is twice the size of image.png (though it could be). What you’re actually declaring is “Use image.png unless the device supports double-pixel density, in which case, use otherimage.png.”

Likewise, when I declare:

srcset="/path/to/image.png 600w 400h"

…it does not mean that image.png is 600 pixels wide by 400 pixels tall. Instead, it means that an action should be taken if the viewport matches those dimensions.

It took me a while to wrap my head around that distinction: I’m used to attributes describing the element they’re attached to, not the viewport.

Now for the really tricky bit: what do those numbers—600w and 400h—mean? Currently the spec is giving conflicting information.

Each image that’s listed in the srcset comma-separated list can have up to three values associated with it: w, h, and x. The x is pretty clear: that’s the pixel density of the device. The w and h values refer to the width and height of the viewport …but it’s not clear if they mean min-width/height or max-width/height.

If I’m taking a “Mobile First” approach to development, then srcset will meet my needs if w and h refer to min-width and min-height.

In this example, I’ll just use w to keep things simple:

(Expected behaviour: use small.png unless the viewport is wider than 600 pixels, in which case use medium.png unless the viewport is wider than 800 pixels, in which case use large.png).

If, on the other hand, w and h refer to max-width and max-height, I have to take a “Desktop First” approach:

(Expected behaviour: use large.png unless the viewport is narrower than 800 pixels, in which case use medium.png unless the viewport is narrower than 600 pixels, in which case use small.png).

One of the advantages of media queries is that, because they support both min- and max- width, they can be used in either use-case: “Mobile First” or “Desktop First”.

Because the srcset syntax will support either min- or max- width (but not both), it will therefore favour one case at the expense of the either.

Both use-cases are valid. Personally, I happen to use the “Mobile First” approach, but that doesn’t mean that other developers shouldn’t be able to take a “Desktop First” approach if they want. By the same logic, I don’t much like the idea of srcset forcing me to take a “Desktop First” approach.

My only alternative, if I want to take a “Mobile First” approach, is to duplicate image paths and declare ludicrous breakpoints:

I hope that this part of the spec offers a way out:

for the purposes of this requirement, omitted width descriptors and height descriptors are considered to have the value “Infinity”

I think that means I should be able to write this:

It’s all quite confusing and srcset doesn’t have anything approaching the extensibility of media queries, but I hope we can get it to work somehow.

Last week I described how I go about getting hold of mobile devices to test with at Clearleft. I finished by throwing open the door to any other web devs in Brighton who want to test on our devices:

We’ve always had an open-door policy here, so if you want to pop around, use our WiFi, and test on our devices, you’re more than welcome.

There was a great response from my fellow Brightonians offering to add some of their devices to the collection:

@adactio I have a Nokia N9 and a 3GS that I can share, if it’d be ok to leave them there?

— Aegir Hallmundur (@aegirthor) April 30, 2012

@adactio @clearleft I’ve got a few phones that you don’t that I’m happy to leave in your capable hands. WP7, B2G, etc

— Remy Sharp (@rem) April 30, 2012

Aegir popped ‘round yesterday and dropped off an iPhone 3GS and a very cute Nokia N9.

This morning I arrived in to the office to find that Aron had dropped off a Samsung Galaxy Tab.

This is great! Just by opening up the testing suite to everyone, it has now expanded beyond what I had been able to gather together by myself.

Josh has put together a page on the Clearleft site that we’ll keep updated with a list of the devices we’ve got.

We’ve ordered some nice stands for the phones so that we can clear away some of the cable clutter. We don’t want the experience of testing to be any more unpleasant than it needs to be. To that end, I’ve installed Adobe Shadow on the iOS and Android devices.

If you want to test a site that isn’t live yet, I find that services like showoff.io and localtunnel are handy for previewing local sites. Do bear in mind that these devices are intended for testing websites: if you want to install apps or update software, sorry; that’s not what the test lab is for.

If you’re based in Brighton, pop ‘round whenever you want to use our devices.

If you’re not based in Brighton, maybe you could kick-start a similar sharing scheme with a local company or co-working space. Belfast, Bristol, Manchester, Birmingham …you’ve got loads of smart geeks and I bet they’ve got plenty of devices they could be sharing.

There are potential pitfalls to opening up a testing suite like this. What about the insurance? What about theft? What about breakage? But the thing about potential pitfalls is that they’re just that: potential. I’m treating all of them as YAGNI issues. I’ll address any problems if and when they occur rather than planning for worst-case scenarios.

“What’s the worst that could happen?” is really only valuable as a rhetorical question.

I’m going to Amsterdam next week for the Mobilism conference. Bizarrely, there are still tickets available. I say “bizarrely” because it’s such an excellent event—it’s like the European equivalent of the Breaking Development conference.

Don’t worry; I won’t be giving a presentation. I’ll leave that to experts like Remy, Lyza, Brad, and Jake. But I will be getting up on the stage. I’m going to moderating not one, but two, panels. I think it’s going to be fun.

We’ll be reprising the Mobile Browser panel from last year. Once again, there will be representatives from Opera, RIM, and Nokia. This year Google is also joining the line-up. As usual, Apple will not be present.

The new addition to the schedule is a panel on device and network APIs. I will be playing the part of a curious but clueless web developer interested in such things …because, well, that’s what I am.

I plan to open up both panels to questions from the audience. While I don’t quite fall into Cennydd’s camp, it would be great if more people would read this article on how to ask a question:

You have not been invited to give a speech. Before you stand up, boil your thoughts down to a single point. Then ask yourself if this point is something you want to assert or something you want to find out. There are exceptions, but if your point falls into the category of assertion, you should probably remain seated.

But I’m not planning to leave the questions entirely to the people in the room. Just as I did last year, I’d like to ask you to tell me what topics are burning in your mind when it comes to mobile browsers or device APIs.

Comments are open for one week. Let fly with your questions.

When I was helping Bevan with making the dConstruct site, I kept banging on to him about the importance of performance.

Don’t get me wrong: I wanted the site to look great, but I also very much wanted it to feel great …and nothing affects the feel of a site (the user’s experience, if you will) more than performance. As Jason wrote:

If you could only do one thing to prepare your desktop site for mobile and had to choose between employing media queries to make it look good on a mobile device or optimizing the site for performance, you would be better served by making the desktop site blazingly fast.

And yet this fundamental aspect of how performant a site is going to be is all too often left until the development phase. I’d really like to see it taken into account much earlier on, during the UX and visual design phases.

Anyway, as the dConstruct site came together, I just kept asking “What would Steve Souders do?”

For a start, that meant ripping out any boilerplate markup and CSS that was there “just in case.” I very much agree with Rachel when she says stop solving problems you don’t yet have. But one of the areas where the unfortunately-named HTML5 Boilerplate excels is in its suggestions for .htaccess rules so I made sure to rip off the best bits.

Initially jQuery was being included, but given how far browsers have come in their JavaScript support, I was able to ditch it and streamline the JavaScript a bit.

Wherever possible, I made sure that background images in CSS were Base64 encoded as data URIs; icons, textures, and the like. That helped to reduce the number of HTTP requests—one of the easy wins for improving performance.

I’ve already mentioned the conditional loading that’s going on.

Then there’s the thorny issue of responsive images. The dConstruct 2012 site is similar to the dConstruct archive in that there is no correlation between browser width and image: quite often, a smaller image is required for wider screens than for narrower viewports because of the presence of a grid. So instead of trying to come up with some complex interplay of client and server cross-communication to figure out which size image would be appropriate to serve up, I instead took the same approach as I did for the archive site: optimise the hell out of images, regardless of whether they’re going to be viewed in a desktop or a mobile device.

Take a look at the original image of Kevin Slavin compared to the version that appears on the dConstruct archive.

See how everything except the face is so much blurrier in the final version? That isn’t just an attempt to introduce some cool bokeh. It makes for much smaller JPGs with fewer jaggy artefacts. And because human beings tend to focus on other human faces, the technique isn’t really consciously noticeable (although you’ll notice it now that I’ve pointed it out to you).

The design of the 2012 dConstruct site called for monochrome images with colour filters applied.

That turned out to be a fortunate boon for optimising the images. This time we were using PNGs rather than JPGs and we were able to get the number of colours down to 32 or even 16. Run them through Image Optim or Smushit and you can squeeze even more bytes out of them.

The funny thing is that sweating the file sizes of images used to be part and parcel of web development. Back in the nineties, there was something of an aesthetic that grew out of the need to optimise images with limited (web-safe!) colour palettes. That was because bandwidth was at a premium and you could be pretty sure that plenty of people were accessing your site on slow connections.

Well, here we are fifteen years later and thanks to the rise of mobile, bandwidth is once again at a premium and we can be pretty sure that plenty of people are accessing our sites on slow connections. Yet again, mobile is highlighting issues that were always there. When did we get so lazy and decide it was acceptable to send giant unoptimised images down the pipe to our long-suffering visitors?

Mathew Pennell recently wrote:

…it’s certainly true that the golden rule I grew up with – no page should ever be over 100Kb – has long since been mothballed.

But why? That seems like a perfectly good and still-relevant rule to me.

Alas, on the dConstruct site I wasn’t able to hit that target. With an unprimed cache, the home page comes in at around 300K (it’s 17K with a primed cache). By far the largest file is the CSS, weighing at 113K, followed by the web font, Futura bold oblique, at 32K.

By the way, when it comes to analysing performance in the browser, this missing manual for the Webkit inspector is really, really handy. I also ran the site through Google Page Speed but it seems that the user-agent chooses an arbitrary browser width (960? 1024?) so some of the advice about scaling images needs to be taken with a pinch of salt when applied to responsive designs.

I took a look at some other conference sites too. The beautiful site for the Build conference comes in at just under a megabyte for the homepage—it has quite a few fonts and images. It also has a monochrome aesthetic going on so I suspect quite a few of those images could be squeezed down (and some far-future expiry dates would help for repeat visitors).

Then there’s site for this year’s Mobilism conference which is blazingly fast. The combined file size on the homepage isn’t that different to the dConstruct site (although the CSS is significantly smaller) and I suspect there’s some server-side wizardry going on. I’ll have to corner Stephen at the conference next week and quiz him about it.

For now, server-side performance optimisation is something beyond my ken. I should really do something about that, especially as I’m expecting the dConstruct site to take a hammering the day that tickets go sale (May 29th—save the date).

In the meantime, there’s still plenty I can do on the front end. As Bruce put it:

It seems to me that old-fashioned, oh-so-dull techniques might not be ready for retirement yet. You know: well-crafted HTML, keeping JavaScript for progressive enhancement rather than a pre-requisite for the page even displaying, and testing across browsers.

All those optimisation techniques we learned in the 90s—and even wacky ideas like lowsrc—are back in fashion. Everything old is new again.