Jeremy Keith

Jeremy Keith

Making websites. Writing books. Speaking at events. Living in Brighton. Working at Clearleft. Playing music. Taking photos. Answering email.

Journal 2581 sparkline Links 8166 sparkline Articles 72 sparkline Notes 4368 sparkline

Monday, June 24th, 2019

Am I cached or not?

When I was writing about the lie-fi strategy I’ve added to, I finished with this thought:

What I’d really like is some way to know—on the client side—whether or not the currently-loaded page came from a cache or from a network. Then I could add some kind of interface element that says, “Hey, this page might be stale—click here if you want to check for a fresher version.”

Trys heard my plea, and came up with a very clever technique to alter the HTML of a page when it’s put into a cache.

It’s a function that reads the response body stream in, returning a new stream. Whilst reading the stream, it searches for the character codes that make up: <html. If it finds them, it tacks on a data-cached attribute.


But then I was discussing this issue with Tantek and Aaron late one night after Indie Web Camp Düsseldorf. I realised that I might have another potential solution that doesn’t involve the service worker at all.

Caveat: this will only work for pages that have some kind of server-side generation. This won’t work for static sites.

In my case, pages are generated by PHP. I’m not doing a database lookup every time you request a page—I’ve got a server-side cache of posts, for example—but there is a little bit of assembly done for every request: get the header from here; get the main content from over there; get the footer; put them all together into a single page and serve that up.

This means I can add a timestamp to the page (using PHP). I can mark the moment that it was served up. Then I can use JavaScript on the client side to compare that timestamp to the current time.

I’ve published the code as a gist.

In a script element on each page, I have this bit of coducken:

var serverTimestamp = <?php echo time(); ?>;

Now the JavaScript variable serverTimestamp holds the timestamp that the page was generated. When the page is put in the cache, this won’t change. This number should be the number of seconds since January 1st, 1970 in the UTC timezone (that’s what my server’s timezone is set to).

Starting with JavaScript’s Date object, I use a caravan of methods like toUTCString() and getTime() to end up with a variable called clientTimestamp. This will give the current number of seconds since January 1st, 1970, regardless of whether the page is coming from the server or from the cache.

var localDate = new Date();
var localUTCString = localDate.toUTCString();
var UTCDate = new Date(localUTCString);
var clientTimestamp = UTCDate.getTime() / 1000;

Then I compare the two and see if there’s a discrepency greater than five minutes:

if (clientTimestamp - serverTimestamp > (60 * 5))

If there is, then I inject some markup into the page, telling the reader that this page might be stale:

  <p class="feedback">
    <button onclick="this.parentNode.remove()">dismiss</button>
    This page might be out of date. You can try <a href="javascript:window.location=window.location.href">refreshing</a>.

The reader has the option to refresh the page or dismiss the message.

This page might be out of date. You can try refreshing.

It’s not foolproof by any means. If the visitor’s computer has their clock set weirdly, then the comparison might return a false positive every time. Still, I thought that using UTC might be a safer bet.

All in all, I think this is a pretty good method for detecting if a page is being served from a cache. Remember, the goal here is not to determine if the user is offline—for that, there’s navigator.onLine.

The upshot is this: if you visit my site with a crappy internet connection (lie-fi), then after three seconds you may be served with a cached version of the page you’re requesting (if you visited that page previously). If that happens, you’ll now also be presented with a little message telling you that the page isn’t fresh. Then it’s up to you whether you want to have another go.

I like the way that this puts control back into the hands of the user.

Sunday, June 23rd, 2019

Julio Biason .Net 4.0 - Things I Learnt The Hard Way (in 30 Years of Software Development)

Lots and lots of programming advice. I can’t attest to the veracity and efficacy of all of it, but this really rang true:

If you have no idea how to start, describe the flow of the application in high level, pure English/your language first. Then fill the spaces between comments with the code.

And this:

Blogging about your stupid solution is still better than being quiet.

You may feel “I’m not start enough to talk about this” or “This must be so stupid I shouldn’t talk about it”.

Create a blog. Post about your stupid solutions.

Saturday, June 22nd, 2019

BBC Shows and Tours - Shows - James Burke: Our Man on the Moon

I wish I were here for this (I’m going to be over in Ireland that week)—an evening with James Burke, Britain’s voice of Apollo 11.

Here is your chance to find out what went on behind the scenes as James revisits the final moments of the Apollo mission. He’ll recreate the drama, struggling to make sense of flickering images from NASA and working with the limitations of 1960s technology. We’ll hear what went wrong as well as what went right on the night! Illustrated with amazing archive material from both the BBC and NASA, this will be the story of the moon landings brought to you by the man who became a broadcasting legend.

Thursday, June 20th, 2019

Wednesday, June 19th, 2019


Shockwaves rippled across the web standards community recently when it appeared that Google Chrome was unilaterally implementing a new element called toast. It turns out that’s not the case, but the confusion is understandable.

First off, this all kicked off with the announcement of “intent to implement”. That makes it sounds like Google are intending to, well, …implement this. In fact “intent to implement” really means “intend to mess around with this behind a flag”. The language is definitely confusing and this is something that will hopefully be addressed.

Secondly, Chrome isn’t going to ship a toast element. Instead, this is a proposal for a custom element currently called std-toast. I’m assuming that should the experiment prove successful, it’s not a foregone conclusion that the final element name will be called toast (minus the sexually-transmitted-disease prefix). If this turns out to be a useful feature, there will surely be a discussion between implementators about the naming of the finished element.

This is the ideal candidate for a web component. It makes total sense to create a custom element along the lines of std-toast. At first I was confused about why this was happening inside of a browser instead of first being created as a standalone web component, but it turns out that there’s been a fair bit of research looking at existing implementations in libraries and web components. So this actually looks like a good example of paving an existing cowpath.

But it didn’t come across that way. The timing of announcements felt like this was something that was happening without prior discussion. Terence Eden writes:

It feels like a Google-designed, Google-approved, Google-benefiting idea which has been dumped onto the Web without any consideration for others.

I know that isn’t the case. And I know how many dedicated people have worked hard on this proposal.

Adrian Roselli also remarks on the optics of this situation:

To be clear, while I think there is value in minting a native HTML element to fill a defined gap, I am wary of the approach Google has taken. A repo from a new-to-the-industry Googler getting a lot of promotion from Googlers, with Googlers on social media doing damage control for the blowback, WHATWG Googlers handling questions on the repo, and Google AMP strongly supporting it (to reduce its own footprint), all add up to raise alarm bells with those who advocated for a community-driven, needs-based, accessible web.

Dave Cramer made a similar point:

But my concern wasn’t so much about the nature of the new elements, but of how we learned about them and what that says about how web standardization works.

So there’s a general feeling (outside of Google) that there’s something screwy here about the order of events. A lot discussion and research seems to have happened in isolation before announcing the intent to implement:

It does not appear that any discussions happened with other browser vendors or standards bodies before the intent to implement.

Why is this a problem? Google is seeking feedback on a solution, not on how to solve the problem.

Going back to my early confusion about putting a web component directly into a browser, this question on Discourse echoes my initial reaction:

Why not release std-toast (and other elements in development) as libraries first?

It turns out that std-toast and other in-browser web components are part of an idea called layered APIs. In theory this is an initiative in the spirit of the extensible web manifesto.

The extensible web movement focused on exposing low-level APIs to developers: the fetch API, the cache API, custom elements, Houdini, and all of those other building blocks. Layered APIs, on the other hand, focuses on high-level features …like, say, an HTML element for displaying “toast” notifications.

Layered APIs is an interesting idea, but I’m worried that it could be used to circumvent discussion between implementers. It’s a route to unilaterally creating new browser features first and standardising after the fact. I know that’s how many features already end up in browsers, but I think that the sooner that authors, implementers, and standards bodies get a say, the better.

I certainly don’t think this is a good look for Google given the debacle of AMP’s “my way or the highway” rollout. I know that’s a completely different team, but the external perception of Google amongst developers has been damaged by the AMP project’s anti-competitive abuse of Google’s power in search.

Right now, a lot of people are jumpy about Microsoft’s move to Chromium for Edge. My friends at Microsoft have been reassuring me that while it’s always a shame to reduce browser engine diversity, this could actually be a good thing for the standards process: Microsoft could theoretically keep Google in check when it comes to what features are introduced to the Chromium engine.

But that only works if there is some kind of standards process. Layered APIs in general—and std-toast in particular—hint at a future where a single browser vendor can plough ahead on their own. I sincerely hope that’s a misreading of the situation and that this has all been an exercise in miscommunication and misunderstanding.

Like Dave Cramer says:

I hear a lot about how anyone can contribute to the web platform. We’ve all heard the preaching about incubation, the Extensible Web, working in public, paving the cowpaths, and so on. But to an outside observer this feels like Google making all the decisions, in private, and then asking for public comment after the feature has been designed.

Tuesday, June 18th, 2019

A song of AIs and fire

The televisual adaption of Game of Thrones wrapped up a few weeks ago, so I hope I can safely share some thoughts with spoilering. That said, if you haven’t seen the final season, and you plan to, please read no further!

There has been much wailing and gnashing of teeth about the style of the final series or two. To many people, it felt weirdly …off. Zeynep’s superb article absolutely nails why the storytelling diverged from its previous style:

For Benioff and Weiss, trying to continue what Game of Thrones had set out to do, tell a compelling sociological story, would be like trying to eat melting ice cream with a fork. Hollywood mostly knows how to tell psychological, individualized stories. They do not have the right tools for sociological stories, nor do they even seem to understand the job.

Let’s leave aside the clumsiness of the execution for now and focus on the outcomes.

The story finishes with Bran as the “winner”, in that he now rules the seve— six kingdoms. I have to admit, I quite like the optics of replacing an iron throne with a wheelchair. Swords into ploughshares, and all that.

By this point, Bran is effectively a non-human character. He’s the Dr. Manhattan of the story. As the three-eyed raven, he has taken on the role of being an emotionless database of historical events. He is Big Data personified. Or, if you squint just right, he’s an Artificial Intelligence.

There’s another AI in the world of Game of Thrones. The commonly accepted reading of the Night King is that he represents climate change: an unstoppable force that’s going to dramatically impact human affairs, but everyone is too busy squabbling in their own politics to pay attention to it. I buy that. But there’s another interpretation. The Night King is rogue AI. He’s a paperclip maximiser.

Clearly, a world ruled by an Artificial Intelligence like that would be a nightmare scenario. But we’re also shown that a world ruled purely by human emotion would be just as bad. That would be the tyrannical reign of the mad queen Daenerys. Both extremes are undesirable.

So why is Bran any better? Well, technically, he isn’t ruling alone. He has a board of (very human) advisors. The emotionless logic of a pure AI is kept in check by a council of people. And the extremes of human nature are kept in check by the impartial AI. To put in another way, humanity is augmented by Artificial Intelligence: Man-computer symbiosis.

Whether it’s the game of chess or the game of thrones, a centaur is your best bet.

Automate your outgoing webmentions

I’ve been kicking the tyres on this great new tool from Remy. Give it a URL and it’ll find all the links in its h-entrys and automatically send webmentions to them. Very cool!

The documentation on the site is excellent, guiding you to the right solution for your particular needs. Read Remy’s announcement:

I’ve also tried very hard to get the documentation to be as welcoming as I can. I’ve tried to think about my dear visitor and what they want to do with the software, rather than type my typical developer approach to documentation - listing all the features and options.

The New Wilderness (Idle Words)

An excellent piece by Maciej on the crucial difference between individual privacy and ambient privacy (and what that means for regulation):

Ambient privacy is not a property of people, or of their data, but of the world around us. Just like you can’t drop out of the oil economy by refusing to drive a car, you can’t opt out of the surveillance economy by forswearing technology (and for many people, that choice is not an option). While there may be worthy reasons to take your life off the grid, the infrastructure will go up around you whether you use it or not.

Because our laws frame privacy as an individual right, we don’t have a mechanism for deciding whether we want to live in a surveillance society. Congress has remained silent on the matter, with both parties content to watch Silicon Valley make up its own rules. The large tech companies point to our willing use of their services as proof that people don’t really care about their privacy. But this is like arguing that inmates are happy to be in jail because they use the prison library. Confronted with the reality of a monitored world, people make the rational decision to make the best of it.

That is not consent.

For more detail, I highly recommend reading his testimony to the senate hearing on Privacy Rights and Data Collection in a Digital Economy.