Tags: sync

29

sparkline

Tuesday, November 20th, 2018

Web workers vs Service workers vs Worklets

A great primer by Ire:

Web workers, service workers, and worklets are all scripts that run on a separate thread. So what are the differences between these three types of workers?

Sunday, November 11th, 2018

Push without notifications

On the first day of Indie Web Camp Berlin, I led a session on going offline with service workers. This covered all the usual use-cases: pre-caching; custom offline pages; saving pages for offline reading.

But on the second day, Sebastiaan spent a fair bit of time investigating a more complex use of service workers with the Push API.

The Push API is what makes push notifications possible on the web. There are a lot of moving parts—browser, server, service worker—and, frankly, it’s way over my head. But I’m familiar with the general gist of how it works. Here’s a typical flow:

  1. A website prompts the user for permission to send push notifications.
  2. The user grants permission.
  3. A whole lot of complicated stuff happens behinds the scenes.
  4. Next time the website publishes something relevant, it fires a push message containing the details of the new URL.
  5. The user’s service worker receives the push message (even if the site isn’t open).
  6. The service worker creates a notification linking to the URL, interrupting the user, and generally adding to the weight of information overload.

Here’s what Sebastiaan wanted to investigate: what if that last step weren’t so intrusive? Here’s the alternate flow he wanted to test:

  1. A website prompts the user for permission to send push notifications.
  2. The user grants permission.
  3. A whole lot of complicated stuff happens behinds the scenes.
  4. Next time the website publishes something relevant, it fires a push message containing the details of the new URL.
  5. The user’s service worker receives the push message (even if the site isn’t open).
  6. The service worker fetches the contents of the URL provided in the push message and caches the page. Silently.

It worked.

I think this could be a real game-changer. I don’t know about you, but I’m very, very wary of granting websites the ability to send me push notifications. In fact, I don’t think I’ve ever given a website permission to interrupt me with push notifications.

You’ve seen the annoying permission dialogues, right?

In Firefox, it looks like this:

Will you allow name-of-website to send notifications?

[Not Now] [Allow Notifications]

In Chrome, it’s:

name-of-website wants to

Show notifications

[Block] [Allow]

But in actual fact, these dialogues are asking for permission to do two things:

  1. Receive messages pushed from the server.
  2. Display notifications based on those messages.

There’s no way to ask for permission just to do the first part. That’s a shame. While I’m very unwilling to grant permission to be interrupted by intrusive notifications, I’d be more than willing to grant permission to allow a website to silently cache timely content in the background. It would be a more calm technology.

Think of the use cases:

  • I grant push permission to a magazine. When the magazine publishes a new article, it’s cached on my device.
  • I grant push permission to a podcast. Whenever a new episode is published, it’s cached on my device.
  • I grant push permission to a blog. When there’s a new blog post, it’s cached on my device.

Then when I’m on a plane, or in the subway, or in any other situation without a network connection, I could still visit these websites and get content that’s fresh to me. It’s kind of like background sync in reverse.

There’s plenty of opportunity for abuse—the cache could get filled with content. But websites can already do that, and they don’t need to be granted any permissions to do so; just by visiting a website, it can add multiple files to a cache.

So it seems that the reason for the permissions dialogue is all about displaying notifications …not so much about receiving push messages from the server.

I wish there were a way to implement this background-caching pattern without requiring the user to grant permission to a dialogue that contains the word “notification.”

I wonder if the act of adding a site to the home screen could implicitly grant permission to allow use of the Push API without notifications?

In the meantime, the proposal for periodic synchronisation (using background sync) could achieve similar results, but in a less elegant way; periodically polling for new content instead of receiving a push message when new content is published. Also, it requires permission. But at least in this case, the permission dialogue should be more specific, and wouldn’t include the word “notification” anywhere.

Sunday, September 23rd, 2018

Service workers in Samsung Internet browser

I was getting reports of some odd behaviour with the service worker on thesession.org, the Irish music website I run. Someone emailed me to say that they kept getting the offline page, even when their internet connection was perfectly fine and the site was up and running.

They didn’t mind answering my pestering follow-on questions to isolate the problem. They told me that they were using the Samsung Internet browser on Android. After a little searching, I found this message on a Github thread about using waitUntil. It’s from someone who works on the Samsung Internet team:

Sadly, the asynchronos waitUntil() is not implemented yet in our browser. Yes, we will implement it but our release cycle is so far. So, for a long time, we might not resolve the issue.

A-ha! That explains the problem. See, here’s the pattern I was using:

  1. When someone requests a file,
  2. fetch that file from the network,
  3. create a copy of the file and cache it,
  4. return the contents.

Step 1 is the event listener:

// 1. When someone requests a file
addEventListener('fetch', fetchEvent => {
  let request = fetchEvent.request;
  fetchEvent.respondWith(

Steps 2, 3, and 4 are inside that respondWith:

// 2. fetch that file from the network
fetch(request)
.then( responseFromFetch => {
  // 3. create a copy of the file and cache it
  let copy = responseFromFetch.clone();
  caches.open(cacheName)
  .then( cache => {
    cache.put(request, copy);
  })
  // 4. return the contents.
  return responseFromFetch;
})

Step 4 might well complete while step 3 is still running (remember, everything in a service worker script is asynchronous so even though I’ve written out the steps sequentially, you never know what order the steps will finish in). That’s why I’m wrapping that third step inside fetchEvent.waitUntil:

// 2. fetch that file from the network
fetch(request)
.then( responseFromFetch => {
  // 3. create a copy of the file and cache it
  let copy = responseFromFetch.clone();
  fetchEvent.waitUntil(
    caches.open(cacheName)
    .then( cache => {
      cache.put(request, copy);
    })
  );
  // 4. return the contents.
  return responseFromFetch;
})

If a browser (like Samsung Internet) doesn’t understand the bit where I say fetchEvent.waitUntil, then it will throw an error and execute the catch clause. That’s where I have my fifth and final step: “try looking in the cache instead, but if that fails, show the offline page”:

.catch( fetchError => {
  console.log(fetchError);
  return caches.match(request)
  .then( responseFromCache => {
    return responseFromCache || caches.match('/offline');
  });
})

Normally in this kind of situation, I’d use feature detection to check whether a browser understands a particular API method. But it’s a bit tricky to test for support for asynchronous waitUntil. That’s okay. I can use a try/catch statement instead. Here’s what my revised code looks like:

fetch(request)
.then( responseFromFetch => {
  let copy = responseFromFetch.clone();
  try {
    fetchEvent.waitUntil(
      caches.open(cacheName)
      .then( cache => {
        cache.put(request, copy);
      })
    );
  } catch (error) {
    console.log(error);
  }
  return responseFromFetch;
})

Now I’ve managed to localise the error. If a browser doesn’t understand the bit where I say fetchEvent.waitUntil, it will execute the code in the catch clause, and then carry on as usual. (I realise it’s a bit confusing that there are two different kinds of catch clauses going on here: on the outside there’s a .then()/.catch() combination; inside is a try{}/catch{} combination.)

At some point, when support for async waitUntil statements is universal, this precautionary measure won’t be needed, but for now wrapping them inside try doesn’t do any harm.

There are a few places in chapter five of Going Offline—the chapter about service worker strategies—where I show examples using async waitUntil. There’s nothing wrong with the code in those examples, but if you want to play it safe (especially while Samsung Internet doesn’t support async waitUntil), feel free to wrap those examples in try/catch statements. But I’m not going to make those changes part of the errata for the book. In this case, the issue isn’t with the code itself, but with browser support.

Monday, April 30th, 2018

Going Offline: Designing An Ideal Offline Experience With Service Workers By Jeremy Keith

Here’s a great even-handed in-depth review of Going Offline:

If you’re interested in the “offline first” movement or want to learn more about Service Workers, Going Offline by Jeremy Keith is a really gentle and highly accessible introduction to the topic. At times, it even felt “too gentle”, with Keith taking a moment here and there to explain what a “variable” is and what “JSON” (JavaScript Object Notation) is. But, this just goes to show you the unassuming and welcoming mindset behind writing a book like this one.

Monday, January 8th, 2018

Async + Await

Slides from a conference talk with a really clear explanation of how async + await works with promises.

Monday, October 9th, 2017

“async” attribute on img, and corresponding “ready” event · Issue #1920 · whatwg/html

It looks like the async attribute is going to ship in Chrome for img elements:

This attribute would have two states:

  • “on”: This indicates that the developer prefers responsiveness and performance over atomic presentation of content.
  • “off”: This indicates that the developer prefers atomic presentation of content over responsiveness.

Friday, September 1st, 2017

Progressively Worse Apps

This article makes a good point about client-rendered pages:

Asynchronously loaded page elements shift click targets, resulting in a usability nightmare.

…but this has nothing, absolutely nothing to do with progressive web apps.

More fuel for the fire of evidence that far too many people think that progressive web apps and single page apps are one and the same.

Sunday, August 20th, 2017

Offline POSTs with Progressive Web Apps – Web Dev @ Microsoft – Medium

This is a smart way to queue up POST submissions for later if the user is offline. It’s not as powerful as background sync (because it requires the user to revisit your site) but it’s a good fallback for browsers that support service workers but don’t yet support background sync

Tuesday, February 21st, 2017

Send messages when you’re back online with Service Workers and Background Sync – Twilio Cloud Communications Blog

This example of using background sync looks like it’s specific to Twilio, but the breakdown of steps is broad enough to apply to many situations:

On the page we need to:

  1. Register a Service Worker
  2. Intercept the “submit” event for our message form
  3. Place the message details into IndexedDB, an in browser database
  4. Register the Service Worker to receive a “sync” event

Then, in the Service Worker we need to:

  1. Listen for sync events
  2. When a sync event is received, retrieve the messages from IndexedDB
  3. For each message, send a request to our server to send the message
  4. If the message is sent successfully, then remove the message from IndexedDB

And that’s it.

Sunday, January 29th, 2017

Callback Hell

At first when I was reading this JavaScript coding guide, I thought “Isn’t this exactly what promises address?” but that is then addressed further down:

Before looking at more advanced solutions, remember that callbacks are a fundamental part of JavaScript (since they are just functions) and you should learn how to read and write them before moving on to more advanced language features, since they all depend on an understanding of callbacks.

Fair enough. In any case, what you’ll find here is mainly good advice for writing modular code.

Saturday, January 28th, 2017

The Promise of a Burger Party - Mariko Kosaka

Mariko has a real knack for explaining technical concepts in a very accessible way. This time it’s JavaScript promises.

Wednesday, January 11th, 2017

Making Resilient Web Design work offline

I’ve written before about taking an online book offline, documenting the process behind the web version of HTML5 For Web Designers. A book is quite a static thing so it’s safe to take a fairly aggressive offline-first approach. In fact, a static unchanging book is one of the few situations that AppCache works for. Of course a service worker is better, but until AppCache is removed from browsers (and until service worker is supported across the board), I’m using both. I wouldn’t recommend that for most sites though—for most sites, use a service worker to enhance it, and avoid AppCache like the plague.

For Resilient Web Design, I took a similar approach to HTML5 For Web Designers but I knew that there was a good chance that some of the content would be getting tweaked at least for a while. So while the approach is still cache-first, I decided to keep the cache fairly fresh.

Here’s my service worker. It starts with the usual stuff: when the service worker is installed, there’s a list of static assets to cache. In this case, that list is literally everything; all the HTML, CSS, JavaScript, and images for the whole site. Again, this is a pattern that works well for a book, but wouldn’t be right for other kinds of websites.

The real heavy lifting happens with the fetch event. This is where the logic sits for what the service worker should do everytime there’s a request for a resource. I’ve documented the logic with comments:

// Look in the cache first, fall back to the network
  // CACHE
  // Did we find the file in the cache?
      // If so, fetch a fresh copy from the network in the background
      // NETWORK
          // Stash the fresh copy in the cache
  // NETWORK
  // If the file wasn't in the cache, make a network request
      // Stash a fresh copy in the cache in the background
  // OFFLINE
  // If the request is for an image, show an offline placeholder
  // If the request is for a page, show an offline message

So my order of preference is:

  1. Try the cache first,
  2. Try the network second,
  3. Fallback to a placeholder as a last resort.

Leaving aside that third part, regardless of whether the response is served straight from the cache or from the network, the cache gets a top-up. If the response is being served from the cache, there’s an additional network request made to get a fresh copy of the resource that was just served. This means that the user might be seeing a slightly stale version of a file, but they’ll get the fresher version next time round.

Again, I think this acceptable for a book where the tweaks and changes should be fairly minor, but I definitely wouldn’t want to do it on a more dynamic site where the freshness matters more.

Here’s what it usually likes like when a file is served up from the cache:

caches.match(request)
  .then( responseFromCache => {
  // Did we find the file in the cache?
  if (responseFromCache) {
      return responseFromCache;
  }

I’ve introduced an extra step where the fresher version is fetched from the network. This is where the code can look a bit confusing: the network request is happening in the background after the cached file has already been returned, but the code appears before the return statement:

caches.match(request)
  .then( responseFromCache => {
  // Did we find the file in the cache?
  if (responseFromCache) {
      // If so, fetch a fresh copy from the network in the background
      event.waitUntil(
          // NETWORK
          fetch(request)
          .then( responseFromFetch => {
              // Stash the fresh copy in the cache
              caches.open(staticCacheName)
              .then( cache => {
                  cache.put(request, responseFromFetch);
              });
          })
      );
      return responseFromCache;
  }

It’s asynchronous, see? So even though all that network code appears before the return statement, it’s pretty much guaranteed to complete after the cache response has been returned. You can verify this by putting in some console.log statements:

caches.match(request)
.then( responseFromCache => {
  if (responseFromCache) {
      event.waitUntil(
          fetch(request)
          .then( responseFromFetch => {
              console.log('Got a response from the network.');
              caches.open(staticCacheName)
              .then( cache => {
                  cache.put(request, responseFromFetch);
              });
          })
      );
      console.log('Got a response from the cache.');
      return responseFromCache;
  }

Those log statements will appear in this order:

Got a response from the cache.
Got a response from the network.

That’s the opposite order in which they appear in the code. Everything inside the event.waitUntil part is asynchronous.

Here’s the catch: this kind of asynchronous waitUntil hasn’t landed in all the browsers yet. The code I’ve written will fail.

But never fear! Jake has written a polyfill. All I need to do is include that at the start of my serviceworker.js file and I’m good to go:

// Import Jake's polyfill for async waitUntil
importScripts('/js/async-waituntil.js');

I’m also using it when a file isn’t found in the cache, and is returned from the network instead. Here’s what the usual network code looks like:

fetch(request)
  .then( responseFromFetch => {
    return responseFromFetch;
  })

I want to also store that response in the cache, but I want to do it asynchronously—I don’t care how long it takes to put the file in the cache as long as the user gets the response straight away.

Technically, I’m not putting the response in the cache; I’m putting a copy of the response in the cache (it’s a stream, so I need to clone it if I want to do more than one thing with it).

fetch(request)
  .then( responseFromFetch => {
    // Stash a fresh copy in the cache in the background
    let responseCopy = responseFromFetch.clone();
    event.waitUntil(
      caches.open(staticCacheName)
      .then( cache => {
          cache.put(request, responseCopy);
      })
    );
    return responseFromFetch;
  })

That all seems to be working well in browsers that support service workers. For legacy browsers, like Mobile Safari, there’s the much blunter caveman logic of an AppCache manifest.

Here’s the JavaScript that decides whether a browser gets the service worker or the AppCache:

if ('serviceWorker' in navigator) {
  // If service workers are supported
  navigator.serviceWorker.register('/serviceworker.js');
} else if ('applicationCache' in window) {
  // Otherwise inject an iframe to use appcache
  var iframe = document.createElement('iframe');
  iframe.setAttribute('src', '/appcache.html');
  iframe.setAttribute('style', 'width: 0; height: 0; border: 0');
  document.querySelector('footer').appendChild(iframe);
}

Either way, people are making full use of the offline nature of the book and that makes me very happy indeed.

Monday, November 21st, 2016

My biggest takeaway from the second Offline Camp in Santa Margarita, CA — plus toast!

J. Renée Beach writes on Ev’s blog about three things to consider when planning for offline experiences:

  • Freshness,
  • Reach, and
  • Assurance.

How will you express to your users that the content is up to date, safe and available across their network?

Tuesday, November 1st, 2016

Web fonts, boy, I don’t know – Monica Dinculescu

Monica takes a look at the options out there for loading web fonts and settles on a smart asynchronous lazy-loading approach.

Wednesday, October 12th, 2016

Enhancing a comment form: From basic to custom error message to BackgroundSync | justmarkup

This is a truly fantastic example of progressive enhancement applied to a form.

What I love about this is that it shows how progressive enhancement isn’t a binary on/off choice: there are layers and layers of enhancements here, from simple inline validation all the way to service workers and background sync, with many options in between.

Superb!

Friday, August 5th, 2016

ServiceWorker: A Basic Guide to BackgroundSync

A nice introduction to using Service Workers to enable syncing in the background: when the user is offline, tasks get queued up and then when the user is back online, those tasks execute.

Thursday, September 10th, 2015

Brighton in September

I know I say this every year, but this month—and this week in particular—is a truly wonderful time to be in Brighton. I am, of course, talking about The Brighton Digital Festival.

It’s already underway. Reasons To Be Creative just wrapped up. I managed to make it over to a few talks—Stacey Mulcahey, Jon, Evan Roth. The activities for the Codebar Code and Chips scavenger hunt are also underway. Tuesday evening’s event was a lot of fun; at the end of the night, everyone wanted to keep on coding.

I popped along to the opening of Georgina’s Familiars exhibition. It’s really good. There’s an accompanying event on Saturday evening called Unfamiliar Matter which looks like it’ll be great. That’s the same night as the Miniclick party though.

I guess clashing events are unavoidable. Like tonight. As well as the Guardians Of The Galaxy screening hosted by Chris (that I’ll be going to), there’s an Async special dedicated to building a 3D Lunar Lander.

But of course the big event is dConstruct tomorrow. I’m really excited about it. Partly that’s because I’m not the one organising it—it’s all down to Andy and Kate—but also because the theme and the line-up is right up my alley.

Andy has asked me to compere the event. I feel a little weird about that seeing as it’s his baby, but I’m also honoured. And, you know, after talking to most of the speakers for the podcast—which I enjoyed immensely—I feel like I can give an informed introduction for each talk.

I’m looking forward to this near future event.

See you there.

Tuesday, May 12th, 2015

Killer page load performance – Async

This Async event at 68 Middle Street on June 11th looks like it’s going to good (and relevant to my interests).

Thursday, July 31st, 2014

How we make RWD sites load fast as heck

Scott shares the code that Filament Group are using to determine which style declarations are critical (and can be inlined) and which are non-critical (and can be loaded asynchronously). It makes quite a difference in perceived performance.

By the way, I really, really like the terminology of “critical” and “non-critical” CSS, rather than “above the fold” and “below the fold” CSS.

Friday, April 25th, 2014

Where Time Comes From on Vimeo

A profile of Demetrios Matsakis, keeper of time at U.S. Naval Observatory, America’s equivalent to Greenwich in its importance for timekeeping in the modern world.

Where Time Comes From