Tags: offline

173

sparkline

Thursday, December 6th, 2018

Introducing Background Fetch  |  Web  |  Google Developers

I’m going to have to read through this article by Jake a few times before I begin to wrap my head around this background fetch thing, but it looks like it would be perfect for something like the dConstruct Audio Archive, where fairly large files can be saved for offline listening.

Mistletoe Offline

This article first appeared in 24 Ways, the online advent calendar for geeks.

It’s that time of year, when we gather together as families to celebrate the life of the greatest person in history. This man walked the Earth long before us, but he left behind words of wisdom. Those words can guide us every single day, but they are at the forefront of our minds during this special season.

I am, of course, talking about Murphy, and the golden rule he gave unto us:

Anything that can go wrong will go wrong.

So true! I mean, that’s why we make sure we’ve got nice 404 pages. It’s not that we want people to ever get served a File Not Found message, but we acknowledge that, despite our best efforts, it’s bound to happen sometime. Murphy’s Law, innit?

But there are some Murphyesque situations where even your lovingly crafted 404 page won’t help. What if your web server is down? What if someone is trying to reach your site but they lose their internet connection? These are all things than can—and will—go wrong.

I guess there’s nothing we can do about those particular situations, right?

Wrong!

A service worker is a Murphy-battling technology that you can inject into a visitor’s device from your website. Once it’s installed, it can intercept any requests made to your domain. If anything goes wrong with a request—as is inevitable—you can provide instructions for the browser. That’s your opportunity to turn those server outage frowns upside down. Take those network connection lemons and make network connection lemonade.

If you’ve got a custom 404 page, why not make a custom offline page too?

Get your server in order

Step one is to make …actually, wait. There’s a step before that. Step zero. Get your site running on HTTPS, if it isn’t already. You won’t be able to use a service worker unless everything’s being served over HTTPS, which makes sense when you consider the awesome power that a service worker wields.

If you’re developing locally, service workers will work fine for localhost, even without HTTPS. But for a live site, HTTPS is a must.

Make an offline page

Alright, assuming your site is being served over HTTPS, then step one is to create an offline page. Make it as serious or as quirky as is appropriate for your particular brand. If the website is for a restaurant, maybe you could put the telephone number and address of the restaurant on the custom offline page (unsolicited advice: you could also put this on the home page, you know). Here’s an example of the custom offline page for this year’s Ampersand conference.

When you’re done, publish the offline page at suitably imaginative URL, like, say /offline.html.

Pre-cache your offline page

Now create a JavaScript file called serviceworker.js. This is the script that the browser will look to when certain events are triggered. The first event to handle is what to do when the service worker is installed on the user’s device. When that happens, an event called install is fired. You can listen out for this event using addEventListener:

addEventListener('install', installEvent => {
// put your instructions here.
}); // end addEventListener

In this case, you want to make sure that your lovingly crafted custom offline page is put into a nice safe cache. You can use the Cache API to do this. You get to create as many caches as you like, and you can call them whatever you want. Here, I’m going to call the cache Johnny just so I can refer to it as JohnnyCache in the code:

addEventListener('install', installEvent => {
  installEvent.waitUntil(
    caches.open('Johnny')
    .then( JohnnyCache => {
      JohnnyCache.addAll([
       '/offline.html'
      ]); // end addAll
     }) // end open.then
  ); // end waitUntil
}); // end addEventListener

I’m betting that your lovely offline page is linking to a CSS file, maybe an image or two, and perhaps some JavaScript. You can cache all of those at this point:

addEventListener('install', installEvent => {
  installEvent.waitUntil(
    caches.open('Johnny')
    .then( JohnnyCache => {
      JohnnyCache.addAll([
       '/offline.html',
       '/path/to/stylesheet.css',
       '/path/to/javascript.js',
         '/path/to/image.jpg'
      ]); // end addAll
     }) // end open.then
  ); // end waitUntil
}); // end addEventListener

Make sure that the URLs are correct. If just one of the URLs in the list fails to resolve, none of the items in the list will be cached.

Intercept requests

The next event you want to listen for is the fetch event. This is probably the most powerful—and, let’s be honest, the creepiest—feature of a service worker. Once it has been installed, the service worker lurks on the user’s device, waiting for any requests made to your site. Every time the user requests a web page from your site, a fetch event will fire. Every time that page requests a style sheet or an image, a fetch event will fire. You can provide instructions for what should happen each time:

addEventListener('fetch', fetchEvent => {
// What happens next is up to you!
}); // end addEventListener

Let’s write a fairly conservative script with the following logic:

  • Whenever a file is requested,
  • First, try to fetch it from the network,
  • But if that doesn’t work, try to find it in the cache,
  • But if that doesn’t work, and it’s a request for a web page, show the custom offline page instead.

Here’s how that translates into JavaScript:

// Whenever a file is requested
addEventListener('fetch', fetchEvent => {
  const request = fetchEvent.request;
  fetchEvent.respondWith(
    // First, try to fetch it from the network
    fetch(request)
    .then( responseFromFetch => {
      return responseFromFetch;
    }) // end fetch.then
    // But if that doesn't work
    .catch( fetchError => {
      // try to find it in the cache
      caches.match(request)
      .then( responseFromCache => {
        if (responseFromCache) {
         return responseFromCache;
       // But if that doesn't work
       } else {
         // and it's a request for a web page
         if (request.headers.get('Accept').includes('text/html')) {
           // show the custom offline page instead
           return caches.match('/offline.html');
         } // end if
       } // end if/else
     }) // end match.then
   }) // end fetch.catch
  ); // end respondWith
}); // end addEventListener

I am fully aware that I may have done some owl-drawing there. If you need a more detailed breakdown of what’s happening at each point in the code, I’ve written a whole book for you. It’s the perfect present for Murphymas.

Hook up your service worker script

You can publish your service worker script at /serviceworker.js but you still need to tell the browser where to look for it. You can do that using JavaScript. Put this in an existing JavaScript file that you’re calling in to every page on your site, or add this in a script element at the end of every page’s HTML:

if (navigator.serviceWorker) {
  navigator.serviceWorker.register('/serviceworker.js');
}

That tells the browser to start installing the service worker, but not without first checking that the browser understands what a service worker is. When it comes to JavaScript, feature detection is your friend.

You might already have some JavaScript files in a folder like /assets/js/ and you might be tempted to put your service worker script in there too. Don’t do that. If you do, the service worker will only be able to handle requests made to for files within /assets/js/. By putting the service worker script in the root directory, you’re making sure that every request can be intercepted.

Go further!

Nicely done! You’ve made sure that if—no, when—a visitor can’t reach your website, they’ll get your hand-tailored offline page. You have temporarily defeated the forces of chaos! You have briefly fought the tide of entropy! You have made a small but ultimately futile gesture against the inevitable heat-death of the universe!

This is just the beginning. You can do more with service workers.

What if, every time you fetched a page from the network, you stored a copy of that page in a cache? Then if that person tries to reach that page later, but they’re offline, you could show them the cached version.

Or, what if instead of reaching out the network first, you checked to see if a file is in the cache first? You could serve up that cached version—which would be blazingly fast—and still fetch a fresh version from the network in the background to pop in the cache for next time. That might be a good strategy for images.

So many options! The hard part isn’t writing the code, it’s figuring out the steps you want to take. Once you’ve got those steps written out, then it’s a matter of translating them into JavaScript.

Inevitably there will be some obstacles along the way—usually it’s a misplaced curly brace or a missing parenthesis. Don’t be too hard on yourself if your code doesn’t work at first. That’s just Murphy’s Law in action.

Wednesday, December 5th, 2018

Going Offline First (Video Series)

A five-part video series from Ire on how she built the “save for offline” functionality on her site.

The first one is about getting a set set up on Ghost so you can probably safely skip that one and go straight to the second video to get down to the nitty-gritty of the Cache API and service workers.

Tuesday, December 4th, 2018

Mistletoe Offline ◆ 24 ways

They let me write a 24 Ways article again. Will they never learn?

This one’s a whirlwind tour of using a service worker to provide a custom offline page, in the style of Going Offline.

By the way, just for the record, I initially rejected this article’s title out of concern that injecting a Cliff Richard song into people’s brains was cruel and unusual punishment. I was overruled.

Monday, November 12th, 2018

Home  |  web.dev

I guess this domain name is why our local developmemnt environments stopped working.

Anyway, it’s a web interface onto Lighthouse (note that it has the same bugs as the version of Lighthouse in Chrome). Kind of like webhint.io.

Sunday, November 11th, 2018

Push without notifications

On the first day of Indie Web Camp Berlin, I led a session on going offline with service workers. This covered all the usual use-cases: pre-caching; custom offline pages; saving pages for offline reading.

But on the second day, Sebastiaan spent a fair bit of time investigating a more complex use of service workers with the Push API.

The Push API is what makes push notifications possible on the web. There are a lot of moving parts—browser, server, service worker—and, frankly, it’s way over my head. But I’m familiar with the general gist of how it works. Here’s a typical flow:

  1. A website prompts the user for permission to send push notifications.
  2. The user grants permission.
  3. A whole lot of complicated stuff happens behinds the scenes.
  4. Next time the website publishes something relevant, it fires a push message containing the details of the new URL.
  5. The user’s service worker receives the push message (even if the site isn’t open).
  6. The service worker creates a notification linking to the URL, interrupting the user, and generally adding to the weight of information overload.

Here’s what Sebastiaan wanted to investigate: what if that last step weren’t so intrusive? Here’s the alternate flow he wanted to test:

  1. A website prompts the user for permission to send push notifications.
  2. The user grants permission.
  3. A whole lot of complicated stuff happens behinds the scenes.
  4. Next time the website publishes something relevant, it fires a push message containing the details of the new URL.
  5. The user’s service worker receives the push message (even if the site isn’t open).
  6. The service worker fetches the contents of the URL provided in the push message and caches the page. Silently.

It worked.

I think this could be a real game-changer. I don’t know about you, but I’m very, very wary of granting websites the ability to send me push notifications. In fact, I don’t think I’ve ever given a website permission to interrupt me with push notifications.

You’ve seen the annoying permission dialogues, right?

In Firefox, it looks like this:

Will you allow name-of-website to send notifications?

[Not Now] [Allow Notifications]

In Chrome, it’s:

name-of-website wants to

Show notifications

[Block] [Allow]

But in actual fact, these dialogues are asking for permission to do two things:

  1. Receive messages pushed from the server.
  2. Display notifications based on those messages.

There’s no way to ask for permission just to do the first part. That’s a shame. While I’m very unwilling to grant permission to be interrupted by intrusive notifications, I’d be more than willing to grant permission to allow a website to silently cache timely content in the background. It would be a more calm technology.

Think of the use cases:

  • I grant push permission to a magazine. When the magazine publishes a new article, it’s cached on my device.
  • I grant push permission to a podcast. Whenever a new episode is published, it’s cached on my device.
  • I grant push permission to a blog. When there’s a new blog post, it’s cached on my device.

Then when I’m on a plane, or in the subway, or in any other situation without a network connection, I could still visit these websites and get content that’s fresh to me. It’s kind of like background sync in reverse.

There’s plenty of opportunity for abuse—the cache could get filled with content. But websites can already do that, and they don’t need to be granted any permissions to do so; just by visiting a website, it can add multiple files to a cache.

So it seems that the reason for the permissions dialogue is all about displaying notifications …not so much about receiving push messages from the server.

I wish there were a way to implement this background-caching pattern without requiring the user to grant permission to a dialogue that contains the word “notification.”

I wonder if the act of adding a site to the home screen could implicitly grant permission to allow use of the Push API without notifications?

In the meantime, the proposal for periodic synchronisation (using background sync) could achieve similar results, but in a less elegant way; periodically polling for new content instead of receiving a push message when new content is published. Also, it requires permission. But at least in this case, the permission dialogue should be more specific, and wouldn’t include the word “notification” anywhere.

Thursday, November 8th, 2018

A Book Apart, Front-End Next Steps

If you buy this bundle of books, you get Going Offline in some very, very good company.

Monday, October 22nd, 2018

PWA Directory

Another directory of progressive web apps, this time maintained by Google.

I quite like the way it links through to a Lighthouse report. Here’s the listing for The Session, for example, and here’s the corresponding Lighthouse report.

Friday, September 28th, 2018

Thoughts on Offline-first | Trys Mudford

Service Workers have such huge potential power, and I feel like we (developers on the web) have barely scratched the surface with what’s possible.

Needless to say, I couldn’t agree more!

Trys is thinking through some of the implicatons of service workers, like how we refresh stale content, and how we deal with slow networks—something that’s actually more of a challenge than dealing with no network connection at all.

There’s some good food for thought here.

I’m so excited to see how we can use Service Workers to improve the web.

Sunday, September 23rd, 2018

Service workers in Samsung Internet browser

I was getting reports of some odd behaviour with the service worker on thesession.org, the Irish music website I run. Someone emailed me to say that they kept getting the offline page, even when their internet connection was perfectly fine and the site was up and running.

They didn’t mind answering my pestering follow-on questions to isolate the problem. They told me that they were using the Samsung Internet browser on Android. After a little searching, I found this message on a Github thread about using waitUntil. It’s from someone who works on the Samsung Internet team:

Sadly, the asynchronos waitUntil() is not implemented yet in our browser. Yes, we will implement it but our release cycle is so far. So, for a long time, we might not resolve the issue.

A-ha! That explains the problem. See, here’s the pattern I was using:

  1. When someone requests a file,
  2. fetch that file from the network,
  3. create a copy of the file and cache it,
  4. return the contents.

Step 1 is the event listener:

// 1. When someone requests a file
addEventListener('fetch', fetchEvent => {
  let request = fetchEvent.request;
  fetchEvent.respondWith(

Steps 2, 3, and 4 are inside that respondWith:

// 2. fetch that file from the network
fetch(request)
.then( responseFromFetch => {
  // 3. create a copy of the file and cache it
  let copy = responseFromFetch.clone();
  caches.open(cacheName)
  .then( cache => {
    cache.put(request, copy);
  })
  // 4. return the contents.
  return responseFromFetch;
})

Step 4 might well complete while step 3 is still running (remember, everything in a service worker script is asynchronous so even though I’ve written out the steps sequentially, you never know what order the steps will finish in). That’s why I’m wrapping that third step inside fetchEvent.waitUntil:

// 2. fetch that file from the network
fetch(request)
.then( responseFromFetch => {
  // 3. create a copy of the file and cache it
  let copy = responseFromFetch.clone();
  fetchEvent.waitUntil(
    caches.open(cacheName)
    .then( cache => {
      cache.put(request, copy);
    })
  );
  // 4. return the contents.
  return responseFromFetch;
})

If a browser (like Samsung Internet) doesn’t understand the bit where I say fetchEvent.waitUntil, then it will throw an error and execute the catch clause. That’s where I have my fifth and final step: “try looking in the cache instead, but if that fails, show the offline page”:

.catch( fetchError => {
  console.log(fetchError);
  return caches.match(request)
  .then( responseFromCache => {
    return responseFromCache || caches.match('/offline');
  });
})

Normally in this kind of situation, I’d use feature detection to check whether a browser understands a particular API method. But it’s a bit tricky to test for support for asynchronous waitUntil. That’s okay. I can use a try/catch statement instead. Here’s what my revised code looks like:

fetch(request)
.then( responseFromFetch => {
  let copy = responseFromFetch.clone();
  try {
    fetchEvent.waitUntil(
      caches.open(cacheName)
      .then( cache => {
        cache.put(request, copy);
      })
    );
  } catch (error) {
    console.log(error);
  }
  return responseFromFetch;
})

Now I’ve managed to localise the error. If a browser doesn’t understand the bit where I say fetchEvent.waitUntil, it will execute the code in the catch clause, and then carry on as usual. (I realise it’s a bit confusing that there are two different kinds of catch clauses going on here: on the outside there’s a .then()/.catch() combination; inside is a try{}/catch{} combination.)

At some point, when support for async waitUntil statements is universal, this precautionary measure won’t be needed, but for now wrapping them inside try doesn’t do any harm.

There are a few places in chapter five of Going Offline—the chapter about service worker strategies—where I show examples using async waitUntil. There’s nothing wrong with the code in those examples, but if you want to play it safe (especially while Samsung Internet doesn’t support async waitUntil), feel free to wrap those examples in try/catch statements. But I’m not going to make those changes part of the errata for the book. In this case, the issue isn’t with the code itself, but with browser support.

Saturday, September 1st, 2018

Offline Content with Service Worker — Chris Ruppel

A step-by-step walkthrough of a really useful service worker pattern: allowing users to save articles for offline reading at the click of a button (kind of like adding the functionality of Instapaper or Pocket to your own site).

Thursday, August 30th, 2018

Offline Web Experiences with Jeremy Keith « CTRL+CLICK CAST

I had a great time chatting with Lea and Emily about service workers on this episode of their podcast—they’re such great hosts!

Here’s the huffduffed audio.

Thursday, August 16th, 2018

Going Offline by Jeremy Keith – a post by Marc Thiele

This is such a lovely, lovely review from Marc!

Jeremy’s way of writing certainly helps, as a specialised or technical book on a topic like Service Workers, could certainly be one, that bores you to death with dry written explanations. But Jeremy has a friendly, fresh and entertaining way of writing books. Sometimes I caught myself with a grin on my face…

Tuesday, August 14th, 2018

005: Service workers - Web Components Club

I strongly recommend that you read Going Offline by Jeremy Keith. Before his book, I found the concept of service workers quite daunting and convinced myself that it’s one of those things that I’ll have to set aside a big chunk of time to learn. I got through Jeremy’s book in a few hours and felt confident and inspired. This is because he’s very good at explaining concepts in a friendly, concise manner.

Friday, August 10th, 2018

PWA: Progressive Web All-the-things - Tales of a Developer Advocate by Paul Kinlan

Very valuable observations from Paul on his travels, talking to developers and business people about progressive web apps—there’s some confusion out there.

My personal feeling is that everyone is really hung up on the A in PWA: ‘App’. It’s the success and failure of the branding of the concept; ‘App’ is in the name, ‘App’ is in the conscious of many users and businesses and so the associations are quite clear.

Tuesday, August 7th, 2018

Seriously, though. What is a progressive web app? – Amberley Romo – Medium

What an excellent question! And what an excellent bit of sleuthing to get to the bottom of it. This is like linguistic spelunking on the World Wide Web.

Oh, and of course I love the little sidenote at the end.

Tuesday, July 24th, 2018

Altering expectations

Luke has written up the selection process he went through when Clearleft was designing the Virgin Holidays app. When it comes to deploying on mobile, there were three options:

  1. Native apps
  2. A progressive web app
  3. A hybrid app

The Virgin Holidays team went with that third option.

Now, it will come as no surprise that I’m a big fan of the second option: building a progressive web app (or turning an existing site into a progressive web app). I think a progressive web app is a great solution for travel apps, and the use-case that Luke describes sounds perfect:

Easy access to resort staff and holiday details that could be viewed offline to help as many customers as possible travel without stress and enjoy a fantastic holiday

Luke explains why they choice not to go with a progressive web app.

The current level of support and leap in understanding meant we’d risk alienating many of our customers.

The issue of support is one that is largely fixed at this point. When Clearleft was working on the Virgin Holidays app, service workers hadn’t landed in iOS. Hence, the risk of alienating a lot of customers. But now that Mobile Safari has offline capabilities, that’s no longer a problem.

But it’s the second reason that’s trickier:

Simply put, customers already expected to find us in the App Store and are familiar with what apps can historically offer over websites.

I think this is the biggest challenge facing progressive web apps: battling expectations.

For over a decade, people have formed ideas about what to expect from the web and what to expect from native. From a technical perspective, native and web have become closer and closer in capabilities. But people’s expectations move slower than technological changes.

First of all, there’s the whole issue of discovery: will people understand that they can “install” a website and expect it to behave exactly like a native app? This is where install prompts and ambient badging come in. I think ambient badging is the way to go, but it’s still a tricky concept to explain to people.

But there’s another way of looking at the current situation. Instead of seeing people’s expectations as a negative factor, maybe it’s an opportunity. There’s an opportunity right now for companies to be as groundbreaking and trendsetting as Wired.com when it switched to CSS for layout, or The Boston Globe when it launched its responsive site.

It makes for a great story. Just look at the Pinterest progressive web app for an example (skip to the end to get to the numbers):

Weekly active users on mobile web have increased 103 percent year-over-year overall, with a 156 percent increase in Brazil and 312 percent increase in India. On the engagement side, session length increased by 296 percent, the number of Pins seen increased by 401 percent and people were 295 percent more likely to save a Pin to a board. Those are amazing in and of themselves, but the growth front is where things really shined. Logins increased by 370 percent and new signups increased by 843 percent year-over-year. Since we shipped the new experience, mobile web has become the top platform for new signups. And for fun, in less than 6 months since fully shipping, we already have 800 thousand weekly users using our PWA like a native app (from their homescreen).

Now admittedly their previous mobile web experience was a dreadful doorslam, but still, those are some amazing statistics!

Maybe we’re underestimating the malleability of people’s expectations when it comes to the web on mobile. Perhaps the inertia we think we’re battling against isn’t such a problem as long as we give people a fast, reliable, engaging experience.

If you build that, they will come.

Wednesday, July 11th, 2018

Monday, July 9th, 2018

devMode.fm // Going Offline: Service Workers with Jeremy Keith

I talked for an hour about service workers ‘n’ stuff

(Also available on Huffduffer.)

Thursday, July 5th, 2018

The trimCache function in Going Offline

Paul Yabsley wrote to let me know about an error in Going Offline. It’s rather embarrassing because it’s code that I’m using in the service worker for adactio.com but for some reason I messed it up in the book.

It’s the trimCache function in Chapter 7: Tidying Up. That’s the reusable piece of code that recursively reduces the number of items in a specified cache (cacheName) to a specified amount (maxItems). On page 95 and 96 I describe the process of creating the function which, in the book, ends up like this:

 function trimCache(cacheName, maxItems) {
   cacheName.open( cache => {
     cache.keys()
     .then( items => {
       if (items.length > maxItems) {
         cache.delete(items[0])
         .then(
           trimCache(cacheName, maxItems)
         ); // end delete then
       } // end if
     }); // end keys then
   }); // end open
 } // end function

See the problem? It’s right there at the start when I try to open the cache like this:

cacheName.open( cache => {

That won’t work. The open method only works on the caches object—I should be passing the name of the cache into the caches.open method. So the code should look like this:

caches.open( cacheName )
.then( cache => {

Everything else remains the same. The corrected trimCache function is here:

function trimCache(cacheName, maxItems) {
  caches.open(cacheName)
  .then( cache => {
    cache.keys()
    .then(items => {
      if (items.length > maxItems) {
        cache.delete(items[0])
        .then(
          trimCache(cacheName, maxItems)
        ); // end delete then
      } // end if
    }); // end keys then
  }); // end open then
} // end function

Sorry about that! I must’ve had some kind of brainfart when I was writing (and describing) that one line of code.

You may want to deface your copy of Going Offline by taking a pen to that code example. Normally I consider the practice of writing in books to be barbarism, but in this case …go for it.