Tags: storage

58

sparkline

Monday, April 6th, 2020

Local-first software: You own your data, in spite of the cloud

The cloud gives us collaboration, but old-fashioned apps give us ownership. Can’t we have the best of both worlds?

We would like both the convenient cross-device access and real-time collaboration provided by cloud apps, and also the personal ownership of your own data embodied by “old-fashioned” software.

This is a very in-depth look at the mindset and the challenges involved in building truly local-first software—something that Tantek has also been thinking about.

Thursday, March 26th, 2020

Apple’s attack on service workers

Apple aren’t the best at developer relations. But, bad as their communications can be, I’m willing to cut them some slack. After all, they’re not used to talking with the developer community.

John Wilander wrote a blog post that starts with some excellent news: Full Third-Party Cookie Blocking and More. Safari is catching up to Firefox and disabling third-party cookies by default. Wonderful! I’ve had third-party cookies disabled for a few years now, and while something occassionally breaks, it’s honestly a pretty great experience all around. Denying companies the ability to track users across sites is A Good Thing.

In the same blog post, John said that client-side cookies will be capped to a seven-day lifespan, as previously announced. Just to be clear, this only applies to client-side cookies. If you’re setting a cookie on the server, using PHP or some other server-side language, it won’t be affected. So persistent logins are still doable.

Then, in an audacious example of burying the lede, towards the end of the blog post, John announces that a whole bunch of other client-side storage technologies will also be capped to seven days. Most of the technologies are APIs that, like cookies, can be used to store data: Indexed DB, Local Storage, and Session Storage (though there’s no mention of the Cache API). At the bottom of the list is this:

Service Worker registrations

Okay, let’s clear up a few things here (because they have been so poorly communicated in the blog post)…

The seven day timer refers to seven days of Safari usage, not seven calendar days (although, given how often most people use their phones, the two are probably interchangable). So if someone returns to your site within a seven day period of using Safari, the timer resets to zero, and your service worker gets a stay of execution. Lucky you.

This only applies to Safari. So if your site has been added to the home screen and your web app manifest has a value for the “display” property like “standalone” or “full screen”, the seven day timer doesn’t apply.

That piece of information was missing from the initial blog post. Since the blog post was updated to include this clarification, some people have taken this to mean that progressive web apps aren’t affected by the upcoming change. Not true. Only progressive web apps that have been added to the home screen (and that have an appropriate “display” value) will be spared. That’s a vanishingly small percentage of progressive web apps, especially on iOS. To add a site to the home screen on iOS, you need to dig and scroll through the share menu to find the right option. And you need to do this unprompted. There is no ambient badging in Safari to indicate that a site is installable. Chrome’s install banner isn’t perfect, but it’s better than nothing.

Just a reminder: a progressive web app is a website that

  • runs on HTTPS,
  • has a service worker,
  • and a web manifest.

Adding to the home screen is something you can do with a progressive web app (or any other website). It is not what defines progressive web apps.

In any case, this move to delete service workers after seven days of using Safari is very odd, and I’m struggling to find the connection to the rest of the blog post, which is about technologies that can store data.

As I understand it, with the crackdown on setting third-party cookies, trackers are moving to first-party technologies. So whereas in the past, a tracking company could tell its customers “Add this script element to your pages”, now they have to say “Add this script element and this script file to your pages.” That JavaScript file can then store a unique idenitifer on the client. This could be done with a cookie, with Local Storage, or with Indexed DB, for example. But I’m struggling to understand how a service worker script could be used in this way. I’d really like to see some examples of this actually happening.

The best explanation I can come up with for this move by Apple is that it feels like the neatest solution. That’s neat as in tidy, not as in nifty. It is definitely not a nifty solution.

If some technologies set by a specific domain are being purged after seven days, then the tidy thing to do is purge all technologies from that domain. Service workers are getting included in that dragnet.

Now, to be fair, browsers and operating systems are free to clean up storage space as they see fit. Caches, Local Storage, Indexed DB—all of those are subject to eventually getting cleaned up.

So I was curious. Wanting to give Apple the benefit of the doubt, I set about trying to find out how long service worker registrations currently last before getting deleted. Maybe this announcement of a seven day time limit would turn out to be not such a big change from current behaviour. Maybe currently service workers last for 90 days, or 60, or just 30.

Nope:

There was no time limit previously.

This is not a minor change. This is a crippling attack on service workers, a technology specifically designed to improve the user experience for return visits, whether it’s through improved performance or offline access.

I wouldn’t be so stunned had this announcement come with an accompanying feature that would allow Safari users to know when a website is a progressive web app that can be added to the home screen. But Safari continues to ignore the existence of progressive web apps. And now it will actively discourage people from using service workers.

If you’d like to give feedback on this ludicrous development, you can file a bug (down in the cellar in the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying “Beware of the Leopard”).

No doubt there will still be plenty of Apple apologists telling us why it’s good that Safari has wished service workers into the cornfield. But make no mistake. This is a terrible move by Apple.

I will say this though: given The Situation we’re all living in right now, some good ol’ fashioned Hot Drama by a browser vendor behaving badly feels almost comforting.

Friday, February 28th, 2020

The Permanent Legacy Foundation

A non-profit that offers digital preservation services for individuals.

Permanence means no subscriptions; a one-time payment for dedicated storage that preserves your most precious memories and an institution that will be there to protect the digital legacy of all people for all time.

Thursday, February 6th, 2020

Local First, Undo Redo, JS-Optional, Create Edit Publish - Tantek

Tantek documents the features he wants his posting interface to have.

Monday, December 2nd, 2019

Just enough Internet | doteveryone

The carbon cost of collecting and storing data no one can use is already a moral issue.

So before you add another field, let alone make a new service, can you be sure it will make enough of a difference to legitimise its impact on the planet?

Tuesday, November 19th, 2019

The GitHub Archive Program will safely store every public GitHub repo for 1,000 years in the Arctic World Archive in Svalbard, Norway.

This is a fascinating project from Github, the Long Now Foundation, the Internet Archive, the Bodleian Library and others. All of the public code on Github on February 2nd, 2020 will be archived for 1000 years in a vault in Svalbard.

Mind you, given the amount of dependencies that most “modern” code projects rely on, I can’t foresee the code working after 1000 days.

Friday, September 6th, 2019

Offline listings

This is brilliant technique by Remy!

If you’ve got a custom offline page that lists previously-visited pages (like I do on my site), you don’t have to choose between localStorage or IndexedDB—you can read the metadata straight from the HTML of the cached pages instead!

This seems forehead-smackingly obvious in hindsight. I’m totally stealing this.

Saturday, July 6th, 2019

The Hiding Place: Inside the World’s First Long-Term Storage Facility for Highly Radioactive Nuclear Waste - Pacific Standard

Robert McFarlane’s new book is an exploration of deep time. In this extract, he visits the Onkalo nuclear waste storage facility in Finland.

Sometimes we bury materials in order that they may be preserved for the future. Sometimes we bury materials in order to preserve the future from them.

Monday, January 28th, 2019

The 500-Year-Long Science Experiment - The Atlantic

Running an experiment for 500 years is hard enough. Then there’s the documentation…

The hard part is ensuring someone will continue doing this on schedule well into the future. The team left a USB stick with instructions, which Möller realizes is far from adequate, given how quickly digital technology becomes obsolete. They also left a hard copy, on paper. “But think about 500-year-old paper,” he says, how it would yellow and crumble. “Should we carve it in stone? Do we have to carve it in a metal plate?” But what if someone who cannot read the writing comes along and decides to take the metal plate as a cool, shiny relic, as tomb raiders once did when looting ancient tombs?

No strategy is likely to be completely foolproof 500 years later. So the team asks that researchers at each 25-year time point copy the instructions so that they remain linguistically and technologically up to date.

Friday, November 23rd, 2018

Home - Memory of Mankind

A time capsule for the long now. Laser-etched ceramic tablets in an Austrian salt mine carry memories of our civilisation in three categories: news editorials, scientific works, and personal stories.

You can contribute a personal story, your favorite poem, or newspaper articles which describe our problems, visions or our daily life.

Tokens that mark the location of the site are also being distributed across the planet.

Sunday, September 9th, 2018

Why the Future of Data Storage is (Still) Magnetic Tape - IEEE Spectrum

It turns out that a whole lot of The So-Called Cloud is relying on magnetic tape for its backups.

Friday, June 22nd, 2018

Offline-Friendly Forms by Max Böck

A clever use of localStorage to stop data from being lost when your visitors are offline.

Friday, March 30th, 2018

Putting Civilization in a Box Means Choosing Our Legacy

A run-down of digital preservation technologies for very, very long-term storage …in space.

Tuesday, March 6th, 2018

Brendan Dawes - Back your sh*t up!

My back-up strategy is similar to Brendan’s (using Super Duper and Backblaze):

In backup parlance there’s a thing called 3-2-1. That is, you should three copies of your files — two locally on different devices and one off site.

But I only do my local back-ups once a week (eek!)—I should do better.

Monday, February 26th, 2018

The Internet Isn’t Forever

A terrific piece by Maria Bustillos on digital preservation and the power of archives, backed up with frightening real-world examples.

Because history is a fight we’re having every day. We’re battling to make the truth first by living it, and then by recording and sharing it, and finally, crucially, by preserving it. Without an archive, there is no history.

Friday, February 9th, 2018

Workers at Your Service | WebKit

Here’s an interesting insight on how WebKit is going to handle the cleanup of unused service workers and caches:

Service worker and Cache API stored information will grow as a user is browsing content. To keep only the stored information that is useful to the user, WebKit will remove unused service worker registrations after a period of a few weeks. Caches that do not get opened after a few weeks will also be removed.

Friday, January 26th, 2018

Arch Mission

Off-site backups of humanity’s knowledge and culture, stored in different media (including pyramidal crystals) placed in near-Earth orbit, the moon, and Mars.

We are developing specialized next-generation devices that we call Archs™ (pronounced “Arks”), which are designed to hold and transmit large amounts of data over long periods of time in extreme environments, including outer space and on the surfaces of other planetary bodies.

Our goal is to collect and curate important data sets and to install them on Archs™ that will be delivered to as many locations as possible for safekeeping.

To increase the chances that Archs™ will be found in the future, we aim for durability and massive redundancy across a broad diversity of locations and materials – a strategy that nature itself has successfully employed.

Tuesday, January 2nd, 2018

Monday, October 2nd, 2017

eBay’s Font Loading Strategy | eBay Tech Blog

Here’s the flow that eBay use for the font-loading. They’ve decided that on the very first page view, seeing a system font is an acceptable trade-off. I think that makes sense for their situation.

Interestingly, they set a flag for subsequent visits using localStorage rather than a cookie. I wonder why that is? For me, the ability to read cookies on the server as well as the client make them quite handy for situations like this.

Friday, September 22nd, 2017

How much storage space is my Progressive Web App using? | Dean Hume

You can use navigator.storage.estimate() to get a (vague) idea of how much space is available on a device for your service worker caches.