Tags: archive

104

sparkline

Thursday, November 2nd, 2017

The dConstruct Audio Archive works offline

The dConstruct conference is as old as Clearleft itself. We put on the first event back in 2005, the year of our founding. The last dConstruct was in 2015. It had a good run.

I’m really proud of the three years I ran the show—2012, 2013, and 2014—and I have great memories from each event. I’m inordinately pleased that the individual websites are still online after all these years. I’m equally pleased with the dConstruct audio archive that we put online in 2012. Now that the event itself is no longer running, it truly is an archive—a treasury of voices from the past.

I think that these kinds of online archives are eminently suitable for some offline design. So I’ve added a service worker script to the dConstruct archive.

Caching

To start with, there’s the no-brainer: as soon as someone hits the website, pre-cache static assets like CSS, JavaScript, the logo, and icon images. Now subsequent page loads will be quicker—those assets are taken straight from the cache.

But what about the individual pages? For something like Resilient Web Design—another site that won’t be updated—I pre-cache everything. I could do that with the dConstruct archive. All of the pages with all of the images add up to less than two megabytes; the entire site weighs less than a single page on Wired.com or The Verge.

In the end, I decided to go with a cache-as-you-go strategy. Every time a page or an image is fetched from the network, it is immediately put in a cache. The next time that page or image is requested, the file is served from that cache instead of the network.

Here’s the logic for fetch requests:

  1. First, look to see if the file is in a cache. If it is, great! Serve that.
  2. If the file isn’t in a cache, make a network request and serve the response …but put a copy of a file in the cache.
  3. The next time that file is requested, go to step one.

Save for offline

That caching strategy works great for pages, images, and other assets. But there’s one kind of file on the dConstruct archive that’s a bit different: the audio files. They can be fairly big, so I don’t want to cache those unless the user specifically requests it.

If you end up on the page for a particular talk, and your browser supports service workers, you’ll get an additional UI element in the list of options: a toggle to “save offline” (under the hood, it’s a checkbox). If you activate that option, then the audio file gets put into a cache.

Now if you lose your network connection while browsing the site, you’ll get a custom offline page with the option to listen to any audio files you saved for offline listening. You’ll also see this collection of talks on the homepage, regardless of whether you’ve got an internet connection or not.

So if you’ve got a long plane journey ahead of you, have a browse around the dConstruct archive and select some talks for your offline listening pleasure.

Or just enjoy the speediness of browsing the site.

Turning another website into a Progressive Web App.

Thursday, September 7th, 2017

Now and Then Cobh

Time-shifted photographs of my hometown in Ireland.

Friday, August 18th, 2017

Poco Apollo

Here’s a beautiful use of the web audio API: Enoesque generative music composed right in your browser. Each piece is generated from one of the 14,226 photos in NASA’s Apollo archive. The darker and murkier the picture, the moodier the music.

Friday, June 9th, 2017

Human Document Project 2017

A conference in my old stomping grounds of Freiburg on archives, preservation, and long-term thinking:

It will present the state of art in long-term archiving as well as the present problems in preservation of information and scientific data in archives and libraries. Perhaps the most interesting aspect is that, since all conceivable systems are finite but can be quite large, a choice on the contents has to be made. This requires thinking of the human condition: Who we are, what we are and what do we find worth to preserve.

Sunday, June 4th, 2017

Month maps

One of the topics I enjoy discussing at Indie Web Camps is how we can use design to display activity over time on personal websites. That’s how I ended up with sparklines on my site—it was the a direct result of a discussion at Indie Web Camp Nuremberg a year ago:

During the discussion at Indie Web Camp, we started looking at how silos design their profile pages to see what we could learn from them. Looking at my Twitter profile, my Instagram profile, my Untappd profile, or just about any other profile, it’s a mixture of bio and stream, with the addition of stats showing activity on the site—signs of life.

Perhaps the most interesting visual example of my activity over time is on my Github profile. Halfway down the page there’s a calendar heatmap that uses colour to indicate the amount of activity. What I find interesting is that it’s using two axes of time over a year: days of the month across the X axis and days of the week down the Y axis.

I wanted to try something similar, but showing activity by time of day down the Y axis. A month of activity feels like the right range to display, so I set about adding a calendar heatmap to monthly archives. I already had the data I needed—timestamps of posts. That’s what I was already using to display sparklines. I wrote some code to loop over those timestamps and organise them by day and by hour. Then I spit out a table with days for the columns and clumps of hours for the rows.

Calendar heatmap on Dribbble

I’m using colour (well, different shades of grey) to indicate the relative amounts of activity, but I decided to use size as well. So it’s also a bubble chart.

It doesn’t work very elegantly on small screens: the table is clipped horizontally and can be swiped left and right. Ideally the visualisation itself would change to accommodate smaller screens.

Still, I kind of like the end result. Here’s last month’s activity on my site. Here’s the same time period ten years ago. I’ve also added month heatmaps to the monthly archives for my journal, links, and notes. They’re kind of like an expanded view of the sparklines that are shown with each month.

From one year ago, here’s the daily distribution of

And then here’s the the daily distribution of everything in that month all together.

I realise that the data being displayed is probably only of interest to me, but then, that’s one of the perks of having your own website—you can do whatever you feel like.

Wednesday, May 3rd, 2017

The Lost Picture Show: Hollywood Archivists Can’t Outpace Obsolescence - IEEE Spectrum

There are three parts to digital preservation: format, medium, and licensing. Film and television archives are struggling with all three.

Format:

Codecs—the software used to compress and decompress digital video files—keep changing, as do the hardware and software for playback.

Medium:

As each new generation of LTO comes to market, an older generation of LTO becomes obsolete. LTO manufacturers guarantee at most two generations of backward compatibility. What that means for film archivists with perhaps tens of thousands of LTO tapes on hand is that every few years they must invest millions of dollars in the latest format of tapes and drives and then migrate all the data on their older tapes—or risk losing access to the information altogether.

Licensing:

Studios didn’t see any revenue potential in their past work. They made money by selling movie tickets; absent the kind of follow-on markets that exist today, long-term archiving didn’t make sense economically.

It adds up to a potential cultural disaster:

If technology companies don’t come through with a long-term solution, it’s possible that humanity could lose a generation’s worth of filmmaking, or more.

Tuesday, May 2nd, 2017

The Last 100 Days, the Next 100 Years

Cancelling the future.

The future lives and dies by the state of the archives. To look hard at this world and honestly, diligently articulate what happened and what it was like in the present is a sort of promise to the future, a new layer to the palimpsest of history that can become someone else’s foundation.

Tuesday, April 25th, 2017

DRM for the Web is a Bad Idea | Internet Archive Blogs

The Encrypted Media Extensions (EME) addition to HTML is effectively DRM with the blessing of the W3C. It’s bad for accessibility, bad for usability, bad for security, and as the Internet Archive rightly points out, it’s bad for digital preservation.

Sunday, April 16th, 2017

Adventure

The Internet Archive is now hosting early Macintosh software emulated right in your browser. That means you can play Adventure: the source of subsequent text adventures, natural language parsing, and chatbots.

Colossal Cave Adventure (also known as ADVENT, Colossal Cave, or Adventure) is a text adventure game, developed originally in 1976, by Will Crowther for the PDP-10 mainframe. The game was expanded upon in 1977, with help from Don Woods, and other programmers created variations on the game and ports to other systems in the following years.

In the game, the player controls a character through simple text commands to explore a cave rumored to be filled with wealth.

Monday, November 7th, 2016

The road to Indie Web Camp LA

After An Event Apart San Francisco, which was—as always—excellent, it was time for me to get to the next event: Indie Web Camp Los Angeles. But I wasn’t going alone. Tantek was going too, and seeing as he has a car—a convertible, even—what better way to travel from San Francisco to LA than on the Pacific Coast Highway?

It was great—travelling through the land of Steinbeck and Guthrie at the speed of Kerouac and Springsteen. We stopped for the night at Pismo Beach and then continued on, rolling into Santa Monica at sunset.

Half Moon Bay. Roadtripping with @t. Pomponio beach. Windswept. Salinas. Refueling. Driving through the Californian night. Pismo Beach. On the beach. On the beach with @t. Stopping for a coffee in Santa Barbara. Leaving Pismo Beach. Chevron. Santa Barbara steps. On the road. Driving through Malibu. Malibu sunset. Sun worshippers. Sunset in Santa Monica.

The weekend was spent in the usual Indie Web Camp fashion: a day of BarCamp-style discussions, followed by a day of hacking on our personal websites.

I decided to follow on from what I did at the Brighton Indie Web Camp. There, I made a combined tag view—a way of seeing, for example, everything tagged with “indieweb” instead of just journal entries tagged with “indieweb” or links tagged with “indieweb”. I wanted to do the same thing with my archives. I have separate archives for my journal, my links, and my notes. What I wanted was a combined view.

After some hacking, I got it working. So now you can see combined archives by year, month, and day (I managed to add a sparkline to the month view as well):

I did face a bit of a conundrum. Both my home page stream and my tag pages show posts in reverse chronological order, with the newest posts at the top. I’ve decided to replicate that for the archive view, but I’m not sure if that’s the right decision. Maybe the list of years should begin with 2001 and end with 2016, instead of the other way around. And maybe when you’re looking at a month of posts, you should see the first posts in that month at the top.

Anyway, I’ll live with it in reverse chronological order for a while and see how it feels. I’m just glad I managed to get it down—I’ve been meaning to do it for quite a while. Once again, I’m amazed by how much gets accomplished when you’re in the same physical space as other helpful, motivated people all working on improving their indie web presence, little by little.

Greetings from Indie Web Camp LA. Indie Web Camping. Hacking away. Day two of Indie Web Camp LA.

Tuesday, November 1st, 2016

The Digital Transition: How the Presidential Transition Works in the Social Media Age | whitehouse.gov

Kori Schulman describes the archiving of social media and other online artefacts of the outgoing US president. It’s a shame that a lot of URLs will break, but I’m glad there’s going to be a public backup available.

Best of all, you can get involved:

In the interim, we’re inviting the American public – from students and data engineers, to artists and researchers – to come up with creative ways to archive this content and make it both useful and available for years to come. From Twitter bots and art projects to printed books and query tools, we’re open to it all.

Thursday, September 1st, 2016

My Decade of Blogging

Heartfelt congratulations to Remy on ten years of blogging.

More importantly, every single URL on my blog that’s ever been published still works, and even better than that (for me) is my archive showing off the decade of writing I’ve been producing over all this time 💪

Wednesday, July 6th, 2016

Persistent Domains by Tim Berners-Lee

This sixteen year old cool URI has not changed. I think this idea of domains entering an archive state is worth pursuing.

Also, I love the science fictional footnote “Note for readers after 2100”.

Saturday, June 11th, 2016

The Web’s Creator Looks to Reinvent It - The New York Times

“The web is already decentralized,” Mr. Berners-Lee said. “The problem is the dominance of one search engine, one big social network, one Twitter for microblogging. We don’t have a technology problem, we have a social problem.”

Wednesday, June 1st, 2016

adactio - Upcoming.org Archive

My old Upcoming.org profile is back online, along with everyone else’s:

This is a static historical archive more than 7 million events saved from Upcoming’s first ten years.

I’m enjoying this trip down memory lane, recalling fun times from the last decade.

Friday, May 13th, 2016

Archiving a Website for Ten Thousand Years - The Atlantic

Prompted by the way Craig is handling the shutdown of hi.co, Glenn Fleishman takes a look at other digital preservation efforts and talk to Laura Welcher at the Long Now Foundation.

A time capsule is bottled optimism. It makes material the belief that human beings will survive long enough to retrieve and decode artifacts of the distant past.

Wednesday, May 11th, 2016

Decentralized Web Summit: Locking the Web Open

Oh, how I wish I could make it to this event!

June 8th-9th at Internet Archive, featuring Vint Cerf, Brewster Kahle, and more.

We are bringing together a diverse group of Web architects, activists, engineers, archivists, scholars, journalists, and other stakeholders to explore the technology required to build a Decentralized Web and its impact.

Thursday, April 14th, 2016

The Internet Archive—Bricks and Mortar Version - Scientific American Blog Network

A profile of the Internet Archive, but this time focusing on its physical space.

The Archive is a third place unlike any other.

Friday, April 8th, 2016

Messages to the Future, by Heather Ryan · The Manual

History, as the future will know it, is happening today on the web. And so it is the web that we must capture, package, and preserve for future generations to see who we are today.

Digital archivists run up against mismatched expectations:

But did you know that a large majority of web users think that when sharing their thoughts, images, and videos online they are going to be preserved in perpetuity? No matter how many licenses the general population clicks “Agree” to, or however many governing policies are developed that state the contrary, the millions of people sharing their content on websites still believe that there is an implicit accountability that should be upheld by the site owners.

Saturday, February 20th, 2016

Eternal 5D data storage could record the history of humankind

360 terabytes of data stored for over 13 billion years:

Coined as the ‘Superman memory crystal’, as the glass memory has been compared to the “memory crystals” used in the Superman films, the data is recorded via self-assembled nanostructures created in fused quartz. The information encoding is realised in five dimensions: the size and orientation in addition to the three dimensional position of these nanostructures.