Journal tags: visual

8

sparkline

Indy web

It was Indie Web Camp Brighton on the weekend. After a day of thought-provoking discussions, I thoroughly enjoyed spending the second day tinkering on my website.

For a while now, I’ve wanted to add maps to my monthly archive pages (to accompany the calendar heatmaps I added at a previous Indie Web Camp). Whenever I post anything to my site—a blog post, a note, a link—it’s timestamped and geotagged. I thought it would be fun to expose that in a glanceable way. A map seems like the right medium for that, but I wanted to avoid the obvious route of dropping a load of pins on a map. Instead I was looking for something more like the maps in Indiana Jones films—a line drawn from place to place to show the movement over time.

I talked to Aaron about this and his advice was that a client-side JavaScript embedded map would be the easiest option. But that seemed like overkill to me. This map didn’t need to be pannable or zoomable; just glanceable. So I decided to see if how far I could get with a static map. I timeboxed two hours for it.

After two hours, I admitted defeat.

I was able to find the kind of static maps I wanted from Mapbox—I’m already using them for my check-ins. I could even add a polyline, which is exactly what I wanted. But instead of passing latitude and longitude co-ordinates for the points on the polyline, the docs explain that I needed to provide …cur ominous thunder and lightning… The Encoded Polyline Algorithm Format.

Go to that link. I’ll wait.

Did you read through the eleven steps of instructions? Did you also think it was a piss take?

  1. Take the initial signed value.
  2. Multiply it by 1e5.
  3. Convert that decimal value to binary.
  4. Left-shift the binary value one bit.
  5. If the original decimal value is negative, invert this encoding.
  6. Break the binary value out into 5-bit chunks.
  7. Place the 5-bit chunks into reverse order.
  8. OR each value with 0x20 if another bit chunk follows.
  9. Convert each value to decimal.
  10. Add 63 to each value.
  11. Convert each value to its ASCII equivalent.

This was way beyond my brain’s pay grade. But surely someone else had written the code I needed? I did some Duck Duck Going and found a piece of PHP code to do the encoding. It didn’t work. I Ducked Ducked and Went some more. I found a different piece of PHP code. That didn’t work either.

At this point, my allotted time was up. If I wanted to have something to demo by the end of the day, I needed to switch gears. So I did.

I used Leaflet.js to create the maps I wanted using client-side JavaScript. Here’s the JavaScript code I wrote.

It waits until the page has finished loading, then it searches for any instances of the h-geo microformat (a way of encoding latitude and longitude coordinates in HTML). If there are three or more, it generates a script element to pull in the Leaflet library, and a corresponding style element. Then it draws the map with the polyline on it. I ended up using Stamen’s beautiful watercolour map tiles.

Had some fun at Indie Web Camp Brighton on the weekend messing around with @Stamen’s lovely watercolour map tiles. (I was trying to create Indiana Jones style travel maps for my site …a different kind of Indy web.)

That’s what I demoed at the end of the day.

But I wasn’t happy with it.

Sure, it looked good, but displaying the map required requests for a script, a style sheet, and multiple map tiles. I made sure that it didn’t hold up the loading of the rest of the page, but it still felt wasteful.

So after Indie Web Camp, I went back to investigate static maps again. This time I did finally manage to find some PHP code for encoding lat/lon coordinates into a polyline that worked. Finally I was able to construct URLs for a static map image that displays a line connecting multiple points with a line.

I’ve put this maps on any of the archive pages that also have calendar heat maps. Some examples:

If you go back much further than that, the maps start to trail off. That’s because I wasn’t geotagging everything from the start.

I’m pretty happy with the final results. It’s certainly far more responsible from a performance point of view. Oh, and I’ve also got the maps inside a picture element so that I can swap out the tiles if you switch to dark mode.

It’s a shame that I can’t use the lovely Stamen watercolour tiles for these static maps though.

Request mapping

The Request Map Generator is a terrific tool. It’s made by Simon Hearne and uses the WebPageTest API.

You pop in a URL, it fetches the page and maps out all the subsequent requests in a nifty interactive diagram of circles, showing how many requests third-party scripts are themselves generating. I’ve found it to be a very effective way of showing the impact of third-party scripts to people who aren’t interested in looking at waterfall diagrams.

I was wondering… Wouldn’t it be great if this were built into browsers?

We already have a “Network” tab in our developer tools. The purpose of this tab is to show requests coming in. The browser already has all the information it needs to make a diagram of requests in the same that the request map generator does.

In Firefox, there’s a little clock icon in the bottom left corner of the “Network” tab. Clicking that shows a pie-chart view of requests. That’s useful, but I’d love it if there were the option to also see the connected circles that the request map generator shows.

Just a thought.

Timelines of the web

Recreating the original WorldWideWeb browser was an exercise in digital archeology. With a working NeXT machine in the room, Kimberly was able to examine the source code for the first every browser and discover a treasure trove within. Like this gem in HTUtils.h:

#define TCP_PORT 80 /* Allocated to http by Jon Postel/ISI 24-Jan-92 */

Sure enough, by June of 1992 port 80 was documented as being officially assigned to the World Wide Web (Gopher got port 70). Jean-François Groff—who worked on the World Wide Web project with Tim Berners-Lee—told us that this was a moment they were very pleased about. It felt like this project of theirs was going places.

Jean-François also told us that the WorldWideWeb browser/editor was kind of like an advanced prototype. The idea was to get something up and running as quickly as possible. Well, the NeXT operating system had a very robust Text Object, so the path of least resistance for Tim Berners-Lee was to take the existing word-processing software and build a hypertext component on top of it. Likewise, instead of creating a brand new format, he used the existing SGML format and added one new piece: linking with A tags.

So the WorldWideWeb application was kind of like a word processor and document viewer mashed up with hypertext. Ted Nelson complains to this day that the original sin of the web was that it borrowed this page-based metaphor. But Nelson’s Project Xanadu, originally proposed in 1974 wouldn’t become a working reality until 2014—a gap of forty years. Whereas Tim Berners-Lee proposed his system in March 1989 and had working code within a year. There’s something to be said for being pragmatic and working with what you’ve got.

The web was also a mashup of ideas. Hypertext existed long before the web—Ted Nelson coined the term in 1963. There were conferences and academic discussions devoted to hypertext and hypermedia. But almost all the existing hypertext systems—including Tim Berners-Lee’s own ENQUIRE system from the early 80s—were confined to a local machine. Meanwhile networked computers were changing everything. First there was the ARPANET, then the internet. Tim Berners-Lee’s ambitious plan was to mash up hypertext with networks.

Going into our recreation of WorldWideWeb at CERN, I knew I wanted to convey this historical context somehow.

The World Wide Web officially celebrates its 30th birthday in March of this year. It’s kind of an arbitrary date: it’s the anniversary of the publication of Information Management: A Proposal. Perhaps a more accurate date would be the day the first website—and first web server—went online. But still. Let’s roll with this date of March 12, 1989. I thought it would be interesting not only to look at what’s happened between 1989 and 2019, but also to look at what happened between 1959 and 1989.

So now I’ve got two time cones that converge in the middle: 1959 – 1989 and 1989 – 2019. For the first time period, I made categories of influences: formats, hypertext, networks, and computing. For the second time period, I catalogued notable results: browsers, servers, and the evolution of HTML.

I did a little bit of sketching and quickly realised that these converging timelines could be represented somewhat like particle collisions. Once I had that idea in my head, I knew how I would be spending my time during the hack week.

Rather than jumping straight into the collider visualisation, I took some time to make a solid foundation to build on. I wanted to be sure that the timeline itself would be understable even if it were, say, viewed in the first ever web browser.

Progressive enhancement. Marking up (and styling) an interactive timeline that looks good in a modern browser and still works in the first ever web browser.

I marked up each timeline as an ordered list of h-events:

<li class="h-event y1968">
  <a href="https://en.wikipedia.org/wiki/NLS_%28computer_system%29" class="u-url">
    <time class="dt-start" datetime="1968-12-09">1968</time>
    <abbr class="p-name" title="oN-Line System">NLS</abbr>
  </a>
</li>

With the markup in place, I could concentrate on making it look halfway decent. For small screens, the layout is very basic—just a series of lists. When the screen gets wide enough, I lay those lists out horzontally one on top of the other. In this view, you can more easily see when events coincide. For example, ENQUIRE, Usenet, and Smalltalk all happen in 1980. But the real beauty comes when the screen is wide enough to display everthing at once. You can see how an explosion of activity in the early 90s. In 1994 alone, we get the release of Netscape Navigator, the creation of HTTPS, and the launch of Amazon.com.

The whole thing is powered by CSS transforms and positioning. Each year on a timeline has its own class that gets moved to the correct chronological point using calc(). I wanted to use translateX() but I couldn’t get the maths to work for that, so I had use plain ol’ left and right:

.y1968 {
  left: calc((1968 - 1959) * (100%/30) - 5em);
}

For events before 1989, it’s the distance of the event from 1959. For events after 1989, it’s the distance of the event from 2019:

.y2014 {
  right: calc((2019 - 2014) * (100%/30) - 5em);
}

(Each h-event has a width of 5em so that’s where the extra bit at the end comes from.)

I had to do some tweaking for legibility: bunches of events happening around the same time period needed to be separated out so that they didn’t overlap too much.

As a finishing touch, I added a few little transitions when the page loaded so that the timeline fans out from its centre point.

Et voilà!

Progressive enhancement. Marking up (and styling) an interactive timeline that looks good in a modern browser and still works in the first ever web browser.

I fiddled with the content a bit after peppering Robert Cailliau with questions over lunch. And I got some very valuable feedback from Jean-François. Some examples he provided:

1971: Unix man pages, one of the first instances of writing documents with a markup language that is interpreted live by a parser before being presented to the user.

1980: Usenet News, because it was THE everyday discussion medium by the time we created the web technology, and the Web first embraced news as a built-in information resource, then various platforms built on the web rendered it obsolete.

1982: Literary Machines, Ted Nelson’s book which was on our desk at all times

I really, really enjoyed building this “collider” timeline. It was a chance for me to smash together my excitement for web history with my enjoyment of using the raw materials of the web; HTML and CSS in this case.

The timeline pales in comparison to the achievement of the rest of the team in recreating the WorldWideWeb application but I was just glad to be able to contribute a little something to the project.

Hello WorldWideWeb.

Month maps

One of the topics I enjoy discussing at Indie Web Camps is how we can use design to display activity over time on personal websites. That’s how I ended up with sparklines on my site—it was the a direct result of a discussion at Indie Web Camp Nuremberg a year ago:

During the discussion at Indie Web Camp, we started looking at how silos design their profile pages to see what we could learn from them. Looking at my Twitter profile, my Instagram profile, my Untappd profile, or just about any other profile, it’s a mixture of bio and stream, with the addition of stats showing activity on the site—signs of life.

Perhaps the most interesting visual example of my activity over time is on my Github profile. Halfway down the page there’s a calendar heatmap that uses colour to indicate the amount of activity. What I find interesting is that it’s using two axes of time over a year: days of the month across the X axis and days of the week down the Y axis.

I wanted to try something similar, but showing activity by time of day down the Y axis. A month of activity feels like the right range to display, so I set about adding a calendar heatmap to monthly archives. I already had the data I needed—timestamps of posts. That’s what I was already using to display sparklines. I wrote some code to loop over those timestamps and organise them by day and by hour. Then I spit out a table with days for the columns and clumps of hours for the rows.

Calendar heatmap on Dribbble

I’m using colour (well, different shades of grey) to indicate the relative amounts of activity, but I decided to use size as well. So it’s also a bubble chart.

It doesn’t work very elegantly on small screens: the table is clipped horizontally and can be swiped left and right. Ideally the visualisation itself would change to accommodate smaller screens.

Still, I kind of like the end result. Here’s last month’s activity on my site. Here’s the same time period ten years ago. I’ve also added month heatmaps to the monthly archives for my journal, links, and notes. They’re kind of like an expanded view of the sparklines that are shown with each month.

From one year ago, here’s the daily distribution of

And then here’s the the daily distribution of everything in that month all together.

I realise that the data being displayed is probably only of interest to me, but then, that’s one of the perks of having your own website—you can do whatever you feel like.

Animating

I’ve noticed a few nice examples of motion design on the web lately.

The Cloud Four gang recently redesigned their site, including a nice little animation on the home page.

Malcolm Gladwell has a new podcast called Revisionist History. The website for the podcast is quite lovely. Each episode is illustrated with an animated image. Lovely!

If you want to see some swishy animations triggered by navigation, the waaark websites has them a-plenty. Personally I find the scroll-triggered animations on internal pages too much to take (I have yet to find an example of scrolljacking that doesn’t infuriate me). But the homepage illustrations have some lovely subtle movement.

When it comes to subtlety in animation, my favourite example comes from Charlotte. She recently refactored the homepage of the website for the Leading Design conference. It originally featured one big background image. Switching over to SVG saved a lot of bandwidth. But what I really love is that the shapes in the background are now moving …ever so gently.

It’s like gazing at a slow-motion lava lamp of geometry.

A new website for dConstruct 2014

dConstruct 2014 has a new website. Huzzah!

When I announced the original website two months ago, I was very, very excited about the line-up, but I was less excited about the design of the site itself. To be honest, it was a somewhat rushed affair. It did the job but it didn’t have much pizzazz. I had some design direction—colour, typography, texture—courtesty of Mikey, but I didn’t push it to do anything very interesting.

dConstruct original 320 dConstruct original 600 dConstruct original 768

So Mikey took some time to iterate and revise, and he came up with a gorgeous new design. I think this does a much better job of capturing the spirit of dConstruct.

As well as a revised colour palette and lusher textures, there was also opportunity to do something quite playful in the masthead. Making sites for our own projects always presents a nice opportunity to try out some whacky stuff that we might not get a chance to do on client work.

In this case, the plan was to play with the theme of this year’s dConstruct—Living With The Network—and use it as part of the visual design, literally networking up parts of the interface.

It was a nice chance for me to play around with canvas. But I didn’t dive into code straight away. I had a think about how I could add this an enhancement to the responsive layout.

My plan was to generate a canvas element under the existing elements in the header using z-index to keep them separated while maintaining the appearance of having everything connected up.

Sketching before coding

It worked out pretty well. But I wanted to push it further. How about making it an interactive element that responds to the user?

I know, I know. It’s very silly and frankly a bit wanky, but y’know, it felt like it would be nice and playful.

I had no idea how to do it though. At an internal code review here at Clearleft, I demoed what I had so far and asked for advice. The general consensus was that I should probably be using SVG rather than canvas for making interactive graphical elements. They’re probably right, but I distinctly remember learning about hit detection and mouse events in canvas during Seb’s excellent Creative JS workshop.

So I stuck with canvas and fiddled around with numbers until I got to something that felt lke it reacted nicely to hover events (or touch/clicks if hover isn’t available …or even if it is). requestAnimationFrame was a godsend when it came to getting smooth animations.

Have a play with it. It’s hard to miss. It’s not exactly a subtle easter egg.

The content of the site remains much the same. While I was disatisfied with the original visual design of the site, I’m still pretty chuffed with the copy.

One small change I made was to give the code of conduct its own page (and expand on it a bit). Previously it was included with terms and conditions but there was a good chance that it could’ve been overlooked there.

Anyway, I hope you like the new site. I think Mikey did a terrific job with the design and it was a lot of fun to put together …especially the silly wanky bit. The only slight disadvantage is that the page weight comes in slightly larger than the previous design. But I’ll keep optimising to see if I can shave off some bytes here and there.

Device testing dConstruct Device testing dConstruct

Oh, and you might notice one significant change on the home page. In addition to the speakers that are currently listed, there’s an addendum that reads “…and more”. That’s because the line-up for this year’s dConstruct, awesome as it is, is not yet complete. It’s going to get even better.

If you don’t have your ticket to this year’s dConstruct yet, what are you waiting for?

See you on September 5th.

Sound and vision

Every creation of Tony Wilson’s was labelled with the letters followed by a number. The first poster was FAC1. was FAC51.

The Joy Division album was FACT10. The album artwork was designed by Peter Saville. The words “Unknown Pleasures” don’t appear on the cover. Neither do the words “Joy Division”. Instead, the cover contains a series of 100 lines representing pulses from —thanks to . It was a groundbreaking piece of graphic design. Its beauty lies in its simplicity: a two-dimensional representation of raw data.

That was almost thirty years ago. This week Radiohead released the video for the song House of Cards from the album In Rainbows …except it isn’t really a video at all. It wasn’t shot on film or video. It is a three-dimensional representation of raw data.

You can play with the data visualisation, altering it while the song plays. You can even download the raw data. You are not just allowed to play around with the data, you are encouraged to do so. There’s a YouTube group for aggregating the results.

Suddenly every other music video seems very flat and passive. I’m reminded of a prescient passage from Douglas Adams’s essay How to Stop Worrying and Learn to Love the Internet:

I expect that history will show “normal” mainstream twentieth century media to be the aberration in all this.

Please, miss, you mean they could only just sit there and watch? They couldn’t do anything? Didn’t everybody feel terribly isolated or alienated or ignored?

Yes, child, that’s why they all went mad. Before the Restoration.

What was the Restoration again, please, miss?

The end of the twentieth century, child. When we started to get interactivity back.

Visual

Jason Kottke points to a a beautiful collection of literary maps by Stefanie Posavec. Meanwhile over on A List Apart there’s a new article by Wilson Miner called Accessible Data Visualization with Web Standards. He shows some of the nifty CSS tricks he used on EveryBlock. The end results are very impressive though I don’t necessarily agree with the assertion that when what we’re really building is navigation, tables are an awkward and often clumsy tool for the job — I still think that tables would have been not just semantically correct but also malleable enough with CSS. But I’m nitpicking. It’s a great article.

There was oodles of data visualisation goodness at BarCamp Brighton 2 courtesy of Robin Harrison. Check out the links from his presentation. As well as the Tufte favourite of Napoleon’s Russian invasion map, he mentioned Florence Nightingale’s map of mortality causes which reminded me of the cholera map of London. That is the subject of the newest book from Steven Johnson called The Ghost Map: The Story of London’s Most Terrifying Epidemic—and How It Changed Science, Cities, and the Modern World.

There are some fine examples of data visualisation over at the New York Times:

Some more data visualisation: