Tags: huffduffer

47

sparkline

Thank you, jQuery

I turned Huffduffer into a progressive web app recently. It was already running on HTTPS so I didn’t have much to do. One manifest file and one basic Service Worker did the trick.

Getting the 'add to home screen' prompt for https://huffduffer.com/ on Android Chrome.

I also did a bit of spring cleaning, refactoring some CSS. The site dates to 2008 so there’s plenty in there that I would do very differently today. Still, considering the age of the code, I wasn’t cursing my past self too much.

After that, I decided to refactor the JavaScript too. There, I had a clear goal: could I remove the dependency on jQuery?

It turned out to be pretty straightforward. I was able to bring my total JavaScript file size down to 3K (gzipped). Pretty much everything I was doing in jQuery could be just as easily accomplished with DOM methods like addEventListener and querySelectorAll, and objects like XMLHttpRequest and classList.

Of course, the reason why half of those handy helpers exist is because of jQuery. Certainly in the case of querySelector and querySelectorAll, jQuery blazed a trail for browsers and standards bodies to pave. In some cases, like animation, the jQuery-led solutions ended up in CSS instead of JavaScript, but the story was the same: developers used the heck of jQuery and browser makers paid attention to that. This is something that Jack spoke about at Render Conf a little while back.

Brian once said of PhoneGap that its ultimate purpose is to cease to exist. I think of jQuery in a similar way.

jQuery turned ten years old this year, and jQuery version 3.0 was just released. Congratulations, jQuery! You have served the web well.

Bookmarklets

Someone at Clearleft asked me a question recently about making bookmarklets. I have a bit of experience in that department. As well as making a bookmarklet for adding links to my own site, there’s the Huffduffer bookmarklet that’s been chugging away since 2008.

I told them that there are basically two approaches:

  1. Have the bookmarklet pop open a new browser window at your service, passing in the URL of the current page. Then do all the heavy lifting on your server.
  2. Have the bookmarklet inject JavaScript to analyse and edit the DOM of the document in the current browser window. All the heavy lifting is done directly in client-side JavaScript.

I favour the first approach. Partly that’s because it makes it easier to update the functionality. As you improve your server-side script, the bookmarklet functionality gets better automatically. But also, if your server-side script doesn’t do its magic, you can always fall back to letting the end user fill in the details.

Here’s an example…

When you click the Huffduffer bookmarklet, it pops open this URL:

https://huffduffer.com/add?page=…

…with that page parameter filled in with whatever page you currently have open. Let’s say I’ve got this page currently open in my browser:

https://adactio.com/journal/6786

If I press the Huffduffer bookmarklet, that will spawn a new window with this URL:

https://huffduffer.com/add?page=https://adactio.com/journal/6786

And that’s all it does. Now it’s up to that page on Huffduffer to figure out what to do with the URL it has been given. In this case, it makes a CURL request to figure out what to use as a title, what to use as a description, what audio file to use, etc. If it can’t figure that out, I can always fill in those fields myself by hand.

I could’ve chosen to get at that information by injecting JavaScript directly into the page open in the browser. But that’s somewhat invasive.

Brian Donohue wrote on Ev’s blog a while back about one of the problems with that approach. Sites that—quite correctly—have a strict Content Security Policy will object to having arbitrary JavaScript injected into their documents.

But remember this only applies to some bookmarklets. If a bookmarklet just spawns a new window—like Huffduffer’s—then there’s no problem. That approach to bookmarklets was dismissed with this justification:

The crux of the issue for bookmarklets is that web authors can control the origin of the JavaScript, network calls, and CSS, all of which are necessary for any non-trivial bookmarklet.

Citation needed. I submit that Huffduffer and Instapaper provide very similar services: “listen later” and “read later”. Both use cases could be described as “non-trivial”. But only one of the bookmarklets works on sites with strict CSPs.

Time and time again, I see over-engineered technical solutions that are built with the justification that “this problem is very complex therefore the solution needs to be complex” (yes, I am talking about web thangs that rely on complex JavaScript). In my experience, it’s exactly the opposite way around. The more complex the problem, the more important it is to solve it in the simplest way possible. It’s the only way of making sure the solution is resilient to unexpected scenarios.

The situation with bookmarklets is a perfect example. It’s not just an issue with strict Content Security Policies either. I’ve seen JavaScript-injecting bookmarklets fail because someone has set their browser cookie preferences to only accept cookies from the originating server.

Bookmarklets are not dead. They may, however, be pining for the fjords. Nobody has a figured out a way to get bookmarklets to work on mobile. Now that might well be a death sentence.

Spamduffing

Running The Session and Huffduffer is immensely rewarding …most of the time. There are occasions when the actions of a few bad apples make it a real pain in the bum.

Yes, I’m talking about SEO spammers.

Huffduffer tends to get it worse than The Session, but even then it’s fairly manageable—just a sign-up or two here or there. This weekend though, there was a veritable spam tsunami. I was up late on Friday night playing a constant game of whack-a-mole with thousands of spam postings by newly-created accounts. (I’m afraid I inadvertently may have deleted some genuine new accounts in the trawl; if you signed up for Huffduffer last Friday and can’t access your account now, I’m really, really sorry.)

Normally these spam SEO accounts would have some pattern to them—either they’d be from the same block of IP addresses or they’d have similar emails. But these all looked different enough to thwart any quick fixes. I knew I’d be spending my Saturday writing some spam-blocking code.

Most “social” websites have a similar sign-up flow: you fill in a form with your details (including your email address), and then you have to go to your email client to click a link to verify that you are indeed who you claim to be. The cynical side of me thinks that this is mostly to verify that you providing a genuine email address so that the site can send you marketing crap.

Neither Huffduffer nor The Session includes that second step of confirming your email address. The only reason for providing your email address is so that you can reset your password if you ever forget it.

I’ve always felt that making a new user break out of the sign-up flow to go check their email was a bit shit. It also strikes me as following the same logic as CAPTCHAs (which I hate): “Because of the bad actions of a minority, we’re going to punish the majority by making them prove to us that they’re human.” It’s such a machine-centric way of thinking.

But after the splurge of spam on Huffduffer, I figured I’d have no choice but to introduce that extra step. Just as I was about to start coding, I thought to myself “No, this is wrong. There must be another way.”

I thought a bit more about the problem. The issue wasn’t so much about spam sign-ups per se. Like I said, there’s always been a steady trickle and it isn’t too onerous to find them and delete them. The problem was the sheer volume of spam posts in a short space of time.

I ended up writing some different code with this logic:

  1. When someone posts to Huffduffer, check to see if they’ve posted at least ten items in the past;
  2. If they have, grab the timestamps for the last ten posts;
  3. Calculate the cumulative elapsed time between those ten posts;
  4. If it’s less than 100 seconds (i.e. an average of one post every ten seconds), delete the user …and delete everything they’ve ever posted.

It worked. I watched as new spam sign-ups began to hammer the site with spam postings …only to self-destruct when they hit the critical mass of posts over time.

I’m still getting SEO spammers signing up but now they’re back to manageable levels. I’m glad that I didn’t end up having to punish genuine new users of Huffduffer for the actions of a few SEO marketing bottom-feeders.

Huffduffing for podcasters

I was pointed to this discussion thread which is talking about how to make podcast episodes findable for services like Huffduffer.

The logic behind Huffduffer’s bookmarklet goes something like this…

  1. Find any a elements that have href values ending in “.mp3” or “.m4a”.
  2. If there’s just one audio on the page, use that.
  3. If there are multiple audio, offer a list to the user and have them choose.

If that doesn’t work…

  1. Look for a link element with a rel value of “enclosure”.
  2. Look for a meta element property value of “og:audio”.
  3. Look for audio elements and grab either the src attribute of the element itself, or the src attribute of any source elements within the audio element.

If that doesn’t work…

  1. Try to find a link to an RSS feed (a link that looks like “rss” or “feed” or “atom”).
  2. If there is a feed, parse that for enclosure elements and present that list to the user.

That covers 80-90% of use cases. There are still situations where the actual audio file for a podcast episode is heavily obfuscated—either with clickjacking JavaScript “download” links, or links that point to a redirection to the actual file.

If you have a podcast and you want your episodes to be sharable and huffduffable, you have a few options:

Have a link to the audio file for the episode somewhere on the page, something like:

<a href="/path/to/file.mp3">download</a>

That’s the simplest option. If you’re hosting with Soundcloud, this is pretty much impossible to accomplish: they deliberately obfuscate and time-limit the audio file, even if you want it to be downloadable (that “download” link literally only allows a user to download that file in that moment).

If you don’t want a visible link on the page, you could use metadata in the head of your document. Either:

<link rel="enclosure" href="/path/to/file.mp3">

Or:

<meta property="og:audio" content="/path/to/file.mp3">

And if you want to encourage people to huffduff an episode of your podcast, you can also include a “huffduff it” link, like this:

<a href="https://huffduffer.com/add?page=referrer">huffduff it</a>

You can also use ?page=referer—that misspelling has become canonised thanks to HTTP.

There you go, my podcasting friends. However you decide to do it, I hope you’ll make your episodes sharable.

100 words 078

I’ve noticed lately that my experience of films is lasting long after leaving the cinema. I end up reading opinion pieces and listening to podcasts about the film for days or even weeks afterwards.

Interstellar, Ex Machina, Mad Max: Fury Road …I enjoyed each of them in the cinema, and then I enjoyed thinking about them again by huffduffing related material to catch up on.

Sometimes I find myself doing it with other media too. I finish a book, and then listen to reckons about it afterwards.

I guess this is the water cooler effect, but extended to the internet.

Huffduffing video

You know what would be awesome? If you could huffduff the audio from videos on YouTube, Vimeo, and other video hosting sites.

To give you an example, A List Apart recently started running online events and once the events are over, they pop ‘em onto YouTube. Now, I’m not saying I don’t want to look at those lovely faces for an hour, but if truth be told, it’s the audio that I’m really interested in.

In the past, my only recourse would’ve been to pester the good people at A List Apart to export audio as well as video, in much the same way as I’ve pestered conference organisers in the past:

I wish conference organisers would export the audio of any talks that they’re publishing as video. Creating the sound file at that point is a simple one-click step. But once the videos are up online—be it on YouTube or Vimeo—it’s a lot, lot harder to get just the audio.

Not everyone wants to watch video. In fact, I bet there are plenty of people who listen to conference talks by opening the video in a separate tab so they can listen to it while they do something else.

Some people have come up with clever workarounds to get the audio track from videos into Huffduffer but they’re fairly convoluted.

Until now!

The brilliant Ryan Barrett has just launched huffduff-video:

He has created a bookmarklet you can use whenever you’re on a YouTube or Vimeo page that you want to huffduff. It works a treat—I’ve already used to huffduff that A List Apart event and a talk by Matt Jones.

It takes a little while to do the initial conversion but you can just leave the pop-up window open while it works its magic. I’ve incorporated it into the Huffduffer bookmarklet now too. So if you’re on a YouTube or Vimeo page, you can hit the usual bookmarklet and it’ll route you through Ryan’s clever creation.

That’s makes two fantastic pieces of software from Ryan that have improved my online life immeasurably: first Bridgy and now huffduff-video. So nifty!

Soundcloudbusting

Matt wrote a great article called Ten Years of Podcasting: Fighting Human Nature (although I’m not entirely sure why he put it on Ev’s site instead of—or in addition to—his own). It’s a look back at the history of podcasting, and how it has grown out of its nerdy origins to become more of a mainstream activity. In it, he kindly gives a shout-out to Huffduffer:

…a way to make piecemeal meta-podcasts on the fly built up from random shows (here’s my feed).

Matt has written about how he uses Huffduffer before: a quick introduction to adding your Huffduffer feed to Instacast. It’s equally straightforward with Overcast, and most other iOS podcast apps.

If you use the iOS app Workflow, there’s a nifty tutorial for extracting the audio from YouTube videos, posting the audio to Dropbox, and subscribing in Huffduffer. I’m letting the side down somewhat though: Huffduffer’s API is currently read-only, but it would so much more powerful if you could post from other apps. I need to wrap my head around OAuth to do this. I was hoping to do OwnYourGram-style API with IndieAuth and micropub (once your Huffduffer profile has your website URL, and that URL has rel="me" links to OAuth providers like Twitter, Flickr, or Github, all the pieces should be in place), but alas IndieAuth only works on a domain or subdomain basis so /username URLs are out.

Anyway, back to Matt’s article about podcasting. He writes:

Personally, I like it when new podcasts use Soundcloud for their hosting, because on a desktop computer it means I can easily dip into their archives and play random episodes, scrub to certain segments and get a feel for the show before I subscribe.

It’s true that if you’re sitting in front of a desktop computer, Soundcloud is a great way to listen to an audio file there and then. But it’s a lousy way to host a podcast.

The whole point of podcasting is that it’s time-shifted. You get to listen to the audio you want, when you want. The whole point of Soundcloud is that you listen to audio then and there. That’s great if you’re a musician, looking to make sure that people can’t make copies of your music, but it’s terrible if you’re a podcaster.

To be fair, Soundcloud’s primary audience is still musicians, rather than podcasters, so it makes sense for them to prioritise that use-case. But still, they really go out of their way to obfuscate the actual audio file. Even if the publisher has checked the right box to allow users to download the audio file, the result is a very literal interpretation of that: you can download the file, but you can’t copy the URL and paste it into, say, an app for listening later (and you certainly can’t huffduff it).

Case in point: Matt finishes his article with:

If you don’t have time to read the above, it’s available as a 14min audio file…

That audio file is hosted on Soundcloud. You can listen to it there, or you can listen to it through the embedded player on the article itself. But that’s it. You can’t take it with you. You can’t listen to it later. You can’t, for example, listen to it in your car, even though as Matt says:

…for most Americans, killing time listening to podcasts in a car is a great place.

If you can figure out a way to get at Matt’s audio file (and maybe even huffduff it), I’d be much obliged.

Like Merlin says:

Overcast and Huffduffer

Marco Arment has released his podcast app Overcast for iOS—you can read his introduction to the app.

It plays nicely with Huffduffer. If you want to listen to any Huffduffer feed in Overcast, it’s a straightforward process.

Step 1

Overcast Add podcast
Launch the app and tap on “add a podcast”. Then tap on “Add URL” in the top right.

Step 2

Add URL Huffduffer URL
Enter the Huffduffer URL e.g. huffduffer.com/adactio.

Step 3

All Podcast episode
That’s it! You’re all set.

It’s pretty straightforward to subscribe to Huffduffer URLs in other iOS apps too. Matt has written up the process of using Huffduffer and Instacast. And there’s also a write-up of using Huffduffer and Downcast.

Huffduff up and up

I had a nice Skype chat with Stan Alcorn yesterday all about Huffduffer, online sharing of audio, and all things podcasty and radioish. I’m sure I must have talked his ear off.

Stan was asking about numbers for Huffduffer’s user base and activity. I have to admit that I’ve got zero analytics running on the site. To be honest, I’m okay with that—one of the perks of having a personal project is that only metric that really matters is your own satisfaction. But I told Stan I’d run some quick database queries to get some feeling for Huffduffer’s usage patterns. Here’s what I found…

There are 5,862 people signed up to Huffduffer.

About 150,919 items have been huffduffed. But those aren’t unique files. The total number of distinct files that have been huffduffed is 5,972. That means that, on average, an audio file is huffduffed around 26 times. And the average user has huffduffed around 30 items. But neither of those distributions would be evenly distributed; they’d be power-law distributions rather than bell curves. For example, the most popular file was huffduffed 329 times.

Looking at the amount of items huffduffed each year, there’s a pleasing upward trend.

1st year 7,382
2nd year 19,080
3rd year 23,403
4th year 31,808
5th year 41,514

I was pleasantly surprised by this. I would’ve assumed that Huffduffer usage would be more of a steady-state affair, but it looks like the site is getting used a bit more with each passing year (the site is currently in its sixth(!) year).

Not that any of that really matters. I built Huffduffer to scratch my own itch. I huffduff an average of 411 audio files each year. So even if nobody else used Huffduffer, it would still provide plenty of value to me.

Like I was saying to Stan, the biggest strength and the biggest weakness of audio—as opposed to text or video—is that you can listen to it while your doing other things. For some people, car journeys are the perfect podcast time. For others, it might be doing the dishes or train journeys. For me, it’s the walk to and from work each day—it takes about 35 minutes each way, and I catch up on my Huffduffer feed during that time.

Jessica and I will often listen to some spoken word audio in the background during dinner—usually something quite radio-y like Radiolab, or NPR stories. Yesterday, we were catching up with Aleks’s BBC documentary series, The Digital Human. It was the episode about voice.

Imagine my surprise when I heard the voice of Stan Alcorn. What a co-inky-dink!

Listen to dConstruct 2013

If you didn’t make it to this year’s dConstruct, at least your ears can catch up. If you did make it to this year’s dConstruct, your ears can experience the fun all over again.

The audio is available, is what I’m saying here.

  1. Amber Case: Ambient Location and the Future of the Interface
  2. Luke Wroblewski: Infinite Inputs
  3. Nicole Sullivan: Don’t Feed the Trolls
  4. Simone Rebaudengo: Great; things are connected, but what will they actually talk about?
  5. Sarah Angliss: Tech and the Uncanny
  6. Keren Elazari: The Heroes and Anti-heroes of the Information Age
  7. Maciej Cegłowski: Fan is a Tool-Using Animal
  8. Dan Williams: Unexpected Item In The Bagging Area
  9. Adam Buxton: Is My Laptop Ruining My Life?

The audio is on Huffduffer for your listening pleasure. If you’d like to take it with you on the go, here’s the RSS feed—just pop that into your podcasting/catching software of choice.

While you’re at it, this might be a nice opportunity to go back and explore the dConstruct archive where you can find every talk from every dConstruct from 2005 to 2013. That’s 70 talks, or about 46 hours of listening pleasure.

Share and enjoy!

Responsive audio out

AudioGO

You never forget your first DMCA takedown notice. In my case it was the Perfect Pitch incident, in which an incompetent business was sending out automatic takedown notices to Google for any website that contained a combination of the words Burge Pitch Torrent. That situation, which affected The Session, was resolved with an apology from the offending party.

Now I’ve received my second DMCA takedown notice. Or rather, my hosting company has. This time it involves Huffduffer.

When I created Huffduffer, I thought about offering hosting for audio files. One of the reasons I decided not to is because of the potential legal pitfalls. As it stands, Huffduffer is pretty much entirely text—it just links to audio files elsewhere on the web. That’s basically what an RSS enclosure is: another form of hypertext.

Linking is simply the act of pointing to a resource, and apart from a few extreme cases, it isn’t illegal.

Now it could be argued that pointing to an audio file on another site through a Flash player (or HTML5 audio element) is more like hotlinking with an img element than regular linking through an a element. The legal status of hotlinking isn’t quite as clear cut as plain ol’ linking, as explained on the Chilling Effects site:

When people complain about inline images, they are most often complaining about web pages that include graphics from external sources. The legal status of inlining images without permission has not been settled.

So the situation with inline audio is similarly murky.

Here’s the threatening email that was sent to the hosting company:

Notice of Copyright Infringement. {Our ref: [$#121809/228552]}

Sender: Robert Nichol
AudioGO Ltd
The Home of BBC Audiobooks
St James House, The Square, Lower Bristol Road
Bath
BA2 3BH
Phone number not available

Recipient: Rackspace Hosting

RE: Copyright Infringement.

This notice complies with the Digital Millennium Copyright Act (17 U.S.C. §512(c)(3))

I, Robert Nichol, swear under penalty of perjury that I am authorised to act on behalf of AudioGO Ltd, the owner(s) of the copyright or of an exclusive licence in the work(s) The Moving Finger by Agatha Christie BBC Audio.

It has come to my attention that the website huffduffer.com is engaged in the electronic distribution of copies of these works. It is my good faith belief that the use of these works in this manner is not authorised by the copyright owner, his agent or the law. This is in clear violation of United States, European Union, and International copyright law, and I now request that you expeditiously remove this material from huffduffer.com, or block or disable access to it, as required under both US and EU law.

The works are The Moving Finger by Agatha Christie BBC Audio.

The following URLs identify the infringing files and the means to locate them.

https://huffduffer.com/TimesPastOTR/68635 (IP: 67.192.7.4)

The information in this notice is accurate and I request that you expeditiously remove or block or disable access to all the infringing material or the entire site.

/Robert Nichol/
Robert Nichol

Wednesday April 18, 2012

Initially, my hosting company rebutted Robert Nichol’s claim but he’s not letting it go. He insists that the offending URL be removed or he will get the servers taken offline. So now I’ve been asked by my host to delete the relevant page on Huffduffer.

But the question of whether audio hotlinking counts as copyright infringement is a moot point in this case…

Go to the page in question. If you try to play the audio file, or click on the “download” link, you will find yourself at a 404 page. Whatever infringing material may have once been located at the end of the link is long gone …and yet AudioGO Ltd are still insisting that the Huffduffer page be removed!

Just to be clear about this, Robert Nichol is using the Digital Millennium Copyright Act—and claiming “good faith belief” while doing so—to have a site removed from the web that mentions the name of a work by his client, and yet that site not only doesn’t host any infringing material, it doesn’t even link to any infringing material!

It seems that, once again, the DMCA is being used in a scattergun approach like a machine-gun in the hands of a child. There could be serious repercussions for Robert Nichol in abusing a piece of legislation in this way.

If I were to remove the page in question, even though it just contains linkrot, it would set a dangerous precedent. It would mean that if someone else—like you, for instance—were to create a page that contains the text “Agatha Christie — The Moving Finger” while pointing to a dead link …well, your hosting company might find themselves slapped with a takedown notice.

In that situation, you wouldn’t be able to copy and paste this markup into your blog, Tumblr, Facebook, or Google+ page:

<a href="http://dc436.4shared.com/img/892695085/b3c907d3/dlink__2Fdownload_2FaKhc8m9b_3Ftsid_3D20120318-72646-b4a59ab0/preview.mp3">Agatha Christie — The Moving Finger</a>

Remember: that link does not point to any infringing material. It points to nothing but a 404 page. There’s absolutely no way that you could have your site taken offline for pointing to a file that doesn’t exist, right?

That would be crazy.

One moment

I use my walk to and from work every day as an opportunity to catch up on my Huffduffer podcast. Today I started listening to a talk I’ve really been looking forward to. It’s a Long Now seminar called Universal Access To All Knowledge by one of my heroes: Brewster Kahle, founder of The Internet Archive.

Brewster Kahle: Universal Access to All Knowledge — The Long Now on Huffduffer

As expected, it’s an excellent talk. I caught the start of it on my walk in to work this morning and I picked up where I left off on my walk home this evening. In fact, I deliberately didn’t get the bus home—despite the cold weather—so that I’d get plenty of listening done.

Round about the 23 minute mark he starts talking about Open Library, the fantastic project that George worked on to provide a web page for every book. He describes how it works as a lending library where an electronic version of a book can be checked out by one person at a time:

You can click on: hey! there’s this HTML5 For Web Designers. We bought this book—we bought this book from a publisher such that we could lend it. So you can say “Oh, I want to borrow this book” and it says “Oh, it’s checked out.” Darn! And you can add it to your list and remind yourself to go and get it some other time.

Holy crap! Did Brewster Kahle just use my book to demonstrate Open Library‽

It literally stopped me in my tracks. I stopped walking and stared at my phone, gobsmacked.

It was a very surreal moment. It was also a very happy moment.

Now I’m documenting that moment—and I don’t just mean on a third-party service like Twitter or Facebook. I want to be able to revisit that moment in the future so I’m documenting it at my own URL …though I’m very happy that the Internet Archive will also have a copy.

Command lines

Here’s a nifty piece of in-browser behaviour…

Fire up Chrome and in the address field type “huffduffer.com” followed by a space and BOOM! …the address field transforms into a site-specific search field: “Search Huffduffer.”

That’s thanks to this XML file that’s been on Huffduffer since day one. It’s the most straightforward example of the OpenSearch format.

It’s the same format that powers Firefox’s search providers. If you visit Huffduffer with Firefox and click in the browser’s search field, you’ll see the option to “Add Huffduffer” to the list of search engines.

I can’t imagine many people will actually do that but still, no harm in providing the option.

Another option that’s been added to Huffduffer recently (thanks to Andy’s hacking) is the ability to access the API through YQL. Feel free to open up the console and have a play around.

Lazy loading on Huffduffer

If you look at my profile page on Huffduffer, this is what you’ll see:

  • my details,
  • what I’ve huffduffed,
  • links to subscribe to my podcast and
  • my tag cloud.

That’s the core information for that page, preceded by a header with site navigation and followed by a footer with some additional links.

Because I’ve provided a URL with my details, there’s some extra information displayed in the sidebar:

It’s a similar situation if you look at a piece of audio I’ve huffduffed. The core information is:

  • all the details about the audio (title, description, tags),
  • who else has huffduffed this,
  • possibly-related items and
  • links to share and embed the audio.

In addition, because I’ve used a machine tagbook:author=cory doctorow—the sidebar contains:

  • related articles from The Guardian,
  • sales information from The New York Times,
  • books on Amazon.

In both cases, this supporting information isn’t essential; it’s just nice to have.

There’s a potential performance problem though. Because this extra information is coming from third-party services—and despite the fact that I’m doing some caching—it could delay the display of the whole page. So I took some time on the weekend to adjust the architecture a little bit. Now the extra information is requested with Ajax once the core information has already loaded. This is .

Now I’ve introduced a dependency on JavaScript, which is far from ideal, but because this is just “nice to have” information, I think it’s okay if it isn’t seen by a subset of visitors.

In fact, because this extra lazy-loaded stuff takes up valuable screen real estate, I think it might be acceptable to only serve it up to visitors who have the screen real estate to accommodate it:

if ($(document).width() > 640) {
// do lazy loading here
}

So if you load my profile on a small screen, you won’t see my latest tweets or my Last.fm recommendations. Likewise if you look at something I’ve huffduffed that’s tagged with music:artist=radiohead you won’t see information from Last.fm, pictures from Flickr or albums on Amazon unless you load the page with a wide enough viewport.

Now it could be that the real issue here isn’t viewport size, but connection speed …in which case I’m making the classic error of conflating small screen size with limited bandwidth. A script like Boomerang, which attempts to measure a user’s connection speed, could be very handy in this situation.

Lazy loading is the new fold

I was chatting with James about the implications that lazy loading could have for earlier phases of the design process: wireframing, page description diagrams, and so on.

Traditionally, you’ve got only two choices when judging what content to include: either something is on the page or it isn’t. You can use hierarchy, position and contrast to emphasise or de-emphasise the content but fundamentally, it’s a binary choice. But with conditional lazy-loading there’s a third option: include some content if the user’s circumstances warrant it.

Once again, Luke’s Mobile First approach is a useful thought experiment. It can help prioritise which elements are core content and which could be lazy-loaded:

Mobile devices require software development teams to focus on only the most important data and actions in an application. There simply isn’t room in a 320 by 480 pixel screen for extraneous, unnecessary elements. You have to prioritize.

So when a team designs mobile first, the end result is an experience focused on the key tasks users want to accomplish without the extraneous detours and general interface debris that litter today’s desktop-accessed Web sites. That’s good user experience and good for business.

Sometimes there are political reasons for wanting the “extraneous detours and general interface debris.” Lazy loading for large-screen users could be the least worst option in that situation. Semantically speaking, the kind of content that might be marked up in an aside element might be a good candidate for lazy loading …if the viewport is large enough.

I have a feeling that we’re going to be seeing a lot more of lazy loading as the responsive web design revolution rolls on. Used judiciously, it could provide plenty of performance benefits. But if it’s taken too far, lazy-loading could be disastrous, resulting in sites that rely on JavaScript to load their core content—I’m looking at you, Twitter.

The Huffduffer Hotline

After seeing (and hearing) what Brian was doing at History Hack Day, I decided I’d have to have a play with Tropo. Like Twilio, it’s a service that allows you to build voice-activated apps that you call up and talk to.

The API is pretty straightforward and it seems like there’s quite a lot that you can do as a developer before upgrading to a paid account. They’ll also host your code for you, and you have a choice of scripting languages.

At the most basic level, you can send text-to-voice messages:

say("Hello world")

But you can also give it audio files to play:

say(http://example.com/helloworld.mp3)

Huffduffer has the locations of thousands of audio files, so I thought a voice interface onto Huffduffer’s collection would be fun.

Call +1 202 600 8751 in the US, +44 2035 142722 in the UK, or use Skype. When the nice digital man on the other end picks up the phone and asks you want you want to hear, you can respond with “what’s new”, “what’s popular”, or say a tag like music, science, history, politics, technology, etc.

The script then fetches the latest files with that tag and will go through them with you one by one, asking “Would you like to hear… ?” followed by the title. If you don’t like the sound of it, just say no. When you find something you do want to hear, say yes. It will then start playing and you will be listening to a podcast down a telephone line.

Audioboo / searching huffduffer.com audio by phone on Huffduffer

I call it the Huffduffer Hotline. The code is on Github. If you fancy playing around with the Tropo API and want to use Huffduffer’s links to audio files, go ahead. You should find everything you need through the Huffduffer API.

If people find the Huffduffer Hotline useful or just plain fun, I’ll upgrade from the developer account to get better performance. Let me know your thoughts on Get Satisfaction.

Tagdiving

Speaking of URLs

We were having a discussion in the Clearleft office recently about that perennially-tricky navigation pivot: tags. Specifically, we were discussing how to represent the interface for combinatorial tags i.e. displaying results of items that have been tagged with tag A and tag B.

I realised that this was functionality that I wasn’t even offering on Huffduffer, so I set to work on implementing it. I decided to dodge the interface question completely by only offering this functionality through the browser address bar. As a fairly niche, power-user feature, I’m not sure it warrants valuable interface real estate—though I may revisit that challenge later.

I can’t use the + symbol as a tag separator because Huffduffer allows spaces in tags (and spaces are converted to pluses in URLs), so I’ve settled on commas instead.

For example, there are plenty of items tagged with “music” (/tags/music) and plenty of items tagged with “science” (/tags/science) but there’s only a handful of items tagged with both “music” and “science” (/tags/music,science).

This being Huffduffer, where just about every page has corresponding JSON, RSS and Atom representations, you can also subscribe to the podcast of everything tagged with both “music” and “science” (/tags/music,science/rss).

There’s an OR operator as well; the vertical pipe symbol. You can view the 60 items tagged with “html5”, the 14 items tagged with “css3”, or the 66 items tagged with either “html5” or “css3” (/tags/html5|css3).

Wait a minute …66 items? But 60 plus 14 equals 74, not 66!

The discrepancy can be explained by the 8 items tagged with both “css3” and “html5” (/tags/html5,css3).

The AND and OR operators can be combined, so you can find items tagged with either “science” or “religion” that are also tagged with “politics” (/tags/science|religion,politics).

While it’s fun to do this in the browser’s address bar, I think the real power is in the way that the corresponding podcast allows you to subscribe to precisely-tailored content. Find just the right combination of tags, click on the RSS link, and you’re basically telling iTunes to automatically download audio whenever there’s something new that matches criteria like:

I’m sure there are plenty of intriguing combinations out there. Now I can use Huffduffer’s URLs to go spelunking for audio gems at the most promising intersections of tags.

Tweaking Huffduffer

Because I was so busy, the two-year anniversary of Huffduffer passed unnoticed back in October. Two years! It’s hard to believe. It seems like just yesterday that I launched it. It’s been ticking along nicely for all that time and I’ve been tweaking it whenever I get the chance.

I recently added oEmbed support. I’m very impressed with the humble little format. It’s basically a unified API onto the multiple embed codes provided by so many websites. You request a URL from an endpoint such as https://huffduffer.com/oembed?url= and you get back a JSON (or XML) file with the details of the HTML you need to embed the content—video, photo, whatever. Something like https://huffduffer.com/oembed?url=https://huffduffer.com/adactio/32454

YouTube, Flickr, Vimeo and a whole host of other sites support oEmbed and the Embedly service provides easy access to all of them. Now Huffduffer is listed amongst the 160 oEmbed providers supported by Embedly. Maybe if I make the right ritual sacrifices, perhaps Huffduffer players might start showing up in New Twitter: it uses a combination of oEmbed and a whitelist to display third-party content in the side panel.

I made some tweaks to the front-end of Huffduffer recently too. For starters, you might notice that the body copy font size has been bumped up from fourteen pixels to sixteen. While fourteen pixels is perfectly fine for Helvetica or Georgia, it’s just that little bit too small for .

While I was in there messing around with the CSS, I took the opportunity to tweak the small screen rendering.

Huffduffer on iOS

For a start, I changed the way that the media queries are executed. Instead of beginning with the wide-screen “desktop” layout as the default and then undoing the widths and floats for smaller screens, I’m now using the same technique I’ve tried out here on adactio.com: begin with a linear layout-less flow and only add widths and floats within media query blocks. That way, mobile devices that don’t support media queries will still get the linearised view.

The elephant in the room is, once again, Internet Explorer (below version nine, anyway). While I can quite merrily say “screw ‘em” here on adactio.com, I need to make more of an effort for Huffduffer. So I split up my CSS into two files: a global.css file that contains all the typography and colour rules, and layout.css that contains a default wide-screen “desktop” view followed by media queries for narrower widths. This is how I’m calling both stylesheets:

<link rel="stylesheet" href="/css/global.css" media="all">
<link rel="stylesheet" href="/css/layout.css" media="all and (min-width: 30em)">
<!--[if lt IE 9]>
<link rel="stylesheet" href="/css/layout.css" media="all">
<![endif]-->

See how the layout.css file is being called twice? Once for browsers that support media queries (with a browser width wider than thirty ems) and again for Internet Explorer less than version nine.

Mobile devices that don’t support media queries or conditional comments will never load the layout.css file. Browsers that do support media queries, be they mobile or desktop, will only execute layout.css if the viewport is at least thirty ems wide. Legacy versions of Internet Explorer will always load layout.css because of the conditional comment. It’s entirely possible that Windows Mobile 7 will also load layout.css because the browser is currently using an IE7 codebase (Trident 3.1, to be precise). Screw ‘em …at least until Microsoft bring out an update.

The disadvantage of this technique is that my CSS is now split up into two separate files. I’d much rather keep HTTP requests to a minimum by having just one style sheet, but I think that, in this case, the reward in cross-browser compatibility is worth the expense of that extra hit.

While I was testing the changes, I noticed something interesting on my iPod Touch when I was at the Clearleft office, where we have the stereo connected to the WiFi network. The most recent iOS update allows you to stream directly from your device to your stereo or television. What I didn’t realise was that this is true of any media, including HTML5 video and audio content in a web page. Nice!

Huffduffer on iOS

Postscript to Space

One of the mailing lists I subscribe to is the Brighton Speculative Fiction group. If I rightly recall, I signed up whilst drunk at a party I had gatecrashed in Kemptown.

What? Like it’s never happened to you. I suppose you’ve never woken up the morning after the night before, clutching your aching head and moaning “Oh man, I hope I didn’t edit any wikis last night!”

Anyway.

The Brighton Speculative Fiction group meets regularly in the excellent Basketmaker’s Arms to talk sci-fi and swap books. My copy of is making the rounds while I’ve snagged a copy of one of ’s earliest works, . It reads like an novel, imagining what it would have been like if the space race had been led from the UK rather than the US.

Early on the book, a character explains that peculiarly British word “”:

Good lord, don’t you know that word? It goes back to the War, and means any long-haired scientific type with a slide-rule in his vest pocket.

That reminded me of the thoroughly enjoyable book Backroom Boys by Francis Spufford, filled with stories of post-war British innovation: everything from “spitfires in space” rocketry ambitions through to the creation of and .

But when Clarke published Prelude To Space in 1953, the idea of Britain leading the charge into space wasn’t a far-fetched flight of fancy. If anything, it was a straightforward linear extrapolation. Before the PR battle of the superpowers kicked off with Sputnik, America had shown no interest in spaceflight, much less putting men on the moon.

I know this, not just because Arthur C. Clarke mentions it in the foreword, but also because of the first episode of the Space Dog podcast which features an interview with Arthur C. Clarke gleaned from The Science Fiction Oral History Association, wherein he talks about Prelude To Space.

Space Dog Podcast Episode One on Huffduffer

In fact, I’ve been huffduffing a host of Arthur C. Clarke-related material lately. The motherlode is this three-way discussion with Clarke, and on , technology, and the future of mankind. They discuss the idea of without explicitly calling it that—this was recorded long before coined the term.

Arthur C. Clarke, Alvin Toffler, and Margaret Mead on Man’s Future on Huffduffer

Listening to this on my iPod on my walk into work, I had a pleasant tingle of recognition when Alvin Toffler, author of , was introduced as a consultant to the Institute For The Future …the organisation that provided the location for a latter-day gathering of web-enabled boffins: Science Hack Day San Francisco.

Listening

Whenever I take a trip somewhere—like Copenhagen, for example—it’s a good opportunity to catch up on what I’ve been huffduffing. Trains, planes and buses are the killer apps of personal podcasting. In many ways, Huffduffer becomes more useful the further away you are from a computer and an internet connection.

I didn’t get the chance to see Mark speak at this year’s Web Directions @media in London, but now that I’ve listened to his talk on Designing Grid Systems, I’m cursing the two-track format of the conference and the fact that I couldn’t be in two places at once. This talk is superb; one of the best presentations I’ve ever heard. It’s got a fantastic long-zoom perspective and completely crystalises and clarifies the fundamental problem with the approach taken to most web design today: canvas in, rather than content out. Do yourself a favour and huffduff this today.

Mark Boulton — Designing grid systems on Huffduffer

The audio from the hot topics panel I moderated at the same conference is also available for your huffduffing pleasure and you can read a transcript of the panel right here in the articles section of my site.

Web Directions @media: Jeremy Keith — Hot Topics on Huffduffer

Matt Ridley’s usual area of expertise is in evolutionary biology but lately he’s turned his Darwinian gaze to the evolution of man-made systems. His talk on How Prosperity Evolves, based on his latest book The Rational Optimist is a fascinating look at how ideas have sex with each other.

How Prosperity Evolves on Huffduffer

Two new podcasts showed up on my radar recently. One is The Box from web designer Tim Van Damme. Episode 1 features a short, snappy interview with Neven Mrgan, one of the creators of the iPhone game The Incident. Expect more short snappy interviews to follow.

The Box - Episode 1: Neven Mrgan on Huffduffer

The other new podcast is called The Incomparable, a chat show about sci-fi and geek culture. The first episode, We’ll Always Have Zeppelins began with a discussion of China Miéville’s The City and The City (which I’m planning to read now) and finished with a look at Cory Doctorow’s For The Win. While I was sitting in a chair in the sky listening to the discussion, I remembered that I had downloaded the ePub version from ManyBooks.net. I began reading it on my iPod Touch and now I’m hooked.

We’ll Always Have Zeppelins — The Incomparable on Huffduffer

So that’s just some of the stuff I’ve been listening to:

…and I haven’t even mentioned the prolific audio output of Dan’s excellent 5by5 network.

If audio isn’t your bag, then you might enjoy the beautiful-looking videos from Put This On, a web series about dressing like a grown-up from the ever-brilliant Jesse Thorn and Adam Lisagor. You’re welcome.