Journal

2999 sparkline

Wednesday, September 20th, 2023

Trabaja remoto

August was a month of travels. You can press play on that month’s map to follow the journey.

But check out the map for September too because the travels continue. This time my adventures are confined to Europe.

I’m in Spain. Jessica and I flew into Madrid on Saturday. The next day we took a train ride across the Extremaduran landscape to Cáceres, our home for the week.

This is like the sequel to our Sicilian trip. We’re both working remotely. We just happen to do be doing it from a beautiful old town with amazing cuisine.

We’re in a nice apartment that—crucially—has good WiFi. It’s right on the main square, but it’s remarkably quiet.

There’s a time difference of one hour with Brighton. Fortunately everything in Spain happens at least an hour later than it does at home. Waking up. Lunch. Dinner. Everything is time-shifted so that I’m on the same schedule as my colleagues.

I swear I’m more productive working this way. Maybe it’s the knowledge that tapas of Iberican ham await me after work, but I’m getting a lot done this week.

And when the working week is done, the fun begins. Cáceres is hosting its annual Irish fleadh this weekend.

I’ve always wanted to go to it, but it’s quite a hassle to get here just for a weekend. Combining it with a week of remote work makes it more doable.

I’m already having a really nice time here and the tunes haven’t even started yet.

Tuesday, September 19th, 2023

Simon’s rule

I got a nice email from someone regarding my recent posts about performance on The Session. They said:

I hope this message finds you well. First and foremost, I want to express how impressed I am with the overall performance of https://thesession.org/. It’s a fantastic resource for music enthusiasts like me.

How nice! I responded, thanking them for the kind words.

They sent a follow-up clarification:

Awesome, anyway there was an issue in my message.

The line ‘It’s a fantastic resource for music enthusiasts like me.’ added by chatGPT and I didn’t notice.

I imagine this is what it feels like when you’re on a phone call with someone and towards the end of the call you hear a distinct flushing sound.

I wrote back and told them about Simon’s rule:

I will not publish anything that takes someone else longer to read than it took me to write.

That just feels so rude!

I think that’s a good rule.

Monday, September 18th, 2023

Secure tunes

The caching strategy for The Session that I wrote about is working a treat.

There are currently about 45,000 different tune settings on the site. One week after introducing the server-side caching, over 40,000 of those settings are already cached.

But even though it’s currently working well, I’m going to change the caching mechanism.

The eagle-eyed amongst you might have raised an eagle eyebrow when I described how the caching happens:

The first time anyone hits a tune page, the ABCs getting converted to SVGs as usual. But now there’s one additional step. I grab the generated markup and send it as an Ajax payload to an endpoint on my server. That endpoint stores the sheetmusic as a file in a cache.

I knew when I came up with this plan that there was a flaw. The endpoint that receives the markup via Ajax is accepting data from the client. That data could be faked by a malicious actor.

Sure, I’m doing a whole bunch of checks and sanitisation on the server, but there’s always going to be a way of working around that. You can never trust data sent from the client. I was kind of relying on security through obscurity …except it wasn’t even that obscure because I blogged about it.

So I’m switching over to using a headless browser to extract the sheetmusic. You may recall that I wrote:

I could spin up a headless browser, run the JavaScript and take a snapshot. But that’s a bit beyond my backend programming skills.

That’s still true. So I’m outsourcing the work to Browserless.

There’s a reason I didn’t go with that solution to begin with. Like I said, over 40,000 tune settings have already been cached. If I had used the Browserless API to do that work, it would’ve been quite pricey. But now that the flood is over and there’s a just a trickle of caching happening, Browserless is a reasonable option.

Anyway, that security hole has now been closed. Thank you to everyone who wrote in to let me know about it. Like I said, I was aware of it, but it was good to have it confirmed.

Funnily enough, the security lesson here is the same as my conclusion when talking about performance:

If that means shifting the work from the browser to the server, do it!

Friday, September 15th, 2023

Speedy tunes

Performance is a high priority for me with The Session. It needs to work for people all over the world using all kinds of devices.

My main strategy for ensuring good performance is to diligently apply progressive enhancement. The core content is available to any device that can render HTML.

To keep things performant, I’ve avoided as many assets (or, more accurately, liabilities) as possible. No uneccessary images. No superfluous JavaScript libraries. Not even any web fonts (gasp!). And definitely no third-party resources.

The pay-off is a speedy site. If you want to see lab data, run a page from The Session through lighthouse. To see field data, take a look at data from Chrome UX Report (Crux).

But the devil is in the details. Even though most pages on The Session are speedy, the outliers have bothered me for a while.

Take a typical tune page on the site. The data is delivered from the server as HTML, which loads nice and quick. That data includes the notes for the tune settings, written in ABC notation, a nice lightweight text format.

Then the enhancement happens. Using Paul Rosen’s brilliant abcjs JavaScript library, those ABCs are converted into SVG sheetmusic.

So on tune pages there’s an additional download for that JavaScript library. That’s not so bad though—I’m using a service worker to cache that file so there’ll only ever be one initial network request.

If a tune has just a few different versions, the page remains nice and zippy. But if a tune has lots of settings, the computation starts to add up. Converting all those settings from ABC to SVG starts to take a cumulative toll on the main thread.

I pondered ways to avoid that conversion step. Was there some way of pre-generating the SVGs on the server rather than doing it all on the client?

In theory, yes. I could spin up a headless browser, run the JavaScript and take a snapshot. But that’s a bit beyond my backend programming skills, so I’ve taken a slightly different approach.

The first time anyone hits a tune page, the ABCs getting converted to SVGs as usual. But now there’s one additional step. I grab the generated markup and send it as an Ajax payload to an endpoint on my server. That endpoint stores the sheetmusic as a file in a cache.

Next time someone hits that page, there’s a server-side check to see if the sheetmusic has been cached. If it has, send that down the wire embedded directly in the HTML.

The idea is that over time, most of the sheetmusic on the site will transition from being generated in the browser to being stored on the server.

So far it’s working out well.

Take a really popular tune like The Lark In The Morning. There are twenty settings, and each one has four parts. Previously that would have resulted in a few seconds of parsing and rendering time on the main thread. Now everything is delivered up-front.

I’m not out of the woods. A page like that with lots of sheetmusic and plenty of comments is going to have a hefty page weight and a large DOM size. I’ve still got a fair bit of main-thread work happening, but now the bulk of it is style and layout, whereas previously I had the JavaScript overhead on top of that.

I’ll keep working on it. But overall, the speed improvement is great. A typical tune page is now very speedy indeed.

It’s like a microcosm of web performance in general: respect your users’ time, network connection and battery life. If that means shifting the work from the browser to the server, do it!

Wednesday, September 13th, 2023

Multi-page web apps

I received this email recently:

Subject: multi-page web apps

Hi Jeremy,

lately I’ve been following you through videos and texts and I’m curious as to why you advocate the use of multi-page web apps and not single-page ones.

Perhaps you can refer me to some sources where your position and reasoning is evident?

Here’s the response I sent…

Hi,

You can find a lot of my reasoning laid out in this (short and free) online book I wrote called Resilient Web Design:

https://resilientwebdesign.com/

The short answer to your question is this: user experience.

The slightly longer answer…

For most use cases, a website (or multi-page app if you prefer) is going to provide the most robust experience for the most number of users. That’s because a user’s web browser takes care of most of the heavy lifting.

Navigating from one page to another? That’s taken care of with links.

Gathering information from a user to process on a server? That’s taken care of with forms.

This frees me up to concentrate on the content and the design without having to reinvent the wheels of links and form fields.

These (let’s call them) multi-page apps are stateless, and for most use cases that’s absolutely fine.

There are some cases where you’d want a state to persist across pages. Let’s say you’re playing a song, or a podcast episode. Ideally you’d want that player to continue seamlessly playing even as the user navigates around the site. In that situation, a single-page app would be a suitable architecture.

But that architecture comes at a cost. Now you’ve got stop the browser doing what it would normally do with links and forms. It’s up to you to recreate that functionality. And you can’t do it with HTML, a robust fault-tolerant declarative language. You need to reimplement all that functionality in JavaScript, a less tolerant, more brittle language.

Then you’ve got to ship all that code to the user before they can use your site. It might be JavaScript code you’ve written yourself or it might be a third-party library designed for building single-page apps. Either way, the user pays a download tax (and a parsing tax, and an execution tax). Whereas with links and forms, all of that functionality is pre-bundled into the user’s web browser.

So that’s my reasoning. At least nine times out of ten, a multi-page approach is leaner, more robust, and simpler.

Like I said, there are times when a single-page approach makes sense—it all comes down to whether state needs to be constantly preserved. But these use cases are the exceptions, not the rule.

That’s why I find the framing of your question a little concerning. It should be inverted. The default approach should be to assume a multi-page approach (which is the way the web works by default). Deciding to take a JavaScript-driven single-page approach should be the exception.

It’s kind of like when people ask, “Why don’t you have children?” Surely the decision to have a child should require deliberation and commitment, rather than the other way around.

When it comes to front-end development, I’m worried that we’ve reached a state where the more complex over-engineered approach is viewed as the default.

I may be committing a fundamental attribution error here, but I think that we’ve reached this point not because of any consideration for users, but rather because of how it makes us developers feel. Perhaps building an old-fashioned website that uses HTML for navigations feels too easy, like it’s beneath us. But building an “app” that requires JavaScript just to render text on a screen feels like real programming.

I hope I’m wrong. I hope that other developers will start to consider user experience first and foremost when making architectural decisions.

Anyway. That’s my answer. User experience.

Cheers,

Jeremy

Monday, September 11th, 2023

Performative performance

Web Summer Camp in Croatia finished with an interesting discussion. It was labelled a town-hall meeting, but it was more like an Oxford debating club.

Two speakers had two minutes each to speak for or against a particular statement. Their stances were assigned to them so they didn’t necessarily believe what they said.

One of the propositions was something like:

In the future, sustainable design will be as important as UX or performance.

That’s a tough one to argue against! But that’s what Sophia had to do.

She actually made a fairly compelling argument. She said that real impact isn’t going to come from individual websites changing their colour schemes. Real impact is going to come from making server farms run on renewable energy. She advocated for political action to change the system rather than having the responsibility heaped on the shoulders of the individuals making websites.

It’s a fair point. Much like the concept of a personal carbon footprint started life at BP to distract from corporate responsibility, perhaps we’re going to end up navel-gazing into our individual websites when we should be collectively lobbying for real change.

It’s akin to clicktivism—thinking you’re taking action by sharing something on social media, when real action requires hassling your political representative.

I’ve definitely seen some examples of performative sustainability on websites.

For example, at the start of this particular debate at Web Summer Camp we were shown a screenshot of a municipal website that has a toggle. The toggle supposedly enables a low-carbon mode. High resolution images are removed and for some reason the colour scheme goes grayscale. But even if those measures genuinely reduced energy consumption, it’s a bit late to enact them only after the toggle has been activated. Those hi-res images have already been downloaded by then.

Defaults matter. To be truly effective, the toggle needs to work the other way. Start in low-carbon mode, and only download the hi-res images when someone specifically requests them. (Hopefully browsers will implement prefers-reduced-data soon so that we can have our sustainable cake and eat it.)

Likewise I’ve seen statistics bandied about around the energy-savings that could be made if we used dark colour schemes. I’m sure the statistics are correct, but I’d like to see them presented side-by-side with, say, the energy impact of Google Tag Manager or React or any other wasteful dependencies that impact performance invisibly.

That’s the crux. Most of the important work around energy usage on websites is invisible. It’s the work done to not add more images, more JavaScript or more web fonts.

And it’s not just performance. I feel like the more important the work, the more likely it is to be invisible: privacy, security, accessibility …those matter enormously but you can’t see when a website is secure, or accessible, or not tracking you.

I suspect this is why those areas are all frustratingly under-resourced. Why pour time and effort into something you can’t point at?

Now that I think about it, this could explain the rise of web accessibility overlays. If you do the real work of actually making a website accessible, your work will be invisible. But if you slap an overlay on your website, it looks like you’re making a statement about how much you care about accessibility (even though the overlay is total shit and does more harm than good).

I suspect there might be a similar mindset at work when it comes to interface toggles for low-carbon mode. It might make you feel good. It might make you look good. But it’s a poor substitute for making your website carbon-neutral by default.

Friday, September 8th, 2023

Lovers in a dangerous time

Being in Croatia last week got me thinking about the country’s history.

I remember the break-up of Yugoslavia, but I was quite out of touch with the news for a while back in 1991. That’s because I was hitch-hiking and busking around Europe with my musical partner Polly from Cornwall. I had my mandolin, she had her fiddle.

We went from Ireland to England to France to Germany to Czechoslovakia (still a single country back then), to Austria to Italy, back to France, and back to England. A loop around Europe.

We set off on August 21st, 1991. The only reason I know the date is because I remember we had been to a gig in Cork the night before.

Sonic Youth were playing in Sir Henry’s (a great venue that no longer exists). The support band was a group from Seattle called Nirvana. I remember that some of my friends decided to skip the support band to stay in the pub next door until Sonic Youth came on because the pints were cheaper there.

By the time Polly and I got back from our travels, Nirvana were the biggest band on the planet. It all happened very quickly.

The same could be said for the situation in Yugoslavia.

I remember when we were stuck for a day at a petrol station in the alps trying to get from Austria to Italy. There was a bureau de change listing currency exchange rates. This was before the euro came in so there were lots of different currencies; pounds, francs, lira, deutsche marks. Then there was the listing for the Yugoslav dinar. It read:

  • We buy: 00.00
  • We sell: 00.00

That really struck me, seeing the situation summarised so clinically.

But what really got to me was an encounter in Vienna.

Polly and I did well in that city. On our first evening of busking, not only did we make some good money, but we also met a local folk singer. This young man very generously took us in and put us up in his flat.

At some point during our stay, we were on one of the city’s trams. That’s when we met another young couple who were on the road. Somehow there was always a connection between fellow travellers. I can’t remember who spoke to who first, but we bonded straight away.

It soon became clear that our situations were only superfically similar. This was a young couple deeply in love. One of them was Serbian. The other was Croatian. It wasn’t safe for either of them back where they used to call home.

I could return home at any point. I always knew that when I was sleeping rough, or struggling to make enough money to eat.

They couldn’t return. All they wanted was to be together somewhere safe. They started asking us about Ireland and England. “Do you think they’d give us asylum?” they asked with so much hope. It broke my heart to see their desperation, the pleading look in their eyes.

I felt so useless. I wished there was something I could’ve done for them.

I think about them a lot.

Wednesday, September 6th, 2023

Travels

He drew a deep breath. ‘Well, I’m back,’ he said.

I know how you feel, Samwise Gamgee.

I have returned from my travels—a week aboard the Queen Mary 2 crossing the Atlantic, followed by a weekend in New York, finishing with a week in Saint Augustine, Florida.

The Atlantic crossing was just as much fun as last time. In fact it was better because this time Jessica and I got to share the experience with our dear friends Dan and Sue.

There was dressing up! There was precarious ballet! There were waves! There were even some dolphins!

The truth is that this kind of Atlantic crossing is a bit like cosplaying a former age of travel. You get out of it what you put it into it. If you’re into LARPing as an Edwardian-era traveller, you’re going to have a good time.

We got very into it. Dressing up for dinner. Putting on a tux for the gala night. Donning masks for the masquerade evening.

Me and Jessica all dressed up wearing eye masks. Dan and Sue in wild outfits wearing eye masks.

It’s actually quite a practical way of travelling if you don’t mind being cut off from all digital communication for a week (this is a feature, not a bug). You adjust your clock by one hour most nights so that by the time you show up in New York, you’re on the right timezone with zero jetlag.

That was just as well because we had a packed weekend of activities in New York. By pure coincidence, two separate groups of friends were also in town from far away. We all met up and had a grand old time. Brunch in Tribeca; a John Cale concert in Prospect Park; the farmer’s market in Union Square; walking the high line …good times with good friends.

A brunch table with me and eight friends all smiling.

New York was hot, but not as hot as what followed in Florida. A week lazing about on Saint Augustine beach. I ate shrimp every single day. I regret nothing.

A sandy beach with gentle waves crashing under a blue sky with wisps of cloud.

We timed our exit just right. We flew out of Florida before the tropical storm hit. Then we landed in Gatwick right before the air-traffic control chaos erupted.

I had one day of rest before going back to work.

Well, I say “work”, but the first item in my calendar was speaking at Web Summer Camp in Croatia. Back to the airport.

The talk went well, and I got to attend a performance workshop by Harry. But best of all was the location. Opatija is an idyllic paradise. Imagine crossing a web conference with White Lotus, but in a good way. It felt like a continuation of Florida, but with more placid clear waters.

A beautiful old town interspersed with lush greenery sweeps down to a tranquil bay with blue/green water.

But now I’m really back. And fortunately the English weather is playing along by being unseasonably warm . It’s as if the warm temperatures are following me around. I like it.

Thursday, August 10th, 2023

Crossing

I managed to get out and play in trad sessions for four days straight: Sunday, Monday, Tuesday, and Wednesday.

Two fiddlers, a whistler and a guitarist playing around a pub table. A concertina player, banjo player, and fiddler playing round a pub table festooned with pints. A bodhrán player, guitarist and fiddler playing around a pub table with drinks on it. Two fiddlers and an accordion player gathered round a pub table, playing.

Today I rest. Or rather, I pack.

There won’t be any Irish music sessions in my calendar for at least the next two weeks. There won’t be much of anything in my calendar. I’m about to be incommunicado as I travel to Amerikay on the wide Atlantic ocean.

You might have seen that video that’s been doing the rounds about how cruise ships got so big. I’m not going on a cruise. I’m going on a crossing. That video ends with:

There are over 320 cruise ships sailing around the world right now. But there’s only one ocean liner left. The Queen Mary 2.

That’s going to be my mode of transportation and my home for the next week.

Then I’ll be in New York for a weekend, followed by most of a week lying on St. Augustine beach in Florida.

All of my sparklines are about to flatline for a while: no blog posts, no links, no notes. I won’t have any internet access on board the ship.

Time for me to switch off my work email. I thought about setting up an autoresponder to let people know that I won’t be replying to their email for a few weeks. But then I thought, how is that any different from how I normally respond to email?

Wednesday, August 9th, 2023

Automation

I just described prototype code as code to be thrown away. On that topic…

I’ve been observing how people are programming with large language models and I’ve seen a few trends.

The first thing that just about everyone agrees on is that the code produced by a generative tool is not fit for public consumption. At least not straight away. It definitely needs to be checked and tested. If you enjoy debugging and doing code reviews, this might be right up your street.

The other option is to not use these tools for production code at all. Instead use them for throwaway code. That could be prototyping. But it could also be the code for those annoying admin tasks that you don’t do very often.

Take content migration. Say you need to grab a data dump, do some operations on the data to transform it in some way, and then pipe the results into a new content management system.

That’s almost certainly something you’d want to automate with bespoke code. Once the content migration is done, the code can be thrown away.

Read Matt’s account of coding up his Braggoscope. The code needed to spider a thousand web pages, extract data from those pages, find similarities, and output the newly-structured data in a different format.

I’ve noticed that these are just the kind of tasks that large language models are pretty good at. In effect you’re training the tool on your own very specific data and getting it to do your drudge work for you.

To me, it feels right that the usefulness happens on your own machine. You don’t put the machine-generated code in front of other humans.