Tags: tool

I just described prototype code as code to be thrown away. On that topic…

I’ve been observing how people are programming with large language models and I’ve seen a few trends.

The first thing that just about everyone agrees on is that the code produced by a generative tool is not fit for public consumption. At least not straight away. It definitely needs to be checked and tested. If you enjoy debugging and doing code reviews, this might be right up your street.

The other option is to not use these tools for production code at all. Instead use them for throwaway code. That could be prototyping. But it could also be the code for those annoying admin tasks that you don’t do very often.

Take content migration. Say you need to grab a data dump, do some operations on the data to transform it in some way, and then pipe the results into a new content management system.

That’s almost certainly something you’d want to automate with bespoke code. Once the content migration is done, the code can be thrown away.

Read Matt’s account of coding up his Braggoscope. The code needed to spider a thousand web pages, extract data from those pages, find similarities, and output the newly-structured data in a different format.

I’ve noticed that these are just the kind of tasks that large language models are pretty good at. In effect you’re training the tool on your own very specific data and getting it to do your drudge work for you.

To me, it feels right that the usefulness happens on your own machine. You don’t put the machine-generated code in front of other humans.

• Download the slides

Hello. I was asked by the Design Business Association to talk to you today about “web3 and AI.”

I’d like to explain what those terms mean.

“Web3”

Let’s start with “web3.” Fortunately I don’t have to come up with an explanation for this term because my friend Heydon Pickering has recorded a video entitled “what is web 3.0?”

What is web trois point nought?

Web uno dot zilch was/is a system of interconnected documents traversible by hyperlink.

However, web deux full stop nowt was/is a system of interconnected documents traversible by hyperlink.

On the other hand, web drei dot zilch is a system of interconnected documents traversible by hyperlink.

Should you wish to upgrade to web three point uno, expect a system of interconnected documents traversible by hyperlink.

If we ever get to web noventa y cinco, you can bet your sweet @rse, it will be a system of interconnected documents traversible by f*!king hyperlink.

There you have it. “Web3” is a completely meaningless term. If someone uses it, they’re probably trying to sell you something.

If you ask for a definition, you’ll get a response like “something something decentralisation something something blockchain.”

As soon as someone mentions blockchain, you can tune out. It’s the classic example of a solution in search of a problem (although it’s still early days; it’s only been …more than a decade).

I can give you a defintion of what a blockchain is. A blockchain is multiple copies of a spreadsheet.

I find it useful to be able to do mental substitions like that when it comes to buzzwords. Like, remember when everyone was talking about “the cloud” but no one was asking what that actually meant? Well, by mentally substituting “the cloud” with “someone else’s server” you get a much better handle on the buzzword.

So, with “web3” out of the way, we can move onto the next buzzword. AI.

“AI”

The letters A and I are supposed to stand for Artificial Intelligence. It’s a term that’s almost as old as digital computing itself. It goes right back to the 1950s.

These days we’d use the term Artificial General Intelligence—AGI—to talk about that original vision of making computers as smart as people.

Vision is the right term here, because AGI remains a thought experiment. This is the realm of super intelligence: world-ending AI overlords; paperclip maximisers; Roko’s basilisk.

These are all fascinating thought experiments but they’re in the same arena as speculative technologies like faster-than-light travel or time travel. I’m happy to talk about any of those theoretically-possible topics, but that’s not what we’re here to talk about today.

When you hear about AI today, you’re probably hearing about specific technologies like large language models and machine learning.

Let’s take a look at large language models and their visual counterparts, diffusion models. They both work in the same way. You take a metric shit ton of data and you assign each one to a token. So you’ve got a numeric token that represents a bigger item: a phrase in a piece of text, or an object in an image.

The author Ted Chiang used a really good analogy to describe this process when he said ChatGPT is like a blurry JPEG of the web.

Just as image formats like JPG use compression to smush image data, these models use compression to smush data into tokens.

By the way, the GPT part of ChatGPT stands for Generative Pre-trained Transformer. The pre-training is that metric shit ton of data I mentioned. The generative part is about combining—or transforming—tokens in a way that should make probabalistic sense.

Terminology

Here’s some more terminology that comes up when people talk about these tools.

Overfitting. This is when the output produced by a generative pre-trained transformer is too close to the original data that fed the model. Another word for overfitting is plagiarism.

Hallucinations. People use this word when the output produced by a generative pre-trained transformer strays too far from reality. Another word for this is lying. Although the truth is that all of the output is a form of hallucination—that’s the generative part. Sometimes the output happens to match objective reality. Sometimes it doesn’t.

What about the term AI itself? Is there a more accurate term we could be using?

I’m going to quote Ted Chiang again. He proposes that a more accurate term is applied statistics. I like that. It points to the probabalistic nature of these tools: take an enormous amount of inputs, then generate something that feels similar based on implied correlations.

I like to think of “AI” as a kind of advanced autocomplete. I don’t say that to denigrate it. Quite the opposite. Autocomplete is something that appears mundane on the surface but has an incredible amount of complexity underneath: real-time parsing of input, a massive database of existing language, and on-the-fly predictions of the next most suitable word. Large language models do the same thing, but on a bigger scale.

What’s it good for?

So what is AI good for? Or rather, what is a language or diffusion model good for? Or what is applied statistics or advanced autocomplete good for?

Transformation. These tools are really good at transforming between formats. Text to speech. Speech to text. Text to images. Long form to short form. Short form to long form.

Think of transcripts. Summaries. These are smart uses of this kind of technology.

Coding, to a certain extent, can be considered a form of transformation. I’ve written books on programming, and I always advise people to first write out what they want in English. Then translate each line of English into the programming language. Large language models do a pretty good job of this right now, but you still need a knowledgable programmer to check the output for errors—there will be errors.

(As for long-form and short-form text transformations, the end game may be an internet filled with large language models endlessly converting our written communications.)

When it comes to the design process, these tools are good at quantity, not quality. If you need to generate some lorem ipsum placeholder text—or images—go for it.

What they won’t help with is problem definition. And it turns out that understanding and defining the problem is the really hard part of the design process.

Use these tools for inputs, not outputs. I would never publish the output of one of these tools publicly. But I might use one of these tools at the beginning of the process to get over the blank page. If I want to get a bunch of mediocre ideas out of the way quickly, these tools can help.

There’s an older definition of the intialism AI that dairy farmers would be familiar with, when “the AI man” would visit the farm. In that context, AI stands for artificial insemination. Perhaps thats also a more helpful definition of AI tools in the design process.

But, like I said, the outputs are not for public release. For one thing, the generated outputs aren’t automatically copyrighted. That’s only fair. Technically, it’s not your work. It is quite literally derivative.

Why all the hype?

Everything I’ve described here is potentially useful in some circumstances, but not Earth-shattering. So what’s with all the hype?

Venture capital. With this model of funding, belief in a technology’s future matters more than the technology’s actual future.

We’ve already seen this in action with self-driving cars, the metaverse, and cryptobollocks. Reality never matched the over-inflated expectations but that made no difference to the people profiting from the investments in those technologies (as long as they make sure to get out in time).

By the way, have you noticed how all your crypto spam has been replaced by AI spam? Your spam folder is a good gauge of what’s hot in venture capital circles right now.

The hype around AI is benefiting from a namespace clash. Remember, AI as in applied statistics or advanced autocomplete has nothing in common with AI as in Artificial General Intelligence. But because the same term is applied to both, the AI hype machine can piggyback on the AGI discourse.

It’s as if we decided to call self-driving cars “time machines”— we’d be debating the ethics of time travel as though it were plausible.

For a refreshing counter-example, take a look at what Apple is saying about AI. Or rather, what it isn’t saying. In the most recent Apple keynote, the term AI wasn’t mentioned once.

Technology blogger Om Malik wrote:

One of the most noticeable aspects of the keynote was the distinct lack of mention of AI or ChatGPT.

I think this was a missed marketing opportunity for the company.

I couldn’t disagree more. Apple is using machine learning a-plenty: facial recognition, categorising your photos, and more. But instead of over-inflating that work with the term AI, they stick to the more descriptive term of machine learning.

I think this will pay off when the inevitable hype crash comes. Other companies, that have tied their value to the mast of AI will see their stock prices tank. But because Apple is not associating themselves with that term, they’re well positioned to ride out that crash.

What should you do?

Alright, it’s time for me to wrap this up with some practical words of advice.

Beware of the Law of the instrument. You know the one: when all you have is a hammer, everything looks a nail. There’s a corollary to that: when the market is investing heavily in hammers, everyone’s going to try to convince you that the world is full of nails. See if you can instead cultivate a genuine sense of nailspotting.

It should ring alarm bells if you find yourself thinking “how can I find a use for this technology?” Rather, spend your time figuring out what problem you’re trying to solve and only then evaluate which technologies might help you.

Never make any decision out of fear. FOMO—Fear Of Missing Out—has been weaponised again and again, by crypto, by “web3”, by “AI”.

The message is always the same: “don’t get left behind!”

“It’s inevitable!” they cry. But you know what’s genuinely inevitable? Climate change. So maybe focus your energy there.

Links

I’ll leave you with some links.

I highly recommend you get a copy of the book, The Intelligence Illusion by Baldur Bjarnason. You can find it at illusion.baldurbjarnason.com

The subtitle is “a practical guide to the business risks of generative AI.” It doesn’t get into philosophical debates on potential future advances. Instead it concentrates squarely on the pros and cons of using these tools in your business today. It’s backed up by tons of research with copious amounts of footnotes and citations if you want to dive deeper into any of the issues.

If you don’t have time to read the whole book, Baldur has also created a kind of cheat sheet. Go to needtoknow.fyi and you can a one-page list of cards to help you become an AI bullshit detector.

I keep track of interesting developments in this space on my own website, tagging with “machine learning” at adactio.com/tags/machinelearning

Thank you very much for your time today.

I’m sure you’ve heard the law of the instrument: when all you have is a hammer, everything looks like a nail.

There’s another side to it. If you’re selling hammers, you’ll depict a world full of nails.

Recent hammers include cryptobollocks and virtual reality. It wasn’t enough for blockchains and the metaverse to be potentially useful for some situations; they staked their reputations on being utterly transformative, disrupting absolutely every facet of life.

This kind of hype is a terrible strategy in the long-term. But if you can convince enough people in the short term, you can make a killing on the stock market. In truth, the technology itself is superfluous. It’s the hype that matters. And if the hype is over-inflated enough, you can even get your critics to do your work for you, broadcasting their fears about these supposedly world-changing technologies.

You’d think we’d learn. If an industry cries wolf enough times, surely we’d become less trusting of extraordinary claims. But the tech industry continues to cry wolf—or rather, “hammer!”—at regular intervals.

The latest hammer is machine learning, usually—incorrectly—referred to as Artificial Intelligence. What makes this hype cycle particularly infuriating is that there are genuine use cases. There are some nails for this hammer. They’re just not as plentiful as the breathless hype—both positive and negative—would have you believe.

When I was hosting the DiBi conference last week, there was a little section on generative “AI” tools. Matt Garbutt covered the visual side, demoing tools like Midjourney. Scott Salisbury covered the text side, showing how you can generate code. Afterwards we had a panel discussion.

During the panel I asked some fairly straightforward questions that nobody could answer. Who owns the input (the data used by these generative tools)? Who owns the output?

On the whole, it stayed quite grounded and mercifully free of hyperbole. Both speakers were treating the current crop of technologies as tools. Everyone agreed we were on the hype cycle, probably the peak of inflated expectations, looking forward to reaching the plateau of productivity.

Scott explicitly warned people off using generative tools for production code. His advice was to stick to side projects for now.

Matt took a closer look at where these tools could fit into your day-to-day design work. Mostly it was pretty sensible, except when he suggested that there could be any merit to using these tools as a replacement for user testing. That’s a terrible idea. A classic hammer/nail mismatch.

I think I moderated the panel reasonably well, but I have one regret. I wish I had first read Baldur Bjarnason’s new book, The Intelligence Illusion. I started reading it on the train journey back from Edinburgh but it would have been perfect for the panel.

The Intelligence Illusion is very level-headed. It is neither pro- nor anti-AI. Instead it takes a pragmatic look at both the benefits and the risks of using these tools in your business.

It has excellent advice for spotting genuine nails. For example:

Generative AI has impressive capabilities for converting and modifying seemingly unstructured data, such as prose, images, and audio. Using these tools for this purpose has less copyright risk, fewer legal risks, and is less error prone than using it to generate original output.

Think about transcripts of videos or podcasts—an excellent use of this technology. As Baldur puts it:

The safest and, probably, the most productive way to use generative AI is to not use it as generative AI. Instead, use it to explain, convert, or modify.

He also says:

Prefer internal tools over externally-facing chatbots.

That chimes with what I’ve been seeing. The most interesting uses of this technology that I’ve seen involve a constrained dataset. Like the way Luke trained a language model on his own content to create a useful chat interface.

Anyway, The Intelligence Illusion is full of practical down-to-earth advice based on plenty of research backed up with copious citations. I’m only halfway through it and it’s already helped me separate the hype from the reality.