Optimizing for LLMs - A side effect I hope to see
I had this in draft already, but I will take every opportunity to point people at Simon Willison as the ultimate source for modern AI/LLM news, in a hype-free way. And thus, I will set up the context for my thoughts by heavily citing from his recent post:
A crucial characteristic of any model is its training cut-off date. This is the date at which the data they were trained on stopped being collected.
…
LLMs can still help you work with libraries that exist outside their training data, but you need to put in more work—you’ll need to feed them recent examples of how those libraries should be used as part of your prompt.
…
Context is king
Most of the craft of getting good results out of an LLM comes down to managing its context—the text that is part of your current conversation.
…
One of my favorite code prompting techniques is to drop in several full examples relating to something I want to build, then prompt the LLM to use them as inspiration for a new project.
…
Good LLMs are great at answering questions about code.
I’ve noticed a trend, or shift, in approaches to SEO. In particular, how the goal for many website owners will no longer be traditional SEO. Rather, the discussion seems to be about how to build content that LLMs can easily ingest, and possibly cite, and remix later.
What I want to happen
There’s a big chance that, as things keep proceeding, everything is written “for the machines” in an even less human-readable way. That would be disappointing.
As someone who works across languages, libraries, and frameworks, and has to reference documentation a lot: I’m hoping that the side-effects of vehemently trying to please LLMs is way better documentation.
- Clear concise examples, in narrative documentation. Great examples help an LLM know how to use your tools. It works on me, as well!
- More robust documentation, covering the “when/why would you use this particular method” and not just “Method to fetch list of Widgets”.
- Better information architecture of said material. As folks take more care to clearly, intuitively structure things.
At least as of this writing, it seems like writing more robust and clear examples and documentation is a key way to win with LLMs. Particularly if your content (docs, library, framework, product, whatever) came out after an LLMs “cut-off date”.
So far we’ve been able to rely on Search Engine companies telling us to make content “good for the human” to win at SEO. The hope being that as their systems get better, then “good for the human” will continually be closer to “good for the machine” as well.
I hope the same remains true for whatever we’re calling the next version of “SEO”.
We’ll see! 🤷♂️
PS: Some will point out that their hope is to never have to reference documentation at all – and so why hold out hope that it remains easy for humans to read it? I’ve got thoughts and opinions there for another time. ;)