My notes on AI creative writing
- All commercial LLMs that I have tried, at their present capabilities as of September 2026, are mediocre at best at creative writing tasks.
- Yes, even the fancy expensive ones.
- This is partially due to blah blah blah post training reinforcement learning mode collapse etc.
- An LLM base model is a bag of text heuristics snaffled up from statistical patterns across a vast collection of text.
- Some of these capture specific factual relationships (e.g. ‘The biggest city in New Zealand is’ will be followed by ‘Auckland’) and some of these capture general relationships that exist within the training data corpus (e.g. a text that mentions Bulbasaur and Pallet Town probably has something to do with Pokemon).
- If you play around with base models, you can see that the ‘world model’ is still there even before Reinforcement Learning is applied.
- Some of these capture specific factual relationships (e.g. ‘The biggest city in New Zealand is’ will be followed by ‘Auckland’) and some of these capture general relationships that exist within the training data corpus (e.g. a text that mentions Bulbasaur and Pallet Town probably has something to do with Pokemon).
- Reinforcement Learning doesn’t add extra reasoning capacity to a model but ‘greases the wheels’ towards paths that lead to rewards in training.
- This tends towards the simplest possible behaviour that increases reward, which means models will fall into the attractor of ‘do the most obvious thing that leads to a reward’.
- In practise this shows up as models converging on a particular set of superficially fluent but hackneyed and predictable writing styles, even for open ended domains like creative fiction writing.
- Meanwhile ‘writing that seems fancy and literary but doesn’t make any sense’ is presumably pretty easy and cheap for an LLM to learn as it involves basically just memorising a small set of ~ meaningful ~ words and phrase templates.
- Major commercial models are all presumably trained on similar data and techniques, and so there’s little variation of writing style across models. This makes makes signs of LLM writing quite blatant (if you read enough of it) even if it were possible to prompt them towards ‘good’ writing, as most users would prompt them in the laziest way possible.
- An LLM base model is a bag of text heuristics snaffled up from statistical patterns across a vast collection of text.
- There are things LLMs can do though which may be useful for creative writing, which are:
- Create linear combinations of styles (e.g. pancake recipe in the style of Shakespeare)
- Write coherently with a ‘world-model’ (this seems to have gotten better as models have gotten more capable)
- Smaller open weight models may be better for creative writing despite their difference in benchmark capabilities, as they are
- less constricted by guardrails
- can set sampler parameters1
- can be fine tuned on (hopefully) higher quality samples of creative writing
- And beyond that, text completion models (no post training) might be better.
- This is as they are ‘bags of heuristics’ which capture the features of various styles and genres of text given a plausible context, and which may be less mode collapsed due to their lack of reinforcement learning.
- They seem (at least from my limited playing around with them) to be a lot worse at maintaining coherency over more than a few sentences.
- But fundamentally I believe that LLMs (in their present architecture) they can’t come up with novel ideas that aren’t linear combinations of things that already exist in their training data.
- It seems like this won’t change until robust continual learning is a thing.
Essentially, changing the statistical probabilities of certain tokens being outputted. Some of these options (like temperature) is just injecting an adjustable level of randomness into the generation, but there are clever tricks out there to guide text generations to more unexpected directions ↩︎