| Failure mode | What’s going on |
|---|---|
| Hallucination | The objective rewards plausible continuations, not true ones. When the model lacks the knowledge, it can still produce fluent, confident, wrong text. Post-training reduces this but doesn’t eliminate it. |
| Knowledge cutoff | Weights are frozen after training. Anything after the data cutoff is unknown unless provided in context or via tools. |
| Tokenization artifacts | Letter counting, exact string manipulation, some arithmetic (see 6. Tokenization). |
| Context limits and degradation | Performance can drop on very long inputs, especially for info buried in the middle. |
| Sycophancy | Tendency to agree with the user because that’s what preference training rewarded (see 11. Post-training). |
| Prompt injection | The model can’t fundamentally separate “instructions” from “data” in its context. Text in a webpage or file can try to hijack it. |
| Brittleness and distribution shift | Weird inputs far from the training distribution can produce weird behavior. |
| Non-determinism | Sampling plus floating-point and batching effects mean you won’t get identical outputs every time. |
| Inherited bias | The data reflects the internet, with all that implies. |
← 13. What’s actually going on inside · Contents · 15. Tiny code examples →