From a single neuron to a trained model
What this is
A bottom-up tour: neurons → networks → how networks learn → how that scales into something that talks. If you only read two sections, read 8. The transformer block and 10. Pretraining at scale. That’s where the real stuff is.
How to get the most out of this guide
- Click any link to jump to that section. Every formula links back to the Symbol cheat sheet.
- Hover a link to preview the section without leaving where you are.
- Collapsed boxes (marked with a small arrow) hold worked examples and extra detail. Click to expand.
- Diagrams are live Mermaid charts and equations are rendered LaTeX.
- Want to play with the ideas? Jump to Interactive visualizers.
Table of contents
| # | Section | In one line |
|---|---|---|
| 0 | Symbol cheat sheet | What Σ, ∂, θ and friends actually mean |
| 0 | The big picture | One diagram of the whole pipeline |
| 1 | 1. The artificial neuron | Multiply, add, squash |
| 2 | 2. Layers and why non-linearity matters | Matrix multiplies and the “stacking” trick |
| 3 | 3. Loss and how wrong we are | A single number that says “bad” or “good” |
| 4 | 4. Gradient descent and backpropagation | How the weights actually improve |
| 5 | 5. From generic networks to language | Two problems to solve for text |
| 6 | 6. Tokenization | Text → integer IDs |
| 7 | 7. Embeddings | IDs → meaningful vectors |
| 8 | 8. The transformer block | Attention, MLP, residuals, norm |
| 9 | 9. Stacking it up and producing output | Full forward pass |
| 10 | 10. Pretraining at scale | Data, objective, scaling laws, hardware |
| 11 | 11. Post-training | Turning a base model into an assistant |
| 12 | 12. Inference | What happens when you hit enter |
| 13 | 13. What’s actually going on inside | Interpretability |
| 14 | 14. Limitations and failure modes | Where it breaks |
| 15 | 15. Tiny code examples | The skeleton in ~50 lines |
| 16 | 16. Where to learn more | Resources + Interactive visualizers |
Prefer one long page? Read the full guide on a single page.