Before the details, here is the whole machine on one page. Every box below gets its own section.
flowchart LR A["Raw text"] --> B["Tokenizer<br/>(section 6)"] B --> C["Embeddings<br/>(section 7)"] C --> D["Transformer blocks<br/>x L layers<br/>(section 8)"] D --> E["Unembedding<br/>(section 9)"] E --> F["Softmax<br/>probabilities"] F --> G["Sample next token<br/>(section 12)"] G -.->|"append and repeat"| A
And here is how it learns (sections 3 and 4):
flowchart LR A["Batch of text"] --> B["Forward pass<br/>make predictions"] B --> C["Loss<br/>how wrong?"] C --> D["Backward pass<br/>compute gradients"] D --> E["Update weights<br/>w ← w − η·gradient"] E --> A
← Symbol cheat sheet · Contents · 1. The artificial neuron →