The #1 guide for people who have never done it
What the parts are, how they wire together, what data goes in, and what it costs. Written from a working build, including the parts that went wrong.
AI4DUMMIES · Book One
Most explanations of language models are either three equations or three hundred pages. This is the middle: enough to actually plan a build, decide what data you need, and know what you are looking at when the numbers on screen start moving.
No code here on purpose. Code goes stale and every framework spells things differently. The shape of the thing does not change, and the shape is what nobody explains.
It predicts the next word. That is the whole job.
Everything else — answering questions, writing code, having a personality — falls out of doing that one thing extremely well over a very large amount of text. Nobody programs in "be helpful." You train next-word prediction, and helpfulness is a behaviour that emerges when the training text contains helpful exchanges.
This matters practically: you never write rules. If the model does something wrong, the fix is almost always in the data, not in a rule you add afterward. A hand-written rule is a patch over a hole in the training set.
A model is a stack of identical layers. Text goes in the bottom, comes out the top as a prediction. Each layer does the same two jobs.
Attention lets every word look at the other words and decide which ones matter. In "the dog that chased the cat was tired," attention is what connects "tired" back to "dog" and not to "cat." Without it a model only sees the previous word or two, which produces text that is topically right and grammatically scrambled.
After looking around, each word gets processed on its own through a small expansion-and-contraction. This is where most of the model's actual stored knowledge lives — roughly two thirds of the parameters.
| Part | What it does | Why you care |
|---|---|---|
| Embedding table | Turns each token into a list of numbers | The model's vocabulary lives here |
| Position information | Tells the model word order | Without it, sentences are just word bags. Modern builds use rotary positions (RoPE) |
| Normalisation | Keeps numbers in a sane range between layers | Skip it and training explodes. RMSNorm is the current default |
| Output head | Turns the final numbers back into "which word next" | Where the actual prediction pops out |
You hear that a model "has 7 billion parameters" and it sounds like a filing cabinet with 7 billion facts in it. It is not. It is closer to 7 billion dials.
Picture an old mixing desk, the kind with rows of knobs. Sound goes in one end, comes out the other, and every knob nudges what happens in between. Nobody labelled the knobs. Nobody decided what any single one does. You just turn them until the output sounds right.
A parameter is one of those knobs. It is a single number, usually something small and unremarkable like 0.0384 or -1.27. On its own it means nothing at all, and you cannot look at one and say what it is for.
Nobody. That is the whole trick, and it is the part people find hardest to believe.
Training is a loop that runs millions of times:
Step 4 is the whole of machine learning. The nudge is tiny on purpose — far too small to fix the current mistake. It is not meant to. Each nudge is a fraction of a correction, and the model only becomes good because it gets billions of them, from billions of different sentences, and the pressures that keep pointing the same way accumulate while the random ones cancel out.
Nobody chooses what any dial means. Meaning arrives on its own: some dials end up tracking whether the subject is plural, some end up caring about the tone of a sentence, some do jobs nobody has a word for. You do not assign these roles and you cannot read them off afterward. They emerge because they were useful for guessing the next word.
More dials means more capacity to represent things. It also means more dials that need enough examples to settle sensibly. Give a model 600 million dials and only a few million sentences to learn from, and most of those dials never receive enough pressure to point anywhere useful — they end up memorising the training text rather than learning language.
That is the ratio in the next chapter, and it is the reason a small well-fed model beats a large starved one.
Four dials decide how big your model is:
Real configurations from a working build, so these are not invented:
| Name | Width | Layers | Context | Runs on |
|---|---|---|---|---|
| tiny | 256 | 6 | 256 | A phone, no graphics card |
| small | 512 | 8 | 768 | Any gaming PC |
| large | 1536 | 24 | 1024 | One good graphics card (~600M parameters) |
This chapter matters more than the rest combined. The architecture is nearly a solved problem you can copy. The data is where your model becomes good or stays useless, and it is where nearly all the real work goes.
The easiest text to get legally is old — public-domain books, out-of-copyright encyclopedias. It is free, it is clean, and it will give your model the voice of a Victorian gentleman. It will not know what a phone is. Everything it says will sound like 1890 because it is 1890.
You need modern text, and you need conversation, and those are different problems from getting text at all.
| Type | What it teaches | Watch out for |
|---|---|---|
| Filtered web text | Modern language, current concepts, range | Use a quality-filtered set, not raw scrape. Raw web is mostly garbage |
| Books | Long-range structure, narrative, real paragraphs | Age. Cap the old stuff or it dominates the voice |
| Conversation / dialogue | Turn-taking, answering rather than continuing | Without this it monologues instead of replying |
| Reference (dictionary, encyclopedia) | Facts, definitions, vocabulary breadth | Dense and repetitive. A little goes a long way |
| Code | Structure, logic, precision — helps reasoning even if you never want code out | Optional, but cheap gains |
| Instructions (task then response) | Doing what it is told | Usually a later stage, not the main run |
Text is measured in tokens, roughly three-quarters of a word each. A useful yardstick: a full-length novel is about 120,000 tokens. So one billion tokens is around 8,000 books.
Ratios matter as much as volume. A defensible starting mix for a general conversational model:
| Share | Source | Why this much |
|---|---|---|
| 60–70% | Quality-filtered modern web | The backbone. Sets the register as contemporary |
| 15–20% | Books | Teaches structure longer than a paragraph. Cap it — this is the Victorian-voice risk |
| 8–12% | Conversation | The difference between a model that answers and one that rambles |
| 3–5% | Reference | Vocabulary and facts, without letting dictionary cadence take over |
| 0–10% | Code | Optional. Improves structured reasoning |
Models cannot read letters. A tokenizer chops text into pieces and assigns each a number. Pieces are usually chunks of words: "unhappiness" might become "un" + "happi" + "ness."
Decisions that matter:
Everything so far is the shape of the thing. This is the shopping list: names, what each one is for, and where it lives.
PyTorch is what nearly everyone trains in. It runs the maths on the graphics card and, critically, works out the nudge for every single dial automatically, so you never write that part yourself. If you install one thing, this is it. Match the version to your card's CUDA release; the site gives you the exact command.
The alternative is JAX, which is genuinely good and common in research. But the tutorials, the example code and the answers to your questions are overwhelmingly PyTorch. Start where the help is.
| What | Does | Where |
|---|---|---|
| Transformers | Model architectures, ready-built — and worth reading, the code is unusually clear | huggingface/transformers |
| tokenizers | Trains and runs the text-to-numbers step | huggingface/tokenizers |
| SentencePiece | The other standard tokenizer, used by many well-known models | google/sentencepiece |
| datasets | Downloads and streams big corpora without filling your disk | huggingface/datasets |
| Accelerate | Multi-card and mixed-precision plumbing | huggingface/accelerate |
| PEFT | LoRA and friends — see Chapter 11 | huggingface/peft |
| bitsandbytes | Shrinks a finished model for running | bitsandbytes |
| TensorBoard or Weights & Biases | Draws the loss curve, so you see the trend instead of one jumpy line | tensorboard / wandb.ai |
This is the part that is hardest to find written down anywhere, and it is the part that decides whether your model is any good.
| Source | What it is | Rough size |
|---|---|---|
| FineWeb-Edu | Web text filtered for educational quality. Modern register. The best single starting point | 1.3T tokens — take a slice |
| FineWeb | The larger sibling, not education-filtered | 15T tokens |
| Dolma | Open corpus with a documented mix and permissive terms | 3T tokens |
| RedPajama | Open reconstruction of a well-known training mix | 1T tokens |
| The Pile (uncopyrighted) | Long-standing mixed corpus, academic and technical heavy | ~400B tokens |
| Project Gutenberg | Public-domain books, free and clean. This is the Victorian-voice trap from Chapter 5 — cap it. Bulk mirror at the offline catalogues | ~75,000 books |
| Wikipedia dumps | Facts and clear expository prose. Simple English is a good small start; extract with wikiextractor | ~4B tokens (English) |
| The Stack v2 | Permissively-licensed source code, if you want the reasoning boost | ~900B tokens |
| UltraChat, Alpaca, OpenAssistant | Conversation and instruction data — the turn-taking without which it monologues | Hundreds of thousands of exchanges |
| Internet Archive texts, Wikisource | Reference works, old encyclopedias, dictionaries. Same age caveat as Gutenberg | Millions of items |
| You have | Realistic target |
|---|---|
| No graphics card | Read nanoGPT and run a tiny model on the processor to watch the loop work. Do not try to train anything real |
| 8 GB card | A small model (Chapter 4). Genuinely coherent output is reachable |
| 16–24 GB card | Several hundred million dials. This is where it gets interesting — and it is an ordinary gaming card |
| Rented cloud hours | Sensible for one long run. Rent before buying hardware to find out whether you enjoy this |
Memory matters more than speed. A slower card with more of it beats a faster card that cannot hold your model.
| Knob | Plain meaning | Practical note |
|---|---|---|
| Learning rate | How big a correction each step makes | The one most likely to ruin a run. Too high and it diverges; too low and it crawls |
| Batch size | How much text per step | Bigger is steadier. Limited by memory |
| Gradient accumulation | Faking a big batch on small hardware | How consumer cards train serious models |
| Warmup | Starting gently before full speed | Skipping it is a classic early-blowup cause |
| Steps | How many rounds total | With batch size, decides how many times it sees your data |
A training run prints a line every few seconds. It looks like this, and this one is real:
step 2457/3000 lr 5.27e-05 loss 2.550 140,221 tok/s| Loss | Reality |
|---|---|
| 10+ | Random. Just started, or something is broken |
| 4–6 | Learning letter and word patterns |
| 2.5–4 | Real grammar appearing |
| 1.5–2.5 | Coherent for a small model |
| Under 1.5 | Strong — or you are memorising, check against held-out text |
You have a base model. It writes decent English and has no self. Now you want a character — a support agent, a narrator, someone with a voice. There are three ways to do that and they cost wildly different amounts.
Put "you are a helpful pirate" at the front of every conversation. Free, instant, changes nothing permanent. It also eats context space, and the character drifts as the conversation gets long, because nothing about the model actually changed.
Fine for trying an idea. Not how you ship a character.
Keep training the whole base model on text in your character's voice. Every dial is still free to move, so the voice sinks in properly.
The catch is cost: you need memory for every dial plus the optimiser's bookkeeping for every dial, which is typically several times the model's own size. And you get a whole new model per character. Ten characters, ten full copies.
It is the right call when you have one character and enough memory. Our own runs do exactly this: the base is trained once over hours, then each character is a short run of a few hundred steps starting from a copy of it — about twelve minutes each, because the grammar is already paid for and only the voice has to move.
LoRA stands for Low-Rank Adaptation, and the idea is better than the name.
Rather than adjusting the model's millions of dials, you freeze all of them and bolt a small extra set of dials alongside. Only the small set trains. At runtime the model's original output and the little add-on's output are added together, and the result behaves like a model that learned your character.
Why it works: the change from "general English" to "this specific voice" is a much simpler change than the model itself. It does not need millions of free dials to express. A few million will carry it.
| Full fine-tune | LoRA | |
|---|---|---|
| Dials being trained | All of them | Typically well under 1% |
| Memory needed | Large — often several times the model | A fraction of that |
| What you end up with | A whole new model | A small file, often a few MB |
| Ten characters | Ten full models | One model, ten small files |
| Swapping between them | Load a different model | Swap the add-on, keep the model loaded |
| Depth of change | Anything, including new knowledge | Voice, style, format — excellent. New facts — weaker |
The practical dial in LoRA is its rank — how many extra dials you allow. Small (8–16) is plenty for a voice or an output format. Larger (32–64) if the behaviour is more involved. Bigger is not better here either; too large and it starts memorising your examples instead of learning the pattern.
PEFT implements this and it is a handful of lines to wire in. The original paper is readable. QLoRA combines it with a shrunk base model so you can adapt a large model on modest hardware.
Far less than you would guess. A few hundred to a few thousand examples in the voice you want, and quality beats quantity every time — a hundred sharp examples beat ten thousand mediocre ones.
Write them in the shape the model will actually be used in. If it will answer questions, write questions and answers, not prose about the character. The model learns the form as much as the content, which is why a character trained only on monologue will monologue at you forever.
Every one of those billions of dials has to be stored somewhere, and each takes up a certain amount of memory. That size is the single biggest reason serious AI has needed rooms full of expensive hardware.
Think of it as how many decimal places you keep. A number stored generously might be 0.038471629. Stored leanly it becomes 0.038. Same number, less room, slightly less exact. Cut the storage in half and your model is half the size. Cut it to a quarter and a model that needed four graphics cards now needs one.
People conflate these constantly, and the difference is the whole story.
Shrinking a finished model is easy and everyone does it. Train it generously, then round all the dials down afterward. This is called quantization. It works well, it is well understood, and it is why you can run a capable model on a laptop.
| Storage per dial | A 600M model | Quality after rounding |
|---|---|---|
| Generous (16-bit) | ~1.2 GB | The reference |
| Half (8-bit) | ~600 MB | Essentially identical |
| Quarter (4-bit) | ~300 MB | Slightly worse, usually fine |
Training in low precision is a different problem entirely, and it is genuinely hard.
Remember that each training nudge is tiny — deliberately far smaller than the error it is correcting. Now store the dial with very few decimal places. The nudge is smaller than the smallest change that storage can represent. So the dial receives the nudge and does not move. Then it happens again. And again.
The model stops learning entirely. Not slowly, not badly. It sits there while the training loop runs, and the numbers never change. Everyone hits this, which is why nearly all training happens at generous precision and the shrinking is saved for the end.
This is not hypothetical. A hardware manufacturer published work getting most of the way there on their own datacenter equipment, and their paper ended by listing what remained unsolved — getting the whole model down to the small format without training falling apart.
That remaining part has since been done, on a consumer graphics card of the sort people buy to play games, in a house. The published techniques did not transfer; what was actually killing the training turned out to be something not previously written down, and fixing that was the work.
The result that matters is not the smaller file. It is this: because everything takes less room, a bigger model fits on the same card — and the bigger model trained in the lean format outperformed the best smaller model trained the generous way.
So it is not a tradeoff where you accept a worse model to save money. Same hardware, better result.
| Symptom | Usual cause |
|---|---|
| Loss goes to NaN | Learning rate too high, or no warmup |
| Loss flat from the start | Learning rate near zero, or data not actually loading |
| Output is word salad | Undertrained, or too little data for the size |
| Sounds antique | Corpus is all old public-domain text |
| Rambles, never answers | No conversational data in the mix |
| Repeats one phrase forever | Needs a repetition penalty at generation time |
| Claims to be a different model | Identity leaked in from the base corpus |
| Out of memory immediately | Batch or context too large. Cut batch, raise accumulation |
| Great on training text, bad on new text | Memorising. More data, or fewer passes over it |