PAPERSLoRA in plain English
Fine-tune a large model by training less than one percent of it. How low-rank adapters work, what QLoRA adds and when to use them.
PAPERSSparse attention explained: sliding windows, global tokens and block sparsity
Why full attention gets expensive, and the three main ideas that make long contexts affordable, with the trade-offs each one makes.
PAPERSSpeculative decoding: faster answers without changing them
A small model drafts, a big model checks several tokens at once, and the output distribution stays exactly the same. How and when it speeds things up.
PAPERSScaling laws, in plain English
Compute, data and parameters: the three-way trade-off behind every model release, explained without equations you have to memorise.
PAPERSMixture of experts, without the hype
Many parameters, few used per token. How MoE routing works, what "active parameters" really means, and the trade-offs for anyone running or buying these models.
PAPERSPreference tuning: RLHF and DPO side by side
Reward models and PPO versus a single elegant loss. How RLHF and DPO turn human preferences into model behaviour, and what each costs.
PAPERSTokenizers: the layer everyone ignores until it bites
Why models struggle to count letters, why some languages cost more, and why numbers behave strangely. The tokenizer explains more than you think.
PAPERSChain of thought and self-consistency, revisited
Write the steps, sample several paths, take the majority answer. The ideas behind modern reasoning, what still works, and what reasoning models changed.