Showing 1–2 of 2 · #inference
PAPERSSpeculative decoding: faster answers without changing them
A small model drafts, a big model checks several tokens at once, and the output distribution stays exactly the same. How and when it speeds things up.
JLJonas LindOct 11 · 6 min
PAPERSMixture of experts, without the hype
Many parameters, few used per token. How MoE routing works, what "active parameters" really means, and the trade-offs for anyone running or buying these models.
MOMira OkaforSep 14 · 6 min