Home/Archive/Papers

The archive

48 posts
Showing 1–1 of 1 · #latency
Speculative decoding: faster answers without changing themPAPERS

Speculative decoding: faster answers without changing them

A small model drafts, a big model checks several tokens at once, and the output distribution stays exactly the same. How and when it speeds things up.

JLJonas LindOct 11 · 6 min

Get the Sunday Diff — every post worth reading, once a week.