Home/Archive/Models

The archive

48 posts
Showing 1–1 of 1 · #quantization
Quantization without tears: what you lose at 8, 4 and 3 bitsMODELS

Quantization without tears: what you lose at 8, 4 and 3 bits

Smaller weights, faster inference, and a quality cost that depends far more on the task than on the bit count. A practical guide to choosing a level.

JLJonas LindOct 28 · 6 min

Get the Sunday Diff — every post worth reading, once a week.