Showing 1–2 of 2 · #local-models
MODELSQuantization without tears: what you lose at 8, 4 and 3 bits
Smaller weights, faster inference, and a quality cost that depends far more on the task than on the bit count. A practical guide to choosing a level.
JLJonas LindOct 28 · 6 min
TESTSHow to run an 8B LLM on a laptop: hardware, quantization and tools
Memory maths, quantization choices, tool options and a quick way to predict speed. Everything you need to run a capable model locally.
JLJonas LindOct 26 · 6 min