Home/Archive/Benchmarks

The archive

48 posts
Showing 1–1 of 1 · #statistics
Your 2-point improvement might be noise: confidence intervals for LLM evalsBENCHMARKS

Your 2-point improvement might be noise: confidence intervals for LLM evals

A 2-point jump on 200 questions is usually noise. How to compute error bars, run paired comparisons and size your eval set to detect real gains.

JLJonas LindOct 01 · 6 min

Get the Sunday Diff — every post worth reading, once a week.