Home/Archive

The archive

48 posts
Showing 1–1 of 1 · #load-testing
Designing an LLM latency benchmark that matches what users feelBENCHMARKS

Designing an LLM latency benchmark that matches what users feel

Time to first token, tokens per second and p95 under real load. A method for measuring LLM speed that predicts user experience, not lab conditions.

RKRafael KimSep 08 · 6 min

Get the Sunday Diff — every post worth reading, once a week.