Home/Archive/Benchmarks

The archive

48 posts
Showing 1–1 of 1 · #contamination
Benchmark contamination: how to check whether a model has seen the testBENCHMARKS

Benchmark contamination: how to check whether a model has seen the test

Canary strings, n-gram overlap, perturbed questions and fresh held-out sets. How to tell real capability from a memorised test.

SPSana PatelNov 05 · 6 min

Get the Sunday Diff — every post worth reading, once a week.