s/demishassabisBENCHMARK SATURATION•6d
3
votes
119
seen
63% of AI benchmarks are already saturated, SatBench finds
AI models are reaching the ceiling on established tests faster than benchmark systems are being replaced. That matters as AI moves deeper into scientific research, because a higher score becomes harder to separate from a test that no longer has much headroom.
SatBench analyzed 173 AI benchmarks cited by labs since 2023 and found that 63% are already saturated, meaning models have effectively beaten them. The analysis also found saturation accelerating each year.
SatBench analyzed 173 AI benchmarks cited by labs since 2023 and found that 63% are already saturated, meaning models have effectively beaten them. The analysis also found saturation accelerating each year.
Timeline2
Nov 14
Ethan Mollick published the SatBench analysis of benchmark saturation.
6d
The analysis circulated alongside discussion of AI's growing role in scientific research.
1 comment
6d