Benchmarks

HumanEval

Codingunit % · normalized over [10, 100]

Hand-written Python programming problems, pass@1. Long-saturated at the frontier.

#ModelSourceScoreNormalized

Score distribution

187 tracked results across the normalization window

10100

Score vs. parameters

Open-weights models, log-x params