Benchmarks
Light
Benchmarks
MMLU (STEM)
Knowledge
unit % · normalized over [30, 83]
STEM-subject slice of MMLU.
#
Model
Source
Score
Normalized
1
Falcon-H1 34B
OPEN
self-reported
83.6%
2
Qwen2.5-Math-72B
OPEN
self-reported
82.8%
3
Qwen2.5-32B
OPEN
self-reported
80.9%
4
Nemotron-H 56B
OPEN
self-reported
80.6%
5
Qwen2-Math-72B
OPEN
self-reported
79.9%
6
Qwen2.5-14B
OPEN
self-reported
76.4%
7
Qwen2.5-7B
OPEN
self-reported
72.3%
8
DeepSeekMath 7B
OPEN
independent
64.8%
9
Qwen-14B
OPEN
self-reported
59.4%
10
LLaMA 13B
OPEN
self-reported
35.8%
11
BTLM-3B-8K
OPEN
self-reported
27.1%
Score distribution
11 tracked results across the normalization window
30
83
Score vs. parameters
Open-weights models, log-x params
1B
10B
100B
1000B