Benchmarks
Light
Benchmarks
PIQA
Reasoning
unit % · normalized over [50, 92]
Physical-interaction commonsense QA.
#
Model
Source
Score
Normalized
1
GPT-4o mini
CLOSED
independent
93.1%
2
UL2 20B
OPEN
self-reported
90.7%
3
Phi-3.5-MoE (16x3.8B, 6.6B active)
OPEN
self-reported
88.6%
4
Hunyuan-Large (A52B)
OPEN
self-reported
88.3%
5
Phi-3-medium (14B)
OPEN
self-reported
87.9%
6
Phi-3-small (7B)
OPEN
self-reported
86.9%
7
NAVER HyperCLOVA X
CLOSED
self-reported
86.4%
8
Hermes 3 405B
OPEN
self-reported
86.0%
9
Nemotron-H 56B
OPEN
self-reported
85.0%
10
PaLM 2
CLOSED
self-reported
85.0%
11
Inflection-1
CLOSED
self-reported
84.2%
12
Inflection-2
CLOSED
self-reported
84.2%
13
Nous Hermes 2 Mixtral 8x7B DPO
OPEN
self-reported
83.8%
14
Mixtral 8x7B
OPEN
self-reported
83.6%
15
Phi-3-mini (3.8B)
OPEN
self-reported
83.5%
16
AI21 Jamba
OPEN
self-reported
83.2%
17
Gemma 2 27B
OPEN
self-reported
83.2%
18
Mistral 7B
OPEN
self-reported
83.0%
19
GPT-3 175B
CLOSED
self-reported
82.8%
20
Llama 2 70B
OPEN
self-reported
82.8%
21
LLaMA 65B
OPEN
self-reported
82.8%
22
OpenHermes 2.5 Mistral 7B
OPEN
self-reported
82.7%
23
H2O-Danube3 4B
OPEN
self-reported
82.6%
24
DeepSeek LLM 67B
OPEN
self-reported
82.6%
25
Nemotron-4 15B
OPEN
self-reported
82.4%
26
LLaMA 33B
OPEN
self-reported
82.3%
27
PaLM (540B)
CLOSED
self-reported
82.3%
28
Megatron-Turing NLG 530B
CLOSED
self-reported
82.0%
29
Chinchilla
CLOSED
self-reported
81.8%
30
Gopher
CLOSED
self-reported
81.8%
31
Gemma 2 9B
OPEN
self-reported
81.7%
32
Apple DCLM-Baseline 7B
OPEN
self-reported
81.3%
33
Gemma 7B
OPEN
self-reported
81.2%
34
Jais 70B
OPEN
self-reported
81.2%
35
OPT-175B
OPEN
independent
81.1%
36
Phi-3.5-mini (3.8B)
OPEN
self-reported
81.0%
37
Llama 2 13B
OPEN
self-reported
80.5%
38
Qwen1.5-MoE-A2.7B
OPEN
independent
80.5%
39
Nemotron-3 8B
OPEN
self-reported
80.4%
40
Falcon-7B
OPEN
self-reported
80.3%
41
DeepSeekMoE 16B
OPEN
self-reported
80.2%
42
ChatGLM3-6B
OPEN
self-reported
80.1%
43
LLaMA 13B
OPEN
self-reported
80.1%
44
Luminous Supreme (70B)
CLOSED
self-reported
79.9%
45
Qwen-14B
OPEN
self-reported
79.9%
46
Skywork-13B
OPEN
self-reported
79.9%
47
LLaMA 7B
OPEN
self-reported
79.8%
48
OLMoE-1B-7B-0924
OPEN
self-reported
79.8%
49
OPT-66B
OPEN
independent
79.8%
50
Falcon3-7B
OPEN
self-reported
78.8%
51
Llama 2 7B
OPEN
self-reported
78.8%
52
Falcon3-10B
OPEN
self-reported
78.4%
53
OLMo 7B
OPEN
self-reported
78.4%
54
Apple OpenELM (270M–3B)
OPEN
self-reported
78.2%
55
Phi-4-mini (3.8B)
OPEN
self-reported
77.6%
56
Phi-4-mini-instruct
OPEN
self-reported
77.6%
57
SmolLM2 1.7B
OPEN
self-reported
77.6%
58
RedPajama-INCITE-7B-Base
OPEN
self-reported
77.4%
59
Gemma 2B
OPEN
self-reported
77.3%
60
OPT-IML 30B
OPEN
self-reported
77.3%
61
BTLM-3B-8K
OPEN
self-reported
77.2%
62
OpenLLaMA 7B
OPEN
self-reported
77.0%
63
RedPajama-INCITE-7B-Instruct
OPEN
self-reported
77.0%
64
H2O-Danube 1.8B
OPEN
self-reported
76.7%
65
NAVER HyperCLOVA X SEED Think 32B
OPEN
self-reported
76.7%
66
Cerebras-GPT 13B
OPEN
self-reported
76.6%
67
Phi-1.5 (1.3B)
OPEN
self-reported
76.6%
68
GPT-J-6B
OPEN
self-reported
76.5%
69
Pythia-12B
OPEN
self-reported
76.0%
70
XGen-7B 8K
OPEN
self-reported
75.5%
71
Pythia-6.9B
OPEN
self-reported
75.2%
72
RedPajama-INCITE-Base-3B-v1
OPEN
self-reported
74.7%
73
Comma v0.1-2T
OPEN
self-reported
72.5%
74
GPT-Neo 2.7B
OPEN
self-reported
72.1%
75
Pythia-1.4B
OPEN
self-reported
70.9%
76
ChatGLM2-6B
OPEN
self-reported
69.6%
77
Jais-13B
OPEN
self-reported
67.6%
78
ChatGLM-6B
OPEN
self-reported
65.7%
79
Phi-2 (2.7B)
OPEN
self-reported
60.2%
Score distribution
79 tracked results across the normalization window
50
92
Score vs. parameters
Open-weights models, log-x params
1B
10B
100B
1000B