Model
Model explorer

Pythia-12B

OPEN
EleutherAI · Pythia family · released Apr 3, 2023

Flagship of the 8-model Pythia suite; 154 released checkpoints per model for interpretability/training-dynamics research

ReasoningCodingVisionFunction callingTool useAgentic
-475.5
Elo · rank #448
Parameters
11.8B
Active params
11.8B (dense)
Context
2K tokens
Architecture
Decoder-only Transformer (GPT-NeoX architecture, RoPE, parallel attention+FF, 36 layers)
License
Apache-2.0
Languages
1+
API price (in/out)
No hosted API
Modalities
text
Benchmark results
Bar shows position within the tracked field; marker = field best
ARC-ChallengeReasoning31.8%#161
best: Llama 3.1 405B · 96.9%
ARC-EasyReasoning70.2%#48
best: Phi-3-medium (14B) · 97.7%
DROPReasoning4.45#74
best: Hunyuan-T1 · 93.1
GSM8KMath1.7%#191
best: Llama 3.1 405B · 96.8%
LAMBADAReasoning70.5%#18
best: GPT-3 175B · 86.4%
MMLU: AnatomyKnowledge26.7%#4
best: Palmyra Med 70B · 83.7%
MMLU: Clinical KnowledgeKnowledge23.4%#4
best: Palmyra Med 70B · 90.9%
MMLU: Medical GeneticsKnowledge38.0%#3
best: Palmyra Med 70B · 94.0%
MMLUKnowledge26.8%#279
best: OpenAI o3 · 92.9%
PIQAReasoning76.0%#69
best: GPT-4o mini · 93.1%
TruthfulQAKnowledge31.8%#103
best: Phi-3.5-MoE (16x3.8B, 6.6B active) · 77.5%
WinoGrandeReasoning63.9%#128
best: PaLM 2 · 90.9%
Run it locally
VRAM @ Q4
7.9 GB
VRAM @ FP16
25.7 GB
Fits on (Q4)
RTX 3060 12GBRTX 4070 Ti 16GBRTX 3090 24GBRTX 4090 24GBRTX 5090 32GBM4 Pro 48GBM3 Max 128GBM3 Ultra 512GBA100 80GBH100 80GBH200 141GBB200 192GB
Throughput data unavailable.
Quantizations
GGUF
Pythia family
Elo progression across releases
Pythia-1.4BApr 2023-483.4Pythia-12BApr 2023-475.5Pythia-6.9BApr 2023-400.3
Resources
HFEleutherAI/pythia-12bGHEleutherAI/pythia
API price weights · each benchmark row carries its own source badge (see methodology)