Model
Model explorer

Pythia-6.9B

OPEN
EleutherAI · Pythia family · released Apr 3, 2023

Mid-large Pythia size widely used as a reproducible base for training-dynamics and scaling-law studies

ReasoningCodingVisionFunction callingTool useAgentic
-400.3
Elo · rank #442
Parameters
6.9B
Active params
6.9B (dense)
Context
2K tokens
Architecture
Decoder-only Transformer (GPT-NeoX architecture, RoPE, parallel attention+FF, 32 layers)
License
Apache-2.0
Languages
1+
API price (in/out)
No hosted API
Modalities
text
Benchmark results
Bar shows position within the tracked field; marker = field best
ARC-ChallengeReasoning31.3%#162
best: Llama 3.1 405B · 96.9%
ARC-EasyReasoning67.3%#55
best: Phi-3-medium (14B) · 97.7%
BIG-Bench HardReasoning32.3%#125
best: ERNIE 4.5 300B-A47B · 94.3%
GPQA DiamondReasoning25.2%#277
best: GPT-5.6 · 94.6%
IFEvalReasoning22.8%#149
best: Gemma 4 26B A4B · 98.5%
LAMBADAReasoning67.3%#23
best: GPT-3 175B · 86.4%
MMLU-ProKnowledge11.5%#181
best: Claude Fable 5 · 91.5%
PIQAReasoning75.2%#71
best: GPT-4o mini · 93.1%
WinoGrandeReasoning60.9%#130
best: PaLM 2 · 90.9%
Run it locally
VRAM @ Q4
4.7 GB
VRAM @ FP16
15 GB
Fits on (Q4)
RTX 3060 12GBRTX 4070 Ti 16GBRTX 3090 24GBRTX 4090 24GBRTX 5090 32GBM4 Pro 48GBM3 Max 128GBM3 Ultra 512GBA100 80GBH100 80GBH200 141GBB200 192GB
Throughput data unavailable.
Quantizations
GGUF
Pythia family
Elo progression across releases
Pythia-1.4BApr 2023-483.4Pythia-12BApr 2023-475.5Pythia-6.9BApr 2023-400.3
Resources
HFEleutherAI/pythia-6.9bGHEleutherAI/pythia
API price weights · each benchmark row carries its own source badge (see methodology)