Model
Model explorer

Llama-3.1-Minitron 4B

OPEN
NVIDIA · Minitron family · released Aug 14, 2024

4B model pruned (width) and distilled from Llama-3.1-8B, using ~40x fewer training tokens than training from scratch.

ReasoningCodingVisionFunction callingTool useAgentic
102.4
Elo · rank #384
Parameters
4.5B
Active params
4.5B (dense)
Context
8K tokens
Architecture
Dense Transformer (width-pruned + distilled from Llama-3.1-8B)
License
NVIDIA Open Model License
Languages
—
API price (in/out)
No hosted API
Modalities
text
Benchmark results
Bar shows position within the tracked field; marker = field best
ARC-ChallengeReasoning55.6%#105
best: Llama 3.1 405B · 96.9%
GSM8KMath41.2%#143
best: Llama 3.1 405B · 96.8%
HellaSwagReasoning76.1%#107
best: Claude 3 Opus · 95.4%
MBPPCoding32.4%#94
best: Llama-3.3-Nemotron-Super-49B v1 (Reasoning On) · 91.3%
MMLUKnowledge60.5%#203
best: OpenAI o3 · 92.9%
TruthfulQAKnowledge42.9%#71
best: Phi-3.5-MoE (16x3.8B, 6.6B active) · 77.5%
WinoGrandeReasoning73.5%#77
best: PaLM 2 · 90.9%
Run it locally
VRAM @ Q4
3.5 GB
VRAM @ FP16
10 GB
Fits on (Q4)
RTX 3060 12GBRTX 4070 Ti 16GBRTX 3090 24GBRTX 4090 24GBRTX 5090 32GBM4 Pro 48GBM3 Max 128GBM3 Ultra 512GBA100 80GBH100 80GBH200 141GBB200 192GB
Throughput data unavailable.
Quantizations
GGUF Q4 · MLX
Minitron family
Elo progression across releases
Llama-3.1-Minitron 4BAug 2024102.4
Resources
HFnvidia/Llama-3.1-Minitron-4B-Width-BaseDOCSdeveloper.nvidia.com
API price weights · each benchmark row carries its own source badge (see methodology)