Model
Model explorer

MPT-30B-Instruct

OPEN
MosaicML · MPT family · released Jun 22, 2023

Instruction-tuned MPT-30B for long-context single-turn tasks, fine-tuned on Dolly/HH-RLHF-derived data, released under CC-BY-SA-3.0.

ReasoningCodingVisionFunction callingTool useAgentic
124.4
Elo · rank #378
Parameters
30B
Active params
30B (dense)
Context
8K tokens
Architecture
Decoder-only Transformer w/ ALiBi + FlashAttention
License
CC-BY-SA-3.0
Languages
—
API price (in/out)
No hosted API
Modalities
text
Benchmark results
Bar shows position within the tracked field; marker = field best
ARC-ChallengeReasoning58.4%#98
best: Llama 3.1 405B · 96.9%
DROPReasoning38.3#57
best: Hunyuan-T1 · 93.1
GSM8KMath15.3%#170
best: Llama 3.1 405B · 96.8%
HellaSwagReasoning84.3%#51
best: Claude 3 Opus · 95.4%
MMLUKnowledge49.2%#234
best: OpenAI o3 · 92.9%
TruthfulQAKnowledge38.0%#88
best: Phi-3.5-MoE (16x3.8B, 6.6B active) · 77.5%
WinoGrandeReasoning75.1%#66
best: PaLM 2 · 90.9%
Run it locally
VRAM @ Q4
19.1 GB
VRAM @ FP16
60 GB
Fits on (Q4)
RTX 3090 24GBRTX 4090 24GBRTX 5090 32GBM4 Pro 48GBM3 Max 128GBM3 Ultra 512GBA100 80GBH100 80GBH200 141GBB200 192GB
Throughput data unavailable.
Quantizations
GGUF Q4 · AWQ
MPT family
Elo progression across releases
MPT-7BMay 2023-262.2MPT-7B-InstructMay 2023-129.7MPT-7B-StoryWriter-65k+May 2023-372.6MPT-30BJun 202384.2MPT-30B-InstructJun 2023124.4
Resources
HFmosaicml/mpt-30b-instructGHmosaicml/llm-foundryDOCSdatabricks.com
API price weights · each benchmark row carries its own source badge (see methodology)