Model
Model explorer

MPT-30B

OPEN
MosaicML · MPT family · released Jun 22, 2023

Base 30B decoder pretrained on 1T tokens of English text and code, 8k context via ALiBi, sized to run on a single 80GB GPU.

ReasoningCodingVisionFunction callingTool useAgentic
84.2
Elo · rank #389
Parameters
30B
Active params
30B (dense)
Context
8K tokens
Architecture
Decoder-only Transformer w/ ALiBi + FlashAttention
License
Apache-2.0
Languages
—
API price (in/out)
No hosted API
Modalities
text
Benchmark results
Bar shows position within the tracked field; marker = field best
ARC-ChallengeReasoning56.0%#104
best: Llama 3.1 405B · 96.9%
FIN-benchKnowledge53.2%#3
best: Poro 34B · 66.3%
GSM8KMath16.9%#168
best: Llama 3.1 405B · 96.8%
HellaSwagReasoning82.4%#64
best: Claude 3 Opus · 95.4%
MMLUKnowledge48.0%#238
best: OpenAI o3 · 92.9%
TruthfulQAKnowledge38.4%#87
best: Phi-3.5-MoE (16x3.8B, 6.6B active) · 77.5%
WinoGrandeReasoning74.9%#68
best: PaLM 2 · 90.9%
Run it locally
VRAM @ Q4
19.1 GB
VRAM @ FP16
60 GB
Fits on (Q4)
RTX 3090 24GBRTX 4090 24GBRTX 5090 32GBM4 Pro 48GBM3 Max 128GBM3 Ultra 512GBA100 80GBH100 80GBH200 141GBB200 192GB
Throughput data unavailable.
Quantizations
GGUF Q4 · AWQ
MPT family
Elo progression across releases
MPT-7BMay 2023-262.2MPT-7B-InstructMay 2023-129.7MPT-7B-StoryWriter-65k+May 2023-372.6MPT-30BJun 202384.2MPT-30B-InstructJun 2023124.4
Resources
HFmosaicml/mpt-30bGHmosaicml/llm-foundryDOCSdatabricks.com
API price weights · each benchmark row carries its own source badge (see methodology)