Model
Model explorer

MPT-7B-Instruct

OPEN
MosaicML · MPT family · released May 5, 2023

MPT-7B fine-tuned on Dolly-15k + Anthropic HH-RLHF (dolly_hhrlhf) for short-form instruction following; licensed CC-BY-SA-3.0 (more restrictive than the Apache-2.0 base model) due to the derived instruction dataset.

ReasoningCodingVisionFunction callingTool useAgentic
-129.7
Elo · rank #415
Parameters
6.7B
Active params
6.7B (dense)
Context
2.048K tokens
Architecture
Decoder-only Transformer, no positional embeddings (ALiBi), FlashAttention; 32L/32H, d_model 4096
License
CC-BY-SA-3.0
Languages
—
API price (in/out)
No hosted API
Modalities
text
Benchmark results
Bar shows position within the tracked field; marker = field best
ARC-ChallengeReasoning50.3%#123
best: Llama 3.1 405B · 96.9%
DROPReasoning29.4#60
best: Hunyuan-T1 · 93.1
GSM8KMath2.8%#189
best: Llama 3.1 405B · 96.8%
HellaSwagReasoning77.9%#91
best: Claude 3 Opus · 95.4%
MMLUKnowledge32.3%#265
best: OpenAI o3 · 92.9%
TruthfulQAKnowledge35.1%#95
best: Phi-3.5-MoE (16x3.8B, 6.6B active) · 77.5%
WinoGrandeReasoning70.5%#98
best: PaLM 2 · 90.9%
Run it locally
VRAM @ Q4
4.5 GB
VRAM @ FP16
14.5 GB
Fits on (Q4)
RTX 3060 12GBRTX 4070 Ti 16GBRTX 3090 24GBRTX 4090 24GBRTX 5090 32GBM4 Pro 48GBM3 Max 128GBM3 Ultra 512GBA100 80GBH100 80GBH200 141GBB200 192GB
Throughput data unavailable.
Quantizations
GGUF · GGML
MPT family
Elo progression across releases
MPT-7BMay 2023-262.2MPT-7B-InstructMay 2023-129.7MPT-7B-StoryWriter-65k+May 2023-372.6MPT-30BJun 202384.2MPT-30B-InstructJun 2023124.4
Resources
HFmosaicml/mpt-7b-instructGHmosaicml/llm-foundryDOCSdatabricks.com/blog/mpt-7b
API price weights · each benchmark row carries its own source badge (see methodology)