Model

MPT-7B

OPEN

MosaicML · MPT family · released May 5, 2023

Apache-2.0 decoder-only transformer trained from scratch on 1T tokens; ALiBi removes fixed context limits, FlashAttention for efficient training/inference.

ReasoningCodingVisionFunction callingTool useAgentic

-262.2

Elo · rank #429

Parameters

6.7B

Active params

6.7B (dense)

Context

2.048K tokens

Architecture

Decoder-only Transformer, no positional embeddings (ALiBi), FlashAttention; 32L/32H, d_model 4096

License

Apache 2.0

Languages

—

API price (in/out)

No hosted API

Modalities

text

Benchmark results

Bar shows position within the tracked field; marker = field best

AGIEvalReasoning21.3%#42

best: OLMo 3-Think 32B · 88.2%

ARC-ChallengeReasoning47.7%#129

best: Llama 3.1 405B · 96.9%

C-EvalKnowledge23.5%#54

best: Qwen3.6-Plus · 93.3%

CMMLUKnowledge25.9%#31

best: Doubao-1.5-Pro · 90.9%

GSM8KMath9.1%#178

best: Llama 3.1 405B · 96.8%

HellaSwagReasoning75.0%#113

best: Claude 3 Opus · 95.4%

HumanEvalCoding17.1%#172

best: Claude Opus 4.5 · 99.4%

MATH-500Math2.9%#203

best: GPT-5 · 99.4%

MBPPCoding22.8%#103

best: Llama-3.3-Nemotron-Super-49B v1 (Reasoning On) · 91.3%

MMLUKnowledge27.5%#276

best: OpenAI o3 · 92.9%

TruthfulQAKnowledge33.5%#99

best: Phi-3.5-MoE (16x3.8B, 6.6B active) · 77.5%

Run it locally

VRAM @ Q4

4.5 GB

VRAM @ FP16

14.5 GB

Fits on (Q4)

RTX 3060 12GBRTX 4070 Ti 16GBRTX 3090 24GBRTX 4090 24GBRTX 5090 32GBM4 Pro 48GBM3 Max 128GBM3 Ultra 512GBA100 80GBH100 80GBH200 141GBB200 192GB

Throughput data unavailable.

Quantizations

GGUF · GGML

MPT family

Elo progression across releases

MPT-7BMay 2023-262.2 MPT-7B-InstructMay 2023-129.7 MPT-7B-StoryWriter-65k+May 2023-372.6 MPT-30BJun 202384.2 MPT-30B-InstructJun 2023124.4

Resources

HFmosaicml/mpt-7b GHmosaicml/llm-foundry DOCSdatabricks.com/blog/mpt-7b

API price weights · each benchmark row carries its own source badge (see methodology)