Model

DBRX

OPEN

Databricks · DBRX family · released Mar 27, 2024

Fine-grained MoE (132B total/36B active) pretrained on 12T tokens; a leading open model at launch, beating GPT-3.5 on MMLU/GSM8K/HumanEval.

ReasoningCodingVisionFunction callingTool useAgentic

593.2

Elo · rank #282

Parameters

132B

Active params

36B (MoE)

Context

32K tokens

Architecture

Fine-grained MoE, 16 experts (4 active per token)

License

Databricks Open Model License

Languages

—

API price (in/out)

No hosted API

Modalities

text

Benchmark results

Bar shows position within the tracked field; marker = field best

ARC-ChallengeReasoning68.9%#51

best: Llama 3.1 405B · 96.9%

BIG-Bench HardReasoning54.1%#88

best: ERNIE 4.5 300B-A47B · 94.3%

GPQA DiamondReasoning34.9%#241

best: GPT-5.6 · 94.6%

GSM8KMath66.9%#105

best: Llama 3.1 405B · 96.8%

HellaSwagReasoning89.0%#11

best: Claude 3 Opus · 95.4%

HumanEvalCoding70.1%#85

best: Claude Opus 4.5 · 99.4%

IFEvalReasoning48.2%#134

best: Gemma 4 26B A4B · 98.5%

MMLU-ProKnowledge36.8%#156

best: Claude Fable 5 · 91.5%

MMLUKnowledge73.7%#129

best: OpenAI o3 · 92.9%

MT-BenchHuman preference8.39#32

best: Hunyuan-Large (A52B) · 9.4

TruthfulQAKnowledge66.9%#9

best: Phi-3.5-MoE (16x3.8B, 6.6B active) · 77.5%

WinoGrandeReasoning81.8%#24

best: PaLM 2 · 90.9%

Run it locally

VRAM @ Q4

66 GB

VRAM @ FP16

264 GB

Fits on (Q4)

M3 Max 128GBM3 Ultra 512GBA100 80GBH100 80GBH200 141GBB200 192GB

Exceeds a single RTX 4090's 24GB VRAM even at Q4 (~66GB needed); requires multi-GPU or CPU offload

Quantizations

GGUF Q4

DBRX family

Elo progression across releases

DBRXMar 2024593.2

Resources

HFdatabricks/dbrx-instruct GHdatabricks/dbrx DOCSwww.databricks.com/blog/introducing-dbrx-new-state-art-open-llm APIAvailable via Databricks API

API price weights · each benchmark row carries its own source badge (see methodology)