Model

Tülu 3 405B

OPEN

Allen Institute for AI (Ai2) · Tülu 3 family · released Jan 30, 2025

Scaled the fully open Tülu 3 RLVR recipe to 405B (on Llama 3.1); matches or beats DeepSeek V3 and GPT-4o on key benchmarks.

ReasoningCodingVisionFunction callingTool useAgentic

1238.9

Elo · rank #171

Parameters

405.85B

Active params

405.85B (dense)

Context

128K tokens

Architecture

Dense transformer (decoder-only, Llama 3.1 405B base)

License

Llama 3.1 Community License Agreement

Languages

API price (in/out)

No hosted API

Modalities

text

Benchmark results

Bar shows position within the tracked field; marker = field best

AlpacaEvalHuman preference51.4%#16

best: Tülu 2+DPO 70B · 95.1%

BIG-Bench HardReasoning88.6%#9

best: ERNIE 4.5 300B-A47B · 94.3%

GSM8KMath95.5%#6

best: Llama 3.1 405B · 96.8%

HumanEval+Coding92.9%#4

best: Mistral Small 3.2 (24B) · 92.9%

HumanEvalCoding95.9%#6

best: Claude Opus 4.5 · 99.4%

IFEvalReasoning86.0%#51

best: Gemma 4 26B A4B · 98.5%

MATH-500Math67.3%#103

best: GPT-5 · 99.4%

MMLUKnowledge87.0%#36

best: OpenAI o3 · 92.9%

Run it locally

VRAM @ Q4

230 GB

VRAM @ FP16

810 GB

Fits on (Q4)

M3 Ultra 512GB

Too large to run on a single RTX 4090 at any practical quant; requires multi-GPU or CPU offload.

Quantizations

GGUF Q4

Tülu 3 family

Elo progression across releases

Tülu 3 70BNov 20241023.6 Tülu 3 405BJan 20251238.9

Resources

HFallenai/Llama-3.1-Tulu-3-405B GHallenai/open-instruct DOCSallenai.org/blog/tulu-3-405b APIAvailable via Allen Institute for AI (Ai2) API

API price weights · each benchmark row carries its own source badge (see methodology)