Model

Qwen1.5-72B

OPEN

Alibaba · Qwen1.5 family · released Feb 4, 2024

Gen-1.5 flagship with a uniform 32K context across the whole lineup, but at 72B this model uses standard multi-head attention, not GQA (config: num_key_value_heads = num_attention_heads = 64) — GQA in Qwen1.5 was applied only to the 32B/110B sizes.

ReasoningCodingVisionFunction callingTool useAgentic

545.1

Elo · rank #291

Parameters

72B

Active params

72B (dense)

Context

32K tokens

Architecture

Dense decoder-only transformer, 80 layers, standard multi-head attention (SwiGLU, RMSNorm, attention QKV bias)

License

Tongyi Qianwen LICENSE AGREEMENT (custom, free commercial use below usage threshold)

Languages

12+

API price (in/out)

No hosted API

Modalities

text

Benchmark results

Bar shows position within the tracked field; marker = field best

BIG-Bench HardReasoning65.5%#72

best: ERNIE 4.5 300B-A47B · 94.3%

C-EvalKnowledge84.1%#25

best: Qwen3.6-Plus · 93.3%

CMMLUKnowledge83.5%#10

best: Doubao-1.5-Pro · 90.9%

GSM8KMath79.5%#87

best: Llama 3.1 405B · 96.8%

HumanEvalCoding41.5%#135

best: Claude Opus 4.5 · 99.4%

MATH-500Math34.1%#156

best: GPT-5 · 99.4%

MBPPCoding53.4%#66

best: Llama-3.3-Nemotron-Super-49B v1 (Reasoning On) · 91.3%

MMLUKnowledge77.5%#110

best: OpenAI o3 · 92.9%

Run it locally

VRAM @ Q4

48 GB

VRAM @ FP16

144 GB

Fits on (Q4)

M3 Max 128GBM3 Ultra 512GBA100 80GBH100 80GBH200 141GBB200 192GB

Q4 weights (~48GB) exceed a single RTX 4090's 24GB VRAM; requires multi-GPU or CPU/disk offload.

Quantizations

GPTQ Int4 · GPTQ Int8 · AWQ · GGUF

Qwen1.5 family

Elo progression across releases

Qwen1.5-72BFeb 2024545.1 Qwen1.5-MoE-A2.7BMar 202494.1 CodeQwen1.5-7BApr 2024526.5 Qwen1.5-110BApr 2024665.5

Resources

HFQwen/Qwen1.5-72B DOCSqwenlm.github.io/blog/qwen1.5

API price weights · each benchmark row carries its own source badge (see methodology)