Model

Llama 2 70B Chat

OPEN

Meta · Llama 2 family · released Jul 18, 2023

Flagship RLHF chat model fine-tuned on 1M+ human annotations; Meta reported it on par with ChatGPT/PaLM in human helpfulness and safety evaluations.

ReasoningCodingVisionFunction callingTool useAgentic

363.8

Elo · rank #342

Parameters

69B

Active params

69B (dense)

Context

4K tokens

Architecture

Dense decoder-only Transformer with Grouped-Query Attention (GQA)

License

Llama 2 Community License Agreement

Languages

—

API price (in/out)

No hosted API

Modalities

text

Benchmark results

Bar shows position within the tracked field; marker = field best

ARC-ChallengeReasoning64.6%#65

best: Llama 3.1 405B · 96.9%

Arena-HardHuman preference11.6%#53

best: Qwen3-235B-A22B (Non-Thinking) · 96.1%

Arena EloHuman preference1051#50

best: Claude Fable 5 · 1505

DROPReasoning10.5#65

best: Hunyuan-T1 · 93.1

GSM8KMath26.7%#158

best: Llama 3.1 405B · 96.8%

HellaSwagReasoning85.9%#33

best: Claude 3 Opus · 95.4%

MMLUKnowledge63.0%#193

best: OpenAI o3 · 92.9%

MT-BenchHuman preference6.86#58

best: Hunyuan-Large (A52B) · 9.4

TruthfulQAKnowledge64.1%#18

best: Phi-3.5-MoE (16x3.8B, 6.6B active) · 77.5%

WinoGrandeReasoning80.5%#36

best: PaLM 2 · 90.9%

Run it locally

VRAM @ Q4

38 GB

VRAM @ FP16

140 GB

Fits on (Q4)

M3 Max 128GBM3 Ultra 512GBA100 80GBH100 80GBH200 141GBB200 192GB

Q4_K_M weights (~38-41GB) exceed the RTX 4090's 24GB VRAM, forcing CPU offload; no reliably sourced single-GPU 4090 tokens/sec figure was found specifically for this model.

Quantizations

GGUF · GPTQ · AWQ · GGML

Llama 2 family

Elo progression across releases

Llama 2 13BJul 202369.7 Llama 2 13B ChatJul 2023105.1 Llama 2 70BJul 2023365.0 Llama 2 70B ChatJul 2023363.8 Llama 2 7BJul 2023-121.8 Llama 2 7B ChatJul 2023-10.6

Resources

HFmeta-llama/Llama-2-70b-chat GHmeta-llama/llama DOCSai.meta.com/llama APIAvailable via Meta API

API price weights · each benchmark row carries its own source badge (see methodology)