Model

Cerebras-GPT 13B

OPEN

Cerebras · Cerebras-GPT family · released Mar 28, 2023

Largest of Cerebras' seven-model, Chinchilla-optimal, fully-open GPT family trained entirely on CS-2 wafer-scale hardware.

ReasoningCodingVisionFunction callingTool useAgentic

-454.6

Elo · rank #445

Parameters

13B

Active params

13B (dense)

Context

2K tokens

Architecture

Dense transformer, GPT-3 style (40 layers, d_model 5120, 40 heads)

License

Apache-2.0

Languages

API price (in/out)

No hosted API

Modalities

text

Benchmark results

Bar shows position within the tracked field; marker = field best

ARC-ChallengeReasoning36.7%#155

best: Llama 3.1 405B · 96.9%

ARC-EasyReasoning71.4%#46

best: Phi-3-medium (14B) · 97.7%

DROPReasoning4.39#75

best: Hunyuan-T1 · 93.1

GSM8KMath1.3%#197

best: Llama 3.1 405B · 96.8%

HellaSwagReasoning51.3%#150

best: Claude 3 Opus · 95.4%

LAMBADAReasoning69.6%#20

best: GPT-3 175B · 86.4%

MMLUKnowledge25.9%#284

best: OpenAI o3 · 92.9%

OpenBookQAReasoning28.6%#52

best: Claude 1 · 90.8%

PIQAReasoning76.6%#66

best: GPT-4o mini · 93.1%

TruthfulQAKnowledge39.2%#83

best: Phi-3.5-MoE (16x3.8B, 6.6B active) · 77.5%

WinoGrandeReasoning64.6%#125

best: PaLM 2 · 90.9%

Run it locally

VRAM @ Q4

7 GB

VRAM @ FP16

26 GB

Fits on (Q4)

RTX 3060 12GBRTX 4070 Ti 16GBRTX 3090 24GBRTX 4090 24GBRTX 5090 32GBM4 Pro 48GBM3 Max 128GBM3 Ultra 512GBA100 80GBH100 80GBH200 141GBB200 192GB

Throughput data unavailable.

Quantizations

—

Cerebras-GPT family

Elo progression across releases

Cerebras-GPT 13BMar 2023-454.6

Resources

HFcerebras/Cerebras-GPT-13B DOCSarxiv.org/abs/2304.03208

API price weights · each benchmark row carries its own source badge (see methodology)