Model

BTLM-3B-8K

OPEN

Cerebras · BTLM family · released Jul 1, 2023

Bittensor Language Model: 2.6B (marketed as 3B) with 7B-class quality and 8K context, trained on 627B tokens of SlimPajama, trained on the Condor Galaxy 1 (CG-1) supercomputer (Cerebras/G42 partnership).

ReasoningCodingVisionFunction callingTool useAgentic

-364.6

Elo · rank #436

Parameters

2.6B

Active params

2.6B (dense)

Context

8K tokens

Architecture

Dense transformer, GPT-2 style + SwiGLU + ALiBi + muP (32 layers, d_model 2560)

License

Apache-2.0

Languages

API price (in/out)

No hosted API

Modalities

text

Benchmark results

Bar shows position within the tracked field; marker = field best

ARC-ChallengeReasoning37.6%#154

best: Llama 3.1 405B · 96.9%

ARC-EasyReasoning66.9%#58

best: Phi-3-medium (14B) · 97.7%

HellaSwagReasoning69.8%#124

best: Claude 3 Opus · 95.4%

HumanEvalCoding9.9%#184

best: Claude Opus 4.5 · 99.4%

MMLU (STEM)Knowledge27.1%#11

best: Falcon-H1 34B · 83.6%

MMLUKnowledge28.1%#274

best: OpenAI o3 · 92.9%

OpenBookQAReasoning40.4%#42

best: Claude 1 · 90.8%

PIQAReasoning77.2%#61

best: GPT-4o mini · 93.1%

RACE-HReasoning39.4%#21

best: Claude 3 Opus · 92.9%

Social IQaReasoning46.5%#31

best: Apple DCLM-Baseline 7B · 82.9%

TriviaQAKnowledge34.9%#48

best: Sarvam-1 (2B) · 90.6%

TruthfulQAKnowledge35.9%#94

best: Phi-3.5-MoE (16x3.8B, 6.6B active) · 77.5%

WinoGrandeReasoning65.8%#115

best: PaLM 2 · 90.9%

Run it locally

VRAM @ Q4

2 GB

VRAM @ FP16

5.5 GB

Fits on (Q4)

RTX 3060 12GBRTX 4070 Ti 16GBRTX 3090 24GBRTX 4090 24GBRTX 5090 32GBM4 Pro 48GBM3 Max 128GBM3 Ultra 512GBA100 80GBH100 80GBH200 141GBB200 192GB

Cerebras reports the model fits in under 3GB of memory at 4-bit precision for mobile/edge deployment.

Quantizations

—

BTLM family

Elo progression across releases

BTLM-3B-8KJul 2023-364.6

Resources

HFcerebras/btlm-3b-8k-base DOCSarxiv.org/abs/2309.11568

API price weights · each benchmark row carries its own source badge (see methodology)