Model

OLMo 7B

OPEN

Allen Institute for AI (Ai2) · OLMo family · released Feb 1, 2024

Ai2's first truly open LLM: weights, Dolma pretraining data, training code and every-1000-step checkpoints all released

ReasoningCodingVisionFunction callingTool useAgentic

-182.0

Elo · rank #422

Parameters

6.9B

Active params

6.9B (dense)

Context

2K tokens

Architecture

Dense Transformer (32 layers, non-parametric LayerNorm, RoPE, SwiGLU, full attention)

License

Apache-2.0

Languages

—

API price (in/out)

No hosted API

Modalities

text

Benchmark results

Bar shows position within the tracked field; marker = field best

ARC-ChallengeReasoning48.5%#128

best: Llama 3.1 405B · 96.9%

ARC-EasyReasoning65.4%#59

best: Phi-3-medium (14B) · 97.7%

GSM8KMath8.5%#180

best: Llama 3.1 405B · 96.8%

HellaSwagReasoning76.4%#104

best: Claude 3 Opus · 95.4%

MMLUKnowledge28.3%#273

best: OpenAI o3 · 92.9%

OpenBookQAReasoning50.2%#25

best: Claude 1 · 90.8%

PIQAReasoning78.4%#53

best: GPT-4o mini · 93.1%

TruthfulQAKnowledge36.0%#92

best: Phi-3.5-MoE (16x3.8B, 6.6B active) · 77.5%

WinoGrandeReasoning67.9%#108

best: PaLM 2 · 90.9%

Run it locally

VRAM @ Q4

4.2 GB

VRAM @ FP16

14 GB

Fits on (Q4)

RTX 3060 12GBRTX 4070 Ti 16GBRTX 3090 24GBRTX 4090 24GBRTX 5090 32GBM4 Pro 48GBM3 Max 128GBM3 Ultra 512GBA100 80GBH100 80GBH200 141GBB200 192GB

Throughput data unavailable.

Quantizations

GGUF

OLMo family

Elo progression across releases

OLMo 7BFeb 2024-182.0

Resources

HFallenai/OLMo-7B GHallenai/OLMo DOCSallenai.org/olmo

API price weights · each benchmark row carries its own source badge (see methodology)