Model

GPT-Neo 2.7B

OPEN

EleutherAI · GPT-Neo family · released Mar 21, 2021

First open GPT-3-style replication trained on The Pile; largest freely available autoregressive LM at its March 2021 launch

ReasoningCodingVisionFunction callingTool useAgentic

-431.6

Elo · rank #444

Parameters

2.7B

Active params

2.7B (dense)

Context

2K tokens

Architecture

Decoder-only Transformer, alternating local (256-token window) and global sparse attention (GPT-3 replication)

License

MIT

Languages

API price (in/out)

No hosted API

Modalities

text

Benchmark results

Bar shows position within the tracked field; marker = field best

ARC-ChallengeReasoning33.4%#159

best: Llama 3.1 405B · 96.9%

BIG-Bench HardReasoning31.4%#129

best: ERNIE 4.5 300B-A47B · 94.3%

DROPReasoning4.77#73

best: Hunyuan-T1 · 93.1

GPQA DiamondReasoning26.6%#272

best: GPT-5.6 · 94.6%

GSM8KMath1.3%#198

best: Llama 3.1 405B · 96.8%

HellaSwagReasoning42.7%#152

best: Claude 3 Opus · 95.4%

IFEvalReasoning25.9%#144

best: Gemma 4 26B A4B · 98.5%

LAMBADAReasoning62.2%#26

best: GPT-3 175B · 86.4%

MMLU-ProKnowledge11.6%#179

best: Claude Fable 5 · 91.5%

MMLUKnowledge26.4%#281

best: OpenAI o3 · 92.9%

PIQAReasoning72.1%#74

best: GPT-4o mini · 93.1%

PubMedQAKnowledge57.5%#10

best: Palmyra Med 70B · 79.6%

TruthfulQAKnowledge39.8%#82

best: Phi-3.5-MoE (16x3.8B, 6.6B active) · 77.5%

WinoGrandeReasoning56.5%#135

best: PaLM 2 · 90.9%

Run it locally

VRAM @ Q4

2.2 GB

VRAM @ FP16

6.4 GB

Fits on (Q4)

RTX 3060 12GBRTX 4070 Ti 16GBRTX 3090 24GBRTX 4090 24GBRTX 5090 32GBM4 Pro 48GBM3 Max 128GBM3 Ultra 512GBA100 80GBH100 80GBH200 141GBB200 192GB

Throughput data unavailable.

Quantizations

Int8

GPT-Neo family

Elo progression across releases

GPT-Neo 2.7BMar 2021-431.6

Resources

HFEleutherAI/gpt-neo-2.7B GHEleutherAI/gpt-neo DOCShuggingface.co/docs/transformers/model_doc/gpt_neo

API price weights · each benchmark row carries its own source badge (see methodology)