Model
Model explorer

Megatron-Turing NLG 530B

CLOSED
NVIDIA · Megatron-Turing NLG family · released Oct 11, 2021

Joint NVIDIA-Microsoft dense transformer; largest monolithic LLM at launch, a scaling landmark; weights never released.

ReasoningCodingVisionFunction callingTool useAgentic
210.0
Elo · unrated
Parameters
530B
Active params
530B (dense)
Context
2K tokens
Architecture
Dense Transformer (105 layers, 20480 hidden dim, 128 attention heads)
License
None (research paper/blog only; weights never publicly released)
Languages
—
API price (in/out)
No hosted API
Modalities
text
Benchmark results
Bar shows position within the tracked field; marker = field best
ANLIReasoning36.6%#9
best: Phi-3-small (7B) · 58.1%
HellaSwagReasoning80.2%#80
best: Claude 3 Opus · 95.4%
LAMBADAReasoning76.6%#7
best: GPT-3 175B · 86.4%
PIQAReasoning82.0%#28
best: GPT-4o mini · 93.1%
RACE-HReasoning47.9%#14
best: Claude 3 Opus · 92.9%
WinoGrandeReasoning73.0%#80
best: PaLM 2 · 90.9%
Run it locally
Closed weights — available via API only. No local deployment.
Input / M tok
—
Output / M tok
—
Megatron-Turing NLG family
Elo progression across releases
Megatron-Turing NLG 530BOct 2021210.0
Resources
GHdeepspeedai/Megatron-DeepSpeedDOCSdeveloper.nvidia.com
API price — · each benchmark row carries its own source badge (see methodology)