Model

Mistral NeMo 12B

OPEN

Mistral AI · Mistral NeMo family · released Jul 18, 2024

12B dense model co-built with NVIDIA; 128k context, Apache 2.0, new Tekken tokenizer; now in Mistral's retirement window.

ReasoningCodingVisionFunction callingTool useAgentic

435.2

Elo · rank #324

Parameters

12B

Active params

12B (dense)

Context

128K tokens

Architecture

Dense Transformer (40 layers, 5120d, Tekken tokenizer)

License

Apache 2.0

Languages

11+

API price (in/out)

$0.15 / $0.15

Modalities

text

Benchmark results

Bar shows position within the tracked field; marker = field best

HellaSwagReasoning83.5%#56

best: Claude 3 Opus · 95.4%

LogicKorHuman preference5.9#27

best: GPT-4o · 9.33

MMLU (EU-21 languages)Knowledge59.5%#3

best: Llama 3.1 70B · 77.1%

MMLU-ProKnowledge44.8%#145

best: Claude Fable 5 · 91.5%

MMLUKnowledge68.0%#160

best: OpenAI o3 · 92.9%

OpenBookQAReasoning60.6%#12

best: Claude 1 · 90.8%

TriviaQAKnowledge73.8%#18

best: Sarvam-1 (2B) · 90.6%

TruthfulQAKnowledge50.3%#54

best: Phi-3.5-MoE (16x3.8B, 6.6B active) · 77.5%

WinoGrandeReasoning76.8%#53

best: PaLM 2 · 90.9%

Run it locally

VRAM @ Q4

7 GB

VRAM @ FP16

24 GB

Fits on (Q4)

RTX 3060 12GBRTX 4070 Ti 16GBRTX 3090 24GBRTX 4090 24GBRTX 5090 32GBM4 Pro 48GBM3 Max 128GBM3 Ultra 512GBA100 80GBH100 80GBH200 141GBB200 192GB

Throughput data unavailable.

Quantizations

GGUF Q4 · MLX · AWQ

Mistral NeMo family

Elo progression across releases

Mistral NeMo 12BJul 2024435.2

Resources

HFmistralai/Mistral-Nemo-Instruct-2407 DOCSdocs.mistral.ai/models/overview APIAvailable via Mistral AI API

API price $0.15/$0.15 · each benchmark row carries its own source badge (see methodology)