Model

Phi-3-small (7B)

OPEN

Microsoft · Phi-3 family · released May 21, 2024

7B dense mid-tier Phi-3 with blocksparse attention; 8K (also 128K variant), 75.7 MMLU.

ReasoningCodingVisionFunction callingTool useAgentic

615.7

Elo · rank #294

Parameters

Active params

7B (dense)

Context

8K tokens

Architecture

Dense decoder-only Transformer (alternating dense + blocksparse attention)

License

MIT

Languages

—

API price (in/out)

$0.15 / $0.6

Modalities

text

Benchmark results

Bar shows position within the tracked field; marker = field best

AGIEvalReasoning45.1%#26

best: OLMo 3-Think 32B · 88.2%

ANLIReasoning58.1%#1

best: this model · 58.1%

ARC-ChallengeReasoning90.7%#24

best: Llama 3.1 405B · 96.9%

ARC-EasyReasoning97.0%#2

best: Phi-3-medium (14B) · 97.7%

BIG-Bench HardReasoning79.1%#36

best: ERNIE 4.5 300B-A47B · 94.3%

GPQA DiamondReasoning34.3%#250

best: GPT-6 Astra · 96.0%

GSM8KMath89.6%#44

best: Llama 3.1 405B · 96.8%

HellaSwagReasoning77.0%#99

best: Claude 3 Opus · 95.4%

HumanEvalCoding61.0%#100

best: Claude Opus 4.5 · 99.4%

KoMT-BenchHuman preference4.87#6

best: LG EXAONE 3.0 7.8B Instruct · 8.92

MATH-500Math34.6%#153

best: GPT-5 · 99.4%

MBPPCoding71.7%#35

best: Llama-3.3-Nemotron-Super-49B v1 (Reasoning On) · 91.3%

MMLUKnowledge75.7%#122

best: OpenAI o3 · 92.9%

MT-BenchHuman preference8.7#19

best: Hunyuan-Large (A52B) · 9.4

OpenBookQAReasoning88.0%#3

best: Claude 1 · 90.8%

PIQAReasoning86.9%#6

best: GPT-4o mini · 93.1%

Social IQaReasoning79.2%#3

best: Apple DCLM-Baseline 7B · 82.9%

TriviaQAKnowledge58.1%#35

best: Sarvam-1 (2B) · 90.6%

TruthfulQAKnowledge70.2%#4

best: Phi-3.5-MoE (16x3.8B, 6.6B active) · 77.5%

WinoGrandeReasoning81.5%#29

best: PaLM 2 · 90.9%

Run it locally

VRAM @ Q4

6 GB

VRAM @ FP16

15 GB

Fits on (Q4)

RTX 3060 12GBRTX 4070 Ti 16GBRTX 3090 24GBRTX 4090 24GBRTX 5090 32GBM4 Pro 48GBM3 Max 128GBM3 Ultra 512GBA100 80GBH100 80GBH200 141GBB200 192GB

Throughput data unavailable.

Quantizations

GGUF Q4 · ONNX INT4

Fine-tune it

Permissive

QLoRA6.4 GB1× RTX 3060 12GB

LoRA16.6 GB1× RTX 3090 24GB

Full fine-tune114.0 GB1× H200 141GB

QLoRA SFT on ~10k samples ≈ $4.10 (1× RTX 3060 12GB)

Plan a fine-tune →

Phi-3 family

Elo progression across releases

Phi-3-mini (3.8B)Apr 2024440.4 Phi-3-medium (14B)May 2024714.8 Phi-3-small (7B)May 2024615.7

Resources

HFmicrosoft/Phi-3-small-8k-instruct GHmicrosoft/PhiCookBook DOCSazure.microsoft.com/en-us/blog/introducing-phi-3-redefining-whats-possible-with-slms APIAvailable via Microsoft API

API price $0.15/$0.6 · each benchmark row carries its own source badge (see methodology)