Model

DeciLM-7B

OPEN

Deci AI · DeciLM family · released Dec 12, 2023

AutoNAC-searched variable-GQA 7B; billed as the fastest and most accurate 7B base LLM at launch (up to 4.4x Mistral-7B throughput).

ReasoningCodingVisionFunction callingTool useAgentic

245.3

Elo · rank #356

Parameters

7.04B

Active params

7.04B (dense)

Context

8K tokens

Architecture

AutoNAC (NAS)-designed dense transformer with variable per-layer Grouped-Query Attention

License

Apache 2.0

Languages

—

API price (in/out)

No hosted API

Modalities

text

Benchmark results

Bar shows position within the tracked field; marker = field best

ARC-ChallengeReasoning59.4%#95

best: Llama 3.1 405B · 96.9%

GSM8KMath47.4%#135

best: Llama 3.1 405B · 96.8%

HellaSwagReasoning82.5%#63

best: Claude 3 Opus · 95.4%

MMLUKnowledge59.8%#207

best: OpenAI o3 · 92.9%

TruthfulQAKnowledge40.3%#79

best: Phi-3.5-MoE (16x3.8B, 6.6B active) · 77.5%

WinoGrandeReasoning80.0%#38

best: PaLM 2 · 90.9%

Run it locally

VRAM @ Q4

5 GB

VRAM @ FP16

14 GB

Fits on (Q4)

RTX 3060 12GBRTX 4070 Ti 16GBRTX 3090 24GBRTX 4090 24GBRTX 5090 32GBM4 Pro 48GBM3 Max 128GBM3 Ultra 512GBA100 80GBH100 80GBH200 141GBB200 192GB

Vendor reports 1,174 tok/s on A100 with PyTorch — not an RTX 4090 llama.cpp measurement

Quantizations

GGUF Q4 · GGUF INT8

DeciLM family

Elo progression across releases

DeciLM-7BDec 2023245.3

Resources

HFDeci/DeciLM-7B APIAvailable via Deci AI API

API price weights · each benchmark row carries its own source badge (see methodology)