Model

Vicuna 13B v1.5

OPEN

LMSYS (UC Berkeley/CMU/Stanford/UCSD) · Vicuna family · released Aug 1, 2023

Llama-2-13B fine-tuned on ~125K ShareGPT chats; the more-permissive, commercially-usable successor to the original LLaMA-based Vicuna.

ReasoningCodingVisionFunction callingTool useAgentic

135.2

Elo · rank #375

Parameters

13B

Active params

13B (dense)

Context

4K tokens

Architecture

Llama-2-13B fine-tuned dense transformer (SFT on ShareGPT); a 16K-context variant was also released

License

Llama 2 Community License Agreement

Languages

—

API price (in/out)

No hosted API

Modalities

text

Benchmark results

Bar shows position within the tracked field; marker = field best

ARC-ChallengeReasoning56.6%#102

best: Llama 3.1 405B · 96.9%

Arena EloHuman preference1041#51

best: Claude Fable 5 · 1505

DROPReasoning27.73#61

best: Hunyuan-T1 · 93.1

GSM8KMath11.3%#176

best: Llama 3.1 405B · 96.8%

HellaSwagReasoning81.1%#72

best: Claude 3 Opus · 95.4%

MMLU (EU-21 languages)Knowledge44.8%#11

best: Llama 3.1 70B · 77.1%

MMLUKnowledge55.8%#220

best: OpenAI o3 · 92.9%

MT-BenchHuman preference6.57#60

best: Hunyuan-Large (A52B) · 9.4

TruthfulQAKnowledge51.1%#52

best: Phi-3.5-MoE (16x3.8B, 6.6B active) · 77.5%

WinoGrandeReasoning74.7%#70

best: PaLM 2 · 90.9%

Run it locally

VRAM @ Q4

9 GB

VRAM @ FP16

26 GB

Fits on (Q4)

RTX 3060 12GBRTX 4070 Ti 16GBRTX 3090 24GBRTX 4090 24GBRTX 5090 32GBM4 Pro 48GBM3 Max 128GBM3 Ultra 512GBA100 80GBH100 80GBH200 141GBB200 192GB

Throughput data unavailable.

Quantizations

GGUF Q4 · GPTQ · AWQ

Vicuna family

Elo progression across releases

Vicuna 13B (v1.1)Apr 202371.3 Vicuna 13B v1.5Aug 2023135.2

Resources

HFlmsys/vicuna-13b-v1.5 GHlm-sys/FastChat APIAvailable via LMSYS (UC Berkeley/CMU/Stanford/UCSD) API

API price weights · each benchmark row carries its own source badge (see methodology)