Model

Guanaco 65B

OPEN

University of Washington (Dettmers et al.) · Guanaco family · released May 23, 2023

4-bit QLoRA fine-tune of LLaMA 65B on OASST1, reaching 99.3% of ChatGPT's performance on the Vicuna benchmark in ~24 GPU-hours on one 48GB GPU.

ReasoningCodingVisionFunction callingTool useAgentic

396.3

Elo · rank #332

Parameters

65B

Active params

65B (dense)

Context

2K tokens

Architecture

LLaMA (dense decoder-only transformer) + 4-bit QLoRA adapter

License

Apache-2.0 (LoRA adapter weights); base LLaMA weights remain under Meta's non-commercial research license

Languages

—

API price (in/out)

No hosted API

Modalities

text

Benchmark results

Bar shows position within the tracked field; marker = field best

AlpacaEvalHuman preference71.8%#8

best: Tülu 2+DPO 70B · 95.1%

ARC-ChallengeReasoning65.4%#63

best: Llama 3.1 405B · 96.9%

GSM8KMath26.0%#160

best: Llama 3.1 405B · 96.8%

HellaSwagReasoning86.5%#25

best: Claude 3 Opus · 95.4%

MMLUKnowledge62.2%#195

best: OpenAI o3 · 92.9%

MT-BenchHuman preference6.41#63

best: Hunyuan-Large (A52B) · 9.4

OpenAssistant BenchHuman preference1008#3

best: GPT-4 · 1294

TruthfulQAKnowledge52.8%#46

best: Phi-3.5-MoE (16x3.8B, 6.6B active) · 77.5%

Vicuna BenchHuman preference1023#2

best: GPT-4 · 1176

WinoGrandeReasoning82.4%#22

best: PaLM 2 · 90.9%

Run it locally

VRAM @ Q4

39 GB

VRAM @ FP16

130 GB

Fits on (Q4)

M3 Max 128GBM3 Ultra 512GBA100 80GBH100 80GBH200 141GBB200 192GB

65B does not fit in a single RTX 4090's 24GB VRAM even at Q4 (~39GB needed); no verified single-GPU llama.cpp measurement found.

Quantizations

GGUF Q4 · GPTQ · GGML

Guanaco family

Elo progression across releases

Guanaco 33BMay 2023280.6 Guanaco 65BMay 2023396.3

Resources

HFtimdettmers/guanaco-65b GHartidoro/qlora

API price weights · each benchmark row carries its own source badge (see methodology)