Benchmarks

AutomationBench

Agentsunit % · normalized over [0, 58]

Zapier's cross-application workflow-orchestration benchmark: whether an agent can complete real business tasks end to end via REST/tool calls. Every assertion must pass. Zapier's own leaderboard scores a held-out private set; several labs instead report the 600 publicly released tasks (v1.0.6), and each row's note says which set it used.

#ModelSourceScoreNormalized

Score distribution

20 tracked results across the normalization window

058

Score vs. parameters

Open-weights models, log-x params