Harness Bench Leaderboard
Seven native harnesses evaluated with ten model backends on 274 general tasks and with seven model backends on 26 multimodal tasks. Every configuration receives identical task inputs, budgets, and scoring rules while keeping its own prompting, tools, context management, and recovery behavior.
Overall Ranking
Switch between configuration, model, and harness views. Click a column header to sort.
| Loading... |
Tokens are per-task API input and output tokens in thousands, including cached input. Requests include auxiliary and delegated calls. Cost is the estimated per-task API cost at official provider prices; locally hosted Gemma-4-31B and Qwen3.8-27B have no API cost.
Score Matrix
Mean oracle score for every model-harness pair. The best harness for each model is outlined; no single harness is best on every model.
Category Breakdown
General-suite outcomes by work category. Each task contributes 70 attempts (7 harnesses × 10 models).
Harness Profile
Mean score of each harness across the models of the selected suite.
Delivery Quality and Resource Use
Pooled over all general and multimodal configuration-task evaluations, ordered by mean score. Higher expenditure does not ensure better delivery.
| Rank | Harness | Score | R0.8 | R1 | Traj. (K) | Tokens (K) | Requests | Cost ($) |
|---|---|---|---|---|---|---|---|---|
| #1 | OpenCode | 66.3 | 50.4% | 27.0% | 21.1 | 331 | 14.8 | $0.242 |
| #2 | Hermes | 65.8 | 50.0% | 26.6% | 27.0 | 310 | 17.8 | $0.244 |
| #3 | OpenClaw | 65.5 | 49.6% | 27.1% | 23.2 | 178 | 13.6 | $0.153 |
| #4 | Claude Code | 65.5 | 49.0% | 24.6% | 22.9 | 288 | 17.7 | $0.174 |
| #5 | Codex | 65.3 | 49.8% | 23.8% | 17.8 | 372 | 15.9 | $0.222 |
| #6 | Goose | 62.0 | 47.2% | 26.1% | 17.8 | 249 | 12.8 | $0.277 |
| #7 | Nanobot | 59.7 | 45.6% | 24.1% | 22.4 | 637 | 20.4 | $0.435 |
| — | All evaluations | 64.3 | 48.8% | 25.6% | 21.7 | 338 | 16.1 | $0.249 |