fixbench leaderboard
Mini AI coding-agent evaluation harness.
grader deterministic · trials 1 · tasks 5 · agents 2 · updated 2026-08-15
Leaderboard (per-agent)
| # |
agent |
mean pass@1 |
avg tokens |
avg cost |
avg latency |
regressions |
| 1 |
codex |
100% |
143,177 |
$0.0000 |
76224 ms |
0 |
| 2 |
claude-code |
100% |
569 |
$0.0982 |
13747 ms |
0 |
pass@1 grid (agent × task)
| agent \ task | 001-parse-duration | 002-paginate | 003-split-fee | 004-mask-phone | 005-order-state |
|---|
codex | 100% | 100% | 100% | 100% | 100% |
claude-code | 100% | 100% | 100% | 100% | 100% |