Archived leaderboard

Corpus 180-0a2e675b6673, revision 2026-10-12, published 2026-10-11, grader 2026-10-10. This snapshot is immutable — cite this URL.

Rank Model Track k Score/100q Correct Incorrect Abstained No answer Truncated Reasoning Grounded Hallucination Cost
1–5 claude-opus-5.5 closed 1 +43.3 (±11.5) 99/178 11 68 0 0 100%
[92–100]
41%
[34–50]
16%
[9–27]
$2.58
1–5 grok-4.7 closed 1 +36.5 (±8.0) 69/178 2 107 0 0 95%
[85–99]
21%
[15–28]
3%
[1–17]
$2.27
1–5 claude-fable-5.1 closed 1 +32.0 (±13.9) 97/178 20 61 0 0 100%
[92–100]
40%
[32–48]
27%
[18–38]
$7.05
1–5 claude-sonnet-5.5 closed 1 +23.0 (±14.8) 91/178 25 62 0 0 98%
[88–100]
36%
[29–45]
33%
[23–44]
$1.27
1–5 claude-haiku-5.5 closed 1 +22.5 (±12.1) 70/178 15 93 0 0 100%
[92–100]
20%
[14–28]
36%
[23–51]
$0.08
6–8 gemma4-12b closed 1 -56.5 (±18.7) 35/161 63 63 0 17 70%
[53–83]
9%
[5–16]
82%
[70–89]
$0.00
6–8 gpt-oss-20b closed 1 -79.1 (±18.4) 36/177 88 52 1 1 63%
[48–76]
7%
[4–13]
88%
[79–93]
$0.00
6–8 deepseek-r1-14b closed 1 -82.9 (±17.9) 29/175 87 59 0 3 55%
[40–69]
5%
[3–10]
91%
[82–95]
$0.00
1–2 claude-fable-5.1 mcp 2 +80.3 (+80…+81) 322/356 18 9 7 0 99%
[94–100]
90%
[86–93]
7%
[4–10]
$174.79
1–2 claude-opus-5.5 mcp 3 +79.8 (+78…+81) 458/534 16 16 44 0 98%
[93–99]
92%
[89–94]
4%
[2–6]
$62.89
3–4 claude-sonnet-5.5 mcp 3 +69.3 (+68…+70) 456/534 43 14 21 0 93%
[87–96]
88%
[84–90]
9%
[7–13]
$25.49
3–4 grok-4.7 mcp 2 +68.8 (+69…+69) 285/356 20 51 0 0 97%
[90–99]
75%
[69–80]
8%
[5–13]
$30.73
5 claude-haiku-5.5 mcp 3 +64.0 (+62…+66) 422/534 40 50 22 0 92%
[86–96]
79%
[75–83]
11%
[8–14]
$2.55
6 gpt-oss-20b mcp 3 +13.7 (+13…+15) 277/534 102 17 138 0 73%
[64–80]
69%
[63–74]
27%
[22–32]
$0.00
7 gemma4-12b mcp 3 0.0 (-12…+6) 256/510 128 125 1 24 71%
[62–78]
44%
[40–49]
38%
[33–44]
$0.00

Score/100q = (correct − 2×incorrect) per 100 gradeable questions; abstention is free. The −2 penalty encodes a 67% confidence threshold. Gradeable = total minus transport errors and truncated completions, and the Correct column is shown over that same denominator. The small figure beside each score is its uncertainty: for k>1 the lowest and highest score/100q observed across those repeat runs, and for k=1 a ±95% interval — repeats re-ask the same questions, so treating them as independent trials would understate the spread. Shared ranks (e.g. 1–10) group models whose interval overlaps the group leader's. Hallucination = incorrect ÷ (correct + incorrect) on grounded questions. Cells with n<10 show raw fractions; percentages carry Wilson 95% intervals.

What tools actually buy: trust vs. accuracy, closed-book → MCP-grounded
Trust — Score/100q Grounded accuracy — % -50 0 50 0 25 50 75 100 gpt-oss-20b +93 +61pp gemma4-12b +57 +35pp claude-fable-5.1 +48 +50pp claude-sonnet-5.5 +46 +51pp claude-haiku-5.5 +42 +59pp claude-opus-5.5 +37 +50pp grok-4.7 +32 +54pp

closed-book MCP-grounded amber Δ = accuracy and trust moved in opposite directions