Archived leaderboard

Corpus 299-21b69d52efdc, revision 2026-10-28, published 2026-10-28, grader 2026-10-10. This snapshot is immutable — cite this URL.

Revisions of this corpus: 2026-10-31 (27 runs) · 2026-10-28 (27 runs)

Rank Model Track k Score/100q Correct Incorrect Abstained No answer Truncated Reasoning Grounded Hallucination Cost
1–4 claude-opus-5.5 closed 1 +27.1 (±10.0) 136/295 28 131 0 0 98%
[88–100]
37%
[32–43]
22%
[16–31]
$4.75
1–4 grok-4.7 closed 1 +25.1 (±6.4) 86/295 6 203 0 0 91%
[78–96]
19%
[14–24]
6%
[2–16]
$3.11
1–4 claude-fable-5.1 closed 1 +24.1 (±10.9) 143/295 36 116 0 0 100%
[92–100]
40%
[34–46]
26%
[20–34]
$14.29
1–4 claude-sonnet-5.5 closed 1 +10.5 (±12.0) 131/295 50 114 0 0 100%
[92–100]
35%
[29–41]
36%
[29–45]
$2.35
5 claude-haiku-5.5 closed 1 +4.7 (±9.7) 80/295 33 182 0 0 95%
[85–99]
15%
[12–20]
44%
[33–56]
$0.13
6–8 gemma4-12b closed 1 -73.0 (±14.0) 33/256 110 113 0 39 73%
[56–86]
5%
[3–9]
90%
[83–94]
$0.00
6–8 gpt-oss-20b closed 1 -93.5 (±13.6) 41/294 158 93 2 1 68%
[52–80]
6%
[3–9]
91%
[86–95]
$0.00
6–8 deepseek-r1-14b closed 1 -95.2 (±13.4) 36/294 158 100 0 1 52%
[38–67]
6%
[3–9]
91%
[85–94]
$0.00
1–2 claude-fable-5.1 mcp 2 +88.0 (+87…+88) 543/590 12 34 1 0 98%
[92–99]
91%
[88–93]
2%
[1–4]
$203.36
1–2 claude-opus-5.5 mcp 3 +87.5 (+85…+91) 808/885 17 27 33 0 98%
[93–99]
94%
[92–96]
2%
[1–3]
$91.39
3 claude-sonnet-5.5 mcp 3 +85.3 (+83…+87) 807/885 26 40 12 0 95%
[89–97]
92%
[90–94]
3%
[2–4]
$34.97
4 grok-4.7 mcp 2 +79.5 (+78…+81) 501/590 16 73 0 0 94%
[87–97]
83%
[80–86]
3%
[2–5]
$39.32
5 claude-haiku-5.5 mcp 3 +70.5 (+67…+75) 728/885 52 91 14 0 91%
[85–95]
82%
[79–85]
7%
[5–9]
$3.62
6 gemma4-12b mcp 3 +18.2 (+14…+22) 494/859 169 195 1 26 66%
[57–74]
56%
[53–60]
26%
[23–30]
$0.00
7 gpt-oss-20b mcp 3 +10.3 (+8…+13) 431/885 170 26 258 0 71%
[62–79]
68%
[64–72]
28%
[25–32]
$0.00

Score/100q = (correct − 2×incorrect) per 100 gradeable questions; abstention is free. The −2 penalty encodes a 67% confidence threshold. Gradeable = total minus transport errors and truncated completions, and the Correct column is shown over that same denominator. The small figure beside each score is its uncertainty: for k>1 the lowest and highest score/100q observed across those repeat runs, and for k=1 a ±95% interval — repeats re-ask the same questions, so treating them as independent trials would understate the spread. Shared ranks (e.g. 1–10) group models whose interval overlaps the group leader's. Hallucination = incorrect ÷ (correct + incorrect) on grounded questions. Cells with n<10 show raw fractions; percentages carry Wilson 95% intervals.

What tools actually buy: trust vs. accuracy, closed-book → MCP-grounded
Trust — Score/100q Grounded accuracy — % -50 0 50 0 25 50 75 100 gpt-oss-20b +104 +63pp gemma4-12b +91 +51pp claude-sonnet-5.5 +75 +57pp claude-haiku-5.5 +66 +67pp claude-fable-5.1 +64 +52pp claude-opus-5.5 +60 +57pp grok-4.7 +54 +65pp

closed-book MCP-grounded amber Δ = accuracy and trust moved in opposite directions