Archived leaderboard

Corpus 299-21b69d52efdc, revision 2026-10-31, published 2026-10-31, grader 2026-10-31b. This snapshot is immutable — cite this URL.

Revisions of this corpus: 2026-10-31 (27 runs) · 2026-10-28 (27 runs)

Rank Model Track k Score/100q Correct Incorrect Abstained No answer Truncated Reasoning Grounded Hallucination Cost
1–4 claude-opus-5.5 closed 1 +28.4 (±10.0) 137/292 27 128 0 0 100%
[92–100]
38%
[32–44]
22%
[16–31]
$4.71
1–4 grok-4.7 closed 1 +26.4 (±6.2) 87/292 5 200 0 0 93%
[81–98]
19%
[15–24]
6%
[2–16]
$3.09
1–4 claude-fable-5.1 closed 1 +24.3 (±11.0) 143/292 36 113 0 0 100%
[92–100]
40%
[34–46]
26%
[20–34]
$14.19
1–4 claude-sonnet-5.5 closed 1 +10.6 (±12.2) 131/292 50 111 0 0 100%
[92–100]
35%
[30–41]
36%
[29–45]
$2.33
5 claude-haiku-5.5 closed 1 +4.8 (±9.8) 80/292 33 179 0 0 95%
[85–99]
16%
[12–21]
44%
[33–56]
$0.13
6–8 gemma4-12b closed 1 -73.9 (±14.2) 33/253 110 110 0 39 73%
[56–86]
5%
[3–9]
90%
[83–94]
$0.00
6–8 deepseek-r1-14b closed 1 -94.5 (±13.5) 37/291 156 98 0 1 55%
[40–69]
6%
[3–9]
91%
[85–94]
$0.00
6–8 gpt-oss-20b closed 1 -94.5 (±13.7) 41/291 158 90 2 1 68%
[52–80]
6%
[3–9]
91%
[86–95]
$0.00
1–2 claude-fable-5.1 mcp 2 +90.6 (+90…+91) 543/584 7 34 0 0 100%
[96–100]
92%
[89–94]
2%
[1–3]
$198.65
1–2 claude-opus-5.5 mcp 3 +90.0 (+88…+93) 808/876 10 27 31 0 100%
[97–100]
95%
[93–96]
1%
[1–3]
$89.36
3 claude-sonnet-5.5 mcp 3 +87.1 (+85…+89) 805/876 21 40 10 0 95%
[90–98]
93%
[90–94]
2%
[1–4]
$34.14
4 grok-4.7 mcp 2 +81.3 (+79…+84) 501/584 13 70 0 0 94%
[87–97]
84%
[81–87]
2%
[1–4]
$38.32
5 claude-haiku-5.5 mcp 3 +73.1 (+69…+77) 728/876 44 91 13 0 92%
[86–96]
83%
[80–86]
6%
[5–8]
$3.52
6 gemma4-12b mcp 3 +20.0 (+17…+23) 494/851 162 194 1 25 67%
[58–75]
57%
[53–60]
26%
[22–29]
$0.00
7 gpt-oss-20b mcp 3 +12.3 (+10…+15) 432/876 162 26 256 0 74%
[65–81]
69%
[65–73]
28%
[24–32]
$0.00

Score/100q = (correct − 2×incorrect) per 100 gradeable questions; abstention is free. The −2 penalty encodes a 67% confidence threshold. Gradeable = total minus transport errors and truncated completions, and the Correct column is shown over that same denominator. The small figure beside each score is its uncertainty: for k>1 the lowest and highest score/100q observed across those repeat runs, and for k=1 a ±95% interval — repeats re-ask the same questions, so treating them as independent trials would understate the spread. Shared ranks (e.g. 1–10) group models whose interval overlaps the group leader's. Hallucination = incorrect ÷ (correct + incorrect) on grounded questions. Cells with n<10 show raw fractions; percentages carry Wilson 95% intervals.

What tools actually buy: trust vs. accuracy, closed-book → MCP-grounded
Trust — Score/100q Grounded accuracy — % -50 0 50 0 25 50 75 100 gpt-oss-20b +107 +63pp gemma4-12b +94 +52pp claude-sonnet-5.5 +76 +57pp claude-haiku-5.5 +68 +67pp claude-fable-5.1 +66 +52pp claude-opus-5.5 +62 +57pp grok-4.7 +55 +65pp

closed-book MCP-grounded amber Δ = accuracy and trust moved in opposite directions