We ran the space benchmark three times. The winner changed.
SpaceBench round 2 · July 2026 · 83 questions · 3 runs per model · 68 runs pooled
Round 1 ran each model once and warned that its rankings were provisional — the top of the table was a statistical tie, and one lucky run could crown the wrong model. So we ran it again. Three times per model, same 83 questions, answers re-frozen from the live database first. Here's what changed, and what didn't.
1. The tools champion was luck.
closed-book MCP-grounded amber Δ = accuracy and trust moved in opposite directions
Round 1 named Grok 4.5 the tool-grounded winner on a single run of +43 points of grounded accuracy. Pooled over three runs, that run was its ceiling, not its center: Grok settles to fifth on the grounded table — third among the eight models we ran three times. Gemini 3.5 Flash takes the top line instead, gaining accuracy with tools while barely denting its calibration. But it leads on the point estimate only: the models just behind it still overlap it on uncertainty, so the leaderboard hands them a shared top rank rather than crowning one. This is exactly the failure mode repeat runs exist to catch, and we'd rather show it happening than hide it.
2. Everything else replicated.
The calibration story from round 1 held precisely. Closed-book, the GPT-5.4 family again hallucinated on 60–65% of the grounded questions they attempted, while the abstainers stayed clean across all three runs. Run-to-run variance was small for most models — the numbers were stable; only the rankings between statistically tied models shuffled, which is the honest limit of a single run.
3. Tripling the data didn't break the closed-book tie.
With three times the evidence, the statistical tie at the top of the closed-book table narrowed but didn't break: round 1's ten-way pileup is now a five-way one — Opus 4.8 leads, but four models overlap it. That isn't a limitation to fix; it's a finding. On stable space knowledge, the frontier has converged, and what separates a good model from a bad one is no longer what it knows but whether it admits what it doesn't. With tools, that consensus breaks apart — the field spreads across a far wider range, though even there the leaders overlap.
What's still true, and still not shown
Claude Fable 5 made its tools debut here and lands fourth on the grounded table — second among the eight models with three runs, and the highest raw grounded accuracy on the board, taxed by the same Claude-family tendency to over-answer with tools in hand. Five models still have only a single MCP run (marked k=1, with wide intervals). The grounded track still measures the model and our tool surface together. The round-1 write-up is preserved unchanged as the record of what one run showed. Every number here is reproducible from the current leaderboard, the published questions, and the methodology — CC BY 4.0.