Can you trust gpt-oss 20B for space work?
By Anthony Vincent Caracappa, Founder and Self-Directed Space Researcher, Viventine Space Systems · updated
On its own, it earns "Trust it" on none of the jobs. Even connected to Orbit Sentinel, it earns "Trust it" on none of the jobs.
| Job | Closed-book | With Orbit Sentinel tools | n | ||||
|---|---|---|---|---|---|---|---|
| right / wrong | range | declined | right / wrong | range | declined | ||
| Catalog lookup | 7 / 62 | 2–18 | 31 | 76 / 10 | 61–86 | 14 | 45×3 |
| Constellation & fleet status | 5 / 51 | 1–15 | 44 | 29 / 36 | 18–44 | 34 | 43×3 |
| Debris & conjunction context | 12 / 64 | 5–25 | 24 | 44 / 19 | 31–59 | 37 | 42×3 |
| Regulatory & licensing | 8 / 60 | 3–20 | 33 | 13 / 25 | 6–27 | 62 | 40×3 |
| Company & market intelligence | 7 / 59 | 3–19 | 34 | 46 / 21 | 32–61 | 33 | 41×3 |
| Ground infrastructure | 9 / 45 | 3–28 | 45 | 23 / 18 | 10–43 | 59 | 22×3 |
| Baseline knowledge | 75 / 17 | 55–88 | 8 | 85 / 7 | 67–94 | 8 | 25×3 |
| Orbit Sentinel-only questions | 0 / 38 | 0–23 | 62 | 67 / 28 | 40–86 | 5 | 13×3 |
% right / % wrong per cell; color is the verdict (green trust it, amber verify, red don't use). Range is the 95% interval for % right, counting each question once; n is questions × runs for the tools track.
- How it was run
- Open weights, run locally (Ollama on one RTX 4090)
- Tool loop
- SpaceBench tool loop
- Runs
- 1 on its own, 3 with tools; rates pool all runs, ranges count each question once
- Verdict rule
- Trust it: wrong ≤ 5% and right ≥ 80%. Don't use it: wrong > 20%. Otherwise verify its answers.
Limits, harness differences and corrections: methodology.