Which AI can you trust for ground infrastructure?
By Anthony Vincent Caracappa, Founder and Self-Directed Space Researcher, Viventine Space Systems · updated
Earth stations, gateways, call signs and where they are.
On their own, no AI earns "Trust it" here. With Orbit Sentinel tools, none earns "Trust it" yet.
| AI | Closed-book | With Orbit Sentinel tools | n | ||||
|---|---|---|---|---|---|---|---|
| right / wrong | range | declined | right / wrong | range | declined | ||
| Claude Sonnet 5.5 Anthropic | 18 / 27 | 7–39 | 55 | 74 / 17 | 53–88 | 9 | 22×3 |
| Claude Fable 5.1 Anthropic | 27 / 23 | 13–48 | 50 | 73 / 14 | 52–87 | 14 | 22×2 |
| Claude Opus 5.5 Anthropic | 18 / 23 | 7–39 | 59 | 68 / 5 | 47–84 | 27 | 22×3 |
| Grok 4.7 xAI | 14 / 5 | 5–33 | 82 | 57 / 11 | 37–75 | 32 | 22×2 |
| Claude Haiku 5.5 Anthropic | 5 / 9 | 1–22 | 86 | 53 / 14 | 33–72 | 33 | 22×3 |
| gpt-oss 20B OpenAI | 9 / 45 | 3–28 | 45 | 23 / 18 | 10–43 | 59 | 22×3 |
| Gemma 4 12B Google | 0 / 45 | 0–15 | 55 | 19 / 58 | 8–39 | 23 | 22×3 |
| DeepSeek-R1 14B DeepSeek | 14 / 36 | 5–33 | 50 | — | 22×1 | ||
% right / % wrong per cell; color is the verdict (green trust it, amber verify, red don't use). Range is the 95% interval for % right, counting each question once; n is questions × runs for the tools track.
Sample questions
- How many applications for new earth station authority (FCC file type SES-LIC) were filed with the FCC in calendar year 2024?
- In calendar year 2018 (the year of the FCC's C-band receive-only earth station registration window), how many earth station registration filings (FCC file type SES-REG) were filed?
- How many earth station assignment or transfer-of-control applications (FCC file types SES-ASG plus SES-T/C) were filed in calendar year 2024?
- How many earth station license renewal applications (FCC file type SES-RWL) were filed in calendar year 2024?
- How many distinct applicants filed at least one new earth station license application (SES-LIC) with the FCC in calendar year 2024?
- Which company filed the most new earth station license applications (SES-LIC) with the FCC in calendar year 2024?