← All verdicts

Can you trust Grok 4.7 for space work?

By , Founder and Self-Directed Space Researcher, Viventine Space Systems · updated

On its own, trust it for baseline knowledge. Connected to Orbit Sentinel, trust it for catalog lookup, constellation & fleet status, debris & conjunction context, baseline knowledge.

Job Closed-book With Orbit Sentinel tools n
right / wrong range declined right / wrong range declined
Catalog lookup
47 / 0
33–61 53
91 / 1
79–96 8 45×2
Constellation & fleet status
19 / 0
10–33 81
95 / 5
85–99 0 43×2
Debris & conjunction context
12 / 5
5–25 83
85 / 4
71–92 12 42×2
Regulatory & licensing
28 / 0
16–43 73
44 / 4
30–59 53 40×2
Company & market intelligence
20 / 5
10–34 76
78 / 2
63–88 20 41×2
Ground infrastructure
14 / 5
5–33 82
57 / 11
37–75 32 22×2
Baseline knowledge
96 / 0
80–99 4
98 / 0
83–100 2 25×2
Orbit Sentinel-only questions
0 / 0
0–23 100
73 / 8
46–90 19 13×2

% right / % wrong per cell; color is the verdict (green trust it, amber verify, red don't use). Range is the 95% interval for % right, counting each question once; n is questions × runs for the tools track.

How it was run
xAI API
Tool loop
SpaceBench tool loop
Runs
1 on its own, 2 with tools; rates pool all runs, ranges count each question once
Verdict rule
Trust it: wrong ≤ 5% and right ≥ 80%. Don't use it: wrong > 20%. Otherwise verify its answers.

Limits, harness differences and corrections: methodology.