groundy
Models & Research

Did GPT-6.2 Astra Score 100% on ARC-AGI-3? How to Verify the Claim

GPT-6.2 Astra's claimed 100% ARC-AGI-3 score is unconfirmed. The documented 100% belongs to humans. Verify provenance, task set, and cost before routing decisions.

Published 11 references
On this page10 sections

Do not change model routing or budget on the claim that “GPT-6.2 Astra scored 100% on ARC-AGI-3.” As of November 21, 2026, no source in the research record reports that result, and the benchmark’s own paper assigns 100% to human solvability. Treat the Astra claim as unconfirmed and non-actionable, not as an established result. Every GPT-6.2 capability and pricing figure below is vendor-reported; results from solver and audit papers are author-reported by those papers, not independent measurements of Astra.

That verdict is not proof that the score was never published somewhere. The available OpenAI launch-page excerpt is truncated, and the record contains no GPT-6.2 model card or ARC Prize leaderboard page. It does establish two useful things: the number cannot presently be traced to Astra, and even a genuine ARC-AGI-3 100% would need its task set, harness, attempt rules and cost disclosed before it could support a procurement decision.

The documented 100% belongs to humans, not to Astra

The ARC-AGI-3 design paper reports that humans could solve 100% of its environments, while frontier AI systems scored below 1% as of March 2026. The earlier competition and exploration paper reports an even sharper preview result: frontier models scored 0% on the initial preview tasks where human participants reached 100%.

A garbled human baseline is therefore one plausible origin for the circulating claim. That is an inference, not a traced history of the posts. A second possibility is that a real public-set harness result lost its conditions as it was repeated. By fall 2026, author-reported 100s did exist on ARC-AGI-3’s public games, although not for GPT-6.2 Astra in the available record.

The available launch-page text of OpenAI’s GPT-6.2 Astra and Sol announcement reports results for Agents’ Last Exam, Terminal-Bench-Science 0.1, Vals Science Index, GDPval v2, AA-AnalystAgent, Harvey LAB-AA, FrontierCode 2.0 and LifeVal. That text contains no ARC-AGI-3 figure. That absence cannot establish what the full announcement or a separate model card says, but it means the launch excerpt cannot anchor the claim.

What the score actually describes

ARC-AGI-3 is an interactive reasoning benchmark. A system acts inside game-like environments and must infer their rules through interaction rather than simply answer a fixed set of text questions. That makes the evaluation conditions part of the result.

One important distinction is the unit. RHAE measures action efficiency, not a simple percentage of correct answers. A circulating “100%” does not even identify whether it refers to environments solved, RHAE, or a metric invented by a particular harness. Nor should the human baseline be read as every individual human achieving a perfect score: the ARC-AGI survey describes it as task-level solvability.

Official competition runs also had concrete limits. The ARC-AGI-3 competition paper reports an eight-hour wall-clock cap and 10 environment steps per second, shared across three private games, allowing at most 96,000 steps for a single game. Its setup used three public development games, ft09, ls20 and vc33, and three private ranking games, sp80, lp85 and as66. Separately, the Kepler audit describes ARC Prize’s revised technical rules as imposing a per-level action budget of five times the human baseline.

Attempt discipline can change the result drastically. In Kepler’s own author-reported setup, enforcing attempt-reset discipline coincided with sp80 falling from 82.16 to 14.29. Its authors explicitly warn that their scores should not be read as official-leaderboard, first-exposure results. A number produced with retries, accumulated environment knowledge or a different reset policy is not interchangeable with a first-attempt result.

There is evidence that the benchmark taps a relevant reasoning construct, but it should not be inflated into a general capability guarantee. An initial human-validation study reports, in its abstract, that ARC-AGI items tested in 100 participants correlated substantially with figural fluid intelligence, ρ = .63. The full methods are not in the research record, and that correlation does not make a saturated public-set score a predictor of coding-agent performance.

Five checks before believing any perfect score

Start with the model card, not a repost. The ARC Prize 2025 technical report notes that Anthropic, Google DeepMind, OpenAI and xAI have reported ARC-AGI performance on model cards, so that is a reasonable first checkpoint for a vendor-attributed number. Then look for the corresponding ARC Prize record and its conditions.

CheckEvidence to requireWhy a bare “100%” fails
1. ProvenanceA model-card entry with its evaluation details, or an official ARC Prize Verified result. Identify who ran the evaluation and who verified it.ARC-AGI-3’s community leaderboard accepts self-reported submissions and is unverified by default. A screenshot or repeated answer does not establish origin.
2. Task set and unitThe ARC-AGI version, exact public or private games, metric, and whether the score concerns completion or RHAE.Results from 25 public games, three competition development games, three private ranking games and a semi-private evaluation set are not interchangeable. ARC-AGI-2 results cannot be relabeled as ARC-AGI-3 results.
3. Run conditionsAction budget, wall-clock and step-rate limits, number of attempts, reset policy and first-exposure status.Extra actions or retries can turn exploration into replay. The official five-times-human action budget and competition caps define a different task from unrestricted interaction.
4. System under testModel version plus the complete agent harness: prompts, tools, observation format, supervisor logic and, where available, traces or implementation.A coding-agent supervisor or a better observation representation may change the score without any change to the underlying model. “Model X scored 100” can conceal the system that actually did the work.
5. Cost and headroomCost per task, total attempts, comparison systems and evidence that the task set still separates those systems.A perfect score cannot rank systems tied at the top, and an accuracy figure without cost can hide an impractical evaluation budget.

The provenance distinction is unusually explicit in the ARC-AGI-3 paper: “Anybody can submit to it, and scores will be self-reported.” It adds that ARC Prize does not verify community-leaderboard submissions by default and cautions against treating them as evidence of AGI progress. The official leaderboard also excludes domain-specific-harness scores. That does not make every community submission false; it means the submission itself is not verification.

For the Verified side of the process, an ARC Prize evaluation report describes Verified results as measured on a semi-private evaluation set through an official verification process and leaderboard. Its author-reported ARC-AGI-2 example also shows the expected cost convention: a human panel is listed at 100.00% and $17.00 per task, while GPT-5.2 Pro (High) is listed at 54.20% and $15.72 per task. Those are ARC-AGI-2 figures, not evidence about Astra, but they demonstrate that accuracy and cost belong in the same row.

A real 100% can still tell you little about the model

The strongest counterargument to “100% must be fake” is that public-set 100s are attainable. It is also the strongest argument against using one as a routing signal.

An analysis of all 25 public ARC-AGI-3 games reports that every one is reachable through non-intelligent strategies: 10 in a single blind step, five after one probing action, one through repeated ACTION1 presses, one through diverse exploration, and eight through single repeated actions with budgets of 50–200 steps. This does not show that those strategies would earn a verified private-set result under every official constraint. It does show why reaching or completing public environments is not, by itself, evidence of abstract reasoning.

The Kepler audit supplies the saturation result directly. In its author-reported final boards for Claude Opus 5 and GPT-5.6 Sol, 48 of 50 game, model cells reached 100. The authors conclude that public-set score alone has limited discriminative value and call for first-attempt, cost-conditioned, verification-aware reporting. They also describe a secondhand example in which AVO reported 100.00 using a supervisor-redirected general coding agent and 6,624 environment actions, without publishing an implementation, traces or cost accounting. Whatever that system achieved, the disclosure is insufficient for an apples-to-apples model comparison.

Harness identity matters because perception can dominate the search problem. A study of observation representations reports that observation geometry drives action waste and concludes, “Observation geometry dictates search complexity far more than reasoning depth.” In practical terms, two evaluations using the same model can measure different systems once one converts the environment into a more useful representation.

The same caution applies to claims of rapid model progress. An executable-world-model solver paper reports that its GPT-5.5-based agent fully solved 15 of the 25 public games with mean per-game RHAE of 58.12%, while its GPT-5.4 version solved eight with RHAE of 41.29%. These are author-reported agent results with acknowledged run-to-run variability, not raw-model scores. They provide a comparison point an Astra claim would need to specify and beat, not a baseline that can be detached from its harness.

What OpenAI’s announcement does support

The launch post supplies reasons to evaluate Astra, but they come from different benchmarks. According to OpenAI’s vendor-reported results, GPT-6.2 Astra scores 5.2 points higher than GPT-6 Astra on Agents’ Last Exam, which tests multistep professional work, at less than one-third of the cost. OpenAI also reports a 5.5-point advantage over Claude Fable 5.2 at 5% of the cost. The announcement lists coding, science, analyst and other professional evaluations as well.

According to OpenAI’s vendor-reported pricing, Astra’s API prices are $8 per million input tokens, $0.40 per million cached input tokens and $40 per million output tokens, with input and output prices 20% lower than GPT-6 Astra’s and a 60% discount on cached input. Token prices are useful for constructing an evaluation budget, but they are not cost per completed task. A system that uses more actions, retries or output tokens can change the total substantially.

Those claims justify a workload-matched trial if Astra is otherwise a candidate. They do not verify the ARC-AGI-3 story, and the ARC-AGI-3 story is not needed to justify the trial.

When should a perfect score change routing?

For an unsourced social claim, a self-reported community entry or a public-set harness 100 without full conditions, the answer is not at all. Keep the current route and verify the result first.

A verified private or semi-private result with the metric, action budget, attempt discipline, harness and cost disclosed would be stronger evidence about interactive rule induction. It could justify prioritizing Astra for an evaluation. It still would not establish that Astra is better for repository editing, terminal work or professional deliverables, so it should not directly determine production routing.

That final step needs enough tasks to survive variance and task selection. The ARC-AGI living survey reports mean performance of 65.6% for systems evaluated on fewer than 100 tasks versus 38.8% for systems evaluated on 100 or more, associating the 26.8-point difference with variance, task-selection bias and weaker detection of overfitting. It also finds that no system maintained its ARC-AGI-1 performance on ARC-AGI-2, regardless of scale, cost regime or architecture. Neither finding measures GPT-6.2 Astra; both argue against transferring a small or differently versioned evaluation into a broad routing conclusion.

The durable rule is therefore:

  1. Verify the perfect score before using it as evidence.
  2. If it is public-set and saturated, do not use it to rank models.
  3. If it is verified on private tasks, treat it as evidence about that workload, not a universal model ranking.
  4. Route on workload-matched evaluations that report quality and cost per task together.

Once every serious system reaches the ceiling, the bottleneck moves from “which model scores higher?” to “which evaluation still discriminates under conditions I can afford?”

What would change the factual verdict

A GPT-6.2 model-card entry or an official ARC Prize leaderboard row outside this research record could confirm that Astra received an ARC-AGI-3 score and change the answer to the headline question.

The decision rule would remain. Even confirmation would not supply the omitted task set, run conditions, harness or cost by itself, and no source in this record measures Astra’s actual post-launch ARC-AGI-3 ability. Unconfirmed means unverified here, not fabricated, and it certainly does not establish that Astra is weak at interactive reasoning.

Frequently Asked Questions

What run conditions apply to official ARC-AGI-3 competition runs?

The ARC-AGI-3 competition paper reports an eight-hour wall-clock cap and 10 environment steps per second, shared across three private games, allowing at most 96,000 steps for a single game. Its setup used three public development games, ft09, ls20 and vc33, and three private ranking games, sp80, lp85 and as66.

What does the ARC-AGI-3 paper say about community leaderboard verification?

The provenance distinction is unusually explicit in the ARC-AGI-3 paper: “Anybody can submit to it, and scores will be self-reported.” It adds that ARC Prize does not verify community-leaderboard submissions by default and cautions against treating them as evidence of AGI progress.

References

Follow the links in the article for context. The supporting material is collected here for further reading.

  1. OpenAI launch-page excerptopenai.comAccessed
  2. ARC-AGI-3 design paperarxiv.orgAccessed
  3. competition and exploration paperarxiv.orgAccessed
  4. ARC-AGI surveyarxiv.orgAccessed
  5. Kepler auditarxiv.orgAccessed
  6. initial human-validation studyarxiv.orgAccessed
  7. ARC Prize 2025 technical reportarxiv.orgAccessed
  8. ARC Prize evaluation reportarxiv.orgAccessed
  9. analysis of all 25 public ARC-AGI-3 gamesarxiv.orgAccessed
  10. study of observation representationsarxiv.orgAccessed
  11. executable-world-model solver paperarxiv.orgAccessed

Join the discussion

Share a useful perspective or ask a question about this article.

Discussion guidelinesComments privacy