groundy
Models & Research

Gemini 4 Neon vs Opus 5.5: How Much Agent Speed Is Worth

Gemini 4 Neon offers speed advantages over Carbon, but unresolved Elo intervals and vendor-reported costs prevent replacing Opus 5.5 for difficult agent tasks.

Published 6 references
On this page11 sections

Gemini 4 Neon is worth a trial in high-volume agent loops where waiting time is expensive, but the evidence does not support replacing Opus 5.5 on the hardest work. Google’s capability rankings and cost savings are vendor-reported; the speed observations and Agent Duel comparison are author-reported by one analyst. That analyst projects Neon at Elo 1,066 against Opus 5.5’s 1,078, but gives Neon only a 48.3% projected win rate and a 90% interval of -94 to +68 for the Elo difference. The head-to-head is unresolved, not a demonstrated 12-point defeat.

The practical consequence is that a static capability rank is no longer enough routing information. If generation speed influences users nearly as much as model identity, as the Agent Duel analysis reports, teams need to measure how much of their own task time is actually spent generating tokens, then calculate cost per accepted, finished task. Neither source supplies that measurement for a real Neon-versus-Opus deployment.

Two source kinds, one comparison

Google introduced Gemini 4 Neon on November 18, 2026, positioning it for fast end-to-end completion across software engineering, knowledge work and science. The launch post reports a Vals Index rank of #2 at 70.9%, with Pareto-frontier placement, and #3 overall on the Artificial Analysis Intelligence Index with a score of 61 at the xhigh thinking level. Google explicitly describes Neon as landing behind larger, slower models on that index.

Those are useful vendor-reported signals, but they are not a Neon-versus-Opus result. Google’s post never names Opus 5.5. Its cost comparisons are against Gemini 4 Carbon or unnamed frontier competitors, not a documented Opus workload.

The independent source asks a different question. The micjm analysis models preference votes on Agent Duel using model identity, generation speed, whether the user voted early, and an interaction between speed and early voting. Under the explicit assumption that Neon performs like Gemini 4 Carbon while generating at 127 tokens per second, it projects Neon just below Opus 5.5 on that leaderboard.

This distinction matters because the two evidence forms answer different questions:

  • Google’s results concern reported capability and cost positions under selected benchmarks and harnesses.
  • Agent Duel concerns modeled user preferences under a particular voting process.
  • Neither establishes accepted coding tasks per hour, failure rates or total spend on your repository.

The strongest reading is therefore not “rank beats speed” or the reverse. It is that routing needs both a capability gate and a workload-specific latency calculation.

The speed number is not an Opus benchmark

On launch day, micjm observed Neon being served through OpenRouter at 127 tokens per second, compared with 46 tokens per second for Gemini 4 Carbon. That is roughly a 2.8x author-reported throughput advantage over Carbon, not over Opus 5.5. The evidence contains no corresponding generation-speed measurement for Opus 5.5.

It is also one early-release observation. It does not establish steady throughput across providers, regions, context lengths, thinking levels or production loads. Treating 127 as a permanent Neon property would go well beyond the observation.

As bounded arithmetic, if two runs generated the same number of output tokens and generation were the entire task, moving from 46 to 127 tokens per second would make the generation component about 36% as long. That is not a forecast of 2.8x faster task completion. Tool execution, queueing, retrieval, review and retries remain in the wall-clock total.

The preference model nevertheless makes the observation important. Its author reports that speed appears roughly as important as model identity, with an additional speed-by-early-voting interaction. The same analysis identifies Carbon as the best model when early voting is excluded, with slowness as its main drawback. That provides a coherent rationale for the Neon projection: preserve an assumed Carbon-like capability level while removing much of the perceived waiting penalty.

It does not prove that faster output causes better work, or that the preference weighting transfers to asynchronous batch jobs. A user reacting quickly to a visible response and an unattended coding agent finishing a tested patch are different evaluation settings.

Why the interval matters more than the Elo gap

The projected snapshot puts Opus 5.5 at 1,078 and Neon at 1,066. Read alone, that suggests a small Opus advantage. The same projection gives a 90% interval of -94 to +68 for Neon’s Elo difference and a 48.3% projected win rate, counting ties as half a win.

That interval accommodates Neon ahead, Opus ahead or an effective tie. It should prevent two opposite mistakes: declaring Opus the winner from the point estimates, or declaring Neon superior because its speed story sounds more decisive. The result is conditional on one analyst’s model, one leaderboard and the assumption that Neon matches Carbon’s performance.

There is also a limit to what even a precise Elo result would settle. A preference win can combine perceived answer quality and waiting experience. It does not directly report how many attempts were needed, whether tests passed, how many tokens each model consumed or what a completed task cost.

The evidence, axis by axis

Decision axisGemini 4 NeonOpus 5.5What the routing decision can conclude
Agent Duel projectionAuthor-projected Elo 1,066, assuming Carbon-level performance and 127 tpsElo 1,078 in the analyst’s later snapshotThe 90% interval of -94 to +68 and 48.3% Neon win rate leave the comparison unresolved. Source
Generation speedAuthor-observed at 127 tps via OpenRouter on launch day; Carbon was 46 tpsNo speed measurement in the supplied evidenceDo not apply the 2.8x Neon-versus-Carbon observation to Opus.
Static capabilityVendor-reported Vals #2 at 70.9%; AAII #3 at 61 at xhighNo comparable static rank in the supplied evidenceNeon appears near the frontier in Google’s reporting, but these ranks do not settle the head-to-head. Source
Price and reported cost per taskVendor-listed at $3 per million input tokens, $0.15 per million cached input tokens and $15 per million output tokensNo Opus pricing suppliedA direct Neon-versus-Opus cost-per-task claim cannot be calculated from these sources.
Independent operational sightingNo independent end-to-end result in the supplied evidenceUsed as an LLM auditing DeepSeek and GLM trajectories in an independent paperThis confirms real use of Opus 5.5, not superior coding quality, speed or cost.

Neon’s remaining vendor-reported economics are attractive enough to justify measurement. At its default high thinking level, Google claims a 32% lower cost per task than Carbon and placement on the Pareto frontier of both Vals and Agent Arena, with roughly 50% to 80% lower cost per task than top competing frontier models. At xhigh, Google claims at least 36% lower cost per task than competitors at their highest settings. None of those claims names Opus 5.5, and the underlying workload totals are not independently reproduced in the supplied evidence.

A worksheet for time and cost per finished task

Start with your own traces, not the leaderboard snapshot. For each attempt, record:

  • I: uncached input tokens.
  • C: cached input tokens.
  • O: generated output tokens.
  • S: measured output tokens per second under the actual provider and thinking level.
  • N: non-generation time, including tools, queueing and other overhead.
  • Whether the attempt passed the same acceptance test, such as tests plus review.

A simplified per-attempt time estimate is:

T ≈ O / S + N

For retries, sum the generation and overhead across every attempt. Do not count only the successful run.

Using Google’s listed Neon prices, dollar cost per attempt is:

(3I + 0.15C + 15O) / 1,000,000

Then calculate:

cost per finished task = total spend on successful and failed attempts / number of accepted finished tasks

That denominator is where a fast but error-prone route can lose its advantage. Conversely, a slower model can justify itself if it avoids enough failed attempts. The supplied evidence cannot determine which happens here.

Token volume needs the same scrutiny. An independent agent-benchmark paper budgets 2 million processed tokens per run for Commit0 and SWE-EVO, with a 90-minute limit per SWE-EVO run. Those budgets show why generation speed can matter across long agent loops, but processed tokens are not all generated output. Multiplying the full budget by a tokens-per-second figure would produce a misleading completion-time estimate.

A separate study of coding-agent runs reports 20,087 to 22,690 agent tokens per run. Adding a delegation contract increased tokens by 13.0% and wall-clock by 38.3%, with measured wall-clock rising from 32.7 to 45.2 seconds. These were not Neon or Opus measurements. Their value here is methodological: process changes can alter overhead and token use differently, so throughput alone cannot predict task time.

For a fair model comparison, let each model have its own measured I, C, O, S and N. Holding token counts fixed is useful for an isolated arithmetic illustration, but real models differ in verbosity, tool calls, retry behavior and thinking-token spend.

What speed-weighted preferences change about leaderboards

If the Agent Duel finding holds, a capability-only leaderboard leaves out a variable that can affect user choice. That does not make preference Elo a sufficient replacement.

Static benchmarks and preference systems have different blind spots. The Open-LLM-Leaderboard paper discusses selection bias in multiple-choice evaluation and presents automatic open-style ranking as faster and cheaper than crowd-user evaluation, without human intervention. Automation changes the evaluation cost and process; it does not measure a deployment’s sensitivity to latency, tool overhead or failed attempts.

Preference data has its own risk. An audit framework for rater-state bias argues that judgments produced under shared conditions can contain structured, correlated variation, and that large annotation volume does not guarantee its statistical cancellation. This is a plausible limitation on preference labels, not a finding that Agent Duel is biased. It lowers confidence in treating its Elo as a universal routing score without invalidating the speed question.

The defensible conclusion is narrower: use capability results to establish a candidate set, use preference results to generate hypotheses about interactive experience, and use deployment traces to make the final route.

A routing playbook by workload

High-volume, latency-sensitive loops with tolerable retries: Neon is the stronger candidate for a controlled trial. The author-reported speed advantage over Carbon and Google’s vendor-reported cost-per-task claims justify that trial, especially when users are waiting on each step. Start at the default high thinking level, then measure time and spend per accepted result rather than response time alone.

The hardest tasks or failures with expensive consequences: retain an Opus 5.5-class option or your current strongest route until task-level evaluation supports a change. This is a conservative choice under uncertainty, not proof that Opus completes difficult tasks better. The projected Elo interval does not justify a blanket migration in either direction.

Tool-bound agents: inspect N before paying attention to throughput. If most elapsed time is spent executing tests, searching or waiting on external systems, faster generation may buy little. If generation dominates and a hard wall-clock deadline applies, Neon’s observed throughput becomes more consequential.

Cost-sensitive routes: compare total spend per accepted task, including abandoned attempts. Neon exposes four thinking levels, and Google reports its highest intelligence score at xhigh. High and xhigh therefore deserve separate evaluations; the cheaper setting is not cheaper if it requires more retries. Cached-input pricing can matter when your workload genuinely reuses eligible context, but it should be measured rather than assumed.

Different agent harnesses: evaluate inside the scaffold you actually operate. Google reports Neon at 12.8% on SWE-sweep with the mini-SWE harness and 39.0% in Google Antigravity. Both figures are vendor-reported and are not Opus comparisons, but their difference shows why a model name alone is an incomplete description of an agent system.

For teams able to trial it, Google says Neon is rolling out through the Gemini API in AI Studio, Gemini for business, paid Antigravity tiers for AI Pro and Ultra subscribers, and the Gemini app for Pro and Ultra subscribers, with a 1 million-token context window.

What remains unmeasured

There is no independent end-to-end measurement of either model in the supplied evidence. Neon’s 127 tps is a single early observation; its benchmark and cost-per-task wins come from Google; and the Opus comparison comes from one analyst’s conditional preference model. Opus 5.5’s appearance as a trajectory auditor confirms use, not comparative agent performance.

I would route retryable, latency-sensitive work through a measured Neon trial while preserving the existing high-capability route for difficult tasks. I would not choose a fleet-wide winner from the 1,066-versus-1,078 snapshot. The durable decision input is your own time and cost per accepted task, with speed treated as one component rather than a proxy for completion.

Frequently Asked Questions

What is the projected Elo difference between Gemini 4 Neon and Opus 5.5?

The projected snapshot puts Opus 5.5 at 1,078 and Neon at 1,066. Read alone, that suggests a small Opus advantage. The same projection gives a 90% interval of -94 to +68 for Neon’s Elo difference and a 48.3% projected win rate, counting ties as half a win.

What is the formula for calculating cost per finished task?

cost per finished task = total spend on successful and failed attempts / number of accepted finished tasks

References

Follow the links in the article for context. The supporting material is collected here for further reading.

  1. Agent Duel analysismicjm.comAccessed
  2. Introducing Gemini 4 Neonblog.googleAccessed
  3. Independent paper on LLM auditingarxiv.orgAccessed
  4. Study of coding-agent runsarxiv.orgAccessed
  5. Open-LLM-Leaderboard paperarxiv.orgAccessed
  6. Audit framework for rater-state biasarxiv.orgAccessed

Join the discussion

Share a useful perspective or ask a question about this article.

Discussion guidelinesComments privacy