What Context Does a Coding Agent Need? What Input Ablations Show

ORACLE-SWE, a preprint feed-observed on 2026-10-16 (arXiv:2604.07789), answers a question most agent builders currently settle by intuition: of the signals you can feed a bug-fixing agent on top of the issue text, which ones actually move the resolution rate, and what does each cost to obtain? The paper ablates five signals: Reproduction Test, Regression Test, Edit Location, Execution Context and API Usage. The headline finding is that every tested signal helps, but not equally, and not for the reason you might assume. Replacing source code and test criteria with a model-written summary of the task requirements cost only 4% of the signal’s contribution, ORACLE-SWE reports, which suggests the agent mostly needs a clear statement of what to do, not raw artifacts. Ordering the signals differently changed results by less than 2%, per the same preprint. And the cheapest signal to extract flips depending on which benchmark you are running.

That combination turns context provisioning from a writing problem into an allocation problem. This article reads the ablation results as a budgeting decision: which signals to guarantee in every task packet, which to spend extraction effort on, and which to let the agent fetch on demand. One caveat belongs up front, because it conditions everything below: every threshold here is author-reported from a single, unreplicated preprint, measured on SWE-bench-style public repositories with oracle-injected signals. Oracle injection is an upper bound, not a deployment number.

Why the no-information baseline matters

The reason signal provisioning is the lever at all is how poorly models do without it. The SWE-bench evaluations cited in the MarsCode Agent paper found that directly applying Claude 2 and GPT-4 to issues solved only 4.8% and 1.7% of instances respectively (MarsCode Agent paper). That is the floor: a capable model, an issue, and no scaffolding for finding, understanding, or verifying a fix.

ORACLE-SWE’s own baseline is an agentic loop with no extra information injected, and against it the paper reports that “injecting any oracle signal consistently improves the success rate.” Two things follow. First, the gap between a bare model and a provisioned agent is where most of the engineering value lives, which is why context curation deserves the same rigor as model selection. Second, and this cuts the other way, a no-information baseline is far weaker than any real production setup, so the absolute contribution sizes ORACLE-SWE reports will look smaller once your baseline already includes issue text and repository access. Treat the paper’s deltas as rankings and relative magnitudes, not as gains you should expect to replicate point-for-point.

What the ablations actually show

The study’s design is simple to state: take a working agent configuration, inject one information signal at a time (an oracle version of it, meaning the correct signal rather than an agent-extracted approximation), and measure how much the success rate moves. Then vary the order, vary the agent, and vary whether the signal arrives as raw artifact or as summarized requirement.

Three findings carry most of the practical weight.

Clarified requirements beat artifacts. When the authors replaced the source code and test criteria with natural-language requirements summarized by a language model, the ORACLE-SWE paper reports, “the resulting contribution only drops by 4%, suggesting the high contribution is due to the clarified task requirements, not the leakage of test criteria that agents cannot infer.” This is the most useful result in the paper for anyone writing provisioning code, because it separates two things that are easy to conflate: the value of knowing what the task asks, and the value of having the technical artifacts. Most of the measurable value sits with the former. A well-written restatement of the issue, what behavior is wrong, what correct behavior looks like, what constraints apply, captures nearly all of what the expensive signals deliver.

Order is not a lever. Per ORACLE-SWE, “Different signal injection orders are tested with no significant performance difference (<2%).” If you have been deliberating over whether the stack trace goes before or after the issue text, the reported answer is: stop. Below the 2% effect the paper reports, ordering is noise relative to the other decisions available to you.

Signals compound. The authors report that “improving multiple fac together yields substantially larger gains than any single factor alone, because the outcome of one signal may help the model understand another signal.” The mechanism is plausible on its face: a reproduction test is more interpretable when you already know the edit location, and an edit location is more actionable when the requirements are clear. This is the argument against a minimal one-signal pipeline, and also the argument that makes budgeting necessary, because compounding gains are exactly what tempts teams to load everything.

Contribution versus cost: the ranked decision

The contribution side says more signals are better. The cost side complicates it, because obtaining a signal is not free, and the price is not stable across task distributions. ORACLE-SWE’s cost analysis (Table 10) flips depending on both the benchmark and what you measure:

Both rows are author-reported, and they disagree with each other in an instructive way: the signal that is cheapest to produce is not necessarily the signal that makes resolution cheapest. The flip also means there is no universal “always extract this first” rule. The right reading is narrower: extraction cost is a per-distribution property, so if your tasks look like SWE-bench-Verified (curated, human-validated issues), your cheapest first signal may differ from what a live-issue stream would give you. Measure it on your own task mix before committing the budget.

This is also where the oracle caveat bites hardest. ORACLE-SWE injects correct signals. A deployed system has to extract them, and extraction introduces its own failure mode: a wrong edit location or a reproduction test that tests the wrong thing is not a neutral signal, it is a costly, misleading one. The paper’s cost numbers include agent-based extraction, which is the right instinct, but the contribution numbers come from oracle injections. The gap between those two columns is where your engineering risk lives.

Does the finding transfer across agents?

Reasonably well, on the paper’s own measurements. On SWE-bench-Live under the same setting, the contribution difference for any signal between ClaudeCode and SWE-agent stayed within ±2% (ORACLE-SWE). That is a useful portability result: a signal ranking you establish on one scaffold is, per this preprint, likely to hold on another, so you do not need to re-run the full ablation when you switch agent frameworks.

The sharper boundary on transfer is not the agent, it is the model. The SWEnergy study tested small language models inside agentic issue-resolution frameworks and found that “the only configuration to achieve any success was AutoCodeRover paired with Qwen 1.7B, resolving an average of 2 out of 50 tasks, a 4% success rate” (SWEnergy). Context provisioning cannot lift a model past its capability floor. If the model cannot reason about a clarified requirement, handing it a better-clarified requirement does nothing. Before investing in a context pipeline, the SWEnergy result implies, confirm your model clears the floor with a minimal setup; otherwise the constraint is the model, and no signal ranking will fix it.

The token ledger: why “load everything” is not the safe default

The stacking result argues for more signals; the cost evidence argues for discipline. Three data points frame the tradeoff.

Overflow is a failure mode, not just a bill. SWE-Pruner reports that “in the high-impact scenario, pruning prevents catastrophic context overflow that leads to task abandonment, effectively converting failures into successes.” An agent that abandons a task because its context filled with low-value material has not saved you anything; it has spent tokens to produce nothing. The same paper posits that removing irrelevant information lets the model allocate more attention to task-critical signals, which is a mechanism claim worth treating as the authors’ interpretation rather than a settled fact.

Rich context is expensive in absolute terms. A multi-agent context-engineering case study measured “around 30-40 messages across all agents for a single task and consumed roughly 100k tokens in total (input + output),” against a single-agent loop that “might use 10k-20k tokens” for a few prompt-response turns. That system succeeded on 4 of 5 tasks versus 2 of 5 for the lean baseline, so the spend bought real outcomes on a small sample, but a cost of “about 3–5× more tokens on successful tasks,” per the same study, is a decision, not a default. At 5 tasks it is anecdote; at five thousand tasks a month it is a budget line that has to be justified per signal.

And the supply side is getting cheaper to generate. SWE-at-Scale synthesized 13,407 training transformations from historical bug fixes, fine-tuning with LoRA under a 64K-token context window. Signal-like material, realistic issue text, fix locations, reproduction cases, can increasingly be manufactured from repository history rather than hand-collected, which lowers the acquisition cost of exactly the signals ORACLE-SWE ranks. That does not change which signals matter; it changes what you pay to get them.

A provisioning checklist

The framework below is a planning heuristic derived from the ablation deltas and cost rankings. It is anchored to those numbers; it is not a validated production recipe, and no deployed pipeline in the evidence demonstrates it end to end.

Guarantee: clarified task requirements. This is the one signal the evidence supports giving every task, every time. The 4% result from ORACLE-SWE means a model-generated restatement of the issue retains nearly all of the contribution of the raw artifacts, so the guaranteed packet should include a clear statement of the wrong behavior, the expected behavior, and the known constraints, written for the agent, not pasted raw from the tracker if the tracker entry is vague. This is cheap to produce and carries the highest measured value density.

Budget: one extraction-heavy signal, chosen per task distribution. Reproduction tests and edit locations are where real extraction cost lives, and the cheapest one flips by benchmark (and by whether you count extraction cost or resolution cost). Pick based on your own measured costs on your task mix, not on the paper’s benchmark-specific rankings. Spend here only after requirements are guaranteed, because this is where the money goes.

Fetch on demand: everything else. Stack traces, surrounding comments, repository maps, historical fixes: leave these behind tool calls the agent can make when the guaranteed and budgeted signals prove insufficient. The compounding result means a second and third signal add real value when the agent actually needs them; the overflow evidence means adding them unconditionally can convert successes into abandoned tasks. On-demand fetch is how you get the stacking benefit without paying the 100k-token-per-task price on tasks that did not need it.

Skip: injection-order tuning. Below the 2% order effect ORACLE-SWE reports, this is not where your effort belongs.

If forced to compress it to one rule: I would spend the first engineering hour on requirement clarity and the second on measuring extraction cost for my own repos, in that order, because the paper’s largest reported effects sit behind those two decisions and its smallest sits behind ordering.

Limits worth keeping in view

The strongest limitation is the evidence base itself. Every quantitative threshold in this article comes from ORACLE-SWE, one unreplicated preprint: the 4% summary drop, the <2% order effect, the ±2% cross-agent difference, the benchmark-dependent cost flips. Beyond that: the signals were oracle-injected, so contributions are upper bounds on what noisy, agent-extracted signals will deliver. The benchmarks are public SWE-bench-style repositories; private monorepos with unfamiliar conventions, generated code, or sparse issue text may not behave the same way. The comparison does not include model swaps, so “context curation beats switching models” is a hypothesis the angle suggests, not a result the paper reports. And nothing here contradicts the capability floor: for models below it, the entire framework is moot.

What the preprint does establish, if its numbers hold, is a reordering of priorities. The scarce resource in a coding-agent pipeline is not prompt cleverness or signal volume; it is knowing which two or three signals carry the compounding gains, and what each costs to obtain on your tasks. That is a measurable question now, which it was not before.

Frequently Asked Questions

What is the cheapest signal to extract for SWE-bench-Verified tasks?

Does the order of signal injection significantly affect performance?

Per ORACLE-SWE, “Different signal injection orders are tested with no significant performance difference (<2%).” If you have been deliberating over whether the stack trace goes before or after the issue text, the reported answer is: stop. Below the 2% effect the paper reports, ordering is noise relative to the other decisions available to you.

How much does replacing source code with a model-written summary reduce the signal’s contribution?

Replacing source code and test criteria with a model-written summary of the task requirements cost only 4% of the signal’s contribution, ORACLE-SWE reports, which suggests the agent mostly needs a clear statement of what to do, not raw artifacts.