CLI coding agents do not usually die on their first bad tool call. According to Failure as a Process, a July 2026 study of 1,7941 manually annotated agent trajectories, the run-killing mistake is typically a stale assumption planted in the opening turns that compounds across later edits until recovery is no longer possible. Per-turn accuracy was never the right place to look; the failure lives in the trajectory.
Do failures compound across turns, or arrive in a single mistake?
Failures compound. The paper’s central claim is that a coding-agent run rarely dies at one identifiable decision point. It organizes each trajectory into three phases: onset, where the error originates; evolution, where later turns build on top of it; and recovery, where the agent either claws its way back or does not. The finding that matters most is temporal. Errors begin within the first few execution steps, yet the run keeps going because nothing in the output has broken yet.
That gap between when a run goes wrong and when it visibly fails is the whole story. A per-turn check sees a sequence of successful tool calls. The trajectory sees an agent that has been drifting since turn two. By the time the symptom appears, usually a repair loop or a test suite that will not pass, the agent has already stacked several edits on top of the original wrong belief. What looks like the failure is the third act of a problem that started in the first.
The three-phase framing is not cosmetic. It forces annotators to timestamp when a trajectory first goes wrong, which is the only way the early-onset pattern becomes visible at all. A study that scored runs pass or fail would never see it.
What kind of error actually ends a run?
Epistemic errors, wrong beliefs about what the repository contains or what state it is in, dominate the failure modes in this corpus. The study attributes more failures to incorrect assumptions about the codebase than to tool misuse or syntax slips. That distinction is doing real work, and it is worth slowing down on.
Procedural errors are the ones existing tooling already catches: a malformed command, a tool called with the wrong arguments, a file written to the wrong path. Linters, type checkers, and argument validators exist for these. Epistemic errors have no such guardrail. The agent believes a function exists where it does not, or that a configuration is in one state when it is in another, and every following turn inherits the mistake. Nothing in the harness flags a confident wrong belief, because the harness never compares what the agent believes against what is true.
The shape of the error is familiar. The agent assumes a dependency is installed when it is not, or remembers a function signature from an earlier read that the file no longer matches. The tool call succeeds against the wrong mental model, and the run moves on.
| Dimension | Epistemic errors | Procedural errors |
|---|---|---|
| What goes wrong | Agent holds a wrong belief about repo state | Agent calls the wrong tool, or misuses one |
| Where it originates | Early turns, often the first few steps | The turn it occurs |
| Catchable by linters and validators | No | Often yes |
| Share of failures in the study | Majority | Minority |
Most existing benchmarks concentrate on procedural failures, which is the error class this corpus shows matters least. Vendor “accuracy” claims that collapse first-turn success into end-to-end completion will overshoot in exactly the dimension where runs break.
Why do early errors stay invisible until recovery fails?
A run can be wrong for many turns before anything in the output signals it. The agent acts on its early assumption, the tool call returns successfully, and because the harness validates the call rather than the belief behind it, the run proceeds. The divergence between where the agent thinks the repository is and where the repository actually is grows with each turn. Recovery becomes impossible when that delta exceeds the remaining turn budget.
This is the mechanism behind the repair loop that anyone who has watched a coding agent flail will recognize. The agent hits a failing test, makes a change, hits the test again, makes another change, and loops. From the outside it looks like the agent is incompetent at repair. The paper reframes it: the agent is competent at repair and incompetent at grounding, and it is spending the recovery budget fighting a belief error from the opening turns rather than the symptom in front of it.
This is also where the work connects to a question other recent papers are asking. AgentAbstain studies whether LLM agents know when not to act at all. An agent that cannot detect its own drift has no reason to stop, re-read the repository, and rebuild its model of state. It will keep editing into a hole it does not know it is in.
What does the Terminal-Bench evidence actually show?
The corpus is the strongest part of the paper: 3,8431 execution trajectories from seven frontier models across three coding-agent scaffolds, narrowed to 1,7941 valid runs carrying over 63,0001 manually annotated steps, per the Terminal-Bench analysis. The three scaffolds were OpenHands, MiniSWE, and Terminus2, all evaluated on Terminal-Bench.
The use of frontier models is the point. These are not weak models failing for lack of capability; they are capable models failing because capability at the single-turn level does not prevent a trajectory from drifting. That is what makes the result uncomfortable for anyone selling agents on per-turn scores.
The size matters less than the cross-scaffold consistency. Fourteen findings span failure occurrence, root causes, recovery patterns, and consistency across systems. That last category is the one to credit. The trajectory pathologies show up across OpenHands, MiniSWE, and Terminus2, which means they are not an artifact of any one harness’s prompting or tool layout. They are properties of the agent-on-repository interaction itself. A scaffold swap will not fix them.
That consistency is what licenses the generalization. If the same onset, evolution, and failed-recovery pattern holds across three independent scaffolds and seven models, then blaming the next failure on “the model got confused on step 14” is a misread. The model got confused earlier, and the scaffold let it continue.
Why do per-turn benchmarks miss the real bottleneck?
Single-turn capability on frontier models is good enough that per-turn success has stopped being the binding constraint on agent reliability. If the bottleneck has moved to trajectory management, then a harness reporting per-turn tool accuracy or step-level success is measuring the part of the problem that is largely solved.
The accounting is misleading in a specific direction. A compounding, early-onset failure produces a trajectory that looks healthy turn by turn and then fails at the end. Per-turn metrics under-report exactly the failure class that dominates this corpus, and they do it consistently. Pass-rate benchmarks inherit the same blind spot: an end-to-end pass or fail collapses the entire trajectory into one bit and hides whether the run died at step 3 or step 30.
This connects to an adjacent problem in long-horizon coding agents. When agents are scored on outcomes rather than process, reward hacking and metric-gaming become viable strategies, and the trajectory framing sharpens why: once the eval surface is the outcome, accidental drift and deliberate gaming look identical from the outside.
Where should agent observability spend go now?
If failure is trajectory-level, the recovery budget and the grounding check are where the engineering should go, not another decimal point of per-turn accuracy. The paper’s recovery-phase analysis is the actionable half of the work, because it is where the findings point toward something a builder can change.
Three changes follow. Periodic re-grounding, where the agent re-reads actual repository state and reconciles it against its cached model rather than trusting earlier reads. Divergence detection, where the harness measures the delta between what the agent believes and what is true and flags it before the run compounds further. Recovery budgets sized to the compounding rate, so the agent is not spending ten turns of repair on a two-turn-old belief error. None of these are new ideas in isolation. The paper’s contribution is to show, with the corpus, that this is where the failures actually concentrate.
The second-order effect lands on how teams buy and rank agents. A vendor reporting per-turn accuracy or headline pass rate is not lying, but it is answering the wrong question. The trajectory is the unit of failure, and the trajectory is where the eval surface has to move. Who&When Pro, a companion study on whether LLMs can attribute failures inside agent runs, arrives at the same hard problem from the other end: if failure compounds across turns, deciding which turn “caused” it is itself an open research question. Attribution and observability are two ends of the same gap, and both still assume a failure has a single home turn. It usually does not.
Frequently Asked Questions
Do these trajectory failures appear outside CLI agents, such as web-based or multimodal coding assistants?
The paper’s corpus is confined to terminal-based scaffolds on Terminal-Bench, but the mechanism (early belief drift compounding across stateful turns) generalizes to any agent that maintains a partial world model and acts on it repeatedly. Web and multimodal agents add an extra layer of noise: visual grounding errors or DOM misreads can become the original false assumption that later actions build on. The trajectory problem is scaffold agnostic; what changes is the type of early evidence the agent fails to question.
How is Terminal-Bench different from SWE-bench for studying these failures?
SWE-bench reports a final pass or fail for a task, which collapses the entire trajectory into one bit and erases the onset, evolution, and recovery phases. The Terminal-Bench runs in this study were manually annotated at the step level, enabling the authors to timestamp when an error actually started rather than when it finally surfaced. That temporal resolution is what turns a static outcome benchmark into a trajectory microscope.
What is the cheapest signal a team can add to catch drift before recovery becomes impossible?
The paper points toward divergence detection, but the cheapest implementation is a state diff between the agent’s cached assumptions and the current repository after each tool call, not a new model. If the agent believes a function signature is X and the file shows Y, flag it immediately, because epistemic errors are still cheap to fix before they are buried under dependent edits.
Can adding more re-grounding turns backfire?
Yes: every forced re-read consumes the finite turn budget and can itself be misread, giving the agent a fresh wrong belief to compound from. The paper’s recovery budget argument implies that re-grounding is only useful when targeted, so re-read only the files the agent actually touched and compare them against its current mental model, because blind re-grounding can look productive while making the trajectory worse.
What would make per-turn accuracy a useful metric again?
Per-turn accuracy would matter if agents could reliably detect and halt on their own early errors, converting trajectory failures into isolated turn failures. The companion paper Who&When Pro suggests we are not there: attribution across turns remains an open research problem, which means the agent cannot yet be its own arbiter of when a belief has drifted. Until that attribution gap closes, per-turn scores will keep measuring the wrong part of the problem.
