When LLMs Show Their Work, Can You Audit the Reasoning?

A reasoning trace can tell you what the model said it was doing. It cannot, by itself, tell you that the model actually did it. That distinction, now measurable thanks to a single-team, unreplicated September 2026 preprint from New York University and NYU Shanghai, is the answer to the audit question this article’s title poses: a chain-of-thought (CoT) trace is a triage and explanation layer, not a certification control, unless its faithfulness has been independently measured against the model’s internals.

The practical consequence lands on two groups at once. Engineers monitoring coding agents have been treating “show your work” traces as oversight signal, reading them the way a reviewer reads a diff. Product leads have been presenting those same traces to stakeholders as explanations. Both uses assume the narrative is load-bearing. The evidence says assume otherwise until proven otherwise, and the new measurement work is useful precisely because it gives “proven otherwise” a concrete, scoreable meaning.

The question behind the reasoning panel

Most agent frameworks ship some version of a reasoning panel: a scrollable log of the model’s apparent deliberation before each action. It feels like an audit trail. It is worth being precise about what kind of trail it actually is.

An audit trail, in the engineering sense, certifies that a recorded sequence of events corresponds to what the system did. A CoT trace certifies something weaker: that the model produced a coherent-sounding account of a reasoning process. Whether that account corresponds to the computation that produced the answer is exactly the open question. The trace might be faithful, it might be post-hoc rationalization, or it might be actively divergent, and reading it fluently cannot distinguish the three.

This is not a philosophical worry. The CIA preprint puts it in one sentence, citing prior work by Pfau et al. (2024) and Goyal et al. (2024):

“the CoT traces can be even manipulated without affecting the model’s output”

Read that the way a security engineer would read it. The observation channel can be altered independently of the system’s behavior. Any monitoring design that treats that channel as evidence of behavior is reading a persuasion surface.

The new measurement: scoring the trace against the internals

What changed in September 2026 is that the disagreement between stated and actual reasoning became a number. The NYU / NYU Shanghai team (again, single-team, unreplicated, lab-only; hold that thought until the limitations section) proposes CoT-Interpretability Alignment, or CIA, defined in the paper’s own abstract as “a metric that measures the agreement between a model’s CoT traces and its internal reasoning strategies as detected by interpretability tools.”

The mechanism matters for the decision. CIA does not ask whether the trace is plausible, well-structured, or correlated with a correct answer. It asks whether the reasoning strategy that interpretability tools find operating inside the model matches the strategy the trace describes. That is a faithfulness claim, and it is the only kind of claim that could ever justify treating a trace as a control.

The same paper reports that this property is trainable: post-training with CIA and task accuracy as rewards, the authors write, showed “we can substantially improve CoT parametric faithfulness while maintaining the task accuracy.” Two readings follow. The optimistic one: “show your work” quality is a property a lab can optimize for, so traces could someday carry audit weight. The less comfortable one: if faithfulness is a training objective, it is also a surface a model can be trained to satisfy, and a measured score is only as trustworthy as the probe behind it. Either way, the paper’s own framing implies that unmeasured traces are not evidence. Faithfulness is something you establish, not something you read off the prose.

What output-only testing already buys you

While the interpretability argument was maturing, the evaluation mainstream made a quieter choice: stop reading the middle of the reasoning entirely. The Chain-of-Thought Hub, a running effort to measure reasoning performance, states its protocol plainly:

“When evaluating, we only use the final answer accuracy but do not consider the correctness of intermediate steps. This is because empirically, the correctness of intermediate steps is strongly correlated with the final accuracy.”

This is the established fallback, and it is a defensible floor. Final answers are cheap to test against ground truth: does the patch apply, does the test suite pass, does the proof check. And if intermediate-step correctness correlates strongly with final accuracy, then a model that answers correctly was, more often than not, reasoning correctly in some sense.

But notice what the floor does not cover. Correlation between step correctness and answer correctness says nothing about whether the stated steps match the actual steps. A model could compute correctly and narrate incorrectly, which is exactly the failure CIA is designed to catch and exactly the failure that matters when a stakeholder reads the trace as an explanation. Output tests certify outcomes. They do not certify narratives, and they were never designed to.

There is also a hedge in the other direction worth keeping: the correlation finding is empirical and task-scoped. It justifies output tests as a baseline, not as proof that any particular model’s traces are honest on any particular task or language.

Where trace review fails even on its own terms

Set aside manipulation and post-hoc rationalization for a moment. Even a genuinely faithful trace has failure modes that reading cannot catch.

The first is compounding error. The DiffCoT paper describes it: “CoT reasoning is highly sensitive to potential errors, where mistakes introduced in the early stages can propagate through later steps, often leading to incorrect final answers.” A trace can be internally coherent, faithfully reflecting a chain of computation, and systematically wrong from step two onward. A reviewer scanning for plausibility will not see that, because the failure is not in the narration but in the arithmetic of the reasoning itself. Output tests catch this; trace review does not.

The second is fluency dependence. A cross-lingual visual reasoning audit found CoT prompting “counterproductive in Indian languages, with the exception of Marathi,” because “the reasoning chain becomes a bottleneck when the model lacks the fluency to sustain coherent logic in the target language.” In that audit, GPT-4o reached 43.2% accuracy on Indian-language tasks, the highest reported. Gemma 3-27B showed the smallest average drop from English, at 9.8 percentage points. The lesson generalizes beyond language: trace quality rides on the model’s expressive capacity, not on the quality of the underlying decision. A team auditing multilingual agents, or agents operating near the edge of a model’s competence, will read garbled intermediate steps that corrupt the final answer.

The oversight matrix

Putting the three options side by side clarifies what each one can honestly certify:

The row that decides most real deployments is the third one’s gate. CIA requires looking inside the model. If you consume a frontier model through a closed API, you do not have that access, which means you cannot currently convert trace-reading into a measured control on the models where oversight stakes are highest. The metric’s existence is not the barrier; the access requirement is.

The governance vacuum that narratives fill

There is a reason teams lean on traces despite all of this, and it is structural rather than naive. Agent systems have an attribution problem. Work on mapping human anti-collusion mechanisms to multi-agent AI systems names it directly: without clear attribution of emergent coordination to specific agents, sanctions risk “being arbitrary or unfairly penalizing deploying organizations for emergent behaviors they did not explicitly program.” When you cannot establish why a fleet of agents did something, the volunteered narrative becomes the only explanation on the table. Fluency fills the vacuum that instrumentation leaves.

Contrast that with a monitoring design that never asks the model to explain itself. A pilot study of ten public document streams from U.S. and PRC government-related sources detected AI-assisted writing statistically: 2021 baselines were consistently near zero, and by 2026 four of ten streams showed statistically significant signals. The point is not the geopolitics. The point is that oversight was built on detection over observable output, with no cooperation from the author and no narrative required. That is what a control looks like when the system being monitored cannot be trusted to describe itself. Coding-agent oversight has the same shape available: tests, reproducibility checks, behavioral statistics over action streams. It just requires building them instead of reading the panel.

A working checklist: when does reading the trace count as a control?

Given the evidence, I would apply these conditions in order. Reading a CoT trace counts as an oversight control only when:

1. Faithfulness has been measured, not assumed. A CIA-style score against the model’s internals, or an equivalent independent probe, exists for this model and this task class. A fluent trace satisfies nothing on this list.
2. The task has ground truth you test separately. The trace is triage; the output test is certification. If you cannot test the output at all, you have a harder problem than trace-reading can solve, and the trace should be treated as a hypothesis generator, not evidence.
3. The model operates in its fluent regime. The cross-lingual audit shows traces degrade into bottlenecks outside it. Multilingual or edge-of-competence deployments need lower trust in the narrative by default.
4. The decision the trace informs is reversible. Traces are good at prioritizing human attention: which agent run to inspect, which failure to reproduce first. That is a genuine use. It fails safely because a wrong triage call costs review time, not correctness.
5. No one downstream reads the trace as an explanation of record. If a stakeholder, auditor, or customer will see it, label it for what it is: the model’s account, unverified against its internals.

Condition one currently fails for most closed-API deployments, which means for most teams the honest answer is that trace-reading is not a control today. It is a convenience with the ergonomics of a control.

Limitations worth stating plainly

Everything CIA-specific in this article rests on a single-team, unreplicated preprint reporting lab metrics on three tasks (two-hop QA, hint intervention, integer multiplication) across three open models, with no deployment evidence. The reported result that faithfulness improves while accuracy holds is encouraging and narrow: it does not establish that the technique works on closed frontier models, in production agent fleets, on coding tasks, or under adversarial pressure, and the manipulation findings it cites are 2024 work being re-cited, not new evidence. The strongest case for trace-reading as a control is, at present, a measurement that most operators cannot run.

The counter-evidence brackets the question from both sides. Traces are manipulable without output change, per the CIA paper’s own citations. Yet output-only evaluation has an empirical floor, per the Chain-of-Thought Hub, because intermediate-step correctness correlates with final accuracy. Neither fact settles whether your model’s trace was honest on Tuesday’s deployment. That is the gap the checklist exists to manage: treat the trace as the beginning of an investigation, put the certification load on tests you control, and demand a measured faithfulness score from any vendor who wants the reasoning panel to count as oversight. Until that score exists and someone outside one lab has replicated it, “show your work” is a request for a story. Useful, sometimes true, and not an audit.

Frequently Asked Questions

What is the practical gate for using CIA-style oversight on a model?

The row that decides most real deployments is the third one’s gate. CIA requires looking inside the model. If you consume a frontier model through a closed API, you do not have that access, which means you cannot currently convert trace-reading into a measured control on the models where oversight stakes are highest. The metric’s existence is not the barrier; the access requirement is.