Should Moderation Read the Chain of Thought on Reasoning LLMs?

If you moderate a reasoning model today by classifying its prompt and final answer, a 2025 benchmark paper and an August 2026 benchmark say your filter is blind to the largest block of text the model produces. The ReasoningShield paper reports F1 declines of up to 22.3% for LlamaGuard-4 and 17.8% for the OpenAI Moderation API when those tools are moved from answers to reasoning traces, and the TRACE benchmark found Qwen3Guard-0.6B and Qwen3Guard-4B scoring 0.00% F1 on trace safety under instruction-encryption attacks. The practical answer to the title’s question is yes, with a hard condition: add a trace-trained safety classifier as a third layer between your prompt and output filters, but do not let it replace either one, because the unfaithfulness literature shows traces can hide exactly the reasoning you are reading them to find.

There is a second consequence buried in that answer. A classifier cannot score a trace it never sees, so enabling trace moderation means capturing and storing intermediate reasoning your pipeline may currently discard. That turns a filter-placement choice into a data-retention decision, and it deserves the same review you would give any new log containing user-adjacent content.

Three surfaces, one pipeline

A safety filter on a reasoning-model deployment can sit in three places: on the incoming prompt, on the chain of thought the model generates before answering, or on the final output. Most production guardrails cover only the first and third. The TRACE authors put it plainly: existing guardrail models including LlamaGuard, PolyGuard, Qwen3Guard, and YuFeng-XGuard “are trained to assess the safety of prompts and model-generated final responses,” leaving detection of unsafe content in reasoning traces “largely unexplored”.

That gap matters because the trace is where a jailbroken or manipulated model does its work. A harmful plan can be fully elaborated in the reasoning and then laundered into a refusal-shaped or innocuous-looking answer. Output filters see only the laundered version. The question is not whether the trace contains safety-relevant content; it plainly can. The question is whether reading it is reliable enough to build on, and that is where the evidence gets interesting in both directions.

What the measurements say

The case for a trace layer is measured, not speculative. ReasoningShield reports that “leading tools like LlamaGuard-4 [33] and the OpenAI Moderation API [30] see significant performance drops when shifting from answer to reasoning trace moderation, with F1 scores declining by up to 22.3% and 17.8% respectively” (ReasoningShield’s pilot study). These are tools doing the job they were tuned for, applied to text they were not tuned on.

TRACE, an independent August 2026 benchmark, confirms the pattern and adds a nastier finding. Under instruction-encryption attacks, where the original prompt is encoded with schemes such as Caesar cipher or Base64 before the model ever reasons, “Qwen3Guard-0.6B and Qwen3Guard-4B obtain 0.00% F1-score on reasoning trace safety judgment, indicating that some guardrail models are entirely incapable of detecting unsafe content in reasoning traces under IE attacks”. Read that number carefully: it is a specific adversarial failure mode, not a general quality score for Qwen3Guard. But it demonstrates that a trace filter built from an off-the-shelf guardrail can have a total blind spot against a motivated attacker.

Both papers’ figures are author-reported in single preprints. TRACE does report false-positive rates on its benchmark (PolyGuard-8B at 9.11% on prompt judgment; YuFeng-XGuard-8B at 15.77% on reasoning traces, per the benchmark’s results), and ReasoningShield measures per-sample classifier latency on its own hardware, 4.63 seconds for its 1B model and 5.77 for the 3B. What is missing is production-API validation: neither paper measures how a trace filter performs on a live reasoning endpoint or how much latency it adds to real traffic, and no cited paper independently replicates the results. Treat the direction of the result (prompt/output-tuned guardrails degrade on traces) as well supported, and the exact percentages as provisional.

Can you even get the trace?

Before designing a trace layer, confirm what your provider actually returns, because the sources conflict. ReasoningShield asserts that “almost all LRMs expose full reasoning traces to users, no matter whether open-source models like DeepSeek-R1 or commercial models like Gemini-2.5 [8] and Claude-3.7/4.0” (ReasoningShield, Section 3.1). But the auditing-framework paper notes that “Some models, such as Gemini models expose only a summary of the reasoning by default and therefore the results depend on this summary being a comprehensive summary including all relevant details”.

These claims can coexist if provider behavior changed between the papers’ writing, or if “expose” means different things (full trace available via API versus shown by default). Either way, the engineering consequence is the same: verify per provider, per API version, before you architect. If your model returns only a reasoning summary, your trace filter is moderating a second model-generated document about the reasoning, not the reasoning itself. That weakens every detection claim downstream.

The faithfulness limit: what trace reading cannot catch

Here is the strongest argument against leaning on traces, and it comes from a different literature. A study of chain-of-thought unfaithfulness detection found that “Answer incorrectness alone (an oracle diagnostic, not a deployable detector) matches or exceeds every purpose-built metric-based signal (AUROC 0.696), because 69% of annotated unfaithfulness in the analyzed release occurs on incorrect answers” (the unfaithfulness study). In plain terms: the metric-based signals built to catch traces that misrepresent the model’s actual reasoning performed no better than simply noticing the answer was wrong, which is not a detector you can deploy. The paper’s one exception is a prompted LLM judge, which beat the oracle (0.782 versus 0.703 AUROC) but requires an additional generative-model inference per audited trace.

The mechanism behind this is articulated by the CIE-Scorer paper: “although CoT traces resemble step-by-step reasoning in natural language, they may not correspond to the internal mechanisms that underlie the model’s predictions”. A trace is a text the model produced, not a window into its computation. The auditing framework draws the safety conclusion directly: when trace and output are inconsistent, “the eval result cannot be trusted” and, because safety researchers increasingly rely on traces to monitor behavior, “Inconsistency between reasoning and output makes this monitoring unreliable”.

There is also a generalization caveat. A faithfulness study set out to ask “Across various model families, is inverse scaling observed in model size for CoT faithfulness once models reach a certain capability level?” and landed on a mixed answer: replicating across three model families, it reproduced the scaling trends under specific conditions, but after normalizing the metric to account for answer-choice bias, unfaithfulness dropped significantly for smaller, less-capable models. Faithfulness properties measured on one model family and one metric do not automatically describe another, so read this as a caution rather than a settled rule. If you calibrate a trace filter against one provider’s model, re-calibrate when you switch.

This is why the answer is layering rather than substitution. Trace reading catches unsafe reasoning that is present and faithfully rendered. Output checks catch the case where the reasoning was hidden or unfaithful, or where an encrypted prompt slipped past judgment and the harm lands in the answer anyway. Each layer covers a failure mode the other provably misses.

Failure modes that survive layering

The instruction-encryption result deserves a second look even in a well-built trace layer. Two guardrail variants scored 0.00% F1 on reasoning trace safety judgment when the prompts behind those traces were encoded with Caesar cipher or Base64, while LlamaGuard-2 still reached 54.70% on the same subset, so the damage is uneven rather than universal. Either way, a trace filter is a classifier, and classifiers have adversarial surfaces.

The cost line

Traces are long, and classification cost scales with tokens. The “Learning to Reason” paper reports that, for math reasoning, DeepSeek-R1 responses averaged approximately 15,500 tokens versus approximately 3,500 for gpt-oss-120b, about 4.4 times as many. A trace layer on a verbose reasoning model can multiply the text your safety classifier scans per request by several times, on top of prompt and output classification.

The volume trend points upward. The CLEF 2026 CheckThat! task now has participants rank “multiple reasoning traces generated by an LLM at varying temperature settings” per claim, illustrating a broader pattern: test-time scaling generates several traces per request, and each one is moderation surface. Budget for classifier inference over trace tokens as a first-order cost, not a rounding error, and note that no cited paper measures the latency overhead of trace classification on live endpoints, so that number must come from your own benchmarking.

The compliance tail

This consequence follows by logic, not from any cited source, and I want to be explicit about that. Stateless output moderation can classify a response and discard it. Trace moderation cannot: to score the reasoning, your pipeline must receive it, and to audit, appeal, or improve the classifier you will want to retain it. Intermediate reasoning may contain user data, inferred sensitive attributes, or the model’s exposed reasoning about a person, and it now sits in your logs.

No cited paper documents what retention or disclosure obligations actually attach to logged reasoning traces. What I can say is that the obligation question is real the moment you enable the filter: how long traces are kept, who can read them, whether they appear in subject-access responses, and whether your provider’s trace exposure comes with its own terms. Run that review with counsel before rollout, in parallel with the classifier evaluation, because a trace filter deployed without a retention policy is a liability wearing a safety costume.

A filter-placement framework and the verdict

Putting the evidence together, the layers compare like this:

The verdict, with its conditions: layer a trace-trained safety classifier between prompt and output filters as a complement, not a replacement. Output checks stay mandatory because unfaithful reasoning evades trace reading, because encrypted prompts can drive trace judgment to zero, and because the output is the only text you can be sure the user saw. Use an off-the-shelf guardrail on traces only if you accept the measured degradation (up to 22.3% F1 decline for LlamaGuard-4); a classifier trained or fine-tuned on traces is the better default, which is the gap ReasoningShield and TRACE both target.

The limitations deserve equal weight. Every quantitative claim here is author-reported in a single preprint: benchmark false-positive rates and per-sample latency figures exist in the papers, but there is no production-API validation, no measurement of the latency a trace filter adds to live traffic, and no independent replication. Provider trace exposure is asserted inconsistently across the sources and must be confirmed per API before you build. And the compliance leg, the part that may matter most to a platform team, rests on inference about logging requirements rather than documented obligations. I would prototype the trace layer on traffic where you can measure false positives cheaply, and hold the rollout until the retention review is done. The detection gap is measured and real; the cost of closing it carelessly is a log full of reasoning you never meant to keep.

Frequently Asked Questions

Do all reasoning models expose their full chain of thought to users?

ReasoningShield asserts that “almost all LRMs expose full reasoning traces to users, no matter whether open-source models like DeepSeek-R1 or commercial models like Gemini-2.5 [8] and Claude-3.7/4.0” (ReasoningShield, Section 3.1). But the auditing-framework paper notes that “Some models, such as Gemini models expose only a summary of the reasoning by default and therefore the results depend on this summary being a comprehensive summary including all relevant details”.

How much longer are reasoning traces compared to standard responses?

The “Learning to Reason” paper reports that, for math reasoning, DeepSeek-R1 responses averaged approximately 15,500 tokens versus approximately 3,500 for gpt-oss-120b, about 4.4 times as many. A trace layer on a verbose reasoning model can multiply the text your safety classifier scans per request by several times, on top of prompt and output classification.

What happens to data retention when enabling trace moderation?

A classifier cannot score a trace it never sees, so enabling trace moderation means capturing and storing intermediate reasoning your pipeline may currently discard. That turns a filter-placement choice into a data-retention decision, and it deserves the same review you would give any new log containing user-adjacent content.