Cascading Hallucination in Agentic RAG: When One Bad Retrieval Poisons the Chain

Most RAG hallucination detectors check the final answer against retrieved context. That works for single-hop queries. For multi-step agentic pipelines, where one bad retrieval feeds into the next reasoning step and the next, CHARM (arXiv:2606.04435), short for Cascading Hallucination Aware Resolution and Mitigation, formally defines a failure mode in which a fabricated retrieval at hop one is propagated forward as valid context and per-step grounding checks never notice. In its evaluation, output-level checks reduced final-answer errors by 18.5% relative to no detection at all; CHARM’s mid-chain intervention reduced them by 82.1% (arXiv:2606.04435), and the strongest existing detector it tested caught 41.7% of injected cascades (arXiv:2606.04435).

Why per-step grounding is not enough

The standard playbook for RAG reliability runs something like this: retrieve chunks, check relevance, generate an answer, then run a fact-checking pass on the output. Tools like SelfCheckGPT and RAGAS operationalize this by comparing the final response against the retrieved context or by sampling multiple outputs and measuring consistency. The assumption is that errors are local: if each step is grounded, the chain is sound.

That assumption holds for single-hop retrieval. It breaks when an agent reasons across multiple retrieval steps, because a fabrication at step i becomes part of the context window for step i+1. The downstream step retrieves additional information, reasons over the combined context (which now contains a fabricated claim), and produces output that is internally consistent but factually contaminated. Standard detectors see a well-grounded step; they do not see that the ground shifted underneath it.

CHARM’s evaluation quantifies the gap two ways. On error propagation, the metric closest to production pain, output-level checks reduced final-answer errors by 18.5% against a no-detection baseline, while mid-chain intervention reduced them by 82.1% (arXiv:2606.04435), roughly a 4.4× difference. On detection, the paper reports RAGAS catching 41.7% of injected cascades and LLM self-correction 12.8% (arXiv:2606.04435). The paper’s text gives SelfCheckGPT, its pure output-level baseline, no stated detection rate, though its cross-dataset comparison implies one in the low-to-mid 20s: CHARM’s stated per-dataset advantages of 66.4, 63.7 and 66.0 percentage points subtracted from detection rates of 92.5%, 86.1% and 87.8% leave roughly 22–26% for the output-level detector. Whichever number you anchor on, output-only monitoring is blind to most of the failure class most likely to produce plausible-sounding falsehoods.

Four conditions that define a cascading hallucination

CHARM formalizes cascading hallucination via four conditions that must all hold simultaneously (arXiv:2606.04435, §3):

1. Factual error at stage i. A retrieval or reasoning step produces output containing a factual error with respect to ground truth.
2. Corrupted context propagated as valid. The error is passed to stage i+1 not as a candidate claim but as established context, indistinguishable from verified facts.
3. Local coherence under global falsity. Downstream output is conditionally coherent given the corrupted input. The reasoning is valid; the premises are not.
4. Monotonically non-decreasing error magnitude. Errors do not self-correct across hops. They accumulate or amplify.

The third condition is the critical one. It is what makes cascading hallucination invisible to per-step detectors: each stage, examined in isolation, looks correct. SelfCheckGPT sees a response that follows from its context. RAGAS sees high faithfulness scores. The problem is that the context itself was poisoned two hops ago, and no single-step check traces back that far. The paper states the conclusion as a corollary: per-step detection is inherently insufficient for cascade identification.

Why existing detection tools fail on cascaded errors

CHARM evaluates against the detection approaches its authors treat as the current state-of-the-art. The paper’s background section groups existing detectors into output-level, retrieval-level and consistency-based categories and argues that all three share structural blind spots in cascading scenarios; its direct baselines are SelfCheckGPT, RAGAS and LLM self-correction:

• SelfCheckGPT and self-consistency methods compare multiple sampled outputs or check response consistency. These catch contradictions within a single step but cannot detect errors that are consistently propagated across steps. If stage 3 confidently repeats the fabrication from stage 1, self-consistency flags nothing.
• RAGAS-style faithfulness metrics measure whether a response is supported by its retrieved context. In a cascade, it is. The context was corrupted upstream, but the response is faithful to that corrupted context. The paper credits RAGAS with 41.7% cascade detection, catching retrieval-stage errors while missing the inference and confidence-inflation cascades that occur after the stage it monitors (arXiv:2606.04435).
• LLM self-correction, where the model reviews its own output, fails for a different reason: confirmation bias. As the CHARM paper notes, the agent reinforces the cascade because downstream reasoning appears logically sound relative to the corrupted intermediate context. The model is checking its work against the same poisoned premises that produced the work; it caught 12.8% of cascades (arXiv:2606.04435).

A comprehensive hallucination survey (arXiv:2510.06265) categorizes detection into five approaches (retrieval-, uncertainty-, embedding-, learning-, and self-consistency-based) and mitigation into four (prompt, retrieval, reasoning, model-centric training), concluding that no single approach suffices. CHARM’s chain-level framing explains why: these approaches are designed for per-step or per-output checks, not for tracking error propagation across a multi-hop DAG.

CHARM’s detection architecture

CHARM models the multi-step reasoning pipeline as a weighted directed acyclic graph where nodes are pipeline stages and edge weights represent the error propagation probability P(ε_{i+1}|ε_i). Cascade detection identifies paths where the cumulative product of propagation probabilities exceeds a safety threshold θ (arXiv:2606.04435). Exact path probabilities are intractable at inference time, so the operational trigger is a linear weighted vote over three anomaly scores (weights 0.4, 0.4, 0.2) that fires at θ = 0.55 (arXiv:2606.04435).

The framework has four components:

1. Stage-level fact verification. Each retrieval and reasoning step is checked independently against source data, similar to existing grounding checks but applied at every node in the DAG rather than only at the output.
2. Cross-stage consistency tracking. Claims from earlier stages are traced through downstream reasoning. If stage 4 cites a “fact” that stage 1 fabricated, the consistency tracker flags the provenance gap.
3. Confidence propagation monitoring. Rather than treating each stage’s confidence score in isolation, CHARM models how confidence degrades (or, tellingly, does not degrade) as errors propagate. A stage that produces high-confidence output from low-confidence input is a cascade signal.
4. Cascade resolution triggering. When the aggregated score crosses θ, a resolution step intervenes: re-retrieving, re-verifying, or halting the chain.

Component ablations are concrete: stage-level verification alone reaches 61.2% detection on HotpotQA, consistency tracking adds 18.2 percentage points, and confidence monitoring contributes a further 6.4 points despite managing only 38.3% alone (arXiv:2606.04435). Each module earns its place; removing any one degrades coverage.

Benchmark results

CHARM was evaluated on three established multi-hop QA benchmarks (HotpotQA, MuSiQue, 2WikiMultiHopQA) and a custom adversarial dataset, across LangChain agentic pipeline configurations. Cascades were injected by protocol rather than harvested from production, so every number below measures controlled failure (arXiv:2606.04435):

Two reading notes on that table. First, the 18.5% figure that headlines the abstract is error reduction, not detection: output-level checks left most final-answer errors in place. Second, the paper argues that a direct per-stage latency comparison understates CHARM’s relative efficiency: the 450 ms for SelfCheckGPT and 380 ms for RAGAS reflect additional LLM sampling and faithfulness-scoring calls, while CHARM’s monitors run on local NLI and embedding models with no additional LLM calls (arXiv:2606.04435).

The 4.4× gap in error reduction is the headline number, but the false positive rate matters for production viability. CHARM’s 5.3% false-positive rate is measured on clean trajectories: roughly one legitimate run in twenty gets flagged as a cascade it is not (arXiv:2606.04435). Whether that is acceptable depends on the cost of your resolution step: if resolution means re-retrieving a document and re-running a reasoning pass, one unnecessary intervention every twenty runs is manageable. If resolution means flagging for human review, it is almost certainly too noisy.

The detection rate of 89.4% also means roughly one in ten cascades slips through (arXiv:2606.04435). This is not a solved problem; it is a substantially better tool for a problem that was previously nearly invisible.

What mid-chain intervention costs

The 215 ms per-stage overhead is the number that should make engineering teams pause. In a 5-hop pipeline, that is roughly one second of added latency before accounting for any re-retrieval or re-reasoning triggered by cascade resolution. In a 10-hop pipeline, it is two seconds. For real-time conversational agents, that is a serious cost. For batch analytical workflows, it is likely negligible.

The paper argues the effective cost is lower than the per-stage figure suggests: cascades are caught at stage 2.1 on average, halting the pipeline before stages 3–5 run and saving 2–3 full LLM inference calls per detected cascade, and the per-stage figure excludes the backbone model’s own latency (arXiv:2606.04435). That saving applies only when a cascade is actually detected. Clean runs pay the monitoring overhead and get nothing back.

The real cost is not latency. It is architectural. Mid-chain intervention requires:

• Provenance metadata at every stage. Each node in the DAG needs to carry structured claims, their source references, and their confidence scores. The paper’s own integration story points the same way: CHARM works by wrapping existing LangChain or LlamaIndex pipelines as an external layer, without teardown (arXiv:2606.04435). My reading of that design is that chain-level provenance is something you add alongside the framework rather than something the framework hands you, so budget for wrapping or replacing framework-provided chain primitives to carry this metadata.
• A cross-stage consistency store. Something needs to hold the state that tracks which claims from stage 1 are being cited at stage 4, either a shared memory layer or an external state store. This is the architectural requirement that separates CHARM from prior work in its own related-work comparison: cross-stage semantic drift tracking and confidence-inflation monitoring are, in the paper’s words, “capabilities not jointly addressed by any prior single framework” (arXiv:2606.04435). If your stack does not already persist inter-stage claim state, that is new plumbing you are signing up for.
• A decision policy for the threshold θ. Set it too low and every pipeline triggers resolution. Set it too high and you miss the cascades you are trying to catch. CHARM’s own value is θ = 0.55, selected by grid search over 0.40–0.65 on held-out validation data to balance detection against false positives, with per-domain recalibration recommended for production (arXiv:2606.04435). Start from 0.55, then recalibrate against your own error distribution; the grid-searched value is a starting point, not a universal constant.

Adding provenance tracking to existing pipelines

The paper does not detail a step-by-step integration path for production LangChain deployments (it describes CHARM as operating “alongside” existing pipelines), but the DAG model suggests a practical approach:

Step 1: Instrument claim extraction. After each retrieval and reasoning step, extract the specific factual claims the step produces. This does not require CHARM; it is structured logging. Each claim gets a unique ID, a source reference, and a confidence score.

Step 2: Add cross-hop claim tracking. Before a downstream step runs, check whether its input context contains claims from prior steps. Flag any claim that was not verified against source data at the point of origin. This is the provenance gap that cascading hallucination exploits.

Step 3: Propagate confidence scores. If step 2 produces output with high confidence but its input contains a low-confidence claim from step 1, surface that mismatch. The downstream step’s confidence should be conditioned on the upstream claim’s reliability, not computed independently.

Step 4: Set a cumulative propagation threshold. When the product of per-hop propagation probabilities exceeds your tolerance, trigger re-retrieval or halt. The threshold is domain-specific, though CHARM’s grid-searched θ = 0.55 is a reasonable starting point (arXiv:2606.04435). A medical diagnosis agent should threshold lower than a news summarizer.

None of this requires adopting CHARM as a library. The value of the paper is the formal model and the empirical evidence that the strongest existing detector it tested caught 41.7% of injected cascades while mid-chain tracking caught 89.4% (arXiv:2606.04435). The implementation is an engineering problem, and it is one that teams running multi-hop agentic RAG should start thinking about now, because the alternative is deploying agents that confidently propagate fabrications across reasoning chains while every per-step monitor reports green.

A separate line of work takes a different approach to the same problem. A study on agentic hallucination mitigation (arXiv:2501.13946) reported that passing 310 deliberately hallucination-inducing prompts through a front-end agent and two successive reviewer agents, connected by structured JSON messages in the OVON framework, lowered its Total Hallucination Score at each stage, with a fourth agent doing the scoring. The caveat sits in the metric: the score is built from four components (factual claim density, factual grounding references, fictional-disclaimer frequency and an explicit contextualization score), and the authors state their KPIs “don’t rely only on absolute factual truth but rather on linguistic and stylistic indicators” (arXiv:2501.13946); their limitations section concedes the scores are no infallible benchmark for factual correctness. That approach stacks independent agents as reviewers rather than tracking provenance within a single chain. Whether stacking or tracking proves more practical in production likely depends on pipeline length and latency budget: stacking adds entire model calls, while provenance tracking adds overhead per hop but keeps the chain intact. The two approaches are complementary, not competing, and neither is a complete solution on its own.

A counterpoint: cascades that do not always amplify

CHARM’s fourth condition assumes error magnitude is monotonically non-decreasing across hops. A second June 2026 preprint complicates that assumption. Hallucination Cascade (arXiv:2606.07937), from Jamshidi and colleagues at Polytechnique Montréal, tracked factual inconsistency as responses passed between separate agents in multi-agent chains rather than within a single retrieval pipeline. Across GPT-5.3, DeepSeek-V3, and LLaMA-3-70B-Instruct, the normalized hallucination score fell from 0.422 at the first agent to 0.272 at the final agent in three-agent chains (arXiv:2606.07937). Deeper chains suppressed fabrication instead of compounding it.

That is the mechanism difference between the two papers, not a contradiction. CHARM models one agent carrying its own corrupted context forward, where nothing re-examines the poisoned premise. A multi-agent chain inserts a fresh model at each hop, and a downstream agent that did not generate the upstream claim can challenge it. The later agents behave as implicit reviewers, which is the same intuition behind stacked-reviewer designs like OVON.

The suppression is not free. Jamshidi’s group report factual accuracy slipping from 0.789 to 0.769 as chains deepened, and describe hallucination suppression across agent-to-agent transitions as not lossless, with small but consistent losses in factual consistency and response quality (arXiv:2606.07937). An agent that overwrites a predecessor’s hallucination sometimes overwrites a correct answer too. The study also runs across 10 knowledge domains with a domain-level breakdown of results, so check that breakdown before assuming the aggregate numbers describe your corpus. A defense tuned on one corpus will not carry its numbers to another, which echoes CHARM’s own caveat about transfer beyond multi-hop QA.

Read side by side, the two results sharpen the guidance. If your architecture is a single agent looping over retrievals, assume cascades amplify and instrument the chain the way CHARM prescribes. If it is several distinct agents passing messages, some self-correction comes for free, and the problem shifts toward preserving the correct answers the reviewers might trample. Groundy’s coverage of council-mode multi-agent voting found the same tension: reviewers cut hallucination, and their votes are paid for in tokens.

The bad retrieval is sometimes planted

Both papers treat the stage-one error as accidental: a noisy retriever, a misranked chunk, a model that guessed. In adversarial settings it is not. A poisoned corpus can plant the hop-one fabrication that CHARM’s first condition requires, using documents engineered to pass stage-level fact verification. Work on conflict-aware retriever poisoning shows attackers can inject claims that read as well-sourced precisely so they survive the per-step grounding check. CHARM’s own adversarial-robustness discussion concedes the surface: a sophisticated adversary aware of its detection mechanisms could construct counterfactual documents semantically proximate to ground truth to evade drift detection, and entailment-ambiguous injections “could evade the SFV’s NLI threshold” (arXiv:2606.04435). Its distractor stress test measures the pressure without proving defeat: with three semantically proximate false documents alongside the correct source, detection fell to 84.1% and false positives rose to 7.8% (arXiv:2606.04435). Cascading hallucination is the propagation half of that attack: poison the retrieval, and the agent’s own reasoning carries the payload the rest of the way. Provenance tracking helps here for a reason CHARM does not stress, which is that it forces every downstream citation back to a named source and raises the cost of a fabrication that has no real provenance to point at.

Treat CHARM’s numbers as a preprint

CHARM is a June 2026 preprint with no peer review, and no independent reproduction is cited in the paper. Its 89.4% detection rate comes from the paper’s own evaluation harness, on injected cascades its authors constructed and labeled (arXiv:2606.04435), and its false-positive methodology in the text describes CHARM’s measurement only. The closest thing to natural-cascade evidence is a 50-trajectory pilot on naturally occurring HotpotQA failures, in which 76% were flagged and 89.5% of the flagged cases looked genuinely cascade-like on manual inspection (arXiv:2606.04435). The DAG formalism and the four-condition definition are the durable contribution; the headline percentages are a first data point, not a settled benchmark. The authors flag that embedding- and NLI-based detectors lose effectiveness on “semantic illusion” hallucinations, outputs that stay semantically close to the correct answer while being wrong (arXiv:2606.04435). The per-stage verifier CHARM depends on is an open problem, not a solved primitive, and a cascade detector is only as good as the stage-level check feeding it.