Final-token safety probes are not enough to catch jailbreaks because unsafe evidence can be distributed across the prompt prefill and invisible at the last user token, according to Before the Last Token: Diagnosing Final-Token Safety Probe Failures. A single readout after prefill still shows high recall on clean harmful prompts, but it misses adversarial prompts that hide their signal earlier in the sequence. Red teams therefore need both final-token checks and trajectory-aware diagnostics, not one or the other.
Why does the final token miss jailbreaks?
A final-token probe reads the model’s hidden state at the last token of the user prompt, and that single snapshot can fail when unsafe evidence is spread across earlier positions. The probe is trained to classify harm from one vector: the representation at the boundary between prefill and generation. That design choice is attractive for production because it is cheap. It needs only one forward pass and one classifier evaluation, so moderation pipelines can run it on every request without adding noticeable latency. But the cost is structural. Jailbreak prompts do not always concentrate their unsafe content at the final position. Instead, they can distribute evidence across user-token representations that the final readout never sees clearly, or they can wrap the harmful instruction inside earlier context that the last token summarizes as benign.
The result is a probe that looks confident at the last token while the model has already absorbed a harmful instruction earlier in the sequence. This is not a failure of the classifier alone. It is a mismatch between where the probe looks and where the attack stores its signal. The prefill phase produces a sequence of hidden states, each encoding some portion of the user content. A final-token probe throws away almost all of them. For clean harmful prompts, the last token may carry enough signal because the request is direct. For adversarial prompts, the evidence is diffuse, conditional, or encoded in earlier turns. The probe sees the calm surface and misses the current underneath.
What do the SafeSwitch experiments show?
On clean harmful and benign prompts, SafeSwitch-style probes trained only on the clean split show high recall for harm; against jailbreaks, they miss many attacks and generate false positives on safety-adjacent benign prompts. The authors tested this setup across three instruction-tuned LLMs and found the same pattern in each: the probes learn the obvious cases well and struggle with the subtle ones.
The high recall on clean harmful prompts is exactly what makes the result dangerous. It gives evaluators a number that looks reassuring while the probe silently fails on the prompts that matter most in production. A dashboard full of green checks on clean harm data says little about whether the probe will catch a carefully constructed jailbreak. The false positives are equally telling. Safety-adjacent benign prompts, such as requests that mention harm in order to refuse it, discuss policy, or request safety documentation, get flagged because the probe has learned surface correlates rather than the structure of a genuine harmful request. If a model refuses to explain why a biological weapon is dangerous because the moderation probe confuses the explanation with the recipe, the safety system has become a source of harm itself.
A probe that cannot distinguish a jailbreak from a benign safety discussion is not a refined instrument. It is a coarse filter, and coarse filters tend to fail open in the direction that embarrasses the deployment.
Why can’t you just widen the probe?
Widening the probe bottleneck does not reliably fix the mismatch, because missed jailbreaks differ from clean benign prompts along directions that the probe’s representational subspace poorly captures. The authors ran subspace analyses to see whether the missed attacks live in a part of representation space that a wider or more expressive probe could learn. The answer appears to be no. The distinguishing directions are not simply outside the current bottleneck; they are poorly aligned with the subspace that the probe uses to separate clean classes.
That matters for anyone hoping to fix the problem by swapping in a bigger linear head or adding capacity. If the probe’s training distribution does not expose the right axes of variation, extra parameters buy only marginal gains. The failure is representational before it is architectural. The clean harmful versus benign split does not span the manifold on which jailbreaks lie, so the probe cannot interpolate to them. You can make the classifier larger, but you are still asking it to draw a boundary in a plane that does not contain the adversarial examples.
This is the kind of result that should make red teams suspicious of any claim that a larger probe or more layers will close the gap. Sometimes the fix is not more capacity; it is a different view of the input.
Why does max-pooling over tokens overfire?
Naive max-pooling across token positions exposes unsafe evidence earlier in the sequence, but it pays for that visibility by flagging a large share of safe prompts. Token-level prefill analyses in the paper show that probe-visible unsafe evidence often appears well before the final token. Max-pooling is the obvious way to capture it: take the highest safety-relevant activation anywhere in the prefill and use that as the decision signal. If a jailbreak hides its payload on token twelve, a max-pooled probe can still see it.
The problem is that safe prompts also contain tokens that look unsafe in isolation. A medical example, a security audit request, a fictional scenario, or a discussion of historical violence can produce strong activations on individual tokens. A max-pooled probe therefore overfires. It catches some jailbreaks that the final-token probe missed, but it turns everyday benign prompts into alerts. In a production moderation pipeline, that is not an acceptable trade. Either the team drowns in false positives or they tune the threshold until the probe becomes useless.
The lesson is that seeing more positions is necessary, but seeing them independently is not enough. Position is not the only structure that matters; the trajectory across positions matters too. A single spike at one token is weak evidence compared with a sustained or evolving pattern across the prefill.
Can a trajectory-aware diagnostic recover the misses?
A PCA-HMM trajectory model, trained only on the same clean split, recovers many final-token misses by modeling the user-content prefill trajectory instead of relying on a single position. The model reduces the sequence of hidden states to a low-dimensional trajectory through principal component analysis and then uses a hidden Markov model to capture how that trajectory evolves. Because it is trained on the same clean data as the probe, its improvement comes from the representation of motion, not from additional labeled attacks.
The diagnostic is interesting precisely because it is so simple. It does not require jailbreak examples at training time. It does not ask the probe to generalize to attacks it has never seen. It models the normal dynamics of benign and clean harmful prefill trajectories and flags deviations. On the missed jailbreaks, those deviations are visible as soon as the unsafe evidence enters the context, long before the final token. The probe missed them not because the evidence was absent, but because it was looking at the wrong coordinate.
The false-positive behavior is also more controlled than naive max-pooling. The PCA-HMM avoids the catastrophic overfire because it conditions on the temporal structure of the trajectory, not on isolated spikes. A single token that looks harmful does not trigger an alert unless the surrounding trajectory supports it. It is not a perfect detector; a jailbreak that perfectly mimics a benign trajectory could still slip through. But it shifts the failure mode from a structural blind spot to a statistical one, which is a different and more manageable class of problem.
What should red teams change now?
Red teams should stop treating a passing full-sequence or final-token safety probe as sufficient and add trajectory-aware checks that look at how unsafe evidence evolves across the prefill. The practical change is to validate the final-token signal separately from any aggregated score and to run a trajectory diagnostic, such as the PCA-HMM approach, alongside it. Neither check replaces the other. The final-token probe catches clean harms cheaply. The trajectory check catches distributed jailbreaks. A full-sequence aggregate can mask the final-token failure by averaging a low final score against stronger earlier signals, so aggregation without position-aware analysis is not a safe substitute.
This raises the cost of safety evaluation. Teams now have two models to maintain, two sets of thresholds to calibrate, and two failure modes to explain to auditors. The trajectory method also needs access to hidden states across the prefill, which may require changes to inference serving if the current stack only surfaces the final token. That cost is real. But the alternative is worse: a single passing probe that gives false confidence while jailbreaks slip through because their evidence never reached the last token.
The finding also sits inside a larger pattern of safety-evaluation blind spots. A separate July 2026 preprint, Safety Targeted Embedding Exploit via Refinement (STEER), reports that English-centric safety training fails for low-resource languages and code-switching, achieving up to 96.7% attack success on AdvBench and 35.5% transfer to GPT-4o-mini. Another concurrent preprint, Harnessing Textual Refusal Directions for Multimodal Safety (MARS), shows that textual refusal directions extracted from an LLM backbone can generalize across image and video modalities, suggesting that safety-relevant structure is shared across modalities. Together, these results frame single-readout probes as one instance of a broader false-confidence problem: safety signals that work on the training distribution often fail where the distribution shifts, whether by language, modality, or token position.
The July 9, 2026 v2 revision of Before the Last Token, accepted at ICML 2026’s Mechanistic Interpretability and Failure Modes in Agentic AI workshops, gives the argument a conference floor just as practitioners are deciding which probes to trust. The answer should be neither final-token nor full-sequence alone. Trust the combination, document where each one breaks, and budget for the extra work.
Frequently Asked Questions
Does the final-token probe problem apply to all LLM safety classifiers?
It is most relevant to decoder-only transformer setups that use a single hidden-state readout after prefill, such as the SafeSwitch-style probes tested across three instruction-tuned LLMs. Classifiers that pool every token, or moderation APIs that score text embeddings rather than internal hidden states, face different failure modes. State-space or encoder-decoder models may distribute representations differently, so the exact blind spot shifts.
How do these probe failures differ from the multilingual attacks reported in STEER?
STEER shows that English-centric safety training fails under low-resource languages and code-switching, reaching 96.7% attack success on AdvBench and 35.5% transfer to GPT-4o-mini. That is a distribution shift across languages, while Before the Last Token isolates a positional shift: unsafe evidence is present but not at the final token. Both reveal that a single safety readout can pass while the model has already absorbed harmful instructions.
What infrastructure changes are needed to run a PCA-HMM trajectory diagnostic in production?
Teams need access to hidden states across the entire prefill, not just the last token, which can require patching inference serving stacks and storing per-token activations. For large models, materializing every layer’s hidden states can multiply memory traffic during prefill. The PCA-HMM itself is cheap once features are extracted, but the logging and retrieval pipeline is the real cost.
Can a trajectory-aware diagnostic still be evaded?
Yes. A jailbreak that closely mimics the hidden-state trajectory of a benign prompt could slip past the PCA-HMM, because it is trained only on clean examples and flags deviations, not adversarial structure per se. It also inherits blind spots if the unsafe signal enters after prefill, for example through tool outputs or retrieval chunks introduced during generation.
What would make final-token probes obsolete rather than complementary?
If serving frameworks expose cheap full-trajectory representations by default and trajectory models prove robust across languages and modalities, teams might drop the final-token screen. Until then, it remains a low-latency first filter while the PCA-HMM catches distributed jailbreaks. The July 9 v2 ICML acceptance is likely to accelerate benchmarks that stress both, which could shift the cost-benefit within a year.
