On 2026-09-17, a preprint called HALT (arXiv:2602.02888) surfaced in the research feed with a specific proposal: read a language model’s per-token log-probabilities as a time series, and use that sequence to detect hallucinations, rather than collapsing each response into one confidence score. For audit and compliance teams, the consequence is concrete. A single hallucination score per response may no longer be sufficient evidence, and whether you can produce anything better depends on what your model provider’s API actually exposes.
Everything quantitative in this article comes from that one preprint’s abstract. The results are author-reported, with no independent replication and no third-party critique in the sources reviewed for this piece, and nothing here is a regulatory mandate or a compliance requirement. What follows is a decision frame the paper’s design makes possible: which hallucination detection method a team can actually run, given the access tier its provider grants.
The premise: one log-prob per token, read as a sequence
Language models do not emit words; they emit tokens. As Merriam-Webster’s token definition notes for LLMs, short words may be a single token while longer words split into two or more, and it is to these tokens that probabilities attach. Every time a model generates a token, it also computes how likely that token was given the context. Most deployments discard that signal or average it away. HALT’s bet is that the discarded stream is where hallucinations live.
According to the HALT abstract, the detector uses “only the top-20 token log-probabilities from LLM generations as a time series.” Two properties of that sentence matter. First, top-20: the detector does not need the full vocabulary distribution, only the twenty most likely alternatives at each step. Second, time series: the order and shape of the log-prob curve across a response is the input, not a summary statistic derived from it. A response that glides confidently through boilerplate and then stumbles at the sentence containing a specific date or citation looks different in sequence form than it does in any average.
This is the design-level answer to why hallucinations are hard to catch with confidence scores. Models “can confidently invent facts, citations, or APIs that don’t exist. They’re predicting plausible text, not retrieving truth,” as a 2026 beginner’s guide to LLMs puts it. Plausible text is usually locally confident. The fabrication tends to show up as a local disturbance, a few tokens where the model’s alternatives fragment, buried inside an otherwise fluent paragraph. An aggregate cannot see that. A sequence can, at least in principle.
Why a single score hides the signal
The canonical single score is perplexity. Per Wikipedia’s language model entry, perplexity is “the exponential of the average negative log likelihood per token”: every token’s surprise, averaged, exponentiated, delivered as one number for the whole text. That is a legitimate measure of how well a model predicts a corpus. It is a structurally lossy measure of where in a specific response the model hesitated.
Consider what averaging does to a long, multi-fact answer. Suppose most of the response is high-confidence and one fabricated clause carries low-confidence tokens. The average dilutes the spike; the response passes a threshold set for the typical case. This is a mechanism argument, not a measured result: the fetched evidence contains no experiment showing perplexity thresholds failing on long-form answers, and this article does not claim one. What the evidence does establish is the definitional contrast. Perplexity averages per-token negative log-likelihood into one number, and HALT exists specifically because its authors believe that average throws away the detectable part.
Sampling-based approaches attack the problem from a different direction. As Groundy’s earlier comparison of uncertainty methods describes, semantic entropy generates multiple candidate answers, clusters them by meaning, and treats dispersion across clusters as the hallucination signal, at the literal cost of paying for the extra generations. One important limitation for this article: the fetched HALT materials contain no comparison against semantic entropy or any other sampling-based method. Nothing here supports a claim that HALT beats them. The honest statement is narrower: HALT proposes a single-generation, sequence-level signal, and its authors have not (in the evidence available) benchmarked it against the sampling-based family.
What HALT adds, according to its authors
The abstract positions HALT against three alternatives, and each positioning is really a claim about access:
- Versus white-box methods. HALT “does not require access to hidden states or attention maps, relying only on output log-probabilities,” per the abstract. White-box detectors can be strong, but they presume you can open the model, which rules out every API-only deployment.
- Versus black-box methods. HALT “operates on log-probs rather than surface-form text, which enables stronger domain generalization and compatibility with proprietary LLMs without requiring access to internal weights.” The generalization claim is the authors’ own, and “compatible with proprietary LLMs” deserves an asterisk this article will return to: it holds only where the vendor actually returns log-probs.
- Versus encoder-based detectors. HALT uses “a gated recurrent unit model combined with entropy-based features to learn model calibration bias, providing an extremely efficient alternative to large encoders.” The abstract’s own figure for that efficiency is 30x smaller than Lettuce, and the entropy features summarize the shape of the top-20 distribution at each step.
The phrase “learn model calibration bias” is doing quiet work. A raw log-prob is not a calibrated probability of correctness; some models are systematically overconfident, others underconfident. Rather than assuming a threshold that transfers across models, HALT learns each model’s tendency as part of training. That is a sensible design choice; what training data it requires, the abstract does not say.
The access-tier comparison
The practical differences collapse into one table. The question is not which method is best in the abstract; it is which method your provider’s API surface permits.
| Method family | Signal used | Access required | Granularity | Extra generation cost | Evidence status in this piece |
|---|---|---|---|---|---|
| HALT (log-prob time series) | Top-20 token log-probs, sequenced | Provider must expose log-probs | Per-token trace | None; single generation | Author-reported only, on authors’ own benchmark |
| White-box detectors | Hidden states, attention maps | Full model internals | Per-token or per-layer | None; single generation | Not evaluated in fetched set |
| Encoder detectors (e.g., Lettuce) | Surface text of the response | None beyond the output text | Per-response score | One encoder pass | Lettuce appears only as HALT’s baseline |
| Perplexity threshold | Average negative log-likelihood | Log-probs (or computable locally) | Single aggregate score | None | Definitional contrast only |
| Sampling-based (e.g., semantic entropy) | Multiple sampled generations | Text output only | Per-response score | Yes; many extra generations | No head-to-head with HALT in fetched set |
Two rows carry the argument. The white-box row is empty for most buyers: if you rent the model through an API, hidden states are not on the menu. And the HALT row’s access requirement is the whole ballgame. The paper’s compatibility claim is conditional on a vendor feature that not every vendor provides, and that vendors can withdraw.
HUB and the numbers: measured by whom, on what
HALT’s headline result is that “while being 30x smaller, HALT outperforms Lettuce, a fine-tuned modernBERT-base encoder, achieving a 60x speedup gain on HUB,” according to the abstract. Both numbers, 30x smaller and 60x faster, are author-reported and unreplicated.
The evaluation harness deserves as much attention as the result. The same paper introduces HUB, the Hallucination detection Unified Benchmark, which consolidates prior datasets into ten capabilities: Algorithmic, Commonsense, Mathematical, Symbolic, Code Generation, Chat, Data-to-Text, Question Answering, Summarization, and World Knowledge. That breadth is genuinely useful; hallucination detection papers have a history of evaluating on one narrow QA slice. But HUB is the authors’ own benchmark, HALT is the authors’ own detector, and Lettuce is the only named baseline in the abstract. There is no independent replication, no long-form or multi-fact evaluation in the fetched record, and no comparison against sampling-based methods. The advantage may be real and may not survive contact with a benchmark the authors did not build. Both things can be true, and as of today only the authors’ account exists.
This is a familiar posture for preprint-driven audit decisions. When Groundy covered variance in LLM-judge scores, the working conclusion was that a single unrepeated score is not audit evidence. The same discipline applies here in both directions: HALT’s 30x and 60x are single-source numbers, and so is any one hallucination score a detector assigns to a response you are about to sign off on.
The audit consequence: ask for the trace, not the score
Here is the consequence that survives even if HALT itself fails replication. A detector that reads log-probs as a sequence produces, as a byproduct, a token-level uncertainty trace: which tokens were confident, which were contested, where in the response the model hesitated. A single score produces a number. When an auditor asks “why was this response flagged?” or “why was this one cleared?”, the trace is inspectable evidence and the number is a conclusion.
This is an inference from HALT’s design, not a claim the paper makes, and it should be labeled as such. But it fits a pattern Groundy has tracked elsewhere: work on chain-of-thought monitoring reached the same structural conclusion about reasoning transcripts, that an incomplete trace is not an audit log, and that evidence of record has to come from signals you can actually inspect. Token-level uncertainty traces are a candidate for that kind of evidence in hallucination review, with one large caveat: a trace is only as good as the detector interpreting it, and HALT’s interpretation is currently validated only by its own authors on its own benchmark.
The burden shift is real regardless. If your audit regime currently accepts a vendor’s single hallucination or confidence score per response, HALT’s existence is a reason to start asking a harder question in procurement: does the API expose top-k log-probs, at what depth, and can we retain them per response? That question costs nothing to ask and repositions the negotiation before it becomes urgent.
Deployment verdict: match the method to the access tier
The practical rule, given everything the evidence does and does not support:
- If your provider exposes log-probs (top-20 is what HALT uses), sequence-level detection is runnable, cheap per response, and yields inspectable token-level traces. HALT is the current author-reported implementation of that idea; treat its 30x smaller and 60x speedup figures as unreplicated until independent results appear.
- If your provider returns text only, token-level uncertainty evidence is unobtainable from that vendor. Your options are surface-form black-box detectors, which HALT’s authors claim generalize worse (their claim, their benchmark), or sampling-based methods like semantic entropy, which need only text output but multiply generation cost per query. Renegotiating log-prob access is now a legitimate audit-driven ask.
- If you run models in-house, white-box methods are available and the access constraint disappears; the fetched evidence says nothing about how HALT compares to them on accuracy, only that it needs less access.
The strongest limitation bears repeating plainly: every performance claim here traces to one abstract, one author group, and one benchmark that same group introduced. There is no independent replication, no semantic-entropy comparison, and no evidence on the long-form, multi-fact responses where aggregate scores are most suspect. What HALT adds today is not a proven detector. It is a workable proof that the signal your model already emits, read as a sequence rather than an average, is worth demanding access to, and a concrete reason to stop treating one score per response as sufficient evidence.
Frequently Asked Questions
What specific data does the HALT detector use to identify hallucinations?
According to the HALT abstract, the detector uses “only the top-20 token log-probabilities from LLM generations as a time series.”
What is the main access requirement for using HALT with a language model?
HALT “does not require access to hidden states or attention maps, relying only on output log-probabilities,” per the abstract.
How does HALT compare in size and speed to the Lettuce encoder?
HALT’s headline result is that “while being 30x smaller, HALT outperforms Lettuce, a fine-tuned modernBERT-base encoder, achieving a 60x speedup gain on HUB,” according to the abstract.
