How LLM Hallucination Detection Works: Attention Graphs (CHARM), Output Signatures (LOS-Net), Semantic Entropy and Practical Safeguards
A guide to detecting LLM hallucinations: white-box methods that read attention and activations (CHARM, ICLR 2026), black-box methods using output probability distributions (LOS-Net), sampling-based checks like semantic entropy and SelfCheckGPT, their limits, and the safeguards production systems should use.
Hallucination detection has moved from external fact-checking toward reading signals from the model itself. Two research directions stand out: white-box methods that analyse a model’s internal attention and activations, and black-box methods that analyse its output probabilities or the consistency of repeated answers. A group including Technion’s Haggai Maron has published examples of both, CHARM and LOS-Net. These methods can flag likely errors cheaply, but none is reliable enough to replace grounding in sources and human review for high-stakes uses. This guide explains how each approach works, what it needs and how to combine them in practice.
The main families of detection methods
| Approach | Example | What it needs | Strengths | Limits |
|---|---|---|---|---|
| Internal states (white-box) | CHARM (attention graphs + activations) | Access to model internals | Can localise suspicious tokens; low extra cost per answer | Only for self-hosted or open-weight models; needs training data |
| Output distributions (grey-box) | LOS-Net (LLM Output Signature) | Next-token probability distributions | Works without internals; very low latency | Commercial APIs usually expose only top token probabilities |
| Sampling consistency (black-box) | SelfCheckGPT, semantic entropy | Several sampled answers | Works with any model | Multiplies cost; misses consistently held false beliefs |
| Self-assessment | Asking the model whether its answer is correct | Nothing extra | Simple | Calibration varies by model and task |
| External verification | Retrieval, code execution, database checks | Trusted sources or tools | Most reliable for checkable facts | Requires infrastructure; not every claim is checkable |
CHARM: reading attention as graphs
Earlier white-box methods summarised attention with simple statistics, such as how much the model attended to the prompt versus its own generated text. CHARM, presented at ICLR 2026 by Fabrizio Frasca, Guy Bar-Shalom, Yftah Ziser and Haggai Maron, instead represents a model’s computation as an attributed graph: tokens are nodes, edges follow attention flows, and both carry features from attention scores and activations (arXiv:2509.24770). A graph neural network then classifies whether the output is likely hallucinated.
The authors show that CHARM provably subsumes earlier attention-based heuristics and report that it consistently outperformed leading approaches across benchmarks, with promising zero-shot transfer across datasets. The practical catch is access: it needs a model’s internal attention and activations, which only self-hosted or open-weight deployments provide.
LOS-Net: detection from output probabilities
A companion paper by an overlapping team targets settings without access to internals (arXiv:2503.14043). Many earlier black-box methods used only the probabilities of the tokens actually generated. The authors define the LLM Output Signature (LOS) as the full sequence of next-token probability distributions plus the probabilities of the chosen tokens, and train a lightweight attention-based model, LOS-Net, on it. They report strong results for detecting both hallucinations and training-data contamination, with very low latency and some transfer across datasets and models. Code is public on GitHub.
There is an important practical limitation: most commercial APIs expose only the log probabilities of the top few candidate tokens, not the full distribution. Methods like LOS-Net therefore fit best with self-hosted models or providers that expose richer probability data; with limited logprobs, performance may differ from what the paper reports.
From explanation to monitoring
Both papers reflect a broader shift in interpretability work. Rather than trying to fully explain how a model with billions of parameters reaches an answer, researchers are building practical monitors that flag likely problems as they happen. For businesses, that is the useful part: a cheap confidence signal that can route doubtful answers to retrieval, a second model or a human before they reach users.
Where detection matters most
In medicine, law and finance, a fabricated drug interaction, citation or figure can cause real harm, and domain-specific fine-tuning does not eliminate hallucination. Detection signals are most valuable here as triage: they decide which answers need checking, not which answers can be trusted blindly.
| Deployment | Practical options |
|---|---|
| Commercial API | Sampling consistency, self-assessment prompts, retrieval with citation checks, limited logprob signals where available |
| Self-hosted open-weight model | All of the above plus internal-state monitors such as CHARM and output-distribution methods |
| Regulated workflows | Detection as triage, plus mandatory source grounding, audit logs and human sign-off |
Research approaches worth knowing
- Semantic entropy (Farquhar et al., Nature, 2024): Sample several answers, cluster them by meaning, and measure uncertainty across meaning clusters rather than exact wording. High semantic entropy signals likely confabulation (Nature).
- SelfCheckGPT (Manakul et al., 2023): A black-box method that checks whether a model’s claims are consistent across multiple sampled responses; facts the model truly “knows” tend to recur, while hallucinated details vary (arXiv:2303.08896).
- Language models (mostly) know what they know (Kadavath et al., 2022): Anthropic researchers found that larger models can be reasonably calibrated when asked to estimate whether their own answers are correct (arXiv:2207.05221).
- Why models hallucinate (Kalai et al., OpenAI, 2025): Argues that standard training and evaluation reward guessing over abstaining, and that benchmarks should give credit for expressing uncertainty (arXiv:2509.04664).
Practical techniques for production systems
- Ground answers in retrieved sources and require citations; then check that each cited passage actually supports the claim.
- Allow abstention. Instruct the model that “I don’t know” is acceptable, and reward it in evaluations.
- Sample-and-compare for high-stakes answers: generate multiple responses and flag disagreement for human review.
- Use verifiers: run code, check calculations, or query databases instead of trusting free-text claims.
- Measure hallucination on your own data with a labeled test set; public benchmark rates rarely transfer directly.
Limits of self-monitoring
Self-checking methods add cost (multiple samples) and can miss errors the model makes consistently—if a model confidently and repeatedly believes a false fact, consistency checks will not catch it. External grounding and human review remain necessary in medical, legal, and financial settings.
Our analysis
Self-monitoring research is promising but not a substitute for grounding. White-box monitors like CHARM need internal access that API customers lack, and output-distribution methods like LOS-Net need richer probability data than most APIs expose. Sampling methods work anywhere but cost more and miss consistent errors. The practical approach is layered: ground answers in retrieved sources, verify checkable claims with tools, use cheap detection signals to route uncertain answers for review, allow the model to abstain, and measure hallucination rates on your own data rather than trusting benchmark figures.