How LLM Hallucination Detection Works: Attention Graphs (CHARM), Output Signatures (LOS-Net), Semantic Entropy and Practical Safeguards

A guide to detecting LLM hallucinations: white-box methods that read attention and activations (CHARM, ICLR 2026), black-box methods using output probability distributions (LOS-Net), sampling-based checks like semantic entropy and SelfCheckGPT, their limits, and the safeguards production systems should use.

PUBLISHED // UPDATED
DURATION // 06 MIN READ
Abstract visualization of neural network nodes connected by glowing pathways, with some nodes highlighted in red to represent detected anomalies
FIG.01 // VISUAL DOCUMENTATION Abstract visualization of neural network nodes connected by glowing pathways, with some nodes highlighted in red to represent detected anomalies

Hallucination detection has moved from external fact-checking toward reading signals from the model itself. Two research directions stand out: white-box methods that analyse a model’s internal attention and activations, and black-box methods that analyse its output probabilities or the consistency of repeated answers. A group including Technion’s Haggai Maron has published examples of both, CHARM and LOS-Net. These methods can flag likely errors cheaply, but none is reliable enough to replace grounding in sources and human review for high-stakes uses. This guide explains how each approach works, what it needs and how to combine them in practice.

The main families of detection methods

ApproachExampleWhat it needsStrengthsLimits
Internal states (white-box)CHARM (attention graphs + activations)Access to model internalsCan localise suspicious tokens; low extra cost per answerOnly for self-hosted or open-weight models; needs training data
Output distributions (grey-box)LOS-Net (LLM Output Signature)Next-token probability distributionsWorks without internals; very low latencyCommercial APIs usually expose only top token probabilities
Sampling consistency (black-box)SelfCheckGPT, semantic entropySeveral sampled answersWorks with any modelMultiplies cost; misses consistently held false beliefs
Self-assessmentAsking the model whether its answer is correctNothing extraSimpleCalibration varies by model and task
External verificationRetrieval, code execution, database checksTrusted sources or toolsMost reliable for checkable factsRequires infrastructure; not every claim is checkable

CHARM: reading attention as graphs

Earlier white-box methods summarised attention with simple statistics, such as how much the model attended to the prompt versus its own generated text. CHARM, presented at ICLR 2026 by Fabrizio Frasca, Guy Bar-Shalom, Yftah Ziser and Haggai Maron, instead represents a model’s computation as an attributed graph: tokens are nodes, edges follow attention flows, and both carry features from attention scores and activations (arXiv:2509.24770). A graph neural network then classifies whether the output is likely hallucinated.

The authors show that CHARM provably subsumes earlier attention-based heuristics and report that it consistently outperformed leading approaches across benchmarks, with promising zero-shot transfer across datasets. The practical catch is access: it needs a model’s internal attention and activations, which only self-hosted or open-weight deployments provide.

LOS-Net: detection from output probabilities

A companion paper by an overlapping team targets settings without access to internals (arXiv:2503.14043). Many earlier black-box methods used only the probabilities of the tokens actually generated. The authors define the LLM Output Signature (LOS) as the full sequence of next-token probability distributions plus the probabilities of the chosen tokens, and train a lightweight attention-based model, LOS-Net, on it. They report strong results for detecting both hallucinations and training-data contamination, with very low latency and some transfer across datasets and models. Code is public on GitHub.

There is an important practical limitation: most commercial APIs expose only the log probabilities of the top few candidate tokens, not the full distribution. Methods like LOS-Net therefore fit best with self-hosted models or providers that expose richer probability data; with limited logprobs, performance may differ from what the paper reports.

From explanation to monitoring

Both papers reflect a broader shift in interpretability work. Rather than trying to fully explain how a model with billions of parameters reaches an answer, researchers are building practical monitors that flag likely problems as they happen. For businesses, that is the useful part: a cheap confidence signal that can route doubtful answers to retrieval, a second model or a human before they reach users.

Where detection matters most

In medicine, law and finance, a fabricated drug interaction, citation or figure can cause real harm, and domain-specific fine-tuning does not eliminate hallucination. Detection signals are most valuable here as triage: they decide which answers need checking, not which answers can be trusted blindly.

DeploymentPractical options
Commercial APISampling consistency, self-assessment prompts, retrieval with citation checks, limited logprob signals where available
Self-hosted open-weight modelAll of the above plus internal-state monitors such as CHARM and output-distribution methods
Regulated workflowsDetection as triage, plus mandatory source grounding, audit logs and human sign-off

Research approaches worth knowing

  • Semantic entropy (Farquhar et al., Nature, 2024): Sample several answers, cluster them by meaning, and measure uncertainty across meaning clusters rather than exact wording. High semantic entropy signals likely confabulation (Nature).
  • SelfCheckGPT (Manakul et al., 2023): A black-box method that checks whether a model’s claims are consistent across multiple sampled responses; facts the model truly “knows” tend to recur, while hallucinated details vary (arXiv:2303.08896).
  • Language models (mostly) know what they know (Kadavath et al., 2022): Anthropic researchers found that larger models can be reasonably calibrated when asked to estimate whether their own answers are correct (arXiv:2207.05221).
  • Why models hallucinate (Kalai et al., OpenAI, 2025): Argues that standard training and evaluation reward guessing over abstaining, and that benchmarks should give credit for expressing uncertainty (arXiv:2509.04664).

Practical techniques for production systems

  1. Ground answers in retrieved sources and require citations; then check that each cited passage actually supports the claim.
  2. Allow abstention. Instruct the model that “I don’t know” is acceptable, and reward it in evaluations.
  3. Sample-and-compare for high-stakes answers: generate multiple responses and flag disagreement for human review.
  4. Use verifiers: run code, check calculations, or query databases instead of trusting free-text claims.
  5. Measure hallucination on your own data with a labeled test set; public benchmark rates rarely transfer directly.

Limits of self-monitoring

Self-checking methods add cost (multiple samples) and can miss errors the model makes consistently—if a model confidently and repeatedly believes a false fact, consistency checks will not catch it. External grounding and human review remain necessary in medical, legal, and financial settings.

Our analysis

Self-monitoring research is promising but not a substitute for grounding. White-box monitors like CHARM need internal access that API customers lack, and output-distribution methods like LOS-Net need richer probability data than most APIs expose. Sampling methods work anywhere but cost more and miss consistent errors. The practical approach is layered: ground answers in retrieved sources, verify checkable claims with tools, use cheap detection signals to route uncertain answers for review, allow the model to abstain, and measure hallucination rates on your own data rather than trusting benchmark figures.