Latent Entropy-Aware Decoding (LEAD) mitigates hallucinations by switching between discrete token and continuous probability-weighted embeddings based on real-time uncertainty.
arXiv · March 17, 2026 · 2603.13366
The Takeaway
By identifying that transition words (e.g., 'however', 'wait') represent high-entropy states where models typically hallucinate, LEAD injects latent semantic context to stabilize the reasoning trajectory. It is an efficient, plug-and-play decoding strategy for multimodal models.
From the abstract
Recent advancements in multimodal large reasoning models (MLRMs) have significantly improved performance in visual question answering. However, we observe that transition words (e.g., because, however, and wait) are closely associated with hallucinations and tend to exhibit high-entropy states. We argue that adequate contextual reasoning information can be directly extracted from the token probability distribution. Inspired by superposed representation theory, we propose leveraging latent superp