Multimodal LLMs suffer from a 'cognitive mismatch' where they succeed at complex reasoning while failing at basic discrete symbol recognition.
March 20, 2026
Original Paper
Cognitive Mismatch in Multimodal Large Language Models for Discrete Symbol Understanding
arXiv · 2603.18472
The Takeaway
The paper reveals that VLM 'reasoning' is often just linguistic probability rather than true visual perception. This challenges the assumption that benchmarks requiring high-level thought are valid proxies for a model's underlying visual understanding of mathematical or scientific symbols.
From the abstract
While Multimodal Large Language Models (MLLMs) have achieved remarkable success in interpreting natural scenes, their ability to process discrete symbols -- the fundamental building blocks of human cognition -- remains a critical open question. Unlike continuous visual data, symbols such as mathematical formulas, chemical structures, and linguistic characters require precise, deeper interpretation. This paper introduces a comprehensive benchmark to evaluate how top-tier MLLMs navigate these "dis