Self-driving AI doesn't actually see the physics of the road. it just predicts the words that describe driving.
April 29, 2026
Original Paper
EgoDyn-Bench: Evaluating Ego-Motion Understanding in Vision-Centric Foundation Models for Autonomous Driving
arXiv · 2604.22851
The Takeaway
Vision-language models are increasingly used to help autonomous vehicles understand their environment. This benchmark reveals that these models derive almost all their physical reasoning from their text training rather than the actual video feed. When the visual input is changed, the model's understanding of the vehicle's motion remains stagnant. This structural deficit means the AI is essentially driving by repeating a textbook rather than looking out the windshield. Relying on these models for safety-critical driving tasks could be dangerous until they can truly integrate visual physics.
From the abstract
While Vision-Language Models (VLMs) have advanced highlevel reasoning in autonomous driving, their ability to ground this reasoning in the underlying physics of ego-motion remains poorly understood. We introduce EgoDyn-Bench, a diagnostic benchmark for evaluating the semantic ego-motion understanding of vision-centric foundation models. By mapping continuous vehicle kinematics to discrete motion concepts via a deterministic oracle, we decouple a model's internal physical logic from its visual pe