Photon enables efficient 3D medical volume understanding through adaptive token scheduling and a novel 'gradient restoration' backpropagation rule.
March 27, 2026
Original Paper
Photon: Speedup Volume Understanding with Efficient Multimodal Large Language Models
arXiv · 2603.25155
The Takeaway
Standard 3D medical VQA is crippled by token limits or fixed compression that loses detail. Photon's variable-length scheduling reduces compute while preserving subtle diagnostic features, making 3D Foundation Models more practical for clinical use.
From the abstract
Multimodal large language models are promising for clinical visual question answering tasks, but scaling to 3D imaging is hindered by high computational costs. Prior methods often rely on 2D slices or fixed-length token compression, disrupting volumetric continuity and obscuring subtle findings. We present Photon, a framework that represents 3D medical volumes with token sequences of variable length. Photon introduces instruction-conditioned token scheduling and surrogate gradient propagation to