Detects when object detectors fail to see safety-critical objects by measuring semantic misalignment with foundation model embeddings.
March 27, 2026
Original Paper
Knowledge-Guided Failure Prediction: Detecting When Object Detectors Miss Safety-Critical Objects
arXiv · 2603.25499
The Takeaway
Unlike traditional OOD detection, this monitors the internal consistency of a detector's perception at runtime. It improves recall of missed pedestrians from 64% to 84% without retraining the primary detector.
From the abstract
Object detectors deployed in safety-critical environments can fail silently, e.g. missing pedestrians, workers, or other safety-critical objects without emitting any warning. Traditional Out Of Distribution (OOD) detection methods focus on identifying unfamiliar inputs, but do not directly predict functional failures of the detector itself. We introduce Knowledge Guided Failure Prediction (KGFP), a representation-based monitoring framework that treats missed safety-critical detections as anomali