Legal AI errors are rarely about 'hallucinations' and almost always about picking the wrong level of detail.
April 23, 2026
Original Paper
Not Hallucination but Granularity: Error Taxonomy and Quality Audit of LLM-Based Legal Information Extraction
SSRN · 6496861
The Takeaway
The industry is obsessed with stopping AI from lying, but in specialized fields, the problem is different. High-performance pipelines for legal data usually get the facts right but the granularity wrong. A model might identify a contract clause but fail to capture the specific sub-condition that matters. This granularity mismatch is far more common than outright fabrication in professional domains. Improving legal AI requires focusing on precision and hierarchy rather than just truthfulness. We need to stop worrying about ghosts and start worrying about blurry vision.
From the abstract
The dominant concern in LLM-based information extraction is hallucination-the fabrication of content not present in the source text. We present an end-to-end expert audit of a production multi-stage legal extraction pipeline, evaluating 1,042 items across 100 Brazilian court decisions from four tribunals. With its production model configuration (Grok-3-mini, Claude Sonnet 4.6, GPT-5-4), the pipeline achieves 96.0% precision with zero hallucinations. The dominant error is not fabrication but gran