Giving an AI more time to think or access to the internet actually makes it more likely to be confidently wrong.
April 3, 2026
Original Paper
Bayesian Elicitation with LLMs: Model Size Helps, Extra "Reasoning" Doesn't Always
arXiv · 2604.01896
The Takeaway
The common belief that extra reasoning steps lead to better results is debunked for uncertainty; extra effort often leads to extreme arrogance. This shows that more 'thought' doesn't help an AI know when it doesn't know something.
From the abstract
Large language models (LLMs) have been proposed as alternatives to human experts for estimating unknown quantities with associated uncertainty, a process known as Bayesian elicitation. We test this by asking eleven LLMs to estimate population statistics, such as health prevalence rates, personality trait distributions, and labor market figures, and to express their uncertainty as 95\% credible intervals. We vary each model's reasoning effort (low, medium, high) to test whether more "thinking" im