Longitudinal evidence reveals that successive ChatGPT versions are converging in output diversity, suggesting potential model collapse from synthetic data saturation.
arXiv · March 16, 2026 · 2603.12683
Why it matters
It provides empirical verification that LLM outputs are becoming less diverse over time, likely due to the recursive training on synthetic data that has flooded the internet. For researchers, this highlights an urgent need for data provenance and non-synthetic data sources to prevent terminal model degradation.
From the abstract
Large Language Models (LLMs) that undergo recursive training on synthetically generated data are susceptible to model collapse, a phenomenon marked by the generation of meaningless output. Existing research has examined this issue from either theoretical or empirical perspectives, often focusing on a single model trained recursively on its own outputs. While prior studies have cautioned against the potential degradation of LLM output quality under such conditions, no longitudinal investigation h