Model collapse
Also known as: AI model collapse, model autophagy disorder, recursive training collapse
Model collapse is a decline in AI model quality that can happen when models are trained repeatedly on content made by other AI models instead of by people.
Each generation loses a bit of the variety and rare details found in real human data, until outputs become bland, repetitive, or wrong. While the risk is real in experiments, AI labs reduce it by mixing in fresh human data, filtering carefully, and using synthetic data in controlled ways.
Model collapse, explained at three levels
At home
Imagine photocopying a photocopy over and over. Each copy gets a little blurrier. Model collapse happens when AI learns mostly from other AI's work, and quality slowly fades.
At work
Model collapse is a reason AI companies value fresh, high-quality human data, since too much AI-made data can make models worse.
At home
AI tends to produce the most typical answers. If new AI learns only from those, unusual facts and styles disappear over time, and everything starts sounding the same.
At work
Collapse shows up first in the 'tails' of the data distribution: rare topics, minority languages, and unusual cases. Mitigations include keeping original data in the mix, labelling and filtering synthetic content, and verifying synthetic data quality.
At home
As more online content is AI-written, it may get harder to find purely human writing to train on. In my opinion, that could make authentic human work more valuable, not less.
At work
Labs manage collapse through data provenance, synthetic data generated with verification (such as checking code compiles or math is correct), and preserving human-data reserves. Content credentials and watermarking may help identify synthetic data, though detection remains unreliable.
Real-world examples
At home
Fatima noticed that many online recipes she found started sounding identical and oddly generic. A friend explained that much of the content was AI-written, and that training future AI on it could make the problem worse.
At work
Jerome manages data for a Canadian translation company. He keeps a carefully labelled archive of human-translated documents, because training only on machine translations made the company's model lose regional phrasing.
Common confusions
- Any synthetic data causes model collapse. — Collapse comes from replacing real data over repeated generations. Carefully verified synthetic data mixed with real data is widely used safely.
- Model collapse means AI will suddenly stop working. — It's a gradual loss of variety and accuracy, mostly affecting rare information first.
For analysts
If your company trains or fine-tunes models, I'd preserve high-quality human data as a long-term asset. Clear labelling of AI-generated content now saves cleanup later.
History & origin
Concerns grew as AI-generated text and images spread across the web after ChatGPT's release in late 2022. In May 2023, Ilia Shumailov and colleagues from Oxford, Cambridge, and other institutions posted a preprint called 'The Curse of Recursion,' showing models forgot rare information when trained on their own outputs. Their peer-reviewed version appeared in the journal Nature in July 2024. Other studies, including work from Stanford and Rice University, showed similar effects for image models, sometimes called 'model autophagy disorder.' Later research suggested that accumulating synthetic data alongside real data, rather than replacing it, largely avoids collapse. In my view, model collapse is less an AI apocalypse than a strong argument for valuing original human writing.