Model collapse
Also known as: AI model collapse, model autophagy disorder, recursive training collapse
Model collapse is a decline in AI model quality that can happen when models are trained repeatedly on content made by other AI models instead of by people.
Each generation loses a bit of the variety and rare details found in real human data, until outputs become bland, repetitive, or wrong. While the risk is real in experiments, AI labs reduce it by mixing in fresh human data, filtering carefully, and using synthetic data in controlled ways.
Model collapse, explained at three levels
At home
Imagine photocopying a photocopy, then copying that copy, again and again. Each round loses a little detail until the image turns blurry and smudged. Model collapse is the AI version. When a model learns mostly from text or images produced by earlier models instead of by people, each generation keeps the common patterns but loses the rare details, and quality slowly fades.
At work
Model collapse is a decline in quality and variety that happens when AI models are trained repeatedly on AI-generated content instead of human-made data. As more of the web becomes AI-written, it is a real concern for anyone training models. For businesses, it means original human data, such as expert writing or human translations, is becoming a valuable asset worth protecting.
At home
AI models tend to produce the most typical answers, the safe middle of what they learned. If a new model trains on those answers, it sees even fewer unusual examples, and its own outputs become more average still. Repeat that over several generations and rare facts, quirky styles, and minority dialects disappear. Eventually outputs can turn repetitive or nonsensical, as researchers showed in experiments with both text and image models.
At work
Collapse starts in the 'tails' of the data distribution, meaning rare topics, minority languages, and unusual cases, because sampling from a model under-represents them and each generation compounds the loss. Research shows the key factor is replacement: when synthetic data replaces real data, quality degrades, but when it accumulates alongside retained real data, collapse is largely avoided. Mitigations include keeping original datasets, labelling synthetic content at creation, filtering it, and verifying synthetic data before training.
At home
As AI writing fills the web, every future scrape contains more synthetic text, and current tools can't reliably tell it apart from human work. Collections captured before 2022, along with books and licensed archives, are becoming prized for that reason. While labs can manage collapse with careful filtering, they need clean human data to do it. In my opinion, that could make authentic human work more valuable, not less.
At work
Labs manage collapse through data provenance, preserved human-data reserves, and synthetic data generated with verification, such as code that must compile or math that must check out, which filters errors before they compound. Content credentials under C2PA and watermarks such as Google's SynthID can help flag synthetic data, but they cover only cooperating tools and detection remains unreliable. Teams fine-tuning in-house models should label AI-generated content at creation and track the human-to-synthetic ratio in every dataset.
Real-world examples
At home
Fatima noticed that many online recipes she found started sounding identical and oddly generic. A friend explained that much of the content was AI-written, and that training future AI on it could make the problem worse.
At work
Jerome manages data for a Canadian translation company. He keeps a carefully labelled archive of human-translated documents, because training only on machine translations made the company's model lose regional phrasing.
Common confusions
- Any synthetic data causes model collapse. — Collapse comes from replacing real data over repeated generations. Carefully verified synthetic data mixed with real data is widely used safely.
- Model collapse means AI will suddenly stop working. — It's a gradual loss of variety and accuracy, mostly affecting rare information first.
For analysts
If your company trains or fine-tunes models, I'd preserve high-quality human data as a long-term asset. Clear labelling of AI-generated content now saves cleanup later.
History & origin
Concerns grew as AI-generated text and images spread across the web after ChatGPT's release in late 2022. In May 2023, Ilia Shumailov and colleagues from Oxford, Cambridge, and other institutions posted a preprint called 'The Curse of Recursion,' showing models forgot rare information when trained on their own outputs. Their peer-reviewed version appeared in the journal Nature in July 2024. Other studies, including work from Stanford and Rice University, showed similar effects for image models, sometimes called 'model autophagy disorder.' Later research suggested that accumulating synthetic data alongside real data, rather than replacing it, largely avoids collapse. In my view, model collapse is less an AI apocalypse than a strong argument for valuing original human writing.