The internet is running out of fresh material
For years, the recipe behind better language models was simple. Feed a bigger model more internet text, and it improved. That recipe worked well. Then researchers hit a limit nobody had planned for. There’s only so much high-quality, human-written text on the open web. Some researchers now think the usable supply could run thin within the next several years. That leaves the field with a genuinely new problem: what do you use for synthetic data AI training once fresh human writing starts drying up?
The industry’s answer has largely been synthetic data. Labs generate text, images, and other material with AI, then feed it back in to train the next round of models. It sounds reasonable on paper. But this approach to synthetic data AI training carries a real risk, and researchers have given it an unsettling name: Model Autophagy Disorder, or MAD. It’s part of a wider conversation known more broadly as model collapse.
What is Model Autophagy Disorder?
“Autophagy” comes from biology. It means “self-eating.” Borrowed for AI, the term describes what happens when a model trains repeatedly on data produced by earlier versions of itself, or by other AI systems, instead of on material written by real people. This doesn’t produce a stable loop. It produces a slow, fairly predictable decline.
The root cause is straightforward. No generative model fully captures the texture of real human data — its variety, its oddities, its long tail of rare examples. Each time a model generates output, that output comes out a little narrower than the distribution it learned from. Common patterns get reinforced. Rare ones start to fade. Feed that narrowed data into a new model, then train the next model on its output, and the narrowing doesn’t just continue. It compounds. Picture a photocopy of a photocopy of a photocopy. Each pass loses a bit more of the original.
How the collapse plays out
A few failure modes show up consistently in the research:
- Diversity thins out. Across several generations of recursive training, models lose the rare and unusual patterns present in the original human data. Output drifts toward something flatter and more “averaged.”
- Mistakes get inherited, not corrected. Quirks or errors in an earlier model’s output become part of the synthetic training data. The next model absorbs them rather than fixing them. It’s a feedback loop that reinforces itself.
- Edge cases suffer first. Rare examples disappear early in this narrowing process. Models raised on recycled synthetic data get noticeably worse at unusual or novel inputs — exactly where reliability tends to matter most.
None of this is hypothetical. Researchers demonstrated it directly, running models through successive generations of recursive training and watching quality drop once too little real data remained in the mix.
Why the pressure is mounting
A few trends are converging to make synthetic data AI training more urgent than it looked a few years ago.
AI writing is everywhere online now
More people lean on generative tools to draft articles, produce images, and churn out other content. A growing share of what’s on the internet was never written by a person. Models scraping the web for fresh training material increasingly swallow AI-generated text, often without anyone flagging or filtering it.
Human-written material isn’t growing as fast
The internet keeps expanding. But the pace of new, high-quality writing from real people lags well behind how fast frontier labs want to scale their data. Synthetic data has rushed in to fill that gap.
Synthetic data still has real value
None of this means synthetic data is bad. Used carefully, it remains a genuinely valuable tool, especially where real-world data is rare, sensitive, or costly to collect. The related field of Generative Reality Engineering builds synthetic training worlds for self-driving cars and robotics, and depends on exactly this kind of data. The real trick is avoiding the collapse dynamics described above.
How researchers are fighting back
A handful of strategies have emerged for capturing the upside of synthetic data without the downside.
Keep a solid core of real data. This is the most reliable fix researchers have found, and it’s almost anticlimactic in its simplicity: never let synthetic data fully replace human-written data. Even a modest share of authentic, diverse material in each training run goes a long way toward holding quality steady.
Track where the data came from. Labs now invest heavily in tools that identify and label AI-generated content, both in their own pipelines and in scraped web data. That lets them control the synthetic-to-human ratio on purpose. This overlaps with broader content provenance efforts like the C2PA standard, built originally for spotting deepfakes but useful here too.
Generate synthetic data on purpose. Rather than letting a model freely produce training data that loops back in unchecked, many labs run tighter pipelines. Human reviewers check output. Generation targets specific gaps in the real dataset. Quality filters run before anything synthetic enters the training set.
Protect the tail of the distribution. Some research focuses on sampling and filtering techniques built specifically to keep rare, unusual examples alive in synthetic datasets. This pushes back against a model’s natural tendency to smooth everything toward the average.
Avoid a single model feeding on itself. Labs increasingly generate synthetic data from several different model architectures, rather than one model repeatedly consuming its own output. This appears to reduce the risk of baking in one model’s blind spots.
Why this problem is so strange to sit with
There’s something eerie about this whole situation. An intelligence built from human writing is running low on that writing. The proposed fix has it feeding on its own output, slowly losing richness in the process, like a photocopier blurring its own copies. It reads like science fiction. Except it’s sitting in peer-reviewed papers with real consequences for how the next wave of AI gets built.
It also puts something in perspective. The messy, sprawling diversity of human writing — the disagreements, the odd personal styles, the obscure niche knowledge — is a genuinely scarce resource. AI progress depends on it more than most people realized.
How to get involved in this field
- Learn machine learning fundamentals, especially data distributions, statistical sampling, and generalization.
- Follow published research on model collapse and synthetic data, a fast-growing corner of AI research.
- Study data curation and provenance work, an increasingly specialized skill as labs move away from raw web scraping.
- Get hands-on with synthetic data generation tools used in production, and learn where they go wrong.
If you’re building out training pipelines yourself, our guide on data quality auditing for machine learning teams walks through practical filtering steps, and our piece on Generative Reality Engineering covers a related use case in robotics and autonomous vehicles.
Where this leaves us
Model Autophagy Disorder is a reminder that AI progress was never purely about bigger models and more compute. It’s a story about data. Even on an internet that feels bottomless, data has real limits. Whoever learns to blend synthetic and authentic data responsibly, preserving diversity instead of quietly grinding it down, will shape whether AI keeps improving over the next decade.