Why Human-Created Content Matters: Synthetic Data and Model Collapse

Human-created content matters because generative models trained over and over on their own output gradually lose information, starting with the rare cases at the edges of the data. Researchers call this model collapse. A 2024 study in Nature demonstrated it in language models and other generative models. Follow-up research shows that collapse can be avoided: in experiments, it did not occur when original human data was kept and new data accumulated alongside it, and checking synthetic data before use also prevented it.

The message, then, is not that synthetic data is bad. Synthetic data is a useful tool, and developers increasingly rely on it. What the research shows is that fresh, well-documented human data remains the anchor that keeps models accurate and varied.

This article explains what synthetic data is used for, and what the model collapse research found and did not find. It then covers why edited human content keeps its value, how provenance standards such as C2PA Content Credentials fit in, and what content owners can do in practice.

Key takeaways

• Synthetic data, meaning text or other content generated by AI models, is increasingly used to fill gaps where real data is scarce, private or expensive to collect.
• Model collapse has been shown most clearly when models are trained repeatedly on generated data that replaces the original data, and the rare "tails" of the distribution disappear first.
• Follow-up studies found that collapse can be avoided by keeping the original real data and accumulating synthetic data alongside it, or by verifying synthetic data before training on it.
• Human-created, edited content is valuable because it carries the variety, rare cases and ground truth that generated data tends to lose.
• Provenance signals such as C2PA Content Credentials record where content came from, but they are optional and do not judge whether content is true.

What synthetic data is and why developers use it

Synthetic data is content produced by a model rather than by people: generated text, question-and-answer pairs, code, images or exercises. A 2024 survey of best practices by Liu and colleagues explains the motivation. Large, varied, high-quality datasets can be "challenging to obtain due to data scarcity, privacy concerns, and high costs", and synthetic data can mimic real-world patterns to fill the gap. The same authors stress the importance of ensuring its "factuality, fidelity, and unbiasedness".

Some results are impressive. The "Textbooks Are All You Need" study trained phi-1, a 1.3-billion-parameter coding model, on 6 billion tokens of "textbook quality" web data plus 1 billion tokens of synthetic textbooks and exercises generated with GPT-3.5. It reached 50.6% pass@1 on the HumanEval coding benchmark. Epoch AI's 2024 analysis noted the limits too: AI-generated synthetic data "has only been shown to reliably improve capabilities in relatively narrow domains like math and coding".

What model collapse is

In July 2024, Shumailov and colleagues published "AI models collapse when trained on recursively generated data" in Nature. They defined model collapse as "a degenerative process affecting generations of learned generative models, in which the data they generate end up polluting the training set of the next generation".

The process has two phases:

• Early collapse: "the model begins losing information about the tails of the distribution". The rare, unusual cases go first.
• Late collapse: "the model converges to a distribution that carries little resemblance to the original one".

The authors showed the effect in large language models, variational autoencoders and Gaussian mixture models. In the language experiments, they fine-tuned Meta's OPT-125m model on the wikitext2 dataset, used it to generate new data, trained the next generation on that output, and repeated the cycle. When no original data was kept, quality declined across generations. When a random 10% of the original training data was mixed into each generation, there was "only minor degradation".

A related study of image generators by Alemohammad and colleagues, titled "Self-Consuming Generative Models Go MAD", reached a similar conclusion. Without "enough fresh real data in each generation", later models are "doomed to have their quality (precision) or diversity (recall) progressively decrease". In its experiments, a fixed set of real data kept alongside the synthetic data only slowed the decline, while enough fresh real data in each generation could prevent it.

Why the tails matter

The tails of a distribution are where the unusual material sits: rare words, minority languages and dialects, specialist terminology, uncommon events, edge cases in a technical field. The Nature authors note that such low-probability events "are often relevant to marginalized groups". In the early stage, a model that has lost them can still look competent on common material, but it becomes more generic and less reliable on exactly the questions where expertise matters. Because the common cases are largely unaffected at first, this kind of loss is easy to miss.

What follow-up research found: collapse is not inevitable

Other researchers pointed out that most earlier studies assumed each generation's synthetic data replaced the data before it, and asked what happens under other assumptions.

• Accumulating data. Gerstgrasser and colleagues (2024) confirmed that "replacing the original real data by each generation's synthetic data does indeed tend towards model collapse". But "accumulating the successive generations of synthetic data alongside the original real data avoids model collapse". In their theoretical model, test error then has "a finite upper bound independent of the number of iterations". The result held for language models, diffusion models and variational autoencoders.
• Verifying data. Feng and colleagues (2024) started from the observation that it is easier to tell good examples from bad than to generate good ones. They showed that "verifiers, even imperfect ones, can indeed be harnessed to prevent model collapse", in experiments on matrix eigenvalue computation and news summarisation.
• Keeping some original data. The Nature study itself found that preserving 10% of the original data in each generation reduced degradation to a minor level.

Across these studies, the outcome depends on how synthetic data is used. Collapse appeared when real data was replaced or generated data went unchecked. It was avoided, or reduced to minor degradation, when real data stayed in the mix or generated data was verified. The studies used different models and experimental set-ups, so their results are not directly comparable.

Why human-created, edited content keeps its value

Most of these mitigations depend on a continuing supply of real, human-created data: the original data that gets kept, the real data that accumulates over time, and the fresh real data each new generation needs. The Nature authors concluded that "the value of data collected about genuine human interactions with systems will be increasingly valuable". They added that, without coordination on provenance, training newer models could become increasingly difficult for anyone lacking web data collected before these tools were widely adopted, or human-generated data at scale.

Supply is also finite. In a 2024 analysis, Epoch AI estimated the effective stock of public human-generated text, adjusted for quality and repetition, at around 300 trillion tokens, with a 90% confidence interval of 100 to 1,000 trillion. Its 80% confidence interval was that, if trends continued, this stock would be fully used at some point between 2026 and 2032.

Content from professional archives has three properties that matter here:

1. Known human origin. Material reliably dated before generative AI tools were widely adopted is very likely to be human-made.
2. Editorial checking. Journalism, textbooks, reference works and peer-reviewed research have been edited, fact-checked or reviewed. That makes them more dependable as reference material, not just as volume.
3. Coverage of the tails. Specialist archives hold exactly the rare vocabulary, local detail and expert knowledge that collapse erodes first.

For the qualities that make content most useful, see what makes content valuable to AI developers. For the stages at which content is used, see how AI models use data.

Provenance signals: telling human and machine content apart

The Nature authors noted that the need to distinguish model-generated data from other data "raises questions about the provenance of content". They also observed that it is unclear how generated content "can be tracked at scale", and suggested community-wide coordination as one option.

One widely discussed technical approach is the C2PA specification from the Coalition for Content Provenance and Authenticity, which sets out how to record where digital content came from and how it was changed. The core unit is the Content Credential, which C2PA describes as "a cryptographically bound structure that records an asset's provenance". It holds statements about the asset, called assertions, such as:

• Origin, such as when and where an asset was created
• Modifications, such as what was done and with which tools
• Use of AI, including a field that identifies actions performed by AI
• Assertions that others can add, such as who created the content or the creator's stated preference about its use for AI training

C2PA's own explainer is clear about the limits. Content Credentials "do not provide value judgments" about whether provenance data is true. Provenance "alone cannot tell you whether the digital content is true, accurate or factual". Adding credentials is optional, and provenance "is not always complete" if an asset passes through tools that do not support the standard. The metadata can also be removed, which is why C2PA describes "durable" credentials that pair cryptographic hashing with watermarking or fingerprinting, so a credential stripped from a file can still be found in cloud-hosted storage. The explainer also says no assumption should be made about an asset's trustworthiness based only on whether it carries credentials.

For archives, the most practical provenance signals are often simpler: reliable creation dates, bylines, edition and version records, and a documented editorial process. Writing these down in a structured way is covered in our guide to documenting a dataset.

Practical steps for content owners

1. Keep originals. Retain the source files and the earliest digital versions, not just processed derivatives.
2. Record creation dates reliably. Publication and creation dates are the simplest evidence of human origin for older material.
3. Flag AI-assisted content in your own production. A metadata field noting where AI tools generated or edited material keeps your archive's record accurate.
4. Document the editorial process. Note which material was edited, fact-checked or peer-reviewed.
5. Consider Content Credentials for new media. Where your tools support C2PA, credentials can be attached to the images and other media you publish.
6. Separate human and generated material in any delivery. Label each item clearly, so a developer can decide how to mix them.
7. Do not overclaim. Describe provenance as it is recorded, including gaps.

How Druid Learning helps

Druid Learning is an AI archival processing platform, based in Dublin and New York, that turns content archives into AI-ready datasets. Archive Optimization processes archives in formats including OCR scans, PDF, XML, IDML, EPUB, WAV, MP3 and MP4 into an indexed, labelled dataset. Druid has processed 10M+ assets and handles content in 140+ languages. Through Data Partnerships, Druid prepares an archive, raw or structured, and connects content owners with AI companies.

Frequently asked questions

What is model collapse in simple terms?

Model collapse is a gradual loss of quality that happens when generative AI models are trained, generation after generation, on content produced by earlier models instead of on original human data. Rare and unusual patterns disappear first, so outputs become more generic. Eventually, the model's view of the world can drift far from the original data. A 2024 study in Nature by Shumailov and colleagues showed the effect in language models and other generative models.

Is synthetic data bad for AI models?

No. Synthetic data is increasingly used and can help, especially in narrow domains such as coding and maths. Research suggests problems arise when synthetic data replaces real data over many generations. Studies found that keeping original human data and accumulating synthetic data alongside it can avoid collapse, and that filtering synthetic data with a verifier, even an imperfect one, can also prevent it. Careful mixing and checking are what matter.

Does model collapse mean AI models will inevitably get worse?

Not according to current research. Model collapse has been demonstrated in controlled experiments where generated data replaced real data at each step. Follow-up work by Gerstgrasser and colleagues found that when real data is kept and new data accumulates, error stays bounded and collapse is avoided. The findings point to a practical need: developers must keep access to sufficient real, human-created data and to good ways of checking generated data.

How can anyone tell whether content was created by a human?

There is no single reliable test. Provenance standards such as C2PA Content Credentials can record an asset's origin, edits and use of AI in a cryptographically bound record, but they are optional, can be removed, and do not judge whether content is true. For archives, reliable creation dates, bylines and documented editorial processes are often the clearest practical evidence of human origin.

Sources

• AI models collapse when trained on recursively generated data — Nature
• Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data — arXiv
• Self-Consuming Generative Models Go MAD — arXiv
• Beyond Model Collapse: Scaling Up with Synthesized Data Requires Verification — arXiv
• Best Practices and Lessons Learned on Synthetic Data — arXiv
• Textbooks Are All You Need — arXiv
• Will we run out of data? Limits of LLM scaling based on human-generated data — Epoch AI
• C2PA and Content Credentials Explainer — C2PA