Model Collapseedit

Model Collapse is a degenerative process in which a model trained recursively on generated or biased data loses information about the original data distribution. Collapse can appear as mode loss, reduced diversity, distorted class proportions, or worsening sample quality over generations.¹

Research contextedit

Model collapse is a failure mode studied in Synthetic Data research. Synthetic data are not inherently harmful: the failure depends on how generated data are selected, mixed, and reused. This risk motivates careful data governance and collaborative verification. Low-resource conditions are particularly important: if tail regions are poorly covered from the beginning, collapse may arrive earlier and affect underrepresented content more severely.

Connection to Qiao's workedit

Qiao's ICML 2026 paper studies when sample-selection bias precipitates collapse in low-resource verification regimes. The work is connected to Wasserstein geometry because distributional distances can provide signals about drift, and to data silos because no single party may have the full distribution. This work addresses a broader reliability problem: data processes can silently degrade models even when the model architecture remains unchanged.

See alsoedit

• When Sample Selection Bias Precipitates Model Collapse
• Recursive Synthetic Data Training
• Sample Selection Bias
• Collaborative Evaluation

Footnotesedit

1. Shumailov et al., "AI models collapse when trained on recursively generated data", Nature 631 (2024), define model collapse in the context of indiscriminate recursive use of generated data and report the phenomenon across language models, variational autoencoders, and Gaussian mixture models. ↩