Recursive Synthetic Data Trainingedit

Recursive Synthetic Data Training is a training process in which generated data from one model generation become part of the training set for a later generation. The process can be intentional, as in self-training or synthetic-data bootstrapping, or incidental, as generated content enters future training corpora.¹

Research contextedit

Recursive reuse can lead to model collapse: a one-time synthetic augmentation may be useful, while repeated reuse can amplify distributional errors. Recursive training is the loop; collapse is one possible degenerative result.

Connection to Qiao's workedit

When Sample Selection Bias Precipitates Model Collapse studies recursive training under local sample-selection bias. The paper's setting is especially relevant to AI and networks because the data process is distributed and often low-resource: parties may see different data, select different samples, and only share limited signals. Recursive synthetic-data training therefore becomes a cross-silo reliability problem, not only a generative-model problem.

See alsoedit

• Synthetic Data
• Model Collapse
• Sample Selection Bias
• Data Silos

Footnotesedit

1. Shumailov et al., "AI models collapse when trained on recursively generated data", Nature 631 (2024), study this recursive framing and show that indiscriminate reuse can progressively erase information from the original distribution; their experiments also show less degradation when some original data are retained. ↩