When Models Don’t Collapse: On the Consistency of Iterative MLE

This research explores model collapse, a phenomenon where generative models degrade after being repeatedly trained on their own synthetic outputs. The authors provide a theoretical framework using Maximum Likelihood Estimation (MLE) to determine when this process can be avoided. They demonstrate that if models meet specific regularity and smoothness assumptions, they can remain consistent and accurate even as the proportion of real data diminishes. Conversely, the study provides the first rigorous proof that without these structural assumptions, model collapse can occur abruptly or over time, even when real data is preserved. Ultimately, the findings suggest that data accumulation alone does not guarantee stability; rather, the underlying mathematical properties of the distribution family are what prevent performance failure.