LLMs getting trained on LLM-generated data is likely to lead to model collapse. Meanwhile, a good 70% / 80%? maybe even 90% of the internet is now LLM-generated.
What are the next sources of data that can be used for future training?
(example from a 2024 Nature by Shumailov et al. go jackrabbits!)
