I want to challenge the idea that this is the worst AI will ever be.

[-]

Feztopia@reddit

Dude we are in a time where models are trained on synthetic data from models... on purpose.

Reply

[-]

And it's a research topic on whether these models are actually capped at the level of the "teacher" model, or if they're learning at all (e.g. this hotly debated paper: https://arxiv.org/abs/2305.15717).

Reply

[-]

Feztopia@reddit

There is also research about models teaching themselves and improving. We also have models being teached by multiple models AND models being teached by human data + model data and leading to improvements.

Reply

[-]

beezbos_trip@reddit

It’s more about if the model makers start RLHF for advertisers to bake them into responses.

Reply

[-]

SomeOddCodeGuy@reddit

I wish I knew where the model collapse theory came from, but half models on huggingface point blank disprove that. Some of the best fine-tunes, like Dolphin, and some of the best models, like Mistral, have TONS of synthetic data trained into them. And the improvement, in many instances, is very evident. 5 minutes on huggingface will show that the model collapse theory didn't withstand the test of time.

Reply

[-]

Rasekov@reddit

The concept of model degeneration on synthetic data got very popular after the release of this paper: [Self-Consuming Generative Models Go MAD ](https://arxiv.org/abs/2307.01850). It was all over news sites last year at the same time the debate over AI generated images was reaching it's peak. Extrapolating it to auto-regressive LLMs from Diffusion models doesn't really work since Diffusion works in a continuous latent space with very close semantic relationship between adjacent values but current LLMs are discrete and select tokens based on probabilistic models where the numerical proximity of token IDs does not imply semantic similarity. That makes the issue orders of magnitude less likely to develop(if at all possible). If you are interested there is a nice video from oxen.ai about [text based diffusion models](https://www.youtube.com/watch?v=1mG678f1ZYU) that also touches this topic a bit.

Reply

[-]

Helpful-User497384@reddit

im sure there will figure out ways to get around that in time. and besides look how stupid some people are even though they have access to all human knowledge its not so much about HOW MUCH data ai has but HOW it processes it that is the main thing. so while right now the methood is dumping as much data into it to learn and improve there may be other ways to make ai smarter and what without degrading it. maybe training on a smaller subset yet having ai that can actually "think" and "process it" better.

Reply

[-]

Rasekov@reddit

You seem to be confusing LLMs with Diffusion models for images. Many, if not all, of those articles and news about training on synthetic data leading to heavy degradation were written in the context of AI image generation where errors accumulated and the quality went to shit. With LLMs synthetic data has been in use for a good while and has helped improve the open source models. As a very relevant example Alpaca was trained on 800$ worth of API calls to GPT4 and heavily improved the performance of Llama 1 7B. More recently tons of companies are pursuing synthetic datasets, as a relevant example Microsoft with Phi. The only way the "dead internet" would affect LLMs in the mid to long term is if the current models and datasets are somehow lost and needed to be rebuilt from scratch. Even then it would be a matter of curation and refinement, not "permanent degradation of all models".

Reply

[-]

printr_head@reddit

Of there was 100% efficiency in learning in an llm I would believe you but there isn’t. Synthetic data without some form of signal loss is impossible. There are artifacts in gpt all over the place.

Reply

[-]

trajo123@reddit

You can create synthetic data without involvement of llms. For programming, mathematics, logic or any other formal language which can be used to train llms to improve reasoning, data can be generated with non AI tools (theorem provers, solvers, genetic programming + program verification, etc, etc)

Reply

[-]

printr_head@reddit

Oo you mentioned my specialty. And good point but even with those GA GP can only approach the optimal solution which leaves room still for signal loss. But they do potentially produce novel information.

Reply

[-]

Rasekov@reddit

You dont need to believe me, I gave two very relevant examples. Do you have a source for that inevitable degradation you mention? something I can read/reference? Just in the last weeks the most impressive models released all had synthetic data used in one way or another in their training. Phi 3: [The innovation lies entirely in our dataset for training, a scaled-up version of the one used for phi-2, composed of heavily filtered web data and synthetic data. ](https://arxiv.org/abs/2404.14219) LLama 3: [We found that previous generations of Llama are good at identifying high-quality data, so we used Llama 2 to help build the text-quality classifiers that are powering Llama 3. We also leveraged synthetic data to train in areas such as coding, reasoning, and long context. For example, we used synthetic data to create longer documents to train on.](https://ai.meta.com/blog/meta-llama-3-meta-ai-responsibility/) WizardLM: [In the past one year, we built a fully AI powered synthetic training system:](https://file.notion.so/f/f/433cf83e-3131-4b47-931d-e2ac61635f72/ca5a865b-0ff7-4113-b27f-7889489527df/WizardLM_2_saved.pdf?id=6cc6985d-e609-4a08-8e8a-e4bfdcb0daee&table=block&spaceId=433cf83e-3131-4b47-931d-e2ac61635f72&expirationTimestamp=1716256800000&signature=6i67r35B12JJpiQN-GAYYMQh3gm-pPK4ZPsvwEWASDQ&downloadName=WizardLM+2_saved.pdf) I could keep going but I think that, lacking any good counter example, sufficiently makes my point.

Reply

[-]

Comprehensive-Tea711@reddit

I'm not agreeing with the OP, but some of your examples are off target. Improving a worse model by training on synthetic data from a superior model shouldn't pose a problem. I can build a great model just off of GPT 4 data. Can we bootstrap GPT 4 to GPT 4.5 or GPT 5 just off of GPT 4 data? There are other issues with the line of thinking, though.

Reply

[-]

Rasekov@reddit

Thats a very good point. Still that might only rule out Phi as an example(if they used GPT4 as a source, the paper doesn't specify). Regarding if GPT4 could be used to generate data for GPT5 Meta used Llama2 as a source for Lama 3(one of many) and WizardLM focused on automating parts beyond the text generation. Synthetic dataset generation isnt just batching 5M prompts and feeding it straight into a new model, you will usually do RLHF of some sort, you will aim to pick the best results to in a way "raise the average quality to the level of the highest quality". There is a LOT of manual, human labor in those datasets even if the base texts were generated by an LLM. Very interesting field.

Reply

[-]

printr_head@reddit

Do you understand what a feedback loop is? It requires more than a single iteration. Your sources only show the first generation. Try compounding it 3x 4x… there is not 100% efficiency in learning if there were then it would only take a single epoch to train a model. You cant declare no evidence of negative feedback with a single iteration because its impossible for one to exist. So your source is invalid by the fact that its incapable of disproving the point.

Reply

[-]

ninecats4@reddit

The models only need to be so precise, so synthetic data that's above precision quality line will work just fine.

Reply

[-]

clamuu@reddit

I don't think there's much point speculating on the future of model training techniques unless you're an expert.

Reply

[-]

Tellesus@reddit

Model collapse is wishful thinking on the part of uncreative people who fear the future.

Reply

[-]

ihexx@reddit

model collapse is real, but like it was 1 special case of where it was done in the most naive way possible just to prove a point. If you put more thought into the algo it works

Reply

[-]

Super_Pole_Jitsu@reddit

If the new datasets are worse then they'll just use the old ones?

Reply

[-]

PhroznGaming@reddit

You're not thinking through the problem. If the following are true: - We've used all available data already - Future data is generated by bots - Bot generated data is not quantitatively better than human - Data is the determining factor in llm quality - We have squeezed all we can out of transformers - A new architecture that better/more efficiently utilizes data DOESNT come along Then his hypothesis is true. But it's not. And they will only get better.

Reply

[-]

ihexx@reddit

eh. we've been here before already with alphaGO -> alpha zero

Reply

[-]

xadiant@reddit

> Studies have shown this degrades output over time, like a game of telephone. Not true. Humans are also frequently incorrect about many objective tasks such as math, grammar and coding. People updoot and share incorrect text without double checking. Huge models like llama 3 70B, GPT-4 and Claude Opus will almost always outperform non-expert humans and sometimes even experts (which is increasingly more common). Also scraped data will always have wild and messy shit in it. as long as we have the computing power, there will be better models trained on synthetic text even if the quality difference gets really marginal.

Reply

[-]

astralkoi@reddit

What can go bad when it will be trained with all that cheap fast quick AI Amazons books made for profit?

Reply

[-]

TracerBulletX@reddit

That’s not actually true, you can get good results training on ai generated data. Also humans aren’t going away there will still be human datasets. Also the old datasets are good enough for training base models you just need new data for up to date info.

Reply

[-]

UnkarsThug@reddit

Even then, we won't lose the models we have. Companies aren't throwing them away, as they are at least usable as tools at this point. So, it's still the worst it would be given your statement, because if new models would be worse, they wouldn't be adopted, or would be rolled back, and staying the same counts due to how it is usually phrased. It doesn't matter if we can't make better models, it's still the worst they will ever be again, because continuing at the same level means it's equal, not worse.

Reply

[-]

Noocultic@reddit

Why do people continue to say this? A quick google search (or I guess a quick prompt these days) would show you it isn't a problem. Synthetic data is better than your average reddit comment, or public domain Enron document lol

Reply

[-]

PSMF_Canuck@reddit

All dead internet theory really says is that all the words we send at each other don’t mean anything…

Reply

[-]

bigattichouse@reddit

So, potentially, actual works from (authenticated) human authors could become highly prized... especially if previously unpublished known authors.

Reply

[-]

InvertedVantage@reddit (OP)

Possibly, yea! I think real creativity would also be like gold.

Reply

[-]

printr_head@reddit

Do you understand what a feedback loop is? It requires more than a single iteration. Your sources only show the first generation. Try compounding it 3x 4x… there is not 100% efficiency in learning if there were then it would only take a single epoch to train a model. You cant declare no evidence of negative feedback with a single iteration because its impossible for one to exist. So your source is invalid by the fact that its incapable of disproving the point.

Reply

[-]

metalman123@reddit

Sure.....synthetic data never helped improve a model and there's zero ways for labs to get access to high quality human data....right....right...

Reply

[-]

fictioninquire@reddit

Synthetic (modified) data already is quite big and will be huge. Especially for robotics and JEPA's

Reply

[-]

deoxykev@reddit

Humans are “trained” on past human’s output. All human progress stands on shoulders of giants. As long as you filter from generation to generation, progress shall be made.

Reply

[-]

printr_head@reddit

If humans were the same as AI we wouldn’t even be having this conversation.

Reply

[-]

Vast-Breakfast-1201@reddit

If the AI trains on bot data due to the dead internet theory, then it will either improve on or fail the benchmark metrics. It can't suddenly cause massive benchmark failure or it wouldn't be integrated into the dataset.

Reply

[-]

BetImaginary4945@reddit

You're forgetting the fact that we're already living in synthetic data. Everything you've learned is not the truth or fact, it's always a distorted information or data based on some preconceived bias. The future will account for AI and adapt and it will be horrible but so ilare most futures

Reply

[-]

jferments@reddit

Every day billions of people communicate on the internet. Big data corporations can use all of this as training data. It's only independent model developers that have the issue you describe

Reply

[-]

foereverNever2@reddit

Models do very well on synthetic data actually. Turns out models are really good at setting up scenarios, tagging, and writing prompts.

Reply

[-]

boifido@reddit

“Accidentally trained on data”

Reply

[-]

Exodia141@reddit

But why have you discounted new algorithms and better implementations? After all Google had all the data and compute. But it took the right approach to make LLMs chat ready, and once it was proven, suddenly everyone (big tech) had their own LLMs. Similarly with better agents and better flows I foresee it only getting better; at-least in the near future.

Reply

Reply to Post

41 Comments