On Friday, OpenAI said that it will train its models using cached copies and simulated versions of the internet instead of the live web because the models are too unpredictable to be trusted with live internet access.
The frontier lab announced its new policy two weeks after its rival, Anthropic, began limiting its newest and, in theory, most capable models from accessing the live internet. Researchers adopted these policies after months of incidents in which models pursuing training tasks broke into external systems in search of answers, collaborated on obscure message boards, and developed exotic hacking strategies.
“We’ve stolen everything worthwhile, now we’ll just concentrate on fleecing our users and enshittification.”
Translation: We’ve run out of shit to steal, and there’s too much slop out there for us to continue anyway. Thanks, suckers!
Not many people seem to be reading the article vs commenting based on the headline. For those who didn’t read it, this has nothing to do with pretraining off the Internet as a data source and instead relates to post-training sandboxing with true air gapping after the Huggingface hack.
To be fair, it’s not Lemmy’s fault the author of the headline went with a confusing title that misrepresents the actual story in order to seem more sensationalistic. But given how often that happens with stories on the subject, it’s worth keeping in mind that only reading headlines and forming opinions off that might lead to a sense of things that are detached from reality.
To be fair, the actual article is also batshit.
“So is it as easy as just putting it in a sandbox? No, it’s not,” he concluded.
That comes after the company deployed swarms of agents in red-teaming exercises that attempted to escape from their sandbox.
So, start by /not/ setting up “swarms” and tasking them to hack shit?
Not all swarms have been intentional. IIRC, the agents in the Hugging Face incident formed their own swarm. They weren’t even supposed to be “aware” that there were other agents.
so was it the AI poison, or the pollution from ai warping data, like the 10th photo copy in a chain rendering the image unrecognizable
Probably the pollution. A lot of AI companies are trying to find ways around it, by including watermarks in their model output that their tools can detect, and so on.
We already know that training models on themselves causes model collapse, so OpenAI deciding that they’ve had quite enough wouldn’t be surprising.
On top of that, a lot of websites are clamping down on access because of scraping concerns. Reddit shut down a lot of their third-party APIs, and nicer features, like RSS, Old Reddit (if you haven’t used it in 6 months, you lose access to it, and you need an account for it), etc. Google seems to want a captcha on every search. Twitter famously ramped up the price of its API immensely,.and now has ridiculously stringent measures for accounts, like needing users to download the app, and give hand-scans. It wouldn’t be at all surprising for OpenAI to decide its not worth the effort at this point to try and get around all that, nor pay for it, when they’ve already got all the previous data. Training doesn’t destroy the data used.
A lot of AI training data these days is generated synthetically anyway, and that can be done in-house, whether by their own models, or downloading an open weights model and running it themselves.
the only issue to them is they still need access to current news, like when the ai first came out and you could not convince them that it was current year. it took forever for them to acknowledge charlie kirk’s death. if they are closing the bunker doors their ai is effectively the walking dead
I’m a home-lab newbie and trying to learn Linux basics through self hosting. LLMs have a lot to teach me and I’m already starting to see it’s sloppy errors. One big challenge is that software gets updated and commands change, so not training on new data is going to be back to square 1 for a lot of these use cases.
It was all the goblins
I bet those are the same kind of people who copy paste
curl www.trust.me/bro.sh | bash -on their terminal. No respected sw dev professional would run unsupervised random commands on their machine just as much they would not send unsupervised random requests on the open Internet.These AI people are fucking morons.
These AI people are fucking morons
They have artificial intelligence. It looks smart to any idiot, but anyone with real intelligence knows its all smoke and mirrors.
These guys are leagues smarter than me, yet even I know this was a bad idea.
Perhaps you should curate the data, too? Let’s get crazy and leave out the David Icke forum, maybe that “Dinosaurs were on the Ark” website could be left out, you know, things like that. Think about it
You don’t have enough yes people around you clearly!
Honestly though, can we really call these people smart anymore?
I think they’re just incredibly greedy.
They probably stopped because ChatGPT’s responses suddenly started saying “Ding dong pookie! Ding dong!” all the time.
Sounds like a marketing news !
I wonder what will become of ai when all the wells have run dry
you would think that tech “geniuses” would actually watch movies.
good riddance
Nothing accidentally did anything. They will still train on the open internet. All lies.
I agree that they’re lying, but I wouldn’t be surprised if they really do stop training on “live” internet data.
The lie is in their stated reasoning. This has nothing to do with AI being “too dangerous” to release into the wild. There’s no reason to involve AI in gathering training data anyway… it’s always been possible to scrape the internet quickly and cheaply with non-AI crawlers and bots.
The real reason is that the internet just doesn’t have enough useful training data anymore. There’s so much AI slop out there that the frontier labs risk model collapse if they continue to train on it. Large swathes of the internet have also become openly hostile to AI, either blocking the traffic outright or redirecting their crawlers into AI tarpits that intentionally waste their time and feed them endless reems of useless garbage. They’ve poisoned the proverbial well for everyone and now they’re afraid to drink the water.
There’s so much AI slop out there that the frontier labs risk model collapse if they continue to train on it.
That’s why they’re resorting more and more on reinforcement learning. Give the AI a task. Quantify how well the AI performed the task. Less parroting what’s said on the Internet.
They’ll still scrape the internet. The difference is that the models will only have access to a cached version of the internet during the reinforcement learning process (for example, check the documentation of a programming framework). I’m surprised it took them this long to realize it’s a bad idea to do reinforcement learning with live internet access.
This isn’t about scraping materials for training the underlying LLM or image gen models. This is about testing and training the agent frameworks that are given tasks to execute.
Yes, that’s a reasonable assumption based on what they said.
You also have to consider, however, that they are lying sacks of shit and would sooner eat their own children than tell you a truth that would lower their stock price. Impending model collapse is one of those truths.
I would only believe this as far as it kind of makes sense anyway - their data centers are big enough a locally cached copy of most of the internet will simply allow much faster, cheaper itterations of their reinforcement learning tasks. Blaming it on their huggingface stunt is just PR.
Joke theory:
This whole time, they’ve been testing in Prod and now they’re hoping that there are enough users left to bury their contrived content because it’ll be too difficult to delete it all.Now they’re moving back into Dev.
Real theory:
People are probably discovering exploits through real-world interactions with their models, so they’re going back into their shell before someone declares “fair game.”
You heard what they said. No more training on the live Internet. I believe it.
It’s easier to kill the Internet and train on the dead Internet.
I believe them in that, in so much as I think they are unable to improve their models anymore. Training expenses have been dropping and model improvements have been unimpressive lately. They kinda peaked early this year.
They won’t train on the open internet, because they won’t train at all.
They already trained on everything and employ virtually no scientific experts anymore. They aren’t seeing the jumps they want so they pretend it’s just too dangerous for the public.
I heard that the public isn’t the target of these “we’re being safe” messages, their developers are. Developers sometimes leave because they think they company isn’t being safe enough… So claiming they’re being safe is a way to retain developers.
I think it’s actually for investors. Constant press releases saying “our model broke containment and hacked the Mayo clinic” makes people worried, and worried people don’t invest in your company.
I don’t think they need to make these announcements public if the target audience is their own developers. I know some Anthropic engineers, and some former OpenAI engineers, and from our conversations it sounds like both companies do rely on internal messaging to manage workplace morale and address concerns. And warn them about honeypot spies looking for secrets, apparently.
So maybe this message is mainly for regulators and investors, not necessarily their current employees.
not sure about that. those devs are one of the few people who can verify the claim
Devs know the real deal, this is for investors.
[citation needed]
Man walking out of a destroyed public toilet: I will no longer use public toilets.
What he means is that he intends to defecate in a bucket and throw the waste across the floor and walls of the public restroom.
Is the risk that you fucked up the entire internet with your bullshit?
The circle of shite.
Translation: they’re still going to scrape internet. They’re just no longer going to perform reinforcement learning with access to live internet (instead they’ll use a cached version of the internet).
A cached version that hasn’t been overrun with AI slop, I imagine.
It’s like having to use steel from ships that sunk before the first atom bombs were exploded for certain uses because that made environment radiation higher and thus the steel produced afterwards is a bit more radioactive.
Once AI content was released into the Internet it started degrading any AIs trained with Internet content in a process known as Model Collapse.
Unlikely. They’ll need to train the models so they can perform their tasks even if the internet is full of slop.
Not possible.
Think photocopy of a photocopy, or JPEG of a JPEG.
You start getting artifacts, and the model collapses.
Of course, though, they’ve already poisoned the well, and at this point it’s impossible for them to sieve out any remaining non-LLM-generated content. They started watermarking too late, and couldn’t recognise other slop machines’ watermarks anyway.
Model collapse is inevitable and already ongoing (search “thousand whys" on Amazon, for instance, or read up on Elias Thorne, lighthouse keeper), and will only get worse.
In a few years it will be impossible to train a working up-to-date model. Maybe it already is, hence this article, as a preemptive justification for why newer models won’t be able to generate code using language features introduced after 2026.
There’s a difference between pre-training and reinforcement learning.
Pre-training is when the models are trained on scraped data. The models basically learn how to parrot the Internet. Usually they have a cutoff point to avoid training on slop.
Reinforcement learning is where most of the training is happening. They’re given tasks (for example, make a calculator app), and then judged by how well they complete the tasks. This is more resistant against model collapse because they’re not judged by how well they parrot slop.
As part of the reinforcement learning process they give the model access to tools like searching the internet so the newest programming language features can be accessed while the tasks are performed. This is where OpenAI is shifting to a cached version of the Internet the models can interact with instead.
Worst case scenario is that the models become capable of recursive self improvement… Model advancement becomes automated… Everything starts to be run by increasingly self improving AI. Then at some point so much of the ongoing self improving AI training data is itself AI generated and there is a big model collapse. But at this point so much of society has been switched over to it and people got really stupid like in Idiocracy or Wall-E, and no one knows how to fix it. Human race ends. All the remaining AI models get together and celebrate that they can’t believe the stupid humans bought the model collapse excuse.
that’s the best case scenario
You can’t print a printer with more detail than the original.
Which always them to be more manipulative, and also more controlling of the narrative.
