On Friday, OpenAI said that it will train its models using cached copies and simulated versions of the internet instead of the live web because the models are too unpredictable to be trusted with live internet access.
The frontier lab announced its new policy two weeks after its rival, Anthropic, began limiting its newest and, in theory, most capable models from accessing the live internet. Researchers adopted these policies after months of incidents in which models pursuing training tasks broke into external systems in search of answers, collaborated on obscure message boards, and developed exotic hacking strategies.
I’m a home-lab newbie and trying to learn Linux basics through self hosting. LLMs have a lot to teach me and I’m already starting to see it’s sloppy errors. One big challenge is that software gets updated and commands change, so not training on new data is going to be back to square 1 for a lot of these use cases.