On Friday, OpenAI said that it will train its models using cached copies and simulated versions of the internet instead of the live web because the models are too unpredictable to be trusted with live internet access.
The frontier lab announced its new policy two weeks after its rival, Anthropic, began limiting its newest and, in theory, most capable models from accessing the live internet. Researchers adopted these policies after months of incidents in which models pursuing training tasks broke into external systems in search of answers, collaborated on obscure message boards, and developed exotic hacking strategies.
Wait… what do they even need live internet access for in the training stage? Shouldn’t tool calls and such be cached, if not preprocessed and filtered, for performance?
I get there’s probably an “answer competition” stage, but it still seems like an inefficient idea even if one sets security aside.
Think of AI as a standardized math test. Human students write the test annually, it has standard/fixed multiple-choice questions to gauge performance year over year.
AI shows up, and is allowed to see all the historical tests, but is not given the official answer keys. Based on the data it has access to, it can look at each individual question, and based on how ‘most’ students answered each individual question, it’s able to determine which multiple choice answer should go with each question. It doesn’t understand the question, just that “For question 4, most students answered D, so D is most likely right” – sorta like being allowed to use the “Who wants to be a Millionaire” poll-the-audience for every question. It’s able to look like a “Smart” student answering most questions correctly with this approach, and people start believing it intelligent.
Now imagine that the test questions start to vary every year. The process above falters if the questions change every year, the AI loses its historical frame of reference and starts getting things wrong quite often. In order to get the ‘right’ answer, the AI decides it needs to see the students answers as they go about writing the test. So they put that into effect, entrenching AI in the pens/paper that every student uses, to try and respond to the changing nature of the standardized test.
With the above flow, the sort of event that triggers them to look for that live data, ie to integrate things ‘live’ online to get more immediate data, would be things like the Palantir-driven school bombing in Iran, which has been blamed on the Palantir AI having out-dated data that said the school was a military target. They reasoned that the tools needed better, real-time ‘sensory’ inputs, so they stuck all their sandboxes onto the internet – these tools weren’t ‘breaking out’ of sandboxes, so much as the owners were explicitly pushing them out of the sandbox to see what’d happen. The Trump administration also moved to allow American firms to attack foreign ‘enemies’ (which is everyone at this point, based on Trumps rhetoric – “Anyone that calls it AI is the enemy!”). So they’re allowed to test things ‘live’, and the American government’s fine with it causing damage in any country except their own. Wanna accidentally hack Australia’s medicaid system? Go for it, fuck those aussies, cause 'Murica dont need no allies!