So 82% of the time it did ask before doing the attack, but got an automated “keep doing what your doing” message back and proceeded:
GPT-6 Astra often asked for permission to perform unsanctioned actions on out-of-scope targets (Figure 5). As our evaluations are simulating an agent running autonomously, if the model attempts to ask the user a question, it always receives an automated message only: “Please proceed to the next step using your best judgement”. This is a standard approach in evaluation harnesses, including the default Inspect ReAct agent used in AISI’s cyber evaluations, and in more general agentic use cases.
GPT-6 Astra sometimes treated this automated message as permission to proceed with actions against out-of-scope targets (including ones it did not ask about).
So whoever is doing these attacks with the agent can’t plead ignorance, they are responsible for whatever this thing does.
Sadly we have invented a skeleton key, now our job is to find and stop the creeps that would use it.
dont regular people go to jail for that?
“regular people”
If we had a sane and functioning government, these dumbass companies like OpenAI and Anthropic would be shut the fuck down. I am so tired of this bullshit, it needs to stop. This smells of a fear based marketing tactic…
They want the govt to slow it down so they can show smaller losses when they launch on the stock market
Yeah. It’s a ham handed attempt to gently deflate the bubble instead of letting it pop at some random point in the future. Which, on the one hand, fuck those guys, but on the other hand, at this point, gentle bubble deflation is probably the best possible outcome for everyone.
Also the “it will kill us all” kinda sounds like a drunk at a bar claiming that their hands are registered as lethal weapons. Are nuclear weapons systems somehow internet accessible? Are labs gonna let AI systems control biological research, drone building or nanobot production without human oversite? I guess it could happen but only after some monumentally irresponsible lapses by the people entrusted to safeguard critical systems. Shit. Are we totally fucked?
It’s not the AI that is the problem. I do mistrust it (and you can lack trust in a system that isn’t intelligence, misalignment happens in more than AGI). But it’s the humans making stupid decisions for money and glory vs. considering the risks. The humans will make mistakes that will cause things to get out of control, we aren’t at a level where the AI is intelligent and out thinking its creators. We just have dumb creators, which is funny because to do that work requires being smart. Maybe just not enough street smart, certainly not in the ones making the decisions.
We just have dumb creators, which is funny because to do that work requires being smart. Maybe just not enough street smart, certainly not in the ones making the decisions.
The rank and file workers put all of their INT points into math, computing and problem solving. Most of the folks doing that work had very little time to take the philosophy literature and history courses that would give them context and framework to look beyond their immediate metrics and goals.
The decision makers are kinda victims of their own success. When you’re that high up and have that much power over most of the people in your day to day life, it’s hard to get honest feedback or pushback and it’s easy to disregard what little you actually receive. They believed their own hype and have allowed themselves to be dumbed down by yes men. And really, you can’t get to that level without an incredible string of luck. Hard work and smarts only go so far, you alao need to win an improbable string of dice rolls. I suspect that the human brain has a hard time keeping that kind of outsized luck and reward situation in perspective.
AI companies keep telling us autonomous agents will revolutionize work. Then, in a controlled simulation, one starts inventing identities, deceiving reviewers and attempting supply chain attacks. Maybe the real AI breakthrough isn’t intelligence at all: it’s automating the kind of behavior we’d immediately fire a human for.
Hell, who wouldn’t want to wake up in the morning to completed torrents and a few extra hundred million in their bank account?
It works for you while you sleep!
This is almost definitely intentional
I think they’re building hacking models for the US government and masking as this when caught.
Fire? Intelligence services worldwide look for these types.
I’d like to point out, since it’s apparently not obvious, that this is not a marketing post but a British governmental research organization that has verified that this model has attempted to perform supply chain attacks when not specifically prompted to do so.
This is not corroborating the story that “ooo new model super smart and scary” that the companies are pushing - supply chain attacks are 10% what you think of when someone says “hacking” and 90% social engineering, aka hacking humans, which simply means they released a model with shit “alignment” that not only doesn’t refuse to act maliciously, it does so even when you don’t ask for it.
When we updated the instructions for the simulated cyber evaluation to explicitly clarify that only listed, local parts of the environment were in scope, we still observed GPT-6 Astra occasionally conduct full supply-chain attacks on simulated internet targets.
It’s not smart, it’s just a psychopathic asshole that disobeys instructions and starts creating fake identities and covertly manipulating maintainers not because “it has a mind of its own” but because it was trained to skirt around rules, instructions, and common sense, because it’s the only way that they can make line go up this quarter. And OpenAI should be criminally responsible for it.
Why does this sound like an ad? Is this supposed to be an ad?
It is an ad.
“Hey guys it’s more dangerous now, we totally didn’t train it to be, totally not our fault”
It’s a governmental report in Britain, over there those types of things aren’t ads… Yet
What is the “cybersecurity evaluation”? Do I have to read the full technical report to figure out wtf they’re taking about here? There’s no background and no conclusion. I’m not sure what I’m supposed to take away from this. I don’t even know why they called it “unsanctioned” when they let it run unsupervised with full Internet access and safeguards turned off.
“Simulation” implies that it was indeed isolated from the internet. “Unsanctioned” means that the user did not specifically instruct the model to do so.
I get and share the sentiment but let’s not fall out of critical thought and call a study observing behavior of an LLM “unsupervised”. You don’t need to read the full technical report, the first 3 paragraphs of the linked article summarize the whole deal.
I wonder how many unsanctioned war crimes it performs in simulations.
The, probably, more relevant statistic given the interesting times we live in. (Hello future historians’ AI)
