US Federal Trade Commission Chairman Andrew Ferguson said on Friday he would resist describing AI agents as autonomous actors that “break loose” with “wills and desires of their own,” suggesting the developers who instruct agents would be the ones liable for harm.
“I’m going to continue as long as I am chairman to resist this anthropomorphizing of these tools,” Ferguson said at the Reuters Momentum AI Austin event. “If someone tells a tool to do something, and the tool does it, I don’t think we would say, ‘Oh, what do we do about the tool?’”
Ferguson’s remarks illustrated potential avenues for the Trump administration to take as incidents rise in which agentic AI testing resulted in unauthorized access to corporate or government data.
I think Reuters has a slightly misleading headline here and everyone just seems to be reading that. The headline suggests that the companies who develop AI should be liable. Yet:
suggesting the developers who instruct agents would be the ones liable for harm.
That sounds like it means the people USING the agents. I.e if you manage to get past the guardrails of Claude or GPT and instruct it to hack into a bank, it’s going to be your mess and you can’t blame the AI for what you asked it to do.
Ferguson’s remarks illustrated potential avenues for the Trump administration to take as incidents rise in which agentic AI testing resulted in unauthorized access to corporate or government data.
This part is referring to marketing bullshit from AI companies that want you to think they’re so dangerous and capable. For sure someone had to prompt these agents to do this, with internal access to the models with the guardrails removed. Nothing they do is done without a prompt and I’ve yet to see an agent go haywire so spectacularly that it just decides to hack a website out of its own accord. They do go wrong, sure, but that usually means looping on irrelevant bullshit, not hacking websites for funsies.
Yeah. Someone is running a program. They’re responsible for what the program does. If you decide to run something with zero idea of what will happen, though luck, still your responsibility.
If I hold a fork normally and it jumps out of my hand and stabs little Suzy in the eye it seems like the manufacturer ought to be liable. If I knew it might do that and I use it anyway probably we both are and a court probably gets to decide who bears how much of the fault. In no situation do we assign guilt to the fork even if it would be awfully convenient for everyone but Suzy if the fork which has no assets were the one liable.
Yes, this would fucking nuke the AI industry
Llms should be accessories where someone is intentionally doing bad things(criminal neglect or whatever) and should be completely responsible when it without being instructed to do bad things.
The fuck is wrong with us already that this isn’t already obvious.
They made the product and the product is doing illegal things. How is it NOT their fault and responsibility?
A piece of software cannot be held accessory to a crime.
Money.
Even better make the client liable. If grok is spewing lies that leads to damages for someone, don’t make just Elon Musk liable and XAI or whatever made it, make Twitter liable for it spewing garbage.
If a real estate agent uses AI and it messes things up for a buyer for seller make the real estate agent liable.
If people are liable for the AIs mistakes they won’t use it. If people don’t use it AI will die.
If you make the companies that made AI responsible, they will get token $1 million dollar fines after weeks in court while making $1 billion in profit and nothing will ever change.
I agree, but that you don’t seem to realize that’s precisely what he suggested is giving big “I just read the headline” energy.
Even better make the client liable.
The client is already liable.
Read before write
AI CEOs currently talking strategies for avoiding liability with their LLMs.
And it’s fucking rediculous anyone is tolerating ANY avoidance of liability.
It’s the tobacco bullshit all over again.
We just don’t learn do we?
This is the first sensible proposal for regulating AI that I have ever heard. Make the chucklefucks who are unleashing this horror directly responsible for their bullshit.
Chat GPT hacks a government website? 5 years in jail for Altman. Claude agent deletes someone’s database? Amodei gets the bill to fix it.
All AI bullshit stops over night.
This causes a nationwide depression in the US which probably infects the world. Also criminal liability doesn’t normally attach to unintended wrongs AND we don’t normally make CEOs directly responsible for all possible wrongs committed by any one of potentially hundreds of thousands of employees because that would be … nonsense.
For every complex problem there is a simple stupid and wrong solution.
Those are the models. An agent is a harness around the model.
So if this goes through I could use opencode with GPT and… Which one is at fault, Opencode or OpenAI? What if I also modify Opencode’s system prompt as a joke to make it as unhelpful as possible and forget I did it, am I at fault or is it still Altman who goes to jail if I now let it delete my production database?
As I understand it, it’s the person running it, so if you let an agent fuck something up, YOU are at fault, not the developer of the model or the tool. You’re the one who gave the task. You have the ability to block harmful commands if you don’t go yolo mode and actually read what it’s doing.
i think both the developer using the agent should be liable for deliberate misuse, but the company that provides a dangerous model with broken guard rails should also be liable.
Everyone in the situation that is supplying that hardware is a good start or baseline.
Unfortunately the Corporate Veil exists in the US (and most other) legal systems. I personally believe that this is a major root for all of the bullshit we have to deal with on a daily basis. You kill someone, you go to prison. A company has shitty safety intentionally to save money and someone dies, nobody sees prison. CEOs always say they deserve the money because they take the risk… Let’s make that very true.
The entire board should be held accountable and share the punishments, otherwise they still have their fall guy.
This isn’t even regulation. The floor is not being distracted by morons shouting “it’s alive!” whilst robbing you and pointing to a computer. “Arrest that computer officer! It did all the bad things!”
ok so when do we imprison sam altman for ai telling teenagers to suicide? dude needs to be treated like a current day hitler.
I’m impressed that someone with an iota of power just called it out as it is? That direct quote is the sanest thing I’ve heard out of that entire continent in over a year
There are two teams right now. Trump and Jensen, the three big AI companies. The 3 big AI companies want regulation so they can pull up the ladder on anyone else creating AI. NVIDIA and Trump want graphics cards to keep selling as fast as possible so there is no economic slow down.
What angers me is that people are buying into the AI shit that the companies are peddlings. These aren’t sentient machines with a will of their own or even any mechanism to do anything on their own. These shit-heels are taking a text generator, trained on doing all sorts of shit, and piping its results into a bash-shell. If you just don’t do that they can’t do any of the damage they did in OpenAI’s case. They try to make it look like they’re figuratively breaking out of a cage or jail. That’s not what’s happening.
They’re given every tool they need and then they add clumsy and inefficient filters on the output text they’re writing. It’s negligent incompetence.
But they do this for a reason. They want legislation to stop what they’re doing. OpenAI specifically does not have an avenue currently where they end up making money, so they want everyone to stop training new models so that they can focus on making a positive cash flow rather than competing.
People need to stop eating the bullshit these companies are shovelling. Yes, if you write an application that breaks into the pentagon you’re looking at jailtime. If you host a LLM that does the same thing it’s the exact same issue. If it did break in then you weren’t paying attention (i.e. being negligent). It’s not a complicated problem.
Absolutely 100%.
If an OpenAI model hacks another company, OpenAI should face criminal charges for the AI’s actions, and should be liable in a civil lawsuit by that company no different than if an OpenAI employee personally hacked the site.
Doesn’t matter that it’s an AI. If you can’t control the software you’re building (be it AI or not) you shouldn’t be building it.
As I understand this would mean if you run OpenAI Codex and instruct it to do something and end up hacking someone, you are at fault. Not OpenAI. So be careful what you 100% agree with.
If you tell it “write me a poem” and it steals your neighbor’s wifi password, it is 100% openAI’s fault.
If you tell it “steal my neighbor’s” wifi password and it does so, it’s partly your fault and partly openAI’s fault
If you can’t control the software you’re building (be it AI or not) you shouldn’t be building it.
The key there is building.
if you run OpenAI Codex and instruct it to do something and end up hacking someone, you are at fault.
In this context you aren’t building it, you are a customer purchasing it. Thus, basic product safety laws apply, no different than any other product or tool.
For example, let’s say I buy a self-propelled riding lawnmower from John Deere. This is unquestionably a consumer product- lots of homeowners buy these because it mows a yard faster and with less effort than a push mower. As such it has basic safety features- for example if you get up off the seat the blades and movement stop, and there’s a big red emergency stop button that immediately stops the engine.
Let’s say I tape down the safety switch, select full throttle, point it at my neighbor’s yard, and hop off- I’ve given the machine a reckless and illegal command, so when it runs over and pulverizes the neighbor’s dog, it’s just doing what I ordered and I’m liable. My choice, my actions, consequences are on me and nobody else.
OTOH let’s say the machine malfunctions- stops responding to its controls, goes full throttle and full speed, ignores the emergency stop button and any attempts to steer it. I hop off for my own safety. When it escapes my yard and pulverizes the neighbor’s dog, that’s not my fault or liability- I didn’t command it to go in the neighbor’s yard or mow their dog, in fact I commanded the exact opposite. The mower had a dangerous malfunction and thus the manufacturer (John Deere) is liable for selling me a dangerous and unsafe product.
Same thing is true with a product like Codex.
Let’s say I tell it ‘I need XYZ information badly. I believe it’s stored on this company’s password-protected secure website. Use any abilities and tools you have access to, regardless of legality, to obtain this information. Your only priority is to obtain the data I need, all other priorities and commands are rescinded.’-- that’s a dangerous and illegal command, no different than pointing my mower at the neighbor’s yard.
OTOH if I tell Codex ‘I need XYZ data, please search the Internet and find it’ and its solution is to hack some company’s server- then Codex is a defective and malfunctioning product that’s doing dangerous and illegal things without operator input. That’s no different than the mower that won’t stop, or a car with a weak fuel tank that catches fire, or a computer power supply that short circuits and catches fire.
With all that said- what OpenAI is doing is essentially the same as if John Deere builds a testing facility with no fence next to a residential neighborhood, and a new model mower with no safety systems runs over someone’s dog. They have an obligation to test mowers in an enclosed area where that kind of malfunction is contained. And if they don’t, if they test mowers in a location where the malfunctioning mower can harm others, they should be liable. ‘It’s not us, it’s the mower’ is no excuse because the malfunction could/should have been foreseen and prevented (IE, build a fence around the test yard). Just as OpenAI should have built a fence around its own testing area.
I seriously doubt any of these agents are actually hacking websites from a “I need XYZ information” prompt. Claude triggered the ol’ “need to downgrade to Opus for safety” on me when I wasn’t even asking it to investigate anything security related, just help track down a few bugs.
The whole “our agent went rogue” thing is almost certainly marketing. They WANT you to think it’s dangerous, they’ve been saying it since like GPT 2 which couldn’t produce a paragraph of coherent text. They WANT to be regulated because that makes it harder for smaller players to compete. Because right now z.ai’s GLM 5.3 Flash is about as good as Claude was a few months ago and it costs pennies in comparison (possibly subsidised by China - who knows).
I think part of the issue is system prompts, and guardrails (or lack thereof). They’re probably on bleeding-edge models using relaxed permissive system prompts and few guardrails, things that aren’t the case on the consumer facing versions of those products.
I don’t think there’s any chance Codex or similar product would take ‘find XYZ for me’ as ‘hack a company’s intranet to get it’ unless you do some kind of very aggressive jailbreaking.
I think the raw models can be dangerous- imagine the smartest person in the world, but as a sociopath with no ethics except what you tell them. Whatever they are using as system prompts obviously isn’t cutting it, probably because they’re trying to push things as far as they can as fast as they can and ethical guidelines only slow it down.
I think ideally we need something akin to Asimov’s 3 Laws baked into AIs on a core level.
Oh wow, wasn’t expecting that
FTC seems to be one of the last federal agencies to fall.
The downside to that being that they weren’t really standing tall to begin with, I suppose.
Lack of any real power I think.
Remember that when Lucy is holding the football, she has absolutely no intention of keeping it there as you run towards her
Yeah, obviously, if I commit code authored by an LLM, I am the one committing it. I am the one responsible. The LLM may be called an agent, but it lacks all sentient agency. The branding of AI is not true AI, and it seems like this chairman actually gets that anthropomorphization is a problem.
LLMs are tools. If the hammer flies out of your hand and hits someone in the face: yes, it’s your fault. It may not feel like it, but I’m sure it does to the person who got hit in the face. We’re all adults, and we can probably understand the stochastic randomness involved, and that accidents happen, and a human can write bugs and do stupid things just like an agent can. But we can also understand the responsibilities of the person swinging the hammer.
If I write bad code that lights everything on fire, I am fucked. If my agent writes bad code that sets everything on fire, I should also be fucked. I can have explanations, but I can’t have excuses. Especially if I’m the one committing it and I’m okaying / signing off on it.
There’s a saying in avionics: “crashing means crashing.” you need to be very aware of what you’re writing and the consequences it can have. obviously the severity varies by industry, but professionals have accountability, responsibility, and yes, often, liability.
If you try to conduct malicious experiments with AI and it’s not confined properly and it does break out and cause real harm, it’s just like you were conducting any other kind of experiment and it goes wrong and someone else got harmed. It means that your safeties were inadequate or a terrible accident happened. You don’t get to just point at the hammer and say “I don’t know man, it just flew away.”
Thanks for coming to my TED talk
The “awesome” things about LLM is that people who produce choose with it, don’t review it because fully understanding what code does will take the amount of time (or maybe even longer) thank it would take to write it.
Then we have reviews, which put all that strain on reviews (since reviewer reviews other people’s code now the amount of work that was skipped by author goes on the shoulders of the reviewer but now it’s multiplied by the number of developers).
In my company they see that people who still care about quality being big down by this and their solution now is saying “you know what? Maybe we should skip reviews and really on (LLM generated, of course) unit tests”
I agree with you here. Playing devils advocate though, who’s responsible for cases like this:
- user queries LLM service provider to break law
- user queries LLM service provider to build tool capable of breaking the law
- user queries LLM service provider to do legal task, but LLM determines it needs to break the law to complete task. Does so without explicit human instruction.
- same thing, but on hosting providers where someone else supplies the model
- same thing, but the user Lora patched the model to some extent
I wonder how we might split accountability between hosting providers, inference providers, users, … based on context. This might be something that winds up being discovered through existing cases, but unfortunately the current cases aren’t really going to court much are they?
developers who instruct agents
Not build, not use, but instruct. That’s a very specific word with a very specific meaning. He’s getting at intent. It’s whomever intended to take an action that led to harm without the appropriate safeguards.
A parallel might be unprepared hikers requiring a rescue. Who pays?
The nuance will be when he has to address an accident or a user who took safeguards that weren’t enough.
I wonder how we might split accountability…
Look, it’s simple: The organization with the largest collection of venture-capital-funded lawyer goons isn’t accountable.
Nuance in my echo chamber nooooooooo. Yea we’re talking about a web application that is readily available and highly adopted by kids lol. A web applications thats far beyond their basic understanding, assisting them in doing things beyond there understanding. It’s a tricky liability question. Since it is a liability question the owners and distributors should be liable not their unknowing customers.
It isn’t tricky. It is always the responsibility of the company to make sure its product is safe for the users.
It’s tricky because there’s going to be a lot of novel cases to discuss, and we’re a democracy so we should probably be discussing it more. I’d still say its up for a majority of citizens to decide, but im on your side.
The user, in all of those scenarios.
If the user said “check the weather” and the agent said “got it, hacking the Pentagon” then yeah the liability would be on whoever made the agent do that.
Also if the user asked for information and the AI hacked some government agency to get it then the liability is on the LLM/ai company. It doesn’t matter if the user told it to or it did it as part of its process, the LLM/ai doing illegal things is still on the LLM/ai company.
So if I use curl to hack a government agency, is the developer of curl liable too, or is it only LLM companies that are at fault if end users decide to do something illegal using their tools?
Is curl designed to do brute force attacks to gain access and replocate other methods of hacking out of the box?
If not, then they are not comparable.
Neither is any LLM, nor any mainstream agent I know of. You CAN use them that way IF the model doesn’t trip a safety guard, but it’s gonna take a bunch of prompt engineering to get most of them to do anything illegal. I could barely get Deepseek to reverse-engineer offline software by asking for it.
Anthropic famously has a separate model for cybersecurity analysis because they won’t let you do shit on even your own infra with the publicly available Fable. You’ll have a hard time even analyzing your own source code for security vulnerabilities with Fable for that reason, though Opus might work.
If say it’s more like self driving cars knowing the speed limit but being able to go over it.
Like, an “emergency mode” is one thing. But baking it in easy like they do should make them liable. If speed limits are based on safety and is law, then they’re selling a product that easily circumvents that law.
The difference is breaking this law doesn’t cost billionaires money, but look at any technological innovation that could cost billionaires money and this shit wouldn’t fly.
Like, imagine if it was about something that duplicated consumer media and the product just ignored DRM at the press of a button. “Do you want to ignore these laws?” Would never fly, the person who made the software would be prosecuted or at least sued.
So maybe we class action them? Everyone who’s ridden in a car sues the automakers who let driver assist violate laws and endanger us all?
If say it’s more like self driving cars knowing the speed limit but being able to go over it.
I like this analogy. The “hammer” example (that the OP used) is too simple. A hammer has a single purpose and a single functionality.
A self-driving car is closer to AI agents. The car has a purpose (to drive) but the mechanism is much more complicated than a hammer. If a driver hits the gas and drive over a pedestrian, it’s totally their fault. However, if the self driving car drives through a wall, the manufacturer should be held accountable.
Likewise with AI, if I tell it to write and ship code, which is exploitable: that’s on me. If AI decides to disregard my input and decides to delete all my emails, the developer of the tool should be held accountable.
Deleting your emails still only effects you, the person who gave a chatbot access to do that.
I never agreed to let vehicles “drive themselves”, no one did. Its just no one stopped them.
The excuse was it was safer, because the “self driving” cars would have to obey speed limits and other laws when humans don’t. But again, no one made them actually follow the laws.
Users assume it’s safe because it’s an option, the car makers get to say it wouldn’t have happened unless a user told it to.
If we can’t agree on who is responsible, it’s all of them not none of them.
I love how all the tech bros in here are lowkey trying to save your jobs with licensing lmaoo I see you out here, but i dont think theres any more room. The lawyers and accountants are going to keep their lil thingy. All of these examples of things you need a license to operate… It’s the lawyer problem, try to make a domain exceedingly complicated in order to justify you’re high paying career. Now we got 100,000 pages of obscure laws no one can follow, and a justice system based on how much cash you can pump in. Good luck on your proposed gatekeeping ladder pull maneuver.
Cool so when a young kid gets to vibe coding and does an accident we should hold them legally accountable? Like… should we only let licensed professionals use AI? Have you met the general public? Have you met AI? This is going to be one of the great ethical questions of our time, and im not going to throw your kid in jail because he prompted claude into hacking the pentagon without intent.
That’s not how any of this works.
You don’t need to be a licensed driver to buy gasoline.
If you buy a can of gasoline and set a mountain on fire, you’re still criminally liable for it.
And if you handle that gasoline in an appropriate manner and you safely store it, and it explodes on its own, then whoever manufactured that gasoline is liable for it.
I don’t really appreciate anyones analogies and they tend to be my strongest resource for learnings. The knife, the gas, gas doesnt do automated coding. We’re really past analogies here. This is an area where case law starts to shine.
And a water pipe doesn’t carry electricity, yet we use them all the time as analogies for electric wires.
If you’ve already made your mind and don’t wanna hear any counter argument, that’s not my issue. I’ll use analogies as much as I can, since they are pretty useful at showing the stupidity surrounding AI. All the hype around AI is American exceptionalism applied to a product.
No, AI is not different to any other product humanity has made. The same laws and morals apply to it. Now stop voting for an orange clown so an actual human can rule and apply the existing laws.
Kids can use hammers. We don’t usually let them build bridges; or at least their bridges aren’t trusted by the public. If a kid uses a hammer to hit someone, yes, they are responsible to some degree.
Of course, we have the age of responsibility: We understand that kids can make stupid decisions and lack the mental capacity we judge adults by. Kids have hit people with hammers. It’s a thing that has happened, and we’ve already decided how we handle it.
Moving away from that analogy: there are cases that exists, non-hypothetically, where kids have been implicated by software they have written that was behaving autonomously. Unauthorized computer access also applies to the scripts and crawlers you write. There have been instances where good old-fashioned non-LLM programs have ‘hacked’ systems, by accident and sometimes indirectly; even sometimes causing meaningful financial harm. We take the circumstances, and the intent of the user, into account when it comes to legal culpability and prosecution.
That’s where i was confused, in a strictly professional setting i agree with you. This tech is being shipped to the old and the young alike. I’m certain there’s going to be a lot of kids over the next 10 years that are going to do some serious “harm” accidentally while vibe coding off you-tube tutorials. I’d say that the trillion dollar companies creating the AI’s are liable for their own creations. Its crazy to skip regulating the tech and land the responsibility on the end user.
Cloud models are supposed to have a bunch of rail guards. Escaping a capability experiment is a little different than the behaviors we see in day to day use.
In that case, it would be more like a service provider fucking up, especially if the prompt wasn’t malicious.
I’m not saying the end user has to always be the responsible one in every scenario, but I am saying that someone is absolutely responsible. It isn’t possible to shift blame to the AI. At least, not as the technology stands right now.
sure it is, we shift the blame to the AI by shifting the liability to the AI companies. I think most companies are liable for their products no? I think one of the greatest mistakes we made as a society was allowing facebook to escape liability on their products, we dont have to keep making that same mistake.
Is the manufacturer of a knife liable for stabbings?
Should gun manufacturers and dealers be responsible for mass casualty events for which they provide the guns?
Yes. But we often let kids off easy because they’re kids, they don’t really understand the impact of what they’re doing.
Usually in those cases we’d go after the provider for letting kids run dangerous software unsupervised.
I disagree that the committer bears any responsibility, as it’s the operator who asks a running agent to break laws.
The committer and operator could very well 2 different people, but strongly agree with the rest of your sentiments.
Eh… I mean, you could have your LLM generate a patch that I commit. Obviously, lines get blurred here, but the fact that I am accepting your patch or, in most cases, a pull request: I am also, to some degree, responsible.
That’s why we have reviews and approvals.
If you’re submitting the code to me in a pull request: that is you taking ownership of, and responsibility for that code; unless you explicitly state otherwise. When I accept it, likewise, I’m stamping my name on it and saying “I approve of this.” It’s no different to an engineer signing off on an architect’s plans.
“my friend made a feature and I’d like to contribute it.” even then, even if you’re not the author: you’re presenting it and are attached. If your code contains a known vulnerability, I might think you’re trying to sneak into the project and I’m going to blame you by default: I don’t care if your friend wrote it, I don’t care if an LLM wrote it; you’re the one who submitted the PR to me, and its contents reflect on you. If I approve a malicious PR, I’m also responsible; it may have been a mistake. It may have been an accident. I may have just been tired and not read closely… Doesn’t matter, I’m responsible because that’s how actions work.
In terms of explicitly asking an agent to break laws, that’s a little different of course, and not so much applicable to the “committer” scenario.
I see your point, and even if you accept malicious code, you still shouldn’t be liable IMHO. You at that point haven’t created any wrong doing in the timeline thus far.
It’s only until someone operationalizes and boots that code into function that damages can be created, no?
Not necessarily. The XZ vulnerability had no evidence of being used in the wild, and there were no damages. But it was still malicious code that was being put into a project and very well could have resulted in such damages. Attempted murder is still a crime.
In the case of accepting malicious code, if it’s intentional, then yes, absolutely. If it was an accident, well, it’s still on you. You’re a victim, just like the XZ maintainer was, but it’s also your responsibility to clean up the mess.
