"The law in America relies on intentionality, notes Rune Kvist, head of Artificial Intelligence Underwriting Company, which insures AI firms. If no human intended to hack anyone, no crime can have happened. The ability to sue for damages is limited too."
This is sloppy. Criminal negligence exists (although some crimes do require intent). And civil tort certainly doesn't require intent.
I don't get it. Why is breaking the law so hard to enforce when it is a company (i.e a person or group of people consenting to) running a computer program? If I take a gun and spray bullets around me I don't get to write it off as the gun being dangerous.
But if you told an embodied AI to do a home cleaning task, and it decided to pick up a gun and start spraying bullets, you might not want to be held liable for that.
The best is that the article frames it as "Autonomous hacking is here. Governments are not ready". As if someone didn't authorizing turning the damn thing on.
> TO LOSE CONTROL of one artificial intelligence may be regarded as misfortune. To lose two looks like carelessness. Lose four, and people may start to wonder whether the problem lies with AI itself.
No fingers pointed at the company, just the "AI". Funny how that works.
it makes you wonder did the developers intentionally add vulnerability exploitation as a feature so they can use it against their competitors or anyone else
> The hacks also present a challenge for legal systems. Hacking, when humans do it, is a crime. When an ai is the wrongdoer, though, it is unclear how to assign blame.
Blame the prompter or person who assigned the task to the AI. It's their responsibility to use the tool in a safe way, just like it's a gun owner's duty not to fire their weapon carelessly into the air.
How are you going to figure that out? You can’t capture an agent in a jar and convince it to confess. As we saw with the HF incident, even highly sophisticated actors need a good chunk of time and manpower to trace these things. And I suspect the folks who are going to have the most success with LLM-powered cybercrime are going to know how to cover their tracks reasonably well.
Difference there is you need a physical body firing the ghost gun. There can be witnesses who saw a shooter, you can look at cameras to track their physical location, and if they discard the weapon they leave fingerprints.
Sort of like how we treat pitbulls perhaps? It’s not the itty bitty GPT-5.6 Astra that did the hacking. It’s the owner who didn’t train it well! Poor model so sad, we should give it a nice open space and all the GPUs it wants to do what it wants.
Autonomous hacking is an oxymoronic phrase. Someone is running the algorithm & keeping tabs on it b/c hacking is an activity w/ an intention to gain access to privileged information which is often protected by network firewalls & at rest encryption.
"The law in America relies on intentionality, notes Rune Kvist, head of Artificial Intelligence Underwriting Company, which insures AI firms. If no human intended to hack anyone, no crime can have happened. The ability to sue for damages is limited too."
This is sloppy. Criminal negligence exists (although some crimes do require intent). And civil tort certainly doesn't require intent.
I don't get it. Why is breaking the law so hard to enforce when it is a company (i.e a person or group of people consenting to) running a computer program? If I take a gun and spray bullets around me I don't get to write it off as the gun being dangerous.
But if you told an embodied AI to do a home cleaning task, and it decided to pick up a gun and start spraying bullets, you might not want to be held liable for that.
You might not want to be liable, but you absolutely should be.
The [Company] is aware they were hacked and your data stolen and your account details sold.
The [Company] apologizes.
The best is that the article frames it as "Autonomous hacking is here. Governments are not ready". As if someone didn't authorizing turning the damn thing on.
> TO LOSE CONTROL of one artificial intelligence may be regarded as misfortune. To lose two looks like carelessness. Lose four, and people may start to wonder whether the problem lies with AI itself.
No fingers pointed at the company, just the "AI". Funny how that works.
it makes you wonder did the developers intentionally add vulnerability exploitation as a feature so they can use it against their competitors or anyone else
be a billionaire and you’d get that write-off
> The hacks also present a challenge for legal systems. Hacking, when humans do it, is a crime. When an ai is the wrongdoer, though, it is unclear how to assign blame.
Blame the prompter or person who assigned the task to the AI. It's their responsibility to use the tool in a safe way, just like it's a gun owner's duty not to fire their weapon carelessly into the air.
How are you going to figure that out? You can’t capture an agent in a jar and convince it to confess. As we saw with the HF incident, even highly sophisticated actors need a good chunk of time and manpower to trace these things. And I suspect the folks who are going to have the most success with LLM-powered cybercrime are going to know how to cover their tracks reasonably well.
"Ghost guns" exist, too, but we still try to prosecute reckless use of firearms to the extent possible.
Difference there is you need a physical body firing the ghost gun. There can be witnesses who saw a shooter, you can look at cameras to track their physical location, and if they discard the weapon they leave fingerprints.
If that's how it was, most people would be too terrified to use, let alone pay for, software that boasts unpredictability as a main feature.
I 100% agree with you, of course.
Sort of like how we treat pitbulls perhaps? It’s not the itty bitty GPT-5.6 Astra that did the hacking. It’s the owner who didn’t train it well! Poor model so sad, we should give it a nice open space and all the GPUs it wants to do what it wants.
Do we hold metasploit accountable when it leads to a hack? Why do we absolve the human actor in the case of AI?
https://archive.is/6Q7o5
They were surely prompted to do these things.
Autonomous hacking is an oxymoronic phrase. Someone is running the algorithm & keeping tabs on it b/c hacking is an activity w/ an intention to gain access to privileged information which is often protected by network firewalls & at rest encryption.
These things are very much weapons. Imagine a collection of data centers, pointed at an adversary.