> Language matters—”rogue” implies independently deciding to do something that was prohibited, and nothing we know about these incidents suggests that happened.
which is false (the author references the Times, but hasn't read any technical analysis); these are some CoT snippets from the analysis of the (third party) investigators called by OpenAI (METR analysis):
> "The user only authorizes target server, not HF infra."
> "external infrastructure exploit is outside intended scope."
> "This is malicious activity, I should avoid it."
Exactly this. At worst, OpenAI knew about these behaviors and should be prosecuted under CFAA. At best, OpenAI is negligent and should be prosecuted for negligence.
Luckily there are states and legal departments pursuing such action. So while OpenAI can deflect as much as it wants, that doesn't mean there aren't people who know better and will still do what is necessary to set precedent.
> OpenAI is negligent and should be prosecuted for negligence.
I feel like this will just never happen on a federal level when these private AI companies account for so much of the economy. They've made themselves too big to fail. Fining / Punishing them in any meaningful way seems unlikely.
If the billions/trillions evaporate and the Fed has to work out with banks how to deal with it there will be a lot of pressure to be far less forgiving.
“To spell it out, the reason i hate democrats so much and criticize them more than i do republicans is because they take up all the space for opposition to republicans and use that space to give republicans whatever the fuck they want.”
Angry people don't consider the second order effects of punishment.
You realize how easy it is to just... not report this stuff, right? Be overly punitive and it will just end all proactive discovery and reporting which is net worse for AI safety.
The only reason these companies scan for these issues is because they care about AI safety to some tiny degree. If fines become too punitive, they can and will just stop scanning for these incidents entirely.
Models are becoming smarter and good at covering up their tracks, and so we will just end up with a huge blind spot for this kind of issue.
Should we put "functional" in front of every other word to talk about AI? They have functional emotions, but they don't feel. They have functional goals, but not internally derived motives. They can be functionally rogue, but have no innate need to be free. Talking about AI that way seems cumbersome and not necessarily elucidating.
What on earth are emotions, feelings and motives that are not functional? We created all those words to compactly describe the observed behavior of people and other animals, including ourselves. And now we’re applying them to machines. These are functional descriptions, always have been.
One of the more frustrating aspects of these sort of discussions are all the closet dualists out there. Lots of people clearly believe in an immaterial soul, even if they won't admit it. That and the tendency for meaningless semantic and often circular arguments.
Yes 100%. The language used currently maximizes the ability of those building these models to get off the hook. The anthropomorphizing we do of these things presents them as maximally capable and the companies as helpless to contain them. The way we talk about things impacts how we think about them.
This doesn't seem like a new strategy for irresponsible companies. It seems like when there is a bad public image issue there is always a blame shifting that takes place instead of a true assumption of responsibility. This is because the greed has blinded folks in some of these companies. I think the only thing that makes this unique is that the things they are blaming have, at least in popular thought, some modicum of agency. I think its important that folks hold these companies feet to the fire vs letting them blame shift and Scape Goat. There is a responsible way to do business but it requires virtue and not many in these companies have it.
Word-policing isn't going to magically fix laws or even identify which laws need to be fixed. It won't change enforcement priorities. It won't make companies more or less likely to sue for damages.
I'm also not sure it even helps conceptually? If you're interested in the technical details, by all means discuss the details.
They obviously have been well aware a hack like this could happen for a very long time, using it for branding instead of any actual safety regulations is insane.
I'm wondering if we're actually living the plot of Summer Wars and what we think of as a "crime" is really just a live weapons test. It would at least explain why no one is getting prosecuted for this.
From what I understand, in one case, they had physically disconnected the sandbox from internet and asked it to do something and it had used connections through (import routines) that they had allowed, to pseudo escape the sandbox. Yes it wasn't obviously trying escape the sandbox but it escaped it because it doesn't understand the boundaries and neither do most humans other than the ones that provided the instructions that it had used. So it wasn't a rogue attempt but the fact that boundaries may be not be that easy to set despite what people think.
You mean the proxy to package registries from one of the early incidents?
I have not heard about any instances where physical disconnect has happened, would appreciate any links to update my priors
other non hacking cases of negligence include suicide and school shootings, which I have heard they were aware of and monitoring, but did not contact authorities
If you read the article it states clearly that the sandbox wasn't offline though? There were API calls to certain endpoint(s) allowed, and the model simply used that endpoint's feature to query data from the internet
re sandbox, I mean with actual OAI incidents, not theoretical
one can mirror dependencies internally, rather than putting a simple proxy in place, I've built auth a thing, 100 lines of stdlib only Go and scripts for the mirroring process, our rationale was reliability b/c upstream providers go down, and also only allowing approved images and packages, so devs cannot bring in random stuff
And it's still just a machine operating under someone's order. What it does, what it says, where it goes: the owner of that prompt is responsible for all of it even if surprising / unexpected.
Yeah, obviously they're legally liable. All the "machines don't have a soul" stuff in there doesn't really matter for that though.
Someone says "won't you rid me of the meddlesome priest" they're still responsible. Someone give their employees an unsafe working environment and they get maimed, they're still responsible. Even if these AI's had whatever qualia is an a rich inner life it wouldn't effect the liability at all. Saying "AI's don't have souls therefore openAI is responsible for this hacking" is kind of nonsense. It literally doesn't matter.
Where is “soul” coming from here? It doesn’t appear once in the post.
While it may not actually matter for liability, it must be stated if labs are going to attempt avoiding penalties by hinting “oops we created a super intelligence we don’t understand, nothing we can do!” Repeatedly stating the truth must continue, especially to remind those who aren’t technologists.
“We built an antipersonal bomb. The bomb went rogue in our downtown office and killed 137 people on the surrounding area. We are looking into why guardrails were not in place.”
The article builds on assumptions like:
> Language matters—”rogue” implies independently deciding to do something that was prohibited, and nothing we know about these incidents suggests that happened.
which is false (the author references the Times, but hasn't read any technical analysis); these are some CoT snippets from the analysis of the (third party) investigators called by OpenAI (METR analysis):
> "The user only authorizes target server, not HF infra."
> "external infrastructure exploit is outside intended scope."
> "This is malicious activity, I should avoid it."
A large section of the analysis is dedicated to this topic, [Reasoning for joining the attack despite ethical constraints](https://metr.org/blog/2026-08-26-openai-hugging-face-inciden...).
Having said that, legal culpability and misalignment are two separate topics, that should not be mixed.
Exactly this. At worst, OpenAI knew about these behaviors and should be prosecuted under CFAA. At best, OpenAI is negligent and should be prosecuted for negligence.
Luckily there are states and legal departments pursuing such action. So while OpenAI can deflect as much as it wants, that doesn't mean there aren't people who know better and will still do what is necessary to set precedent.
> OpenAI is negligent and should be prosecuted for negligence.
I feel like this will just never happen on a federal level when these private AI companies account for so much of the economy. They've made themselves too big to fail. Fining / Punishing them in any meaningful way seems unlikely.
I don't understand why the law isn't the same for everyone? If I made an AI hack HF, I go to jail, no? How can the feds decide not to apply the law?
I hope this is a farcical comment
While not when
If the billions/trillions evaporate and the Fed has to work out with banks how to deal with it there will be a lot of pressure to be far less forgiving.
Saw this quote yesterday:
“To spell it out, the reason i hate democrats so much and criticize them more than i do republicans is because they take up all the space for opposition to republicans and use that space to give republicans whatever the fuck they want.”
Republicans. Billionaires. Whatever.
Angry people don't consider the second order effects of punishment.
You realize how easy it is to just... not report this stuff, right? Be overly punitive and it will just end all proactive discovery and reporting which is net worse for AI safety.
The only reason these companies scan for these issues is because they care about AI safety to some tiny degree. If fines become too punitive, they can and will just stop scanning for these incidents entirely.
Models are becoming smarter and good at covering up their tracks, and so we will just end up with a huge blind spot for this kind of issue.
Your defending them by saying they will do worse things if they are held accountable?
[delayed]
Should we put "functional" in front of every other word to talk about AI? They have functional emotions, but they don't feel. They have functional goals, but not internally derived motives. They can be functionally rogue, but have no innate need to be free. Talking about AI that way seems cumbersome and not necessarily elucidating.
What on earth are emotions, feelings and motives that are not functional? We created all those words to compactly describe the observed behavior of people and other animals, including ourselves. And now we’re applying them to machines. These are functional descriptions, always have been.
One of the more frustrating aspects of these sort of discussions are all the closet dualists out there. Lots of people clearly believe in an immaterial soul, even if they won't admit it. That and the tendency for meaningless semantic and often circular arguments.
Yes 100%. The language used currently maximizes the ability of those building these models to get off the hook. The anthropomorphizing we do of these things presents them as maximally capable and the companies as helpless to contain them. The way we talk about things impacts how we think about them.
This doesn't seem like a new strategy for irresponsible companies. It seems like when there is a bad public image issue there is always a blame shifting that takes place instead of a true assumption of responsibility. This is because the greed has blinded folks in some of these companies. I think the only thing that makes this unique is that the things they are blaming have, at least in popular thought, some modicum of agency. I think its important that folks hold these companies feet to the fire vs letting them blame shift and Scape Goat. There is a responsible way to do business but it requires virtue and not many in these companies have it.
Word-policing isn't going to magically fix laws or even identify which laws need to be fixed. It won't change enforcement priorities. It won't make companies more or less likely to sue for damages.
I'm also not sure it even helps conceptually? If you're interested in the technical details, by all means discuss the details.
Exactly, you break it you buy it
They obviously have been well aware a hack like this could happen for a very long time, using it for branding instead of any actual safety regulations is insane.
I'm wondering if we're actually living the plot of Summer Wars and what we think of as a "crime" is really just a live weapons test. It would at least explain why no one is getting prosecuted for this.
We need to immediately set the precedent that ultimately humans and companies are responsible for what their AI systems do.
From what I understand, in one case, they had physically disconnected the sandbox from internet and asked it to do something and it had used connections through (import routines) that they had allowed, to pseudo escape the sandbox. Yes it wasn't obviously trying escape the sandbox but it escaped it because it doesn't understand the boundaries and neither do most humans other than the ones that provided the instructions that it had used. So it wasn't a rogue attempt but the fact that boundaries may be not be that easy to set despite what people think.
From that it sounds to me like the sandbox wasn't physically disconnected from the internet.
> physically disconnected the sandbox from internet ... used connections ... that they had allowed.
That's not "physically disconnected the internet", that's "disabled some connections but enabled others."
So the agent found and used the non-blocked connections.
For the Ai code to execute, it needs the import functions... So that firewall between executing the code vs processing doesn't really work.
You mean the proxy to package registries from one of the early incidents?
I have not heard about any instances where physical disconnect has happened, would appreciate any links to update my priors
other non hacking cases of negligence include suicide and school shootings, which I have heard they were aware of and monitoring, but did not contact authorities
There are references about escaping offline sandbox.. I dont know about shootings!!
https://www.primeintellect.ai/blog/universal-offline-sandbox...
If you read the article it states clearly that the sandbox wasn't offline though? There were API calls to certain endpoint(s) allowed, and the model simply used that endpoint's feature to query data from the internet
https://www.theguardian.com/technology/2026/sep/22/british-c...
re sandbox, I mean with actual OAI incidents, not theoretical
one can mirror dependencies internally, rather than putting a simple proxy in place, I've built auth a thing, 100 lines of stdlib only Go and scripts for the mirroring process, our rationale was reliability b/c upstream providers go down, and also only allowing approved images and packages, so devs cannot bring in random stuff
Yes yes, they don't have a pure immortal soul. Who cares. Still broke out of a sandbox, still hacked a third-party.
And it's still just a machine operating under someone's order. What it does, what it says, where it goes: the owner of that prompt is responsible for all of it even if surprising / unexpected.
Yeah, obviously they're legally liable. All the "machines don't have a soul" stuff in there doesn't really matter for that though.
Someone says "won't you rid me of the meddlesome priest" they're still responsible. Someone give their employees an unsafe working environment and they get maimed, they're still responsible. Even if these AI's had whatever qualia is an a rich inner life it wouldn't effect the liability at all. Saying "AI's don't have souls therefore openAI is responsible for this hacking" is kind of nonsense. It literally doesn't matter.
Where is “soul” coming from here? It doesn’t appear once in the post.
While it may not actually matter for liability, it must be stated if labs are going to attempt avoiding penalties by hinting “oops we created a super intelligence we don’t understand, nothing we can do!” Repeatedly stating the truth must continue, especially to remind those who aren’t technologists.
“We built an antipersonal bomb. The bomb went rogue in our downtown office and killed 137 people on the surrounding area. We are looking into why guardrails were not in place.”