>...researchers found a bug in the way that the community-discussion forum Discourse processed certain image files. The researchers had access to a special version of Claude Opus 4.8...
>At first, it didn’t work. That evening, however, Anthropic released Opus 5 and by the next day, Claude had found a way to exploit the bug...
Is this speed of capability because hacking is almost entirely machine verifiable, thus training quicker/deeper than other domains?
>Interestingly, the vulnerable code had been changed upstream the previous year, but the commit was not documented as a security fix and received no CVE.3 This might be a reason why Debian 12 and 13 have not received the security relevant backports in time.
Ooof, keeping packages like this up to date with the rate of updates and churn is a mess.
I suspect you don’t really know what you are talking about. “It pays.” is not the only reason people like doing bad things. You’re right about the movies bit though, people tend to like black and white narratives as your naive “That’s why people like doing bad things. It pays.” comment perfectly demonstrates.
This whole blog-post is impressive with the chain of vulnerabilities involved. However...
> OpenAI also paid us a $6,500 bounty.
?
That amount for this payout is beyond pathetic for a near $1.2T company, who just got themselves breached with a complete potential source code leak.
This is like getting close to breaching the main monorepo at Google: google3.
If this was on the black market and the leak included unreleased models and training material, it would easily be worth tens of millions. Even reporting crypto smart contract flaw pay way more than that on average of $100k - $10M.
The unfortunate truth of doing the right thing. Also, correct me if I'm wrong but there are too many bad things out there and companies can't give 1 million bounty for stuff like that. I'm sure they could but in the long run, wouldn't it be unsustainable?
Pay next to nothing every time, accept one financially-depressed researcher sale to blackhats causing tremendous business disruption every n years. Cheaper than honest payouts to [keep] researchers [honest]? Keep paying chump change. (Booo)
No more unsustainable than these companies already are by default. The bounty should have been proportionate to how important and pressing the findings were.
>...researchers found a bug in the way that the community-discussion forum Discourse processed certain image files. The researchers had access to a special version of Claude Opus 4.8... >At first, it didn’t work. That evening, however, Anthropic released Opus 5 and by the next day, Claude had found a way to exploit the bug...
Is this speed of capability because hacking is almost entirely machine verifiable, thus training quicker/deeper than other domains?
>Interestingly, the vulnerable code had been changed upstream the previous year, but the commit was not documented as a security fix and received no CVE.3 This might be a reason why Debian 12 and 13 have not received the security relevant backports in time.
Ooof, keeping packages like this up to date with the rate of updates and churn is a mess.
$6 500 bounty for this is a joke. The black market price would be smth like $6 500 000 or more
There is probably no black market for this at all.
https://news.ycombinator.com/item?id=43025038
As someone who spent some time on the receiving end on a very public bug bounty program, thank you for nailing all of it.
I also thought that's crazy. Why even bother for these kind of bounties.
That's why people like doing bad things. It pays. Why do you think movies like using this single theme over and over again? It's always happening
I suspect you don’t really know what you are talking about. “It pays.” is not the only reason people like doing bad things. You’re right about the movies bit though, people tend to like black and white narratives as your naive “That’s why people like doing bad things. It pays.” comment perfectly demonstrates.
I really hope those model weights are more secure than this
This whole blog-post is impressive with the chain of vulnerabilities involved. However...
> OpenAI also paid us a $6,500 bounty.
?
That amount for this payout is beyond pathetic for a near $1.2T company, who just got themselves breached with a complete potential source code leak.
This is like getting close to breaching the main monorepo at Google: google3.
If this was on the black market and the leak included unreleased models and training material, it would easily be worth tens of millions. Even reporting crypto smart contract flaw pay way more than that on average of $100k - $10M.
Come on.
My guess is that OpenAI has done a lot more to prevent exfil of their model weights than the codebase of their main web app and client.
The unfortunate truth of doing the right thing. Also, correct me if I'm wrong but there are too many bad things out there and companies can't give 1 million bounty for stuff like that. I'm sure they could but in the long run, wouldn't it be unsustainable?
It’s an interesting bet then.
Pay next to nothing every time, accept one financially-depressed researcher sale to blackhats causing tremendous business disruption every n years. Cheaper than honest payouts to [keep] researchers [honest]? Keep paying chump change. (Booo)
How much would a nation state pay for a complete copy of OpenAI’s github repositories? I doubt there are many full chains laying around like this.
No more unsustainable than these companies already are by default. The bounty should have been proportionate to how important and pressing the findings were.