1. Prosecute and lock people up over known instances of AI-enabled hacking, so companies playing fast and loose know “oops my AI did it” isn’t a useful defense.
2. If you’re actually worried about AI killing lots of people, you immediately stop training AI specifically at the art of killing people, like selecting bombing targets, flying fighter jets, etc.
This seems to be a case where Anthropic just can't win in the public court, and the other big ones (Google, Microsoft) have realized it / don't care enough, so they are rather quiet. Anthropic constantly gets accused of trying to lobby for regulation in order to lock out smaller competitors, but they've consistently lobbied for regulation aimed at frontier labs with exceptions for smaller labs. That's what this bill is about as well.
> "regulation aimed at frontier labs with exceptions for smaller labs"
I didn't read it that way. This is a bill that would ban open-weight models (good ones); effectively mandating gated-access, locked-down, high-refusal rate services, like what Anthropic provides.
If a life is only worth $20m, by their own calculus: saving > 50k lives is worth more than Anthropic. Tobacco ostensibly causes ~8m deaths a year, so banning tobacco (easy intervention) should be worth 160 Anthropic sized labs. :D
I don't think this is avoidable even at the present day. Anyone with sufficient capital could download a leading open-source model and run it on ~$10-25K hardware. They could align it to their own interest, including making a biological weapon. This law is akin to blocking Torrent downloads. Sure, you can slow them, but there will always be back channels.
The next step is demanding anyone who wishes to release a Certified Safe AI™ model to prove they can insure the $1B in damages and pass a compliance audit for how they will prevent the LTM death count from reaching 50, effectively making it so that only big labs can serve government and enterprise.
It’s going to be fun reading the reasoning transcripts where it argues with itself over whether it can afford to spend 1 or 2 human lives on the current task.
With a new model every couple of months this can only be triggered if it happens all at once.
What is the average damage caused by OpenAI/Anthropic hacks? I feel like we should aim at lowering that number, not increasing it.
What's the rule for fully autonomous cars?
Should we allow this as unsupervised at all if we are really afraid of deaths, while there are practically no states that allow 100% autonomous cars (where we are actually afraid of deaths)?
If "an independent third party registered with the Department of Commerce would periodically evaluate the lab", will this turn into an expensive certification/insurance that cuts off smaller competition (totally by mistake, I'm sure)?
I am also very very sure this is most definitely not at all going to become like product certification if EU, where there are literally 2 companies certifying various standards: they police each other and a stupid electric fan can cost you tens of thousands of euros to certify.
On the bright side, don’t offer an exchange rate based on ethnicity or other background criteria but I get the feeling that your AI won’t be able to kill 49 billionaires without triggering something. Can probably do 49 millionaires without hitting the $1B threshold? Maybe someone will come up with a risk management SaaS as not all millionaires are the same and maybe you’ll hit the $1B before the 49th.
The dark humor aside, is it really a damage when AI kills 49 millionaires? After all it probably creates even more millionaires as their wealth is distributed to survivors. So the actual risk could be when AI kills 49 engineers since that actually may translate to losses for their employers.
Who's keeping score? Also define "death". If a non-alive person has social media, a blog, a diary, meeting recordings, emails - we'll make it so they never left. Hell, we'll partner with Elon and Mark and a make it a deactivation hook.
$1B is an extraordinarily easy bar to clear with a single crypto hack and Anthropic was repeatedly refusing to give access to Mythos to defenders in the crypto world. I think this stems from bias rather than a conspiracy theory but it's an entirely avoidable disaster waiting to happen.
> Anthropic was repeatedly refusing to give access to Mythos to defenders in the crypto world.
This almost makes sense. You almost can't risk vulnerabilities even being discovered in certain systems that are impossible to patch, like immutable distributed smart contracts.
Not sure that justifies withholding Mythos access entirely, but what do I know. I apparently got access to Mythos a few days ago since Anthropic folded Project Glasswing into the CVP.
Most smart contracts are not impossible to patch because everyone knows that's bad for security. They use a delegate/proxy pattern where the deployer can change the implementation any time but users can still access the old one with a third party interface.
And if a problem is critical enough the community can decide to hard fork and rewrite the contracts source code even if it's immutable. It already happened with the Ethereum DAO hack and most people were okay with it.
The issue with stuff like this is inheriting all the funds held in the old contract. I guess a hard fork can do it. There's also the issue of trustworthiness of the smart contract owner. Trustworthiness in general is a huge issue in the crypto world, because if you can get away with something you can do it pretty much instantly and irrevocably (without another hard fork anyway). There's a chance that if you share a vulnerability that would allow one of the contract owners to take advantage, they will just do it and run. Plenty of cases of previously-trustworthy crypto people seeing an opportunity and disappearing with a lifetime of money.
Yes, and if Anthropic are trying what people are saying they are, that would be intentional. $1B could be reached quickly if cyberattacks supported by AI count.
Do we even know if ChatGPT's sycophancy killed 50 people? The last figure I've seen for suicide-by-ChatGPT was like 13 or 14 cases, not sure if there have been more.
This alleviates them from harm on the individual level, consumers and communities, we the people. We've already seen encouraged suicides and school shootings. These would not qualify.
That is why the bar is set where it is. The corporate circle jerk designed this as a cost of doing business, not as a check on responsibility or restitution for real damages to people and society.
All these "safety" bills are to stop open source models by painting them as "dangerous". Regulatory capture.
Companies are responsible for their agents; you don't blame the car for running over the person.
Proposal:
1. Prosecute and lock people up over known instances of AI-enabled hacking, so companies playing fast and loose know “oops my AI did it” isn’t a useful defense.
2. If you’re actually worried about AI killing lots of people, you immediately stop training AI specifically at the art of killing people, like selecting bombing targets, flying fighter jets, etc.
Of course we all know none of these will happen.
This seems to be a case where Anthropic just can't win in the public court, and the other big ones (Google, Microsoft) have realized it / don't care enough, so they are rather quiet. Anthropic constantly gets accused of trying to lobby for regulation in order to lock out smaller competitors, but they've consistently lobbied for regulation aimed at frontier labs with exceptions for smaller labs. That's what this bill is about as well.
> "regulation aimed at frontier labs with exceptions for smaller labs"
I didn't read it that way. This is a bill that would ban open-weight models (good ones); effectively mandating gated-access, locked-down, high-refusal rate services, like what Anthropic provides.
If a life is only worth $20m, by their own calculus: saving > 50k lives is worth more than Anthropic. Tobacco ostensibly causes ~8m deaths a year, so banning tobacco (easy intervention) should be worth 160 Anthropic sized labs. :D
I don't think this is avoidable even at the present day. Anyone with sufficient capital could download a leading open-source model and run it on ~$10-25K hardware. They could align it to their own interest, including making a biological weapon. This law is akin to blocking Torrent downloads. Sure, you can slow them, but there will always be back channels.
The next step is demanding anyone who wishes to release a Certified Safe AI™ model to prove they can insure the $1B in damages and pass a compliance audit for how they will prevent the LTM death count from reaching 50, effectively making it so that only big labs can serve government and enterprise.
With existing players given a free pass as is necessary for the functioning of a democratic society
> Certified Safe SI™
There is an idea for a pooled, mutualized insurance fund. Ai companies would contribute based on various factors
https://www.lawfaremedia.org/article/bootstrapping-frontier-...
It’s going to be fun reading the reasoning transcripts where it argues with itself over whether it can afford to spend 1 or 2 human lives on the current task.
With a new model every couple of months this can only be triggered if it happens all at once.
What is the average damage caused by OpenAI/Anthropic hacks? I feel like we should aim at lowering that number, not increasing it.
What's the rule for fully autonomous cars?
Should we allow this as unsupervised at all if we are really afraid of deaths, while there are practically no states that allow 100% autonomous cars (where we are actually afraid of deaths)?
If "an independent third party registered with the Department of Commerce would periodically evaluate the lab", will this turn into an expensive certification/insurance that cuts off smaller competition (totally by mistake, I'm sure)?
I am also very very sure this is most definitely not at all going to become like product certification if EU, where there are literally 2 companies certifying various standards: they police each other and a stupid electric fan can cost you tens of thousands of euros to certify.
I guess they solved the trolley problem internally
1e1,000,000 lines of lean. Just needs human review.
At least we now know that a life is worth: 20mil
Not sure what kicks in, but if a tiny company worth very little is destroyed by an AI hack they should be 100% liable for all damages.
Why should a small business suffer because it wasnt worth a billion dollars?
I think you answered your own question. Government protection is directly proportional to net worth.
well its because amount of money has
if you has debt of 1 million, you has problem
but if you have debt of 1 billion, its bank problem
On the bright side, don’t offer an exchange rate based on ethnicity or other background criteria but I get the feeling that your AI won’t be able to kill 49 billionaires without triggering something. Can probably do 49 millionaires without hitting the $1B threshold? Maybe someone will come up with a risk management SaaS as not all millionaires are the same and maybe you’ll hit the $1B before the 49th.
I feel like the surviving family of even a single millionaire could claim $1B in damages if the millionaire was young enough.
Right, so maybe they had an exchange rate after all. Just implicit.
I got the feeling that what they had in their essentially culling the energy consumers with low economic output.
There's always been an exchange rate based on the quality of your legal team. Get one sociopathic enough and they'll pick a judge right apart.
The dark humor aside, is it really a damage when AI kills 49 millionaires? After all it probably creates even more millionaires as their wealth is distributed to survivors. So the actual risk could be when AI kills 49 engineers since that actually may translate to losses for their employers.
Who's keeping score? Also define "death". If a non-alive person has social media, a blog, a diary, meeting recordings, emails - we'll make it so they never left. Hell, we'll partner with Elon and Mark and a make it a deactivation hook.
So Claude figured out a human life is worth 20 million dollars?
So Anthropic is now the authority on life or death decisions? How far has the US deteriorated?
$1B is a pretty easy bar to clear.
$1B is an extraordinarily easy bar to clear with a single crypto hack and Anthropic was repeatedly refusing to give access to Mythos to defenders in the crypto world. I think this stems from bias rather than a conspiracy theory but it's an entirely avoidable disaster waiting to happen.
> Anthropic was repeatedly refusing to give access to Mythos to defenders in the crypto world.
This almost makes sense. You almost can't risk vulnerabilities even being discovered in certain systems that are impossible to patch, like immutable distributed smart contracts.
Not sure that justifies withholding Mythos access entirely, but what do I know. I apparently got access to Mythos a few days ago since Anthropic folded Project Glasswing into the CVP.
Most smart contracts are not impossible to patch because everyone knows that's bad for security. They use a delegate/proxy pattern where the deployer can change the implementation any time but users can still access the old one with a third party interface.
And if a problem is critical enough the community can decide to hard fork and rewrite the contracts source code even if it's immutable. It already happened with the Ethereum DAO hack and most people were okay with it.
The issue with stuff like this is inheriting all the funds held in the old contract. I guess a hard fork can do it. There's also the issue of trustworthiness of the smart contract owner. Trustworthiness in general is a huge issue in the crypto world, because if you can get away with something you can do it pretty much instantly and irrevocably (without another hard fork anyway). There's a chance that if you share a vulnerability that would allow one of the contract owners to take advantage, they will just do it and run. Plenty of cases of previously-trustworthy crypto people seeing an opportunity and disappearing with a lifetime of money.
Seems kind of low, actually.
Yes, and if Anthropic are trying what people are saying they are, that would be intentional. $1B could be reached quickly if cyberattacks supported by AI count.
Do we even know if ChatGPT's sycophancy killed 50 people? The last figure I've seen for suicide-by-ChatGPT was like 13 or 14 cases, not sure if there have been more.
how about the LLMs being used for target selection in Iran? Or does collateral damage not count?
I haven't been following the wars very much these days. That sounds pretty bad indeed.
Also the LLM that nearly started a war with China recently.
so 49 totally okay.
I would not like to see a world where companies push their models as close as possible to killing exactly 49 people without reaching 50
This alleviates them from harm on the individual level, consumers and communities, we the people. We've already seen encouraged suicides and school shootings. These would not qualify.
That is why the bar is set where it is. The corporate circle jerk designed this as a cost of doing business, not as a check on responsibility or restitution for real damages to people and society.
Good to know the going rate for a soul is $20M.
/s
With a minimum quantity of 50.
After the AISI and MAISI AI promotion institutes, let us create SISI (pronounced sissy) for investigating a super-intelligence safety bill.
This is all theater, The whole Trump clan is invested in the AI bubble.
Nepotism is the primary benefit to being president. What don't we get about this?
I guess it’s unclear why an entire congress goes along with it (I blame democrats too for sane washing and going about their workday)
Oligarchy, they are all in on the winnings. Democrats are only marginally better because they say the things we like to hear on social causes.