> It found the U.S. “failed in its obligation to do everything feasible to verify” that the school was a military objective and that the failure “went beyond mere negligence.” The report said the United States “directed the strikes at the building of the school while being aware of a substantial risk of striking a civilian object and acting recklessly as regards the possibility that this would happen.”
Reading the details, "AI" doesn't really seem like the culprit -it's a scapegoat.
The intelligence that it was no longer a military target never entered the target database, the team that was responsible for vetting the target list was gutted, and said team was never even consulted.
The White House wanted 1000 targets and pulled from their database without any due diligence. Whether it was an AI call or an SQL query - this was from pure human maliciousness and incompetence.
While I think that it's important to do root cause analysis, we need to be very careful not to absolve blame from the people that decided to shift a significant weight of decision-making authority to an AI. An AI can be tried in a court. There has to be a responsible human for everyone one of its actions, especially when it's killing (-people-) children.
> The Minab site, which was cataloged as an Islamic Revolutionary Guard Corps facility due to outdated data, was fed into Maven with other candidates and came out as a recommended day-one target. Target-list work that once took hours was condensed into minutes.
I feel like we're optimizing the wrong metric here.
The pentagon says it was Palantir software’s fault, Palantir says it was the fault of bad data input to their software. Where’s the accountability by both organizations? How did we get to a world where military decisions leading to innocent deaths are responded to as if it was just another miscommunication implementing a B2B SaaS platform.
> Officials said some users expected Maven to flag stale records or contradictions in the intelligence assembled for potential targets, though it is not clear why they thought the system would do that
This is what happens when people who don't understand AI and its limitations embrace it wholeheartedly, thinking it's going to behave like The ultimate analyst. Except it's not. Its got blind spots in more places than anyone knows. And not to mention the system itself has multiple areas of failure.
Is that really the case? Stories like this always have all sides muddying waters.
The terrorists are well known to use children and the rest of the public as hostage meatshields, and the US military is always looking for something/someone else to blame for the dirty work they have to do.
The broader public is so polarized by performative politics and botspam that they eat these stories up, but nothing I'm saying is secret. It's been this way since forever. Slow news today?
I'm not even defending AI at all and think these are crappy choices, but compared to humans gathering intelligence who can go rogue, what alternative does anyone really have?
nice, Karp really reeled in a nice big fish, and ofcourse he is going to blame OpenAI/Anthropic for giving the targets, this explains his CNBC rants about the frontiers, his tool was the trigger in a killchain of hallucination.
Completely disagree. When we’re talking about something like air, missile, or artillery strikes, the operator of the weapon system may literally never even see the target, on a screen or otherwise. They may simply be entering targeting information provided to them by an operations center. These decisions are social and group decisions, with distributed responsibility, which is why it’s so important to have institutions like the Civilian Protection Center of Excellence which historically has identified and helped implement the policies which prevent these kinds of disasters, but which unfortunately the current administration decided to shut down without a replacement.
Sometimes, the difference between a successful strike on a lawful combatant and disaster is one analyst pointing out that the building the enemy is hiding in is next to a square that’s used for a market on Tuesdays and wouldn’t you know it, it’s Tuesday. That’s not something the person “pulling the trigger,” can always be expected to know, which is why every ops center needs to be designed to elevate those voices, when the time comes
I would argue `s/pulls the trigger/gives the go ahead/` is intuitively more appropriate. The sailor on the boat that fired off the tomahawk (iirc) was likely not in a position to make an in-the-moment determination
DOGE strikes again. Edit actually looks like it was Hegseth himself, king Testosterone.
> Staffing on civilian harm mitigation teams across the U.S. Department of Defense had also fallen by roughly 90% over recent years, shrinking to fewer than 20 people overall. Centcom’s group shrank from 10 people to one.
> It found the U.S. “failed in its obligation to do everything feasible to verify” that the school was a military objective and that the failure “went beyond mere negligence.” The report said the United States “directed the strikes at the building of the school while being aware of a substantial risk of striking a civilian object and acting recklessly as regards the possibility that this would happen.”
Reading the details, "AI" doesn't really seem like the culprit -it's a scapegoat.
The intelligence that it was no longer a military target never entered the target database, the team that was responsible for vetting the target list was gutted, and said team was never even consulted.
The White House wanted 1000 targets and pulled from their database without any due diligence. Whether it was an AI call or an SQL query - this was from pure human maliciousness and incompetence.
While I think that it's important to do root cause analysis, we need to be very careful not to absolve blame from the people that decided to shift a significant weight of decision-making authority to an AI. An AI can be tried in a court. There has to be a responsible human for everyone one of its actions, especially when it's killing (-people-) children.
I cannot believe this has come to pass.
can -> can't
> The Minab site, which was cataloged as an Islamic Revolutionary Guard Corps facility due to outdated data, was fed into Maven with other candidates and came out as a recommended day-one target. Target-list work that once took hours was condensed into minutes.
I feel like we're optimizing the wrong metric here.
Related: the U.S. nearly boarded a Chinese boat that AI incorrectly flagged as carrying nuclear weapons materiel.
https://gizmodo.com/almost-started-a-war-us-military-nearly-...
The pentagon says it was Palantir software’s fault, Palantir says it was the fault of bad data input to their software. Where’s the accountability by both organizations? How did we get to a world where military decisions leading to innocent deaths are responded to as if it was just another miscommunication implementing a B2B SaaS platform.
> Officials said some users expected Maven to flag stale records or contradictions in the intelligence assembled for potential targets, though it is not clear why they thought the system would do that
This is what happens when people who don't understand AI and its limitations embrace it wholeheartedly, thinking it's going to behave like The ultimate analyst. Except it's not. Its got blind spots in more places than anyone knows. And not to mention the system itself has multiple areas of failure.
I'm going to bet that absolutely nobody higher up will end up going to jail for this.
Whole world is turning into an unlivable place and people who are doing it are not taking responsibility.
They never have. Their money insulates them from the consequences of their actions.
There’s a threshold where the money won’t insulate anymore.
Tragically this won't be the last time AI kills a lot of innocent people.
AI didn’t kill innocent people here, the Department of Defense and its decision-makers did.
Blowing up a school is proven route to diplomatic triumph and regime change!
Is that really the case? Stories like this always have all sides muddying waters.
The terrorists are well known to use children and the rest of the public as hostage meatshields, and the US military is always looking for something/someone else to blame for the dirty work they have to do.
The broader public is so polarized by performative politics and botspam that they eat these stories up, but nothing I'm saying is secret. It's been this way since forever. Slow news today?
I'm not even defending AI at all and think these are crappy choices, but compared to humans gathering intelligence who can go rogue, what alternative does anyone really have?
It was a school. No one on either side is denying this.
so who is paying for this? "humans must always remain accountable for artificial intelligence (AI) decisions"
An Maven AI used Claude, which was denied here multiple times. Maybe call the next model after Fable Minab!
In a just world, Pete Hegseth and the entire board of Palantir would be in prison for this.
time to make Palantir the next Flock
France gave them the boot, others can too
nice, Karp really reeled in a nice big fish, and ofcourse he is going to blame OpenAI/Anthropic for giving the targets, this explains his CNBC rants about the frontiers, his tool was the trigger in a killchain of hallucination.
The one that pulls the trigger bears the responsibility.
No. That's too easy on the higher ups, they never pull triggers.
Completely disagree. When we’re talking about something like air, missile, or artillery strikes, the operator of the weapon system may literally never even see the target, on a screen or otherwise. They may simply be entering targeting information provided to them by an operations center. These decisions are social and group decisions, with distributed responsibility, which is why it’s so important to have institutions like the Civilian Protection Center of Excellence which historically has identified and helped implement the policies which prevent these kinds of disasters, but which unfortunately the current administration decided to shut down without a replacement.
Sometimes, the difference between a successful strike on a lawful combatant and disaster is one analyst pointing out that the building the enemy is hiding in is next to a square that’s used for a market on Tuesdays and wouldn’t you know it, it’s Tuesday. That’s not something the person “pulling the trigger,” can always be expected to know, which is why every ops center needs to be designed to elevate those voices, when the time comes
the operator was following orders...
I would argue `s/pulls the trigger/gives the go ahead/` is intuitively more appropriate. The sailor on the boat that fired off the tomahawk (iirc) was likely not in a position to make an in-the-moment determination
[delayed]
Given that everything emanating from this administration has a political agenda, does this mean that Thiel is on the outs?
[delayed]
> unreleased Pentagon probe
From TFA.
DOGE strikes again. Edit actually looks like it was Hegseth himself, king Testosterone.
> Staffing on civilian harm mitigation teams across the U.S. Department of Defense had also fallen by roughly 90% over recent years, shrinking to fewer than 20 people overall. Centcom’s group shrank from 10 people to one.
https://www.propublica.org/article/hegseth-civilian-harm-mit...
"stupid and energetic, the worst kind of general" - Indy Neidell
Paraphrasing the German General Kurt von Hammerstein-Equord
in reference to a WW1 Austrian General (for others who pass by and are curious, since you seem to be in the know)
https://en.wikipedia.org/wiki/Franz_Conrad_von_H%C3%B6tzendo...
"maximum lethality, not tepid legality."
Despite what AI told them, they knew it was a school full of kids. They still chose to kill them all with bomb. Only maniac would do such thing.
> they knew it was a school full of kids.
I would like to know how you know this