I feel like this will become something that runs in the provider backend and where users can't turn it off. Esp with all the current-day security signal. They don't care about Ada's model deleting her drive. They want to ensure Ada's Claude Code doesn't hack a merchant trying to get a refund.
If regulation requires this type of thing - there are possible implications for any form of alternative/open-source harness - as there's fairly tight integration with being able to monitor intent effectively & how harnesses shape session data. Def a bear signal for any ai monitoring cyber startup.
This is a duplicate of https://news.ycombinator.com/item?id=49220827
I feel like this will become something that runs in the provider backend and where users can't turn it off. Esp with all the current-day security signal. They don't care about Ada's model deleting her drive. They want to ensure Ada's Claude Code doesn't hack a merchant trying to get a refund.
If regulation requires this type of thing - there are possible implications for any form of alternative/open-source harness - as there's fairly tight integration with being able to monitor intent effectively & how harnesses shape session data. Def a bear signal for any ai monitoring cyber startup.
Stated my full thoughts around this & sandboxing here: https://xcancel.com/backnotprop/status/2085801458486911128?s...