Good catch on CTOs part. That scenario is one that keeps me awake at night -- that some large PR of mine has come functionality or feature that I didn't ask for and didn't notice.
How did the CTO respond to "Claude did it and I didn't catch it"? Did the AI PR summary mention it? Or did the CTO just read the code?
I am fighting against people doing this every day, and we've made it extremely clear that your computer and processes running on it are an extension of you, a human, who is responsible for the actions on your computer.
CTO responded well to it - frankly because we're all using some coding agent and it's just the natural process of code review for them to sniff things out. No mention of it in any PR summary - CTO found it in the migration that would mutate the User record to accept `null` email.
> The reasoning was in a transcript on my laptop. Claude Code deletes those after 30 days by default. Two of my projects lost their whole history that way.
Wait what? Claude Code (and Codex) transcripts are auto-deleted??
That's a treasure trove of information for all my projects and I had built some tooling and workflows tahat deal with extracting insights from them -- withut ever realizing I mightbe losing old chats. Thanks for this -- I may need to include an archival step for retention.
For me this is basically the opposite of how it would need to be, the decisions are all in my messages to the agent, almost never directly in the agent's replies, sometimes indirectly though I suppose.
Interesting to see that apparently some people use these tools and are the ones listening and doing what the LLM decide and says, rather than the opposite.
> Ran a blind eval before shipping, published it including the misses (REPORT.md). Agents with Grepathy answered the "why" questions right. Baseline agents made up confident wrong answers.
Is this eval public anywhere? The whole ecosystem severely lacks transparency, and people continue making strong claims without any sort of data or results that backs these up. Is there any public somewhere backing up the idea that it's more accurate that just using whatever agent harness?
> Claude made a decision nobody approved
It irks me a tiny bit that this is essentially clickbait, you don't know for sure Claude "made a decision nobody approved" as you've lost the data because you never thought sufficiently about storing these things in the first place. Most likely it misunderstood you at one point, you didn't carefully read the reply and approved it all, so most surely what you've done lead to that being done. These big models hardly ever just randomly do whatever, you can more often than not trace it to something you did wrong.
Can we get the transcript of the session where you had Claude write this post? I want to understand the "why" behind this project, and whether a human being was at any point involved in the "blind, pre-registered evaluation of Grepathy against an honest baseline".
Your .ai/why/main.md is at 97KB and 148 entries. Does anything prune that, or does a long-lived branch keep growing?
Good catch on CTOs part. That scenario is one that keeps me awake at night -- that some large PR of mine has come functionality or feature that I didn't ask for and didn't notice.
How did the CTO respond to "Claude did it and I didn't catch it"? Did the AI PR summary mention it? Or did the CTO just read the code?
I am fighting against people doing this every day, and we've made it extremely clear that your computer and processes running on it are an extension of you, a human, who is responsible for the actions on your computer.
CTO responded well to it - frankly because we're all using some coding agent and it's just the natural process of code review for them to sniff things out. No mention of it in any PR summary - CTO found it in the migration that would mutate the User record to accept `null` email.
> The reasoning was in a transcript on my laptop. Claude Code deletes those after 30 days by default. Two of my projects lost their whole history that way.
Wait what? Claude Code (and Codex) transcripts are auto-deleted??
That's a treasure trove of information for all my projects and I had built some tooling and workflows tahat deal with extracting insights from them -- withut ever realizing I mightbe losing old chats. Thanks for this -- I may need to include an archival step for retention.
Well done.
This is not just Claude’s behaviour :)
It could be used for every agentic coding as well, because I have experienced such mess so many times in last three years.
One issue repaired, another one touched and changed…
thank you so much! started with a Claude Code adapter, but you think Codex would be a good next one?
> Decisions only, never your messages
For me this is basically the opposite of how it would need to be, the decisions are all in my messages to the agent, almost never directly in the agent's replies, sometimes indirectly though I suppose.
Interesting to see that apparently some people use these tools and are the ones listening and doing what the LLM decide and says, rather than the opposite.
> Ran a blind eval before shipping, published it including the misses (REPORT.md). Agents with Grepathy answered the "why" questions right. Baseline agents made up confident wrong answers.
Is this eval public anywhere? The whole ecosystem severely lacks transparency, and people continue making strong claims without any sort of data or results that backs these up. Is there any public somewhere backing up the idea that it's more accurate that just using whatever agent harness?
> Claude made a decision nobody approved
It irks me a tiny bit that this is essentially clickbait, you don't know for sure Claude "made a decision nobody approved" as you've lost the data because you never thought sufficiently about storing these things in the first place. Most likely it misunderstood you at one point, you didn't carefully read the reply and approved it all, so most surely what you've done lead to that being done. These big models hardly ever just randomly do whatever, you can more often than not trace it to something you did wrong.
> I didn't know! I didn't make that decision!
Thanks for today's dose of impostor syndrome cure.
Can we get the transcript of the session where you had Claude write this post? I want to understand the "why" behind this project, and whether a human being was at any point involved in the "blind, pre-registered evaluation of Grepathy against an honest baseline".