On my last project, it kept trying to use the system python instead of the project's virtual environment. It also kept using the wrong build tool. Both things wasted considerable tokens because the agent got sidetracked trying to understand why it could not run the tests - and that repeated on each new session.
It mentions that a chatbot generated AGENTS.md does nothing, which makes sense.
I added one when it kept making the same mistake and using things from the wrong library version making compiler errors, adding in the common mistakes pre emptively.
Just because it doesn't materially impact task success rates does not mean it's not useful. I use that file to give my agent information about the environment (operating system, architecture, command-line utilities I have installed), as well as how to do certain things (such as using `uv` for running Python when needed). I want my agent to work how I do, so I also tell it things like my preferred version control system (jj), my preferred implementation languages for things like shell scripts (zsh), and other things like that. I care more about how the work is done than the final result. It still slips up sometimes, but on the whole I think it works alright.
On my last project, it kept trying to use the system python instead of the project's virtual environment. It also kept using the wrong build tool. Both things wasted considerable tokens because the agent got sidetracked trying to understand why it could not run the tests - and that repeated on each new session.
A simple instruction in AGENTS.md fixed that.
It mentions that a chatbot generated AGENTS.md does nothing, which makes sense.
I added one when it kept making the same mistake and using things from the wrong library version making compiler errors, adding in the common mistakes pre emptively.
Just because it doesn't materially impact task success rates does not mean it's not useful. I use that file to give my agent information about the environment (operating system, architecture, command-line utilities I have installed), as well as how to do certain things (such as using `uv` for running Python when needed). I want my agent to work how I do, so I also tell it things like my preferred version control system (jj), my preferred implementation languages for things like shell scripts (zsh), and other things like that. I care more about how the work is done than the final result. It still slips up sometimes, but on the whole I think it works alright.