> Guardrails matter here: mandatory human authored test specs (in partnership with an experienced engineer), strict code review, and explicit reasoning requirements (“explain why this change is correct”) help prevent silent failure modes.
The third of these (reasoning requirements) is the most reasonable to me. The others require that review capacity scales to match the capacity of the junior to produce garbage, which capacity LLMs now make unlimited.
IMO the onus must be on the author to produce an explanation that matches their change and makes sense (even if it’s wrong), and until they do the reviewer must be able to just say “nope, try again” as many times as necessary. It’s sad because thoughtful feedback is the best way to learn, but bad actors using LLMs have decided to skip learning and so should not waste the time of a reviewer who could be helping someone else.
> Making high-level architectural decisions
> Understanding unfamiliar codebases quickly
> Debugging systems you didn’t write
> Reasoning about scale and failure modes
> Keeping systems secure
> When you look at that list, it reads a lot like a Staff+ job description
I did most of my career in a backwater without big tech and I only heard of the “staff+” label a few years ago. I assumed it meant something fancy until I heard descriptions of what these people do; like this one, which to me just sounds like “competent experienced programmer”. 5 years ago in my backwater it was pretty normal to meet very good “senior developers” with 20 years of experience who could do all this and more, because “senior” wasn’t a phase, it was the IC career destination.
I find that often tasks and code which don’t match the checklist of cases where agents are useful can be broken down into tasks that do match the criteria. Sometimes when the context is wrong for the tool we are guilty of assuming that there’s nothing that can be done to change the context.
> Guardrails matter here: mandatory human authored test specs (in partnership with an experienced engineer), strict code review, and explicit reasoning requirements (“explain why this change is correct”) help prevent silent failure modes.
The third of these (reasoning requirements) is the most reasonable to me. The others require that review capacity scales to match the capacity of the junior to produce garbage, which capacity LLMs now make unlimited.
IMO the onus must be on the author to produce an explanation that matches their change and makes sense (even if it’s wrong), and until they do the reviewer must be able to just say “nope, try again” as many times as necessary. It’s sad because thoughtful feedback is the best way to learn, but bad actors using LLMs have decided to skip learning and so should not waste the time of a reviewer who could be helping someone else.
> Making high-level architectural decisions > Understanding unfamiliar codebases quickly > Debugging systems you didn’t write > Reasoning about scale and failure modes > Keeping systems secure
> When you look at that list, it reads a lot like a Staff+ job description
I did most of my career in a backwater without big tech and I only heard of the “staff+” label a few years ago. I assumed it meant something fancy until I heard descriptions of what these people do; like this one, which to me just sounds like “competent experienced programmer”. 5 years ago in my backwater it was pretty normal to meet very good “senior developers” with 20 years of experience who could do all this and more, because “senior” wasn’t a phase, it was the IC career destination.
What happened?
I find that often tasks and code which don’t match the checklist of cases where agents are useful can be broken down into tasks that do match the criteria. Sometimes when the context is wrong for the tool we are guilty of assuming that there’s nothing that can be done to change the context.