I’m slowly coming around to the idea that if it works, I’m good with that. I never used to personally check all my colleagues work, either, I just trusted that they’d done a good job.
That’s not to say I blindly trust an LLM - my project is an autovectorising compiler. I have it write a bazillion tests, and I have real, non-trivial programs that it compiled. I have it run a ‘gate’ before releases that takes literally hours on 5 different architectures. To say this code is tested is … masterful understatement.
I think what helps is a long planning session, going into details, working out corner cases, designing test-cases, you know work and effort. Once that’s done it’s easier to let it work through the night and do a major production feature.
I still review the code, but hand on heart I couldn’t say I’d reviewed everything, sometimes I miss something in a complicated check-in. I try to keep checkins small and contained, but sometimes that’s not easy to manage alongside the “don’t break stuff” rule as well.
So it comes down to: I’ve been working on this for 6 months or so, and so far this has worked out for me. You know, kind of like getting used to a colleague’s idiosyncrasies over a period of time.
I will say that in my experience the code quality is fairly high. Maybe that’s down to the effort put in ahead of time in the planning stages, I don’t know, but I don’t seem to suffer from the ‘slop’ that people complain of. YMMV.
I am not in META but for what I read on blind, assuming it is true, they are not spending any time in long planning sessions or in getting into details of the implementation plan at all, and they could not anymore considering how many PRs in parallel they are requested to manage
wow, “quality is no longer an objective” is a wild idea that feels dangerously negligent.
quality is all we have, it’s what makes us different from these agents.
quality is perplexing to agents because it requires a strange loop into asking why. there’s not always a clear answer. but i think the power is in the
ambiguity.
if we don’t know why (or don’t care) then what happens if we all adopt this attitude at scale?
The entire industry is dangerously negligent, we just have better tools and stronger incentives now. Why spend 3h code-reviewing a PR when you know if it breaks you just tell your agent to watch out for that specific scenario? Which executive will see that as time well spent?
Sort of. What if you asked your second agent to fuzz test the work of the first agent, adversarial code review, or make a video to explain all the code.
I had same problem, but after a while I just accepted that is inevitable
I’m slowly coming around to the idea that if it works, I’m good with that. I never used to personally check all my colleagues work, either, I just trusted that they’d done a good job.
That’s not to say I blindly trust an LLM - my project is an autovectorising compiler. I have it write a bazillion tests, and I have real, non-trivial programs that it compiled. I have it run a ‘gate’ before releases that takes literally hours on 5 different architectures. To say this code is tested is … masterful understatement.
I think what helps is a long planning session, going into details, working out corner cases, designing test-cases, you know work and effort. Once that’s done it’s easier to let it work through the night and do a major production feature.
I still review the code, but hand on heart I couldn’t say I’d reviewed everything, sometimes I miss something in a complicated check-in. I try to keep checkins small and contained, but sometimes that’s not easy to manage alongside the “don’t break stuff” rule as well.
So it comes down to: I’ve been working on this for 6 months or so, and so far this has worked out for me. You know, kind of like getting used to a colleague’s idiosyncrasies over a period of time.
I will say that in my experience the code quality is fairly high. Maybe that’s down to the effort put in ahead of time in the planning stages, I don’t know, but I don’t seem to suffer from the ‘slop’ that people complain of. YMMV.
I am not in META but for what I read on blind, assuming it is true, they are not spending any time in long planning sessions or in getting into details of the implementation plan at all, and they could not anymore considering how many PRs in parallel they are requested to manage
The solution is more AI. Throw another agent in to watch your logs, then throw in another agent to watch that agent.
Quality is no longer an objective. If it breaks, rebuild it and tell it not to break.
wow, “quality is no longer an objective” is a wild idea that feels dangerously negligent.
quality is all we have, it’s what makes us different from these agents.
quality is perplexing to agents because it requires a strange loop into asking why. there’s not always a clear answer. but i think the power is in the ambiguity.
if we don’t know why (or don’t care) then what happens if we all adopt this attitude at scale?
The entire industry is dangerously negligent, we just have better tools and stronger incentives now. Why spend 3h code-reviewing a PR when you know if it breaks you just tell your agent to watch out for that specific scenario? Which executive will see that as time well spent?
Sort of. What if you asked your second agent to fuzz test the work of the first agent, adversarial code review, or make a video to explain all the code.
It's agents all the way down!
<that scene from the Matrix with hundreds of Agent Smith>.gif