Observations From Building a PR Review Loop

Building my own agent has been fun. Sure, I could have opted for an off-the-shelf solution, but I wanted control. The ironic part is that I'm using that control to offload more control to the agent. Recent experiments have included letting the agent work on itself. Super meta, but didn't quite go as planned considering the guardrails I have in place around the size of PRs. Agent work tends to be heavy lifting, whereas most project work is small and fits nicely inside the constraints. Part of the issue there is that the PR reviewer would constantly request changes on those larger PRs. Changes would happen, PR message body would diverge. Infinite loop death spiral until eventually I was called. Around this same time, I was also tracking some issues that started to creep in after I changed some models from Opus 4.8 to Opus 5. Approval rate dropped by about half. I like to think that each new model is going to be better than the last, so I thought perhaps it was time to revisit the…