The Wall Confronting Reliable Coding Agent Autonomy

It’s a classic mistake. We develop a technology in N years for X dollars that’s 90% reliable, and extrapolate that traversing that final 10% will only take another 0.1N years and 0.1X dollars. This leads to the kinds of extraordinary predictions about fully autonomous self-driving vehicles or coding agents being just around the corner. We see agents build, say, a web browser that kind-of sort-of works and extrapolate that it must surely only be months before they can build browsers that *really* work. But folks are looking at the wrong %. To take agents from 90% reliability to 99% reliability isn’t an improvement of 9%, it’s a 10x improvement. Those of us who’ve worked on high-integrity software will know that the journey from 90% to 99% reliability is just as hard as it was to get to 90%. And from 99% to 99.9% is just as hard again. In his famous article Software Quality at Top Speed, Steve McConnell includes a graph showing engineering effort vs. defects caught and fixed before…