Human vs. Agent Reliability Over Long Horizons – How Can We Do What They Can’t?

Multiple studies find that – without exception – the reliability of autonomous agent workflows decays geometrically over many steps. There are no examples of complex, reliable software created autonomously by agents – or anything even close. On a per step basis, humans are – of course – equally unreliable. And yet, somehow, there are many examples of complex, reliable software created by humans. Despite being just as fallible, humans are able to stabilise reliability in a way LLM-based agents can’t – despite all our best attempts to make them “self-correcting” or “self-healing”. I think the missing piece of the jigsaw lies in my simplified model of the reliability of a step in development workflows: R = 1 – (1 – C)(1 – P) Where C is the probability of it being correct, and P is the probability of any errors being caught before they propagate and compound – before the wheels start to wobble. I think what we’re looking for is in 1 – P. If P represents the distribution of errors that get…