Human vs. Agent Reliability Over Long Horizons – How Can We Do What They Can’t?

Multiple studies find that – without exception – the reliability of autonomous agent workflows decays geometrically over many steps. There are no examples of complex, reliable software created autonomously by agents – or anything even close. On a per step basis, humans are – of course – equally unreliable. And yet, somehow, there are many examples of complex, reliable software created by humans. Despite being just as fallible, humans are able to stabilise reliability in a way LLM-based agents can’t – despite all our best attempts to make them “self-correcting” or “self-healing”. I think the missing piece of the jigsaw lies in my simplified model of the reliability of a step in development workflows: R = 1 – (1 – C)(1 – P) Where C is the probability of it being correct, and P is the probability of any errors being caught before they propagate and compound – before the wheels start to wobble. I think what we’re looking for is in 1 – P. If P represents the distribution of errors that get…

Commenting on Bubbles has been removed. Why? If you want to talk about this post, write to the author, use the comments on their blog if it has them, or reply with a "RE:" post on your own blog. Suggestions? feedback@bubbles.town