Out-Of-Distribution Checks – How Human Intelligence Stabilises Agentic Workflows 0 ▲ Codemanship's Blog 65 days ago · Tech · hide · Fediverse post Back to my “Special Theory of Autonomous Agent Reliability” (STAAR). The reliability of a step (a model interaction) in an agentic workflow: R = 1 – (1 – C)(1 – P) Where C is the probability of correct inference, and P is the probability of catching errors before they propagate and compound. If P represents the distribution of anticipated errors – what we test for, what we lint for, what the model knows to look for etc – then, by definition, 1 – P represents out-of-distribution errors – we didn’t write a test for that, the linter doesn’t check for that, the model wasn’t trained on that etc. P is always < 1, and there will always be errors outside the distribution of our automated checks – be they deterministic or probabilistic. So, without checking for out-of-distribution errors, RN will inevitably decay over N steps, even at high values of R. What, or who, could spot unanticipated errors? What can operate outside of its distribution? Humans, that’s what! We can know what we don’t… Commenting on Bubbles has been removed. Why? If you want to talk about this post, write to the author, use the comments on their blog if it has them, or reply with a "RE:" post on your own blog. Suggestions? feedback@bubbles.town