Out-Of-Distribution Checks – How Human Intelligence Stabilises Agentic Workflows

Back to my “Special Theory of Autonomous Agent Reliability” (STAAR). The reliability of a step (a model interaction) in an agentic workflow: R = 1 – (1 – C)(1 – P) Where C is the probability of correct inference, and P is the probability of catching errors before they propagate and compound. If P represents the distribution of anticipated errors – what we test for, what we lint for, what the model knows to look for etc – then, by definition, 1 – P represents out-of-distribution errors – we didn’t write a test for that, the linter doesn’t check for that, the model wasn’t trained on that etc. P is always < 1, and there will always be errors outside the distribution of our automated checks – be they deterministic or probabilistic. So, without checking for out-of-distribution errors, RN will inevitably decay over N steps, even at high values of R. What, or who, could spot unanticipated errors? What can operate outside of its distribution? Humans, that’s what! We can know what we don’t…