Context complexity: what is the Big-O of an agent API?

TLDR; small, local models are a good instrument for measuring what a harness costs an agent. an API is only as good as its outputs’ specificity, parsimony, and truthfulness. obvious code repair is cheaper done in your tools than elicited from the model through another retry, provided the tool discloses what it changed. Earlier this year I saw a post comparing the token efficiencies of different programming languages. The post itself wasn’t a thorough treatment of the issue but it did create a pretty interesting discussion around what makes some languages better for LLMs than others. I’ve also seen assertions flying around online claiming flavours of the following: a language with a good compiler is perfect for LLMs because the compiler acts as a correctness oracle of sorts Python is the best programming language to use for AI since there are so many examples of it terse/information dense languages are better for LLMs since, compared to verbose languages, it takes fewer tokens to…