What is Inference?

Welcome to a new series! For most developers today, using a large language model means one thing: an HTTP call to somebody else’s computer. You send a prompt to an API, tokens come back, and everything in between is somebody else’s magic. But here’s what I find much more interesting: you can run these models locally, inside your own process, on your own hardware — and if you’re a Go developer, you can do it directly from Go. Two projects make this genuinely pleasant today: Yzma, which lets Go call the llama.cpp libraries directly (without cgo — that gets its own article later in the series), and Kronk, which builds a high-level, OpenAI-API-feeling SDK and model server on top of Yzma. And underneath both of them sits llama.cpp, the C/C++ inference engine that made running LLMs on ordinary hardware practical.