On local inference

I woke up at 2 a.m. because the fans were screaming. The sound was different from the soft whoosh of a GPU under load. It was the high-pitched panic of a model that had swallowed its own context and was eating the swap file. I sat up for a second, then walked to the other room and killed the process. The house was quiet again. I went back to bed angry but not surprised. That is local inference now. The romance of rebellion against the cloud wore off long ago, now it's mostly quiet labor of checking on a sick animal at night. For a couple of years it was a hobby. I downloaded weights the way some people "download" vinyl. I ran quants I barely understood and felt clever when a reply came back. Then agents arrived, and the model stopped being a toy I visit. It became the room my work lives in. OpenCode drafts here and Hermes listens here. When it breaks, the damage is immediate, my notes stop, my drafts stall, ideas do not validate themselves, and I have to decide whether to fix it now…