Dev Stack, Part XIII: Local AI

This is the latest post documenting my tech stack built around using coding agents safely. The previous posts were: Part I: Intro and Rationale Part II: Linux Part III: Python/Django Part IV: HTMX Part V: VSCode and Copilot Part VI: Bulma Part VII: Sqlite Part VIII: uv Part IX: tooling Part X: networking Part XI: Sandboxing Part XII: Sandboxing update (Docker) Up until this past week, I had been using cloud based AI, but the endgame of this setup was to get onto local AI. In Part II, I mentioned that I moved to Linux on a Framework Desktop. That machine is using an AMD Ryzen AI Max+ with 64GB of unified memory. This is definitely good enough to run some interesting local models, but I do regret not getting a 128GB back in December when I bought it. When prices come down again, I will be upgrading. For hosting models, I use Ollama. I tried a bunch of models. For each one, I gave it a simple command as a way to “eval” it: “Build and run nebulas tests”. I’ll explain more in the next post…