How I run LLMs locally on Mac

Harness OpenCode & oh-my-pi come with batteries included and works out of the box. Pi is barebones & lightweight. There are quality plugins available to extend the functionality & customise the behaviour. It is extremely extensible. Inference Server Ollama MLX doesn't generate tokens efficiently. Rapix-MLX, omlx are excellent at utilizing resources and generating tokens efficiently. With M5 Pro 48GB RAM, they are able to generate ~50 tokens/second with Qwen3.6-35B-A3B-4bit model. Llama.cpp is cross-platform and it is extremely fast as well. Local LLMs Qwen3.6-35B-A3B model made locall LLMs usable on decent hardware. Qwen3.6-35B-A3B - has 35B parameters & only 3B are active at a time. I use 4bit quantized model as it provides a better tradeoff between speed and accuracy. It fits in 48GB RAM with some headroom for other applications. Bonsai has highest intelligent densisty and it can run on ~5 GB RAM. It is surprising how fast & powerful the local LLMs have become.