llama.cpp

llama.cpp is a LLM inference framework written in C++ and primarily meant for edge computers like laptops. Install Install using brew: $ brew install llama.cpp Run a LLM Download and run a HuggingFace model: $ llama-cli -hf ggml-org/gemma-3-1b-it-GGUF