How to fit Qwen3.8-27B into 16GB of VRAM and run it on a single RTX 3080 card: the best quantizations and Llama.cpp flags I've found

Despite being a notorious AI skeptic, when Qwen3.8-27B came out, I spent several afternoons trying to get it to run on a 16gb Nvidia card with halfway decent results.Eventually I did, and this is how.Note: This is the sequel to the nearly-identical saga of trying to get Qwen3.6 to run on the same hardware.Step 1: Lower Your ExpectationsThe cool kids are all running dual RTX PRO 6000 Blackwells or DGX Sparks, or, if they’re really impoverished, AMD Strix Halo with 128gb unified memory.The average HackerNews reader seems to run inference on their spare M6 MacBook with 960gb of RAM.I just bought a new old laptop, with four times the GPU of my last one: 16gb of VRAM seemed like a dream to someone who couldn't even run models above 14b, let along run them with all layers on the GPU. I soon learned that, actually, no, the dream life begins at 32gb of VRAM, and that on the internet, if you use "16gb" in the same sentence as "local AI", expect pity/sniggering/incredulity, and recommendations…

Commenting on Bubbles has been removed. Why? If you want to talk about this post, write to the author, use the comments on their blog if it has them, or reply with a "RE:" post on your own blog. Suggestions? feedback@bubbles.town