How to Convert GGUF to MLX (and When You Should Not)

Short answer: There is no direct GGUF to MLX converter. mlx_lm.convert only reads Hugging Face safetensors, so pointing it at a .gguf file fails. What to do instead, in order of preference: Download an MLX build that already exists. Most popular models have one on mlx-community. Convert the original weights, not the GGUF. mlx_lm.convert --hf-path <original-repo> -q takes a few seconds. If a GGUF really is all you have, dequantize it with transformers and then convert. Take config.json and the tokenizer from the original repo, not from the GGUF. If you skip that, a Llama 3 model converts without any error and falls apart on long prompts. The script is below. The rest of this post shows why, with the exact commands and what each path costs in quality. I tested everything on an M4 Pro, and the versions are listed at the end. In my GGUF vs MLX decision guide I argued that GGUF is the safer default and MLX is worth it for specific workloads. The search data for that post shows the…