Inference Cards

Why skip past the why When someone says “I run Qwen 3.6 at 25 tokens per second”, or makes any similar performance claim about their self-hosted LLM setup, this is only meaningful if we know several other things. Which model variant? Qwen 3.6 could be the dense 27B or the 35B-A3B MoE, totally different architectures. Better, just link to the repo you downloaded the weights from. Which quantization? Q8, Q4_K_XL, and IQ3_XXS are at different points in quality/speed/size space. What hardware and inference engine? Is this vLLM on an H100 or llama.cpp on a Raspberry Pi? I could ask if you’re using speculative decoding or various weird stuff, but better, show the command you used to invoke the inference engine. How did you measure the speed? Are you exercising an HTTP API (which adds a possibly-large chat template in the context window), or using something like llama-bench (which skips the template and HTTP/network delay)? Or better, what command did you use to run the test? Also, knowing…