Squeezing Open Models At Scale 0 ▲ Tao of Mac 110 days ago · Tech · hide · Fediverse post This is the kind of inference optimisation work I have been doing in go-pherence and my personal llama.cpp fork, but at Cloudflare scale. I have been trying to squeeze more useful models into the hardware I already own; they are trying to squeeze more requests into a global GPU fleet, and many of the trade-offs are the same. I expect “mid-tier” open-weight models to become much more popular throughout the rest of the year as optimisations like these make them cheaper to serve, hopefully putting a ceiling on OpenAI and Anthropic pricing and forcing some competition. I would like this to help burst the AI bubble, but OpenAI and Anthropic’s fantasy pricing is hardly the only thing keeping it aloft. Commenting on Bubbles has been removed. Why? If you want to talk about this post, write to the author, use the comments on their blog if it has them, or reply with a "RE:" post on your own blog. Suggestions? feedback@bubbles.town