The Model Server 0 ▲ Internals for Interns 46 days ago · Tech · hide · Fediverse post Over the last two articles we went through everything needed to run a model: an SDK that runs inference, and a management layer that downloads models, sizes them for your hardware and loads them. You can use the SDK directly from your Go code, but one of the most popular ways of consuming LLMs is through an HTTP API, and that’s what’s left: the server that lets your existing OpenAI client talk to all of it. Commenting on Bubbles has been removed. Why? If you want to talk about this post, write to the author, use the comments on their blog if it has them, or reply with a "RE:" post on your own blog. Suggestions? feedback@bubbles.town
