A Local LLM Infrastructure 0 ▲ Zit Seng's Blog 89 days ago · 6 min read1132 words · Tech · hide · Fediverse post I built an LLM API gateway. Between that and all the infrastructure modernisation I’ve been doing at work, the months have blurred. I’ve been really busy, and there was never a good moment to write. But this is really exciting. So here goes. The project is called SoCLaaS. Nevermind the name — it means what it means. It’s our LLM infrastructure so that we can make LLM API service broadly available free-of-charge to our NUS School of Computing community, scalable for volume consumption without easily falling apart from overload or contenton. SoCLaaS begun as an experiment with Codex. I was exploring some ideas and thought to give Codex some work. That experiment grew into what it is today. It’s not at all a small project as it stands now. If not for AI, I estimate it would have taken 3 experienced developers 3 months to complete. Codex itself estimates 18 man-months. It’s amazing what we can do with AI. In a nutshell, SoCLaaS gives developers one stable OpenAI-compatible API while… Commenting on Bubbles has been removed. Why? If you want to talk about this post, write to the author, use the comments on their blog if it has them, or reply with a "RE:" post on your own blog. Suggestions? feedback@bubbles.town