11 hours ago · Tech · hide · 0 comments

An LLM serving simulator drained all four GPU replicas at its first autoscaler tick, before any request arrived. 0 of 200 requests completed. The repro, the root cause, the fix, and how real autoscalers set a floor.

No comments yet. Log in to reply on the Fediverse. Comments will appear here.