An inference journey

I got curious about how DeepSeek actually works. Not the model architecture. That part is well documented. I wanted to understand what happens when you hit their API. How does the server receive a request, queue it, load the model, run the forward pass, stream tokens back. The whole pipeline. So I started from first principles. If I had to build my own inference server, where would I even start. What would the request lifecycle look like. How do you batch requests. How do you manage GPU memory. What happens when the model is too big for a single GPU. I assumed my laptop was enough. Not for running GPT-4 at scale. But for understanding the system. For proving to myself that I could write a server that receives a prompt, runs it through a model, and streams tokens back. That curiosity turned into a page: tools/inference.html. But something else happened along the way. I was not just learning how inference works. I was using DeepSeek itself to learn it. Asking it to explain tokenizers.…