105 days ago · Tech · hide · Fediverse post

I got curious about how DeepSeek actually works. Not the model architecture. That part is well documented. I wanted to understand what happens when you hit their API. How does the server receive a request, queue it, load the model, run the forward pass, stream tokens back. The whole pipeline. So I started from first principles. If I had to build my own inference server, where would I even start. What would the request lifecycle look like. How do you batch requests. How do you manage GPU memory. What happens when the model is too big for a single GPU. I assumed my laptop was enough. Not for running GPT-4 at scale. But for understanding the system. For proving to myself that I could write a server that receives a prompt, runs it through a model, and streams tokens back. That curiosity turned into a page: tools/inference.html. But something else happened along the way. I was not just learning how inference works. I was using DeepSeek itself to learn it. Asking it to explain tokenizers.…

Commenting on Bubbles has been removed. Why? If you want to talk about this post, write to the author, use the comments on their blog if it has them, or reply with a "RE:" post on your own blog. Suggestions? feedback@bubbles.town