Back to Basics: Build Your Own LLM from Scratch

I did a workshop titled “Back to Basics: Build Your Own LLM from Scratch”, which kind of included some basic theory on how a transformer works, and then building a very small LLM. The idea was to demystify an LLM (or transformer) by understanding what goes on and then building one to deepen our understanding. I had to skip some slides because the planned session was only two hours. Ideally, I want it to be around 4 hours, split into 2 sessions: one for theory and one for lab. Maybe next time, when I plan, I will make it 4 hours so I can do it at a slower pace. Of course, there are other similar workshops available online, and some of them are linked in the references section. This is just my take on it and what I used for my own understanding. Suppose you want to try it at your own pace. Try the slides below and then use annotated code to read and run. Slides and code are in a repo too if you prefer that. #!/usr/bin/env -S uv run --script # /// script # requires-python = ">=3.10" #…