Edge LM Devlog: Running a language model on a Raspberry Pi 5

Working on a Raspberry Pi (RPi) feels like a trip back to memory lane. My undergraduate degree was in electronics engineering, and I still remember my final year project which involves an RPi.1 Now that I’m in the field of NLP, I’m quite curious how we can fit large language models into these small devices. In this blog post, I document my journey in running a language model in a Raspberry Pi 5! First, I want to lay down the price list: Raspberry Pi 5 16 GB RAM (230 GBP): it was my birthday so I had money to spare. Kidding aside, I chose to max out on specs because I want to measure the ceiling in which I can deploy language models. To the best of my knowledge, this is the largest you can buy unmodded. 32 GB microSD Card (27.50 GBP): I chose 32 GB because I want to work on a size that is a bit constrained, but not too limiting that I can’t store a mid-sized model. Active Cooler (4.80 GBP): I often see this recommended in all the tutorials I found and I’m glad I bought it. You actually…