Trying out open weights LLM-s on hardware that I can actually afford 0 ▲ ./techtipsy 77 days ago · 16 min read3128 words · Tech · hide · Fediverse post I’ve been aware of open weights local language models for over a year. They have gone from being a fun but useless novelty to something that with the right harness and hardware could get closer to becoming usable for day-to-day work. Not at the level of something like Opus 5, sure, but usable enough for doing useful work without a lot of headaches. After a Friday evening e-mail from a government across the ocean resulted in a (allegedly super capable) model being turned off, I was instantly motivated to give open weights models a try on hardware that I can actually afford. Local LLM crash course If you’re unfamiliar with hosting local large language models, then a lot of this may seem like gibberish. Don’t worry, I had the same reaction initially. With trial, error, and a lot of guidance, I kind of know what to keep in mind when playing in this area, so here’s a quick crash course on common terms and gotchas. Large language model: it’s just a file on your disk that contains the… Commenting on Bubbles has been removed. Why? If you want to talk about this post, write to the author, use the comments on their blog if it has them, or reply with a "RE:" post on your own blog. Suggestions? feedback@bubbles.town