62 days ago · Tech · hide · Fediverse post

Alright, so if you’ve read the title and thought you were having a stroke, you might not be the target audience for this post. If, however, you thought “more tokens/s?”, you should keep on reading. If you remember my review of the new Lenovo X1 Carbon Gen 14 Aura with Intel Panther Lake Core Ultra X7 368H vPro from a while ago, you might recall that I had tested its local “AI” performance using Ollama, via Vulkan, which obviously didn’t perform particularly well across various models. The proper way to run local LLMs is to use Intel’s official oneAPI framework and compile llama.cpp with SYCL enabled, which will greatly increase performance. This post is a very brief write-up of how that can be done and primarily serves as yet another documentation for future-me. Gentoo I’m assuming that you’re already running your Gentoo system with the xe driver and VIDEO_CARDS="intel". First, install the necessary dependencies: emerge -av dev-libs/intel-compute-runtime dev-libs/level-zero \…

Commenting on Bubbles has been removed. Why? If you want to talk about this post, write to the author, use the comments on their blog if it has them, or reply with a "RE:" post on your own blog. Suggestions? feedback@bubbles.town