Your AI, Your Rules: Running a Local LLM with GPU Acceleration on Proxmox

From 3 tok/s frustration to 21 tok/s GPU-hybrid inference - a real engineer's guide to self-hosted AI that actually works. Why Bother Running Local LLMs? Before we get into the how, let's address the obvious question: why not just use Claude, GPT, or Gemini? The honest answer is - for many tasks, you should. But local LLMs make sense when: Privacy matters. Code, internal documents, proprietary configs - none of it leaves your machine. Cost at scale. API calls add up fast when you're running a co