LuaJIT is a better LLM runtime than Python

The title is clickbait, forgive me. You clicked, time to read :D Not really sure how to start this blog post, so I'm just gonna wing it. I have 32 GB of RAM, a Ryzen 9 9950X and an RTX 3060. Not enough to run 14B in Q8 without CPU offload, and definitely not enough to let Python eat half my memory before the model is even loaded. I've been writing Lua for nearly 14 years, and I have a strong taste for tools that don't leave a huge footprint on my system: few packages, little RAM, little noise. P