The Future of Android is Local: How to Run Custom LLMs (Llama, Gemma) On-Device with MediaPipe and Kotlin

For years, the promise of Large Language Models (LLMs) in the mobile ecosystem has been tethered to the cloud. We’ve treated these powerful models as remote black boxes, accessed through REST APIs and hidden behind paywalls. While this "Cloud-Centric" approach allowed us to tap into the power of GPT-4 or Claude, it came with a heavy price: high latency, a mandatory internet connection, and significant privacy concerns. For developers, it meant unpredictable API costs and the constant risk of dat