Why Your Gemma 4 Fine-Tuning is Failing (and How to Fix It)

TL;DR: Gemma 4 is a multimodal beast with an Apache 2.0 license, but its new ClippableLinear layers and dynamic image tokens will break standard LoRA scripts. Use target_modules="all-linear" and backward-search masking to hit 94%+ accuracy on Cloud Run. Gemma 4 has officially landed, and with an Apache 2.0 license and a 356K context window, it’s the new king of open-weight models. But if you try to drop it into your old Gemma 3 or Llama scripts, it will fail. I’ve been deep-diving into the archi