vLLM on Google Cloud TPU: A Model Size vs Chip Cheat Sheet (With Interactive Tool)

Picking a Cloud TPU slice for vLLM inference involves three decisions that most tutorials skip over: how much HBM your model actually needs at runtime, how many chips to shard across, and whether the cost is justified for your workload. Get it wrong in either direction and you're either OOMing on startup or paying for memory you're not using. This post walks through how to make that decision, with a reference table for popular models and a live interactive tool where you can select your model, t