Stop Letting Your LLM Bill Spiral: Building a Multi-Tenant Gateway in Spring Boot

A team I worked with shipped their first LLM feature in two weeks. Six weeks later, they got a $47,000 OpenAI bill — for a free tier product. The post-mortem found three things: one tenant ran a script that retried failed requests indefinitely, another had a buggy prompt that asked the model to "respond in ten thousand tokens," and a third was just abusive — they had discovered the API key was effectively unlimited and were running batch jobs through it. There was no rate limit. No per-tenant bu