Anthropic prompt caching cut our RCA cost by 90%

Originally published at theculprit.ai/blog/anthropic-prompt-caching-90-percent . LLM costs in production scale faster than the post-mortem of the demo bill suggests they will. The shape of the problem: you ship a feature that calls Claude on every meaningful event. The first month the bill is rounding error and nobody looks at it. The second month a customer's traffic ramps and the line item is suddenly five percent of revenue. The third month your finance person sends a polite Slack about wheth