How I Took Down Prod With a 400ms Migration (And...

How I Took Down Prod With a 400ms Migration (And The Playbook I Use Now) The 3 AM ALTER TABLE That Ruined My Weekend It was a regular Tuesday deploy. The Jira ticket was straightforward: just add a foreign key constraint linking orders to customers. The table had 50 million rows, but I had explicitly tested it against a staging dump. It took exactly 400 milliseconds. "It works locally and on staging. Ship it," I confidently told my team lead. What I totally missed was the underlying database loc