When DynamoDB Global Tables Go Stale: Chaos Testing Replication Lag with AWS FIS

DynamoDB Global Tables replicate data across regions in seconds, but replication is still asynchronous. That means a simple read from a replica region can occasionally return stale data, which is acceptable in most application as the user doesn’t require the latest available data all the time, but in some systems, stale reads can break important processes and stability of a platform. So the question becomes: How do you detect and react to stale replica reads automatically? So, I set out to find