“The downside is that caching introduces bimodal behavior.”
Prime Day 2021 hit 89.2 million DynamoDB requests per second, and that is only Amazon’s own use. The interesting part is the cache that hit 99.75 percent and was still a hazard. A cold restart would have sent all traffic to the metadata service at once. Amazon built a second system to keep load constant, which is the lesson most teams learn during an outage.