↓ Skip to main content

“During the roughly 3.5 hours that these services were impacted, about 55% of the logs we normally send to customers were not sent and were lost.”

Jamie Herre, Tom Walwyn, Christian Endres, Gabriele Viglianisi, Mik Kocikowski and Rian van der Merweblog.cloudflare.com ↗

A bug pushed a blank config, and a failsafe built back when traffic was small failed open and sent logs for every customer. Buftee jumped from about 1 million buffers to 40 million and locked up. The limits meant to stop exactly that existed but were never configured. Customers who pay for logs for compliance now have a 3.5-hour hole.