“We went from thousands, or hundreds of thousands, of stale WireGuard peers on a gateway to what rounds to none.”
Fly.io pushed a WireGuard peer to a gateway every time flyctl ran, and nothing ever cleaned them up. CI jobs minted a fresh peer on every run, so gateways hoarded hundreds of thousands of dead configs, crawled on reboot and hit kernel panics. The provisioning ran over NATS, which Fly admits was losing too many messages to trust. The fix sniffs handshake packets and installs peers on demand, which is how it should have worked from day one.