Cloudflare, the internet infrastructure company, changed how its Zone Builder service turns DNS changes into data for its network. The service now receives about 700 record changes per second. Some events, such as zone setting changes, forced a full zone rebuild, and some zones hold more than 20 million records. The team added a planner that picks the smallest build it can prove produces identical output. The company says full builds fell about 40%, peak memory dropped from 79 GiB to 12 GiB, and median build delay fell from about 7.7 to 3.3 seconds.
The trade-offs are explicit. Above a record-count threshold, the system never runs a full build, because one cannot finish; rare cases need manual rebuilds. After a restart, Kafka, the message queue feeding the pipeline, replays work rather than risk losing a change. Every release runs against a mirror that compares output for 20% of zones. The team also found its latency charts overstated tail delays and switched to finer-grained histograms.
Source · google.com ↗
