Cloudflare, the internet infrastructure company, is testing a new way to keep the servers behind its internal control plane, which coordinates settings across its global network, in agreement. Consensus algorithms such as Raft rely on a single leader server. When the leader slows down, writes stall. In 2020, a partly failed network switch made one such system's dashboard 80x slower for six hours and blocked writes. Cloudflare built Meerkat, an experimental service based on QuePaxa, an algorithm where any replica can propose and clients send requests to every replica.
The company says over 95% of decisions finish in one network round trip through the leader. In one leader failure, the decision rate fell to about 65% of normal and recovered within a minute, without halting. The cost is extra network traffic and slower decisions while the leader is down. The authors concede QuePaxa is far less battle-tested than Raft and plan formal verification and fault-injection testing.
