Beyond Raft: Cloudflare Meerkat and Leaderless Consensus

Distributed systems running across the open internet face a fundamental physical reality: the wide-area network is highly unpredictable. Fiber cables get severed, routing tables flap, queue depths spike, and packet loss is a constant companion. For a global edge platform operating across more than 330 data centers, coordinating critical control-plane state under these conditions is an ongoing battle.

Traditionally, systems have relied on consensus algorithms like Raft to keep distributed state in sync. But Raft has a glaring weakness when deployed over wide-area networks (WANs): it is highly dependent on a single leader and strict timeouts. When the network degrades, leaders become unreachable, timeouts trigger, and the system halts to elect a new leader.

To solve this, Cloudflare has introduced Meerkat, an experimental internal consensus service. Instead of relying on Raft, Meerkat is powered by QuePaxa, a consensus algorithm published in 2023 by researchers at EPFL. Meerkat represents a significant architectural shift away from leader-centric consensus toward a multi-writer, timeout-free model designed specifically for the chaos of the WAN.

The Raft Bottleneck on the WAN

To understand why Cloudflare is building Meerkat, you have to look at how traditional consensus engines behave when latency spikes.

In a standard Raft-based system, like etcd, a single node is elected as the leader. All writes must go through this leader. The leader maintains its status by sending periodic heartbeats to follower nodes. If followers do not hear from the leader within a specific timeout window, they assume the leader has failed and initiate a new election.

On a local area network (LAN) inside a single data center, this model works beautifully. Latency is low and predictable, allowing you to set aggressive timeouts of a few milliseconds. But on a global WAN, latency is highly variable. If you set the timeout too low, transient network congestion will trigger accidental elections, causing unnecessary write unavailability. If you set the timeout too high, the system will hang for a long time before realizing a leader has actually crashed.

Cloudflare has admitted to experiencing multiple internal incidents caused by unavailable leaders in consensus-driven systems. When a leader goes offline or becomes isolated by a network partition, the entire control plane can grind to a halt. For services that require linearizability (the guarantee that every read returns the most recent write), waiting for leader election loops to resolve is a major liability.

How QuePaxa Flips the Script

QuePaxa, the algorithm behind Meerkat, takes a fundamentally different approach to achieving consensus. It abandons the single-leader dependency entirely.

In a QuePaxa-based system, all replicas are permitted to perform writes at any time. Because there is no single leader that must be kept alive via heartbeats, the progress of the system is never halted due to a timeout. If a replica or a network link fails, the remaining nodes can continue to agree on the sequence of values without waiting for an election cycle to complete.

Meerkat uses this consensus log to build higher-level abstractions, including a transactional key-value store and a distributed leasing system. It provides linearizability, allowing developers to reason about distributed state as if they were writing to local memory on a single-threaded machine. If a write is acknowledged, any subsequent read anywhere in the world is guaranteed to see that write.

The Developer Angle: What This Means in Practice

For working software engineers, managing distributed state usually means running etcd or Consul. For example, Cloudflare has historically used etcd for distributed cluster leadership consensus and configuration consistency when operating distributed PostgreSQL clusters.

If you have ever managed an etcd cluster across multiple regions, you know the pain of tuning parameters like election-timeout and heartbeat-interval. It is a delicate balancing act. A network hiccup between your US-East and EU-West regions can easily trigger a leader election storm, blocking writes to your configuration store and degrading your entire application.

If the QuePaxa model pans out and eventually makes its way into open-source tools or managed cloud services, it could fundamentally change how we design global applications.

• No More Timeout Tuning: You would no longer need to guess the optimal timeout values for a globally distributed database. The consensus engine would naturally adapt to fluctuating WAN latencies without dropping write availability.
• True Multi-Region Active-Active: Instead of routing all configuration writes to a single primary region (and suffering the associated round-trip latency), developers could write to the nearest local replica with strong consistency guarantees.
• Resilience to Partitioning: In a traditional setup, if the network splits, the minority partition becomes completely read-only or entirely unavailable. A timeout-free, multi-writer system degrades much more gracefully.

However, there are major caveats to keep in mind. Meerkat is currently an internal experiment and remains under active development. Cloudflare is initially targeting it at very small pieces of control-plane state, such as database leadership leases and resource placement metadata.

This is not a drop-in replacement for high-throughput databases. Leaderless consensus algorithms typically trade off write throughput and CPU overhead to achieve their high availability and low latency on WANs. If you are building an application that requires millions of writes per second, a centralized or weakly consistent database is still your only realistic option.

A Realistic Outlook

Meerkat is the first known industrial deployment of QuePaxa at a global scale. It is a bold attempt to bridge the gap between academic consensus research and real-world infrastructure engineering.

For now, developers cannot download Meerkat or use it in their own stacks. But the project serves as an important proof of concept. It proves that the industry is beginning to hit the physical limits of Raft and Paxos at the edge, and that the future of global state coordination lies in leaderless, timeout-free architectures. If Cloudflare successfully stabilizes Meerkat for its internal control plane, expect other cloud providers and open-source projects to quickly follow suit with their own QuePaxa implementations.