Making hundreds of Postgres nodes look like one

A single Postgres primary hits a wall long before marketing slides admit it. CPU and IOPS saturate. The write-ahead log becomes a shared choke point. Backups of a multi-terabyte monolith drag on for hours or days. Read replicas buy time, sometimes a lot of it, but they do not grow storage and they do not multiply write throughput. At some point the only proven path past a few terabytes is to split the data across many independent primaries.

That split is the easy sentence to write. The hard product is the control and routing layer that makes hundreds of those primaries behave like one database to every application connection. Without it, ops runs a fleet and developers maintain a graph of connection strings, shard maps, and special-case query paths. The interesting engineering is not the shard math. It is the abstraction that erases the fleet from application code.

Where vertical scale and replicas stop

Most applications start with the classic three-tier picture: clients, app servers, one database. Scale the database box (more cores, more RAM) and add read replicas when SELECT traffic climbs. Writes stay on the primary; replicas stream WAL changes and serve reads. That pattern is well understood and still the right first move.

It has hard limits. Writes remain serialized through a single primary’s WAL and durable flush path. No number of read-only replicas removes that bottleneck. Replicas also copy the entire dataset, so they multiply read capacity without increasing total storage. Backups of a large monolithic data directory stay bound by the bandwidth of one node talking to object storage. Organizations that need tight recovery objectives eventually find those windows unacceptable.

The Universal Scalability Law describes the same curve in other words: contention makes scaling sub-linear, and past a point incoherence makes more resources hurt. Postgres is not exempt. Extreme replica counts exist in production (one public example is dozens of replicas off a single primary), but they are still vertical-plus-read-scale designs. They do not distribute data ownership.

Sharding as capacity, not elegance

Sharding distributes both data and query load across many distinct primaries. A few-terabyte workload might land on four shards of roughly 500 GB each, each handling a fraction of traffic. A petabyte-scale system can require far more: hundreds of shards, each sized in the low terabytes, each with its own primary and a small replica set for availability. One concrete configuration is 256 shards with a primary plus two replicas apiece—768 servers total, each responsible for on the order of 4 TB.

That arithmetic is straightforward. The operational surface is not. Every shard is a full Postgres instance with its own connections, backups, failovers, and upgrade cadence. Cross-shard joins, transactions that touch multiple keys, and global secondary indexes become first-class problems instead of free features. Resharding when a key range or hash bucket grows unevenly is a migration project, not a config tweak.

None of that is new. What has changed is how much of the graph can be pushed out of application code and into a proxy and metadata plane that presents one logical database.

The proxy is the product

A fleet-wide abstraction typically sits between app servers and the shard primaries (and their replicas). Clients open a normal Postgres connection to a proxy endpoint. The proxy owns the shard map: given a shard key (user ID, tenant ID, or whatever the schema designers chose), it routes the statement to the correct primary or replica. When the map changes, the proxies learn the new topology; applications keep the same connection string.

That design has several practical consequences. Connection pooling and prepared-statement handling move into the proxy layer so app servers are not opening thousands of raw connections to hundreds of nodes. Read/write splitting can stay policy-driven instead of hard-coded in every service. Failover of a single shard primary becomes a proxy routing update rather than a wave of application redeploys. Multiple proxy instances sit in front of the same logical database so the front door scales and stays highly available even though the data plane is deliberately partitioned.

The cost is honesty about what the abstraction cannot hide. Queries that cannot be routed by a single shard key either fan out (and pay latency and consistency complexity) or are forbidden. Schema changes must be coordinated so every shard stays compatible. Observability has to stitch per-shard metrics into a fleet view without lying about p99s. Teams that treat the proxy as a pure transparent pass-through discover those edges the hard way.

What this means for developers in practice

If your working set still fits on a well-provisioned primary with a handful of replicas, stay there. Vertical scale plus read replicas remains cheaper to operate and reason about than any sharded system. Reach for fleet sharding when write volume saturates one WAL path, when storage growth makes single-node backups or restores operationally painful, or when you are already planning multi-tenant isolation that maps cleanly onto a shard key.

Adoption work is mostly schema and query discipline, not new client libraries:

• Choose a shard key that appears in almost every hot path and that keeps related rows together. User or tenant ID is the common default. Bad keys create hot shards that reintroduce the original bottleneck.
• Audit joins and multi-row transactions. Anything that spans keys will either be rewritten, fan out, or move to application-level orchestration.
• Keep connection configuration simple: one logical DSN (or a small set for read vs write policies) aimed at the proxy. Do not let services cache direct shard endpoints.
• Plan migrations and online schema changes as fleet operations. A change that works on one Postgres instance must be safe when applied 256 times with partial failure.
• Budget for the control plane. Shard maps, proxy fleets, backup orchestration, and rebalancing are the real ongoing cost; the leaf Postgres processes are the easy part.

Competing approaches occupy the same design space with different trade-offs: extension-based distributed Postgres (for example Citus-style setups), Vitess-style proxy and topology management (historically MySQL-first, with lessons that transfer), and fully managed products that sell the abstraction as the unit of purchase. Manual application-level sharding still exists and still fails the same way when the map lives in every service. The common winning pattern is the same: one logical endpoint, explicit shard-key discipline, and operations that treat the fleet as a single database with internal partitions.

Honest scope

Most teams will never run 768 database servers. The pattern still matters earlier than that number. The moment data and write load force more than one primary, you either build a private version of the proxy-and-map machinery or you buy someone else’s. Vertical scale and replicas remain the correct default; sharding is the escape hatch when the Universal Scalability Law and the WAL stop negotiating. The teams that stay sane after that transition are the ones that invested in making the fleet disappear from application code rather than celebrating how many boxes they own.

Treat the proxy, the shard key, and the operational control plane as the product. The 768 servers are just the storage layout underneath.