The Real Bottlenecks in Postgres Queues Aren't Where You Think

The "just use Postgres for your job queue" argument has been won for years — Rails ships Solid Queue as its default backend, and half of Hacker News rolls its eyes every time the topic resurfaces. What's actually new in DBOS's recent write-up on scaling Postgres queues to 30,000 executions per second isn't the conclusion. It's the failure map. The bottlenecks they hit on the way from 100 jobs/sec to 30K are not the ones the folklore warns you about, and that gap between lore and reality is worth your attention if you run — or are about to rip out — a database-backed queue.

The received wisdom says Postgres queues die by vacuum: churn a status column hard enough and dead tuples pile up, the planner degrades, and you spend your weekends tuning autovacuum. DBOS's numbers tell a different story. Their walls, in order, were lock contention, transaction isolation, and index maintenance CPU. Vacuum shows up only indirectly, as a cost you cut with better index design.

The three walls, and where they actually are

The first wall is the one everybody knows. Naive polling — every worker running SELECT ... ORDER BY created_at LIMIT n FOR UPDATE — collapses around 100 jobs/sec because all workers fight over the same oldest rows. The fix is FOR UPDATE SKIP LOCKED, which lets each worker lock a disjoint batch and skip rows another worker already claimed. This has been in Postgres since 9.5, shipped in early 2016, and it's the founding trick of every serious Postgres queue: Que, graphile-worker, Oban, River, pgmq, Solid Queue. If a 2026 article had stopped there, it would deserve the eye-rolls.

The second wall is more interesting because almost nobody writes about it. DBOS ran dequeues under REPEATABLE READ to enforce global concurrency limits — "at most N workflows running across the whole fleet" needs a consistent snapshot to count correctly. Past roughly 1,000 dequeues/sec, most of those transactions started dying with serialization failures and retrying, and throughput fell off a cliff. Their fix was to make isolation conditional: queues with a global limit keep REPEATABLE READ; queues with only per-worker limits ("at most 10 per process") drop to READ COMMITTED, where the failures vanish entirely. The general lesson generalizes well beyond queues: isolation level is a per-transaction knob, and paying for a fleet-wide invariant on transactions that only need a local one is a self-inflicted wound. Most homegrown queues get this wrong in the other direction — they run everything at READ COMMITTED and their "global" concurrency limit is quietly approximate.

The third wall was CPU, around 8,000 jobs/sec, and the culprit was index churn. Every status transition — enqueued, started, completed — rewrites index entries on a table where the overwhelming majority of rows are terminal and will never be dequeued again. The fix was partial indexes: index the dequeue path only WHERE status = 'ENQUEUED', with priority and creation time in the key so the hot query needs no sort. In practice that looks like:

CREATE INDEX queue_dequeue_idx
  ON workflow_status (queue_name, priority, created_at)
  WHERE status = 'ENQUEUED';

A completed job falls out of the index instead of lingering as a dead entry, which is also how you keep autovacuum bills down without heroic tuning. Partial indexes are one of Postgres's most underused features generally; on a queue table, where the live working set is a sliver of total rows, they're closer to mandatory than optional.

Read the numbers with the vendor discount

Now the caveats, because nobody is editing this claim chain but us. The 30K/sec figure — about 80 billion executions a month — comes from the vendor whose product is the queue, and the post doesn't disclose the Postgres instance size or hardware behind the benchmark. Treat it as an existence proof, not a capacity plan. It's a credible one, though: Oban has published benchmarks in the 12K jobs/sec range on Postgres, and plenty of teams run millions of queued rows in production without drama. The direction of the claim — that a single well-indexed Postgres can cover queue throughput far beyond what most companies will ever generate — is consistent with everything else in the record.

The honest limits are elsewhere. If your workload is write-heavy across the board, the queue shares headroom with everything else on the primary, and Postgres's single-writer architecture becomes the ceiling — that's a sharding conversation, not an indexing one. If you need a queue as an integration boundary between teams that don't share a database, SQS or Kafka is still the right shape. And MVCC bloat is real; partial indexes shrink the problem but don't repeal it, which is why designs like pgmq lean on partition truncation instead of row deletes for cleanup.

What this is really about

The subtext of the post matters more than the benchmark. DBOS sells durable execution — workflows that survive crashes and resume — as a library that piggybacks on your existing Postgres, in deliberate contrast to Temporal, which runs a dedicated orchestration cluster. That architecture only holds if Postgres can absorb the queue traffic of a large fleet, so this post is DBOS defending its load-bearing wall. That's fine; it's also why the engineering detail is unusually concrete for vendor content, and why the techniques transfer wholesale to any queue you build yourself.

The practical takeaway: if you're on Redis-plus-Sidekiq or Celery-plus-RabbitMQ mostly out of habit, the operational argument for that second system keeps shrinking. A queue in Postgres enqueues jobs in the same transaction as your business writes — the transactional-outbox problem disappears instead of getting solved. Reach for an existing implementation (River in Go, Oban in Elixir, Solid Queue in Rails, pgmq if you want it in-database) before writing your own, and if you do write your own, steal all three fixes on day one: SKIP LOCKED, isolation matched to the invariant, and a partial index on the enqueued state. The teams that genuinely need more than that already know who they are — and they're not reading Postgres queue articles for the verdict.