Your Flaky Test Might Be a Use-After-Free

Buildkite's engineering team published a postmortem this month that every team with a "flaky" tag in their test suite should sit with for a minute. A set of intermittently failing Rails feature tests — Selenium, ActionCable, Redis pub/sub, the usual suspects — turned out not to be a timing problem at all. It was a use-after-free in hiredis-client, the C driver that backs Ruby's redis-client gem. The same code was running in production.

That's the uncomfortable part. Nobody quarantined a race condition in a test helper; they were about to quarantine heap corruption.

The bug: pub/sub plus threads plus C

The redis-client gem — the transport underneath the mainline redis gem since version 5.0 — ships in two flavors: a pure-Ruby driver and hiredis-client, a native extension wrapping hiredis, the minimalist C client. The C driver exists for speed, and most Rails apps that care about Redis throughput reach for it without thinking twice.

The failure mode lived exactly where you'd predict if you've spent time around native extensions: the one place the client is genuinely multithreaded. ActionCable's Redis adapter runs a dedicated reader thread that sits in next_event waiting for pub/sub messages, while the main thread issues subscribe and unsubscribe commands. In hiredis-client, the read path also tried to flush the write buffer — a buffer the writing thread frees and reallocates. Two threads, one buffer, no lock that covers both: the reader ends up draining memory the writer already freed. AddressSanitizer's verdict, in the upstream report, was a textbook heap-use-after-free.

Buildkite's symptoms were what heap corruption always looks like from the outside: tests that fail a few percent of the time, occasional connection resets, then the tells — malloc: Double free of object and segfaults. The detail that gives the game away in their core dump analysis is a memmove with a size argument of 0x00a0ffffffffffb6, roughly 45 petabytes. A length field like that isn't a bug in your test; it's freed memory that got reused and reinterpreted.

Rian McGuire reported it upstream as redis-client issue #208 in July 2024, with a one-line fix merged the same day: the read path's flush was redundant — write already flushes — so deleting it closed the race. The fix shipped in redis-client 0.23.0, with a follow-up crash fix for subscriptions in 0.23.1. If you're on ActionCable with the hiredis driver and haven't bumped past 0.23.1, that's your action item from this piece.

The playbook is more valuable than the bug

The specific bug is fixed. What's worth stealing is the debugging chain, because almost none of it is Buildkite-specific.

First: they capture core dumps in CI. A Buildkite plugin detects core files after a step and uploads them as build artifacts. That's the whole trick, and it's the step most teams are missing. When a segfault happens once a week on an ephemeral CI agent, the evidence evaporates with the container — unless you've arranged for it not to. The equivalent in GitHub Actions is a few lines: set ulimit -c unlimited, point kernel.core_pattern somewhere sane, and actions/upload-artifact on failure.

Second: gdb on the core got them to "this is memory corruption in native code," which reframed the entire investigation. That reframing matters more than the stack trace itself. The moment you see an impossible allocation size, hypotheses about WebSocket timing and Selenium waits go in the bin.

Third: AddressSanitizer found the root cause in about 30 minutes of runtime, after a couple of hours building an instrumented Ruby. This is quietly a bigger deal than it sounds for Rubyists. ASan has been standard kit in C and C++ shops for a decade, but running a whole Ruby interpreter under it used to be miserable; the upstream report used Ruby 3.4 with ASan enabled, riding recent work in CRuby to make sanitizer builds viable. If your stack has native extensions — and every nontrivial Ruby, Python, or Node stack does — an ASan-built interpreter is now a realistic weapon rather than a research project.

Retry culture is a bug-hiding machine

Here's the editorial part, and there's some irony in a company that sells flaky-test detection handing us the evidence: the industry's default response to flakiness is engineered suppression. rspec-retry, @pytest.mark.flaky, CI-level auto-retries, quarantine queues. Those tools exist because most flakes really are timing noise, and burning engineer-days on every one of them is a bad trade.

But "flaky" is a description of a symptom, not a diagnosis, and we've let the tooling flatten that distinction. A test that fails with a stale-element error is a different species from a test whose process died. Segfaults, bus errors, malloc diagnostics, mysterious connection resets from a client library — these aren't flakes, they're crashes with a low reproduction rate, and a retry loop is precisely the wrong response because it converts a screaming signal into a green checkmark. Buildkite's tests had been failing intermittently within days of a Redis gem upgrade; a team with aggressive auto-retry might never have looked, and the use-after-free would have kept firing under production ActionCable load, silently or not.

The practical fix is to classify flakes by failure signature before they enter the retry pipeline. Exit codes from signals (anything 128+), SIGSEGV/SIGABRT in output, allocator error strings — route those to a human, never to quarantine. That's a grep in your CI pipeline, not a platform feature.

There's a broader lesson about where memory-unsafety actually lives in 2026. The Rust-versus-C discourse tends to fixate on operating systems and browsers, but the everyday exposure for most working developers is this: a "memory-safe" application language sitting on a stack of native extensions — hiredis bindings, JSON parsers, grpc, nokogiri — each one a small C codebase exercised by concurrency patterns its authors may never have tested. The pure-Ruby driver never had this bug. Buildkite's interim mitigation was simply to switch to it, trading some raw parsing speed for an entire vanished bug class. For a pub/sub connection that spends its life blocked on a socket, that trade is close to free — and it's worth asking, for each native extension in your Gemfile or requirements.txt, whether the speed you're buying is speed you can measure.

Flaky tests are the one place your system voluntarily confesses its rarest bugs. The least you can do is keep the body for the autopsy.