Profile a Rust Web Service with cargo-flamegraph to Eliminate CPU Hot Paths

What you'll build

Profile a running Axum web service to identify a CPU hot path (regex compilation per request), fix it with a zero-cost static initializer, then confirm the improvement with before-and-after Criterion benchmark numbers.

Prerequisites

• Linux only. perf doesn't work on macOS or WSL2; use a native Linux VM or bare metal.
• Rust 1.80 or newer (rustc --version). std::sync::LazyLock stabilized in 1.80.
• linux-tools for perf, wrk for load generation.

sudo apt install linux-tools-common linux-tools-$(uname -r) wrk

If linux-tools-$(uname -r) isn't found, try linux-tools-generic. In Docker or most CI containers, perf won't work at all — use a VM.

1. Set Up the Project

cargo new perf-demo && cd perf-demo

Cargo.toml:

[package]
name = "perf-demo"
version = "0.1.0"
edition = "2021"

[dependencies]
axum = "0.7"
tokio = { version = "1", features = ["full"] }
regex = "1"

[dev-dependencies]
criterion = { version = "0.5", features = ["html_reports"] }

[[bench]]
name = "handler"
harness = false

[profile.release]
debug = 1

debug = 1 adds line-table DWARF info to the release binary. Without it, every frame in your flamegraph will read [unknown].

src/main.rs:

use axum::{routing::get, Router};
use regex::Regex;
use std::net::SocketAddr;

async fn check_dates() -> String {
    // Intentional hot path: Regex::new compiles a full DFA on every call.
    let re = Regex::new(r"^\d{4}-\d{2}-\d{2}$").unwrap();
    let input = ["2024-01-15", "not-a-date", "2024-12-31", "2025-06-01"];
    let count = input.iter().filter(|s| re.is_match(s)).count();
    format!("{count} valid dates\n")
}

#[tokio::main]
async fn main() {
    let app = Router::new().route("/check", get(check_dates));
    let addr = SocketAddr::from(([127, 0, 0, 1], 3000));
    let listener = tokio::net::TcpListener::bind(addr).await.unwrap();
    println!("Listening on {addr}");
    axum::serve(listener, app).await.unwrap();
}

2. Install cargo-flamegraph and Configure perf

cargo install flamegraph

This installs the cargo flamegraph subcommand. On Linux it drives perf record under the hood, then pipes output through inferno-collapse-perf and inferno-flamegraph to produce an SVG.

Relax perf's default access restrictions:

echo -1 | sudo tee /proc/sys/kernel/perf_event_paranoid
echo 0 | sudo tee /proc/sys/kernel/kptr_restrict

These reset on reboot. For a permanent dev machine, put them in /etc/sysctl.d/99-perf.conf.

3. Capture the Before Flamegraph

Open two terminals.

Terminal A — start the service under the profiler:

cargo flamegraph --bin perf-demo -o flamegraph-before.svg

This builds in release mode and runs the binary under perf record -F 99. The service is now listening on port 3000.

Terminal B — drive load for the duration of the profile:

wrk -t4 -c50 -d20s http://127.0.0.1:3000/check

Let wrk finish its 20 seconds, then press Ctrl+C in Terminal A. cargo-flamegraph writes the SVG.

xdg-open flamegraph-before.svg

Find your check_dates frame. Nested under it you'll see regex_automata or regex_syntax consuming a significant share of CPU time. That's DFA construction, repeating on every single request.

4. Write the Benchmarks (Both Versions)

benches/handler.rs:

use criterion::{criterion_group, criterion_main, Criterion};
use regex::Regex;
use std::sync::LazyLock;

fn bench_slow(c: &mut Criterion) {
    c.bench_function("regex_per_request", |b| {
        b.iter(|| {
            let re = Regex::new(r"^\d{4}-\d{2}-\d{2}$").unwrap();
            ["2024-01-15", "not-a-date", "2024-12-31", "2025-06-01"]
                .iter()
                .filter(|s| re.is_match(s))
                .count()
        })
    });
}

fn bench_fast(c: &mut Criterion) {
    static DATE_RE: LazyLock<Regex> =
        LazyLock::new(|| Regex::new(r"^\d{4}-\d{2}-\d{2}$").unwrap());

    c.bench_function("regex_cached", |b| {
        b.iter(|| {
            ["2024-01-15", "not-a-date", "2024-12-31", "2025-06-01"]
                .iter()
                .filter(|s| DATE_RE.is_match(s))
                .count()
        })
    });
}

criterion_group!(benches, bench_slow, bench_fast);
criterion_main!(benches);

Note that DATE_RE must be static, not a local variable. A local LazyLock doesn't survive between iterations, so you'd rebuild it every time and benchmark nothing useful.

cargo bench --bench handler

5. Fix the Hot Path

Update src/main.rs:

use axum::{routing::get, Router};
use regex::Regex;
use std::net::SocketAddr;
use std::sync::LazyLock;

static DATE_RE: LazyLock<Regex> =
    LazyLock::new(|| Regex::new(r"^\d{4}-\d{2}-\d{2}$").unwrap());

async fn check_dates() -> String {
    let input = ["2024-01-15", "not-a-date", "2024-12-31", "2025-06-01"];
    let count = input.iter().filter(|s| DATE_RE.is_match(s)).count();
    format!("{count} valid dates\n")
}

#[tokio::main]
async fn main() {
    let app = Router::new().route("/check", get(check_dates));
    let addr = SocketAddr::from(([127, 0, 0, 1], 3000));
    let listener = tokio::net::TcpListener::bind(addr).await.unwrap();
    println!("Listening on {addr}");
    axum::serve(listener, app).await.unwrap();
}

LazyLock initializes DATE_RE once on first access. Every subsequent request skips construction entirely and executes the already-compiled DFA.

Verify It Works

Run the benchmarks and compare both outputs:

cargo bench --bench handler

Expected output (numbers vary by CPU):

regex_per_request  time:   [2.094 µs 2.207 µs 2.330 µs]
regex_cached       time:   [44.8 ns  45.6 ns  46.5 ns]

Microseconds versus nanoseconds — roughly 45x on this workload. The gap narrows on complex regexes and widens on trivially short ones, but the direction never changes.

Capture a second flamegraph with the fixed binary using the same Terminal A/B steps, saving as flamegraph-after.svg. The regex_automata frames will shrink to near-zero.

Confirm the service returns the right answer:

curl http://127.0.0.1:3000/check
# 3 valid dates

Troubleshooting

perf not found or perf_event_open fails. Run sudo apt install linux-tools-$(uname -r). If that package doesn't exist for your exact kernel version, install linux-tools-generic. perf requires real Linux kernel support — containers without SYS_PERF_EVENT and WSL2 won't work.

Flamegraph is all [unknown] frames. You're missing debug symbols. Confirm [profile.release] debug = 1 is in Cargo.toml, and that you're letting cargo flamegraph do the build rather than running a separately-built binary. It always rebuilds with the correct profile.

cargo flamegraph exits immediately, no SVG written. perf needs at least a few hundred samples. Make sure wrk is actively hitting the service before you press Ctrl+C — profiling a sleeping server produces nothing. Also check for a port conflict: ss -tlnp | grep 3000.

Criterion shows no improvement in bench_fast. The LazyLock must be declared static inside bench_fast, not as a plain let. A let LazyLock is dropped and recreated each iteration.

Next Steps

• perf stat gives hardware counter summaries (cache miss rate, branch mispredictions) without generating a graph. Useful for memory-bound paths that look flat in flamegraphs because samples land in cache stall cycles.
• CPU flamegraphs show tokio runtime frames but hide async task stalls. Pair this workflow with tokio-console to surface tasks blocked on I/O or mutex contention.
• Once algorithmic waste is gone, look at cargo-pgo for Profile-Guided Optimization. It uses runtime profiles to guide inlining and branch prediction hints — typically another 10-20% on throughput-sensitive services.
• The Criterion HTML report in target/criterion/ renders interactive charts with run-to-run variance. Check it before declaring a regression fixed; a noisy machine can hide real differences in the terminal output.