Hunt Down Go Memory Leaks with pprof and Live Heap Diffing

What you'll build

A small Go HTTP service with a deliberate memory leak, instrumented with net/http/pprof — and a repeatable snapshot-diff workflow that takes you from "memory keeps climbing" to the exact source line responsible. The same commands work unchanged against any production Go service that exposes pprof.

Prerequisites

• Go — any currently supported release (1.24+). Current stable is go1.26.5; every command below was run and verified against go1.25.7, and the pprof features used here have been stable for years.
• curl for capturing snapshots and generating load.
• macOS or Linux. On Windows, run the shell loops under WSL.
• Optional: Graphviz for pprof's graph view — the flame-graph view needs nothing extra.

1. Create the leaky service

The bug baked in here is the most common Go leak in the wild: a map used as a cache with no eviction. Every request stores its payload "for later," and later never comes.

package main

import (
	"fmt"
	"log"
	"net/http"
	_ "net/http/pprof"
	"sync"
)

var (
	mu     sync.Mutex
	cache  = make(map[int][]byte)
	nextID int
)

func handleOrder(w http.ResponseWriter, r *http.Request) {
	payload := make([]byte, 64*1024) // pretend this is a parsed order

	mu.Lock()
	nextID++
	cache[nextID] = payload // BUG: nothing ever deletes from cache
	id := nextID
	mu.Unlock()

	fmt.Fprintf(w, "order %d accepted\n", id)
}

func main() {
	http.HandleFunc("/order", handleOrder)
	log.Println("listening on localhost:8080, pprof at /debug/pprof/")
	log.Fatal(http.ListenAndServe("localhost:8080", nil))
}

The blank import of net/http/pprof is the entire instrumentation: its init registers profiling handlers under /debug/pprof/ on http.DefaultServeMux, which this server uses because it passes nil to ListenAndServe. In production, keep app routes on their own mux and serve pprof from a second, localhost-only listener so the profiling endpoints are never internet-facing.

Save it as main.go and start it:

go mod init leakdemo
go run .

2. Take a baseline heap snapshot

In a second terminal:

curl -s -o heap1.pb.gz 'http://localhost:8080/debug/pprof/heap?gc=1'

gc=1 runs a garbage-collection cycle before the profile is written, so the snapshot counts only live memory rather than collectable garbage — heap profiles otherwise report state as of the last completed GC, which makes diffs noisy.

3. Apply load and snapshot again

Simulate traffic — 3,000 requests retaining 64 KB each is roughly 190 MB of leak:

for i in $(seq 1 3000); do curl -s localhost:8080/order > /dev/null; done
curl -s -o heap2.pb.gz 'http://localhost:8080/debug/pprof/heap?gc=1'

4. Diff the snapshots

-diff_base subtracts the baseline from the second snapshot, leaving only memory that appeared — and stayed live — between the two. Rank it with pprof's top report:

go tool pprof -top -diff_base heap1.pb.gz heap2.pb.gz

File: leakdemo
Type: inuse_space
Showing nodes accounting for 194.61MB, 12948.96% of 1.50MB total
      flat  flat%   sum%        cum   cum%
  193.61MB 12882.29% 12882.29%   193.61MB 12882.29%  main.handleOrder
       1MB 66.67% 12948.96%        1MB 66.67%  runtime.allocm
         0     0% 12948.96%   193.61MB 12882.29%  net/http.(*ServeMux).ServeHTTP

Don't let 12,882% alarm you — in a diff, percentages are relative to the baseline's total (1.50 MB here), so any real leak on a small heap produces huge numbers. The signal is the flat column: main.handleOrder holds 193.61 MB it didn't hold before, and the math checks out (3,000 × 64 KB = 187.5 MB, plus map bucket overhead).

5. Pin the exact line

Drop -top, and in the interactive shell run list (from the directory containing the source):

go tool pprof -diff_base heap1.pb.gz heap2.pb.gz
(pprof) list handleOrder

Total: 1.50MB
ROUTINE ======================== main.handleOrder in /tmp/leakdemo/main.go
  193.61MB   193.61MB (flat, cum) 12882.29% of Total
         .          .     17:func handleOrder(w http.ResponseWriter, r *http.Request) {
  193.61MB   193.61MB     18:	payload := make([]byte, 64*1024)
         .          .     19:
         .          .     20:	mu.Lock()
         .          .     21:	nextID++
         .          .     22:	cache[nextID] = payload
         .          .     23:	id := nextID

One nuance: pprof attributes leaked bytes to the line that allocated them (the make on line 18), not the line that retains them. The actual bug is four lines down — a map insert with no matching delete. With the allocation site in hand, spotting the retainer is usually a ten-second read.

6. Diff a live service in one command

For a quick look without saving files, ask the endpoint for a delta profile — it snapshots, waits N seconds, snapshots again, and returns the difference:

go tool pprof -top 'http://localhost:8080/debug/pprof/heap?seconds=15'

With load running you'll see the same culprit (73.40MB 99.32% main.handleOrder in a 15-second window). Prefer the two-snapshot workflow for real hunts, though: deltas only register if a GC cycle completes inside the window (see Troubleshooting), and saved snapshots are evidence you can re-analyze later. For a visual diff, go tool pprof -http=localhost:8081 -diff_base heap1.pb.gz heap2.pb.gz serves a web UI with a flame-graph view.

Verify it works

Two checks. First, your step 4 output should show main.handleOrder dominating the flat column at roughly request-count × 64 KB. Second, prove the diagnosis by fixing the leak — bound the cache right after the insert:

	cache[nextID] = payload
	delete(cache, nextID-100) // keep only the last 100 entries

Restart the server, repeat steps 2–3 with identical load, and re-run the diff. Expected result:

Showing nodes accounting for 4804.26kB, 186.68% of 2573.53kB total
      flat  flat%   sum%        cum   cum%
    3268kB 126.98% 126.98%     3268kB 126.98%  main.handleOrder

Growth drops from 193.61 MB to ~3 MB — the bounded 100-entry window. Confirmed find, confirmed fix.

Troubleshooting

Duration: 15s, Total samples = 0 from a ?seconds= delta profile. Heap statistics only update when a GC cycle completes. On a process with a large live heap, the default GOGC=100 means the next collection may be hundreds of megabytes of allocation away — nothing completes inside your window, so the delta is empty. Use the two-snapshot gc=1 workflow instead, or lengthen the window.

server response: 404 Not Found when fetching a profile. Your service uses its own router (chi, gin, gorilla), so the handlers net/http/pprof registered on DefaultServeMux are unreachable. Either mount them on your router (mux.HandleFunc("/debug/pprof/", pprof.Index) and friends, importing the package non-blank) or start a side listener with go http.ListenAndServe("localhost:6060", nil).

Failed to execute dot. Is Graphviz installed? — followed by exec: "dot": executable file not found in $PATH. The web UI's default Graph view shells out to Graphviz. Install it (brew install graphviz / apt install graphviz) or switch to the Flame Graph or Top views, which don't need it.

Error: could not find file main.go on path ... from list. You're analyzing the profile on a machine (or in a directory) without the source that produced it — profiles embed the build machine's absolute paths. Run pprof from the source checkout, or point it there with -source_path (add -trim_path if the build paths differ).

Next steps

• Re-run any command with -sample_index=alloc_space to rank by total bytes allocated instead of live bytes — that's how you hunt allocation churn and GC pressure rather than leaks.
• Leaked goroutines pin memory too: /debug/pprof/goroutine supports the same ?seconds= delta trick for finding goroutines that start but never exit.
• Read the heap-accounting fine print in the runtime/pprof docs, then the broader tour of Go's profilers in the official diagnostics guide.
• For leaks that only show up after days in production, look at continuous profiling: capture heap?gc=1 on a cron, keep the files, and -diff_base across any two points in time.