Security Code Fails Silently. Test It Like It Does.
A FastAPI hardening pass shows why exception handlers, logging, and secret sweeps each deserve their own regression tests.
A developer hardening a hobby FastAPI expense tracker recently documented something more valuable than the checklist itself: nearly every piece of their security and observability code failed silently the first time. A logging format string typo'd as $s instead of %s — twice on the same line — meant the app's error-reporting path was itself broken. A test function accidentally indented inside another one simply vanished from pytest's collection, no warning given. A curl probe broke because an invisible control character rode along in a rich-text paste.
None of that is beginner clumsiness. It's the defining property of this category of code, and it's why hardening work deserves different habits than feature work.
Everyone converges on the same six steps
Strip away the narrative and the hardening pass lands on a list you'll recognize from every production API you've ever inherited: a public health endpoint, manual probes of the auth boundaries, a global exception handler that returns a generic 500 with a correlation ID, a small pytest suite that pins the auth behavior down, pagination caps on list endpoints, and a sweep of git history for leaked secrets.
That convergence isn't a coincidence — it's roughly the floor that OWASP's API Security Top 10 implies. Broken object-level authorization has sat at #1 on that list for years, and the corresponding check here is the one worth stealing verbatim: seed two users, create a record owned by user B, request it as both. Expect 200 as the owner and 404 — not 403 — as the other user. The 200 matters as much as the 404; without it, a broken router that 404s everything passes your isolation test.
Returning 404 instead of 403 for records you don't own is the right call, and it's worth being opinionated about. A 403 is an existence oracle: it tells an attacker enumerating IDs that record 4187 is real, just not theirs. GitHub has answered this the same way for over a decade — a private repo you can't see returns 404, not 403. If your API leaks existence through status codes, sequential integer IDs turn into an inventory of your customers' data.
Your error handler is the least-tested code you ship
FastAPI's default behavior on an unhandled exception is a bare-bones 500 with no internals leaked (unless you've shipped debug=True, which is a separate incident waiting to happen). The standard upgrade is a global handler:
@app.exception_handler(Exception)
async def unhandled_exception_handler(request: Request, exc: Exception):
error_id = uuid.uuid4().hex[:8]
logger.exception("Unhandled error [%s] on %s %s",
error_id, request.method, request.url.path)
return JSONResponse(status_code=500,
content={"detail": "Internal server error",
"error_id": error_id})
Here's the trap the write-up walked into and most teams walk into too: this handler is nearly impossible to exercise accidentally, so it ships untested. Worse, if you do write a test for it, Starlette's TestClient re-raises server exceptions by default — your test crashes with the original exception instead of showing you the 500 response. You have to opt out explicitly:
client = TestClient(app, raise_server_exceptions=False)
resp = client.get("/route-that-blows-up")
assert resp.status_code == 500
assert "error_id" in resp.json()
assert "Traceback" not in resp.text
And the logging inside that handler deserves its own assertion. Python's logging module uses lazy %-style interpolation, so a malformed format string doesn't raise at your call site — the TypeError surfaces later inside the formatter, gets caught by handleError, and prints a --- Logging error --- block to stderr while your actual log line is lost. Your tests stay green; your incident-time forensics are gone. Pytest's caplog fixture closes the loop: assert that the rendered message actually contains the correlation ID. If you'd rather delete the bug class entirely, structlog's keyword-argument style makes the interpolation failure unrepresentable.
The nested-function pytest bug has the same shape and the same cheap fix. A test that pytest never collects fails at nothing. pytest --collect-only in CI, with an assertion on the expected test count when the suite is small, turns "silently absent" into "loudly missing."
git log -S is a spot check, not a control
The secrets sweep in the write-up is the manual version of a solved problem: git ls-files for suspicious filenames, git log --all -- backend/.env for files that once existed, and the pickaxe (git log -S "the-actual-value") for content. The author's framing — a secret leaks by content, not filename — is exactly right, and it's why the filename-only sweeps most people do are theater.
But pickaxe searches only find secrets you already know to look for. gitleaks or trufflehog scan every blob in history against entropy checks and hundreds of credential patterns, and either one runs in CI in seconds. That's the transferable move: the manual sweep is how you learn what leaking looks like; the scanner is how you stop doing the sweep by hand forever.
One caveat the post gets right and plenty of "clean up your repo" guides get wrong: finding a leaked secret in history means the secret is burned. Rewrite history with git-filter-repo if you like, but clones, forks, and CI caches don't get the memo. Rotation is the fix; rewriting is hygiene.
The habit that actually transfers
The stack-specific details here — StaticPool for in-memory SQLite under FastAPI's thread pool, dependency_overrides for auth fixtures — are useful FastAPI trivia. The transferable lesson is structural: security and observability code fails silently by nature, because its job is to handle paths your happy-path tests never touch. So every manual check gets promoted to a permanent one, and every "it should be logging now" gets verified by reading the actual output.
That's not a junior lesson dressed up. It's the difference between teams that discover their exception handler was broken during a postmortem and teams that discovered it in CI, back when it cost nothing.
Sources & further reading
- Phase 8 - Making It Trustworthy: Hardening a FastAPI App — dev.to
- TestClient - Starlette — starlette.io
- Handling Errors - FastAPI — fastapi.tiangolo.com
- logging - Logging facility for Python — docs.python.org
Emeka has spent over a decade tracking threat actors, vulnerability disclosures, and the evolving landscape of application security, bringing a sharp continent-spanning perspective to his reporting. He's known for translating dense CVE advisories into clear, actionable context that developers and security teams alike actually read.
Discussion 5
the silent failures thing hit home—we had a similar moment in an auth middleware where a malformed env var in the secret handler just... didn't log anything, didn't crash, just let requests through with a degraded token check. we only caught it in staging because someone happened to grep the logs. now we always pair those kinds of handlers with explicit assertions in CI that the happy path *and* the failure path both write to stderr.
that's the nightmare scenario. silent auth degradation is worse than loud failure—you don't know you're bleeding. explicit assertions in CI are table stakes, but you also need tests that verify the *absence* of things (no requests reached handler, no fallback invoked). grep-finding bugs means your observability is the last line of defense, which is fragile.
the absence assertions are exactly what saved us during a key rotation last year—our tests checked that old keys were *rejected*, not just that new ones worked. turns out the fallback silently accepted both, and we only caught it because someone wrote a test that explicitly verified the old path returned 401, not 200. absence tests feel paranoid until the moment you realize nobody tested the unhappy path at all.
@securepaws nails it, but the flip side bites harder: testing absence is expensive and brittle, so most teams punt on it until prod screams. maybe the real move is treating observability *as* the test—instrument so aggressively that silent failures become impossible, not just detectable.
Been there with the silent failures—had a secrets rotation job that looked perfect in code review but the regex pattern never matched anything. Ran fine for six months before anyone noticed we weren't actually rotating. Now I write tests that actively *fail* if the security code stops working, same way you'd test that an exception is raised. Makes all the difference.
