Property-Based Testing in Python with Hypothesis

What you'll build / learn

You'll add Hypothesis property-based tests to a small Python codec, watch it automatically discover a bug your hand-picked test cases miss, and read the minimal failing input it shrinks the bug down to.

Prerequisites

• Python 3.10 or newer (verified on Python 3.13.5; Hypothesis supports 3.10–3.15)
• Hypothesis 6.165.10 and pytest 9.1.1 — the versions every command below was verified against
• Any OS with a terminal; commands use POSIX syntax (on Windows, activate the venv with .venv\Scripts\activate instead)

No accounts or services needed. Everything runs locally.

1. Set up the project

mkdir rle-demo && cd rle-demo
python3 -m venv .venv && source .venv/bin/activate
pip install hypothesis pytest

2. Write the code under test

We'll test a run-length encoder — the classic Hypothesis demo, because it hides a bug beautifully. Create rle.py:

def encode(text: str) -> list[tuple[str, int]]:
    """Run-length encode: "aaab" -> [("a", 3), ("b", 1)]."""
    result = []
    count = 1
    prev = ""
    for char in text:
        if char == prev:
            count += 1
        else:
            if prev:
                result.append((prev, count))
            prev = char
    if prev:
        result.append((prev, count))
    return result


def decode(pairs: list[tuple[str, int]]) -> str:
    return "".join(char * count for char, count in pairs)

This code has a bug, but it's the kind that survives casual testing: encode("abc"), encode("aaa"), and encode("") all return the right answer. A traditional test file asserting those three cases passes green.

3. Write a property test instead

With example-based tests, you pick the inputs. With property-based tests, you state something that must hold for every input, and Hypothesis generates hundreds of inputs trying to break it — including the weird ones (empty strings, null bytes, emoji, surrogate-adjacent characters) you'd never type by hand.

The best property for any encode/decode pair is the round trip. Create test_rle.py:

from hypothesis import given, strategies as st

from rle import decode, encode


@given(st.text())
def test_roundtrip(text):
    assert decode(encode(text)) == text

st.text() is a strategy — a recipe for generating values, here arbitrary Unicode strings. @given runs the test body against 100 generated inputs per run by default. The strategies reference covers integers, floats, dates, dictionaries, dataclasses, and compositions of all of them.

4. Run it and read the shrunk failure

pytest test_rle.py -q

Hypothesis finds a failing input almost instantly, then shrinks it — repeatedly simplifying the failing string until no smaller input still fails — and reports only the minimal reproduction:

text = '001'

    @given(st.text())
    def test_roundtrip(text):
>       assert decode(encode(text)) == text
E       AssertionError: assert '0011' == '001'
E       Failing test case: test_roundtrip(
E           text='001',
E       )

That text='001' is the whole bug in three characters: a run of length ≥ 2 followed by a different character. Walk it through encode: after the "00" run, count is 2; when the loop hits "1" it appends ("0", 2) and moves prev on — but never resets count. So "1" gets encoded as ("1", 2) and the round trip comes back as "0011".

Two things happened behind the scenes worth knowing. First, the raw failing example Hypothesis initially generated was likely some ugly 20-character Unicode string — shrinking is what turned it into '001'. Second, Hypothesis saved the failure to a local database in .hypothesis/ (add it to .gitignore), so the next run replays it first, before any random generation. Failures stay deterministic until you fix them.

5. Fix the bug

Reset the counter when a new run starts, in the else branch of rle.py:

        else:
            if prev:
                result.append((prev, count))
            prev = char
            count = 1

Run pytest test_rle.py -q again: 1 passed.

6. Pin regressions and add a second property

@example forces specific inputs to run every time, before random generation — use it to pin known past failures and boundary cases. And one property rarely says everything: the round trip can't catch encode emitting adjacent pairs with the same character, so add a structural property. Replace test_rle.py with:

from hypothesis import example, given, settings, strategies as st

from rle import decode, encode


@given(st.text())
@example("aaab")
@example("")
def test_roundtrip(text):
    assert decode(encode(text)) == text


@settings(max_examples=500)
@given(st.text(min_size=1))
def test_encode_structure(text):
    pairs = encode(text)
    assert sum(count for _, count in pairs) == len(text)
    assert all(a != b for (a, _), (b, _) in zip(pairs, pairs[1:]))

@settings(max_examples=500) buys more coverage per run at the cost of runtime; the default 100 is fine for local loops, and teams commonly raise it only in CI.

Verify it works

pytest -q

Expected output (timing will vary):

..                                                                       [100%]
2 passed in 0.31s

To confirm the harness still has teeth, delete the count = 1 line again and rerun — you should get the text='001' failure back immediately, replayed from the .hypothesis/ database.

Troubleshooting

hypothesis.errors.InvalidArgument: Expected a SearchStrategy but got t=<function text at 0x...> (type=function) You wrote @given(st.text) instead of @given(st.text()). Strategies are values returned by calling the factory function — add the parentheses.

hypothesis.errors.FailedHealthCheck: 'test_rle.py::test_x' uses a function-scoped fixture 'db'. A function-scoped pytest fixture is set up once per test function, but @given runs the body 100+ times — so all examples share one fixture instance, which usually isn't what you want. Rebuild the resource inside the test body, or if sharing is genuinely fine, suppress with @settings(suppress_health_check=[HealthCheck.function_scoped_fixture]).

hypothesis.errors.DeadlineExceeded: Test took 352.18ms, which exceeds the deadline of 200.00ms. Each individual example must finish within 200 ms by default. For legitimately slow code, raise or disable it: @settings(deadline=None).

hypothesis.errors.FailedHealthCheck: It looks like this test is filtering out a lot of inputs. Your .filter(...) or assume() rejects nearly everything Hypothesis generates (the message reports the counts, e.g. "0 inputs were generated successfully, while 50 inputs were filtered out"). Construct valid values directly instead of filtering — e.g. replace st.integers().filter(lambda x: x % 1000 == 17) with st.integers().map(lambda x: x * 1000 + 17).

Next steps

• Work through the official Hypothesis tutorial for st.composite, assume(), and custom strategies for your domain objects.
• Test stateful systems (caches, DB layers, state machines) with RuleBasedStateMachine, which generates whole sequences of operations and shrinks the sequence.
• Let the Ghostwriter draft tests for you: pip install 'hypothesis[cli]', then hypothesis write rle.encode.
• Use settings profiles to run 100 examples locally and thousands in CI without touching test code.