Advanced

Locking a fix in place

I fixed a bug and shipped, then the same bug came back a month later after another change. A regression test is the claim that stays red until the fix is in, so the fix cannot silently regress.

What you will see

============================== 1 passed in 0.01s ===============================

About 2 min of reading

Try It

Write a regression test that was red before the fix and green after, and keep it.

The buggy version crashes on empty input. Write the test that captures the bug.

def render(s): return s[0] + s[1:] def test_empty_renders_empty(): assert render("") == ""

Before you run it, say out loud what it will print. Then open the answer.

What it prints

Traceback (most recent call last): File "main.py", line 1, in AssertionError

Fix the function so empty input is handled. The same test now passes.

def render(s): return s pytest test_reg.py

Before you run it, say out loud what it will print. Then open the answer.

What it prints

collected 1 item test_reg.py . ============================== 1 passed in 0.01s ===============================

The test stays in the file. If a later edit breaks the fix, this is the test that turns red.

pytest test_reg.py

Before you run it, say out loud what it will print. Then open the answer.

What it prints

============================== 1 passed in 0.01s ===============================

Deleting a passing regression test is the mistake. It is a guard, not a checkpoint. Keep it so the fix cannot regress quietly.

Put it in order

Before you write anything, put these lines back in the right order. Click a line to move it up one step. This support step is easier than writing code from scratch, and it is here to make the last example stick.

Data Flow

A regression test is red before the fix and green after, kept forever, so any change that reintroduces the bug turns it red again.

1 Bug found, test written red

↓

2 Fix makes it green

↓

3 Test kept in the suite

↓

4 Future change turns it red

Why this works

Every fix gets a test

When you fix a bug, write the smallest test that was red before the fix and green after. Keep it. It is now a guard: any future edit that reintroduces that bug turns it red. This is what turns a one-time fix into a permanent one.

When you have a choice

Regression versus feature

A feature test states what the code should do. A regression test states that a specific past bug stays fixed. They are the same mechanism, different intent, and the name usually says which. A suite of regression tests is a wall of guards.

Examples

Name the bug

def test_empty_input_does_not_crash():
    assert render("") == ""

This test exists only to hold one bug fixed. Its name says which bug. When someone regresses the fix, this test is the one that goes red.

Keep it forever

pytest test_regression.py

A regression test is not deleted when it passes. It stays in the suite so that any future change that reintroduces the bug is caught on the next run.

The red is a gift

FAILED test_empty_input_does_not_crash

When this goes red after a change, it tells you that change broke a fix you already made. That is a regression, caught before it reaches users.

Words this lesson uses

Each one is defined here on this site. Open the entry for the full definition and the other words it points at.

← Setting up state for a test

Previous Lesson

Setting up state for a test

Next Lesson

Reading a stack trace

Reading a stack trace →