Basic

Reading the runner output

The runner printed a wall of text and I scrolled past it. The summary line is the answer, and the failure block under it is the reason. Learn to read those two and the rest is noise.

What you will see

FAILED test_add.py::test_add_zero assert add(0, 0) == 1 ============================== 1 failed ==============================

About 1 min of reading

Quick check

Before this new lesson, one small question about the lesson you just finished. Recall it, do not look it up.

Try It

Run a file with a deliberate failure and read only the verdict and the failure block.

1

Run the file. The verdict line tells you one claim failed.

pytest test_add.py

Before you run it, say out loud what it will print. Then open the answer.

What it prints
collected 2 items test_add.py .! FAILED test_add.py::test_add_zero ============================== 1 failed, 1 passed ==============================
2

Open the failure block. It names the function and the assert line that did not hold.

pytest test_add.py -k test_add_zero

Before you run it, say out loud what it will print. Then open the answer.

What it prints
FAILED test_add.py::test_add_zero assert add(0, 0) == 1 ============================== 1 failed ==============================
Read the assert line in the block, not a stack trace of your program. The runner already points at the claim that broke, which is why the function name matters.

Data Flow

The runner prints a verdict line with counts and, on failure, a block naming the function and the line that broke.
1 Runner runs all claims
↓
2 Verdict line counts pass/fail
↓
3 Failure blocks name the line
↓
4 You read verdict then block
Why this works
Two things to read

Every runner output has a verdict line (counts) and, when something failed, a block per failure (which function, which line). Everything else is decoration. Train your eye to jump to the counts, then to the first FAILED block.

Green does not mean correct

A passing run means the claims you wrote held. It does not mean the program is right, it means it is right where you tested. The empty space between your claims is untested. That gap is the honest meaning of green.

Examples

The summary line

3 passed, 1 failed

This is the whole result in one line. If you read nothing else, read this. It counts the claims and how many broke.

The failure block

FAILED test_add.py::test_add_negative
  assert add(-1, -1) == 0

Under the summary, each failure names its function and shows the line that did not hold. This is the reason, not a stack of your program.

Quiet on request

pytest -q test_add.py

-q trims the run to the summary and failures. Useful when the full output is too long and you only want the verdict.

Words this lesson uses

Each one is defined here on this site. Open the entry for the full definition and the other words it points at.

← What to assert on
Previous Lesson
What to assert on
Next Lesson
Debugging from a failure
Debugging from a failure →