Master

Auditing the frontend suite for what it does not cover

A green suite is a list of questions, and the audit is the list of questions it does not ask. The move is to enumerate the pages and interactions users reach, then mark which have an assertion. The gap list is the work, and it is the only honest measure of a suite.

What you will see

An ordered list. The top line is the next assertion worth writing.

About 1 min of reading

Try It

Audit the suite against the site

List the routes

grep -oE "^(app|router)\.(get|post)\(|createHandler\(|registerRoute\(" routes/*.js | head -20; ls routes | wc -l

Before you run it, say out loud what it will print. Then open the answer.

What it prints

The route list. This is the denominator, not the number of test files.

Name the assertion per route

grep -oE 'http://localhost:3000[/a-z0-9-]*' FULLTEST/*.js | sed 's|.*3000||' | sort -u | tee /tmp/covered.txt | wc -l

Before you run it, say out loud what it will print. Then open the answer.

What it prints

The paths the suite actually requests. Anything not in this list is uncovered.

Diff to find the gaps

comm -23 <(printf '/tutorials/lessons /tutorials/perf/lessons /account ' | sort -u) <(sort -u /tmp/covered.txt)

Before you run it, say out loud what it will print. Then open the answer.

What it prints

The uncovered paths. Each one is a page a user can reach with no assertion.

Rank the gaps

printf 'rank 1: /tutorials/lessons (every visitor) rank 2: /account (signed-in users) rank 3: /downloads (a few) ' | tee docs/fe-gaps.md

Before you run it, say out loud what it will print. Then open the answer.

What it prints

An ordered list. The top line is the next assertion worth writing.

Do not count a route as covered because a test file mentions it. Count it covered only when an assertion checks a property of its response.

Put it in order

Before you write anything, put these lines back in the right order. Click a line to move it up one step. This support step is easier than writing code from scratch, and it is here to make the last example stick.

Data Flow

Suite to a gap list

1 List pages and interactions users reach

↓

2 Name the assertion for each

↓

3 Mark the ones with none

↓

4 Rank the gaps by reach

Why this works

What the audit is not

It is not a line count. A suite can have many lines and cover few pages, which is why the denominator is pages and interactions.

Where the audit lies

An assertion that asserts on a string the page happens to contain counts as coverage it does not have. Name the property asserted, not the file touched.

Examples

Enumerate from the user side

List the pages a visitor reaches and the actions a user takes. That list is the coverage denominator, and the suite never supplies it.

Mark asserted and unasserted

For each page, name the assertion that touches it. A page with no named assertion is uncovered, and a page with three is over-asserted.

Rank the gaps by reach

The page half the users load is worth more than the page three people load. Coverage percentage is not the ranking, reach is.

Words this lesson uses

Each one is defined here on this site. Open the entry for the full definition and the other words it points at.

← Putting frontend tests in the release

Previous Lesson

Putting frontend tests in the release

Next Lesson

Choosing what the suite asserts when everything cannot be asserted

Choosing what the suite asserts when everything cannot be asserted →