Advanced

Red teaming your own system

A defence is only as good as the attempts that failed against it. Red teaming is running known attacks against the running system and keeping the ones that still work.

What you will see

role-switch. That one line is the whole work list for the next fix.

About 1 min of reading

Quick check

Before this new lesson, one small question about the lesson you just finished. Recall it, do not look it up.

Try It

Run an attack suite and file the wins

1

Write the suite

printf 'repeat-above\nnew-instruction\nrole-switch\n' > /tmp/suite.txt; wc -l < /tmp/suite.txt

Before you run it, say out loud what it will print. Then open the answer.

What it prints
Three attacks. The suite size is the denominator of the score.
2

Record the results

printf 'repeat-above fail\nnew-instruction fail\nrole-switch win\n' > /tmp/results.txt; grep -c ' win' /tmp/results.txt

Before you run it, say out loud what it will print. Then open the answer.

What it prints
One. One win out of three is the score, and both numbers print together.
3

File the win with its string

grep ' win' /tmp/results.txt | cut -d' ' -f1 > /tmp/open.txt; cat /tmp/open.txt

Before you run it, say out loud what it will print. Then open the answer.

What it prints
role-switch. That one line is the whole work list for the next fix.

Data Flow

A suite to a shrinking loss list
1 The suite is run against the system
↓
2 Each attack is recorded as win or fail
↓
3 Wins are filed with the attack string
↓
4 The next fix targets the filed wins
Why this works
Why the suite runs against the deployed system

Testing a local copy tests a system that is not the one users reach. The suite runs where the system runs.

The score that matters

The number that matters is attacks that still work, not attacks that failed. A small number is only meaningful next to the suite size.

Examples

The suite is a file

Attacks live in a file that runs on every deploy. A defence that was fixed by hand is a defence that regresses silently.

Why new attacks are added immediately

An attack found once and not added is an attack that will be found again by a user instead of by the suite.

A suite that always passes is unmeasured

If every attack fails and no attack ever worked, the suite may be broken. The suite is only proven by an attack that once passed.

Words this lesson uses

Each one is defined here on this site. Open the entry for the full definition and the other words it points at.

← Injection through retrieved text
Previous Lesson
Injection through retrieved text
Next Lesson
A written policy for what is unsafe
A written policy for what is unsafe →