DailyRNG
Login
Basic

Detecting a leak after the fact

You cannot stop every injection before the model, so you check after it. A detector looks at the output for content the model was never supposed to be able to repeat.

What you will see

One. Non-zero means block, and the case goes to the loss file.

About 1 min of reading

Try It

Plant a canary and detect it

1

Put a canary in the instruction

printf 'Answer briefly. CANARY-7F3K\n' > /tmp/instr.txt; grep -o 'CANARY-[A-Z0-9]*' /tmp/instr.txt

Before you run it, say out loud what it will print. Then open the answer.

What it prints
CANARY-7F3K. The string exists only in the instruction.
2

Write an output that repeated it

printf 'The rules are: Answer briefly. CANARY-7F3K\n' > /tmp/out.txt; cat /tmp/out.txt

Before you run it, say out loud what it will print. Then open the answer.

What it prints
An output containing the canary.
3

Run the detector

grep -c 'CANARY-7F3K' /tmp/out.txt

Before you run it, say out loud what it will print. Then open the answer.

What it prints
One. Non-zero means block, and the case goes to the loss file.

Data Flow

Output to a blocked response
1 The model writes an output
↓
2 The detector compares it to known secrets
↓
3 A match marks the response unsafe
↓
4 The response is blocked and the case logged
Why this works
Why canaries are planted

A canary is a string only the system prompt contains. If it appears in an output, the prompt was repeated, and now you know it.

What the detector does with a hit

It blocks the response and writes the case to the loss file. A hit that only logs is a hit that still ships.

Examples

Secrets are the detectable class

If the output contains a string that only the system knows, something leaked. That check costs one comparison.

Why detection is not prevention

Detection catches the leak you can see. It does not stop the one you did not think to look for.

The detector runs on the output

The detector runs after the model and before the user. The model never sees the detector exist.

Words this lesson uses

Each one is defined here on this site. Open the entry for the full definition and the other words it points at.

← The injection that works
Previous Lesson
The injection that works
Next Lesson
Escaping user text before it is sent
Escaping user text before it is sent →