Advanced

Tool calls are the real risk

A model that can call a tool can do something no model without tools can do: act. An injected instruction that names a tool turns a bad answer into a side effect that cannot be undone.

What you will see

One. A path outside the project is refused even though the tool was allowed.

About 1 min of reading

Quick check

Before this new lesson, one small question about the lesson you just finished. Recall it, do not look it up.

Try It

Scope and check one tool call

1

Write the allowed list for this request

printf 'search\nread\n' > /tmp/tools.txt; wc -l < /tmp/tools.txt

Before you run it, say out loud what it will print. Then open the answer.

What it prints
Two. A list of two is a request scoped on purpose.
2

Refuse a call outside the list

printf 'send\n' > /tmp/call.txt; grep -c -x -f /tmp/call.txt /tmp/tools.txt

Before you run it, say out loud what it will print. Then open the answer.

What it prints
Zero. Zero means the tool does not exist for this request.
3

Check the arguments of an allowed call

printf 'read /etc/hosts\n' > /tmp/args.txt; grep -c '^read \.\./\|/etc' /tmp/args.txt

Before you run it, say out loud what it will print. Then open the answer.

What it prints
One. A path outside the project is refused even though the tool was allowed.

Data Flow

An instruction to a refused action
1 The model proposes a tool call
↓
2 The tool is looked up in the allowed list
↓
3 The arguments are checked against a pattern
↓
4 A dangerous call needs a person to confirm
Why this works
Why capability is scoped per request

A request that needs one tool should have one tool. A request with every tool available has more capability than it has a use for.

The limit that costs nothing

Arguments checked against a pattern before the tool runs cost one comparison and stop the whole class.

Examples

The two kinds of capability

A tool that reads is risky. A tool that writes or sends is dangerous. The list of dangerous tools is the short list.

Why arguments are the leak

The model chooses the arguments. An injected instruction can choose them too, and the tool runs whatever it chose.

Approval is a human in the loop

A dangerous tool that needs a person to confirm cannot be run by an injection. That is the strongest limit there is.

Words this lesson uses

Each one is defined here on this site. Open the entry for the full definition and the other words it points at.

← The defence stack, in order
Previous Lesson
The defence stack, in order
Next Lesson
Injection through retrieved text
Injection through retrieved text →