Basic

Why retrieval exists

A model knows what it was trained on and nothing else: not your documents, not yesterday's prices, not your project's rules. Retrieval is the step that fetches the right pieces of your own text and hands them to the model as context. It is the difference between a model that guesses and one that reads.

What you will see

That sentence is what retrieval must find. If your system cannot find it, the system is broken, not the model.

About 1 min of reading

Try It

State the contract in one line.

1

The question

q = 'What is the refund policy for a damaged item?'

Before you run it, say out loud what it will print. Then open the answer.

What it prints
A question that a document in your system answers.
2

The passage that answers it

print('If an item arrives damaged, return it within 30 days for a full refund.')

Before you run it, say out loud what it will print. Then open the answer.

What it prints
That sentence is what retrieval must find. If your system cannot find it, the system is broken, not the model.

Data Flow

A question selects text that becomes context.
1 Take the question
↓
2 Search your documents
↓
3 Pick the best passages
↓
4 Put them in the prompt
Why this works
Start with the question

Write the question the system must answer, then find the passage that answers it. If you cannot find a passage that answers it, retrieval cannot help and the document is missing.

Read what was retrieved

Print the passages that were chosen before the model sees them. Most bad answers are a bad retrieval, and reading the passages is the fastest way to see that.

Examples

Context is the contract

The model answers from the text you put in the prompt. If the answer needs a fact from your files, that fact has to be in the prompt. Retrieval is the machinery that decides which facts go in.

Guessing looks fluent

A model with no retrieval writes a plausible answer about your system that is wrong in the details. That fluency is why the failure is invisible, and why the citation of the source you retrieved is worth requiring.

Retrieval is not training

You change what the model reads without changing the model. That is why retrieval updates in a day, works on private documents, and can be checked by reading the passages it chose.

Words this lesson uses

Each one is defined here on this site. Open the entry for the full definition and the other words it points at.

Next Lesson
Chunking documents
Chunking documents →