Advanced

Rewriting the question before searching

A user's question is rarely shaped like a document. It is short, it refers to something earlier in the conversation, and it uses the words a person uses rather than the words the manual uses. Rewriting the question into search shaped queries is the step that fixes those three, and it costs one cheap model call.

What you will see

A standalone question that carries the topic, which is the shape the search needs.

About 1 min of reading

Try It

Rewrite a follow-up.

1

The follow-up

history = 'Q: refund for a broken laptop\nA: full refund within 30 days.' q = 'and for a phone?' print(history, q)

Before you run it, say out loud what it will print. Then open the answer.

What it prints
A follow-up that names nothing on its own - retrieval on this string alone finds the wrong things.
2

The rewrite

q2 = 'refund policy for a damaged phone' print(q2)

Before you run it, say out loud what it will print. Then open the answer.

What it prints
A standalone question that carries the topic, which is the shape the search needs.

Put it in order

Before you write anything, put these lines back in the right order. Click a line to move it up one step. This support step is easier than writing code from scratch, and it is here to make the last example stick.

Data Flow

A question becomes several searches.
1 Read the conversation
↓
2 Rewrite into standalone questions
↓
3 Run the search for each
↓
4 Merge and rerank the union
Line by line

Read the first example one line at a time. A line that prints shows what appears on screen; the rest set things up quietly.

history = 'Q: refund for a broken laptop\nA: full refund within 30 days.' sets nothing on screen
q = 'and for a phone?' sets nothing on screen
print(history, q) A follow-up that names nothing on its own - retrieval on this string alone finds the wrong things.
Why this works
Rewrite, never answer

The rewrite step must not answer the question. A rewrite that guesses an answer and searches for it retrieves evidence for a wrong claim, which is the failure mode that makes a system agree with itself.

Keep the original too

Search the original question as well as the rewrites. A rewrite that changes meaning is then still answered by the original query, and the merge keeps both in the candidate set.

Examples

Resolve the reference

'And for the annual one?' only makes sense with the previous turn. Rewriting it into a standalone question using the conversation is what makes retrieval work at all, since retrieval never sees the conversation.

Search the words it uses

Ask for the words the document uses, not the words the user used. 'Is it open on Sundays?' and 'trading hours' share no tokens, so a keyword search finds nothing and an embedding search finds it weakly.

Several queries, one list

Generate two or three phrasings and run the search for each, then merge. A question whose answer sits in one phrasing's blind spot is covered by another, which is cheaper and broader than tuning one query.

Words this lesson uses

Each one is defined here on this site. Open the entry for the full definition and the other words it points at.

← Reranking with a stronger model
Previous Lesson
Reranking with a stronger model
Next Lesson
Chunking strategies that hold up
Chunking strategies that hold up →