Advanced

Reranking with a stronger model

The fast first pass compares a question to a chunk with no view of either. A second pass reads both together and scores the pair, which is slower and much better. Two stages, cheap then careful, is the standard shape of a good retrieval system.

What you will see

4 passages into the prompt. The narrowing happens where it is cheap to be careful, which is the point.

About 1 min of reading

Quick check

Before this new lesson, one small question about the lesson you just finished. Recall it, do not look it up.

Try It

Widen then narrow.

1

Wide first

first = ranked_all(q)[:50] print(len(first))

Before you run it, say out loud what it will print. Then open the answer.

What it prints
50 candidates going into the careful stage.
2

Careful second

final = sorted(first, key=lambda c: pair_score(q, c))[:4] print(len(final))

Before you run it, say out loud what it will print. Then open the answer.

What it prints
4 passages into the prompt. The narrowing happens where it is cheap to be careful, which is the point.

Data Flow

A cheap pass feeds a careful one.
1 Cheap search for a wide list
↓
2 Read the question with each candidate
↓
3 Score each pair
↓
4 Take the best few
Line by line

Read the first example one line at a time. A line that prints shows what appears on screen; the rest set things up quietly.

first = ranked_all(q)[:50] sets nothing on screen
print(len(first)) 50 candidates going into the careful stage.
Why this works
Widen the first stage

The first stage should pass more than you need, since the second stage is what picks. A first stage at k=3 and a second at 3 can only reorder three, which is a second stage that does nothing.

Keep both scores

A candidate strong in both stages is the best evidence. Keeping both scores makes the final list explainable, which is what lets a reviewer see why an answer cited what it cited.

Examples

Two stage retrieval

The first stage takes the top few dozen cheaply. The second reads the question against each of those and reorders to the best few. The answer only ever sees the final few.

The pair sees the negation

A question asking for what is not supported and a chunk describing support point the same way to a vector model. Reading both together, the second stage sees the mismatch that the first could not.

Budget the second stage

The second stage costs per candidate. Passing a hundred candidates is a hundred reads per question, so the first stage has to be good enough to make the list short without cutting the right chunk.

Words this lesson uses

Each one is defined here on this site. Open the entry for the full definition and the other words it points at.

← Filtering by metadata
Previous Lesson
Filtering by metadata
Next Lesson
Rewriting the question before searching
Rewriting the question before searching →