Advanced

Filtering by metadata

Semantic search finds similar text, and it will happily find similar text from the wrong place. A version, a language, a product, or an access rule is a filter applied before or after the search, and forgetting one is how a system answers about the wrong thing.

What you will see

An empty list. That empty list must produce 'no documents match', not an unfiltered search.

About 1 min of reading

Quick check

Before this new lesson, one small question about the lesson you just finished. Recall it, do not look it up.

Try It

Filter then rank.

1

Tag the chunks

chunks = [(1, 'v2'), (2, 'v1'), (3, 'v2')] want = 'v2' allowed = [i for i, v in chunks if v == want] print(allowed)

Before you run it, say out loud what it will print. Then open the answer.

What it prints
[1, 3] - only the v2 chunks are candidates, which is what stops a v1 answer being returned.
2

Prove the empty case

print([i for i, v in chunks if v == 'v9'])

Before you run it, say out loud what it will print. Then open the answer.

What it prints
An empty list. That empty list must produce 'no documents match', not an unfiltered search.

Put it in order

Before you write anything, put these lines back in the right order. Click a line to move it up one step. This support step is easier than writing code from scratch, and it is here to make the last example stick.

Data Flow

A filter narrows before ranking.
1 Read the metadata on each chunk
↓
2 Extract constraints from the question
↓
3 Keep only the allowed candidates
↓
4 Rank the survivors
Line by line

Read the first example one line at a time. A line that prints shows what appears on screen; the rest set things up quietly.

chunks = [(1, 'v2'), (2, 'v1'), (3, 'v2')] sets nothing on screen
want = 'v2' sets nothing on screen
allowed = [i for i, v in chunks if v == want] sets nothing on screen
print(allowed) [1, 3] - only the v2 chunks are candidates, which is what stops a v1 answer being returned.
Why this works
Empty filter results need a message

A filter that leaves nothing should say so rather than answer from an unfiltered fallback. A fallback that ignores the filter is the bug that turns a wrong answer into a plausible one.

Filter values come from the question

If the question names a version or a product, that has to become a filter. Extracting it is a small step, and skipping it is why a system answers about the default version every time.

Examples

Filter before you rank

Restrict the candidates to the allowed set, then rank inside it. Filtering after ranking can return fewer results than you asked for, and a short list is how a system silently stops answering.

Metadata is part of the chunk

Store the version, the section, and the source with each chunk. Every answer then carries what it was about, and a question about a specific version has something to match against.

Access rules are filters

A user who cannot see a document must not get a chunk from it in a prompt. The permission check belongs in the retrieval step, because that is the only place the candidate set exists.

Words this lesson uses

Each one is defined here on this site. Open the entry for the full definition and the other words it points at.

← Storing vectors so search is fast
Previous Lesson
Storing vectors so search is fast
Next Lesson
Reranking with a stronger model
Reranking with a stronger model →