Advanced

Chunking strategies that hold up

Fixed size chunking is a starting point, not a plan. The strategies that hold up give each chunk a context it can be understood in, and they let a small precise chunk pull a larger passage into the prompt when it matches.

What you will see

The whole section, including the sentence about sale items that the short chunk left out - the detail the model needed.

About 1 min of reading

Quick check

Before this new lesson, one small question about the lesson you just finished. Recall it, do not look it up.

Try It

Small chunk, large passage.

1

The pair

chunk = {'find': 'Refunds cover damaged items.', 'full': '# Returns\nRefunds cover damaged items. Sale items are final.', 'id': 'doc3#2'} print(chunk['find'])

Before you run it, say out loud what it will print. Then open the answer.

What it prints
The short text is what is embedded; the full text is what is sent when it matches.
2

Use it

print(chunk['full'])

Before you run it, say out loud what it will print. Then open the answer.

What it prints
The whole section, including the sentence about sale items that the short chunk left out - the detail the model needed.

Data Flow

A small chunk points to a large passage.
1 Split into findable pieces
↓
2 Attach the heading and the pointer
↓
3 Match on the small piece
↓
4 Send the larger passage to the model
Line by line

Read the first example one line at a time. A line that prints shows what appears on screen; the rest set things up quietly.

chunk = {'find': 'Refunds cover damaged items.', 'full': '# Returns\nRefunds cover damaged items. Sale items are final.', 'id': 'doc3#2'} sets nothing on screen
print(chunk['find']) The short text is what is embedded; the full text is what is sent when it matches.
Why this works
Headings carry the subject

A chunk that repeats the heading it belongs to can be found by a question naming that subject. Without it, the chunk says 'it' and the question has nothing to match against.

Test the boundary cases

A question answered by a sentence in the middle of a section is the test for your chunking. If that question misses, the chunking is wrong, and it is the middle of a section that most splitting mistakes break.

Examples

Small to find, large to read

Embed a short chunk and store a pointer to the section it came from. When the small chunk matches, put the whole section in the prompt. Finding is precise and reading has its context.

Sentence windows

Embed each sentence and include the sentences around it in what is sent to the model. That keeps the pronoun that named the subject, without making the whole section one blurry vector.

One chunk per item, not per page

A table, a list of options, or a set of steps is one item. Chunking it per row makes a chunk that answers nothing; chunking it as one item with a heading makes it findable.

Words this lesson uses

Each one is defined here on this site. Open the entry for the full definition and the other words it points at.

← Rewriting the question before searching
Previous Lesson
Rewriting the question before searching
Next Lesson
A retrieval system you can run
A retrieval system you can run →