Basic

Chunking documents

You cannot embed a whole manual as one item and search it well. It is split into pieces, and the size and shape of those pieces decide whether a question can find its answer. Chunking is the one decision that most retrieval quality is made of.

What you will see

2 chunks, each carrying its heading, which is the shape that makes a question about returns find the right one.

About 1 min of reading

Try It

Chunk a small document by heading.

1

The document

doc = '''# Shipping We ship within two business days. # Returns Damaged items get a full refund within 30 days. ''' print(doc.count('# '))

Before you run it, say out loud what it will print. Then open the answer.

What it prints
2 - two sections, which is the number of chunks a heading split produces.
2

The chunks

chunks = [('# Shipping\nWe ship within two business days.'), ('# Returns\nDamaged items get a full refund within 30 days.')] print(len(chunks))

Before you run it, say out loud what it will print. Then open the answer.

What it prints
2 chunks, each carrying its heading, which is the shape that makes a question about returns find the right one.

Put it in order

Before you write anything, put these lines back in the right order. Click a line to move it up one step. This support step is easier than writing code from scratch, and it is here to make the last example stick.

Data Flow

A document becomes searchable pieces.
1 Read the document
↓
2 Split at structure
↓
3 Attach the heading and the id
↓
4 Store each chunk
Line by line

Read the first example one line at a time. A line that prints shows what appears on screen; the rest set things up quietly.

doc = '''# Shipping sets nothing on screen
We ship within two business days. sets nothing on screen
# Returns sets nothing on screen
Damaged items get a full refund within 30 days. sets nothing on screen
''' sets nothing on screen
print(doc.count('# ')) 2 - two sections, which is the number of chunks a heading split produces.
Why this works
Carry the heading down

Prepend the section heading to every chunk in that section. Now a chunk says which section it belongs to, and a question naming that section matches strongly. That one change fixes most weak retrieval.

Keep the chunk id

Every chunk needs a stable identifier and a pointer back to the document. Without it you cannot cite the source, and a citation is what makes the answer checkable.

Examples

Small enough to be focused

A chunk that holds one idea answers one question cleanly. A chunk that holds six topics dilutes the match, because the embedding now describes all six.

Big enough to keep context

A chunk that is one sentence loses the sentence before it that named the subject. Overlap, or a heading carried into each chunk, is what keeps the reference.

Split at structure, not length

Split on paragraph or heading boundaries so a chunk is a whole thought. Splitting at a fixed character count cuts a list in half and creates a chunk that matches nothing well.

Words this lesson uses

Each one is defined here on this site. Open the entry for the full definition and the other words it points at.

← Why retrieval exists
Previous Lesson
Why retrieval exists
Next Lesson
What an embedding is
What an embedding is →