Advanced

Adding documents without a rebuild

The index is built once, and the library is not static. The difference is the whole design decision, and the decision is a list of parts rather than one file.

What you will see

refuse old part still holds it. The part is named, which is what must be replaced.

About 1 min of reading

Try It

Write the parts and the limit

1

Write three parts

printf 'part1 docs 1000 date=2026-01\npart2 docs 10 date=2026-10\npart3 docs 5 date=2026-10\n' > /tmp/p.txt; wc -l < /tmp/p.txt

Before you run it, say out loud what it will print. Then open the answer.

What it prints
3. Three parts is the index, and the answer is their sum.
2

Write the part limit

printf 'parts_max 4\n' > /tmp/m.txt; cat /tmp/m.txt

Before you run it, say out loud what it will print. Then open the answer.

What it prints
parts_max 4. Over the limit the oldest parts are merged.
3

Refuse a changed document

printf 'doc 7 in part1\n' | awk '{print "refuse old part still holds it"}'

Before you run it, say out loud what it will print. Then open the answer.

What it prints
refuse old part still holds it. The part is named, which is what must be replaced.

Data Flow

A new document to a part
1 The new document becomes its own part
↓
2 The answer is the sum of the parts
↓
3 The part count is checked against the limit
↓
4 Over the limit the oldest parts are merged
Why this works
Why the merge is scheduled

A merge during an answer is a slow answer. The merge happens when no answer is running, and the when is written.

The record of a part

Its size and its date. The date is what makes an old part findable for merging.

Examples

A new document is a new part

The old parts are not touched. Not touched is what makes the answer the sum of the parts.

The part count is a limit

Too many parts is a slow answer. The limit is written, and over the limit the parts are merged.

A changed document is a refusal first

The old part still holds it. Refusing first names the part, and the part is what must be replaced.

Words this lesson uses

Each one is defined here on this site. Open the entry for the full definition and the other words it points at.

← Paging an answer that is long
Previous Lesson
Paging an answer that is long
Next Lesson
Building an index that fits
Building an index that fits →