Basic

Data feeds and what they hide

A feed is a promise about the past. It hides delistings, revisions, and the fact that the instrument you backtested may not have been tradable on the day it is priced.

What you will see

Two. Two versions in one comparison is the bug, and the count is what catches it.

About 1 min of reading

Try It

Stamp a run with a feed version

1

Name the feed version

printf 'feed=2026-10-01 rows=120000\n' > /tmp/feed.txt; cat /tmp/feed.txt

Before you run it, say out loud what it will print. Then open the answer.

What it prints
One line with a date and a row count.
2

Stamp the run with it

printf 'run=42 feed=2026-10-01\n' > /tmp/run.txt; grep -c 'feed=2026-10-01' /tmp/run.txt

Before you run it, say out loud what it will print. Then open the answer.

What it prints
One. A run with no feed stamp is a run that cannot be reproduced.
3

Prove two versions are not comparable

printf 'run=43 feed=2026-11-01\n' >> /tmp/run.txt; awk -F' ' '$1=="feed"{print $2}' /tmp/run.txt | sort -u | wc -l

Before you run it, say out loud what it will print. Then open the answer.

What it prints
Two. Two versions in one comparison is the bug, and the count is what catches it.

Data Flow

A feed to a comparable run
1 The feed is stored at a version
↓
2 The run names that version
↓
3 Delistings and revisions are kept in the file
↓
4 Two runs are compared only inside one version
Why this works
Why the feed is stored, not fetched

A fetched feed changes under you. A stored feed is the thing you can point at when a result is questioned.

The cost of a correct feed

It is storage, not thinking. Getting the data right is the part that is cheap, and it is the part most people skip.

Examples

Survivorship is a feed bug

A feed that only carries what still exists carries the winners. The strategy is trained on a filtered world.

Revisions are the second bug

A number that was revised was wrong when it was read. A backtest that reads the revised value read the future.

The feed version is the fix

The version names what was known when. Two runs on different versions are not comparable, and that is the point of stamping.

← Market microstructure
Previous Lesson
Market microstructure
Next Lesson
Calibrating the error, not the value
Calibrating the error, not the value →