Optimization finds the settings that maximize past performance, and it is one of the most seductive ways to fool yourself. The best settings you can find in historical data are usually not a real edge, they're the noise wearing a costume. The skill is knowing the difference.
About 2 min of reading
Optimization finds the settings that maximize past performance, and that is almost always a lie the data tells you. I search a grid for the best in-sample result, then penalize the number of knobs so a model cannot just memorize noise.
I run a grid search over fast and slow settings and print the best in-sample return, acknowledging this winner probably overfit the random patterns in the past.
Before you run it, say out loud what it will print. Then open the answer.
I subtract a penalty for every parameter added so the optimizer is discouraged from memorizing the past, printing the penalized net score.
Before you run it, say out loud what it will print. Then open the answer.
Before you write anything, put these lines back in the right order. Click a line to move it up one step. This support step is easier than writing code from scratch, and it is here to make the last example stick.
Read the first example one line at a time. A line that prints shows what appears on screen; the rest set things up quietly.
Trading costs, taxes, and slippage quietly eat your returns. A strategy that looks fantastic before costs will very often lose money after them. Always optimize on the net number, because that's the number that actually lands in your account.
Split the data: use in-sample to search and out-of-sample to judge. If the best settings collapse on out-of-sample data, they were worthless, the search only made a prettier story.
Search 10,000 settings
Keep the one with the best past returnThat winning setting probably overfit the noise. When you let the data pick the answer for you, you're almost always memorizing random patterns. The only defense is to check those settings on data they never got to train on, and to pretend nothing will survive.
best = None
for fast in range(10, 100):
for slow in range(100, 500):
ret = backtest(fast, slow)
if ret > best:
best = retThis finds the historical winner, but it also finds the settings that happened to land on noise and got lucky. The safest winners are the ones that still look good after you throw them at fresh data. Search is easy; judging the search is the hard part.
score = return_on_test - penalty * num_parametersEvery parameter you add lets the optimizer memorize the past a little more. Penalizing complexity keeps it from doing that, and simpler models almost always generalize better to a future they never saw than anything you can overfit.
Each one is defined here on this site. Open the entry for the full definition and the other words it points at.