Embed every chunk once, embed the question, score every pair, and take the top few. That is the whole search, and the tuning is in how many you take and what you do with the scores.
About 1 min of reading
Rank toy vectors.
Set up
Before you run it, say out loud what it will print. Then open the answer.
Take two
Before you run it, say out loud what it will print. Then open the answer.
Before you write anything, put these lines back in the right order. Click a line to move it up one step. This support step is easier than writing code from scratch, and it is here to make the last example stick.
Read the first example one line at a time. A line that prints shows what appears on screen; the rest set things up quietly.
Print the top five with their scores. A big drop between the second and third tells you the question had two good answers; a flat list tells you the question matched nothing well.
Embedding the same text every query is slow and wasteful. Do it once at build time, store the vectors, and embed only the question at search time.
Take the best few, not everything above a threshold. A fixed count keeps the prompt size predictable, which matters because context costs and a growing list eventually overflows.
A score of 0.62 does not mean 62 percent likely correct. Comparing scores to each other is valid; reading one as a confidence is not, which is why the threshold is tuned on your own data.
Rank every candidate, then take the top k. Cutting first and sorting after gives you an arbitrary subset, and the right chunk may never have been in it.
Each one is defined here on this site. Open the entry for the full definition and the other words it points at.