Basic

What an embedding is

An embedding turns text into a list of numbers, and text with similar meaning gets numbers that are close together. That closeness is what a search uses. The numbers themselves are meaningless to read, and the relationship between two of them is the whole point.

What you will see

1 0 - identical vectors score 1, perpendicular ones score 0. That score is the ranking signal.

About 1 min of reading

Try It

See the shape of an embedding.

1

One vector

print(len([0.1, -0.2, 0.5, 0.3]))

Before you run it, say out loud what it will print. Then open the answer.

What it prints
4 - a toy length. Real ones are hundreds long, and the length is what you cannot read.
2

The comparison

a = [1, 0]; b = [1, 0]; c = [0, 1] print(a[0]*b[0] + a[1]*b[1], a[0]*c[0] + a[1]*c[1])

Before you run it, say out loud what it will print. Then open the answer.

What it prints
1 0 - identical vectors score 1, perpendicular ones score 0. That score is the ranking signal.

Data Flow

Text becomes numbers that compare.
1 Send the text to the model
↓
2 Receive a list of numbers
↓
3 Store it with the text
↓
4 Compare two by their angle
Line by line

Read the first example one line at a time. A line that prints shows what appears on screen; the rest set things up quietly.

print(len([0.1, -0.2, 0.5, 0.3])) 4 - a toy length. Real ones are hundreds long, and the length is what you cannot read.
Why this works
You cannot read them

Printing an embedding tells you nothing. The check is whether two texts you expect to be similar score high, which is a test you can run and read.

Store them next to the text

Keep the chunk text with its embedding. When a search returns a match, you need the words to show the reader, and a match with no text is a match you cannot use.

Examples

Text in, numbers out

A model turns a string into a few hundred numbers. The same string always gives the same numbers, and two strings about the same thing give numbers that point the same direction.

Distance is the match

Compare two embeddings by how close they point. A high score means similar meaning, a low score means unrelated. That score is what ranks the chunks for a question.

Same model, always

Embeddings from two different models are not comparable, because the numbers mean different things in each. Embedding the documents once and the query with the same model every time is the rule that keeps the scores valid.

Words this lesson uses

Each one is defined here on this site. Open the entry for the full definition and the other words it points at.

← Chunking documents
Previous Lesson
Chunking documents
Next Lesson
Ranking the chunks
Ranking the chunks →