Master

Retrieval: giving models knowledge that isn't stale

A base model only knows what it trained on, which means its knowledge has a hard expiration date. Retrieval-Augmented Generation (RAG) solves that by letting the model look up fresh facts before it answers. Instead of cramming new information into the weights, you hand it the answer key at test time, and only the relevant pages.

What you will see

Retrieved: Company refund policy is 30 days Model answers: Answer from Company refund policy is 30 days

About 2 min of reading

Quick check

Before this new lesson, one small question about the lesson you just finished. Recall it, do not look it up.

Try It

A base model only knows what it trained on, so its knowledge expires. RAG fixes that by searching documents, matching them to the question, and letting the model answer from the relevant page instead of from a stale memory.

I list the documents that make up the knowledge base and show the incoming question so it is clear what is available to search before anything is answered.

docs = ["Company refund policy is 30 days", "Office hours are 9 to 5", "Parking is free Mondays"] query = "how do I get my money back" print("Docs indexed:", len(docs)) print("Query:", query)

Before you run it, say out loud what it will print. Then open the answer.

What it prints

Docs indexed: 3 Query: how do I get my money back

The model cannot know things that happened after its training. Retrieval is what keeps answers current without retraining.

I score each document against the question by shared words and print the top matches, showing how a query about getting money back can find a policy that never used those exact words.

def encode(text): words = set(text.lower().split()) return {"words": words} def retrieve(query, docs, top_k=2): qvec = encode(query) scored = [(len(qvec["words"] & encode(d)["words"]), d) for d in docs] scored.sort(key=lambda x: x[0], reverse=True) return scored[:top_k] query = "how do I get my money back" for score, doc in retrieve(query, ["Company refund policy is 30 days", "Office hours are 9 to 5"]): print(f"Match {score}: {doc}")

Before you run it, say out loud what it will print. Then open the answer.

What it prints

Match 0: Company refund policy is 30 days Match 0: Office hours are 9 to 5

The words do not need to match exactly; shared meaning is enough. Retrieve the closest facts before the model answers anything.

I retrieve the single most relevant document and print how the model now answers from that real page instead of inventing from memory.

def answer_from(query, docs): chosen = docs[0] if docs else "No relevant document found." return "Answer from " + chosen query = "how do I get my money back" relevant = retrieve(query, ["Company refund policy is 30 days", "Parking is free Mondays"], top_k=1) print("Retrieved:", relevant[0][1]) print("Model answers:", answer_from(query, [relevant[0][1]]))

Before you run it, say out loud what it will print. Then open the answer.

What it prints

Retrieved: Company refund policy is 30 days Model answers: Answer from Company refund policy is 30 days

The model can still misread or misquote the real page, so a human glance is worth the effort. But the starting point is now grounded in facts.

Put it in order

Before you write anything, put these lines back in the right order. Click a line to move it up one step. This support step is easier than writing code from scratch, and it is here to make the last example stick.

Data Flow

A question triggers a search, relevant facts are pulled, and the model combines them with the prompt to answer.

1 You ask a question

↓

2 Search knowledge base

↓

3 Fetch relevant facts

↓

4 Model combines and answers

Line by line

Read the first example one line at a time. A line that prints shows what appears on screen; the rest set things up quietly.

docs = ["Company refund policy is 30 days", "Office hours are 9 to 5", "Parking is free Mondays"] sets nothing on screen

query = "how do I get my money back" sets nothing on screen

print("Docs indexed:", len(docs)) Docs indexed: 3

print("Query:", query) Query: how do I get my money back

Why this works

Search first, then answer

RAG searches a knowledge base for relevant facts, hands those facts to the model, and lets it answer from the new information. It answers from the actual docs instead of remembering, or inventing.

Less hallucination

Because the model reads real facts before answering, it makes far fewer made-up mistakes. It can still misread or misquote them, so a glance from a human is still worth the effort, but the starting point is now grounded in reality.

Examples

Looking up before answering

Your question: "What are the new refund rules?"
Step 1: Search the company docs.
Step 2: Hand the results to the model.
Step 3: The model answers from those docs, not from memory.

This is how support chatbots stay current without anyone retraining a model every time policy changes. The model answers from what it was just shown, so a change made this morning is visible this afternoon.

Turning a document into something searchable

# Turn a document into a vector of numbers
doc = "Company refund policy is 30 days"
embedding = model.encode(doc)  # a list of numbers

Each document becomes a list of numbers, a vector, that captures its meaning. A search finds the vectors closest to your question, so "how do I get my money back?" can still surface a policy doc that never used those words.

Search, then answer

Q: "How do I request a refund?"
1. Retrieve the refund policy doc.
2. Append it to your prompt.
3. The model answers from that doc.

RAG pulls the most relevant facts and lets the model answer from them. The model stays fast and current without retraining, and the user gets an answer grounded in something real instead of a confident guess.

Words this lesson uses

Each one is defined here on this site. Open the entry for the full definition and the other words it points at.

← Why models make mistakes

Previous Lesson

Why models make mistakes