DailyRNG
Login
Master

Context management

Every chatbot has a limit on how much text the model can hold. Learn to compress, trim, and prune history so long conversations never run out of room.

What you will see

After trim to 10 turns: 21 messages Kept role of first entry: system

About 2 min of reading

Quick check

Before this new lesson, one small question about the lesson you just finished. Recall it, do not look it up.

Try It

Trim conversation history into a context budget

1

Keep only the most recent turns, prepending a running summary for the older ones.

import math def trim_history(history, max_turns=10): turns = math.ceil(len(history) / 2) if turns > max_turns: keep = history[len(history) - max_turns * 2:] keep = [{"role": "system", "content": "Summary of the earlier conversation."}] + keep return keep return history # Simulate a long conversation growing past the budget. history = [] for i in range(24): history.append({"role": "user", "content": "msg %d" % i}) history.append({"role": "assistant", "content": "reply %d" % i}) print() print("Context management") print("Raw history length: %d messages" % len(history))

Before you run it, say out loud what it will print. Then open the answer.

What it prints
Context management Raw history length: 48 messages
2

Trim to the budget and confirm a summary header is preserved.

trimmed = trim_history(history, max_turns=10) print("After trim to 10 turns: %d messages" % len(trimmed)) print("Kept role of first entry: %s" % trimmed[0]["role"])

Before you run it, say out loud what it will print. Then open the answer.

What it prints
After trim to 10 turns: 21 messages Kept role of first entry: system

Put it in order

Before you write anything, put these lines back in the right order. Click a line to move it up one step. This support step is easier than writing code from scratch, and it is here to make the last example stick.

Data Flow

Recent turns are kept; when history exceeds the budget, older turns are summarized into a compact note that replaces them.
1 New message appended to history
↓
2 Current token count is checked
↓
3 Count within budget? Keep it all
↓
4 Over budget: summarize past turns
↓
5 Old messages are replaced by the summary
↓
6 The model continues with a compact thread
Why this works
The context budget

Every model has a fixed context window measured in tokens. As a chat grows, it eventually fills that window.

Manage it deliberately:

  • Recent messages are always kept, the model needs them.
  • Old messages are trimmed or summarized.
  • A running summary preserves long-term knowledge.

Balance memory against cost: keep what matters, forget what does not. A well-tuned bot stays fast and affordable while remembering enough to feel natural.

Examples

Trimming old turns

You cannot keep forever. Drop old messages when the history grows:

function trimHistory(history, maxTurns = 10) {
  const turns = Math.ceil(history.length / 2);
  if (turns > maxTurns) {
    const keep = history.slice(history.length - maxTurns * 2);
    keep.unshift({
      role: 'system',
      content: 'Summary of the earlier conversation.'
    });
    return keep;
  }
  return history;
}

When the conversation gets long, cut the oldest exchanges and keep a short note. This keeps the current chat crisp and the token count flat, no matter how long a user talks.

Summarizing on the fly

For very long chats, compress the past into a summary. Store the summary alongside the recent turns:

async function summarize(history) {
  const summary = await ask([
    { role: 'system', content: 'Summarize this conversation in 3 sentences.' },
    { role: 'user', content: history.map(m => m.content).join('\n') }
  ]);
  return summary;
}

A compressed summary means the model still "remembers" the beginning of a long chat without paying for every word twice. Smart context management is what separates a demo from a production chatbot.

Words this lesson uses

Each one is defined here on this site. Open the entry for the full definition and the other words it points at.

← Cost and rate limits
Previous Lesson
Cost and rate limits
Next Lesson
Deploying to production
Deploying to production →