Advanced

Streaming responses

AI replies can take seconds. Streaming shows words as they arrive, like a real person typing. Learn to read the model's output piece by piece and display it live.

What you will see

Streaming responses Chunks received: 8 Received chunk: 'Hello' Received chunk: ', ' Received chunk: 'this ' Received chunk: 'reply '

The full result is shown at the end of the exercise below.

About 2 min of reading

Try It

A full AI reply can take seconds, so streaming shows words the moment they arrive instead of a blank screen. The server reads the reply as small chunks and forwards each one to the browser, which glues them back together in real time.

I split a model reply into chunks, receive them one at a time, and print each as it arrives, accumulating them into a complete message.

chunks = ["Hello", ", ", "this ", "reply ", "is ", "streamed ", "in ", "parts."] buffer = "" print("Streaming responses") print("Chunks received: %d" % len(chunks)) for c in chunks: buffer += c print("Received chunk: %r" % c) print("Full message reconstructed: %s" % buffer)

Before you run it, say out loud what it will print. Then open the answer.

What it prints

Streaming responses Chunks received: 8 Received chunk: 'Hello' Received chunk: ', ' Received chunk: 'this ' Received chunk: 'reply ' Received chunk: 'is ' Received chunk: 'streamed ' Received chunk: 'in ' Received chunk: 'parts.' Full message reconstructed: Hello, this reply is streamed in parts.

Each chunk is just a tiny piece with no words on either side, so the browser is responsible for gluing them back in order. If one chunk is dropped, a word silently disappears from the middle.

Put it in order

Before you write anything, put these lines back in the right order. Click a line to move it up one step. This support step is easier than writing code from scratch, and it is here to make the last example stick.

Data Flow

The model sends the reply as a stream; the server forwards each chunk to the browser, which displays words as they arrive.

1 Client requests a streamed reply

↓

2 Server calls the model with stream:true

↓

3 The model emits the reply in chunks

↓

4 The server decodes each chunk

↓

5 Each chunk is forwarded to the client

↓

6 The browser shows words in real time

Why this works

Why streaming feels better

A full AI reply can take two to five seconds. Without streaming, the user sees nothing during that time and assumes it is broken.

Streaming solves it:

Words arrive continuously.
The user sees progress instantly.
Long answers stay comfortable.

Streaming also lets you cancel a reply mid-sentence and saves bandwidth by not buffering everything. It is the difference between a terminal that hangs and a conversation that flows.

Examples

Reading chunks on the server

The API can send the answer as a continuous stream. On the server, read it in parts:

const stream = await fetch(API_URL, {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    model: 'gpt-4o-mini',
    messages: history,
    stream: true
  })
});

const reader = stream.body.getReader();
const decoder = new TextDecoder();
while (true) {
  const { done, value } = await reader.read();
  if (done) break;
  const chunk = decoder.decode(value);
  sendToClient(chunk);   // forward to the browser
}

Each chunk is a tiny piece of the answer. You forward it to the browser, which shows it immediately. The user never watches a blank screen.

Showing words in real time

The browser can attach each incoming chunk to the chat box:

const es = new EventSource('/stream');
es.onmessage = (event) => {
  addWord(event.data);   // one chunk at a time
};

Instead of waiting for the whole reply, words appear as the model writes them. This small change makes a chatbot feel alive, and it is how every polished AI app responds.

Words this lesson uses

Each one is defined here on this site. Open the entry for the full definition and the other words it points at.

← Conversation memory

Previous Lesson

Conversation memory

Next Lesson

Tuning model behaviour

Tuning model behaviour →