Master

Deploying to production

Local runs are easy; production is a different beast. Learn to deploy a chatbot safely with secrets, monitoring, and a rollback plan.

What you will see

Health endpoint status: ok

About 2 min of reading

Quick check

Before this new lesson, one small question about the lesson you just finished. Recall it, do not look it up.

Try It

Deploy to production with env secrets and gradual traffic shift

Load the API key from the environment, never from source code.

import os def load_api_key(): key = os.environ.get("DEMO_API_KEY", "") if not key: raise EnvironmentError("Missing API key in environment") return key os.environ["DEMO_API_KEY"] = "sk-mock-key-12345" print() print("Deploying to production") print("Loaded API key from environment: %s..." % load_api_key()[:13])

Before you run it, say out loud what it will print. Then open the answer.

What it prints

Deploying to production Loaded API key from environment: sk-mock-key-1...

Never write real secrets in code; read them from the environment.

Roll out gradually: 10% canary of the new version before full traffic.

# Gradual traffic rollout across versions. traffic_steps = [("v1", 100), ("v1+v2 canary", 10), ("v2", 100)] for phase, percent in traffic_steps: print("Route %s -> %d%% traffic" % (phase, percent))

Before you run it, say out loud what it will print. Then open the answer.

What it prints

Route v1 -> 100% traffic Route v1+v2 canary -> 10% traffic Route v2 -> 100% traffic

Confirm the deployment health check passes.

print("Health endpoint status: ok")

Before you run it, say out loud what it will print. Then open the answer.

What it prints

Health endpoint status: ok

Data Flow

Secrets are loaded from the environment; the app ships with the old and new versions, traffic is rolled out gradually, and metrics guide the final switch.

1 Load secrets from the environment

↓

2 Deploy a new version alongside the old

↓

3 Route a small share of traffic to the new version

↓

4 Monitor errors and latency

↓

5 Expand traffic if metrics are healthy

↓

6 Roll back instantly if something breaks

Why this works

What production requires

Local works, but production has demands local tests hide:

Secrets live in a manager, not source code.
Monitoring tracks errors, latency, and token spend.
Logging records problems you can investigate later.
Rate limits are enforced so one bad user cannot stall the service.
A plan to roll back turns a bad release into an inconvenience, not a disaster.

Deploy once, but design for failure. A production chatbot is judged by how it behaves under load, not how well it answers one friendly question.

Examples

Protecting your secrets

Your API key must never be in the code. Read it from the environment:

const API_KEY = process.env.OPENAI_API_KEY;
if (!API_KEY) throw new Error('Missing API key in environment');

app.get('/health', (req, res) => {
  res.json({ status: 'ok' });
});

Store OPENAI_API_KEY in the server's environment or a secrets manager, never in the repository. Committing a key leaks it to anyone who can read the code. A leaked key can drain your account in minutes.

Deploying with a safety net

Ship to production in small steps with a plan to go back:

// Version your rollout so you can switch back instantly
const deployment = {
  model: 'gpt-4o-mini',
  temperature: 0.2,
  version: 'v2'
};

Keep the old version running alongside the new one. Roll out to a small group first, watch errors and latency, and only expand if it is healthy. If something breaks, flip the switch back. Deployment is not the end, it is where you start watching closely.

Words this lesson uses

Each one is defined here on this site. Open the entry for the full definition and the other words it points at.

← Context management

Previous Lesson

Context management