DailyRNG
Login
Basic

Safe auto-reply design

Trustworthy auto-replies live behind three guardrails: a clear scope for what the agent may do, a way for users to reach a real person, and logging so you can see what happened.

What you will see

connect to human auto reply connect to human

About 2 min of reading

Try It

Trustworthy auto-replies live behind three guardrails: a scope for allowed actions, a log so replies can be reviewed, and a human escape hatch. I run actions past the allowed list, log each reply, and print when a person is connected.

1

A guardrail lists which actions the agent may take and which must escalate. I run three actions and print whether each runs or goes to a human.

ALLOWED = {"answer_faq", "book_appointment"} FORBIDDEN = {"refund_money", "delete_account"} def check(action): if action in FORBIDDEN: return "human" elif action in ALLOWED: return "run" else: return "unknown" for a in ["refund_money", "answer_faq", "delete_account"]: print(check(a))

Before you run it, say out loud what it will print. Then open the answer.

What it prints
human run human
Listing forbidden actions is safer than hoping the model behaves. Anything not on the allowed list stays unapproved.
2

Every reply should be recorded so bad ones can be fixed later. I loop over three replies with their confidence and print the log lines.

replies = [("hello", 0.7), ("cancel", 0.9), ("??", 0.3)] for msg, score in replies: print(f"log: {msg} | confidence={score}")

Before you run it, say out loud what it will print. Then open the answer.

What it prints
log: hello | confidence=0.7 log: cancel | confidence=0.9 log: ?? | confidence=0.3
You cannot fix what you do not measure. Logging both the message and the confidence is what lets you spot weak replies.
3

The human escape hatch opens when a user asks for a person or the bot is unsure. I test three inputs and print whether a real person is connected.

def hatch(message, confidence): if "speak to a person" in message.lower() or confidence < 0.5: return "connect to human" else: return "auto reply" for m, s in [("I want a person", 0.6), ("hello", 0.9), ("?", 0.4)]: print(hatch(m, s))

Before you run it, say out loud what it will print. Then open the answer.

What it prints
connect to human auto reply connect to human
Trust needs an escape hatch. If a user can always reach a person, a wrong auto-reply costs them less.

Put it in order

Before you write anything, put these lines back in the right order. Click a line to move it up one step. This support step is easier than writing code from scratch, and it is here to make the last example stick.

Data Flow

Rules and logging wrap the auto-reply so risky actions escalate and every reply can be reviewed.
1 Run the reply inside a defined scope
↓
2 Block forbidden actions automatically
↓
3 Log the message and reply
↓
4 Review logs to add new rules
Why this works
The three guardrails

Keep every auto-reply system behind three rails:

  1. **Scope*: only certain topics or actions.
  2. **Human hand-off*: a clear escape hatch.
  3. **Logging*: record what happened.

Scope + Hand-off + Logging = trustworthy auto-reply

Without all three, a smart reply can still cause real harm.

Improving over time

An auto-reply system improves when you:

  • Watch where it hands off too often.
  • Read the replies users rated poorly.
  • Add new rules for repeated questions.

Collect -> Review -> Update -> Repeat

This turns a static script into something that gets better every week.

Examples

Setting boundaries

What the agent is allowed to do

ALLOWED_TO = {"answer_faq", "book_appointment"}
FORBIDDEN_TO = {"refund_money", "delete_account"}

if action in FORBIDDEN_TO:
    escalate_to_human()
else:
    run(action)

Listing forbidden actions prevents the agent from doing costly things on its own.

Why it works: explicit limits are easier to audit than hoping the model behaves.

Logging and review

Every reply should be recorded

log({"msg": message, "reply": reply, "confidence": score})

Logging lets you review bad replies and improve the system over time.

Logged replies -> weekly review -> better rules

You cannot fix what you do not measure.

The human escape hatch

Always let a person step in

if 'speak to a person' in message or confidence < 0.5:
    connect_to_agent()
else:
    send_auto_reply()

An escape hatch lets users reach a human whenever they want, or when the bot is unsure. It builds trust and catches mistakes.

Why it works: people feel safe knowing a real person is one click away.

Words this lesson uses

Each one is defined here on this site. Open the entry for the full definition and the other words it points at.

← How auto-reply works
Previous Lesson
How auto-reply works
Next Lesson
Intent & entity extraction
Intent & entity extraction →