DailyRNG
Login
Advanced

Tuning model behaviour

The same prompt can give wildly different answers. Learn the settings that control how creative or precise the model is, and how to keep answers consistent and safe.

What you will see

max_tokens cap set to 250 top_p limit set to 0.9

About 2 min of reading

Try It

The same prompt can produce wildly different answers, because a setting called temperature controls how random the model is. Low values focus on the single most likely word; high values let it wander. Other settings cap length and narrow the word pool.

1

I print how the model picks a reply style at three temperature values, showing that low is precise and high is creative.

def generate(temperature, top_p, max_tokens): if temperature <= 0.2: return "A precise, factual, consistent answer." if temperature <= 0.7: return "A balanced mix of precise and creative." return "A very imaginative, sometimes surprising answer." print("Tuning model behaviour") print("temperature=0.0 -> %s" % generate(0.0, 1.0, 250)) print("temperature=0.7 -> %s" % generate(0.7, 0.9, 250)) print("temperature=1.5 -> %s" % generate(1.5, 1.0, 250))

Before you run it, say out loud what it will print. Then open the answer.

What it prints
Tuning model behaviour temperature=0.0 -> A precise, factual, consistent answer. temperature=0.7 -> A balanced mix of precise and creative. temperature=1.5 -> A very imaginative, sometimes surprising answer.
There is no perfect setting, only the right one for the job. A calculator bot should sit near 0; a creative writing companion can wander freely.
2

I set the two other controls, printing that max_tokens caps how long a reply can run and top_p narrows the words the model is allowed to pick.

print("max_tokens cap set to 250") print("top_p limit set to 0.9")

Before you run it, say out loud what it will print. Then open the answer.

What it prints
max_tokens cap set to 250 top_p limit set to 0.9
Change one knob at a time and watch the answers shift. The setting that looks best in isolation is rarely the best combination for the app.

Data Flow

Settings like temperature and max tokens shape how the model picks its next words before the reply is generated.
1 App sends the prompt with settings
↓
2 Temperature sets the randomness of sampling
↓
3 top_p narrows the candidate words
↓
4 max_tokens caps the reply length
↓
5 The model picks likely words in order
↓
6 The reply is generated within limits
Line by line

Read the first example one line at a time. A line that prints shows what appears on screen; the rest set things up quietly.

def generate(temperature, top_p, max_tokens): sets nothing on screen
if temperature <= 0.2: sets nothing on screen
return "A precise, factual, consistent answer." sets nothing on screen
if temperature <= 0.7: sets nothing on screen
return "A balanced mix of precise and creative." sets nothing on screen
return "A very imaginative, sometimes surprising answer." sets nothing on screen
print("Tuning model behaviour") Tuning model behaviour
print("temperature=0.0 -> %s" % generate(0.0, 1.0, 250)) temperature=0.0 -> A precise, factual, consistent answer.
print("temperature=0.7 -> %s" % generate(0.7, 0.9, 250)) temperature=0.7 -> A balanced mix of precise and creative.
print("temperature=1.5 -> %s" % generate(1.5, 1.0, 250)) temperature=1.5 -> A very imaginative, sometimes surprising answer.
Why this works
Matching settings to purpose

There is no perfect setting, there is the right one for your app:

  • A coding helper should use low temperature and a max token cap. Predictability matters more than flair.
  • A creative writing companion can use higher temperature for surprises.
  • A factual assistant should stay low and add safety parameters.

Learn how the knobs interact by testing one at a time. Change temperature, then tokens, and watch how the answers shift. That experiment is how you develop an eye for the right values.

Examples

Understanding temperature

The temperature controls randomness. Low values are focused; high values are creative:

const params = {
  model: 'gpt-4o-mini',
  temperature: 0.2,   // low = precise, consistent
  messages: history
};

  • 0: the model always picks the most likely next word. Great for facts and code.
  • 0.7: a balanced mix of precise and creative.
  • 1.5: very imaginative, sometimes surprising.

Choose the setting for the job. A calculator bot should sit near 0.

Max tokens and safety

Two more settings keep a bot well-behaved:

const params = {
  model: 'gpt-4o-mini',
  temperature: 0.4,
  max_tokens: 250,     // cap how long the reply can be
  top_p: 0.9           // limit the pool of possible words
};

  • max_tokens stops one answer from running forever.
  • top_p narrows the words the model may choose, making replies safer and more on-topic.

Tuning these gives you control over length, focus, and tone without rewriting your code every time.

Words this lesson uses

Each one is defined here on this site. Open the entry for the full definition and the other words it points at.

← Streaming responses
Previous Lesson
Streaming responses
Next Lesson
Input and output safety
Input and output safety →