DailyRNG

LLM Evals

LLM Evals

Every lesson in order, step by step. No difficulty tiers. Just follow the arrows.

15 lessons, about 16 min of reading

1

Why a model needs a score, not an opinion →

Step 1 · Basic

2

The golden set →

Step 2 · Basic

3

One assertion against one output →

Step 3 · Basic

4

Grading with a rubric instead of a string →

Step 4 · Basic

5

Scoring a whole run →

Step 5 · Basic

6

Comparing two runs honestly →

Step 6 · Basic

7

Eval set drift →

Step 7 · Basic

8

Pairwise comparison →

Step 8 · Advanced

9

Regression evals →

Step 9 · Advanced

10

Grading faithfulness →

Step 10 · Advanced

11

Budgeting an eval run →

Step 11 · Advanced

12

The eval gate →

Step 12 · Advanced

13

A harness that survives a model swap →

Step 13 · Master

14

Instrumenting the eval itself →

Step 14 · Master

15

Shipping a model change with its number →

Step 15 · Master