Basic

Computing a new column

Derived columns carry the analysis forward: ratio, difference, category. Compute them once, store them, and reuse. Recomputing the same expression in five places is how five subtly different answers end up in one report.

What you will see

1.0 (or 0.9999999999999999). If it is not near 1, the shares are not shares of the same total.

About 2 min of reading

Try It

Add a share column and check it sums to one.

1

Compute the share

total = sum(int(r['count']) for r in rows) for r in rows: r['share'] = int(r['count']) / total print([r['share'] for r in rows])

Before you run it, say out loud what it will print. Then open the answer.

What it prints
[0.136..., 0.545..., 0.318...].
2

Prove it is consistent

print(sum(r['share'] for r in rows))

Before you run it, say out loud what it will print. Then open the answer.

What it prints
1.0 (or 0.9999999999999999). If it is not near 1, the shares are not shares of the same total.

Put it in order

Before you write anything, put these lines back in the right order. Click a line to move it up one step. This support step is easier than writing code from scratch, and it is here to make the last example stick.

Data Flow

A rule turns each row into a new value.
1 Decide the formula once
↓
2 Apply it to every row
↓
3 Store the result on the row
↓
4 Round only when printing
Line by line

Read the first example one line at a time. A line that prints shows what appears on screen; the rest set things up quietly.

total = sum(int(r['count']) for r in rows) sets nothing on screen
for r in rows: sets nothing on screen
r['share'] = int(r['count']) / total sets nothing on screen
print([r['share'] for r in rows]) [0.136..., 0.545..., 0.318...].
Why this works
Name the column after the question

A column called 'share' beats 'x2'. When the column name states the question it answers, the analysis reads like an argument and reviewers can check each step against the claim it supports.

One definition per idea

If two places compute 'is_over_180_days', one of them will eventually drift. Put the rule in one function and call it. The bug you are looking for is not a wrong number, it is two different numbers.

Examples

Add the column to the row

for r in rows: r['share'] = int(r['count']) / total. Now every later step sees the same value. In JavaScript, rows.forEach(r => r.share = r.count / total) or map to new objects.

Guard the denominator

Division by zero from an empty group is a crash or an infinity that prints as 'inf' and quietly makes your total wrong. Test the denominator and decide what the answer should be when it is zero: usually 0 or None, never a silent infinity.

Round at the end, not in the middle

Rounding a derived column early means the parts no longer add up to the whole. Keep full precision through the pipeline and round only when you print. This is also why percentages should be reported as fractions of the same denominator.

Words this lesson uses

Each one is defined here on this site. Open the entry for the full definition and the other words it points at.

← Selecting and filtering rows
Previous Lesson
Selecting and filtering rows
Next Lesson
Grouping and aggregating
Grouping and aggregating →