Advanced

CSV rows

CSV is lines of fields with a few rules. Quoting and separators are where a hand-rolled split quietly loses data.

What you will see

About 2 min of reading

Quick check

Before this new lesson, one small question about the lesson you just finished. Recall it, do not look it up.

Try It

Check the csv rules.

Plain row split.

row = "a,b,c" print(len(row.split(",")))

Before you run it, say out loud what it will print. Then open the answer.

What it prints

Quoted field count.

import csv row = "a,\"b,c\",d" parts = next(csv.reader([row])) print(len(parts))

Before you run it, say out loud what it will print. Then open the answer.

What it prints

Semicolon separator.

import csv row = "a;b" parts = next(csv.reader([row], delimiter=";")) print(len(parts))

Before you run it, say out loud what it will print. Then open the answer.

What it prints

Trailing comma.

import csv parts = next(csv.reader(["a,b,"])) print(len(parts))

Before you run it, say out loud what it will print. Then open the answer.

What it prints

Put it in order

Before you write anything, put these lines back in the right order. Click a line to move it up one step. This support step is easier than writing code from scratch, and it is here to make the last example stick.

Data Flow

CSV is lines of fields where quoting protects the separator. Read with a real reader and write with a real writer.

1 read a row

↓

2 respect quoting

↓

3 use header names

↓

4 write with quoting

Line by line

Read the first example one line at a time. A line that prints shows what appears on screen; the rest set things up quietly.

row = "a,b,c" sets nothing on screen

print(len(row.split(","))) 3

Why this works

Never hand-split a row

A quoted field can contain the separator, a newline, or a quote. Only a reader that knows the rules handles all three.

Names beat positions

Read by header name so adding a column does not move everything. Position code breaks silently on the first change.

Examples

Split on a comma

row = "a,b,c"
parts = row.split(",")
print(len(parts))

Three fields for a plain row. A quoted comma inside a field breaks this.

A quoted comma

import csv
row = "a,\"b,c\",d"
parts = next(csv.reader([row]))
print(len(parts))

Quoting protects the separator. A plain split gives four parts instead of three.

Fields by name

import csv
rows = [{"a": 1, "b": 2}]
print(rows[0]["b"])

A header names each field. Index positions shift when a column is added.

A separator change

import csv
row = "a;b"
parts = next(csv.reader([row], delimiter=";"))
print(len(parts))

The separator is a parameter. Files from other tools often use a semicolon.

A trailing comma

import csv
parts = next(csv.reader(["a,b,"]))
print(len(parts))

A trailing comma means an empty last field. The count is three, not two.

Write with quoting

import csv
out = ",".join(["a", "b c"])
print(out)

A field with a separator needs quoting on write. Let the writer add it.

Words this lesson uses

Each one is defined here on this site. Open the entry for the full definition and the other words it points at.

← Reading JSON files

Previous Lesson

Reading JSON files

Next Lesson

Streaming a big file

Streaming a big file →