Master

Checksums and hashes

A checksum is a number computed from the bytes of a file. Store it next to the data and recompute it later, and a changed file no longer matches.

What you will see

a hex string

About 1 min of reading

Quick check

Before this new lesson, one small question about the lesson you just finished. Recall it, do not look it up.

Try It

Compute and compare checksums.

Hash three bytes.

import zlib print(zlib.crc32(b"owl"))

Before you run it, say out loud what it will print. Then open the answer.

What it prints

1289493952

Hash one byte more and compare.

import zlib print(zlib.crc32(b"owl") == zlib.crc32(b"owlx"))

Before you run it, say out loud what it will print. Then open the answer.

What it prints

False

Show a longer fingerprint.

import hashlib print(hashlib.sha256(b"owl").hexdigest()[:16])

Before you run it, say out loud what it will print. Then open the answer.

What it prints

a longer hex string

Fingerprint the probe file.

import hashlib print(hashlib.sha256(open("/tmp/probe.bin", "rb").read()).hexdigest()[:16])

Before you run it, say out loud what it will print. Then open the answer.

What it prints

a hex string

Run this after the earlier lesson wrote that file.

Data Flow

A checksum is a number computed from bytes, and recomputing it later tells you whether the data still matches.

1 bytes hashed to a number

↓

2 same bytes same number

↓

3 one byte changed differs

↓

4 stored next to the data

Line by line

Read the first example one line at a time. A line that prints shows what appears on screen; the rest set things up quietly.

import zlib sets nothing on screen

print(zlib.crc32(b"owl")) 1289493952

Why this works

Checksums catch accidents

A checksum catches a truncated or corrupted file. It is not a security test, since a deliberate change can be made to match.

Hashes name content

A content hash is used as a name, so identical files share storage and any change gives a new name. That is how versions and caches work.

Examples

A number from bytes

import zlib
print(zlib.crc32(b"owl"))

The same bytes always give the same number. One byte changed gives a different number, which is the check.

A stronger hash

import hashlib
print(hashlib.sha256(b"owl").hexdigest()[:16])

A hash is a bigger fingerprint. It is used to name files and to check downloads, not just to detect accidents.

One byte off

import zlib
print(zlib.crc32(b"owl") == zlib.crc32(b"owlx"))

Adding one letter changes the number. A corrupted file fails this comparison, so the damage is caught.

Over a file

import hashlib
h = hashlib.sha256(open("/tmp/probe.bin", "rb").read())
print(h.hexdigest()[:16])

Hashing the bytes of a file gives a fingerprint of the whole file. Same file, same fingerprint.

Words this lesson uses

Each one is defined here on this site. Open the entry for the full definition and the other words it points at.

← A string from end to end

Previous Lesson

A string from end to end

Next Lesson

The bit layout of a real format

The bit layout of a real format →