Master

A string from end to end

Text becomes bytes by a rule, bytes become numbers by a layout, and both steps have to agree on both sides. Tracing one word through all of it makes the whole picture concrete.

What you will see

owlww

About 2 min of reading

Quick check

Before this new lesson, one small question about the lesson you just finished. Recall it, do not look it up.

Try It

Build and read a self-describing record.

1

Encode a word to bytes.

print("owlww".encode("utf-8"))

Before you run it, say out loud what it will print. Then open the answer.

What it prints
b'owlww'
2

Show the byte numbers.

print(list("owlww".encode("utf-8")))

Before you run it, say out loud what it will print. Then open the answer.

What it prints
[111, 119, 108, 119, 119]
3

Put a two-byte length in front.

b = "owlww".encode("utf-8") print(len(b).to_bytes(2, "big") + b)

Before you run it, say out loud what it will print. Then open the answer.

What it prints
b'\x00\x05owlww'
4

Read the length and decode the rest.

b = "owlww".encode("utf-8") rec = len(b).to_bytes(2, "big") + b n = int.from_bytes(rec[:2], "big") print(rec[2:2 + n].decode("utf-8"))

Before you run it, say out loud what it will print. Then open the answer.

What it prints
owlww

Put it in order

Before you write anything, put these lines back in the right order. Click a line to move it up one step. This support step is easier than writing code from scratch, and it is here to make the last example stick.

Data Flow

Text is encoded to bytes, a length can be written in front, and a reader that knows both rules recovers the original.
1 text encoded to bytes
↓
2 bytes are numbers
↓
3 a length written in front
↓
4 the reader undoes both
Line by line

Read the first example one line at a time. A line that prints shows what appears on screen; the rest set things up quietly.

print("owlww".encode("utf-8")) b'owlww'
Why this works
Every step is a choice

The encoding, the byte order, and the length width are all choices. A format writes them down once so a reader can undo them exactly.

Where this shows up

Every text file, JSON body, and database string goes through this. When one side picks a different rule, characters change or records line up wrong.

Examples

Word to bytes

print("owlww".encode("utf-8"))

Five letters become five bytes, one each, because they are all plain. The rule is utf-8 and the result is what a file holds.

Bytes to numbers

print(list("owlww".encode("utf-8")))

Each byte is a number from 0 to 255. Those numbers are what a hex editor shows you, and what a checksum runs over.

Length then bytes

b = "owlww".encode("utf-8")
print(len(b).to_bytes(2, "big") + b)

A two-byte length in front makes a self-describing record. The reader knows where the next record starts.

Read it back

b = "owlww".encode("utf-8")
rec = len(b).to_bytes(2, "big") + b
n = int.from_bytes(rec[:2], "big")
print(rec[2:2 + n].decode("utf-8"))

The reader takes the length first, then that many bytes. Same rule both sides, so the word comes back.

Words this lesson uses

Each one is defined here on this site. Open the entry for the full definition and the other words it points at.

← Reading a file as bytes
Previous Lesson
Reading a file as bytes
Next Lesson
Checksums and hashes
Checksums and hashes →