Advanced

Threads and the GIL

A thread is a second stream of execution inside one process. Python gives threads a shared gate, which changes what runs truly at once and what does not.

What you will see

True

About 2 min of reading

Quick check

Before this new lesson, one small question about the lesson you just finished. Recall it, do not look it up.

Try It

Check what a thread hands back.

Tag printed.

import threading out = [] t = threading.Thread(target=lambda: out.append("a")) t.start() t.join() print(out[0])

Before you run it, say out loud what it will print. Then open the answer.

What it prints

Threads counted.

import threading threads = [threading.Thread(target=lambda: None) for _ in range(2)] for t in threads: t.start() for t in threads: t.join() print(len(threads))

Before you run it, say out loud what it will print. Then open the answer.

What it prints

Workers picked.

workers = min(4, 8) print(workers)

Before you run it, say out loud what it will print. Then open the answer.

What it prints

Cores found.

import os print(os.cpu_count() > 0)

Before you run it, say out loud what it will print. Then open the answer.

What it prints

True

Put it in order

Before you write anything, put these lines back in the right order. Click a line to move it up one step. This support step is easier than writing code from scratch, and it is here to make the last example stick.

Data Flow

Threads are started, each runs its own stream, the gate lets one Python body at a time, and joins collect the finished work.

1 start each thread

↓

2 the gate picks one

↓

3 each body runs

↓

4 joins collect results

Line by line

Read the first example one line at a time. A line that prints shows what appears on screen; the rest set things up quietly.

import threading sets nothing on screen

out = [] sets nothing on screen

t = threading.Thread(target=lambda: out.append("a")) sets nothing on screen

t.start() sets nothing on screen

t.join() sets nothing on screen

print(out[0]) a

Why this works

The gate is one at a time

Threads let you express parallel work, but the gate lets only one run Python at a time. Measure before adding workers.

Waiting is not gated

A thread that spends its time waiting for a socket or a file is not holding the gate. That is where threads pay off in plain Python.

Examples

Two threads started

import threading

def show(tag):
    print(tag)

a = threading.Thread(target=show, args=("a",))
b = threading.Thread(target=show, args=("b",))
a.start()
b.start()
a.join()
b.join()

Both bodies run in their own stream. The join waits for each one to finish.

A thread that returns

import threading

out = []

def show(tag):
    out.append(tag)

t = threading.Thread(target=show, args=("a",))
t.start()
t.join()
print(out[0])

A thread has no return channel. The value comes back through a shared place you provide.

The gate

import threading

threads = [threading.Thread(target=lambda: None) for _ in range(2)]
for t in threads:
    t.start()
for t in threads:
    t.join()
print(len(threads))

Only one thread runs Python bytecode at a time. The gate is why more threads do not mean more speed.

Where threads do help

import time
start = time.time()
time.sleep(0.01)
print(round(time.time() - start, 2) > 0)

Work that waits outside Python, a socket or a subprocess, is not gated. Threads overlap that wait.

Processors instead

import os
print(os.cpu_count() > 0)

A process pool hands whole calls to other interpreters, so the gate stops being the limit.

A thread count to pick

workers = min(4, 8)
print(workers)

More workers than cores buys nothing once the gate is the limit. Pick the smaller number.

Words this lesson uses

Each one is defined here on this site. Open the entry for the full definition and the other words it points at.

← A queue feeds workers

Previous Lesson

A queue feeds workers

Next Lesson

Coordinating two streams

Coordinating two streams →