Consume large data lazily with generators and the yield keyword.
What you will see
[0, 4, 16]
About 2 min of reading
Quick check
Before this new lesson, one small question about the lesson you just finished. Recall it, do not look it up.
Try It
Produce lazy sequences with generators and the `yield` keyword. Each step runs a generator and prints the values it yields one at a time.
1
Write a generator that counts down with yield
def countdown(n):
while n > 0:
yield n
n -= 1
for x in countdown(3):
print(x)
Before you run it, say out loud what it will print. Then open the answer.
What it prints
3
2
1
2
Compare a list comprehension with a generator expression
n = 5
mylist = [x * x for x in range(n)] # list
mygen = (x * x for x in range(n)) # generator
print(mylist)
print(type(mygen))
Before you run it, say out loud what it will print. Then open the answer.
What it prints
0
1
4
9
16
3
Pull values one at a time from a generator
squares = (x * x for x in range(10))
print(next(squares))
print(next(squares))
print(next(squares))
Before you run it, say out loud what it will print. Then open the answer.
What it prints
0
1
4
4
Chain generators together with yield from
def nums():
yield from range(3)
def doubled(gen):
for n in gen:
yield n * 2
for v in doubled(nums()):
print(v)
Before you run it, say out loud what it will print. Then open the answer.
What it prints
0
2
4
5
Build a pipeline: squares then filter even results
def gen():
for i in range(5):
yield i
squares = (x * x for x in gen())
even = [x for x in squares if x % 2 == 0]
print(even)
Before you run it, say out loud what it will print. Then open the answer.
What it prints
[0, 4, 16]
Put it in order
Before you write anything, put these lines back in the right order. Click a line to move it up one step. This support step is easier than writing code from scratch, and it is here to make the last example stick.
Data Flow
A generator pauses at each yield, producing one value per next() call, keeping memory flat.
1Iterator created but does not run yet
↓
2next() runs body up to yield -> returns value
↓
3Call next() again -> resumes after yield
↓
4StopIteration when the function returns
Why this worksLazy evaluation benefits
Generators compute values on demand. For huge or infinite sequences this avoids building large lists in memory. A list comprehension [x for x in range(109)] would exhaust RAM; a generator (x for x in range(109)) streams it.
The iterator protocol
An iterator implements __iter__ and __next__. for calls iter() then repeatedly calls next() until StopIteration. Custom classes can be iterable by defining __iter__ to return an iterator, enabling generators to slot into any for loop or built-in like sum().
Examples
Creating a generator
Lazy sequences
defcountdown(n):
while n > 0:
yield n
n -= 1for x incountdown(3):
print(x) # 3 2 1
yield produces values one at a time without storing the whole sequence in memory.
Generator expressions
Comprehension, lazy
squares = (x * x for x inrange(1000000))
print(next(squares)) # 0print(next(squares)) # 1
Parentheses make a generator expression, same syntax as a comprehension but lazy, so it uses almost no memory.
Chaining generators
Pipeline data
defnums():
yieldfromrange(5)
defdoubled(gen):
for n in gen:
yield n * 2print(list(doubled(nums()))) # [0, 2, 4, 6, 8]
yield from delegates to another iterator, letting you compose generators into pipelines.
Words this lesson uses
Each one is defined here on this site. Open the entry for the full definition and the other words it points at.