Iterators & the Iterator Protocol
Iteration is everywhere in Python - for loops, comprehensions, unpacking, and sum. Objects participate by implementing the iterator protocol or delegating to generators.
Recipe
class Countdown:
def __init__(self, start: int) -> None:
self.current = start
def __iter__(self):
return self
def __next__(self) -> int:
if self.current < 0:
raise StopIteration
value = self.current
self.current -= 1
return value
for n in Countdown(3):
print(n)When to reach for this:
- Custom sequence traversal without storing full list
- Parsing streaming data chunk by chunk
- Lazy transformation pipelines
- Integrating with
itertoolsandforloops
Working Example
from collections.abc import Iterable, Iterator
class CSVLines:
def __init__(self, lines: Iterable[str]) -> None:
self._lines = iter(lines)
def __iter__(self) -> Iterator[list[str]]:
for raw in self._lines:
yield [cell.strip() for cell in raw.split(",")]
def take(iterator: Iterator[int], n: int) -> list[int]:
result: list[int] = []
for _ in range(n):
try:
result.append(next(iterator))
except StopIteration:
break
return result
if __name__ == "__main__":
rows = CSVLines(["a,b", "c,d"])
for row in rows:
print(row)
print(take(iter(range(10)), 3))What this demonstrates:
- Iterable can return generator from
__iter__(generator is iterator) iter()obtains iterator;next()pulls items untilStopIterationtakemanually consumes iterator with bounded read- Separating iterable (replayable via new iter) vs iterator (single pass)
Deep Dive
How It Works
- Iterable - Implements
__iter__returning iterator. - Iterator - Implements
__iter__(returns self) and__next__. - StopIteration - Signals end; must not be caught broadly around
for. - Generator functions -
yieldbuilds iterator automatically. - iter(callable, sentinel) - Calls callable until sentinel value.
Protocol Summary
| Object | Required methods |
|---|---|
| Iterable | __iter__ |
| Iterator | __iter__, __next__ |
Python Notes
# iter with sentinel for binary chunks
with open("data.bin", "rb") as fh:
for chunk in iter(lambda: fh.read(4096), b""):
process(chunk)Gotchas
- Iterable conflated with iterator - Iterator exhausted after one
for. Fix: Calliter()again only on iterables. - Raising StopIteration in generator - In generators use bare
return; in__next__raise StopIteration. - Mutable iterable during iteration - Dict/list size changes error. Fix: Snapshot or defer deletes.
- getitem legacy iteration - Old sequence protocol; prefer
__iter__. - Infinite iterator without break - Hangs consumer. Fix: Document or provide
take/islice.
Alternatives
| Alternative | Use When | Don't Use When |
|---|---|---|
| Generator function | Most custom iteration | Need class state beyond yield |
itertools | Standard iterator algebra | One-off simple loop |
list materialize | Need reuse/random access | Streams too large |
| async iterator | Async for with I/O | Sync-only pipeline |
FAQs
What is the difference between iterable and iterator?
Iterable produces iterators. Iterator yields items once until exhausted.
Why StopIteration?
Signals end of iteration to for loop machinery - not meant for general flow control in app code.
Can I iterate twice?
Only if object is iterable returning fresh iterator each iter() call. Generators exhausted after one pass.
What does for loop do?
it = iter(obj); while True: try: x = next(it) ... except StopIteration: break
iter() two-arg form?
Repeatedly calls callable until sentinel - useful for stream reads.
collections.abc Iterable?
Structural isinstance check - custom classes can register or implement __iter__.
generator vs iterator class?
Generators faster to write; classes when complex state or multiple methods needed.
enumerate zip iteration?
Built-ins return iterators - consume once, memory efficient.
async iteration?
__aiter__/__anext__ with async for - see asyncio section.
testing iterators?
list(iterator) materialize expected; assert StopIteration on extra next calls.
Related
- Generators & yield - yield-based iterators
- itertools - iterator utilities
- Comprehensions & Generator Expressions - genexp syntax
- Abstract Base Classes - Iterable ABC
Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.