itertools
itertools provides fast, memory-efficient iterator building blocks - chaining, slicing, combinatorics, and grouping without intermediate lists.
Recipe
from itertools import chain, islice, product
flat = chain([1, 2], [3, 4])
first_three = list(islice(range(10), 3))
pairs = list(product(["A", "B"], [1, 2]))When to reach for this:
- Flattening nested iterables
- Pagination via
isliceon infinite counter - Grouping sorted log lines by key
- Cartesian products for test case generation
Working Example
from itertools import chain, groupby, islice, accumulate
def group_sorted(rows: list[tuple[str, int]]):
for key, group in groupby(rows, key=lambda r: r[0]):
yield key, sum(v for _, v in group)
def paginate(iterator, page_size: int):
page = 0
while True:
chunk = list(islice(iterator, page_size))
if not chunk:
break
yield page, chunk
page += 1
def running_totals(values):
return accumulate(values)
if __name__ == "__main__":
rows = [("a", 1), ("a", 2), ("b", 3)]
print(list(group_sorted(sorted(rows))))
print(list(paginate(iter(range(7)), 3)))
print(list(running_totals([1, 2, 3, 4])))What this demonstrates:
groupbyneeds sort by same key function firstislicetakes from any iterator without len()paginatepattern for streaming APIsaccumulateyields running reductions (default sum)
Deep Dive
How It Works
- Iterator algebra - Functions consume iterables, return iterators.
- groupby - Groups consecutive equal keys only - not SQL GROUP BY.
- tee - Splits iterator into independent iterators with buffered storage.
- cycle/repeat/count - Infinite iterators - always bound consumer loop.
- combinations/permutations - Combinatorial generation lazily.
Frequently Used
| Function | Purpose |
|---|---|
chain | Concatenate iterables |
islice | Slice iterator |
groupby | Consecutive grouping |
product | Cartesian product |
zip_longest | Zip unequal lengths |
Python Notes
from itertools import pairwise # 3.10+
for a, b in pairwise([1, 2, 3]):
print(a, b)Gotchas
- groupby without sort - Splits groups incorrectly when keys not contiguous. Fix:
sorted(data, key=key)first. - tee memory - Buffers values for secondary iterators. Fix: Use sparingly on large streams.
- infinite product -
product(count(), repeat=2)never ends. Fix: islice bound. - chain.from_iterable - Flattens one level - deeper nesting needs recursion or nested chain.
- zip vs zip_longest - zip stops at shortest; zip_longest needs fillvalue for padding.
Alternatives
| Alternative | Use When | Don't Use When |
|---|---|---|
| nested for loops | Two small iterables | Many combinations - itertools clearer |
| list comprehensions | Materialize small result | Large cross product |
| pandas groupby | DataFrame analytics | Plain Python tuples |
| more-itertools | Extra recipes (chunked) | Stdlib-only constraint |
FAQs
Why sort before groupby?
groupby only groups adjacent equal keys - sorting clusters equal keys together.
islice vs slicing list?
islice works on iterators without random access or len().
chain vs yield from?
chain for known iterables at once; yield from for generator delegation with protocol.
product memory?
Lazy - generates tuples on demand. Materializing huge product explodes memory.
accumulate custom func?
accumulate(values, func=operator.mul) for running product etc.
pairwise availability?
stdlib 3.10+ - else zip(items, items[1:]) pattern.
starmap?
Like map but unpacks tuple args - starmap(pow, [(2,3), (3,2)]).
filterfalse?
Opposite of filter - elements where pred false.
tee when?
Need two passes over iterator without storing all - pays buffer memory.
combinations vs permutations?
combinations order-insensitive subsets; permutations order matters.
Related
- Generators & yield - custom iterators
- functools - reduce related to accumulate
- Comprehensions & Generator Expressions - genexp
- collections Module - Counter with iterables
Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.