collections, itertools & functools Recap
Three stdlib modules form Python's data toolkit: collections (specialized containers), itertools (iterator algebra), functools (higher-order functions). Master a small subset before reaching for PyPI.
Recipe
from collections import Counter, defaultdict
from functools import lru_cache
from itertools import islice, chain
@lru_cache(maxsize=128)
def fib(n: int) -> int:
return n if n < 2 else fib(n - 1) + fib(n - 2)
counts = Counter(chain.from_iterable(["a", "b", "a"]))When to reach for this:
- Counting frequencies -> Counter
- Grouping -> defaultdict, itertools.groupby
- Memoization -> lru_cache
- Streaming large data -> itertools chain/islice
- Single-dispatch -> functools.singledispatch
Working Example
from collections import Counter, deque
from functools import partial
from itertools import groupby
def top_n(words: list[str], n: int = 3) -> list[tuple[str, int]]:
return Counter(words).most_common(n)
def sliding_window(iterable, size: int):
window = deque(maxlen=size)
for item in iterable:
window.append(item)
if len(window) == size:
yield tuple(window)
def group_sorted(rows: list[tuple[str, int]]):
for key, group in groupby(sorted(rows), key=lambda r: r[0]):
yield key, list(group)
words = ["api", "api", "cli", "ui", "api"]
print(top_n(words))
print(list(sliding_window(range(6), 3)))
print(list(group_sorted([("a", 1), ("a", 2), ("b", 3)])))
double = partial(pow, 2)
print(double(5))What this demonstrates:
- Counter for tallies without manual dict increments
- deque maxlen for O(1) sliding windows
- groupby requires sorted input for all keys grouped
- partial fixes leftmost callable arguments
Deep Dive
collections Highlights
| Type | Use |
|---|---|
| deque | FIFO/LRU ends |
| Counter | multiset counts |
| defaultdict | default missing keys |
| namedtuple | light records (dataclass often better) |
itertools Highlights
chain,islice,batched(3.12+)product,permutationscombinatorics- Lazy - memory friendly
functools Highlights
lru_cache,cache(3.9+ unbounded)singledispatch,wrapsfor decorators
Gotchas
- groupby unsorted input - splits same key twice. Fix: sort first or use defaultdict.
- lru_cache mutable args - unhashable dict/list params fail. Fix: tuple keys or disable cache.
- infinite itertools without islice - infinite loops. Fix: bound consumption.
- Counter negative counts - allowed but confusing. Fix: use only increment paths.
- premature more-itertools - stdlib batched/islice covers many cases in 3.12+.
Alternatives
| Alternative | Use When | Don't Use When |
|---|---|---|
| dataclasses | Structured records | Pure counting |
| pandas | Columnar stats | Tiny in-memory tallies |
| more-itertools | Extra recipes | Policy stdlib-only |
FAQs
Counter most_common?
Efficient heap for top-k frequencies.
deque vs list queue?
deque O(1) pops from left; list pop(0) is O(n).
cache vs lru_cache?
cache unbounded - only for finite small argument space.
itertools.batched?
3.12+ chunks iterable into tuples of size n - replaces manual zip loops.
namedtuple still?
Immutable light tuples - dataclasses often clearer for new code.
reduce?
Available but explicit loops often more readable in Python.
chain.from_iterable?
Flatten list of lists without copy.
singledispatchmodule?
See pythonic patterns page for type-based dispatch.
Performance?
itertools lazy; Counter C-implemented - fast for typical sizes.
Where deeper coverage?
See data-structures and iterators-generators sections for extended treatment.
Related
- dataclasses, enum & contextlib - records
- The Strategy & Dispatch Patterns - singledispatch
- Standard Library Overview - map
- Standard Library Best Practices - toolkit rules
Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.