collections, itertools & functools Recap

Q: cache vs lru_cache?

cache unbounded - only for finite small argument space.

Three stdlib modules form Python's data toolkit: collections (specialized containers), itertools (iterator algebra), functools (higher-order functions). Master a small subset before reaching for PyPI.

Recipe

from collections import Counter, defaultdict
from functools import lru_cache
from itertools import islice, chain
 
@lru_cache(maxsize=128)
def fib(n: int) -> int:
    return n if n < 2 else fib(n - 1) + fib(n - 2)
 
counts = Counter(chain.from_iterable(["a", "b", "a"]))

When to reach for this:

Counting frequencies -> Counter
Grouping -> defaultdict, itertools.groupby
Memoization -> lru_cache
Streaming large data -> itertools chain/islice
Single-dispatch -> functools.singledispatch

Working Example

from collections import Counter, deque
from functools import partial
from itertools import groupby
 
def top_n(words: list[str], n: int = 3) -> list[tuple[str, int]]:
    return Counter(words).most_common(n)
 
def sliding_window(iterable, size: int):
    window = deque(maxlen=size)
    for item in iterable:
        window.append(item)
        if len(window) == size:
            yield tuple(window)
 
def group_sorted(rows: list[tuple[str, int]]):
    for key, group in groupby(sorted(rows), key=lambda r: r[0]):
        yield key, list(group)
 
words = ["api", "api", "cli", "ui", "api"]
print(top_n(words))
print(list(sliding_window(range(6), 3)))
print(list(group_sorted([("a", 1), ("a", 2), ("b", 3)])))
 
double = partial(pow, 2)
print(double(5))

What this demonstrates:

Counter for tallies without manual dict increments
deque maxlen for O(1) sliding windows
groupby requires sorted input for all keys grouped
partial fixes leftmost callable arguments

Deep Dive

collections Highlights

Type	Use
deque	FIFO/LRU ends
Counter	multiset counts
defaultdict	default missing keys
namedtuple	light records (dataclass often better)

itertools Highlights

chain, islice, batched (3.12+)
product, permutations combinatorics
Lazy - memory friendly

functools Highlights

lru_cache, cache (3.9+ unbounded)
singledispatch, wraps for decorators

Gotchas

groupby unsorted input - splits same key twice. Fix: sort first or use defaultdict.
lru_cache mutable args - unhashable dict/list params fail. Fix: tuple keys or disable cache.
infinite itertools without islice - infinite loops. Fix: bound consumption.
Counter negative counts - allowed but confusing. Fix: use only increment paths.
premature more-itertools - stdlib batched/islice covers many cases in 3.12+.

Alternatives

Alternative	Use When	Don't Use When
dataclasses	Structured records	Pure counting
pandas	Columnar stats	Tiny in-memory tallies
more-itertools	Extra recipes	Policy stdlib-only

FAQs

Counter most_common?

Efficient heap for top-k frequencies.

deque vs list queue?

deque O(1) pops from left; list pop(0) is O(n).

cache vs lru_cache?

cache unbounded - only for finite small argument space.

itertools.batched?

3.12+ chunks iterable into tuples of size n - replaces manual zip loops.

namedtuple still?

Immutable light tuples - dataclasses often clearer for new code.

reduce?

Available but explicit loops often more readable in Python.

chain.from_iterable?

Flatten list of lists without copy.

singledispatchmodule?

See pythonic patterns page for type-based dispatch.

Performance?

itertools lazy; Counter C-implemented - fast for typical sizes.

Where deeper coverage?

See data-structures and iterators-generators sections for extended treatment.

dataclasses, enum & contextlib - records
The Strategy & Dispatch Patterns - singledispatch
Standard Library Overview - map
Standard Library Best Practices - toolkit rules

Python Standard Library Reference - Complete documentation for built-in modules and functions.
Official Python Documentation - The definitive source (Tutorial, Library Reference, Language Reference, HOWTOs).
Real Python - In-depth tutorials, courses, and practical guides.
Fluent Python (2nd Edition) - Intermediate to advanced Python idioms (Luciano Ramalho).
The Quick Python Book (4th Edition) - Concise Python overview (Naomi Ceder).

Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.

Recipe

from collections import Counter, defaultdict
from functools import lru_cache
from itertools import islice, chain
 
@lru_cache(maxsize=128)
def fib(n: int) -> int:
    return n if n < 2 else fib(n - 1) + fib(n - 2)
 
counts = Counter(chain.from_iterable(["a", "b", "a"]))

When to reach for this:

Counting frequencies -> Counter
Grouping -> defaultdict, itertools.groupby
Memoization -> lru_cache
Streaming large data -> itertools chain/islice
Single-dispatch -> functools.singledispatch

Working Example

from collections import Counter, deque
from functools import partial
from itertools import groupby
 
def top_n(words: list[str], n: int = 3) -> list[tuple[str, int]]:
    return Counter(words).most_common(n)
 
def sliding_window(iterable, size: int):
    window = deque(maxlen=size)
    for item in iterable:
        window.append(item)
        if len(window) == size:
            yield tuple(window)
 
def group_sorted(rows: list[tuple[str, int]]):
    for key, group in groupby(sorted(rows), key=lambda r: r[0]):
        yield key, list(group)
 
words = ["api", "api", "cli", "ui", "api"]
print(top_n(words))
print(list(sliding_window(range(6), 3)))
print(list(group_sorted([("a", 1), ("a", 2), ("b", 3)])))
 
double = partial(pow, 2)
print(double(5))

What this demonstrates:

Counter for tallies without manual dict increments
deque maxlen for O(1) sliding windows
groupby requires sorted input for all keys grouped
partial fixes leftmost callable arguments

Deep Dive

collections Highlights

Type	Use
deque	FIFO/LRU ends
Counter	multiset counts
defaultdict	default missing keys
namedtuple	light records (dataclass often better)

itertools Highlights

chain, islice, batched (3.12+)
product, permutations combinatorics
Lazy - memory friendly

functools Highlights

lru_cache, cache (3.9+ unbounded)
singledispatch, wraps for decorators

Gotchas

groupby unsorted input - splits same key twice. Fix: sort first or use defaultdict.
lru_cache mutable args - unhashable dict/list params fail. Fix: tuple keys or disable cache.
infinite itertools without islice - infinite loops. Fix: bound consumption.
Counter negative counts - allowed but confusing. Fix: use only increment paths.
premature more-itertools - stdlib batched/islice covers many cases in 3.12+.

Alternatives

Alternative	Use When	Don't Use When
dataclasses	Structured records	Pure counting
pandas	Columnar stats	Tiny in-memory tallies
more-itertools	Extra recipes	Policy stdlib-only

FAQs

Counter most_common?

Efficient heap for top-k frequencies.

deque vs list queue?

deque O(1) pops from left; list pop(0) is O(n).

cache vs lru_cache?

cache unbounded - only for finite small argument space.

itertools.batched?

3.12+ chunks iterable into tuples of size n - replaces manual zip loops.

namedtuple still?

Immutable light tuples - dataclasses often clearer for new code.

reduce?

Available but explicit loops often more readable in Python.

chain.from_iterable?

Flatten list of lists without copy.

singledispatchmodule?

See pythonic patterns page for type-based dispatch.

Performance?

itertools lazy; Counter C-implemented - fast for typical sizes.

Where deeper coverage?

See data-structures and iterators-generators sections for extended treatment.

dataclasses, enum & contextlib - records
The Strategy & Dispatch Patterns - singledispatch
Standard Library Overview - map
Standard Library Best Practices - toolkit rules

Python Standard Library Reference - Complete documentation for built-in modules and functions.
Official Python Documentation - The definitive source (Tutorial, Library Reference, Language Reference, HOWTOs).
Real Python - In-depth tutorials, courses, and practical guides.
Fluent Python (2nd Edition) - Intermediate to advanced Python idioms (Luciano Ramalho).
The Quick Python Book (4th Edition) - Concise Python overview (Naomi Ceder).

Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.

collections, itertools & functools Recap

Search Documentation

collections, itertools & functools Recap