timeit & Microbenchmarks
Microbenchmarks measure small code snippets precisely. The timeit module eliminates common timing pitfalls (garbage collection, setup overhead) for fair comparisons.
Recipe
import timeit
time_a = timeit.timeit("[x*x for x in range(1000)]", number=10000)
time_b = timeit.timeit("list(map(lambda x: x*x, range(1000)))", number=10000)
print(f"comprehension: {time_a:.4f}s, map: {time_b:.4f}s")When to reach for this:
- Comparing two implementations of the same logic
- Validating that an optimization actually helps
- Benchmarking stdlib vs third-party alternatives
Working Example
import timeit
setup = """
from pathlib import Path
data = list(range(10000))
"""
stmt_list = "[x * 2 for x in data]"
stmt_gen = "list(x * 2 for x in data)"
t_list = timeit.timeit(stmt_list, setup=setup, number=1000)
t_gen = timeit.timeit(stmt_gen, setup=setup, number=1000)
print(f"list comp: {t_list:.4f}s, gen: {t_gen:.4f}s, ratio: {t_gen/t_list:.2f}x")# Command line
# python -m timeit "[x*x for x in range(1000)]"
# python -m timeit -s "import math" "math.sqrt(2)"What this demonstrates:
setupruns once;stmtrunsnumbertimes- Fair comparison with identical setup
- CLI
python -m timeitfor quick one-liners - Report ratio, not just absolute times
Deep Dive
Benchmark Rules
- Run on an idle machine (close other apps)
- Increase
numberuntil total time is > 0.2s - Run multiple rounds; report median, not best
- Never benchmark I/O or network in timeit
Common Pitfalls
| Pitfall | Fix |
|---|---|
| GC during timing | timeit disables GC by default |
| Cold cache | Warm up before timing |
| Debug mode | Benchmark without -O unless measuring production |
Tiny number | Increase until timer resolution is negligible |
Gotchas
- Benchmarking debug builds -
-Oor production config differs. Fix: benchmark the deployment configuration. - Single run conclusions - noise dominates. Fix: multiple runs, report median and stddev.
- Comparing different input sizes - O(n) vs O(n^2) crossover. Fix: benchmark at realistic input sizes.
- Optimizing the non-hot path - microbenchmark wins but no real impact. Fix: profile the full application first.
- Dead code elimination - compiler removes unused results. Fix: store or use the result in the benchmark loop.
Alternatives
| Alternative | Use When | Don't Use When |
|---|---|---|
| pytest-benchmark | Regression tracking in CI | Quick one-off comparison |
| cProfile | Finding what to benchmark | Comparing two implementations |
| pyperf | Rigorous statistical benchmarks | Quick checks |
FAQs
timeit vs time.perf_counter?
timeit for isolated snippets. perf_counter for full function timing.
What number should I use?
Increase until total elapsed is 0.2-2 seconds. timeit auto-calibrates with autorange.
Can I benchmark async code?
Use asyncio.run() in setup/stmt or dedicated async benchmark tools.
How do I benchmark with arguments?
Use setup to define data; reference it in stmt string.
Is timeit thread-safe?
Each call is independent. For threaded code, benchmark the full concurrent scenario separately.
python -m timeit examples?
python -m timeit "sum(range(1000))" runs with auto-calibrated repetitions.
How do I prevent GC interference?
timeit disables GC during timing by default.
Should I use %timeit in Jupyter?
Convenient for exploration. Use module timeit for reproducible scripts.
How do I share benchmark results?
Commit a benchmark script with fixed number and document hardware/OS.
When is microbenchmarking misleading?
When the snippet is not the application hotspot. Profile first.
Related
- Performance Basics - measurement mindset
- cProfile & Profilers - finding hotspots
- Performance Best Practices - when to optimize
- Vectorization with NumPy - numeric benchmarks
Stack versions: This page was written for Python 3.14.0, FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.