Performance Best Practices
Performance rules that prevent premature optimization and focus effort on measured bottlenecks.
How to Use This List
- Default stance: readability over micro-optimization
- Break these rules only with profile data proving the hotspot
- Document every optimization with before/after numbers
A - Measurement
- Profile before optimizing. cProfile, py-spy, or APM - never guess.
- Define SLA before tuning. Know the target p95 latency or throughput.
- Benchmark realistic workloads. Production data sizes and concurrency.
- Record before/after numbers. Every optimization needs proof.
B - Algorithm & Data
- Right algorithm first. O(n log n) beats micro-optimized O(n^2).
- Vectorize numeric loops. NumPy/Polars for array and DataFrame ops.
- Stream large data. Generators over materialized lists.
- Right data structure. set for membership, deque for queues, dict for lookup.
C - Concurrency
- asyncio for I/O-bound. httpx, asyncpg, aiofiles.
- multiprocessing for CPU-bound. Not threads (GIL).
- Cap concurrency. Semaphores and pool limits prevent overload.
- No blocking in async handlers.
asyncio.to_threadfor legacy blocking code.
D - Caching & I/O
- Cache with bounds. maxsize and TTL on every cache.
- Connection pooling. Reuse DB and HTTP connections.
- Batch I/O operations. Bulk inserts, gather HTTP requests.
- Efficient serialization. orjson for hot JSON paths.
E - When to Stop
- SLA met? Stop optimizing. Further work has diminishing returns.
- Readability preserved. Complex optimization needs comment explaining why.
- Tests still pass. Correctness before speed.
- No premature Cython/Rust. Python is fast enough until profiled otherwise.
FAQs
Is Python slow?
Python is fast enough for most web and data workloads. Optimize hotspots, not the language.
Should I use PyPy?
Only if benchmarks on your specific workload show clear wins.
List comprehension or map?
Readability first. Profile if hot. Differences are usually negligible.
When to use Cython?
When a profiled hotspot cannot be fixed with NumPy, better algorithms, or caching.
How do I prevent performance regression?
Benchmark critical paths in CI with pytest-benchmark. Alert on threshold breach.
asyncio everywhere?
No. Only I/O-bound services that benefit from concurrency. Adds complexity.
What about database performance?
Indexes, query plans, and N+1 elimination often beat application-level tuning.
Should I use __slots__?
For millions of small objects with fixed attributes. Not by default.
How do I set performance budgets?
Define p95 latency per endpoint. Block deploys that exceed budget in load tests.
Micro-optimize string concatenation?
Only in profiled hot loops. f-strings are fine for normal code.
Related
- Performance Basics - getting started
- Performance Audit Checklist - systematic review
- cProfile & Profilers - finding hotspots
- Async & Concurrency for Throughput - parallelism
Stack versions: This page was written for Python 3.14.0, FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.