Performance Best Practices

Performance rules that prevent premature optimization and focus effort on measured bottlenecks.

How to Use This List

Default stance: readability over micro-optimization
Break these rules only with profile data proving the hotspot
Document every optimization with before/after numbers

A - Measurement

Profile before optimizing. cProfile, py-spy, or APM - never guess.
Define SLA before tuning. Know the target p95 latency or throughput.
Benchmark realistic workloads. Production data sizes and concurrency.
Record before/after numbers. Every optimization needs proof.

B - Algorithm & Data

Right algorithm first. O(n log n) beats micro-optimized O(n^2).
Vectorize numeric loops. NumPy/Polars for array and DataFrame ops.
Stream large data. Generators over materialized lists.
Right data structure. set for membership, deque for queues, dict for lookup.

C - Concurrency

asyncio for I/O-bound. httpx, asyncpg, aiofiles.
multiprocessing for CPU-bound. Not threads (GIL).
Cap concurrency. Semaphores and pool limits prevent overload.
No blocking in async handlers. asyncio.to_thread for legacy blocking code.

D - Caching & I/O

Cache with bounds. maxsize and TTL on every cache.
Connection pooling. Reuse DB and HTTP connections.
Batch I/O operations. Bulk inserts, gather HTTP requests.
Efficient serialization. orjson for hot JSON paths.

E - When to Stop

SLA met? Stop optimizing. Further work has diminishing returns.
Readability preserved. Complex optimization needs comment explaining why.
Tests still pass. Correctness before speed.
No premature Cython/Rust. Python is fast enough until profiled otherwise.

FAQs

Is Python slow?

Python is fast enough for most web and data workloads. Optimize hotspots, not the language.

Should I use PyPy?

Only if benchmarks on your specific workload show clear wins.

List comprehension or map?

Readability first. Profile if hot. Differences are usually negligible.

When to use Cython?

When a profiled hotspot cannot be fixed with NumPy, better algorithms, or caching.

How do I prevent performance regression?

Benchmark critical paths in CI with pytest-benchmark. Alert on threshold breach.

asyncio everywhere?

No. Only I/O-bound services that benefit from concurrency. Adds complexity.

What about database performance?

Indexes, query plans, and N+1 elimination often beat application-level tuning.

Should I use __slots__?

For millions of small objects with fixed attributes. Not by default.

How do I set performance budgets?

Define p95 latency per endpoint. Block deploys that exceed budget in load tests.

Micro-optimize string concatenation?

Only in profiled hot loops. f-strings are fine for normal code.

Performance Basics - getting started
Performance Audit Checklist - systematic review
cProfile & Profilers - finding hotspots
Async & Concurrency for Throughput - parallelism

Stack versions: This page was written for Python 3.14.0, FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.

Performance Best Practices

Performance rules that prevent premature optimization and focus effort on measured bottlenecks.

How to Use This List

Default stance: readability over micro-optimization
Break these rules only with profile data proving the hotspot
Document every optimization with before/after numbers

A - Measurement

Profile before optimizing. cProfile, py-spy, or APM - never guess.
Define SLA before tuning. Know the target p95 latency or throughput.
Benchmark realistic workloads. Production data sizes and concurrency.
Record before/after numbers. Every optimization needs proof.

B - Algorithm & Data

Right algorithm first. O(n log n) beats micro-optimized O(n^2).
Vectorize numeric loops. NumPy/Polars for array and DataFrame ops.
Stream large data. Generators over materialized lists.
Right data structure. set for membership, deque for queues, dict for lookup.

C - Concurrency

asyncio for I/O-bound. httpx, asyncpg, aiofiles.
multiprocessing for CPU-bound. Not threads (GIL).
Cap concurrency. Semaphores and pool limits prevent overload.
No blocking in async handlers. asyncio.to_thread for legacy blocking code.

D - Caching & I/O

Cache with bounds. maxsize and TTL on every cache.
Connection pooling. Reuse DB and HTTP connections.
Batch I/O operations. Bulk inserts, gather HTTP requests.
Efficient serialization. orjson for hot JSON paths.

E - When to Stop

SLA met? Stop optimizing. Further work has diminishing returns.
Readability preserved. Complex optimization needs comment explaining why.
Tests still pass. Correctness before speed.
No premature Cython/Rust. Python is fast enough until profiled otherwise.

FAQs

Is Python slow?

Python is fast enough for most web and data workloads. Optimize hotspots, not the language.

Should I use PyPy?

Only if benchmarks on your specific workload show clear wins.

List comprehension or map?

Readability first. Profile if hot. Differences are usually negligible.

When to use Cython?

When a profiled hotspot cannot be fixed with NumPy, better algorithms, or caching.

How do I prevent performance regression?

Benchmark critical paths in CI with pytest-benchmark. Alert on threshold breach.

asyncio everywhere?

No. Only I/O-bound services that benefit from concurrency. Adds complexity.

What about database performance?

Indexes, query plans, and N+1 elimination often beat application-level tuning.

Should I use __slots__?

For millions of small objects with fixed attributes. Not by default.

How do I set performance budgets?

Define p95 latency per endpoint. Block deploys that exceed budget in load tests.

Micro-optimize string concatenation?

Only in profiled hot loops. f-strings are fine for normal code.

Performance Basics - getting started
Performance Audit Checklist - systematic review
cProfile & Profilers - finding hotspots
Async & Concurrency for Throughput - parallelism

Stack versions: This page was written for Python 3.14.0, FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.

Performance Best Practices

Search Documentation

Performance Best Practices