Concurrency Best Practices
Safety and operability rules for threads, processes, and bridges to async code.
How to Use This List
- Apply before increasing
max_workersor task fan-out. - Load-test shutdown paths - races appear on deploy/restart.
- Prefer message passing; treat shared memory as guilty until proven safe.
A - Model Selection
- Profile before picking asyncio vs threads vs processes. Labels deceive without timing data.
- Keep CPU work off asyncio event loop thread.
to_threador process pool. - Use ProcessPool for CPU-bound pure Python on GIL builds. Do not expect thread linear speedup.
- Size thread pools to I/O concurrency limits. Connection pools, not
cpu_count() * 5by default. - Document model choice in ADR. Future maintainers need the why.
B - Shared State
- Prefer queues over shared mutable structures. Producer/consumer simplifies reasoning.
- Protect multi-step invariants with locks. Single bytecode ops are not whole transactions.
- Acquire locks in fixed global order. Document order to prevent deadlocks.
- Avoid global singletons mutated by workers. Pass explicit context or use
threading.local(). - Use immutable messages between stages. Snapshots, not live object graphs.
C - Processes & Pools
- Guard multiprocessing entry with
if __name__ == "__main__". Required on spawn platforms. - Keep worker targets picklable top-level functions. No lambdas in ProcessPool.
- Set
maxtasksperchildfor leaky C extensions. Long-running pools recycle workers. - Cap workers by memory budget. RSS × workers must fit container limit.
- Configure logging in worker initializer. Avoid inherited broken handlers.
D - Shutdown & Errors
- Join threads with timeout during shutdown. Log stragglers; avoid daemon mid-critical-section.
- Use queue sentinels per consumer. One poison pill per worker thread/process.
- Call
task_donefor everyqueue.getin threading.Queue pipelines. Orjoinhangs. - Handle Future exceptions at
result(). Silent failed futures lose errors. - Cancel outstanding futures on timeout. Do not leak background work.
E - Async Bridge
- Never call blocking I/O inside coroutines without executor. Blocks all tasks.
- Share one loop per thread. Do not call
asyncio.runrepeatedly in hot servers use lifespan. - Propagate contextvars to thread workers when needed. Request ids for logs.
- Test concurrent code under stress. Timing-only tests flake; use barriers/events.
- Revisit concurrency model when dependencies go async. Thread pools may become unnecessary.
FAQs
Top production incident cause?
Shared mutable cache without locks plus concurrent workers - use queues or proper synchronization.
How many processes?
Start at cpu_count(), reduce if memory per worker is large.
Is asyncio always safer?
Safer from data races on one thread, but blocking-the-loop causes total outage - different failure mode.
Should I use threading.local?
For per-request state in sync WSGI/threaded servers - not a substitute for proper async contextvars in ASGI.
How to debug deadlocks?
Dump all thread stacks (faulthandler.dump_traceback_all) during staging hang reproduction.
Are atomic integers enough?
itertools and operator won't fix compound business invariants - use locks or single-thread ownership.
Process pool in async app?
loop.run_in_executor(process_pool, fn) - keep pool lifecycle tied to app startup/shutdown.
Gevent legacy?
Maintain if required; greenfield FastAPI/asyncio avoids monkey-patching stdlib.
Load test focus?
Shutdown, burst enqueue, and slow consumer backpressure - not only steady-state throughput.
Free-threaded deployment?
Re-audit all practices - data races on dict/list become more likely without GIL serialization.
Related
- Choosing a Concurrency Model - pick model
- threading - locks detail
- Queues & Producer/Consumer - pipelines
- Async Best Practices - async-specific rules
Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.