Choosing a Concurrency Model
Pick asyncio, threads, or processes based on workload shape, library APIs, and operability - not hype. This checklist walks from profiling data to a defensible choice.
How to Use This Checklist
- Profile first (
py-spy,cProfile, APM) - labels lie. - Document the choice in an ADR when the service is long-lived.
- Revisit after dependency changes (sync SDK to async).
Workload Classification
- Measure dominant wait: If >60% time waits on I/O, treat as I/O-bound; if CPU saturates cores, CPU-bound.
- Library API shape: Native async client -> asyncio; blocking only -> threads or
to_thread. - Parallelism need: Many concurrent sockets -> asyncio; few blocking calls -> small thread pool.
- CPU cores: Python CPU loops ->
ProcessPoolExecutor; NumPy/C extensions may release GIL in threads. - Isolation: Crash-prone native code -> processes; shared in-memory cache -> threads/async with care.
Model Decision Table
| Profile | First choice | Fallback |
|---|---|---|
| Many HTTP/DB waits, async libs | asyncio | ThreadPool + blocking libs |
| Blocking SDK, moderate concurrency | ThreadPoolExecutor | asyncio + to_thread |
| CPU pure Python | ProcessPoolExecutor | C extension / vectorize |
| Mixed API + CPU stage | asyncio + process pool | Thread pool + process pool split |
| Simple script, <5 tasks | Sequential | threads if I/O slow |
Step-by-Step Workflow
Step 1: Is the task I/O-bound?
If yes and async-capable libraries exist, prefer asyncio for hundreds+ concurrent waits. If libraries are blocking, use ThreadPoolExecutor sized to connection limits.
Step 2: Is the task CPU-bound in Python bytecode?
If yes on default GIL build, use multiprocessing or ProcessPoolExecutor. Do not expect thread speedup for tight loops.
Step 3: Is workload mixed?
Split pipeline: async fetch -> asyncio.to_thread or run_in_executor(ProcessPool) for transform. Keep CPU work off the event loop thread.
Step 4: What are operability constraints?
Windows spawn requires picklable workers; daemon threads complicate graceful shutdown; process memory = baseline × workers. Factor into choice.
Step 5: Did you measure after implementing?
Compare p50/p99 latency and CPU under realistic load. Wrong model shows as event-loop blockage, GIL contention, or process spawn thrash.
FAQs
Default for FastAPI services?
asyncio - framework is async-native; offload blocking to thread pool sparingly.
When is sequential fine?
Low QPS admin scripts, cron with single connection, prototypes before scale proof.
Can I mix threads and asyncio?
Yes via executor - never call blocking I/O directly on loop thread.
What about gevent?
Legacy monkey-patch model - prefer asyncio for greenfield unless stuck on sync stack.
How many thread pool workers?
Connection pool size or concurrent blocking calls - not CPU count.
Process pool on small tasks?
Avoid - task must amortize spawn/pickle cost.
Does Polars/pandas change choice?
Heavy ops run in native code - often release GIL; still profile, may stay in-process threads.
Free-threaded Python impact?
May shift CPU-bound thread choice - re-run checklist after extension audit.
Redis/Celery workers?
Process-based workers are separate choice - often multiprocessing by design for isolation.
Document where?
ADR: workload, measured profile, chosen model, rejected alternatives, revisit trigger.
Related
- Concurrency Basics - terminology
- The GIL & Free-Threaded Python - CPU threads
- Asyncio Basics - async path
- Concurrency Best Practices - safety rules
Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.