Tasks & Gather
Tasks wrap coroutines for concurrent scheduling on the event loop. asyncio.gather awaits multiple awaitables together - the bread-and-butter of parallel I/O in asyncio.
Recipe
import asyncio
async def fetch(n: int) -> int:
await asyncio.sleep(0.1)
return n
async def main() -> None:
tasks = [asyncio.create_task(fetch(i)) for i in range(3)]
print(await asyncio.gather(*tasks))
asyncio.run(main())When to reach for this:
- Concurrent HTTP/DB calls with independent results
- Background tasks alongside request handling
- Fan-in before responding (all data required)
- Optional partial failure with
return_exceptions=True - Bridging callback APIs via
create_task
Working Example
import asyncio
async def download(url_id: int) -> dict:
await asyncio.sleep(0.05)
if url_id == 2:
raise ValueError("bad response")
return {"id": url_id, "bytes": 100}
async def all_or_nothing() -> None:
try:
await asyncio.gather(*(download(i) for i in range(4)))
except ValueError as exc:
print("failed", exc)
async def collect_errors() -> None:
results = await asyncio.gather(
*(download(i) for i in range(4)),
return_exceptions=True,
)
for r in results:
print(type(r).__name__, r)
asyncio.run(all_or_nothing())
asyncio.run(collect_errors())What this demonstrates:
- Default
gatherfails fast on first exception return_exceptions=Trueyields exception objects in result list- Tasks start running as soon as
create_taskschedules them - Order of gather results matches submission order, not completion order
Deep Dive
Task vs Coroutine
| Coroutine | Task | |
|---|---|---|
| Scheduling | Await to run | Scheduled immediately |
| Cancellation | N/A | task.cancel() |
| Inspection | Limited | .done(), .exception() |
gather vs TaskGroup
gather- convenient fan-in, legacy-friendlyTaskGroup(3.11+) - structured concurrency, auto-cancel siblings on failure
Gotchas
- create_task without reference - task garbage-collected mid-run in edge cases. Fix: keep reference until done.
- gather without return_exceptions in partial success APIs - one failure loses all results. Fix:
return_exceptions=Trueor TaskGroup. - Blocking code inside tasks - stalls all tasks. Fix:
to_thread. - Unbounded task creation - million tasks exhaust memory. Fix: semaphores, worker pools, batches.
- Fire-and-forget without error handler -
task.exception()never retrieved logs warning. Fix: callback orasyncio.create_taskwrapper logging failures.
Alternatives
| Alternative | Use When | Don't Use When |
|---|---|---|
| TaskGroup | New code, strict parent/child | Need gather's return_exceptions ergonomics |
| asyncio.as_completed | Process results as ready | Need full list at end |
| Semaphore + gather | Cap concurrency | Tiny fan-out |
FAQs
create_task vs ensure_future?
create_task for coroutines on running loop; ensure_future broader (legacy interop).
Does gather run in parallel?
Concurrent on one thread - interleaved awaits, not OS parallel threads.
How to limit concurrency?
asyncio.Semaphore(n) around work inside each task.
Cancel all on timeout?
async with asyncio.timeout(5): wrapping gather or TaskGroup.
gather vs wait?
wait returns done/pending sets; gather returns values/exceptions list.
Can I gather mixed types?
Yes - awaitables include tasks, coroutines, futures.
Task naming?
create_task(coro, name="fetch-user") helps debug logs (3.8+).
Exception in fire-and-forget?
Add task.add_done_callback logging task.exception().
gather preserves order?
Yes - result list order matches input awaitables order.
Nested gather?
Works but flatten when possible; semaphore at outer level for clarity.
Related
- Structured Concurrency - TaskGroup
- Asyncio Basics - coroutine primer
- Async HTTP with httpx/aiohttp - concurrent requests
- Debugging Async Code - task leaks
Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.