tenacity
tenacity adds configurable retry logic to any callable - exponential backoff, jitter, stop conditions, and hooks - without hand-rolling while loops around flaky network or database calls.
Recipe
from tenacity import retry, stop_after_attempt, wait_exponential_jitter
@retry(stop=stop_after_attempt(3), wait=wait_exponential_jitter(initial=1, max=10))
def fetch_status() -> dict:
response = httpx.get("https://api.example.com/health", timeout=5.0)
response.raise_for_status()
return response.json()When to reach for this:
- Transient HTTP 502/503/timeout errors from upstream APIs
- Database deadlocks or connection blips
- Cloud API throttling (combined with rate-limit awareness)
- Worker tasks that should survive brief infrastructure hiccups
Working Example
from __future__ import annotations
import logging
import httpx
from tenacity import (
before_sleep_log,
retry,
retry_if_exception_type,
stop_after_attempt,
wait_exponential_jitter,
)
logger = logging.getLogger(__name__)
class TransientError(Exception):
pass
def _raise_for_transient(response: httpx.Response) -> None:
if response.status_code >= 500:
raise TransientError(f"server error {response.status_code}")
response.raise_for_status()
@retry(
retry=retry_if_exception_type((TransientError, httpx.TimeoutException)),
stop=stop_after_attempt(5),
wait=wait_exponential_jitter(initial=0.5, max=8),
before_sleep=before_sleep_log(logger, logging.WARNING),
reraise=True,
)
def charge_customer(customer_id: str, amount_cents: int) -> str:
with httpx.Client(timeout=10.0) as client:
response = client.post(
"/charges",
json={"customer_id": customer_id, "amount_cents": amount_cents},
headers={"Idempotency-Key": f"{customer_id}:{amount_cents}"},
)
_raise_for_transient(response)
return response.json()["id"]
if __name__ == "__main__":
logging.basicConfig(level=logging.INFO)
print(charge_customer("cus_1", 1999))What this demonstrates:
- Retries only
TransientErrorand timeouts - not 4xx client mistakes - Exponential backoff with jitter reduces thundering herds
before_sleep_logsurfaces retry attempts in logs- Idempotency header paired with retries for payment safety
reraise=Truepreserves the final exception after exhaustion
Deep Dive
How It Works
- Decorators -
@retrywraps sync functions; use@retryon async functions with async-compatible wait/stop (orAsyncRetryingcontext). - Stop policies - Combine
stop_after_attempt,stop_after_delay,stop_any. - Wait policies - Fixed, exponential, random, or custom callable.
- Retry predicates -
retry_if_exception,retry_if_result,retry_unless_exception_type.
Retry Decision Matrix
| Error Type | Retry? | Notes |
|---|---|---|
| HTTP 408/429/502/503/504 | Often | Respect Retry-After when present |
| HTTP 400/401/404 | No | Fix request or auth |
| DB deadlock | Yes | Short backoff, limited attempts |
| ValidationError | No | Data will not change on retry |
Python Notes
from tenacity import Retrying
# Imperative style inside a loop body
for attempt in Retrying(stop=stop_after_attempt(3)):
with attempt:
do_work()Gotchas
- Retrying non-idempotent POST - Double charges or duplicate records. Fix: idempotency keys or dedupe table.
- Infinite retry on bad config - Wrong URL retries forever with cron. Fix:
stop_after_attemptplus alert on final failure. - Catching bare
Exception- Masks programming bugs. Fix: narrowretry_if_exception_type. - No jitter - Synchronized clients retry together and DDoS the recovering service. Fix:
wait_exponential_jitter. - Retry inside retry - Nested decorators multiply attempts. Fix: one retry boundary per outbound call.
- Silent failures - Swallowing the last exception hides outages. Fix:
reraise=Trueand metrics on retry count.
Alternatives
| Alternative | Use When | Don't Use When |
|---|---|---|
Manual for loop | One-off scripts with 2 retries | Shared policies across services |
urllib3.util.retry | Only requests/urllib3 stack | httpx-native apps |
Celery autoretry_for | Task queue retries at worker level | In-process HTTP client calls |
| Circuit breaker (pybreaker) | Sustained upstream failure | Brief transient blips only |
FAQs
Does tenacity work with async functions?
Yes - use async retry helpers or AsyncRetrying depending on tenacity version; keep one event loop per client.
How do I retry on HTTP status without custom exceptions?
Raise a domain TransientError after inspecting response.status_code inside the wrapped function.
Can I notify Slack on final failure?
Use retry_error_callback or wrap the call and handle the re-raised exception in the caller.
How many attempts is enough?
Start with 3-5 for HTTP; tune from p99 latency and upstream SLO - log retry metrics to decide.
Should workers retry or the client?
Both layers are valid - client retries handle sub-second blips; Celery retries handle process restarts.
How do I test retry logic?
Mock the flaky dependency to fail twice then succeed; assert call count equals three.
Does tenacity respect Retry-After headers?
Not automatically - parse the header in before_sleep and return a custom wait if needed.
Can I disable retries in tests?
Pass a no-op retry policy via dependency injection or set an env flag that returns the bare function.
What about database session rollback?
Retry at the unit-of-work boundary after session.rollback() - do not retry mid-transaction.
Is tenacity thread-safe?
The decorator is re-entrant per call; share no mutable state inside retried functions without locks.
Related
- httpx & requests - HTTP clients to wrap
- Retries and Backoff - application-level patterns
- celery - distributed task retries
- Rate Limiting - when not to retry
Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.