httpx & requests

requests is the classic synchronous HTTP client; httpx is its modern successor with async support, HTTP/2, and stricter defaults. Both belong in every Python developer's toolkit for APIs, webhooks, and service-to-service calls.

Recipe

import httpx
 
with httpx.Client(timeout=httpx.Timeout(5.0, connect=2.0)) as client:
    response = client.get("https://httpbin.org/get", params={"q": "python"})
    response.raise_for_status()
    data = response.json()

When to reach for this:

• Calling REST/JSON APIs from scripts, workers, or FastAPI background tasks
• Need async HTTP in asyncio routes (httpx AsyncClient)
• Uploading files or streaming large responses
• Replacing ad-hoc urllib calls with readable, testable code

Working Example

from __future__ import annotations
 
import httpx
from pydantic import BaseModel
 
class User(BaseModel):
    id: int
    email: str
 
def fetch_user(user_id: int) -> User:
    timeout = httpx.Timeout(10.0, connect=3.0)
    with httpx.Client(base_url="https://api.example.com", timeout=timeout) as client:
        response = client.get(f"/users/{user_id}")
        response.raise_for_status()
        return User.model_validate(response.json())
 
async def fetch_user_async(user_id: int) -> User:
    timeout = httpx.Timeout(10.0, connect=3.0)
    async with httpx.AsyncClient(base_url="https://api.example.com", timeout=timeout) as client:
        response = await client.get(f"/users/{user_id}")
        response.raise_for_status()
        return User.model_validate(response.json())
 
if __name__ == "__main__":
    print(fetch_user(1))

What this demonstrates:

• base_url avoids repeating host prefixes
• Explicit Timeout splits connect vs read limits
• raise_for_status() turns 4xx/5xx into exceptions
• Pydantic validates JSON payloads at the boundary
• Same pattern works sync (Client) and async (AsyncClient)

Deep Dive

How It Works

• Connection pooling - Reusing a Client keeps TCP/TLS sessions warm; one-off httpx.get() creates a new pool each call.
• Timeouts - httpx.Timeout accepts connect, read, write, and pool limits; unset values fall back to the default (5s in httpx).
• Redirects - Followed by default; disable with follow_redirects=False for OAuth or signed URLs.
• Streaming - client.stream("GET", url) yields chunks without loading the full body into memory.

Client Lifecycle

Python Notes

# Map transport errors to your domain layer
try:
    response = client.post("/orders", json=payload)
    response.raise_for_status()
except httpx.TimeoutException as exc:
    raise ServiceUnavailable("upstream timeout") from exc
except httpx.HTTPStatusError as exc:
    if exc.response.status_code == 404:
        raise NotFound("order missing")
    raise

Gotchas

• No timeout set - Infinite hangs stall workers and obscure outages. Fix: always pass timeout=.
• Creating a new client per request - Destroys pooling and slows TLS handshakes. Fix: reuse Client/AsyncClient.
• Trusting JSON without validation - Schema drift causes KeyError deep in business logic. Fix: validate with Pydantic at the HTTP boundary.
• Blocking requests inside async routes - Blocks the event loop under load. Fix: use httpx.AsyncClient or asyncio.to_thread.
• Ignoring redirects on POST - Some APIs return 302 to a different host. Fix: log response.history when debugging auth flows.
• Hard-coded secrets in URLs - Query tokens end up in logs. Fix: use headers (Authorization) and redact in logging middleware.

cGJxcnRodnFyZi52YnxwdHZiNzQ0fDIwMjYwNw==

Alternatives

FAQs

with open("report.pdf", "rb") as f:
    client.post("/upload", files={"file": ("report.pdf", f, "application/pdf")})

Related

• tenacity - retries and backoff for flaky upstreams
• pydantic-settings - typed config for API base URLs
• FastAPI Basics - async routes that call external APIs
• beautifulsoup4 & lxml - parsing HTML fetched over HTTP

External Related Links

• Python Package Index (PyPI) - Official repository of Python packages.
• Python Standard Library Reference - Complete documentation for built-in modules and functions.
• Real Python - In-depth tutorials, courses, and practical guides.
• Fluent Python (2nd Edition, O'Reilly) - Same book on O'Reilly.
• Python Crash Course (3rd Edition) - Beginner-friendly projects (Eric Matthes).

Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.