Working with APIs & Webhooks
Automation scripts integrate with SaaS and internal APIs via HTTP clients. Use httpx or requests with timeouts, retries, pagination loops, and optional webhook receivers for event-driven triggers.
Search across all documentation pages
Automation scripts integrate with SaaS and internal APIs via HTTP clients. Use httpx or requests with timeouts, retries, pagination loops, and optional webhook receivers for event-driven triggers.
import httpx
with httpx.Client(timeout=30.0) as client:
resp = client.get(
"https://api.example.com/v1/items",
headers={"Authorization": "Bearer TOKEN"},
params={"page": 1, "limit": 100},
)
resp.raise_for_status()
data = resp.json()When to reach for this:
Paginated GET client with retry, plus minimal webhook signature verification stub.
import hashlib
import hmac
import time
from typing import Any, Iterator
import httpx
class ApiClient:
def __init__(self, base_url: str, token: str) -> None:
self._client = httpx.Client(
base_url=base_url,
headers={"Authorization": f"Bearer {token}"},
timeout=httpx.Timeout(30.0),
)
def list_items(self, page_size: int = 100) -> Iterator[dict[str, Any]]:
cursor: str | None = None
while True:
params = {"limit": page_size}
if cursor:
params["cursor"] = cursor
resp = self._client.get("/v1/items", params=params)
resp.raise_for_status()
payload = resp.json()
for item in payload.get("items", []):
yield item
cursor = payload.get("next_cursor")
if not cursor:
break
def close(self) -> None:
self._client.close()
def verify_webhook(body: bytes, signature: str, secret: str) -> bool:
expected = hmac.new(secret.encode(), body, hashlib.sha256).hexdigest()
return hmac.compare_digest(expected, signature)
if __name__ == "__main__":
client = ApiClient("https://api.example.com", "dev-token")
for i, item in enumerate(client.list_items()):
if i >= 3:
break
print(item.get("id"))
client.close()What this demonstrates:
next_cursor absenthttpx.Client reuses connections across pageshmac.compare_digest prevents timing leaks on webhook verification| Mode | Pros | Cons |
|---|---|---|
| Polling | Simple, no inbound port | Delay, rate limits |
| Webhooks | Real-time | Signature verify, retries, idempotency |
Retry-After header on 429import httpx
from tenacity import retry, stop_after_attempt, wait_exponential
@retry(stop=stop_after_attempt(5), wait=wait_exponential(multiplier=0.5))
def fetch(client: httpx.Client, url: str) -> httpx.Response:
resp = client.get(url)
resp.raise_for_status()
return resphttpx.Timeout always set.| Alternative | Use When | Don't Use When |
|---|---|---|
| Official SDK | Vendor provides maintained Python SDK | Thin REST surface |
| SQS/Lambda webhook | Scale inbound events | Single cron poller enough |
| GraphQL client | API is GraphQL-only | Simple REST pagination |
httpx supports HTTP/2 and async; requests is sync-only but ubiquitous - both fine for scripts with timeouts.
Read X-RateLimit-* headers, sleep or backoff on 429, reduce concurrency.
Prefer cursor for large datasets - offset pagination breaks when data mutates during walk.
FastAPI/Flask endpoint or Lambda Function URL - validate signature before work.
httpx.MockTransport or pytest-httpx fixtures - no network in unit tests.
Sync is simpler for cron; async when orchestrating many parallel downloads in one event loop.
Log method, URL path, status, duration - not Authorization header or full PII bodies.
Some APIs use RFC 5988 Link header - parse rel="next" instead of JSON cursor field.
client.post("/hooks", json={"ok": True}) sets content-type automatically.
Poll-based scripts are often cron-fired; webhooks reduce schedule frequency.
Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.
Reviewed by Chris St. John·Last updated Jul 19, 2026