Essential Libraries Best Practices

A condensed summary of the 25 most important practices for choosing and using Python essential libraries - drawn from every page in this section.

Prefer httpx for new HTTP clients: Reuse Client/AsyncClient with explicit Timeout and raise_for_status() (httpx & requests).
Never block asyncio with sync HTTP: Use httpx.AsyncClient or asyncio.to_thread - not requests inside async FastAPI routes.
Validate JSON at the HTTP boundary: Parse responses with Pydantic models before business logic - schema drift causes deep KeyErrors.
Load settings with pydantic-settings: Fail fast on missing secrets at startup; use extra="ignore" to catch typos (pydantic-settings).
Cache settings factories: @lru_cache on get_settings() avoids re-parsing env per request without import-time globals.
Retry transient failures only: Narrow retry_if_exception_type in tenacity - do not retry 4xx client errors (tenacity).
Add jitter to backoff: wait_exponential_jitter prevents synchronized retry storms against recovering upstreams.
Pair retries with idempotency keys: Especially for payments and writes - retries without dedupe double-apply side effects.
Log before_sleep on retries: before_sleep_log makes retry loops visible in observability tooling.
Store UTC in databases: Convert with zoneinfo or pendulum at display boundaries - not naive local datetimes (pendulum / arrow).
Apply EXIF transpose before thumbnails: Phone photos appear sideways without ImageOps.exif_transpose (Pillow).
Cap bytes before image decode: Reject uploads over a size limit to mitigate decompression bombs.
Whitelist image formats after open: Trust img.format from Pillow, not client Content-Type alone.
Use in-memory BytesIO for API exports: openpyxl and reportlab pipelines should not require temp files on disk (openpyxl / python-docx / reportlab).
Keep report templates in version control: Word and PDF layouts maintained as templates beat hard-coded coordinates.
Separate fetch from parse in scrapers: Test BeautifulSoup parsers against HTML fixtures without network (beautifulsoup4 & lxml).
Prefer APIs over scraping: Respect robots.txt, rate limits, and terms of service when HTML is the only source.
Use lxml parser backend for large HTML: Faster and lower memory than pure-Python parsers at scale.
Celery tasks pass IDs, not ORM objects: JSON-serialize primitive payloads only (celery).
Configure JSON serializers in Celery: Avoid pickle - arbitrary deserialization is a security risk.
Set acks_late and low prefetch: Fair recovery when workers die mid-task without losing broker messages.
Use Alembic for schema changes: create_all is for dev only - production needs migration history (SQLModel / SQLAlchemy).
SQLAlchemy 2.0 select() style: session.exec(select(Model)) - not legacy session.query.
Rich human output on stderr: Keep stdout machine-readable when CLIs pipe data (Rich & Textual).
Pin and audit dependencies: Lock versions in pyproject.toml/uv.lock; review transitive licenses and CVEs quarterly.

FAQs

How many HTTP libraries should one service use?

Standardize on httpx per service - multiple client stacks multiply timeout and retry policies.

When is stdlib enough without pendulum?

When APIs enforce ISO 8601 UTC strings - use datetime + zoneinfo and skip extra dependencies.

Celery for every background job?

No - FastAPI BackgroundTasks and in-process queues handle sub-second work; Celery when you need horizontal workers.

How do I choose between SQLModel and Django ORM?

Match the web framework boundary - SQLModel with FastAPI; Django ORM inside Django apps.

Environments & Packaging Basics - dependency management
Dependency and Supply Chain Security - auditing deps
FastAPI Basics - framework integration
Databases Basics - data layer depth

Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.

Essential Libraries Best Practices

A condensed summary of the 25 most important practices for choosing and using Python essential libraries - drawn from every page in this section.

Prefer httpx for new HTTP clients: Reuse Client/AsyncClient with explicit Timeout and raise_for_status() (httpx & requests).

Never block asyncio with sync HTTP: Use httpx.AsyncClient or asyncio.to_thread - not requests inside async FastAPI routes.

Validate JSON at the HTTP boundary: Parse responses with Pydantic models before business logic - schema drift causes deep KeyErrors.

Load settings with pydantic-settings: Fail fast on missing secrets at startup; use extra="ignore" to catch typos (pydantic-settings).

Cache settings factories: @lru_cache on get_settings() avoids re-parsing env per request without import-time globals.

Retry transient failures only: Narrow retry_if_exception_type in tenacity - do not retry 4xx client errors (tenacity).

Add jitter to backoff: wait_exponential_jitter prevents synchronized retry storms against recovering upstreams.

Pair retries with idempotency keys: Especially for payments and writes - retries without dedupe double-apply side effects.

Log before_sleep on retries: before_sleep_log makes retry loops visible in observability tooling.

Store UTC in databases: Convert with zoneinfo or pendulum at display boundaries - not naive local datetimes (pendulum / arrow).

Apply EXIF transpose before thumbnails: Phone photos appear sideways without ImageOps.exif_transpose (Pillow).

Cap bytes before image decode: Reject uploads over a size limit to mitigate decompression bombs.

Whitelist image formats after open: Trust img.format from Pillow, not client Content-Type alone.

Use in-memory BytesIO for API exports: openpyxl and reportlab pipelines should not require temp files on disk (openpyxl / python-docx / reportlab).

Keep report templates in version control: Word and PDF layouts maintained as templates beat hard-coded coordinates.

Separate fetch from parse in scrapers: Test BeautifulSoup parsers against HTML fixtures without network (beautifulsoup4 & lxml).

Prefer APIs over scraping: Respect robots.txt, rate limits, and terms of service when HTML is the only source.

Use lxml parser backend for large HTML: Faster and lower memory than pure-Python parsers at scale.

Celery tasks pass IDs, not ORM objects: JSON-serialize primitive payloads only (celery).

Configure JSON serializers in Celery: Avoid pickle - arbitrary deserialization is a security risk.

Set acks_late and low prefetch: Fair recovery when workers die mid-task without losing broker messages.

Use Alembic for schema changes: create_all is for dev only - production needs migration history (SQLModel / SQLAlchemy).

SQLAlchemy 2.0 select() style: session.exec(select(Model)) - not legacy session.query.

Rich human output on stderr: Keep stdout machine-readable when CLIs pipe data (Rich & Textual).

Pin and audit dependencies: Lock versions in pyproject.toml/uv.lock; review transitive licenses and CVEs quarterly.

FAQs

How many HTTP libraries should one service use?

Standardize on httpx per service - multiple client stacks multiply timeout and retry policies.

When is stdlib enough without pendulum?

When APIs enforce ISO 8601 UTC strings - use datetime + zoneinfo and skip extra dependencies.

Celery for every background job?

No - FastAPI BackgroundTasks and in-process queues handle sub-second work; Celery when you need horizontal workers.

How do I choose between SQLModel and Django ORM?

Match the web framework boundary - SQLModel with FastAPI; Django ORM inside Django apps.

Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.

Essential Libraries Best Practices

FAQs

Related

Search Documentation

Essential Libraries Best Practices

FAQs

Related