Structured Logging

Structured logging attaches key-value context to every log line - typically JSON - so operators can filter by user_id, order_id, or trace_id without regex archaeology.

Recipe

import structlog
 
structlog.configure(
    processors=[
        structlog.processors.add_log_level,
        structlog.processors.TimeStamper(fmt="iso"),
        structlog.processors.JSONRenderer(),
    ],
)
log = structlog.get_logger()
log.info("order_created", order_id="ord_1", amount_cents=4200)

When to reach for this:

Every production Python service behind containers or systemd
Centralized log platforms (Loki, Datadog, CloudWatch Logs Insights)
Correlation with traces and metrics fields
Security audits needing searchable actor/action fields

Working Example

structlog with contextvars for request_id and FastAPI middleware binding.

import uuid
from contextvars import ContextVar
 
import structlog
from fastapi import FastAPI, Request
 
request_id_var: ContextVar[str] = ContextVar("request_id", default="-")
 
def configure_logging() -> None:
    structlog.configure(
        processors=[
            structlog.contextvars.merge_contextvars,
            structlog.processors.add_log_level,
            structlog.processors.TimeStamper(fmt="iso"),
            structlog.processors.JSONRenderer(),
        ],
    )
 
configure_logging()
log = structlog.get_logger()
app = FastAPI()
 
@app.middleware("http")
async def bind_request_context(request: Request, call_next):
    rid = request.headers.get("x-request-id", str(uuid.uuid4()))
    structlog.contextvars.clear_contextvars()
    structlog.contextvars.bind_contextvars(request_id=rid, path=request.url.path)
    response = await call_next(request)
    response.headers["X-Request-ID"] = rid
    return response
 
@app.get("/orders/{order_id}")
async def get_order(order_id: str):
    log.info("fetch_order", order_id=order_id)
    return {"order_id": order_id}

What this demonstrates:

bind_contextvars adds fields to all logs in request scope
Middleware propagates or generates X-Request-ID
JSON renderer outputs one parseable object per line

Deep Dive

Processors Pipeline

Processor	Role
merge_contextvars	Inject request context
add_log_level	`level` field
TimeStamper	ISO timestamp
JSONRenderer	Serialize

stdlib Integration

import logging
structlog.stdlib.recreate_defaults()  # bridge to logging config

Python Notes

# Never log secrets
log.info("token_refreshed", user_id=u.id)  # not: token=token

Gotchas

f-strings in log messages - fields not indexed. Fix: keyword args log.info("event", user_id=id).
Logging PII without policy - GDPR violations. Fix: allowlist fields, hash user ids in non-prod.
Huge payload dumps - cost and noise. Fix: log size/hash, not full body.
Mixed plain text and JSON - parser breaks. Fix: JSON only in prod; pretty console in dev.
Missing trace_id - cannot jump to APM. Fix: bind OpenTelemetry trace context in processor.

Alternatives

Alternative	Use When	Don't Use When
stdlib logging Formatter JSON	Minimal deps	Want processor pipeline ergonomics
loguru	Scripts and prototypes	Team standardized on structlog
print(JSON)	Quick debug	Production services

FAQs

structlog vs logging?

structlog builds structured events; integrates with stdlib logging handlers for transport.

How do I pretty-print locally?

Use ConsoleRenderer() in dev processor chain; swap to JSONRenderer in prod via env.

Django integration?

Middleware binds contextvars; configure structlog in LOGGING dict replacing formatters.

How do I log exceptions?

log.exception("payment_failed", order_id=oid) or exc_info=True on stdlib bridge.

Log sampling?

Sample debug/trace volume at processor level; never sample ERROR without explicit policy.

How do workers get context?

Bind job_id at task start; propagate trace headers from enqueue message attributes.

CloudWatch Logs Insights?

JSON fields become queryable - fields @timestamp, order_id | filter level='error'.

Performance cost?

JSON serialization cheap vs I/O; avoid synchronous network appenders blocking request path.

How does this relate to parsing logs?

Apps should emit JSON upstream; parsing text logs is fallback for legacy systems.

Required fields?

timestamp, level, event/message, service name, environment, request_id/trace_id when available.

Observability Basics - three pillars
Distributed Tracing - trace_id in logs
Error Tracking - Sentry complements logs
Health & Readiness - minimal access logs
Observability Best Practices - logging policy

Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.

Structured Logging

Structured logging attaches key-value context to every log line - typically JSON - so operators can filter by user_id, order_id, or trace_id without regex archaeology.

Recipe

import structlog
 
structlog.configure(
    processors=[
        structlog.processors.add_log_level,
        structlog.processors.TimeStamper(fmt="iso"),
        structlog.processors.JSONRenderer(),
    ],
)
log = structlog.get_logger()
log.info("order_created", order_id="ord_1", amount_cents=4200)

When to reach for this:

Every production Python service behind containers or systemd
Centralized log platforms (Loki, Datadog, CloudWatch Logs Insights)
Correlation with traces and metrics fields
Security audits needing searchable actor/action fields

Working Example

structlog with contextvars for request_id and FastAPI middleware binding.

import uuid
from contextvars import ContextVar
 
import structlog
from fastapi import FastAPI, Request
 
request_id_var: ContextVar[str] = ContextVar("request_id", default="-")
 
def configure_logging() -> None:
    structlog.configure(
        processors=[
            structlog.contextvars.merge_contextvars,
            structlog.processors.add_log_level,
            structlog.processors.TimeStamper(fmt="iso"),
            structlog.processors.JSONRenderer(),
        ],
    )
 
configure_logging()
log = structlog.get_logger()
app = FastAPI()
 
@app.middleware("http")
async def bind_request_context(request: Request, call_next):
    rid = request.headers.get("x-request-id", str(uuid.uuid4()))
    structlog.contextvars.clear_contextvars()
    structlog.contextvars.bind_contextvars(request_id=rid, path=request.url.path)
    response = await call_next(request)
    response.headers["X-Request-ID"] = rid
    return response
 
@app.get("/orders/{order_id}")
async def get_order(order_id: str):
    log.info("fetch_order", order_id=order_id)
    return {"order_id": order_id}

What this demonstrates:

bind_contextvars adds fields to all logs in request scope
Middleware propagates or generates X-Request-ID
JSON renderer outputs one parseable object per line

Deep Dive

Processors Pipeline

Processor	Role
merge_contextvars	Inject request context
add_log_level	`level` field
TimeStamper	ISO timestamp
JSONRenderer	Serialize

stdlib Integration

import logging
structlog.stdlib.recreate_defaults()  # bridge to logging config

Python Notes

# Never log secrets
log.info("token_refreshed", user_id=u.id)  # not: token=token

Gotchas

f-strings in log messages - fields not indexed. Fix: keyword args log.info("event", user_id=id).
Logging PII without policy - GDPR violations. Fix: allowlist fields, hash user ids in non-prod.
Huge payload dumps - cost and noise. Fix: log size/hash, not full body.
Mixed plain text and JSON - parser breaks. Fix: JSON only in prod; pretty console in dev.
Missing trace_id - cannot jump to APM. Fix: bind OpenTelemetry trace context in processor.

Alternatives

Alternative	Use When	Don't Use When
stdlib logging Formatter JSON	Minimal deps	Want processor pipeline ergonomics
loguru	Scripts and prototypes	Team standardized on structlog
print(JSON)	Quick debug	Production services

FAQs

structlog vs logging?

structlog builds structured events; integrates with stdlib logging handlers for transport.

How do I pretty-print locally?

Use ConsoleRenderer() in dev processor chain; swap to JSONRenderer in prod via env.

Django integration?

Middleware binds contextvars; configure structlog in LOGGING dict replacing formatters.

How do I log exceptions?

log.exception("payment_failed", order_id=oid) or exc_info=True on stdlib bridge.

Log sampling?

Sample debug/trace volume at processor level; never sample ERROR without explicit policy.

How do workers get context?

Bind job_id at task start; propagate trace headers from enqueue message attributes.

CloudWatch Logs Insights?

JSON fields become queryable - fields @timestamp, order_id | filter level='error'.

Performance cost?

JSON serialization cheap vs I/O; avoid synchronous network appenders blocking request path.

How does this relate to parsing logs?

Apps should emit JSON upstream; parsing text logs is fallback for legacy systems.

Required fields?

timestamp, level, event/message, service name, environment, request_id/trace_id when available.

Observability Basics - three pillars
Distributed Tracing - trace_id in logs
Error Tracking - Sentry complements logs
Health & Readiness - minimal access logs
Observability Best Practices - logging policy

Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.

Structured Logging

Search Documentation

Structured Logging