Lambda
AWS Lambda runs Python handlers without managing servers. boto3 lets you invoke functions, publish versions, and wire automation around deployments while keeping function code in your repo.
Recipe
import json
import boto3
client = boto3.client("lambda")
resp = client.invoke(
FunctionName="process-order",
InvocationType="RequestResponse",
Payload=json.dumps({"order_id": "ord_123"}),
)
print(resp["StatusCode"], resp["Payload"].read())When to reach for this:
- Event-driven tasks (S3, SQS, EventBridge)
- Infrequent workloads where always-on servers waste money
- Async fan-out with
InvocationType="Event" - Glue code between managed AWS services
Working Example
Deploy artifact metadata update, synchronous invoke, and async invoke with error handling.
import json
import logging
import boto3
from botocore.exceptions import ClientError
log = logging.getLogger(__name__)
lambda_client = boto3.client("lambda")
def invoke_sync(function_name: str, payload: dict) -> dict:
try:
resp = lambda_client.invoke(
FunctionName=function_name,
InvocationType="RequestResponse",
Payload=json.dumps(payload).encode(),
)
except ClientError as exc:
log.error("invoke failed: %s", exc.response["Error"]["Code"])
raise
raw = resp["Payload"].read()
return json.loads(raw or b"{}")
def invoke_async(function_name: str, payload: dict) -> int:
resp = lambda_client.invoke(
FunctionName=function_name,
InvocationType="Event",
Payload=json.dumps(payload).encode(),
)
return resp["StatusCode"]
if __name__ == "__main__":
result = invoke_sync("demo-python-fn", {"action": "ping"})
print(result)
status = invoke_async("demo-python-fn", {"action": "enqueue"})
print("async status", status)What this demonstrates:
RequestResponsewaits for handler return;Eventis fire-and-forget- Payload must be JSON bytes for structured events
ClientErrorcodes includeResourceNotFoundException,TooManyRequestsException
Deep Dive
How It Works
- Upload zip/container image to Lambda; handler string points to
module.function - Cold start loads Python runtime and imports; reuse clients outside handler
- Concurrency limits and reserved concurrency protect downstream systems
- Versions and aliases (
prod) enable safe rollouts
Packaging Notes
| Approach | Pros |
|---|---|
| Zip + layers | Simple deps separation |
| Container image | Heavy deps (ML libs) |
| SAR / IaC | Repeatable deploys |
Python Notes
# handler.py - create boto3 clients outside handler
import boto3
s3 = boto3.client("s3")
def handler(event, context):
return {"ok": True, "remaining_ms": context.get_remaining_time_in_millis()}Gotchas
- Creating boto3 client per invocation - slows cold/warm starts. Fix: module-level client reuse.
- Huge deployment packages - slow cold starts. Fix: slim deps, layers, or container with prebuilt venv.
- Blocking the event loop in async handlers - timeouts. Fix: offload CPU work or use sync handler with appropriate timeout/memory.
- No idempotency on async retries - duplicate side effects. Fix: idempotent handlers with dedupe keys (SQS FIFO or DynamoDB conditional writes).
- Logging to stdout only without structure - hard to trace. Fix: JSON logs with
request_idfromcontext.aws_request_id.
Alternatives
| Alternative | Use When | Don't Use When |
|---|---|---|
| ECS/Fargate | Long-running HTTP, websockets | Short sporadic jobs |
| Step Functions | Multi-step orchestration | Single-function transforms |
| Celery workers | Existing Redis/Rabbit ops culture | Want zero server management |
FAQs
How do I package Python 3.14 for Lambda?
Match Lambda runtime version in AWS; build deps in Amazon Linux container or CI image matching runtime ABI.
What timeout should I set?
Just above p99 runtime with headroom; pair with SQS visibility timeout > Lambda timeout for queue triggers.
How do I invoke from another account?
Resource-based policy on function allowing caller account principal.
RequestResponse vs Event?
RequestResponse for RPC-style; Event for decoupled processing where caller does not need result inline.
How do versions and aliases work?
Publish immutable version from $LATEST, point prod alias to version for rollback by retargeting alias.
Can I use pydantic in handlers?
Yes - validate event dict early; keep models small to reduce import time.
How do I limit concurrency?
Reserved concurrency on function or account limits; SQS max concurrency for event source mapping.
What about secrets?
Fetch from Secrets Manager at init if needed; prefer env vars injected from SSM/Secrets at deploy.
How do I test locally?
Invoke handler function directly in pytest with sample events; use SAM or LocalStack for integration.
How does boto3 deploy code?
update_function_code uploads zip from CI; prefer IaC (SAM, Pulumi, Terraform) wrapping that API.
Related
- AWS Lambda & Serverless - packaging and cold starts
- SQS & SNS - event sources
- S3 - object triggers
- boto3 Basics - client patterns
- Secrets Manager & SSM - env secrets
Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.