Cloud SDK Best Practices

Safe, efficient, well-scoped AWS SDK usage keeps Python services fast under load and limits blast radius when credentials leak or scripts mis-target an account.

How to Use This List

• Apply on every new boto3 integration before production traffic.
• Enforce IAM and retry settings via shared internal library, not copy-paste per service.
• Review after incidents involving throttling, wrong-account access, or credential exposure.
• Cross-check with Credential & Session Patterns for identity setup.

A - Credentials and Identity

• No long-lived access keys in code or images. Use IAM roles, SSO profiles, or CI OIDC federation.
• Call sts.get_caller_identity() in destructive CLIs. Confirm account before delete operations.
• Scope IAM policies to resource ARNs and actions required. Avoid * actions on * resources.
• Use separate AWS profiles per account. AWS_PROFILE documented in runbooks and CI jobs.
• Rotate and audit access keys if legacy users remain. Prefer migration plan to SSO roles.

B - Clients, Sessions, and Performance

• Create boto3.Session once per process context. Reuse clients across requests in workers and Lambda handlers.
• Set explicit region_name on sessions. Avoid ambiguous cross-region redirects and signing bugs.
• Tune botocore.config.Config retries and timeouts. Match service p99 latency and throttle behavior.
• Use paginators for all list operations. Never assume a single page is complete.
• Raise max_pool_connections for parallel uploads. Prevent connection pool starvation.

C - Data and Security

• Fetch secrets from Secrets Manager/SSM with TTL cache. Not on every HTTP request without bounds.
• Never log SecretString, presigned URLs, or auth headers. Log ARNs and request IDs only.
• Use SSE and block public access for S3 buckets. Presigned URLs with short expiry for external access.
• Validate error codes, not message strings. Branch on ClientError.response["Error"]["Code"].
• Keep payloads under service limits. Store large blobs in S3; pass references in SQS messages.

D - Reliability

• Handle throttling with backoff and jitter. Combine botocore retries with app-level caps.
• Make consumers idempotent. SQS and Lambda may deliver duplicates - use dedupe keys.
• Use DLQs and alarms on queue depth. Poison messages must not spin forever.
• Delete SQS messages only after successful processing. Tune visibility timeout to handler p99.
• Test with moto/LocalStack in unit tests. Sandbox account smoke tests for integration paths.

E - Operations

• Pin boto3/botocore versions in lockfiles. Reproducible behavior across laptops and CI.
• Wrap cloud calls with structured logs. Include service, operation, latency, and correlation ID.
• Expose CloudWatch metrics for SDK error rates. Alert on AccessDenied spikes after IAM changes.
• Document which roles each service uses. On-call should not guess execution role at 3am.
• Prefer IaC for resource creation. boto3 in apps for runtime operations, not one-off admin scripts without review.

FAQs

Related

• boto3 Basics - sessions, clients, resources
• Credential & Session Patterns - roles and profiles
• Retry, Pagination & Throttling - resilient calls
• Secrets Manager & SSM - secret retrieval
• Infrastructure Automation Best Practices - IaC complement

Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.