google-cloud & azure-sdk
Not every workload runs on AWS. Google Cloud (google-cloud-*) and Azure (azure-*) Python SDKs follow similar client patterns to boto3: credentials from the environment, regional endpoints, and paginated list operations.
Recipe
from google.cloud import storage
client = storage.Client()
bucket = client.bucket("my-gcs-bucket")
blob = bucket.blob("data/report.csv")
blob.upload_from_filename("report.csv")from azure.storage.blob import BlobServiceClient
service = BlobServiceClient.from_connection_string(conn_str)
container = service.get_container_client("uploads")
container.upload_blob(name="report.csv", data=b"hello", overwrite=True)When to reach for this:
- Multi-cloud strategy or acquisition integrations
- GCP BigQuery / GCS or Azure Blob / Service Bus as primary data plane
- Portable patterns abstracting storage behind an interface
- Hybrid workloads with per-service best-fit cloud
Working Example
Thin abstraction over GCS and Azure Blob upload with shared interface.
from __future__ import annotations
from dataclasses import dataclass
from typing import Protocol
@dataclass
class ObjectRef:
bucket: str
key: str
class ObjectStore(Protocol):
def upload_bytes(self, ref: ObjectRef, data: bytes, content_type: str) -> None: ...
class GCSStore:
def __init__(self, project: str | None = None) -> None:
from google.cloud import storage
self._client = storage.Client(project=project)
def upload_bytes(self, ref: ObjectRef, data: bytes, content_type: str) -> None:
bucket = self._client.bucket(ref.bucket)
blob = bucket.blob(ref.key)
blob.upload_from_string(data, content_type=content_type)
class AzureBlobStore:
def __init__(self, connection_string: str) -> None:
from azure.storage.blob import BlobServiceClient
self._service = BlobServiceClient.from_connection_string(connection_string)
def upload_bytes(self, ref: ObjectRef, data: bytes, content_type: str) -> None:
container = self._service.get_container_client(ref.bucket)
container.upload_blob(name=ref.key, data=data, overwrite=True, content_type=content_type)
def persist_report(store: ObjectStore, ref: ObjectRef, payload: bytes) -> None:
store.upload_bytes(ref, payload, "text/csv")
if __name__ == "__main__":
ref = ObjectRef(bucket="reports", key="daily/summary.csv")
# persist_report(GCSStore(), ref, b"date,total\n2026-07-09,42\n")
print(ref)What this demonstrates:
- Protocol interface keeps app code cloud-agnostic
- GCS uses
google.cloud.storage; Azure usesazure-storage-blob - Credentials come from ADC (
GOOGLE_APPLICATION_CREDENTIALS) or Azure connection string /DefaultAzureCredential
Deep Dive
Credential Models
| Cloud | Python credential source |
|---|---|
| GCP | Application Default Credentials, service account JSON |
| Azure | DefaultAzureCredential, connection strings, managed identity |
| AWS | boto3 chain (compare Credential & Session Patterns) |
SDK Layout
- GCP: one package per product (
google-cloud-storage,google-cloud-pubsub) - Azure: product packages (
azure-storage-blob,azure-identity,azure-servicebus) - Install only packages you need - keep Lambda layers slim
Python Notes
from azure.identity import DefaultAzureCredential
from azure.storage.blob import BlobServiceClient
credential = DefaultAzureCredential()
service = BlobServiceClient(account_url="https://acct.blob.core.windows.net", credential=credential)Gotchas
- Connection strings in git - full storage access if leaked. Fix: managed identity in Azure; workload identity in GCP/GKE.
- Mixing sync SDK in async FastAPI - blocks event loop. Fix: threadpool or async-compatible clients where available.
- Different pagination APIs - GCP iterators vs Azure continuations. Fix: wrap in shared helper per provider.
- Assuming boto3 IAM patterns - GCP/Azure RBAC roles differ. Fix: least privilege per service account / app registration.
- Installing
google-cloudmeta - pulls enormous deps. Fix: pin specificgoogle-cloud-storageetc.
Alternatives
| Alternative | Use When | Don't Use When |
|---|---|---|
| boto3 only | AWS-exclusive shop | Contract requires GCP/Azure |
| REST + httpx | Tiny integration surface | Full SDK feature set needed |
| Terraform providers | Infra provisioning | App-runtime object uploads |
FAQs
How do GCP credentials work locally?
gcloud auth application-default login sets ADC; CI uses service account JSON or workload identity federation.
What replaces AWS S3 on Azure?
Azure Blob Storage via azure-storage-blob - containers map to buckets, blobs to keys.
Can I share code between clouds?
Yes - interface + provider implementations; avoid leaking SDK types above the adapter layer.
How do I handle retries?
SDKs include retry policies; tune similarly to botocore Config for timeouts and max attempts.
BigQuery from Python?
google-cloud-bigquery client runs queries and loads from GCS - common GCP data pattern.
Azure Service Bus vs SNS/SQS?
Service Bus queues/topics for Azure-native messaging - similar decoupling patterns as AWS messaging docs.
How do I test?
Use emulators (Azurite, fake-gcs-server) or mock adapters in pytest - avoid hitting prod storage in unit tests.
Package management with uv?
uv add google-cloud-storage azure-storage-blob azure-identity pins versions in pyproject.toml.
Multi-cloud logging?
Emit cloud-agnostic structured logs; include cloud.provider field for routing dashboards.
When not to abstract?
When single-cloud for years - thin wrappers add indirection without portability benefit.
Related
- boto3 Basics - AWS SDK parallels
- S3 - AWS object storage patterns
- Credential & Session Patterns - identity patterns
- Cloud SDK Best Practices - multi-cloud hygiene
- Provisioning Cloud Resources - account bootstrap
Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.