Caching & Memoization
Caching stores computed results for reuse. functools.lru_cache handles in-process memoization; Redis/Memcached handle shared caches across processes.
Recipe
from functools import lru_cache
@lru_cache(maxsize=128)
def expensive_lookup(key: str) -> dict:
return fetch_from_database(key)When to reach for this:
- Pure functions called repeatedly with same arguments
- Expensive I/O (database, HTTP) with stable results
- Configuration or reference data that changes infrequently
Working Example
from functools import lru_cache
import httpx
@lru_cache(maxsize=256)
def get_exchange_rate(currency: str) -> float:
response = httpx.get(f"https://api.example/rates/{currency}")
response.raise_for_status()
return response.json()["rate"]
# First call: HTTP request
# Subsequent calls: cached
rate1 = get_exchange_rate("EUR")
rate2 = get_exchange_rate("EUR") # cache hit
print(get_exchange_rate.cache_info())
# CacheInfo(hits=1, misses=1, maxsize=256, currsize=1)# TTL cache with cachetools
from cachetools import TTLCache, cached
cache = TTLCache(maxsize=100, ttl=300)
@cached(cache)
def get_user(user_id: int) -> dict:
return db.query(user_id)What this demonstrates:
lru_cachefor unbounded-reuse pure functionscache_info()monitors hit rate- TTL cache for data that expires
- External API results cached to reduce latency and load
Deep Dive
Cache Layers
| Layer | Tool | Scope |
|---|---|---|
| In-process | lru_cache, cachetools | Single process |
| Shared | Redis, Memcached | All app instances |
| HTTP | CDN, Cache-Control | Client and edge |
| Framework | FastAPI @lru_cache on deps | Per-worker |
Invalidation
get_exchange_rate.cache_clear() # manual invalidationGotchas
- Caching mutable return values - caller mutates cache. Fix: return copies or immutable types.
- No TTL on lru_cache - stale data forever. Fix: cachetools TTLCache or Redis with expiry.
- Caching functions with side effects - wrong results, missed writes. Fix: cache only pure reads.
- Unbounded cache - memory leak. Fix: always set
maxsize. - Cache key misses unhashable args - TypeError. Fix: convert to hashable types or use custom cache key.
Alternatives
| Alternative | Use When | Don't Use When |
|---|---|---|
| Eager precomputation | Data changes rarely | Dynamic inputs |
| Database query cache | ORM-level | Application-level control needed |
| No cache | Data always fresh | Repeated expensive lookups |
FAQs
lru_cache vs Redis?
lru_cache for single-process, pure functions. Redis for multi-instance shared cache.
How do I cache async functions?
async_lru package or manual dict with asyncio lock.
What maxsize?
Start 128-256. Monitor cache_info() hit rate; adjust.
How do I cache FastAPI responses?
@lru_cache on dependency functions. HTTP caching with Cache-Control headers for clients.
Cache stampede?
Use lock per key or stale-while-revalidate pattern for popular keys.
Can I cache instance methods?
Yes but cache is per-method, not per-instance. Consider functools on standalone function.
How do I test cached code?
func.cache_clear() in test teardown. Or mock the underlying data source.
Django caching?
django.core.cache with Redis backend. @cache_page for view caching.
When is caching wrong?
Financial transactions, real-time data, security-sensitive per-user data without key isolation.
How do I monitor cache hit rate?
cache_info() for lru_cache. Redis INFO stats for shared caches.
Related
- Performance Basics - lru_cache intro
- Memory Profiling - cache memory cost
- Async & Concurrency for Throughput - async caching
- FastAPI Dependency Injection - cached deps
Stack versions: This page was written for Python 3.14.0, FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.