Search across all documentation pages
New Python tools - orchestrators, vector DBs, frameworks - need disciplined spikes and PoCs before fleet-wide adoption. Time boxes, written criteria, and ADR outcomes prevent resume-driven development and production surprises.
Quick-reference recipe card - copy-paste ready.
# Spike charter
**Question:** Can Polars cut ETL time 50% on orders mart?
**Time box:** 3 days (ends Friday EOD)
**Owner:** @data-eng
**Success:** staging job runtime < 12 min (baseline 25 min)
**Non-goals:** production cutover, full refactor
**Output:** README in `spikes/2026-07-polars/` + recommendation ADR draftWhen to reach for this:
# spikes/2026-07-polars/README.md
## Result: Adopt for new marts; do not rewrite legacy yet
| Criterion | Met? |
| Runtime 50% improvement | yes (22→10 min) |
| Team can maintain | yes (2 engineers trained) |
| Ops cost acceptable | yes (same K8s job size) |
| Failure modes understood | partial - document memory spike on wide joins |
## Next steps
- ADR-021 adopt Polars for new pipelines
- Ticket: add Polars to data-platform cookiecutter
- PoC hardening NOT done - production needs observability ticket# spikes/2026-07-polars/benchmark.py - minimal evidence script
import time
import polars as pl
start = time.perf_counter()
df = pl.scan_parquet("sample.parquet").group_by("tenant_id").len().collect()
print(f"rows={df.height} elapsed={time.perf_counter() - start:.2f}s")What this demonstrates:
spikes/ folder.| Dimension | Ask |
|---|---|
| Operability | On-call at 3am? |
| Security | CVE history, supply chain |
| Hiring | Can we recruit skill? |
| Exit cost | Lock-in severity |
| Fit | Python 3.14 compat |
spikes/ folder gitignored for large data; commit only README + scripts| Alternative | Use When | Don't Use When |
|---|---|---|
| Buy SaaS | Commodity capability | Core differentiator |
| Do nothing ADR | Spike shows insufficient gain | Emergency CVE replacement |
| Vendor POC support | Large deal | Tiny experiment |
| Internal tool forever in spike | Never - decide adopt/reject |
Spike answers technical question; PoC demos product value - may be longer but still time-boxed.
Tech lead + platform + security for infra tools; EM for schedule impact.
Include eval set, cost model, and data residency before pilot.
High - document why not; saves future time.
Limit per squad WIP; spikes count toward capacity planning.
Check bus factor, release cadence, CVE response in evaluation table.
Would need extraordinary evidence; spike compares CI time and lock compat.
Only pilot with flag, SLO monitor, and rollback - not silent shadow forever.
Only patterns/tests merge; throwaway scripts stay in spikes/ or deleted.
Estimation & Risk uses spike outcomes.
Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.
Reviewed by Chris St. John·Last updated Jul 16, 2026