Spikes, PoCs & Adopting New Tools
New Python tools - orchestrators, vector DBs, frameworks - need disciplined spikes and PoCs before fleet-wide adoption. Time boxes, written criteria, and ADR outcomes prevent resume-driven development and production surprises.
Recipe
Quick-reference recipe card - copy-paste ready.
# Spike charter
**Question:** Can Polars cut ETL time 50% on orders mart?
**Time box:** 3 days (ends Friday EOD)
**Owner:** @data-eng
**Success:** staging job runtime < 12 min (baseline 25 min)
**Non-goals:** production cutover, full refactor
**Output:** README in `spikes/2026-07-polars/` + recommendation ADR draftWhen to reach for this:
- RFC open question
- Vendor demo looks amazing
- Team wants to replace Celery/Airflow/xyz
- Estimation blocked on unknown
Working Example
# spikes/2026-07-polars/README.md
## Result: Adopt for new marts; do not rewrite legacy yet
| Criterion | Met? |
| Runtime 50% improvement | yes (22→10 min) |
| Team can maintain | yes (2 engineers trained) |
| Ops cost acceptable | yes (same K8s job size) |
| Failure modes understood | partial - document memory spike on wide joins |
## Next steps
- ADR-021 adopt Polars for new pipelines
- Ticket: add Polars to data-platform cookiecutter
- PoC hardening NOT done - production needs observability ticket# spikes/2026-07-polars/benchmark.py - minimal evidence script
import time
import polars as pl
start = time.perf_counter()
df = pl.scan_parquet("sample.parquet").group_by("tenant_id").len().collect()
print(f"rows={df.height} elapsed={time.perf_counter() - start:.2f}s")What this demonstrates:
- Charter question is falsifiable
- Spike output is short README with decision recommendation
- PoC explicitly not production-ready
- ADR follows for adopt/reject fleet-wide
Deep Dive
How It Works
- Spike - Learning; throwaway code OK in
spikes/folder. - PoC - Demonstrates value to stakeholders; still not prod SLO.
- Pilot - One service in prod with flags and runbook - bridge to adoption.
- Adoption - Cookiecutter, CI, training, on-call runbook updated.
- Reject - ADR documents why not chosen - prevents re-debate monthly.
Evaluation Dimensions
| Dimension | Ask |
|---|---|
| Operability | On-call at 3am? |
| Security | CVE history, supply chain |
| Hiring | Can we recruit skill? |
| Exit cost | Lock-in severity |
| Fit | Python 3.14 compat |
Python Notes
spikes/ folder gitignored for large data; commit only README + scriptsGotchas
- Spike becomes production - No tests or runbook. Fix: Hardening checklist gate for pilot.
- No time box - Endless research. Fix: Calendar end; present whatever learned.
- Success criteria after results - Bias. Fix: Criteria in charter before code.
- Adopt because demo shiny - Ops burden ignored. Fix: Operator sign-off on evaluation table.
- Reject without ADR - Relitigate quarterly. Fix: Short ADR "rejected Polars 2024; superseded by ADR-021."
Alternatives
| Alternative | Use When | Don't Use When |
|---|---|---|
| Buy SaaS | Commodity capability | Core differentiator |
| Do nothing ADR | Spike shows insufficient gain | Emergency CVE replacement |
| Vendor POC support | Large deal | Tiny experiment |
| Internal tool forever in spike | Never - decide adopt/reject |
FAQs
Spike vs PoC?
Spike answers technical question; PoC demos product value - may be longer but still time-boxed.
Who approves adoption?
Tech lead + platform + security for infra tools; EM for schedule impact.
LLM tool spikes?
Include eval set, cost model, and data residency before pilot.
Failed spike value?
High - document why not; saves future time.
Multiple spikes parallel?
Limit per squad WIP; spikes count toward capacity planning.
Open source health?
Check bus factor, release cadence, CVE response in evaluation table.
Replace uv?
Would need extraordinary evidence; spike compares CI time and lock compat.
PoC in prod?
Only pilot with flag, SLO monitor, and rollback - not silent shadow forever.
Spike code merge?
Only patterns/tests merge; throwaway scripts stay in spikes/ or deleted.
Link to estimation?
Estimation & Risk uses spike outcomes.
Related
- Architecture Decision Records (ADRs) - adopt/reject record
- Technical Decision-Making - ranked options
- Running Design Reviews - spike outcomes in RFC
- Dependency & Supply-Chain Governance - new dep approval
- Prioritizing Platform & Tech Debt - capacity for spikes
Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.