The Notebook-to-Production Lifecycle
Every page in this section - experiment tracking, model registries, serving, monitoring - is a stage in one continuous lifecycle, and MLOps as a discipline exists because none of those stages work in isolation.
A model trained in a notebook is a liability, not an asset, until it has passed through a pipeline that makes it reproducible, comparable, deployable, and observable.
This page describes that lifecycle end to end, so that the more specific pages in this section read as chapters of one story rather than a list of unrelated tools.
Summary
- MLOps is the discipline of moving a model through a repeatable lifecycle - experiment, register, serve, monitor - so that "it worked on my machine" becomes "it works, and we can prove it, in production."
- Insight: Software engineering solved reproducibility with version control; machine learning adds two more moving parts, data and model weights, that also need versioning, or results silently stop being reproducible.
- Key Concepts: experiment tracking, model registry, model versioning, serving, monitoring, drift.
- When to Use: Any model that will be retrained more than once, touched by more than one person, or run against live traffic needs this lifecycle - a one-off analysis notebook does not.
- Limitations/Trade-offs: The lifecycle adds real overhead (tracking infrastructure, registry servers, monitoring pipelines) that is not worth paying for a model that will be trained once and never touched again.
- Related Topics: data versioning, CI/CD for ML, feature stores, model governance.
Foundations
A notebook is an excellent tool for exploration and a poor tool for production, because it conflates code, state, and output in a way that is nearly impossible to reproduce exactly later.
Experiment tracking is the first stage of the lifecycle, and it exists to solve exactly that problem: every training run logs its parameters, metrics, and resulting artifact to a system built for comparison, not just execution.
Without tracking, "which run produced our best model" becomes an unanswerable question after the fifth or sixth iteration, because notebook cell order and manual print statements do not survive a restarted kernel.
Once a run produces a model worth keeping, it needs to move into a model registry, which is the second stage.
A registry is not just file storage; it is a catalog of versions with lifecycle stages attached (commonly something like None, Staging, Production, Archived), so that "which model is live right now" is always a single, answerable query rather than a guess based on filenames.
The third stage is serving: exposing a registered model version behind an interface, typically an HTTP endpoint, so that other systems can request predictions without needing to know anything about how the model was trained.
The fourth and most frequently skipped stage is monitoring, which watches the live model's inputs and predictions over time to catch drift - the slow divergence between the data a model was trained on and the data it now sees in production.
A simple analogy: training a model is like calibrating a thermometer in a lab.
Experiment tracking is the lab notebook proving which calibration procedure was used.
The registry is the label on the thermometer stating which calibration version it carries.
Serving is putting the thermometer to work in the field.
Monitoring is periodically checking that the thermometer still reads true against a known reference, because the world it measures keeps changing even if the thermometer itself does not.
Mechanics & Interactions
The lifecycle only works if each stage produces an artifact the next stage can consume unambiguously, which is why versioning threads through every stage rather than living in just one place.
An experiment run should version three things together: the code (git commit), the data (a hash or a DVC-tracked pointer), and the resulting parameters and metrics, because a model's behavior is a function of all three, and changing any one without recording it breaks reproducibility.
The registry's job is to take the winning run from experiment tracking and give it a stable identity - a name and version number - that serving can reference without caring which run produced it.
This decoupling matters mechanically: serving code should say "load models:/churn-predictor/Production," never "load the file my colleague trained last Tuesday," because the former survives retraining and redeployment while the latter does not.
Promotion between registry stages (Staging to Production) is where testing and validation actually get enforced - it is the mechanical gate between "a model that scored well offline" and "a model allowed to answer real requests."
Serving itself has its own internal trade-off: batch inference processes large volumes on a schedule with high throughput and no latency pressure, while online serving answers individual requests in real time with strict latency budgets, and a model's serving pattern should be chosen before, not after, the deployment architecture is built.
Monitoring closes the loop back to experiment tracking, because detecting drift is only useful if it triggers a new experiment, using fresh data, that produces a new candidate version to register and promote.
# The lifecycle as a chain of lookups, not a single script
run = experiment_tracker.best_run(metric="f1", experiment="churn")
version = registry.register(run.model_artifact, name="churn-predictor")
registry.promote(version, stage="Staging") # gate: offline validation
# ... validation passes ...
registry.promote(version, stage="Production") # gate: now serving reads this
serving_model = registry.load("churn-predictor", stage="Production")The point of this snippet is the shape, not the specific API: each stage hands off a small, stable reference (a run ID, a version number, a stage label) rather than passing raw files or notebook state directly between people.
Advanced Considerations & Applications
At scale, the lifecycle has to handle two things a single-model demo never surfaces: multiple models competing for the same production slot, and models whose accuracy degrades without any code change at all.
Champion/challenger patterns extend the registry model beyond a single "Production" label, running a new candidate alongside the incumbent on a fraction of traffic before a full promotion, which turns promotion from a single risky cutover into a gradual, measured rollout.
Drift itself splits into at least two distinct failure modes that require different monitoring: data drift, where the distribution of incoming features shifts (new customer segment, seasonal behavior change), and concept drift, where the relationship between features and the true label changes even if feature distributions look stable (fraud patterns evolving, for example).
Neither kind of drift raises an exception or a 500 error - the model keeps returning confident predictions - which is exactly why monitoring has to be a deliberate, separate stage rather than something you assume error logs will catch.
GPU-backed training and serving introduce a cost dimension that CPU-only lifecycles do not have to reason about as carefully: idle GPU capacity between training runs, and serving infrastructure that must be right-sized against genuinely bursty traffic rather than provisioned for peak at all times.
Compliance-heavy environments add a further requirement on top of the base lifecycle: every promotion needs an audit trail linking a live production version back through its registry entry, its experiment run, and the exact data snapshot it trained on.
| Approach | Strength | Weakness | Best Fit |
|---|---|---|---|
| Manual notebook + file copy | Zero setup, fastest for a single analyst | No reproducibility, no rollback, no audit trail | One-off analysis, throwaway prototypes |
| Experiment tracking only (no registry) | Comparable runs, easy adoption | No clear "what is live" answer, manual promotion | Small teams, few models, low deployment frequency |
| Full lifecycle (tracking + registry + serving + monitoring) | Reproducible, auditable, safe rollback, drift-aware | Real infrastructure and process overhead | Any model retrained regularly or serving live traffic |
| Champion/challenger rollout | Safer promotions, real-traffic validation | Needs traffic-splitting infra and clear success metrics | High-stakes models, frequent retraining cadence |
Common Misconceptions
- "MLOps is just DevOps applied to ML." DevOps versions code; MLOps must also version data and model weights, which change independently of code and are usually far larger, so the tooling and workflows differ meaningfully.
- "Once a model is deployed, the work is done." Deployment is the midpoint of the lifecycle, not the end - a model with no monitoring is a model nobody will notice has gone stale until users complain.
- "High accuracy in training guarantees good production performance." Offline metrics are computed against a fixed, historical dataset; production traffic is a moving target, which is precisely the gap that drift monitoring exists to catch.
- "A model registry is just a folder of files with version numbers in the name." A folder does not enforce stage transitions, does not link back to the run that produced each version, and gives serving code no stable way to ask "which version is authoritative right now."
- "Retraining on a schedule solves drift." Scheduled retraining helps, but without monitoring you cannot tell whether the schedule is frequent enough, and you retrain blind to whether the previous version had actually degraded yet.
FAQs
What is the difference between experiment tracking and a model registry?
- Experiment tracking records many runs for comparison, most of which are never deployed.
- The registry holds only the runs that were promoted to a real version with a lifecycle stage.
- Think: tracking is the lab notebook, the registry is the shipped, labeled product.
Do I need MLOps tooling for a small side project?
Probably not the full stack. A single notebook and manual file saves are fine until more than one person touches the model, it gets retrained more than once, or it serves real traffic - at that point reproducibility problems appear quickly.
How does versioning actually work across code, data, and models together?
Each experiment run should record a git commit (code), a data hash or DVC pointer (data), and a registered artifact (model), so any past result can be reconstructed from three coordinates rather than tribal memory.
What is the mechanical difference between batch and online serving?
Batch inference runs on a schedule over a large dataset with relaxed latency and high throughput; online serving answers individual requests synchronously under a strict latency budget, which usually means a smaller, faster model or extra optimization work.
Why doesn't a model raise an error when it goes stale from drift?
The model has no awareness that the world changed; it applies the same learned function to new inputs and returns a confident-looking prediction regardless of whether that function still matches reality, which is why drift needs explicit statistical monitoring, not error logs.
What's the difference between data drift and concept drift?
Data drift is a shift in the incoming feature distributions themselves; concept drift is a shift in the relationship between features and the true outcome, even if the features look statistically similar to before.
When should I use champion/challenger instead of a direct promotion?
Use it when the cost of a bad promotion is high enough to justify validating a new version against a slice of real traffic before it fully replaces the incumbent, rather than relying only on offline metrics.
Is GPU cost part of the MLOps lifecycle or a separate concern?
It's part of the same lifecycle - training and serving cadence directly determine GPU utilization, and idle or oversized GPU capacity is a recurring, measurable cost that the lifecycle's design choices directly influence.
Why can't I just keep deploying the latest trained model automatically?
Without a staging gate and validation step, a regression in data quality or a training bug ships straight to production with no checkpoint to catch it - the registry's stage transitions exist specifically to insert a review point.
What does "reproducibility" actually require in ML, beyond code?
The exact training data snapshot, the exact code version, the exact hyperparameters, and often the exact library versions - any one of these changing can change the resulting model even with identical code.
How is this lifecycle different for a simple scikit-learn model versus a large deep learning model?
The stages are the same, but the cost and cadence differ - large models make retraining and champion/challenger rollouts more expensive, which pushes teams toward heavier monitoring to avoid unnecessary retrains.
What's the single biggest lifecycle mistake teams make?
Treating deployment as the finish line and skipping monitoring entirely, which means the first sign of drift is a business metric quietly declining weeks after the model actually stopped being accurate.
Related
- MLOps Basics - hands-on walkthrough of the same lifecycle stages
- Experiment Tracking - the first stage in depth
- Model Registries & Versioning - promotion and rollback mechanics
- Serving with FastAPI - exposing a registered model as an endpoint
- Monitoring and Drift - detecting data and concept drift in production
- Data & Model Versioning (DVC) - versioning the data half of reproducibility
Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.