Data Engineering

The mental model underneath every data pipeline - batch vs. streaming as a latency/completeness trade-off, DAGs as dependency graphs rather than execution order, and why idempotency is what makes retries and backfills safe - the model behind every other page in this section.

Batch vs. streaming and the pipeline lifecycle.

Transformations and the analytics-engineering workflow.

Great Expectations and Pandera checks.

Parquet, Arrow, and columnar storage.

Testable, observable, reproducible pipelines.