The Standard Library Mental Model
Python's standard library is often described as "batteries included," but that phrase undersells what it actually is: a versioned, dependency-free contract that ships with the interpreter and covers a fixed set of platform-level jobs - reading and writing files, tracking time, exchanging structured data, matching text, and running other programs.
The pages in this section walk through the concrete modules that do this work - pathlib & os, datetime, zoneinfo & time, json & Serialization, re, subprocess - and Standard Library Overview maps the full module list.
This page is the layer underneath all of them: what "owning a job" actually means for a stdlib module, why that ownership model is worth understanding before reaching for a PyPI package, and where the honest boundary sits between what stdlib solves well and what it deliberately leaves to the ecosystem.
Summary
- The standard library is a set of modules each responsible for exactly one platform-level job, shipped and versioned with the interpreter itself rather than installed separately.
- Insight: Reaching for a PyPI package before checking whether stdlib already solves the problem adds a dependency, a supply-chain surface, and a version to track - for a job that may already have a stable, free answer.
- Key Concepts: module ownership, stdlib stability policy, the str/bytes boundary, batteries-included philosophy, PEP-gated additions.
- When to Use: File paths and OS interaction, civil time and timezones, structured data interchange, text pattern matching, and running external processes - the five jobs this section covers - before adding a dependency for the same job.
- Limitations/Trade-offs: Stdlib deliberately favors stability over ergonomics and newest features, so it lacks things like a modern async HTTP client or built-in data validation, which is exactly where PyPI earns its place.
- Related Topics: packaging and environments, type hints, the PyPI ecosystem, configuration and settings.
Foundations
"Batteries included" means the interpreter ships with working answers to the jobs almost every program eventually needs, so pip install python is never a step - the capability already exists the moment Python is installed.
Each core module in this section maps to exactly one of those jobs and does not overlap with its neighbors: pathlib and os own filesystem paths and OS-level process context, datetime and zoneinfo own civil time and timezone-aware conversions, json owns structured data interchange, re owns text pattern matching, and subprocess owns spawning and communicating with external programs.
That non-overlap is deliberate design, not accident - Python's standard library follows a PEP-driven process for what gets added, so a new stdlib module (like tomllib in 3.11 or zoneinfo in 3.9) had to justify itself as a genuinely stable, broadly needed capability before joining the set, not just a convenient helper someone wanted.
A useful way to think about it: stdlib is the set of tools that came in the toolbox when you bought the house, and PyPI is the hardware store down the street - the toolbox tools are reliable, already paid for, and will still be there next year, while the hardware store has options the toolbox doesn't, at the cost of a trip and an ongoing relationship with that store.
from pathlib import Path
import json
# Each import is a job, not a convenience wrapper around another job
config = json.loads(Path("app.json").read_text(encoding="utf-8"))Mechanics & Interactions
The five modules this section covers interact along one recurring seam: the boundary between text (str) and bytes, which every one of them has to cross explicitly rather than hide from you.
Path.read_text() requires an encoding because a file is bytes on disk and Python strings are Unicode - guessing the encoding wrong produces mojibake, not an error, which is why explicit encoding="utf-8" is the safe default rather than an optional nicety.
json.loads/json.dumps operate on str, meaning any bytes coming from a socket or subprocess have to be decoded first - json itself has no opinion about encoding, it just assumes you already resolved that boundary before calling it.
subprocess.run(..., text=True) decodes the child process's stdout/stderr bytes into str for you, which is the same boundary again, just handled at the process layer instead of the filesystem layer.
re sits on the far side of that boundary once text already exists as str: its patterns are a small, separate language compiled into a matching engine, and that engine's cost model (backtracking on ambiguous patterns) is worth understanding before reaching for a regex to parse something structured that json or tomllib would parse correctly and faster.
datetime and zoneinfo split civil time into two concerns on purpose - a naive datetime is just numbers with no timezone attached, and it becomes aware only once a zoneinfo.ZoneInfo is attached to it - conflating the two is the single most common stdlib time bug, because naive datetimes compare and arithmetic-combine without ever warning you they don't know what timezone they mean.
Advanced Considerations & Applications
Knowing when to graduate from stdlib to a PyPI package is as much a part of this mental model as knowing stdlib itself - the honest answer is usually "when stdlib's contract stops matching your actual requirement," not "when a nicer API exists."
urllib.request handles simple synchronous HTTP fine, but production services need connection pooling, HTTP/2, retries, and async support that stdlib deliberately does not provide - that's the exact gap httpx and aiohttp fill, and it's a considered upgrade, not a stdlib failure.
dataclasses and typing cover structured data and type-level intent well, but once validation rules, coercion, and JSON schema generation matter, pydantic earns its dependency by doing a job stdlib was never designed to do.
Stdlib itself is not static - PEP 594 ("Removing dead batteries") removed a set of legacy modules by Python 3.13, tomllib (3.11) and zoneinfo (3.9) were both added within the last several years, and tracking "what changed" per release is a real, recurring maintenance task, not a one-time learning curve.
Security defaults matter more in this layer than they first appear: subprocess with shell=True interpolates a string into an actual shell, which is an injection vector the moment any part of that string comes from user input, while the list-argument form with shell=False (the default) never invokes a shell at all.
| Job | Stdlib Module | Typical PyPI Upgrade | Upgrade When |
|---|---|---|---|
| Filesystem paths | pathlib / os | rarely needed | almost never - stdlib is the complete answer here |
| Civil time & timezones | datetime / zoneinfo | pendulum, arrow | you want fluent parsing/arithmetic ergonomics across many timezone edge cases |
| Data interchange | json / tomllib | orjson, pydantic | you need speed at scale, or validation and coercion, not just parsing |
| Text pattern matching | re | regex | you need features re lacks (recursive patterns, better Unicode property support) |
| Process orchestration | subprocess | sh, plumbum | you want a shell-scripting-like ergonomic layer over many chained commands |
Common Misconceptions
- "Stdlib is the old, basic stuff - PyPI is where the real tools are." Several stdlib modules are recent additions gated by PEP review (
tomllibin 3.11,zoneinfoin 3.9), and the modules covered here are actively maintained, not legacy leftovers. - "datetime handles timezones correctly by default." A plain
datetimeis naive - it carries no timezone information at all - and naive datetimes will silently compare and combine with each other without ever telling you the comparison may be meaningless;zoneinfois what makes a datetime aware. - "re can parse anything text-shaped, including config and structured data."
reis a pattern-matching engine with its own backtracking cost model, not a parser; reaching forjson,tomllib, orconfigparserfor genuinely structured formats is both more correct and usually faster. - "subprocess with shell=True is just a convenience flag." It hands your string to an actual shell for interpretation, which is a real injection vector the instant any part of the command is influenced by external input - the list-argument,
shell=Falseform avoids this entirely. - "json.dumps just works on any Python object." It only serializes a specific set of built-in types by default and raises
TypeErroron anything else (adatetime, a custom class), which is why real code needs an explicit encoder or a library likepydanticfor anything beyond plain data.
FAQs
What does it mean for a stdlib module to "own" a job?
It means that job - filesystem paths, civil time, data interchange, text matching, process orchestration - has one canonical stdlib answer that is stable across Python versions, rather than several competing conventions.
Why does the standard library evolve so much more slowly than PyPI packages?
Because stdlib changes ship with the interpreter itself and are expected to hold for years across a huge installed base, additions go through a formal PEP review process, and removals follow a long deprecation cycle (PEP 594 removed old modules only by 3.13).
Why do pathlib, json, and subprocess all care about encoding?
Because each one crosses the boundary between bytes (files on disk, subprocess I/O, sockets) and Python's Unicode str at some point, and that boundary never resolves itself - something in the code has to specify the encoding explicitly.
Is a naive datetime ever safe to use?
Only when the entire program agrees on one implicit timezone and never has to compare against another - the moment two datetimes from different sources need to be compared or combined, naive datetimes become a latent bug, and attaching a zoneinfo.ZoneInfo is the fix.
Should I always use re for text validation?
Only when the format genuinely is a text pattern (an identifier shape, a log line format); structured formats like JSON, TOML, or key-value config are parsed more correctly and often faster with json, tomllib, or configparser than with a hand-written regex.
When is subprocess.run safe from shell injection?
When you pass a list of arguments and leave shell=False (the default) - that form never invokes a shell to interpret the string, so there's nothing for an attacker-controlled substring to inject into.
Why doesn't json.dumps just serialize any Python object?
Because arbitrary objects have no canonical JSON representation - json only knows the built-in types (str, int, float, bool, None, list, dict) by default, and anything else needs an explicit encoder function or a library that defines that mapping for you.
What decides whether something belongs in stdlib versus PyPI?
Breadth of need and long-term stability commitment - a capability nearly every program needs, that the core team is willing to support for years, becomes a PEP-reviewed stdlib addition; something narrower, faster-moving, or opinionated stays on PyPI where iteration is cheap.
Is tomllib the same as configparser?
No - tomllib (3.11+) parses TOML, the format pyproject.toml uses, while configparser parses INI-style files; they solve the same category of problem (structured config) for two different formats.
Does stdlib have anything for async HTTP?
Not natively at production quality - urllib.request is synchronous only, which is exactly the gap that httpx and aiohttp fill; this is one of the clearest cases where stdlib's contract stops and a PyPI package is the right upgrade, not a workaround.
Why does knowing "which module owns this job" matter more than memorizing APIs?
Because the API surface changes release to release, but the ownership map - paths go to pathlib, time to datetime/zoneinfo, interchange to json, patterns to re, processes to subprocess - stays stable, so it's the faster thing to actually internalize.
Is it ever wrong to reach for stdlib first?
Rarely for the five jobs this section covers, but yes in general - if the job needs validation, async I/O, or performance stdlib was never designed to deliver, checking stdlib first should take minutes, not become a rule to reinvent a worse version of a mature PyPI library.
Related
- Standard Library Overview - the full module map this page's model sits underneath
- pathlib & os - the filesystem and OS-context job in depth
- datetime, zoneinfo & time - the civil time and timezone job in depth
- json & Serialization - the data interchange job in depth
- subprocess - the process orchestration job in depth
- Standard Library Best Practices - operational rules that follow from this model
Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.