The Python Dependency Model
Python ships with one of the largest standard libraries of any mainstream language, covering JSON, HTTP servers, unit testing, concurrency primitives, and more, yet almost every real Python codebase still pulls in dozens of packages from PyPI.
That isn't a contradiction so much as the actual shape of the ecosystem.
The standard library is the broad, generic floor every installation can assume, while PyPI is where domain-specific depth, ergonomics, and raw speed actually live - a typed HTTP client, a validation layer, a dataframe engine tuned in Rust.
This page is the reasoning underneath the packages documented elsewhere in this section - httpx & requests, pydantic-settings, tenacity, SQLModel / SQLAlchemy, celery - each of those pages assumes the model described here: where a dependency belongs, what a version number actually promises, and what you take on every time you add one.
Summary
- Python's standard library covers the generic case adequately; nearly every production concern that needs to be fast, ergonomic, or domain-specific - HTTP, validation, ORMs, retries - is filled by a deliberately chosen PyPI package, not accumulated by default.
- Insight: Dependency choices compound - a package added casually today is a version to track, a transitive graph to audit, and a behavior to understand for as long as the project lives.
- Key Concepts: standard library, PyPI, virtual environment, dependency resolver, PEP 440 versioning, lockfile.
- When to Use: Deciding whether the standard library already solves a problem, evaluating a candidate package before adding it, and reasoning about why an existing dependency was chosen over an alternative.
- Limitations/Trade-offs: This model describes how to choose well - it doesn't eliminate the underlying cost of any dependency, and even the best-chosen package is still code you didn't write, running in your interpreter.
- Related Topics: virtual environments, packaging and publishing, supply-chain auditing, boundary validation.
Foundations
Python's standard library is genuinely broad - json, http.server, unittest, asyncio, dataclasses, sqlite3, and dozens more ship with every interpreter, no install required.
That's a real difference from ecosystems with a deliberately minimal core, and it means small scripts and internal tools often need zero third-party packages at all.
Production services still reach past it constantly, because the standard library optimizes for "works everywhere, changes slowly" rather than "fastest, most ergonomic option available today."
PyPI (the Python Package Index) is where that faster-moving layer lives - packages published independently, versioned on their own schedule, installed into a project's environment rather than baked into the interpreter.
A useful analogy: the standard library is a city's public utilities, water and electricity that every building already has connected. PyPI packages are the specialists you hire per project - a plumber who only does high-pressure systems, an electrician who only does data centers - because the generic utility connection doesn't cover what a particular building actually needs.
A virtual environment (venv, or one managed by uv) is that project's private meter room: its own isolated set of installed packages, separate from every other project's, so two projects on the same machine can depend on completely different versions of the same library without conflict.
[project]
dependencies = [
"httpx>=0.27", # PyPI: a project-specific choice, versioned independently
"pydantic>=2.0", # PyPI: same
]
# json, asyncio, dataclasses need no entry here - they ship with the interpreterThis split is exactly why "essential libraries" is its own subject at all - a project's dependency list isn't an accident of whatever got installed along the way, it's a set of deliberate choices about which gaps the standard library leaves, and with what.
Mechanics & Interactions
The single highest-leverage question when deciding where a dependency's responsibility should start and stop is: is this a boundary or is this domain logic?
A boundary is any point where data crosses from outside your program's control into it: an HTTP request body, an environment variable, HTML scraped from a page, a row read from an untrusted spreadsheet.
Boundaries are exactly where a validation layer like pydantic earns its keep, because untrusted shape and untrusted values both arrive there first, before anything downstream can safely trust them.
Domain logic, by contrast, operates on data that's already been validated and normalized - it generally shouldn't need to re-import a validation library or guess at a payload's shape, because that work already happened at the edge.
That same boundary instinct applies to outbound HTTP (retry and timeout policy belongs at the client boundary, see tenacity and httpx & requests) and to configuration (pydantic-settings parses environment variables once, at startup, rather than scattering os.environ.get() calls through business logic).
Underneath every pip install or uv add, a dependency resolver is what turns a list of top-level requirements into one consistent, installable graph.
Two packages that each depend on different, incompatible versions of a third package create a conflict the resolver has to solve, backtracking through candidate versions until it finds a combination that satisfies every constraint simultaneously, or reports that none exists.
uv add "sqlmodel>=0.0.16" # resolver walks the graph, picks compatible versions
uv lock # pins the exact resolved graph to uv.lock
uv sync --frozen # every machine/CI installs precisely what's lockedPEP 440 is Python's version-string convention (MAJOR.MINOR.PATCH, plus pre-release and post-release suffixes), and like semantic versioning elsewhere, it's a promise a maintainer chooses to follow, not something PyPI mechanically verifies before accepting an upload.
A lockfile (uv.lock, or a hash-pinned requirements.txt) is what actually makes an install reproducible: pyproject.toml records an accepted range for each dependency, while the lockfile pins the exact versions - and, ideally, cryptographic hashes - of the entire graph that was installed and tested.
Advanced Considerations & Applications
Every dependency you add also adds supply chain surface: code you didn't write, running with the same privileges as your own, pulled in transitively by packages you did choose deliberately.
Python's packaging story adds a wrinkle most pure-source ecosystems don't have: many popular packages (NumPy, cryptography, some database drivers) ship compiled C or Rust extensions, distributed as prebuilt wheels for common platforms so most installs never trigger a local compiler. When no matching wheel exists for a platform, pip or uv falls back to building from a source distribution (sdist) - a build that can fail on machines without the right compiler toolchain, and one more reason to pin exact versions in CI rather than letting a fresh resolve silently switch to a source build.
Packaging tooling itself has consolidated over the past several years: setup.py and ad-hoc requirements.txt files have given way to pyproject.toml (PEP 621) as the standard project metadata format, with uv emerging as a fast, Rust-based resolver and installer that unifies virtual-environment creation, dependency resolution, and locking into one tool rather than three.
Not every "do I need a library" decision looks the same, either. A single-purpose package (httpx for HTTP) is a very different bet than a large framework that bundles its own opinions about routing, ORM, and admin tooling (Django) - the framework case trades more control for more structure, and is usually decided once, near a project's inception, rather than incrementally.
| Approach | Strength | Weakness | Best Fit |
|---|---|---|---|
| Standard library only | Zero install, zero third-party risk, always available | Missing validation, async HTTP clients, ORMs, and more - real gaps to fill yourself | Small scripts, internal tools with no real production surface |
| Single-purpose package | Small footprint, one clear job, easy to swap later | You assemble and wire several of them yourself | Most production services - the default posture this section documents |
| Bundled framework (e.g. Django) | Consistent conventions across a large team, less individual wiring | Heavier footprint, harder to swap one piece later, steeper onboarding | Larger teams that value consistency over incremental flexibility |
| Roll your own | Exactly the behavior you need, no external maintenance dependency | You now own testing, edge cases, and security review forever | Genuinely novel problems with no well-maintained existing package |
Data science and ML projects add another axis: a package like PyTorch or Polars pulls in large native binaries and a different install-size calculus than a typical web dependency, which is one more reason "does this need a library at all" is worth asking before "which library."
Common Misconceptions
- "The standard library being large means Python has no real dependency problem." It reduces the frequency of needing a package for simple tasks, but production concerns like validation, async HTTP, and ORMs still live in PyPI, with the same version-tracking and supply-chain cost as any other ecosystem.
- "Pinning a version in
pyproject.tomlis the same as a reproducible install."pyproject.tomlrecords an accepted range; only the lockfile pins the exact resolved graph that was actually tested - without it, two installs of the "same" project can resolve differently over time. - "A wheel install means nothing was compiled anywhere." It means compilation already happened upstream, by the package's maintainers, for common platforms - a platform without a matching wheel still falls back to building from source locally.
- "A popular package on PyPI is automatically a safe, well-maintained one." Popularity correlates with scrutiny but doesn't guarantee it - maintenance activity, security response history, and typosquatting risk are separate questions worth checking directly.
- "PEP 440 version numbers guarantee a minor release can't break my code." It's a convention a maintainer opts into, not something PyPI enforces before accepting an upload - a mistagged release can still violate it.
FAQs
What's the actual difference between the standard library and a PyPI package?
The standard library ships with every Python interpreter and needs no installation; a PyPI package is published independently, installed into a project's virtual environment, and versioned on its own schedule, separate from the interpreter's release cycle.
Why doesn't Python's standard library just include validation, async HTTP clients, and an ORM?
The standard library favors broad compatibility and slow, careful change, while validation, HTTP, and ORM needs evolve quickly and benefit from independent iteration. Some especially universal needs (asyncio, dataclasses) have moved into core over time, but the general gap is intentional.
How do I decide whether something needs a dependency at all?
Ask whether the problem sits at a genuine boundary - untrusted input, an external system, a cross-cutting concern like configuration - versus internal domain logic that a well-chosen dependency at the boundary should have already made safe to work with.
What does "parse at the boundary" mean in Python terms?
It means validating and normalizing data once, where it enters the program - a FastAPI request body, an environment variable read through pydantic-settings, a scraped HTML page - so everything downstream can trust its shape instead of re-checking it.
What is a dependency resolver actually solving?
It's finding one consistent set of package versions that satisfies every top-level and transitive requirement at once, backtracking through candidates when two dependencies want incompatible versions of a shared package.
Why do I need both `pyproject.toml` and a lockfile?
pyproject.toml records an accepted range for each dependency; the lockfile (uv.lock, or a hash-pinned requirements.txt) pins the exact resolved versions of the entire graph that was installed and tested. Without the lockfile, the same project file can resolve to a different graph on a different machine.
What is a Python package's "supply chain," and why does it matter?
It's the full set of packages, direct and transitive, that end up running inside the interpreter as a result of your choices. A single direct dependency can pull in dozens of others you never explicitly chose, each one now part of your security and maintenance surface.
What's the difference between a wheel and a source distribution (sdist)?
A wheel is a prebuilt package for a specific platform, so installing it needs no local compiler. An sdist is raw source that gets built locally at install time - a step that can fail on machines missing the right toolchain, especially for packages with C or Rust extensions.
Why has `pyproject.toml` replaced `setup.py` as the standard?
pyproject.toml (PEP 621) declares project metadata and dependencies in one standard, tool-agnostic format, whereas setup.py was executable Python code that different tools had to actually run to discover the same information - a slower and less predictable approach.
Does a virtual environment protect against a malicious package?
No - it isolates one project's dependency graph from another project's on the same machine, but any package installed inside it still runs with the same privileges as your own code once imported. Isolation limits cross-project damage, not what an installed package can do within its own environment.
When does a bundled framework like Django make more sense than assembling single-purpose libraries?
When a larger team benefits more from shared, enforced conventions (routing, ORM, admin tooling) than from the flexibility of choosing and swapping each piece independently - it's a heavier commitment, usually made once near a project's inception.
Is it ever right to roll your own instead of using a library?
Yes, but rarely - mainly when the problem is genuinely novel and no well-maintained package fits, since rolling your own means you now own testing, edge cases, and security review indefinitely, work a maintained package's community already does.
Related
- httpx & requests - a boundary-facing HTTP client choice in practice
- pydantic-settings - validating configuration at the boundary
- tenacity - retry policy as a boundary-adjacent concern
- SQLModel / SQLAlchemy - a bundled-vs-single-purpose dependency decision in the data layer
- Essential Libraries Best Practices - condensed operational rules that follow from this model
Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13) and uv 0.6+.