pathlib & os
pathlib provides object-oriented paths (Path) for join, read, glob, and mkdir. os and os.path remain for low-level flags, environment variables, and process ids - combine both in production code.
Recipe
from pathlib import Path
root = Path(__file__).resolve().parent
config = root / "config" / "app.toml"
config.parent.mkdir(parents=True, exist_ok=True)
config.write_text("key = 'value'\n", encoding="utf-8")When to reach for this:
- Building portable file paths
- Walking directories with
glob/rglob - Reading/writing text or bytes files
- Checking existence and file metadata
- Environment and cwd via
os.environ,os.getcwd
Working Example
import os
from pathlib import Path
def find_py_files(src: Path, limit: int = 5) -> list[Path]:
return sorted(src.rglob("*.py"))[:limit]
def ensure_cache() -> Path:
cache = Path(os.environ.get("APP_CACHE", Path.home() / ".cache" / "demo"))
cache.mkdir(parents=True, exist_ok=True)
return cache
root = Path.cwd()
print("cwd", root)
print("sample files", find_py_files(root))
stamp = ensure_cache() / "last.run"
stamp.write_text("ok", encoding="utf-8")
print(stamp.read_text(encoding="utf-8"))What this demonstrates:
/operator joins path segments portablyrglobrecursive glob from rootmkdir(parents=True, exist_ok=True)creates tree- Env override pattern for configurable directories
Deep Dive
pathlib vs os.path
| Task | pathlib | os |
|---|---|---|
| Join | base / "x" | os.path.join |
| Exists | p.exists() | os.path.exists |
| Env | - | os.environ |
| chmod | p.chmod | os.chmod |
Text vs Binary
read_text/write_textneed encodingread_bytesfor images, pickles, archives
Gotchas
- String path concatenation - breaks on Windows. Fix: Path objects.
- Relative paths without anchor - depend on cwd surprises. Fix:
resolve()or pass base dir. - glob on huge trees - slow cold start. Fix: narrow root, use
os.walkpruning, or dedicated index. - Race TOCTOU - exists then open fails. Fix: EAFP open or try/except.
- Missing encoding - mojibake on Windows. Fix: always utf-8 for text unless known locale.
Alternatives
| Alternative | Use When | Don't Use When |
|---|---|---|
| os.walk | Fine-grained prune | Simple glob enough |
| tempfile module | Secure temp files | Fixed cache dir |
| shutil | copy/move tree | Single file pathlib |
FAQs
Path vs str for APIs?
Accept os.PathLike (Path | str) in public functions; convert internal to Path.
Absolute vs resolve?
resolve() follows symlinks and makes absolute - use before comparing paths.
Home directory?
Path.home() or ~ expand via expanduser().
Delete tree?
shutil.rmtree - pathlib has no recursive delete until careful unlink/rmdir.
Watch filesystem?
No stdlib great watcher - watchdog PyPI on some platforms; polling glob for simple cases.
os.environ mutation?
Affects child processes - document side effects in CLI tools.
Symlinks?
is_symlink, readlink, resolve behavior - know your deployment FS.
Pathlib thread-safe?
Path objects immutable; filesystem itself can race - same as os.
Unicode paths?
Python 3 paths are str (unicode) - still watch NFC normalization on macOS.
Django MEDIA_ROOT?
Configure as str path; use Path in new code inside Django for consistency.
Related
- subprocess - run commands with paths
- json & Serialization - read config files
- Standard Library Overview - map
- Files & I/O - fundamentals
Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.