Files & I/O

Reading and writing files is a daily task - logs, config, uploads, and exports. pathlib.Path provides an object-oriented path API, while context managers ensure descriptors close even when errors occur.

Recipe

from pathlib import Path
 
path = Path("data/config.json")
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text('{"debug": true}', encoding="utf-8")
raw = path.read_bytes()
text = raw.decode("utf-8")

When to reach for this:

Loading JSON/YAML/TOML configuration at startup
Writing audit logs and CSV exports
Streaming large files without loading all bytes
Atomic config updates via temp file + rename

Working Example

import json
import tempfile
from pathlib import Path
 
def atomic_write_json(path: Path, payload: dict) -> None:
    path.parent.mkdir(parents=True, exist_ok=True)
    with tempfile.NamedTemporaryFile(
        mode="w"

What this demonstrates:

Atomic write via temp file in target directory then Path.replace
Text reads always pass encoding="utf-8"
Binary copy streams chunks - bounded memory for large files
mkdir(parents=True, exist_ok=True) creates nested directories safely

Deep Dive

How It Works

Text vs binary - "r"/"w" return str; "rb"/"wb" return bytes.
Context managers - with open(...) as f calls f.close() in finally.
pathlib - Wraps os/open with Path methods (read_text, glob, iterdir).
Encodings - UTF-8 default for cross-platform text; specify errors policy (strict, replace).
File descriptors - OS limits open files; always close or use with.

Common Path Operations

Task	pathlib
Join paths	`base / "child" / "file.txt"`
Exists check	`path.exists()`
Glob	`path.glob("*/.py")`
Resolve	`path.resolve()` absolute path

Python Notes

from contextlib import ExitStack
 
with ExitStack() as stack:
    a = stack.enter_context(path_a.open())
    b = stack.enter_context(path_b.open())
    # both close on exit

Gotchas

Missing encoding - Platform default may not be UTF-8 on Windows. Fix: encoding="utf-8" always for text.
Reading huge files with read_text - Loads entire file into RAM. Fix: Iterate lines or chunk binary reads.
Path string concat - "dir" + "/file" breaks on Windows. Fix: Path("dir") / "file".
Toctou races - Check exists() then open - another process may delete. Fix: try/except FileNotFoundError.
Forgetting newline on write - Logs run together. Fix: Write line + "\n" or print(..., file=fh).

Alternatives

Alternative	Use When	Don't Use When
`pathlib`	Most file tasks	Need lowest-level os.open flags
`aiofiles`	Async FastAPI uploads	Sync CLI script
`tempfile`	Atomic writes, scratch	Permanent storage paths
Object storage (S3)	Durable shared assets	Local dev config only

FAQs

pathlib or os.path?

Prefer pathlib for new code - readable operators and high-level read/write helpers.

How do I append to a log file?

with path.open("a", encoding="utf-8") as fh: fh.write(line + "\n")

What mode for binary images?

"rb" and "wb" - never decode images as UTF-8 text.

How do I list files recursively?

for p in Path("src").rglob("*.py"): print(p)

What is atomic replace?

Write temp in same filesystem directory, then Path.replace - readers see old or new file, never partial.

How do I handle missing files?

Catch FileNotFoundError or check path.exists() when optional config is acceptable.

Can pathlib read JSON directly?

No - read text then json.loads. Or use json.load with an open file handle.

What encoding errors policy should I use?

strict for validated pipelines; replace or backslashreplace for lossy log ingestion only.

How do I get file size?

path.stat().st_size without reading contents.

Does with close on exception?

Yes - context manager protocol runs __exit__ and closes the file even when an error propagates.

Strings & f-strings - encode/decode text
Context Managers - advanced with patterns
arrays, bytes & memoryview - binary buffers
Virtual Environments Quickstart - project paths

Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.

Search Documentation

Files & I/O