Files & I/O

Reading and writing files is a daily task - logs, config, uploads, and exports. pathlib.Path provides an object-oriented path API, while context managers ensure descriptors close even when errors occur.

Recipe

from pathlib import Path
 
path = Path("data/config.json")
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text('{"debug": true}', encoding="utf-8")
raw = path.read_bytes()
text = raw.decode("utf-8")

When to reach for this:

• Loading JSON/YAML/TOML configuration at startup
• Writing audit logs and CSV exports
• Streaming large files without loading all bytes
• Atomic config updates via temp file + rename

Working Example

import json
import tempfile
from pathlib import Path
 
def atomic_write_json(path: Path, payload: dict) -> None:
    path.parent.mkdir(parents=True, exist_ok=True)
    with tempfile.NamedTemporaryFile(
        mode="w",
        encoding="utf-8",
        dir=path.parent,
        delete=False,
        suffix=".tmp",
    ) as tmp:
        json.dump(payload, tmp, indent=2)
        tmp_path = Path(tmp.name)
    tmp_path.replace(path)
 
def tail_lines(path: Path, max_lines: int = 50) -> list[str]:
    if not path.exists():
        return []
    lines = path.read_text(encoding="utf-8").splitlines()
    return lines[-max_lines:]
 
def copy_binary(src: Path, dst: Path, chunk_size: int = 1024 * 64) -> None:
    dst.parent.mkdir(parents=True, exist_ok=True)
    with src.open("rb") as rf, dst.open("wb") as wf:
        while chunk := rf.read(chunk_size):
            wf.write(chunk)
 
if __name__ == "__main__":
    cfg = Path("data/settings.json")
    atomic_write_json(cfg, {"retries": 3, "timeout": 30})
    print(tail_lines(Path("data/app.log")))

What this demonstrates:

• Atomic write via temp file in target directory then Path.replace
• Text reads always pass encoding="utf-8"
• Binary copy streams chunks - bounded memory for large files
• mkdir(parents=True, exist_ok=True) creates nested directories safely

Deep Dive

How It Works

• Text vs binary - "r"/"w" return str; "rb"/"wb" return bytes.
• Context managers - with open(...) as f calls f.close() in finally.
• pathlib - Wraps os/open with Path methods (read_text, glob, iterdir).
• Encodings - UTF-8 default for cross-platform text; specify errors policy (strict, replace).
• File descriptors - OS limits open files; always close or use with.

Common Path Operations

Python Notes

from contextlib import ExitStack
 
with ExitStack() as stack:
    a = stack.enter_context(path_a.open())
    b = stack.enter_context(path_b.open())
    # both close on exit

Gotchas

• Missing encoding - Platform default may not be UTF-8 on Windows. Fix: encoding="utf-8" always for text.
• Reading huge files with read_text - Loads entire file into RAM. Fix: Iterate lines or chunk binary reads.
• Path string concat - "dir" + "/file" breaks on Windows. Fix: Path("dir") / "file".
• Toctou races - Check exists() then open - another process may delete. Fix: try/except FileNotFoundError.
• Forgetting newline on write - Logs run together. Fix: Write line + "\n" or print(..., file=fh).

Alternatives

FAQs

Related

• Strings & f-strings - encode/decode text
• Context Managers - advanced with patterns
• arrays, bytes & memoryview - binary buffers
• Virtual Environments Quickstart - project paths

External Related Links

• Python Standard Library Reference - Complete documentation for built-in modules and functions.
• Python Tutorial (Official) - Official beginner-to-intermediate tutorial.
• Automate the Boring Stuff with Python (online book) - Free practical book focused on automation.
• Automate the Boring Stuff with Python (3rd Edition) - Practical automation (Al Sweigart).
• Python Crash Course (3rd Edition) - Beginner-friendly projects (Eric Matthes).

Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.