subprocess & Shell Interop

subprocess runs external binaries from Python with controlled arguments, timeouts, and captured output. Avoid shell=True unless you accept injection risk and quoting complexity.

Recipe

import subprocess
 
proc = subprocess.run(
    ["git", "rev-parse", "HEAD"],
    check=True,
    capture_output=True,
    text=True,
    timeout=30,
)
print(proc.stdout.strip())

When to reach for this:

• Git, rsync, tar, ffmpeg and other mature CLI tools
• Legacy shell scripts you are wrapping incrementally
• Capture stdout for parsing in Python
• Timeout long-running commands to prevent hung cron jobs

Working Example

Run command with timeout, map errors to exit codes, stream large output line by line.

import logging
import subprocess
from collections.abc import Sequence
 
log = logging.getLogger(__name__)
 
def run_command(cmd: Sequence[str], timeout: float = 60) -> subprocess.CompletedProcess[str]:
    log.info("exec %s", list(cmd))
    try:
        return subprocess.run(
            cmd,
            check=True,
            capture_output=True,
            text=True,
            timeout=timeout,
        )
    except subprocess.TimeoutExpired as exc:
        log.error("timeout after %ss: %s", timeout, cmd)
        raise
    except subprocess.CalledProcessError as exc:
        log.error("exit %s stderr=%s", exc.returncode, exc.stderr.strip())
        raise
 
def stream_lines(cmd: Sequence[str]) -> int:
    with subprocess.Popen(cmd, stdout=subprocess.PIPE, text=True) as proc:
        assert proc.stdout is not None
        for line in proc.stdout:
            print(line.rstrip())
        return proc.wait()
 
if __name__ == "__main__":
    result = run_command(["python3", "--version"])
    print(result.stdout)

What this demonstrates:

• Sequence of args, not one shell string - no injection surface
• check=True raises CalledProcessError with returncode and stderr
• Popen streams stdout without buffering entire output in memory

Deep Dive

API Choice

shell=True Risk

• User input in command string enables shell injection
• Prefer list argv; if shell required, use shlex.quote on user segments

Python Notes

import shlex
 
user_file = "report; rm -rf /"
# BAD: subprocess.run(f"cat {user_file}", shell=True)
subprocess.run(["cat", user_file], check=True)

Gotchas

• shell=True with variables - classic injection bug. Fix: argv list or shlex.split on trusted scripts only.
• No timeout - zombie cron jobs forever. Fix: always set timeout proportional to expected runtime.
• Capturing huge stdout - memory blowup. Fix: stream with Popen or redirect to temp file.
• Ignoring stderr - hidden failure context. Fix: log stderr on non-zero exit.
• Text mode encoding - binary tools break. Fix: text=False and decode explicitly when needed.

Alternatives

FAQs

Related

• Scripting Basics - CLI entry points
• SSH & Remote Execution - remote commands
• Robust Scripts - error handling
• Parsing Logs & Text - parse command output
• System Scripting Best Practices - safe execution

Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.