Process & Resource Management
ps, htop, signals, nohup, and resource limits on Linux - manage Python web workers, Celery processes, and long-running scripts without orphan zombies or OOM kills.
Recipe
pgrep -af "uvicorn|celery|gunicorn"
kill -TERM <pid>
sleep 5
kill -KILL <pid> # only if still running
htop -p <pid>When to reach for this:
- Deploy left duplicate uvicorn on same port
- Worker memory leak needs restart
- Graceful shutdown before maintenance window
- Diagnose CPU spike from runaway loop
Working Example
# Find billing-api master and workers
pgrep -af gunicorn
# Graceful reload (if USR2/HUP configured) or TERM
sudo kill -TERM "$(pgrep -f 'gunicorn.*billing-api' | head -1)"
sleep 10
sudo ss -tlnp | grep 8000
# Check open files for connection leak
PID=$(pgrep -f 'uvicorn app.main:app' | head -1)
ls -l /proc/$PID/fd | wc -l
cat /proc/$PID/limits | grep "open files"What this demonstrates:
pgrep -afshows full command line for Python processes- TERM allows FastAPI/Gunicorn graceful close
- KILL only after timeout
/procinspection for fd leaks
Deep Dive
How It Works
- Signals - SIGTERM polite shutdown; SIGKILL immediate; SIGUSR1/2 app-specific.
- Parent/child - Gunicorn master spawns workers; kill master to recycle all.
- OOM killer - Linux kills high RSS process under memory pressure.
- cgroups - Container memory/CPU limits in Docker/K8s.
Signal Cheat Sheet
| Signal | Python service effect |
|---|---|
| SIGTERM | Uvicorn/Gunicorn stops accepting; drains requests |
| SIGINT | Same as Ctrl+C in foreground dev |
| SIGHUP | Reload config in some daemons |
| SIGKILL | Immediate death - no finally blocks |
Python Notes
# app registers SIGTERM handler for cleanup
import signal
def handle_term(signum, frame):
close_db_pool()
signal.signal(signal.SIGTERM, handle_term)Gotchas
- SIGKILL first - Drops in-flight HTTP and DB transactions. Fix: TERM, wait, then KILL.
- Killing wrong PID from pgrep - Pattern too broad matches two apps. Fix: verify cmdline in
ps -fp. - nohup in production - Orphans on SSH disconnect without log rotation. Fix: systemd unit.
- Zombie processes - Parent not reaping children. Fix: restart parent supervisor.
- Ignoring OOM dmesg - Mystery restarts. Fix:
dmesg -T | rg -i oom. - Too low ulimit - "Too many open files" under load. Fix:
LimitNOFILEin systemd.
Alternatives
| Alternative | Use When | Don't Use When |
|---|---|---|
| systemd restart | Production services | Quick local experiment |
| Docker stop | Container grace period | Bare metal VM |
| Kubernetes delete pod | Orchestrated roll | SSH troubleshooting |
| py-spy | Sampling stuck Python | Simple restart suffices |
FAQs
How long to wait after SIGTERM?
Match upstream load balancer drain timeout - often 10-30s for APIs.
Multiple uvicorn workers?
One process per CPU with --workers N under gunicorn; each appears in pgrep.
Celery warm shutdown?
celery control shutdown or TERM worker; allow task acks_late to finish.
nice/ionice for batch jobs?
nice -n 10 uv run python batch.py reduces CPU contention with API on same VM.
cgroups v2 memory max?
systemd-run -p MemoryMax=2G ... caps runaway worker on shared host.
strace when?
Last resort performance/debug - huge overhead; try py-spy first.
lsof vs ss?
ss for listening ports; lsof -p for open files/sockets per PID.
Load average vs CPU?
High load with low CPU may be IO wait - check iostat.
Child defunct zombie?
Identify parent with ps -o ppid= -p ZPID; restart parent process.
Free memory on Linux?
available column in free -h matters more than free cache reading.
Related
- Sysadmin Essentials - systemd
- SSH & Remote Work - remote kill
- Deploying FastAPI - gunicorn layout
- Performance Basics - profiling
Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.