Scheduling & Cron

Python scripts become reliable jobs when scheduled with system cron, systemd timers, or in-process schedulers like APScheduler. The script itself must be idempotent, logged, and exit with meaningful codes.

Recipe

# crontab -e
0 2 * * * cd /opt/jobs && /opt/jobs/.venv/bin/python backup.py >>/var/log/backup.log 2>&1

from apscheduler.schedulers.blocking import BlockingScheduler
 
sched = BlockingScheduler()
sched.add_job(lambda: print("tick"), "interval", minutes=5)
sched.start()

When to reach for this:

• Nightly ETL and report generation
• Certificate and log rotation checks
• Polling APIs when webhooks unavailable
• Dev/long-running agent with APScheduler in one process

Working Example

Cron-friendly script with lock file, logging, and APScheduler alternative for dev.

import fcntl
import logging
import sys
from contextlib import contextmanager
from pathlib import Path
 
LOCK = Path("/tmp/daily-sync.lock")
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(message)s")
log = logging.getLogger("daily-sync")
 
@contextmanager
def single_instance(lock_path: Path):
    lock_path.parent.mkdir(parents=True, exist_ok=True)
    with lock_path.open("w") as f:
        try:
            fcntl.flock(f.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)
        except BlockingIOError:
            log.warning("another instance running")
            raise SystemExit(0)
        yield
 
def job() -> int:
    log.info("sync start")
    # ... work ...
    log.info("sync done")
    return 0
 
def main() -> int:
    with single_instance(LOCK):
        return job()
 
if __name__ == "__main__":
    raise SystemExit(main())

What this demonstrates:

• File lock prevents overlapping cron invocations when job exceeds interval
• Exit 0 when skipping duplicate run - cron does not alert falsely
• Absolute paths to venv python in crontab avoid wrong interpreter

Deep Dive

Scheduler Options

Cron Hygiene

• Set PATH and cd to project root
• Use venv python absolute path
• Redirect stdout/stderr to rotated logs
• Monitor last-success timestamp externally

Python Notes

from apscheduler.schedulers.background import BackgroundScheduler
 
scheduler = BackgroundScheduler()
scheduler.add_job(send_heartbeat, "cron", hour=3)
scheduler.start()  # non-blocking in web app - shutdown on exit

Gotchas

• Relative paths in cron - file not found. Fix: cd to app dir or absolute paths everywhere.
• Overlapping runs - double processing. Fix: flock or distributed lock (Redis/DynamoDB).
• Silent cron failure - empty log means success even on crash before logging. Fix: wrap top-level try/except, mail on non-zero exit.
• APScheduler in multi-worker Gunicorn - duplicate jobs. Fix: only scheduler in dedicated process or use OS cron.
• Timezone surprises - cron uses server TZ. Fix: document UTC vs local; use TZ= in crontab if needed.

Alternatives

FAQs

Related

• Robust Scripts - locks and exit codes
• Scripting Basics - CLI entry point
• Working with APIs & Webhooks - polling fallback
• System Scripting Best Practices - unattended operations
• Prefect & Dagster - workflow orchestration at scale

Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.