re - Regular Expressions
The re module matches, searches, and substitutes text via regex. Compile patterns for reuse; use raw strings r"..." for readable escapes; prefer str methods when regex is overkill.
Recipe
import re
EMAIL = re.compile(r"^[\w.+-]+@[\w-]+\.[\w.-]+$")
def valid_email(value: str) -> bool:
return bool(EMAIL.match(value))When to reach for this:
- Log parsing and extraction
- Tokenizing structured but irregular text
- Validation with known pattern limits
- sed-like substitutions
- Splitting on variable whitespace
Working Example
import re
LOG_LINE = re.compile(
r"^(?P<level>INFO|ERROR)\s+(?P<msg>.+)$"
)
def parse_log_lines(text: str) -> list[dict[str, str]]:
rows = []
for line in text.strip().splitlines():
m = LOG_LINE.match(line)
if m:
rows.append(m.groupdict())
return rows
sample = "INFO started\nERROR disk full\n"
print(parse_log_lines(sample))
redacted = re.sub(r"\b\d{4}-\d{4}-\d{4}-\d{4}\b", "[CARD]", "4111-1111-1111-1111")
print(redacted)What this demonstrates:
- Named groups via
(?P<name>...) matchanchors start;searchscans anywheresubreplaces sensitive data patterns- Compile once at module level for hot paths
Deep Dive
Common Methods
| Method | Purpose |
|---|---|
re.search | First match anywhere |
re.match | Start of string |
re.findall | All non-overlapping |
re.sub | Replace |
Flags
re.IGNORECASE,re.MULTILINE,re.DOTALLre.compile(..., flags)combine
Gotchas
- Catastrophic backtracking - evil nested quantifiers. Fix: possessive limits, atomic groups, or regex engine timeout.
- Over-regex - email/url validation incomplete vs real RFC. Fix: pragmatic pattern + secondary verification.
- Recompiling in loop - slow. Fix: module-level compiled patterns.
- match vs search confusion -
matchmisses mid-string. Fix: pick correct method. - Locale-dependent
\w- unicode surprises. Fix: explicit character classes[A-Za-z0-9_].
Alternatives
| Alternative | Use When | Don't Use When |
|---|---|---|
| str.split/partition | Fixed delimiter | Variable pattern |
| parsimonious/lark | Grammar languages | Simple extract |
| pandas str.extract | Columnar data | Single string |
FAQs
Raw string required?
Not required but r"\n" vs "\\n" clarity strongly recommended.
findall groups?
Returns tuples if multiple capturing groups - use finditer for objects.
greedy vs lazy?
*?, +? lazy quantifiers - match minimal span.
verbose regex?
re.compile(r"...", re.VERBOSE) allows comments and whitespace in pattern.
Unicode?
Default unicode str patterns in Python 3; use (?a) for ASCII-only when needed.
Replace count?
sub(pattern, repl, s, count=1) limit replacements.
Lookahead?
(?=...) positive lookahead - assert without consuming.
Testing regex?
Table-driven cases in pytest; document pattern in docstring.
JSON in regex?
Do not parse JSON with regex - use json module.
re vs regex PyPI?
regex module adds features; stdlib sufficient for most services.
Related
- json & Serialization - structured parse instead
- subprocess - parse command output
- Standard Library Overview - map
- Strings & f-strings - text basics
Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.