Prompt Engineering
Prompt engineering shapes LLM behavior through instructions, examples, and structure. Good prompts reduce hallucinations, enforce output format, and improve task accuracy without retraining the model.
Recipe
Quick-reference recipe card - copy-paste ready.
SYSTEM = """You are a Python code reviewer.
- Flag bugs, security issues, and style problems.
- Respond in bullet points.
- If code is fine, say "LGTM"."""
messages = [
{"role": "system", "content": SYSTEM},
{"role": "user", "content": f"Review this code:\n```python\n{code}\n```"},
]When to reach for this:
- Defining consistent behavior across application features.
- Teaching output format with few-shot examples.
- Improving reasoning with chain-of-thought prompting.
- Reducing hallucinations with grounding instructions.
Working Example
"""prompt_engineering.py - system, few-shot, chain-of-thought, and structured prompts."""
from __future__ import annotations
from openai import OpenAI
client = OpenAI()
# Chain-of-thought for reasoning
cot_messages = [
{"role": "system", "content": "Solve math problems step by step. Show your work, then give the final answer on the last line as ANSWER: <number>."},
{"role": "user", "content": "A store has 24 apples. They sell 3/8 of them. How many remain?"},
]
cot = client.chat.completions.create(model="gpt-4o-mini", messages=cot_messages, temperature=0)
print(cot.choices[0].message.content)
# Few-shot classification
few_shot = [
{"role": "system", "content": "Classify support tickets as billing, technical, or account. Reply with one word only."},
{"role": "user", "content": "I was charged twice this month"},
{"role": "assistant", "content": "billing"},
{"role": "user", "content": "The API returns 500 errors on POST /users"},
{"role": "assistant", "content": "technical"},
{"role": "user", "content": "I need to change my email address"},
]
result = client.chat.completions.create(model="gpt-4o-mini", messages=few_shot, temperature=0)
print("classification:", result.choices[0].message.content)
# Grounded QA - only use provided context
context = "Our API rate limit is 1000 requests/minute for Pro plans and 100/minute for Free plans."
grounded = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "system", "content": "Answer ONLY using the provided context. If the answer is not in the context, say 'I don't have that information.'"},
{"role": "user", "content": f"Context:\n{context}\n\nQuestion: What is the Pro plan rate limit?"},
],
temperature=0,
)
print(grounded.choices[0].message.content)What this demonstrates:
- Chain-of-thought with explicit answer format.
- Few-shot examples teaching classification labels.
- Grounded QA restricting answers to provided context.
temperature=0for deterministic classification and extraction.
Deep Dive
How It Works
- System prompt sets persistent instructions (role, constraints, format).
- Few-shot provides input/output examples in the message history.
- Chain-of-thought asks the model to reason step-by-step before answering.
- Grounding restricts answers to provided context (RAG pattern).
- Output format instructions specify JSON, markdown, or bullet structure.
Prompt Patterns
| Pattern | Technique | Best For |
|---|---|---|
| Zero-shot | Instructions only | Simple, well-defined tasks |
| Few-shot | 2-5 examples | Classification, extraction |
| Chain-of-thought | "Think step by step" | Math, logic, multi-step reasoning |
| Grounded | "Use only this context" | RAG, factual QA |
| Role-based | "You are a senior X" | Tone and expertise level |
Python Notes
# Template prompts with variables
def build_review_prompt(code: str, focus: str) -> list[dict]:
return [
{"role": "system", "content": f"You are a code reviewer focusing on {focus}."},
{"role": "user", "content": f"```python\n{code}\n```"},
]
# Store prompts in version-controlled files
from pathlib import Path
SYSTEM_PROMPT = Path("prompts/reviewer.txt").read_text()Gotchas
- Vague instructions - "be helpful" produces inconsistent output. Fix: specific role, format, and constraints.
- Too many few-shot examples - wastes tokens without improving accuracy. Fix: 2-5 diverse examples; trim if performance plateaus.
- No output format specification - model returns prose when you need JSON. Fix: explicit format in system prompt or
response_format. - Prompt injection from user input - users override system instructions. Fix: separate system/user clearly; validate outputs; use delimiters.
- Not iterating - first prompt is rarely optimal. Fix: test on 20+ examples; measure accuracy before deploying.
- Mixing instructions and data - model confuses what to follow vs what to process. Fix: use clear delimiters (
---, XML tags, markdown sections).
Alternatives
| Alternative | Use When | Don't Use When |
|---|---|---|
| Prompt engineering | Quick iteration, no training data | Need consistent behavior at scale |
| Fine-tuning | Thousands of labeled examples | Few examples (use few-shot) |
| RAG | Answers need current/private data | Model already knows the information |
| Structured output / tools | Need guaranteed schema | Free-form text is acceptable |
FAQs
How long should a system prompt be?
- As long as needed, but every token costs money.
- Start concise; add constraints when you see failures.
- Move stable instructions to system; keep user messages for task data.
What is chain-of-thought?
- Ask the model to show reasoning before the final answer.
- Improves accuracy on math, logic, and multi-step tasks.
- Use
temperature=0for reproducible reasoning.
How do I prevent prompt injection?
- Never concatenate user input into system prompts.
- Use delimiters:
<user_input>...</user_input>. - Validate and sanitize model outputs before execution.
Should I put examples in system or user messages?
- Few-shot examples go in user/assistant message pairs.
- System prompt holds rules and format constraints.
How do I test prompt quality?
- Build an eval set of 20-50 input/expected-output pairs.
- Measure accuracy, not vibes.
- See Evaluation & Guardrails.
What is the ReAct pattern?
- Reason + Act: model thinks, calls a tool, observes result, repeats.
- Implemented in agent frameworks like LangGraph.
How do I handle long inputs?
- Summarize or chunk input before sending.
- Use models with larger context windows.
- RAG retrieves only relevant chunks.
Does prompt order matter?
- Yes - recent tokens have more influence.
- Put the most important instructions in system prompt.
- Place the actual task last in the user message.
How do I version prompts?
- Store in git-tracked files (
prompts/v2/reviewer.txt). - Log prompt version with each API call.
- A/B test prompt variants on eval sets.
When should I fine-tune instead?
- Thousands of consistent input/output pairs.
- Prompt engineering plateaus on your eval set.
- Need lower latency or smaller model with same quality.
What are XML tags for prompts?
<context>{retrieved_docs}</context>
<question>{user_query}</question>- Claude and GPT respond well to structured delimiters.
How do I reduce hallucinations?
- Ground with retrieved context (RAG).
- Instruct "say I don't know if not in context."
- Lower temperature; require citations.
Related
- LLMs Basics - messages and roles
- Structured Output - JSON schemas
- RAG Basics - grounded answers
- Evaluation & Guardrails - prompt testing
- Tokens, Cost & Rate Limits - prompt length cost
Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.