LLM App / RAG Skill
Retrieval and agent app patterns - an Agent Skill for Python 3.14 RAG/LLM services.
What This Skill Does
Produces RAG application checklist: document loaders, chunking strategy, embedding pipeline, vector store adapter, retrieval API, LLM orchestration (LangChain/LlamaIndex/LangGraph stubs), evaluation harness, and observability (latency, token usage, retrieval hit rate).
When to Invoke
- New internal knowledge-base chat over Confluence/S3 docs
- PR review on chunk size, metadata filters, or missing citations
- Adding tool-use agent loop with human approval gate
- Post-incident hardening after prompt injection or data leak
Inputs
| Input | Why |
|---|---|
| Corpus source | PDF, HTML, tickets, code repos |
| Latency budget | p95 target for user-facing chat |
| Model provider | OpenAI, Anthropic, local vLLM |
| Vector DB | pgvector, Pinecone, Chroma |
| Compliance | PII redaction, retention, audit log |
Outputs
ingest/andretrieval/modules with typed config- Chunking parameters documented (size, overlap, splitter)
- Eval JSON with question, expected_doc_id, min_score
- Prompt templates with mandatory citation format
- Tool registry with auth scopes for agent skills
- Verification:
uv run pytest tests/test_retrieval.py
Guardrails
- Ground answers in retrieved chunks - show citations or refuse.
- Sanitize uploads - no arbitrary code execution from parsed PDF macros.
- Separate system and user prompts - resist injection via document content.
- Rate limit and log prompts - redact secrets and PII in traces.
- Version embeddings - reindex when model changes.
- Human approval for destructive tools - delete, charge, email external.
Recipe
uv run python -m rag.ingest --source data/docs/
uv run python -m rag.index --rebuild
uv run pytest tests/test_golden.py -q
uv run python -m rag.chat --question "What is the refund policy?"Working Example (retrieval stub)
from dataclasses import dataclass
@dataclass
class Chunk:
doc_id: str
text: str
score: float
def retrieve(query: str, store, k: int = 5) -> list[Chunk]:
hits = store.similarity_search_with_score(query, k=k)
return [Chunk(doc_id=h[0].metadata["doc_id"], text=h[0].page_content, score=h[1]) for h in hits]FAQs
LangChain or LlamaIndex?
Follow team ADR - skill outputs interface stubs both can implement; human docs in ai-agents-rag are authoritative.
When to add agents vs single-shot RAG?
Agents when multi-step tools needed - start RAG-only until retrieval quality plateaus.
Related
- RAG Basics - cookbook
- Chunking and Ingestion - depth
- Evaluation and Guardrails - safety
- Vector Databases - storage
Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.