LLM App / RAG Skill

Retrieval and agent app patterns - an Agent Skill for Python 3.14 RAG/LLM services.

What This Skill Does

Produces RAG application checklist: document loaders, chunking strategy, embedding pipeline, vector store adapter, retrieval API, LLM orchestration (LangChain/LlamaIndex/LangGraph stubs), evaluation harness, and observability (latency, token usage, retrieval hit rate).

When to Invoke

• New internal knowledge-base chat over Confluence/S3 docs
• PR review on chunk size, metadata filters, or missing citations
• Adding tool-use agent loop with human approval gate
• Post-incident hardening after prompt injection or data leak

Inputs

Outputs

• ingest/ and retrieval/ modules with typed config
• Chunking parameters documented (size, overlap, splitter)
• Eval JSON with question, expected_doc_id, min_score
• Prompt templates with mandatory citation format
• Tool registry with auth scopes for agent skills
• Verification: uv run pytest tests/test_retrieval.py

Guardrails

1. Ground answers in retrieved chunks - show citations or refuse.
2. Sanitize uploads - no arbitrary code execution from parsed PDF macros.
3. Separate system and user prompts - resist injection via document content.
4. Rate limit and log prompts - redact secrets and PII in traces.
5. Version embeddings - reindex when model changes.
6. Human approval for destructive tools - delete, charge, email external.

Recipe

uv run python -m rag.ingest --source data/docs/
uv run python -m rag.index --rebuild
uv run pytest tests/test_golden.py -q
uv run python -m rag.chat --question "What is the refund policy?"

Working Example (retrieval stub)

from dataclasses import dataclass
 
 
@dataclass
class Chunk:
    doc_id: str
    text: str
    score: float
 
 
def retrieve(query: str, store, k: int = 5) -> list[Chunk]:
    hits = store.similarity_search_with_score(query, k=k)
    return [Chunk(doc_id=h[0].metadata["doc_id"], text=h[0].page_content, score=h[1]) for h in hits]

FAQs

Related

• RAG Basics - cookbook
• Chunking and Ingestion - depth
• Evaluation and Guardrails - safety
• Vector Databases - storage

Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.