AI Agents & RAG Highlights Summary
Every highlight bullet from the 12 pages in this section, gathered on one page and grouped by the page it came from.
- RAG is a five-stage pipeline, not a single vector database call
- Each pipeline stage trades off recall, precision, latency, and cost
- Agents wrap an LLM in a loop that can call tools and re-observe results
- Tool use turns free-text generation into structured, callable actions
- Both models fail differently - retrieval failures vs runaway loops
- Frameworks like LangChain, LlamaIndex, and LangGraph implement these models
- RAG grounds LLM answers in retrieved documents
- Ingest: chunk, embed, and index documents
- Query: embed question, retrieve top-k, generate answer
- Reduces hallucinations on domain-specific questions
- Chunk size balances context richness and retrieval precision
- Overlap prevents losing context at boundaries
- Clean HTML, headers, and boilerplate before chunking
- Store metadata (source, page, section) with each chunk
- Vector DBs store embeddings and run similarity search
- pgvector adds vector search to PostgreSQL
- Chroma is ideal for local prototyping
- Pin embedding model and dimensions per collection
- Hybrid search combines BM25 keywords with dense embeddings
- Re-rankers rescore top-k candidates for better precision
- Retrieve more (k=20) then re-rank down to k=5
- Evaluate retrieval with recall@k on labeled queries
- LangChain composes LLM calls, retrievers, and tools into chains
- LCEL (pipe syntax) chains Runnable components
- Retrievers abstract vector store queries
- Use LangGraph for stateful agent loops
- LlamaIndex focuses on data ingestion and querying
- VectorStoreIndex is the core retrieval primitive
- Query engines combine retrieval and synthesis
- Supports 100+ data connectors
- LangGraph models agents as state machines with nodes and edges
- Cycles enable tool-call loops until the task completes
- Checkpointers persist state across turns
- Conditional edges route based on LLM decisions
- Tools extend LLMs with database, API, and code execution abilities
- Define tools with JSON schema describing parameters
- Validate tool arguments before execution
- Limit tool scope to minimum required permissions
- MCP standardizes how LLMs connect to tools and data sources
- Servers expose tools, resources, and prompts
- Clients discover and invoke server capabilities
- Enables reusable tool servers across applications
- Eval sets with labeled inputs and expected outputs
- Faithfulness checks if answers match retrieved context
- Guardrails filter harmful or off-topic outputs
- LLM-as-judge for semantic evaluation at scale
- Ground answers in retrieved context
- Validate tool inputs and limit permissions
- Set max iterations on agent loops
- Evaluate retrieval and generation separately
Reviewed by Chris St. John·Last updated Jul 31, 2026