LlamaIndex
LlamaIndex is a data framework for ingesting, indexing, and querying private data with LLMs. It handles chunking, embedding, retrieval, and response synthesis.
Recipe
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, Settings
from llama_index.embeddings.openai import OpenAIEmbedding
from llama_index.llms.openai import OpenAI
Settings.llm = OpenAI(model="gpt-4o-mini")
Settings.embed_model = OpenAIEmbedding(model="text-embedding-3-small")
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()
response = query_engine.query("What is pytest?")Working Example
"""llamaindex.py - build index and query."""
from llama_index.core import Document, VectorStoreIndex, Settings
from llama_index.embeddings.openai import OpenAIEmbedding
from llama_index.llms.openai import OpenAI as LlamaOpenAI
Settings.llm = LlamaOpenAI(model="gpt-4o-mini", temperature=0)
Settings.embed_model = OpenAIEmbedding(model="text-embedding-3-small")
documents = [
Document(text="pytest uses fixtures to share test setup.", metadata={"source": "testing"}),
Document(text="FastAPI validates request bodies with Pydantic.", metadata={"source": "web"}),
]
index = VectorStoreIndex.from_documents(documents)
engine = index.as_query_engine(similarity_top_k=2)
response = engine.query("How do pytest fixtures work?")
print(response)
print("sources:", [n.metadata for n in response.source_nodes])Gotchas
- Default chunk settings - may not fit your data. Fix: configure
SentenceSplitterchunk_size/overlap. - No source citation by default - enable
response_mode="compact"with source nodes. - Global Settings - thread-unsafe in multi-tenant apps. Fix: pass llm/embed_model per index.
Alternatives
| Alternative | Use When | Don't Use When |
|---|---|---|
| LlamaIndex | Data-heavy RAG with connectors | Simple one-off chains |
| LangChain | Tool/agent composition | Connector-heavy ingestion |
| Raw SDK | Minimal dependencies | Many data sources |
FAQs
LlamaIndex vs LangChain?
LlamaIndex excels at ingestion/querying; LangChain at agent orchestration.How do I load PDFs?
SimpleDirectoryReader("./data") auto-detects file types.What is a query engine?
Retriever + response synthesizer that generates answers from retrieved nodes.Chat engine vs query engine?
Chat engine maintains conversation history across turns.Custom retriever?
Subclass BaseRetriever or configure VectorIndexRetriever params.Persistent index?
StorageContext with ChromaVectorStore or save_to_disk/load_from_disk.How do I cite sources?
response.source_nodes contains retrieved chunks with metadata.Structured output?
PydanticProgram or OpenAIPydanticProgram for typed responses.Agents in LlamaIndex?
ReActAgent with tools; consider LangGraph for complex loops.How do I evaluate?
LlamaIndex eval modules: retriever eval, faithfulness, relevancy.Embedding model change?
Rebuild index - vectors are model-specific.Production serving?
llama_deploy or wrap query_engine in FastAPI.Related
Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.