Drop-in wrapper: litellm.completion(model="claude-sonnet-4-20250514", ...). Routes to the correct provider SDK based on model name prefix.

OpenAI & Other SDKs

Q: Can I use OpenAI embeddings with pgvector?

Yes - store text-embedding-3-small vectors (1536 dims) in PostgreSQL. See Vector Databases.

Q: What about open-source models via OpenAI-compatible API?

vLLM and Ollama expose /v1/chat/completions compatible endpoints. Point base_url to the local server.

Q: How do I handle content policy refusals?

Check finish_reason == "content_filter". Provide fallback messaging to users.

The OpenAI Python SDK is the reference pattern for chat completions, embeddings, and tool calling. LiteLLM and similar libraries abstract multiple providers when you need vendor flexibility.

Recipe

Quick-reference recipe card - copy-paste ready.

from openai import OpenAI
 
client = OpenAI()  # reads OPENAI_API_KEY
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello"}],
)
print(response.choices[0].message.content)

When to reach for this:

Building on GPT-4o, GPT-4o-mini, or embedding models.
Migrating from OpenAI SDK v0 (openai.ChatCompletion) to v1.
Abstracting across OpenAI, Anthropic, and local models with LiteLLM.
Calling the Responses API or batch endpoints for cost savings.

Working Example

"""openai_sdks.py - chat, embeddings, tools, and LiteLLM fallback."""
from __future__ import annotations
 
import json
import os
from openai import OpenAI, AsyncOpenAI
 
client = OpenAI()
 
# Chat completion
chat = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is FastAPI?"},
    ],
    temperature=0.2,
)
print(chat.choices[0].message.content)
 
# Embeddings
emb = client.embeddings.create(
    model="text-embedding-3-small",
    input=["Python data pipelines", "Machine learning basics"],
)
print("embedding dim:", len(emb.data[0].embedding))
 
# Function calling
tools = [{
    "type": "function",
    "function": {
        "name": "search_docs",
        "description": "Search internal documentation",
        "parameters": {
            "type": "object",
            "properties": {"query": {"type": "string"}},
            "required": ["query"],
        },
    },
}]
tool_resp = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Find docs about pydantic validators"}],
    tools=tools,
)
msg = tool_resp.choices[0].message
if msg.tool_calls:
    call = msg.tool_calls[0]
    print(f"tool: {call.function.name}({call.function.arguments})")
 
# LiteLLM multi-provider
try:
    import litellm
    response = litellm.completion(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": "Hi"}],
    )
    print("litellm:", response.choices[0].message.content[:50])
except ImportError:
    print("litellm not installed - skip")

What this demonstrates:

OpenAI SDK v1 client pattern for chat and embeddings.
Tool/function calling with JSON schema parameters.
Embedding vectors for semantic search pipelines.
LiteLLM as a drop-in multi-provider wrapper.

Deep Dive

How It Works

Client instance holds API key, base URL, and HTTP configuration.
Chat completions send messages; model returns a completion choice.
Embeddings map text to dense vectors for similarity search.
Tool calls let the model request function execution with parsed arguments.
AsyncOpenAI mirrors the sync API for async web servers.

Provider Comparison

Provider	SDK	Strengths
OpenAI	`openai`	GPT models, embeddings, wide tooling
Anthropic	`anthropic`	Claude, long context, tool use
Google	`google-genai`	Gemini, multimodal
LiteLLM	`litellm`	Unified API across 100+ models

Python Notes

# Custom base URL (Azure OpenAI, local proxy)
client = OpenAI(base_url=os.environ["OPENAI_BASE_URL"], api_key=os.environ["OPENAI_API_KEY"])
 
# Async usage in FastAPI
async def ask(question: str) -> str:
    client = AsyncOpenAI()
    resp = await client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": question}],
    )
    return resp.choices[0].message.content or ""

Gotchas

Using deprecated v0 API - openai.ChatCompletion.create is removed. Fix: migrate to OpenAI() client.
Hardcoded API keys - keys in source get committed to git. Fix: environment variables and secrets managers.
Not handling tool_calls - model requests a function but code ignores it. Fix: check message.tool_calls and loop.
Ignoring rate limits - 429 errors under load. Fix: exponential backoff; respect retry-after headers.
Wrong embedding model dimensions - mixing models breaks vector search. Fix: pin one embedding model per index.
Assuming non-null content - refusals or tool calls leave content as None. Fix: check before using .content.

Alternatives

Alternative	Use When	Don't Use When
OpenAI SDK	GPT models directly	Need Claude or Gemini
LiteLLM	Multi-provider apps	Single provider, no abstraction needed
LangChain LLM wrappers	Chains and agents	Simple one-off API calls
Raw HTTP (httpx)	Full control, no SDK dep	Standard SDK features suffice

FAQs

How do I migrate from openai v0 to v1?

# v0 (deprecated)
# openai.ChatCompletion.create(model="gpt-3.5-turbo", messages=[...])
# v1
client = OpenAI()
client.chat.completions.create(model="gpt-4o-mini", messages=[...])

What is the Responses API?

Newer OpenAI API for agentic workflows with built-in tool handling.
Check current docs for migration from chat completions.

How do I use Azure OpenAI?

client = OpenAI(
    api_key=os.environ["AZURE_OPENAI_API_KEY"],
    base_url=f"{os.environ['AZURE_OPENAI_ENDPOINT']}/openai/v1/",
)

How do I batch requests for cost savings?

OpenAI Batch API processes requests asynchronously at lower cost.
Good for offline embedding generation or bulk evaluation.

What is LiteLLM?

Drop-in wrapper: litellm.completion(model="claude-sonnet-4-20250514", ...).
Routes to the correct provider SDK based on model name prefix.

How do I list available models?

models = client.models.list()
for m in models.data:
    print(m.id)

How do I set timeouts?

client = OpenAI(timeout=30.0, max_retries=3)

Can I use OpenAI embeddings with pgvector?

Yes - store text-embedding-3-small vectors (1536 dims) in PostgreSQL.
See Vector Databases.

How do I log requests?

OpenAI dashboard shows usage and costs.
Log prompt hashes and token counts in your application.

What about open-source models via OpenAI-compatible API?

vLLM and Ollama expose /v1/chat/completions compatible endpoints.
Point base_url to the local server.

How do I handle content policy refusals?

Check finish_reason == "content_filter".
Provide fallback messaging to users.

How do I test without API calls?

Mock the client in unit tests.
Use recorded fixtures (VCR-style) for integration tests.

Anthropic Claude SDK - Claude-specific patterns
Embeddings & Similarity - embedding models
Structured Output - JSON mode
Local & Open Models - Ollama and vLLM
Tool Use & Function Calling - tool loops

Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.

Recipe

Quick-reference recipe card - copy-paste ready.

from openai import OpenAI
 
client = OpenAI()  # reads OPENAI_API_KEY
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello"}],
)
print(response.choices[0].message.content)

When to reach for this:

Building on GPT-4o, GPT-4o-mini, or embedding models.
Migrating from OpenAI SDK v0 (openai.ChatCompletion) to v1.
Abstracting across OpenAI, Anthropic, and local models with LiteLLM.
Calling the Responses API or batch endpoints for cost savings.

Working Example

"""openai_sdks.py - chat, embeddings, tools, and LiteLLM fallback."""
from __future__ import annotations
 
import json
import os
from openai import OpenAI, AsyncOpenAI
 
client = OpenAI()
 
# Chat completion
chat = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is FastAPI?"},
    ],
    temperature=0.2,
)
print(chat.choices[0].message.content)
 
# Embeddings
emb = client.embeddings.create(
    model="text-embedding-3-small",
    input=["Python data pipelines", "Machine learning basics"],
)
print("embedding dim:", len(emb.data[0].embedding))
 
# Function calling
tools = [{
    "type": "function",
    "function": {
        "name": "search_docs",
        "description": "Search internal documentation",
        "parameters": {
            "type": "object",
            "properties": {"query": {"type": "string"}},
            "required": ["query"],
        },
    },
}]
tool_resp = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Find docs about pydantic validators"}],
    tools=tools,
)
msg = tool_resp.choices[0].message
if msg.tool_calls:
    call = msg.tool_calls[0]
    print(f"tool: {call.function.name}({call.function.arguments})")
 
# LiteLLM multi-provider
try:
    import litellm
    response = litellm.completion(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": "Hi"}],
    )
    print("litellm:", response.choices[0].message.content[:50])
except ImportError:
    print("litellm not installed - skip")

What this demonstrates:

OpenAI SDK v1 client pattern for chat and embeddings.
Tool/function calling with JSON schema parameters.
Embedding vectors for semantic search pipelines.
LiteLLM as a drop-in multi-provider wrapper.

Deep Dive

How It Works

Client instance holds API key, base URL, and HTTP configuration.
Chat completions send messages; model returns a completion choice.
Embeddings map text to dense vectors for similarity search.
Tool calls let the model request function execution with parsed arguments.
AsyncOpenAI mirrors the sync API for async web servers.

Provider Comparison

Provider	SDK	Strengths
OpenAI	`openai`	GPT models, embeddings, wide tooling
Anthropic	`anthropic`	Claude, long context, tool use
Google	`google-genai`	Gemini, multimodal
LiteLLM	`litellm`	Unified API across 100+ models

Python Notes

# Custom base URL (Azure OpenAI, local proxy)
client = OpenAI(base_url=os.environ["OPENAI_BASE_URL"], api_key=os.environ["OPENAI_API_KEY"])
 
# Async usage in FastAPI
async def ask(question: str) -> str:
    client = AsyncOpenAI()
    resp = await client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": question}],
    )
    return resp.choices[0].message.content or ""

Gotchas

Using deprecated v0 API - openai.ChatCompletion.create is removed. Fix: migrate to OpenAI() client.
Hardcoded API keys - keys in source get committed to git. Fix: environment variables and secrets managers.
Not handling tool_calls - model requests a function but code ignores it. Fix: check message.tool_calls and loop.
Ignoring rate limits - 429 errors under load. Fix: exponential backoff; respect retry-after headers.
Wrong embedding model dimensions - mixing models breaks vector search. Fix: pin one embedding model per index.
Assuming non-null content - refusals or tool calls leave content as None. Fix: check before using .content.

Alternatives

Alternative	Use When	Don't Use When
OpenAI SDK	GPT models directly	Need Claude or Gemini
LiteLLM	Multi-provider apps	Single provider, no abstraction needed
LangChain LLM wrappers	Chains and agents	Simple one-off API calls
Raw HTTP (httpx)	Full control, no SDK dep	Standard SDK features suffice

FAQs

How do I migrate from openai v0 to v1?

# v0 (deprecated)
# openai.ChatCompletion.create(model="gpt-3.5-turbo", messages=[...])
# v1
client = OpenAI()
client.chat.completions.create(model="gpt-4o-mini", messages=[...])

What is the Responses API?

Newer OpenAI API for agentic workflows with built-in tool handling.
Check current docs for migration from chat completions.

How do I use Azure OpenAI?

client = OpenAI(
    api_key=os.environ["AZURE_OPENAI_API_KEY"],
    base_url=f"{os.environ['AZURE_OPENAI_ENDPOINT']}/openai/v1/",
)

How do I batch requests for cost savings?

OpenAI Batch API processes requests asynchronously at lower cost.
Good for offline embedding generation or bulk evaluation.

What is LiteLLM?

Drop-in wrapper: litellm.completion(model="claude-sonnet-4-20250514", ...).
Routes to the correct provider SDK based on model name prefix.

How do I list available models?

models = client.models.list()
for m in models.data:
    print(m.id)

How do I set timeouts?

client = OpenAI(timeout=30.0, max_retries=3)

Can I use OpenAI embeddings with pgvector?

Yes - store text-embedding-3-small vectors (1536 dims) in PostgreSQL.
See Vector Databases.

How do I log requests?

OpenAI dashboard shows usage and costs.
Log prompt hashes and token counts in your application.

What about open-source models via OpenAI-compatible API?

vLLM and Ollama expose /v1/chat/completions compatible endpoints.
Point base_url to the local server.

How do I handle content policy refusals?

Check finish_reason == "content_filter".
Provide fallback messaging to users.

How do I test without API calls?

Mock the client in unit tests.
Use recorded fixtures (VCR-style) for integration tests.

Anthropic Claude SDK - Claude-specific patterns
Embeddings & Similarity - embedding models
Structured Output - JSON mode
Local & Open Models - Ollama and vLLM
Tool Use & Function Calling - tool loops

Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.

OpenAI & Other SDKs

Search Documentation

OpenAI & Other SDKs