pandas Series & DataFrames

pandas adds labeled axes to columnar data: a Series is one typed column; a DataFrame is a table of aligned Series sharing an index.

Recipe

Quick-reference recipe card - copy-paste ready.

import pandas as pd
 
df = pd.DataFrame(
    {"region": ["East", "West"], "revenue": [120, 340]},
    index=pd.Index(["a", "b"], name="row"),
)
east = df.loc[df["region"] == "East", "revenue"]
subset = df.iloc[0:1, [0, 1]]

When to reach for this:

Tabular EDA with mixed column types and missing values
Labeled time series indexed by timestamps
Joining multiple tables on keys before aggregation
Exporting analysis results to CSV, Parquet, or Excel

Working Example

import pandas as pd
 
orders = pd.DataFrame(
    {
        "order_id": [101, 102, 103, 104],
        "region": pd.Categorical(["East", "West", "East", "West"]),
        "revenue": [120.0, 340.0, None, 210.0],
        "ordered_at": pd.to_datetime(
            ["2025-01-01", "2025-01-02", "2025-01-03", "2025-01-04"], utc=True
        ),
    }
).set_index("order_id")
 
# Label-based selection
east = orders.loc[orders["region"] == "East", ["region", "revenue"]]
 
# Position-based head
head = orders.iloc[:2]
 
# Add computed column safely
orders = orders.copy()
orders["revenue_filled"] = orders["revenue"].fillna(orders["revenue"].median())
 
# Series reduction
monthly_total = orders["revenue_filled"].sum()
 
print(east)
print("monthly_total:", monthly_total)

What this demonstrates:

Construction with categorical and nullable float columns
UTC datetime parsing at load time
.loc boolean filtering with column subset
.copy() before assignment to avoid SettingWithCopyWarning

Deep Dive

How It Works

Index labels align rows across columns - operations match on index automatically.
Block manager (pandas 2.x) stores columns in Arrow/numpy-backed blocks by dtype.
.loc includes both endpoints for labels; .iloc is half-open like Python slices.
dtype backend (string[pyarrow]) changes memory and missing-value behavior.

Selection Cheat Sheet

Goal	API
Filter rows	`df.loc[mask]`
Pick columns	`df[["a", "b"]]`
Scalar by label	`df.loc[row, "col"]`
Scalar by position	`df.iloc[i, j]`

Parameters & Return Values

Parameter	Type	Description
`index`	Index-like	Row labels; defaults to RangeIndex
`columns`	list[str]	Column names for DataFrame
`dtype`	str/np.dtype	Force column type on construction

Gotchas

Chained indexing assignment - df[df["x"] > 0]["y"] = 1 triggers SettingWithCopyWarning. Fix: df.loc[df["x"] > 0, "y"] = 1.
Duplicate index labels - .loc["a"] returns multiple rows; scalar access fails. Fix: reset_index() or enforce unique keys upstream.
Object dtype strings - default object columns use lots of RAM. Fix: df["col"] = df["col"].astype("string[pyarrow]") in pandas 2.2+.
Silent dtype coercion - mixing ints and floats upgrades column to float. Fix: use nullable Int64 when NA is possible.
Timezone-naive vs aware - comparing mixed datetimes errors in pandas 2.x. Fix: pd.to_datetime(..., utc=True) everywhere at ingest.

Alternatives

Alternative	Use When	Don't Use When
Polars DataFrame	Lazy scans, very large data, strict typing	Team already standardized on pandas idioms
PyArrow Table	Zero-copy interchange between systems	You need rich interactive indexing
SQL via DuckDB	Data already in warehouse tables	Tiny in-memory transforms
NumPy structured arrays	Fixed schema numeric records	Heterogeneous column types

FAQs

When should I reset_index?

After groupby when you need the group key as a column.
When duplicate index labels break scalar .loc access.

What is the difference between [] and .loc?

df["col"] selects one column Series.
df.loc[rows, cols] is label-based 2-D selection with alignment rules.

How do I avoid copies on filter?

Filtering always returns a new frame unless using inplace=True (discouraged).
For huge data, use Polars lazy filter or chunked reads.

How do I read only some columns from CSV?

import pandas as pd
df = pd.read_csv("big.csv", usecols=["region", "revenue"])

Can a Series have a name?

Yes - s.name appears when the Series becomes a DataFrame column.
Set on creation: pd.Series(..., name="revenue").

How do I detect duplicate column names?

import pandas as pd
df.columns.duplicated().any()

Duplicate names make df["x"] return multiple columns.

What does copy-on-write do in pandas 2.x?

With pd.options.mode.copy_on_write = True, chained ops defer copies until write.
Still prefer explicit .loc assignment for clarity.

How do I convert index to column?

df = df.reset_index()

Or df.rename_axis("order_id").reset_index() to name the former index.

Why is my numeric column object dtype?

Mixed strings and numbers force object.
Re-read with dtype or pd.to_numeric(errors="coerce").

How do I sample rows reproducibly?

sample = df.sample(n=100, random_state=42)

Cleaning & Transforming Data - dtypes and missing values
GroupBy & Aggregation - split-apply-combine
Joins & Merges - combining frames
Data Analysis Basics - intro tour

Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.

pandas Series & DataFrames

pandas adds labeled axes to columnar data: a Series is one typed column; a DataFrame is a table of aligned Series sharing an index.

Recipe

Quick-reference recipe card - copy-paste ready.

import pandas as pd
 
df = pd.DataFrame(
    {"region": ["East", "West"], "revenue": [120, 340]},
    index=pd.Index(["a", "b"], name="row"),
)
east = df.loc[df["region"] == "East", "revenue"]
subset = df.iloc[0:1, [0, 1]]

When to reach for this:

Tabular EDA with mixed column types and missing values
Labeled time series indexed by timestamps
Joining multiple tables on keys before aggregation
Exporting analysis results to CSV, Parquet, or Excel

Working Example

import pandas as pd
 
orders = pd.DataFrame(
    {
        "order_id": [101, 102, 103, 104],
        "region": pd.Categorical(["East", "West", "East", "West"]),
        "revenue": [120.0, 340.0, None, 210.0],
        "ordered_at": pd.to_datetime(
            ["2025-01-01", "2025-01-02", "2025-01-03", "2025-01-04"], utc=True
        ),
    }
).set_index("order_id")
 
# Label-based selection
east = orders.loc[orders["region"] == "East", ["region", "revenue"]]
 
# Position-based head
head = orders.iloc[:2]
 
# Add computed column safely
orders = orders.copy()
orders["revenue_filled"] = orders["revenue"].fillna(orders["revenue"].median())
 
# Series reduction
monthly_total = orders["revenue_filled"].sum()
 
print(east)
print("monthly_total:", monthly_total)

What this demonstrates:

Construction with categorical and nullable float columns
UTC datetime parsing at load time
.loc boolean filtering with column subset
.copy() before assignment to avoid SettingWithCopyWarning

Deep Dive

How It Works

Index labels align rows across columns - operations match on index automatically.
Block manager (pandas 2.x) stores columns in Arrow/numpy-backed blocks by dtype.
.loc includes both endpoints for labels; .iloc is half-open like Python slices.
dtype backend (string[pyarrow]) changes memory and missing-value behavior.

Selection Cheat Sheet

Goal	API
Filter rows	`df.loc[mask]`
Pick columns	`df[["a", "b"]]`
Scalar by label	`df.loc[row, "col"]`
Scalar by position	`df.iloc[i, j]`

Parameters & Return Values

Parameter	Type	Description
`index`	Index-like	Row labels; defaults to RangeIndex
`columns`	list[str]	Column names for DataFrame
`dtype`	str/np.dtype	Force column type on construction

Gotchas

Chained indexing assignment - df[df["x"] > 0]["y"] = 1 triggers SettingWithCopyWarning. Fix: df.loc[df["x"] > 0, "y"] = 1.
Duplicate index labels - .loc["a"] returns multiple rows; scalar access fails. Fix: reset_index() or enforce unique keys upstream.
Object dtype strings - default object columns use lots of RAM. Fix: df["col"] = df["col"].astype("string[pyarrow]") in pandas 2.2+.
Silent dtype coercion - mixing ints and floats upgrades column to float. Fix: use nullable Int64 when NA is possible.
Timezone-naive vs aware - comparing mixed datetimes errors in pandas 2.x. Fix: pd.to_datetime(..., utc=True) everywhere at ingest.

Alternatives

Alternative	Use When	Don't Use When
Polars DataFrame	Lazy scans, very large data, strict typing	Team already standardized on pandas idioms
PyArrow Table	Zero-copy interchange between systems	You need rich interactive indexing
SQL via DuckDB	Data already in warehouse tables	Tiny in-memory transforms
NumPy structured arrays	Fixed schema numeric records	Heterogeneous column types

FAQs

When should I reset_index?

After groupby when you need the group key as a column.
When duplicate index labels break scalar .loc access.

What is the difference between [] and .loc?

df["col"] selects one column Series.
df.loc[rows, cols] is label-based 2-D selection with alignment rules.

How do I avoid copies on filter?

Filtering always returns a new frame unless using inplace=True (discouraged).
For huge data, use Polars lazy filter or chunked reads.

How do I read only some columns from CSV?

import pandas as pd
df = pd.read_csv("big.csv", usecols=["region", "revenue"])

Can a Series have a name?

Yes - s.name appears when the Series becomes a DataFrame column.
Set on creation: pd.Series(..., name="revenue").

How do I detect duplicate column names?

import pandas as pd
df.columns.duplicated().any()

Duplicate names make df["x"] return multiple columns.

What does copy-on-write do in pandas 2.x?

With pd.options.mode.copy_on_write = True, chained ops defer copies until write.
Still prefer explicit .loc assignment for clarity.

How do I convert index to column?

df = df.reset_index()

Or df.rename_axis("order_id").reset_index() to name the former index.

Why is my numeric column object dtype?

Mixed strings and numbers force object.
Re-read with dtype or pd.to_numeric(errors="coerce").

How do I sample rows reproducibly?

sample = df.sample(n=100, random_state=42)

Cleaning & Transforming Data - dtypes and missing values
GroupBy & Aggregation - split-apply-combine
Joins & Merges - combining frames
Data Analysis Basics - intro tour

Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.

pandas Series & DataFrames

Search Documentation

pandas Series & DataFrames