Why Pandas exists

The problem of parallel lists

You already know how to store data in Python lists. Here are three employees the "list way": one list for each attribute, and the hope that they never go out of sync.

nomes = ["Ana", "Bruno", "Carla"]
idades = [24, 28, 26]
salarios = [4500, 6500, 5000]

print(sum(salarios) / len(salarios))
print([n for n, s in zip(nomes, salarios) if s > 4800])

5333.333333333333
['Bruno', 'Carla']

It works for three people. Now imagine 500 employees, with missing salaries, repeated entries, and mixed dates. The first obstacle is the missing data, which lists cannot represent:

salarios_com_falta = [4500, None, 5000]
try:
    sum(salarios_com_falta)
except TypeError as erro:
    print("TypeError:", erro)

TypeError: unsupported operand type(s) for +: 'int' and 'NoneType'

Each filter requires a manually written loop, grouping "by department" requires its own bucket logic, and none of this scales beyond a toy example. Pandas offers fast and flexible data structures for real-world structured data, with the same relationship that a spreadsheet has with a sheet of paper with numbers written on it.

The same data in Pandas

import pandas as pd

df = pd.DataFrame({"nome": nomes, "idade": idades, "salario": salarios_com_falta})
print(df)
print(df["salario"].mean())
print(df[df["salario"] > 4800])

    nome  idade  salario
0    Ana     24   4500.0
1  Bruno     28      NaN
2  Carla     26   5000.0
4750.0
    nome  idade  salario
2  Carla     26   5000.0

Three things happened effortlessly: the missing data became NaN and was ignored in the average (4750 is the average of 4500 and 5000), the columns became aligned (one row is a complete employee), and the filter became an expression, not a loop.

From NumPy to Pandas

Pandas is built on top of NumPy. What it adds is, in essence, what a spreadsheet has and an array does not:

The anatomy: `Series` and `DataFrame`

A Series is a labeled column. A DataFrame is a table made of columns that share the same index. The pieces have names:

print(df.index)
print(df.columns.tolist())
print(df.loc[1])
print(df["salario"])
print(df.at[0, "nome"])

RangeIndex(start=0, stop=3, step=1)
['nome', 'idade', 'salario']
nome       Bruno
idade         28
salario      NaN
Name: 1, dtype: object
0    4500.0
1       NaN
2    5000.0
Name: salario, dtype: float64
Ana

And NumPy is still there underneath. Any numeric column delivers its array:

valores = df["salario"].to_numpy()
print(type(valores).__name__, valores.dtype)

ndarray float64

Notice the type float64: the salary column was of integers, but the missing data (NaN) only exists as decimal, and Pandas promoted the entire column. Chapter 3 explains this rule.

The mistake of those coming from NumPy

To treat a DataFrame as "a NumPy array with extra steps". Old habits get in the way: trusting in the position instead of the label, and forgetting that a column can hold text alongside a column of numbers. Pandas aligns by label, and Chapter 4 shows what this changes.

When Pandas is not the answer

Pandas loads the data into memory, integers. For datasets that exceed memory size, the outputs are to process in parts (Chapter 32) or use tools designed for that, like DuckDB or Polars, which this course does not cover. For purely numerical calculations on a uniform grid, NumPy remains simpler and faster.