Introduction

Pandas Course

From dirty CSV to answers, in three levels, with included data, code verified by execution and three complete projects.

Who is this material for

Pandas is the tool that transforms a messy spreadsheet into an answer. Almost all data analysis in Python starts with it: loading a CSV, figuring out what is wrong, cleaning, asking, and summarizing. This course continues the NumPy Course: here the array gets labels, columns of different types, and the ability to handle missing data.

I organized the notes into six parts. All use the same datasets, which accompany the course, and each idea appears first as intuition and then as code that you execute.

Part	For whom	What you will be able to do by the end
Environment	Who just came from NumPy	Understand what a `DataFrame` is internally and install everything
Basic	Who has never analyzed data in Python	Load, inspect, select, filter, clean, create columns, sort, and group
Intermediate	Who already does simple analyses	Join tables, reformat, work with dates, texts, and categories
Advanced	Who writes analyses that need to be fast and correct	Performance, memory, method chaining, large files, and data testing
Projects	Who wants to prove they learned	Deliver three projects with tests, one per level

If you have never seen NumPy, at least do chapters 5 to 14 of the NumPy Course. Here I assume you know what an array is, what vectorization is, and what a dtype.

The original course, and what I did with it

This course is based on a course that covers seven topics (from "why Pandas" to grouping) and that intentionally states that table joining, reformatting, and time series will be left for another day. I kept the seven topics, in the same order and with the same dataset (translated into Portuguese, with values in reais), and added what was missing for real work, at the Intermediate and Advanced levels. Where the code from the original course breaks or teaches a habit that Pandas 3 discourages, I show the problem with a real execution.

How each chapter works

The output is real. I executed each snippet and copied what Python printed. This applies to the Pandas 3.0, the version that the course assumes. In 2.x versions, some outputs change (mainly the type str to text).
Each block indicates where it is. At the top of each snippet appears the file and the lines from the repository (for example, basico/cap13_agrupar_e_agregar.py), and you can execute it on your own.
The data accompanies the course. The chapters read CSV files from the folder dados/, which are in the repository. Run the files from the root of the repository.
The exercises have solutions. Try first. The solution is hidden and ends with assert, so you find out for yourself if you got it right.

The datasets

File	Rows	What it is for
`funcionarios_15.csv`	15	The course set, translated, with problems hidden on purpose. Used at the Basic level
`funcionarios_100.csv`	104	A larger and messier version (repeated ids, dates in two formats). Used in the following levels
`clientes.csv`, `produtos.csv`, `pedidos.csv`	60, 20 and 800	A sales model in three tables, for joining, reformatting, and time series

The repository

Estrutura do repositório

pandas-notes/
  README.md
  pyproject.toml
  docs/
    pandas-notes.html
  dados/
  ambiente/
  basico/
  intermediario/
  avancado/
  projetos/

Each level folder has a README.md (objective, environment, data dictionary) and a recording script. To start, from the root of the repository:

Terminal

uv venv
uv sync
uv run basico/cap04_series.py

Versions that the course assumes

Python 3.11 or higher (Pandas 3 requires this) and Pandas 3.0. The code also uses the pyarrow, which Pandas 3 leverages to store texts and to read and write the Parquet format:

Terminal

uv add pandas pyarrow openpyxl

A rule I repeat in this course

Think in columns, not in rows. When you write a for about the rows of a DataFrame, stop and ask if there is an operation on the entire column. Almost always there is, and it is shorter, faster, and less prone to error. The same instinct from NumPy applies here, with an addition: be suspicious of the data before trusting the calculations. A mean() that runs without error on a column that Pandas read as text is not an analysis, it is a silent error.