Skip to content

    Chapter 30, Advanced

    Method chaining

    First, the way we learn: one step at a time, with a new variable (or a change) on each line. Then, the same in

    The same calculation, in two ways

    a chain The results are:

    avancado/cap30_encadeamento.pylinhas 10 a 46
    import pandas as pd

pd.set_option("display.width", 170)
pd.set_option("display.max_columns", 20)

pedidos = pd.read_csv("dados/pedidos.csv", parse_dates=["data_pedido"])
produtos = pd.read_csv("dados/produtos.csv")

# passo a passo
v = pedidos.merge(produtos, on="id_produto", how="left", validate="m:1")
v["receita"] = v["quantidade"] * v["preco_unitario"] * (1 - v["desconto"].fillna(0))
v["mes"] = v["data_pedido"].dt.to_period("M").astype(str)
v = v[v["quantidade"] >= 2]
passo_a_passo = (
    v.groupby(["categoria", "mes"], as_index=False)
    .agg(receita=("receita", "sum"), pedidos=("id_pedido", "count"))
    .sort_values(["categoria", "mes"])
    .reset_index(drop=True)
)

# em cadeia
encadeado = (
    pedidos
    .merge(produtos, on="id_produto", how="left", validate="m:1")
    .assign(
        receita=lambda d: d["quantidade"] * d["preco_unitario"] * (1 - d["desconto"].fillna(0)),
        mes=lambda d: d["data_pedido"].dt.to_period("M").astype(str),
    )
    .query("quantidade >= 2")
    .groupby(["categoria", "mes"], as_index=False)
    .agg(receita=("receita", "sum"), pedidos=("id_pedido", "count"))
    .sort_values(["categoria", "mes"])
    .reset_index(drop=True)
)
pd.testing.assert_frame_equal(passo_a_passo, encadeado)
print(encadeado.shape, "as duas versões são idênticas")
print(encadeado.head(3).round(1))
    Output
    (48, 4) as duas versões são idênticas
  categoria      mes  receita  pedidos
0      Casa  2025-01  19382.5       14
1      Casa  2025-02  20281.5       14
2      Casa  2025-03  13254.6       11

    identical . The chain wins in three ways: it is read from top to bottom like a recipe, it does not leave loose names (was reassigned four times in the first version), andv does not alter any existing table. And this directly relates to the copy-on-write from chapter 27. creates columns and returns a

    The pieces that make the chain work

    The assign new table . The. A lambda d: receives the table from that point in the chain (and not a variable from outside), and it allows using columns that have just been created. The query filters with an expression. And the .loc[lambda d: ...] filters with a function, when the condition doesn't fit in text:

    avancado/cap30_encadeamento.pylinhas 51 a 57
    com_total = (
    pedidos
    .assign(total=lambda d: d["quantidade"] * 10)
    .assign(grande=lambda d: d["total"] > 50)
    .loc[lambda d: d["grande"], ["id_pedido", "total"]]
)
print(com_total.shape, int(com_total["total"].min()))
    Output
    (401, 2) 60

    The second assign uses the total that the first just created. This only works with the lambda d:: write pedidos["total"] there would fail, because the pedidos original does not have that column.

    Look inside the chain: `pipe`

    A long chain has a flaw: when something goes wrong, where? The pipe fits any function in the chain, and a function that only logs and returns the table is the clean way to peek in the middle:

    avancado/cap30_encadeamento.pylinhas 62 a 75
    def registrar(d, etapa):
    print(f"{etapa:<12} {d.shape[0]:>4} linhas, {d.shape[1]:>2} colunas")
    return d


(
    pedidos
    .pipe(registrar, "lidos")
    .merge(produtos, on="id_produto", how="left", validate="m:1")
    .pipe(registrar, "com produto")
    .query("quantidade >= 5")
    .pipe(registrar, "filtrados")
    .groupby("categoria").size()
)
    Output
    lidos         800 linhas,  7 colunas
com produto   800 linhas, 10 colunas
filtrados     494 linhas, 10 colunas

    Small and named functions

    A chain of ten lines is still hard to read. The best design combines small named functions, each receiving and returning a table, and the pipe composes them. The analysis becomes a list of named steps:

    avancado/cap30_encadeamento.pylinhas 80 a 97
    def enriquecer(d: pd.DataFrame) -> pd.DataFrame:
    return (
        d.merge(produtos, on="id_produto", how="left", validate="m:1")
        .assign(receita=lambda t: t["quantidade"] * t["preco_unitario"] * (1 - t["desconto"].fillna(0)))
    )


def so_grandes(d: pd.DataFrame, minimo: int = 2) -> pd.DataFrame:
    return d.query("quantidade >= @minimo")


def resumir_por_categoria(d: pd.DataFrame) -> pd.DataFrame:
    return d.groupby("categoria", as_index=False).agg(receita=("receita", "sum")).sort_values("categoria")


resumo = pedidos.pipe(enriquecer).pipe(so_grandes, minimo=2).pipe(resumir_por_categoria)
print(resumo.round(0).to_dict("records")[0])
print(bool(resumo["receita"].sum() <= enriquecer(pedidos)["receita"].sum()))
    Output
    {'categoria': 'Casa', 'receita': 233575.0}
True

    Each function can be tested alone (chapter 33), and the final line tells the story of the analysis.

    What breaks a chain

    The methods that alter in place and return None (inplace=True, df.sort_values(inplace=True)) cannot be in a chain: they return None, and the next method fails. It's another reason to avoid the inplace, and what the ruff points out with the rule PD002.

    Exercise 1

    Rewrite in chain

    Rewrite in The results are this code: d = pedidos[pedidos["canal"] == "site"], then d["total"] = d["quantidade"] * 2, then d.sort_values("total", ascending=False).head(3), returning only the columns id_pedido and total. Check that the result is the same as the step-by-step (done with copy).