Aggregations and the axis parameter

The most used aggregations

import numpy as np

dados = np.array([[4, 9, 2], [11, 6, 15]])
print(dados.sum(), dados.mean(), np.median(dados))
print(dados.sum(axis=0), dados.sum(axis=1))
print(dados.max(axis=1), dados.argmax(), dados.argmax(axis=1))

47 7.833333333333333 7.5
[15 15 17] [15 32]
[ 9 15] 5 [1 2]

Without axis, aggregation flattens the array and summarizes everything into a single number. With axis, NumPy summarizes along that axis, and the dimension disappears from the result.

What the axis does

I use this mental model: axis=k is the axis that disappears from the result. In axis=0, the row axis disappears and the columns remain.

The argmax and argmin return the position of the largest and smallest value (without axis, the position in the flattened array), instead of the value.

Dispersion and position

The standard deviation and variance have a subtlety: by default, NumPy divides by N (the variance of the population). To treat the data as a sample, divide by N - 1, with ddof=1:

print(dados.std().round(3), dados.var().round(3), dados.std(ddof=1).round(3))
print(np.percentile(dados, [25, 50, 75]))

4.375 19.139 4.792
[ 4.5  7.5 10.5]

The np.percentile returns the values below which the requested percentage lies: the 50% is the median.

Median resists extreme values, mean does not

A single extreme value drags the mean away from what is "typical", and the median barely moves. Therefore, in salary data, real estate prices, and response times, the median is often the honest measure:

salarios = np.array([3000, 3200, 3100, 2900, 50000])
print(salarios.mean(), np.median(salarios))

12440.0 3100.0

Count distinct values

The np.unique with return_counts=True returns the different values and how many times each appears, without a loop:

votos = np.array([2, 1, 2, 3, 2, 1])
valores, contagens = np.unique(votos, return_counts=True)
print(valores, contagens)

[1 2 3] [2 3 1]

Aggregate with `keepdims`

Keeping the dimension reduced, the result still "fits" into the original array through broadcasting. This is what the previous chapter used to normalize rows:

print(dados.sum(axis=1, keepdims=True))
print(dados / dados.sum(axis=1, keepdims=True))

[[15]
 [32]]
[[0.26666667 0.6        0.13333333]
 [0.34375    0.1875     0.46875   ]]

Aggregations on booleans

As True is 1 and False is worth 0, sum counts, mean gives the proportion, any asks if any is true and all asks if all are. They combined with the masks from chapter 9 solve most questions about data.