Broadcasting

The idea: stretch, without copying

Think of the smaller array being stretched to match the larger one. NumPy does not create copies of the data: it reuses memory as if it were repeated. A row added to a grid applies to all rows, and a column applies to all columns:

import numpy as np

grade = np.ones((3, 4))
linha = np.array([1, 2, 3, 4])
print(grade + linha)

coluna = np.array([[10], [20], [30]])
print(grade + coluna)
print((np.arange(3).reshape(3, 1) + np.arange(4)).shape)

[[2. 3. 4. 5.]
 [2. 3. 4. 5.]
 [2. 3. 4. 5.]]
[[11. 11. 11. 11.]
 [21. 21. 21. 21.]
 [31. 31. 31. 31.]]
(3, 4)

The last example is the most powerful: a column (3, 1) added to a row (4,) produces a grid (3, 4), with all combinations.

The rule

Compare the shapes from right to left, one dimension at a time. Two dimensions are compatible if they are equal or if one of them is 1. If all dimensions pass this test, the shapes combine. Missing dimensions on the left are treated as 1 by NumPy. You can check the rule by code:

def compativel(a, b):
    try:
        return np.broadcast_shapes(a, b)
    except ValueError:
        return None


print(compativel((3, 4), (4,)))
print(compativel((3, 1), (1, 4)))
print(compativel((5, 1, 3), (3,)))
print(compativel((3, 4), (3,)))

(3, 4)
(3, 4)
(5, 1, 3)
None

The last case fails: (3, 4) and (3,) align the 4 with the 3 (from right to left), which are not equal and neither is 1. The actual error has this message:

try:
    np.ones((3, 4)) + np.ones(3)
except ValueError as erro:
    print(erro)

operands could not be broadcast together with shapes (3,4) (3,) 

Real uses

Centering each column (subtracting the mean of each) and normalizing each row (dividing by the sum of the row) are the two most common operations, and both depend on broadcasting. The keepdims=True keeps the reduced axis with size 1, which is exactly what broadcasting needs:

notas = np.array([[7.0, 8.0, 9.0], [4.0, 6.0, 8.0]])
print(notas - notas.mean(axis=0))
print(notas / notas.sum(axis=1, keepdims=True))

[[ 1.5  1.   0.5]
 [-1.5 -1.  -0.5]]
[[0.29166667 0.33333333 0.375     ]
 [0.22222222 0.33333333 0.44444444]]

The subtraction uses a mean per column (shape (3,)). The division needs a sum per row in column shape, (2, 1), and that’s why the keepdims=True is necessary.

Another use is to calculate all differences between pairs at once, without a double loop:

indices = np.arange(4)
print(indices[:, None] - indices)

[[ 0 -1 -2 -3]
 [ 1  0 -1 -2]
 [ 2  1  0 -1]
 [ 3  2  1  0]]

Memory is not copied

Broadcasting is a virtualstretching. You can see this in the strides: a step of 0 bytes means "repeat the same data". And the result is read-only, because writing to elements that are actually the same would be ambiguous:

esticado = np.broadcast_to(np.array([1, 2, 3]), (1000, 3))
print(esticado.shape, esticado.strides, esticado.flags["WRITEABLE"])

(1000, 3) (0, 8) False

A thousand rows "exist", but take up the space of one. That’s why broadcasting is fast and economical. The caution comes from the other side: the result of the calculation (the grid (3, 4) of the example) is a real array, and at large scales it can blow up memory (chapter 31).

It’s not "any smaller array will do"

The typical error is thinking that if an array is smaller, it always combines. What matters is each dimension. Test the shape with np.broadcast_shapes before running a large calculation, or read the error message: it shows the two shapes side by side.