Memory from the inside: strides, contiguity, and order

The array is a block plus a list of `strides`

The data resides in a continuous block of memory. What shapes it are the strides: how many bytes to skip to advance one position in each axis. A 3 by 4 array of int64 (8 bytes per element) in row-major order ( C, the default) advances 8 bytes to the next column and 32 (4 columns × 8) to the next row. The transpose does not move a byte: it only swaps the strides:

import numpy as np

a = np.arange(12, dtype=np.int64).reshape(3, 4)
print(a.strides, a.flags["C_CONTIGUOUS"], a.flags["F_CONTIGUOUS"])

t = a.T
print(t.strides, t.flags["C_CONTIGUOUS"], t.flags["F_CONTIGUOUS"], np.shares_memory(a, t))

(32, 8) True False
(8, 32) False True True

The transpose is contiguous in Fortran order (F, the column-major order used in Fortran and MATLAB): the elements of the same column are side by side. The two arrays share memory, and that is why a.T costs almost nothing, even in a gigabyte array.

Slicing only changes the strides

A slice with a step is a new read of the same block, with different strides. No data moves:

print(a[:, ::2].strides, a[::2].strides, a[:, 0].strides)

(32, 16) (64, 8) (32,)

Taking every other row doubles the stride of the row axis (from 32 to 64), and taking every other column doubles the column stride (from 8 to 16). And the isolated column 0 has a stride of 32 bytes: to traverse it, NumPy jumps through memory, and that jump has a cost.

Why contiguity matters

The processor reads memory in cache lines (blocks of 64 bytes). Traversing neighboring elements takes advantage of each entire cache line. Traversing with large jumps wastes: for each element, a new cache line. Compare summing the same amount of elements, contiguous and spaced:

import timeit

x = np.ones(16_000_000, dtype=np.float32)
contiguo = x[:1_000_000]
espacado = x[::16]
print(contiguo.size == espacado.size)
t_contiguo = min(timeit.repeat(lambda: contiguo.sum(), number=5, repeat=5))
t_espacado = min(timeit.repeat(lambda: espacado.sum(), number=5, repeat=5))
print("o contíguo foi mais rápido:", t_contiguo < t_espacado)

True
o contíguo foi mais rápido: True

They are the same 1 million elements, but the spaced one needs to traverse 16 times more memory. When an operation is unexpectedly slow, strides are the first suspect, and np.ascontiguousarray(x) makes a contiguous copy when it is worth it.

Reinterpret bytes with `view`

The view with another dtype does not convert the values: it reads the same bytes in a different way. It is useful for inspecting the internal representation of numbers and for processing binary data:

inteiros = np.array([1, 256, 65536], dtype=np.int32)
print(inteiros.view(np.uint8))

bits = np.array([1.0], dtype=np.float32).view(np.uint32)[0]
print(hex(int(bits)))

[1 0 0 0 0 1 0 0 0 0 1 0]
0x3f800000

The result of the first view shows that NumPy stores integers in order little-endian on this machine (the least significant byte first). The 0x3f800000 is the binary representation of the number 1.0 in float32, according to the IEEE 754 standard.

`as_strided`: power with danger

The function as_strided builds any view you describe with shape and strides. With it, you manually assemble sliding windows. It is the basis of what the sliding_window_view in chapter 28 does safely:

from numpy.lib.stride_tricks import as_strided

y = np.arange(6)
janelas = as_strided(y, shape=(4, 3), strides=(y.strides[0], y.strides[0]))
print(janelas)

[[0 1 2]
 [1 2 3]
 [2 3 4]
 [3 4 5]]

Do not use as_strided unnecessarily

The as_strided it does not check anything. A shape or a stride that exceeds the memory block reads (or writes) bytes that do not belong to the array, without error and without warning. This corrupts data or crashes the program unpredictably. Always prefer the sliding_window_view, which calculates everything for you, and reserve the as_strided for those who have a strong reason and test very carefully.