Chapter 37, Advanced
Typing, API and NumPy 2
How to declare what a function accepts and returns, how to validate shapes at runtime, and what changed in version 2 of NumPy, because much of the material you find still uses version 1.
Code for this chapter: avancado/cap37_tipagem_numpy2.py
Annotating functions that use arrays
The module numpy.typing offers two names that solve almost everything. The ArrayLike is "anything that NumPy converts to an array" (lists, tuples, numbers, arrays), and it is the right type for inputs. The NDArray[...] is an array of a specific type, and it is the right type for outputs. The standard is to accept generously and return precisely, converting on input with np.asarray:
import numpy as np
import numpy.typing as npt
def normalizar(x: npt.ArrayLike) -> npt.NDArray[np.float64]:
a = np.asarray(x, dtype=np.float64)
return (a - a.mean()) / a.std()
print(normalizar([1, 2, 3]).round(3))
print(normalizar(np.array([10, 20, 30])).round(3))
[-1.225 0. 1.225]
[-1.225 0. 1.225]
The np.asarray does not copy what is already an array of the right type (chapter 10), and converts lists, so the function accepts both forms at no extra cost. A type checker like the mypy checks the annotations, but cannot check the shape (how many dimensions, how many rows): the NumPy type system knows the type of the elements, not the dimensions.
Validating shape at runtime
Since the shape is not checked statically, a function that depends on it must validate on input, with a clear message. It is better to fail early, with the explanation, than later, with an incomprehensible broadcasting error. A None in the expected shape means "any size on that axis":
def exigir_forma(a, esperada):
if a.ndim != len(esperada) or any(e is not None and e != s for e, s in zip(esperada, a.shape)):
raise ValueError(f"forma {a.shape} incompatível com {esperada}")
return a
try:
exigir_forma(np.zeros((3, 4)), (None, 5))
except ValueError as erro:
print(erro)
forma (3, 4) incompatível com (None, 5)
I also document the expected shape in the docstring, in the style X : (n_amostras, n_características), which is the scikit-learn convention.
What changed in NumPy 2
Version 2.0 was the first incompatible change in many years. Many old names were removed to make the API leaner, and other behaviors changed. I checked, in version 2.4 used in this course, which common names no longer exist:
removidos = ["float_", "NaN", "Inf", "in1d", "trapz", "round_", "msort"]
print({nome: hasattr(np, nome) for nome in removidos})
print(hasattr(np, "trapezoid"), hasattr(np, "isin"))
{'float_': False, 'NaN': False, 'Inf': False, 'in1d': False, 'trapz': False, 'round_': False, 'msort': False}
True True
| Old name | Use now |
|---|---|
np.float_, np.complex_ | np.float64, np.complex128 |
np.NaN, np.Inf, np.PINF | np.nan, np.inf |
np.in1d | np.isin |
np.trapz | np.trapezoid |
np.round_ | np.round |
np.msort(a) | np.sort(a, axis=0) |
Other changes that appear in practice:
- The representation of scalars. The
reprof a scalar now shows the type, and theprintremains the same. In a result like(np.int64(1), np.int64(0))the numbers are the same as always. - Type promotion (NEP 50). A pure Python number does not promote the type of the array (chapter 7).
- The default integer is
int64also on Windows. In the 1.x series, Windows usedint32, and the same code gave different results across systems. copy=Falsechanged meaning. Before: "avoid copying, if you can". Now: "never copy, and fail if necessary". For the old behavior, usecopy=None(the default) ornp.asarray.
print(repr(np.float64(3.0)), str(np.float64(3.0)))
lista = [1, 2, 3]
try:
np.array(lista, copy=False)
except ValueError as erro:
print("copy=False falha quando é preciso copiar:", type(erro).__name__)
x = np.array([1.0, 2.0])
print(np.asarray(x, copy=False) is x, np.array(x, copy=None) is x)
np.float64(3.0) 3.0
copy=False falha quando é preciso copiar: ValueError
True True
Version 2 also brought new functions aligned with the community array standard, such as the np.unstack (the inverse of the stack), the np.vecdot and np.linalg.matrix_transpose:
print(np.unstack(np.arange(6).reshape(2, 3)))
(array([0, 1, 2]), array([3, 4, 5]))
Find the old code automatically
You don't need to hunt for the old names manually. The ruff has a migration rule, the NPY201, which marks each use of a removed name and, in several cases, suggests (or applies with --fix) the replacement. This file mixes names from series 1:
import numpy as np
x = np.float_(3)
y = np.NaN
z = np.in1d([1], [1])
w = np.trapz([1, 2])
cd exemplos
ruff check --select NPY201 velho_numpy1.py --output-format concise
velho_numpy1.py:3:5: NPY201 [*] `np.float_` will be removed in NumPy 2.0. Use `numpy.float64` instead.
velho_numpy1.py:4:5: NPY201 [*] `np.NaN` will be removed in NumPy 2.0. Use `numpy.nan` instead.
velho_numpy1.py:5:5: NPY201 `np.in1d` will be removed in NumPy 2.0. Use `np.isin` instead. Unlike `np.in1d`, `np.isin` preserves the shape of its input, so `np.in1d(ar1, ar2)` is equivalent to `np.isin(ar1, ar2).ravel()`.
velho_numpy1.py:6:5: NPY201 `np.trapz` will be removed in NumPy 2.0. Use `numpy.trapezoid` on NumPy 2.0, or ignore this warning on earlier versions.
Found 4 errors.
[*] 2 fixable with the `--fix` option (1 hidden fix can be enabled with the `--unsafe-fixes` option).
Dependencies that have not migrated yet
If a library you use was compiled against series 1.x, it may break with NumPy 2. When updating a project, check if your dependencies declare support for NumPy 2, and fix
numpy>=2in thepyproject.tomlonly after checking. Theuvandpipwarn of version conflicts at resolution time.
Exercise 1
Validate an input array
Write validar_matriz(a) that converts the input to float64, reject (with ValueError) anything that is not 2D or has nan or infinities, and return the validated array.