Measure real performance

`timeit` and the best of several

A time measured once is noise: the processor may be busy, the cache may be cold. The module timeit repeats the measurement, and the rule is to use the smallest of the times, which is the least affected by what your computer was doing at the same time:

import timeit

import numpy as np

a = np.arange(1_000_000)
tempos = timeit.repeat(lambda: a.sum(), number=20, repeat=5)
print("melhor tempo por chamada abaixo de 10 ms:", min(tempos) / 20 < 0.01)

melhor tempo por chamada abaixo de 10 ms: True

In a notebook, %timeit a.sum() does all this for you. In a script, use timeit.repeat and look at the min.

How much it costs to start: small sizes don't tell the story

Calling a NumPy function has a fixed cost (checking types, allocating the output array). For a few elements, this cost dominates, and the array may even lose to a list. The advantage only appears, and grows, when the data grows:

def razao(n):
    lista = list(range(n))
    array = np.arange(n)
    t_lista = min(timeit.repeat(lambda: [x * 2 for x in lista], number=50, repeat=5))
    t_array = min(timeit.repeat(lambda: array * 2, number=50, repeat=5))
    return t_lista / t_array


print("a vantagem cresce com o tamanho:", razao(100_000) > razao(100))
print("em 100 mil elementos o array ganha:", razao(100_000) > 5)

a vantagem cresce com o tamanho: True
em 100 mil elementos o array ganha: True

That’s why measuring with 10 elements and concluding that "NumPy is not faster" is a classic mistake. The test must be done at the real size of your data.

Measure memory

Time is just one half. Many analyses fail due to lack of memory, and not due to slowness. The tracemalloc measures the memory allocated by Python, and NumPy reports its arrays to it:

import tracemalloc

tracemalloc.start()
x = np.zeros(1_000_000)
atual, pico = tracemalloc.get_traced_memory()
tracemalloc.stop()
print(round(atual / 1e6, 1), "MB")

8.0 MB

A million of float64 occupies 8 MB, exactly size * itemsize. For an array that already exists, the nbytes gives the answer without measuring anything.

Measurement traps

Measure before changing

I only optimize after knowing where the time goes. For an entire program, the cProfile (chapter 48 of the Python course) shows which function consumes the time. For a snippet, the timeit compares alternatives. And, before accepting a faster version, I check that it produces the same result.