Chapter 42, Advanced
The data model and protocols
Almost everything that seems magical in Python is the interpreter calling special methods on its objects. Those who master these protocols write classes that fit into the language.
Code for this chapter: avancado/cap42_modelo_dados.py
A sequence with two methods
If an object has __len__ and __getitem__, it already behaves like a sequence: it accepts index, slice, in, iteration, reversed and random.choice. No inheritance is necessary. It's the duck typing taken seriously:
import random
from collections import namedtuple
Carta = namedtuple("Carta", ["valor", "naipe"])
class Baralho:
valores = [str(n) for n in range(2, 11)] + list("JQKA")
naipes = ["paus", "ouros", "copas", "espadas"]
def __init__(self):
self._cartas = [Carta(v, n) for n in self.naipes for v in self.valores]
def __len__(self):
return len(self._cartas)
def __getitem__(self, posicao):
return self._cartas[posicao]
baralho = Baralho()
print(len(baralho), baralho[0], baralho[-1])
print(baralho[:3])
print(Carta("Q", "copas") in baralho)
print(random.choice(baralho) in baralho)
52 Carta(valor='2', naipe='paus') Carta(valor='A', naipe='espadas')
[Carta(valor='2', naipe='paus'), Carta(valor='3', naipe='paus'), Carta(valor='4', naipe='paus')]
True
True
The in works because, without __contains__ or __iter__, Python falls into the old iteration protocol: it calls __getitem__ with 0, 1, 2... until it receives IndexError.
To actually be a Sequence
The fact that the object behaves like a sequence does not make it a Sequence for type checking. Abstract classes of collections.abc test for the presence of specific methods, and this check is more demanding than actual iteration:
from collections.abc import Container, Iterable, Sequence, Sized
print(isinstance(baralho, Sized), isinstance(baralho, Iterable), isinstance(baralho, Container))
True False False
It is Sized (has __len__), but it is not Iterable (lacks __iter__), although the for works. The practical lesson: to know if something is iterable, try iter(obj). And, if you want the full contract, inherit from Sequence: you only implement __len__ and __getitem__, and get __contains__, __iter__, __reversed__, index and count:
class BaralhoSeq(Sequence):
def __init__(self):
self._cartas = Baralho()._cartas
def __len__(self):
return len(self._cartas)
def __getitem__(self, posicao):
return self._cartas[posicao]
b = BaralhoSeq()
print(isinstance(b, Sequence), Carta("A", "copas") in b)
print(b.index(Carta("3", "paus")), b.count(Carta("2", "paus")))
True True
1 1
The contract between __eq__ and __hash__
for free. Two equal objects must have the same hash, and the hash of an object cannot change while it is in a set or is a dictionary key. Breaking the second rule produces one of the most confusing bugs in the language: the object is in the set, but the search does not find it:
class Chave:
def __init__(self, valor):
self.valor = valor
def __eq__(self, outro):
return self.valor == outro.valor
try:
{Chave(1)}
except TypeError as erro:
print(erro)
class Mutavel:
def __init__(self, v):
self.v = v
def __eq__(self, outro):
return self.v == outro.v
def __hash__(self):
return hash(self.v)
m = Mutavel(1)
conjunto = {m}
m.v = 2
print(m in conjunto)
unhashable type: 'Chave'
False
The first class defines __eq__ , and therefore, Python removes the __hash__ from it. The second defines the hash based on a field mutable: after m.v = 2, the object remains in the set, stored in the "bucket" of the old hash, and the query looks in the new bucket. The rule I follow: only define __hash__ for immutable objects, and calculate the hash only from immutable fields.
Dynamic access to attributes
The method __getattr__ is only called when the normal search fails. It is the basis for proxies, configuration objects, and dynamic API clients. The __format__ controls what comes after the colon in an f-string:
class Config:
def __init__(self, **dados):
self._dados = dados
def __getattr__(self, nome):
try:
return self._dados[nome]
except KeyError:
raise AttributeError(nome) from None
c = Config(porta=8080)
print(c.porta, getattr(c, "host", "padrão"), hasattr(c, "porta"))
class Dinheiro:
def __init__(self, centavos):
self.centavos = centavos
def __format__(self, formato):
return format(self.centavos / 100, formato or ".2f")
print(f"{Dinheiro(1050)}", f"{Dinheiro(1050):.1f}")
8080 padrão True
10.50 10.5
Be careful with __getattr__ and internal attributes
In the
__getattr__above, if_dadosdoes not yet exist (duringcopyorpickle, for example), readingself._dadoscall__getattr__again, and the recursion has no end. In production code, handle names that start with an underscore explicitly.
Exercise 1
An interval that behaves like a range
Write the class Intervalo(inicio, fim) with __len__, __getitem__ (accepting negative indices and raising IndexError out of bounds) and __contains__.