Skip to content

    Chapter 42, Advanced

    The data model and protocols

    Almost everything that seems magical in Python is the interpreter calling special methods on its objects. Those who master these protocols write classes that fit into the language.

    Code for this chapter: avancado/cap42_modelo_dados.py

    A sequence with two methods

    If an object has __len__ and __getitem__, it already behaves like a sequence: it accepts index, slice, in, iteration, reversed and random.choice. No inheritance is necessary. It's the duck typing taken seriously:

    avancado/cap42_modelo_dados.pylinhas 10 a 34
    import random
    from collections import namedtuple
    
    Carta = namedtuple("Carta", ["valor", "naipe"])
    
    
    class Baralho:
        valores = [str(n) for n in range(2, 11)] + list("JQKA")
        naipes = ["paus", "ouros", "copas", "espadas"]
    
        def __init__(self):
            self._cartas = [Carta(v, n) for n in self.naipes for v in self.valores]
    
        def __len__(self):
            return len(self._cartas)
    
        def __getitem__(self, posicao):
            return self._cartas[posicao]
    
    
    baralho = Baralho()
    print(len(baralho), baralho[0], baralho[-1])
    print(baralho[:3])
    print(Carta("Q", "copas") in baralho)
    print(random.choice(baralho) in baralho)
    
    Output
    52 Carta(valor='2', naipe='paus') Carta(valor='A', naipe='espadas')
    [Carta(valor='2', naipe='paus'), Carta(valor='3', naipe='paus'), Carta(valor='4', naipe='paus')]
    True
    True
    

    The in works because, without __contains__ or __iter__, Python falls into the old iteration protocol: it calls __getitem__ with 0, 1, 2... until it receives IndexError.

    To actually be a Sequence

    The fact that the object behaves like a sequence does not make it a Sequence for type checking. Abstract classes of collections.abc test for the presence of specific methods, and this check is more demanding than actual iteration:

    avancado/cap42_modelo_dados.pylinhas 39 a 41
    from collections.abc import Container, Iterable, Sequence, Sized
    
    print(isinstance(baralho, Sized), isinstance(baralho, Iterable), isinstance(baralho, Container))
    
    Output
    True False False
    

    It is Sized (has __len__), but it is not Iterable (lacks __iter__), although the for works. The practical lesson: to know if something is iterable, try iter(obj). And, if you want the full contract, inherit from Sequence: you only implement __len__ and __getitem__, and get __contains__, __iter__, __reversed__, index and count:

    avancado/cap42_modelo_dados.pylinhas 43 a 56
    class BaralhoSeq(Sequence):
        def __init__(self):
            self._cartas = Baralho()._cartas
    
        def __len__(self):
            return len(self._cartas)
    
        def __getitem__(self, posicao):
            return self._cartas[posicao]
    
    
    b = BaralhoSeq()
    print(isinstance(b, Sequence), Carta("A", "copas") in b)
    print(b.index(Carta("3", "paus")), b.count(Carta("2", "paus")))
    
    Output
    True True
    1 1
    

    The contract between __eq__ and __hash__

    for free. Two equal objects must have the same hash, and the hash of an object cannot change while it is in a set or is a dictionary key. Breaking the second rule produces one of the most confusing bugs in the language: the object is in the set, but the search does not find it:

    avancado/cap42_modelo_dados.pylinhas 61 a 89
    class Chave:
        def __init__(self, valor):
            self.valor = valor
    
        def __eq__(self, outro):
            return self.valor == outro.valor
    
    
    try:
        {Chave(1)}
    except TypeError as erro:
        print(erro)
    
    
    class Mutavel:
        def __init__(self, v):
            self.v = v
    
        def __eq__(self, outro):
            return self.v == outro.v
    
        def __hash__(self):
            return hash(self.v)
    
    
    m = Mutavel(1)
    conjunto = {m}
    m.v = 2
    print(m in conjunto)
    
    Output
    unhashable type: 'Chave'
    False
    

    The first class defines __eq__ , and therefore, Python removes the __hash__ from it. The second defines the hash based on a field mutable: after m.v = 2, the object remains in the set, stored in the "bucket" of the old hash, and the query looks in the new bucket. The rule I follow: only define __hash__ for immutable objects, and calculate the hash only from immutable fields.

    Dynamic access to attributes

    The method __getattr__ is only called when the normal search fails. It is the basis for proxies, configuration objects, and dynamic API clients. The __format__ controls what comes after the colon in an f-string:

    avancado/cap42_modelo_dados.pylinhas 94 a 117
    class Config:
        def __init__(self, **dados):
            self._dados = dados
    
        def __getattr__(self, nome):
            try:
                return self._dados[nome]
            except KeyError:
                raise AttributeError(nome) from None
    
    
    c = Config(porta=8080)
    print(c.porta, getattr(c, "host", "padrão"), hasattr(c, "porta"))
    
    
    class Dinheiro:
        def __init__(self, centavos):
            self.centavos = centavos
    
        def __format__(self, formato):
            return format(self.centavos / 100, formato or ".2f")
    
    
    print(f"{Dinheiro(1050)}", f"{Dinheiro(1050):.1f}")
    
    Output
    8080 padrão True
    10.50 10.5
    

    Be careful with __getattr__ and internal attributes

    In the __getattr__ above, if _dados does not yet exist (during copy or pickle, for example), reading self._dados call __getattr__ again, and the recursion has no end. In production code, handle names that start with an underscore explicitly.

    Exercise 1

    An interval that behaves like a range

    Write the class Intervalo(inicio, fim) with __len__, __getitem__ (accepting negative indices and raising IndexError out of bounds) and __contains__.