Python's Attribute Protocol in Depth — the toll booth and the signpost

Every attribute access in Python is a method call. Most programmers spend years not needing to know that.

Ask where the value lives. The answer decides which of the two mechanisms you actually needed.

Every attribute access in Python is a method call. Most programmers spend years not needing to know that.

§IFrame

The Ops lesson this morning left a defect sitting in plain view.

@dataclass(frozen=True)
class RetryPolicy:
    max_attempts: int = 4
    base_delay: float = 0.5

RetryPolicy(max_attempts=-1) constructs without complaint. RetryPolicy(base_delay="0.5") constructs too, and the int and float annotations do nothing at runtime, as August's typing lesson established at length. The string survives construction, survives the first attempt, and raises a TypeError inside random.uniform on the first retry, in a stack frame three calls away from the mistake that caused it.

The obvious patch is a __post_init__ that checks each field. Write it once and the shape becomes clear: five fields, five near-identical checks, and the same five checks copied into the next policy class, and the one after it.

Python has a better answer, and it is the answer that property is built out of, that dataclass fields are looked up through, that every ORM column and every validation library in the ecosystem uses. Ramalho gives it a chapter of its own. Lutz gives it two sections and a warning.

Start from the fact underneath all of it. obj.x is not a lookup in a dictionary. It is a protocol, and the protocol has hooks.

§IILanguage Idiom: the attribute protocol

What obj.x actually does

When Python evaluates obj.x, object.__getattribute__ runs. Roughly, in order:

1. Look for x in type(obj).__mro__. If it is found and the class attribute defines both __get__ and __set__, call its __get__ and return. That is a data descriptor, and it wins outright.
2. Otherwise look in obj.__dict__. If x is there, return it.
3. Otherwise fall back to the class attribute found in step 1. If it defines only __get__, call it. That is a non-data descriptor.
4. If nothing was found, call type(obj).__getattr__(obj, "x") if the class defines one. Otherwise raise AttributeError.

Ramalho names the two kinds overriding and nonoverriding descriptors, and the distinction is the whole point of step 1 sitting above step 2. A data descriptor takes precedence over the instance dictionary. A non-data descriptor does not.

This is why property works. A property defines __get__, __set__, and __delete__, so it is a data descriptor, so assigning to the attribute reaches the property's setter rather than quietly shadowing it in obj.__dict__. And it is why a plain function works as a method: functions define __get__ and nothing else, so they are non-data descriptors, and an instance attribute of the same name shadows the method exactly as most programmers expect.

Coin it and keep it: a data descriptor is a toll booth, a non-data descriptor is a signpost. One is unavoidable; one is a default the instance can override.

The three descriptor methods

A descriptor is any class implementing at least one of __get__, __set__, __delete__. The signatures matter.

__get__(self, instance, owner) receives the instance the attribute was read from, which is None when the attribute is read from the class itself. That None case is not an edge case to ignore. RetryPolicy.max_attempts accessed on the class should return the descriptor, so that help(), inspect, and documentation tools see something useful.

__set__(self, instance, value) receives the value being assigned. Its presence is what makes the descriptor a data descriptor, whether or not it does anything interesting.

__set_name__(self, owner, name) is the one that turns descriptors from awkward into pleasant. Python 3.6 added it; the interpreter calls it on every class attribute at class-creation time, handing the descriptor the name it was bound to. Before it existed, a descriptor either had to be told its own name redundantly (max_attempts = Positive("max_attempts")) or generate a unique storage key by counter, which is precisely the awkwardness Ramalho works through across his LineItem takes before arriving at the automatic version.

Where the value lives

A descriptor is a class attribute. One object, shared by every instance. Storing the value on the descriptor itself is therefore the classic beginner bug: every instance would share one value.

The value belongs in the instance's own __dict__, under a key the descriptor derives from the name __set_name__ gave it. The one trap here is name collision. If the descriptor is bound to max_attempts and stores under the key max_attempts, then instance.__dict__["max_attempts"] exists, and step 2 of the lookup would find it, except that step 1 already returned because the descriptor is a data descriptor. The lookup is safe. Readers of vars(obj) are the ones who get confused, which is why a private storage key is worth the small cost.

__getattr__ versus __getattribute__

Lutz's treatment separates these two carefully, and confusing them is the most common way to hang an interpreter.

__getattr__ is a fallback. Python calls it only when normal lookup has already failed. It is cheap, safe, and the right tool for proxies, lazy loading, and friendly error messages.

__getattribute__ is a toll booth. Python calls it on every attribute access, including accesses made from inside __getattribute__ itself. Write return self.__dict__[name] inside it and the interpreter recurses until the stack dies, because self.__dict__ is itself an attribute access. The only safe move inside __getattribute__ is to delegate through the base class: object.__getattribute__(self, name).

Lutz adds a caution that catches people building proxy objects. Implicit lookups of dunder methods for built-in operations skip the instance entirely and are found on the type. A proxy that forwards __len__ through __getattr__ will still fail under len(proxy), because len() looks on type(proxy) and never consults the instance protocol. Dunder forwarding has to be done on the class.

The rule that comes out of this: reach for the fallback first, and reach for the toll booth only when the requirement is genuinely to intercept everything. Taking __getattribute__ means taking on the cost of every attribute read in the process.

§IIICode Worked Example: the policy object that cannot hold a bad value

Build a small validating descriptor, then the base class that uses it, then the policy.

The base descriptor handles storage and naming, and leaves the actual check to subclasses.

from abc import ABC, abstractmethod

class Validated(ABC):
    def __set_name__(self, owner, name):
        self.public_name = name
        self.private_name = "_" + name

    def __get__(self, instance, owner=None):
        if instance is None:
            return self
        return getattr(instance, self.private_name)

    def __set__(self, instance, value):
        value = self.validate(value)
        setattr(instance, self.private_name, value)

    @abstractmethod
    def validate(self, value):
        ...

__set_name__ receives the name at class-creation time, so the field is written as max_attempts = PositiveInt() with no repetition. The instance is None branch in __get__ returns the descriptor itself, which is what makes class-level introspection work. __set__ runs the check before storage, so an instance whose construction completed holds only values that passed.

The concrete validators are small enough to read at a glance.

class PositiveInt(Validated):
    def validate(self, value):
        if not isinstance(value, int) or isinstance(value, bool):
            raise TypeError(f"{self.public_name} must be int, got {type(value).__name__}")
        if value < 1:
            raise ValueError(f"{self.public_name} must be >= 1, got {value}")
        return value

class Seconds(Validated):
    def __init__(self, lo=0.0, hi=3600.0):
        self.lo, self.hi = lo, hi

    def validate(self, value):
        if isinstance(value, bool) or not isinstance(value, (int, float)):
            raise TypeError(f"{self.public_name} must be a number")
        value = float(value)
        if not (self.lo <= value <= self.hi):
            raise ValueError(f"{self.public_name} must be in [{self.lo}, {self.hi}], got {value}")
        return value

The explicit bool rejection is deliberate. isinstance(True, int) is True in Python, so max_attempts=True would otherwise pass as the integer 1 and produce a client that retries once for reasons nobody could reconstruct from the config file.

Now the policy from the Ops lesson, rebuilt.

class RetryPolicy:
    max_attempts = PositiveInt()
    base_delay   = Seconds(lo=0.001, hi=60.0)
    max_delay    = Seconds(lo=0.001, hi=600.0)
    deadline     = Seconds(lo=0.1, hi=3600.0)

    def __init__(self, max_attempts=4, base_delay=0.5, max_delay=20.0, deadline=60.0):
        self.max_attempts = max_attempts
        self.base_delay = base_delay
        self.max_delay = max_delay
        self.deadline = deadline
        if self.base_delay > self.max_delay:
            raise ValueError("base_delay must not exceed max_delay")

Four assignments in __init__, each of which is a __set__ call that runs a check. RetryPolicy(base_delay="0.5") now raises a TypeError naming the field, at the line that assigned it. RetryPolicy(max_attempts=-1) raises a ValueError naming the field and the value.

The cross-field rule sits after the assignments because it needs two validated values to compare. Per-field checks belong in descriptors; relations between fields belong in the constructor. That division holds for every validation system worth using.

One property worth noticing: this validates on every assignment, not only at construction. A configuration reloader that does policy.max_attempts = new_value at runtime gets the same check. The frozen=True dataclass in the Ops lesson forbade that assignment entirely. Descriptors permit it and police it, which is the behavior an ops tool that reloads config actually wants.

For the small subset of cases where the whole object should be inert after construction, __set_name__ and a _frozen flag compose cleanly:

    def __set__(self, instance, value):
        if getattr(instance, "_frozen", False):
            raise AttributeError(f"{self.public_name} is read-only after init")
        setattr(instance, self.private_name, self.validate(value))

The lookup cost, briefly

Each descriptor read is a Python-level __get__ call rather than a dictionary hit. In a tight numeric loop that matters; in the configuration object of an ops tool it does not, because the reads happen once per retry rather than once per element. Wednesday's performance lesson gave the framing this reuses: measure the ratio of interpreter overhead to real work before optimizing anything. A policy object read four times per remote call is entirely on the wrong side of that ratio to care.

§IVConnection to Today's Ops Lesson

The Ops lesson built four controls: timeout, retry policy, retry budget, circuit breaker. Three of the four are configured by numbers, and every one of those numbers has a range outside which the control is meaningless or harmful. A max_attempts of zero disables retries silently. A base_delay of 600 seconds turns a ten-second tool into a ten-minute one. A cooldown of zero on the breaker makes the half-open probe fire continuously, which is the load pattern the breaker exists to prevent.

None of those are caught by type annotations, and none of them are caught by a frozen dataclass. They are caught at the assignment, by a descriptor that knows the field's name and its legal range.

The Breaker class from the Ops lesson takes the same treatment: threshold = PositiveInt(), cooldown = Seconds(lo=1.0, hi=600.0). Two lines, and a category of misconfiguration stops existing.

§VPrior-Lesson Reach

Wednesday's typing lesson made the case that annotations describe intent to a checker and are erased at runtime, and that a library's boundary needs runtime enforcement in addition to static types. Descriptors are that runtime enforcement, and the two compose rather than compete: annotate max_attempts: int for the checker, validate it with PositiveInt() for the caller who ignored the checker.

The performance lesson supplies the discipline for not over-applying this. A descriptor is an interpreter-level call on every read. Config objects, model fields, and API boundaries are worth it. Array elements in a numerical inner loop are not.

The packaging lesson matters because Validated, PositiveInt, and Seconds belong in the shared library rather than in one tool. A fleet where every tool wrote its own validator is a fleet with as many error-message formats as it has tools.

§VIClosing

Three facts are worth carrying out of this.

obj.x runs type(obj).__getattribute__, and the order it searches decides whether a class attribute or an instance attribute wins. A data descriptor defines __set__ and wins over the instance dictionary; a non-data descriptor does not, which is exactly why property is unavoidable and a method is shadowable. __getattr__ catches failures and __getattribute__ catches everything, and only one of those two is safe to reach for casually.

Ramalho spends a chapter on this because it is the machinery beneath a large share of Python's ecosystem. Once the protocol is visible, property, classmethod, functools.cached_property, dataclass fields, SQLAlchemy columns, and Pydantic fields stop being separate pieces of magic and become one mechanism applied six ways.

Open any validation library the fleet imports and find its field class. It defines __set_name__. Read what it does with the name.