ruff/crates/red_knot_python_semantic/resources/mdtest/call/dunder.md

Dunder calls

Introduction

This test suite explains and documents how dunder methods are looked up and called. Throughout the document, we use __getitem__ as an example, but the same principles apply to other dunder methods.

Dunder methods are implicitly called when using certain syntax. For example, the index operator obj[key] calls the __getitem__ method under the hood. Exactly how a dunder method is looked up and called works slightly different from regular methods. Dunder methods are not looked up on obj directly, but rather on type(obj). But in many ways, they still act as if they were called on obj directly. If the __getitem__ member of type(obj) is a descriptor, it is called with obj as the instance argument to __get__. A desugared version of obj[key] is roughly equivalent to getitem_desugared(obj, key) as defined below:

from typing import Any

def find_name_in_mro(typ: type, name: str) -> Any:
    # See implementation in https://docs.python.org/3/howto/descriptor.html#invocation-from-an-instance
    pass

def getitem_desugared(obj: object, key: object) -> object:
    getitem_callable = find_name_in_mro(type(obj), "__getitem__")
    if hasattr(getitem_callable, "__get__"):
        getitem_callable = getitem_callable.__get__(obj, type(obj))

    return getitem_callable(key)

In the following tests, we demonstrate that we implement this behavior correctly.

Operating on class objects

If we invoke a dunder method on a class, it is looked up on the meta class, since any class is an instance of its metaclass:

class Meta(type):
    def __getitem__(cls, key: int) -> str:
        return str(key)

class DunderOnMetaClass(metaclass=Meta):
    pass

reveal_type(DunderOnMetaClass[0])  # revealed: str

Operating on instances

When invoking a dunder method on an instance of a class, it is looked up on the class:

class ClassWithNormalDunder:
    def __getitem__(self, key: int) -> str:
        return str(key)

class_with_normal_dunder = ClassWithNormalDunder()

reveal_type(class_with_normal_dunder[0])  # revealed: str

Which can be demonstrated by trying to attach a dunder method to an instance, which will not work:

def external_getitem(instance, key: int) -> str:
    return str(key)

class ThisFails:
    def __init__(self):
        self.__getitem__ = external_getitem

this_fails = ThisFails()

# error: [non-subscriptable] "Cannot subscript object of type `ThisFails` with no `__getitem__` method"
reveal_type(this_fails[0])  # revealed: Unknown

However, the attached dunder method can be called if accessed directly:

# TODO: `this_fails.__getitem__` is incorrectly treated as a bound method. This
# should be fixed with https://github.com/astral-sh/ruff/issues/16367
# error: [too-many-positional-arguments]
# error: [invalid-argument-type]
reveal_type(this_fails.__getitem__(this_fails, 0))  # revealed: Unknown | str

When the dunder is not a method

A dunder can also be a non-method callable:

class SomeCallable:
    def __call__(self, key: int) -> str:
        return str(key)

class ClassWithNonMethodDunder:
    __getitem__: SomeCallable = SomeCallable()

class_with_callable_dunder = ClassWithNonMethodDunder()

reveal_type(class_with_callable_dunder[0])  # revealed: str

Dunders are looked up using the descriptor protocol

Here, we demonstrate that the descriptor protocol is invoked when looking up a dunder method. Note that the instance argument is on object of type ClassWithDescriptorDunder:

from __future__ import annotations

class SomeCallable:
    def __call__(self, key: int) -> str:
        return str(key)

class Descriptor:
    def __get__(self, instance: ClassWithDescriptorDunder, owner: type[ClassWithDescriptorDunder]) -> SomeCallable:
        return SomeCallable()

class ClassWithDescriptorDunder:
    __getitem__: Descriptor = Descriptor()

class_with_descriptor_dunder = ClassWithDescriptorDunder()

reveal_type(class_with_descriptor_dunder[0])  # revealed: str