[red-knot] Correct modeling of dunder calls (#16368)
## Summary
Model dunder-calls correctly (and in one single place), by implementing
this behavior (using `__getitem__` as an example).
```py
def getitem_desugared(obj: object, key: object) -> object:
getitem_callable = find_in_mro(type(obj), "__getitem__")
if hasattr(getitem_callable, "__get__"):
getitem_callable = getitem_callable.__get__(obj, type(obj))
return getitem_callable(key)
```
See the new `calls/dunder.md` test suite for more information. The new
behavior also needs much fewer lines of code (the diff is positive due
to new tests).
## Test Plan
New tests; fix TODOs in existing tests.
This commit is contained in:
@@ -82,7 +82,7 @@ class C:
|
||||
|
||||
c = C()
|
||||
|
||||
# error: 15 [invalid-argument-type] "Object of type `Literal["foo"]` cannot be assigned to parameter 2 (`x`) of function `__call__`; expected type `int`"
|
||||
# error: 15 [invalid-argument-type] "Object of type `Literal["foo"]` cannot be assigned to parameter 2 (`x`) of bound method `__call__`; expected type `int`"
|
||||
reveal_type(c("foo")) # revealed: int
|
||||
```
|
||||
|
||||
@@ -96,7 +96,7 @@ class C:
|
||||
|
||||
c = C()
|
||||
|
||||
# error: 13 [invalid-argument-type] "Object of type `C` cannot be assigned to parameter 1 (`self`) of function `__call__`; expected type `int`"
|
||||
# error: 13 [invalid-argument-type] "Object of type `C` cannot be assigned to parameter 1 (`self`) of bound method `__call__`; expected type `int`"
|
||||
reveal_type(c()) # revealed: int
|
||||
```
|
||||
|
||||
|
||||
128
crates/red_knot_python_semantic/resources/mdtest/call/dunder.md
Normal file
128
crates/red_knot_python_semantic/resources/mdtest/call/dunder.md
Normal file
@@ -0,0 +1,128 @@
|
||||
# Dunder calls
|
||||
|
||||
## Introduction
|
||||
|
||||
This test suite explains and documents how dunder methods are looked up and called. Throughout the
|
||||
document, we use `__getitem__` as an example, but the same principles apply to other dunder methods.
|
||||
|
||||
Dunder methods are implicitly called when using certain syntax. For example, the index operator
|
||||
`obj[key]` calls the `__getitem__` method under the hood. Exactly *how* a dunder method is looked up
|
||||
and called works slightly different from regular methods. Dunder methods are not looked up on `obj`
|
||||
directly, but rather on `type(obj)`. But in many ways, they still *act* as if they were called on
|
||||
`obj` directly. If the `__getitem__` member of `type(obj)` is a descriptor, it is called with `obj`
|
||||
as the `instance` argument to `__get__`. A desugared version of `obj[key]` is roughly equivalent to
|
||||
`getitem_desugared(obj, key)` as defined below:
|
||||
|
||||
```py
|
||||
from typing import Any
|
||||
|
||||
def find_name_in_mro(typ: type, name: str) -> Any:
|
||||
# See implementation in https://docs.python.org/3/howto/descriptor.html#invocation-from-an-instance
|
||||
pass
|
||||
|
||||
def getitem_desugared(obj: object, key: object) -> object:
|
||||
getitem_callable = find_name_in_mro(type(obj), "__getitem__")
|
||||
if hasattr(getitem_callable, "__get__"):
|
||||
getitem_callable = getitem_callable.__get__(obj, type(obj))
|
||||
|
||||
return getitem_callable(key)
|
||||
```
|
||||
|
||||
In the following tests, we demonstrate that we implement this behavior correctly.
|
||||
|
||||
## Operating on class objects
|
||||
|
||||
If we invoke a dunder method on a class, it is looked up on the *meta* class, since any class is an
|
||||
instance of its metaclass:
|
||||
|
||||
```py
|
||||
class Meta(type):
|
||||
def __getitem__(cls, key: int) -> str:
|
||||
return str(key)
|
||||
|
||||
class DunderOnMetaClass(metaclass=Meta):
|
||||
pass
|
||||
|
||||
reveal_type(DunderOnMetaClass[0]) # revealed: str
|
||||
```
|
||||
|
||||
## Operating on instances
|
||||
|
||||
When invoking a dunder method on an instance of a class, it is looked up on the class:
|
||||
|
||||
```py
|
||||
class ClassWithNormalDunder:
|
||||
def __getitem__(self, key: int) -> str:
|
||||
return str(key)
|
||||
|
||||
class_with_normal_dunder = ClassWithNormalDunder()
|
||||
|
||||
reveal_type(class_with_normal_dunder[0]) # revealed: str
|
||||
```
|
||||
|
||||
Which can be demonstrated by trying to attach a dunder method to an instance, which will not work:
|
||||
|
||||
```py
|
||||
def external_getitem(instance, key: int) -> str:
|
||||
return str(key)
|
||||
|
||||
class ThisFails:
|
||||
def __init__(self):
|
||||
self.__getitem__ = external_getitem
|
||||
|
||||
this_fails = ThisFails()
|
||||
|
||||
# error: [non-subscriptable] "Cannot subscript object of type `ThisFails` with no `__getitem__` method"
|
||||
reveal_type(this_fails[0]) # revealed: Unknown
|
||||
```
|
||||
|
||||
However, the attached dunder method *can* be called if accessed directly:
|
||||
|
||||
```py
|
||||
# TODO: `this_fails.__getitem__` is incorrectly treated as a bound method. This
|
||||
# should be fixed with https://github.com/astral-sh/ruff/issues/16367
|
||||
# error: [too-many-positional-arguments]
|
||||
# error: [invalid-argument-type]
|
||||
reveal_type(this_fails.__getitem__(this_fails, 0)) # revealed: Unknown | str
|
||||
```
|
||||
|
||||
## When the dunder is not a method
|
||||
|
||||
A dunder can also be a non-method callable:
|
||||
|
||||
```py
|
||||
class SomeCallable:
|
||||
def __call__(self, key: int) -> str:
|
||||
return str(key)
|
||||
|
||||
class ClassWithNonMethodDunder:
|
||||
__getitem__: SomeCallable = SomeCallable()
|
||||
|
||||
class_with_callable_dunder = ClassWithNonMethodDunder()
|
||||
|
||||
reveal_type(class_with_callable_dunder[0]) # revealed: str
|
||||
```
|
||||
|
||||
## Dunders are looked up using the descriptor protocol
|
||||
|
||||
Here, we demonstrate that the descriptor protocol is invoked when looking up a dunder method. Note
|
||||
that the `instance` argument is on object of type `ClassWithDescriptorDunder`:
|
||||
|
||||
```py
|
||||
from __future__ import annotations
|
||||
|
||||
class SomeCallable:
|
||||
def __call__(self, key: int) -> str:
|
||||
return str(key)
|
||||
|
||||
class Descriptor:
|
||||
def __get__(self, instance: ClassWithDescriptorDunder, owner: type[ClassWithDescriptorDunder]) -> SomeCallable:
|
||||
return SomeCallable()
|
||||
|
||||
class ClassWithDescriptorDunder:
|
||||
__getitem__: Descriptor = Descriptor()
|
||||
|
||||
class_with_descriptor_dunder = ClassWithDescriptorDunder()
|
||||
|
||||
reveal_type(class_with_descriptor_dunder[0]) # revealed: str
|
||||
```
|
||||
Reference in New Issue
Block a user