From 2c0c5ff4e7a0f347e81bdcdb6e38180dcee0688b Mon Sep 17 00:00:00 2001 From: Shunsuke Shibayama <45118249+mtshiba@users.noreply.github.com> Date: Thu, 27 Nov 2025 01:50:26 +0900 Subject: [PATCH] [ty] handle recursive type inference properly (#20566) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary Derived from #17371 Fixes astral-sh/ty#256 Fixes https://github.com/astral-sh/ty/issues/1415 Fixes https://github.com/astral-sh/ty/issues/1433 Fixes https://github.com/astral-sh/ty/issues/1524 Properly handles any kind of recursive inference and prevents panics. --- Let me explain techniques for converging fixed-point iterations during recursive type inference. There are two types of type inference that naively don't converge (causing salsa to panic): divergent type inference and oscillating type inference. ### Divergent type inference Divergent type inference occurs when eagerly expanding a recursive type. A typical example is this: ```python class C: def f(self, other: "C"): self.x = (other.x, 1) reveal_type(C().x) # revealed: Unknown | tuple[Unknown | tuple[Unknown | tuple[..., Literal[1]], Literal[1]], Literal[1]] ``` To solve this problem, we have already introduced `Divergent` types (https://github.com/astral-sh/ruff/pull/20312). `Divergent` types are treated as a kind of dynamic type [^1]. ```python Unknown | tuple[Unknown | tuple[Unknown | tuple[..., Literal[1]], Literal[1]], Literal[1]] => Unknown | tuple[Divergent, Literal[1]] ``` When a query function that returns a type enters a cycle, it sets `Divergent` as the cycle initial value (instead of `Never`). Then, in the cycle recovery function, it reduces the nesting of types containing `Divergent` to converge. ```python 0th: Divergent 1st: Unknown | tuple[Divergent, Literal[1]] 2nd: Unknown | tuple[Unknown | tuple[Divergent, Literal[1]], Literal[1]] => Unknown | tuple[Divergent, Literal[1]] ``` Each cycle recovery function for each query should operate only on the `Divergent` type originating from that query. For this reason, while `Divergent` appears the same as `Any` to the user, it internally carries some information: the location where the cycle occurred. Previously, we roughly identified this by having the scope where the cycle occurred, but with the update to salsa, functions that create cycle initial values ​​can now receive a `salsa::Id` (https://github.com/salsa-rs/salsa/pull/1012). This is an opaque ID that uniquely identifies the cycle head (the query that is the starting point for the fixed-point iteration). `Divergent` now has this `salsa::Id`. ### Oscillating type inference Now, another thing to consider is oscillating type inference. Oscillating type inference arises from the fact that monotonicity is broken. Monotonicity here means that for a query function, if it enters a cycle, the calculation must start from a "bottom value" and progress towards the final result with each cycle. Monotonicity breaks down in type systems that have features like overloading and overriding. ```python class Base: def flip(self) -> "Sub": return Sub() class Sub(Base): def flip(self) -> "Base": return Base() class C: def __init__(self, x: Sub): self.x = x def replace_with(self, other: "C"): self.x = other.x.flip() reveal_type(C(Sub()).x) ``` Naive fixed-point iteration results in `Divergent -> Sub -> Base -> Sub -> ...`, which oscillates forever without diverging or converging. To address this, the salsa API has been modified so that the cycle recovery function receives the value of the previous cycle (https://github.com/salsa-rs/salsa/pull/1012). The cycle recovery function returns the union type of the current cycle and the previous cycle. In the above example, the result type for each cycle is `Divergent -> Sub -> Base (= Sub | Base) -> Base`, which converges. The final result of oscillating type inference does not contain `Divergent` because `Divergent` that appears in a union type can be removed, as is clear from the expansion. This simplification is performed at the same time as nesting reduction. ``` T | Divergent = T | (T | (T | ...)) = T ``` [^1]: In theory, it may be possible to strictly treat types containing `Divergent` types as recursive types, but we probably shouldn't go that deep yet. (AFAIK, there are no PEPs that specify how to handle implicitly recursive types that aren't named by type aliases) ## Performance analysis A happy side effect of this PR is that we've observed widespread performance improvements! This is likely due to the removal of the `ITERATIONS_BEFORE_FALLBACK` and max-specialization depth trick (https://github.com/astral-sh/ty/issues/1433, https://github.com/astral-sh/ty/issues/1415), which means we reach a fixed point much sooner. ## Ecosystem analysis The changes look good overall. You may notice changes in the converged values ​​for recursive types, this is because the way recursive types are normalized has been changed. Previously, types containing `Divergent` types were normalized by replacing them with the `Divergent` type itself, but in this PR, types with a nesting level of 2 or more that contain `Divergent` types are normalized by replacing them with a type with a nesting level of 1. This means that information about the non-divergent parts of recursive types is no longer lost. ```python # previous tuple[tuple[Divergent, int], int] => Divergent # now tuple[tuple[Divergent, int], int] => tuple[Divergent, int] ``` The false positive error introduced in this PR occurs in class definitions with self-referential base classes, such as the one below. ```python from typing_extensions import Generic, TypeVar T = TypeVar("T") U = TypeVar("U") class Base2(Generic[T, U]): ... # TODO: no error # error: [unsupported-base] "Unsupported class base with type ` | `" class Sub2(Base2["Sub2", U]): ... ``` This is due to the lack of support for unions of MROs, or because cyclic legacy generic types are not inferred as generic types early in the query cycle. ## Test Plan All samples listed in astral-sh/ty#256 are tested and passed without any panic! ## Acknowledgments Thanks to @MichaReiser for working on bug fixes and improvements to salsa for this PR. @carljm also contributed early on to the discussion of the query convergence mechanism proposed in this PR. --------- Co-authored-by: Carl Meyer --- crates/ty/docs/rules.md | 168 ++-- .../corpus/classliteral_decorators_cycle.py | 26 + .../resources/corpus/cyclic_reassignment.py | 5 + .../resources/corpus/cyclic_type_alias.py | 20 + .../resources/mdtest/attributes.md | 64 +- .../resources/mdtest/cycle.md | 49 + .../mdtest/generics/legacy/classes.md | 10 + .../resources/mdtest/import/cyclic.md | 2 +- .../resources/mdtest/pep613_type_aliases.md | 31 +- .../resources/mdtest/pep695_type_aliases.md | 66 +- .../1377_iteration_count_mismatch.md | 5 +- crates/ty_python_semantic/src/place.rs | 69 +- .../ty_python_semantic/src/semantic_model.rs | 3 +- crates/ty_python_semantic/src/types.rs | 880 ++++++++++++++++-- .../src/types/bound_super.rs | 38 +- .../ty_python_semantic/src/types/builder.rs | 14 +- crates/ty_python_semantic/src/types/class.rs | 92 +- .../src/types/class_base.rs | 18 +- crates/ty_python_semantic/src/types/cyclic.rs | 17 + .../src/types/diagnostic.rs | 23 + .../ty_python_semantic/src/types/function.rs | 28 + .../ty_python_semantic/src/types/generics.rs | 54 +- crates/ty_python_semantic/src/types/infer.rs | 296 +++--- .../src/types/infer/builder.rs | 123 ++- .../types/infer/builder/type_expression.rs | 39 +- .../ty_python_semantic/src/types/instance.rs | 84 +- crates/ty_python_semantic/src/types/member.rs | 2 +- .../ty_python_semantic/src/types/newtype.rs | 23 +- .../src/types/protocol_class.rs | 61 ++ .../src/types/signatures.rs | 125 +++ .../src/types/subclass_of.rs | 37 +- crates/ty_python_semantic/src/types/tuple.rs | 112 ++- .../src/types/type_ordering.rs | 4 +- .../ty_python_semantic/src/types/unpacker.rs | 25 +- .../ty_python_semantic/src/types/visitor.rs | 152 +-- crates/ty_python_semantic/tests/corpus.rs | 3 - .../e2e__commands__debug_command.snap | 1 + ty.schema.json | 10 + 38 files changed, 2223 insertions(+), 556 deletions(-) create mode 100644 crates/ty_python_semantic/resources/corpus/classliteral_decorators_cycle.py create mode 100644 crates/ty_python_semantic/resources/corpus/cyclic_reassignment.py create mode 100644 crates/ty_python_semantic/resources/corpus/cyclic_type_alias.py diff --git a/crates/ty/docs/rules.md b/crates/ty/docs/rules.md index a6b571c402..b039ba94f7 100644 --- a/crates/ty/docs/rules.md +++ b/crates/ty/docs/rules.md @@ -39,7 +39,7 @@ def test(): -> "int": Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -63,7 +63,7 @@ Calling a non-callable object will raise a `TypeError` at runtime. Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -95,7 +95,7 @@ f(int) # error Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -126,7 +126,7 @@ a = 1 Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -158,7 +158,7 @@ class C(A, B): ... Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -184,13 +184,41 @@ class B(A): ... [method resolution order]: https://docs.python.org/3/glossary.html#term-method-resolution-order +## `cyclic-type-alias-definition` + + +Default level: error · +Preview (since 1.0.0) · +Related issues · +View source + + + +**What it does** + +Checks for type alias definitions that (directly or mutually) refer to themselves. + +**Why is it bad?** + +Although it is permitted to define a recursive type alias, it is not meaningful +to have a type alias whose expansion can only result in itself, and is therefore not allowed. + +**Examples** + +```python +type Itself = Itself + +type A = B +type B = A +``` + ## `duplicate-base` Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -217,7 +245,7 @@ class B(A, A): ... Default level: error · Added in 0.0.1-alpha.12 · Related issues · -View source +View source @@ -329,7 +357,7 @@ def test(): -> "Literal[5]": Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -359,7 +387,7 @@ class C(A, B): ... Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -385,7 +413,7 @@ t[3] # IndexError: tuple index out of range Default level: error · Added in 0.0.1-alpha.12 · Related issues · -View source +View source @@ -474,7 +502,7 @@ an atypical memory layout. Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -501,7 +529,7 @@ func("foo") # error: [invalid-argument-type] Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -529,7 +557,7 @@ a: int = '' Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -563,7 +591,7 @@ C.instance_var = 3 # error: Cannot assign to instance variable Default level: error · Added in 0.0.1-alpha.19 · Related issues · -View source +View source @@ -599,7 +627,7 @@ asyncio.run(main()) Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -623,7 +651,7 @@ class A(42): ... # error: [invalid-base] Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -650,7 +678,7 @@ with 1: Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -679,7 +707,7 @@ a: str Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -723,7 +751,7 @@ except ZeroDivisionError: Default level: error · Added in 0.0.1-alpha.28 · Related issues · -View source +View source @@ -765,7 +793,7 @@ class D(A): Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -798,7 +826,7 @@ class C[U](Generic[T]): ... Default level: error · Added in 0.0.1-alpha.17 · Related issues · -View source +View source @@ -837,7 +865,7 @@ carol = Person(name="Carol", age=25) # typo! Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -872,7 +900,7 @@ def f(t: TypeVar("U")): ... Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -906,7 +934,7 @@ class B(metaclass=f): ... Default level: error · Added in 0.0.1-alpha.20 · Related issues · -View source +View source @@ -992,7 +1020,7 @@ and `__ne__` methods accept `object` as their second argument. Default level: error · Added in 0.0.1-alpha.19 · Related issues · -View source +View source @@ -1024,7 +1052,7 @@ TypeError: can only inherit from a NamedTuple type and Generic Default level: error · Preview (since 1.0.0) · Related issues · -View source +View source @@ -1054,7 +1082,7 @@ Baz = NewType("Baz", int | str) # error: invalid base for `typing.NewType` Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1104,7 +1132,7 @@ def foo(x: int) -> int: ... Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1130,7 +1158,7 @@ def f(a: int = ''): ... Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1161,7 +1189,7 @@ P2 = ParamSpec("S2") # error: ParamSpec name must match the variable it's assig Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1195,7 +1223,7 @@ TypeError: Protocols can only inherit from other protocols, got Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1244,7 +1272,7 @@ def g(): Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1269,7 +1297,7 @@ def func() -> int: Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1327,7 +1355,7 @@ TODO #14889 Default level: error · Added in 0.0.1-alpha.6 · Related issues · -View source +View source @@ -1354,7 +1382,7 @@ NewAlias = TypeAliasType(get_name(), int) # error: TypeAliasType name mus Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1384,7 +1412,7 @@ TYPE_CHECKING = '' Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1414,7 +1442,7 @@ b: Annotated[int] # `Annotated` expects at least two arguments Default level: error · Added in 0.0.1-alpha.11 · Related issues · -View source +View source @@ -1448,7 +1476,7 @@ f(10) # Error Default level: error · Added in 0.0.1-alpha.11 · Related issues · -View source +View source @@ -1482,7 +1510,7 @@ class C: Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1517,7 +1545,7 @@ T = TypeVar('T', bound=str) # valid bound TypeVar Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1542,7 +1570,7 @@ func() # TypeError: func() missing 1 required positional argument: 'x' Default level: error · Added in 0.0.1-alpha.20 · Related issues · -View source +View source @@ -1575,7 +1603,7 @@ alice["age"] # KeyError Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1604,7 +1632,7 @@ func("string") # error: [no-matching-overload] Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1628,7 +1656,7 @@ Subscripting an object that does not support it will raise a `TypeError` at runt Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1654,7 +1682,7 @@ for i in 34: # TypeError: 'int' object is not iterable Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1681,7 +1709,7 @@ f(1, x=2) # Error raised here Default level: error · Added in 0.0.1-alpha.22 · Related issues · -View source +View source @@ -1739,7 +1767,7 @@ def test(): -> "int": Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1769,7 +1797,7 @@ static_assert(int(2.0 * 3.0) == 6) # error: does not have a statically known tr Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1798,7 +1826,7 @@ class B(A): ... # Error raised here Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1825,7 +1853,7 @@ f("foo") # Error raised here Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1853,7 +1881,7 @@ def _(x: int): Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1899,7 +1927,7 @@ class A: Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1926,7 +1954,7 @@ f(x=1, y=2) # Error raised here Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1954,7 +1982,7 @@ A().foo # AttributeError: 'A' object has no attribute 'foo' Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1979,7 +2007,7 @@ import foo # ModuleNotFoundError: No module named 'foo' Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -2004,7 +2032,7 @@ print(x) # NameError: name 'x' is not defined Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -2041,7 +2069,7 @@ b1 < b2 < b1 # exception raised here Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -2069,7 +2097,7 @@ A() + A() # TypeError: unsupported operand type(s) for +: 'A' and 'A' Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -2094,7 +2122,7 @@ l[1:10:0] # ValueError: slice step cannot be zero Default level: warn · Added in 0.0.1-alpha.20 · Related issues · -View source +View source @@ -2135,7 +2163,7 @@ class SubProto(BaseProto, Protocol): Default level: warn · Added in 0.0.1-alpha.16 · Related issues · -View source +View source @@ -2223,7 +2251,7 @@ a = 20 / 0 # type: ignore Default level: warn · Added in 0.0.1-alpha.22 · Related issues · -View source +View source @@ -2251,7 +2279,7 @@ A.c # AttributeError: type object 'A' has no attribute 'c' Default level: warn · Added in 0.0.1-alpha.22 · Related issues · -View source +View source @@ -2283,7 +2311,7 @@ A()[0] # TypeError: 'A' object is not subscriptable Default level: warn · Added in 0.0.1-alpha.22 · Related issues · -View source +View source @@ -2315,7 +2343,7 @@ from module import a # ImportError: cannot import name 'a' from 'module' Default level: warn · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -2342,7 +2370,7 @@ cast(int, f()) # Redundant Default level: warn · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -2366,7 +2394,7 @@ reveal_type(1) # NameError: name 'reveal_type' is not defined Default level: warn · Added in 0.0.1-alpha.15 · Related issues · -View source +View source @@ -2424,7 +2452,7 @@ def g(): Default level: warn · Added in 0.0.1-alpha.7 · Related issues · -View source +View source @@ -2463,7 +2491,7 @@ class D(C): ... # error: [unsupported-base] Default level: warn · Added in 0.0.1-alpha.22 · Related issues · -View source +View source @@ -2526,7 +2554,7 @@ def foo(x: int | str) -> int | str: Default level: ignore · Preview (since 0.0.1-alpha.1) · Related issues · -View source +View source @@ -2550,7 +2578,7 @@ Dividing by zero raises a `ZeroDivisionError` at runtime. Default level: ignore · Added in 0.0.1-alpha.1 · Related issues · -View source +View source diff --git a/crates/ty_python_semantic/resources/corpus/classliteral_decorators_cycle.py b/crates/ty_python_semantic/resources/corpus/classliteral_decorators_cycle.py new file mode 100644 index 0000000000..7130432c69 --- /dev/null +++ b/crates/ty_python_semantic/resources/corpus/classliteral_decorators_cycle.py @@ -0,0 +1,26 @@ +try: + type name_4 = name_1 +finally: + from .. import name_3 + +try: + pass +except* 0: + pass +else: + def name_1() -> name_4: + pass + + @name_1 + def name_3(): + pass +finally: + try: + pass + except* 0: + assert name_3 + finally: + + @name_3 + class name_1: + pass diff --git a/crates/ty_python_semantic/resources/corpus/cyclic_reassignment.py b/crates/ty_python_semantic/resources/corpus/cyclic_reassignment.py new file mode 100644 index 0000000000..5df2694654 --- /dev/null +++ b/crates/ty_python_semantic/resources/corpus/cyclic_reassignment.py @@ -0,0 +1,5 @@ +class foo[_: foo](object): ... + +[_] = (foo,) = foo + +def foo(): ... diff --git a/crates/ty_python_semantic/resources/corpus/cyclic_type_alias.py b/crates/ty_python_semantic/resources/corpus/cyclic_type_alias.py new file mode 100644 index 0000000000..38a2e3def8 --- /dev/null +++ b/crates/ty_python_semantic/resources/corpus/cyclic_type_alias.py @@ -0,0 +1,20 @@ +name_3: Foo = 0 +name_4 = 0 + +if _0: + type name_3 = name_5 + type name_4 = name_3 + +_1: name_3 + +def name_1(_2: name_4): + pass + +match 0: + case name_1._3: + pass + case 1: + type name_5 = name_4 + case name_5: + pass +name_3 = name_5 diff --git a/crates/ty_python_semantic/resources/mdtest/attributes.md b/crates/ty_python_semantic/resources/mdtest/attributes.md index 176865a04d..bef0451ea2 100644 --- a/crates/ty_python_semantic/resources/mdtest/attributes.md +++ b/crates/ty_python_semantic/resources/mdtest/attributes.md @@ -2383,6 +2383,34 @@ class B: reveal_type(B().x) # revealed: Unknown | Literal[1] reveal_type(A().x) # revealed: Unknown | Literal[1] + +class Base: + def flip(self) -> "Sub": + return Sub() + +class Sub(Base): + # error: [invalid-method-override] + def flip(self) -> "Base": + return Base() + +class C2: + def __init__(self, x: Sub): + self.x = x + + def replace_with(self, other: "C2"): + self.x = other.x.flip() + +reveal_type(C2(Sub()).x) # revealed: Unknown | Base + +class C3: + def __init__(self, x: Sub): + self.x = [x] + + def replace_with(self, other: "C3"): + self.x = [self.x[0].flip()] + +# TODO: should be `Unknown | list[Unknown | Sub] | list[Unknown | Base]` +reveal_type(C3(Sub()).x) # revealed: Unknown | list[Unknown | Sub] | list[Divergent] ``` And cycles between many attributes: @@ -2432,6 +2460,30 @@ class ManyCycles: reveal_type(self.x5) # revealed: Unknown | int reveal_type(self.x6) # revealed: Unknown | int reveal_type(self.x7) # revealed: Unknown | int + +class ManyCycles2: + def __init__(self: "ManyCycles2"): + self.x1 = [0] + self.x2 = [1] + self.x3 = [1] + + def f1(self: "ManyCycles2"): + # TODO: should be Unknown | list[Unknown | int] | list[Divergent] + reveal_type(self.x3) # revealed: Unknown | list[Unknown | int] | list[Divergent] | list[Divergent] + + self.x1 = [self.x2] + [self.x3] + self.x2 = [self.x1] + [self.x3] + self.x3 = [self.x1] + [self.x2] + + def f2(self: "ManyCycles2"): + self.x1 = self.x2 + self.x3 + self.x2 = self.x1 + self.x3 + self.x3 = self.x1 + self.x2 + + def f3(self: "ManyCycles2"): + self.x1 = self.x2 + self.x3 + self.x2 = self.x1 + self.x3 + self.x3 = self.x1 + self.x2 ``` This case additionally tests our union/intersection simplification logic: @@ -2611,12 +2663,18 @@ reveal_type(Answer.__members__) # revealed: MappingProxyType[str, Unknown] ## Divergent inferred implicit instance attribute types +If an implicit attribute is defined recursively and type inference diverges, the divergent part is +filled in with the dynamic type `Divergent`. Types containing `Divergent` can be seen as "cheap" +recursive types: they are not true recursive types based on recursive type theory, so no unfolding +is performed when you use them. + ```py class C: def f(self, other: "C"): self.x = (other.x, 1) reveal_type(C().x) # revealed: Unknown | tuple[Divergent, Literal[1]] +reveal_type(C().x[0]) # revealed: Unknown | Divergent ``` This also works if the tuple is not constructed directly: @@ -2655,11 +2713,11 @@ And it also works for homogeneous tuples: def make_homogeneous_tuple(x: T) -> tuple[T, ...]: return (x, x) -class E: - def f(self, other: "E"): +class F: + def f(self, other: "F"): self.x = make_homogeneous_tuple(other.x) -reveal_type(E().x) # revealed: Unknown | tuple[Divergent, ...] +reveal_type(F().x) # revealed: Unknown | tuple[Divergent, ...] ``` ## Attributes of standard library modules that aren't yet defined diff --git a/crates/ty_python_semantic/resources/mdtest/cycle.md b/crates/ty_python_semantic/resources/mdtest/cycle.md index e52641a05c..30d8e9554a 100644 --- a/crates/ty_python_semantic/resources/mdtest/cycle.md +++ b/crates/ty_python_semantic/resources/mdtest/cycle.md @@ -32,6 +32,55 @@ reveal_type(p.x) # revealed: Unknown | int reveal_type(p.y) # revealed: Unknown | int ``` +## Self-referential bare type alias + +```toml +[environment] +python-version = "3.12" # typing.TypeAliasType +``` + +```py +from typing import Union, TypeAliasType, Sequence, Mapping + +A = list["A" | None] + +def f(x: A): + # TODO: should be `list[A | None]`? + reveal_type(x) # revealed: list[Divergent] + # TODO: should be `A | None`? + reveal_type(x[0]) # revealed: Divergent + +JSONPrimitive = Union[str, int, float, bool, None] +JSONValue = TypeAliasType("JSONValue", 'Union[JSONPrimitive, Sequence["JSONValue"], Mapping[str, "JSONValue"]]') +``` + +## Self-referential legacy type variables + +```py +from typing import Generic, TypeVar + +B = TypeVar("B", bound="Base") + +class Base(Generic[B]): + pass + +T = TypeVar("T", bound="Foo[int]") + +class Foo(Generic[T]): ... +``` + +## Self-referential PEP-695 type variables + +```toml +[environment] +python-version = "3.12" +``` + +```py +class Node[T: "Node[int]"]: + pass +``` + ## Parameter default values This is a regression test for . When a parameter has a diff --git a/crates/ty_python_semantic/resources/mdtest/generics/legacy/classes.md b/crates/ty_python_semantic/resources/mdtest/generics/legacy/classes.md index 09df84b42e..9d6cd6ded7 100644 --- a/crates/ty_python_semantic/resources/mdtest/generics/legacy/classes.md +++ b/crates/ty_python_semantic/resources/mdtest/generics/legacy/classes.md @@ -732,6 +732,14 @@ class Base(Generic[T]): ... class Sub(Base["Sub"]): ... reveal_type(Sub) # revealed: + +U = TypeVar("U") + +class Base2(Generic[T, U]): ... + +# TODO: no error +# error: [unsupported-base] "Unsupported class base with type ` | `" +class Sub2(Base2["Sub2", U]): ... ``` #### Without string forward references @@ -756,6 +764,8 @@ from typing_extensions import Generic, TypeVar T = TypeVar("T") +# TODO: no error "Unsupported class base with type ` | `" +# error: [unsupported-base] class Derived(list[Derived[T]], Generic[T]): ... ``` diff --git a/crates/ty_python_semantic/resources/mdtest/import/cyclic.md b/crates/ty_python_semantic/resources/mdtest/import/cyclic.md index 4af714c7de..81c04c4620 100644 --- a/crates/ty_python_semantic/resources/mdtest/import/cyclic.md +++ b/crates/ty_python_semantic/resources/mdtest/import/cyclic.md @@ -38,7 +38,7 @@ See: from pkg.sub import A # TODO: This should be `` -reveal_type(A) # revealed: Never +reveal_type(A) # revealed: Divergent ``` `pkg/outer.py`: diff --git a/crates/ty_python_semantic/resources/mdtest/pep613_type_aliases.md b/crates/ty_python_semantic/resources/mdtest/pep613_type_aliases.md index 13f05ddd5d..40e5d2c477 100644 --- a/crates/ty_python_semantic/resources/mdtest/pep613_type_aliases.md +++ b/crates/ty_python_semantic/resources/mdtest/pep613_type_aliases.md @@ -144,17 +144,46 @@ def _(x: IntOrStr): ## Cyclic ```py -from typing import TypeAlias +from typing import TypeAlias, TypeVar, Union +from types import UnionType RecursiveTuple: TypeAlias = tuple[int | "RecursiveTuple", str] def _(rec: RecursiveTuple): + # TODO should be `tuple[int | RecursiveTuple, str]` reveal_type(rec) # revealed: tuple[Divergent, str] RecursiveHomogeneousTuple: TypeAlias = tuple[int | "RecursiveHomogeneousTuple", ...] def _(rec: RecursiveHomogeneousTuple): + # TODO should be `tuple[int | RecursiveHomogeneousTuple, ...]` reveal_type(rec) # revealed: tuple[Divergent, ...] + +ClassInfo: TypeAlias = type | UnionType | tuple["ClassInfo", ...] +reveal_type(ClassInfo) # revealed: types.UnionType + +def my_isinstance(obj: object, classinfo: ClassInfo) -> bool: + # TODO should be `type | UnionType | tuple[ClassInfo, ...]` + reveal_type(classinfo) # revealed: type | UnionType | tuple[Divergent, ...] + return isinstance(obj, classinfo) + +K = TypeVar("K") +V = TypeVar("V") +NestedDict: TypeAlias = dict[K, Union[V, "NestedDict[K, V]"]] + +def _(nested: NestedDict[str, int]): + # TODO should be `dict[str, int | NestedDict[str, int]]` + reveal_type(nested) # revealed: @Todo(specialized generic alias in type expression) + +my_isinstance(1, int) +my_isinstance(1, int | str) +my_isinstance(1, (int, str)) +my_isinstance(1, (int, (str, float))) +my_isinstance(1, (int, (str | float))) +# error: [invalid-argument-type] +my_isinstance(1, 1) +# TODO should be an invalid-argument-type error +my_isinstance(1, (int, (str, 1))) ``` ## Conditionally imported on Python < 3.10 diff --git a/crates/ty_python_semantic/resources/mdtest/pep695_type_aliases.md b/crates/ty_python_semantic/resources/mdtest/pep695_type_aliases.md index f0c73fc16a..d4e4fafc73 100644 --- a/crates/ty_python_semantic/resources/mdtest/pep695_type_aliases.md +++ b/crates/ty_python_semantic/resources/mdtest/pep695_type_aliases.md @@ -106,6 +106,29 @@ def _(flag: bool): ```py type ListOrSet[T] = list[T] | set[T] reveal_type(ListOrSet.__type_params__) # revealed: tuple[TypeVar | ParamSpec | TypeVarTuple, ...] +type Tuple1[T] = tuple[T] + +def _(cond: bool): + Generic = ListOrSet if cond else Tuple1 + + def _(x: Generic[int]): + reveal_type(x) # revealed: list[int] | set[int] | tuple[int] + +try: + class Foo[T]: + x: T + def foo(self) -> T: + return self.x + + ... +except Exception: + class Foo[T]: + x: T + def foo(self) -> T: + return self.x + +def f(x: Foo[int]): + reveal_type(x.foo()) # revealed: int ``` ## In unions and intersections @@ -244,6 +267,47 @@ def f(x: IntOr, y: OrInt): reveal_type(x) # revealed: Never if not isinstance(y, int): reveal_type(y) # revealed: Never + +# error: [cyclic-type-alias-definition] "Cyclic definition of `Itself`" +type Itself = Itself + +def foo( + # this is a very strange thing to do, but this is a regression test to ensure it doesn't panic + Itself: Itself, +): + x: Itself + reveal_type(Itself) # revealed: Divergent + +# A type alias defined with invalid recursion behaves as a dynamic type. +foo(42) +foo("hello") + +# error: [cyclic-type-alias-definition] "Cyclic definition of `A`" +type A = B +# error: [cyclic-type-alias-definition] "Cyclic definition of `B`" +type B = A + +def bar(B: B): + x: B + reveal_type(B) # revealed: Divergent + +# error: [cyclic-type-alias-definition] "Cyclic definition of `G`" +type G[T] = G[T] +# error: [cyclic-type-alias-definition] "Cyclic definition of `H`" +type H[T] = I[T] +# error: [cyclic-type-alias-definition] "Cyclic definition of `I`" +type I[T] = H[T] + +# It's not possible to create an element of this type, but it's not an error for now +type DirectRecursiveList[T] = list[DirectRecursiveList[T]] + +# TODO: this should probably be a cyclic-type-alias-definition error +type Foo[T] = list[T] | Bar[T] +type Bar[T] = int | Foo[T] + +def _(x: Bar[int]): + # TODO: should be `int | list[int]` + reveal_type(x) # revealed: int | list[int] | Any ``` ### With legacy generic @@ -327,7 +391,7 @@ class C(P[T]): pass reveal_type(C[int]()) # revealed: C[int] -reveal_type(C()) # revealed: C[Divergent] +reveal_type(C()) # revealed: C[C[Divergent]] ``` ### Union inside generic diff --git a/crates/ty_python_semantic/resources/mdtest/regression/1377_iteration_count_mismatch.md b/crates/ty_python_semantic/resources/mdtest/regression/1377_iteration_count_mismatch.md index 600e408331..309a41422c 100644 --- a/crates/ty_python_semantic/resources/mdtest/regression/1377_iteration_count_mismatch.md +++ b/crates/ty_python_semantic/resources/mdtest/regression/1377_iteration_count_mismatch.md @@ -2,10 +2,7 @@ Regression test for . -The code is an excerpt from that is minimal enough to -trigger the iteration count mismatch bug in Salsa. - - +The code is an excerpt from . ```toml [environment] diff --git a/crates/ty_python_semantic/src/place.rs b/crates/ty_python_semantic/src/place.rs index 4957a53914..589a4720f4 100644 --- a/crates/ty_python_semantic/src/place.rs +++ b/crates/ty_python_semantic/src/place.rs @@ -87,7 +87,7 @@ impl TypeOrigin { /// bound_or_declared: Place::Defined(Literal[1], TypeOrigin::Inferred, Definedness::PossiblyUndefined), /// non_existent: Place::Undefined, /// ``` -#[derive(Debug, Clone, PartialEq, Eq, salsa::Update, get_size2::GetSize)] +#[derive(Debug, Clone, Copy, PartialEq, Eq, salsa::Update, get_size2::GetSize)] pub(crate) enum Place<'db> { Defined(Type<'db>, TypeOrigin, Definedness), Undefined, @@ -532,7 +532,7 @@ impl<'db> PlaceFromDeclarationsResult<'db> { /// that this comes with a [`CLASS_VAR`] type qualifier. /// /// [`CLASS_VAR`]: crate::types::TypeQualifiers::CLASS_VAR -#[derive(Debug, Clone, PartialEq, Eq, salsa::Update, get_size2::GetSize)] +#[derive(Debug, Clone, Copy, PartialEq, Eq, salsa::Update, get_size2::GetSize)] pub(crate) struct PlaceAndQualifiers<'db> { pub(crate) place: Place<'db>, pub(crate) qualifiers: TypeQualifiers, @@ -689,6 +689,51 @@ impl<'db> PlaceAndQualifiers<'db> { .or_else(|lookup_error| lookup_error.or_fall_back_to(db, fallback_fn())) .into() } + + pub(crate) fn cycle_normalized( + self, + db: &'db dyn Db, + previous_place: Self, + cycle: &salsa::Cycle, + ) -> Self { + let place = match (previous_place.place, self.place) { + // In fixed-point iteration of type inference, the member type must be monotonically widened and not "oscillate". + // Here, monotonicity is guaranteed by pre-unioning the type of the previous iteration into the current result. + (Place::Defined(prev_ty, _, _), Place::Defined(ty, origin, definedness)) => { + Place::Defined(ty.cycle_normalized(db, prev_ty, cycle), origin, definedness) + } + // If a `Place` in the current cycle is `Defined` but `Undefined` in the previous cycle, + // that means that its definedness depends on the truthiness of the previous cycle value. + // In this case, the definedness of the current cycle `Place` is set to `PossiblyUndefined`. + // Actually, this branch is unreachable. We evaluate the truthiness of non-definitely-bound places as Ambiguous (see #19579), + // so convergence is guaranteed without resorting to this handling. + // However, the handling described above may reduce the exactness of reachability analysis, + // so it may be better to remove it. In that case, this branch is necessary. + (Place::Undefined, Place::Defined(ty, origin, _definedness)) => Place::Defined( + ty.recursive_type_normalized(db, cycle), + origin, + Definedness::PossiblyUndefined, + ), + // If a `Place` that was `Defined(Divergent)` in the previous cycle is actually found to be unreachable in the current cycle, + // it is set to `Undefined` (because the cycle initial value does not include meaningful reachability information). + (Place::Defined(ty, origin, _definedness), Place::Undefined) => { + if cycle.head_ids().any(|id| ty == Type::divergent(id)) { + Place::Undefined + } else { + Place::Defined( + ty.recursive_type_normalized(db, cycle), + origin, + Definedness::PossiblyUndefined, + ) + } + } + (Place::Undefined, Place::Undefined) => Place::Undefined, + }; + PlaceAndQualifiers { + place, + qualifiers: self.qualifiers, + } + } } impl<'db> From> for PlaceAndQualifiers<'db> { @@ -699,16 +744,30 @@ impl<'db> From> for PlaceAndQualifiers<'db> { fn place_cycle_initial<'db>( _db: &'db dyn Db, - _id: salsa::Id, + id: salsa::Id, _scope: ScopeId<'db>, _place_id: ScopedPlaceId, _requires_explicit_reexport: RequiresExplicitReExport, _considered_definitions: ConsideredDefinitions, ) -> PlaceAndQualifiers<'db> { - Place::bound(Type::Never).into() + Place::bound(Type::divergent(id)).into() } -#[salsa::tracked(cycle_initial=place_cycle_initial, heap_size=ruff_memory_usage::heap_size)] +#[allow(clippy::too_many_arguments)] +fn place_cycle_recover<'db>( + db: &'db dyn Db, + cycle: &salsa::Cycle, + previous_place: &PlaceAndQualifiers<'db>, + place: PlaceAndQualifiers<'db>, + _scope: ScopeId<'db>, + _place_id: ScopedPlaceId, + _requires_explicit_reexport: RequiresExplicitReExport, + _considered_definitions: ConsideredDefinitions, +) -> PlaceAndQualifiers<'db> { + place.cycle_normalized(db, *previous_place, cycle) +} + +#[salsa::tracked(cycle_fn=place_cycle_recover, cycle_initial=place_cycle_initial, heap_size=ruff_memory_usage::heap_size)] pub(crate) fn place_by_id<'db>( db: &'db dyn Db, scope: ScopeId<'db>, diff --git a/crates/ty_python_semantic/src/semantic_model.rs b/crates/ty_python_semantic/src/semantic_model.rs index 24c3f09ecb..3f2a377c20 100644 --- a/crates/ty_python_semantic/src/semantic_model.rs +++ b/crates/ty_python_semantic/src/semantic_model.rs @@ -12,8 +12,7 @@ use crate::module_resolver::{KnownModule, Module, list_modules, resolve_module}; use crate::semantic_index::definition::Definition; use crate::semantic_index::scope::FileScopeId; use crate::semantic_index::semantic_index; -use crate::types::ide_support::all_declarations_and_bindings; -use crate::types::ide_support::{Member, all_members}; +use crate::types::ide_support::{Member, all_declarations_and_bindings, all_members}; use crate::types::{Type, binding_type, infer_scope_types}; /// The primary interface the LSP should use for querying semantic information about a [`File`]. diff --git a/crates/ty_python_semantic/src/types.rs b/crates/ty_python_semantic/src/types.rs index 6194965659..c6b6c540f5 100644 --- a/crates/ty_python_semantic/src/types.rs +++ b/crates/ty_python_semantic/src/types.rs @@ -1,7 +1,6 @@ use compact_str::{CompactString, ToCompactString}; use infer::nearest_enclosing_class; use itertools::{Either, Itertools}; -use ruff_db::parsed::parsed_module; use std::borrow::Cow; use std::time::Duration; @@ -13,6 +12,7 @@ use diagnostic::{INVALID_CONTEXT_MANAGER, NOT_ITERABLE, POSSIBLY_MISSING_IMPLICI use ruff_db::Instant; use ruff_db::diagnostic::{Annotation, Diagnostic, Span, SubDiagnostic, SubDiagnosticSeverity}; use ruff_db::files::File; +use ruff_db::parsed::parsed_module; use ruff_python_ast as ast; use ruff_python_ast::name::Name; use ruff_text_size::{Ranged, TextRange}; @@ -29,7 +29,7 @@ pub(crate) use self::infer::{ TypeContext, infer_deferred_types, infer_definition_types, infer_expression_type, infer_expression_types, infer_scope_types, static_expression_truthiness, }; -pub(crate) use self::signatures::{CallableSignature, Parameter, Parameters, Signature}; +pub(crate) use self::signatures::{CallableSignature, Signature}; pub(crate) use self::subclass_of::{SubclassOfInner, SubclassOfType}; pub use crate::diagnostic::add_inferred_python_version_hint_to_diagnostic; use crate::module_name::ModuleName; @@ -61,16 +61,16 @@ use crate::types::generics::{ InferableTypeVars, PartialSpecialization, Specialization, bind_typevar, typing_self, walk_generic_context, }; -use crate::types::infer::infer_unpack_types; use crate::types::mro::{Mro, MroError, MroIterator}; pub(crate) use crate::types::narrow::infer_narrowing_constraint; use crate::types::newtype::NewType; +pub(crate) use crate::types::signatures::{Parameter, Parameters}; use crate::types::signatures::{ParameterForm, walk_signature}; use crate::types::tuple::{TupleSpec, TupleSpecBuilder}; pub(crate) use crate::types::typed_dict::{TypedDictParams, TypedDictType, walk_typed_dict_type}; pub use crate::types::variance::TypeVarVariance; use crate::types::variance::VarianceInferable; -use crate::types::visitor::{any_over_type, exceeds_max_specialization_depth}; +use crate::types::visitor::any_over_type; use crate::unpack::EvaluationMode; use crate::{Db, FxOrderSet, Module, Program}; pub use class::KnownClass; @@ -387,22 +387,46 @@ impl Default for MemberLookupPolicy { fn member_lookup_cycle_initial<'db>( _db: &'db dyn Db, - _id: salsa::Id, + id: salsa::Id, _self: Type<'db>, _name: Name, _policy: MemberLookupPolicy, ) -> PlaceAndQualifiers<'db> { - Place::bound(Type::Never).into() + Place::bound(Type::divergent(id)).into() +} + +fn member_lookup_cycle_recover<'db>( + db: &'db dyn Db, + cycle: &salsa::Cycle, + previous_member: &PlaceAndQualifiers<'db>, + member: PlaceAndQualifiers<'db>, + _self_type: Type<'db>, + _name: Name, + _policy: MemberLookupPolicy, +) -> PlaceAndQualifiers<'db> { + member.cycle_normalized(db, *previous_member, cycle) } fn class_lookup_cycle_initial<'db>( _db: &'db dyn Db, - _id: salsa::Id, + id: salsa::Id, _self: Type<'db>, _name: Name, _policy: MemberLookupPolicy, ) -> PlaceAndQualifiers<'db> { - Place::bound(Type::Never).into() + Place::bound(Type::divergent(id)).into() +} + +fn class_lookup_cycle_recover<'db>( + db: &'db dyn Db, + cycle: &salsa::Cycle, + previous_member: &PlaceAndQualifiers<'db>, + member: PlaceAndQualifiers<'db>, + _self_type: Type<'db>, + _name: Name, + _policy: MemberLookupPolicy, +) -> PlaceAndQualifiers<'db> { + member.cycle_normalized(db, *previous_member, cycle) } fn variance_cycle_initial<'db, T>( @@ -543,6 +567,32 @@ impl<'db> PropertyInstanceType<'db> { ) } + fn recursive_type_normalized_impl( + self, + db: &'db dyn Db, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> Option { + let getter = match self.getter(db) { + Some(ty) if nested => Some(ty.recursive_type_normalized_impl(db, div, true, visitor)?), + Some(ty) => Some( + ty.recursive_type_normalized_impl(db, div, true, visitor) + .unwrap_or(div), + ), + None => None, + }; + let setter = match self.setter(db) { + Some(ty) if nested => Some(ty.recursive_type_normalized_impl(db, div, true, visitor)?), + Some(ty) => Some( + ty.recursive_type_normalized_impl(db, div, true, visitor) + .unwrap_or(div), + ), + None => None, + }; + Some(Self::new(db, getter, setter)) + } + fn find_legacy_typevars_impl( self, db: &'db dyn Db, @@ -712,7 +762,7 @@ impl<'db> DataclassParams<'db> { #[derive(Copy, Clone, Debug, PartialEq, Eq, Hash, salsa::Update, get_size2::GetSize)] pub enum Type<'db> { /// The dynamic type: a statically unknown set of values - Dynamic(DynamicType<'db>), + Dynamic(DynamicType), /// The empty set of values Never, /// A specific function object @@ -829,8 +879,8 @@ impl<'db> Type<'db> { Self::Dynamic(DynamicType::Unknown) } - pub(crate) fn divergent(scope: Option>) -> Self { - Self::Dynamic(DynamicType::Divergent(DivergentType { scope })) + pub(crate) fn divergent(id: salsa::Id) -> Self { + Self::Dynamic(DynamicType::Divergent(DivergentType { id })) } pub(crate) const fn is_divergent(&self) -> bool { @@ -850,6 +900,16 @@ impl<'db> Type<'db> { matches!(self, Type::Callable(..)) } + pub(crate) fn cycle_normalized( + self, + db: &'db dyn Db, + previous: Self, + cycle: &salsa::Cycle, + ) -> Self { + UnionType::from_elements_cycle_recovery(db, [self, previous]) + .recursive_type_normalized(db, cycle) + } + fn is_none(&self, db: &'db dyn Db) -> bool { self.is_instance_of(db, KnownClass::NoneType) } @@ -1080,7 +1140,7 @@ impl<'db> Type<'db> { } } - pub(crate) const fn as_dynamic(self) -> Option> { + pub(crate) const fn as_dynamic(self) -> Option { match self { Type::Dynamic(dynamic_type) => Some(dynamic_type), _ => None, @@ -1094,7 +1154,7 @@ impl<'db> Type<'db> { } } - pub(crate) const fn expect_dynamic(self) -> DynamicType<'db> { + pub(crate) const fn expect_dynamic(self) -> DynamicType { self.as_dynamic().expect("Expected a Type::Dynamic variant") } @@ -1475,6 +1535,164 @@ impl<'db> Type<'db> { } } + /// Performs nest reduction for recursive types (types that contain `Divergent` types). + /// For example, consider the following implicit attribute inference: + /// ```python + /// class C: + /// def f(self, other: "C"): + /// self.x = (other.x, 1) + /// + /// reveal_type(C().x) # revealed: Unknown | tuple[Divergent, Literal[1]] + /// ``` + /// + /// A query that performs implicit attribute type inference enters a cycle because the attribute is recursively defined, and the cycle initial value is set to `Divergent`. + /// In the next (1st) cycle it is inferred to be `tuple[Divergent, Literal[1]]`, and in the 2nd cycle it becomes `tuple[tuple[Divergent, Literal[1]], Literal[1]]`. + /// If this continues, the query will not converge, so this method is called in the cycle recovery function. + /// Then `tuple[tuple[Divergent, Literal[1]], Literal[1]]` is replaced with `tuple[Divergent, Literal[1]]` and the query converges. + #[must_use] + pub(crate) fn recursive_type_normalized(self, db: &'db dyn Db, cycle: &salsa::Cycle) -> Self { + cycle.head_ids().fold(self, |ty, id| { + let visitor = NormalizedVisitor::new(Type::divergent(id)); + ty.recursive_type_normalized_impl(db, Type::divergent(id), false, &visitor) + .unwrap_or(Type::divergent(id)) + }) + } + + /// Normalizes types including divergent types (recursive types), which is necessary for convergence of fixed-point iteration. + /// When nested is true, propagate `None`. That is, if the type contains a `Divergent` type, the return value of this method is `None`. + /// When nested is false, create a type containing `Divergent` types instead of propagating `None`. + /// This is to preserve the structure of the non-divergent parts of the type instead of completely collapsing the type containing a `Divergent` type into a `Divergent` type. + /// ```python + /// tuple[tuple[Divergent, Literal[1]], Literal[1]].recursive_type_normalized(nested: false) + /// => tuple[ + /// tuple[Divergent, Literal[1]].recursive_type_normalized_impl(nested: true).unwrap_or(Divergent), + /// Literal[1].recursive_type_normalized_impl(nested: true).unwrap_or(Divergent) + /// ] + /// => tuple[Divergent, Literal[1]] + /// ``` + /// Generic nominal types such as `list[T]` and `tuple[T]` should send `nested=true` for `T`. This is necessary for normalization. + /// Structural types such as union and intersection do not need to send `nested=true` for element types; that is, types that are "flat" from the perspective of recursive types. `T | U` should send `nested` as is for `T`, `U`. + /// For other types, the decision depends on whether they are interpreted as nominal or structural. + /// For example, `KnownInstanceType::UnionType` should simply send `nested` as is. + fn recursive_type_normalized_impl( + self, + db: &'db dyn Db, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> Option { + if nested && self == div { + return None; + } + match self { + Type::Union(union) => visitor.try_visit(self, || { + union.recursive_type_normalized_impl(db, div, nested, visitor) + }), + Type::Intersection(intersection) => visitor.try_visit(self, || { + intersection + .recursive_type_normalized_impl(db, div, nested, visitor) + .map(Type::Intersection) + }), + Type::Callable(callable) => visitor.try_visit(self, || { + callable + .recursive_type_normalized_impl(db, div, nested, visitor) + .map(Type::Callable) + }), + Type::ProtocolInstance(protocol) => visitor.try_visit(self, || { + protocol + .recursive_type_normalized_impl(db, div, nested, visitor) + .map(Type::ProtocolInstance) + }), + Type::NominalInstance(instance) => visitor.try_visit(self, || { + instance + .recursive_type_normalized_impl(db, div, nested, visitor) + .map(Type::NominalInstance) + }), + Type::FunctionLiteral(function) => visitor.try_visit(self, || { + function + .recursive_type_normalized_impl(db, div, nested, visitor) + .map(Type::FunctionLiteral) + }), + Type::PropertyInstance(property) => visitor.try_visit(self, || { + property + .recursive_type_normalized_impl(db, div, nested, visitor) + .map(Type::PropertyInstance) + }), + Type::KnownBoundMethod(method_kind) => visitor.try_visit(self, || { + method_kind + .recursive_type_normalized_impl(db, div, nested, visitor) + .map(Type::KnownBoundMethod) + }), + Type::BoundMethod(method) => visitor.try_visit(self, || { + method + .recursive_type_normalized_impl(db, div, nested, visitor) + .map(Type::BoundMethod) + }), + Type::BoundSuper(bound_super) => visitor.try_visit(self, || { + bound_super + .recursive_type_normalized_impl(db, div, nested, visitor) + .map(Type::BoundSuper) + }), + Type::GenericAlias(generic) => visitor.try_visit(self, || { + generic + .recursive_type_normalized_impl(db, div, nested, visitor) + .map(Type::GenericAlias) + }), + Type::SubclassOf(subclass_of) => visitor.try_visit(self, || { + subclass_of + .recursive_type_normalized_impl(db, div, nested, visitor) + .map(Type::SubclassOf) + }), + Type::TypeVar(_) => Some(self), + Type::KnownInstance(known_instance) => visitor.try_visit(self, || { + known_instance + .recursive_type_normalized_impl(db, div, nested, visitor) + .map(Type::KnownInstance) + }), + Type::TypeIs(type_is) => visitor.try_visit(self, || { + let ty = if nested { + type_is + .return_type(db) + .recursive_type_normalized_impl(db, div, true, visitor)? + } else { + type_is + .return_type(db) + .recursive_type_normalized_impl(db, div, true, visitor) + .unwrap_or(div) + }; + Some(type_is.with_type(db, ty)) + }), + Type::Dynamic(dynamic) => Some(Type::Dynamic(dynamic.recursive_type_normalized())), + Type::TypedDict(_) => { + // TODO: Normalize TypedDicts + Some(self) + } + Type::TypeAlias(_) => Some(self), + Type::NewTypeInstance(newtype) => visitor.try_visit(self, || { + newtype + .try_map_base_class_type(db, |class_type| { + class_type.recursive_type_normalized_impl(db, div, nested, visitor) + }) + .map(Type::NewTypeInstance) + }), + Type::LiteralString + | Type::AlwaysFalsy + | Type::AlwaysTruthy + | Type::BooleanLiteral(_) + | Type::BytesLiteral(_) + | Type::EnumLiteral(_) + | Type::StringLiteral(_) + | Type::Never + | Type::WrapperDescriptor(_) + | Type::DataclassDecorator(_) + | Type::DataclassTransformer(_) + | Type::ModuleLiteral(_) + | Type::ClassLiteral(_) + | Type::SpecialForm(_) + | Type::IntLiteral(_) => Some(self), + } + } + /// Return `true` if subtyping is always reflexive for this type; `T <: T` is always true for /// any `T` of this type. /// @@ -3912,7 +4130,7 @@ impl<'db> Type<'db> { self.class_member_with_policy(db, name, MemberLookupPolicy::default()) } - #[salsa::tracked(cycle_initial=class_lookup_cycle_initial, heap_size=ruff_memory_usage::heap_size)] + #[salsa::tracked(cycle_fn=class_lookup_cycle_recover, cycle_initial=class_lookup_cycle_initial, heap_size=ruff_memory_usage::heap_size)] fn class_member_with_policy( self, db: &'db dyn Db, @@ -4380,7 +4598,7 @@ impl<'db> Type<'db> { /// Similar to [`Type::member`], but allows the caller to specify what policy should be used /// when looking up attributes. See [`MemberLookupPolicy`] for more information. - #[salsa::tracked(cycle_initial=member_lookup_cycle_initial, heap_size=ruff_memory_usage::heap_size)] + #[salsa::tracked(cycle_fn=member_lookup_cycle_recover, cycle_initial=member_lookup_cycle_initial, heap_size=ruff_memory_usage::heap_size)] pub(crate) fn member_lookup_with_policy( self, db: &'db dyn Db, @@ -4786,7 +5004,7 @@ impl<'db> Type<'db> { let owner_attr = bound_super.find_name_in_mro_after_pivot(db, name_str, policy); bound_super - .try_call_dunder_get_on_attribute(db, owner_attr.clone()) + .try_call_dunder_get_on_attribute(db, owner_attr) .unwrap_or(owner_attr) } } @@ -7056,7 +7274,6 @@ impl<'db> Type<'db> { .unwrap_or(SubclassOfInner::unknown()), ), }, - Type::StringLiteral(_) | Type::LiteralString => KnownClass::Str.to_class_literal(db), Type::Dynamic(dynamic) => SubclassOfType::from(db, SubclassOfInner::Dynamic(dynamic)), // TODO intersections @@ -7113,7 +7330,7 @@ impl<'db> Type<'db> { /// Note that this does not specialize generic classes, functions, or type aliases! That is a /// different operation that is performed explicitly (via a subscript operation), or implicitly /// via a call to the generic object. - #[salsa::tracked(heap_size=ruff_memory_usage::heap_size, cycle_initial=apply_specialization_cycle_initial)] + #[salsa::tracked(heap_size=ruff_memory_usage::heap_size, cycle_fn=apply_specialization_cycle_recover, cycle_initial=apply_specialization_cycle_initial)] pub(crate) fn apply_specialization( self, db: &'db dyn Db, @@ -7162,7 +7379,7 @@ impl<'db> Type<'db> { match self { Type::TypeVar(bound_typevar) => match type_mapping { TypeMapping::Specialization(specialization) => { - specialization.get(db, bound_typevar).unwrap_or(self).fallback_to_divergent(db) + specialization.get(db, bound_typevar).unwrap_or(self) } TypeMapping::PartialSpecialization(partial) => { partial.get(db, bound_typevar).unwrap_or(self) @@ -7189,7 +7406,8 @@ impl<'db> Type<'db> { } TypeMapping::PromoteLiterals(_) | TypeMapping::ReplaceParameterDefaults - | TypeMapping::BindLegacyTypevars(_) => self, + | TypeMapping::BindLegacyTypevars(_) + | TypeMapping::EagerExpansion => self, TypeMapping::Materialize(materialization_kind) => { Type::TypeVar(bound_typevar.materialize_impl(db, *materialization_kind, visitor)) } @@ -7205,7 +7423,8 @@ impl<'db> Type<'db> { TypeMapping::BindSelf(_) | TypeMapping::ReplaceSelf { .. } | TypeMapping::Materialize(_) | - TypeMapping::ReplaceParameterDefaults => self, + TypeMapping::ReplaceParameterDefaults | + TypeMapping::EagerExpansion => self, } Type::FunctionLiteral(function) => { @@ -7306,6 +7525,9 @@ impl<'db> Type<'db> { Type::TypeIs(type_is) => type_is.with_type(db, type_is.return_type(db).apply_type_mapping(db, type_mapping, tcx)), Type::TypeAlias(alias) => { + if TypeMapping::EagerExpansion == *type_mapping { + return alias.raw_value_type(db).expand_eagerly(db); + } // Do not call `value_type` here. `value_type` does the specialization internally, so `apply_type_mapping` is performed without `visitor` inheritance. // In the case of recursive type aliases, this leads to infinite recursion. // Instead, call `raw_value_type` and perform the specialization after the `visitor` cache has been created. @@ -7327,6 +7549,7 @@ impl<'db> Type<'db> { TypeMapping::ReplaceSelf { .. } | TypeMapping::Materialize(_) | TypeMapping::ReplaceParameterDefaults | + TypeMapping::EagerExpansion | TypeMapping::PromoteLiterals(PromoteLiteralsMode::Off) => self, TypeMapping::PromoteLiterals(PromoteLiteralsMode::On) => self.promote_literals_impl(db, tcx) } @@ -7338,7 +7561,8 @@ impl<'db> Type<'db> { TypeMapping::BindSelf(_) | TypeMapping::ReplaceSelf { .. } | TypeMapping::PromoteLiterals(_) | - TypeMapping::ReplaceParameterDefaults => self, + TypeMapping::ReplaceParameterDefaults | + TypeMapping::EagerExpansion => self, TypeMapping::Materialize(materialization_kind) => match materialization_kind { MaterializationKind::Top => Type::object(), MaterializationKind::Bottom => Type::Never, @@ -7545,6 +7769,18 @@ impl<'db> Type<'db> { ) } + /// Returns the eagerly expanded type. + /// In the case of recursive type aliases, this will diverge, so that part will be replaced with `Divergent`. + fn expand_eagerly(self, db: &'db dyn Db) -> Type<'db> { + self.expand_eagerly_(db, ()) + } + + #[allow(clippy::used_underscore_binding)] + #[salsa::tracked(heap_size=ruff_memory_usage::heap_size, cycle_fn=expand_eagerly_cycle_recover, cycle_initial=expand_eagerly_cycle_initial)] + fn expand_eagerly_(self, db: &'db dyn Db, _unit: ()) -> Type<'db> { + self.apply_type_mapping(db, &TypeMapping::EagerExpansion, TypeContext::default()) + } + /// Return the string representation of this type when converted to string as it would be /// provided by the `__str__` method. /// @@ -7757,20 +7993,6 @@ impl<'db> Type<'db> { _ => None, } } - - pub(super) fn has_divergent_type(self, db: &'db dyn Db, div: Type<'db>) -> bool { - any_over_type(db, self, &|ty| ty == div, false) - } - - /// If the specialization depth of `self` exceeds the maximum limit allowed, - /// return `Divergent`. Otherwise, return `self`. - pub(super) fn fallback_to_divergent(self, db: &'db dyn Db) -> Type<'db> { - if exceeds_max_specialization_depth(db, self) { - Type::divergent(None) - } else { - self - } - } } impl<'db> From<&Type<'db>> for Type<'db> { @@ -7889,13 +8111,44 @@ fn is_redundant_with_cycle_initial<'db>( true } -fn apply_specialization_cycle_initial<'db>( - _db: &'db dyn Db, - _id: salsa::Id, +fn apply_specialization_cycle_recover<'db>( + db: &'db dyn Db, + cycle: &salsa::Cycle, + previous_value: &Type<'db>, + value: Type<'db>, _self: Type<'db>, _specialization: Specialization<'db>, ) -> Type<'db> { - Type::Never + value.cycle_normalized(db, *previous_value, cycle) +} + +fn apply_specialization_cycle_initial<'db>( + _db: &'db dyn Db, + id: salsa::Id, + _self: Type<'db>, + _specialization: Specialization<'db>, +) -> Type<'db> { + Type::divergent(id) +} + +fn expand_eagerly_cycle_initial<'db>( + _db: &'db dyn Db, + id: salsa::Id, + _self: Type<'db>, + _unit: (), +) -> Type<'db> { + Type::divergent(id) +} + +fn expand_eagerly_cycle_recover<'db>( + db: &'db dyn Db, + cycle: &salsa::Cycle, + previous_value: &Type<'db>, + value: Type<'db>, + _self: Type<'db>, + _unit: (), +) -> Type<'db> { + value.cycle_normalized(db, *previous_value, cycle) } #[derive(Clone, Copy, Debug, Eq, Hash, PartialEq, get_size2::GetSize)] @@ -7940,6 +8193,9 @@ pub enum TypeMapping<'a, 'db> { /// Replace default types in parameters of callables with `Unknown`. This is used to avoid infinite /// recursion when the type of the default value of a parameter depends on the callable itself. ReplaceParameterDefaults, + /// Apply eager expansion to the type. + /// In the case of recursive type aliases, this will diverge, so that part will be replaced with `Divergent`. + EagerExpansion, } impl<'db> TypeMapping<'_, 'db> { @@ -7955,7 +8211,8 @@ impl<'db> TypeMapping<'_, 'db> { | TypeMapping::PromoteLiterals(_) | TypeMapping::BindLegacyTypevars(_) | TypeMapping::Materialize(_) - | TypeMapping::ReplaceParameterDefaults => context, + | TypeMapping::ReplaceParameterDefaults + | TypeMapping::EagerExpansion => context, TypeMapping::BindSelf(_) => GenericContext::from_typevar_instances( db, context @@ -7991,7 +8248,8 @@ impl<'db> TypeMapping<'_, 'db> { | TypeMapping::BindLegacyTypevars(_) | TypeMapping::BindSelf(_) | TypeMapping::ReplaceSelf { .. } - | TypeMapping::ReplaceParameterDefaults => self.clone(), + | TypeMapping::ReplaceParameterDefaults + | TypeMapping::EagerExpansion => self.clone(), } } } @@ -8184,6 +8442,56 @@ impl<'db> KnownInstanceType<'db> { } } + fn recursive_type_normalized_impl( + self, + db: &'db dyn Db, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> Option { + match self { + // Nothing to normalize + Self::SubscriptedProtocol(context) => Some(Self::SubscriptedProtocol(context)), + Self::SubscriptedGeneric(context) => Some(Self::SubscriptedGeneric(context)), + Self::Deprecated(deprecated) => Some(Self::Deprecated(deprecated)), + Self::ConstraintSet(set) => Some(Self::ConstraintSet(set)), + Self::TypeVar(typevar) => Some(Self::TypeVar(typevar)), + Self::TypeAliasType(type_alias) => type_alias + .recursive_type_normalized_impl(db, div, visitor) + .map(Self::TypeAliasType), + Self::Field(field) => field + .recursive_type_normalized_impl(db, div, nested, visitor) + .map(Self::Field), + Self::UnionType(union_type) => union_type + .recursive_type_normalized_impl(db, div, nested, visitor) + .map(Self::UnionType), + Self::Literal(ty) => ty + .recursive_type_normalized_impl(db, div, true, visitor) + .map(Self::Literal), + Self::Annotated(ty) => ty + .recursive_type_normalized_impl(db, div, true, visitor) + .map(Self::Annotated), + Self::TypeGenericAlias(ty) => ty + .recursive_type_normalized_impl(db, div, true, visitor) + .map(Self::TypeGenericAlias), + Self::LiteralStringAlias(ty) => ty + .recursive_type_normalized_impl(db, div, true, visitor) + .map(Self::LiteralStringAlias), + Self::Callable(callable) => callable + .recursive_type_normalized_impl(db, div, nested, visitor) + .map(Self::Callable), + Self::NewType(newtype) => newtype + .try_map_base_class_type(db, |class_type| { + class_type.recursive_type_normalized_impl(db, div, true, visitor) + }) + .map(Self::NewType), + Self::GenericContext(generic) => Some(Self::GenericContext(generic)), + Self::Specialization(specialization) => specialization + .recursive_type_normalized_impl(db, div, true, visitor) + .map(Self::Specialization), + } + } + fn class(self, db: &'db dyn Db) -> KnownClass { match self { Self::SubscriptedProtocol(_) | Self::SubscriptedGeneric(_) => KnownClass::SpecialForm, @@ -8239,14 +8547,17 @@ impl<'db> KnownInstanceType<'db> { /// (e.g. `Divergent` is assignable to `@Todo`, but `@Todo | Divergent` must not be reducted to `@Todo`). /// Otherwise, type inference cannot converge properly. /// For detailed properties of this type, see the unit test at the end of the file. -#[derive(Copy, Clone, Debug, Eq, Hash, PartialEq, salsa::Update, get_size2::GetSize)] -pub struct DivergentType<'db> { - /// The scope where this divergence was detected. - scope: Option>, +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord, salsa::Update)] +pub struct DivergentType { + /// The query ID that caused the cycle. + id: salsa::Id, } +// The Salsa heap is tracked separately. +impl get_size2::GetSize for DivergentType {} + #[derive(Copy, Clone, Debug, Eq, Hash, PartialEq, salsa::Update, get_size2::GetSize)] -pub enum DynamicType<'db> { +pub enum DynamicType { /// An explicitly annotated `typing.Any` Any, /// An unannotated value, or a dynamic type resulting from an error @@ -8263,11 +8574,11 @@ pub enum DynamicType<'db> { Todo(TodoType), /// A special Todo-variant for `Unpack[Ts]`, so that we can treat it specially in `Generic[Unpack[Ts]]` TodoUnpack, - /// A type that is determined to be divergent during type inference for a recursive function. - Divergent(DivergentType<'db>), + /// A type that is determined to be divergent during recursive type inference. + Divergent(DivergentType), } -impl DynamicType<'_> { +impl DynamicType { fn normalized(self) -> Self { if matches!(self, Self::Divergent(_)) { self @@ -8276,12 +8587,16 @@ impl DynamicType<'_> { } } + fn recursive_type_normalized(self) -> Self { + self + } + pub(crate) fn is_todo(&self) -> bool { matches!(self, Self::Todo(_) | Self::TodoUnpack) } } -impl std::fmt::Display for DynamicType<'_> { +impl std::fmt::Display for DynamicType { fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { match self { DynamicType::Any => f.write_str("Any"), @@ -8398,6 +8713,17 @@ impl<'db> TypeAndQualifiers<'db> { pub(crate) fn qualifiers(&self) -> TypeQualifiers { self.qualifiers } + + pub(crate) fn map_type( + &self, + f: impl FnOnce(Type<'db>) -> Type<'db>, + ) -> TypeAndQualifiers<'db> { + TypeAndQualifiers { + inner: f(self.inner), + origin: self.origin, + qualifiers: self.qualifiers, + } + } } /// Error struct providing information on type(s) that were deemed to be invalid @@ -8633,6 +8959,33 @@ impl<'db> FieldInstance<'db> { self.alias(db), ) } + + fn recursive_type_normalized_impl( + self, + db: &'db dyn Db, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> Option { + let default_type = match self.default_type(db) { + Some(default) if nested => { + Some(default.recursive_type_normalized_impl(db, div, true, visitor)?) + } + Some(default) => Some( + default + .recursive_type_normalized_impl(db, div, true, visitor) + .unwrap_or(div), + ), + None => None, + }; + Some(FieldInstance::new( + db, + default_type, + self.init(db), + self.kw_only(db), + self.alias(db), + )) + } } /// Whether this typevar was created via the legacy `TypeVar` constructor, using PEP 695 syntax, @@ -8991,7 +9344,7 @@ impl<'db> TypeVarInstance<'db> { Some(TypeVarBoundOrConstraints::Constraints(ty)) } - #[salsa::tracked(cycle_initial=lazy_default_cycle_initial, heap_size=ruff_memory_usage::heap_size)] + #[salsa::tracked(cycle_fn=lazy_default_cycle_recover, cycle_initial=lazy_default_cycle_initial, heap_size=ruff_memory_usage::heap_size)] fn lazy_default(self, db: &'db dyn Db) -> Option> { let definition = self.definition(db)?; let module = parsed_module(db, definition.file(db)).load(db); @@ -9048,12 +9401,28 @@ fn lazy_bound_or_constraints_cycle_initial<'db>( None } +#[allow(clippy::ref_option)] +fn lazy_default_cycle_recover<'db>( + db: &'db dyn Db, + cycle: &salsa::Cycle, + previous_default: &Option>, + default: Option>, + _typevar: TypeVarInstance<'db>, +) -> Option> { + match (previous_default, default) { + (Some(prev), Some(default)) => Some(default.cycle_normalized(db, *prev, cycle)), + (None, Some(default)) => Some(default.recursive_type_normalized(db, cycle)), + (_, None) => None, + } +} + +#[allow(clippy::unnecessary_wraps)] fn lazy_default_cycle_initial<'db>( _db: &'db dyn Db, - _id: salsa::Id, + id: salsa::Id, _self: TypeVarInstance<'db>, ) -> Option> { - None + Some(Type::divergent(id)) } /// Where a type variable is bound and usable. @@ -9184,13 +9553,9 @@ impl<'db> BoundTypeVarInstance<'db> { match self.typevar(db).explicit_variance(db) { Some(explicit_variance) => explicit_variance.compose(polarity), None => match self.binding_context(db) { - BindingContext::Definition(definition) => { - let type_inference = infer_definition_types(db, definition); - type_inference - .binding_type(definition) - .with_polarity(polarity) - .variance_of(db, self) - } + BindingContext::Definition(definition) => binding_type(db, definition) + .with_polarity(polarity) + .variance_of(db, self), BindingContext::Synthetic => TypeVarVariance::Invariant, }, } @@ -9488,6 +9853,44 @@ impl<'db> UnionTypeInstance<'db> { Self::new(db, value_expr_types, union_type) } + + fn recursive_type_normalized_impl( + self, + db: &'db dyn Db, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> Option { + // The `Divergent` elimination rules are different within union types. + // See `UnionType::recursive_type_normalized_impl` for details. + let value_expr_types = match self._value_expr_types(db).as_ref() { + Some(types) if nested => Some( + types + .iter() + .map(|ty| ty.recursive_type_normalized_impl(db, div, nested, visitor)) + .collect::>>()?, + ), + Some(types) => Some( + types + .iter() + .map(|ty| { + ty.recursive_type_normalized_impl(db, div, nested, visitor) + .unwrap_or(div) + }) + .collect::>(), + ), + None => None, + }; + let union_type = match self.union_type(db).clone() { + Ok(ty) if nested => Ok(ty.recursive_type_normalized_impl(db, div, nested, visitor)?), + Ok(ty) => Ok(ty + .recursive_type_normalized_impl(db, div, nested, visitor) + .unwrap_or(div)), + Err(err) => Err(err), + }; + + Some(Self::new(db, value_expr_types, union_type)) + } } /// A salsa-interned `Type` @@ -9507,6 +9910,24 @@ impl<'db> InternedType<'db> { pub(crate) fn normalized_impl(self, db: &'db dyn Db, visitor: &NormalizedVisitor<'db>) -> Self { InternedType::new(db, self.inner(db).normalized_impl(db, visitor)) } + + fn recursive_type_normalized_impl( + self, + db: &'db dyn Db, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> Option { + let inner = if nested { + self.inner(db) + .recursive_type_normalized_impl(db, div, nested, visitor)? + } else { + self.inner(db) + .recursive_type_normalized_impl(db, div, nested, visitor) + .unwrap_or(div) + }; + Some(InternedType::new(db, inner)) + } } /// Error returned if a type is not awaitable. @@ -10809,6 +11230,22 @@ impl<'db> BoundMethodType<'db> { ) } + fn recursive_type_normalized_impl( + self, + db: &'db dyn Db, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> Option { + Some(Self::new( + db, + self.function(db) + .recursive_type_normalized_impl(db, div, nested, visitor)?, + self.self_instance(db) + .recursive_type_normalized_impl(db, div, true, visitor)?, + )) + } + fn has_relation_to_impl( self, db: &'db dyn Db, @@ -10965,6 +11402,21 @@ impl<'db> CallableType<'db> { ) } + fn recursive_type_normalized_impl( + self, + db: &'db dyn Db, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> Option { + Some(CallableType::new( + db, + self.signatures(db) + .recursive_type_normalized_impl(db, div, nested, visitor)?, + self.is_function_like(db), + )) + } + fn apply_type_mapping_impl<'a>( self, db: &'db dyn Db, @@ -11393,6 +11845,45 @@ impl<'db> KnownBoundMethodType<'db> { } } + fn recursive_type_normalized_impl( + self, + db: &'db dyn Db, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> Option { + match self { + KnownBoundMethodType::FunctionTypeDunderGet(function) => { + Some(KnownBoundMethodType::FunctionTypeDunderGet( + function.recursive_type_normalized_impl(db, div, nested, visitor)?, + )) + } + KnownBoundMethodType::FunctionTypeDunderCall(function) => { + Some(KnownBoundMethodType::FunctionTypeDunderCall( + function.recursive_type_normalized_impl(db, div, nested, visitor)?, + )) + } + KnownBoundMethodType::PropertyDunderGet(property) => { + Some(KnownBoundMethodType::PropertyDunderGet( + property.recursive_type_normalized_impl(db, div, nested, visitor)?, + )) + } + KnownBoundMethodType::PropertyDunderSet(property) => { + Some(KnownBoundMethodType::PropertyDunderSet( + property.recursive_type_normalized_impl(db, div, nested, visitor)?, + )) + } + KnownBoundMethodType::StrStartswith(_) + | KnownBoundMethodType::ConstraintSetRange + | KnownBoundMethodType::ConstraintSetAlways + | KnownBoundMethodType::ConstraintSetNever + | KnownBoundMethodType::ConstraintSetImpliesSubtypeOf(_) + | KnownBoundMethodType::ConstraintSetSatisfies(_) + | KnownBoundMethodType::ConstraintSetSatisfiedByAllTypeVars(_) + | KnownBoundMethodType::GenericContextSpecializeConstrained(_) => Some(self), + } + } + /// Return the [`KnownClass`] that inhabitants of this type are instances of at runtime fn class(self) -> KnownClass { match self { @@ -11868,8 +12359,9 @@ impl<'db> PEP695TypeAliasType<'db> { } /// The RHS type of a PEP-695 style type alias with *no* specialization applied. - #[salsa::tracked(cycle_initial=value_type_cycle_initial, heap_size=ruff_memory_usage::heap_size)] - pub(crate) fn raw_value_type(self, db: &'db dyn Db) -> Type<'db> { + /// Returns `Divergent` if the type alias is defined cyclically. + #[salsa::tracked(cycle_fn=value_type_cycle_recover, cycle_initial=value_type_cycle_initial, heap_size=ruff_memory_usage::heap_size)] + fn raw_value_type(self, db: &'db dyn Db) -> Type<'db> { let scope = self.rhs_scope(db); let module = parsed_module(db, scope.file(db)).load(db); let type_alias_stmt_node = scope.node(db).expect_type_alias(); @@ -11951,10 +12443,20 @@ fn generic_context_cycle_initial<'db>( fn value_type_cycle_initial<'db>( _db: &'db dyn Db, - _id: salsa::Id, + id: salsa::Id, _self: PEP695TypeAliasType<'db>, ) -> Type<'db> { - Type::Never + Type::divergent(id) +} + +fn value_type_cycle_recover<'db>( + db: &'db dyn Db, + cycle: &salsa::Cycle, + previous_value: &Type<'db>, + value: Type<'db>, + _self: PEP695TypeAliasType<'db>, +) -> Type<'db> { + value.cycle_normalized(db, *previous_value, cycle) } /// A PEP 695 `types.TypeAliasType` created by manually calling the constructor. @@ -11991,6 +12493,21 @@ impl<'db> ManualPEP695TypeAliasType<'db> { self.value(db).normalized_impl(db, visitor), ) } + + fn recursive_type_normalized_impl( + self, + db: &'db dyn Db, + div: Type<'db>, + visitor: &NormalizedVisitor<'db>, + ) -> Option { + Some(Self::new( + db, + self.name(db), + self.definition(db), + self.value(db) + .recursive_type_normalized_impl(db, div, true, visitor)?, + )) + } } #[derive( @@ -12033,6 +12550,20 @@ impl<'db> TypeAliasType<'db> { } } + fn recursive_type_normalized_impl( + self, + db: &'db dyn Db, + div: Type<'db>, + visitor: &NormalizedVisitor<'db>, + ) -> Option { + match self { + TypeAliasType::PEP695(type_alias) => Some(TypeAliasType::PEP695(type_alias)), + TypeAliasType::ManualPEP695(type_alias) => Some(TypeAliasType::ManualPEP695( + type_alias.recursive_type_normalized_impl(db, div, visitor)?, + )), + } + } + pub(crate) fn name(self, db: &'db dyn Db) -> &'db str { match self { TypeAliasType::PEP695(type_alias) => type_alias.name(db), @@ -12162,6 +12693,20 @@ impl<'db> UnionType<'db> { .build() } + fn from_elements_cycle_recovery(db: &'db dyn Db, elements: I) -> Type<'db> + where + I: IntoIterator, + T: Into>, + { + elements + .into_iter() + .fold( + UnionBuilder::new(db).cycle_recovery(true), + |builder, element| builder.add(element.into()), + ) + .build() + } + /// A fallible version of [`UnionType::from_elements`]. /// /// If all items in `elements` are `Some()`, the result of unioning all elements is returned. @@ -12334,6 +12879,46 @@ impl<'db> UnionType<'db> { .build() } + fn recursive_type_normalized_impl( + self, + db: &'db dyn Db, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> Option> { + let mut builder = UnionBuilder::new(db) + .order_elements(false) + .unpack_aliases(false) + .cycle_recovery(true); + let mut empty = true; + for ty in self.elements(db) { + if nested { + // list[T | Divergent] => list[Divergent] + let ty = ty.recursive_type_normalized_impl(db, div, nested, visitor)?; + if ty == div { + return Some(ty); + } + builder = builder.add(ty); + empty = false; + } else { + // `Divergent` in a union type does not mean true divergence, so we skip it if not nested. + // e.g. T | Divergent == T | (T | (T | (T | ...))) == T + if ty == &div { + continue; + } + builder = builder.add( + ty.recursive_type_normalized_impl(db, div, nested, visitor) + .unwrap_or(div), + ); + empty = false; + } + } + if empty { + builder = builder.add(div); + } + Some(builder.build()) + } + pub(crate) fn is_equivalent_to_impl( self, db: &'db dyn Db, @@ -12435,6 +13020,56 @@ impl<'db> IntersectionType<'db> { ) } + pub(crate) fn recursive_type_normalized_impl( + self, + db: &'db dyn Db, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> Option { + fn opt_normalized_set<'db>( + db: &'db dyn Db, + elements: &FxOrderSet>, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> Option>> { + elements + .iter() + .map(|ty| ty.recursive_type_normalized_impl(db, div, nested, visitor)) + .collect() + } + + fn normalized_set<'db>( + db: &'db dyn Db, + elements: &FxOrderSet>, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> FxOrderSet> { + elements + .iter() + .map(|ty| { + ty.recursive_type_normalized_impl(db, div, nested, visitor) + .unwrap_or(div) + }) + .collect() + } + + let positive = if nested { + opt_normalized_set(db, self.positive(db), div, nested, visitor)? + } else { + normalized_set(db, self.positive(db), div, nested, visitor) + }; + let negative = if nested { + opt_normalized_set(db, self.negative(db), div, nested, visitor)? + } else { + normalized_set(db, self.negative(db), div, nested, visitor) + }; + + Some(IntersectionType::new(db, positive, negative)) + } + /// Return `true` if `self` represents exactly the same set of possible runtime objects as `other` pub(crate) fn is_equivalent_to_impl( self, @@ -12766,8 +13401,6 @@ pub(crate) mod tests { use super::*; use crate::db::tests::{TestDbBuilder, setup_db}; use crate::place::{typing_extensions_symbol, typing_symbol}; - use crate::semantic_index::FileScopeId; - use ruff_db::files::system_path_to_file; use ruff_db::system::DbWithWritableSystem as _; use ruff_python_ast::PythonVersion; use test_case::test_case; @@ -12854,14 +13487,8 @@ pub(crate) mod tests { #[test] fn divergent_type() { - let mut db = setup_db(); - - db.write_dedented("src/foo.py", "").unwrap(); - let file = system_path_to_file(&db, "src/foo.py").unwrap(); - let file_scope_id = FileScopeId::global(); - let scope = file_scope_id.to_scope_id(&db, file); - - let div = Type::Dynamic(DynamicType::Divergent(DivergentType { scope: Some(scope) })); + let db = setup_db(); + let div = Type::divergent(salsa::plumbing::Id::from_bits(1)); // The `Divergent` type must not be eliminated in union with other dynamic types, // as this would prevent detection of divergent type inference using `Divergent`. @@ -12908,6 +13535,31 @@ pub(crate) mod tests { let union = UnionType::from_elements(&db, [KnownClass::Object.to_instance(&db), div]); assert_eq!(union.display(&db).to_string(), "object"); + let recursive = UnionType::from_elements( + &db, + [ + KnownClass::List.to_specialized_instance(&db, [div]), + Type::none(&db), + ], + ); + let nested_rec = KnownClass::List.to_specialized_instance(&db, [recursive]); + assert_eq!( + nested_rec.display(&db).to_string(), + "list[list[Divergent] | None]" + ); + let visitor = NormalizedVisitor::default(); + let normalized = nested_rec + .recursive_type_normalized_impl(&db, div, false, &visitor) + .unwrap(); + assert_eq!(normalized.display(&db).to_string(), "list[Divergent]"); + + let union = UnionType::from_elements(&db, [div, KnownClass::Int.to_instance(&db)]); + assert_eq!(union.display(&db).to_string(), "Divergent | int"); + let normalized = union + .recursive_type_normalized_impl(&db, div, false, &visitor) + .unwrap(); + assert_eq!(normalized.display(&db).to_string(), "int"); + // The same can be said about intersections for the `Never` type. let intersection = IntersectionBuilder::new(&db) .add_positive(Type::Never) @@ -13019,4 +13671,82 @@ type RecursiveAlias2[T] = None | list[T] | list[RecursiveAlias2[T]] TypeVarVariance::Invariant ); } + + #[test] + fn eager_expansion() { + use crate::db::tests::TestDb; + use crate::place::global_symbol; + + fn get_type_alias<'db>(db: &'db TestDb, name: &str) -> Type<'db> { + let module = ruff_db::files::system_path_to_file(db, "/src/a.py").unwrap(); + let ty = global_symbol(db, module, name).place.expect_type(); + let Type::KnownInstance(KnownInstanceType::TypeAliasType(TypeAliasType::PEP695( + type_alias, + ))) = ty + else { + panic!("Expected `{name}` to be a type alias"); + }; + Type::TypeAlias(TypeAliasType::PEP695(type_alias)) + } + + let mut db = setup_db(); + db.write_dedented( + "/src/a.py", + r#" + +type IntStr = int | str +type ListIntStr = list[IntStr] +type RecursiveList[T] = list[T | RecursiveList[T]] +type RecursiveIntList = RecursiveList[int] +type Itself = Itself +type A = B +type B = A +type G[T] = H[T] +type H[T] = G[T] +"#, + ) + .unwrap(); + + let int_str = get_type_alias(&db, "IntStr"); + assert_eq!( + int_str.expand_eagerly(&db).display(&db).to_string(), + "int | str", + ); + + let list_int_str = get_type_alias(&db, "ListIntStr"); + assert_eq!( + list_int_str.expand_eagerly(&db).display(&db).to_string(), + "list[int | str]", + ); + + let rec_list = get_type_alias(&db, "RecursiveList"); + assert_eq!( + rec_list.expand_eagerly(&db).display(&db).to_string(), + "list[Divergent]", + ); + + let rec_int_list = get_type_alias(&db, "RecursiveIntList"); + assert_eq!( + rec_int_list.expand_eagerly(&db).display(&db).to_string(), + "list[Divergent]", + ); + + let itself = get_type_alias(&db, "Itself"); + assert_eq!( + itself.expand_eagerly(&db).display(&db).to_string(), + "Divergent", + ); + + let a = get_type_alias(&db, "A"); + assert_eq!(a.expand_eagerly(&db).display(&db).to_string(), "Divergent",); + + let b = get_type_alias(&db, "B"); + assert_eq!(b.expand_eagerly(&db).display(&db).to_string(), "Divergent",); + + let g = get_type_alias(&db, "G"); + assert_eq!(g.expand_eagerly(&db).display(&db).to_string(), "Divergent",); + + let h = get_type_alias(&db, "H"); + assert_eq!(h.expand_eagerly(&db).display(&db).to_string(), "Divergent",); + } } diff --git a/crates/ty_python_semantic/src/types/bound_super.rs b/crates/ty_python_semantic/src/types/bound_super.rs index 65f9295a5e..95ac7b6148 100644 --- a/crates/ty_python_semantic/src/types/bound_super.rs +++ b/crates/ty_python_semantic/src/types/bound_super.rs @@ -172,7 +172,7 @@ impl<'db> BoundSuperError<'db> { #[derive(Debug, Copy, Clone, Hash, PartialEq, Eq, get_size2::GetSize)] pub enum SuperOwnerKind<'db> { - Dynamic(DynamicType<'db>), + Dynamic(DynamicType), Class(ClassType<'db>), Instance(NominalInstanceType<'db>), } @@ -192,6 +192,26 @@ impl<'db> SuperOwnerKind<'db> { } } + fn recursive_type_normalized_impl( + self, + db: &'db dyn Db, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> Option { + match self { + SuperOwnerKind::Dynamic(dynamic) => { + Some(SuperOwnerKind::Dynamic(dynamic.recursive_type_normalized())) + } + SuperOwnerKind::Class(class) => Some(SuperOwnerKind::Class( + class.recursive_type_normalized_impl(db, div, nested, visitor)?, + )), + SuperOwnerKind::Instance(instance) => Some(SuperOwnerKind::Instance( + instance.recursive_type_normalized_impl(db, div, nested, visitor)?, + )), + } + } + fn iter_mro(self, db: &'db dyn Db) -> impl Iterator> { match self { SuperOwnerKind::Dynamic(dynamic) => { @@ -582,4 +602,20 @@ impl<'db> BoundSuperType<'db> { self.owner(db).normalized_impl(db, visitor), ) } + + pub(super) fn recursive_type_normalized_impl( + self, + db: &'db dyn Db, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> Option { + Some(Self::new( + db, + self.pivot_class(db) + .recursive_type_normalized_impl(db, div, nested, visitor)?, + self.owner(db) + .recursive_type_normalized_impl(db, div, nested, visitor)?, + )) + } } diff --git a/crates/ty_python_semantic/src/types/builder.rs b/crates/ty_python_semantic/src/types/builder.rs index ac6603f8f5..0618682837 100644 --- a/crates/ty_python_semantic/src/types/builder.rs +++ b/crates/ty_python_semantic/src/types/builder.rs @@ -214,6 +214,9 @@ pub(crate) struct UnionBuilder<'db> { db: &'db dyn Db, unpack_aliases: bool, order_elements: bool, + // This is enabled when joining types in a `cycle_recovery` function. + // Since a cycle cannot be created within a `cycle_recovery` function, execution of `is_redundant_with` is skipped. + cycle_recovery: bool, } impl<'db> UnionBuilder<'db> { @@ -223,6 +226,7 @@ impl<'db> UnionBuilder<'db> { elements: vec![], unpack_aliases: true, order_elements: false, + cycle_recovery: false, } } @@ -236,6 +240,14 @@ impl<'db> UnionBuilder<'db> { self } + pub(crate) fn cycle_recovery(mut self, val: bool) -> Self { + self.cycle_recovery = val; + if self.cycle_recovery { + self.unpack_aliases = false; + } + self + } + pub(crate) fn is_empty(&self) -> bool { self.elements.is_empty() } @@ -466,7 +478,7 @@ impl<'db> UnionBuilder<'db> { // If an alias gets here, it means we aren't unpacking aliases, and we also // shouldn't try to simplify aliases out of the union, because that will require // unpacking them. - let should_simplify_full = !matches!(ty, Type::TypeAlias(_)); + let should_simplify_full = !matches!(ty, Type::TypeAlias(_)) && !self.cycle_recovery; let mut to_remove = SmallVec::<[usize; 2]>::new(); let ty_negated = if should_simplify_full { diff --git a/crates/ty_python_semantic/src/types/class.rs b/crates/ty_python_semantic/src/types/class.rs index 151d329d47..033f326c39 100644 --- a/crates/ty_python_semantic/src/types/class.rs +++ b/crates/ty_python_semantic/src/types/class.rs @@ -6,7 +6,7 @@ use super::TypeVarVariance; use super::{ BoundTypeVarInstance, IntersectionBuilder, MemberLookupPolicy, Mro, MroError, MroIterator, SpecialFormType, SubclassOfType, Truthiness, Type, TypeQualifiers, class_base::ClassBase, - function::FunctionType, infer_expression_type, infer_unpack_types, + function::FunctionType, }; use crate::module_resolver::KnownModule; use crate::place::TypeOrigin; @@ -25,7 +25,7 @@ use crate::types::function::{DataclassTransformerParams, KnownFunction}; use crate::types::generics::{ GenericContext, InferableTypeVars, Specialization, walk_generic_context, walk_specialization, }; -use crate::types::infer::nearest_enclosing_class; +use crate::types::infer::{infer_expression_type, infer_unpack_types, nearest_enclosing_class}; use crate::types::member::{Member, class_member}; use crate::types::signatures::{CallableSignature, Parameter, Parameters, Signature}; use crate::types::tuple::{TupleSpec, TupleType}; @@ -37,8 +37,7 @@ use crate::types::{ HasRelationToVisitor, IsDisjointVisitor, IsEquivalentVisitor, KnownInstanceType, ManualPEP695TypeAliasType, MaterializationKind, NormalizedVisitor, PropertyInstanceType, StringLiteralType, TypeAliasType, TypeContext, TypeMapping, TypeRelation, TypedDictParams, - UnionBuilder, VarianceInferable, declaration_type, determine_upper_bound, - exceeds_max_specialization_depth, infer_definition_types, + UnionBuilder, VarianceInferable, binding_type, declaration_type, determine_upper_bound, }; use crate::{ Db, FxIndexMap, FxIndexSet, FxOrderSet, Program, @@ -87,12 +86,30 @@ fn inheritance_cycle_initial<'db>( fn implicit_attribute_initial<'db>( _db: &'db dyn Db, - _id: salsa::Id, + id: salsa::Id, _class_body_scope: ScopeId<'db>, _name: String, _target_method_decorator: MethodDecorator, ) -> Member<'db> { - Member::unbound() + Member { + inner: Place::bound(Type::divergent(id)).into(), + } +} + +#[allow(clippy::too_many_arguments)] +fn implicit_attribute_cycle_recover<'db>( + db: &'db dyn Db, + cycle: &salsa::Cycle, + previous_member: &Member<'db>, + member: Member<'db>, + _class_body_scope: ScopeId<'db>, + _name: String, + _target_method_decorator: MethodDecorator, +) -> Member<'db> { + let inner = member + .inner + .cycle_normalized(db, previous_member.inner, cycle); + Member { inner } } fn try_mro_cycle_initial<'db>( @@ -107,7 +124,6 @@ fn try_mro_cycle_initial<'db>( )) } -#[allow(clippy::unnecessary_wraps)] fn is_typed_dict_cycle_initial<'db>( _db: &'db dyn Db, _id: salsa::Id, @@ -127,6 +143,14 @@ fn try_metaclass_cycle_initial<'db>( }) } +fn decorators_cycle_initial<'db>( + _db: &'db dyn Db, + _id: salsa::Id, + _self: ClassLiteral<'db>, +) -> Box<[Type<'db>]> { + Box::default() +} + fn fields_cycle_initial<'db>( _db: &'db dyn Db, _id: salsa::Id, @@ -255,6 +279,21 @@ impl<'db> GenericAlias<'db> { ) } + pub(super) fn recursive_type_normalized_impl( + self, + db: &'db dyn Db, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> Option { + Some(Self::new( + db, + self.origin(db), + self.specialization(db) + .recursive_type_normalized_impl(db, div, nested, visitor)?, + )) + } + pub(crate) fn definition(self, db: &'db dyn Db) -> Definition<'db> { self.origin(db).definition(db) } @@ -399,6 +438,21 @@ impl<'db> ClassType<'db> { } } + pub(super) fn recursive_type_normalized_impl( + self, + db: &'db dyn Db, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> Option { + match self { + Self::NonGeneric(_) => Some(self), + Self::Generic(generic) => Some(Self::Generic( + generic.recursive_type_normalized_impl(db, div, nested, visitor)?, + )), + } + } + pub(super) fn has_pep_695_type_params(self, db: &'db dyn Db) -> bool { match self { Self::NonGeneric(class) => class.has_pep_695_type_params(db), @@ -1533,17 +1587,7 @@ impl<'db> ClassLiteral<'db> { match self.generic_context(db) { None => ClassType::NonGeneric(self), Some(generic_context) => { - let mut specialization = f(generic_context); - - for (idx, ty) in specialization.types(db).iter().enumerate() { - if exceeds_max_specialization_depth(db, *ty) { - specialization = specialization.with_replaced_type( - db, - idx, - Type::divergent(Some(self.body_scope(db))), - ); - } - } + let specialization = f(generic_context); ClassType::Generic(GenericAlias::new(db, self, specialization)) } @@ -1698,7 +1742,7 @@ impl<'db> ClassLiteral<'db> { } /// Return the types of the decorators on this class - #[salsa::tracked(returns(deref), heap_size=ruff_memory_usage::heap_size)] + #[salsa::tracked(returns(deref), cycle_initial=decorators_cycle_initial, heap_size=ruff_memory_usage::heap_size)] fn decorators(self, db: &'db dyn Db) -> Box<[Type<'db>]> { tracing::trace!("ClassLiteral::decorators: {}", self.name(db)); @@ -3136,10 +3180,12 @@ impl<'db> ClassLiteral<'db> { ) } - #[salsa::tracked(cycle_initial=implicit_attribute_initial, + #[salsa::tracked( + cycle_fn=implicit_attribute_cycle_recover, + cycle_initial=implicit_attribute_initial, heap_size=ruff_memory_usage::heap_size, )] - fn implicit_attribute_inner( + pub(super) fn implicit_attribute_inner( db: &'db dyn Db, class_body_scope: ScopeId<'db>, name: String, @@ -3159,7 +3205,6 @@ impl<'db> ClassLiteral<'db> { let index = semantic_index(db, file); let class_map = use_def_map(db, class_body_scope); let class_table = place_table(db, class_body_scope); - let is_valid_scope = |method_scope: &Scope| { if let Some(method_def) = method_scope.node().as_function() { let method_name = method_def.node(&module).name.as_str(); @@ -5468,8 +5513,7 @@ impl KnownClass { }; let definition = index.expect_single_definition(first_param); - let first_param = - infer_definition_types(db, definition).binding_type(definition); + let first_param = binding_type(db, definition); let bound_super = BoundSuperType::build( db, diff --git a/crates/ty_python_semantic/src/types/class_base.rs b/crates/ty_python_semantic/src/types/class_base.rs index 6d0af17427..74833406f5 100644 --- a/crates/ty_python_semantic/src/types/class_base.rs +++ b/crates/ty_python_semantic/src/types/class_base.rs @@ -18,7 +18,7 @@ use crate::types::{ /// automatically construct the default specialization for that class. #[derive(Debug, Copy, Clone, Hash, PartialEq, Eq, salsa::Update, get_size2::GetSize)] pub enum ClassBase<'db> { - Dynamic(DynamicType<'db>), + Dynamic(DynamicType), Class(ClassType<'db>), /// Although `Protocol` is not a class in typeshed's stubs, it is at runtime, /// and can appear in the MRO of a class. @@ -43,6 +43,22 @@ impl<'db> ClassBase<'db> { } } + pub(super) fn recursive_type_normalized_impl( + self, + db: &'db dyn Db, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> Option { + match self { + Self::Dynamic(dynamic) => Some(Self::Dynamic(dynamic.recursive_type_normalized())), + Self::Class(class) => Some(Self::Class( + class.recursive_type_normalized_impl(db, div, nested, visitor)?, + )), + Self::Protocol | Self::Generic | Self::TypedDict => Some(self), + } + } + pub(crate) fn name(self, db: &'db dyn Db) -> &'db str { match self { ClassBase::Class(class) => class.name(db), diff --git a/crates/ty_python_semantic/src/types/cyclic.rs b/crates/ty_python_semantic/src/types/cyclic.rs index e46d30f40f..344881303a 100644 --- a/crates/ty_python_semantic/src/types/cyclic.rs +++ b/crates/ty_python_semantic/src/types/cyclic.rs @@ -89,6 +89,23 @@ impl CycleDetector { ret } + + pub fn try_visit(&self, item: T, func: impl FnOnce() -> Option) -> Option { + if let Some(val) = self.cache.borrow().get(&item) { + return Some(val.clone()); + } + + // We hit a cycle + if !self.seen.borrow_mut().insert(item.clone()) { + return Some(self.fallback.clone()); + } + + let ret = func()?; + self.seen.borrow_mut().pop(); + self.cache.borrow_mut().insert(item, ret.clone()); + + Some(ret) + } } impl Default for CycleDetector { diff --git a/crates/ty_python_semantic/src/types/diagnostic.rs b/crates/ty_python_semantic/src/types/diagnostic.rs index 82db60c7b9..0738c41330 100644 --- a/crates/ty_python_semantic/src/types/diagnostic.rs +++ b/crates/ty_python_semantic/src/types/diagnostic.rs @@ -54,6 +54,7 @@ pub(crate) fn register_lints(registry: &mut LintRegistryBuilder) { registry.register_lint(&CONFLICTING_DECLARATIONS); registry.register_lint(&CONFLICTING_METACLASS); registry.register_lint(&CYCLIC_CLASS_DEFINITION); + registry.register_lint(&CYCLIC_TYPE_ALIAS_DEFINITION); registry.register_lint(&DEPRECATED); registry.register_lint(&DIVISION_BY_ZERO); registry.register_lint(&DUPLICATE_BASE); @@ -274,6 +275,28 @@ declare_lint! { } } +declare_lint! { + /// ## What it does + /// Checks for type alias definitions that (directly or mutually) refer to themselves. + /// + /// ## Why is it bad? + /// Although it is permitted to define a recursive type alias, it is not meaningful + /// to have a type alias whose expansion can only result in itself, and is therefore not allowed. + /// + /// ## Examples + /// ```python + /// type Itself = Itself + /// + /// type A = B + /// type B = A + /// ``` + pub(crate) static CYCLIC_TYPE_ALIAS_DEFINITION = { + summary: "detects cyclic type alias definitions", + status: LintStatus::preview("1.0.0"), + default_level: Level::Error, + } +} + declare_lint! { /// ## What it does /// It detects division by zero. diff --git a/crates/ty_python_semantic/src/types/function.rs b/crates/ty_python_semantic/src/types/function.rs index 222d1cfb77..3c440ba2a2 100644 --- a/crates/ty_python_semantic/src/types/function.rs +++ b/crates/ty_python_semantic/src/types/function.rs @@ -1053,6 +1053,34 @@ impl<'db> FunctionType<'db> { updated_last_definition_signature, ) } + + pub(crate) fn recursive_type_normalized_impl( + self, + db: &'db dyn Db, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> Option { + let literal = self.literal(db); + let updated_signature = match self.updated_signature(db) { + Some(signature) => { + Some(signature.recursive_type_normalized_impl(db, div, nested, visitor)?) + } + None => None, + }; + let updated_last_definition_signature = match self.updated_last_definition_signature(db) { + Some(signature) => { + Some(signature.recursive_type_normalized_impl(db, div, nested, visitor)?) + } + None => None, + }; + Some(Self::new( + db, + literal, + updated_signature, + updated_last_definition_signature, + )) + } } /// Evaluate an `isinstance` call. Return `Truthiness::AlwaysTrue` if we can definitely infer that diff --git a/crates/ty_python_semantic/src/types/generics.rs b/crates/ty_python_semantic/src/types/generics.rs index 0603413396..df15fdafc0 100644 --- a/crates/ty_python_semantic/src/types/generics.rs +++ b/crates/ty_python_semantic/src/types/generics.rs @@ -1073,6 +1073,41 @@ impl<'db> Specialization<'db> { ) } + pub(super) fn recursive_type_normalized_impl( + self, + db: &'db dyn Db, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> Option { + let types = if nested { + self.types(db) + .iter() + .map(|ty| ty.recursive_type_normalized_impl(db, div, true, visitor)) + .collect::>>()? + } else { + self.types(db) + .iter() + .map(|ty| { + ty.recursive_type_normalized_impl(db, div, true, visitor) + .unwrap_or(div) + }) + .collect::>() + }; + let tuple_inner = match self.tuple_inner(db) { + Some(tuple) => Some(tuple.recursive_type_normalized_impl(db, div, nested, visitor)?), + None => None, + }; + let context = self.generic_context(db); + Some(Self::new( + db, + context, + types, + self.materialization_kind(db), + tuple_inner, + )) + } + pub(super) fn materialize_impl( self, db: &'db dyn Db, @@ -1281,25 +1316,6 @@ impl<'db> Specialization<'db> { // A tuple's specialization will include all of its element types, so we don't need to also // look in `self.tuple`. } - - /// Returns a copy of this specialization with the type at a given index replaced. - pub(crate) fn with_replaced_type( - self, - db: &'db dyn Db, - index: usize, - new_type: Type<'db>, - ) -> Self { - let mut new_types: Box<[_]> = self.types(db).to_vec().into_boxed_slice(); - new_types[index] = new_type; - - Self::new( - db, - self.generic_context(db), - new_types, - self.materialization_kind(db), - self.tuple_inner(db), - ) - } } /// A mapping between type variables and types. diff --git a/crates/ty_python_semantic/src/types/infer.rs b/crates/ty_python_semantic/src/types/infer.rs index 1d77a76e78..fe39deb36a 100644 --- a/crates/ty_python_semantic/src/types/infer.rs +++ b/crates/ty_python_semantic/src/types/infer.rs @@ -31,7 +31,7 @@ //! //! Many of our type inference Salsa queries implement cycle recovery via fixed-point iteration. In //! general, they initiate fixed-point iteration by returning an `Inference` type that returns -//! `Type::Never` for all expressions, bindings, and declarations, and then they continue iterating +//! the `Divergent` type for all expressions, bindings, and declarations, and then they continue iterating //! the query cycle until a fixed-point is reached. Salsa has a built-in fixed limit on the number //! of iterations, so if we fail to converge, Salsa will eventually panic. (This should of course //! be considered a bug.) @@ -52,7 +52,9 @@ use crate::types::diagnostic::TypeCheckDiagnostics; use crate::types::function::FunctionType; use crate::types::generics::Specialization; use crate::types::unpacker::{UnpackResult, Unpacker}; -use crate::types::{ClassLiteral, KnownClass, Truthiness, Type, TypeAndQualifiers}; +use crate::types::{ + ClassLiteral, KnownClass, Truthiness, Type, TypeAndQualifiers, declaration_type, +}; use crate::unpack::Unpack; use builder::TypeInferenceBuilder; @@ -60,13 +62,10 @@ mod builder; #[cfg(test)] mod tests; -/// How many fixpoint iterations to allow before falling back to Divergent type. -const ITERATIONS_BEFORE_FALLBACK: u32 = 10; - /// Infer all types for a [`ScopeId`], including all definitions and expressions in that scope. /// Use when checking a scope, or needing to provide a type for an arbitrary expression in the /// scope. -#[salsa::tracked(returns(ref), cycle_initial=scope_cycle_initial, heap_size=ruff_memory_usage::heap_size)] +#[salsa::tracked(returns(ref), cycle_fn=scope_cycle_recover, cycle_initial=scope_cycle_initial, heap_size=ruff_memory_usage::heap_size)] pub(crate) fn infer_scope_types<'db>(db: &'db dyn Db, scope: ScopeId<'db>) -> ScopeInference<'db> { let file = scope.file(db); let _span = tracing::trace_span!("infer_scope_types", scope=?scope.as_id(), ?file).entered(); @@ -80,12 +79,22 @@ pub(crate) fn infer_scope_types<'db>(db: &'db dyn Db, scope: ScopeId<'db>) -> Sc TypeInferenceBuilder::new(db, InferenceRegion::Scope(scope), index, &module).finish_scope() } +fn scope_cycle_recover<'db>( + db: &'db dyn Db, + cycle: &salsa::Cycle, + previous_inference: &ScopeInference<'db>, + inference: ScopeInference<'db>, + _scope: ScopeId<'db>, +) -> ScopeInference<'db> { + inference.cycle_normalized(db, previous_inference, cycle) +} + fn scope_cycle_initial<'db>( _db: &'db dyn Db, - _id: salsa::Id, - scope: ScopeId<'db>, + id: salsa::Id, + _scope: ScopeId<'db>, ) -> ScopeInference<'db> { - ScopeInference::cycle_initial(scope) + ScopeInference::cycle_initial(Type::divergent(id)) } /// Infer all types for a [`Definition`] (including sub-expressions). @@ -113,30 +122,26 @@ pub(crate) fn infer_definition_types<'db>( fn definition_cycle_recover<'db>( db: &'db dyn Db, cycle: &salsa::Cycle, - last_provisional_value: &DefinitionInference<'db>, - value: DefinitionInference<'db>, - definition: Definition<'db>, + previous_inference: &DefinitionInference<'db>, + inference: DefinitionInference<'db>, + _definition: Definition<'db>, ) -> DefinitionInference<'db> { - if &value == last_provisional_value || cycle.iteration() != ITERATIONS_BEFORE_FALLBACK { - value - } else { - DefinitionInference::cycle_fallback(definition.scope(db)) - } + inference.cycle_normalized(db, previous_inference, cycle) } fn definition_cycle_initial<'db>( db: &'db dyn Db, - _id: salsa::Id, + id: salsa::Id, definition: Definition<'db>, ) -> DefinitionInference<'db> { - DefinitionInference::cycle_initial(definition.scope(db)) + DefinitionInference::cycle_initial(definition.scope(db), Type::divergent(id)) } /// Infer types for all deferred type expressions in a [`Definition`]. /// /// Deferred expressions are type expressions (annotations, base classes, aliases...) in a stub /// file, or in a file with `from __future__ import annotations`, or stringified annotations. -#[salsa::tracked(returns(ref), cycle_initial=deferred_cycle_initial, heap_size=ruff_memory_usage::heap_size)] +#[salsa::tracked(returns(ref), cycle_fn=deferred_cycle_recovery, cycle_initial=deferred_cycle_initial, heap_size=ruff_memory_usage::heap_size)] pub(crate) fn infer_deferred_types<'db>( db: &'db dyn Db, definition: Definition<'db>, @@ -157,12 +162,22 @@ pub(crate) fn infer_deferred_types<'db>( .finish_definition() } +fn deferred_cycle_recovery<'db>( + db: &'db dyn Db, + cycle: &salsa::Cycle, + previous_inference: &DefinitionInference<'db>, + inference: DefinitionInference<'db>, + _definition: Definition<'db>, +) -> DefinitionInference<'db> { + inference.cycle_normalized(db, previous_inference, cycle) +} + fn deferred_cycle_initial<'db>( db: &'db dyn Db, - _id: salsa::Id, + id: salsa::Id, definition: Definition<'db>, ) -> DefinitionInference<'db> { - DefinitionInference::cycle_initial(definition.scope(db)) + DefinitionInference::cycle_initial(definition.scope(db), Type::divergent(id)) } /// Infer all types for an [`Expression`] (including sub-expressions). @@ -178,7 +193,7 @@ pub(crate) fn infer_expression_types<'db>( } #[salsa::tracked(returns(ref), cycle_fn=expression_cycle_recover, cycle_initial=expression_cycle_initial, heap_size=ruff_memory_usage::heap_size)] -fn infer_expression_types_impl<'db>( +pub(super) fn infer_expression_types_impl<'db>( db: &'db dyn Db, input: InferExpression<'db>, ) -> ExpressionInference<'db> { @@ -208,23 +223,20 @@ fn infer_expression_types_impl<'db>( fn expression_cycle_recover<'db>( db: &'db dyn Db, cycle: &salsa::Cycle, - last_provisional_value: &ExpressionInference<'db>, - value: ExpressionInference<'db>, - input: InferExpression<'db>, + previous_inference: &ExpressionInference<'db>, + inference: ExpressionInference<'db>, + _input: InferExpression<'db>, ) -> ExpressionInference<'db> { - if &value == last_provisional_value || cycle.iteration() != ITERATIONS_BEFORE_FALLBACK { - value - } else { - ExpressionInference::cycle_fallback(input.expression(db).scope(db)) - } + inference.cycle_normalized(db, previous_inference, cycle) } fn expression_cycle_initial<'db>( db: &'db dyn Db, - _id: salsa::Id, + id: salsa::Id, input: InferExpression<'db>, ) -> ExpressionInference<'db> { - ExpressionInference::cycle_initial(input.expression(db).scope(db)) + let cycle_recovery = Type::divergent(id); + ExpressionInference::cycle_initial(input.expression(db).scope(db), cycle_recovery) } /// Infers the type of an `expression` that is guaranteed to be in the same file as the calling query. @@ -232,7 +244,7 @@ fn expression_cycle_initial<'db>( /// This is a small helper around [`infer_expression_types()`] to reduce the boilerplate. /// Use [`infer_expression_type()`] if it isn't guaranteed that `expression` is in the same file to /// avoid cross-file query dependencies. -pub(super) fn infer_same_file_expression_type<'db>( +pub(crate) fn infer_same_file_expression_type<'db>( db: &'db dyn Db, expression: Expression<'db>, tcx: TypeContext<'db>, @@ -257,22 +269,33 @@ pub(crate) fn infer_expression_type<'db>( infer_expression_type_impl(db, InferExpression::new(db, expression, tcx)) } -#[salsa::tracked(cycle_initial=single_expression_cycle_initial, heap_size=ruff_memory_usage::heap_size)] +#[salsa::tracked(cycle_fn=single_expression_cycle_recover, cycle_initial=single_expression_cycle_initial, heap_size=ruff_memory_usage::heap_size)] fn infer_expression_type_impl<'db>(db: &'db dyn Db, input: InferExpression<'db>) -> Type<'db> { let file = input.expression(db).file(db); let module = parsed_module(db, file).load(db); // It's okay to call the "same file" version here because we're inside a salsa query. let inference = infer_expression_types_impl(db, input); + inference.expression_type(input.expression(db).node_ref(db, &module)) } +fn single_expression_cycle_recover<'db>( + db: &'db dyn Db, + cycle: &salsa::Cycle, + previous_cycle_value: &Type<'db>, + result: Type<'db>, + _input: InferExpression<'db>, +) -> Type<'db> { + result.cycle_normalized(db, *previous_cycle_value, cycle) +} + fn single_expression_cycle_initial<'db>( _db: &'db dyn Db, - _id: salsa::Id, + id: salsa::Id, _input: InferExpression<'db>, ) -> Type<'db> { - Type::Never + Type::divergent(id) } /// An `Expression` with an optional `TypeContext`. @@ -280,13 +303,13 @@ fn single_expression_cycle_initial<'db>( /// This is a Salsa supertype used as the input to `infer_expression_types` to avoid /// interning an `ExpressionWithContext` unnecessarily when no type context is provided. #[derive(Debug, Clone, Copy, Eq, Hash, PartialEq, salsa::Supertype, salsa::Update)] -enum InferExpression<'db> { +pub(super) enum InferExpression<'db> { Bare(Expression<'db>), WithContext(ExpressionWithContext<'db>), } impl<'db> InferExpression<'db> { - fn new( + pub(super) fn new( db: &'db dyn Db, expression: Expression<'db>, tcx: TypeContext<'db>, @@ -320,7 +343,7 @@ impl<'db> InferExpression<'db> { /// An `Expression` with a `TypeContext`. #[salsa::interned(debug, heap_size=ruff_memory_usage::heap_size)] -struct ExpressionWithContext<'db> { +pub(super) struct ExpressionWithContext<'db> { expression: Expression<'db>, tcx: TypeContext<'db>, } @@ -381,7 +404,6 @@ pub(crate) fn static_expression_truthiness<'db>( let file = expression.file(db); let module = parsed_module(db, file).load(db); let node = expression.node_ref(db, &module); - inference.expression_type(node).bool(db) } @@ -399,7 +421,7 @@ fn static_expression_truthiness_cycle_initial<'db>( /// involved in an unpacking operation. It returns a result-like object that can be used to get the /// type of the variables involved in this unpacking along with any violations that are detected /// during this unpacking. -#[salsa::tracked(returns(ref), cycle_initial=unpack_cycle_initial, heap_size=ruff_memory_usage::heap_size)] +#[salsa::tracked(returns(ref), cycle_fn=unpack_cycle_recover, cycle_initial=unpack_cycle_initial, heap_size=ruff_memory_usage::heap_size)] pub(super) fn infer_unpack_types<'db>(db: &'db dyn Db, unpack: Unpack<'db>) -> UnpackResult<'db> { let file = unpack.file(db); let module = parsed_module(db, file).load(db); @@ -413,10 +435,20 @@ pub(super) fn infer_unpack_types<'db>(db: &'db dyn Db, unpack: Unpack<'db>) -> U fn unpack_cycle_initial<'db>( _db: &'db dyn Db, - _id: salsa::Id, + id: salsa::Id, _unpack: Unpack<'db>, ) -> UnpackResult<'db> { - UnpackResult::cycle_initial(Type::Never) + UnpackResult::cycle_initial(Type::divergent(id)) +} + +fn unpack_cycle_recover<'db>( + db: &'db dyn Db, + cycle: &salsa::Cycle, + previous_cycle_result: &UnpackResult<'db>, + result: UnpackResult<'db>, + _unpack: Unpack<'db>, +) -> UnpackResult<'db> { + result.cycle_normalized(db, previous_cycle_result, cycle) } /// Returns the type of the nearest enclosing class for the given scope. @@ -438,8 +470,7 @@ pub(crate) fn nearest_enclosing_class<'db>( .find_map(|(_, ancestor_scope)| { let class = ancestor_scope.node().as_class()?; let definition = semantic.expect_single_definition(class); - infer_definition_types(db, definition) - .declaration_type(definition) + declaration_type(db, definition) .inner_type() .as_class_literal() }) @@ -494,35 +525,6 @@ impl<'db> InferenceRegion<'db> { } } -#[derive(Debug, Clone, Copy, Eq, PartialEq, get_size2::GetSize, salsa::Update)] -enum CycleRecovery<'db> { - /// An initial-value for fixpoint iteration; all types are `Type::Never`. - Initial, - /// A divergence-fallback value for fixpoint iteration; all types are `Divergent`. - Divergent(ScopeId<'db>), -} - -impl<'db> CycleRecovery<'db> { - fn merge(self, other: Option>) -> Self { - if let Some(other) = other { - match (self, other) { - // It's important here that we keep the scope of `self` if merging two `Divergent`. - (Self::Divergent(scope), _) | (_, Self::Divergent(scope)) => Self::Divergent(scope), - _ => Self::Initial, - } - } else { - self - } - } - - fn fallback_type(self) -> Type<'db> { - match self { - Self::Initial => Type::Never, - Self::Divergent(scope) => Type::divergent(Some(scope)), - } - } -} - /// The inferred types for a scope region. #[derive(Debug, Eq, PartialEq, salsa::Update, get_size2::GetSize)] pub(crate) struct ScopeInference<'db> { @@ -538,26 +540,38 @@ struct ScopeInferenceExtra<'db> { /// String annotations found in this region string_annotations: FxHashSet, - /// Is this a cycle-recovery inference result, and if so, what kind? - cycle_recovery: Option>, + /// The fallback type for missing expressions/bindings/declarations or recursive type inference. + cycle_recovery: Option>, /// The diagnostics for this region. diagnostics: TypeCheckDiagnostics, } impl<'db> ScopeInference<'db> { - fn cycle_initial(scope: ScopeId<'db>) -> Self { - let _ = scope; - + fn cycle_initial(cycle_recovery: Type<'db>) -> Self { Self { extra: Some(Box::new(ScopeInferenceExtra { - cycle_recovery: Some(CycleRecovery::Initial), + cycle_recovery: Some(cycle_recovery), ..ScopeInferenceExtra::default() })), expressions: FxHashMap::default(), } } + fn cycle_normalized( + mut self, + db: &'db dyn Db, + previous_inference: &ScopeInference<'db>, + cycle: &salsa::Cycle, + ) -> ScopeInference<'db> { + for (expr, ty) in &mut self.expressions { + let previous_ty = previous_inference.expression_type(*expr); + *ty = ty.cycle_normalized(db, previous_ty, cycle); + } + + self + } + pub(crate) fn diagnostics(&self) -> Option<&TypeCheckDiagnostics> { self.extra.as_deref().map(|extra| &extra.diagnostics) } @@ -578,9 +592,7 @@ impl<'db> ScopeInference<'db> { } fn fallback_type(&self) -> Option> { - self.extra - .as_ref() - .and_then(|extra| extra.cycle_recovery.map(CycleRecovery::fallback_type)) + self.extra.as_ref().and_then(|extra| extra.cycle_recovery) } /// Returns whether the given expression is a string annotation @@ -608,7 +620,7 @@ pub(crate) struct DefinitionInference<'db> { /// /// Almost all definition regions have less than 10 bindings. There are very few with more than 10 (but still less than 20). /// Because of that, use a slice with linear search over a hash map. - bindings: Box<[(Definition<'db>, Type<'db>)]>, + pub(crate) bindings: Box<[(Definition<'db>, Type<'db>)]>, /// The types and type qualifiers of every declaration in this region. /// @@ -626,8 +638,8 @@ struct DefinitionInferenceExtra<'db> { /// String annotations found in this region string_annotations: FxHashSet, - /// Is this a cycle-recovery inference result, and if so, what kind? - cycle_recovery: Option>, + /// The fallback type for missing expressions/bindings/declarations or recursive type inference. + cycle_recovery: Option>, /// The definitions that have some deferred parts. deferred: Box<[Definition<'db>]>, @@ -640,7 +652,7 @@ struct DefinitionInferenceExtra<'db> { } impl<'db> DefinitionInference<'db> { - fn cycle_initial(scope: ScopeId<'db>) -> Self { + fn cycle_initial(scope: ScopeId<'db>, cycle_recovery: Type<'db>) -> Self { let _ = scope; Self { @@ -650,26 +662,49 @@ impl<'db> DefinitionInference<'db> { #[cfg(debug_assertions)] scope, extra: Some(Box::new(DefinitionInferenceExtra { - cycle_recovery: Some(CycleRecovery::Initial), + cycle_recovery: Some(cycle_recovery), ..DefinitionInferenceExtra::default() })), } } - fn cycle_fallback(scope: ScopeId<'db>) -> Self { - let _ = scope; - - Self { - expressions: FxHashMap::default(), - bindings: Box::default(), - declarations: Box::default(), - #[cfg(debug_assertions)] - scope, - extra: Some(Box::new(DefinitionInferenceExtra { - cycle_recovery: Some(CycleRecovery::Divergent(scope)), - ..DefinitionInferenceExtra::default() - })), + fn cycle_normalized( + mut self, + db: &'db dyn Db, + previous_inference: &DefinitionInference<'db>, + cycle: &salsa::Cycle, + ) -> DefinitionInference<'db> { + for (expr, ty) in &mut self.expressions { + let previous_ty = previous_inference.expression_type(*expr); + *ty = ty.cycle_normalized(db, previous_ty, cycle); } + for (binding, binding_ty) in &mut self.bindings { + if let Some((_, previous_binding)) = previous_inference + .bindings + .iter() + .find(|(previous_binding, _)| previous_binding == binding) + { + *binding_ty = binding_ty.cycle_normalized(db, *previous_binding, cycle); + } else { + *binding_ty = binding_ty.recursive_type_normalized(db, cycle); + } + } + for (declaration, declaration_ty) in &mut self.declarations { + if let Some((_, previous_declaration)) = previous_inference + .declarations + .iter() + .find(|(previous_declaration, _)| previous_declaration == declaration) + { + *declaration_ty = declaration_ty.map_type(|decl_ty| { + decl_ty.cycle_normalized(db, previous_declaration.inner_type(), cycle) + }); + } else { + *declaration_ty = + declaration_ty.map_type(|decl_ty| decl_ty.recursive_type_normalized(db, cycle)); + } + } + + self } pub(crate) fn expression_type(&self, expression: impl Into) -> Type<'db> { @@ -735,10 +770,8 @@ impl<'db> DefinitionInference<'db> { self.declarations.iter().map(|(_, qualifiers)| *qualifiers) } - fn fallback_type(&self) -> Option> { - self.extra - .as_ref() - .and_then(|extra| extra.cycle_recovery.map(CycleRecovery::fallback_type)) + pub(crate) fn fallback_type(&self) -> Option> { + self.extra.as_ref().and_then(|extra| extra.cycle_recovery) } pub(crate) fn undecorated_type(&self) -> Option> { @@ -773,42 +806,55 @@ struct ExpressionInferenceExtra<'db> { /// The diagnostics for this region. diagnostics: TypeCheckDiagnostics, - /// Is this a cycle recovery inference result, and if so, what kind? - cycle_recovery: Option>, + /// The fallback type for missing expressions/bindings/declarations or recursive type inference. + cycle_recovery: Option>, /// `true` if all places in this expression are definitely bound all_definitely_bound: bool, } impl<'db> ExpressionInference<'db> { - fn cycle_initial(scope: ScopeId<'db>) -> Self { + fn cycle_initial(scope: ScopeId<'db>, cycle_recovery: Type<'db>) -> Self { let _ = scope; Self { extra: Some(Box::new(ExpressionInferenceExtra { - cycle_recovery: Some(CycleRecovery::Initial), + cycle_recovery: Some(cycle_recovery), all_definitely_bound: true, ..ExpressionInferenceExtra::default() })), expressions: FxHashMap::default(), - #[cfg(debug_assertions)] scope, } } - fn cycle_fallback(scope: ScopeId<'db>) -> Self { - let _ = scope; - Self { - extra: Some(Box::new(ExpressionInferenceExtra { - cycle_recovery: Some(CycleRecovery::Divergent(scope)), - all_definitely_bound: true, - ..ExpressionInferenceExtra::default() - })), - expressions: FxHashMap::default(), - - #[cfg(debug_assertions)] - scope, + fn cycle_normalized( + mut self, + db: &'db dyn Db, + previous: &ExpressionInference<'db>, + cycle: &salsa::Cycle, + ) -> ExpressionInference<'db> { + if let Some(extra) = self.extra.as_mut() { + for (binding, binding_ty) in &mut extra.bindings { + if let Some((_, previous_binding)) = previous.extra.as_deref().and_then(|extra| { + extra + .bindings + .iter() + .find(|(previous_binding, _)| previous_binding == binding) + }) { + *binding_ty = binding_ty.cycle_normalized(db, *previous_binding, cycle); + } else { + *binding_ty = binding_ty.recursive_type_normalized(db, cycle); + } + } } + + for (expr, ty) in &mut self.expressions { + let previous_ty = previous.expression_type(*expr); + *ty = ty.cycle_normalized(db, previous_ty, cycle); + } + + self } pub(crate) fn try_expression_type( @@ -827,9 +873,7 @@ impl<'db> ExpressionInference<'db> { } fn fallback_type(&self) -> Option> { - self.extra - .as_ref() - .and_then(|extra| extra.cycle_recovery.map(CycleRecovery::fallback_type)) + self.extra.as_ref().and_then(|extra| extra.cycle_recovery) } /// Returns true if all places in this expression are definitely bound. diff --git a/crates/ty_python_semantic/src/types/infer/builder.rs b/crates/ty_python_semantic/src/types/infer/builder.rs index dc4dc2a7e9..b2f8cc5687 100644 --- a/crates/ty_python_semantic/src/types/infer/builder.rs +++ b/crates/ty_python_semantic/src/types/infer/builder.rs @@ -14,10 +14,10 @@ use rustc_hash::{FxHashMap, FxHashSet}; use smallvec::SmallVec; use super::{ - CycleRecovery, DefinitionInference, DefinitionInferenceExtra, ExpressionInference, - ExpressionInferenceExtra, InferenceRegion, ScopeInference, ScopeInferenceExtra, - infer_deferred_types, infer_definition_types, infer_expression_types, - infer_same_file_expression_type, infer_scope_types, infer_unpack_types, + DefinitionInference, DefinitionInferenceExtra, ExpressionInference, ExpressionInferenceExtra, + InferenceRegion, ScopeInference, ScopeInferenceExtra, infer_deferred_types, + infer_definition_types, infer_expression_types, infer_same_file_expression_type, + infer_unpack_types, }; use crate::diagnostic::format_enumeration; use crate::module_name::{ModuleName, ModuleNameResolutionError}; @@ -56,16 +56,16 @@ use crate::types::context::{InNoTypeCheck, InferContext}; use crate::types::cyclic::CycleDetector; use crate::types::diagnostic::{ CALL_NON_CALLABLE, CONFLICTING_DECLARATIONS, CONFLICTING_METACLASS, CYCLIC_CLASS_DEFINITION, - DIVISION_BY_ZERO, DUPLICATE_KW_ONLY, INCONSISTENT_MRO, INVALID_ARGUMENT_TYPE, - INVALID_ASSIGNMENT, INVALID_ATTRIBUTE_ACCESS, INVALID_BASE, INVALID_DECLARATION, - INVALID_GENERIC_CLASS, INVALID_KEY, INVALID_LEGACY_TYPE_VARIABLE, INVALID_METACLASS, - INVALID_NAMED_TUPLE, INVALID_NEWTYPE, INVALID_OVERLOAD, INVALID_PARAMETER_DEFAULT, - INVALID_PARAMSPEC, INVALID_PROTOCOL, INVALID_TYPE_FORM, INVALID_TYPE_GUARD_CALL, - INVALID_TYPE_VARIABLE_CONSTRAINTS, IncompatibleBases, NON_SUBSCRIPTABLE, - POSSIBLY_MISSING_ATTRIBUTE, POSSIBLY_MISSING_IMPLICIT_CALL, POSSIBLY_MISSING_IMPORT, - SUBCLASS_OF_FINAL_CLASS, UNDEFINED_REVEAL, UNRESOLVED_ATTRIBUTE, UNRESOLVED_GLOBAL, - UNRESOLVED_IMPORT, UNRESOLVED_REFERENCE, UNSUPPORTED_OPERATOR, USELESS_OVERLOAD_BODY, - hint_if_stdlib_attribute_exists_on_other_versions, + CYCLIC_TYPE_ALIAS_DEFINITION, DIVISION_BY_ZERO, DUPLICATE_KW_ONLY, INCONSISTENT_MRO, + INVALID_ARGUMENT_TYPE, INVALID_ASSIGNMENT, INVALID_ATTRIBUTE_ACCESS, INVALID_BASE, + INVALID_DECLARATION, INVALID_GENERIC_CLASS, INVALID_KEY, INVALID_LEGACY_TYPE_VARIABLE, + INVALID_METACLASS, INVALID_NAMED_TUPLE, INVALID_NEWTYPE, INVALID_OVERLOAD, + INVALID_PARAMETER_DEFAULT, INVALID_PARAMSPEC, INVALID_PROTOCOL, INVALID_TYPE_FORM, + INVALID_TYPE_GUARD_CALL, INVALID_TYPE_VARIABLE_CONSTRAINTS, IncompatibleBases, + NON_SUBSCRIPTABLE, POSSIBLY_MISSING_ATTRIBUTE, POSSIBLY_MISSING_IMPLICIT_CALL, + POSSIBLY_MISSING_IMPORT, SUBCLASS_OF_FINAL_CLASS, UNDEFINED_REVEAL, UNRESOLVED_ATTRIBUTE, + UNRESOLVED_GLOBAL, UNRESOLVED_IMPORT, UNRESOLVED_REFERENCE, UNSUPPORTED_OPERATOR, + USELESS_OVERLOAD_BODY, hint_if_stdlib_attribute_exists_on_other_versions, hint_if_stdlib_submodule_exists_on_other_versions, report_attempted_protocol_instantiation, report_bad_dunder_set_call, report_cannot_pop_required_field_on_typed_dict, report_duplicate_bases, report_implicit_return_type, report_index_out_of_bounds, @@ -92,7 +92,7 @@ use crate::types::infer::nearest_enclosing_function; use crate::types::instance::SliceLiteral; use crate::types::mro::MroErrorKind; use crate::types::newtype::NewType; -use crate::types::signatures::Signature; +use crate::types::signatures::{Parameter, Parameters, Signature}; use crate::types::subclass_of::SubclassOfInner; use crate::types::tuple::{Tuple, TupleLength, TupleSpec, TupleType}; use crate::types::typed_dict::{ @@ -104,11 +104,11 @@ use crate::types::{ CallDunderError, CallableBinding, CallableType, CallableTypes, ClassLiteral, ClassType, DataclassParams, DynamicType, InternedType, IntersectionBuilder, IntersectionType, KnownClass, KnownInstanceType, LintDiagnosticGuard, MemberLookupPolicy, MetaclassCandidate, - PEP695TypeAliasType, Parameter, ParameterForm, Parameters, SpecialFormType, SubclassOfType, - TrackedConstraintSet, Truthiness, Type, TypeAliasType, TypeAndQualifiers, TypeContext, - TypeQualifiers, TypeVarBoundOrConstraintsEvaluation, TypeVarDefaultEvaluation, TypeVarIdentity, + PEP695TypeAliasType, ParameterForm, SpecialFormType, SubclassOfType, TrackedConstraintSet, + Truthiness, Type, TypeAliasType, TypeAndQualifiers, TypeContext, TypeQualifiers, + TypeVarBoundOrConstraintsEvaluation, TypeVarDefaultEvaluation, TypeVarIdentity, TypeVarInstance, TypeVarKind, TypeVarVariance, TypedDictType, UnionBuilder, UnionType, - UnionTypeInstance, binding_type, liskov, todo_type, + UnionTypeInstance, binding_type, infer_scope_types, liskov, todo_type, }; use crate::types::{ClassBase, add_inferred_python_version_hint_to_diagnostic}; use crate::unpack::{EvaluationMode, UnpackPosition}; @@ -288,11 +288,16 @@ pub(super) struct TypeInferenceBuilder<'db, 'ast> { multi_inference_state: MultiInferenceState, + /// If you cannot avoid the possibility of calling `infer(_type)_expression` multiple times for a given expression, + /// set this to `Get` after the expression has been inferred for the first time. + /// While this is `Get`, any expressions will be considered to have already been inferred. + inner_expression_inference_state: InnerExpressionInferenceState, + /// For function definitions, the undecorated type of the function. undecorated_type: Option>, - /// Did we merge in a sub-region with a cycle-recovery fallback, and if so, what kind? - cycle_recovery: Option>, + /// The fallback type for missing expressions/bindings/declarations or recursive type inference. + cycle_recovery: Option>, /// `true` if all places in this expression are definitely bound all_definitely_bound: bool, @@ -327,6 +332,7 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { called_functions: FxIndexSet::default(), deferred_state: DeferredExpressionState::None, multi_inference_state: MultiInferenceState::Panic, + inner_expression_inference_state: InnerExpressionInferenceState::Infer, expressions: FxHashMap::default(), string_annotations: FxHashSet::default(), bindings: VecMap::default(), @@ -340,15 +346,22 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { } } - fn extend_cycle_recovery(&mut self, other_recovery: Option>) { - match &mut self.cycle_recovery { - Some(recovery) => *recovery = recovery.merge(other_recovery), - recovery @ None => *recovery = other_recovery, - } + fn fallback_type(&self) -> Option> { + self.cycle_recovery } - fn fallback_type(&self) -> Option> { - self.cycle_recovery.map(CycleRecovery::fallback_type) + fn extend_cycle_recovery(&mut self, other: Option>) { + if let Some(other) = other { + match self.cycle_recovery { + Some(existing) => { + self.cycle_recovery = + Some(UnionType::from_elements(self.db(), [existing, other])); + } + None => { + self.cycle_recovery = Some(other); + } + } + } } fn extend_definition(&mut self, inference: &DefinitionInference<'db>) { @@ -1905,7 +1918,38 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { } fn infer_type_alias(&mut self, type_alias: &ast::StmtTypeAlias) { - self.infer_annotation_expression(&type_alias.value, DeferredExpressionState::None); + let value_ty = + self.infer_annotation_expression(&type_alias.value, DeferredExpressionState::None); + + // A type alias where a value type points to itself, i.e. the expanded type is `Divergent` is meaningless + // (but a type alias that expands to something like `list[Divergent]` may be a valid recursive type alias) + // and would lead to infinite recursion. Therefore, such type aliases should not be exposed. + // ```python + // type Itself = Itself # error: "Cyclic definition of `Itself`" + // type A = B # error: "Cyclic definition of `A`" + // type B = A # error: "Cyclic definition of `B`" + // type G[T] = G[T] # error: "Cyclic definition of `G`" + // type RecursiveList[T] = list[T | RecursiveList[T]] # OK + // type RecursiveList2[T] = list[RecursiveList2[T]] # It's not possible to create an element of this, but it's not an error for now + // type IntOr = int | IntOr # It's redundant, but OK for now + // type IntOrStr = int | StrOrInt # It's redundant, but OK + // type StrOrInt = str | IntOrStr # It's redundant, but OK + // ``` + let expanded = value_ty.inner_type().expand_eagerly(self.db()); + if expanded.is_divergent() { + if let Some(builder) = self + .context + .report_lint(&CYCLIC_TYPE_ALIAS_DEFINITION, type_alias) + { + builder.into_diagnostic(format_args!( + "Cyclic definition of `{}`", + &type_alias.name.as_name_expr().unwrap().id, + )); + } + // Replace with `Divergent`. + self.expressions + .insert(type_alias.value.as_ref().into(), expanded); + } } /// If the current scope is a method inside an enclosing class, @@ -1974,6 +2018,7 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { // `@overload`ed functions without a body in unreachable code. true } + Type::Dynamic(DynamicType::Divergent(_)) => true, _ => false, } }) @@ -5978,6 +6023,7 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { let inferred = infer_definition_types(self.db(), *definition); // Check non-star imports for deprecations if definition.kind(self.db()).as_star_import().is_none() { + // In the initial cycle, `declaration_types()` is empty, so no deprecation check is performed. for ty in inferred.declaration_types() { self.check_deprecated(alias, ty.inner); } @@ -6957,6 +7003,9 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { expression: &ast::Expr, tcx: TypeContext<'db>, ) -> Type<'db> { + if self.inner_expression_inference_state.is_get() { + return self.expression_type(expression); + } let ty = match expression { ast::Expr::NoneLiteral(ast::ExprNoneLiteral { range: _, @@ -11725,6 +11774,7 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { typevar_binding_context: _, deferred_state: _, multi_inference_state: _, + inner_expression_inference_state: _, called_functions: _, index: _, region: _, @@ -11791,6 +11841,7 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { typevar_binding_context: _, deferred_state: _, multi_inference_state: _, + inner_expression_inference_state: _, called_functions: _, index: _, region: _, @@ -11867,6 +11918,7 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { typevar_binding_context: _, deferred_state: _, multi_inference_state: _, + inner_expression_inference_state: _, called_functions: _, index: _, region: _, @@ -11941,6 +11993,19 @@ impl MultiInferenceState { } } +#[derive(Default, Debug, Clone, Copy)] +enum InnerExpressionInferenceState { + #[default] + Infer, + Get, +} + +impl InnerExpressionInferenceState { + const fn is_get(self) -> bool { + matches!(self, InnerExpressionInferenceState::Get) + } +} + /// The deferred state of a specific expression in an inference region. #[derive(Default, Debug, Clone, Copy)] enum DeferredExpressionState { diff --git a/crates/ty_python_semantic/src/types/infer/builder/type_expression.rs b/crates/ty_python_semantic/src/types/infer/builder/type_expression.rs index fde99daf92..7c2addf9f1 100644 --- a/crates/ty_python_semantic/src/types/infer/builder/type_expression.rs +++ b/crates/ty_python_semantic/src/types/infer/builder/type_expression.rs @@ -6,20 +6,24 @@ use crate::types::diagnostic::{ self, INVALID_TYPE_FORM, NON_SUBSCRIPTABLE, report_invalid_argument_number_to_special_form, report_invalid_arguments_to_callable, }; -use crate::types::signatures::Signature; +use crate::types::infer::builder::InnerExpressionInferenceState; +use crate::types::signatures::{Parameter, Parameters, Signature}; use crate::types::string_annotation::parse_string_annotation; use crate::types::tuple::{TupleSpecBuilder, TupleType}; use crate::types::visitor::any_over_type; use crate::types::{ CallableType, DynamicType, IntersectionBuilder, KnownClass, KnownInstanceType, - LintDiagnosticGuard, Parameter, Parameters, SpecialFormType, SubclassOfType, Type, - TypeAliasType, TypeContext, TypeIsType, UnionBuilder, UnionType, todo_type, + LintDiagnosticGuard, SpecialFormType, SubclassOfType, Type, TypeAliasType, TypeContext, + TypeIsType, UnionBuilder, UnionType, todo_type, }; /// Type expressions impl<'db> TypeInferenceBuilder<'db, '_> { /// Infer the type of a type expression. pub(super) fn infer_type_expression(&mut self, expression: &ast::Expr) -> Type<'db> { + if self.inner_expression_inference_state.is_get() { + return self.expression_type(expression); + } let previous_deferred_state = self.deferred_state; // `DeferredExpressionState::InStringAnnotation` takes precedence over other states. @@ -33,13 +37,9 @@ impl<'db> TypeInferenceBuilder<'db, '_> { } DeferredExpressionState::InStringAnnotation(_) | DeferredExpressionState::Deferred => {} } - let mut ty = self.infer_type_expression_no_store(expression); - self.deferred_state = previous_deferred_state; - let divergent = Type::divergent(Some(self.scope())); - if ty.has_divergent_type(self.db(), divergent) { - ty = divergent; - } + let ty = self.infer_type_expression_no_store(expression); + self.deferred_state = previous_deferred_state; self.store_expression_type(expression, ty); ty } @@ -82,6 +82,9 @@ impl<'db> TypeInferenceBuilder<'db, '_> { /// Infer the type of a type expression without storing the result. pub(super) fn infer_type_expression_no_store(&mut self, expression: &ast::Expr) -> Type<'db> { + if self.inner_expression_inference_state.is_get() { + return self.expression_type(expression); + } // https://typing.python.org/en/latest/spec/annotations.html#grammar-token-expression-grammar-type_expression match expression { ast::Expr::Name(name) => match name.ctx { @@ -605,7 +608,7 @@ impl<'db> TypeInferenceBuilder<'db, '_> { // TODO: emit a diagnostic } } else { - element_types.push(element_ty.fallback_to_divergent(self.db())); + element_types.push(element_ty); } } @@ -960,6 +963,22 @@ impl<'db> TypeInferenceBuilder<'db, '_> { // For stringified TypeAlias; remove once properly supported todo_type!("string literal subscripted in type expression") } + Type::Union(union) => { + self.infer_type_expression(slice); + let previous_slice_inference_state = std::mem::replace( + &mut self.inner_expression_inference_state, + InnerExpressionInferenceState::Get, + ); + let union = union + .elements(self.db()) + .iter() + .fold(UnionBuilder::new(self.db()), |builder, elem| { + builder.add(self.infer_subscript_type_expression(subscript, *elem)) + }) + .build(); + self.inner_expression_inference_state = previous_slice_inference_state; + union + } _ => { self.infer_type_expression(slice); if let Some(builder) = self.context.report_lint(&INVALID_TYPE_FORM, subscript) { diff --git a/crates/ty_python_semantic/src/types/instance.rs b/crates/ty_python_semantic/src/types/instance.rs index 04a560be83..028ce648cd 100644 --- a/crates/ty_python_semantic/src/types/instance.rs +++ b/crates/ty_python_semantic/src/types/instance.rs @@ -11,7 +11,7 @@ use crate::types::constraints::{ConstraintSet, IteratorConstraintsExtension}; use crate::types::enums::is_single_member_enum; use crate::types::generics::{InferableTypeVars, walk_specialization}; use crate::types::protocol_class::{ProtocolClass, walk_protocol_interface}; -use crate::types::tuple::{TupleSpec, TupleType}; +use crate::types::tuple::{TupleSpec, TupleType, walk_tuple_type}; use crate::types::{ ApplyTypeMappingVisitor, ClassBase, ClassLiteral, FindLegacyTypeVarsVisitor, HasRelationToVisitor, IsDisjointVisitor, IsEquivalentVisitor, NormalizedVisitor, TypeContext, @@ -76,10 +76,7 @@ impl<'db> Type<'db> { { Type::tuple(TupleType::heterogeneous( db, - elements - .into_iter() - .map(Into::into) - .map(|element| element.fallback_to_divergent(db)), + elements.into_iter().map(Into::into), )) } @@ -207,7 +204,15 @@ pub(super) fn walk_nominal_instance_type<'db, V: super::visitor::TypeVisitor<'db nominal: NominalInstanceType<'db>, visitor: &V, ) { - visitor.visit_type(db, nominal.class(db).into()); + match nominal.0 { + NominalInstanceInner::ExactTuple(tuple) => { + walk_tuple_type(db, tuple, visitor); + } + NominalInstanceInner::Object => {} + NominalInstanceInner::NonTuple(class) => { + visitor.visit_type(db, class.into()); + } + } } impl<'db> NominalInstanceType<'db> { @@ -369,6 +374,26 @@ impl<'db> NominalInstanceType<'db> { } } + pub(super) fn recursive_type_normalized_impl( + self, + db: &'db dyn Db, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> Option { + match self.0 { + NominalInstanceInner::ExactTuple(tuple) => { + Some(Self(NominalInstanceInner::ExactTuple( + tuple.recursive_type_normalized_impl(db, div, nested, visitor)?, + ))) + } + NominalInstanceInner::NonTuple(class) => Some(Self(NominalInstanceInner::NonTuple( + class.recursive_type_normalized_impl(db, div, nested, visitor)?, + ))), + NominalInstanceInner::Object => Some(Self(NominalInstanceInner::Object)), + } + } + pub(super) fn has_relation_to_impl( self, db: &'db dyn Db, @@ -720,6 +745,21 @@ impl<'db> ProtocolInstanceType<'db> { } } + pub(super) fn recursive_type_normalized_impl( + self, + db: &'db dyn Db, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> Option { + Some(Self { + inner: self + .inner + .recursive_type_normalized_impl(db, div, nested, visitor)?, + _phantom: PhantomData, + }) + } + /// Return `true` if this protocol type is equivalent to the protocol `other`. /// /// TODO: consider the types of the members as well as their existence @@ -831,6 +871,23 @@ impl<'db> Protocol<'db> { Self::Synthesized(synthesized) => synthesized.interface(), } } + + fn recursive_type_normalized_impl( + self, + db: &'db dyn Db, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> Option { + match self { + Self::FromClass(class) => Some(Self::FromClass( + class.recursive_type_normalized_impl(db, div, nested, visitor)?, + )), + Self::Synthesized(synthesized) => Some(Self::Synthesized( + synthesized.recursive_type_normalized_impl(db, div, nested, visitor)?, + )), + } + } } impl<'db> VarianceInferable<'db> for Protocol<'db> { @@ -849,7 +906,7 @@ mod synthesized_protocol { use crate::types::protocol_class::ProtocolInterface; use crate::types::{ ApplyTypeMappingVisitor, BoundTypeVarInstance, FindLegacyTypeVarsVisitor, - NormalizedVisitor, TypeContext, TypeMapping, TypeVarVariance, VarianceInferable, + NormalizedVisitor, Type, TypeContext, TypeMapping, TypeVarVariance, VarianceInferable, }; use crate::{Db, FxOrderSet}; @@ -900,6 +957,19 @@ mod synthesized_protocol { pub(in crate::types) fn interface(self) -> ProtocolInterface<'db> { self.0 } + + pub(in crate::types) fn recursive_type_normalized_impl( + self, + db: &'db dyn Db, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> Option { + Some(Self( + self.0 + .recursive_type_normalized_impl(db, div, nested, visitor)?, + )) + } } impl<'db> VarianceInferable<'db> for SynthesizedProtocolType<'db> { diff --git a/crates/ty_python_semantic/src/types/member.rs b/crates/ty_python_semantic/src/types/member.rs index 4090533221..11a8648f50 100644 --- a/crates/ty_python_semantic/src/types/member.rs +++ b/crates/ty_python_semantic/src/types/member.rs @@ -8,7 +8,7 @@ use crate::types::Type; /// The return type of certain member-lookup operations. Contains information /// about the type, type qualifiers, boundness/declaredness. -#[derive(Debug, Clone, PartialEq, Eq, salsa::Update, get_size2::GetSize, Default)] +#[derive(Debug, Clone, Copy, PartialEq, Eq, salsa::Update, get_size2::GetSize, Default)] pub(super) struct Member<'db> { /// Type, qualifiers, and boundness information of this member pub(super) inner: PlaceAndQualifiers<'db>, diff --git a/crates/ty_python_semantic/src/types/newtype.rs b/crates/ty_python_semantic/src/types/newtype.rs index fe08fa7bee..84a6e18f50 100644 --- a/crates/ty_python_semantic/src/types/newtype.rs +++ b/crates/ty_python_semantic/src/types/newtype.rs @@ -141,11 +141,11 @@ impl<'db> NewType<'db> { /// Create a new `NewType` by mapping the underlying `ClassType`. This descends through any /// number of nested `NewType` layers and rebuilds the whole chain. In the rare case of cyclic /// `NewType`s with no underlying `ClassType`, this has no effect and does not call `f`. - pub(crate) fn map_base_class_type( + pub(crate) fn try_map_base_class_type( self, db: &'db dyn Db, - f: impl FnOnce(ClassType<'db>) -> ClassType<'db>, - ) -> Self { + f: impl FnOnce(ClassType<'db>) -> Option>, + ) -> Option { // Modifying the base class type requires unwrapping and re-wrapping however many base // newtypes there are between here and there. Normally recursion would be natural for this, // but the bases iterator does cycle detection, and I think using that with a stack is a @@ -162,7 +162,7 @@ impl<'db> NewType<'db> { // We've reached the `ClassType`. NewTypeBase::ClassType(base_class_type) => { // Call `f`. - let mut mapped_base = NewTypeBase::ClassType(f(base_class_type)); + let mut mapped_base = NewTypeBase::ClassType(f(base_class_type)?); // Re-wrap the mapped base class in however many newtypes we unwrapped. for inner_newtype in inner_newtype_stack.into_iter().rev() { mapped_base = NewTypeBase::NewType(NewType::new( @@ -172,18 +172,27 @@ impl<'db> NewType<'db> { Some(mapped_base), )); } - return NewType::new( + return Some(NewType::new( db, self.name(db).clone(), self.definition(db), Some(mapped_base), - ); + )); } } } // If we get here, there is no `ClassType` (because this newtype is cyclic), and we don't // call `f` at all. - self + Some(self) + } + + pub(crate) fn map_base_class_type( + self, + db: &'db dyn Db, + f: impl FnOnce(ClassType<'db>) -> ClassType<'db>, + ) -> Self { + self.try_map_base_class_type(db, |class_type| Some(f(class_type))) + .unwrap() } } diff --git a/crates/ty_python_semantic/src/types/protocol_class.rs b/crates/ty_python_semantic/src/types/protocol_class.rs index 8bfc0525a6..48b76bf98d 100644 --- a/crates/ty_python_semantic/src/types/protocol_class.rs +++ b/crates/ty_python_semantic/src/types/protocol_class.rs @@ -144,6 +144,19 @@ impl<'db> ProtocolClass<'db> { .apply_type_mapping_impl(db, type_mapping, tcx, visitor), ) } + + pub(super) fn recursive_type_normalized_impl( + self, + db: &'db dyn Db, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> Option { + Some(Self( + self.0 + .recursive_type_normalized_impl(db, div, nested, visitor)?, + )) + } } impl<'db> Deref for ProtocolClass<'db> { @@ -365,6 +378,27 @@ impl<'db> ProtocolInterface<'db> { ) } + pub(super) fn recursive_type_normalized_impl( + self, + db: &'db dyn Db, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> Option { + Some(Self::new( + db, + self.inner(db) + .iter() + .map(|(name, data)| { + Some(( + name.clone(), + data.recursive_type_normalized_impl(db, div, nested, visitor)?, + )) + }) + .collect::>>()?, + )) + } + pub(super) fn specialized_and_normalized<'a>( self, db: &'db dyn Db, @@ -456,6 +490,33 @@ impl<'db> ProtocolMemberData<'db> { } } + fn recursive_type_normalized_impl( + &self, + db: &'db dyn Db, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> Option { + Some(Self { + kind: match &self.kind { + ProtocolMemberKind::Method(callable) => ProtocolMemberKind::Method( + callable.recursive_type_normalized_impl(db, div, nested, visitor)?, + ), + ProtocolMemberKind::Property(property) => ProtocolMemberKind::Property( + property.recursive_type_normalized_impl(db, div, nested, visitor)?, + ), + ProtocolMemberKind::Other(ty) if nested => ProtocolMemberKind::Other( + ty.recursive_type_normalized_impl(db, div, true, visitor)?, + ), + ProtocolMemberKind::Other(ty) => ProtocolMemberKind::Other( + ty.recursive_type_normalized_impl(db, div, true, visitor) + .unwrap_or(div), + ), + }, + qualifiers: self.qualifiers, + }) + } + fn apply_type_mapping_impl<'a>( &self, db: &'db dyn Db, diff --git a/crates/ty_python_semantic/src/types/signatures.rs b/crates/ty_python_semantic/src/types/signatures.rs index 0460f31f08..ebe2945693 100644 --- a/crates/ty_python_semantic/src/types/signatures.rs +++ b/crates/ty_python_semantic/src/types/signatures.rs @@ -174,6 +174,22 @@ impl<'db> CallableSignature<'db> { ) } + pub(super) fn recursive_type_normalized_impl( + &self, + db: &'db dyn Db, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> Option { + Some(Self { + overloads: self + .overloads + .iter() + .map(|signature| signature.recursive_type_normalized_impl(db, div, nested, visitor)) + .collect::>>()?, + }) + } + pub(crate) fn apply_type_mapping_impl<'a>( &self, db: &'db dyn Db, @@ -553,6 +569,36 @@ impl<'db> Signature<'db> { } } + pub(super) fn recursive_type_normalized_impl( + &self, + db: &'db dyn Db, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> Option { + let return_ty = match self.return_ty { + Some(return_ty) if nested => { + Some(return_ty.recursive_type_normalized_impl(db, div, true, visitor)?) + } + Some(return_ty) => Some( + return_ty + .recursive_type_normalized_impl(db, div, true, visitor) + .unwrap_or(div), + ), + None => None, + }; + Some(Self { + generic_context: self.generic_context, + definition: self.definition, + parameters: self + .parameters + .iter() + .map(|param| param.recursive_type_normalized_impl(db, div, nested, visitor)) + .collect::>()?, + return_ty, + }) + } + pub(crate) fn apply_type_mapping_impl<'a>( &self, db: &'db dyn Db, @@ -1853,6 +1899,85 @@ impl<'db> Parameter<'db> { } } + pub(super) fn recursive_type_normalized_impl( + &self, + db: &'db dyn Db, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> Option { + let Parameter { + annotated_type, + inferred_annotation, + kind, + form, + } = self; + + let annotated_type = match annotated_type { + Some(ty) if nested => Some(ty.recursive_type_normalized_impl(db, div, true, visitor)?), + Some(ty) => Some( + ty.recursive_type_normalized_impl(db, div, true, visitor) + .unwrap_or(div), + ), + None => None, + }; + + let kind = match kind { + ParameterKind::PositionalOnly { name, default_type } => ParameterKind::PositionalOnly { + name: name.clone(), + default_type: match default_type { + Some(ty) if nested => { + Some(ty.recursive_type_normalized_impl(db, div, true, visitor)?) + } + Some(ty) => Some( + ty.recursive_type_normalized_impl(db, div, true, visitor) + .unwrap_or(div), + ), + None => None, + }, + }, + ParameterKind::PositionalOrKeyword { name, default_type } => { + ParameterKind::PositionalOrKeyword { + name: name.clone(), + default_type: match default_type { + Some(ty) if nested => { + Some(ty.recursive_type_normalized_impl(db, div, true, visitor)?) + } + Some(ty) => Some( + ty.recursive_type_normalized_impl(db, div, true, visitor) + .unwrap_or(div), + ), + None => None, + }, + } + } + ParameterKind::KeywordOnly { name, default_type } => ParameterKind::KeywordOnly { + name: name.clone(), + default_type: match default_type { + Some(ty) if nested => { + Some(ty.recursive_type_normalized_impl(db, div, true, visitor)?) + } + Some(ty) => Some( + ty.recursive_type_normalized_impl(db, div, true, visitor) + .unwrap_or(div), + ), + None => None, + }, + }, + ParameterKind::Variadic { name } => ParameterKind::Variadic { name: name.clone() }, + ParameterKind::KeywordVariadic { name } => { + ParameterKind::KeywordVariadic { name: name.clone() } + } + }; + + Some(Self { + annotated_type, + inferred_annotation: *inferred_annotation, + kind, + form: *form, + }) + } + fn from_node_and_kind( db: &'db dyn Db, definition: Definition<'db>, diff --git a/crates/ty_python_semantic/src/types/subclass_of.rs b/crates/ty_python_semantic/src/types/subclass_of.rs index 687d5aaad0..b5b9e00c06 100644 --- a/crates/ty_python_semantic/src/types/subclass_of.rs +++ b/crates/ty_python_semantic/src/types/subclass_of.rs @@ -194,6 +194,20 @@ impl<'db> SubclassOfType<'db> { } } + pub(super) fn recursive_type_normalized_impl( + self, + db: &'db dyn Db, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> Option { + Some(Self { + subclass_of: self + .subclass_of + .recursive_type_normalized_impl(db, div, nested, visitor)?, + }) + } + pub(crate) fn to_instance(self, db: &'db dyn Db) -> Type<'db> { match self.subclass_of { SubclassOfInner::Class(class) => Type::instance(db, class), @@ -234,7 +248,7 @@ impl<'db> VarianceInferable<'db> for SubclassOfType<'db> { #[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, salsa::Update, get_size2::GetSize)] pub(crate) enum SubclassOfInner<'db> { Class(ClassType<'db>), - Dynamic(DynamicType<'db>), + Dynamic(DynamicType), } impl<'db> SubclassOfInner<'db> { @@ -253,7 +267,7 @@ impl<'db> SubclassOfInner<'db> { } } - pub(crate) const fn into_dynamic(self) -> Option> { + pub(crate) const fn into_dynamic(self) -> Option { match self { Self::Class(_) => None, Self::Dynamic(dynamic) => Some(dynamic), @@ -267,6 +281,21 @@ impl<'db> SubclassOfInner<'db> { } } + pub(super) fn recursive_type_normalized_impl( + self, + db: &'db dyn Db, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> Option { + match self { + Self::Class(class) => Some(Self::Class( + class.recursive_type_normalized_impl(db, div, nested, visitor)?, + )), + Self::Dynamic(dynamic) => Some(Self::Dynamic(dynamic.recursive_type_normalized())), + } + } + pub(crate) fn try_from_type(db: &'db dyn Db, ty: Type<'db>) -> Option { match ty { Type::Dynamic(dynamic) => Some(Self::Dynamic(dynamic)), @@ -284,8 +313,8 @@ impl<'db> From> for SubclassOfInner<'db> { } } -impl<'db> From> for SubclassOfInner<'db> { - fn from(value: DynamicType<'db>) -> Self { +impl From for SubclassOfInner<'_> { + fn from(value: DynamicType) -> Self { SubclassOfInner::Dynamic(value) } } diff --git a/crates/ty_python_semantic/src/types/tuple.rs b/crates/ty_python_semantic/src/types/tuple.rs index e773315d68..5fb8fc982a 100644 --- a/crates/ty_python_semantic/src/types/tuple.rs +++ b/crates/ty_python_semantic/src/types/tuple.rs @@ -229,6 +229,20 @@ impl<'db> TupleType<'db> { TupleType::new(db, &self.tuple(db).normalized_impl(db, visitor)) } + pub(super) fn recursive_type_normalized_impl( + self, + db: &'db dyn Db, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> Option { + Some(Self::new_internal( + db, + self.tuple(db) + .recursive_type_normalized_impl(db, div, nested, visitor)?, + )) + } + pub(crate) fn apply_type_mapping_impl<'a>( self, db: &'db dyn Db, @@ -292,7 +306,7 @@ impl<'db> TupleType<'db> { fn to_class_type_cycle_initial<'db>( db: &'db dyn Db, - _id: salsa::Id, + id: salsa::Id, self_: TupleType<'db>, ) -> ClassType<'db> { let tuple_class = KnownClass::Tuple @@ -301,7 +315,7 @@ fn to_class_type_cycle_initial<'db>( tuple_class.apply_specialization(db, |generic_context| { if generic_context.variables(db).len() == 1 { - generic_context.specialize_tuple(db, Type::Never, self_) + generic_context.specialize_tuple(db, Type::divergent(id), self_) } else { generic_context.default_specialization(db, Some(KnownClass::Tuple)) } @@ -392,6 +406,33 @@ impl<'db> FixedLengthTuple> { Self::from_elements(self.0.iter().map(|ty| ty.normalized_impl(db, visitor))) } + fn recursive_type_normalized_impl( + &self, + db: &'db dyn Db, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> Option { + if nested { + Some(Self::from_elements( + self.0 + .iter() + .map(|ty| ty.recursive_type_normalized_impl(db, div, true, visitor)) + .collect::>>()?, + )) + } else { + Some(Self::from_elements( + self.0 + .iter() + .map(|ty| { + ty.recursive_type_normalized_impl(db, div, true, visitor) + .unwrap_or(div) + }) + .collect::>(), + )) + } + } + fn apply_type_mapping_impl<'a>( &self, db: &'db dyn Db, @@ -758,6 +799,56 @@ impl<'db> VariableLengthTuple> { }) } + fn recursive_type_normalized_impl( + &self, + db: &'db dyn Db, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> Option { + let prefix = if nested { + self.prefix + .iter() + .map(|ty| ty.recursive_type_normalized_impl(db, div, true, visitor)) + .collect::>>()? + } else { + self.prefix + .iter() + .map(|ty| { + ty.recursive_type_normalized_impl(db, div, true, visitor) + .unwrap_or(div) + }) + .collect::>() + }; + let suffix = if nested { + self.suffix + .iter() + .map(|ty| ty.recursive_type_normalized_impl(db, div, true, visitor)) + .collect::>>()? + } else { + self.suffix + .iter() + .map(|ty| { + ty.recursive_type_normalized_impl(db, div, true, visitor) + .unwrap_or(div) + }) + .collect::>() + }; + let variable = if nested { + self.variable + .recursive_type_normalized_impl(db, div, true, visitor)? + } else { + self.variable + .recursive_type_normalized_impl(db, div, true, visitor) + .unwrap_or(div) + }; + Some(Self { + prefix, + variable, + suffix, + }) + } + fn apply_type_mapping_impl<'a>( &self, db: &'db dyn Db, @@ -1154,6 +1245,23 @@ impl<'db> Tuple> { } } + pub(super) fn recursive_type_normalized_impl( + &self, + db: &'db dyn Db, + div: Type<'db>, + nested: bool, + visitor: &NormalizedVisitor<'db>, + ) -> Option { + match self { + Tuple::Fixed(tuple) => Some(Tuple::Fixed( + tuple.recursive_type_normalized_impl(db, div, nested, visitor)?, + )), + Tuple::Variable(tuple) => Some(Tuple::Variable( + tuple.recursive_type_normalized_impl(db, div, nested, visitor)?, + )), + } + } + pub(crate) fn apply_type_mapping_impl<'a>( &self, db: &'db dyn Db, diff --git a/crates/ty_python_semantic/src/types/type_ordering.rs b/crates/ty_python_semantic/src/types/type_ordering.rs index 1b9b00a8fd..cb2f75b108 100644 --- a/crates/ty_python_semantic/src/types/type_ordering.rs +++ b/crates/ty_python_semantic/src/types/type_ordering.rs @@ -269,9 +269,7 @@ fn dynamic_elements_ordering(left: DynamicType, right: DynamicType) -> Ordering (DynamicType::TodoUnpack, _) => Ordering::Less, (_, DynamicType::TodoUnpack) => Ordering::Greater, - (DynamicType::Divergent(left), DynamicType::Divergent(right)) => { - left.scope.cmp(&right.scope) - } + (DynamicType::Divergent(left), DynamicType::Divergent(right)) => left.cmp(&right), (DynamicType::Divergent(_), _) => Ordering::Less, (_, DynamicType::Divergent(_)) => Ordering::Greater, } diff --git a/crates/ty_python_semantic/src/types/unpacker.rs b/crates/ty_python_semantic/src/types/unpacker.rs index 9af330d8e0..eddd3aab0b 100644 --- a/crates/ty_python_semantic/src/types/unpacker.rs +++ b/crates/ty_python_semantic/src/types/unpacker.rs @@ -180,10 +180,11 @@ impl<'db, 'ast> Unpacker<'db, 'ast> { pub(crate) fn finish(mut self) -> UnpackResult<'db> { self.targets.shrink_to_fit(); + UnpackResult { diagnostics: self.context.finish(), targets: self.targets, - cycle_fallback_type: None, + cycle_recovery: None, } } } @@ -196,7 +197,7 @@ pub(crate) struct UnpackResult<'db> { /// The fallback type for missing expressions. /// /// This is used only when constructing a cycle-recovery `UnpackResult`. - cycle_fallback_type: Option>, + cycle_recovery: Option>, } impl<'db> UnpackResult<'db> { @@ -222,7 +223,7 @@ impl<'db> UnpackResult<'db> { self.targets .get(&expr.into()) .copied() - .or(self.cycle_fallback_type) + .or(self.cycle_recovery) } /// Returns the diagnostics in this unpacking assignment. @@ -230,11 +231,25 @@ impl<'db> UnpackResult<'db> { &self.diagnostics } - pub(crate) fn cycle_initial(cycle_fallback_type: Type<'db>) -> Self { + pub(crate) fn cycle_initial(cycle_recovery: Type<'db>) -> Self { Self { targets: FxHashMap::default(), diagnostics: TypeCheckDiagnostics::default(), - cycle_fallback_type: Some(cycle_fallback_type), + cycle_recovery: Some(cycle_recovery), } } + + pub(crate) fn cycle_normalized( + mut self, + db: &'db dyn Db, + previous_cycle_result: &UnpackResult<'db>, + cycle: &salsa::Cycle, + ) -> Self { + for (expr, ty) in &mut self.targets { + let previous_ty = previous_cycle_result.expression_type(*expr); + *ty = ty.cycle_normalized(db, previous_ty, cycle); + } + + self + } } diff --git a/crates/ty_python_semantic/src/types/visitor.rs b/crates/ty_python_semantic/src/types/visitor.rs index 7692c205ff..54ce30cc53 100644 --- a/crates/ty_python_semantic/src/types/visitor.rs +++ b/crates/ty_python_semantic/src/types/visitor.rs @@ -1,5 +1,3 @@ -use rustc_hash::FxHashMap; - use crate::{ Db, FxIndexSet, types::{ @@ -19,10 +17,7 @@ use crate::{ walk_typed_dict_type, walk_typeis_type, walk_union, }, }; -use std::{ - cell::{Cell, RefCell}, - collections::hash_map::Entry, -}; +use std::cell::{Cell, RefCell}; /// A visitor trait that recurses into nested types. /// @@ -330,148 +325,3 @@ pub(super) fn any_over_type<'db>( visitor.visit_type(db, ty); visitor.found_matching_type.get() } - -/// Returns the maximum number of layers of generic specializations for a given type. -/// -/// For example, `int` has a depth of `0`, `list[int]` has a depth of `1`, and `list[set[int]]` -/// has a depth of `2`. A set-theoretic type like `list[int] | list[list[int]]` has a maximum -/// depth of `2`. -fn specialization_depth(db: &dyn Db, ty: Type<'_>) -> usize { - #[derive(Debug, Default)] - struct SpecializationDepthVisitor<'db> { - seen_types: RefCell, Option>>, - max_depth: Cell, - } - - impl<'db> TypeVisitor<'db> for SpecializationDepthVisitor<'db> { - fn should_visit_lazy_type_attributes(&self) -> bool { - false - } - - fn visit_type(&self, db: &'db dyn Db, ty: Type<'db>) { - match TypeKind::from(ty) { - TypeKind::Atomic => { - if ty.is_divergent() { - self.max_depth.set(usize::MAX); - } - } - TypeKind::NonAtomic(non_atomic_type) => { - match self.seen_types.borrow_mut().entry(non_atomic_type) { - Entry::Occupied(cached_depth) => { - self.max_depth - .update(|current| current.max(cached_depth.get().unwrap_or(0))); - return; - } - Entry::Vacant(entry) => { - entry.insert(None); - } - } - - let self_depth: usize = - matches!(non_atomic_type, NonAtomicType::GenericAlias(_)).into(); - - let previous_max_depth = self.max_depth.replace(0); - walk_non_atomic_type(db, non_atomic_type, self); - - self.max_depth.update(|max_child_depth| { - previous_max_depth.max(max_child_depth.saturating_add(self_depth)) - }); - - self.seen_types - .borrow_mut() - .insert(non_atomic_type, Some(self.max_depth.get())); - } - } - } - } - - let visitor = SpecializationDepthVisitor::default(); - visitor.visit_type(db, ty); - visitor.max_depth.get() -} - -pub(super) fn exceeds_max_specialization_depth(db: &dyn Db, ty: Type<'_>) -> bool { - // To prevent infinite recursion during type inference for infinite types, we fall back to - // `C[Divergent]` once a certain amount of levels of specialization have occurred. For - // example: - // - // ```py - // x = 1 - // while random_bool(): - // x = [x] - // - // reveal_type(x) # Unknown | Literal[1] | list[Divergent] - // ``` - const MAX_SPECIALIZATION_DEPTH: usize = 10; - - specialization_depth(db, ty) > MAX_SPECIALIZATION_DEPTH -} - -#[cfg(test)] -mod tests { - use super::*; - use crate::{db::tests::setup_db, types::KnownClass}; - - #[test] - fn test_generics_layering_depth() { - let db = setup_db(); - - let int = || KnownClass::Int.to_instance(&db); - let list = |element| KnownClass::List.to_specialized_instance(&db, [element]); - let dict = |key, value| KnownClass::Dict.to_specialized_instance(&db, [key, value]); - let set = |element| KnownClass::Set.to_specialized_instance(&db, [element]); - let str = || KnownClass::Str.to_instance(&db); - let bytes = || KnownClass::Bytes.to_instance(&db); - - let list_of_int = list(int()); - assert_eq!(specialization_depth(&db, list_of_int), 1); - - let list_of_list_of_int = list(list_of_int); - assert_eq!(specialization_depth(&db, list_of_list_of_int), 2); - - let list_of_list_of_list_of_int = list(list_of_list_of_int); - assert_eq!(specialization_depth(&db, list_of_list_of_list_of_int), 3); - - assert_eq!(specialization_depth(&db, set(dict(str(), list_of_int))), 3); - - assert_eq!( - specialization_depth( - &db, - UnionType::from_elements(&db, [list_of_list_of_list_of_int, list_of_list_of_int]) - ), - 3 - ); - - assert_eq!( - specialization_depth( - &db, - UnionType::from_elements(&db, [list_of_list_of_int, list_of_list_of_list_of_int]) - ), - 3 - ); - - assert_eq!( - specialization_depth( - &db, - Type::heterogeneous_tuple(&db, [Type::heterogeneous_tuple(&db, [int()])]) - ), - 2 - ); - - assert_eq!( - specialization_depth(&db, Type::heterogeneous_tuple(&db, [list_of_int, str()])), - 2 - ); - - assert_eq!( - specialization_depth( - &db, - list(UnionType::from_elements( - &db, - [list(int()), list(str()), list(bytes())] - )) - ), - 2 - ); - } -} diff --git a/crates/ty_python_semantic/tests/corpus.rs b/crates/ty_python_semantic/tests/corpus.rs index db2b1f3342..c09929bcbd 100644 --- a/crates/ty_python_semantic/tests/corpus.rs +++ b/crates/ty_python_semantic/tests/corpus.rs @@ -169,9 +169,6 @@ fn run_corpus_tests(pattern: &str) -> anyhow::Result<()> { /// Whether or not the .py/.pyi version of this file is expected to fail #[rustfmt::skip] const KNOWN_FAILURES: &[(&str, bool, bool)] = &[ - // Fails with too-many-cycle-iterations due to a self-referential - // type alias, see https://github.com/astral-sh/ty/issues/256 - ("crates/ruff_linter/resources/test/fixtures/pyflakes/F401_34.py", true, true), ]; #[salsa::db] diff --git a/crates/ty_server/tests/e2e/snapshots/e2e__commands__debug_command.snap b/crates/ty_server/tests/e2e/snapshots/e2e__commands__debug_command.snap index e2480cae4e..b6c245a91f 100644 --- a/crates/ty_server/tests/e2e/snapshots/e2e__commands__debug_command.snap +++ b/crates/ty_server/tests/e2e/snapshots/e2e__commands__debug_command.snap @@ -35,6 +35,7 @@ Settings: Settings { "conflicting-declarations": Error (Default), "conflicting-metaclass": Error (Default), "cyclic-class-definition": Error (Default), + "cyclic-type-alias-definition": Error (Default), "deprecated": Warning (Default), "duplicate-base": Error (Default), "duplicate-kw-only": Error (Default), diff --git a/ty.schema.json b/ty.schema.json index fda46afe4b..edc493f2e3 100644 --- a/ty.schema.json +++ b/ty.schema.json @@ -373,6 +373,16 @@ } ] }, + "cyclic-type-alias-definition": { + "title": "detects cyclic type alias definitions", + "description": "## What it does\nChecks for type alias definitions that (directly or mutually) refer to themselves.\n\n## Why is it bad?\nAlthough it is permitted to define a recursive type alias, it is not meaningful\nto have a type alias whose expansion can only result in itself, and is therefore not allowed.\n\n## Examples\n```python\ntype Itself = Itself\n\ntype A = B\ntype B = A\n```", + "default": "error", + "oneOf": [ + { + "$ref": "#/definitions/Level" + } + ] + }, "deprecated": { "title": "detects uses of deprecated items", "description": "## What it does\nChecks for uses of deprecated items\n\n## Why is this bad?\nDeprecated items should no longer be used.\n\n## Examples\n```python\n@warnings.deprecated(\"use new_func instead\")\ndef old_func(): ...\n\nold_func() # emits [deprecated] diagnostic\n```",