Compare commits

...

16 Commits

Author SHA1 Message Date
Dhruv Manilawala
752ad4d579 Handle untitled files in Neovim 2025-02-23 10:49:01 +05:30
Alex Waygood
64effa4aea [red-knot] Add a regression test for recent improvement to TypeInferenceBuilder::infer_name_load() (#16310) 2025-02-21 22:28:42 +00:00
Alex Waygood
224a36f5f3 Teach red-knot that type(x) is the same as x.__class__ (#16301) 2025-02-21 21:05:48 +00:00
Alex Waygood
5347abc766 [red-knot] Generalise special-casing for KnownClasses in Type::bool (#16300) 2025-02-21 20:46:36 +00:00
Micha Reiser
5fab97f1ef [red-knot] Diagnostics for incorrect bool usages (#16238) 2025-02-21 19:26:05 +01:00
David Peter
3aa7ba31b1 [red-knot] Fix descriptor __get__ call on class objects (#16304)
## Summary

I spotted a minor mistake in my descriptor protocol implementation where
`C.descriptor` would pass the meta type (`type`) of the type of `C`
(`Literal[C]`) as the owner argument to `__get__`, instead of passing
`Literal[C]` directly.

## Test Plan

New test.
2025-02-21 15:35:41 +01:00
Douglas Creager
4dae09ecff [red-knot] Better handling of visibility constraint copies (#16276)
Two related changes.  For context:

1. We were maintaining two separate arenas of `Constraint`s in each
use-def map. One was used for narrowing constraints, and the other for
visibility constraints. The visibility constraint arena was interned,
ensuring that we always used the same ID for any particular
`Constraint`. The narrowing constraint arena was not interned.

2. The TDD code relies on _all_ TDD nodes being interned and reduced.
This is an important requirement for TDDs to be a canonical form, which
allows us to use a single int comparison to test for "always true/false"
and to compare two TDDs for equivalence. But we also need to support an
individual `Constraint` having multiple values in a TDD evaluation (e.g.
to handle a `while` condition having different values the first time
it's evaluated vs later times). Previously, we handled that by
introducing a "copy" number, which was only there as a disambiguator, to
allow an interned, deduplicated constraint ID to appear in the TDD
formula multiple times.

A better way to handle (2) is to not intern the constraints in the
visibility constraint arena! The caller now gets to decide: if they add
a `Constraint` to the arena more than once, they get distinct
`ScopedConstraintId`s — which the TDD code will treat as distinct
variables, allowing them to take on different values in the ternary
function.

With that in place, we can then consolidate on a single (non-interned)
arena, which is shared for both narrowing and visibility constraints.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2025-02-21 09:16:25 -05:00
Darius Carrier
b9b094869a [pylint] Fix false positives, add missing methods, and support positional-only parameters (PLE0302) (#16263)
## Summary

Resolves 3/4 requests in #16217:

-  Remove not special methods: `__cmp__`, `__div__`, `__nonzero__`, and
`__unicode__`.
-  Add special methods: `__next__`, `__buffer__`, `__class_getitem__`,
`__mro_entries__`, `__release_buffer__`, and `__subclasshook__`.
-  Support positional-only arguments.
-  Add support for module functions `__dir__` and `__getattr__`. As
mentioned in the issue the check is scoped for methods rather than
module functions. I am hesitant to expand the scope of this check
without a discussion.

## Test Plan

- Manually confirmed each example file from the issue functioned as
expected.
- Ran cargo nextest to ensure `unexpected_special_method_signature` test
still passed.

Fixes #16217.
2025-02-21 08:38:51 -05:00
Alex Waygood
b3c5932fda [red-knot] Restrict visibility of the module_type_symbols function (#16290) 2025-02-21 10:55:22 +00:00
Alex Waygood
fe3ae587ea [red-knot] Fix subtle detail in where the types.ModuleType attribute lookup should happen in TypeInferenceBuilder::infer_name_load() (#16284) 2025-02-21 10:48:52 +00:00
Dhruv Manilawala
c2b9fa84f7 Refactor workspace logic into workspace.rs (#16295)
## Summary

This is just a small refactor to move workspace related structs and impl
out from `server.rs` where `Server` is defined and into a new
`workspace.rs`.
2025-02-21 08:37:29 +00:00
Victorien
793264db13 [ruff] Add more Pydantic models variants to the list of default copy semantics (RUF012) (#16291) 2025-02-21 08:28:13 +01:00
Carl Meyer
4d63c16c19 [red-knot] update to latest Salsa (#16293)
Update to latest Salsa main branch. This provides a point of comparison
for the perf impact of fixpoint iteration, which is based on latest
Salsa main.

This requires an update to the locked version of our boxcar dep, since
Salsa now depends on a newer version of boxcar.
2025-02-20 18:15:58 -08:00
David Peter
d2e034adcd [red-knot] Method calls and the descriptor protocol (#16121)
## Summary

This PR achieves the following:

* Add support for checking method calls, and inferring return types from
method calls. For example:
  ```py
  reveal_type("abcde".find("abc"))  # revealed: int
  reveal_type("foo".encode(encoding="utf-8"))  # revealed: bytes
  
  "abcde".find(123)  # error: [invalid-argument-type]
  
  class C:
      def f(self) -> int:
          pass
  
  reveal_type(C.f)  # revealed: <function `f`>
  reveal_type(C().f)  # revealed: <bound method: `f` of `C`>
  
  C.f()  # error: [missing-argument]
  reveal_type(C().f())  # revealed: int
  ```
* Implement the descriptor protocol, i.e. properly call the `__get__`
method when a descriptor object is accessed through a class object or an
instance of a class. For example:
  ```py
  from typing import Literal
  
  class Ten:
def __get__(self, instance: object, owner: type | None = None) ->
Literal[10]:
          return 10
  
  class C:
      ten: Ten = Ten()
  
  reveal_type(C.ten)  # revealed: Literal[10]
  reveal_type(C().ten)  # revealed: Literal[10]
  ```
* Add support for member lookup on intersection types.
* Support type inference for `inspect.getattr_static(obj, attr)` calls.
This was mostly used as a debugging tool during development, but seems
more generally useful. It can be used to bypass the descriptor protocol.
For the example above:
  ```py
  from inspect import getattr_static
  
  reveal_type(getattr_static(C, "ten"))  # revealed: Ten
  ```
* Add a new `Type::Callable(…)` variant with the following sub-variants:
* `Type::Callable(CallableType::BoundMethod(…))` — represents bound
method objects, e.g. `C().f` above
* `Type::Callable(CallableType::MethodWrapperDunderGet(…))` — represents
`f.__get__` where `f` is a function
* `Type::Callable(WrapperDescriptorDunderGet)` — represents
`FunctionType.__get__`
* Add new known classes:
  * `types.MethodType`
  * `types.MethodWrapperType`
  * `types.WrapperDescriptorType`
  * `builtins.range`

## Performance analysis

On this branch, we do more work. We need to do more call checking, since
we now check all method calls. We also need to do ~twice as many member
lookups, because we need to check if a `__get__` attribute exists on
accessed members.

A brief analysis on `tomllib` shows that we now call `Type::call` 1780
times, compared to 612 calls before.

## Limitations

* Data descriptors are not yet supported, i.e. we do not infer correct
types for descriptor attribute accesses in `Store` context and do not
check writes to descriptor attributes. I felt like this was something
that could be split out as a follow-up without risking a major
architectural change.
* We currently distinguish between `Type::member` (with descriptor
protocol) and `Type::static_member` (without descriptor protocol). The
former corresponds to `obj.attr`, the latter corresponds to
`getattr_static(obj, "attr")`. However, to model some details correctly,
we would also need to distinguish between a static member lookup *with*
and *without* instance variables. The lookup without instance variables
corresponds to `find_name_in_mro`
[here](https://docs.python.org/3/howto/descriptor.html#invocation-from-an-instance).
We currently approximate both using `member_static`, which leads to two
open TODOs. Changing this would be a larger refactoring of
`Type::own_instance_member`, so I chose to leave it out of this PR.

## Test Plan

* New `call/methods.md` test suite for method calls
* New tests in `descriptor_protocol.md`
* New `call/getattr_static.md` test suite for `inspect.getattr_static`
* Various updated tests
2025-02-20 23:22:26 +01:00
David Peter
f62e5406f2 [red-knot] Short-circuit bool calls on bool (#16292)
## Summary

This avoids looking up `__bool__` on class `bool` for every
`Type::Instance(bool).bool()` call. 1% performance win on cold cache, 4%
win on incremental performance.
2025-02-20 23:06:11 +01:00
Douglas Creager
1be4394155 [red-knot] Consolidate SymbolBindings/SymbolDeclarations state (#16286)
This updates the `SymbolBindings` and `SymbolDeclarations` types to use
a single smallvec of live bindings/declarations, instead of splitting
that out into separate containers for each field.

I'm seeing an 11-13% `cargo bench` performance improvement with this
locally (for both cold and incremental). I'm interested to see if
Codspeed agrees!

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2025-02-20 16:20:23 -05:00
71 changed files with 4006 additions and 1309 deletions

146
Cargo.lock generated
View File

@@ -8,18 +8,6 @@ version = "2.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "512761e0bb2578dd7380c6baaa0f4ce03e84f95e960231d1dec8bf4d7d6e2627"
[[package]]
name = "ahash"
version = "0.8.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e89da841a80418a9b391ebaea17f5c112ffaaa96f621d2c285b5174da76b9011"
dependencies = [
"cfg-if",
"once_cell",
"version_check",
"zerocopy 0.7.35",
]
[[package]]
name = "aho-corasick"
version = "1.1.3"
@@ -140,12 +128,6 @@ version = "1.0.95"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "34ac096ce696dc2fcabef30516bb13c0a68a11d30131d3df6f04711467681b04"
[[package]]
name = "append-only-vec"
version = "0.1.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7992085ec035cfe96992dd31bfd495a2ebd31969bb95f624471cb6c0b349e571"
[[package]]
name = "arc-swap"
version = "1.7.1"
@@ -227,9 +209,12 @@ dependencies = [
[[package]]
name = "boxcar"
version = "0.2.8"
version = "0.2.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2721c3c5a6f0e7f7e607125d963fedeb765f545f67adc9d71ed934693881eb42"
checksum = "225450ee9328e1e828319b48a89726cffc1b0ad26fd9211ad435de9fa376acae"
dependencies = [
"loom",
]
[[package]]
name = "bstr"
@@ -1013,6 +998,19 @@ dependencies = [
"libc",
]
[[package]]
name = "generator"
version = "0.8.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cc6bd114ceda131d3b1d665eba35788690ad37f5916457286b32ab6fd3c438dd"
dependencies = [
"cfg-if",
"libc",
"log",
"rustversion",
"windows",
]
[[package]]
name = "generic-array"
version = "0.14.7"
@@ -1102,10 +1100,6 @@ name = "hashbrown"
version = "0.14.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e5274423e17b7c9fc20b6e7e208532f9b19825d82dfd615708b70edd83df41f1"
dependencies = [
"ahash",
"allocator-api2",
]
[[package]]
name = "hashbrown"
@@ -1113,17 +1107,18 @@ version = "0.15.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bf151400ff0baff5465007dd2f3e717f3fe502074ca563069ce3a6629d07b289"
dependencies = [
"allocator-api2",
"equivalent",
"foldhash",
]
[[package]]
name = "hashlink"
version = "0.9.1"
version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6ba4ff7128dee98c7dc9794b6a411377e1404dba1c97deb8d1a55297bd25d8af"
checksum = "7382cf6263419f2d8df38c55d7da83da5c18aef87fc7a7fc1fb1e344edfe14c1"
dependencies = [
"hashbrown 0.14.5",
"hashbrown 0.15.2",
]
[[package]]
@@ -1179,7 +1174,7 @@ dependencies = [
"iana-time-zone-haiku",
"js-sys",
"wasm-bindgen",
"windows-core",
"windows-core 0.52.0",
]
[[package]]
@@ -1683,6 +1678,19 @@ version = "0.4.25"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "04cbf5b083de1c7e0222a7a51dbfdba1cbe1c6ab0b15e29fff3f6c077fd9cd9f"
[[package]]
name = "loom"
version = "0.7.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "419e0dc8046cb947daa77eb95ae174acfbddb7673b4151f56d1eed8e93fbfaca"
dependencies = [
"cfg-if",
"generator",
"scoped-tls",
"tracing",
"tracing-subscriber",
]
[[package]]
name = "lsp-server"
version = "0.7.8"
@@ -3315,14 +3323,14 @@ checksum = "6ea1a2d0a644769cc99faa24c3ad26b379b786fe7c36fd3c546254801650e6dd"
[[package]]
name = "salsa"
version = "0.18.0"
source = "git+https://github.com/salsa-rs/salsa.git?rev=351d9cf0037be949d17800d0c7b4838e533c2ed6#351d9cf0037be949d17800d0c7b4838e533c2ed6"
source = "git+https://github.com/salsa-rs/salsa.git?rev=c8826fa4d1d9e3cba4c6e578763878b71fa9a10d#c8826fa4d1d9e3cba4c6e578763878b71fa9a10d"
dependencies = [
"append-only-vec",
"arc-swap",
"boxcar",
"compact_str",
"crossbeam",
"crossbeam-queue",
"dashmap 6.1.0",
"hashbrown 0.14.5",
"hashbrown 0.15.2",
"hashlink",
"indexmap",
"parking_lot",
@@ -3337,12 +3345,12 @@ dependencies = [
[[package]]
name = "salsa-macro-rules"
version = "0.1.0"
source = "git+https://github.com/salsa-rs/salsa.git?rev=351d9cf0037be949d17800d0c7b4838e533c2ed6#351d9cf0037be949d17800d0c7b4838e533c2ed6"
source = "git+https://github.com/salsa-rs/salsa.git?rev=c8826fa4d1d9e3cba4c6e578763878b71fa9a10d#c8826fa4d1d9e3cba4c6e578763878b71fa9a10d"
[[package]]
name = "salsa-macros"
version = "0.18.0"
source = "git+https://github.com/salsa-rs/salsa.git?rev=351d9cf0037be949d17800d0c7b4838e533c2ed6#351d9cf0037be949d17800d0c7b4838e533c2ed6"
source = "git+https://github.com/salsa-rs/salsa.git?rev=c8826fa4d1d9e3cba4c6e578763878b71fa9a10d#c8826fa4d1d9e3cba4c6e578763878b71fa9a10d"
dependencies = [
"heck",
"proc-macro2",
@@ -3384,6 +3392,12 @@ dependencies = [
"syn 2.0.98",
]
[[package]]
name = "scoped-tls"
version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e1cf6437eb19a8f4a6cc0f7dca544973b0b78843adbfeb3683d1a94a0024a294"
[[package]]
name = "scopeguard"
version = "1.2.0"
@@ -4451,6 +4465,16 @@ version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f"
[[package]]
name = "windows"
version = "0.58.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dd04d41d93c4992d421894c18c8b43496aa748dd4c081bac0dc93eb0489272b6"
dependencies = [
"windows-core 0.58.0",
"windows-targets 0.52.6",
]
[[package]]
name = "windows-core"
version = "0.52.0"
@@ -4460,6 +4484,60 @@ dependencies = [
"windows-targets 0.52.6",
]
[[package]]
name = "windows-core"
version = "0.58.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6ba6d44ec8c2591c134257ce647b7ea6b20335bf6379a27dac5f1641fcf59f99"
dependencies = [
"windows-implement",
"windows-interface",
"windows-result",
"windows-strings",
"windows-targets 0.52.6",
]
[[package]]
name = "windows-implement"
version = "0.58.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2bbd5b46c938e506ecbce286b6628a02171d56153ba733b6c741fc627ec9579b"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.98",
]
[[package]]
name = "windows-interface"
version = "0.58.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "053c4c462dc91d3b1504c6fe5a726dd15e216ba718e84a0e46a88fbe5ded3515"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.98",
]
[[package]]
name = "windows-result"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1d1043d8214f791817bab27572aaa8af63732e11bf84aa21a45a78d6c317ae0e"
dependencies = [
"windows-targets 0.52.6",
]
[[package]]
name = "windows-strings"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4cd9b125c486025df0eabcb585e62173c6c9eddcec5d117d3b6e8c30e2ee4d10"
dependencies = [
"windows-result",
"windows-targets 0.52.6",
]
[[package]]
name = "windows-sys"
version = "0.48.0"

View File

@@ -123,7 +123,7 @@ rayon = { version = "1.10.0" }
regex = { version = "1.10.2" }
rustc-hash = { version = "2.0.0" }
# When updating salsa, make sure to also update the revision in `fuzz/Cargo.toml`
salsa = { git = "https://github.com/salsa-rs/salsa.git", rev = "351d9cf0037be949d17800d0c7b4838e533c2ed6" }
salsa = { git = "https://github.com/salsa-rs/salsa.git", rev = "c8826fa4d1d9e3cba4c6e578763878b71fa9a10d" }
schemars = { version = "0.8.16" }
seahash = { version = "4.1.0" }
serde = { version = "1.0.197", features = ["derive"] }

View File

@@ -73,12 +73,12 @@ qux = (foo, bar)
reveal_type(qux) # revealed: tuple[Literal["foo"], Literal["bar"]]
# TODO: Infer "LiteralString"
reveal_type(foo.join(qux)) # revealed: @Todo(Attribute access on `StringLiteral` types)
reveal_type(foo.join(qux)) # revealed: @Todo(decorated method)
template: LiteralString = "{}, {}"
reveal_type(template) # revealed: Literal["{}, {}"]
# TODO: Infer `LiteralString`
reveal_type(template.format(foo, bar)) # revealed: @Todo(Attribute access on `StringLiteral` types)
reveal_type(template.format(foo, bar)) # revealed: @Todo(decorated method)
```
### Assignability

View File

@@ -884,13 +884,18 @@ def _(flag: bool):
## Objects of all types have a `__class__` method
The type of `x.__class__` is the same as `x`'s meta-type. `x.__class__` is always the same value as
`type(x)`.
```py
import typing_extensions
reveal_type(typing_extensions.__class__) # revealed: Literal[ModuleType]
reveal_type(type(typing_extensions)) # revealed: Literal[ModuleType]
a = 42
reveal_type(a.__class__) # revealed: Literal[int]
reveal_type(type(a)) # revealed: Literal[int]
b = "42"
reveal_type(b.__class__) # revealed: Literal[str]
@@ -906,8 +911,13 @@ reveal_type(e.__class__) # revealed: Literal[tuple]
def f(a: int, b: typing_extensions.LiteralString, c: int | str, d: type[str]):
reveal_type(a.__class__) # revealed: type[int]
reveal_type(type(a)) # revealed: type[int]
reveal_type(b.__class__) # revealed: Literal[str]
reveal_type(type(b)) # revealed: Literal[str]
reveal_type(c.__class__) # revealed: type[int] | type[str]
reveal_type(type(c)) # revealed: type[int] | type[str]
# `type[type]`, a.k.a., either the class `type` or some subclass of `type`.
# It would be incorrect to infer `Literal[type]` here,
@@ -1005,8 +1015,8 @@ reveal_type(f.__kwdefaults__) # revealed: @Todo(generics) | None
Some attributes are special-cased, however:
```py
reveal_type(f.__get__) # revealed: @Todo(`__get__` method on functions)
reveal_type(f.__call__) # revealed: @Todo(`__call__` method on functions)
reveal_type(f.__get__) # revealed: <method-wrapper `__get__` of `f`>
reveal_type(f.__call__) # revealed: <bound method `__call__` of `Literal[f]`>
```
### Int-literal attributes
@@ -1015,7 +1025,7 @@ Most attribute accesses on int-literal types are delegated to `builtins.int`, si
integers are instances of that class:
```py
reveal_type((2).bit_length) # revealed: @Todo(bound method)
reveal_type((2).bit_length) # revealed: <bound method `bit_length` of `Literal[2]`>
reveal_type((2).denominator) # revealed: @Todo(@property)
```
@@ -1029,11 +1039,11 @@ reveal_type((2).real) # revealed: Literal[2]
### Bool-literal attributes
Most attribute accesses on bool-literal types are delegated to `builtins.bool`, since all literal
bols are instances of that class:
bools are instances of that class:
```py
reveal_type(True.__and__) # revealed: @Todo(bound method)
reveal_type(False.__or__) # revealed: @Todo(bound method)
reveal_type(True.__and__) # revealed: @Todo(decorated method)
reveal_type(False.__or__) # revealed: @Todo(decorated method)
```
Some attributes are special-cased, however:
@@ -1045,11 +1055,11 @@ reveal_type(False.real) # revealed: Literal[0]
### Bytes-literal attributes
All attribute access on literal `bytes` types is currently delegated to `buitins.bytes`:
All attribute access on literal `bytes` types is currently delegated to `builtins.bytes`:
```py
reveal_type(b"foo".join) # revealed: @Todo(bound method)
reveal_type(b"foo".endswith) # revealed: @Todo(bound method)
reveal_type(b"foo".join) # revealed: <bound method `join` of `Literal[b"foo"]`>
reveal_type(b"foo".endswith) # revealed: <bound method `endswith` of `Literal[b"foo"]`>
```
## Instance attribute edge cases
@@ -1136,6 +1146,40 @@ class C:
reveal_type(C().x) # revealed: Unknown
```
### Builtin types attributes
This test can probably be removed eventually, but we currently include it because we do not yet
understand generic bases and protocols, and we want to make sure that we can still use builtin types
in our tests in the meantime. See the corresponding TODO in `Type::static_member` for more
information.
```py
class C:
a_int: int = 1
a_str: str = "a"
a_bytes: bytes = b"a"
a_bool: bool = True
a_float: float = 1.0
a_complex: complex = 1 + 1j
a_tuple: tuple[int] = (1,)
a_range: range = range(1)
a_slice: slice = slice(1)
a_type: type = int
a_none: None = None
reveal_type(C.a_int) # revealed: int
reveal_type(C.a_str) # revealed: str
reveal_type(C.a_bytes) # revealed: bytes
reveal_type(C.a_bool) # revealed: bool
reveal_type(C.a_float) # revealed: int | float
reveal_type(C.a_complex) # revealed: int | float | complex
reveal_type(C.a_tuple) # revealed: tuple[int]
reveal_type(C.a_range) # revealed: range
reveal_type(C.a_slice) # revealed: slice
reveal_type(C.a_type) # revealed: type
reveal_type(C.a_none) # revealed: None
```
## References
Some of the tests in the *Class and instance variables* section draw inspiration from

View File

@@ -351,6 +351,20 @@ class Y(Foo): ...
reveal_type(X() + Y()) # revealed: int
```
## Operations involving types with invalid `__bool__` methods
<!-- snapshot-diagnostics -->
```py
class NotBoolable:
__bool__ = 3
a = NotBoolable()
# error: [unsupported-bool-conversion]
10 and a and True
```
## Unsupported
### Dunder as instance attribute

View File

@@ -0,0 +1,37 @@
# Calling builtins
## `bool` with incorrect arguments
```py
class NotBool:
__bool__ = None
# TODO: We should emit an `invalid-argument` error here for `2` because `bool` only takes one argument.
bool(1, 2)
# TODO: We should emit an `unsupported-bool-conversion` error here because the argument doesn't implement `__bool__` correctly.
bool(NotBool())
```
## Calls to `type()`
A single-argument call to `type()` returns an object that has the argument's meta-type. (This is
tested more extensively in `crates/red_knot_python_semantic/resources/mdtest/attributes.md`,
alongside the tests for the `__class__` attribute.)
```py
reveal_type(type(1)) # revealed: Literal[int]
```
But a three-argument call to type creates a dynamic instance of the `type` class:
```py
reveal_type(type("Foo", (), {})) # revealed: type
```
Other numbers of arguments are invalid (TODO -- these should emit a diagnostic)
```py
type("Foo", ())
type("Foo", (), {}, weird_other_arg=42)
```

View File

@@ -0,0 +1,133 @@
# `inspect.getattr_static`
## Basic usage
`inspect.getattr_static` is a function that returns attributes of an object without invoking the
descriptor protocol (for caveats, see the [official documentation]).
Consider the following example:
```py
import inspect
class Descriptor:
def __get__(self, instance, owner) -> str:
return 1
class C:
normal: int = 1
descriptor: Descriptor = Descriptor()
```
If we access attributes on an instance of `C` as usual, the descriptor protocol is invoked, and we
get a type of `str` for the `descriptor` attribute:
```py
c = C()
reveal_type(c.normal) # revealed: int
reveal_type(c.descriptor) # revealed: str
```
However, if we use `inspect.getattr_static`, we can see the underlying `Descriptor` type:
```py
reveal_type(inspect.getattr_static(c, "normal")) # revealed: int
reveal_type(inspect.getattr_static(c, "descriptor")) # revealed: Descriptor
```
For non-existent attributes, a default value can be provided:
```py
reveal_type(inspect.getattr_static(C, "normal", "default-arg")) # revealed: int
reveal_type(inspect.getattr_static(C, "non_existent", "default-arg")) # revealed: Literal["default-arg"]
```
When a non-existent attribute is accessed without a default value, the runtime raises an
`AttributeError`. We could emit a diagnostic for this case, but that is currently not supported:
```py
# TODO: we could emit a diagnostic here
reveal_type(inspect.getattr_static(C, "non_existent")) # revealed: Never
```
We can access attributes on objects of all kinds:
```py
import sys
reveal_type(inspect.getattr_static(sys, "platform")) # revealed: LiteralString
reveal_type(inspect.getattr_static(inspect, "getattr_static")) # revealed: Literal[getattr_static]
reveal_type(inspect.getattr_static(1, "real")) # revealed: Literal[1]
```
(Implicit) instance attributes can also be accessed through `inspect.getattr_static`:
```py
class D:
def __init__(self) -> None:
self.instance_attr: int = 1
reveal_type(inspect.getattr_static(D(), "instance_attr")) # revealed: int
```
## Error cases
We can only infer precise types if the attribute is a literal string. In all other cases, we fall
back to `Any`:
```py
import inspect
class C:
x: int = 1
def _(attr_name: str):
reveal_type(inspect.getattr_static(C(), attr_name)) # revealed: Any
reveal_type(inspect.getattr_static(C(), attr_name, 1)) # revealed: Any
```
But we still detect errors in the number or type of arguments:
```py
# error: [missing-argument] "No arguments provided for required parameters `obj`, `attr` of function `getattr_static`"
inspect.getattr_static()
# error: [missing-argument] "No argument provided for required parameter `attr`"
inspect.getattr_static(C())
# error: [invalid-argument-type] "Object of type `Literal[1]` cannot be assigned to parameter 2 (`attr`) of function `getattr_static`; expected type `str`"
inspect.getattr_static(C(), 1)
# error: [too-many-positional-arguments] "Too many positional arguments to function `getattr_static`: expected 3, got 4"
inspect.getattr_static(C(), "x", "default-arg", "one too many")
```
## Possibly unbound attributes
```py
import inspect
def _(flag: bool):
class C:
if flag:
x: int = 1
reveal_type(inspect.getattr_static(C, "x", "default")) # revealed: int | Literal["default"]
```
## Gradual types
```py
import inspect
from typing import Any
def _(a: Any, tuple_of_any: tuple[Any]):
reveal_type(inspect.getattr_static(a, "x", "default")) # revealed: Any | Literal["default"]
# TODO: Ideally, this would just be `Literal[index]`
reveal_type(inspect.getattr_static(tuple_of_any, "index", "default")) # revealed: Literal[index] | Literal["default"]
```
[official documentation]: https://docs.python.org/3/library/inspect.html#inspect.getattr_static

View File

@@ -0,0 +1,258 @@
# Methods
## Background: Functions as descriptors
> Note: See also this related section in the descriptor guide: [Functions and methods].
Say we have a simple class `C` with a function definition `f` inside its body:
```py
class C:
def f(self, x: int) -> str:
return "a"
```
Whenever we access the `f` attribute through the class object itself (`C.f`) or through an instance
(`C().f`), this access happens via the descriptor protocol. Functions are (non-data) descriptors
because they implement a `__get__` method. This is crucial in making sure that method calls work as
expected. In general, the signature of the `__get__` method in the descriptor protocol is
`__get__(self, instance, owner)`. The `self` argument is the descriptor object itself (`f`). The
passed value for the `instance` argument depends on whether the attribute is accessed from the class
object (in which case it is `None`), or from an instance (in which case it is the instance of type
`C`). The `owner` argument is the class itself (`C` of type `Literal[C]`). To summarize:
- `C.f` is equivalent to `getattr_static(C, "f").__get__(None, C)`
- `C().f` is equivalent to `getattr_static(C, "f").__get__(C(), C)`
Here, `inspect.getattr_static` is used to bypass the descriptor protocol and directly access the
function attribute. The way the special `__get__` method *on functions* works is as follows. In the
former case, if the `instance` argument is `None`, `__get__` simply returns the function itself. In
the latter case, it returns a *bound method* object:
```py
from inspect import getattr_static
reveal_type(getattr_static(C, "f")) # revealed: Literal[f]
reveal_type(getattr_static(C, "f").__get__) # revealed: <method-wrapper `__get__` of `f`>
reveal_type(getattr_static(C, "f").__get__(None, C)) # revealed: Literal[f]
reveal_type(getattr_static(C, "f").__get__(C(), C)) # revealed: <bound method `f` of `C`>
```
In conclusion, this is why we see the following two types when accessing the `f` attribute on the
class object `C` and on an instance `C()`:
```py
reveal_type(C.f) # revealed: Literal[f]
reveal_type(C().f) # revealed: <bound method `f` of `C`>
```
A bound method is a callable object that contains a reference to the `instance` that it was called
on (can be inspected via `__self__`), and the function object that it refers to (can be inspected
via `__func__`):
```py
bound_method = C().f
reveal_type(bound_method.__self__) # revealed: C
reveal_type(bound_method.__func__) # revealed: Literal[f]
```
When we call the bound method, the `instance` is implicitly passed as the first argument (`self`):
```py
reveal_type(C().f(1)) # revealed: str
reveal_type(bound_method(1)) # revealed: str
```
When we call the function object itself, we need to pass the `instance` explicitly:
```py
C.f(1) # error: [missing-argument]
reveal_type(C.f(C(), 1)) # revealed: str
```
When we access methods from derived classes, they will be bound to instances of the derived class:
```py
class D(C):
pass
reveal_type(D().f) # revealed: <bound method `f` of `D`>
```
If we access an attribute on a bound method object itself, it will defer to `types.MethodType`:
```py
reveal_type(bound_method.__hash__) # revealed: <bound method `__hash__` of `MethodType`>
```
If an attribute is not available on the bound method object, it will be looked up on the underlying
function object. We model this explicitly, which means that we can access `__kwdefaults__` on bound
methods, even though it is not available on `types.MethodType`:
```py
reveal_type(bound_method.__kwdefaults__) # revealed: @Todo(generics) | None
```
## Basic method calls on class objects and instances
```py
class Base:
def method_on_base(self, x: int | None) -> str:
return "a"
class Derived(Base):
def method_on_derived(self, x: bytes) -> tuple[int, str]:
return (1, "a")
reveal_type(Base().method_on_base(1)) # revealed: str
reveal_type(Base.method_on_base(Base(), 1)) # revealed: str
Base().method_on_base("incorrect") # error: [invalid-argument-type]
Base().method_on_base() # error: [missing-argument]
Base().method_on_base(1, 2) # error: [too-many-positional-arguments]
reveal_type(Derived().method_on_base(1)) # revealed: str
reveal_type(Derived().method_on_derived(b"abc")) # revealed: tuple[int, str]
reveal_type(Derived.method_on_base(Derived(), 1)) # revealed: str
reveal_type(Derived.method_on_derived(Derived(), b"abc")) # revealed: tuple[int, str]
```
## Method calls on literals
### Boolean literals
```py
reveal_type(True.bit_length()) # revealed: int
reveal_type(True.as_integer_ratio()) # revealed: tuple[int, Literal[1]]
```
### Integer literals
```py
reveal_type((42).bit_length()) # revealed: int
```
### String literals
```py
reveal_type("abcde".find("abc")) # revealed: int
reveal_type("foo".encode(encoding="utf-8")) # revealed: bytes
"abcde".find(123) # error: [invalid-argument-type]
```
### Bytes literals
```py
reveal_type(b"abcde".startswith(b"abc")) # revealed: bool
```
## Method calls on `LiteralString`
```py
from typing_extensions import LiteralString
def f(s: LiteralString) -> None:
reveal_type(s.find("a")) # revealed: int
```
## Method calls on `tuple`
```py
def f(t: tuple[int, str]) -> None:
reveal_type(t.index("a")) # revealed: int
```
## Method calls on unions
```py
from typing import Any
class A:
def f(self) -> int:
return 1
class B:
def f(self) -> str:
return "a"
def f(a_or_b: A | B, any_or_a: Any | A):
reveal_type(a_or_b.f) # revealed: <bound method `f` of `A`> | <bound method `f` of `B`>
reveal_type(a_or_b.f()) # revealed: int | str
reveal_type(any_or_a.f) # revealed: Any | <bound method `f` of `A`>
reveal_type(any_or_a.f()) # revealed: Any | int
```
## Method calls on `KnownInstance` types
```toml
[environment]
python-version = "3.12"
```
```py
type IntOrStr = int | str
reveal_type(IntOrStr.__or__) # revealed: <bound method `__or__` of `typing.TypeAliasType`>
```
## Error cases: Calling `__get__` for methods
The `__get__` method on `types.FunctionType` has the following overloaded signature in typeshed:
```py
from types import FunctionType, MethodType
from typing import overload
@overload
def __get__(self, instance: None, owner: type, /) -> FunctionType: ...
@overload
def __get__(self, instance: object, owner: type | None = None, /) -> MethodType: ...
```
Here, we test that this signature is enforced correctly:
```py
from inspect import getattr_static
class C:
def f(self, x: int) -> str:
return "a"
method_wrapper = getattr_static(C, "f").__get__
reveal_type(method_wrapper) # revealed: <method-wrapper `__get__` of `f`>
# All of these are fine:
method_wrapper(C(), C)
method_wrapper(C())
method_wrapper(C(), None)
method_wrapper(None, C)
# Passing `None` without an `owner` argument is an
# error: [missing-argument] "No argument provided for required parameter `owner`"
method_wrapper(None)
# Passing something that is not assignable to `type` as the `owner` argument is an
# error: [invalid-argument-type] "Object of type `Literal[1]` cannot be assigned to parameter 2 (`owner`); expected type `type`"
method_wrapper(None, 1)
# Passing `None` as the `owner` argument when `instance` is `None` is an
# error: [invalid-argument-type] "Object of type `None` cannot be assigned to parameter 2 (`owner`); expected type `type`"
method_wrapper(None, None)
# Calling `__get__` without any arguments is an
# error: [missing-argument] "No argument provided for required parameter `instance`"
method_wrapper()
# Calling `__get__` with too many positional arguments is an
# error: [too-many-positional-arguments] "Too many positional arguments: expected 2, got 3"
method_wrapper(C(), C, "one too many")
```
[functions and methods]: https://docs.python.org/3/howto/descriptor.html#functions-and-methods

View File

@@ -160,3 +160,45 @@ reveal_type(42 in A()) # revealed: bool
# error: [unsupported-operator] "Operator `in` is not supported for types `str` and `A`, in comparing `Literal["hello"]` with `A`"
reveal_type("hello" in A()) # revealed: bool
```
## Return type that doesn't implement `__bool__` correctly
`in` and `not in` operations will fail at runtime if the object on the right-hand side of the
operation has a `__contains__` method that returns a type which is not convertible to `bool`. This
is because of the way these operations are handled by the Python interpreter at runtime. If we
assume that `y` is an object that has a `__contains__` method, the Python expression `x in y`
desugars to a `contains(y, x)` call, where `contains` looks something like this:
```ignore
def contains(y, x):
return bool(type(y).__contains__(y, x))
```
where the `bool()` conversion itself implicitly calls `__bool__` under the hood.
TODO: Ideally the message would explain to the user what's wrong. E.g,
```ignore
error: [operator] cannot use `in` operator on object of type `WithContains`
note: This is because the `in` operator implicitly calls `WithContains.__contains__`, but `WithContains.__contains__` is invalidly defined
note: `WithContains.__contains__` is invalidly defined because it returns an instance of `NotBoolable`, which cannot be evaluated in a boolean context
note: `NotBoolable` cannot be evaluated in a boolean context because its `__bool__` attribute is not callable
```
It may also be more appropriate to use `unsupported-operator` as the error code.
<!-- snapshot-diagnostics -->
```py
class NotBoolable:
__bool__ = 3
class WithContains:
def __contains__(self, item) -> NotBoolable:
return NotBoolable()
# error: [unsupported-bool-conversion]
10 in WithContains()
# error: [unsupported-bool-conversion]
10 not in WithContains()
```

View File

@@ -345,3 +345,29 @@ def f(x: bool, y: int):
reveal_type(4.2 < x) # revealed: bool
reveal_type(x < 4.2) # revealed: bool
```
## Chained comparisons with objects that don't implement `__bool__` correctly
<!-- snapshot-diagnostics -->
Python implicitly calls `bool` on the comparison result of preceding elements (but not for the last
element) of a chained comparison.
```py
class NotBoolable:
__bool__ = 3
class Comparable:
def __lt__(self, item) -> NotBoolable:
return NotBoolable()
def __gt__(self, item) -> NotBoolable:
return NotBoolable()
# error: [unsupported-bool-conversion]
10 < Comparable() < 20
# error: [unsupported-bool-conversion]
10 < Comparable() < Comparable()
Comparable() < Comparable() # fine
```

View File

@@ -334,3 +334,61 @@ reveal_type(a is not c) # revealed: Literal[True]
For tuples like `tuple[int, ...]`, `tuple[Any, ...]`
// TODO
## Chained comparisons with elements that incorrectly implement `__bool__`
<!-- snapshot-diagnostics -->
For an operation `A() < A()` to succeed at runtime, the `A.__lt__` method does not necessarily need
to return an object that is convertible to a `bool`. However, the return type _does_ need to be
convertible to a `bool` for the operation `A() < A() < A()` (a _chained_ comparison) to succeed.
This is because `A() < A() < A()` desugars to something like this, which involves several implicit
conversions to `bool`:
```ignore
def compute_chained_comparison():
a1 = A()
a2 = A()
first_comparison = a1 < a2
return first_comparison and (a2 < A())
```
```py
class NotBoolable:
__bool__ = 5
class Comparable:
def __lt__(self, other) -> NotBoolable:
return NotBoolable()
def __gt__(self, other) -> NotBoolable:
return NotBoolable()
a = (1, Comparable())
b = (1, Comparable())
# error: [unsupported-bool-conversion]
a < b < b
a < b # fine
```
## Equality with elements that incorrectly implement `__bool__`
<!-- snapshot-diagnostics -->
Python does not generally attempt to coerce the result of `==` and `!=` operations between two
arbitrary objects to a `bool`, but a comparison of tuples will fail if the result of comparing any
pair of elements at equivalent positions cannot be converted to a `bool`:
```py
class A:
def __eq__(self, other) -> NotBoolable:
return NotBoolable()
class NotBoolable:
__bool__ = None
# error: [unsupported-bool-conversion]
(A(),) == (A(),)
```

View File

@@ -35,3 +35,13 @@ def _(flag: bool):
x = 1 if flag else None
reveal_type(x) # revealed: Literal[1] | None
```
## Condition with object that implements `__bool__` incorrectly
```py
class NotBoolable:
__bool__ = 3
# error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable"
3 if NotBoolable() else 4
```

View File

@@ -147,3 +147,17 @@ def _(flag: bool):
reveal_type(y) # revealed: Literal[0, 1]
```
## Condition with object that implements `__bool__` incorrectly
```py
class NotBoolable:
__bool__ = 3
# error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable"
if NotBoolable():
...
# error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable"
elif NotBoolable():
...
```

View File

@@ -43,3 +43,21 @@ def _(target: int):
reveal_type(y) # revealed: Literal[2, 3, 4]
```
## Guard with object that implements `__bool__` incorrectly
```py
class NotBoolable:
__bool__ = 3
def _(target: int, flag: NotBoolable):
y = 1
match target:
# error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable"
case 1 if flag:
y = 2
case 2:
y = 3
reveal_type(y) # revealed: Literal[1, 2, 3]
```

View File

@@ -22,22 +22,26 @@ class Ten:
pass
class C:
ten = Ten()
ten: Ten = Ten()
c = C()
# TODO: this should be `Literal[10]`
reveal_type(c.ten) # revealed: Unknown | Ten
reveal_type(c.ten) # revealed: Literal[10]
# TODO: This should `Literal[10]`
reveal_type(C.ten) # revealed: Unknown | Ten
reveal_type(C.ten) # revealed: Literal[10]
# These are fine:
c.ten = 10
# TODO: This should not be an error
c.ten = 10 # error: [invalid-assignment]
C.ten = 10
# TODO: Both of these should be errors
# TODO: This should be an error (as the wrong type is being implicitly passed to `Ten.__set__`),
# but the error message is misleading.
# error: [invalid-assignment] "Object of type `Literal[11]` is not assignable to attribute `ten` of type `Ten`"
c.ten = 11
# TODO: same as above
# error: [invalid-assignment] "Object of type `Literal[11]` is not assignable to attribute `ten` of type `Literal[10]`"
C.ten = 11
```
@@ -57,24 +61,86 @@ class FlexibleInt:
self._value = int(value)
class C:
flexible_int = FlexibleInt()
flexible_int: FlexibleInt = FlexibleInt()
c = C()
# TODO: should be `int | None`
reveal_type(c.flexible_int) # revealed: Unknown | FlexibleInt
reveal_type(c.flexible_int) # revealed: int | None
# TODO: These should not be errors
# error: [invalid-assignment]
c.flexible_int = 42 # okay
# error: [invalid-assignment]
c.flexible_int = "42" # also okay!
# TODO: should be `int | None`
reveal_type(c.flexible_int) # revealed: Unknown | FlexibleInt
reveal_type(c.flexible_int) # revealed: int | None
# TODO: should be an error
# TODO: This should be an error, but the message needs to be improved.
# error: [invalid-assignment] "Object of type `None` is not assignable to attribute `flexible_int` of type `FlexibleInt`"
c.flexible_int = None # not okay
# TODO: should be `int | None`
reveal_type(c.flexible_int) # revealed: Unknown | FlexibleInt
reveal_type(c.flexible_int) # revealed: int | None
```
## Data and non-data descriptors
Descriptors that define `__set__` or `__delete__` are called *data descriptors*. An example\
of a data descriptor is a `property` with a setter and/or a deleter.\
Descriptors that only define `__get__`, meanwhile, are called *non-data descriptors*. Examples
include\
functions, `classmethod` or `staticmethod`).
The precedence chain for attribute access is (1) data descriptors, (2) instance attributes, and (3)
non-data descriptors.
```py
from typing import Literal
class DataDescriptor:
def __get__(self, instance: object, owner: type | None = None) -> Literal["data"]:
return "data"
def __set__(self, instance: int, value) -> None:
pass
class NonDataDescriptor:
def __get__(self, instance: object, owner: type | None = None) -> Literal["non-data"]:
return "non-data"
class C:
data_descriptor = DataDescriptor()
non_data_descriptor = NonDataDescriptor()
def f(self):
# This explains why data descriptors come first in the precedence chain. If
# instance attributes would take priority, we would override the descriptor
# here. Instead, this calls `DataDescriptor.__set__`, i.e. it does not affect
# the type of the `data_descriptor` attribute.
self.data_descriptor = 1
# However, for non-data descriptors, instance attributes do take precedence.
# So it is possible to override them.
self.non_data_descriptor = 1
c = C()
# TODO: This should ideally be `Unknown | Literal["data"]`.
#
# - Pyright also wrongly shows `int | Literal['data']` here
# - Mypy shows Literal["data"] here, but also shows Literal["non-data"] below.
#
reveal_type(c.data_descriptor) # revealed: Unknown | Literal["data", 1]
reveal_type(c.non_data_descriptor) # revealed: Unknown | Literal["non-data", 1]
reveal_type(C.data_descriptor) # revealed: Unknown | Literal["data"]
reveal_type(C.non_data_descriptor) # revealed: Unknown | Literal["non-data"]
# It is possible to override data descriptors via class objects. The following
# assignment does not call `DataDescriptor.__set__`. For this reason, we infer
# `Unknown | …` for all (descriptor) attributes.
C.data_descriptor = "something else" # This is okay
```
## Built-in `property` descriptor
@@ -101,7 +167,7 @@ c = C()
reveal_type(c._name) # revealed: str | None
# Should be `str`
reveal_type(c.name) # revealed: @Todo(bound method)
reveal_type(c.name) # revealed: @Todo(decorated method)
# Should be `builtins.property`
reveal_type(C.name) # revealed: Literal[name]
@@ -142,7 +208,7 @@ reveal_type(c1) # revealed: @Todo(return type)
reveal_type(C.get_name()) # revealed: @Todo(return type)
# TODO: should be `str`
reveal_type(C("42").get_name()) # revealed: @Todo(bound method)
reveal_type(C("42").get_name()) # revealed: @Todo(decorated method)
```
## Descriptors only work when used as class variables
@@ -160,9 +226,10 @@ class Ten:
class C:
def __init__(self):
self.ten = Ten()
self.ten: Ten = Ten()
reveal_type(C().ten) # revealed: Unknown | Ten
# TODO: Should be Ten
reveal_type(C().ten) # revealed: Literal[10]
```
## Descriptors distinguishing between class and instance access
@@ -186,13 +253,191 @@ class Descriptor:
return "called on class object"
class C:
d = Descriptor()
d: Descriptor = Descriptor()
# TODO: should be `Literal["called on class object"]
reveal_type(C.d) # revealed: Unknown | Descriptor
reveal_type(C.d) # revealed: LiteralString
# TODO: should be `Literal["called on instance"]
reveal_type(C().d) # revealed: Unknown | Descriptor
reveal_type(C().d) # revealed: LiteralString
```
## Undeclared descriptor arguments
If a descriptor attribute is not declared, we union with `Unknown`, just like for regular
attributes, since that attribute could be overwritten externally. Even a data descriptor with a
`__set__` method can be overwritten when accessed through a class object.
```py
class Descriptor:
def __get__(self, instance: object, owner: type | None = None) -> int:
return 1
def __set__(self, instance: object, value: int) -> None:
pass
class C:
descriptor = Descriptor()
C.descriptor = "something else"
# This could also be `Literal["something else"]` if we support narrowing of attribute types based on assignments
reveal_type(C.descriptor) # revealed: Unknown | int
```
## `__get__` is called with correct arguments
```py
from __future__ import annotations
class TailoredForClassObjectAccess:
def __get__(self, instance: None, owner: type[C]) -> int:
return 1
class TailoredForInstanceAccess:
def __get__(self, instance: C, owner: type[C] | None = None) -> str:
return "a"
class C:
class_object_access: TailoredForClassObjectAccess = TailoredForClassObjectAccess()
instance_access: TailoredForInstanceAccess = TailoredForInstanceAccess()
reveal_type(C.class_object_access) # revealed: int
reveal_type(C().instance_access) # revealed: str
# TODO: These should emit a diagnostic
reveal_type(C().class_object_access) # revealed: TailoredForClassObjectAccess
reveal_type(C.instance_access) # revealed: TailoredForInstanceAccess
```
## Descriptors with incorrect `__get__` signature
```py
class Descriptor:
# `__get__` method with missing parameters:
def __get__(self) -> int:
return 1
class C:
descriptor: Descriptor = Descriptor()
# TODO: This should be an error
reveal_type(C.descriptor) # revealed: Descriptor
```
## Possibly-unbound `__get__` method
```py
def _(flag: bool):
class MaybeDescriptor:
if flag:
def __get__(self, instance: object, owner: type | None = None) -> int:
return 1
class C:
descriptor: MaybeDescriptor = MaybeDescriptor()
# TODO: This should be `MaybeDescriptor | int`
reveal_type(C.descriptor) # revealed: int
```
## Dunder methods
Dunder methods are looked up on the meta type, but we still need to invoke the descriptor protocol:
```py
class SomeCallable:
def __call__(self, x: int) -> str:
return "a"
class Descriptor:
def __get__(self, instance: object, owner: type | None = None) -> SomeCallable:
return SomeCallable()
class B:
__call__: Descriptor = Descriptor()
b_instance = B()
reveal_type(b_instance(1)) # revealed: str
b_instance("bla") # error: [invalid-argument-type]
```
## Functions as descriptors
Functions are descriptors because they implement a `__get__` method. This is crucial in making sure
that method calls work as expected. See [this test suite](./call/methods.md) for more information.
Here, we only demonstrate how `__get__` works on functions:
```py
from inspect import getattr_static
def f(x: object) -> str:
return "a"
reveal_type(f) # revealed: Literal[f]
reveal_type(f.__get__) # revealed: <method-wrapper `__get__` of `f`>
reveal_type(f.__get__(None, type(f))) # revealed: Literal[f]
reveal_type(f.__get__(None, type(f))(1)) # revealed: str
wrapper_descriptor = getattr_static(f, "__get__")
reveal_type(wrapper_descriptor) # revealed: <wrapper-descriptor `__get__` of `function` objects>
reveal_type(wrapper_descriptor(f, None, type(f))) # revealed: Literal[f]
# Attribute access on the method-wrapper `f.__get__` falls back to `MethodWrapperType`:
reveal_type(f.__get__.__hash__) # revealed: <bound method `__hash__` of `MethodWrapperType`>
# Attribute access on the wrapper-descriptor falls back to `WrapperDescriptorType`:
reveal_type(wrapper_descriptor.__qualname__) # revealed: @Todo(@property)
```
We can also bind the free function `f` to an instance of a class `C`:
```py
class C: ...
bound_method = wrapper_descriptor(f, C(), C)
reveal_type(bound_method) # revealed: <bound method `f` of `C`>
```
We can then call it, and the instance of `C` is implicitly passed to the first parameter of `f`
(`x`):
```py
reveal_type(bound_method()) # revealed: str
```
Finally, we test some error cases for the call to the wrapper descriptor:
```py
# Calling the wrapper descriptor without any arguments is an
# error: [missing-argument] "No arguments provided for required parameters `self`, `instance`"
wrapper_descriptor()
# Calling it without the `instance` argument is an also an
# error: [missing-argument] "No argument provided for required parameter `instance`"
wrapper_descriptor(f)
# Calling it without the `owner` argument if `instance` is not `None` is an
# error: [missing-argument] "No argument provided for required parameter `owner`"
wrapper_descriptor(f, None)
# But calling it with an instance is fine (in this case, the `owner` argument is optional):
wrapper_descriptor(f, C())
# Calling it with something that is not a `FunctionType` as the first argument is an
# error: [invalid-argument-type] "Object of type `Literal[1]` cannot be assigned to parameter 1 (`self`); expected type `FunctionType`"
wrapper_descriptor(1, None, type(f))
# Calling it with something that is not a `type` as the `owner` argument is an
# error: [invalid-argument-type] "Object of type `Literal[f]` cannot be assigned to parameter 3 (`owner`); expected type `type`"
wrapper_descriptor(f, None, f)
# Calling it with too many positional arguments is an
# error: [too-many-positional-arguments] "Too many positional arguments: expected 3, got 4"
wrapper_descriptor(f, None, type(f), "one too many")
```
[descriptors]: https://docs.python.org/3/howto/descriptor.html

View File

@@ -0,0 +1,9 @@
## Condition with object that implements `__bool__` incorrectly
```py
class NotBoolable:
__bool__ = 3
# error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable"
assert NotBoolable()
```

View File

@@ -101,3 +101,55 @@ reveal_type(bool([])) # revealed: bool
reveal_type(bool({})) # revealed: bool
reveal_type(bool(set())) # revealed: bool
```
## `__bool__` returning `NoReturn`
```py
from typing import NoReturn
class NotBoolable:
def __bool__(self) -> NoReturn:
raise NotImplementedError("This object can't be converted to a boolean")
# TODO: This should emit an error that `NotBoolable` can't be converted to a bool but it currently doesn't
# because `Never` is assignable to `bool`. This probably requires dead code analysis to fix.
if NotBoolable():
...
```
## Not callable `__bool__`
```py
class NotBoolable:
__bool__ = None
# error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable"
if NotBoolable():
...
```
## Not-boolable union
```py
def test(cond: bool):
class NotBoolable:
__bool__ = None if cond else 3
# error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `NotBoolable`; it incorrectly implements `__bool__`"
if NotBoolable():
...
```
## Union with some variants implementing `__bool__` incorrectly
```py
def test(cond: bool):
class NotBoolable:
__bool__: int
a = 10 if cond else NotBoolable()
# error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `Literal[10] | NotBoolable`; its `__bool__` method isn't callable"
if a:
...
```

View File

@@ -116,3 +116,14 @@ def _(flag: bool, flag2: bool):
# error: [possibly-unresolved-reference]
y
```
## Condition with object that implements `__bool__` incorrectly
```py
class NotBoolable:
__bool__ = 3
# error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable"
while NotBoolable():
...
```

View File

@@ -97,12 +97,7 @@ else:
## No narrowing for instances of `builtins.type`
```py
def _(flag: bool):
t = type("t", (), {})
# This isn't testing what we want it to test if we infer anything more precise here:
reveal_type(t) # revealed: type
def _(flag: bool, t: type):
x = 1 if flag else "foo"
if isinstance(x, t):

View File

@@ -112,8 +112,7 @@ def _(flag: bool):
reveal_type(t) # revealed: Literal[NoneType]
if issubclass(t, type(None)):
# TODO: this should be just `Literal[NoneType]`
reveal_type(t) # revealed: Literal[int, NoneType]
reveal_type(t) # revealed: Literal[NoneType]
```
## `classinfo` contains multiple types

View File

@@ -266,7 +266,7 @@ def _(
if af:
reveal_type(af) # revealed: type[AmbiguousClass] & ~AlwaysFalsy
# TODO: Emit a diagnostic (`d` is not valid in boolean context)
# error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `MetaDeferred`; the return type of its bool method (`MetaAmbiguous`) isn't assignable to `bool"
if d:
# TODO: Should be `Unknown`
reveal_type(d) # revealed: type[DeferredClass] & ~AlwaysFalsy

View File

@@ -0,0 +1,15 @@
# Protocols
We do not support protocols yet, but to avoid false positives, we *partially* support some known
protocols.
## `typing.SupportsIndex`
```py
from typing import SupportsIndex, Literal
def _(some_int: int, some_literal_int: Literal[1], some_indexable: SupportsIndex):
a: SupportsIndex = some_int
b: SupportsIndex = some_literal_int
c: SupportsIndex = some_indexable
```

View File

@@ -9,7 +9,7 @@ is unbound.
```py
reveal_type(__name__) # revealed: str
reveal_type(__file__) # revealed: str | None
reveal_type(__loader__) # revealed: LoaderProtocol | None
reveal_type(__loader__) # revealed: @Todo(instance attribute on class with dynamic base) | None
reveal_type(__package__) # revealed: str | None
reveal_type(__doc__) # revealed: str | None
@@ -54,10 +54,10 @@ inside the module:
import typing
reveal_type(typing.__name__) # revealed: str
reveal_type(typing.__init__) # revealed: @Todo(bound method)
reveal_type(typing.__init__) # revealed: <bound method `__init__` of `ModuleType`>
# These come from `builtins.object`, not `types.ModuleType`:
reveal_type(typing.__eq__) # revealed: @Todo(bound method)
reveal_type(typing.__eq__) # revealed: <bound method `__eq__` of `ModuleType`>
reveal_type(typing.__class__) # revealed: Literal[ModuleType]
@@ -136,3 +136,42 @@ if returns_bool():
reveal_type(__file__) # revealed: Literal[42]
reveal_type(__name__) # revealed: Literal[1] | str
```
## Implicit global attributes in the current module override implicit globals from builtins
Here, we take the type of the implicit global symbol `__name__` from the `types.ModuleType` stub
(which in this custom typeshed specifies the type as `bytes`). This is because the `main` module has
an implicit `__name__` global that shadows the builtin `__name__` symbol.
```toml
[environment]
typeshed = "/typeshed"
```
`/typeshed/stdlib/builtins.pyi`:
```pyi
class int: ...
class bytes: ...
__name__: int = 42
```
`/typeshed/stdlib/types.pyi`:
```pyi
class ModuleType:
__name__: bytes
```
`/typeshed/stdlib/typing_extensions.pyi`:
```pyi
def reveal_type(obj, /): ...
```
`main.py`:
```py
reveal_type(__name__) # revealed: bytes
```

View File

@@ -0,0 +1,35 @@
---
source: crates/red_knot_test/src/lib.rs
expression: snapshot
---
---
mdtest name: instances.md - Binary operations on instances - Operations involving types with invalid `__bool__` methods
mdtest path: crates/red_knot_python_semantic/resources/mdtest/binary/instances.md
---
# Python source files
## mdtest_snippet.py
```
1 | class NotBoolable:
2 | __bool__ = 3
3 |
4 | a = NotBoolable()
5 |
6 | # error: [unsupported-bool-conversion]
7 | 10 and a and True
```
# Diagnostics
```
error: lint:unsupported-bool-conversion
--> /src/mdtest_snippet.py:7:8
|
6 | # error: [unsupported-bool-conversion]
7 | 10 and a and True
| ^ Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable
|
```

View File

@@ -0,0 +1,53 @@
---
source: crates/red_knot_test/src/lib.rs
expression: snapshot
---
---
mdtest name: membership_test.md - Comparison: Membership Test - Return type that doesn't implement `__bool__` correctly
mdtest path: crates/red_knot_python_semantic/resources/mdtest/comparison/instances/membership_test.md
---
# Python source files
## mdtest_snippet.py
```
1 | class NotBoolable:
2 | __bool__ = 3
3 |
4 | class WithContains:
5 | def __contains__(self, item) -> NotBoolable:
6 | return NotBoolable()
7 |
8 | # error: [unsupported-bool-conversion]
9 | 10 in WithContains()
10 | # error: [unsupported-bool-conversion]
11 | 10 not in WithContains()
```
# Diagnostics
```
error: lint:unsupported-bool-conversion
--> /src/mdtest_snippet.py:9:1
|
8 | # error: [unsupported-bool-conversion]
9 | 10 in WithContains()
| ^^^^^^^^^^^^^^^^^^^^ Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable
10 | # error: [unsupported-bool-conversion]
11 | 10 not in WithContains()
|
```
```
error: lint:unsupported-bool-conversion
--> /src/mdtest_snippet.py:11:1
|
9 | 10 in WithContains()
10 | # error: [unsupported-bool-conversion]
11 | 10 not in WithContains()
| ^^^^^^^^^^^^^^^^^^^^^^^^ Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable
|
```

View File

@@ -0,0 +1,33 @@
---
source: crates/red_knot_test/src/lib.rs
expression: snapshot
---
---
mdtest name: not.md - Unary not - Object that implements `__bool__` incorrectly
mdtest path: crates/red_knot_python_semantic/resources/mdtest/unary/not.md
---
# Python source files
## mdtest_snippet.py
```
1 | class NotBoolable:
2 | __bool__ = 3
3 |
4 | # error: [unsupported-bool-conversion]
5 | not NotBoolable()
```
# Diagnostics
```
error: lint:unsupported-bool-conversion
--> /src/mdtest_snippet.py:5:1
|
4 | # error: [unsupported-bool-conversion]
5 | not NotBoolable()
| ^^^^^^^^^^^^^^^^^ Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable
|
```

View File

@@ -0,0 +1,60 @@
---
source: crates/red_knot_test/src/lib.rs
expression: snapshot
---
---
mdtest name: rich_comparison.md - Comparison: Rich Comparison - Chained comparisons with objects that don't implement `__bool__` correctly
mdtest path: crates/red_knot_python_semantic/resources/mdtest/comparison/instances/rich_comparison.md
---
# Python source files
## mdtest_snippet.py
```
1 | class NotBoolable:
2 | __bool__ = 3
3 |
4 | class Comparable:
5 | def __lt__(self, item) -> NotBoolable:
6 | return NotBoolable()
7 |
8 | def __gt__(self, item) -> NotBoolable:
9 | return NotBoolable()
10 |
11 | # error: [unsupported-bool-conversion]
12 | 10 < Comparable() < 20
13 | # error: [unsupported-bool-conversion]
14 | 10 < Comparable() < Comparable()
15 |
16 | Comparable() < Comparable() # fine
```
# Diagnostics
```
error: lint:unsupported-bool-conversion
--> /src/mdtest_snippet.py:12:1
|
11 | # error: [unsupported-bool-conversion]
12 | 10 < Comparable() < 20
| ^^^^^^^^^^^^^^^^^ Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable
13 | # error: [unsupported-bool-conversion]
14 | 10 < Comparable() < Comparable()
|
```
```
error: lint:unsupported-bool-conversion
--> /src/mdtest_snippet.py:14:1
|
12 | 10 < Comparable() < 20
13 | # error: [unsupported-bool-conversion]
14 | 10 < Comparable() < Comparable()
| ^^^^^^^^^^^^^^^^^ Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable
15 |
16 | Comparable() < Comparable() # fine
|
```

View File

@@ -0,0 +1,47 @@
---
source: crates/red_knot_test/src/lib.rs
expression: snapshot
---
---
mdtest name: tuples.md - Comparison: Tuples - Chained comparisons with elements that incorrectly implement `__bool__`
mdtest path: crates/red_knot_python_semantic/resources/mdtest/comparison/tuples.md
---
# Python source files
## mdtest_snippet.py
```
1 | class NotBoolable:
2 | __bool__ = 5
3 |
4 | class Comparable:
5 | def __lt__(self, other) -> NotBoolable:
6 | return NotBoolable()
7 |
8 | def __gt__(self, other) -> NotBoolable:
9 | return NotBoolable()
10 |
11 | a = (1, Comparable())
12 | b = (1, Comparable())
13 |
14 | # error: [unsupported-bool-conversion]
15 | a < b < b
16 |
17 | a < b # fine
```
# Diagnostics
```
error: lint:unsupported-bool-conversion
--> /src/mdtest_snippet.py:15:1
|
14 | # error: [unsupported-bool-conversion]
15 | a < b < b
| ^^^^^ Boolean conversion is unsupported for type `NotBoolable | Literal[False]`; its `__bool__` method isn't callable
16 |
17 | a < b # fine
|
```

View File

@@ -0,0 +1,37 @@
---
source: crates/red_knot_test/src/lib.rs
expression: snapshot
---
---
mdtest name: tuples.md - Comparison: Tuples - Equality with elements that incorrectly implement `__bool__`
mdtest path: crates/red_knot_python_semantic/resources/mdtest/comparison/tuples.md
---
# Python source files
## mdtest_snippet.py
```
1 | class A:
2 | def __eq__(self, other) -> NotBoolable:
3 | return NotBoolable()
4 |
5 | class NotBoolable:
6 | __bool__ = None
7 |
8 | # error: [unsupported-bool-conversion]
9 | (A(),) == (A(),)
```
# Diagnostics
```
error: lint:unsupported-bool-conversion
--> /src/mdtest_snippet.py:9:1
|
8 | # error: [unsupported-bool-conversion]
9 | (A(),) == (A(),)
| ^^^^^^^^^^^^^^^^ Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable
|
```

View File

@@ -66,6 +66,6 @@ It is [recommended](https://docs.python.org/3/library/sys.html#sys.platform) to
```py
import sys
reveal_type(sys.platform.startswith("freebsd")) # revealed: @Todo(Attribute access on `LiteralString` types)
reveal_type(sys.platform.startswith("linux")) # revealed: @Todo(Attribute access on `LiteralString` types)
reveal_type(sys.platform.startswith("freebsd")) # revealed: bool
reveal_type(sys.platform.startswith("linux")) # revealed: bool
```

View File

@@ -223,7 +223,7 @@ class InvalidBoolDunder:
def __bool__(self) -> int:
return 1
# error: "Static assertion error: argument of type `InvalidBoolDunder` has an ambiguous static truthiness"
# error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `InvalidBoolDunder`; the return type of its bool method (`int`) isn't assignable to `bool"
static_assert(InvalidBoolDunder())
```

View File

@@ -35,7 +35,7 @@ in strict mode.
```py
def f(x: type):
reveal_type(x) # revealed: type
reveal_type(x.__repr__) # revealed: @Todo(bound method)
reveal_type(x.__repr__) # revealed: <bound method `__repr__` of `type`>
class A: ...
@@ -50,7 +50,7 @@ x: type = A() # error: [invalid-assignment]
```py
def f(x: type[object]):
reveal_type(x) # revealed: type
reveal_type(x.__repr__) # revealed: @Todo(bound method)
reveal_type(x.__repr__) # revealed: <bound method `__repr__` of `type`>
class A: ...

View File

@@ -75,3 +75,48 @@ class Boom:
reveal_type(bool(Boom())) # revealed: bool
```
### Possibly unbound __bool__ method
```py
from typing import Literal
def flag() -> bool:
return True
class PossiblyUnboundTrue:
if flag():
def __bool__(self) -> Literal[True]:
return True
reveal_type(bool(PossiblyUnboundTrue())) # revealed: bool
```
### Special-cased classes
Some special-cased `@final` classes are known by red-knot to have instances that are either always
truthy or always falsy.
```toml
[environment]
python-version = "3.12"
```
```py
import types
import typing
import sys
from knot_extensions import AlwaysTruthy, static_assert, is_subtype_of
from typing_extensions import _NoDefaultType
static_assert(is_subtype_of(sys.version_info.__class__, AlwaysTruthy))
static_assert(is_subtype_of(types.EllipsisType, AlwaysTruthy))
static_assert(is_subtype_of(_NoDefaultType, AlwaysTruthy))
static_assert(is_subtype_of(slice, AlwaysTruthy))
static_assert(is_subtype_of(types.FunctionType, AlwaysTruthy))
static_assert(is_subtype_of(types.MethodType, AlwaysTruthy))
static_assert(is_subtype_of(typing.TypeVar, AlwaysTruthy))
static_assert(is_subtype_of(typing.TypeAliasType, AlwaysTruthy))
static_assert(is_subtype_of(types.MethodWrapperType, AlwaysTruthy))
static_assert(is_subtype_of(types.WrapperDescriptorType, AlwaysTruthy))
```

View File

@@ -183,12 +183,11 @@ class WithBothLenAndBool2:
# revealed: Literal[False]
reveal_type(not WithBothLenAndBool2())
# TODO: raise diagnostic when __bool__ method is not valid: [unsupported-operator] "Method __bool__ for type `MethodBoolInvalid` should return `bool`, returned type `int`"
# https://docs.python.org/3/reference/datamodel.html#object.__bool__
class MethodBoolInvalid:
def __bool__(self) -> int:
return 0
# error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `MethodBoolInvalid`; the return type of its bool method (`int`) isn't assignable to `bool"
# revealed: bool
reveal_type(not MethodBoolInvalid())
@@ -204,3 +203,15 @@ class PossiblyUnboundBool:
# revealed: bool
reveal_type(not PossiblyUnboundBool())
```
## Object that implements `__bool__` incorrectly
<!-- snapshot-diagnostics -->
```py
class NotBoolable:
__bool__ = 3
# error: [unsupported-bool-conversion]
not NotBoolable()
```

View File

@@ -109,6 +109,7 @@ pub enum KnownModule {
#[allow(dead_code)]
Abc, // currently only used in tests
Collections,
Inspect,
KnotExtensions,
}
@@ -123,6 +124,7 @@ impl KnownModule {
Self::Sys => "sys",
Self::Abc => "abc",
Self::Collections => "collections",
Self::Inspect => "inspect",
Self::KnotExtensions => "knot_extensions",
}
}
@@ -149,6 +151,7 @@ impl KnownModule {
"sys" => Some(Self::Sys),
"abc" => Some(Self::Abc),
"collections" => Some(Self::Collections),
"inspect" => Some(Self::Inspect),
"knot_extensions" => Some(Self::KnotExtensions),
_ => None,
}

View File

@@ -15,7 +15,7 @@ use crate::module_name::ModuleName;
use crate::semantic_index::ast_ids::node_key::ExpressionNodeKey;
use crate::semantic_index::ast_ids::AstIdsBuilder;
use crate::semantic_index::attribute_assignment::{AttributeAssignment, AttributeAssignments};
use crate::semantic_index::constraint::PatternConstraintKind;
use crate::semantic_index::constraint::{PatternConstraintKind, ScopedConstraintId};
use crate::semantic_index::definition::{
AssignmentDefinitionNodeRef, ComprehensionDefinitionNodeRef, Definition, DefinitionNodeKey,
DefinitionNodeRef, ForStmtDefinitionNodeRef, ImportFromDefinitionNodeRef,
@@ -26,7 +26,7 @@ use crate::semantic_index::symbol::{
SymbolTableBuilder,
};
use crate::semantic_index::use_def::{
EagerBindingsKey, FlowSnapshot, ScopedConstraintId, ScopedEagerBindingsId, UseDefMapBuilder,
EagerBindingsKey, FlowSnapshot, ScopedEagerBindingsId, UseDefMapBuilder,
};
use crate::semantic_index::SemanticIndex;
use crate::unpack::{Unpack, UnpackValue};
@@ -294,7 +294,7 @@ impl<'db> SemanticIndexBuilder<'db> {
&self.use_def_maps[scope_id]
}
fn current_visibility_constraints_mut(&mut self) -> &mut VisibilityConstraintsBuilder<'db> {
fn current_visibility_constraints_mut(&mut self) -> &mut VisibilityConstraintsBuilder {
let scope_id = self.current_scope();
&mut self.use_def_maps[scope_id].visibility_constraints
}
@@ -406,16 +406,12 @@ impl<'db> SemanticIndexBuilder<'db> {
}
/// Negates a constraint and adds it to the list of all constraints, does not record it.
fn add_negated_constraint(
&mut self,
constraint: Constraint<'db>,
) -> (Constraint<'db>, ScopedConstraintId) {
fn add_negated_constraint(&mut self, constraint: Constraint<'db>) -> ScopedConstraintId {
let negated = Constraint {
node: constraint.node,
is_positive: false,
};
let id = self.current_use_def_map_mut().add_constraint(negated);
(negated, id)
self.current_use_def_map_mut().add_constraint(negated)
}
/// Records a previously added constraint by adding it to all live bindings.
@@ -431,7 +427,7 @@ impl<'db> SemanticIndexBuilder<'db> {
/// Negates the given constraint and then adds it to all live bindings.
fn record_negated_constraint(&mut self, constraint: Constraint<'db>) -> ScopedConstraintId {
let (_, id) = self.add_negated_constraint(constraint);
let id = self.add_negated_constraint(constraint);
self.record_constraint_id(id);
id
}
@@ -460,9 +456,10 @@ impl<'db> SemanticIndexBuilder<'db> {
&mut self,
constraint: Constraint<'db>,
) -> ScopedVisibilityConstraintId {
let constraint_id = self.current_use_def_map_mut().add_constraint(constraint);
let id = self
.current_visibility_constraints_mut()
.add_atom(constraint, 0);
.add_atom(constraint_id);
self.record_visibility_constraint_id(id);
id
}
@@ -1192,12 +1189,14 @@ where
// We need multiple copies of the visibility constraint for the while condition,
// since we need to model situations where the first evaluation of the condition
// returns True, but a later evaluation returns False.
let first_constraint_id = self.current_use_def_map_mut().add_constraint(constraint);
let later_constraint_id = self.current_use_def_map_mut().add_constraint(constraint);
let first_vis_constraint_id = self
.current_visibility_constraints_mut()
.add_atom(constraint, 0);
.add_atom(first_constraint_id);
let later_vis_constraint_id = self
.current_visibility_constraints_mut()
.add_atom(constraint, 1);
.add_atom(later_constraint_id);
// Save aside any break states from an outer loop
let saved_break_states = std::mem::take(&mut self.loop_break_states);
@@ -1778,13 +1777,13 @@ where
// anymore.
if index < values.len() - 1 {
let constraint = self.build_constraint(value);
let (constraint, constraint_id) = match op {
ast::BoolOp::And => (constraint, self.add_constraint(constraint)),
let constraint_id = match op {
ast::BoolOp::And => self.add_constraint(constraint),
ast::BoolOp::Or => self.add_negated_constraint(constraint),
};
let visibility_constraint = self
.current_visibility_constraints_mut()
.add_atom(constraint, 0);
.add_atom(constraint_id);
let after_expr = self.flow_snapshot();

View File

@@ -1,10 +1,40 @@
use ruff_db::files::File;
use ruff_index::{newtype_index, IndexVec};
use ruff_python_ast::Singleton;
use crate::db::Db;
use crate::semantic_index::expression::Expression;
use crate::semantic_index::symbol::{FileScopeId, ScopeId};
// A scoped identifier for each `Constraint` in a scope.
#[newtype_index]
#[derive(Ord, PartialOrd)]
pub(crate) struct ScopedConstraintId;
// A collection of constraints. This is currently stored in `UseDefMap`, which means we maintain a
// separate set of constraints for each scope in a file.
pub(crate) type Constraints<'db> = IndexVec<ScopedConstraintId, Constraint<'db>>;
#[derive(Debug, Default)]
pub(crate) struct ConstraintsBuilder<'db> {
constraints: IndexVec<ScopedConstraintId, Constraint<'db>>,
}
impl<'db> ConstraintsBuilder<'db> {
/// Adds a constraint. Note that we do not deduplicate constraints. If you add a `Constraint`
/// more than once, you will get distinct `ScopedConstraintId`s for each one. (This lets you
/// model constraint expressions that might evaluate to different values at different points of
/// execution.)
pub(crate) fn add_constraint(&mut self, constraint: Constraint<'db>) -> ScopedConstraintId {
self.constraints.push(constraint)
}
pub(crate) fn build(mut self) -> Constraints<'db> {
self.constraints.shrink_to_fit();
self.constraints
}
}
#[derive(Clone, Copy, Debug, Hash, PartialEq, Eq, salsa::Update)]
pub(crate) struct Constraint<'db> {
pub(crate) node: ConstraintNode<'db>,

View File

@@ -165,7 +165,7 @@
//! don't actually store these "list of visible definitions" as a vector of [`Definition`].
//! Instead, [`SymbolBindings`] and [`SymbolDeclarations`] are structs which use bit-sets to track
//! definitions (and constraints, in the case of bindings) in terms of [`ScopedDefinitionId`] and
//! [`ScopedConstraintId`], which are indices into the `all_definitions` and `all_constraints`
//! [`ScopedConstraintId`], which are indices into the `all_definitions` and `constraints`
//! indexvecs in the [`UseDefMap`].
//!
//! There is another special kind of possible "definition" for a symbol: there might be a path from
@@ -255,28 +255,27 @@
//! snapshot, and merging a snapshot into the current state. The logic using these methods lives in
//! [`SemanticIndexBuilder`](crate::semantic_index::builder::SemanticIndexBuilder), e.g. where it
//! visits a `StmtIf` node.
pub(crate) use self::symbol_state::ScopedConstraintId;
use self::symbol_state::{
BindingIdWithConstraintsIterator, ConstraintIdIterator, DeclarationIdIterator,
ScopedDefinitionId, SymbolBindings, SymbolDeclarations, SymbolState,
};
use crate::semantic_index::ast_ids::ScopedUseId;
use crate::semantic_index::definition::Definition;
use crate::semantic_index::symbol::{FileScopeId, ScopedSymbolId};
use crate::semantic_index::use_def::symbol_state::DeclarationIdWithConstraint;
use crate::visibility_constraints::{
ScopedVisibilityConstraintId, VisibilityConstraints, VisibilityConstraintsBuilder,
};
use ruff_index::{newtype_index, IndexVec};
use rustc_hash::FxHashMap;
use super::constraint::Constraint;
use self::symbol_state::{
ConstraintIndexIterator, LiveBindingsIterator, LiveDeclaration, LiveDeclarationsIterator,
ScopedDefinitionId, SymbolBindings, SymbolDeclarations, SymbolState,
};
use crate::semantic_index::ast_ids::ScopedUseId;
use crate::semantic_index::constraint::{
Constraint, Constraints, ConstraintsBuilder, ScopedConstraintId,
};
use crate::semantic_index::definition::Definition;
use crate::semantic_index::symbol::{FileScopeId, ScopedSymbolId};
use crate::visibility_constraints::{
ScopedVisibilityConstraintId, VisibilityConstraints, VisibilityConstraintsBuilder,
};
mod bitset;
mod symbol_state;
type AllConstraints<'db> = IndexVec<ScopedConstraintId, Constraint<'db>>;
/// Applicable definitions and constraints for every use of a name.
#[derive(Debug, PartialEq, Eq, salsa::Update)]
pub(crate) struct UseDefMap<'db> {
@@ -285,10 +284,10 @@ pub(crate) struct UseDefMap<'db> {
all_definitions: IndexVec<ScopedDefinitionId, Option<Definition<'db>>>,
/// Array of [`Constraint`] in this scope.
all_constraints: AllConstraints<'db>,
constraints: Constraints<'db>,
/// Array of visibility constraints in this scope.
visibility_constraints: VisibilityConstraints<'db>,
visibility_constraints: VisibilityConstraints,
/// [`SymbolBindings`] reaching a [`ScopedUseId`].
bindings_by_use: IndexVec<ScopedUseId, SymbolBindings>,
@@ -370,7 +369,7 @@ impl<'db> UseDefMap<'db> {
) -> BindingWithConstraintsIterator<'map, 'db> {
BindingWithConstraintsIterator {
all_definitions: &self.all_definitions,
all_constraints: &self.all_constraints,
constraints: &self.constraints,
visibility_constraints: &self.visibility_constraints,
inner: bindings.iter(),
}
@@ -382,6 +381,7 @@ impl<'db> UseDefMap<'db> {
) -> DeclarationsIterator<'map, 'db> {
DeclarationsIterator {
all_definitions: &self.all_definitions,
constraints: &self.constraints,
visibility_constraints: &self.visibility_constraints,
inner: declarations.iter(),
}
@@ -415,26 +415,26 @@ type EagerBindings = IndexVec<ScopedEagerBindingsId, SymbolBindings>;
#[derive(Debug)]
pub(crate) struct BindingWithConstraintsIterator<'map, 'db> {
all_definitions: &'map IndexVec<ScopedDefinitionId, Option<Definition<'db>>>,
all_constraints: &'map AllConstraints<'db>,
pub(crate) visibility_constraints: &'map VisibilityConstraints<'db>,
inner: BindingIdWithConstraintsIterator<'map>,
pub(crate) constraints: &'map Constraints<'db>,
pub(crate) visibility_constraints: &'map VisibilityConstraints,
inner: LiveBindingsIterator<'map>,
}
impl<'map, 'db> Iterator for BindingWithConstraintsIterator<'map, 'db> {
type Item = BindingWithConstraints<'map, 'db>;
fn next(&mut self) -> Option<Self::Item> {
let all_constraints = self.all_constraints;
let constraints = self.constraints;
self.inner
.next()
.map(|binding_id_with_constraints| BindingWithConstraints {
binding: self.all_definitions[binding_id_with_constraints.definition],
constraints: ConstraintsIterator {
all_constraints,
constraint_ids: binding_id_with_constraints.constraint_ids,
.map(|live_binding| BindingWithConstraints {
binding: self.all_definitions[live_binding.binding],
narrowing_constraints: ConstraintsIterator {
constraints,
constraint_ids: live_binding.narrowing_constraints.iter(),
},
visibility_constraint: binding_id_with_constraints.visibility_constraint,
visibility_constraint: live_binding.visibility_constraint,
})
}
}
@@ -443,13 +443,13 @@ impl std::iter::FusedIterator for BindingWithConstraintsIterator<'_, '_> {}
pub(crate) struct BindingWithConstraints<'map, 'db> {
pub(crate) binding: Option<Definition<'db>>,
pub(crate) constraints: ConstraintsIterator<'map, 'db>,
pub(crate) narrowing_constraints: ConstraintsIterator<'map, 'db>,
pub(crate) visibility_constraint: ScopedVisibilityConstraintId,
}
pub(crate) struct ConstraintsIterator<'map, 'db> {
all_constraints: &'map AllConstraints<'db>,
constraint_ids: ConstraintIdIterator<'map>,
constraints: &'map Constraints<'db>,
constraint_ids: ConstraintIndexIterator<'map>,
}
impl<'db> Iterator for ConstraintsIterator<'_, 'db> {
@@ -458,7 +458,7 @@ impl<'db> Iterator for ConstraintsIterator<'_, 'db> {
fn next(&mut self) -> Option<Self::Item> {
self.constraint_ids
.next()
.map(|constraint_id| self.all_constraints[constraint_id])
.map(|constraint_id| self.constraints[ScopedConstraintId::from_u32(constraint_id)])
}
}
@@ -466,8 +466,9 @@ impl std::iter::FusedIterator for ConstraintsIterator<'_, '_> {}
pub(crate) struct DeclarationsIterator<'map, 'db> {
all_definitions: &'map IndexVec<ScopedDefinitionId, Option<Definition<'db>>>,
pub(crate) visibility_constraints: &'map VisibilityConstraints<'db>,
inner: DeclarationIdIterator<'map>,
pub(crate) constraints: &'map Constraints<'db>,
pub(crate) visibility_constraints: &'map VisibilityConstraints,
inner: LiveDeclarationsIterator<'map>,
}
pub(crate) struct DeclarationWithConstraint<'db> {
@@ -480,13 +481,13 @@ impl<'db> Iterator for DeclarationsIterator<'_, 'db> {
fn next(&mut self) -> Option<Self::Item> {
self.inner.next().map(
|DeclarationIdWithConstraint {
definition,
|LiveDeclaration {
declaration,
visibility_constraint,
}| {
DeclarationWithConstraint {
declaration: self.all_definitions[definition],
visibility_constraint,
declaration: self.all_definitions[*declaration],
visibility_constraint: *visibility_constraint,
}
},
)
@@ -507,11 +508,11 @@ pub(super) struct UseDefMapBuilder<'db> {
/// Append-only array of [`Definition`].
all_definitions: IndexVec<ScopedDefinitionId, Option<Definition<'db>>>,
/// Append-only array of [`Constraint`].
all_constraints: AllConstraints<'db>,
/// Builder of constraints.
constraints: ConstraintsBuilder<'db>,
/// Builder of visibility constraints.
pub(super) visibility_constraints: VisibilityConstraintsBuilder<'db>,
pub(super) visibility_constraints: VisibilityConstraintsBuilder,
/// A constraint which describes the visibility of the unbound/undeclared state, i.e.
/// whether or not the start of the scope is visible. This is important for cases like
@@ -540,7 +541,7 @@ impl Default for UseDefMapBuilder<'_> {
fn default() -> Self {
Self {
all_definitions: IndexVec::from_iter([None]),
all_constraints: IndexVec::new(),
constraints: ConstraintsBuilder::default(),
visibility_constraints: VisibilityConstraintsBuilder::default(),
scope_start_visibility: ScopedVisibilityConstraintId::ALWAYS_TRUE,
bindings_by_use: IndexVec::new(),
@@ -573,7 +574,7 @@ impl<'db> UseDefMapBuilder<'db> {
}
pub(super) fn add_constraint(&mut self, constraint: Constraint<'db>) -> ScopedConstraintId {
self.all_constraints.push(constraint)
self.constraints.add_constraint(constraint)
}
pub(super) fn record_constraint_id(&mut self, constraint: ScopedConstraintId) {
@@ -753,7 +754,6 @@ impl<'db> UseDefMapBuilder<'db> {
pub(super) fn finish(mut self) -> UseDefMap<'db> {
self.all_definitions.shrink_to_fit();
self.all_constraints.shrink_to_fit();
self.symbol_states.shrink_to_fit();
self.bindings_by_use.shrink_to_fit();
self.declarations_by_binding.shrink_to_fit();
@@ -762,7 +762,7 @@ impl<'db> UseDefMapBuilder<'db> {
UseDefMap {
all_definitions: self.all_definitions,
all_constraints: self.all_constraints,
constraints: self.constraints.build(),
visibility_constraints: self.visibility_constraints.build(),
bindings_by_use: self.bindings_by_use,
public_symbols: self.symbol_states,

View File

@@ -25,13 +25,6 @@ impl<const B: usize> Default for BitSet<B> {
}
impl<const B: usize> BitSet<B> {
/// Create and return a new [`BitSet`] with a single `value` inserted.
pub(super) fn with(value: u32) -> Self {
let mut bitset = Self::default();
bitset.insert(value);
bitset
}
/// Convert from Inline to Heap, if needed, and resize the Heap vector, if needed.
fn resize(&mut self, value: u32) {
let num_blocks_needed = (value / 64) + 1;
@@ -93,19 +86,6 @@ impl<const B: usize> BitSet<B> {
}
}
/// Union in-place with another [`BitSet`].
pub(super) fn union(&mut self, other: &BitSet<B>) {
let mut max_len = self.blocks().len();
let other_len = other.blocks().len();
if other_len > max_len {
max_len = other_len;
self.resize_blocks(max_len);
}
for (my_block, other_block) in self.blocks_mut().iter_mut().zip(other.blocks()) {
*my_block |= other_block;
}
}
/// Return an iterator over the values (in ascending order) in this [`BitSet`].
pub(super) fn iter(&self) -> BitSetIterator<'_, B> {
let blocks = self.blocks();
@@ -158,6 +138,15 @@ impl<const B: usize> std::iter::FusedIterator for BitSetIterator<'_, B> {}
mod tests {
use super::BitSet;
impl<const B: usize> BitSet<B> {
/// Create and return a new [`BitSet`] with a single `value` inserted.
pub(super) fn with(value: u32) -> Self {
let mut bitset = Self::default();
bitset.insert(value);
bitset
}
}
fn assert_bitset<const B: usize>(bitset: &BitSet<B>, contents: &[u32]) {
assert_eq!(bitset.iter().collect::<Vec<_>>(), contents);
}
@@ -235,59 +224,6 @@ mod tests {
assert_bitset(&b1, &[89]);
}
#[test]
fn union() {
let mut b1 = BitSet::<1>::with(2);
let b2 = BitSet::<1>::with(4);
b1.union(&b2);
assert_bitset(&b1, &[2, 4]);
}
#[test]
fn union_mixed_1() {
let mut b1 = BitSet::<1>::with(4);
let mut b2 = BitSet::<1>::with(4);
b1.insert(89);
b2.insert(5);
b1.union(&b2);
assert_bitset(&b1, &[4, 5, 89]);
}
#[test]
fn union_mixed_2() {
let mut b1 = BitSet::<1>::with(4);
let mut b2 = BitSet::<1>::with(4);
b1.insert(23);
b2.insert(89);
b1.union(&b2);
assert_bitset(&b1, &[4, 23, 89]);
}
#[test]
fn union_heap() {
let mut b1 = BitSet::<1>::with(4);
let mut b2 = BitSet::<1>::with(4);
b1.insert(89);
b2.insert(90);
b1.union(&b2);
assert_bitset(&b1, &[4, 89, 90]);
}
#[test]
fn union_heap_2() {
let mut b1 = BitSet::<1>::with(89);
let mut b2 = BitSet::<1>::with(89);
b1.insert(91);
b2.insert(90);
b1.union(&b2);
assert_bitset(&b1, &[89, 90, 91]);
}
#[test]
fn multiple_blocks() {
let mut b = BitSet::<2>::with(120);

View File

@@ -46,14 +46,16 @@
use itertools::{EitherOrBoth, Itertools};
use ruff_index::newtype_index;
use smallvec::SmallVec;
use smallvec::{smallvec, SmallVec};
use crate::semantic_index::constraint::ScopedConstraintId;
use crate::semantic_index::use_def::bitset::{BitSet, BitSetIterator};
use crate::semantic_index::use_def::VisibilityConstraintsBuilder;
use crate::visibility_constraints::ScopedVisibilityConstraintId;
/// A newtype-index for a definition in a particular scope.
#[newtype_index]
#[derive(Ord, PartialOrd)]
pub(super) struct ScopedDefinitionId;
impl ScopedDefinitionId {
@@ -65,89 +67,54 @@ impl ScopedDefinitionId {
pub(super) const UNBOUND: ScopedDefinitionId = ScopedDefinitionId::from_u32(0);
}
/// A newtype-index for a constraint expression in a particular scope.
#[newtype_index]
pub(crate) struct ScopedConstraintId;
/// Can reference this * 64 total definitions inline; more will fall back to the heap.
const INLINE_BINDING_BLOCKS: usize = 3;
/// A [`BitSet`] of [`ScopedDefinitionId`], representing live bindings of a symbol in a scope.
type Bindings = BitSet<INLINE_BINDING_BLOCKS>;
type BindingsIterator<'a> = BitSetIterator<'a, INLINE_BINDING_BLOCKS>;
/// Can reference this * 64 total declarations inline; more will fall back to the heap.
const INLINE_DECLARATION_BLOCKS: usize = 3;
/// A [`BitSet`] of [`ScopedDefinitionId`], representing live declarations of a symbol in a scope.
type Declarations = BitSet<INLINE_DECLARATION_BLOCKS>;
type DeclarationsIterator<'a> = BitSetIterator<'a, INLINE_DECLARATION_BLOCKS>;
/// Can reference this * 64 total constraints inline; more will fall back to the heap.
const INLINE_CONSTRAINT_BLOCKS: usize = 2;
/// Can keep inline this many live bindings per symbol at a given time; more will go to heap.
const INLINE_BINDINGS_PER_SYMBOL: usize = 4;
/// Can keep inline this many live bindings or declarations per symbol at a given time; more will
/// go to heap.
const INLINE_DEFINITIONS_PER_SYMBOL: usize = 4;
/// Which constraints apply to a given binding?
type Constraints = BitSet<INLINE_CONSTRAINT_BLOCKS>;
type InlineConstraintArray = [Constraints; INLINE_BINDINGS_PER_SYMBOL];
/// One [`BitSet`] of applicable [`ScopedConstraintId`]s per live binding.
type ConstraintsPerBinding = SmallVec<InlineConstraintArray>;
/// Iterate over all constraints for a single binding.
type ConstraintsIterator<'a> = std::slice::Iter<'a, Constraints>;
const INLINE_VISIBILITY_CONSTRAINTS: usize = 4;
type InlineVisibilityConstraintsArray =
[ScopedVisibilityConstraintId; INLINE_VISIBILITY_CONSTRAINTS];
/// One [`ScopedVisibilityConstraintId`] per live declaration.
type VisibilityConstraintPerDeclaration = SmallVec<InlineVisibilityConstraintsArray>;
/// One [`ScopedVisibilityConstraintId`] per live binding.
type VisibilityConstraintPerBinding = SmallVec<InlineVisibilityConstraintsArray>;
/// Iterator over the visibility constraints for all live bindings/declarations.
type VisibilityConstraintsIterator<'a> = std::slice::Iter<'a, ScopedVisibilityConstraintId>;
pub(super) type ConstraintIndexIterator<'a> = BitSetIterator<'a, INLINE_CONSTRAINT_BLOCKS>;
/// Live declarations for a single symbol at some point in control flow, with their
/// corresponding visibility constraints.
#[derive(Clone, Debug, Default, PartialEq, Eq, salsa::Update)]
pub(super) struct SymbolDeclarations {
/// [`BitSet`]: which declarations (as [`ScopedDefinitionId`]) can reach the current location?
///
/// Invariant: Because this is a `BitSet`, it can be viewed as a _sorted_ set of definition
/// IDs. The `visibility_constraints` field stores constraints for each definition. Therefore
/// those fields must always have the same `len()` as `live_declarations`, and the elements
/// must appear in the same order. Effectively, this means that elements must always be added
/// in sorted order, or via a binary search that determines the correct place to insert new
/// constraints.
pub(crate) live_declarations: Declarations,
/// For each live declaration, which visibility constraint applies to it?
pub(crate) visibility_constraints: VisibilityConstraintPerDeclaration,
/// A list of live declarations for this symbol, sorted by their `ScopedDefinitionId`
live_declarations: SmallVec<[LiveDeclaration; INLINE_DEFINITIONS_PER_SYMBOL]>,
}
/// One of the live declarations for a single symbol at some point in control flow.
#[derive(Clone, Debug, PartialEq, Eq)]
pub(super) struct LiveDeclaration {
pub(super) declaration: ScopedDefinitionId,
pub(super) visibility_constraint: ScopedVisibilityConstraintId,
}
pub(super) type LiveDeclarationsIterator<'a> = std::slice::Iter<'a, LiveDeclaration>;
impl SymbolDeclarations {
fn undeclared(scope_start_visibility: ScopedVisibilityConstraintId) -> Self {
let initial_declaration = LiveDeclaration {
declaration: ScopedDefinitionId::UNBOUND,
visibility_constraint: scope_start_visibility,
};
Self {
live_declarations: Declarations::with(0),
visibility_constraints: VisibilityConstraintPerDeclaration::from_iter([
scope_start_visibility,
]),
live_declarations: smallvec![initial_declaration],
}
}
/// Record a newly-encountered declaration for this symbol.
fn record_declaration(&mut self, declaration_id: ScopedDefinitionId) {
self.live_declarations = Declarations::with(declaration_id.into());
self.visibility_constraints = VisibilityConstraintPerDeclaration::with_capacity(1);
self.visibility_constraints
.push(ScopedVisibilityConstraintId::ALWAYS_TRUE);
fn record_declaration(&mut self, declaration: ScopedDefinitionId) {
// The new declaration replaces all previous live declaration in this path.
self.live_declarations.clear();
self.live_declarations.push(LiveDeclaration {
declaration,
visibility_constraint: ScopedVisibilityConstraintId::ALWAYS_TRUE,
});
}
/// Add given visibility constraint to all live declarations.
@@ -156,45 +123,62 @@ impl SymbolDeclarations {
visibility_constraints: &mut VisibilityConstraintsBuilder,
constraint: ScopedVisibilityConstraintId,
) {
for existing in &mut self.visibility_constraints {
*existing = visibility_constraints.add_and_constraint(*existing, constraint);
for declaration in &mut self.live_declarations {
declaration.visibility_constraint = visibility_constraints
.add_and_constraint(declaration.visibility_constraint, constraint);
}
}
/// Return an iterator over live declarations for this symbol.
pub(super) fn iter(&self) -> DeclarationIdIterator {
DeclarationIdIterator {
declarations: self.live_declarations.iter(),
visibility_constraints: self.visibility_constraints.iter(),
pub(super) fn iter(&self) -> LiveDeclarationsIterator<'_> {
self.live_declarations.iter()
}
/// Iterate over the IDs of each currently live declaration for this symbol
fn iter_declarations(&self) -> impl Iterator<Item = ScopedDefinitionId> + '_ {
self.iter().map(|lb| lb.declaration)
}
fn simplify_visibility_constraints(&mut self, other: SymbolDeclarations) {
// If the set of live declarations hasn't changed, don't simplify.
if self.live_declarations.len() != other.live_declarations.len()
|| !self.iter_declarations().eq(other.iter_declarations())
{
return;
}
for (declaration, other_declaration) in self
.live_declarations
.iter_mut()
.zip(other.live_declarations)
{
declaration.visibility_constraint = other_declaration.visibility_constraint;
}
}
fn merge(&mut self, b: Self, visibility_constraints: &mut VisibilityConstraintsBuilder) {
let a = std::mem::take(self);
self.live_declarations = a.live_declarations.clone();
self.live_declarations.union(&b.live_declarations);
// Invariant: These zips are well-formed since we maintain an invariant that all of our
// fields are sets/vecs with the same length.
let a = (a.live_declarations.iter()).zip(a.visibility_constraints);
let b = (b.live_declarations.iter()).zip(b.visibility_constraints);
// Invariant: merge_join_by consumes the two iterators in sorted order, which ensures that
// the definition IDs and constraints line up correctly in the merged result. If a
// definition is found in both `a` and `b`, we compose the constraints from the two paths
// in an appropriate way (intersection for narrowing constraints; ternary OR for visibility
// constraints). If a definition is found in only one path, it is used as-is.
for zipped in a.merge_join_by(b, |(a_decl, _), (b_decl, _)| a_decl.cmp(b_decl)) {
// the merged `live_declarations` vec remains sorted. If a definition is found in both `a`
// and `b`, we compose the constraints from the two paths in an appropriate way
// (intersection for narrowing constraints; ternary OR for visibility constraints). If a
// definition is found in only one path, it is used as-is.
let a = a.live_declarations.into_iter();
let b = b.live_declarations.into_iter();
for zipped in a.merge_join_by(b, |a, b| a.declaration.cmp(&b.declaration)) {
match zipped {
EitherOrBoth::Both((_, a_vis_constraint), (_, b_vis_constraint)) => {
let vis_constraint = visibility_constraints
.add_or_constraint(a_vis_constraint, b_vis_constraint);
self.visibility_constraints.push(vis_constraint);
EitherOrBoth::Both(a, b) => {
let visibility_constraint = visibility_constraints
.add_or_constraint(a.visibility_constraint, b.visibility_constraint);
self.live_declarations.push(LiveDeclaration {
declaration: a.declaration,
visibility_constraint,
});
}
EitherOrBoth::Left((_, vis_constraint))
| EitherOrBoth::Right((_, vis_constraint)) => {
self.visibility_constraints.push(vis_constraint);
EitherOrBoth::Left(declaration) | EitherOrBoth::Right(declaration) => {
self.live_declarations.push(declaration);
}
}
}
@@ -205,57 +189,52 @@ impl SymbolDeclarations {
/// with a set of narrowing constraints and a visibility constraint.
#[derive(Clone, Debug, Default, PartialEq, Eq, salsa::Update)]
pub(super) struct SymbolBindings {
/// [`BitSet`]: which bindings (as [`ScopedDefinitionId`]) can reach the current location?
///
/// Invariant: Because this is a `BitSet`, it can be viewed as a _sorted_ set of definition
/// IDs. The `constraints` and `visibility_constraints` field stores constraints for each
/// definition. Therefore those fields must always have the same `len()` as
/// `live_bindings`, and the elements must appear in the same order. Effectively, this means
/// that elements must always be added in sorted order, or via a binary search that determines
/// the correct place to insert new constraints.
live_bindings: Bindings,
/// For each live binding, which [`ScopedConstraintId`] apply?
///
/// This is a [`smallvec::SmallVec`] which should always have one [`BitSet`] of constraints per
/// binding in `live_bindings`.
constraints: ConstraintsPerBinding,
/// For each live binding, which visibility constraint applies to it?
visibility_constraints: VisibilityConstraintPerBinding,
/// A list of live bindings for this symbol, sorted by their `ScopedDefinitionId`
live_bindings: SmallVec<[LiveBinding; INLINE_DEFINITIONS_PER_SYMBOL]>,
}
/// One of the live bindings for a single symbol at some point in control flow.
#[derive(Clone, Debug, PartialEq, Eq)]
pub(super) struct LiveBinding {
pub(super) binding: ScopedDefinitionId,
pub(super) narrowing_constraints: Constraints,
pub(super) visibility_constraint: ScopedVisibilityConstraintId,
}
pub(super) type LiveBindingsIterator<'a> = std::slice::Iter<'a, LiveBinding>;
impl SymbolBindings {
fn unbound(scope_start_visibility: ScopedVisibilityConstraintId) -> Self {
let initial_binding = LiveBinding {
binding: ScopedDefinitionId::UNBOUND,
narrowing_constraints: Constraints::default(),
visibility_constraint: scope_start_visibility,
};
Self {
live_bindings: Bindings::with(ScopedDefinitionId::UNBOUND.as_u32()),
constraints: ConstraintsPerBinding::from_iter([Constraints::default()]),
visibility_constraints: VisibilityConstraintPerBinding::from_iter([
scope_start_visibility,
]),
live_bindings: smallvec![initial_binding],
}
}
/// Record a newly-encountered binding for this symbol.
pub(super) fn record_binding(
&mut self,
binding_id: ScopedDefinitionId,
binding: ScopedDefinitionId,
visibility_constraint: ScopedVisibilityConstraintId,
) {
// The new binding replaces all previous live bindings in this path, and has no
// constraints.
self.live_bindings = Bindings::with(binding_id.into());
self.constraints = ConstraintsPerBinding::with_capacity(1);
self.constraints.push(Constraints::default());
self.visibility_constraints = VisibilityConstraintPerBinding::with_capacity(1);
self.visibility_constraints.push(visibility_constraint);
self.live_bindings.clear();
self.live_bindings.push(LiveBinding {
binding,
narrowing_constraints: Constraints::default(),
visibility_constraint,
});
}
/// Add given constraint to all live bindings.
pub(super) fn record_constraint(&mut self, constraint_id: ScopedConstraintId) {
for bitset in &mut self.constraints {
bitset.insert(constraint_id.into());
for binding in &mut self.live_bindings {
binding.narrowing_constraints.insert(constraint_id.into());
}
}
@@ -265,71 +244,67 @@ impl SymbolBindings {
visibility_constraints: &mut VisibilityConstraintsBuilder,
constraint: ScopedVisibilityConstraintId,
) {
for existing in &mut self.visibility_constraints {
*existing = visibility_constraints.add_and_constraint(*existing, constraint);
for binding in &mut self.live_bindings {
binding.visibility_constraint = visibility_constraints
.add_and_constraint(binding.visibility_constraint, constraint);
}
}
/// Iterate over currently live bindings for this symbol
pub(super) fn iter(&self) -> BindingIdWithConstraintsIterator {
BindingIdWithConstraintsIterator {
definitions: self.live_bindings.iter(),
constraints: self.constraints.iter(),
visibility_constraints: self.visibility_constraints.iter(),
pub(super) fn iter(&self) -> LiveBindingsIterator<'_> {
self.live_bindings.iter()
}
/// Iterate over the IDs of each currently live binding for this symbol
fn iter_bindings(&self) -> impl Iterator<Item = ScopedDefinitionId> + '_ {
self.iter().map(|lb| lb.binding)
}
fn simplify_visibility_constraints(&mut self, other: SymbolBindings) {
// If the set of live bindings hasn't changed, don't simplify.
if self.live_bindings.len() != other.live_bindings.len()
|| !self.iter_bindings().eq(other.iter_bindings())
{
return;
}
for (binding, other_binding) in self.live_bindings.iter_mut().zip(other.live_bindings) {
binding.visibility_constraint = other_binding.visibility_constraint;
}
}
fn merge(&mut self, mut b: Self, visibility_constraints: &mut VisibilityConstraintsBuilder) {
let mut a = std::mem::take(self);
self.live_bindings = a.live_bindings.clone();
self.live_bindings.union(&b.live_bindings);
// Invariant: These zips are well-formed since we maintain an invariant that all of our
// fields are sets/vecs with the same length.
//
// Performance: We iterate over the `constraints` smallvecs via mut reference, because the
// individual elements are `BitSet`s (currently 24 bytes in size), and we don't want to
// move them by value multiple times during iteration. By iterating by reference, we only
// have to copy single pointers around. In the loop below, the `std::mem::take` calls
// specify precisely where we want to move them into the merged `constraints` smallvec.
//
// We don't need a similar optimization for `visibility_constraints`, since those elements
// are 32-bit IndexVec IDs, and so are already cheap to move/copy.
let a = (a.live_bindings.iter())
.zip(a.constraints.iter_mut())
.zip(a.visibility_constraints);
let b = (b.live_bindings.iter())
.zip(b.constraints.iter_mut())
.zip(b.visibility_constraints);
fn merge(&mut self, b: Self, visibility_constraints: &mut VisibilityConstraintsBuilder) {
let a = std::mem::take(self);
// Invariant: merge_join_by consumes the two iterators in sorted order, which ensures that
// the definition IDs and constraints line up correctly in the merged result. If a
// definition is found in both `a` and `b`, we compose the constraints from the two paths
// in an appropriate way (intersection for narrowing constraints; ternary OR for visibility
// constraints). If a definition is found in only one path, it is used as-is.
for zipped in a.merge_join_by(b, |((a_def, _), _), ((b_def, _), _)| a_def.cmp(b_def)) {
// the merged `live_bindings` vec remains sorted. If a definition is found in both `a` and
// `b`, we compose the constraints from the two paths in an appropriate way (intersection
// for narrowing constraints; ternary OR for visibility constraints). If a definition is
// found in only one path, it is used as-is.
let a = a.live_bindings.into_iter();
let b = b.live_bindings.into_iter();
for zipped in a.merge_join_by(b, |a, b| a.binding.cmp(&b.binding)) {
match zipped {
EitherOrBoth::Both(
((_, a_constraints), a_vis_constraint),
((_, b_constraints), b_vis_constraint),
) => {
EitherOrBoth::Both(a, b) => {
// If the same definition is visible through both paths, any constraint
// that applies on only one path is irrelevant to the resulting type from
// unioning the two paths, so we intersect the constraints.
let constraints = a_constraints;
constraints.intersect(b_constraints);
self.constraints.push(std::mem::take(constraints));
let mut narrowing_constraints = a.narrowing_constraints;
narrowing_constraints.intersect(&b.narrowing_constraints);
// For visibility constraints, we merge them using a ternary OR operation:
let vis_constraint = visibility_constraints
.add_or_constraint(a_vis_constraint, b_vis_constraint);
self.visibility_constraints.push(vis_constraint);
let visibility_constraint = visibility_constraints
.add_or_constraint(a.visibility_constraint, b.visibility_constraint);
self.live_bindings.push(LiveBinding {
binding: a.binding,
narrowing_constraints,
visibility_constraint,
});
}
EitherOrBoth::Left(((_, constraints), vis_constraint))
| EitherOrBoth::Right(((_, constraints), vis_constraint)) => {
self.constraints.push(std::mem::take(constraints));
self.visibility_constraints.push(vis_constraint);
EitherOrBoth::Left(binding) | EitherOrBoth::Right(binding) => {
self.live_bindings.push(binding);
}
}
}
@@ -379,14 +354,14 @@ impl SymbolState {
.record_visibility_constraint(visibility_constraints, constraint);
}
/// Simplifies this snapshot to have the same visibility constraints as a previous point in the
/// control flow, but only if the set of live bindings or declarations for this symbol hasn't
/// changed.
pub(super) fn simplify_visibility_constraints(&mut self, snapshot_state: SymbolState) {
if self.bindings.live_bindings == snapshot_state.bindings.live_bindings {
self.bindings.visibility_constraints = snapshot_state.bindings.visibility_constraints;
}
if self.declarations.live_declarations == snapshot_state.declarations.live_declarations {
self.declarations.visibility_constraints =
snapshot_state.declarations.visibility_constraints;
}
self.bindings
.simplify_visibility_constraints(snapshot_state.bindings);
self.declarations
.simplify_visibility_constraints(snapshot_state.declarations);
}
/// Record a newly-encountered declaration of this symbol.
@@ -414,98 +389,6 @@ impl SymbolState {
}
}
/// A single binding (as [`ScopedDefinitionId`]) with an iterator of its applicable
/// narrowing constraints ([`ScopedConstraintId`]) and a corresponding visibility
/// visibility constraint ([`ScopedVisibilityConstraintId`]).
#[derive(Debug)]
pub(super) struct BindingIdWithConstraints<'map> {
pub(super) definition: ScopedDefinitionId,
pub(super) constraint_ids: ConstraintIdIterator<'map>,
pub(super) visibility_constraint: ScopedVisibilityConstraintId,
}
#[derive(Debug)]
pub(super) struct BindingIdWithConstraintsIterator<'map> {
definitions: BindingsIterator<'map>,
constraints: ConstraintsIterator<'map>,
visibility_constraints: VisibilityConstraintsIterator<'map>,
}
impl<'map> Iterator for BindingIdWithConstraintsIterator<'map> {
type Item = BindingIdWithConstraints<'map>;
fn next(&mut self) -> Option<Self::Item> {
match (
self.definitions.next(),
self.constraints.next(),
self.visibility_constraints.next(),
) {
(None, None, None) => None,
(Some(def), Some(constraints), Some(visibility_constraint_id)) => {
Some(BindingIdWithConstraints {
definition: ScopedDefinitionId::from_u32(def),
constraint_ids: ConstraintIdIterator {
wrapped: constraints.iter(),
},
visibility_constraint: *visibility_constraint_id,
})
}
// SAFETY: see above.
_ => unreachable!("definitions and constraints length mismatch"),
}
}
}
impl std::iter::FusedIterator for BindingIdWithConstraintsIterator<'_> {}
#[derive(Debug)]
pub(super) struct ConstraintIdIterator<'a> {
wrapped: BitSetIterator<'a, INLINE_CONSTRAINT_BLOCKS>,
}
impl Iterator for ConstraintIdIterator<'_> {
type Item = ScopedConstraintId;
fn next(&mut self) -> Option<Self::Item> {
self.wrapped.next().map(ScopedConstraintId::from_u32)
}
}
impl std::iter::FusedIterator for ConstraintIdIterator<'_> {}
/// A single declaration (as [`ScopedDefinitionId`]) with a corresponding visibility
/// visibility constraint ([`ScopedVisibilityConstraintId`]).
#[derive(Debug)]
pub(super) struct DeclarationIdWithConstraint {
pub(super) definition: ScopedDefinitionId,
pub(super) visibility_constraint: ScopedVisibilityConstraintId,
}
pub(super) struct DeclarationIdIterator<'map> {
pub(crate) declarations: DeclarationsIterator<'map>,
pub(crate) visibility_constraints: VisibilityConstraintsIterator<'map>,
}
impl Iterator for DeclarationIdIterator<'_> {
type Item = DeclarationIdWithConstraint;
fn next(&mut self) -> Option<Self::Item> {
match (self.declarations.next(), self.visibility_constraints.next()) {
(None, None) => None,
(Some(declaration), Some(&visibility_constraint)) => {
Some(DeclarationIdWithConstraint {
definition: ScopedDefinitionId::from_u32(declaration),
visibility_constraint,
})
}
// SAFETY: see above.
_ => unreachable!("declarations and visibility_constraints length mismatch"),
}
}
}
impl std::iter::FusedIterator for DeclarationIdIterator<'_> {}
#[cfg(test)]
mod tests {
use super::*;
@@ -515,16 +398,16 @@ mod tests {
let actual = symbol
.bindings()
.iter()
.map(|def_id_with_constraints| {
let def_id = def_id_with_constraints.definition;
.map(|live_binding| {
let def_id = live_binding.binding;
let def = if def_id == ScopedDefinitionId::UNBOUND {
"unbound".into()
} else {
def_id.as_u32().to_string()
};
let constraints = def_id_with_constraints
.constraint_ids
.map(ScopedConstraintId::as_u32)
let constraints = live_binding
.narrowing_constraints
.iter()
.map(|idx| idx.to_string())
.collect::<Vec<_>>()
.join(", ");
@@ -540,14 +423,14 @@ mod tests {
.declarations()
.iter()
.map(
|DeclarationIdWithConstraint {
definition,
|LiveDeclaration {
declaration,
visibility_constraint: _,
}| {
if definition == ScopedDefinitionId::UNBOUND {
if *declaration == ScopedDefinitionId::UNBOUND {
"undeclared".into()
} else {
definition.as_u32().to_string()
declaration.as_u32().to_string()
}
},
)

View File

@@ -1,10 +1,9 @@
use ruff_db::files::File;
use ruff_python_ast as ast;
use crate::module_resolver::file_to_module;
use crate::semantic_index::definition::Definition;
use crate::semantic_index::symbol::{ScopeId, ScopedSymbolId};
use crate::semantic_index::{self, global_scope, use_def_map, DeclarationWithConstraint};
use crate::semantic_index::{global_scope, use_def_map, DeclarationWithConstraint};
use crate::semantic_index::{
symbol_table, BindingWithConstraints, BindingWithConstraintsIterator, DeclarationsIterator,
};
@@ -14,6 +13,8 @@ use crate::types::{
};
use crate::{resolve_module, Db, KnownModule, Module, Program};
pub(crate) use implicit_globals::module_type_implicit_global_symbol;
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub(crate) enum Boundness {
Bound,
@@ -183,20 +184,34 @@ pub(crate) fn symbol<'db>(db: &'db dyn Db, scope: ScopeId<'db>, name: &str) -> S
symbol_impl(db, scope, name, RequiresExplicitReExport::No)
}
/// Infers the public type of a module-global symbol as seen from within the same file.
/// Infers the public type of an explicit module-global symbol as seen from within the same file.
///
/// If it's not defined explicitly in the global scope, it will look it up in `types.ModuleType`
/// with a few very special exceptions.
/// Note that all global scopes also include various "implicit globals" such as `__name__`,
/// `__doc__` and `__file__`. This function **does not** consider those symbols; it will return
/// `Symbol::Unbound` for them. Use the (currently test-only) `global_symbol` query to also include
/// those additional symbols.
///
/// Use [`imported_symbol`] to perform the lookup as seen from outside the file (e.g. via imports).
pub(crate) fn global_symbol<'db>(db: &'db dyn Db, file: File, name: &str) -> Symbol<'db> {
pub(crate) fn explicit_global_symbol<'db>(db: &'db dyn Db, file: File, name: &str) -> Symbol<'db> {
symbol_impl(
db,
global_scope(db, file),
name,
RequiresExplicitReExport::No,
)
.or_fall_back_to(db, || module_type_symbol(db, name))
}
/// Infers the public type of an explicit module-global symbol as seen from within the same file.
///
/// Unlike [`explicit_global_symbol`], this function also considers various "implicit globals"
/// such as `__name__`, `__doc__` and `__file__`. These are looked up as attributes on `types.ModuleType`
/// rather than being looked up as symbols explicitly defined/declared in the global scope.
///
/// Use [`imported_symbol`] to perform the lookup as seen from outside the file (e.g. via imports).
#[cfg(test)]
pub(crate) fn global_symbol<'db>(db: &'db dyn Db, file: File, name: &str) -> Symbol<'db> {
explicit_global_symbol(db, file, name)
.or_fall_back_to(db, || module_type_implicit_global_symbol(db, name))
}
/// Infers the public type of an imported symbol.
@@ -204,16 +219,16 @@ pub(crate) fn imported_symbol<'db>(db: &'db dyn Db, module: &Module, name: &str)
// If it's not found in the global scope, check if it's present as an instance on
// `types.ModuleType` or `builtins.object`.
//
// We do a more limited version of this in `global_symbol`, but there are two crucial
// differences here:
// We do a more limited version of this in `module_type_implicit_global_symbol`,
// but there are two crucial differences here:
// - If a member is looked up as an attribute, `__init__` is also available on the module, but
// it isn't available as a global from inside the module
// - If a member is looked up as an attribute, members on `builtins.object` are also available
// (because `types.ModuleType` inherits from `object`); these attributes are also not
// available as globals from inside the module.
//
// The same way as in `global_symbol`, however, we need to be careful to ignore
// `__getattr__`. Typeshed has a fake `__getattr__` on `types.ModuleType` to help out with
// The same way as in `module_type_implicit_global_symbol`, however, we need to be careful to
// ignore `__getattr__`. Typeshed has a fake `__getattr__` on `types.ModuleType` to help out with
// dynamic imports; we shouldn't use it for `ModuleLiteral` types where we know exactly which
// module we're dealing with.
external_symbol_impl(db, module.file(), name).or_fall_back_to(db, || {
@@ -239,7 +254,7 @@ pub(crate) fn builtins_symbol<'db>(db: &'db dyn Db, symbol: &str) -> Symbol<'db>
// We're looking up in the builtins namespace and not the module, so we should
// do the normal lookup in `types.ModuleType` and not the special one as in
// `imported_symbol`.
module_type_symbol(db, symbol)
module_type_implicit_global_symbol(db, symbol)
})
})
.unwrap_or(Symbol::Unbound)
@@ -488,6 +503,7 @@ fn symbol_from_bindings_impl<'db>(
bindings_with_constraints: BindingWithConstraintsIterator<'_, 'db>,
requires_explicit_reexport: RequiresExplicitReExport,
) -> Symbol<'db> {
let constraints = bindings_with_constraints.constraints;
let visibility_constraints = bindings_with_constraints.visibility_constraints;
let mut bindings_with_constraints = bindings_with_constraints.peekable();
@@ -499,9 +515,9 @@ fn symbol_from_bindings_impl<'db>(
Some(BindingWithConstraints {
binding,
visibility_constraint,
constraints: _,
narrowing_constraints: _,
}) if binding.map_or(true, is_non_exported) => {
visibility_constraints.evaluate(db, *visibility_constraint)
visibility_constraints.evaluate(db, constraints, *visibility_constraint)
}
_ => Truthiness::AlwaysFalse,
};
@@ -509,7 +525,7 @@ fn symbol_from_bindings_impl<'db>(
let mut types = bindings_with_constraints.filter_map(
|BindingWithConstraints {
binding,
constraints,
narrowing_constraints,
visibility_constraint,
}| {
let binding = binding?;
@@ -518,13 +534,14 @@ fn symbol_from_bindings_impl<'db>(
return None;
}
let static_visibility = visibility_constraints.evaluate(db, visibility_constraint);
let static_visibility =
visibility_constraints.evaluate(db, constraints, visibility_constraint);
if static_visibility.is_always_false() {
return None;
}
let mut constraint_tys = constraints
let mut constraint_tys = narrowing_constraints
.filter_map(|constraint| narrowing_constraint(db, constraint, binding))
.peekable();
@@ -575,6 +592,7 @@ fn symbol_from_declarations_impl<'db>(
declarations: DeclarationsIterator<'_, 'db>,
requires_explicit_reexport: RequiresExplicitReExport,
) -> SymbolFromDeclarationsResult<'db> {
let constraints = declarations.constraints;
let visibility_constraints = declarations.visibility_constraints;
let mut declarations = declarations.peekable();
@@ -587,7 +605,7 @@ fn symbol_from_declarations_impl<'db>(
declaration,
visibility_constraint,
}) if declaration.map_or(true, is_non_exported) => {
visibility_constraints.evaluate(db, *visibility_constraint)
visibility_constraints.evaluate(db, constraints, *visibility_constraint)
}
_ => Truthiness::AlwaysFalse,
};
@@ -603,7 +621,8 @@ fn symbol_from_declarations_impl<'db>(
return None;
}
let static_visibility = visibility_constraints.evaluate(db, visibility_constraint);
let static_visibility =
visibility_constraints.evaluate(db, constraints, visibility_constraint);
if static_visibility.is_always_false() {
None
@@ -658,63 +677,106 @@ fn symbol_from_declarations_impl<'db>(
}
}
/// Return a list of the symbols that typeshed declares in the body scope of
/// the stub for the class `types.ModuleType`.
///
/// Conceptually this could be a `Set` rather than a list,
/// but the number of symbols declared in this scope is likely to be very small,
/// so the cost of hashing the names is likely to be more expensive than it's worth.
#[salsa::tracked(return_ref)]
fn module_type_symbols<'db>(db: &'db dyn Db) -> smallvec::SmallVec<[ast::name::Name; 8]> {
let Some(module_type) = KnownClass::ModuleType
.to_class_literal(db)
.into_class_literal()
else {
// The most likely way we get here is if a user specified a `--custom-typeshed-dir`
// without a `types.pyi` stub in the `stdlib/` directory
return smallvec::SmallVec::default();
};
mod implicit_globals {
use ruff_python_ast as ast;
let module_type_scope = module_type.body_scope(db);
let module_type_symbol_table = symbol_table(db, module_type_scope);
use crate::db::Db;
use crate::semantic_index::{self, symbol_table};
use crate::types::KnownClass;
// `__dict__` and `__init__` are very special members that can be accessed as attributes
// on the module when imported, but cannot be accessed as globals *inside* the module.
//
// `__getattr__` is even more special: it doesn't exist at runtime, but typeshed includes it
// to reduce false positives associated with functions that dynamically import modules
// and return `Instance(types.ModuleType)`. We should ignore it for any known module-literal type.
module_type_symbol_table
.symbols()
.filter(|symbol| symbol.is_declared())
.map(semantic_index::symbol::Symbol::name)
.filter(|symbol_name| !matches!(&***symbol_name, "__dict__" | "__getattr__" | "__init__"))
.cloned()
.collect()
}
use super::Symbol;
/// Return the symbol for a member of `types.ModuleType`.
///
/// ## Notes
///
/// In general we wouldn't check to see whether a symbol exists on a class before doing the
/// [`member`] call on the instance type -- we'd just do the [`member`] call on the instance
/// type, since it has the same end result. The reason to only call [`member`] on [`ModuleType`]
/// instance when absolutely necessary is that it was a fairly significant performance regression
/// to fallback to doing that for every name lookup that wasn't found in the module's globals
/// ([`global_symbol`]). So we use less idiomatic (and much more verbose) code here as a
/// micro-optimisation because it's used in a very hot path.
///
/// [`member`]: Type::member
/// [`ModuleType`]: KnownClass::ModuleType
fn module_type_symbol<'db>(db: &'db dyn Db, name: &str) -> Symbol<'db> {
if module_type_symbols(db)
.iter()
.any(|module_type_member| &**module_type_member == name)
{
KnownClass::ModuleType.to_instance(db).member(db, name)
} else {
Symbol::Unbound
/// Looks up the type of an "implicit global symbol". Returns [`Symbol::Unbound`] if
/// `name` is not present as an implicit symbol in module-global namespaces.
///
/// Implicit global symbols are symbols such as `__doc__`, `__name__`, and `__file__`
/// that are implicitly defined in every module's global scope. Because their type is
/// always the same, we simply look these up as instance attributes on `types.ModuleType`.
///
/// Note that this function should only be used as a fallback if a symbol is being looked
/// up in the global scope **from within the same file**. If the symbol is being looked up
/// from outside the file (e.g. via imports), use [`super::imported_symbol`] (or fallback logic
/// like the logic used in that function) instead. The reason is that this function returns
/// [`Symbol::Unbound`] for `__init__` and `__dict__` (which cannot be found in globals if
/// the lookup is being done from the same file) -- but these symbols *are* available in the
/// global scope if they're being imported **from a different file**.
pub(crate) fn module_type_implicit_global_symbol<'db>(
db: &'db dyn Db,
name: &str,
) -> Symbol<'db> {
// In general we wouldn't check to see whether a symbol exists on a class before doing the
// `.member()` call on the instance type -- we'd just do the `.member`() call on the instance
// type, since it has the same end result. The reason to only call `.member()` on `ModuleType`
// when absolutely necessary is that this function is used in a very hot path (name resolution
// in `infer.rs`). We use less idiomatic (and much more verbose) code here as a micro-optimisation.
if module_type_symbols(db)
.iter()
.any(|module_type_member| &**module_type_member == name)
{
KnownClass::ModuleType.to_instance(db).member(db, name)
} else {
Symbol::Unbound
}
}
/// An internal micro-optimisation for `module_type_implicit_global_symbol`.
///
/// This function returns a list of the symbols that typeshed declares in the
/// body scope of the stub for the class `types.ModuleType`.
///
/// The returned list excludes the attributes `__dict__` and `__init__`. These are very
/// special members that can be accessed as attributes on the module when imported,
/// but cannot be accessed as globals *inside* the module.
///
/// The list also excludes `__getattr__`. `__getattr__` is even more special: it doesn't
/// exist at runtime, but typeshed includes it to reduce false positives associated with
/// functions that dynamically import modules and return `Instance(types.ModuleType)`.
/// We should ignore it for any known module-literal type.
///
/// Conceptually this function could be a `Set` rather than a list,
/// but the number of symbols declared in this scope is likely to be very small,
/// so the cost of hashing the names is likely to be more expensive than it's worth.
#[salsa::tracked(return_ref)]
fn module_type_symbols<'db>(db: &'db dyn Db) -> smallvec::SmallVec<[ast::name::Name; 8]> {
let Some(module_type) = KnownClass::ModuleType
.to_class_literal(db)
.into_class_literal()
else {
// The most likely way we get here is if a user specified a `--custom-typeshed-dir`
// without a `types.pyi` stub in the `stdlib/` directory
return smallvec::SmallVec::default();
};
let module_type_scope = module_type.body_scope(db);
let module_type_symbol_table = symbol_table(db, module_type_scope);
module_type_symbol_table
.symbols()
.filter(|symbol| symbol.is_declared())
.map(semantic_index::symbol::Symbol::name)
.filter(|symbol_name| {
!matches!(&***symbol_name, "__dict__" | "__getattr__" | "__init__")
})
.cloned()
.collect()
}
#[cfg(test)]
mod tests {
use super::*;
use crate::db::tests::setup_db;
#[test]
fn module_type_symbols_includes_declared_types_but_not_referenced_types() {
let db = setup_db();
let symbol_names = module_type_symbols(&db);
let dunder_name_symbol_name = ast::name::Name::new_static("__name__");
assert!(symbol_names.contains(&dunder_name_symbol_name));
let property_symbol_name = ast::name::Name::new_static("property");
assert!(!symbol_names.contains(&property_symbol_name));
}
}
}
@@ -828,15 +890,36 @@ mod tests {
);
}
#[track_caller]
fn assert_bound_string_symbol<'db>(db: &'db dyn Db, symbol: Symbol<'db>) {
assert!(matches!(
symbol,
Symbol::Type(Type::Instance(_), Boundness::Bound)
));
assert_eq!(symbol.expect_type(), KnownClass::Str.to_instance(db));
}
#[test]
fn module_type_symbols_includes_declared_types_but_not_referenced_types() {
fn implicit_builtin_globals() {
let db = setup_db();
let symbol_names = module_type_symbols(&db);
assert_bound_string_symbol(&db, builtins_symbol(&db, "__name__"));
}
let dunder_name_symbol_name = ast::name::Name::new_static("__name__");
assert!(symbol_names.contains(&dunder_name_symbol_name));
#[test]
fn implicit_typing_globals() {
let db = setup_db();
assert_bound_string_symbol(&db, typing_symbol(&db, "__name__"));
}
let property_symbol_name = ast::name::Name::new_static("property");
assert!(!symbol_names.contains(&property_symbol_name));
#[test]
fn implicit_typing_extensions_globals() {
let db = setup_db();
assert_bound_string_symbol(&db, typing_extensions_symbol(&db, "__name__"));
}
#[test]
fn implicit_sys_globals() {
let db = setup_db();
assert_bound_string_symbol(&db, known_module_symbol(&db, KnownModule::Sys, "__name__"));
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -57,11 +57,11 @@ impl<'db> CallOutcome<'db> {
not_callable_ty: Type::Union(union),
})
} else {
Err(CallError::Union {
Err(CallError::Union(UnionCallError {
errors: errors.into(),
bindings: bindings.into(),
called_ty: Type::Union(union),
})
}))
}
}
@@ -96,16 +96,7 @@ pub(super) enum CallError<'db> {
/// can't be called with the given arguments.
///
/// A union where all variants are not callable is represented as a `NotCallable` error.
Union {
/// The variants that can't be called with the given arguments.
errors: Box<[CallError<'db>]>,
/// The bindings for the callable variants (that have no binding errors).
bindings: Box<[CallBinding<'db>]>,
/// The union type that we tried calling.
called_ty: Type<'db>,
},
Union(UnionCallError<'db>),
/// The type has a `__call__` method but it isn't always bound.
PossiblyUnboundDunderCall {
@@ -126,9 +117,9 @@ impl<'db> CallError<'db> {
CallError::NotCallable { .. } => None,
// If some variants are callable, and some are not, return the union of the return types of the callable variants
// combined with `Type::Unknown`
CallError::Union {
errors, bindings, ..
} => Some(UnionType::from_elements(
CallError::Union(UnionCallError {
bindings, errors, ..
}) => Some(UnionType::from_elements(
db,
bindings
.iter()
@@ -158,7 +149,7 @@ impl<'db> CallError<'db> {
Self::NotCallable {
not_callable_ty, ..
} => *not_callable_ty,
Self::Union { called_ty, .. } => *called_ty,
Self::Union(UnionCallError { called_ty, .. }) => *called_ty,
Self::PossiblyUnboundDunderCall { called_type, .. } => *called_type,
Self::BindingError { binding } => binding.callable_type(),
}
@@ -169,6 +160,18 @@ impl<'db> CallError<'db> {
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub(super) struct UnionCallError<'db> {
/// The variants that can't be called with the given arguments.
pub(super) errors: Box<[CallError<'db>]>,
/// The bindings for the callable variants (that have no binding errors).
pub(super) bindings: Box<[CallBinding<'db>]>,
/// The union type that we tried calling.
pub(super) called_ty: Type<'db>,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub(super) enum CallDunderError<'db> {
/// The dunder attribute exists but it can't be called with the given arguments.

View File

@@ -5,6 +5,11 @@ use super::Type;
pub(crate) struct CallArguments<'a, 'db>(Vec<Argument<'a, 'db>>);
impl<'a, 'db> CallArguments<'a, 'db> {
/// Create a [`CallArguments`] with no arguments.
pub(crate) fn none() -> Self {
Self(Vec::new())
}
/// Create a [`CallArguments`] from an iterator over non-variadic positional argument types.
pub(crate) fn positional(positional_tys: impl IntoIterator<Item = Type<'db>>) -> Self {
positional_tys
@@ -29,6 +34,19 @@ impl<'a, 'db> CallArguments<'a, 'db> {
pub(crate) fn first_argument(&self) -> Option<Type<'db>> {
self.0.first().map(Argument::ty)
}
// TODO this should be eliminated in favor of [`bind_call`]
pub(crate) fn exactly_one_argument(&self) -> Option<Type<'db>> {
match &*self.0 {
[arg] => Some(arg.ty()),
_ => None,
}
}
// TODO this should be eliminated in favor of [`bind_call`]
pub(crate) fn second_argument(&self) -> Option<Type<'db>> {
self.0.get(1).map(Argument::ty)
}
}
impl<'db, 'a, 'b> IntoIterator for &'b CallArguments<'a, 'db> {

View File

@@ -193,6 +193,13 @@ impl<'db> CallBinding<'db> {
}
}
pub(crate) fn three_parameter_types(&self) -> Option<(Type<'db>, Type<'db>, Type<'db>)> {
match self.parameter_types() {
[first, second, third] => Some((*first, *second, *third)),
_ => None,
}
}
fn callable_name(&self, db: &'db dyn Db) -> Option<&str> {
match self.callable_ty {
Type::FunctionLiteral(function) => Some(function.name(db)),

View File

@@ -72,6 +72,7 @@ impl<'db> ClassBase<'db> {
Type::Never
| Type::BooleanLiteral(_)
| Type::FunctionLiteral(_)
| Type::Callable(..)
| Type::BytesLiteral(_)
| Type::IntLiteral(_)
| Type::StringLiteral(_)

View File

@@ -44,6 +44,7 @@ pub(crate) fn register_lints(registry: &mut LintRegistryBuilder) {
registry.register_lint(&MISSING_ARGUMENT);
registry.register_lint(&NON_SUBSCRIPTABLE);
registry.register_lint(&NOT_ITERABLE);
registry.register_lint(&UNSUPPORTED_BOOL_CONVERSION);
registry.register_lint(&PARAMETER_ALREADY_ASSIGNED);
registry.register_lint(&POSSIBLY_UNBOUND_ATTRIBUTE);
registry.register_lint(&POSSIBLY_UNBOUND_IMPORT);
@@ -490,6 +491,37 @@ declare_lint! {
}
}
declare_lint! {
/// ## What it does
/// Checks for bool conversions where the object doesn't correctly implement `__bool__`.
///
/// ## Why is this bad?
/// If an exception is raised when you attempt to evaluate the truthiness of an object,
/// using the object in a boolean context will fail at runtime.
///
/// ## Examples
///
/// ```python
/// class NotBoolable:
/// __bool__ = None
///
/// b1 = NotBoolable()
/// b2 = NotBoolable()
///
/// if b1: # exception raised here
/// pass
///
/// b1 and b2 # exception raised here
/// not b1 # exception raised here
/// b1 < b2 < b1 # exception raised here
/// ```
pub(crate) static UNSUPPORTED_BOOL_CONVERSION = {
summary: "detects boolean conversion where the object incorrectly implements `__bool__`",
status: LintStatus::preview("1.0.0"),
default_level: Level::Error,
}
}
declare_lint! {
/// ## What it does
/// Checks for calls which provide more than one argument for a single parameter.

View File

@@ -8,8 +8,8 @@ use ruff_python_literal::escape::AsciiEscape;
use crate::types::class_base::ClassBase;
use crate::types::{
ClassLiteralType, InstanceType, IntersectionType, KnownClass, StringLiteralType, Type,
UnionType,
CallableType, ClassLiteralType, InstanceType, IntersectionType, KnownClass, StringLiteralType,
Type, UnionType,
};
use crate::Db;
use rustc_hash::FxHashMap;
@@ -88,6 +88,24 @@ impl Display for DisplayRepresentation<'_> {
},
Type::KnownInstance(known_instance) => f.write_str(known_instance.repr(self.db)),
Type::FunctionLiteral(function) => f.write_str(function.name(self.db)),
Type::Callable(CallableType::BoundMethod(bound_method)) => {
write!(
f,
"<bound method `{method}` of `{instance}`>",
method = bound_method.function(self.db).name(self.db),
instance = bound_method.self_instance(self.db).display(self.db)
)
}
Type::Callable(CallableType::MethodWrapperDunderGet(function)) => {
write!(
f,
"<method-wrapper `__get__` of `{function}`>",
function = function.name(self.db)
)
}
Type::Callable(CallableType::WrapperDescriptorDunderGet) => {
f.write_str("<wrapper-descriptor `__get__` of `function` objects>")
}
Type::Union(union) => union.display(self.db).fmt(f),
Type::Intersection(intersection) => intersection.display(self.db).fmt(f),
Type::IntLiteral(n) => n.fmt(f),

View File

@@ -33,7 +33,7 @@ use ruff_db::diagnostic::{DiagnosticId, Severity};
use ruff_db::files::File;
use ruff_db::parsed::parsed_module;
use ruff_python_ast::{self as ast, AnyNodeRef, ExprContext};
use ruff_text_size::Ranged;
use ruff_text_size::{Ranged, TextRange};
use rustc_hash::{FxHashMap, FxHashSet};
use salsa;
use salsa::plumbing::AsId;
@@ -50,10 +50,11 @@ use crate::semantic_index::semantic_index;
use crate::semantic_index::symbol::{FileScopeId, NodeWithScopeKind, NodeWithScopeRef, ScopeId};
use crate::semantic_index::SemanticIndex;
use crate::symbol::{
builtins_module_scope, builtins_symbol, symbol, symbol_from_bindings, symbol_from_declarations,
builtins_module_scope, builtins_symbol, explicit_global_symbol,
module_type_implicit_global_symbol, symbol, symbol_from_bindings, symbol_from_declarations,
typing_extensions_symbol, LookupError,
};
use crate::types::call::{Argument, CallArguments};
use crate::types::call::{Argument, CallArguments, UnionCallError};
use crate::types::diagnostic::{
report_invalid_arguments_to_annotated, report_invalid_assignment,
report_invalid_attribute_assignment, report_unresolved_module, TypeCheckDiagnostics,
@@ -68,9 +69,9 @@ use crate::types::mro::MroErrorKind;
use crate::types::unpacker::{UnpackResult, Unpacker};
use crate::types::{
todo_type, Boundness, Class, ClassLiteralType, DynamicType, FunctionType, InstanceType,
IntersectionBuilder, IntersectionType, IterationOutcome, KnownClass, KnownFunction,
KnownInstanceType, MetaclassCandidate, MetaclassErrorKind, SliceLiteralType, SubclassOfType,
Symbol, SymbolAndQualifiers, Truthiness, TupleType, Type, TypeAliasType, TypeAndQualifiers,
IntersectionBuilder, IntersectionType, KnownClass, KnownFunction, KnownInstanceType,
MetaclassCandidate, MetaclassErrorKind, SliceLiteralType, SubclassOfType, Symbol,
SymbolAndQualifiers, Truthiness, TupleType, Type, TypeAliasType, TypeAndQualifiers,
TypeArrayDisplay, TypeQualifiers, TypeVarBoundOrConstraints, TypeVarInstance, UnionBuilder,
UnionType,
};
@@ -90,7 +91,7 @@ use super::slots::check_class_slots;
use super::string_annotation::{
parse_string_annotation, BYTE_STRING_TYPE_ANNOTATION, FSTRING_TYPE_ANNOTATION,
};
use super::{global_symbol, CallDunderError, ParameterExpectation, ParameterExpectations};
use super::{CallDunderError, ParameterExpectation, ParameterExpectations};
/// Infer all types for a [`ScopeId`], including all definitions and expressions in that scope.
/// Use when checking a scope, or needing to provide a type for an arbitrary expression in the
@@ -1480,7 +1481,12 @@ impl<'db> TypeInferenceBuilder<'db> {
elif_else_clauses,
} = if_statement;
self.infer_standalone_expression(test);
let test_ty = self.infer_standalone_expression(test);
if let Err(err) = test_ty.try_bool(self.db()) {
err.report_diagnostic(&self.context, &**test);
}
self.infer_body(body);
for clause in elif_else_clauses {
@@ -1491,7 +1497,11 @@ impl<'db> TypeInferenceBuilder<'db> {
} = clause;
if let Some(test) = &test {
self.infer_standalone_expression(test);
let test_ty = self.infer_standalone_expression(test);
if let Err(err) = test_ty.try_bool(self.db()) {
err.report_diagnostic(&self.context, test);
}
}
self.infer_body(body);
@@ -1887,9 +1897,15 @@ impl<'db> TypeInferenceBuilder<'db> {
guard,
} = case;
self.infer_match_pattern(pattern);
guard
.as_deref()
.map(|guard| self.infer_standalone_expression(guard));
if let Some(guard) = guard.as_deref() {
let guard_ty = self.infer_standalone_expression(guard);
if let Err(err) = guard_ty.try_bool(self.db()) {
err.report_diagnostic(&self.context, guard);
}
}
self.infer_body(body);
}
}
@@ -2358,7 +2374,10 @@ impl<'db> TypeInferenceBuilder<'db> {
} = for_statement;
self.infer_target(target, iter, |db, iter_ty| {
iter_ty.iterate(db).unwrap_without_diagnostic()
// TODO: `infer_for_statement_definition` reports a diagnostic if `iter_ty` isn't iterable
// but only if the target is a name. We should report a diagnostic here if the target isn't a name:
// `for a.x in not_iterable: ...
iter_ty.iterate(db)
});
self.infer_body(body);
@@ -2387,9 +2406,10 @@ impl<'db> TypeInferenceBuilder<'db> {
let name_ast_id = name.scoped_expression_id(self.db(), self.scope());
unpacked.expression_type(name_ast_id)
}
TargetKind::Name => iterable_ty
.iterate(self.db())
.unwrap_with_diagnostic(&self.context, iterable.into()),
TargetKind::Name => iterable_ty.try_iterate(self.db()).unwrap_or_else(|err| {
err.report_diagnostic(&self.context, iterable.into());
err.fallback_element_type()
}),
}
};
@@ -2405,7 +2425,12 @@ impl<'db> TypeInferenceBuilder<'db> {
orelse,
} = while_statement;
self.infer_standalone_expression(test);
let test_ty = self.infer_standalone_expression(test);
if let Err(err) = test_ty.try_bool(self.db()) {
err.report_diagnostic(&self.context, &**test);
}
self.infer_body(body);
self.infer_body(orelse);
}
@@ -2487,7 +2512,12 @@ impl<'db> TypeInferenceBuilder<'db> {
msg,
} = assert;
self.infer_expression(test);
let test_ty = self.infer_expression(test);
if let Err(err) = test_ty.try_bool(self.db()) {
err.report_diagnostic(&self.context, &**test);
}
self.infer_optional_expression(msg.as_deref());
}
@@ -3171,9 +3201,10 @@ impl<'db> TypeInferenceBuilder<'db> {
// TODO: async iterables/iterators! -- Alex
todo_type!("async iterables/iterators")
} else {
iterable_ty
.iterate(self.db())
.unwrap_with_diagnostic(&self.context, iterable.into())
iterable_ty.try_iterate(self.db()).unwrap_or_else(|err| {
err.report_diagnostic(&self.context, iterable.into());
err.fallback_element_type()
})
};
self.types.expressions.insert(
@@ -3229,7 +3260,10 @@ impl<'db> TypeInferenceBuilder<'db> {
let body_ty = self.infer_expression(body);
let orelse_ty = self.infer_expression(orelse);
match test_ty.bool(self.db()) {
match test_ty.try_bool(self.db()).unwrap_or_else(|err| {
err.report_diagnostic(&self.context, &**test);
err.fallback_truthiness()
}) {
Truthiness::AlwaysTrue => body_ty,
Truthiness::AlwaysFalse => orelse_ty,
Truthiness::Ambiguous => UnionType::from_elements(self.db(), [body_ty, orelse_ty]),
@@ -3322,7 +3356,26 @@ impl<'db> TypeInferenceBuilder<'db> {
}
KnownFunction::StaticAssert => {
if let Some((parameter_ty, message)) = binding.two_parameter_types() {
let truthiness = parameter_ty.bool(self.db());
let truthiness = match parameter_ty.try_bool(self.db()) {
Ok(truthiness) => truthiness,
Err(err) => {
let condition = arguments
.find_argument("condition", 0)
.map(|argument| match argument {
ruff_python_ast::ArgOrKeyword::Arg(expr) => {
ast::AnyNodeRef::from(expr)
}
ruff_python_ast::ArgOrKeyword::Keyword(keyword) => {
ast::AnyNodeRef::from(keyword)
}
})
.unwrap_or(ast::AnyNodeRef::from(call_expression));
err.report_diagnostic(&self.context, condition);
continue;
}
};
if !truthiness.is_always_true() {
if let Some(message) =
@@ -3388,14 +3441,8 @@ impl<'db> TypeInferenceBuilder<'db> {
);
}
CallError::Union {
called_ty: _,
bindings: _,
errors,
} => {
// TODO: Remove the `Vec::from` call once we use the Rust 2024 edition
// which adds `Box<[T]>::into_iter`
if let Some(first) = Vec::from(errors).into_iter().next() {
CallError::Union(UnionCallError { errors, .. }) => {
if let Some(first) = IntoIterator::into_iter(errors).next() {
report_call_error(context, first, call_expression);
} else {
debug_assert!(
@@ -3437,9 +3484,10 @@ impl<'db> TypeInferenceBuilder<'db> {
} = starred;
let iterable_ty = self.infer_expression(value);
iterable_ty
.iterate(self.db())
.unwrap_with_diagnostic(&self.context, value.as_ref().into());
iterable_ty.try_iterate(self.db()).unwrap_or_else(|err| {
err.report_diagnostic(&self.context, value.as_ref().into());
err.fallback_element_type()
});
// TODO
todo_type!("starred expression")
@@ -3455,9 +3503,10 @@ impl<'db> TypeInferenceBuilder<'db> {
let ast::ExprYieldFrom { range: _, value } = yield_from;
let iterable_ty = self.infer_expression(value);
iterable_ty
.iterate(self.db())
.unwrap_with_diagnostic(&self.context, value.as_ref().into());
iterable_ty.try_iterate(self.db()).unwrap_or_else(|err| {
err.report_diagnostic(&self.context, value.as_ref().into());
err.fallback_element_type()
});
// TODO get type from `ReturnType` of generator
todo_type!("Generic `typing.Generator` type")
@@ -3571,7 +3620,7 @@ impl<'db> TypeInferenceBuilder<'db> {
}
Symbol::Unbound
// No nonlocal binding? Check the module's globals.
// No nonlocal binding? Check the module's explicit globals.
// Avoid infinite recursion if `self.scope` already is the module's global scope.
.or_fall_back_to(db, || {
if file_scope_id.is_global() {
@@ -3588,8 +3637,12 @@ impl<'db> TypeInferenceBuilder<'db> {
}
}
global_symbol(db, self.file(), symbol_name)
explicit_global_symbol(db, self.file(), symbol_name)
})
// Not found in the module's explicitly declared global symbols?
// Check the "implicit globals" such as `__doc__`, `__file__`, `__name__`, etc.
// These are looked up as attributes on `types.ModuleType`.
.or_fall_back_to(db, || module_type_implicit_global_symbol(db, symbol_name))
// Not found in globals? Fallback to builtins
// (without infinite recursion if we're already in builtins.)
.or_fall_back_to(db, || {
@@ -3749,10 +3802,18 @@ impl<'db> TypeInferenceBuilder<'db> {
Type::IntLiteral(!i64::from(bool))
}
(ast::UnaryOp::Not, ty) => ty.bool(self.db()).negate().into_type(self.db()),
(ast::UnaryOp::Not, ty) => ty
.try_bool(self.db())
.unwrap_or_else(|err| {
err.report_diagnostic(&self.context, unary);
err.fallback_truthiness()
})
.negate()
.into_type(self.db()),
(
op @ (ast::UnaryOp::UAdd | ast::UnaryOp::USub | ast::UnaryOp::Invert),
Type::FunctionLiteral(_)
| Type::Callable(..)
| Type::ModuleLiteral(_)
| Type::ClassLiteral(_)
| Type::SubclassOf(_)
@@ -3780,7 +3841,7 @@ impl<'db> TypeInferenceBuilder<'db> {
match operand_type.try_call_dunder(
self.db(),
unary_dunder_method,
&CallArguments::positional([operand_type]),
&CallArguments::none(),
) {
Ok(outcome) => outcome.return_type(self.db()),
Err(e) => {
@@ -3967,6 +4028,7 @@ impl<'db> TypeInferenceBuilder<'db> {
// fall back on looking for dunder methods on one of the operand types.
(
Type::FunctionLiteral(_)
| Type::Callable(..)
| Type::ModuleLiteral(_)
| Type::ClassLiteral(_)
| Type::SubclassOf(_)
@@ -3983,6 +4045,7 @@ impl<'db> TypeInferenceBuilder<'db> {
| Type::SliceLiteral(_)
| Type::Tuple(_),
Type::FunctionLiteral(_)
| Type::Callable(..)
| Type::ModuleLiteral(_)
| Type::ClassLiteral(_)
| Type::SubclassOf(_)
@@ -4029,7 +4092,7 @@ impl<'db> TypeInferenceBuilder<'db> {
.try_call_dunder(
self.db(),
reflected_dunder,
&CallArguments::positional([right_ty, left_ty]),
&CallArguments::positional([left_ty]),
)
.map(|outcome| outcome.return_type(self.db()))
.or_else(|_| {
@@ -4037,7 +4100,7 @@ impl<'db> TypeInferenceBuilder<'db> {
.try_call_dunder(
self.db(),
op.dunder(),
&CallArguments::positional([left_ty, right_ty]),
&CallArguments::positional([right_ty]),
)
.map(|outcome| outcome.return_type(self.db()))
})
@@ -4091,11 +4154,13 @@ impl<'db> TypeInferenceBuilder<'db> {
*op,
values.iter().enumerate(),
|builder, (index, value)| {
if index == values.len() - 1 {
let ty = if index == values.len() - 1 {
builder.infer_expression(value)
} else {
builder.infer_standalone_expression(value)
}
};
(ty, value.range())
},
)
}
@@ -4112,7 +4177,7 @@ impl<'db> TypeInferenceBuilder<'db> {
) -> Type<'db>
where
Iterator: IntoIterator<Item = Item>,
F: Fn(&mut Self, Item) -> Type<'db>,
F: Fn(&mut Self, Item) -> (Type<'db>, TextRange),
{
let mut done = false;
let db = self.db();
@@ -4120,37 +4185,48 @@ impl<'db> TypeInferenceBuilder<'db> {
let elements = operations
.into_iter()
.with_position()
.map(|(position, ty)| {
let ty = infer_ty(self, ty);
if done {
return Type::Never;
}
.map(|(position, item)| {
let (ty, range) = infer_ty(self, item);
let is_last = matches!(
position,
itertools::Position::Last | itertools::Position::Only
);
match (ty.bool(db), is_last, op) {
(Truthiness::AlwaysTrue, false, ast::BoolOp::And) => Type::Never,
(Truthiness::AlwaysFalse, false, ast::BoolOp::Or) => Type::Never,
(Truthiness::AlwaysFalse, _, ast::BoolOp::And)
| (Truthiness::AlwaysTrue, _, ast::BoolOp::Or) => {
done = true;
if is_last {
if done {
Type::Never
} else {
ty
}
} else {
let truthiness = ty.try_bool(self.db()).unwrap_or_else(|err| {
err.report_diagnostic(&self.context, range);
err.fallback_truthiness()
});
(Truthiness::Ambiguous, false, _) => IntersectionBuilder::new(db)
.add_positive(ty)
.add_negative(match op {
ast::BoolOp::And => Type::AlwaysTruthy,
ast::BoolOp::Or => Type::AlwaysFalsy,
})
.build(),
if done {
return Type::Never;
};
(_, true, _) => ty,
match (truthiness, op) {
(Truthiness::AlwaysTrue, ast::BoolOp::And) => Type::Never,
(Truthiness::AlwaysFalse, ast::BoolOp::Or) => Type::Never,
(Truthiness::AlwaysFalse, ast::BoolOp::And)
| (Truthiness::AlwaysTrue, ast::BoolOp::Or) => {
done = true;
ty
}
(Truthiness::Ambiguous, _) => IntersectionBuilder::new(db)
.add_positive(ty)
.add_negative(match op {
ast::BoolOp::And => Type::AlwaysTruthy,
ast::BoolOp::Or => Type::AlwaysFalsy,
})
.build(),
}
}
});
@@ -4166,9 +4242,6 @@ impl<'db> TypeInferenceBuilder<'db> {
} = compare;
self.infer_expression(left);
for right in comparators {
self.infer_expression(right);
}
// https://docs.python.org/3/reference/expressions.html#comparisons
// > Formally, if `a, b, c, …, y, z` are expressions and `op1, op2, …, opN` are comparison
@@ -4185,15 +4258,17 @@ impl<'db> TypeInferenceBuilder<'db> {
.zip(ops),
|builder, ((left, right), op)| {
let left_ty = builder.expression_type(left);
let right_ty = builder.expression_type(right);
let right_ty = builder.infer_expression(right);
builder
.infer_binary_type_comparison(left_ty, *op, right_ty)
let range = TextRange::new(left.start(), right.end());
let ty = builder
.infer_binary_type_comparison(left_ty, *op, right_ty, range)
.unwrap_or_else(|error| {
// Handle unsupported operators (diagnostic, `bool`/`Unknown` outcome)
builder.context.report_lint(
&UNSUPPORTED_OPERATOR,
AnyNodeRef::ExprCompare(compare),
range,
format_args!(
"Operator `{}` is not supported for types `{}` and `{}`{}",
error.op,
@@ -4220,7 +4295,9 @@ impl<'db> TypeInferenceBuilder<'db> {
// Other operators can return arbitrary types
_ => Type::unknown(),
}
})
});
(ty, range)
},
)
}
@@ -4231,14 +4308,19 @@ impl<'db> TypeInferenceBuilder<'db> {
op: ast::CmpOp,
other: Type<'db>,
intersection_on: IntersectionOn,
range: TextRange,
) -> Result<Type<'db>, CompareUnsupportedError<'db>> {
// If a comparison yields a definitive true/false answer on a (positive) part
// of an intersection type, it will also yield a definitive answer on the full
// intersection type, which is even more specific.
for pos in intersection.positive(self.db()) {
let result = match intersection_on {
IntersectionOn::Left => self.infer_binary_type_comparison(*pos, op, other)?,
IntersectionOn::Right => self.infer_binary_type_comparison(other, op, *pos)?,
IntersectionOn::Left => {
self.infer_binary_type_comparison(*pos, op, other, range)?
}
IntersectionOn::Right => {
self.infer_binary_type_comparison(other, op, *pos, range)?
}
};
if let Type::BooleanLiteral(b) = result {
return Ok(Type::BooleanLiteral(b));
@@ -4249,8 +4331,12 @@ impl<'db> TypeInferenceBuilder<'db> {
// special cases that allow us to narrow down the result type of the comparison.
for neg in intersection.negative(self.db()) {
let result = match intersection_on {
IntersectionOn::Left => self.infer_binary_type_comparison(*neg, op, other).ok(),
IntersectionOn::Right => self.infer_binary_type_comparison(other, op, *neg).ok(),
IntersectionOn::Left => self
.infer_binary_type_comparison(*neg, op, other, range)
.ok(),
IntersectionOn::Right => self
.infer_binary_type_comparison(other, op, *neg, range)
.ok(),
};
match (op, result) {
@@ -4311,8 +4397,12 @@ impl<'db> TypeInferenceBuilder<'db> {
let mut builder = IntersectionBuilder::new(self.db());
for pos in intersection.positive(self.db()) {
let result = match intersection_on {
IntersectionOn::Left => self.infer_binary_type_comparison(*pos, op, other)?,
IntersectionOn::Right => self.infer_binary_type_comparison(other, op, *pos)?,
IntersectionOn::Left => {
self.infer_binary_type_comparison(*pos, op, other, range)?
}
IntersectionOn::Right => {
self.infer_binary_type_comparison(other, op, *pos, range)?
}
};
builder = builder.add_positive(result);
}
@@ -4331,6 +4421,7 @@ impl<'db> TypeInferenceBuilder<'db> {
left: Type<'db>,
op: ast::CmpOp,
right: Type<'db>,
range: TextRange,
) -> Result<Type<'db>, CompareUnsupportedError<'db>> {
// Note: identity (is, is not) for equal builtin types is unreliable and not part of the
// language spec.
@@ -4340,14 +4431,16 @@ impl<'db> TypeInferenceBuilder<'db> {
(Type::Union(union), other) => {
let mut builder = UnionBuilder::new(self.db());
for element in union.elements(self.db()) {
builder = builder.add(self.infer_binary_type_comparison(*element, op, other)?);
builder =
builder.add(self.infer_binary_type_comparison(*element, op, other, range)?);
}
Ok(builder.build())
}
(other, Type::Union(union)) => {
let mut builder = UnionBuilder::new(self.db());
for element in union.elements(self.db()) {
builder = builder.add(self.infer_binary_type_comparison(other, op, *element)?);
builder =
builder.add(self.infer_binary_type_comparison(other, op, *element, range)?);
}
Ok(builder.build())
}
@@ -4358,6 +4451,7 @@ impl<'db> TypeInferenceBuilder<'db> {
op,
right,
IntersectionOn::Left,
range,
),
(left, Type::Intersection(intersection)) => self
.infer_binary_intersection_type_comparison(
@@ -4365,6 +4459,7 @@ impl<'db> TypeInferenceBuilder<'db> {
op,
left,
IntersectionOn::Right,
range,
),
(Type::IntLiteral(n), Type::IntLiteral(m)) => match op {
@@ -4395,29 +4490,38 @@ impl<'db> TypeInferenceBuilder<'db> {
right_ty: right,
}),
},
(Type::IntLiteral(_), Type::Instance(_)) => {
self.infer_binary_type_comparison(KnownClass::Int.to_instance(self.db()), op, right)
}
(Type::Instance(_), Type::IntLiteral(_)) => {
self.infer_binary_type_comparison(left, op, KnownClass::Int.to_instance(self.db()))
}
(Type::IntLiteral(_), Type::Instance(_)) => self.infer_binary_type_comparison(
KnownClass::Int.to_instance(self.db()),
op,
right,
range,
),
(Type::Instance(_), Type::IntLiteral(_)) => self.infer_binary_type_comparison(
left,
op,
KnownClass::Int.to_instance(self.db()),
range,
),
// Booleans are coded as integers (False = 0, True = 1)
(Type::IntLiteral(n), Type::BooleanLiteral(b)) => self.infer_binary_type_comparison(
Type::IntLiteral(n),
op,
Type::IntLiteral(i64::from(b)),
range,
),
(Type::BooleanLiteral(b), Type::IntLiteral(m)) => self.infer_binary_type_comparison(
Type::IntLiteral(i64::from(b)),
op,
Type::IntLiteral(m),
range,
),
(Type::BooleanLiteral(a), Type::BooleanLiteral(b)) => self
.infer_binary_type_comparison(
Type::IntLiteral(i64::from(a)),
op,
Type::IntLiteral(i64::from(b)),
range,
),
(Type::StringLiteral(salsa_s1), Type::StringLiteral(salsa_s2)) => {
@@ -4448,19 +4552,31 @@ impl<'db> TypeInferenceBuilder<'db> {
}
}
}
(Type::StringLiteral(_), _) => {
self.infer_binary_type_comparison(KnownClass::Str.to_instance(self.db()), op, right)
}
(_, Type::StringLiteral(_)) => {
self.infer_binary_type_comparison(left, op, KnownClass::Str.to_instance(self.db()))
}
(Type::StringLiteral(_), _) => self.infer_binary_type_comparison(
KnownClass::Str.to_instance(self.db()),
op,
right,
range,
),
(_, Type::StringLiteral(_)) => self.infer_binary_type_comparison(
left,
op,
KnownClass::Str.to_instance(self.db()),
range,
),
(Type::LiteralString, _) => {
self.infer_binary_type_comparison(KnownClass::Str.to_instance(self.db()), op, right)
}
(_, Type::LiteralString) => {
self.infer_binary_type_comparison(left, op, KnownClass::Str.to_instance(self.db()))
}
(Type::LiteralString, _) => self.infer_binary_type_comparison(
KnownClass::Str.to_instance(self.db()),
op,
right,
range,
),
(_, Type::LiteralString) => self.infer_binary_type_comparison(
left,
op,
KnownClass::Str.to_instance(self.db()),
range,
),
(Type::BytesLiteral(salsa_b1), Type::BytesLiteral(salsa_b2)) => {
let b1 = &**salsa_b1.value(self.db());
@@ -4498,21 +4614,33 @@ impl<'db> TypeInferenceBuilder<'db> {
KnownClass::Bytes.to_instance(self.db()),
op,
right,
range,
),
(_, Type::BytesLiteral(_)) => self.infer_binary_type_comparison(
left,
op,
KnownClass::Bytes.to_instance(self.db()),
range,
),
(Type::Tuple(_), Type::Instance(InstanceType { class }))
if class.is_known(self.db(), KnownClass::VersionInfo) =>
{
self.infer_binary_type_comparison(left, op, Type::version_info_tuple(self.db()))
self.infer_binary_type_comparison(
left,
op,
Type::version_info_tuple(self.db()),
range,
)
}
(Type::Instance(InstanceType { class }), Type::Tuple(_))
if class.is_known(self.db(), KnownClass::VersionInfo) =>
{
self.infer_binary_type_comparison(Type::version_info_tuple(self.db()), op, right)
self.infer_binary_type_comparison(
Type::version_info_tuple(self.db()),
op,
right,
range,
)
}
(Type::Tuple(lhs), Type::Tuple(rhs)) => {
// Note: This only works on heterogeneous tuple types.
@@ -4520,7 +4648,7 @@ impl<'db> TypeInferenceBuilder<'db> {
let rhs_elements = rhs.elements(self.db());
let mut tuple_rich_comparison =
|op| self.infer_tuple_rich_comparison(lhs_elements, op, rhs_elements);
|op| self.infer_tuple_rich_comparison(lhs_elements, op, rhs_elements, range);
match op {
ast::CmpOp::Eq => tuple_rich_comparison(RichCompareOperator::Eq),
@@ -4538,10 +4666,14 @@ impl<'db> TypeInferenceBuilder<'db> {
Type::Tuple(lhs),
ast::CmpOp::Eq,
*ty,
range,
).expect("infer_binary_type_comparison should never return None for `CmpOp::Eq`");
match eq_result {
todo @ Type::Dynamic(DynamicType::Todo(_)) => return Ok(todo),
// It's okay to ignore errors here because Python doesn't call `__bool__`
// for different union variants. Instead, this is just for us to
// evaluate a possibly truthy value to `false` or `true`.
ty => match ty.bool(self.db()) {
Truthiness::AlwaysTrue => eq_count += 1,
Truthiness::AlwaysFalse => not_eq_count += 1,
@@ -4567,6 +4699,9 @@ impl<'db> TypeInferenceBuilder<'db> {
Ok(match eq_result {
todo @ Type::Dynamic(DynamicType::Todo(_)) => todo,
// It's okay to ignore errors here because Python doesn't call `__bool__`
// for `is` and `is not` comparisons. This is an implementation detail
// for how we determine the truthiness of a type.
ty => match ty.bool(self.db()) {
Truthiness::AlwaysFalse => Type::BooleanLiteral(op.is_is_not()),
_ => KnownClass::Bool.to_instance(self.db()),
@@ -4580,8 +4715,9 @@ impl<'db> TypeInferenceBuilder<'db> {
(Type::Instance(left_instance), Type::Instance(right_instance)) => {
let rich_comparison =
|op| self.infer_rich_comparison(left_instance, right_instance, op);
let membership_test_comparison =
|op| self.infer_membership_test_comparison(left_instance, right_instance, op);
let membership_test_comparison = |op, range: TextRange| {
self.infer_membership_test_comparison(left_instance, right_instance, op, range)
};
match op {
ast::CmpOp::Eq => rich_comparison(RichCompareOperator::Eq),
ast::CmpOp::NotEq => rich_comparison(RichCompareOperator::Ne),
@@ -4589,9 +4725,11 @@ impl<'db> TypeInferenceBuilder<'db> {
ast::CmpOp::LtE => rich_comparison(RichCompareOperator::Le),
ast::CmpOp::Gt => rich_comparison(RichCompareOperator::Gt),
ast::CmpOp::GtE => rich_comparison(RichCompareOperator::Ge),
ast::CmpOp::In => membership_test_comparison(MembershipTestCompareOperator::In),
ast::CmpOp::In => {
membership_test_comparison(MembershipTestCompareOperator::In, range)
}
ast::CmpOp::NotIn => {
membership_test_comparison(MembershipTestCompareOperator::NotIn)
membership_test_comparison(MembershipTestCompareOperator::NotIn, range)
}
ast::CmpOp::Is => {
if left.is_disjoint_from(self.db(), right) {
@@ -4640,7 +4778,6 @@ impl<'db> TypeInferenceBuilder<'db> {
let call_dunder = |op: RichCompareOperator,
left: InstanceType<'db>,
right: InstanceType<'db>| {
// TODO: How do we want to handle possibly unbound dunder methods?
match left.class.class_member(db, op.dunder()) {
Symbol::Type(class_member_dunder, Boundness::Bound) => class_member_dunder
.try_call(
@@ -4685,6 +4822,7 @@ impl<'db> TypeInferenceBuilder<'db> {
left: InstanceType<'db>,
right: InstanceType<'db>,
op: MembershipTestCompareOperator,
range: TextRange,
) -> Result<Type<'db>, CompareUnsupportedError<'db>> {
let db = self.db();
@@ -4702,11 +4840,10 @@ impl<'db> TypeInferenceBuilder<'db> {
}
_ => {
// iteration-based membership test
match Type::Instance(right).iterate(db) {
IterationOutcome::Iterable { .. } => Some(KnownClass::Bool.to_instance(db)),
IterationOutcome::NotIterable { .. }
| IterationOutcome::PossiblyUnboundDunderIter { .. } => None,
}
Type::Instance(right)
.try_iterate(db)
.map(|_| KnownClass::Bool.to_instance(db))
.ok()
}
};
@@ -4716,7 +4853,10 @@ impl<'db> TypeInferenceBuilder<'db> {
return ty;
}
let truthiness = ty.bool(db);
let truthiness = ty.try_bool(db).unwrap_or_else(|err| {
err.report_diagnostic(&self.context, range);
err.fallback_truthiness()
});
match op {
MembershipTestCompareOperator::In => truthiness.into_type(db),
@@ -4740,6 +4880,7 @@ impl<'db> TypeInferenceBuilder<'db> {
left: &[Type<'db>],
op: RichCompareOperator,
right: &[Type<'db>],
range: TextRange,
) -> Result<Type<'db>, CompareUnsupportedError<'db>> {
let left_iter = left.iter().copied();
let right_iter = right.iter().copied();
@@ -4748,46 +4889,49 @@ impl<'db> TypeInferenceBuilder<'db> {
for (l_ty, r_ty) in left_iter.zip(right_iter) {
let pairwise_eq_result = self
.infer_binary_type_comparison(l_ty, ast::CmpOp::Eq, r_ty)
.infer_binary_type_comparison(l_ty, ast::CmpOp::Eq, r_ty, range)
.expect("infer_binary_type_comparison should never return None for `CmpOp::Eq`");
match pairwise_eq_result {
// If propagation is required, return the result as is
todo @ Type::Dynamic(DynamicType::Todo(_)) => return Ok(todo),
ty => match ty.bool(self.db()) {
// - AlwaysTrue : Continue to the next pair for lexicographic comparison
Truthiness::AlwaysTrue => continue,
// - AlwaysFalse:
// Lexicographic comparisons will always terminate with this pair.
// Complete the comparison and return the result.
// - Ambiguous:
// Lexicographic comparisons might continue to the next pair (if eq_result is true),
// or terminate here (if eq_result is false).
// To account for cases where the comparison terminates here, add the pairwise comparison result to the union builder.
eq_truthiness @ (Truthiness::AlwaysFalse | Truthiness::Ambiguous) => {
let pairwise_compare_result = match op {
RichCompareOperator::Lt
| RichCompareOperator::Le
| RichCompareOperator::Gt
| RichCompareOperator::Ge => {
self.infer_binary_type_comparison(l_ty, op.into(), r_ty)?
}
// For `==` and `!=`, we already figure out the result from `pairwise_eq_result`
// NOTE: The CPython implementation does not account for non-boolean return types
// or cases where `!=` is not the negation of `==`, we also do not consider these cases.
RichCompareOperator::Eq => Type::BooleanLiteral(false),
RichCompareOperator::Ne => Type::BooleanLiteral(true),
};
builder = builder.add(pairwise_compare_result);
if eq_truthiness.is_ambiguous() {
continue;
match pairwise_eq_result
.try_bool(self.db())
.unwrap_or_else(|err| {
// TODO: We should, whenever possible, pass the range of the left and right elements
// instead of the range of the whole tuple.
err.report_diagnostic(&self.context, range);
err.fallback_truthiness()
}) {
// - AlwaysTrue : Continue to the next pair for lexicographic comparison
Truthiness::AlwaysTrue => continue,
// - AlwaysFalse:
// Lexicographic comparisons will always terminate with this pair.
// Complete the comparison and return the result.
// - Ambiguous:
// Lexicographic comparisons might continue to the next pair (if eq_result is true),
// or terminate here (if eq_result is false).
// To account for cases where the comparison terminates here, add the pairwise comparison result to the union builder.
eq_truthiness @ (Truthiness::AlwaysFalse | Truthiness::Ambiguous) => {
let pairwise_compare_result = match op {
RichCompareOperator::Lt
| RichCompareOperator::Le
| RichCompareOperator::Gt
| RichCompareOperator::Ge => {
self.infer_binary_type_comparison(l_ty, op.into(), r_ty, range)?
}
// For `==` and `!=`, we already figure out the result from `pairwise_eq_result`
// NOTE: The CPython implementation does not account for non-boolean return types
// or cases where `!=` is not the negation of `==`, we also do not consider these cases.
RichCompareOperator::Eq => Type::BooleanLiteral(false),
RichCompareOperator::Ne => Type::BooleanLiteral(true),
};
return Ok(builder.build());
builder = builder.add(pairwise_compare_result);
if eq_truthiness.is_ambiguous() {
continue;
}
},
return Ok(builder.build());
}
}
}
@@ -4953,7 +5097,7 @@ impl<'db> TypeInferenceBuilder<'db> {
match value_ty.try_call_dunder(
self.db(),
"__getitem__",
&CallArguments::positional([value_ty, slice_ty]),
&CallArguments::positional([slice_ty]),
) {
Ok(outcome) => return outcome.return_type(self.db()),
Err(err @ CallDunderError::PossiblyUnbound { .. }) => {
@@ -6245,6 +6389,7 @@ mod tests {
use crate::semantic_index::definition::Definition;
use crate::semantic_index::symbol::FileScopeId;
use crate::semantic_index::{global_scope, semantic_index, symbol_table, use_def_map};
use crate::symbol::global_symbol;
use crate::types::check_types;
use ruff_db::files::{system_path_to_file, File};
use ruff_db::system::DbWithTestSystem;

View File

@@ -29,9 +29,10 @@ use std::sync::{Arc, Mutex, MutexGuard, OnceLock};
use crate::db::tests::{setup_db, TestDb};
use crate::symbol::{builtins_symbol, known_module_symbol};
use crate::types::{
IntersectionBuilder, KnownClass, KnownInstanceType, SubclassOfType, TupleType, Type, UnionType,
BoundMethodType, CallableType, IntersectionBuilder, KnownClass, KnownInstanceType,
SubclassOfType, TupleType, Type, UnionType,
};
use crate::KnownModule;
use crate::{Db, KnownModule};
use quickcheck::{Arbitrary, Gen};
/// A test representation of a type that can be transformed unambiguously into a real Type,
@@ -67,6 +68,24 @@ pub(crate) enum Ty {
SubclassOfAbcClass(&'static str),
AlwaysTruthy,
AlwaysFalsy,
BuiltinsFunction(&'static str),
BuiltinsBoundMethod {
class: &'static str,
method: &'static str,
},
}
#[salsa::tracked]
fn create_bound_method<'db>(
db: &'db dyn Db,
function: Type<'db>,
builtins_class: Type<'db>,
) -> Type<'db> {
Type::Callable(CallableType::BoundMethod(BoundMethodType::new(
db,
function.expect_function_literal(),
builtins_class.to_instance(db),
)))
}
impl Ty {
@@ -123,6 +142,13 @@ impl Ty {
),
Ty::AlwaysTruthy => Type::AlwaysTruthy,
Ty::AlwaysFalsy => Type::AlwaysFalsy,
Ty::BuiltinsFunction(name) => builtins_symbol(db, name).expect_type(),
Ty::BuiltinsBoundMethod { class, method } => {
let builtins_class = builtins_symbol(db, class).expect_type();
let function = builtins_class.static_member(db, method).expect_type();
create_bound_method(db, function, builtins_class)
}
}
}
}
@@ -173,6 +199,16 @@ fn arbitrary_core_type(g: &mut Gen) -> Ty {
Ty::SubclassOfAbcClass("ABCMeta"),
Ty::AlwaysTruthy,
Ty::AlwaysFalsy,
Ty::BuiltinsFunction("chr"),
Ty::BuiltinsFunction("ascii"),
Ty::BuiltinsBoundMethod {
class: "str",
method: "isascii",
},
Ty::BuiltinsBoundMethod {
class: "int",
method: "bit_length",
},
])
.unwrap()
.clone()

View File

@@ -21,6 +21,13 @@ pub(crate) struct Signature<'db> {
}
impl<'db> Signature<'db> {
pub(crate) fn new(parameters: Parameters<'db>, return_ty: Option<Type<'db>>) -> Self {
Self {
parameters,
return_ty,
}
}
/// Return a todo signature: (*args: Todo, **kwargs: Todo) -> Todo
pub(crate) fn todo() -> Self {
Self {
@@ -64,6 +71,10 @@ impl<'db> Signature<'db> {
pub(crate) struct Parameters<'db>(Vec<Parameter<'db>>);
impl<'db> Parameters<'db> {
pub(crate) fn new(parameters: impl IntoIterator<Item = Parameter<'db>>) -> Self {
Self(parameters.into_iter().collect())
}
/// Return todo parameters: (*args: Todo, **kwargs: Todo)
fn todo() -> Self {
Self(vec![
@@ -233,6 +244,18 @@ pub(crate) struct Parameter<'db> {
}
impl<'db> Parameter<'db> {
pub(crate) fn new(
name: Option<Name>,
annotated_ty: Option<Type<'db>>,
kind: ParameterKind<'db>,
) -> Self {
Self {
name,
annotated_ty,
kind,
}
}
fn from_node_and_kind(
db: &'db dyn Db,
definition: Definition<'db>,
@@ -322,7 +345,8 @@ pub(crate) enum ParameterKind<'db> {
mod tests {
use super::*;
use crate::db::tests::{setup_db, TestDb};
use crate::types::{global_symbol, FunctionType, KnownClass};
use crate::symbol::global_symbol;
use crate::types::{FunctionType, KnownClass};
use ruff_db::system::DbWithTestSystem;
#[track_caller]

View File

@@ -64,8 +64,8 @@ impl<'db> SubclassOfType<'db> {
!self.is_dynamic()
}
pub(crate) fn member(self, db: &'db dyn Db, name: &str) -> Symbol<'db> {
Type::from(self.subclass_of).member(db, name)
pub(crate) fn static_member(self, db: &'db dyn Db, name: &str) -> Symbol<'db> {
Type::from(self.subclass_of).static_member(db, name)
}
/// Return `true` if `self` is a subtype of `other`.

View File

@@ -1,5 +1,7 @@
use std::cmp::Ordering;
use crate::types::CallableType;
use super::{
class_base::ClassBase, ClassLiteralType, DynamicType, InstanceType, KnownInstanceType,
TodoType, Type,
@@ -54,6 +56,27 @@ pub(super) fn union_elements_ordering<'db>(left: &Type<'db>, right: &Type<'db>)
(Type::FunctionLiteral(_), _) => Ordering::Less,
(_, Type::FunctionLiteral(_)) => Ordering::Greater,
(
Type::Callable(CallableType::BoundMethod(left)),
Type::Callable(CallableType::BoundMethod(right)),
) => left.cmp(right),
(Type::Callable(CallableType::BoundMethod(_)), _) => Ordering::Less,
(_, Type::Callable(CallableType::BoundMethod(_))) => Ordering::Greater,
(
Type::Callable(CallableType::MethodWrapperDunderGet(left)),
Type::Callable(CallableType::MethodWrapperDunderGet(right)),
) => left.cmp(right),
(Type::Callable(CallableType::MethodWrapperDunderGet(_)), _) => Ordering::Less,
(_, Type::Callable(CallableType::MethodWrapperDunderGet(_))) => Ordering::Greater,
(
Type::Callable(CallableType::WrapperDescriptorDunderGet),
Type::Callable(CallableType::WrapperDescriptorDunderGet),
) => Ordering::Equal,
(Type::Callable(CallableType::WrapperDescriptorDunderGet), _) => Ordering::Less,
(_, Type::Callable(CallableType::WrapperDescriptorDunderGet)) => Ordering::Greater,
(Type::Tuple(left), Type::Tuple(right)) => left.cmp(right),
(Type::Tuple(_), _) => Ordering::Less,
(_, Type::Tuple(_)) => Ordering::Greater,

View File

@@ -57,9 +57,10 @@ impl<'db> Unpacker<'db> {
if value.is_iterable() {
// If the value is an iterable, then the type that needs to be unpacked is the iterator
// type.
value_ty = value_ty
.iterate(self.db())
.unwrap_with_diagnostic(&self.context, value.as_any_node_ref(self.db()));
value_ty = value_ty.try_iterate(self.db()).unwrap_or_else(|err| {
err.report_diagnostic(&self.context, value.as_any_node_ref(self.db()));
err.fallback_element_type()
});
}
self.unpack_inner(target, value.as_any_node_ref(self.db()), value_ty);
@@ -155,8 +156,10 @@ impl<'db> Unpacker<'db> {
let ty = if ty.is_literal_string() {
Type::LiteralString
} else {
ty.iterate(self.db())
.unwrap_with_diagnostic(&self.context, value_expr)
ty.try_iterate(self.db()).unwrap_or_else(|err| {
err.report_diagnostic(&self.context, value_expr);
err.fallback_element_type()
})
};
for target_type in &mut target_types {
target_type.push(ty);

View File

@@ -178,7 +178,9 @@ use std::cmp::Ordering;
use ruff_index::{Idx, IndexVec};
use rustc_hash::FxHashMap;
use crate::semantic_index::constraint::{Constraint, ConstraintNode, PatternConstraintKind};
use crate::semantic_index::constraint::{
Constraint, ConstraintNode, Constraints, PatternConstraintKind, ScopedConstraintId,
};
use crate::types::{infer_expression_type, Truthiness};
use crate::Db;
@@ -231,69 +233,15 @@ impl std::fmt::Debug for ScopedVisibilityConstraintId {
#[derive(Clone, Copy, Debug, Eq, Hash, PartialEq)]
struct InteriorNode {
atom: Atom,
/// A "variable" that is evaluated as part of a TDD ternary function. For visibility
/// constraints, this is a `Constraint` that represents some runtime property of the Python
/// code that we are evaluating.
atom: ScopedConstraintId,
if_true: ScopedVisibilityConstraintId,
if_ambiguous: ScopedVisibilityConstraintId,
if_false: ScopedVisibilityConstraintId,
}
/// A "variable" that is evaluated as part of a TDD ternary function. For visibility constraints,
/// this is a `Constraint` that represents some runtime property of the Python code that we are
/// evaluating. We intern these constraints in an arena ([`VisibilityConstraints::constraints`]).
/// An atom is then an index into this arena.
///
/// By using a 32-bit index, we would typically allow 4 billion distinct constraints within a
/// scope. However, we sometimes have to model how a `Constraint` can have a different runtime
/// value at different points in the execution of the program. To handle this, we reserve the top
/// byte of an atom to represent a "copy number". This is just an opaque value that allows
/// different `Atom`s to evaluate the same `Constraint`. This yields a maximum of 16 million
/// distinct `Constraint`s in a scope, and 256 possible copies of each of those constraints.
#[derive(Clone, Copy, Eq, Hash, Ord, PartialEq, PartialOrd)]
struct Atom(u32);
impl Atom {
/// Deconstruct an atom into a constraint index and a copy number.
#[inline]
fn into_index_and_copy(self) -> (u32, u8) {
let copy = self.0 >> 24;
let index = self.0 & 0x00ff_ffff;
(index, copy as u8)
}
#[inline]
fn copy_of(mut self, copy: u8) -> Self {
// Clear out the previous copy number
self.0 &= 0x00ff_ffff;
// OR in the new one
self.0 |= u32::from(copy) << 24;
self
}
}
// A custom Debug implementation that prints out the constraint index and copy number as distinct
// fields.
impl std::fmt::Debug for Atom {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
let (index, copy) = self.into_index_and_copy();
f.debug_tuple("Atom").field(&index).field(&copy).finish()
}
}
impl Idx for Atom {
#[inline]
fn new(value: usize) -> Self {
assert!(value <= 0x00ff_ffff);
#[allow(clippy::cast_possible_truncation)]
Self(value as u32)
}
#[inline]
fn index(self) -> usize {
let (index, _) = self.into_index_and_copy();
index as usize
}
}
impl ScopedVisibilityConstraintId {
/// A special ID that is used for an "always true" / "always visible" constraint.
pub(crate) const ALWAYS_TRUE: ScopedVisibilityConstraintId =
@@ -336,16 +284,13 @@ const SMALLEST_TERMINAL: ScopedVisibilityConstraintId = ALWAYS_FALSE;
/// A collection of visibility constraints. This is currently stored in `UseDefMap`, which means we
/// maintain a separate set of visibility constraints for each scope in file.
#[derive(Debug, PartialEq, Eq, salsa::Update)]
pub(crate) struct VisibilityConstraints<'db> {
constraints: IndexVec<Atom, Constraint<'db>>,
pub(crate) struct VisibilityConstraints {
interiors: IndexVec<ScopedVisibilityConstraintId, InteriorNode>,
}
#[derive(Debug, Default, PartialEq, Eq)]
pub(crate) struct VisibilityConstraintsBuilder<'db> {
constraints: IndexVec<Atom, Constraint<'db>>,
pub(crate) struct VisibilityConstraintsBuilder {
interiors: IndexVec<ScopedVisibilityConstraintId, InteriorNode>,
constraint_cache: FxHashMap<Constraint<'db>, Atom>,
interior_cache: FxHashMap<InteriorNode, ScopedVisibilityConstraintId>,
not_cache: FxHashMap<ScopedVisibilityConstraintId, ScopedVisibilityConstraintId>,
and_cache: FxHashMap<
@@ -358,10 +303,9 @@ pub(crate) struct VisibilityConstraintsBuilder<'db> {
>,
}
impl<'db> VisibilityConstraintsBuilder<'db> {
pub(crate) fn build(self) -> VisibilityConstraints<'db> {
impl VisibilityConstraintsBuilder {
pub(crate) fn build(self) -> VisibilityConstraints {
VisibilityConstraints {
constraints: self.constraints,
interiors: self.interiors,
}
}
@@ -385,14 +329,6 @@ impl<'db> VisibilityConstraintsBuilder<'db> {
}
}
/// Adds a constraint, ensuring that we only store any particular constraint once.
fn add_constraint(&mut self, constraint: Constraint<'db>, copy: u8) -> Atom {
self.constraint_cache
.entry(constraint)
.or_insert_with(|| self.constraints.push(constraint))
.copy_of(copy)
}
/// Adds an interior node, ensuring that we always use the same visibility constraint ID for
/// equal nodes.
fn add_interior(&mut self, node: InteriorNode) -> ScopedVisibilityConstraintId {
@@ -408,17 +344,23 @@ impl<'db> VisibilityConstraintsBuilder<'db> {
.or_insert_with(|| self.interiors.push(node))
}
/// Adds a new visibility constraint that checks a single [`Constraint`]. Provide different
/// values for `copy` if you need to model that the constraint can evaluate to different
/// results at different points in the execution of the program being modeled.
/// Adds a new visibility constraint that checks a single [`Constraint`].
///
/// [`ScopedConstraintId`]s are the “variables” that are evaluated by a TDD. A TDD variable has
/// the same value no matter how many times it appears in the ternary formula that the TDD
/// represents.
///
/// However, we sometimes have to model how a `Constraint` can have a different runtime
/// value at different points in the execution of the program. To handle this, you can take
/// advantage of the fact that the [`Constraints`] arena does not deduplicate `Constraint`s.
/// You can add a `Constraint` multiple times, yielding different `ScopedConstraintId`s, which
/// you can then create separate TDD atoms for.
pub(crate) fn add_atom(
&mut self,
constraint: Constraint<'db>,
copy: u8,
constraint: ScopedConstraintId,
) -> ScopedVisibilityConstraintId {
let atom = self.add_constraint(constraint, copy);
self.add_interior(InteriorNode {
atom,
atom: constraint,
if_true: ALWAYS_TRUE,
if_ambiguous: AMBIGUOUS,
if_false: ALWAYS_FALSE,
@@ -588,11 +530,12 @@ impl<'db> VisibilityConstraintsBuilder<'db> {
}
}
impl<'db> VisibilityConstraints<'db> {
impl VisibilityConstraints {
/// Analyze the statically known visibility for a given visibility constraint.
pub(crate) fn evaluate(
pub(crate) fn evaluate<'db>(
&self,
db: &'db dyn Db,
constraints: &Constraints<'db>,
mut id: ScopedVisibilityConstraintId,
) -> Truthiness {
loop {
@@ -602,7 +545,7 @@ impl<'db> VisibilityConstraints<'db> {
ALWAYS_FALSE => return Truthiness::AlwaysFalse,
_ => self.interiors[id],
};
let constraint = &self.constraints[node.atom];
let constraint = &constraints[node.atom];
match Self::analyze_single(db, constraint) {
Truthiness::AlwaysTrue => id = node.if_true,
Truthiness::Ambiguous => id = node.if_ambiguous,

View File

@@ -253,10 +253,15 @@ fn run_test(
})
.collect();
if !snapshot_diagnostics.is_empty() {
if snapshot_diagnostics.is_empty() && test.should_snapshot_diagnostics() {
panic!(
"Test `{}` requested snapshotting diagnostics but it didn't produce any.",
test.name()
);
} else if !snapshot_diagnostics.is_empty() {
let snapshot =
create_diagnostic_snapshot(db, relative_fixture_path, test, snapshot_diagnostics);
let name = test.name().replace(' ', "_");
let name = test.name().replace(' ', "_").replace(':', "__");
insta::with_settings!(
{
snapshot_path => snapshot_path,

View File

@@ -595,6 +595,10 @@ impl<'s> Parser<'s> {
return self.process_config_block(code);
}
if lang == "ignore" {
return Ok(());
}
if let Some(explicit_path) = self.explicit_path {
if !lang.is_empty()
&& lang != "text"
@@ -618,7 +622,7 @@ impl<'s> Parser<'s> {
EmbeddedFilePath::Explicit(path)
}
None => match lang {
"py" => EmbeddedFilePath::Autogenerated(PySourceType::Python),
"py" | "python" => EmbeddedFilePath::Autogenerated(PySourceType::Python),
"pyi" => EmbeddedFilePath::Autogenerated(PySourceType::Stub),
"" => {
bail!("Cannot auto-generate file name for code block with empty language specifier in test `{test_name}`");

View File

@@ -68,8 +68,44 @@ class TestClass:
def __eq__(self, **kwargs): # ignore **kwargs
...
def __eq__(self, /, other=42): # ignore positional-only args
def __eq__(self, /, other=42): # support positional-only args
...
def __eq__(self, *, other=42): # ignore positional-only args
def __eq__(self, *, other=42): # ignore keyword-only args
...
def __cmp__(self): # #16217 assert non-special method is skipped, expects 2 parameters
...
def __div__(self): # #16217 assert non-special method is skipped, expects 2 parameters
...
def __nonzero__(self, x): # #16217 assert non-special method is skipped, expects 1 parameter
...
def __unicode__(self, x): # #16217 assert non-special method is skipped, expects 1 parameter
...
def __next__(self, x): # #16217 assert special method is linted, expects 1 parameter
...
def __buffer__(self): # #16217 assert special method is linted, expects 2 parameters
...
def __class_getitem__(self): # #16217 assert special method is linted, expects 2 parameters
...
def __mro_entries__(self): # #16217 assert special method is linted, expects 2 parameters
...
def __release_buffer__(self): # #16217 assert special method is linted, expects 2 parameters
...
def __subclasshook__(self): # #16217 assert special method is linted, expects 2 parameters
...
def __setattr__(self, /, name): # #16217 assert positional-only special method is linted, expects 3 parameters
...
def __setitem__(self, key, /, value, extra_value): # #16217 assert positional-only special method is linted, expects 3 parameters
...

View File

@@ -81,28 +81,42 @@ class H(BaseModel):
final_variable: Final[list[int]] = []
from pydantic.v1 import BaseModel as V1BaseModel
class I(V1BaseModel):
mutable_default: list[int] = []
from pydantic.v1.generics import GenericModel
class J(GenericModel):
mutable_default: list[int] = []
def sqlmodel_import_checker():
from sqlmodel.main import SQLModel
class I(SQLModel):
class K(SQLModel):
id: int
mutable_default: list[int] = []
from sqlmodel import SQLModel
class J(SQLModel):
class L(SQLModel):
id: int
name: str
class K(SQLModel):
class M(SQLModel):
id: int
i_s: list[J] = []
class L(SQLModel):
class N(SQLModel):
id: int
i_j: list[K] = list()
i_j: list[L] = list()
# Lint should account for deferred annotations
# See https://github.com/astral-sh/ruff/issues/15857

View File

@@ -23,10 +23,8 @@ impl ExpectedParams {
| "__neg__" | "__pos__" | "__abs__" | "__invert__" | "__complex__" | "__int__"
| "__float__" | "__index__" | "__trunc__" | "__floor__" | "__ceil__" | "__enter__"
| "__aenter__" | "__getnewargs_ex__" | "__getnewargs__" | "__getstate__"
| "__reduce__" | "__copy__" | "__unicode__" | "__nonzero__" | "__await__"
| "__aiter__" | "__anext__" | "__fspath__" | "__subclasses__" => {
Some(ExpectedParams::Fixed(0))
}
| "__reduce__" | "__copy__" | "__await__" | "__aiter__" | "__anext__"
| "__fspath__" | "__subclasses__" | "__next__" => Some(ExpectedParams::Fixed(0)),
"__format__" | "__lt__" | "__le__" | "__eq__" | "__ne__" | "__gt__" | "__ge__"
| "__getattr__" | "__getattribute__" | "__delattr__" | "__delete__"
| "__instancecheck__" | "__subclasscheck__" | "__getitem__" | "__missing__"
@@ -37,8 +35,9 @@ impl ExpectedParams {
| "__rpow__" | "__rlshift__" | "__rrshift__" | "__rand__" | "__rxor__" | "__ror__"
| "__iadd__" | "__isub__" | "__imul__" | "__itruediv__" | "__ifloordiv__"
| "__imod__" | "__ilshift__" | "__irshift__" | "__iand__" | "__ixor__" | "__ior__"
| "__ipow__" | "__setstate__" | "__reduce_ex__" | "__deepcopy__" | "__cmp__"
| "__matmul__" | "__rmatmul__" | "__imatmul__" | "__div__" => {
| "__ipow__" | "__setstate__" | "__reduce_ex__" | "__deepcopy__" | "__matmul__"
| "__rmatmul__" | "__imatmul__" | "__buffer__" | "__class_getitem__"
| "__mro_entries__" | "__release_buffer__" | "__subclasshook__" => {
Some(ExpectedParams::Fixed(1))
}
"__setattr__" | "__get__" | "__set__" | "__setitem__" | "__set_name__" => {
@@ -147,11 +146,8 @@ pub(crate) fn unexpected_special_method_signature(
return;
}
// Ignore methods with positional-only or keyword-only parameters, or variadic parameters.
if !parameters.posonlyargs.is_empty()
|| !parameters.kwonlyargs.is_empty()
|| parameters.kwarg.is_some()
{
// Ignore methods with keyword-only parameters or variadic parameters.
if !parameters.kwonlyargs.is_empty() || parameters.kwarg.is_some() {
return;
}
@@ -160,10 +156,11 @@ pub(crate) fn unexpected_special_method_signature(
return;
}
let actual_params = parameters.args.len();
let actual_params = parameters.args.len() + parameters.posonlyargs.len();
let mandatory_params = parameters
.args
.iter()
.chain(parameters.posonlyargs.iter())
.filter(|arg| arg.default.is_none())
.count();

View File

@@ -71,3 +71,75 @@ unexpected_special_method_signature.py:65:9: PLE0302 The special method `__round
| ^^^^^^^^^ PLE0302
66 | ...
|
unexpected_special_method_signature.py:89:9: PLE0302 The special method `__next__` expects 1 parameter, 2 were given
|
87 | ...
88 |
89 | def __next__(self, x): # #16217 assert special method is linted, expects 1 parameter
| ^^^^^^^^ PLE0302
90 | ...
|
unexpected_special_method_signature.py:92:9: PLE0302 The special method `__buffer__` expects 2 parameters, 1 was given
|
90 | ...
91 |
92 | def __buffer__(self): # #16217 assert special method is linted, expects 2 parameters
| ^^^^^^^^^^ PLE0302
93 | ...
|
unexpected_special_method_signature.py:95:9: PLE0302 The special method `__class_getitem__` expects 2 parameters, 1 was given
|
93 | ...
94 |
95 | def __class_getitem__(self): # #16217 assert special method is linted, expects 2 parameters
| ^^^^^^^^^^^^^^^^^ PLE0302
96 | ...
|
unexpected_special_method_signature.py:98:9: PLE0302 The special method `__mro_entries__` expects 2 parameters, 1 was given
|
96 | ...
97 |
98 | def __mro_entries__(self): # #16217 assert special method is linted, expects 2 parameters
| ^^^^^^^^^^^^^^^ PLE0302
99 | ...
|
unexpected_special_method_signature.py:101:9: PLE0302 The special method `__release_buffer__` expects 2 parameters, 1 was given
|
99 | ...
100 |
101 | def __release_buffer__(self): # #16217 assert special method is linted, expects 2 parameters
| ^^^^^^^^^^^^^^^^^^ PLE0302
102 | ...
|
unexpected_special_method_signature.py:104:9: PLE0302 The special method `__subclasshook__` expects 2 parameters, 1 was given
|
102 | ...
103 |
104 | def __subclasshook__(self): # #16217 assert special method is linted, expects 2 parameters
| ^^^^^^^^^^^^^^^^ PLE0302
105 | ...
|
unexpected_special_method_signature.py:107:9: PLE0302 The special method `__setattr__` expects 3 parameters, 2 were given
|
105 | ...
106 |
107 | def __setattr__(self, /, name): # #16217 assert positional-only special method is linted, expects 3 parameters
| ^^^^^^^^^^^ PLE0302
108 | ...
|
unexpected_special_method_signature.py:110:9: PLE0302 The special method `__setitem__` expects 3 parameters, 4 were given
|
108 | ...
109 |
110 | def __setitem__(self, key, /, value, extra_value): # #16217 assert positional-only special method is linted, expects 3 parameters
| ^^^^^^^^^^^ PLE0302
111 | ...
|

View File

@@ -165,7 +165,7 @@ pub(super) fn dataclass_kind<'a>(
/// Returns `true` if the given class has "default copy" semantics.
///
/// For example, Pydantic `BaseModel` and `BaseSettings` subclassses copy attribute defaults on
/// For example, Pydantic `BaseModel` and `BaseSettings` subclasses copy attribute defaults on
/// instance creation. As such, the use of mutable default values is safe for such classes.
pub(super) fn has_default_copy_semantics(
class_def: &ast::StmtClassDef,
@@ -174,7 +174,16 @@ pub(super) fn has_default_copy_semantics(
analyze::class::any_qualified_base_class(class_def, semantic, &|qualified_name| {
matches!(
qualified_name.segments(),
["pydantic", "BaseModel" | "BaseSettings" | "BaseConfig"]
[
"pydantic",
"BaseModel" | "RootModel" | "BaseSettings" | "BaseConfig"
] | ["pydantic", "generics", "GenericModel"]
| [
"pydantic",
"v1",
"BaseModel" | "BaseSettings" | "BaseConfig"
]
| ["pydantic", "v1", "generics", "GenericModel"]
| ["pydantic_settings", "BaseSettings"]
| ["msgspec", "Struct"]
| ["sqlmodel", "SQLModel"]

View File

@@ -31,78 +31,78 @@ RUF012.py:25:26: RUF012 Mutable class attributes should be annotated with `typin
27 | class_variable: ClassVar[list[int]] = []
|
RUF012.py:89:38: RUF012 Mutable class attributes should be annotated with `typing.ClassVar`
|
87 | class I(SQLModel):
88 | id: int
89 | mutable_default: list[int] = []
| ^^ RUF012
90 |
91 | from sqlmodel import SQLModel
|
RUF012.py:114:36: RUF012 Mutable class attributes should be annotated with `typing.ClassVar`
RUF012.py:103:38: RUF012 Mutable class attributes should be annotated with `typing.ClassVar`
|
112 | }
113 |
114 | mutable_default: 'list[int]' = []
101 | class K(SQLModel):
102 | id: int
103 | mutable_default: list[int] = []
| ^^ RUF012
104 |
105 | from sqlmodel import SQLModel
|
RUF012.py:128:36: RUF012 Mutable class attributes should be annotated with `typing.ClassVar`
|
126 | }
127 |
128 | mutable_default: 'list[int]' = []
| ^^ RUF012
115 | immutable_annotation: 'Sequence[int]'= []
116 | without_annotation = []
129 | immutable_annotation: 'Sequence[int]'= []
130 | without_annotation = []
|
RUF012.py:115:44: RUF012 Mutable class attributes should be annotated with `typing.ClassVar`
RUF012.py:129:44: RUF012 Mutable class attributes should be annotated with `typing.ClassVar`
|
114 | mutable_default: 'list[int]' = []
115 | immutable_annotation: 'Sequence[int]'= []
128 | mutable_default: 'list[int]' = []
129 | immutable_annotation: 'Sequence[int]'= []
| ^^ RUF012
116 | without_annotation = []
117 | class_variable: 'ClassVar[list[int]]' = []
130 | without_annotation = []
131 | class_variable: 'ClassVar[list[int]]' = []
|
RUF012.py:116:26: RUF012 Mutable class attributes should be annotated with `typing.ClassVar`
RUF012.py:130:26: RUF012 Mutable class attributes should be annotated with `typing.ClassVar`
|
114 | mutable_default: 'list[int]' = []
115 | immutable_annotation: 'Sequence[int]'= []
116 | without_annotation = []
128 | mutable_default: 'list[int]' = []
129 | immutable_annotation: 'Sequence[int]'= []
130 | without_annotation = []
| ^^ RUF012
117 | class_variable: 'ClassVar[list[int]]' = []
118 | final_variable: 'Final[list[int]]' = []
131 | class_variable: 'ClassVar[list[int]]' = []
132 | final_variable: 'Final[list[int]]' = []
|
RUF012.py:117:45: RUF012 Mutable class attributes should be annotated with `typing.ClassVar`
RUF012.py:131:45: RUF012 Mutable class attributes should be annotated with `typing.ClassVar`
|
115 | immutable_annotation: 'Sequence[int]'= []
116 | without_annotation = []
117 | class_variable: 'ClassVar[list[int]]' = []
129 | immutable_annotation: 'Sequence[int]'= []
130 | without_annotation = []
131 | class_variable: 'ClassVar[list[int]]' = []
| ^^ RUF012
118 | final_variable: 'Final[list[int]]' = []
119 | class_variable_without_subscript: 'ClassVar' = []
132 | final_variable: 'Final[list[int]]' = []
133 | class_variable_without_subscript: 'ClassVar' = []
|
RUF012.py:118:42: RUF012 Mutable class attributes should be annotated with `typing.ClassVar`
RUF012.py:132:42: RUF012 Mutable class attributes should be annotated with `typing.ClassVar`
|
116 | without_annotation = []
117 | class_variable: 'ClassVar[list[int]]' = []
118 | final_variable: 'Final[list[int]]' = []
130 | without_annotation = []
131 | class_variable: 'ClassVar[list[int]]' = []
132 | final_variable: 'Final[list[int]]' = []
| ^^ RUF012
119 | class_variable_without_subscript: 'ClassVar' = []
120 | final_variable_without_subscript: 'Final' = []
133 | class_variable_without_subscript: 'ClassVar' = []
134 | final_variable_without_subscript: 'Final' = []
|
RUF012.py:119:52: RUF012 Mutable class attributes should be annotated with `typing.ClassVar`
RUF012.py:133:52: RUF012 Mutable class attributes should be annotated with `typing.ClassVar`
|
117 | class_variable: 'ClassVar[list[int]]' = []
118 | final_variable: 'Final[list[int]]' = []
119 | class_variable_without_subscript: 'ClassVar' = []
131 | class_variable: 'ClassVar[list[int]]' = []
132 | final_variable: 'Final[list[int]]' = []
133 | class_variable_without_subscript: 'ClassVar' = []
| ^^ RUF012
120 | final_variable_without_subscript: 'Final' = []
134 | final_variable_without_subscript: 'Final' = []
|
RUF012.py:120:49: RUF012 Mutable class attributes should be annotated with `typing.ClassVar`
RUF012.py:134:49: RUF012 Mutable class attributes should be annotated with `typing.ClassVar`
|
118 | final_variable: 'Final[list[int]]' = []
119 | class_variable_without_subscript: 'ClassVar' = []
120 | final_variable_without_subscript: 'Final' = []
132 | final_variable: 'Final[list[int]]' = []
133 | class_variable_without_subscript: 'ClassVar' = []
134 | final_variable_without_subscript: 'Final' = []
| ^^ RUF012
|

View File

@@ -2,8 +2,9 @@
pub use edit::{DocumentKey, NotebookDocument, PositionEncoding, TextDocument};
use lsp_types::CodeActionKind;
pub use server::{Server, Workspace, Workspaces};
pub use server::Server;
pub use session::{ClientSettings, DocumentQuery, DocumentSnapshot, Session};
pub use workspace::{Workspace, Workspaces};
#[macro_use]
mod message;
@@ -16,6 +17,7 @@ mod logging;
mod resolve;
mod server;
mod session;
mod workspace;
pub(crate) const SERVER_NAME: &str = "ruff";
pub(crate) const DIAGNOSTIC_NAME: &str = "Ruff";

View File

@@ -81,13 +81,17 @@ pub(crate) fn check(
return DiagnosticsMap::default();
}
detect_package_root(
document_path
.parent()
.expect("a path to a document should have a parent path"),
&settings.linter.namespace_packages,
)
.map(PackageRoot::root)
if let Some(parent) = document_path.parent() {
detect_package_root(parent, &settings.linter.namespace_packages).map(PackageRoot::root)
} else {
// Untitled documents in Neovim are represented using `file:///` URIs which have no
// parent. See https://github.com/astral-sh/ruff/issues/15392.
tracing::info!(
"Cannot detect package root for document with no parent: {:?}",
document_path
);
None
}
} else {
None
};

View File

@@ -3,14 +3,11 @@
use lsp_server as lsp;
use lsp_types as types;
use lsp_types::InitializeParams;
use lsp_types::WorkspaceFolder;
use std::num::NonZeroUsize;
use std::ops::Deref;
// The new PanicInfoHook name requires MSRV >= 1.82
#[allow(deprecated)]
use std::panic::PanicInfo;
use std::str::FromStr;
use thiserror::Error;
use types::ClientCapabilities;
use types::CodeActionKind;
use types::CodeActionOptions;
@@ -24,7 +21,6 @@ use types::OneOf;
use types::TextDocumentSyncCapability;
use types::TextDocumentSyncKind;
use types::TextDocumentSyncOptions;
use types::Url;
use types::WorkDoneProgressOptions;
use types::WorkspaceFoldersServerCapabilities;
@@ -34,9 +30,8 @@ use self::schedule::event_loop_thread;
use self::schedule::Scheduler;
use self::schedule::Task;
use crate::session::AllSettings;
use crate::session::ClientSettings;
use crate::session::Session;
use crate::session::WorkspaceSettingsMap;
use crate::workspace::Workspaces;
use crate::PositionEncoding;
mod api;
@@ -447,122 +442,3 @@ impl FromStr for SupportedCommand {
})
}
}
#[derive(Debug)]
pub struct Workspaces(Vec<Workspace>);
impl Workspaces {
pub fn new(workspaces: Vec<Workspace>) -> Self {
Self(workspaces)
}
/// Create the workspaces from the provided workspace folders as provided by the client during
/// initialization.
fn from_workspace_folders(
workspace_folders: Option<Vec<WorkspaceFolder>>,
mut workspace_settings: WorkspaceSettingsMap,
) -> std::result::Result<Workspaces, WorkspacesError> {
let mut client_settings_for_url = |url: &Url| {
workspace_settings.remove(url).unwrap_or_else(|| {
tracing::info!(
"No workspace settings found for {}, using default settings",
url
);
ClientSettings::default()
})
};
let workspaces =
if let Some(folders) = workspace_folders.filter(|folders| !folders.is_empty()) {
folders
.into_iter()
.map(|folder| {
let settings = client_settings_for_url(&folder.uri);
Workspace::new(folder.uri).with_settings(settings)
})
.collect()
} else {
let current_dir = std::env::current_dir().map_err(WorkspacesError::Io)?;
tracing::info!(
"No workspace(s) were provided during initialization. \
Using the current working directory as a default workspace: {}",
current_dir.display()
);
let uri = Url::from_file_path(current_dir)
.map_err(|()| WorkspacesError::InvalidCurrentDir)?;
let settings = client_settings_for_url(&uri);
vec![Workspace::default(uri).with_settings(settings)]
};
Ok(Workspaces(workspaces))
}
}
impl Deref for Workspaces {
type Target = [Workspace];
fn deref(&self) -> &Self::Target {
&self.0
}
}
#[derive(Error, Debug)]
enum WorkspacesError {
#[error(transparent)]
Io(#[from] std::io::Error),
#[error("Failed to create a URL from the current working directory")]
InvalidCurrentDir,
}
#[derive(Debug)]
pub struct Workspace {
/// The [`Url`] pointing to the root of the workspace.
url: Url,
/// The client settings for this workspace.
settings: Option<ClientSettings>,
/// Whether this is the default workspace as created by the server. This will be the case when
/// no workspace folders were provided during initialization.
is_default: bool,
}
impl Workspace {
/// Create a new workspace with the given root URL.
pub fn new(url: Url) -> Self {
Self {
url,
settings: None,
is_default: false,
}
}
/// Create a new default workspace with the given root URL.
pub fn default(url: Url) -> Self {
Self {
url,
settings: None,
is_default: true,
}
}
/// Set the client settings for this workspace.
#[must_use]
pub fn with_settings(mut self, settings: ClientSettings) -> Self {
self.settings = Some(settings);
self
}
/// Returns the root URL of the workspace.
pub(crate) fn url(&self) -> &Url {
&self.url
}
/// Returns the client settings for this workspace.
pub(crate) fn settings(&self) -> Option<&ClientSettings> {
self.settings.as_ref()
}
/// Returns true if this is the default workspace.
pub(crate) fn is_default(&self) -> bool {
self.is_default
}
}

View File

@@ -7,7 +7,7 @@ use lsp_types::{ClientCapabilities, FileEvent, NotebookDocumentCellChange, Url};
use settings::ResolvedClientSettings;
use crate::edit::{DocumentKey, DocumentVersion, NotebookDocument};
use crate::server::Workspaces;
use crate::workspace::Workspaces;
use crate::{PositionEncoding, TextDocument};
pub(crate) use self::capabilities::ResolvedClientCapabilities;

View File

@@ -11,7 +11,7 @@ use thiserror::Error;
pub(crate) use ruff_settings::RuffSettings;
use crate::edit::LanguageId;
use crate::server::{Workspace, Workspaces};
use crate::workspace::{Workspace, Workspaces};
use crate::{
edit::{DocumentKey, DocumentVersion, NotebookDocument},
PositionEncoding, TextDocument,
@@ -427,12 +427,15 @@ impl WorkspaceSettingsIndex {
let workspace_url = workspace.url();
if workspace_url.scheme() != "file" {
tracing::info!("Ignoring non-file workspace URL: {workspace_url}");
show_warn_msg!("Ruff does not support non-file workspaces; Ignoring {workspace_url}");
show_warn_msg!("Ruff does not support non-file workspaces; ignoring {workspace_url}");
return Ok(());
}
let workspace_path = workspace_url.to_file_path().map_err(|()| {
anyhow!("Failed to convert workspace URL to file path: {workspace_url}")
})?;
let Ok(workspace_path) = workspace_url.to_file_path() else {
tracing::warn!(
"Failed to convert workspace URL to file path; ignoring {workspace_url}"
);
return Ok(());
};
let client_settings = if let Some(workspace_settings) = workspace.settings() {
ResolvedClientSettings::with_workspace(workspace_settings, global_settings)

View File

@@ -0,0 +1,126 @@
use std::ops::Deref;
use lsp_types::{Url, WorkspaceFolder};
use thiserror::Error;
use crate::session::WorkspaceSettingsMap;
use crate::ClientSettings;
#[derive(Debug)]
pub struct Workspaces(Vec<Workspace>);
impl Workspaces {
pub fn new(workspaces: Vec<Workspace>) -> Self {
Self(workspaces)
}
/// Create the workspaces from the provided workspace folders as provided by the client during
/// initialization.
pub(crate) fn from_workspace_folders(
workspace_folders: Option<Vec<WorkspaceFolder>>,
mut workspace_settings: WorkspaceSettingsMap,
) -> std::result::Result<Workspaces, WorkspacesError> {
let mut client_settings_for_url = |url: &Url| {
workspace_settings.remove(url).unwrap_or_else(|| {
tracing::info!(
"No workspace settings found for {}, using default settings",
url
);
ClientSettings::default()
})
};
let workspaces =
if let Some(folders) = workspace_folders.filter(|folders| !folders.is_empty()) {
folders
.into_iter()
.map(|folder| {
let settings = client_settings_for_url(&folder.uri);
Workspace::new(folder.uri).with_settings(settings)
})
.collect()
} else {
let current_dir = std::env::current_dir().map_err(WorkspacesError::Io)?;
tracing::info!(
"No workspace(s) were provided during initialization. \
Using the current working directory as a default workspace: {}",
current_dir.display()
);
let uri = Url::from_file_path(current_dir)
.map_err(|()| WorkspacesError::InvalidCurrentDir)?;
let settings = client_settings_for_url(&uri);
vec![Workspace::default(uri).with_settings(settings)]
};
Ok(Workspaces(workspaces))
}
}
impl Deref for Workspaces {
type Target = [Workspace];
fn deref(&self) -> &Self::Target {
&self.0
}
}
#[derive(Error, Debug)]
pub(crate) enum WorkspacesError {
#[error(transparent)]
Io(#[from] std::io::Error),
#[error("Failed to create a URL from the current working directory")]
InvalidCurrentDir,
}
#[derive(Debug)]
pub struct Workspace {
/// The [`Url`] pointing to the root of the workspace.
url: Url,
/// The client settings for this workspace.
settings: Option<ClientSettings>,
/// Whether this is the default workspace as created by the server. This will be the case when
/// no workspace folders were provided during initialization.
is_default: bool,
}
impl Workspace {
/// Create a new workspace with the given root URL.
pub fn new(url: Url) -> Self {
Self {
url,
settings: None,
is_default: false,
}
}
/// Create a new default workspace with the given root URL.
pub fn default(url: Url) -> Self {
Self {
url,
settings: None,
is_default: true,
}
}
/// Set the client settings for this workspace.
#[must_use]
pub fn with_settings(mut self, settings: ClientSettings) -> Self {
self.settings = Some(settings);
self
}
/// Returns the root URL of the workspace.
pub(crate) fn url(&self) -> &Url {
&self.url
}
/// Returns the client settings for this workspace.
pub(crate) fn settings(&self) -> Option<&ClientSettings> {
self.settings.as_ref()
}
/// Returns true if this is the default workspace.
pub(crate) fn is_default(&self) -> bool {
self.is_default
}
}

View File

@@ -29,7 +29,7 @@ ruff_python_formatter = { path = "../crates/ruff_python_formatter" }
ruff_text_size = { path = "../crates/ruff_text_size" }
libfuzzer-sys = { git = "https://github.com/rust-fuzz/libfuzzer", default-features = false }
salsa = { git = "https://github.com/salsa-rs/salsa.git", rev = "351d9cf0037be949d17800d0c7b4838e533c2ed6" }
salsa = { git = "https://github.com/salsa-rs/salsa.git", rev = "c8826fa4d1d9e3cba4c6e578763878b71fa9a10d" }
similar = { version = "2.5.0" }
tracing = { version = "0.1.40" }

View File

@@ -651,6 +651,16 @@
}
]
},
"unsupported-bool-conversion": {
"title": "detects boolean conversion where the object incorrectly implements `__bool__`",
"description": "## What it does\nChecks for bool conversions where the object doesn't correctly implement `__bool__`.\n\n## Why is this bad?\nIf an exception is raised when you attempt to evaluate the truthiness of an object,\nusing the object in a boolean context will fail at runtime.\n\n## Examples\n\n```python\nclass NotBoolable:\n __bool__ = None\n\nb1 = NotBoolable()\nb2 = NotBoolable()\n\nif b1: # exception raised here\n pass\n\nb1 and b2 # exception raised here\nnot b1 # exception raised here\nb1 < b2 < b1 # exception raised here\n```",
"default": "error",
"oneOf": [
{
"$ref": "#/definitions/Level"
}
]
},
"unsupported-operator": {
"title": "detects binary, unary, or comparison expressions where the operands don't support the operator",
"description": "## What it does\nChecks for binary expressions, comparisons, and unary expressions where the operands don't support the operator.\n\nTODO #14889",