Compare commits

...

12 Commits

Author SHA1 Message Date
Dhruv Manilawala
c705787bb3 WIP 2025-02-22 10:59:20 +05:30
Douglas Creager
4dae09ecff [red-knot] Better handling of visibility constraint copies (#16276)
Two related changes.  For context:

1. We were maintaining two separate arenas of `Constraint`s in each
use-def map. One was used for narrowing constraints, and the other for
visibility constraints. The visibility constraint arena was interned,
ensuring that we always used the same ID for any particular
`Constraint`. The narrowing constraint arena was not interned.

2. The TDD code relies on _all_ TDD nodes being interned and reduced.
This is an important requirement for TDDs to be a canonical form, which
allows us to use a single int comparison to test for "always true/false"
and to compare two TDDs for equivalence. But we also need to support an
individual `Constraint` having multiple values in a TDD evaluation (e.g.
to handle a `while` condition having different values the first time
it's evaluated vs later times). Previously, we handled that by
introducing a "copy" number, which was only there as a disambiguator, to
allow an interned, deduplicated constraint ID to appear in the TDD
formula multiple times.

A better way to handle (2) is to not intern the constraints in the
visibility constraint arena! The caller now gets to decide: if they add
a `Constraint` to the arena more than once, they get distinct
`ScopedConstraintId`s — which the TDD code will treat as distinct
variables, allowing them to take on different values in the ternary
function.

With that in place, we can then consolidate on a single (non-interned)
arena, which is shared for both narrowing and visibility constraints.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2025-02-21 09:16:25 -05:00
Darius Carrier
b9b094869a [pylint] Fix false positives, add missing methods, and support positional-only parameters (PLE0302) (#16263)
## Summary

Resolves 3/4 requests in #16217:

-  Remove not special methods: `__cmp__`, `__div__`, `__nonzero__`, and
`__unicode__`.
-  Add special methods: `__next__`, `__buffer__`, `__class_getitem__`,
`__mro_entries__`, `__release_buffer__`, and `__subclasshook__`.
-  Support positional-only arguments.
-  Add support for module functions `__dir__` and `__getattr__`. As
mentioned in the issue the check is scoped for methods rather than
module functions. I am hesitant to expand the scope of this check
without a discussion.

## Test Plan

- Manually confirmed each example file from the issue functioned as
expected.
- Ran cargo nextest to ensure `unexpected_special_method_signature` test
still passed.

Fixes #16217.
2025-02-21 08:38:51 -05:00
Alex Waygood
b3c5932fda [red-knot] Restrict visibility of the module_type_symbols function (#16290) 2025-02-21 10:55:22 +00:00
Alex Waygood
fe3ae587ea [red-knot] Fix subtle detail in where the types.ModuleType attribute lookup should happen in TypeInferenceBuilder::infer_name_load() (#16284) 2025-02-21 10:48:52 +00:00
Dhruv Manilawala
c2b9fa84f7 Refactor workspace logic into workspace.rs (#16295)
## Summary

This is just a small refactor to move workspace related structs and impl
out from `server.rs` where `Server` is defined and into a new
`workspace.rs`.
2025-02-21 08:37:29 +00:00
Victorien
793264db13 [ruff] Add more Pydantic models variants to the list of default copy semantics (RUF012) (#16291) 2025-02-21 08:28:13 +01:00
Carl Meyer
4d63c16c19 [red-knot] update to latest Salsa (#16293)
Update to latest Salsa main branch. This provides a point of comparison
for the perf impact of fixpoint iteration, which is based on latest
Salsa main.

This requires an update to the locked version of our boxcar dep, since
Salsa now depends on a newer version of boxcar.
2025-02-20 18:15:58 -08:00
David Peter
d2e034adcd [red-knot] Method calls and the descriptor protocol (#16121)
## Summary

This PR achieves the following:

* Add support for checking method calls, and inferring return types from
method calls. For example:
  ```py
  reveal_type("abcde".find("abc"))  # revealed: int
  reveal_type("foo".encode(encoding="utf-8"))  # revealed: bytes
  
  "abcde".find(123)  # error: [invalid-argument-type]
  
  class C:
      def f(self) -> int:
          pass
  
  reveal_type(C.f)  # revealed: <function `f`>
  reveal_type(C().f)  # revealed: <bound method: `f` of `C`>
  
  C.f()  # error: [missing-argument]
  reveal_type(C().f())  # revealed: int
  ```
* Implement the descriptor protocol, i.e. properly call the `__get__`
method when a descriptor object is accessed through a class object or an
instance of a class. For example:
  ```py
  from typing import Literal
  
  class Ten:
def __get__(self, instance: object, owner: type | None = None) ->
Literal[10]:
          return 10
  
  class C:
      ten: Ten = Ten()
  
  reveal_type(C.ten)  # revealed: Literal[10]
  reveal_type(C().ten)  # revealed: Literal[10]
  ```
* Add support for member lookup on intersection types.
* Support type inference for `inspect.getattr_static(obj, attr)` calls.
This was mostly used as a debugging tool during development, but seems
more generally useful. It can be used to bypass the descriptor protocol.
For the example above:
  ```py
  from inspect import getattr_static
  
  reveal_type(getattr_static(C, "ten"))  # revealed: Ten
  ```
* Add a new `Type::Callable(…)` variant with the following sub-variants:
* `Type::Callable(CallableType::BoundMethod(…))` — represents bound
method objects, e.g. `C().f` above
* `Type::Callable(CallableType::MethodWrapperDunderGet(…))` — represents
`f.__get__` where `f` is a function
* `Type::Callable(WrapperDescriptorDunderGet)` — represents
`FunctionType.__get__`
* Add new known classes:
  * `types.MethodType`
  * `types.MethodWrapperType`
  * `types.WrapperDescriptorType`
  * `builtins.range`

## Performance analysis

On this branch, we do more work. We need to do more call checking, since
we now check all method calls. We also need to do ~twice as many member
lookups, because we need to check if a `__get__` attribute exists on
accessed members.

A brief analysis on `tomllib` shows that we now call `Type::call` 1780
times, compared to 612 calls before.

## Limitations

* Data descriptors are not yet supported, i.e. we do not infer correct
types for descriptor attribute accesses in `Store` context and do not
check writes to descriptor attributes. I felt like this was something
that could be split out as a follow-up without risking a major
architectural change.
* We currently distinguish between `Type::member` (with descriptor
protocol) and `Type::static_member` (without descriptor protocol). The
former corresponds to `obj.attr`, the latter corresponds to
`getattr_static(obj, "attr")`. However, to model some details correctly,
we would also need to distinguish between a static member lookup *with*
and *without* instance variables. The lookup without instance variables
corresponds to `find_name_in_mro`
[here](https://docs.python.org/3/howto/descriptor.html#invocation-from-an-instance).
We currently approximate both using `member_static`, which leads to two
open TODOs. Changing this would be a larger refactoring of
`Type::own_instance_member`, so I chose to leave it out of this PR.

## Test Plan

* New `call/methods.md` test suite for method calls
* New tests in `descriptor_protocol.md`
* New `call/getattr_static.md` test suite for `inspect.getattr_static`
* Various updated tests
2025-02-20 23:22:26 +01:00
David Peter
f62e5406f2 [red-knot] Short-circuit bool calls on bool (#16292)
## Summary

This avoids looking up `__bool__` on class `bool` for every
`Type::Instance(bool).bool()` call. 1% performance win on cold cache, 4%
win on incremental performance.
2025-02-20 23:06:11 +01:00
Douglas Creager
1be4394155 [red-knot] Consolidate SymbolBindings/SymbolDeclarations state (#16286)
This updates the `SymbolBindings` and `SymbolDeclarations` types to use
a single smallvec of live bindings/declarations, instead of splitting
that out into separate containers for each field.

I'm seeing an 11-13% `cargo bench` performance improvement with this
locally (for both cold and incremental). I'm interested to see if
Codspeed agrees!

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2025-02-20 16:20:23 -05:00
Micha Reiser
470f852f04 [red-knot] Prevent cross-module query dependencies in own_instance_member (#16268) 2025-02-20 18:46:45 +01:00
51 changed files with 2931 additions and 1094 deletions

148
Cargo.lock generated
View File

@@ -8,18 +8,6 @@ version = "2.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "512761e0bb2578dd7380c6baaa0f4ce03e84f95e960231d1dec8bf4d7d6e2627"
[[package]]
name = "ahash"
version = "0.8.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e89da841a80418a9b391ebaea17f5c112ffaaa96f621d2c285b5174da76b9011"
dependencies = [
"cfg-if",
"once_cell",
"version_check",
"zerocopy 0.7.35",
]
[[package]]
name = "aho-corasick"
version = "1.1.3"
@@ -140,12 +128,6 @@ version = "1.0.95"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "34ac096ce696dc2fcabef30516bb13c0a68a11d30131d3df6f04711467681b04"
[[package]]
name = "append-only-vec"
version = "0.1.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7992085ec035cfe96992dd31bfd495a2ebd31969bb95f624471cb6c0b349e571"
[[package]]
name = "arc-swap"
version = "1.7.1"
@@ -227,9 +209,12 @@ dependencies = [
[[package]]
name = "boxcar"
version = "0.2.8"
version = "0.2.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2721c3c5a6f0e7f7e607125d963fedeb765f545f67adc9d71ed934693881eb42"
checksum = "225450ee9328e1e828319b48a89726cffc1b0ad26fd9211ad435de9fa376acae"
dependencies = [
"loom",
]
[[package]]
name = "bstr"
@@ -1013,6 +998,19 @@ dependencies = [
"libc",
]
[[package]]
name = "generator"
version = "0.8.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cc6bd114ceda131d3b1d665eba35788690ad37f5916457286b32ab6fd3c438dd"
dependencies = [
"cfg-if",
"libc",
"log",
"rustversion",
"windows",
]
[[package]]
name = "generic-array"
version = "0.14.7"
@@ -1102,10 +1100,6 @@ name = "hashbrown"
version = "0.14.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e5274423e17b7c9fc20b6e7e208532f9b19825d82dfd615708b70edd83df41f1"
dependencies = [
"ahash",
"allocator-api2",
]
[[package]]
name = "hashbrown"
@@ -1113,17 +1107,18 @@ version = "0.15.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bf151400ff0baff5465007dd2f3e717f3fe502074ca563069ce3a6629d07b289"
dependencies = [
"allocator-api2",
"equivalent",
"foldhash",
]
[[package]]
name = "hashlink"
version = "0.9.1"
version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6ba4ff7128dee98c7dc9794b6a411377e1404dba1c97deb8d1a55297bd25d8af"
checksum = "7382cf6263419f2d8df38c55d7da83da5c18aef87fc7a7fc1fb1e344edfe14c1"
dependencies = [
"hashbrown 0.14.5",
"hashbrown 0.15.2",
]
[[package]]
@@ -1179,7 +1174,7 @@ dependencies = [
"iana-time-zone-haiku",
"js-sys",
"wasm-bindgen",
"windows-core",
"windows-core 0.52.0",
]
[[package]]
@@ -1683,6 +1678,19 @@ version = "0.4.25"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "04cbf5b083de1c7e0222a7a51dbfdba1cbe1c6ab0b15e29fff3f6c077fd9cd9f"
[[package]]
name = "loom"
version = "0.7.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "419e0dc8046cb947daa77eb95ae174acfbddb7673b4151f56d1eed8e93fbfaca"
dependencies = [
"cfg-if",
"generator",
"scoped-tls",
"tracing",
"tracing-subscriber",
]
[[package]]
name = "lsp-server"
version = "0.7.8"
@@ -2814,6 +2822,7 @@ dependencies = [
"ruff_python_formatter",
"ruff_python_parser",
"ruff_python_trivia",
"ruff_server",
"ruff_workspace",
"schemars",
"serde",
@@ -3164,6 +3173,7 @@ dependencies = [
"ruff_diagnostics",
"ruff_formatter",
"ruff_linter",
"ruff_macros",
"ruff_notebook",
"ruff_python_ast",
"ruff_python_codegen",
@@ -3315,14 +3325,14 @@ checksum = "6ea1a2d0a644769cc99faa24c3ad26b379b786fe7c36fd3c546254801650e6dd"
[[package]]
name = "salsa"
version = "0.18.0"
source = "git+https://github.com/salsa-rs/salsa.git?rev=351d9cf0037be949d17800d0c7b4838e533c2ed6#351d9cf0037be949d17800d0c7b4838e533c2ed6"
source = "git+https://github.com/salsa-rs/salsa.git?rev=c8826fa4d1d9e3cba4c6e578763878b71fa9a10d#c8826fa4d1d9e3cba4c6e578763878b71fa9a10d"
dependencies = [
"append-only-vec",
"arc-swap",
"boxcar",
"compact_str",
"crossbeam",
"crossbeam-queue",
"dashmap 6.1.0",
"hashbrown 0.14.5",
"hashbrown 0.15.2",
"hashlink",
"indexmap",
"parking_lot",
@@ -3337,12 +3347,12 @@ dependencies = [
[[package]]
name = "salsa-macro-rules"
version = "0.1.0"
source = "git+https://github.com/salsa-rs/salsa.git?rev=351d9cf0037be949d17800d0c7b4838e533c2ed6#351d9cf0037be949d17800d0c7b4838e533c2ed6"
source = "git+https://github.com/salsa-rs/salsa.git?rev=c8826fa4d1d9e3cba4c6e578763878b71fa9a10d#c8826fa4d1d9e3cba4c6e578763878b71fa9a10d"
[[package]]
name = "salsa-macros"
version = "0.18.0"
source = "git+https://github.com/salsa-rs/salsa.git?rev=351d9cf0037be949d17800d0c7b4838e533c2ed6#351d9cf0037be949d17800d0c7b4838e533c2ed6"
source = "git+https://github.com/salsa-rs/salsa.git?rev=c8826fa4d1d9e3cba4c6e578763878b71fa9a10d#c8826fa4d1d9e3cba4c6e578763878b71fa9a10d"
dependencies = [
"heck",
"proc-macro2",
@@ -3384,6 +3394,12 @@ dependencies = [
"syn 2.0.98",
]
[[package]]
name = "scoped-tls"
version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e1cf6437eb19a8f4a6cc0f7dca544973b0b78843adbfeb3683d1a94a0024a294"
[[package]]
name = "scopeguard"
version = "1.2.0"
@@ -4451,6 +4467,16 @@ version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f"
[[package]]
name = "windows"
version = "0.58.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dd04d41d93c4992d421894c18c8b43496aa748dd4c081bac0dc93eb0489272b6"
dependencies = [
"windows-core 0.58.0",
"windows-targets 0.52.6",
]
[[package]]
name = "windows-core"
version = "0.52.0"
@@ -4460,6 +4486,60 @@ dependencies = [
"windows-targets 0.52.6",
]
[[package]]
name = "windows-core"
version = "0.58.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6ba6d44ec8c2591c134257ce647b7ea6b20335bf6379a27dac5f1641fcf59f99"
dependencies = [
"windows-implement",
"windows-interface",
"windows-result",
"windows-strings",
"windows-targets 0.52.6",
]
[[package]]
name = "windows-implement"
version = "0.58.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2bbd5b46c938e506ecbce286b6628a02171d56153ba733b6c741fc627ec9579b"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.98",
]
[[package]]
name = "windows-interface"
version = "0.58.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "053c4c462dc91d3b1504c6fe5a726dd15e216ba718e84a0e46a88fbe5ded3515"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.98",
]
[[package]]
name = "windows-result"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1d1043d8214f791817bab27572aaa8af63732e11bf84aa21a45a78d6c317ae0e"
dependencies = [
"windows-targets 0.52.6",
]
[[package]]
name = "windows-strings"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4cd9b125c486025df0eabcb585e62173c6c9eddcec5d117d3b6e8c30e2ee4d10"
dependencies = [
"windows-result",
"windows-targets 0.52.6",
]
[[package]]
name = "windows-sys"
version = "0.48.0"

View File

@@ -123,7 +123,7 @@ rayon = { version = "1.10.0" }
regex = { version = "1.10.2" }
rustc-hash = { version = "2.0.0" }
# When updating salsa, make sure to also update the revision in `fuzz/Cargo.toml`
salsa = { git = "https://github.com/salsa-rs/salsa.git", rev = "351d9cf0037be949d17800d0c7b4838e533c2ed6" }
salsa = { git = "https://github.com/salsa-rs/salsa.git", rev = "c8826fa4d1d9e3cba4c6e578763878b71fa9a10d" }
schemars = { version = "0.8.16" }
seahash = { version = "4.1.0" }
serde = { version = "1.0.197", features = ["derive"] }

View File

@@ -73,12 +73,12 @@ qux = (foo, bar)
reveal_type(qux) # revealed: tuple[Literal["foo"], Literal["bar"]]
# TODO: Infer "LiteralString"
reveal_type(foo.join(qux)) # revealed: @Todo(Attribute access on `StringLiteral` types)
reveal_type(foo.join(qux)) # revealed: @Todo(decorated method)
template: LiteralString = "{}, {}"
reveal_type(template) # revealed: Literal["{}, {}"]
# TODO: Infer `LiteralString`
reveal_type(template.format(foo, bar)) # revealed: @Todo(Attribute access on `StringLiteral` types)
reveal_type(template.format(foo, bar)) # revealed: @Todo(decorated method)
```
### Assignability

View File

@@ -1005,8 +1005,8 @@ reveal_type(f.__kwdefaults__) # revealed: @Todo(generics) | None
Some attributes are special-cased, however:
```py
reveal_type(f.__get__) # revealed: @Todo(`__get__` method on functions)
reveal_type(f.__call__) # revealed: @Todo(`__call__` method on functions)
reveal_type(f.__get__) # revealed: <method-wrapper `__get__` of `f`>
reveal_type(f.__call__) # revealed: <bound method `__call__` of `Literal[f]`>
```
### Int-literal attributes
@@ -1015,7 +1015,7 @@ Most attribute accesses on int-literal types are delegated to `builtins.int`, si
integers are instances of that class:
```py
reveal_type((2).bit_length) # revealed: @Todo(bound method)
reveal_type((2).bit_length) # revealed: <bound method `bit_length` of `Literal[2]`>
reveal_type((2).denominator) # revealed: @Todo(@property)
```
@@ -1029,11 +1029,11 @@ reveal_type((2).real) # revealed: Literal[2]
### Bool-literal attributes
Most attribute accesses on bool-literal types are delegated to `builtins.bool`, since all literal
bols are instances of that class:
bools are instances of that class:
```py
reveal_type(True.__and__) # revealed: @Todo(bound method)
reveal_type(False.__or__) # revealed: @Todo(bound method)
reveal_type(True.__and__) # revealed: @Todo(decorated method)
reveal_type(False.__or__) # revealed: @Todo(decorated method)
```
Some attributes are special-cased, however:
@@ -1045,11 +1045,11 @@ reveal_type(False.real) # revealed: Literal[0]
### Bytes-literal attributes
All attribute access on literal `bytes` types is currently delegated to `buitins.bytes`:
All attribute access on literal `bytes` types is currently delegated to `builtins.bytes`:
```py
reveal_type(b"foo".join) # revealed: @Todo(bound method)
reveal_type(b"foo".endswith) # revealed: @Todo(bound method)
reveal_type(b"foo".join) # revealed: <bound method `join` of `Literal[b"foo"]`>
reveal_type(b"foo".endswith) # revealed: <bound method `endswith` of `Literal[b"foo"]`>
```
## Instance attribute edge cases
@@ -1136,6 +1136,40 @@ class C:
reveal_type(C().x) # revealed: Unknown
```
### Builtin types attributes
This test can probably be removed eventually, but we currently include it because we do not yet
understand generic bases and protocols, and we want to make sure that we can still use builtin types
in our tests in the meantime. See the corresponding TODO in `Type::static_member` for more
information.
```py
class C:
a_int: int = 1
a_str: str = "a"
a_bytes: bytes = b"a"
a_bool: bool = True
a_float: float = 1.0
a_complex: complex = 1 + 1j
a_tuple: tuple[int] = (1,)
a_range: range = range(1)
a_slice: slice = slice(1)
a_type: type = int
a_none: None = None
reveal_type(C.a_int) # revealed: int
reveal_type(C.a_str) # revealed: str
reveal_type(C.a_bytes) # revealed: bytes
reveal_type(C.a_bool) # revealed: bool
reveal_type(C.a_float) # revealed: int | float
reveal_type(C.a_complex) # revealed: int | float | complex
reveal_type(C.a_tuple) # revealed: tuple[int]
reveal_type(C.a_range) # revealed: range
reveal_type(C.a_slice) # revealed: slice
reveal_type(C.a_type) # revealed: type
reveal_type(C.a_none) # revealed: None
```
## References
Some of the tests in the *Class and instance variables* section draw inspiration from

View File

@@ -0,0 +1,133 @@
# `inspect.getattr_static`
## Basic usage
`inspect.getattr_static` is a function that returns attributes of an object without invoking the
descriptor protocol (for caveats, see the [official documentation]).
Consider the following example:
```py
import inspect
class Descriptor:
def __get__(self, instance, owner) -> str:
return 1
class C:
normal: int = 1
descriptor: Descriptor = Descriptor()
```
If we access attributes on an instance of `C` as usual, the descriptor protocol is invoked, and we
get a type of `str` for the `descriptor` attribute:
```py
c = C()
reveal_type(c.normal) # revealed: int
reveal_type(c.descriptor) # revealed: str
```
However, if we use `inspect.getattr_static`, we can see the underlying `Descriptor` type:
```py
reveal_type(inspect.getattr_static(c, "normal")) # revealed: int
reveal_type(inspect.getattr_static(c, "descriptor")) # revealed: Descriptor
```
For non-existent attributes, a default value can be provided:
```py
reveal_type(inspect.getattr_static(C, "normal", "default-arg")) # revealed: int
reveal_type(inspect.getattr_static(C, "non_existent", "default-arg")) # revealed: Literal["default-arg"]
```
When a non-existent attribute is accessed without a default value, the runtime raises an
`AttributeError`. We could emit a diagnostic for this case, but that is currently not supported:
```py
# TODO: we could emit a diagnostic here
reveal_type(inspect.getattr_static(C, "non_existent")) # revealed: Never
```
We can access attributes on objects of all kinds:
```py
import sys
reveal_type(inspect.getattr_static(sys, "platform")) # revealed: LiteralString
reveal_type(inspect.getattr_static(inspect, "getattr_static")) # revealed: Literal[getattr_static]
reveal_type(inspect.getattr_static(1, "real")) # revealed: Literal[1]
```
(Implicit) instance attributes can also be accessed through `inspect.getattr_static`:
```py
class D:
def __init__(self) -> None:
self.instance_attr: int = 1
reveal_type(inspect.getattr_static(D(), "instance_attr")) # revealed: int
```
## Error cases
We can only infer precise types if the attribute is a literal string. In all other cases, we fall
back to `Any`:
```py
import inspect
class C:
x: int = 1
def _(attr_name: str):
reveal_type(inspect.getattr_static(C(), attr_name)) # revealed: Any
reveal_type(inspect.getattr_static(C(), attr_name, 1)) # revealed: Any
```
But we still detect errors in the number or type of arguments:
```py
# error: [missing-argument] "No arguments provided for required parameters `obj`, `attr` of function `getattr_static`"
inspect.getattr_static()
# error: [missing-argument] "No argument provided for required parameter `attr`"
inspect.getattr_static(C())
# error: [invalid-argument-type] "Object of type `Literal[1]` cannot be assigned to parameter 2 (`attr`) of function `getattr_static`; expected type `str`"
inspect.getattr_static(C(), 1)
# error: [too-many-positional-arguments] "Too many positional arguments to function `getattr_static`: expected 3, got 4"
inspect.getattr_static(C(), "x", "default-arg", "one too many")
```
## Possibly unbound attributes
```py
import inspect
def _(flag: bool):
class C:
if flag:
x: int = 1
reveal_type(inspect.getattr_static(C, "x", "default")) # revealed: int | Literal["default"]
```
## Gradual types
```py
import inspect
from typing import Any
def _(a: Any, tuple_of_any: tuple[Any]):
reveal_type(inspect.getattr_static(a, "x", "default")) # revealed: Any | Literal["default"]
# TODO: Ideally, this would just be `Literal[index]`
reveal_type(inspect.getattr_static(tuple_of_any, "index", "default")) # revealed: Literal[index] | Literal["default"]
```
[official documentation]: https://docs.python.org/3/library/inspect.html#inspect.getattr_static

View File

@@ -0,0 +1,258 @@
# Methods
## Background: Functions as descriptors
> Note: See also this related section in the descriptor guide: [Functions and methods].
Say we have a simple class `C` with a function definition `f` inside its body:
```py
class C:
def f(self, x: int) -> str:
return "a"
```
Whenever we access the `f` attribute through the class object itself (`C.f`) or through an instance
(`C().f`), this access happens via the descriptor protocol. Functions are (non-data) descriptors
because they implement a `__get__` method. This is crucial in making sure that method calls work as
expected. In general, the signature of the `__get__` method in the descriptor protocol is
`__get__(self, instance, owner)`. The `self` argument is the descriptor object itself (`f`). The
passed value for the `instance` argument depends on whether the attribute is accessed from the class
object (in which case it is `None`), or from an instance (in which case it is the instance of type
`C`). The `owner` argument is the class itself (`C` of type `Literal[C]`). To summarize:
- `C.f` is equivalent to `getattr_static(C, "f").__get__(None, C)`
- `C().f` is equivalent to `getattr_static(C, "f").__get__(C(), C)`
Here, `inspect.getattr_static` is used to bypass the descriptor protocol and directly access the
function attribute. The way the special `__get__` method *on functions* works is as follows. In the
former case, if the `instance` argument is `None`, `__get__` simply returns the function itself. In
the latter case, it returns a *bound method* object:
```py
from inspect import getattr_static
reveal_type(getattr_static(C, "f")) # revealed: Literal[f]
reveal_type(getattr_static(C, "f").__get__) # revealed: <method-wrapper `__get__` of `f`>
reveal_type(getattr_static(C, "f").__get__(None, C)) # revealed: Literal[f]
reveal_type(getattr_static(C, "f").__get__(C(), C)) # revealed: <bound method `f` of `C`>
```
In conclusion, this is why we see the following two types when accessing the `f` attribute on the
class object `C` and on an instance `C()`:
```py
reveal_type(C.f) # revealed: Literal[f]
reveal_type(C().f) # revealed: <bound method `f` of `C`>
```
A bound method is a callable object that contains a reference to the `instance` that it was called
on (can be inspected via `__self__`), and the function object that it refers to (can be inspected
via `__func__`):
```py
bound_method = C().f
reveal_type(bound_method.__self__) # revealed: C
reveal_type(bound_method.__func__) # revealed: Literal[f]
```
When we call the bound method, the `instance` is implicitly passed as the first argument (`self`):
```py
reveal_type(C().f(1)) # revealed: str
reveal_type(bound_method(1)) # revealed: str
```
When we call the function object itself, we need to pass the `instance` explicitly:
```py
C.f(1) # error: [missing-argument]
reveal_type(C.f(C(), 1)) # revealed: str
```
When we access methods from derived classes, they will be bound to instances of the derived class:
```py
class D(C):
pass
reveal_type(D().f) # revealed: <bound method `f` of `D`>
```
If we access an attribute on a bound method object itself, it will defer to `types.MethodType`:
```py
reveal_type(bound_method.__hash__) # revealed: <bound method `__hash__` of `MethodType`>
```
If an attribute is not available on the bound method object, it will be looked up on the underlying
function object. We model this explicitly, which means that we can access `__kwdefaults__` on bound
methods, even though it is not available on `types.MethodType`:
```py
reveal_type(bound_method.__kwdefaults__) # revealed: @Todo(generics) | None
```
## Basic method calls on class objects and instances
```py
class Base:
def method_on_base(self, x: int | None) -> str:
return "a"
class Derived(Base):
def method_on_derived(self, x: bytes) -> tuple[int, str]:
return (1, "a")
reveal_type(Base().method_on_base(1)) # revealed: str
reveal_type(Base.method_on_base(Base(), 1)) # revealed: str
Base().method_on_base("incorrect") # error: [invalid-argument-type]
Base().method_on_base() # error: [missing-argument]
Base().method_on_base(1, 2) # error: [too-many-positional-arguments]
reveal_type(Derived().method_on_base(1)) # revealed: str
reveal_type(Derived().method_on_derived(b"abc")) # revealed: tuple[int, str]
reveal_type(Derived.method_on_base(Derived(), 1)) # revealed: str
reveal_type(Derived.method_on_derived(Derived(), b"abc")) # revealed: tuple[int, str]
```
## Method calls on literals
### Boolean literals
```py
reveal_type(True.bit_length()) # revealed: int
reveal_type(True.as_integer_ratio()) # revealed: tuple[int, Literal[1]]
```
### Integer literals
```py
reveal_type((42).bit_length()) # revealed: int
```
### String literals
```py
reveal_type("abcde".find("abc")) # revealed: int
reveal_type("foo".encode(encoding="utf-8")) # revealed: bytes
"abcde".find(123) # error: [invalid-argument-type]
```
### Bytes literals
```py
reveal_type(b"abcde".startswith(b"abc")) # revealed: bool
```
## Method calls on `LiteralString`
```py
from typing_extensions import LiteralString
def f(s: LiteralString) -> None:
reveal_type(s.find("a")) # revealed: int
```
## Method calls on `tuple`
```py
def f(t: tuple[int, str]) -> None:
reveal_type(t.index("a")) # revealed: int
```
## Method calls on unions
```py
from typing import Any
class A:
def f(self) -> int:
return 1
class B:
def f(self) -> str:
return "a"
def f(a_or_b: A | B, any_or_a: Any | A):
reveal_type(a_or_b.f) # revealed: <bound method `f` of `A`> | <bound method `f` of `B`>
reveal_type(a_or_b.f()) # revealed: int | str
reveal_type(any_or_a.f) # revealed: Any | <bound method `f` of `A`>
reveal_type(any_or_a.f()) # revealed: Any | int
```
## Method calls on `KnownInstance` types
```toml
[environment]
python-version = "3.12"
```
```py
type IntOrStr = int | str
reveal_type(IntOrStr.__or__) # revealed: <bound method `__or__` of `typing.TypeAliasType`>
```
## Error cases: Calling `__get__` for methods
The `__get__` method on `types.FunctionType` has the following overloaded signature in typeshed:
```py
from types import FunctionType, MethodType
from typing import overload
@overload
def __get__(self, instance: None, owner: type, /) -> FunctionType: ...
@overload
def __get__(self, instance: object, owner: type | None = None, /) -> MethodType: ...
```
Here, we test that this signature is enforced correctly:
```py
from inspect import getattr_static
class C:
def f(self, x: int) -> str:
return "a"
method_wrapper = getattr_static(C, "f").__get__
reveal_type(method_wrapper) # revealed: <method-wrapper `__get__` of `f`>
# All of these are fine:
method_wrapper(C(), C)
method_wrapper(C())
method_wrapper(C(), None)
method_wrapper(None, C)
# Passing `None` without an `owner` argument is an
# error: [missing-argument] "No argument provided for required parameter `owner`"
method_wrapper(None)
# Passing something that is not assignable to `type` as the `owner` argument is an
# error: [invalid-argument-type] "Object of type `Literal[1]` cannot be assigned to parameter 2 (`owner`); expected type `type`"
method_wrapper(None, 1)
# Passing `None` as the `owner` argument when `instance` is `None` is an
# error: [invalid-argument-type] "Object of type `None` cannot be assigned to parameter 2 (`owner`); expected type `type`"
method_wrapper(None, None)
# Calling `__get__` without any arguments is an
# error: [missing-argument] "No argument provided for required parameter `instance`"
method_wrapper()
# Calling `__get__` with too many positional arguments is an
# error: [too-many-positional-arguments] "Too many positional arguments: expected 2, got 3"
method_wrapper(C(), C, "one too many")
```
[functions and methods]: https://docs.python.org/3/howto/descriptor.html#functions-and-methods

View File

@@ -22,22 +22,26 @@ class Ten:
pass
class C:
ten = Ten()
ten: Ten = Ten()
c = C()
# TODO: this should be `Literal[10]`
reveal_type(c.ten) # revealed: Unknown | Ten
reveal_type(c.ten) # revealed: Literal[10]
# TODO: This should `Literal[10]`
reveal_type(C.ten) # revealed: Unknown | Ten
reveal_type(C.ten) # revealed: Literal[10]
# These are fine:
c.ten = 10
# TODO: This should not be an error
c.ten = 10 # error: [invalid-assignment]
C.ten = 10
# TODO: Both of these should be errors
# TODO: This should be an error (as the wrong type is being implicitly passed to `Ten.__set__`),
# but the error message is misleading.
# error: [invalid-assignment] "Object of type `Literal[11]` is not assignable to attribute `ten` of type `Ten`"
c.ten = 11
# TODO: same as above
# error: [invalid-assignment] "Object of type `Literal[11]` is not assignable to attribute `ten` of type `Literal[10]`"
C.ten = 11
```
@@ -57,24 +61,86 @@ class FlexibleInt:
self._value = int(value)
class C:
flexible_int = FlexibleInt()
flexible_int: FlexibleInt = FlexibleInt()
c = C()
# TODO: should be `int | None`
reveal_type(c.flexible_int) # revealed: Unknown | FlexibleInt
reveal_type(c.flexible_int) # revealed: int | None
# TODO: These should not be errors
# error: [invalid-assignment]
c.flexible_int = 42 # okay
# error: [invalid-assignment]
c.flexible_int = "42" # also okay!
# TODO: should be `int | None`
reveal_type(c.flexible_int) # revealed: Unknown | FlexibleInt
reveal_type(c.flexible_int) # revealed: int | None
# TODO: should be an error
# TODO: This should be an error, but the message needs to be improved.
# error: [invalid-assignment] "Object of type `None` is not assignable to attribute `flexible_int` of type `FlexibleInt`"
c.flexible_int = None # not okay
# TODO: should be `int | None`
reveal_type(c.flexible_int) # revealed: Unknown | FlexibleInt
reveal_type(c.flexible_int) # revealed: int | None
```
## Data and non-data descriptors
Descriptors that define `__set__` or `__delete__` are called *data descriptors*. An example\
of a data descriptor is a `property` with a setter and/or a deleter.\
Descriptors that only define `__get__`, meanwhile, are called *non-data descriptors*. Examples
include\
functions, `classmethod` or `staticmethod`).
The precedence chain for attribute access is (1) data descriptors, (2) instance attributes, and (3)
non-data descriptors.
```py
from typing import Literal
class DataDescriptor:
def __get__(self, instance: object, owner: type | None = None) -> Literal["data"]:
return "data"
def __set__(self, instance: int, value) -> None:
pass
class NonDataDescriptor:
def __get__(self, instance: object, owner: type | None = None) -> Literal["non-data"]:
return "non-data"
class C:
data_descriptor = DataDescriptor()
non_data_descriptor = NonDataDescriptor()
def f(self):
# This explains why data descriptors come first in the precedence chain. If
# instance attributes would take priority, we would override the descriptor
# here. Instead, this calls `DataDescriptor.__set__`, i.e. it does not affect
# the type of the `data_descriptor` attribute.
self.data_descriptor = 1
# However, for non-data descriptors, instance attributes do take precedence.
# So it is possible to override them.
self.non_data_descriptor = 1
c = C()
# TODO: This should ideally be `Unknown | Literal["data"]`.
#
# - Pyright also wrongly shows `int | Literal['data']` here
# - Mypy shows Literal["data"] here, but also shows Literal["non-data"] below.
#
reveal_type(c.data_descriptor) # revealed: Unknown | Literal["data", 1]
reveal_type(c.non_data_descriptor) # revealed: Unknown | Literal["non-data", 1]
reveal_type(C.data_descriptor) # revealed: Unknown | Literal["data"]
reveal_type(C.non_data_descriptor) # revealed: Unknown | Literal["non-data"]
# It is possible to override data descriptors via class objects. The following
# assignment does not call `DataDescriptor.__set__`. For this reason, we infer
# `Unknown | …` for all (descriptor) attributes.
C.data_descriptor = "something else" # This is okay
```
## Built-in `property` descriptor
@@ -101,7 +167,7 @@ c = C()
reveal_type(c._name) # revealed: str | None
# Should be `str`
reveal_type(c.name) # revealed: @Todo(bound method)
reveal_type(c.name) # revealed: @Todo(decorated method)
# Should be `builtins.property`
reveal_type(C.name) # revealed: Literal[name]
@@ -142,7 +208,7 @@ reveal_type(c1) # revealed: @Todo(return type)
reveal_type(C.get_name()) # revealed: @Todo(return type)
# TODO: should be `str`
reveal_type(C("42").get_name()) # revealed: @Todo(bound method)
reveal_type(C("42").get_name()) # revealed: @Todo(decorated method)
```
## Descriptors only work when used as class variables
@@ -160,9 +226,10 @@ class Ten:
class C:
def __init__(self):
self.ten = Ten()
self.ten: Ten = Ten()
reveal_type(C().ten) # revealed: Unknown | Ten
# TODO: Should be Ten
reveal_type(C().ten) # revealed: Literal[10]
```
## Descriptors distinguishing between class and instance access
@@ -186,13 +253,166 @@ class Descriptor:
return "called on class object"
class C:
d = Descriptor()
d: Descriptor = Descriptor()
# TODO: should be `Literal["called on class object"]
reveal_type(C.d) # revealed: Unknown | Descriptor
reveal_type(C.d) # revealed: LiteralString
# TODO: should be `Literal["called on instance"]
reveal_type(C().d) # revealed: Unknown | Descriptor
reveal_type(C().d) # revealed: LiteralString
```
## Undeclared descriptor arguments
If a descriptor attribute is not declared, we union with `Unknown`, just like for regular
attributes, since that attribute could be overwritten externally. Even a data descriptor with a
`__set__` method can be overwritten when accessed through a class object.
```py
class Descriptor:
def __get__(self, instance: object, owner: type | None = None) -> int:
return 1
def __set__(self, instance: object, value: int) -> None:
pass
class C:
descriptor = Descriptor()
C.descriptor = "something else"
# This could also be `Literal["something else"]` if we support narrowing of attribute types based on assignments
reveal_type(C.descriptor) # revealed: Unknown | int
```
## Descriptors with incorrect `__get__` signature
```py
class Descriptor:
# `__get__` method with missing parameters:
def __get__(self) -> int:
return 1
class C:
descriptor: Descriptor = Descriptor()
# TODO: This should be an error
reveal_type(C.descriptor) # revealed: Descriptor
```
## Possibly-unbound `__get__` method
```py
def _(flag: bool):
class MaybeDescriptor:
if flag:
def __get__(self, instance: object, owner: type | None = None) -> int:
return 1
class C:
descriptor: MaybeDescriptor = MaybeDescriptor()
# TODO: This should be `MaybeDescriptor | int`
reveal_type(C.descriptor) # revealed: int
```
## Dunder methods
Dunder methods are looked up on the meta type, but we still need to invoke the descriptor protocol:
```py
class SomeCallable:
def __call__(self, x: int) -> str:
return "a"
class Descriptor:
def __get__(self, instance: object, owner: type | None = None) -> SomeCallable:
return SomeCallable()
class B:
__call__: Descriptor = Descriptor()
b_instance = B()
reveal_type(b_instance(1)) # revealed: str
b_instance("bla") # error: [invalid-argument-type]
```
## Functions as descriptors
Functions are descriptors because they implement a `__get__` method. This is crucial in making sure
that method calls work as expected. See [this test suite](./call/methods.md) for more information.
Here, we only demonstrate how `__get__` works on functions:
```py
from inspect import getattr_static
def f(x: object) -> str:
return "a"
reveal_type(f) # revealed: Literal[f]
reveal_type(f.__get__) # revealed: <method-wrapper `__get__` of `f`>
reveal_type(f.__get__(None, type(f))) # revealed: Literal[f]
reveal_type(f.__get__(None, type(f))(1)) # revealed: str
wrapper_descriptor = getattr_static(f, "__get__")
reveal_type(wrapper_descriptor) # revealed: <wrapper-descriptor `__get__` of `function` objects>
reveal_type(wrapper_descriptor(f, None, type(f))) # revealed: Literal[f]
# Attribute access on the method-wrapper `f.__get__` falls back to `MethodWrapperType`:
reveal_type(f.__get__.__hash__) # revealed: <bound method `__hash__` of `MethodWrapperType`>
# Attribute access on the wrapper-descriptor falls back to `WrapperDescriptorType`:
reveal_type(wrapper_descriptor.__qualname__) # revealed: @Todo(@property)
```
We can also bind the free function `f` to an instance of a class `C`:
```py
class C: ...
bound_method = wrapper_descriptor(f, C(), C)
reveal_type(bound_method) # revealed: <bound method `f` of `C`>
```
We can then call it, and the instance of `C` is implicitly passed to the first parameter of `f`
(`x`):
```py
reveal_type(bound_method()) # revealed: str
```
Finally, we test some error cases for the call to the wrapper descriptor:
```py
# Calling the wrapper descriptor without any arguments is an
# error: [missing-argument] "No arguments provided for required parameters `self`, `instance`"
wrapper_descriptor()
# Calling it without the `instance` argument is an also an
# error: [missing-argument] "No argument provided for required parameter `instance`"
wrapper_descriptor(f)
# Calling it without the `owner` argument if `instance` is not `None` is an
# error: [missing-argument] "No argument provided for required parameter `owner`"
wrapper_descriptor(f, None)
# But calling it with an instance is fine (in this case, the `owner` argument is optional):
wrapper_descriptor(f, C())
# Calling it with something that is not a `FunctionType` as the first argument is an
# error: [invalid-argument-type] "Object of type `Literal[1]` cannot be assigned to parameter 1 (`self`); expected type `FunctionType`"
wrapper_descriptor(1, None, type(f))
# Calling it with something that is not a `type` as the `owner` argument is an
# error: [invalid-argument-type] "Object of type `Literal[f]` cannot be assigned to parameter 3 (`owner`); expected type `type`"
wrapper_descriptor(f, None, f)
# Calling it with too many positional arguments is an
# error: [too-many-positional-arguments] "Too many positional arguments: expected 3, got 4"
wrapper_descriptor(f, None, type(f), "one too many")
```
[descriptors]: https://docs.python.org/3/howto/descriptor.html

View File

@@ -0,0 +1,15 @@
# Protocols
We do not support protocols yet, but to avoid false positives, we *partially* support some known
protocols.
## `typing.SupportsIndex`
```py
from typing import SupportsIndex, Literal
def _(some_int: int, some_literal_int: Literal[1], some_indexable: SupportsIndex):
a: SupportsIndex = some_int
b: SupportsIndex = some_literal_int
c: SupportsIndex = some_indexable
```

View File

@@ -9,7 +9,7 @@ is unbound.
```py
reveal_type(__name__) # revealed: str
reveal_type(__file__) # revealed: str | None
reveal_type(__loader__) # revealed: LoaderProtocol | None
reveal_type(__loader__) # revealed: @Todo(instance attribute on class with dynamic base) | None
reveal_type(__package__) # revealed: str | None
reveal_type(__doc__) # revealed: str | None
@@ -54,10 +54,10 @@ inside the module:
import typing
reveal_type(typing.__name__) # revealed: str
reveal_type(typing.__init__) # revealed: @Todo(bound method)
reveal_type(typing.__init__) # revealed: <bound method `__init__` of `ModuleType`>
# These come from `builtins.object`, not `types.ModuleType`:
reveal_type(typing.__eq__) # revealed: @Todo(bound method)
reveal_type(typing.__eq__) # revealed: <bound method `__eq__` of `ModuleType`>
reveal_type(typing.__class__) # revealed: Literal[ModuleType]

View File

@@ -66,6 +66,6 @@ It is [recommended](https://docs.python.org/3/library/sys.html#sys.platform) to
```py
import sys
reveal_type(sys.platform.startswith("freebsd")) # revealed: @Todo(Attribute access on `LiteralString` types)
reveal_type(sys.platform.startswith("linux")) # revealed: @Todo(Attribute access on `LiteralString` types)
reveal_type(sys.platform.startswith("freebsd")) # revealed: bool
reveal_type(sys.platform.startswith("linux")) # revealed: bool
```

View File

@@ -35,7 +35,7 @@ in strict mode.
```py
def f(x: type):
reveal_type(x) # revealed: type
reveal_type(x.__repr__) # revealed: @Todo(bound method)
reveal_type(x.__repr__) # revealed: <bound method `__repr__` of `type`>
class A: ...
@@ -50,7 +50,7 @@ x: type = A() # error: [invalid-assignment]
```py
def f(x: type[object]):
reveal_type(x) # revealed: type
reveal_type(x.__repr__) # revealed: @Todo(bound method)
reveal_type(x.__repr__) # revealed: <bound method `__repr__` of `type`>
class A: ...

View File

@@ -75,3 +75,19 @@ class Boom:
reveal_type(bool(Boom())) # revealed: bool
```
### Possibly unbound __bool__ method
```py
from typing import Literal
def flag() -> bool:
return True
class PossiblyUnboundTrue:
if flag():
def __bool__(self) -> Literal[True]:
return True
reveal_type(bool(PossiblyUnboundTrue())) # revealed: bool
```

View File

@@ -109,6 +109,7 @@ pub enum KnownModule {
#[allow(dead_code)]
Abc, // currently only used in tests
Collections,
Inspect,
KnotExtensions,
}
@@ -123,6 +124,7 @@ impl KnownModule {
Self::Sys => "sys",
Self::Abc => "abc",
Self::Collections => "collections",
Self::Inspect => "inspect",
Self::KnotExtensions => "knot_extensions",
}
}
@@ -149,6 +151,7 @@ impl KnownModule {
"sys" => Some(Self::Sys),
"abc" => Some(Self::Abc),
"collections" => Some(Self::Collections),
"inspect" => Some(Self::Inspect),
"knot_extensions" => Some(Self::KnotExtensions),
_ => None,
}

View File

@@ -15,7 +15,7 @@ use crate::module_name::ModuleName;
use crate::semantic_index::ast_ids::node_key::ExpressionNodeKey;
use crate::semantic_index::ast_ids::AstIdsBuilder;
use crate::semantic_index::attribute_assignment::{AttributeAssignment, AttributeAssignments};
use crate::semantic_index::constraint::PatternConstraintKind;
use crate::semantic_index::constraint::{PatternConstraintKind, ScopedConstraintId};
use crate::semantic_index::definition::{
AssignmentDefinitionNodeRef, ComprehensionDefinitionNodeRef, Definition, DefinitionNodeKey,
DefinitionNodeRef, ForStmtDefinitionNodeRef, ImportFromDefinitionNodeRef,
@@ -26,7 +26,7 @@ use crate::semantic_index::symbol::{
SymbolTableBuilder,
};
use crate::semantic_index::use_def::{
EagerBindingsKey, FlowSnapshot, ScopedConstraintId, ScopedEagerBindingsId, UseDefMapBuilder,
EagerBindingsKey, FlowSnapshot, ScopedEagerBindingsId, UseDefMapBuilder,
};
use crate::semantic_index::SemanticIndex;
use crate::unpack::{Unpack, UnpackValue};
@@ -294,7 +294,7 @@ impl<'db> SemanticIndexBuilder<'db> {
&self.use_def_maps[scope_id]
}
fn current_visibility_constraints_mut(&mut self) -> &mut VisibilityConstraintsBuilder<'db> {
fn current_visibility_constraints_mut(&mut self) -> &mut VisibilityConstraintsBuilder {
let scope_id = self.current_scope();
&mut self.use_def_maps[scope_id].visibility_constraints
}
@@ -346,12 +346,14 @@ impl<'db> SemanticIndexBuilder<'db> {
// SAFETY: `definition_node` is guaranteed to be a child of `self.module`
let kind = unsafe { definition_node.into_owned(self.module.clone()) };
let category = kind.category();
let is_reexported = kind.is_reexported();
let definition = Definition::new(
self.db,
self.file,
self.current_scope(),
symbol,
kind,
is_reexported,
countme::Count::default(),
);
@@ -404,16 +406,12 @@ impl<'db> SemanticIndexBuilder<'db> {
}
/// Negates a constraint and adds it to the list of all constraints, does not record it.
fn add_negated_constraint(
&mut self,
constraint: Constraint<'db>,
) -> (Constraint<'db>, ScopedConstraintId) {
fn add_negated_constraint(&mut self, constraint: Constraint<'db>) -> ScopedConstraintId {
let negated = Constraint {
node: constraint.node,
is_positive: false,
};
let id = self.current_use_def_map_mut().add_constraint(negated);
(negated, id)
self.current_use_def_map_mut().add_constraint(negated)
}
/// Records a previously added constraint by adding it to all live bindings.
@@ -429,7 +427,7 @@ impl<'db> SemanticIndexBuilder<'db> {
/// Negates the given constraint and then adds it to all live bindings.
fn record_negated_constraint(&mut self, constraint: Constraint<'db>) -> ScopedConstraintId {
let (_, id) = self.add_negated_constraint(constraint);
let id = self.add_negated_constraint(constraint);
self.record_constraint_id(id);
id
}
@@ -458,9 +456,10 @@ impl<'db> SemanticIndexBuilder<'db> {
&mut self,
constraint: Constraint<'db>,
) -> ScopedVisibilityConstraintId {
let constraint_id = self.current_use_def_map_mut().add_constraint(constraint);
let id = self
.current_visibility_constraints_mut()
.add_atom(constraint, 0);
.add_atom(constraint_id);
self.record_visibility_constraint_id(id);
id
}
@@ -1190,12 +1189,14 @@ where
// We need multiple copies of the visibility constraint for the while condition,
// since we need to model situations where the first evaluation of the condition
// returns True, but a later evaluation returns False.
let first_constraint_id = self.current_use_def_map_mut().add_constraint(constraint);
let later_constraint_id = self.current_use_def_map_mut().add_constraint(constraint);
let first_vis_constraint_id = self
.current_visibility_constraints_mut()
.add_atom(constraint, 0);
.add_atom(first_constraint_id);
let later_vis_constraint_id = self
.current_visibility_constraints_mut()
.add_atom(constraint, 1);
.add_atom(later_constraint_id);
// Save aside any break states from an outer loop
let saved_break_states = std::mem::take(&mut self.loop_break_states);
@@ -1776,13 +1777,13 @@ where
// anymore.
if index < values.len() - 1 {
let constraint = self.build_constraint(value);
let (constraint, constraint_id) = match op {
ast::BoolOp::And => (constraint, self.add_constraint(constraint)),
let constraint_id = match op {
ast::BoolOp::And => self.add_constraint(constraint),
ast::BoolOp::Or => self.add_negated_constraint(constraint),
};
let visibility_constraint = self
.current_visibility_constraints_mut()
.add_atom(constraint, 0);
.add_atom(constraint_id);
let after_expr = self.flow_snapshot();

View File

@@ -1,10 +1,40 @@
use ruff_db::files::File;
use ruff_index::{newtype_index, IndexVec};
use ruff_python_ast::Singleton;
use crate::db::Db;
use crate::semantic_index::expression::Expression;
use crate::semantic_index::symbol::{FileScopeId, ScopeId};
// A scoped identifier for each `Constraint` in a scope.
#[newtype_index]
#[derive(Ord, PartialOrd)]
pub(crate) struct ScopedConstraintId;
// A collection of constraints. This is currently stored in `UseDefMap`, which means we maintain a
// separate set of constraints for each scope in a file.
pub(crate) type Constraints<'db> = IndexVec<ScopedConstraintId, Constraint<'db>>;
#[derive(Debug, Default)]
pub(crate) struct ConstraintsBuilder<'db> {
constraints: IndexVec<ScopedConstraintId, Constraint<'db>>,
}
impl<'db> ConstraintsBuilder<'db> {
/// Adds a constraint. Note that we do not deduplicate constraints. If you add a `Constraint`
/// more than once, you will get distinct `ScopedConstraintId`s for each one. (This lets you
/// model constraint expressions that might evaluate to different values at different points of
/// execution.)
pub(crate) fn add_constraint(&mut self, constraint: Constraint<'db>) -> ScopedConstraintId {
self.constraints.push(constraint)
}
pub(crate) fn build(mut self) -> Constraints<'db> {
self.constraints.shrink_to_fit();
self.constraints
}
}
#[derive(Clone, Copy, Debug, Hash, PartialEq, Eq, salsa::Update)]
pub(crate) struct Constraint<'db> {
pub(crate) node: ConstraintNode<'db>,

View File

@@ -33,11 +33,16 @@ pub struct Definition<'db> {
/// The symbol defined.
pub(crate) symbol: ScopedSymbolId,
/// WARNING: Only access this field when doing type inference for the same
/// file as where `Definition` is defined to avoid cross-file query dependencies.
#[no_eq]
#[return_ref]
#[tracked]
pub(crate) kind: DefinitionKind<'db>,
/// This is a dedicated field to avoid accessing `kind` to compute this value.
pub(crate) is_reexported: bool,
count: countme::Count<Definition<'static>>,
}
@@ -45,22 +50,6 @@ impl<'db> Definition<'db> {
pub(crate) fn scope(self, db: &'db dyn Db) -> ScopeId<'db> {
self.file_scope(db).to_scope_id(db, self.file(db))
}
pub(crate) fn category(self, db: &'db dyn Db) -> DefinitionCategory {
self.kind(db).category()
}
pub(crate) fn is_declaration(self, db: &'db dyn Db) -> bool {
self.kind(db).category().is_declaration()
}
pub(crate) fn is_binding(self, db: &'db dyn Db) -> bool {
self.kind(db).category().is_binding()
}
pub(crate) fn is_reexported(self, db: &'db dyn Db) -> bool {
self.kind(db).is_reexported()
}
}
#[derive(Copy, Clone, Debug)]

View File

@@ -165,7 +165,7 @@
//! don't actually store these "list of visible definitions" as a vector of [`Definition`].
//! Instead, [`SymbolBindings`] and [`SymbolDeclarations`] are structs which use bit-sets to track
//! definitions (and constraints, in the case of bindings) in terms of [`ScopedDefinitionId`] and
//! [`ScopedConstraintId`], which are indices into the `all_definitions` and `all_constraints`
//! [`ScopedConstraintId`], which are indices into the `all_definitions` and `constraints`
//! indexvecs in the [`UseDefMap`].
//!
//! There is another special kind of possible "definition" for a symbol: there might be a path from
@@ -255,28 +255,27 @@
//! snapshot, and merging a snapshot into the current state. The logic using these methods lives in
//! [`SemanticIndexBuilder`](crate::semantic_index::builder::SemanticIndexBuilder), e.g. where it
//! visits a `StmtIf` node.
pub(crate) use self::symbol_state::ScopedConstraintId;
use self::symbol_state::{
BindingIdWithConstraintsIterator, ConstraintIdIterator, DeclarationIdIterator,
ScopedDefinitionId, SymbolBindings, SymbolDeclarations, SymbolState,
};
use crate::semantic_index::ast_ids::ScopedUseId;
use crate::semantic_index::definition::Definition;
use crate::semantic_index::symbol::{FileScopeId, ScopedSymbolId};
use crate::semantic_index::use_def::symbol_state::DeclarationIdWithConstraint;
use crate::visibility_constraints::{
ScopedVisibilityConstraintId, VisibilityConstraints, VisibilityConstraintsBuilder,
};
use ruff_index::{newtype_index, IndexVec};
use rustc_hash::FxHashMap;
use super::constraint::Constraint;
use self::symbol_state::{
ConstraintIndexIterator, LiveBindingsIterator, LiveDeclaration, LiveDeclarationsIterator,
ScopedDefinitionId, SymbolBindings, SymbolDeclarations, SymbolState,
};
use crate::semantic_index::ast_ids::ScopedUseId;
use crate::semantic_index::constraint::{
Constraint, Constraints, ConstraintsBuilder, ScopedConstraintId,
};
use crate::semantic_index::definition::Definition;
use crate::semantic_index::symbol::{FileScopeId, ScopedSymbolId};
use crate::visibility_constraints::{
ScopedVisibilityConstraintId, VisibilityConstraints, VisibilityConstraintsBuilder,
};
mod bitset;
mod symbol_state;
type AllConstraints<'db> = IndexVec<ScopedConstraintId, Constraint<'db>>;
/// Applicable definitions and constraints for every use of a name.
#[derive(Debug, PartialEq, Eq, salsa::Update)]
pub(crate) struct UseDefMap<'db> {
@@ -285,10 +284,10 @@ pub(crate) struct UseDefMap<'db> {
all_definitions: IndexVec<ScopedDefinitionId, Option<Definition<'db>>>,
/// Array of [`Constraint`] in this scope.
all_constraints: AllConstraints<'db>,
constraints: Constraints<'db>,
/// Array of visibility constraints in this scope.
visibility_constraints: VisibilityConstraints<'db>,
visibility_constraints: VisibilityConstraints,
/// [`SymbolBindings`] reaching a [`ScopedUseId`].
bindings_by_use: IndexVec<ScopedUseId, SymbolBindings>,
@@ -370,7 +369,7 @@ impl<'db> UseDefMap<'db> {
) -> BindingWithConstraintsIterator<'map, 'db> {
BindingWithConstraintsIterator {
all_definitions: &self.all_definitions,
all_constraints: &self.all_constraints,
constraints: &self.constraints,
visibility_constraints: &self.visibility_constraints,
inner: bindings.iter(),
}
@@ -382,6 +381,7 @@ impl<'db> UseDefMap<'db> {
) -> DeclarationsIterator<'map, 'db> {
DeclarationsIterator {
all_definitions: &self.all_definitions,
constraints: &self.constraints,
visibility_constraints: &self.visibility_constraints,
inner: declarations.iter(),
}
@@ -415,26 +415,26 @@ type EagerBindings = IndexVec<ScopedEagerBindingsId, SymbolBindings>;
#[derive(Debug)]
pub(crate) struct BindingWithConstraintsIterator<'map, 'db> {
all_definitions: &'map IndexVec<ScopedDefinitionId, Option<Definition<'db>>>,
all_constraints: &'map AllConstraints<'db>,
pub(crate) visibility_constraints: &'map VisibilityConstraints<'db>,
inner: BindingIdWithConstraintsIterator<'map>,
pub(crate) constraints: &'map Constraints<'db>,
pub(crate) visibility_constraints: &'map VisibilityConstraints,
inner: LiveBindingsIterator<'map>,
}
impl<'map, 'db> Iterator for BindingWithConstraintsIterator<'map, 'db> {
type Item = BindingWithConstraints<'map, 'db>;
fn next(&mut self) -> Option<Self::Item> {
let all_constraints = self.all_constraints;
let constraints = self.constraints;
self.inner
.next()
.map(|binding_id_with_constraints| BindingWithConstraints {
binding: self.all_definitions[binding_id_with_constraints.definition],
constraints: ConstraintsIterator {
all_constraints,
constraint_ids: binding_id_with_constraints.constraint_ids,
.map(|live_binding| BindingWithConstraints {
binding: self.all_definitions[live_binding.binding],
narrowing_constraints: ConstraintsIterator {
constraints,
constraint_ids: live_binding.narrowing_constraints.iter(),
},
visibility_constraint: binding_id_with_constraints.visibility_constraint,
visibility_constraint: live_binding.visibility_constraint,
})
}
}
@@ -443,13 +443,13 @@ impl std::iter::FusedIterator for BindingWithConstraintsIterator<'_, '_> {}
pub(crate) struct BindingWithConstraints<'map, 'db> {
pub(crate) binding: Option<Definition<'db>>,
pub(crate) constraints: ConstraintsIterator<'map, 'db>,
pub(crate) narrowing_constraints: ConstraintsIterator<'map, 'db>,
pub(crate) visibility_constraint: ScopedVisibilityConstraintId,
}
pub(crate) struct ConstraintsIterator<'map, 'db> {
all_constraints: &'map AllConstraints<'db>,
constraint_ids: ConstraintIdIterator<'map>,
constraints: &'map Constraints<'db>,
constraint_ids: ConstraintIndexIterator<'map>,
}
impl<'db> Iterator for ConstraintsIterator<'_, 'db> {
@@ -458,7 +458,7 @@ impl<'db> Iterator for ConstraintsIterator<'_, 'db> {
fn next(&mut self) -> Option<Self::Item> {
self.constraint_ids
.next()
.map(|constraint_id| self.all_constraints[constraint_id])
.map(|constraint_id| self.constraints[ScopedConstraintId::from_u32(constraint_id)])
}
}
@@ -466,8 +466,9 @@ impl std::iter::FusedIterator for ConstraintsIterator<'_, '_> {}
pub(crate) struct DeclarationsIterator<'map, 'db> {
all_definitions: &'map IndexVec<ScopedDefinitionId, Option<Definition<'db>>>,
pub(crate) visibility_constraints: &'map VisibilityConstraints<'db>,
inner: DeclarationIdIterator<'map>,
pub(crate) constraints: &'map Constraints<'db>,
pub(crate) visibility_constraints: &'map VisibilityConstraints,
inner: LiveDeclarationsIterator<'map>,
}
pub(crate) struct DeclarationWithConstraint<'db> {
@@ -480,13 +481,13 @@ impl<'db> Iterator for DeclarationsIterator<'_, 'db> {
fn next(&mut self) -> Option<Self::Item> {
self.inner.next().map(
|DeclarationIdWithConstraint {
definition,
|LiveDeclaration {
declaration,
visibility_constraint,
}| {
DeclarationWithConstraint {
declaration: self.all_definitions[definition],
visibility_constraint,
declaration: self.all_definitions[*declaration],
visibility_constraint: *visibility_constraint,
}
},
)
@@ -507,11 +508,11 @@ pub(super) struct UseDefMapBuilder<'db> {
/// Append-only array of [`Definition`].
all_definitions: IndexVec<ScopedDefinitionId, Option<Definition<'db>>>,
/// Append-only array of [`Constraint`].
all_constraints: AllConstraints<'db>,
/// Builder of constraints.
constraints: ConstraintsBuilder<'db>,
/// Builder of visibility constraints.
pub(super) visibility_constraints: VisibilityConstraintsBuilder<'db>,
pub(super) visibility_constraints: VisibilityConstraintsBuilder,
/// A constraint which describes the visibility of the unbound/undeclared state, i.e.
/// whether or not the start of the scope is visible. This is important for cases like
@@ -540,7 +541,7 @@ impl Default for UseDefMapBuilder<'_> {
fn default() -> Self {
Self {
all_definitions: IndexVec::from_iter([None]),
all_constraints: IndexVec::new(),
constraints: ConstraintsBuilder::default(),
visibility_constraints: VisibilityConstraintsBuilder::default(),
scope_start_visibility: ScopedVisibilityConstraintId::ALWAYS_TRUE,
bindings_by_use: IndexVec::new(),
@@ -573,7 +574,7 @@ impl<'db> UseDefMapBuilder<'db> {
}
pub(super) fn add_constraint(&mut self, constraint: Constraint<'db>) -> ScopedConstraintId {
self.all_constraints.push(constraint)
self.constraints.add_constraint(constraint)
}
pub(super) fn record_constraint_id(&mut self, constraint: ScopedConstraintId) {
@@ -753,7 +754,6 @@ impl<'db> UseDefMapBuilder<'db> {
pub(super) fn finish(mut self) -> UseDefMap<'db> {
self.all_definitions.shrink_to_fit();
self.all_constraints.shrink_to_fit();
self.symbol_states.shrink_to_fit();
self.bindings_by_use.shrink_to_fit();
self.declarations_by_binding.shrink_to_fit();
@@ -762,7 +762,7 @@ impl<'db> UseDefMapBuilder<'db> {
UseDefMap {
all_definitions: self.all_definitions,
all_constraints: self.all_constraints,
constraints: self.constraints.build(),
visibility_constraints: self.visibility_constraints.build(),
bindings_by_use: self.bindings_by_use,
public_symbols: self.symbol_states,

View File

@@ -25,13 +25,6 @@ impl<const B: usize> Default for BitSet<B> {
}
impl<const B: usize> BitSet<B> {
/// Create and return a new [`BitSet`] with a single `value` inserted.
pub(super) fn with(value: u32) -> Self {
let mut bitset = Self::default();
bitset.insert(value);
bitset
}
/// Convert from Inline to Heap, if needed, and resize the Heap vector, if needed.
fn resize(&mut self, value: u32) {
let num_blocks_needed = (value / 64) + 1;
@@ -93,19 +86,6 @@ impl<const B: usize> BitSet<B> {
}
}
/// Union in-place with another [`BitSet`].
pub(super) fn union(&mut self, other: &BitSet<B>) {
let mut max_len = self.blocks().len();
let other_len = other.blocks().len();
if other_len > max_len {
max_len = other_len;
self.resize_blocks(max_len);
}
for (my_block, other_block) in self.blocks_mut().iter_mut().zip(other.blocks()) {
*my_block |= other_block;
}
}
/// Return an iterator over the values (in ascending order) in this [`BitSet`].
pub(super) fn iter(&self) -> BitSetIterator<'_, B> {
let blocks = self.blocks();
@@ -158,6 +138,15 @@ impl<const B: usize> std::iter::FusedIterator for BitSetIterator<'_, B> {}
mod tests {
use super::BitSet;
impl<const B: usize> BitSet<B> {
/// Create and return a new [`BitSet`] with a single `value` inserted.
pub(super) fn with(value: u32) -> Self {
let mut bitset = Self::default();
bitset.insert(value);
bitset
}
}
fn assert_bitset<const B: usize>(bitset: &BitSet<B>, contents: &[u32]) {
assert_eq!(bitset.iter().collect::<Vec<_>>(), contents);
}
@@ -235,59 +224,6 @@ mod tests {
assert_bitset(&b1, &[89]);
}
#[test]
fn union() {
let mut b1 = BitSet::<1>::with(2);
let b2 = BitSet::<1>::with(4);
b1.union(&b2);
assert_bitset(&b1, &[2, 4]);
}
#[test]
fn union_mixed_1() {
let mut b1 = BitSet::<1>::with(4);
let mut b2 = BitSet::<1>::with(4);
b1.insert(89);
b2.insert(5);
b1.union(&b2);
assert_bitset(&b1, &[4, 5, 89]);
}
#[test]
fn union_mixed_2() {
let mut b1 = BitSet::<1>::with(4);
let mut b2 = BitSet::<1>::with(4);
b1.insert(23);
b2.insert(89);
b1.union(&b2);
assert_bitset(&b1, &[4, 23, 89]);
}
#[test]
fn union_heap() {
let mut b1 = BitSet::<1>::with(4);
let mut b2 = BitSet::<1>::with(4);
b1.insert(89);
b2.insert(90);
b1.union(&b2);
assert_bitset(&b1, &[4, 89, 90]);
}
#[test]
fn union_heap_2() {
let mut b1 = BitSet::<1>::with(89);
let mut b2 = BitSet::<1>::with(89);
b1.insert(91);
b2.insert(90);
b1.union(&b2);
assert_bitset(&b1, &[89, 90, 91]);
}
#[test]
fn multiple_blocks() {
let mut b = BitSet::<2>::with(120);

View File

@@ -46,14 +46,16 @@
use itertools::{EitherOrBoth, Itertools};
use ruff_index::newtype_index;
use smallvec::SmallVec;
use smallvec::{smallvec, SmallVec};
use crate::semantic_index::constraint::ScopedConstraintId;
use crate::semantic_index::use_def::bitset::{BitSet, BitSetIterator};
use crate::semantic_index::use_def::VisibilityConstraintsBuilder;
use crate::visibility_constraints::ScopedVisibilityConstraintId;
/// A newtype-index for a definition in a particular scope.
#[newtype_index]
#[derive(Ord, PartialOrd)]
pub(super) struct ScopedDefinitionId;
impl ScopedDefinitionId {
@@ -65,89 +67,54 @@ impl ScopedDefinitionId {
pub(super) const UNBOUND: ScopedDefinitionId = ScopedDefinitionId::from_u32(0);
}
/// A newtype-index for a constraint expression in a particular scope.
#[newtype_index]
pub(crate) struct ScopedConstraintId;
/// Can reference this * 64 total definitions inline; more will fall back to the heap.
const INLINE_BINDING_BLOCKS: usize = 3;
/// A [`BitSet`] of [`ScopedDefinitionId`], representing live bindings of a symbol in a scope.
type Bindings = BitSet<INLINE_BINDING_BLOCKS>;
type BindingsIterator<'a> = BitSetIterator<'a, INLINE_BINDING_BLOCKS>;
/// Can reference this * 64 total declarations inline; more will fall back to the heap.
const INLINE_DECLARATION_BLOCKS: usize = 3;
/// A [`BitSet`] of [`ScopedDefinitionId`], representing live declarations of a symbol in a scope.
type Declarations = BitSet<INLINE_DECLARATION_BLOCKS>;
type DeclarationsIterator<'a> = BitSetIterator<'a, INLINE_DECLARATION_BLOCKS>;
/// Can reference this * 64 total constraints inline; more will fall back to the heap.
const INLINE_CONSTRAINT_BLOCKS: usize = 2;
/// Can keep inline this many live bindings per symbol at a given time; more will go to heap.
const INLINE_BINDINGS_PER_SYMBOL: usize = 4;
/// Can keep inline this many live bindings or declarations per symbol at a given time; more will
/// go to heap.
const INLINE_DEFINITIONS_PER_SYMBOL: usize = 4;
/// Which constraints apply to a given binding?
type Constraints = BitSet<INLINE_CONSTRAINT_BLOCKS>;
type InlineConstraintArray = [Constraints; INLINE_BINDINGS_PER_SYMBOL];
/// One [`BitSet`] of applicable [`ScopedConstraintId`]s per live binding.
type ConstraintsPerBinding = SmallVec<InlineConstraintArray>;
/// Iterate over all constraints for a single binding.
type ConstraintsIterator<'a> = std::slice::Iter<'a, Constraints>;
const INLINE_VISIBILITY_CONSTRAINTS: usize = 4;
type InlineVisibilityConstraintsArray =
[ScopedVisibilityConstraintId; INLINE_VISIBILITY_CONSTRAINTS];
/// One [`ScopedVisibilityConstraintId`] per live declaration.
type VisibilityConstraintPerDeclaration = SmallVec<InlineVisibilityConstraintsArray>;
/// One [`ScopedVisibilityConstraintId`] per live binding.
type VisibilityConstraintPerBinding = SmallVec<InlineVisibilityConstraintsArray>;
/// Iterator over the visibility constraints for all live bindings/declarations.
type VisibilityConstraintsIterator<'a> = std::slice::Iter<'a, ScopedVisibilityConstraintId>;
pub(super) type ConstraintIndexIterator<'a> = BitSetIterator<'a, INLINE_CONSTRAINT_BLOCKS>;
/// Live declarations for a single symbol at some point in control flow, with their
/// corresponding visibility constraints.
#[derive(Clone, Debug, Default, PartialEq, Eq, salsa::Update)]
pub(super) struct SymbolDeclarations {
/// [`BitSet`]: which declarations (as [`ScopedDefinitionId`]) can reach the current location?
///
/// Invariant: Because this is a `BitSet`, it can be viewed as a _sorted_ set of definition
/// IDs. The `visibility_constraints` field stores constraints for each definition. Therefore
/// those fields must always have the same `len()` as `live_declarations`, and the elements
/// must appear in the same order. Effectively, this means that elements must always be added
/// in sorted order, or via a binary search that determines the correct place to insert new
/// constraints.
pub(crate) live_declarations: Declarations,
/// For each live declaration, which visibility constraint applies to it?
pub(crate) visibility_constraints: VisibilityConstraintPerDeclaration,
/// A list of live declarations for this symbol, sorted by their `ScopedDefinitionId`
live_declarations: SmallVec<[LiveDeclaration; INLINE_DEFINITIONS_PER_SYMBOL]>,
}
/// One of the live declarations for a single symbol at some point in control flow.
#[derive(Clone, Debug, PartialEq, Eq)]
pub(super) struct LiveDeclaration {
pub(super) declaration: ScopedDefinitionId,
pub(super) visibility_constraint: ScopedVisibilityConstraintId,
}
pub(super) type LiveDeclarationsIterator<'a> = std::slice::Iter<'a, LiveDeclaration>;
impl SymbolDeclarations {
fn undeclared(scope_start_visibility: ScopedVisibilityConstraintId) -> Self {
let initial_declaration = LiveDeclaration {
declaration: ScopedDefinitionId::UNBOUND,
visibility_constraint: scope_start_visibility,
};
Self {
live_declarations: Declarations::with(0),
visibility_constraints: VisibilityConstraintPerDeclaration::from_iter([
scope_start_visibility,
]),
live_declarations: smallvec![initial_declaration],
}
}
/// Record a newly-encountered declaration for this symbol.
fn record_declaration(&mut self, declaration_id: ScopedDefinitionId) {
self.live_declarations = Declarations::with(declaration_id.into());
self.visibility_constraints = VisibilityConstraintPerDeclaration::with_capacity(1);
self.visibility_constraints
.push(ScopedVisibilityConstraintId::ALWAYS_TRUE);
fn record_declaration(&mut self, declaration: ScopedDefinitionId) {
// The new declaration replaces all previous live declaration in this path.
self.live_declarations.clear();
self.live_declarations.push(LiveDeclaration {
declaration,
visibility_constraint: ScopedVisibilityConstraintId::ALWAYS_TRUE,
});
}
/// Add given visibility constraint to all live declarations.
@@ -156,45 +123,62 @@ impl SymbolDeclarations {
visibility_constraints: &mut VisibilityConstraintsBuilder,
constraint: ScopedVisibilityConstraintId,
) {
for existing in &mut self.visibility_constraints {
*existing = visibility_constraints.add_and_constraint(*existing, constraint);
for declaration in &mut self.live_declarations {
declaration.visibility_constraint = visibility_constraints
.add_and_constraint(declaration.visibility_constraint, constraint);
}
}
/// Return an iterator over live declarations for this symbol.
pub(super) fn iter(&self) -> DeclarationIdIterator {
DeclarationIdIterator {
declarations: self.live_declarations.iter(),
visibility_constraints: self.visibility_constraints.iter(),
pub(super) fn iter(&self) -> LiveDeclarationsIterator<'_> {
self.live_declarations.iter()
}
/// Iterate over the IDs of each currently live declaration for this symbol
fn iter_declarations(&self) -> impl Iterator<Item = ScopedDefinitionId> + '_ {
self.iter().map(|lb| lb.declaration)
}
fn simplify_visibility_constraints(&mut self, other: SymbolDeclarations) {
// If the set of live declarations hasn't changed, don't simplify.
if self.live_declarations.len() != other.live_declarations.len()
|| !self.iter_declarations().eq(other.iter_declarations())
{
return;
}
for (declaration, other_declaration) in self
.live_declarations
.iter_mut()
.zip(other.live_declarations)
{
declaration.visibility_constraint = other_declaration.visibility_constraint;
}
}
fn merge(&mut self, b: Self, visibility_constraints: &mut VisibilityConstraintsBuilder) {
let a = std::mem::take(self);
self.live_declarations = a.live_declarations.clone();
self.live_declarations.union(&b.live_declarations);
// Invariant: These zips are well-formed since we maintain an invariant that all of our
// fields are sets/vecs with the same length.
let a = (a.live_declarations.iter()).zip(a.visibility_constraints);
let b = (b.live_declarations.iter()).zip(b.visibility_constraints);
// Invariant: merge_join_by consumes the two iterators in sorted order, which ensures that
// the definition IDs and constraints line up correctly in the merged result. If a
// definition is found in both `a` and `b`, we compose the constraints from the two paths
// in an appropriate way (intersection for narrowing constraints; ternary OR for visibility
// constraints). If a definition is found in only one path, it is used as-is.
for zipped in a.merge_join_by(b, |(a_decl, _), (b_decl, _)| a_decl.cmp(b_decl)) {
// the merged `live_declarations` vec remains sorted. If a definition is found in both `a`
// and `b`, we compose the constraints from the two paths in an appropriate way
// (intersection for narrowing constraints; ternary OR for visibility constraints). If a
// definition is found in only one path, it is used as-is.
let a = a.live_declarations.into_iter();
let b = b.live_declarations.into_iter();
for zipped in a.merge_join_by(b, |a, b| a.declaration.cmp(&b.declaration)) {
match zipped {
EitherOrBoth::Both((_, a_vis_constraint), (_, b_vis_constraint)) => {
let vis_constraint = visibility_constraints
.add_or_constraint(a_vis_constraint, b_vis_constraint);
self.visibility_constraints.push(vis_constraint);
EitherOrBoth::Both(a, b) => {
let visibility_constraint = visibility_constraints
.add_or_constraint(a.visibility_constraint, b.visibility_constraint);
self.live_declarations.push(LiveDeclaration {
declaration: a.declaration,
visibility_constraint,
});
}
EitherOrBoth::Left((_, vis_constraint))
| EitherOrBoth::Right((_, vis_constraint)) => {
self.visibility_constraints.push(vis_constraint);
EitherOrBoth::Left(declaration) | EitherOrBoth::Right(declaration) => {
self.live_declarations.push(declaration);
}
}
}
@@ -205,57 +189,52 @@ impl SymbolDeclarations {
/// with a set of narrowing constraints and a visibility constraint.
#[derive(Clone, Debug, Default, PartialEq, Eq, salsa::Update)]
pub(super) struct SymbolBindings {
/// [`BitSet`]: which bindings (as [`ScopedDefinitionId`]) can reach the current location?
///
/// Invariant: Because this is a `BitSet`, it can be viewed as a _sorted_ set of definition
/// IDs. The `constraints` and `visibility_constraints` field stores constraints for each
/// definition. Therefore those fields must always have the same `len()` as
/// `live_bindings`, and the elements must appear in the same order. Effectively, this means
/// that elements must always be added in sorted order, or via a binary search that determines
/// the correct place to insert new constraints.
live_bindings: Bindings,
/// For each live binding, which [`ScopedConstraintId`] apply?
///
/// This is a [`smallvec::SmallVec`] which should always have one [`BitSet`] of constraints per
/// binding in `live_bindings`.
constraints: ConstraintsPerBinding,
/// For each live binding, which visibility constraint applies to it?
visibility_constraints: VisibilityConstraintPerBinding,
/// A list of live bindings for this symbol, sorted by their `ScopedDefinitionId`
live_bindings: SmallVec<[LiveBinding; INLINE_DEFINITIONS_PER_SYMBOL]>,
}
/// One of the live bindings for a single symbol at some point in control flow.
#[derive(Clone, Debug, PartialEq, Eq)]
pub(super) struct LiveBinding {
pub(super) binding: ScopedDefinitionId,
pub(super) narrowing_constraints: Constraints,
pub(super) visibility_constraint: ScopedVisibilityConstraintId,
}
pub(super) type LiveBindingsIterator<'a> = std::slice::Iter<'a, LiveBinding>;
impl SymbolBindings {
fn unbound(scope_start_visibility: ScopedVisibilityConstraintId) -> Self {
let initial_binding = LiveBinding {
binding: ScopedDefinitionId::UNBOUND,
narrowing_constraints: Constraints::default(),
visibility_constraint: scope_start_visibility,
};
Self {
live_bindings: Bindings::with(ScopedDefinitionId::UNBOUND.as_u32()),
constraints: ConstraintsPerBinding::from_iter([Constraints::default()]),
visibility_constraints: VisibilityConstraintPerBinding::from_iter([
scope_start_visibility,
]),
live_bindings: smallvec![initial_binding],
}
}
/// Record a newly-encountered binding for this symbol.
pub(super) fn record_binding(
&mut self,
binding_id: ScopedDefinitionId,
binding: ScopedDefinitionId,
visibility_constraint: ScopedVisibilityConstraintId,
) {
// The new binding replaces all previous live bindings in this path, and has no
// constraints.
self.live_bindings = Bindings::with(binding_id.into());
self.constraints = ConstraintsPerBinding::with_capacity(1);
self.constraints.push(Constraints::default());
self.visibility_constraints = VisibilityConstraintPerBinding::with_capacity(1);
self.visibility_constraints.push(visibility_constraint);
self.live_bindings.clear();
self.live_bindings.push(LiveBinding {
binding,
narrowing_constraints: Constraints::default(),
visibility_constraint,
});
}
/// Add given constraint to all live bindings.
pub(super) fn record_constraint(&mut self, constraint_id: ScopedConstraintId) {
for bitset in &mut self.constraints {
bitset.insert(constraint_id.into());
for binding in &mut self.live_bindings {
binding.narrowing_constraints.insert(constraint_id.into());
}
}
@@ -265,71 +244,67 @@ impl SymbolBindings {
visibility_constraints: &mut VisibilityConstraintsBuilder,
constraint: ScopedVisibilityConstraintId,
) {
for existing in &mut self.visibility_constraints {
*existing = visibility_constraints.add_and_constraint(*existing, constraint);
for binding in &mut self.live_bindings {
binding.visibility_constraint = visibility_constraints
.add_and_constraint(binding.visibility_constraint, constraint);
}
}
/// Iterate over currently live bindings for this symbol
pub(super) fn iter(&self) -> BindingIdWithConstraintsIterator {
BindingIdWithConstraintsIterator {
definitions: self.live_bindings.iter(),
constraints: self.constraints.iter(),
visibility_constraints: self.visibility_constraints.iter(),
pub(super) fn iter(&self) -> LiveBindingsIterator<'_> {
self.live_bindings.iter()
}
/// Iterate over the IDs of each currently live binding for this symbol
fn iter_bindings(&self) -> impl Iterator<Item = ScopedDefinitionId> + '_ {
self.iter().map(|lb| lb.binding)
}
fn simplify_visibility_constraints(&mut self, other: SymbolBindings) {
// If the set of live bindings hasn't changed, don't simplify.
if self.live_bindings.len() != other.live_bindings.len()
|| !self.iter_bindings().eq(other.iter_bindings())
{
return;
}
for (binding, other_binding) in self.live_bindings.iter_mut().zip(other.live_bindings) {
binding.visibility_constraint = other_binding.visibility_constraint;
}
}
fn merge(&mut self, mut b: Self, visibility_constraints: &mut VisibilityConstraintsBuilder) {
let mut a = std::mem::take(self);
self.live_bindings = a.live_bindings.clone();
self.live_bindings.union(&b.live_bindings);
// Invariant: These zips are well-formed since we maintain an invariant that all of our
// fields are sets/vecs with the same length.
//
// Performance: We iterate over the `constraints` smallvecs via mut reference, because the
// individual elements are `BitSet`s (currently 24 bytes in size), and we don't want to
// move them by value multiple times during iteration. By iterating by reference, we only
// have to copy single pointers around. In the loop below, the `std::mem::take` calls
// specify precisely where we want to move them into the merged `constraints` smallvec.
//
// We don't need a similar optimization for `visibility_constraints`, since those elements
// are 32-bit IndexVec IDs, and so are already cheap to move/copy.
let a = (a.live_bindings.iter())
.zip(a.constraints.iter_mut())
.zip(a.visibility_constraints);
let b = (b.live_bindings.iter())
.zip(b.constraints.iter_mut())
.zip(b.visibility_constraints);
fn merge(&mut self, b: Self, visibility_constraints: &mut VisibilityConstraintsBuilder) {
let a = std::mem::take(self);
// Invariant: merge_join_by consumes the two iterators in sorted order, which ensures that
// the definition IDs and constraints line up correctly in the merged result. If a
// definition is found in both `a` and `b`, we compose the constraints from the two paths
// in an appropriate way (intersection for narrowing constraints; ternary OR for visibility
// constraints). If a definition is found in only one path, it is used as-is.
for zipped in a.merge_join_by(b, |((a_def, _), _), ((b_def, _), _)| a_def.cmp(b_def)) {
// the merged `live_bindings` vec remains sorted. If a definition is found in both `a` and
// `b`, we compose the constraints from the two paths in an appropriate way (intersection
// for narrowing constraints; ternary OR for visibility constraints). If a definition is
// found in only one path, it is used as-is.
let a = a.live_bindings.into_iter();
let b = b.live_bindings.into_iter();
for zipped in a.merge_join_by(b, |a, b| a.binding.cmp(&b.binding)) {
match zipped {
EitherOrBoth::Both(
((_, a_constraints), a_vis_constraint),
((_, b_constraints), b_vis_constraint),
) => {
EitherOrBoth::Both(a, b) => {
// If the same definition is visible through both paths, any constraint
// that applies on only one path is irrelevant to the resulting type from
// unioning the two paths, so we intersect the constraints.
let constraints = a_constraints;
constraints.intersect(b_constraints);
self.constraints.push(std::mem::take(constraints));
let mut narrowing_constraints = a.narrowing_constraints;
narrowing_constraints.intersect(&b.narrowing_constraints);
// For visibility constraints, we merge them using a ternary OR operation:
let vis_constraint = visibility_constraints
.add_or_constraint(a_vis_constraint, b_vis_constraint);
self.visibility_constraints.push(vis_constraint);
let visibility_constraint = visibility_constraints
.add_or_constraint(a.visibility_constraint, b.visibility_constraint);
self.live_bindings.push(LiveBinding {
binding: a.binding,
narrowing_constraints,
visibility_constraint,
});
}
EitherOrBoth::Left(((_, constraints), vis_constraint))
| EitherOrBoth::Right(((_, constraints), vis_constraint)) => {
self.constraints.push(std::mem::take(constraints));
self.visibility_constraints.push(vis_constraint);
EitherOrBoth::Left(binding) | EitherOrBoth::Right(binding) => {
self.live_bindings.push(binding);
}
}
}
@@ -379,14 +354,14 @@ impl SymbolState {
.record_visibility_constraint(visibility_constraints, constraint);
}
/// Simplifies this snapshot to have the same visibility constraints as a previous point in the
/// control flow, but only if the set of live bindings or declarations for this symbol hasn't
/// changed.
pub(super) fn simplify_visibility_constraints(&mut self, snapshot_state: SymbolState) {
if self.bindings.live_bindings == snapshot_state.bindings.live_bindings {
self.bindings.visibility_constraints = snapshot_state.bindings.visibility_constraints;
}
if self.declarations.live_declarations == snapshot_state.declarations.live_declarations {
self.declarations.visibility_constraints =
snapshot_state.declarations.visibility_constraints;
}
self.bindings
.simplify_visibility_constraints(snapshot_state.bindings);
self.declarations
.simplify_visibility_constraints(snapshot_state.declarations);
}
/// Record a newly-encountered declaration of this symbol.
@@ -414,98 +389,6 @@ impl SymbolState {
}
}
/// A single binding (as [`ScopedDefinitionId`]) with an iterator of its applicable
/// narrowing constraints ([`ScopedConstraintId`]) and a corresponding visibility
/// visibility constraint ([`ScopedVisibilityConstraintId`]).
#[derive(Debug)]
pub(super) struct BindingIdWithConstraints<'map> {
pub(super) definition: ScopedDefinitionId,
pub(super) constraint_ids: ConstraintIdIterator<'map>,
pub(super) visibility_constraint: ScopedVisibilityConstraintId,
}
#[derive(Debug)]
pub(super) struct BindingIdWithConstraintsIterator<'map> {
definitions: BindingsIterator<'map>,
constraints: ConstraintsIterator<'map>,
visibility_constraints: VisibilityConstraintsIterator<'map>,
}
impl<'map> Iterator for BindingIdWithConstraintsIterator<'map> {
type Item = BindingIdWithConstraints<'map>;
fn next(&mut self) -> Option<Self::Item> {
match (
self.definitions.next(),
self.constraints.next(),
self.visibility_constraints.next(),
) {
(None, None, None) => None,
(Some(def), Some(constraints), Some(visibility_constraint_id)) => {
Some(BindingIdWithConstraints {
definition: ScopedDefinitionId::from_u32(def),
constraint_ids: ConstraintIdIterator {
wrapped: constraints.iter(),
},
visibility_constraint: *visibility_constraint_id,
})
}
// SAFETY: see above.
_ => unreachable!("definitions and constraints length mismatch"),
}
}
}
impl std::iter::FusedIterator for BindingIdWithConstraintsIterator<'_> {}
#[derive(Debug)]
pub(super) struct ConstraintIdIterator<'a> {
wrapped: BitSetIterator<'a, INLINE_CONSTRAINT_BLOCKS>,
}
impl Iterator for ConstraintIdIterator<'_> {
type Item = ScopedConstraintId;
fn next(&mut self) -> Option<Self::Item> {
self.wrapped.next().map(ScopedConstraintId::from_u32)
}
}
impl std::iter::FusedIterator for ConstraintIdIterator<'_> {}
/// A single declaration (as [`ScopedDefinitionId`]) with a corresponding visibility
/// visibility constraint ([`ScopedVisibilityConstraintId`]).
#[derive(Debug)]
pub(super) struct DeclarationIdWithConstraint {
pub(super) definition: ScopedDefinitionId,
pub(super) visibility_constraint: ScopedVisibilityConstraintId,
}
pub(super) struct DeclarationIdIterator<'map> {
pub(crate) declarations: DeclarationsIterator<'map>,
pub(crate) visibility_constraints: VisibilityConstraintsIterator<'map>,
}
impl Iterator for DeclarationIdIterator<'_> {
type Item = DeclarationIdWithConstraint;
fn next(&mut self) -> Option<Self::Item> {
match (self.declarations.next(), self.visibility_constraints.next()) {
(None, None) => None,
(Some(declaration), Some(&visibility_constraint)) => {
Some(DeclarationIdWithConstraint {
definition: ScopedDefinitionId::from_u32(declaration),
visibility_constraint,
})
}
// SAFETY: see above.
_ => unreachable!("declarations and visibility_constraints length mismatch"),
}
}
}
impl std::iter::FusedIterator for DeclarationIdIterator<'_> {}
#[cfg(test)]
mod tests {
use super::*;
@@ -515,16 +398,16 @@ mod tests {
let actual = symbol
.bindings()
.iter()
.map(|def_id_with_constraints| {
let def_id = def_id_with_constraints.definition;
.map(|live_binding| {
let def_id = live_binding.binding;
let def = if def_id == ScopedDefinitionId::UNBOUND {
"unbound".into()
} else {
def_id.as_u32().to_string()
};
let constraints = def_id_with_constraints
.constraint_ids
.map(ScopedConstraintId::as_u32)
let constraints = live_binding
.narrowing_constraints
.iter()
.map(|idx| idx.to_string())
.collect::<Vec<_>>()
.join(", ");
@@ -540,14 +423,14 @@ mod tests {
.declarations()
.iter()
.map(
|DeclarationIdWithConstraint {
definition,
|LiveDeclaration {
declaration,
visibility_constraint: _,
}| {
if definition == ScopedDefinitionId::UNBOUND {
if *declaration == ScopedDefinitionId::UNBOUND {
"undeclared".into()
} else {
definition.as_u32().to_string()
declaration.as_u32().to_string()
}
},
)

View File

@@ -1,10 +1,9 @@
use ruff_db::files::File;
use ruff_python_ast as ast;
use crate::module_resolver::file_to_module;
use crate::semantic_index::definition::Definition;
use crate::semantic_index::symbol::{ScopeId, ScopedSymbolId};
use crate::semantic_index::{self, global_scope, use_def_map, DeclarationWithConstraint};
use crate::semantic_index::{global_scope, use_def_map, DeclarationWithConstraint};
use crate::semantic_index::{
symbol_table, BindingWithConstraints, BindingWithConstraintsIterator, DeclarationsIterator,
};
@@ -14,6 +13,8 @@ use crate::types::{
};
use crate::{resolve_module, Db, KnownModule, Module, Program};
pub(crate) use implicit_globals::module_type_implicit_global_symbol;
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub(crate) enum Boundness {
Bound,
@@ -183,20 +184,34 @@ pub(crate) fn symbol<'db>(db: &'db dyn Db, scope: ScopeId<'db>, name: &str) -> S
symbol_impl(db, scope, name, RequiresExplicitReExport::No)
}
/// Infers the public type of a module-global symbol as seen from within the same file.
/// Infers the public type of an explicit module-global symbol as seen from within the same file.
///
/// If it's not defined explicitly in the global scope, it will look it up in `types.ModuleType`
/// with a few very special exceptions.
/// Note that all global scopes also include various "implicit globals" such as `__name__`,
/// `__doc__` and `__file__`. This function **does not** consider those symbols; it will return
/// `Symbol::Unbound` for them. Use the (currently test-only) `global_symbol` query to also include
/// those additional symbols.
///
/// Use [`imported_symbol`] to perform the lookup as seen from outside the file (e.g. via imports).
pub(crate) fn global_symbol<'db>(db: &'db dyn Db, file: File, name: &str) -> Symbol<'db> {
pub(crate) fn explicit_global_symbol<'db>(db: &'db dyn Db, file: File, name: &str) -> Symbol<'db> {
symbol_impl(
db,
global_scope(db, file),
name,
RequiresExplicitReExport::No,
)
.or_fall_back_to(db, || module_type_symbol(db, name))
}
/// Infers the public type of an explicit module-global symbol as seen from within the same file.
///
/// Unlike [`explicit_global_symbol`], this function also considers various "implicit globals"
/// such as `__name__`, `__doc__` and `__file__`. These are looked up as attributes on `types.ModuleType`
/// rather than being looked up as symbols explicitly defined/declared in the global scope.
///
/// Use [`imported_symbol`] to perform the lookup as seen from outside the file (e.g. via imports).
#[cfg(test)]
pub(crate) fn global_symbol<'db>(db: &'db dyn Db, file: File, name: &str) -> Symbol<'db> {
explicit_global_symbol(db, file, name)
.or_fall_back_to(db, || module_type_implicit_global_symbol(db, name))
}
/// Infers the public type of an imported symbol.
@@ -204,16 +219,16 @@ pub(crate) fn imported_symbol<'db>(db: &'db dyn Db, module: &Module, name: &str)
// If it's not found in the global scope, check if it's present as an instance on
// `types.ModuleType` or `builtins.object`.
//
// We do a more limited version of this in `global_symbol`, but there are two crucial
// differences here:
// We do a more limited version of this in `module_type_implicit_global_symbol`,
// but there are two crucial differences here:
// - If a member is looked up as an attribute, `__init__` is also available on the module, but
// it isn't available as a global from inside the module
// - If a member is looked up as an attribute, members on `builtins.object` are also available
// (because `types.ModuleType` inherits from `object`); these attributes are also not
// available as globals from inside the module.
//
// The same way as in `global_symbol`, however, we need to be careful to ignore
// `__getattr__`. Typeshed has a fake `__getattr__` on `types.ModuleType` to help out with
// The same way as in `module_type_implicit_global_symbol`, however, we need to be careful to
// ignore `__getattr__`. Typeshed has a fake `__getattr__` on `types.ModuleType` to help out with
// dynamic imports; we shouldn't use it for `ModuleLiteral` types where we know exactly which
// module we're dealing with.
external_symbol_impl(db, module.file(), name).or_fall_back_to(db, || {
@@ -239,7 +254,7 @@ pub(crate) fn builtins_symbol<'db>(db: &'db dyn Db, symbol: &str) -> Symbol<'db>
// We're looking up in the builtins namespace and not the module, so we should
// do the normal lookup in `types.ModuleType` and not the special one as in
// `imported_symbol`.
module_type_symbol(db, symbol)
module_type_implicit_global_symbol(db, symbol)
})
})
.unwrap_or(Symbol::Unbound)
@@ -293,7 +308,7 @@ fn core_module_scope(db: &dyn Db, core_module: KnownModule) -> Option<ScopeId<'_
/// together with boundness information in a [`Symbol`].
///
/// The type will be a union if there are multiple bindings with different types.
pub(crate) fn symbol_from_bindings<'db>(
pub(super) fn symbol_from_bindings<'db>(
db: &'db dyn Db,
bindings_with_constraints: BindingWithConstraintsIterator<'_, 'db>,
) -> Symbol<'db> {
@@ -479,11 +494,16 @@ fn symbol_impl<'db>(
}
/// Implementation of [`symbol_from_bindings`].
///
/// ## Implementation Note
/// This function gets called cross-module. It, therefore, shouldn't
/// access any AST nodes from the file containing the declarations.
fn symbol_from_bindings_impl<'db>(
db: &'db dyn Db,
bindings_with_constraints: BindingWithConstraintsIterator<'_, 'db>,
requires_explicit_reexport: RequiresExplicitReExport,
) -> Symbol<'db> {
let constraints = bindings_with_constraints.constraints;
let visibility_constraints = bindings_with_constraints.visibility_constraints;
let mut bindings_with_constraints = bindings_with_constraints.peekable();
@@ -495,9 +515,9 @@ fn symbol_from_bindings_impl<'db>(
Some(BindingWithConstraints {
binding,
visibility_constraint,
constraints: _,
narrowing_constraints: _,
}) if binding.map_or(true, is_non_exported) => {
visibility_constraints.evaluate(db, *visibility_constraint)
visibility_constraints.evaluate(db, constraints, *visibility_constraint)
}
_ => Truthiness::AlwaysFalse,
};
@@ -505,7 +525,7 @@ fn symbol_from_bindings_impl<'db>(
let mut types = bindings_with_constraints.filter_map(
|BindingWithConstraints {
binding,
constraints,
narrowing_constraints,
visibility_constraint,
}| {
let binding = binding?;
@@ -514,13 +534,14 @@ fn symbol_from_bindings_impl<'db>(
return None;
}
let static_visibility = visibility_constraints.evaluate(db, visibility_constraint);
let static_visibility =
visibility_constraints.evaluate(db, constraints, visibility_constraint);
if static_visibility.is_always_false() {
return None;
}
let mut constraint_tys = constraints
let mut constraint_tys = narrowing_constraints
.filter_map(|constraint| narrowing_constraint(db, constraint, binding))
.peekable();
@@ -562,11 +583,16 @@ fn symbol_from_bindings_impl<'db>(
}
/// Implementation of [`symbol_from_declarations`].
///
/// ## Implementation Note
/// This function gets called cross-module. It, therefore, shouldn't
/// access any AST nodes from the file containing the declarations.
fn symbol_from_declarations_impl<'db>(
db: &'db dyn Db,
declarations: DeclarationsIterator<'_, 'db>,
requires_explicit_reexport: RequiresExplicitReExport,
) -> SymbolFromDeclarationsResult<'db> {
let constraints = declarations.constraints;
let visibility_constraints = declarations.visibility_constraints;
let mut declarations = declarations.peekable();
@@ -579,7 +605,7 @@ fn symbol_from_declarations_impl<'db>(
declaration,
visibility_constraint,
}) if declaration.map_or(true, is_non_exported) => {
visibility_constraints.evaluate(db, *visibility_constraint)
visibility_constraints.evaluate(db, constraints, *visibility_constraint)
}
_ => Truthiness::AlwaysFalse,
};
@@ -595,7 +621,8 @@ fn symbol_from_declarations_impl<'db>(
return None;
}
let static_visibility = visibility_constraints.evaluate(db, visibility_constraint);
let static_visibility =
visibility_constraints.evaluate(db, constraints, visibility_constraint);
if static_visibility.is_always_false() {
None
@@ -650,63 +677,106 @@ fn symbol_from_declarations_impl<'db>(
}
}
/// Return a list of the symbols that typeshed declares in the body scope of
/// the stub for the class `types.ModuleType`.
///
/// Conceptually this could be a `Set` rather than a list,
/// but the number of symbols declared in this scope is likely to be very small,
/// so the cost of hashing the names is likely to be more expensive than it's worth.
#[salsa::tracked(return_ref)]
fn module_type_symbols<'db>(db: &'db dyn Db) -> smallvec::SmallVec<[ast::name::Name; 8]> {
let Some(module_type) = KnownClass::ModuleType
.to_class_literal(db)
.into_class_literal()
else {
// The most likely way we get here is if a user specified a `--custom-typeshed-dir`
// without a `types.pyi` stub in the `stdlib/` directory
return smallvec::SmallVec::default();
};
mod implicit_globals {
use ruff_python_ast as ast;
let module_type_scope = module_type.body_scope(db);
let module_type_symbol_table = symbol_table(db, module_type_scope);
use crate::db::Db;
use crate::semantic_index::{self, symbol_table};
use crate::types::KnownClass;
// `__dict__` and `__init__` are very special members that can be accessed as attributes
// on the module when imported, but cannot be accessed as globals *inside* the module.
//
// `__getattr__` is even more special: it doesn't exist at runtime, but typeshed includes it
// to reduce false positives associated with functions that dynamically import modules
// and return `Instance(types.ModuleType)`. We should ignore it for any known module-literal type.
module_type_symbol_table
.symbols()
.filter(|symbol| symbol.is_declared())
.map(semantic_index::symbol::Symbol::name)
.filter(|symbol_name| !matches!(&***symbol_name, "__dict__" | "__getattr__" | "__init__"))
.cloned()
.collect()
}
use super::Symbol;
/// Return the symbol for a member of `types.ModuleType`.
///
/// ## Notes
///
/// In general we wouldn't check to see whether a symbol exists on a class before doing the
/// [`member`] call on the instance type -- we'd just do the [`member`] call on the instance
/// type, since it has the same end result. The reason to only call [`member`] on [`ModuleType`]
/// instance when absolutely necessary is that it was a fairly significant performance regression
/// to fallback to doing that for every name lookup that wasn't found in the module's globals
/// ([`global_symbol`]). So we use less idiomatic (and much more verbose) code here as a
/// micro-optimisation because it's used in a very hot path.
///
/// [`member`]: Type::member
/// [`ModuleType`]: KnownClass::ModuleType
fn module_type_symbol<'db>(db: &'db dyn Db, name: &str) -> Symbol<'db> {
if module_type_symbols(db)
.iter()
.any(|module_type_member| &**module_type_member == name)
{
KnownClass::ModuleType.to_instance(db).member(db, name)
} else {
Symbol::Unbound
/// Looks up the type of an "implicit global symbol". Returns [`Symbol::Unbound`] if
/// `name` is not present as an implicit symbol in module-global namespaces.
///
/// Implicit global symbols are symbols such as `__doc__`, `__name__`, and `__file__`
/// that are implicitly defined in every module's global scope. Because their type is
/// always the same, we simply look these up as instance attributes on `types.ModuleType`.
///
/// Note that this function should only be used as a fallback if a symbol is being looked
/// up in the global scope **from within the same file**. If the symbol is being looked up
/// from outside the file (e.g. via imports), use [`super::imported_symbol`] (or fallback logic
/// like the logic used in that function) instead. The reason is that this function returns
/// [`Symbol::Unbound`] for `__init__` and `__dict__` (which cannot be found in globals if
/// the lookup is being done from the same file) -- but these symbols *are* available in the
/// global scope if they're being imported **from a different file**.
pub(crate) fn module_type_implicit_global_symbol<'db>(
db: &'db dyn Db,
name: &str,
) -> Symbol<'db> {
// In general we wouldn't check to see whether a symbol exists on a class before doing the
// `.member()` call on the instance type -- we'd just do the `.member`() call on the instance
// type, since it has the same end result. The reason to only call `.member()` on `ModuleType`
// when absolutely necessary is that this function is used in a very hot path (name resolution
// in `infer.rs`). We use less idiomatic (and much more verbose) code here as a micro-optimisation.
if module_type_symbols(db)
.iter()
.any(|module_type_member| &**module_type_member == name)
{
KnownClass::ModuleType.to_instance(db).member(db, name)
} else {
Symbol::Unbound
}
}
/// An internal micro-optimisation for `module_type_implicit_global_symbol`.
///
/// This function returns a list of the symbols that typeshed declares in the
/// body scope of the stub for the class `types.ModuleType`.
///
/// The returned list excludes the attributes `__dict__` and `__init__`. These are very
/// special members that can be accessed as attributes on the module when imported,
/// but cannot be accessed as globals *inside* the module.
///
/// The list also excludes `__getattr__`. `__getattr__` is even more special: it doesn't
/// exist at runtime, but typeshed includes it to reduce false positives associated with
/// functions that dynamically import modules and return `Instance(types.ModuleType)`.
/// We should ignore it for any known module-literal type.
///
/// Conceptually this function could be a `Set` rather than a list,
/// but the number of symbols declared in this scope is likely to be very small,
/// so the cost of hashing the names is likely to be more expensive than it's worth.
#[salsa::tracked(return_ref)]
fn module_type_symbols<'db>(db: &'db dyn Db) -> smallvec::SmallVec<[ast::name::Name; 8]> {
let Some(module_type) = KnownClass::ModuleType
.to_class_literal(db)
.into_class_literal()
else {
// The most likely way we get here is if a user specified a `--custom-typeshed-dir`
// without a `types.pyi` stub in the `stdlib/` directory
return smallvec::SmallVec::default();
};
let module_type_scope = module_type.body_scope(db);
let module_type_symbol_table = symbol_table(db, module_type_scope);
module_type_symbol_table
.symbols()
.filter(|symbol| symbol.is_declared())
.map(semantic_index::symbol::Symbol::name)
.filter(|symbol_name| {
!matches!(&***symbol_name, "__dict__" | "__getattr__" | "__init__")
})
.cloned()
.collect()
}
#[cfg(test)]
mod tests {
use super::*;
use crate::db::tests::setup_db;
#[test]
fn module_type_symbols_includes_declared_types_but_not_referenced_types() {
let db = setup_db();
let symbol_names = module_type_symbols(&db);
let dunder_name_symbol_name = ast::name::Name::new_static("__name__");
assert!(symbol_names.contains(&dunder_name_symbol_name));
let property_symbol_name = ast::name::Name::new_static("property");
assert!(!symbol_names.contains(&property_symbol_name));
}
}
}
@@ -820,15 +890,36 @@ mod tests {
);
}
#[track_caller]
fn assert_bound_string_symbol<'db>(db: &'db dyn Db, symbol: Symbol<'db>) {
assert!(matches!(
symbol,
Symbol::Type(Type::Instance(_), Boundness::Bound)
));
assert_eq!(symbol.expect_type(), KnownClass::Str.to_instance(db));
}
#[test]
fn module_type_symbols_includes_declared_types_but_not_referenced_types() {
fn implicit_builtin_globals() {
let db = setup_db();
let symbol_names = module_type_symbols(&db);
assert_bound_string_symbol(&db, builtins_symbol(&db, "__name__"));
}
let dunder_name_symbol_name = ast::name::Name::new_static("__name__");
assert!(symbol_names.contains(&dunder_name_symbol_name));
#[test]
fn implicit_typing_globals() {
let db = setup_db();
assert_bound_string_symbol(&db, typing_symbol(&db, "__name__"));
}
let property_symbol_name = ast::name::Name::new_static("property");
assert!(!symbol_names.contains(&property_symbol_name));
#[test]
fn implicit_typing_extensions_globals() {
let db = setup_db();
assert_bound_string_symbol(&db, typing_extensions_symbol(&db, "__name__"));
}
#[test]
fn implicit_sys_globals() {
let db = setup_db();
assert_bound_string_symbol(&db, known_module_symbol(&db, KnownModule::Sys, "__name__"));
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -5,6 +5,11 @@ use super::Type;
pub(crate) struct CallArguments<'a, 'db>(Vec<Argument<'a, 'db>>);
impl<'a, 'db> CallArguments<'a, 'db> {
/// Create a [`CallArguments`] with no arguments.
pub(crate) fn none() -> Self {
Self(Vec::new())
}
/// Create a [`CallArguments`] from an iterator over non-variadic positional argument types.
pub(crate) fn positional(positional_tys: impl IntoIterator<Item = Type<'db>>) -> Self {
positional_tys
@@ -29,6 +34,11 @@ impl<'a, 'db> CallArguments<'a, 'db> {
pub(crate) fn first_argument(&self) -> Option<Type<'db>> {
self.0.first().map(Argument::ty)
}
// TODO this should be eliminated in favor of [`bind_call`]
pub(crate) fn second_argument(&self) -> Option<Type<'db>> {
self.0.get(1).map(Argument::ty)
}
}
impl<'db, 'a, 'b> IntoIterator for &'b CallArguments<'a, 'db> {

View File

@@ -193,6 +193,13 @@ impl<'db> CallBinding<'db> {
}
}
pub(crate) fn three_parameter_types(&self) -> Option<(Type<'db>, Type<'db>, Type<'db>)> {
match self.parameter_types() {
[first, second, third] => Some((*first, *second, *third)),
_ => None,
}
}
fn callable_name(&self, db: &'db dyn Db) -> Option<&str> {
match self.callable_ty {
Type::FunctionLiteral(function) => Some(function.name(db)),

View File

@@ -72,6 +72,7 @@ impl<'db> ClassBase<'db> {
Type::Never
| Type::BooleanLiteral(_)
| Type::FunctionLiteral(_)
| Type::Callable(..)
| Type::BytesLiteral(_)
| Type::IntLiteral(_)
| Type::StringLiteral(_)

View File

@@ -8,8 +8,8 @@ use ruff_python_literal::escape::AsciiEscape;
use crate::types::class_base::ClassBase;
use crate::types::{
ClassLiteralType, InstanceType, IntersectionType, KnownClass, StringLiteralType, Type,
UnionType,
CallableType, ClassLiteralType, InstanceType, IntersectionType, KnownClass, StringLiteralType,
Type, UnionType,
};
use crate::Db;
use rustc_hash::FxHashMap;
@@ -88,6 +88,24 @@ impl Display for DisplayRepresentation<'_> {
},
Type::KnownInstance(known_instance) => f.write_str(known_instance.repr(self.db)),
Type::FunctionLiteral(function) => f.write_str(function.name(self.db)),
Type::Callable(CallableType::BoundMethod(bound_method)) => {
write!(
f,
"<bound method `{method}` of `{instance}`>",
method = bound_method.function(self.db).name(self.db),
instance = bound_method.self_instance(self.db).display(self.db)
)
}
Type::Callable(CallableType::MethodWrapperDunderGet(function)) => {
write!(
f,
"<method-wrapper `__get__` of `{function}`>",
function = function.name(self.db)
)
}
Type::Callable(CallableType::WrapperDescriptorDunderGet) => {
f.write_str("<wrapper-descriptor `__get__` of `function` objects>")
}
Type::Union(union) => union.display(self.db).fmt(f),
Type::Intersection(intersection) => intersection.display(self.db).fmt(f),
Type::IntLiteral(n) => n.fmt(f),

View File

@@ -50,7 +50,8 @@ use crate::semantic_index::semantic_index;
use crate::semantic_index::symbol::{FileScopeId, NodeWithScopeKind, NodeWithScopeRef, ScopeId};
use crate::semantic_index::SemanticIndex;
use crate::symbol::{
builtins_module_scope, builtins_symbol, symbol, symbol_from_bindings, symbol_from_declarations,
builtins_module_scope, builtins_symbol, explicit_global_symbol,
module_type_implicit_global_symbol, symbol, symbol_from_bindings, symbol_from_declarations,
typing_extensions_symbol, LookupError,
};
use crate::types::call::{Argument, CallArguments};
@@ -90,7 +91,7 @@ use super::slots::check_class_slots;
use super::string_annotation::{
parse_string_annotation, BYTE_STRING_TYPE_ANNOTATION, FSTRING_TYPE_ANNOTATION,
};
use super::{global_symbol, CallDunderError, ParameterExpectation, ParameterExpectations};
use super::{CallDunderError, ParameterExpectation, ParameterExpectations};
/// Infer all types for a [`ScopeId`], including all definitions and expressions in that scope.
/// Use when checking a scope, or needing to provide a type for an arbitrary expression in the
@@ -118,7 +119,7 @@ fn infer_definition_types_cycle_recovery<'db>(
) -> TypeInference<'db> {
tracing::trace!("infer_definition_types_cycle_recovery");
let mut inference = TypeInference::empty(input.scope(db));
let category = input.category(db);
let category = input.kind(db).category();
if category.is_declaration() {
inference
.declarations
@@ -198,6 +199,36 @@ pub(crate) fn infer_expression_types<'db>(
TypeInferenceBuilder::new(db, InferenceRegion::Expression(expression), index).finish()
}
/// Infers the type of an `expression` that is guaranteed to be in the same file as the calling query.
///
/// This is a small helper around [`infer_expression_types()`] to reduce the boilerplate.
/// Use [`infer_expression_type()`] if it isn't guaranteed that `expression` is in the same file to
/// avoid cross-file query dependencies.
pub(super) fn infer_same_file_expression_type<'db>(
db: &'db dyn Db,
expression: Expression<'db>,
) -> Type<'db> {
let inference = infer_expression_types(db, expression);
let scope = expression.scope(db);
inference.expression_type(expression.node_ref(db).scoped_expression_id(db, scope))
}
/// Infers the type of an expression where the expression might come from another file.
///
/// Use this over [`infer_expression_types`] if the expression might come from another file than the
/// enclosing query to avoid cross-file query dependencies.
///
/// Use [`infer_same_file_expression_type`] if it is guaranteed that `expression` is in the same
/// to avoid unnecessary salsa ingredients. This is normally the case inside the `TypeInferenceBuilder`.
#[salsa::tracked]
pub(crate) fn infer_expression_type<'db>(
db: &'db dyn Db,
expression: Expression<'db>,
) -> Type<'db> {
// It's okay to call the "same file" version here because we're inside a salsa query.
infer_same_file_expression_type(db, expression)
}
/// Infer the types for an [`Unpack`] operation.
///
/// This infers the expression type and performs structural match against the target expression
@@ -870,7 +901,7 @@ impl<'db> TypeInferenceBuilder<'db> {
}
fn add_binding(&mut self, node: AnyNodeRef, binding: Definition<'db>, ty: Type<'db>) {
debug_assert!(binding.is_binding(self.db()));
debug_assert!(binding.kind(self.db()).category().is_binding());
let use_def = self.index.use_def_map(binding.file_scope(self.db()));
let declarations = use_def.declarations_at_binding(binding);
let mut bound_ty = ty;
@@ -905,7 +936,7 @@ impl<'db> TypeInferenceBuilder<'db> {
declaration: Definition<'db>,
ty: TypeAndQualifiers<'db>,
) {
debug_assert!(declaration.is_declaration(self.db()));
debug_assert!(declaration.kind(self.db()).category().is_declaration());
let use_def = self.index.use_def_map(declaration.file_scope(self.db()));
let prior_bindings = use_def.bindings_at_declaration(declaration);
// unbound_ty is Never because for this check we don't care about unbound
@@ -935,8 +966,8 @@ impl<'db> TypeInferenceBuilder<'db> {
definition: Definition<'db>,
declared_and_inferred_ty: &DeclaredAndInferredType<'db>,
) {
debug_assert!(definition.is_binding(self.db()));
debug_assert!(definition.is_declaration(self.db()));
debug_assert!(definition.kind(self.db()).category().is_binding());
debug_assert!(definition.kind(self.db()).category().is_declaration());
let (declared_ty, inferred_ty) = match *declared_and_inferred_ty {
DeclaredAndInferredType::AreTheSame(ty) => (ty.into(), ty),
@@ -3541,7 +3572,7 @@ impl<'db> TypeInferenceBuilder<'db> {
}
Symbol::Unbound
// No nonlocal binding? Check the module's globals.
// No nonlocal binding? Check the module's explicit globals.
// Avoid infinite recursion if `self.scope` already is the module's global scope.
.or_fall_back_to(db, || {
if file_scope_id.is_global() {
@@ -3558,8 +3589,12 @@ impl<'db> TypeInferenceBuilder<'db> {
}
}
global_symbol(db, self.file(), symbol_name)
explicit_global_symbol(db, self.file(), symbol_name)
})
// Not found in the module's explicitly declared global symbols?
// Check the "implicit globals" such as `__doc__`, `__file__`, `__name__`, etc.
// These are looked up as attributes on `types.ModuleType`.
.or_fall_back_to(db, || module_type_implicit_global_symbol(db, symbol_name))
// Not found in globals? Fallback to builtins
// (without infinite recursion if we're already in builtins.)
.or_fall_back_to(db, || {
@@ -3723,6 +3758,7 @@ impl<'db> TypeInferenceBuilder<'db> {
(
op @ (ast::UnaryOp::UAdd | ast::UnaryOp::USub | ast::UnaryOp::Invert),
Type::FunctionLiteral(_)
| Type::Callable(..)
| Type::ModuleLiteral(_)
| Type::ClassLiteral(_)
| Type::SubclassOf(_)
@@ -3750,7 +3786,7 @@ impl<'db> TypeInferenceBuilder<'db> {
match operand_type.try_call_dunder(
self.db(),
unary_dunder_method,
&CallArguments::positional([operand_type]),
&CallArguments::none(),
) {
Ok(outcome) => outcome.return_type(self.db()),
Err(e) => {
@@ -3937,6 +3973,7 @@ impl<'db> TypeInferenceBuilder<'db> {
// fall back on looking for dunder methods on one of the operand types.
(
Type::FunctionLiteral(_)
| Type::Callable(..)
| Type::ModuleLiteral(_)
| Type::ClassLiteral(_)
| Type::SubclassOf(_)
@@ -3953,6 +3990,7 @@ impl<'db> TypeInferenceBuilder<'db> {
| Type::SliceLiteral(_)
| Type::Tuple(_),
Type::FunctionLiteral(_)
| Type::Callable(..)
| Type::ModuleLiteral(_)
| Type::ClassLiteral(_)
| Type::SubclassOf(_)
@@ -3999,7 +4037,7 @@ impl<'db> TypeInferenceBuilder<'db> {
.try_call_dunder(
self.db(),
reflected_dunder,
&CallArguments::positional([right_ty, left_ty]),
&CallArguments::positional([left_ty]),
)
.map(|outcome| outcome.return_type(self.db()))
.or_else(|_| {
@@ -4007,7 +4045,7 @@ impl<'db> TypeInferenceBuilder<'db> {
.try_call_dunder(
self.db(),
op.dunder(),
&CallArguments::positional([left_ty, right_ty]),
&CallArguments::positional([right_ty]),
)
.map(|outcome| outcome.return_type(self.db()))
})
@@ -4923,7 +4961,7 @@ impl<'db> TypeInferenceBuilder<'db> {
match value_ty.try_call_dunder(
self.db(),
"__getitem__",
&CallArguments::positional([value_ty, slice_ty]),
&CallArguments::positional([slice_ty]),
) {
Ok(outcome) => return outcome.return_type(self.db()),
Err(err @ CallDunderError::PossiblyUnbound { .. }) => {
@@ -6215,6 +6253,7 @@ mod tests {
use crate::semantic_index::definition::Definition;
use crate::semantic_index::symbol::FileScopeId;
use crate::semantic_index::{global_scope, semantic_index, symbol_table, use_def_map};
use crate::symbol::global_symbol;
use crate::types::check_types;
use ruff_db::files::{system_path_to_file, File};
use ruff_db::system::DbWithTestSystem;
@@ -6626,4 +6665,93 @@ mod tests {
Ok(())
}
/// This test verifies that changing a class's declaration in a non-meaningful way (e.g. by adding a comment)
/// doesn't trigger type inference for expressions that depend on the class's members.
#[test]
fn dependency_own_instance_member() -> anyhow::Result<()> {
fn x_rhs_expression(db: &TestDb) -> Expression<'_> {
let file_main = system_path_to_file(db, "/src/main.py").unwrap();
let ast = parsed_module(db, file_main);
// Get the second statement in `main.py` (x = …) and extract the expression
// node on the right-hand side:
let x_rhs_node = &ast.syntax().body[1].as_assign_stmt().unwrap().value;
let index = semantic_index(db, file_main);
index.expression(x_rhs_node.as_ref())
}
let mut db = setup_db();
db.write_dedented(
"/src/mod.py",
r#"
class C:
if random.choice([True, False]):
attr: int = 42
else:
attr: None = None
"#,
)?;
db.write_dedented(
"/src/main.py",
r#"
from mod import C
x = C().attr
"#,
)?;
let file_main = system_path_to_file(&db, "/src/main.py").unwrap();
let attr_ty = global_symbol(&db, file_main, "x").expect_type();
assert_eq!(attr_ty.display(&db).to_string(), "Unknown | int | None");
// Change the type of `attr` to `str | None`; this should trigger the type of `x` to be re-inferred
db.write_dedented(
"/src/mod.py",
r#"
class C:
if random.choice([True, False]):
attr: str = "42"
else:
attr: None = None
"#,
)?;
let events = {
db.clear_salsa_events();
let attr_ty = global_symbol(&db, file_main, "x").expect_type();
assert_eq!(attr_ty.display(&db).to_string(), "Unknown | str | None");
db.take_salsa_events()
};
assert_function_query_was_run(&db, infer_expression_types, x_rhs_expression(&db), &events);
// Add a comment; this should not trigger the type of `x` to be re-inferred
db.write_dedented(
"/src/mod.py",
r#"
class C:
# comment
if random.choice([True, False]):
attr: str = "42"
else:
attr: None = None
"#,
)?;
let events = {
db.clear_salsa_events();
let attr_ty = global_symbol(&db, file_main, "x").expect_type();
assert_eq!(attr_ty.display(&db).to_string(), "Unknown | str | None");
db.take_salsa_events()
};
assert_function_query_was_not_run(
&db,
infer_expression_types,
x_rhs_expression(&db),
&events,
);
Ok(())
}
}

View File

@@ -6,6 +6,7 @@ use crate::semantic_index::definition::Definition;
use crate::semantic_index::expression::Expression;
use crate::semantic_index::symbol::{ScopeId, ScopedSymbolId, SymbolTable};
use crate::semantic_index::symbol_table;
use crate::types::infer::infer_same_file_expression_type;
use crate::types::{
infer_expression_types, ClassLiteralType, IntersectionBuilder, KnownClass, KnownFunction,
SubclassOfType, Truthiness, Type, UnionBuilder,
@@ -497,11 +498,8 @@ impl<'db> NarrowingConstraintsBuilder<'db> {
if let Some(ast::ExprName { id, .. }) = subject.node_ref(self.db).as_name_expr() {
// SAFETY: we should always have a symbol for every Name node.
let symbol = self.symbols().symbol_id_by_name(id).unwrap();
let scope = self.scope();
let inference = infer_expression_types(self.db, cls);
let ty = inference
.expression_type(cls.node_ref(self.db).scoped_expression_id(self.db, scope))
.to_instance(self.db);
let ty = infer_same_file_expression_type(self.db, cls).to_instance(self.db);
let mut constraints = NarrowingConstraints::default();
constraints.insert(symbol, ty);
Some(constraints)

View File

@@ -29,9 +29,10 @@ use std::sync::{Arc, Mutex, MutexGuard, OnceLock};
use crate::db::tests::{setup_db, TestDb};
use crate::symbol::{builtins_symbol, known_module_symbol};
use crate::types::{
IntersectionBuilder, KnownClass, KnownInstanceType, SubclassOfType, TupleType, Type, UnionType,
BoundMethodType, CallableType, IntersectionBuilder, KnownClass, KnownInstanceType,
SubclassOfType, TupleType, Type, UnionType,
};
use crate::KnownModule;
use crate::{Db, KnownModule};
use quickcheck::{Arbitrary, Gen};
/// A test representation of a type that can be transformed unambiguously into a real Type,
@@ -67,6 +68,24 @@ pub(crate) enum Ty {
SubclassOfAbcClass(&'static str),
AlwaysTruthy,
AlwaysFalsy,
BuiltinsFunction(&'static str),
BuiltinsBoundMethod {
class: &'static str,
method: &'static str,
},
}
#[salsa::tracked]
fn create_bound_method<'db>(
db: &'db dyn Db,
function: Type<'db>,
builtins_class: Type<'db>,
) -> Type<'db> {
Type::Callable(CallableType::BoundMethod(BoundMethodType::new(
db,
function.expect_function_literal(),
builtins_class.to_instance(db),
)))
}
impl Ty {
@@ -123,6 +142,13 @@ impl Ty {
),
Ty::AlwaysTruthy => Type::AlwaysTruthy,
Ty::AlwaysFalsy => Type::AlwaysFalsy,
Ty::BuiltinsFunction(name) => builtins_symbol(db, name).expect_type(),
Ty::BuiltinsBoundMethod { class, method } => {
let builtins_class = builtins_symbol(db, class).expect_type();
let function = builtins_class.static_member(db, method).expect_type();
create_bound_method(db, function, builtins_class)
}
}
}
}
@@ -173,6 +199,16 @@ fn arbitrary_core_type(g: &mut Gen) -> Ty {
Ty::SubclassOfAbcClass("ABCMeta"),
Ty::AlwaysTruthy,
Ty::AlwaysFalsy,
Ty::BuiltinsFunction("chr"),
Ty::BuiltinsFunction("ascii"),
Ty::BuiltinsBoundMethod {
class: "str",
method: "isascii",
},
Ty::BuiltinsBoundMethod {
class: "int",
method: "bit_length",
},
])
.unwrap()
.clone()

View File

@@ -21,6 +21,13 @@ pub(crate) struct Signature<'db> {
}
impl<'db> Signature<'db> {
pub(crate) fn new(parameters: Parameters<'db>, return_ty: Option<Type<'db>>) -> Self {
Self {
parameters,
return_ty,
}
}
/// Return a todo signature: (*args: Todo, **kwargs: Todo) -> Todo
pub(crate) fn todo() -> Self {
Self {
@@ -64,6 +71,10 @@ impl<'db> Signature<'db> {
pub(crate) struct Parameters<'db>(Vec<Parameter<'db>>);
impl<'db> Parameters<'db> {
pub(crate) fn new(parameters: impl IntoIterator<Item = Parameter<'db>>) -> Self {
Self(parameters.into_iter().collect())
}
/// Return todo parameters: (*args: Todo, **kwargs: Todo)
fn todo() -> Self {
Self(vec![
@@ -233,6 +244,18 @@ pub(crate) struct Parameter<'db> {
}
impl<'db> Parameter<'db> {
pub(crate) fn new(
name: Option<Name>,
annotated_ty: Option<Type<'db>>,
kind: ParameterKind<'db>,
) -> Self {
Self {
name,
annotated_ty,
kind,
}
}
fn from_node_and_kind(
db: &'db dyn Db,
definition: Definition<'db>,
@@ -322,7 +345,8 @@ pub(crate) enum ParameterKind<'db> {
mod tests {
use super::*;
use crate::db::tests::{setup_db, TestDb};
use crate::types::{global_symbol, FunctionType, KnownClass};
use crate::symbol::global_symbol;
use crate::types::{FunctionType, KnownClass};
use ruff_db::system::DbWithTestSystem;
#[track_caller]

View File

@@ -64,8 +64,8 @@ impl<'db> SubclassOfType<'db> {
!self.is_dynamic()
}
pub(crate) fn member(self, db: &'db dyn Db, name: &str) -> Symbol<'db> {
Type::from(self.subclass_of).member(db, name)
pub(crate) fn static_member(self, db: &'db dyn Db, name: &str) -> Symbol<'db> {
Type::from(self.subclass_of).static_member(db, name)
}
/// Return `true` if `self` is a subtype of `other`.

View File

@@ -1,5 +1,7 @@
use std::cmp::Ordering;
use crate::types::CallableType;
use super::{
class_base::ClassBase, ClassLiteralType, DynamicType, InstanceType, KnownInstanceType,
TodoType, Type,
@@ -54,6 +56,27 @@ pub(super) fn union_elements_ordering<'db>(left: &Type<'db>, right: &Type<'db>)
(Type::FunctionLiteral(_), _) => Ordering::Less,
(_, Type::FunctionLiteral(_)) => Ordering::Greater,
(
Type::Callable(CallableType::BoundMethod(left)),
Type::Callable(CallableType::BoundMethod(right)),
) => left.cmp(right),
(Type::Callable(CallableType::BoundMethod(_)), _) => Ordering::Less,
(_, Type::Callable(CallableType::BoundMethod(_))) => Ordering::Greater,
(
Type::Callable(CallableType::MethodWrapperDunderGet(left)),
Type::Callable(CallableType::MethodWrapperDunderGet(right)),
) => left.cmp(right),
(Type::Callable(CallableType::MethodWrapperDunderGet(_)), _) => Ordering::Less,
(_, Type::Callable(CallableType::MethodWrapperDunderGet(_))) => Ordering::Greater,
(
Type::Callable(CallableType::WrapperDescriptorDunderGet),
Type::Callable(CallableType::WrapperDescriptorDunderGet),
) => Ordering::Equal,
(Type::Callable(CallableType::WrapperDescriptorDunderGet), _) => Ordering::Less,
(_, Type::Callable(CallableType::WrapperDescriptorDunderGet)) => Ordering::Greater,
(Type::Tuple(left), Type::Tuple(right)) => left.cmp(right),
(Type::Tuple(_), _) => Ordering::Less,
(_, Type::Tuple(_)) => Ordering::Greater,

View File

@@ -178,11 +178,10 @@ use std::cmp::Ordering;
use ruff_index::{Idx, IndexVec};
use rustc_hash::FxHashMap;
use crate::semantic_index::{
ast_ids::HasScopedExpressionId,
constraint::{Constraint, ConstraintNode, PatternConstraintKind},
use crate::semantic_index::constraint::{
Constraint, ConstraintNode, Constraints, PatternConstraintKind, ScopedConstraintId,
};
use crate::types::{infer_expression_types, Truthiness};
use crate::types::{infer_expression_type, Truthiness};
use crate::Db;
/// A ternary formula that defines under what conditions a binding is visible. (A ternary formula
@@ -234,69 +233,15 @@ impl std::fmt::Debug for ScopedVisibilityConstraintId {
#[derive(Clone, Copy, Debug, Eq, Hash, PartialEq)]
struct InteriorNode {
atom: Atom,
/// A "variable" that is evaluated as part of a TDD ternary function. For visibility
/// constraints, this is a `Constraint` that represents some runtime property of the Python
/// code that we are evaluating.
atom: ScopedConstraintId,
if_true: ScopedVisibilityConstraintId,
if_ambiguous: ScopedVisibilityConstraintId,
if_false: ScopedVisibilityConstraintId,
}
/// A "variable" that is evaluated as part of a TDD ternary function. For visibility constraints,
/// this is a `Constraint` that represents some runtime property of the Python code that we are
/// evaluating. We intern these constraints in an arena ([`VisibilityConstraints::constraints`]).
/// An atom is then an index into this arena.
///
/// By using a 32-bit index, we would typically allow 4 billion distinct constraints within a
/// scope. However, we sometimes have to model how a `Constraint` can have a different runtime
/// value at different points in the execution of the program. To handle this, we reserve the top
/// byte of an atom to represent a "copy number". This is just an opaque value that allows
/// different `Atom`s to evaluate the same `Constraint`. This yields a maximum of 16 million
/// distinct `Constraint`s in a scope, and 256 possible copies of each of those constraints.
#[derive(Clone, Copy, Eq, Hash, Ord, PartialEq, PartialOrd)]
struct Atom(u32);
impl Atom {
/// Deconstruct an atom into a constraint index and a copy number.
#[inline]
fn into_index_and_copy(self) -> (u32, u8) {
let copy = self.0 >> 24;
let index = self.0 & 0x00ff_ffff;
(index, copy as u8)
}
#[inline]
fn copy_of(mut self, copy: u8) -> Self {
// Clear out the previous copy number
self.0 &= 0x00ff_ffff;
// OR in the new one
self.0 |= u32::from(copy) << 24;
self
}
}
// A custom Debug implementation that prints out the constraint index and copy number as distinct
// fields.
impl std::fmt::Debug for Atom {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
let (index, copy) = self.into_index_and_copy();
f.debug_tuple("Atom").field(&index).field(&copy).finish()
}
}
impl Idx for Atom {
#[inline]
fn new(value: usize) -> Self {
assert!(value <= 0x00ff_ffff);
#[allow(clippy::cast_possible_truncation)]
Self(value as u32)
}
#[inline]
fn index(self) -> usize {
let (index, _) = self.into_index_and_copy();
index as usize
}
}
impl ScopedVisibilityConstraintId {
/// A special ID that is used for an "always true" / "always visible" constraint.
pub(crate) const ALWAYS_TRUE: ScopedVisibilityConstraintId =
@@ -339,16 +284,13 @@ const SMALLEST_TERMINAL: ScopedVisibilityConstraintId = ALWAYS_FALSE;
/// A collection of visibility constraints. This is currently stored in `UseDefMap`, which means we
/// maintain a separate set of visibility constraints for each scope in file.
#[derive(Debug, PartialEq, Eq, salsa::Update)]
pub(crate) struct VisibilityConstraints<'db> {
constraints: IndexVec<Atom, Constraint<'db>>,
pub(crate) struct VisibilityConstraints {
interiors: IndexVec<ScopedVisibilityConstraintId, InteriorNode>,
}
#[derive(Debug, Default, PartialEq, Eq)]
pub(crate) struct VisibilityConstraintsBuilder<'db> {
constraints: IndexVec<Atom, Constraint<'db>>,
pub(crate) struct VisibilityConstraintsBuilder {
interiors: IndexVec<ScopedVisibilityConstraintId, InteriorNode>,
constraint_cache: FxHashMap<Constraint<'db>, Atom>,
interior_cache: FxHashMap<InteriorNode, ScopedVisibilityConstraintId>,
not_cache: FxHashMap<ScopedVisibilityConstraintId, ScopedVisibilityConstraintId>,
and_cache: FxHashMap<
@@ -361,10 +303,9 @@ pub(crate) struct VisibilityConstraintsBuilder<'db> {
>,
}
impl<'db> VisibilityConstraintsBuilder<'db> {
pub(crate) fn build(self) -> VisibilityConstraints<'db> {
impl VisibilityConstraintsBuilder {
pub(crate) fn build(self) -> VisibilityConstraints {
VisibilityConstraints {
constraints: self.constraints,
interiors: self.interiors,
}
}
@@ -388,14 +329,6 @@ impl<'db> VisibilityConstraintsBuilder<'db> {
}
}
/// Adds a constraint, ensuring that we only store any particular constraint once.
fn add_constraint(&mut self, constraint: Constraint<'db>, copy: u8) -> Atom {
self.constraint_cache
.entry(constraint)
.or_insert_with(|| self.constraints.push(constraint))
.copy_of(copy)
}
/// Adds an interior node, ensuring that we always use the same visibility constraint ID for
/// equal nodes.
fn add_interior(&mut self, node: InteriorNode) -> ScopedVisibilityConstraintId {
@@ -411,17 +344,23 @@ impl<'db> VisibilityConstraintsBuilder<'db> {
.or_insert_with(|| self.interiors.push(node))
}
/// Adds a new visibility constraint that checks a single [`Constraint`]. Provide different
/// values for `copy` if you need to model that the constraint can evaluate to different
/// results at different points in the execution of the program being modeled.
/// Adds a new visibility constraint that checks a single [`Constraint`].
///
/// [`ScopedConstraintId`]s are the “variables” that are evaluated by a TDD. A TDD variable has
/// the same value no matter how many times it appears in the ternary formula that the TDD
/// represents.
///
/// However, we sometimes have to model how a `Constraint` can have a different runtime
/// value at different points in the execution of the program. To handle this, you can take
/// advantage of the fact that the [`Constraints`] arena does not deduplicate `Constraint`s.
/// You can add a `Constraint` multiple times, yielding different `ScopedConstraintId`s, which
/// you can then create separate TDD atoms for.
pub(crate) fn add_atom(
&mut self,
constraint: Constraint<'db>,
copy: u8,
constraint: ScopedConstraintId,
) -> ScopedVisibilityConstraintId {
let atom = self.add_constraint(constraint, copy);
self.add_interior(InteriorNode {
atom,
atom: constraint,
if_true: ALWAYS_TRUE,
if_ambiguous: AMBIGUOUS,
if_false: ALWAYS_FALSE,
@@ -591,11 +530,12 @@ impl<'db> VisibilityConstraintsBuilder<'db> {
}
}
impl<'db> VisibilityConstraints<'db> {
impl VisibilityConstraints {
/// Analyze the statically known visibility for a given visibility constraint.
pub(crate) fn evaluate(
pub(crate) fn evaluate<'db>(
&self,
db: &'db dyn Db,
constraints: &Constraints<'db>,
mut id: ScopedVisibilityConstraintId,
) -> Truthiness {
loop {
@@ -605,7 +545,7 @@ impl<'db> VisibilityConstraints<'db> {
ALWAYS_FALSE => return Truthiness::AlwaysFalse,
_ => self.interiors[id],
};
let constraint = &self.constraints[node.atom];
let constraint = &constraints[node.atom];
match Self::analyze_single(db, constraint) {
Truthiness::AlwaysTrue => id = node.if_true,
Truthiness::Ambiguous => id = node.if_ambiguous,
@@ -617,28 +557,14 @@ impl<'db> VisibilityConstraints<'db> {
fn analyze_single(db: &dyn Db, constraint: &Constraint) -> Truthiness {
match constraint.node {
ConstraintNode::Expression(test_expr) => {
let inference = infer_expression_types(db, test_expr);
let scope = test_expr.scope(db);
let ty = inference
.expression_type(test_expr.node_ref(db).scoped_expression_id(db, scope));
let ty = infer_expression_type(db, test_expr);
ty.bool(db).negate_if(!constraint.is_positive)
}
ConstraintNode::Pattern(inner) => match inner.kind(db) {
PatternConstraintKind::Value(value, guard) => {
let subject_expression = inner.subject(db);
let inference = infer_expression_types(db, subject_expression);
let scope = subject_expression.scope(db);
let subject_ty = inference.expression_type(
subject_expression
.node_ref(db)
.scoped_expression_id(db, scope),
);
let inference = infer_expression_types(db, *value);
let scope = value.scope(db);
let value_ty = inference
.expression_type(value.node_ref(db).scoped_expression_id(db, scope));
let subject_ty = infer_expression_type(db, subject_expression);
let value_ty = infer_expression_type(db, *value);
if subject_ty.is_single_valued(db) {
let truthiness =

View File

@@ -22,6 +22,7 @@ ruff_python_codegen = { workspace = true }
ruff_python_formatter = { workspace = true }
ruff_python_parser = { workspace = true }
ruff_python_trivia = { workspace = true }
ruff_server = { workspace = true }
ruff_workspace = { workspace = true, features = ["schemars"] }
anyhow = { workspace = true }

View File

@@ -2,6 +2,7 @@
//!
//! Used for <https://docs.astral.sh/ruff/settings/>.
use itertools::Itertools;
use ruff_server::ClientSettings;
use std::fmt::Write;
use ruff_python_trivia::textwrap;
@@ -15,12 +16,32 @@ pub(crate) fn generate() -> String {
&mut output,
Set::Toplevel(Options::metadata()),
&mut Vec::new(),
SetKind::Ruff,
);
output
}
fn generate_set(output: &mut String, set: Set, parents: &mut Vec<Set>) {
pub(crate) fn generate_server_options() -> String {
let mut output = String::new();
generate_set(
&mut output,
Set::Toplevel(ClientSettings::metadata()),
&mut Vec::new(),
SetKind::RuffServer,
);
output
}
#[derive(Copy, Clone)]
enum SetKind {
Ruff,
RuffServer,
}
fn generate_set(output: &mut String, set: Set, parents: &mut Vec<Set>, set_kind: SetKind) {
match &set {
Set::Toplevel(_) => {
output.push_str("### Top-level\n");
@@ -53,7 +74,7 @@ fn generate_set(output: &mut String, set: Set, parents: &mut Vec<Set>) {
// Generate the fields.
for (name, field) in &fields {
emit_field(output, name, field, parents.as_slice());
emit_field(output, name, field, parents.as_slice(), set_kind);
output.push_str("---\n\n");
}
@@ -66,6 +87,7 @@ fn generate_set(output: &mut String, set: Set, parents: &mut Vec<Set>) {
set: *sub_set,
},
parents,
set_kind,
);
}
@@ -93,7 +115,13 @@ impl Set {
}
}
fn emit_field(output: &mut String, name: &str, field: &OptionField, parents: &[Set]) {
fn emit_field(
output: &mut String,
name: &str,
field: &OptionField,
parents: &[Set],
set_kind: SetKind,
) {
let header_level = if parents.is_empty() { "####" } else { "#####" };
let parents_anchor = parents.iter().filter_map(|parent| parent.name()).join("_");
@@ -137,28 +165,46 @@ fn emit_field(output: &mut String, name: &str, field: &OptionField, parents: &[S
output.push_str(&format!("**Type**: `{}`\n", field.value_type));
output.push('\n');
output.push_str("**Example usage**:\n\n");
output.push_str(&format_tab(
"pyproject.toml",
&format_header(field.scope, parents, ConfigurationFile::PyprojectToml),
field.example,
));
output.push_str(&format_tab(
"ruff.toml",
&format_header(field.scope, parents, ConfigurationFile::RuffToml),
field.example,
));
match set_kind {
SetKind::Ruff => {
output.push_str(&format_tab(
"pyproject.toml",
&format_content(field, parents, ConfigurationFile::PyprojectToml),
));
output.push_str(&format_tab(
"ruff.toml",
&format_content(field, parents, ConfigurationFile::RuffToml),
));
}
SetKind::RuffServer => {}
}
output.push('\n');
}
fn format_tab(tab_name: &str, header: &str, content: &str) -> String {
fn format_tab(tab_name: &str, content: &str) -> String {
format!(
"=== \"{}\"\n\n ```toml\n {}\n{}\n ```\n",
"=== \"{}\"\n\n{}\n\n",
tab_name,
header,
textwrap::indent(content, " ")
)
}
fn format_content(
field: &OptionField,
parents: &[Set],
configuration: ConfigurationFile,
) -> String {
let header = format_header(field.scope, parents, configuration);
format!(
"```toml\n{}\n{}\n```",
header,
textwrap::indent(field.example, " ")
)
}
/// Format the TOML header for the example usage for a given option.
///
/// For example: `[tool.ruff.format]` or `[tool.ruff.lint.isort]`.
@@ -187,6 +233,15 @@ enum ConfigurationFile {
RuffToml,
}
fn format_server_content(field: &OptionField, editor: Editor) -> String {}
#[derive(Debug, Copy, Clone)]
enum Editor {
VSCode,
Neovim,
Zed,
}
#[derive(Default)]
struct CollectOptionsVisitor {
groups: Vec<(String, OptionSet)>,

View File

@@ -46,6 +46,8 @@ enum Command {
GenerateRulesTable,
/// Generate a Markdown-compatible listing of configuration options.
GenerateOptions,
/// Generate a Markdown-compatible listing of server options.
GenerateServerOptions,
/// Generate CLI help.
GenerateCliHelp(generate_cli_help::Args),
/// Generate Markdown docs.
@@ -89,6 +91,9 @@ fn main() -> Result<ExitCode> {
Command::GenerateKnotSchema(args) => generate_knot_schema::main(&args)?,
Command::GenerateRulesTable => println!("{}", generate_rules_table::generate()),
Command::GenerateOptions => println!("{}", generate_options::generate()),
Command::GenerateServerOptions => {
println!("{}", generate_options::generate_server_options())
}
Command::GenerateCliHelp(args) => generate_cli_help::main(&args)?,
Command::GenerateDocs(args) => generate_docs::main(&args)?,
Command::PrintAST(args) => print_ast::main(&args)?,

View File

@@ -68,8 +68,44 @@ class TestClass:
def __eq__(self, **kwargs): # ignore **kwargs
...
def __eq__(self, /, other=42): # ignore positional-only args
def __eq__(self, /, other=42): # support positional-only args
...
def __eq__(self, *, other=42): # ignore positional-only args
def __eq__(self, *, other=42): # ignore keyword-only args
...
def __cmp__(self): # #16217 assert non-special method is skipped, expects 2 parameters
...
def __div__(self): # #16217 assert non-special method is skipped, expects 2 parameters
...
def __nonzero__(self, x): # #16217 assert non-special method is skipped, expects 1 parameter
...
def __unicode__(self, x): # #16217 assert non-special method is skipped, expects 1 parameter
...
def __next__(self, x): # #16217 assert special method is linted, expects 1 parameter
...
def __buffer__(self): # #16217 assert special method is linted, expects 2 parameters
...
def __class_getitem__(self): # #16217 assert special method is linted, expects 2 parameters
...
def __mro_entries__(self): # #16217 assert special method is linted, expects 2 parameters
...
def __release_buffer__(self): # #16217 assert special method is linted, expects 2 parameters
...
def __subclasshook__(self): # #16217 assert special method is linted, expects 2 parameters
...
def __setattr__(self, /, name): # #16217 assert positional-only special method is linted, expects 3 parameters
...
def __setitem__(self, key, /, value, extra_value): # #16217 assert positional-only special method is linted, expects 3 parameters
...

View File

@@ -81,28 +81,42 @@ class H(BaseModel):
final_variable: Final[list[int]] = []
from pydantic.v1 import BaseModel as V1BaseModel
class I(V1BaseModel):
mutable_default: list[int] = []
from pydantic.v1.generics import GenericModel
class J(GenericModel):
mutable_default: list[int] = []
def sqlmodel_import_checker():
from sqlmodel.main import SQLModel
class I(SQLModel):
class K(SQLModel):
id: int
mutable_default: list[int] = []
from sqlmodel import SQLModel
class J(SQLModel):
class L(SQLModel):
id: int
name: str
class K(SQLModel):
class M(SQLModel):
id: int
i_s: list[J] = []
class L(SQLModel):
class N(SQLModel):
id: int
i_j: list[K] = list()
i_j: list[L] = list()
# Lint should account for deferred annotations
# See https://github.com/astral-sh/ruff/issues/15857

View File

@@ -23,10 +23,8 @@ impl ExpectedParams {
| "__neg__" | "__pos__" | "__abs__" | "__invert__" | "__complex__" | "__int__"
| "__float__" | "__index__" | "__trunc__" | "__floor__" | "__ceil__" | "__enter__"
| "__aenter__" | "__getnewargs_ex__" | "__getnewargs__" | "__getstate__"
| "__reduce__" | "__copy__" | "__unicode__" | "__nonzero__" | "__await__"
| "__aiter__" | "__anext__" | "__fspath__" | "__subclasses__" => {
Some(ExpectedParams::Fixed(0))
}
| "__reduce__" | "__copy__" | "__await__" | "__aiter__" | "__anext__"
| "__fspath__" | "__subclasses__" | "__next__" => Some(ExpectedParams::Fixed(0)),
"__format__" | "__lt__" | "__le__" | "__eq__" | "__ne__" | "__gt__" | "__ge__"
| "__getattr__" | "__getattribute__" | "__delattr__" | "__delete__"
| "__instancecheck__" | "__subclasscheck__" | "__getitem__" | "__missing__"
@@ -37,8 +35,9 @@ impl ExpectedParams {
| "__rpow__" | "__rlshift__" | "__rrshift__" | "__rand__" | "__rxor__" | "__ror__"
| "__iadd__" | "__isub__" | "__imul__" | "__itruediv__" | "__ifloordiv__"
| "__imod__" | "__ilshift__" | "__irshift__" | "__iand__" | "__ixor__" | "__ior__"
| "__ipow__" | "__setstate__" | "__reduce_ex__" | "__deepcopy__" | "__cmp__"
| "__matmul__" | "__rmatmul__" | "__imatmul__" | "__div__" => {
| "__ipow__" | "__setstate__" | "__reduce_ex__" | "__deepcopy__" | "__matmul__"
| "__rmatmul__" | "__imatmul__" | "__buffer__" | "__class_getitem__"
| "__mro_entries__" | "__release_buffer__" | "__subclasshook__" => {
Some(ExpectedParams::Fixed(1))
}
"__setattr__" | "__get__" | "__set__" | "__setitem__" | "__set_name__" => {
@@ -147,11 +146,8 @@ pub(crate) fn unexpected_special_method_signature(
return;
}
// Ignore methods with positional-only or keyword-only parameters, or variadic parameters.
if !parameters.posonlyargs.is_empty()
|| !parameters.kwonlyargs.is_empty()
|| parameters.kwarg.is_some()
{
// Ignore methods with keyword-only parameters or variadic parameters.
if !parameters.kwonlyargs.is_empty() || parameters.kwarg.is_some() {
return;
}
@@ -160,10 +156,11 @@ pub(crate) fn unexpected_special_method_signature(
return;
}
let actual_params = parameters.args.len();
let actual_params = parameters.args.len() + parameters.posonlyargs.len();
let mandatory_params = parameters
.args
.iter()
.chain(parameters.posonlyargs.iter())
.filter(|arg| arg.default.is_none())
.count();

View File

@@ -71,3 +71,75 @@ unexpected_special_method_signature.py:65:9: PLE0302 The special method `__round
| ^^^^^^^^^ PLE0302
66 | ...
|
unexpected_special_method_signature.py:89:9: PLE0302 The special method `__next__` expects 1 parameter, 2 were given
|
87 | ...
88 |
89 | def __next__(self, x): # #16217 assert special method is linted, expects 1 parameter
| ^^^^^^^^ PLE0302
90 | ...
|
unexpected_special_method_signature.py:92:9: PLE0302 The special method `__buffer__` expects 2 parameters, 1 was given
|
90 | ...
91 |
92 | def __buffer__(self): # #16217 assert special method is linted, expects 2 parameters
| ^^^^^^^^^^ PLE0302
93 | ...
|
unexpected_special_method_signature.py:95:9: PLE0302 The special method `__class_getitem__` expects 2 parameters, 1 was given
|
93 | ...
94 |
95 | def __class_getitem__(self): # #16217 assert special method is linted, expects 2 parameters
| ^^^^^^^^^^^^^^^^^ PLE0302
96 | ...
|
unexpected_special_method_signature.py:98:9: PLE0302 The special method `__mro_entries__` expects 2 parameters, 1 was given
|
96 | ...
97 |
98 | def __mro_entries__(self): # #16217 assert special method is linted, expects 2 parameters
| ^^^^^^^^^^^^^^^ PLE0302
99 | ...
|
unexpected_special_method_signature.py:101:9: PLE0302 The special method `__release_buffer__` expects 2 parameters, 1 was given
|
99 | ...
100 |
101 | def __release_buffer__(self): # #16217 assert special method is linted, expects 2 parameters
| ^^^^^^^^^^^^^^^^^^ PLE0302
102 | ...
|
unexpected_special_method_signature.py:104:9: PLE0302 The special method `__subclasshook__` expects 2 parameters, 1 was given
|
102 | ...
103 |
104 | def __subclasshook__(self): # #16217 assert special method is linted, expects 2 parameters
| ^^^^^^^^^^^^^^^^ PLE0302
105 | ...
|
unexpected_special_method_signature.py:107:9: PLE0302 The special method `__setattr__` expects 3 parameters, 2 were given
|
105 | ...
106 |
107 | def __setattr__(self, /, name): # #16217 assert positional-only special method is linted, expects 3 parameters
| ^^^^^^^^^^^ PLE0302
108 | ...
|
unexpected_special_method_signature.py:110:9: PLE0302 The special method `__setitem__` expects 3 parameters, 4 were given
|
108 | ...
109 |
110 | def __setitem__(self, key, /, value, extra_value): # #16217 assert positional-only special method is linted, expects 3 parameters
| ^^^^^^^^^^^ PLE0302
111 | ...
|

View File

@@ -165,7 +165,7 @@ pub(super) fn dataclass_kind<'a>(
/// Returns `true` if the given class has "default copy" semantics.
///
/// For example, Pydantic `BaseModel` and `BaseSettings` subclassses copy attribute defaults on
/// For example, Pydantic `BaseModel` and `BaseSettings` subclasses copy attribute defaults on
/// instance creation. As such, the use of mutable default values is safe for such classes.
pub(super) fn has_default_copy_semantics(
class_def: &ast::StmtClassDef,
@@ -174,7 +174,16 @@ pub(super) fn has_default_copy_semantics(
analyze::class::any_qualified_base_class(class_def, semantic, &|qualified_name| {
matches!(
qualified_name.segments(),
["pydantic", "BaseModel" | "BaseSettings" | "BaseConfig"]
[
"pydantic",
"BaseModel" | "RootModel" | "BaseSettings" | "BaseConfig"
] | ["pydantic", "generics", "GenericModel"]
| [
"pydantic",
"v1",
"BaseModel" | "BaseSettings" | "BaseConfig"
]
| ["pydantic", "v1", "generics", "GenericModel"]
| ["pydantic_settings", "BaseSettings"]
| ["msgspec", "Struct"]
| ["sqlmodel", "SQLModel"]

View File

@@ -31,78 +31,78 @@ RUF012.py:25:26: RUF012 Mutable class attributes should be annotated with `typin
27 | class_variable: ClassVar[list[int]] = []
|
RUF012.py:89:38: RUF012 Mutable class attributes should be annotated with `typing.ClassVar`
|
87 | class I(SQLModel):
88 | id: int
89 | mutable_default: list[int] = []
| ^^ RUF012
90 |
91 | from sqlmodel import SQLModel
|
RUF012.py:114:36: RUF012 Mutable class attributes should be annotated with `typing.ClassVar`
RUF012.py:103:38: RUF012 Mutable class attributes should be annotated with `typing.ClassVar`
|
112 | }
113 |
114 | mutable_default: 'list[int]' = []
101 | class K(SQLModel):
102 | id: int
103 | mutable_default: list[int] = []
| ^^ RUF012
104 |
105 | from sqlmodel import SQLModel
|
RUF012.py:128:36: RUF012 Mutable class attributes should be annotated with `typing.ClassVar`
|
126 | }
127 |
128 | mutable_default: 'list[int]' = []
| ^^ RUF012
115 | immutable_annotation: 'Sequence[int]'= []
116 | without_annotation = []
129 | immutable_annotation: 'Sequence[int]'= []
130 | without_annotation = []
|
RUF012.py:115:44: RUF012 Mutable class attributes should be annotated with `typing.ClassVar`
RUF012.py:129:44: RUF012 Mutable class attributes should be annotated with `typing.ClassVar`
|
114 | mutable_default: 'list[int]' = []
115 | immutable_annotation: 'Sequence[int]'= []
128 | mutable_default: 'list[int]' = []
129 | immutable_annotation: 'Sequence[int]'= []
| ^^ RUF012
116 | without_annotation = []
117 | class_variable: 'ClassVar[list[int]]' = []
130 | without_annotation = []
131 | class_variable: 'ClassVar[list[int]]' = []
|
RUF012.py:116:26: RUF012 Mutable class attributes should be annotated with `typing.ClassVar`
RUF012.py:130:26: RUF012 Mutable class attributes should be annotated with `typing.ClassVar`
|
114 | mutable_default: 'list[int]' = []
115 | immutable_annotation: 'Sequence[int]'= []
116 | without_annotation = []
128 | mutable_default: 'list[int]' = []
129 | immutable_annotation: 'Sequence[int]'= []
130 | without_annotation = []
| ^^ RUF012
117 | class_variable: 'ClassVar[list[int]]' = []
118 | final_variable: 'Final[list[int]]' = []
131 | class_variable: 'ClassVar[list[int]]' = []
132 | final_variable: 'Final[list[int]]' = []
|
RUF012.py:117:45: RUF012 Mutable class attributes should be annotated with `typing.ClassVar`
RUF012.py:131:45: RUF012 Mutable class attributes should be annotated with `typing.ClassVar`
|
115 | immutable_annotation: 'Sequence[int]'= []
116 | without_annotation = []
117 | class_variable: 'ClassVar[list[int]]' = []
129 | immutable_annotation: 'Sequence[int]'= []
130 | without_annotation = []
131 | class_variable: 'ClassVar[list[int]]' = []
| ^^ RUF012
118 | final_variable: 'Final[list[int]]' = []
119 | class_variable_without_subscript: 'ClassVar' = []
132 | final_variable: 'Final[list[int]]' = []
133 | class_variable_without_subscript: 'ClassVar' = []
|
RUF012.py:118:42: RUF012 Mutable class attributes should be annotated with `typing.ClassVar`
RUF012.py:132:42: RUF012 Mutable class attributes should be annotated with `typing.ClassVar`
|
116 | without_annotation = []
117 | class_variable: 'ClassVar[list[int]]' = []
118 | final_variable: 'Final[list[int]]' = []
130 | without_annotation = []
131 | class_variable: 'ClassVar[list[int]]' = []
132 | final_variable: 'Final[list[int]]' = []
| ^^ RUF012
119 | class_variable_without_subscript: 'ClassVar' = []
120 | final_variable_without_subscript: 'Final' = []
133 | class_variable_without_subscript: 'ClassVar' = []
134 | final_variable_without_subscript: 'Final' = []
|
RUF012.py:119:52: RUF012 Mutable class attributes should be annotated with `typing.ClassVar`
RUF012.py:133:52: RUF012 Mutable class attributes should be annotated with `typing.ClassVar`
|
117 | class_variable: 'ClassVar[list[int]]' = []
118 | final_variable: 'Final[list[int]]' = []
119 | class_variable_without_subscript: 'ClassVar' = []
131 | class_variable: 'ClassVar[list[int]]' = []
132 | final_variable: 'Final[list[int]]' = []
133 | class_variable_without_subscript: 'ClassVar' = []
| ^^ RUF012
120 | final_variable_without_subscript: 'Final' = []
134 | final_variable_without_subscript: 'Final' = []
|
RUF012.py:120:49: RUF012 Mutable class attributes should be annotated with `typing.ClassVar`
RUF012.py:134:49: RUF012 Mutable class attributes should be annotated with `typing.ClassVar`
|
118 | final_variable: 'Final[list[int]]' = []
119 | class_variable_without_subscript: 'ClassVar' = []
120 | final_variable_without_subscript: 'Final' = []
132 | final_variable: 'Final[list[int]]' = []
133 | class_variable_without_subscript: 'ClassVar' = []
134 | final_variable_without_subscript: 'Final' = []
| ^^ RUF012
|

View File

@@ -24,19 +24,22 @@ pub(crate) fn derive_impl(input: DeriveInput) -> syn::Result<TokenStream> {
}) => {
let mut output = vec![];
let rename_value =
RenameValue::from_attributes(struct_attributes.as_slice()).unwrap_or_default();
for field in &fields.named {
if let Some(attr) = field
.attrs
.iter()
.find(|attr| attr.path().is_ident("option"))
{
output.push(handle_option(field, attr)?);
output.push(handle_option(field, attr, rename_value)?);
} else if field
.attrs
.iter()
.any(|attr| attr.path().is_ident("option_group"))
{
output.push(handle_option_group(field)?);
output.push(handle_option_group(field, rename_value)?);
} else if let Some(serde) = field
.attrs
.iter()
@@ -86,8 +89,8 @@ pub(crate) fn derive_impl(input: DeriveInput) -> syn::Result<TokenStream> {
Ok(quote! {
#[automatically_derived]
impl crate::options_base::OptionsMetadata for #ident {
fn record(visit: &mut dyn crate::options_base::Visit) {
impl ruff_workspace::options_base::OptionsMetadata for #ident {
fn record(visit: &mut dyn ruff_workspace::options_base::Visit) {
#(#output);*
}
@@ -105,7 +108,10 @@ pub(crate) fn derive_impl(input: DeriveInput) -> syn::Result<TokenStream> {
/// For a field with type `Option<Foobar>` where `Foobar` itself is a struct
/// deriving `ConfigurationOptions`, create code that calls retrieves options
/// from that group: `Foobar::get_available_options()`
fn handle_option_group(field: &Field) -> syn::Result<proc_macro2::TokenStream> {
fn handle_option_group(
field: &Field,
rename_value: RenameValue,
) -> syn::Result<proc_macro2::TokenStream> {
let ident = field
.ident
.as_ref()
@@ -122,10 +128,10 @@ fn handle_option_group(field: &Field) -> syn::Result<proc_macro2::TokenStream> {
PathArguments::AngleBracketed(AngleBracketedGenericArguments { args, .. }),
}) if type_ident == "Option" => {
let path = &args[0];
let kebab_name = LitStr::new(&ident.to_string().replace('_', "-"), ident.span());
let renamed_field = rename_value.apply(ident);
Ok(quote_spanned!(
ident.span() => (visit.record_set(#kebab_name, crate::options_base::OptionSet::of::<#path>()))
ident.span() => (visit.record_set(#renamed_field, ruff_workspace::options_base::OptionSet::of::<#path>()))
))
}
_ => Err(syn::Error::new(
@@ -154,7 +160,11 @@ fn parse_doc(doc: &Attribute) -> syn::Result<String> {
/// Parse an `#[option(doc="...", default="...", value_type="...",
/// example="...")]` attribute and return data in the form of an `OptionField`.
fn handle_option(field: &Field, attr: &Attribute) -> syn::Result<proc_macro2::TokenStream> {
fn handle_option(
field: &Field,
attr: &Attribute,
rename_value: RenameValue,
) -> syn::Result<proc_macro2::TokenStream> {
let docs: Vec<&Attribute> = field
.attrs
.iter()
@@ -190,8 +200,7 @@ fn handle_option(field: &Field, attr: &Attribute) -> syn::Result<proc_macro2::To
example,
scope,
} = parse_field_attributes(attr)?;
let kebab_name = LitStr::new(&ident.to_string().replace('_', "-"), ident.span());
let renamed_field = rename_value.apply(ident);
let scope = if let Some(scope) = scope {
quote!(Some(#scope))
} else {
@@ -214,14 +223,14 @@ fn handle_option(field: &Field, attr: &Attribute) -> syn::Result<proc_macro2::To
let note = quote_option(deprecated.note);
let since = quote_option(deprecated.since);
quote!(Some(crate::options_base::Deprecated { since: #since, message: #note }))
quote!(Some(ruff_workspace::options_base::Deprecated { since: #since, message: #note }))
} else {
quote!(None)
};
Ok(quote_spanned!(
ident.span() => {
visit.record_field(#kebab_name, crate::options_base::OptionField{
visit.record_field(#renamed_field, ruff_workspace::options_base::OptionField{
doc: &#doc,
default: &#default,
value_type: &#value_type,
@@ -351,3 +360,66 @@ struct DeprecatedAttribute {
since: Option<String>,
note: Option<String>,
}
#[derive(Copy, Clone, Default)]
enum RenameValue {
#[default]
KebabCase,
CamelCase,
}
impl RenameValue {
fn from_attributes(attrs: &[Attribute]) -> Option<RenameValue> {
let serde = attrs.iter().find(|attr| attr.path().is_ident("serde"))?;
let Meta::List(list) = &serde.meta else {
return None;
};
let mut rename_value = None;
let _ = list.parse_nested_meta(|meta| {
if meta.path.is_ident("rename_all") {
let value = meta.value()?;
let s: LitStr = value.parse()?;
match s.value().as_str() {
"kebab-case" => {
rename_value = Some(RenameValue::KebabCase);
Ok(())
}
"camelCase" => {
rename_value = Some(RenameValue::CamelCase);
Ok(())
}
_ => Err(meta.error("Expected `kebab-case` or `camelCase`")),
}
} else {
Err(meta.error("Expected `rename_all`"))
}
});
rename_value
}
fn apply(self, ident: &syn::Ident) -> syn::LitStr {
let renamed = match self {
RenameValue::KebabCase => ident.to_string().replace('_', "-"),
RenameValue::CamelCase => {
let mut result = String::new();
let mut capitalize = false;
for c in ident.to_string().chars() {
if c == '_' {
capitalize = true;
} else if capitalize {
result.push(c.to_ascii_uppercase());
capitalize = false;
} else {
result.push(c);
}
}
result
}
};
LitStr::new(&renamed, ident.span())
}
}

View File

@@ -16,6 +16,7 @@ license = { workspace = true }
ruff_diagnostics = { workspace = true }
ruff_formatter = { workspace = true }
ruff_linter = { workspace = true }
ruff_macros = { workspace = true }
ruff_notebook = { workspace = true }
ruff_python_ast = { workspace = true }
ruff_python_codegen = { workspace = true }

View File

@@ -2,8 +2,9 @@
pub use edit::{DocumentKey, NotebookDocument, PositionEncoding, TextDocument};
use lsp_types::CodeActionKind;
pub use server::{Server, Workspace, Workspaces};
pub use server::Server;
pub use session::{ClientSettings, DocumentQuery, DocumentSnapshot, Session};
pub use workspace::{Workspace, Workspaces};
#[macro_use]
mod message;
@@ -16,6 +17,7 @@ mod logging;
mod resolve;
mod server;
mod session;
mod workspace;
pub(crate) const SERVER_NAME: &str = "ruff";
pub(crate) const DIAGNOSTIC_NAME: &str = "Ruff";

View File

@@ -3,14 +3,11 @@
use lsp_server as lsp;
use lsp_types as types;
use lsp_types::InitializeParams;
use lsp_types::WorkspaceFolder;
use std::num::NonZeroUsize;
use std::ops::Deref;
// The new PanicInfoHook name requires MSRV >= 1.82
#[allow(deprecated)]
use std::panic::PanicInfo;
use std::str::FromStr;
use thiserror::Error;
use types::ClientCapabilities;
use types::CodeActionKind;
use types::CodeActionOptions;
@@ -24,7 +21,6 @@ use types::OneOf;
use types::TextDocumentSyncCapability;
use types::TextDocumentSyncKind;
use types::TextDocumentSyncOptions;
use types::Url;
use types::WorkDoneProgressOptions;
use types::WorkspaceFoldersServerCapabilities;
@@ -34,9 +30,8 @@ use self::schedule::event_loop_thread;
use self::schedule::Scheduler;
use self::schedule::Task;
use crate::session::AllSettings;
use crate::session::ClientSettings;
use crate::session::Session;
use crate::session::WorkspaceSettingsMap;
use crate::workspace::Workspaces;
use crate::PositionEncoding;
mod api;
@@ -447,122 +442,3 @@ impl FromStr for SupportedCommand {
})
}
}
#[derive(Debug)]
pub struct Workspaces(Vec<Workspace>);
impl Workspaces {
pub fn new(workspaces: Vec<Workspace>) -> Self {
Self(workspaces)
}
/// Create the workspaces from the provided workspace folders as provided by the client during
/// initialization.
fn from_workspace_folders(
workspace_folders: Option<Vec<WorkspaceFolder>>,
mut workspace_settings: WorkspaceSettingsMap,
) -> std::result::Result<Workspaces, WorkspacesError> {
let mut client_settings_for_url = |url: &Url| {
workspace_settings.remove(url).unwrap_or_else(|| {
tracing::info!(
"No workspace settings found for {}, using default settings",
url
);
ClientSettings::default()
})
};
let workspaces =
if let Some(folders) = workspace_folders.filter(|folders| !folders.is_empty()) {
folders
.into_iter()
.map(|folder| {
let settings = client_settings_for_url(&folder.uri);
Workspace::new(folder.uri).with_settings(settings)
})
.collect()
} else {
let current_dir = std::env::current_dir().map_err(WorkspacesError::Io)?;
tracing::info!(
"No workspace(s) were provided during initialization. \
Using the current working directory as a default workspace: {}",
current_dir.display()
);
let uri = Url::from_file_path(current_dir)
.map_err(|()| WorkspacesError::InvalidCurrentDir)?;
let settings = client_settings_for_url(&uri);
vec![Workspace::default(uri).with_settings(settings)]
};
Ok(Workspaces(workspaces))
}
}
impl Deref for Workspaces {
type Target = [Workspace];
fn deref(&self) -> &Self::Target {
&self.0
}
}
#[derive(Error, Debug)]
enum WorkspacesError {
#[error(transparent)]
Io(#[from] std::io::Error),
#[error("Failed to create a URL from the current working directory")]
InvalidCurrentDir,
}
#[derive(Debug)]
pub struct Workspace {
/// The [`Url`] pointing to the root of the workspace.
url: Url,
/// The client settings for this workspace.
settings: Option<ClientSettings>,
/// Whether this is the default workspace as created by the server. This will be the case when
/// no workspace folders were provided during initialization.
is_default: bool,
}
impl Workspace {
/// Create a new workspace with the given root URL.
pub fn new(url: Url) -> Self {
Self {
url,
settings: None,
is_default: false,
}
}
/// Create a new default workspace with the given root URL.
pub fn default(url: Url) -> Self {
Self {
url,
settings: None,
is_default: true,
}
}
/// Set the client settings for this workspace.
#[must_use]
pub fn with_settings(mut self, settings: ClientSettings) -> Self {
self.settings = Some(settings);
self
}
/// Returns the root URL of the workspace.
pub(crate) fn url(&self) -> &Url {
&self.url
}
/// Returns the client settings for this workspace.
pub(crate) fn settings(&self) -> Option<&ClientSettings> {
self.settings.as_ref()
}
/// Returns true if this is the default workspace.
pub(crate) fn is_default(&self) -> bool {
self.is_default
}
}

View File

@@ -7,7 +7,7 @@ use lsp_types::{ClientCapabilities, FileEvent, NotebookDocumentCellChange, Url};
use settings::ResolvedClientSettings;
use crate::edit::{DocumentKey, DocumentVersion, NotebookDocument};
use crate::server::Workspaces;
use crate::workspace::Workspaces;
use crate::{PositionEncoding, TextDocument};
pub(crate) use self::capabilities::ResolvedClientCapabilities;

View File

@@ -11,7 +11,7 @@ use thiserror::Error;
pub(crate) use ruff_settings::RuffSettings;
use crate::edit::LanguageId;
use crate::server::{Workspace, Workspaces};
use crate::workspace::{Workspace, Workspaces};
use crate::{
edit::{DocumentKey, DocumentVersion, NotebookDocument},
PositionEncoding, TextDocument,

View File

@@ -5,6 +5,8 @@ use rustc_hash::FxHashMap;
use serde::Deserialize;
use ruff_linter::{line_width::LineLength, RuleSelector};
use ruff_macros::OptionsMetadata;
use ruff_workspace::options_base::{OptionField, OptionsMetadata};
/// Maps a workspace URI to its associated client settings. Used during server initialization.
pub(crate) type WorkspaceSettingsMap = FxHashMap<Url, ClientSettings>;
@@ -57,27 +59,76 @@ pub(crate) enum ConfigurationPreference {
EditorOnly,
}
/// This is a direct representation of the settings schema sent by the client.
#[derive(Debug, Deserialize, Default)]
#[derive(Debug, Deserialize, Default, OptionsMetadata)]
#[cfg_attr(test, derive(PartialEq, Eq))]
#[serde(rename_all = "camelCase")]
pub struct ClientSettings {
/// Path to a `ruff.toml` or `pyproject.toml` file to use for configuration.
///
/// By default, Ruff will discover configuration for each project from the filesystem,
/// mirroring the behavior of the Ruff CLI.
#[option(
default = r#"null"#,
value_type = "string",
example = r#""~/path/to/ruff.toml""#
)]
configuration: Option<String>,
fix_all: Option<bool>,
organize_imports: Option<bool>,
lint: Option<LintOptions>,
format: Option<FormatOptions>,
code_action: Option<CodeActionOptions>,
exclude: Option<Vec<String>>,
line_length: Option<LineLength>,
/// The strategy to use when resolving settings across VS Code and the filesystem. By default,
/// editor configuration is prioritized over `ruff.toml` and `pyproject.toml` files.
///
/// * `"editorFirst"`: Editor settings take priority over configuration files present in the
/// workspace.
/// * `"filesystemFirst"`: Configuration files present in the workspace takes priority over
/// editor settings.
/// * `"editorOnly"`: Ignore configuration files entirely i.e., only use editor settings.
#[option(
default = r#""editorFirst""#,
value_type = r#""editorFirst" | "filesystemFirst" | "editorOnly""#,
example = r#""filesystemFirst""#
)]
configuration_preference: Option<ConfigurationPreference>,
/// If `true` or [`None`], show syntax errors as diagnostics.
/// A list of file patterns to exclude from linting and formatting. See [the
/// documentation](https://docs.astral.sh/ruff/settings/#exclude) for more details.
#[option(
default = r#"null"#,
value_type = "string[]",
example = r#"["**/tests/**"]"#
)]
exclude: Option<Vec<String>>,
/// The line length to use for the linter and formatter.
#[option(default = "null", value_type = "int", example = "100")]
line_length: Option<LineLength>,
/// Whether to register the server as capable of handling `source.fixAll` code actions.
#[option(default = "true", value_type = "bool", example = "false")]
fix_all: Option<bool>,
/// Whether to register the server as capable of handling `source.organizeImports` code
/// actions.
#[option(default = "true", value_type = "bool", example = "false")]
organize_imports: Option<bool>,
/// _New in Ruff [v0.5.0](https://astral.sh/blog/ruff-v0.5.0#changes-to-e999-and-reporting-of-syntax-errors)_
///
/// Whether to show syntax error diagnostics.
///
/// This is useful when using Ruff with other language servers, allowing the user to refer
/// to syntax errors from only one source.
#[option(default = "true", value_type = "bool", example = "false")]
show_syntax_errors: Option<bool>,
#[option_group]
lint: Option<LintOptions>,
#[option_group]
format: Option<FormatOptions>,
#[option_group]
code_action: Option<CodeActionOptions>,
// These settings are only needed for tracing, and are only read from the global configuration.
// These will not be in the resolved settings.
#[serde(flatten)]
@@ -98,14 +149,26 @@ impl ClientSettings {
}
}
/// Settings needed to initialize tracing. These will only be
/// read from the global configuration.
#[derive(Debug, Deserialize, Default)]
#[derive(Debug, Deserialize, Default, OptionsMetadata)]
#[cfg_attr(test, derive(PartialEq, Eq))]
#[serde(rename_all = "camelCase")]
pub(crate) struct TracingSettings {
/// The log level to use for the server.
#[option(
default = r#""info""#,
value_type = r#""error" | "warn" | "info" | "debug" | "trace""#,
example = r#""debug""#
)]
pub(crate) log_level: Option<crate::logging::LogLevel>,
/// Path to the log file - tildes and environment variables are supported.
/// Path to the log file to use for the server.
///
/// If not set, logs will be written to stderr. Tildes and environment variables are expanded.
#[option(
default = r#"null"#,
value_type = "string",
example = r#""~/path/to/ruff.log""#
)]
pub(crate) log_file: Option<PathBuf>,
}
@@ -121,14 +184,31 @@ struct WorkspaceSettings {
workspace: Url,
}
#[derive(Debug, Default, Deserialize)]
/// Settings specific to the Ruff linter.
#[derive(Debug, Default, Deserialize, OptionsMetadata)]
#[cfg_attr(test, derive(PartialEq, Eq))]
#[serde(rename_all = "camelCase")]
struct LintOptions {
/// Whether to enable linting. Set to `false` to use Ruff exclusively as a formatter.
#[option(default = "true", value_type = "bool", example = "false")]
enable: Option<bool>,
/// Whether to enable Ruff's preview mode when linting.
#[option(default = "null", value_type = "bool", example = "true")]
preview: Option<bool>,
/// Rules to enable by default. See [the
/// documentation](https://docs.astral.sh/ruff/settings/#lint_select).
#[option(default = "null", value_type = "string[]", example = r#"["E", "F"]"#)]
select: Option<Vec<String>>,
/// Rules to enable in addition to those in [`lint.select`](#select).
#[option(default = "null", value_type = "string[]", example = r#"["W"]"#)]
extend_select: Option<Vec<String>>,
/// Rules to disable by default. See [the
/// documentation](https://docs.astral.sh/ruff/settings/#lint_ignore).
#[option(default = "null", value_type = "string[]", example = r#"["E4", "E7"]"#)]
ignore: Option<Vec<String>>,
}
@@ -143,10 +223,13 @@ impl LintOptions {
}
}
#[derive(Debug, Default, Deserialize)]
/// Settings specific to the Ruff formatter.
#[derive(Debug, Default, Deserialize, OptionsMetadata)]
#[cfg_attr(test, derive(PartialEq, Eq))]
#[serde(rename_all = "camelCase")]
struct FormatOptions {
/// Whether to enable Ruff's preview mode when formatting.
#[option(default = "null", value_type = "bool", example = "true")]
preview: Option<bool>,
}
@@ -176,6 +259,38 @@ struct CodeActionParameters {
enable: Option<bool>,
}
impl OptionsMetadata for CodeActionOptions {
fn record(visit: &mut dyn ruff_workspace::options_base::Visit) {
visit.record_field(
"disableRuleComment.enable",
OptionField {
doc: "Whether to display Quick Fix actions to disable rules via `noqa` suppression comments.",
default: "true",
value_type: "bool",
scope: None,
example: "false",
deprecated: None,
},
);
visit.record_field(
"fixViolation.enable",
OptionField {
doc: "Whether to display Quick Fix actions to autofix violations.",
default: "true",
value_type: "bool",
scope: None,
example: "false",
deprecated: None,
},
);
}
fn documentation() -> Option<&'static str> {
Some("Enable or disable code actions provided by the server.")
}
}
/// This is the exact schema for initialization options sent in by the client
/// during initialization.
#[derive(Debug, Deserialize)]

View File

@@ -0,0 +1,126 @@
use std::ops::Deref;
use lsp_types::{Url, WorkspaceFolder};
use thiserror::Error;
use crate::session::WorkspaceSettingsMap;
use crate::ClientSettings;
#[derive(Debug)]
pub struct Workspaces(Vec<Workspace>);
impl Workspaces {
pub fn new(workspaces: Vec<Workspace>) -> Self {
Self(workspaces)
}
/// Create the workspaces from the provided workspace folders as provided by the client during
/// initialization.
pub(crate) fn from_workspace_folders(
workspace_folders: Option<Vec<WorkspaceFolder>>,
mut workspace_settings: WorkspaceSettingsMap,
) -> std::result::Result<Workspaces, WorkspacesError> {
let mut client_settings_for_url = |url: &Url| {
workspace_settings.remove(url).unwrap_or_else(|| {
tracing::info!(
"No workspace settings found for {}, using default settings",
url
);
ClientSettings::default()
})
};
let workspaces =
if let Some(folders) = workspace_folders.filter(|folders| !folders.is_empty()) {
folders
.into_iter()
.map(|folder| {
let settings = client_settings_for_url(&folder.uri);
Workspace::new(folder.uri).with_settings(settings)
})
.collect()
} else {
let current_dir = std::env::current_dir().map_err(WorkspacesError::Io)?;
tracing::info!(
"No workspace(s) were provided during initialization. \
Using the current working directory as a default workspace: {}",
current_dir.display()
);
let uri = Url::from_file_path(current_dir)
.map_err(|()| WorkspacesError::InvalidCurrentDir)?;
let settings = client_settings_for_url(&uri);
vec![Workspace::default(uri).with_settings(settings)]
};
Ok(Workspaces(workspaces))
}
}
impl Deref for Workspaces {
type Target = [Workspace];
fn deref(&self) -> &Self::Target {
&self.0
}
}
#[derive(Error, Debug)]
pub(crate) enum WorkspacesError {
#[error(transparent)]
Io(#[from] std::io::Error),
#[error("Failed to create a URL from the current working directory")]
InvalidCurrentDir,
}
#[derive(Debug)]
pub struct Workspace {
/// The [`Url`] pointing to the root of the workspace.
url: Url,
/// The client settings for this workspace.
settings: Option<ClientSettings>,
/// Whether this is the default workspace as created by the server. This will be the case when
/// no workspace folders were provided during initialization.
is_default: bool,
}
impl Workspace {
/// Create a new workspace with the given root URL.
pub fn new(url: Url) -> Self {
Self {
url,
settings: None,
is_default: false,
}
}
/// Create a new default workspace with the given root URL.
pub fn default(url: Url) -> Self {
Self {
url,
settings: None,
is_default: true,
}
}
/// Set the client settings for this workspace.
#[must_use]
pub fn with_settings(mut self, settings: ClientSettings) -> Self {
self.settings = Some(settings);
self
}
/// Returns the root URL of the workspace.
pub(crate) fn url(&self) -> &Url {
&self.url
}
/// Returns the client settings for this workspace.
pub(crate) fn settings(&self) -> Option<&ClientSettings> {
self.settings.as_ref()
}
/// Returns true if this is the default workspace.
pub(crate) fn is_default(&self) -> bool {
self.is_default
}
}

View File

@@ -6,6 +6,7 @@ use std::collections::{BTreeMap, BTreeSet};
use std::path::PathBuf;
use strum::IntoEnumIterator;
use crate as ruff_workspace;
use crate::options_base::{OptionsMetadata, Visit};
use crate::settings::LineEnding;
use ruff_formatter::IndentStyle;

View File

@@ -29,7 +29,7 @@ ruff_python_formatter = { path = "../crates/ruff_python_formatter" }
ruff_text_size = { path = "../crates/ruff_text_size" }
libfuzzer-sys = { git = "https://github.com/rust-fuzz/libfuzzer", default-features = false }
salsa = { git = "https://github.com/salsa-rs/salsa.git", rev = "351d9cf0037be949d17800d0c7b4838e533c2ed6" }
salsa = { git = "https://github.com/salsa-rs/salsa.git", rev = "c8826fa4d1d9e3cba4c6e578763878b71fa9a10d" }
similar = { version = "2.5.0" }
tracing = { version = "0.1.40" }