Compare commits

...

25 Commits

Author SHA1 Message Date
Brent Westbrook
47a5543109 chore: stabilize RUF059 2025-06-05 12:33:28 -04:00
Andrew Gallant
55100209c7 [ty] IDE: add support for object.<CURSOR> completions (#18468)
This PR adds logic for detecting `Name Dot [Name]` token patterns,
finding the corresponding `ExprAttribute`, getting the type of the
object and returning the members available on that object.

Here's a video demonstrating this working:

https://github.com/user-attachments/assets/42ce78e8-5930-4211-a18a-fa2a0434d0eb

Ref astral-sh/ty#86
2025-06-05 11:15:19 -04:00
chiri
c0bb83b882 [perflint] fix missing parentheses for lambda and ternary conditions (PERF401, PERF403) (#18412)
Closes #18405
2025-06-05 09:57:08 -05:00
Brent Westbrook
74a4e9af3d Combine lint and syntax error handling (#18471)
## Summary

This is a spin-off from
https://github.com/astral-sh/ruff/pull/18447#discussion_r2125844669 to
avoid using `Message::noqa_code` to differentiate between lints and
syntax errors. I went through all of the calls on `main` and on the
branch from #18447, and the instance in `ruff_server` noted in the
linked comment was actually the primary place where this was being done.
Other calls to `noqa_code` are typically some variation of
`message.noqa_code().map_or(String::new, format!(...))`, with the major
exception of the gitlab output format:


a120610b5b/crates/ruff_linter/src/message/gitlab.rs (L93-L105)

which obviously assumes that `None` means syntax error. A simple fix
here would be to use `message.name()` for `check_name` instead of the
noqa code, but I'm not sure how breaking that would be. This could just
be:

```rust
 let description = message.body();
 let description = description.strip_prefix("SyntaxError: ").unwrap_or(description).to_string();
 let check_name = message.name();
```

In that case. This sounds reasonable based on the [Code Quality report
format](https://docs.gitlab.com/ci/testing/code_quality/#code-quality-report-format)
docs:

> | Name | Type | Description|
> |-----|-----|----|
> |`check_name` | String | A unique name representing the check, or
rule, associated with this violation. |

## Test Plan

Existing tests
2025-06-05 12:50:02 +00:00
Alex Waygood
8485dbb324 [ty] Fix --python argument for Windows, and improve error messages for bad --python arguments (#18457)
## Summary

Fixes https://github.com/astral-sh/ty/issues/556.

On Windows, system installations have different layouts to virtual
environments. In Windows virtual environments, the Python executable is
found at `<sys.prefix>/Scripts/python.exe`. But in Windows system
installations, the Python executable is found at
`<sys.prefix>/python.exe`. That means that Windows users were able to
point to Python executables inside virtual environments with the
`--python` flag, but they weren't able to point to Python executables
inside system installations.

This PR fixes that issue. It also makes a couple of other changes:
- Nearly all `sys.prefix` resolution is moved inside `site_packages.rs`.
That was the original design of the `site-packages` resolution logic,
but features implemented since the initial implementation have added
some resolution and validation to `resolver.rs` inside the module
resolver. That means that we've ended up with a somewhat confusing code
structure and a situation where several checks are unnecessarily
duplicated between the two modules.
- I noticed that we had quite bad error messages if you e.g. pointed to
a path that didn't exist on disk with `--python` (we just gave a
somewhat impenetrable message saying that we "failed to canonicalize"
the path). I improved the error messages here and added CLI tests for
`--python` and the `environment.python` configuration setting.

## Test Plan

- Existing tests pass
- Added new CLI tests
- I manually checked that virtual-environment discovery still works if
no configuration is given
- Micha did some manual testing to check that pointing `--python` to a
system-installation executable now works on Windows
2025-06-05 08:19:15 +01:00
Shunsuke Shibayama
0858896bc4 [ty] type narrowing by attribute/subscript assignments (#18041)
## Summary

This PR partially solves https://github.com/astral-sh/ty/issues/164
(derived from #17643).

Currently, the definitions we manage are limited to those for simple
name (symbol) targets, but we expand this to track definitions for
attribute and subscript targets as well.

This was originally planned as part of the work in #17643, but the
changes are significant, so I made it a separate PR.
After merging this PR, I will reflect this changes in #17643.

There is still some incomplete work remaining, but the basic features
have been implemented, so I am publishing it as a draft PR.
Here is the TODO list (there may be more to come):
* [x] Complete rewrite and refactoring of documentation (removing
`Symbol` and replacing it with `Place`)
* [x] More thorough testing
* [x] Consolidation of duplicated code (maybe we can consolidate the
handling related to name, attribute, and subscript)

This PR replaces the current `Symbol` API with the `Place` API, which is
a concept that includes attributes and subscripts (the term is borrowed
from Rust).

## Test Plan

`mdtest/narrow/assignment.md` is added.

---------

Co-authored-by: David Peter <sharkdp@users.noreply.github.com>
Co-authored-by: Carl Meyer <carl@astral.sh>
2025-06-04 17:24:27 -07:00
Alex Waygood
ce8b744f17 [ty] Only calculate information for unresolved-reference subdiagnostic if we know we'll emit the diagnostic (#18465)
## Summary

This optimizes some of the logic added in
https://github.com/astral-sh/ruff/pull/18444. In general, we only
calculate information for subdiagnostics if we know we'll actually emit
the diagnostic. The check to see whether we'll emit the diagnostic is
work we'll definitely have to do whereas the the work to gather
information for a subdiagnostic isn't work we necessarily have to do if
the diagnostic isn't going to be emitted at all.

This PR makes us lazier about gathering the information we need for the
subdiagnostic, and moves all the subdiagnostic logic into one function
rather than having some `unresolved-reference` subdiagnostic logic in
`infer.rs` and some in `diagnostic.rs`.

## Test Plan

`cargo test -p ty_python_semantic`
2025-06-04 20:41:00 +01:00
Alex Waygood
5a8cdab771 [ty] Only consider a type T a subtype of a protocol P if all of P's members are fully bound on T (#18466)
## Summary

Fixes https://github.com/astral-sh/ty/issues/578

## Test Plan

mdtests
2025-06-04 19:39:14 +00:00
Alex Waygood
3a8191529c [ty] Exclude members starting with _abc_ from a protocol interface (#18467)
## Summary

As well as excluding a hardcoded set of special attributes, CPython at
runtime also excludes any attributes or declarations starting with
`_abc_` from the set of members that make up a protocol interface. I
missed this in my initial implementation.

This is a bit of a CPython implementation detail, but I do think it's
important that we try to model the runtime as best we can here. The
closer we are to the runtime behaviour, the closer we come to sound
behaviour when narrowing types from `isinstance()` checks against
runtime-checkable protocols (for example)

## Test Plan

Extended an existing mdtest
2025-06-04 20:34:09 +01:00
lipefree
e658778ced [ty] Add subdiagnostic suggestion to unresolved-reference diagnostic when variable exists on self (#18444)
## Summary

Closes https://github.com/astral-sh/ty/issues/502.

In the following example:
```py
class Foo:
    x: int

    def method(self):
        y = x
```
The user may intended to use `y = self.x` in `method`. 

This is now added as a subdiagnostic in the following form : 

`info: An attribute with the same name as 'x' is defined, consider using
'self.x'`

## Test Plan

Added mdtest with snapshot diagnostics.
2025-06-04 08:13:50 -07:00
David Peter
f1883d71a4 [ty] IDE: only provide declarations and bindings as completions (#18456)
## Summary

Previously, all symbols where provided as possible completions. In an
example like the following, both `foo` and `f` were suggested as
completions, because `f` itself is a symbol.
```py
foo = 1

f<CURSOR>
```
Similarly, in the following example, `hidden_symbol` was suggested, even
though it is not statically visible:
```py
if 1 + 2 != 3:
    hidden_symbol = 1

hidden_<CURSOR>
```

With the change suggested here, we only use statically visible
declarations and bindings as a source for completions.


## Test Plan

- Updated snapshot tests
- New test for statically hidden definitions
- Added test for star import
2025-06-04 16:11:05 +02:00
David Peter
11db567b0b [ty] ty_ide: Hotfix for expression_scope_id panics (#18455)
## Summary

Implement a hotfix for the playground/LSP crashes related to missing
`expression_scope_id`s.

relates to: https://github.com/astral-sh/ty/issues/572

## Test Plan

* Regression tests from https://github.com/astral-sh/ruff/pull/18441
* Ran the playground locally to check if panics occur / completions
still work.

---------

Co-authored-by: Andrew Gallant <andrew@astral.sh>
2025-06-04 10:39:16 +02:00
David Peter
9f8c3de462 [ty] Improve docs for Class{Literal,Type}::instance_member (#18454)
## Summary

Mostly just refer to `Type::instance_member` which has much more
details.
2025-06-04 09:55:45 +02:00
David Peter
293d4ac388 [ty] Add meta-type tests for legavy TypeVars (#18453)
## Summary

Follow up to the comment by @dcreager
[here](https://github.com/astral-sh/ruff/pull/18439#discussion_r2123802784).
2025-06-04 07:44:44 +00:00
Carl Meyer
9e8a7e9353 update to salsa that doesn't panic silently on cycles (#18450) 2025-06-04 07:40:16 +02:00
Dhruv Manilawala
453e5f5934 [ty] Add tests for empty list/tuple unpacking (#18451)
## Summary

This PR is to address this comment:
https://github.com/astral-sh/ruff/pull/18438#issuecomment-2935344415

## Test Plan

Run mdtest
2025-06-04 02:40:26 +00:00
Dhruv Manilawala
7ea773daf2 [ty] Argument type expansion for overload call evaluation (#18382)
## Summary

Part of astral-sh/ty#104, closes: astral-sh/ty#468

This PR implements the argument type expansion which is step 3 of the
overload call evaluation algorithm.

Specifically, this step needs to be taken if type checking resolves to
no matching overload and there are argument types that can be expanded.

## Test Plan

Add new test cases.

## Ecosystem analysis

This PR removes 174 `no-matching-overload` false positives -- I looked
at a lot of them and they all are false positives.

One thing that I'm not able to understand is that in
2b7e3adf27/sphinx/ext/autodoc/preserve_defaults.py (L179)
the inferred type of `value` is `str | None` by ty and Pyright, which is
correct, but it's only ty that raises `invalid-argument-type` error
while Pyright doesn't. The constructor method of `DefaultValue` has
declared type of `str` which is invalid.

There are few cases of false positives resulting due to the fact that ty
doesn't implement narrowing on attribute expressions.
2025-06-04 02:12:00 +00:00
Alex Waygood
0079cc6817 [ty] Minor cleanup for site-packages discovery logic (#18446) 2025-06-03 18:49:14 +00:00
Matthew Mckee
e8ea40012a [ty] Add generic inference for dataclasses (#18443)
## Summary

An issue seen here https://github.com/astral-sh/ty/issues/500

The `__init__` method of dataclasses had no inherited generic context,
so we could not infer the type of an instance from a constructor call
with generics

## Test Plan

Add tests to classes.md` in generics folder
2025-06-03 09:59:43 -07:00
Abhijeet Prasad Bodas
71d8a5da2a [ty] dataclasses: Allow using dataclasses.dataclass as a function. (#18440)
## Summary

Part of https://github.com/astral-sh/ty/issues/111

Using `dataclass` as a function, instead of as a decorator did not work
as expected prior to this.
Fix that by modifying the dataclass overload's return type.

## Test Plan

New mdtests, fixing the existing TODO.
2025-06-03 09:50:29 -07:00
Douglas Creager
2c3b3d3230 [ty] Create separate FunctionLiteral and FunctionType types (#18360)
This updates our representation of functions to more closely match our
representation of classes.

The new `OverloadLiteral` and `FunctionLiteral` classes represent a
function definition in the AST. If a function is generic, this is
unspecialized. `FunctionType` has been updated to represent a function
type, which is specialized if the function is generic. (These names are
chosen to match `ClassLiteral` and `ClassType` on the class side.)

This PR does not add a separate `Type` variant for `FunctionLiteral`.
Maybe we should? Possibly as a follow-on PR?

Part of https://github.com/astral-sh/ty/issues/462

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2025-06-03 10:59:31 -04:00
Dhruv Manilawala
8d98c601d8 [ty] Infer list[T] when unpacking non-tuple type (#18438)
## Summary

Follow-up from #18401, I was looking at whether that would fix the issue
at https://github.com/astral-sh/ty/issues/247#issuecomment-2917656676
and it didn't, which made me realize that the PR only inferred `list[T]`
when the value type was tuple but it could be other types as well.

This PR fixes the actual issue by inferring `list[T]` for the non-tuple
type case.

## Test Plan

Add test cases for starred expression involved with non-tuple type. I
also added a few test cases for list type and list literal.

I also verified that the example in the linked issue comment works:
```py
def _(line: str):
    a, b, *c = line.split(maxsplit=2)
    c.pop()
```
2025-06-03 19:17:47 +05:30
David Peter
0986edf427 [ty] Meta-type of type variables should be type[..] (#18439)
## Summary

Came across this while debugging some ecosystem changes in
https://github.com/astral-sh/ruff/pull/18347. I think the meta-type of a
typevar-annotated variable should be equal to `type`, not `<class
'object'>`.

## Test Plan

New Markdown tests.
2025-06-03 15:22:00 +02:00
chiri
03f1f8e218 [pyupgrade] Make fix unsafe if it deletes comments (UP050) (#18390)
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
  requests.)
- Does this pull request include references to any relevant issues?
-->

## Summary
/closes #18387
<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan
update snapshots
<!-- How was it tested? -->
2025-06-03 09:10:15 -04:00
chiri
628bb2cd1d [pyupgrade] Make fix unsafe if it deletes comments (UP004) (#18393)
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
  requests.)
- Does this pull request include references to any relevant issues?
-->

## Summary
https://github.com/astral-sh/ruff/issues/18387#issuecomment-2923039331
<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan
update snapshots
<!-- How was it tested? -->
2025-06-03 09:09:33 -04:00
81 changed files with 7298 additions and 3790 deletions

7
Cargo.lock generated
View File

@@ -3194,7 +3194,7 @@ checksum = "28d3b2b1366ec20994f1fd18c3c594f05c5dd4bc44d8bb0c1c632c8d6829481f"
[[package]]
name = "salsa"
version = "0.22.0"
source = "git+https://github.com/salsa-rs/salsa.git?rev=2b5188778e91a5ab50cb7d827148caf7eb2f4630#2b5188778e91a5ab50cb7d827148caf7eb2f4630"
source = "git+https://github.com/carljm/salsa.git?rev=0f6d406f6c309964279baef71588746b8c67b4a3#0f6d406f6c309964279baef71588746b8c67b4a3"
dependencies = [
"boxcar",
"compact_str",
@@ -3218,14 +3218,13 @@ dependencies = [
[[package]]
name = "salsa-macro-rules"
version = "0.22.0"
source = "git+https://github.com/salsa-rs/salsa.git?rev=2b5188778e91a5ab50cb7d827148caf7eb2f4630#2b5188778e91a5ab50cb7d827148caf7eb2f4630"
source = "git+https://github.com/carljm/salsa.git?rev=0f6d406f6c309964279baef71588746b8c67b4a3#0f6d406f6c309964279baef71588746b8c67b4a3"
[[package]]
name = "salsa-macros"
version = "0.22.0"
source = "git+https://github.com/salsa-rs/salsa.git?rev=2b5188778e91a5ab50cb7d827148caf7eb2f4630#2b5188778e91a5ab50cb7d827148caf7eb2f4630"
source = "git+https://github.com/carljm/salsa.git?rev=0f6d406f6c309964279baef71588746b8c67b4a3#0f6d406f6c309964279baef71588746b8c67b4a3"
dependencies = [
"heck",
"proc-macro2",
"quote",
"syn",

View File

@@ -129,7 +129,7 @@ regex = { version = "1.10.2" }
rustc-hash = { version = "2.0.0" }
rustc-stable-hash = { version = "0.1.2" }
# When updating salsa, make sure to also update the revision in `fuzz/Cargo.toml`
salsa = { git = "https://github.com/salsa-rs/salsa.git", rev = "2b5188778e91a5ab50cb7d827148caf7eb2f4630" }
salsa = { git = "https://github.com/carljm/salsa.git", rev = "0f6d406f6c309964279baef71588746b8c67b4a3" }
schemars = { version = "0.8.16" }
seahash = { version = "4.1.0" }
serde = { version = "1.0.197", features = ["derive"] }

View File

@@ -4,6 +4,10 @@ extend-exclude = [
"crates/ty_vendored/vendor/**/*",
"**/resources/**/*",
"**/snapshots/**/*",
# Completion tests tend to have a lot of incomplete
# words naturally. It's annoying to have to make all
# of them actually words. So just ignore typos here.
"crates/ty_ide/src/completion.rs",
]
[default.extend-words]

View File

@@ -566,7 +566,7 @@ fn venv() -> Result<()> {
----- stderr -----
ruff failed
Cause: Invalid search path settings
Cause: Failed to discover the site-packages directory: Invalid `--python` argument: `none` could not be canonicalized
Cause: Failed to discover the site-packages directory: Invalid `--python` argument: `none` does not point to a Python executable or a directory on disk
");
});

View File

@@ -10,7 +10,7 @@ use ruff_python_ast::PythonVersion;
use ty_python_semantic::lint::{LintRegistry, RuleSelection};
use ty_python_semantic::{
Db, Program, ProgramSettings, PythonPath, PythonPlatform, PythonVersionSource,
PythonVersionWithSource, SearchPathSettings, default_lint_registry,
PythonVersionWithSource, SearchPathSettings, SysPrefixPathOrigin, default_lint_registry,
};
static EMPTY_VENDORED: std::sync::LazyLock<VendoredFileSystem> = std::sync::LazyLock::new(|| {
@@ -37,7 +37,8 @@ impl ModuleDb {
) -> Result<Self> {
let mut search_paths = SearchPathSettings::new(src_roots);
if let Some(venv_path) = venv_path {
search_paths.python_path = PythonPath::from_cli_flag(venv_path);
search_paths.python_path =
PythonPath::sys_prefix(venv_path, SysPrefixPathOrigin::PythonCliFlag);
}
let db = Self::default();

View File

@@ -266,3 +266,15 @@ def f():
result = list() # this should be replaced with a comprehension
for i in values:
result.append(i + 1) # PERF401
def f():
src = [1]
dst = []
for i in src:
if True if True else False:
dst.append(i)
for i in src:
if lambda: 0:
dst.append(i)

View File

@@ -151,3 +151,16 @@ def foo():
result = {}
for idx, name in indices, fruit:
result[name] = idx # PERF403
def foo():
src = (("x", 1),)
dst = {}
for k, v in src:
if True if True else False:
dst[k] = v
for k, v in src:
if lambda: 0:
dst[k] = v

View File

@@ -1025,7 +1025,7 @@ pub fn code_to_rule(linter: Linter, code: &str) -> Option<(RuleGroup, Rule)> {
(Ruff, "056") => (RuleGroup::Preview, rules::ruff::rules::FalsyDictGetFallback),
(Ruff, "057") => (RuleGroup::Preview, rules::ruff::rules::UnnecessaryRound),
(Ruff, "058") => (RuleGroup::Preview, rules::ruff::rules::StarmapZip),
(Ruff, "059") => (RuleGroup::Preview, rules::ruff::rules::UnusedUnpackedVariable),
(Ruff, "059") => (RuleGroup::Stable, rules::ruff::rules::UnusedUnpackedVariable),
(Ruff, "060") => (RuleGroup::Preview, rules::ruff::rules::InEmptyCollection),
(Ruff, "100") => (RuleGroup::Stable, rules::ruff::rules::UnusedNOQA),
(Ruff, "101") => (RuleGroup::Stable, rules::ruff::rules::RedirectedNOQA),

View File

@@ -346,10 +346,11 @@ fn convert_to_dict_comprehension(
// since if the assignment expression appears
// internally (e.g. as an operand in a boolean
// operation) then it will already be parenthesized.
if test.is_named_expr() {
format!(" if ({})", locator.slice(test.range()))
} else {
format!(" if {}", locator.slice(test.range()))
match test {
Expr::Named(_) | Expr::If(_) | Expr::Lambda(_) => {
format!(" if ({})", locator.slice(test.range()))
}
_ => format!(" if {}", locator.slice(test.range())),
}
}
None => String::new(),

View File

@@ -358,7 +358,7 @@ fn convert_to_list_extend(
fix_type: ComprehensionType,
binding: &Binding,
for_stmt: &ast::StmtFor,
if_test: Option<&ast::Expr>,
if_test: Option<&Expr>,
to_append: &Expr,
checker: &Checker,
) -> Result<Fix> {
@@ -374,10 +374,11 @@ fn convert_to_list_extend(
// since if the assignment expression appears
// internally (e.g. as an operand in a boolean
// operation) then it will already be parenthesized.
if test.is_named_expr() {
format!(" if ({})", locator.slice(test.range()))
} else {
format!(" if {}", locator.slice(test.range()))
match test {
Expr::Named(_) | Expr::If(_) | Expr::Lambda(_) => {
format!(" if ({})", locator.slice(test.range()))
}
_ => format!(" if {}", locator.slice(test.range())),
}
}
None => String::new(),

View File

@@ -219,5 +219,27 @@ PERF401.py:268:9: PERF401 Use a list comprehension to create a transformed list
267 | for i in values:
268 | result.append(i + 1) # PERF401
| ^^^^^^^^^^^^^^^^^^^^ PERF401
269 |
270 | def f():
|
= help: Replace for loop with list comprehension
PERF401.py:276:13: PERF401 Use a list comprehension to create a transformed list
|
274 | for i in src:
275 | if True if True else False:
276 | dst.append(i)
| ^^^^^^^^^^^^^ PERF401
277 |
278 | for i in src:
|
= help: Replace for loop with list comprehension
PERF401.py:280:13: PERF401 Use `list.extend` to create a transformed list
|
278 | for i in src:
279 | if lambda: 0:
280 | dst.append(i)
| ^^^^^^^^^^^^^ PERF401
|
= help: Replace for loop with list.extend

View File

@@ -128,3 +128,23 @@ PERF403.py:153:9: PERF403 Use a dictionary comprehension instead of a for-loop
| ^^^^^^^^^^^^^^^^^^ PERF403
|
= help: Replace for loop with dict comprehension
PERF403.py:162:13: PERF403 Use a dictionary comprehension instead of a for-loop
|
160 | for k, v in src:
161 | if True if True else False:
162 | dst[k] = v
| ^^^^^^^^^^ PERF403
163 |
164 | for k, v in src:
|
= help: Replace for loop with dict comprehension
PERF403.py:166:13: PERF403 Use a dictionary comprehension instead of a for-loop
|
164 | for k, v in src:
165 | if lambda: 0:
166 | dst[k] = v
| ^^^^^^^^^^ PERF403
|
= help: Replace for loop with dict comprehension

View File

@@ -517,6 +517,8 @@ PERF401.py:268:9: PERF401 [*] Use a list comprehension to create a transformed l
267 | for i in values:
268 | result.append(i + 1) # PERF401
| ^^^^^^^^^^^^^^^^^^^^ PERF401
269 |
270 | def f():
|
= help: Replace for loop with list comprehension
@@ -529,3 +531,49 @@ PERF401.py:268:9: PERF401 [*] Use a list comprehension to create a transformed l
268 |- result.append(i + 1) # PERF401
266 |+ # this should be replaced with a comprehension
267 |+ result = [i + 1 for i in values] # PERF401
269 268 |
270 269 | def f():
271 270 | src = [1]
PERF401.py:276:13: PERF401 [*] Use a list comprehension to create a transformed list
|
274 | for i in src:
275 | if True if True else False:
276 | dst.append(i)
| ^^^^^^^^^^^^^ PERF401
277 |
278 | for i in src:
|
= help: Replace for loop with list comprehension
Unsafe fix
269 269 |
270 270 | def f():
271 271 | src = [1]
272 |- dst = []
273 272 |
274 |- for i in src:
275 |- if True if True else False:
276 |- dst.append(i)
273 |+ dst = [i for i in src if (True if True else False)]
277 274 |
278 275 | for i in src:
279 276 | if lambda: 0:
PERF401.py:280:13: PERF401 [*] Use `list.extend` to create a transformed list
|
278 | for i in src:
279 | if lambda: 0:
280 | dst.append(i)
| ^^^^^^^^^^^^^ PERF401
|
= help: Replace for loop with list.extend
Unsafe fix
275 275 | if True if True else False:
276 276 | dst.append(i)
277 277 |
278 |- for i in src:
279 |- if lambda: 0:
280 |- dst.append(i)
278 |+ dst.extend(i for i in src if (lambda: 0))

View File

@@ -305,3 +305,49 @@ PERF403.py:153:9: PERF403 [*] Use a dictionary comprehension instead of a for-lo
152 |- for idx, name in indices, fruit:
153 |- result[name] = idx # PERF403
151 |+ result = {name: idx for idx, name in (indices, fruit)} # PERF403
154 152 |
155 153 |
156 154 | def foo():
PERF403.py:162:13: PERF403 [*] Use a dictionary comprehension instead of a for-loop
|
160 | for k, v in src:
161 | if True if True else False:
162 | dst[k] = v
| ^^^^^^^^^^ PERF403
163 |
164 | for k, v in src:
|
= help: Replace for loop with dict comprehension
Unsafe fix
155 155 |
156 156 | def foo():
157 157 | src = (("x", 1),)
158 |- dst = {}
159 158 |
160 |- for k, v in src:
161 |- if True if True else False:
162 |- dst[k] = v
159 |+ dst = {k: v for k, v in src if (True if True else False)}
163 160 |
164 161 | for k, v in src:
165 162 | if lambda: 0:
PERF403.py:166:13: PERF403 [*] Use `dict.update` instead of a for-loop
|
164 | for k, v in src:
165 | if lambda: 0:
166 | dst[k] = v
| ^^^^^^^^^^ PERF403
|
= help: Replace for loop with `dict.update`
Unsafe fix
161 161 | if True if True else False:
162 162 | dst[k] = v
163 163 |
164 |- for k, v in src:
165 |- if lambda: 0:
166 |- dst[k] = v
164 |+ dst.update({k: v for k, v in src if (lambda: 0)})

View File

@@ -1,6 +1,7 @@
use crate::checkers::ast::Checker;
use crate::fix::edits::{Parentheses, remove_argument};
use crate::{Fix, FixAvailability, Violation};
use ruff_diagnostics::Applicability;
use ruff_macros::{ViolationMetadata, derive_message_formats};
use ruff_python_ast::StmtClassDef;
use ruff_text_size::Ranged;
@@ -63,13 +64,21 @@ pub(crate) fn useless_class_metaclass_type(checker: &Checker, class_def: &StmtCl
);
diagnostic.try_set_fix(|| {
remove_argument(
let edit = remove_argument(
keyword,
arguments,
Parentheses::Remove,
checker.locator().contents(),
)
.map(Fix::safe_edit)
)?;
let range = edit.range();
let applicability = if checker.comment_ranges().intersects(range) {
Applicability::Unsafe
} else {
Applicability::Safe
};
Ok(Fix::applicable_edit(edit, applicability))
});
}
}

View File

@@ -1,3 +1,4 @@
use ruff_diagnostics::Applicability;
use ruff_macros::{ViolationMetadata, derive_message_formats};
use ruff_python_ast as ast;
use ruff_text_size::Ranged;
@@ -61,14 +62,23 @@ pub(crate) fn useless_object_inheritance(checker: &Checker, class_def: &ast::Stm
},
base.range(),
);
diagnostic.try_set_fix(|| {
remove_argument(
let edit = remove_argument(
base,
arguments,
Parentheses::Remove,
checker.locator().contents(),
)
.map(Fix::safe_edit)
)?;
let range = edit.range();
let applicability = if checker.comment_ranges().intersects(range) {
Applicability::Unsafe
} else {
Applicability::Safe
};
Ok(Fix::applicable_edit(edit, applicability))
});
}
}

View File

@@ -51,7 +51,7 @@ UP004.py:16:5: UP004 [*] Class `A` inherits from `object`
|
= help: Remove `object` inheritance
Safe fix
Unsafe fix
12 12 | ...
13 13 |
14 14 |
@@ -75,7 +75,7 @@ UP004.py:24:5: UP004 [*] Class `A` inherits from `object`
|
= help: Remove `object` inheritance
Safe fix
Unsafe fix
19 19 | ...
20 20 |
21 21 |
@@ -99,7 +99,7 @@ UP004.py:31:5: UP004 [*] Class `A` inherits from `object`
|
= help: Remove `object` inheritance
Safe fix
Unsafe fix
26 26 | ...
27 27 |
28 28 |
@@ -122,7 +122,7 @@ UP004.py:37:5: UP004 [*] Class `A` inherits from `object`
|
= help: Remove `object` inheritance
Safe fix
Unsafe fix
33 33 | ...
34 34 |
35 35 |
@@ -146,7 +146,7 @@ UP004.py:45:5: UP004 [*] Class `A` inherits from `object`
|
= help: Remove `object` inheritance
Safe fix
Unsafe fix
40 40 | ...
41 41 |
42 42 |
@@ -171,7 +171,7 @@ UP004.py:53:5: UP004 [*] Class `A` inherits from `object`
|
= help: Remove `object` inheritance
Safe fix
Unsafe fix
48 48 | ...
49 49 |
50 50 |
@@ -196,7 +196,7 @@ UP004.py:61:5: UP004 [*] Class `A` inherits from `object`
|
= help: Remove `object` inheritance
Safe fix
Unsafe fix
56 56 | ...
57 57 |
58 58 |
@@ -221,7 +221,7 @@ UP004.py:69:5: UP004 [*] Class `A` inherits from `object`
|
= help: Remove `object` inheritance
Safe fix
Unsafe fix
64 64 | ...
65 65 |
66 66 |
@@ -320,7 +320,7 @@ UP004.py:98:5: UP004 [*] Class `B` inherits from `object`
|
= help: Remove `object` inheritance
Safe fix
Unsafe fix
95 95 |
96 96 |
97 97 | class B(
@@ -381,7 +381,7 @@ UP004.py:125:5: UP004 [*] Class `A` inherits from `object`
|
= help: Remove `object` inheritance
Safe fix
Unsafe fix
121 121 | ...
122 122 |
123 123 |
@@ -403,7 +403,7 @@ UP004.py:131:5: UP004 [*] Class `A` inherits from `object`
|
= help: Remove `object` inheritance
Safe fix
Unsafe fix
127 127 | ...
128 128 |
129 129 |

View File

@@ -51,7 +51,7 @@ UP050.py:16:5: UP050 [*] Class `A` uses `metaclass=type`, which is redundant
|
= help: Remove `metaclass=type`
Safe fix
Unsafe fix
12 12 | ...
13 13 |
14 14 |
@@ -75,7 +75,7 @@ UP050.py:24:5: UP050 [*] Class `A` uses `metaclass=type`, which is redundant
|
= help: Remove `metaclass=type`
Safe fix
Unsafe fix
19 19 | ...
20 20 |
21 21 |
@@ -98,7 +98,7 @@ UP050.py:30:5: UP050 [*] Class `A` uses `metaclass=type`, which is redundant
|
= help: Remove `metaclass=type`
Safe fix
Unsafe fix
26 26 | ...
27 27 |
28 28 |
@@ -122,7 +122,7 @@ UP050.py:38:5: UP050 [*] Class `A` uses `metaclass=type`, which is redundant
|
= help: Remove `metaclass=type`
Safe fix
Unsafe fix
33 33 | ...
34 34 |
35 35 |
@@ -185,7 +185,7 @@ UP050.py:58:5: UP050 [*] Class `B` uses `metaclass=type`, which is redundant
|
= help: Remove `metaclass=type`
Safe fix
Unsafe fix
54 54 |
55 55 | class B(
56 56 | A,
@@ -205,7 +205,7 @@ UP050.py:69:5: UP050 [*] Class `A` uses `metaclass=type`, which is redundant
|
= help: Remove `metaclass=type`
Safe fix
Unsafe fix
65 65 | ...
66 66 |
67 67 |

View File

@@ -12,7 +12,6 @@ use crate::{
use ruff_diagnostics::{Applicability, Edit, Fix};
use ruff_linter::{
Locator,
codes::Rule,
directives::{Flags, extract_directives},
generate_noqa_edits,
linter::check_path,
@@ -166,26 +165,17 @@ pub(crate) fn check(
messages
.into_iter()
.zip(noqa_edits)
.filter_map(|(message, noqa_edit)| match message.to_rule() {
Some(rule) => Some(to_lsp_diagnostic(
rule,
&message,
noqa_edit,
&source_kind,
locator.to_index(),
encoding,
)),
None => {
if show_syntax_errors {
Some(syntax_error_to_lsp_diagnostic(
&message,
&source_kind,
locator.to_index(),
encoding,
))
} else {
None
}
.filter_map(|(message, noqa_edit)| {
if message.is_syntax_error() && !show_syntax_errors {
None
} else {
Some(to_lsp_diagnostic(
&message,
noqa_edit,
&source_kind,
locator.to_index(),
encoding,
))
}
});
@@ -241,7 +231,6 @@ pub(crate) fn fixes_for_diagnostics(
/// Generates an LSP diagnostic with an associated cell index for the diagnostic to go in.
/// If the source kind is a text document, the cell index will always be `0`.
fn to_lsp_diagnostic(
rule: Rule,
diagnostic: &Message,
noqa_edit: Option<Edit>,
source_kind: &SourceKind,
@@ -253,11 +242,13 @@ fn to_lsp_diagnostic(
let body = diagnostic.body().to_string();
let fix = diagnostic.fix();
let suggestion = diagnostic.suggestion();
let code = diagnostic.to_noqa_code();
let fix = fix.and_then(|fix| fix.applies(Applicability::Unsafe).then_some(fix));
let data = (fix.is_some() || noqa_edit.is_some())
.then(|| {
let code = code?.to_string();
let edits = fix
.into_iter()
.flat_map(Fix::edits)
@@ -274,14 +265,12 @@ fn to_lsp_diagnostic(
title: suggestion.unwrap_or(name).to_string(),
noqa_edit,
edits,
code: rule.noqa_code().to_string(),
code,
})
.ok()
})
.flatten();
let code = rule.noqa_code().to_string();
let range: lsp_types::Range;
let cell: usize;
@@ -297,14 +286,25 @@ fn to_lsp_diagnostic(
range = diagnostic_range.to_range(source_kind.source_code(), index, encoding);
}
let (severity, tags, code) = if let Some(code) = code {
let code = code.to_string();
(
Some(severity(&code)),
tags(&code),
Some(lsp_types::NumberOrString::String(code)),
)
} else {
(None, None, None)
};
(
cell,
lsp_types::Diagnostic {
range,
severity: Some(severity(&code)),
tags: tags(&code),
code: Some(lsp_types::NumberOrString::String(code)),
code_description: rule.url().and_then(|url| {
severity,
tags,
code,
code_description: diagnostic.to_url().and_then(|url| {
Some(lsp_types::CodeDescription {
href: lsp_types::Url::parse(&url).ok()?,
})
@@ -317,45 +317,6 @@ fn to_lsp_diagnostic(
)
}
fn syntax_error_to_lsp_diagnostic(
syntax_error: &Message,
source_kind: &SourceKind,
index: &LineIndex,
encoding: PositionEncoding,
) -> (usize, lsp_types::Diagnostic) {
let range: lsp_types::Range;
let cell: usize;
if let Some(notebook_index) = source_kind.as_ipy_notebook().map(Notebook::index) {
NotebookRange { cell, range } = syntax_error.range().to_notebook_range(
source_kind.source_code(),
index,
notebook_index,
encoding,
);
} else {
cell = usize::default();
range = syntax_error
.range()
.to_range(source_kind.source_code(), index, encoding);
}
(
cell,
lsp_types::Diagnostic {
range,
severity: Some(lsp_types::DiagnosticSeverity::ERROR),
tags: None,
code: None,
code_description: None,
source: Some(DIAGNOSTIC_NAME.into()),
message: syntax_error.body().to_string(),
related_information: None,
data: None,
},
)
}
fn diagnostic_edit_range(
range: TextRange,
source_kind: &SourceKind,

View File

@@ -207,36 +207,23 @@ impl Workspace {
let messages: Vec<ExpandedMessage> = messages
.into_iter()
.map(|msg| {
let message = msg.body().to_string();
let range = msg.range();
match msg.to_noqa_code() {
Some(code) => ExpandedMessage {
code: Some(code.to_string()),
message,
start_location: source_code.line_column(range.start()).into(),
end_location: source_code.line_column(range.end()).into(),
fix: msg.fix().map(|fix| ExpandedFix {
message: msg.suggestion().map(ToString::to_string),
edits: fix
.edits()
.iter()
.map(|edit| ExpandedEdit {
location: source_code.line_column(edit.start()).into(),
end_location: source_code.line_column(edit.end()).into(),
content: edit.content().map(ToString::to_string),
})
.collect(),
}),
},
None => ExpandedMessage {
code: None,
message,
start_location: source_code.line_column(range.start()).into(),
end_location: source_code.line_column(range.end()).into(),
fix: None,
},
}
.map(|msg| ExpandedMessage {
code: msg.to_noqa_code().map(|code| code.to_string()),
message: msg.body().to_string(),
start_location: source_code.line_column(msg.start()).into(),
end_location: source_code.line_column(msg.end()).into(),
fix: msg.fix().map(|fix| ExpandedFix {
message: msg.suggestion().map(ToString::to_string),
edits: fix
.edits()
.iter()
.map(|edit| ExpandedEdit {
location: source_code.line_column(edit.start()).into(),
end_location: source_code.line_column(edit.end()).into(),
content: edit.content().map(ToString::to_string),
})
.collect(),
}),
})
.collect();

View File

@@ -918,6 +918,156 @@ fn cli_unknown_rules() -> anyhow::Result<()> {
Ok(())
}
#[test]
fn python_cli_argument_virtual_environment() -> anyhow::Result<()> {
let path_to_executable = if cfg!(windows) {
"my-venv/Scripts/python.exe"
} else {
"my-venv/bin/python"
};
let other_venv_path = "my-venv/foo/some_other_file.txt";
let case = TestCase::with_files([
("test.py", ""),
(
if cfg!(windows) {
"my-venv/Lib/site-packages/foo.py"
} else {
"my-venv/lib/python3.13/site-packages/foo.py"
},
"",
),
(path_to_executable, ""),
(other_venv_path, ""),
])?;
// Passing a path to the installation works
assert_cmd_snapshot!(case.command().arg("--python").arg("my-venv"), @r"
success: true
exit_code: 0
----- stdout -----
All checks passed!
----- stderr -----
WARN ty is pre-release software and not ready for production use. Expect to encounter bugs, missing features, and fatal errors.
");
// And so does passing a path to the executable inside the installation
assert_cmd_snapshot!(case.command().arg("--python").arg(path_to_executable), @r"
success: true
exit_code: 0
----- stdout -----
All checks passed!
----- stderr -----
WARN ty is pre-release software and not ready for production use. Expect to encounter bugs, missing features, and fatal errors.
");
// But random other paths inside the installation are rejected
assert_cmd_snapshot!(case.command().arg("--python").arg(other_venv_path), @r"
success: false
exit_code: 2
----- stdout -----
----- stderr -----
WARN ty is pre-release software and not ready for production use. Expect to encounter bugs, missing features, and fatal errors.
ty failed
Cause: Invalid search path settings
Cause: Failed to discover the site-packages directory: Invalid `--python` argument: `<temp_dir>/my-venv/foo/some_other_file.txt` does not point to a Python executable or a directory on disk
");
// And so are paths that do not exist on disk
assert_cmd_snapshot!(case.command().arg("--python").arg("not-a-directory-or-executable"), @r"
success: false
exit_code: 2
----- stdout -----
----- stderr -----
WARN ty is pre-release software and not ready for production use. Expect to encounter bugs, missing features, and fatal errors.
ty failed
Cause: Invalid search path settings
Cause: Failed to discover the site-packages directory: Invalid `--python` argument: `<temp_dir>/not-a-directory-or-executable` does not point to a Python executable or a directory on disk
");
Ok(())
}
#[test]
fn python_cli_argument_system_installation() -> anyhow::Result<()> {
let path_to_executable = if cfg!(windows) {
"Python3.11/python.exe"
} else {
"Python3.11/bin/python"
};
let case = TestCase::with_files([
("test.py", ""),
(
if cfg!(windows) {
"Python3.11/Lib/site-packages/foo.py"
} else {
"Python3.11/lib/python3.11/site-packages/foo.py"
},
"",
),
(path_to_executable, ""),
])?;
// Passing a path to the installation works
assert_cmd_snapshot!(case.command().arg("--python").arg("Python3.11"), @r"
success: true
exit_code: 0
----- stdout -----
All checks passed!
----- stderr -----
WARN ty is pre-release software and not ready for production use. Expect to encounter bugs, missing features, and fatal errors.
");
// And so does passing a path to the executable inside the installation
assert_cmd_snapshot!(case.command().arg("--python").arg(path_to_executable), @r"
success: true
exit_code: 0
----- stdout -----
All checks passed!
----- stderr -----
WARN ty is pre-release software and not ready for production use. Expect to encounter bugs, missing features, and fatal errors.
");
Ok(())
}
#[test]
fn config_file_broken_python_setting() -> anyhow::Result<()> {
let case = TestCase::with_files([
(
"pyproject.toml",
r#"
[tool.ty.environment]
python = "not-a-directory-or-executable"
"#,
),
("test.py", ""),
])?;
// TODO: this error message should say "invalid `python` configuration setting" rather than "invalid `--python` argument"
assert_cmd_snapshot!(case.command(), @r"
success: false
exit_code: 2
----- stdout -----
----- stderr -----
WARN ty is pre-release software and not ready for production use. Expect to encounter bugs, missing features, and fatal errors.
ty failed
Cause: Invalid search path settings
Cause: Failed to discover the site-packages directory: Invalid `--python` argument: `<temp_dir>/not-a-directory-or-executable` does not point to a Python executable or a directory on disk
");
Ok(())
}
#[test]
fn exit_code_only_warnings() -> anyhow::Result<()> {
let case = TestCase::with_file("test.py", r"print(x) # [unresolved-reference]")?;

View File

@@ -1,10 +1,13 @@
use std::cmp::Ordering;
use ruff_db::files::File;
use ruff_db::parsed::{ParsedModule, parsed_module};
use ruff_python_parser::TokenAt;
use ruff_python_ast as ast;
use ruff_python_parser::{Token, TokenAt, TokenKind};
use ruff_text_size::{Ranged, TextRange, TextSize};
use crate::Db;
use crate::find_node::{CoveringNode, covering_node};
use crate::find_node::covering_node;
#[derive(Debug, Clone)]
pub struct Completion {
@@ -14,13 +17,16 @@ pub struct Completion {
pub fn completion(db: &dyn Db, file: File, offset: TextSize) -> Vec<Completion> {
let parsed = parsed_module(db.upcast(), file);
let Some(target) = find_target(parsed, offset) else {
let Some(target) = CompletionTargetTokens::find(parsed, offset).ast(parsed) else {
return vec![];
};
let model = ty_python_semantic::SemanticModel::new(db.upcast(), file);
let mut completions = model.completions(target.node());
completions.sort();
let mut completions = match target {
CompletionTargetAst::ObjectDot { expr } => model.attribute_completions(expr),
CompletionTargetAst::Scoped { node } => model.scoped_completions(node),
};
completions.sort_by(|name1, name2| compare_suggestions(name1, name2));
completions.dedup();
completions
.into_iter()
@@ -28,30 +34,253 @@ pub fn completion(db: &dyn Db, file: File, offset: TextSize) -> Vec<Completion>
.collect()
}
fn find_target(parsed: &ParsedModule, offset: TextSize) -> Option<CoveringNode> {
let offset = match parsed.tokens().at_offset(offset) {
TokenAt::None => {
return Some(covering_node(
parsed.syntax().into(),
TextRange::empty(offset),
));
/// The kind of tokens identified under the cursor.
#[derive(Debug)]
enum CompletionTargetTokens<'t> {
/// A `object.attribute` token form was found, where
/// `attribute` may be empty.
///
/// This requires a name token followed by a dot token.
ObjectDot {
/// The token preceding the dot.
object: &'t Token,
/// The token, if non-empty, following the dot.
///
/// This is currently unused, but we should use this
/// eventually to remove completions that aren't a
/// prefix of what has already been typed. (We are
/// currently relying on the LSP client to do this.)
#[expect(dead_code)]
attribute: Option<&'t Token>,
},
/// A token was found under the cursor, but it didn't
/// match any of our anticipated token patterns.
Generic { token: &'t Token },
/// No token was found, but we have the offset of the
/// cursor.
Unknown { offset: TextSize },
}
impl<'t> CompletionTargetTokens<'t> {
/// Look for the best matching token pattern at the given offset.
fn find(parsed: &ParsedModule, offset: TextSize) -> CompletionTargetTokens<'_> {
static OBJECT_DOT_EMPTY: [TokenKind; 2] = [TokenKind::Name, TokenKind::Dot];
static OBJECT_DOT_NON_EMPTY: [TokenKind; 3] =
[TokenKind::Name, TokenKind::Dot, TokenKind::Name];
let offset = match parsed.tokens().at_offset(offset) {
TokenAt::None => return CompletionTargetTokens::Unknown { offset },
TokenAt::Single(tok) => tok.end(),
TokenAt::Between(_, tok) => tok.start(),
};
let before = parsed.tokens().before(offset);
if let Some([object, _dot]) = token_suffix_by_kinds(before, OBJECT_DOT_EMPTY) {
CompletionTargetTokens::ObjectDot {
object,
attribute: None,
}
} else if let Some([object, _dot, attribute]) =
token_suffix_by_kinds(before, OBJECT_DOT_NON_EMPTY)
{
CompletionTargetTokens::ObjectDot {
object,
attribute: Some(attribute),
}
} else {
let Some(last) = before.last() else {
return CompletionTargetTokens::Unknown { offset };
};
CompletionTargetTokens::Generic { token: last }
}
TokenAt::Single(tok) => tok.end(),
TokenAt::Between(_, tok) => tok.start(),
};
let before = parsed.tokens().before(offset);
let last = before.last()?;
let covering_node = covering_node(parsed.syntax().into(), last.range());
Some(covering_node)
}
/// Returns a corresponding AST node for these tokens.
///
/// If no plausible AST node could be found, then `None` is returned.
fn ast(&self, parsed: &'t ParsedModule) -> Option<CompletionTargetAst<'t>> {
match *self {
CompletionTargetTokens::ObjectDot { object, .. } => {
let covering_node = covering_node(parsed.syntax().into(), object.range())
.find(|node| node.is_expr_attribute())
.ok()?;
match covering_node.node() {
ast::AnyNodeRef::ExprAttribute(expr) => {
Some(CompletionTargetAst::ObjectDot { expr })
}
_ => None,
}
}
CompletionTargetTokens::Generic { token } => {
let covering_node = covering_node(parsed.syntax().into(), token.range());
Some(CompletionTargetAst::Scoped {
node: covering_node.node(),
})
}
CompletionTargetTokens::Unknown { offset } => {
let range = TextRange::empty(offset);
let covering_node = covering_node(parsed.syntax().into(), range);
Some(CompletionTargetAst::Scoped {
node: covering_node.node(),
})
}
}
}
}
/// The AST node patterns that we support identifying under the cursor.
#[derive(Debug)]
enum CompletionTargetAst<'t> {
/// A `object.attribute` scenario, where we want to
/// list attributes on `object` for completions.
ObjectDot { expr: &'t ast::ExprAttribute },
/// A scoped scenario, where we want to list all items available in
/// the most narrow scope containing the giving AST node.
Scoped { node: ast::AnyNodeRef<'t> },
}
/// Returns a suffix of `tokens` corresponding to the `kinds` given.
///
/// If a suffix of `tokens` with the given `kinds` could not be found,
/// then `None` is returned.
///
/// This is useful for matching specific patterns of token sequences
/// in order to identify what kind of completions we should offer.
fn token_suffix_by_kinds<const N: usize>(
tokens: &[Token],
kinds: [TokenKind; N],
) -> Option<[&Token; N]> {
if kinds.len() > tokens.len() {
return None;
}
for (token, expected_kind) in tokens.iter().rev().zip(kinds.iter().rev()) {
if &token.kind() != expected_kind {
return None;
}
}
Some(std::array::from_fn(|i| {
&tokens[tokens.len() - (kinds.len() - i)]
}))
}
/// Order completions lexicographically, with these exceptions:
///
/// 1) A `_[^_]` prefix sorts last and
/// 2) A `__` prefix sorts last except before (1)
///
/// This has the effect of putting all dunder attributes after "normal"
/// attributes, and all single-underscore attributes after dunder attributes.
fn compare_suggestions(name1: &str, name2: &str) -> Ordering {
/// A helper type for sorting completions based only on name.
///
/// This sorts "normal" names first, then dunder names and finally
/// single-underscore names. This matches the order of the variants defined for
/// this enum, which is in turn picked up by the derived trait implementation
/// for `Ord`.
#[derive(Clone, Copy, Eq, PartialEq, PartialOrd, Ord)]
enum Kind {
Normal,
Dunder,
Sunder,
}
impl Kind {
fn classify(name: &str) -> Kind {
// Dunder needs a prefix and suffix double underscore.
// When there's only a prefix double underscore, this
// results in explicit name mangling. We let that be
// classified as-if they were single underscore names.
//
// Ref: <https://docs.python.org/3/reference/lexical_analysis.html#reserved-classes-of-identifiers>
if name.starts_with("__") && name.ends_with("__") {
Kind::Dunder
} else if name.starts_with('_') {
Kind::Sunder
} else {
Kind::Normal
}
}
}
let (kind1, kind2) = (Kind::classify(name1), Kind::classify(name2));
kind1.cmp(&kind2).then_with(|| name1.cmp(name2))
}
#[cfg(test)]
mod tests {
use insta::assert_snapshot;
use ruff_python_parser::{Mode, ParseOptions, TokenKind, Tokens};
use crate::completion;
use crate::tests::{CursorTest, cursor_test};
use super::token_suffix_by_kinds;
#[test]
fn token_suffixes_match() {
insta::assert_debug_snapshot!(
token_suffix_by_kinds(&tokenize("foo.x"), [TokenKind::Newline]),
@r"
Some(
[
Newline 5..5,
],
)
",
);
insta::assert_debug_snapshot!(
token_suffix_by_kinds(&tokenize("foo.x"), [TokenKind::Name, TokenKind::Newline]),
@r"
Some(
[
Name 4..5,
Newline 5..5,
],
)
",
);
let all = [
TokenKind::Name,
TokenKind::Dot,
TokenKind::Name,
TokenKind::Newline,
];
insta::assert_debug_snapshot!(
token_suffix_by_kinds(&tokenize("foo.x"), all),
@r"
Some(
[
Name 0..3,
Dot 3..4,
Name 4..5,
Newline 5..5,
],
)
",
);
}
#[test]
fn token_suffixes_nomatch() {
insta::assert_debug_snapshot!(
token_suffix_by_kinds(&tokenize("foo.x"), [TokenKind::Name]),
@"None",
);
let too_many = [
TokenKind::Dot,
TokenKind::Name,
TokenKind::Dot,
TokenKind::Name,
TokenKind::Newline,
];
insta::assert_debug_snapshot!(
token_suffix_by_kinds(&tokenize("foo.x"), too_many),
@"None",
);
}
// At time of writing (2025-05-22), the tests below show some of the
// naivete of our completions. That is, we don't even take what has been
// typed into account. We just kind return all possible completions
@@ -113,8 +342,7 @@ import re
re.<CURSOR>
",
);
assert_snapshot!(test.completions(), @"re");
test.assert_completions_include("findall");
}
#[test]
@@ -127,10 +355,7 @@ f<CURSOR>
",
);
assert_snapshot!(test.completions(), @r"
f
foo
");
assert_snapshot!(test.completions(), @"foo");
}
#[test]
@@ -143,10 +368,7 @@ g<CURSOR>
",
);
assert_snapshot!(test.completions(), @r"
foo
g
");
assert_snapshot!(test.completions(), @"foo");
}
#[test]
@@ -175,10 +397,7 @@ f<CURSOR>
",
);
assert_snapshot!(test.completions(), @r"
f
foo
");
assert_snapshot!(test.completions(), @"foo");
}
#[test]
@@ -208,7 +427,6 @@ def foo():
);
assert_snapshot!(test.completions(), @r"
f
foo
foofoo
");
@@ -259,7 +477,6 @@ def foo():
);
assert_snapshot!(test.completions(), @r"
f
foo
foofoo
");
@@ -276,7 +493,6 @@ def foo():
);
assert_snapshot!(test.completions(), @r"
f
foo
foofoo
");
@@ -295,7 +511,6 @@ def frob(): ...
);
assert_snapshot!(test.completions(), @r"
f
foo
foofoo
frob
@@ -315,7 +530,6 @@ def frob(): ...
);
assert_snapshot!(test.completions(), @r"
f
foo
frob
");
@@ -334,7 +548,6 @@ def frob(): ...
);
assert_snapshot!(test.completions(), @r"
f
foo
foofoo
foofoofoo
@@ -451,15 +664,10 @@ def frob(): ...
",
);
// It's not totally clear why `for` shows up in the
// symbol tables of the detected scopes here. My guess
// is that there's perhaps some sub-optimal behavior
// here because the list comprehension as written is not
// valid.
assert_snapshot!(test.completions(), @r"
bar
for
");
// TODO: it would be good if `bar` was included here, but
// the list comprehension is not yet valid and so we do not
// detect this as a definition of `bar`.
assert_snapshot!(test.completions(), @"<No completions found>");
}
#[test]
@@ -470,10 +678,7 @@ def frob(): ...
",
);
assert_snapshot!(test.completions(), @r"
f
foo
");
assert_snapshot!(test.completions(), @"foo");
}
#[test]
@@ -484,10 +689,7 @@ def frob(): ...
",
);
assert_snapshot!(test.completions(), @r"
f
foo
");
assert_snapshot!(test.completions(), @"foo");
}
#[test]
@@ -498,10 +700,7 @@ def frob(): ...
",
);
assert_snapshot!(test.completions(), @r"
f
foo
");
assert_snapshot!(test.completions(), @"foo");
}
#[test]
@@ -512,10 +711,7 @@ def frob(): ...
",
);
assert_snapshot!(test.completions(), @r"
f
foo
");
assert_snapshot!(test.completions(), @"foo");
}
#[test]
@@ -526,10 +722,7 @@ def frob(): ...
",
);
assert_snapshot!(test.completions(), @r"
f
foo
");
assert_snapshot!(test.completions(), @"foo");
}
#[test]
@@ -602,7 +795,6 @@ class Foo:
assert_snapshot!(test.completions(), @r"
Foo
b
bar
frob
quux
@@ -621,7 +813,6 @@ class Foo:
assert_snapshot!(test.completions(), @r"
Foo
b
bar
quux
");
@@ -734,6 +925,119 @@ class Foo(<CURSOR>",
");
}
#[test]
fn class_init1() {
let test = cursor_test(
"\
class Quux:
def __init__(self):
self.foo = 1
self.bar = 2
self.baz = 3
quux = Quux()
quux.<CURSOR>
",
);
assert_snapshot!(test.completions(), @r"
bar
baz
foo
__annotations__
__class__
__delattr__
__dict__
__dir__
__doc__
__eq__
__format__
__getattribute__
__getstate__
__hash__
__init__
__init_subclass__
__module__
__ne__
__new__
__reduce__
__reduce_ex__
__repr__
__setattr__
__sizeof__
__str__
__subclasshook__
");
}
#[test]
fn class_init2() {
let test = cursor_test(
"\
class Quux:
def __init__(self):
self.foo = 1
self.bar = 2
self.baz = 3
quux = Quux()
quux.b<CURSOR>
",
);
assert_snapshot!(test.completions(), @r"
bar
baz
foo
__annotations__
__class__
__delattr__
__dict__
__dir__
__doc__
__eq__
__format__
__getattribute__
__getstate__
__hash__
__init__
__init_subclass__
__module__
__ne__
__new__
__reduce__
__reduce_ex__
__repr__
__setattr__
__sizeof__
__str__
__subclasshook__
");
}
#[test]
fn class_init3() {
let test = cursor_test(
"\
class Quux:
def __init__(self):
self.foo = 1
self.bar = 2
self.<CURSOR>
self.baz = 3
",
);
// FIXME: This should list completions on `self`, which should
// include, at least, `foo` and `bar`. At time of writing
// (2025-06-04), the type of `self` is inferred as `Unknown` in
// this context. This in turn prevents us from getting a list
// of available attributes.
//
// See: https://github.com/astral-sh/ty/issues/159
assert_snapshot!(test.completions(), @"<No completions found>");
}
// We don't yet take function parameters into account.
#[test]
fn call_prefix1() {
@@ -750,7 +1054,6 @@ bar(o<CURSOR>
assert_snapshot!(test.completions(), @r"
bar
foo
o
");
}
@@ -788,7 +1091,6 @@ class C:
assert_snapshot!(test.completions(), @r"
C
bar
f
foo
self
");
@@ -825,7 +1127,6 @@ class C:
assert_snapshot!(test.completions(), @r"
C
bar
f
foo
self
");
@@ -854,15 +1155,267 @@ print(f\"{some<CURSOR>
",
);
assert_snapshot!(test.completions(), @"some_symbol");
}
#[test]
fn statically_invisible_symbols() {
let test = cursor_test(
"\
if 1 + 2 != 3:
hidden_symbol = 1
hidden_<CURSOR>
",
);
assert_snapshot!(test.completions(), @"<No completions found>");
}
#[test]
fn completions_inside_unreachable_sections() {
let test = cursor_test(
"\
import sys
if sys.platform == \"not-my-current-platform\":
only_available_in_this_branch = 1
on<CURSOR>
",
);
// TODO: ideally, `only_available_in_this_branch` should be available here, but we
// currently make no effort to provide a good IDE experience within sections that
// are unreachable
assert_snapshot!(test.completions(), @"sys");
}
#[test]
fn star_import() {
let test = cursor_test(
"\
from typing import *
Re<CURSOR>
",
);
test.assert_completions_include("Reversible");
// `ReadableBuffer` is a symbol in `typing`, but it is not re-exported
test.assert_completions_do_not_include("ReadableBuffer");
}
#[test]
fn nested_attribute_access() {
let test = cursor_test(
"\
class A:
x: str
class B:
a: A
b = B()
b.a.<CURSOR>
",
);
// FIXME: These should be flipped.
test.assert_completions_include("a");
test.assert_completions_do_not_include("x");
}
#[test]
fn ordering() {
let test = cursor_test(
"\
class A:
foo: str
_foo: str
__foo__: str
__foo: str
FOO: str
_FOO: str
__FOO__: str
__FOO: str
A.<CURSOR>
",
);
assert_snapshot!(
test.completions_if(|name| name.contains("FOO") || name.contains("foo")),
@r"
FOO
foo
__FOO__
__foo__
_FOO
__FOO
__foo
_foo
",
);
}
// Ref: https://github.com/astral-sh/ty/issues/572
#[test]
fn scope_id_missing_function_identifier1() {
let test = cursor_test(
"\
def m<CURSOR>
",
);
assert_snapshot!(test.completions(), @"<No completions found>");
}
// Ref: https://github.com/astral-sh/ty/issues/572
#[test]
fn scope_id_missing_function_identifier2() {
let test = cursor_test(
"\
def m<CURSOR>(): pass
",
);
assert_snapshot!(test.completions(), @"<No completions found>");
}
// Ref: https://github.com/astral-sh/ty/issues/572
#[test]
fn fscope_id_missing_function_identifier3() {
let test = cursor_test(
"\
def m(): pass
<CURSOR>
",
);
assert_snapshot!(test.completions(), @r"
print
some
some_symbol
m
");
}
// Ref: https://github.com/astral-sh/ty/issues/572
#[test]
fn scope_id_missing_class_identifier1() {
let test = cursor_test(
"\
class M<CURSOR>
",
);
assert_snapshot!(test.completions(), @"<No completions found>");
}
// Ref: https://github.com/astral-sh/ty/issues/572
#[test]
fn scope_id_missing_type_alias1() {
let test = cursor_test(
"\
Fo<CURSOR> = float
",
);
assert_snapshot!(test.completions(), @"Fo");
}
// Ref: https://github.com/astral-sh/ty/issues/572
#[test]
fn scope_id_missing_import1() {
let test = cursor_test(
"\
import fo<CURSOR>
",
);
assert_snapshot!(test.completions(), @"<No completions found>");
}
// Ref: https://github.com/astral-sh/ty/issues/572
#[test]
fn scope_id_missing_import2() {
let test = cursor_test(
"\
import foo as ba<CURSOR>
",
);
assert_snapshot!(test.completions(), @"<No completions found>");
}
// Ref: https://github.com/astral-sh/ty/issues/572
#[test]
fn scope_id_missing_from_import1() {
let test = cursor_test(
"\
from fo<CURSOR> import wat
",
);
assert_snapshot!(test.completions(), @"<No completions found>");
}
// Ref: https://github.com/astral-sh/ty/issues/572
#[test]
fn scope_id_missing_from_import2() {
let test = cursor_test(
"\
from foo import wa<CURSOR>
",
);
assert_snapshot!(test.completions(), @"<No completions found>");
}
// Ref: https://github.com/astral-sh/ty/issues/572
#[test]
fn scope_id_missing_from_import3() {
let test = cursor_test(
"\
from foo import wat as ba<CURSOR>
",
);
assert_snapshot!(test.completions(), @"<No completions found>");
}
// Ref: https://github.com/astral-sh/ty/issues/572
#[test]
fn scope_id_missing_try_except1() {
let test = cursor_test(
"\
try:
pass
except Type<CURSOR>:
pass
",
);
assert_snapshot!(test.completions(), @"<No completions found>");
}
// Ref: https://github.com/astral-sh/ty/issues/572
#[test]
fn scope_id_missing_global1() {
let test = cursor_test(
"\
def _():
global fo<CURSOR>
",
);
assert_snapshot!(test.completions(), @"<No completions found>");
}
impl CursorTest {
fn completions(&self) -> String {
self.completions_if(|_| true)
}
fn completions_if(&self, predicate: impl Fn(&str) -> bool) -> String {
let completions = completion(&self.db, self.file, self.cursor_offset);
if completions.is_empty() {
return "<No completions found>".to_string();
@@ -870,8 +1423,39 @@ print(f\"{some<CURSOR>
completions
.into_iter()
.map(|completion| completion.label)
.filter(|label| predicate(label))
.collect::<Vec<String>>()
.join("\n")
}
#[track_caller]
fn assert_completions_include(&self, expected: &str) {
let completions = completion(&self.db, self.file, self.cursor_offset);
assert!(
completions
.iter()
.any(|completion| completion.label == expected),
"Expected completions to include `{expected}`"
);
}
#[track_caller]
fn assert_completions_do_not_include(&self, unexpected: &str) {
let completions = completion(&self.db, self.file, self.cursor_offset);
assert!(
completions
.iter()
.all(|completion| completion.label != unexpected),
"Expected completions to not include `{unexpected}`",
);
}
}
fn tokenize(src: &str) -> Tokens {
let parsed = ruff_python_parser::parse(src, ParseOptions::from(Mode::Module))
.expect("valid Python source for token stream");
parsed.tokens().clone()
}
}

View File

@@ -0,0 +1,11 @@
# This is a regression test for `infer_expression_types`.
# ref: https://github.com/astral-sh/ruff/pull/18041#discussion_r2094573989
class C:
def f(self, other: "C"):
if self.a > other.b or self.b:
return False
if self:
return True
C().a

View File

@@ -12,7 +12,7 @@ use thiserror::Error;
use ty_python_semantic::lint::{GetLintError, Level, LintSource, RuleSelection};
use ty_python_semantic::{
ProgramSettings, PythonPath, PythonPlatform, PythonVersionFileSource, PythonVersionSource,
PythonVersionWithSource, SearchPathSettings,
PythonVersionWithSource, SearchPathSettings, SysPrefixPathOrigin,
};
use super::settings::{Settings, TerminalSettings};
@@ -182,19 +182,27 @@ impl Options {
custom_typeshed: typeshed.map(|path| path.absolute(project_root, system)),
python_path: python
.map(|python_path| {
PythonPath::from_cli_flag(python_path.absolute(project_root, system))
PythonPath::sys_prefix(
python_path.absolute(project_root, system),
SysPrefixPathOrigin::PythonCliFlag,
)
})
.or_else(|| {
std::env::var("VIRTUAL_ENV")
.ok()
.map(PythonPath::from_virtual_env_var)
std::env::var("VIRTUAL_ENV").ok().map(|virtual_env| {
PythonPath::sys_prefix(virtual_env, SysPrefixPathOrigin::VirtualEnvVar)
})
})
.or_else(|| {
std::env::var("CONDA_PREFIX")
.ok()
.map(PythonPath::from_conda_prefix_var)
std::env::var("CONDA_PREFIX").ok().map(|path| {
PythonPath::sys_prefix(path, SysPrefixPathOrigin::CondaPrefixVar)
})
})
.unwrap_or_else(|| PythonPath::Discover(project_root.to_path_buf())),
.unwrap_or_else(|| {
PythonPath::sys_prefix(
project_root.to_path_buf(),
SysPrefixPathOrigin::LocalVenv,
)
}),
}
}

View File

@@ -37,7 +37,9 @@ reveal_type(c_instance.inferred_from_other_attribute) # revealed: Unknown
# See https://github.com/astral-sh/ruff/issues/15960 for a related discussion.
reveal_type(c_instance.inferred_from_param) # revealed: Unknown | int | None
reveal_type(c_instance.declared_only) # revealed: bytes
# TODO: Should be `bytes` with no error, like mypy and pyright?
# error: [unresolved-attribute]
reveal_type(c_instance.declared_only) # revealed: Unknown
reveal_type(c_instance.declared_and_bound) # revealed: bool
@@ -64,12 +66,10 @@ C.inferred_from_value = "overwritten on class"
# This assignment is fine:
c_instance.declared_and_bound = False
# TODO: After this assignment to the attribute within this scope, we may eventually want to narrow
# the `bool` type (see above) for this instance variable to `Literal[False]` here. This is unsound
# in general (we don't know what else happened to `c_instance` between the assignment and the use
# here), but mypy and pyright support this. In conclusion, this could be `bool` but should probably
# be `Literal[False]`.
reveal_type(c_instance.declared_and_bound) # revealed: bool
# Strictly speaking, inferring this as `Literal[False]` rather than `bool` is unsound in general
# (we don't know what else happened to `c_instance` between the assignment and the use here),
# but mypy and pyright support this.
reveal_type(c_instance.declared_and_bound) # revealed: Literal[False]
```
#### Variable declared in class body and possibly bound in `__init__`
@@ -149,14 +149,16 @@ class C:
c_instance = C(True)
reveal_type(c_instance.only_declared_in_body) # revealed: str | None
reveal_type(c_instance.only_declared_in_init) # revealed: str | None
# TODO: should be `str | None` without error
# error: [unresolved-attribute]
reveal_type(c_instance.only_declared_in_init) # revealed: Unknown
reveal_type(c_instance.declared_in_body_and_init) # revealed: str | None
reveal_type(c_instance.declared_in_body_defined_in_init) # revealed: str | None
# TODO: This should be `str | None`. Fixing this requires an overhaul of the `Symbol` API,
# which is planned in https://github.com/astral-sh/ruff/issues/14297
reveal_type(c_instance.bound_in_body_declared_in_init) # revealed: Unknown | str | None
reveal_type(c_instance.bound_in_body_declared_in_init) # revealed: Unknown | Literal["a"]
reveal_type(c_instance.bound_in_body_and_init) # revealed: Unknown | None | Literal["a"]
```
@@ -187,7 +189,9 @@ reveal_type(c_instance.inferred_from_other_attribute) # revealed: Unknown
reveal_type(c_instance.inferred_from_param) # revealed: Unknown | int | None
reveal_type(c_instance.declared_only) # revealed: bytes
# TODO: should be `bytes` with no error, like mypy and pyright?
# error: [unresolved-attribute]
reveal_type(c_instance.declared_only) # revealed: Unknown
reveal_type(c_instance.declared_and_bound) # revealed: bool
@@ -260,8 +264,8 @@ class C:
self.w += None
# TODO: Mypy and pyright do not support this, but it would be great if we could
# infer `Unknown | str` or at least `Unknown | Weird | str` here.
reveal_type(C().w) # revealed: Unknown | Weird
# infer `Unknown | str` here (`Weird` is not a possible type for the `w` attribute).
reveal_type(C().w) # revealed: Unknown
```
#### Attributes defined in tuple unpackings
@@ -410,14 +414,41 @@ class C:
[... for self.a in IntIterable()]
[... for (self.b, self.c) in TupleIterable()]
[... for self.d in IntIterable() for self.e in IntIterable()]
[[... for self.f in IntIterable()] for _ in IntIterable()]
[[... for self.g in IntIterable()] for self in [D()]]
class D:
g: int
c_instance = C()
reveal_type(c_instance.a) # revealed: Unknown | int
reveal_type(c_instance.b) # revealed: Unknown | int
reveal_type(c_instance.c) # revealed: Unknown | str
reveal_type(c_instance.d) # revealed: Unknown | int
reveal_type(c_instance.e) # revealed: Unknown | int
# TODO: no error, reveal Unknown | int
# error: [unresolved-attribute]
reveal_type(c_instance.a) # revealed: Unknown
# TODO: no error, reveal Unknown | int
# error: [unresolved-attribute]
reveal_type(c_instance.b) # revealed: Unknown
# TODO: no error, reveal Unknown | str
# error: [unresolved-attribute]
reveal_type(c_instance.c) # revealed: Unknown
# TODO: no error, reveal Unknown | int
# error: [unresolved-attribute]
reveal_type(c_instance.d) # revealed: Unknown
# TODO: no error, reveal Unknown | int
# error: [unresolved-attribute]
reveal_type(c_instance.e) # revealed: Unknown
# TODO: no error, reveal Unknown | int
# error: [unresolved-attribute]
reveal_type(c_instance.f) # revealed: Unknown
# This one is correctly not resolved as an attribute:
# error: [unresolved-attribute]
reveal_type(c_instance.g) # revealed: Unknown
```
#### Conditionally declared / bound attributes
@@ -721,10 +752,7 @@ reveal_type(C.pure_class_variable) # revealed: Unknown
# error: [invalid-attribute-access] "Cannot assign to instance attribute `pure_class_variable` from the class object `<class 'C'>`"
C.pure_class_variable = "overwritten on class"
# TODO: should be `Unknown | Literal["value set in class method"]` or
# Literal["overwritten on class"]`, once/if we support local narrowing.
# error: [unresolved-attribute]
reveal_type(C.pure_class_variable) # revealed: Unknown
reveal_type(C.pure_class_variable) # revealed: Literal["overwritten on class"]
c_instance = C()
reveal_type(c_instance.pure_class_variable) # revealed: Unknown | Literal["value set in class method"]
@@ -762,19 +790,12 @@ reveal_type(c_instance.variable_with_class_default2) # revealed: Unknown | Lite
c_instance.variable_with_class_default1 = "value set on instance"
reveal_type(C.variable_with_class_default1) # revealed: str
# TODO: Could be Literal["value set on instance"], or still `str` if we choose not to
# narrow the type.
reveal_type(c_instance.variable_with_class_default1) # revealed: str
reveal_type(c_instance.variable_with_class_default1) # revealed: Literal["value set on instance"]
C.variable_with_class_default1 = "overwritten on class"
# TODO: Could be `Literal["overwritten on class"]`, or still `str` if we choose not to
# narrow the type.
reveal_type(C.variable_with_class_default1) # revealed: str
# TODO: should still be `Literal["value set on instance"]`, or `str`.
reveal_type(c_instance.variable_with_class_default1) # revealed: str
reveal_type(C.variable_with_class_default1) # revealed: Literal["overwritten on class"]
reveal_type(c_instance.variable_with_class_default1) # revealed: Literal["value set on instance"]
```
#### Descriptor attributes as class variables
@@ -928,6 +949,42 @@ def _(flag1: bool, flag2: bool):
reveal_type(C5.attr1) # revealed: Unknown | Literal["metaclass value", "class value"]
```
## Invalid access to attribute
<!-- snapshot-diagnostics -->
If a non-declared variable is used and an attribute with the same name is defined and accessible,
then we emit a subdiagnostic suggesting the use of `self.`.
(`An attribute with the same name as 'x' is defined, consider using 'self.x'` in these cases)
```py
class Foo:
x: int
def method(self):
# error: [unresolved-reference] "Name `x` used when not defined"
y = x
```
```py
class Foo:
x: int = 1
def method(self):
# error: [unresolved-reference] "Name `x` used when not defined"
y = x
```
```py
class Foo:
def __init__(self):
self.x = 1
def method(self):
# error: [unresolved-reference] "Name `x` used when not defined"
y = x
```
## Unions of attributes
If the (meta)class is a union type or if the attribute on the (meta) class has a union type, we

View File

@@ -0,0 +1,401 @@
# Overloads
When ty evaluates the call of an overloaded function, it attempts to "match" the supplied arguments
with one or more overloads. This document describes the algorithm that it uses for overload
matching, which is the same as the one mentioned in the
[spec](https://typing.python.org/en/latest/spec/overload.html#overload-call-evaluation).
## Arity check
The first step is to perform arity check. The non-overloaded cases are described in the
[function](./function.md) document.
`overloaded.pyi`:
```pyi
from typing import overload
@overload
def f() -> None: ...
@overload
def f(x: int) -> int: ...
```
```py
from overloaded import f
# These match a single overload
reveal_type(f()) # revealed: None
reveal_type(f(1)) # revealed: int
# error: [no-matching-overload] "No overload of function `f` matches arguments"
reveal_type(f("a", "b")) # revealed: Unknown
```
## Type checking
The second step is to perform type checking. This is done for all the overloads that passed the
arity check.
### Single match
`overloaded.pyi`:
```pyi
from typing import overload
@overload
def f(x: int) -> int: ...
@overload
def f(x: str) -> str: ...
@overload
def f(x: bytes) -> bytes: ...
```
Here, all of the calls below pass the arity check for all overloads, so we proceed to type checking
which filters out all but the matching overload:
```py
from overloaded import f
reveal_type(f(1)) # revealed: int
reveal_type(f("a")) # revealed: str
reveal_type(f(b"b")) # revealed: bytes
```
### Single match error
`overloaded.pyi`:
```pyi
from typing import overload
@overload
def f() -> None: ...
@overload
def f(x: int) -> int: ...
```
If the arity check only matches a single overload, it should be evaluated as a regular
(non-overloaded) function call. This means that any diagnostics resulted during type checking that
call should be reported directly and not as a `no-matching-overload` error.
```py
from overloaded import f
reveal_type(f()) # revealed: None
# TODO: This should be `invalid-argument-type` instead
# error: [no-matching-overload]
reveal_type(f("a")) # revealed: Unknown
```
### Multiple matches
`overloaded.pyi`:
```pyi
from typing import overload
class A: ...
class B(A): ...
@overload
def f(x: A) -> A: ...
@overload
def f(x: B, y: int = 0) -> B: ...
```
```py
from overloaded import A, B, f
# These calls pass the arity check, and type checking matches both overloads:
reveal_type(f(A())) # revealed: A
reveal_type(f(B())) # revealed: A
# But, in this case, the arity check filters out the first overload, so we only have one match:
reveal_type(f(B(), 1)) # revealed: B
```
## Argument type expansion
This step is performed only if the previous steps resulted in **no matches**.
In this case, the algorithm would perform
[argument type expansion](https://typing.python.org/en/latest/spec/overload.html#argument-type-expansion)
and loops over from the type checking step, evaluating the argument lists.
### Expanding the only argument
`overloaded.pyi`:
```pyi
from typing import overload
class A: ...
class B: ...
class C: ...
@overload
def f(x: A) -> A: ...
@overload
def f(x: B) -> B: ...
@overload
def f(x: C) -> C: ...
```
```py
from overloaded import A, B, C, f
def _(ab: A | B, ac: A | C, bc: B | C):
reveal_type(f(ab)) # revealed: A | B
reveal_type(f(bc)) # revealed: B | C
reveal_type(f(ac)) # revealed: A | C
```
### Expanding first argument
If the set of argument lists created by expanding the first argument evaluates successfully, the
algorithm shouldn't expand the second argument.
`overloaded.pyi`:
```pyi
from typing import Literal, overload
class A: ...
class B: ...
class C: ...
class D: ...
@overload
def f(x: A, y: C) -> A: ...
@overload
def f(x: A, y: D) -> B: ...
@overload
def f(x: B, y: C) -> C: ...
@overload
def f(x: B, y: D) -> D: ...
```
```py
from overloaded import A, B, C, D, f
def _(a_b: A | B):
reveal_type(f(a_b, C())) # revealed: A | C
reveal_type(f(a_b, D())) # revealed: B | D
# But, if it doesn't, it should expand the second argument and try again:
def _(a_b: A | B, c_d: C | D):
reveal_type(f(a_b, c_d)) # revealed: A | B | C | D
```
### Expanding second argument
If the first argument cannot be expanded, the algorithm should move on to the second argument,
keeping the first argument as is.
`overloaded.pyi`:
```pyi
from typing import overload
class A: ...
class B: ...
class C: ...
class D: ...
@overload
def f(x: A, y: B) -> B: ...
@overload
def f(x: A, y: C) -> C: ...
@overload
def f(x: B, y: D) -> D: ...
```
```py
from overloaded import A, B, C, D, f
def _(a: A, bc: B | C, cd: C | D):
# This also tests that partial matching works correctly as the argument type expansion results
# in matching the first and second overloads, but not the third one.
reveal_type(f(a, bc)) # revealed: B | C
# error: [no-matching-overload] "No overload of function `f` matches arguments"
reveal_type(f(a, cd)) # revealed: Unknown
```
### Generics (legacy)
`overloaded.pyi`:
```pyi
from typing import TypeVar, overload
_T = TypeVar("_T")
class A: ...
class B: ...
@overload
def f(x: A) -> A: ...
@overload
def f(x: _T) -> _T: ...
```
```py
from overloaded import A, f
def _(x: int, y: A | int):
reveal_type(f(x)) # revealed: int
reveal_type(f(y)) # revealed: A | int
```
### Generics (PEP 695)
```toml
[environment]
python-version = "3.12"
```
`overloaded.pyi`:
```pyi
from typing import overload
class A: ...
class B: ...
@overload
def f(x: B) -> B: ...
@overload
def f[T](x: T) -> T: ...
```
```py
from overloaded import B, f
def _(x: int, y: B | int):
reveal_type(f(x)) # revealed: int
reveal_type(f(y)) # revealed: B | int
```
### Expanding `bool`
`overloaded.pyi`:
```pyi
from typing import Literal, overload
class T: ...
class F: ...
@overload
def f(x: Literal[True]) -> T: ...
@overload
def f(x: Literal[False]) -> F: ...
```
```py
from overloaded import f
def _(flag: bool):
reveal_type(f(True)) # revealed: T
reveal_type(f(False)) # revealed: F
reveal_type(f(flag)) # revealed: T | F
```
### Expanding `tuple`
`overloaded.pyi`:
```pyi
from typing import Literal, overload
class A: ...
class B: ...
class C: ...
class D: ...
@overload
def f(x: tuple[A, int], y: tuple[int, Literal[True]]) -> A: ...
@overload
def f(x: tuple[A, int], y: tuple[int, Literal[False]]) -> B: ...
@overload
def f(x: tuple[B, int], y: tuple[int, Literal[True]]) -> C: ...
@overload
def f(x: tuple[B, int], y: tuple[int, Literal[False]]) -> D: ...
```
```py
from overloaded import A, B, f
def _(x: tuple[A | B, int], y: tuple[int, bool]):
reveal_type(f(x, y)) # revealed: A | B | C | D
```
### Expanding `type`
There's no special handling for expanding `type[A | B]` type because ty stores this type in it's
distributed form, which is `type[A] | type[B]`.
`overloaded.pyi`:
```pyi
from typing import overload
class A: ...
class B: ...
@overload
def f(x: type[A]) -> A: ...
@overload
def f(x: type[B]) -> B: ...
```
```py
from overloaded import A, B, f
def _(x: type[A | B]):
reveal_type(x) # revealed: type[A] | type[B]
reveal_type(f(x)) # revealed: A | B
```
### Expanding enums
`overloaded.pyi`:
```pyi
from enum import Enum
from typing import Literal, overload
class SomeEnum(Enum):
A = 1
B = 2
C = 3
class A: ...
class B: ...
class C: ...
@overload
def f(x: Literal[SomeEnum.A]) -> A: ...
@overload
def f(x: Literal[SomeEnum.B]) -> B: ...
@overload
def f(x: Literal[SomeEnum.C]) -> C: ...
```
```py
from overloaded import SomeEnum, A, B, C, f
def _(x: SomeEnum):
reveal_type(f(SomeEnum.A)) # revealed: A
# TODO: This should be `B` once enums are supported and are expanded
reveal_type(f(SomeEnum.B)) # revealed: A
# TODO: This should be `C` once enums are supported and are expanded
reveal_type(f(SomeEnum.C)) # revealed: A
# TODO: This should be `A | B | C` once enums are supported and are expanded
reveal_type(f(x)) # revealed: A
```

View File

@@ -797,7 +797,20 @@ C(1) < C(2) # ok
### Using `dataclass` as a function
To do
```py
from dataclasses import dataclass
class B:
x: int
# error: [missing-argument]
dataclass(B)()
# error: [invalid-argument-type]
dataclass(B)("a")
reveal_type(dataclass(B)(3).x) # revealed: int
```
## Internals

View File

@@ -699,9 +699,7 @@ class C:
descriptor = Descriptor()
C.descriptor = "something else"
# This could also be `Literal["something else"]` if we support narrowing of attribute types based on assignments
reveal_type(C.descriptor) # revealed: Unknown | int
reveal_type(C.descriptor) # revealed: Literal["something else"]
```
### Possibly unbound descriptor attributes

View File

@@ -379,6 +379,21 @@ C[None](b"bytes") # error: [no-matching-overload]
C[None](12)
```
### Synthesized methods with dataclasses
```py
from dataclasses import dataclass
from typing import Generic, TypeVar
T = TypeVar("T")
@dataclass
class A(Generic[T]):
x: T
reveal_type(A(x=1)) # revealed: A[int]
```
## Generic subclass
When a generic subclass fills its superclass's type parameter with one of its own, the actual types

View File

@@ -196,4 +196,33 @@ def constrained(f: T):
reveal_type(f()) # revealed: int | str
```
## Meta-type
The meta-type of a typevar is the same as the meta-type of the upper bound, or the union of the
meta-types of the constraints:
```py
from typing import TypeVar
T_normal = TypeVar("T_normal")
def normal(x: T_normal):
reveal_type(type(x)) # revealed: type
T_bound_object = TypeVar("T_bound_object", bound=object)
def bound_object(x: T_bound_object):
reveal_type(type(x)) # revealed: type
T_bound_int = TypeVar("T_bound_int", bound=int)
def bound_int(x: T_bound_int):
reveal_type(type(x)) # revealed: type[int]
T_constrained = TypeVar("T_constrained", int, str)
def constrained(x: T_constrained):
reveal_type(type(x)) # revealed: type[int] | type[str]
```
[generics]: https://typing.python.org/en/latest/spec/generics.html

View File

@@ -354,6 +354,18 @@ C[None](b"bytes") # error: [no-matching-overload]
C[None](12)
```
### Synthesized methods with dataclasses
```py
from dataclasses import dataclass
@dataclass
class A[T]:
x: T
reveal_type(A(x=1)) # revealed: A[int]
```
## Generic subclass
When a generic subclass fills its superclass's type parameter with one of its own, the actual types

View File

@@ -766,4 +766,23 @@ def constrained[T: (Callable[[], int], Callable[[], str])](f: T):
reveal_type(f()) # revealed: int | str
```
## Meta-type
The meta-type of a typevar is the same as the meta-type of the upper bound, or the union of the
meta-types of the constraints:
```py
def normal[T](x: T):
reveal_type(type(x)) # revealed: type
def bound_object[T: object](x: T):
reveal_type(type(x)) # revealed: type
def bound_int[T: int](x: T):
reveal_type(type(x)) # revealed: type[int]
def constrained[T: (int, str)](x: T):
reveal_type(type(x)) # revealed: type[int] | type[str]
```
[pep 695]: https://peps.python.org/pep-0695/

View File

@@ -134,9 +134,7 @@ class Property[T](NamedTuple):
name: str
value: T
# TODO: this should be supported (no error, revealed type of `Property[float]`)
# error: [invalid-argument-type]
reveal_type(Property("height", 3.4)) # revealed: Property[Unknown]
reveal_type(Property("height", 3.4)) # revealed: Property[float]
```
## Attributes on `NamedTuple`

View File

@@ -0,0 +1,318 @@
# Narrowing by assignment
## Attribute
### Basic
```py
class A:
x: int | None = None
y = None
def __init__(self):
self.z = None
a = A()
a.x = 0
a.y = 0
a.z = 0
reveal_type(a.x) # revealed: Literal[0]
reveal_type(a.y) # revealed: Literal[0]
reveal_type(a.z) # revealed: Literal[0]
# Make sure that we infer the narrowed type for eager
# scopes (class, comprehension) and the non-narrowed
# public type for lazy scopes (function)
class _:
reveal_type(a.x) # revealed: Literal[0]
reveal_type(a.y) # revealed: Literal[0]
reveal_type(a.z) # revealed: Literal[0]
[reveal_type(a.x) for _ in range(1)] # revealed: Literal[0]
[reveal_type(a.y) for _ in range(1)] # revealed: Literal[0]
[reveal_type(a.z) for _ in range(1)] # revealed: Literal[0]
def _():
reveal_type(a.x) # revealed: Unknown | int | None
reveal_type(a.y) # revealed: Unknown | None
reveal_type(a.z) # revealed: Unknown | None
if False:
a = A()
reveal_type(a.x) # revealed: Literal[0]
reveal_type(a.y) # revealed: Literal[0]
reveal_type(a.z) # revealed: Literal[0]
if True:
a = A()
reveal_type(a.x) # revealed: int | None
reveal_type(a.y) # revealed: Unknown | None
reveal_type(a.z) # revealed: Unknown | None
a.x = 0
a.y = 0
a.z = 0
reveal_type(a.x) # revealed: Literal[0]
reveal_type(a.y) # revealed: Literal[0]
reveal_type(a.z) # revealed: Literal[0]
class _:
a = A()
reveal_type(a.x) # revealed: int | None
reveal_type(a.y) # revealed: Unknown | None
reveal_type(a.z) # revealed: Unknown | None
def cond() -> bool:
return True
class _:
if False:
a = A()
reveal_type(a.x) # revealed: Literal[0]
reveal_type(a.y) # revealed: Literal[0]
reveal_type(a.z) # revealed: Literal[0]
if cond():
a = A()
reveal_type(a.x) # revealed: int | None
reveal_type(a.y) # revealed: Unknown | None
reveal_type(a.z) # revealed: Unknown | None
class _:
a = A()
class Inner:
reveal_type(a.x) # revealed: int | None
reveal_type(a.y) # revealed: Unknown | None
reveal_type(a.z) # revealed: Unknown | None
# error: [unresolved-reference]
does.nt.exist = 0
# error: [unresolved-reference]
reveal_type(does.nt.exist) # revealed: Unknown
```
### Narrowing chain
```py
class D: ...
class C:
d: D | None = None
class B:
c1: C | None = None
c2: C | None = None
class A:
b: B | None = None
a = A()
a.b = B()
a.b.c1 = C()
a.b.c2 = C()
a.b.c1.d = D()
a.b.c2.d = D()
reveal_type(a.b) # revealed: B
reveal_type(a.b.c1) # revealed: C
reveal_type(a.b.c1.d) # revealed: D
a.b.c1 = C()
reveal_type(a.b) # revealed: B
reveal_type(a.b.c1) # revealed: C
reveal_type(a.b.c1.d) # revealed: D | None
reveal_type(a.b.c2.d) # revealed: D
a.b.c1.d = D()
a.b = B()
reveal_type(a.b) # revealed: B
reveal_type(a.b.c1) # revealed: C | None
reveal_type(a.b.c2) # revealed: C | None
# error: [possibly-unbound-attribute]
reveal_type(a.b.c1.d) # revealed: D | None
# error: [possibly-unbound-attribute]
reveal_type(a.b.c2.d) # revealed: D | None
```
### Do not narrow the type of a `property` by assignment
```py
class C:
def __init__(self):
self._x: int = 0
@property
def x(self) -> int:
return self._x
@x.setter
def x(self, value: int) -> None:
self._x = abs(value)
c = C()
c.x = -1
# Don't infer `c.x` to be `Literal[-1]`
reveal_type(c.x) # revealed: int
```
### Do not narrow the type of a descriptor by assignment
```py
class Descriptor:
def __get__(self, instance: object, owner: type) -> int:
return 1
def __set__(self, instance: object, value: int) -> None:
pass
class C:
desc: Descriptor = Descriptor()
c = C()
c.desc = -1
# Don't infer `c.desc` to be `Literal[-1]`
reveal_type(c.desc) # revealed: int
```
## Subscript
### Specialization for builtin types
Type narrowing based on assignment to a subscript expression is generally unsound, because arbitrary
`__getitem__`/`__setitem__` methods on a class do not necessarily guarantee that the passed-in value
for `__setitem__` is stored and can be retrieved unmodified via `__getitem__`. Therefore, we
currently only perform assignment-based narrowing on a few built-in classes (`list`, `dict`,
`bytesarray`, `TypedDict` and `collections` types) where we are confident that this kind of
narrowing can be performed soundly. This is the same approach as pyright.
```py
from typing import TypedDict
from collections import ChainMap, defaultdict
l: list[int | None] = [None]
l[0] = 0
d: dict[int, int] = {1: 1}
d[0] = 0
b: bytearray = bytearray(b"abc")
b[0] = 0
dd: defaultdict[int, int] = defaultdict(int)
dd[0] = 0
cm: ChainMap[int, int] = ChainMap({1: 1}, {0: 0})
cm[0] = 0
# TODO: should be ChainMap[int, int]
reveal_type(cm) # revealed: ChainMap[Unknown, Unknown]
reveal_type(l[0]) # revealed: Literal[0]
reveal_type(d[0]) # revealed: Literal[0]
reveal_type(b[0]) # revealed: Literal[0]
reveal_type(dd[0]) # revealed: Literal[0]
# TODO: should be Literal[0]
reveal_type(cm[0]) # revealed: Unknown
class C:
reveal_type(l[0]) # revealed: Literal[0]
reveal_type(d[0]) # revealed: Literal[0]
reveal_type(b[0]) # revealed: Literal[0]
reveal_type(dd[0]) # revealed: Literal[0]
# TODO: should be Literal[0]
reveal_type(cm[0]) # revealed: Unknown
[reveal_type(l[0]) for _ in range(1)] # revealed: Literal[0]
[reveal_type(d[0]) for _ in range(1)] # revealed: Literal[0]
[reveal_type(b[0]) for _ in range(1)] # revealed: Literal[0]
[reveal_type(dd[0]) for _ in range(1)] # revealed: Literal[0]
# TODO: should be Literal[0]
[reveal_type(cm[0]) for _ in range(1)] # revealed: Unknown
def _():
reveal_type(l[0]) # revealed: int | None
reveal_type(d[0]) # revealed: int
reveal_type(b[0]) # revealed: int
reveal_type(dd[0]) # revealed: int
reveal_type(cm[0]) # revealed: int
class D(TypedDict):
x: int
label: str
td = D(x=1, label="a")
td["x"] = 0
# TODO: should be Literal[0]
reveal_type(td["x"]) # revealed: @Todo(TypedDict)
# error: [unresolved-reference]
does["not"]["exist"] = 0
# error: [unresolved-reference]
reveal_type(does["not"]["exist"]) # revealed: Unknown
non_subscriptable = 1
# error: [non-subscriptable]
non_subscriptable[0] = 0
# error: [non-subscriptable]
reveal_type(non_subscriptable[0]) # revealed: Unknown
```
### No narrowing for custom classes with arbitrary `__getitem__` / `__setitem__`
```py
class C:
def __init__(self):
self.l: list[str] = []
def __getitem__(self, index: int) -> str:
return self.l[index]
def __setitem__(self, index: int, value: str | int) -> None:
if len(self.l) == index:
self.l.append(str(value))
else:
self.l[index] = str(value)
c = C()
c[0] = 0
reveal_type(c[0]) # revealed: str
```
## Complex target
```py
class A:
x: list[int | None] = []
class B:
a: A | None = None
b = B()
b.a = A()
b.a.x[0] = 0
reveal_type(b.a.x[0]) # revealed: Literal[0]
class C:
reveal_type(b.a.x[0]) # revealed: Literal[0]
def _():
# error: [possibly-unbound-attribute]
reveal_type(b.a.x[0]) # revealed: Unknown | int | None
# error: [possibly-unbound-attribute]
reveal_type(b.a.x) # revealed: Unknown | list[int | None]
reveal_type(b.a) # revealed: Unknown | A | None
```
## Invalid assignments are not used for narrowing
```py
class C:
x: int | None
l: list[int]
def f(c: C, s: str):
c.x = s # error: [invalid-assignment]
reveal_type(c.x) # revealed: int | None
s = c.x # error: [invalid-assignment]
# TODO: This assignment is invalid and should result in an error.
c.l[0] = s
reveal_type(c.l[0]) # revealed: int
```

View File

@@ -53,11 +53,114 @@ constraints may no longer be valid due to a "time lag". However, it may be possi
that some of them are valid by performing a more detailed analysis (e.g. checking that the narrowing
target has not changed in all places where the function is called).
### Narrowing by attribute/subscript assignments
```py
class A:
x: str | None = None
def update_x(self, value: str | None):
self.x = value
a = A()
a.x = "a"
class B:
reveal_type(a.x) # revealed: Literal["a"]
def f():
reveal_type(a.x) # revealed: Unknown | str | None
[reveal_type(a.x) for _ in range(1)] # revealed: Literal["a"]
a = A()
class C:
reveal_type(a.x) # revealed: str | None
def g():
reveal_type(a.x) # revealed: Unknown | str | None
[reveal_type(a.x) for _ in range(1)] # revealed: str | None
a = A()
a.x = "a"
a.update_x("b")
class D:
# TODO: should be `str | None`
reveal_type(a.x) # revealed: Literal["a"]
def h():
reveal_type(a.x) # revealed: Unknown | str | None
# TODO: should be `str | None`
[reveal_type(a.x) for _ in range(1)] # revealed: Literal["a"]
```
### Narrowing by attribute/subscript assignments in nested scopes
```py
class D: ...
class C:
d: D | None = None
class B:
c1: C | None = None
c2: C | None = None
class A:
b: B | None = None
a = A()
a.b = B()
class _:
a.b.c1 = C()
class _:
a.b.c1.d = D()
a = 1
class _3:
reveal_type(a) # revealed: A
reveal_type(a.b.c1.d) # revealed: D
class _:
a = 1
# error: [unresolved-attribute]
a.b.c1.d = D()
class _3:
reveal_type(a) # revealed: A
# TODO: should be `D | None`
reveal_type(a.b.c1.d) # revealed: D
a.b.c1 = C()
a.b.c1.d = D()
class _:
a.b = B()
class _:
# error: [possibly-unbound-attribute]
reveal_type(a.b.c1.d) # revealed: D | None
reveal_type(a.b.c1) # revealed: C | None
```
### Narrowing constraints introduced in eager nested scopes
```py
g: str | None = "a"
class A:
x: str | None = None
a = A()
l: list[str | None] = [None]
def f(x: str | None):
def _():
if x is not None:
@@ -69,6 +172,14 @@ def f(x: str | None):
if g is not None:
reveal_type(g) # revealed: str
if a.x is not None:
# TODO(#17643): should be `Unknown | str`
reveal_type(a.x) # revealed: Unknown | str | None
if l[0] is not None:
# TODO(#17643): should be `str`
reveal_type(l[0]) # revealed: str | None
class C:
if x is not None:
reveal_type(x) # revealed: str
@@ -79,6 +190,14 @@ def f(x: str | None):
if g is not None:
reveal_type(g) # revealed: str
if a.x is not None:
# TODO(#17643): should be `Unknown | str`
reveal_type(a.x) # revealed: Unknown | str | None
if l[0] is not None:
# TODO(#17643): should be `str`
reveal_type(l[0]) # revealed: str | None
# TODO: should be str
# This could be fixed if we supported narrowing with if clauses in comprehensions.
[reveal_type(x) for _ in range(1) if x is not None] # revealed: str | None
@@ -89,6 +208,13 @@ def f(x: str | None):
```py
g: str | None = "a"
class A:
x: str | None = None
a = A()
l: list[str | None] = [None]
def f(x: str | None):
if x is not None:
def _():
@@ -109,6 +235,28 @@ def f(x: str | None):
reveal_type(g) # revealed: str
[reveal_type(g) for _ in range(1)] # revealed: str
if a.x is not None:
def _():
reveal_type(a.x) # revealed: Unknown | str | None
class D:
# TODO(#17643): should be `Unknown | str`
reveal_type(a.x) # revealed: Unknown | str | None
# TODO(#17643): should be `Unknown | str`
[reveal_type(a.x) for _ in range(1)] # revealed: Unknown | str | None
if l[0] is not None:
def _():
reveal_type(l[0]) # revealed: str | None
class D:
# TODO(#17643): should be `str`
reveal_type(l[0]) # revealed: str | None
# TODO(#17643): should be `str`
[reveal_type(l[0]) for _ in range(1)] # revealed: str | None
```
### Narrowing constraints introduced in multiple scopes
@@ -118,6 +266,13 @@ from typing import Literal
g: str | Literal[1] | None = "a"
class A:
x: str | Literal[1] | None = None
a = A()
l: list[str | Literal[1] | None] = [None]
def f(x: str | Literal[1] | None):
class C:
if x is not None:
@@ -140,6 +295,28 @@ def f(x: str | Literal[1] | None):
class D:
if g != 1:
reveal_type(g) # revealed: str
if a.x is not None:
def _():
if a.x != 1:
# TODO(#17643): should be `Unknown | str | None`
reveal_type(a.x) # revealed: Unknown | str | Literal[1] | None
class D:
if a.x != 1:
# TODO(#17643): should be `Unknown | str`
reveal_type(a.x) # revealed: Unknown | str | Literal[1] | None
if l[0] is not None:
def _():
if l[0] != 1:
# TODO(#17643): should be `str | None`
reveal_type(l[0]) # revealed: str | Literal[1] | None
class D:
if l[0] != 1:
# TODO(#17643): should be `str`
reveal_type(l[0]) # revealed: str | Literal[1] | None
```
### Narrowing constraints with bindings in class scope, and nested scopes

View File

@@ -26,7 +26,7 @@ def f(x: Foo):
else:
reveal_type(x) # revealed: Foo
def y(x: Bar):
def g(x: Bar):
if hasattr(x, "spam"):
reveal_type(x) # revealed: Never
reveal_type(x.spam) # revealed: Never
@@ -35,4 +35,25 @@ def y(x: Bar):
# error: [unresolved-attribute]
reveal_type(x.spam) # revealed: Unknown
def returns_bool() -> bool:
return False
class Baz:
if returns_bool():
x: int = 42
def h(obj: Baz):
reveal_type(obj) # revealed: Baz
# error: [possibly-unbound-attribute]
reveal_type(obj.x) # revealed: int
if hasattr(obj, "x"):
reveal_type(obj) # revealed: Baz & <Protocol with members 'x'>
reveal_type(obj.x) # revealed: int
else:
reveal_type(obj) # revealed: Baz & ~<Protocol with members 'x'>
# TODO: should emit `[unresolved-attribute]` and reveal `Unknown`
reveal_type(obj.x) # revealed: @Todo(map_with_boundness: intersections with negative contributions)
```

View File

@@ -389,6 +389,7 @@ not be considered protocol members by type checkers either:
class Lumberjack(Protocol):
__slots__ = ()
__match_args__ = ()
_abc_foo: str # any attribute starting with `_abc_` is excluded as a protocol attribute
x: int
def __new__(cls, x: int) -> "Lumberjack":

View File

@@ -0,0 +1,82 @@
---
source: crates/ty_test/src/lib.rs
expression: snapshot
---
---
mdtest name: attributes.md - Attributes - Invalid access to attribute
mdtest path: crates/ty_python_semantic/resources/mdtest/attributes.md
---
# Python source files
## mdtest_snippet.py
```
1 | class Foo:
2 | x: int
3 |
4 | def method(self):
5 | # error: [unresolved-reference] "Name `x` used when not defined"
6 | y = x
7 | class Foo:
8 | x: int = 1
9 |
10 | def method(self):
11 | # error: [unresolved-reference] "Name `x` used when not defined"
12 | y = x
13 | class Foo:
14 | def __init__(self):
15 | self.x = 1
16 |
17 | def method(self):
18 | # error: [unresolved-reference] "Name `x` used when not defined"
19 | y = x
```
# Diagnostics
```
error[unresolved-reference]: Name `x` used when not defined
--> src/mdtest_snippet.py:6:13
|
4 | def method(self):
5 | # error: [unresolved-reference] "Name `x` used when not defined"
6 | y = x
| ^
7 | class Foo:
8 | x: int = 1
|
info: An attribute `x` is available: consider using `self.x`
info: rule `unresolved-reference` is enabled by default
```
```
error[unresolved-reference]: Name `x` used when not defined
--> src/mdtest_snippet.py:12:13
|
10 | def method(self):
11 | # error: [unresolved-reference] "Name `x` used when not defined"
12 | y = x
| ^
13 | class Foo:
14 | def __init__(self):
|
info: An attribute `x` is available: consider using `self.x`
info: rule `unresolved-reference` is enabled by default
```
```
error[unresolved-reference]: Name `x` used when not defined
--> src/mdtest_snippet.py:19:13
|
17 | def method(self):
18 | # error: [unresolved-reference] "Name `x` used when not defined"
19 | y = x
| ^
|
info: An attribute `x` is available: consider using `self.x`
info: rule `unresolved-reference` is enabled by default
```

View File

@@ -1,8 +1,8 @@
# Unpacking
If there are not enough or too many values when unpacking, an error will occur and the types of
all variables (if nested tuple unpacking fails, only the variables within the failed tuples) is
inferred to be `Unknown`.
If there are not enough or too many values when unpacking, an error will occur and the types of all
variables (if nested tuple unpacking fails, only the variables within the failed tuples) is inferred
to be `Unknown`.
## Tuple
@@ -207,6 +207,57 @@ reveal_type(c) # revealed: int
reveal_type(d) # revealed: Literal[2]
```
## List
### Literal unpacking
```py
a, b = [1, 2]
# TODO: should be `int` for both `a` and `b`
reveal_type(a) # revealed: Unknown
reveal_type(b) # revealed: Unknown
```
### Simple unpacking
```py
def _(value: list[int]):
a, b = value
reveal_type(a) # revealed: int
reveal_type(b) # revealed: int
```
### Nested unpacking
```py
def _(value: list[list[int]]):
a, (b, c) = value
reveal_type(a) # revealed: list[int]
reveal_type(b) # revealed: int
reveal_type(c) # revealed: int
```
### Invalid nested unpacking
```py
def _(value: list[int]):
# error: [not-iterable] "Object of type `int` is not iterable"
a, (b, c) = value
reveal_type(a) # revealed: int
reveal_type(b) # revealed: Unknown
reveal_type(c) # revealed: Unknown
```
### Starred expression
```py
def _(value: list[int]):
a, *b, c = value
reveal_type(a) # revealed: int
reveal_type(b) # revealed: list[int]
reveal_type(c) # revealed: int
```
## String
### Simple unpacking
@@ -293,6 +344,18 @@ reveal_type(b) # revealed: LiteralString
reveal_type(c) # revealed: list[LiteralString]
```
### Starred expression (6)
```py
from typing_extensions import LiteralString
def _(s: LiteralString):
a, b, *c = s
reveal_type(a) # revealed: LiteralString
reveal_type(b) # revealed: LiteralString
reveal_type(c) # revealed: list[LiteralString]
```
### Unicode
```py
@@ -788,3 +851,14 @@ def _(arg: tuple[tuple[int, str], Iterable]):
# revealed: tuple[int | bytes, str | bytes]
[reveal_type((a, b)) for a, b in arg]
```
## Empty
Unpacking an empty tuple or list shouldn't raise any diagnostics.
```py
[] = []
() = ()
[] = ()
() = []
```

View File

@@ -7,7 +7,7 @@ use ruff_python_ast::statement_visitor::{StatementVisitor, walk_stmt};
use ruff_python_ast::{self as ast};
use crate::semantic_index::ast_ids::HasScopedExpressionId;
use crate::semantic_index::symbol::ScopeId;
use crate::semantic_index::place::ScopeId;
use crate::semantic_index::{SemanticIndex, global_scope, semantic_index};
use crate::types::{Truthiness, Type, infer_expression_types};
use crate::{Db, ModuleName, resolve_module};

View File

@@ -24,13 +24,13 @@ pub(crate) mod list;
mod module_name;
mod module_resolver;
mod node_key;
pub(crate) mod place;
mod program;
mod python_platform;
pub mod semantic_index;
mod semantic_model;
pub(crate) mod site_packages;
mod suppression;
pub(crate) mod symbol;
pub mod types;
mod unpack;
mod util;

View File

@@ -139,15 +139,6 @@ pub(crate) fn search_paths(db: &dyn Db) -> SearchPathIterator {
Program::get(db).search_paths(db).iter(db)
}
/// Searches for a `.venv` directory in `project_root` that contains a `pyvenv.cfg` file.
fn discover_venv_in(system: &dyn System, project_root: &SystemPath) -> Option<SystemPathBuf> {
let virtual_env_directory = project_root.join(".venv");
system
.is_file(&virtual_env_directory.join("pyvenv.cfg"))
.then_some(virtual_env_directory)
}
#[derive(Debug, PartialEq, Eq)]
pub struct SearchPaths {
/// Search paths that have been statically determined purely from reading Ruff's configuration settings.
@@ -243,68 +234,34 @@ impl SearchPaths {
static_paths.push(stdlib_path);
let (site_packages_paths, python_version) = match python_path {
PythonPath::SysPrefix(sys_prefix, origin) => {
tracing::debug!(
"Discovering site-packages paths from sys-prefix `{sys_prefix}` ({origin}')"
);
// TODO: We may want to warn here if the venv's python version is older
// than the one resolved in the program settings because it indicates
// that the `target-version` is incorrectly configured or that the
// venv is out of date.
PythonEnvironment::new(sys_prefix, *origin, system)?.into_settings(system)?
}
PythonPath::IntoSysPrefix(path, origin) => {
if *origin == SysPrefixPathOrigin::LocalVenv {
tracing::debug!("Discovering virtual environment in `{path}`");
let virtual_env_directory = path.join(".venv");
PythonPath::Resolve(target, origin) => {
tracing::debug!("Resolving {origin}: {target}");
let root = system
// If given a file, assume it's a Python executable, e.g., `.venv/bin/python3`,
// and search for a virtual environment in the root directory. Ideally, we'd
// invoke the target to determine `sys.prefix` here, but that's more complicated
// and may be deferred to uv.
.is_file(target)
.then(|| target.as_path())
.take_if(|target| {
// Avoid using the target if it doesn't look like a Python executable, e.g.,
// to deny cases like `.venv/bin/foo`
target
.file_name()
.is_some_and(|name| name.starts_with("python"))
})
.and_then(SystemPath::parent)
.and_then(SystemPath::parent)
// If not a file, use the path as given and allow let `PythonEnvironment::new`
// handle the error.
.unwrap_or(target);
PythonEnvironment::new(root, *origin, system)?.into_settings(system)?
}
PythonPath::Discover(root) => {
tracing::debug!("Discovering virtual environment in `{root}`");
discover_venv_in(db.system(), root)
.and_then(|virtual_env_path| {
tracing::debug!("Found `.venv` folder at `{}`", virtual_env_path);
PythonEnvironment::new(
virtual_env_path.clone(),
SysPrefixPathOrigin::LocalVenv,
system,
)
.and_then(|env| env.into_settings(system))
.inspect_err(|err| {
PythonEnvironment::new(
&virtual_env_directory,
SysPrefixPathOrigin::LocalVenv,
system,
)
.and_then(|venv| venv.into_settings(system))
.inspect_err(|err| {
if system.is_directory(&virtual_env_directory) {
tracing::debug!(
"Ignoring automatically detected virtual environment at `{}`: {}",
virtual_env_path,
&virtual_env_directory,
err
);
})
.ok()
}
})
.unwrap_or_else(|| {
.unwrap_or_else(|_| {
tracing::debug!("No virtual environment found");
(SitePackagesPaths::default(), None)
})
} else {
tracing::debug!("Resolving {origin}: {path}");
PythonEnvironment::new(path, *origin, system)?.into_settings(system)?
}
}
PythonPath::KnownSitePackages(paths) => (

View File

@@ -262,8 +262,10 @@ impl SearchPathSettings {
#[derive(Debug, Clone, Eq, PartialEq)]
#[cfg_attr(feature = "serde", derive(serde::Serialize, serde::Deserialize))]
pub enum PythonPath {
/// A path that represents the value of [`sys.prefix`] at runtime in Python
/// for a given Python executable.
/// A path that either represents the value of [`sys.prefix`] at runtime in Python
/// for a given Python executable, or which represents a path relative to `sys.prefix`
/// that we will attempt later to resolve into `sys.prefix`. Exactly which this variant
/// represents depends on the [`SysPrefixPathOrigin`] element in the tuple.
///
/// For the case of a virtual environment, where a
/// Python binary is at `/.venv/bin/python`, `sys.prefix` is the path to
@@ -275,13 +277,7 @@ pub enum PythonPath {
/// `/opt/homebrew/lib/python3.X/site-packages`.
///
/// [`sys.prefix`]: https://docs.python.org/3/library/sys.html#sys.prefix
SysPrefix(SystemPathBuf, SysPrefixPathOrigin),
/// Resolve a path to an executable (or environment directory) into a usable environment.
Resolve(SystemPathBuf, SysPrefixPathOrigin),
/// Tries to discover a virtual environment in the given path.
Discover(SystemPathBuf),
IntoSysPrefix(SystemPathBuf, SysPrefixPathOrigin),
/// Resolved site packages paths.
///
@@ -291,16 +287,8 @@ pub enum PythonPath {
}
impl PythonPath {
pub fn from_virtual_env_var(path: impl Into<SystemPathBuf>) -> Self {
Self::SysPrefix(path.into(), SysPrefixPathOrigin::VirtualEnvVar)
}
pub fn from_conda_prefix_var(path: impl Into<SystemPathBuf>) -> Self {
Self::Resolve(path.into(), SysPrefixPathOrigin::CondaPrefixVar)
}
pub fn from_cli_flag(path: SystemPathBuf) -> Self {
Self::Resolve(path, SysPrefixPathOrigin::PythonCliFlag)
pub fn sys_prefix(path: impl Into<SystemPathBuf>, origin: SysPrefixPathOrigin) -> Self {
Self::IntoSysPrefix(path.into(), origin)
}
}

View File

@@ -19,9 +19,9 @@ use crate::semantic_index::builder::SemanticIndexBuilder;
use crate::semantic_index::definition::{Definition, DefinitionNodeKey, Definitions};
use crate::semantic_index::expression::Expression;
use crate::semantic_index::narrowing_constraints::ScopedNarrowingConstraint;
use crate::semantic_index::symbol::{
FileScopeId, NodeWithScopeKey, NodeWithScopeRef, Scope, ScopeId, ScopeKind, ScopedSymbolId,
SymbolTable,
use crate::semantic_index::place::{
FileScopeId, NodeWithScopeKey, NodeWithScopeRef, PlaceExpr, PlaceTable, Scope, ScopeId,
ScopeKind, ScopedPlaceId,
};
use crate::semantic_index::use_def::{EagerSnapshotKey, ScopedEagerSnapshotId, UseDefMap};
@@ -30,9 +30,9 @@ mod builder;
pub mod definition;
pub mod expression;
pub(crate) mod narrowing_constraints;
pub mod place;
pub(crate) mod predicate;
mod re_exports;
pub mod symbol;
mod use_def;
mod visibility_constraints;
@@ -41,7 +41,7 @@ pub(crate) use self::use_def::{
DeclarationsIterator,
};
type SymbolMap = hashbrown::HashMap<ScopedSymbolId, (), FxBuildHasher>;
type PlaceSet = hashbrown::HashMap<ScopedPlaceId, (), FxBuildHasher>;
/// Returns the semantic index for `file`.
///
@@ -55,18 +55,18 @@ pub(crate) fn semantic_index(db: &dyn Db, file: File) -> SemanticIndex<'_> {
SemanticIndexBuilder::new(db, file, parsed).build()
}
/// Returns the symbol table for a specific `scope`.
/// Returns the place table for a specific `scope`.
///
/// Using [`symbol_table`] over [`semantic_index`] has the advantage that
/// Salsa can avoid invalidating dependent queries if this scope's symbol table
/// Using [`place_table`] over [`semantic_index`] has the advantage that
/// Salsa can avoid invalidating dependent queries if this scope's place table
/// is unchanged.
#[salsa::tracked(returns(deref))]
pub(crate) fn symbol_table<'db>(db: &'db dyn Db, scope: ScopeId<'db>) -> Arc<SymbolTable> {
pub(crate) fn place_table<'db>(db: &'db dyn Db, scope: ScopeId<'db>) -> Arc<PlaceTable> {
let file = scope.file(db);
let _span = tracing::trace_span!("symbol_table", scope=?scope.as_id(), ?file).entered();
let _span = tracing::trace_span!("place_table", scope=?scope.as_id(), ?file).entered();
let index = semantic_index(db, file);
index.symbol_table(scope.file_scope_id(db))
index.place_table(scope.file_scope_id(db))
}
/// Returns the set of modules that are imported anywhere in `file`.
@@ -113,13 +113,10 @@ pub(crate) fn attribute_assignments<'db, 's>(
let index = semantic_index(db, file);
attribute_scopes(db, class_body_scope).filter_map(|function_scope_id| {
let attribute_table = index.instance_attribute_table(function_scope_id);
let symbol = attribute_table.symbol_id_by_name(name)?;
let place_table = index.place_table(function_scope_id);
let place = place_table.place_id_by_instance_attribute_name(name)?;
let use_def = &index.use_def_maps[function_scope_id];
Some((
use_def.instance_attribute_bindings(symbol),
function_scope_id,
))
Some((use_def.public_bindings(place), function_scope_id))
})
}
@@ -167,14 +164,11 @@ pub(crate) enum EagerSnapshotResult<'map, 'db> {
NoLongerInEagerContext,
}
/// The symbol tables and use-def maps for all scopes in a file.
/// The place tables and use-def maps for all scopes in a file.
#[derive(Debug, Update)]
pub(crate) struct SemanticIndex<'db> {
/// List of all symbol tables in this file, indexed by scope.
symbol_tables: IndexVec<FileScopeId, Arc<SymbolTable>>,
/// List of all instance attribute tables in this file, indexed by scope.
instance_attribute_tables: IndexVec<FileScopeId, SymbolTable>,
/// List of all place tables in this file, indexed by scope.
place_tables: IndexVec<FileScopeId, Arc<PlaceTable>>,
/// List of all scopes in this file.
scopes: IndexVec<FileScopeId, Scope>,
@@ -195,7 +189,7 @@ pub(crate) struct SemanticIndex<'db> {
scope_ids_by_scope: IndexVec<FileScopeId, ScopeId<'db>>,
/// Map from the file-local [`FileScopeId`] to the set of explicit-global symbols it contains.
globals_by_scope: FxHashMap<FileScopeId, FxHashSet<ScopedSymbolId>>,
globals_by_scope: FxHashMap<FileScopeId, FxHashSet<ScopedPlaceId>>,
/// Use-def map for each scope in this file.
use_def_maps: IndexVec<FileScopeId, Arc<UseDefMap<'db>>>,
@@ -223,17 +217,13 @@ pub(crate) struct SemanticIndex<'db> {
}
impl<'db> SemanticIndex<'db> {
/// Returns the symbol table for a specific scope.
/// Returns the place table for a specific scope.
///
/// Use the Salsa cached [`symbol_table()`] query if you only need the
/// symbol table for a single scope.
/// Use the Salsa cached [`place_table()`] query if you only need the
/// place table for a single scope.
#[track_caller]
pub(super) fn symbol_table(&self, scope_id: FileScopeId) -> Arc<SymbolTable> {
self.symbol_tables[scope_id].clone()
}
pub(super) fn instance_attribute_table(&self, scope_id: FileScopeId) -> &SymbolTable {
&self.instance_attribute_tables[scope_id]
pub(super) fn place_table(&self, scope_id: FileScopeId) -> Arc<PlaceTable> {
self.place_tables[scope_id].clone()
}
/// Returns the use-def map for a specific scope.
@@ -259,6 +249,14 @@ impl<'db> SemanticIndex<'db> {
self.scopes_by_expression[&expression.into()]
}
/// Returns the ID of the `expression`'s enclosing scope.
pub(crate) fn try_expression_scope_id(
&self,
expression: impl Into<ExpressionNodeKey>,
) -> Option<FileScopeId> {
self.scopes_by_expression.get(&expression.into()).copied()
}
/// Returns the [`Scope`] of the `expression`'s enclosing scope.
#[allow(unused)]
#[track_caller]
@@ -278,7 +276,7 @@ impl<'db> SemanticIndex<'db> {
pub(crate) fn symbol_is_global_in_scope(
&self,
symbol: ScopedSymbolId,
symbol: ScopedPlaceId,
scope: FileScopeId,
) -> bool {
self.globals_by_scope
@@ -436,7 +434,7 @@ impl<'db> SemanticIndex<'db> {
pub(crate) fn eager_snapshot(
&self,
enclosing_scope: FileScopeId,
symbol: &str,
expr: &PlaceExpr,
nested_scope: FileScopeId,
) -> EagerSnapshotResult<'_, 'db> {
for (ancestor_scope_id, ancestor_scope) in self.ancestor_scopes(nested_scope) {
@@ -447,12 +445,12 @@ impl<'db> SemanticIndex<'db> {
return EagerSnapshotResult::NoLongerInEagerContext;
}
}
let Some(symbol_id) = self.symbol_tables[enclosing_scope].symbol_id_by_name(symbol) else {
let Some(place_id) = self.place_tables[enclosing_scope].place_id_by_expr(expr) else {
return EagerSnapshotResult::NotFound;
};
let key = EagerSnapshotKey {
enclosing_scope,
enclosing_symbol: symbol_id,
enclosing_place: place_id,
nested_scope,
};
let Some(id) = self.eager_snapshots.get(&key) else {
@@ -472,9 +470,9 @@ pub struct AncestorsIter<'a> {
}
impl<'a> AncestorsIter<'a> {
fn new(module_symbol_table: &'a SemanticIndex, start: FileScopeId) -> Self {
fn new(module_table: &'a SemanticIndex, start: FileScopeId) -> Self {
Self {
scopes: &module_symbol_table.scopes,
scopes: &module_table.scopes,
next_id: Some(start),
}
}
@@ -500,9 +498,9 @@ pub struct DescendantsIter<'a> {
}
impl<'a> DescendantsIter<'a> {
fn new(symbol_table: &'a SemanticIndex, scope_id: FileScopeId) -> Self {
let scope = &symbol_table.scopes[scope_id];
let scopes = &symbol_table.scopes[scope.descendants()];
fn new(index: &'a SemanticIndex, scope_id: FileScopeId) -> Self {
let scope = &index.scopes[scope_id];
let scopes = &index.scopes[scope.descendants()];
Self {
next_id: scope_id + 1,
@@ -537,8 +535,8 @@ pub struct ChildrenIter<'a> {
}
impl<'a> ChildrenIter<'a> {
pub(crate) fn new(module_symbol_table: &'a SemanticIndex, parent: FileScopeId) -> Self {
let descendants = DescendantsIter::new(module_symbol_table, parent);
pub(crate) fn new(module_index: &'a SemanticIndex, parent: FileScopeId) -> Self {
let descendants = DescendantsIter::new(module_index, parent);
Self {
parent,
@@ -569,21 +567,19 @@ mod tests {
use crate::db::tests::{TestDb, TestDbBuilder};
use crate::semantic_index::ast_ids::{HasScopedUseId, ScopedUseId};
use crate::semantic_index::definition::{Definition, DefinitionKind};
use crate::semantic_index::symbol::{
FileScopeId, Scope, ScopeKind, ScopedSymbolId, SymbolTable,
};
use crate::semantic_index::place::{FileScopeId, PlaceTable, Scope, ScopeKind, ScopedPlaceId};
use crate::semantic_index::use_def::UseDefMap;
use crate::semantic_index::{global_scope, semantic_index, symbol_table, use_def_map};
use crate::semantic_index::{global_scope, place_table, semantic_index, use_def_map};
impl UseDefMap<'_> {
fn first_public_binding(&self, symbol: ScopedSymbolId) -> Option<Definition<'_>> {
fn first_public_binding(&self, symbol: ScopedPlaceId) -> Option<Definition<'_>> {
self.public_bindings(symbol)
.find_map(|constrained_binding| constrained_binding.binding)
.find_map(|constrained_binding| constrained_binding.binding.definition())
}
fn first_binding_at_use(&self, use_id: ScopedUseId) -> Option<Definition<'_>> {
self.bindings_at_use(use_id)
.find_map(|constrained_binding| constrained_binding.binding)
.find_map(|constrained_binding| constrained_binding.binding.definition())
}
}
@@ -605,17 +601,17 @@ mod tests {
TestCase { db, file }
}
fn names(table: &SymbolTable) -> Vec<String> {
fn names(table: &PlaceTable) -> Vec<String> {
table
.symbols()
.map(|symbol| symbol.name().to_string())
.places()
.filter_map(|expr| Some(expr.as_name()?.to_string()))
.collect()
}
#[test]
fn empty() {
let TestCase { db, file } = test_case("");
let global_table = symbol_table(&db, global_scope(&db, file));
let global_table = place_table(&db, global_scope(&db, file));
let global_names = names(global_table);
@@ -625,7 +621,7 @@ mod tests {
#[test]
fn simple() {
let TestCase { db, file } = test_case("x");
let global_table = symbol_table(&db, global_scope(&db, file));
let global_table = place_table(&db, global_scope(&db, file));
assert_eq!(names(global_table), vec!["x"]);
}
@@ -633,7 +629,7 @@ mod tests {
#[test]
fn annotation_only() {
let TestCase { db, file } = test_case("x: int");
let global_table = symbol_table(&db, global_scope(&db, file));
let global_table = place_table(&db, global_scope(&db, file));
assert_eq!(names(global_table), vec!["int", "x"]);
// TODO record definition
@@ -643,10 +639,10 @@ mod tests {
fn import() {
let TestCase { db, file } = test_case("import foo");
let scope = global_scope(&db, file);
let global_table = symbol_table(&db, scope);
let global_table = place_table(&db, scope);
assert_eq!(names(global_table), vec!["foo"]);
let foo = global_table.symbol_id_by_name("foo").unwrap();
let foo = global_table.place_id_by_name("foo").unwrap();
let use_def = use_def_map(&db, scope);
let binding = use_def.first_public_binding(foo).unwrap();
@@ -656,7 +652,7 @@ mod tests {
#[test]
fn import_sub() {
let TestCase { db, file } = test_case("import foo.bar");
let global_table = symbol_table(&db, global_scope(&db, file));
let global_table = place_table(&db, global_scope(&db, file));
assert_eq!(names(global_table), vec!["foo"]);
}
@@ -664,7 +660,7 @@ mod tests {
#[test]
fn import_as() {
let TestCase { db, file } = test_case("import foo.bar as baz");
let global_table = symbol_table(&db, global_scope(&db, file));
let global_table = place_table(&db, global_scope(&db, file));
assert_eq!(names(global_table), vec!["baz"]);
}
@@ -673,12 +669,12 @@ mod tests {
fn import_from() {
let TestCase { db, file } = test_case("from bar import foo");
let scope = global_scope(&db, file);
let global_table = symbol_table(&db, scope);
let global_table = place_table(&db, scope);
assert_eq!(names(global_table), vec!["foo"]);
assert!(
global_table
.symbol_by_name("foo")
.place_by_name("foo")
.is_some_and(|symbol| { symbol.is_bound() && !symbol.is_used() }),
"symbols that are defined get the defined flag"
);
@@ -687,7 +683,7 @@ mod tests {
let binding = use_def
.first_public_binding(
global_table
.symbol_id_by_name("foo")
.place_id_by_name("foo")
.expect("symbol to exist"),
)
.unwrap();
@@ -698,18 +694,18 @@ mod tests {
fn assign() {
let TestCase { db, file } = test_case("x = foo");
let scope = global_scope(&db, file);
let global_table = symbol_table(&db, scope);
let global_table = place_table(&db, scope);
assert_eq!(names(global_table), vec!["foo", "x"]);
assert!(
global_table
.symbol_by_name("foo")
.place_by_name("foo")
.is_some_and(|symbol| { !symbol.is_bound() && symbol.is_used() }),
"a symbol used but not bound in a scope should have only the used flag"
);
let use_def = use_def_map(&db, scope);
let binding = use_def
.first_public_binding(global_table.symbol_id_by_name("x").expect("symbol exists"))
.first_public_binding(global_table.place_id_by_name("x").expect("symbol exists"))
.unwrap();
assert!(matches!(binding.kind(&db), DefinitionKind::Assignment(_)));
}
@@ -718,13 +714,13 @@ mod tests {
fn augmented_assignment() {
let TestCase { db, file } = test_case("x += 1");
let scope = global_scope(&db, file);
let global_table = symbol_table(&db, scope);
let global_table = place_table(&db, scope);
assert_eq!(names(global_table), vec!["x"]);
let use_def = use_def_map(&db, scope);
let binding = use_def
.first_public_binding(global_table.symbol_id_by_name("x").unwrap())
.first_public_binding(global_table.place_id_by_name("x").unwrap())
.unwrap();
assert!(matches!(
@@ -742,7 +738,7 @@ class C:
y = 2
",
);
let global_table = symbol_table(&db, global_scope(&db, file));
let global_table = place_table(&db, global_scope(&db, file));
assert_eq!(names(global_table), vec!["C", "y"]);
@@ -757,12 +753,12 @@ y = 2
assert_eq!(class_scope.kind(), ScopeKind::Class);
assert_eq!(class_scope_id.to_scope_id(&db, file).name(&db), "C");
let class_table = index.symbol_table(class_scope_id);
let class_table = index.place_table(class_scope_id);
assert_eq!(names(&class_table), vec!["x"]);
let use_def = index.use_def_map(class_scope_id);
let binding = use_def
.first_public_binding(class_table.symbol_id_by_name("x").expect("symbol exists"))
.first_public_binding(class_table.place_id_by_name("x").expect("symbol exists"))
.unwrap();
assert!(matches!(binding.kind(&db), DefinitionKind::Assignment(_)));
}
@@ -777,7 +773,7 @@ y = 2
",
);
let index = semantic_index(&db, file);
let global_table = index.symbol_table(FileScopeId::global());
let global_table = index.place_table(FileScopeId::global());
assert_eq!(names(&global_table), vec!["func", "y"]);
@@ -790,16 +786,12 @@ y = 2
assert_eq!(function_scope.kind(), ScopeKind::Function);
assert_eq!(function_scope_id.to_scope_id(&db, file).name(&db), "func");
let function_table = index.symbol_table(function_scope_id);
let function_table = index.place_table(function_scope_id);
assert_eq!(names(&function_table), vec!["x"]);
let use_def = index.use_def_map(function_scope_id);
let binding = use_def
.first_public_binding(
function_table
.symbol_id_by_name("x")
.expect("symbol exists"),
)
.first_public_binding(function_table.place_id_by_name("x").expect("symbol exists"))
.unwrap();
assert!(matches!(binding.kind(&db), DefinitionKind::Assignment(_)));
}
@@ -814,7 +806,7 @@ def f(a: str, /, b: str, c: int = 1, *args, d: int = 2, **kwargs):
);
let index = semantic_index(&db, file);
let global_table = symbol_table(&db, global_scope(&db, file));
let global_table = place_table(&db, global_scope(&db, file));
assert_eq!(names(global_table), vec!["str", "int", "f"]);
@@ -825,7 +817,7 @@ def f(a: str, /, b: str, c: int = 1, *args, d: int = 2, **kwargs):
panic!("Expected a function scope")
};
let function_table = index.symbol_table(function_scope_id);
let function_table = index.place_table(function_scope_id);
assert_eq!(
names(&function_table),
vec!["a", "b", "c", "d", "args", "kwargs"],
@@ -836,7 +828,7 @@ def f(a: str, /, b: str, c: int = 1, *args, d: int = 2, **kwargs):
let binding = use_def
.first_public_binding(
function_table
.symbol_id_by_name(name)
.place_id_by_name(name)
.expect("symbol exists"),
)
.unwrap();
@@ -845,7 +837,7 @@ def f(a: str, /, b: str, c: int = 1, *args, d: int = 2, **kwargs):
let args_binding = use_def
.first_public_binding(
function_table
.symbol_id_by_name("args")
.place_id_by_name("args")
.expect("symbol exists"),
)
.unwrap();
@@ -856,7 +848,7 @@ def f(a: str, /, b: str, c: int = 1, *args, d: int = 2, **kwargs):
let kwargs_binding = use_def
.first_public_binding(
function_table
.symbol_id_by_name("kwargs")
.place_id_by_name("kwargs")
.expect("symbol exists"),
)
.unwrap();
@@ -871,7 +863,7 @@ def f(a: str, /, b: str, c: int = 1, *args, d: int = 2, **kwargs):
let TestCase { db, file } = test_case("lambda a, b, c=1, *args, d=2, **kwargs: None");
let index = semantic_index(&db, file);
let global_table = symbol_table(&db, global_scope(&db, file));
let global_table = place_table(&db, global_scope(&db, file));
assert!(names(global_table).is_empty());
@@ -882,7 +874,7 @@ def f(a: str, /, b: str, c: int = 1, *args, d: int = 2, **kwargs):
panic!("Expected a lambda scope")
};
let lambda_table = index.symbol_table(lambda_scope_id);
let lambda_table = index.place_table(lambda_scope_id);
assert_eq!(
names(&lambda_table),
vec!["a", "b", "c", "d", "args", "kwargs"],
@@ -891,14 +883,14 @@ def f(a: str, /, b: str, c: int = 1, *args, d: int = 2, **kwargs):
let use_def = index.use_def_map(lambda_scope_id);
for name in ["a", "b", "c", "d"] {
let binding = use_def
.first_public_binding(lambda_table.symbol_id_by_name(name).expect("symbol exists"))
.first_public_binding(lambda_table.place_id_by_name(name).expect("symbol exists"))
.unwrap();
assert!(matches!(binding.kind(&db), DefinitionKind::Parameter(_)));
}
let args_binding = use_def
.first_public_binding(
lambda_table
.symbol_id_by_name("args")
.place_id_by_name("args")
.expect("symbol exists"),
)
.unwrap();
@@ -909,7 +901,7 @@ def f(a: str, /, b: str, c: int = 1, *args, d: int = 2, **kwargs):
let kwargs_binding = use_def
.first_public_binding(
lambda_table
.symbol_id_by_name("kwargs")
.place_id_by_name("kwargs")
.expect("symbol exists"),
)
.unwrap();
@@ -930,7 +922,7 @@ def f(a: str, /, b: str, c: int = 1, *args, d: int = 2, **kwargs):
);
let index = semantic_index(&db, file);
let global_table = index.symbol_table(FileScopeId::global());
let global_table = index.place_table(FileScopeId::global());
assert_eq!(names(&global_table), vec!["iter1"]);
@@ -947,7 +939,7 @@ def f(a: str, /, b: str, c: int = 1, *args, d: int = 2, **kwargs):
"<listcomp>"
);
let comprehension_symbol_table = index.symbol_table(comprehension_scope_id);
let comprehension_symbol_table = index.place_table(comprehension_scope_id);
assert_eq!(names(&comprehension_symbol_table), vec!["x", "y"]);
@@ -956,7 +948,7 @@ def f(a: str, /, b: str, c: int = 1, *args, d: int = 2, **kwargs):
let binding = use_def
.first_public_binding(
comprehension_symbol_table
.symbol_id_by_name(name)
.place_id_by_name(name)
.expect("symbol exists"),
)
.unwrap();
@@ -1023,7 +1015,7 @@ def f(a: str, /, b: str, c: int = 1, *args, d: int = 2, **kwargs):
);
let index = semantic_index(&db, file);
let global_table = index.symbol_table(FileScopeId::global());
let global_table = index.place_table(FileScopeId::global());
assert_eq!(names(&global_table), vec!["iter1"]);
@@ -1040,7 +1032,7 @@ def f(a: str, /, b: str, c: int = 1, *args, d: int = 2, **kwargs):
"<listcomp>"
);
let comprehension_symbol_table = index.symbol_table(comprehension_scope_id);
let comprehension_symbol_table = index.place_table(comprehension_scope_id);
assert_eq!(names(&comprehension_symbol_table), vec!["y", "iter2"]);
@@ -1059,7 +1051,7 @@ def f(a: str, /, b: str, c: int = 1, *args, d: int = 2, **kwargs):
"<setcomp>"
);
let inner_comprehension_symbol_table = index.symbol_table(inner_comprehension_scope_id);
let inner_comprehension_symbol_table = index.place_table(inner_comprehension_scope_id);
assert_eq!(names(&inner_comprehension_symbol_table), vec!["x"]);
}
@@ -1074,14 +1066,14 @@ with item1 as x, item2 as y:
);
let index = semantic_index(&db, file);
let global_table = index.symbol_table(FileScopeId::global());
let global_table = index.place_table(FileScopeId::global());
assert_eq!(names(&global_table), vec!["item1", "x", "item2", "y"]);
let use_def = index.use_def_map(FileScopeId::global());
for name in ["x", "y"] {
let binding = use_def
.first_public_binding(global_table.symbol_id_by_name(name).expect("symbol exists"))
.first_public_binding(global_table.place_id_by_name(name).expect("symbol exists"))
.expect("Expected with item definition for {name}");
assert!(matches!(binding.kind(&db), DefinitionKind::WithItem(_)));
}
@@ -1097,14 +1089,14 @@ with context() as (x, y):
);
let index = semantic_index(&db, file);
let global_table = index.symbol_table(FileScopeId::global());
let global_table = index.place_table(FileScopeId::global());
assert_eq!(names(&global_table), vec!["context", "x", "y"]);
let use_def = index.use_def_map(FileScopeId::global());
for name in ["x", "y"] {
let binding = use_def
.first_public_binding(global_table.symbol_id_by_name(name).expect("symbol exists"))
.first_public_binding(global_table.place_id_by_name(name).expect("symbol exists"))
.expect("Expected with item definition for {name}");
assert!(matches!(binding.kind(&db), DefinitionKind::WithItem(_)));
}
@@ -1121,7 +1113,7 @@ def func():
",
);
let index = semantic_index(&db, file);
let global_table = index.symbol_table(FileScopeId::global());
let global_table = index.place_table(FileScopeId::global());
assert_eq!(names(&global_table), vec!["func"]);
let [
@@ -1140,8 +1132,8 @@ def func():
assert_eq!(func_scope_2.kind(), ScopeKind::Function);
assert_eq!(func_scope2_id.to_scope_id(&db, file).name(&db), "func");
let func1_table = index.symbol_table(func_scope1_id);
let func2_table = index.symbol_table(func_scope2_id);
let func1_table = index.place_table(func_scope1_id);
let func2_table = index.place_table(func_scope2_id);
assert_eq!(names(&func1_table), vec!["x"]);
assert_eq!(names(&func2_table), vec!["y"]);
@@ -1149,7 +1141,7 @@ def func():
let binding = use_def
.first_public_binding(
global_table
.symbol_id_by_name("func")
.place_id_by_name("func")
.expect("symbol exists"),
)
.unwrap();
@@ -1166,7 +1158,7 @@ def func[T]():
);
let index = semantic_index(&db, file);
let global_table = index.symbol_table(FileScopeId::global());
let global_table = index.place_table(FileScopeId::global());
assert_eq!(names(&global_table), vec!["func"]);
@@ -1179,7 +1171,7 @@ def func[T]():
assert_eq!(ann_scope.kind(), ScopeKind::Annotation);
assert_eq!(ann_scope_id.to_scope_id(&db, file).name(&db), "func");
let ann_table = index.symbol_table(ann_scope_id);
let ann_table = index.place_table(ann_scope_id);
assert_eq!(names(&ann_table), vec!["T"]);
let [(func_scope_id, func_scope)] =
@@ -1189,7 +1181,7 @@ def func[T]():
};
assert_eq!(func_scope.kind(), ScopeKind::Function);
assert_eq!(func_scope_id.to_scope_id(&db, file).name(&db), "func");
let func_table = index.symbol_table(func_scope_id);
let func_table = index.place_table(func_scope_id);
assert_eq!(names(&func_table), vec!["x"]);
}
@@ -1203,7 +1195,7 @@ class C[T]:
);
let index = semantic_index(&db, file);
let global_table = index.symbol_table(FileScopeId::global());
let global_table = index.place_table(FileScopeId::global());
assert_eq!(names(&global_table), vec!["C"]);
@@ -1216,11 +1208,11 @@ class C[T]:
assert_eq!(ann_scope.kind(), ScopeKind::Annotation);
assert_eq!(ann_scope_id.to_scope_id(&db, file).name(&db), "C");
let ann_table = index.symbol_table(ann_scope_id);
let ann_table = index.place_table(ann_scope_id);
assert_eq!(names(&ann_table), vec!["T"]);
assert!(
ann_table
.symbol_by_name("T")
.place_by_name("T")
.is_some_and(|s| s.is_bound() && !s.is_used()),
"type parameters are defined by the scope that introduces them"
);
@@ -1233,7 +1225,7 @@ class C[T]:
assert_eq!(class_scope.kind(), ScopeKind::Class);
assert_eq!(class_scope_id.to_scope_id(&db, file).name(&db), "C");
assert_eq!(names(&index.symbol_table(class_scope_id)), vec!["x"]);
assert_eq!(names(&index.place_table(class_scope_id)), vec!["x"]);
}
#[test]
@@ -1361,9 +1353,9 @@ match subject:
);
let global_scope_id = global_scope(&db, file);
let global_table = symbol_table(&db, global_scope_id);
let global_table = place_table(&db, global_scope_id);
assert!(global_table.symbol_by_name("Foo").unwrap().is_used());
assert!(global_table.place_by_name("Foo").unwrap().is_used());
assert_eq!(
names(global_table),
vec![
@@ -1387,7 +1379,7 @@ match subject:
("l", 1),
] {
let binding = use_def
.first_public_binding(global_table.symbol_id_by_name(name).expect("symbol exists"))
.first_public_binding(global_table.place_id_by_name(name).expect("symbol exists"))
.expect("Expected with item definition for {name}");
if let DefinitionKind::MatchPattern(pattern) = binding.kind(&db) {
assert_eq!(pattern.index(), expected_index);
@@ -1410,14 +1402,14 @@ match 1:
);
let global_scope_id = global_scope(&db, file);
let global_table = symbol_table(&db, global_scope_id);
let global_table = place_table(&db, global_scope_id);
assert_eq!(names(global_table), vec!["first", "second"]);
let use_def = use_def_map(&db, global_scope_id);
for (name, expected_index) in [("first", 0), ("second", 0)] {
let binding = use_def
.first_public_binding(global_table.symbol_id_by_name(name).expect("symbol exists"))
.first_public_binding(global_table.place_id_by_name(name).expect("symbol exists"))
.expect("Expected with item definition for {name}");
if let DefinitionKind::MatchPattern(pattern) = binding.kind(&db) {
assert_eq!(pattern.index(), expected_index);
@@ -1431,13 +1423,13 @@ match 1:
fn for_loops_single_assignment() {
let TestCase { db, file } = test_case("for x in a: pass");
let scope = global_scope(&db, file);
let global_table = symbol_table(&db, scope);
let global_table = place_table(&db, scope);
assert_eq!(&names(global_table), &["a", "x"]);
let use_def = use_def_map(&db, scope);
let binding = use_def
.first_public_binding(global_table.symbol_id_by_name("x").unwrap())
.first_public_binding(global_table.place_id_by_name("x").unwrap())
.unwrap();
assert!(matches!(binding.kind(&db), DefinitionKind::For(_)));
@@ -1447,16 +1439,16 @@ match 1:
fn for_loops_simple_unpacking() {
let TestCase { db, file } = test_case("for (x, y) in a: pass");
let scope = global_scope(&db, file);
let global_table = symbol_table(&db, scope);
let global_table = place_table(&db, scope);
assert_eq!(&names(global_table), &["a", "x", "y"]);
let use_def = use_def_map(&db, scope);
let x_binding = use_def
.first_public_binding(global_table.symbol_id_by_name("x").unwrap())
.first_public_binding(global_table.place_id_by_name("x").unwrap())
.unwrap();
let y_binding = use_def
.first_public_binding(global_table.symbol_id_by_name("y").unwrap())
.first_public_binding(global_table.place_id_by_name("y").unwrap())
.unwrap();
assert!(matches!(x_binding.kind(&db), DefinitionKind::For(_)));
@@ -1467,13 +1459,13 @@ match 1:
fn for_loops_complex_unpacking() {
let TestCase { db, file } = test_case("for [((a,) b), (c, d)] in e: pass");
let scope = global_scope(&db, file);
let global_table = symbol_table(&db, scope);
let global_table = place_table(&db, scope);
assert_eq!(&names(global_table), &["e", "a", "b", "c", "d"]);
let use_def = use_def_map(&db, scope);
let binding = use_def
.first_public_binding(global_table.symbol_id_by_name("a").unwrap())
.first_public_binding(global_table.place_id_by_name("a").unwrap())
.unwrap();
assert!(matches!(binding.kind(&db), DefinitionKind::For(_)));

View File

@@ -6,14 +6,14 @@ use ruff_python_ast::ExprRef;
use crate::Db;
use crate::semantic_index::ast_ids::node_key::ExpressionNodeKey;
use crate::semantic_index::place::ScopeId;
use crate::semantic_index::semantic_index;
use crate::semantic_index::symbol::ScopeId;
/// AST ids for a single scope.
///
/// The motivation for building the AST ids per scope isn't about reducing invalidation because
/// the struct changes whenever the parsed AST changes. Instead, it's mainly that we can
/// build the AST ids struct when building the symbol table and also keep the property that
/// build the AST ids struct when building the place table and also keep the property that
/// IDs of outer scopes are unaffected by changes in inner scopes.
///
/// For example, we don't want that adding new statements to `foo` changes the statement id of `x = foo()` in:
@@ -28,7 +28,7 @@ use crate::semantic_index::symbol::ScopeId;
pub(crate) struct AstIds {
/// Maps expressions to their expression id.
expressions_map: FxHashMap<ExpressionNodeKey, ScopedExpressionId>,
/// Maps expressions which "use" a symbol (that is, [`ast::ExprName`]) to a use id.
/// Maps expressions which "use" a place (that is, [`ast::ExprName`], [`ast::ExprAttribute`] or [`ast::ExprSubscript`]) to a use id.
uses_map: FxHashMap<ExpressionNodeKey, ScopedUseId>,
}
@@ -49,7 +49,7 @@ fn ast_ids<'db>(db: &'db dyn Db, scope: ScopeId) -> &'db AstIds {
semantic_index(db, scope.file(db)).ast_ids(scope.file_scope_id(db))
}
/// Uniquely identifies a use of a name in a [`crate::semantic_index::symbol::FileScopeId`].
/// Uniquely identifies a use of a name in a [`crate::semantic_index::place::FileScopeId`].
#[newtype_index]
pub struct ScopedUseId;
@@ -72,6 +72,20 @@ impl HasScopedUseId for ast::ExprName {
}
}
impl HasScopedUseId for ast::ExprAttribute {
fn scoped_use_id(&self, db: &dyn Db, scope: ScopeId) -> ScopedUseId {
let expression_ref = ExprRef::from(self);
expression_ref.scoped_use_id(db, scope)
}
}
impl HasScopedUseId for ast::ExprSubscript {
fn scoped_use_id(&self, db: &dyn Db, scope: ScopeId) -> ScopedUseId {
let expression_ref = ExprRef::from(self);
expression_ref.scoped_use_id(db, scope)
}
}
impl HasScopedUseId for ast::ExprRef<'_> {
fn scoped_use_id(&self, db: &dyn Db, scope: ScopeId) -> ScopedUseId {
let ast_ids = ast_ids(db, scope);
@@ -79,7 +93,7 @@ impl HasScopedUseId for ast::ExprRef<'_> {
}
}
/// Uniquely identifies an [`ast::Expr`] in a [`crate::semantic_index::symbol::FileScopeId`].
/// Uniquely identifies an [`ast::Expr`] in a [`crate::semantic_index::place::FileScopeId`].
#[newtype_index]
#[derive(salsa::Update)]
pub struct ScopedExpressionId;

View File

@@ -24,24 +24,22 @@ use crate::semantic_index::SemanticIndex;
use crate::semantic_index::ast_ids::AstIdsBuilder;
use crate::semantic_index::ast_ids::node_key::ExpressionNodeKey;
use crate::semantic_index::definition::{
AnnotatedAssignmentDefinitionKind, AnnotatedAssignmentDefinitionNodeRef,
AssignmentDefinitionKind, AssignmentDefinitionNodeRef, ComprehensionDefinitionKind,
ComprehensionDefinitionNodeRef, Definition, DefinitionCategory, DefinitionKind,
DefinitionNodeKey, DefinitionNodeRef, Definitions, ExceptHandlerDefinitionNodeRef,
ForStmtDefinitionKind, ForStmtDefinitionNodeRef, ImportDefinitionNodeRef,
ImportFromDefinitionNodeRef, MatchPatternDefinitionNodeRef, StarImportDefinitionNodeRef,
TargetKind, WithItemDefinitionKind, WithItemDefinitionNodeRef,
AnnotatedAssignmentDefinitionNodeRef, AssignmentDefinitionNodeRef,
ComprehensionDefinitionNodeRef, Definition, DefinitionCategory, DefinitionNodeKey,
DefinitionNodeRef, Definitions, ExceptHandlerDefinitionNodeRef, ForStmtDefinitionNodeRef,
ImportDefinitionNodeRef, ImportFromDefinitionNodeRef, MatchPatternDefinitionNodeRef,
StarImportDefinitionNodeRef, WithItemDefinitionNodeRef,
};
use crate::semantic_index::expression::{Expression, ExpressionKind};
use crate::semantic_index::place::{
FileScopeId, NodeWithScopeKey, NodeWithScopeKind, NodeWithScopeRef, PlaceExpr,
PlaceTableBuilder, Scope, ScopeId, ScopeKind, ScopedPlaceId,
};
use crate::semantic_index::predicate::{
PatternPredicate, PatternPredicateKind, Predicate, PredicateNode, ScopedPredicateId,
StarImportPlaceholderPredicate,
};
use crate::semantic_index::re_exports::exported_names;
use crate::semantic_index::symbol::{
FileScopeId, NodeWithScopeKey, NodeWithScopeKind, NodeWithScopeRef, Scope, ScopeId, ScopeKind,
ScopedSymbolId, SymbolTableBuilder,
};
use crate::semantic_index::use_def::{
EagerSnapshotKey, FlowSnapshot, ScopedEagerSnapshotId, UseDefMapBuilder,
};
@@ -100,13 +98,12 @@ pub(super) struct SemanticIndexBuilder<'db> {
// Semantic Index fields
scopes: IndexVec<FileScopeId, Scope>,
scope_ids_by_scope: IndexVec<FileScopeId, ScopeId<'db>>,
symbol_tables: IndexVec<FileScopeId, SymbolTableBuilder>,
instance_attribute_tables: IndexVec<FileScopeId, SymbolTableBuilder>,
place_tables: IndexVec<FileScopeId, PlaceTableBuilder>,
ast_ids: IndexVec<FileScopeId, AstIdsBuilder>,
use_def_maps: IndexVec<FileScopeId, UseDefMapBuilder<'db>>,
scopes_by_node: FxHashMap<NodeWithScopeKey, FileScopeId>,
scopes_by_expression: FxHashMap<ExpressionNodeKey, FileScopeId>,
globals_by_scope: FxHashMap<FileScopeId, FxHashSet<ScopedSymbolId>>,
globals_by_scope: FxHashMap<FileScopeId, FxHashSet<ScopedPlaceId>>,
definitions_by_node: FxHashMap<DefinitionNodeKey, Definitions<'db>>,
expressions_by_node: FxHashMap<ExpressionNodeKey, Expression<'db>>,
imported_modules: FxHashSet<ModuleName>,
@@ -135,8 +132,7 @@ impl<'db> SemanticIndexBuilder<'db> {
has_future_annotations: false,
scopes: IndexVec::new(),
symbol_tables: IndexVec::new(),
instance_attribute_tables: IndexVec::new(),
place_tables: IndexVec::new(),
ast_ids: IndexVec::new(),
scope_ids_by_scope: IndexVec::new(),
use_def_maps: IndexVec::new(),
@@ -259,9 +255,7 @@ impl<'db> SemanticIndexBuilder<'db> {
self.try_node_context_stack_manager.enter_nested_scope();
let file_scope_id = self.scopes.push(scope);
self.symbol_tables.push(SymbolTableBuilder::default());
self.instance_attribute_tables
.push(SymbolTableBuilder::default());
self.place_tables.push(PlaceTableBuilder::default());
self.use_def_maps
.push(UseDefMapBuilder::new(is_class_scope));
let ast_id_scope = self.ast_ids.push(AstIdsBuilder::default());
@@ -301,36 +295,35 @@ impl<'db> SemanticIndexBuilder<'db> {
// If the scope that we just popped off is an eager scope, we need to "lock" our view of
// which bindings reach each of the uses in the scope. Loop through each enclosing scope,
// looking for any that bind each symbol.
// looking for any that bind each place.
for enclosing_scope_info in self.scope_stack.iter().rev() {
let enclosing_scope_id = enclosing_scope_info.file_scope_id;
let enclosing_scope_kind = self.scopes[enclosing_scope_id].kind();
let enclosing_symbol_table = &self.symbol_tables[enclosing_scope_id];
let enclosing_place_table = &self.place_tables[enclosing_scope_id];
for nested_symbol in self.symbol_tables[popped_scope_id].symbols() {
// Skip this symbol if this enclosing scope doesn't contain any bindings for it.
// Note that even if this symbol is bound in the popped scope,
for nested_place in self.place_tables[popped_scope_id].places() {
// Skip this place if this enclosing scope doesn't contain any bindings for it.
// Note that even if this place is bound in the popped scope,
// it may refer to the enclosing scope bindings
// so we also need to snapshot the bindings of the enclosing scope.
let Some(enclosing_symbol_id) =
enclosing_symbol_table.symbol_id_by_name(nested_symbol.name())
let Some(enclosing_place_id) = enclosing_place_table.place_id_by_expr(nested_place)
else {
continue;
};
let enclosing_symbol = enclosing_symbol_table.symbol(enclosing_symbol_id);
let enclosing_place = enclosing_place_table.place_expr(enclosing_place_id);
// Snapshot the state of this symbol that are visible at this point in this
// Snapshot the state of this place that are visible at this point in this
// enclosing scope.
let key = EagerSnapshotKey {
enclosing_scope: enclosing_scope_id,
enclosing_symbol: enclosing_symbol_id,
enclosing_place: enclosing_place_id,
nested_scope: popped_scope_id,
};
let eager_snapshot = self.use_def_maps[enclosing_scope_id].snapshot_eager_state(
enclosing_symbol_id,
enclosing_place_id,
enclosing_scope_kind,
enclosing_symbol.is_bound(),
enclosing_place,
);
self.eager_snapshots.insert(key, eager_snapshot);
}
@@ -338,7 +331,7 @@ impl<'db> SemanticIndexBuilder<'db> {
// Lazy scopes are "sticky": once we see a lazy scope we stop doing lookups
// eagerly, even if we would encounter another eager enclosing scope later on.
// Also, narrowing constraints outside a lazy scope are not applicable.
// TODO: If the symbol has never been rewritten, they are applicable.
// TODO: If the place has never been rewritten, they are applicable.
if !enclosing_scope_kind.is_eager() {
break;
}
@@ -347,14 +340,9 @@ impl<'db> SemanticIndexBuilder<'db> {
popped_scope_id
}
fn current_symbol_table(&mut self) -> &mut SymbolTableBuilder {
fn current_place_table(&mut self) -> &mut PlaceTableBuilder {
let scope_id = self.current_scope();
&mut self.symbol_tables[scope_id]
}
fn current_attribute_table(&mut self) -> &mut SymbolTableBuilder {
let scope_id = self.current_scope();
&mut self.instance_attribute_tables[scope_id]
&mut self.place_tables[scope_id]
}
fn current_use_def_map_mut(&mut self) -> &mut UseDefMapBuilder<'db> {
@@ -389,34 +377,36 @@ impl<'db> SemanticIndexBuilder<'db> {
self.current_use_def_map_mut().merge(state);
}
/// Add a symbol to the symbol table and the use-def map.
/// Return the [`ScopedSymbolId`] that uniquely identifies the symbol in both.
fn add_symbol(&mut self, name: Name) -> ScopedSymbolId {
let (symbol_id, added) = self.current_symbol_table().add_symbol(name);
/// Add a symbol to the place table and the use-def map.
/// Return the [`ScopedPlaceId`] that uniquely identifies the symbol in both.
fn add_symbol(&mut self, name: Name) -> ScopedPlaceId {
let (place_id, added) = self.current_place_table().add_symbol(name);
if added {
self.current_use_def_map_mut().add_symbol(symbol_id);
self.current_use_def_map_mut().add_place(place_id);
}
symbol_id
place_id
}
fn add_attribute(&mut self, name: Name) -> ScopedSymbolId {
let (symbol_id, added) = self.current_attribute_table().add_symbol(name);
/// Add a place to the place table and the use-def map.
/// Return the [`ScopedPlaceId`] that uniquely identifies the place in both.
fn add_place(&mut self, place_expr: PlaceExpr) -> ScopedPlaceId {
let (place_id, added) = self.current_place_table().add_place(place_expr);
if added {
self.current_use_def_map_mut().add_attribute(symbol_id);
self.current_use_def_map_mut().add_place(place_id);
}
symbol_id
place_id
}
fn mark_symbol_bound(&mut self, id: ScopedSymbolId) {
self.current_symbol_table().mark_symbol_bound(id);
fn mark_place_bound(&mut self, id: ScopedPlaceId) {
self.current_place_table().mark_place_bound(id);
}
fn mark_symbol_declared(&mut self, id: ScopedSymbolId) {
self.current_symbol_table().mark_symbol_declared(id);
fn mark_place_declared(&mut self, id: ScopedPlaceId) {
self.current_place_table().mark_place_declared(id);
}
fn mark_symbol_used(&mut self, id: ScopedSymbolId) {
self.current_symbol_table().mark_symbol_used(id);
fn mark_place_used(&mut self, id: ScopedPlaceId) {
self.current_place_table().mark_place_used(id);
}
fn add_entry_for_definition_key(&mut self, key: DefinitionNodeKey) -> &mut Definitions<'db> {
@@ -432,11 +422,10 @@ impl<'db> SemanticIndexBuilder<'db> {
/// for all nodes *except* [`ast::Alias`] nodes representing `*` imports.
fn add_definition(
&mut self,
symbol: ScopedSymbolId,
place: ScopedPlaceId,
definition_node: impl Into<DefinitionNodeRef<'db>> + std::fmt::Debug + Copy,
) -> Definition<'db> {
let (definition, num_definitions) =
self.push_additional_definition(symbol, definition_node);
let (definition, num_definitions) = self.push_additional_definition(place, definition_node);
debug_assert_eq!(
num_definitions, 1,
"Attempted to create multiple `Definition`s associated with AST node {definition_node:?}"
@@ -444,6 +433,22 @@ impl<'db> SemanticIndexBuilder<'db> {
definition
}
fn delete_associated_bindings(&mut self, place: ScopedPlaceId) {
let scope = self.current_scope();
// Don't delete associated bindings if the scope is a class scope & place is a name (it's never visible to nested scopes)
if self.scopes[scope].kind() == ScopeKind::Class
&& self.place_tables[scope].place_expr(place).is_name()
{
return;
}
for associated_place in self.place_tables[scope].associated_place_ids(place) {
let is_place_name = self.place_tables[scope]
.place_expr(associated_place)
.is_name();
self.use_def_maps[scope].delete_binding(associated_place, is_place_name);
}
}
/// Push a new [`Definition`] onto the list of definitions
/// associated with the `definition_node` AST node.
///
@@ -457,7 +462,7 @@ impl<'db> SemanticIndexBuilder<'db> {
/// prefer to use `self.add_definition()`, which ensures that this invariant is maintained.
fn push_additional_definition(
&mut self,
symbol: ScopedSymbolId,
place: ScopedPlaceId,
definition_node: impl Into<DefinitionNodeRef<'db>>,
) -> (Definition<'db>, usize) {
let definition_node: DefinitionNodeRef<'_> = definition_node.into();
@@ -471,7 +476,7 @@ impl<'db> SemanticIndexBuilder<'db> {
self.db,
self.file,
self.current_scope(),
symbol,
place,
kind,
is_reexported,
countme::Count::default(),
@@ -484,19 +489,24 @@ impl<'db> SemanticIndexBuilder<'db> {
};
if category.is_binding() {
self.mark_symbol_bound(symbol);
self.mark_place_bound(place);
}
if category.is_declaration() {
self.mark_symbol_declared(symbol);
self.mark_place_declared(place);
}
let is_place_name = self.current_place_table().place_expr(place).is_name();
let use_def = self.current_use_def_map_mut();
match category {
DefinitionCategory::DeclarationAndBinding => {
use_def.record_declaration_and_binding(symbol, definition);
use_def.record_declaration_and_binding(place, definition, is_place_name);
self.delete_associated_bindings(place);
}
DefinitionCategory::Declaration => use_def.record_declaration(place, definition),
DefinitionCategory::Binding => {
use_def.record_binding(place, definition, is_place_name);
self.delete_associated_bindings(place);
}
DefinitionCategory::Declaration => use_def.record_declaration(symbol, definition),
DefinitionCategory::Binding => use_def.record_binding(symbol, definition),
}
let mut try_node_stack_manager = std::mem::take(&mut self.try_node_context_stack_manager);
@@ -506,25 +516,6 @@ impl<'db> SemanticIndexBuilder<'db> {
(definition, num_definitions)
}
fn add_attribute_definition(
&mut self,
symbol: ScopedSymbolId,
definition_kind: DefinitionKind<'db>,
) -> Definition {
let definition = Definition::new(
self.db,
self.file,
self.current_scope(),
symbol,
definition_kind,
false,
countme::Count::default(),
);
self.current_use_def_map_mut()
.record_attribute_binding(symbol, definition);
definition
}
fn record_expression_narrowing_constraint(
&mut self,
precide_node: &ast::Expr,
@@ -684,28 +675,6 @@ impl<'db> SemanticIndexBuilder<'db> {
self.current_assignments.last_mut()
}
/// Records the fact that we saw an attribute assignment of the form
/// `object.attr: <annotation>( = …)` or `object.attr = <value>`.
fn register_attribute_assignment(
&mut self,
object: &ast::Expr,
attr: &'db ast::Identifier,
definition_kind: DefinitionKind<'db>,
) {
if self.is_method_of_class().is_some() {
// We only care about attribute assignments to the first parameter of a method,
// i.e. typically `self` or `cls`.
let accessed_object_refers_to_first_parameter =
object.as_name_expr().map(|name| name.id.as_str())
== self.current_first_parameter_name;
if accessed_object_refers_to_first_parameter {
let symbol = self.add_attribute(attr.id().clone());
self.add_attribute_definition(symbol, definition_kind);
}
}
}
fn predicate_kind(&mut self, pattern: &ast::Pattern) -> PatternPredicateKind<'db> {
match pattern {
ast::Pattern::MatchValue(pattern) => {
@@ -850,8 +819,8 @@ impl<'db> SemanticIndexBuilder<'db> {
// TODO create Definition for PEP 695 typevars
// note that the "bound" on the typevar is a totally different thing than whether
// or not a name is "bound" by a typevar declaration; the latter is always true.
self.mark_symbol_bound(symbol);
self.mark_symbol_declared(symbol);
self.mark_place_bound(symbol);
self.mark_place_declared(symbol);
if let Some(bounds) = bound {
self.visit_expr(bounds);
}
@@ -1022,7 +991,7 @@ impl<'db> SemanticIndexBuilder<'db> {
));
Some(unpackable.as_current_assignment(unpack))
}
ast::Expr::Name(_) | ast::Expr::Attribute(_) => {
ast::Expr::Name(_) | ast::Expr::Attribute(_) | ast::Expr::Subscript(_) => {
Some(unpackable.as_current_assignment(None))
}
_ => None,
@@ -1050,18 +1019,12 @@ impl<'db> SemanticIndexBuilder<'db> {
assert_eq!(&self.current_assignments, &[]);
let mut symbol_tables: IndexVec<_, _> = self
.symbol_tables
let mut place_tables: IndexVec<_, _> = self
.place_tables
.into_iter()
.map(|builder| Arc::new(builder.finish()))
.collect();
let mut instance_attribute_tables: IndexVec<_, _> = self
.instance_attribute_tables
.into_iter()
.map(SymbolTableBuilder::finish)
.collect();
let mut use_def_maps: IndexVec<_, _> = self
.use_def_maps
.into_iter()
@@ -1075,8 +1038,7 @@ impl<'db> SemanticIndexBuilder<'db> {
.collect();
self.scopes.shrink_to_fit();
symbol_tables.shrink_to_fit();
instance_attribute_tables.shrink_to_fit();
place_tables.shrink_to_fit();
use_def_maps.shrink_to_fit();
ast_ids.shrink_to_fit();
self.scopes_by_expression.shrink_to_fit();
@@ -1089,8 +1051,7 @@ impl<'db> SemanticIndexBuilder<'db> {
self.globals_by_scope.shrink_to_fit();
SemanticIndex {
symbol_tables,
instance_attribute_tables,
place_tables,
scopes: self.scopes,
definitions_by_node: self.definitions_by_node,
expressions_by_node: self.expressions_by_node,
@@ -1213,7 +1174,7 @@ where
// used to collect all the overloaded definitions of a function. This needs to be
// done on the `Identifier` node as opposed to `ExprName` because that's what the
// AST uses.
self.mark_symbol_used(symbol);
self.mark_place_used(symbol);
let use_id = self.current_ast_ids().record_use(name);
self.current_use_def_map_mut()
.record_use(symbol, use_id, NodeKey::from_node(name));
@@ -1356,7 +1317,10 @@ where
// For more details, see the doc-comment on `StarImportPlaceholderPredicate`.
for export in exported_names(self.db, referenced_module) {
let symbol_id = self.add_symbol(export.clone());
let node_ref = StarImportDefinitionNodeRef { node, symbol_id };
let node_ref = StarImportDefinitionNodeRef {
node,
place_id: symbol_id,
};
let star_import = StarImportPlaceholderPredicate::new(
self.db,
self.file,
@@ -1365,7 +1329,7 @@ where
);
let pre_definition =
self.current_use_def_map().single_symbol_snapshot(symbol_id);
self.current_use_def_map().single_place_snapshot(symbol_id);
self.push_additional_definition(symbol_id, node_ref);
self.current_use_def_map_mut()
.record_and_negate_star_import_visibility_constraint(
@@ -1920,8 +1884,8 @@ where
ast::Stmt::Global(ast::StmtGlobal { range: _, names }) => {
for name in names {
let symbol_id = self.add_symbol(name.id.clone());
let symbol_table = self.current_symbol_table();
let symbol = symbol_table.symbol(symbol_id);
let symbol_table = self.current_place_table();
let symbol = symbol_table.place_expr(symbol_id);
if symbol.is_bound() || symbol.is_declared() || symbol.is_used() {
self.report_semantic_error(SemanticSyntaxError {
kind: SemanticSyntaxErrorKind::LoadBeforeGlobalDeclaration {
@@ -1942,9 +1906,9 @@ where
}
ast::Stmt::Delete(ast::StmtDelete { targets, range: _ }) => {
for target in targets {
if let ast::Expr::Name(ast::ExprName { id, .. }) = target {
let symbol_id = self.add_symbol(id.clone());
self.current_symbol_table().mark_symbol_used(symbol_id);
if let Ok(target) = PlaceExpr::try_from(target) {
let place_id = self.add_place(target);
self.current_place_table().mark_place_used(place_id);
}
}
walk_stmt(self, stmt);
@@ -1971,109 +1935,133 @@ where
let node_key = NodeKey::from_node(expr);
match expr {
ast::Expr::Name(ast::ExprName { id, ctx, .. }) => {
let (is_use, is_definition) = match (ctx, self.current_assignment()) {
(ast::ExprContext::Store, Some(CurrentAssignment::AugAssign(_))) => {
// For augmented assignment, the target expression is also used.
(true, true)
}
(ast::ExprContext::Load, _) => (true, false),
(ast::ExprContext::Store, _) => (false, true),
(ast::ExprContext::Del, _) => (false, true),
(ast::ExprContext::Invalid, _) => (false, false),
};
let symbol = self.add_symbol(id.clone());
ast::Expr::Name(ast::ExprName { ctx, .. })
| ast::Expr::Attribute(ast::ExprAttribute { ctx, .. })
| ast::Expr::Subscript(ast::ExprSubscript { ctx, .. }) => {
if let Ok(mut place_expr) = PlaceExpr::try_from(expr) {
if self.is_method_of_class().is_some() {
// We specifically mark attribute assignments to the first parameter of a method,
// i.e. typically `self` or `cls`.
let accessed_object_refers_to_first_parameter = self
.current_first_parameter_name
.is_some_and(|fst| place_expr.root_name().as_str() == fst);
if is_use {
self.mark_symbol_used(symbol);
let use_id = self.current_ast_ids().record_use(expr);
if accessed_object_refers_to_first_parameter && place_expr.is_member() {
place_expr.mark_instance_attribute();
}
}
let (is_use, is_definition) = match (ctx, self.current_assignment()) {
(ast::ExprContext::Store, Some(CurrentAssignment::AugAssign(_))) => {
// For augmented assignment, the target expression is also used.
(true, true)
}
(ast::ExprContext::Load, _) => (true, false),
(ast::ExprContext::Store, _) => (false, true),
(ast::ExprContext::Del, _) => (false, true),
(ast::ExprContext::Invalid, _) => (false, false),
};
let place_id = self.add_place(place_expr);
if is_use {
self.mark_place_used(place_id);
let use_id = self.current_ast_ids().record_use(expr);
self.current_use_def_map_mut()
.record_use(place_id, use_id, node_key);
}
if is_definition {
match self.current_assignment() {
Some(CurrentAssignment::Assign { node, unpack }) => {
self.add_definition(
place_id,
AssignmentDefinitionNodeRef {
unpack,
value: &node.value,
target: expr,
},
);
}
Some(CurrentAssignment::AnnAssign(ann_assign)) => {
self.add_standalone_type_expression(&ann_assign.annotation);
self.add_definition(
place_id,
AnnotatedAssignmentDefinitionNodeRef {
node: ann_assign,
annotation: &ann_assign.annotation,
value: ann_assign.value.as_deref(),
target: expr,
},
);
}
Some(CurrentAssignment::AugAssign(aug_assign)) => {
self.add_definition(place_id, aug_assign);
}
Some(CurrentAssignment::For { node, unpack }) => {
self.add_definition(
place_id,
ForStmtDefinitionNodeRef {
unpack,
iterable: &node.iter,
target: expr,
is_async: node.is_async,
},
);
}
Some(CurrentAssignment::Named(named)) => {
// TODO(dhruvmanila): If the current scope is a comprehension, then the
// named expression is implicitly nonlocal. This is yet to be
// implemented.
self.add_definition(place_id, named);
}
Some(CurrentAssignment::Comprehension {
unpack,
node,
first,
}) => {
self.add_definition(
place_id,
ComprehensionDefinitionNodeRef {
unpack,
iterable: &node.iter,
target: expr,
first,
is_async: node.is_async,
},
);
}
Some(CurrentAssignment::WithItem {
item,
is_async,
unpack,
}) => {
self.add_definition(
place_id,
WithItemDefinitionNodeRef {
unpack,
context_expr: &item.context_expr,
target: expr,
is_async,
},
);
}
None => {}
}
}
if let Some(unpack_position) = self
.current_assignment_mut()
.and_then(CurrentAssignment::unpack_position_mut)
{
*unpack_position = UnpackPosition::Other;
}
}
// Track reachability of attribute expressions to silence `unresolved-attribute`
// diagnostics in unreachable code.
if expr.is_attribute_expr() {
self.current_use_def_map_mut()
.record_use(symbol, use_id, node_key);
}
if is_definition {
match self.current_assignment() {
Some(CurrentAssignment::Assign { node, unpack }) => {
self.add_definition(
symbol,
AssignmentDefinitionNodeRef {
unpack,
value: &node.value,
target: expr,
},
);
}
Some(CurrentAssignment::AnnAssign(ann_assign)) => {
self.add_definition(
symbol,
AnnotatedAssignmentDefinitionNodeRef {
node: ann_assign,
annotation: &ann_assign.annotation,
value: ann_assign.value.as_deref(),
target: expr,
},
);
}
Some(CurrentAssignment::AugAssign(aug_assign)) => {
self.add_definition(symbol, aug_assign);
}
Some(CurrentAssignment::For { node, unpack }) => {
self.add_definition(
symbol,
ForStmtDefinitionNodeRef {
unpack,
iterable: &node.iter,
target: expr,
is_async: node.is_async,
},
);
}
Some(CurrentAssignment::Named(named)) => {
// TODO(dhruvmanila): If the current scope is a comprehension, then the
// named expression is implicitly nonlocal. This is yet to be
// implemented.
self.add_definition(symbol, named);
}
Some(CurrentAssignment::Comprehension {
unpack,
node,
first,
}) => {
self.add_definition(
symbol,
ComprehensionDefinitionNodeRef {
unpack,
iterable: &node.iter,
target: expr,
first,
is_async: node.is_async,
},
);
}
Some(CurrentAssignment::WithItem {
item,
is_async,
unpack,
}) => {
self.add_definition(
symbol,
WithItemDefinitionNodeRef {
unpack,
context_expr: &item.context_expr,
target: expr,
is_async,
},
);
}
None => {}
}
}
if let Some(unpack_position) = self
.current_assignment_mut()
.and_then(CurrentAssignment::unpack_position_mut)
{
*unpack_position = UnpackPosition::Other;
.record_node_reachability(node_key);
}
walk_expr(self, expr);
@@ -2239,125 +2227,6 @@ where
self.simplify_visibility_constraints(pre_op);
}
ast::Expr::Attribute(ast::ExprAttribute {
value: object,
attr,
ctx,
range: _,
}) => {
if ctx.is_store() {
match self.current_assignment() {
Some(CurrentAssignment::Assign { node, unpack, .. }) => {
// SAFETY: `value` and `expr` belong to the `self.module` tree
#[expect(unsafe_code)]
let assignment = AssignmentDefinitionKind::new(
TargetKind::from(unpack),
unsafe { AstNodeRef::new(self.module.clone(), &node.value) },
unsafe { AstNodeRef::new(self.module.clone(), expr) },
);
self.register_attribute_assignment(
object,
attr,
DefinitionKind::Assignment(assignment),
);
}
Some(CurrentAssignment::AnnAssign(ann_assign)) => {
self.add_standalone_type_expression(&ann_assign.annotation);
// SAFETY: `annotation`, `value` and `expr` belong to the `self.module` tree
#[expect(unsafe_code)]
let assignment = AnnotatedAssignmentDefinitionKind::new(
unsafe {
AstNodeRef::new(self.module.clone(), &ann_assign.annotation)
},
ann_assign.value.as_deref().map(|value| unsafe {
AstNodeRef::new(self.module.clone(), value)
}),
unsafe { AstNodeRef::new(self.module.clone(), expr) },
);
self.register_attribute_assignment(
object,
attr,
DefinitionKind::AnnotatedAssignment(assignment),
);
}
Some(CurrentAssignment::For { node, unpack, .. }) => {
// // SAFETY: `iter` and `expr` belong to the `self.module` tree
#[expect(unsafe_code)]
let assignment = ForStmtDefinitionKind::new(
TargetKind::from(unpack),
unsafe { AstNodeRef::new(self.module.clone(), &node.iter) },
unsafe { AstNodeRef::new(self.module.clone(), expr) },
node.is_async,
);
self.register_attribute_assignment(
object,
attr,
DefinitionKind::For(assignment),
);
}
Some(CurrentAssignment::WithItem {
item,
unpack,
is_async,
..
}) => {
// SAFETY: `context_expr` and `expr` belong to the `self.module` tree
#[expect(unsafe_code)]
let assignment = WithItemDefinitionKind::new(
TargetKind::from(unpack),
unsafe { AstNodeRef::new(self.module.clone(), &item.context_expr) },
unsafe { AstNodeRef::new(self.module.clone(), expr) },
is_async,
);
self.register_attribute_assignment(
object,
attr,
DefinitionKind::WithItem(assignment),
);
}
Some(CurrentAssignment::Comprehension {
unpack,
node,
first,
}) => {
// SAFETY: `iter` and `expr` belong to the `self.module` tree
#[expect(unsafe_code)]
let assignment = ComprehensionDefinitionKind {
target_kind: TargetKind::from(unpack),
iterable: unsafe {
AstNodeRef::new(self.module.clone(), &node.iter)
},
target: unsafe { AstNodeRef::new(self.module.clone(), expr) },
first,
is_async: node.is_async,
};
// Temporarily move to the scope of the method to which the instance attribute is defined.
// SAFETY: `self.scope_stack` is not empty because the targets in comprehensions should always introduce a new scope.
let scope = self.scope_stack.pop().expect("The popped scope must be a comprehension, which must have a parent scope");
self.register_attribute_assignment(
object,
attr,
DefinitionKind::Comprehension(assignment),
);
self.scope_stack.push(scope);
}
Some(CurrentAssignment::AugAssign(_)) => {
// TODO:
}
Some(CurrentAssignment::Named(_)) => {
// A named expression whose target is an attribute is syntactically prohibited
}
None => {}
}
}
// Track reachability of attribute expressions to silence `unresolved-attribute`
// diagnostics in unreachable code.
self.current_use_def_map_mut()
.record_node_reachability(node_key);
walk_expr(self, expr);
}
ast::Expr::StringLiteral(_) => {
// Track reachability of string literals, as they could be a stringified annotation
// with child expressions whose reachability we are interested in.

View File

@@ -8,16 +8,16 @@ use ruff_text_size::{Ranged, TextRange};
use crate::Db;
use crate::ast_node_ref::AstNodeRef;
use crate::node_key::NodeKey;
use crate::semantic_index::symbol::{FileScopeId, ScopeId, ScopedSymbolId};
use crate::semantic_index::place::{FileScopeId, ScopeId, ScopedPlaceId};
use crate::unpack::{Unpack, UnpackPosition};
/// A definition of a symbol.
/// A definition of a place.
///
/// ## ID stability
/// The `Definition`'s ID is stable when the only field that change is its `kind` (AST node).
///
/// The `Definition` changes when the `file`, `scope`, or `symbol` change. This can be
/// because a new scope gets inserted before the `Definition` or a new symbol is inserted
/// The `Definition` changes when the `file`, `scope`, or `place` change. This can be
/// because a new scope gets inserted before the `Definition` or a new place is inserted
/// before this `Definition`. However, the ID can be considered stable and it is okay to use
/// `Definition` in cross-module` salsa queries or as a field on other salsa tracked structs.
#[salsa::tracked(debug)]
@@ -28,8 +28,8 @@ pub struct Definition<'db> {
/// The scope in which the definition occurs.
pub(crate) file_scope: FileScopeId,
/// The symbol defined.
pub(crate) symbol: ScopedSymbolId,
/// The place ID of the definition.
pub(crate) place: ScopedPlaceId,
/// WARNING: Only access this field when doing type inference for the same
/// file as where `Definition` is defined to avoid cross-file query dependencies.
@@ -89,6 +89,39 @@ impl<'a, 'db> IntoIterator for &'a Definitions<'db> {
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, salsa::Update)]
pub(crate) enum DefinitionState<'db> {
Defined(Definition<'db>),
/// Represents the implicit "unbound"/"undeclared" definition of every place.
Undefined,
/// Represents a definition that has been deleted.
/// This used when an attribute/subscript definition (such as `x.y = ...`, `x[0] = ...`) becomes obsolete due to a reassignment of the root place.
Deleted,
}
impl<'db> DefinitionState<'db> {
pub(crate) fn is_defined_and(self, f: impl Fn(Definition<'db>) -> bool) -> bool {
matches!(self, DefinitionState::Defined(def) if f(def))
}
pub(crate) fn is_undefined_or(self, f: impl Fn(Definition<'db>) -> bool) -> bool {
matches!(self, DefinitionState::Undefined)
|| matches!(self, DefinitionState::Defined(def) if f(def))
}
pub(crate) fn is_undefined(self) -> bool {
matches!(self, DefinitionState::Undefined)
}
#[allow(unused)]
pub(crate) fn definition(self) -> Option<Definition<'db>> {
match self {
DefinitionState::Defined(def) => Some(def),
DefinitionState::Deleted | DefinitionState::Undefined => None,
}
}
}
#[derive(Copy, Clone, Debug)]
pub(crate) enum DefinitionNodeRef<'a> {
Import(ImportDefinitionNodeRef<'a>),
@@ -232,7 +265,7 @@ pub(crate) struct ImportDefinitionNodeRef<'a> {
#[derive(Copy, Clone, Debug)]
pub(crate) struct StarImportDefinitionNodeRef<'a> {
pub(crate) node: &'a ast::StmtImportFrom,
pub(crate) symbol_id: ScopedSymbolId,
pub(crate) place_id: ScopedPlaceId,
}
#[derive(Copy, Clone, Debug)]
@@ -323,10 +356,10 @@ impl<'db> DefinitionNodeRef<'db> {
is_reexported,
}),
DefinitionNodeRef::ImportStar(star_import) => {
let StarImportDefinitionNodeRef { node, symbol_id } = star_import;
let StarImportDefinitionNodeRef { node, place_id } = star_import;
DefinitionKind::StarImport(StarImportDefinitionKind {
node: unsafe { AstNodeRef::new(parsed, node) },
symbol_id,
place_id,
})
}
DefinitionNodeRef::Function(function) => {
@@ -456,7 +489,7 @@ impl<'db> DefinitionNodeRef<'db> {
// INVARIANT: for an invalid-syntax statement such as `from foo import *, bar, *`,
// we only create a `StarImportDefinitionKind` for the *first* `*` alias in the names list.
Self::ImportStar(StarImportDefinitionNodeRef { node, symbol_id: _ }) => node
Self::ImportStar(StarImportDefinitionNodeRef { node, place_id: _ }) => node
.names
.iter()
.find(|alias| &alias.name == "*")
@@ -517,7 +550,7 @@ pub(crate) enum DefinitionCategory {
}
impl DefinitionCategory {
/// True if this definition establishes a "declared type" for the symbol.
/// True if this definition establishes a "declared type" for the place.
///
/// If so, any assignments reached by this definition are in error if they assign a value of a
/// type not assignable to the declared type.
@@ -530,7 +563,7 @@ impl DefinitionCategory {
)
}
/// True if this definition assigns a value to the symbol.
/// True if this definition assigns a value to the place.
///
/// False only for annotated assignments without a RHS.
pub(crate) fn is_binding(self) -> bool {
@@ -591,8 +624,8 @@ impl DefinitionKind<'_> {
/// Returns the [`TextRange`] of the definition target.
///
/// A definition target would mainly be the node representing the symbol being defined i.e.,
/// [`ast::ExprName`] or [`ast::Identifier`] but could also be other nodes.
/// A definition target would mainly be the node representing the place being defined i.e.,
/// [`ast::ExprName`], [`ast::Identifier`], [`ast::ExprAttribute`] or [`ast::ExprSubscript`] but could also be other nodes.
pub(crate) fn target_range(&self) -> TextRange {
match self {
DefinitionKind::Import(import) => import.alias().range(),
@@ -700,14 +733,15 @@ impl DefinitionKind<'_> {
#[derive(Copy, Clone, Debug, PartialEq, Hash)]
pub(crate) enum TargetKind<'db> {
Sequence(UnpackPosition, Unpack<'db>),
NameOrAttribute,
/// Name, attribute, or subscript.
Single,
}
impl<'db> From<Option<(UnpackPosition, Unpack<'db>)>> for TargetKind<'db> {
fn from(value: Option<(UnpackPosition, Unpack<'db>)>) -> Self {
match value {
Some((unpack_position, unpack)) => TargetKind::Sequence(unpack_position, unpack),
None => TargetKind::NameOrAttribute,
None => TargetKind::Single,
}
}
}
@@ -715,7 +749,7 @@ impl<'db> From<Option<(UnpackPosition, Unpack<'db>)>> for TargetKind<'db> {
#[derive(Clone, Debug)]
pub struct StarImportDefinitionKind {
node: AstNodeRef<ast::StmtImportFrom>,
symbol_id: ScopedSymbolId,
place_id: ScopedPlaceId,
}
impl StarImportDefinitionKind {
@@ -737,8 +771,8 @@ impl StarImportDefinitionKind {
)
}
pub(crate) fn symbol_id(&self) -> ScopedSymbolId {
self.symbol_id
pub(crate) fn place_id(&self) -> ScopedPlaceId {
self.place_id
}
}
@@ -759,13 +793,18 @@ impl MatchPatternDefinitionKind {
}
}
/// Note that the elements of a comprehension can be in different scopes.
/// If the definition target of a comprehension is a name, it is in the comprehension's scope.
/// But if the target is an attribute or subscript, its definition is not in the comprehension's scope;
/// it is in the scope in which the root variable is bound.
/// TODO: currently we don't model this correctly and simply assume that it is in a scope outside the comprehension.
#[derive(Clone, Debug)]
pub struct ComprehensionDefinitionKind<'db> {
pub(super) target_kind: TargetKind<'db>,
pub(super) iterable: AstNodeRef<ast::Expr>,
pub(super) target: AstNodeRef<ast::Expr>,
pub(super) first: bool,
pub(super) is_async: bool,
target_kind: TargetKind<'db>,
iterable: AstNodeRef<ast::Expr>,
target: AstNodeRef<ast::Expr>,
first: bool,
is_async: bool,
}
impl<'db> ComprehensionDefinitionKind<'db> {
@@ -840,18 +879,6 @@ pub struct AssignmentDefinitionKind<'db> {
}
impl<'db> AssignmentDefinitionKind<'db> {
pub(crate) fn new(
target_kind: TargetKind<'db>,
value: AstNodeRef<ast::Expr>,
target: AstNodeRef<ast::Expr>,
) -> Self {
Self {
target_kind,
value,
target,
}
}
pub(crate) fn target_kind(&self) -> TargetKind<'db> {
self.target_kind
}
@@ -873,18 +900,6 @@ pub struct AnnotatedAssignmentDefinitionKind {
}
impl AnnotatedAssignmentDefinitionKind {
pub(crate) fn new(
annotation: AstNodeRef<ast::Expr>,
value: Option<AstNodeRef<ast::Expr>>,
target: AstNodeRef<ast::Expr>,
) -> Self {
Self {
annotation,
value,
target,
}
}
pub(crate) fn value(&self) -> Option<&ast::Expr> {
self.value.as_deref()
}
@@ -907,20 +922,6 @@ pub struct WithItemDefinitionKind<'db> {
}
impl<'db> WithItemDefinitionKind<'db> {
pub(crate) fn new(
target_kind: TargetKind<'db>,
context_expr: AstNodeRef<ast::Expr>,
target: AstNodeRef<ast::Expr>,
is_async: bool,
) -> Self {
Self {
target_kind,
context_expr,
target,
is_async,
}
}
pub(crate) fn context_expr(&self) -> &ast::Expr {
self.context_expr.node()
}
@@ -947,20 +948,6 @@ pub struct ForStmtDefinitionKind<'db> {
}
impl<'db> ForStmtDefinitionKind<'db> {
pub(crate) fn new(
target_kind: TargetKind<'db>,
iterable: AstNodeRef<ast::Expr>,
target: AstNodeRef<ast::Expr>,
is_async: bool,
) -> Self {
Self {
target_kind,
iterable,
target,
is_async,
}
}
pub(crate) fn iterable(&self) -> &ast::Expr {
self.iterable.node()
}
@@ -1031,6 +1018,18 @@ impl From<&ast::ExprName> for DefinitionNodeKey {
}
}
impl From<&ast::ExprAttribute> for DefinitionNodeKey {
fn from(node: &ast::ExprAttribute) -> Self {
Self(NodeKey::from_node(node))
}
}
impl From<&ast::ExprSubscript> for DefinitionNodeKey {
fn from(node: &ast::ExprSubscript) -> Self {
Self(NodeKey::from_node(node))
}
}
impl From<&ast::ExprNamed> for DefinitionNodeKey {
fn from(node: &ast::ExprNamed) -> Self {
Self(NodeKey::from_node(node))

View File

@@ -1,6 +1,6 @@
use crate::ast_node_ref::AstNodeRef;
use crate::db::Db;
use crate::semantic_index::symbol::{FileScopeId, ScopeId};
use crate::semantic_index::place::{FileScopeId, ScopeId};
use ruff_db::files::File;
use ruff_python_ast as ast;
use salsa;

View File

@@ -1,7 +1,7 @@
//! # Narrowing constraints
//!
//! When building a semantic index for a file, we associate each binding with a _narrowing
//! constraint_, which constrains the type of the binding's symbol. Note that a binding can be
//! constraint_, which constrains the type of the binding's place. Note that a binding can be
//! associated with a different narrowing constraint at different points in a file. See the
//! [`use_def`][crate::semantic_index::use_def] module for more details.
//!
@@ -34,7 +34,7 @@ use crate::semantic_index::predicate::ScopedPredicateId;
/// A narrowing constraint associated with a live binding.
///
/// A constraint is a list of [`Predicate`]s that each constrain the type of the binding's symbol.
/// A constraint is a list of [`Predicate`]s that each constrain the type of the binding's place.
///
/// [`Predicate`]: crate::semantic_index::predicate::Predicate
pub(crate) type ScopedNarrowingConstraint = List<ScopedNarrowingConstraintPredicate>;
@@ -46,7 +46,7 @@ pub(crate) enum ConstraintKey {
}
/// One of the [`Predicate`]s in a narrowing constraint, which constraints the type of the
/// binding's symbol.
/// binding's place.
///
/// Note that those [`Predicate`]s are stored in [their own per-scope
/// arena][crate::semantic_index::predicate::Predicates], so internally we use a

View File

@@ -0,0 +1,942 @@
use std::convert::Infallible;
use std::hash::{Hash, Hasher};
use std::ops::Range;
use bitflags::bitflags;
use hashbrown::hash_map::RawEntryMut;
use ruff_db::files::File;
use ruff_db::parsed::ParsedModule;
use ruff_index::{IndexVec, newtype_index};
use ruff_python_ast as ast;
use ruff_python_ast::name::Name;
use rustc_hash::FxHasher;
use smallvec::{SmallVec, smallvec};
use crate::Db;
use crate::ast_node_ref::AstNodeRef;
use crate::node_key::NodeKey;
use crate::semantic_index::visibility_constraints::ScopedVisibilityConstraintId;
use crate::semantic_index::{PlaceSet, SemanticIndex, semantic_index};
#[derive(Debug, Clone, PartialEq, Eq, Hash, salsa::Update)]
pub(crate) enum PlaceExprSubSegment {
/// A member access, e.g. `.y` in `x.y`
Member(ast::name::Name),
/// An integer-based index access, e.g. `[1]` in `x[1]`
IntSubscript(ast::Int),
/// A string-based index access, e.g. `["foo"]` in `x["foo"]`
StringSubscript(String),
}
impl PlaceExprSubSegment {
pub(crate) fn as_member(&self) -> Option<&ast::name::Name> {
match self {
PlaceExprSubSegment::Member(name) => Some(name),
_ => None,
}
}
}
/// An expression that can be the target of a `Definition`.
/// If you want to perform a comparison based on the equality of segments (without including
/// flags), use [`PlaceSegments`].
#[derive(Eq, PartialEq, Debug)]
pub struct PlaceExpr {
root_name: Name,
sub_segments: SmallVec<[PlaceExprSubSegment; 1]>,
flags: PlaceFlags,
}
impl std::fmt::Display for PlaceExpr {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "{}", self.root_name)?;
for segment in &self.sub_segments {
match segment {
PlaceExprSubSegment::Member(name) => write!(f, ".{name}")?,
PlaceExprSubSegment::IntSubscript(int) => write!(f, "[{int}]")?,
PlaceExprSubSegment::StringSubscript(string) => write!(f, "[\"{string}\"]")?,
}
}
Ok(())
}
}
impl TryFrom<&ast::name::Name> for PlaceExpr {
type Error = Infallible;
fn try_from(name: &ast::name::Name) -> Result<Self, Infallible> {
Ok(PlaceExpr::name(name.clone()))
}
}
impl TryFrom<ast::name::Name> for PlaceExpr {
type Error = Infallible;
fn try_from(name: ast::name::Name) -> Result<Self, Infallible> {
Ok(PlaceExpr::name(name))
}
}
impl TryFrom<&ast::ExprAttribute> for PlaceExpr {
type Error = ();
fn try_from(attr: &ast::ExprAttribute) -> Result<Self, ()> {
let mut place = PlaceExpr::try_from(&*attr.value)?;
place
.sub_segments
.push(PlaceExprSubSegment::Member(attr.attr.id.clone()));
Ok(place)
}
}
impl TryFrom<ast::ExprAttribute> for PlaceExpr {
type Error = ();
fn try_from(attr: ast::ExprAttribute) -> Result<Self, ()> {
let mut place = PlaceExpr::try_from(&*attr.value)?;
place
.sub_segments
.push(PlaceExprSubSegment::Member(attr.attr.id));
Ok(place)
}
}
impl TryFrom<&ast::ExprSubscript> for PlaceExpr {
type Error = ();
fn try_from(subscript: &ast::ExprSubscript) -> Result<Self, ()> {
let mut place = PlaceExpr::try_from(&*subscript.value)?;
match &*subscript.slice {
ast::Expr::NumberLiteral(ast::ExprNumberLiteral {
value: ast::Number::Int(index),
..
}) => {
place
.sub_segments
.push(PlaceExprSubSegment::IntSubscript(index.clone()));
}
ast::Expr::StringLiteral(string) => {
place
.sub_segments
.push(PlaceExprSubSegment::StringSubscript(
string.value.to_string(),
));
}
_ => {
return Err(());
}
}
Ok(place)
}
}
impl TryFrom<ast::ExprSubscript> for PlaceExpr {
type Error = ();
fn try_from(subscript: ast::ExprSubscript) -> Result<Self, ()> {
PlaceExpr::try_from(&subscript)
}
}
impl TryFrom<&ast::Expr> for PlaceExpr {
type Error = ();
fn try_from(expr: &ast::Expr) -> Result<Self, ()> {
match expr {
ast::Expr::Name(name) => Ok(PlaceExpr::name(name.id.clone())),
ast::Expr::Attribute(attr) => PlaceExpr::try_from(attr),
ast::Expr::Subscript(subscript) => PlaceExpr::try_from(subscript),
_ => Err(()),
}
}
}
impl PlaceExpr {
pub(super) fn name(name: Name) -> Self {
Self {
root_name: name,
sub_segments: smallvec![],
flags: PlaceFlags::empty(),
}
}
fn insert_flags(&mut self, flags: PlaceFlags) {
self.flags.insert(flags);
}
pub(super) fn mark_instance_attribute(&mut self) {
self.flags.insert(PlaceFlags::IS_INSTANCE_ATTRIBUTE);
}
pub(crate) fn root_name(&self) -> &Name {
&self.root_name
}
pub(crate) fn sub_segments(&self) -> &[PlaceExprSubSegment] {
&self.sub_segments
}
pub(crate) fn as_name(&self) -> Option<&Name> {
if self.is_name() {
Some(&self.root_name)
} else {
None
}
}
/// Assumes that the place expression is a name.
#[track_caller]
pub(crate) fn expect_name(&self) -> &Name {
debug_assert_eq!(self.sub_segments.len(), 0);
&self.root_name
}
/// Does the place expression have the form `self.{name}` (`self` is the first parameter of the method)?
pub(super) fn is_instance_attribute_named(&self, name: &str) -> bool {
self.is_instance_attribute()
&& self.sub_segments.len() == 1
&& self.sub_segments[0].as_member().unwrap().as_str() == name
}
/// Is the place an instance attribute?
pub fn is_instance_attribute(&self) -> bool {
self.flags.contains(PlaceFlags::IS_INSTANCE_ATTRIBUTE)
}
/// Is the place used in its containing scope?
pub fn is_used(&self) -> bool {
self.flags.contains(PlaceFlags::IS_USED)
}
/// Is the place defined in its containing scope?
pub fn is_bound(&self) -> bool {
self.flags.contains(PlaceFlags::IS_BOUND)
}
/// Is the place declared in its containing scope?
pub fn is_declared(&self) -> bool {
self.flags.contains(PlaceFlags::IS_DECLARED)
}
/// Is the place just a name?
pub fn is_name(&self) -> bool {
self.sub_segments.is_empty()
}
pub fn is_name_and(&self, f: impl FnOnce(&str) -> bool) -> bool {
self.is_name() && f(&self.root_name)
}
/// Does the place expression have the form `<object>.member`?
pub fn is_member(&self) -> bool {
self.sub_segments
.last()
.is_some_and(|last| last.as_member().is_some())
}
pub(crate) fn segments(&self) -> PlaceSegments {
PlaceSegments {
root_name: Some(&self.root_name),
sub_segments: &self.sub_segments,
}
}
// TODO: Ideally this would iterate PlaceSegments instead of RootExprs, both to reduce
// allocation and to avoid having both flagged and non-flagged versions of PlaceExprs.
fn root_exprs(&self) -> RootExprs<'_> {
RootExprs {
expr: self,
len: self.sub_segments.len(),
}
}
}
struct RootExprs<'e> {
expr: &'e PlaceExpr,
len: usize,
}
impl Iterator for RootExprs<'_> {
type Item = PlaceExpr;
fn next(&mut self) -> Option<Self::Item> {
if self.len == 0 {
return None;
}
self.len -= 1;
Some(PlaceExpr {
root_name: self.expr.root_name.clone(),
sub_segments: self.expr.sub_segments[..self.len].iter().cloned().collect(),
flags: PlaceFlags::empty(),
})
}
}
bitflags! {
/// Flags that can be queried to obtain information about a place in a given scope.
///
/// See the doc-comment at the top of [`super::use_def`] for explanations of what it
/// means for a place to be *bound* as opposed to *declared*.
#[derive(Copy, Clone, Debug, Eq, PartialEq)]
struct PlaceFlags: u8 {
const IS_USED = 1 << 0;
const IS_BOUND = 1 << 1;
const IS_DECLARED = 1 << 2;
/// TODO: This flag is not yet set by anything
const MARKED_GLOBAL = 1 << 3;
/// TODO: This flag is not yet set by anything
const MARKED_NONLOCAL = 1 << 4;
const IS_INSTANCE_ATTRIBUTE = 1 << 5;
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum PlaceSegment<'a> {
/// A first segment of a place expression (root name), e.g. `x` in `x.y.z[0]`.
Name(&'a ast::name::Name),
Member(&'a ast::name::Name),
IntSubscript(&'a ast::Int),
StringSubscript(&'a str),
}
#[derive(Debug, PartialEq, Eq)]
pub struct PlaceSegments<'a> {
root_name: Option<&'a ast::name::Name>,
sub_segments: &'a [PlaceExprSubSegment],
}
impl<'a> Iterator for PlaceSegments<'a> {
type Item = PlaceSegment<'a>;
fn next(&mut self) -> Option<Self::Item> {
if let Some(name) = self.root_name.take() {
return Some(PlaceSegment::Name(name));
}
if self.sub_segments.is_empty() {
return None;
}
let segment = &self.sub_segments[0];
self.sub_segments = &self.sub_segments[1..];
Some(match segment {
PlaceExprSubSegment::Member(name) => PlaceSegment::Member(name),
PlaceExprSubSegment::IntSubscript(int) => PlaceSegment::IntSubscript(int),
PlaceExprSubSegment::StringSubscript(string) => PlaceSegment::StringSubscript(string),
})
}
}
/// ID that uniquely identifies a place in a file.
#[derive(Copy, Clone, Debug, Eq, PartialEq, Hash)]
pub struct FilePlaceId {
scope: FileScopeId,
scoped_place_id: ScopedPlaceId,
}
impl FilePlaceId {
pub fn scope(self) -> FileScopeId {
self.scope
}
pub(crate) fn scoped_place_id(self) -> ScopedPlaceId {
self.scoped_place_id
}
}
impl From<FilePlaceId> for ScopedPlaceId {
fn from(val: FilePlaceId) -> Self {
val.scoped_place_id()
}
}
/// ID that uniquely identifies a place inside a [`Scope`].
#[newtype_index]
#[derive(salsa::Update)]
pub struct ScopedPlaceId;
/// A cross-module identifier of a scope that can be used as a salsa query parameter.
#[salsa::tracked(debug)]
pub struct ScopeId<'db> {
pub file: File,
pub file_scope_id: FileScopeId,
count: countme::Count<ScopeId<'static>>,
}
impl<'db> ScopeId<'db> {
pub(crate) fn is_function_like(self, db: &'db dyn Db) -> bool {
self.node(db).scope_kind().is_function_like()
}
pub(crate) fn is_type_parameter(self, db: &'db dyn Db) -> bool {
self.node(db).scope_kind().is_type_parameter()
}
pub(crate) fn node(self, db: &dyn Db) -> &NodeWithScopeKind {
self.scope(db).node()
}
pub(crate) fn scope(self, db: &dyn Db) -> &Scope {
semantic_index(db, self.file(db)).scope(self.file_scope_id(db))
}
#[cfg(test)]
pub(crate) fn name(self, db: &'db dyn Db) -> &'db str {
match self.node(db) {
NodeWithScopeKind::Module => "<module>",
NodeWithScopeKind::Class(class) | NodeWithScopeKind::ClassTypeParameters(class) => {
class.name.as_str()
}
NodeWithScopeKind::Function(function)
| NodeWithScopeKind::FunctionTypeParameters(function) => function.name.as_str(),
NodeWithScopeKind::TypeAlias(type_alias)
| NodeWithScopeKind::TypeAliasTypeParameters(type_alias) => type_alias
.name
.as_name_expr()
.map(|name| name.id.as_str())
.unwrap_or("<type alias>"),
NodeWithScopeKind::Lambda(_) => "<lambda>",
NodeWithScopeKind::ListComprehension(_) => "<listcomp>",
NodeWithScopeKind::SetComprehension(_) => "<setcomp>",
NodeWithScopeKind::DictComprehension(_) => "<dictcomp>",
NodeWithScopeKind::GeneratorExpression(_) => "<generator>",
}
}
}
/// ID that uniquely identifies a scope inside of a module.
#[newtype_index]
#[derive(salsa::Update)]
pub struct FileScopeId;
impl FileScopeId {
/// Returns the scope id of the module-global scope.
pub fn global() -> Self {
FileScopeId::from_u32(0)
}
pub fn is_global(self) -> bool {
self == FileScopeId::global()
}
pub fn to_scope_id(self, db: &dyn Db, file: File) -> ScopeId<'_> {
let index = semantic_index(db, file);
index.scope_ids_by_scope[self]
}
pub(crate) fn is_generator_function(self, index: &SemanticIndex) -> bool {
index.generator_functions.contains(&self)
}
}
#[derive(Debug, salsa::Update)]
pub struct Scope {
parent: Option<FileScopeId>,
node: NodeWithScopeKind,
descendants: Range<FileScopeId>,
reachability: ScopedVisibilityConstraintId,
}
impl Scope {
pub(super) fn new(
parent: Option<FileScopeId>,
node: NodeWithScopeKind,
descendants: Range<FileScopeId>,
reachability: ScopedVisibilityConstraintId,
) -> Self {
Scope {
parent,
node,
descendants,
reachability,
}
}
pub fn parent(&self) -> Option<FileScopeId> {
self.parent
}
pub fn node(&self) -> &NodeWithScopeKind {
&self.node
}
pub fn kind(&self) -> ScopeKind {
self.node().scope_kind()
}
pub fn descendants(&self) -> Range<FileScopeId> {
self.descendants.clone()
}
pub(super) fn extend_descendants(&mut self, children_end: FileScopeId) {
self.descendants = self.descendants.start..children_end;
}
pub(crate) fn is_eager(&self) -> bool {
self.kind().is_eager()
}
pub(crate) fn reachability(&self) -> ScopedVisibilityConstraintId {
self.reachability
}
}
#[derive(Copy, Clone, Debug, PartialEq, Eq)]
pub enum ScopeKind {
Module,
Annotation,
Class,
Function,
Lambda,
Comprehension,
TypeAlias,
}
impl ScopeKind {
pub(crate) fn is_eager(self) -> bool {
match self {
ScopeKind::Module | ScopeKind::Class | ScopeKind::Comprehension => true,
ScopeKind::Annotation
| ScopeKind::Function
| ScopeKind::Lambda
| ScopeKind::TypeAlias => false,
}
}
pub(crate) fn is_function_like(self) -> bool {
// Type parameter scopes behave like function scopes in terms of name resolution; CPython
// place table also uses the term "function-like" for these scopes.
matches!(
self,
ScopeKind::Annotation
| ScopeKind::Function
| ScopeKind::Lambda
| ScopeKind::TypeAlias
| ScopeKind::Comprehension
)
}
pub(crate) fn is_class(self) -> bool {
matches!(self, ScopeKind::Class)
}
pub(crate) fn is_type_parameter(self) -> bool {
matches!(self, ScopeKind::Annotation | ScopeKind::TypeAlias)
}
}
/// [`PlaceExpr`] table for a specific [`Scope`].
#[derive(Default, salsa::Update)]
pub struct PlaceTable {
/// The place expressions in this scope.
places: IndexVec<ScopedPlaceId, PlaceExpr>,
/// The set of places.
place_set: PlaceSet,
}
impl PlaceTable {
fn shrink_to_fit(&mut self) {
self.places.shrink_to_fit();
}
pub(crate) fn place_expr(&self, place_id: impl Into<ScopedPlaceId>) -> &PlaceExpr {
&self.places[place_id.into()]
}
/// Iterate over the "root" expressions of the place (e.g. `x.y.z`, `x.y`, `x` for `x.y.z[0]`).
pub(crate) fn root_place_exprs(
&self,
place_expr: &PlaceExpr,
) -> impl Iterator<Item = &PlaceExpr> {
place_expr
.root_exprs()
.filter_map(|place_expr| self.place_by_expr(&place_expr))
}
#[expect(unused)]
pub(crate) fn place_ids(&self) -> impl Iterator<Item = ScopedPlaceId> {
self.places.indices()
}
pub fn places(&self) -> impl Iterator<Item = &PlaceExpr> {
self.places.iter()
}
pub fn symbols(&self) -> impl Iterator<Item = &PlaceExpr> {
self.places().filter(|place_expr| place_expr.is_name())
}
pub fn instance_attributes(&self) -> impl Iterator<Item = &PlaceExpr> {
self.places()
.filter(|place_expr| place_expr.is_instance_attribute())
}
/// Returns the place named `name`.
#[allow(unused)] // used in tests
pub(crate) fn place_by_name(&self, name: &str) -> Option<&PlaceExpr> {
let id = self.place_id_by_name(name)?;
Some(self.place_expr(id))
}
/// Returns the flagged place by the unflagged place expression.
///
/// TODO: Ideally this would take a [`PlaceSegments`] instead of [`PlaceExpr`], to avoid the
/// awkward distinction between "flagged" (canonical) and unflagged [`PlaceExpr`]; in that
/// world, we would only create [`PlaceExpr`] in semantic indexing; in type inference we'd
/// create [`PlaceSegments`] if we need to look up a [`PlaceExpr`]. The [`PlaceTable`] would
/// need to gain the ability to hash and look up by a [`PlaceSegments`].
pub(crate) fn place_by_expr(&self, place_expr: &PlaceExpr) -> Option<&PlaceExpr> {
let id = self.place_id_by_expr(place_expr)?;
Some(self.place_expr(id))
}
/// Returns the [`ScopedPlaceId`] of the place named `name`.
pub(crate) fn place_id_by_name(&self, name: &str) -> Option<ScopedPlaceId> {
let (id, ()) = self
.place_set
.raw_entry()
.from_hash(Self::hash_name(name), |id| {
self.place_expr(*id).as_name().map(Name::as_str) == Some(name)
})?;
Some(*id)
}
/// Returns the [`ScopedPlaceId`] of the place expression.
pub(crate) fn place_id_by_expr(&self, place_expr: &PlaceExpr) -> Option<ScopedPlaceId> {
let (id, ()) = self
.place_set
.raw_entry()
.from_hash(Self::hash_place_expr(place_expr), |id| {
self.place_expr(*id).segments() == place_expr.segments()
})?;
Some(*id)
}
pub(crate) fn place_id_by_instance_attribute_name(&self, name: &str) -> Option<ScopedPlaceId> {
self.places
.indices()
.find(|id| self.places[*id].is_instance_attribute_named(name))
}
fn hash_name(name: &str) -> u64 {
let mut hasher = FxHasher::default();
name.hash(&mut hasher);
hasher.finish()
}
fn hash_place_expr(place_expr: &PlaceExpr) -> u64 {
let mut hasher = FxHasher::default();
place_expr.root_name().as_str().hash(&mut hasher);
for segment in &place_expr.sub_segments {
match segment {
PlaceExprSubSegment::Member(name) => name.hash(&mut hasher),
PlaceExprSubSegment::IntSubscript(int) => int.hash(&mut hasher),
PlaceExprSubSegment::StringSubscript(string) => string.hash(&mut hasher),
}
}
hasher.finish()
}
}
impl PartialEq for PlaceTable {
fn eq(&self, other: &Self) -> bool {
// We don't need to compare the place_set because the place is already captured in `PlaceExpr`.
self.places == other.places
}
}
impl Eq for PlaceTable {}
impl std::fmt::Debug for PlaceTable {
/// Exclude the `place_set` field from the debug output.
/// It's very noisy and not useful for debugging.
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_tuple("PlaceTable")
.field(&self.places)
.finish_non_exhaustive()
}
}
#[derive(Debug, Default)]
pub(super) struct PlaceTableBuilder {
table: PlaceTable,
associated_place_ids: IndexVec<ScopedPlaceId, Vec<ScopedPlaceId>>,
}
impl PlaceTableBuilder {
pub(super) fn add_symbol(&mut self, name: Name) -> (ScopedPlaceId, bool) {
let hash = PlaceTable::hash_name(&name);
let entry = self
.table
.place_set
.raw_entry_mut()
.from_hash(hash, |id| self.table.places[*id].as_name() == Some(&name));
match entry {
RawEntryMut::Occupied(entry) => (*entry.key(), false),
RawEntryMut::Vacant(entry) => {
let symbol = PlaceExpr::name(name);
let id = self.table.places.push(symbol);
entry.insert_with_hasher(hash, id, (), |id| {
PlaceTable::hash_place_expr(&self.table.places[*id])
});
let new_id = self.associated_place_ids.push(vec![]);
debug_assert_eq!(new_id, id);
(id, true)
}
}
}
pub(super) fn add_place(&mut self, place_expr: PlaceExpr) -> (ScopedPlaceId, bool) {
let hash = PlaceTable::hash_place_expr(&place_expr);
let entry = self.table.place_set.raw_entry_mut().from_hash(hash, |id| {
self.table.places[*id].segments() == place_expr.segments()
});
match entry {
RawEntryMut::Occupied(entry) => (*entry.key(), false),
RawEntryMut::Vacant(entry) => {
let id = self.table.places.push(place_expr);
entry.insert_with_hasher(hash, id, (), |id| {
PlaceTable::hash_place_expr(&self.table.places[*id])
});
let new_id = self.associated_place_ids.push(vec![]);
debug_assert_eq!(new_id, id);
for root in self.table.places[id].root_exprs() {
if let Some(root_id) = self.table.place_id_by_expr(&root) {
self.associated_place_ids[root_id].push(id);
}
}
(id, true)
}
}
}
pub(super) fn mark_place_bound(&mut self, id: ScopedPlaceId) {
self.table.places[id].insert_flags(PlaceFlags::IS_BOUND);
}
pub(super) fn mark_place_declared(&mut self, id: ScopedPlaceId) {
self.table.places[id].insert_flags(PlaceFlags::IS_DECLARED);
}
pub(super) fn mark_place_used(&mut self, id: ScopedPlaceId) {
self.table.places[id].insert_flags(PlaceFlags::IS_USED);
}
pub(super) fn places(&self) -> impl Iterator<Item = &PlaceExpr> {
self.table.places()
}
pub(super) fn place_id_by_expr(&self, place_expr: &PlaceExpr) -> Option<ScopedPlaceId> {
self.table.place_id_by_expr(place_expr)
}
pub(super) fn place_expr(&self, place_id: impl Into<ScopedPlaceId>) -> &PlaceExpr {
self.table.place_expr(place_id)
}
/// Returns the place IDs associated with the place (e.g. `x.y`, `x.y.z`, `x.y.z[0]` for `x`).
pub(super) fn associated_place_ids(
&self,
place: ScopedPlaceId,
) -> impl Iterator<Item = ScopedPlaceId> {
self.associated_place_ids[place].iter().copied()
}
pub(super) fn finish(mut self) -> PlaceTable {
self.table.shrink_to_fit();
self.table
}
}
/// Reference to a node that introduces a new scope.
#[derive(Copy, Clone, Debug)]
pub(crate) enum NodeWithScopeRef<'a> {
Module,
Class(&'a ast::StmtClassDef),
Function(&'a ast::StmtFunctionDef),
Lambda(&'a ast::ExprLambda),
FunctionTypeParameters(&'a ast::StmtFunctionDef),
ClassTypeParameters(&'a ast::StmtClassDef),
TypeAlias(&'a ast::StmtTypeAlias),
TypeAliasTypeParameters(&'a ast::StmtTypeAlias),
ListComprehension(&'a ast::ExprListComp),
SetComprehension(&'a ast::ExprSetComp),
DictComprehension(&'a ast::ExprDictComp),
GeneratorExpression(&'a ast::ExprGenerator),
}
impl NodeWithScopeRef<'_> {
/// Converts the unowned reference to an owned [`NodeWithScopeKind`].
///
/// # Safety
/// The node wrapped by `self` must be a child of `module`.
#[expect(unsafe_code)]
pub(super) unsafe fn to_kind(self, module: ParsedModule) -> NodeWithScopeKind {
unsafe {
match self {
NodeWithScopeRef::Module => NodeWithScopeKind::Module,
NodeWithScopeRef::Class(class) => {
NodeWithScopeKind::Class(AstNodeRef::new(module, class))
}
NodeWithScopeRef::Function(function) => {
NodeWithScopeKind::Function(AstNodeRef::new(module, function))
}
NodeWithScopeRef::TypeAlias(type_alias) => {
NodeWithScopeKind::TypeAlias(AstNodeRef::new(module, type_alias))
}
NodeWithScopeRef::TypeAliasTypeParameters(type_alias) => {
NodeWithScopeKind::TypeAliasTypeParameters(AstNodeRef::new(module, type_alias))
}
NodeWithScopeRef::Lambda(lambda) => {
NodeWithScopeKind::Lambda(AstNodeRef::new(module, lambda))
}
NodeWithScopeRef::FunctionTypeParameters(function) => {
NodeWithScopeKind::FunctionTypeParameters(AstNodeRef::new(module, function))
}
NodeWithScopeRef::ClassTypeParameters(class) => {
NodeWithScopeKind::ClassTypeParameters(AstNodeRef::new(module, class))
}
NodeWithScopeRef::ListComprehension(comprehension) => {
NodeWithScopeKind::ListComprehension(AstNodeRef::new(module, comprehension))
}
NodeWithScopeRef::SetComprehension(comprehension) => {
NodeWithScopeKind::SetComprehension(AstNodeRef::new(module, comprehension))
}
NodeWithScopeRef::DictComprehension(comprehension) => {
NodeWithScopeKind::DictComprehension(AstNodeRef::new(module, comprehension))
}
NodeWithScopeRef::GeneratorExpression(generator) => {
NodeWithScopeKind::GeneratorExpression(AstNodeRef::new(module, generator))
}
}
}
}
pub(crate) fn node_key(self) -> NodeWithScopeKey {
match self {
NodeWithScopeRef::Module => NodeWithScopeKey::Module,
NodeWithScopeRef::Class(class) => NodeWithScopeKey::Class(NodeKey::from_node(class)),
NodeWithScopeRef::Function(function) => {
NodeWithScopeKey::Function(NodeKey::from_node(function))
}
NodeWithScopeRef::Lambda(lambda) => {
NodeWithScopeKey::Lambda(NodeKey::from_node(lambda))
}
NodeWithScopeRef::FunctionTypeParameters(function) => {
NodeWithScopeKey::FunctionTypeParameters(NodeKey::from_node(function))
}
NodeWithScopeRef::ClassTypeParameters(class) => {
NodeWithScopeKey::ClassTypeParameters(NodeKey::from_node(class))
}
NodeWithScopeRef::TypeAlias(type_alias) => {
NodeWithScopeKey::TypeAlias(NodeKey::from_node(type_alias))
}
NodeWithScopeRef::TypeAliasTypeParameters(type_alias) => {
NodeWithScopeKey::TypeAliasTypeParameters(NodeKey::from_node(type_alias))
}
NodeWithScopeRef::ListComprehension(comprehension) => {
NodeWithScopeKey::ListComprehension(NodeKey::from_node(comprehension))
}
NodeWithScopeRef::SetComprehension(comprehension) => {
NodeWithScopeKey::SetComprehension(NodeKey::from_node(comprehension))
}
NodeWithScopeRef::DictComprehension(comprehension) => {
NodeWithScopeKey::DictComprehension(NodeKey::from_node(comprehension))
}
NodeWithScopeRef::GeneratorExpression(generator) => {
NodeWithScopeKey::GeneratorExpression(NodeKey::from_node(generator))
}
}
}
}
/// Node that introduces a new scope.
#[derive(Clone, Debug, salsa::Update)]
pub enum NodeWithScopeKind {
Module,
Class(AstNodeRef<ast::StmtClassDef>),
ClassTypeParameters(AstNodeRef<ast::StmtClassDef>),
Function(AstNodeRef<ast::StmtFunctionDef>),
FunctionTypeParameters(AstNodeRef<ast::StmtFunctionDef>),
TypeAliasTypeParameters(AstNodeRef<ast::StmtTypeAlias>),
TypeAlias(AstNodeRef<ast::StmtTypeAlias>),
Lambda(AstNodeRef<ast::ExprLambda>),
ListComprehension(AstNodeRef<ast::ExprListComp>),
SetComprehension(AstNodeRef<ast::ExprSetComp>),
DictComprehension(AstNodeRef<ast::ExprDictComp>),
GeneratorExpression(AstNodeRef<ast::ExprGenerator>),
}
impl NodeWithScopeKind {
pub(crate) const fn scope_kind(&self) -> ScopeKind {
match self {
Self::Module => ScopeKind::Module,
Self::Class(_) => ScopeKind::Class,
Self::Function(_) => ScopeKind::Function,
Self::Lambda(_) => ScopeKind::Lambda,
Self::FunctionTypeParameters(_)
| Self::ClassTypeParameters(_)
| Self::TypeAliasTypeParameters(_) => ScopeKind::Annotation,
Self::TypeAlias(_) => ScopeKind::TypeAlias,
Self::ListComprehension(_)
| Self::SetComprehension(_)
| Self::DictComprehension(_)
| Self::GeneratorExpression(_) => ScopeKind::Comprehension,
}
}
pub fn expect_class(&self) -> &ast::StmtClassDef {
match self {
Self::Class(class) => class.node(),
_ => panic!("expected class"),
}
}
pub(crate) const fn as_class(&self) -> Option<&ast::StmtClassDef> {
match self {
Self::Class(class) => Some(class.node()),
_ => None,
}
}
pub fn expect_function(&self) -> &ast::StmtFunctionDef {
self.as_function().expect("expected function")
}
pub fn expect_type_alias(&self) -> &ast::StmtTypeAlias {
match self {
Self::TypeAlias(type_alias) => type_alias.node(),
_ => panic!("expected type alias"),
}
}
pub const fn as_function(&self) -> Option<&ast::StmtFunctionDef> {
match self {
Self::Function(function) => Some(function.node()),
_ => None,
}
}
}
#[derive(Copy, Clone, Debug, Eq, PartialEq, Hash)]
pub(crate) enum NodeWithScopeKey {
Module,
Class(NodeKey),
ClassTypeParameters(NodeKey),
Function(NodeKey),
FunctionTypeParameters(NodeKey),
TypeAlias(NodeKey),
TypeAliasTypeParameters(NodeKey),
Lambda(NodeKey),
ListComprehension(NodeKey),
SetComprehension(NodeKey),
DictComprehension(NodeKey),
GeneratorExpression(NodeKey),
}

View File

@@ -14,7 +14,7 @@ use ruff_python_ast::Singleton;
use crate::db::Db;
use crate::semantic_index::expression::Expression;
use crate::semantic_index::global_scope;
use crate::semantic_index::symbol::{FileScopeId, ScopeId, ScopedSymbolId};
use crate::semantic_index::place::{FileScopeId, ScopeId, ScopedPlaceId};
// A scoped identifier for each `Predicate` in a scope.
#[newtype_index]
@@ -144,13 +144,13 @@ pub(crate) struct StarImportPlaceholderPredicate<'db> {
/// Each symbol imported by a `*` import has a separate predicate associated with it:
/// this field identifies which symbol that is.
///
/// Note that a [`ScopedSymbolId`] is only meaningful if you also know the scope
/// Note that a [`ScopedPlaceId`] is only meaningful if you also know the scope
/// it is relative to. For this specific struct, however, there's no need to store a
/// separate field to hold the ID of the scope. `StarImportPredicate`s are only created
/// for valid `*`-import definitions, and valid `*`-import definitions can only ever
/// exist in the global scope; thus, we know that the `symbol_id` here will be relative
/// to the global scope of the importing file.
pub(crate) symbol_id: ScopedSymbolId,
pub(crate) symbol_id: ScopedPlaceId,
pub(crate) referenced_file: File,
}

View File

@@ -1,589 +0,0 @@
use std::hash::{Hash, Hasher};
use std::ops::Range;
use bitflags::bitflags;
use hashbrown::hash_map::RawEntryMut;
use ruff_db::files::File;
use ruff_db::parsed::ParsedModule;
use ruff_index::{IndexVec, newtype_index};
use ruff_python_ast as ast;
use ruff_python_ast::name::Name;
use rustc_hash::FxHasher;
use crate::Db;
use crate::ast_node_ref::AstNodeRef;
use crate::node_key::NodeKey;
use crate::semantic_index::visibility_constraints::ScopedVisibilityConstraintId;
use crate::semantic_index::{SemanticIndex, SymbolMap, semantic_index};
#[derive(Eq, PartialEq, Debug)]
pub struct Symbol {
name: Name,
flags: SymbolFlags,
}
impl Symbol {
fn new(name: Name) -> Self {
Self {
name,
flags: SymbolFlags::empty(),
}
}
fn insert_flags(&mut self, flags: SymbolFlags) {
self.flags.insert(flags);
}
/// The symbol's name.
pub fn name(&self) -> &Name {
&self.name
}
/// Is the symbol used in its containing scope?
pub fn is_used(&self) -> bool {
self.flags.contains(SymbolFlags::IS_USED)
}
/// Is the symbol defined in its containing scope?
pub fn is_bound(&self) -> bool {
self.flags.contains(SymbolFlags::IS_BOUND)
}
/// Is the symbol declared in its containing scope?
pub fn is_declared(&self) -> bool {
self.flags.contains(SymbolFlags::IS_DECLARED)
}
}
bitflags! {
/// Flags that can be queried to obtain information about a symbol in a given scope.
///
/// See the doc-comment at the top of [`super::use_def`] for explanations of what it
/// means for a symbol to be *bound* as opposed to *declared*.
#[derive(Copy, Clone, Debug, Eq, PartialEq)]
struct SymbolFlags: u8 {
const IS_USED = 1 << 0;
const IS_BOUND = 1 << 1;
const IS_DECLARED = 1 << 2;
/// TODO: This flag is not yet set by anything
const MARKED_GLOBAL = 1 << 3;
/// TODO: This flag is not yet set by anything
const MARKED_NONLOCAL = 1 << 4;
}
}
/// ID that uniquely identifies a symbol in a file.
#[derive(Copy, Clone, Debug, Eq, PartialEq, Hash)]
pub struct FileSymbolId {
scope: FileScopeId,
scoped_symbol_id: ScopedSymbolId,
}
impl FileSymbolId {
pub fn scope(self) -> FileScopeId {
self.scope
}
pub(crate) fn scoped_symbol_id(self) -> ScopedSymbolId {
self.scoped_symbol_id
}
}
impl From<FileSymbolId> for ScopedSymbolId {
fn from(val: FileSymbolId) -> Self {
val.scoped_symbol_id()
}
}
/// Symbol ID that uniquely identifies a symbol inside a [`Scope`].
#[newtype_index]
#[derive(salsa::Update)]
pub struct ScopedSymbolId;
/// A cross-module identifier of a scope that can be used as a salsa query parameter.
#[salsa::tracked(debug)]
pub struct ScopeId<'db> {
pub file: File,
pub file_scope_id: FileScopeId,
count: countme::Count<ScopeId<'static>>,
}
impl<'db> ScopeId<'db> {
pub(crate) fn is_function_like(self, db: &'db dyn Db) -> bool {
self.node(db).scope_kind().is_function_like()
}
pub(crate) fn is_type_parameter(self, db: &'db dyn Db) -> bool {
self.node(db).scope_kind().is_type_parameter()
}
pub(crate) fn node(self, db: &dyn Db) -> &NodeWithScopeKind {
self.scope(db).node()
}
pub(crate) fn scope(self, db: &dyn Db) -> &Scope {
semantic_index(db, self.file(db)).scope(self.file_scope_id(db))
}
#[cfg(test)]
pub(crate) fn name(self, db: &'db dyn Db) -> &'db str {
match self.node(db) {
NodeWithScopeKind::Module => "<module>",
NodeWithScopeKind::Class(class) | NodeWithScopeKind::ClassTypeParameters(class) => {
class.name.as_str()
}
NodeWithScopeKind::Function(function)
| NodeWithScopeKind::FunctionTypeParameters(function) => function.name.as_str(),
NodeWithScopeKind::TypeAlias(type_alias)
| NodeWithScopeKind::TypeAliasTypeParameters(type_alias) => type_alias
.name
.as_name_expr()
.map(|name| name.id.as_str())
.unwrap_or("<type alias>"),
NodeWithScopeKind::Lambda(_) => "<lambda>",
NodeWithScopeKind::ListComprehension(_) => "<listcomp>",
NodeWithScopeKind::SetComprehension(_) => "<setcomp>",
NodeWithScopeKind::DictComprehension(_) => "<dictcomp>",
NodeWithScopeKind::GeneratorExpression(_) => "<generator>",
}
}
}
/// ID that uniquely identifies a scope inside of a module.
#[newtype_index]
#[derive(salsa::Update)]
pub struct FileScopeId;
impl FileScopeId {
/// Returns the scope id of the module-global scope.
pub fn global() -> Self {
FileScopeId::from_u32(0)
}
pub fn is_global(self) -> bool {
self == FileScopeId::global()
}
pub fn to_scope_id(self, db: &dyn Db, file: File) -> ScopeId<'_> {
let index = semantic_index(db, file);
index.scope_ids_by_scope[self]
}
pub(crate) fn is_generator_function(self, index: &SemanticIndex) -> bool {
index.generator_functions.contains(&self)
}
}
#[derive(Debug, salsa::Update)]
pub struct Scope {
parent: Option<FileScopeId>,
node: NodeWithScopeKind,
descendants: Range<FileScopeId>,
reachability: ScopedVisibilityConstraintId,
}
impl Scope {
pub(super) fn new(
parent: Option<FileScopeId>,
node: NodeWithScopeKind,
descendants: Range<FileScopeId>,
reachability: ScopedVisibilityConstraintId,
) -> Self {
Scope {
parent,
node,
descendants,
reachability,
}
}
pub fn parent(&self) -> Option<FileScopeId> {
self.parent
}
pub fn node(&self) -> &NodeWithScopeKind {
&self.node
}
pub fn kind(&self) -> ScopeKind {
self.node().scope_kind()
}
pub fn descendants(&self) -> Range<FileScopeId> {
self.descendants.clone()
}
pub(super) fn extend_descendants(&mut self, children_end: FileScopeId) {
self.descendants = self.descendants.start..children_end;
}
pub(crate) fn is_eager(&self) -> bool {
self.kind().is_eager()
}
pub(crate) fn reachability(&self) -> ScopedVisibilityConstraintId {
self.reachability
}
}
#[derive(Copy, Clone, Debug, PartialEq, Eq)]
pub enum ScopeKind {
Module,
Annotation,
Class,
Function,
Lambda,
Comprehension,
TypeAlias,
}
impl ScopeKind {
pub(crate) fn is_eager(self) -> bool {
match self {
ScopeKind::Module | ScopeKind::Class | ScopeKind::Comprehension => true,
ScopeKind::Annotation
| ScopeKind::Function
| ScopeKind::Lambda
| ScopeKind::TypeAlias => false,
}
}
pub(crate) fn is_function_like(self) -> bool {
// Type parameter scopes behave like function scopes in terms of name resolution; CPython
// symbol table also uses the term "function-like" for these scopes.
matches!(
self,
ScopeKind::Annotation
| ScopeKind::Function
| ScopeKind::Lambda
| ScopeKind::TypeAlias
| ScopeKind::Comprehension
)
}
pub(crate) fn is_class(self) -> bool {
matches!(self, ScopeKind::Class)
}
pub(crate) fn is_type_parameter(self) -> bool {
matches!(self, ScopeKind::Annotation | ScopeKind::TypeAlias)
}
}
/// Symbol table for a specific [`Scope`].
#[derive(Default, salsa::Update)]
pub struct SymbolTable {
/// The symbols in this scope.
symbols: IndexVec<ScopedSymbolId, Symbol>,
/// The symbols indexed by name.
symbols_by_name: SymbolMap,
}
impl SymbolTable {
fn shrink_to_fit(&mut self) {
self.symbols.shrink_to_fit();
}
pub(crate) fn symbol(&self, symbol_id: impl Into<ScopedSymbolId>) -> &Symbol {
&self.symbols[symbol_id.into()]
}
#[expect(unused)]
pub(crate) fn symbol_ids(&self) -> impl Iterator<Item = ScopedSymbolId> {
self.symbols.indices()
}
pub fn symbols(&self) -> impl Iterator<Item = &Symbol> {
self.symbols.iter()
}
/// Returns the symbol named `name`.
pub(crate) fn symbol_by_name(&self, name: &str) -> Option<&Symbol> {
let id = self.symbol_id_by_name(name)?;
Some(self.symbol(id))
}
/// Returns the [`ScopedSymbolId`] of the symbol named `name`.
pub(crate) fn symbol_id_by_name(&self, name: &str) -> Option<ScopedSymbolId> {
let (id, ()) = self
.symbols_by_name
.raw_entry()
.from_hash(Self::hash_name(name), |id| {
self.symbol(*id).name().as_str() == name
})?;
Some(*id)
}
fn hash_name(name: &str) -> u64 {
let mut hasher = FxHasher::default();
name.hash(&mut hasher);
hasher.finish()
}
}
impl PartialEq for SymbolTable {
fn eq(&self, other: &Self) -> bool {
// We don't need to compare the symbols_by_name because the name is already captured in `Symbol`.
self.symbols == other.symbols
}
}
impl Eq for SymbolTable {}
impl std::fmt::Debug for SymbolTable {
/// Exclude the `symbols_by_name` field from the debug output.
/// It's very noisy and not useful for debugging.
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_tuple("SymbolTable")
.field(&self.symbols)
.finish_non_exhaustive()
}
}
#[derive(Debug, Default)]
pub(super) struct SymbolTableBuilder {
table: SymbolTable,
}
impl SymbolTableBuilder {
pub(super) fn add_symbol(&mut self, name: Name) -> (ScopedSymbolId, bool) {
let hash = SymbolTable::hash_name(&name);
let entry = self
.table
.symbols_by_name
.raw_entry_mut()
.from_hash(hash, |id| self.table.symbols[*id].name() == &name);
match entry {
RawEntryMut::Occupied(entry) => (*entry.key(), false),
RawEntryMut::Vacant(entry) => {
let symbol = Symbol::new(name);
let id = self.table.symbols.push(symbol);
entry.insert_with_hasher(hash, id, (), |id| {
SymbolTable::hash_name(self.table.symbols[*id].name().as_str())
});
(id, true)
}
}
}
pub(super) fn mark_symbol_bound(&mut self, id: ScopedSymbolId) {
self.table.symbols[id].insert_flags(SymbolFlags::IS_BOUND);
}
pub(super) fn mark_symbol_declared(&mut self, id: ScopedSymbolId) {
self.table.symbols[id].insert_flags(SymbolFlags::IS_DECLARED);
}
pub(super) fn mark_symbol_used(&mut self, id: ScopedSymbolId) {
self.table.symbols[id].insert_flags(SymbolFlags::IS_USED);
}
pub(super) fn symbols(&self) -> impl Iterator<Item = &Symbol> {
self.table.symbols()
}
pub(super) fn symbol_id_by_name(&self, name: &str) -> Option<ScopedSymbolId> {
self.table.symbol_id_by_name(name)
}
pub(super) fn symbol(&self, symbol_id: impl Into<ScopedSymbolId>) -> &Symbol {
self.table.symbol(symbol_id)
}
pub(super) fn finish(mut self) -> SymbolTable {
self.table.shrink_to_fit();
self.table
}
}
/// Reference to a node that introduces a new scope.
#[derive(Copy, Clone, Debug)]
pub(crate) enum NodeWithScopeRef<'a> {
Module,
Class(&'a ast::StmtClassDef),
Function(&'a ast::StmtFunctionDef),
Lambda(&'a ast::ExprLambda),
FunctionTypeParameters(&'a ast::StmtFunctionDef),
ClassTypeParameters(&'a ast::StmtClassDef),
TypeAlias(&'a ast::StmtTypeAlias),
TypeAliasTypeParameters(&'a ast::StmtTypeAlias),
ListComprehension(&'a ast::ExprListComp),
SetComprehension(&'a ast::ExprSetComp),
DictComprehension(&'a ast::ExprDictComp),
GeneratorExpression(&'a ast::ExprGenerator),
}
impl NodeWithScopeRef<'_> {
/// Converts the unowned reference to an owned [`NodeWithScopeKind`].
///
/// # Safety
/// The node wrapped by `self` must be a child of `module`.
#[expect(unsafe_code)]
pub(super) unsafe fn to_kind(self, module: ParsedModule) -> NodeWithScopeKind {
unsafe {
match self {
NodeWithScopeRef::Module => NodeWithScopeKind::Module,
NodeWithScopeRef::Class(class) => {
NodeWithScopeKind::Class(AstNodeRef::new(module, class))
}
NodeWithScopeRef::Function(function) => {
NodeWithScopeKind::Function(AstNodeRef::new(module, function))
}
NodeWithScopeRef::TypeAlias(type_alias) => {
NodeWithScopeKind::TypeAlias(AstNodeRef::new(module, type_alias))
}
NodeWithScopeRef::TypeAliasTypeParameters(type_alias) => {
NodeWithScopeKind::TypeAliasTypeParameters(AstNodeRef::new(module, type_alias))
}
NodeWithScopeRef::Lambda(lambda) => {
NodeWithScopeKind::Lambda(AstNodeRef::new(module, lambda))
}
NodeWithScopeRef::FunctionTypeParameters(function) => {
NodeWithScopeKind::FunctionTypeParameters(AstNodeRef::new(module, function))
}
NodeWithScopeRef::ClassTypeParameters(class) => {
NodeWithScopeKind::ClassTypeParameters(AstNodeRef::new(module, class))
}
NodeWithScopeRef::ListComprehension(comprehension) => {
NodeWithScopeKind::ListComprehension(AstNodeRef::new(module, comprehension))
}
NodeWithScopeRef::SetComprehension(comprehension) => {
NodeWithScopeKind::SetComprehension(AstNodeRef::new(module, comprehension))
}
NodeWithScopeRef::DictComprehension(comprehension) => {
NodeWithScopeKind::DictComprehension(AstNodeRef::new(module, comprehension))
}
NodeWithScopeRef::GeneratorExpression(generator) => {
NodeWithScopeKind::GeneratorExpression(AstNodeRef::new(module, generator))
}
}
}
}
pub(crate) fn node_key(self) -> NodeWithScopeKey {
match self {
NodeWithScopeRef::Module => NodeWithScopeKey::Module,
NodeWithScopeRef::Class(class) => NodeWithScopeKey::Class(NodeKey::from_node(class)),
NodeWithScopeRef::Function(function) => {
NodeWithScopeKey::Function(NodeKey::from_node(function))
}
NodeWithScopeRef::Lambda(lambda) => {
NodeWithScopeKey::Lambda(NodeKey::from_node(lambda))
}
NodeWithScopeRef::FunctionTypeParameters(function) => {
NodeWithScopeKey::FunctionTypeParameters(NodeKey::from_node(function))
}
NodeWithScopeRef::ClassTypeParameters(class) => {
NodeWithScopeKey::ClassTypeParameters(NodeKey::from_node(class))
}
NodeWithScopeRef::TypeAlias(type_alias) => {
NodeWithScopeKey::TypeAlias(NodeKey::from_node(type_alias))
}
NodeWithScopeRef::TypeAliasTypeParameters(type_alias) => {
NodeWithScopeKey::TypeAliasTypeParameters(NodeKey::from_node(type_alias))
}
NodeWithScopeRef::ListComprehension(comprehension) => {
NodeWithScopeKey::ListComprehension(NodeKey::from_node(comprehension))
}
NodeWithScopeRef::SetComprehension(comprehension) => {
NodeWithScopeKey::SetComprehension(NodeKey::from_node(comprehension))
}
NodeWithScopeRef::DictComprehension(comprehension) => {
NodeWithScopeKey::DictComprehension(NodeKey::from_node(comprehension))
}
NodeWithScopeRef::GeneratorExpression(generator) => {
NodeWithScopeKey::GeneratorExpression(NodeKey::from_node(generator))
}
}
}
}
/// Node that introduces a new scope.
#[derive(Clone, Debug, salsa::Update)]
pub enum NodeWithScopeKind {
Module,
Class(AstNodeRef<ast::StmtClassDef>),
ClassTypeParameters(AstNodeRef<ast::StmtClassDef>),
Function(AstNodeRef<ast::StmtFunctionDef>),
FunctionTypeParameters(AstNodeRef<ast::StmtFunctionDef>),
TypeAliasTypeParameters(AstNodeRef<ast::StmtTypeAlias>),
TypeAlias(AstNodeRef<ast::StmtTypeAlias>),
Lambda(AstNodeRef<ast::ExprLambda>),
ListComprehension(AstNodeRef<ast::ExprListComp>),
SetComprehension(AstNodeRef<ast::ExprSetComp>),
DictComprehension(AstNodeRef<ast::ExprDictComp>),
GeneratorExpression(AstNodeRef<ast::ExprGenerator>),
}
impl NodeWithScopeKind {
pub(crate) const fn scope_kind(&self) -> ScopeKind {
match self {
Self::Module => ScopeKind::Module,
Self::Class(_) => ScopeKind::Class,
Self::Function(_) => ScopeKind::Function,
Self::Lambda(_) => ScopeKind::Lambda,
Self::FunctionTypeParameters(_)
| Self::ClassTypeParameters(_)
| Self::TypeAliasTypeParameters(_) => ScopeKind::Annotation,
Self::TypeAlias(_) => ScopeKind::TypeAlias,
Self::ListComprehension(_)
| Self::SetComprehension(_)
| Self::DictComprehension(_)
| Self::GeneratorExpression(_) => ScopeKind::Comprehension,
}
}
pub fn expect_class(&self) -> &ast::StmtClassDef {
match self {
Self::Class(class) => class.node(),
_ => panic!("expected class"),
}
}
pub(crate) const fn as_class(&self) -> Option<&ast::StmtClassDef> {
match self {
Self::Class(class) => Some(class.node()),
_ => None,
}
}
pub fn expect_function(&self) -> &ast::StmtFunctionDef {
self.as_function().expect("expected function")
}
pub fn expect_type_alias(&self) -> &ast::StmtTypeAlias {
match self {
Self::TypeAlias(type_alias) => type_alias.node(),
_ => panic!("expected type alias"),
}
}
pub const fn as_function(&self) -> Option<&ast::StmtFunctionDef> {
match self {
Self::Function(function) => Some(function.node()),
_ => None,
}
}
}
#[derive(Copy, Clone, Debug, Eq, PartialEq, Hash)]
pub(crate) enum NodeWithScopeKey {
Module,
Class(NodeKey),
ClassTypeParameters(NodeKey),
Function(NodeKey),
FunctionTypeParameters(NodeKey),
TypeAlias(NodeKey),
TypeAliasTypeParameters(NodeKey),
Lambda(NodeKey),
ListComprehension(NodeKey),
SetComprehension(NodeKey),
DictComprehension(NodeKey),
GeneratorExpression(NodeKey),
}

View File

@@ -1,12 +1,20 @@
//! First, some terminology:
//!
//! * A "binding" gives a new value to a variable. This includes many different Python statements
//! * A "place" is semantically a location where a value can be read or written, and syntactically,
//! an expression that can be the target of an assignment, e.g. `x`, `x[0]`, `x.y`. (The term is
//! borrowed from Rust). In Python syntax, an expression like `f().x` is also allowed as the
//! target so it can be called a place, but we do not record declarations / bindings like `f().x:
//! int`, `f().x = ...`. Type checking itself can be done by recording only assignments to names,
//! but in order to perform type narrowing by attribute/subscript assignments, they must also be
//! recorded.
//!
//! * A "binding" gives a new value to a place. This includes many different Python statements
//! (assignment statements of course, but also imports, `def` and `class` statements, `as`
//! clauses in `with` and `except` statements, match patterns, and others) and even one
//! expression kind (named expressions). It notably does not include annotated assignment
//! statements without a right-hand side value; these do not assign any new value to the
//! variable. We consider function parameters to be bindings as well, since (from the perspective
//! of the function's internal scope), a function parameter begins the scope bound to a value.
//! statements without a right-hand side value; these do not assign any new value to the place.
//! We consider function parameters to be bindings as well, since (from the perspective of the
//! function's internal scope), a function parameter begins the scope bound to a value.
//!
//! * A "declaration" establishes an upper bound type for the values that a variable may be
//! permitted to take on. Annotated assignment statements (with or without an RHS value) are
@@ -67,12 +75,12 @@
//! Path(path)`, with the explicit `: Path` annotation, is permitted.
//!
//! The general rule is that whatever declaration(s) can reach a given binding determine the
//! validity of that binding. If there is a path in which the symbol is not declared, that is a
//! validity of that binding. If there is a path in which the place is not declared, that is a
//! declaration of `Unknown`. If multiple declarations can reach a binding, we union them, but by
//! default we also issue a type error, since this implicit union of declared types may hide an
//! error.
//!
//! To support type inference, we build a map from each use of a symbol to the bindings live at
//! To support type inference, we build a map from each use of a place to the bindings live at
//! that use, and the type narrowing constraints that apply to each binding.
//!
//! Let's take this code sample:
@@ -103,12 +111,12 @@
//! bindings and infer a type of `Literal[3, 4]` -- the union of `Literal[3]` and `Literal[4]` --
//! for the second use of `x`.
//!
//! So that's one question our use-def map needs to answer: given a specific use of a symbol, which
//! So that's one question our use-def map needs to answer: given a specific use of a place, which
//! binding(s) can reach that use. In [`AstIds`](crate::semantic_index::ast_ids::AstIds) we number
//! all uses (that means a `Name` node with `Load` context) so we have a `ScopedUseId` to
//! efficiently represent each use.
//! all uses (that means a `Name`/`ExprAttribute`/`ExprSubscript` node with `Load` context)
//! so we have a `ScopedUseId` to efficiently represent each use.
//!
//! We also need to know, for a given definition of a symbol, what type narrowing constraints apply
//! We also need to know, for a given definition of a place, what type narrowing constraints apply
//! to it. For instance, in this code sample:
//!
//! ```python
@@ -122,70 +130,70 @@
//! can rule out the possibility that `x` is `None` here, which should give us the type
//! `Literal[1]` for this use.
//!
//! For declared types, we need to be able to answer the question "given a binding to a symbol,
//! which declarations of that symbol can reach the binding?" This allows us to emit a diagnostic
//! For declared types, we need to be able to answer the question "given a binding to a place,
//! which declarations of that place can reach the binding?" This allows us to emit a diagnostic
//! if the binding is attempting to bind a value of a type that is not assignable to the declared
//! type for that symbol, at that point in control flow.
//! type for that place, at that point in control flow.
//!
//! We also need to know, given a declaration of a symbol, what the inferred type of that symbol is
//! We also need to know, given a declaration of a place, what the inferred type of that place is
//! at that point. This allows us to emit a diagnostic in a case like `x = "foo"; x: int`. The
//! binding `x = "foo"` occurs before the declaration `x: int`, so according to our
//! control-flow-sensitive interpretation of declarations, the assignment is not an error. But the
//! declaration is an error, since it would violate the "inferred type must be assignable to
//! declared type" rule.
//!
//! Another case we need to handle is when a symbol is referenced from a different scope (for
//! example, an import or a nonlocal reference). We call this "public" use of a symbol. For public
//! use of a symbol, we prefer the declared type, if there are any declarations of that symbol; if
//! Another case we need to handle is when a place is referenced from a different scope (for
//! example, an import or a nonlocal reference). We call this "public" use of a place. For public
//! use of a place, we prefer the declared type, if there are any declarations of that place; if
//! not, we fall back to the inferred type. So we also need to know which declarations and bindings
//! can reach the end of the scope.
//!
//! Technically, public use of a symbol could occur from any point in control flow of the scope
//! where the symbol is defined (via inline imports and import cycles, in the case of an import, or
//! via a function call partway through the local scope that ends up using a symbol from the scope
//! Technically, public use of a place could occur from any point in control flow of the scope
//! where the place is defined (via inline imports and import cycles, in the case of an import, or
//! via a function call partway through the local scope that ends up using a place from the scope
//! via a global or nonlocal reference.) But modeling this fully accurately requires whole-program
//! analysis that isn't tractable for an efficient analysis, since it means a given symbol could
//! analysis that isn't tractable for an efficient analysis, since it means a given place could
//! have a different type every place it's referenced throughout the program, depending on the
//! shape of arbitrarily-sized call/import graphs. So we follow other Python type checkers in
//! making the simplifying assumption that usually the scope will finish execution before its
//! symbols are made visible to other scopes; for instance, most imports will import from a
//! places are made visible to other scopes; for instance, most imports will import from a
//! complete module, not a partially-executed module. (We may want to get a little smarter than
//! this in the future for some closures, but for now this is where we start.)
//!
//! The data structure we build to answer these questions is the `UseDefMap`. It has a
//! `bindings_by_use` vector of [`SymbolBindings`] indexed by [`ScopedUseId`], a
//! `declarations_by_binding` vector of [`SymbolDeclarations`] indexed by [`ScopedDefinitionId`], a
//! `bindings_by_declaration` vector of [`SymbolBindings`] indexed by [`ScopedDefinitionId`], and
//! `public_bindings` and `public_definitions` vectors indexed by [`ScopedSymbolId`]. The values in
//! `bindings_by_use` vector of [`Bindings`] indexed by [`ScopedUseId`], a
//! `declarations_by_binding` vector of [`Declarations`] indexed by [`ScopedDefinitionId`], a
//! `bindings_by_declaration` vector of [`Bindings`] indexed by [`ScopedDefinitionId`], and
//! `public_bindings` and `public_definitions` vectors indexed by [`ScopedPlaceId`]. The values in
//! each of these vectors are (in principle) a list of live bindings at that use/definition, or at
//! the end of the scope for that symbol, with a list of the dominating constraints for each
//! the end of the scope for that place, with a list of the dominating constraints for each
//! binding.
//!
//! In order to avoid vectors-of-vectors-of-vectors and all the allocations that would entail, we
//! don't actually store these "list of visible definitions" as a vector of [`Definition`].
//! Instead, [`SymbolBindings`] and [`SymbolDeclarations`] are structs which use bit-sets to track
//! Instead, [`Bindings`] and [`Declarations`] are structs which use bit-sets to track
//! definitions (and constraints, in the case of bindings) in terms of [`ScopedDefinitionId`] and
//! [`ScopedPredicateId`], which are indices into the `all_definitions` and `predicates`
//! indexvecs in the [`UseDefMap`].
//!
//! There is another special kind of possible "definition" for a symbol: there might be a path from
//! the scope entry to a given use in which the symbol is never bound. We model this with a special
//! "unbound" definition (a `None` entry at the start of the `all_definitions` vector). If that
//! sentinel definition is present in the live bindings at a given use, it means that there is a
//! possible path through control flow in which that symbol is unbound. Similarly, if that sentinel
//! is present in the live declarations, it means that the symbol is (possibly) undeclared.
//! There is another special kind of possible "definition" for a place: there might be a path from
//! the scope entry to a given use in which the place is never bound. We model this with a special
//! "unbound/undeclared" definition (a [`DefinitionState::Undefined`] entry at the start of the
//! `all_definitions` vector). If that sentinel definition is present in the live bindings at a
//! given use, it means that there is a possible path through control flow in which that place is
//! unbound. Similarly, if that sentinel is present in the live declarations, it means that the
//! place is (possibly) undeclared.
//!
//! To build a [`UseDefMap`], the [`UseDefMapBuilder`] is notified of each new use, definition, and
//! constraint as they are encountered by the
//! [`SemanticIndexBuilder`](crate::semantic_index::builder::SemanticIndexBuilder) AST visit. For
//! each symbol, the builder tracks the `SymbolState` (`SymbolBindings` and `SymbolDeclarations`)
//! for that symbol. When we hit a use or definition of a symbol, we record the necessary parts of
//! the current state for that symbol that we need for that use or definition. When we reach the
//! end of the scope, it records the state for each symbol as the public definitions of that
//! symbol.
//! each place, the builder tracks the `PlaceState` (`Bindings` and `Declarations`) for that place.
//! When we hit a use or definition of a place, we record the necessary parts of the current state
//! for that place that we need for that use or definition. When we reach the end of the scope, it
//! records the state for each place as the public definitions of that place.
//!
//! Let's walk through the above example. Initially we do not have any record of `x`. When we add
//! the new symbol (before we process the first binding), we create a new undefined `SymbolState`
//! the new place (before we process the first binding), we create a new undefined `PlaceState`
//! which has a single live binding (the "unbound" definition) and a single live declaration (the
//! "undeclared" definition). When we see `x = 1`, we record that as the sole live binding of `x`.
//! The "unbound" binding is no longer visible. Then we see `x = 2`, and we replace `x = 1` as the
@@ -193,11 +201,11 @@
//! of `x` are just the `x = 2` definition.
//!
//! Then we hit the `if` branch. We visit the `test` node (`flag` in this case), since that will
//! happen regardless. Then we take a pre-branch snapshot of the current state for all symbols,
//! happen regardless. Then we take a pre-branch snapshot of the current state for all places,
//! which we'll need later. Then we record `flag` as a possible constraint on the current binding
//! (`x = 2`), and go ahead and visit the `if` body. When we see `x = 3`, it replaces `x = 2`
//! (constrained by `flag`) as the sole live binding of `x`. At the end of the `if` body, we take
//! another snapshot of the current symbol state; we'll call this the post-if-body snapshot.
//! another snapshot of the current place state; we'll call this the post-if-body snapshot.
//!
//! Now we need to visit the `else` clause. The conditions when entering the `else` clause should
//! be the pre-if conditions; if we are entering the `else` clause, we know that the `if` test
@@ -247,7 +255,7 @@
//! `__bool__` method of `test` returns type `bool`, we can see both bindings.
//!
//! Note that we also record visibility constraints for the start of the scope. This is important
//! to determine if a symbol is definitely bound, possibly unbound, or definitely unbound. In the
//! to determine if a place is definitely bound, possibly unbound, or definitely unbound. In the
//! example above, The `y = <unbound>` binding is constrained by `~test`, so `y` would only be
//! definitely-bound if `test` is always truthy.
//!
@@ -259,34 +267,34 @@
use ruff_index::{IndexVec, newtype_index};
use rustc_hash::FxHashMap;
use self::symbol_state::{
EagerSnapshot, LiveBindingsIterator, LiveDeclaration, LiveDeclarationsIterator,
ScopedDefinitionId, SymbolBindings, SymbolDeclarations, SymbolState,
use self::place_state::{
Bindings, Declarations, EagerSnapshot, LiveBindingsIterator, LiveDeclaration,
LiveDeclarationsIterator, PlaceState, ScopedDefinitionId,
};
use crate::node_key::NodeKey;
use crate::semantic_index::EagerSnapshotResult;
use crate::semantic_index::ast_ids::ScopedUseId;
use crate::semantic_index::definition::Definition;
use crate::semantic_index::definition::{Definition, DefinitionState};
use crate::semantic_index::narrowing_constraints::{
ConstraintKey, NarrowingConstraints, NarrowingConstraintsBuilder, NarrowingConstraintsIterator,
};
use crate::semantic_index::place::{FileScopeId, PlaceExpr, ScopeKind, ScopedPlaceId};
use crate::semantic_index::predicate::{
Predicate, Predicates, PredicatesBuilder, ScopedPredicateId, StarImportPlaceholderPredicate,
};
use crate::semantic_index::symbol::{FileScopeId, ScopeKind, ScopedSymbolId};
use crate::semantic_index::visibility_constraints::{
ScopedVisibilityConstraintId, VisibilityConstraints, VisibilityConstraintsBuilder,
};
use crate::types::{IntersectionBuilder, Truthiness, Type, infer_narrowing_constraint};
mod symbol_state;
mod place_state;
/// Applicable definitions and constraints for every use of a name.
#[derive(Debug, PartialEq, Eq, salsa::Update)]
pub(crate) struct UseDefMap<'db> {
/// Array of [`Definition`] in this scope. Only the first entry should be `None`;
/// this represents the implicit "unbound"/"undeclared" definition of every symbol.
all_definitions: IndexVec<ScopedDefinitionId, Option<Definition<'db>>>,
/// Array of [`Definition`] in this scope. Only the first entry should be [`DefinitionState::Undefined`];
/// this represents the implicit "unbound"/"undeclared" definition of every place.
all_definitions: IndexVec<ScopedDefinitionId, DefinitionState<'db>>,
/// Array of predicates in this scope.
predicates: Predicates<'db>,
@@ -297,34 +305,31 @@ pub(crate) struct UseDefMap<'db> {
/// Array of visibility constraints in this scope.
visibility_constraints: VisibilityConstraints,
/// [`SymbolBindings`] reaching a [`ScopedUseId`].
bindings_by_use: IndexVec<ScopedUseId, SymbolBindings>,
/// [`Bindings`] reaching a [`ScopedUseId`].
bindings_by_use: IndexVec<ScopedUseId, Bindings>,
/// Tracks whether or not a given AST node is reachable from the start of the scope.
node_reachability: FxHashMap<NodeKey, ScopedVisibilityConstraintId>,
/// If the definition is a binding (only) -- `x = 1` for example -- then we need
/// [`SymbolDeclarations`] to know whether this binding is permitted by the live declarations.
/// [`Declarations`] to know whether this binding is permitted by the live declarations.
///
/// If the definition is both a declaration and a binding -- `x: int = 1` for example -- then
/// we don't actually need anything here, all we'll need to validate is that our own RHS is a
/// valid assignment to our own annotation.
declarations_by_binding: FxHashMap<Definition<'db>, SymbolDeclarations>,
declarations_by_binding: FxHashMap<Definition<'db>, Declarations>,
/// If the definition is a declaration (only) -- `x: int` for example -- then we need
/// [`SymbolBindings`] to know whether this declaration is consistent with the previously
/// [`Bindings`] to know whether this declaration is consistent with the previously
/// inferred type.
///
/// If the definition is both a declaration and a binding -- `x: int = 1` for example -- then
/// we don't actually need anything here, all we'll need to validate is that our own RHS is a
/// valid assignment to our own annotation.
bindings_by_declaration: FxHashMap<Definition<'db>, SymbolBindings>,
bindings_by_declaration: FxHashMap<Definition<'db>, Bindings>,
/// [`SymbolState`] visible at end of scope for each symbol.
public_symbols: IndexVec<ScopedSymbolId, SymbolState>,
/// [`SymbolState`] for each instance attribute.
instance_attributes: IndexVec<ScopedSymbolId, SymbolState>,
/// [`PlaceState`] visible at end of scope for each place.
public_places: IndexVec<ScopedPlaceId, PlaceState>,
/// Snapshot of bindings in this scope that can be used to resolve a reference in a nested
/// eager scope.
@@ -402,16 +407,9 @@ impl<'db> UseDefMap<'db> {
pub(crate) fn public_bindings(
&self,
symbol: ScopedSymbolId,
place: ScopedPlaceId,
) -> BindingWithConstraintsIterator<'_, 'db> {
self.bindings_iterator(self.public_symbols[symbol].bindings())
}
pub(crate) fn instance_attribute_bindings(
&self,
symbol: ScopedSymbolId,
) -> BindingWithConstraintsIterator<'_, 'db> {
self.bindings_iterator(self.instance_attributes[symbol].bindings())
self.bindings_iterator(self.public_places[place].bindings())
}
pub(crate) fn eager_snapshot(
@@ -422,8 +420,8 @@ impl<'db> UseDefMap<'db> {
Some(EagerSnapshot::Constraint(constraint)) => {
EagerSnapshotResult::FoundConstraint(*constraint)
}
Some(EagerSnapshot::Bindings(symbol_bindings)) => {
EagerSnapshotResult::FoundBindings(self.bindings_iterator(symbol_bindings))
Some(EagerSnapshot::Bindings(bindings)) => {
EagerSnapshotResult::FoundBindings(self.bindings_iterator(bindings))
}
None => EagerSnapshotResult::NotFound,
}
@@ -445,27 +443,27 @@ impl<'db> UseDefMap<'db> {
pub(crate) fn public_declarations<'map>(
&'map self,
symbol: ScopedSymbolId,
place: ScopedPlaceId,
) -> DeclarationsIterator<'map, 'db> {
let declarations = self.public_symbols[symbol].declarations();
let declarations = self.public_places[place].declarations();
self.declarations_iterator(declarations)
}
pub(crate) fn all_public_declarations<'map>(
&'map self,
) -> impl Iterator<Item = (ScopedSymbolId, DeclarationsIterator<'map, 'db>)> + 'map {
(0..self.public_symbols.len())
.map(ScopedSymbolId::from_usize)
.map(|symbol_id| (symbol_id, self.public_declarations(symbol_id)))
) -> impl Iterator<Item = (ScopedPlaceId, DeclarationsIterator<'map, 'db>)> + 'map {
(0..self.public_places.len())
.map(ScopedPlaceId::from_usize)
.map(|place_id| (place_id, self.public_declarations(place_id)))
}
pub(crate) fn all_public_bindings<'map>(
&'map self,
) -> impl Iterator<Item = (ScopedSymbolId, BindingWithConstraintsIterator<'map, 'db>)> + 'map
) -> impl Iterator<Item = (ScopedPlaceId, BindingWithConstraintsIterator<'map, 'db>)> + 'map
{
(0..self.public_symbols.len())
.map(ScopedSymbolId::from_usize)
.map(|symbol_id| (symbol_id, self.public_bindings(symbol_id)))
(0..self.public_places.len())
.map(ScopedPlaceId::from_usize)
.map(|place_id| (place_id, self.public_bindings(place_id)))
}
/// This function is intended to be called only once inside `TypeInferenceBuilder::infer_function_body`.
@@ -487,7 +485,7 @@ impl<'db> UseDefMap<'db> {
fn bindings_iterator<'map>(
&'map self,
bindings: &'map SymbolBindings,
bindings: &'map Bindings,
) -> BindingWithConstraintsIterator<'map, 'db> {
BindingWithConstraintsIterator {
all_definitions: &self.all_definitions,
@@ -500,7 +498,7 @@ impl<'db> UseDefMap<'db> {
fn declarations_iterator<'map>(
&'map self,
declarations: &'map SymbolDeclarations,
declarations: &'map Declarations,
) -> DeclarationsIterator<'map, 'db> {
DeclarationsIterator {
all_definitions: &self.all_definitions,
@@ -511,12 +509,12 @@ impl<'db> UseDefMap<'db> {
}
}
/// Uniquely identifies a snapshot of a symbol state that can be used to resolve a reference in a
/// Uniquely identifies a snapshot of a place state that can be used to resolve a reference in a
/// nested eager scope.
///
/// An eager scope has its entire body executed immediately at the location where it is defined.
/// For any free references in the nested scope, we use the bindings that are visible at the point
/// where the nested scope is defined, instead of using the public type of the symbol.
/// where the nested scope is defined, instead of using the public type of the place.
///
/// There is a unique ID for each distinct [`EagerSnapshotKey`] in the file.
#[newtype_index]
@@ -526,18 +524,18 @@ pub(crate) struct ScopedEagerSnapshotId;
pub(crate) struct EagerSnapshotKey {
/// The enclosing scope containing the bindings
pub(crate) enclosing_scope: FileScopeId,
/// The referenced symbol (in the enclosing scope)
pub(crate) enclosing_symbol: ScopedSymbolId,
/// The referenced place (in the enclosing scope)
pub(crate) enclosing_place: ScopedPlaceId,
/// The nested eager scope containing the reference
pub(crate) nested_scope: FileScopeId,
}
/// A snapshot of symbol states that can be used to resolve a reference in a nested eager scope.
/// A snapshot of place states that can be used to resolve a reference in a nested eager scope.
type EagerSnapshots = IndexVec<ScopedEagerSnapshotId, EagerSnapshot>;
#[derive(Debug)]
pub(crate) struct BindingWithConstraintsIterator<'map, 'db> {
all_definitions: &'map IndexVec<ScopedDefinitionId, Option<Definition<'db>>>,
all_definitions: &'map IndexVec<ScopedDefinitionId, DefinitionState<'db>>,
pub(crate) predicates: &'map Predicates<'db>,
pub(crate) narrowing_constraints: &'map NarrowingConstraints,
pub(crate) visibility_constraints: &'map VisibilityConstraints,
@@ -568,7 +566,7 @@ impl<'map, 'db> Iterator for BindingWithConstraintsIterator<'map, 'db> {
impl std::iter::FusedIterator for BindingWithConstraintsIterator<'_, '_> {}
pub(crate) struct BindingWithConstraints<'map, 'db> {
pub(crate) binding: Option<Definition<'db>>,
pub(crate) binding: DefinitionState<'db>,
pub(crate) narrowing_constraint: ConstraintsIterator<'map, 'db>,
pub(crate) visibility_constraint: ScopedVisibilityConstraintId,
}
@@ -595,10 +593,10 @@ impl<'db> ConstraintsIterator<'_, 'db> {
self,
db: &'db dyn crate::Db,
base_ty: Type<'db>,
symbol: ScopedSymbolId,
place: ScopedPlaceId,
) -> Type<'db> {
let constraint_tys: Vec<_> = self
.filter_map(|constraint| infer_narrowing_constraint(db, constraint, symbol))
.filter_map(|constraint| infer_narrowing_constraint(db, constraint, place))
.collect();
if constraint_tys.is_empty() {
@@ -618,14 +616,14 @@ impl<'db> ConstraintsIterator<'_, 'db> {
#[derive(Clone)]
pub(crate) struct DeclarationsIterator<'map, 'db> {
all_definitions: &'map IndexVec<ScopedDefinitionId, Option<Definition<'db>>>,
all_definitions: &'map IndexVec<ScopedDefinitionId, DefinitionState<'db>>,
pub(crate) predicates: &'map Predicates<'db>,
pub(crate) visibility_constraints: &'map VisibilityConstraints,
inner: LiveDeclarationsIterator<'map>,
}
pub(crate) struct DeclarationWithConstraint<'db> {
pub(crate) declaration: Option<Definition<'db>>,
pub(crate) declaration: DefinitionState<'db>,
pub(crate) visibility_constraint: ScopedVisibilityConstraintId,
}
@@ -652,8 +650,7 @@ impl std::iter::FusedIterator for DeclarationsIterator<'_, '_> {}
/// A snapshot of the definitions and constraints state at a particular point in control flow.
#[derive(Clone, Debug)]
pub(super) struct FlowSnapshot {
symbol_states: IndexVec<ScopedSymbolId, SymbolState>,
instance_attribute_states: IndexVec<ScopedSymbolId, SymbolState>,
place_states: IndexVec<ScopedPlaceId, PlaceState>,
scope_start_visibility: ScopedVisibilityConstraintId,
reachability: ScopedVisibilityConstraintId,
}
@@ -661,7 +658,7 @@ pub(super) struct FlowSnapshot {
#[derive(Debug)]
pub(super) struct UseDefMapBuilder<'db> {
/// Append-only array of [`Definition`].
all_definitions: IndexVec<ScopedDefinitionId, Option<Definition<'db>>>,
all_definitions: IndexVec<ScopedDefinitionId, DefinitionState<'db>>,
/// Builder of predicates.
pub(super) predicates: PredicatesBuilder<'db>,
@@ -673,7 +670,7 @@ pub(super) struct UseDefMapBuilder<'db> {
pub(super) visibility_constraints: VisibilityConstraintsBuilder,
/// A constraint which describes the visibility of the unbound/undeclared state, i.e.
/// whether or not a use of a symbol at the current point in control flow would see
/// whether or not a use of a place at the current point in control flow would see
/// the fake `x = <unbound>` binding at the start of the scope. This is important for
/// cases like the following, where we need to hide the implicit unbound binding in
/// the "else" branch:
@@ -688,7 +685,7 @@ pub(super) struct UseDefMapBuilder<'db> {
pub(super) scope_start_visibility: ScopedVisibilityConstraintId,
/// Live bindings at each so-far-recorded use.
bindings_by_use: IndexVec<ScopedUseId, SymbolBindings>,
bindings_by_use: IndexVec<ScopedUseId, Bindings>,
/// Tracks whether or not the scope start is visible at the current point in control flow.
/// This is subtly different from `scope_start_visibility`, as we apply these constraints
@@ -725,18 +722,15 @@ pub(super) struct UseDefMapBuilder<'db> {
node_reachability: FxHashMap<NodeKey, ScopedVisibilityConstraintId>,
/// Live declarations for each so-far-recorded binding.
declarations_by_binding: FxHashMap<Definition<'db>, SymbolDeclarations>,
declarations_by_binding: FxHashMap<Definition<'db>, Declarations>,
/// Live bindings for each so-far-recorded declaration.
bindings_by_declaration: FxHashMap<Definition<'db>, SymbolBindings>,
bindings_by_declaration: FxHashMap<Definition<'db>, Bindings>,
/// Currently live bindings and declarations for each symbol.
symbol_states: IndexVec<ScopedSymbolId, SymbolState>,
/// Currently live bindings and declarations for each place.
place_states: IndexVec<ScopedPlaceId, PlaceState>,
/// Currently live bindings for each instance attribute.
instance_attribute_states: IndexVec<ScopedSymbolId, SymbolState>,
/// Snapshots of symbol states in this scope that can be used to resolve a reference in a
/// Snapshots of place states in this scope that can be used to resolve a reference in a
/// nested eager scope.
eager_snapshots: EagerSnapshots,
@@ -747,7 +741,7 @@ pub(super) struct UseDefMapBuilder<'db> {
impl<'db> UseDefMapBuilder<'db> {
pub(super) fn new(is_class_scope: bool) -> Self {
Self {
all_definitions: IndexVec::from_iter([None]),
all_definitions: IndexVec::from_iter([DefinitionState::Undefined]),
predicates: PredicatesBuilder::default(),
narrowing_constraints: NarrowingConstraintsBuilder::default(),
visibility_constraints: VisibilityConstraintsBuilder::default(),
@@ -757,9 +751,8 @@ impl<'db> UseDefMapBuilder<'db> {
node_reachability: FxHashMap::default(),
declarations_by_binding: FxHashMap::default(),
bindings_by_declaration: FxHashMap::default(),
symbol_states: IndexVec::new(),
place_states: IndexVec::new(),
eager_snapshots: EagerSnapshots::default(),
instance_attribute_states: IndexVec::new(),
is_class_scope,
}
}
@@ -768,38 +761,29 @@ impl<'db> UseDefMapBuilder<'db> {
self.reachability = ScopedVisibilityConstraintId::ALWAYS_FALSE;
}
pub(super) fn add_symbol(&mut self, symbol: ScopedSymbolId) {
let new_symbol = self
.symbol_states
.push(SymbolState::undefined(self.scope_start_visibility));
debug_assert_eq!(symbol, new_symbol);
pub(super) fn add_place(&mut self, place: ScopedPlaceId) {
let new_place = self
.place_states
.push(PlaceState::undefined(self.scope_start_visibility));
debug_assert_eq!(place, new_place);
}
pub(super) fn add_attribute(&mut self, symbol: ScopedSymbolId) {
let new_symbol = self
.instance_attribute_states
.push(SymbolState::undefined(self.scope_start_visibility));
debug_assert_eq!(symbol, new_symbol);
}
pub(super) fn record_binding(&mut self, symbol: ScopedSymbolId, binding: Definition<'db>) {
let def_id = self.all_definitions.push(Some(binding));
let symbol_state = &mut self.symbol_states[symbol];
self.declarations_by_binding
.insert(binding, symbol_state.declarations().clone());
symbol_state.record_binding(def_id, self.scope_start_visibility, self.is_class_scope);
}
pub(super) fn record_attribute_binding(
pub(super) fn record_binding(
&mut self,
symbol: ScopedSymbolId,
place: ScopedPlaceId,
binding: Definition<'db>,
is_place_name: bool,
) {
let def_id = self.all_definitions.push(Some(binding));
let attribute_state = &mut self.instance_attribute_states[symbol];
let def_id = self.all_definitions.push(DefinitionState::Defined(binding));
let place_state = &mut self.place_states[place];
self.declarations_by_binding
.insert(binding, attribute_state.declarations().clone());
attribute_state.record_binding(def_id, self.scope_start_visibility, self.is_class_scope);
.insert(binding, place_state.declarations().clone());
place_state.record_binding(
def_id,
self.scope_start_visibility,
self.is_class_scope,
is_place_name,
);
}
pub(super) fn add_predicate(&mut self, predicate: Predicate<'db>) -> ScopedPredicateId {
@@ -808,11 +792,7 @@ impl<'db> UseDefMapBuilder<'db> {
pub(super) fn record_narrowing_constraint(&mut self, predicate: ScopedPredicateId) {
let narrowing_constraint = predicate.into();
for state in &mut self.symbol_states {
state
.record_narrowing_constraint(&mut self.narrowing_constraints, narrowing_constraint);
}
for state in &mut self.instance_attribute_states {
for state in &mut self.place_states {
state
.record_narrowing_constraint(&mut self.narrowing_constraints, narrowing_constraint);
}
@@ -822,10 +802,7 @@ impl<'db> UseDefMapBuilder<'db> {
&mut self,
constraint: ScopedVisibilityConstraintId,
) {
for state in &mut self.symbol_states {
state.record_visibility_constraint(&mut self.visibility_constraints, constraint);
}
for state in &mut self.instance_attribute_states {
for state in &mut self.place_states {
state.record_visibility_constraint(&mut self.visibility_constraints, constraint);
}
self.scope_start_visibility = self
@@ -833,13 +810,13 @@ impl<'db> UseDefMapBuilder<'db> {
.add_and_constraint(self.scope_start_visibility, constraint);
}
/// Snapshot the state of a single symbol at the current point in control flow.
/// Snapshot the state of a single place at the current point in control flow.
///
/// This is only used for `*`-import visibility constraints, which are handled differently
/// to most other visibility constraints. See the doc-comment for
/// [`Self::record_and_negate_star_import_visibility_constraint`] for more details.
pub(super) fn single_symbol_snapshot(&self, symbol: ScopedSymbolId) -> SymbolState {
self.symbol_states[symbol].clone()
pub(super) fn single_place_snapshot(&self, place: ScopedPlaceId) -> PlaceState {
self.place_states[place].clone()
}
/// This method exists solely for handling `*`-import visibility constraints.
@@ -863,10 +840,10 @@ impl<'db> UseDefMapBuilder<'db> {
/// Doing things this way is cheaper in and of itself. However, it also allows us to avoid
/// calling [`Self::simplify_visibility_constraints`] after the constraint has been applied to
/// the "if-predicate-true" branch and negated for the "if-predicate-false" branch. Simplifying
/// the visibility constraints is only important for symbols that did not have any new
/// the visibility constraints is only important for places that did not have any new
/// definitions inside either the "if-predicate-true" branch or the "if-predicate-false" branch.
///
/// - We only snapshot the state for a single symbol prior to the definition, rather than doing
/// - We only snapshot the state for a single place prior to the definition, rather than doing
/// expensive calls to [`Self::snapshot`]. Again, this is possible because we know
/// that only a single definition occurs inside the "if-predicate-true" predicate branch.
///
@@ -880,8 +857,8 @@ impl<'db> UseDefMapBuilder<'db> {
pub(super) fn record_and_negate_star_import_visibility_constraint(
&mut self,
star_import: StarImportPlaceholderPredicate<'db>,
symbol: ScopedSymbolId,
pre_definition_state: SymbolState,
symbol: ScopedPlaceId,
pre_definition_state: PlaceState,
) {
let predicate_id = self.add_predicate(star_import.into());
let visibility_id = self.visibility_constraints.add_atom(predicate_id);
@@ -890,22 +867,22 @@ impl<'db> UseDefMapBuilder<'db> {
.add_not_constraint(visibility_id);
let mut post_definition_state =
std::mem::replace(&mut self.symbol_states[symbol], pre_definition_state);
std::mem::replace(&mut self.place_states[symbol], pre_definition_state);
post_definition_state
.record_visibility_constraint(&mut self.visibility_constraints, visibility_id);
self.symbol_states[symbol]
self.place_states[symbol]
.record_visibility_constraint(&mut self.visibility_constraints, negated_visibility_id);
self.symbol_states[symbol].merge(
self.place_states[symbol].merge(
post_definition_state,
&mut self.narrowing_constraints,
&mut self.visibility_constraints,
);
}
/// This method resets the visibility constraints for all symbols to a previous state
/// This method resets the visibility constraints for all places to a previous state
/// *if* there have been no new declarations or bindings since then. Consider the
/// following example:
/// ```py
@@ -924,10 +901,7 @@ impl<'db> UseDefMapBuilder<'db> {
/// constraint for the `x = 0` binding as well, but at the `RESET` point, we can get rid
/// of it, as the `if`-`elif`-`elif` chain doesn't include any new bindings of `x`.
pub(super) fn simplify_visibility_constraints(&mut self, snapshot: FlowSnapshot) {
debug_assert!(self.symbol_states.len() >= snapshot.symbol_states.len());
debug_assert!(
self.instance_attribute_states.len() >= snapshot.instance_attribute_states.len()
);
debug_assert!(self.place_states.len() >= snapshot.place_states.len());
// If there are any control flow paths that have become unreachable between `snapshot` and
// now, then it's not valid to simplify any visibility constraints to `snapshot`.
@@ -935,20 +909,13 @@ impl<'db> UseDefMapBuilder<'db> {
return;
}
// Note that this loop terminates when we reach a symbol not present in the snapshot.
// This means we keep visibility constraints for all new symbols, which is intended,
// since these symbols have been introduced in the corresponding branch, which might
// Note that this loop terminates when we reach a place not present in the snapshot.
// This means we keep visibility constraints for all new places, which is intended,
// since these places have been introduced in the corresponding branch, which might
// be subject to visibility constraints. We only simplify/reset visibility constraints
// for symbols that have the same bindings and declarations present compared to the
// for places that have the same bindings and declarations present compared to the
// snapshot.
for (current, snapshot) in self.symbol_states.iter_mut().zip(snapshot.symbol_states) {
current.simplify_visibility_constraints(snapshot);
}
for (current, snapshot) in self
.instance_attribute_states
.iter_mut()
.zip(snapshot.instance_attribute_states)
{
for (current, snapshot) in self.place_states.iter_mut().zip(snapshot.place_states) {
current.simplify_visibility_constraints(snapshot);
}
}
@@ -965,43 +932,64 @@ impl<'db> UseDefMapBuilder<'db> {
pub(super) fn record_declaration(
&mut self,
symbol: ScopedSymbolId,
place: ScopedPlaceId,
declaration: Definition<'db>,
) {
let def_id = self.all_definitions.push(Some(declaration));
let symbol_state = &mut self.symbol_states[symbol];
let def_id = self
.all_definitions
.push(DefinitionState::Defined(declaration));
let place_state = &mut self.place_states[place];
self.bindings_by_declaration
.insert(declaration, symbol_state.bindings().clone());
symbol_state.record_declaration(def_id);
.insert(declaration, place_state.bindings().clone());
place_state.record_declaration(def_id);
}
pub(super) fn record_declaration_and_binding(
&mut self,
symbol: ScopedSymbolId,
place: ScopedPlaceId,
definition: Definition<'db>,
is_place_name: bool,
) {
// We don't need to store anything in self.bindings_by_declaration or
// self.declarations_by_binding.
let def_id = self.all_definitions.push(Some(definition));
let symbol_state = &mut self.symbol_states[symbol];
symbol_state.record_declaration(def_id);
symbol_state.record_binding(def_id, self.scope_start_visibility, self.is_class_scope);
let def_id = self
.all_definitions
.push(DefinitionState::Defined(definition));
let place_state = &mut self.place_states[place];
place_state.record_declaration(def_id);
place_state.record_binding(
def_id,
self.scope_start_visibility,
self.is_class_scope,
is_place_name,
);
}
pub(super) fn delete_binding(&mut self, place: ScopedPlaceId, is_place_name: bool) {
let def_id = self.all_definitions.push(DefinitionState::Deleted);
let place_state = &mut self.place_states[place];
place_state.record_binding(
def_id,
self.scope_start_visibility,
self.is_class_scope,
is_place_name,
);
}
pub(super) fn record_use(
&mut self,
symbol: ScopedSymbolId,
place: ScopedPlaceId,
use_id: ScopedUseId,
node_key: NodeKey,
) {
// We have a use of a symbol; clone the current bindings for that symbol, and record them
// We have a use of a place; clone the current bindings for that place, and record them
// as the live bindings for this use.
let new_use = self
.bindings_by_use
.push(self.symbol_states[symbol].bindings().clone());
.push(self.place_states[place].bindings().clone());
debug_assert_eq!(use_id, new_use);
// Track reachability of all uses of symbols to silence `unresolved-reference`
// Track reachability of all uses of places to silence `unresolved-reference`
// diagnostics in unreachable code.
self.record_node_reachability(node_key);
}
@@ -1012,66 +1000,59 @@ impl<'db> UseDefMapBuilder<'db> {
pub(super) fn snapshot_eager_state(
&mut self,
enclosing_symbol: ScopedSymbolId,
enclosing_place: ScopedPlaceId,
scope: ScopeKind,
is_bound: bool,
enclosing_place_expr: &PlaceExpr,
) -> ScopedEagerSnapshotId {
// Names bound in class scopes are never visible to nested scopes, so we never need to
// save eager scope bindings in a class scope.
if scope.is_class() || !is_bound {
// Names bound in class scopes are never visible to nested scopes (but attributes/subscripts are visible),
// so we never need to save eager scope bindings in a class scope.
if (scope.is_class() && enclosing_place_expr.is_name()) || !enclosing_place_expr.is_bound()
{
self.eager_snapshots.push(EagerSnapshot::Constraint(
self.symbol_states[enclosing_symbol]
self.place_states[enclosing_place]
.bindings()
.unbound_narrowing_constraint(),
))
} else {
self.eager_snapshots.push(EagerSnapshot::Bindings(
self.symbol_states[enclosing_symbol].bindings().clone(),
self.place_states[enclosing_place].bindings().clone(),
))
}
}
/// Take a snapshot of the current visible-symbols state.
/// Take a snapshot of the current visible-places state.
pub(super) fn snapshot(&self) -> FlowSnapshot {
FlowSnapshot {
symbol_states: self.symbol_states.clone(),
instance_attribute_states: self.instance_attribute_states.clone(),
place_states: self.place_states.clone(),
scope_start_visibility: self.scope_start_visibility,
reachability: self.reachability,
}
}
/// Restore the current builder symbols state to the given snapshot.
/// Restore the current builder places state to the given snapshot.
pub(super) fn restore(&mut self, snapshot: FlowSnapshot) {
// We never remove symbols from `symbol_states` (it's an IndexVec, and the symbol
// IDs must line up), so the current number of known symbols must always be equal to or
// greater than the number of known symbols in a previously-taken snapshot.
let num_symbols = self.symbol_states.len();
debug_assert!(num_symbols >= snapshot.symbol_states.len());
let num_attributes = self.instance_attribute_states.len();
debug_assert!(num_attributes >= snapshot.instance_attribute_states.len());
// We never remove places from `place_states` (it's an IndexVec, and the place
// IDs must line up), so the current number of known places must always be equal to or
// greater than the number of known places in a previously-taken snapshot.
let num_places = self.place_states.len();
debug_assert!(num_places >= snapshot.place_states.len());
// Restore the current visible-definitions state to the given snapshot.
self.symbol_states = snapshot.symbol_states;
self.instance_attribute_states = snapshot.instance_attribute_states;
self.place_states = snapshot.place_states;
self.scope_start_visibility = snapshot.scope_start_visibility;
self.reachability = snapshot.reachability;
// If the snapshot we are restoring is missing some symbols we've recorded since, we need
// to fill them in so the symbol IDs continue to line up. Since they don't exist in the
// If the snapshot we are restoring is missing some places we've recorded since, we need
// to fill them in so the place IDs continue to line up. Since they don't exist in the
// snapshot, the correct state to fill them in with is "undefined".
self.symbol_states.resize(
num_symbols,
SymbolState::undefined(self.scope_start_visibility),
);
self.instance_attribute_states.resize(
num_attributes,
SymbolState::undefined(self.scope_start_visibility),
self.place_states.resize(
num_places,
PlaceState::undefined(self.scope_start_visibility),
);
}
/// Merge the given snapshot into the current state, reflecting that we might have taken either
/// path to get here. The new state for each symbol should include definitions from both the
/// path to get here. The new state for each place should include definitions from both the
/// prior state and the snapshot.
pub(super) fn merge(&mut self, snapshot: FlowSnapshot) {
// As an optimization, if we know statically that either of the snapshots is always
@@ -1089,16 +1070,13 @@ impl<'db> UseDefMapBuilder<'db> {
return;
}
// We never remove symbols from `symbol_states` (it's an IndexVec, and the symbol
// IDs must line up), so the current number of known symbols must always be equal to or
// greater than the number of known symbols in a previously-taken snapshot.
debug_assert!(self.symbol_states.len() >= snapshot.symbol_states.len());
debug_assert!(
self.instance_attribute_states.len() >= snapshot.instance_attribute_states.len()
);
// We never remove places from `place_states` (it's an IndexVec, and the place
// IDs must line up), so the current number of known places must always be equal to or
// greater than the number of known places in a previously-taken snapshot.
debug_assert!(self.place_states.len() >= snapshot.place_states.len());
let mut snapshot_definitions_iter = snapshot.symbol_states.into_iter();
for current in &mut self.symbol_states {
let mut snapshot_definitions_iter = snapshot.place_states.into_iter();
for current in &mut self.place_states {
if let Some(snapshot) = snapshot_definitions_iter.next() {
current.merge(
snapshot,
@@ -1107,27 +1085,11 @@ impl<'db> UseDefMapBuilder<'db> {
);
} else {
current.merge(
SymbolState::undefined(snapshot.scope_start_visibility),
&mut self.narrowing_constraints,
&mut self.visibility_constraints,
);
// Symbol not present in snapshot, so it's unbound/undeclared from that path.
}
}
let mut snapshot_definitions_iter = snapshot.instance_attribute_states.into_iter();
for current in &mut self.instance_attribute_states {
if let Some(snapshot) = snapshot_definitions_iter.next() {
current.merge(
snapshot,
&mut self.narrowing_constraints,
&mut self.visibility_constraints,
);
} else {
current.merge(
SymbolState::undefined(snapshot.scope_start_visibility),
PlaceState::undefined(snapshot.scope_start_visibility),
&mut self.narrowing_constraints,
&mut self.visibility_constraints,
);
// Place not present in snapshot, so it's unbound/undeclared from that path.
}
}
@@ -1142,8 +1104,7 @@ impl<'db> UseDefMapBuilder<'db> {
pub(super) fn finish(mut self) -> UseDefMap<'db> {
self.all_definitions.shrink_to_fit();
self.symbol_states.shrink_to_fit();
self.instance_attribute_states.shrink_to_fit();
self.place_states.shrink_to_fit();
self.bindings_by_use.shrink_to_fit();
self.node_reachability.shrink_to_fit();
self.declarations_by_binding.shrink_to_fit();
@@ -1157,8 +1118,7 @@ impl<'db> UseDefMapBuilder<'db> {
visibility_constraints: self.visibility_constraints.build(),
bindings_by_use: self.bindings_by_use,
node_reachability: self.node_reachability,
public_symbols: self.symbol_states,
instance_attributes: self.instance_attribute_states,
public_places: self.place_states,
declarations_by_binding: self.declarations_by_binding,
bindings_by_declaration: self.bindings_by_declaration,
eager_snapshots: self.eager_snapshots,

View File

@@ -1,4 +1,4 @@
//! Track live bindings per symbol, applicable constraints per binding, and live declarations.
//! Track live bindings per place, applicable constraints per binding, and live declarations.
//!
//! These data structures operate entirely on scope-local newtype-indices for definitions and
//! constraints, referring to their location in the `all_definitions` and `all_constraints`
@@ -60,9 +60,9 @@ pub(super) struct ScopedDefinitionId;
impl ScopedDefinitionId {
/// A special ID that is used to describe an implicit start-of-scope state. When
/// we see that this definition is live, we know that the symbol is (possibly)
/// we see that this definition is live, we know that the place is (possibly)
/// unbound or undeclared at a given usage site.
/// When creating a use-def-map builder, we always add an empty `None` definition
/// When creating a use-def-map builder, we always add an empty `DefinitionState::Undefined` definition
/// at index 0, so this ID is always present.
pub(super) const UNBOUND: ScopedDefinitionId = ScopedDefinitionId::from_u32(0);
@@ -71,19 +71,19 @@ impl ScopedDefinitionId {
}
}
/// Can keep inline this many live bindings or declarations per symbol at a given time; more will
/// Can keep inline this many live bindings or declarations per place at a given time; more will
/// go to heap.
const INLINE_DEFINITIONS_PER_SYMBOL: usize = 4;
const INLINE_DEFINITIONS_PER_PLACE: usize = 4;
/// Live declarations for a single symbol at some point in control flow, with their
/// Live declarations for a single place at some point in control flow, with their
/// corresponding visibility constraints.
#[derive(Clone, Debug, Default, PartialEq, Eq, salsa::Update)]
pub(super) struct SymbolDeclarations {
/// A list of live declarations for this symbol, sorted by their `ScopedDefinitionId`
live_declarations: SmallVec<[LiveDeclaration; INLINE_DEFINITIONS_PER_SYMBOL]>,
pub(super) struct Declarations {
/// A list of live declarations for this place, sorted by their `ScopedDefinitionId`
live_declarations: SmallVec<[LiveDeclaration; INLINE_DEFINITIONS_PER_PLACE]>,
}
/// One of the live declarations for a single symbol at some point in control flow.
/// One of the live declarations for a single place at some point in control flow.
#[derive(Clone, Debug, PartialEq, Eq)]
pub(super) struct LiveDeclaration {
pub(super) declaration: ScopedDefinitionId,
@@ -92,7 +92,7 @@ pub(super) struct LiveDeclaration {
pub(super) type LiveDeclarationsIterator<'a> = std::slice::Iter<'a, LiveDeclaration>;
impl SymbolDeclarations {
impl Declarations {
fn undeclared(scope_start_visibility: ScopedVisibilityConstraintId) -> Self {
let initial_declaration = LiveDeclaration {
declaration: ScopedDefinitionId::UNBOUND,
@@ -103,7 +103,7 @@ impl SymbolDeclarations {
}
}
/// Record a newly-encountered declaration for this symbol.
/// Record a newly-encountered declaration for this place.
fn record_declaration(&mut self, declaration: ScopedDefinitionId) {
// The new declaration replaces all previous live declaration in this path.
self.live_declarations.clear();
@@ -125,17 +125,17 @@ impl SymbolDeclarations {
}
}
/// Return an iterator over live declarations for this symbol.
/// Return an iterator over live declarations for this place.
pub(super) fn iter(&self) -> LiveDeclarationsIterator<'_> {
self.live_declarations.iter()
}
/// Iterate over the IDs of each currently live declaration for this symbol
/// Iterate over the IDs of each currently live declaration for this place
fn iter_declarations(&self) -> impl Iterator<Item = ScopedDefinitionId> + '_ {
self.iter().map(|lb| lb.declaration)
}
fn simplify_visibility_constraints(&mut self, other: SymbolDeclarations) {
fn simplify_visibility_constraints(&mut self, other: Declarations) {
// If the set of live declarations hasn't changed, don't simplify.
if self.live_declarations.len() != other.live_declarations.len()
|| !self.iter_declarations().eq(other.iter_declarations())
@@ -181,7 +181,7 @@ impl SymbolDeclarations {
}
}
/// A snapshot of a symbol state that can be used to resolve a reference in a nested eager scope.
/// A snapshot of a place state that can be used to resolve a reference in a nested eager scope.
/// If there are bindings in a (non-class) scope , they are stored in `Bindings`.
/// Even if it's a class scope (class variables are not visible to nested scopes) or there are no
/// bindings, the current narrowing constraint is necessary for narrowing, so it's stored in
@@ -189,34 +189,30 @@ impl SymbolDeclarations {
#[derive(Clone, Debug, PartialEq, Eq, salsa::Update)]
pub(super) enum EagerSnapshot {
Constraint(ScopedNarrowingConstraint),
Bindings(SymbolBindings),
Bindings(Bindings),
}
/// Live bindings for a single symbol at some point in control flow. Each live binding comes
/// Live bindings for a single place at some point in control flow. Each live binding comes
/// with a set of narrowing constraints and a visibility constraint.
#[derive(Clone, Debug, Default, PartialEq, Eq, salsa::Update)]
pub(super) struct SymbolBindings {
pub(super) struct Bindings {
/// The narrowing constraint applicable to the "unbound" binding, if we need access to it even
/// when it's not visible. This happens in class scopes, where local bindings are not visible
/// when it's not visible. This happens in class scopes, where local name bindings are not visible
/// to nested scopes, but we still need to know what narrowing constraints were applied to the
/// "unbound" binding.
unbound_narrowing_constraint: Option<ScopedNarrowingConstraint>,
/// A list of live bindings for this symbol, sorted by their `ScopedDefinitionId`
live_bindings: SmallVec<[LiveBinding; INLINE_DEFINITIONS_PER_SYMBOL]>,
/// A list of live bindings for this place, sorted by their `ScopedDefinitionId`
live_bindings: SmallVec<[LiveBinding; INLINE_DEFINITIONS_PER_PLACE]>,
}
impl SymbolBindings {
impl Bindings {
pub(super) fn unbound_narrowing_constraint(&self) -> ScopedNarrowingConstraint {
debug_assert!(
self.unbound_narrowing_constraint.is_some()
|| self.live_bindings[0].binding.is_unbound()
);
self.unbound_narrowing_constraint
.unwrap_or(self.live_bindings[0].narrowing_constraint)
}
}
/// One of the live bindings for a single symbol at some point in control flow.
/// One of the live bindings for a single place at some point in control flow.
#[derive(Clone, Debug, PartialEq, Eq)]
pub(super) struct LiveBinding {
pub(super) binding: ScopedDefinitionId,
@@ -226,7 +222,7 @@ pub(super) struct LiveBinding {
pub(super) type LiveBindingsIterator<'a> = std::slice::Iter<'a, LiveBinding>;
impl SymbolBindings {
impl Bindings {
fn unbound(scope_start_visibility: ScopedVisibilityConstraintId) -> Self {
let initial_binding = LiveBinding {
binding: ScopedDefinitionId::UNBOUND,
@@ -239,16 +235,17 @@ impl SymbolBindings {
}
}
/// Record a newly-encountered binding for this symbol.
/// Record a newly-encountered binding for this place.
pub(super) fn record_binding(
&mut self,
binding: ScopedDefinitionId,
visibility_constraint: ScopedVisibilityConstraintId,
is_class_scope: bool,
is_place_name: bool,
) {
// If we are in a class scope, and the unbound binding was previously visible, but we will
// If we are in a class scope, and the unbound name binding was previously visible, but we will
// now replace it, record the narrowing constraints on it:
if is_class_scope && self.live_bindings[0].binding.is_unbound() {
if is_class_scope && is_place_name && self.live_bindings[0].binding.is_unbound() {
self.unbound_narrowing_constraint = Some(self.live_bindings[0].narrowing_constraint);
}
// The new binding replaces all previous live bindings in this path, and has no
@@ -285,17 +282,17 @@ impl SymbolBindings {
}
}
/// Iterate over currently live bindings for this symbol
/// Iterate over currently live bindings for this place
pub(super) fn iter(&self) -> LiveBindingsIterator<'_> {
self.live_bindings.iter()
}
/// Iterate over the IDs of each currently live binding for this symbol
/// Iterate over the IDs of each currently live binding for this place
fn iter_bindings(&self) -> impl Iterator<Item = ScopedDefinitionId> + '_ {
self.iter().map(|lb| lb.binding)
}
fn simplify_visibility_constraints(&mut self, other: SymbolBindings) {
fn simplify_visibility_constraints(&mut self, other: Bindings) {
// If the set of live bindings hasn't changed, don't simplify.
if self.live_bindings.len() != other.live_bindings.len()
|| !self.iter_bindings().eq(other.iter_bindings())
@@ -360,30 +357,35 @@ impl SymbolBindings {
}
#[derive(Clone, Debug, PartialEq, Eq)]
pub(in crate::semantic_index) struct SymbolState {
declarations: SymbolDeclarations,
bindings: SymbolBindings,
pub(in crate::semantic_index) struct PlaceState {
declarations: Declarations,
bindings: Bindings,
}
impl SymbolState {
/// Return a new [`SymbolState`] representing an unbound, undeclared symbol.
impl PlaceState {
/// Return a new [`PlaceState`] representing an unbound, undeclared place.
pub(super) fn undefined(scope_start_visibility: ScopedVisibilityConstraintId) -> Self {
Self {
declarations: SymbolDeclarations::undeclared(scope_start_visibility),
bindings: SymbolBindings::unbound(scope_start_visibility),
declarations: Declarations::undeclared(scope_start_visibility),
bindings: Bindings::unbound(scope_start_visibility),
}
}
/// Record a newly-encountered binding for this symbol.
/// Record a newly-encountered binding for this place.
pub(super) fn record_binding(
&mut self,
binding_id: ScopedDefinitionId,
visibility_constraint: ScopedVisibilityConstraintId,
is_class_scope: bool,
is_place_name: bool,
) {
debug_assert_ne!(binding_id, ScopedDefinitionId::UNBOUND);
self.bindings
.record_binding(binding_id, visibility_constraint, is_class_scope);
self.bindings.record_binding(
binding_id,
visibility_constraint,
is_class_scope,
is_place_name,
);
}
/// Add given constraint to all live bindings.
@@ -409,24 +411,24 @@ impl SymbolState {
}
/// Simplifies this snapshot to have the same visibility constraints as a previous point in the
/// control flow, but only if the set of live bindings or declarations for this symbol hasn't
/// control flow, but only if the set of live bindings or declarations for this place hasn't
/// changed.
pub(super) fn simplify_visibility_constraints(&mut self, snapshot_state: SymbolState) {
pub(super) fn simplify_visibility_constraints(&mut self, snapshot_state: PlaceState) {
self.bindings
.simplify_visibility_constraints(snapshot_state.bindings);
self.declarations
.simplify_visibility_constraints(snapshot_state.declarations);
}
/// Record a newly-encountered declaration of this symbol.
/// Record a newly-encountered declaration of this place.
pub(super) fn record_declaration(&mut self, declaration_id: ScopedDefinitionId) {
self.declarations.record_declaration(declaration_id);
}
/// Merge another [`SymbolState`] into this one.
/// Merge another [`PlaceState`] into this one.
pub(super) fn merge(
&mut self,
b: SymbolState,
b: PlaceState,
narrowing_constraints: &mut NarrowingConstraintsBuilder,
visibility_constraints: &mut VisibilityConstraintsBuilder,
) {
@@ -436,11 +438,11 @@ impl SymbolState {
.merge(b.declarations, visibility_constraints);
}
pub(super) fn bindings(&self) -> &SymbolBindings {
pub(super) fn bindings(&self) -> &Bindings {
&self.bindings
}
pub(super) fn declarations(&self) -> &SymbolDeclarations {
pub(super) fn declarations(&self) -> &Declarations {
&self.declarations
}
}
@@ -454,10 +456,10 @@ mod tests {
#[track_caller]
fn assert_bindings(
narrowing_constraints: &NarrowingConstraintsBuilder,
symbol: &SymbolState,
place: &PlaceState,
expected: &[&str],
) {
let actual = symbol
let actual = place
.bindings()
.iter()
.map(|live_binding| {
@@ -479,8 +481,8 @@ mod tests {
}
#[track_caller]
pub(crate) fn assert_declarations(symbol: &SymbolState, expected: &[&str]) {
let actual = symbol
pub(crate) fn assert_declarations(place: &PlaceState, expected: &[&str]) {
let actual = place
.declarations()
.iter()
.map(
@@ -502,7 +504,7 @@ mod tests {
#[test]
fn unbound() {
let narrowing_constraints = NarrowingConstraintsBuilder::default();
let sym = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
let sym = PlaceState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
assert_bindings(&narrowing_constraints, &sym, &["unbound<>"]);
}
@@ -510,11 +512,12 @@ mod tests {
#[test]
fn with() {
let narrowing_constraints = NarrowingConstraintsBuilder::default();
let mut sym = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
let mut sym = PlaceState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
sym.record_binding(
ScopedDefinitionId::from_u32(1),
ScopedVisibilityConstraintId::ALWAYS_TRUE,
false,
true,
);
assert_bindings(&narrowing_constraints, &sym, &["1<>"]);
@@ -523,11 +526,12 @@ mod tests {
#[test]
fn record_constraint() {
let mut narrowing_constraints = NarrowingConstraintsBuilder::default();
let mut sym = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
let mut sym = PlaceState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
sym.record_binding(
ScopedDefinitionId::from_u32(1),
ScopedVisibilityConstraintId::ALWAYS_TRUE,
false,
true,
);
let predicate = ScopedPredicateId::from_u32(0).into();
sym.record_narrowing_constraint(&mut narrowing_constraints, predicate);
@@ -541,20 +545,22 @@ mod tests {
let mut visibility_constraints = VisibilityConstraintsBuilder::default();
// merging the same definition with the same constraint keeps the constraint
let mut sym1a = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
let mut sym1a = PlaceState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
sym1a.record_binding(
ScopedDefinitionId::from_u32(1),
ScopedVisibilityConstraintId::ALWAYS_TRUE,
false,
true,
);
let predicate = ScopedPredicateId::from_u32(0).into();
sym1a.record_narrowing_constraint(&mut narrowing_constraints, predicate);
let mut sym1b = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
let mut sym1b = PlaceState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
sym1b.record_binding(
ScopedDefinitionId::from_u32(1),
ScopedVisibilityConstraintId::ALWAYS_TRUE,
false,
true,
);
let predicate = ScopedPredicateId::from_u32(0).into();
sym1b.record_narrowing_constraint(&mut narrowing_constraints, predicate);
@@ -568,20 +574,22 @@ mod tests {
assert_bindings(&narrowing_constraints, &sym1, &["1<0>"]);
// merging the same definition with differing constraints drops all constraints
let mut sym2a = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
let mut sym2a = PlaceState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
sym2a.record_binding(
ScopedDefinitionId::from_u32(2),
ScopedVisibilityConstraintId::ALWAYS_TRUE,
false,
true,
);
let predicate = ScopedPredicateId::from_u32(1).into();
sym2a.record_narrowing_constraint(&mut narrowing_constraints, predicate);
let mut sym1b = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
let mut sym1b = PlaceState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
sym1b.record_binding(
ScopedDefinitionId::from_u32(2),
ScopedVisibilityConstraintId::ALWAYS_TRUE,
false,
true,
);
let predicate = ScopedPredicateId::from_u32(2).into();
sym1b.record_narrowing_constraint(&mut narrowing_constraints, predicate);
@@ -595,16 +603,17 @@ mod tests {
assert_bindings(&narrowing_constraints, &sym2, &["2<>"]);
// merging a constrained definition with unbound keeps both
let mut sym3a = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
let mut sym3a = PlaceState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
sym3a.record_binding(
ScopedDefinitionId::from_u32(3),
ScopedVisibilityConstraintId::ALWAYS_TRUE,
false,
true,
);
let predicate = ScopedPredicateId::from_u32(3).into();
sym3a.record_narrowing_constraint(&mut narrowing_constraints, predicate);
let sym2b = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
let sym2b = PlaceState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
sym3a.merge(
sym2b,
@@ -626,14 +635,14 @@ mod tests {
#[test]
fn no_declaration() {
let sym = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
let sym = PlaceState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
assert_declarations(&sym, &["undeclared"]);
}
#[test]
fn record_declaration() {
let mut sym = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
let mut sym = PlaceState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
sym.record_declaration(ScopedDefinitionId::from_u32(1));
assert_declarations(&sym, &["1"]);
@@ -641,7 +650,7 @@ mod tests {
#[test]
fn record_declaration_override() {
let mut sym = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
let mut sym = PlaceState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
sym.record_declaration(ScopedDefinitionId::from_u32(1));
sym.record_declaration(ScopedDefinitionId::from_u32(2));
@@ -652,10 +661,10 @@ mod tests {
fn record_declaration_merge() {
let mut narrowing_constraints = NarrowingConstraintsBuilder::default();
let mut visibility_constraints = VisibilityConstraintsBuilder::default();
let mut sym = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
let mut sym = PlaceState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
sym.record_declaration(ScopedDefinitionId::from_u32(1));
let mut sym2 = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
let mut sym2 = PlaceState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
sym2.record_declaration(ScopedDefinitionId::from_u32(2));
sym.merge(
@@ -671,10 +680,10 @@ mod tests {
fn record_declaration_merge_partial_undeclared() {
let mut narrowing_constraints = NarrowingConstraintsBuilder::default();
let mut visibility_constraints = VisibilityConstraintsBuilder::default();
let mut sym = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
let mut sym = PlaceState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
sym.record_declaration(ScopedDefinitionId::from_u32(1));
let sym2 = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
let sym2 = PlaceState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
sym.merge(
sym2,

View File

@@ -180,12 +180,12 @@ use rustc_hash::FxHashMap;
use crate::Db;
use crate::dunder_all::dunder_all_names;
use crate::place::{RequiresExplicitReExport, imported_symbol};
use crate::semantic_index::expression::Expression;
use crate::semantic_index::place_table;
use crate::semantic_index::predicate::{
PatternPredicate, PatternPredicateKind, Predicate, PredicateNode, Predicates, ScopedPredicateId,
};
use crate::semantic_index::symbol_table;
use crate::symbol::{RequiresExplicitReExport, imported_symbol};
use crate::types::{Truthiness, Type, infer_expression_type};
/// A ternary formula that defines under what conditions a binding is visible. (A ternary formula
@@ -654,8 +654,10 @@ impl VisibilityConstraints {
}
PredicateNode::Pattern(inner) => Self::analyze_single_pattern_predicate(db, inner),
PredicateNode::StarImportPlaceholder(star_import) => {
let symbol_table = symbol_table(db, star_import.scope(db));
let symbol_name = symbol_table.symbol(star_import.symbol_id(db)).name();
let place_table = place_table(db, star_import.scope(db));
let symbol_name = place_table
.place_expr(star_import.symbol_id(db))
.expect_name();
let referenced_file = star_import.referenced_file(db);
let requires_explicit_reexport = match dunder_all_names(db, referenced_file) {
@@ -675,15 +677,15 @@ impl VisibilityConstraints {
};
match imported_symbol(db, referenced_file, symbol_name, requires_explicit_reexport)
.symbol
.place
{
crate::symbol::Symbol::Type(_, crate::symbol::Boundness::Bound) => {
crate::place::Place::Type(_, crate::place::Boundness::Bound) => {
Truthiness::AlwaysTrue
}
crate::symbol::Symbol::Type(_, crate::symbol::Boundness::PossiblyUnbound) => {
crate::place::Place::Type(_, crate::place::Boundness::PossiblyUnbound) => {
Truthiness::Ambiguous
}
crate::symbol::Symbol::Unbound => Truthiness::AlwaysFalse,
crate::place::Place::Unbound => Truthiness::AlwaysFalse,
}
}
}

View File

@@ -8,8 +8,9 @@ use crate::Db;
use crate::module_name::ModuleName;
use crate::module_resolver::{Module, resolve_module};
use crate::semantic_index::ast_ids::HasScopedExpressionId;
use crate::semantic_index::place::FileScopeId;
use crate::semantic_index::semantic_index;
use crate::semantic_index::symbol::FileScopeId;
use crate::types::ide_support::all_declarations_and_bindings;
use crate::types::{Type, binding_type, infer_scope_types};
pub struct SemanticModel<'db> {
@@ -40,28 +41,42 @@ impl<'db> SemanticModel<'db> {
resolve_module(self.db, module_name)
}
/// Returns completions for symbols available in a `object.<CURSOR>` context.
pub fn attribute_completions(&self, node: &ast::ExprAttribute) -> Vec<Name> {
let ty = node.value.inferred_type(self);
crate::types::all_members(self.db, ty).into_iter().collect()
}
/// Returns completions for symbols available in the scope containing the
/// given expression.
///
/// If a scope could not be determined, then completions for the global
/// scope of this model's `File` are returned.
pub fn completions(&self, node: ast::AnyNodeRef<'_>) -> Vec<Name> {
pub fn scoped_completions(&self, node: ast::AnyNodeRef<'_>) -> Vec<Name> {
let index = semantic_index(self.db, self.file);
let file_scope = match node {
ast::AnyNodeRef::Identifier(identifier) => index.expression_scope_id(identifier),
// TODO: We currently use `try_expression_scope_id` here as a hotfix for [1].
// Revert this to use `expression_scope_id` once a proper fix is in place.
//
// [1] https://github.com/astral-sh/ty/issues/572
let Some(file_scope) = (match node {
ast::AnyNodeRef::Identifier(identifier) => index.try_expression_scope_id(identifier),
node => match node.as_expr_ref() {
// If we couldn't identify a specific
// expression that we're in, then just
// fall back to the global scope.
None => FileScopeId::global(),
Some(expr) => index.expression_scope_id(expr),
None => Some(FileScopeId::global()),
Some(expr) => index.try_expression_scope_id(expr),
},
}) else {
return vec![];
};
let mut symbols = vec![];
for (file_scope, _) in index.ancestor_scopes(file_scope) {
for symbol in index.symbol_table(file_scope).symbols() {
symbols.push(symbol.name().clone());
}
symbols.extend(all_declarations_and_bindings(
self.db,
file_scope.to_scope_id(self.db, self.file),
));
}
symbols
}

View File

@@ -95,15 +95,13 @@ impl PythonEnvironment {
origin: SysPrefixPathOrigin,
system: &dyn System,
) -> SitePackagesDiscoveryResult<Self> {
let path = SysPrefixPath::new(path, origin, system)?;
let path = SysPrefixPath::new(path.as_ref(), origin, system)?;
// Attempt to inspect as a virtual environment first
// TODO(zanieb): Consider avoiding the clone here by checking for `pyvenv.cfg` ahead-of-time
match VirtualEnvironment::new(path.clone(), system) {
match VirtualEnvironment::new(path, system) {
Ok(venv) => Ok(Self::Virtual(venv)),
// If there's not a `pyvenv.cfg` marker, attempt to inspect as a system environment
//
Err(SitePackagesDiscoveryError::NoPyvenvCfgFile(_, _))
Err(SitePackagesDiscoveryError::NoPyvenvCfgFile(path, _))
if !origin.must_be_virtual_env() =>
{
Ok(Self::System(SystemEnvironment::new(path)))
@@ -207,9 +205,10 @@ impl VirtualEnvironment {
let pyvenv_cfg_path = path.join("pyvenv.cfg");
tracing::debug!("Attempting to parse virtual environment metadata at '{pyvenv_cfg_path}'");
let pyvenv_cfg = system
.read_to_string(&pyvenv_cfg_path)
.map_err(|io_err| SitePackagesDiscoveryError::NoPyvenvCfgFile(path.origin, io_err))?;
let pyvenv_cfg = match system.read_to_string(&pyvenv_cfg_path) {
Ok(pyvenv_cfg) => pyvenv_cfg,
Err(err) => return Err(SitePackagesDiscoveryError::NoPyvenvCfgFile(path, err)),
};
let parsed_pyvenv_cfg =
PyvenvCfgParser::new(&pyvenv_cfg)
@@ -530,20 +529,48 @@ impl SystemEnvironment {
}
}
/// Enumeration of ways in which `site-packages` discovery can fail.
#[derive(Debug, thiserror::Error)]
pub(crate) enum SitePackagesDiscoveryError {
/// `site-packages` discovery failed because the provided path couldn't be canonicalized.
#[error("Invalid {1}: `{0}` could not be canonicalized")]
EnvDirCanonicalizationError(SystemPathBuf, SysPrefixPathOrigin, #[source] io::Error),
#[error("Invalid {1}: `{0}` does not point to a directory on disk")]
EnvDirNotDirectory(SystemPathBuf, SysPrefixPathOrigin),
#[error("{0} points to a broken venv with no pyvenv.cfg file")]
NoPyvenvCfgFile(SysPrefixPathOrigin, #[source] io::Error),
CanonicalizationError(SystemPathBuf, SysPrefixPathOrigin, #[source] io::Error),
/// `site-packages` discovery failed because the provided path doesn't appear to point to
/// a Python executable or a `sys.prefix` directory.
#[error(
"Invalid {1}: `{0}` does not point to a {thing}",
thing = if .1.must_point_directly_to_sys_prefix() {
"directory on disk"
} else {
"Python executable or a directory on disk"
}
)]
PathNotExecutableOrDirectory(SystemPathBuf, SysPrefixPathOrigin),
/// `site-packages` discovery failed because the [`SysPrefixPathOrigin`] indicated that
/// the provided path should point to the `sys.prefix` of a virtual environment,
/// but there was no file at `<sys.prefix>/pyvenv.cfg`.
#[error("{} points to a broken venv with no pyvenv.cfg file", .0.origin)]
NoPyvenvCfgFile(SysPrefixPath, #[source] io::Error),
/// `site-packages` discovery failed because the `pyvenv.cfg` file could not be parsed.
#[error("Failed to parse the pyvenv.cfg file at {0} because {1}")]
PyvenvCfgParseError(SystemPathBuf, PyvenvCfgParseErrorKind),
/// `site-packages` discovery failed because we're on a Unix system,
/// we weren't able to figure out from the `pyvenv.cfg` file exactly where `site-packages`
/// would be relative to the `sys.prefix` path, and we tried to fallback to iterating
/// through the `<sys.prefix>/lib` directory looking for a `site-packages` directory,
/// but we came across some I/O error while trying to do so.
#[error(
"Failed to search the `lib` directory of the Python installation at {1} for `site-packages`"
"Failed to iterate over the contents of the `lib` directory of the Python installation at {1}"
)]
CouldNotReadLibDirectory(#[source] io::Error, SysPrefixPath),
/// We looked everywhere we could think of for the `site-packages` directory,
/// but none could be found despite our best endeavours.
#[error("Could not find the `site-packages` directory for the Python installation at {0}")]
NoSitePackagesDirFound(SysPrefixPath),
}
@@ -709,14 +736,6 @@ pub(crate) struct SysPrefixPath {
impl SysPrefixPath {
fn new(
unvalidated_path: impl AsRef<SystemPath>,
origin: SysPrefixPathOrigin,
system: &dyn System,
) -> SitePackagesDiscoveryResult<Self> {
Self::new_impl(unvalidated_path.as_ref(), origin, system)
}
fn new_impl(
unvalidated_path: &SystemPath,
origin: SysPrefixPathOrigin,
system: &dyn System,
@@ -727,24 +746,79 @@ impl SysPrefixPath {
let canonicalized = system
.canonicalize_path(unvalidated_path)
.map_err(|io_err| {
SitePackagesDiscoveryError::EnvDirCanonicalizationError(
unvalidated_path.to_path_buf(),
origin,
io_err,
)
let unvalidated_path = unvalidated_path.to_path_buf();
if io_err.kind() == io::ErrorKind::NotFound {
SitePackagesDiscoveryError::PathNotExecutableOrDirectory(
unvalidated_path,
origin,
)
} else {
SitePackagesDiscoveryError::CanonicalizationError(
unvalidated_path,
origin,
io_err,
)
}
})?;
system
.is_directory(&canonicalized)
.then_some(Self {
inner: canonicalized,
origin,
})
.ok_or_else(|| {
SitePackagesDiscoveryError::EnvDirNotDirectory(
if origin.must_point_directly_to_sys_prefix() {
return system
.is_directory(&canonicalized)
.then_some(Self {
inner: canonicalized,
origin,
})
.ok_or_else(|| {
SitePackagesDiscoveryError::PathNotExecutableOrDirectory(
unvalidated_path.to_path_buf(),
origin,
)
});
}
let sys_prefix = if system.is_file(&canonicalized)
&& canonicalized
.file_name()
.is_some_and(|name| name.starts_with("python"))
{
// It looks like they passed us a path to a Python executable, e.g. `.venv/bin/python3`.
// Try to figure out the `sys.prefix` value from the Python executable.
let sys_prefix = if cfg!(windows) {
// On Windows, the relative path to the Python executable from `sys.prefix`
// is different depending on whether it's a virtual environment or a system installation.
// System installations have their executable at `<sys.prefix>/python.exe`,
// whereas virtual environments have their executable at `<sys.prefix>/Scripts/python.exe`.
canonicalized.parent().and_then(|parent| {
if parent.file_name() == Some("Scripts") {
parent.parent()
} else {
Some(parent)
}
})
} else {
// On Unix, `sys.prefix` is always the grandparent directory of the Python executable,
// regardless of whether it's a virtual environment or a system installation.
canonicalized.ancestors().nth(2)
};
sys_prefix.map(SystemPath::to_path_buf).ok_or_else(|| {
SitePackagesDiscoveryError::PathNotExecutableOrDirectory(
unvalidated_path.to_path_buf(),
origin,
)
})
})?
} else if system.is_directory(&canonicalized) {
canonicalized
} else {
return Err(SitePackagesDiscoveryError::PathNotExecutableOrDirectory(
unvalidated_path.to_path_buf(),
origin,
));
};
Ok(Self {
inner: sys_prefix,
origin,
})
}
fn from_executable_home_path(path: &PythonHomePath) -> Option<Self> {
@@ -801,12 +875,26 @@ pub enum SysPrefixPathOrigin {
impl SysPrefixPathOrigin {
/// Whether the given `sys.prefix` path must be a virtual environment (rather than a system
/// Python environment).
pub(crate) fn must_be_virtual_env(self) -> bool {
pub(crate) const fn must_be_virtual_env(self) -> bool {
match self {
Self::LocalVenv | Self::VirtualEnvVar => true,
Self::PythonCliFlag | Self::DerivedFromPyvenvCfg | Self::CondaPrefixVar => false,
}
}
/// Whether paths with this origin always point directly to the `sys.prefix` directory.
///
/// Some variants can point either directly to `sys.prefix` or to a Python executable inside
/// the `sys.prefix` directory, e.g. the `--python` CLI flag.
pub(crate) const fn must_point_directly_to_sys_prefix(self) -> bool {
match self {
Self::PythonCliFlag => false,
Self::VirtualEnvVar
| Self::CondaPrefixVar
| Self::DerivedFromPyvenvCfg
| Self::LocalVenv => true,
}
}
}
impl Display for SysPrefixPathOrigin {
@@ -1367,7 +1455,7 @@ mod tests {
let system = TestSystem::default();
assert!(matches!(
PythonEnvironment::new("/env", SysPrefixPathOrigin::PythonCliFlag, &system),
Err(SitePackagesDiscoveryError::EnvDirCanonicalizationError(..))
Err(SitePackagesDiscoveryError::PathNotExecutableOrDirectory(..))
));
}
@@ -1380,7 +1468,7 @@ mod tests {
.unwrap();
assert!(matches!(
PythonEnvironment::new("/env", SysPrefixPathOrigin::PythonCliFlag, &system),
Err(SitePackagesDiscoveryError::EnvDirNotDirectory(..))
Err(SitePackagesDiscoveryError::PathNotExecutableOrDirectory(..))
));
}

File diff suppressed because it is too large Load Diff

View File

@@ -1,6 +1,11 @@
use std::borrow::Cow;
use std::ops::{Deref, DerefMut};
use itertools::{Either, Itertools};
use crate::Db;
use crate::types::{KnownClass, TupleType};
use super::Type;
/// Arguments for a single call, in source order.
@@ -86,6 +91,10 @@ impl<'a, 'db> CallArgumentTypes<'a, 'db> {
Self { arguments, types }
}
pub(crate) fn types(&self) -> &[Type<'db>] {
&self.types
}
/// Prepend an optional extra synthetic argument (for a `self` or `cls` parameter) to the front
/// of this argument list. (If `bound_self` is none, we return the argument list
/// unmodified.)
@@ -108,6 +117,72 @@ impl<'a, 'db> CallArgumentTypes<'a, 'db> {
pub(crate) fn iter(&self) -> impl Iterator<Item = (Argument<'a>, Type<'db>)> + '_ {
self.arguments.iter().zip(self.types.iter().copied())
}
/// Returns an iterator on performing [argument type expansion].
///
/// Each element of the iterator represents a set of argument lists, where each argument list
/// contains the same arguments, but with one or more of the argument types expanded.
///
/// [argument type expansion]: https://typing.python.org/en/latest/spec/overload.html#argument-type-expansion
pub(crate) fn expand(&self, db: &'db dyn Db) -> impl Iterator<Item = Vec<Vec<Type<'db>>>> + '_ {
/// Represents the state of the expansion process.
///
/// This is useful to avoid cloning the initial types vector if none of the types can be
/// expanded.
enum State<'a, 'db> {
Initial(&'a Vec<Type<'db>>),
Expanded(Vec<Vec<Type<'db>>>),
}
impl<'db> State<'_, 'db> {
fn len(&self) -> usize {
match self {
State::Initial(_) => 1,
State::Expanded(expanded) => expanded.len(),
}
}
fn iter(&self) -> impl Iterator<Item = &Vec<Type<'db>>> + '_ {
match self {
State::Initial(types) => std::slice::from_ref(*types).iter(),
State::Expanded(expanded) => expanded.iter(),
}
}
}
let mut index = 0;
std::iter::successors(Some(State::Initial(&self.types)), move |previous| {
// Find the next type that can be expanded.
let expanded_types = loop {
let arg_type = self.types.get(index)?;
if let Some(expanded_types) = expand_type(db, *arg_type) {
break expanded_types;
}
index += 1;
};
let mut expanded_arg_types = Vec::with_capacity(expanded_types.len() * previous.len());
for pre_expanded_types in previous.iter() {
for subtype in &expanded_types {
let mut new_expanded_types = pre_expanded_types.clone();
new_expanded_types[index] = *subtype;
expanded_arg_types.push(new_expanded_types);
}
}
// Increment the index to move to the next argument type for the next iteration.
index += 1;
Some(State::Expanded(expanded_arg_types))
})
.skip(1) // Skip the initial state, which has no expanded types.
.map(|state| match state {
State::Initial(_) => unreachable!("initial state should be skipped"),
State::Expanded(expanded) => expanded,
})
}
}
impl<'a> Deref for CallArgumentTypes<'a, '_> {
@@ -122,3 +197,138 @@ impl<'a> DerefMut for CallArgumentTypes<'a, '_> {
&mut self.arguments
}
}
/// Expands a type into its possible subtypes, if applicable.
///
/// Returns [`None`] if the type cannot be expanded.
fn expand_type<'db>(db: &'db dyn Db, ty: Type<'db>) -> Option<Vec<Type<'db>>> {
// TODO: Expand enums to their variants
match ty {
Type::NominalInstance(instance) if instance.class.is_known(db, KnownClass::Bool) => {
Some(vec![
Type::BooleanLiteral(true),
Type::BooleanLiteral(false),
])
}
Type::Tuple(tuple) => {
// Note: This should only account for tuples of known length, i.e., `tuple[bool, ...]`
// should not be expanded here.
let expanded = tuple
.iter(db)
.map(|element| {
if let Some(expanded) = expand_type(db, element) {
Either::Left(expanded.into_iter())
} else {
Either::Right(std::iter::once(element))
}
})
.multi_cartesian_product()
.map(|types| TupleType::from_elements(db, types))
.collect::<Vec<_>>();
if expanded.len() == 1 {
// There are no elements in the tuple type that can be expanded.
None
} else {
Some(expanded)
}
}
Type::Union(union) => Some(union.iter(db).copied().collect()),
// We don't handle `type[A | B]` here because it's already stored in the expanded form
// i.e., `type[A] | type[B]` which is handled by the `Type::Union` case.
_ => None,
}
}
#[cfg(test)]
mod tests {
use crate::db::tests::setup_db;
use crate::types::{KnownClass, TupleType, Type, UnionType};
use super::expand_type;
#[test]
fn expand_union_type() {
let db = setup_db();
let types = [
KnownClass::Int.to_instance(&db),
KnownClass::Str.to_instance(&db),
KnownClass::Bytes.to_instance(&db),
];
let union_type = UnionType::from_elements(&db, types);
let expanded = expand_type(&db, union_type).unwrap();
assert_eq!(expanded.len(), types.len());
assert_eq!(expanded, types);
}
#[test]
fn expand_bool_type() {
let db = setup_db();
let bool_instance = KnownClass::Bool.to_instance(&db);
let expanded = expand_type(&db, bool_instance).unwrap();
let expected_types = [Type::BooleanLiteral(true), Type::BooleanLiteral(false)];
assert_eq!(expanded.len(), expected_types.len());
assert_eq!(expanded, expected_types);
}
#[test]
fn expand_tuple_type() {
let db = setup_db();
let int_ty = KnownClass::Int.to_instance(&db);
let str_ty = KnownClass::Str.to_instance(&db);
let bytes_ty = KnownClass::Bytes.to_instance(&db);
let bool_ty = KnownClass::Bool.to_instance(&db);
let true_ty = Type::BooleanLiteral(true);
let false_ty = Type::BooleanLiteral(false);
// Empty tuple
let empty_tuple = TupleType::empty(&db);
let expanded = expand_type(&db, empty_tuple);
assert!(expanded.is_none());
// None of the elements can be expanded.
let tuple_type1 = TupleType::from_elements(&db, [int_ty, str_ty]);
let expanded = expand_type(&db, tuple_type1);
assert!(expanded.is_none());
// All elements can be expanded.
let tuple_type2 = TupleType::from_elements(
&db,
[
bool_ty,
UnionType::from_elements(&db, [int_ty, str_ty, bytes_ty]),
],
);
let expected_types = [
TupleType::from_elements(&db, [true_ty, int_ty]),
TupleType::from_elements(&db, [true_ty, str_ty]),
TupleType::from_elements(&db, [true_ty, bytes_ty]),
TupleType::from_elements(&db, [false_ty, int_ty]),
TupleType::from_elements(&db, [false_ty, str_ty]),
TupleType::from_elements(&db, [false_ty, bytes_ty]),
];
let expanded = expand_type(&db, tuple_type2).unwrap();
assert_eq!(expanded.len(), expected_types.len());
assert_eq!(expanded, expected_types);
// Mixed set of elements where some can be expanded while others cannot be.
let tuple_type3 = TupleType::from_elements(
&db,
[
bool_ty,
int_ty,
UnionType::from_elements(&db, [str_ty, bytes_ty]),
str_ty,
],
);
let expected_types = [
TupleType::from_elements(&db, [true_ty, int_ty, str_ty, str_ty]),
TupleType::from_elements(&db, [true_ty, int_ty, bytes_ty, str_ty]),
TupleType::from_elements(&db, [false_ty, int_ty, str_ty, str_ty]),
TupleType::from_elements(&db, [false_ty, int_ty, bytes_ty, str_ty]),
];
let expanded = expand_type(&db, tuple_type3).unwrap();
assert_eq!(expanded.len(), expected_types.len());
assert_eq!(expanded, expected_types);
}
}

View File

@@ -12,19 +12,19 @@ use super::{
};
use crate::db::Db;
use crate::dunder_all::dunder_all_names;
use crate::symbol::{Boundness, Symbol};
use crate::place::{Boundness, Place};
use crate::types::diagnostic::{
CALL_NON_CALLABLE, CONFLICTING_ARGUMENT_FORMS, INVALID_ARGUMENT_TYPE, MISSING_ARGUMENT,
NO_MATCHING_OVERLOAD, PARAMETER_ALREADY_ASSIGNED, TOO_MANY_POSITIONAL_ARGUMENTS,
UNKNOWN_ARGUMENT,
};
use crate::types::function::{DataclassTransformerParams, FunctionDecorators, KnownFunction};
use crate::types::generics::{Specialization, SpecializationBuilder, SpecializationError};
use crate::types::signatures::{Parameter, ParameterForm};
use crate::types::{
BoundMethodType, DataclassParams, DataclassTransformerParams, FunctionDecorators, FunctionType,
KnownClass, KnownFunction, KnownInstanceType, MethodWrapperKind, PropertyInstanceType,
SpecialFormType, TupleType, TypeMapping, UnionType, WrapperDescriptorKind, ide_support,
todo_type,
BoundMethodType, ClassLiteral, DataclassParams, KnownClass, KnownInstanceType,
MethodWrapperKind, PropertyInstanceType, SpecialFormType, TupleType, TypeMapping, UnionType,
WrapperDescriptorKind, ide_support, todo_type,
};
use ruff_db::diagnostic::{Annotation, Diagnostic, Severity, SubDiagnostic};
use ruff_python_ast as ast;
@@ -770,7 +770,7 @@ impl<'db> Bindings<'db> {
// TODO: we could emit a diagnostic here (if default is not set)
overload.set_return_type(
match instance_ty.static_member(db, attr_name.value(db)) {
Symbol::Type(ty, Boundness::Bound) => {
Place::Type(ty, Boundness::Bound) => {
if instance_ty.is_fully_static(db) {
ty
} else {
@@ -782,10 +782,10 @@ impl<'db> Bindings<'db> {
union_with_default(ty)
}
}
Symbol::Type(ty, Boundness::PossiblyUnbound) => {
Place::Type(ty, Boundness::PossiblyUnbound) => {
union_with_default(ty)
}
Symbol::Unbound => default,
Place::Unbound => default,
},
);
}
@@ -839,6 +839,21 @@ impl<'db> Bindings<'db> {
overload.set_return_type(Type::DataclassDecorator(params));
}
// `dataclass` being used as a non-decorator
if let [Some(Type::ClassLiteral(class_literal))] =
overload.parameter_types()
{
let params = DataclassParams::default();
overload.set_return_type(Type::from(ClassLiteral::new(
db,
class_literal.name(db),
class_literal.body_scope(db),
class_literal.known(db),
Some(params),
class_literal.dataclass_transformer_params(db),
)));
}
}
Some(KnownFunction::DataclassTransform) => {
@@ -871,47 +886,47 @@ impl<'db> Bindings<'db> {
}
_ => {
let mut handle_dataclass_transformer_params =
|function_type: &FunctionType| {
if let Some(params) =
function_type.dataclass_transformer_params(db)
{
// This is a call to a custom function that was decorated with `@dataclass_transformer`.
// If this function was called with a keyword argument like `order=False`, we extract
// the argument type and overwrite the corresponding flag in `dataclass_params` after
// constructing them from the `dataclass_transformer`-parameter defaults.
let mut dataclass_params = DataclassParams::from(params);
if let Some(Some(Type::BooleanLiteral(order))) = overload
.signature
.parameters()
.keyword_by_name("order")
.map(|(idx, _)| idx)
.and_then(|idx| overload.parameter_types().get(idx))
{
dataclass_params.set(DataclassParams::ORDER, *order);
}
overload.set_return_type(Type::DataclassDecorator(
dataclass_params,
));
}
};
// Ideally, either the implementation, or exactly one of the overloads
// of the function can have the dataclass_transform decorator applied.
// However, we do not yet enforce this, and in the case of multiple
// applications of the decorator, we will only consider the last one
// for the return value, since the prior ones will be over-written.
if let Some(overloaded) = function_type.to_overloaded(db) {
overloaded
.overloads
.iter()
.for_each(&mut handle_dataclass_transformer_params);
}
let return_type = function_type
.iter_overloads_and_implementation(db)
.filter_map(|function_overload| {
function_overload.dataclass_transformer_params(db).map(
|params| {
// This is a call to a custom function that was decorated with `@dataclass_transformer`.
// If this function was called with a keyword argument like `order=False`, we extract
// the argument type and overwrite the corresponding flag in `dataclass_params` after
// constructing them from the `dataclass_transformer`-parameter defaults.
handle_dataclass_transformer_params(&function_type);
let mut dataclass_params =
DataclassParams::from(params);
if let Some(Some(Type::BooleanLiteral(order))) =
overload
.signature
.parameters()
.keyword_by_name("order")
.map(|(idx, _)| idx)
.and_then(|idx| {
overload.parameter_types().get(idx)
})
{
dataclass_params
.set(DataclassParams::ORDER, *order);
}
Type::DataclassDecorator(dataclass_params)
},
)
})
.last();
if let Some(return_type) = return_type {
overload.set_return_type(return_type);
}
}
},
@@ -997,6 +1012,7 @@ impl<'db> From<Binding<'db>> for Bindings<'db> {
signature_type,
dunder_call_is_possibly_unbound: false,
bound_type: None,
return_type: None,
overloads: smallvec![from],
};
Bindings {
@@ -1015,14 +1031,9 @@ impl<'db> From<Binding<'db>> for Bindings<'db> {
/// If the callable has multiple overloads, the first one that matches is used as the overall
/// binding match.
///
/// TODO: Implement the call site evaluation algorithm in the [proposed updated typing
/// spec][overloads], which is much more subtle than “first match wins”.
///
/// If the arguments cannot be matched to formal parameters, we store information about the
/// specific errors that occurred when trying to match them up. If the callable has multiple
/// overloads, we store this error information for each overload.
///
/// [overloads]: https://github.com/python/typing/pull/1839
#[derive(Debug)]
pub(crate) struct CallableBinding<'db> {
/// The type that is (hopefully) callable.
@@ -1040,6 +1051,14 @@ pub(crate) struct CallableBinding<'db> {
/// The type of the bound `self` or `cls` parameter if this signature is for a bound method.
pub(crate) bound_type: Option<Type<'db>>,
/// The return type of this callable.
///
/// This is only `Some` if it's an overloaded callable, "argument type expansion" was
/// performed, and one of the expansion evaluated successfully for all of the argument lists.
/// This type is then the union of all the return types of the matched overloads for the
/// expanded argument lists.
return_type: Option<Type<'db>>,
/// The bindings of each overload of this callable. Will be empty if the type is not callable.
///
/// By using `SmallVec`, we avoid an extra heap allocation for the common case of a
@@ -1061,6 +1080,7 @@ impl<'db> CallableBinding<'db> {
signature_type,
dunder_call_is_possibly_unbound: false,
bound_type: None,
return_type: None,
overloads,
}
}
@@ -1071,6 +1091,7 @@ impl<'db> CallableBinding<'db> {
signature_type,
dunder_call_is_possibly_unbound: false,
bound_type: None,
return_type: None,
overloads: smallvec![],
}
}
@@ -1099,12 +1120,6 @@ impl<'db> CallableBinding<'db> {
// before checking.
let arguments = arguments.with_self(self.bound_type);
// TODO: This checks every overload. In the proposed more detailed call checking spec [1],
// arguments are checked for arity first, and are only checked for type assignability against
// the matching overloads. Make sure to implement that as part of separating call binding into
// two phases.
//
// [1] https://typing.python.org/en/latest/spec/overload.html#overload-call-evaluation
for overload in &mut self.overloads {
overload.match_parameters(arguments.as_ref(), argument_forms, conflicting_forms);
}
@@ -1114,9 +1129,154 @@ impl<'db> CallableBinding<'db> {
// If this callable is a bound method, prepend the self instance onto the arguments list
// before checking.
let argument_types = argument_types.with_self(self.bound_type);
for overload in &mut self.overloads {
overload.check_types(db, argument_types.as_ref());
// Step 1: Check the result of the arity check which is done by `match_parameters`
let matching_overload_indexes = match self.matching_overload_index() {
MatchingOverloadIndex::None => {
// If no candidate overloads remain from the arity check, we can stop here. We
// still perform type checking for non-overloaded function to provide better user
// experience.
if let [overload] = self.overloads.as_mut_slice() {
overload.check_types(db, argument_types.as_ref(), argument_types.types());
}
return;
}
MatchingOverloadIndex::Single(index) => {
// If only one candidate overload remains, it is the winning match.
// TODO: Evaluate it as a regular (non-overloaded) call. This means that any
// diagnostics reported in this check should be reported directly instead of
// reporting it as `no-matching-overload`.
self.overloads[index].check_types(
db,
argument_types.as_ref(),
argument_types.types(),
);
return;
}
MatchingOverloadIndex::Multiple(indexes) => {
// If two or more candidate overloads remain, proceed to step 2.
indexes
}
};
let snapshotter = MatchingOverloadsSnapshotter::new(matching_overload_indexes);
// State of the bindings _before_ evaluating (type checking) the matching overloads using
// the non-expanded argument types.
let pre_evaluation_snapshot = snapshotter.take(self);
// Step 2: Evaluate each remaining overload as a regular (non-overloaded) call to determine
// whether it is compatible with the supplied argument list.
for (_, overload) in self.matching_overloads_mut() {
overload.check_types(db, argument_types.as_ref(), argument_types.types());
}
match self.matching_overload_index() {
MatchingOverloadIndex::None => {
// If all overloads result in errors, proceed to step 3.
}
MatchingOverloadIndex::Single(_) => {
// If only one overload evaluates without error, it is the winning match.
return;
}
MatchingOverloadIndex::Multiple(_) => {
// If two or more candidate overloads remain, proceed to step 4.
// TODO: Step 4 and Step 5 goes here...
// We're returning here because this shouldn't lead to argument type expansion.
return;
}
}
// Step 3: Perform "argument type expansion". Reference:
// https://typing.python.org/en/latest/spec/overload.html#argument-type-expansion
let mut expansions = argument_types.expand(db).peekable();
if expansions.peek().is_none() {
// Return early if there are no argument types to expand.
return;
}
// State of the bindings _after_ evaluating (type checking) the matching overloads using
// the non-expanded argument types.
let post_evaluation_snapshot = snapshotter.take(self);
// Restore the bindings state to the one prior to the type checking step in preparation
// for evaluating the expanded argument lists.
snapshotter.restore(self, pre_evaluation_snapshot);
for expanded_argument_lists in expansions {
// This is the merged state of the bindings after evaluating all of the expanded
// argument lists. This will be the final state to restore the bindings to if all of
// the expanded argument lists evaluated successfully.
let mut merged_evaluation_state: Option<MatchingOverloadsSnapshot<'db>> = None;
let mut return_types = Vec::new();
for expanded_argument_types in &expanded_argument_lists {
let pre_evaluation_snapshot = snapshotter.take(self);
for (_, overload) in self.matching_overloads_mut() {
overload.check_types(db, argument_types.as_ref(), expanded_argument_types);
}
let return_type = match self.matching_overload_index() {
MatchingOverloadIndex::None => None,
MatchingOverloadIndex::Single(index) => {
Some(self.overloads[index].return_type())
}
MatchingOverloadIndex::Multiple(index) => {
// TODO: Step 4 and Step 5 goes here... but for now we just use the return
// type of the first matched overload.
Some(self.overloads[index[0]].return_type())
}
};
// This split between initializing and updating the merged evaluation state is
// required because otherwise it's difficult to differentiate between the
// following:
// 1. An initial unmatched overload becomes a matched overload when evaluating the
// first argument list
// 2. An unmatched overload after evaluating the first argument list becomes a
// matched overload when evaluating the second argument list
if let Some(merged_evaluation_state) = merged_evaluation_state.as_mut() {
merged_evaluation_state.update(self);
} else {
merged_evaluation_state = Some(snapshotter.take(self));
}
// Restore the bindings state before evaluating the next argument list.
snapshotter.restore(self, pre_evaluation_snapshot);
if let Some(return_type) = return_type {
return_types.push(return_type);
} else {
// No need to check the remaining argument lists if the current argument list
// doesn't evaluate successfully. Move on to expanding the next argument type.
break;
}
}
if return_types.len() == expanded_argument_lists.len() {
// If the number of return types is equal to the number of expanded argument lists,
// they all evaluated successfully. So, we need to combine their return types by
// union to determine the final return type.
self.return_type = Some(UnionType::from_elements(db, return_types));
// Restore the bindings state to the one that merges the bindings state evaluating
// each of the expanded argument list.
if let Some(merged_evaluation_state) = merged_evaluation_state {
snapshotter.restore(self, merged_evaluation_state);
}
return;
}
}
// If the type expansion didn't yield any successful return type, we need to restore the
// bindings state back to the one after the type checking step using the non-expanded
// argument types. This is necessary because we restore the state to the pre-evaluation
// snapshot when processing the expanded argument lists.
snapshotter.restore(self, post_evaluation_snapshot);
}
fn as_result(&self) -> Result<(), CallErrorKind> {
@@ -1145,6 +1305,25 @@ impl<'db> CallableBinding<'db> {
self.matching_overloads().next().is_none()
}
/// Returns the index of the matching overload in the form of [`MatchingOverloadIndex`].
fn matching_overload_index(&self) -> MatchingOverloadIndex {
let mut matching_overloads = self.matching_overloads();
match matching_overloads.next() {
None => MatchingOverloadIndex::None,
Some((first, _)) => {
if let Some((second, _)) = matching_overloads.next() {
let mut indexes = vec![first, second];
for (index, _) in matching_overloads {
indexes.push(index);
}
MatchingOverloadIndex::Multiple(indexes)
} else {
MatchingOverloadIndex::Single(first)
}
}
}
}
/// Returns an iterator over all the overloads that matched for this call binding.
pub(crate) fn matching_overloads(&self) -> impl Iterator<Item = (usize, &Binding<'db>)> {
self.overloads
@@ -1163,16 +1342,20 @@ impl<'db> CallableBinding<'db> {
.filter(|(_, overload)| overload.as_result().is_ok())
}
/// Returns the return type of this call. For a valid call, this is the return type of the
/// first overload that the arguments matched against. For an invalid call to a non-overloaded
/// function, this is the return type of the function. For an invalid call to an overloaded
/// function, we return `Type::unknown`, since we cannot make any useful conclusions about
/// which overload was intended to be called.
/// Returns the return type of this call.
///
/// For a valid call, this is the return type of either a successful argument type expansion of
/// an overloaded function, or the return type of the first overload that the arguments matched
/// against.
///
/// For an invalid call to a non-overloaded function, this is the return type of the function.
///
/// For an invalid call to an overloaded function, we return `Type::unknown`, since we cannot
/// make any useful conclusions about which overload was intended to be called.
pub(crate) fn return_type(&self) -> Type<'db> {
// TODO: Implement the overload call evaluation algorithm as mentioned in the spec [1] to
// get the matching overload and use that to get the return type.
//
// [1]: https://typing.python.org/en/latest/spec/overload.html#overload-call-evaluation
if let Some(return_type) = self.return_type {
return return_type;
}
if let Some((_, first_overload)) = self.matching_overloads().next() {
return first_overload.return_type();
}
@@ -1261,47 +1444,49 @@ impl<'db> CallableBinding<'db> {
_ => None,
};
if let Some((kind, function)) = function_type_and_kind {
if let Some(overloaded_function) = function.to_overloaded(context.db()) {
if let Some(spans) = overloaded_function
.overloads
.first()
.and_then(|overload| overload.spans(context.db()))
{
let mut sub =
SubDiagnostic::new(Severity::Info, "First overload defined here");
sub.annotate(Annotation::primary(spans.signature));
diag.sub(sub);
}
let (overloads, implementation) =
function.overloads_and_implementation(context.db());
if let Some(spans) = overloads
.first()
.and_then(|overload| overload.spans(context.db()))
{
let mut sub =
SubDiagnostic::new(Severity::Info, "First overload defined here");
sub.annotate(Annotation::primary(spans.signature));
diag.sub(sub);
}
diag.info(format_args!(
"Possible overloads for {kind} `{}`:",
function.name(context.db())
));
for overload in overloads.iter().take(MAXIMUM_OVERLOADS) {
diag.info(format_args!(
"Possible overloads for {kind} `{}`:",
function.name(context.db())
" {}",
overload.signature(context.db(), None).display(context.db())
));
}
if overloads.len() > MAXIMUM_OVERLOADS {
diag.info(format_args!(
"... omitted {remaining} overloads",
remaining = overloads.len() - MAXIMUM_OVERLOADS
));
}
let overloads = &function.signature(context.db()).overloads.overloads;
for overload in overloads.iter().take(MAXIMUM_OVERLOADS) {
diag.info(format_args!(" {}", overload.display(context.db())));
}
if overloads.len() > MAXIMUM_OVERLOADS {
diag.info(format_args!(
"... omitted {remaining} overloads",
remaining = overloads.len() - MAXIMUM_OVERLOADS
));
}
if let Some(spans) = overloaded_function
.implementation
.and_then(|function| function.spans(context.db()))
{
let mut sub = SubDiagnostic::new(
Severity::Info,
"Overload implementation defined here",
);
sub.annotate(Annotation::primary(spans.signature));
diag.sub(sub);
}
if let Some(spans) =
implementation.and_then(|function| function.spans(context.db()))
{
let mut sub = SubDiagnostic::new(
Severity::Info,
"Overload implementation defined here",
);
sub.annotate(Annotation::primary(spans.signature));
diag.sub(sub);
}
}
if let Some(union_diag) = union_diag {
union_diag.add_union_context(context.db(), &mut diag);
}
@@ -1319,6 +1504,18 @@ impl<'a, 'db> IntoIterator for &'a CallableBinding<'db> {
}
}
#[derive(Debug)]
enum MatchingOverloadIndex {
/// No matching overloads found.
None,
/// Exactly one matching overload found at the given index.
Single(usize),
/// Multiple matching overloads found at the given indexes.
Multiple(Vec<usize>),
}
/// Binding information for one of the overloads of a callable.
#[derive(Debug)]
pub(crate) struct Binding<'db> {
@@ -1493,7 +1690,12 @@ impl<'db> Binding<'db> {
self.parameter_tys = vec![None; parameters.len()].into_boxed_slice();
}
fn check_types(&mut self, db: &'db dyn Db, argument_types: &CallArgumentTypes<'_, 'db>) {
fn check_types(
&mut self,
db: &'db dyn Db,
arguments: &CallArguments<'_>,
argument_types: &[Type<'db>],
) {
let mut num_synthetic_args = 0;
let get_argument_index = |argument_index: usize, num_synthetic_args: usize| {
if argument_index >= num_synthetic_args {
@@ -1507,13 +1709,20 @@ impl<'db> Binding<'db> {
}
};
let enumerate_argument_types = || {
arguments
.iter()
.zip(argument_types.iter().copied())
.enumerate()
};
// If this overload is generic, first see if we can infer a specialization of the function
// from the arguments that were passed in.
let signature = &self.signature;
let parameters = signature.parameters();
if signature.generic_context.is_some() || signature.inherited_generic_context.is_some() {
let mut builder = SpecializationBuilder::new(db);
for (argument_index, (argument, argument_type)) in argument_types.iter().enumerate() {
for (argument_index, (argument, argument_type)) in enumerate_argument_types() {
if matches!(argument, Argument::Synthetic) {
num_synthetic_args += 1;
}
@@ -1545,7 +1754,7 @@ impl<'db> Binding<'db> {
}
num_synthetic_args = 0;
for (argument_index, (argument, mut argument_type)) in argument_types.iter().enumerate() {
for (argument_index, (argument, mut argument_type)) in enumerate_argument_types() {
if matches!(argument, Argument::Synthetic) {
num_synthetic_args += 1;
}
@@ -1648,6 +1857,133 @@ impl<'db> Binding<'db> {
}
Ok(())
}
fn snapshot(&self) -> BindingSnapshot<'db> {
BindingSnapshot {
return_ty: self.return_ty,
specialization: self.specialization,
inherited_specialization: self.inherited_specialization,
argument_parameters: self.argument_parameters.clone(),
parameter_tys: self.parameter_tys.clone(),
errors: self.errors.clone(),
}
}
fn restore(&mut self, snapshot: BindingSnapshot<'db>) {
let BindingSnapshot {
return_ty,
specialization,
inherited_specialization,
argument_parameters,
parameter_tys,
errors,
} = snapshot;
self.return_ty = return_ty;
self.specialization = specialization;
self.inherited_specialization = inherited_specialization;
self.argument_parameters = argument_parameters;
self.parameter_tys = parameter_tys;
self.errors = errors;
}
}
#[derive(Clone, Debug)]
struct BindingSnapshot<'db> {
return_ty: Type<'db>,
specialization: Option<Specialization<'db>>,
inherited_specialization: Option<Specialization<'db>>,
argument_parameters: Box<[Option<usize>]>,
parameter_tys: Box<[Option<Type<'db>>]>,
errors: Vec<BindingError<'db>>,
}
/// Represents the snapshot of the matched overload bindings.
///
/// The reason that this only contains the matched overloads are:
/// 1. Avoid creating snapshots for the overloads that have been filtered by the arity check
/// 2. Avoid duplicating errors when merging the snapshots on a successful evaluation of all the
/// expanded argument lists
#[derive(Clone, Debug)]
struct MatchingOverloadsSnapshot<'db>(Vec<(usize, BindingSnapshot<'db>)>);
impl<'db> MatchingOverloadsSnapshot<'db> {
/// Update the state of the matched overload bindings in this snapshot with the current
/// state in the given `binding`.
fn update(&mut self, binding: &CallableBinding<'db>) {
// Here, the `snapshot` is the state of this binding for the previous argument list and
// `binding` would contain the state after evaluating the current argument list.
for (snapshot, binding) in self
.0
.iter_mut()
.map(|(index, snapshot)| (snapshot, &binding.overloads[*index]))
{
if binding.errors.is_empty() {
// If the binding has no errors, this means that the current argument list was
// evaluated successfully and this is the matching overload.
//
// Clear the errors from the snapshot of this overload to signal this change ...
snapshot.errors.clear();
// ... and update the snapshot with the current state of the binding.
snapshot.return_ty = binding.return_ty;
snapshot.specialization = binding.specialization;
snapshot.inherited_specialization = binding.inherited_specialization;
snapshot
.argument_parameters
.clone_from(&binding.argument_parameters);
snapshot.parameter_tys.clone_from(&binding.parameter_tys);
}
// If the errors in the snapshot was empty, then this binding is the matching overload
// for a previously evaluated argument list. This means that we don't need to change
// any information for an already matched overload binding.
//
// If it does have errors, we could extend it with the errors from evaluating the
// current argument list. Arguably, this isn't required, since the errors in the
// snapshot should already signal that this is an unmatched overload which is why we
// don't do it. Similarly, due to this being an unmatched overload, there's no point in
// updating the binding state.
}
}
}
/// A helper to take snapshots of the matched overload bindings for the current state of the
/// bindings.
struct MatchingOverloadsSnapshotter(Vec<usize>);
impl MatchingOverloadsSnapshotter {
/// Creates a new snapshotter for the given indexes of the matched overloads.
fn new(indexes: Vec<usize>) -> Self {
debug_assert!(indexes.len() > 1);
MatchingOverloadsSnapshotter(indexes)
}
/// Takes a snapshot of the current state of the matched overload bindings.
///
/// # Panics
///
/// Panics if the indexes of the matched overloads are not valid for the given binding.
fn take<'db>(&self, binding: &CallableBinding<'db>) -> MatchingOverloadsSnapshot<'db> {
MatchingOverloadsSnapshot(
self.0
.iter()
.map(|index| (*index, binding.overloads[*index].snapshot()))
.collect(),
)
}
/// Restores the state of the matched overload bindings from the given snapshot.
fn restore<'db>(
&self,
binding: &mut CallableBinding<'db>,
snapshot: MatchingOverloadsSnapshot<'db>,
) {
debug_assert_eq!(self.0.len(), snapshot.0.len());
for (index, snapshot) in snapshot.0 {
binding.overloads[index].restore(snapshot);
}
}
}
/// Describes a callable for the purposes of diagnostics.

View File

@@ -2,32 +2,31 @@ use std::hash::BuildHasherDefault;
use std::sync::{LazyLock, Mutex};
use super::{
IntersectionBuilder, KnownFunction, MemberLookupPolicy, Mro, MroError, MroIterator,
SpecialFormType, SubclassOfType, Truthiness, Type, TypeQualifiers, class_base::ClassBase,
infer_expression_type, infer_unpack_types,
IntersectionBuilder, MemberLookupPolicy, Mro, MroError, MroIterator, SpecialFormType,
SubclassOfType, Truthiness, Type, TypeQualifiers, class_base::ClassBase, infer_expression_type,
infer_unpack_types,
};
use crate::semantic_index::DeclarationWithConstraint;
use crate::semantic_index::definition::Definition;
use crate::semantic_index::definition::{Definition, DefinitionState};
use crate::types::function::{DataclassTransformerParams, KnownFunction};
use crate::types::generics::{GenericContext, Specialization};
use crate::types::signatures::{CallableSignature, Parameter, Parameters, Signature};
use crate::types::{
CallableType, DataclassParams, DataclassTransformerParams, KnownInstanceType, TypeMapping,
TypeVarInstance,
CallableType, DataclassParams, KnownInstanceType, TypeMapping, TypeVarInstance,
};
use crate::{
Db, FxOrderSet, KnownModule, Program,
module_resolver::file_to_module,
place::{
Boundness, LookupError, LookupResult, Place, PlaceAndQualifiers, class_symbol,
known_module_symbol, place_from_bindings, place_from_declarations,
},
semantic_index::{
ast_ids::HasScopedExpressionId,
attribute_assignments,
definition::{DefinitionKind, TargetKind},
semantic_index,
symbol::ScopeId,
symbol_table, use_def_map,
},
symbol::{
Boundness, LookupError, LookupResult, Symbol, SymbolAndQualifiers, class_symbol,
known_module_symbol, symbol_from_bindings, symbol_from_declarations,
place::ScopeId,
place_table, semantic_index, use_def_map,
},
types::{
CallArgumentTypes, CallError, CallErrorKind, DynamicType, MetaclassCandidate, TupleType,
@@ -454,7 +453,7 @@ impl<'db> ClassType<'db> {
db: &'db dyn Db,
name: &str,
policy: MemberLookupPolicy,
) -> SymbolAndQualifiers<'db> {
) -> PlaceAndQualifiers<'db> {
let (class_literal, specialization) = self.class_literal(db);
class_literal.class_member_inner(db, specialization, name, policy)
}
@@ -462,23 +461,20 @@ impl<'db> ClassType<'db> {
/// Returns the inferred type of the class member named `name`. Only bound members
/// or those marked as ClassVars are considered.
///
/// Returns [`Symbol::Unbound`] if `name` cannot be found in this class's scope
/// Returns [`Place::Unbound`] if `name` cannot be found in this class's scope
/// directly. Use [`ClassType::class_member`] if you require a method that will
/// traverse through the MRO until it finds the member.
pub(super) fn own_class_member(self, db: &'db dyn Db, name: &str) -> SymbolAndQualifiers<'db> {
pub(super) fn own_class_member(self, db: &'db dyn Db, name: &str) -> PlaceAndQualifiers<'db> {
let (class_literal, specialization) = self.class_literal(db);
class_literal
.own_class_member(db, specialization, name)
.map_type(|ty| ty.apply_optional_specialization(db, specialization))
}
/// Returns the `name` attribute of an instance of this class.
/// Look up an instance attribute (available in `__dict__`) of the given name.
///
/// The attribute could be defined in the class body, but it could also be an implicitly
/// defined attribute that is only present in a method (typically `__init__`).
///
/// The attribute might also be defined in a superclass of this class.
pub(super) fn instance_member(self, db: &'db dyn Db, name: &str) -> SymbolAndQualifiers<'db> {
/// See [`Type::instance_member`] for more details.
pub(super) fn instance_member(self, db: &'db dyn Db, name: &str) -> PlaceAndQualifiers<'db> {
let (class_literal, specialization) = self.class_literal(db);
class_literal
.instance_member(db, specialization, name)
@@ -487,7 +483,7 @@ impl<'db> ClassType<'db> {
/// A helper function for `instance_member` that looks up the `name` attribute only on
/// this class, not on its superclasses.
fn own_instance_member(self, db: &'db dyn Db, name: &str) -> SymbolAndQualifiers<'db> {
fn own_instance_member(self, db: &'db dyn Db, name: &str) -> PlaceAndQualifiers<'db> {
let (class_literal, specialization) = self.class_literal(db);
class_literal
.own_instance_member(db, name)
@@ -505,9 +501,9 @@ impl<'db> ClassType<'db> {
MemberLookupPolicy::NO_INSTANCE_FALLBACK
| MemberLookupPolicy::META_CLASS_NO_TYPE_FALLBACK,
)
.symbol;
.place;
if let Symbol::Type(Type::BoundMethod(metaclass_dunder_call_function), _) =
if let Place::Type(Type::BoundMethod(metaclass_dunder_call_function), _) =
metaclass_dunder_call_function_symbol
{
// TODO: this intentionally diverges from step 1 in
@@ -523,10 +519,10 @@ impl<'db> ClassType<'db> {
"__new__".into(),
MemberLookupPolicy::MRO_NO_OBJECT_FALLBACK,
)
.symbol;
.place;
let dunder_new_function =
if let Symbol::Type(Type::FunctionLiteral(dunder_new_function), _) =
if let Place::Type(Type::FunctionLiteral(dunder_new_function), _) =
dunder_new_function_symbol
{
// Step 3: If the return type of the `__new__` evaluates to a type that is not a subclass of this class,
@@ -565,7 +561,7 @@ impl<'db> ClassType<'db> {
MemberLookupPolicy::MRO_NO_OBJECT_FALLBACK
| MemberLookupPolicy::META_CLASS_NO_TYPE_FALLBACK,
)
.symbol;
.place;
let correct_return_type = self_ty.to_instance(db).unwrap_or_else(Type::unknown);
@@ -573,7 +569,7 @@ impl<'db> ClassType<'db> {
// same parameters as the `__init__` method after it is bound, and with the return type of
// the concrete type of `Self`.
let synthesized_dunder_init_callable =
if let Symbol::Type(Type::FunctionLiteral(dunder_init_function), _) =
if let Place::Type(Type::FunctionLiteral(dunder_init_function), _) =
dunder_init_function_symbol
{
let synthesized_signature = |signature: Signature<'db>| {
@@ -615,9 +611,9 @@ impl<'db> ClassType<'db> {
"__new__".into(),
MemberLookupPolicy::META_CLASS_NO_TYPE_FALLBACK,
)
.symbol;
.place;
if let Symbol::Type(Type::FunctionLiteral(new_function), _) = new_function_symbol {
if let Place::Type(Type::FunctionLiteral(new_function), _) = new_function_symbol {
new_function.into_bound_method_type(db, self_ty)
} else {
// Fallback if no `object.__new__` is found.
@@ -1139,7 +1135,7 @@ impl<'db> ClassLiteral<'db> {
db: &'db dyn Db,
name: &str,
policy: MemberLookupPolicy,
) -> SymbolAndQualifiers<'db> {
) -> PlaceAndQualifiers<'db> {
self.class_member_inner(db, None, name, policy)
}
@@ -1149,10 +1145,10 @@ impl<'db> ClassLiteral<'db> {
specialization: Option<Specialization<'db>>,
name: &str,
policy: MemberLookupPolicy,
) -> SymbolAndQualifiers<'db> {
) -> PlaceAndQualifiers<'db> {
if name == "__mro__" {
let tuple_elements = self.iter_mro(db, specialization).map(Type::from);
return Symbol::bound(TupleType::from_elements(db, tuple_elements)).into();
return Place::bound(TupleType::from_elements(db, tuple_elements)).into();
}
self.class_member_from_mro(db, name, policy, self.iter_mro(db, specialization))
@@ -1164,7 +1160,7 @@ impl<'db> ClassLiteral<'db> {
name: &str,
policy: MemberLookupPolicy,
mro_iter: impl Iterator<Item = ClassBase<'db>>,
) -> SymbolAndQualifiers<'db> {
) -> PlaceAndQualifiers<'db> {
// If we encounter a dynamic type in this class's MRO, we'll save that dynamic type
// in this variable. After we've traversed the MRO, we'll either:
// (1) Use that dynamic type as the type for this attribute,
@@ -1211,18 +1207,18 @@ impl<'db> ClassLiteral<'db> {
}
match (
SymbolAndQualifiers::from(lookup_result),
PlaceAndQualifiers::from(lookup_result),
dynamic_type_to_intersect_with,
) {
(symbol_and_qualifiers, None) => symbol_and_qualifiers,
(
SymbolAndQualifiers {
symbol: Symbol::Type(ty, _),
PlaceAndQualifiers {
place: Place::Type(ty, _),
qualifiers,
},
Some(dynamic_type),
) => Symbol::bound(
) => Place::bound(
IntersectionBuilder::new(db)
.add_positive(ty)
.add_positive(dynamic_type)
@@ -1231,19 +1227,19 @@ impl<'db> ClassLiteral<'db> {
.with_qualifiers(qualifiers),
(
SymbolAndQualifiers {
symbol: Symbol::Unbound,
PlaceAndQualifiers {
place: Place::Unbound,
qualifiers,
},
Some(dynamic_type),
) => Symbol::bound(dynamic_type).with_qualifiers(qualifiers),
) => Place::bound(dynamic_type).with_qualifiers(qualifiers),
}
}
/// Returns the inferred type of the class member named `name`. Only bound members
/// or those marked as ClassVars are considered.
///
/// Returns [`Symbol::Unbound`] if `name` cannot be found in this class's scope
/// Returns [`Place::Unbound`] if `name` cannot be found in this class's scope
/// directly. Use [`ClassLiteral::class_member`] if you require a method that will
/// traverse through the MRO until it finds the member.
pub(super) fn own_class_member(
@@ -1251,10 +1247,10 @@ impl<'db> ClassLiteral<'db> {
db: &'db dyn Db,
specialization: Option<Specialization<'db>>,
name: &str,
) -> SymbolAndQualifiers<'db> {
) -> PlaceAndQualifiers<'db> {
if name == "__dataclass_fields__" && self.dataclass_params(db).is_some() {
// Make this class look like a subclass of the `DataClassInstance` protocol
return Symbol::bound(KnownClass::Dict.to_specialized_instance(
return Place::bound(KnownClass::Dict.to_specialized_instance(
db,
[
KnownClass::Str.to_instance(db),
@@ -1290,10 +1286,10 @@ impl<'db> ClassLiteral<'db> {
}
});
if symbol.symbol.is_unbound() {
if symbol.place.is_unbound() {
if let Some(synthesized_member) = self.own_synthesized_member(db, specialization, name)
{
return Symbol::bound(synthesized_member).into();
return Place::bound(synthesized_member).into();
}
}
@@ -1325,7 +1321,7 @@ impl<'db> ClassLiteral<'db> {
// itself in this case, so we skip the special descriptor handling.
if attr_ty.is_fully_static(db) {
let dunder_set = attr_ty.class_member(db, "__set__".into());
if let Some(dunder_set) = dunder_set.symbol.ignore_possibly_unbound() {
if let Some(dunder_set) = dunder_set.place.ignore_possibly_unbound() {
// This type of this attribute is a data descriptor. Instead of overwriting the
// descriptor attribute, data-classes will (implicitly) call the `__set__` method
// of the descriptor. This means that the synthesized `__init__` parameter for
@@ -1372,7 +1368,8 @@ impl<'db> ClassLiteral<'db> {
parameters.push(parameter);
}
let signature = Signature::new(Parameters::new(parameters), Some(Type::none(db)));
let mut signature = Signature::new(Parameters::new(parameters), Some(Type::none(db)));
signature.inherited_generic_context = self.generic_context(db);
Some(CallableType::function_like(db, signature))
};
@@ -1430,7 +1427,7 @@ impl<'db> ClassLiteral<'db> {
.to_class_literal(db)
.into_class_literal()?
.own_class_member(db, None, name)
.symbol
.place
.ignore_possibly_unbound()
}
_ => None,
@@ -1492,10 +1489,10 @@ impl<'db> ClassLiteral<'db> {
let mut attributes = FxOrderMap::default();
let class_body_scope = self.body_scope(db);
let table = symbol_table(db, class_body_scope);
let table = place_table(db, class_body_scope);
let use_def = use_def_map(db, class_body_scope);
for (symbol_id, declarations) in use_def.all_public_declarations() {
for (place_id, declarations) in use_def.all_public_declarations() {
// Here, we exclude all declarations that are not annotated assignments. We need this because
// things like function definitions and nested classes would otherwise be considered dataclass
// fields. The check is too broad in the sense that it also excludes (weird) constructs where
@@ -1506,7 +1503,7 @@ impl<'db> ClassLiteral<'db> {
if !declarations
.clone()
.all(|DeclarationWithConstraint { declaration, .. }| {
declaration.is_some_and(|declaration| {
declaration.is_defined_and(|declaration| {
matches!(
declaration.kind(db),
DefinitionKind::AnnotatedAssignment(..)
@@ -1517,18 +1514,18 @@ impl<'db> ClassLiteral<'db> {
continue;
}
let symbol = table.symbol(symbol_id);
let place_expr = table.place_expr(place_id);
if let Ok(attr) = symbol_from_declarations(db, declarations) {
if let Ok(attr) = place_from_declarations(db, declarations) {
if attr.is_class_var() {
continue;
}
if let Some(attr_ty) = attr.symbol.ignore_possibly_unbound() {
let bindings = use_def.public_bindings(symbol_id);
let default_ty = symbol_from_bindings(db, bindings).ignore_possibly_unbound();
if let Some(attr_ty) = attr.place.ignore_possibly_unbound() {
let bindings = use_def.public_bindings(place_id);
let default_ty = place_from_bindings(db, bindings).ignore_possibly_unbound();
attributes.insert(symbol.name().clone(), (attr_ty, default_ty));
attributes.insert(place_expr.expect_name().clone(), (attr_ty, default_ty));
}
}
}
@@ -1536,18 +1533,15 @@ impl<'db> ClassLiteral<'db> {
attributes
}
/// Returns the `name` attribute of an instance of this class.
/// Look up an instance attribute (available in `__dict__`) of the given name.
///
/// The attribute could be defined in the class body, but it could also be an implicitly
/// defined attribute that is only present in a method (typically `__init__`).
///
/// The attribute might also be defined in a superclass of this class.
/// See [`Type::instance_member`] for more details.
pub(super) fn instance_member(
self,
db: &'db dyn Db,
specialization: Option<Specialization<'db>>,
name: &str,
) -> SymbolAndQualifiers<'db> {
) -> PlaceAndQualifiers<'db> {
let mut union = UnionBuilder::new(db);
let mut union_qualifiers = TypeQualifiers::empty();
@@ -1557,13 +1551,13 @@ impl<'db> ClassLiteral<'db> {
// Skip over these very special class bases that aren't really classes.
}
ClassBase::Dynamic(_) => {
return SymbolAndQualifiers::todo(
return PlaceAndQualifiers::todo(
"instance attribute on class with dynamic base",
);
}
ClassBase::Class(class) => {
if let member @ SymbolAndQualifiers {
symbol: Symbol::Type(ty, boundness),
if let member @ PlaceAndQualifiers {
place: Place::Type(ty, boundness),
qualifiers,
} = class.own_instance_member(db, name)
{
@@ -1576,7 +1570,7 @@ impl<'db> ClassLiteral<'db> {
return member;
}
return Symbol::bound(union.add(ty).build())
return Place::bound(union.add(ty).build())
.with_qualifiers(union_qualifiers);
}
@@ -1589,13 +1583,12 @@ impl<'db> ClassLiteral<'db> {
}
if union.is_empty() {
Symbol::Unbound.with_qualifiers(TypeQualifiers::empty())
Place::Unbound.with_qualifiers(TypeQualifiers::empty())
} else {
// If we have reached this point, we know that we have only seen possibly-unbound symbols.
// If we have reached this point, we know that we have only seen possibly-unbound places.
// This means that the final result is still possibly-unbound.
Symbol::Type(union.build(), Boundness::PossiblyUnbound)
.with_qualifiers(union_qualifiers)
Place::Type(union.build(), Boundness::PossiblyUnbound).with_qualifiers(union_qualifiers)
}
}
@@ -1605,7 +1598,7 @@ impl<'db> ClassLiteral<'db> {
db: &'db dyn Db,
class_body_scope: ScopeId<'db>,
name: &str,
) -> Symbol<'db> {
) -> Place<'db> {
// If we do not see any declarations of an attribute, neither in the class body nor in
// any method, we build a union of `Unknown` with the inferred types of all bindings of
// that attribute. We include `Unknown` in that union to account for the fact that the
@@ -1617,7 +1610,7 @@ impl<'db> ClassLiteral<'db> {
let file = class_body_scope.file(db);
let index = semantic_index(db, file);
let class_map = use_def_map(db, class_body_scope);
let class_table = symbol_table(db, class_body_scope);
let class_table = place_table(db, class_body_scope);
for (attribute_assignments, method_scope_id) in
attribute_assignments(db, class_body_scope, name)
@@ -1628,11 +1621,11 @@ impl<'db> ClassLiteral<'db> {
// The attribute assignment inherits the visibility of the method which contains it
let is_method_visible = if let Some(method_def) = method_scope.node(db).as_function() {
let method = index.expect_single_definition(method_def);
let method_symbol = class_table.symbol_id_by_name(&method_def.name).unwrap();
let method_place = class_table.place_id_by_name(&method_def.name).unwrap();
class_map
.public_bindings(method_symbol)
.public_bindings(method_place)
.find_map(|bind| {
(bind.binding == Some(method))
(bind.binding.is_defined_and(|def| def == method))
.then(|| class_map.is_binding_visible(db, &bind))
})
.unwrap_or(Truthiness::AlwaysFalse)
@@ -1647,7 +1640,7 @@ impl<'db> ClassLiteral<'db> {
let unbound_visibility = attribute_assignments
.peek()
.map(|attribute_assignment| {
if attribute_assignment.binding.is_none() {
if attribute_assignment.binding.is_undefined() {
method_map.is_binding_visible(db, attribute_assignment)
} else {
Truthiness::AlwaysFalse
@@ -1656,7 +1649,7 @@ impl<'db> ClassLiteral<'db> {
.unwrap_or(Truthiness::AlwaysFalse);
for attribute_assignment in attribute_assignments {
let Some(binding) = attribute_assignment.binding else {
let DefinitionState::Defined(binding) = attribute_assignment.binding else {
continue;
};
match method_map
@@ -1701,10 +1694,10 @@ impl<'db> ClassLiteral<'db> {
// TODO: check if there are conflicting declarations
match is_attribute_bound {
Truthiness::AlwaysTrue => {
return Symbol::bound(annotation_ty);
return Place::bound(annotation_ty);
}
Truthiness::Ambiguous => {
return Symbol::possibly_unbound(annotation_ty);
return Place::possibly_unbound(annotation_ty);
}
Truthiness::AlwaysFalse => unreachable!(
"If the attribute assignments are all invisible, inference of their types should be skipped"
@@ -1727,7 +1720,7 @@ impl<'db> ClassLiteral<'db> {
union_of_inferred_types = union_of_inferred_types.add(inferred_ty);
}
TargetKind::NameOrAttribute => {
TargetKind::Single => {
// We found an un-annotated attribute assignment of the form:
//
// self.name = <value>
@@ -1753,7 +1746,7 @@ impl<'db> ClassLiteral<'db> {
union_of_inferred_types = union_of_inferred_types.add(inferred_ty);
}
TargetKind::NameOrAttribute => {
TargetKind::Single => {
// We found an attribute assignment like:
//
// for self.name in <iterable>:
@@ -1783,7 +1776,7 @@ impl<'db> ClassLiteral<'db> {
union_of_inferred_types = union_of_inferred_types.add(inferred_ty);
}
TargetKind::NameOrAttribute => {
TargetKind::Single => {
// We found an attribute assignment like:
//
// with <context_manager> as self.name:
@@ -1813,7 +1806,7 @@ impl<'db> ClassLiteral<'db> {
union_of_inferred_types = union_of_inferred_types.add(inferred_ty);
}
TargetKind::NameOrAttribute => {
TargetKind::Single => {
// We found an attribute assignment like:
//
// [... for self.name in <iterable>]
@@ -1841,42 +1834,42 @@ impl<'db> ClassLiteral<'db> {
}
match is_attribute_bound {
Truthiness::AlwaysTrue => Symbol::bound(union_of_inferred_types.build()),
Truthiness::Ambiguous => Symbol::possibly_unbound(union_of_inferred_types.build()),
Truthiness::AlwaysFalse => Symbol::Unbound,
Truthiness::AlwaysTrue => Place::bound(union_of_inferred_types.build()),
Truthiness::Ambiguous => Place::possibly_unbound(union_of_inferred_types.build()),
Truthiness::AlwaysFalse => Place::Unbound,
}
}
/// A helper function for `instance_member` that looks up the `name` attribute only on
/// this class, not on its superclasses.
fn own_instance_member(self, db: &'db dyn Db, name: &str) -> SymbolAndQualifiers<'db> {
fn own_instance_member(self, db: &'db dyn Db, name: &str) -> PlaceAndQualifiers<'db> {
// TODO: There are many things that are not yet implemented here:
// - `typing.Final`
// - Proper diagnostics
let body_scope = self.body_scope(db);
let table = symbol_table(db, body_scope);
let table = place_table(db, body_scope);
if let Some(symbol_id) = table.symbol_id_by_name(name) {
if let Some(place_id) = table.place_id_by_name(name) {
let use_def = use_def_map(db, body_scope);
let declarations = use_def.public_declarations(symbol_id);
let declared_and_qualifiers = symbol_from_declarations(db, declarations);
let declarations = use_def.public_declarations(place_id);
let declared_and_qualifiers = place_from_declarations(db, declarations);
match declared_and_qualifiers {
Ok(SymbolAndQualifiers {
symbol: mut declared @ Symbol::Type(declared_ty, declaredness),
Ok(PlaceAndQualifiers {
place: mut declared @ Place::Type(declared_ty, declaredness),
qualifiers,
}) => {
// For the purpose of finding instance attributes, ignore `ClassVar`
// declarations:
if qualifiers.contains(TypeQualifiers::CLASS_VAR) {
declared = Symbol::Unbound;
declared = Place::Unbound;
}
// The attribute is declared in the class body.
let bindings = use_def.public_bindings(symbol_id);
let inferred = symbol_from_bindings(db, bindings);
let bindings = use_def.public_bindings(place_id);
let inferred = place_from_bindings(db, bindings);
let has_binding = !inferred.is_unbound();
if has_binding {
@@ -1892,7 +1885,7 @@ impl<'db> ClassLiteral<'db> {
// we trust the declared type.
declared.with_qualifiers(qualifiers)
} else {
Symbol::Type(
Place::Type(
UnionType::from_elements(db, [declared_ty, implicit_ty]),
declaredness,
)
@@ -1905,7 +1898,7 @@ impl<'db> ClassLiteral<'db> {
// has a class-level default value, but it would not be
// found in a `__dict__` lookup.
Symbol::Unbound.into()
Place::Unbound.into()
}
} else {
// The attribute is declared but not bound in the class body.
@@ -1921,7 +1914,7 @@ impl<'db> ClassLiteral<'db> {
Self::implicit_instance_attribute(db, body_scope, name)
.ignore_possibly_unbound()
{
Symbol::Type(
Place::Type(
UnionType::from_elements(db, [declared_ty, implicit_ty]),
declaredness,
)
@@ -1933,8 +1926,8 @@ impl<'db> ClassLiteral<'db> {
}
}
Ok(SymbolAndQualifiers {
symbol: Symbol::Unbound,
Ok(PlaceAndQualifiers {
place: Place::Unbound,
qualifiers: _,
}) => {
// The attribute is not *declared* in the class body. It could still be declared/bound
@@ -1944,7 +1937,7 @@ impl<'db> ClassLiteral<'db> {
}
Err((declared, _conflicting_declarations)) => {
// There are conflicting declarations for this attribute in the class body.
Symbol::bound(declared.inner_type()).with_qualifiers(declared.qualifiers())
Place::bound(declared.inner_type()).with_qualifiers(declared.qualifiers())
}
}
} else {
@@ -2459,16 +2452,16 @@ impl<'db> KnownClass {
self,
db: &'db dyn Db,
) -> Result<ClassLiteral<'db>, KnownClassLookupError<'db>> {
let symbol = known_module_symbol(db, self.canonical_module(db), self.name(db)).symbol;
let symbol = known_module_symbol(db, self.canonical_module(db), self.name(db)).place;
match symbol {
Symbol::Type(Type::ClassLiteral(class_literal), Boundness::Bound) => Ok(class_literal),
Symbol::Type(Type::ClassLiteral(class_literal), Boundness::PossiblyUnbound) => {
Place::Type(Type::ClassLiteral(class_literal), Boundness::Bound) => Ok(class_literal),
Place::Type(Type::ClassLiteral(class_literal), Boundness::PossiblyUnbound) => {
Err(KnownClassLookupError::ClassPossiblyUnbound { class_literal })
}
Symbol::Type(found_type, _) => {
Place::Type(found_type, _) => {
Err(KnownClassLookupError::SymbolNotAClass { found_type })
}
Symbol::Unbound => Err(KnownClassLookupError::ClassNotFound),
Place::Unbound => Err(KnownClassLookupError::ClassNotFound),
}
}

View File

@@ -11,13 +11,14 @@ use ruff_text_size::{Ranged, TextRange};
use super::{Type, TypeCheckDiagnostics, binding_type};
use crate::lint::LintSource;
use crate::semantic_index::symbol::ScopeId;
use crate::semantic_index::place::ScopeId;
use crate::semantic_index::semantic_index;
use crate::types::function::FunctionDecorators;
use crate::{
Db,
lint::{LintId, LintMetadata},
suppression::suppressions,
};
use crate::{semantic_index::semantic_index, types::FunctionDecorators};
/// Context for inferring the types of a single file.
///

View File

@@ -8,17 +8,17 @@ use super::{
use crate::lint::{Level, LintRegistryBuilder, LintStatus};
use crate::suppression::FileSuppressionId;
use crate::types::LintDiagnosticGuard;
use crate::types::function::KnownFunction;
use crate::types::string_annotation::{
BYTE_STRING_TYPE_ANNOTATION, ESCAPE_CHARACTER_IN_FORWARD_ANNOTATION, FSTRING_TYPE_ANNOTATION,
IMPLICIT_CONCATENATED_STRING_TYPE_ANNOTATION, INVALID_SYNTAX_IN_FORWARD_ANNOTATION,
RAW_STRING_TYPE_ANNOTATION,
};
use crate::types::{KnownFunction, SpecialFormType, Type, protocol_class::ProtocolClassLiteral};
use crate::types::{SpecialFormType, Type, protocol_class::ProtocolClassLiteral};
use crate::{Db, Module, ModuleName, Program, declare_lint};
use itertools::Itertools;
use ruff_db::diagnostic::{Annotation, Diagnostic, Severity, SubDiagnostic};
use ruff_python_ast::{self as ast, AnyNodeRef};
use ruff_python_stdlib::builtins::version_builtin_was_added;
use ruff_text_size::{Ranged, TextRange};
use rustc_hash::FxHashSet;
use std::fmt::Formatter;
@@ -1777,25 +1777,6 @@ pub(super) fn report_possibly_unbound_attribute(
));
}
pub(super) fn report_unresolved_reference(context: &InferContext, expr_name_node: &ast::ExprName) {
let Some(builder) = context.report_lint(&UNRESOLVED_REFERENCE, expr_name_node) else {
return;
};
let ast::ExprName { id, .. } = expr_name_node;
let mut diagnostic = builder.into_diagnostic(format_args!("Name `{id}` used when not defined"));
if let Some(version_added_to_builtins) = version_builtin_was_added(id) {
diagnostic.info(format_args!(
"`{id}` was added as a builtin in Python 3.{version_added_to_builtins}"
));
add_inferred_python_version_hint_to_diagnostic(
context.db(),
&mut diagnostic,
"resolving types",
);
}
}
pub(super) fn report_invalid_exception_caught(context: &InferContext, node: &ast::Expr, ty: Type) {
let Some(builder) = context.report_lint(&INVALID_EXCEPTION_CAUGHT, node) else {
return;

View File

@@ -7,6 +7,7 @@ use ruff_python_ast::str::{Quote, TripleQuotes};
use ruff_python_literal::escape::AsciiEscape;
use crate::types::class::{ClassLiteral, ClassType, GenericAlias};
use crate::types::function::{FunctionType, OverloadLiteral};
use crate::types::generics::{GenericContext, Specialization};
use crate::types::signatures::{CallableSignature, Parameter, Parameters, Signature};
use crate::types::{
@@ -112,34 +113,7 @@ impl Display for DisplayRepresentation<'_> {
},
Type::SpecialForm(special_form) => special_form.fmt(f),
Type::KnownInstance(known_instance) => known_instance.repr(self.db).fmt(f),
Type::FunctionLiteral(function) => {
let signature = function.signature(self.db);
// TODO: when generic function types are supported, we should add
// the generic type parameters to the signature, i.e.
// show `def foo[T](x: T) -> T`.
match signature.overloads.as_slice() {
[signature] => {
write!(
f,
// "def {name}{specialization}{signature}",
"def {name}{signature}",
name = function.name(self.db),
signature = signature.display(self.db)
)
}
signatures => {
// TODO: How to display overloads?
f.write_str("Overload[")?;
let mut join = f.join(", ");
for signature in signatures {
join.entry(&signature.display(self.db));
}
f.write_str("]")
}
}
}
Type::FunctionLiteral(function) => function.display(self.db).fmt(f),
Type::Callable(callable) => callable.display(self.db).fmt(f),
Type::BoundMethod(bound_method) => {
let function = bound_method.function(self.db);
@@ -241,6 +215,71 @@ impl Display for DisplayRepresentation<'_> {
}
}
impl<'db> OverloadLiteral<'db> {
// Not currently used, but useful for debugging.
#[expect(dead_code)]
pub(crate) fn display(self, db: &'db dyn Db) -> DisplayOverloadLiteral<'db> {
DisplayOverloadLiteral { literal: self, db }
}
}
pub(crate) struct DisplayOverloadLiteral<'db> {
literal: OverloadLiteral<'db>,
db: &'db dyn Db,
}
impl Display for DisplayOverloadLiteral<'_> {
fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {
let signature = self.literal.signature(self.db, None);
write!(
f,
"def {name}{signature}",
name = self.literal.name(self.db),
signature = signature.display(self.db)
)
}
}
impl<'db> FunctionType<'db> {
pub(crate) fn display(self, db: &'db dyn Db) -> DisplayFunctionType<'db> {
DisplayFunctionType { ty: self, db }
}
}
pub(crate) struct DisplayFunctionType<'db> {
ty: FunctionType<'db>,
db: &'db dyn Db,
}
impl Display for DisplayFunctionType<'_> {
fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {
let signature = self.ty.signature(self.db);
// TODO: We should consider adding the type parameters to the signature of a generic
// function, i.e. `def foo[T](x: T) -> T`.
match signature.overloads.as_slice() {
[signature] => {
write!(
f,
"def {name}{signature}",
name = self.ty.name(self.db),
signature = signature.display(self.db)
)
}
signatures => {
// TODO: How to display overloads?
f.write_str("Overload[")?;
let mut join = f.join(", ");
for signature in signatures {
join.entry(&signature.display(self.db));
}
f.write_str("]")
}
}
}
}
impl<'db> GenericAlias<'db> {
pub(crate) fn display(&'db self, db: &'db dyn Db) -> DisplayGenericAlias<'db> {
DisplayGenericAlias {
@@ -755,7 +794,7 @@ mod tests {
use crate::Db;
use crate::db::tests::setup_db;
use crate::symbol::typing_extensions_symbol;
use crate::place::typing_extensions_symbol;
use crate::types::{KnownClass, Parameter, Parameters, Signature, StringLiteralType, Type};
#[test]
@@ -794,7 +833,7 @@ mod tests {
);
let iterator_synthesized = typing_extensions_symbol(&db, "Iterator")
.symbol
.place
.ignore_possibly_unbound()
.unwrap()
.to_instance(&db)

View File

@@ -0,0 +1,996 @@
//! Contains representations of function literals. There are several complicating factors:
//!
//! - Functions can be generic, and can have specializations applied to them. These are not the
//! same thing! For instance, a method of a generic class might not itself be generic, but it can
//! still have the class's specialization applied to it.
//!
//! - Functions can be overloaded, and each overload can be independently generic or not, with
//! different sets of typevars for different generic overloads. In some cases we need to consider
//! each overload separately; in others we need to consider all of the overloads (and any
//! implementation) as a single collective entity.
//!
//! - Certain “known” functions need special treatment — for instance, inferring a special return
//! type, or raising custom diagnostics.
//!
//! - TODO: Some functions don't correspond to a function definition in the AST, and are instead
//! synthesized as we mimic the behavior of the Python interpreter. Even though they are
//! synthesized, and are “implemented” as Rust code, they are still functions from the POV of the
//! rest of the type system.
//!
//! Given these constraints, we have the following representation: a function is a list of one or
//! more overloads, with zero or more specializations (more specifically, “type mappings”) applied
//! to it. [`FunctionType`] is the outermost type, which is what [`Type::FunctionLiteral`] wraps.
//! It contains the list of type mappings to apply. It wraps a [`FunctionLiteral`], which collects
//! together all of the overloads (and implementation) of an overloaded function. An
//! [`OverloadLiteral`] represents an individual function definition in the AST — that is, each
//! overload (and implementation) of an overloaded function, or the single definition of a
//! non-overloaded function.
//!
//! Technically, each `FunctionLiteral` wraps a particular overload and all _previous_ overloads.
//! So it's only true that it wraps _all_ overloads if you are looking at the last definition. For
//! instance, in
//!
//! ```py
//! @overload
//! def f(x: int) -> None: ...
//! # <-- 1
//!
//! @overload
//! def f(x: str) -> None: ...
//! # <-- 2
//!
//! def f(x): pass
//! # <-- 3
//! ```
//!
//! resolving `f` at each of the three numbered positions will give you a `FunctionType`, which
//! wraps a `FunctionLiteral`, which contain `OverloadLiteral`s only for the definitions that
//! appear before that position. We rely on the fact that later definitions shadow earlier ones, so
//! the public type of `f` is resolved at position 3, correctly giving you all of the overloads
//! (and the implementation).
use std::str::FromStr;
use bitflags::bitflags;
use ruff_db::diagnostic::Span;
use ruff_db::files::{File, FileRange};
use ruff_python_ast as ast;
use ruff_text_size::Ranged;
use crate::module_resolver::{KnownModule, file_to_module};
use crate::place::{Boundness, Place, place_from_bindings};
use crate::semantic_index::ast_ids::HasScopedUseId;
use crate::semantic_index::definition::Definition;
use crate::semantic_index::place::ScopeId;
use crate::semantic_index::semantic_index;
use crate::types::generics::GenericContext;
use crate::types::narrow::ClassInfoConstraintFunction;
use crate::types::signatures::{CallableSignature, Signature};
use crate::types::{BoundMethodType, CallableType, Type, TypeMapping, TypeVarInstance};
use crate::{Db, FxOrderSet};
/// A collection of useful spans for annotating functions.
///
/// This can be retrieved via `FunctionType::spans` or
/// `Type::function_spans`.
pub(crate) struct FunctionSpans {
/// The span of the entire function "signature." This includes
/// the name, parameter list and return type (if present).
pub(crate) signature: Span,
/// The span of the function name. i.e., `foo` in `def foo(): ...`.
pub(crate) name: Span,
/// The span of the parameter list, including the opening and
/// closing parentheses.
#[expect(dead_code)]
pub(crate) parameters: Span,
/// The span of the annotated return type, if present.
pub(crate) return_type: Option<Span>,
}
bitflags! {
#[derive(Copy, Clone, Debug, Eq, PartialEq, Default, Hash)]
pub struct FunctionDecorators: u8 {
/// `@classmethod`
const CLASSMETHOD = 1 << 0;
/// `@typing.no_type_check`
const NO_TYPE_CHECK = 1 << 1;
/// `@typing.overload`
const OVERLOAD = 1 << 2;
/// `@abc.abstractmethod`
const ABSTRACT_METHOD = 1 << 3;
/// `@typing.final`
const FINAL = 1 << 4;
/// `@typing.override`
const OVERRIDE = 1 << 6;
}
}
bitflags! {
/// Used for the return type of `dataclass_transform(…)` calls. Keeps track of the
/// arguments that were passed in. For the precise meaning of the fields, see [1].
///
/// [1]: https://docs.python.org/3/library/typing.html#typing.dataclass_transform
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, salsa::Update)]
pub struct DataclassTransformerParams: u8 {
const EQ_DEFAULT = 1 << 0;
const ORDER_DEFAULT = 1 << 1;
const KW_ONLY_DEFAULT = 1 << 2;
const FROZEN_DEFAULT = 1 << 3;
}
}
impl Default for DataclassTransformerParams {
fn default() -> Self {
Self::EQ_DEFAULT
}
}
/// Representation of a function definition in the AST: either a non-generic function, or a generic
/// function that has not been specialized.
///
/// If a function has multiple overloads, each overload is represented by a separate function
/// definition in the AST, and is therefore a separate `OverloadLiteral` instance.
///
/// # Ordering
/// Ordering is based on the function's id assigned by salsa and not on the function literal's
/// values. The id may change between runs, or when the function literal was garbage collected and
/// recreated.
#[salsa::interned(debug)]
#[derive(PartialOrd, Ord)]
pub struct OverloadLiteral<'db> {
/// Name of the function at definition.
#[returns(ref)]
pub name: ast::name::Name,
/// Is this a function that we special-case somehow? If so, which one?
pub(crate) known: Option<KnownFunction>,
/// The scope that's created by the function, in which the function body is evaluated.
pub(crate) body_scope: ScopeId<'db>,
/// A set of special decorators that were applied to this function
pub(crate) decorators: FunctionDecorators,
/// The arguments to `dataclass_transformer`, if this function was annotated
/// with `@dataclass_transformer(...)`.
pub(crate) dataclass_transformer_params: Option<DataclassTransformerParams>,
}
#[salsa::tracked]
impl<'db> OverloadLiteral<'db> {
fn with_dataclass_transformer_params(
self,
db: &'db dyn Db,
params: DataclassTransformerParams,
) -> Self {
Self::new(
db,
self.name(db).clone(),
self.known(db),
self.body_scope(db),
self.decorators(db),
Some(params),
)
}
fn file(self, db: &'db dyn Db) -> File {
// NOTE: Do not use `self.definition(db).file(db)` here, as that could create a
// cross-module dependency on the full AST.
self.body_scope(db).file(db)
}
pub(crate) fn has_known_decorator(self, db: &dyn Db, decorator: FunctionDecorators) -> bool {
self.decorators(db).contains(decorator)
}
pub(crate) fn is_overload(self, db: &dyn Db) -> bool {
self.has_known_decorator(db, FunctionDecorators::OVERLOAD)
}
fn node(self, db: &'db dyn Db, file: File) -> &'db ast::StmtFunctionDef {
debug_assert_eq!(
file,
self.file(db),
"OverloadLiteral::node() must be called with the same file as the one where \
the function is defined."
);
self.body_scope(db).node(db).expect_function()
}
/// Returns the [`FileRange`] of the function's name.
pub(crate) fn focus_range(self, db: &dyn Db) -> FileRange {
FileRange::new(
self.file(db),
self.body_scope(db).node(db).expect_function().name.range,
)
}
/// Returns the [`Definition`] of this function.
///
/// ## Warning
///
/// This uses the semantic index to find the definition of the function. This means that if the
/// calling query is not in the same file as this function is defined in, then this will create
/// a cross-module dependency directly on the full AST which will lead to cache
/// over-invalidation.
fn definition(self, db: &'db dyn Db) -> Definition<'db> {
let body_scope = self.body_scope(db);
let index = semantic_index(db, body_scope.file(db));
index.expect_single_definition(body_scope.node(db).expect_function())
}
/// Returns the overload immediately before this one in the AST. Returns `None` if there is no
/// previous overload.
fn previous_overload(self, db: &'db dyn Db) -> Option<FunctionLiteral<'db>> {
// The semantic model records a use for each function on the name node. This is used
// here to get the previous function definition with the same name.
let scope = self.definition(db).scope(db);
let use_def = semantic_index(db, scope.file(db)).use_def_map(scope.file_scope_id(db));
let use_id = self
.body_scope(db)
.node(db)
.expect_function()
.name
.scoped_use_id(db, scope);
let Place::Type(Type::FunctionLiteral(previous_type), Boundness::Bound) =
place_from_bindings(db, use_def.bindings_at_use(use_id))
else {
return None;
};
let previous_literal = previous_type.literal(db);
let previous_overload = previous_literal.last_definition(db);
if !previous_overload.is_overload(db) {
return None;
}
Some(previous_literal)
}
/// Typed internally-visible signature for this function.
///
/// This represents the annotations on the function itself, unmodified by decorators and
/// overloads.
///
/// ## Warning
///
/// This uses the semantic index to find the definition of the function. This means that if the
/// calling query is not in the same file as this function is defined in, then this will create
/// a cross-module dependency directly on the full AST which will lead to cache
/// over-invalidation.
pub(crate) fn signature(
self,
db: &'db dyn Db,
inherited_generic_context: Option<GenericContext<'db>>,
) -> Signature<'db> {
let scope = self.body_scope(db);
let function_stmt_node = scope.node(db).expect_function();
let definition = self.definition(db);
let generic_context = function_stmt_node.type_params.as_ref().map(|type_params| {
let index = semantic_index(db, scope.file(db));
GenericContext::from_type_params(db, index, type_params)
});
Signature::from_function(
db,
generic_context,
inherited_generic_context,
definition,
function_stmt_node,
)
}
fn parameter_span(
self,
db: &'db dyn Db,
parameter_index: Option<usize>,
) -> Option<(Span, Span)> {
let function_scope = self.body_scope(db);
let span = Span::from(function_scope.file(db));
let node = function_scope.node(db);
let func_def = node.as_function()?;
let range = parameter_index
.and_then(|parameter_index| {
func_def
.parameters
.iter()
.nth(parameter_index)
.map(|param| param.range())
})
.unwrap_or(func_def.parameters.range);
let name_span = span.clone().with_range(func_def.name.range);
let parameter_span = span.with_range(range);
Some((name_span, parameter_span))
}
pub(crate) fn spans(self, db: &'db dyn Db) -> Option<FunctionSpans> {
let function_scope = self.body_scope(db);
let span = Span::from(function_scope.file(db));
let node = function_scope.node(db);
let func_def = node.as_function()?;
let return_type_range = func_def.returns.as_ref().map(|returns| returns.range());
let mut signature = func_def.name.range.cover(func_def.parameters.range);
if let Some(return_type_range) = return_type_range {
signature = signature.cover(return_type_range);
}
Some(FunctionSpans {
signature: span.clone().with_range(signature),
name: span.clone().with_range(func_def.name.range),
parameters: span.clone().with_range(func_def.parameters.range),
return_type: return_type_range.map(|range| span.clone().with_range(range)),
})
}
}
/// Representation of a function definition in the AST, along with any previous overloads of the
/// function. Each overload can be separately generic or not, and each generic overload uses
/// distinct typevars.
///
/// # Ordering
/// Ordering is based on the function's id assigned by salsa and not on the function literal's
/// values. The id may change between runs, or when the function literal was garbage collected and
/// recreated.
#[salsa::interned(debug)]
#[derive(PartialOrd, Ord)]
pub struct FunctionLiteral<'db> {
pub(crate) last_definition: OverloadLiteral<'db>,
/// The inherited generic context, if this function is a constructor method (`__new__` or
/// `__init__`) being used to infer the specialization of its generic class. If any of the
/// method's overloads are themselves generic, this is in addition to those per-overload
/// generic contexts (which are created lazily in [`OverloadLiteral::signature`]).
///
/// If the function is not a constructor method, this field will always be `None`.
///
/// If the function is a constructor method, we will end up creating two `FunctionLiteral`
/// instances for it. The first is created in [`TypeInferenceBuilder`][infer] when we encounter
/// the function definition during type inference. At this point, we don't yet know if the
/// function is a constructor method, so we create a `FunctionLiteral` with `None` for this
/// field.
///
/// If at some point we encounter a call expression, which invokes the containing class's
/// constructor, as will create a _new_ `FunctionLiteral` instance for the function, with this
/// field [updated][] to contain the containing class's generic context.
///
/// [infer]: crate::types::infer::TypeInferenceBuilder::infer_function_definition
/// [updated]: crate::types::class::ClassLiteral::own_class_member
inherited_generic_context: Option<GenericContext<'db>>,
}
#[salsa::tracked]
impl<'db> FunctionLiteral<'db> {
fn with_inherited_generic_context(
self,
db: &'db dyn Db,
inherited_generic_context: GenericContext<'db>,
) -> Self {
// A function cannot inherit more than one generic context from its containing class.
debug_assert!(self.inherited_generic_context(db).is_none());
Self::new(
db,
self.last_definition(db),
Some(inherited_generic_context),
)
}
fn name(self, db: &'db dyn Db) -> &'db ast::name::Name {
// All of the overloads of a function literal should have the same name.
self.last_definition(db).name(db)
}
fn known(self, db: &'db dyn Db) -> Option<KnownFunction> {
// Whether a function is known is based on its name (and its containing module's name), so
// all overloads should be known (or not) equivalently.
self.last_definition(db).known(db)
}
fn has_known_decorator(self, db: &dyn Db, decorator: FunctionDecorators) -> bool {
self.iter_overloads_and_implementation(db)
.any(|overload| overload.decorators(db).contains(decorator))
}
fn definition(self, db: &'db dyn Db) -> Definition<'db> {
self.last_definition(db).definition(db)
}
fn parameter_span(
self,
db: &'db dyn Db,
parameter_index: Option<usize>,
) -> Option<(Span, Span)> {
self.last_definition(db).parameter_span(db, parameter_index)
}
fn spans(self, db: &'db dyn Db) -> Option<FunctionSpans> {
self.last_definition(db).spans(db)
}
#[salsa::tracked(returns(ref))]
fn overloads_and_implementation(
self,
db: &'db dyn Db,
) -> (Box<[OverloadLiteral<'db>]>, Option<OverloadLiteral<'db>>) {
let self_overload = self.last_definition(db);
let mut current = self_overload;
let mut overloads = vec![];
while let Some(previous) = current.previous_overload(db) {
let overload = previous.last_definition(db);
overloads.push(overload);
current = overload;
}
// Overloads are inserted in reverse order, from bottom to top.
overloads.reverse();
let implementation = if self_overload.is_overload(db) {
overloads.push(self_overload);
None
} else {
Some(self_overload)
};
(overloads.into_boxed_slice(), implementation)
}
fn iter_overloads_and_implementation(
self,
db: &'db dyn Db,
) -> impl Iterator<Item = OverloadLiteral<'db>> + 'db {
let (implementation, overloads) = self.overloads_and_implementation(db);
overloads.iter().chain(implementation).copied()
}
/// Typed externally-visible signature for this function.
///
/// This is the signature as seen by external callers, possibly modified by decorators and/or
/// overloaded.
///
/// ## Warning
///
/// This uses the semantic index to find the definition of the function. This means that if the
/// calling query is not in the same file as this function is defined in, then this will create
/// a cross-module dependency directly on the full AST which will lead to cache
/// over-invalidation.
fn signature<'a>(
self,
db: &'db dyn Db,
type_mappings: &'a [TypeMapping<'a, 'db>],
) -> CallableSignature<'db>
where
'db: 'a,
{
// We only include an implementation (i.e. a definition not decorated with `@overload`) if
// it's the only definition.
let inherited_generic_context = self.inherited_generic_context(db);
let (overloads, implementation) = self.overloads_and_implementation(db);
if let Some(implementation) = implementation {
if overloads.is_empty() {
return CallableSignature::single(type_mappings.iter().fold(
implementation.signature(db, inherited_generic_context),
|ty, mapping| ty.apply_type_mapping(db, mapping),
));
}
}
CallableSignature::from_overloads(overloads.iter().map(|overload| {
type_mappings.iter().fold(
overload.signature(db, inherited_generic_context),
|ty, mapping| ty.apply_type_mapping(db, mapping),
)
}))
}
fn normalized(self, db: &'db dyn Db) -> Self {
let context = self
.inherited_generic_context(db)
.map(|ctx| ctx.normalized(db));
Self::new(db, self.last_definition(db), context)
}
}
/// Represents a function type, which might be a non-generic function, or a specialization of a
/// generic function.
#[salsa::interned(debug)]
#[derive(PartialOrd, Ord)]
pub struct FunctionType<'db> {
pub(crate) literal: FunctionLiteral<'db>,
/// Type mappings that should be applied to the function's parameter and return types. This
/// might include specializations of enclosing generic contexts (e.g. for non-generic methods
/// of a specialized generic class).
#[returns(deref)]
type_mappings: Box<[TypeMapping<'db, 'db>]>,
}
#[salsa::tracked]
impl<'db> FunctionType<'db> {
pub(crate) fn with_inherited_generic_context(
self,
db: &'db dyn Db,
inherited_generic_context: GenericContext<'db>,
) -> Self {
let literal = self
.literal(db)
.with_inherited_generic_context(db, inherited_generic_context);
Self::new(db, literal, self.type_mappings(db))
}
pub(crate) fn with_type_mapping<'a>(
self,
db: &'db dyn Db,
type_mapping: &TypeMapping<'a, 'db>,
) -> Self {
let type_mappings: Box<[_]> = self
.type_mappings(db)
.iter()
.cloned()
.chain(std::iter::once(type_mapping.to_owned()))
.collect();
Self::new(db, self.literal(db), type_mappings)
}
pub(crate) fn with_dataclass_transformer_params(
self,
db: &'db dyn Db,
params: DataclassTransformerParams,
) -> Self {
// A decorator only applies to the specific overload that it is attached to, not to all
// previous overloads.
let literal = self.literal(db);
let last_definition = literal
.last_definition(db)
.with_dataclass_transformer_params(db, params);
let literal =
FunctionLiteral::new(db, last_definition, literal.inherited_generic_context(db));
Self::new(db, literal, self.type_mappings(db))
}
/// Returns the [`File`] in which this function is defined.
pub(crate) fn file(self, db: &'db dyn Db) -> File {
self.literal(db).last_definition(db).file(db)
}
/// Returns the AST node for this function.
pub(crate) fn node(self, db: &'db dyn Db, file: File) -> &'db ast::StmtFunctionDef {
self.literal(db).last_definition(db).node(db, file)
}
pub(crate) fn name(self, db: &'db dyn Db) -> &'db ast::name::Name {
self.literal(db).name(db)
}
pub(crate) fn known(self, db: &'db dyn Db) -> Option<KnownFunction> {
self.literal(db).known(db)
}
pub(crate) fn is_known(self, db: &'db dyn Db, known_function: KnownFunction) -> bool {
self.known(db) == Some(known_function)
}
/// Returns if any of the overloads of this function have a particular decorator.
///
/// Some decorators are expected to appear on every overload; others are expected to appear
/// only the implementation or first overload. This method does not check either of those
/// conditions.
pub(crate) fn has_known_decorator(self, db: &dyn Db, decorator: FunctionDecorators) -> bool {
self.literal(db).has_known_decorator(db, decorator)
}
/// Returns the [`Definition`] of the implementation or first overload of this function.
///
/// ## Warning
///
/// This uses the semantic index to find the definition of the function. This means that if the
/// calling query is not in the same file as this function is defined in, then this will create
/// a cross-module dependency directly on the full AST which will lead to cache
/// over-invalidation.
pub(crate) fn definition(self, db: &'db dyn Db) -> Definition<'db> {
self.literal(db).definition(db)
}
/// Returns a tuple of two spans. The first is
/// the span for the identifier of the function
/// definition for `self`. The second is
/// the span for the parameter in the function
/// definition for `self`.
///
/// If there are no meaningful spans, then this
/// returns `None`. For example, when this type
/// isn't callable.
///
/// When `parameter_index` is `None`, then the
/// second span returned covers the entire parameter
/// list.
///
/// # Performance
///
/// Note that this may introduce cross-module
/// dependencies. This can have an impact on
/// the effectiveness of incremental caching
/// and should therefore be used judiciously.
///
/// An example of a good use case is to improve
/// a diagnostic.
pub(crate) fn parameter_span(
self,
db: &'db dyn Db,
parameter_index: Option<usize>,
) -> Option<(Span, Span)> {
self.literal(db).parameter_span(db, parameter_index)
}
/// Returns a collection of useful spans for a
/// function signature. These are useful for
/// creating annotations on diagnostics.
///
/// # Performance
///
/// Note that this may introduce cross-module
/// dependencies. This can have an impact on
/// the effectiveness of incremental caching
/// and should therefore be used judiciously.
///
/// An example of a good use case is to improve
/// a diagnostic.
pub(crate) fn spans(self, db: &'db dyn Db) -> Option<FunctionSpans> {
self.literal(db).spans(db)
}
/// Returns all of the overload signatures and the implementation definition, if any, of this
/// function. The overload signatures will be in source order.
pub(crate) fn overloads_and_implementation(
self,
db: &'db dyn Db,
) -> &'db (Box<[OverloadLiteral<'db>]>, Option<OverloadLiteral<'db>>) {
self.literal(db).overloads_and_implementation(db)
}
/// Returns an iterator of all of the definitions of this function, including both overload
/// signatures and any implementation, all in source order.
pub(crate) fn iter_overloads_and_implementation(
self,
db: &'db dyn Db,
) -> impl Iterator<Item = OverloadLiteral<'db>> + 'db {
self.literal(db).iter_overloads_and_implementation(db)
}
/// Typed externally-visible signature for this function.
///
/// This is the signature as seen by external callers, possibly modified by decorators and/or
/// overloaded.
///
/// ## Why is this a salsa query?
///
/// This is a salsa query to short-circuit the invalidation
/// when the function's AST node changes.
///
/// Were this not a salsa query, then the calling query
/// would depend on the function's AST and rerun for every change in that file.
#[salsa::tracked(returns(ref), cycle_fn=signature_cycle_recover, cycle_initial=signature_cycle_initial)]
pub(crate) fn signature(self, db: &'db dyn Db) -> CallableSignature<'db> {
self.literal(db).signature(db, self.type_mappings(db))
}
/// Convert the `FunctionType` into a [`Type::Callable`].
pub(crate) fn into_callable_type(self, db: &'db dyn Db) -> Type<'db> {
Type::Callable(CallableType::new(db, self.signature(db), false))
}
/// Convert the `FunctionType` into a [`Type::BoundMethod`].
pub(crate) fn into_bound_method_type(
self,
db: &'db dyn Db,
self_instance: Type<'db>,
) -> Type<'db> {
Type::BoundMethod(BoundMethodType::new(db, self, self_instance))
}
pub(crate) fn is_subtype_of(self, db: &'db dyn Db, other: Self) -> bool {
// A function type is the subtype of itself, and not of any other function type. However,
// our representation of a function type includes any specialization that should be applied
// to the signature. Different specializations of the same function type are only subtypes
// of each other if they result in subtype signatures.
if self.normalized(db) == other.normalized(db) {
return true;
}
if self.literal(db) != other.literal(db) {
return false;
}
let self_signature = self.signature(db);
let other_signature = other.signature(db);
if !self_signature.is_fully_static(db) || !other_signature.is_fully_static(db) {
return false;
}
self_signature.is_subtype_of(db, other_signature)
}
pub(crate) fn is_assignable_to(self, db: &'db dyn Db, other: Self) -> bool {
// A function type is assignable to itself, and not to any other function type. However,
// our representation of a function type includes any specialization that should be applied
// to the signature. Different specializations of the same function type are only
// assignable to each other if they result in assignable signatures.
self.literal(db) == other.literal(db)
&& self.signature(db).is_assignable_to(db, other.signature(db))
}
pub(crate) fn is_equivalent_to(self, db: &'db dyn Db, other: Self) -> bool {
if self.normalized(db) == other.normalized(db) {
return true;
}
if self.literal(db) != other.literal(db) {
return false;
}
let self_signature = self.signature(db);
let other_signature = other.signature(db);
if !self_signature.is_fully_static(db) || !other_signature.is_fully_static(db) {
return false;
}
self_signature.is_equivalent_to(db, other_signature)
}
pub(crate) fn is_gradual_equivalent_to(self, db: &'db dyn Db, other: Self) -> bool {
self.literal(db) == other.literal(db)
&& self
.signature(db)
.is_gradual_equivalent_to(db, other.signature(db))
}
pub(crate) fn find_legacy_typevars(
self,
db: &'db dyn Db,
typevars: &mut FxOrderSet<TypeVarInstance<'db>>,
) {
let signatures = self.signature(db);
for signature in &signatures.overloads {
signature.find_legacy_typevars(db, typevars);
}
}
pub(crate) fn normalized(self, db: &'db dyn Db) -> Self {
let mappings: Box<_> = self
.type_mappings(db)
.iter()
.map(|mapping| mapping.normalized(db))
.collect();
Self::new(db, self.literal(db).normalized(db), mappings)
}
}
fn signature_cycle_recover<'db>(
_db: &'db dyn Db,
_value: &CallableSignature<'db>,
_count: u32,
_function: FunctionType<'db>,
) -> salsa::CycleRecoveryAction<CallableSignature<'db>> {
salsa::CycleRecoveryAction::Iterate
}
fn signature_cycle_initial<'db>(
db: &'db dyn Db,
_function: FunctionType<'db>,
) -> CallableSignature<'db> {
CallableSignature::single(Signature::bottom(db))
}
/// Non-exhaustive enumeration of known functions (e.g. `builtins.reveal_type`, ...) that might
/// have special behavior.
#[derive(
Debug, Copy, Clone, PartialEq, Eq, Hash, strum_macros::EnumString, strum_macros::IntoStaticStr,
)]
#[strum(serialize_all = "snake_case")]
#[cfg_attr(test, derive(strum_macros::EnumIter))]
pub enum KnownFunction {
/// `builtins.isinstance`
#[strum(serialize = "isinstance")]
IsInstance,
/// `builtins.issubclass`
#[strum(serialize = "issubclass")]
IsSubclass,
/// `builtins.hasattr`
#[strum(serialize = "hasattr")]
HasAttr,
/// `builtins.reveal_type`, `typing.reveal_type` or `typing_extensions.reveal_type`
RevealType,
/// `builtins.len`
Len,
/// `builtins.repr`
Repr,
/// `typing(_extensions).final`
Final,
/// [`typing(_extensions).no_type_check`](https://typing.python.org/en/latest/spec/directives.html#no-type-check)
NoTypeCheck,
/// `typing(_extensions).assert_type`
AssertType,
/// `typing(_extensions).assert_never`
AssertNever,
/// `typing(_extensions).cast`
Cast,
/// `typing(_extensions).overload`
Overload,
/// `typing(_extensions).override`
Override,
/// `typing(_extensions).is_protocol`
IsProtocol,
/// `typing(_extensions).get_protocol_members`
GetProtocolMembers,
/// `typing(_extensions).runtime_checkable`
RuntimeCheckable,
/// `typing(_extensions).dataclass_transform`
DataclassTransform,
/// `abc.abstractmethod`
#[strum(serialize = "abstractmethod")]
AbstractMethod,
/// `dataclasses.dataclass`
Dataclass,
/// `inspect.getattr_static`
GetattrStatic,
/// `ty_extensions.static_assert`
StaticAssert,
/// `ty_extensions.is_equivalent_to`
IsEquivalentTo,
/// `ty_extensions.is_subtype_of`
IsSubtypeOf,
/// `ty_extensions.is_assignable_to`
IsAssignableTo,
/// `ty_extensions.is_disjoint_from`
IsDisjointFrom,
/// `ty_extensions.is_gradual_equivalent_to`
IsGradualEquivalentTo,
/// `ty_extensions.is_fully_static`
IsFullyStatic,
/// `ty_extensions.is_singleton`
IsSingleton,
/// `ty_extensions.is_single_valued`
IsSingleValued,
/// `ty_extensions.generic_context`
GenericContext,
/// `ty_extensions.dunder_all_names`
DunderAllNames,
/// `ty_extensions.all_members`
AllMembers,
}
impl KnownFunction {
pub fn into_classinfo_constraint_function(self) -> Option<ClassInfoConstraintFunction> {
match self {
Self::IsInstance => Some(ClassInfoConstraintFunction::IsInstance),
Self::IsSubclass => Some(ClassInfoConstraintFunction::IsSubclass),
_ => None,
}
}
pub(crate) fn try_from_definition_and_name<'db>(
db: &'db dyn Db,
definition: Definition<'db>,
name: &str,
) -> Option<Self> {
let candidate = Self::from_str(name).ok()?;
candidate
.check_module(file_to_module(db, definition.file(db))?.known()?)
.then_some(candidate)
}
/// Return `true` if `self` is defined in `module` at runtime.
const fn check_module(self, module: KnownModule) -> bool {
match self {
Self::IsInstance | Self::IsSubclass | Self::HasAttr | Self::Len | Self::Repr => {
module.is_builtins()
}
Self::AssertType
| Self::AssertNever
| Self::Cast
| Self::Overload
| Self::Override
| Self::RevealType
| Self::Final
| Self::IsProtocol
| Self::GetProtocolMembers
| Self::RuntimeCheckable
| Self::DataclassTransform
| Self::NoTypeCheck => {
matches!(module, KnownModule::Typing | KnownModule::TypingExtensions)
}
Self::AbstractMethod => {
matches!(module, KnownModule::Abc)
}
Self::Dataclass => {
matches!(module, KnownModule::Dataclasses)
}
Self::GetattrStatic => module.is_inspect(),
Self::IsAssignableTo
| Self::IsDisjointFrom
| Self::IsEquivalentTo
| Self::IsGradualEquivalentTo
| Self::IsFullyStatic
| Self::IsSingleValued
| Self::IsSingleton
| Self::IsSubtypeOf
| Self::GenericContext
| Self::DunderAllNames
| Self::StaticAssert
| Self::AllMembers => module.is_ty_extensions(),
}
}
}
#[cfg(test)]
pub(crate) mod tests {
use strum::IntoEnumIterator;
use super::*;
use crate::db::tests::setup_db;
use crate::place::known_module_symbol;
#[test]
fn known_function_roundtrip_from_str() {
let db = setup_db();
for function in KnownFunction::iter() {
let function_name: &'static str = function.into();
let module = match function {
KnownFunction::Len
| KnownFunction::Repr
| KnownFunction::IsInstance
| KnownFunction::HasAttr
| KnownFunction::IsSubclass => KnownModule::Builtins,
KnownFunction::AbstractMethod => KnownModule::Abc,
KnownFunction::Dataclass => KnownModule::Dataclasses,
KnownFunction::GetattrStatic => KnownModule::Inspect,
KnownFunction::Cast
| KnownFunction::Final
| KnownFunction::Overload
| KnownFunction::Override
| KnownFunction::RevealType
| KnownFunction::AssertType
| KnownFunction::AssertNever
| KnownFunction::IsProtocol
| KnownFunction::GetProtocolMembers
| KnownFunction::RuntimeCheckable
| KnownFunction::DataclassTransform
| KnownFunction::NoTypeCheck => KnownModule::TypingExtensions,
KnownFunction::IsSingleton
| KnownFunction::IsSubtypeOf
| KnownFunction::GenericContext
| KnownFunction::DunderAllNames
| KnownFunction::StaticAssert
| KnownFunction::IsFullyStatic
| KnownFunction::IsDisjointFrom
| KnownFunction::IsSingleValued
| KnownFunction::IsAssignableTo
| KnownFunction::IsEquivalentTo
| KnownFunction::IsGradualEquivalentTo
| KnownFunction::AllMembers => KnownModule::TyExtensions,
};
let function_definition = known_module_symbol(&db, module, function_name)
.place
.expect_type()
.expect_function_literal()
.definition(&db);
assert_eq!(
KnownFunction::try_from_definition_and_name(
&db,
function_definition,
function_name
),
Some(function),
"The strum `EnumString` implementation appears to be incorrect for `{function_name}`"
);
}
}
}

View File

@@ -1,13 +1,43 @@
use crate::Db;
use crate::semantic_index::symbol::ScopeId;
use crate::place::{imported_symbol, place_from_bindings, place_from_declarations};
use crate::semantic_index::place::ScopeId;
use crate::semantic_index::{
attribute_scopes, global_scope, semantic_index, symbol_table, use_def_map,
attribute_scopes, global_scope, place_table, semantic_index, use_def_map,
};
use crate::symbol::{imported_symbol, symbol_from_bindings, symbol_from_declarations};
use crate::types::{ClassBase, ClassLiteral, KnownClass, Type};
use ruff_python_ast::name::Name;
use rustc_hash::FxHashSet;
pub(crate) fn all_declarations_and_bindings<'db>(
db: &'db dyn Db,
scope_id: ScopeId<'db>,
) -> impl Iterator<Item = Name> + 'db {
let use_def_map = use_def_map(db, scope_id);
let table = place_table(db, scope_id);
use_def_map
.all_public_declarations()
.filter_map(move |(symbol_id, declarations)| {
place_from_declarations(db, declarations)
.ok()
.and_then(|result| {
result
.place
.ignore_possibly_unbound()
.and_then(|_| table.place_expr(symbol_id).as_name().cloned())
})
})
.chain(
use_def_map
.all_public_bindings()
.filter_map(move |(symbol_id, bindings)| {
place_from_bindings(db, bindings)
.ignore_possibly_unbound()
.and_then(|_| table.place_expr(symbol_id).as_name().cloned())
}),
)
}
struct AllMembers {
members: FxHashSet<Name>,
}
@@ -101,16 +131,18 @@ impl AllMembers {
let module_scope = global_scope(db, file);
let use_def_map = use_def_map(db, module_scope);
let symbol_table = symbol_table(db, module_scope);
let place_table = place_table(db, module_scope);
for (symbol_id, _) in use_def_map.all_public_declarations() {
let symbol_name = symbol_table.symbol(symbol_id).name();
let Some(symbol_name) = place_table.place_expr(symbol_id).as_name() else {
continue;
};
if !imported_symbol(db, file, symbol_name, None)
.symbol
.place
.is_unbound()
{
self.members
.insert(symbol_table.symbol(symbol_id).name().clone());
.insert(place_table.place_expr(symbol_id).expect_name().clone());
}
}
}
@@ -118,24 +150,8 @@ impl AllMembers {
}
fn extend_with_declarations_and_bindings(&mut self, db: &dyn Db, scope_id: ScopeId) {
let use_def_map = use_def_map(db, scope_id);
let symbol_table = symbol_table(db, scope_id);
for (symbol_id, declarations) in use_def_map.all_public_declarations() {
if symbol_from_declarations(db, declarations)
.is_ok_and(|result| !result.symbol.is_unbound())
{
self.members
.insert(symbol_table.symbol(symbol_id).name().clone());
}
}
for (symbol_id, bindings) in use_def_map.all_public_bindings() {
if !symbol_from_bindings(db, bindings).is_unbound() {
self.members
.insert(symbol_table.symbol(symbol_id).name().clone());
}
}
self.members
.extend(all_declarations_and_bindings(db, scope_id));
}
fn extend_with_class_members<'db>(
@@ -167,9 +183,10 @@ impl AllMembers {
let file = class_body_scope.file(db);
let index = semantic_index(db, file);
for function_scope_id in attribute_scopes(db, class_body_scope) {
let attribute_table = index.instance_attribute_table(function_scope_id);
for symbol in attribute_table.symbols() {
self.members.insert(symbol.name().clone());
let place_table = index.place_table(function_scope_id);
for instance_attribute in place_table.instance_attributes() {
let name = instance_attribute.sub_segments()[0].as_member().unwrap();
self.members.insert(name.clone());
}
}
}
@@ -178,6 +195,6 @@ impl AllMembers {
/// List all members of a given type: anything that would be valid when accessed
/// as an attribute on an object of the given type.
pub(crate) fn all_members<'db>(db: &'db dyn Db, ty: Type<'db>) -> FxHashSet<Name> {
pub fn all_members<'db>(db: &'db dyn Db, ty: Type<'db>) -> FxHashSet<Name> {
AllMembers::of(db, ty).members
}

File diff suppressed because it is too large Load Diff

View File

@@ -4,7 +4,7 @@ use std::marker::PhantomData;
use super::protocol_class::ProtocolInterface;
use super::{ClassType, KnownClass, SubclassOfType, Type};
use crate::symbol::{Symbol, SymbolAndQualifiers};
use crate::place::{Boundness, Place, PlaceAndQualifiers};
use crate::types::{ClassLiteral, TypeMapping, TypeVarInstance};
use crate::{Db, FxOrderSet};
@@ -45,12 +45,12 @@ impl<'db> Type<'db> {
protocol: ProtocolInstanceType<'db>,
) -> bool {
// TODO: this should consider the types of the protocol members
// as well as whether each member *exists* on `self`.
protocol
.inner
.interface(db)
.members(db)
.all(|member| !self.member(db, member.name()).symbol.is_unbound())
protocol.inner.interface(db).members(db).all(|member| {
matches!(
self.member(db, member.name()).place,
Place::Type(_, Boundness::Bound)
)
})
}
}
@@ -294,14 +294,14 @@ impl<'db> ProtocolInstanceType<'db> {
false
}
pub(crate) fn instance_member(self, db: &'db dyn Db, name: &str) -> SymbolAndQualifiers<'db> {
pub(crate) fn instance_member(self, db: &'db dyn Db, name: &str) -> PlaceAndQualifiers<'db> {
match self.inner {
Protocol::FromClass(class) => class.instance_member(db, name),
Protocol::Synthesized(synthesized) => synthesized
.interface()
.member_by_name(db, name)
.map(|member| SymbolAndQualifiers {
symbol: Symbol::bound(member.ty()),
.map(|member| PlaceAndQualifiers {
place: Place::bound(member.ty()),
qualifiers: member.qualifiers(),
})
.unwrap_or_else(|| KnownClass::Object.to_instance(db).instance_member(db, name)),

View File

@@ -1,11 +1,12 @@
use crate::Db;
use crate::semantic_index::ast_ids::HasScopedExpressionId;
use crate::semantic_index::expression::Expression;
use crate::semantic_index::place::{PlaceTable, ScopeId, ScopedPlaceId};
use crate::semantic_index::place_table;
use crate::semantic_index::predicate::{
PatternPredicate, PatternPredicateKind, Predicate, PredicateNode,
};
use crate::semantic_index::symbol::{ScopeId, ScopedSymbolId, SymbolTable};
use crate::semantic_index::symbol_table;
use crate::types::function::KnownFunction;
use crate::types::infer::infer_same_file_expression_type;
use crate::types::{
IntersectionBuilder, KnownClass, SubclassOfType, Truthiness, Type, UnionBuilder,
@@ -20,7 +21,7 @@ use ruff_python_ast::{BoolOp, ExprBoolOp};
use rustc_hash::FxHashMap;
use std::collections::hash_map::Entry;
use super::{KnownFunction, UnionType};
use super::UnionType;
/// Return the type constraint that `test` (if true) would place on `symbol`, if any.
///
@@ -41,7 +42,7 @@ use super::{KnownFunction, UnionType};
pub(crate) fn infer_narrowing_constraint<'db>(
db: &'db dyn Db,
predicate: Predicate<'db>,
symbol: ScopedSymbolId,
place: ScopedPlaceId,
) -> Option<Type<'db>> {
let constraints = match predicate.node {
PredicateNode::Expression(expression) => {
@@ -61,7 +62,7 @@ pub(crate) fn infer_narrowing_constraint<'db>(
PredicateNode::StarImportPlaceholder(_) => return None,
};
if let Some(constraints) = constraints {
constraints.get(&symbol).copied()
constraints.get(&place).copied()
} else {
None
}
@@ -189,7 +190,7 @@ impl ClassInfoConstraintFunction {
}
}
type NarrowingConstraints<'db> = FxHashMap<ScopedSymbolId, Type<'db>>;
type NarrowingConstraints<'db> = FxHashMap<ScopedPlaceId, Type<'db>>;
fn merge_constraints_and<'db>(
into: &mut NarrowingConstraints<'db>,
@@ -234,7 +235,7 @@ fn merge_constraints_or<'db>(
}
fn negate_if<'db>(constraints: &mut NarrowingConstraints<'db>, db: &'db dyn Db, yes: bool) {
for (_symbol, ty) in constraints.iter_mut() {
for (_place, ty) in constraints.iter_mut() {
*ty = ty.negate_if(db, yes);
}
}
@@ -346,8 +347,8 @@ impl<'db> NarrowingConstraintsBuilder<'db> {
})
}
fn symbols(&self) -> &'db SymbolTable {
symbol_table(self.db, self.scope())
fn places(&self) -> &'db PlaceTable {
place_table(self.db, self.scope())
}
fn scope(&self) -> ScopeId<'db> {
@@ -359,9 +360,9 @@ impl<'db> NarrowingConstraintsBuilder<'db> {
}
#[track_caller]
fn expect_expr_name_symbol(&self, symbol: &str) -> ScopedSymbolId {
self.symbols()
.symbol_id_by_name(symbol)
fn expect_expr_name_symbol(&self, symbol: &str) -> ScopedPlaceId {
self.places()
.place_id_by_name(symbol)
.expect("We should always have a symbol for every `Name` node")
}

View File

@@ -1,5 +1,5 @@
use crate::db::tests::TestDb;
use crate::symbol::{builtins_symbol, known_module_symbol};
use crate::place::{builtins_symbol, known_module_symbol};
use crate::types::{
BoundMethodType, CallableType, IntersectionBuilder, KnownClass, Parameter, Parameters,
Signature, SpecialFormType, SubclassOfType, TupleType, Type, UnionType,
@@ -130,20 +130,20 @@ impl Ty {
Ty::LiteralString => Type::LiteralString,
Ty::BytesLiteral(s) => Type::bytes_literal(db, s.as_bytes()),
Ty::BuiltinInstance(s) => builtins_symbol(db, s)
.symbol
.place
.expect_type()
.to_instance(db)
.unwrap(),
Ty::AbcInstance(s) => known_module_symbol(db, KnownModule::Abc, s)
.symbol
.place
.expect_type()
.to_instance(db)
.unwrap(),
Ty::AbcClassLiteral(s) => known_module_symbol(db, KnownModule::Abc, s)
.symbol
.place
.expect_type(),
Ty::TypingLiteral => Type::SpecialForm(SpecialFormType::Literal),
Ty::BuiltinClassLiteral(s) => builtins_symbol(db, s).symbol.expect_type(),
Ty::BuiltinClassLiteral(s) => builtins_symbol(db, s).place.expect_type(),
Ty::KnownClassInstance(known_class) => known_class.to_instance(db),
Ty::Union(tys) => {
UnionType::from_elements(db, tys.into_iter().map(|ty| ty.into_type(db)))
@@ -166,7 +166,7 @@ impl Ty {
Ty::SubclassOfBuiltinClass(s) => SubclassOfType::from(
db,
builtins_symbol(db, s)
.symbol
.place
.expect_type()
.expect_class_literal()
.default_specialization(db),
@@ -174,17 +174,17 @@ impl Ty {
Ty::SubclassOfAbcClass(s) => SubclassOfType::from(
db,
known_module_symbol(db, KnownModule::Abc, s)
.symbol
.place
.expect_type()
.expect_class_literal()
.default_specialization(db),
),
Ty::AlwaysTruthy => Type::AlwaysTruthy,
Ty::AlwaysFalsy => Type::AlwaysFalsy,
Ty::BuiltinsFunction(name) => builtins_symbol(db, name).symbol.expect_type(),
Ty::BuiltinsFunction(name) => builtins_symbol(db, name).place.expect_type(),
Ty::BuiltinsBoundMethod { class, method } => {
let builtins_class = builtins_symbol(db, class).symbol.expect_type();
let function = builtins_class.member(db, method).symbol.expect_type();
let builtins_class = builtins_symbol(db, class).place.expect_type();
let function = builtins_class.member(db, method).place.expect_type();
create_bound_method(db, function, builtins_class)
}

View File

@@ -5,8 +5,8 @@ use itertools::{Either, Itertools};
use ruff_python_ast::name::Name;
use crate::{
semantic_index::{symbol_table, use_def_map},
symbol::{symbol_from_bindings, symbol_from_declarations},
place::{place_from_bindings, place_from_declarations},
semantic_index::{place_table, use_def_map},
types::{
ClassBase, ClassLiteral, KnownFunction, Type, TypeMapping, TypeQualifiers, TypeVarInstance,
},
@@ -274,7 +274,7 @@ impl<'a, 'db> ProtocolMember<'a, 'db> {
/// The list of excluded members is subject to change between Python versions,
/// especially for dunders, but it probably doesn't matter *too* much if this
/// list goes out of date. It's up to date as of Python commit 87b1ea016b1454b1e83b9113fa9435849b7743aa
/// (<https://github.com/python/cpython/blob/87b1ea016b1454b1e83b9113fa9435849b7743aa/Lib/typing.py#L1776-L1791>)
/// (<https://github.com/python/cpython/blob/87b1ea016b1454b1e83b9113fa9435849b7743aa/Lib/typing.py#L1776-L1814>)
fn excluded_from_proto_members(member: &str) -> bool {
matches!(
member,
@@ -304,7 +304,7 @@ fn excluded_from_proto_members(member: &str) -> bool {
| "__annotate__"
| "__annotate_func__"
| "__annotations_cache__"
)
) || member.starts_with("_abc_")
}
/// Inner Salsa query for [`ProtocolClassLiteral::interface`].
@@ -322,19 +322,19 @@ fn cached_protocol_interface<'db>(
{
let parent_scope = parent_protocol.body_scope(db);
let use_def_map = use_def_map(db, parent_scope);
let symbol_table = symbol_table(db, parent_scope);
let place_table = place_table(db, parent_scope);
members.extend(
use_def_map
.all_public_declarations()
.flat_map(|(symbol_id, declarations)| {
symbol_from_declarations(db, declarations).map(|symbol| (symbol_id, symbol))
.flat_map(|(place_id, declarations)| {
place_from_declarations(db, declarations).map(|place| (place_id, place))
})
.filter_map(|(symbol_id, symbol)| {
symbol
.symbol
.filter_map(|(place_id, place)| {
place
.place
.ignore_possibly_unbound()
.map(|ty| (symbol_id, ty, symbol.qualifiers))
.map(|ty| (place_id, ty, place.qualifiers))
})
// Bindings in the class body that are not declared in the class body
// are not valid protocol members, and we plan to emit diagnostics for them
@@ -347,14 +347,18 @@ fn cached_protocol_interface<'db>(
.chain(
use_def_map
.all_public_bindings()
.filter_map(|(symbol_id, bindings)| {
symbol_from_bindings(db, bindings)
.filter_map(|(place_id, bindings)| {
place_from_bindings(db, bindings)
.ignore_possibly_unbound()
.map(|ty| (symbol_id, ty, TypeQualifiers::default()))
.map(|ty| (place_id, ty, TypeQualifiers::default()))
}),
)
.map(|(symbol_id, member, qualifiers)| {
(symbol_table.symbol(symbol_id).name(), member, qualifiers)
.filter_map(|(place_id, member, qualifiers)| {
Some((
place_table.place_expr(place_id).as_name()?,
member,
qualifiers,
))
})
.filter(|(name, _, _)| !excluded_from_proto_members(name))
.map(|(name, ty, qualifiers)| {

View File

@@ -53,10 +53,6 @@ impl<'db> CallableSignature<'db> {
self.overloads.iter()
}
pub(crate) fn as_slice(&self) -> &[Signature<'db>] {
self.overloads.as_slice()
}
pub(crate) fn normalized(&self, db: &'db dyn Db) -> Self {
Self::from_overloads(
self.overloads
@@ -1537,15 +1533,15 @@ pub(crate) enum ParameterForm {
mod tests {
use super::*;
use crate::db::tests::{TestDb, setup_db};
use crate::symbol::global_symbol;
use crate::types::{FunctionSignature, FunctionType, KnownClass};
use crate::place::global_symbol;
use crate::types::{FunctionType, KnownClass};
use ruff_db::system::DbWithWritableSystem as _;
#[track_caller]
fn get_function_f<'db>(db: &'db TestDb, file: &'static str) -> FunctionType<'db> {
let module = ruff_db::files::system_path_to_file(db, file).unwrap();
global_symbol(db, module, "f")
.symbol
.place
.expect_type()
.expect_function_literal()
}
@@ -1559,9 +1555,11 @@ mod tests {
fn empty() {
let mut db = setup_db();
db.write_dedented("/src/a.py", "def f(): ...").unwrap();
let func = get_function_f(&db, "/src/a.py");
let func = get_function_f(&db, "/src/a.py")
.literal(&db)
.last_definition(&db);
let sig = func.internal_signature(&db, None);
let sig = func.signature(&db, None);
assert!(sig.return_ty.is_none());
assert_params(&sig, &[]);
@@ -1582,9 +1580,11 @@ mod tests {
",
)
.unwrap();
let func = get_function_f(&db, "/src/a.py");
let func = get_function_f(&db, "/src/a.py")
.literal(&db)
.last_definition(&db);
let sig = func.internal_signature(&db, None);
let sig = func.signature(&db, None);
assert_eq!(sig.return_ty.unwrap().display(&db).to_string(), "bytes");
assert_params(
@@ -1633,9 +1633,11 @@ mod tests {
",
)
.unwrap();
let func = get_function_f(&db, "/src/a.py");
let func = get_function_f(&db, "/src/a.py")
.literal(&db)
.last_definition(&db);
let sig = func.internal_signature(&db, None);
let sig = func.signature(&db, None);
let [
Parameter {
@@ -1669,9 +1671,11 @@ mod tests {
",
)
.unwrap();
let func = get_function_f(&db, "/src/a.pyi");
let func = get_function_f(&db, "/src/a.pyi")
.literal(&db)
.last_definition(&db);
let sig = func.internal_signature(&db, None);
let sig = func.signature(&db, None);
let [
Parameter {
@@ -1705,9 +1709,11 @@ mod tests {
",
)
.unwrap();
let func = get_function_f(&db, "/src/a.py");
let func = get_function_f(&db, "/src/a.py")
.literal(&db)
.last_definition(&db);
let sig = func.internal_signature(&db, None);
let sig = func.signature(&db, None);
let [
Parameter {
@@ -1751,9 +1757,11 @@ mod tests {
",
)
.unwrap();
let func = get_function_f(&db, "/src/a.pyi");
let func = get_function_f(&db, "/src/a.pyi")
.literal(&db)
.last_definition(&db);
let sig = func.internal_signature(&db, None);
let sig = func.signature(&db, None);
let [
Parameter {
@@ -1789,15 +1797,13 @@ mod tests {
.unwrap();
let func = get_function_f(&db, "/src/a.py");
let expected_sig = func.internal_signature(&db, None);
let overload = func.literal(&db).last_definition(&db);
let expected_sig = overload.signature(&db, None);
// With no decorators, internal and external signature are the same
assert_eq!(
func.signature(&db),
&FunctionSignature {
overloads: CallableSignature::single(expected_sig),
implementation: None
},
&CallableSignature::single(expected_sig)
);
}
}

View File

@@ -1,7 +1,7 @@
use ruff_python_ast as ast;
use crate::db::Db;
use crate::symbol::{Boundness, Symbol};
use crate::place::{Boundness, Place};
use crate::types::class_base::ClassBase;
use crate::types::diagnostic::report_base_with_incompatible_slots;
use crate::types::{ClassLiteral, Type};
@@ -24,7 +24,7 @@ enum SlotsKind {
impl SlotsKind {
fn from(db: &dyn Db, base: ClassLiteral) -> Self {
let Symbol::Type(slots_ty, bound) = base.own_class_member(db, None, "__slots__").symbol
let Place::Type(slots_ty, bound) = base.own_class_member(db, None, "__slots__").place
else {
return Self::NotSpecified;
};

View File

@@ -1,4 +1,4 @@
use crate::symbol::SymbolAndQualifiers;
use crate::place::PlaceAndQualifiers;
use crate::types::{
ClassType, DynamicType, KnownClass, MemberLookupPolicy, Type, TypeMapping, TypeVarInstance,
};
@@ -99,7 +99,7 @@ impl<'db> SubclassOfType<'db> {
db: &'db dyn Db,
name: &str,
policy: MemberLookupPolicy,
) -> Option<SymbolAndQualifiers<'db>> {
) -> Option<PlaceAndQualifiers<'db>> {
Type::from(self.subclass_of).find_name_in_mro_with_policy(db, name, policy)
}

View File

@@ -7,7 +7,7 @@ use ruff_python_ast::{self as ast, AnyNodeRef};
use crate::Db;
use crate::semantic_index::ast_ids::{HasScopedExpressionId, ScopedExpressionId};
use crate::semantic_index::symbol::ScopeId;
use crate::semantic_index::place::ScopeId;
use crate::types::{Type, TypeCheckDiagnostics, infer_expression_types};
use crate::unpack::{UnpackKind, UnpackValue};
@@ -84,7 +84,7 @@ impl<'db> Unpacker<'db> {
value_ty: Type<'db>,
) {
match target {
ast::Expr::Name(_) | ast::Expr::Attribute(_) => {
ast::Expr::Name(_) | ast::Expr::Attribute(_) | ast::Expr::Subscript(_) => {
self.targets.insert(
target.scoped_expression_id(self.db(), self.target_scope),
value_ty,
@@ -192,8 +192,15 @@ impl<'db> Unpacker<'db> {
err.fallback_element_type(self.db())
})
};
for target_type in &mut target_types {
target_type.push(ty);
// Both `elts` and `target_types` are guaranteed to have the same length.
for (element, target_type) in elts.iter().zip(&mut target_types) {
if element.is_starred_expr() {
target_type.push(
KnownClass::List.to_specialized_instance(self.db(), [ty]),
);
} else {
target_type.push(ty);
}
}
}
}

View File

@@ -6,7 +6,7 @@ use crate::Db;
use crate::ast_node_ref::AstNodeRef;
use crate::semantic_index::ast_ids::{HasScopedExpressionId, ScopedExpressionId};
use crate::semantic_index::expression::Expression;
use crate::semantic_index::symbol::{FileScopeId, ScopeId};
use crate::semantic_index::place::{FileScopeId, ScopeId};
/// This ingredient represents a single unpacking.
///

View File

@@ -180,6 +180,7 @@ impl Server {
completion_provider: experimental
.is_some_and(Experimental::is_completions_enabled)
.then_some(lsp_types::CompletionOptions {
trigger_characters: Some(vec!['.'.to_string()]),
..Default::default()
}),
..Default::default()

View File

@@ -272,7 +272,7 @@ fn run_test(
python_path: configuration
.python()
.map(|sys_prefix| {
PythonPath::SysPrefix(
PythonPath::IntoSysPrefix(
sys_prefix.to_path_buf(),
SysPrefixPathOrigin::PythonCliFlag,
)

View File

@@ -30,7 +30,7 @@ ty_python_semantic = { path = "../crates/ty_python_semantic" }
ty_vendored = { path = "../crates/ty_vendored" }
libfuzzer-sys = { git = "https://github.com/rust-fuzz/libfuzzer", default-features = false }
salsa = { git = "https://github.com/salsa-rs/salsa.git", rev = "2b5188778e91a5ab50cb7d827148caf7eb2f4630" }
salsa = { git = "https://github.com/carljm/salsa.git", rev = "0f6d406f6c309964279baef71588746b8c67b4a3" }
similar = { version = "2.5.0" }
tracing = { version = "0.1.40" }

View File

@@ -179,7 +179,7 @@ class PlaygroundServer
monaco.languages.registerDocumentFormattingEditProvider("python", this);
}
triggerCharacters: undefined;
triggerCharacters: string[] = ["."];
provideCompletionItems(
model: editor.ITextModel,