Compare commits

...

11 Commits

Author SHA1 Message Date
Brent Westbrook
47a5543109 chore: stabilize RUF059 2025-06-05 12:33:28 -04:00
Andrew Gallant
55100209c7 [ty] IDE: add support for object.<CURSOR> completions (#18468)
This PR adds logic for detecting `Name Dot [Name]` token patterns,
finding the corresponding `ExprAttribute`, getting the type of the
object and returning the members available on that object.

Here's a video demonstrating this working:

https://github.com/user-attachments/assets/42ce78e8-5930-4211-a18a-fa2a0434d0eb

Ref astral-sh/ty#86
2025-06-05 11:15:19 -04:00
chiri
c0bb83b882 [perflint] fix missing parentheses for lambda and ternary conditions (PERF401, PERF403) (#18412)
Closes #18405
2025-06-05 09:57:08 -05:00
Brent Westbrook
74a4e9af3d Combine lint and syntax error handling (#18471)
## Summary

This is a spin-off from
https://github.com/astral-sh/ruff/pull/18447#discussion_r2125844669 to
avoid using `Message::noqa_code` to differentiate between lints and
syntax errors. I went through all of the calls on `main` and on the
branch from #18447, and the instance in `ruff_server` noted in the
linked comment was actually the primary place where this was being done.
Other calls to `noqa_code` are typically some variation of
`message.noqa_code().map_or(String::new, format!(...))`, with the major
exception of the gitlab output format:


a120610b5b/crates/ruff_linter/src/message/gitlab.rs (L93-L105)

which obviously assumes that `None` means syntax error. A simple fix
here would be to use `message.name()` for `check_name` instead of the
noqa code, but I'm not sure how breaking that would be. This could just
be:

```rust
 let description = message.body();
 let description = description.strip_prefix("SyntaxError: ").unwrap_or(description).to_string();
 let check_name = message.name();
```

In that case. This sounds reasonable based on the [Code Quality report
format](https://docs.gitlab.com/ci/testing/code_quality/#code-quality-report-format)
docs:

> | Name | Type | Description|
> |-----|-----|----|
> |`check_name` | String | A unique name representing the check, or
rule, associated with this violation. |

## Test Plan

Existing tests
2025-06-05 12:50:02 +00:00
Alex Waygood
8485dbb324 [ty] Fix --python argument for Windows, and improve error messages for bad --python arguments (#18457)
## Summary

Fixes https://github.com/astral-sh/ty/issues/556.

On Windows, system installations have different layouts to virtual
environments. In Windows virtual environments, the Python executable is
found at `<sys.prefix>/Scripts/python.exe`. But in Windows system
installations, the Python executable is found at
`<sys.prefix>/python.exe`. That means that Windows users were able to
point to Python executables inside virtual environments with the
`--python` flag, but they weren't able to point to Python executables
inside system installations.

This PR fixes that issue. It also makes a couple of other changes:
- Nearly all `sys.prefix` resolution is moved inside `site_packages.rs`.
That was the original design of the `site-packages` resolution logic,
but features implemented since the initial implementation have added
some resolution and validation to `resolver.rs` inside the module
resolver. That means that we've ended up with a somewhat confusing code
structure and a situation where several checks are unnecessarily
duplicated between the two modules.
- I noticed that we had quite bad error messages if you e.g. pointed to
a path that didn't exist on disk with `--python` (we just gave a
somewhat impenetrable message saying that we "failed to canonicalize"
the path). I improved the error messages here and added CLI tests for
`--python` and the `environment.python` configuration setting.

## Test Plan

- Existing tests pass
- Added new CLI tests
- I manually checked that virtual-environment discovery still works if
no configuration is given
- Micha did some manual testing to check that pointing `--python` to a
system-installation executable now works on Windows
2025-06-05 08:19:15 +01:00
Shunsuke Shibayama
0858896bc4 [ty] type narrowing by attribute/subscript assignments (#18041)
## Summary

This PR partially solves https://github.com/astral-sh/ty/issues/164
(derived from #17643).

Currently, the definitions we manage are limited to those for simple
name (symbol) targets, but we expand this to track definitions for
attribute and subscript targets as well.

This was originally planned as part of the work in #17643, but the
changes are significant, so I made it a separate PR.
After merging this PR, I will reflect this changes in #17643.

There is still some incomplete work remaining, but the basic features
have been implemented, so I am publishing it as a draft PR.
Here is the TODO list (there may be more to come):
* [x] Complete rewrite and refactoring of documentation (removing
`Symbol` and replacing it with `Place`)
* [x] More thorough testing
* [x] Consolidation of duplicated code (maybe we can consolidate the
handling related to name, attribute, and subscript)

This PR replaces the current `Symbol` API with the `Place` API, which is
a concept that includes attributes and subscripts (the term is borrowed
from Rust).

## Test Plan

`mdtest/narrow/assignment.md` is added.

---------

Co-authored-by: David Peter <sharkdp@users.noreply.github.com>
Co-authored-by: Carl Meyer <carl@astral.sh>
2025-06-04 17:24:27 -07:00
Alex Waygood
ce8b744f17 [ty] Only calculate information for unresolved-reference subdiagnostic if we know we'll emit the diagnostic (#18465)
## Summary

This optimizes some of the logic added in
https://github.com/astral-sh/ruff/pull/18444. In general, we only
calculate information for subdiagnostics if we know we'll actually emit
the diagnostic. The check to see whether we'll emit the diagnostic is
work we'll definitely have to do whereas the the work to gather
information for a subdiagnostic isn't work we necessarily have to do if
the diagnostic isn't going to be emitted at all.

This PR makes us lazier about gathering the information we need for the
subdiagnostic, and moves all the subdiagnostic logic into one function
rather than having some `unresolved-reference` subdiagnostic logic in
`infer.rs` and some in `diagnostic.rs`.

## Test Plan

`cargo test -p ty_python_semantic`
2025-06-04 20:41:00 +01:00
Alex Waygood
5a8cdab771 [ty] Only consider a type T a subtype of a protocol P if all of P's members are fully bound on T (#18466)
## Summary

Fixes https://github.com/astral-sh/ty/issues/578

## Test Plan

mdtests
2025-06-04 19:39:14 +00:00
Alex Waygood
3a8191529c [ty] Exclude members starting with _abc_ from a protocol interface (#18467)
## Summary

As well as excluding a hardcoded set of special attributes, CPython at
runtime also excludes any attributes or declarations starting with
`_abc_` from the set of members that make up a protocol interface. I
missed this in my initial implementation.

This is a bit of a CPython implementation detail, but I do think it's
important that we try to model the runtime as best we can here. The
closer we are to the runtime behaviour, the closer we come to sound
behaviour when narrowing types from `isinstance()` checks against
runtime-checkable protocols (for example)

## Test Plan

Extended an existing mdtest
2025-06-04 20:34:09 +01:00
lipefree
e658778ced [ty] Add subdiagnostic suggestion to unresolved-reference diagnostic when variable exists on self (#18444)
## Summary

Closes https://github.com/astral-sh/ty/issues/502.

In the following example:
```py
class Foo:
    x: int

    def method(self):
        y = x
```
The user may intended to use `y = self.x` in `method`. 

This is now added as a subdiagnostic in the following form : 

`info: An attribute with the same name as 'x' is defined, consider using
'self.x'`

## Test Plan

Added mdtest with snapshot diagnostics.
2025-06-04 08:13:50 -07:00
David Peter
f1883d71a4 [ty] IDE: only provide declarations and bindings as completions (#18456)
## Summary

Previously, all symbols where provided as possible completions. In an
example like the following, both `foo` and `f` were suggested as
completions, because `f` itself is a symbol.
```py
foo = 1

f<CURSOR>
```
Similarly, in the following example, `hidden_symbol` was suggested, even
though it is not statically visible:
```py
if 1 + 2 != 3:
    hidden_symbol = 1

hidden_<CURSOR>
```

With the change suggested here, we only use statically visible
declarations and bindings as a source for completions.


## Test Plan

- Updated snapshot tests
- New test for statically hidden definitions
- Added test for star import
2025-06-04 16:11:05 +02:00
65 changed files with 4746 additions and 2808 deletions

View File

@@ -566,7 +566,7 @@ fn venv() -> Result<()> {
----- stderr -----
ruff failed
Cause: Invalid search path settings
Cause: Failed to discover the site-packages directory: Invalid `--python` argument: `none` could not be canonicalized
Cause: Failed to discover the site-packages directory: Invalid `--python` argument: `none` does not point to a Python executable or a directory on disk
");
});

View File

@@ -10,7 +10,7 @@ use ruff_python_ast::PythonVersion;
use ty_python_semantic::lint::{LintRegistry, RuleSelection};
use ty_python_semantic::{
Db, Program, ProgramSettings, PythonPath, PythonPlatform, PythonVersionSource,
PythonVersionWithSource, SearchPathSettings, default_lint_registry,
PythonVersionWithSource, SearchPathSettings, SysPrefixPathOrigin, default_lint_registry,
};
static EMPTY_VENDORED: std::sync::LazyLock<VendoredFileSystem> = std::sync::LazyLock::new(|| {
@@ -37,7 +37,8 @@ impl ModuleDb {
) -> Result<Self> {
let mut search_paths = SearchPathSettings::new(src_roots);
if let Some(venv_path) = venv_path {
search_paths.python_path = PythonPath::from_cli_flag(venv_path);
search_paths.python_path =
PythonPath::sys_prefix(venv_path, SysPrefixPathOrigin::PythonCliFlag);
}
let db = Self::default();

View File

@@ -266,3 +266,15 @@ def f():
result = list() # this should be replaced with a comprehension
for i in values:
result.append(i + 1) # PERF401
def f():
src = [1]
dst = []
for i in src:
if True if True else False:
dst.append(i)
for i in src:
if lambda: 0:
dst.append(i)

View File

@@ -151,3 +151,16 @@ def foo():
result = {}
for idx, name in indices, fruit:
result[name] = idx # PERF403
def foo():
src = (("x", 1),)
dst = {}
for k, v in src:
if True if True else False:
dst[k] = v
for k, v in src:
if lambda: 0:
dst[k] = v

View File

@@ -1025,7 +1025,7 @@ pub fn code_to_rule(linter: Linter, code: &str) -> Option<(RuleGroup, Rule)> {
(Ruff, "056") => (RuleGroup::Preview, rules::ruff::rules::FalsyDictGetFallback),
(Ruff, "057") => (RuleGroup::Preview, rules::ruff::rules::UnnecessaryRound),
(Ruff, "058") => (RuleGroup::Preview, rules::ruff::rules::StarmapZip),
(Ruff, "059") => (RuleGroup::Preview, rules::ruff::rules::UnusedUnpackedVariable),
(Ruff, "059") => (RuleGroup::Stable, rules::ruff::rules::UnusedUnpackedVariable),
(Ruff, "060") => (RuleGroup::Preview, rules::ruff::rules::InEmptyCollection),
(Ruff, "100") => (RuleGroup::Stable, rules::ruff::rules::UnusedNOQA),
(Ruff, "101") => (RuleGroup::Stable, rules::ruff::rules::RedirectedNOQA),

View File

@@ -346,10 +346,11 @@ fn convert_to_dict_comprehension(
// since if the assignment expression appears
// internally (e.g. as an operand in a boolean
// operation) then it will already be parenthesized.
if test.is_named_expr() {
format!(" if ({})", locator.slice(test.range()))
} else {
format!(" if {}", locator.slice(test.range()))
match test {
Expr::Named(_) | Expr::If(_) | Expr::Lambda(_) => {
format!(" if ({})", locator.slice(test.range()))
}
_ => format!(" if {}", locator.slice(test.range())),
}
}
None => String::new(),

View File

@@ -358,7 +358,7 @@ fn convert_to_list_extend(
fix_type: ComprehensionType,
binding: &Binding,
for_stmt: &ast::StmtFor,
if_test: Option<&ast::Expr>,
if_test: Option<&Expr>,
to_append: &Expr,
checker: &Checker,
) -> Result<Fix> {
@@ -374,10 +374,11 @@ fn convert_to_list_extend(
// since if the assignment expression appears
// internally (e.g. as an operand in a boolean
// operation) then it will already be parenthesized.
if test.is_named_expr() {
format!(" if ({})", locator.slice(test.range()))
} else {
format!(" if {}", locator.slice(test.range()))
match test {
Expr::Named(_) | Expr::If(_) | Expr::Lambda(_) => {
format!(" if ({})", locator.slice(test.range()))
}
_ => format!(" if {}", locator.slice(test.range())),
}
}
None => String::new(),

View File

@@ -219,5 +219,27 @@ PERF401.py:268:9: PERF401 Use a list comprehension to create a transformed list
267 | for i in values:
268 | result.append(i + 1) # PERF401
| ^^^^^^^^^^^^^^^^^^^^ PERF401
269 |
270 | def f():
|
= help: Replace for loop with list comprehension
PERF401.py:276:13: PERF401 Use a list comprehension to create a transformed list
|
274 | for i in src:
275 | if True if True else False:
276 | dst.append(i)
| ^^^^^^^^^^^^^ PERF401
277 |
278 | for i in src:
|
= help: Replace for loop with list comprehension
PERF401.py:280:13: PERF401 Use `list.extend` to create a transformed list
|
278 | for i in src:
279 | if lambda: 0:
280 | dst.append(i)
| ^^^^^^^^^^^^^ PERF401
|
= help: Replace for loop with list.extend

View File

@@ -128,3 +128,23 @@ PERF403.py:153:9: PERF403 Use a dictionary comprehension instead of a for-loop
| ^^^^^^^^^^^^^^^^^^ PERF403
|
= help: Replace for loop with dict comprehension
PERF403.py:162:13: PERF403 Use a dictionary comprehension instead of a for-loop
|
160 | for k, v in src:
161 | if True if True else False:
162 | dst[k] = v
| ^^^^^^^^^^ PERF403
163 |
164 | for k, v in src:
|
= help: Replace for loop with dict comprehension
PERF403.py:166:13: PERF403 Use a dictionary comprehension instead of a for-loop
|
164 | for k, v in src:
165 | if lambda: 0:
166 | dst[k] = v
| ^^^^^^^^^^ PERF403
|
= help: Replace for loop with dict comprehension

View File

@@ -517,6 +517,8 @@ PERF401.py:268:9: PERF401 [*] Use a list comprehension to create a transformed l
267 | for i in values:
268 | result.append(i + 1) # PERF401
| ^^^^^^^^^^^^^^^^^^^^ PERF401
269 |
270 | def f():
|
= help: Replace for loop with list comprehension
@@ -529,3 +531,49 @@ PERF401.py:268:9: PERF401 [*] Use a list comprehension to create a transformed l
268 |- result.append(i + 1) # PERF401
266 |+ # this should be replaced with a comprehension
267 |+ result = [i + 1 for i in values] # PERF401
269 268 |
270 269 | def f():
271 270 | src = [1]
PERF401.py:276:13: PERF401 [*] Use a list comprehension to create a transformed list
|
274 | for i in src:
275 | if True if True else False:
276 | dst.append(i)
| ^^^^^^^^^^^^^ PERF401
277 |
278 | for i in src:
|
= help: Replace for loop with list comprehension
Unsafe fix
269 269 |
270 270 | def f():
271 271 | src = [1]
272 |- dst = []
273 272 |
274 |- for i in src:
275 |- if True if True else False:
276 |- dst.append(i)
273 |+ dst = [i for i in src if (True if True else False)]
277 274 |
278 275 | for i in src:
279 276 | if lambda: 0:
PERF401.py:280:13: PERF401 [*] Use `list.extend` to create a transformed list
|
278 | for i in src:
279 | if lambda: 0:
280 | dst.append(i)
| ^^^^^^^^^^^^^ PERF401
|
= help: Replace for loop with list.extend
Unsafe fix
275 275 | if True if True else False:
276 276 | dst.append(i)
277 277 |
278 |- for i in src:
279 |- if lambda: 0:
280 |- dst.append(i)
278 |+ dst.extend(i for i in src if (lambda: 0))

View File

@@ -305,3 +305,49 @@ PERF403.py:153:9: PERF403 [*] Use a dictionary comprehension instead of a for-lo
152 |- for idx, name in indices, fruit:
153 |- result[name] = idx # PERF403
151 |+ result = {name: idx for idx, name in (indices, fruit)} # PERF403
154 152 |
155 153 |
156 154 | def foo():
PERF403.py:162:13: PERF403 [*] Use a dictionary comprehension instead of a for-loop
|
160 | for k, v in src:
161 | if True if True else False:
162 | dst[k] = v
| ^^^^^^^^^^ PERF403
163 |
164 | for k, v in src:
|
= help: Replace for loop with dict comprehension
Unsafe fix
155 155 |
156 156 | def foo():
157 157 | src = (("x", 1),)
158 |- dst = {}
159 158 |
160 |- for k, v in src:
161 |- if True if True else False:
162 |- dst[k] = v
159 |+ dst = {k: v for k, v in src if (True if True else False)}
163 160 |
164 161 | for k, v in src:
165 162 | if lambda: 0:
PERF403.py:166:13: PERF403 [*] Use `dict.update` instead of a for-loop
|
164 | for k, v in src:
165 | if lambda: 0:
166 | dst[k] = v
| ^^^^^^^^^^ PERF403
|
= help: Replace for loop with `dict.update`
Unsafe fix
161 161 | if True if True else False:
162 162 | dst[k] = v
163 163 |
164 |- for k, v in src:
165 |- if lambda: 0:
166 |- dst[k] = v
164 |+ dst.update({k: v for k, v in src if (lambda: 0)})

View File

@@ -12,7 +12,6 @@ use crate::{
use ruff_diagnostics::{Applicability, Edit, Fix};
use ruff_linter::{
Locator,
codes::Rule,
directives::{Flags, extract_directives},
generate_noqa_edits,
linter::check_path,
@@ -166,26 +165,17 @@ pub(crate) fn check(
messages
.into_iter()
.zip(noqa_edits)
.filter_map(|(message, noqa_edit)| match message.to_rule() {
Some(rule) => Some(to_lsp_diagnostic(
rule,
&message,
noqa_edit,
&source_kind,
locator.to_index(),
encoding,
)),
None => {
if show_syntax_errors {
Some(syntax_error_to_lsp_diagnostic(
&message,
&source_kind,
locator.to_index(),
encoding,
))
} else {
None
}
.filter_map(|(message, noqa_edit)| {
if message.is_syntax_error() && !show_syntax_errors {
None
} else {
Some(to_lsp_diagnostic(
&message,
noqa_edit,
&source_kind,
locator.to_index(),
encoding,
))
}
});
@@ -241,7 +231,6 @@ pub(crate) fn fixes_for_diagnostics(
/// Generates an LSP diagnostic with an associated cell index for the diagnostic to go in.
/// If the source kind is a text document, the cell index will always be `0`.
fn to_lsp_diagnostic(
rule: Rule,
diagnostic: &Message,
noqa_edit: Option<Edit>,
source_kind: &SourceKind,
@@ -253,11 +242,13 @@ fn to_lsp_diagnostic(
let body = diagnostic.body().to_string();
let fix = diagnostic.fix();
let suggestion = diagnostic.suggestion();
let code = diagnostic.to_noqa_code();
let fix = fix.and_then(|fix| fix.applies(Applicability::Unsafe).then_some(fix));
let data = (fix.is_some() || noqa_edit.is_some())
.then(|| {
let code = code?.to_string();
let edits = fix
.into_iter()
.flat_map(Fix::edits)
@@ -274,14 +265,12 @@ fn to_lsp_diagnostic(
title: suggestion.unwrap_or(name).to_string(),
noqa_edit,
edits,
code: rule.noqa_code().to_string(),
code,
})
.ok()
})
.flatten();
let code = rule.noqa_code().to_string();
let range: lsp_types::Range;
let cell: usize;
@@ -297,14 +286,25 @@ fn to_lsp_diagnostic(
range = diagnostic_range.to_range(source_kind.source_code(), index, encoding);
}
let (severity, tags, code) = if let Some(code) = code {
let code = code.to_string();
(
Some(severity(&code)),
tags(&code),
Some(lsp_types::NumberOrString::String(code)),
)
} else {
(None, None, None)
};
(
cell,
lsp_types::Diagnostic {
range,
severity: Some(severity(&code)),
tags: tags(&code),
code: Some(lsp_types::NumberOrString::String(code)),
code_description: rule.url().and_then(|url| {
severity,
tags,
code,
code_description: diagnostic.to_url().and_then(|url| {
Some(lsp_types::CodeDescription {
href: lsp_types::Url::parse(&url).ok()?,
})
@@ -317,45 +317,6 @@ fn to_lsp_diagnostic(
)
}
fn syntax_error_to_lsp_diagnostic(
syntax_error: &Message,
source_kind: &SourceKind,
index: &LineIndex,
encoding: PositionEncoding,
) -> (usize, lsp_types::Diagnostic) {
let range: lsp_types::Range;
let cell: usize;
if let Some(notebook_index) = source_kind.as_ipy_notebook().map(Notebook::index) {
NotebookRange { cell, range } = syntax_error.range().to_notebook_range(
source_kind.source_code(),
index,
notebook_index,
encoding,
);
} else {
cell = usize::default();
range = syntax_error
.range()
.to_range(source_kind.source_code(), index, encoding);
}
(
cell,
lsp_types::Diagnostic {
range,
severity: Some(lsp_types::DiagnosticSeverity::ERROR),
tags: None,
code: None,
code_description: None,
source: Some(DIAGNOSTIC_NAME.into()),
message: syntax_error.body().to_string(),
related_information: None,
data: None,
},
)
}
fn diagnostic_edit_range(
range: TextRange,
source_kind: &SourceKind,

View File

@@ -207,36 +207,23 @@ impl Workspace {
let messages: Vec<ExpandedMessage> = messages
.into_iter()
.map(|msg| {
let message = msg.body().to_string();
let range = msg.range();
match msg.to_noqa_code() {
Some(code) => ExpandedMessage {
code: Some(code.to_string()),
message,
start_location: source_code.line_column(range.start()).into(),
end_location: source_code.line_column(range.end()).into(),
fix: msg.fix().map(|fix| ExpandedFix {
message: msg.suggestion().map(ToString::to_string),
edits: fix
.edits()
.iter()
.map(|edit| ExpandedEdit {
location: source_code.line_column(edit.start()).into(),
end_location: source_code.line_column(edit.end()).into(),
content: edit.content().map(ToString::to_string),
})
.collect(),
}),
},
None => ExpandedMessage {
code: None,
message,
start_location: source_code.line_column(range.start()).into(),
end_location: source_code.line_column(range.end()).into(),
fix: None,
},
}
.map(|msg| ExpandedMessage {
code: msg.to_noqa_code().map(|code| code.to_string()),
message: msg.body().to_string(),
start_location: source_code.line_column(msg.start()).into(),
end_location: source_code.line_column(msg.end()).into(),
fix: msg.fix().map(|fix| ExpandedFix {
message: msg.suggestion().map(ToString::to_string),
edits: fix
.edits()
.iter()
.map(|edit| ExpandedEdit {
location: source_code.line_column(edit.start()).into(),
end_location: source_code.line_column(edit.end()).into(),
content: edit.content().map(ToString::to_string),
})
.collect(),
}),
})
.collect();

108
crates/ty/docs/rules.md generated
View File

@@ -52,7 +52,7 @@ Calling a non-callable object will raise a `TypeError` at runtime.
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20call-non-callable)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L93)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L92)
</details>
## `conflicting-argument-forms`
@@ -83,7 +83,7 @@ f(int) # error
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20conflicting-argument-forms)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L137)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L136)
</details>
## `conflicting-declarations`
@@ -113,7 +113,7 @@ a = 1
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20conflicting-declarations)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L163)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L162)
</details>
## `conflicting-metaclass`
@@ -144,7 +144,7 @@ class C(A, B): ...
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20conflicting-metaclass)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L188)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L187)
</details>
## `cyclic-class-definition`
@@ -175,7 +175,7 @@ class B(A): ...
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20cyclic-class-definition)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L214)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L213)
</details>
## `duplicate-base`
@@ -201,7 +201,7 @@ class B(A, A): ...
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20duplicate-base)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L258)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L257)
</details>
## `escape-character-in-forward-annotation`
@@ -338,7 +338,7 @@ TypeError: multiple bases have instance lay-out conflict
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20incompatible-slots)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L279)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L278)
</details>
## `inconsistent-mro`
@@ -367,7 +367,7 @@ class C(A, B): ...
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20inconsistent-mro)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L365)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L364)
</details>
## `index-out-of-bounds`
@@ -392,7 +392,7 @@ t[3] # IndexError: tuple index out of range
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20index-out-of-bounds)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L389)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L388)
</details>
## `invalid-argument-type`
@@ -418,7 +418,7 @@ func("foo") # error: [invalid-argument-type]
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20invalid-argument-type)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L409)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L408)
</details>
## `invalid-assignment`
@@ -445,7 +445,7 @@ a: int = ''
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20invalid-assignment)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L449)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L448)
</details>
## `invalid-attribute-access`
@@ -478,7 +478,7 @@ C.instance_var = 3 # error: Cannot assign to instance variable
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20invalid-attribute-access)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1397)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1396)
</details>
## `invalid-base`
@@ -501,7 +501,7 @@ class A(42): ... # error: [invalid-base]
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20invalid-base)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L471)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L470)
</details>
## `invalid-context-manager`
@@ -527,7 +527,7 @@ with 1:
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20invalid-context-manager)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L522)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L521)
</details>
## `invalid-declaration`
@@ -555,7 +555,7 @@ a: str
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20invalid-declaration)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L543)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L542)
</details>
## `invalid-exception-caught`
@@ -596,7 +596,7 @@ except ZeroDivisionError:
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20invalid-exception-caught)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L566)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L565)
</details>
## `invalid-generic-class`
@@ -627,7 +627,7 @@ class C[U](Generic[T]): ...
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20invalid-generic-class)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L602)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L601)
</details>
## `invalid-legacy-type-variable`
@@ -660,7 +660,7 @@ def f(t: TypeVar("U")): ...
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20invalid-legacy-type-variable)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L628)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L627)
</details>
## `invalid-metaclass`
@@ -692,7 +692,7 @@ class B(metaclass=f): ...
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20invalid-metaclass)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L677)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L676)
</details>
## `invalid-overload`
@@ -740,7 +740,7 @@ def foo(x: int) -> int: ...
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20invalid-overload)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L704)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L703)
</details>
## `invalid-parameter-default`
@@ -765,7 +765,7 @@ def f(a: int = ''): ...
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20invalid-parameter-default)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L747)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L746)
</details>
## `invalid-protocol`
@@ -798,7 +798,7 @@ TypeError: Protocols can only inherit from other protocols, got <class 'int'>
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20invalid-protocol)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L337)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L336)
</details>
## `invalid-raise`
@@ -846,7 +846,7 @@ def g():
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20invalid-raise)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L767)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L766)
</details>
## `invalid-return-type`
@@ -870,7 +870,7 @@ def func() -> int:
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20invalid-return-type)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L430)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L429)
</details>
## `invalid-super-argument`
@@ -914,7 +914,7 @@ super(B, A) # error: `A` does not satisfy `issubclass(A, B)`
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20invalid-super-argument)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L810)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L809)
</details>
## `invalid-syntax-in-forward-annotation`
@@ -954,7 +954,7 @@ NewAlias = TypeAliasType(get_name(), int) # error: TypeAliasType name mus
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20invalid-type-alias-type)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L656)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L655)
</details>
## `invalid-type-checking-constant`
@@ -983,7 +983,7 @@ TYPE_CHECKING = ''
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20invalid-type-checking-constant)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L849)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L848)
</details>
## `invalid-type-form`
@@ -1012,7 +1012,7 @@ b: Annotated[int] # `Annotated` expects at least two arguments
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20invalid-type-form)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L873)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L872)
</details>
## `invalid-type-variable-constraints`
@@ -1046,7 +1046,7 @@ T = TypeVar('T', bound=str) # valid bound TypeVar
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20invalid-type-variable-constraints)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L897)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L896)
</details>
## `missing-argument`
@@ -1070,7 +1070,7 @@ func() # TypeError: func() missing 1 required positional argument: 'x'
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20missing-argument)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L926)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L925)
</details>
## `no-matching-overload`
@@ -1098,7 +1098,7 @@ func("string") # error: [no-matching-overload]
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20no-matching-overload)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L945)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L944)
</details>
## `non-subscriptable`
@@ -1121,7 +1121,7 @@ Subscripting an object that does not support it will raise a `TypeError` at runt
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20non-subscriptable)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L968)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L967)
</details>
## `not-iterable`
@@ -1146,7 +1146,7 @@ for i in 34: # TypeError: 'int' object is not iterable
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20not-iterable)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L986)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L985)
</details>
## `parameter-already-assigned`
@@ -1172,7 +1172,7 @@ f(1, x=2) # Error raised here
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20parameter-already-assigned)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1037)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1036)
</details>
## `raw-string-type-annotation`
@@ -1231,7 +1231,7 @@ static_assert(int(2.0 * 3.0) == 6) # error: does not have a statically known tr
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20static-assert-error)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1373)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1372)
</details>
## `subclass-of-final-class`
@@ -1259,7 +1259,7 @@ class B(A): ... # Error raised here
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20subclass-of-final-class)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1128)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1127)
</details>
## `too-many-positional-arguments`
@@ -1285,7 +1285,7 @@ f("foo") # Error raised here
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20too-many-positional-arguments)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1173)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1172)
</details>
## `type-assertion-failure`
@@ -1312,7 +1312,7 @@ def _(x: int):
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20type-assertion-failure)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1151)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1150)
</details>
## `unavailable-implicit-super-arguments`
@@ -1356,7 +1356,7 @@ class A:
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20unavailable-implicit-super-arguments)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1194)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1193)
</details>
## `unknown-argument`
@@ -1382,7 +1382,7 @@ f(x=1, y=2) # Error raised here
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20unknown-argument)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1251)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1250)
</details>
## `unresolved-attribute`
@@ -1409,7 +1409,7 @@ A().foo # AttributeError: 'A' object has no attribute 'foo'
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20unresolved-attribute)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1272)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1271)
</details>
## `unresolved-import`
@@ -1433,7 +1433,7 @@ import foo # ModuleNotFoundError: No module named 'foo'
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20unresolved-import)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1294)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1293)
</details>
## `unresolved-reference`
@@ -1457,7 +1457,7 @@ print(x) # NameError: name 'x' is not defined
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20unresolved-reference)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1313)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1312)
</details>
## `unsupported-bool-conversion`
@@ -1493,7 +1493,7 @@ b1 < b2 < b1 # exception raised here
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20unsupported-bool-conversion)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1006)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1005)
</details>
## `unsupported-operator`
@@ -1520,7 +1520,7 @@ A() + A() # TypeError: unsupported operand type(s) for +: 'A' and 'A'
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20unsupported-operator)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1332)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1331)
</details>
## `zero-stepsize-in-slice`
@@ -1544,7 +1544,7 @@ l[1:10:0] # ValueError: slice step cannot be zero
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20zero-stepsize-in-slice)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1354)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1353)
</details>
## `invalid-ignore-comment`
@@ -1600,7 +1600,7 @@ A.c # AttributeError: type object 'A' has no attribute 'c'
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20possibly-unbound-attribute)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1058)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1057)
</details>
## `possibly-unbound-implicit-call`
@@ -1631,7 +1631,7 @@ A()[0] # TypeError: 'A' object is not subscriptable
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20possibly-unbound-implicit-call)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L111)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L110)
</details>
## `possibly-unbound-import`
@@ -1662,7 +1662,7 @@ from module import a # ImportError: cannot import name 'a' from 'module'
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20possibly-unbound-import)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1080)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1079)
</details>
## `redundant-cast`
@@ -1688,7 +1688,7 @@ cast(int, f()) # Redundant
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20redundant-cast)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1425)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1424)
</details>
## `undefined-reveal`
@@ -1711,7 +1711,7 @@ reveal_type(1) # NameError: name 'reveal_type' is not defined
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20undefined-reveal)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1233)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1232)
</details>
## `unknown-rule`
@@ -1779,7 +1779,7 @@ class D(C): ... # error: [unsupported-base]
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20unsupported-base)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L489)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L488)
</details>
## `division-by-zero`
@@ -1802,7 +1802,7 @@ Dividing by zero raises a `ZeroDivisionError` at runtime.
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20division-by-zero)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L240)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L239)
</details>
## `possibly-unresolved-reference`
@@ -1829,7 +1829,7 @@ print(x) # NameError: name 'x' is not defined
### Links
* [Related issues](https://github.com/astral-sh/ty/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20possibly-unresolved-reference)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1106)
* [View source](https://github.com/astral-sh/ruff/blob/main/crates%2Fty_python_semantic%2Fsrc%2Ftypes%2Fdiagnostic.rs#L1105)
</details>
## `unused-ignore-comment`

View File

@@ -918,6 +918,156 @@ fn cli_unknown_rules() -> anyhow::Result<()> {
Ok(())
}
#[test]
fn python_cli_argument_virtual_environment() -> anyhow::Result<()> {
let path_to_executable = if cfg!(windows) {
"my-venv/Scripts/python.exe"
} else {
"my-venv/bin/python"
};
let other_venv_path = "my-venv/foo/some_other_file.txt";
let case = TestCase::with_files([
("test.py", ""),
(
if cfg!(windows) {
"my-venv/Lib/site-packages/foo.py"
} else {
"my-venv/lib/python3.13/site-packages/foo.py"
},
"",
),
(path_to_executable, ""),
(other_venv_path, ""),
])?;
// Passing a path to the installation works
assert_cmd_snapshot!(case.command().arg("--python").arg("my-venv"), @r"
success: true
exit_code: 0
----- stdout -----
All checks passed!
----- stderr -----
WARN ty is pre-release software and not ready for production use. Expect to encounter bugs, missing features, and fatal errors.
");
// And so does passing a path to the executable inside the installation
assert_cmd_snapshot!(case.command().arg("--python").arg(path_to_executable), @r"
success: true
exit_code: 0
----- stdout -----
All checks passed!
----- stderr -----
WARN ty is pre-release software and not ready for production use. Expect to encounter bugs, missing features, and fatal errors.
");
// But random other paths inside the installation are rejected
assert_cmd_snapshot!(case.command().arg("--python").arg(other_venv_path), @r"
success: false
exit_code: 2
----- stdout -----
----- stderr -----
WARN ty is pre-release software and not ready for production use. Expect to encounter bugs, missing features, and fatal errors.
ty failed
Cause: Invalid search path settings
Cause: Failed to discover the site-packages directory: Invalid `--python` argument: `<temp_dir>/my-venv/foo/some_other_file.txt` does not point to a Python executable or a directory on disk
");
// And so are paths that do not exist on disk
assert_cmd_snapshot!(case.command().arg("--python").arg("not-a-directory-or-executable"), @r"
success: false
exit_code: 2
----- stdout -----
----- stderr -----
WARN ty is pre-release software and not ready for production use. Expect to encounter bugs, missing features, and fatal errors.
ty failed
Cause: Invalid search path settings
Cause: Failed to discover the site-packages directory: Invalid `--python` argument: `<temp_dir>/not-a-directory-or-executable` does not point to a Python executable or a directory on disk
");
Ok(())
}
#[test]
fn python_cli_argument_system_installation() -> anyhow::Result<()> {
let path_to_executable = if cfg!(windows) {
"Python3.11/python.exe"
} else {
"Python3.11/bin/python"
};
let case = TestCase::with_files([
("test.py", ""),
(
if cfg!(windows) {
"Python3.11/Lib/site-packages/foo.py"
} else {
"Python3.11/lib/python3.11/site-packages/foo.py"
},
"",
),
(path_to_executable, ""),
])?;
// Passing a path to the installation works
assert_cmd_snapshot!(case.command().arg("--python").arg("Python3.11"), @r"
success: true
exit_code: 0
----- stdout -----
All checks passed!
----- stderr -----
WARN ty is pre-release software and not ready for production use. Expect to encounter bugs, missing features, and fatal errors.
");
// And so does passing a path to the executable inside the installation
assert_cmd_snapshot!(case.command().arg("--python").arg(path_to_executable), @r"
success: true
exit_code: 0
----- stdout -----
All checks passed!
----- stderr -----
WARN ty is pre-release software and not ready for production use. Expect to encounter bugs, missing features, and fatal errors.
");
Ok(())
}
#[test]
fn config_file_broken_python_setting() -> anyhow::Result<()> {
let case = TestCase::with_files([
(
"pyproject.toml",
r#"
[tool.ty.environment]
python = "not-a-directory-or-executable"
"#,
),
("test.py", ""),
])?;
// TODO: this error message should say "invalid `python` configuration setting" rather than "invalid `--python` argument"
assert_cmd_snapshot!(case.command(), @r"
success: false
exit_code: 2
----- stdout -----
----- stderr -----
WARN ty is pre-release software and not ready for production use. Expect to encounter bugs, missing features, and fatal errors.
ty failed
Cause: Invalid search path settings
Cause: Failed to discover the site-packages directory: Invalid `--python` argument: `<temp_dir>/not-a-directory-or-executable` does not point to a Python executable or a directory on disk
");
Ok(())
}
#[test]
fn exit_code_only_warnings() -> anyhow::Result<()> {
let case = TestCase::with_file("test.py", r"print(x) # [unresolved-reference]")?;

View File

@@ -1,10 +1,13 @@
use std::cmp::Ordering;
use ruff_db::files::File;
use ruff_db::parsed::{ParsedModule, parsed_module};
use ruff_python_parser::TokenAt;
use ruff_python_ast as ast;
use ruff_python_parser::{Token, TokenAt, TokenKind};
use ruff_text_size::{Ranged, TextRange, TextSize};
use crate::Db;
use crate::find_node::{CoveringNode, covering_node};
use crate::find_node::covering_node;
#[derive(Debug, Clone)]
pub struct Completion {
@@ -14,13 +17,16 @@ pub struct Completion {
pub fn completion(db: &dyn Db, file: File, offset: TextSize) -> Vec<Completion> {
let parsed = parsed_module(db.upcast(), file);
let Some(target) = find_target(parsed, offset) else {
let Some(target) = CompletionTargetTokens::find(parsed, offset).ast(parsed) else {
return vec![];
};
let model = ty_python_semantic::SemanticModel::new(db.upcast(), file);
let mut completions = model.completions(target.node());
completions.sort();
let mut completions = match target {
CompletionTargetAst::ObjectDot { expr } => model.attribute_completions(expr),
CompletionTargetAst::Scoped { node } => model.scoped_completions(node),
};
completions.sort_by(|name1, name2| compare_suggestions(name1, name2));
completions.dedup();
completions
.into_iter()
@@ -28,30 +34,253 @@ pub fn completion(db: &dyn Db, file: File, offset: TextSize) -> Vec<Completion>
.collect()
}
fn find_target(parsed: &ParsedModule, offset: TextSize) -> Option<CoveringNode> {
let offset = match parsed.tokens().at_offset(offset) {
TokenAt::None => {
return Some(covering_node(
parsed.syntax().into(),
TextRange::empty(offset),
));
/// The kind of tokens identified under the cursor.
#[derive(Debug)]
enum CompletionTargetTokens<'t> {
/// A `object.attribute` token form was found, where
/// `attribute` may be empty.
///
/// This requires a name token followed by a dot token.
ObjectDot {
/// The token preceding the dot.
object: &'t Token,
/// The token, if non-empty, following the dot.
///
/// This is currently unused, but we should use this
/// eventually to remove completions that aren't a
/// prefix of what has already been typed. (We are
/// currently relying on the LSP client to do this.)
#[expect(dead_code)]
attribute: Option<&'t Token>,
},
/// A token was found under the cursor, but it didn't
/// match any of our anticipated token patterns.
Generic { token: &'t Token },
/// No token was found, but we have the offset of the
/// cursor.
Unknown { offset: TextSize },
}
impl<'t> CompletionTargetTokens<'t> {
/// Look for the best matching token pattern at the given offset.
fn find(parsed: &ParsedModule, offset: TextSize) -> CompletionTargetTokens<'_> {
static OBJECT_DOT_EMPTY: [TokenKind; 2] = [TokenKind::Name, TokenKind::Dot];
static OBJECT_DOT_NON_EMPTY: [TokenKind; 3] =
[TokenKind::Name, TokenKind::Dot, TokenKind::Name];
let offset = match parsed.tokens().at_offset(offset) {
TokenAt::None => return CompletionTargetTokens::Unknown { offset },
TokenAt::Single(tok) => tok.end(),
TokenAt::Between(_, tok) => tok.start(),
};
let before = parsed.tokens().before(offset);
if let Some([object, _dot]) = token_suffix_by_kinds(before, OBJECT_DOT_EMPTY) {
CompletionTargetTokens::ObjectDot {
object,
attribute: None,
}
} else if let Some([object, _dot, attribute]) =
token_suffix_by_kinds(before, OBJECT_DOT_NON_EMPTY)
{
CompletionTargetTokens::ObjectDot {
object,
attribute: Some(attribute),
}
} else {
let Some(last) = before.last() else {
return CompletionTargetTokens::Unknown { offset };
};
CompletionTargetTokens::Generic { token: last }
}
TokenAt::Single(tok) => tok.end(),
TokenAt::Between(_, tok) => tok.start(),
};
let before = parsed.tokens().before(offset);
let last = before.last()?;
let covering_node = covering_node(parsed.syntax().into(), last.range());
Some(covering_node)
}
/// Returns a corresponding AST node for these tokens.
///
/// If no plausible AST node could be found, then `None` is returned.
fn ast(&self, parsed: &'t ParsedModule) -> Option<CompletionTargetAst<'t>> {
match *self {
CompletionTargetTokens::ObjectDot { object, .. } => {
let covering_node = covering_node(parsed.syntax().into(), object.range())
.find(|node| node.is_expr_attribute())
.ok()?;
match covering_node.node() {
ast::AnyNodeRef::ExprAttribute(expr) => {
Some(CompletionTargetAst::ObjectDot { expr })
}
_ => None,
}
}
CompletionTargetTokens::Generic { token } => {
let covering_node = covering_node(parsed.syntax().into(), token.range());
Some(CompletionTargetAst::Scoped {
node: covering_node.node(),
})
}
CompletionTargetTokens::Unknown { offset } => {
let range = TextRange::empty(offset);
let covering_node = covering_node(parsed.syntax().into(), range);
Some(CompletionTargetAst::Scoped {
node: covering_node.node(),
})
}
}
}
}
/// The AST node patterns that we support identifying under the cursor.
#[derive(Debug)]
enum CompletionTargetAst<'t> {
/// A `object.attribute` scenario, where we want to
/// list attributes on `object` for completions.
ObjectDot { expr: &'t ast::ExprAttribute },
/// A scoped scenario, where we want to list all items available in
/// the most narrow scope containing the giving AST node.
Scoped { node: ast::AnyNodeRef<'t> },
}
/// Returns a suffix of `tokens` corresponding to the `kinds` given.
///
/// If a suffix of `tokens` with the given `kinds` could not be found,
/// then `None` is returned.
///
/// This is useful for matching specific patterns of token sequences
/// in order to identify what kind of completions we should offer.
fn token_suffix_by_kinds<const N: usize>(
tokens: &[Token],
kinds: [TokenKind; N],
) -> Option<[&Token; N]> {
if kinds.len() > tokens.len() {
return None;
}
for (token, expected_kind) in tokens.iter().rev().zip(kinds.iter().rev()) {
if &token.kind() != expected_kind {
return None;
}
}
Some(std::array::from_fn(|i| {
&tokens[tokens.len() - (kinds.len() - i)]
}))
}
/// Order completions lexicographically, with these exceptions:
///
/// 1) A `_[^_]` prefix sorts last and
/// 2) A `__` prefix sorts last except before (1)
///
/// This has the effect of putting all dunder attributes after "normal"
/// attributes, and all single-underscore attributes after dunder attributes.
fn compare_suggestions(name1: &str, name2: &str) -> Ordering {
/// A helper type for sorting completions based only on name.
///
/// This sorts "normal" names first, then dunder names and finally
/// single-underscore names. This matches the order of the variants defined for
/// this enum, which is in turn picked up by the derived trait implementation
/// for `Ord`.
#[derive(Clone, Copy, Eq, PartialEq, PartialOrd, Ord)]
enum Kind {
Normal,
Dunder,
Sunder,
}
impl Kind {
fn classify(name: &str) -> Kind {
// Dunder needs a prefix and suffix double underscore.
// When there's only a prefix double underscore, this
// results in explicit name mangling. We let that be
// classified as-if they were single underscore names.
//
// Ref: <https://docs.python.org/3/reference/lexical_analysis.html#reserved-classes-of-identifiers>
if name.starts_with("__") && name.ends_with("__") {
Kind::Dunder
} else if name.starts_with('_') {
Kind::Sunder
} else {
Kind::Normal
}
}
}
let (kind1, kind2) = (Kind::classify(name1), Kind::classify(name2));
kind1.cmp(&kind2).then_with(|| name1.cmp(name2))
}
#[cfg(test)]
mod tests {
use insta::assert_snapshot;
use ruff_python_parser::{Mode, ParseOptions, TokenKind, Tokens};
use crate::completion;
use crate::tests::{CursorTest, cursor_test};
use super::token_suffix_by_kinds;
#[test]
fn token_suffixes_match() {
insta::assert_debug_snapshot!(
token_suffix_by_kinds(&tokenize("foo.x"), [TokenKind::Newline]),
@r"
Some(
[
Newline 5..5,
],
)
",
);
insta::assert_debug_snapshot!(
token_suffix_by_kinds(&tokenize("foo.x"), [TokenKind::Name, TokenKind::Newline]),
@r"
Some(
[
Name 4..5,
Newline 5..5,
],
)
",
);
let all = [
TokenKind::Name,
TokenKind::Dot,
TokenKind::Name,
TokenKind::Newline,
];
insta::assert_debug_snapshot!(
token_suffix_by_kinds(&tokenize("foo.x"), all),
@r"
Some(
[
Name 0..3,
Dot 3..4,
Name 4..5,
Newline 5..5,
],
)
",
);
}
#[test]
fn token_suffixes_nomatch() {
insta::assert_debug_snapshot!(
token_suffix_by_kinds(&tokenize("foo.x"), [TokenKind::Name]),
@"None",
);
let too_many = [
TokenKind::Dot,
TokenKind::Name,
TokenKind::Dot,
TokenKind::Name,
TokenKind::Newline,
];
insta::assert_debug_snapshot!(
token_suffix_by_kinds(&tokenize("foo.x"), too_many),
@"None",
);
}
// At time of writing (2025-05-22), the tests below show some of the
// naivete of our completions. That is, we don't even take what has been
// typed into account. We just kind return all possible completions
@@ -113,8 +342,7 @@ import re
re.<CURSOR>
",
);
assert_snapshot!(test.completions(), @"re");
test.assert_completions_include("findall");
}
#[test]
@@ -127,10 +355,7 @@ f<CURSOR>
",
);
assert_snapshot!(test.completions(), @r"
f
foo
");
assert_snapshot!(test.completions(), @"foo");
}
#[test]
@@ -143,10 +368,7 @@ g<CURSOR>
",
);
assert_snapshot!(test.completions(), @r"
foo
g
");
assert_snapshot!(test.completions(), @"foo");
}
#[test]
@@ -175,10 +397,7 @@ f<CURSOR>
",
);
assert_snapshot!(test.completions(), @r"
f
foo
");
assert_snapshot!(test.completions(), @"foo");
}
#[test]
@@ -208,7 +427,6 @@ def foo():
);
assert_snapshot!(test.completions(), @r"
f
foo
foofoo
");
@@ -259,7 +477,6 @@ def foo():
);
assert_snapshot!(test.completions(), @r"
f
foo
foofoo
");
@@ -276,7 +493,6 @@ def foo():
);
assert_snapshot!(test.completions(), @r"
f
foo
foofoo
");
@@ -295,7 +511,6 @@ def frob(): ...
);
assert_snapshot!(test.completions(), @r"
f
foo
foofoo
frob
@@ -315,7 +530,6 @@ def frob(): ...
);
assert_snapshot!(test.completions(), @r"
f
foo
frob
");
@@ -334,7 +548,6 @@ def frob(): ...
);
assert_snapshot!(test.completions(), @r"
f
foo
foofoo
foofoofoo
@@ -451,15 +664,10 @@ def frob(): ...
",
);
// It's not totally clear why `for` shows up in the
// symbol tables of the detected scopes here. My guess
// is that there's perhaps some sub-optimal behavior
// here because the list comprehension as written is not
// valid.
assert_snapshot!(test.completions(), @r"
bar
for
");
// TODO: it would be good if `bar` was included here, but
// the list comprehension is not yet valid and so we do not
// detect this as a definition of `bar`.
assert_snapshot!(test.completions(), @"<No completions found>");
}
#[test]
@@ -470,10 +678,7 @@ def frob(): ...
",
);
assert_snapshot!(test.completions(), @r"
f
foo
");
assert_snapshot!(test.completions(), @"foo");
}
#[test]
@@ -484,10 +689,7 @@ def frob(): ...
",
);
assert_snapshot!(test.completions(), @r"
f
foo
");
assert_snapshot!(test.completions(), @"foo");
}
#[test]
@@ -498,10 +700,7 @@ def frob(): ...
",
);
assert_snapshot!(test.completions(), @r"
f
foo
");
assert_snapshot!(test.completions(), @"foo");
}
#[test]
@@ -512,10 +711,7 @@ def frob(): ...
",
);
assert_snapshot!(test.completions(), @r"
f
foo
");
assert_snapshot!(test.completions(), @"foo");
}
#[test]
@@ -526,10 +722,7 @@ def frob(): ...
",
);
assert_snapshot!(test.completions(), @r"
f
foo
");
assert_snapshot!(test.completions(), @"foo");
}
#[test]
@@ -602,7 +795,6 @@ class Foo:
assert_snapshot!(test.completions(), @r"
Foo
b
bar
frob
quux
@@ -621,7 +813,6 @@ class Foo:
assert_snapshot!(test.completions(), @r"
Foo
b
bar
quux
");
@@ -734,6 +925,119 @@ class Foo(<CURSOR>",
");
}
#[test]
fn class_init1() {
let test = cursor_test(
"\
class Quux:
def __init__(self):
self.foo = 1
self.bar = 2
self.baz = 3
quux = Quux()
quux.<CURSOR>
",
);
assert_snapshot!(test.completions(), @r"
bar
baz
foo
__annotations__
__class__
__delattr__
__dict__
__dir__
__doc__
__eq__
__format__
__getattribute__
__getstate__
__hash__
__init__
__init_subclass__
__module__
__ne__
__new__
__reduce__
__reduce_ex__
__repr__
__setattr__
__sizeof__
__str__
__subclasshook__
");
}
#[test]
fn class_init2() {
let test = cursor_test(
"\
class Quux:
def __init__(self):
self.foo = 1
self.bar = 2
self.baz = 3
quux = Quux()
quux.b<CURSOR>
",
);
assert_snapshot!(test.completions(), @r"
bar
baz
foo
__annotations__
__class__
__delattr__
__dict__
__dir__
__doc__
__eq__
__format__
__getattribute__
__getstate__
__hash__
__init__
__init_subclass__
__module__
__ne__
__new__
__reduce__
__reduce_ex__
__repr__
__setattr__
__sizeof__
__str__
__subclasshook__
");
}
#[test]
fn class_init3() {
let test = cursor_test(
"\
class Quux:
def __init__(self):
self.foo = 1
self.bar = 2
self.<CURSOR>
self.baz = 3
",
);
// FIXME: This should list completions on `self`, which should
// include, at least, `foo` and `bar`. At time of writing
// (2025-06-04), the type of `self` is inferred as `Unknown` in
// this context. This in turn prevents us from getting a list
// of available attributes.
//
// See: https://github.com/astral-sh/ty/issues/159
assert_snapshot!(test.completions(), @"<No completions found>");
}
// We don't yet take function parameters into account.
#[test]
fn call_prefix1() {
@@ -750,7 +1054,6 @@ bar(o<CURSOR>
assert_snapshot!(test.completions(), @r"
bar
foo
o
");
}
@@ -788,7 +1091,6 @@ class C:
assert_snapshot!(test.completions(), @r"
C
bar
f
foo
self
");
@@ -825,7 +1127,6 @@ class C:
assert_snapshot!(test.completions(), @r"
C
bar
f
foo
self
");
@@ -854,11 +1155,108 @@ print(f\"{some<CURSOR>
",
);
assert_snapshot!(test.completions(), @r"
print
some
some_symbol
");
assert_snapshot!(test.completions(), @"some_symbol");
}
#[test]
fn statically_invisible_symbols() {
let test = cursor_test(
"\
if 1 + 2 != 3:
hidden_symbol = 1
hidden_<CURSOR>
",
);
assert_snapshot!(test.completions(), @"<No completions found>");
}
#[test]
fn completions_inside_unreachable_sections() {
let test = cursor_test(
"\
import sys
if sys.platform == \"not-my-current-platform\":
only_available_in_this_branch = 1
on<CURSOR>
",
);
// TODO: ideally, `only_available_in_this_branch` should be available here, but we
// currently make no effort to provide a good IDE experience within sections that
// are unreachable
assert_snapshot!(test.completions(), @"sys");
}
#[test]
fn star_import() {
let test = cursor_test(
"\
from typing import *
Re<CURSOR>
",
);
test.assert_completions_include("Reversible");
// `ReadableBuffer` is a symbol in `typing`, but it is not re-exported
test.assert_completions_do_not_include("ReadableBuffer");
}
#[test]
fn nested_attribute_access() {
let test = cursor_test(
"\
class A:
x: str
class B:
a: A
b = B()
b.a.<CURSOR>
",
);
// FIXME: These should be flipped.
test.assert_completions_include("a");
test.assert_completions_do_not_include("x");
}
#[test]
fn ordering() {
let test = cursor_test(
"\
class A:
foo: str
_foo: str
__foo__: str
__foo: str
FOO: str
_FOO: str
__FOO__: str
__FOO: str
A.<CURSOR>
",
);
assert_snapshot!(
test.completions_if(|name| name.contains("FOO") || name.contains("foo")),
@r"
FOO
foo
__FOO__
__foo__
_FOO
__FOO
__foo
_foo
",
);
}
// Ref: https://github.com/astral-sh/ty/issues/572
@@ -921,10 +1319,7 @@ Fo<CURSOR> = float
",
);
assert_snapshot!(test.completions(), @r"
Fo
float
");
assert_snapshot!(test.completions(), @"Fo");
}
// Ref: https://github.com/astral-sh/ty/issues/572
@@ -999,9 +1394,7 @@ except Type<CURSOR>:
",
);
assert_snapshot!(test.completions(), @r"
Type
");
assert_snapshot!(test.completions(), @"<No completions found>");
}
// Ref: https://github.com/astral-sh/ty/issues/572
@@ -1019,6 +1412,10 @@ def _():
impl CursorTest {
fn completions(&self) -> String {
self.completions_if(|_| true)
}
fn completions_if(&self, predicate: impl Fn(&str) -> bool) -> String {
let completions = completion(&self.db, self.file, self.cursor_offset);
if completions.is_empty() {
return "<No completions found>".to_string();
@@ -1026,8 +1423,39 @@ def _():
completions
.into_iter()
.map(|completion| completion.label)
.filter(|label| predicate(label))
.collect::<Vec<String>>()
.join("\n")
}
#[track_caller]
fn assert_completions_include(&self, expected: &str) {
let completions = completion(&self.db, self.file, self.cursor_offset);
assert!(
completions
.iter()
.any(|completion| completion.label == expected),
"Expected completions to include `{expected}`"
);
}
#[track_caller]
fn assert_completions_do_not_include(&self, unexpected: &str) {
let completions = completion(&self.db, self.file, self.cursor_offset);
assert!(
completions
.iter()
.all(|completion| completion.label != unexpected),
"Expected completions to not include `{unexpected}`",
);
}
}
fn tokenize(src: &str) -> Tokens {
let parsed = ruff_python_parser::parse(src, ParseOptions::from(Mode::Module))
.expect("valid Python source for token stream");
parsed.tokens().clone()
}
}

View File

@@ -0,0 +1,11 @@
# This is a regression test for `infer_expression_types`.
# ref: https://github.com/astral-sh/ruff/pull/18041#discussion_r2094573989
class C:
def f(self, other: "C"):
if self.a > other.b or self.b:
return False
if self:
return True
C().a

View File

@@ -12,7 +12,7 @@ use thiserror::Error;
use ty_python_semantic::lint::{GetLintError, Level, LintSource, RuleSelection};
use ty_python_semantic::{
ProgramSettings, PythonPath, PythonPlatform, PythonVersionFileSource, PythonVersionSource,
PythonVersionWithSource, SearchPathSettings,
PythonVersionWithSource, SearchPathSettings, SysPrefixPathOrigin,
};
use super::settings::{Settings, TerminalSettings};
@@ -182,19 +182,27 @@ impl Options {
custom_typeshed: typeshed.map(|path| path.absolute(project_root, system)),
python_path: python
.map(|python_path| {
PythonPath::from_cli_flag(python_path.absolute(project_root, system))
PythonPath::sys_prefix(
python_path.absolute(project_root, system),
SysPrefixPathOrigin::PythonCliFlag,
)
})
.or_else(|| {
std::env::var("VIRTUAL_ENV")
.ok()
.map(PythonPath::from_virtual_env_var)
std::env::var("VIRTUAL_ENV").ok().map(|virtual_env| {
PythonPath::sys_prefix(virtual_env, SysPrefixPathOrigin::VirtualEnvVar)
})
})
.or_else(|| {
std::env::var("CONDA_PREFIX")
.ok()
.map(PythonPath::from_conda_prefix_var)
std::env::var("CONDA_PREFIX").ok().map(|path| {
PythonPath::sys_prefix(path, SysPrefixPathOrigin::CondaPrefixVar)
})
})
.unwrap_or_else(|| PythonPath::Discover(project_root.to_path_buf())),
.unwrap_or_else(|| {
PythonPath::sys_prefix(
project_root.to_path_buf(),
SysPrefixPathOrigin::LocalVenv,
)
}),
}
}

View File

@@ -37,7 +37,9 @@ reveal_type(c_instance.inferred_from_other_attribute) # revealed: Unknown
# See https://github.com/astral-sh/ruff/issues/15960 for a related discussion.
reveal_type(c_instance.inferred_from_param) # revealed: Unknown | int | None
reveal_type(c_instance.declared_only) # revealed: bytes
# TODO: Should be `bytes` with no error, like mypy and pyright?
# error: [unresolved-attribute]
reveal_type(c_instance.declared_only) # revealed: Unknown
reveal_type(c_instance.declared_and_bound) # revealed: bool
@@ -64,12 +66,10 @@ C.inferred_from_value = "overwritten on class"
# This assignment is fine:
c_instance.declared_and_bound = False
# TODO: After this assignment to the attribute within this scope, we may eventually want to narrow
# the `bool` type (see above) for this instance variable to `Literal[False]` here. This is unsound
# in general (we don't know what else happened to `c_instance` between the assignment and the use
# here), but mypy and pyright support this. In conclusion, this could be `bool` but should probably
# be `Literal[False]`.
reveal_type(c_instance.declared_and_bound) # revealed: bool
# Strictly speaking, inferring this as `Literal[False]` rather than `bool` is unsound in general
# (we don't know what else happened to `c_instance` between the assignment and the use here),
# but mypy and pyright support this.
reveal_type(c_instance.declared_and_bound) # revealed: Literal[False]
```
#### Variable declared in class body and possibly bound in `__init__`
@@ -149,14 +149,16 @@ class C:
c_instance = C(True)
reveal_type(c_instance.only_declared_in_body) # revealed: str | None
reveal_type(c_instance.only_declared_in_init) # revealed: str | None
# TODO: should be `str | None` without error
# error: [unresolved-attribute]
reveal_type(c_instance.only_declared_in_init) # revealed: Unknown
reveal_type(c_instance.declared_in_body_and_init) # revealed: str | None
reveal_type(c_instance.declared_in_body_defined_in_init) # revealed: str | None
# TODO: This should be `str | None`. Fixing this requires an overhaul of the `Symbol` API,
# which is planned in https://github.com/astral-sh/ruff/issues/14297
reveal_type(c_instance.bound_in_body_declared_in_init) # revealed: Unknown | str | None
reveal_type(c_instance.bound_in_body_declared_in_init) # revealed: Unknown | Literal["a"]
reveal_type(c_instance.bound_in_body_and_init) # revealed: Unknown | None | Literal["a"]
```
@@ -187,7 +189,9 @@ reveal_type(c_instance.inferred_from_other_attribute) # revealed: Unknown
reveal_type(c_instance.inferred_from_param) # revealed: Unknown | int | None
reveal_type(c_instance.declared_only) # revealed: bytes
# TODO: should be `bytes` with no error, like mypy and pyright?
# error: [unresolved-attribute]
reveal_type(c_instance.declared_only) # revealed: Unknown
reveal_type(c_instance.declared_and_bound) # revealed: bool
@@ -260,8 +264,8 @@ class C:
self.w += None
# TODO: Mypy and pyright do not support this, but it would be great if we could
# infer `Unknown | str` or at least `Unknown | Weird | str` here.
reveal_type(C().w) # revealed: Unknown | Weird
# infer `Unknown | str` here (`Weird` is not a possible type for the `w` attribute).
reveal_type(C().w) # revealed: Unknown
```
#### Attributes defined in tuple unpackings
@@ -410,14 +414,41 @@ class C:
[... for self.a in IntIterable()]
[... for (self.b, self.c) in TupleIterable()]
[... for self.d in IntIterable() for self.e in IntIterable()]
[[... for self.f in IntIterable()] for _ in IntIterable()]
[[... for self.g in IntIterable()] for self in [D()]]
class D:
g: int
c_instance = C()
reveal_type(c_instance.a) # revealed: Unknown | int
reveal_type(c_instance.b) # revealed: Unknown | int
reveal_type(c_instance.c) # revealed: Unknown | str
reveal_type(c_instance.d) # revealed: Unknown | int
reveal_type(c_instance.e) # revealed: Unknown | int
# TODO: no error, reveal Unknown | int
# error: [unresolved-attribute]
reveal_type(c_instance.a) # revealed: Unknown
# TODO: no error, reveal Unknown | int
# error: [unresolved-attribute]
reveal_type(c_instance.b) # revealed: Unknown
# TODO: no error, reveal Unknown | str
# error: [unresolved-attribute]
reveal_type(c_instance.c) # revealed: Unknown
# TODO: no error, reveal Unknown | int
# error: [unresolved-attribute]
reveal_type(c_instance.d) # revealed: Unknown
# TODO: no error, reveal Unknown | int
# error: [unresolved-attribute]
reveal_type(c_instance.e) # revealed: Unknown
# TODO: no error, reveal Unknown | int
# error: [unresolved-attribute]
reveal_type(c_instance.f) # revealed: Unknown
# This one is correctly not resolved as an attribute:
# error: [unresolved-attribute]
reveal_type(c_instance.g) # revealed: Unknown
```
#### Conditionally declared / bound attributes
@@ -721,10 +752,7 @@ reveal_type(C.pure_class_variable) # revealed: Unknown
# error: [invalid-attribute-access] "Cannot assign to instance attribute `pure_class_variable` from the class object `<class 'C'>`"
C.pure_class_variable = "overwritten on class"
# TODO: should be `Unknown | Literal["value set in class method"]` or
# Literal["overwritten on class"]`, once/if we support local narrowing.
# error: [unresolved-attribute]
reveal_type(C.pure_class_variable) # revealed: Unknown
reveal_type(C.pure_class_variable) # revealed: Literal["overwritten on class"]
c_instance = C()
reveal_type(c_instance.pure_class_variable) # revealed: Unknown | Literal["value set in class method"]
@@ -762,19 +790,12 @@ reveal_type(c_instance.variable_with_class_default2) # revealed: Unknown | Lite
c_instance.variable_with_class_default1 = "value set on instance"
reveal_type(C.variable_with_class_default1) # revealed: str
# TODO: Could be Literal["value set on instance"], or still `str` if we choose not to
# narrow the type.
reveal_type(c_instance.variable_with_class_default1) # revealed: str
reveal_type(c_instance.variable_with_class_default1) # revealed: Literal["value set on instance"]
C.variable_with_class_default1 = "overwritten on class"
# TODO: Could be `Literal["overwritten on class"]`, or still `str` if we choose not to
# narrow the type.
reveal_type(C.variable_with_class_default1) # revealed: str
# TODO: should still be `Literal["value set on instance"]`, or `str`.
reveal_type(c_instance.variable_with_class_default1) # revealed: str
reveal_type(C.variable_with_class_default1) # revealed: Literal["overwritten on class"]
reveal_type(c_instance.variable_with_class_default1) # revealed: Literal["value set on instance"]
```
#### Descriptor attributes as class variables
@@ -928,6 +949,42 @@ def _(flag1: bool, flag2: bool):
reveal_type(C5.attr1) # revealed: Unknown | Literal["metaclass value", "class value"]
```
## Invalid access to attribute
<!-- snapshot-diagnostics -->
If a non-declared variable is used and an attribute with the same name is defined and accessible,
then we emit a subdiagnostic suggesting the use of `self.`.
(`An attribute with the same name as 'x' is defined, consider using 'self.x'` in these cases)
```py
class Foo:
x: int
def method(self):
# error: [unresolved-reference] "Name `x` used when not defined"
y = x
```
```py
class Foo:
x: int = 1
def method(self):
# error: [unresolved-reference] "Name `x` used when not defined"
y = x
```
```py
class Foo:
def __init__(self):
self.x = 1
def method(self):
# error: [unresolved-reference] "Name `x` used when not defined"
y = x
```
## Unions of attributes
If the (meta)class is a union type or if the attribute on the (meta) class has a union type, we

View File

@@ -699,9 +699,7 @@ class C:
descriptor = Descriptor()
C.descriptor = "something else"
# This could also be `Literal["something else"]` if we support narrowing of attribute types based on assignments
reveal_type(C.descriptor) # revealed: Unknown | int
reveal_type(C.descriptor) # revealed: Literal["something else"]
```
### Possibly unbound descriptor attributes

View File

@@ -0,0 +1,318 @@
# Narrowing by assignment
## Attribute
### Basic
```py
class A:
x: int | None = None
y = None
def __init__(self):
self.z = None
a = A()
a.x = 0
a.y = 0
a.z = 0
reveal_type(a.x) # revealed: Literal[0]
reveal_type(a.y) # revealed: Literal[0]
reveal_type(a.z) # revealed: Literal[0]
# Make sure that we infer the narrowed type for eager
# scopes (class, comprehension) and the non-narrowed
# public type for lazy scopes (function)
class _:
reveal_type(a.x) # revealed: Literal[0]
reveal_type(a.y) # revealed: Literal[0]
reveal_type(a.z) # revealed: Literal[0]
[reveal_type(a.x) for _ in range(1)] # revealed: Literal[0]
[reveal_type(a.y) for _ in range(1)] # revealed: Literal[0]
[reveal_type(a.z) for _ in range(1)] # revealed: Literal[0]
def _():
reveal_type(a.x) # revealed: Unknown | int | None
reveal_type(a.y) # revealed: Unknown | None
reveal_type(a.z) # revealed: Unknown | None
if False:
a = A()
reveal_type(a.x) # revealed: Literal[0]
reveal_type(a.y) # revealed: Literal[0]
reveal_type(a.z) # revealed: Literal[0]
if True:
a = A()
reveal_type(a.x) # revealed: int | None
reveal_type(a.y) # revealed: Unknown | None
reveal_type(a.z) # revealed: Unknown | None
a.x = 0
a.y = 0
a.z = 0
reveal_type(a.x) # revealed: Literal[0]
reveal_type(a.y) # revealed: Literal[0]
reveal_type(a.z) # revealed: Literal[0]
class _:
a = A()
reveal_type(a.x) # revealed: int | None
reveal_type(a.y) # revealed: Unknown | None
reveal_type(a.z) # revealed: Unknown | None
def cond() -> bool:
return True
class _:
if False:
a = A()
reveal_type(a.x) # revealed: Literal[0]
reveal_type(a.y) # revealed: Literal[0]
reveal_type(a.z) # revealed: Literal[0]
if cond():
a = A()
reveal_type(a.x) # revealed: int | None
reveal_type(a.y) # revealed: Unknown | None
reveal_type(a.z) # revealed: Unknown | None
class _:
a = A()
class Inner:
reveal_type(a.x) # revealed: int | None
reveal_type(a.y) # revealed: Unknown | None
reveal_type(a.z) # revealed: Unknown | None
# error: [unresolved-reference]
does.nt.exist = 0
# error: [unresolved-reference]
reveal_type(does.nt.exist) # revealed: Unknown
```
### Narrowing chain
```py
class D: ...
class C:
d: D | None = None
class B:
c1: C | None = None
c2: C | None = None
class A:
b: B | None = None
a = A()
a.b = B()
a.b.c1 = C()
a.b.c2 = C()
a.b.c1.d = D()
a.b.c2.d = D()
reveal_type(a.b) # revealed: B
reveal_type(a.b.c1) # revealed: C
reveal_type(a.b.c1.d) # revealed: D
a.b.c1 = C()
reveal_type(a.b) # revealed: B
reveal_type(a.b.c1) # revealed: C
reveal_type(a.b.c1.d) # revealed: D | None
reveal_type(a.b.c2.d) # revealed: D
a.b.c1.d = D()
a.b = B()
reveal_type(a.b) # revealed: B
reveal_type(a.b.c1) # revealed: C | None
reveal_type(a.b.c2) # revealed: C | None
# error: [possibly-unbound-attribute]
reveal_type(a.b.c1.d) # revealed: D | None
# error: [possibly-unbound-attribute]
reveal_type(a.b.c2.d) # revealed: D | None
```
### Do not narrow the type of a `property` by assignment
```py
class C:
def __init__(self):
self._x: int = 0
@property
def x(self) -> int:
return self._x
@x.setter
def x(self, value: int) -> None:
self._x = abs(value)
c = C()
c.x = -1
# Don't infer `c.x` to be `Literal[-1]`
reveal_type(c.x) # revealed: int
```
### Do not narrow the type of a descriptor by assignment
```py
class Descriptor:
def __get__(self, instance: object, owner: type) -> int:
return 1
def __set__(self, instance: object, value: int) -> None:
pass
class C:
desc: Descriptor = Descriptor()
c = C()
c.desc = -1
# Don't infer `c.desc` to be `Literal[-1]`
reveal_type(c.desc) # revealed: int
```
## Subscript
### Specialization for builtin types
Type narrowing based on assignment to a subscript expression is generally unsound, because arbitrary
`__getitem__`/`__setitem__` methods on a class do not necessarily guarantee that the passed-in value
for `__setitem__` is stored and can be retrieved unmodified via `__getitem__`. Therefore, we
currently only perform assignment-based narrowing on a few built-in classes (`list`, `dict`,
`bytesarray`, `TypedDict` and `collections` types) where we are confident that this kind of
narrowing can be performed soundly. This is the same approach as pyright.
```py
from typing import TypedDict
from collections import ChainMap, defaultdict
l: list[int | None] = [None]
l[0] = 0
d: dict[int, int] = {1: 1}
d[0] = 0
b: bytearray = bytearray(b"abc")
b[0] = 0
dd: defaultdict[int, int] = defaultdict(int)
dd[0] = 0
cm: ChainMap[int, int] = ChainMap({1: 1}, {0: 0})
cm[0] = 0
# TODO: should be ChainMap[int, int]
reveal_type(cm) # revealed: ChainMap[Unknown, Unknown]
reveal_type(l[0]) # revealed: Literal[0]
reveal_type(d[0]) # revealed: Literal[0]
reveal_type(b[0]) # revealed: Literal[0]
reveal_type(dd[0]) # revealed: Literal[0]
# TODO: should be Literal[0]
reveal_type(cm[0]) # revealed: Unknown
class C:
reveal_type(l[0]) # revealed: Literal[0]
reveal_type(d[0]) # revealed: Literal[0]
reveal_type(b[0]) # revealed: Literal[0]
reveal_type(dd[0]) # revealed: Literal[0]
# TODO: should be Literal[0]
reveal_type(cm[0]) # revealed: Unknown
[reveal_type(l[0]) for _ in range(1)] # revealed: Literal[0]
[reveal_type(d[0]) for _ in range(1)] # revealed: Literal[0]
[reveal_type(b[0]) for _ in range(1)] # revealed: Literal[0]
[reveal_type(dd[0]) for _ in range(1)] # revealed: Literal[0]
# TODO: should be Literal[0]
[reveal_type(cm[0]) for _ in range(1)] # revealed: Unknown
def _():
reveal_type(l[0]) # revealed: int | None
reveal_type(d[0]) # revealed: int
reveal_type(b[0]) # revealed: int
reveal_type(dd[0]) # revealed: int
reveal_type(cm[0]) # revealed: int
class D(TypedDict):
x: int
label: str
td = D(x=1, label="a")
td["x"] = 0
# TODO: should be Literal[0]
reveal_type(td["x"]) # revealed: @Todo(TypedDict)
# error: [unresolved-reference]
does["not"]["exist"] = 0
# error: [unresolved-reference]
reveal_type(does["not"]["exist"]) # revealed: Unknown
non_subscriptable = 1
# error: [non-subscriptable]
non_subscriptable[0] = 0
# error: [non-subscriptable]
reveal_type(non_subscriptable[0]) # revealed: Unknown
```
### No narrowing for custom classes with arbitrary `__getitem__` / `__setitem__`
```py
class C:
def __init__(self):
self.l: list[str] = []
def __getitem__(self, index: int) -> str:
return self.l[index]
def __setitem__(self, index: int, value: str | int) -> None:
if len(self.l) == index:
self.l.append(str(value))
else:
self.l[index] = str(value)
c = C()
c[0] = 0
reveal_type(c[0]) # revealed: str
```
## Complex target
```py
class A:
x: list[int | None] = []
class B:
a: A | None = None
b = B()
b.a = A()
b.a.x[0] = 0
reveal_type(b.a.x[0]) # revealed: Literal[0]
class C:
reveal_type(b.a.x[0]) # revealed: Literal[0]
def _():
# error: [possibly-unbound-attribute]
reveal_type(b.a.x[0]) # revealed: Unknown | int | None
# error: [possibly-unbound-attribute]
reveal_type(b.a.x) # revealed: Unknown | list[int | None]
reveal_type(b.a) # revealed: Unknown | A | None
```
## Invalid assignments are not used for narrowing
```py
class C:
x: int | None
l: list[int]
def f(c: C, s: str):
c.x = s # error: [invalid-assignment]
reveal_type(c.x) # revealed: int | None
s = c.x # error: [invalid-assignment]
# TODO: This assignment is invalid and should result in an error.
c.l[0] = s
reveal_type(c.l[0]) # revealed: int
```

View File

@@ -53,11 +53,114 @@ constraints may no longer be valid due to a "time lag". However, it may be possi
that some of them are valid by performing a more detailed analysis (e.g. checking that the narrowing
target has not changed in all places where the function is called).
### Narrowing by attribute/subscript assignments
```py
class A:
x: str | None = None
def update_x(self, value: str | None):
self.x = value
a = A()
a.x = "a"
class B:
reveal_type(a.x) # revealed: Literal["a"]
def f():
reveal_type(a.x) # revealed: Unknown | str | None
[reveal_type(a.x) for _ in range(1)] # revealed: Literal["a"]
a = A()
class C:
reveal_type(a.x) # revealed: str | None
def g():
reveal_type(a.x) # revealed: Unknown | str | None
[reveal_type(a.x) for _ in range(1)] # revealed: str | None
a = A()
a.x = "a"
a.update_x("b")
class D:
# TODO: should be `str | None`
reveal_type(a.x) # revealed: Literal["a"]
def h():
reveal_type(a.x) # revealed: Unknown | str | None
# TODO: should be `str | None`
[reveal_type(a.x) for _ in range(1)] # revealed: Literal["a"]
```
### Narrowing by attribute/subscript assignments in nested scopes
```py
class D: ...
class C:
d: D | None = None
class B:
c1: C | None = None
c2: C | None = None
class A:
b: B | None = None
a = A()
a.b = B()
class _:
a.b.c1 = C()
class _:
a.b.c1.d = D()
a = 1
class _3:
reveal_type(a) # revealed: A
reveal_type(a.b.c1.d) # revealed: D
class _:
a = 1
# error: [unresolved-attribute]
a.b.c1.d = D()
class _3:
reveal_type(a) # revealed: A
# TODO: should be `D | None`
reveal_type(a.b.c1.d) # revealed: D
a.b.c1 = C()
a.b.c1.d = D()
class _:
a.b = B()
class _:
# error: [possibly-unbound-attribute]
reveal_type(a.b.c1.d) # revealed: D | None
reveal_type(a.b.c1) # revealed: C | None
```
### Narrowing constraints introduced in eager nested scopes
```py
g: str | None = "a"
class A:
x: str | None = None
a = A()
l: list[str | None] = [None]
def f(x: str | None):
def _():
if x is not None:
@@ -69,6 +172,14 @@ def f(x: str | None):
if g is not None:
reveal_type(g) # revealed: str
if a.x is not None:
# TODO(#17643): should be `Unknown | str`
reveal_type(a.x) # revealed: Unknown | str | None
if l[0] is not None:
# TODO(#17643): should be `str`
reveal_type(l[0]) # revealed: str | None
class C:
if x is not None:
reveal_type(x) # revealed: str
@@ -79,6 +190,14 @@ def f(x: str | None):
if g is not None:
reveal_type(g) # revealed: str
if a.x is not None:
# TODO(#17643): should be `Unknown | str`
reveal_type(a.x) # revealed: Unknown | str | None
if l[0] is not None:
# TODO(#17643): should be `str`
reveal_type(l[0]) # revealed: str | None
# TODO: should be str
# This could be fixed if we supported narrowing with if clauses in comprehensions.
[reveal_type(x) for _ in range(1) if x is not None] # revealed: str | None
@@ -89,6 +208,13 @@ def f(x: str | None):
```py
g: str | None = "a"
class A:
x: str | None = None
a = A()
l: list[str | None] = [None]
def f(x: str | None):
if x is not None:
def _():
@@ -109,6 +235,28 @@ def f(x: str | None):
reveal_type(g) # revealed: str
[reveal_type(g) for _ in range(1)] # revealed: str
if a.x is not None:
def _():
reveal_type(a.x) # revealed: Unknown | str | None
class D:
# TODO(#17643): should be `Unknown | str`
reveal_type(a.x) # revealed: Unknown | str | None
# TODO(#17643): should be `Unknown | str`
[reveal_type(a.x) for _ in range(1)] # revealed: Unknown | str | None
if l[0] is not None:
def _():
reveal_type(l[0]) # revealed: str | None
class D:
# TODO(#17643): should be `str`
reveal_type(l[0]) # revealed: str | None
# TODO(#17643): should be `str`
[reveal_type(l[0]) for _ in range(1)] # revealed: str | None
```
### Narrowing constraints introduced in multiple scopes
@@ -118,6 +266,13 @@ from typing import Literal
g: str | Literal[1] | None = "a"
class A:
x: str | Literal[1] | None = None
a = A()
l: list[str | Literal[1] | None] = [None]
def f(x: str | Literal[1] | None):
class C:
if x is not None:
@@ -140,6 +295,28 @@ def f(x: str | Literal[1] | None):
class D:
if g != 1:
reveal_type(g) # revealed: str
if a.x is not None:
def _():
if a.x != 1:
# TODO(#17643): should be `Unknown | str | None`
reveal_type(a.x) # revealed: Unknown | str | Literal[1] | None
class D:
if a.x != 1:
# TODO(#17643): should be `Unknown | str`
reveal_type(a.x) # revealed: Unknown | str | Literal[1] | None
if l[0] is not None:
def _():
if l[0] != 1:
# TODO(#17643): should be `str | None`
reveal_type(l[0]) # revealed: str | Literal[1] | None
class D:
if l[0] != 1:
# TODO(#17643): should be `str`
reveal_type(l[0]) # revealed: str | Literal[1] | None
```
### Narrowing constraints with bindings in class scope, and nested scopes

View File

@@ -26,7 +26,7 @@ def f(x: Foo):
else:
reveal_type(x) # revealed: Foo
def y(x: Bar):
def g(x: Bar):
if hasattr(x, "spam"):
reveal_type(x) # revealed: Never
reveal_type(x.spam) # revealed: Never
@@ -35,4 +35,25 @@ def y(x: Bar):
# error: [unresolved-attribute]
reveal_type(x.spam) # revealed: Unknown
def returns_bool() -> bool:
return False
class Baz:
if returns_bool():
x: int = 42
def h(obj: Baz):
reveal_type(obj) # revealed: Baz
# error: [possibly-unbound-attribute]
reveal_type(obj.x) # revealed: int
if hasattr(obj, "x"):
reveal_type(obj) # revealed: Baz & <Protocol with members 'x'>
reveal_type(obj.x) # revealed: int
else:
reveal_type(obj) # revealed: Baz & ~<Protocol with members 'x'>
# TODO: should emit `[unresolved-attribute]` and reveal `Unknown`
reveal_type(obj.x) # revealed: @Todo(map_with_boundness: intersections with negative contributions)
```

View File

@@ -389,6 +389,7 @@ not be considered protocol members by type checkers either:
class Lumberjack(Protocol):
__slots__ = ()
__match_args__ = ()
_abc_foo: str # any attribute starting with `_abc_` is excluded as a protocol attribute
x: int
def __new__(cls, x: int) -> "Lumberjack":

View File

@@ -0,0 +1,82 @@
---
source: crates/ty_test/src/lib.rs
expression: snapshot
---
---
mdtest name: attributes.md - Attributes - Invalid access to attribute
mdtest path: crates/ty_python_semantic/resources/mdtest/attributes.md
---
# Python source files
## mdtest_snippet.py
```
1 | class Foo:
2 | x: int
3 |
4 | def method(self):
5 | # error: [unresolved-reference] "Name `x` used when not defined"
6 | y = x
7 | class Foo:
8 | x: int = 1
9 |
10 | def method(self):
11 | # error: [unresolved-reference] "Name `x` used when not defined"
12 | y = x
13 | class Foo:
14 | def __init__(self):
15 | self.x = 1
16 |
17 | def method(self):
18 | # error: [unresolved-reference] "Name `x` used when not defined"
19 | y = x
```
# Diagnostics
```
error[unresolved-reference]: Name `x` used when not defined
--> src/mdtest_snippet.py:6:13
|
4 | def method(self):
5 | # error: [unresolved-reference] "Name `x` used when not defined"
6 | y = x
| ^
7 | class Foo:
8 | x: int = 1
|
info: An attribute `x` is available: consider using `self.x`
info: rule `unresolved-reference` is enabled by default
```
```
error[unresolved-reference]: Name `x` used when not defined
--> src/mdtest_snippet.py:12:13
|
10 | def method(self):
11 | # error: [unresolved-reference] "Name `x` used when not defined"
12 | y = x
| ^
13 | class Foo:
14 | def __init__(self):
|
info: An attribute `x` is available: consider using `self.x`
info: rule `unresolved-reference` is enabled by default
```
```
error[unresolved-reference]: Name `x` used when not defined
--> src/mdtest_snippet.py:19:13
|
17 | def method(self):
18 | # error: [unresolved-reference] "Name `x` used when not defined"
19 | y = x
| ^
|
info: An attribute `x` is available: consider using `self.x`
info: rule `unresolved-reference` is enabled by default
```

View File

@@ -7,7 +7,7 @@ use ruff_python_ast::statement_visitor::{StatementVisitor, walk_stmt};
use ruff_python_ast::{self as ast};
use crate::semantic_index::ast_ids::HasScopedExpressionId;
use crate::semantic_index::symbol::ScopeId;
use crate::semantic_index::place::ScopeId;
use crate::semantic_index::{SemanticIndex, global_scope, semantic_index};
use crate::types::{Truthiness, Type, infer_expression_types};
use crate::{Db, ModuleName, resolve_module};

View File

@@ -24,13 +24,13 @@ pub(crate) mod list;
mod module_name;
mod module_resolver;
mod node_key;
pub(crate) mod place;
mod program;
mod python_platform;
pub mod semantic_index;
mod semantic_model;
pub(crate) mod site_packages;
mod suppression;
pub(crate) mod symbol;
pub mod types;
mod unpack;
mod util;

View File

@@ -139,15 +139,6 @@ pub(crate) fn search_paths(db: &dyn Db) -> SearchPathIterator {
Program::get(db).search_paths(db).iter(db)
}
/// Searches for a `.venv` directory in `project_root` that contains a `pyvenv.cfg` file.
fn discover_venv_in(system: &dyn System, project_root: &SystemPath) -> Option<SystemPathBuf> {
let virtual_env_directory = project_root.join(".venv");
system
.is_file(&virtual_env_directory.join("pyvenv.cfg"))
.then_some(virtual_env_directory)
}
#[derive(Debug, PartialEq, Eq)]
pub struct SearchPaths {
/// Search paths that have been statically determined purely from reading Ruff's configuration settings.
@@ -243,68 +234,34 @@ impl SearchPaths {
static_paths.push(stdlib_path);
let (site_packages_paths, python_version) = match python_path {
PythonPath::SysPrefix(sys_prefix, origin) => {
tracing::debug!(
"Discovering site-packages paths from sys-prefix `{sys_prefix}` ({origin}')"
);
// TODO: We may want to warn here if the venv's python version is older
// than the one resolved in the program settings because it indicates
// that the `target-version` is incorrectly configured or that the
// venv is out of date.
PythonEnvironment::new(sys_prefix, *origin, system)?.into_settings(system)?
}
PythonPath::IntoSysPrefix(path, origin) => {
if *origin == SysPrefixPathOrigin::LocalVenv {
tracing::debug!("Discovering virtual environment in `{path}`");
let virtual_env_directory = path.join(".venv");
PythonPath::Resolve(target, origin) => {
tracing::debug!("Resolving {origin}: {target}");
let root = system
// If given a file, assume it's a Python executable, e.g., `.venv/bin/python3`,
// and search for a virtual environment in the root directory. Ideally, we'd
// invoke the target to determine `sys.prefix` here, but that's more complicated
// and may be deferred to uv.
.is_file(target)
.then(|| target.as_path())
.take_if(|target| {
// Avoid using the target if it doesn't look like a Python executable, e.g.,
// to deny cases like `.venv/bin/foo`
target
.file_name()
.is_some_and(|name| name.starts_with("python"))
})
.and_then(SystemPath::parent)
.and_then(SystemPath::parent)
// If not a file, use the path as given and allow let `PythonEnvironment::new`
// handle the error.
.unwrap_or(target);
PythonEnvironment::new(root, *origin, system)?.into_settings(system)?
}
PythonPath::Discover(root) => {
tracing::debug!("Discovering virtual environment in `{root}`");
discover_venv_in(db.system(), root)
.and_then(|virtual_env_path| {
tracing::debug!("Found `.venv` folder at `{}`", virtual_env_path);
PythonEnvironment::new(
virtual_env_path.clone(),
SysPrefixPathOrigin::LocalVenv,
system,
)
.and_then(|env| env.into_settings(system))
.inspect_err(|err| {
PythonEnvironment::new(
&virtual_env_directory,
SysPrefixPathOrigin::LocalVenv,
system,
)
.and_then(|venv| venv.into_settings(system))
.inspect_err(|err| {
if system.is_directory(&virtual_env_directory) {
tracing::debug!(
"Ignoring automatically detected virtual environment at `{}`: {}",
virtual_env_path,
&virtual_env_directory,
err
);
})
.ok()
}
})
.unwrap_or_else(|| {
.unwrap_or_else(|_| {
tracing::debug!("No virtual environment found");
(SitePackagesPaths::default(), None)
})
} else {
tracing::debug!("Resolving {origin}: {path}");
PythonEnvironment::new(path, *origin, system)?.into_settings(system)?
}
}
PythonPath::KnownSitePackages(paths) => (

View File

@@ -262,8 +262,10 @@ impl SearchPathSettings {
#[derive(Debug, Clone, Eq, PartialEq)]
#[cfg_attr(feature = "serde", derive(serde::Serialize, serde::Deserialize))]
pub enum PythonPath {
/// A path that represents the value of [`sys.prefix`] at runtime in Python
/// for a given Python executable.
/// A path that either represents the value of [`sys.prefix`] at runtime in Python
/// for a given Python executable, or which represents a path relative to `sys.prefix`
/// that we will attempt later to resolve into `sys.prefix`. Exactly which this variant
/// represents depends on the [`SysPrefixPathOrigin`] element in the tuple.
///
/// For the case of a virtual environment, where a
/// Python binary is at `/.venv/bin/python`, `sys.prefix` is the path to
@@ -275,13 +277,7 @@ pub enum PythonPath {
/// `/opt/homebrew/lib/python3.X/site-packages`.
///
/// [`sys.prefix`]: https://docs.python.org/3/library/sys.html#sys.prefix
SysPrefix(SystemPathBuf, SysPrefixPathOrigin),
/// Resolve a path to an executable (or environment directory) into a usable environment.
Resolve(SystemPathBuf, SysPrefixPathOrigin),
/// Tries to discover a virtual environment in the given path.
Discover(SystemPathBuf),
IntoSysPrefix(SystemPathBuf, SysPrefixPathOrigin),
/// Resolved site packages paths.
///
@@ -291,16 +287,8 @@ pub enum PythonPath {
}
impl PythonPath {
pub fn from_virtual_env_var(path: impl Into<SystemPathBuf>) -> Self {
Self::SysPrefix(path.into(), SysPrefixPathOrigin::VirtualEnvVar)
}
pub fn from_conda_prefix_var(path: impl Into<SystemPathBuf>) -> Self {
Self::Resolve(path.into(), SysPrefixPathOrigin::CondaPrefixVar)
}
pub fn from_cli_flag(path: SystemPathBuf) -> Self {
Self::Resolve(path, SysPrefixPathOrigin::PythonCliFlag)
pub fn sys_prefix(path: impl Into<SystemPathBuf>, origin: SysPrefixPathOrigin) -> Self {
Self::IntoSysPrefix(path.into(), origin)
}
}

View File

@@ -19,9 +19,9 @@ use crate::semantic_index::builder::SemanticIndexBuilder;
use crate::semantic_index::definition::{Definition, DefinitionNodeKey, Definitions};
use crate::semantic_index::expression::Expression;
use crate::semantic_index::narrowing_constraints::ScopedNarrowingConstraint;
use crate::semantic_index::symbol::{
FileScopeId, NodeWithScopeKey, NodeWithScopeRef, Scope, ScopeId, ScopeKind, ScopedSymbolId,
SymbolTable,
use crate::semantic_index::place::{
FileScopeId, NodeWithScopeKey, NodeWithScopeRef, PlaceExpr, PlaceTable, Scope, ScopeId,
ScopeKind, ScopedPlaceId,
};
use crate::semantic_index::use_def::{EagerSnapshotKey, ScopedEagerSnapshotId, UseDefMap};
@@ -30,9 +30,9 @@ mod builder;
pub mod definition;
pub mod expression;
pub(crate) mod narrowing_constraints;
pub mod place;
pub(crate) mod predicate;
mod re_exports;
pub mod symbol;
mod use_def;
mod visibility_constraints;
@@ -41,7 +41,7 @@ pub(crate) use self::use_def::{
DeclarationsIterator,
};
type SymbolMap = hashbrown::HashMap<ScopedSymbolId, (), FxBuildHasher>;
type PlaceSet = hashbrown::HashMap<ScopedPlaceId, (), FxBuildHasher>;
/// Returns the semantic index for `file`.
///
@@ -55,18 +55,18 @@ pub(crate) fn semantic_index(db: &dyn Db, file: File) -> SemanticIndex<'_> {
SemanticIndexBuilder::new(db, file, parsed).build()
}
/// Returns the symbol table for a specific `scope`.
/// Returns the place table for a specific `scope`.
///
/// Using [`symbol_table`] over [`semantic_index`] has the advantage that
/// Salsa can avoid invalidating dependent queries if this scope's symbol table
/// Using [`place_table`] over [`semantic_index`] has the advantage that
/// Salsa can avoid invalidating dependent queries if this scope's place table
/// is unchanged.
#[salsa::tracked(returns(deref))]
pub(crate) fn symbol_table<'db>(db: &'db dyn Db, scope: ScopeId<'db>) -> Arc<SymbolTable> {
pub(crate) fn place_table<'db>(db: &'db dyn Db, scope: ScopeId<'db>) -> Arc<PlaceTable> {
let file = scope.file(db);
let _span = tracing::trace_span!("symbol_table", scope=?scope.as_id(), ?file).entered();
let _span = tracing::trace_span!("place_table", scope=?scope.as_id(), ?file).entered();
let index = semantic_index(db, file);
index.symbol_table(scope.file_scope_id(db))
index.place_table(scope.file_scope_id(db))
}
/// Returns the set of modules that are imported anywhere in `file`.
@@ -113,13 +113,10 @@ pub(crate) fn attribute_assignments<'db, 's>(
let index = semantic_index(db, file);
attribute_scopes(db, class_body_scope).filter_map(|function_scope_id| {
let attribute_table = index.instance_attribute_table(function_scope_id);
let symbol = attribute_table.symbol_id_by_name(name)?;
let place_table = index.place_table(function_scope_id);
let place = place_table.place_id_by_instance_attribute_name(name)?;
let use_def = &index.use_def_maps[function_scope_id];
Some((
use_def.instance_attribute_bindings(symbol),
function_scope_id,
))
Some((use_def.public_bindings(place), function_scope_id))
})
}
@@ -167,14 +164,11 @@ pub(crate) enum EagerSnapshotResult<'map, 'db> {
NoLongerInEagerContext,
}
/// The symbol tables and use-def maps for all scopes in a file.
/// The place tables and use-def maps for all scopes in a file.
#[derive(Debug, Update)]
pub(crate) struct SemanticIndex<'db> {
/// List of all symbol tables in this file, indexed by scope.
symbol_tables: IndexVec<FileScopeId, Arc<SymbolTable>>,
/// List of all instance attribute tables in this file, indexed by scope.
instance_attribute_tables: IndexVec<FileScopeId, SymbolTable>,
/// List of all place tables in this file, indexed by scope.
place_tables: IndexVec<FileScopeId, Arc<PlaceTable>>,
/// List of all scopes in this file.
scopes: IndexVec<FileScopeId, Scope>,
@@ -195,7 +189,7 @@ pub(crate) struct SemanticIndex<'db> {
scope_ids_by_scope: IndexVec<FileScopeId, ScopeId<'db>>,
/// Map from the file-local [`FileScopeId`] to the set of explicit-global symbols it contains.
globals_by_scope: FxHashMap<FileScopeId, FxHashSet<ScopedSymbolId>>,
globals_by_scope: FxHashMap<FileScopeId, FxHashSet<ScopedPlaceId>>,
/// Use-def map for each scope in this file.
use_def_maps: IndexVec<FileScopeId, Arc<UseDefMap<'db>>>,
@@ -223,17 +217,13 @@ pub(crate) struct SemanticIndex<'db> {
}
impl<'db> SemanticIndex<'db> {
/// Returns the symbol table for a specific scope.
/// Returns the place table for a specific scope.
///
/// Use the Salsa cached [`symbol_table()`] query if you only need the
/// symbol table for a single scope.
/// Use the Salsa cached [`place_table()`] query if you only need the
/// place table for a single scope.
#[track_caller]
pub(super) fn symbol_table(&self, scope_id: FileScopeId) -> Arc<SymbolTable> {
self.symbol_tables[scope_id].clone()
}
pub(super) fn instance_attribute_table(&self, scope_id: FileScopeId) -> &SymbolTable {
&self.instance_attribute_tables[scope_id]
pub(super) fn place_table(&self, scope_id: FileScopeId) -> Arc<PlaceTable> {
self.place_tables[scope_id].clone()
}
/// Returns the use-def map for a specific scope.
@@ -286,7 +276,7 @@ impl<'db> SemanticIndex<'db> {
pub(crate) fn symbol_is_global_in_scope(
&self,
symbol: ScopedSymbolId,
symbol: ScopedPlaceId,
scope: FileScopeId,
) -> bool {
self.globals_by_scope
@@ -444,7 +434,7 @@ impl<'db> SemanticIndex<'db> {
pub(crate) fn eager_snapshot(
&self,
enclosing_scope: FileScopeId,
symbol: &str,
expr: &PlaceExpr,
nested_scope: FileScopeId,
) -> EagerSnapshotResult<'_, 'db> {
for (ancestor_scope_id, ancestor_scope) in self.ancestor_scopes(nested_scope) {
@@ -455,12 +445,12 @@ impl<'db> SemanticIndex<'db> {
return EagerSnapshotResult::NoLongerInEagerContext;
}
}
let Some(symbol_id) = self.symbol_tables[enclosing_scope].symbol_id_by_name(symbol) else {
let Some(place_id) = self.place_tables[enclosing_scope].place_id_by_expr(expr) else {
return EagerSnapshotResult::NotFound;
};
let key = EagerSnapshotKey {
enclosing_scope,
enclosing_symbol: symbol_id,
enclosing_place: place_id,
nested_scope,
};
let Some(id) = self.eager_snapshots.get(&key) else {
@@ -480,9 +470,9 @@ pub struct AncestorsIter<'a> {
}
impl<'a> AncestorsIter<'a> {
fn new(module_symbol_table: &'a SemanticIndex, start: FileScopeId) -> Self {
fn new(module_table: &'a SemanticIndex, start: FileScopeId) -> Self {
Self {
scopes: &module_symbol_table.scopes,
scopes: &module_table.scopes,
next_id: Some(start),
}
}
@@ -508,9 +498,9 @@ pub struct DescendantsIter<'a> {
}
impl<'a> DescendantsIter<'a> {
fn new(symbol_table: &'a SemanticIndex, scope_id: FileScopeId) -> Self {
let scope = &symbol_table.scopes[scope_id];
let scopes = &symbol_table.scopes[scope.descendants()];
fn new(index: &'a SemanticIndex, scope_id: FileScopeId) -> Self {
let scope = &index.scopes[scope_id];
let scopes = &index.scopes[scope.descendants()];
Self {
next_id: scope_id + 1,
@@ -545,8 +535,8 @@ pub struct ChildrenIter<'a> {
}
impl<'a> ChildrenIter<'a> {
pub(crate) fn new(module_symbol_table: &'a SemanticIndex, parent: FileScopeId) -> Self {
let descendants = DescendantsIter::new(module_symbol_table, parent);
pub(crate) fn new(module_index: &'a SemanticIndex, parent: FileScopeId) -> Self {
let descendants = DescendantsIter::new(module_index, parent);
Self {
parent,
@@ -577,21 +567,19 @@ mod tests {
use crate::db::tests::{TestDb, TestDbBuilder};
use crate::semantic_index::ast_ids::{HasScopedUseId, ScopedUseId};
use crate::semantic_index::definition::{Definition, DefinitionKind};
use crate::semantic_index::symbol::{
FileScopeId, Scope, ScopeKind, ScopedSymbolId, SymbolTable,
};
use crate::semantic_index::place::{FileScopeId, PlaceTable, Scope, ScopeKind, ScopedPlaceId};
use crate::semantic_index::use_def::UseDefMap;
use crate::semantic_index::{global_scope, semantic_index, symbol_table, use_def_map};
use crate::semantic_index::{global_scope, place_table, semantic_index, use_def_map};
impl UseDefMap<'_> {
fn first_public_binding(&self, symbol: ScopedSymbolId) -> Option<Definition<'_>> {
fn first_public_binding(&self, symbol: ScopedPlaceId) -> Option<Definition<'_>> {
self.public_bindings(symbol)
.find_map(|constrained_binding| constrained_binding.binding)
.find_map(|constrained_binding| constrained_binding.binding.definition())
}
fn first_binding_at_use(&self, use_id: ScopedUseId) -> Option<Definition<'_>> {
self.bindings_at_use(use_id)
.find_map(|constrained_binding| constrained_binding.binding)
.find_map(|constrained_binding| constrained_binding.binding.definition())
}
}
@@ -613,17 +601,17 @@ mod tests {
TestCase { db, file }
}
fn names(table: &SymbolTable) -> Vec<String> {
fn names(table: &PlaceTable) -> Vec<String> {
table
.symbols()
.map(|symbol| symbol.name().to_string())
.places()
.filter_map(|expr| Some(expr.as_name()?.to_string()))
.collect()
}
#[test]
fn empty() {
let TestCase { db, file } = test_case("");
let global_table = symbol_table(&db, global_scope(&db, file));
let global_table = place_table(&db, global_scope(&db, file));
let global_names = names(global_table);
@@ -633,7 +621,7 @@ mod tests {
#[test]
fn simple() {
let TestCase { db, file } = test_case("x");
let global_table = symbol_table(&db, global_scope(&db, file));
let global_table = place_table(&db, global_scope(&db, file));
assert_eq!(names(global_table), vec!["x"]);
}
@@ -641,7 +629,7 @@ mod tests {
#[test]
fn annotation_only() {
let TestCase { db, file } = test_case("x: int");
let global_table = symbol_table(&db, global_scope(&db, file));
let global_table = place_table(&db, global_scope(&db, file));
assert_eq!(names(global_table), vec!["int", "x"]);
// TODO record definition
@@ -651,10 +639,10 @@ mod tests {
fn import() {
let TestCase { db, file } = test_case("import foo");
let scope = global_scope(&db, file);
let global_table = symbol_table(&db, scope);
let global_table = place_table(&db, scope);
assert_eq!(names(global_table), vec!["foo"]);
let foo = global_table.symbol_id_by_name("foo").unwrap();
let foo = global_table.place_id_by_name("foo").unwrap();
let use_def = use_def_map(&db, scope);
let binding = use_def.first_public_binding(foo).unwrap();
@@ -664,7 +652,7 @@ mod tests {
#[test]
fn import_sub() {
let TestCase { db, file } = test_case("import foo.bar");
let global_table = symbol_table(&db, global_scope(&db, file));
let global_table = place_table(&db, global_scope(&db, file));
assert_eq!(names(global_table), vec!["foo"]);
}
@@ -672,7 +660,7 @@ mod tests {
#[test]
fn import_as() {
let TestCase { db, file } = test_case("import foo.bar as baz");
let global_table = symbol_table(&db, global_scope(&db, file));
let global_table = place_table(&db, global_scope(&db, file));
assert_eq!(names(global_table), vec!["baz"]);
}
@@ -681,12 +669,12 @@ mod tests {
fn import_from() {
let TestCase { db, file } = test_case("from bar import foo");
let scope = global_scope(&db, file);
let global_table = symbol_table(&db, scope);
let global_table = place_table(&db, scope);
assert_eq!(names(global_table), vec!["foo"]);
assert!(
global_table
.symbol_by_name("foo")
.place_by_name("foo")
.is_some_and(|symbol| { symbol.is_bound() && !symbol.is_used() }),
"symbols that are defined get the defined flag"
);
@@ -695,7 +683,7 @@ mod tests {
let binding = use_def
.first_public_binding(
global_table
.symbol_id_by_name("foo")
.place_id_by_name("foo")
.expect("symbol to exist"),
)
.unwrap();
@@ -706,18 +694,18 @@ mod tests {
fn assign() {
let TestCase { db, file } = test_case("x = foo");
let scope = global_scope(&db, file);
let global_table = symbol_table(&db, scope);
let global_table = place_table(&db, scope);
assert_eq!(names(global_table), vec!["foo", "x"]);
assert!(
global_table
.symbol_by_name("foo")
.place_by_name("foo")
.is_some_and(|symbol| { !symbol.is_bound() && symbol.is_used() }),
"a symbol used but not bound in a scope should have only the used flag"
);
let use_def = use_def_map(&db, scope);
let binding = use_def
.first_public_binding(global_table.symbol_id_by_name("x").expect("symbol exists"))
.first_public_binding(global_table.place_id_by_name("x").expect("symbol exists"))
.unwrap();
assert!(matches!(binding.kind(&db), DefinitionKind::Assignment(_)));
}
@@ -726,13 +714,13 @@ mod tests {
fn augmented_assignment() {
let TestCase { db, file } = test_case("x += 1");
let scope = global_scope(&db, file);
let global_table = symbol_table(&db, scope);
let global_table = place_table(&db, scope);
assert_eq!(names(global_table), vec!["x"]);
let use_def = use_def_map(&db, scope);
let binding = use_def
.first_public_binding(global_table.symbol_id_by_name("x").unwrap())
.first_public_binding(global_table.place_id_by_name("x").unwrap())
.unwrap();
assert!(matches!(
@@ -750,7 +738,7 @@ class C:
y = 2
",
);
let global_table = symbol_table(&db, global_scope(&db, file));
let global_table = place_table(&db, global_scope(&db, file));
assert_eq!(names(global_table), vec!["C", "y"]);
@@ -765,12 +753,12 @@ y = 2
assert_eq!(class_scope.kind(), ScopeKind::Class);
assert_eq!(class_scope_id.to_scope_id(&db, file).name(&db), "C");
let class_table = index.symbol_table(class_scope_id);
let class_table = index.place_table(class_scope_id);
assert_eq!(names(&class_table), vec!["x"]);
let use_def = index.use_def_map(class_scope_id);
let binding = use_def
.first_public_binding(class_table.symbol_id_by_name("x").expect("symbol exists"))
.first_public_binding(class_table.place_id_by_name("x").expect("symbol exists"))
.unwrap();
assert!(matches!(binding.kind(&db), DefinitionKind::Assignment(_)));
}
@@ -785,7 +773,7 @@ y = 2
",
);
let index = semantic_index(&db, file);
let global_table = index.symbol_table(FileScopeId::global());
let global_table = index.place_table(FileScopeId::global());
assert_eq!(names(&global_table), vec!["func", "y"]);
@@ -798,16 +786,12 @@ y = 2
assert_eq!(function_scope.kind(), ScopeKind::Function);
assert_eq!(function_scope_id.to_scope_id(&db, file).name(&db), "func");
let function_table = index.symbol_table(function_scope_id);
let function_table = index.place_table(function_scope_id);
assert_eq!(names(&function_table), vec!["x"]);
let use_def = index.use_def_map(function_scope_id);
let binding = use_def
.first_public_binding(
function_table
.symbol_id_by_name("x")
.expect("symbol exists"),
)
.first_public_binding(function_table.place_id_by_name("x").expect("symbol exists"))
.unwrap();
assert!(matches!(binding.kind(&db), DefinitionKind::Assignment(_)));
}
@@ -822,7 +806,7 @@ def f(a: str, /, b: str, c: int = 1, *args, d: int = 2, **kwargs):
);
let index = semantic_index(&db, file);
let global_table = symbol_table(&db, global_scope(&db, file));
let global_table = place_table(&db, global_scope(&db, file));
assert_eq!(names(global_table), vec!["str", "int", "f"]);
@@ -833,7 +817,7 @@ def f(a: str, /, b: str, c: int = 1, *args, d: int = 2, **kwargs):
panic!("Expected a function scope")
};
let function_table = index.symbol_table(function_scope_id);
let function_table = index.place_table(function_scope_id);
assert_eq!(
names(&function_table),
vec!["a", "b", "c", "d", "args", "kwargs"],
@@ -844,7 +828,7 @@ def f(a: str, /, b: str, c: int = 1, *args, d: int = 2, **kwargs):
let binding = use_def
.first_public_binding(
function_table
.symbol_id_by_name(name)
.place_id_by_name(name)
.expect("symbol exists"),
)
.unwrap();
@@ -853,7 +837,7 @@ def f(a: str, /, b: str, c: int = 1, *args, d: int = 2, **kwargs):
let args_binding = use_def
.first_public_binding(
function_table
.symbol_id_by_name("args")
.place_id_by_name("args")
.expect("symbol exists"),
)
.unwrap();
@@ -864,7 +848,7 @@ def f(a: str, /, b: str, c: int = 1, *args, d: int = 2, **kwargs):
let kwargs_binding = use_def
.first_public_binding(
function_table
.symbol_id_by_name("kwargs")
.place_id_by_name("kwargs")
.expect("symbol exists"),
)
.unwrap();
@@ -879,7 +863,7 @@ def f(a: str, /, b: str, c: int = 1, *args, d: int = 2, **kwargs):
let TestCase { db, file } = test_case("lambda a, b, c=1, *args, d=2, **kwargs: None");
let index = semantic_index(&db, file);
let global_table = symbol_table(&db, global_scope(&db, file));
let global_table = place_table(&db, global_scope(&db, file));
assert!(names(global_table).is_empty());
@@ -890,7 +874,7 @@ def f(a: str, /, b: str, c: int = 1, *args, d: int = 2, **kwargs):
panic!("Expected a lambda scope")
};
let lambda_table = index.symbol_table(lambda_scope_id);
let lambda_table = index.place_table(lambda_scope_id);
assert_eq!(
names(&lambda_table),
vec!["a", "b", "c", "d", "args", "kwargs"],
@@ -899,14 +883,14 @@ def f(a: str, /, b: str, c: int = 1, *args, d: int = 2, **kwargs):
let use_def = index.use_def_map(lambda_scope_id);
for name in ["a", "b", "c", "d"] {
let binding = use_def
.first_public_binding(lambda_table.symbol_id_by_name(name).expect("symbol exists"))
.first_public_binding(lambda_table.place_id_by_name(name).expect("symbol exists"))
.unwrap();
assert!(matches!(binding.kind(&db), DefinitionKind::Parameter(_)));
}
let args_binding = use_def
.first_public_binding(
lambda_table
.symbol_id_by_name("args")
.place_id_by_name("args")
.expect("symbol exists"),
)
.unwrap();
@@ -917,7 +901,7 @@ def f(a: str, /, b: str, c: int = 1, *args, d: int = 2, **kwargs):
let kwargs_binding = use_def
.first_public_binding(
lambda_table
.symbol_id_by_name("kwargs")
.place_id_by_name("kwargs")
.expect("symbol exists"),
)
.unwrap();
@@ -938,7 +922,7 @@ def f(a: str, /, b: str, c: int = 1, *args, d: int = 2, **kwargs):
);
let index = semantic_index(&db, file);
let global_table = index.symbol_table(FileScopeId::global());
let global_table = index.place_table(FileScopeId::global());
assert_eq!(names(&global_table), vec!["iter1"]);
@@ -955,7 +939,7 @@ def f(a: str, /, b: str, c: int = 1, *args, d: int = 2, **kwargs):
"<listcomp>"
);
let comprehension_symbol_table = index.symbol_table(comprehension_scope_id);
let comprehension_symbol_table = index.place_table(comprehension_scope_id);
assert_eq!(names(&comprehension_symbol_table), vec!["x", "y"]);
@@ -964,7 +948,7 @@ def f(a: str, /, b: str, c: int = 1, *args, d: int = 2, **kwargs):
let binding = use_def
.first_public_binding(
comprehension_symbol_table
.symbol_id_by_name(name)
.place_id_by_name(name)
.expect("symbol exists"),
)
.unwrap();
@@ -1031,7 +1015,7 @@ def f(a: str, /, b: str, c: int = 1, *args, d: int = 2, **kwargs):
);
let index = semantic_index(&db, file);
let global_table = index.symbol_table(FileScopeId::global());
let global_table = index.place_table(FileScopeId::global());
assert_eq!(names(&global_table), vec!["iter1"]);
@@ -1048,7 +1032,7 @@ def f(a: str, /, b: str, c: int = 1, *args, d: int = 2, **kwargs):
"<listcomp>"
);
let comprehension_symbol_table = index.symbol_table(comprehension_scope_id);
let comprehension_symbol_table = index.place_table(comprehension_scope_id);
assert_eq!(names(&comprehension_symbol_table), vec!["y", "iter2"]);
@@ -1067,7 +1051,7 @@ def f(a: str, /, b: str, c: int = 1, *args, d: int = 2, **kwargs):
"<setcomp>"
);
let inner_comprehension_symbol_table = index.symbol_table(inner_comprehension_scope_id);
let inner_comprehension_symbol_table = index.place_table(inner_comprehension_scope_id);
assert_eq!(names(&inner_comprehension_symbol_table), vec!["x"]);
}
@@ -1082,14 +1066,14 @@ with item1 as x, item2 as y:
);
let index = semantic_index(&db, file);
let global_table = index.symbol_table(FileScopeId::global());
let global_table = index.place_table(FileScopeId::global());
assert_eq!(names(&global_table), vec!["item1", "x", "item2", "y"]);
let use_def = index.use_def_map(FileScopeId::global());
for name in ["x", "y"] {
let binding = use_def
.first_public_binding(global_table.symbol_id_by_name(name).expect("symbol exists"))
.first_public_binding(global_table.place_id_by_name(name).expect("symbol exists"))
.expect("Expected with item definition for {name}");
assert!(matches!(binding.kind(&db), DefinitionKind::WithItem(_)));
}
@@ -1105,14 +1089,14 @@ with context() as (x, y):
);
let index = semantic_index(&db, file);
let global_table = index.symbol_table(FileScopeId::global());
let global_table = index.place_table(FileScopeId::global());
assert_eq!(names(&global_table), vec!["context", "x", "y"]);
let use_def = index.use_def_map(FileScopeId::global());
for name in ["x", "y"] {
let binding = use_def
.first_public_binding(global_table.symbol_id_by_name(name).expect("symbol exists"))
.first_public_binding(global_table.place_id_by_name(name).expect("symbol exists"))
.expect("Expected with item definition for {name}");
assert!(matches!(binding.kind(&db), DefinitionKind::WithItem(_)));
}
@@ -1129,7 +1113,7 @@ def func():
",
);
let index = semantic_index(&db, file);
let global_table = index.symbol_table(FileScopeId::global());
let global_table = index.place_table(FileScopeId::global());
assert_eq!(names(&global_table), vec!["func"]);
let [
@@ -1148,8 +1132,8 @@ def func():
assert_eq!(func_scope_2.kind(), ScopeKind::Function);
assert_eq!(func_scope2_id.to_scope_id(&db, file).name(&db), "func");
let func1_table = index.symbol_table(func_scope1_id);
let func2_table = index.symbol_table(func_scope2_id);
let func1_table = index.place_table(func_scope1_id);
let func2_table = index.place_table(func_scope2_id);
assert_eq!(names(&func1_table), vec!["x"]);
assert_eq!(names(&func2_table), vec!["y"]);
@@ -1157,7 +1141,7 @@ def func():
let binding = use_def
.first_public_binding(
global_table
.symbol_id_by_name("func")
.place_id_by_name("func")
.expect("symbol exists"),
)
.unwrap();
@@ -1174,7 +1158,7 @@ def func[T]():
);
let index = semantic_index(&db, file);
let global_table = index.symbol_table(FileScopeId::global());
let global_table = index.place_table(FileScopeId::global());
assert_eq!(names(&global_table), vec!["func"]);
@@ -1187,7 +1171,7 @@ def func[T]():
assert_eq!(ann_scope.kind(), ScopeKind::Annotation);
assert_eq!(ann_scope_id.to_scope_id(&db, file).name(&db), "func");
let ann_table = index.symbol_table(ann_scope_id);
let ann_table = index.place_table(ann_scope_id);
assert_eq!(names(&ann_table), vec!["T"]);
let [(func_scope_id, func_scope)] =
@@ -1197,7 +1181,7 @@ def func[T]():
};
assert_eq!(func_scope.kind(), ScopeKind::Function);
assert_eq!(func_scope_id.to_scope_id(&db, file).name(&db), "func");
let func_table = index.symbol_table(func_scope_id);
let func_table = index.place_table(func_scope_id);
assert_eq!(names(&func_table), vec!["x"]);
}
@@ -1211,7 +1195,7 @@ class C[T]:
);
let index = semantic_index(&db, file);
let global_table = index.symbol_table(FileScopeId::global());
let global_table = index.place_table(FileScopeId::global());
assert_eq!(names(&global_table), vec!["C"]);
@@ -1224,11 +1208,11 @@ class C[T]:
assert_eq!(ann_scope.kind(), ScopeKind::Annotation);
assert_eq!(ann_scope_id.to_scope_id(&db, file).name(&db), "C");
let ann_table = index.symbol_table(ann_scope_id);
let ann_table = index.place_table(ann_scope_id);
assert_eq!(names(&ann_table), vec!["T"]);
assert!(
ann_table
.symbol_by_name("T")
.place_by_name("T")
.is_some_and(|s| s.is_bound() && !s.is_used()),
"type parameters are defined by the scope that introduces them"
);
@@ -1241,7 +1225,7 @@ class C[T]:
assert_eq!(class_scope.kind(), ScopeKind::Class);
assert_eq!(class_scope_id.to_scope_id(&db, file).name(&db), "C");
assert_eq!(names(&index.symbol_table(class_scope_id)), vec!["x"]);
assert_eq!(names(&index.place_table(class_scope_id)), vec!["x"]);
}
#[test]
@@ -1369,9 +1353,9 @@ match subject:
);
let global_scope_id = global_scope(&db, file);
let global_table = symbol_table(&db, global_scope_id);
let global_table = place_table(&db, global_scope_id);
assert!(global_table.symbol_by_name("Foo").unwrap().is_used());
assert!(global_table.place_by_name("Foo").unwrap().is_used());
assert_eq!(
names(global_table),
vec![
@@ -1395,7 +1379,7 @@ match subject:
("l", 1),
] {
let binding = use_def
.first_public_binding(global_table.symbol_id_by_name(name).expect("symbol exists"))
.first_public_binding(global_table.place_id_by_name(name).expect("symbol exists"))
.expect("Expected with item definition for {name}");
if let DefinitionKind::MatchPattern(pattern) = binding.kind(&db) {
assert_eq!(pattern.index(), expected_index);
@@ -1418,14 +1402,14 @@ match 1:
);
let global_scope_id = global_scope(&db, file);
let global_table = symbol_table(&db, global_scope_id);
let global_table = place_table(&db, global_scope_id);
assert_eq!(names(global_table), vec!["first", "second"]);
let use_def = use_def_map(&db, global_scope_id);
for (name, expected_index) in [("first", 0), ("second", 0)] {
let binding = use_def
.first_public_binding(global_table.symbol_id_by_name(name).expect("symbol exists"))
.first_public_binding(global_table.place_id_by_name(name).expect("symbol exists"))
.expect("Expected with item definition for {name}");
if let DefinitionKind::MatchPattern(pattern) = binding.kind(&db) {
assert_eq!(pattern.index(), expected_index);
@@ -1439,13 +1423,13 @@ match 1:
fn for_loops_single_assignment() {
let TestCase { db, file } = test_case("for x in a: pass");
let scope = global_scope(&db, file);
let global_table = symbol_table(&db, scope);
let global_table = place_table(&db, scope);
assert_eq!(&names(global_table), &["a", "x"]);
let use_def = use_def_map(&db, scope);
let binding = use_def
.first_public_binding(global_table.symbol_id_by_name("x").unwrap())
.first_public_binding(global_table.place_id_by_name("x").unwrap())
.unwrap();
assert!(matches!(binding.kind(&db), DefinitionKind::For(_)));
@@ -1455,16 +1439,16 @@ match 1:
fn for_loops_simple_unpacking() {
let TestCase { db, file } = test_case("for (x, y) in a: pass");
let scope = global_scope(&db, file);
let global_table = symbol_table(&db, scope);
let global_table = place_table(&db, scope);
assert_eq!(&names(global_table), &["a", "x", "y"]);
let use_def = use_def_map(&db, scope);
let x_binding = use_def
.first_public_binding(global_table.symbol_id_by_name("x").unwrap())
.first_public_binding(global_table.place_id_by_name("x").unwrap())
.unwrap();
let y_binding = use_def
.first_public_binding(global_table.symbol_id_by_name("y").unwrap())
.first_public_binding(global_table.place_id_by_name("y").unwrap())
.unwrap();
assert!(matches!(x_binding.kind(&db), DefinitionKind::For(_)));
@@ -1475,13 +1459,13 @@ match 1:
fn for_loops_complex_unpacking() {
let TestCase { db, file } = test_case("for [((a,) b), (c, d)] in e: pass");
let scope = global_scope(&db, file);
let global_table = symbol_table(&db, scope);
let global_table = place_table(&db, scope);
assert_eq!(&names(global_table), &["e", "a", "b", "c", "d"]);
let use_def = use_def_map(&db, scope);
let binding = use_def
.first_public_binding(global_table.symbol_id_by_name("a").unwrap())
.first_public_binding(global_table.place_id_by_name("a").unwrap())
.unwrap();
assert!(matches!(binding.kind(&db), DefinitionKind::For(_)));

View File

@@ -6,14 +6,14 @@ use ruff_python_ast::ExprRef;
use crate::Db;
use crate::semantic_index::ast_ids::node_key::ExpressionNodeKey;
use crate::semantic_index::place::ScopeId;
use crate::semantic_index::semantic_index;
use crate::semantic_index::symbol::ScopeId;
/// AST ids for a single scope.
///
/// The motivation for building the AST ids per scope isn't about reducing invalidation because
/// the struct changes whenever the parsed AST changes. Instead, it's mainly that we can
/// build the AST ids struct when building the symbol table and also keep the property that
/// build the AST ids struct when building the place table and also keep the property that
/// IDs of outer scopes are unaffected by changes in inner scopes.
///
/// For example, we don't want that adding new statements to `foo` changes the statement id of `x = foo()` in:
@@ -28,7 +28,7 @@ use crate::semantic_index::symbol::ScopeId;
pub(crate) struct AstIds {
/// Maps expressions to their expression id.
expressions_map: FxHashMap<ExpressionNodeKey, ScopedExpressionId>,
/// Maps expressions which "use" a symbol (that is, [`ast::ExprName`]) to a use id.
/// Maps expressions which "use" a place (that is, [`ast::ExprName`], [`ast::ExprAttribute`] or [`ast::ExprSubscript`]) to a use id.
uses_map: FxHashMap<ExpressionNodeKey, ScopedUseId>,
}
@@ -49,7 +49,7 @@ fn ast_ids<'db>(db: &'db dyn Db, scope: ScopeId) -> &'db AstIds {
semantic_index(db, scope.file(db)).ast_ids(scope.file_scope_id(db))
}
/// Uniquely identifies a use of a name in a [`crate::semantic_index::symbol::FileScopeId`].
/// Uniquely identifies a use of a name in a [`crate::semantic_index::place::FileScopeId`].
#[newtype_index]
pub struct ScopedUseId;
@@ -72,6 +72,20 @@ impl HasScopedUseId for ast::ExprName {
}
}
impl HasScopedUseId for ast::ExprAttribute {
fn scoped_use_id(&self, db: &dyn Db, scope: ScopeId) -> ScopedUseId {
let expression_ref = ExprRef::from(self);
expression_ref.scoped_use_id(db, scope)
}
}
impl HasScopedUseId for ast::ExprSubscript {
fn scoped_use_id(&self, db: &dyn Db, scope: ScopeId) -> ScopedUseId {
let expression_ref = ExprRef::from(self);
expression_ref.scoped_use_id(db, scope)
}
}
impl HasScopedUseId for ast::ExprRef<'_> {
fn scoped_use_id(&self, db: &dyn Db, scope: ScopeId) -> ScopedUseId {
let ast_ids = ast_ids(db, scope);
@@ -79,7 +93,7 @@ impl HasScopedUseId for ast::ExprRef<'_> {
}
}
/// Uniquely identifies an [`ast::Expr`] in a [`crate::semantic_index::symbol::FileScopeId`].
/// Uniquely identifies an [`ast::Expr`] in a [`crate::semantic_index::place::FileScopeId`].
#[newtype_index]
#[derive(salsa::Update)]
pub struct ScopedExpressionId;

View File

@@ -24,24 +24,22 @@ use crate::semantic_index::SemanticIndex;
use crate::semantic_index::ast_ids::AstIdsBuilder;
use crate::semantic_index::ast_ids::node_key::ExpressionNodeKey;
use crate::semantic_index::definition::{
AnnotatedAssignmentDefinitionKind, AnnotatedAssignmentDefinitionNodeRef,
AssignmentDefinitionKind, AssignmentDefinitionNodeRef, ComprehensionDefinitionKind,
ComprehensionDefinitionNodeRef, Definition, DefinitionCategory, DefinitionKind,
DefinitionNodeKey, DefinitionNodeRef, Definitions, ExceptHandlerDefinitionNodeRef,
ForStmtDefinitionKind, ForStmtDefinitionNodeRef, ImportDefinitionNodeRef,
ImportFromDefinitionNodeRef, MatchPatternDefinitionNodeRef, StarImportDefinitionNodeRef,
TargetKind, WithItemDefinitionKind, WithItemDefinitionNodeRef,
AnnotatedAssignmentDefinitionNodeRef, AssignmentDefinitionNodeRef,
ComprehensionDefinitionNodeRef, Definition, DefinitionCategory, DefinitionNodeKey,
DefinitionNodeRef, Definitions, ExceptHandlerDefinitionNodeRef, ForStmtDefinitionNodeRef,
ImportDefinitionNodeRef, ImportFromDefinitionNodeRef, MatchPatternDefinitionNodeRef,
StarImportDefinitionNodeRef, WithItemDefinitionNodeRef,
};
use crate::semantic_index::expression::{Expression, ExpressionKind};
use crate::semantic_index::place::{
FileScopeId, NodeWithScopeKey, NodeWithScopeKind, NodeWithScopeRef, PlaceExpr,
PlaceTableBuilder, Scope, ScopeId, ScopeKind, ScopedPlaceId,
};
use crate::semantic_index::predicate::{
PatternPredicate, PatternPredicateKind, Predicate, PredicateNode, ScopedPredicateId,
StarImportPlaceholderPredicate,
};
use crate::semantic_index::re_exports::exported_names;
use crate::semantic_index::symbol::{
FileScopeId, NodeWithScopeKey, NodeWithScopeKind, NodeWithScopeRef, Scope, ScopeId, ScopeKind,
ScopedSymbolId, SymbolTableBuilder,
};
use crate::semantic_index::use_def::{
EagerSnapshotKey, FlowSnapshot, ScopedEagerSnapshotId, UseDefMapBuilder,
};
@@ -100,13 +98,12 @@ pub(super) struct SemanticIndexBuilder<'db> {
// Semantic Index fields
scopes: IndexVec<FileScopeId, Scope>,
scope_ids_by_scope: IndexVec<FileScopeId, ScopeId<'db>>,
symbol_tables: IndexVec<FileScopeId, SymbolTableBuilder>,
instance_attribute_tables: IndexVec<FileScopeId, SymbolTableBuilder>,
place_tables: IndexVec<FileScopeId, PlaceTableBuilder>,
ast_ids: IndexVec<FileScopeId, AstIdsBuilder>,
use_def_maps: IndexVec<FileScopeId, UseDefMapBuilder<'db>>,
scopes_by_node: FxHashMap<NodeWithScopeKey, FileScopeId>,
scopes_by_expression: FxHashMap<ExpressionNodeKey, FileScopeId>,
globals_by_scope: FxHashMap<FileScopeId, FxHashSet<ScopedSymbolId>>,
globals_by_scope: FxHashMap<FileScopeId, FxHashSet<ScopedPlaceId>>,
definitions_by_node: FxHashMap<DefinitionNodeKey, Definitions<'db>>,
expressions_by_node: FxHashMap<ExpressionNodeKey, Expression<'db>>,
imported_modules: FxHashSet<ModuleName>,
@@ -135,8 +132,7 @@ impl<'db> SemanticIndexBuilder<'db> {
has_future_annotations: false,
scopes: IndexVec::new(),
symbol_tables: IndexVec::new(),
instance_attribute_tables: IndexVec::new(),
place_tables: IndexVec::new(),
ast_ids: IndexVec::new(),
scope_ids_by_scope: IndexVec::new(),
use_def_maps: IndexVec::new(),
@@ -259,9 +255,7 @@ impl<'db> SemanticIndexBuilder<'db> {
self.try_node_context_stack_manager.enter_nested_scope();
let file_scope_id = self.scopes.push(scope);
self.symbol_tables.push(SymbolTableBuilder::default());
self.instance_attribute_tables
.push(SymbolTableBuilder::default());
self.place_tables.push(PlaceTableBuilder::default());
self.use_def_maps
.push(UseDefMapBuilder::new(is_class_scope));
let ast_id_scope = self.ast_ids.push(AstIdsBuilder::default());
@@ -301,36 +295,35 @@ impl<'db> SemanticIndexBuilder<'db> {
// If the scope that we just popped off is an eager scope, we need to "lock" our view of
// which bindings reach each of the uses in the scope. Loop through each enclosing scope,
// looking for any that bind each symbol.
// looking for any that bind each place.
for enclosing_scope_info in self.scope_stack.iter().rev() {
let enclosing_scope_id = enclosing_scope_info.file_scope_id;
let enclosing_scope_kind = self.scopes[enclosing_scope_id].kind();
let enclosing_symbol_table = &self.symbol_tables[enclosing_scope_id];
let enclosing_place_table = &self.place_tables[enclosing_scope_id];
for nested_symbol in self.symbol_tables[popped_scope_id].symbols() {
// Skip this symbol if this enclosing scope doesn't contain any bindings for it.
// Note that even if this symbol is bound in the popped scope,
for nested_place in self.place_tables[popped_scope_id].places() {
// Skip this place if this enclosing scope doesn't contain any bindings for it.
// Note that even if this place is bound in the popped scope,
// it may refer to the enclosing scope bindings
// so we also need to snapshot the bindings of the enclosing scope.
let Some(enclosing_symbol_id) =
enclosing_symbol_table.symbol_id_by_name(nested_symbol.name())
let Some(enclosing_place_id) = enclosing_place_table.place_id_by_expr(nested_place)
else {
continue;
};
let enclosing_symbol = enclosing_symbol_table.symbol(enclosing_symbol_id);
let enclosing_place = enclosing_place_table.place_expr(enclosing_place_id);
// Snapshot the state of this symbol that are visible at this point in this
// Snapshot the state of this place that are visible at this point in this
// enclosing scope.
let key = EagerSnapshotKey {
enclosing_scope: enclosing_scope_id,
enclosing_symbol: enclosing_symbol_id,
enclosing_place: enclosing_place_id,
nested_scope: popped_scope_id,
};
let eager_snapshot = self.use_def_maps[enclosing_scope_id].snapshot_eager_state(
enclosing_symbol_id,
enclosing_place_id,
enclosing_scope_kind,
enclosing_symbol.is_bound(),
enclosing_place,
);
self.eager_snapshots.insert(key, eager_snapshot);
}
@@ -338,7 +331,7 @@ impl<'db> SemanticIndexBuilder<'db> {
// Lazy scopes are "sticky": once we see a lazy scope we stop doing lookups
// eagerly, even if we would encounter another eager enclosing scope later on.
// Also, narrowing constraints outside a lazy scope are not applicable.
// TODO: If the symbol has never been rewritten, they are applicable.
// TODO: If the place has never been rewritten, they are applicable.
if !enclosing_scope_kind.is_eager() {
break;
}
@@ -347,14 +340,9 @@ impl<'db> SemanticIndexBuilder<'db> {
popped_scope_id
}
fn current_symbol_table(&mut self) -> &mut SymbolTableBuilder {
fn current_place_table(&mut self) -> &mut PlaceTableBuilder {
let scope_id = self.current_scope();
&mut self.symbol_tables[scope_id]
}
fn current_attribute_table(&mut self) -> &mut SymbolTableBuilder {
let scope_id = self.current_scope();
&mut self.instance_attribute_tables[scope_id]
&mut self.place_tables[scope_id]
}
fn current_use_def_map_mut(&mut self) -> &mut UseDefMapBuilder<'db> {
@@ -389,34 +377,36 @@ impl<'db> SemanticIndexBuilder<'db> {
self.current_use_def_map_mut().merge(state);
}
/// Add a symbol to the symbol table and the use-def map.
/// Return the [`ScopedSymbolId`] that uniquely identifies the symbol in both.
fn add_symbol(&mut self, name: Name) -> ScopedSymbolId {
let (symbol_id, added) = self.current_symbol_table().add_symbol(name);
/// Add a symbol to the place table and the use-def map.
/// Return the [`ScopedPlaceId`] that uniquely identifies the symbol in both.
fn add_symbol(&mut self, name: Name) -> ScopedPlaceId {
let (place_id, added) = self.current_place_table().add_symbol(name);
if added {
self.current_use_def_map_mut().add_symbol(symbol_id);
self.current_use_def_map_mut().add_place(place_id);
}
symbol_id
place_id
}
fn add_attribute(&mut self, name: Name) -> ScopedSymbolId {
let (symbol_id, added) = self.current_attribute_table().add_symbol(name);
/// Add a place to the place table and the use-def map.
/// Return the [`ScopedPlaceId`] that uniquely identifies the place in both.
fn add_place(&mut self, place_expr: PlaceExpr) -> ScopedPlaceId {
let (place_id, added) = self.current_place_table().add_place(place_expr);
if added {
self.current_use_def_map_mut().add_attribute(symbol_id);
self.current_use_def_map_mut().add_place(place_id);
}
symbol_id
place_id
}
fn mark_symbol_bound(&mut self, id: ScopedSymbolId) {
self.current_symbol_table().mark_symbol_bound(id);
fn mark_place_bound(&mut self, id: ScopedPlaceId) {
self.current_place_table().mark_place_bound(id);
}
fn mark_symbol_declared(&mut self, id: ScopedSymbolId) {
self.current_symbol_table().mark_symbol_declared(id);
fn mark_place_declared(&mut self, id: ScopedPlaceId) {
self.current_place_table().mark_place_declared(id);
}
fn mark_symbol_used(&mut self, id: ScopedSymbolId) {
self.current_symbol_table().mark_symbol_used(id);
fn mark_place_used(&mut self, id: ScopedPlaceId) {
self.current_place_table().mark_place_used(id);
}
fn add_entry_for_definition_key(&mut self, key: DefinitionNodeKey) -> &mut Definitions<'db> {
@@ -432,11 +422,10 @@ impl<'db> SemanticIndexBuilder<'db> {
/// for all nodes *except* [`ast::Alias`] nodes representing `*` imports.
fn add_definition(
&mut self,
symbol: ScopedSymbolId,
place: ScopedPlaceId,
definition_node: impl Into<DefinitionNodeRef<'db>> + std::fmt::Debug + Copy,
) -> Definition<'db> {
let (definition, num_definitions) =
self.push_additional_definition(symbol, definition_node);
let (definition, num_definitions) = self.push_additional_definition(place, definition_node);
debug_assert_eq!(
num_definitions, 1,
"Attempted to create multiple `Definition`s associated with AST node {definition_node:?}"
@@ -444,6 +433,22 @@ impl<'db> SemanticIndexBuilder<'db> {
definition
}
fn delete_associated_bindings(&mut self, place: ScopedPlaceId) {
let scope = self.current_scope();
// Don't delete associated bindings if the scope is a class scope & place is a name (it's never visible to nested scopes)
if self.scopes[scope].kind() == ScopeKind::Class
&& self.place_tables[scope].place_expr(place).is_name()
{
return;
}
for associated_place in self.place_tables[scope].associated_place_ids(place) {
let is_place_name = self.place_tables[scope]
.place_expr(associated_place)
.is_name();
self.use_def_maps[scope].delete_binding(associated_place, is_place_name);
}
}
/// Push a new [`Definition`] onto the list of definitions
/// associated with the `definition_node` AST node.
///
@@ -457,7 +462,7 @@ impl<'db> SemanticIndexBuilder<'db> {
/// prefer to use `self.add_definition()`, which ensures that this invariant is maintained.
fn push_additional_definition(
&mut self,
symbol: ScopedSymbolId,
place: ScopedPlaceId,
definition_node: impl Into<DefinitionNodeRef<'db>>,
) -> (Definition<'db>, usize) {
let definition_node: DefinitionNodeRef<'_> = definition_node.into();
@@ -471,7 +476,7 @@ impl<'db> SemanticIndexBuilder<'db> {
self.db,
self.file,
self.current_scope(),
symbol,
place,
kind,
is_reexported,
countme::Count::default(),
@@ -484,19 +489,24 @@ impl<'db> SemanticIndexBuilder<'db> {
};
if category.is_binding() {
self.mark_symbol_bound(symbol);
self.mark_place_bound(place);
}
if category.is_declaration() {
self.mark_symbol_declared(symbol);
self.mark_place_declared(place);
}
let is_place_name = self.current_place_table().place_expr(place).is_name();
let use_def = self.current_use_def_map_mut();
match category {
DefinitionCategory::DeclarationAndBinding => {
use_def.record_declaration_and_binding(symbol, definition);
use_def.record_declaration_and_binding(place, definition, is_place_name);
self.delete_associated_bindings(place);
}
DefinitionCategory::Declaration => use_def.record_declaration(place, definition),
DefinitionCategory::Binding => {
use_def.record_binding(place, definition, is_place_name);
self.delete_associated_bindings(place);
}
DefinitionCategory::Declaration => use_def.record_declaration(symbol, definition),
DefinitionCategory::Binding => use_def.record_binding(symbol, definition),
}
let mut try_node_stack_manager = std::mem::take(&mut self.try_node_context_stack_manager);
@@ -506,25 +516,6 @@ impl<'db> SemanticIndexBuilder<'db> {
(definition, num_definitions)
}
fn add_attribute_definition(
&mut self,
symbol: ScopedSymbolId,
definition_kind: DefinitionKind<'db>,
) -> Definition {
let definition = Definition::new(
self.db,
self.file,
self.current_scope(),
symbol,
definition_kind,
false,
countme::Count::default(),
);
self.current_use_def_map_mut()
.record_attribute_binding(symbol, definition);
definition
}
fn record_expression_narrowing_constraint(
&mut self,
precide_node: &ast::Expr,
@@ -684,28 +675,6 @@ impl<'db> SemanticIndexBuilder<'db> {
self.current_assignments.last_mut()
}
/// Records the fact that we saw an attribute assignment of the form
/// `object.attr: <annotation>( = …)` or `object.attr = <value>`.
fn register_attribute_assignment(
&mut self,
object: &ast::Expr,
attr: &'db ast::Identifier,
definition_kind: DefinitionKind<'db>,
) {
if self.is_method_of_class().is_some() {
// We only care about attribute assignments to the first parameter of a method,
// i.e. typically `self` or `cls`.
let accessed_object_refers_to_first_parameter =
object.as_name_expr().map(|name| name.id.as_str())
== self.current_first_parameter_name;
if accessed_object_refers_to_first_parameter {
let symbol = self.add_attribute(attr.id().clone());
self.add_attribute_definition(symbol, definition_kind);
}
}
}
fn predicate_kind(&mut self, pattern: &ast::Pattern) -> PatternPredicateKind<'db> {
match pattern {
ast::Pattern::MatchValue(pattern) => {
@@ -850,8 +819,8 @@ impl<'db> SemanticIndexBuilder<'db> {
// TODO create Definition for PEP 695 typevars
// note that the "bound" on the typevar is a totally different thing than whether
// or not a name is "bound" by a typevar declaration; the latter is always true.
self.mark_symbol_bound(symbol);
self.mark_symbol_declared(symbol);
self.mark_place_bound(symbol);
self.mark_place_declared(symbol);
if let Some(bounds) = bound {
self.visit_expr(bounds);
}
@@ -1022,7 +991,7 @@ impl<'db> SemanticIndexBuilder<'db> {
));
Some(unpackable.as_current_assignment(unpack))
}
ast::Expr::Name(_) | ast::Expr::Attribute(_) => {
ast::Expr::Name(_) | ast::Expr::Attribute(_) | ast::Expr::Subscript(_) => {
Some(unpackable.as_current_assignment(None))
}
_ => None,
@@ -1050,18 +1019,12 @@ impl<'db> SemanticIndexBuilder<'db> {
assert_eq!(&self.current_assignments, &[]);
let mut symbol_tables: IndexVec<_, _> = self
.symbol_tables
let mut place_tables: IndexVec<_, _> = self
.place_tables
.into_iter()
.map(|builder| Arc::new(builder.finish()))
.collect();
let mut instance_attribute_tables: IndexVec<_, _> = self
.instance_attribute_tables
.into_iter()
.map(SymbolTableBuilder::finish)
.collect();
let mut use_def_maps: IndexVec<_, _> = self
.use_def_maps
.into_iter()
@@ -1075,8 +1038,7 @@ impl<'db> SemanticIndexBuilder<'db> {
.collect();
self.scopes.shrink_to_fit();
symbol_tables.shrink_to_fit();
instance_attribute_tables.shrink_to_fit();
place_tables.shrink_to_fit();
use_def_maps.shrink_to_fit();
ast_ids.shrink_to_fit();
self.scopes_by_expression.shrink_to_fit();
@@ -1089,8 +1051,7 @@ impl<'db> SemanticIndexBuilder<'db> {
self.globals_by_scope.shrink_to_fit();
SemanticIndex {
symbol_tables,
instance_attribute_tables,
place_tables,
scopes: self.scopes,
definitions_by_node: self.definitions_by_node,
expressions_by_node: self.expressions_by_node,
@@ -1213,7 +1174,7 @@ where
// used to collect all the overloaded definitions of a function. This needs to be
// done on the `Identifier` node as opposed to `ExprName` because that's what the
// AST uses.
self.mark_symbol_used(symbol);
self.mark_place_used(symbol);
let use_id = self.current_ast_ids().record_use(name);
self.current_use_def_map_mut()
.record_use(symbol, use_id, NodeKey::from_node(name));
@@ -1356,7 +1317,10 @@ where
// For more details, see the doc-comment on `StarImportPlaceholderPredicate`.
for export in exported_names(self.db, referenced_module) {
let symbol_id = self.add_symbol(export.clone());
let node_ref = StarImportDefinitionNodeRef { node, symbol_id };
let node_ref = StarImportDefinitionNodeRef {
node,
place_id: symbol_id,
};
let star_import = StarImportPlaceholderPredicate::new(
self.db,
self.file,
@@ -1365,7 +1329,7 @@ where
);
let pre_definition =
self.current_use_def_map().single_symbol_snapshot(symbol_id);
self.current_use_def_map().single_place_snapshot(symbol_id);
self.push_additional_definition(symbol_id, node_ref);
self.current_use_def_map_mut()
.record_and_negate_star_import_visibility_constraint(
@@ -1920,8 +1884,8 @@ where
ast::Stmt::Global(ast::StmtGlobal { range: _, names }) => {
for name in names {
let symbol_id = self.add_symbol(name.id.clone());
let symbol_table = self.current_symbol_table();
let symbol = symbol_table.symbol(symbol_id);
let symbol_table = self.current_place_table();
let symbol = symbol_table.place_expr(symbol_id);
if symbol.is_bound() || symbol.is_declared() || symbol.is_used() {
self.report_semantic_error(SemanticSyntaxError {
kind: SemanticSyntaxErrorKind::LoadBeforeGlobalDeclaration {
@@ -1942,9 +1906,9 @@ where
}
ast::Stmt::Delete(ast::StmtDelete { targets, range: _ }) => {
for target in targets {
if let ast::Expr::Name(ast::ExprName { id, .. }) = target {
let symbol_id = self.add_symbol(id.clone());
self.current_symbol_table().mark_symbol_used(symbol_id);
if let Ok(target) = PlaceExpr::try_from(target) {
let place_id = self.add_place(target);
self.current_place_table().mark_place_used(place_id);
}
}
walk_stmt(self, stmt);
@@ -1971,109 +1935,133 @@ where
let node_key = NodeKey::from_node(expr);
match expr {
ast::Expr::Name(ast::ExprName { id, ctx, .. }) => {
let (is_use, is_definition) = match (ctx, self.current_assignment()) {
(ast::ExprContext::Store, Some(CurrentAssignment::AugAssign(_))) => {
// For augmented assignment, the target expression is also used.
(true, true)
}
(ast::ExprContext::Load, _) => (true, false),
(ast::ExprContext::Store, _) => (false, true),
(ast::ExprContext::Del, _) => (false, true),
(ast::ExprContext::Invalid, _) => (false, false),
};
let symbol = self.add_symbol(id.clone());
ast::Expr::Name(ast::ExprName { ctx, .. })
| ast::Expr::Attribute(ast::ExprAttribute { ctx, .. })
| ast::Expr::Subscript(ast::ExprSubscript { ctx, .. }) => {
if let Ok(mut place_expr) = PlaceExpr::try_from(expr) {
if self.is_method_of_class().is_some() {
// We specifically mark attribute assignments to the first parameter of a method,
// i.e. typically `self` or `cls`.
let accessed_object_refers_to_first_parameter = self
.current_first_parameter_name
.is_some_and(|fst| place_expr.root_name().as_str() == fst);
if is_use {
self.mark_symbol_used(symbol);
let use_id = self.current_ast_ids().record_use(expr);
if accessed_object_refers_to_first_parameter && place_expr.is_member() {
place_expr.mark_instance_attribute();
}
}
let (is_use, is_definition) = match (ctx, self.current_assignment()) {
(ast::ExprContext::Store, Some(CurrentAssignment::AugAssign(_))) => {
// For augmented assignment, the target expression is also used.
(true, true)
}
(ast::ExprContext::Load, _) => (true, false),
(ast::ExprContext::Store, _) => (false, true),
(ast::ExprContext::Del, _) => (false, true),
(ast::ExprContext::Invalid, _) => (false, false),
};
let place_id = self.add_place(place_expr);
if is_use {
self.mark_place_used(place_id);
let use_id = self.current_ast_ids().record_use(expr);
self.current_use_def_map_mut()
.record_use(place_id, use_id, node_key);
}
if is_definition {
match self.current_assignment() {
Some(CurrentAssignment::Assign { node, unpack }) => {
self.add_definition(
place_id,
AssignmentDefinitionNodeRef {
unpack,
value: &node.value,
target: expr,
},
);
}
Some(CurrentAssignment::AnnAssign(ann_assign)) => {
self.add_standalone_type_expression(&ann_assign.annotation);
self.add_definition(
place_id,
AnnotatedAssignmentDefinitionNodeRef {
node: ann_assign,
annotation: &ann_assign.annotation,
value: ann_assign.value.as_deref(),
target: expr,
},
);
}
Some(CurrentAssignment::AugAssign(aug_assign)) => {
self.add_definition(place_id, aug_assign);
}
Some(CurrentAssignment::For { node, unpack }) => {
self.add_definition(
place_id,
ForStmtDefinitionNodeRef {
unpack,
iterable: &node.iter,
target: expr,
is_async: node.is_async,
},
);
}
Some(CurrentAssignment::Named(named)) => {
// TODO(dhruvmanila): If the current scope is a comprehension, then the
// named expression is implicitly nonlocal. This is yet to be
// implemented.
self.add_definition(place_id, named);
}
Some(CurrentAssignment::Comprehension {
unpack,
node,
first,
}) => {
self.add_definition(
place_id,
ComprehensionDefinitionNodeRef {
unpack,
iterable: &node.iter,
target: expr,
first,
is_async: node.is_async,
},
);
}
Some(CurrentAssignment::WithItem {
item,
is_async,
unpack,
}) => {
self.add_definition(
place_id,
WithItemDefinitionNodeRef {
unpack,
context_expr: &item.context_expr,
target: expr,
is_async,
},
);
}
None => {}
}
}
if let Some(unpack_position) = self
.current_assignment_mut()
.and_then(CurrentAssignment::unpack_position_mut)
{
*unpack_position = UnpackPosition::Other;
}
}
// Track reachability of attribute expressions to silence `unresolved-attribute`
// diagnostics in unreachable code.
if expr.is_attribute_expr() {
self.current_use_def_map_mut()
.record_use(symbol, use_id, node_key);
}
if is_definition {
match self.current_assignment() {
Some(CurrentAssignment::Assign { node, unpack }) => {
self.add_definition(
symbol,
AssignmentDefinitionNodeRef {
unpack,
value: &node.value,
target: expr,
},
);
}
Some(CurrentAssignment::AnnAssign(ann_assign)) => {
self.add_definition(
symbol,
AnnotatedAssignmentDefinitionNodeRef {
node: ann_assign,
annotation: &ann_assign.annotation,
value: ann_assign.value.as_deref(),
target: expr,
},
);
}
Some(CurrentAssignment::AugAssign(aug_assign)) => {
self.add_definition(symbol, aug_assign);
}
Some(CurrentAssignment::For { node, unpack }) => {
self.add_definition(
symbol,
ForStmtDefinitionNodeRef {
unpack,
iterable: &node.iter,
target: expr,
is_async: node.is_async,
},
);
}
Some(CurrentAssignment::Named(named)) => {
// TODO(dhruvmanila): If the current scope is a comprehension, then the
// named expression is implicitly nonlocal. This is yet to be
// implemented.
self.add_definition(symbol, named);
}
Some(CurrentAssignment::Comprehension {
unpack,
node,
first,
}) => {
self.add_definition(
symbol,
ComprehensionDefinitionNodeRef {
unpack,
iterable: &node.iter,
target: expr,
first,
is_async: node.is_async,
},
);
}
Some(CurrentAssignment::WithItem {
item,
is_async,
unpack,
}) => {
self.add_definition(
symbol,
WithItemDefinitionNodeRef {
unpack,
context_expr: &item.context_expr,
target: expr,
is_async,
},
);
}
None => {}
}
}
if let Some(unpack_position) = self
.current_assignment_mut()
.and_then(CurrentAssignment::unpack_position_mut)
{
*unpack_position = UnpackPosition::Other;
.record_node_reachability(node_key);
}
walk_expr(self, expr);
@@ -2239,125 +2227,6 @@ where
self.simplify_visibility_constraints(pre_op);
}
ast::Expr::Attribute(ast::ExprAttribute {
value: object,
attr,
ctx,
range: _,
}) => {
if ctx.is_store() {
match self.current_assignment() {
Some(CurrentAssignment::Assign { node, unpack, .. }) => {
// SAFETY: `value` and `expr` belong to the `self.module` tree
#[expect(unsafe_code)]
let assignment = AssignmentDefinitionKind::new(
TargetKind::from(unpack),
unsafe { AstNodeRef::new(self.module.clone(), &node.value) },
unsafe { AstNodeRef::new(self.module.clone(), expr) },
);
self.register_attribute_assignment(
object,
attr,
DefinitionKind::Assignment(assignment),
);
}
Some(CurrentAssignment::AnnAssign(ann_assign)) => {
self.add_standalone_type_expression(&ann_assign.annotation);
// SAFETY: `annotation`, `value` and `expr` belong to the `self.module` tree
#[expect(unsafe_code)]
let assignment = AnnotatedAssignmentDefinitionKind::new(
unsafe {
AstNodeRef::new(self.module.clone(), &ann_assign.annotation)
},
ann_assign.value.as_deref().map(|value| unsafe {
AstNodeRef::new(self.module.clone(), value)
}),
unsafe { AstNodeRef::new(self.module.clone(), expr) },
);
self.register_attribute_assignment(
object,
attr,
DefinitionKind::AnnotatedAssignment(assignment),
);
}
Some(CurrentAssignment::For { node, unpack, .. }) => {
// // SAFETY: `iter` and `expr` belong to the `self.module` tree
#[expect(unsafe_code)]
let assignment = ForStmtDefinitionKind::new(
TargetKind::from(unpack),
unsafe { AstNodeRef::new(self.module.clone(), &node.iter) },
unsafe { AstNodeRef::new(self.module.clone(), expr) },
node.is_async,
);
self.register_attribute_assignment(
object,
attr,
DefinitionKind::For(assignment),
);
}
Some(CurrentAssignment::WithItem {
item,
unpack,
is_async,
..
}) => {
// SAFETY: `context_expr` and `expr` belong to the `self.module` tree
#[expect(unsafe_code)]
let assignment = WithItemDefinitionKind::new(
TargetKind::from(unpack),
unsafe { AstNodeRef::new(self.module.clone(), &item.context_expr) },
unsafe { AstNodeRef::new(self.module.clone(), expr) },
is_async,
);
self.register_attribute_assignment(
object,
attr,
DefinitionKind::WithItem(assignment),
);
}
Some(CurrentAssignment::Comprehension {
unpack,
node,
first,
}) => {
// SAFETY: `iter` and `expr` belong to the `self.module` tree
#[expect(unsafe_code)]
let assignment = ComprehensionDefinitionKind {
target_kind: TargetKind::from(unpack),
iterable: unsafe {
AstNodeRef::new(self.module.clone(), &node.iter)
},
target: unsafe { AstNodeRef::new(self.module.clone(), expr) },
first,
is_async: node.is_async,
};
// Temporarily move to the scope of the method to which the instance attribute is defined.
// SAFETY: `self.scope_stack` is not empty because the targets in comprehensions should always introduce a new scope.
let scope = self.scope_stack.pop().expect("The popped scope must be a comprehension, which must have a parent scope");
self.register_attribute_assignment(
object,
attr,
DefinitionKind::Comprehension(assignment),
);
self.scope_stack.push(scope);
}
Some(CurrentAssignment::AugAssign(_)) => {
// TODO:
}
Some(CurrentAssignment::Named(_)) => {
// A named expression whose target is an attribute is syntactically prohibited
}
None => {}
}
}
// Track reachability of attribute expressions to silence `unresolved-attribute`
// diagnostics in unreachable code.
self.current_use_def_map_mut()
.record_node_reachability(node_key);
walk_expr(self, expr);
}
ast::Expr::StringLiteral(_) => {
// Track reachability of string literals, as they could be a stringified annotation
// with child expressions whose reachability we are interested in.

View File

@@ -8,16 +8,16 @@ use ruff_text_size::{Ranged, TextRange};
use crate::Db;
use crate::ast_node_ref::AstNodeRef;
use crate::node_key::NodeKey;
use crate::semantic_index::symbol::{FileScopeId, ScopeId, ScopedSymbolId};
use crate::semantic_index::place::{FileScopeId, ScopeId, ScopedPlaceId};
use crate::unpack::{Unpack, UnpackPosition};
/// A definition of a symbol.
/// A definition of a place.
///
/// ## ID stability
/// The `Definition`'s ID is stable when the only field that change is its `kind` (AST node).
///
/// The `Definition` changes when the `file`, `scope`, or `symbol` change. This can be
/// because a new scope gets inserted before the `Definition` or a new symbol is inserted
/// The `Definition` changes when the `file`, `scope`, or `place` change. This can be
/// because a new scope gets inserted before the `Definition` or a new place is inserted
/// before this `Definition`. However, the ID can be considered stable and it is okay to use
/// `Definition` in cross-module` salsa queries or as a field on other salsa tracked structs.
#[salsa::tracked(debug)]
@@ -28,8 +28,8 @@ pub struct Definition<'db> {
/// The scope in which the definition occurs.
pub(crate) file_scope: FileScopeId,
/// The symbol defined.
pub(crate) symbol: ScopedSymbolId,
/// The place ID of the definition.
pub(crate) place: ScopedPlaceId,
/// WARNING: Only access this field when doing type inference for the same
/// file as where `Definition` is defined to avoid cross-file query dependencies.
@@ -89,6 +89,39 @@ impl<'a, 'db> IntoIterator for &'a Definitions<'db> {
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, salsa::Update)]
pub(crate) enum DefinitionState<'db> {
Defined(Definition<'db>),
/// Represents the implicit "unbound"/"undeclared" definition of every place.
Undefined,
/// Represents a definition that has been deleted.
/// This used when an attribute/subscript definition (such as `x.y = ...`, `x[0] = ...`) becomes obsolete due to a reassignment of the root place.
Deleted,
}
impl<'db> DefinitionState<'db> {
pub(crate) fn is_defined_and(self, f: impl Fn(Definition<'db>) -> bool) -> bool {
matches!(self, DefinitionState::Defined(def) if f(def))
}
pub(crate) fn is_undefined_or(self, f: impl Fn(Definition<'db>) -> bool) -> bool {
matches!(self, DefinitionState::Undefined)
|| matches!(self, DefinitionState::Defined(def) if f(def))
}
pub(crate) fn is_undefined(self) -> bool {
matches!(self, DefinitionState::Undefined)
}
#[allow(unused)]
pub(crate) fn definition(self) -> Option<Definition<'db>> {
match self {
DefinitionState::Defined(def) => Some(def),
DefinitionState::Deleted | DefinitionState::Undefined => None,
}
}
}
#[derive(Copy, Clone, Debug)]
pub(crate) enum DefinitionNodeRef<'a> {
Import(ImportDefinitionNodeRef<'a>),
@@ -232,7 +265,7 @@ pub(crate) struct ImportDefinitionNodeRef<'a> {
#[derive(Copy, Clone, Debug)]
pub(crate) struct StarImportDefinitionNodeRef<'a> {
pub(crate) node: &'a ast::StmtImportFrom,
pub(crate) symbol_id: ScopedSymbolId,
pub(crate) place_id: ScopedPlaceId,
}
#[derive(Copy, Clone, Debug)]
@@ -323,10 +356,10 @@ impl<'db> DefinitionNodeRef<'db> {
is_reexported,
}),
DefinitionNodeRef::ImportStar(star_import) => {
let StarImportDefinitionNodeRef { node, symbol_id } = star_import;
let StarImportDefinitionNodeRef { node, place_id } = star_import;
DefinitionKind::StarImport(StarImportDefinitionKind {
node: unsafe { AstNodeRef::new(parsed, node) },
symbol_id,
place_id,
})
}
DefinitionNodeRef::Function(function) => {
@@ -456,7 +489,7 @@ impl<'db> DefinitionNodeRef<'db> {
// INVARIANT: for an invalid-syntax statement such as `from foo import *, bar, *`,
// we only create a `StarImportDefinitionKind` for the *first* `*` alias in the names list.
Self::ImportStar(StarImportDefinitionNodeRef { node, symbol_id: _ }) => node
Self::ImportStar(StarImportDefinitionNodeRef { node, place_id: _ }) => node
.names
.iter()
.find(|alias| &alias.name == "*")
@@ -517,7 +550,7 @@ pub(crate) enum DefinitionCategory {
}
impl DefinitionCategory {
/// True if this definition establishes a "declared type" for the symbol.
/// True if this definition establishes a "declared type" for the place.
///
/// If so, any assignments reached by this definition are in error if they assign a value of a
/// type not assignable to the declared type.
@@ -530,7 +563,7 @@ impl DefinitionCategory {
)
}
/// True if this definition assigns a value to the symbol.
/// True if this definition assigns a value to the place.
///
/// False only for annotated assignments without a RHS.
pub(crate) fn is_binding(self) -> bool {
@@ -591,8 +624,8 @@ impl DefinitionKind<'_> {
/// Returns the [`TextRange`] of the definition target.
///
/// A definition target would mainly be the node representing the symbol being defined i.e.,
/// [`ast::ExprName`] or [`ast::Identifier`] but could also be other nodes.
/// A definition target would mainly be the node representing the place being defined i.e.,
/// [`ast::ExprName`], [`ast::Identifier`], [`ast::ExprAttribute`] or [`ast::ExprSubscript`] but could also be other nodes.
pub(crate) fn target_range(&self) -> TextRange {
match self {
DefinitionKind::Import(import) => import.alias().range(),
@@ -700,14 +733,15 @@ impl DefinitionKind<'_> {
#[derive(Copy, Clone, Debug, PartialEq, Hash)]
pub(crate) enum TargetKind<'db> {
Sequence(UnpackPosition, Unpack<'db>),
NameOrAttribute,
/// Name, attribute, or subscript.
Single,
}
impl<'db> From<Option<(UnpackPosition, Unpack<'db>)>> for TargetKind<'db> {
fn from(value: Option<(UnpackPosition, Unpack<'db>)>) -> Self {
match value {
Some((unpack_position, unpack)) => TargetKind::Sequence(unpack_position, unpack),
None => TargetKind::NameOrAttribute,
None => TargetKind::Single,
}
}
}
@@ -715,7 +749,7 @@ impl<'db> From<Option<(UnpackPosition, Unpack<'db>)>> for TargetKind<'db> {
#[derive(Clone, Debug)]
pub struct StarImportDefinitionKind {
node: AstNodeRef<ast::StmtImportFrom>,
symbol_id: ScopedSymbolId,
place_id: ScopedPlaceId,
}
impl StarImportDefinitionKind {
@@ -737,8 +771,8 @@ impl StarImportDefinitionKind {
)
}
pub(crate) fn symbol_id(&self) -> ScopedSymbolId {
self.symbol_id
pub(crate) fn place_id(&self) -> ScopedPlaceId {
self.place_id
}
}
@@ -759,13 +793,18 @@ impl MatchPatternDefinitionKind {
}
}
/// Note that the elements of a comprehension can be in different scopes.
/// If the definition target of a comprehension is a name, it is in the comprehension's scope.
/// But if the target is an attribute or subscript, its definition is not in the comprehension's scope;
/// it is in the scope in which the root variable is bound.
/// TODO: currently we don't model this correctly and simply assume that it is in a scope outside the comprehension.
#[derive(Clone, Debug)]
pub struct ComprehensionDefinitionKind<'db> {
pub(super) target_kind: TargetKind<'db>,
pub(super) iterable: AstNodeRef<ast::Expr>,
pub(super) target: AstNodeRef<ast::Expr>,
pub(super) first: bool,
pub(super) is_async: bool,
target_kind: TargetKind<'db>,
iterable: AstNodeRef<ast::Expr>,
target: AstNodeRef<ast::Expr>,
first: bool,
is_async: bool,
}
impl<'db> ComprehensionDefinitionKind<'db> {
@@ -840,18 +879,6 @@ pub struct AssignmentDefinitionKind<'db> {
}
impl<'db> AssignmentDefinitionKind<'db> {
pub(crate) fn new(
target_kind: TargetKind<'db>,
value: AstNodeRef<ast::Expr>,
target: AstNodeRef<ast::Expr>,
) -> Self {
Self {
target_kind,
value,
target,
}
}
pub(crate) fn target_kind(&self) -> TargetKind<'db> {
self.target_kind
}
@@ -873,18 +900,6 @@ pub struct AnnotatedAssignmentDefinitionKind {
}
impl AnnotatedAssignmentDefinitionKind {
pub(crate) fn new(
annotation: AstNodeRef<ast::Expr>,
value: Option<AstNodeRef<ast::Expr>>,
target: AstNodeRef<ast::Expr>,
) -> Self {
Self {
annotation,
value,
target,
}
}
pub(crate) fn value(&self) -> Option<&ast::Expr> {
self.value.as_deref()
}
@@ -907,20 +922,6 @@ pub struct WithItemDefinitionKind<'db> {
}
impl<'db> WithItemDefinitionKind<'db> {
pub(crate) fn new(
target_kind: TargetKind<'db>,
context_expr: AstNodeRef<ast::Expr>,
target: AstNodeRef<ast::Expr>,
is_async: bool,
) -> Self {
Self {
target_kind,
context_expr,
target,
is_async,
}
}
pub(crate) fn context_expr(&self) -> &ast::Expr {
self.context_expr.node()
}
@@ -947,20 +948,6 @@ pub struct ForStmtDefinitionKind<'db> {
}
impl<'db> ForStmtDefinitionKind<'db> {
pub(crate) fn new(
target_kind: TargetKind<'db>,
iterable: AstNodeRef<ast::Expr>,
target: AstNodeRef<ast::Expr>,
is_async: bool,
) -> Self {
Self {
target_kind,
iterable,
target,
is_async,
}
}
pub(crate) fn iterable(&self) -> &ast::Expr {
self.iterable.node()
}
@@ -1031,6 +1018,18 @@ impl From<&ast::ExprName> for DefinitionNodeKey {
}
}
impl From<&ast::ExprAttribute> for DefinitionNodeKey {
fn from(node: &ast::ExprAttribute) -> Self {
Self(NodeKey::from_node(node))
}
}
impl From<&ast::ExprSubscript> for DefinitionNodeKey {
fn from(node: &ast::ExprSubscript) -> Self {
Self(NodeKey::from_node(node))
}
}
impl From<&ast::ExprNamed> for DefinitionNodeKey {
fn from(node: &ast::ExprNamed) -> Self {
Self(NodeKey::from_node(node))

View File

@@ -1,6 +1,6 @@
use crate::ast_node_ref::AstNodeRef;
use crate::db::Db;
use crate::semantic_index::symbol::{FileScopeId, ScopeId};
use crate::semantic_index::place::{FileScopeId, ScopeId};
use ruff_db::files::File;
use ruff_python_ast as ast;
use salsa;

View File

@@ -1,7 +1,7 @@
//! # Narrowing constraints
//!
//! When building a semantic index for a file, we associate each binding with a _narrowing
//! constraint_, which constrains the type of the binding's symbol. Note that a binding can be
//! constraint_, which constrains the type of the binding's place. Note that a binding can be
//! associated with a different narrowing constraint at different points in a file. See the
//! [`use_def`][crate::semantic_index::use_def] module for more details.
//!
@@ -34,7 +34,7 @@ use crate::semantic_index::predicate::ScopedPredicateId;
/// A narrowing constraint associated with a live binding.
///
/// A constraint is a list of [`Predicate`]s that each constrain the type of the binding's symbol.
/// A constraint is a list of [`Predicate`]s that each constrain the type of the binding's place.
///
/// [`Predicate`]: crate::semantic_index::predicate::Predicate
pub(crate) type ScopedNarrowingConstraint = List<ScopedNarrowingConstraintPredicate>;
@@ -46,7 +46,7 @@ pub(crate) enum ConstraintKey {
}
/// One of the [`Predicate`]s in a narrowing constraint, which constraints the type of the
/// binding's symbol.
/// binding's place.
///
/// Note that those [`Predicate`]s are stored in [their own per-scope
/// arena][crate::semantic_index::predicate::Predicates], so internally we use a

View File

@@ -0,0 +1,942 @@
use std::convert::Infallible;
use std::hash::{Hash, Hasher};
use std::ops::Range;
use bitflags::bitflags;
use hashbrown::hash_map::RawEntryMut;
use ruff_db::files::File;
use ruff_db::parsed::ParsedModule;
use ruff_index::{IndexVec, newtype_index};
use ruff_python_ast as ast;
use ruff_python_ast::name::Name;
use rustc_hash::FxHasher;
use smallvec::{SmallVec, smallvec};
use crate::Db;
use crate::ast_node_ref::AstNodeRef;
use crate::node_key::NodeKey;
use crate::semantic_index::visibility_constraints::ScopedVisibilityConstraintId;
use crate::semantic_index::{PlaceSet, SemanticIndex, semantic_index};
#[derive(Debug, Clone, PartialEq, Eq, Hash, salsa::Update)]
pub(crate) enum PlaceExprSubSegment {
/// A member access, e.g. `.y` in `x.y`
Member(ast::name::Name),
/// An integer-based index access, e.g. `[1]` in `x[1]`
IntSubscript(ast::Int),
/// A string-based index access, e.g. `["foo"]` in `x["foo"]`
StringSubscript(String),
}
impl PlaceExprSubSegment {
pub(crate) fn as_member(&self) -> Option<&ast::name::Name> {
match self {
PlaceExprSubSegment::Member(name) => Some(name),
_ => None,
}
}
}
/// An expression that can be the target of a `Definition`.
/// If you want to perform a comparison based on the equality of segments (without including
/// flags), use [`PlaceSegments`].
#[derive(Eq, PartialEq, Debug)]
pub struct PlaceExpr {
root_name: Name,
sub_segments: SmallVec<[PlaceExprSubSegment; 1]>,
flags: PlaceFlags,
}
impl std::fmt::Display for PlaceExpr {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "{}", self.root_name)?;
for segment in &self.sub_segments {
match segment {
PlaceExprSubSegment::Member(name) => write!(f, ".{name}")?,
PlaceExprSubSegment::IntSubscript(int) => write!(f, "[{int}]")?,
PlaceExprSubSegment::StringSubscript(string) => write!(f, "[\"{string}\"]")?,
}
}
Ok(())
}
}
impl TryFrom<&ast::name::Name> for PlaceExpr {
type Error = Infallible;
fn try_from(name: &ast::name::Name) -> Result<Self, Infallible> {
Ok(PlaceExpr::name(name.clone()))
}
}
impl TryFrom<ast::name::Name> for PlaceExpr {
type Error = Infallible;
fn try_from(name: ast::name::Name) -> Result<Self, Infallible> {
Ok(PlaceExpr::name(name))
}
}
impl TryFrom<&ast::ExprAttribute> for PlaceExpr {
type Error = ();
fn try_from(attr: &ast::ExprAttribute) -> Result<Self, ()> {
let mut place = PlaceExpr::try_from(&*attr.value)?;
place
.sub_segments
.push(PlaceExprSubSegment::Member(attr.attr.id.clone()));
Ok(place)
}
}
impl TryFrom<ast::ExprAttribute> for PlaceExpr {
type Error = ();
fn try_from(attr: ast::ExprAttribute) -> Result<Self, ()> {
let mut place = PlaceExpr::try_from(&*attr.value)?;
place
.sub_segments
.push(PlaceExprSubSegment::Member(attr.attr.id));
Ok(place)
}
}
impl TryFrom<&ast::ExprSubscript> for PlaceExpr {
type Error = ();
fn try_from(subscript: &ast::ExprSubscript) -> Result<Self, ()> {
let mut place = PlaceExpr::try_from(&*subscript.value)?;
match &*subscript.slice {
ast::Expr::NumberLiteral(ast::ExprNumberLiteral {
value: ast::Number::Int(index),
..
}) => {
place
.sub_segments
.push(PlaceExprSubSegment::IntSubscript(index.clone()));
}
ast::Expr::StringLiteral(string) => {
place
.sub_segments
.push(PlaceExprSubSegment::StringSubscript(
string.value.to_string(),
));
}
_ => {
return Err(());
}
}
Ok(place)
}
}
impl TryFrom<ast::ExprSubscript> for PlaceExpr {
type Error = ();
fn try_from(subscript: ast::ExprSubscript) -> Result<Self, ()> {
PlaceExpr::try_from(&subscript)
}
}
impl TryFrom<&ast::Expr> for PlaceExpr {
type Error = ();
fn try_from(expr: &ast::Expr) -> Result<Self, ()> {
match expr {
ast::Expr::Name(name) => Ok(PlaceExpr::name(name.id.clone())),
ast::Expr::Attribute(attr) => PlaceExpr::try_from(attr),
ast::Expr::Subscript(subscript) => PlaceExpr::try_from(subscript),
_ => Err(()),
}
}
}
impl PlaceExpr {
pub(super) fn name(name: Name) -> Self {
Self {
root_name: name,
sub_segments: smallvec![],
flags: PlaceFlags::empty(),
}
}
fn insert_flags(&mut self, flags: PlaceFlags) {
self.flags.insert(flags);
}
pub(super) fn mark_instance_attribute(&mut self) {
self.flags.insert(PlaceFlags::IS_INSTANCE_ATTRIBUTE);
}
pub(crate) fn root_name(&self) -> &Name {
&self.root_name
}
pub(crate) fn sub_segments(&self) -> &[PlaceExprSubSegment] {
&self.sub_segments
}
pub(crate) fn as_name(&self) -> Option<&Name> {
if self.is_name() {
Some(&self.root_name)
} else {
None
}
}
/// Assumes that the place expression is a name.
#[track_caller]
pub(crate) fn expect_name(&self) -> &Name {
debug_assert_eq!(self.sub_segments.len(), 0);
&self.root_name
}
/// Does the place expression have the form `self.{name}` (`self` is the first parameter of the method)?
pub(super) fn is_instance_attribute_named(&self, name: &str) -> bool {
self.is_instance_attribute()
&& self.sub_segments.len() == 1
&& self.sub_segments[0].as_member().unwrap().as_str() == name
}
/// Is the place an instance attribute?
pub fn is_instance_attribute(&self) -> bool {
self.flags.contains(PlaceFlags::IS_INSTANCE_ATTRIBUTE)
}
/// Is the place used in its containing scope?
pub fn is_used(&self) -> bool {
self.flags.contains(PlaceFlags::IS_USED)
}
/// Is the place defined in its containing scope?
pub fn is_bound(&self) -> bool {
self.flags.contains(PlaceFlags::IS_BOUND)
}
/// Is the place declared in its containing scope?
pub fn is_declared(&self) -> bool {
self.flags.contains(PlaceFlags::IS_DECLARED)
}
/// Is the place just a name?
pub fn is_name(&self) -> bool {
self.sub_segments.is_empty()
}
pub fn is_name_and(&self, f: impl FnOnce(&str) -> bool) -> bool {
self.is_name() && f(&self.root_name)
}
/// Does the place expression have the form `<object>.member`?
pub fn is_member(&self) -> bool {
self.sub_segments
.last()
.is_some_and(|last| last.as_member().is_some())
}
pub(crate) fn segments(&self) -> PlaceSegments {
PlaceSegments {
root_name: Some(&self.root_name),
sub_segments: &self.sub_segments,
}
}
// TODO: Ideally this would iterate PlaceSegments instead of RootExprs, both to reduce
// allocation and to avoid having both flagged and non-flagged versions of PlaceExprs.
fn root_exprs(&self) -> RootExprs<'_> {
RootExprs {
expr: self,
len: self.sub_segments.len(),
}
}
}
struct RootExprs<'e> {
expr: &'e PlaceExpr,
len: usize,
}
impl Iterator for RootExprs<'_> {
type Item = PlaceExpr;
fn next(&mut self) -> Option<Self::Item> {
if self.len == 0 {
return None;
}
self.len -= 1;
Some(PlaceExpr {
root_name: self.expr.root_name.clone(),
sub_segments: self.expr.sub_segments[..self.len].iter().cloned().collect(),
flags: PlaceFlags::empty(),
})
}
}
bitflags! {
/// Flags that can be queried to obtain information about a place in a given scope.
///
/// See the doc-comment at the top of [`super::use_def`] for explanations of what it
/// means for a place to be *bound* as opposed to *declared*.
#[derive(Copy, Clone, Debug, Eq, PartialEq)]
struct PlaceFlags: u8 {
const IS_USED = 1 << 0;
const IS_BOUND = 1 << 1;
const IS_DECLARED = 1 << 2;
/// TODO: This flag is not yet set by anything
const MARKED_GLOBAL = 1 << 3;
/// TODO: This flag is not yet set by anything
const MARKED_NONLOCAL = 1 << 4;
const IS_INSTANCE_ATTRIBUTE = 1 << 5;
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum PlaceSegment<'a> {
/// A first segment of a place expression (root name), e.g. `x` in `x.y.z[0]`.
Name(&'a ast::name::Name),
Member(&'a ast::name::Name),
IntSubscript(&'a ast::Int),
StringSubscript(&'a str),
}
#[derive(Debug, PartialEq, Eq)]
pub struct PlaceSegments<'a> {
root_name: Option<&'a ast::name::Name>,
sub_segments: &'a [PlaceExprSubSegment],
}
impl<'a> Iterator for PlaceSegments<'a> {
type Item = PlaceSegment<'a>;
fn next(&mut self) -> Option<Self::Item> {
if let Some(name) = self.root_name.take() {
return Some(PlaceSegment::Name(name));
}
if self.sub_segments.is_empty() {
return None;
}
let segment = &self.sub_segments[0];
self.sub_segments = &self.sub_segments[1..];
Some(match segment {
PlaceExprSubSegment::Member(name) => PlaceSegment::Member(name),
PlaceExprSubSegment::IntSubscript(int) => PlaceSegment::IntSubscript(int),
PlaceExprSubSegment::StringSubscript(string) => PlaceSegment::StringSubscript(string),
})
}
}
/// ID that uniquely identifies a place in a file.
#[derive(Copy, Clone, Debug, Eq, PartialEq, Hash)]
pub struct FilePlaceId {
scope: FileScopeId,
scoped_place_id: ScopedPlaceId,
}
impl FilePlaceId {
pub fn scope(self) -> FileScopeId {
self.scope
}
pub(crate) fn scoped_place_id(self) -> ScopedPlaceId {
self.scoped_place_id
}
}
impl From<FilePlaceId> for ScopedPlaceId {
fn from(val: FilePlaceId) -> Self {
val.scoped_place_id()
}
}
/// ID that uniquely identifies a place inside a [`Scope`].
#[newtype_index]
#[derive(salsa::Update)]
pub struct ScopedPlaceId;
/// A cross-module identifier of a scope that can be used as a salsa query parameter.
#[salsa::tracked(debug)]
pub struct ScopeId<'db> {
pub file: File,
pub file_scope_id: FileScopeId,
count: countme::Count<ScopeId<'static>>,
}
impl<'db> ScopeId<'db> {
pub(crate) fn is_function_like(self, db: &'db dyn Db) -> bool {
self.node(db).scope_kind().is_function_like()
}
pub(crate) fn is_type_parameter(self, db: &'db dyn Db) -> bool {
self.node(db).scope_kind().is_type_parameter()
}
pub(crate) fn node(self, db: &dyn Db) -> &NodeWithScopeKind {
self.scope(db).node()
}
pub(crate) fn scope(self, db: &dyn Db) -> &Scope {
semantic_index(db, self.file(db)).scope(self.file_scope_id(db))
}
#[cfg(test)]
pub(crate) fn name(self, db: &'db dyn Db) -> &'db str {
match self.node(db) {
NodeWithScopeKind::Module => "<module>",
NodeWithScopeKind::Class(class) | NodeWithScopeKind::ClassTypeParameters(class) => {
class.name.as_str()
}
NodeWithScopeKind::Function(function)
| NodeWithScopeKind::FunctionTypeParameters(function) => function.name.as_str(),
NodeWithScopeKind::TypeAlias(type_alias)
| NodeWithScopeKind::TypeAliasTypeParameters(type_alias) => type_alias
.name
.as_name_expr()
.map(|name| name.id.as_str())
.unwrap_or("<type alias>"),
NodeWithScopeKind::Lambda(_) => "<lambda>",
NodeWithScopeKind::ListComprehension(_) => "<listcomp>",
NodeWithScopeKind::SetComprehension(_) => "<setcomp>",
NodeWithScopeKind::DictComprehension(_) => "<dictcomp>",
NodeWithScopeKind::GeneratorExpression(_) => "<generator>",
}
}
}
/// ID that uniquely identifies a scope inside of a module.
#[newtype_index]
#[derive(salsa::Update)]
pub struct FileScopeId;
impl FileScopeId {
/// Returns the scope id of the module-global scope.
pub fn global() -> Self {
FileScopeId::from_u32(0)
}
pub fn is_global(self) -> bool {
self == FileScopeId::global()
}
pub fn to_scope_id(self, db: &dyn Db, file: File) -> ScopeId<'_> {
let index = semantic_index(db, file);
index.scope_ids_by_scope[self]
}
pub(crate) fn is_generator_function(self, index: &SemanticIndex) -> bool {
index.generator_functions.contains(&self)
}
}
#[derive(Debug, salsa::Update)]
pub struct Scope {
parent: Option<FileScopeId>,
node: NodeWithScopeKind,
descendants: Range<FileScopeId>,
reachability: ScopedVisibilityConstraintId,
}
impl Scope {
pub(super) fn new(
parent: Option<FileScopeId>,
node: NodeWithScopeKind,
descendants: Range<FileScopeId>,
reachability: ScopedVisibilityConstraintId,
) -> Self {
Scope {
parent,
node,
descendants,
reachability,
}
}
pub fn parent(&self) -> Option<FileScopeId> {
self.parent
}
pub fn node(&self) -> &NodeWithScopeKind {
&self.node
}
pub fn kind(&self) -> ScopeKind {
self.node().scope_kind()
}
pub fn descendants(&self) -> Range<FileScopeId> {
self.descendants.clone()
}
pub(super) fn extend_descendants(&mut self, children_end: FileScopeId) {
self.descendants = self.descendants.start..children_end;
}
pub(crate) fn is_eager(&self) -> bool {
self.kind().is_eager()
}
pub(crate) fn reachability(&self) -> ScopedVisibilityConstraintId {
self.reachability
}
}
#[derive(Copy, Clone, Debug, PartialEq, Eq)]
pub enum ScopeKind {
Module,
Annotation,
Class,
Function,
Lambda,
Comprehension,
TypeAlias,
}
impl ScopeKind {
pub(crate) fn is_eager(self) -> bool {
match self {
ScopeKind::Module | ScopeKind::Class | ScopeKind::Comprehension => true,
ScopeKind::Annotation
| ScopeKind::Function
| ScopeKind::Lambda
| ScopeKind::TypeAlias => false,
}
}
pub(crate) fn is_function_like(self) -> bool {
// Type parameter scopes behave like function scopes in terms of name resolution; CPython
// place table also uses the term "function-like" for these scopes.
matches!(
self,
ScopeKind::Annotation
| ScopeKind::Function
| ScopeKind::Lambda
| ScopeKind::TypeAlias
| ScopeKind::Comprehension
)
}
pub(crate) fn is_class(self) -> bool {
matches!(self, ScopeKind::Class)
}
pub(crate) fn is_type_parameter(self) -> bool {
matches!(self, ScopeKind::Annotation | ScopeKind::TypeAlias)
}
}
/// [`PlaceExpr`] table for a specific [`Scope`].
#[derive(Default, salsa::Update)]
pub struct PlaceTable {
/// The place expressions in this scope.
places: IndexVec<ScopedPlaceId, PlaceExpr>,
/// The set of places.
place_set: PlaceSet,
}
impl PlaceTable {
fn shrink_to_fit(&mut self) {
self.places.shrink_to_fit();
}
pub(crate) fn place_expr(&self, place_id: impl Into<ScopedPlaceId>) -> &PlaceExpr {
&self.places[place_id.into()]
}
/// Iterate over the "root" expressions of the place (e.g. `x.y.z`, `x.y`, `x` for `x.y.z[0]`).
pub(crate) fn root_place_exprs(
&self,
place_expr: &PlaceExpr,
) -> impl Iterator<Item = &PlaceExpr> {
place_expr
.root_exprs()
.filter_map(|place_expr| self.place_by_expr(&place_expr))
}
#[expect(unused)]
pub(crate) fn place_ids(&self) -> impl Iterator<Item = ScopedPlaceId> {
self.places.indices()
}
pub fn places(&self) -> impl Iterator<Item = &PlaceExpr> {
self.places.iter()
}
pub fn symbols(&self) -> impl Iterator<Item = &PlaceExpr> {
self.places().filter(|place_expr| place_expr.is_name())
}
pub fn instance_attributes(&self) -> impl Iterator<Item = &PlaceExpr> {
self.places()
.filter(|place_expr| place_expr.is_instance_attribute())
}
/// Returns the place named `name`.
#[allow(unused)] // used in tests
pub(crate) fn place_by_name(&self, name: &str) -> Option<&PlaceExpr> {
let id = self.place_id_by_name(name)?;
Some(self.place_expr(id))
}
/// Returns the flagged place by the unflagged place expression.
///
/// TODO: Ideally this would take a [`PlaceSegments`] instead of [`PlaceExpr`], to avoid the
/// awkward distinction between "flagged" (canonical) and unflagged [`PlaceExpr`]; in that
/// world, we would only create [`PlaceExpr`] in semantic indexing; in type inference we'd
/// create [`PlaceSegments`] if we need to look up a [`PlaceExpr`]. The [`PlaceTable`] would
/// need to gain the ability to hash and look up by a [`PlaceSegments`].
pub(crate) fn place_by_expr(&self, place_expr: &PlaceExpr) -> Option<&PlaceExpr> {
let id = self.place_id_by_expr(place_expr)?;
Some(self.place_expr(id))
}
/// Returns the [`ScopedPlaceId`] of the place named `name`.
pub(crate) fn place_id_by_name(&self, name: &str) -> Option<ScopedPlaceId> {
let (id, ()) = self
.place_set
.raw_entry()
.from_hash(Self::hash_name(name), |id| {
self.place_expr(*id).as_name().map(Name::as_str) == Some(name)
})?;
Some(*id)
}
/// Returns the [`ScopedPlaceId`] of the place expression.
pub(crate) fn place_id_by_expr(&self, place_expr: &PlaceExpr) -> Option<ScopedPlaceId> {
let (id, ()) = self
.place_set
.raw_entry()
.from_hash(Self::hash_place_expr(place_expr), |id| {
self.place_expr(*id).segments() == place_expr.segments()
})?;
Some(*id)
}
pub(crate) fn place_id_by_instance_attribute_name(&self, name: &str) -> Option<ScopedPlaceId> {
self.places
.indices()
.find(|id| self.places[*id].is_instance_attribute_named(name))
}
fn hash_name(name: &str) -> u64 {
let mut hasher = FxHasher::default();
name.hash(&mut hasher);
hasher.finish()
}
fn hash_place_expr(place_expr: &PlaceExpr) -> u64 {
let mut hasher = FxHasher::default();
place_expr.root_name().as_str().hash(&mut hasher);
for segment in &place_expr.sub_segments {
match segment {
PlaceExprSubSegment::Member(name) => name.hash(&mut hasher),
PlaceExprSubSegment::IntSubscript(int) => int.hash(&mut hasher),
PlaceExprSubSegment::StringSubscript(string) => string.hash(&mut hasher),
}
}
hasher.finish()
}
}
impl PartialEq for PlaceTable {
fn eq(&self, other: &Self) -> bool {
// We don't need to compare the place_set because the place is already captured in `PlaceExpr`.
self.places == other.places
}
}
impl Eq for PlaceTable {}
impl std::fmt::Debug for PlaceTable {
/// Exclude the `place_set` field from the debug output.
/// It's very noisy and not useful for debugging.
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_tuple("PlaceTable")
.field(&self.places)
.finish_non_exhaustive()
}
}
#[derive(Debug, Default)]
pub(super) struct PlaceTableBuilder {
table: PlaceTable,
associated_place_ids: IndexVec<ScopedPlaceId, Vec<ScopedPlaceId>>,
}
impl PlaceTableBuilder {
pub(super) fn add_symbol(&mut self, name: Name) -> (ScopedPlaceId, bool) {
let hash = PlaceTable::hash_name(&name);
let entry = self
.table
.place_set
.raw_entry_mut()
.from_hash(hash, |id| self.table.places[*id].as_name() == Some(&name));
match entry {
RawEntryMut::Occupied(entry) => (*entry.key(), false),
RawEntryMut::Vacant(entry) => {
let symbol = PlaceExpr::name(name);
let id = self.table.places.push(symbol);
entry.insert_with_hasher(hash, id, (), |id| {
PlaceTable::hash_place_expr(&self.table.places[*id])
});
let new_id = self.associated_place_ids.push(vec![]);
debug_assert_eq!(new_id, id);
(id, true)
}
}
}
pub(super) fn add_place(&mut self, place_expr: PlaceExpr) -> (ScopedPlaceId, bool) {
let hash = PlaceTable::hash_place_expr(&place_expr);
let entry = self.table.place_set.raw_entry_mut().from_hash(hash, |id| {
self.table.places[*id].segments() == place_expr.segments()
});
match entry {
RawEntryMut::Occupied(entry) => (*entry.key(), false),
RawEntryMut::Vacant(entry) => {
let id = self.table.places.push(place_expr);
entry.insert_with_hasher(hash, id, (), |id| {
PlaceTable::hash_place_expr(&self.table.places[*id])
});
let new_id = self.associated_place_ids.push(vec![]);
debug_assert_eq!(new_id, id);
for root in self.table.places[id].root_exprs() {
if let Some(root_id) = self.table.place_id_by_expr(&root) {
self.associated_place_ids[root_id].push(id);
}
}
(id, true)
}
}
}
pub(super) fn mark_place_bound(&mut self, id: ScopedPlaceId) {
self.table.places[id].insert_flags(PlaceFlags::IS_BOUND);
}
pub(super) fn mark_place_declared(&mut self, id: ScopedPlaceId) {
self.table.places[id].insert_flags(PlaceFlags::IS_DECLARED);
}
pub(super) fn mark_place_used(&mut self, id: ScopedPlaceId) {
self.table.places[id].insert_flags(PlaceFlags::IS_USED);
}
pub(super) fn places(&self) -> impl Iterator<Item = &PlaceExpr> {
self.table.places()
}
pub(super) fn place_id_by_expr(&self, place_expr: &PlaceExpr) -> Option<ScopedPlaceId> {
self.table.place_id_by_expr(place_expr)
}
pub(super) fn place_expr(&self, place_id: impl Into<ScopedPlaceId>) -> &PlaceExpr {
self.table.place_expr(place_id)
}
/// Returns the place IDs associated with the place (e.g. `x.y`, `x.y.z`, `x.y.z[0]` for `x`).
pub(super) fn associated_place_ids(
&self,
place: ScopedPlaceId,
) -> impl Iterator<Item = ScopedPlaceId> {
self.associated_place_ids[place].iter().copied()
}
pub(super) fn finish(mut self) -> PlaceTable {
self.table.shrink_to_fit();
self.table
}
}
/// Reference to a node that introduces a new scope.
#[derive(Copy, Clone, Debug)]
pub(crate) enum NodeWithScopeRef<'a> {
Module,
Class(&'a ast::StmtClassDef),
Function(&'a ast::StmtFunctionDef),
Lambda(&'a ast::ExprLambda),
FunctionTypeParameters(&'a ast::StmtFunctionDef),
ClassTypeParameters(&'a ast::StmtClassDef),
TypeAlias(&'a ast::StmtTypeAlias),
TypeAliasTypeParameters(&'a ast::StmtTypeAlias),
ListComprehension(&'a ast::ExprListComp),
SetComprehension(&'a ast::ExprSetComp),
DictComprehension(&'a ast::ExprDictComp),
GeneratorExpression(&'a ast::ExprGenerator),
}
impl NodeWithScopeRef<'_> {
/// Converts the unowned reference to an owned [`NodeWithScopeKind`].
///
/// # Safety
/// The node wrapped by `self` must be a child of `module`.
#[expect(unsafe_code)]
pub(super) unsafe fn to_kind(self, module: ParsedModule) -> NodeWithScopeKind {
unsafe {
match self {
NodeWithScopeRef::Module => NodeWithScopeKind::Module,
NodeWithScopeRef::Class(class) => {
NodeWithScopeKind::Class(AstNodeRef::new(module, class))
}
NodeWithScopeRef::Function(function) => {
NodeWithScopeKind::Function(AstNodeRef::new(module, function))
}
NodeWithScopeRef::TypeAlias(type_alias) => {
NodeWithScopeKind::TypeAlias(AstNodeRef::new(module, type_alias))
}
NodeWithScopeRef::TypeAliasTypeParameters(type_alias) => {
NodeWithScopeKind::TypeAliasTypeParameters(AstNodeRef::new(module, type_alias))
}
NodeWithScopeRef::Lambda(lambda) => {
NodeWithScopeKind::Lambda(AstNodeRef::new(module, lambda))
}
NodeWithScopeRef::FunctionTypeParameters(function) => {
NodeWithScopeKind::FunctionTypeParameters(AstNodeRef::new(module, function))
}
NodeWithScopeRef::ClassTypeParameters(class) => {
NodeWithScopeKind::ClassTypeParameters(AstNodeRef::new(module, class))
}
NodeWithScopeRef::ListComprehension(comprehension) => {
NodeWithScopeKind::ListComprehension(AstNodeRef::new(module, comprehension))
}
NodeWithScopeRef::SetComprehension(comprehension) => {
NodeWithScopeKind::SetComprehension(AstNodeRef::new(module, comprehension))
}
NodeWithScopeRef::DictComprehension(comprehension) => {
NodeWithScopeKind::DictComprehension(AstNodeRef::new(module, comprehension))
}
NodeWithScopeRef::GeneratorExpression(generator) => {
NodeWithScopeKind::GeneratorExpression(AstNodeRef::new(module, generator))
}
}
}
}
pub(crate) fn node_key(self) -> NodeWithScopeKey {
match self {
NodeWithScopeRef::Module => NodeWithScopeKey::Module,
NodeWithScopeRef::Class(class) => NodeWithScopeKey::Class(NodeKey::from_node(class)),
NodeWithScopeRef::Function(function) => {
NodeWithScopeKey::Function(NodeKey::from_node(function))
}
NodeWithScopeRef::Lambda(lambda) => {
NodeWithScopeKey::Lambda(NodeKey::from_node(lambda))
}
NodeWithScopeRef::FunctionTypeParameters(function) => {
NodeWithScopeKey::FunctionTypeParameters(NodeKey::from_node(function))
}
NodeWithScopeRef::ClassTypeParameters(class) => {
NodeWithScopeKey::ClassTypeParameters(NodeKey::from_node(class))
}
NodeWithScopeRef::TypeAlias(type_alias) => {
NodeWithScopeKey::TypeAlias(NodeKey::from_node(type_alias))
}
NodeWithScopeRef::TypeAliasTypeParameters(type_alias) => {
NodeWithScopeKey::TypeAliasTypeParameters(NodeKey::from_node(type_alias))
}
NodeWithScopeRef::ListComprehension(comprehension) => {
NodeWithScopeKey::ListComprehension(NodeKey::from_node(comprehension))
}
NodeWithScopeRef::SetComprehension(comprehension) => {
NodeWithScopeKey::SetComprehension(NodeKey::from_node(comprehension))
}
NodeWithScopeRef::DictComprehension(comprehension) => {
NodeWithScopeKey::DictComprehension(NodeKey::from_node(comprehension))
}
NodeWithScopeRef::GeneratorExpression(generator) => {
NodeWithScopeKey::GeneratorExpression(NodeKey::from_node(generator))
}
}
}
}
/// Node that introduces a new scope.
#[derive(Clone, Debug, salsa::Update)]
pub enum NodeWithScopeKind {
Module,
Class(AstNodeRef<ast::StmtClassDef>),
ClassTypeParameters(AstNodeRef<ast::StmtClassDef>),
Function(AstNodeRef<ast::StmtFunctionDef>),
FunctionTypeParameters(AstNodeRef<ast::StmtFunctionDef>),
TypeAliasTypeParameters(AstNodeRef<ast::StmtTypeAlias>),
TypeAlias(AstNodeRef<ast::StmtTypeAlias>),
Lambda(AstNodeRef<ast::ExprLambda>),
ListComprehension(AstNodeRef<ast::ExprListComp>),
SetComprehension(AstNodeRef<ast::ExprSetComp>),
DictComprehension(AstNodeRef<ast::ExprDictComp>),
GeneratorExpression(AstNodeRef<ast::ExprGenerator>),
}
impl NodeWithScopeKind {
pub(crate) const fn scope_kind(&self) -> ScopeKind {
match self {
Self::Module => ScopeKind::Module,
Self::Class(_) => ScopeKind::Class,
Self::Function(_) => ScopeKind::Function,
Self::Lambda(_) => ScopeKind::Lambda,
Self::FunctionTypeParameters(_)
| Self::ClassTypeParameters(_)
| Self::TypeAliasTypeParameters(_) => ScopeKind::Annotation,
Self::TypeAlias(_) => ScopeKind::TypeAlias,
Self::ListComprehension(_)
| Self::SetComprehension(_)
| Self::DictComprehension(_)
| Self::GeneratorExpression(_) => ScopeKind::Comprehension,
}
}
pub fn expect_class(&self) -> &ast::StmtClassDef {
match self {
Self::Class(class) => class.node(),
_ => panic!("expected class"),
}
}
pub(crate) const fn as_class(&self) -> Option<&ast::StmtClassDef> {
match self {
Self::Class(class) => Some(class.node()),
_ => None,
}
}
pub fn expect_function(&self) -> &ast::StmtFunctionDef {
self.as_function().expect("expected function")
}
pub fn expect_type_alias(&self) -> &ast::StmtTypeAlias {
match self {
Self::TypeAlias(type_alias) => type_alias.node(),
_ => panic!("expected type alias"),
}
}
pub const fn as_function(&self) -> Option<&ast::StmtFunctionDef> {
match self {
Self::Function(function) => Some(function.node()),
_ => None,
}
}
}
#[derive(Copy, Clone, Debug, Eq, PartialEq, Hash)]
pub(crate) enum NodeWithScopeKey {
Module,
Class(NodeKey),
ClassTypeParameters(NodeKey),
Function(NodeKey),
FunctionTypeParameters(NodeKey),
TypeAlias(NodeKey),
TypeAliasTypeParameters(NodeKey),
Lambda(NodeKey),
ListComprehension(NodeKey),
SetComprehension(NodeKey),
DictComprehension(NodeKey),
GeneratorExpression(NodeKey),
}

View File

@@ -14,7 +14,7 @@ use ruff_python_ast::Singleton;
use crate::db::Db;
use crate::semantic_index::expression::Expression;
use crate::semantic_index::global_scope;
use crate::semantic_index::symbol::{FileScopeId, ScopeId, ScopedSymbolId};
use crate::semantic_index::place::{FileScopeId, ScopeId, ScopedPlaceId};
// A scoped identifier for each `Predicate` in a scope.
#[newtype_index]
@@ -144,13 +144,13 @@ pub(crate) struct StarImportPlaceholderPredicate<'db> {
/// Each symbol imported by a `*` import has a separate predicate associated with it:
/// this field identifies which symbol that is.
///
/// Note that a [`ScopedSymbolId`] is only meaningful if you also know the scope
/// Note that a [`ScopedPlaceId`] is only meaningful if you also know the scope
/// it is relative to. For this specific struct, however, there's no need to store a
/// separate field to hold the ID of the scope. `StarImportPredicate`s are only created
/// for valid `*`-import definitions, and valid `*`-import definitions can only ever
/// exist in the global scope; thus, we know that the `symbol_id` here will be relative
/// to the global scope of the importing file.
pub(crate) symbol_id: ScopedSymbolId,
pub(crate) symbol_id: ScopedPlaceId,
pub(crate) referenced_file: File,
}

View File

@@ -1,589 +0,0 @@
use std::hash::{Hash, Hasher};
use std::ops::Range;
use bitflags::bitflags;
use hashbrown::hash_map::RawEntryMut;
use ruff_db::files::File;
use ruff_db::parsed::ParsedModule;
use ruff_index::{IndexVec, newtype_index};
use ruff_python_ast as ast;
use ruff_python_ast::name::Name;
use rustc_hash::FxHasher;
use crate::Db;
use crate::ast_node_ref::AstNodeRef;
use crate::node_key::NodeKey;
use crate::semantic_index::visibility_constraints::ScopedVisibilityConstraintId;
use crate::semantic_index::{SemanticIndex, SymbolMap, semantic_index};
#[derive(Eq, PartialEq, Debug)]
pub struct Symbol {
name: Name,
flags: SymbolFlags,
}
impl Symbol {
fn new(name: Name) -> Self {
Self {
name,
flags: SymbolFlags::empty(),
}
}
fn insert_flags(&mut self, flags: SymbolFlags) {
self.flags.insert(flags);
}
/// The symbol's name.
pub fn name(&self) -> &Name {
&self.name
}
/// Is the symbol used in its containing scope?
pub fn is_used(&self) -> bool {
self.flags.contains(SymbolFlags::IS_USED)
}
/// Is the symbol defined in its containing scope?
pub fn is_bound(&self) -> bool {
self.flags.contains(SymbolFlags::IS_BOUND)
}
/// Is the symbol declared in its containing scope?
pub fn is_declared(&self) -> bool {
self.flags.contains(SymbolFlags::IS_DECLARED)
}
}
bitflags! {
/// Flags that can be queried to obtain information about a symbol in a given scope.
///
/// See the doc-comment at the top of [`super::use_def`] for explanations of what it
/// means for a symbol to be *bound* as opposed to *declared*.
#[derive(Copy, Clone, Debug, Eq, PartialEq)]
struct SymbolFlags: u8 {
const IS_USED = 1 << 0;
const IS_BOUND = 1 << 1;
const IS_DECLARED = 1 << 2;
/// TODO: This flag is not yet set by anything
const MARKED_GLOBAL = 1 << 3;
/// TODO: This flag is not yet set by anything
const MARKED_NONLOCAL = 1 << 4;
}
}
/// ID that uniquely identifies a symbol in a file.
#[derive(Copy, Clone, Debug, Eq, PartialEq, Hash)]
pub struct FileSymbolId {
scope: FileScopeId,
scoped_symbol_id: ScopedSymbolId,
}
impl FileSymbolId {
pub fn scope(self) -> FileScopeId {
self.scope
}
pub(crate) fn scoped_symbol_id(self) -> ScopedSymbolId {
self.scoped_symbol_id
}
}
impl From<FileSymbolId> for ScopedSymbolId {
fn from(val: FileSymbolId) -> Self {
val.scoped_symbol_id()
}
}
/// Symbol ID that uniquely identifies a symbol inside a [`Scope`].
#[newtype_index]
#[derive(salsa::Update)]
pub struct ScopedSymbolId;
/// A cross-module identifier of a scope that can be used as a salsa query parameter.
#[salsa::tracked(debug)]
pub struct ScopeId<'db> {
pub file: File,
pub file_scope_id: FileScopeId,
count: countme::Count<ScopeId<'static>>,
}
impl<'db> ScopeId<'db> {
pub(crate) fn is_function_like(self, db: &'db dyn Db) -> bool {
self.node(db).scope_kind().is_function_like()
}
pub(crate) fn is_type_parameter(self, db: &'db dyn Db) -> bool {
self.node(db).scope_kind().is_type_parameter()
}
pub(crate) fn node(self, db: &dyn Db) -> &NodeWithScopeKind {
self.scope(db).node()
}
pub(crate) fn scope(self, db: &dyn Db) -> &Scope {
semantic_index(db, self.file(db)).scope(self.file_scope_id(db))
}
#[cfg(test)]
pub(crate) fn name(self, db: &'db dyn Db) -> &'db str {
match self.node(db) {
NodeWithScopeKind::Module => "<module>",
NodeWithScopeKind::Class(class) | NodeWithScopeKind::ClassTypeParameters(class) => {
class.name.as_str()
}
NodeWithScopeKind::Function(function)
| NodeWithScopeKind::FunctionTypeParameters(function) => function.name.as_str(),
NodeWithScopeKind::TypeAlias(type_alias)
| NodeWithScopeKind::TypeAliasTypeParameters(type_alias) => type_alias
.name
.as_name_expr()
.map(|name| name.id.as_str())
.unwrap_or("<type alias>"),
NodeWithScopeKind::Lambda(_) => "<lambda>",
NodeWithScopeKind::ListComprehension(_) => "<listcomp>",
NodeWithScopeKind::SetComprehension(_) => "<setcomp>",
NodeWithScopeKind::DictComprehension(_) => "<dictcomp>",
NodeWithScopeKind::GeneratorExpression(_) => "<generator>",
}
}
}
/// ID that uniquely identifies a scope inside of a module.
#[newtype_index]
#[derive(salsa::Update)]
pub struct FileScopeId;
impl FileScopeId {
/// Returns the scope id of the module-global scope.
pub fn global() -> Self {
FileScopeId::from_u32(0)
}
pub fn is_global(self) -> bool {
self == FileScopeId::global()
}
pub fn to_scope_id(self, db: &dyn Db, file: File) -> ScopeId<'_> {
let index = semantic_index(db, file);
index.scope_ids_by_scope[self]
}
pub(crate) fn is_generator_function(self, index: &SemanticIndex) -> bool {
index.generator_functions.contains(&self)
}
}
#[derive(Debug, salsa::Update)]
pub struct Scope {
parent: Option<FileScopeId>,
node: NodeWithScopeKind,
descendants: Range<FileScopeId>,
reachability: ScopedVisibilityConstraintId,
}
impl Scope {
pub(super) fn new(
parent: Option<FileScopeId>,
node: NodeWithScopeKind,
descendants: Range<FileScopeId>,
reachability: ScopedVisibilityConstraintId,
) -> Self {
Scope {
parent,
node,
descendants,
reachability,
}
}
pub fn parent(&self) -> Option<FileScopeId> {
self.parent
}
pub fn node(&self) -> &NodeWithScopeKind {
&self.node
}
pub fn kind(&self) -> ScopeKind {
self.node().scope_kind()
}
pub fn descendants(&self) -> Range<FileScopeId> {
self.descendants.clone()
}
pub(super) fn extend_descendants(&mut self, children_end: FileScopeId) {
self.descendants = self.descendants.start..children_end;
}
pub(crate) fn is_eager(&self) -> bool {
self.kind().is_eager()
}
pub(crate) fn reachability(&self) -> ScopedVisibilityConstraintId {
self.reachability
}
}
#[derive(Copy, Clone, Debug, PartialEq, Eq)]
pub enum ScopeKind {
Module,
Annotation,
Class,
Function,
Lambda,
Comprehension,
TypeAlias,
}
impl ScopeKind {
pub(crate) fn is_eager(self) -> bool {
match self {
ScopeKind::Module | ScopeKind::Class | ScopeKind::Comprehension => true,
ScopeKind::Annotation
| ScopeKind::Function
| ScopeKind::Lambda
| ScopeKind::TypeAlias => false,
}
}
pub(crate) fn is_function_like(self) -> bool {
// Type parameter scopes behave like function scopes in terms of name resolution; CPython
// symbol table also uses the term "function-like" for these scopes.
matches!(
self,
ScopeKind::Annotation
| ScopeKind::Function
| ScopeKind::Lambda
| ScopeKind::TypeAlias
| ScopeKind::Comprehension
)
}
pub(crate) fn is_class(self) -> bool {
matches!(self, ScopeKind::Class)
}
pub(crate) fn is_type_parameter(self) -> bool {
matches!(self, ScopeKind::Annotation | ScopeKind::TypeAlias)
}
}
/// Symbol table for a specific [`Scope`].
#[derive(Default, salsa::Update)]
pub struct SymbolTable {
/// The symbols in this scope.
symbols: IndexVec<ScopedSymbolId, Symbol>,
/// The symbols indexed by name.
symbols_by_name: SymbolMap,
}
impl SymbolTable {
fn shrink_to_fit(&mut self) {
self.symbols.shrink_to_fit();
}
pub(crate) fn symbol(&self, symbol_id: impl Into<ScopedSymbolId>) -> &Symbol {
&self.symbols[symbol_id.into()]
}
#[expect(unused)]
pub(crate) fn symbol_ids(&self) -> impl Iterator<Item = ScopedSymbolId> {
self.symbols.indices()
}
pub fn symbols(&self) -> impl Iterator<Item = &Symbol> {
self.symbols.iter()
}
/// Returns the symbol named `name`.
pub(crate) fn symbol_by_name(&self, name: &str) -> Option<&Symbol> {
let id = self.symbol_id_by_name(name)?;
Some(self.symbol(id))
}
/// Returns the [`ScopedSymbolId`] of the symbol named `name`.
pub(crate) fn symbol_id_by_name(&self, name: &str) -> Option<ScopedSymbolId> {
let (id, ()) = self
.symbols_by_name
.raw_entry()
.from_hash(Self::hash_name(name), |id| {
self.symbol(*id).name().as_str() == name
})?;
Some(*id)
}
fn hash_name(name: &str) -> u64 {
let mut hasher = FxHasher::default();
name.hash(&mut hasher);
hasher.finish()
}
}
impl PartialEq for SymbolTable {
fn eq(&self, other: &Self) -> bool {
// We don't need to compare the symbols_by_name because the name is already captured in `Symbol`.
self.symbols == other.symbols
}
}
impl Eq for SymbolTable {}
impl std::fmt::Debug for SymbolTable {
/// Exclude the `symbols_by_name` field from the debug output.
/// It's very noisy and not useful for debugging.
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_tuple("SymbolTable")
.field(&self.symbols)
.finish_non_exhaustive()
}
}
#[derive(Debug, Default)]
pub(super) struct SymbolTableBuilder {
table: SymbolTable,
}
impl SymbolTableBuilder {
pub(super) fn add_symbol(&mut self, name: Name) -> (ScopedSymbolId, bool) {
let hash = SymbolTable::hash_name(&name);
let entry = self
.table
.symbols_by_name
.raw_entry_mut()
.from_hash(hash, |id| self.table.symbols[*id].name() == &name);
match entry {
RawEntryMut::Occupied(entry) => (*entry.key(), false),
RawEntryMut::Vacant(entry) => {
let symbol = Symbol::new(name);
let id = self.table.symbols.push(symbol);
entry.insert_with_hasher(hash, id, (), |id| {
SymbolTable::hash_name(self.table.symbols[*id].name().as_str())
});
(id, true)
}
}
}
pub(super) fn mark_symbol_bound(&mut self, id: ScopedSymbolId) {
self.table.symbols[id].insert_flags(SymbolFlags::IS_BOUND);
}
pub(super) fn mark_symbol_declared(&mut self, id: ScopedSymbolId) {
self.table.symbols[id].insert_flags(SymbolFlags::IS_DECLARED);
}
pub(super) fn mark_symbol_used(&mut self, id: ScopedSymbolId) {
self.table.symbols[id].insert_flags(SymbolFlags::IS_USED);
}
pub(super) fn symbols(&self) -> impl Iterator<Item = &Symbol> {
self.table.symbols()
}
pub(super) fn symbol_id_by_name(&self, name: &str) -> Option<ScopedSymbolId> {
self.table.symbol_id_by_name(name)
}
pub(super) fn symbol(&self, symbol_id: impl Into<ScopedSymbolId>) -> &Symbol {
self.table.symbol(symbol_id)
}
pub(super) fn finish(mut self) -> SymbolTable {
self.table.shrink_to_fit();
self.table
}
}
/// Reference to a node that introduces a new scope.
#[derive(Copy, Clone, Debug)]
pub(crate) enum NodeWithScopeRef<'a> {
Module,
Class(&'a ast::StmtClassDef),
Function(&'a ast::StmtFunctionDef),
Lambda(&'a ast::ExprLambda),
FunctionTypeParameters(&'a ast::StmtFunctionDef),
ClassTypeParameters(&'a ast::StmtClassDef),
TypeAlias(&'a ast::StmtTypeAlias),
TypeAliasTypeParameters(&'a ast::StmtTypeAlias),
ListComprehension(&'a ast::ExprListComp),
SetComprehension(&'a ast::ExprSetComp),
DictComprehension(&'a ast::ExprDictComp),
GeneratorExpression(&'a ast::ExprGenerator),
}
impl NodeWithScopeRef<'_> {
/// Converts the unowned reference to an owned [`NodeWithScopeKind`].
///
/// # Safety
/// The node wrapped by `self` must be a child of `module`.
#[expect(unsafe_code)]
pub(super) unsafe fn to_kind(self, module: ParsedModule) -> NodeWithScopeKind {
unsafe {
match self {
NodeWithScopeRef::Module => NodeWithScopeKind::Module,
NodeWithScopeRef::Class(class) => {
NodeWithScopeKind::Class(AstNodeRef::new(module, class))
}
NodeWithScopeRef::Function(function) => {
NodeWithScopeKind::Function(AstNodeRef::new(module, function))
}
NodeWithScopeRef::TypeAlias(type_alias) => {
NodeWithScopeKind::TypeAlias(AstNodeRef::new(module, type_alias))
}
NodeWithScopeRef::TypeAliasTypeParameters(type_alias) => {
NodeWithScopeKind::TypeAliasTypeParameters(AstNodeRef::new(module, type_alias))
}
NodeWithScopeRef::Lambda(lambda) => {
NodeWithScopeKind::Lambda(AstNodeRef::new(module, lambda))
}
NodeWithScopeRef::FunctionTypeParameters(function) => {
NodeWithScopeKind::FunctionTypeParameters(AstNodeRef::new(module, function))
}
NodeWithScopeRef::ClassTypeParameters(class) => {
NodeWithScopeKind::ClassTypeParameters(AstNodeRef::new(module, class))
}
NodeWithScopeRef::ListComprehension(comprehension) => {
NodeWithScopeKind::ListComprehension(AstNodeRef::new(module, comprehension))
}
NodeWithScopeRef::SetComprehension(comprehension) => {
NodeWithScopeKind::SetComprehension(AstNodeRef::new(module, comprehension))
}
NodeWithScopeRef::DictComprehension(comprehension) => {
NodeWithScopeKind::DictComprehension(AstNodeRef::new(module, comprehension))
}
NodeWithScopeRef::GeneratorExpression(generator) => {
NodeWithScopeKind::GeneratorExpression(AstNodeRef::new(module, generator))
}
}
}
}
pub(crate) fn node_key(self) -> NodeWithScopeKey {
match self {
NodeWithScopeRef::Module => NodeWithScopeKey::Module,
NodeWithScopeRef::Class(class) => NodeWithScopeKey::Class(NodeKey::from_node(class)),
NodeWithScopeRef::Function(function) => {
NodeWithScopeKey::Function(NodeKey::from_node(function))
}
NodeWithScopeRef::Lambda(lambda) => {
NodeWithScopeKey::Lambda(NodeKey::from_node(lambda))
}
NodeWithScopeRef::FunctionTypeParameters(function) => {
NodeWithScopeKey::FunctionTypeParameters(NodeKey::from_node(function))
}
NodeWithScopeRef::ClassTypeParameters(class) => {
NodeWithScopeKey::ClassTypeParameters(NodeKey::from_node(class))
}
NodeWithScopeRef::TypeAlias(type_alias) => {
NodeWithScopeKey::TypeAlias(NodeKey::from_node(type_alias))
}
NodeWithScopeRef::TypeAliasTypeParameters(type_alias) => {
NodeWithScopeKey::TypeAliasTypeParameters(NodeKey::from_node(type_alias))
}
NodeWithScopeRef::ListComprehension(comprehension) => {
NodeWithScopeKey::ListComprehension(NodeKey::from_node(comprehension))
}
NodeWithScopeRef::SetComprehension(comprehension) => {
NodeWithScopeKey::SetComprehension(NodeKey::from_node(comprehension))
}
NodeWithScopeRef::DictComprehension(comprehension) => {
NodeWithScopeKey::DictComprehension(NodeKey::from_node(comprehension))
}
NodeWithScopeRef::GeneratorExpression(generator) => {
NodeWithScopeKey::GeneratorExpression(NodeKey::from_node(generator))
}
}
}
}
/// Node that introduces a new scope.
#[derive(Clone, Debug, salsa::Update)]
pub enum NodeWithScopeKind {
Module,
Class(AstNodeRef<ast::StmtClassDef>),
ClassTypeParameters(AstNodeRef<ast::StmtClassDef>),
Function(AstNodeRef<ast::StmtFunctionDef>),
FunctionTypeParameters(AstNodeRef<ast::StmtFunctionDef>),
TypeAliasTypeParameters(AstNodeRef<ast::StmtTypeAlias>),
TypeAlias(AstNodeRef<ast::StmtTypeAlias>),
Lambda(AstNodeRef<ast::ExprLambda>),
ListComprehension(AstNodeRef<ast::ExprListComp>),
SetComprehension(AstNodeRef<ast::ExprSetComp>),
DictComprehension(AstNodeRef<ast::ExprDictComp>),
GeneratorExpression(AstNodeRef<ast::ExprGenerator>),
}
impl NodeWithScopeKind {
pub(crate) const fn scope_kind(&self) -> ScopeKind {
match self {
Self::Module => ScopeKind::Module,
Self::Class(_) => ScopeKind::Class,
Self::Function(_) => ScopeKind::Function,
Self::Lambda(_) => ScopeKind::Lambda,
Self::FunctionTypeParameters(_)
| Self::ClassTypeParameters(_)
| Self::TypeAliasTypeParameters(_) => ScopeKind::Annotation,
Self::TypeAlias(_) => ScopeKind::TypeAlias,
Self::ListComprehension(_)
| Self::SetComprehension(_)
| Self::DictComprehension(_)
| Self::GeneratorExpression(_) => ScopeKind::Comprehension,
}
}
pub fn expect_class(&self) -> &ast::StmtClassDef {
match self {
Self::Class(class) => class.node(),
_ => panic!("expected class"),
}
}
pub(crate) const fn as_class(&self) -> Option<&ast::StmtClassDef> {
match self {
Self::Class(class) => Some(class.node()),
_ => None,
}
}
pub fn expect_function(&self) -> &ast::StmtFunctionDef {
self.as_function().expect("expected function")
}
pub fn expect_type_alias(&self) -> &ast::StmtTypeAlias {
match self {
Self::TypeAlias(type_alias) => type_alias.node(),
_ => panic!("expected type alias"),
}
}
pub const fn as_function(&self) -> Option<&ast::StmtFunctionDef> {
match self {
Self::Function(function) => Some(function.node()),
_ => None,
}
}
}
#[derive(Copy, Clone, Debug, Eq, PartialEq, Hash)]
pub(crate) enum NodeWithScopeKey {
Module,
Class(NodeKey),
ClassTypeParameters(NodeKey),
Function(NodeKey),
FunctionTypeParameters(NodeKey),
TypeAlias(NodeKey),
TypeAliasTypeParameters(NodeKey),
Lambda(NodeKey),
ListComprehension(NodeKey),
SetComprehension(NodeKey),
DictComprehension(NodeKey),
GeneratorExpression(NodeKey),
}

View File

@@ -1,12 +1,20 @@
//! First, some terminology:
//!
//! * A "binding" gives a new value to a variable. This includes many different Python statements
//! * A "place" is semantically a location where a value can be read or written, and syntactically,
//! an expression that can be the target of an assignment, e.g. `x`, `x[0]`, `x.y`. (The term is
//! borrowed from Rust). In Python syntax, an expression like `f().x` is also allowed as the
//! target so it can be called a place, but we do not record declarations / bindings like `f().x:
//! int`, `f().x = ...`. Type checking itself can be done by recording only assignments to names,
//! but in order to perform type narrowing by attribute/subscript assignments, they must also be
//! recorded.
//!
//! * A "binding" gives a new value to a place. This includes many different Python statements
//! (assignment statements of course, but also imports, `def` and `class` statements, `as`
//! clauses in `with` and `except` statements, match patterns, and others) and even one
//! expression kind (named expressions). It notably does not include annotated assignment
//! statements without a right-hand side value; these do not assign any new value to the
//! variable. We consider function parameters to be bindings as well, since (from the perspective
//! of the function's internal scope), a function parameter begins the scope bound to a value.
//! statements without a right-hand side value; these do not assign any new value to the place.
//! We consider function parameters to be bindings as well, since (from the perspective of the
//! function's internal scope), a function parameter begins the scope bound to a value.
//!
//! * A "declaration" establishes an upper bound type for the values that a variable may be
//! permitted to take on. Annotated assignment statements (with or without an RHS value) are
@@ -67,12 +75,12 @@
//! Path(path)`, with the explicit `: Path` annotation, is permitted.
//!
//! The general rule is that whatever declaration(s) can reach a given binding determine the
//! validity of that binding. If there is a path in which the symbol is not declared, that is a
//! validity of that binding. If there is a path in which the place is not declared, that is a
//! declaration of `Unknown`. If multiple declarations can reach a binding, we union them, but by
//! default we also issue a type error, since this implicit union of declared types may hide an
//! error.
//!
//! To support type inference, we build a map from each use of a symbol to the bindings live at
//! To support type inference, we build a map from each use of a place to the bindings live at
//! that use, and the type narrowing constraints that apply to each binding.
//!
//! Let's take this code sample:
@@ -103,12 +111,12 @@
//! bindings and infer a type of `Literal[3, 4]` -- the union of `Literal[3]` and `Literal[4]` --
//! for the second use of `x`.
//!
//! So that's one question our use-def map needs to answer: given a specific use of a symbol, which
//! So that's one question our use-def map needs to answer: given a specific use of a place, which
//! binding(s) can reach that use. In [`AstIds`](crate::semantic_index::ast_ids::AstIds) we number
//! all uses (that means a `Name` node with `Load` context) so we have a `ScopedUseId` to
//! efficiently represent each use.
//! all uses (that means a `Name`/`ExprAttribute`/`ExprSubscript` node with `Load` context)
//! so we have a `ScopedUseId` to efficiently represent each use.
//!
//! We also need to know, for a given definition of a symbol, what type narrowing constraints apply
//! We also need to know, for a given definition of a place, what type narrowing constraints apply
//! to it. For instance, in this code sample:
//!
//! ```python
@@ -122,70 +130,70 @@
//! can rule out the possibility that `x` is `None` here, which should give us the type
//! `Literal[1]` for this use.
//!
//! For declared types, we need to be able to answer the question "given a binding to a symbol,
//! which declarations of that symbol can reach the binding?" This allows us to emit a diagnostic
//! For declared types, we need to be able to answer the question "given a binding to a place,
//! which declarations of that place can reach the binding?" This allows us to emit a diagnostic
//! if the binding is attempting to bind a value of a type that is not assignable to the declared
//! type for that symbol, at that point in control flow.
//! type for that place, at that point in control flow.
//!
//! We also need to know, given a declaration of a symbol, what the inferred type of that symbol is
//! We also need to know, given a declaration of a place, what the inferred type of that place is
//! at that point. This allows us to emit a diagnostic in a case like `x = "foo"; x: int`. The
//! binding `x = "foo"` occurs before the declaration `x: int`, so according to our
//! control-flow-sensitive interpretation of declarations, the assignment is not an error. But the
//! declaration is an error, since it would violate the "inferred type must be assignable to
//! declared type" rule.
//!
//! Another case we need to handle is when a symbol is referenced from a different scope (for
//! example, an import or a nonlocal reference). We call this "public" use of a symbol. For public
//! use of a symbol, we prefer the declared type, if there are any declarations of that symbol; if
//! Another case we need to handle is when a place is referenced from a different scope (for
//! example, an import or a nonlocal reference). We call this "public" use of a place. For public
//! use of a place, we prefer the declared type, if there are any declarations of that place; if
//! not, we fall back to the inferred type. So we also need to know which declarations and bindings
//! can reach the end of the scope.
//!
//! Technically, public use of a symbol could occur from any point in control flow of the scope
//! where the symbol is defined (via inline imports and import cycles, in the case of an import, or
//! via a function call partway through the local scope that ends up using a symbol from the scope
//! Technically, public use of a place could occur from any point in control flow of the scope
//! where the place is defined (via inline imports and import cycles, in the case of an import, or
//! via a function call partway through the local scope that ends up using a place from the scope
//! via a global or nonlocal reference.) But modeling this fully accurately requires whole-program
//! analysis that isn't tractable for an efficient analysis, since it means a given symbol could
//! analysis that isn't tractable for an efficient analysis, since it means a given place could
//! have a different type every place it's referenced throughout the program, depending on the
//! shape of arbitrarily-sized call/import graphs. So we follow other Python type checkers in
//! making the simplifying assumption that usually the scope will finish execution before its
//! symbols are made visible to other scopes; for instance, most imports will import from a
//! places are made visible to other scopes; for instance, most imports will import from a
//! complete module, not a partially-executed module. (We may want to get a little smarter than
//! this in the future for some closures, but for now this is where we start.)
//!
//! The data structure we build to answer these questions is the `UseDefMap`. It has a
//! `bindings_by_use` vector of [`SymbolBindings`] indexed by [`ScopedUseId`], a
//! `declarations_by_binding` vector of [`SymbolDeclarations`] indexed by [`ScopedDefinitionId`], a
//! `bindings_by_declaration` vector of [`SymbolBindings`] indexed by [`ScopedDefinitionId`], and
//! `public_bindings` and `public_definitions` vectors indexed by [`ScopedSymbolId`]. The values in
//! `bindings_by_use` vector of [`Bindings`] indexed by [`ScopedUseId`], a
//! `declarations_by_binding` vector of [`Declarations`] indexed by [`ScopedDefinitionId`], a
//! `bindings_by_declaration` vector of [`Bindings`] indexed by [`ScopedDefinitionId`], and
//! `public_bindings` and `public_definitions` vectors indexed by [`ScopedPlaceId`]. The values in
//! each of these vectors are (in principle) a list of live bindings at that use/definition, or at
//! the end of the scope for that symbol, with a list of the dominating constraints for each
//! the end of the scope for that place, with a list of the dominating constraints for each
//! binding.
//!
//! In order to avoid vectors-of-vectors-of-vectors and all the allocations that would entail, we
//! don't actually store these "list of visible definitions" as a vector of [`Definition`].
//! Instead, [`SymbolBindings`] and [`SymbolDeclarations`] are structs which use bit-sets to track
//! Instead, [`Bindings`] and [`Declarations`] are structs which use bit-sets to track
//! definitions (and constraints, in the case of bindings) in terms of [`ScopedDefinitionId`] and
//! [`ScopedPredicateId`], which are indices into the `all_definitions` and `predicates`
//! indexvecs in the [`UseDefMap`].
//!
//! There is another special kind of possible "definition" for a symbol: there might be a path from
//! the scope entry to a given use in which the symbol is never bound. We model this with a special
//! "unbound" definition (a `None` entry at the start of the `all_definitions` vector). If that
//! sentinel definition is present in the live bindings at a given use, it means that there is a
//! possible path through control flow in which that symbol is unbound. Similarly, if that sentinel
//! is present in the live declarations, it means that the symbol is (possibly) undeclared.
//! There is another special kind of possible "definition" for a place: there might be a path from
//! the scope entry to a given use in which the place is never bound. We model this with a special
//! "unbound/undeclared" definition (a [`DefinitionState::Undefined`] entry at the start of the
//! `all_definitions` vector). If that sentinel definition is present in the live bindings at a
//! given use, it means that there is a possible path through control flow in which that place is
//! unbound. Similarly, if that sentinel is present in the live declarations, it means that the
//! place is (possibly) undeclared.
//!
//! To build a [`UseDefMap`], the [`UseDefMapBuilder`] is notified of each new use, definition, and
//! constraint as they are encountered by the
//! [`SemanticIndexBuilder`](crate::semantic_index::builder::SemanticIndexBuilder) AST visit. For
//! each symbol, the builder tracks the `SymbolState` (`SymbolBindings` and `SymbolDeclarations`)
//! for that symbol. When we hit a use or definition of a symbol, we record the necessary parts of
//! the current state for that symbol that we need for that use or definition. When we reach the
//! end of the scope, it records the state for each symbol as the public definitions of that
//! symbol.
//! each place, the builder tracks the `PlaceState` (`Bindings` and `Declarations`) for that place.
//! When we hit a use or definition of a place, we record the necessary parts of the current state
//! for that place that we need for that use or definition. When we reach the end of the scope, it
//! records the state for each place as the public definitions of that place.
//!
//! Let's walk through the above example. Initially we do not have any record of `x`. When we add
//! the new symbol (before we process the first binding), we create a new undefined `SymbolState`
//! the new place (before we process the first binding), we create a new undefined `PlaceState`
//! which has a single live binding (the "unbound" definition) and a single live declaration (the
//! "undeclared" definition). When we see `x = 1`, we record that as the sole live binding of `x`.
//! The "unbound" binding is no longer visible. Then we see `x = 2`, and we replace `x = 1` as the
@@ -193,11 +201,11 @@
//! of `x` are just the `x = 2` definition.
//!
//! Then we hit the `if` branch. We visit the `test` node (`flag` in this case), since that will
//! happen regardless. Then we take a pre-branch snapshot of the current state for all symbols,
//! happen regardless. Then we take a pre-branch snapshot of the current state for all places,
//! which we'll need later. Then we record `flag` as a possible constraint on the current binding
//! (`x = 2`), and go ahead and visit the `if` body. When we see `x = 3`, it replaces `x = 2`
//! (constrained by `flag`) as the sole live binding of `x`. At the end of the `if` body, we take
//! another snapshot of the current symbol state; we'll call this the post-if-body snapshot.
//! another snapshot of the current place state; we'll call this the post-if-body snapshot.
//!
//! Now we need to visit the `else` clause. The conditions when entering the `else` clause should
//! be the pre-if conditions; if we are entering the `else` clause, we know that the `if` test
@@ -247,7 +255,7 @@
//! `__bool__` method of `test` returns type `bool`, we can see both bindings.
//!
//! Note that we also record visibility constraints for the start of the scope. This is important
//! to determine if a symbol is definitely bound, possibly unbound, or definitely unbound. In the
//! to determine if a place is definitely bound, possibly unbound, or definitely unbound. In the
//! example above, The `y = <unbound>` binding is constrained by `~test`, so `y` would only be
//! definitely-bound if `test` is always truthy.
//!
@@ -259,34 +267,34 @@
use ruff_index::{IndexVec, newtype_index};
use rustc_hash::FxHashMap;
use self::symbol_state::{
EagerSnapshot, LiveBindingsIterator, LiveDeclaration, LiveDeclarationsIterator,
ScopedDefinitionId, SymbolBindings, SymbolDeclarations, SymbolState,
use self::place_state::{
Bindings, Declarations, EagerSnapshot, LiveBindingsIterator, LiveDeclaration,
LiveDeclarationsIterator, PlaceState, ScopedDefinitionId,
};
use crate::node_key::NodeKey;
use crate::semantic_index::EagerSnapshotResult;
use crate::semantic_index::ast_ids::ScopedUseId;
use crate::semantic_index::definition::Definition;
use crate::semantic_index::definition::{Definition, DefinitionState};
use crate::semantic_index::narrowing_constraints::{
ConstraintKey, NarrowingConstraints, NarrowingConstraintsBuilder, NarrowingConstraintsIterator,
};
use crate::semantic_index::place::{FileScopeId, PlaceExpr, ScopeKind, ScopedPlaceId};
use crate::semantic_index::predicate::{
Predicate, Predicates, PredicatesBuilder, ScopedPredicateId, StarImportPlaceholderPredicate,
};
use crate::semantic_index::symbol::{FileScopeId, ScopeKind, ScopedSymbolId};
use crate::semantic_index::visibility_constraints::{
ScopedVisibilityConstraintId, VisibilityConstraints, VisibilityConstraintsBuilder,
};
use crate::types::{IntersectionBuilder, Truthiness, Type, infer_narrowing_constraint};
mod symbol_state;
mod place_state;
/// Applicable definitions and constraints for every use of a name.
#[derive(Debug, PartialEq, Eq, salsa::Update)]
pub(crate) struct UseDefMap<'db> {
/// Array of [`Definition`] in this scope. Only the first entry should be `None`;
/// this represents the implicit "unbound"/"undeclared" definition of every symbol.
all_definitions: IndexVec<ScopedDefinitionId, Option<Definition<'db>>>,
/// Array of [`Definition`] in this scope. Only the first entry should be [`DefinitionState::Undefined`];
/// this represents the implicit "unbound"/"undeclared" definition of every place.
all_definitions: IndexVec<ScopedDefinitionId, DefinitionState<'db>>,
/// Array of predicates in this scope.
predicates: Predicates<'db>,
@@ -297,34 +305,31 @@ pub(crate) struct UseDefMap<'db> {
/// Array of visibility constraints in this scope.
visibility_constraints: VisibilityConstraints,
/// [`SymbolBindings`] reaching a [`ScopedUseId`].
bindings_by_use: IndexVec<ScopedUseId, SymbolBindings>,
/// [`Bindings`] reaching a [`ScopedUseId`].
bindings_by_use: IndexVec<ScopedUseId, Bindings>,
/// Tracks whether or not a given AST node is reachable from the start of the scope.
node_reachability: FxHashMap<NodeKey, ScopedVisibilityConstraintId>,
/// If the definition is a binding (only) -- `x = 1` for example -- then we need
/// [`SymbolDeclarations`] to know whether this binding is permitted by the live declarations.
/// [`Declarations`] to know whether this binding is permitted by the live declarations.
///
/// If the definition is both a declaration and a binding -- `x: int = 1` for example -- then
/// we don't actually need anything here, all we'll need to validate is that our own RHS is a
/// valid assignment to our own annotation.
declarations_by_binding: FxHashMap<Definition<'db>, SymbolDeclarations>,
declarations_by_binding: FxHashMap<Definition<'db>, Declarations>,
/// If the definition is a declaration (only) -- `x: int` for example -- then we need
/// [`SymbolBindings`] to know whether this declaration is consistent with the previously
/// [`Bindings`] to know whether this declaration is consistent with the previously
/// inferred type.
///
/// If the definition is both a declaration and a binding -- `x: int = 1` for example -- then
/// we don't actually need anything here, all we'll need to validate is that our own RHS is a
/// valid assignment to our own annotation.
bindings_by_declaration: FxHashMap<Definition<'db>, SymbolBindings>,
bindings_by_declaration: FxHashMap<Definition<'db>, Bindings>,
/// [`SymbolState`] visible at end of scope for each symbol.
public_symbols: IndexVec<ScopedSymbolId, SymbolState>,
/// [`SymbolState`] for each instance attribute.
instance_attributes: IndexVec<ScopedSymbolId, SymbolState>,
/// [`PlaceState`] visible at end of scope for each place.
public_places: IndexVec<ScopedPlaceId, PlaceState>,
/// Snapshot of bindings in this scope that can be used to resolve a reference in a nested
/// eager scope.
@@ -402,16 +407,9 @@ impl<'db> UseDefMap<'db> {
pub(crate) fn public_bindings(
&self,
symbol: ScopedSymbolId,
place: ScopedPlaceId,
) -> BindingWithConstraintsIterator<'_, 'db> {
self.bindings_iterator(self.public_symbols[symbol].bindings())
}
pub(crate) fn instance_attribute_bindings(
&self,
symbol: ScopedSymbolId,
) -> BindingWithConstraintsIterator<'_, 'db> {
self.bindings_iterator(self.instance_attributes[symbol].bindings())
self.bindings_iterator(self.public_places[place].bindings())
}
pub(crate) fn eager_snapshot(
@@ -422,8 +420,8 @@ impl<'db> UseDefMap<'db> {
Some(EagerSnapshot::Constraint(constraint)) => {
EagerSnapshotResult::FoundConstraint(*constraint)
}
Some(EagerSnapshot::Bindings(symbol_bindings)) => {
EagerSnapshotResult::FoundBindings(self.bindings_iterator(symbol_bindings))
Some(EagerSnapshot::Bindings(bindings)) => {
EagerSnapshotResult::FoundBindings(self.bindings_iterator(bindings))
}
None => EagerSnapshotResult::NotFound,
}
@@ -445,27 +443,27 @@ impl<'db> UseDefMap<'db> {
pub(crate) fn public_declarations<'map>(
&'map self,
symbol: ScopedSymbolId,
place: ScopedPlaceId,
) -> DeclarationsIterator<'map, 'db> {
let declarations = self.public_symbols[symbol].declarations();
let declarations = self.public_places[place].declarations();
self.declarations_iterator(declarations)
}
pub(crate) fn all_public_declarations<'map>(
&'map self,
) -> impl Iterator<Item = (ScopedSymbolId, DeclarationsIterator<'map, 'db>)> + 'map {
(0..self.public_symbols.len())
.map(ScopedSymbolId::from_usize)
.map(|symbol_id| (symbol_id, self.public_declarations(symbol_id)))
) -> impl Iterator<Item = (ScopedPlaceId, DeclarationsIterator<'map, 'db>)> + 'map {
(0..self.public_places.len())
.map(ScopedPlaceId::from_usize)
.map(|place_id| (place_id, self.public_declarations(place_id)))
}
pub(crate) fn all_public_bindings<'map>(
&'map self,
) -> impl Iterator<Item = (ScopedSymbolId, BindingWithConstraintsIterator<'map, 'db>)> + 'map
) -> impl Iterator<Item = (ScopedPlaceId, BindingWithConstraintsIterator<'map, 'db>)> + 'map
{
(0..self.public_symbols.len())
.map(ScopedSymbolId::from_usize)
.map(|symbol_id| (symbol_id, self.public_bindings(symbol_id)))
(0..self.public_places.len())
.map(ScopedPlaceId::from_usize)
.map(|place_id| (place_id, self.public_bindings(place_id)))
}
/// This function is intended to be called only once inside `TypeInferenceBuilder::infer_function_body`.
@@ -487,7 +485,7 @@ impl<'db> UseDefMap<'db> {
fn bindings_iterator<'map>(
&'map self,
bindings: &'map SymbolBindings,
bindings: &'map Bindings,
) -> BindingWithConstraintsIterator<'map, 'db> {
BindingWithConstraintsIterator {
all_definitions: &self.all_definitions,
@@ -500,7 +498,7 @@ impl<'db> UseDefMap<'db> {
fn declarations_iterator<'map>(
&'map self,
declarations: &'map SymbolDeclarations,
declarations: &'map Declarations,
) -> DeclarationsIterator<'map, 'db> {
DeclarationsIterator {
all_definitions: &self.all_definitions,
@@ -511,12 +509,12 @@ impl<'db> UseDefMap<'db> {
}
}
/// Uniquely identifies a snapshot of a symbol state that can be used to resolve a reference in a
/// Uniquely identifies a snapshot of a place state that can be used to resolve a reference in a
/// nested eager scope.
///
/// An eager scope has its entire body executed immediately at the location where it is defined.
/// For any free references in the nested scope, we use the bindings that are visible at the point
/// where the nested scope is defined, instead of using the public type of the symbol.
/// where the nested scope is defined, instead of using the public type of the place.
///
/// There is a unique ID for each distinct [`EagerSnapshotKey`] in the file.
#[newtype_index]
@@ -526,18 +524,18 @@ pub(crate) struct ScopedEagerSnapshotId;
pub(crate) struct EagerSnapshotKey {
/// The enclosing scope containing the bindings
pub(crate) enclosing_scope: FileScopeId,
/// The referenced symbol (in the enclosing scope)
pub(crate) enclosing_symbol: ScopedSymbolId,
/// The referenced place (in the enclosing scope)
pub(crate) enclosing_place: ScopedPlaceId,
/// The nested eager scope containing the reference
pub(crate) nested_scope: FileScopeId,
}
/// A snapshot of symbol states that can be used to resolve a reference in a nested eager scope.
/// A snapshot of place states that can be used to resolve a reference in a nested eager scope.
type EagerSnapshots = IndexVec<ScopedEagerSnapshotId, EagerSnapshot>;
#[derive(Debug)]
pub(crate) struct BindingWithConstraintsIterator<'map, 'db> {
all_definitions: &'map IndexVec<ScopedDefinitionId, Option<Definition<'db>>>,
all_definitions: &'map IndexVec<ScopedDefinitionId, DefinitionState<'db>>,
pub(crate) predicates: &'map Predicates<'db>,
pub(crate) narrowing_constraints: &'map NarrowingConstraints,
pub(crate) visibility_constraints: &'map VisibilityConstraints,
@@ -568,7 +566,7 @@ impl<'map, 'db> Iterator for BindingWithConstraintsIterator<'map, 'db> {
impl std::iter::FusedIterator for BindingWithConstraintsIterator<'_, '_> {}
pub(crate) struct BindingWithConstraints<'map, 'db> {
pub(crate) binding: Option<Definition<'db>>,
pub(crate) binding: DefinitionState<'db>,
pub(crate) narrowing_constraint: ConstraintsIterator<'map, 'db>,
pub(crate) visibility_constraint: ScopedVisibilityConstraintId,
}
@@ -595,10 +593,10 @@ impl<'db> ConstraintsIterator<'_, 'db> {
self,
db: &'db dyn crate::Db,
base_ty: Type<'db>,
symbol: ScopedSymbolId,
place: ScopedPlaceId,
) -> Type<'db> {
let constraint_tys: Vec<_> = self
.filter_map(|constraint| infer_narrowing_constraint(db, constraint, symbol))
.filter_map(|constraint| infer_narrowing_constraint(db, constraint, place))
.collect();
if constraint_tys.is_empty() {
@@ -618,14 +616,14 @@ impl<'db> ConstraintsIterator<'_, 'db> {
#[derive(Clone)]
pub(crate) struct DeclarationsIterator<'map, 'db> {
all_definitions: &'map IndexVec<ScopedDefinitionId, Option<Definition<'db>>>,
all_definitions: &'map IndexVec<ScopedDefinitionId, DefinitionState<'db>>,
pub(crate) predicates: &'map Predicates<'db>,
pub(crate) visibility_constraints: &'map VisibilityConstraints,
inner: LiveDeclarationsIterator<'map>,
}
pub(crate) struct DeclarationWithConstraint<'db> {
pub(crate) declaration: Option<Definition<'db>>,
pub(crate) declaration: DefinitionState<'db>,
pub(crate) visibility_constraint: ScopedVisibilityConstraintId,
}
@@ -652,8 +650,7 @@ impl std::iter::FusedIterator for DeclarationsIterator<'_, '_> {}
/// A snapshot of the definitions and constraints state at a particular point in control flow.
#[derive(Clone, Debug)]
pub(super) struct FlowSnapshot {
symbol_states: IndexVec<ScopedSymbolId, SymbolState>,
instance_attribute_states: IndexVec<ScopedSymbolId, SymbolState>,
place_states: IndexVec<ScopedPlaceId, PlaceState>,
scope_start_visibility: ScopedVisibilityConstraintId,
reachability: ScopedVisibilityConstraintId,
}
@@ -661,7 +658,7 @@ pub(super) struct FlowSnapshot {
#[derive(Debug)]
pub(super) struct UseDefMapBuilder<'db> {
/// Append-only array of [`Definition`].
all_definitions: IndexVec<ScopedDefinitionId, Option<Definition<'db>>>,
all_definitions: IndexVec<ScopedDefinitionId, DefinitionState<'db>>,
/// Builder of predicates.
pub(super) predicates: PredicatesBuilder<'db>,
@@ -673,7 +670,7 @@ pub(super) struct UseDefMapBuilder<'db> {
pub(super) visibility_constraints: VisibilityConstraintsBuilder,
/// A constraint which describes the visibility of the unbound/undeclared state, i.e.
/// whether or not a use of a symbol at the current point in control flow would see
/// whether or not a use of a place at the current point in control flow would see
/// the fake `x = <unbound>` binding at the start of the scope. This is important for
/// cases like the following, where we need to hide the implicit unbound binding in
/// the "else" branch:
@@ -688,7 +685,7 @@ pub(super) struct UseDefMapBuilder<'db> {
pub(super) scope_start_visibility: ScopedVisibilityConstraintId,
/// Live bindings at each so-far-recorded use.
bindings_by_use: IndexVec<ScopedUseId, SymbolBindings>,
bindings_by_use: IndexVec<ScopedUseId, Bindings>,
/// Tracks whether or not the scope start is visible at the current point in control flow.
/// This is subtly different from `scope_start_visibility`, as we apply these constraints
@@ -725,18 +722,15 @@ pub(super) struct UseDefMapBuilder<'db> {
node_reachability: FxHashMap<NodeKey, ScopedVisibilityConstraintId>,
/// Live declarations for each so-far-recorded binding.
declarations_by_binding: FxHashMap<Definition<'db>, SymbolDeclarations>,
declarations_by_binding: FxHashMap<Definition<'db>, Declarations>,
/// Live bindings for each so-far-recorded declaration.
bindings_by_declaration: FxHashMap<Definition<'db>, SymbolBindings>,
bindings_by_declaration: FxHashMap<Definition<'db>, Bindings>,
/// Currently live bindings and declarations for each symbol.
symbol_states: IndexVec<ScopedSymbolId, SymbolState>,
/// Currently live bindings and declarations for each place.
place_states: IndexVec<ScopedPlaceId, PlaceState>,
/// Currently live bindings for each instance attribute.
instance_attribute_states: IndexVec<ScopedSymbolId, SymbolState>,
/// Snapshots of symbol states in this scope that can be used to resolve a reference in a
/// Snapshots of place states in this scope that can be used to resolve a reference in a
/// nested eager scope.
eager_snapshots: EagerSnapshots,
@@ -747,7 +741,7 @@ pub(super) struct UseDefMapBuilder<'db> {
impl<'db> UseDefMapBuilder<'db> {
pub(super) fn new(is_class_scope: bool) -> Self {
Self {
all_definitions: IndexVec::from_iter([None]),
all_definitions: IndexVec::from_iter([DefinitionState::Undefined]),
predicates: PredicatesBuilder::default(),
narrowing_constraints: NarrowingConstraintsBuilder::default(),
visibility_constraints: VisibilityConstraintsBuilder::default(),
@@ -757,9 +751,8 @@ impl<'db> UseDefMapBuilder<'db> {
node_reachability: FxHashMap::default(),
declarations_by_binding: FxHashMap::default(),
bindings_by_declaration: FxHashMap::default(),
symbol_states: IndexVec::new(),
place_states: IndexVec::new(),
eager_snapshots: EagerSnapshots::default(),
instance_attribute_states: IndexVec::new(),
is_class_scope,
}
}
@@ -768,38 +761,29 @@ impl<'db> UseDefMapBuilder<'db> {
self.reachability = ScopedVisibilityConstraintId::ALWAYS_FALSE;
}
pub(super) fn add_symbol(&mut self, symbol: ScopedSymbolId) {
let new_symbol = self
.symbol_states
.push(SymbolState::undefined(self.scope_start_visibility));
debug_assert_eq!(symbol, new_symbol);
pub(super) fn add_place(&mut self, place: ScopedPlaceId) {
let new_place = self
.place_states
.push(PlaceState::undefined(self.scope_start_visibility));
debug_assert_eq!(place, new_place);
}
pub(super) fn add_attribute(&mut self, symbol: ScopedSymbolId) {
let new_symbol = self
.instance_attribute_states
.push(SymbolState::undefined(self.scope_start_visibility));
debug_assert_eq!(symbol, new_symbol);
}
pub(super) fn record_binding(&mut self, symbol: ScopedSymbolId, binding: Definition<'db>) {
let def_id = self.all_definitions.push(Some(binding));
let symbol_state = &mut self.symbol_states[symbol];
self.declarations_by_binding
.insert(binding, symbol_state.declarations().clone());
symbol_state.record_binding(def_id, self.scope_start_visibility, self.is_class_scope);
}
pub(super) fn record_attribute_binding(
pub(super) fn record_binding(
&mut self,
symbol: ScopedSymbolId,
place: ScopedPlaceId,
binding: Definition<'db>,
is_place_name: bool,
) {
let def_id = self.all_definitions.push(Some(binding));
let attribute_state = &mut self.instance_attribute_states[symbol];
let def_id = self.all_definitions.push(DefinitionState::Defined(binding));
let place_state = &mut self.place_states[place];
self.declarations_by_binding
.insert(binding, attribute_state.declarations().clone());
attribute_state.record_binding(def_id, self.scope_start_visibility, self.is_class_scope);
.insert(binding, place_state.declarations().clone());
place_state.record_binding(
def_id,
self.scope_start_visibility,
self.is_class_scope,
is_place_name,
);
}
pub(super) fn add_predicate(&mut self, predicate: Predicate<'db>) -> ScopedPredicateId {
@@ -808,11 +792,7 @@ impl<'db> UseDefMapBuilder<'db> {
pub(super) fn record_narrowing_constraint(&mut self, predicate: ScopedPredicateId) {
let narrowing_constraint = predicate.into();
for state in &mut self.symbol_states {
state
.record_narrowing_constraint(&mut self.narrowing_constraints, narrowing_constraint);
}
for state in &mut self.instance_attribute_states {
for state in &mut self.place_states {
state
.record_narrowing_constraint(&mut self.narrowing_constraints, narrowing_constraint);
}
@@ -822,10 +802,7 @@ impl<'db> UseDefMapBuilder<'db> {
&mut self,
constraint: ScopedVisibilityConstraintId,
) {
for state in &mut self.symbol_states {
state.record_visibility_constraint(&mut self.visibility_constraints, constraint);
}
for state in &mut self.instance_attribute_states {
for state in &mut self.place_states {
state.record_visibility_constraint(&mut self.visibility_constraints, constraint);
}
self.scope_start_visibility = self
@@ -833,13 +810,13 @@ impl<'db> UseDefMapBuilder<'db> {
.add_and_constraint(self.scope_start_visibility, constraint);
}
/// Snapshot the state of a single symbol at the current point in control flow.
/// Snapshot the state of a single place at the current point in control flow.
///
/// This is only used for `*`-import visibility constraints, which are handled differently
/// to most other visibility constraints. See the doc-comment for
/// [`Self::record_and_negate_star_import_visibility_constraint`] for more details.
pub(super) fn single_symbol_snapshot(&self, symbol: ScopedSymbolId) -> SymbolState {
self.symbol_states[symbol].clone()
pub(super) fn single_place_snapshot(&self, place: ScopedPlaceId) -> PlaceState {
self.place_states[place].clone()
}
/// This method exists solely for handling `*`-import visibility constraints.
@@ -863,10 +840,10 @@ impl<'db> UseDefMapBuilder<'db> {
/// Doing things this way is cheaper in and of itself. However, it also allows us to avoid
/// calling [`Self::simplify_visibility_constraints`] after the constraint has been applied to
/// the "if-predicate-true" branch and negated for the "if-predicate-false" branch. Simplifying
/// the visibility constraints is only important for symbols that did not have any new
/// the visibility constraints is only important for places that did not have any new
/// definitions inside either the "if-predicate-true" branch or the "if-predicate-false" branch.
///
/// - We only snapshot the state for a single symbol prior to the definition, rather than doing
/// - We only snapshot the state for a single place prior to the definition, rather than doing
/// expensive calls to [`Self::snapshot`]. Again, this is possible because we know
/// that only a single definition occurs inside the "if-predicate-true" predicate branch.
///
@@ -880,8 +857,8 @@ impl<'db> UseDefMapBuilder<'db> {
pub(super) fn record_and_negate_star_import_visibility_constraint(
&mut self,
star_import: StarImportPlaceholderPredicate<'db>,
symbol: ScopedSymbolId,
pre_definition_state: SymbolState,
symbol: ScopedPlaceId,
pre_definition_state: PlaceState,
) {
let predicate_id = self.add_predicate(star_import.into());
let visibility_id = self.visibility_constraints.add_atom(predicate_id);
@@ -890,22 +867,22 @@ impl<'db> UseDefMapBuilder<'db> {
.add_not_constraint(visibility_id);
let mut post_definition_state =
std::mem::replace(&mut self.symbol_states[symbol], pre_definition_state);
std::mem::replace(&mut self.place_states[symbol], pre_definition_state);
post_definition_state
.record_visibility_constraint(&mut self.visibility_constraints, visibility_id);
self.symbol_states[symbol]
self.place_states[symbol]
.record_visibility_constraint(&mut self.visibility_constraints, negated_visibility_id);
self.symbol_states[symbol].merge(
self.place_states[symbol].merge(
post_definition_state,
&mut self.narrowing_constraints,
&mut self.visibility_constraints,
);
}
/// This method resets the visibility constraints for all symbols to a previous state
/// This method resets the visibility constraints for all places to a previous state
/// *if* there have been no new declarations or bindings since then. Consider the
/// following example:
/// ```py
@@ -924,10 +901,7 @@ impl<'db> UseDefMapBuilder<'db> {
/// constraint for the `x = 0` binding as well, but at the `RESET` point, we can get rid
/// of it, as the `if`-`elif`-`elif` chain doesn't include any new bindings of `x`.
pub(super) fn simplify_visibility_constraints(&mut self, snapshot: FlowSnapshot) {
debug_assert!(self.symbol_states.len() >= snapshot.symbol_states.len());
debug_assert!(
self.instance_attribute_states.len() >= snapshot.instance_attribute_states.len()
);
debug_assert!(self.place_states.len() >= snapshot.place_states.len());
// If there are any control flow paths that have become unreachable between `snapshot` and
// now, then it's not valid to simplify any visibility constraints to `snapshot`.
@@ -935,20 +909,13 @@ impl<'db> UseDefMapBuilder<'db> {
return;
}
// Note that this loop terminates when we reach a symbol not present in the snapshot.
// This means we keep visibility constraints for all new symbols, which is intended,
// since these symbols have been introduced in the corresponding branch, which might
// Note that this loop terminates when we reach a place not present in the snapshot.
// This means we keep visibility constraints for all new places, which is intended,
// since these places have been introduced in the corresponding branch, which might
// be subject to visibility constraints. We only simplify/reset visibility constraints
// for symbols that have the same bindings and declarations present compared to the
// for places that have the same bindings and declarations present compared to the
// snapshot.
for (current, snapshot) in self.symbol_states.iter_mut().zip(snapshot.symbol_states) {
current.simplify_visibility_constraints(snapshot);
}
for (current, snapshot) in self
.instance_attribute_states
.iter_mut()
.zip(snapshot.instance_attribute_states)
{
for (current, snapshot) in self.place_states.iter_mut().zip(snapshot.place_states) {
current.simplify_visibility_constraints(snapshot);
}
}
@@ -965,43 +932,64 @@ impl<'db> UseDefMapBuilder<'db> {
pub(super) fn record_declaration(
&mut self,
symbol: ScopedSymbolId,
place: ScopedPlaceId,
declaration: Definition<'db>,
) {
let def_id = self.all_definitions.push(Some(declaration));
let symbol_state = &mut self.symbol_states[symbol];
let def_id = self
.all_definitions
.push(DefinitionState::Defined(declaration));
let place_state = &mut self.place_states[place];
self.bindings_by_declaration
.insert(declaration, symbol_state.bindings().clone());
symbol_state.record_declaration(def_id);
.insert(declaration, place_state.bindings().clone());
place_state.record_declaration(def_id);
}
pub(super) fn record_declaration_and_binding(
&mut self,
symbol: ScopedSymbolId,
place: ScopedPlaceId,
definition: Definition<'db>,
is_place_name: bool,
) {
// We don't need to store anything in self.bindings_by_declaration or
// self.declarations_by_binding.
let def_id = self.all_definitions.push(Some(definition));
let symbol_state = &mut self.symbol_states[symbol];
symbol_state.record_declaration(def_id);
symbol_state.record_binding(def_id, self.scope_start_visibility, self.is_class_scope);
let def_id = self
.all_definitions
.push(DefinitionState::Defined(definition));
let place_state = &mut self.place_states[place];
place_state.record_declaration(def_id);
place_state.record_binding(
def_id,
self.scope_start_visibility,
self.is_class_scope,
is_place_name,
);
}
pub(super) fn delete_binding(&mut self, place: ScopedPlaceId, is_place_name: bool) {
let def_id = self.all_definitions.push(DefinitionState::Deleted);
let place_state = &mut self.place_states[place];
place_state.record_binding(
def_id,
self.scope_start_visibility,
self.is_class_scope,
is_place_name,
);
}
pub(super) fn record_use(
&mut self,
symbol: ScopedSymbolId,
place: ScopedPlaceId,
use_id: ScopedUseId,
node_key: NodeKey,
) {
// We have a use of a symbol; clone the current bindings for that symbol, and record them
// We have a use of a place; clone the current bindings for that place, and record them
// as the live bindings for this use.
let new_use = self
.bindings_by_use
.push(self.symbol_states[symbol].bindings().clone());
.push(self.place_states[place].bindings().clone());
debug_assert_eq!(use_id, new_use);
// Track reachability of all uses of symbols to silence `unresolved-reference`
// Track reachability of all uses of places to silence `unresolved-reference`
// diagnostics in unreachable code.
self.record_node_reachability(node_key);
}
@@ -1012,66 +1000,59 @@ impl<'db> UseDefMapBuilder<'db> {
pub(super) fn snapshot_eager_state(
&mut self,
enclosing_symbol: ScopedSymbolId,
enclosing_place: ScopedPlaceId,
scope: ScopeKind,
is_bound: bool,
enclosing_place_expr: &PlaceExpr,
) -> ScopedEagerSnapshotId {
// Names bound in class scopes are never visible to nested scopes, so we never need to
// save eager scope bindings in a class scope.
if scope.is_class() || !is_bound {
// Names bound in class scopes are never visible to nested scopes (but attributes/subscripts are visible),
// so we never need to save eager scope bindings in a class scope.
if (scope.is_class() && enclosing_place_expr.is_name()) || !enclosing_place_expr.is_bound()
{
self.eager_snapshots.push(EagerSnapshot::Constraint(
self.symbol_states[enclosing_symbol]
self.place_states[enclosing_place]
.bindings()
.unbound_narrowing_constraint(),
))
} else {
self.eager_snapshots.push(EagerSnapshot::Bindings(
self.symbol_states[enclosing_symbol].bindings().clone(),
self.place_states[enclosing_place].bindings().clone(),
))
}
}
/// Take a snapshot of the current visible-symbols state.
/// Take a snapshot of the current visible-places state.
pub(super) fn snapshot(&self) -> FlowSnapshot {
FlowSnapshot {
symbol_states: self.symbol_states.clone(),
instance_attribute_states: self.instance_attribute_states.clone(),
place_states: self.place_states.clone(),
scope_start_visibility: self.scope_start_visibility,
reachability: self.reachability,
}
}
/// Restore the current builder symbols state to the given snapshot.
/// Restore the current builder places state to the given snapshot.
pub(super) fn restore(&mut self, snapshot: FlowSnapshot) {
// We never remove symbols from `symbol_states` (it's an IndexVec, and the symbol
// IDs must line up), so the current number of known symbols must always be equal to or
// greater than the number of known symbols in a previously-taken snapshot.
let num_symbols = self.symbol_states.len();
debug_assert!(num_symbols >= snapshot.symbol_states.len());
let num_attributes = self.instance_attribute_states.len();
debug_assert!(num_attributes >= snapshot.instance_attribute_states.len());
// We never remove places from `place_states` (it's an IndexVec, and the place
// IDs must line up), so the current number of known places must always be equal to or
// greater than the number of known places in a previously-taken snapshot.
let num_places = self.place_states.len();
debug_assert!(num_places >= snapshot.place_states.len());
// Restore the current visible-definitions state to the given snapshot.
self.symbol_states = snapshot.symbol_states;
self.instance_attribute_states = snapshot.instance_attribute_states;
self.place_states = snapshot.place_states;
self.scope_start_visibility = snapshot.scope_start_visibility;
self.reachability = snapshot.reachability;
// If the snapshot we are restoring is missing some symbols we've recorded since, we need
// to fill them in so the symbol IDs continue to line up. Since they don't exist in the
// If the snapshot we are restoring is missing some places we've recorded since, we need
// to fill them in so the place IDs continue to line up. Since they don't exist in the
// snapshot, the correct state to fill them in with is "undefined".
self.symbol_states.resize(
num_symbols,
SymbolState::undefined(self.scope_start_visibility),
);
self.instance_attribute_states.resize(
num_attributes,
SymbolState::undefined(self.scope_start_visibility),
self.place_states.resize(
num_places,
PlaceState::undefined(self.scope_start_visibility),
);
}
/// Merge the given snapshot into the current state, reflecting that we might have taken either
/// path to get here. The new state for each symbol should include definitions from both the
/// path to get here. The new state for each place should include definitions from both the
/// prior state and the snapshot.
pub(super) fn merge(&mut self, snapshot: FlowSnapshot) {
// As an optimization, if we know statically that either of the snapshots is always
@@ -1089,16 +1070,13 @@ impl<'db> UseDefMapBuilder<'db> {
return;
}
// We never remove symbols from `symbol_states` (it's an IndexVec, and the symbol
// IDs must line up), so the current number of known symbols must always be equal to or
// greater than the number of known symbols in a previously-taken snapshot.
debug_assert!(self.symbol_states.len() >= snapshot.symbol_states.len());
debug_assert!(
self.instance_attribute_states.len() >= snapshot.instance_attribute_states.len()
);
// We never remove places from `place_states` (it's an IndexVec, and the place
// IDs must line up), so the current number of known places must always be equal to or
// greater than the number of known places in a previously-taken snapshot.
debug_assert!(self.place_states.len() >= snapshot.place_states.len());
let mut snapshot_definitions_iter = snapshot.symbol_states.into_iter();
for current in &mut self.symbol_states {
let mut snapshot_definitions_iter = snapshot.place_states.into_iter();
for current in &mut self.place_states {
if let Some(snapshot) = snapshot_definitions_iter.next() {
current.merge(
snapshot,
@@ -1107,27 +1085,11 @@ impl<'db> UseDefMapBuilder<'db> {
);
} else {
current.merge(
SymbolState::undefined(snapshot.scope_start_visibility),
&mut self.narrowing_constraints,
&mut self.visibility_constraints,
);
// Symbol not present in snapshot, so it's unbound/undeclared from that path.
}
}
let mut snapshot_definitions_iter = snapshot.instance_attribute_states.into_iter();
for current in &mut self.instance_attribute_states {
if let Some(snapshot) = snapshot_definitions_iter.next() {
current.merge(
snapshot,
&mut self.narrowing_constraints,
&mut self.visibility_constraints,
);
} else {
current.merge(
SymbolState::undefined(snapshot.scope_start_visibility),
PlaceState::undefined(snapshot.scope_start_visibility),
&mut self.narrowing_constraints,
&mut self.visibility_constraints,
);
// Place not present in snapshot, so it's unbound/undeclared from that path.
}
}
@@ -1142,8 +1104,7 @@ impl<'db> UseDefMapBuilder<'db> {
pub(super) fn finish(mut self) -> UseDefMap<'db> {
self.all_definitions.shrink_to_fit();
self.symbol_states.shrink_to_fit();
self.instance_attribute_states.shrink_to_fit();
self.place_states.shrink_to_fit();
self.bindings_by_use.shrink_to_fit();
self.node_reachability.shrink_to_fit();
self.declarations_by_binding.shrink_to_fit();
@@ -1157,8 +1118,7 @@ impl<'db> UseDefMapBuilder<'db> {
visibility_constraints: self.visibility_constraints.build(),
bindings_by_use: self.bindings_by_use,
node_reachability: self.node_reachability,
public_symbols: self.symbol_states,
instance_attributes: self.instance_attribute_states,
public_places: self.place_states,
declarations_by_binding: self.declarations_by_binding,
bindings_by_declaration: self.bindings_by_declaration,
eager_snapshots: self.eager_snapshots,

View File

@@ -1,4 +1,4 @@
//! Track live bindings per symbol, applicable constraints per binding, and live declarations.
//! Track live bindings per place, applicable constraints per binding, and live declarations.
//!
//! These data structures operate entirely on scope-local newtype-indices for definitions and
//! constraints, referring to their location in the `all_definitions` and `all_constraints`
@@ -60,9 +60,9 @@ pub(super) struct ScopedDefinitionId;
impl ScopedDefinitionId {
/// A special ID that is used to describe an implicit start-of-scope state. When
/// we see that this definition is live, we know that the symbol is (possibly)
/// we see that this definition is live, we know that the place is (possibly)
/// unbound or undeclared at a given usage site.
/// When creating a use-def-map builder, we always add an empty `None` definition
/// When creating a use-def-map builder, we always add an empty `DefinitionState::Undefined` definition
/// at index 0, so this ID is always present.
pub(super) const UNBOUND: ScopedDefinitionId = ScopedDefinitionId::from_u32(0);
@@ -71,19 +71,19 @@ impl ScopedDefinitionId {
}
}
/// Can keep inline this many live bindings or declarations per symbol at a given time; more will
/// Can keep inline this many live bindings or declarations per place at a given time; more will
/// go to heap.
const INLINE_DEFINITIONS_PER_SYMBOL: usize = 4;
const INLINE_DEFINITIONS_PER_PLACE: usize = 4;
/// Live declarations for a single symbol at some point in control flow, with their
/// Live declarations for a single place at some point in control flow, with their
/// corresponding visibility constraints.
#[derive(Clone, Debug, Default, PartialEq, Eq, salsa::Update)]
pub(super) struct SymbolDeclarations {
/// A list of live declarations for this symbol, sorted by their `ScopedDefinitionId`
live_declarations: SmallVec<[LiveDeclaration; INLINE_DEFINITIONS_PER_SYMBOL]>,
pub(super) struct Declarations {
/// A list of live declarations for this place, sorted by their `ScopedDefinitionId`
live_declarations: SmallVec<[LiveDeclaration; INLINE_DEFINITIONS_PER_PLACE]>,
}
/// One of the live declarations for a single symbol at some point in control flow.
/// One of the live declarations for a single place at some point in control flow.
#[derive(Clone, Debug, PartialEq, Eq)]
pub(super) struct LiveDeclaration {
pub(super) declaration: ScopedDefinitionId,
@@ -92,7 +92,7 @@ pub(super) struct LiveDeclaration {
pub(super) type LiveDeclarationsIterator<'a> = std::slice::Iter<'a, LiveDeclaration>;
impl SymbolDeclarations {
impl Declarations {
fn undeclared(scope_start_visibility: ScopedVisibilityConstraintId) -> Self {
let initial_declaration = LiveDeclaration {
declaration: ScopedDefinitionId::UNBOUND,
@@ -103,7 +103,7 @@ impl SymbolDeclarations {
}
}
/// Record a newly-encountered declaration for this symbol.
/// Record a newly-encountered declaration for this place.
fn record_declaration(&mut self, declaration: ScopedDefinitionId) {
// The new declaration replaces all previous live declaration in this path.
self.live_declarations.clear();
@@ -125,17 +125,17 @@ impl SymbolDeclarations {
}
}
/// Return an iterator over live declarations for this symbol.
/// Return an iterator over live declarations for this place.
pub(super) fn iter(&self) -> LiveDeclarationsIterator<'_> {
self.live_declarations.iter()
}
/// Iterate over the IDs of each currently live declaration for this symbol
/// Iterate over the IDs of each currently live declaration for this place
fn iter_declarations(&self) -> impl Iterator<Item = ScopedDefinitionId> + '_ {
self.iter().map(|lb| lb.declaration)
}
fn simplify_visibility_constraints(&mut self, other: SymbolDeclarations) {
fn simplify_visibility_constraints(&mut self, other: Declarations) {
// If the set of live declarations hasn't changed, don't simplify.
if self.live_declarations.len() != other.live_declarations.len()
|| !self.iter_declarations().eq(other.iter_declarations())
@@ -181,7 +181,7 @@ impl SymbolDeclarations {
}
}
/// A snapshot of a symbol state that can be used to resolve a reference in a nested eager scope.
/// A snapshot of a place state that can be used to resolve a reference in a nested eager scope.
/// If there are bindings in a (non-class) scope , they are stored in `Bindings`.
/// Even if it's a class scope (class variables are not visible to nested scopes) or there are no
/// bindings, the current narrowing constraint is necessary for narrowing, so it's stored in
@@ -189,34 +189,30 @@ impl SymbolDeclarations {
#[derive(Clone, Debug, PartialEq, Eq, salsa::Update)]
pub(super) enum EagerSnapshot {
Constraint(ScopedNarrowingConstraint),
Bindings(SymbolBindings),
Bindings(Bindings),
}
/// Live bindings for a single symbol at some point in control flow. Each live binding comes
/// Live bindings for a single place at some point in control flow. Each live binding comes
/// with a set of narrowing constraints and a visibility constraint.
#[derive(Clone, Debug, Default, PartialEq, Eq, salsa::Update)]
pub(super) struct SymbolBindings {
pub(super) struct Bindings {
/// The narrowing constraint applicable to the "unbound" binding, if we need access to it even
/// when it's not visible. This happens in class scopes, where local bindings are not visible
/// when it's not visible. This happens in class scopes, where local name bindings are not visible
/// to nested scopes, but we still need to know what narrowing constraints were applied to the
/// "unbound" binding.
unbound_narrowing_constraint: Option<ScopedNarrowingConstraint>,
/// A list of live bindings for this symbol, sorted by their `ScopedDefinitionId`
live_bindings: SmallVec<[LiveBinding; INLINE_DEFINITIONS_PER_SYMBOL]>,
/// A list of live bindings for this place, sorted by their `ScopedDefinitionId`
live_bindings: SmallVec<[LiveBinding; INLINE_DEFINITIONS_PER_PLACE]>,
}
impl SymbolBindings {
impl Bindings {
pub(super) fn unbound_narrowing_constraint(&self) -> ScopedNarrowingConstraint {
debug_assert!(
self.unbound_narrowing_constraint.is_some()
|| self.live_bindings[0].binding.is_unbound()
);
self.unbound_narrowing_constraint
.unwrap_or(self.live_bindings[0].narrowing_constraint)
}
}
/// One of the live bindings for a single symbol at some point in control flow.
/// One of the live bindings for a single place at some point in control flow.
#[derive(Clone, Debug, PartialEq, Eq)]
pub(super) struct LiveBinding {
pub(super) binding: ScopedDefinitionId,
@@ -226,7 +222,7 @@ pub(super) struct LiveBinding {
pub(super) type LiveBindingsIterator<'a> = std::slice::Iter<'a, LiveBinding>;
impl SymbolBindings {
impl Bindings {
fn unbound(scope_start_visibility: ScopedVisibilityConstraintId) -> Self {
let initial_binding = LiveBinding {
binding: ScopedDefinitionId::UNBOUND,
@@ -239,16 +235,17 @@ impl SymbolBindings {
}
}
/// Record a newly-encountered binding for this symbol.
/// Record a newly-encountered binding for this place.
pub(super) fn record_binding(
&mut self,
binding: ScopedDefinitionId,
visibility_constraint: ScopedVisibilityConstraintId,
is_class_scope: bool,
is_place_name: bool,
) {
// If we are in a class scope, and the unbound binding was previously visible, but we will
// If we are in a class scope, and the unbound name binding was previously visible, but we will
// now replace it, record the narrowing constraints on it:
if is_class_scope && self.live_bindings[0].binding.is_unbound() {
if is_class_scope && is_place_name && self.live_bindings[0].binding.is_unbound() {
self.unbound_narrowing_constraint = Some(self.live_bindings[0].narrowing_constraint);
}
// The new binding replaces all previous live bindings in this path, and has no
@@ -285,17 +282,17 @@ impl SymbolBindings {
}
}
/// Iterate over currently live bindings for this symbol
/// Iterate over currently live bindings for this place
pub(super) fn iter(&self) -> LiveBindingsIterator<'_> {
self.live_bindings.iter()
}
/// Iterate over the IDs of each currently live binding for this symbol
/// Iterate over the IDs of each currently live binding for this place
fn iter_bindings(&self) -> impl Iterator<Item = ScopedDefinitionId> + '_ {
self.iter().map(|lb| lb.binding)
}
fn simplify_visibility_constraints(&mut self, other: SymbolBindings) {
fn simplify_visibility_constraints(&mut self, other: Bindings) {
// If the set of live bindings hasn't changed, don't simplify.
if self.live_bindings.len() != other.live_bindings.len()
|| !self.iter_bindings().eq(other.iter_bindings())
@@ -360,30 +357,35 @@ impl SymbolBindings {
}
#[derive(Clone, Debug, PartialEq, Eq)]
pub(in crate::semantic_index) struct SymbolState {
declarations: SymbolDeclarations,
bindings: SymbolBindings,
pub(in crate::semantic_index) struct PlaceState {
declarations: Declarations,
bindings: Bindings,
}
impl SymbolState {
/// Return a new [`SymbolState`] representing an unbound, undeclared symbol.
impl PlaceState {
/// Return a new [`PlaceState`] representing an unbound, undeclared place.
pub(super) fn undefined(scope_start_visibility: ScopedVisibilityConstraintId) -> Self {
Self {
declarations: SymbolDeclarations::undeclared(scope_start_visibility),
bindings: SymbolBindings::unbound(scope_start_visibility),
declarations: Declarations::undeclared(scope_start_visibility),
bindings: Bindings::unbound(scope_start_visibility),
}
}
/// Record a newly-encountered binding for this symbol.
/// Record a newly-encountered binding for this place.
pub(super) fn record_binding(
&mut self,
binding_id: ScopedDefinitionId,
visibility_constraint: ScopedVisibilityConstraintId,
is_class_scope: bool,
is_place_name: bool,
) {
debug_assert_ne!(binding_id, ScopedDefinitionId::UNBOUND);
self.bindings
.record_binding(binding_id, visibility_constraint, is_class_scope);
self.bindings.record_binding(
binding_id,
visibility_constraint,
is_class_scope,
is_place_name,
);
}
/// Add given constraint to all live bindings.
@@ -409,24 +411,24 @@ impl SymbolState {
}
/// Simplifies this snapshot to have the same visibility constraints as a previous point in the
/// control flow, but only if the set of live bindings or declarations for this symbol hasn't
/// control flow, but only if the set of live bindings or declarations for this place hasn't
/// changed.
pub(super) fn simplify_visibility_constraints(&mut self, snapshot_state: SymbolState) {
pub(super) fn simplify_visibility_constraints(&mut self, snapshot_state: PlaceState) {
self.bindings
.simplify_visibility_constraints(snapshot_state.bindings);
self.declarations
.simplify_visibility_constraints(snapshot_state.declarations);
}
/// Record a newly-encountered declaration of this symbol.
/// Record a newly-encountered declaration of this place.
pub(super) fn record_declaration(&mut self, declaration_id: ScopedDefinitionId) {
self.declarations.record_declaration(declaration_id);
}
/// Merge another [`SymbolState`] into this one.
/// Merge another [`PlaceState`] into this one.
pub(super) fn merge(
&mut self,
b: SymbolState,
b: PlaceState,
narrowing_constraints: &mut NarrowingConstraintsBuilder,
visibility_constraints: &mut VisibilityConstraintsBuilder,
) {
@@ -436,11 +438,11 @@ impl SymbolState {
.merge(b.declarations, visibility_constraints);
}
pub(super) fn bindings(&self) -> &SymbolBindings {
pub(super) fn bindings(&self) -> &Bindings {
&self.bindings
}
pub(super) fn declarations(&self) -> &SymbolDeclarations {
pub(super) fn declarations(&self) -> &Declarations {
&self.declarations
}
}
@@ -454,10 +456,10 @@ mod tests {
#[track_caller]
fn assert_bindings(
narrowing_constraints: &NarrowingConstraintsBuilder,
symbol: &SymbolState,
place: &PlaceState,
expected: &[&str],
) {
let actual = symbol
let actual = place
.bindings()
.iter()
.map(|live_binding| {
@@ -479,8 +481,8 @@ mod tests {
}
#[track_caller]
pub(crate) fn assert_declarations(symbol: &SymbolState, expected: &[&str]) {
let actual = symbol
pub(crate) fn assert_declarations(place: &PlaceState, expected: &[&str]) {
let actual = place
.declarations()
.iter()
.map(
@@ -502,7 +504,7 @@ mod tests {
#[test]
fn unbound() {
let narrowing_constraints = NarrowingConstraintsBuilder::default();
let sym = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
let sym = PlaceState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
assert_bindings(&narrowing_constraints, &sym, &["unbound<>"]);
}
@@ -510,11 +512,12 @@ mod tests {
#[test]
fn with() {
let narrowing_constraints = NarrowingConstraintsBuilder::default();
let mut sym = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
let mut sym = PlaceState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
sym.record_binding(
ScopedDefinitionId::from_u32(1),
ScopedVisibilityConstraintId::ALWAYS_TRUE,
false,
true,
);
assert_bindings(&narrowing_constraints, &sym, &["1<>"]);
@@ -523,11 +526,12 @@ mod tests {
#[test]
fn record_constraint() {
let mut narrowing_constraints = NarrowingConstraintsBuilder::default();
let mut sym = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
let mut sym = PlaceState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
sym.record_binding(
ScopedDefinitionId::from_u32(1),
ScopedVisibilityConstraintId::ALWAYS_TRUE,
false,
true,
);
let predicate = ScopedPredicateId::from_u32(0).into();
sym.record_narrowing_constraint(&mut narrowing_constraints, predicate);
@@ -541,20 +545,22 @@ mod tests {
let mut visibility_constraints = VisibilityConstraintsBuilder::default();
// merging the same definition with the same constraint keeps the constraint
let mut sym1a = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
let mut sym1a = PlaceState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
sym1a.record_binding(
ScopedDefinitionId::from_u32(1),
ScopedVisibilityConstraintId::ALWAYS_TRUE,
false,
true,
);
let predicate = ScopedPredicateId::from_u32(0).into();
sym1a.record_narrowing_constraint(&mut narrowing_constraints, predicate);
let mut sym1b = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
let mut sym1b = PlaceState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
sym1b.record_binding(
ScopedDefinitionId::from_u32(1),
ScopedVisibilityConstraintId::ALWAYS_TRUE,
false,
true,
);
let predicate = ScopedPredicateId::from_u32(0).into();
sym1b.record_narrowing_constraint(&mut narrowing_constraints, predicate);
@@ -568,20 +574,22 @@ mod tests {
assert_bindings(&narrowing_constraints, &sym1, &["1<0>"]);
// merging the same definition with differing constraints drops all constraints
let mut sym2a = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
let mut sym2a = PlaceState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
sym2a.record_binding(
ScopedDefinitionId::from_u32(2),
ScopedVisibilityConstraintId::ALWAYS_TRUE,
false,
true,
);
let predicate = ScopedPredicateId::from_u32(1).into();
sym2a.record_narrowing_constraint(&mut narrowing_constraints, predicate);
let mut sym1b = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
let mut sym1b = PlaceState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
sym1b.record_binding(
ScopedDefinitionId::from_u32(2),
ScopedVisibilityConstraintId::ALWAYS_TRUE,
false,
true,
);
let predicate = ScopedPredicateId::from_u32(2).into();
sym1b.record_narrowing_constraint(&mut narrowing_constraints, predicate);
@@ -595,16 +603,17 @@ mod tests {
assert_bindings(&narrowing_constraints, &sym2, &["2<>"]);
// merging a constrained definition with unbound keeps both
let mut sym3a = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
let mut sym3a = PlaceState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
sym3a.record_binding(
ScopedDefinitionId::from_u32(3),
ScopedVisibilityConstraintId::ALWAYS_TRUE,
false,
true,
);
let predicate = ScopedPredicateId::from_u32(3).into();
sym3a.record_narrowing_constraint(&mut narrowing_constraints, predicate);
let sym2b = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
let sym2b = PlaceState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
sym3a.merge(
sym2b,
@@ -626,14 +635,14 @@ mod tests {
#[test]
fn no_declaration() {
let sym = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
let sym = PlaceState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
assert_declarations(&sym, &["undeclared"]);
}
#[test]
fn record_declaration() {
let mut sym = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
let mut sym = PlaceState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
sym.record_declaration(ScopedDefinitionId::from_u32(1));
assert_declarations(&sym, &["1"]);
@@ -641,7 +650,7 @@ mod tests {
#[test]
fn record_declaration_override() {
let mut sym = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
let mut sym = PlaceState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
sym.record_declaration(ScopedDefinitionId::from_u32(1));
sym.record_declaration(ScopedDefinitionId::from_u32(2));
@@ -652,10 +661,10 @@ mod tests {
fn record_declaration_merge() {
let mut narrowing_constraints = NarrowingConstraintsBuilder::default();
let mut visibility_constraints = VisibilityConstraintsBuilder::default();
let mut sym = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
let mut sym = PlaceState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
sym.record_declaration(ScopedDefinitionId::from_u32(1));
let mut sym2 = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
let mut sym2 = PlaceState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
sym2.record_declaration(ScopedDefinitionId::from_u32(2));
sym.merge(
@@ -671,10 +680,10 @@ mod tests {
fn record_declaration_merge_partial_undeclared() {
let mut narrowing_constraints = NarrowingConstraintsBuilder::default();
let mut visibility_constraints = VisibilityConstraintsBuilder::default();
let mut sym = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
let mut sym = PlaceState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
sym.record_declaration(ScopedDefinitionId::from_u32(1));
let sym2 = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
let sym2 = PlaceState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
sym.merge(
sym2,

View File

@@ -180,12 +180,12 @@ use rustc_hash::FxHashMap;
use crate::Db;
use crate::dunder_all::dunder_all_names;
use crate::place::{RequiresExplicitReExport, imported_symbol};
use crate::semantic_index::expression::Expression;
use crate::semantic_index::place_table;
use crate::semantic_index::predicate::{
PatternPredicate, PatternPredicateKind, Predicate, PredicateNode, Predicates, ScopedPredicateId,
};
use crate::semantic_index::symbol_table;
use crate::symbol::{RequiresExplicitReExport, imported_symbol};
use crate::types::{Truthiness, Type, infer_expression_type};
/// A ternary formula that defines under what conditions a binding is visible. (A ternary formula
@@ -654,8 +654,10 @@ impl VisibilityConstraints {
}
PredicateNode::Pattern(inner) => Self::analyze_single_pattern_predicate(db, inner),
PredicateNode::StarImportPlaceholder(star_import) => {
let symbol_table = symbol_table(db, star_import.scope(db));
let symbol_name = symbol_table.symbol(star_import.symbol_id(db)).name();
let place_table = place_table(db, star_import.scope(db));
let symbol_name = place_table
.place_expr(star_import.symbol_id(db))
.expect_name();
let referenced_file = star_import.referenced_file(db);
let requires_explicit_reexport = match dunder_all_names(db, referenced_file) {
@@ -675,15 +677,15 @@ impl VisibilityConstraints {
};
match imported_symbol(db, referenced_file, symbol_name, requires_explicit_reexport)
.symbol
.place
{
crate::symbol::Symbol::Type(_, crate::symbol::Boundness::Bound) => {
crate::place::Place::Type(_, crate::place::Boundness::Bound) => {
Truthiness::AlwaysTrue
}
crate::symbol::Symbol::Type(_, crate::symbol::Boundness::PossiblyUnbound) => {
crate::place::Place::Type(_, crate::place::Boundness::PossiblyUnbound) => {
Truthiness::Ambiguous
}
crate::symbol::Symbol::Unbound => Truthiness::AlwaysFalse,
crate::place::Place::Unbound => Truthiness::AlwaysFalse,
}
}
}

View File

@@ -8,8 +8,9 @@ use crate::Db;
use crate::module_name::ModuleName;
use crate::module_resolver::{Module, resolve_module};
use crate::semantic_index::ast_ids::HasScopedExpressionId;
use crate::semantic_index::place::FileScopeId;
use crate::semantic_index::semantic_index;
use crate::semantic_index::symbol::FileScopeId;
use crate::types::ide_support::all_declarations_and_bindings;
use crate::types::{Type, binding_type, infer_scope_types};
pub struct SemanticModel<'db> {
@@ -40,12 +41,18 @@ impl<'db> SemanticModel<'db> {
resolve_module(self.db, module_name)
}
/// Returns completions for symbols available in a `object.<CURSOR>` context.
pub fn attribute_completions(&self, node: &ast::ExprAttribute) -> Vec<Name> {
let ty = node.value.inferred_type(self);
crate::types::all_members(self.db, ty).into_iter().collect()
}
/// Returns completions for symbols available in the scope containing the
/// given expression.
///
/// If a scope could not be determined, then completions for the global
/// scope of this model's `File` are returned.
pub fn completions(&self, node: ast::AnyNodeRef<'_>) -> Vec<Name> {
pub fn scoped_completions(&self, node: ast::AnyNodeRef<'_>) -> Vec<Name> {
let index = semantic_index(self.db, self.file);
// TODO: We currently use `try_expression_scope_id` here as a hotfix for [1].
@@ -66,9 +73,10 @@ impl<'db> SemanticModel<'db> {
};
let mut symbols = vec![];
for (file_scope, _) in index.ancestor_scopes(file_scope) {
for symbol in index.symbol_table(file_scope).symbols() {
symbols.push(symbol.name().clone());
}
symbols.extend(all_declarations_and_bindings(
self.db,
file_scope.to_scope_id(self.db, self.file),
));
}
symbols
}

View File

@@ -536,10 +536,18 @@ pub(crate) enum SitePackagesDiscoveryError {
#[error("Invalid {1}: `{0}` could not be canonicalized")]
CanonicalizationError(SystemPathBuf, SysPrefixPathOrigin, #[source] io::Error),
/// `site-packages` discovery failed because the [`SysPrefixPathOrigin`] indicated that
/// the provided path should point to `sys.prefix` directly, but the path wasn't a directory.
#[error("Invalid {1}: `{0}` does not point to a directory on disk")]
SysPrefixNotADirectory(SystemPathBuf, SysPrefixPathOrigin),
/// `site-packages` discovery failed because the provided path doesn't appear to point to
/// a Python executable or a `sys.prefix` directory.
#[error(
"Invalid {1}: `{0}` does not point to a {thing}",
thing = if .1.must_point_directly_to_sys_prefix() {
"directory on disk"
} else {
"Python executable or a directory on disk"
}
)]
PathNotExecutableOrDirectory(SystemPathBuf, SysPrefixPathOrigin),
/// `site-packages` discovery failed because the [`SysPrefixPathOrigin`] indicated that
/// the provided path should point to the `sys.prefix` of a virtual environment,
@@ -738,24 +746,79 @@ impl SysPrefixPath {
let canonicalized = system
.canonicalize_path(unvalidated_path)
.map_err(|io_err| {
SitePackagesDiscoveryError::CanonicalizationError(
unvalidated_path.to_path_buf(),
origin,
io_err,
)
let unvalidated_path = unvalidated_path.to_path_buf();
if io_err.kind() == io::ErrorKind::NotFound {
SitePackagesDiscoveryError::PathNotExecutableOrDirectory(
unvalidated_path,
origin,
)
} else {
SitePackagesDiscoveryError::CanonicalizationError(
unvalidated_path,
origin,
io_err,
)
}
})?;
system
.is_directory(&canonicalized)
.then_some(Self {
inner: canonicalized,
origin,
})
.ok_or_else(|| {
SitePackagesDiscoveryError::SysPrefixNotADirectory(
if origin.must_point_directly_to_sys_prefix() {
return system
.is_directory(&canonicalized)
.then_some(Self {
inner: canonicalized,
origin,
})
.ok_or_else(|| {
SitePackagesDiscoveryError::PathNotExecutableOrDirectory(
unvalidated_path.to_path_buf(),
origin,
)
});
}
let sys_prefix = if system.is_file(&canonicalized)
&& canonicalized
.file_name()
.is_some_and(|name| name.starts_with("python"))
{
// It looks like they passed us a path to a Python executable, e.g. `.venv/bin/python3`.
// Try to figure out the `sys.prefix` value from the Python executable.
let sys_prefix = if cfg!(windows) {
// On Windows, the relative path to the Python executable from `sys.prefix`
// is different depending on whether it's a virtual environment or a system installation.
// System installations have their executable at `<sys.prefix>/python.exe`,
// whereas virtual environments have their executable at `<sys.prefix>/Scripts/python.exe`.
canonicalized.parent().and_then(|parent| {
if parent.file_name() == Some("Scripts") {
parent.parent()
} else {
Some(parent)
}
})
} else {
// On Unix, `sys.prefix` is always the grandparent directory of the Python executable,
// regardless of whether it's a virtual environment or a system installation.
canonicalized.ancestors().nth(2)
};
sys_prefix.map(SystemPath::to_path_buf).ok_or_else(|| {
SitePackagesDiscoveryError::PathNotExecutableOrDirectory(
unvalidated_path.to_path_buf(),
origin,
)
})
})?
} else if system.is_directory(&canonicalized) {
canonicalized
} else {
return Err(SitePackagesDiscoveryError::PathNotExecutableOrDirectory(
unvalidated_path.to_path_buf(),
origin,
));
};
Ok(Self {
inner: sys_prefix,
origin,
})
}
fn from_executable_home_path(path: &PythonHomePath) -> Option<Self> {
@@ -812,12 +875,26 @@ pub enum SysPrefixPathOrigin {
impl SysPrefixPathOrigin {
/// Whether the given `sys.prefix` path must be a virtual environment (rather than a system
/// Python environment).
pub(crate) fn must_be_virtual_env(self) -> bool {
pub(crate) const fn must_be_virtual_env(self) -> bool {
match self {
Self::LocalVenv | Self::VirtualEnvVar => true,
Self::PythonCliFlag | Self::DerivedFromPyvenvCfg | Self::CondaPrefixVar => false,
}
}
/// Whether paths with this origin always point directly to the `sys.prefix` directory.
///
/// Some variants can point either directly to `sys.prefix` or to a Python executable inside
/// the `sys.prefix` directory, e.g. the `--python` CLI flag.
pub(crate) const fn must_point_directly_to_sys_prefix(self) -> bool {
match self {
Self::PythonCliFlag => false,
Self::VirtualEnvVar
| Self::CondaPrefixVar
| Self::DerivedFromPyvenvCfg
| Self::LocalVenv => true,
}
}
}
impl Display for SysPrefixPathOrigin {
@@ -1378,7 +1455,7 @@ mod tests {
let system = TestSystem::default();
assert!(matches!(
PythonEnvironment::new("/env", SysPrefixPathOrigin::PythonCliFlag, &system),
Err(SitePackagesDiscoveryError::CanonicalizationError(..))
Err(SitePackagesDiscoveryError::PathNotExecutableOrDirectory(..))
));
}
@@ -1391,7 +1468,7 @@ mod tests {
.unwrap();
assert!(matches!(
PythonEnvironment::new("/env", SysPrefixPathOrigin::PythonCliFlag, &system),
Err(SitePackagesDiscoveryError::SysPrefixNotADirectory(..))
Err(SitePackagesDiscoveryError::PathNotExecutableOrDirectory(..))
));
}

File diff suppressed because it is too large Load Diff

View File

@@ -12,7 +12,7 @@ use super::{
};
use crate::db::Db;
use crate::dunder_all::dunder_all_names;
use crate::symbol::{Boundness, Symbol};
use crate::place::{Boundness, Place};
use crate::types::diagnostic::{
CALL_NON_CALLABLE, CONFLICTING_ARGUMENT_FORMS, INVALID_ARGUMENT_TYPE, MISSING_ARGUMENT,
NO_MATCHING_OVERLOAD, PARAMETER_ALREADY_ASSIGNED, TOO_MANY_POSITIONAL_ARGUMENTS,
@@ -770,7 +770,7 @@ impl<'db> Bindings<'db> {
// TODO: we could emit a diagnostic here (if default is not set)
overload.set_return_type(
match instance_ty.static_member(db, attr_name.value(db)) {
Symbol::Type(ty, Boundness::Bound) => {
Place::Type(ty, Boundness::Bound) => {
if instance_ty.is_fully_static(db) {
ty
} else {
@@ -782,10 +782,10 @@ impl<'db> Bindings<'db> {
union_with_default(ty)
}
}
Symbol::Type(ty, Boundness::PossiblyUnbound) => {
Place::Type(ty, Boundness::PossiblyUnbound) => {
union_with_default(ty)
}
Symbol::Unbound => default,
Place::Unbound => default,
},
);
}

View File

@@ -7,7 +7,7 @@ use super::{
infer_unpack_types,
};
use crate::semantic_index::DeclarationWithConstraint;
use crate::semantic_index::definition::Definition;
use crate::semantic_index::definition::{Definition, DefinitionState};
use crate::types::function::{DataclassTransformerParams, KnownFunction};
use crate::types::generics::{GenericContext, Specialization};
use crate::types::signatures::{CallableSignature, Parameter, Parameters, Signature};
@@ -17,17 +17,16 @@ use crate::types::{
use crate::{
Db, FxOrderSet, KnownModule, Program,
module_resolver::file_to_module,
place::{
Boundness, LookupError, LookupResult, Place, PlaceAndQualifiers, class_symbol,
known_module_symbol, place_from_bindings, place_from_declarations,
},
semantic_index::{
ast_ids::HasScopedExpressionId,
attribute_assignments,
definition::{DefinitionKind, TargetKind},
semantic_index,
symbol::ScopeId,
symbol_table, use_def_map,
},
symbol::{
Boundness, LookupError, LookupResult, Symbol, SymbolAndQualifiers, class_symbol,
known_module_symbol, symbol_from_bindings, symbol_from_declarations,
place::ScopeId,
place_table, semantic_index, use_def_map,
},
types::{
CallArgumentTypes, CallError, CallErrorKind, DynamicType, MetaclassCandidate, TupleType,
@@ -454,7 +453,7 @@ impl<'db> ClassType<'db> {
db: &'db dyn Db,
name: &str,
policy: MemberLookupPolicy,
) -> SymbolAndQualifiers<'db> {
) -> PlaceAndQualifiers<'db> {
let (class_literal, specialization) = self.class_literal(db);
class_literal.class_member_inner(db, specialization, name, policy)
}
@@ -462,10 +461,10 @@ impl<'db> ClassType<'db> {
/// Returns the inferred type of the class member named `name`. Only bound members
/// or those marked as ClassVars are considered.
///
/// Returns [`Symbol::Unbound`] if `name` cannot be found in this class's scope
/// Returns [`Place::Unbound`] if `name` cannot be found in this class's scope
/// directly. Use [`ClassType::class_member`] if you require a method that will
/// traverse through the MRO until it finds the member.
pub(super) fn own_class_member(self, db: &'db dyn Db, name: &str) -> SymbolAndQualifiers<'db> {
pub(super) fn own_class_member(self, db: &'db dyn Db, name: &str) -> PlaceAndQualifiers<'db> {
let (class_literal, specialization) = self.class_literal(db);
class_literal
.own_class_member(db, specialization, name)
@@ -475,7 +474,7 @@ impl<'db> ClassType<'db> {
/// Look up an instance attribute (available in `__dict__`) of the given name.
///
/// See [`Type::instance_member`] for more details.
pub(super) fn instance_member(self, db: &'db dyn Db, name: &str) -> SymbolAndQualifiers<'db> {
pub(super) fn instance_member(self, db: &'db dyn Db, name: &str) -> PlaceAndQualifiers<'db> {
let (class_literal, specialization) = self.class_literal(db);
class_literal
.instance_member(db, specialization, name)
@@ -484,7 +483,7 @@ impl<'db> ClassType<'db> {
/// A helper function for `instance_member` that looks up the `name` attribute only on
/// this class, not on its superclasses.
fn own_instance_member(self, db: &'db dyn Db, name: &str) -> SymbolAndQualifiers<'db> {
fn own_instance_member(self, db: &'db dyn Db, name: &str) -> PlaceAndQualifiers<'db> {
let (class_literal, specialization) = self.class_literal(db);
class_literal
.own_instance_member(db, name)
@@ -502,9 +501,9 @@ impl<'db> ClassType<'db> {
MemberLookupPolicy::NO_INSTANCE_FALLBACK
| MemberLookupPolicy::META_CLASS_NO_TYPE_FALLBACK,
)
.symbol;
.place;
if let Symbol::Type(Type::BoundMethod(metaclass_dunder_call_function), _) =
if let Place::Type(Type::BoundMethod(metaclass_dunder_call_function), _) =
metaclass_dunder_call_function_symbol
{
// TODO: this intentionally diverges from step 1 in
@@ -520,10 +519,10 @@ impl<'db> ClassType<'db> {
"__new__".into(),
MemberLookupPolicy::MRO_NO_OBJECT_FALLBACK,
)
.symbol;
.place;
let dunder_new_function =
if let Symbol::Type(Type::FunctionLiteral(dunder_new_function), _) =
if let Place::Type(Type::FunctionLiteral(dunder_new_function), _) =
dunder_new_function_symbol
{
// Step 3: If the return type of the `__new__` evaluates to a type that is not a subclass of this class,
@@ -562,7 +561,7 @@ impl<'db> ClassType<'db> {
MemberLookupPolicy::MRO_NO_OBJECT_FALLBACK
| MemberLookupPolicy::META_CLASS_NO_TYPE_FALLBACK,
)
.symbol;
.place;
let correct_return_type = self_ty.to_instance(db).unwrap_or_else(Type::unknown);
@@ -570,7 +569,7 @@ impl<'db> ClassType<'db> {
// same parameters as the `__init__` method after it is bound, and with the return type of
// the concrete type of `Self`.
let synthesized_dunder_init_callable =
if let Symbol::Type(Type::FunctionLiteral(dunder_init_function), _) =
if let Place::Type(Type::FunctionLiteral(dunder_init_function), _) =
dunder_init_function_symbol
{
let synthesized_signature = |signature: Signature<'db>| {
@@ -612,9 +611,9 @@ impl<'db> ClassType<'db> {
"__new__".into(),
MemberLookupPolicy::META_CLASS_NO_TYPE_FALLBACK,
)
.symbol;
.place;
if let Symbol::Type(Type::FunctionLiteral(new_function), _) = new_function_symbol {
if let Place::Type(Type::FunctionLiteral(new_function), _) = new_function_symbol {
new_function.into_bound_method_type(db, self_ty)
} else {
// Fallback if no `object.__new__` is found.
@@ -1136,7 +1135,7 @@ impl<'db> ClassLiteral<'db> {
db: &'db dyn Db,
name: &str,
policy: MemberLookupPolicy,
) -> SymbolAndQualifiers<'db> {
) -> PlaceAndQualifiers<'db> {
self.class_member_inner(db, None, name, policy)
}
@@ -1146,10 +1145,10 @@ impl<'db> ClassLiteral<'db> {
specialization: Option<Specialization<'db>>,
name: &str,
policy: MemberLookupPolicy,
) -> SymbolAndQualifiers<'db> {
) -> PlaceAndQualifiers<'db> {
if name == "__mro__" {
let tuple_elements = self.iter_mro(db, specialization).map(Type::from);
return Symbol::bound(TupleType::from_elements(db, tuple_elements)).into();
return Place::bound(TupleType::from_elements(db, tuple_elements)).into();
}
self.class_member_from_mro(db, name, policy, self.iter_mro(db, specialization))
@@ -1161,7 +1160,7 @@ impl<'db> ClassLiteral<'db> {
name: &str,
policy: MemberLookupPolicy,
mro_iter: impl Iterator<Item = ClassBase<'db>>,
) -> SymbolAndQualifiers<'db> {
) -> PlaceAndQualifiers<'db> {
// If we encounter a dynamic type in this class's MRO, we'll save that dynamic type
// in this variable. After we've traversed the MRO, we'll either:
// (1) Use that dynamic type as the type for this attribute,
@@ -1208,18 +1207,18 @@ impl<'db> ClassLiteral<'db> {
}
match (
SymbolAndQualifiers::from(lookup_result),
PlaceAndQualifiers::from(lookup_result),
dynamic_type_to_intersect_with,
) {
(symbol_and_qualifiers, None) => symbol_and_qualifiers,
(
SymbolAndQualifiers {
symbol: Symbol::Type(ty, _),
PlaceAndQualifiers {
place: Place::Type(ty, _),
qualifiers,
},
Some(dynamic_type),
) => Symbol::bound(
) => Place::bound(
IntersectionBuilder::new(db)
.add_positive(ty)
.add_positive(dynamic_type)
@@ -1228,19 +1227,19 @@ impl<'db> ClassLiteral<'db> {
.with_qualifiers(qualifiers),
(
SymbolAndQualifiers {
symbol: Symbol::Unbound,
PlaceAndQualifiers {
place: Place::Unbound,
qualifiers,
},
Some(dynamic_type),
) => Symbol::bound(dynamic_type).with_qualifiers(qualifiers),
) => Place::bound(dynamic_type).with_qualifiers(qualifiers),
}
}
/// Returns the inferred type of the class member named `name`. Only bound members
/// or those marked as ClassVars are considered.
///
/// Returns [`Symbol::Unbound`] if `name` cannot be found in this class's scope
/// Returns [`Place::Unbound`] if `name` cannot be found in this class's scope
/// directly. Use [`ClassLiteral::class_member`] if you require a method that will
/// traverse through the MRO until it finds the member.
pub(super) fn own_class_member(
@@ -1248,10 +1247,10 @@ impl<'db> ClassLiteral<'db> {
db: &'db dyn Db,
specialization: Option<Specialization<'db>>,
name: &str,
) -> SymbolAndQualifiers<'db> {
) -> PlaceAndQualifiers<'db> {
if name == "__dataclass_fields__" && self.dataclass_params(db).is_some() {
// Make this class look like a subclass of the `DataClassInstance` protocol
return Symbol::bound(KnownClass::Dict.to_specialized_instance(
return Place::bound(KnownClass::Dict.to_specialized_instance(
db,
[
KnownClass::Str.to_instance(db),
@@ -1287,10 +1286,10 @@ impl<'db> ClassLiteral<'db> {
}
});
if symbol.symbol.is_unbound() {
if symbol.place.is_unbound() {
if let Some(synthesized_member) = self.own_synthesized_member(db, specialization, name)
{
return Symbol::bound(synthesized_member).into();
return Place::bound(synthesized_member).into();
}
}
@@ -1322,7 +1321,7 @@ impl<'db> ClassLiteral<'db> {
// itself in this case, so we skip the special descriptor handling.
if attr_ty.is_fully_static(db) {
let dunder_set = attr_ty.class_member(db, "__set__".into());
if let Some(dunder_set) = dunder_set.symbol.ignore_possibly_unbound() {
if let Some(dunder_set) = dunder_set.place.ignore_possibly_unbound() {
// This type of this attribute is a data descriptor. Instead of overwriting the
// descriptor attribute, data-classes will (implicitly) call the `__set__` method
// of the descriptor. This means that the synthesized `__init__` parameter for
@@ -1428,7 +1427,7 @@ impl<'db> ClassLiteral<'db> {
.to_class_literal(db)
.into_class_literal()?
.own_class_member(db, None, name)
.symbol
.place
.ignore_possibly_unbound()
}
_ => None,
@@ -1490,10 +1489,10 @@ impl<'db> ClassLiteral<'db> {
let mut attributes = FxOrderMap::default();
let class_body_scope = self.body_scope(db);
let table = symbol_table(db, class_body_scope);
let table = place_table(db, class_body_scope);
let use_def = use_def_map(db, class_body_scope);
for (symbol_id, declarations) in use_def.all_public_declarations() {
for (place_id, declarations) in use_def.all_public_declarations() {
// Here, we exclude all declarations that are not annotated assignments. We need this because
// things like function definitions and nested classes would otherwise be considered dataclass
// fields. The check is too broad in the sense that it also excludes (weird) constructs where
@@ -1504,7 +1503,7 @@ impl<'db> ClassLiteral<'db> {
if !declarations
.clone()
.all(|DeclarationWithConstraint { declaration, .. }| {
declaration.is_some_and(|declaration| {
declaration.is_defined_and(|declaration| {
matches!(
declaration.kind(db),
DefinitionKind::AnnotatedAssignment(..)
@@ -1515,18 +1514,18 @@ impl<'db> ClassLiteral<'db> {
continue;
}
let symbol = table.symbol(symbol_id);
let place_expr = table.place_expr(place_id);
if let Ok(attr) = symbol_from_declarations(db, declarations) {
if let Ok(attr) = place_from_declarations(db, declarations) {
if attr.is_class_var() {
continue;
}
if let Some(attr_ty) = attr.symbol.ignore_possibly_unbound() {
let bindings = use_def.public_bindings(symbol_id);
let default_ty = symbol_from_bindings(db, bindings).ignore_possibly_unbound();
if let Some(attr_ty) = attr.place.ignore_possibly_unbound() {
let bindings = use_def.public_bindings(place_id);
let default_ty = place_from_bindings(db, bindings).ignore_possibly_unbound();
attributes.insert(symbol.name().clone(), (attr_ty, default_ty));
attributes.insert(place_expr.expect_name().clone(), (attr_ty, default_ty));
}
}
}
@@ -1542,7 +1541,7 @@ impl<'db> ClassLiteral<'db> {
db: &'db dyn Db,
specialization: Option<Specialization<'db>>,
name: &str,
) -> SymbolAndQualifiers<'db> {
) -> PlaceAndQualifiers<'db> {
let mut union = UnionBuilder::new(db);
let mut union_qualifiers = TypeQualifiers::empty();
@@ -1552,13 +1551,13 @@ impl<'db> ClassLiteral<'db> {
// Skip over these very special class bases that aren't really classes.
}
ClassBase::Dynamic(_) => {
return SymbolAndQualifiers::todo(
return PlaceAndQualifiers::todo(
"instance attribute on class with dynamic base",
);
}
ClassBase::Class(class) => {
if let member @ SymbolAndQualifiers {
symbol: Symbol::Type(ty, boundness),
if let member @ PlaceAndQualifiers {
place: Place::Type(ty, boundness),
qualifiers,
} = class.own_instance_member(db, name)
{
@@ -1571,7 +1570,7 @@ impl<'db> ClassLiteral<'db> {
return member;
}
return Symbol::bound(union.add(ty).build())
return Place::bound(union.add(ty).build())
.with_qualifiers(union_qualifiers);
}
@@ -1584,13 +1583,12 @@ impl<'db> ClassLiteral<'db> {
}
if union.is_empty() {
Symbol::Unbound.with_qualifiers(TypeQualifiers::empty())
Place::Unbound.with_qualifiers(TypeQualifiers::empty())
} else {
// If we have reached this point, we know that we have only seen possibly-unbound symbols.
// If we have reached this point, we know that we have only seen possibly-unbound places.
// This means that the final result is still possibly-unbound.
Symbol::Type(union.build(), Boundness::PossiblyUnbound)
.with_qualifiers(union_qualifiers)
Place::Type(union.build(), Boundness::PossiblyUnbound).with_qualifiers(union_qualifiers)
}
}
@@ -1600,7 +1598,7 @@ impl<'db> ClassLiteral<'db> {
db: &'db dyn Db,
class_body_scope: ScopeId<'db>,
name: &str,
) -> Symbol<'db> {
) -> Place<'db> {
// If we do not see any declarations of an attribute, neither in the class body nor in
// any method, we build a union of `Unknown` with the inferred types of all bindings of
// that attribute. We include `Unknown` in that union to account for the fact that the
@@ -1612,7 +1610,7 @@ impl<'db> ClassLiteral<'db> {
let file = class_body_scope.file(db);
let index = semantic_index(db, file);
let class_map = use_def_map(db, class_body_scope);
let class_table = symbol_table(db, class_body_scope);
let class_table = place_table(db, class_body_scope);
for (attribute_assignments, method_scope_id) in
attribute_assignments(db, class_body_scope, name)
@@ -1623,11 +1621,11 @@ impl<'db> ClassLiteral<'db> {
// The attribute assignment inherits the visibility of the method which contains it
let is_method_visible = if let Some(method_def) = method_scope.node(db).as_function() {
let method = index.expect_single_definition(method_def);
let method_symbol = class_table.symbol_id_by_name(&method_def.name).unwrap();
let method_place = class_table.place_id_by_name(&method_def.name).unwrap();
class_map
.public_bindings(method_symbol)
.public_bindings(method_place)
.find_map(|bind| {
(bind.binding == Some(method))
(bind.binding.is_defined_and(|def| def == method))
.then(|| class_map.is_binding_visible(db, &bind))
})
.unwrap_or(Truthiness::AlwaysFalse)
@@ -1642,7 +1640,7 @@ impl<'db> ClassLiteral<'db> {
let unbound_visibility = attribute_assignments
.peek()
.map(|attribute_assignment| {
if attribute_assignment.binding.is_none() {
if attribute_assignment.binding.is_undefined() {
method_map.is_binding_visible(db, attribute_assignment)
} else {
Truthiness::AlwaysFalse
@@ -1651,7 +1649,7 @@ impl<'db> ClassLiteral<'db> {
.unwrap_or(Truthiness::AlwaysFalse);
for attribute_assignment in attribute_assignments {
let Some(binding) = attribute_assignment.binding else {
let DefinitionState::Defined(binding) = attribute_assignment.binding else {
continue;
};
match method_map
@@ -1696,10 +1694,10 @@ impl<'db> ClassLiteral<'db> {
// TODO: check if there are conflicting declarations
match is_attribute_bound {
Truthiness::AlwaysTrue => {
return Symbol::bound(annotation_ty);
return Place::bound(annotation_ty);
}
Truthiness::Ambiguous => {
return Symbol::possibly_unbound(annotation_ty);
return Place::possibly_unbound(annotation_ty);
}
Truthiness::AlwaysFalse => unreachable!(
"If the attribute assignments are all invisible, inference of their types should be skipped"
@@ -1722,7 +1720,7 @@ impl<'db> ClassLiteral<'db> {
union_of_inferred_types = union_of_inferred_types.add(inferred_ty);
}
TargetKind::NameOrAttribute => {
TargetKind::Single => {
// We found an un-annotated attribute assignment of the form:
//
// self.name = <value>
@@ -1748,7 +1746,7 @@ impl<'db> ClassLiteral<'db> {
union_of_inferred_types = union_of_inferred_types.add(inferred_ty);
}
TargetKind::NameOrAttribute => {
TargetKind::Single => {
// We found an attribute assignment like:
//
// for self.name in <iterable>:
@@ -1778,7 +1776,7 @@ impl<'db> ClassLiteral<'db> {
union_of_inferred_types = union_of_inferred_types.add(inferred_ty);
}
TargetKind::NameOrAttribute => {
TargetKind::Single => {
// We found an attribute assignment like:
//
// with <context_manager> as self.name:
@@ -1808,7 +1806,7 @@ impl<'db> ClassLiteral<'db> {
union_of_inferred_types = union_of_inferred_types.add(inferred_ty);
}
TargetKind::NameOrAttribute => {
TargetKind::Single => {
// We found an attribute assignment like:
//
// [... for self.name in <iterable>]
@@ -1836,42 +1834,42 @@ impl<'db> ClassLiteral<'db> {
}
match is_attribute_bound {
Truthiness::AlwaysTrue => Symbol::bound(union_of_inferred_types.build()),
Truthiness::Ambiguous => Symbol::possibly_unbound(union_of_inferred_types.build()),
Truthiness::AlwaysFalse => Symbol::Unbound,
Truthiness::AlwaysTrue => Place::bound(union_of_inferred_types.build()),
Truthiness::Ambiguous => Place::possibly_unbound(union_of_inferred_types.build()),
Truthiness::AlwaysFalse => Place::Unbound,
}
}
/// A helper function for `instance_member` that looks up the `name` attribute only on
/// this class, not on its superclasses.
fn own_instance_member(self, db: &'db dyn Db, name: &str) -> SymbolAndQualifiers<'db> {
fn own_instance_member(self, db: &'db dyn Db, name: &str) -> PlaceAndQualifiers<'db> {
// TODO: There are many things that are not yet implemented here:
// - `typing.Final`
// - Proper diagnostics
let body_scope = self.body_scope(db);
let table = symbol_table(db, body_scope);
let table = place_table(db, body_scope);
if let Some(symbol_id) = table.symbol_id_by_name(name) {
if let Some(place_id) = table.place_id_by_name(name) {
let use_def = use_def_map(db, body_scope);
let declarations = use_def.public_declarations(symbol_id);
let declared_and_qualifiers = symbol_from_declarations(db, declarations);
let declarations = use_def.public_declarations(place_id);
let declared_and_qualifiers = place_from_declarations(db, declarations);
match declared_and_qualifiers {
Ok(SymbolAndQualifiers {
symbol: mut declared @ Symbol::Type(declared_ty, declaredness),
Ok(PlaceAndQualifiers {
place: mut declared @ Place::Type(declared_ty, declaredness),
qualifiers,
}) => {
// For the purpose of finding instance attributes, ignore `ClassVar`
// declarations:
if qualifiers.contains(TypeQualifiers::CLASS_VAR) {
declared = Symbol::Unbound;
declared = Place::Unbound;
}
// The attribute is declared in the class body.
let bindings = use_def.public_bindings(symbol_id);
let inferred = symbol_from_bindings(db, bindings);
let bindings = use_def.public_bindings(place_id);
let inferred = place_from_bindings(db, bindings);
let has_binding = !inferred.is_unbound();
if has_binding {
@@ -1887,7 +1885,7 @@ impl<'db> ClassLiteral<'db> {
// we trust the declared type.
declared.with_qualifiers(qualifiers)
} else {
Symbol::Type(
Place::Type(
UnionType::from_elements(db, [declared_ty, implicit_ty]),
declaredness,
)
@@ -1900,7 +1898,7 @@ impl<'db> ClassLiteral<'db> {
// has a class-level default value, but it would not be
// found in a `__dict__` lookup.
Symbol::Unbound.into()
Place::Unbound.into()
}
} else {
// The attribute is declared but not bound in the class body.
@@ -1916,7 +1914,7 @@ impl<'db> ClassLiteral<'db> {
Self::implicit_instance_attribute(db, body_scope, name)
.ignore_possibly_unbound()
{
Symbol::Type(
Place::Type(
UnionType::from_elements(db, [declared_ty, implicit_ty]),
declaredness,
)
@@ -1928,8 +1926,8 @@ impl<'db> ClassLiteral<'db> {
}
}
Ok(SymbolAndQualifiers {
symbol: Symbol::Unbound,
Ok(PlaceAndQualifiers {
place: Place::Unbound,
qualifiers: _,
}) => {
// The attribute is not *declared* in the class body. It could still be declared/bound
@@ -1939,7 +1937,7 @@ impl<'db> ClassLiteral<'db> {
}
Err((declared, _conflicting_declarations)) => {
// There are conflicting declarations for this attribute in the class body.
Symbol::bound(declared.inner_type()).with_qualifiers(declared.qualifiers())
Place::bound(declared.inner_type()).with_qualifiers(declared.qualifiers())
}
}
} else {
@@ -2454,16 +2452,16 @@ impl<'db> KnownClass {
self,
db: &'db dyn Db,
) -> Result<ClassLiteral<'db>, KnownClassLookupError<'db>> {
let symbol = known_module_symbol(db, self.canonical_module(db), self.name(db)).symbol;
let symbol = known_module_symbol(db, self.canonical_module(db), self.name(db)).place;
match symbol {
Symbol::Type(Type::ClassLiteral(class_literal), Boundness::Bound) => Ok(class_literal),
Symbol::Type(Type::ClassLiteral(class_literal), Boundness::PossiblyUnbound) => {
Place::Type(Type::ClassLiteral(class_literal), Boundness::Bound) => Ok(class_literal),
Place::Type(Type::ClassLiteral(class_literal), Boundness::PossiblyUnbound) => {
Err(KnownClassLookupError::ClassPossiblyUnbound { class_literal })
}
Symbol::Type(found_type, _) => {
Place::Type(found_type, _) => {
Err(KnownClassLookupError::SymbolNotAClass { found_type })
}
Symbol::Unbound => Err(KnownClassLookupError::ClassNotFound),
Place::Unbound => Err(KnownClassLookupError::ClassNotFound),
}
}

View File

@@ -11,8 +11,8 @@ use ruff_text_size::{Ranged, TextRange};
use super::{Type, TypeCheckDiagnostics, binding_type};
use crate::lint::LintSource;
use crate::semantic_index::place::ScopeId;
use crate::semantic_index::semantic_index;
use crate::semantic_index::symbol::ScopeId;
use crate::types::function::FunctionDecorators;
use crate::{
Db,

View File

@@ -19,7 +19,6 @@ use crate::{Db, Module, ModuleName, Program, declare_lint};
use itertools::Itertools;
use ruff_db::diagnostic::{Annotation, Diagnostic, Severity, SubDiagnostic};
use ruff_python_ast::{self as ast, AnyNodeRef};
use ruff_python_stdlib::builtins::version_builtin_was_added;
use ruff_text_size::{Ranged, TextRange};
use rustc_hash::FxHashSet;
use std::fmt::Formatter;
@@ -1778,25 +1777,6 @@ pub(super) fn report_possibly_unbound_attribute(
));
}
pub(super) fn report_unresolved_reference(context: &InferContext, expr_name_node: &ast::ExprName) {
let Some(builder) = context.report_lint(&UNRESOLVED_REFERENCE, expr_name_node) else {
return;
};
let ast::ExprName { id, .. } = expr_name_node;
let mut diagnostic = builder.into_diagnostic(format_args!("Name `{id}` used when not defined"));
if let Some(version_added_to_builtins) = version_builtin_was_added(id) {
diagnostic.info(format_args!(
"`{id}` was added as a builtin in Python 3.{version_added_to_builtins}"
));
add_inferred_python_version_hint_to_diagnostic(
context.db(),
&mut diagnostic,
"resolving types",
);
}
}
pub(super) fn report_invalid_exception_caught(context: &InferContext, node: &ast::Expr, ty: Type) {
let Some(builder) = context.report_lint(&INVALID_EXCEPTION_CAUGHT, node) else {
return;

View File

@@ -794,7 +794,7 @@ mod tests {
use crate::Db;
use crate::db::tests::setup_db;
use crate::symbol::typing_extensions_symbol;
use crate::place::typing_extensions_symbol;
use crate::types::{KnownClass, Parameter, Parameters, Signature, StringLiteralType, Type};
#[test]
@@ -833,7 +833,7 @@ mod tests {
);
let iterator_synthesized = typing_extensions_symbol(&db, "Iterator")
.symbol
.place
.ignore_possibly_unbound()
.unwrap()
.to_instance(&db)

View File

@@ -58,11 +58,11 @@ use ruff_python_ast as ast;
use ruff_text_size::Ranged;
use crate::module_resolver::{KnownModule, file_to_module};
use crate::place::{Boundness, Place, place_from_bindings};
use crate::semantic_index::ast_ids::HasScopedUseId;
use crate::semantic_index::definition::Definition;
use crate::semantic_index::place::ScopeId;
use crate::semantic_index::semantic_index;
use crate::semantic_index::symbol::ScopeId;
use crate::symbol::{Boundness, Symbol, symbol_from_bindings};
use crate::types::generics::GenericContext;
use crate::types::narrow::ClassInfoConstraintFunction;
use crate::types::signatures::{CallableSignature, Signature};
@@ -234,8 +234,8 @@ impl<'db> OverloadLiteral<'db> {
.name
.scoped_use_id(db, scope);
let Symbol::Type(Type::FunctionLiteral(previous_type), Boundness::Bound) =
symbol_from_bindings(db, use_def.bindings_at_use(use_id))
let Place::Type(Type::FunctionLiteral(previous_type), Boundness::Bound) =
place_from_bindings(db, use_def.bindings_at_use(use_id))
else {
return None;
};
@@ -927,7 +927,7 @@ pub(crate) mod tests {
use super::*;
use crate::db::tests::setup_db;
use crate::symbol::known_module_symbol;
use crate::place::known_module_symbol;
#[test]
fn known_function_roundtrip_from_str() {
@@ -977,7 +977,7 @@ pub(crate) mod tests {
};
let function_definition = known_module_symbol(&db, module, function_name)
.symbol
.place
.expect_type()
.expect_function_literal()
.definition(&db);

View File

@@ -1,13 +1,43 @@
use crate::Db;
use crate::semantic_index::symbol::ScopeId;
use crate::place::{imported_symbol, place_from_bindings, place_from_declarations};
use crate::semantic_index::place::ScopeId;
use crate::semantic_index::{
attribute_scopes, global_scope, semantic_index, symbol_table, use_def_map,
attribute_scopes, global_scope, place_table, semantic_index, use_def_map,
};
use crate::symbol::{imported_symbol, symbol_from_bindings, symbol_from_declarations};
use crate::types::{ClassBase, ClassLiteral, KnownClass, Type};
use ruff_python_ast::name::Name;
use rustc_hash::FxHashSet;
pub(crate) fn all_declarations_and_bindings<'db>(
db: &'db dyn Db,
scope_id: ScopeId<'db>,
) -> impl Iterator<Item = Name> + 'db {
let use_def_map = use_def_map(db, scope_id);
let table = place_table(db, scope_id);
use_def_map
.all_public_declarations()
.filter_map(move |(symbol_id, declarations)| {
place_from_declarations(db, declarations)
.ok()
.and_then(|result| {
result
.place
.ignore_possibly_unbound()
.and_then(|_| table.place_expr(symbol_id).as_name().cloned())
})
})
.chain(
use_def_map
.all_public_bindings()
.filter_map(move |(symbol_id, bindings)| {
place_from_bindings(db, bindings)
.ignore_possibly_unbound()
.and_then(|_| table.place_expr(symbol_id).as_name().cloned())
}),
)
}
struct AllMembers {
members: FxHashSet<Name>,
}
@@ -101,16 +131,18 @@ impl AllMembers {
let module_scope = global_scope(db, file);
let use_def_map = use_def_map(db, module_scope);
let symbol_table = symbol_table(db, module_scope);
let place_table = place_table(db, module_scope);
for (symbol_id, _) in use_def_map.all_public_declarations() {
let symbol_name = symbol_table.symbol(symbol_id).name();
let Some(symbol_name) = place_table.place_expr(symbol_id).as_name() else {
continue;
};
if !imported_symbol(db, file, symbol_name, None)
.symbol
.place
.is_unbound()
{
self.members
.insert(symbol_table.symbol(symbol_id).name().clone());
.insert(place_table.place_expr(symbol_id).expect_name().clone());
}
}
}
@@ -118,24 +150,8 @@ impl AllMembers {
}
fn extend_with_declarations_and_bindings(&mut self, db: &dyn Db, scope_id: ScopeId) {
let use_def_map = use_def_map(db, scope_id);
let symbol_table = symbol_table(db, scope_id);
for (symbol_id, declarations) in use_def_map.all_public_declarations() {
if symbol_from_declarations(db, declarations)
.is_ok_and(|result| !result.symbol.is_unbound())
{
self.members
.insert(symbol_table.symbol(symbol_id).name().clone());
}
}
for (symbol_id, bindings) in use_def_map.all_public_bindings() {
if !symbol_from_bindings(db, bindings).is_unbound() {
self.members
.insert(symbol_table.symbol(symbol_id).name().clone());
}
}
self.members
.extend(all_declarations_and_bindings(db, scope_id));
}
fn extend_with_class_members<'db>(
@@ -167,9 +183,10 @@ impl AllMembers {
let file = class_body_scope.file(db);
let index = semantic_index(db, file);
for function_scope_id in attribute_scopes(db, class_body_scope) {
let attribute_table = index.instance_attribute_table(function_scope_id);
for symbol in attribute_table.symbols() {
self.members.insert(symbol.name().clone());
let place_table = index.place_table(function_scope_id);
for instance_attribute in place_table.instance_attributes() {
let name = instance_attribute.sub_segments()[0].as_member().unwrap();
self.members.insert(name.clone());
}
}
}
@@ -178,6 +195,6 @@ impl AllMembers {
/// List all members of a given type: anything that would be valid when accessed
/// as an attribute on an object of the given type.
pub(crate) fn all_members<'db>(db: &'db dyn Db, ty: Type<'db>) -> FxHashSet<Name> {
pub fn all_members<'db>(db: &'db dyn Db, ty: Type<'db>) -> FxHashSet<Name> {
AllMembers::of(db, ty).members
}

File diff suppressed because it is too large Load Diff

View File

@@ -4,7 +4,7 @@ use std::marker::PhantomData;
use super::protocol_class::ProtocolInterface;
use super::{ClassType, KnownClass, SubclassOfType, Type};
use crate::symbol::{Symbol, SymbolAndQualifiers};
use crate::place::{Boundness, Place, PlaceAndQualifiers};
use crate::types::{ClassLiteral, TypeMapping, TypeVarInstance};
use crate::{Db, FxOrderSet};
@@ -45,12 +45,12 @@ impl<'db> Type<'db> {
protocol: ProtocolInstanceType<'db>,
) -> bool {
// TODO: this should consider the types of the protocol members
// as well as whether each member *exists* on `self`.
protocol
.inner
.interface(db)
.members(db)
.all(|member| !self.member(db, member.name()).symbol.is_unbound())
protocol.inner.interface(db).members(db).all(|member| {
matches!(
self.member(db, member.name()).place,
Place::Type(_, Boundness::Bound)
)
})
}
}
@@ -294,14 +294,14 @@ impl<'db> ProtocolInstanceType<'db> {
false
}
pub(crate) fn instance_member(self, db: &'db dyn Db, name: &str) -> SymbolAndQualifiers<'db> {
pub(crate) fn instance_member(self, db: &'db dyn Db, name: &str) -> PlaceAndQualifiers<'db> {
match self.inner {
Protocol::FromClass(class) => class.instance_member(db, name),
Protocol::Synthesized(synthesized) => synthesized
.interface()
.member_by_name(db, name)
.map(|member| SymbolAndQualifiers {
symbol: Symbol::bound(member.ty()),
.map(|member| PlaceAndQualifiers {
place: Place::bound(member.ty()),
qualifiers: member.qualifiers(),
})
.unwrap_or_else(|| KnownClass::Object.to_instance(db).instance_member(db, name)),

View File

@@ -1,11 +1,11 @@
use crate::Db;
use crate::semantic_index::ast_ids::HasScopedExpressionId;
use crate::semantic_index::expression::Expression;
use crate::semantic_index::place::{PlaceTable, ScopeId, ScopedPlaceId};
use crate::semantic_index::place_table;
use crate::semantic_index::predicate::{
PatternPredicate, PatternPredicateKind, Predicate, PredicateNode,
};
use crate::semantic_index::symbol::{ScopeId, ScopedSymbolId, SymbolTable};
use crate::semantic_index::symbol_table;
use crate::types::function::KnownFunction;
use crate::types::infer::infer_same_file_expression_type;
use crate::types::{
@@ -42,7 +42,7 @@ use super::UnionType;
pub(crate) fn infer_narrowing_constraint<'db>(
db: &'db dyn Db,
predicate: Predicate<'db>,
symbol: ScopedSymbolId,
place: ScopedPlaceId,
) -> Option<Type<'db>> {
let constraints = match predicate.node {
PredicateNode::Expression(expression) => {
@@ -62,7 +62,7 @@ pub(crate) fn infer_narrowing_constraint<'db>(
PredicateNode::StarImportPlaceholder(_) => return None,
};
if let Some(constraints) = constraints {
constraints.get(&symbol).copied()
constraints.get(&place).copied()
} else {
None
}
@@ -190,7 +190,7 @@ impl ClassInfoConstraintFunction {
}
}
type NarrowingConstraints<'db> = FxHashMap<ScopedSymbolId, Type<'db>>;
type NarrowingConstraints<'db> = FxHashMap<ScopedPlaceId, Type<'db>>;
fn merge_constraints_and<'db>(
into: &mut NarrowingConstraints<'db>,
@@ -235,7 +235,7 @@ fn merge_constraints_or<'db>(
}
fn negate_if<'db>(constraints: &mut NarrowingConstraints<'db>, db: &'db dyn Db, yes: bool) {
for (_symbol, ty) in constraints.iter_mut() {
for (_place, ty) in constraints.iter_mut() {
*ty = ty.negate_if(db, yes);
}
}
@@ -347,8 +347,8 @@ impl<'db> NarrowingConstraintsBuilder<'db> {
})
}
fn symbols(&self) -> &'db SymbolTable {
symbol_table(self.db, self.scope())
fn places(&self) -> &'db PlaceTable {
place_table(self.db, self.scope())
}
fn scope(&self) -> ScopeId<'db> {
@@ -360,9 +360,9 @@ impl<'db> NarrowingConstraintsBuilder<'db> {
}
#[track_caller]
fn expect_expr_name_symbol(&self, symbol: &str) -> ScopedSymbolId {
self.symbols()
.symbol_id_by_name(symbol)
fn expect_expr_name_symbol(&self, symbol: &str) -> ScopedPlaceId {
self.places()
.place_id_by_name(symbol)
.expect("We should always have a symbol for every `Name` node")
}

View File

@@ -1,5 +1,5 @@
use crate::db::tests::TestDb;
use crate::symbol::{builtins_symbol, known_module_symbol};
use crate::place::{builtins_symbol, known_module_symbol};
use crate::types::{
BoundMethodType, CallableType, IntersectionBuilder, KnownClass, Parameter, Parameters,
Signature, SpecialFormType, SubclassOfType, TupleType, Type, UnionType,
@@ -130,20 +130,20 @@ impl Ty {
Ty::LiteralString => Type::LiteralString,
Ty::BytesLiteral(s) => Type::bytes_literal(db, s.as_bytes()),
Ty::BuiltinInstance(s) => builtins_symbol(db, s)
.symbol
.place
.expect_type()
.to_instance(db)
.unwrap(),
Ty::AbcInstance(s) => known_module_symbol(db, KnownModule::Abc, s)
.symbol
.place
.expect_type()
.to_instance(db)
.unwrap(),
Ty::AbcClassLiteral(s) => known_module_symbol(db, KnownModule::Abc, s)
.symbol
.place
.expect_type(),
Ty::TypingLiteral => Type::SpecialForm(SpecialFormType::Literal),
Ty::BuiltinClassLiteral(s) => builtins_symbol(db, s).symbol.expect_type(),
Ty::BuiltinClassLiteral(s) => builtins_symbol(db, s).place.expect_type(),
Ty::KnownClassInstance(known_class) => known_class.to_instance(db),
Ty::Union(tys) => {
UnionType::from_elements(db, tys.into_iter().map(|ty| ty.into_type(db)))
@@ -166,7 +166,7 @@ impl Ty {
Ty::SubclassOfBuiltinClass(s) => SubclassOfType::from(
db,
builtins_symbol(db, s)
.symbol
.place
.expect_type()
.expect_class_literal()
.default_specialization(db),
@@ -174,17 +174,17 @@ impl Ty {
Ty::SubclassOfAbcClass(s) => SubclassOfType::from(
db,
known_module_symbol(db, KnownModule::Abc, s)
.symbol
.place
.expect_type()
.expect_class_literal()
.default_specialization(db),
),
Ty::AlwaysTruthy => Type::AlwaysTruthy,
Ty::AlwaysFalsy => Type::AlwaysFalsy,
Ty::BuiltinsFunction(name) => builtins_symbol(db, name).symbol.expect_type(),
Ty::BuiltinsFunction(name) => builtins_symbol(db, name).place.expect_type(),
Ty::BuiltinsBoundMethod { class, method } => {
let builtins_class = builtins_symbol(db, class).symbol.expect_type();
let function = builtins_class.member(db, method).symbol.expect_type();
let builtins_class = builtins_symbol(db, class).place.expect_type();
let function = builtins_class.member(db, method).place.expect_type();
create_bound_method(db, function, builtins_class)
}

View File

@@ -5,10 +5,11 @@ use itertools::{Either, Itertools};
use ruff_python_ast::name::Name;
use crate::{
semantic_index::{symbol_table, use_def_map},
symbol::{symbol_from_bindings, symbol_from_declarations},
types::function::KnownFunction,
types::{ClassBase, ClassLiteral, Type, TypeMapping, TypeQualifiers, TypeVarInstance},
place::{place_from_bindings, place_from_declarations},
semantic_index::{place_table, use_def_map},
types::{
ClassBase, ClassLiteral, KnownFunction, Type, TypeMapping, TypeQualifiers, TypeVarInstance,
},
{Db, FxOrderSet},
};
@@ -273,7 +274,7 @@ impl<'a, 'db> ProtocolMember<'a, 'db> {
/// The list of excluded members is subject to change between Python versions,
/// especially for dunders, but it probably doesn't matter *too* much if this
/// list goes out of date. It's up to date as of Python commit 87b1ea016b1454b1e83b9113fa9435849b7743aa
/// (<https://github.com/python/cpython/blob/87b1ea016b1454b1e83b9113fa9435849b7743aa/Lib/typing.py#L1776-L1791>)
/// (<https://github.com/python/cpython/blob/87b1ea016b1454b1e83b9113fa9435849b7743aa/Lib/typing.py#L1776-L1814>)
fn excluded_from_proto_members(member: &str) -> bool {
matches!(
member,
@@ -303,7 +304,7 @@ fn excluded_from_proto_members(member: &str) -> bool {
| "__annotate__"
| "__annotate_func__"
| "__annotations_cache__"
)
) || member.starts_with("_abc_")
}
/// Inner Salsa query for [`ProtocolClassLiteral::interface`].
@@ -321,19 +322,19 @@ fn cached_protocol_interface<'db>(
{
let parent_scope = parent_protocol.body_scope(db);
let use_def_map = use_def_map(db, parent_scope);
let symbol_table = symbol_table(db, parent_scope);
let place_table = place_table(db, parent_scope);
members.extend(
use_def_map
.all_public_declarations()
.flat_map(|(symbol_id, declarations)| {
symbol_from_declarations(db, declarations).map(|symbol| (symbol_id, symbol))
.flat_map(|(place_id, declarations)| {
place_from_declarations(db, declarations).map(|place| (place_id, place))
})
.filter_map(|(symbol_id, symbol)| {
symbol
.symbol
.filter_map(|(place_id, place)| {
place
.place
.ignore_possibly_unbound()
.map(|ty| (symbol_id, ty, symbol.qualifiers))
.map(|ty| (place_id, ty, place.qualifiers))
})
// Bindings in the class body that are not declared in the class body
// are not valid protocol members, and we plan to emit diagnostics for them
@@ -346,14 +347,18 @@ fn cached_protocol_interface<'db>(
.chain(
use_def_map
.all_public_bindings()
.filter_map(|(symbol_id, bindings)| {
symbol_from_bindings(db, bindings)
.filter_map(|(place_id, bindings)| {
place_from_bindings(db, bindings)
.ignore_possibly_unbound()
.map(|ty| (symbol_id, ty, TypeQualifiers::default()))
.map(|ty| (place_id, ty, TypeQualifiers::default()))
}),
)
.map(|(symbol_id, member, qualifiers)| {
(symbol_table.symbol(symbol_id).name(), member, qualifiers)
.filter_map(|(place_id, member, qualifiers)| {
Some((
place_table.place_expr(place_id).as_name()?,
member,
qualifiers,
))
})
.filter(|(name, _, _)| !excluded_from_proto_members(name))
.map(|(name, ty, qualifiers)| {

View File

@@ -1533,16 +1533,15 @@ pub(crate) enum ParameterForm {
mod tests {
use super::*;
use crate::db::tests::{TestDb, setup_db};
use crate::symbol::global_symbol;
use crate::types::KnownClass;
use crate::types::function::FunctionType;
use crate::place::global_symbol;
use crate::types::{FunctionType, KnownClass};
use ruff_db::system::DbWithWritableSystem as _;
#[track_caller]
fn get_function_f<'db>(db: &'db TestDb, file: &'static str) -> FunctionType<'db> {
let module = ruff_db::files::system_path_to_file(db, file).unwrap();
global_symbol(db, module, "f")
.symbol
.place
.expect_type()
.expect_function_literal()
}

View File

@@ -1,7 +1,7 @@
use ruff_python_ast as ast;
use crate::db::Db;
use crate::symbol::{Boundness, Symbol};
use crate::place::{Boundness, Place};
use crate::types::class_base::ClassBase;
use crate::types::diagnostic::report_base_with_incompatible_slots;
use crate::types::{ClassLiteral, Type};
@@ -24,7 +24,7 @@ enum SlotsKind {
impl SlotsKind {
fn from(db: &dyn Db, base: ClassLiteral) -> Self {
let Symbol::Type(slots_ty, bound) = base.own_class_member(db, None, "__slots__").symbol
let Place::Type(slots_ty, bound) = base.own_class_member(db, None, "__slots__").place
else {
return Self::NotSpecified;
};

View File

@@ -1,4 +1,4 @@
use crate::symbol::SymbolAndQualifiers;
use crate::place::PlaceAndQualifiers;
use crate::types::{
ClassType, DynamicType, KnownClass, MemberLookupPolicy, Type, TypeMapping, TypeVarInstance,
};
@@ -99,7 +99,7 @@ impl<'db> SubclassOfType<'db> {
db: &'db dyn Db,
name: &str,
policy: MemberLookupPolicy,
) -> Option<SymbolAndQualifiers<'db>> {
) -> Option<PlaceAndQualifiers<'db>> {
Type::from(self.subclass_of).find_name_in_mro_with_policy(db, name, policy)
}

View File

@@ -7,7 +7,7 @@ use ruff_python_ast::{self as ast, AnyNodeRef};
use crate::Db;
use crate::semantic_index::ast_ids::{HasScopedExpressionId, ScopedExpressionId};
use crate::semantic_index::symbol::ScopeId;
use crate::semantic_index::place::ScopeId;
use crate::types::{Type, TypeCheckDiagnostics, infer_expression_types};
use crate::unpack::{UnpackKind, UnpackValue};
@@ -84,7 +84,7 @@ impl<'db> Unpacker<'db> {
value_ty: Type<'db>,
) {
match target {
ast::Expr::Name(_) | ast::Expr::Attribute(_) => {
ast::Expr::Name(_) | ast::Expr::Attribute(_) | ast::Expr::Subscript(_) => {
self.targets.insert(
target.scoped_expression_id(self.db(), self.target_scope),
value_ty,

View File

@@ -6,7 +6,7 @@ use crate::Db;
use crate::ast_node_ref::AstNodeRef;
use crate::semantic_index::ast_ids::{HasScopedExpressionId, ScopedExpressionId};
use crate::semantic_index::expression::Expression;
use crate::semantic_index::symbol::{FileScopeId, ScopeId};
use crate::semantic_index::place::{FileScopeId, ScopeId};
/// This ingredient represents a single unpacking.
///

View File

@@ -180,6 +180,7 @@ impl Server {
completion_provider: experimental
.is_some_and(Experimental::is_completions_enabled)
.then_some(lsp_types::CompletionOptions {
trigger_characters: Some(vec!['.'.to_string()]),
..Default::default()
}),
..Default::default()

View File

@@ -272,7 +272,7 @@ fn run_test(
python_path: configuration
.python()
.map(|sys_prefix| {
PythonPath::SysPrefix(
PythonPath::IntoSysPrefix(
sys_prefix.to_path_buf(),
SysPrefixPathOrigin::PythonCliFlag,
)

View File

@@ -179,7 +179,7 @@ class PlaygroundServer
monaco.languages.registerDocumentFormattingEditProvider("python", this);
}
triggerCharacters: undefined;
triggerCharacters: string[] = ["."];
provideCompletionItems(
model: editor.ITextModel,