Compare commits

...

66 Commits

Author SHA1 Message Date
Micha Reiser
c13a1814f7 Shrink TypeInference struct by 6 words 2025-02-16 15:40:50 +01:00
Alex Waygood
93aff36147 [red-knot] Improve handling of inherited class attributes (#16160) 2025-02-15 18:22:35 +00:00
Ayush Baweja
df45a9db64 [flake8-comprehensions]: Handle trailing comma in C403 fix (#16110)
## Summary

Resolves [#16099 ](https://github.com/astral-sh/ruff/issues/16099) based
on [#15929 ](https://github.com/astral-sh/ruff/pull/15929)

## Test Plan

Added test case `s = set([x for x in range(3)],)` and updated snapshot.

---------

Co-authored-by: dylwil3 <dylwil3@gmail.com>
2025-02-15 11:45:41 -06:00
InSync
3c69b685ee [ruff] Implicit class variable in dataclass (RUF045) (#14349)
## Summary

Implement lint rule to flag un-annotated variable assignments in dataclass definitions.

Resolves #12877.

---------

Co-authored-by: dylwil3 <dylwil3@gmail.com>
2025-02-15 09:08:13 -06:00
github-actions[bot]
171facd960 Sync vendored typeshed stubs (#16173)
Close and reopen this PR to trigger CI

Co-authored-by: typeshedbot <>
2025-02-15 10:01:34 +00:00
InSync
977447f9b8 Sort linters alphabetically (#16168)
## Summary

Resolves #16164.

Linters are now sorted by their names case-insensitively.

## Test Plan


![](https://github.com/user-attachments/assets/87ffd4d8-1ba5-4a4b-8fed-dd21a020bd27)

Also unit tests.
2025-02-14 22:05:08 +01:00
Micha Reiser
b3e99b25bf Fix missing serde feature for red_knot_python_semantic (#16169)
## Summary

Running `cargo test -p red_knot_python_semantic` failed because of a
missing serde feature. This PR enables the `ruff_python_ast`'`s `serde`
if the crate's `serde` feature is enabled

## Test Plan

`cargo test -p red_knot_python_semantic` compiles again
2025-02-14 20:31:55 +00:00
Carl Meyer
dcabb948f3 [red-knot] add special case for float/complex (#16166)
When adjusting the existing tests, I aimed to avoid dealing with the
special case in other tests if it's not necessary to do so (that is,
avoid using `float` and `complex` as examples where we just need "some
type"), and keep the tests for the special case mostly collected in the
mdtest dedicated to that purpose.

Fixes https://github.com/astral-sh/ruff/issues/14932
2025-02-14 12:24:10 -08:00
Vlad Nedelcu
219712860c [refurb] Check for subclasses includes subscript expressions (FURB189) (#16155)
## Summary

Added checks for subscript expressions on builtin classes as in FURB189.
The object is changed to use the collections objects and the types from
the subscript are kept.

Resolves #16130 

> Note: Added some comments in the code explaining why
## Test Plan


- Added a subscript dict and list class to the test file.
- Tested locally to check that the symbols are changed and the types are
kept.
- No modifications changed on optional `str` values.
2025-02-14 20:21:26 +01:00
Brent Westbrook
f58a54f043 Move red_knot_python_semantic::PythonVersion to the ruff_python_ast crate (#16147)
## Summary

This PR moves the `PythonVersion` struct from the
`red_knot_python_semantic` crate to the `ruff_python_ast` crate so that
it can be used more easily in the syntax error detection work. Compared
to that [prototype](https://github.com/astral-sh/ruff/pull/16090/) these
changes reduce us from 2 `PythonVersion` structs to 1.

This does not unify any of the `PythonVersion` *enums*, but I hope to
make some progress on that in a follow-up.

## Test Plan

Existing tests, this should not change any external behavior.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-02-14 12:48:08 -05:00
Junhson Jean-Baptiste
fa28dc5ccf [internal] Move Linter OperatorPrecedence into ruff_python_ast crate (#16162)
## Summary

This change begins to resolve #16071 by moving the `OperatorPrecedence`
structs from the `ruff_python_linter` crate into `ruff_python_ast`. This
PR also implements `precedence()` methods on the `Expr` and `ExprRef`
enums.

## Test Plan

Since this change mainly shifts existing logic, I didn't add any
additional tests. Existing tests do pass.
2025-02-14 15:55:07 +01:00
Dhruv Manilawala
63dd68e0ed Refactor symbol lookup APIs to hide re-export implementation details (#16133)
## Summary

This PR refactors the symbol lookup APIs to better facilitate the
re-export implementation. Specifically,
* Add `module_type_symbol` which returns the `Symbol` that's a member of
`types.ModuleType`
* Rename `symbol` -> `symbol_impl`; add `symbol` which delegates to
`symbol_impl` with `RequireExplicitReExport::No`
* Update `global_symbol` to do `symbol_impl` -> fall back to
`module_type_symbol` and default to `RequireExplicitReExport::No`
* Add `imported_symbol` to do `symbol_impl` with
`RequireExplicitReExport` as `Yes` if the module is in a stub file else
`No`
* Update `known_module_symbol` to use `imported_symbol` with a fallback
to `module_type_symbol`
* Update `ModuleLiteralType::member` to use `imported_symbol` with a
custom fallback

We could potentially also update `symbol_from_declarations` and
`symbol_from_bindings` to avoid passing in the `RequireExplicitReExport`
as it would be always `No` if called directly. We could add
`symbol_from_declarations_impl` and `symbol_from_bindings_impl`.

Looking at the `_impl` functions, I think we should move all of these
symbol related logic into `symbol.rs` where `Symbol` is defined and the
`_impl` could be private while we expose the public APIs at the crate
level. This would also make the `RequireExplicitReExport` an
implementation detail and the caller doesn't need to worry about it.
2025-02-14 15:25:48 +05:30
Dhruv Manilawala
60b3ef2c98 [red-knot] Support re-export conventions for stub files (#16073)
This is an alternative implementation to #15848.

## Summary

This PR adds support for re-export conventions for imports for stub
files.

**How does this work?**
* Add a new flag on the `Import` and `ImportFrom` definitions to
indicate whether they're being exported or not
* Add a new enum to indicate whether the symbol lookup is happening
within the same file or is being queried from another file (e.g., an
import statement)
* When a `Symbol` is being queried, we'll skip the definitions that are
(a) coming from a stub file (b) external lookup and (c) check the
re-export flag on the definition

This implementation does not yet support `__all__` and `*` imports as
both are features that needs to be implemented independently.

closes: #14099
closes: #15476 

## Test Plan

Add test cases, update existing ones if required.
2025-02-14 15:17:51 +05:30
InSync
3d0a58eb60 [pyupgrade] Unwrap unary expressions correctly (UP018) (#15919)
## Summary

Resolves #15859.

The rule now adds parentheses if the original call wraps an unary
expression and is:

* The left-hand side of a binary expression where the operator is `**`.
* The caller of a call expression.
* The subscripted of a subscript expression.
* The object of an attribute access.

The fix will also be marked as unsafe if there are any comments in its
range.

## Test Plan

`cargo nextest run` and `cargo insta test`.
2025-02-14 08:42:00 +01:00
InSync
1db8392a5a Check for backtick-quoted shortcut links in CI (#16114)
## Summary

Follow-up to #16035.

`check_docs_formatted.py` will now report backtick-quoted shortcut links
in rule documentation. It uses a regular expression to find them. Such a
link:

* Starts with `[`, followed by <code>\`</code>, then a "name" sequence
of at least one non-backtick non-newline character, followed by another
<code>\`</code>, then ends with `]`.
* Is not followed by either a `[` or a `(`.
* Is not placed within a code block.

If the name is a known Ruff option name, that link is not considered a
violation.

## Test Plan

Manual.
2025-02-14 08:37:46 +01:00
Micha Reiser
81e202ed52 Make CallBinding::callable_ty required (#16135)
## Summary

The `callable_ty` is always known except in some TODO code where we can
use a `TODO` type instead.

## Test Plan

`cargo test`
2025-02-14 08:15:24 +01:00
Vlad Nedelcu
63c67750b1 Replace dead link for rome tools playground (#16153)
## Summary

Rome Tools Playground was renamed to Biome Playground. The link was
replaced to the new website.

Resolves #16143


## Test Plan

- Checked the linked is accessible from the README
2025-02-14 12:27:14 +05:30
Shaygan Hooshyari
0a75a1d56b Replace is-macro with implementation in enums (#16144) 2025-02-13 22:49:00 +00:00
Shaygan Hooshyari
bb15c7653a Use ubuntu-24 to run benchmarks (#16145) 2025-02-13 22:05:51 +00:00
Vlad Nedelcu
cb8b23d609 [flake8-pyi] Avoid flagging custom-typevar-for-self on metaclass methods (PYI019) (#16141) 2025-02-13 18:44:11 +00:00
Alex Waygood
be49151a3d [red-knot] Remove a parameter from the symbol_by_id() query (#16138) 2025-02-13 13:33:40 +00:00
InSync
7d2e40be2d [pylint] Do not offer fix for raw strings (PLE251) (#16132)
## Summary

Resolves #13294, follow-up to #13882.

At #13882, it was concluded that a fix should not be offered for raw
strings. This change implements that. The five rules in question are now
no longer always fixable.

## Test Plan

`cargo nextest run` and `cargo insta test`.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2025-02-13 08:36:11 +00:00
Brent Westbrook
f8093b65ea [flake8-builtins] Update documentation (A005) (#16097)
Follow-up to https://github.com/astral-sh/ruff/pull/15951 to update
* the options links in A005 to reference
`lint.flake8-builtins.builtins-strict-checking`
* the description of the rule to explain strict vs non-strict checking
* the option documentation to point back to the rule
2025-02-12 12:50:13 -05:00
Alex Waygood
c31352f52b [ruff] Skip RUF001 diagnostics when visiting string type definitions (#16122) 2025-02-12 16:27:38 +00:00
Andrew Gallant
a9671e7008 ruff_db: add a vector for configuring diagnostic output (#16118)
For now, the only thing one can configure is whether color is enabled or
not. This avoids needing to ask the `colored` crate whether colors have
been globally enabled or disabled. And, more crucially, avoids the need
to _set_ this global flag for testing diagnostic output. Doing so can
have unintended consequences, as outlined in #16115.

Fixes #16115
2025-02-12 14:38:05 +00:00
Micha Reiser
03f08283ad [red-knot] Fallback to requires-python if no python-version is specified (#16028)
## Summary

Add support for the `project.requires-python` field in `pyproject.toml`
files.

Fall back to the resolved lower bound of `project.requires-python` if
the `environment.python-version` field is `None` (or more accurately,
initialize `environment.python-version with `requires-python`'s lower
bound if left unspecified).

## UX design

There are two options on how we can handle the fallback to
`requires-python`'s lower bound:

1. Store the resolved lower bound in `environment.python-version` if
that field is `None` (Implemented in this PR)
2. Store the `requires-python` constraint separately. 

There's no observed difference unless a user-level configuration (or any
other inherited configuration is used). Let's discuss it on the given
example


**User configuration**

```toml
[environment]
python-version = "3.10"
```

**Project configuration (`pyproject.toml`)**

```toml
[project]
name = "test"
requires-python = ">= 3.12"

[tool.knot]
# No environment table
```

The resolved version for 1. is 3.12 because the `requires-python`
constraint precedence takes precedence over the `python-version` in the
user configuration. 2. resolves to 3.10 because all `python-version`
constraints take precedence before falling back to `requires-python`.

Ruff implements 1. It's also the easier to implement and it does seem
intuitive to me that the more local `requires-python` constraint takes
precedence.


## Test plan

Added CLI and unit tests.
2025-02-12 11:47:59 +00:00
Vasco Schiavo
ae1b381c06 [pylint] Correct ordering of arguments in fix for if-stmt-min-max (PLR1730) (#16080)
The PR addresses the issue #16040 .

---

The logic used into the rule is the following:

Suppose to have an expression of the form 

```python
if a cmp b:
    c = d
```
where `a`,` b`, `c` and `d` are Python obj and `cmp` one of `<`, `>`,
`<=`, `>=`.

Then:

- `if a=c and b=d`
    
    - if `<=` fix with `a = max(b, a)`
    - if `>=`  fix with `a = min(b, a)`
    - if `>` fix with `a = min(a, b)`
    - if `<` fix with `a = max(a, b)`

- `if a=d and b=c`

    - if `<=` fix with `b = min(a, b)`
    - if `>=`  fix with `b = max(a, b)`
    - if `>` fix with `b = max(b, a)`
    - if `<` fix with `b = min(b, a)`
 
- do nothing, i.e., we cannot fix this case.

---

In total we have 8 different and possible cases.

```

| Case  | Expression       | Fix           |
|-------|------------------|---------------|
| 1     | if a >= b: a = b | a = min(b, a) |
| 2     | if a <= b: a = b | a = max(b, a) |
| 3     | if a <= b: b = a | b = min(a, b) |
| 4     | if a >= b: b = a | b = max(a, b) |
| 5     | if a > b: a = b  | a = min(a, b) |
| 6     | if a < b: a = b  | a = max(a, b) |
| 7     | if a < b: b = a  | b = min(b, a) |
| 8     | if a > b: b = a  | b = max(b, a) |
```

I added them in the tests. 

Please double-check that I didn't make any mistakes. It's quite easy to
mix up > and <.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2025-02-12 10:27:46 +01:00
David Peter
366ae1feaa [red-knot] Document 'public type of undeclared symbols' behavior (#16096)
## Summary

After I was asked twice within the same day, I thought it would be a
good idea to write some *user facing* documentation that explains our
reasoning behind inferring `Unknown | T_inferred` for public uses of
undeclared symbols. This is a major deviation from the behavior of other
type checkers and it seems like a good practice to defend our choice
like this.
2025-02-12 08:52:11 +01:00
Wei Lee
86c5cba472 [airflow] Fix ImportPathMoved / ProviderName misuse (AIR303) (#16013)
## Summary


* fix ImportPathMoved / ProviderName misuse
* oncrete names, such as `["airflow", "config_templates",
"default_celery", "DEFAULT_CELERY_CONFIG"]`, should use `ProviderName`.
In contrast, module paths like `"airflow", "operators", "weekday", ...`
should use `ImportPathMoved`. Misuse may lead to incorrect detection.

## Test Plan

update test fixture
2025-02-12 12:34:16 +05:30
Andrew Gallant
6e34f74c16 add diagnostic Span (couples File and TextRange) (#16101)
This essentially makes it impossible to construct a `Diagnostic`
that has a `TextRange` but no `File`.

This is meant to be a precursor to multi-span support.

(Note that I consider this more of a prototyping-change and not
necessarily what this is going to look like longer term.)

Reviewers can probably review this PR as one big diff instead of
commit-by-commit.
2025-02-11 14:55:12 -05:00
Micha Reiser
9c179314ed Remove Hash and Eq from AstNodeRef for types not implementing Eq or Hash (#16100)
## Summary

This is a follow up to
https://github.com/astral-sh/ruff/pull/15763#discussion_r1949681336

It reverts the change to using ptr equality for `AstNodeRef`s, which in
turn removes the `Eq`, `PartialEq`, and `Hash` implementations for
`AstNodeRef`s parametrized with AST nodes.
Cheap comparisons shouldn't be needed because the node field is
generally marked as `[#tracked]` and `#[no_eq]` and removing the
implementations even enforces that those
attributes are set on all `AstNodeRef` fields (which is good).

The only downside this has is that we technically wouldn't have to mark
the `Unpack::target` as `#[tracked]` because
the `target` field is accessed in every query accepting `Unpack` as an
argument.

Overall, enforcing the use of `#[tracked]` seems like a good trade off,
espacially considering that it's very likely that
we'd probably forget to mark the `Unpack::target` field as tracked if we
add a new `Unpack` query that doesn't access the target.

## Test Plan

`cargo test`
2025-02-11 19:55:50 +01:00
Micha Reiser
ce31c2693b Fix release build warning about unused todo type message (#16102) 2025-02-11 18:38:41 +00:00
Brent Westbrook
7b487d853a [pydocstyle] Handle arguments with the same names as sections (D417) (#16011)
## Summary

Fixes #16007. The logic from the last fix for this (#9427) was
sufficient, it just wasn't being applied because `Attributes` sections
aren't expected to have nested sections. I just deleted the outer
conditional, which should hopefully fix this for all section types.

## Test Plan

New regression test, plus the existing D417 tests.
2025-02-11 12:05:29 -05:00
Alex Waygood
df1d430294 [red-knot] Reduce usage of From<Type> implementations when working with Symbols (#16076) 2025-02-11 11:09:37 +00:00
Ibraheem Ahmed
69d86d1d69 Transition to salsa coarse-grained tracked structs (#15763)
## Summary

Transition to using coarse-grained tracked structs (depends on
https://github.com/salsa-rs/salsa/pull/657). For now, this PR doesn't
add any `#[tracked]` fields, meaning that any changes cause the entire
struct to be invalidated. It also changes `AstNodeRef` to be
compared/hashed by pointer address, instead of performing a deep AST
comparison.

## Test Plan

This yields a 10-15% improvement on my machine (though weirdly some runs
were 5-10% without being flagged as inconsistent by criterion, is there
some non-determinism involved?). It's possible that some of this is
unrelated, I'll try applying the patch to the current salsa version to
make sure.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2025-02-11 11:38:50 +01:00
InSync
7fbd89cb39 [pyupgrade] Handle micro version numbers correctly (UP036) (#16091)
## Summary

Resolves #16082.

`UP036` will now also take into consideration whether or not a micro
version number is set:

* If a third element doesn't exist, the existing logic is preserved.
* If it exists but is not an integer literal, the check will not be
reported.
* If it is an integer literal but doesn't fit into a `u8`, the check
will be reported as invalid.
* Otherwise, the compared version is determined to always be less than
the target version when:
	* The target's minor version is smaller than that of the comparator, or
* The operator is `<`, the micro version is 0, and the two minor
versions compare equal.

As this is considered a bugfix, it is not preview-gated.

## Test Plan

`cargo nextest run` and `cargo insta test`.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2025-02-11 07:40:56 +00:00
David Peter
0019d39f6e [red-knot] T | object == object (#16088)
## Summary

- Simplify unions with `object` to `object`.
- Add a new `Type::object(db)` constructor to abbreviate
`KnownClass::Object.to_instance(db)` in some places.
- Add a `Type::is_object` and `Class::is_object` function to make some
tests for a bit easier to read.

closes #16084

## Test Plan

New Markdown tests.
2025-02-10 23:07:06 +01:00
Dylan
f30fac6326 [ruff] Skip singleton starred expressions for incorrectly-parenthesized-tuple-in-subscript (RUF031) (#16083)
The index in subscript access like `d[*y]` will not be linted or
autofixed with parentheses, even when
`lint.ruff.parenthesize-tuple-in-subscript = true`.

Closes #16077
2025-02-10 11:30:07 -06:00
Micha Reiser
a4c8c49ac2 Delete left-over `verbosity.rs (#16081) 2025-02-10 16:06:05 +00:00
Micha Reiser
af832560fc [red-knot] User-level configuration (#16021)
## Summary

This PR adds support for user-level configurations
(`~/.config/knot/knot.toml`) to Red Knot.

Red Knot will watch the user-level configuration file for changes but
only if it exists
when the process start. It doesn't watch for new configurations, 
mainly to simplify things for now (it would require watching the entire
`.config` directory because the `knot` subfolder might not exist
either).

The new `ConfigurationFile` struct seems a bit overkill for now but I
plan to use it for
hierarchical configurations as well. 


Red Knot uses the same strategy as uv and Ruff by using the etcetera
crate.

## Test Plan

Added CLI and file watching test
2025-02-10 16:44:23 +01:00
Micha Reiser
f7819e553f Add user_configuration_directory to System (#16020)
## Summary

This PR adds a new `user_configuration_directory` method to `System`. We
need it to resolve where to lookup a user-level `knot.toml`
configuration file.
The method belongs to `System` because not all platforms have a
convention of where to store such configuration files (e.g. wasm).


I refactored `TestSystem` to be a simple wrapper around an `Arc<dyn
System...>` and use the `System.as_any` method instead to cast it down
to an `InMemory` system. I also removed some `System` specific methods
from `InMemoryFileSystem`, they don't belong there.

This PR removes the `os` feature as a default feature from `ruff_db`.
Most crates depending on `ruff_db` don't need it because they only
depend on `System` or only depend on `os` for testing. This was
necessary to fix a compile error with `red_knot_wasm`

## Test Plan

I'll make use of the method in my next PR. So I guess we won't know if
it works before then but I copied the code from Ruff/uv, so I have high
confidence that it is correct.

`cargo test`
2025-02-10 15:50:55 +01:00
Micha Reiser
678b0c2d39 [red-knot] Resolve Options to Settings (#16000)
## Summary

This PR generalize the idea that we may want to emit diagnostics for 
invalid or incompatible configuration values similar to how we already 
do it for `rules`. 

This PR introduces a new `Settings` struct that is similar to `Options`
but, unlike
`Options`, are fields have their default values filled in and they use a
representation optimized for reads.

The diagnostics created during loading the `Settings` are stored on the
`Project` so that we can emit them when calling `check`.

The motivation for this work is that it simplifies adding new settings.
That's also why I went ahead and added the `terminal.error-on-warning`
setting to demonstrate how new settings are added.

## Test Plan

Existing tests, new CLI test.
2025-02-10 15:28:45 +01:00
Dhruv Manilawala
524cf6e515 Bump version to 0.9.6 (#16074) 2025-02-10 18:14:04 +05:30
Dhruv Manilawala
857cf0deb0 Revert tailwindcss v4 update (#16075)
## Summary

Revert the v4 update for now until the codebase is updated
(https://github.com/astral-sh/ruff/pull/16069).

Update renovate config to disable updating it.

## Test Plan

```console
$ npx --yes --package renovate -- renovate-config-validator
(node:98977) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
 INFO: Validating .github/renovate.json5
 INFO: Config validated successfully
```

And run `npm run build` in the `playground/` directory.
2025-02-10 18:13:32 +05:30
Dhruv Manilawala
0f1eb1e2fc Improve migration document (#16072)
## Summary

This PR improves the migration document based on recent feedback.

[Rendered
version](https://github.com/astral-sh/ruff/blob/dhruv/migration/docs/editors/migration.md)

### Preview

<img width="1897" alt="Screenshot 2025-02-10 at 2 52 31 PM"
src="https://github.com/user-attachments/assets/596a3217-6598-4274-ab49-a89b9cb60fe0"
/>
2025-02-10 16:30:03 +05:30
InSync
b69eb9099a Fix reference definition labels for backtick-quoted shortcut links (#16035)
## Summary

Resolves #16010.

The changes boil down to something like this:

```diff
-/// The [FastAPI documentation] recommends the use of [`typing.Annotated`]
+/// The [FastAPI documentation] recommends the use of [`typing.Annotated`][typing-annotated]

-/// [typing.Annotated]: https://docs.python.org/3/library/typing.html#typing.Annotated
+/// [typing-annotated]: https://docs.python.org/3/library/typing.html#typing.Annotated
```

## Test Plan

Mkdocs:


![](https://github.com/user-attachments/assets/a2e6bf22-56fa-4b2c-9500-1c1256c5a218)

GitHub:

> ## Why is this bad?
> The [FastAPI documentation] recommends the use of
[`typing.Annotated`][typing-annotated]
> 
> ...
>
> [FastAPI documentation]:
https://fastapi.tiangolo.com/tutorial/query-params-str-validations/?h=annotated#advantages-of-annotated
> [typing-annotated]:
https://docs.python.org/3/library/typing.html#typing.Annotated

[CommonMark
dingus](https://spec.commonmark.org/dingus/?text=%23%23%20Why%20is%20this%20bad%3F%0AThe%20%5BFastAPI%20documentation%5D%20recommends%20the%20use%20of%20%5B%60typing.Annotated%60%5D%5Btyping-annotated%5D%0A%0A...%0A%0A%5BFastAPI%20documentation%5D%3A%20https%3A%2F%2Ffastapi.tiangolo.com%2Ftutorial%2Fquery-params-str-validations%2F%3Fh%3Dannotated%23advantages-of-annotated%0A%5Btyping-annotated%5D%3A%20https%3A%2F%2Fdocs.python.org%2F3%2Flibrary%2Ftyping.html%23typing.Annotated):

```html
<h2>Why is this bad?</h2>
<p>The <a href="https://fastapi.tiangolo.com/tutorial/query-params-str-validations/?h=annotated#advantages-of-annotated">FastAPI documentation</a> recommends the use of <a href="https://docs.python.org/3/library/typing.html#typing.Annotated"><code>typing.Annotated</code></a></p>
<p>...</p>
```
2025-02-10 09:54:22 +01:00
ABDULRAHMAN ALRAHMA
d2f661f795 RUF009 should behave similar to B008 and ignore attributes with immutable types (#16048)
This PR resolved #15772

Before PR:
```
def _(
    this_is_fine: int = f(),           # No error
    this_is_not: list[int] = f()       # B008: Do not perform function call `f` in argument defaults
): ...


@dataclass
class _:
    this_is_not_fine: list[int] = f()  # RUF009: Do not perform function call `f` in dataclass defaults
    this_is_also_not: int = f()        # RUF009: Do not perform function call `f` in dataclass defaults
```

After PR:
```
def _(
    this_is_fine: int = f(),           # No error
    this_is_not: list[int] = f()       # B008: Do not perform function call `f` in argument defaults
): ...


@dataclass
class _:
    this_is_not_fine: list[int] = f()  # RUF009: Do not perform function call `f` in dataclass defaults
    this_is_fine: int = f()
```
2025-02-10 09:46:23 +01:00
InSync
07cf8852a3 [pylint] Also report when the object isn't a literal (PLE1310) (#15985)
## Summary

Follow-up to #15984.

Previously, `PLE1310` would only report when the object is a literal:

```python
'a'.strip('//')  # error

foo = ''
foo.strip('//')  # no error
```

After this change, objects whose type can be inferred to be either `str`
or `bytes` will also be reported in preview.

## Test Plan

`cargo nextest run` and `cargo insta test`.
2025-02-10 09:31:27 +01:00
renovate[bot]
c08989692b Update Rust crate rustc-hash to v2.1.1 (#16060)
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [rustc-hash](https://redirect.github.com/rust-lang/rustc-hash) |
workspace.dependencies | patch | `2.1.0` -> `2.1.1` |

---

> [!WARNING]
> Some dependencies could not be looked up. Check the Dependency
Dashboard for more information.

---

### Release Notes

<details>
<summary>rust-lang/rustc-hash (rustc-hash)</summary>

###
[`v2.1.1`](https://redirect.github.com/rust-lang/rustc-hash/blob/HEAD/CHANGELOG.md#211)

[Compare
Source](https://redirect.github.com/rust-lang/rustc-hash/compare/v2.1.0...v2.1.1)

-   Change the internal algorithm to better accomodate large hashmaps.
This mitigates a [regression with 2.0 in
rustc](https://redirect.github.com/rust-lang/rust/issues/135477).
See [PR#55](https://redirect.github.com/rust-lang/rustc-hash/pull/55)
for more details on the change (this PR was not merged).
This problem might be improved with changes to hashbrown in the future.

#### 2.1.0

-   Implement `Clone` for `FxRandomState`
-   Implement `Clone` for `FxSeededState`
-   Use SPDX license expression in license field

#### 2.0.0

-   Replace hash with faster and better finalized hash.
    This replaces the previous "fxhash" algorithm originating in Firefox
with a custom hasher designed and implemented by Orson Peters
([`@orlp`](https://redirect.github.com/orlp)).
It was measured to have slightly better performance for rustc, has
better theoretical properties
    and also includes a significantly better string hasher.
-   Fix `no_std` builds

#### 1.2.0 (**YANKED**)

**Note: This version has been yanked due to issues with the `no_std`
feature!**

-   Add a `FxBuildHasher` unit struct
-   Improve documentation
-   Add seed API for supplying custom seeds other than 0
- Add `FxRandomState` based on `rand` (behind the `rand` feature) for
random seeds
-   Make many functions `const fn`
-   Implement `Clone` for `FxHasher` struct

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "before 4am on Monday" (UTC),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/astral-sh/ruff).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xNjQuMSIsInVwZGF0ZWRJblZlciI6IjM5LjE2NC4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJpbnRlcm5hbCJdfQ==-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-02-10 08:22:52 +01:00
Dhruv Manilawala
869a9543e4 Root exclusions in the server to project root (#16043)
## Summary

fixes: #16041 

## Test Plan

Using the [project](https://github.com/bwcc-clan/polebot) in the linked
issue:

Notice how the project "polebot" is in the "play" directory which is
included in the `exclude` setting as:

```toml
exclude = ["play"]
```

**Before this fix**

```
DEBUG ruff:worker:0 ruff_server::resolve: Ignored path via `exclude`: /private/tmp/ruff-test/play/polebot/src/utils/log_tools.py
```

**After this fix**

```
DEBUG ruff:worker:2 ruff_server::resolve: Included path via `include`: /private/tmp/ruff-test/play/polebot/src/utils/log_tools.py
```

I also updated the same project to remove the "play" directory from the
`exclude` setting and made sure that anything under the `polebot/play`
directory is included:

```
DEBUG  ruff:worker:4 ruff_server::resolve: Included path via `include`: /private/tmp/ruff-test/play/polebot/play/test.py
```

And, excluded when I add the directory back:

```
DEBUG  ruff:worker:2 ruff_server::resolve: Ignored path via `exclude`: /private/tmp/ruff-test/play/polebot/play/test.py
```
2025-02-10 04:57:14 +00:00
Dhruv Manilawala
cc0a5dd14a Directly include Settings struct for the server (#16042)
## Summary

This PR refactors the `RuffSettings` struct to directly include the
resolved `Settings` instead of including the specific fields from it.
The server utilizes a lot of it already, so it makes sense to just
include the entire struct for simplicity.

### `Deref`

I implemented `Deref` on `RuffSettings` to return the `Settings` because
`RuffSettings` is now basically a wrapper around it with the config path
as the other field. This path field is only used for debugging
("printDebugInformation" command).
2025-02-10 10:20:01 +05:30
renovate[bot]
b54e390cb4 Update Rust crate clap to v4.5.28 (#16059) 2025-02-09 21:36:36 -05:00
renovate[bot]
5e1403a8a6 Update Rust crate strum_macros to 0.27.0 (#16065) 2025-02-10 02:36:08 +00:00
renovate[bot]
a6b86e3de2 Update NPM Development dependencies (#16067) 2025-02-09 21:29:45 -05:00
renovate[bot]
798725ccf9 Update Rust crate uuid to v1.13.1 (#16066) 2025-02-09 21:29:38 -05:00
renovate[bot]
81749164bc Update Rust crate strum to 0.27.0 (#16064) 2025-02-09 21:29:20 -05:00
renovate[bot]
b3ea17f128 Update pre-commit dependencies (#16063) 2025-02-09 21:29:12 -05:00
renovate[bot]
8fb69d3b05 Update dependency ruff to v0.9.5 (#16062) 2025-02-09 21:29:05 -05:00
renovate[bot]
3b69a8833d Update Rust crate toml to v0.8.20 (#16061) 2025-02-09 21:28:58 -05:00
Brent Westbrook
88b543d73a [flake8-builtins] Make strict module name comparison optional (A005) (#15951)
## Summary

This PR adds the configuration option
`lint.flake8-builtins.builtins-strict-checking`, which is used in A005
to determine whether the fully-qualified module name (relative to the
project root or source directories) should be checked instead of just
the final component as is currently the case.

As discussed in
https://github.com/astral-sh/ruff/issues/15399#issuecomment-2587017147,
the default value of the new option is `false` on preview, so modules
like `utils.logging` from the initial report are no longer flagged by
default. For non-preview the default is still strict checking.

## Test Plan

New A005 test module with the structure reported in #15399.

Fixes #15399
2025-02-09 19:33:03 -05:00
InSync
f367aa8367 [ruff] Indented form feeds (RUF054) (#16049)
## Summary

Resolves #12321.

The physical-line-based `RUF054` checks for form feed characters that
are preceded by only tabs and spaces, but not any other characters,
including form feeds.

## Test Plan

`cargo nextest run` and `cargo insta test`.
2025-02-09 19:23:48 -05:00
David Salvisberg
9ae98d4a09 [flake8-type-checking] Avoid TC004 false positive with __getattr__ (#16052) 2025-02-09 16:27:06 +00:00
Dylan
0af4b23d9f [ruff] Skip type definitions for missing-f-string-syntax (RUF027) (#16054)
As an f-string is never correct in a type definition context, we skip
[missing-f-string-syntax
(RUF027)](https://docs.astral.sh/ruff/rules/missing-f-string-syntax/#missing-f-string-syntax-ruf027)
in this case.

Closes #16037
2025-02-09 10:16:28 -06:00
Dylan
f178ecc2d7 [flake8-pyi] Extend fix to Python <= 3.9 for redundant-none-literal (PYI061) (#16044)
This PR extends the fix offered for [redundant-none-literal
(PYI061)](https://docs.astral.sh/ruff/rules/redundant-none-literal/#redundant-none-literal-pyi061)
to include Python versions <= 3.9 by using `typing.Optional` instead of
the operator `|`. We also offer the fix with `|` for any target version
on stub files.

Closes #15795
2025-02-09 09:58:53 -06:00
InSync
a46fbda948 [flake8-datetime] Ignore .replace() calls while looking for .astimezone (#16050)
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2025-02-09 15:48:59 +00:00
Alex Waygood
fc59e1b17f [red-knot] Merge TypeInferenceBuilder::infer_name_load and TypeInferenceBuilder::lookup_name (#16019)
## Summary

No functional change here; this is another simplification split out from
my outcome-refactor branch to reduce the diff there. This merges
`TypeInferenceBuilder::infer_name_load` and
`TypeInferenceBuilder::lookup_name`. This removes the need to have
extensive doc-comments about the purpose of
`TypeInferenceBuilder::lookup_name`, since the method only makes sense
when called from the specific context of
`TypeInferenceBuilder::infer_name_load`.

## Test Plan

`cargo test -p red_knot_python_semantic`
2025-02-08 19:42:14 +00:00
267 changed files with 12330 additions and 4559 deletions

View File

@@ -58,6 +58,12 @@
description: "Disable PRs updating GitHub runners (e.g. 'runs-on: macos-14')",
enabled: false,
},
{
// TODO: Remove this once the codebase is upgrade to v4 (https://github.com/astral-sh/ruff/pull/16069)
matchPackageNames: ["tailwindcss"],
matchManagers: ["npm"],
enabled: false,
},
{
// Disable updates of `zip-rs`; intentionally pinned for now due to ownership change
// See: https://github.com/astral-sh/uv/issues/3642

View File

@@ -712,7 +712,7 @@ jobs:
just test
benchmarks:
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
needs: determine_changes
if: ${{ github.repository == 'astral-sh/ruff' && !contains(github.event.pull_request.labels.*.name, 'no-test') && (needs.determine_changes.outputs.code == 'true' || github.ref == 'refs/heads/main') }}
timeout-minutes: 20

View File

@@ -74,7 +74,7 @@ repos:
pass_filenames: false # This makes it a lot faster
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.9.4
rev: v0.9.5
hooks:
- id: ruff-format
- id: ruff
@@ -92,7 +92,7 @@ repos:
# zizmor detects security vulnerabilities in GitHub Actions workflows.
# Additional configuration for the tool is found in `.github/zizmor.yml`
- repo: https://github.com/woodruffw/zizmor-pre-commit
rev: v1.3.0
rev: v1.3.1
hooks:
- id: zizmor

View File

@@ -1,5 +1,44 @@
# Changelog
## 0.9.6
### Preview features
- \[`airflow`\] Add `external_task.{ExternalTaskMarker, ExternalTaskSensor}` for `AIR302` ([#16014](https://github.com/astral-sh/ruff/pull/16014))
- \[`flake8-builtins`\] Make strict module name comparison optional (`A005`) ([#15951](https://github.com/astral-sh/ruff/pull/15951))
- \[`flake8-pyi`\] Extend fix to Python \<= 3.9 for `redundant-none-literal` (`PYI061`) ([#16044](https://github.com/astral-sh/ruff/pull/16044))
- \[`pylint`\] Also report when the object isn't a literal (`PLE1310`) ([#15985](https://github.com/astral-sh/ruff/pull/15985))
- \[`ruff`\] Implement `indented-form-feed` (`RUF054`) ([#16049](https://github.com/astral-sh/ruff/pull/16049))
- \[`ruff`\] Skip type definitions for `missing-f-string-syntax` (`RUF027`) ([#16054](https://github.com/astral-sh/ruff/pull/16054))
### Rule changes
- \[`flake8-annotations`\] Correct syntax for `typing.Union` in suggested return type fixes for `ANN20x` rules ([#16025](https://github.com/astral-sh/ruff/pull/16025))
- \[`flake8-builtins`\] Match upstream module name comparison (`A005`) ([#16006](https://github.com/astral-sh/ruff/pull/16006))
- \[`flake8-comprehensions`\] Detect overshadowed `list`/`set`/`dict`, ignore variadics and named expressions (`C417`) ([#15955](https://github.com/astral-sh/ruff/pull/15955))
- \[`flake8-pie`\] Remove following comma correctly when the unpacked dictionary is empty (`PIE800`) ([#16008](https://github.com/astral-sh/ruff/pull/16008))
- \[`flake8-simplify`\] Only trigger `SIM401` on known dictionaries ([#15995](https://github.com/astral-sh/ruff/pull/15995))
- \[`pylint`\] Do not report calls when object type and argument type mismatch, remove custom escape handling logic (`PLE1310`) ([#15984](https://github.com/astral-sh/ruff/pull/15984))
- \[`pyupgrade`\] Comments within parenthesized value ranges should not affect applicability (`UP040`) ([#16027](https://github.com/astral-sh/ruff/pull/16027))
- \[`pyupgrade`\] Don't introduce invalid syntax when upgrading old-style type aliases with parenthesized multiline values (`UP040`) ([#16026](https://github.com/astral-sh/ruff/pull/16026))
- \[`pyupgrade`\] Ensure we do not rename two type parameters to the same name (`UP049`) ([#16038](https://github.com/astral-sh/ruff/pull/16038))
- \[`pyupgrade`\] \[`ruff`\] Don't apply renamings if the new name is shadowed in a scope of one of the references to the binding (`UP049`, `RUF052`) ([#16032](https://github.com/astral-sh/ruff/pull/16032))
- \[`ruff`\] Update `RUF009` to behave similar to `B008` and ignore attributes with immutable types ([#16048](https://github.com/astral-sh/ruff/pull/16048))
### Server
- Root exclusions in the server to project root ([#16043](https://github.com/astral-sh/ruff/pull/16043))
### Bug fixes
- \[`flake8-datetime`\] Ignore `.replace()` calls while looking for `.astimezone` ([#16050](https://github.com/astral-sh/ruff/pull/16050))
- \[`flake8-type-checking`\] Avoid `TC004` false positive where the runtime definition is provided by `__getattr__` ([#16052](https://github.com/astral-sh/ruff/pull/16052))
### Documentation
- Improve `ruff-lsp` migration document ([#16072](https://github.com/astral-sh/ruff/pull/16072))
- Undeprecate `ruff.nativeServer` ([#16039](https://github.com/astral-sh/ruff/pull/16039))
## 0.9.5
### Preview features

116
Cargo.lock generated
View File

@@ -29,6 +29,12 @@ dependencies = [
"memchr",
]
[[package]]
name = "allocator-api2"
version = "0.2.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "683d7910e743518b0e34f1186f92494becacb047c7b6bf616c96772180fef923"
[[package]]
name = "android-tzdata"
version = "0.1.1"
@@ -354,9 +360,9 @@ dependencies = [
[[package]]
name = "clap"
version = "4.5.27"
version = "4.5.28"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "769b0145982b4b48713e01ec42d61614425f27b7058bda7180a3a41f30104796"
checksum = "3e77c3243bd94243c03672cb5154667347c457ca271254724f9f393aee1c05ff"
dependencies = [
"clap_builder",
"clap_derive",
@@ -407,9 +413,9 @@ dependencies = [
[[package]]
name = "clap_derive"
version = "4.5.24"
version = "4.5.28"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "54b755194d6389280185988721fffba69495eed5ee9feeee9a599b53db80318c"
checksum = "bf4ced95c6f4a675af3da73304b9ac4ed991640c36374e4b46795c49e17cf1ed"
dependencies = [
"heck",
"proc-macro2",
@@ -471,7 +477,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "117725a109d387c937a1533ce01b450cbde6b88abceea8473c4d7a85853cda3c"
dependencies = [
"lazy_static",
"windows-sys 0.48.0",
"windows-sys 0.59.0",
]
[[package]]
@@ -480,7 +486,7 @@ version = "3.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fde0e0ec90c9dfb3b4b1a0891a7dcd0e2bffde2f7efed5fe7c9bb00e5bfb915e"
dependencies = [
"windows-sys 0.48.0",
"windows-sys 0.59.0",
]
[[package]]
@@ -897,7 +903,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "33d852cb9b869c2a9b3df2f71a3074817f01e1844f839a144f5fcef059a4eb5d"
dependencies = [
"libc",
"windows-sys 0.52.0",
"windows-sys 0.59.0",
]
[[package]]
@@ -1031,10 +1037,8 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c4567c8db10ae91089c99af84c68c38da3ec2f087c3f82960bcdbf3656b6f4d7"
dependencies = [
"cfg-if",
"js-sys",
"libc",
"wasi 0.11.0+wasi-snapshot-preview1",
"wasm-bindgen",
]
[[package]]
@@ -1098,6 +1102,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e5274423e17b7c9fc20b6e7e208532f9b19825d82dfd615708b70edd83df41f1"
dependencies = [
"ahash",
"allocator-api2",
]
[[package]]
@@ -1475,7 +1480,7 @@ checksum = "e19b23d53f35ce9f56aebc7d1bb4e6ac1e9c0db7ac85c8d1760c04379edced37"
dependencies = [
"hermit-abi 0.4.0",
"libc",
"windows-sys 0.52.0",
"windows-sys 0.59.0",
]
[[package]]
@@ -2049,7 +2054,7 @@ dependencies = [
"once_cell",
"pep440_rs",
"regex",
"rustc-hash 2.1.0",
"rustc-hash 2.1.1",
"serde",
"smallvec",
"thiserror 1.0.69",
@@ -2409,6 +2414,7 @@ dependencies = [
"red_knot_server",
"regex",
"ruff_db",
"ruff_python_ast",
"ruff_python_trivia",
"salsa",
"tempfile",
@@ -2437,7 +2443,7 @@ dependencies = [
"ruff_macros",
"ruff_python_ast",
"ruff_text_size",
"rustc-hash 2.1.0",
"rustc-hash 2.1.1",
"salsa",
"schemars",
"serde",
@@ -2477,7 +2483,7 @@ dependencies = [
"ruff_python_trivia",
"ruff_source_file",
"ruff_text_size",
"rustc-hash 2.1.0",
"rustc-hash 2.1.1",
"salsa",
"schemars",
"serde",
@@ -2505,7 +2511,7 @@ dependencies = [
"ruff_python_ast",
"ruff_source_file",
"ruff_text_size",
"rustc-hash 2.1.0",
"rustc-hash 2.1.1",
"serde",
"serde_json",
"shellexpand",
@@ -2531,7 +2537,7 @@ dependencies = [
"ruff_python_trivia",
"ruff_source_file",
"ruff_text_size",
"rustc-hash 2.1.0",
"rustc-hash 2.1.1",
"salsa",
"serde",
"smallvec",
@@ -2558,9 +2564,9 @@ dependencies = [
"js-sys",
"log",
"red_knot_project",
"red_knot_python_semantic",
"ruff_db",
"ruff_notebook",
"ruff_python_ast",
"wasm-bindgen",
"wasm-bindgen-test",
]
@@ -2642,7 +2648,7 @@ dependencies = [
[[package]]
name = "ruff"
version = "0.9.5"
version = "0.9.6"
dependencies = [
"anyhow",
"argfile",
@@ -2682,7 +2688,7 @@ dependencies = [
"ruff_source_file",
"ruff_text_size",
"ruff_workspace",
"rustc-hash 2.1.0",
"rustc-hash 2.1.1",
"serde",
"serde_json",
"shellexpand",
@@ -2721,14 +2727,13 @@ dependencies = [
"mimalloc",
"rayon",
"red_knot_project",
"red_knot_python_semantic",
"ruff_db",
"ruff_linter",
"ruff_python_ast",
"ruff_python_formatter",
"ruff_python_parser",
"ruff_python_trivia",
"rustc-hash 2.1.0",
"rustc-hash 2.1.1",
"tikv-jemallocator",
]
@@ -2750,10 +2755,10 @@ name = "ruff_db"
version = "0.0.0"
dependencies = [
"camino",
"colored 3.0.0",
"countme",
"dashmap 6.1.0",
"dunce",
"etcetera",
"filetime",
"glob",
"ignore",
@@ -2768,7 +2773,7 @@ dependencies = [
"ruff_python_trivia",
"ruff_source_file",
"ruff_text_size",
"rustc-hash 2.1.0",
"rustc-hash 2.1.1",
"salsa",
"schemars",
"serde",
@@ -2839,7 +2844,7 @@ dependencies = [
"ruff_cache",
"ruff_macros",
"ruff_text_size",
"rustc-hash 2.1.0",
"rustc-hash 2.1.1",
"schemars",
"serde",
"static_assertions",
@@ -2871,12 +2876,13 @@ name = "ruff_index"
version = "0.0.0"
dependencies = [
"ruff_macros",
"salsa",
"static_assertions",
]
[[package]]
name = "ruff_linter"
version = "0.9.5"
version = "0.9.6"
dependencies = [
"aho-corasick",
"anyhow",
@@ -2918,7 +2924,7 @@ dependencies = [
"ruff_python_trivia",
"ruff_source_file",
"ruff_text_size",
"rustc-hash 2.1.0",
"rustc-hash 2.1.1",
"schemars",
"serde",
"serde_json",
@@ -2980,7 +2986,8 @@ dependencies = [
"ruff_python_trivia",
"ruff_source_file",
"ruff_text_size",
"rustc-hash 2.1.0",
"rustc-hash 2.1.1",
"salsa",
"schemars",
"serde",
]
@@ -3027,7 +3034,7 @@ dependencies = [
"ruff_python_trivia",
"ruff_source_file",
"ruff_text_size",
"rustc-hash 2.1.0",
"rustc-hash 2.1.1",
"schemars",
"serde",
"serde_json",
@@ -3074,7 +3081,7 @@ dependencies = [
"ruff_python_trivia",
"ruff_source_file",
"ruff_text_size",
"rustc-hash 2.1.0",
"rustc-hash 2.1.1",
"static_assertions",
"unicode-ident",
"unicode-normalization",
@@ -3105,7 +3112,7 @@ dependencies = [
"ruff_python_parser",
"ruff_python_stdlib",
"ruff_text_size",
"rustc-hash 2.1.0",
"rustc-hash 2.1.1",
"schemars",
"serde",
"smallvec",
@@ -3164,7 +3171,7 @@ dependencies = [
"ruff_source_file",
"ruff_text_size",
"ruff_workspace",
"rustc-hash 2.1.0",
"rustc-hash 2.1.1",
"serde",
"serde_json",
"shellexpand",
@@ -3194,7 +3201,7 @@ dependencies = [
[[package]]
name = "ruff_wasm"
version = "0.9.5"
version = "0.9.6"
dependencies = [
"console_error_panic_hook",
"console_log",
@@ -3246,7 +3253,7 @@ dependencies = [
"ruff_python_semantic",
"ruff_python_stdlib",
"ruff_source_file",
"rustc-hash 2.1.0",
"rustc-hash 2.1.1",
"schemars",
"serde",
"shellexpand",
@@ -3273,9 +3280,9 @@ checksum = "08d43f7aa6b08d49f382cde6a7982047c3426db949b1424bc4b7ec9ae12c6ce2"
[[package]]
name = "rustc-hash"
version = "2.1.0"
version = "2.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c7fb8039b3032c191086b10f11f319a6e99e1e82889c5cc6046f515c9db1d497"
checksum = "357703d41365b4b27c590e3ed91eabb1b663f07c4c084095e60cbed4362dff0d"
[[package]]
name = "rustix"
@@ -3287,7 +3294,7 @@ dependencies = [
"errno",
"libc",
"linux-raw-sys",
"windows-sys 0.52.0",
"windows-sys 0.59.0",
]
[[package]]
@@ -3305,17 +3312,19 @@ checksum = "6ea1a2d0a644769cc99faa24c3ad26b379b786fe7c36fd3c546254801650e6dd"
[[package]]
name = "salsa"
version = "0.18.0"
source = "git+https://github.com/salsa-rs/salsa.git?rev=88a1d7774d78f048fbd77d40abca9ebd729fd1f0#88a1d7774d78f048fbd77d40abca9ebd729fd1f0"
source = "git+https://github.com/salsa-rs/salsa.git?rev=351d9cf0037be949d17800d0c7b4838e533c2ed6#351d9cf0037be949d17800d0c7b4838e533c2ed6"
dependencies = [
"append-only-vec",
"arc-swap",
"compact_str",
"crossbeam",
"dashmap 6.1.0",
"hashbrown 0.14.5",
"hashlink",
"indexmap",
"parking_lot",
"rayon",
"rustc-hash 2.1.0",
"rustc-hash 2.1.1",
"salsa-macro-rules",
"salsa-macros",
"smallvec",
@@ -3325,12 +3334,12 @@ dependencies = [
[[package]]
name = "salsa-macro-rules"
version = "0.1.0"
source = "git+https://github.com/salsa-rs/salsa.git?rev=88a1d7774d78f048fbd77d40abca9ebd729fd1f0#88a1d7774d78f048fbd77d40abca9ebd729fd1f0"
source = "git+https://github.com/salsa-rs/salsa.git?rev=351d9cf0037be949d17800d0c7b4838e533c2ed6#351d9cf0037be949d17800d0c7b4838e533c2ed6"
[[package]]
name = "salsa-macros"
version = "0.18.0"
source = "git+https://github.com/salsa-rs/salsa.git?rev=88a1d7774d78f048fbd77d40abca9ebd729fd1f0#88a1d7774d78f048fbd77d40abca9ebd729fd1f0"
source = "git+https://github.com/salsa-rs/salsa.git?rev=351d9cf0037be949d17800d0c7b4838e533c2ed6#351d9cf0037be949d17800d0c7b4838e533c2ed6"
dependencies = [
"heck",
"proc-macro2",
@@ -3601,18 +3610,18 @@ checksum = "7da8b5736845d9f2fcb837ea5d9e2628564b3b043a70948a3f0b778838c5fb4f"
[[package]]
name = "strum"
version = "0.26.3"
version = "0.27.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8fec0f0aef304996cf250b31b5a10dee7980c85da9d759361292b8bca5a18f06"
checksum = "ce1475c515a4f03a8a7129bb5228b81a781a86cb0b3fbbc19e1c556d491a401f"
dependencies = [
"strum_macros",
]
[[package]]
name = "strum_macros"
version = "0.26.4"
version = "0.27.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4c6bee85a5a24955dc440386795aa378cd9cf82acd5f764469152d2270e581be"
checksum = "9688894b43459159c82bfa5a5fa0435c19cbe3c9b427fa1dd7b1ce0c279b18a7"
dependencies = [
"heck",
"proc-macro2",
@@ -3665,7 +3674,7 @@ dependencies = [
"getrandom 0.3.1",
"once_cell",
"rustix",
"windows-sys 0.52.0",
"windows-sys 0.59.0",
]
[[package]]
@@ -3854,9 +3863,9 @@ checksum = "1f3ccbac311fea05f86f61904b462b55fb3df8837a366dfc601a0161d0532f20"
[[package]]
name = "toml"
version = "0.8.19"
version = "0.8.20"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a1ed1f98e3fdc28d6d910e6737ae6ab1a93bf1985935a1193e68f93eeb68d24e"
checksum = "cd87a5cdd6ffab733b2f74bc4fd7ee5fff6634124999ac278c35fc78c6120148"
dependencies = [
"serde",
"serde_spanned",
@@ -4153,21 +4162,22 @@ checksum = "06abde3611657adf66d383f00b093d7faecc7fa57071cce2578660c9f1010821"
[[package]]
name = "uuid"
version = "1.12.1"
version = "1.13.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b3758f5e68192bb96cc8f9b7e2c2cfdabb435499a28499a42f8f984092adad4b"
checksum = "ced87ca4be083373936a67f8de945faa23b6b42384bd5b64434850802c6dccd0"
dependencies = [
"getrandom 0.2.15",
"rand 0.8.5",
"getrandom 0.3.1",
"js-sys",
"rand 0.9.0",
"uuid-macro-internal",
"wasm-bindgen",
]
[[package]]
name = "uuid-macro-internal"
version = "1.12.1"
version = "1.13.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8a86d88347b61a0e17b9908a67efcc594130830bf1045653784358dd023e294"
checksum = "d28dd23acb5f2fa7bd2155ab70b960e770596b3bb6395119b40476c3655dfba4"
dependencies = [
"proc-macro2",
"quote",
@@ -4429,7 +4439,7 @@ version = "0.1.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cf221c93e13a30d793f7645a0e7762c55d169dbb0a49671918a2319d289b10bb"
dependencies = [
"windows-sys 0.48.0",
"windows-sys 0.59.0",
]
[[package]]

View File

@@ -123,7 +123,7 @@ rayon = { version = "1.10.0" }
regex = { version = "1.10.2" }
rustc-hash = { version = "2.0.0" }
# When updating salsa, make sure to also update the revision in `fuzz/Cargo.toml`
salsa = { git = "https://github.com/salsa-rs/salsa.git", rev = "88a1d7774d78f048fbd77d40abca9ebd729fd1f0" }
salsa = { git = "https://github.com/salsa-rs/salsa.git", rev = "351d9cf0037be949d17800d0c7b4838e533c2ed6" }
schemars = { version = "0.8.16" }
seahash = { version = "4.1.0" }
serde = { version = "1.0.197", features = ["derive"] }
@@ -143,8 +143,8 @@ snapbox = { version = "0.6.0", features = [
"examples",
] }
static_assertions = "1.1.0"
strum = { version = "0.26.0", features = ["strum_macros"] }
strum_macros = { version = "0.26.0" }
strum = { version = "0.27.0", features = ["strum_macros"] }
strum_macros = { version = "0.27.0" }
syn = { version = "2.0.55" }
tempfile = { version = "3.9.0" }
test-case = { version = "3.3.1" }

View File

@@ -149,8 +149,8 @@ curl -LsSf https://astral.sh/ruff/install.sh | sh
powershell -c "irm https://astral.sh/ruff/install.ps1 | iex"
# For a specific version.
curl -LsSf https://astral.sh/ruff/0.9.5/install.sh | sh
powershell -c "irm https://astral.sh/ruff/0.9.5/install.ps1 | iex"
curl -LsSf https://astral.sh/ruff/0.9.6/install.sh | sh
powershell -c "irm https://astral.sh/ruff/0.9.6/install.ps1 | iex"
```
You can also install Ruff via [Homebrew](https://formulae.brew.sh/formula/ruff), [Conda](https://anaconda.org/conda-forge/ruff),
@@ -183,7 +183,7 @@ Ruff can also be used as a [pre-commit](https://pre-commit.com/) hook via [`ruff
```yaml
- repo: https://github.com/astral-sh/ruff-pre-commit
# Ruff version.
rev: v0.9.5
rev: v0.9.6
hooks:
# Run the linter.
- id: ruff

View File

@@ -16,6 +16,7 @@ red_knot_python_semantic = { workspace = true }
red_knot_project = { workspace = true, features = ["zstd"] }
red_knot_server = { workspace = true }
ruff_db = { workspace = true, features = ["os", "cache"] }
ruff_python_ast = { workspace = true }
anyhow = { workspace = true }
chrono = { workspace = true }

View File

@@ -1,7 +1,7 @@
use crate::logging::Verbosity;
use crate::python_version::PythonVersion;
use clap::{ArgAction, ArgMatches, Error, Parser};
use red_knot_project::metadata::options::{EnvironmentOptions, Options};
use red_knot_project::metadata::options::{EnvironmentOptions, Options, TerminalOptions};
use red_knot_project::metadata::value::{RangedValue, RelativePathBuf};
use red_knot_python_semantic::lint;
use ruff_db::system::SystemPathBuf;
@@ -67,8 +67,8 @@ pub(crate) struct CheckCommand {
pub(crate) rules: RulesArg,
/// Use exit code 1 if there are any warning-level diagnostics.
#[arg(long, conflicts_with = "exit_zero")]
pub(crate) error_on_warning: bool,
#[arg(long, conflicts_with = "exit_zero", default_missing_value = "true", num_args=0..1)]
pub(crate) error_on_warning: Option<bool>,
/// Always use exit code 0, even when there are error-level diagnostics.
#[arg(long)]
@@ -107,6 +107,9 @@ impl CheckCommand {
}),
..EnvironmentOptions::default()
}),
terminal: Some(TerminalOptions {
error_on_warning: self.error_on_warning,
}),
rules,
..Default::default()
}

View File

@@ -11,18 +11,17 @@ use clap::Parser;
use colored::Colorize;
use crossbeam::channel as crossbeam_channel;
use red_knot_project::metadata::options::Options;
use red_knot_project::watch;
use red_knot_project::watch::ProjectWatcher;
use red_knot_project::{watch, Db};
use red_knot_project::{ProjectDatabase, ProjectMetadata};
use red_knot_server::run_server;
use ruff_db::diagnostic::{Diagnostic, Severity};
use ruff_db::diagnostic::{Diagnostic, DisplayDiagnosticConfig, Severity};
use ruff_db::system::{OsSystem, System, SystemPath, SystemPathBuf};
use salsa::plumbing::ZalsaDatabase;
mod args;
mod logging;
mod python_version;
mod verbosity;
mod version;
#[allow(clippy::print_stdout, clippy::unnecessary_wraps, clippy::print_stderr)]
@@ -97,19 +96,15 @@ fn run_check(args: CheckCommand) -> anyhow::Result<ExitStatus> {
let system = OsSystem::new(cwd);
let watch = args.watch;
let exit_zero = args.exit_zero;
let min_error_severity = if args.error_on_warning {
Severity::Warning
} else {
Severity::Error
};
let cli_options = args.into_options();
let mut workspace_metadata = ProjectMetadata::discover(system.current_directory(), &system)?;
workspace_metadata.apply_cli_options(cli_options.clone());
let mut project_metadata = ProjectMetadata::discover(system.current_directory(), &system)?;
project_metadata.apply_cli_options(cli_options.clone());
project_metadata.apply_configuration_files(&system)?;
let mut db = ProjectDatabase::new(workspace_metadata, system)?;
let mut db = ProjectDatabase::new(project_metadata, system)?;
let (main_loop, main_loop_cancellation_token) = MainLoop::new(cli_options, min_error_severity);
let (main_loop, main_loop_cancellation_token) = MainLoop::new(cli_options);
// Listen to Ctrl+C and abort the watch mode.
let main_loop_cancellation_token = Mutex::new(Some(main_loop_cancellation_token));
@@ -167,18 +162,10 @@ struct MainLoop {
watcher: Option<ProjectWatcher>,
cli_options: Options,
/// The minimum severity to consider an error when deciding the exit status.
///
/// TODO(micha): Get from the terminal settings.
min_error_severity: Severity,
}
impl MainLoop {
fn new(
cli_options: Options,
min_error_severity: Severity,
) -> (Self, MainLoopCancellationToken) {
fn new(cli_options: Options) -> (Self, MainLoopCancellationToken) {
let (sender, receiver) = crossbeam_channel::bounded(10);
(
@@ -187,7 +174,6 @@ impl MainLoop {
receiver,
watcher: None,
cli_options,
min_error_severity,
},
MainLoopCancellationToken { sender },
)
@@ -245,14 +231,24 @@ impl MainLoop {
result,
revision: check_revision,
} => {
let display_config = DisplayDiagnosticConfig::default()
.color(colored::control::SHOULD_COLORIZE.should_colorize());
let min_error_severity =
if db.project().settings(db).terminal().error_on_warning {
Severity::Warning
} else {
Severity::Error
};
let failed = result
.iter()
.any(|diagnostic| diagnostic.severity() >= self.min_error_severity);
.any(|diagnostic| diagnostic.severity() >= min_error_severity);
if check_revision == revision {
#[allow(clippy::print_stdout)]
for diagnostic in result {
println!("{}", diagnostic.display(db));
println!("{}", diagnostic.display(db, &display_config));
}
} else {
tracing::debug!(

View File

@@ -40,7 +40,7 @@ impl std::fmt::Display for PythonVersion {
}
}
impl From<PythonVersion> for red_knot_python_semantic::PythonVersion {
impl From<PythonVersion> for ruff_python_ast::python_version::PythonVersion {
fn from(value: PythonVersion) -> Self {
match value {
PythonVersion::Py37 => Self::PY37,
@@ -61,8 +61,8 @@ mod tests {
#[test]
fn same_default_as_python_version() {
assert_eq!(
red_knot_python_semantic::PythonVersion::from(PythonVersion::default()),
red_knot_python_semantic::PythonVersion::default()
ruff_python_ast::python_version::PythonVersion::from(PythonVersion::default()),
ruff_python_ast::python_version::PythonVersion::default()
);
}
}

View File

@@ -1 +0,0 @@

View File

@@ -98,7 +98,7 @@ fn cli_arguments_are_relative_to_the_current_directory() -> anyhow::Result<()> {
])?;
// Make sure that the CLI fails when the `libs` directory is not in the search path.
assert_cmd_snapshot!(case.command().current_dir(case.project_dir().join("child")), @r###"
assert_cmd_snapshot!(case.command().current_dir(case.root().join("child")), @r###"
success: false
exit_code: 1
----- stdout -----
@@ -115,7 +115,7 @@ fn cli_arguments_are_relative_to_the_current_directory() -> anyhow::Result<()> {
----- stderr -----
"###);
assert_cmd_snapshot!(case.command().current_dir(case.project_dir().join("child")).arg("--extra-search-path").arg("../libs"), @r"
assert_cmd_snapshot!(case.command().current_dir(case.root().join("child")).arg("--extra-search-path").arg("../libs"), @r"
success: true
exit_code: 0
----- stdout -----
@@ -167,7 +167,7 @@ fn paths_in_configuration_files_are_relative_to_the_project_root() -> anyhow::Re
),
])?;
assert_cmd_snapshot!(case.command().current_dir(case.project_dir().join("child")), @r"
assert_cmd_snapshot!(case.command().current_dir(case.root().join("child")), @r"
success: true
exit_code: 0
----- stdout -----
@@ -575,6 +575,37 @@ fn exit_code_no_errors_but_error_on_warning_is_true() -> anyhow::Result<()> {
Ok(())
}
#[test]
fn exit_code_no_errors_but_error_on_warning_is_enabled_in_configuration() -> anyhow::Result<()> {
let case = TestCase::with_files([
("test.py", r"print(x) # [unresolved-reference]"),
(
"knot.toml",
r#"
[terminal]
error-on-warning = true
"#,
),
])?;
assert_cmd_snapshot!(case.command(), @r###"
success: false
exit_code: 1
----- stdout -----
warning: lint:unresolved-reference
--> <temp_dir>/test.py:1:7
|
1 | print(x) # [unresolved-reference]
| - Name `x` used when not defined
|
----- stderr -----
"###);
Ok(())
}
#[test]
fn exit_code_both_warnings_and_errors() -> anyhow::Result<()> {
let case = TestCase::with_file(
@@ -686,6 +717,109 @@ fn exit_code_exit_zero_is_true() -> anyhow::Result<()> {
Ok(())
}
#[test]
fn user_configuration() -> anyhow::Result<()> {
let case = TestCase::with_files([
(
"project/knot.toml",
r#"
[rules]
division-by-zero = "warn"
"#,
),
(
"project/main.py",
r#"
y = 4 / 0
for a in range(0, y):
x = a
print(x)
"#,
),
])?;
let config_directory = case.root().join("home/.config");
let config_env_var = if cfg!(windows) {
"APPDATA"
} else {
"XDG_CONFIG_HOME"
};
assert_cmd_snapshot!(
case.command().current_dir(case.root().join("project")).env(config_env_var, config_directory.as_os_str()),
@r###"
success: true
exit_code: 0
----- stdout -----
warning: lint:division-by-zero
--> <temp_dir>/project/main.py:2:5
|
2 | y = 4 / 0
| ----- Cannot divide object of type `Literal[4]` by zero
3 |
4 | for a in range(0, y):
|
warning: lint:possibly-unresolved-reference
--> <temp_dir>/project/main.py:7:7
|
5 | x = a
6 |
7 | print(x)
| - Name `x` used when possibly not defined
|
----- stderr -----
"###
);
// The user-level configuration promotes `possibly-unresolved-reference` to an error.
// Changing the level for `division-by-zero` has no effect, because the project-level configuration
// has higher precedence.
case.write_file(
config_directory.join("knot/knot.toml"),
r#"
[rules]
division-by-zero = "error"
possibly-unresolved-reference = "error"
"#,
)?;
assert_cmd_snapshot!(
case.command().current_dir(case.root().join("project")).env(config_env_var, config_directory.as_os_str()),
@r###"
success: false
exit_code: 1
----- stdout -----
warning: lint:division-by-zero
--> <temp_dir>/project/main.py:2:5
|
2 | y = 4 / 0
| ----- Cannot divide object of type `Literal[4]` by zero
3 |
4 | for a in range(0, y):
|
error: lint:possibly-unresolved-reference
--> <temp_dir>/project/main.py:7:7
|
5 | x = a
6 |
7 | print(x)
| ^ Name `x` used when possibly not defined
|
----- stderr -----
"###
);
Ok(())
}
struct TestCase {
_temp_dir: TempDir,
_settings_scope: SettingsBindDropGuard,
@@ -753,7 +887,7 @@ impl TestCase {
Ok(())
}
fn project_dir(&self) -> &Path {
fn root(&self) -> &Path {
&self.project_dir
}

View File

@@ -9,11 +9,14 @@ use red_knot_project::metadata::pyproject::{PyProject, Tool};
use red_knot_project::metadata::value::{RangedValue, RelativePathBuf};
use red_knot_project::watch::{directory_watcher, ChangeEvent, ProjectWatcher};
use red_knot_project::{Db, ProjectDatabase, ProjectMetadata};
use red_knot_python_semantic::{resolve_module, ModuleName, PythonPlatform, PythonVersion};
use red_knot_python_semantic::{resolve_module, ModuleName, PythonPlatform};
use ruff_db::files::{system_path_to_file, File, FileError};
use ruff_db::source::source_text;
use ruff_db::system::{OsSystem, SystemPath, SystemPathBuf};
use ruff_db::system::{
OsSystem, System, SystemPath, SystemPathBuf, UserConfigDirectoryOverrideGuard,
};
use ruff_db::Upcast;
use ruff_python_ast::python_version::PythonVersion;
struct TestCase {
db: ProjectDatabase,
@@ -220,17 +223,44 @@ where
}
trait SetupFiles {
fn setup(self, root_path: &SystemPath, project_path: &SystemPath) -> anyhow::Result<()>;
fn setup(self, context: &SetupContext) -> anyhow::Result<()>;
}
struct SetupContext<'a> {
system: &'a OsSystem,
root_path: &'a SystemPath,
}
impl<'a> SetupContext<'a> {
fn system(&self) -> &'a OsSystem {
self.system
}
fn join_project_path(&self, relative: impl AsRef<SystemPath>) -> SystemPathBuf {
self.project_path().join(relative)
}
fn project_path(&self) -> &SystemPath {
self.system.current_directory()
}
fn root_path(&self) -> &'a SystemPath {
self.root_path
}
fn join_root_path(&self, relative: impl AsRef<SystemPath>) -> SystemPathBuf {
self.root_path().join(relative)
}
}
impl<const N: usize, P> SetupFiles for [(P, &'static str); N]
where
P: AsRef<SystemPath>,
{
fn setup(self, _root_path: &SystemPath, project_path: &SystemPath) -> anyhow::Result<()> {
fn setup(self, context: &SetupContext) -> anyhow::Result<()> {
for (relative_path, content) in self {
let relative_path = relative_path.as_ref();
let absolute_path = project_path.join(relative_path);
let absolute_path = context.join_project_path(relative_path);
if let Some(parent) = absolute_path.parent() {
std::fs::create_dir_all(parent).with_context(|| {
format!("Failed to create parent directory for file `{relative_path}`")
@@ -250,10 +280,10 @@ where
impl<F> SetupFiles for F
where
F: FnOnce(&SystemPath, &SystemPath) -> anyhow::Result<()>,
F: FnOnce(&SetupContext) -> anyhow::Result<()>,
{
fn setup(self, root_path: &SystemPath, project_path: &SystemPath) -> anyhow::Result<()> {
self(root_path, project_path)
fn setup(self, context: &SetupContext) -> anyhow::Result<()> {
self(context)
}
}
@@ -261,13 +291,12 @@ fn setup<F>(setup_files: F) -> anyhow::Result<TestCase>
where
F: SetupFiles,
{
setup_with_options(setup_files, |_root, _project_path| None)
setup_with_options(setup_files, |_context| None)
}
// TODO: Replace with configuration?
fn setup_with_options<F>(
setup_files: F,
create_options: impl FnOnce(&SystemPath, &SystemPath) -> Option<Options>,
create_options: impl FnOnce(&SetupContext) -> Option<Options>,
) -> anyhow::Result<TestCase>
where
F: SetupFiles,
@@ -295,13 +324,17 @@ where
std::fs::create_dir_all(project_path.as_std_path())
.with_context(|| format!("Failed to create project directory `{project_path}`"))?;
let system = OsSystem::new(&project_path);
let setup_context = SetupContext {
system: &system,
root_path: &root_path,
};
setup_files
.setup(&root_path, &project_path)
.setup(&setup_context)
.context("Failed to setup test files")?;
let system = OsSystem::new(&project_path);
if let Some(options) = create_options(&root_path, &project_path) {
if let Some(options) = create_options(&setup_context) {
std::fs::write(
project_path.join("pyproject.toml").as_std_path(),
toml::to_string(&PyProject {
@@ -315,7 +348,9 @@ where
.context("Failed to write configuration")?;
}
let project = ProjectMetadata::discover(&project_path, &system)?;
let mut project = ProjectMetadata::discover(&project_path, &system)?;
project.apply_configuration_files(&system)?;
let program_settings = project.to_program_settings(&system);
for path in program_settings
@@ -789,10 +824,12 @@ fn directory_deleted() -> anyhow::Result<()> {
#[test]
fn search_path() -> anyhow::Result<()> {
let mut case = setup_with_options([("bar.py", "import sub.a")], |root_path, _project_path| {
let mut case = setup_with_options([("bar.py", "import sub.a")], |context| {
Some(Options {
environment: Some(EnvironmentOptions {
extra_paths: Some(vec![RelativePathBuf::cli(root_path.join("site_packages"))]),
extra_paths: Some(vec![RelativePathBuf::cli(
context.join_root_path("site_packages"),
)]),
..EnvironmentOptions::default()
}),
..Options::default()
@@ -853,10 +890,12 @@ fn add_search_path() -> anyhow::Result<()> {
#[test]
fn remove_search_path() -> anyhow::Result<()> {
let mut case = setup_with_options([("bar.py", "import sub.a")], |root_path, _project_path| {
let mut case = setup_with_options([("bar.py", "import sub.a")], |context| {
Some(Options {
environment: Some(EnvironmentOptions {
extra_paths: Some(vec![RelativePathBuf::cli(root_path.join("site_packages"))]),
extra_paths: Some(vec![RelativePathBuf::cli(
context.join_root_path("site_packages"),
)]),
..EnvironmentOptions::default()
}),
..Options::default()
@@ -894,7 +933,7 @@ import os
print(sys.last_exc, os.getegid())
"#,
)],
|_root_path, _project_path| {
|_context| {
Some(Options {
environment: Some(EnvironmentOptions {
python_version: Some(RangedValue::cli(PythonVersion::PY311)),
@@ -942,21 +981,31 @@ print(sys.last_exc, os.getegid())
#[test]
fn changed_versions_file() -> anyhow::Result<()> {
let mut case = setup_with_options(
|root_path: &SystemPath, project_path: &SystemPath| {
std::fs::write(project_path.join("bar.py").as_std_path(), "import sub.a")?;
std::fs::create_dir_all(root_path.join("typeshed/stdlib").as_std_path())?;
std::fs::write(root_path.join("typeshed/stdlib/VERSIONS").as_std_path(), "")?;
|context: &SetupContext| {
std::fs::write(
root_path.join("typeshed/stdlib/os.pyi").as_std_path(),
context.join_project_path("bar.py").as_std_path(),
"import sub.a",
)?;
std::fs::create_dir_all(context.join_root_path("typeshed/stdlib").as_std_path())?;
std::fs::write(
context
.join_root_path("typeshed/stdlib/VERSIONS")
.as_std_path(),
"",
)?;
std::fs::write(
context
.join_root_path("typeshed/stdlib/os.pyi")
.as_std_path(),
"# not important",
)?;
Ok(())
},
|root_path, _project_path| {
|context| {
Some(Options {
environment: Some(EnvironmentOptions {
typeshed: Some(RelativePathBuf::cli(root_path.join("typeshed"))),
typeshed: Some(RelativePathBuf::cli(context.join_root_path("typeshed"))),
..EnvironmentOptions::default()
}),
..Options::default()
@@ -1007,12 +1056,12 @@ fn changed_versions_file() -> anyhow::Result<()> {
/// we're seeing is that Windows only emits a single event, similar to Linux.
#[test]
fn hard_links_in_project() -> anyhow::Result<()> {
let mut case = setup(|_root: &SystemPath, project: &SystemPath| {
let foo_path = project.join("foo.py");
let mut case = setup(|context: &SetupContext| {
let foo_path = context.join_project_path("foo.py");
std::fs::write(foo_path.as_std_path(), "print('Version 1')")?;
// Create a hardlink to `foo`
let bar_path = project.join("bar.py");
let bar_path = context.join_project_path("bar.py");
std::fs::hard_link(foo_path.as_std_path(), bar_path.as_std_path())
.context("Failed to create hard link from foo.py -> bar.py")?;
@@ -1078,12 +1127,12 @@ fn hard_links_in_project() -> anyhow::Result<()> {
ignore = "windows doesn't support observing changes to hard linked files."
)]
fn hard_links_to_target_outside_project() -> anyhow::Result<()> {
let mut case = setup(|root: &SystemPath, project: &SystemPath| {
let foo_path = root.join("foo.py");
let mut case = setup(|context: &SetupContext| {
let foo_path = context.join_root_path("foo.py");
std::fs::write(foo_path.as_std_path(), "print('Version 1')")?;
// Create a hardlink to `foo`
let bar_path = project.join("bar.py");
let bar_path = context.join_project_path("bar.py");
std::fs::hard_link(foo_path.as_std_path(), bar_path.as_std_path())
.context("Failed to create hard link from foo.py -> bar.py")?;
@@ -1186,9 +1235,9 @@ mod unix {
ignore = "FSEvents doesn't emit change events for symlinked directories outside of the watched paths."
)]
fn symlink_target_outside_watched_paths() -> anyhow::Result<()> {
let mut case = setup(|root: &SystemPath, project: &SystemPath| {
let mut case = setup(|context: &SetupContext| {
// Set up the symlink target.
let link_target = root.join("bar");
let link_target = context.join_root_path("bar");
std::fs::create_dir_all(link_target.as_std_path())
.context("Failed to create link target directory")?;
let baz_original = link_target.join("baz.py");
@@ -1196,7 +1245,7 @@ mod unix {
.context("Failed to write link target file")?;
// Create a symlink inside the project
let bar = project.join("bar");
let bar = context.join_project_path("bar");
std::os::unix::fs::symlink(link_target.as_std_path(), bar.as_std_path())
.context("Failed to create symlink to bar package")?;
@@ -1267,9 +1316,9 @@ mod unix {
/// ```
#[test]
fn symlink_inside_project() -> anyhow::Result<()> {
let mut case = setup(|_root: &SystemPath, project: &SystemPath| {
let mut case = setup(|context: &SetupContext| {
// Set up the symlink target.
let link_target = project.join("patched/bar");
let link_target = context.join_project_path("patched/bar");
std::fs::create_dir_all(link_target.as_std_path())
.context("Failed to create link target directory")?;
let baz_original = link_target.join("baz.py");
@@ -1277,7 +1326,7 @@ mod unix {
.context("Failed to write link target file")?;
// Create a symlink inside site-packages
let bar_in_project = project.join("bar");
let bar_in_project = context.join_project_path("bar");
std::os::unix::fs::symlink(link_target.as_std_path(), bar_in_project.as_std_path())
.context("Failed to create symlink to bar package")?;
@@ -1358,9 +1407,9 @@ mod unix {
#[test]
fn symlinked_module_search_path() -> anyhow::Result<()> {
let mut case = setup_with_options(
|root: &SystemPath, project: &SystemPath| {
|context: &SetupContext| {
// Set up the symlink target.
let site_packages = root.join("site-packages");
let site_packages = context.join_root_path("site-packages");
let bar = site_packages.join("bar");
std::fs::create_dir_all(bar.as_std_path())
.context("Failed to create bar directory")?;
@@ -1369,7 +1418,8 @@ mod unix {
.context("Failed to write baz.py")?;
// Symlink the site packages in the venv to the global site packages
let venv_site_packages = project.join(".venv/lib/python3.12/site-packages");
let venv_site_packages =
context.join_project_path(".venv/lib/python3.12/site-packages");
std::fs::create_dir_all(venv_site_packages.parent().unwrap())
.context("Failed to create .venv directory")?;
std::os::unix::fs::symlink(
@@ -1380,7 +1430,7 @@ mod unix {
Ok(())
},
|_root, _project| {
|_context| {
Some(Options {
environment: Some(EnvironmentOptions {
extra_paths: Some(vec![RelativePathBuf::cli(
@@ -1450,9 +1500,9 @@ mod unix {
#[test]
fn nested_projects_delete_root() -> anyhow::Result<()> {
let mut case = setup(|root: &SystemPath, project_root: &SystemPath| {
let mut case = setup(|context: &SetupContext| {
std::fs::write(
project_root.join("pyproject.toml").as_std_path(),
context.join_project_path("pyproject.toml").as_std_path(),
r#"
[project]
name = "inner"
@@ -1462,7 +1512,7 @@ fn nested_projects_delete_root() -> anyhow::Result<()> {
)?;
std::fs::write(
root.join("pyproject.toml").as_std_path(),
context.join_root_path("pyproject.toml").as_std_path(),
r#"
[project]
name = "outer"
@@ -1487,3 +1537,79 @@ fn nested_projects_delete_root() -> anyhow::Result<()> {
Ok(())
}
#[test]
fn changes_to_user_configuration() -> anyhow::Result<()> {
let mut _config_dir_override: Option<UserConfigDirectoryOverrideGuard> = None;
let mut case = setup(|context: &SetupContext| {
std::fs::write(
context.join_project_path("pyproject.toml").as_std_path(),
r#"
[project]
name = "test"
"#,
)?;
std::fs::write(
context.join_project_path("foo.py").as_std_path(),
"a = 10 / 0",
)?;
let config_directory = context.join_root_path("home/.config");
std::fs::create_dir_all(config_directory.join("knot").as_std_path())?;
std::fs::write(
config_directory.join("knot/knot.toml").as_std_path(),
r#"
[rules]
division-by-zero = "ignore"
"#,
)?;
_config_dir_override = Some(
context
.system()
.with_user_config_directory(Some(config_directory)),
);
Ok(())
})?;
let foo = case
.system_file(case.project_path("foo.py"))
.expect("foo.py to exist");
let diagnostics = case
.db()
.check_file(foo)
.context("Failed to check project.")?;
assert!(
diagnostics.is_empty(),
"Expected no diagnostics but got: {diagnostics:#?}"
);
// Enable division-by-zero in the user configuration with warning severity
update_file(
case.root_path().join("home/.config/knot/knot.toml"),
r#"
[rules]
division-by-zero = "warn"
"#,
)?;
let changes = case.stop_watch(event_for_file("knot.toml"));
case.apply_changes(changes);
let diagnostics = case
.db()
.check_file(foo)
.context("Failed to check project.")?;
assert!(
diagnostics.len() == 1,
"Expected exactly one diagnostic but got: {diagnostics:#?}"
);
Ok(())
}

View File

@@ -13,7 +13,7 @@ license.workspace = true
[dependencies]
ruff_cache = { workspace = true }
ruff_db = { workspace = true, features = ["os", "cache", "serde"] }
ruff_db = { workspace = true, features = ["cache", "serde"] }
ruff_macros = { workspace = true }
ruff_python_ast = { workspace = true, features = ["serde"] }
ruff_text_size = { workspace = true }
@@ -24,7 +24,7 @@ anyhow = { workspace = true }
crossbeam = { workspace = true }
glob = { workspace = true }
notify = { workspace = true }
pep440_rs = { workspace = true }
pep440_rs = { workspace = true, features = ["version-ranges"] }
rayon = { workspace = true }
rustc-hash = { workspace = true }
salsa = { workspace = true }

View File

@@ -1,7 +1,8 @@
use std::{collections::HashMap, hash::BuildHasher};
use red_knot_python_semantic::{PythonPlatform, PythonVersion, SitePackages};
use red_knot_python_semantic::{PythonPlatform, SitePackages};
use ruff_db::system::SystemPathBuf;
use ruff_python_ast::python_version::PythonVersion;
/// Combine two values, preferring the values in `self`.
///

View File

@@ -114,8 +114,8 @@ impl SemanticDb for ProjectDatabase {
project.is_file_open(self, file)
}
fn rule_selection(&self) -> &RuleSelection {
self.project().rule_selection(self)
fn rule_selection(&self) -> Arc<RuleSelection> {
self.project().rules(self)
}
fn lint_registry(&self) -> &LintRegistry {
@@ -186,7 +186,6 @@ pub(crate) mod tests {
files: Files,
system: TestSystem,
vendored: VendoredFileSystem,
rule_selection: RuleSelection,
project: Option<Project>,
}
@@ -198,7 +197,6 @@ pub(crate) mod tests {
vendored: red_knot_vendored::file_system().clone(),
files: Files::default(),
events: Arc::default(),
rule_selection: RuleSelection::from_registry(&DEFAULT_LINT_REGISTRY),
project: None,
};
@@ -270,8 +268,8 @@ pub(crate) mod tests {
!file.path(self).is_vendored_path()
}
fn rule_selection(&self) -> &RuleSelection {
&self.rule_selection
fn rule_selection(&self) -> Arc<RuleSelection> {
self.project().rules(self)
}
fn lint_registry(&self) -> &LintRegistry {

View File

@@ -8,6 +8,7 @@ use ruff_db::files::{system_path_to_file, File, Files};
use ruff_db::system::walk_directory::WalkState;
use ruff_db::system::SystemPath;
use ruff_db::Db as _;
use ruff_python_ast::PySourceType;
use rustc_hash::FxHashSet;
impl ProjectDatabase {
@@ -47,7 +48,7 @@ impl ProjectDatabase {
if let Some(path) = change.system_path() {
if matches!(
path.file_name(),
Some(".gitignore" | ".ignore" | "ruff.toml" | ".ruff.toml" | "pyproject.toml")
Some(".gitignore" | ".ignore" | "knot.toml" | "pyproject.toml")
) {
// Changes to ignore files or settings can change the project structure or add/remove files.
project_changed = true;
@@ -144,6 +145,12 @@ impl ProjectDatabase {
metadata.apply_cli_options(cli_options.clone());
}
if let Err(error) = metadata.apply_configuration_files(self.system()) {
tracing::error!(
"Failed to apply configuration files, continuing without applying them: {error}"
);
}
let program_settings = metadata.to_program_settings(self.system());
let program = Program::get(self);
@@ -201,9 +208,16 @@ impl ProjectDatabase {
return WalkState::Continue;
}
let mut paths = added_paths.lock().unwrap();
if entry
.path()
.extension()
.and_then(PySourceType::try_from_extension)
.is_some()
{
let mut paths = added_paths.lock().unwrap();
paths.push(entry.into_path());
paths.push(entry.into_path());
}
WalkState::Continue
})

View File

@@ -3,18 +3,18 @@
use crate::metadata::options::OptionDiagnostic;
pub use db::{Db, ProjectDatabase};
use files::{Index, Indexed, IndexedFiles};
use metadata::settings::Settings;
pub use metadata::{ProjectDiscoveryError, ProjectMetadata};
use red_knot_python_semantic::lint::{LintRegistry, LintRegistryBuilder, RuleSelection};
use red_knot_python_semantic::register_lints;
use red_knot_python_semantic::types::check_types;
use ruff_db::diagnostic::{Diagnostic, DiagnosticId, ParseDiagnostic, Severity};
use ruff_db::diagnostic::{Diagnostic, DiagnosticId, ParseDiagnostic, Severity, Span};
use ruff_db::files::{system_path_to_file, File};
use ruff_db::parsed::parsed_module;
use ruff_db::source::{source_text, SourceTextError};
use ruff_db::system::walk_directory::WalkState;
use ruff_db::system::{FileType, SystemPath};
use ruff_python_ast::PySourceType;
use ruff_text_size::TextRange;
use rustc_hash::{FxBuildHasher, FxHashSet};
use salsa::Durability;
use salsa::Setter;
@@ -66,12 +66,22 @@ pub struct Project {
/// The metadata describing the project, including the unresolved options.
#[return_ref]
pub metadata: ProjectMetadata,
/// The resolved project settings.
#[return_ref]
pub settings: Settings,
/// Diagnostics that were generated when resolving the project settings.
#[return_ref]
settings_diagnostics: Vec<OptionDiagnostic>,
}
#[salsa::tracked]
impl Project {
pub fn from_metadata(db: &dyn Db, metadata: ProjectMetadata) -> Self {
Project::builder(metadata)
let (settings, settings_diagnostics) = metadata.options().to_settings(db);
Project::builder(metadata, settings, settings_diagnostics)
.durability(Durability::MEDIUM)
.open_fileset_durability(Durability::LOW)
.file_set_durability(Durability::LOW)
@@ -86,30 +96,37 @@ impl Project {
self.metadata(db).name()
}
/// Returns the resolved linter rules for the project.
///
/// This is a salsa query to prevent re-computing queries if other, unrelated
/// settings change. For example, we don't want that changing the terminal settings
/// invalidates any type checking queries.
#[salsa::tracked]
pub fn rules(self, db: &dyn Db) -> Arc<RuleSelection> {
self.settings(db).to_rules()
}
pub fn reload(self, db: &mut dyn Db, metadata: ProjectMetadata) {
tracing::debug!("Reloading project");
assert_eq!(self.root(db), metadata.root());
if &metadata != self.metadata(db) {
let (settings, settings_diagnostics) = metadata.options().to_settings(db);
if self.settings(db) != &settings {
self.set_settings(db).to(settings);
}
if self.settings_diagnostics(db) != &settings_diagnostics {
self.set_settings_diagnostics(db).to(settings_diagnostics);
}
self.set_metadata(db).to(metadata);
}
self.reload_files(db);
}
pub fn rule_selection(self, db: &dyn Db) -> &RuleSelection {
let (selection, _) = self.rule_selection_with_diagnostics(db);
selection
}
#[salsa::tracked(return_ref)]
fn rule_selection_with_diagnostics(
self,
db: &dyn Db,
) -> (RuleSelection, Vec<OptionDiagnostic>) {
self.metadata(db).options().to_rule_selection(db)
}
/// Checks all open files in the project and its dependencies.
pub(crate) fn check(self, db: &ProjectDatabase) -> Vec<Box<dyn Diagnostic>> {
let project_span = tracing::debug_span!("Project::check");
@@ -118,8 +135,7 @@ impl Project {
tracing::debug!("Checking project '{name}'", name = self.name(db));
let mut diagnostics: Vec<Box<dyn Diagnostic>> = Vec::new();
let (_, options_diagnostics) = self.rule_selection_with_diagnostics(db);
diagnostics.extend(options_diagnostics.iter().map(|diagnostic| {
diagnostics.extend(self.settings_diagnostics(db).iter().map(|diagnostic| {
let diagnostic: Box<dyn Diagnostic> = Box::new(diagnostic.clone());
diagnostic
}));
@@ -151,9 +167,8 @@ impl Project {
}
pub(crate) fn check_file(self, db: &dyn Db, file: File) -> Vec<Box<dyn Diagnostic>> {
let (_, options_diagnostics) = self.rule_selection_with_diagnostics(db);
let mut file_diagnostics: Vec<_> = options_diagnostics
let mut file_diagnostics: Vec<_> = self
.settings_diagnostics(db)
.iter()
.map(|diagnostic| {
let diagnostic: Box<dyn Diagnostic> = Box::new(diagnostic.clone());
@@ -329,7 +344,13 @@ fn check_file_impl(db: &dyn Db, file: File) -> Vec<Box<dyn Diagnostic>> {
boxed
}));
diagnostics.sort_unstable_by_key(|diagnostic| diagnostic.range().unwrap_or_default().start());
diagnostics.sort_unstable_by_key(|diagnostic| {
diagnostic
.span()
.and_then(|span| span.range())
.unwrap_or_default()
.start()
});
diagnostics
}
@@ -442,12 +463,8 @@ impl Diagnostic for IOErrorDiagnostic {
self.error.to_string().into()
}
fn file(&self) -> Option<File> {
Some(self.file)
}
fn range(&self) -> Option<TextRange> {
None
fn span(&self) -> Option<Span> {
Some(Span::from(self.file))
}
fn severity(&self) -> Severity {

View File

@@ -1,3 +1,4 @@
use configuration_file::{ConfigurationFile, ConfigurationFileError};
use red_knot_python_semantic::ProgramSettings;
use ruff_db::system::{System, SystemPath, SystemPathBuf};
use ruff_python_ast::name::Name;
@@ -5,13 +6,15 @@ use std::sync::Arc;
use thiserror::Error;
use crate::combine::Combine;
use crate::metadata::pyproject::{Project, PyProject, PyProjectError};
use crate::metadata::pyproject::{Project, PyProject, PyProjectError, ResolveRequiresPythonError};
use crate::metadata::value::ValueSource;
use options::KnotTomlError;
use options::Options;
mod configuration_file;
pub mod options;
pub mod pyproject;
pub mod settings;
pub mod value;
#[derive(Debug, PartialEq, Eq)]
@@ -23,6 +26,15 @@ pub struct ProjectMetadata {
/// The raw options
pub(super) options: Options,
/// Paths of configurations other than the project's configuration that were combined into [`Self::options`].
///
/// This field stores the paths of the configuration files, mainly for
/// knowing which files to watch for changes.
///
/// The path ordering doesn't imply precedence.
#[cfg_attr(test, serde(skip_serializing_if = "Vec::is_empty"))]
pub(super) extra_configuration_paths: Vec<SystemPathBuf>,
}
impl ProjectMetadata {
@@ -31,12 +43,16 @@ impl ProjectMetadata {
Self {
name,
root,
extra_configuration_paths: Vec::default(),
options: Options::default(),
}
}
/// Loads a project from a `pyproject.toml` file.
pub(crate) fn from_pyproject(pyproject: PyProject, root: SystemPathBuf) -> Self {
pub(crate) fn from_pyproject(
pyproject: PyProject,
root: SystemPathBuf,
) -> Result<Self, ResolveRequiresPythonError> {
Self::from_options(
pyproject
.tool
@@ -49,21 +65,37 @@ impl ProjectMetadata {
/// Loads a project from a set of options with an optional pyproject-project table.
pub(crate) fn from_options(
options: Options,
mut options: Options,
root: SystemPathBuf,
project: Option<&Project>,
) -> Self {
) -> Result<Self, ResolveRequiresPythonError> {
let name = project
.and_then(|project| project.name.as_ref())
.map(|name| Name::new(&***name))
.and_then(|project| project.name.as_deref())
.map(|name| Name::new(&**name))
.unwrap_or_else(|| Name::new(root.file_name().unwrap_or("root")));
// TODO(https://github.com/astral-sh/ruff/issues/15491): Respect requires-python
Self {
// If the `options` don't specify a python version but the `project.requires-python` field is set,
// use that as a lower bound instead.
if let Some(project) = project {
if !options
.environment
.as_ref()
.is_some_and(|env| env.python_version.is_some())
{
if let Some(requires_python) = project.resolve_requires_python_lower_bound()? {
let mut environment = options.environment.unwrap_or_default();
environment.python_version = Some(requires_python);
options.environment = Some(environment);
}
}
}
Ok(Self {
name,
root,
options,
}
extra_configuration_paths: Vec::new(),
})
}
/// Discovers the closest project at `path` and returns its metadata.
@@ -131,19 +163,34 @@ impl ProjectMetadata {
}
tracing::debug!("Found project at '{}'", project_root);
return Ok(ProjectMetadata::from_options(
let metadata = ProjectMetadata::from_options(
options,
project_root.to_path_buf(),
pyproject
.as_ref()
.and_then(|pyproject| pyproject.project.as_ref()),
));
)
.map_err(|err| {
ProjectDiscoveryError::InvalidRequiresPythonConstraint {
source: err,
path: pyproject_path,
}
})?;
return Ok(metadata);
}
if let Some(pyproject) = pyproject {
let has_knot_section = pyproject.knot().is_some();
let metadata =
ProjectMetadata::from_pyproject(pyproject, project_root.to_path_buf());
ProjectMetadata::from_pyproject(pyproject, project_root.to_path_buf())
.map_err(
|err| ProjectDiscoveryError::InvalidRequiresPythonConstraint {
source: err,
path: pyproject_path,
},
)?;
if has_knot_section {
tracing::debug!("Found project at '{}'", project_root);
@@ -191,6 +238,10 @@ impl ProjectMetadata {
&self.options
}
pub fn extra_configuration_paths(&self) -> &[SystemPathBuf] {
&self.extra_configuration_paths
}
pub fn to_program_settings(&self, system: &dyn System) -> ProgramSettings {
self.options.to_program_settings(self.root(), system)
}
@@ -200,9 +251,31 @@ impl ProjectMetadata {
self.options = options.combine(std::mem::take(&mut self.options));
}
/// Combine the project options with the user options where project options take precedence.
pub fn apply_user_options(&mut self, options: Options) {
self.options.combine_with(options);
/// Applies the options from the configuration files to the project's options.
///
/// This includes:
///
/// * The user-level configuration
pub fn apply_configuration_files(
&mut self,
system: &dyn System,
) -> Result<(), ConfigurationFileError> {
if let Some(user) = ConfigurationFile::user(system)? {
tracing::debug!(
"Applying user-level configuration loaded from `{path}`.",
path = user.path()
);
self.apply_configuration_file(user);
}
Ok(())
}
/// Applies a lower-precedence configuration files to the project's options.
fn apply_configuration_file(&mut self, options: ConfigurationFile) {
self.extra_configuration_paths
.push(options.path().to_owned());
self.options.combine_with(options.into_options());
}
}
@@ -222,16 +295,22 @@ pub enum ProjectDiscoveryError {
source: Box<KnotTomlError>,
path: SystemPathBuf,
},
#[error("Invalid `requires-python` version specifier (`{path}`): {source}")]
InvalidRequiresPythonConstraint {
source: ResolveRequiresPythonError,
path: SystemPathBuf,
},
}
#[cfg(test)]
mod tests {
//! Integration tests for project discovery
use crate::snapshot_project;
use anyhow::{anyhow, Context};
use insta::assert_ron_snapshot;
use ruff_db::system::{SystemPathBuf, TestSystem};
use ruff_python_ast::python_version::PythonVersion;
use crate::{ProjectDiscoveryError, ProjectMetadata};
@@ -250,7 +329,15 @@ mod tests {
assert_eq!(project.root(), &*root);
snapshot_project!(project);
with_escaped_paths(|| {
assert_ron_snapshot!(&project, @r#"
ProjectMetadata(
name: Name("app"),
root: "/app",
options: Options(),
)
"#);
});
Ok(())
}
@@ -279,7 +366,16 @@ mod tests {
ProjectMetadata::discover(&root, &system).context("Failed to discover project")?;
assert_eq!(project.root(), &*root);
snapshot_project!(project);
with_escaped_paths(|| {
assert_ron_snapshot!(&project, @r#"
ProjectMetadata(
name: Name("backend"),
root: "/app",
options: Options(),
)
"#);
});
// Discovering the same package from a subdirectory should give the same result
let from_src = ProjectMetadata::discover(&root.join("db"), &system)
@@ -362,7 +458,19 @@ expected `.`, `]`
let sub_project = ProjectMetadata::discover(&root.join("packages/a"), &system)?;
snapshot_project!(sub_project);
with_escaped_paths(|| {
assert_ron_snapshot!(sub_project, @r#"
ProjectMetadata(
name: Name("nested-project"),
root: "/app/packages/a",
options: Options(
src: Some(SrcOptions(
root: Some("src"),
)),
),
)
"#);
});
Ok(())
}
@@ -400,7 +508,19 @@ expected `.`, `]`
let root = ProjectMetadata::discover(&root, &system)?;
snapshot_project!(root);
with_escaped_paths(|| {
assert_ron_snapshot!(root, @r#"
ProjectMetadata(
name: Name("project-root"),
root: "/app",
options: Options(
src: Some(SrcOptions(
root: Some("src"),
)),
),
)
"#);
});
Ok(())
}
@@ -432,7 +552,15 @@ expected `.`, `]`
let sub_project = ProjectMetadata::discover(&root.join("packages/a"), &system)?;
snapshot_project!(sub_project);
with_escaped_paths(|| {
assert_ron_snapshot!(sub_project, @r#"
ProjectMetadata(
name: Name("nested-project"),
root: "/app/packages/a",
options: Options(),
)
"#);
});
Ok(())
}
@@ -467,7 +595,19 @@ expected `.`, `]`
let root = ProjectMetadata::discover(&root.join("packages/a"), &system)?;
snapshot_project!(root);
with_escaped_paths(|| {
assert_ron_snapshot!(root, @r#"
ProjectMetadata(
name: Name("project-root"),
root: "/app",
options: Options(
environment: Some(EnvironmentOptions(
r#python-version: Some("3.10"),
)),
),
)
"#);
});
Ok(())
}
@@ -487,27 +627,304 @@ expected `.`, `]`
(
root.join("pyproject.toml"),
r#"
[project]
name = "super-app"
requires-python = ">=3.12"
[project]
name = "super-app"
requires-python = ">=3.12"
[tool.knot.src]
root = "this_option_is_ignored"
"#,
[tool.knot.src]
root = "this_option_is_ignored"
"#,
),
(
root.join("knot.toml"),
r#"
[src]
root = "src"
"#,
[src]
root = "src"
"#,
),
])
.context("Failed to write files")?;
let root = ProjectMetadata::discover(&root, &system)?;
snapshot_project!(root);
with_escaped_paths(|| {
assert_ron_snapshot!(root, @r#"
ProjectMetadata(
name: Name("super-app"),
root: "/app",
options: Options(
environment: Some(EnvironmentOptions(
r#python-version: Some("3.12"),
)),
src: Some(SrcOptions(
root: Some("src"),
)),
),
)
"#);
});
Ok(())
}
#[test]
fn requires_python_major_minor() -> anyhow::Result<()> {
let system = TestSystem::default();
let root = SystemPathBuf::from("/app");
system
.memory_file_system()
.write_file(
root.join("pyproject.toml"),
r#"
[project]
requires-python = ">=3.12"
"#,
)
.context("Failed to write file")?;
let root = ProjectMetadata::discover(&root, &system)?;
assert_eq!(
root.options
.environment
.unwrap_or_default()
.python_version
.as_deref(),
Some(&PythonVersion::PY312)
);
Ok(())
}
#[test]
fn requires_python_major_only() -> anyhow::Result<()> {
let system = TestSystem::default();
let root = SystemPathBuf::from("/app");
system
.memory_file_system()
.write_file(
root.join("pyproject.toml"),
r#"
[project]
requires-python = ">=3"
"#,
)
.context("Failed to write file")?;
let root = ProjectMetadata::discover(&root, &system)?;
assert_eq!(
root.options
.environment
.unwrap_or_default()
.python_version
.as_deref(),
Some(&PythonVersion::from((3, 0)))
);
Ok(())
}
/// A `requires-python` constraint with major, minor and patch can be simplified
/// to major and minor (e.g. 3.12.1 -> 3.12).
#[test]
fn requires_python_major_minor_patch() -> anyhow::Result<()> {
let system = TestSystem::default();
let root = SystemPathBuf::from("/app");
system
.memory_file_system()
.write_file(
root.join("pyproject.toml"),
r#"
[project]
requires-python = ">=3.12.8"
"#,
)
.context("Failed to write file")?;
let root = ProjectMetadata::discover(&root, &system)?;
assert_eq!(
root.options
.environment
.unwrap_or_default()
.python_version
.as_deref(),
Some(&PythonVersion::PY312)
);
Ok(())
}
#[test]
fn requires_python_beta_version() -> anyhow::Result<()> {
let system = TestSystem::default();
let root = SystemPathBuf::from("/app");
system
.memory_file_system()
.write_file(
root.join("pyproject.toml"),
r#"
[project]
requires-python = ">= 3.13.0b0"
"#,
)
.context("Failed to write file")?;
let root = ProjectMetadata::discover(&root, &system)?;
assert_eq!(
root.options
.environment
.unwrap_or_default()
.python_version
.as_deref(),
Some(&PythonVersion::PY313)
);
Ok(())
}
#[test]
fn requires_python_greater_than_major_minor() -> anyhow::Result<()> {
let system = TestSystem::default();
let root = SystemPathBuf::from("/app");
system
.memory_file_system()
.write_file(
root.join("pyproject.toml"),
r#"
[project]
# This is somewhat nonsensical because 3.12.1 > 3.12 is true.
# That's why simplifying the constraint to >= 3.12 is correct
requires-python = ">3.12"
"#,
)
.context("Failed to write file")?;
let root = ProjectMetadata::discover(&root, &system)?;
assert_eq!(
root.options
.environment
.unwrap_or_default()
.python_version
.as_deref(),
Some(&PythonVersion::PY312)
);
Ok(())
}
/// `python-version` takes precedence if both `requires-python` and `python-version` are configured.
#[test]
fn requires_python_and_python_version() -> anyhow::Result<()> {
let system = TestSystem::default();
let root = SystemPathBuf::from("/app");
system
.memory_file_system()
.write_file(
root.join("pyproject.toml"),
r#"
[project]
requires-python = ">=3.12"
[tool.knot.environment]
python-version = "3.10"
"#,
)
.context("Failed to write file")?;
let root = ProjectMetadata::discover(&root, &system)?;
assert_eq!(
root.options
.environment
.unwrap_or_default()
.python_version
.as_deref(),
Some(&PythonVersion::PY310)
);
Ok(())
}
#[test]
fn requires_python_less_than() -> anyhow::Result<()> {
let system = TestSystem::default();
let root = SystemPathBuf::from("/app");
system
.memory_file_system()
.write_file(
root.join("pyproject.toml"),
r#"
[project]
requires-python = "<3.12"
"#,
)
.context("Failed to write file")?;
let Err(error) = ProjectMetadata::discover(&root, &system) else {
return Err(anyhow!("Expected project discovery to fail because the `requires-python` doesn't specify a lower bound (it only specifies an upper bound)."));
};
assert_error_eq(&error, "Invalid `requires-python` version specifier (`/app/pyproject.toml`): value `<3.12` does not contain a lower bound. Add a lower bound to indicate the minimum compatible Python version (e.g., `>=3.13`) or specify a version in `environment.python-version`.");
Ok(())
}
#[test]
fn requires_python_no_specifiers() -> anyhow::Result<()> {
let system = TestSystem::default();
let root = SystemPathBuf::from("/app");
system
.memory_file_system()
.write_file(
root.join("pyproject.toml"),
r#"
[project]
requires-python = ""
"#,
)
.context("Failed to write file")?;
let Err(error) = ProjectMetadata::discover(&root, &system) else {
return Err(anyhow!("Expected project discovery to fail because the `requires-python` specifiers are empty and don't define a lower bound."));
};
assert_error_eq(&error, "Invalid `requires-python` version specifier (`/app/pyproject.toml`): value `` does not contain a lower bound. Add a lower bound to indicate the minimum compatible Python version (e.g., `>=3.13`) or specify a version in `environment.python-version`.");
Ok(())
}
#[test]
fn requires_python_too_large_major_version() -> anyhow::Result<()> {
let system = TestSystem::default();
let root = SystemPathBuf::from("/app");
system
.memory_file_system()
.write_file(
root.join("pyproject.toml"),
r#"
[project]
requires-python = ">=999.0"
"#,
)
.context("Failed to write file")?;
let Err(error) = ProjectMetadata::discover(&root, &system) else {
return Err(anyhow!("Expected project discovery to fail because of the requires-python major version that is larger than 255."));
};
assert_error_eq(&error, "Invalid `requires-python` version specifier (`/app/pyproject.toml`): The major version `999` is larger than the maximum supported value 255");
Ok(())
}
@@ -517,15 +934,12 @@ expected `.`, `]`
assert_eq!(error.to_string().replace('\\', "/"), message);
}
/// Snapshots a project but with all paths using unix separators.
#[macro_export]
macro_rules! snapshot_project {
($project:expr) => {{
assert_ron_snapshot!($project,{
".root" => insta::dynamic_redaction(|content, _content_path| {
content.as_str().unwrap().replace("\\", "/")
}),
fn with_escaped_paths<R>(f: impl FnOnce() -> R) -> R {
let mut settings = insta::Settings::clone_current();
settings.add_dynamic_redaction(".root", |content, _path| {
content.as_str().unwrap().replace('\\', "/")
});
}};
}
settings.bind(f)
}
}

View File

@@ -0,0 +1,69 @@
use std::sync::Arc;
use ruff_db::system::{System, SystemPath, SystemPathBuf};
use thiserror::Error;
use crate::metadata::value::ValueSource;
use super::options::{KnotTomlError, Options};
/// A `knot.toml` configuration file with the options it contains.
pub(crate) struct ConfigurationFile {
path: SystemPathBuf,
options: Options,
}
impl ConfigurationFile {
/// Loads the user-level configuration file if it exists.
///
/// Returns `None` if the file does not exist or if the concept of user-level configurations
/// doesn't exist on `system`.
pub(crate) fn user(system: &dyn System) -> Result<Option<Self>, ConfigurationFileError> {
let Some(configuration_directory) = system.user_config_directory() else {
return Ok(None);
};
let knot_toml_path = configuration_directory.join("knot").join("knot.toml");
tracing::debug!(
"Searching for a user-level configuration at `{path}`",
path = &knot_toml_path
);
let Ok(knot_toml_str) = system.read_to_string(&knot_toml_path) else {
return Ok(None);
};
match Options::from_toml_str(
&knot_toml_str,
ValueSource::File(Arc::new(knot_toml_path.clone())),
) {
Ok(options) => Ok(Some(Self {
path: knot_toml_path,
options,
})),
Err(error) => Err(ConfigurationFileError::InvalidKnotToml {
source: Box::new(error),
path: knot_toml_path,
}),
}
}
/// Returns the path to the configuration file.
pub(crate) fn path(&self) -> &SystemPath {
&self.path
}
pub(crate) fn into_options(self) -> Options {
self.options
}
}
#[derive(Debug, Error)]
pub enum ConfigurationFileError {
#[error("{path} is not a valid `knot.toml`: {source}")]
InvalidKnotToml {
source: Box<KnotTomlError>,
path: SystemPathBuf,
},
}

View File

@@ -1,20 +1,20 @@
use crate::metadata::value::{RangedValue, RelativePathBuf, ValueSource, ValueSourceGuard};
use crate::Db;
use red_knot_python_semantic::lint::{GetLintError, Level, LintSource, RuleSelection};
use red_knot_python_semantic::{
ProgramSettings, PythonPlatform, PythonVersion, SearchPathSettings, SitePackages,
};
use ruff_db::diagnostic::{Diagnostic, DiagnosticId, Severity};
use ruff_db::files::{system_path_to_file, File};
use red_knot_python_semantic::{ProgramSettings, PythonPlatform, SearchPathSettings, SitePackages};
use ruff_db::diagnostic::{Diagnostic, DiagnosticId, Severity, Span};
use ruff_db::files::system_path_to_file;
use ruff_db::system::{System, SystemPath};
use ruff_macros::Combine;
use ruff_text_size::TextRange;
use ruff_python_ast::python_version::PythonVersion;
use rustc_hash::FxHashMap;
use serde::{Deserialize, Serialize};
use std::borrow::Cow;
use std::fmt::Debug;
use thiserror::Error;
use super::settings::{Settings, TerminalSettings};
/// The options for the project.
#[derive(Debug, Default, Clone, PartialEq, Eq, Combine, Serialize, Deserialize)]
#[serde(rename_all = "kebab-case", deny_unknown_fields)]
@@ -30,6 +30,9 @@ pub struct Options {
/// Configures the enabled lints and their severity.
#[serde(skip_serializing_if = "Option::is_none")]
pub rules: Option<Rules>,
#[serde(skip_serializing_if = "Option::is_none")]
pub terminal: Option<TerminalOptions>,
}
impl Options {
@@ -110,7 +113,22 @@ impl Options {
}
#[must_use]
pub(crate) fn to_rule_selection(&self, db: &dyn Db) -> (RuleSelection, Vec<OptionDiagnostic>) {
pub(crate) fn to_settings(&self, db: &dyn Db) -> (Settings, Vec<OptionDiagnostic>) {
let (rules, diagnostics) = self.to_rule_selection(db);
let mut settings = Settings::new(rules);
if let Some(terminal) = self.terminal.as_ref() {
settings.set_terminal(TerminalSettings {
error_on_warning: terminal.error_on_warning.unwrap_or_default(),
});
}
(settings, diagnostics)
}
#[must_use]
fn to_rule_selection(&self, db: &dyn Db) -> (RuleSelection, Vec<OptionDiagnostic>) {
let registry = db.lint_registry();
let mut diagnostics = Vec::new();
@@ -169,7 +187,14 @@ impl Options {
),
};
diagnostics.push(diagnostic.with_file(file).with_range(rule_name.range()));
let span = file.map(Span::from).map(|span| {
if let Some(range) = rule_name.range() {
span.with_range(range)
} else {
span
}
});
diagnostics.push(diagnostic.with_span(span));
}
}
}
@@ -244,6 +269,16 @@ impl FromIterator<(RangedValue<String>, RangedValue<Level>)> for Rules {
}
}
#[derive(Debug, Default, Clone, Eq, PartialEq, Combine, Serialize, Deserialize)]
#[serde(rename_all = "kebab-case", deny_unknown_fields)]
#[cfg_attr(feature = "schemars", derive(schemars::JsonSchema))]
pub struct TerminalOptions {
/// Use exit code 1 if there are any warning-level diagnostics.
///
/// Defaults to `false`.
pub error_on_warning: Option<bool>,
}
#[cfg(feature = "schemars")]
mod schema {
use crate::DEFAULT_LINT_REGISTRY;
@@ -318,8 +353,7 @@ pub struct OptionDiagnostic {
id: DiagnosticId,
message: String,
severity: Severity,
file: Option<File>,
range: Option<TextRange>,
span: Option<Span>,
}
impl OptionDiagnostic {
@@ -328,21 +362,13 @@ impl OptionDiagnostic {
id,
message,
severity,
file: None,
range: None,
span: None,
}
}
#[must_use]
fn with_file(mut self, file: Option<File>) -> Self {
self.file = file;
self
}
#[must_use]
fn with_range(mut self, range: Option<TextRange>) -> Self {
self.range = range;
self
fn with_span(self, span: Option<Span>) -> Self {
OptionDiagnostic { span, ..self }
}
}
@@ -355,12 +381,8 @@ impl Diagnostic for OptionDiagnostic {
Cow::Borrowed(&self.message)
}
fn file(&self) -> Option<File> {
self.file
}
fn range(&self) -> Option<TextRange> {
self.range
fn span(&self) -> Option<Span> {
self.span.clone()
}
fn severity(&self) -> Severity {

View File

@@ -1,10 +1,11 @@
use pep440_rs::{Version, VersionSpecifiers};
use serde::{Deserialize, Deserializer, Serialize};
use std::ops::Deref;
use thiserror::Error;
use crate::metadata::options::Options;
use crate::metadata::value::{RangedValue, ValueSource, ValueSourceGuard};
use pep440_rs::{release_specifiers_to_ranges, Version, VersionSpecifiers};
use ruff_python_ast::python_version::PythonVersion;
use serde::{Deserialize, Deserializer, Serialize};
use std::collections::Bound;
use std::ops::Deref;
use thiserror::Error;
/// A `pyproject.toml` as specified in PEP 517.
#[derive(Deserialize, Serialize, Debug, Default, Clone)]
@@ -55,6 +56,73 @@ pub struct Project {
pub requires_python: Option<RangedValue<VersionSpecifiers>>,
}
impl Project {
pub(super) fn resolve_requires_python_lower_bound(
&self,
) -> Result<Option<RangedValue<PythonVersion>>, ResolveRequiresPythonError> {
let Some(requires_python) = self.requires_python.as_ref() else {
return Ok(None);
};
tracing::debug!("Resolving requires-python constraint: `{requires_python}`");
let ranges = release_specifiers_to_ranges((**requires_python).clone());
let Some((lower, _)) = ranges.bounding_range() else {
return Ok(None);
};
let version = match lower {
// Ex) `>=3.10.1` -> `>=3.10`
Bound::Included(version) => version,
// Ex) `>3.10.1` -> `>=3.10` or `>3.10` -> `>=3.10`
// The second example looks obscure at first but it is required because
// `3.10.1 > 3.10` is true but we only have two digits here. So including 3.10 is the
// right move. Overall, using `>` without a patch release is most likely bogus.
Bound::Excluded(version) => version,
// Ex) `<3.10` or ``
Bound::Unbounded => {
return Err(ResolveRequiresPythonError::NoLowerBound(
requires_python.to_string(),
))
}
};
// Take the major and minor version
let mut versions = version.release().iter().take(2);
let Some(major) = versions.next().copied() else {
return Ok(None);
};
let minor = versions.next().copied().unwrap_or_default();
tracing::debug!("Resolved requires-python constraint to: {major}.{minor}");
let major =
u8::try_from(major).map_err(|_| ResolveRequiresPythonError::TooLargeMajor(major))?;
let minor =
u8::try_from(minor).map_err(|_| ResolveRequiresPythonError::TooLargeMajor(minor))?;
Ok(Some(
requires_python
.clone()
.map_value(|_| PythonVersion::from((major, minor))),
))
}
}
#[derive(Debug, Error)]
pub enum ResolveRequiresPythonError {
#[error("The major version `{0}` is larger than the maximum supported value 255")]
TooLargeMajor(u64),
#[error("The minor version `{0}` is larger than the maximum supported value 255")]
TooLargeMinor(u64),
#[error("value `{0}` does not contain a lower bound. Add a lower bound to indicate the minimum compatible Python version (e.g., `>=3.13`) or specify a version in `environment.python-version`.")]
NoLowerBound(String),
}
#[derive(Deserialize, Serialize, Debug, Clone, PartialEq, Eq)]
#[serde(rename_all = "kebab-case")]
pub struct Tool {

View File

@@ -0,0 +1,53 @@
use std::sync::Arc;
use red_knot_python_semantic::lint::RuleSelection;
/// The resolved [`super::Options`] for the project.
///
/// Unlike [`super::Options`], the struct has default values filled in and
/// uses representations that are optimized for reads (instead of preserving the source representation).
/// It's also not required that this structure precisely resembles the TOML schema, although
/// it's encouraged to use a similar structure.
///
/// It's worth considering to adding a salsa query for specific settings to
/// limit the blast radius when only some settings change. For example,
/// changing the terminal settings shouldn't invalidate any core type-checking queries.
/// This can be achieved by adding a salsa query for the type checking specific settings.
///
/// Settings that are part of [`red_knot_python_semantic::ProgramSettings`] are not included here.
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct Settings {
rules: Arc<RuleSelection>,
terminal: TerminalSettings,
}
impl Settings {
pub fn new(rules: RuleSelection) -> Self {
Self {
rules: Arc::new(rules),
terminal: TerminalSettings::default(),
}
}
pub fn rules(&self) -> &RuleSelection {
&self.rules
}
pub fn to_rules(&self) -> Arc<RuleSelection> {
self.rules.clone()
}
pub fn terminal(&self) -> &TerminalSettings {
&self.terminal
}
pub fn set_terminal(&mut self, terminal: TerminalSettings) {
self.terminal = terminal;
}
}
#[derive(Debug, Clone, PartialEq, Eq, Default)]
pub struct TerminalSettings {
pub error_on_warning: bool,
}

View File

@@ -118,6 +118,15 @@ impl<T> RangedValue<T> {
self
}
#[must_use]
pub fn map_value<R>(self, f: impl FnOnce(T) -> R) -> RangedValue<R> {
RangedValue {
value: f(self.value),
source: self.source,
range: self.range,
}
}
pub fn into_inner(self) -> T {
self.value
}

View File

@@ -1,13 +0,0 @@
---
source: crates/red_knot_project/src/metadata.rs
expression: root
---
ProjectMetadata(
name: Name("project-root"),
root: "/app",
options: Options(
src: Some(SrcOptions(
root: Some("src"),
)),
),
)

View File

@@ -1,13 +0,0 @@
---
source: crates/red_knot_project/src/metadata.rs
expression: sub_project
---
ProjectMetadata(
name: Name("nested-project"),
root: "/app/packages/a",
options: Options(
src: Some(SrcOptions(
root: Some("src"),
)),
),
)

View File

@@ -1,13 +0,0 @@
---
source: crates/red_knot_project/src/metadata.rs
expression: root
---
ProjectMetadata(
name: Name("project-root"),
root: "/app",
options: Options(
environment: Some(EnvironmentOptions(
r#python-version: Some("3.10"),
)),
),
)

View File

@@ -1,9 +0,0 @@
---
source: crates/red_knot_project/src/metadata.rs
expression: sub_project
---
ProjectMetadata(
name: Name("nested-project"),
root: "/app/packages/a",
options: Options(),
)

View File

@@ -1,13 +0,0 @@
---
source: crates/red_knot_project/src/metadata.rs
expression: root
---
ProjectMetadata(
name: Name("super-app"),
root: "/app",
options: Options(
src: Some(SrcOptions(
root: Some("src"),
)),
),
)

View File

@@ -1,9 +0,0 @@
---
source: crates/red_knot_project/src/metadata.rs
expression: project
---
ProjectMetadata(
name: Name("backend"),
root: "/app",
options: Options(),
)

View File

@@ -1,9 +0,0 @@
---
source: crates/red_knot_project/src/metadata.rs
expression: project
---
ProjectMetadata(
name: Name("app"),
root: "/app",
options: Options(),
)

View File

@@ -73,6 +73,13 @@ impl ProjectWatcher {
.canonicalize_path(&project_path)
.unwrap_or(project_path);
let config_paths = db
.project()
.metadata(db)
.extra_configuration_paths()
.iter()
.cloned();
// Find the non-overlapping module search paths and filter out paths that are already covered by the project.
// Module search paths are already canonicalized.
let unique_module_paths = ruff_db::system::deduplicate_nested_paths(
@@ -83,8 +90,11 @@ impl ProjectWatcher {
.map(SystemPath::to_path_buf);
// Now add the new paths, first starting with the project path and then
// adding the library search paths.
for path in std::iter::once(project_path).chain(unique_module_paths) {
// adding the library search paths, and finally the paths for configurations.
for path in std::iter::once(project_path)
.chain(unique_module_paths)
.chain(config_paths)
{
// Log a warning. It's not worth aborting if registering a single folder fails because
// Ruff otherwise stills works as expected.
if let Err(error) = self.watcher.watch(&path) {

View File

@@ -12,9 +12,9 @@ license = { workspace = true }
[dependencies]
ruff_db = { workspace = true }
ruff_index = { workspace = true }
ruff_index = { workspace = true, features = ["salsa"] }
ruff_macros = { workspace = true }
ruff_python_ast = { workspace = true }
ruff_python_ast = { workspace = true, features = ["salsa"] }
ruff_python_parser = { workspace = true }
ruff_python_stdlib = { workspace = true }
ruff_source_file = { workspace = true }
@@ -31,7 +31,7 @@ drop_bomb = { workspace = true }
indexmap = { workspace = true }
itertools = { workspace = true }
ordermap = { workspace = true }
salsa = { workspace = true }
salsa = { workspace = true, features = ["compact_str"] }
thiserror = { workspace = true }
tracing = { workspace = true }
rustc-hash = { workspace = true }
@@ -44,7 +44,7 @@ test-case = { workspace = true }
memchr = { workspace = true }
[dev-dependencies]
ruff_db = { workspace = true, features = ["os", "testing"] }
ruff_db = { workspace = true, features = ["testing", "os"] }
ruff_python_parser = { workspace = true }
red_knot_test = { workspace = true }
red_knot_vendored = { workspace = true }
@@ -57,7 +57,7 @@ quickcheck = { version = "1.0.3", default-features = false }
quickcheck_macros = { version = "1.0.0" }
[features]
serde = ["ruff_db/serde", "dep:serde"]
serde = ["ruff_db/serde", "dep:serde", "ruff_python_ast/serde"]
[lints]
workspace = true

View File

@@ -0,0 +1,90 @@
# Special cases for int/float/complex in annotations
In order to support common use cases, an annotation of `float` actually means `int | float`, and an
annotation of `complex` actually means `int | float | complex`. See
[the specification](https://typing.readthedocs.io/en/latest/spec/special-types.html#special-cases-for-float-and-complex)
## float
An annotation of `float` means `int | float`, so `int` is assignable to it:
```py
def takes_float(x: float):
pass
def passes_int_to_float(x: int):
# no error!
takes_float(x)
```
It also applies to variable annotations:
```py
def assigns_int_to_float(x: int):
# no error!
y: float = x
```
It doesn't work the other way around:
```py
def takes_int(x: int):
pass
def passes_float_to_int(x: float):
# error: [invalid-argument-type]
takes_int(x)
def assigns_float_to_int(x: float):
# error: [invalid-assignment]
y: int = x
```
Unlike other type checkers, we choose not to obfuscate this special case by displaying `int | float`
as just `float`; we display the actual type:
```py
def f(x: float):
reveal_type(x) # revealed: int | float
```
## complex
An annotation of `complex` means `int | float | complex`, so `int` and `float` are both assignable
to it (but not the other way around):
```py
def takes_complex(x: complex):
pass
def passes_to_complex(x: float, y: int):
# no errors!
takes_complex(x)
takes_complex(y)
def assigns_to_complex(x: float, y: int):
# no errors!
a: complex = x
b: complex = y
def takes_int(x: int):
pass
def takes_float(x: float):
pass
def passes_complex(x: complex):
# error: [invalid-argument-type]
takes_int(x)
# error: [invalid-argument-type]
takes_float(x)
def assigns_complex(x: complex):
# error: [invalid-assignment]
y: int = x
# error: [invalid-assignment]
z: float = x
def f(x: complex):
reveal_type(x) # revealed: int | float | complex
```

View File

@@ -9,9 +9,9 @@ from typing import Union
a: Union[int, str]
a1: Union[int, bool]
a2: Union[int, Union[float, str]]
a2: Union[int, Union[bytes, str]]
a3: Union[int, None]
a4: Union[Union[float, str]]
a4: Union[Union[bytes, str]]
a5: Union[int]
a6: Union[()]
@@ -21,11 +21,11 @@ def f():
# Since bool is a subtype of int we simplify to int here. But we do allow assigning boolean values (see below).
# revealed: int
reveal_type(a1)
# revealed: int | float | str
# revealed: int | bytes | str
reveal_type(a2)
# revealed: int | None
reveal_type(a3)
# revealed: float | str
# revealed: bytes | str
reveal_type(a4)
# revealed: int
reveal_type(a5)

View File

@@ -9,7 +9,7 @@ reveal_type(x) # revealed: Literal[2]
x = 1.0
x /= 2
reveal_type(x) # revealed: float
reveal_type(x) # revealed: int | float
```
## Dunder methods
@@ -24,12 +24,12 @@ x -= 1
reveal_type(x) # revealed: str
class C:
def __iadd__(self, other: str) -> float:
return 1.0
def __iadd__(self, other: str) -> int:
return 1
x = C()
x += "Hello"
reveal_type(x) # revealed: float
reveal_type(x) # revealed: int
```
## Unsupported types
@@ -130,10 +130,10 @@ def _(flag: bool):
if flag:
f = Foo()
else:
f = 42.0
f = 42
f += 12
reveal_type(f) # revealed: str | float
reveal_type(f) # revealed: str | Literal[54]
```
## Partially bound target union with `__add__`

View File

@@ -804,6 +804,67 @@ def _(flag: bool, flag1: bool, flag2: bool):
reveal_type(C.x) # revealed: Unknown | Literal[1, 2, 3]
```
### Attribute possibly unbound on a subclass but not on a superclass
```py
def _(flag: bool):
class Foo:
x = 1
class Bar(Foo):
if flag:
x = 2
reveal_type(Bar.x) # revealed: Unknown | Literal[2, 1]
```
### Attribute possibly unbound on a subclass and on a superclass
```py
def _(flag: bool):
class Foo:
if flag:
x = 1
class Bar(Foo):
if flag:
x = 2
# error: [possibly-unbound-attribute]
reveal_type(Bar.x) # revealed: Unknown | Literal[2, 1]
```
### Attribute access on `Any`
The union of the set of types that `Any` could materialise to is equivalent to `object`. It follows
from this that attribute access on `Any` resolves to `Any` if the attribute does not exist on
`object` -- but if the attribute *does* exist on `object`, the type of the attribute is
`<type as it exists on object> & Any`.
```py
from typing import Any
class Foo(Any): ...
reveal_type(Foo.bar) # revealed: Any
reveal_type(Foo.__repr__) # revealed: Literal[__repr__] & Any
```
Similar principles apply if `Any` appears in the middle of an inheritance hierarchy:
```py
from typing import ClassVar, Literal
class A:
x: ClassVar[Literal[1]] = 1
class B(Any): ...
class C(B, A): ...
reveal_type(C.__mro__) # revealed: tuple[Literal[C], Literal[B], Any, Literal[A], Literal[object]]
reveal_type(C.x) # revealed: Literal[1] & Any
```
### Unions with all paths unbound
If the symbol is unbound in all elements of the union, we detect that:

View File

@@ -56,7 +56,7 @@ def _(a: bool):
reveal_type(x - a) # revealed: int
reveal_type(x * a) # revealed: int
reveal_type(x // a) # revealed: int
reveal_type(x / a) # revealed: float
reveal_type(x / a) # revealed: int | float
reveal_type(x % a) # revealed: int
def rhs_is_int(x: int):
@@ -64,7 +64,7 @@ def _(a: bool):
reveal_type(a - x) # revealed: int
reveal_type(a * x) # revealed: int
reveal_type(a // x) # revealed: int
reveal_type(a / x) # revealed: float
reveal_type(a / x) # revealed: int | float
reveal_type(a % x) # revealed: int
def lhs_is_bool(x: bool):
@@ -72,7 +72,7 @@ def _(a: bool):
reveal_type(x - a) # revealed: int
reveal_type(x * a) # revealed: int
reveal_type(x // a) # revealed: int
reveal_type(x / a) # revealed: float
reveal_type(x / a) # revealed: int | float
reveal_type(x % a) # revealed: int
def rhs_is_bool(x: bool):
@@ -80,7 +80,7 @@ def _(a: bool):
reveal_type(a - x) # revealed: int
reveal_type(a * x) # revealed: int
reveal_type(a // x) # revealed: int
reveal_type(a / x) # revealed: float
reveal_type(a / x) # revealed: int | float
reveal_type(a % x) # revealed: int
def both_are_bool(x: bool, y: bool):
@@ -88,6 +88,6 @@ def _(a: bool):
reveal_type(x - y) # revealed: int
reveal_type(x * y) # revealed: int
reveal_type(x // y) # revealed: int
reveal_type(x / y) # revealed: float
reveal_type(x / y) # revealed: int | float
reveal_type(x % y) # revealed: int
```

View File

@@ -268,23 +268,28 @@ reveal_type(B() + B()) # revealed: Unknown | int
## Integration test: numbers from typeshed
We get less precise results from binary operations on float/complex literals due to the special case
for annotations of `float` or `complex`, which applies also to return annotations for typeshed
dunder methods. Perhaps we could have a special-case on the special-case, to exclude these typeshed
return annotations from the widening, and preserve a bit more precision here?
```py
reveal_type(3j + 3.14) # revealed: complex
reveal_type(4.2 + 42) # revealed: float
reveal_type(3j + 3) # revealed: complex
reveal_type(3j + 3.14) # revealed: int | float | complex
reveal_type(4.2 + 42) # revealed: int | float
reveal_type(3j + 3) # revealed: int | float | complex
# TODO should be complex, need to check arg type and fall back to `rhs.__radd__`
reveal_type(3.14 + 3j) # revealed: float
# TODO should be int | float | complex, need to check arg type and fall back to `rhs.__radd__`
reveal_type(3.14 + 3j) # revealed: int | float
# TODO should be float, need to check arg type and fall back to `rhs.__radd__`
# TODO should be int | float, need to check arg type and fall back to `rhs.__radd__`
reveal_type(42 + 4.2) # revealed: int
# TODO should be complex, need to check arg type and fall back to `rhs.__radd__`
# TODO should be int | float | complex, need to check arg type and fall back to `rhs.__radd__`
reveal_type(3 + 3j) # revealed: int
def _(x: bool, y: int):
reveal_type(x + y) # revealed: int
reveal_type(4.2 + x) # revealed: float
reveal_type(4.2 + x) # revealed: int | float
# TODO should be float, need to check arg type and fall back to `rhs.__radd__`
reveal_type(y + 4.12) # revealed: int

View File

@@ -19,7 +19,7 @@ def lhs(x: int):
reveal_type(x - 4) # revealed: int
reveal_type(x * -1) # revealed: int
reveal_type(x // 3) # revealed: int
reveal_type(x / 3) # revealed: float
reveal_type(x / 3) # revealed: int | float
reveal_type(x % 3) # revealed: int
def rhs(x: int):
@@ -27,7 +27,7 @@ def rhs(x: int):
reveal_type(3 - x) # revealed: int
reveal_type(3 * x) # revealed: int
reveal_type(-3 // x) # revealed: int
reveal_type(-3 / x) # revealed: float
reveal_type(-3 / x) # revealed: int | float
reveal_type(5 % x) # revealed: int
def both(x: int):
@@ -35,7 +35,7 @@ def both(x: int):
reveal_type(x - x) # revealed: int
reveal_type(x * x) # revealed: int
reveal_type(x // x) # revealed: int
reveal_type(x / x) # revealed: float
reveal_type(x / x) # revealed: int | float
reveal_type(x % x) # revealed: int
```
@@ -80,24 +80,20 @@ c = 3 % 0 # error: "Cannot reduce object of type `Literal[3]` modulo zero"
reveal_type(c) # revealed: int
# error: "Cannot divide object of type `int` by zero"
# revealed: float
reveal_type(int() / 0)
reveal_type(int() / 0) # revealed: int | float
# error: "Cannot divide object of type `Literal[1]` by zero"
# revealed: float
reveal_type(1 / False)
reveal_type(1 / False) # revealed: float
# error: [division-by-zero] "Cannot divide object of type `Literal[True]` by zero"
True / False
# error: [division-by-zero] "Cannot divide object of type `Literal[True]` by zero"
bool(1) / False
# error: "Cannot divide object of type `float` by zero"
# revealed: float
reveal_type(1.0 / 0)
reveal_type(1.0 / 0) # revealed: int | float
class MyInt(int): ...
# No error for a subclass of int
# revealed: float
reveal_type(MyInt(3) / 0)
reveal_type(MyInt(3) / 0) # revealed: int | float
```

View File

@@ -4,14 +4,14 @@
```py
class Multiplier:
def __init__(self, factor: float):
def __init__(self, factor: int):
self.factor = factor
def __call__(self, number: float) -> float:
def __call__(self, number: int) -> int:
return number * self.factor
a = Multiplier(2.0)(3.0)
reveal_type(a) # revealed: float
a = Multiplier(2)(3)
reveal_type(a) # revealed: int
class Unit: ...

View File

@@ -20,8 +20,8 @@ class A:
def __eq__(self, other: A) -> int:
return 42
def __ne__(self, other: A) -> float:
return 42.0
def __ne__(self, other: A) -> bytearray:
return bytearray()
def __lt__(self, other: A) -> str:
return "42"
@@ -36,7 +36,7 @@ class A:
return {42}
reveal_type(A() == A()) # revealed: int
reveal_type(A() != A()) # revealed: float
reveal_type(A() != A()) # revealed: bytearray
reveal_type(A() < A()) # revealed: str
reveal_type(A() <= A()) # revealed: bytes
reveal_type(A() > A()) # revealed: list
@@ -55,8 +55,8 @@ class A:
def __eq__(self, other: B) -> int:
return 42
def __ne__(self, other: B) -> float:
return 42.0
def __ne__(self, other: B) -> bytearray:
return bytearray()
def __lt__(self, other: B) -> str:
return "42"
@@ -73,7 +73,7 @@ class A:
class B: ...
reveal_type(A() == B()) # revealed: int
reveal_type(A() != B()) # revealed: float
reveal_type(A() != B()) # revealed: bytearray
reveal_type(A() < B()) # revealed: str
reveal_type(A() <= B()) # revealed: bytes
reveal_type(A() > B()) # revealed: list
@@ -93,8 +93,8 @@ class A:
def __eq__(self, other: B) -> int:
return 42
def __ne__(self, other: B) -> float:
return 42.0
def __ne__(self, other: B) -> bytearray:
return bytearray()
def __lt__(self, other: B) -> str:
return "42"
@@ -117,7 +117,7 @@ class B:
def __ne__(self, other: str) -> B:
return B()
# TODO: should be `int` and `float`.
# TODO: should be `int` and `bytearray`.
# Need to check arg type and fall back to `rhs.__eq__` and `rhs.__ne__`.
#
# Because `object.__eq__` and `object.__ne__` accept `object` in typeshed,
@@ -136,11 +136,11 @@ class C:
def __gt__(self, other: C) -> int:
return 42
def __ge__(self, other: C) -> float:
return 42.0
def __ge__(self, other: C) -> bytearray:
return bytearray()
reveal_type(C() < C()) # revealed: int
reveal_type(C() <= C()) # revealed: float
reveal_type(C() <= C()) # revealed: bytearray
```
## Reflected Comparisons with Subclasses
@@ -175,8 +175,8 @@ class B(A):
def __eq__(self, other: A) -> int:
return 42
def __ne__(self, other: A) -> float:
return 42.0
def __ne__(self, other: A) -> bytearray:
return bytearray()
def __lt__(self, other: A) -> str:
return "42"
@@ -191,7 +191,7 @@ class B(A):
return {42}
reveal_type(A() == B()) # revealed: int
reveal_type(A() != B()) # revealed: float
reveal_type(A() != B()) # revealed: bytearray
reveal_type(A() < B()) # revealed: list
reveal_type(A() <= B()) # revealed: set

View File

@@ -151,11 +151,11 @@ class A:
def __ne__(self, o: object) -> bytes:
return b"world"
def __lt__(self, o: A) -> float:
return 3.14
def __lt__(self, o: A) -> bytearray:
return bytearray()
def __le__(self, o: A) -> complex:
return complex(0.5, -0.5)
def __le__(self, o: A) -> memoryview:
return memoryview(b"")
def __gt__(self, o: A) -> tuple:
return (1, 2, 3)
@@ -167,8 +167,8 @@ a = (A(), A())
reveal_type(a == a) # revealed: bool
reveal_type(a != a) # revealed: bool
reveal_type(a < a) # revealed: float | Literal[False]
reveal_type(a <= a) # revealed: complex | Literal[True]
reveal_type(a < a) # revealed: bytearray | Literal[False]
reveal_type(a <= a) # revealed: memoryview | Literal[True]
reveal_type(a > a) # revealed: tuple | Literal[False]
reveal_type(a >= a) # revealed: list | Literal[True]
@@ -187,7 +187,7 @@ class B:
def __lt__(self, o: B) -> set:
return set()
reveal_type((A(), B()) < (A(), B())) # revealed: float | set | Literal[False]
reveal_type((A(), B()) < (A(), B())) # revealed: bytearray | set | Literal[False]
```
#### Special Handling of Eq and NotEq in Lexicographic Comparisons

View File

@@ -0,0 +1,2 @@
This directory contains user-facing documentation, but also doubles as an extended test suite that
makes sure that our documentation stays up to date.

View File

@@ -0,0 +1,125 @@
# Public type of undeclared symbols
## Summary
One major deviation from the behavior of existing Python type checkers is our handling of 'public'
types for undeclared symbols. This is best illustrated with an example:
```py
class Wrapper:
value = None
wrapper = Wrapper()
reveal_type(wrapper.value) # revealed: Unknown | None
wrapper.value = 1
```
Mypy and Pyright both infer a type of `None` for the type of `wrapper.value`. Consequently, both
tools emit an error when trying to assign `1` to `wrapper.value`. But there is nothing wrong with
this program. Emitting an error here violates the [gradual guarantee] which states that *"Removing
type annotations (making the program more dynamic) should not result in additional static type
errors."*: If `value` were annotated with `int | None` here, Mypy and Pyright would not emit any
errors.
By inferring `Unknown | None` instead, we allow arbitrary values to be assigned to `wrapper.value`.
This is a deliberate choice to prevent false positive errors on untyped code.
More generally, we infer `Unknown | T_inferred` for undeclared symbols, where `T_inferred` is the
inferred type of the right-hand side of the assignment. This gradual type represents an *unknown*
fully-static type that is *at least as large as* `T_inferred`. It accurately describes our static
knowledge about this type. In the example above, we don't know what values `wrapper.value` could
possibly contain, but we *do know* that `None` is a possibility. This allows us to catch errors
where `wrapper.value` is used in a way that is incompatible with `None`:
```py
def accepts_int(i: int) -> None:
pass
def f(w: Wrapper) -> None:
# This is fine
v: int | None = w.value
# This function call is incorrect, because `w.value` could be `None`. We therefore emit the following
# error: "`Unknown | None` cannot be assigned to parameter 1 (`i`) of function `accepts_int`; expected type `int`"
c = accepts_int(w.value)
```
## Explicit lack of knowledge
The following example demonstrates how Mypy and Pyright's type inference of fully-static types in
these situations can lead to false-negatives, even though everything appears to be (statically)
typed. To make this a bit more realistic, imagine that `OptionalInt` is imported from an external,
untyped module:
`optional_int.py`:
```py
class OptionalInt:
value = 10
def reset(o):
o.value = None
```
It is then used like this:
```py
from optional_int import OptionalInt, reset
o = OptionalInt()
reset(o) # Oh no...
# Mypy and Pyright infer a fully-static type of `int` here, which appears to make the
# subsequent division operation safe -- but it is not. We infer the following type:
reveal_type(o.value) # revealed: Unknown | Literal[10]
print(o.value // 2) # Runtime error!
```
We do not catch this mistake either, but we accurately reflect our lack of knowledge about
`o.value`. Together with a possible future type-checker mode that would detect the prevalence of
dynamic types, this could help developers catch such mistakes.
## Stricter behavior
Users can always opt in to stricter behavior by adding type annotations. For the `OptionalInt`
class, this would probably be:
```py
class OptionalInt:
value: int | None = 10
o = OptionalInt()
# The following public type is now
# revealed: int | None
reveal_type(o.value)
# Incompatible assignments are now caught:
# error: "Object of type `Literal["a"]` is not assignable to attribute `value` of type `int | None`"
o.value = "a"
```
## What is meant by 'public' type?
We apply different semantics depending on whether a symbol is accessed from the same scope in which
it was originally defined, or whether it is accessed from an external scope. External scopes will
see the symbol's "public type", which has been discussed above. But within the same scope the symbol
was defined in, we use a narrower type of `T_inferred` for undeclared symbols. This is because, from
the perspective of this scope, there is no way that the value of the symbol could have been
reassigned from external scopes. For example:
```py
class Wrapper:
value = None
# Type as seen from the same scope:
reveal_type(value) # revealed: None
# Type as seen from another scope:
reveal_type(Wrapper.value) # revealed: Unknown | None
```
[gradual guarantee]: https://typing.readthedocs.io/en/latest/spec/concepts.html#the-gradual-guarantee

View File

@@ -303,8 +303,8 @@ An example with multiple `except` branches and a `finally` branch:
def could_raise_returns_memoryview() -> memoryview:
return memoryview(b"")
def could_raise_returns_float() -> float:
return 3.14
def could_raise_returns_bytearray() -> bytearray:
return bytearray()
x = 1
@@ -322,13 +322,13 @@ except ValueError:
reveal_type(x) # revealed: Literal[1] | str
x = could_raise_returns_memoryview()
reveal_type(x) # revealed: memoryview
x = could_raise_returns_float()
reveal_type(x) # revealed: float
x = could_raise_returns_bytearray()
reveal_type(x) # revealed: bytearray
finally:
# TODO: should be `Literal[1] | str | bytes | bool | memoryview | float`
reveal_type(x) # revealed: str | bool | float
# TODO: should be `Literal[1] | str | bytes | bool | memoryview | bytearray`
reveal_type(x) # revealed: str | bool | bytearray
reveal_type(x) # revealed: str | bool | float
reveal_type(x) # revealed: str | bool | bytearray
```
## Combining `except`, `else` and `finally` branches
@@ -350,8 +350,8 @@ def could_raise_returns_bool() -> bool:
def could_raise_returns_memoryview() -> memoryview:
return memoryview(b"")
def could_raise_returns_float() -> float:
return 3.14
def could_raise_returns_bytearray() -> bytearray:
return bytearray()
x = 1
@@ -369,13 +369,13 @@ else:
reveal_type(x) # revealed: str
x = could_raise_returns_memoryview()
reveal_type(x) # revealed: memoryview
x = could_raise_returns_float()
reveal_type(x) # revealed: float
x = could_raise_returns_bytearray()
reveal_type(x) # revealed: bytearray
finally:
# TODO: should be `Literal[1] | str | bytes | bool | memoryview | float`
reveal_type(x) # revealed: bool | float
# TODO: should be `Literal[1] | str | bytes | bool | memoryview | bytearray`
reveal_type(x) # revealed: bool | bytearray
reveal_type(x) # revealed: bool | float
reveal_type(x) # revealed: bool | bytearray
```
The same again, this time with multiple `except` branches:
@@ -403,8 +403,8 @@ except ValueError:
reveal_type(x) # revealed: Literal[1] | str
x = could_raise_returns_memoryview()
reveal_type(x) # revealed: memoryview
x = could_raise_returns_float()
reveal_type(x) # revealed: float
x = could_raise_returns_bytearray()
reveal_type(x) # revealed: bytearray
else:
reveal_type(x) # revealed: str
x = could_raise_returns_range()
@@ -412,10 +412,10 @@ else:
x = could_raise_returns_slice()
reveal_type(x) # revealed: slice
finally:
# TODO: should be `Literal[1] | str | bytes | bool | memoryview | float | range | slice`
reveal_type(x) # revealed: bool | float | slice
# TODO: should be `Literal[1] | str | bytes | bool | memoryview | bytearray | range | slice`
reveal_type(x) # revealed: bool | bytearray | slice
reveal_type(x) # revealed: bool | float | slice
reveal_type(x) # revealed: bool | bytearray | slice
```
## Nested `try`/`except` blocks
@@ -441,8 +441,8 @@ def could_raise_returns_bool() -> bool:
def could_raise_returns_memoryview() -> memoryview:
return memoryview(b"")
def could_raise_returns_float() -> float:
return 3.14
def could_raise_returns_property() -> property:
return property()
def could_raise_returns_range() -> range:
return range(42)
@@ -450,8 +450,8 @@ def could_raise_returns_range() -> range:
def could_raise_returns_slice() -> slice:
return slice(None)
def could_raise_returns_complex() -> complex:
return 3j
def could_raise_returns_super() -> super:
return super()
def could_raise_returns_bytearray() -> bytearray:
return bytearray()
@@ -482,8 +482,8 @@ try:
reveal_type(x) # revealed: Literal[1] | str
x = could_raise_returns_memoryview()
reveal_type(x) # revealed: memoryview
x = could_raise_returns_float()
reveal_type(x) # revealed: float
x = could_raise_returns_property()
reveal_type(x) # revealed: property
else:
reveal_type(x) # revealed: str
x = could_raise_returns_range()
@@ -491,15 +491,15 @@ try:
x = could_raise_returns_slice()
reveal_type(x) # revealed: slice
finally:
# TODO: should be `Literal[1] | str | bytes | bool | memoryview | float | range | slice`
reveal_type(x) # revealed: bool | float | slice
# TODO: should be `Literal[1] | str | bytes | bool | memoryview | property | range | slice`
reveal_type(x) # revealed: bool | property | slice
x = 2
reveal_type(x) # revealed: Literal[2]
reveal_type(x) # revealed: Literal[2]
except:
reveal_type(x) # revealed: Literal[1, 2] | str | bytes | bool | memoryview | float | range | slice
x = could_raise_returns_complex()
reveal_type(x) # revealed: complex
reveal_type(x) # revealed: Literal[1, 2] | str | bytes | bool | memoryview | property | range | slice
x = could_raise_returns_super()
reveal_type(x) # revealed: super
x = could_raise_returns_bytearray()
reveal_type(x) # revealed: bytearray
else:
@@ -509,7 +509,7 @@ else:
x = could_raise_returns_Bar()
reveal_type(x) # revealed: Bar
finally:
# TODO: should be `Literal[1, 2] | str | bytes | bool | memoryview | float | range | slice | complex | bytearray | Foo | Bar`
# TODO: should be `Literal[1, 2] | str | bytes | bool | memoryview | property | range | slice | super | bytearray | Foo | Bar`
reveal_type(x) # revealed: bytearray | Bar
# Either one `except` branch or the `else`
@@ -535,8 +535,8 @@ def could_raise_returns_range() -> range:
def could_raise_returns_bytearray() -> bytearray:
return bytearray()
def could_raise_returns_float() -> float:
return 3.14
def could_raise_returns_memoryview() -> memoryview:
return memoryview(b"")
x = 1
@@ -553,12 +553,12 @@ try:
reveal_type(x) # revealed: str | bytes
x = could_raise_returns_bytearray()
reveal_type(x) # revealed: bytearray
x = could_raise_returns_float()
reveal_type(x) # revealed: float
x = could_raise_returns_memoryview()
reveal_type(x) # revealed: memoryview
finally:
# TODO: should be `str | bytes | bytearray | float`
reveal_type(x) # revealed: bytes | float
reveal_type(x) # revealed: bytes | float
# TODO: should be `str | bytes | bytearray | memoryview`
reveal_type(x) # revealed: bytes | memoryview
reveal_type(x) # revealed: bytes | memoryview
x = foo
reveal_type(x) # revealed: Literal[foo]
except:

View File

@@ -0,0 +1,371 @@
# Import conventions
This document describes the conventions for importing symbols.
Reference:
- <https://typing.readthedocs.io/en/latest/spec/distributing.html#import-conventions>
## Builtins scope
When looking up for a name, red knot will fallback to using the builtins scope if the name is not
found in the global scope. The `builtins.pyi` file, that will be used to resolve any symbol in the
builtins scope, contains multiple symbols from other modules (e.g., `typing`) but those are not
re-exported.
```py
# These symbols are being imported in `builtins.pyi` but shouldn't be considered as being
# available in the builtins scope.
# error: "Name `Literal` used when not defined"
reveal_type(Literal) # revealed: Unknown
# error: "Name `sys` used when not defined"
reveal_type(sys) # revealed: Unknown
```
## Builtins import
Similarly, trying to import the symbols from the builtins module which aren't re-exported should
also raise an error.
```py
# error: "Module `builtins` has no member `Literal`"
# error: "Module `builtins` has no member `sys`"
from builtins import Literal, sys
reveal_type(Literal) # revealed: Unknown
reveal_type(sys) # revealed: Unknown
# error: "Module `math` has no member `Iterable`"
from math import Iterable
reveal_type(Iterable) # revealed: Unknown
```
## Re-exported symbols in stub files
When a symbol is re-exported, importing it should not raise an error. This tests both `import ...`
and `from ... import ...` forms.
Note: Submodule imports in `import ...` form doesn't work because it's a syntax error. For example,
in `import os.path as os.path` the `os.path` is not a valid identifier.
```py
from b import Any, Literal, foo
reveal_type(Any) # revealed: typing.Any
reveal_type(Literal) # revealed: typing.Literal
reveal_type(foo) # revealed: <module 'foo'>
```
`b.pyi`:
```pyi
import foo as foo
from typing import Any as Any, Literal as Literal
```
`foo.py`:
```py
```
## Non-exported symbols in stub files
Here, none of the symbols are being re-exported in the stub file.
```py
# error: 15 [unresolved-import] "Module `b` has no member `foo`"
# error: 20 [unresolved-import] "Module `b` has no member `Any`"
# error: 25 [unresolved-import] "Module `b` has no member `Literal`"
from b import foo, Any, Literal
reveal_type(Any) # revealed: Unknown
reveal_type(Literal) # revealed: Unknown
reveal_type(foo) # revealed: Unknown
```
`b.pyi`:
```pyi
import foo
from typing import Any, Literal
```
`foo.pyi`:
```pyi
```
## Nested non-exports
Here, a chain of modules all don't re-export an import.
```py
# error: "Module `a` has no member `Any`"
from a import Any
reveal_type(Any) # revealed: Unknown
```
`a.pyi`:
```pyi
# error: "Module `b` has no member `Any`"
from b import Any
reveal_type(Any) # revealed: Unknown
```
`b.pyi`:
```pyi
# error: "Module `c` has no member `Any`"
from c import Any
reveal_type(Any) # revealed: Unknown
```
`c.pyi`:
```pyi
from typing import Any
reveal_type(Any) # revealed: typing.Any
```
## Nested mixed re-export and not
But, if the symbol is being re-exported explicitly in one of the modules in the chain, it should not
raise an error at that step in the chain.
```py
# error: "Module `a` has no member `Any`"
from a import Any
reveal_type(Any) # revealed: Unknown
```
`a.pyi`:
```pyi
from b import Any
reveal_type(Any) # revealed: Unknown
```
`b.pyi`:
```pyi
# error: "Module `c` has no member `Any`"
from c import Any as Any
reveal_type(Any) # revealed: Unknown
```
`c.pyi`:
```pyi
from typing import Any
reveal_type(Any) # revealed: typing.Any
```
## Exported as different name
The re-export convention only works when the aliased name is exactly the same as the original name.
```py
# error: "Module `a` has no member `Foo`"
from a import Foo
reveal_type(Foo) # revealed: Unknown
```
`a.pyi`:
```pyi
from b import AnyFoo as Foo
reveal_type(Foo) # revealed: Literal[AnyFoo]
```
`b.pyi`:
```pyi
class AnyFoo: ...
```
## Exported using `__all__`
Here, the symbol is re-exported using the `__all__` variable.
```py
# TODO: This should *not* be an error but we don't understand `__all__` yet.
# error: "Module `a` has no member `Foo`"
from a import Foo
```
`a.pyi`:
```pyi
from b import Foo
__all__ = ['Foo']
```
`b.pyi`:
```pyi
class Foo: ...
```
## Re-exports in `__init__.pyi`
Similarly, for an `__init__.pyi` (stub) file, importing a non-exported name should raise an error
but the inference would be `Unknown`.
```py
# error: 15 "Module `a` has no member `Foo`"
# error: 20 "Module `a` has no member `c`"
from a import Foo, c, foo
reveal_type(Foo) # revealed: Unknown
reveal_type(c) # revealed: Unknown
reveal_type(foo) # revealed: <module 'a.foo'>
```
`a/__init__.pyi`:
```pyi
from .b import c
from .foo import Foo
```
`a/foo.pyi`:
```pyi
class Foo: ...
```
`a/b/__init__.pyi`:
```pyi
```
`a/b/c.pyi`:
```pyi
```
## Conditional re-export in stub file
The following scenarios are when a re-export happens conditionally in a stub file.
### Global import
```py
# error: "Member `Foo` of module `a` is possibly unbound"
from a import Foo
reveal_type(Foo) # revealed: str
```
`a.pyi`:
```pyi
from b import Foo
def coinflip() -> bool: ...
if coinflip():
Foo: str = ...
reveal_type(Foo) # revealed: Literal[Foo] | str
```
`b.pyi`:
```pyi
class Foo: ...
```
### Both branch is an import
Here, both the branches of the condition are import statements where one of them re-exports while
the other does not.
```py
# error: "Member `Foo` of module `a` is possibly unbound"
from a import Foo
reveal_type(Foo) # revealed: Literal[Foo]
```
`a.pyi`:
```pyi
def coinflip() -> bool: ...
if coinflip():
from b import Foo
else:
from b import Foo as Foo
reveal_type(Foo) # revealed: Literal[Foo]
```
`b.pyi`:
```pyi
class Foo: ...
```
### Re-export in one branch
```py
# error: "Member `Foo` of module `a` is possibly unbound"
from a import Foo
reveal_type(Foo) # revealed: Literal[Foo]
```
`a.pyi`:
```pyi
def coinflip() -> bool: ...
if coinflip():
from b import Foo as Foo
```
`b.pyi`:
```pyi
class Foo: ...
```
### Non-export in one branch
```py
# error: "Module `a` has no member `Foo`"
from a import Foo
reveal_type(Foo) # revealed: Unknown
```
`a.pyi`:
```pyi
def coinflip() -> bool: ...
if coinflip():
from b import Foo
```
`b.pyi`:
```pyi
class Foo: ...
```

View File

@@ -11,11 +11,15 @@ See the [typing documentation] for more information.
- `bool` is a subtype of `int`. This is modeled after Python's runtime behavior, where `int` is a
supertype of `bool` (present in `bool`s bases and MRO).
- `int` is not a subtype of `float`/`complex`, even though `float`/`complex` can be used in place of
`int` in some contexts (see [special case for float and complex]).
- `int` is not a subtype of `float`/`complex`, although this is muddied by the
[special case for float and complex] where annotations of `float` and `complex` are interpreted
as `int | float` and `int | float | complex`, respectively.
```py
from knot_extensions import is_subtype_of, static_assert
from knot_extensions import is_subtype_of, static_assert, TypeOf
type JustFloat = TypeOf[1.0]
type JustComplex = TypeOf[1j]
static_assert(is_subtype_of(bool, bool))
static_assert(is_subtype_of(bool, int))
@@ -30,8 +34,8 @@ static_assert(not is_subtype_of(int, bool))
static_assert(not is_subtype_of(int, str))
static_assert(not is_subtype_of(object, int))
static_assert(not is_subtype_of(int, float))
static_assert(not is_subtype_of(int, complex))
static_assert(not is_subtype_of(int, JustFloat))
static_assert(not is_subtype_of(int, JustComplex))
static_assert(is_subtype_of(TypeError, Exception))
static_assert(is_subtype_of(FloatingPointError, Exception))
@@ -79,7 +83,9 @@ static_assert(is_subtype_of(C, object))
```py
from typing_extensions import Literal, LiteralString
from knot_extensions import is_subtype_of, static_assert
from knot_extensions import is_subtype_of, static_assert, TypeOf
type JustFloat = TypeOf[1.0]
# Boolean literals
static_assert(is_subtype_of(Literal[True], bool))
@@ -92,8 +98,7 @@ static_assert(is_subtype_of(Literal[1], object))
static_assert(not is_subtype_of(Literal[1], bool))
# See the note above (or link below) concerning int and float/complex
static_assert(not is_subtype_of(Literal[1], float))
static_assert(not is_subtype_of(Literal[1], JustFloat))
# String literals
static_assert(is_subtype_of(Literal["foo"], LiteralString))

View File

@@ -37,6 +37,31 @@ def noreturn(u1: int | NoReturn, u2: int | NoReturn | str) -> None:
reveal_type(u2) # revealed: int | str
```
## `object` subsumes everything
Unions with `object` can be simplified to `object`:
```py
from typing_extensions import Never, Any
def _(
u1: int | object,
u2: object | int,
u3: Any | object,
u4: object | Any,
u5: object | Never,
u6: Never | object,
u7: int | str | object | bytes | Any,
) -> None:
reveal_type(u1) # revealed: object
reveal_type(u2) # revealed: object
reveal_type(u3) # revealed: object
reveal_type(u4) # revealed: object
reveal_type(u5) # revealed: object
reveal_type(u6) # revealed: object
reveal_type(u7) # revealed: object
```
## Flattening of nested unions
```py
@@ -45,11 +70,11 @@ from typing import Literal
def _(
u1: (int | str) | bytes,
u2: int | (str | bytes),
u3: int | (str | (bytes | complex)),
u3: int | (str | (bytes | bytearray)),
) -> None:
reveal_type(u1) # revealed: int | str | bytes
reveal_type(u2) # revealed: int | str | bytes
reveal_type(u3) # revealed: int | str | bytes | complex
reveal_type(u3) # revealed: int | str | bytes | bytearray
```
## Simplification using subtyping
@@ -120,8 +145,8 @@ Simplifications still apply when `Unknown` is present.
```py
from knot_extensions import Unknown
def _(u1: str | Unknown | int | object):
reveal_type(u1) # revealed: Unknown | object
def _(u1: int | Unknown | bool) -> None:
reveal_type(u1) # revealed: int | Unknown
```
## Union of intersections

View File

@@ -12,14 +12,32 @@ use ruff_db::parsed::ParsedModule;
/// Holding on to any [`AstNodeRef`] prevents the [`ParsedModule`] from being released.
///
/// ## Equality
/// Two `AstNodeRef` are considered equal if their wrapped nodes are equal.
/// Two `AstNodeRef` are considered equal if their pointer addresses are equal.
///
/// ## Usage in salsa tracked structs
/// It's important that [`AstNodeRef`] fields in salsa tracked structs are tracked fields
/// (attributed with `#[tracked`]). It prevents that the tracked struct gets a new ID
/// everytime the AST changes, which in turn, invalidates the result of any query
/// that takes said tracked struct as a query argument or returns the tracked struct as part of its result.
///
/// For example, marking the [`AstNodeRef`] as tracked on `Expression`
/// has the effect that salsa will consider the expression as "unchanged" for as long as it:
///
/// * belongs to the same file
/// * belongs to the same scope
/// * has the same kind
/// * was created in the same order
///
/// This means that changes to expressions in other scopes don't invalidate the expression's id, giving
/// us some form of scope-stable identity for expressions. Only queries accessing the node field
/// run on every AST change. All other queries only run when the expression's identity changes.
#[derive(Clone)]
pub struct AstNodeRef<T> {
/// Owned reference to the node's [`ParsedModule`].
///
/// The node's reference is guaranteed to remain valid as long as it's enclosing
/// [`ParsedModule`] is alive.
_parsed: ParsedModule,
parsed: ParsedModule,
/// Pointer to the referenced node.
node: std::ptr::NonNull<T>,
@@ -37,7 +55,7 @@ impl<T> AstNodeRef<T> {
/// the invariant `node belongs to parsed` is upheld.
pub(super) unsafe fn new(parsed: ParsedModule, node: &T) -> Self {
Self {
_parsed: parsed,
parsed,
node: std::ptr::NonNull::from(node),
}
}
@@ -72,7 +90,14 @@ where
T: PartialEq,
{
fn eq(&self, other: &Self) -> bool {
self.node().eq(other.node())
if self.parsed == other.parsed {
// Comparing the pointer addresses is sufficient to determine equality
// if the parsed are the same.
self.node.eq(&other.node)
} else {
// Otherwise perform a deep comparison.
self.node().eq(other.node())
}
}
}
@@ -87,6 +112,20 @@ where
}
}
#[allow(unsafe_code)]
unsafe impl<T> salsa::Update for AstNodeRef<T> {
unsafe fn maybe_update(old_pointer: *mut Self, new_value: Self) -> bool {
let old_ref = &mut (*old_pointer);
if old_ref.parsed == new_value.parsed && old_ref.node.eq(&new_value.node) {
false
} else {
*old_ref = new_value;
true
}
}
}
#[allow(unsafe_code)]
unsafe impl<T> Send for AstNodeRef<T> where T: Send {}
#[allow(unsafe_code)]

View File

@@ -1,3 +1,5 @@
use std::sync::Arc;
use crate::lint::{LintRegistry, RuleSelection};
use ruff_db::files::File;
use ruff_db::{Db as SourceDb, Upcast};
@@ -7,7 +9,7 @@ use ruff_db::{Db as SourceDb, Upcast};
pub trait Db: SourceDb + Upcast<dyn SourceDb> {
fn is_file_open(&self, file: File) -> bool;
fn rule_selection(&self) -> &RuleSelection;
fn rule_selection(&self) -> Arc<RuleSelection>;
fn lint_registry(&self) -> &LintRegistry;
}
@@ -17,7 +19,6 @@ pub(crate) mod tests {
use std::sync::Arc;
use crate::program::{Program, SearchPathSettings};
use crate::python_version::PythonVersion;
use crate::{default_lint_registry, ProgramSettings, PythonPlatform};
use super::Db;
@@ -27,6 +28,7 @@ pub(crate) mod tests {
use ruff_db::system::{DbWithTestSystem, System, SystemPathBuf, TestSystem};
use ruff_db::vendored::VendoredFileSystem;
use ruff_db::{Db as SourceDb, Upcast};
use ruff_python_ast::python_version::PythonVersion;
#[salsa::db]
#[derive(Clone)]
@@ -111,8 +113,8 @@ pub(crate) mod tests {
!file.path(self).is_vendored_path()
}
fn rule_selection(&self) -> &RuleSelection {
&self.rule_selection
fn rule_selection(&self) -> Arc<RuleSelection> {
self.rule_selection.clone()
}
fn lint_registry(&self) -> &LintRegistry {

View File

@@ -9,7 +9,6 @@ pub use module_name::ModuleName;
pub use module_resolver::{resolve_module, system_module_search_paths, KnownModule, Module};
pub use program::{Program, ProgramSettings, SearchPathSettings, SitePackages};
pub use python_platform::PythonPlatform;
pub use python_version::PythonVersion;
pub use semantic_model::{HasType, SemanticModel};
pub mod ast_node_ref;
@@ -20,7 +19,6 @@ mod module_resolver;
mod node_key;
mod program;
mod python_platform;
mod python_version;
pub mod semantic_index;
mod semantic_model;
pub(crate) mod site_packages;

View File

@@ -631,10 +631,10 @@ impl PartialEq<SearchPath> for VendoredPathBuf {
#[cfg(test)]
mod tests {
use ruff_db::Db;
use ruff_python_ast::python_version::PythonVersion;
use crate::db::tests::TestDb;
use crate::module_resolver::testing::{FileSpec, MockedTypeshed, TestCase, TestCaseBuilder};
use crate::python_version::PythonVersion;
use super::*;

View File

@@ -6,12 +6,13 @@ use rustc_hash::{FxBuildHasher, FxHashSet};
use ruff_db::files::{File, FilePath, FileRootKind};
use ruff_db::system::{DirectoryEntry, System, SystemPath, SystemPathBuf};
use ruff_db::vendored::{VendoredFileSystem, VendoredPath};
use ruff_python_ast::python_version::PythonVersion;
use crate::db::Db;
use crate::module_name::ModuleName;
use crate::module_resolver::typeshed::{vendored_typeshed_versions, TypeshedVersions};
use crate::site_packages::VirtualEnvironment;
use crate::{Program, PythonVersion, SearchPathSettings, SitePackages};
use crate::{Program, SearchPathSettings, SitePackages};
use super::module::{Module, ModuleKind};
use super::path::{ModulePath, SearchPath, SearchPathValidationError};
@@ -133,7 +134,7 @@ pub(crate) fn search_paths(db: &dyn Db) -> SearchPathIterator {
}
#[derive(Debug, PartialEq, Eq)]
pub(crate) struct SearchPaths {
pub struct SearchPaths {
/// Search paths that have been statically determined purely from reading Ruff's configuration settings.
/// These shouldn't ever change unless the config settings themselves change.
static_paths: Vec<SearchPath>,
@@ -724,12 +725,12 @@ mod tests {
assert_const_function_query_was_not_run, assert_function_query_was_not_run,
};
use ruff_db::Db;
use ruff_python_ast::python_version::PythonVersion;
use crate::db::tests::TestDb;
use crate::module_name::ModuleName;
use crate::module_resolver::module::ModuleKind;
use crate::module_resolver::testing::{FileSpec, MockedTypeshed, TestCase, TestCaseBuilder};
use crate::PythonVersion;
use crate::{ProgramSettings, PythonPlatform};
use super::*;

View File

@@ -1,9 +1,9 @@
use ruff_db::system::{DbWithTestSystem, SystemPath, SystemPathBuf};
use ruff_db::vendored::VendoredPathBuf;
use ruff_python_ast::python_version::PythonVersion;
use crate::db::tests::TestDb;
use crate::program::{Program, SearchPathSettings};
use crate::python_version::PythonVersion;
use crate::{ProgramSettings, PythonPlatform, SitePackages};
/// A test case for the module resolver.

View File

@@ -4,11 +4,12 @@ use std::num::{NonZeroU16, NonZeroUsize};
use std::ops::{RangeFrom, RangeInclusive};
use std::str::FromStr;
use ruff_python_ast::python_version::PythonVersion;
use rustc_hash::FxHashMap;
use crate::db::Db;
use crate::module_name::ModuleName;
use crate::{Program, PythonVersion};
use crate::Program;
pub(in crate::module_resolver) fn vendored_typeshed_versions(db: &dyn Db) -> TypeshedVersions {
TypeshedVersions::from_str(
@@ -278,12 +279,12 @@ impl FromStr for PyVersionRange {
let mut parts = s.split('-').map(str::trim);
match (parts.next(), parts.next(), parts.next()) {
(Some(lower), Some(""), None) => {
let lower = PythonVersion::from_versions_file_string(lower)?;
let lower = python_version_from_versions_file_string(lower)?;
Ok(Self::AvailableFrom(lower..))
}
(Some(lower), Some(upper), None) => {
let lower = PythonVersion::from_versions_file_string(lower)?;
let upper = PythonVersion::from_versions_file_string(upper)?;
let lower = python_version_from_versions_file_string(lower)?;
let upper = python_version_from_versions_file_string(upper)?;
Ok(Self::AvailableWithin(lower..=upper))
}
_ => Err(TypeshedVersionsParseErrorKind::UnexpectedNumberOfHyphens),
@@ -302,21 +303,21 @@ impl fmt::Display for PyVersionRange {
}
}
impl PythonVersion {
fn from_versions_file_string(s: &str) -> Result<Self, TypeshedVersionsParseErrorKind> {
let mut parts = s.split('.').map(str::trim);
let (Some(major), Some(minor), None) = (parts.next(), parts.next(), parts.next()) else {
return Err(TypeshedVersionsParseErrorKind::UnexpectedNumberOfPeriods(
s.to_string(),
));
};
PythonVersion::try_from((major, minor)).map_err(|int_parse_error| {
TypeshedVersionsParseErrorKind::IntegerParsingFailure {
version: s.to_string(),
err: int_parse_error,
}
})
}
fn python_version_from_versions_file_string(
s: &str,
) -> Result<PythonVersion, TypeshedVersionsParseErrorKind> {
let mut parts = s.split('.').map(str::trim);
let (Some(major), Some(minor), None) = (parts.next(), parts.next(), parts.next()) else {
return Err(TypeshedVersionsParseErrorKind::UnexpectedNumberOfPeriods(
s.to_string(),
));
};
PythonVersion::try_from((major, minor)).map_err(|int_parse_error| {
TypeshedVersionsParseErrorKind::IntegerParsingFailure {
version: s.to_string(),
err: int_parse_error,
}
})
}
#[cfg(test)]

View File

@@ -1,10 +1,10 @@
use crate::module_resolver::SearchPaths;
use crate::python_platform::PythonPlatform;
use crate::python_version::PythonVersion;
use crate::Db;
use anyhow::Context;
use ruff_db::system::{SystemPath, SystemPathBuf};
use ruff_python_ast::python_version::PythonVersion;
use salsa::Durability;
use salsa::Setter;

View File

@@ -1,13 +1,14 @@
use std::iter::FusedIterator;
use std::sync::Arc;
use rustc_hash::{FxBuildHasher, FxHashMap, FxHashSet};
use salsa::plumbing::AsId;
use ruff_db::files::File;
use ruff_db::parsed::parsed_module;
use ruff_index::{IndexSlice, IndexVec};
use rustc_hash::{FxBuildHasher, FxHashMap, FxHashSet};
use salsa::plumbing::AsId;
use salsa::Update;
use crate::module_name::ModuleName;
use crate::semantic_index::ast_ids::node_key::ExpressionNodeKey;
use crate::semantic_index::ast_ids::AstIds;
@@ -123,7 +124,7 @@ pub(crate) fn global_scope(db: &dyn Db, file: File) -> ScopeId<'_> {
}
/// The symbol tables and use-def maps for all scopes in a file.
#[derive(Debug)]
#[derive(Debug, Update)]
pub(crate) struct SemanticIndex<'db> {
/// List of all symbol tables in this file, indexed by scope.
symbol_tables: IndexVec<FileScopeId, Arc<SymbolTable>>,

View File

@@ -24,7 +24,7 @@ use crate::Db;
///
/// x = foo()
/// ```
#[derive(Debug)]
#[derive(Debug, salsa::Update)]
pub(crate) struct AstIds {
/// Maps expressions to their expression id.
expressions_map: FxHashMap<ExpressionNodeKey, ScopedExpressionId>,
@@ -74,6 +74,7 @@ impl HasScopedUseId for ast::ExprRef<'_> {
/// Uniquely identifies an [`ast::Expr`] in a [`crate::semantic_index::symbol::FileScopeId`].
#[newtype_index]
#[derive(salsa::Update)]
pub struct ScopedExpressionId;
pub trait HasScopedExpressionId {
@@ -181,7 +182,7 @@ pub(crate) mod node_key {
use crate::node_key::NodeKey;
#[derive(Copy, Clone, Eq, PartialEq, Hash, Debug)]
#[derive(Copy, Clone, Eq, PartialEq, Hash, Debug, salsa::Update)]
pub(crate) struct ExpressionNodeKey(NodeKey);
impl From<ast::ExprRef<'_>> for ExpressionNodeKey {

View File

@@ -9,7 +9,7 @@ use rustc_hash::FxHashMap;
/// Describes an (annotated) attribute assignment that we discovered in a method
/// body, typically of the form `self.x: int`, `self.x: int = …` or `self.x = …`.
#[derive(Debug, Clone, PartialEq, Eq)]
#[derive(Debug, Clone, PartialEq, Eq, salsa::Update)]
pub(crate) enum AttributeAssignment<'db> {
/// An attribute assignment with an explicit type annotation, either
/// `self.x: <annotation>` or `self.x: <annotation> = …`.

View File

@@ -33,8 +33,8 @@ use crate::Db;
use super::constraint::{Constraint, ConstraintNode, PatternConstraint};
use super::definition::{
DefinitionCategory, ExceptHandlerDefinitionNodeRef, MatchPatternDefinitionNodeRef,
WithItemDefinitionNodeRef,
DefinitionCategory, ExceptHandlerDefinitionNodeRef, ImportDefinitionNodeRef,
MatchPatternDefinitionNodeRef, WithItemDefinitionNodeRef,
};
mod except_handlers;
@@ -886,22 +886,28 @@ where
self.imported_modules.extend(module_name.ancestors());
}
let symbol_name = if let Some(asname) = &alias.asname {
asname.id.clone()
let (symbol_name, is_reexported) = if let Some(asname) = &alias.asname {
(asname.id.clone(), asname.id == alias.name.id)
} else {
Name::new(alias.name.id.split('.').next().unwrap())
(Name::new(alias.name.id.split('.').next().unwrap()), false)
};
let symbol = self.add_symbol(symbol_name);
self.add_definition(symbol, alias);
self.add_definition(
symbol,
ImportDefinitionNodeRef {
alias,
is_reexported,
},
);
}
}
ast::Stmt::ImportFrom(node) => {
for (alias_index, alias) in node.names.iter().enumerate() {
let symbol_name = if let Some(asname) = &alias.asname {
&asname.id
let (symbol_name, is_reexported) = if let Some(asname) = &alias.asname {
(&asname.id, asname.id == alias.name.id)
} else {
&alias.name.id
(&alias.name.id, false)
};
// Look for imports `from __future__ import annotations`, ignore `as ...`
@@ -914,7 +920,14 @@ where
let symbol = self.add_symbol(symbol_name.clone());
self.add_definition(symbol, ImportFromDefinitionNodeRef { node, alias_index });
self.add_definition(
symbol,
ImportFromDefinitionNodeRef {
node,
alias_index,
is_reexported,
},
);
}
}
ast::Stmt::Assign(node) => {

View File

@@ -5,20 +5,20 @@ use crate::db::Db;
use crate::semantic_index::expression::Expression;
use crate::semantic_index::symbol::{FileScopeId, ScopeId};
#[derive(Clone, Copy, Debug, Hash, PartialEq, Eq)]
#[derive(Clone, Copy, Debug, Hash, PartialEq, Eq, salsa::Update)]
pub(crate) struct Constraint<'db> {
pub(crate) node: ConstraintNode<'db>,
pub(crate) is_positive: bool,
}
#[derive(Clone, Copy, Debug, Hash, PartialEq, Eq)]
#[derive(Clone, Copy, Debug, Hash, PartialEq, Eq, salsa::Update)]
pub(crate) enum ConstraintNode<'db> {
Expression(Expression<'db>),
Pattern(PatternConstraint<'db>),
}
/// Pattern kinds for which we support type narrowing and/or static visibility analysis.
#[derive(Debug, Clone, Hash, PartialEq)]
#[derive(Debug, Clone, Hash, PartialEq, salsa::Update)]
pub(crate) enum PatternConstraintKind<'db> {
Singleton(Singleton, Option<Expression<'db>>),
Value(Expression<'db>, Option<Expression<'db>>),
@@ -28,21 +28,15 @@ pub(crate) enum PatternConstraintKind<'db> {
#[salsa::tracked]
pub(crate) struct PatternConstraint<'db> {
#[id]
pub(crate) file: File,
#[id]
pub(crate) file_scope: FileScopeId,
#[no_eq]
#[return_ref]
pub(crate) subject: Expression<'db>,
#[no_eq]
#[return_ref]
pub(crate) kind: PatternConstraintKind<'db>,
#[no_eq]
count: countme::Count<PatternConstraint<'static>>,
}

View File

@@ -25,22 +25,19 @@ use crate::Db;
#[salsa::tracked]
pub struct Definition<'db> {
/// The file in which the definition occurs.
#[id]
pub(crate) file: File,
/// The scope in which the definition occurs.
#[id]
pub(crate) file_scope: FileScopeId,
/// The symbol defined.
#[id]
pub(crate) symbol: ScopedSymbolId,
#[no_eq]
#[return_ref]
#[tracked]
pub(crate) kind: DefinitionKind<'db>,
#[no_eq]
count: countme::Count<Definition<'static>>,
}
@@ -60,11 +57,15 @@ impl<'db> Definition<'db> {
pub(crate) fn is_binding(self, db: &'db dyn Db) -> bool {
self.kind(db).category().is_binding()
}
pub(crate) fn is_reexported(self, db: &'db dyn Db) -> bool {
self.kind(db).is_reexported()
}
}
#[derive(Copy, Clone, Debug)]
pub(crate) enum DefinitionNodeRef<'a> {
Import(&'a ast::Alias),
Import(ImportDefinitionNodeRef<'a>),
ImportFrom(ImportFromDefinitionNodeRef<'a>),
For(ForStmtDefinitionNodeRef<'a>),
Function(&'a ast::StmtFunctionDef),
@@ -122,12 +123,6 @@ impl<'a> From<&'a ast::StmtAugAssign> for DefinitionNodeRef<'a> {
}
}
impl<'a> From<&'a ast::Alias> for DefinitionNodeRef<'a> {
fn from(node_ref: &'a ast::Alias) -> Self {
Self::Import(node_ref)
}
}
impl<'a> From<&'a ast::TypeParamTypeVar> for DefinitionNodeRef<'a> {
fn from(value: &'a ast::TypeParamTypeVar) -> Self {
Self::TypeVar(value)
@@ -146,6 +141,12 @@ impl<'a> From<&'a ast::TypeParamTypeVarTuple> for DefinitionNodeRef<'a> {
}
}
impl<'a> From<ImportDefinitionNodeRef<'a>> for DefinitionNodeRef<'a> {
fn from(node_ref: ImportDefinitionNodeRef<'a>) -> Self {
Self::Import(node_ref)
}
}
impl<'a> From<ImportFromDefinitionNodeRef<'a>> for DefinitionNodeRef<'a> {
fn from(node_ref: ImportFromDefinitionNodeRef<'a>) -> Self {
Self::ImportFrom(node_ref)
@@ -188,10 +189,17 @@ impl<'a> From<MatchPatternDefinitionNodeRef<'a>> for DefinitionNodeRef<'a> {
}
}
#[derive(Copy, Clone, Debug)]
pub(crate) struct ImportDefinitionNodeRef<'a> {
pub(crate) alias: &'a ast::Alias,
pub(crate) is_reexported: bool,
}
#[derive(Copy, Clone, Debug)]
pub(crate) struct ImportFromDefinitionNodeRef<'a> {
pub(crate) node: &'a ast::StmtImportFrom,
pub(crate) alias_index: usize,
pub(crate) is_reexported: bool,
}
#[derive(Copy, Clone, Debug)]
@@ -247,15 +255,22 @@ impl<'db> DefinitionNodeRef<'db> {
#[allow(unsafe_code)]
pub(super) unsafe fn into_owned(self, parsed: ParsedModule) -> DefinitionKind<'db> {
match self {
DefinitionNodeRef::Import(alias) => {
DefinitionKind::Import(AstNodeRef::new(parsed, alias))
}
DefinitionNodeRef::ImportFrom(ImportFromDefinitionNodeRef { node, alias_index }) => {
DefinitionKind::ImportFrom(ImportFromDefinitionKind {
node: AstNodeRef::new(parsed, node),
alias_index,
})
}
DefinitionNodeRef::Import(ImportDefinitionNodeRef {
alias,
is_reexported,
}) => DefinitionKind::Import(ImportDefinitionKind {
alias: AstNodeRef::new(parsed, alias),
is_reexported,
}),
DefinitionNodeRef::ImportFrom(ImportFromDefinitionNodeRef {
node,
alias_index,
is_reexported,
}) => DefinitionKind::ImportFrom(ImportFromDefinitionKind {
node: AstNodeRef::new(parsed, node),
alias_index,
is_reexported,
}),
DefinitionNodeRef::Function(function) => {
DefinitionKind::Function(AstNodeRef::new(parsed, function))
}
@@ -357,10 +372,15 @@ impl<'db> DefinitionNodeRef<'db> {
pub(super) fn key(self) -> DefinitionNodeKey {
match self {
Self::Import(node) => node.into(),
Self::ImportFrom(ImportFromDefinitionNodeRef { node, alias_index }) => {
(&node.names[alias_index]).into()
}
Self::Import(ImportDefinitionNodeRef {
alias,
is_reexported: _,
}) => alias.into(),
Self::ImportFrom(ImportFromDefinitionNodeRef {
node,
alias_index,
is_reexported: _,
}) => (&node.names[alias_index]).into(),
Self::Function(node) => node.into(),
Self::Class(node) => node.into(),
Self::TypeAlias(node) => node.into(),
@@ -435,9 +455,16 @@ impl DefinitionCategory {
}
}
/// The kind of a definition.
///
/// ## Usage in salsa tracked structs
///
/// [`DefinitionKind`] fields in salsa tracked structs should be tracked (attributed with `#[tracked]`)
/// because the kind is a thin wrapper around [`AstNodeRef`]. See the [`AstNodeRef`] documentation
/// for an in-depth explanation of why this is necessary.
#[derive(Clone, Debug)]
pub enum DefinitionKind<'db> {
Import(AstNodeRef<ast::Alias>),
Import(ImportDefinitionKind),
ImportFrom(ImportFromDefinitionKind),
Function(AstNodeRef<ast::StmtFunctionDef>),
Class(AstNodeRef<ast::StmtClassDef>),
@@ -460,6 +487,14 @@ pub enum DefinitionKind<'db> {
}
impl DefinitionKind<'_> {
pub(crate) fn is_reexported(&self) -> bool {
match self {
DefinitionKind::Import(import) => import.is_reexported(),
DefinitionKind::ImportFrom(import) => import.is_reexported(),
_ => true,
}
}
/// Returns the [`TextRange`] of the definition target.
///
/// A definition target would mainly be the node representing the symbol being defined i.e.,
@@ -468,7 +503,7 @@ impl DefinitionKind<'_> {
/// This is mainly used for logging and debugging purposes.
pub(crate) fn target_range(&self) -> TextRange {
match self {
DefinitionKind::Import(alias) => alias.range(),
DefinitionKind::Import(import) => import.alias().range(),
DefinitionKind::ImportFrom(import) => import.alias().range(),
DefinitionKind::Function(function) => function.name.range(),
DefinitionKind::Class(class) => class.name.range(),
@@ -540,7 +575,7 @@ impl DefinitionKind<'_> {
}
}
#[derive(Copy, Clone, Debug, PartialEq)]
#[derive(Copy, Clone, Debug, PartialEq, Hash)]
pub(crate) enum TargetKind<'db> {
Sequence(Unpack<'db>),
Name,
@@ -599,10 +634,27 @@ impl ComprehensionDefinitionKind {
}
}
#[derive(Clone, Debug)]
pub struct ImportDefinitionKind {
alias: AstNodeRef<ast::Alias>,
is_reexported: bool,
}
impl ImportDefinitionKind {
pub(crate) fn alias(&self) -> &ast::Alias {
self.alias.node()
}
pub(crate) fn is_reexported(&self) -> bool {
self.is_reexported
}
}
#[derive(Clone, Debug)]
pub struct ImportFromDefinitionKind {
node: AstNodeRef<ast::StmtImportFrom>,
alias_index: usize,
is_reexported: bool,
}
impl ImportFromDefinitionKind {
@@ -613,6 +665,10 @@ impl ImportFromDefinitionKind {
pub(crate) fn alias(&self) -> &ast::Alias {
&self.node.node().names[self.alias_index]
}
pub(crate) fn is_reexported(&self) -> bool {
self.is_reexported
}
}
#[derive(Clone, Debug)]
@@ -713,7 +769,7 @@ impl ExceptHandlerDefinitionKind {
}
}
#[derive(Copy, Clone, Eq, PartialEq, Hash, Debug)]
#[derive(Copy, Clone, Eq, PartialEq, Hash, Debug, salsa::Update)]
pub(crate) struct DefinitionNodeKey(NodeKey);
impl From<&ast::Alias> for DefinitionNodeKey {

View File

@@ -33,23 +33,20 @@ pub(crate) enum ExpressionKind {
#[salsa::tracked]
pub(crate) struct Expression<'db> {
/// The file in which the expression occurs.
#[id]
pub(crate) file: File,
/// The scope in which the expression occurs.
#[id]
pub(crate) file_scope: FileScopeId,
/// The expression node.
#[no_eq]
#[tracked]
#[return_ref]
pub(crate) node_ref: AstNodeRef<ast::Expr>,
/// Should this expression be inferred as a normal expression or a type expression?
#[id]
pub(crate) kind: ExpressionKind,
#[no_eq]
count: countme::Count<Expression<'static>>,
}

View File

@@ -96,18 +96,16 @@ impl From<FileSymbolId> for ScopedSymbolId {
/// Symbol ID that uniquely identifies a symbol inside a [`Scope`].
#[newtype_index]
#[derive(salsa::Update)]
pub struct ScopedSymbolId;
/// A cross-module identifier of a scope that can be used as a salsa query parameter.
#[salsa::tracked]
pub struct ScopeId<'db> {
#[id]
pub file: File,
#[id]
pub file_scope_id: FileScopeId,
#[no_eq]
count: countme::Count<ScopeId<'static>>,
}
@@ -159,6 +157,7 @@ impl<'db> ScopeId<'db> {
/// ID that uniquely identifies a scope inside of a module.
#[newtype_index]
#[derive(salsa::Update)]
pub struct FileScopeId;
impl FileScopeId {
@@ -177,7 +176,7 @@ impl FileScopeId {
}
}
#[derive(Debug)]
#[derive(Debug, salsa::Update)]
pub struct Scope {
pub(super) parent: Option<FileScopeId>,
pub(super) node: NodeWithScopeKind,
@@ -216,7 +215,7 @@ impl ScopeKind {
}
/// Symbol table for a specific [`Scope`].
#[derive(Debug, Default)]
#[derive(Debug, Default, salsa::Update)]
pub struct SymbolTable {
/// The symbols in this scope.
symbols: IndexVec<ScopedSymbolId, Symbol>,
@@ -424,7 +423,7 @@ impl NodeWithScopeRef<'_> {
}
/// Node that introduces a new scope.
#[derive(Clone, Debug)]
#[derive(Clone, Debug, salsa::Update)]
pub enum NodeWithScopeKind {
Module,
Class(AstNodeRef<ast::StmtClassDef>),

View File

@@ -278,7 +278,7 @@ mod symbol_state;
type AllConstraints<'db> = IndexVec<ScopedConstraintId, Constraint<'db>>;
/// Applicable definitions and constraints for every use of a name.
#[derive(Debug, PartialEq, Eq)]
#[derive(Debug, PartialEq, Eq, salsa::Update)]
pub(crate) struct UseDefMap<'db> {
/// Array of [`Definition`] in this scope. Only the first entry should be `None`;
/// this represents the implicit "unbound"/"undeclared" definition of every symbol.
@@ -384,7 +384,7 @@ impl<'db> UseDefMap<'db> {
}
/// Either live bindings or live declarations for a symbol.
#[derive(Debug, PartialEq, Eq)]
#[derive(Debug, PartialEq, Eq, salsa::Update)]
enum SymbolDefinitions {
Bindings(SymbolBindings),
Declarations(SymbolDeclarations),

View File

@@ -14,8 +14,7 @@ use std::num::NonZeroUsize;
use std::ops::Deref;
use ruff_db::system::{System, SystemPath, SystemPathBuf};
use crate::PythonVersion;
use ruff_python_ast::python_version::PythonVersion;
type SitePackagesDiscoveryResult<T> = Result<T, SitePackagesDiscoveryError>;

View File

@@ -2,7 +2,7 @@ use crate::module_resolver::{resolve_module, KnownModule};
use crate::semantic_index::global_scope;
use crate::semantic_index::symbol::ScopeId;
use crate::symbol::Symbol;
use crate::types::global_symbol;
use crate::types::imported_symbol;
use crate::Db;
/// Lookup the type of `symbol` in a given known module
@@ -14,18 +14,10 @@ pub(crate) fn known_module_symbol<'db>(
symbol: &str,
) -> Symbol<'db> {
resolve_module(db, &known_module.name())
.map(|module| global_symbol(db, module.file(), symbol))
.map(|module| imported_symbol(db, &module, symbol))
.unwrap_or(Symbol::Unbound)
}
/// Lookup the type of `symbol` in the builtins namespace.
///
/// Returns `Symbol::Unbound` if the `builtins` module isn't available for some reason.
#[inline]
pub(crate) fn builtins_symbol<'db>(db: &'db dyn Db, symbol: &str) -> Symbol<'db> {
known_module_symbol(db, KnownModule::Builtins, symbol)
}
/// Lookup the type of `symbol` in the `typing` module namespace.
///
/// Returns `Symbol::Unbound` if the `typing` module isn't available for some reason.

View File

@@ -1,5 +1,5 @@
use crate::{
types::{Type, UnionType},
types::{todo_type, Type, UnionType},
Db,
};
@@ -26,13 +26,25 @@ pub(crate) enum Boundness {
/// possibly_unbound: Symbol::Type(Type::IntLiteral(2), Boundness::PossiblyUnbound),
/// non_existent: Symbol::Unbound,
/// ```
#[derive(Debug, Clone, PartialEq, Eq)]
#[derive(Debug, Clone, PartialEq, Eq, salsa::Update)]
pub(crate) enum Symbol<'db> {
Type(Type<'db>, Boundness),
Unbound,
}
impl<'db> Symbol<'db> {
/// Constructor that creates a `Symbol` with boundness [`Boundness::Bound`].
pub(crate) fn bound(ty: impl Into<Type<'db>>) -> Self {
Symbol::Type(ty.into(), Boundness::Bound)
}
/// Constructor that creates a [`Symbol`] with a [`crate::types::TodoType`] type
/// and boundness [`Boundness::Bound`].
#[allow(unused_variables)] // Only unused in release builds
pub(crate) fn todo(message: &'static str) -> Self {
Symbol::Type(todo_type!(message), Boundness::Bound)
}
pub(crate) fn is_unbound(&self) -> bool {
matches!(self, Symbol::Unbound)
}
@@ -55,6 +67,30 @@ impl<'db> Symbol<'db> {
.expect("Expected a (possibly unbound) type, not an unbound symbol")
}
/// Transform the symbol into a [`LookupResult`],
/// a [`Result`] type in which the `Ok` variant represents a definitely bound symbol
/// and the `Err` variant represents a symbol that is either definitely or possibly unbound.
pub(crate) fn into_lookup_result(self) -> LookupResult<'db> {
match self {
Symbol::Type(ty, Boundness::Bound) => Ok(ty),
Symbol::Type(ty, Boundness::PossiblyUnbound) => Err(LookupError::PossiblyUnbound(ty)),
Symbol::Unbound => Err(LookupError::Unbound),
}
}
/// Safely unwrap the symbol into a [`Type`].
///
/// If the symbol is definitely unbound or possibly unbound, it will be transformed into a
/// [`LookupError`] and `diagnostic_fn` will be applied to the error value before returning
/// the result of `diagnostic_fn` (which will be a [`Type`]). This allows the caller to ensure
/// that a diagnostic is emitted if the symbol is possibly or definitely unbound.
pub(crate) fn unwrap_with_diagnostic(
self,
diagnostic_fn: impl FnOnce(LookupError<'db>) -> Type<'db>,
) -> Type<'db> {
self.into_lookup_result().unwrap_or_else(diagnostic_fn)
}
/// Fallback (partially or fully) to another symbol if `self` is partially or fully unbound.
///
/// 1. If `self` is definitely bound, return `self` without evaluating `fallback_fn()`.
@@ -71,17 +107,9 @@ impl<'db> Symbol<'db> {
db: &'db dyn Db,
fallback_fn: impl FnOnce() -> Self,
) -> Self {
match self {
Symbol::Type(_, Boundness::Bound) => self,
Symbol::Unbound => fallback_fn(),
Symbol::Type(self_ty, Boundness::PossiblyUnbound) => match fallback_fn() {
Symbol::Unbound => self,
Symbol::Type(fallback_ty, fallback_boundness) => Symbol::Type(
UnionType::from_elements(db, [self_ty, fallback_ty]),
fallback_boundness,
),
},
}
self.into_lookup_result()
.or_else(|lookup_error| lookup_error.or_fall_back_to(db, fallback_fn()))
.into()
}
#[must_use]
@@ -93,6 +121,51 @@ impl<'db> Symbol<'db> {
}
}
impl<'db> From<LookupResult<'db>> for Symbol<'db> {
fn from(value: LookupResult<'db>) -> Self {
match value {
Ok(ty) => Symbol::Type(ty, Boundness::Bound),
Err(LookupError::Unbound) => Symbol::Unbound,
Err(LookupError::PossiblyUnbound(ty)) => Symbol::Type(ty, Boundness::PossiblyUnbound),
}
}
}
/// Possible ways in which a symbol lookup can (possibly or definitely) fail.
#[derive(Copy, Clone, PartialEq, Eq, Debug)]
pub(crate) enum LookupError<'db> {
Unbound,
PossiblyUnbound(Type<'db>),
}
impl<'db> LookupError<'db> {
/// Fallback (wholly or partially) to `fallback` to create a new [`LookupResult`].
pub(crate) fn or_fall_back_to(
self,
db: &'db dyn Db,
fallback: Symbol<'db>,
) -> LookupResult<'db> {
let fallback = fallback.into_lookup_result();
match (&self, &fallback) {
(LookupError::Unbound, _) => fallback,
(LookupError::PossiblyUnbound { .. }, Err(LookupError::Unbound)) => Err(self),
(LookupError::PossiblyUnbound(ty), Ok(ty2)) => {
Ok(UnionType::from_elements(db, [ty, ty2]))
}
(LookupError::PossiblyUnbound(ty), Err(LookupError::PossiblyUnbound(ty2))) => Err(
LookupError::PossiblyUnbound(UnionType::from_elements(db, [ty, ty2])),
),
}
}
}
/// A [`Result`] type in which the `Ok` variant represents a definitely bound symbol
/// and the `Err` variant represents a symbol that is either definitely or possibly unbound.
///
/// Note that this type is exactly isomorphic to [`Symbol`].
/// In the future, we could possibly consider removing `Symbol` and using this type everywhere instead.
pub(crate) type LookupResult<'db> = Result<Type<'db>, LookupError<'db>>;
#[cfg(test)]
mod tests {
use super::*;

View File

@@ -8,6 +8,7 @@ use itertools::Itertools;
use ruff_db::diagnostic::Severity;
use ruff_db::files::File;
use ruff_python_ast as ast;
use ruff_python_ast::python_version::PythonVersion;
use type_ordering::union_elements_ordering;
pub(crate) use self::builder::{IntersectionBuilder, UnionBuilder};
@@ -32,9 +33,9 @@ use crate::semantic_index::{
use_def_map, BindingWithConstraints, BindingWithConstraintsIterator, DeclarationWithConstraint,
DeclarationsIterator,
};
use crate::stdlib::{builtins_symbol, known_module_symbol, typing_extensions_symbol};
use crate::stdlib::{known_module_symbol, typing_extensions_symbol};
use crate::suppression::check_suppressions;
use crate::symbol::{Boundness, Symbol};
use crate::symbol::{Boundness, LookupError, LookupResult, Symbol};
use crate::types::call::{
bind_call, CallArguments, CallBinding, CallDunderResult, CallOutcome, StaticAssertionErrorKind,
};
@@ -43,7 +44,7 @@ use crate::types::diagnostic::INVALID_TYPE_FORM;
use crate::types::infer::infer_unpack_types;
use crate::types::mro::{Mro, MroError, MroIterator};
use crate::types::narrow::narrowing_constraint;
use crate::{Db, FxOrderSet, Module, Program, PythonVersion};
use crate::{Db, FxOrderSet, Module, Program};
mod builder;
mod call;
@@ -56,7 +57,6 @@ mod mro;
mod narrow;
mod signatures;
mod slots;
mod statistics;
mod string_annotation;
mod subclass_of;
mod type_ordering;
@@ -107,14 +107,30 @@ fn widen_type_for_undeclared_public_symbol<'db>(
}
}
/// Infer the public type of a symbol (its type as seen from outside its scope).
fn symbol<'db>(db: &'db dyn Db, scope: ScopeId<'db>, name: &str) -> Symbol<'db> {
#[derive(Copy, Clone, Debug, PartialEq, Eq, Hash)]
enum RequiresExplicitReExport {
Yes,
No,
}
impl RequiresExplicitReExport {
const fn is_yes(self) -> bool {
matches!(self, RequiresExplicitReExport::Yes)
}
}
fn symbol_impl<'db>(
db: &'db dyn Db,
scope: ScopeId<'db>,
name: &str,
requires_explicit_reexport: RequiresExplicitReExport,
) -> Symbol<'db> {
#[salsa::tracked]
fn symbol_by_id<'db>(
db: &'db dyn Db,
scope: ScopeId<'db>,
is_dunder_slots: bool,
symbol_id: ScopedSymbolId,
requires_explicit_reexport: RequiresExplicitReExport,
) -> Symbol<'db> {
let use_def = use_def_map(db, scope);
@@ -122,7 +138,7 @@ fn symbol<'db>(db: &'db dyn Db, scope: ScopeId<'db>, name: &str) -> Symbol<'db>
// on inference from bindings.
let declarations = use_def.public_declarations(symbol_id);
let declared = symbol_from_declarations(db, declarations);
let declared = symbol_from_declarations(db, declarations, requires_explicit_reexport);
let is_final = declared.as_ref().is_ok_and(SymbolAndQualifiers::is_final);
let declared = declared.map(|SymbolAndQualifiers(symbol, _)| symbol);
@@ -132,7 +148,7 @@ fn symbol<'db>(db: &'db dyn Db, scope: ScopeId<'db>, name: &str) -> Symbol<'db>
// Symbol is possibly declared
Ok(Symbol::Type(declared_ty, Boundness::PossiblyUnbound)) => {
let bindings = use_def.public_bindings(symbol_id);
let inferred = symbol_from_bindings(db, bindings);
let inferred = symbol_from_bindings(db, bindings, requires_explicit_reexport);
match inferred {
// Symbol is possibly undeclared and definitely unbound
@@ -144,7 +160,7 @@ fn symbol<'db>(db: &'db dyn Db, scope: ScopeId<'db>, name: &str) -> Symbol<'db>
}
// Symbol is possibly undeclared and (possibly) bound
Symbol::Type(inferred_ty, boundness) => Symbol::Type(
UnionType::from_elements(db, [inferred_ty, declared_ty].iter().copied()),
UnionType::from_elements(db, [inferred_ty, declared_ty]),
boundness,
),
}
@@ -152,15 +168,23 @@ fn symbol<'db>(db: &'db dyn Db, scope: ScopeId<'db>, name: &str) -> Symbol<'db>
// Symbol is undeclared, return the union of `Unknown` with the inferred type
Ok(Symbol::Unbound) => {
let bindings = use_def.public_bindings(symbol_id);
let inferred = symbol_from_bindings(db, bindings);
let inferred = symbol_from_bindings(db, bindings, requires_explicit_reexport);
widen_type_for_undeclared_public_symbol(db, inferred, is_dunder_slots || is_final)
// `__slots__` is a symbol with special behavior in Python's runtime. It can be
// modified externally, but those changes do not take effect. We therefore issue
// a diagnostic if we see it being modified externally. In type inference, we
// can assign a "narrow" type to it even if it is not *declared*. This means, we
// do not have to call [`widen_type_for_undeclared_public_symbol`].
let is_considered_non_modifiable =
is_final || symbol_table(db, scope).symbol(symbol_id).name() == "__slots__";
widen_type_for_undeclared_public_symbol(db, inferred, is_considered_non_modifiable)
}
// Symbol has conflicting declared types
Err((declared_ty, _)) => {
// Intentionally ignore conflicting declared types; that's not our problem,
// it's the problem of the module we are importing from.
declared_ty.inner_type().into()
Symbol::bound(declared_ty.inner_type())
}
}
@@ -188,7 +212,7 @@ fn symbol<'db>(db: &'db dyn Db, scope: ScopeId<'db>, name: &str) -> Symbol<'db>
&& file_to_module(db, scope.file(db))
.is_some_and(|module| module.is_known(KnownModule::Typing))
{
return Symbol::Type(Type::BooleanLiteral(true), Boundness::Bound);
return Symbol::bound(Type::BooleanLiteral(true));
}
if name == "platform"
&& file_to_module(db, scope.file(db))
@@ -196,10 +220,7 @@ fn symbol<'db>(db: &'db dyn Db, scope: ScopeId<'db>, name: &str) -> Symbol<'db>
{
match Program::get(db).python_platform(db) {
crate::PythonPlatform::Identifier(platform) => {
return Symbol::Type(
Type::StringLiteral(StringLiteralType::new(db, platform.as_str())),
Boundness::Bound,
);
return Symbol::bound(Type::string_literal(db, platform.as_str()));
}
crate::PythonPlatform::All => {
// Fall through to the looked up type
@@ -207,16 +228,9 @@ fn symbol<'db>(db: &'db dyn Db, scope: ScopeId<'db>, name: &str) -> Symbol<'db>
}
}
let table = symbol_table(db, scope);
// `__slots__` is a symbol with special behavior in Python's runtime. It can be
// modified externally, but those changes do not take effect. We therefore issue
// a diagnostic if we see it being modified externally. In type inference, we
// can assign a "narrow" type to it even if it is not *declared*. This means, we
// do not have to call [`widen_type_for_undeclared_public_symbol`].
let is_dunder_slots = name == "__slots__";
table
symbol_table(db, scope)
.symbol_id_by_name(name)
.map(|symbol| symbol_by_id(db, scope, is_dunder_slots, symbol))
.map(|symbol| symbol_by_id(db, scope, symbol, requires_explicit_reexport))
.unwrap_or(Symbol::Unbound)
}
@@ -255,23 +269,99 @@ fn module_type_symbols<'db>(db: &'db dyn Db) -> smallvec::SmallVec<[ast::name::N
.collect()
}
/// Looks up a module-global symbol by name in a file.
/// Return the symbol for a member of `types.ModuleType`.
pub(crate) fn module_type_symbol<'db>(db: &'db dyn Db, name: &str) -> Symbol<'db> {
if module_type_symbols(db)
.iter()
.any(|module_type_member| &**module_type_member == name)
{
KnownClass::ModuleType.to_instance(db).member(db, name)
} else {
Symbol::Unbound
}
}
/// Infer the public type of a symbol (its type as seen from outside its scope) in the given
/// `scope`.
fn symbol<'db>(db: &'db dyn Db, scope: ScopeId<'db>, name: &str) -> Symbol<'db> {
symbol_impl(db, scope, name, RequiresExplicitReExport::No)
}
/// Infers the public type of a module-global symbol as seen from within the same file.
///
/// If it's not defined explicitly in the global scope, it will look it up in `types.ModuleType`
/// with a few very special exceptions.
///
/// Use [`imported_symbol`] to perform the lookup as seen from outside the file (e.g. via imports).
pub(crate) fn global_symbol<'db>(db: &'db dyn Db, file: File, name: &str) -> Symbol<'db> {
// Not defined explicitly in the global scope?
// All modules are instances of `types.ModuleType`;
// look it up there (with a few very special exceptions)
symbol(db, global_scope(db, file), name).or_fall_back_to(db, || {
if module_type_symbols(db)
.iter()
.any(|module_type_member| &**module_type_member == name)
{
KnownClass::ModuleType.to_instance(db).member(db, name)
} else {
symbol_impl(
db,
global_scope(db, file),
name,
RequiresExplicitReExport::No,
)
.or_fall_back_to(db, || module_type_symbol(db, name))
}
/// Infers the public type of an imported symbol.
pub(crate) fn imported_symbol<'db>(db: &'db dyn Db, module: &Module, name: &str) -> Symbol<'db> {
// If it's not found in the global scope, check if it's present as an instance on
// `types.ModuleType` or `builtins.object`.
//
// We do a more limited version of this in `global_symbol`, but there are two crucial
// differences here:
// - If a member is looked up as an attribute, `__init__` is also available on the module, but
// it isn't available as a global from inside the module
// - If a member is looked up as an attribute, members on `builtins.object` are also available
// (because `types.ModuleType` inherits from `object`); these attributes are also not
// available as globals from inside the module.
//
// The same way as in `global_symbol`, however, we need to be careful to ignore
// `__getattr__`. Typeshed has a fake `__getattr__` on `types.ModuleType` to help out with
// dynamic imports; we shouldn't use it for `ModuleLiteral` types where we know exactly which
// module we're dealing with.
external_symbol_impl(db, module.file(), name).or_fall_back_to(db, || {
if name == "__getattr__" {
Symbol::Unbound
} else {
KnownClass::ModuleType.to_instance(db).member(db, name)
}
})
}
/// Lookup the type of `symbol` in the builtins namespace.
///
/// Returns `Symbol::Unbound` if the `builtins` module isn't available for some reason.
///
/// Note that this function is only intended for use in the context of the builtins *namespace*
/// and should not be used when a symbol is being explicitly imported from the `builtins` module
/// (e.g. `from builtins import int`).
pub(crate) fn builtins_symbol<'db>(db: &'db dyn Db, symbol: &str) -> Symbol<'db> {
resolve_module(db, &KnownModule::Builtins.name())
.map(|module| {
external_symbol_impl(db, module.file(), symbol).or_fall_back_to(db, || {
// We're looking up in the builtins namespace and not the module, so we should
// do the normal lookup in `types.ModuleType` and not the special one as in
// `imported_symbol`.
module_type_symbol(db, symbol)
})
})
.unwrap_or(Symbol::Unbound)
}
fn external_symbol_impl<'db>(db: &'db dyn Db, file: File, name: &str) -> Symbol<'db> {
symbol_impl(
db,
global_scope(db, file),
name,
if file.is_stub(db.upcast()) {
RequiresExplicitReExport::Yes
} else {
RequiresExplicitReExport::No
},
)
}
/// Infer the type of a binding.
pub(crate) fn binding_type<'db>(db: &'db dyn Db, definition: Definition<'db>) -> Type<'db> {
let inference = infer_definition_types(db, definition);
@@ -321,19 +411,24 @@ fn definition_expression_type<'db>(
fn symbol_from_bindings<'db>(
db: &'db dyn Db,
bindings_with_constraints: BindingWithConstraintsIterator<'_, 'db>,
requires_explicit_reexport: RequiresExplicitReExport,
) -> Symbol<'db> {
let visibility_constraints = bindings_with_constraints.visibility_constraints;
let mut bindings_with_constraints = bindings_with_constraints.peekable();
let unbound_visibility = if let Some(BindingWithConstraints {
binding: None,
constraints: _,
visibility_constraint,
}) = bindings_with_constraints.peek()
{
visibility_constraints.evaluate(db, *visibility_constraint)
} else {
Truthiness::AlwaysFalse
let is_non_exported = |binding: Definition<'db>| {
requires_explicit_reexport.is_yes() && !binding.is_reexported(db)
};
let unbound_visibility = match bindings_with_constraints.peek() {
Some(BindingWithConstraints {
binding,
visibility_constraint,
constraints: _,
}) if binding.map_or(true, is_non_exported) => {
visibility_constraints.evaluate(db, *visibility_constraint)
}
_ => Truthiness::AlwaysFalse,
};
let mut types = bindings_with_constraints.filter_map(
@@ -343,6 +438,11 @@ fn symbol_from_bindings<'db>(
visibility_constraint,
}| {
let binding = binding?;
if is_non_exported(binding) {
return None;
}
let static_visibility = visibility_constraints.evaluate(db, visibility_constraint);
if static_visibility.is_always_false() {
@@ -402,9 +502,16 @@ fn symbol_from_bindings<'db>(
/// If we look up the declared type of `variable` in the scope of class `C`, we will get
/// the type `int`, a "declaredness" of [`Boundness::PossiblyUnbound`], and the information
/// that this comes with a [`TypeQualifiers::CLASS_VAR`] type qualifier.
#[derive(Debug)]
pub(crate) struct SymbolAndQualifiers<'db>(Symbol<'db>, TypeQualifiers);
impl SymbolAndQualifiers<'_> {
/// Constructor that creates a [`SymbolAndQualifiers`] instance with a [`TodoType`] type
/// and no qualifiers.
fn todo(message: &'static str) -> Self {
Self(Symbol::todo(message), TypeQualifiers::empty())
}
fn is_class_var(&self) -> bool {
self.1.contains(TypeQualifiers::CLASS_VAR)
}
@@ -420,12 +527,6 @@ impl<'db> From<Symbol<'db>> for SymbolAndQualifiers<'db> {
}
}
impl<'db> From<Type<'db>> for SymbolAndQualifiers<'db> {
fn from(ty: Type<'db>) -> Self {
SymbolAndQualifiers(ty.into(), TypeQualifiers::empty())
}
}
/// The result of looking up a declared type from declarations; see [`symbol_from_declarations`].
type SymbolFromDeclarationsResult<'db> =
Result<SymbolAndQualifiers<'db>, (TypeAndQualifiers<'db>, Box<[Type<'db>]>)>;
@@ -441,18 +542,23 @@ type SymbolFromDeclarationsResult<'db> =
fn symbol_from_declarations<'db>(
db: &'db dyn Db,
declarations: DeclarationsIterator<'_, 'db>,
requires_explicit_reexport: RequiresExplicitReExport,
) -> SymbolFromDeclarationsResult<'db> {
let visibility_constraints = declarations.visibility_constraints;
let mut declarations = declarations.peekable();
let undeclared_visibility = if let Some(DeclarationWithConstraint {
declaration: None,
visibility_constraint,
}) = declarations.peek()
{
visibility_constraints.evaluate(db, *visibility_constraint)
} else {
Truthiness::AlwaysFalse
let is_non_exported = |declaration: Definition<'db>| {
requires_explicit_reexport.is_yes() && !declaration.is_reexported(db)
};
let undeclared_visibility = match declarations.peek() {
Some(DeclarationWithConstraint {
declaration,
visibility_constraint,
}) if declaration.map_or(true, is_non_exported) => {
visibility_constraints.evaluate(db, *visibility_constraint)
}
_ => Truthiness::AlwaysFalse,
};
let mut types = declarations.filter_map(
@@ -461,6 +567,11 @@ fn symbol_from_declarations<'db>(
visibility_constraint,
}| {
let declaration = declaration?;
if is_non_exported(declaration) {
return None;
}
let static_visibility = visibility_constraints.evaluate(db, visibility_constraint);
if static_visibility.is_always_false() {
@@ -561,6 +672,11 @@ macro_rules! todo_type {
$crate::types::TodoType::Message($message),
))
};
($message:ident) => {
$crate::types::Type::Dynamic($crate::types::DynamicType::Todo(
$crate::types::TodoType::Message($message),
))
};
}
#[cfg(not(debug_assertions))]
@@ -571,6 +687,9 @@ macro_rules! todo_type {
($message:literal) => {
$crate::types::Type::Dynamic($crate::types::DynamicType::Todo(crate::types::TodoType))
};
($message:ident) => {
$crate::types::Type::Dynamic($crate::types::DynamicType::Todo(crate::types::TodoType))
};
}
pub(crate) use todo_type;
@@ -631,6 +750,10 @@ impl<'db> Type<'db> {
Self::Dynamic(DynamicType::Unknown)
}
pub fn object(db: &'db dyn Db) -> Self {
KnownClass::Object.to_instance(db)
}
pub const fn is_unknown(&self) -> bool {
matches!(self, Type::Dynamic(DynamicType::Unknown))
}
@@ -639,6 +762,11 @@ impl<'db> Type<'db> {
matches!(self, Type::Never)
}
pub fn is_object(&self, db: &'db dyn Db) -> bool {
self.into_instance()
.is_some_and(|instance| instance.class.is_object(db))
}
pub const fn is_todo(&self) -> bool {
matches!(self, Type::Dynamic(DynamicType::Todo(_)))
}
@@ -895,7 +1023,7 @@ impl<'db> Type<'db> {
// `object` is the only type that can be known to be a supertype of any intersection,
// even an intersection with no positive elements
(Type::Intersection(_), Type::Instance(InstanceType { class }))
if class.is_known(db, KnownClass::Object) =>
if class.is_object(db) =>
{
true
}
@@ -949,7 +1077,7 @@ impl<'db> Type<'db> {
(left, Type::AlwaysTruthy) => left.bool(db).is_always_true(),
// Currently, the only supertype of `AlwaysFalsy` and `AlwaysTruthy` is the universal set (object instance).
(Type::AlwaysFalsy | Type::AlwaysTruthy, _) => {
target.is_equivalent_to(db, KnownClass::Object.to_instance(db))
target.is_equivalent_to(db, Type::object(db))
}
// All `StringLiteral` types are a subtype of `LiteralString`.
@@ -1088,11 +1216,7 @@ impl<'db> Type<'db> {
// All types are assignable to `object`.
// TODO this special case might be removable once the below cases are comprehensive
(_, Type::Instance(InstanceType { class }))
if class.is_known(db, KnownClass::Object) =>
{
true
}
(_, Type::Instance(InstanceType { class })) if class.is_object(db) => true,
// A union is assignable to a type T iff every element of the union is assignable to T.
(Type::Union(union), ty) => union
@@ -1641,6 +1765,7 @@ impl<'db> Type<'db> {
| KnownClass::Type
| KnownClass::Int
| KnownClass::Float
| KnownClass::Complex
| KnownClass::Str
| KnownClass::List
| KnownClass::Tuple
@@ -1684,17 +1809,17 @@ impl<'db> Type<'db> {
#[must_use]
pub(crate) fn member(&self, db: &'db dyn Db, name: &str) -> Symbol<'db> {
if name == "__class__" {
return self.to_meta_type(db).into();
return Symbol::bound(self.to_meta_type(db));
}
match self {
Type::Dynamic(_) => self.into(),
Type::Dynamic(_) => Symbol::bound(self),
Type::Never => todo_type!("attribute lookup on Never").into(),
Type::Never => Symbol::todo("attribute lookup on Never"),
Type::FunctionLiteral(_) => match name {
"__get__" => todo_type!("`__get__` method on functions").into(),
"__call__" => todo_type!("`__call__` method on functions").into(),
"__get__" => Symbol::todo("`__get__` method on functions"),
"__call__" => Symbol::todo("`__call__` method on functions"),
_ => KnownClass::FunctionType.to_instance(db).member(db, name),
},
@@ -1707,12 +1832,12 @@ impl<'db> Type<'db> {
Type::KnownInstance(known_instance) => known_instance.member(db, name),
Type::Instance(InstanceType { class }) => match (class.known(db), name) {
(Some(KnownClass::VersionInfo), "major") => {
Type::IntLiteral(Program::get(db).python_version(db).major.into()).into()
}
(Some(KnownClass::VersionInfo), "minor") => {
Type::IntLiteral(Program::get(db).python_version(db).minor.into()).into()
}
(Some(KnownClass::VersionInfo), "major") => Symbol::bound(Type::IntLiteral(
Program::get(db).python_version(db).major.into(),
)),
(Some(KnownClass::VersionInfo), "minor") => Symbol::bound(Type::IntLiteral(
Program::get(db).python_version(db).minor.into(),
)),
_ => {
let SymbolAndQualifiers(symbol, _) = class.instance_member(db, name);
symbol
@@ -1758,30 +1883,30 @@ impl<'db> Type<'db> {
Type::Intersection(_) => {
// TODO perform the get_member on each type in the intersection
// TODO return the intersection of those results
todo_type!("Attribute access on `Intersection` types").into()
Symbol::todo("Attribute access on `Intersection` types")
}
Type::IntLiteral(_) => match name {
"real" | "numerator" => self.into(),
"real" | "numerator" => Symbol::bound(self),
// TODO more attributes could probably be usefully special-cased
_ => KnownClass::Int.to_instance(db).member(db, name),
},
Type::BooleanLiteral(bool_value) => match name {
"real" | "numerator" => Type::IntLiteral(i64::from(*bool_value)).into(),
"real" | "numerator" => Symbol::bound(Type::IntLiteral(i64::from(*bool_value))),
_ => KnownClass::Bool.to_instance(db).member(db, name),
},
Type::StringLiteral(_) => {
// TODO defer to `typing.LiteralString`/`builtins.str` methods
// from typeshed's stubs
todo_type!("Attribute access on `StringLiteral` types").into()
Symbol::todo("Attribute access on `StringLiteral` types")
}
Type::LiteralString => {
// TODO defer to `typing.LiteralString`/`builtins.str` methods
// from typeshed's stubs
todo_type!("Attribute access on `LiteralString` types").into()
Symbol::todo("Attribute access on `LiteralString` types")
}
Type::BytesLiteral(_) => KnownClass::Bytes.to_instance(db).member(db, name),
@@ -1793,15 +1918,15 @@ impl<'db> Type<'db> {
Type::Tuple(_) => {
// TODO: implement tuple methods
todo_type!("Attribute access on heterogeneous tuple types").into()
Symbol::todo("Attribute access on heterogeneous tuple types")
}
Type::AlwaysTruthy | Type::AlwaysFalsy => match name {
"__bool__" => {
// TODO should be `Callable[[], Literal[True/False]]`
todo_type!("`__bool__` for `AlwaysTruthy`/`AlwaysFalsy` Type variants").into()
Symbol::todo("`__bool__` for `AlwaysTruthy`/`AlwaysFalsy` Type variants")
}
_ => KnownClass::Object.to_instance(db).member(db, name),
_ => Type::object(db).member(db, name),
},
}
}
@@ -1932,7 +2057,7 @@ impl<'db> Type<'db> {
fn call(self, db: &'db dyn Db, arguments: &CallArguments<'_, 'db>) -> CallOutcome<'db> {
match self {
Type::FunctionLiteral(function_type) => {
let mut binding = bind_call(db, arguments, function_type.signature(db), Some(self));
let mut binding = bind_call(db, arguments, function_type.signature(db), self);
match function_type.known(db) {
Some(KnownFunction::RevealType) => {
let revealed_ty = binding.one_parameter_type().unwrap_or(Type::unknown());
@@ -2309,6 +2434,31 @@ impl<'db> Type<'db> {
db: &'db dyn Db,
) -> Result<Type<'db>, InvalidTypeExpressionError<'db>> {
match self {
// Special cases for `float` and `complex`
// https://typing.readthedocs.io/en/latest/spec/special-types.html#special-cases-for-float-and-complex
Type::ClassLiteral(ClassLiteralType { class })
if class.is_known(db, KnownClass::Float) =>
{
Ok(UnionType::from_elements(
db,
[
KnownClass::Int.to_instance(db),
KnownClass::Float.to_instance(db),
],
))
}
Type::ClassLiteral(ClassLiteralType { class })
if class.is_known(db, KnownClass::Complex) =>
{
Ok(UnionType::from_elements(
db,
[
KnownClass::Int.to_instance(db),
KnownClass::Float.to_instance(db),
KnownClass::Complex.to_instance(db),
],
))
}
// In a type expression, a bare `type` is interpreted as "instance of `type`", which is
// equivalent to `type[object]`.
Type::ClassLiteral(_) | Type::SubclassOf(_) => Ok(self.to_instance(db)),
@@ -2524,18 +2674,6 @@ impl<'db> From<&Type<'db>> for Type<'db> {
}
}
impl<'db> From<Type<'db>> for Symbol<'db> {
fn from(value: Type<'db>) -> Self {
Symbol::Type(value, Boundness::Bound)
}
}
impl<'db> From<&Type<'db>> for Symbol<'db> {
fn from(value: &Type<'db>) -> Self {
Self::from(*value)
}
}
#[derive(Copy, Clone, Debug, Eq, Hash, PartialEq)]
pub enum DynamicType {
// An explicitly annotated `typing.Any`
@@ -2568,7 +2706,7 @@ impl std::fmt::Display for DynamicType {
bitflags! {
/// Type qualifiers that appear in an annotation expression.
#[derive(Copy, Clone, Debug, Eq, PartialEq)]
#[derive(Copy, Clone, Debug, Eq, PartialEq, Default)]
pub(crate) struct TypeQualifiers: u8 {
/// `typing.ClassVar`
const CLASS_VAR = 1 << 0;
@@ -2584,7 +2722,7 @@ bitflags! {
///
/// Example: `Annotated[ClassVar[tuple[int]], "metadata"]` would have type `tuple[int]` and the
/// qualifier `ClassVar`.
#[derive(Clone, Debug, Copy, Eq, PartialEq)]
#[derive(Clone, Debug, Copy, Eq, PartialEq, salsa::Update)]
pub(crate) struct TypeAndQualifiers<'db> {
inner: Type<'db>,
qualifiers: TypeQualifiers,
@@ -2595,6 +2733,14 @@ impl<'db> TypeAndQualifiers<'db> {
Self { inner, qualifiers }
}
/// Constructor that creates a [`TypeAndQualifiers`] instance with type `Unknown` and no qualifiers.
pub(crate) fn unknown() -> Self {
Self {
inner: Type::unknown(),
qualifiers: TypeQualifiers::empty(),
}
}
/// Forget about type qualifiers and only return the inner type.
pub(crate) fn inner_type(&self) -> Type<'db> {
self.inner
@@ -2688,6 +2834,7 @@ pub enum KnownClass {
Type,
Int,
Float,
Complex,
Str,
List,
Tuple,
@@ -2733,6 +2880,7 @@ impl<'db> KnownClass {
Self::Tuple => "tuple",
Self::Int => "int",
Self::Float => "float",
Self::Complex => "complex",
Self::FrozenSet => "frozenset",
Self::Str => "str",
Self::Set => "set",
@@ -2802,6 +2950,7 @@ impl<'db> KnownClass {
| Self::Type
| Self::Int
| Self::Float
| Self::Complex
| Self::Str
| Self::List
| Self::Tuple
@@ -2851,6 +3000,7 @@ impl<'db> KnownClass {
| Self::Tuple
| Self::Int
| Self::Float
| Self::Complex
| Self::Str
| Self::Set
| Self::FrozenSet
@@ -2887,6 +3037,7 @@ impl<'db> KnownClass {
"type" => Self::Type,
"int" => Self::Int,
"float" => Self::Float,
"complex" => Self::Complex,
"str" => Self::Str,
"set" => Self::Set,
"frozenset" => Self::FrozenSet,
@@ -2926,6 +3077,7 @@ impl<'db> KnownClass {
| Self::Type
| Self::Int
| Self::Float
| Self::Complex
| Self::Str
| Self::List
| Self::Tuple
@@ -3326,7 +3478,7 @@ impl<'db> KnownInstanceType<'db> {
(Self::TypeAliasType(alias), "__name__") => Type::string_literal(db, alias.name(db)),
_ => return self.instance_fallback(db).member(db, name),
};
ty.into()
Symbol::bound(ty)
}
}
@@ -3784,34 +3936,12 @@ impl<'db> ModuleLiteralType<'db> {
full_submodule_name.extend(&submodule_name);
if imported_submodules.contains(&full_submodule_name) {
if let Some(submodule) = resolve_module(db, &full_submodule_name) {
let submodule_ty = Type::module_literal(db, importing_file, submodule);
return Symbol::Type(submodule_ty, Boundness::Bound);
return Symbol::bound(Type::module_literal(db, importing_file, submodule));
}
}
}
// If it's not found in the global scope, check if it's present as an instance
// on `types.ModuleType` or `builtins.object`.
//
// We do a more limited version of this in `global_symbol_ty`,
// but there are two crucial differences here:
// - If a member is looked up as an attribute, `__init__` is also available
// on the module, but it isn't available as a global from inside the module
// - If a member is looked up as an attribute, members on `builtins.object`
// are also available (because `types.ModuleType` inherits from `object`);
// these attributes are also not available as globals from inside the module.
//
// The same way as in `global_symbol_ty`, however, we need to be careful to
// ignore `__getattr__`. Typeshed has a fake `__getattr__` on `types.ModuleType`
// to help out with dynamic imports; we shouldn't use it for `ModuleLiteral` types
// where we know exactly which module we're dealing with.
symbol(db, global_scope(db, self.module(db).file()), name).or_fall_back_to(db, || {
if name == "__getattr__" {
Symbol::Unbound
} else {
KnownClass::ModuleType.to_instance(db).member(db, name)
}
})
imported_symbol(db, &self.module(db), name)
}
}
@@ -3853,6 +3983,11 @@ impl<'db> Class<'db> {
self.known(db) == Some(known_class)
}
/// Return `true` if this class represents the builtin class `object`
pub fn is_object(self, db: &'db dyn Db) -> bool {
self.is_known(db, KnownClass::Object)
}
/// Return an iterator over the inferred types of this class's *explicit* bases.
///
/// Note that any class (except for `object`) that has no explicit
@@ -4114,24 +4249,48 @@ impl<'db> Class<'db> {
pub(crate) fn class_member(self, db: &'db dyn Db, name: &str) -> Symbol<'db> {
if name == "__mro__" {
let tuple_elements = self.iter_mro(db).map(Type::from);
return TupleType::from_elements(db, tuple_elements).into();
return Symbol::bound(TupleType::from_elements(db, tuple_elements));
}
// If we encounter a dynamic type in this class's MRO, we'll save that dynamic type
// in this variable. After we've traversed the MRO, we'll either:
// (1) Use that dynamic type as the type for this attribute,
// if no other classes in the MRO define the attribute; or,
// (2) Intersect that dynamic type with the type of the attribute
// from the non-dynamic members of the class's MRO.
let mut dynamic_type_to_intersect_with: Option<Type<'db>> = None;
let mut lookup_result: LookupResult<'db> = Err(LookupError::Unbound);
for superclass in self.iter_mro(db) {
match superclass {
// TODO we may instead want to record the fact that we encountered dynamic, and intersect it with
// the type found on the next "real" class.
ClassBase::Dynamic(_) => return Type::from(superclass).member(db, name),
ClassBase::Class(class) => {
let member = class.own_class_member(db, name);
if !member.is_unbound() {
return member;
}
ClassBase::Dynamic(_) => {
// Note: calling `Type::from(superclass).member()` would be incorrect here.
// What we'd really want is a `Type::Any.own_class_member()` method,
// but adding such a method wouldn't make much sense -- it would always return `Any`!
dynamic_type_to_intersect_with.get_or_insert(Type::from(superclass));
}
ClassBase::Class(class) => {
lookup_result = lookup_result.or_else(|lookup_error| {
lookup_error.or_fall_back_to(db, class.own_class_member(db, name))
});
}
}
if lookup_result.is_ok() {
break;
}
}
Symbol::Unbound
match (Symbol::from(lookup_result), dynamic_type_to_intersect_with) {
(symbol, None) => symbol,
(Symbol::Type(ty, _), Some(dynamic_type)) => Symbol::bound(
IntersectionBuilder::new(db)
.add_positive(ty)
.add_positive(dynamic_type)
.build(),
),
(Symbol::Unbound, Some(dynamic_type)) => Symbol::bound(dynamic_type),
}
}
/// Returns the inferred type of the class member named `name`.
@@ -4154,7 +4313,9 @@ impl<'db> Class<'db> {
for superclass in self.iter_mro(db) {
match superclass {
ClassBase::Dynamic(_) => {
return todo_type!("instance attribute on class with dynamic base").into();
return SymbolAndQualifiers::todo(
"instance attribute on class with dynamic base",
);
}
ClassBase::Class(class) => {
if let member @ SymbolAndQualifiers(Symbol::Type(_, _), _) =
@@ -4204,7 +4365,7 @@ impl<'db> Class<'db> {
.and_then(|assignments| assignments.get(name))
else {
if inferred_type_from_class_body.is_some() {
return union_of_inferred_types.build().into();
return Symbol::bound(union_of_inferred_types.build());
}
return Symbol::Unbound;
};
@@ -4221,7 +4382,7 @@ impl<'db> Class<'db> {
let annotation_ty = infer_expression_type(db, *annotation);
// TODO: check if there are conflicting declarations
return annotation_ty.into();
return Symbol::bound(annotation_ty);
}
AttributeAssignment::Unannotated { value } => {
// We found an un-annotated attribute assignment of the form:
@@ -4261,7 +4422,7 @@ impl<'db> Class<'db> {
}
}
union_of_inferred_types.build().into()
Symbol::bound(union_of_inferred_types.build())
}
/// A helper function for `instance_member` that looks up the `name` attribute only on
@@ -4281,7 +4442,7 @@ impl<'db> Class<'db> {
let declarations = use_def.public_declarations(symbol_id);
match symbol_from_declarations(db, declarations) {
match symbol_from_declarations(db, declarations, RequiresExplicitReExport::No) {
Ok(SymbolAndQualifiers(Symbol::Type(declared_ty, _), qualifiers)) => {
// The attribute is declared in the class body.
@@ -4290,12 +4451,12 @@ impl<'db> Class<'db> {
// just a temporary heuristic to provide a broad categorization into properties
// and non-property methods.
if function.has_decorator(db, KnownClass::Property.to_class_literal(db)) {
todo_type!("@property").into()
SymbolAndQualifiers::todo("@property")
} else {
todo_type!("bound method").into()
SymbolAndQualifiers::todo("bound method")
}
} else {
SymbolAndQualifiers(Symbol::Type(declared_ty, Boundness::Bound), qualifiers)
SymbolAndQualifiers(Symbol::bound(declared_ty), qualifiers)
}
}
Ok(SymbolAndQualifiers(Symbol::Unbound, _)) => {
@@ -4303,14 +4464,17 @@ impl<'db> Class<'db> {
// in a method, and it could also be *bound* in the class body (and/or in a method).
let bindings = use_def.public_bindings(symbol_id);
let inferred = symbol_from_bindings(db, bindings);
let inferred = symbol_from_bindings(db, bindings, RequiresExplicitReExport::No);
let inferred_ty = inferred.ignore_possibly_unbound();
Self::implicit_instance_attribute(db, body_scope, name, inferred_ty).into()
}
Err((declared_ty, _conflicting_declarations)) => {
// There are conflicting declarations for this attribute in the class body.
SymbolAndQualifiers(declared_ty.inner_type().into(), declared_ty.qualifiers())
SymbolAndQualifiers(
Symbol::bound(declared_ty.inner_type()),
declared_ty.qualifiers(),
)
}
}
} else {
@@ -4390,7 +4554,7 @@ impl<'db> TypeAliasType<'db> {
}
/// Either the explicit `metaclass=` keyword of the class, or the inferred metaclass of one of its base classes.
#[derive(Debug, Clone, PartialEq, Eq)]
#[derive(Debug, Clone, PartialEq, Eq, salsa::Update)]
pub(super) struct MetaclassCandidate<'db> {
metaclass: Class<'db>,
explicit_metaclass_of: Class<'db>,
@@ -4433,7 +4597,7 @@ impl<'db> From<InstanceType<'db>> for Type<'db> {
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
#[derive(Debug, Clone, PartialEq, Eq, salsa::Update)]
pub(super) struct MetaclassError<'db> {
kind: MetaclassErrorKind<'db>,
}
@@ -4445,7 +4609,7 @@ impl<'db> MetaclassError<'db> {
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
#[derive(Debug, Clone, PartialEq, Eq, salsa::Update)]
pub(super) enum MetaclassErrorKind<'db> {
/// The class has incompatible metaclasses in its inheritance hierarchy.
///
@@ -4842,12 +5006,12 @@ pub(crate) mod tests {
use super::*;
use crate::db::tests::{setup_db, TestDbBuilder};
use crate::stdlib::typing_symbol;
use crate::PythonVersion;
use ruff_db::files::system_path_to_file;
use ruff_db::parsed::parsed_module;
use ruff_db::system::DbWithTestSystem;
use ruff_db::testing::assert_function_query_was_not_run;
use ruff_python_ast as ast;
use ruff_python_ast::python_version::PythonVersion;
use test_case::test_case;
/// Explicitly test for Python version <3.13 and >=3.13, to ensure that

View File

@@ -43,6 +43,13 @@ impl<'db> UnionBuilder<'db> {
}
}
/// Collapse the union to a single type: `object`.
fn collapse_to_object(mut self) -> Self {
self.elements.clear();
self.elements.push(Type::object(self.db));
self
}
/// Adds a type to this union.
pub(crate) fn add(mut self, ty: Type<'db>) -> Self {
match ty {
@@ -53,7 +60,12 @@ impl<'db> UnionBuilder<'db> {
self = self.add(*element);
}
}
// Adding `Never` to a union is a no-op.
Type::Never => {}
// Adding `object` to a union results in `object`.
ty if ty.is_object(self.db) => {
return self.collapse_to_object();
}
_ => {
let bool_pair = if let Type::BooleanLiteral(b) = ty {
Some(Type::BooleanLiteral(!b))
@@ -76,7 +88,10 @@ impl<'db> UnionBuilder<'db> {
break;
}
if ty.is_same_gradual_form(*element) || ty.is_subtype_of(self.db, *element) {
if ty.is_same_gradual_form(*element)
|| ty.is_subtype_of(self.db, *element)
|| element.is_object(self.db)
{
return self;
} else if element.is_subtype_of(self.db, ty) {
to_remove.push(index);
@@ -88,9 +103,7 @@ impl<'db> UnionBuilder<'db> {
// `element | ty` must be `object` (object has no other supertypes). This means we can simplify
// the whole union to just `object`, since all other potential elements would also be subtypes of
// `object`.
self.elements.clear();
self.elements.push(KnownClass::Object.to_instance(self.db));
return self;
return self.collapse_to_object();
}
}
match to_remove[..] {
@@ -416,7 +429,7 @@ impl<'db> InnerIntersectionBuilder<'db> {
Type::Never => {
// Adding ~Never to an intersection is a no-op.
}
Type::Instance(instance) if instance.class.is_known(db, KnownClass::Object) => {
Type::Instance(instance) if instance.class.is_object(db) => {
// Adding ~object to an intersection results in Never.
*self = Self::default();
self.positive.insert(Type::Never);
@@ -481,7 +494,7 @@ impl<'db> InnerIntersectionBuilder<'db> {
fn build(mut self, db: &'db dyn Db) -> Type<'db> {
match (self.positive.len(), self.negative.len()) {
(0, 0) => KnownClass::Object.to_instance(db),
(0, 0) => Type::object(db),
(1, 0) => self.positive[0],
_ => {
self.positive.shrink_to_fit();
@@ -534,7 +547,7 @@ mod tests {
let db = setup_db();
let intersection = IntersectionBuilder::new(&db).build();
assert_eq!(intersection, KnownClass::Object.to_instance(&db));
assert_eq!(intersection, Type::object(&db));
}
#[test_case(Type::BooleanLiteral(true))]
@@ -548,7 +561,7 @@ mod tests {
// We add t_object in various orders (in first or second position) in
// the tests below to ensure that the boolean simplification eliminates
// everything from the intersection, not just `bool`.
let t_object = KnownClass::Object.to_instance(&db);
let t_object = Type::object(&db);
let t_bool = KnownClass::Bool.to_instance(&db);
let ty = IntersectionBuilder::new(&db)

View File

@@ -5,7 +5,7 @@ use crate::types::diagnostic::{
TOO_MANY_POSITIONAL_ARGUMENTS, UNKNOWN_ARGUMENT,
};
use crate::types::signatures::Parameter;
use crate::types::UnionType;
use crate::types::{todo_type, UnionType};
use ruff_python_ast as ast;
/// Bind a [`CallArguments`] against a callable [`Signature`].
@@ -16,7 +16,7 @@ pub(crate) fn bind_call<'db>(
db: &'db dyn Db,
arguments: &CallArguments<'_, 'db>,
signature: &Signature<'db>,
callable_ty: Option<Type<'db>>,
callable_ty: Type<'db>,
) -> CallBinding<'db> {
let parameters = signature.parameters();
// The type assigned to each parameter at this call site.
@@ -138,7 +138,7 @@ pub(crate) fn bind_call<'db>(
#[derive(Debug, Clone, PartialEq, Eq)]
pub(crate) struct CallBinding<'db> {
/// Type of the callable object (function, class...)
callable_ty: Option<Type<'db>>,
callable_ty: Type<'db>,
/// Return type of the call.
return_ty: Type<'db>,
@@ -154,7 +154,7 @@ impl<'db> CallBinding<'db> {
// TODO remove this constructor and construct always from `bind_call`
pub(crate) fn from_return_type(return_ty: Type<'db>) -> Self {
Self {
callable_ty: None,
callable_ty: todo_type!("CallBinding::from_return_type"),
return_ty,
parameter_tys: Box::default(),
errors: vec![],
@@ -189,8 +189,8 @@ impl<'db> CallBinding<'db> {
fn callable_name(&self, db: &'db dyn Db) -> Option<&str> {
match self.callable_ty {
Some(Type::FunctionLiteral(function)) => Some(function.name(db)),
Some(Type::ClassLiteral(class_type)) => Some(class_type.class.name(db)),
Type::FunctionLiteral(function) => Some(function.name(db)),
Type::ClassLiteral(class_type) => Some(class_type.class.name(db)),
_ => None,
}
}

View File

@@ -8,7 +8,7 @@ use crate::types::string_annotation::{
};
use crate::types::{ClassLiteralType, KnownInstanceType, Type};
use crate::{declare_lint, Db};
use ruff_db::diagnostic::{Diagnostic, DiagnosticId, Severity};
use ruff_db::diagnostic::{Diagnostic, DiagnosticId, Severity, Span};
use ruff_db::files::File;
use ruff_python_ast::{self as ast, AnyNodeRef};
use ruff_text_size::{Ranged, TextRange};
@@ -802,12 +802,8 @@ impl Diagnostic for TypeCheckDiagnostic {
TypeCheckDiagnostic::message(self).into()
}
fn file(&self) -> Option<File> {
Some(TypeCheckDiagnostic::file(self))
}
fn range(&self) -> Option<TextRange> {
Some(Ranged::range(self))
fn span(&self) -> Option<Span> {
Some(Span::from(self.file).with_range(self.range))
}
fn severity(&self) -> Severity {
@@ -830,41 +826,77 @@ impl Ranged for TypeCheckDiagnostic {
/// each Salsa-struct comes with an overhead.
#[derive(Default, Eq, PartialEq)]
pub struct TypeCheckDiagnostics {
inner: Option<Box<TypeCheckDiagnosticsInner>>,
}
#[derive(Default, Eq, PartialEq)]
struct TypeCheckDiagnosticsInner {
diagnostics: Vec<Arc<TypeCheckDiagnostic>>,
used_suppressions: FxHashSet<FileSuppressionId>,
}
impl TypeCheckDiagnostics {
pub(crate) fn push(&mut self, diagnostic: TypeCheckDiagnostic) {
self.diagnostics.push(Arc::new(diagnostic));
let inner = self.get_mut_inner();
inner.diagnostics.push(Arc::new(diagnostic));
}
pub(super) fn extend(&mut self, other: &TypeCheckDiagnostics) {
self.diagnostics.extend_from_slice(&other.diagnostics);
self.used_suppressions.extend(&other.used_suppressions);
let Some(other_inner) = other.inner.as_ref() else {
return;
};
let inner = self.get_mut_inner();
inner
.diagnostics
.extend_from_slice(&other_inner.diagnostics);
inner
.used_suppressions
.extend(&other_inner.used_suppressions);
}
pub(crate) fn mark_used(&mut self, suppression_id: FileSuppressionId) {
self.used_suppressions.insert(suppression_id);
let inner = self.get_mut_inner();
inner.used_suppressions.insert(suppression_id);
}
pub(crate) fn diagnostics(&self) -> &[Arc<TypeCheckDiagnostic>] {
self.get_inner()
.map(|inner| &*inner.diagnostics)
.unwrap_or_default()
}
pub(crate) fn is_used(&self, suppression_id: FileSuppressionId) -> bool {
self.used_suppressions.contains(&suppression_id)
self.get_inner()
.is_some_and(|inner| inner.used_suppressions.contains(&suppression_id))
}
pub(crate) fn used_len(&self) -> usize {
self.used_suppressions.len()
self.get_inner()
.map(|inner| inner.used_suppressions.len())
.unwrap_or_default()
}
pub(crate) fn shrink_to_fit(&mut self) {
self.used_suppressions.shrink_to_fit();
self.diagnostics.shrink_to_fit();
if let Some(inner) = self.inner.as_mut() {
inner.used_suppressions.shrink_to_fit();
inner.diagnostics.shrink_to_fit();
}
}
fn get_mut_inner(&mut self) -> &mut TypeCheckDiagnosticsInner {
self.inner
.get_or_insert_with(|| Box::new(TypeCheckDiagnosticsInner::default()))
}
fn get_inner(&self) -> Option<&TypeCheckDiagnosticsInner> {
self.inner.as_deref()
}
}
impl std::fmt::Debug for TypeCheckDiagnostics {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
self.diagnostics.fmt(f)
self.diagnostics().fmt(f)
}
}
@@ -872,7 +904,7 @@ impl Deref for TypeCheckDiagnostics {
type Target = [std::sync::Arc<TypeCheckDiagnostic>];
fn deref(&self) -> &Self::Target {
&self.diagnostics
self.diagnostics()
}
}
@@ -881,7 +913,10 @@ impl IntoIterator for TypeCheckDiagnostics {
type IntoIter = std::vec::IntoIter<std::sync::Arc<TypeCheckDiagnostic>>;
fn into_iter(self) -> Self::IntoIter {
self.diagnostics.into_iter()
self.inner
.map(|inner| inner.diagnostics)
.unwrap_or_default()
.into_iter()
}
}
@@ -890,7 +925,7 @@ impl<'a> IntoIterator for &'a TypeCheckDiagnostics {
type IntoIter = std::slice::Iter<'a, std::sync::Arc<TypeCheckDiagnostic>>;
fn into_iter(self) -> Self::IntoIter {
self.diagnostics.iter()
self.diagnostics().iter()
}
}

View File

@@ -49,6 +49,7 @@ use crate::semantic_index::semantic_index;
use crate::semantic_index::symbol::{NodeWithScopeKind, NodeWithScopeRef, ScopeId};
use crate::semantic_index::SemanticIndex;
use crate::stdlib::builtins_module_scope;
use crate::symbol::LookupError;
use crate::types::call::{Argument, CallArguments};
use crate::types::diagnostic::{
report_invalid_arguments_to_annotated, report_invalid_assignment,
@@ -61,16 +62,15 @@ use crate::types::diagnostic::{
UNDEFINED_REVEAL, UNRESOLVED_ATTRIBUTE, UNRESOLVED_IMPORT, UNSUPPORTED_OPERATOR,
};
use crate::types::mro::MroErrorKind;
use crate::types::statistics::TypeStatistics;
use crate::types::unpacker::{UnpackResult, Unpacker};
use crate::types::{
builtins_symbol, global_symbol, symbol, symbol_from_bindings, symbol_from_declarations,
todo_type, typing_extensions_symbol, Boundness, CallDunderResult, Class, ClassLiteralType,
DynamicType, FunctionType, InstanceType, IntersectionBuilder, IntersectionType,
IterationOutcome, KnownClass, KnownFunction, KnownInstanceType, MetaclassCandidate,
MetaclassErrorKind, SliceLiteralType, SubclassOfType, Symbol, SymbolAndQualifiers, Truthiness,
TupleType, Type, TypeAliasType, TypeAndQualifiers, TypeArrayDisplay, TypeQualifiers,
TypeVarBoundOrConstraints, TypeVarInstance, UnionBuilder, UnionType,
builtins_symbol, symbol, symbol_from_bindings, symbol_from_declarations, todo_type,
typing_extensions_symbol, Boundness, CallDunderResult, Class, ClassLiteralType, DynamicType,
FunctionType, InstanceType, IntersectionBuilder, IntersectionType, IterationOutcome,
KnownClass, KnownFunction, KnownInstanceType, MetaclassCandidate, MetaclassErrorKind,
RequiresExplicitReExport, SliceLiteralType, SubclassOfType, Symbol, SymbolAndQualifiers,
Truthiness, TupleType, Type, TypeAliasType, TypeAndQualifiers, TypeArrayDisplay,
TypeQualifiers, TypeVarBoundOrConstraints, TypeVarInstance, UnionBuilder, UnionType,
};
use crate::unpack::Unpack;
use crate::util::subscript::{PyIndex, PySlice};
@@ -87,7 +87,7 @@ use super::slots::check_class_slots;
use super::string_annotation::{
parse_string_annotation, BYTE_STRING_TYPE_ANNOTATION, FSTRING_TYPE_ANNOTATION,
};
use super::{ParameterExpectation, ParameterExpectations};
use super::{global_symbol, ParameterExpectation, ParameterExpectations};
/// Infer all types for a [`ScopeId`], including all definitions and expressions in that scope.
/// Use when checking a scope, or needing to provide a type for an arbitrary expression in the
@@ -117,7 +117,9 @@ fn infer_definition_types_cycle_recovery<'db>(
let mut inference = TypeInference::empty(input.scope(db));
let category = input.category(db);
if category.is_declaration() {
inference.declarations.insert(input, Type::unknown().into());
inference
.declarations
.insert(input, TypeAndQualifiers::unknown());
}
if category.is_binding() {
inference.bindings.insert(input, Type::unknown());
@@ -237,7 +239,7 @@ impl<'db> InferenceRegion<'db> {
}
/// The inferred types for a single region.
#[derive(Debug, Eq, PartialEq)]
#[derive(Debug, Eq, PartialEq, salsa::Update)]
pub(crate) struct TypeInference<'db> {
/// The types of every expression in this region.
expressions: FxHashMap<ScopedExpressionId, Type<'db>>,
@@ -300,14 +302,6 @@ impl<'db> TypeInference<'db> {
self.diagnostics.shrink_to_fit();
self.deferred.shrink_to_fit();
}
pub(super) fn statistics(&self) -> TypeStatistics {
let mut statistics = TypeStatistics::default();
for ty in self.expressions.values() {
statistics.increment(*ty);
}
statistics
}
}
impl WithDiagnostics for TypeInference<'_> {
@@ -742,7 +736,7 @@ impl<'db> TypeInferenceBuilder<'db> {
self.infer_type_alias_definition(type_alias.node(), definition);
}
DefinitionKind::Import(import) => {
self.infer_import_definition(import.node(), definition);
self.infer_import_definition(import.alias(), definition);
}
DefinitionKind::ImportFrom(import_from) => {
self.infer_import_from_definition(
@@ -878,22 +872,25 @@ impl<'db> TypeInferenceBuilder<'db> {
let use_def = self.index.use_def_map(binding.file_scope(self.db()));
let declarations = use_def.declarations_at_binding(binding);
let mut bound_ty = ty;
let declared_ty = symbol_from_declarations(self.db(), declarations)
.map(|SymbolAndQualifiers(s, _)| s.ignore_possibly_unbound().unwrap_or(Type::unknown()))
.unwrap_or_else(|(ty, conflicting)| {
// TODO point out the conflicting declarations in the diagnostic?
let symbol_table = self.index.symbol_table(binding.file_scope(self.db()));
let symbol_name = symbol_table.symbol(binding.symbol(self.db())).name();
self.context.report_lint(
&CONFLICTING_DECLARATIONS,
node,
format_args!(
"Conflicting declared types for `{symbol_name}`: {}",
conflicting.display(self.db())
),
);
ty.inner_type()
});
let declared_ty =
symbol_from_declarations(self.db(), declarations, RequiresExplicitReExport::No)
.map(|SymbolAndQualifiers(s, _)| {
s.ignore_possibly_unbound().unwrap_or(Type::unknown())
})
.unwrap_or_else(|(ty, conflicting)| {
// TODO point out the conflicting declarations in the diagnostic?
let symbol_table = self.index.symbol_table(binding.file_scope(self.db()));
let symbol_name = symbol_table.symbol(binding.symbol(self.db())).name();
self.context.report_lint(
&CONFLICTING_DECLARATIONS,
node,
format_args!(
"Conflicting declared types for `{symbol_name}`: {}",
conflicting.display(self.db())
),
);
ty.inner_type()
});
if !bound_ty.is_assignable_to(self.db(), declared_ty) {
report_invalid_assignment(&self.context, node, declared_ty, bound_ty);
// allow declarations to override inference in case of invalid assignment
@@ -913,9 +910,10 @@ impl<'db> TypeInferenceBuilder<'db> {
let use_def = self.index.use_def_map(declaration.file_scope(self.db()));
let prior_bindings = use_def.bindings_at_declaration(declaration);
// unbound_ty is Never because for this check we don't care about unbound
let inferred_ty = symbol_from_bindings(self.db(), prior_bindings)
.ignore_possibly_unbound()
.unwrap_or(Type::Never);
let inferred_ty =
symbol_from_bindings(self.db(), prior_bindings, RequiresExplicitReExport::No)
.ignore_possibly_unbound()
.unwrap_or(Type::Never);
let ty = if inferred_ty.is_assignable_to(self.db(), ty.inner_type()) {
ty
} else {
@@ -928,7 +926,7 @@ impl<'db> TypeInferenceBuilder<'db> {
inferred_ty.display(self.db())
),
);
Type::unknown().into()
TypeAndQualifiers::unknown()
};
self.types.declarations.insert(declaration, ty);
}
@@ -3297,38 +3295,84 @@ impl<'db> TypeInferenceBuilder<'db> {
todo_type!("generic `typing.Awaitable` type")
}
/// Look up a name reference that isn't bound in the local scope.
fn lookup_name(&mut self, name_node: &ast::ExprName) -> Symbol<'db> {
/// Infer the type of a [`ast::ExprName`] expression, assuming a load context.
fn infer_name_load(&mut self, name_node: &ast::ExprName) -> Type<'db> {
let ast::ExprName {
range: _,
id: symbol_name,
ctx: _,
} = name_node;
let db = self.db();
let ast::ExprName { id: name, .. } = name_node;
let file_scope_id = self.scope().file_scope_id(db);
let is_bound =
if let Some(symbol) = self.index.symbol_table(file_scope_id).symbol_by_name(name) {
symbol.is_bound()
let scope = self.scope();
let file_scope_id = scope.file_scope_id(db);
let symbol_table = self.index.symbol_table(file_scope_id);
let use_def = self.index.use_def_map(file_scope_id);
// If we're inferring types of deferred expressions, always treat them as public symbols
let local_scope_symbol = if self.is_deferred() {
if let Some(symbol_id) = symbol_table.symbol_id_by_name(symbol_name) {
symbol_from_bindings(
db,
use_def.public_bindings(symbol_id),
RequiresExplicitReExport::No,
)
} else {
assert!(
self.deferred_state.in_string_annotation(),
"Expected the symbol table to create a symbol for every Name node"
);
false
Symbol::Unbound
}
} else {
let use_id = name_node.scoped_use_id(db, scope);
symbol_from_bindings(
db,
use_def.bindings_at_use(use_id),
RequiresExplicitReExport::No,
)
};
let symbol = local_scope_symbol.or_fall_back_to(db, || {
let has_bindings_in_this_scope = match symbol_table.symbol_by_name(symbol_name) {
Some(symbol) => symbol.is_bound(),
None => {
assert!(
self.deferred_state.in_string_annotation(),
"Expected the symbol table to create a symbol for every Name node"
);
false
}
};
// In function-like scopes, any local variable (symbol that is bound in this scope) can
// only have a definition in this scope, or error; it never references another scope.
// (At runtime, it would use the `LOAD_FAST` opcode.)
if !is_bound || !self.scope().is_function_like(db) {
// If it's a function-like scope and there is one or more binding in this scope (but
// none of those bindings are visible from where we are in the control flow), we cannot
// fallback to any bindings in enclosing scopes. As such, we can immediately short-circuit
// here and return `Symbol::Unbound`.
//
// This is because Python is very strict in its categorisation of whether a variable is
// a local variable or not in function-like scopes. If a variable has any bindings in a
// function-like scope, it is considered a local variable; it never references another
// scope. (At runtime, it would use the `LOAD_FAST` opcode.)
if has_bindings_in_this_scope && scope.is_function_like(db) {
return Symbol::Unbound;
}
let current_file = self.file();
// Walk up parent scopes looking for a possible enclosing scope that may have a
// definition of this name visible to us (would be `LOAD_DEREF` at runtime.)
for (enclosing_scope_file_id, _) in self.index.ancestor_scopes(file_scope_id) {
// Class scopes are not visible to nested scopes, and we need to handle global
// scope differently (because an unbound name there falls back to builtins), so
// check only function-like scopes.
let enclosing_scope_id = enclosing_scope_file_id.to_scope_id(db, self.file());
let enclosing_scope_id = enclosing_scope_file_id.to_scope_id(db, current_file);
if !enclosing_scope_id.is_function_like(db) {
continue;
}
let enclosing_symbol_table = self.index.symbol_table(enclosing_scope_file_id);
let Some(enclosing_symbol) = enclosing_symbol_table.symbol_by_name(name) else {
let Some(enclosing_symbol) = enclosing_symbol_table.symbol_by_name(symbol_name)
else {
continue;
};
if enclosing_symbol.is_bound() {
@@ -3337,7 +3381,7 @@ impl<'db> TypeInferenceBuilder<'db> {
// runtime, it is the scope that creates the cell for our closure.) If the name
// isn't bound in that scope, we should get an unbound name, not continue
// falling back to other scopes / globals / builtins.
return symbol(db, enclosing_scope_id, name);
return symbol(db, enclosing_scope_id, symbol_name);
}
}
@@ -3348,7 +3392,7 @@ impl<'db> TypeInferenceBuilder<'db> {
if file_scope_id.is_global() {
Symbol::Unbound
} else {
global_symbol(db, self.file(), name)
global_symbol(db, self.file(), symbol_name)
}
})
// Not found in globals? Fallback to builtins
@@ -3357,12 +3401,12 @@ impl<'db> TypeInferenceBuilder<'db> {
if Some(self.scope()) == builtins_module_scope(db) {
Symbol::Unbound
} else {
builtins_symbol(db, name)
builtins_symbol(db, symbol_name)
}
})
// Still not found? It might be `reveal_type`...
.or_fall_back_to(db, || {
if name == "reveal_type" {
if symbol_name == "reveal_type" {
self.context.report_lint(
&UNDEFINED_REVEAL,
name_node.into(),
@@ -3371,70 +3415,23 @@ impl<'db> TypeInferenceBuilder<'db> {
this is allowed for debugging convenience but will fail at runtime"
),
);
typing_extensions_symbol(db, name)
typing_extensions_symbol(db, symbol_name)
} else {
Symbol::Unbound
}
})
} else {
Symbol::Unbound
}
}
});
/// Infer the type of a [`ast::ExprName`] expression, assuming a load context.
fn infer_name_load(&mut self, name: &ast::ExprName) -> Type<'db> {
let ast::ExprName {
range: _,
id,
ctx: _,
} = name;
let file_scope_id = self.scope().file_scope_id(self.db());
let use_def = self.index.use_def_map(file_scope_id);
// If we're inferring types of deferred expressions, always treat them as public symbols
let inferred = if self.is_deferred() {
if let Some(symbol) = self.index.symbol_table(file_scope_id).symbol_id_by_name(id) {
symbol_from_bindings(self.db(), use_def.public_bindings(symbol))
} else {
assert!(
self.deferred_state.in_string_annotation(),
"Expected the symbol table to create a symbol for every Name node"
);
Symbol::Unbound
symbol.unwrap_with_diagnostic(|lookup_error| match lookup_error {
LookupError::Unbound => {
report_unresolved_reference(&self.context, name_node);
Type::unknown()
}
} else {
let use_id = name.scoped_use_id(self.db(), self.scope());
symbol_from_bindings(self.db(), use_def.bindings_at_use(use_id))
};
if let Symbol::Type(ty, Boundness::Bound) = inferred {
ty
} else {
match self.lookup_name(name) {
Symbol::Type(looked_up_ty, looked_up_boundness) => {
if looked_up_boundness == Boundness::PossiblyUnbound {
report_possibly_unresolved_reference(&self.context, name);
}
inferred
.ignore_possibly_unbound()
.map(|ty| UnionType::from_elements(self.db(), [ty, looked_up_ty]))
.unwrap_or(looked_up_ty)
}
Symbol::Unbound => match inferred {
Symbol::Type(ty, Boundness::PossiblyUnbound) => {
report_possibly_unresolved_reference(&self.context, name);
ty
}
Symbol::Unbound => {
report_unresolved_reference(&self.context, name);
Type::unknown()
}
Symbol::Type(_, Boundness::Bound) => unreachable!("Handled above"),
},
LookupError::PossiblyUnbound(type_when_bound) => {
report_possibly_unresolved_reference(&self.context, name_node);
type_when_bound
}
}
})
}
fn infer_name_expression(&mut self, name: &ast::ExprName) -> Type<'db> {
@@ -3454,36 +3451,37 @@ impl<'db> TypeInferenceBuilder<'db> {
ctx: _,
} = attribute;
let value_ty = self.infer_expression(value);
match value_ty.member(self.db(), &attr.id) {
Symbol::Type(member_ty, boundness) => {
if boundness == Boundness::PossiblyUnbound {
let value_type = self.infer_expression(value);
let db = self.db();
value_type
.member(db, &attr.id)
.unwrap_with_diagnostic(|lookup_error| match lookup_error {
LookupError::Unbound => {
self.context.report_lint(
&UNRESOLVED_ATTRIBUTE,
attribute.into(),
format_args!(
"Type `{}` has no attribute `{}`",
value_type.display(db),
attr.id
),
);
Type::unknown()
}
LookupError::PossiblyUnbound(type_when_bound) => {
self.context.report_lint(
&POSSIBLY_UNBOUND_ATTRIBUTE,
attribute.into(),
format_args!(
"Attribute `{}` on type `{}` is possibly unbound",
attr.id,
value_ty.display(self.db()),
value_type.display(db),
),
);
type_when_bound
}
member_ty
}
Symbol::Unbound => {
self.context.report_lint(
&UNRESOLVED_ATTRIBUTE,
attribute.into(),
format_args!(
"Type `{}` has no attribute `{}`",
value_ty.display(self.db()),
attr.id
),
);
Type::unknown()
}
}
})
}
fn infer_attribute_expression(&mut self, attribute: &ast::ExprAttribute) -> Type<'db> {
@@ -3841,6 +3839,8 @@ impl<'db> TypeInferenceBuilder<'db> {
if left_ty != right_ty && right_ty.is_subtype_of(self.db(), left_ty) {
let reflected_dunder = op.reflected_dunder();
let rhs_reflected = right_class.member(self.db(), reflected_dunder);
// TODO: if `rhs_reflected` is possibly unbound, we should union the two possible
// CallOutcomes together
if !rhs_reflected.is_unbound()
&& rhs_reflected != left_class.member(self.db(), reflected_dunder)
{
@@ -4862,7 +4862,7 @@ impl<'db> TypeInferenceBuilder<'db> {
bytes.into(),
format_args!("Type expressions cannot use bytes literal"),
);
Type::unknown().into()
TypeAndQualifiers::unknown()
}
ast::Expr::FString(fstring) => {
@@ -4872,7 +4872,7 @@ impl<'db> TypeInferenceBuilder<'db> {
format_args!("Type expressions cannot use f-strings"),
);
self.infer_fstring_expression(fstring);
Type::unknown().into()
TypeAndQualifiers::unknown()
}
ast::Expr::Name(name) => match name.ctx {
@@ -4893,7 +4893,7 @@ impl<'db> TypeInferenceBuilder<'db> {
.into(),
}
}
ast::ExprContext::Invalid => Type::unknown().into(),
ast::ExprContext::Invalid => TypeAndQualifiers::unknown(),
ast::ExprContext::Store | ast::ExprContext::Del => todo_type!().into(),
},
@@ -4931,7 +4931,7 @@ impl<'db> TypeInferenceBuilder<'db> {
inner_annotation_ty
} else {
self.infer_type_expression(slice);
Type::unknown().into()
TypeAndQualifiers::unknown()
}
} else {
report_invalid_arguments_to_annotated(
@@ -5000,7 +5000,7 @@ impl<'db> TypeInferenceBuilder<'db> {
DeferredExpressionState::InStringAnnotation,
)
}
None => Type::unknown().into(),
None => TypeAndQualifiers::unknown(),
}
}
}
@@ -6093,7 +6093,7 @@ mod tests {
let mut db = setup_db();
let content = format!(
r#"
from typing_extensions import assert_type
from typing_extensions import Literal, assert_type
assert_type(not "{y}", bool)
assert_type(not 10*"{y}", bool)
@@ -6115,7 +6115,7 @@ mod tests {
let mut db = setup_db();
let content = format!(
r#"
from typing_extensions import assert_type
from typing_extensions import Literal, LiteralString, assert_type
assert_type(2 * "hello", Literal["hellohello"])
assert_type("goodbye" * 3, Literal["goodbyegoodbyegoodbye"])
@@ -6140,7 +6140,7 @@ mod tests {
let mut db = setup_db();
let content = format!(
r#"
from typing_extensions import assert_type
from typing_extensions import Literal, LiteralString, assert_type
assert_type("{y}", LiteralString)
assert_type(10*"{y}", LiteralString)
@@ -6162,7 +6162,7 @@ mod tests {
let mut db = setup_db();
let content = format!(
r#"
from typing_extensions import assert_type
from typing_extensions import LiteralString, assert_type
assert_type("{y}", LiteralString)
assert_type("a" + "{z}", LiteralString)
@@ -6182,7 +6182,7 @@ mod tests {
let mut db = setup_db();
let content = format!(
r#"
from typing_extensions import assert_type
from typing_extensions import LiteralString, assert_type
assert_type("{y}", LiteralString)
assert_type("{y}" + "a", LiteralString)
@@ -6441,6 +6441,7 @@ mod tests {
assert_eq!(attr_ty.display(&db).to_string(), "Unknown | str | None");
db.take_salsa_events()
};
assert_function_query_was_not_run(
&db,
infer_expression_types,

View File

@@ -4,13 +4,13 @@ use std::ops::Deref;
use rustc_hash::FxHashSet;
use crate::types::class_base::ClassBase;
use crate::types::{Class, KnownClass, Type};
use crate::types::{Class, Type};
use crate::Db;
/// The inferred method resolution order of a given class.
///
/// See [`Class::iter_mro`] for more details.
#[derive(PartialEq, Eq, Clone, Debug)]
#[derive(PartialEq, Eq, Clone, Debug, salsa::Update)]
pub(super) struct Mro<'db>(Box<[ClassBase<'db>]>);
impl<'db> Mro<'db> {
@@ -52,9 +52,7 @@ impl<'db> Mro<'db> {
match class_bases {
// `builtins.object` is the special case:
// the only class in Python that has an MRO with length <2
[] if class.is_known(db, KnownClass::Object) => {
Ok(Self::from([ClassBase::Class(class)]))
}
[] if class.is_object(db) => Ok(Self::from([ClassBase::Class(class)])),
// All other classes in Python have an MRO with length >=2.
// Even if a class has no explicit base classes,
@@ -238,7 +236,7 @@ impl<'db> Iterator for MroIterator<'db> {
impl std::iter::FusedIterator for MroIterator<'_> {}
#[derive(Debug, PartialEq, Eq)]
#[derive(Debug, PartialEq, Eq, salsa::Update)]
pub(super) struct MroError<'db> {
kind: MroErrorKind<'db>,
fallback_mro: Mro<'db>,
@@ -258,7 +256,7 @@ impl<'db> MroError<'db> {
}
/// Possible ways in which attempting to resolve the MRO of a class might fail.
#[derive(Debug, PartialEq, Eq)]
#[derive(Debug, PartialEq, Eq, salsa::Update)]
pub(super) enum MroErrorKind<'db> {
/// The class inherits from one or more invalid bases.
///

View File

@@ -152,13 +152,13 @@ fn merge_constraints_or<'db>(
*entry.get_mut() = UnionBuilder::new(db).add(*entry.get()).add(*value).build();
}
Entry::Vacant(entry) => {
entry.insert(KnownClass::Object.to_instance(db));
entry.insert(Type::object(db));
}
}
}
for (key, value) in into.iter_mut() {
if !from.contains_key(key) {
*value = KnownClass::Object.to_instance(db);
*value = Type::object(db);
}
}
}
@@ -231,10 +231,10 @@ impl<'db> NarrowingConstraintsBuilder<'db> {
match pattern.kind(self.db) {
PatternConstraintKind::Singleton(singleton, _guard) => {
self.evaluate_match_pattern_singleton(*subject, *singleton)
self.evaluate_match_pattern_singleton(subject, *singleton)
}
PatternConstraintKind::Class(cls, _guard) => {
self.evaluate_match_pattern_class(*subject, *cls)
self.evaluate_match_pattern_class(subject, *cls)
}
// TODO: support more pattern kinds
PatternConstraintKind::Value(..) | PatternConstraintKind::Unsupported => None,

View File

@@ -329,7 +329,7 @@ fn union<'db>(db: &'db TestDb, tys: impl IntoIterator<Item = Type<'db>>) -> Type
mod stable {
use super::union;
use crate::types::{KnownClass, Type};
use crate::types::Type;
// Reflexivity: `T` is equivalent to itself.
type_property_test!(
@@ -419,13 +419,13 @@ mod stable {
// All types should be assignable to `object`
type_property_test!(
all_types_assignable_to_object, db,
forall types t. t.is_assignable_to(db, KnownClass::Object.to_instance(db))
forall types t. t.is_assignable_to(db, Type::object(db))
);
// And for fully static types, they should also be subtypes of `object`
type_property_test!(
all_fully_static_types_subtype_of_object, db,
forall types t. t.is_fully_static(db) => t.is_subtype_of(db, KnownClass::Object.to_instance(db))
forall types t. t.is_fully_static(db) => t.is_subtype_of(db, Type::object(db))
);
// Never should be assignable to every type

View File

@@ -4,7 +4,7 @@ use crate::{semantic_index::definition::Definition, types::todo_type};
use ruff_python_ast::{self as ast, name::Name};
/// A typed callable signature.
#[derive(Clone, Debug, PartialEq, Eq)]
#[derive(Clone, Debug, PartialEq, Eq, salsa::Update)]
pub(crate) struct Signature<'db> {
/// Parameters, in source order.
///
@@ -60,7 +60,7 @@ impl<'db> Signature<'db> {
}
// TODO: use SmallVec here once invariance bug is fixed
#[derive(Clone, Debug, PartialEq, Eq)]
#[derive(Clone, Debug, PartialEq, Eq, salsa::Update)]
pub(crate) struct Parameters<'db>(Vec<Parameter<'db>>);
impl<'db> Parameters<'db> {
@@ -218,7 +218,7 @@ impl<'db> std::ops::Index<usize> for Parameters<'db> {
}
}
#[derive(Clone, Debug, PartialEq, Eq)]
#[derive(Clone, Debug, PartialEq, Eq, salsa::Update)]
pub(crate) struct Parameter<'db> {
/// Parameter name.
///
@@ -304,7 +304,7 @@ impl<'db> Parameter<'db> {
}
}
#[derive(Clone, Debug, PartialEq, Eq)]
#[derive(Clone, Debug, PartialEq, Eq, salsa::Update)]
pub(crate) enum ParameterKind<'db> {
/// Positional-only parameter, e.g. `def f(x, /): ...`
PositionalOnly { default_ty: Option<Type<'db>> },
@@ -357,6 +357,8 @@ mod tests {
db.write_dedented(
"/src/a.py",
"
from typing import Literal
def f(a, b: int, c = 1, d: int = 2, /,
e = 3, f: Literal[4] = 4, *args: object,
g = 5, h: Literal[6] = 6, **kwargs: str) -> bytes: ...
@@ -411,7 +413,7 @@ mod tests {
},
Parameter {
name: Some(Name::new_static("args")),
annotated_ty: Some(KnownClass::Object.to_instance(&db)),
annotated_ty: Some(Type::object(&db)),
kind: ParameterKind::Variadic,
},
Parameter {

View File

@@ -1,121 +0,0 @@
use crate::types::{infer_scope_types, semantic_index, Type};
use crate::Db;
use ruff_db::files::File;
use rustc_hash::FxHashMap;
/// Get type-coverage statistics for a file.
#[salsa::tracked(return_ref)]
pub fn type_statistics<'db>(db: &'db dyn Db, file: File) -> TypeStatistics<'db> {
let _span = tracing::trace_span!("type_statistics", file=?file.path(db)).entered();
tracing::debug!(
"Gathering statistics for file '{path}'",
path = file.path(db)
);
let index = semantic_index(db, file);
let mut statistics = TypeStatistics::default();
for scope_id in index.scope_ids() {
let result = infer_scope_types(db, scope_id);
statistics.extend(&result.statistics());
}
statistics
}
/// Map each type to count of expressions with that type.
#[derive(Debug, Default, Eq, PartialEq)]
pub(super) struct TypeStatistics<'db>(FxHashMap<Type<'db>, u32>);
impl<'db> TypeStatistics<'db> {
fn extend(&mut self, other: &TypeStatistics<'db>) {
for (ty, count) in &other.0 {
self.0
.entry(*ty)
.and_modify(|my_count| *my_count += count)
.or_insert(*count);
}
}
pub(super) fn increment(&mut self, ty: Type<'db>) {
self.0
.entry(ty)
.and_modify(|count| *count += 1)
.or_insert(1);
}
#[allow(unused)]
fn expression_count(&self) -> u32 {
self.0.values().sum()
}
#[allow(unused)]
fn todo_count(&self) -> u32 {
self.0
.iter()
.filter(|(key, _)| key.is_todo())
.map(|(_, count)| count)
.sum()
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::db::tests::{setup_db, TestDb};
use ruff_db::files::system_path_to_file;
use ruff_db::system::DbWithTestSystem;
fn get_stats<'db>(
db: &'db mut TestDb,
filename: &str,
source: &str,
) -> &'db TypeStatistics<'db> {
db.write_dedented(filename, source).unwrap();
type_statistics(db, system_path_to_file(db, filename).unwrap())
}
#[test]
fn all_static() {
let mut db = setup_db();
let stats = get_stats(&mut db, "src/foo.py", "1");
assert_eq!(stats.0, FxHashMap::from_iter([(Type::IntLiteral(1), 1)]));
}
#[test]
fn todo_and_expression_count() {
let mut db = setup_db();
let stats = get_stats(
&mut db,
"src/foo.py",
r#"
x = [x for x in [1]]
"#,
);
assert_eq!(stats.todo_count(), 4);
assert_eq!(stats.expression_count(), 6);
}
#[test]
fn sum() {
let mut db = setup_db();
let stats = get_stats(
&mut db,
"src/foo.py",
r#"
1
def f():
1
"#,
);
assert_eq!(stats.0[&Type::IntLiteral(1)], 2);
}
}

View File

@@ -26,7 +26,7 @@ impl<'db> SubclassOfType<'db> {
ClassBase::Class(class) => {
if class.is_final(db) {
Type::ClassLiteral(ClassLiteralType { class })
} else if class.is_known(db, KnownClass::Object) {
} else if class.is_object(db) {
KnownClass::Type.to_instance(db)
} else {
Type::SubclassOf(Self { subclass_of })

View File

@@ -261,7 +261,7 @@ impl<'db> Unpacker<'db> {
}
}
#[derive(Debug, Default, PartialEq, Eq)]
#[derive(Debug, Default, PartialEq, Eq, salsa::Update)]
pub(crate) struct UnpackResult<'db> {
targets: FxHashMap<ScopedExpressionId, Type<'db>>,
diagnostics: TypeCheckDiagnostics,

View File

@@ -28,16 +28,15 @@ use crate::Db;
/// * an argument of a cross-module query
#[salsa::tracked]
pub(crate) struct Unpack<'db> {
#[id]
pub(crate) file: File,
#[id]
pub(crate) file_scope: FileScopeId,
/// The target expression that is being unpacked. For example, in `(a, b) = (1, 2)`, the target
/// expression is `(a, b)`.
#[no_eq]
#[return_ref]
#[tracked]
pub(crate) target: AstNodeRef<ast::Expr>,
/// The ingredient representing the value expression of the unpacking. For example, in
@@ -45,7 +44,6 @@ pub(crate) struct Unpack<'db> {
#[no_eq]
pub(crate) value: UnpackValue<'db>,
#[no_eq]
count: countme::Count<Unpack<'static>>,
}
@@ -62,7 +60,7 @@ impl<'db> Unpack<'db> {
}
/// The expression that is being unpacked.
#[derive(Clone, Copy, Debug)]
#[derive(Clone, Copy, Debug, Hash)]
pub(crate) enum UnpackValue<'db> {
/// An iterable expression like the one in a `for` loop or a comprehension.
Iterable(Expression<'db>),

View File

@@ -338,7 +338,7 @@ const SMALLEST_TERMINAL: ScopedVisibilityConstraintId = ALWAYS_FALSE;
/// A collection of visibility constraints. This is currently stored in `UseDefMap`, which means we
/// maintain a separate set of visibility constraints for each scope in file.
#[derive(Debug, PartialEq, Eq)]
#[derive(Debug, PartialEq, Eq, salsa::Update)]
pub(crate) struct VisibilityConstraints<'db> {
constraints: IndexVec<Atom, Constraint<'db>>,
interiors: IndexVec<ScopedVisibilityConstraintId, InteriorNode>,
@@ -627,7 +627,7 @@ impl<'db> VisibilityConstraints<'db> {
ConstraintNode::Pattern(inner) => match inner.kind(db) {
PatternConstraintKind::Value(value, guard) => {
let subject_expression = inner.subject(db);
let inference = infer_expression_types(db, *subject_expression);
let inference = infer_expression_types(db, subject_expression);
let scope = subject_expression.scope(db);
let subject_ty = inference.expression_type(
subject_expression

View File

@@ -12,7 +12,7 @@ license = { workspace = true }
[dependencies]
red_knot_project = { workspace = true }
ruff_db = { workspace = true }
ruff_db = { workspace = true, features = ["os"] }
ruff_notebook = { workspace = true }
ruff_python_ast = { workspace = true }
ruff_source_file = { workspace = true }

View File

@@ -75,11 +75,13 @@ fn to_lsp_diagnostic(
diagnostic: &dyn ruff_db::diagnostic::Diagnostic,
encoding: crate::PositionEncoding,
) -> Diagnostic {
let range = if let (Some(file), Some(range)) = (diagnostic.file(), diagnostic.range()) {
let index = line_index(db.upcast(), file);
let source = source_text(db.upcast(), file);
let range = if let Some(span) = diagnostic.span() {
let index = line_index(db.upcast(), span.file());
let source = source_text(db.upcast(), span.file());
range.to_range(&source, &index, encoding)
span.range()
.map(|range| range.to_range(&source, &index, encoding))
.unwrap_or_default()
} else {
Range::default()
};

View File

@@ -68,7 +68,9 @@ impl Session {
let system = LSPSystem::new(index.clone());
// TODO(dhruvmanila): Get the values from the client settings
let metadata = ProjectMetadata::discover(system_path, &system)?;
let mut metadata = ProjectMetadata::discover(system_path, &system)?;
metadata.apply_configuration_files(&system)?;
// TODO(micha): Handle the case where the program settings are incorrect more gracefully.
workspaces.insert(path, ProjectDatabase::new(metadata, system)?);
}

View File

@@ -187,6 +187,10 @@ impl System for LSPSystem {
self.os_system.current_directory()
}
fn user_config_directory(&self) -> Option<SystemPathBuf> {
self.os_system.user_config_directory()
}
fn read_directory<'a>(
&'a self,
path: &SystemPath,

View File

@@ -9,7 +9,8 @@
//! ```
use anyhow::Context;
use red_knot_python_semantic::{PythonPlatform, PythonVersion};
use red_knot_python_semantic::PythonPlatform;
use ruff_python_ast::python_version::PythonVersion;
use serde::Deserialize;
#[derive(Deserialize, Debug, Default, Clone)]

View File

@@ -1,12 +1,15 @@
use std::sync::Arc;
use red_knot_python_semantic::lint::{LintRegistry, RuleSelection};
use red_knot_python_semantic::{
default_lint_registry, Db as SemanticDb, Program, ProgramSettings, PythonPlatform,
PythonVersion, SearchPathSettings,
SearchPathSettings,
};
use ruff_db::files::{File, Files};
use ruff_db::system::{DbWithTestSystem, System, SystemPath, SystemPathBuf, TestSystem};
use ruff_db::vendored::VendoredFileSystem;
use ruff_db::{Db as SourceDb, Upcast};
use ruff_python_ast::python_version::PythonVersion;
#[salsa::db]
#[derive(Clone)]
@@ -16,7 +19,7 @@ pub(crate) struct Db {
files: Files,
system: TestSystem,
vendored: VendoredFileSystem,
rule_selection: RuleSelection,
rule_selection: Arc<RuleSelection>,
}
impl Db {
@@ -29,7 +32,7 @@ impl Db {
system: TestSystem::default(),
vendored: red_knot_vendored::file_system().clone(),
files: Files::default(),
rule_selection,
rule_selection: Arc::new(rule_selection),
};
db.memory_file_system()
@@ -94,8 +97,8 @@ impl SemanticDb for Db {
!file.path(self).is_vendored_path()
}
fn rule_selection(&self) -> &RuleSelection {
&self.rule_selection
fn rule_selection(&self) -> Arc<RuleSelection> {
self.rule_selection.clone()
}
fn lint_registry(&self) -> &LintRegistry {

View File

@@ -26,7 +26,8 @@ where
.into_iter()
.map(|diagnostic| DiagnosticWithLine {
line_number: diagnostic
.range()
.span()
.and_then(|span| span.range())
.map_or(OneIndexed::from_zero_indexed(0), |range| {
line_index.line_index(range.start())
}),
@@ -144,7 +145,7 @@ struct DiagnosticWithLine<T> {
mod tests {
use crate::db::Db;
use crate::diagnostic::Diagnostic;
use ruff_db::diagnostic::{DiagnosticId, LintName, Severity};
use ruff_db::diagnostic::{DiagnosticId, LintName, Severity, Span};
use ruff_db::files::{system_path_to_file, File};
use ruff_db::source::line_index;
use ruff_db::system::{DbWithTestSystem, SystemPathBuf};
@@ -198,12 +199,8 @@ mod tests {
"dummy".into()
}
fn file(&self) -> Option<File> {
Some(self.file)
}
fn range(&self) -> Option<TextRange> {
Some(self.range)
fn span(&self) -> Option<Span> {
Some(Span::from(self.file).with_range(self.range))
}
fn severity(&self) -> Severity {

View File

@@ -5,7 +5,7 @@ use colored::Colorize;
use parser as test_parser;
use red_knot_python_semantic::types::check_types;
use red_knot_python_semantic::{Program, ProgramSettings, SearchPathSettings, SitePackages};
use ruff_db::diagnostic::{Diagnostic, ParseDiagnostic};
use ruff_db::diagnostic::{Diagnostic, DisplayDiagnosticConfig, ParseDiagnostic};
use ruff_db::files::{system_path_to_file, File, Files};
use ruff_db::panic::catch_unwind;
use ruff_db::parsed::parsed_module;
@@ -300,9 +300,7 @@ fn create_diagnostic_snapshot<D: Diagnostic>(
test: &parser::MarkdownTest,
diagnostics: impl IntoIterator<Item = D>,
) -> String {
// TODO(ag): Do something better than requiring this
// global state to be twiddled everywhere.
colored::control::set_override(false);
let display_config = DisplayDiagnosticConfig::default().color(false);
let mut snapshot = String::new();
writeln!(snapshot).unwrap();
@@ -340,7 +338,7 @@ fn create_diagnostic_snapshot<D: Diagnostic>(
writeln!(snapshot).unwrap();
}
writeln!(snapshot, "```").unwrap();
writeln!(snapshot, "{}", diag.display(db)).unwrap();
writeln!(snapshot, "{}", diag.display(db, &display_config)).unwrap();
writeln!(snapshot, "```").unwrap();
}
snapshot

View File

@@ -257,7 +257,8 @@ impl Matcher {
fn column<T: Diagnostic>(&self, diagnostic: &T) -> OneIndexed {
diagnostic
.range()
.span()
.and_then(|span| span.range())
.map(|range| {
self.line_index
.source_location(range.start(), &self.source)
@@ -334,7 +335,7 @@ impl Matcher {
#[cfg(test)]
mod tests {
use super::FailuresByLine;
use ruff_db::diagnostic::{Diagnostic, DiagnosticId, Severity};
use ruff_db::diagnostic::{Diagnostic, DiagnosticId, Severity, Span};
use ruff_db::files::{system_path_to_file, File};
use ruff_db::system::{DbWithTestSystem, SystemPathBuf};
use ruff_python_trivia::textwrap::dedent;
@@ -385,12 +386,8 @@ mod tests {
self.message.into()
}
fn file(&self) -> Option<File> {
Some(self.file)
}
fn range(&self) -> Option<TextRange> {
Some(self.range)
fn span(&self) -> Option<Span> {
Some(Span::from(self.file).with_range(self.range))
}
fn severity(&self) -> Severity {

View File

@@ -1 +1 @@
c193cd2a36839c8e6336f350397f51ce52fedd5e
cc8ca939c0477a49fcce0554fa1743bd5c656a11

View File

@@ -78,7 +78,7 @@ if sys.platform == "win32":
SO_EXCLUSIVEADDRUSE: int
if sys.platform != "win32":
SO_REUSEPORT: int
if sys.platform != "darwin" or sys.version_info >= (3, 13):
if sys.platform != "darwin":
SO_BINDTODEVICE: int
if sys.platform != "win32" and sys.platform != "darwin":

View File

@@ -2,7 +2,7 @@ import sys
from _typeshed import SupportsWrite, sentinel
from collections.abc import Callable, Generator, Iterable, Sequence
from re import Pattern
from typing import IO, Any, ClassVar, Final, Generic, NewType, NoReturn, Protocol, TypeVar, overload
from typing import IO, Any, ClassVar, Final, Generic, NoReturn, Protocol, TypeVar, overload
from typing_extensions import Self, TypeAlias, deprecated
__all__ = [
@@ -33,25 +33,14 @@ _ActionT = TypeVar("_ActionT", bound=Action)
_ArgumentParserT = TypeVar("_ArgumentParserT", bound=ArgumentParser)
_N = TypeVar("_N")
_ActionType: TypeAlias = Callable[[str], Any] | FileType | str
# more precisely, Literal["store", "store_const", "store_true",
# "store_false", "append", "append_const", "count", "help", "version",
# "extend"], but using this would make it hard to annotate callers
# that don't use a literal argument
_ActionStr: TypeAlias = str
# more precisely, Literal["?", "*", "+", "...", "A...",
# "==SUPPRESS=="], but using this would make it hard to annotate
# callers that don't use a literal argument
_NArgsStr: TypeAlias = str
ONE_OR_MORE: Final = "+"
OPTIONAL: Final = "?"
PARSER: Final = "A..."
REMAINDER: Final = "..."
_SUPPRESS_T = NewType("_SUPPRESS_T", str)
SUPPRESS: _SUPPRESS_T | str # not using Literal because argparse sometimes compares SUPPRESS with is
# the | str is there so that foo = argparse.SUPPRESS; foo = "test" checks out in mypy
SUPPRESS: Final = "==SUPPRESS=="
ZERO_OR_MORE: Final = "*"
_UNRECOGNIZED_ARGS_ATTR: Final[str] # undocumented
_UNRECOGNIZED_ARGS_ATTR: Final = "_unrecognized_args" # undocumented
class ArgumentError(Exception):
argument_name: str | None
@@ -86,8 +75,13 @@ class _ActionsContainer:
def add_argument(
self,
*name_or_flags: str,
action: _ActionStr | type[Action] = ...,
nargs: int | _NArgsStr | _SUPPRESS_T | None = None,
# str covers predefined actions ("store_true", "count", etc.)
# and user registered actions via the `register` method.
action: str | type[Action] = ...,
# more precisely, Literal["?", "*", "+", "...", "A...", "==SUPPRESS=="],
# but using this would make it hard to annotate callers that don't use a
# literal argument and for subclasses to override this method.
nargs: int | str | None = None,
const: Any = ...,
default: Any = ...,
type: _ActionType = ...,

View File

@@ -79,6 +79,7 @@ if sys.version_info >= (3, 12):
_FutureLike: TypeAlias = Future[_T] | Awaitable[_T]
else:
_FutureLike: TypeAlias = Future[_T] | Generator[Any, None, _T] | Awaitable[_T]
_TaskYieldType: TypeAlias = Future[object] | None
FIRST_COMPLETED = concurrent.futures.FIRST_COMPLETED
@@ -347,7 +348,8 @@ else:
*coros_or_futures: _FutureLike[_T], loop: AbstractEventLoop | None = None, return_exceptions: bool
) -> Future[list[_T | BaseException]]: ...
def run_coroutine_threadsafe(coro: _FutureLike[_T], loop: AbstractEventLoop) -> concurrent.futures.Future[_T]: ...
# unlike some asyncio apis, This does strict runtime checking of actually being a coroutine, not of any future-like.
def run_coroutine_threadsafe(coro: Coroutine[Any, Any, _T], loop: AbstractEventLoop) -> concurrent.futures.Future[_T]: ...
if sys.version_info >= (3, 10):
def shield(arg: _FutureLike[_T]) -> Future[_T]: ...

View File

@@ -1,6 +1,7 @@
import sys
from _typeshed import ExcInfo, TraceFunction, Unused
from collections.abc import Callable, Iterable, Mapping
from collections.abc import Callable, Iterable, Iterator, Mapping
from contextlib import contextmanager
from types import CodeType, FrameType, TracebackType
from typing import IO, Any, Final, SupportsInt, TypeVar
from typing_extensions import ParamSpec
@@ -30,6 +31,10 @@ class Bdb:
def __init__(self, skip: Iterable[str] | None = None) -> None: ...
def canonic(self, filename: str) -> str: ...
def reset(self) -> None: ...
if sys.version_info >= (3, 12):
@contextmanager
def set_enterframe(self, frame: FrameType) -> Iterator[None]: ...
def trace_dispatch(self, frame: FrameType, event: str, arg: Any) -> TraceFunction: ...
def dispatch_line(self, frame: FrameType) -> TraceFunction: ...
def dispatch_call(self, frame: FrameType, arg: None) -> TraceFunction: ...
@@ -73,7 +78,7 @@ class Bdb:
def get_file_breaks(self, filename: str) -> list[Breakpoint]: ...
def get_all_breaks(self) -> list[Breakpoint]: ...
def get_stack(self, f: FrameType | None, t: TracebackType | None) -> tuple[list[tuple[FrameType, int]], int]: ...
def format_stack_entry(self, frame_lineno: int, lprefix: str = ": ") -> str: ...
def format_stack_entry(self, frame_lineno: tuple[FrameType, int], lprefix: str = ": ") -> str: ...
def run(
self, cmd: str | CodeType, globals: dict[str, Any] | None = None, locals: Mapping[str, Any] | None = None
) -> None: ...

View File

@@ -1295,7 +1295,7 @@ def ascii(obj: object, /) -> str: ...
def bin(number: int | SupportsIndex, /) -> str: ...
def breakpoint(*args: Any, **kws: Any) -> None: ...
def callable(obj: object, /) -> TypeIs[Callable[..., object]]: ...
def chr(i: int, /) -> str: ...
def chr(i: int | SupportsIndex, /) -> str: ...
# We define this here instead of using os.PathLike to avoid import cycle issues.
# See https://github.com/python/typeshed/pull/991#issuecomment-288160993

View File

@@ -32,9 +32,9 @@ _T = TypeVar("_T")
_T_co = TypeVar("_T_co", covariant=True)
_T_io = TypeVar("_T_io", bound=IO[str] | None)
_ExitT_co = TypeVar("_ExitT_co", covariant=True, bound=bool | None, default=bool | None)
_F = TypeVar("_F", bound=Callable[..., Any])
_G = TypeVar("_G", bound=Generator[Any, Any, Any] | AsyncGenerator[Any, Any], covariant=True)
_P = ParamSpec("_P")
_R = TypeVar("_R")
_SendT_contra = TypeVar("_SendT_contra", contravariant=True, default=None)
_ReturnT_co = TypeVar("_ReturnT_co", covariant=True, default=None)
@@ -64,13 +64,9 @@ class AbstractAsyncContextManager(ABC, Protocol[_T_co, _ExitT_co]): # type: ign
self, exc_type: type[BaseException] | None, exc_value: BaseException | None, traceback: TracebackType | None, /
) -> _ExitT_co: ...
class _WrappedCallable(Generic[_P, _R]):
__wrapped__: Callable[_P, _R]
def __call__(self, *args: _P.args, **kwargs: _P.kwargs) -> _R: ...
class ContextDecorator:
def _recreate_cm(self) -> Self: ...
def __call__(self, func: Callable[_P, _R]) -> _WrappedCallable[_P, _R]: ...
def __call__(self, func: _F) -> _F: ...
class _GeneratorContextManagerBase(Generic[_G]):
# Ideally this would use ParamSpec, but that requires (*args, **kwargs), which this isn't. see #6676
@@ -97,11 +93,11 @@ class _GeneratorContextManager(
def contextmanager(func: Callable[_P, Iterator[_T_co]]) -> Callable[_P, _GeneratorContextManager[_T_co]]: ...
if sys.version_info >= (3, 10):
_AR = TypeVar("_AR", bound=Awaitable[Any])
_AF = TypeVar("_AF", bound=Callable[..., Awaitable[Any]])
class AsyncContextDecorator:
def _recreate_cm(self) -> Self: ...
def __call__(self, func: Callable[_P, _AR]) -> _WrappedCallable[_P, _AR]: ...
def __call__(self, func: _AF) -> _AF: ...
class _AsyncGeneratorContextManager(
_GeneratorContextManagerBase[AsyncGenerator[_T_co, _SendT_contra]],

Some files were not shown because too many files have changed in this diff Show More