Compare commits

..

95 Commits

Author SHA1 Message Date
David Peter
500b9a2691 EllipsisType test 2024-11-29 22:15:37 +01:00
David Peter
15476be531 Add TODO 2024-11-29 21:52:17 +01:00
David Peter
e7a361699d Fix behavior for NoDefault 2024-11-29 21:49:24 +01:00
David Peter
a218e1901b Declarations 2024-11-29 21:16:34 +01:00
David Peter
f108043d2d Reset typeshed 2024-11-28 22:51:10 +01:00
David Peter
77b45aeee9 Remove hard-coded EllipsisType 2024-11-28 22:49:25 +01:00
David Peter
c6f4c106b0 Updates 2024-11-28 22:47:18 +01:00
David Peter
dc55b4c8a2 Properly restore constraints 2024-11-28 22:38:08 +01:00
David Peter
41d19c3c29 Update 2024-11-28 20:00:58 +01:00
David Peter
99d44299e8 Revert typing change 2024-11-28 19:58:26 +01:00
David Peter
32ad489d79 More debugging output 2024-11-28 19:42:24 +01:00
David Peter
a3e7e7d8b6 Remove TODOs 2024-11-28 19:39:58 +01:00
David Peter
aea4bbbb30 Adapt debug test 2024-11-28 19:39:33 +01:00
David Peter
167e445243 Reset test file 2024-11-28 19:39:16 +01:00
David Peter
dccfd6e4f8 New tests 2024-11-28 19:38:56 +01:00
David Peter
eb4ae2b910 statically known to be True branches 2024-11-28 17:33:35 +01:00
David Peter
5be842b1c3 Patch typeshed and increase default Python version to 3.13 2024-11-28 17:33:35 +01:00
David Peter
2a21d79ec4 Cleanup 2024-11-28 17:33:35 +01:00
David Peter
1964ecdbb7 Base check on type, not expr 2024-11-28 17:33:35 +01:00
David Peter
6d167672f1 First version based on end-of-branch constraints 2024-11-28 17:33:35 +01:00
David Peter
ae45b897ea Restore to main 2024-11-28 17:33:35 +01:00
David Peter
90f48f45b0 Dump save 2024-11-28 17:33:35 +01:00
David Salvisberg
d9cbf2fe44 Avoids unnecessary overhead for TC004, when TC001-003 are disabled (#14657) 2024-11-28 16:28:24 +01:00
Samodya Abeysiriwardane
3f6c65e78c [red-knot] Fix merged type after if-else without explicit else branch (#14621)
## Summary

Closes: https://github.com/astral-sh/ruff/issues/14593

The final type of a variable after if-statement without explicit else
branch should be similar to having an explicit else branch.

## Test Plan

Originally failed test cases from the bug are added.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-11-28 06:23:55 -08:00
Dhruv Manilawala
976c37a849 Bump version to 0.8.1 (#14655) 2024-11-28 19:12:50 +05:30
David Peter
a378ff38dc [red-knot] Fix Boolean flags in mdtests (#14654)
## Summary

Similar to #14652, but now with conditions that are `Literal[True]`
(instead of `Literal[False]`), where we want them to be `bool`.
2024-11-28 14:29:35 +01:00
Alex Waygood
d8bca0d3a2 Fix bug where methods defined using lambdas were flagged by FURB118 (#14639) 2024-11-28 12:58:23 +00:00
David Peter
6f1cf5b686 [red-knot] Minor fix in MRO tests (#14652)
## Summary

`bool()` is equal to `False`, and we infer `Literal[False]` for it. Which
means that the test here will fail as soon as we treat the body of
this `if` as unreachable.
2024-11-28 10:17:15 +01:00
David Peter
8639f8c1a6 CI: Treat mdtest Markdown files as code (#14653)
## Summary

Make sure we run the tests for mdtest-only changes.

## Test Plan

Tested if positive glob patterns override negative patterns here:
https://codepen.io/mrmlnc/pen/OXQjMe
2024-11-28 10:04:20 +01:00
Alex Waygood
f1b2e85339 py-fuzzer: recommend using uvx rather than uv run to run the fuzzer (#14645) 2024-11-27 22:19:52 +00:00
David Salvisberg
6d61c8aa16 Fixes minor bug in SemanticModel::lookup_symbol (#14643)
## Summary

This came up as part of #12927 when implementing
`SemanticModel::simulate_runtime_load`.

Should be fairly self-explanatory, if the scope returns a binding with
`BindingKind::Annotation` the bottom part of the loop gets skipped, so
there's no chance for `seen_function` to have been updated. So unless
there's something subtle going on here, like function scopes never
containing bindings with `BindingKind::Annotation`, this seems like a
bug.

## Test Plan

`cargo nextest run`
2024-11-27 16:50:19 -05:00
David Salvisberg
8a7ba5d2df [flake8-type-checking] Fixes quote_type_expression (#14634) 2024-11-27 18:58:48 +01:00
Brent Westbrook
6fcbe8efb4 [ruff] Detect redirected-noqa in file-level comments (RUF101) (#14635) 2024-11-27 18:25:47 +01:00
Alexandra Valentine-Ketchum
c40b37aa36 N811 & N814: eliminate false positives for single-letter names (#14584)
Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2024-11-27 14:38:36 +00:00
Alex Waygood
ef0e2a6e1b Refactor crates/ruff_python_stdlib/src/builtins.rs to make it easier to add support for new Python versions (#14632) 2024-11-27 12:20:21 +00:00
Alex Waygood
4fb1416bf4 Minor stylistic improvements for functions detecting PEP-604 unions (#14633) 2024-11-27 11:29:37 +00:00
Simon Brugman
8a860b89b4 Add social icons to the footer (#14591)
## Summary

Add social icons to the footer

`mkdocs-material` update is required for the `x-twitter` icon.

## Test Plan

Tested locally. 

---------

Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
2024-11-27 11:07:45 +00:00
Dhruv Manilawala
f96fa6b0e2 Do not consider f-strings with escaped newlines as multiline (#14624)
## Summary

This PR fixes a bug in the f-string formatting to not consider the
escaped newlines for `is_multiline`. This is done by checking if the
f-string is triple-quoted or not similar to normal string literals.

This is not required to be gated behind preview because the logic change
for `is_multiline` was added in
https://github.com/astral-sh/ruff/pull/14454.

## Test Plan

Add a test case which formats differently on `main`:
https://play.ruff.rs/ea3c55c2-f0fe-474e-b6b8-e3365e0ede5e
2024-11-27 10:25:38 +00:00
Dhruv Manilawala
4cd2b9926e Gate is_multiline change behind preview (#14630)
## Summary

Ref:
https://github.com/astral-sh/ruff/pull/14624#pullrequestreview-2464127254

## Test Plan

The test case in the follow-up PR showcases the difference between
preview and non-preview formatting:
https://github.com/astral-sh/ruff/pull/14624/files#diff-dc25bd4df280d9a9180598075b5bc2d0bac30af956767b373561029309c8f024
2024-11-27 15:50:28 +05:30
Simon Brugman
11a2929ed7 [ruff] Implement unnecessary-nested-literal (RUF041) (#14323)
Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-11-27 10:01:50 +00:00
InSync
187974eff4 [flake8-use-pathlib] Recommend Path.iterdir() over os.listdir() (PTH208) (#14509)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-11-27 09:53:13 +00:00
Micha Reiser
14ba469fc0 Use a derive macro for Violations (#14557)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-11-27 09:41:40 +00:00
David Salvisberg
6fd10e2fe7 [flake8-type-checking] Adds implementation for TC007 and TC008 (#12927)
Co-authored-by: Simon Brugman <sbrugman@users.noreply.github.com>
Co-authored-by: Carl Meyer <carl@oddbird.net>
2024-11-27 09:51:20 +01:00
Alex Waygood
e0f3eaf1dd Turn the fuzz-parser script into a properly packaged Python project (#14606)
## Summary

This PR gets rid of the `requirements.in` and `requirements.txt` files
in the `scripts/fuzz-parser` directory, and replaces them with
`pyproject.toml` and `uv.lock` files. The script is renamed from
`fuzz-parser` to `py-fuzzer` (since it can now also be used to fuzz
red-knot as well as the parser, following
https://github.com/astral-sh/ruff/pull/14566), and moved from the
`scripts/` directory to the `python/` directory, since it's now a
(uv)-pip-installable project in its own right.

I've been resisting this for a while, because conceptually this script
just doesn't feel "complicated" enough to me for it to be a full-blown
package. However, I think it's time to do this. Making it a proper
package has several advantages:
- It means we can run it from the project root using `uv run` without
having to activate a virtual environment and ensure that all required
dependencies are installed into that environment
- Using a `pyproject.toml` file means that we can express that the
project requires Python 3.12+ to run properly; this wasn't possible
before
- I've been running mypy on the project locally when I've been working
on it or reviewing other people's PRs; now I can put the mypy config for
the project in the `pyproject.toml` file

## Test Plan

I manually tested that all the commands detailed in
`python/py-fuzzer/README.md` work for me locally.

---------

Co-authored-by: David Peter <sharkdp@users.noreply.github.com>
2024-11-27 08:09:04 +00:00
Dhruv Manilawala
c84c690f1e Avoid invalid syntax for format-spec with quotes for all Python versions (#14625)
## Summary

fixes: #14608

The logic that was only applied for 3.12+ target version needs to be
applied for other versions as well.

## Test Plan

I've moved the existing test cases for 3.12 only to `f_string.py` so
that it's tested against the default target version.

I think we should probably enabled testing for two target version (pre
3.12 and 3.12) but it won't highlight any issue because the parser
doesn't consider this. Maybe we should enable this once we have target
version specific syntax errors in place
(https://github.com/astral-sh/ruff/issues/6591).
2024-11-27 13:19:33 +05:30
Dhruv Manilawala
0d649f9afd Check that airflow module is seen for AIR001 (#14627) 2024-11-27 07:25:08 +00:00
Lokejoke
82c01aa662 [pylint] Implement len-test (PLC1802) (#14309)
## Summary

This PR implements [`use-implicit-booleaness-not-len` /
`C1802`](https://pylint.pycqa.org/en/latest/user_guide/messages/convention/use-implicit-booleaness-not-len.html)
> For sequences, (strings, lists, tuples), use the fact that empty
sequences are false.

---------

Co-authored-by: xbrtnik1 <524841@mail.muni.cz>
Co-authored-by: xbrtnik1 <xbrtnik1@mail.muni.cz>
2024-11-26 13:30:17 -06:00
Brent Westbrook
9f446faa6c [pyflakes] Avoid false positives in @no_type_check contexts (F821, F722) (#14615) 2024-11-26 19:13:43 +00:00
David Peter
b94d6cf567 [red-knot] Fix panic related to f-strings in annotations (#14613)
## Summary

Fix panics related to expressions without inferred types in invalid
syntax examples like:
```py
x: f"Literal[{1 + 2}]" = 3
```
where the `1 + 2` expression (and its sub-expressions) inside the
annotation did not have an inferred type.

## Test Plan

Added new corpus test.
2024-11-26 16:35:44 +01:00
David Peter
cd0c97211c [red-knot] Update KNOWN_FAILURES (#14612)
## Summary

Remove entry that was prevously fixed in
5a30ec0df6.

## Test Plan

```sh
cargo test -p red_knot_workspace -- --ignored linter_af linter_gz
```
2024-11-26 15:56:42 +01:00
David Peter
0e71c9e3bb [red-knot] Fix unit tests in release mode (#14604)
## Summary

This is about the easiest patch that I can think of. It has a drawback
in that there is no real guarantee this won't happen again. I think this
might be acceptable, given that all of this is a temporary thing.

And we also add a new CI job to prevent regressions like this in the
future.

For the record though, I'm listing alternative approaches I thought of:

- We could get rid of the debug/release distinction and just add `@Todo`
type metadata everywhere. This has possible affects on runtime. The main
reason I didn't follow through with this is that the size of `Type`
increases. We would either have to adapt the `assert_eq_size!` test or
get rid of it. Even if we add messages everywhere and get rid of the
file-and-line-variant in the enum, it's not enough to get back to the
current release-mode size of `Type`.
- We could generally discard `@Todo` meta information when using it in
tests. I think this would be a huge drawback. I like that we can have
the actual messages in the mdtest. And make sure we get the expected
`@Todo` type, not just any `@Todo`. It's also helpful when debugging
tests.

closes #14594

## Test Plan

```rs
cargo nextest run --release
```
2024-11-26 15:40:02 +01:00
Dylan
24c90d6953 [pylint] Do not wrap function calls in parentheses in the fix for unnecessary-dunder-call (PLC2801) (#14601) 2024-11-26 06:47:01 -06:00
Tzu-ping Chung
fbff4dec3a [airflow] Avoid implicit DAG schedule (AIR301) (#14581) 2024-11-26 13:38:18 +01:00
Dhruv Manilawala
f3dac27e9a Fix f-string formatting in assignment statement (#14454)
## Summary

fixes: #13813

This PR fixes a bug in the formatting assignment statement when the
value is an f-string.

This is resolved by using custom best fit layouts if the f-string is (a)
not already a flat f-string (thus, cannot be multiline) and (b) is not a
multiline string (thus, cannot be flattened). So, it is used in cases
like the following:
```py
aaaaaaaaaaaaaaaaaa = f"testeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee{
    expression}moreeeeeeeeeeeeeeeee"
```
Which is (a) `FStringLayout::Multiline` and (b) not a multiline.

There are various other examples in the PR diff along with additional
explanation and context as code comments.

## Test Plan

Add multiple test cases for various scenarios.
2024-11-26 15:07:18 +05:30
Simon Brugman
e4cefd9bf9 Extend test cases for flake8-pyi (#14280) 2024-11-26 09:10:38 +01:00
Lokejoke
9e4ee98109 [ruff] Implement invalid-assert-message-literal-argument (RUF040) (#14488)
## Summary

This PR implements new rule discussed
[here](https://github.com/astral-sh/ruff/discussions/14449).
In short, it searches for assert messages which were unintentionally
used as a expression to be matched against.

## Test Plan

`cargo test` and review of `ruff-ecosystem`
2024-11-25 17:41:07 -06:00
Shaygan Hooshyari
557d583e32 Support typing.NoReturn and typing.Never (#14559)
Fix #14558 
## Summary

- Add `typing.NoReturn` and `typing.Never` to known instances and infer
them as `Type::Never`
- Add `is_assignable_to` cases for `Type::Never`

I skipped emitting diagnostic for when a function is annotated as
`NoReturn` but it actually returns.

## Test Plan

Added tests from

https://github.com/python/typing/blob/main/conformance/tests/specialtypes_never.py
except from generics and checking if the return value of the function
and the annotations match.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Carl Meyer <carl@astral.sh>
2024-11-25 21:37:55 +00:00
cake-monotone
f98eebdbab [red-knot] Fix Leaking Narrowing Constraint in ast::ExprIf (#14590)
## Summary

Closes #14588


```py
x: Literal[42, "hello"] = 42 if bool_instance() else "hello"
reveal_type(x)  # revealed: Literal[42] | Literal["hello"]

_ = ... if isinstance(x, str) else ...

# The `isinstance` test incorrectly narrows the type of `x`.
# As a result, `x` is revealed as Literal["hello"], but it should remain Literal[42, "hello"].
reveal_type(x)  # revealed: Literal["hello"]
```

## Test Plan
mdtest included!

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-11-25 10:36:37 -08:00
Simon Brugman
c606bf014e [flake8-pyi] Improve autofix safety for redundant-none-literal (PYI061) (#14583)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-11-25 17:40:57 +00:00
Simon Brugman
e8fce20736 [ruff] Improve autofix safety for never-union (RUF020) (#14589) 2024-11-25 18:35:07 +01:00
Dhruv Manilawala
5a30ec0df6 Avoid inferring invalid expr types for string annotation (#14447)
## Summary

fixes: #14440

## Test Plan

Add a test case with all the invalid expressions in a string annotation
context.
2024-11-25 21:27:03 +05:30
Alex Waygood
fab1b0d546 fuzz-parser: catch exceptions from pysource-minimize (#14586) 2024-11-25 15:14:01 +00:00
Connor Skees
66abef433b red-knot: adapt fuzz-parser to work with red-knot (#14566)
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2024-11-25 13:12:28 +00:00
Harutaka Kawamura
fa22bd604a Fix pytest.mark.parametrize rules to check calls instead of decorators (#14515)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-11-25 13:55:18 +01:00
Dhruv Manilawala
0c9165fc3a Use Result for failed text document retrieval in LSP requests (#14579)
## Summary

Ref:
https://github.com/astral-sh/ruff-vscode/issues/644#issuecomment-2496588452

## Test Plan

Not sure how to test this as this is mainly to get more context on the
panic that the server is raising.
2024-11-25 15:14:30 +05:30
renovate[bot]
9f6147490b Update NPM Development dependencies (#14577)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-11-25 08:54:51 +01:00
renovate[bot]
b7571c3e24 Update Rust crate syn to v2.0.89 (#14573)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-11-25 07:46:06 +00:00
renovate[bot]
d178d115f3 Update dependency mdformat to v0.7.19 (#14576)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-11-25 08:40:53 +01:00
renovate[bot]
6501782678 Update Rust crate libcst to v1.5.1 (#14570)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-11-25 08:39:22 +01:00
renovate[bot]
bca4341dcc Update Rust crate hashbrown to v0.15.2 (#14569)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-11-25 08:38:34 +01:00
renovate[bot]
31ede11774 Update Rust crate quick-junit to v0.5.1 (#14572)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-11-25 08:38:12 +01:00
renovate[bot]
ba9f881687 Update Rust crate proc-macro2 to v1.0.92 (#14571)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-11-25 08:38:00 +01:00
renovate[bot]
4357a0a3c2 Update Rust crate unicode-ident to v1.0.14 (#14574)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-11-25 08:36:50 +01:00
renovate[bot]
c18afa93b3 Update Rust crate url to v2.5.4 (#14575)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-11-25 08:36:28 +01:00
renovate[bot]
8f04202ee4 Update Rust crate dir-test to 0.4.0 (#14578)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-11-25 08:36:06 +01:00
Dhruv Manilawala
efe54081d6 Remove FormatFStringPart (#14448)
## Summary

This is just a small refactor to remove the `FormatFStringPart` as it's
only used in the case when the f-string is not implicitly concatenated
in which case the only part is going to be `FString`. In implicitly
concatenated f-strings, we use `StringLike` instead.
2024-11-25 10:29:22 +05:30
Alex Waygood
ac23c99744 [ruff] Mark fixes for unsorted-dunder-all and unsorted-dunder-slots as unsafe when there are complex comments in the sequence (RUF022, RUF023) (#14560) 2024-11-24 12:49:29 +00:00
InSync
e5c7d87461 Add @astropy/astropy to ecosystem checks (#14565) 2024-11-24 12:47:11 +01:00
Charlie Marsh
de62e39eba Use truthiness check in auto_attribs detection (#14562) 2024-11-23 22:06:10 -05:00
InSync
d285717da8 [ruff] Handle attrs's auto_attribs correctly (RUF009) (#14520)
## Summary

Resolves #14519.

## Test Plan

`cargo nextest run` and `cargo insta test`.
2024-11-23 21:46:38 -05:00
InSync
545e9deba3 [flake8-builtins] Exempt private built-in modules (A005) (#14505)
## Summary

Resolves #12949.

## Test Plan

`cargo nextest run` and `cargo insta test`.
2024-11-23 21:39:04 -05:00
Harutaka Kawamura
e3d792605f [flake8-bugbear] Fix mutable-contextvar-default (B039) to resolve annotated function calls properly (#14532)
## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

Fix #14525

## Test Plan

<!-- How was it tested? -->

New test cases

---------

Signed-off-by: harupy <hkawamura0130@gmail.com>
2024-11-23 21:29:25 -05:00
Harutaka Kawamura
1f303a5eb6 Simplify flake8_pytest_style::rules::fail_call implementation (#14556) 2024-11-23 15:14:28 +01:00
Nikolas Hearp
07d13c6b4a [B028-doc-update] Update documentation for B028 (#14338)
## Summary
Resolves #14289
The documentation for B028 no_explicit_stacklevel is updated to be more
clear.

---------

Co-authored-by: dylwil3 <dylwil3@gmail.com>
2024-11-23 07:45:28 +00:00
Dylan
e1838aac29 Ignore more rules for stub files (#14541)
This PR causes the following rules to ignore stub files, on the grounds
that it is not under the author's control to appease these lints:

- `PLR0904` https://docs.astral.sh/ruff/rules/too-many-public-methods/
- `PLR0913` https://docs.astral.sh/ruff/rules/too-many-arguments/
- `PLR0917`
https://docs.astral.sh/ruff/rules/too-many-positional-arguments/
- `PLW3201` https://docs.astral.sh/ruff/rules/bad-dunder-method-name/
- `SLOT` https://docs.astral.sh/ruff/rules/#flake8-slots-slot
- `FBT` https://docs.astral.sh/ruff/rules/#flake8-boolean-trap-fbt
(except for FBT003 since that involves a function call.)

Progress towards #14535
2024-11-23 07:41:10 +00:00
Carl Meyer
4ba847f250 [red-knot] remove wrong typevar attribute implementations (#14540) 2024-11-22 13:17:16 -08:00
renovate[bot]
13e9fc9362 Update dependency smol-toml to v1.3.1 (#14542)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-11-22 21:16:05 +00:00
Dylan
3fda2d17c7 [ruff] Auto-add r prefix when string has no backslashes for unraw-re-pattern (RUF039) (#14536)
This PR adds a sometimes-available, safe autofix for [unraw-re-pattern
(RUF039)](https://docs.astral.sh/ruff/rules/unraw-re-pattern/#unraw-re-pattern-ruf039),
which prepends an `r` prefix. It is used only when the string in
question has no backslahses (and also does not have a `u` prefix, since
that causes a syntax error.)

Closes #14527

Notes: 
- Test fixture unchanged, but snapshot changed to include fix messages.
- This fix is automatically only available in preview since the rule
itself is in preview
2024-11-22 15:09:53 -06:00
Harutaka Kawamura
931fa06d85 Extend invalid-envvar-default (PLW1508) to detect os.environ.get (#14512) 2024-11-22 19:13:58 +00:00
Micha Reiser
e53ac7985d Enable logging for directory-renamed test (#14533) 2024-11-22 16:41:46 +00:00
David Salvisberg
e25e7044ba [flake8-type-checking] Adds implementation for TC006 (#14511)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-11-22 15:22:59 +01:00
Micha Reiser
b80de52592 Consider quotes inside format-specs when choosing the quotes for an f-string (#14493) 2024-11-22 12:43:53 +00:00
Alex Waygood
2917534279 Fix broken link to PYI063 (#14526) 2024-11-22 12:27:52 +00:00
David Peter
f6b2cd5588 [red-knot] Semantic index: handle invalid breaks (#14522)
## Summary

This fix addresses panics related to invalid syntax like the following
where a `break` statement is used in a nested definition inside a
loop:

```py
while True:

    def b():
        x: int

        break
```

closes #14342

## Test Plan

* New corpus regression tests.
* New unit test to make sure we handle nested while loops correctly.
This test is passing on `main`, but can easily fail if the
`is_inside_loop` state isn't properly saved/restored.
2024-11-22 13:13:55 +01:00
Micha Reiser
302fe76c2b Fix unnecessary space around power op in overlong f-string expressions (#14489) 2024-11-22 13:01:22 +01:00
852 changed files with 15130 additions and 4581 deletions

5
.github/CODEOWNERS vendored
View File

@@ -13,9 +13,10 @@
# flake8-pyi
/crates/ruff_linter/src/rules/flake8_pyi/ @AlexWaygood
# Script for fuzzing the parser
/scripts/fuzz-parser/ @AlexWaygood
# Script for fuzzing the parser/red-knot etc.
/python/py-fuzzer/ @AlexWaygood
# red-knot
/crates/red_knot* @carljm @MichaReiser @AlexWaygood @sharkdp
/crates/ruff_db/ @carljm @MichaReiser @AlexWaygood @sharkdp
/scripts/knot_benchmark/ @carljm @MichaReiser @AlexWaygood @sharkdp

View File

@@ -49,7 +49,7 @@ jobs:
- crates/ruff_text_size/**
- crates/ruff_python_ast/**
- crates/ruff_python_parser/**
- scripts/fuzz-parser/**
- python/py-fuzzer/**
- .github/workflows/ci.yaml
linter:
@@ -82,6 +82,7 @@ jobs:
code:
- "**/*"
- "!**/*.md"
- "crates/red_knot_python_semantic/resources/mdtest/**/*.md"
- "!docs/**"
- "!assets/**"
@@ -157,6 +158,33 @@ jobs:
name: ruff
path: target/debug/ruff
cargo-test-linux-release:
name: "cargo test (linux, release)"
runs-on: depot-ubuntu-22.04-16
needs: determine_changes
if: ${{ needs.determine_changes.outputs.code == 'true' || github.ref == 'refs/heads/main' }}
timeout-minutes: 20
steps:
- uses: actions/checkout@v4
- name: "Install Rust toolchain"
run: rustup show
- name: "Install mold"
uses: rui314/setup-mold@v1
- name: "Install cargo nextest"
uses: taiki-e/install-action@v2
with:
tool: cargo-nextest
- name: "Install cargo insta"
uses: taiki-e/install-action@v2
with:
tool: cargo-insta
- uses: Swatinem/rust-cache@v2
- name: "Run tests"
shell: bash
env:
NEXTEST_PROFILE: "ci"
run: cargo insta test --release --all-features --unreferenced reject --test-runner nextest
cargo-test-windows:
name: "cargo test (windows)"
runs-on: windows-latest-xlarge
@@ -212,7 +240,6 @@ jobs:
cargo-build-release:
name: "cargo build (release)"
runs-on: macos-latest
needs: determine_changes
if: ${{ github.ref == 'refs/heads/main' }}
timeout-minutes: 20
steps:
@@ -291,13 +318,7 @@ jobs:
FORCE_COLOR: 1
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Install uv
run: curl -LsSf https://astral.sh/uv/install.sh | sh
- name: Install Python requirements
run: uv pip install -r scripts/fuzz-parser/requirements.txt --system
- uses: astral-sh/setup-uv@v4
- uses: actions/download-artifact@v4
name: Download Ruff binary to test
id: download-cached-binary
@@ -309,7 +330,15 @@ jobs:
# Make executable, since artifact download doesn't preserve this
chmod +x ${{ steps.download-cached-binary.outputs.download-path }}/ruff
python scripts/fuzz-parser/fuzz.py 0-500 --test-executable ${{ steps.download-cached-binary.outputs.download-path }}/ruff
(
uvx \
--python=${{ env.PYTHON_VERSION }} \
--from=./python/py-fuzzer \
fuzz \
--test-executable=${{ steps.download-cached-binary.outputs.download-path }}/ruff \
--bin=ruff \
0-500
)
scripts:
name: "test scripts"

View File

@@ -32,13 +32,7 @@ jobs:
if: ${{ github.repository == 'astral-sh/ruff' || github.event_name != 'schedule' }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install uv
run: curl -LsSf https://astral.sh/uv/install.sh | sh
- name: Install Python requirements
run: uv pip install -r scripts/fuzz-parser/requirements.txt --system
- uses: astral-sh/setup-uv@v4
- name: "Install Rust toolchain"
run: rustup show
- name: "Install mold"
@@ -49,7 +43,16 @@ jobs:
# but this is outweighed by the fact that a release build takes *much* longer to compile in CI
run: cargo build --locked
- name: Fuzz
run: python scripts/fuzz-parser/fuzz.py $(shuf -i 0-9999999999999999999 -n 1000) --test-executable target/debug/ruff
run: |
(
uvx \
--python=3.12 \
--from=./python/py-fuzzer \
fuzz \
--test-executable=target/debug/ruff \
--bin=ruff \
$(shuf -i 0-9999999999999999999 -n 1000)
)
create-issue-on-failure:
name: Create an issue if the daily fuzz surfaced any bugs

View File

@@ -1,5 +1,43 @@
# Changelog
## 0.8.1
### Preview features
- Formatter: Avoid invalid syntax for format-spec with quotes for all Python versions ([#14625](https://github.com/astral-sh/ruff/pull/14625))
- Formatter: Consider quotes inside format-specs when choosing the quotes for an f-string ([#14493](https://github.com/astral-sh/ruff/pull/14493))
- Formatter: Do not consider f-strings with escaped newlines as multiline ([#14624](https://github.com/astral-sh/ruff/pull/14624))
- Formatter: Fix f-string formatting in assignment statement ([#14454](https://github.com/astral-sh/ruff/pull/14454))
- Formatter: Fix unnecessary space around power operator (`**`) in overlong f-string expressions ([#14489](https://github.com/astral-sh/ruff/pull/14489))
- \[`airflow`\] Avoid implicit `schedule` argument to `DAG` and `@dag` (`AIR301`) ([#14581](https://github.com/astral-sh/ruff/pull/14581))
- \[`flake8-builtins`\] Exempt private built-in modules (`A005`) ([#14505](https://github.com/astral-sh/ruff/pull/14505))
- \[`flake8-pytest-style`\] Fix `pytest.mark.parametrize` rules to check calls instead of decorators ([#14515](https://github.com/astral-sh/ruff/pull/14515))
- \[`flake8-type-checking`\] Implement `runtime-cast-value` (`TC006`) ([#14511](https://github.com/astral-sh/ruff/pull/14511))
- \[`flake8-type-checking`\] Implement `unquoted-type-alias` (`TC007`) and `quoted-type-alias` (`TC008`) ([#12927](https://github.com/astral-sh/ruff/pull/12927))
- \[`flake8-use-pathlib`\] Recommend `Path.iterdir()` over `os.listdir()` (`PTH208`) ([#14509](https://github.com/astral-sh/ruff/pull/14509))
- \[`pylint`\] Extend `invalid-envvar-default` to detect `os.environ.get` (`PLW1508`) ([#14512](https://github.com/astral-sh/ruff/pull/14512))
- \[`pylint`\] Implement `len-test` (`PLC1802`) ([#14309](https://github.com/astral-sh/ruff/pull/14309))
- \[`refurb`\] Fix bug where methods defined using lambdas were flagged by `FURB118` ([#14639](https://github.com/astral-sh/ruff/pull/14639))
- \[`ruff`\] Auto-add `r` prefix when string has no backslashes for `unraw-re-pattern` (`RUF039`) ([#14536](https://github.com/astral-sh/ruff/pull/14536))
- \[`ruff`\] Implement `invalid-assert-message-literal-argument` (`RUF040`) ([#14488](https://github.com/astral-sh/ruff/pull/14488))
- \[`ruff`\] Implement `unnecessary-nested-literal` (`RUF041`) ([#14323](https://github.com/astral-sh/ruff/pull/14323))
### Rule changes
- Ignore more rules for stub files ([#14541](https://github.com/astral-sh/ruff/pull/14541))
- \[`pep8-naming`\] Eliminate false positives for single-letter names (`N811`, `N814`) ([#14584](https://github.com/astral-sh/ruff/pull/14584))
- \[`pyflakes`\] Avoid false positives in `@no_type_check` contexts (`F821`, `F722`) ([#14615](https://github.com/astral-sh/ruff/pull/14615))
- \[`ruff`\] Detect redirected-noqa in file-level comments (`RUF101`) ([#14635](https://github.com/astral-sh/ruff/pull/14635))
- \[`ruff`\] Mark fixes for `unsorted-dunder-all` and `unsorted-dunder-slots` as unsafe when there are complex comments in the sequence (`RUF022`, `RUF023`) ([#14560](https://github.com/astral-sh/ruff/pull/14560))
### Bug fixes
- Avoid fixing code to `None | None` for `redundant-none-literal` (`PYI061`) and `never-union` (`RUF020`) ([#14583](https://github.com/astral-sh/ruff/pull/14583), [#14589](https://github.com/astral-sh/ruff/pull/14589))
- \[`flake8-bugbear`\] Fix `mutable-contextvar-default` to resolve annotated function calls properly (`B039`) ([#14532](https://github.com/astral-sh/ruff/pull/14532))
- \[`flake8-type-checking`\] Avoid syntax errors and type checking problem for quoted annotations autofix (`TC003`, `TC006`) ([#14634](https://github.com/astral-sh/ruff/pull/14634))
- \[`pylint`\] Do not wrap function calls in parentheses in the fix for unnecessary-dunder-call (`PLC2801`) ([#14601](https://github.com/astral-sh/ruff/pull/14601))
- \[`ruff`\] Handle `attrs`'s `auto_attribs` correctly (`RUF009`) ([#14520](https://github.com/astral-sh/ruff/pull/14520))
## 0.8.0
Check out the [blog post](https://astral.sh/blog/ruff-v0.8.0) for a migration guide and overview of the changes!
@@ -57,7 +95,7 @@ The following rules have been stabilized and are no longer in preview:
- [`fast-api-redundant-response-model`](https://docs.astral.sh/ruff/rules/fast-api-redundant-response-model/) (`FAST001`)
- [`fast-api-non-annotated-dependency`](https://docs.astral.sh/ruff/rules/fast-api-non-annotated-dependency/) (`FAST002`)
- [`dict-index-missing-items`](https://docs.astral.sh/ruff/rules/dict-index-missing-items/) (`PLC0206`)
- [`pep484-style-positional-only-argument`](https://docs.astral.sh/ruff/rules/pep484-style-positional-only-argument/) (`PYI063`)
- [`pep484-style-positional-only-parameter`](https://docs.astral.sh/ruff/rules/pep484-style-positional-only-parameter/) (`PYI063`)
- [`redundant-final-literal`](https://docs.astral.sh/ruff/rules/redundant-final-literal/) (`PYI064`)
- [`bad-version-info-order`](https://docs.astral.sh/ruff/rules/bad-version-info-order/) (`PYI066`)
- [`parenthesize-chained-operators`](https://docs.astral.sh/ruff/rules/parenthesize-chained-operators/) (`RUF021`)
@@ -1077,7 +1115,7 @@ The following deprecated CLI commands have been removed:
### Preview features
- \[`flake8-bugbear`\] Implement `return-in-generator` (`B901`) ([#11644](https://github.com/astral-sh/ruff/pull/11644))
- \[`flake8-pyi`\] Implement `PYI063` ([#11699](https://github.com/astral-sh/ruff/pull/11699))
- \[`flake8-pyi`\] Implement `pep484-style-positional-only-parameter` (`PYI063`) ([#11699](https://github.com/astral-sh/ruff/pull/11699))
- \[`pygrep_hooks`\] Check blanket ignores via file-level pragmas (`PGH004`) ([#11540](https://github.com/astral-sh/ruff/pull/11540))
### Rule changes

View File

@@ -139,7 +139,7 @@ At a high level, the steps involved in adding a new lint rule are as follows:
1. Create a file for your rule (e.g., `crates/ruff_linter/src/rules/flake8_bugbear/rules/assert_false.rs`).
1. In that file, define a violation struct (e.g., `pub struct AssertFalse`). You can grep for
`#[violation]` to see examples.
`#[derive(ViolationMetadata)]` to see examples.
1. In that file, define a function that adds the violation to the diagnostic list as appropriate
(e.g., `pub(crate) fn assert_false`) based on whatever inputs are required for the rule (e.g.,

129
Cargo.lock generated
View File

@@ -413,7 +413,7 @@ dependencies = [
"heck",
"proc-macro2",
"quote",
"syn 2.0.87",
"syn",
]
[[package]]
@@ -693,7 +693,7 @@ dependencies = [
"proc-macro2",
"quote",
"strsim 0.10.0",
"syn 2.0.87",
"syn",
]
[[package]]
@@ -704,7 +704,7 @@ checksum = "a668eda54683121533a393014d8692171709ff57a7d61f187b6e782719f8933f"
dependencies = [
"darling_core",
"quote",
"syn 2.0.87",
"syn",
]
[[package]]
@@ -758,23 +758,23 @@ dependencies = [
[[package]]
name = "dir-test"
version = "0.3.0"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5c44bdf9319ad5223afb7eb15a7110452b0adf0373ea6756561b2c708eef0dd1"
checksum = "b12781621d53fd9087021f5a338df5c57c04f84a6231c1f4726f45e2e333470b"
dependencies = [
"dir-test-macros",
]
[[package]]
name = "dir-test-macros"
version = "0.3.0"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "644f96047137dfaa7a09e34d4623f9e52a1926ecc25ba32ad2ba3fc422536b25"
checksum = "1340852f50b2285d01a7f598cc5d08b572669c3e09e614925175cc3c26787b91"
dependencies = [
"glob",
"proc-macro2",
"quote",
"syn 1.0.109",
"syn",
]
[[package]]
@@ -826,7 +826,7 @@ checksum = "97369cbbc041bc366949bc74d34658d6cda5621039731c6310521892a3a20ae0"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.87",
"syn",
]
[[package]]
@@ -1068,9 +1068,9 @@ dependencies = [
[[package]]
name = "hashbrown"
version = "0.15.1"
version = "0.15.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3a9bfc1af68b1726ea47d3d5109de126281def866b33970e10fbab11b5dafab3"
checksum = "bf151400ff0baff5465007dd2f3e717f3fe502074ca563069ce3a6629d07b289"
[[package]]
name = "hashlink"
@@ -1246,7 +1246,7 @@ checksum = "1ec89e9337638ecdc08744df490b221a7399bf8d164eb52a665454e60e075ad6"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.87",
"syn",
]
[[package]]
@@ -1319,7 +1319,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "707907fe3c25f5424cce2cb7e1cbcafee6bdbe735ca90ef77c29e84591e5b9da"
dependencies = [
"equivalent",
"hashbrown 0.15.1",
"hashbrown 0.15.2",
"serde",
]
@@ -1420,7 +1420,7 @@ dependencies = [
"heck",
"proc-macro2",
"quote",
"syn 2.0.87",
"syn",
]
[[package]]
@@ -1526,9 +1526,9 @@ checksum = "433bfe06b8c75da9b2e3fbea6e5329ff87748f0b144ef75306e674c3f6f7c13f"
[[package]]
name = "libcst"
version = "1.5.0"
version = "1.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1586dd7a857d8a61a577afde1a24cc9573ff549eff092d5ce968b1ec93cc61b6"
checksum = "fa3e60579a8cba3d86aa4a5f7fc98973cc0fd2ac270bf02f85a9bef09700b075"
dependencies = [
"chic",
"libcst_derive",
@@ -1546,7 +1546,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a2ae40017ac09cd2c6a53504cb3c871c7f2b41466eac5bc66ba63f39073b467b"
dependencies = [
"quote",
"syn 2.0.87",
"syn",
]
[[package]]
@@ -1710,9 +1710,9 @@ checksum = "308d96db8debc727c3fd9744aac51751243420e46edf401010908da7f8d5e57c"
[[package]]
name = "newtype-uuid"
version = "1.1.0"
version = "1.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3526cb7c660872e401beaf3297f95f548ce3b4b4bdd8121b7c0713771d7c4a6e"
checksum = "4c8781e2ef64806278a55ad223f0bc875772fd40e1fe6e73e8adbf027817229d"
dependencies = [
"uuid",
]
@@ -2012,7 +2012,7 @@ dependencies = [
"pest_meta",
"proc-macro2",
"quote",
"syn 2.0.87",
"syn",
]
[[package]]
@@ -2127,9 +2127,9 @@ dependencies = [
[[package]]
name = "proc-macro2"
version = "1.0.89"
version = "1.0.92"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f139b0662de085916d1fb67d2b4169d1addddda1919e696f3252b740b629986e"
checksum = "37d3544b3f2748c54e147655edb5025752e2303145b5aefb3c3ea2c78b973bb0"
dependencies = [
"unicode-ident",
]
@@ -2150,24 +2150,24 @@ dependencies = [
[[package]]
name = "quick-junit"
version = "0.5.0"
version = "0.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "62ffd2f9a162cfae131bed6d9d1ed60adced33be340a94f96952897d7cb0c240"
checksum = "3ed1a693391a16317257103ad06a88c6529ac640846021da7c435a06fffdacd7"
dependencies = [
"chrono",
"indexmap",
"newtype-uuid",
"quick-xml",
"strip-ansi-escapes",
"thiserror 1.0.67",
"thiserror 2.0.3",
"uuid",
]
[[package]]
name = "quick-xml"
version = "0.36.1"
version = "0.37.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "96a05e2e8efddfa51a84ca47cec303fac86c8541b686d37cac5efc0e094417bc"
checksum = "f22f29bdff3987b4d8632ef95fd6424ec7e4e0a57e2f4fc63e489e75357f6a03"
dependencies = [
"memchr",
]
@@ -2266,7 +2266,7 @@ dependencies = [
"compact_str",
"countme",
"dir-test",
"hashbrown 0.15.1",
"hashbrown 0.15.2",
"indexmap",
"insta",
"itertools 0.13.0",
@@ -2489,7 +2489,7 @@ dependencies = [
[[package]]
name = "ruff"
version = "0.8.0"
version = "0.8.1"
dependencies = [
"anyhow",
"argfile",
@@ -2708,7 +2708,7 @@ dependencies = [
[[package]]
name = "ruff_linter"
version = "0.8.0"
version = "0.8.1"
dependencies = [
"aho-corasick",
"annotate-snippets 0.9.2",
@@ -2775,7 +2775,7 @@ dependencies = [
"proc-macro2",
"quote",
"ruff_python_trivia",
"syn 2.0.87",
"syn",
]
[[package]]
@@ -3023,7 +3023,7 @@ dependencies = [
[[package]]
name = "ruff_wasm"
version = "0.8.0"
version = "0.8.1"
dependencies = [
"console_error_panic_hook",
"console_log",
@@ -3195,7 +3195,7 @@ dependencies = [
"heck",
"proc-macro2",
"quote",
"syn 2.0.87",
"syn",
"synstructure",
]
@@ -3229,7 +3229,7 @@ dependencies = [
"proc-macro2",
"quote",
"serde_derive_internals",
"syn 2.0.87",
"syn",
]
[[package]]
@@ -3278,7 +3278,7 @@ checksum = "ad1e866f866923f252f05c889987993144fb74e722403468a4ebd70c3cd756c0"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.87",
"syn",
]
[[package]]
@@ -3289,7 +3289,7 @@ checksum = "330f01ce65a3a5fe59a60c82f3c9a024b573b8a6e875bd233fe5f934e71d54e3"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.87",
"syn",
]
[[package]]
@@ -3312,7 +3312,7 @@ checksum = "6c64451ba24fc7a6a2d60fc75dd9c83c90903b19028d4eff35e88fc1e86564e9"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.87",
"syn",
]
[[package]]
@@ -3353,7 +3353,7 @@ dependencies = [
"darling",
"proc-macro2",
"quote",
"syn 2.0.87",
"syn",
]
[[package]]
@@ -3461,7 +3461,7 @@ dependencies = [
"proc-macro2",
"quote",
"rustversion",
"syn 2.0.87",
"syn",
]
[[package]]
@@ -3472,20 +3472,9 @@ checksum = "81cdd64d312baedb58e21336b31bc043b77e01cc99033ce76ef539f78e965ebc"
[[package]]
name = "syn"
version = "1.0.109"
version = "2.0.89"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "72b64191b275b66ffe2469e8af2c1cfe3bafa67b529ead792a6d0160888b4237"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]]
name = "syn"
version = "2.0.87"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "25aa4ce346d03a6dcd68dd8b4010bcb74e54e62c90c573f394c46eae99aba32d"
checksum = "44d46482f1c1c87acd84dea20c1bf5ebff4c757009ed6bf19cfd36fb10e92c4e"
dependencies = [
"proc-macro2",
"quote",
@@ -3500,7 +3489,7 @@ checksum = "c8af7666ab7b6390ab78131fb5b0fce11d6b7a6951602017c35fa82800708971"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.87",
"syn",
]
[[package]]
@@ -3563,7 +3552,7 @@ dependencies = [
"cfg-if",
"proc-macro2",
"quote",
"syn 2.0.87",
"syn",
]
[[package]]
@@ -3574,7 +3563,7 @@ checksum = "5c89e72a01ed4c579669add59014b9a524d609c0c88c6a585ce37485879f6ffb"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.87",
"syn",
"test-case-core",
]
@@ -3604,7 +3593,7 @@ checksum = "b607164372e89797d78b8e23a6d67d5d1038c1c65efd52e1389ef8b77caba2a6"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.87",
"syn",
]
[[package]]
@@ -3615,7 +3604,7 @@ checksum = "f077553d607adc1caf65430528a576c757a71ed73944b66ebb58ef2bbd243568"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.87",
"syn",
]
[[package]]
@@ -3737,7 +3726,7 @@ checksum = "34704c8d6ebcbc939824180af020566b01a7c01f80641264eba0999f6c2b6be7"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.87",
"syn",
]
[[package]]
@@ -3876,9 +3865,9 @@ dependencies = [
[[package]]
name = "unicode-ident"
version = "1.0.13"
version = "1.0.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e91b56cd4cadaeb79bbf1a5645f6b4f8dc5bde8834ad5894a8db35fda9efa1fe"
checksum = "adb9e6ca4f869e1180728b7950e35922a7fc6397f7b641499e8f3ef06e50dc83"
[[package]]
name = "unicode-normalization"
@@ -3953,9 +3942,9 @@ dependencies = [
[[package]]
name = "url"
version = "2.5.3"
version = "2.5.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8d157f1b96d14500ffdc1f10ba712e780825526c03d9a49b4d0324b0d9113ada"
checksum = "32f8b686cadd1473f4bd0117a5d28d36b1ade384ea9b5069a1c40aefed7fda60"
dependencies = [
"form_urlencoded",
"idna",
@@ -4007,7 +3996,7 @@ checksum = "6b91f57fe13a38d0ce9e28a03463d8d3c2468ed03d75375110ec71d93b449a08"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.87",
"syn",
]
[[package]]
@@ -4102,7 +4091,7 @@ dependencies = [
"once_cell",
"proc-macro2",
"quote",
"syn 2.0.87",
"syn",
"wasm-bindgen-shared",
]
@@ -4136,7 +4125,7 @@ checksum = "26c6ab57572f7a24a4985830b120de1594465e5d500f24afe89e16b4e833ef68"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.87",
"syn",
"wasm-bindgen-backend",
"wasm-bindgen-shared",
]
@@ -4170,7 +4159,7 @@ checksum = "c97b2ef2c8d627381e51c071c2ab328eac606d3f69dd82bcbca20a9e389d95f0"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.87",
"syn",
]
[[package]]
@@ -4473,7 +4462,7 @@ checksum = "28cc31741b18cb6f1d5ff12f5b7523e3d6eb0852bbbad19d73905511d9849b95"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.87",
"syn",
"synstructure",
]
@@ -4494,7 +4483,7 @@ checksum = "9ce1b18ccd8e73a9321186f97e46f9f04b778851177567b1975109d26a08d2a6"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.87",
"syn",
]
[[package]]
@@ -4514,7 +4503,7 @@ checksum = "0ea7b4a3637ea8669cedf0f1fd5c286a17f3de97b8dd5a70a6c167a1730e63a5"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.87",
"syn",
"synstructure",
]
@@ -4543,7 +4532,7 @@ checksum = "6eafa6dfb17584ea3e2bd6e76e0cc15ad7af12b09abdd1ca55961bed9b1063c6"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.87",
"syn",
]
[[package]]

View File

@@ -65,7 +65,7 @@ compact_str = "0.8.0"
criterion = { version = "0.5.1", default-features = false }
crossbeam = { version = "0.8.4" }
dashmap = { version = "6.0.1" }
dir-test = { version = "0.3.0" }
dir-test = { version = "0.4.0" }
dunce = { version = "1.0.5" }
drop_bomb = { version = "0.1.5" }
env_logger = { version = "0.11.0" }

View File

@@ -136,8 +136,8 @@ curl -LsSf https://astral.sh/ruff/install.sh | sh
powershell -c "irm https://astral.sh/ruff/install.ps1 | iex"
# For a specific version.
curl -LsSf https://astral.sh/ruff/0.8.0/install.sh | sh
powershell -c "irm https://astral.sh/ruff/0.8.0/install.ps1 | iex"
curl -LsSf https://astral.sh/ruff/0.8.1/install.sh | sh
powershell -c "irm https://astral.sh/ruff/0.8.1/install.ps1 | iex"
```
You can also install Ruff via [Homebrew](https://formulae.brew.sh/formula/ruff), [Conda](https://anaconda.org/conda-forge/ruff),
@@ -170,7 +170,7 @@ Ruff can also be used as a [pre-commit](https://pre-commit.com/) hook via [`ruff
```yaml
- repo: https://github.com/astral-sh/ruff-pre-commit
# Ruff version.
rev: v0.8.0
rev: v0.8.1
hooks:
# Run the linter.
- id: ruff

View File

@@ -1,21 +1,25 @@
doc-valid-idents = [
"..",
"CodeQL",
"FastAPI",
"IPython",
"LangChain",
"LibCST",
"McCabe",
"NumPy",
"SCREAMING_SNAKE_CASE",
"SQLAlchemy",
"StackOverflow",
"PyCharm",
"..",
"CodeQL",
"FastAPI",
"IPython",
"LangChain",
"LibCST",
"McCabe",
"NumPy",
"SCREAMING_SNAKE_CASE",
"SQLAlchemy",
"StackOverflow",
"PyCharm",
"SNMPv1",
"SNMPv2",
"SNMPv3",
"PyFlakes"
]
ignore-interior-mutability = [
# Interned is read-only. The wrapped `Rc` never gets updated.
"ruff_formatter::format_element::Interned",
# The expression is read-only.
"ruff_python_ast::hashable::HashableExpr",
# Interned is read-only. The wrapped `Rc` never gets updated.
"ruff_formatter::format_element::Interned",
# The expression is read-only.
"ruff_python_ast::hashable::HashableExpr",
]

View File

@@ -5,11 +5,11 @@
pub enum TargetVersion {
Py37,
Py38,
#[default]
Py39,
Py310,
Py311,
Py312,
#[default]
Py313,
}

View File

@@ -4,7 +4,6 @@ use std::io::Write;
use std::time::Duration;
use anyhow::{anyhow, Context};
use red_knot_python_semantic::{resolve_module, ModuleName, Program, PythonVersion, SitePackages};
use red_knot_workspace::db::{Db, RootDatabase};
use red_knot_workspace::watch;
@@ -14,7 +13,7 @@ use red_knot_workspace::workspace::WorkspaceMetadata;
use ruff_db::files::{system_path_to_file, File, FileError};
use ruff_db::source::source_text;
use ruff_db::system::{OsSystem, SystemPath, SystemPathBuf};
use ruff_db::testing::setup_logging;
use ruff_db::testing::{setup_logging, setup_logging_with_filter};
use ruff_db::Upcast;
struct TestCase {
@@ -47,6 +46,8 @@ impl TestCase {
}
fn try_stop_watch(&mut self, timeout: Duration) -> Option<Vec<watch::ChangeEvent>> {
tracing::debug!("Try stopping watch with timeout {:?}", timeout);
let watcher = self
.watcher
.take()
@@ -56,8 +57,11 @@ impl TestCase {
.changes_receiver
.recv_timeout(timeout)
.unwrap_or_default();
watcher.flush();
tracing::debug!("Flushed file watcher");
watcher.stop();
tracing::debug!("Stopping file watcher");
for event in &self.changes_receiver {
all_events.extend(event);
@@ -600,6 +604,8 @@ fn directory_moved_to_trash() -> anyhow::Result<()> {
#[test]
fn directory_renamed() -> anyhow::Result<()> {
let _tracing = setup_logging_with_filter("file_watching=TRACE,red_knot=TRACE");
let mut case = setup([
("bar.py", "import sub.a"),
("sub/__init__.py", ""),
@@ -640,6 +646,10 @@ fn directory_renamed() -> anyhow::Result<()> {
let changes = case.stop_watch();
for event in &changes {
tracing::debug!("Event: {:?}", event);
}
case.apply_changes(changes);
// `import sub.a` should no longer resolve

View File

@@ -0,0 +1,62 @@
# NoReturn & Never
`NoReturn` is used to annotate the return type for functions that never return. `Never` is the
bottom type, representing the empty set of Python objects. These two annotations can be used
interchangeably.
## Function Return Type Annotation
```py
from typing import NoReturn
def stop() -> NoReturn:
raise RuntimeError("no way")
# revealed: Never
reveal_type(stop())
```
## Assignment
```py
from typing import NoReturn, Never, Any
# error: [invalid-type-parameter] "Type `typing.Never` expected no type parameter"
x: Never[int]
a1: NoReturn
# TODO: Test `Never` is only available in python >= 3.11
a2: Never
b1: Any
b2: int
def f():
# revealed: Never
reveal_type(a1)
# revealed: Never
reveal_type(a2)
# Never is assignable to all types.
v1: int = a1
v2: str = a1
# Other types are not assignable to Never except for Never (and Any).
v3: Never = b1
v4: Never = a2
v5: Any = b2
# error: [invalid-assignment] "Object of type `Literal[1]` is not assignable to `Never`"
v6: Never = 1
```
## Typing Extensions
```py
from typing_extensions import NoReturn, Never
x: NoReturn
y: Never
def f():
# revealed: Never
reveal_type(x)
# revealed: Never
reveal_type(y)
```

View File

@@ -189,3 +189,31 @@ reveal_type(d) # revealed: Foo
## Parameter
TODO: Add tests once parameter inference is supported
## Invalid expressions
The expressions in these string annotations aren't valid expressions in this context but we
shouldn't panic.
```py
a: "1 or 2"
b: "(x := 1)"
c: "1 + 2"
d: "lambda x: x"
e: "x if True else y"
f: "{'a': 1, 'b': 2}"
g: "{1, 2}"
h: "[i for i in range(5)]"
i: "{i for i in range(5)}"
j: "{i: i for i in range(5)}"
k: "(i for i in range(5))"
l: "await 1"
# error: [forward-annotation-syntax-error]
m: "yield 1"
# error: [forward-annotation-syntax-error]
n: "yield from 1"
o: "1 < 2"
p: "call()"
r: "[1, 2]"
s: "(1, 2)"
```

View File

@@ -38,7 +38,7 @@ if (x := 1) and bool_instance():
if True or (x := 1):
# TODO: infer that the second arm is never executed, and raise `unresolved-reference`.
# error: [possibly-unresolved-reference]
reveal_type(x) # revealed: Literal[1]
reveal_type(x) # revealed: Never
if True and (x := 1):
# TODO: infer that the second arm is always executed, do not raise a diagnostic

View File

@@ -22,3 +22,22 @@ reveal_type(1 if None else 2) # revealed: Literal[2]
reveal_type(1 if "" else 2) # revealed: Literal[2]
reveal_type(1 if 0 else 2) # revealed: Literal[2]
```
## Leaked Narrowing Constraint
(issue #14588)
The test inside an if expression should not affect code outside of the expression.
```py
def bool_instance() -> bool:
return True
x: Literal[42, "hello"] = 42 if bool_instance() else "hello"
reveal_type(x) # revealed: Literal[42] | Literal["hello"]
_ = ... if isinstance(x, str) else ...
reveal_type(x) # revealed: Literal[42] | Literal["hello"]
```

View File

@@ -60,52 +60,20 @@ reveal_type(S) # revealed: Literal[S]
## Type params
A PEP695 type variable defines a value of type `typing.TypeVar` with attributes `__name__`,
`__bounds__`, `__constraints__`, and `__default__` (the latter three all lazily evaluated):
A PEP695 type variable defines a value of type `typing.TypeVar`.
```py
def f[T, U: A, V: (A, B), W = A, X: A = A1]():
def f[T]():
reveal_type(T) # revealed: T
reveal_type(T.__name__) # revealed: Literal["T"]
reveal_type(T.__bound__) # revealed: None
reveal_type(T.__constraints__) # revealed: tuple[()]
reveal_type(T.__default__) # revealed: NoDefault
reveal_type(U) # revealed: U
reveal_type(U.__name__) # revealed: Literal["U"]
reveal_type(U.__bound__) # revealed: type[A]
reveal_type(U.__constraints__) # revealed: tuple[()]
reveal_type(U.__default__) # revealed: NoDefault
reveal_type(V) # revealed: V
reveal_type(V.__name__) # revealed: Literal["V"]
reveal_type(V.__bound__) # revealed: None
reveal_type(V.__constraints__) # revealed: tuple[type[A], type[B]]
reveal_type(V.__default__) # revealed: NoDefault
reveal_type(W) # revealed: W
reveal_type(W.__name__) # revealed: Literal["W"]
reveal_type(W.__bound__) # revealed: None
reveal_type(W.__constraints__) # revealed: tuple[()]
reveal_type(W.__default__) # revealed: type[A]
reveal_type(X) # revealed: X
reveal_type(X.__name__) # revealed: Literal["X"]
reveal_type(X.__bound__) # revealed: type[A]
reveal_type(X.__constraints__) # revealed: tuple[()]
reveal_type(X.__default__) # revealed: type[A1]
class A: ...
class B: ...
class A1(A): ...
```
## Minimum two constraints
A typevar with less than two constraints emits a diagnostic and is treated as unconstrained:
A typevar with less than two constraints emits a diagnostic:
```py
# error: [invalid-typevar-constraints] "TypeVar must have at least two constrained types"
def f[T: (int,)]():
reveal_type(T.__constraints__) # revealed: tuple[()]
pass
```

View File

@@ -52,3 +52,29 @@ else:
reveal_type(x) # revealed: Literal[2, 3]
reveal_type(y) # revealed: Literal[1, 2, 4]
```
## Nested while loops
```py
def flag() -> bool:
return True
x = 1
while flag():
x = 2
while flag():
x = 3
if flag():
break
else:
x = 4
if flag():
break
else:
x = 5
reveal_type(x) # revealed: Literal[3, 4, 5]
```

View File

@@ -256,7 +256,7 @@ class O: ...
class X(O): ...
class Y(O): ...
if bool():
if returns_bool():
foo = Y
else:
foo = object

View File

@@ -0,0 +1,64 @@
# Consolidating narrowed types after if statement
## After if-else statements, narrowing has no effect if the variable is not mutated in any branch
```py
def optional_int() -> int | None: ...
x = optional_int()
if x is None:
pass
else:
pass
reveal_type(x) # revealed: int | None
```
## Narrowing can have a persistent effect if the variable is mutated in one branch
```py
def optional_int() -> int | None: ...
x = optional_int()
if x is None:
x = 10
else:
pass
reveal_type(x) # revealed: int
```
## An if statement without an explicit `else` branch is equivalent to one with a no-op `else` branch
```py
def optional_int() -> int | None: ...
x = optional_int()
y = optional_int()
if x is None:
x = 0
if y is None:
pass
reveal_type(x) # revealed: int
reveal_type(y) # revealed: int | None
```
## An if-elif without an explicit else branch is equivalent to one with an empty else branch
```py
def optional_int() -> int | None: ...
x = optional_int()
if x is None:
x = 0
elif x > 50:
x = 50
reveal_type(x) # revealed: int
```

View File

@@ -0,0 +1,303 @@
# Statically-known branches
## Always false
### If
```py
x = 1
if False:
x = 2
reveal_type(x) # revealed: Literal[1]
```
### Else
```py
x = 1
if True:
pass
else:
x = 2
reveal_type(x) # revealed: Literal[1]
```
## Always true
### If
```py
x = 1
if True:
x = 2
reveal_type(x) # revealed: Literal[2]
```
### Else
```py
x = 1
if False:
pass
else:
x = 2
reveal_type(x) # revealed: Literal[2]
```
## Combination
```py
x = 1
if True:
x = 2
else:
x = 3
reveal_type(x) # revealed: Literal[2]
```
## Nested
```py path=nested_if_true_if_true.py
x = 1
if True:
if True:
x = 2
else:
x = 3
else:
x = 4
reveal_type(x) # revealed: Literal[2]
```
```py path=nested_if_true_if_false.py
x = 1
if True:
if False:
x = 2
else:
x = 3
else:
x = 4
reveal_type(x) # revealed: Literal[3]
```
```py path=nested_if_true_if_bool.py
def flag() -> bool: ...
x = 1
if True:
if flag():
x = 2
else:
x = 3
else:
x = 4
reveal_type(x) # revealed: Literal[2, 3]
```
```py path=nested_if_bool_if_true.py
def flag() -> bool: ...
x = 1
if flag():
if True:
x = 2
else:
x = 3
else:
x = 4
reveal_type(x) # revealed: Literal[2, 4]
```
```py path=nested_else_if_true.py
x = 1
if False:
x = 2
else:
if True:
x = 3
else:
x = 4
reveal_type(x) # revealed: Literal[3]
```
```py path=nested_else_if_false.py
x = 1
if False:
x = 2
else:
if False:
x = 3
else:
x = 4
reveal_type(x) # revealed: Literal[4]
```
```py path=nested_else_if_bool.py
def flag() -> bool: ...
x = 1
if False:
x = 2
else:
if flag():
x = 3
else:
x = 4
reveal_type(x) # revealed: Literal[3, 4]
```
## If-expressions
### Always true
```py
x = 1 if True else 2
reveal_type(x) # revealed: Literal[1]
```
### Always false
```py
x = 1 if False else 2
reveal_type(x) # revealed: Literal[2]
```
## Boolean expressions
### Always true
```py
(x := 1) == 1 or (x := 2)
reveal_type(x) # revealed: Literal[1]
```
### Always false
```py
(x := 1) == 0 or (x := 2)
reveal_type(x) # revealed: Literal[2]
```
## Conditional declarations
```py path=if_false.py
x: str
if False:
x: int
def f() -> None:
reveal_type(x) # revealed: str
```
```py path=if_true_else.py
x: str
if True:
pass
else:
x: int
def f() -> None:
reveal_type(x) # revealed: str
```
```py path=if_true.py
x: str
if True:
x: int
def f() -> None:
reveal_type(x) # revealed: int
```
```py path=if_false_else.py
x: str
if False:
pass
else:
x: int
def f() -> None:
reveal_type(x) # revealed: int
```
```py path=if_bool.py
def flag() -> bool: ...
x: str
if flag():
x: int
def f() -> None:
reveal_type(x) # revealed: str | int
```
## Conditionally defined functions
```py
def f() -> int: ...
def g() -> int: ...
if True:
def f() -> str: ...
else:
def g() -> str: ...
reveal_type(f()) # revealed: str
reveal_type(g()) # revealed: int
```
## Conditionally defined class attributes
```py
class C:
if True:
x: int = 1
else:
x: str = "a"
reveal_type(C.x) # revealed: int
```
## TODO
- declarations vs bindings => NoDefault: NoDefaultType
- conditional imports
- conditional class definitions
- compare with tests in if.md=>Statically known branches
- boundness
- TODO in `issubclass.md`

View File

@@ -21,10 +21,11 @@ reveal_type(Identity[0]) # revealed: str
## Class getitem union
```py
flag = True
def bool_instance() -> bool:
return True
class UnionClassGetItem:
if flag:
if bool_instance():
def __class_getitem__(cls, item: int) -> str:
return item
@@ -59,9 +60,10 @@ reveal_type(x[0]) # revealed: str | int
## Class getitem with unbound method union
```py
flag = True
def bool_instance() -> bool:
return True
if flag:
if bool_instance():
class Spam:
def __class_getitem__(self, x: int) -> str:
return "foo"
@@ -77,9 +79,10 @@ reveal_type(Spam[42])
## TODO: Class getitem non-class union
```py
flag = True
def bool_instance() -> bool:
return True
if flag:
if bool_instance():
class Eggs:
def __class_getitem__(self, x: int) -> str:
return "foo"

View File

@@ -30,10 +30,11 @@ reveal_type(Identity()[0]) # revealed: int
## Getitem union
```py
flag = True
def bool_instance() -> bool:
return True
class Identity:
if flag:
if bool_instance():
def __getitem__(self, index: int) -> int:
return index

View File

@@ -49,14 +49,14 @@ sometimes not:
```py
import sys
reveal_type(sys.version_info >= (3, 9, 1)) # revealed: bool
reveal_type(sys.version_info >= (3, 9, 1, "final", 0)) # revealed: bool
reveal_type(sys.version_info >= (3, 9, 1)) # revealed: Literal[True]
reveal_type(sys.version_info >= (3, 9, 1, "final", 0)) # revealed: Literal[True]
# TODO: While this won't fail at runtime, the user has probably made a mistake
# if they're comparing a tuple of length >5 with `sys.version_info`
# (`sys.version_info` is a tuple of length 5). It might be worth
# emitting a lint diagnostic of some kind warning them about the probable error?
reveal_type(sys.version_info >= (3, 9, 1, "final", 0, 5)) # revealed: bool
reveal_type(sys.version_info >= (3, 9, 1, "final", 0, 5)) # revealed: Literal[True]
reveal_type(sys.version_info == (3, 8, 1, "finallllll", 0)) # revealed: Literal[False]
```
@@ -102,8 +102,8 @@ The fields of `sys.version_info` can be accessed by name:
import sys
reveal_type(sys.version_info.major >= 3) # revealed: Literal[True]
reveal_type(sys.version_info.minor >= 9) # revealed: Literal[True]
reveal_type(sys.version_info.minor >= 10) # revealed: Literal[False]
reveal_type(sys.version_info.minor >= 13) # revealed: Literal[True]
reveal_type(sys.version_info.minor >= 14) # revealed: Literal[False]
```
But the `micro`, `releaselevel` and `serial` fields are inferred as `@Todo` until we support
@@ -125,14 +125,14 @@ The fields of `sys.version_info` can be accessed by index or by slice:
import sys
reveal_type(sys.version_info[0] < 3) # revealed: Literal[False]
reveal_type(sys.version_info[1] > 9) # revealed: Literal[False]
reveal_type(sys.version_info[1] > 13) # revealed: Literal[False]
# revealed: tuple[Literal[3], Literal[9], int, Literal["alpha", "beta", "candidate", "final"], int]
# revealed: tuple[Literal[3], Literal[13], int, Literal["alpha", "beta", "candidate", "final"], int]
reveal_type(sys.version_info[:5])
reveal_type(sys.version_info[:2] >= (3, 9)) # revealed: Literal[True]
reveal_type(sys.version_info[0:2] >= (3, 10)) # revealed: Literal[False]
reveal_type(sys.version_info[:3] >= (3, 10, 1)) # revealed: Literal[False]
reveal_type(sys.version_info[:2] >= (3, 13)) # revealed: Literal[True]
reveal_type(sys.version_info[0:2] >= (3, 14)) # revealed: Literal[False]
reveal_type(sys.version_info[:3] >= (3, 14, 1)) # revealed: Literal[False]
reveal_type(sys.version_info[3] == "final") # revealed: bool
reveal_type(sys.version_info[3] == "finalllllll") # revealed: Literal[False]
```

View File

@@ -39,7 +39,7 @@ impl PythonVersion {
impl Default for PythonVersion {
fn default() -> Self {
Self::PY39
Self::PY313 // TODO: temporarily changed to 3.13 to activate all sys.version_info branches
}
}

View File

@@ -1229,4 +1229,32 @@ match 1:
assert!(matches!(binding.kind(&db), DefinitionKind::For(_)));
}
#[test]
#[ignore]
fn if_statement() {
let TestCase { db, file } = test_case(
"
x = False
if True:
x: bool
",
);
let index = semantic_index(&db, file);
// let global_table = index.symbol_table(FileScopeId::global());
let use_def = index.use_def_map(FileScopeId::global());
// use_def
use_def.print(&db);
assert!(false);
// let binding = use_def
// .first_public_binding(global_table.symbol_id_by_name(name).expect("symbol exists"))
// .expect("Expected with item definition for {name}");
// assert!(matches!(binding.kind(&db), DefinitionKind::WithItem(_)));
}
}

View File

@@ -23,7 +23,7 @@ use crate::semantic_index::symbol::{
FileScopeId, NodeWithScopeKey, NodeWithScopeRef, Scope, ScopeId, ScopedSymbolId,
SymbolTableBuilder,
};
use crate::semantic_index::use_def::{FlowSnapshot, UseDefMapBuilder};
use crate::semantic_index::use_def::{ActiveConstraintsSnapshot, FlowSnapshot, UseDefMapBuilder};
use crate::semantic_index::SemanticIndex;
use crate::unpack::Unpack;
use crate::Db;
@@ -36,12 +36,25 @@ use super::definition::{
mod except_handlers;
/// Are we in a state where a `break` statement is allowed?
#[derive(Clone, Copy, Debug)]
enum LoopState {
InLoop,
NotInLoop,
}
impl LoopState {
fn is_inside(self) -> bool {
matches!(self, LoopState::InLoop)
}
}
pub(super) struct SemanticIndexBuilder<'db> {
// Builder state
db: &'db dyn Db,
file: File,
module: &'db ParsedModule,
scope_stack: Vec<FileScopeId>,
scope_stack: Vec<(FileScopeId, LoopState)>,
/// The assignments we're currently visiting, with
/// the most recent visit at the end of the Vec
current_assignments: Vec<CurrentAssignment<'db>>,
@@ -103,9 +116,24 @@ impl<'db> SemanticIndexBuilder<'db> {
*self
.scope_stack
.last()
.map(|(scope, _)| scope)
.expect("Always to have a root scope")
}
fn loop_state(&self) -> LoopState {
self.scope_stack
.last()
.expect("Always to have a root scope")
.1
}
fn set_inside_loop(&mut self, state: LoopState) {
self.scope_stack
.last_mut()
.expect("Always to have a root scope")
.1 = state;
}
fn push_scope(&mut self, node: NodeWithScopeRef) {
let parent = self.current_scope();
self.push_scope_with_parent(node, Some(parent));
@@ -136,11 +164,11 @@ impl<'db> SemanticIndexBuilder<'db> {
debug_assert_eq!(ast_id_scope, file_scope_id);
self.scope_stack.push(file_scope_id);
self.scope_stack.push((file_scope_id, LoopState::NotInLoop));
}
fn pop_scope(&mut self) -> FileScopeId {
let id = self.scope_stack.pop().expect("Root scope to be present");
let (id, _) = self.scope_stack.pop().expect("Root scope to be present");
let children_end = self.scopes.next_index();
let scope = &mut self.scopes[id];
scope.descendents = scope.descendents.start..children_end;
@@ -172,12 +200,20 @@ impl<'db> SemanticIndexBuilder<'db> {
self.current_use_def_map().snapshot()
}
fn flow_restore(&mut self, state: FlowSnapshot) {
self.current_use_def_map_mut().restore(state);
fn constraints_snapshot(&self) -> ActiveConstraintsSnapshot {
self.current_use_def_map().constraints_snapshot()
}
fn flow_merge(&mut self, state: FlowSnapshot) {
fn flow_restore(&mut self, state: FlowSnapshot, active_constraints: ActiveConstraintsSnapshot) {
self.current_use_def_map_mut().restore(state);
self.current_use_def_map_mut()
.restore_constraints(active_constraints);
}
fn flow_merge(&mut self, state: FlowSnapshot, active_constraints: ActiveConstraintsSnapshot) {
self.current_use_def_map_mut().merge(state);
self.current_use_def_map_mut()
.restore_constraints(active_constraints);
}
fn add_symbol(&mut self, name: Name) -> ScopedSymbolId {
@@ -737,37 +773,44 @@ where
ast::Stmt::If(node) => {
self.visit_expr(&node.test);
let pre_if = self.flow_snapshot();
let pre_if_constraints = self.constraints_snapshot();
let constraint = self.record_expression_constraint(&node.test);
let mut constraints = vec![constraint];
self.visit_body(&node.body);
let mut post_clauses: Vec<FlowSnapshot> = vec![];
for clause in &node.elif_else_clauses {
let elif_else_clauses = node
.elif_else_clauses
.iter()
.map(|clause| (clause.test.as_ref(), clause.body.as_slice()));
let has_else = node
.elif_else_clauses
.last()
.is_some_and(|clause| clause.test.is_none());
let elif_else_clauses = elif_else_clauses.chain(if has_else {
// if there's an `else` clause already, we don't need to add another
None
} else {
// if there's no `else` branch, we should add a no-op `else` branch
Some((None, Default::default()))
});
for (clause_test, clause_body) in elif_else_clauses {
// snapshot after every block except the last; the last one will just become
// the state that we merge the other snapshots into
post_clauses.push(self.flow_snapshot());
// we can only take an elif/else branch if none of the previous ones were
// taken, so the block entry state is always `pre_if`
self.flow_restore(pre_if.clone());
self.flow_restore(pre_if.clone(), pre_if_constraints.clone());
for constraint in &constraints {
self.record_negated_constraint(*constraint);
}
if let Some(elif_test) = &clause.test {
if let Some(elif_test) = clause_test {
self.visit_expr(elif_test);
constraints.push(self.record_expression_constraint(elif_test));
}
self.visit_body(&clause.body);
self.visit_body(clause_body);
}
for post_clause_state in post_clauses {
self.flow_merge(post_clause_state);
}
let has_else = node
.elif_else_clauses
.last()
.is_some_and(|clause| clause.test.is_none());
if !has_else {
// if there's no else clause, then it's possible we took none of the branches,
// and the pre_if state can reach here
self.flow_merge(pre_if);
self.flow_merge(post_clause_state, pre_if_constraints.clone());
}
}
ast::Stmt::While(ast::StmtWhile {
@@ -779,13 +822,17 @@ where
self.visit_expr(test);
let pre_loop = self.flow_snapshot();
let pre_loop_constraints = self.constraints_snapshot();
// Save aside any break states from an outer loop
let saved_break_states = std::mem::take(&mut self.loop_break_states);
// TODO: definitions created inside the body should be fully visible
// to other statements/expressions inside the body --Alex/Carl
let outer_loop_state = self.loop_state();
self.set_inside_loop(LoopState::InLoop);
self.visit_body(body);
self.set_inside_loop(outer_loop_state);
// Get the break states from the body of this loop, and restore the saved outer
// ones.
@@ -794,13 +841,13 @@ where
// We may execute the `else` clause without ever executing the body, so merge in
// the pre-loop state before visiting `else`.
self.flow_merge(pre_loop);
self.flow_merge(pre_loop, pre_loop_constraints.clone());
self.visit_body(orelse);
// Breaking out of a while loop bypasses the `else` clause, so merge in the break
// states after visiting `else`.
for break_state in break_states {
self.flow_merge(break_state);
self.flow_merge(break_state, pre_loop_constraints.clone()); // TODO?
}
}
ast::Stmt::With(ast::StmtWith {
@@ -824,7 +871,9 @@ where
self.visit_body(body);
}
ast::Stmt::Break(_) => {
self.loop_break_states.push(self.flow_snapshot());
if self.loop_state().is_inside() {
self.loop_break_states.push(self.flow_snapshot());
}
}
ast::Stmt::For(
@@ -841,6 +890,7 @@ where
self.visit_expr(iter);
let pre_loop = self.flow_snapshot();
let pre_loop_constraints = self.constraints_snapshot();
let saved_break_states = std::mem::take(&mut self.loop_break_states);
debug_assert_eq!(&self.current_assignments, &[]);
@@ -851,20 +901,23 @@ where
// TODO: Definitions created by loop variables
// (and definitions created inside the body)
// are fully visible to other statements/expressions inside the body --Alex/Carl
let outer_loop_state = self.loop_state();
self.set_inside_loop(LoopState::InLoop);
self.visit_body(body);
self.set_inside_loop(outer_loop_state);
let break_states =
std::mem::replace(&mut self.loop_break_states, saved_break_states);
// We may execute the `else` clause without ever executing the body, so merge in
// the pre-loop state before visiting `else`.
self.flow_merge(pre_loop);
self.flow_merge(pre_loop, pre_loop_constraints.clone());
self.visit_body(orelse);
// Breaking out of a `for` loop bypasses the `else` clause, so merge in the break
// states after visiting `else`.
for break_state in break_states {
self.flow_merge(break_state);
self.flow_merge(break_state, pre_loop_constraints.clone());
}
}
ast::Stmt::Match(ast::StmtMatch {
@@ -876,6 +929,7 @@ where
self.visit_expr(subject);
let after_subject = self.flow_snapshot();
let after_subject_cs = self.constraints_snapshot();
let Some((first, remaining)) = cases.split_first() else {
return;
};
@@ -885,18 +939,18 @@ where
let mut post_case_snapshots = vec![];
for case in remaining {
post_case_snapshots.push(self.flow_snapshot());
self.flow_restore(after_subject.clone());
self.flow_restore(after_subject.clone(), after_subject_cs.clone());
self.add_pattern_constraint(subject, &case.pattern);
self.visit_match_case(case);
}
for post_clause_state in post_case_snapshots {
self.flow_merge(post_clause_state);
self.flow_merge(post_clause_state, after_subject_cs.clone());
}
if !cases
.last()
.is_some_and(|case| case.guard.is_none() && case.pattern.is_wildcard())
{
self.flow_merge(after_subject);
self.flow_merge(after_subject, after_subject_cs.clone());
}
}
ast::Stmt::Try(ast::StmtTry {
@@ -914,6 +968,7 @@ where
// We will merge this state with all of the intermediate
// states during the `try` block before visiting those suites.
let pre_try_block_state = self.flow_snapshot();
let pre_try_block_constraints = self.constraints_snapshot();
self.try_node_context_stack_manager.push_context();
@@ -934,14 +989,17 @@ where
// as there necessarily must have been 0 `except` blocks executed
// if we hit the `else` block.
let post_try_block_state = self.flow_snapshot();
let post_try_block_constraints = self.constraints_snapshot();
// Prepare for visiting the `except` block(s)
self.flow_restore(pre_try_block_state);
self.flow_restore(pre_try_block_state, pre_try_block_constraints.clone());
for state in try_block_snapshots {
self.flow_merge(state);
self.flow_merge(state, pre_try_block_constraints.clone());
// TODO?
}
let pre_except_state = self.flow_snapshot();
let pre_except_constraints = self.constraints_snapshot();
let num_handlers = handlers.len();
for (i, except_handler) in handlers.iter().enumerate() {
@@ -980,19 +1038,22 @@ where
// as we'll immediately call `self.flow_restore()` to a different state
// as soon as this loop over the handlers terminates.
if i < (num_handlers - 1) {
self.flow_restore(pre_except_state.clone());
self.flow_restore(
pre_except_state.clone(),
pre_except_constraints.clone(),
);
}
}
// If we get to the `else` block, we know that 0 of the `except` blocks can have been executed,
// and the entire `try` block must have been executed:
self.flow_restore(post_try_block_state);
self.flow_restore(post_try_block_state, post_try_block_constraints);
}
self.visit_body(orelse);
for post_except_state in post_except_states {
self.flow_merge(post_except_state);
self.flow_merge(post_except_state, pre_try_block_constraints.clone());
}
// TODO: there's lots of complexity here that isn't yet handled by our model.
@@ -1149,19 +1210,17 @@ where
ast::Expr::If(ast::ExprIf {
body, test, orelse, ..
}) => {
// TODO detect statically known truthy or falsy test (via type inference, not naive
// AST inspection, so we can't simplify here, need to record test expression for
// later checking)
self.visit_expr(test);
let constraint = self.record_expression_constraint(test);
let pre_if = self.flow_snapshot();
let pre_if_constraints = self.constraints_snapshot();
let constraint = self.record_expression_constraint(test);
self.visit_expr(body);
let post_body = self.flow_snapshot();
self.flow_restore(pre_if);
self.flow_restore(pre_if, pre_if_constraints.clone());
self.record_negated_constraint(constraint);
self.visit_expr(orelse);
self.flow_merge(post_body);
self.flow_merge(post_body, pre_if_constraints);
}
ast::Expr::ListComp(
list_comprehension @ ast::ExprListComp {
@@ -1222,7 +1281,7 @@ where
// AST inspection, so we can't simplify here, need to record test expression for
// later checking)
let mut snapshots = vec![];
let pre_op_constraints = self.constraints_snapshot();
for (index, value) in values.iter().enumerate() {
self.visit_expr(value);
// In the last value we don't need to take a snapshot nor add a constraint
@@ -1237,7 +1296,7 @@ where
}
}
for snapshot in snapshots {
self.flow_merge(snapshot);
self.flow_merge(snapshot, pre_op_constraints.clone());
}
}
_ => {

View File

@@ -221,6 +221,8 @@
//! snapshot, and merging a snapshot into the current state. The logic using these methods lives in
//! [`SemanticIndexBuilder`](crate::semantic_index::builder::SemanticIndexBuilder), e.g. where it
//! visits a `StmtIf` node.
use std::collections::HashSet;
use self::symbol_state::{
BindingIdWithConstraintsIterator, ConstraintIdIterator, DeclarationIdIterator,
ScopedConstraintId, ScopedDefinitionId, SymbolBindings, SymbolDeclarations, SymbolState,
@@ -268,6 +270,109 @@ pub(crate) struct UseDefMap<'db> {
}
impl<'db> UseDefMap<'db> {
#[cfg(test)]
pub(crate) fn print(&self, db: &dyn crate::db::Db) {
use crate::semantic_index::constraint::ConstraintNode;
println!("all_definitions:");
println!("================");
for (id, d) in self.all_definitions.iter_enumerated() {
println!(
"{:?}: {:?} {:?} {:?}",
id,
d.category(db),
d.scope(db),
d.symbol(db),
);
println!(" {:?}", d.kind(db));
println!();
}
println!("all_constraints:");
println!("================");
for (id, c) in self.all_constraints.iter_enumerated() {
println!("{:?}: {:?}", id, c.node);
if let ConstraintNode::Expression(e) = c.node {
println!(" {:?}", e.node_ref(db));
}
}
println!();
println!("bindings_by_use:");
println!("================");
for (id, bindings) in self.bindings_by_use.iter_enumerated() {
println!("{:?}:", id);
for binding in bindings.iter() {
let definition = self.all_definitions[binding.definition];
let mut constraint_ids = binding.constraint_ids.peekable();
let mut active_constraint_ids =
binding.constraints_active_at_binding_ids.peekable();
println!(" * {:?}", definition);
if constraint_ids.peek().is_some() {
println!(" Constraints:");
for constraint_id in constraint_ids {
println!(" {:?}", self.all_constraints[constraint_id]);
}
} else {
println!(" No constraints");
}
println!();
if active_constraint_ids.peek().is_some() {
println!(" Active constraints at binding:");
for constraint_id in active_constraint_ids {
println!(" {:?}", self.all_constraints[constraint_id]);
}
} else {
println!(" No active constraints at binding");
}
}
}
println!();
println!("public_symbols:");
println!("================");
for (id, symbol) in self.public_symbols.iter_enumerated() {
println!("{:?}:", id);
println!(" * Bindings:");
for binding in symbol.bindings().iter() {
let definition = self.all_definitions[binding.definition];
let mut constraint_ids = binding.constraint_ids.peekable();
println!(" {:?}", definition);
if constraint_ids.peek().is_some() {
println!(" Constraints:");
for constraint_id in constraint_ids {
println!(" {:?}", self.all_constraints[constraint_id]);
}
} else {
println!(" No constraints");
}
}
println!(" * Declarations:");
for (declaration, _) in symbol.declarations().iter() {
let definition = self.all_definitions[declaration];
println!(" {:?}", definition);
}
println!();
}
println!();
println!();
}
pub(crate) fn bindings_at_use(
&self,
use_id: ScopedUseId,
@@ -352,6 +457,7 @@ impl<'db> UseDefMap<'db> {
) -> DeclarationsIterator<'a, 'db> {
DeclarationsIterator {
all_definitions: &self.all_definitions,
all_constraints: &self.all_constraints,
inner: declarations.iter(),
may_be_undeclared: declarations.may_be_undeclared(),
}
@@ -365,7 +471,7 @@ enum SymbolDefinitions {
Declarations(SymbolDeclarations),
}
#[derive(Debug)]
#[derive(Debug, Clone)]
pub(crate) struct BindingWithConstraintsIterator<'map, 'db> {
all_definitions: &'map IndexVec<ScopedDefinitionId, Definition<'db>>,
all_constraints: &'map IndexVec<ScopedConstraintId, Constraint<'db>>,
@@ -384,6 +490,10 @@ impl<'map, 'db> Iterator for BindingWithConstraintsIterator<'map, 'db> {
all_constraints: self.all_constraints,
constraint_ids: def_id_with_constraints.constraint_ids,
},
constraints_active_at_binding: ConstraintsIterator {
all_constraints: self.all_constraints,
constraint_ids: def_id_with_constraints.constraints_active_at_binding_ids,
},
})
}
}
@@ -393,8 +503,10 @@ impl std::iter::FusedIterator for BindingWithConstraintsIterator<'_, '_> {}
pub(crate) struct BindingWithConstraints<'map, 'db> {
pub(crate) binding: Definition<'db>,
pub(crate) constraints: ConstraintsIterator<'map, 'db>,
pub(crate) constraints_active_at_binding: ConstraintsIterator<'map, 'db>,
}
#[derive(Debug, Clone)]
pub(crate) struct ConstraintsIterator<'map, 'db> {
all_constraints: &'map IndexVec<ScopedConstraintId, Constraint<'db>>,
constraint_ids: ConstraintIdIterator<'map>,
@@ -414,6 +526,7 @@ impl std::iter::FusedIterator for ConstraintsIterator<'_, '_> {}
pub(crate) struct DeclarationsIterator<'map, 'db> {
all_definitions: &'map IndexVec<ScopedDefinitionId, Definition<'db>>,
all_constraints: &'map IndexVec<ScopedConstraintId, Constraint<'db>>,
inner: DeclarationIdIterator<'map>,
may_be_undeclared: bool,
}
@@ -425,10 +538,18 @@ impl DeclarationsIterator<'_, '_> {
}
impl<'map, 'db> Iterator for DeclarationsIterator<'map, 'db> {
type Item = Definition<'db>;
type Item = (Definition<'db>, ConstraintsIterator<'map, 'db>);
fn next(&mut self) -> Option<Self::Item> {
self.inner.next().map(|def_id| self.all_definitions[def_id])
self.inner.next().map(|(def_id, constraints)| {
(
self.all_definitions[def_id],
ConstraintsIterator {
all_constraints: self.all_constraints,
constraint_ids: constraints,
},
)
})
}
}
@@ -440,6 +561,9 @@ pub(super) struct FlowSnapshot {
symbol_states: IndexVec<ScopedSymbolId, SymbolState>,
}
#[derive(Clone, Debug)]
pub(super) struct ActiveConstraintsSnapshot(HashSet<ScopedConstraintId>);
#[derive(Debug, Default)]
pub(super) struct UseDefMapBuilder<'db> {
/// Append-only array of [`Definition`].
@@ -448,6 +572,8 @@ pub(super) struct UseDefMapBuilder<'db> {
/// Append-only array of [`Constraint`].
all_constraints: IndexVec<ScopedConstraintId, Constraint<'db>>,
active_constraints: HashSet<ScopedConstraintId>,
/// Live bindings at each so-far-recorded use.
bindings_by_use: IndexVec<ScopedUseId, SymbolBindings>,
@@ -471,7 +597,7 @@ impl<'db> UseDefMapBuilder<'db> {
binding,
SymbolDefinitions::Declarations(symbol_state.declarations().clone()),
);
symbol_state.record_binding(def_id);
symbol_state.record_binding(def_id, &self.active_constraints);
}
pub(super) fn record_constraint(&mut self, constraint: Constraint<'db>) {
@@ -479,6 +605,7 @@ impl<'db> UseDefMapBuilder<'db> {
for state in &mut self.symbol_states {
state.record_constraint(constraint_id);
}
self.active_constraints.insert(constraint_id);
}
pub(super) fn record_declaration(
@@ -492,7 +619,7 @@ impl<'db> UseDefMapBuilder<'db> {
declaration,
SymbolDefinitions::Bindings(symbol_state.bindings().clone()),
);
symbol_state.record_declaration(def_id);
symbol_state.record_declaration(def_id, &self.active_constraints);
}
pub(super) fn record_declaration_and_binding(
@@ -503,8 +630,8 @@ impl<'db> UseDefMapBuilder<'db> {
// We don't need to store anything in self.definitions_by_definition.
let def_id = self.all_definitions.push(definition);
let symbol_state = &mut self.symbol_states[symbol];
symbol_state.record_declaration(def_id);
symbol_state.record_binding(def_id);
symbol_state.record_declaration(def_id, &self.active_constraints);
symbol_state.record_binding(def_id, &self.active_constraints);
}
pub(super) fn record_use(&mut self, symbol: ScopedSymbolId, use_id: ScopedUseId) {
@@ -523,6 +650,10 @@ impl<'db> UseDefMapBuilder<'db> {
}
}
pub(super) fn constraints_snapshot(&self) -> ActiveConstraintsSnapshot {
ActiveConstraintsSnapshot(self.active_constraints.clone())
}
/// Restore the current builder symbols state to the given snapshot.
pub(super) fn restore(&mut self, snapshot: FlowSnapshot) {
// We never remove symbols from `symbol_states` (it's an IndexVec, and the symbol
@@ -541,6 +672,10 @@ impl<'db> UseDefMapBuilder<'db> {
.resize(num_symbols, SymbolState::undefined());
}
pub(super) fn restore_constraints(&mut self, snapshot: ActiveConstraintsSnapshot) {
self.active_constraints = snapshot.0;
}
/// Merge the given snapshot into the current state, reflecting that we might have taken either
/// path to get here. The new state for each symbol should include definitions from both the
/// prior state and the snapshot.

View File

@@ -122,7 +122,7 @@ impl<const B: usize> BitSet<B> {
}
/// Iterator over values in a [`BitSet`].
#[derive(Debug)]
#[derive(Debug, Clone)]
pub(super) struct BitSetIterator<'a, const B: usize> {
/// The blocks we are iterating over.
blocks: &'a [u64],

View File

@@ -43,6 +43,8 @@
//!
//! Tracking live declarations is simpler, since constraints are not involved, but otherwise very
//! similar to tracking live bindings.
use std::collections::HashSet;
use super::bitset::{BitSet, BitSetIterator};
use ruff_index::newtype_index;
use smallvec::SmallVec;
@@ -87,6 +89,8 @@ pub(super) struct SymbolDeclarations {
/// [`BitSet`]: which declarations (as [`ScopedDefinitionId`]) can reach the current location?
live_declarations: Declarations,
constraints_active_at_declaration: Constraints, // TODO: rename to constraints_active_at_declaration
/// Could the symbol be un-declared at this point?
may_be_undeclared: bool,
}
@@ -95,14 +99,27 @@ impl SymbolDeclarations {
fn undeclared() -> Self {
Self {
live_declarations: Declarations::default(),
constraints_active_at_declaration: Constraints::default(),
may_be_undeclared: true,
}
}
/// Record a newly-encountered declaration for this symbol.
fn record_declaration(&mut self, declaration_id: ScopedDefinitionId) {
fn record_declaration(
&mut self,
declaration_id: ScopedDefinitionId,
active_constraints: &HashSet<ScopedConstraintId>,
) {
self.live_declarations = Declarations::with(declaration_id.into());
self.may_be_undeclared = false;
// TODO: unify code with below
self.constraints_active_at_declaration = Constraints::with_capacity(1);
self.constraints_active_at_declaration
.push(BitSet::default());
for active_constraint_id in active_constraints {
self.constraints_active_at_declaration[0].insert(active_constraint_id.as_u32());
}
}
/// Add undeclared as a possibility for this symbol.
@@ -114,6 +131,7 @@ impl SymbolDeclarations {
pub(super) fn iter(&self) -> DeclarationIdIterator {
DeclarationIdIterator {
inner: self.live_declarations.iter(),
constraints_active_at_binding: self.constraints_active_at_declaration.iter(),
}
}
@@ -138,6 +156,8 @@ pub(super) struct SymbolBindings {
/// binding in `live_bindings`.
constraints: Constraints,
constraints_active_at_binding: Constraints,
/// Could the symbol be unbound at this point?
may_be_unbound: bool,
}
@@ -147,6 +167,7 @@ impl SymbolBindings {
Self {
live_bindings: Bindings::default(),
constraints: Constraints::default(),
constraints_active_at_binding: Constraints::default(),
may_be_unbound: true,
}
}
@@ -157,12 +178,21 @@ impl SymbolBindings {
}
/// Record a newly-encountered binding for this symbol.
pub(super) fn record_binding(&mut self, binding_id: ScopedDefinitionId) {
pub(super) fn record_binding(
&mut self,
binding_id: ScopedDefinitionId,
active_constraints: &HashSet<ScopedConstraintId>,
) {
// The new binding replaces all previous live bindings in this path, and has no
// constraints.
self.live_bindings = Bindings::with(binding_id.into());
self.constraints = Constraints::with_capacity(1);
self.constraints.push(BitSet::default());
self.constraints_active_at_binding = Constraints::with_capacity(1);
self.constraints_active_at_binding.push(BitSet::default());
for active_constraint_id in active_constraints {
self.constraints_active_at_binding[0].insert(active_constraint_id.as_u32());
}
self.may_be_unbound = false;
}
@@ -178,6 +208,7 @@ impl SymbolBindings {
BindingIdWithConstraintsIterator {
definitions: self.live_bindings.iter(),
constraints: self.constraints.iter(),
constraints_active_at_binding: self.constraints_active_at_binding.iter(),
}
}
@@ -207,8 +238,12 @@ impl SymbolState {
}
/// Record a newly-encountered binding for this symbol.
pub(super) fn record_binding(&mut self, binding_id: ScopedDefinitionId) {
self.bindings.record_binding(binding_id);
pub(super) fn record_binding(
&mut self,
binding_id: ScopedDefinitionId,
active_constraints: &HashSet<ScopedConstraintId>,
) {
self.bindings.record_binding(binding_id, active_constraints);
}
/// Add given constraint to all live bindings.
@@ -222,8 +257,13 @@ impl SymbolState {
}
/// Record a newly-encountered declaration of this symbol.
pub(super) fn record_declaration(&mut self, declaration_id: ScopedDefinitionId) {
self.declarations.record_declaration(declaration_id);
pub(super) fn record_declaration(
&mut self,
declaration_id: ScopedDefinitionId,
active_constraints: &HashSet<ScopedConstraintId>,
) {
self.declarations
.record_declaration(declaration_id, active_constraints);
}
/// Merge another [`SymbolState`] into this one.
@@ -232,24 +272,93 @@ impl SymbolState {
bindings: SymbolBindings {
live_bindings: Bindings::default(),
constraints: Constraints::default(),
constraints_active_at_binding: Constraints::default(), // TODO
may_be_unbound: self.bindings.may_be_unbound || b.bindings.may_be_unbound,
},
declarations: SymbolDeclarations {
live_declarations: self.declarations.live_declarations.clone(),
constraints_active_at_declaration: Constraints::default(), // TODO
may_be_undeclared: self.declarations.may_be_undeclared
|| b.declarations.may_be_undeclared,
},
};
// let mut constraints_active_at_binding = BitSet::default();
// for active_constraint_id in active_constraints.0 {
// constraints_active_at_binding.insert(active_constraint_id.as_u32());
// }
std::mem::swap(&mut a, self);
self.declarations
.live_declarations
.union(&b.declarations.live_declarations);
// self.declarations
// .live_declarations
// .union(&b.declarations.live_declarations);
let mut a_decls_iter = a.declarations.live_declarations.iter();
let mut b_decls_iter = b.declarations.live_declarations.iter();
let mut a_constraints_active_at_declaration_iter =
a.declarations.constraints_active_at_declaration.into_iter();
let mut b_constraints_active_at_declaration_iter =
b.declarations.constraints_active_at_declaration.into_iter();
let mut opt_a_decl: Option<u32> = a_decls_iter.next();
let mut opt_b_decl: Option<u32> = b_decls_iter.next();
let push = |decl,
constraints_active_at_declaration_iter: &mut ConstraintsIntoIterator,
merged: &mut Self| {
merged.declarations.live_declarations.insert(decl);
let constraints_active_at_binding = constraints_active_at_declaration_iter
.next()
.expect("declarations and constraints_active_at_binding length mismatch");
merged
.declarations
.constraints_active_at_declaration
.push(constraints_active_at_binding);
};
loop {
match (opt_a_decl, opt_b_decl) {
(Some(a_decl), Some(b_decl)) => match a_decl.cmp(&b_decl) {
std::cmp::Ordering::Less => {
push(a_decl, &mut a_constraints_active_at_declaration_iter, self);
opt_a_decl = a_decls_iter.next();
}
std::cmp::Ordering::Greater => {
push(b_decl, &mut b_constraints_active_at_declaration_iter, self);
opt_b_decl = b_decls_iter.next();
}
std::cmp::Ordering::Equal => {
push(a_decl, &mut b_constraints_active_at_declaration_iter, self);
self.declarations
.constraints_active_at_declaration
.last_mut()
.unwrap()
.intersect(&a_constraints_active_at_declaration_iter.next().unwrap());
opt_a_decl = a_decls_iter.next();
opt_b_decl = b_decls_iter.next();
}
},
(Some(a_decl), None) => {
push(a_decl, &mut a_constraints_active_at_declaration_iter, self);
opt_a_decl = a_decls_iter.next();
}
(None, Some(b_decl)) => {
push(b_decl, &mut b_constraints_active_at_declaration_iter, self);
opt_b_decl = b_decls_iter.next();
}
(None, None) => break,
}
}
let mut a_defs_iter = a.bindings.live_bindings.iter();
let mut b_defs_iter = b.bindings.live_bindings.iter();
let mut a_constraints_iter = a.bindings.constraints.into_iter();
let mut b_constraints_iter = b.bindings.constraints.into_iter();
let mut a_constraints_active_at_binding_iter =
a.bindings.constraints_active_at_binding.into_iter();
let mut b_constraints_active_at_binding_iter =
b.bindings.constraints_active_at_binding.into_iter();
let mut opt_a_def: Option<u32> = a_defs_iter.next();
let mut opt_b_def: Option<u32> = b_defs_iter.next();
@@ -261,7 +370,10 @@ impl SymbolState {
// path is irrelevant.
// Helper to push `def`, with constraints in `constraints_iter`, onto `self`.
let push = |def, constraints_iter: &mut ConstraintsIntoIterator, merged: &mut Self| {
let push = |def,
constraints_iter: &mut ConstraintsIntoIterator,
constraints_active_at_binding_iter: &mut ConstraintsIntoIterator,
merged: &mut Self| {
merged.bindings.live_bindings.insert(def);
// SAFETY: we only ever create SymbolState with either no definitions and no constraint
// bitsets (`::unbound`) or one definition and one constraint bitset (`::with`), and
@@ -271,7 +383,14 @@ impl SymbolState {
let constraints = constraints_iter
.next()
.expect("definitions and constraints length mismatch");
let constraints_active_at_binding = constraints_active_at_binding_iter
.next()
.expect("definitions and constraints_active_at_binding length mismatch");
merged.bindings.constraints.push(constraints);
merged
.bindings
.constraints_active_at_binding
.push(constraints_active_at_binding);
};
loop {
@@ -279,17 +398,32 @@ impl SymbolState {
(Some(a_def), Some(b_def)) => match a_def.cmp(&b_def) {
std::cmp::Ordering::Less => {
// Next definition ID is only in `a`, push it to `self` and advance `a`.
push(a_def, &mut a_constraints_iter, self);
push(
a_def,
&mut a_constraints_iter,
&mut a_constraints_active_at_binding_iter,
self,
);
opt_a_def = a_defs_iter.next();
}
std::cmp::Ordering::Greater => {
// Next definition ID is only in `b`, push it to `self` and advance `b`.
push(b_def, &mut b_constraints_iter, self);
push(
b_def,
&mut b_constraints_iter,
&mut b_constraints_active_at_binding_iter,
self,
);
opt_b_def = b_defs_iter.next();
}
std::cmp::Ordering::Equal => {
// Next definition is in both; push to `self` and intersect constraints.
push(a_def, &mut b_constraints_iter, self);
push(
a_def,
&mut b_constraints_iter,
&mut b_constraints_active_at_binding_iter,
self,
);
// SAFETY: we only ever create SymbolState with either no definitions and
// no constraint bitsets (`::unbound`) or one definition and one constraint
// bitset (`::with`), and `::merge` always pushes one definition and one
@@ -298,6 +432,11 @@ impl SymbolState {
let a_constraints = a_constraints_iter
.next()
.expect("definitions and constraints length mismatch");
// let _a_constraints_active_at_binding =
// a_constraints_active_at_binding_iter.next().expect(
// "definitions and constraints_active_at_binding length mismatch",
// ); // TODO: perform check that we see the same constraints in both paths
// If the same definition is visible through both paths, any constraint
// that applies on only one path is irrelevant to the resulting type from
// unioning the two paths, so we intersect the constraints.
@@ -306,18 +445,29 @@ impl SymbolState {
.last_mut()
.unwrap()
.intersect(&a_constraints);
opt_a_def = a_defs_iter.next();
opt_b_def = b_defs_iter.next();
}
},
(Some(a_def), None) => {
// We've exhausted `b`, just push the def from `a` and move on to the next.
push(a_def, &mut a_constraints_iter, self);
push(
a_def,
&mut a_constraints_iter,
&mut a_constraints_active_at_binding_iter,
self,
);
opt_a_def = a_defs_iter.next();
}
(None, Some(b_def)) => {
// We've exhausted `a`, just push the def from `b` and move on to the next.
push(b_def, &mut b_constraints_iter, self);
push(
b_def,
&mut b_constraints_iter,
&mut b_constraints_active_at_binding_iter,
self,
);
opt_b_def = b_defs_iter.next();
}
(None, None) => break,
@@ -353,26 +503,37 @@ impl Default for SymbolState {
pub(super) struct BindingIdWithConstraints<'a> {
pub(super) definition: ScopedDefinitionId,
pub(super) constraint_ids: ConstraintIdIterator<'a>,
pub(super) constraints_active_at_binding_ids: ConstraintIdIterator<'a>,
}
#[derive(Debug)]
#[derive(Debug, Clone)]
pub(super) struct BindingIdWithConstraintsIterator<'a> {
definitions: BindingsIterator<'a>,
constraints: ConstraintsIterator<'a>,
constraints_active_at_binding: ConstraintsIterator<'a>,
}
impl<'a> Iterator for BindingIdWithConstraintsIterator<'a> {
type Item = BindingIdWithConstraints<'a>;
fn next(&mut self) -> Option<Self::Item> {
match (self.definitions.next(), self.constraints.next()) {
(None, None) => None,
(Some(def), Some(constraints)) => Some(BindingIdWithConstraints {
definition: ScopedDefinitionId::from_u32(def),
constraint_ids: ConstraintIdIterator {
wrapped: constraints.iter(),
},
}),
match (
self.definitions.next(),
self.constraints.next(),
self.constraints_active_at_binding.next(),
) {
(None, None, None) => None,
(Some(def), Some(constraints), Some(constraints_active_at_binding)) => {
Some(BindingIdWithConstraints {
definition: ScopedDefinitionId::from_u32(def),
constraint_ids: ConstraintIdIterator {
wrapped: constraints.iter(),
},
constraints_active_at_binding_ids: ConstraintIdIterator {
wrapped: constraints_active_at_binding.iter(),
},
})
}
// SAFETY: see above.
_ => unreachable!("definitions and constraints length mismatch"),
}
@@ -381,7 +542,7 @@ impl<'a> Iterator for BindingIdWithConstraintsIterator<'a> {
impl std::iter::FusedIterator for BindingIdWithConstraintsIterator<'_> {}
#[derive(Debug)]
#[derive(Debug, Clone)]
pub(super) struct ConstraintIdIterator<'a> {
wrapped: BitSetIterator<'a, INLINE_CONSTRAINT_BLOCKS>,
}
@@ -399,13 +560,25 @@ impl std::iter::FusedIterator for ConstraintIdIterator<'_> {}
#[derive(Debug)]
pub(super) struct DeclarationIdIterator<'a> {
inner: DeclarationsIterator<'a>,
constraints_active_at_binding: ConstraintsIterator<'a>,
}
impl<'a> Iterator for DeclarationIdIterator<'a> {
type Item = ScopedDefinitionId;
type Item = (ScopedDefinitionId, ConstraintIdIterator<'a>);
fn next(&mut self) -> Option<Self::Item> {
self.inner.next().map(ScopedDefinitionId::from_u32)
// self.inner.next().map(ScopedDefinitionId::from_u32)
match (self.inner.next(), self.constraints_active_at_binding.next()) {
(None, None) => None,
(Some(declaration), Some(constraints_active_at_binding)) => Some((
ScopedDefinitionId::from_u32(declaration),
ConstraintIdIterator {
wrapped: constraints_active_at_binding.iter(),
},
)),
// SAFETY: see above.
_ => unreachable!("declarations and constraints_active_at_binding length mismatch"),
}
}
}
@@ -413,7 +586,7 @@ impl std::iter::FusedIterator for DeclarationIdIterator<'_> {}
#[cfg(test)]
mod tests {
use super::{ScopedConstraintId, ScopedDefinitionId, SymbolState};
use super::{ScopedConstraintId, SymbolState};
fn assert_bindings(symbol: &SymbolState, may_be_unbound: bool, expected: &[&str]) {
assert_eq!(symbol.may_be_unbound(), may_be_unbound);
@@ -445,7 +618,7 @@ mod tests {
let actual = symbol
.declarations()
.iter()
.map(ScopedDefinitionId::as_u32)
.map(|(d, _)| d.as_u32()) // TODO: constraints
.collect::<Vec<_>>();
assert_eq!(actual, expected);
}
@@ -457,76 +630,76 @@ mod tests {
assert_bindings(&sym, true, &[]);
}
#[test]
fn with() {
let mut sym = SymbolState::undefined();
sym.record_binding(ScopedDefinitionId::from_u32(0));
// #[test]
// fn with() {
// let mut sym = SymbolState::undefined();
// sym.record_binding(ScopedDefinitionId::from_u32(0));
assert_bindings(&sym, false, &["0<>"]);
}
// assert_bindings(&sym, false, &["0<>"]);
// }
#[test]
fn set_may_be_unbound() {
let mut sym = SymbolState::undefined();
sym.record_binding(ScopedDefinitionId::from_u32(0));
sym.set_may_be_unbound();
// #[test]
// fn set_may_be_unbound() {
// let mut sym = SymbolState::undefined();
// sym.record_binding(ScopedDefinitionId::from_u32(0));
// sym.set_may_be_unbound();
assert_bindings(&sym, true, &["0<>"]);
}
// assert_bindings(&sym, true, &["0<>"]);
// }
#[test]
fn record_constraint() {
let mut sym = SymbolState::undefined();
sym.record_binding(ScopedDefinitionId::from_u32(0));
sym.record_constraint(ScopedConstraintId::from_u32(0));
// #[test]
// fn record_constraint() {
// let mut sym = SymbolState::undefined();
// sym.record_binding(ScopedDefinitionId::from_u32(0));
// sym.record_constraint(ScopedConstraintId::from_u32(0));
assert_bindings(&sym, false, &["0<0>"]);
}
// assert_bindings(&sym, false, &["0<0>"]);
// }
#[test]
fn merge() {
// merging the same definition with the same constraint keeps the constraint
let mut sym0a = SymbolState::undefined();
sym0a.record_binding(ScopedDefinitionId::from_u32(0));
sym0a.record_constraint(ScopedConstraintId::from_u32(0));
// #[test]
// fn merge() {
// // merging the same definition with the same constraint keeps the constraint
// let mut sym0a = SymbolState::undefined();
// sym0a.record_binding(ScopedDefinitionId::from_u32(0));
// sym0a.record_constraint(ScopedConstraintId::from_u32(0));
let mut sym0b = SymbolState::undefined();
sym0b.record_binding(ScopedDefinitionId::from_u32(0));
sym0b.record_constraint(ScopedConstraintId::from_u32(0));
// let mut sym0b = SymbolState::undefined();
// sym0b.record_binding(ScopedDefinitionId::from_u32(0));
// sym0b.record_constraint(ScopedConstraintId::from_u32(0));
sym0a.merge(sym0b);
let mut sym0 = sym0a;
assert_bindings(&sym0, false, &["0<0>"]);
// sym0a.merge(sym0b);
// let mut sym0 = sym0a;
// assert_bindings(&sym0, false, &["0<0>"]);
// merging the same definition with differing constraints drops all constraints
let mut sym1a = SymbolState::undefined();
sym1a.record_binding(ScopedDefinitionId::from_u32(1));
sym1a.record_constraint(ScopedConstraintId::from_u32(1));
// // merging the same definition with differing constraints drops all constraints
// let mut sym1a = SymbolState::undefined();
// sym1a.record_binding(ScopedDefinitionId::from_u32(1));
// sym1a.record_constraint(ScopedConstraintId::from_u32(1));
let mut sym1b = SymbolState::undefined();
sym1b.record_binding(ScopedDefinitionId::from_u32(1));
sym1b.record_constraint(ScopedConstraintId::from_u32(2));
// let mut sym1b = SymbolState::undefined();
// sym1b.record_binding(ScopedDefinitionId::from_u32(1));
// sym1b.record_constraint(ScopedConstraintId::from_u32(2));
sym1a.merge(sym1b);
let sym1 = sym1a;
assert_bindings(&sym1, false, &["1<>"]);
// sym1a.merge(sym1b);
// let sym1 = sym1a;
// assert_bindings(&sym1, false, &["1<>"]);
// merging a constrained definition with unbound keeps both
let mut sym2a = SymbolState::undefined();
sym2a.record_binding(ScopedDefinitionId::from_u32(2));
sym2a.record_constraint(ScopedConstraintId::from_u32(3));
// // merging a constrained definition with unbound keeps both
// let mut sym2a = SymbolState::undefined();
// sym2a.record_binding(ScopedDefinitionId::from_u32(2));
// sym2a.record_constraint(ScopedConstraintId::from_u32(3));
let sym2b = SymbolState::undefined();
// let sym2b = SymbolState::undefined();
sym2a.merge(sym2b);
let sym2 = sym2a;
assert_bindings(&sym2, true, &["2<3>"]);
// sym2a.merge(sym2b);
// let sym2 = sym2a;
// assert_bindings(&sym2, true, &["2<3>"]);
// merging different definitions keeps them each with their existing constraints
sym0.merge(sym2);
let sym = sym0;
assert_bindings(&sym, true, &["0<0>", "2<3>"]);
}
// // merging different definitions keeps them each with their existing constraints
// sym0.merge(sym2);
// let sym = sym0;
// assert_bindings(&sym, true, &["0<0>", "2<3>"]);
// }
#[test]
fn no_declaration() {
@@ -535,54 +708,54 @@ mod tests {
assert_declarations(&sym, true, &[]);
}
#[test]
fn record_declaration() {
let mut sym = SymbolState::undefined();
sym.record_declaration(ScopedDefinitionId::from_u32(1));
// #[test]
// fn record_declaration() {
// let mut sym = SymbolState::undefined();
// sym.record_declaration(ScopedDefinitionId::from_u32(1));
assert_declarations(&sym, false, &[1]);
}
// assert_declarations(&sym, false, &[1]);
// }
#[test]
fn record_declaration_override() {
let mut sym = SymbolState::undefined();
sym.record_declaration(ScopedDefinitionId::from_u32(1));
sym.record_declaration(ScopedDefinitionId::from_u32(2));
// #[test]
// fn record_declaration_override() {
// let mut sym = SymbolState::undefined();
// sym.record_declaration(ScopedDefinitionId::from_u32(1));
// sym.record_declaration(ScopedDefinitionId::from_u32(2));
assert_declarations(&sym, false, &[2]);
}
// assert_declarations(&sym, false, &[2]);
// }
#[test]
fn record_declaration_merge() {
let mut sym = SymbolState::undefined();
sym.record_declaration(ScopedDefinitionId::from_u32(1));
// #[test]
// fn record_declaration_merge() {
// let mut sym = SymbolState::undefined();
// sym.record_declaration(ScopedDefinitionId::from_u32(1));
let mut sym2 = SymbolState::undefined();
sym2.record_declaration(ScopedDefinitionId::from_u32(2));
// let mut sym2 = SymbolState::undefined();
// sym2.record_declaration(ScopedDefinitionId::from_u32(2));
sym.merge(sym2);
// sym.merge(sym2);
assert_declarations(&sym, false, &[1, 2]);
}
// assert_declarations(&sym, false, &[1, 2]);
// }
#[test]
fn record_declaration_merge_partial_undeclared() {
let mut sym = SymbolState::undefined();
sym.record_declaration(ScopedDefinitionId::from_u32(1));
// #[test]
// fn record_declaration_merge_partial_undeclared() {
// let mut sym = SymbolState::undefined();
// sym.record_declaration(ScopedDefinitionId::from_u32(1));
let sym2 = SymbolState::undefined();
// let sym2 = SymbolState::undefined();
sym.merge(sym2);
// sym.merge(sym2);
assert_declarations(&sym, true, &[1]);
}
// assert_declarations(&sym, true, &[1]);
// }
#[test]
fn set_may_be_undeclared() {
let mut sym = SymbolState::undefined();
sym.record_declaration(ScopedDefinitionId::from_u32(0));
sym.set_may_be_undeclared();
// #[test]
// fn set_may_be_undeclared() {
// let mut sym = SymbolState::undefined();
// sym.record_declaration(ScopedDefinitionId::from_u32(0));
// sym.set_may_be_undeclared();
assert_declarations(&sym, true, &[0]);
}
// assert_declarations(&sym, true, &[0]);
// }
}

View File

@@ -15,6 +15,7 @@ pub(crate) use self::infer::{
pub(crate) use self::signatures::Signature;
use crate::module_resolver::file_to_module;
use crate::semantic_index::ast_ids::HasScopedExpressionId;
use crate::semantic_index::constraint::ConstraintNode;
use crate::semantic_index::definition::Definition;
use crate::semantic_index::symbol::{self as symbol, ScopeId, ScopedSymbolId};
use crate::semantic_index::{
@@ -222,6 +223,12 @@ fn definition_expression_ty<'db>(
}
}
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
enum UnconditionallyVisible {
Yes,
No,
}
/// Infer the combined type of an iterator of bindings.
///
/// Will return a union if there is more than one binding.
@@ -229,29 +236,88 @@ fn bindings_ty<'db>(
db: &'db dyn Db,
bindings_with_constraints: BindingWithConstraintsIterator<'_, 'db>,
) -> Option<Type<'db>> {
let mut def_types = bindings_with_constraints.map(
let def_types = bindings_with_constraints.map(
|BindingWithConstraints {
binding,
constraints,
constraints_active_at_binding,
}| {
let mut constraint_tys = constraints
.filter_map(|constraint| narrowing_constraint(db, constraint, binding))
.peekable();
let test_expr_tys = || {
constraints_active_at_binding.clone().map(|c| {
let ty = if let ConstraintNode::Expression(test_expr) = c.node {
let inference = infer_expression_types(db, test_expr);
let scope = test_expr.scope(db);
inference
.expression_ty(test_expr.node_ref(db).scoped_expression_id(db, scope))
} else {
// TODO: handle other constraint nodes
todo_type!()
};
let binding_ty = binding_ty(db, binding);
if constraint_tys.peek().is_some() {
constraint_tys
.fold(
IntersectionBuilder::new(db).add_positive(binding_ty),
IntersectionBuilder::add_positive,
)
.build()
(c, ty)
})
};
if test_expr_tys().any(|(c, test_expr_ty)| {
if c.is_positive {
test_expr_ty.bool(db).is_always_false()
} else {
test_expr_ty.bool(db).is_always_true()
}
}) {
// TODO: do we need to call binding_ty(…) even if we don't need the result?
(Type::Never, UnconditionallyVisible::No)
} else {
binding_ty
let mut test_expr_tys_iter = test_expr_tys().peekable();
let unconditionally_visible = if test_expr_tys_iter.peek().is_some()
&& test_expr_tys_iter.all(|(c, test_expr_ty)| {
if c.is_positive {
test_expr_ty.bool(db).is_always_true()
} else {
test_expr_ty.bool(db).is_always_false()
}
}) {
UnconditionallyVisible::Yes
} else {
UnconditionallyVisible::No
};
let mut constraint_tys = constraints
.filter_map(|constraint| narrowing_constraint(db, constraint, binding))
.peekable();
let binding_ty = binding_ty(db, binding);
if constraint_tys.peek().is_some() {
let intersection_ty = constraint_tys
.fold(
IntersectionBuilder::new(db).add_positive(binding_ty),
IntersectionBuilder::add_positive,
)
.build();
(intersection_ty, unconditionally_visible)
} else {
(binding_ty, unconditionally_visible)
}
}
},
);
// TODO: get rid of all the collects and clean up, obviously
let def_types: Vec<_> = def_types.collect();
// shrink the vector to only include everything from the last unconditionally visible binding
let def_types: Vec<_> = def_types
.iter()
.rev()
.take_while_inclusive(|(_, unconditionally_visible)| {
*unconditionally_visible != UnconditionallyVisible::Yes
})
.map(|(ty, _)| *ty)
.collect();
let mut def_types = def_types.into_iter().rev();
if let Some(first) = def_types.next() {
if let Some(second) = def_types.next() {
Some(UnionType::from_elements(
@@ -287,7 +353,63 @@ fn declarations_ty<'db>(
declarations: DeclarationsIterator<'_, 'db>,
undeclared_ty: Option<Type<'db>>,
) -> DeclaredTypeResult<'db> {
let decl_types = declarations.map(|declaration| declaration_ty(db, declaration));
let decl_types = declarations.map(|(declaration, constraints_active_at_declaration)| {
let test_expr_tys = || {
constraints_active_at_declaration.clone().map(|c| {
let ty = if let ConstraintNode::Expression(test_expr) = c.node {
let inference = infer_expression_types(db, test_expr);
let scope = test_expr.scope(db);
inference.expression_ty(test_expr.node_ref(db).scoped_expression_id(db, scope))
} else {
// TODO: handle other constraint nodes
todo_type!()
};
(c, ty)
})
};
if test_expr_tys().any(|(c, test_expr_ty)| {
if c.is_positive {
test_expr_ty.bool(db).is_always_false()
} else {
test_expr_ty.bool(db).is_always_true()
}
}) {
(Type::Never, UnconditionallyVisible::No)
} else {
let mut test_expr_tys_iter = test_expr_tys().peekable();
if test_expr_tys_iter.peek().is_some()
&& test_expr_tys_iter.all(|(c, test_expr_ty)| {
if c.is_positive {
test_expr_ty.bool(db).is_always_true()
} else {
test_expr_ty.bool(db).is_always_false()
}
})
{
(declaration_ty(db, declaration), UnconditionallyVisible::Yes)
} else {
(declaration_ty(db, declaration), UnconditionallyVisible::No)
}
}
});
// TODO: get rid of all the collects and clean up, obviously
let decl_types: Vec<_> = decl_types.collect();
// shrink the vector to only include everything from the last unconditionally visible binding
let decl_types: Vec<_> = decl_types
.iter()
.rev()
.take_while_inclusive(|(_, unconditionally_visible)| {
*unconditionally_visible != UnconditionallyVisible::Yes
})
.map(|(ty, _)| *ty)
.collect();
let decl_types = decl_types.into_iter().rev();
let mut all_types = undeclared_ty.into_iter().chain(decl_types);
@@ -537,6 +659,19 @@ impl<'db> Type<'db> {
.expect("Expected a Type::IntLiteral variant")
}
pub const fn into_known_instance(self) -> Option<KnownInstanceType<'db>> {
match self {
Type::KnownInstance(known_instance) => Some(known_instance),
_ => None,
}
}
#[track_caller]
pub fn expect_known_instance(self) -> KnownInstanceType<'db> {
self.into_known_instance()
.expect("Expected a Type::KnownInstance variant")
}
pub const fn is_boolean_literal(&self) -> bool {
matches!(self, Type::BooleanLiteral(..))
}
@@ -755,23 +890,7 @@ impl<'db> Type<'db> {
// TODO: Once we have support for final classes, we can establish that
// `Type::SubclassOf('FinalClass')` is equivalent to `Type::ClassLiteral('FinalClass')`.
// TODO: The following is a workaround that is required to unify the two different versions
// of `NoneType` and `NoDefaultType` in typeshed. This should not be required anymore once
// we understand `sys.version_info` branches.
self == other
|| matches!((self, other), (Type::Todo(_), Type::Todo(_)))
|| matches!((self, other),
(
Type::Instance(InstanceType { class: self_class }),
Type::Instance(InstanceType { class: target_class })
)
if {
let self_known = self_class.known(db);
matches!(self_known, Some(KnownClass::NoneType | KnownClass::NoDefaultType))
&& self_known == target_class.known(db)
}
)
self == other || matches!((self, other), (Type::Todo(_), Type::Todo(_)))
}
/// Return true if this type and `other` have no common elements.
@@ -1540,6 +1659,9 @@ impl<'db> Type<'db> {
// TODO map this to a new `Type::TypeVar` variant
Type::KnownInstance(KnownInstanceType::TypeVar(_)) => *self,
Type::KnownInstance(KnownInstanceType::TypeAliasType(alias)) => alias.value_ty(db),
Type::KnownInstance(KnownInstanceType::Never | KnownInstanceType::NoReturn) => {
Type::Never
}
_ => todo_type!(),
}
}
@@ -1747,13 +1869,13 @@ impl<'db> KnownClass {
}
pub fn to_class_literal(self, db: &'db dyn Db) -> Type<'db> {
core_module_symbol(db, self.canonical_module(), self.as_str())
core_module_symbol(db, self.canonical_module(db), self.as_str())
.ignore_possibly_unbound()
.unwrap_or(Type::Unknown)
}
/// Return the module in which we should look up the definition for this class
pub(crate) const fn canonical_module(self) -> CoreStdlibModule {
pub(crate) fn canonical_module(self, db: &'db dyn Db) -> CoreStdlibModule {
match self {
Self::Bool
| Self::Object
@@ -1771,10 +1893,18 @@ impl<'db> KnownClass {
Self::GenericAlias | Self::ModuleType | Self::FunctionType => CoreStdlibModule::Types,
Self::NoneType => CoreStdlibModule::Typeshed,
Self::SpecialForm | Self::TypeVar | Self::TypeAliasType => CoreStdlibModule::Typing,
// TODO when we understand sys.version_info, we will need an explicit fallback here,
// because typing_extensions has a 3.13+ re-export for the `typing.NoDefault`
// singleton, but not for `typing._NoDefaultType`
Self::NoDefaultType => CoreStdlibModule::TypingExtensions,
Self::NoDefaultType => {
let python_version = Program::get(db).target_version(db);
// typing_extensions has a 3.13+ re-export for the `typing.NoDefault`
// singleton, but not for `typing._NoDefaultType`. So we need to switch
// to `typing.NoDefault` for newer versions:
if python_version.major >= 3 && python_version.minor >= 13 {
CoreStdlibModule::Typing
} else {
CoreStdlibModule::TypingExtensions
}
}
}
}
@@ -1834,11 +1964,11 @@ impl<'db> KnownClass {
};
let module = file_to_module(db, file)?;
candidate.check_module(&module).then_some(candidate)
candidate.check_module(db, &module).then_some(candidate)
}
/// Return `true` if the module of `self` matches `module_name`
fn check_module(self, module: &Module) -> bool {
fn check_module(self, db: &dyn Db, module: &Module) -> bool {
if !module.search_path().is_standard_library() {
return false;
}
@@ -1858,7 +1988,7 @@ impl<'db> KnownClass {
| Self::GenericAlias
| Self::ModuleType
| Self::VersionInfo
| Self::FunctionType => module.name() == self.canonical_module().as_str(),
| Self::FunctionType => module.name() == self.canonical_module(db).as_str(),
Self::NoneType => matches!(module.name().as_str(), "_typeshed" | "types"),
Self::SpecialForm | Self::TypeVar | Self::TypeAliasType | Self::NoDefaultType => {
matches!(module.name().as_str(), "typing" | "typing_extensions")
@@ -1876,6 +2006,10 @@ pub enum KnownInstanceType<'db> {
Optional,
/// The symbol `typing.Union` (which can also be found as `typing_extensions.Union`)
Union,
/// The symbol `typing.NoReturn` (which can also be found as `typing_extensions.NoReturn`)
NoReturn,
/// The symbol `typing.Never` available since 3.11 (which can also be found as `typing_extensions.Never`)
Never,
/// A single instance of `typing.TypeVar`
TypeVar(TypeVarInstance<'db>),
/// A single instance of `typing.TypeAliasType` (PEP 695 type alias)
@@ -1886,11 +2020,13 @@ pub enum KnownInstanceType<'db> {
impl<'db> KnownInstanceType<'db> {
pub const fn as_str(self) -> &'static str {
match self {
KnownInstanceType::Literal => "Literal",
KnownInstanceType::Optional => "Optional",
KnownInstanceType::Union => "Union",
KnownInstanceType::TypeVar(_) => "TypeVar",
KnownInstanceType::TypeAliasType(_) => "TypeAliasType",
Self::Literal => "Literal",
Self::Optional => "Optional",
Self::Union => "Union",
Self::TypeVar(_) => "TypeVar",
Self::NoReturn => "NoReturn",
Self::Never => "Never",
Self::TypeAliasType(_) => "TypeAliasType",
}
}
@@ -1901,6 +2037,8 @@ impl<'db> KnownInstanceType<'db> {
| Self::Optional
| Self::TypeVar(_)
| Self::Union
| Self::NoReturn
| Self::Never
| Self::TypeAliasType(_) => Truthiness::AlwaysTrue,
}
}
@@ -1911,6 +2049,8 @@ impl<'db> KnownInstanceType<'db> {
Self::Literal => "typing.Literal",
Self::Optional => "typing.Optional",
Self::Union => "typing.Union",
Self::NoReturn => "typing.NoReturn",
Self::Never => "typing.Never",
Self::TypeVar(typevar) => typevar.name(db),
Self::TypeAliasType(_) => "typing.TypeAliasType",
}
@@ -1922,6 +2062,8 @@ impl<'db> KnownInstanceType<'db> {
Self::Literal => KnownClass::SpecialForm,
Self::Optional => KnownClass::SpecialForm,
Self::Union => KnownClass::SpecialForm,
Self::NoReturn => KnownClass::SpecialForm,
Self::Never => KnownClass::SpecialForm,
Self::TypeVar(_) => KnownClass::TypeVar,
Self::TypeAliasType(_) => KnownClass::TypeAliasType,
}
@@ -1944,6 +2086,8 @@ impl<'db> KnownInstanceType<'db> {
("typing" | "typing_extensions", "Literal") => Some(Self::Literal),
("typing" | "typing_extensions", "Optional") => Some(Self::Optional),
("typing" | "typing_extensions", "Union") => Some(Self::Union),
("typing" | "typing_extensions", "NoReturn") => Some(Self::NoReturn),
("typing" | "typing_extensions", "Never") => Some(Self::Never),
_ => None,
}
}
@@ -1951,23 +2095,6 @@ impl<'db> KnownInstanceType<'db> {
fn member(self, db: &'db dyn Db, name: &str) -> Symbol<'db> {
let ty = match (self, name) {
(Self::TypeVar(typevar), "__name__") => Type::string_literal(db, typevar.name(db)),
(Self::TypeVar(typevar), "__bound__") => typevar
.upper_bound(db)
.map(|ty| ty.to_meta_type(db))
.unwrap_or_else(|| KnownClass::NoneType.to_instance(db)),
(Self::TypeVar(typevar), "__constraints__") => {
let tuple_elements: Vec<Type<'db>> = typevar
.constraints(db)
.unwrap_or_default()
.iter()
.map(|ty| ty.to_meta_type(db))
.collect();
Type::tuple(db, &tuple_elements)
}
(Self::TypeVar(typevar), "__default__") => typevar
.default_ty(db)
.map(|ty| ty.to_meta_type(db))
.unwrap_or_else(|| KnownClass::NoDefaultType.to_instance(db)),
(Self::TypeAliasType(alias), "__name__") => Type::string_literal(db, alias.name(db)),
_ => return self.instance_fallback(db).member(db, name),
};
@@ -2000,6 +2127,7 @@ pub struct TypeVarInstance<'db> {
}
impl<'db> TypeVarInstance<'db> {
#[allow(unused)]
pub(crate) fn upper_bound(self, db: &'db dyn Db) -> Option<Type<'db>> {
if let Some(TypeVarBoundOrConstraints::UpperBound(ty)) = self.bound_or_constraints(db) {
Some(ty)
@@ -2008,6 +2136,7 @@ impl<'db> TypeVarInstance<'db> {
}
}
#[allow(unused)]
pub(crate) fn constraints(self, db: &'db dyn Db) -> Option<&[Type<'db>]> {
if let Some(TypeVarBoundOrConstraints::Constraints(tuple)) = self.bound_or_constraints(db) {
Some(tuple.elements(db))
@@ -2381,6 +2510,14 @@ impl Truthiness {
matches!(self, Truthiness::Ambiguous)
}
const fn is_always_false(self) -> bool {
matches!(self, Truthiness::AlwaysFalse)
}
const fn is_always_true(self) -> bool {
matches!(self, Truthiness::AlwaysTrue)
}
const fn negate(self) -> Self {
match self {
Self::AlwaysTrue => Self::AlwaysFalse,
@@ -3021,7 +3158,7 @@ pub(crate) mod tests {
use ruff_python_ast as ast;
use test_case::test_case;
pub(crate) fn setup_db() -> TestDb {
pub(crate) fn setup_db_with_python_version(python_version: PythonVersion) -> TestDb {
let db = TestDb::new();
let src_root = SystemPathBuf::from("/src");
@@ -3032,7 +3169,7 @@ pub(crate) mod tests {
Program::from_settings(
&db,
&ProgramSettings {
target_version: PythonVersion::default(),
target_version: python_version,
search_paths: SearchPathSettings::new(src_root),
},
)
@@ -3041,6 +3178,10 @@ pub(crate) mod tests {
db
}
pub(crate) fn setup_db() -> TestDb {
setup_db_with_python_version(PythonVersion::default())
}
/// A test representation of a type that can be transformed unambiguously into a real Type,
/// given a db.
#[derive(Debug, Clone)]
@@ -3488,13 +3629,23 @@ pub(crate) mod tests {
#[test_case(Ty::None)]
#[test_case(Ty::BooleanLiteral(true))]
#[test_case(Ty::BooleanLiteral(false))]
#[test_case(Ty::KnownClassInstance(KnownClass::NoDefaultType))]
fn is_singleton(from: Ty) {
let db = setup_db();
assert!(from.into_type(&db).is_singleton(&db));
}
/// TODO: test documentation
#[test_case(PythonVersion::PY312)]
#[test_case(PythonVersion::PY313)]
fn no_default_type_is_singleton(python_version: PythonVersion) {
let db = setup_db_with_python_version(python_version);
let no_default = Ty::KnownClassInstance(KnownClass::NoDefaultType).into_type(&db);
assert!(no_default.is_singleton(&db));
}
#[test_case(Ty::None)]
#[test_case(Ty::BooleanLiteral(true))]
#[test_case(Ty::IntLiteral(1))]

View File

@@ -4229,6 +4229,7 @@ impl<'db> TypeInferenceBuilder<'db> {
"annotation-f-string",
format_args!("Type expressions cannot use f-strings"),
);
self.infer_fstring_expression(fstring);
Type::Unknown
}
@@ -4374,6 +4375,10 @@ impl<'db> TypeInferenceBuilder<'db> {
todo_type!()
}
// Avoid inferring the types of invalid type expressions that have been parsed from a
// string annotation, as they are not present in the semantic index.
_ if self.deferred_state.in_string_annotation() => Type::Unknown,
// Forms which are invalid in the context of annotation expressions: we infer their
// nested expressions as normal expressions, but the type of the top-level expression is
// always `Type::Unknown` in these cases.
@@ -4457,7 +4462,6 @@ impl<'db> TypeInferenceBuilder<'db> {
self.infer_slice_expression(slice);
Type::Unknown
}
ast::Expr::IpyEscapeCommand(_) => todo!("Implement Ipy escape command support"),
}
}
@@ -4572,7 +4576,7 @@ impl<'db> TypeInferenceBuilder<'db> {
match value_ty {
Type::KnownInstance(known_instance) => {
self.infer_parameterized_known_instance_type_expression(known_instance, slice)
self.infer_parameterized_known_instance_type_expression(subscript, known_instance)
}
_ => {
self.infer_type_expression(slice);
@@ -4583,9 +4587,10 @@ impl<'db> TypeInferenceBuilder<'db> {
fn infer_parameterized_known_instance_type_expression(
&mut self,
subscript: &ast::ExprSubscript,
known_instance: KnownInstanceType,
parameters: &ast::Expr,
) -> Type<'db> {
let parameters = &*subscript.slice;
match known_instance {
KnownInstanceType::Literal => match self.infer_literal_parameter_type(parameters) {
Ok(ty) => ty,
@@ -4626,6 +4631,17 @@ impl<'db> TypeInferenceBuilder<'db> {
self.infer_type_expression(parameters);
todo_type!("generic type alias")
}
KnownInstanceType::NoReturn | KnownInstanceType::Never => {
self.diagnostics.add(
subscript.into(),
"invalid-type-parameter",
format_args!(
"Type `{}` expected no type parameter",
known_instance.repr(self.db)
),
);
Type::Unknown
}
}
}
@@ -5015,7 +5031,7 @@ mod tests {
use crate::db::tests::TestDb;
use crate::program::{Program, SearchPathSettings};
use crate::python_version::PythonVersion;
use crate::python_version::{self, PythonVersion};
use crate::semantic_index::definition::Definition;
use crate::semantic_index::symbol::FileScopeId;
use crate::semantic_index::{global_scope, semantic_index, symbol_table, use_def_map};
@@ -5025,10 +5041,11 @@ mod tests {
use ruff_db::parsed::parsed_module;
use ruff_db::system::{DbWithTestSystem, SystemPathBuf};
use ruff_db::testing::assert_function_query_was_not_run;
use test_case::test_case;
use super::*;
fn setup_db() -> TestDb {
fn setup_db_with_python_version(python_version: PythonVersion) -> TestDb {
let db = TestDb::new();
let src_root = SystemPathBuf::from("/src");
@@ -5039,7 +5056,7 @@ mod tests {
Program::from_settings(
&db,
&ProgramSettings {
target_version: PythonVersion::default(),
target_version: python_version,
search_paths: SearchPathSettings::new(src_root),
},
)
@@ -5048,6 +5065,10 @@ mod tests {
db
}
fn setup_db() -> TestDb {
setup_db_with_python_version(PythonVersion::default())
}
fn setup_db_with_custom_typeshed<'a>(
typeshed: &str,
files: impl IntoIterator<Item = (&'a str, &'a str)>,
@@ -5319,9 +5340,10 @@ mod tests {
Ok(())
}
#[test]
fn ellipsis_type() -> anyhow::Result<()> {
let mut db = setup_db();
#[test_case(PythonVersion::PY39, "ellipsis")]
#[test_case(PythonVersion::PY310, "EllipsisType")]
fn ellipsis_type(version: PythonVersion, expected_type: &str) -> anyhow::Result<()> {
let mut db = setup_db_with_python_version(version);
db.write_dedented(
"src/a.py",
@@ -5330,8 +5352,7 @@ mod tests {
",
)?;
// TODO: sys.version_info
assert_public_ty(&db, "src/a.py", "x", "EllipsisType | ellipsis");
assert_public_ty(&db, "src/a.py", "x", expected_type);
Ok(())
}
@@ -5970,7 +5991,11 @@ mod tests {
"src/a.py",
&["foo", "<listcomp>"],
"x",
"@Todo(async iterables/iterators)",
if cfg!(debug_assertions) {
"@Todo(async iterables/iterators)"
} else {
"@Todo"
},
);
Ok(())
@@ -6000,7 +6025,11 @@ mod tests {
"src/a.py",
&["foo", "<listcomp>"],
"x",
"@Todo(async iterables/iterators)",
if cfg!(debug_assertions) {
"@Todo(async iterables/iterators)"
} else {
"@Todo"
},
);
Ok(())
@@ -6035,6 +6064,72 @@ mod tests {
);
}
#[test]
fn pep695_type_params() {
let mut db = setup_db();
db.write_dedented(
"src/a.py",
"
def f[T, U: A, V: (A, B), W = A, X: A = A1, Y: (int,)]():
pass
class A: ...
class B: ...
class A1(A): ...
",
)
.unwrap();
let check_typevar = |var: &'static str,
upper_bound: Option<&'static str>,
constraints: Option<&[&'static str]>,
default: Option<&'static str>| {
let var_ty = get_symbol(&db, "src/a.py", &["f"], var).expect_type();
assert_eq!(var_ty.display(&db).to_string(), var);
let expected_name_ty = format!(r#"Literal["{var}"]"#);
let name_ty = var_ty.member(&db, "__name__").expect_type();
assert_eq!(name_ty.display(&db).to_string(), expected_name_ty);
let KnownInstanceType::TypeVar(typevar) = var_ty.expect_known_instance() else {
panic!("expected TypeVar");
};
assert_eq!(
typevar
.upper_bound(&db)
.map(|ty| ty.display(&db).to_string()),
upper_bound.map(std::borrow::ToOwned::to_owned)
);
assert_eq!(
typevar.constraints(&db).map(|tys| tys
.iter()
.map(|ty| ty.display(&db).to_string())
.collect::<Vec<_>>()),
constraints.map(|strings| strings
.iter()
.map(std::string::ToString::to_string)
.collect::<Vec<_>>())
);
assert_eq!(
typevar
.default_ty(&db)
.map(|ty| ty.display(&db).to_string()),
default.map(std::borrow::ToOwned::to_owned)
);
};
check_typevar("T", None, None, None);
check_typevar("U", Some("A"), None, None);
check_typevar("V", None, Some(&["A", "B"]), None);
check_typevar("W", None, None, Some("A"));
check_typevar("X", Some("A"), None, Some("A1"));
// a typevar with less than two constraints is treated as unconstrained
check_typevar("Y", None, None, None);
}
// Incremental inference tests
fn first_public_binding<'db>(db: &'db TestDb, file: File, name: &str) -> Definition<'db> {

View File

@@ -375,6 +375,8 @@ impl<'db> ClassBase<'db> {
| KnownInstanceType::TypeAliasType(_)
| KnownInstanceType::Literal
| KnownInstanceType::Union
| KnownInstanceType::NoReturn
| KnownInstanceType::Never
| KnownInstanceType::Optional => None,
},
}

View File

@@ -180,6 +180,16 @@ where
}
}
/// Discard `@Todo`-type metadata from expected types, which is not available
/// when running in release mode.
#[cfg(not(debug_assertions))]
fn discard_todo_metadata(ty: &str) -> std::borrow::Cow<'_, str> {
static TODO_METADATA_REGEX: std::sync::LazyLock<regex::Regex> =
std::sync::LazyLock::new(|| regex::Regex::new(r"@Todo\([^)]*\)").unwrap());
TODO_METADATA_REGEX.replace_all(ty, "@Todo")
}
struct Matcher {
line_index: LineIndex,
source: SourceText,
@@ -276,6 +286,9 @@ impl Matcher {
}
}
Assertion::Revealed(expected_type) => {
#[cfg(not(debug_assertions))]
let expected_type = discard_todo_metadata(&expected_type);
let mut matched_revealed_type = None;
let mut matched_undefined_reveal = None;
let expected_reveal_type_message = format!("Revealed type is `{expected_type}`");

View File

@@ -185,11 +185,11 @@ impl Settings {
pub enum TargetVersion {
Py37,
Py38,
#[default]
Py39,
Py310,
Py311,
Py312,
#[default]
Py313,
}

View File

@@ -0,0 +1,6 @@
while True:
class A:
x: int
break

View File

@@ -0,0 +1,6 @@
while True:
def b():
x: int
break

View File

@@ -0,0 +1,6 @@
for _ in range(1):
class A:
x: int
break

View File

@@ -0,0 +1,6 @@
for _ in range(1):
def b():
x: int
break

View File

@@ -0,0 +1 @@
x: f"Literal[{1 + 2}]" = 3

View File

@@ -1,7 +1,6 @@
---
source: crates/red_knot_workspace/src/workspace/metadata.rs
expression: "&workspace"
snapshot_kind: text
---
WorkspaceMetadata(
root: "/app",
@@ -24,7 +23,7 @@ WorkspaceMetadata(
program: ProgramSettings(
target_version: PythonVersion(
major: 3,
minor: 9,
minor: 13,
),
search_paths: SearchPathSettings(
extra_paths: [],

View File

@@ -1,7 +1,6 @@
---
source: crates/red_knot_workspace/src/workspace/metadata.rs
expression: workspace
snapshot_kind: text
---
WorkspaceMetadata(
root: "/app",
@@ -24,7 +23,7 @@ WorkspaceMetadata(
program: ProgramSettings(
target_version: PythonVersion(
major: 3,
minor: 9,
minor: 13,
),
search_paths: SearchPathSettings(
extra_paths: [],

View File

@@ -1,7 +1,6 @@
---
source: crates/red_knot_workspace/src/workspace/metadata.rs
expression: workspace
snapshot_kind: text
---
WorkspaceMetadata(
root: "/app",
@@ -24,7 +23,7 @@ WorkspaceMetadata(
program: ProgramSettings(
target_version: PythonVersion(
major: 3,
minor: 9,
minor: 13,
),
search_paths: SearchPathSettings(
extra_paths: [],

View File

@@ -1,7 +1,6 @@
---
source: crates/red_knot_workspace/src/workspace/metadata.rs
expression: workspace
snapshot_kind: text
---
WorkspaceMetadata(
root: "/app",
@@ -24,7 +23,7 @@ WorkspaceMetadata(
program: ProgramSettings(
target_version: PythonVersion(
major: 3,
minor: 9,
minor: 13,
),
search_paths: SearchPathSettings(
extra_paths: [],

View File

@@ -1,7 +1,6 @@
---
source: crates/red_knot_workspace/src/workspace/metadata.rs
expression: workspace
snapshot_kind: text
---
WorkspaceMetadata(
root: "/app",
@@ -37,7 +36,7 @@ WorkspaceMetadata(
program: ProgramSettings(
target_version: PythonVersion(
major: 3,
minor: 9,
minor: 13,
),
search_paths: SearchPathSettings(
extra_paths: [],

View File

@@ -1,7 +1,6 @@
---
source: crates/red_knot_workspace/src/workspace/metadata.rs
expression: workspace
snapshot_kind: text
---
WorkspaceMetadata(
root: "/app",
@@ -50,7 +49,7 @@ WorkspaceMetadata(
program: ProgramSettings(
target_version: PythonVersion(
major: 3,
minor: 9,
minor: 13,
),
search_paths: SearchPathSettings(
extra_paths: [],

View File

@@ -272,8 +272,7 @@ const KNOWN_FAILURES: &[(&str, bool, bool)] = &[
("crates/ruff_linter/resources/test/fixtures/pyupgrade/UP039.py", true, false),
// related to circular references in type aliases (salsa cycle panic):
("crates/ruff_python_parser/resources/inline/err/type_alias_invalid_value_expr.py", true, true),
// related to string annotations (https://github.com/astral-sh/ruff/issues/14440)
// related to circular references in f-string annotations (invalid syntax)
("crates/ruff_linter/resources/test/fixtures/pyflakes/F821_15.py", true, true),
("crates/ruff_linter/resources/test/fixtures/pyflakes/F821_14.py", false, true),
("crates/ruff_linter/resources/test/fixtures/pyflakes/F632.py", true, true),
];

View File

@@ -1,6 +1,6 @@
[package]
name = "ruff"
version = "0.8.0"
version = "0.8.1"
publish = true
authors = { workspace = true }
edition = { workspace = true }

View File

@@ -2,7 +2,7 @@ pub use diagnostic::{Diagnostic, DiagnosticKind};
pub use edit::Edit;
pub use fix::{Applicability, Fix, IsolationLevel};
pub use source_map::{SourceMap, SourceMarker};
pub use violation::{AlwaysFixableViolation, FixAvailability, Violation};
pub use violation::{AlwaysFixableViolation, FixAvailability, Violation, ViolationMetadata};
mod diagnostic;
mod edit;

View File

@@ -1,3 +1,4 @@
use crate::DiagnosticKind;
use std::fmt::{Debug, Display};
#[derive(Debug, Copy, Clone)]
@@ -17,7 +18,16 @@ impl Display for FixAvailability {
}
}
pub trait Violation: Debug + PartialEq + Eq {
pub trait ViolationMetadata {
/// Returns the rule name of this violation
fn rule_name() -> &'static str;
/// Returns an explanation of what this violation catches,
/// why it's bad, and what users should do instead.
fn explain() -> Option<&'static str>;
}
pub trait Violation: ViolationMetadata {
/// `None` in the case a fix is never available or otherwise Some
/// [`FixAvailability`] describing the available fix.
const FIX_AVAILABILITY: FixAvailability = FixAvailability::None;
@@ -41,7 +51,7 @@ pub trait Violation: Debug + PartialEq + Eq {
/// This trait exists just to make implementing the [`Violation`] trait more
/// convenient for violations that can always be fixed.
pub trait AlwaysFixableViolation: Debug + PartialEq + Eq {
pub trait AlwaysFixableViolation: ViolationMetadata {
/// The message used to describe the violation.
fn message(&self) -> String;
@@ -69,3 +79,16 @@ impl<V: AlwaysFixableViolation> Violation for V {
<Self as AlwaysFixableViolation>::message_formats()
}
}
impl<T> From<T> for DiagnosticKind
where
T: Violation,
{
fn from(value: T) -> Self {
Self {
body: Violation::message(&value),
suggestion: Violation::fix_title(&value),
name: T::rule_name().to_string(),
}
}
}

View File

@@ -1,10 +1,11 @@
use super::{write, Arguments, FormatElement};
use crate::format_element::Interned;
use crate::prelude::LineMode;
use crate::prelude::{LineMode, Tag};
use crate::{FormatResult, FormatState};
use rustc_hash::FxHashMap;
use std::any::{Any, TypeId};
use std::fmt::Debug;
use std::num::NonZeroUsize;
use std::ops::{Deref, DerefMut};
/// A trait for writing or formatting into [`FormatElement`]-accepting buffers or streams.
@@ -294,10 +295,11 @@ where
}
}
/// A Buffer that removes any soft line breaks.
/// A Buffer that removes any soft line breaks or [`if_group_breaks`](crate::builders::if_group_breaks) elements.
///
/// - Removes [`lines`](FormatElement::Line) with the mode [`Soft`](LineMode::Soft).
/// - Replaces [`lines`](FormatElement::Line) with the mode [`Soft`](LineMode::SoftOrSpace) with a [`Space`](FormatElement::Space)
/// - Removes [`if_group_breaks`](crate::builders::if_group_breaks) elements.
///
/// # Examples
///
@@ -350,6 +352,8 @@ pub struct RemoveSoftLinesBuffer<'a, Context> {
/// It's fine to not snapshot the cache. The worst that can happen is that it holds on interned elements
/// that are now unused. But there's little harm in that and the cache is cleaned when dropping the buffer.
interned_cache: FxHashMap<Interned, Interned>,
state: RemoveSoftLineBreaksState,
}
impl<'a, Context> RemoveSoftLinesBuffer<'a, Context> {
@@ -357,6 +361,7 @@ impl<'a, Context> RemoveSoftLinesBuffer<'a, Context> {
pub fn new(inner: &'a mut dyn Buffer<Context = Context>) -> Self {
Self {
inner,
state: RemoveSoftLineBreaksState::default(),
interned_cache: FxHashMap::default(),
}
}
@@ -375,6 +380,8 @@ fn clean_interned(
if let Some(cleaned) = interned_cache.get(interned) {
cleaned.clone()
} else {
let mut state = RemoveSoftLineBreaksState::default();
// Find the first soft line break element or interned element that must be changed
let result = interned
.iter()
@@ -382,8 +389,9 @@ fn clean_interned(
.find_map(|(index, element)| match element {
FormatElement::Line(LineMode::Soft | LineMode::SoftOrSpace) => {
let mut cleaned = Vec::new();
cleaned.extend_from_slice(&interned[..index]);
Some((cleaned, &interned[index..]))
let (before, after) = interned.split_at(index);
cleaned.extend_from_slice(before);
Some((cleaned, &after[1..]))
}
FormatElement::Interned(inner) => {
let cleaned_inner = clean_interned(inner, interned_cache);
@@ -398,19 +406,33 @@ fn clean_interned(
}
}
_ => None,
element => {
if state.should_drop(element) {
let mut cleaned = Vec::new();
let (before, after) = interned.split_at(index);
cleaned.extend_from_slice(before);
Some((cleaned, &after[1..]))
} else {
None
}
}
});
let result = match result {
// Copy the whole interned buffer so that becomes possible to change the necessary elements.
Some((mut cleaned, rest)) => {
for element in rest {
if state.should_drop(element) {
continue;
}
let element = match element {
FormatElement::Line(LineMode::Soft) => continue,
FormatElement::Line(LineMode::SoftOrSpace) => FormatElement::Space,
FormatElement::Interned(interned) => {
FormatElement::Interned(clean_interned(interned, interned_cache))
}
element => element.clone(),
};
cleaned.push(element);
@@ -431,12 +453,17 @@ impl<Context> Buffer for RemoveSoftLinesBuffer<'_, Context> {
type Context = Context;
fn write_element(&mut self, element: FormatElement) {
if self.state.should_drop(&element) {
return;
}
let element = match element {
FormatElement::Line(LineMode::Soft) => return,
FormatElement::Line(LineMode::SoftOrSpace) => FormatElement::Space,
FormatElement::Interned(interned) => {
FormatElement::Interned(self.clean_interned(&interned))
}
element => element,
};
@@ -456,14 +483,77 @@ impl<Context> Buffer for RemoveSoftLinesBuffer<'_, Context> {
}
fn snapshot(&self) -> BufferSnapshot {
self.inner.snapshot()
BufferSnapshot::Any(Box::new(RemoveSoftLinebreaksSnapshot {
inner: self.inner.snapshot(),
state: self.state,
}))
}
fn restore_snapshot(&mut self, snapshot: BufferSnapshot) {
self.inner.restore_snapshot(snapshot);
let RemoveSoftLinebreaksSnapshot { inner, state } = snapshot.unwrap_any();
self.inner.restore_snapshot(inner);
self.state = state;
}
}
#[derive(Copy, Clone, Debug, Default)]
enum RemoveSoftLineBreaksState {
#[default]
Default,
InIfGroupBreaks {
conditional_content_level: NonZeroUsize,
},
}
impl RemoveSoftLineBreaksState {
fn should_drop(&mut self, element: &FormatElement) -> bool {
match self {
Self::Default => {
// Entered the start of an `if_group_breaks`
if let FormatElement::Tag(Tag::StartConditionalContent(condition)) = element {
if condition.mode.is_expanded() {
*self = Self::InIfGroupBreaks {
conditional_content_level: NonZeroUsize::new(1).unwrap(),
};
return true;
}
}
false
}
Self::InIfGroupBreaks {
conditional_content_level,
} => {
match element {
// A nested `if_group_breaks` or `if_group_fits`
FormatElement::Tag(Tag::StartConditionalContent(_)) => {
*conditional_content_level = conditional_content_level.saturating_add(1);
}
// The end of an `if_group_breaks` or `if_group_fits`.
FormatElement::Tag(Tag::EndConditionalContent) => {
if let Some(level) = NonZeroUsize::new(conditional_content_level.get() - 1)
{
*conditional_content_level = level;
} else {
// Found the end tag of the initial `if_group_breaks`. Skip this element but retain
// the elements coming after
*self = RemoveSoftLineBreaksState::Default;
}
}
_ => {}
}
true
}
}
}
}
struct RemoveSoftLinebreaksSnapshot {
inner: BufferSnapshot,
state: RemoveSoftLineBreaksState,
}
pub trait BufferExtensions: Buffer + Sized {
/// Returns a new buffer that calls the passed inspector for every element that gets written to the output
#[must_use]

View File

@@ -1,6 +1,6 @@
[package]
name = "ruff_linter"
version = "0.8.0"
version = "0.8.1"
publish = false
authors = { workspace = true }
edition = { workspace = true }

View File

@@ -0,0 +1,15 @@
from airflow import DAG, dag
DAG(dag_id="class_default_schedule")
DAG(dag_id="class_schedule", schedule="@hourly")
@dag()
def decorator_default_schedule():
pass
@dag(schedule="0 * * * *")
def decorator_schedule():
pass

View File

@@ -14,6 +14,8 @@ ContextVar("cv", default=frozenset())
ContextVar("cv", default=MappingProxyType({}))
ContextVar("cv", default=re.compile("foo"))
ContextVar("cv", default=float(1))
ContextVar("cv", default=frozenset[str]())
ContextVar[frozenset[str]]("cv", default=frozenset[str]())
# Bad
ContextVar("cv", default=[])
@@ -25,6 +27,8 @@ ContextVar("cv", default=[char for char in "foo"])
ContextVar("cv", default={char for char in "foo"})
ContextVar("cv", default={char: idx for idx, char in enumerate("foo")})
ContextVar("cv", default=collections.deque())
ContextVar("cv", default=set[str]())
ContextVar[set[str]]("cv", default=set[str]())
def bar() -> list[int]:
return [1, 2, 3]

View File

@@ -84,3 +84,27 @@ field24: typing.Union[int, typing.Union[int, int]] # PYI016: Duplicate union me
# duplicates of the outer `int`), but not three times (which would indicate that
# we incorrectly re-checked the nested union).
field25: typing.Union[int, int | int] # PYI016: Duplicate union member `int`
# Should emit in cases with nested `typing.Union`
field26: typing.Union[typing.Union[int, int]] # PYI016: Duplicate union member `int`
# Should emit in cases with nested `typing.Union`
field27: typing.Union[typing.Union[typing.Union[int, int]]] # PYI016: Duplicate union member `int`
# Should emit in cases with mixed `typing.Union` and `|`
field28: typing.Union[int | int] # Error
# Should emit twice in cases with multiple nested `typing.Union`
field29: typing.Union[int, typing.Union[typing.Union[int, int]]] # Error
# Should emit once in cases with multiple nested `typing.Union`
field30: typing.Union[int, typing.Union[typing.Union[int, str]]] # Error
# Should emit once, and fix to `typing.Union[float, int]`
field31: typing.Union[float, typing.Union[int | int]] # Error
# Should emit once, and fix to `typing.Union[float, int]`
field32: typing.Union[float, typing.Union[int | int | int]] # Error
# Test case for mixed union type fix
field33: typing.Union[typing.Union[int | int] | typing.Union[int | int]] # Error

View File

@@ -84,3 +84,27 @@ field24: typing.Union[int, typing.Union[int, int]] # PYI016: Duplicate union me
# duplicates of the outer `int`), but not three times (which would indicate that
# we incorrectly re-checked the nested union).
field25: typing.Union[int, int | int] # PYI016: Duplicate union member `int`
# Should emit in cases with nested `typing.Union`
field26: typing.Union[typing.Union[int, int]] # PYI016: Duplicate union member `int`
# Should emit in cases with nested `typing.Union`
field27: typing.Union[typing.Union[typing.Union[int, int]]] # PYI016: Duplicate union member `int`
# Should emit in cases with mixed `typing.Union` and `|`
field28: typing.Union[int | int] # Error
# Should emit twice in cases with multiple nested `typing.Union`
field29: typing.Union[int, typing.Union[typing.Union[int, int]]] # Error
# Should emit once in cases with multiple nested `typing.Union`
field30: typing.Union[int, typing.Union[typing.Union[int, str]]] # Error
# Should emit once, and fix to `typing.Union[float, int]`
field31: typing.Union[float, typing.Union[int | int]] # Error
# Should emit once, and fix to `typing.Union[float, int]`
field32: typing.Union[float, typing.Union[int | int | int]] # Error
# Test case for mixed union type fix
field33: typing.Union[typing.Union[int | int] | typing.Union[int | int]] # Error

View File

@@ -39,14 +39,30 @@ async def f4(**kwargs: int | int | float) -> None:
...
def f5(
def f5(arg1: int, *args: Union[int, int, float]) -> None:
...
def f6(arg1: int, *args: Union[Union[int, int, float]]) -> None:
...
def f7(arg1: int, *args: Union[Union[Union[int, int, float]]]) -> None:
...
def f8(arg1: int, *args: Union[Union[Union[int | int | float]]]) -> None:
...
def f9(
arg: Union[ # comment
float, # another
complex, int]
) -> None:
...
def f6(
def f10(
arg: (
int | # comment
float | # another

View File

@@ -46,6 +46,18 @@ def f6(
)
) -> None: ... # PYI041
def f5(arg1: int, *args: Union[int, int, float]) -> None: ... # PYI041
def f6(arg1: int, *args: Union[Union[int, int, float]]) -> None: ... # PYI041
def f7(arg1: int, *args: Union[Union[Union[int, int, float]]]) -> None: ... # PYI041
def f8(arg1: int, *args: Union[Union[Union[int | int | float]]]) -> None: ... # PYI041
class Foo:
def good(self, arg: int) -> None: ...

View File

@@ -5,6 +5,9 @@ A: str | Literal["foo"]
B: TypeAlias = typing.Union[Literal[b"bar", b"foo"], bytes, str]
C: TypeAlias = typing.Union[Literal[5], int, typing.Union[Literal["foo"], str]]
D: TypeAlias = typing.Union[Literal[b"str_bytes", 42], bytes, int]
E: TypeAlias = typing.Union[typing.Union[typing.Union[typing.Union[Literal["foo"], str]]]]
F: TypeAlias = typing.Union[str, typing.Union[typing.Union[typing.Union[Literal["foo"], int]]]]
G: typing.Union[str, typing.Union[typing.Union[typing.Union[Literal["foo"], int]]]]
def func(x: complex | Literal[1J], y: Union[Literal[3.14], float]): ...

View File

@@ -5,6 +5,9 @@ A: str | Literal["foo"]
B: TypeAlias = typing.Union[Literal[b"bar", b"foo"], bytes, str]
C: TypeAlias = typing.Union[Literal[5], int, typing.Union[Literal["foo"], str]]
D: TypeAlias = typing.Union[Literal[b"str_bytes", 42], bytes, int]
E: TypeAlias = typing.Union[typing.Union[typing.Union[typing.Union[Literal["foo"], str]]]]
F: TypeAlias = typing.Union[str, typing.Union[typing.Union[typing.Union[Literal["foo"], int]]]]
G: typing.Union[str, typing.Union[typing.Union[typing.Union[Literal["foo"], int]]]]
def func(x: complex | Literal[1J], y: Union[Literal[3.14], float]): ...

View File

@@ -1,11 +1,14 @@
import builtins
from typing import Union
w: builtins.type[int] | builtins.type[str] | builtins.type[complex]
x: type[int] | type[str] | type[float]
y: builtins.type[int] | type[str] | builtins.type[complex]
z: Union[type[float], type[complex]]
z: Union[type[float, int], type[complex]]
s: builtins.type[int] | builtins.type[str] | builtins.type[complex]
t: type[int] | type[str] | type[float]
u: builtins.type[int] | type[str] | builtins.type[complex]
v: Union[type[float], type[complex]]
w: Union[type[float, int], type[complex]]
x: Union[Union[type[float, int], type[complex]]]
y: Union[Union[Union[type[float, int], type[complex]]]]
z: Union[type[complex], Union[Union[type[float, int]]]]
def func(arg: type[int] | str | type[float]) -> None:

View File

@@ -1,11 +1,14 @@
import builtins
from typing import Union
w: builtins.type[int] | builtins.type[str] | builtins.type[complex]
x: type[int] | type[str] | type[float]
y: builtins.type[int] | type[str] | builtins.type[complex]
z: Union[type[float], type[complex]]
z: Union[type[float, int], type[complex]]
s: builtins.type[int] | builtins.type[str] | builtins.type[complex]
t: type[int] | type[str] | type[float]
u: builtins.type[int] | type[str] | builtins.type[complex]
v: Union[type[float], type[complex]]
w: Union[type[float, int], type[complex]]
x: Union[Union[type[float, int], type[complex]]]
y: Union[Union[Union[type[float, int], type[complex]]]]
z: Union[type[complex], Union[Union[type[float, int]]]]
def func(arg: type[int] | str | type[float]) -> None: ...

View File

@@ -1,4 +1,4 @@
from typing import Literal
from typing import Literal, Union
def func1(arg1: Literal[None]):
@@ -17,7 +17,7 @@ def func4(arg1: Literal[int, None, float]):
...
def func5(arg1: Literal[None, None]):
def func5(arg1: Literal[None, None]):
...
@@ -25,13 +25,21 @@ def func6(arg1: Literal[
"hello",
None # Comment 1
, "world"
]):
]):
...
def func7(arg1: Literal[
None # Comment 1
]):
]):
...
def func8(arg1: Literal[None] | None):
...
def func9(arg1: Union[Literal[None], None]):
...
@@ -58,3 +66,16 @@ Literal[1, None, "foo", None] # Y061 None inside "Literal[]" expression. Replac
# and there are no None members in the Literal[] slice,
# only emit Y062:
Literal[None, True, None, True] # Y062 Duplicate "Literal[]" member "True"
# Regression tests for https://github.com/astral-sh/ruff/issues/14567
x: Literal[None] | None
y: None | Literal[None]
z: Union[Literal[None], None]
a: int | Literal[None] | None
b: None | Literal[None] | None
c: (None | Literal[None]) | None
d: None | (Literal[None] | None)
e: None | ((None | Literal[None]) | None) | None
f: Literal[None] | Literal[None]

View File

@@ -1,4 +1,4 @@
from typing import Literal
from typing import Literal, Union
def func1(arg1: Literal[None]): ...
@@ -28,6 +28,12 @@ def func7(arg1: Literal[
]): ...
def func8(arg1: Literal[None] | None):...
def func9(arg1: Union[Literal[None], None]): ...
# OK
def good_func(arg1: Literal[int] | None): ...
@@ -35,3 +41,16 @@ def good_func(arg1: Literal[int] | None): ...
# From flake8-pyi
Literal[None] # PYI061 None inside "Literal[]" expression. Replace with "None"
Literal[True, None] # PYI061 None inside "Literal[]" expression. Replace with "Literal[True] | None"
# Regression tests for https://github.com/astral-sh/ruff/issues/14567
x: Literal[None] | None
y: None | Literal[None]
z: Union[Literal[None], None]
a: int | Literal[None] | None
b: None | Literal[None] | None
c: (None | Literal[None]) | None
d: None | (Literal[None] | None)
e: None | ((None | Literal[None]) | None) | None
f: Literal[None] | Literal[None]

View File

@@ -25,3 +25,9 @@ Literal[
MyType = Literal["foo", Literal[True, False, True], "bar"] # PYI062
n: Literal["No", "duplicates", "here", 1, "1"]
# nested literals, all equivalent to `Literal[1]`
Literal[Literal[1]] # no duplicate
Literal[Literal[Literal[1], Literal[1]]] # once
Literal[Literal[1], Literal[Literal[Literal[1]]]] # once

View File

@@ -25,3 +25,9 @@ Literal[
MyType = Literal["foo", Literal[True, False, True], "bar"] # PYI062
n: Literal["No", "duplicates", "here", 1, "1"]
# nested literals, all equivalent to `Literal[1]`
Literal[Literal[1]] # no duplicate
Literal[Literal[Literal[1], Literal[1]]] # once
Literal[Literal[1], Literal[Literal[Literal[1]]]] # once

View File

@@ -69,3 +69,15 @@ def test_implicit_str_concat_with_multi_parens(param1, param2, param3):
@pytest.mark.parametrize(("param1,param2"), [(1, 2), (3, 4)])
def test_csv_with_parens(param1, param2):
...
parametrize = pytest.mark.parametrize(("param1,param2"), [(1, 2), (3, 4)])
@parametrize
def test_not_decorator(param1, param2):
...
@pytest.mark.parametrize(argnames=("param1,param2"), argvalues=[(1, 2), (3, 4)])
def test_keyword_arguments(param1, param2):
...

View File

@@ -0,0 +1,82 @@
def f():
from typing import cast
cast(int, 3.0) # TC006
def f():
from typing import cast
cast(list[tuple[bool | float | int | str]], 3.0) # TC006
def f():
from typing import Union, cast
cast(list[tuple[Union[bool, float, int, str]]], 3.0) # TC006
def f():
from typing import cast
cast("int", 3.0) # OK
def f():
from typing import cast
cast("list[tuple[bool | float | int | str]]", 3.0) # OK
def f():
from typing import Union, cast
cast("list[tuple[Union[bool, float, int, str]]]", 3.0) # OK
def f():
from typing import cast as typecast
typecast(int, 3.0) # TC006
def f():
import typing
typing.cast(int, 3.0) # TC006
def f():
import typing as t
t.cast(t.Literal["3.0", '3'], 3.0) # TC006
def f():
from typing import cast
cast(
int # TC006 (unsafe, because it will get rid of this comment)
| None,
3.0
)
def f():
# Regression test for #14554
import typing
typing.cast(M-())
def f():
# Simple case with Literal that should lead to nested quotes
from typing import cast, Literal
cast(Literal["A"], 'A')
def f():
# Really complex case with nested forward references
from typing import cast, Annotated, Literal
cast(list[Annotated["list['Literal[\"A\"]']", "Foo"]], ['A'])

View File

@@ -0,0 +1,31 @@
from __future__ import annotations
from typing import Dict, TypeAlias, TYPE_CHECKING
if TYPE_CHECKING:
from typing import Dict
from foo import Foo
OptStr: TypeAlias = str | None
Bar: TypeAlias = Foo[int]
a: TypeAlias = int # OK
b: TypeAlias = Dict # OK
c: TypeAlias = Foo # TC007
d: TypeAlias = Foo | None # TC007
e: TypeAlias = OptStr # TC007
f: TypeAlias = Bar # TC007
g: TypeAlias = Foo | Bar # TC007 x2
h: TypeAlias = Foo[str] # TC007
i: TypeAlias = (Foo | # TC007 x2 (fix removes comment currently)
Bar)
type C = Foo # OK
type D = Foo | None # OK
type E = OptStr # OK
type F = Bar # OK
type G = Foo | Bar # OK
type H = Foo[str] # OK
type I = (Foo | # OK
Bar)

View File

@@ -0,0 +1,52 @@
from __future__ import annotations
from typing import TypeAlias, TYPE_CHECKING
from foo import Foo
if TYPE_CHECKING:
from typing import Dict
OptStr: TypeAlias = str | None
Bar: TypeAlias = Foo[int]
else:
Bar = Foo
a: TypeAlias = 'int' # TC008
b: TypeAlias = 'Dict' # OK
c: TypeAlias = 'Foo' # TC008
d: TypeAlias = 'Foo[str]' # OK
e: TypeAlias = 'Foo.bar' # OK
f: TypeAlias = 'Foo | None' # TC008
g: TypeAlias = 'OptStr' # OK
h: TypeAlias = 'Bar' # TC008
i: TypeAlias = Foo['str'] # TC008
j: TypeAlias = 'Baz' # OK
k: TypeAlias = 'k | None' # OK
l: TypeAlias = 'int' | None # TC008 (because TC010 is not enabled)
m: TypeAlias = ('int' # TC008
| None)
n: TypeAlias = ('int' # TC008 (fix removes comment currently)
' | None')
type B = 'Dict' # TC008
type D = 'Foo[str]' # TC008
type E = 'Foo.bar' # TC008
type G = 'OptStr' # TC008
type I = Foo['str'] # TC008
type J = 'Baz' # TC008
type K = 'K | None' # TC008
type L = 'int' | None # TC008 (because TC010 is not enabled)
type M = ('int' # TC008
| None)
type N = ('int' # TC008 (fix removes comment currently)
' | None')
class Baz:
a: TypeAlias = 'Baz' # OK
type A = 'Baz' # TC008
class Nested:
a: TypeAlias = 'Baz' # OK
type A = 'Baz' # TC008

View File

@@ -101,3 +101,12 @@ def f():
def test_annotated_non_typing_reference(user: Annotated[str, Depends(get_foo)]):
pass
def f():
from typing import TypeAlias, TYPE_CHECKING
if TYPE_CHECKING:
from pandas import DataFrame
x: TypeAlias = DataFrame | None

View File

@@ -58,7 +58,7 @@ def f():
from typing import Literal
from third_party import Type
def test_string_contains_opposite_quote_do_not_fix(self, type1: Type[Literal["'"]], type2: Type[Literal["\'"]]):
def test_string_contains_opposite_quote(self, type1: Type[Literal["'"]], type2: Type[Literal["\'"]]):
pass

View File

@@ -0,0 +1,23 @@
import os
os.listdir('.')
os.listdir(b'.')
string_path = '.'
os.listdir(string_path)
bytes_path = b'.'
os.listdir(bytes_path)
from pathlib import Path
path_path = Path('.')
os.listdir(path_path)
if os.listdir("dir"):
...
if "file" in os.listdir("dir"):
...

View File

@@ -1,3 +1,9 @@
import mod.CONST as const
from mod import CONSTANT as constant
from mod import ANOTHER_CONSTANT as another_constant
import mod.CON as c
from mod import C as c
# These are all OK:
import django.db.models.Q as Query1
from django.db.models import Q as Query2

View File

@@ -1,3 +1,7 @@
import mod.Camel as CAMEL
from mod import CamelCase as CAMELCASE
from mod import AnotherCamelCase as ANOTHER_CAMELCASE
# These are all OK:
import mod.AppleFruit as A
from mod import BananaFruit as B

View File

@@ -0,0 +1,21 @@
"""Regression test for #13824.
Don't report an error when the function being annotated has the
`@no_type_check` decorator.
However, we still want to ignore this annotation on classes. See
https://github.com/python/typing/pull/1615/files and the discussion on #14615.
"""
from typing import no_type_check
@no_type_check
def f(arg: "this isn't python") -> "this isn't python either":
x: "this also isn't python" = 0
@no_type_check
class C:
def f(arg: "this isn't python") -> "this isn't python either":
x: "this also isn't python" = 1

View File

@@ -0,0 +1,21 @@
"""Regression test for #13824.
Don't report an error when the function being annotated has the
`@no_type_check` decorator.
However, we still want to ignore this annotation on classes. See
https://github.com/python/typing/pull/1615/files and the discussion on #14615.
"""
import typing
@typing.no_type_check
def f(arg: "A") -> "R":
x: "A" = 1
@typing.no_type_check
class C:
def f(self, arg: "B") -> "S":
x: "B" = 1

View File

@@ -12,3 +12,4 @@ os.getenv("AA", "GOOD %s" % "BAR")
os.getenv("B", Z)
os.getenv("AA", "GOOD" if Z else "BAR")
os.getenv("AA", 1 if Z else "BAR") # [invalid-envvar-default]
os.environ.get("TEST", 12) # [invalid-envvar-default]

View File

@@ -0,0 +1,234 @@
if len('TEST'): # [PLC1802]
pass
if not len('TEST'): # [PLC1802]
pass
z = []
if z and len(['T', 'E', 'S', 'T']): # [PLC1802]
pass
if True or len('TEST'): # [PLC1802]
pass
if len('TEST') == 0: # Should be fine
pass
if len('TEST') < 1: # Should be fine
pass
if len('TEST') <= 0: # Should be fine
pass
if 1 > len('TEST'): # Should be fine
pass
if 0 >= len('TEST'): # Should be fine
pass
if z and len('TEST') == 0: # Should be fine
pass
if 0 == len('TEST') < 10: # Should be fine
pass
# Should be fine
if 0 < 1 <= len('TEST') < 10: # [comparison-of-constants]
pass
if 10 > len('TEST') != 0: # Should be fine
pass
if 10 > len('TEST') > 1 > 0: # Should be fine
pass
if 0 <= len('TEST') < 100: # Should be fine
pass
if z or 10 > len('TEST') != 0: # Should be fine
pass
if z:
pass
elif len('TEST'): # [PLC1802]
pass
if z:
pass
elif not len('TEST'): # [PLC1802]
pass
while len('TEST'): # [PLC1802]
pass
while not len('TEST'): # [PLC1802]
pass
while z and len('TEST'): # [PLC1802]
pass
while not len('TEST') and z: # [PLC1802]
pass
assert len('TEST') > 0 # Should be fine
x = 1 if len('TEST') != 0 else 2 # Should be fine
f_o_o = len('TEST') or 42 # Should be fine
a = x and len(x) # Should be fine
def some_func():
return len('TEST') > 0 # Should be fine
def github_issue_1325():
l = [1, 2, 3]
length = len(l) if l else 0 # Should be fine
return length
def github_issue_1331(*args):
assert False, len(args) # Should be fine
def github_issue_1331_v2(*args):
assert len(args), args # [PLC1802]
def github_issue_1331_v3(*args):
assert len(args) or z, args # [PLC1802]
def github_issue_1331_v4(*args):
assert z and len(args), args # [PLC1802]
def github_issue_1331_v5(**args):
assert z and len(args), args # [PLC1802]
b = bool(len(z)) # [PLC1802]
c = bool(len('TEST') or 42) # [PLC1802]
def github_issue_1879():
class ClassWithBool(list):
def __bool__(self):
return True
class ClassWithoutBool(list):
pass
class ChildClassWithBool(ClassWithBool):
pass
class ChildClassWithoutBool(ClassWithoutBool):
pass
assert len(ClassWithBool())
assert len(ChildClassWithBool())
assert len(ClassWithoutBool()) # unintuitive?, in pylint: [PLC1802]
assert len(ChildClassWithoutBool()) # unintuitive?, in pylint: [PLC1802]
assert len(range(0)) # [PLC1802]
assert len([t + 1 for t in []]) # [PLC1802]
# assert len(u + 1 for u in []) generator has no len
assert len({"1":(v + 1) for v in {}}) # [PLC1802]
assert len(set((w + 1) for w in set())) # [PLC1802]
import numpy
numpy_array = numpy.array([0])
if len(numpy_array) > 0:
print('numpy_array')
if len(numpy_array):
print('numpy_array')
if numpy_array:
print('b')
import pandas as pd
pandas_df = pd.DataFrame()
if len(pandas_df):
print("this works, but pylint tells me not to use len() without comparison")
if len(pandas_df) > 0:
print("this works and pylint likes it, but it's not the solution intended by PEP-8")
if pandas_df:
print("this does not work (truth value of dataframe is ambiguous)")
def function_returning_list(r):
if r==1:
return [1]
return [2]
def function_returning_int(r):
if r==1:
return 1
return 2
def function_returning_generator(r):
for i in [r, 1, 2, 3]:
yield i
def function_returning_comprehension(r):
return [x+1 for x in [r, 1, 2, 3]]
def function_returning_function(r):
return function_returning_generator(r)
assert len(function_returning_list(z)) # [PLC1802] differs from pylint
assert len(function_returning_int(z))
# This should raise a PLC1802 once astroid can infer it
# See https://github.com/pylint-dev/pylint/pull/3821#issuecomment-743771514
assert len(function_returning_generator(z))
assert len(function_returning_comprehension(z))
assert len(function_returning_function(z))
def github_issue_4215():
# Test undefined variables
# https://github.com/pylint-dev/pylint/issues/4215
if len(undefined_var): # [undefined-variable]
pass
if len(undefined_var2[0]): # [undefined-variable]
pass
def f(cond:bool):
x = [1,2,3]
if cond:
x = [4,5,6]
if len(x): # this should be addressed
print(x)
def g(cond:bool):
x = [1,2,3]
if cond:
x = [4,5,6]
if len(x): # this should be addressed
print(x)
del x
def h(cond:bool):
x = [1,2,3]
x = 123
if len(x): # ok
print(x)
def outer():
x = [1,2,3]
def inner(x:int):
return x+1
if len(x): # [PLC1802]
print(x)
def redefined():
x = 123
x = [1, 2, 3]
if len(x): # this should be addressed
print(x)
global_seq = [1, 2, 3]
def i():
global global_seq
if len(global_seq): # ok
print(global_seq)
def j():
if False:
x = [1, 2, 3]
if len(x): # [PLC1802] should be fine
print(x)

View File

@@ -102,3 +102,6 @@ blah = lambda: {"a": 1}.__delitem__("a") # OK
blah = dict[{"a": 1}.__delitem__("a")] # OK
"abc".__contains__("a")
# https://github.com/astral-sh/ruff/issues/14597
assert "abc".__str__() == "abc"

View File

@@ -93,3 +93,53 @@ op_itemgetter = lambda x: x[1, :]
# Without a slice, trivia is retained
op_itemgetter = lambda x: x[1, 2]
# All methods in classes are ignored, even those defined using lambdas:
class Foo:
def x(self, other):
return self == other
class Bar:
y = lambda self, other: self == other
from typing import Callable
class Baz:
z: Callable = lambda self, other: self == other
# Lambdas wrapped in function calls could also still be method definitions!
# To avoid false positives, we shouldn't flag any of these either:
from typing import final, override, no_type_check
class Foo:
a = final(lambda self, other: self == other)
b = override(lambda self, other: self == other)
c = no_type_check(lambda self, other: self == other)
d = final(override(no_type_check(lambda self, other: self == other)))
# lambdas used in decorators do not constitute method definitions,
# so these *should* be flagged:
class TheLambdasHereAreNotMethods:
@pytest.mark.parametrize(
"slicer, expected",
[
(lambda x: x[-2:], "foo"),
(lambda x: x[-5:-3], "bar"),
],
)
def test_inlet_asset_alias_extra_slice(self, slicer, expected):
assert slice("whatever") == expected
class NotAMethodButHardToDetect:
# In an ideal world, perhaps we'd emit a diagnostic here,
# since this `lambda` is clearly not a method definition,
# and *could* be safely replaced with an `operator` function.
# Practically speaking, however, it's hard to see how we'd accurately determine
# that the `lambda` is *not* a method definition
# without risking false positives elsewhere or introducing complex heuristics
# that users would find surprising and confusing
FOO = sorted([x for x in BAR], key=lambda x: x.baz)

View File

@@ -0,0 +1,125 @@
import attr
from attrs import define, field, frozen, mutable
foo = int
@define # auto_attribs = None => True
class C:
a: str = 0
b = field()
c: int = foo()
d = list()
@define() # auto_attribs = None => True
class C:
a: str = 0
b = field()
c: int = foo()
d = list()
@define(auto_attribs=None) # auto_attribs = None => True
class C:
a: str = 0
b = field()
c: int = foo()
d = list()
@frozen # auto_attribs = None => True
class C:
a: str = 0
b = field()
c: int = foo()
d = list()
@frozen() # auto_attribs = None => True
class C:
a: str = 0
b = field()
c: int = foo()
d = list()
@frozen(auto_attribs=None) # auto_attribs = None => True
class C:
a: str = 0
b = field()
c: int = foo()
d = list()
@mutable # auto_attribs = None => True
class C:
a: str = 0
b = field()
c: int = foo()
d = list()
@mutable() # auto_attribs = None => True
class C:
a: str = 0
b = field()
c: int = foo()
d = list()
@mutable(auto_attribs=None) # auto_attribs = None => True
class C:
a: str = 0
b = field()
c: int = foo()
d = list()
@attr.s # auto_attribs = False
class C:
a: str = 0
b = field()
c: int = foo()
d = list()
@attr.s() # auto_attribs = False
class C:
a: str = 0
b = field()
c: int = foo()
d = list()
@attr.s(auto_attribs=None) # auto_attribs = None => True
class C:
a: str = 0
b = field()
c: int = foo()
d = list()
@attr.s(auto_attribs=False) # auto_attribs = False
class C:
a: str = 0
b = field()
c: int = foo()
d = list()
@attr.s(auto_attribs=True) # auto_attribs = True
class C:
a: str = 0
b = field()
c: int = foo()
d = list()
@attr.s(auto_attribs=[1, 2, 3]) # auto_attribs = False
class C:
a: str = 0
b = field()
c: int = foo()
d = list()

View File

@@ -6,3 +6,12 @@ Never | int
NoReturn | int
Union[Union[Never, int], Union[NoReturn, int]]
Union[NoReturn, int, float]
# Regression tests for https://github.com/astral-sh/ruff/issues/14567
x: None | Never | None
y: (None | Never) | None
z: None | (Never | None)
a: int | Never | None
b: Never | Never | None

View File

@@ -0,0 +1,5 @@
fruits = ["apples", "plums", "pear"]
fruits.filter(lambda fruit: fruit.startwith("p"))
assert len(fruits), 2
assert True, "always true"

View File

@@ -0,0 +1,31 @@
from typing import Literal
import typing as t
import typing_extensions
y: Literal[1, print("hello"), 3, Literal[4, 1]]
Literal[1, Literal[1]]
Literal[1, 2, Literal[1, 2]]
Literal[1, Literal[1], Literal[1]]
Literal[1, Literal[2], Literal[2]]
t.Literal[1, t.Literal[2, t.Literal[1]]]
Literal[
1, # comment 1
Literal[ # another comment
1 # yet annother comment
]
] # once
# Ensure issue is only raised once, even on nested literals
MyType = Literal["foo", Literal[True, False, True], "bar"]
# nested literals, all equivalent to `Literal[1]`
Literal[Literal[1]]
Literal[Literal[Literal[1], Literal[1]]]
Literal[Literal[1], Literal[Literal[Literal[1]]]]
# OK
x: Literal[True, False, True, False]
z: Literal[{1, 3, 5}, "foobar", {1,3,5}]
typing_extensions.Literal[1, 1, 1]
n: Literal["No", "duplicates", "here", 1, "1"]

View File

@@ -0,0 +1,31 @@
from typing import Literal
import typing as t
import typing_extensions
y: Literal[1, print("hello"), 3, Literal[4, 1]]
Literal[1, Literal[1]]
Literal[1, 2, Literal[1, 2]]
Literal[1, Literal[1], Literal[1]]
Literal[1, Literal[2], Literal[2]]
t.Literal[1, t.Literal[2, t.Literal[1]]]
Literal[
1, # comment 1
Literal[ # another comment
1 # yet annother comment
]
] # once
# Ensure issue is only raised once, even on nested literals
MyType = Literal["foo", Literal[True, False, True], "bar"]
# nested literals, all equivalent to `Literal[1]`
Literal[Literal[1]]
Literal[Literal[Literal[1], Literal[1]]]
Literal[Literal[1], Literal[Literal[Literal[1]]]]
# OK
x: Literal[True, False, True, False]
z: Literal[{1, 3, 5}, "foobar", {1,3,5}]
typing_extensions.Literal[1, 1, 1]
n: Literal["No", "duplicates", "here", 1, "1"]

View File

@@ -0,0 +1,13 @@
"""Regression test for #14531.
RUF101 should trigger here because the TCH rules have been recoded to TC.
"""
# ruff: noqa: TCH002
from __future__ import annotations
import local_module
def func(sized: local_module.Container) -> int:
return len(sized)

View File

@@ -3,7 +3,9 @@ use ruff_text_size::Ranged;
use crate::checkers::ast::Checker;
use crate::codes::Rule;
use crate::rules::{flake8_import_conventions, flake8_pyi, pyflakes, pylint, ruff};
use crate::rules::{
flake8_import_conventions, flake8_pyi, flake8_type_checking, pyflakes, pylint, ruff,
};
/// Run lint rules over the [`Binding`]s.
pub(crate) fn bindings(checker: &mut Checker) {
@@ -15,6 +17,7 @@ pub(crate) fn bindings(checker: &mut Checker) {
Rule::UnconventionalImportAlias,
Rule::UnsortedDunderSlots,
Rule::UnusedVariable,
Rule::UnquotedTypeAlias,
]) {
return;
}
@@ -72,6 +75,13 @@ pub(crate) fn bindings(checker: &mut Checker) {
checker.diagnostics.push(diagnostic);
}
}
if checker.enabled(Rule::UnquotedTypeAlias) {
if let Some(diagnostics) =
flake8_type_checking::rules::unquoted_type_alias(checker, binding)
{
checker.diagnostics.extend(diagnostics);
}
}
if checker.enabled(Rule::UnsortedDunderSlots) {
if let Some(diagnostic) = ruff::rules::sort_dunder_slots(checker, binding) {
checker.diagnostics.push(diagnostic);

View File

@@ -2,7 +2,7 @@ use ruff_python_ast::Expr;
use crate::checkers::ast::Checker;
use crate::codes::Rule;
use crate::rules::{flake8_builtins, flake8_pie, pylint, refurb};
use crate::rules::{flake8_builtins, flake8_pie, pylint};
/// Run lint rules over all deferred lambdas in the [`SemanticModel`].
pub(crate) fn deferred_lambdas(checker: &mut Checker) {
@@ -21,9 +21,6 @@ pub(crate) fn deferred_lambdas(checker: &mut Checker) {
if checker.enabled(Rule::ReimplementedContainerBuiltin) {
flake8_pie::rules::reimplemented_container_builtin(checker, lambda);
}
if checker.enabled(Rule::ReimplementedOperator) {
refurb::rules::reimplemented_operator(checker, &lambda.into());
}
if checker.enabled(Rule::BuiltinLambdaArgumentShadowing) {
flake8_builtins::rules::builtin_lambda_argument_shadowing(checker, lambda);
}

View File

@@ -52,14 +52,13 @@ pub(crate) fn deferred_scopes(checker: &mut Checker) {
// Identify any valid runtime imports. If a module is imported at runtime, and
// used at runtime, then by default, we avoid flagging any other
// imports from that model as typing-only.
let enforce_typing_imports = !checker.source_type.is_stub()
let enforce_typing_only_imports = !checker.source_type.is_stub()
&& checker.any_enabled(&[
Rule::RuntimeImportInTypeCheckingBlock,
Rule::TypingOnlyFirstPartyImport,
Rule::TypingOnlyStandardLibraryImport,
Rule::TypingOnlyThirdPartyImport,
]);
let runtime_imports: Vec<Vec<&Binding>> = if enforce_typing_imports {
let runtime_imports: Vec<Vec<&Binding>> = if enforce_typing_only_imports {
checker
.semantic
.scopes
@@ -375,7 +374,16 @@ pub(crate) fn deferred_scopes(checker: &mut Checker) {
}
if matches!(scope.kind, ScopeKind::Function(_) | ScopeKind::Module) {
if enforce_typing_imports {
if !checker.source_type.is_stub()
&& checker.enabled(Rule::RuntimeImportInTypeCheckingBlock)
{
flake8_type_checking::rules::runtime_import_in_type_checking_block(
checker,
scope,
&mut diagnostics,
);
}
if enforce_typing_only_imports {
let runtime_imports: Vec<&Binding> = checker
.semantic
.scopes
@@ -384,26 +392,12 @@ pub(crate) fn deferred_scopes(checker: &mut Checker) {
.copied()
.collect();
if checker.enabled(Rule::RuntimeImportInTypeCheckingBlock) {
flake8_type_checking::rules::runtime_import_in_type_checking_block(
checker,
scope,
&mut diagnostics,
);
}
if checker.any_enabled(&[
Rule::TypingOnlyFirstPartyImport,
Rule::TypingOnlyStandardLibraryImport,
Rule::TypingOnlyThirdPartyImport,
]) {
flake8_type_checking::rules::typing_only_runtime_import(
checker,
scope,
&runtime_imports,
&mut diagnostics,
);
}
flake8_type_checking::rules::typing_only_runtime_import(
checker,
scope,
&runtime_imports,
&mut diagnostics,
);
}
if checker.enabled(Rule::UnusedImport) {

View File

@@ -11,8 +11,8 @@ use ruff_text_size::Ranged;
use crate::checkers::ast::Checker;
use crate::registry::Rule;
use crate::rules::{
flake8_2020, flake8_async, flake8_bandit, flake8_boolean_trap, flake8_bugbear, flake8_builtins,
flake8_comprehensions, flake8_datetimez, flake8_debugger, flake8_django,
airflow, flake8_2020, flake8_async, flake8_bandit, flake8_boolean_trap, flake8_bugbear,
flake8_builtins, flake8_comprehensions, flake8_datetimez, flake8_debugger, flake8_django,
flake8_future_annotations, flake8_gettext, flake8_implicit_str_concat, flake8_logging,
flake8_logging_format, flake8_pie, flake8_print, flake8_pyi, flake8_pytest_style, flake8_self,
flake8_simplify, flake8_tidy_imports, flake8_type_checking, flake8_use_pathlib, flynt, numpy,
@@ -108,6 +108,7 @@ pub(crate) fn expression(expr: &Expr, checker: &mut Checker) {
Rule::DuplicateLiteralMember,
Rule::RedundantBoolLiteral,
Rule::RedundantNoneLiteral,
Rule::UnnecessaryNestedLiteral,
]) {
if !checker.semantic.in_nested_literal() {
if checker.enabled(Rule::DuplicateLiteralMember) {
@@ -119,6 +120,9 @@ pub(crate) fn expression(expr: &Expr, checker: &mut Checker) {
if checker.enabled(Rule::RedundantNoneLiteral) {
flake8_pyi::rules::redundant_none_literal(checker, expr);
}
if checker.enabled(Rule::UnnecessaryNestedLiteral) {
ruff::rules::unnecessary_nested_literal(checker, expr);
}
}
}
@@ -494,6 +498,9 @@ pub(crate) fn expression(expr: &Expr, checker: &mut Checker) {
if checker.enabled(Rule::SuperWithoutBrackets) {
pylint::rules::super_without_brackets(checker, func);
}
if checker.enabled(Rule::LenTest) {
pylint::rules::len_test(checker, call);
}
if checker.enabled(Rule::BitCount) {
refurb::rules::bit_count(checker, call);
}
@@ -856,6 +863,15 @@ pub(crate) fn expression(expr: &Expr, checker: &mut Checker) {
checker.diagnostics.push(diagnostic);
}
}
if checker.settings.preview.is_enabled()
&& checker.any_enabled(&[
Rule::PytestParametrizeNamesWrongType,
Rule::PytestParametrizeValuesWrongType,
Rule::PytestDuplicateParametrizeTestCases,
])
{
flake8_pytest_style::rules::parametrize(checker, call);
}
if checker.enabled(Rule::PytestUnittestAssertion) {
if let Some(diagnostic) = flake8_pytest_style::rules::unittest_assertion(
checker, expr, func, args, keywords,
@@ -953,6 +969,7 @@ pub(crate) fn expression(expr: &Expr, checker: &mut Checker) {
Rule::OsPathGetmtime,
Rule::OsPathGetctime,
Rule::Glob,
Rule::OsListdir,
]) {
flake8_use_pathlib::rules::replaceable_by_pathlib(checker, call);
}
@@ -1061,6 +1078,9 @@ pub(crate) fn expression(expr: &Expr, checker: &mut Checker) {
if checker.enabled(Rule::UnrawRePattern) {
ruff::rules::unraw_re_pattern(checker, call);
}
if checker.enabled(Rule::AirflowDagNoScheduleArgument) {
airflow::rules::dag_no_schedule_argument(checker, expr);
}
}
Expr::Dict(dict) => {
if checker.any_enabled(&[
@@ -1630,6 +1650,11 @@ pub(crate) fn expression(expr: &Expr, checker: &mut Checker) {
ruff::rules::assignment_in_assert(checker, expr);
}
}
Expr::Lambda(lambda_expr) => {
if checker.enabled(Rule::ReimplementedOperator) {
refurb::rules::reimplemented_operator(checker, &lambda_expr.into());
}
}
_ => {}
};
}

View File

@@ -309,12 +309,20 @@ pub(crate) fn statement(stmt: &Stmt, checker: &mut Checker) {
body,
);
}
if checker.any_enabled(&[
Rule::PytestParametrizeNamesWrongType,
Rule::PytestParametrizeValuesWrongType,
Rule::PytestDuplicateParametrizeTestCases,
]) {
flake8_pytest_style::rules::parametrize(checker, decorator_list);
// In preview mode, calls are analyzed. To avoid duplicate diagnostics,
// skip analyzing the decorators.
if !checker.settings.preview.is_enabled()
&& checker.any_enabled(&[
Rule::PytestParametrizeNamesWrongType,
Rule::PytestParametrizeValuesWrongType,
Rule::PytestDuplicateParametrizeTestCases,
])
{
for decorator in decorator_list {
if let Some(call) = decorator.expression.as_call_expr() {
flake8_pytest_style::rules::parametrize(checker, call);
}
}
}
if checker.any_enabled(&[
Rule::PytestIncorrectMarkParenthesesStyle,
@@ -1268,6 +1276,9 @@ pub(crate) fn statement(stmt: &Stmt, checker: &mut Checker) {
if checker.enabled(Rule::AssertWithPrintMessage) {
ruff::rules::assert_with_print_message(checker, assert_stmt);
}
if checker.enabled(Rule::InvalidAssertMessageLiteralArgument) {
ruff::rules::invalid_assert_message_literal_argument(checker, assert_stmt);
}
}
Stmt::With(
with_stmt @ ast::StmtWith {

View File

@@ -723,6 +723,12 @@ impl<'a> Visitor<'a> for Checker<'a> {
// Visit the decorators and arguments, but avoid the body, which will be
// deferred.
for decorator in decorator_list {
if self
.semantic
.match_typing_expr(&decorator.expression, "no_type_check")
{
self.semantic.flags |= SemanticModelFlags::NO_TYPE_CHECK;
}
self.visit_decorator(decorator);
}
@@ -882,9 +888,7 @@ impl<'a> Visitor<'a> for Checker<'a> {
if let Some(type_params) = type_params {
self.visit_type_params(type_params);
}
self.visit
.type_param_definitions
.push((value, self.semantic.snapshot()));
self.visit_deferred_type_alias_value(value);
self.semantic.pop_scope();
self.visit_expr(name);
}
@@ -955,7 +959,7 @@ impl<'a> Visitor<'a> for Checker<'a> {
if let Some(expr) = value {
if self.semantic.match_typing_expr(annotation, "TypeAlias") {
self.visit_type_definition(expr);
self.visit_annotated_type_alias_value(expr);
} else {
self.visit_expr(expr);
}
@@ -1280,6 +1284,10 @@ impl<'a> Visitor<'a> for Checker<'a> {
let mut args = arguments.args.iter();
if let Some(arg) = args.next() {
self.visit_type_definition(arg);
if self.enabled(Rule::RuntimeCastValue) {
flake8_type_checking::rules::runtime_cast_value(self, arg);
}
}
for arg in args {
self.visit_expr(arg);
@@ -1845,8 +1853,50 @@ impl<'a> Checker<'a> {
self.semantic.flags = snapshot;
}
/// Visit an [`Expr`], and treat it as the value expression
/// of a [PEP 613] type alias.
///
/// For example:
/// ```python
/// from typing import TypeAlias
///
/// OptStr: TypeAlias = str | None # We're visiting the RHS
/// ```
///
/// [PEP 613]: https://peps.python.org/pep-0613/
fn visit_annotated_type_alias_value(&mut self, expr: &'a Expr) {
let snapshot = self.semantic.flags;
self.semantic.flags |= SemanticModelFlags::ANNOTATED_TYPE_ALIAS;
self.visit_type_definition(expr);
self.semantic.flags = snapshot;
}
/// Visit an [`Expr`], and treat it as the value expression
/// of a [PEP 695] type alias.
///
/// For example:
/// ```python
/// type OptStr = str | None # We're visiting the RHS
/// ```
///
/// [PEP 695]: https://peps.python.org/pep-0695/#generic-type-alias
fn visit_deferred_type_alias_value(&mut self, expr: &'a Expr) {
let snapshot = self.semantic.flags;
// even though we don't visit these nodes immediately we need to
// modify the semantic flags before we push the expression and its
// corresponding semantic snapshot
self.semantic.flags |= SemanticModelFlags::DEFERRED_TYPE_ALIAS;
self.visit
.type_param_definitions
.push((expr, self.semantic.snapshot()));
self.semantic.flags = snapshot;
}
/// Visit an [`Expr`], and treat it as a type definition.
fn visit_type_definition(&mut self, expr: &'a Expr) {
if self.semantic.in_no_type_check() {
return;
}
let snapshot = self.semantic.flags;
self.semantic.flags |= SemanticModelFlags::TYPE_DEFINITION;
self.visit_expr(expr);
@@ -2004,6 +2054,21 @@ impl<'a> Checker<'a> {
flags.insert(BindingFlags::UNPACKED_ASSIGNMENT);
}
match parent {
Stmt::TypeAlias(_) => flags.insert(BindingFlags::DEFERRED_TYPE_ALIAS),
Stmt::AnnAssign(ast::StmtAnnAssign { annotation, .. }) => {
// TODO: It is a bit unfortunate that we do this check twice
// maybe we should change how we visit this statement
// so the semantic flag for the type alias sticks around
// until after we've handled this store, so we can check
// the flag instead of duplicating this check
if self.semantic.match_typing_expr(annotation, "TypeAlias") {
flags.insert(BindingFlags::ANNOTATED_TYPE_ALIAS);
}
}
_ => {}
}
let scope = self.semantic.current_scope();
if scope.kind.is_module()
@@ -2259,7 +2324,17 @@ impl<'a> Checker<'a> {
self.semantic.flags |=
SemanticModelFlags::TYPE_DEFINITION | type_definition_flag;
self.visit_expr(parsed_annotation.expression());
let parsed_expr = parsed_annotation.expression();
self.visit_expr(parsed_expr);
if self.semantic.in_type_alias_value() {
if self.enabled(Rule::QuotedTypeAlias) {
flake8_type_checking::rules::quoted_type_alias(
self,
parsed_expr,
string_expr,
);
}
}
self.parsed_type_annotation = None;
} else {
if self.enabled(Rule::ForwardAnnotationSyntaxError) {

View File

@@ -211,6 +211,7 @@ pub(crate) fn check_noqa(
&& !exemption.includes(Rule::RedirectedNOQA)
{
ruff::rules::redirected_noqa(diagnostics, &noqa_directives);
ruff::rules::redirected_file_noqa(diagnostics, &file_noqa_directives);
}
if settings.rules.enabled(Rule::BlanketNOQA)

View File

@@ -192,6 +192,7 @@ pub fn code_to_rule(linter: Linter, code: &str) -> Option<(RuleGroup, Rule)> {
(Pylint, "C0208") => (RuleGroup::Stable, rules::pylint::rules::IterationOverSet),
(Pylint, "C0414") => (RuleGroup::Stable, rules::pylint::rules::UselessImportAlias),
(Pylint, "C0415") => (RuleGroup::Preview, rules::pylint::rules::ImportOutsideTopLevel),
(Pylint, "C1802") => (RuleGroup::Preview, rules::pylint::rules::LenTest),
(Pylint, "C1901") => (RuleGroup::Preview, rules::pylint::rules::CompareToEmptyString),
(Pylint, "C2401") => (RuleGroup::Stable, rules::pylint::rules::NonAsciiName),
(Pylint, "C2403") => (RuleGroup::Stable, rules::pylint::rules::NonAsciiImportName),
@@ -856,6 +857,9 @@ pub fn code_to_rule(linter: Linter, code: &str) -> Option<(RuleGroup, Rule)> {
(Flake8TypeChecking, "003") => (RuleGroup::Stable, rules::flake8_type_checking::rules::TypingOnlyStandardLibraryImport),
(Flake8TypeChecking, "004") => (RuleGroup::Stable, rules::flake8_type_checking::rules::RuntimeImportInTypeCheckingBlock),
(Flake8TypeChecking, "005") => (RuleGroup::Stable, rules::flake8_type_checking::rules::EmptyTypeCheckingBlock),
(Flake8TypeChecking, "006") => (RuleGroup::Preview, rules::flake8_type_checking::rules::RuntimeCastValue),
(Flake8TypeChecking, "007") => (RuleGroup::Preview, rules::flake8_type_checking::rules::UnquotedTypeAlias),
(Flake8TypeChecking, "008") => (RuleGroup::Preview, rules::flake8_type_checking::rules::QuotedTypeAlias),
(Flake8TypeChecking, "010") => (RuleGroup::Stable, rules::flake8_type_checking::rules::RuntimeStringUnion),
// tryceratops
@@ -904,6 +908,7 @@ pub fn code_to_rule(linter: Linter, code: &str) -> Option<(RuleGroup, Rule)> {
(Flake8UsePathlib, "205") => (RuleGroup::Stable, rules::flake8_use_pathlib::rules::OsPathGetctime),
(Flake8UsePathlib, "206") => (RuleGroup::Stable, rules::flake8_use_pathlib::rules::OsSepSplit),
(Flake8UsePathlib, "207") => (RuleGroup::Stable, rules::flake8_use_pathlib::rules::Glob),
(Flake8UsePathlib, "208") => (RuleGroup::Preview, rules::flake8_use_pathlib::violations::OsListdir),
// flake8-logging-format
(Flake8LoggingFormat, "001") => (RuleGroup::Stable, rules::flake8_logging_format::violations::LoggingStringFormat),
@@ -975,8 +980,10 @@ pub fn code_to_rule(linter: Linter, code: &str) -> Option<(RuleGroup, Rule)> {
(Ruff, "035") => (RuleGroup::Preview, rules::ruff::rules::UnsafeMarkupUse),
(Ruff, "036") => (RuleGroup::Preview, rules::ruff::rules::NoneNotAtEndOfUnion),
(Ruff, "038") => (RuleGroup::Preview, rules::ruff::rules::RedundantBoolLiteral),
(Ruff, "048") => (RuleGroup::Preview, rules::ruff::rules::MapIntVersionParsing),
(Ruff, "039") => (RuleGroup::Preview, rules::ruff::rules::UnrawRePattern),
(Ruff, "040") => (RuleGroup::Preview, rules::ruff::rules::InvalidAssertMessageLiteralArgument),
(Ruff, "041") => (RuleGroup::Preview, rules::ruff::rules::UnnecessaryNestedLiteral),
(Ruff, "048") => (RuleGroup::Preview, rules::ruff::rules::MapIntVersionParsing),
(Ruff, "100") => (RuleGroup::Stable, rules::ruff::rules::UnusedNOQA),
(Ruff, "101") => (RuleGroup::Stable, rules::ruff::rules::RedirectedNOQA),
@@ -1031,6 +1038,7 @@ pub fn code_to_rule(linter: Linter, code: &str) -> Option<(RuleGroup, Rule)> {
// airflow
(Airflow, "001") => (RuleGroup::Stable, rules::airflow::rules::AirflowVariableNameTaskIdMismatch),
(Airflow, "301") => (RuleGroup::Preview, rules::airflow::rules::AirflowDagNoScheduleArgument),
// perflint
(Perflint, "101") => (RuleGroup::Stable, rules::perflint::rules::UnnecessaryListCast),

View File

@@ -319,7 +319,7 @@ impl<'a> From<&'a FileNoqaDirectives<'a>> for FileExemption<'a> {
if directives
.lines()
.iter()
.any(|line| ParsedFileExemption::All == line.parsed_file_exemption)
.any(|line| matches!(line.parsed_file_exemption, ParsedFileExemption::All))
{
FileExemption::All(codes)
} else {
@@ -362,7 +362,7 @@ impl<'a> FileNoqaDirectives<'a> {
let mut lines = vec![];
for range in comment_ranges {
match ParsedFileExemption::try_extract(&locator.contents()[range]) {
match ParsedFileExemption::try_extract(range, locator.contents()) {
Err(err) => {
#[allow(deprecated)]
let line = locator.compute_line_index(range.start());
@@ -384,6 +384,7 @@ impl<'a> FileNoqaDirectives<'a> {
}
ParsedFileExemption::Codes(codes) => {
codes.iter().filter_map(|code| {
let code = code.as_str();
// Ignore externally-defined rules.
if external.iter().any(|external| code.starts_with(external)) {
return None;
@@ -424,21 +425,26 @@ impl<'a> FileNoqaDirectives<'a> {
/// An individual file-level exemption (e.g., `# ruff: noqa` or `# ruff: noqa: F401, F841`). Like
/// [`FileNoqaDirectives`], but only for a single line, as opposed to an aggregated set of exemptions
/// across a source file.
#[derive(Debug, PartialEq, Eq)]
#[derive(Debug)]
pub(crate) enum ParsedFileExemption<'a> {
/// The file-level exemption ignores all rules (e.g., `# ruff: noqa`).
All,
/// The file-level exemption ignores specific rules (e.g., `# ruff: noqa: F401, F841`).
Codes(Vec<&'a str>),
Codes(Codes<'a>),
}
impl<'a> ParsedFileExemption<'a> {
/// Return a [`ParsedFileExemption`] for a given comment line.
fn try_extract(line: &'a str) -> Result<Option<Self>, ParseError> {
/// Return a [`ParsedFileExemption`] for a given `comment_range` in `source`.
fn try_extract(comment_range: TextRange, source: &'a str) -> Result<Option<Self>, ParseError> {
let line = &source[comment_range];
let offset = comment_range.start();
let init_line_len = line.text_len();
let line = Self::lex_whitespace(line);
let Some(line) = Self::lex_char(line, '#') else {
return Ok(None);
};
let comment_start = init_line_len - line.text_len() - '#'.text_len();
let line = Self::lex_whitespace(line);
let Some(line) = Self::lex_flake8(line).or_else(|| Self::lex_ruff(line)) else {
@@ -469,7 +475,11 @@ impl<'a> ParsedFileExemption<'a> {
let mut codes = vec![];
let mut line = line;
while let Some(code) = Self::lex_code(line) {
codes.push(code);
let codes_end = init_line_len - line.text_len();
codes.push(Code {
code,
range: TextRange::at(codes_end, code.text_len()).add(offset),
});
line = &line[code.len()..];
// Codes can be comma- or whitespace-delimited.
@@ -485,7 +495,12 @@ impl<'a> ParsedFileExemption<'a> {
return Err(ParseError::MissingCodes);
}
Self::Codes(codes)
let codes_end = init_line_len - line.text_len();
let range = TextRange::new(comment_start, codes_end);
Self::Codes(Codes {
range: range.add(offset),
codes,
})
}))
}
@@ -1059,7 +1074,7 @@ mod tests {
use ruff_diagnostics::{Diagnostic, Edit};
use ruff_python_trivia::CommentRanges;
use ruff_source_file::LineEnding;
use ruff_text_size::{TextRange, TextSize};
use ruff_text_size::{TextLen, TextRange, TextSize};
use crate::noqa::{add_noqa_inner, Directive, NoqaMapping, ParsedFileExemption};
use crate::rules::pycodestyle::rules::{AmbiguousVariableName, UselessSemicolon};
@@ -1226,50 +1241,74 @@ mod tests {
#[test]
fn flake8_exemption_all() {
let source = "# flake8: noqa";
assert_debug_snapshot!(ParsedFileExemption::try_extract(source));
assert_debug_snapshot!(ParsedFileExemption::try_extract(
TextRange::up_to(source.text_len()),
source,
));
}
#[test]
fn ruff_exemption_all() {
let source = "# ruff: noqa";
assert_debug_snapshot!(ParsedFileExemption::try_extract(source));
assert_debug_snapshot!(ParsedFileExemption::try_extract(
TextRange::up_to(source.text_len()),
source,
));
}
#[test]
fn flake8_exemption_all_no_space() {
let source = "#flake8:noqa";
assert_debug_snapshot!(ParsedFileExemption::try_extract(source));
assert_debug_snapshot!(ParsedFileExemption::try_extract(
TextRange::up_to(source.text_len()),
source,
));
}
#[test]
fn ruff_exemption_all_no_space() {
let source = "#ruff:noqa";
assert_debug_snapshot!(ParsedFileExemption::try_extract(source));
assert_debug_snapshot!(ParsedFileExemption::try_extract(
TextRange::up_to(source.text_len()),
source,
));
}
#[test]
fn flake8_exemption_codes() {
// Note: Flake8 doesn't support this; it's treated as a blanket exemption.
let source = "# flake8: noqa: F401, F841";
assert_debug_snapshot!(ParsedFileExemption::try_extract(source));
assert_debug_snapshot!(ParsedFileExemption::try_extract(
TextRange::up_to(source.text_len()),
source,
));
}
#[test]
fn ruff_exemption_codes() {
let source = "# ruff: noqa: F401, F841";
assert_debug_snapshot!(ParsedFileExemption::try_extract(source));
assert_debug_snapshot!(ParsedFileExemption::try_extract(
TextRange::up_to(source.text_len()),
source,
));
}
#[test]
fn flake8_exemption_all_case_insensitive() {
let source = "# flake8: NoQa";
assert_debug_snapshot!(ParsedFileExemption::try_extract(source));
assert_debug_snapshot!(ParsedFileExemption::try_extract(
TextRange::up_to(source.text_len()),
source,
));
}
#[test]
fn ruff_exemption_all_case_insensitive() {
let source = "# ruff: NoQa";
assert_debug_snapshot!(ParsedFileExemption::try_extract(source));
assert_debug_snapshot!(ParsedFileExemption::try_extract(
TextRange::up_to(source.text_len()),
source,
));
}
#[test]

View File

@@ -13,6 +13,7 @@ mod tests {
use crate::{assert_messages, settings};
#[test_case(Rule::AirflowVariableNameTaskIdMismatch, Path::new("AIR001.py"))]
#[test_case(Rule::AirflowDagNoScheduleArgument, Path::new("AIR301.py"))]
fn rules(rule_code: Rule, path: &Path) -> Result<()> {
let snapshot = format!("{}_{}", rule_code.noqa_code(), path.to_string_lossy());
let diagnostics = test_path(

View File

@@ -0,0 +1,84 @@
use ruff_diagnostics::{Diagnostic, Violation};
use ruff_macros::{derive_message_formats, ViolationMetadata};
use ruff_python_ast::Expr;
use ruff_python_ast::{self as ast};
use ruff_python_semantic::Modules;
use ruff_text_size::Ranged;
use crate::checkers::ast::Checker;
/// ## What it does
/// Checks for a `DAG()` class or `@dag()` decorator without an explicit
/// `schedule` parameter.
///
/// ## Why is this bad?
/// The default `schedule` value on Airflow 2 is `timedelta(days=1)`, which is
/// almost never what a user is looking for. Airflow 3 changes this the default
/// to *None*, and would break existing DAGs using the implicit default.
///
/// If your DAG does not have an explicit `schedule` argument, Airflow 2
/// schedules a run for it every day (at the time determined by `start_date`).
/// Such a DAG will no longer be scheduled on Airflow 3 at all, without any
/// exceptions or other messages visible to the user.
///
/// ## Example
/// ```python
/// from airflow import DAG
///
///
/// # Using the implicit default schedule.
/// dag = DAG(dag_id="my_dag")
/// ```
///
/// Use instead:
/// ```python
/// from datetime import timedelta
///
/// from airflow import DAG
///
///
/// dag = DAG(dag_id="my_dag", schedule=timedelta(days=1))
/// ```
#[derive(ViolationMetadata)]
pub(crate) struct AirflowDagNoScheduleArgument;
impl Violation for AirflowDagNoScheduleArgument {
#[derive_message_formats]
fn message(&self) -> String {
"DAG should have an explicit `schedule` argument".to_string()
}
}
/// AIR301
pub(crate) fn dag_no_schedule_argument(checker: &mut Checker, expr: &Expr) {
if !checker.semantic().seen_module(Modules::AIRFLOW) {
return;
}
// Don't check non-call expressions.
let Expr::Call(ast::ExprCall {
func, arguments, ..
}) = expr
else {
return;
};
// We don't do anything unless this is a `DAG` (class) or `dag` (decorator
// function) from Airflow.
if !checker
.semantic()
.resolve_qualified_name(func)
.is_some_and(|qualname| matches!(qualname.segments(), ["airflow", .., "DAG" | "dag"]))
{
return;
}
// If there's a `schedule` keyword argument, we are good.
if arguments.find_keyword("schedule").is_some() {
return;
}
// Produce a diagnostic when the `schedule` keyword argument is not found.
let diagnostic = Diagnostic::new(AirflowDagNoScheduleArgument, expr.range());
checker.diagnostics.push(diagnostic);
}

Some files were not shown because too many files have changed in this diff Show More