Compare commits

...

308 Commits

Author SHA1 Message Date
Dhruv Manilawala
2b8c801a79 Make tracing aware that walkdir threads run as part of request 2025-01-16 14:09:16 +05:30
Dhruv Manilawala
79e52c7fdf [pyflakes] Show syntax error message for F722 (#15523)
## Summary

Ref: https://github.com/astral-sh/ruff/pull/15387#discussion_r1917796907

This PR updates `F722` to show syntax error message instead of the
string content.

I think it's more useful to show the syntax error message than the
string content. In the future, when the diagnostics renderer is more
capable, we could even highlight the exact location of the syntax error
along with the annotation string.

This is also in line with how we show the diagnostic in red knot.

## Test Plan

Update existing test snapshots.
2025-01-16 12:44:01 +05:30
Shaygan Hooshyari
cf4ab7cba1 Parse triple quoted string annotations as if parenthesized (#15387)
## Summary

Resolves #9467 

Parse quoted annotations as if the string content is inside parenthesis.
With this logic `x` and `y` in this example are equal:

```python
y: """
   int |
   str
"""

z: """(
    int |
    str
)
"""
```

Also this rule only applies to triple
quotes([link](https://github.com/python/typing-council/issues/9#issuecomment-1890808610)).

This PR is based on the
[comments](https://github.com/astral-sh/ruff/issues/9467#issuecomment-2579180991)
on the issue.

I did one extra change, since we don't want any indentation tokens I am
setting the `State::Other` as the initial state of the Lexer.

Remaining work:

- [x] Add a test case for red-knot.
- [x] Add more tests.

## Test Plan

Added a test which previously failed because quoted annotation contained
indentation.
Added an mdtest for red-knot.
Updated previous test.

Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
Co-authored-by: Micha Reiser <micha@reiser.io>
2025-01-16 11:38:15 +05:30
Dylan
d2656e88a3 [flake8-todos] Allow VSCode GitHub PR extension style links in missing-todo-link (TD003) (#15519)
## Summary
Allow links to issues that appear on the same line as the TODO
directive, if they conform to the format that VSCode's GitHub PR
extension produces.

Revival of #9627 (the branch was stale enough that rebasing was a lot
harder than just making the changes anew). Credit should go to the
author of that PR though.

Closes #8061

Co-authored-by: Martin Bernstorff <martinbernstorff@gmail.com>
2025-01-15 23:47:33 +00:00
Alex Waygood
c53ee608a1 Typeshed-sync workflow: add appropriate labels, link directly to failing run (#15520) 2025-01-15 23:42:35 +00:00
David Peter
c034e280a9 [red-knot] Instance attributes: type inference clarifications (#15512)
## Summary

Some clarifications in the instance-attributes tests, mostly regarding
type inference behavior following this discussion:

https://github.com/astral-sh/ruff/pull/15474#discussion_r1917044566
2025-01-15 21:17:55 +01:00
Alex Waygood
49557a9129 [red-knot] Simplify object out of intersections (#15511) 2025-01-15 20:06:48 +00:00
Andrew Gallant
c9b99e4bee ruff_linter: adjust empty spans after line terminator more generally
Instead of doing this on a lint-by-lint basis, we now just do it right
before rendering. This is more broadly applicable.

Note that this doesn't fix the diagnostic rendering for the Python
parser. But that's using a different path anyway (`annotate-snippets` is
only used in tests).
2025-01-15 13:37:52 -05:00
Andrew Gallant
2ff2a54f56 test: update a few indentation related diagnostics
Previously, these were pointing to the right place, but were missing the
`^`. With the `annotate-snippets` upgrade, the `^` was added, but they
started pointing to the end of the previous line instead of the
beginning of the following line. In this case, we really want it to
point to the beginning of the following line since we're calling out
indentation issues.

As in a prior commit, we fix this by tweaking the offsets emitted by the
lint itself. Instead of an empty range at the beginning of the line, we
point to the first character in the line. This "forces" the renderer to
point to the beginning of the line instead of the end of the preceding
line.

The end effect here is that the rendering is fixed by adding `^` in the
proper location.
2025-01-15 13:37:52 -05:00
Andrew Gallant
17f01a4355 test: add more missing carets
This update includes some missing `^` in the diagnostic annotations.

This update also includes some shifting of "syntax error" annotations to
the end of the preceding line. I believe this is technically a
regression, but fixing them has proven quite difficult. I *think* the
best way to do that might be to tweak the spans generated by the Python
parser errors, but I didn't want to dig into that. (Another approach
would be to change the `annotate-snippets` rendering, but when I tried
that and managed to fix these regressions, I ended up causing a bunch of
other regressions.)

Ref 77d454525e (r1915458616)
2025-01-15 13:37:52 -05:00
Andrew Gallant
5021f32449 test: another update to add back a caret
This change also requires some shuffling to the offsets we generate for
the diagnostic. Previously, we were generating an empty range
immediately *after* the line terminator and immediate before the first
byte of the subsequent line. How this is rendered is somewhat open to
interpretation, but the new version of `annotate-snippets` chooses to
render this at the end of the preceding line instead of the beginning of
the following line.

In this case, we want the diagnostic to point to the beginning of the
following line. So we either need to change `annotate-snippets` to
render such spans at the beginning of the following line, or we need to
change our span to point to the first full character in the following
line. The latter will force `annotate-snippets` to move the caret to the
proper location.

I ended up deciding to change our spans instead of changing how
`annotate-snippets` renders empty spans after a line terminator. While I
didn't investigate it, my guess is that they probably had good reason
for doing so, and it doesn't necessarily strike me as _wrong_.
Furthermore, fixing up our spans seems like a good idea regardless, and
was pretty easy to do.
2025-01-15 13:37:52 -05:00
Andrew Gallant
75b4ed5ad1 codeowners: make BurntSushi owner of ruff_annotate_snippets 2025-01-15 13:37:52 -05:00
Andrew Gallant
e6e610c274 test: tweak in alignment involving unprintable characters
This looks like a bug fix since the caret is now pointing right at the
position of the unprintable character. I'm not sure if this is a result
of an improvement via the `annotate-snippets` upgrade, or because of
more accurate tracking of annotation ranges even after unprintable
characters are replaced. I'm tempted to say the former since in theory
the offsets were never wrong before because they were codepoint offsets.

Regardless, this looks like an improvement.
2025-01-15 13:37:52 -05:00
Andrew Gallant
670fcecd1b test: update snapshots with trimmed lines
This updates snapshots where long lines now get trimmed with
`annotate-snippets`. And an ellipsis is inserted to indicate trimming.

This is a little hokey to test since in tests we don't do any styling.
And I believe this just uses the default "max term width" for rendering.
But in real life, it seems like a big improvement to have long lines
trimmed if they would otherwise wrap in the terminal. So this seems like
an improvement to me.

There are some other fixes here that overlap with previous categories.
2025-01-15 13:37:52 -05:00
Andrew Gallant
84ba4ecaf5 ruff_annotate_snippets: support overriding the "cut indicator"
We do this because `...` is valid Python, which makes it pretty likely
that some line trimming will lead to ambiguous output. So we add support
for overriding the cut indicator. This also requires changing some of
the alignment math, which was previously tightly coupled to `...`.

For Ruff, we go with `…` (`U+2026 HORIZONTAL ELLIPSIS`) for our cut
indicator.

For more details, see the patch sent to upstream:
https://github.com/rust-lang/annotate-snippets-rs/pull/172
2025-01-15 13:37:52 -05:00
Andrew Gallant
a45f4de683 ruff_annotate_snippets: fix false positive line trimming
This fix was sent upstream and the PR description includes more details:
https://github.com/rust-lang/annotate-snippets-rs/pull/170

Without this fix, there was an errant snapshot diff that looked like
this:

  |
1 |   version = "0.1.0"
2 |   # Ensure that the spans from toml handle utf-8 correctly
3 |   authors = [
  |  ___________^
4 | |     { name = "Z͑ͫ̓ͪ̂ͫ̽͏̴̙...A̴̵̜̰͔ͫ͗͢L̠ͨͧͩ͘G̴̻͈͍̑͗̎̅͛́Ǫ̵̹̻̝̳͂̌̌͘", email = 1 }
5 | | ]
  | |_^ RUF200
  |

That ellipsis should _not_ be inserted since the line is not actually
truncated. The handling of line length (in bytes versus actual rendered
length) wasn't quite being handled correctly in all cases.

With this fix, there's (correctly) no snapshot diff.
2025-01-15 13:37:52 -05:00
Andrew Gallant
88df168b63 ruff_annotate_snippets: update snapshot for single ASCII whitespace source
The change to the rendering code is elaborated on in more detail here,
where I attempted to upstream it:
https://github.com/rust-lang/annotate-snippets-rs/pull/169

Otherwise, the snapshot diff also shows a bug fix: a `^` is now rendered
where as it previously was not.
2025-01-15 13:37:52 -05:00
Andrew Gallant
59edee2aca test: update another improperly rendered range
This one almost looks like it fits into the other failure categories,
but without identifying root causes, it's hard to say for sure. The span
here does end after a line terminator, so it feels like it's like the
rest.

I also isolated this change since I found the snapshot diff pretty hard
to read and wanted to look at it more closely. In this case, the before
is:

    E204.py:31:2: E204 [*] Whitespace after decorator
       |
    30 |   # E204
    31 |   @ \
       |  __^
    32 | | foo
       | |_^ E204
    33 |   def baz():
    34 |       print('baz')
       |
       = help: Remove whitespace

And the after is:

    E204.py:31:2: E204 [*] Whitespace after decorator
       |
    30 | # E204
    31 | @ \
       |  ^^ E204
    32 | foo
    33 | def baz():
    34 |     print('baz')
       |
       = help: Remove whitespace

The updated rendering is clearly an improvement, since `foo` itself is
not really the subject of the diagnostic. The whitespace is.

Also, the new rendering matches the span fed to `annotate-snippets`,
where as the old rendering does not.
2025-01-15 13:37:52 -05:00
Andrew Gallant
9fdb1e9bc8 test: update snapshot with fixed annotation but carets include whitespace
I separated out this snapshot update since the string of `^` including
whitespace looked a little odd. I investigated this one specifically,
and indeed, our span in this case is telling `annotate-snippets` to
point at the whitespace. So this is `annotate-snippets` doing what it's
told with a mildly sub-optimal span.

For clarity, the before rendering is:

    skip.py:34:1: I001 [*] Import block is un-sorted or un-formatted
       |
    32 |       import sys; import os  # isort:skip
    33 |       import sys; import os  # isort:skip  # isort:skip
    34 | /     import sys; import os
       |
       = help: Organize imports

And now after is:

    skip.py:34:1: I001 [*] Import block is un-sorted or un-formatted
       |
    32 |     import sys; import os  # isort:skip
    33 |     import sys; import os  # isort:skip  # isort:skip
    34 |     import sys; import os
       | ^^^^^^^^^^^^^^^^^^^^^^^^^ I001
       |
       = help: Organize imports

This is a clear bug fix since it adds in the `I001` annotation, even
though the carets look a little funny by including the whitespace
preceding `import sys; import os`.
2025-01-15 13:37:52 -05:00
Andrew Gallant
eed0595b18 test: another set of updates related to line terminator handling
This group of updates is similar to the last one, but they call out the
fact that while the change is an improvement, it does still seem to be a
little buggy.

As one example, previously we would have this:

       |
     1 | / from __future__ import annotations
     2 | |
     3 | | from typing import Any
     4 | |
     5 | | from requests import Session
     6 | |
     7 | | from my_first_party import my_first_party_object
     8 | |
     9 | | from . import my_local_folder_object
    10 | |
    11 | |
    12 | |
    13 | | class Thing(object):
       | |_^ I001
    14 |     name: str
    15 |     def __init__(self, name: str):
       |
       = help: Organize imports

And now here's what it looks like after:

       |
     1 | / from __future__ import annotations
     2 | |
     3 | | from typing import Any
     4 | |
     5 | | from requests import Session
     6 | |
     7 | | from my_first_party import my_first_party_object
     8 | |
     9 | | from . import my_local_folder_object
    10 | |
    11 | |
    12 | |
       | |__^ Organize imports
    13 |   class Thing(object):
    14 |     name: str
    15 |     def __init__(self, name: str):
       |
       = help: Organize imports

So at least now, the diagnostic is not pointing to a completely
unrelated thing (`class Thing`), but it's still not quite pointing to
the imports directly. And the `^` is a bit offset. After looking at
some examples more closely, I think this is probably more of a bug
with how we're generating offsets, since we are actually pointing to
a location that is a few empty lines _below_ the last import. And
`annotate-snippets` is rendering that part correctly. However, the
offset from the left (the `^` is pointing at `r` instead of `f` or even
at the end of `from . import my_local_folder_object`) appears to be a
problem with `annotate-snippets` itself.

We accept this under the reasoning that it's an improvement, albeit not
perfect.
2025-01-15 13:37:52 -05:00
Andrew Gallant
79e71cbbcd test: another line terminator bug fix
I believe this case is different from the last in that it happens when
the end of a *multi-line* annotation occurs after a line terminator.
Previously, the diagnostic would render on the next line, which is
definitely a bit weird. This new update renders it at the end of the
line the annotation ends on.

In some cases, the annotation was previously rendered to point at source
lines below where the error occurred, which is probably pretty
confusing.
2025-01-15 13:37:52 -05:00
Andrew Gallant
5caef89af3 test: update snapshots with improper end-of-line placement
This looks like a bug fix that occurs when the annotation is a
zero-width span immediately following a line terminator. Previously, the
caret seems to be rendered on the next line, but it should be rendered
at the end of the line the span corresponds to.

I admit that this one is kinda weird. I would somewhat expect that our
spans here are actually incorrect, and that to obtain this sort of
rendering, we should identify a span just immediately _before_ the line
terminator and not after it. But I don't want to dive into that rabbit
hole for now (and given how `annotate-snippets` now renders these
spans, perhaps there is more to it than I see), and this does seem like
a clear improvement given the spans we feed to `annotate-snippets`.
2025-01-15 13:37:52 -05:00
Andrew Gallant
f49cfb6c28 test: update snapshots with missing ^
The previous rendering just seems wrong in that a `^` is omitted. The
new version of `annotate-snippets` seems to get this right. I checked a
pseudo random sample of these, and it seems to only happen when the
position pointed at a line terminator.
2025-01-15 13:37:52 -05:00
Andrew Gallant
f29f58105b test: update formatting of multi-line annotations
It's hard to grok the change from the snapshot diffs alone, so here's
one example. Before:

    PYI021.pyi:15:5: PYI021 [*] Docstrings should not be included in stubs
       |
    14 |   class Baz:
    15 |       """Multiline docstring
       |  _____^
    16 | |
    17 | |     Lorem ipsum dolor sit amet
    18 | |     """
       | |_______^ PYI021
    19 |
    20 |       def __init__(self) -> None: ...
       |
       = help: Remove docstring

And now after:

    PYI021.pyi:15:5: PYI021 [*] Docstrings should not be included in stubs
       |
    14 |   class Baz:
    15 | /     """Multiline docstring
    16 | |
    17 | |     Lorem ipsum dolor sit amet
    18 | |     """
       | |_______^ PYI021
    19 |
    20 |       def __init__(self) -> None: ...
       |
       = help: Remove docstring

I personally think both of these are fine. If we felt strongly, I could
investigate reverting to the old style, but the new style seems okay to
me.

In other words, these updates I believe are just cosmetic and not a bug
fix.
2025-01-15 13:37:52 -05:00
Andrew Gallant
3fa4479c85 test: update snapshots with missing annotations
These updates center around the addition of annotations in the
diagnostic rendering. Previously, the annotation was just not rendered
at all. With the `annotate-snippets` upgrade, it is now rendered. I
examined a pseudo random sample of these, and they all look correct.

As will be true in future batches, some of these snapshots also have
changes to whitespace in them as well.
2025-01-15 13:37:52 -05:00
Andrew Gallant
0de8216a25 test: update snapshots with just whitespace changes
These snapshot changes should *all* only be a result of changes to
trailing whitespace in the output. I checked a psuedo random sample of
these, and the whitespace found in the previous snapshots seems to be an
artifact of the rendering and _not_ of the source data. So this seems
like a strict bug fix to me.

There are other snapshots with whitespace changes, but they also have
other changes that we split out into separate commits. Basically, we're
going to do approximately one commit per category of change.

This represents, by far, the biggest chunk of changes to snapshots as a
result of the `annotate-snippets` upgrade.
2025-01-15 13:37:52 -05:00
Andrew Gallant
2922490cb8 ruff_linter: fix handling of unprintable characters
Previously, we were replacing unprintable ASCII characters with a
printable representation of them via fancier Unicode characters. Since
`annotate-snippets` used to use codepoint offsets, this didn't make our
ranges incorrect: we swapped one codepoint for another.

But now, with the `annotate-snippets` upgrade, we use byte offsets
(which is IMO the correct choice). However, this means our ranges can be
thrown off since an ASCII codepoint is always one byte and a non-ASCII
codepoint is always more than one byte.

Instead of tweaking the `ShowNonprinting` trait and making it more
complicated (which is used in places other than this diagnostic
rendering it seems), we instead change `replace_whitespace` to handle
non-printable characters. This works out because `replace_whitespace`
was already updating the annotation range to account for the tab
replacement. We copy that approach for unprintable characters.
2025-01-15 13:37:52 -05:00
Andrew Gallant
84179aaa96 ruff_linter,ruff_python_parser: migrate to updated annotate-snippets
This is pretty much just moving to the new API and taking care to use
byte offsets. This is *almost* enough. The next commit will fix a bug
involving the handling of unprintable characters as a result of
switching to byte offsets.
2025-01-15 13:37:52 -05:00
Andrew Gallant
1b97677779 ruff_annotate_snippets: make small change to enable omitting header
This is a tiny change that, perhaps slightly shady, permits us to use
the `annotate-snippets` renderer without its mandatory header (which
wasn't there in `annotate-snippets 0.9`). Specifically, we can now do
this:

    Level::None.title("")

The combination of a "none" level and an empty label results in the
`annotate-snippets` header being skipped entirely. (Not even an empty
line is written.)

This is maybe not the right API for upstream `annotate-snippets`, but
it's very easy for us to do and unblocks the upgrade (albeit relying on
a vendored copy).

Ref https://github.com/rust-lang/annotate-snippets-rs/issues/167
2025-01-15 13:37:52 -05:00
Andrew Gallant
9c27c57b5b crates: vendor annotate-snippets crate
This merely adds the crate to our repository. Some cosmetic changes are
made to make it work in our repo and follow our conventions, such as
changing the name to `ruff_annotate_snippets`. We retain the original
license information. We do drop some things, such as benchmarks, but
keep tests and examples.
2025-01-15 13:37:52 -05:00
David Peter
4f3209a3ec [red-knot] More comprehensive 'is_subtype_of' tests (#15490)
## Summary

Make the `is_subtype_of` tests a bit easier to understand and
more comprehensive.
2025-01-15 18:33:29 +00:00
Brent Westbrook
1a77a75935 [FastAPI] Update Annotated fixes (FAST002) (#15462)
## Summary

The initial purpose was to fix #15043, where code like this:
```python
from fastapi import FastAPI, Query

app = FastAPI()

@app.get("/test")
def handler(echo: str = Query("")):
    return echo
```

was being fixed to the invalid code below:

```python
from typing import Annotated
from fastapi import FastAPI, Query

app = FastAPI()

@app.get("/test")
def handler(echo: Annotated[str, Query("")]): # changed
    return echo
```

As @MichaReiser pointed out, the correct fix is:

```python
from typing import Annotated
from fastapi import FastAPI, Query

app = FastAPI()

@app.get("/test")
def handler(echo: Annotated[str, Query()] = ""): # changed
    return echo 
```

After fixing the issue for `Query`, I realized that other classes like
`Path`, `Body`, `Cookie`, `Header`, `File`, and `Form` also looked
susceptible to this issue. The last few commits should handle these too,
which I think means this will also close #12913.

I had to reorder the arguments to the `do_stuff` test case because the
new fix removes some default argument values (eg for `Path`:
`some_path_param: str = Path()` becomes `some_path_param: Annotated[str,
Path()]`).

There's also #14484 related to this rule. I'm happy to take a stab at
that here or in a follow up PR too.

## Test Plan

`cargo test`

I also checked the fixed output with `uv run --with fastapi
FAST002_0.py`, but it required making a bunch of additional changes to
the test file that I wasn't sure we wanted in this PR.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2025-01-15 13:05:53 -05:00
David Peter
48e6541893 [red-knot] Negation reverses subtyping order (#15503)
## Summary

If `S <: T`, then `~T <: ~S`. This test currently fails with example
like:

```
S = tuple[()]
T = ~Literal[True] & ~Literal[False]
```

`T` is equivalent to `~(Literal[True] | Literal[False])` and therefore
equivalent to `~bool`, but the minimal example for a failure is what is
stated above. We correctly recognize that `S <: T`, but fail to see that
`~T <: ~S`, i.e. `bool <: ~tuple[()]`.

This is why the tests goes into the "flaky" section as well.

## Test Plan

```
export QUICKCHECK_TESTS=100000
while cargo test --release -p red_knot_python_semantic -- --ignored types::property_tests::flaky::negation_reverses_subtype_order; do :; done
```
2025-01-15 16:32:21 +01:00
Alex Waygood
55a7f72035 [red-knot] Fix more edge cases for intersection simplification with LiteralString and AlwaysTruthy/AlwaysFalsy (#15496) 2025-01-15 15:02:41 +00:00
David Peter
8712438aec [red-knot] Initial tests for instance attributes (#15474)
## Summary

Adds some initial tests for class and instance attributes, mostly to
document (and discuss) what we want to support eventually. These
tests are not exhaustive yet. The idea is to specify the coarse-grained
behavior first.

Things that we'll eventually want to test:

- Interplay with inheritance
- Support `Final` in addition to `ClassVar`
- Specific tests for `ClassVar`, like making sure that we support things
like `x: Annotated[ClassVar[int], "metadata"]`
- … or making sure that we raise an error here:
  ```py
  class Foo:
      def __init__(self):
          self.x: ClassVar[str] = "x"
  ```
- Add tests for `__new__` in addition to the tests for `__init__`
- Add tests that show that we use the union of types if multiple methods
define the symbol with different types
- Make sure that diagnostics are raised if, e.g., the inferred type of
an assignment within a method does not match the declared type in the
class body.
- https://github.com/astral-sh/ruff/pull/15474#discussion_r1916556284
- Method calls are completely left out for now.
- Same for `@property`
- … and the descriptor protocol

## Test Plan

New Markdown tests

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-01-15 14:43:41 +00:00
Dhruv Manilawala
b5dbb2a1d7 Avoid indexing the same workspace multiple times (#15495)
## Summary

This is not lazy indexing but it should somewhat help with #13686.

Currently, processing the change notifications for config files doesn't
account for the fact that multiple config files could belong to the same
workspace. This means that the server will re-index the same workspace
`n` times where `n` is the number of file events which belongs to the
same workspace. This is evident in the following trace logs:

**Trace logs:**

```
[Trace - 6:21:15 PM] Sending notification 'workspace/didChangeWatchedFiles'.
Params: {
    "changes": [
        {
            "uri": "file:///Users/dhruv/work/astral/parser-checkouts/home-assistant-core/pylint/ruff.toml",
            "type": 1
        },
        {
            "uri": "file:///Users/dhruv/work/astral/parser-checkouts/home-assistant-core/script/ruff.toml",
            "type": 2
        },
        {
            "uri": "file:///Users/dhruv/work/astral/parser-checkouts/home-assistant-core/script/scaffold/templates/ruff.toml",
            "type": 2
        },
        {
            "uri": "file:///Users/dhruv/work/astral/parser-checkouts/home-assistant-core/pyproject.toml",
            "type": 2
        }
    ]
}

...

[Trace - 6:21:19 PM] Sending notification 'workspace/didChangeWatchedFiles'.
Params: {
    "changes": [
        {
            "uri": "file:///Users/dhruv/work/astral/parser-checkouts/home-assistant-core/tests/testing_config/custom_components/ruff.toml",
            "type": 1
        },
        {
            "uri": "file:///Users/dhruv/work/astral/parser-checkouts/home-assistant-core/tests/ruff.toml",
            "type": 2
        }
    ]
}

...
```

**Server logs:**

```
 14.838004208s TRACE     ruff:main notification{method="workspace/didChangeWatchedFiles"}: ruff_server::server::api: enter
  14.838043583s DEBUG     ruff:main notification{method="workspace/didChangeWatchedFiles"}: ruff_server::session::index::ruff_settings: Indexing settings for workspace: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core
  14.854324541s DEBUG ThreadId(55) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/.vscode
  14.854388500s DEBUG ThreadId(55) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/.git
  14.937713291s DEBUG     ruff:main notification{method="workspace/didChangeWatchedFiles"}: ruff_server::session::index::ruff_settings: Indexing settings for workspace: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core
  14.954429833s DEBUG ThreadId(75) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/.git
  14.954675708s DEBUG ThreadId(66) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/.vscode
  15.041465500s DEBUG     ruff:main notification{method="workspace/didChangeWatchedFiles"}: ruff_server::session::index::ruff_settings: Indexing settings for workspace: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core
  15.056731541s DEBUG ThreadId(78) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/.vscode
  15.056796833s DEBUG ThreadId(78) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/.git
  15.117545833s DEBUG     ruff:main notification{method="workspace/didChangeWatchedFiles"}: ruff_server::session::index::ruff_settings: Indexing settings for workspace: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core
  15.133091666s DEBUG ThreadId(90) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/.vscode
  15.133146500s DEBUG ThreadId(90) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/.git
  15.220340666s TRACE ruff:worker:6 request{id=5 method="textDocument/diagnostic"}: ruff_server::server::api: enter
  15.220401458s DEBUG ruff:worker:6 request{id=5 method="textDocument/diagnostic"}: ruff_server::resolve: Included path via `include`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/homeassistant/bootstrap.py
  18.577521250s TRACE     ruff:main notification{method="workspace/didChangeWatchedFiles"}: ruff_server::server::api: enter
  18.577561291s DEBUG     ruff:main notification{method="workspace/didChangeWatchedFiles"}: ruff_server::session::index::ruff_settings: Indexing settings for workspace: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core
  18.616564583s DEBUG ThreadId(102) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/.vscode
  18.616627291s DEBUG ThreadId(102) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/.git
  18.687424250s DEBUG     ruff:main notification{method="workspace/didChangeWatchedFiles"}: ruff_server::session::index::ruff_settings: Indexing settings for workspace: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core
  18.704441416s DEBUG ThreadId(114) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/.vscode
  18.704694958s DEBUG ThreadId(121) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/.git
  18.769627500s TRACE ruff:worker:4 request{id=6 method="textDocument/diagnostic"}: ruff_server::server::api: enter
  18.769696791s DEBUG ruff:worker:4 request{id=6 method="textDocument/diagnostic"}: ruff_server::resolve: Included path via `include`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/homeassistant/bootstrap.py
```

This PR updates the logic to consider all the change events at once
keeping track of the workspace path that have been already indexed.

I want to include this in tomorrow's release to check how this change
would affect the linked issue.

## Test Plan

Run the same scenario as above and check the logs to see that the server
isn't re-indexing the same workspace multiple times:

**Trace logs:**

```
[Trace - 6:04:07 PM] Sending notification 'workspace/didChangeWatchedFiles'.
Params: {
    "changes": [
        {
            "uri": "file:///Users/dhruv/work/astral/parser-checkouts/home-assistant-core/script/ruff.toml",
            "type": 1
        },
        {
            "uri": "file:///Users/dhruv/work/astral/parser-checkouts/home-assistant-core/script/scaffold/templates/ruff.toml",
            "type": 1
        },
        {
            "uri": "file:///Users/dhruv/work/astral/parser-checkouts/home-assistant-core/pylint/ruff.toml",
            "type": 2
        },
        {
            "uri": "file:///Users/dhruv/work/astral/parser-checkouts/home-assistant-core/pyproject.toml",
            "type": 2
        }
    ]
}

...

[Trace - 6:04:11 PM] Sending notification 'workspace/didChangeWatchedFiles'.
Params: {
    "changes": [
        {
            "uri": "file:///Users/dhruv/work/astral/parser-checkouts/home-assistant-core/tests/testing_config/custom_components/ruff.toml",
            "type": 1
        },
        {
            "uri": "file:///Users/dhruv/work/astral/parser-checkouts/home-assistant-core/tests/ruff.toml",
            "type": 2
        }
    ]
}

...
```

**Server logs:**

```
  17.047706750s TRACE     ruff:main notification{method="workspace/didChangeWatchedFiles"}: ruff_server::server::api: enter
  17.047747875s DEBUG     ruff:main notification{method="workspace/didChangeWatchedFiles"}: ruff_server::session::index::ruff_settings: Indexing settings for workspace: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core
  17.080006083s DEBUG ThreadId(54) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/.vscode
  17.080085708s DEBUG ThreadId(54) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/.git
  17.145328791s TRACE ruff:worker:6 request{id=5 method="textDocument/diagnostic"}: ruff_server::server::api: enter
  17.145386166s DEBUG ruff:worker:6 request{id=5 method="textDocument/diagnostic"}: ruff_server::resolve: Included path via `include`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/homeassistant/bootstrap.py
  20.756845958s TRACE     ruff:main notification{method="workspace/didChangeWatchedFiles"}: ruff_server::server::api: enter
  20.756923375s DEBUG     ruff:main notification{method="workspace/didChangeWatchedFiles"}: ruff_server::session::index::ruff_settings: Indexing settings for workspace: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core
  20.781733916s DEBUG ThreadId(66) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/.vscode
  20.781825875s DEBUG ThreadId(75) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/.git
  20.848340750s TRACE ruff:worker:7 request{id=6 method="textDocument/diagnostic"}: ruff_server::server::api: enter
  20.848408041s DEBUG ruff:worker:7 request{id=6 method="textDocument/diagnostic"}: ruff_server::resolve: Included path via `include`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/homeassistant/bootstrap.py
```
2025-01-15 18:58:28 +05:30
David Salvisberg
73488e71f8 [flake8-type-checking] Avoid false positives for | in TC008 (#15201) 2025-01-15 14:27:24 +01:00
InSync
8331326cb6 Remove legacy issue template (#15163) 2025-01-15 11:59:22 +01:00
David Peter
3a6238d8c2 [red-knot] Typeshed sync and sys.platform fixes (#15492)
## Summary

The next sync of typeshed would have failed without manual changes
anyway, so I'm doing one manual sync + the required changes in our
`sys.platform` tests (which are necessary because of my tiny typeshed PR
here: https://github.com/python/typeshed/pull/13378).

closes #15485 (the next run of the pipeline in two weeks should be fine
as the bug has been fixed upstream)
2025-01-15 11:21:01 +01:00
David Peter
d4862844f1 [red-knot] 'is_equivalent_to' is an equivalence relation (#15488)
## Summary

Adds two additional tests for `is_equivalent_to` so that we cover all
properties of an [equivalence relation].

## Test Plan

```
while cargo test --release -p red_knot_python_semantic -- --ignored types::property_tests::stable; do :; done
```

[equivalence relation]:
https://en.wikipedia.org/wiki/Equivalence_relation
2025-01-15 09:25:46 +01:00
Micha Reiser
96c2d0996d Fix curly bracket spacing around curly f-string expressions (#15471) 2025-01-15 09:22:47 +01:00
Dhruv Manilawala
6aef4ad008 Fix LSP show message macro to allow format args (#15487)
## Summary

This PR fixes the `show_*_msg` macros to pass all the tokens instead of
just a single token. This allows for using various expressions right in
the macro similar to how it would be in `format_args!`.

## Test Plan

`cargo clippy`
2025-01-15 08:11:49 +00:00
Micha Reiser
18d5dbfb7f Remove workspace support (#15472) 2025-01-15 09:03:38 +01:00
Dhruv Manilawala
bec8441cf5 Use tool specific function to perform exclude checks (#15486)
## Summary

This PR creates separate functions to check whether the document path is
excluded for linting or formatting. The main motivation is to avoid the
double `Option` for the call sites and makes passing the correct
settings simpler.
2025-01-15 13:18:46 +05:30
InSync
aefb607405 [red-knot] Migrate is_equivalent_to unit tests to Markdown tests (#15470)
## Summary

Part of #15397, built on top of #15469.

## Test Plan

Markdown tests.
2025-01-14 18:57:23 +00:00
Alex Waygood
bcf0a715c2 [red-knot] Corrections and improvements to intersection simplification (#15475) 2025-01-14 18:15:38 +00:00
InSync
5ed7b55b15 [red-knot] Migrate is_subtype_of unit tests to Markdown tests (#15469)
## Summary

Part of #15397.

## Test Plan

Markdown tests.

---------

Co-authored-by: David Peter <mail@david-peter.de>
2025-01-14 15:57:24 +01:00
David Peter
8aac69bb2e [red-knot] Add boundness and declaredness tests (#15453)
## Summary

This changeset adds new tests for public uses of symbols,
considering all possible declaredness and boundness states.

Note that this is a mere documentation of the current behavior. There is
still an [open ticket] questioning some of these choices (or unintential
behaviors).

## Test plan

Made sure that the respective test fails if I add the questionable case
again in `symbol_by_id`:

```rs
Symbol::Type(inferred_ty, Boundness::Bound) => {
    Symbol::Type(inferred_ty, Boundness::Bound)
}
```

[open ticket]: https://github.com/astral-sh/ruff/issues/14297
2025-01-14 13:07:16 +01:00
Tom Kuson
9dfc61bf09 [flake8-pytest-style] Tweak documentation and message (#15465) 2025-01-14 08:47:45 +01:00
Tom Kuson
369cbb5424 [flake8-builtins] Improve A005 documentation (#15466) 2025-01-14 08:42:13 +01:00
Garrett Reynolds
dc491e8ade [ruff] Fix false positive on global keyword (RUF052) (#15235) 2025-01-14 08:36:40 +01:00
Wei Lee
a2dc8c93ef [airflow] Replace typo "security_managr" as "security_manager" (AIR303) (#15463)
## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

Replace typo "security_managr" in AIR303 as "security_manager"

## Test Plan

<!-- How was it tested? -->

a test fixture has been updated
2025-01-13 18:38:09 -05:00
Carl Meyer
d54c19b983 [red-knot] remove CallOutcome::Cast variant (#15461)
## Summary

Simplification follow-up to #15413.

There's no need to have a dedicated `CallOutcome` variant for every
known function, it's only necessary if the special-cased behavior of the
known function includes emitting extra diagnostics. For `typing.cast`,
there's no such need; we can use the regular `Callable` outcome variant,
and update the return type according to the cast. (This is the same way
we already handle `len`.)

One reason to avoid proliferating unnecessary `CallOutcome` variants is
that currently we have to explicitly add emitting call-binding
diagnostics, for each outcome variant. So we were previously wrongly
silencing any binding diagnostics on calls to `typing.cast`. Fixing this
revealed a separate bug, that we were emitting a bogus error anytime
more than one keyword argument mapped to a `**kwargs` parameter. So this
PR also adds test and fix for that bug.

## Test Plan

Existing `cast` tests pass unchanged, added new test for `**kwargs` bug.
2025-01-13 10:58:53 -08:00
Micha Reiser
5ad546f187 Change ProgramSettings::python_platform to return a reference (#15457) 2025-01-13 16:23:34 +01:00
InSync
47d0a8ba96 [flake8-pytest-style] Test function parameters with default arguments (PT028) (#15449) 2025-01-13 13:40:54 +01:00
Dhruv Manilawala
56b14454dc Display context for ruff.configuration errors (#15452)
## Summary

I noticed this while trying out
https://github.com/astral-sh/ruff-vscode/issues/665 that we use the
`Display` implementation to show the error which hides the context. This
PR changes it to use the `Debug` implementation and adds the message as
a context.

## Test Plan

**Before:**

```
   0.001228084s ERROR main ruff_server::session::index::ruff_settings: Unable to find editor-specified configuration file: Failed to parse /private/tmp/hatch-test/ruff.toml
```

**After:**

```
   0.002348750s ERROR main ruff_server::session::index::ruff_settings: Unable to load editor-specified configuration file

Caused by:
    0: Failed to parse /private/tmp/hatch-test/ruff.toml
    1: TOML parse error at line 2, column 18
         |
       2 | extend-select = ["ASYNC101"]
         |                  ^^^^^^^^^^
       Unknown rule selector: `ASYNC101`
```
2025-01-13 15:43:20 +05:30
David Peter
eb3cb8d4b2 [red-knot] Use BitSet::union for merging of declarations (#15451)
## Summary

In `SymbolState` merging, use `BitSet::union` instead of inserting
declarations one by one. This used to be the case but was changed in
https://github.com/astral-sh/ruff/pull/15019 because we had to iterate
over declarations anyway.

This is an alternative to https://github.com/astral-sh/ruff/pull/15419
by @MichaReiser. It's similar in performance, but a bit more
declarative and less imperative.
2025-01-13 11:10:42 +01:00
InSync
6f35a4d8d5 [fastapi] Handle parameters with Depends correctly (FAST003) (#15364)
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
Co-authored-by: Micha Reiser <micha@reiser.io>
2025-01-13 08:51:02 +00:00
cake-monotone
82d06a198d [red-knot] Remove duplicate property test (#15450)
## Summary

Follow-up PR from https://github.com/astral-sh/ruff/pull/15415  🥲 

The exact same property test already exists:
`intersection_assignable_to_both` and
`all_type_pairs_can_be_assigned_from_their_intersection`

## Test Plan

`cargo test -p red_knot_python_semantic -- --ignored
types::property_tests::flaky`
2025-01-13 08:18:41 +01:00
InSync
70c3be88b9 [flake8-pie] Reuse parsed tokens (PIE800) (#15438)
## Summary

Follow-up to #15394. See [this review
comment](https://github.com/astral-sh/ruff/pull/15394#discussion_r1910526741).

## Test Plan

`cargo nextest run` and `cargo insta test`.
2025-01-12 21:03:11 -05:00
Tom Kuson
347ab5b47a [flake8-pytest-style] Implement pytest.warns diagnostics (PT029, PT030, PT031) (#15444)
## Summary

Implements upstream diagnostics `PT029`, `PT030`, `PT031` that function
as pytest.warns corollaries of `PT010`, `PT011`, `PT012` respectively.
Most of the implementation and documentation is designed to mirror those
existing diagnostics.

Closes #14239

## Test Plan

Tests for `PT029`, `PT030`, `PT031` largely copied from `PT010`,
`PT011`, `PT012` respectively.

`cargo nextest run`

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2025-01-13 01:46:59 +00:00
renovate[bot]
fa11b08766 Update dependency @types/react to v19.0.6 (#15448) 2025-01-13 01:11:51 +00:00
renovate[bot]
6f3e4e5062 Update NPM Development dependencies to v19.0.5 (#15445) 2025-01-12 20:06:02 -05:00
Charlie Marsh
2454305ef8 [flake8-pathlib] Fix --select for os-path-dirname (PTH120) (#15446)
## Summary

Closes https://github.com/astral-sh/ruff/issues/15439.
2025-01-13 00:55:46 +00:00
InSync
4f37fdeff2 [flake8-bandit] Check for builtins instead of builtin (S102, PTH123) (#15443)
## Summary

Resolves #15442.

## Test Plan

`cargo nextest run` and `cargo insta test`.
2025-01-12 19:45:31 -05:00
InSync
d1666fbbee [red-knot] Add AlwaysTruthy and AlwaysFalsy to knot_extensions (#15437)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-01-12 17:00:57 +00:00
Alex Waygood
06b7f4495e [red-knot] Minor improvements to KnownFunction API (#15441)
A small PR to reduce some of the code duplication between the various
branches, make it a little more readable and move the API closer to what
we already have for `KnownClass`
2025-01-12 16:06:31 +00:00
Alex Waygood
c8795fcb37 [red-knot] Minor improvements to property_tests.rs (#15440) 2025-01-12 13:55:18 +00:00
cake-monotone
ccfde37619 [red-knot] Add Property Tests for Intersection and Union (#15415) 2025-01-12 13:21:29 +00:00
InSync
6ae3e8f8d7 [red-knot] Support cast (#15413)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-01-12 13:05:45 +00:00
renovate[bot]
60d7a464fb Update Rust crate colored to v3 (#15434)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-01-11 17:52:52 +00:00
renovate[bot]
c0259e7bf2 Update dependency ruff to v0.9.1 (#15432)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-01-11 17:18:38 +00:00
renovate[bot]
22edee2353 Update pre-commit dependencies (#15433)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2025-01-11 17:18:13 +00:00
renovate[bot]
7d20277111 Update Rust crate libcst to v1.6.0 (#15431)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-01-11 17:14:44 +00:00
renovate[bot]
bce07f6564 Update Rust crate bitflags to v2.7.0 (#15430)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-01-11 17:14:19 +00:00
renovate[bot]
8ea6605a6d Update NPM Development dependencies (#15428)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-01-11 17:13:32 +00:00
renovate[bot]
d323f2019b Update dependency uuid to v11.0.5 (#15427)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-01-11 17:13:17 +00:00
renovate[bot]
ad883d9b31 Update Rust crate uuid to v1.11.1 (#15426)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-01-11 17:12:54 +00:00
renovate[bot]
7240212d27 Update Rust crate thiserror to v2.0.11 (#15425)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-01-11 17:12:34 +00:00
renovate[bot]
925ee41317 Update Rust crate syn to v2.0.96 (#15424)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-01-11 17:12:23 +00:00
renovate[bot]
78b242fe3f Update Rust crate proc-macro2 to v1.0.93 (#15422)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-01-11 17:12:07 +00:00
renovate[bot]
7ed46d0823 Update Rust crate serde_json to v1.0.135 (#15423)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-01-11 17:11:48 +00:00
renovate[bot]
bff4edb717 Update Rust crate clap to v4.5.26 (#15420)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-01-11 17:09:54 +00:00
Dhruv Manilawala
38f873ba52 Remove flatten to improve deserialization error messages (#15414)
## Summary

Closes: #9719  

## Test Plan

**Before:**

```
ruff failed
  Cause: Failed to parse /Users/dhruv/playground/ruff/pyproject.toml
  Cause: TOML parse error at line 22, column 1
   |
22 | [tool.ruff.lint]
   | ^^^^^^^^^^^^^^^^
invalid type: string "false", expected a boolean
```

**After:**

```
ruff failed
  Cause: Failed to parse /Users/dhruv/playground/ruff/pyproject.toml
  Cause: TOML parse error at line 27, column 20
   |
27 | mypy-init-return = "false"
   |                    ^^^^^^^
invalid type: string "false", expected a boolean
```
2025-01-11 22:08:21 +05:30
Micha Reiser
c39ca8fe6d Upgrade Rust toolchain to 1.84.0 (#15408) 2025-01-11 09:51:58 +01:00
David Peter
2d82445794 [red-knot] Simplify unions of T and ~T (#15400)
## Summary

Simplify unions of `T` and `~T` to `object`.

## Test Plan

Adapted existing tests.
2025-01-10 23:00:52 +01:00
David Peter
398f2e8b0c [red-knot] Minor fixes in intersection-types tests (#15410)
## Summary

Minor fixes in intersection-types tests
2025-01-10 22:53:03 +01:00
InSync
232fbc1300 [red-knot] Understand type[Unknown] (#15409)
## Summary

Follow-up to #15194.

## Test Plan

Markdown tests.
2025-01-10 13:25:59 -08:00
Alex Waygood
c82932e580 [red-knot] Refactor KnownFunction::takes_expression_arguments() (#15406) 2025-01-10 19:09:03 +00:00
Micha Reiser
12f86f39a4 Ruff 0.9.1 (#15407) 2025-01-10 19:45:06 +01:00
Micha Reiser
2b28d566a4 Associate a trailing end-of-line comment in a parenthesized implicit concatenated string with the last literal (#15378) 2025-01-10 19:21:34 +01:00
Calum Young
adca7bd95c Remove pygments pin (#15404)
## Summary

The recent release of Pygments
([2.19.1](https://github.com/pygments/pygments/releases/tag/2.19.1))
allows the pinned version to be removed as the PYI alias for Python
syntax highlighting has been removed.

## Test Plan

- Follow the steps outlined in
https://github.com/astral-sh/ruff/blob/main/CONTRIBUTING.md#mkdocs to
get the documentation site running locally.
- Spot test rules pages that have PYI code blocks to ensure that syntax
highlighting remains e.g.
[http://127.0.0.1:8000/ruff/rules/complex-if-statement-in-stub/](http://127.0.0.1:8000/ruff/rules/complex-if-statement-in-stub/).

**Note:** I am unable to test the insiders build but would assume that
it functions locally as I do not have access to MkDocs Insiders, but I
would like to assume that it functions in the same way as the
non-insiders build.
2025-01-10 12:15:13 -05:00
InSync
6b98a26452 [red-knot] Support assert_type (#15194)
## Summary

See #15103.

## Test Plan

Markdown tests and unit tests.
2025-01-10 08:45:02 -08:00
David Peter
c87463842a [red-knot] Move tuple-containing-Never tests to Markdown (#15402)
## Summary

See title.

Part of #15397

## Test Plan

Ran new Markdown test.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-01-10 15:31:30 +00:00
InSync
c364b586f9 [flake8-pie] Correctly remove wrapping parentheses (PIE800) (#15394)
Co-authored-by: Micha Reiser <micha@reiser.io>
2025-01-10 14:52:32 +00:00
Antoine Dechaume
73d424ee5e Fix outdated doc for handling the default file types with the pre-commit hook (#15401)
Co-authored-by: Antoine DECHAUME <>
2025-01-10 15:49:23 +01:00
Dhruv Manilawala
6e9ff445fd Insert the cells from the start position (#15398)
## Summary

The cause of this bug is from
https://github.com/astral-sh/ruff/pull/12575 which was itself a bug fix
but the fix wasn't completely correct.

fixes: #14768 
fixes: https://github.com/astral-sh/ruff-vscode/issues/644

## Test Plan

Consider the following three cells:

1.
```python
class Foo:
    def __init__(self):
        self.x = 1

    def __str__(self):
        return f"Foo({self.x})"
```

2.
```python
def hello():
    print("hello world")
```

3.
```python
y = 1
```

The test case is moving cell 2 to the top i.e., cell 2 goes to position
1 and cell 1 goes to position 2.

Before this fix, it can be seen that the cells were pushed at the end of
the vector:

```
  12.643269917s  INFO ruff:main ruff_server::edit:📓 Before update: [
    NotebookCell {
        document: TextDocument {
            contents: "class Foo:\n    def __init__(self):\n        self.x = 1\n\n    def __str__(self):\n        return f\"Foo({self.x})\"",
        },
    },
    NotebookCell {
        document: TextDocument {
            contents: "def hello():\n    print(\"hello world\")",
        },
    },
    NotebookCell {
        document: TextDocument {
            contents: "y = 1",
        },
    },
]
  12.643777667s  INFO ruff:main ruff_server::edit:📓 After update: [
    NotebookCell {
        document: TextDocument {
            contents: "y = 1",
        },
    },
    NotebookCell {
        document: TextDocument {
            contents: "class Foo:\n    def __init__(self):\n        self.x = 1\n\n    def __str__(self):\n        return f\"Foo({self.x})\"",
        },
    },
    NotebookCell {
        document: TextDocument {
            contents: "def hello():\n    print(\"hello world\")",
        },
    },
]
```

After the fix in this PR, it can be seen that the cells are being pushed
at the correct `start` index:

```
   6.520570917s  INFO ruff:main ruff_server::edit:📓 Before update: [
    NotebookCell {
        document: TextDocument {
            contents: "class Foo:\n    def __init__(self):\n        self.x = 1\n\n    def __str__(self):\n        return f\"Foo({self.x})\"",
        },
    },
    NotebookCell {
        document: TextDocument {
            contents: "def hello():\n    print(\"hello world\")",
        },
    },
    NotebookCell {
        document: TextDocument {
            contents: "y = 1",
        },
    },
]
   6.521084792s  INFO ruff:main ruff_server::edit:📓 After update: [
    NotebookCell {
        document: TextDocument {
            contents: "def hello():\n    print(\"hello world\")",
        },
    },
    NotebookCell {
        document: TextDocument {
            contents: "class Foo:\n    def __init__(self):\n        self.x = 1\n\n    def __str__(self):\n        return f\"Foo({self.x})\"",
        },
    },
    NotebookCell {
        document: TextDocument {
            contents: "y = 1",
        },
    },
]
```
2025-01-10 13:11:56 +00:00
David Peter
f2c3ddc5ea [red-knot] Move intersection type tests to Markdown (#15396)
## Summary

[**Rendered version of the new test
suite**](https://github.com/astral-sh/ruff/blob/david/intersection-type-tests/crates/red_knot_python_semantic/resources/mdtest/intersection_types.md)

Moves most of our existing intersection-types tests to a dedicated
Markdown test suite, extends the test coverage, unifies the notation for
these tests, groups tests into a proper structure, and adds some
explanations for various simplification strategies.

This changeset also:
- Adds a new simplification where `~Never` is removed from
intersections.
- Adds a new simplification where adding `~object` simplifies the whole
intersection to `Never`
- Avoids unnecessary assignment-checks between inferred and declared
type. This was added to this changeset to avoid many false positive
errors in this test suite.

Resolves the task described in this old comment
[here](e01da82a5a..e7e432bca2 (r1819924085)).

## Test Plan

Running the new Markdown tests

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-01-10 14:04:03 +01:00
Dhruv Manilawala
b861551b6a Remove unnecessary backticks (#15393)
Ref: https://github.com/astral-sh/ruff/pull/15367#discussion_r1909448140
2025-01-10 09:22:26 +00:00
Dylan
443bf38565 [ruff] Omit diagnostic for shadowed private function parameters in used-dummy-variable (RUF052) (#15376) 2025-01-10 03:09:25 -06:00
Tom Kuson
23ad319b55 [flake8-bugbear] Improve assert-raises-exception (B017) message (#15389) 2025-01-10 08:48:18 +01:00
InSync
3d9433ca66 [pyupgrade] Handle comments and multiline expressions correctly (UP037) (#15337) 2025-01-10 08:46:01 +01:00
Douglas Creager
baf068361a [red-knot] Consolidate all gradual types into single Type variant (#15386)
Prompted by

> One nit: I think we need to consider `Any` and `Unknown` and `Todo` as
all (gradually) equivalent to each other, and thus `type & Any` and
`type & Unknown` and `type & Todo` as also equivalent. The distinction
between `Any` vs `Unknown` vs `Todo` is entirely about
provenance/debugging, there is no type level distinction. (And I've been
wondering if the `Any` vs `Unknown` distinction is really worth it.)

The thought here is that _most_ places want to treat `Any`, `Unknown`,
and `Todo` identically. So this PR simplifies things by having a single
`Type::Any` variant, and moves the provenance part into a new `AnyType`
type. If you need to treat e.g. `Todo` differently, you still can by
pattern-matching into the `AnyType`. But if you don't, you can just use
`Type::Any(_)`.

(This would also allow us to (more easily) distinguish "unknown via an
unannotated value" from "unknown because of a typing error" should we
want to do that in the future)

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Carl Meyer <carl@astral.sh>
2025-01-09 21:32:20 -05:00
David Peter
b33cf5baba [red-knot] Move UnionBuilder tests to Markdown (#15374)
## Summary

This moves almost all of our existing `UnionBuilder` tests to a
Markdown-based test suite.

I see how this could be a more controversial change, since these tests
where written specifically for `UnionBuilder`, and by creating the union
types using Python type expressions, we add an additional layer on top
(parsing and inference of these expressions) that moves these tests away
from clean unit tests more in the direction of integration tests. Also,
there are probably a few implementation details of `UnionBuilder` hidden
in the test assertions (e.g. order of union elements after
simplifications).

That said, I think we would like to see all those properties that are
being tested here from *any* implementation of union types. And the
Markdown tests come with the usual advantages:

- More consice
- Better readability
- No re-compiliation when working on tests
- Easier to add additional explanations and structure to the test suite

This changeset adds a few additional tests, but keeps the logic of the
existing tests except for a few minor modifications for consistency.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: T-256 <132141463+T-256@users.noreply.github.com>
2025-01-09 21:45:06 +01:00
Dylan
b0905c4b04 [pycodestyle] Handle each cell separately for too-many-newlines-at-end-of-file (W391) (#15308)
Jupyter notebooks are converted into source files by joining with
newlines, which confuses the check [too-many-newlines-at-end-of-file
(W391)](https://docs.astral.sh/ruff/rules/too-many-newlines-at-end-of-file/#too-many-newlines-at-end-of-file-w391).
This PR introduces logic to apply the check cell-wise (and, in
particular, correctly handles empty cells.)

Closes #13763
2025-01-09 10:50:39 -06:00
Micha Reiser
d0b2bbd55e Release 0.9.0 (#15371)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-01-09 14:53:08 +01:00
Josiah Outram Halstead
8628f169e9 [ruff] Stop parsing diagnostics from other sources for code action requests (#15373) 2025-01-09 14:38:13 +01:00
InSync
8bc11c49b2 [flake8-django] Recognize other magic methods (DJ012) (#15365) 2025-01-09 14:36:42 +01:00
David Peter
bf5b0c2688 [red-knot] Minor refactor of red_knot_vendored/build.rs (#15372)
## Summary

See https://github.com/astral-sh/ruff/pull/15370#discussion_r1908611461:

- Rename `zip_dir` to `write_zipped_typeshed_to` to clarify it's not a
generic function (anymore)
- Hard-code `TYPESHED_SOURCE_DIR` instead of using a `directory_path`
argument
2025-01-09 12:23:42 +00:00
David Peter
097aa04c04 [red-knot] Typeshed patching: use build.rs instead of workflow (#15370)
## Summary

The symlink-approach in the typeshed-sync workflow caused some problems
on Windows, even though it seemed to work fine in CI:

https://github.com/astral-sh/ruff/pull/15138#issuecomment-2578642129

Here, we rely on `build.rs` to patch typeshed instead, which allows us
to get rid of the modifications in the workflow (thank you
@MichaReiser for the idea).

## Test Plan

- Made sure that changes to `knot_extensions.pyi` result in a recompile
  of `red_knot_vendored`.
2025-01-09 11:50:32 +01:00
Dhruv Manilawala
f706c3fdf2 Add f-string formatting to the docs (#15367)
Revive https://github.com/astral-sh/ruff/pull/15341 as it got removed
from the latest rebase in https://github.com/astral-sh/ruff/pull/15238.
2025-01-09 10:20:06 +01:00
Micha Reiser
29f6653318 [ruff] Stabilize useless-if-else (RUF034) (#15351) 2025-01-09 10:20:06 +01:00
Micha Reiser
d645525afc [pylint]: Stabilize boolean-chained-comparison (PLR1716) (#15354) 2025-01-09 10:20:06 +01:00
Micha Reiser
6dcf7b35b9 [ruff] Stabilize post-init-default (RUF033) (#15352) 2025-01-09 10:20:06 +01:00
Micha Reiser
943d4fc160 Update formatter preview documentation (#15349) 2025-01-09 10:20:06 +01:00
Micha Reiser
3ea4c63d2c [flake8-pyi] Stabilize: include all python file types for PYI006 (#15340) 2025-01-09 10:20:06 +01:00
Micha Reiser
8e8a07144d [flake8-pyi]: Stabilize: Provide more automated fixes for duplicate-union-members (PYI016) (#15342) 2025-01-09 10:20:06 +01:00
Micha Reiser
225dd0a027 [ruff] Stabilize: Detect attrs dataclasses (RUF008, RUF009) (#15345) 2025-01-09 10:20:06 +01:00
Micha Reiser
52aeb8ae11 [flake8-pyi] Stabilize autofix for redundant-numeric-union (PYI041) (#15343) 2025-01-09 10:20:06 +01:00
Micha Reiser
71b6ac81a6 Remove unnecessary PreviewMode::Enabled in tests (#15344) 2025-01-09 10:20:06 +01:00
Alex Waygood
75fc2c3116 [ruff-0.9] Stabilise two flake8-builtins rules (#15322) 2025-01-09 10:20:06 +01:00
Micha Reiser
9c4d124ba0 [pycodestyle] Stabilize: Exempt pytest.importorskip calls (E402) (#15338) 2025-01-09 10:20:06 +01:00
InSync
8c620b9b4b [flake8-pytest-style] Stabilize "Detect more pytest.mark.parametrize calls" (PT006) (#15327)
Co-authored-by: Micha Reiser <micha@reiser.io>
Resolves #15324. Stabilizes the behavior changes introduced in #14515.
2025-01-09 10:20:06 +01:00
Dylan
1eda27d1a5 [ruff-0.9] Stabilize decimal-from-float-literal (RUF032) (#15333) 2025-01-09 10:20:06 +01:00
Alex Waygood
aaa86cf38d [ruff-0.9] Stabilise slice-to-remove-prefix-or-suffix (FURB188) (#15329)
Stabilise [`slice-to-remove-prefix-or-suffix`](https://docs.astral.sh/ruff/rules/slice-to-remove-prefix-or-suffix/) (`FURB188`) for the Ruff 0.9 release.

This is a stylistic rule, but I think it's a pretty uncontroversial one. There are no open issues or PRs regarding it and it's been in preview for a while now.
2025-01-09 10:20:06 +01:00
Micha Reiser
7821206b7b Update Black deviations to reflect 2025 style changes (#15127) 2025-01-09 10:20:06 +01:00
Micha Reiser
b76d05e283 Remove formatter incompatibility warning for ISC001 (#15123) 2025-01-09 10:20:06 +01:00
Micha Reiser
424b720c19 Ruff 2025 style guide (#13906)
Closes #13371
2025-01-09 10:20:06 +01:00
Carl Meyer
a95deec00f [red-knot] handle synthetic 'self' argument in call-binding diagnostics (#15362) 2025-01-09 00:36:48 -08:00
InSync
21aa12a073 [red-knot] More precise inference for classes with non-class metaclasses (#15138)
## Summary

Resolves #14208.

## Test Plan

Markdown tests.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2025-01-09 00:34:04 +00:00
Douglas Creager
5f5eb7c0dd [red-knot] Print non-string panic payloads and (sometimes) backtraces (#15363)
More refinements to the panic messages for failing mdtests to mimic the
output of the default panic hook more closely:

- We now print out `Box<dyn Any>` if the panic payload is not a string
(which is typically the case for salsa panics).
- We now include the panic's backtrace if you set the `RUST_BACKTRACE`
environment variable.
2025-01-08 18:12:16 -05:00
David Peter
b6562ed57e [red-knot] Property test workflow: Fix issue label, link to CI run (#15361)
## Summary

See title. Had to make a minor change, because it failed the zizmor
pre-commit check otherwise:

```
error[template-injection]: code injection via template expansion
  --> /home/shark/ruff/.github/workflows/daily_fuzz.yaml:68:9
   |
68 |          - uses: actions/github-script@v7
   |  __________^
69 | |          with:
70 | |            github-token: ${{ secrets.GITHUB_TOKEN }}
71 | |            script: |
   | | ___________^
72 | ||             await github.rest.issues.create({
...  ||
77 | ||               labels: ["bug", "parser", "fuzzer"],
78 | ||             })
   | ||               ^
   | ||_______________|
   |  |_______________this step
   |                  github.server_url may expand into attacker-controllable code
   |
   = note: audit confidence → High
```
2025-01-08 22:47:16 +01:00
David Peter
4fd82d5f35 [red-knot] Property test improvements (#15358)
## Summary

- Add a workflow to run property tests on a daily basis (based on
`daily_fuzz.yaml`)
- Mark `assignable_to_is_reflexive` as flaky (related to #14899)
- Add new (failing) `intersection_assignable_to_both` test (also related
to #14899)

## Test Plan

Ran:

```bash
export QUICKCHECK_TESTS=100000
while cargo test --release -p red_knot_python_semantic -- \
  --ignored types::property_tests::stable; do :; done
```

Observed successful property_tests CI run
2025-01-08 22:24:57 +01:00
David Peter
beb8e2dfe0 [red-knot] More comprehensive is_assignable_to tests (#15353)
## Summary

This changeset migrates all existing `is_assignable_to` tests to a
Markdown-based test. It also increases our test coverage in a hopefully
meaningful way (not claiming to be complete in any sense). But at least
I found and fixed one bug while doing so.

## Test Plan

Ran property tests to make sure the new test succeeds after fixing it.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-01-08 20:25:08 +01:00
Alex Waygood
88d07202c1 [red-knot] Reduce Name clones in call signature checking (#15335) 2025-01-08 18:29:35 +00:00
Douglas Creager
2ca31e4b43 Fall back on previous panic hook when not in catch_unwind wrapper (#15319)
This fixes #15317. Our `catch_unwind` wrapper installs a panic hook that
captures (the rendered contents of) the panic info when a panic occurs.
Since the intent is that the caller will render the panic info in some
custom way, the hook silences the default stderr panic output.

However, the panic hook is a global resource, so if any one thread was
in the middle of a `catch_unwind` call, we would silence the default
panic output for _all_ threads.

The solution is to also keep a thread local that indicates whether the
current thread is in the middle of our `catch_unwind`, and to fall back
on the default panic hook if not.

## Test Plan

Artificially added an mdtest parse error, ran tests via `cargo test -p
red_knot_python_semantic` to run a large number of tests in parallel.
Before this patch, the panic message was swallowed as reported in
#15317. After, the panic message was shown.
2025-01-08 11:34:51 -05:00
Auguste Lalande
450d4e0e0c [pylint] Fix unreachable infinite loop (PLW0101) (#15278)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

Fix infinite loop issue reported here #15248.
The issue was caused by the break inside the if block, which caused the
flow to exit in an unforeseen way. This caused other issues, eventually
leading to an infinite loop.

Resolves #15248. Resolves #15336.

## Test Plan

Added failing code to fixture.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: dylwil3 <dylwil3@gmail.com>
2025-01-08 09:45:04 -06:00
Alex Waygood
7284d68157 fix invalid syntax in workflow file (#15357) 2025-01-08 15:34:33 +00:00
InSync
3820af2f1b [pycodestyle] Avoid false positives related to type aliases (E252) (#15356) 2025-01-08 16:04:08 +01:00
Alex Waygood
ee9a912f47 [flake8-builtins] Disapply A005 to stub files (#15350) 2025-01-08 12:59:27 +00:00
Dhruv Manilawala
1447553bc2 Improve logging system using logLevel, avoid trace value (#15232)
## Summary

Refer to the VS Code PR
(https://github.com/astral-sh/ruff-vscode/pull/659) for details on the
change.

This PR changes the following:

1. Add tracing span for both request (request id and method name) and
notification (method name) handler
2. Remove the `RUFF_TRACE` environment variable. This was being used to
turn on / off logging for the server
3. Similarly, remove reading the `trace` value from the initialization
options
4. Remove handling the `$/setTrace` notification
5. Remove the specialized `TraceLogWriter` used for Zed and VS Code
(https://github.com/astral-sh/ruff/pull/12564)

Regarding the (5) for the Zed editor, the reason that was implemented
was because there was no way of looking at the stderr messages in the
editor which has been changed. Now, it captures the stderr as part of
the "Server Logs".
(82492d74a8/crates/language_tools/src/lsp_log.rs (L548-L552))

### Question

Regarding (1), I think having just a simple trace level message should
be good for now as the spans are not hierarchical. This could be tackled
with #12744. The difference between the two:

<details><summary>Using <code>tracing::trace</code></summary>
<p>

```
   0.019243416s DEBUG ThreadId(08) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/playground/ruff/.vscode
   0.026398750s  INFO main ruff_server::session::index: Registering workspace: /Users/dhruv/playground/ruff
   0.026802125s TRACE ruff:main ruff_server::server::api: Received notification "textDocument/didOpen"
   0.026930666s TRACE ruff:main ruff_server::server::api: Received notification "textDocument/didOpen"
   0.026962333s TRACE ruff:main ruff_server::server::api: Received request "textDocument/diagnostic" (1)
   0.027042875s TRACE ruff:main ruff_server::server::api: Received request "textDocument/diagnostic" (2)
   0.027097500s TRACE ruff:main ruff_server::server::api: Received request "textDocument/codeAction" (3)
   0.027107458s DEBUG ruff:worker:0 ruff_server::resolve: Included path via `include`: /Users/dhruv/playground/ruff/lsp/play.py
   0.027123541s DEBUG ruff:worker:3 ruff_server::resolve: Included path via `include`: /Users/dhruv/playground/ruff/lsp/organize_imports.py
   0.027514875s  INFO     ruff:main ruff_server::server: Configuration file watcher successfully registered
   0.285689833s TRACE     ruff:main ruff_server::server::api: Received request "textDocument/codeAction" (4)
  45.741101666s TRACE     ruff:main ruff_server::server::api: Received notification "textDocument/didClose"
  47.108745500s TRACE     ruff:main ruff_server::server::api: Received notification "textDocument/didOpen"
  47.109802041s TRACE     ruff:main ruff_server::server::api: Received request "textDocument/diagnostic" (5)
  47.109926958s TRACE     ruff:main ruff_server::server::api: Received request "textDocument/codeAction" (6)
  47.110027791s DEBUG ruff:worker:6 ruff_server::resolve: Included path via `include`: /Users/dhruv/playground/ruff/lsp/play.py
  51.863679125s TRACE     ruff:main ruff_server::server::api: Received request "textDocument/hover" (7)
```

</p>
</details> 

<details><summary>Using <code>tracing::trace_span</code></summary>
<p>

Only logging the enter event:

```
   0.018638750s DEBUG ThreadId(11) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/playground/ruff/.vscode
   0.025895791s  INFO main ruff_server::session::index: Registering workspace: /Users/dhruv/playground/ruff
   0.026378791s TRACE ruff:main notification{method="textDocument/didOpen"}: ruff_server::server::api: enter
   0.026531208s TRACE ruff:main notification{method="textDocument/didOpen"}: ruff_server::server::api: enter
   0.026567583s TRACE ruff:main request{id=1 method="textDocument/diagnostic"}: ruff_server::server::api: enter
   0.026652541s TRACE ruff:main request{id=2 method="textDocument/diagnostic"}: ruff_server::server::api: enter
   0.026711041s DEBUG ruff:worker:2 ruff_server::resolve: Included path via `include`: /Users/dhruv/playground/ruff/lsp/organize_imports.py
   0.026729166s DEBUG ruff:worker:1 ruff_server::resolve: Included path via `include`: /Users/dhruv/playground/ruff/lsp/play.py
   0.027023083s  INFO     ruff:main ruff_server::server: Configuration file watcher successfully registered
   5.197554750s TRACE     ruff:main notification{method="textDocument/didClose"}: ruff_server::server::api: enter
   6.534458000s TRACE     ruff:main notification{method="textDocument/didOpen"}: ruff_server::server::api: enter
   6.535027958s TRACE     ruff:main request{id=3 method="textDocument/diagnostic"}: ruff_server::server::api: enter
   6.535271166s DEBUG ruff:worker:3 ruff_server::resolve: Included path via `include`: /Users/dhruv/playground/ruff/lsp/organize_imports.py
   6.544240583s TRACE     ruff:main request{id=4 method="textDocument/codeAction"}: ruff_server::server::api: enter
   7.049692458s TRACE     ruff:main request{id=5 method="textDocument/codeAction"}: ruff_server::server::api: enter
   7.508142541s TRACE     ruff:main request{id=6 method="textDocument/hover"}: ruff_server::server::api: enter
   7.872421958s TRACE     ruff:main request{id=7 method="textDocument/hover"}: ruff_server::server::api: enter
   8.024498583s TRACE     ruff:main request{id=8 method="textDocument/codeAction"}: ruff_server::server::api: enter
  13.895063666s TRACE     ruff:main request{id=9 method="textDocument/codeAction"}: ruff_server::server::api: enter
  14.774706083s TRACE     ruff:main request{id=10 method="textDocument/hover"}: ruff_server::server::api: enter
  16.058918958s TRACE     ruff:main notification{method="textDocument/didChange"}: ruff_server::server::api: enter
  16.060562208s TRACE     ruff:main request{id=11 method="textDocument/diagnostic"}: ruff_server::server::api: enter
  16.061109083s DEBUG ruff:worker:8 ruff_server::resolve: Included path via `include`: /Users/dhruv/playground/ruff/lsp/play.py
  21.561742875s TRACE     ruff:main notification{method="textDocument/didChange"}: ruff_server::server::api: enter
  21.563573791s TRACE     ruff:main request{id=12 method="textDocument/diagnostic"}: ruff_server::server::api: enter
  21.564206750s DEBUG ruff:worker:4 ruff_server::resolve: Included path via `include`: /Users/dhruv/playground/ruff/lsp/play.py
  21.826691375s TRACE     ruff:main request{id=13 method="textDocument/codeAction"}: ruff_server::server::api: enter
  22.091080125s TRACE     ruff:main request{id=14 method="textDocument/codeAction"}: ruff_server::server::api: enter
```

</p>
</details> 


**Todo**

- [x] Update documentation (I'll be adding a troubleshooting section
under "Editors" as a follow-up which is for all editors)
- [x] Check for backwards compatibility. I don't think this should break
backwards compatibility as it's mainly targeted towards improving the
debugging experience.

~**Before I go on to updating the documentation, I'd appreciate initial
review on the chosen approach.**~

resolves: #14959 

## Test Plan

Refer to the test plan in
https://github.com/astral-sh/ruff-vscode/pull/659.

Example logs at `debug` level:

```
   0.010770083s DEBUG ThreadId(15) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/playground/ruff/.vscode
   0.018101916s  INFO main ruff_server::session::index: Registering workspace: /Users/dhruv/playground/ruff
   0.018559916s DEBUG ruff:worker:4 ruff_server::resolve: Included path via `include`: /Users/dhruv/playground/ruff/lsp/play.py
   0.018992375s  INFO     ruff:main ruff_server::server: Configuration file watcher successfully registered
  23.408802375s DEBUG ruff:worker:11 ruff_server::resolve: Included path via `include`: /Users/dhruv/playground/ruff/lsp/play.py
  24.329127416s DEBUG  ruff:worker:6 ruff_server::resolve: Included path via `include`: /Users/dhruv/playground/ruff/lsp/play.py
```

Example logs at `trace` level:

```
   0.010296375s DEBUG ThreadId(13) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/playground/ruff/.vscode
   0.017422583s  INFO main ruff_server::session::index: Registering workspace: /Users/dhruv/playground/ruff
   0.018034458s TRACE ruff:main notification{method="textDocument/didOpen"}: ruff_server::server::api: enter
   0.018199708s TRACE ruff:worker:0 request{id=1 method="textDocument/diagnostic"}: ruff_server::server::api: enter
   0.018251167s DEBUG ruff:worker:0 request{id=1 method="textDocument/diagnostic"}: ruff_server::resolve: Included path via `include`: /Users/dhruv/playground/ruff/lsp/play.py
   0.018528708s  INFO     ruff:main ruff_server::server: Configuration file watcher successfully registered
   1.611798417s TRACE ruff:worker:1 request{id=2 method="textDocument/codeAction"}: ruff_server::server::api: enter
   1.861757542s TRACE ruff:worker:4 request{id=3 method="textDocument/codeAction"}: ruff_server::server::api: enter
   7.027361792s TRACE ruff:worker:2 request{id=4 method="textDocument/codeAction"}: ruff_server::server::api: enter
   7.851361500s TRACE ruff:worker:5 request{id=5 method="textDocument/codeAction"}: ruff_server::server::api: enter
   7.901690875s TRACE     ruff:main notification{method="textDocument/didChange"}: ruff_server::server::api: enter
   7.903063167s TRACE ruff:worker:10 request{id=6 method="textDocument/diagnostic"}: ruff_server::server::api: enter
   7.903183500s DEBUG ruff:worker:10 request{id=6 method="textDocument/diagnostic"}: ruff_server::resolve: Included path via `include`: /Users/dhruv/playground/ruff/lsp/play.py
   8.702385292s TRACE      ruff:main notification{method="textDocument/didChange"}: ruff_server::server::api: enter
   8.704106625s TRACE  ruff:worker:3 request{id=7 method="textDocument/diagnostic"}: ruff_server::server::api: enter
   8.704304875s DEBUG  ruff:worker:3 request{id=7 method="textDocument/diagnostic"}: ruff_server::resolve: Included path via `include`: /Users/dhruv/playground/ruff/lsp/play.py
   8.966853458s TRACE  ruff:worker:9 request{id=8 method="textDocument/codeAction"}: ruff_server::server::api: enter
   9.229622792s TRACE  ruff:worker:6 request{id=9 method="textDocument/codeAction"}: ruff_server::server::api: enter
  10.513111583s TRACE  ruff:worker:7 request{id=10 method="textDocument/codeAction"}: ruff_server::server::api: enter
```
2025-01-08 18:18:00 +05:30
Alex Waygood
9a27b37a91 [flake8-builtins] Rename A005 and improve its error message (#15348) 2025-01-08 12:38:34 +00:00
Alex Waygood
487f2f5df0 Spruce up docs for pydoclint rules (#15325) 2025-01-08 12:22:37 +00:00
David Salvisberg
339167d372 [flake8-type-checking] Apply TC008 more eagerly in TYPE_CHECKING blocks and disapply it in stubs (#15180)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-01-08 12:09:06 +00:00
David Peter
235fdfc57a [red-knot] knot_extensions Python API (#15103)
## Summary

Adds a type-check-time Python API that allows us to create and
manipulate types and to test various of their properties. For example,
this can be used to write a Markdown test to make sure that `A & B` is a
subtype of `A` and `B`, but not of an unrelated class `C` (something
that requires quite a bit more code to do in Rust):
```py
from knot_extensions import Intersection, is_subtype_of, static_assert

class A: ...
class B: ...

type AB = Intersection[A, B]

static_assert(is_subtype_of(AB, A))
static_assert(is_subtype_of(AB, B))

class C: ...
static_assert(not is_subtype_of(AB, C))
```

I think this functionality is also helpful for interactive debugging
sessions, in order to query various properties of Red Knot's type
system. Which is something that otherwise requires a custom Rust unit
test, some boilerplate code and constant re-compilation.

## Test Plan

- New Markdown tests
- Tested the modified typeshed_sync workflow locally
2025-01-08 12:52:07 +01:00
Shaygan Hooshyari
03ff883626 Display Union of Literals as a Literal (#14993)
## Summary

Resolves #14988

Display union of Literals like other type checkers do.

With this change we lose the sorting behavior. And we show the types as
they appeared. So it's deterministic and tests should not be flaky.
This is similar to how Mypy [reveals the
type](https://mypy-play.net/?mypy=latest&python=3.12&gist=51ad03b153bfca3b940d5084345e230f).

In some cases this makes it harder to know what is the order in revealed
type when writing tests but since it's consistent after the test fails
we know the order.

## Test Plan

I adjusted mdtests for this change. Basically merged the int and string
types of the unions.

In cases where we have types other than numbers and strings like this
[one](https://github.com/astral-sh/ruff/pull/14993/files#diff-ac50bce02b9f0ad4dc7d6b8e1046d60dad919ac52d0aeb253e5884f89ea42bfeL51).
We only group the strings and numbers as the issue suggsted.

```
def _(flag: bool, flag2: bool):
    if flag:
        f = 1
    elif flag2:
        f = "foo"
    else:
        def f() -> int:
            return 1
    # error: "Object of type `Literal[1, "foo", f]` is not callable (due to union elements Literal[1], Literal["foo"])"
    # revealed: Unknown | int
    reveal_type(f())
```

[pyright
example](https://pyright-play.net/?code=GYJw9gtgBALgngBwJYDsDmUkQWEMoAySMApiAIYA2AUNQCYnBQD6AFMJeWgFxQBGYMJQA0UDlwBMvAUICU3alCWYm4nouWamAXigBGDUpKUkqzmimHNYqLoBEwQXavGAziQXXlDVa1lQAWgA%2BTBQYTy9rEBIYAFcQFH0rAGIoMnAQXjsAeT4AKxIAY3wwJngEEigAAyJSCkoAbT1RBydRYABdKsxXKBQwfEKqTj5KStY6WMqYMChYlCQwROMSCBIw3tqyKiaO0S36htawOw7ZZ01U6IA3EioSOl4AVRQAa36Ad0SAH1CYKxud0ozHKJHYflk1CAA)

[mypy
example](https://mypy-play.net/?mypy=latest&python=3.12&gist=31c8bdaa5521860cfeca4b92841cb3b7)

---------

Co-authored-by: Carl Meyer <carl@oddbird.net>
2025-01-08 00:58:38 +00:00
Carl Meyer
fdca2b422e [red-knot] all types are assignable to object (#15332)
## Summary

`Type[Any]` should be assignable to `object`. All types should be
assignable to `object`.

We specifically didn't understand the former; this PR adds a test for
it, and a case to ensure that `Type[Any]` is assignable to anything that
`type` is assignable to (which includes `object`).

This PR also adds a property test that all types are assignable to
object. In order to make it pass, I added a special case to check early
if we are assigning to `object` and just return `true`. In principle,
once we get all the more general cases correct, this special case might
be removable. But having the special case for now allows the property
test to pass.

And we add a property test that all types are subtypes of object. This
failed for the case of an intersection with no positive elements (that
is, a negation type). This really does need to be a special case for
`object`, because there is no other type we can know that a negation
type is a subtype of.

## Test Plan

Added unit test and property test.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-01-07 15:19:07 -08:00
Dylan
71ad9a2ab1 [ruff] Parenthesize arguments to int when removing int would change semantics in unnecessary-cast-to-int (RUF046) (#15277)
When removing `int` in calls like `int(expr)` we may need to keep
parentheses around `expr` even when it is a function call or subscript,
since there may be newlines in between the function/value name and the
opening parentheses/bracket of the argument.

This PR implements that logic.

Closes #15263

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2025-01-07 21:43:50 +00:00
InSync
3b3c2c5aa4 [eradicate] Correctly handle metadata blocks directly followed by normal blocks (ERA001) (#15330)
## Summary

Resolves #15321.

## Test Plan

`cargo nextest run` and `cargo insta test`.
2025-01-07 16:22:22 -05:00
Douglas Creager
b2a0d68d70 Narrowing for class patterns in match statements (#15223)
We now support class patterns in a match statement, adding a narrowing
constraint that within the body of that match arm, we can assume that
the subject is an instance of that class.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
Co-authored-by: Micha Reiser <micha@reiser.io>
2025-01-07 15:58:12 -05:00
Carl Meyer
f2a86fcfda [red-knot] add call checking (#15200)
## Summary

This implements checking of calls.

I ended up following Micha's original suggestion from back when the
signature representation was first introduced, and flattening it to a
single array of parameters. This turned out to be easier to manage,
because we can represent parameters using indices into that array, and
represent the bound argument types as an array of the same length.

Starred and double-starred arguments are still TODO; these won't be very
useful until we have generics.

The handling of diagnostics is just hacked into `return_ty_result`,
which was already inconsistent about whether it emitted diagnostics or
not; now it's even more inconsistent. This needs to be addressed, but
could be a follow-up.

The new benchmark errors here surface the need for intersection support
in `is_assignable_to`.

Fixes #14161.

## Test Plan

Added mdtests.
2025-01-07 20:39:45 +00:00
Alex Waygood
ac72aca27c Spruce up docs for slice-to-remove-prefix-or-suffix (FURB188) (#15328) 2025-01-07 19:58:35 +00:00
Dylan
a876090715 [internal] Return statements in finally block point to end block for unreachable (PLW0101) (#15276)
Note: `PLW0101` remains in testing rather than preview, so this PR does
not modify any public behavior (hence the title beginning with
`internal` rather than `pylint`, for the sake of the changelog.)

Fixes an error in the processing of `try` statements in the control flow
graph builder.

When processing a try statement, the block following a `return` was
forced to point to the `finally` block. However, if the return was _in_
the `finally` block, this caused the block to point to itself. In the
case where the whole `try-finally` statement was also included inside of
a loop, this caused an infinite loop in the builder for the control flow
graph as it attempted to resolve edges.

Closes #15248

## Test function
### Source
```python
def l():
    while T:
        try:
            while ():
                if 3:
                    break
        finally:
            return
```

### Control Flow Graph
```mermaid
flowchart TD
  start(("Start"))
  return(("End"))
  block0[["`*(empty)*`"]]
  block1[["Loop continue"]]
  block2["return\n"]
  block3[["Loop continue"]]
  block4["break\n"]
  block5["if 3:
                    break\n"]
  block6["while ():
                if 3:
                    break\n"]
  block7[["Exception raised"]]
  block8["try:
            while ():
                if 3:
                    break
        finally:
            return\n"]
  block9["while T:
        try:
            while ():
                if 3:
                    break
        finally:
            return\n"]
  start --> block9
  block9 -- "T" --> block8
  block9 -- "else" --> block0
  block8 -- "Exception raised" --> block7
  block8 -- "else" --> block6
  block7 --> block2
  block6 -- "()" --> block5
  block6 -- "else" --> block2
  block5 -- "3" --> block4
  block5 -- "else" --> block3
  block4 --> block2
  block3 --> block6
  block2 --> return
  block1 --> block9
  block0 --> return
```
2025-01-07 11:26:04 -06:00
InSync
e4139568b8 [ruff] Treat ) as a regex metacharacter (RUF043, RUF055) (#15318)
## Summary

Resolves #15316.

## Test Plan

`cargo nextest run` and `cargo insta test`.
2025-01-07 12:11:05 -05:00
Charlie Marsh
5567e7c26b Use uv consistently throughout the documentation (#15302)
## Summary

Closes
https://github.com/astral-sh/ruff/issues/15301#issuecomment-2573350821.
2025-01-07 14:43:25 +00:00
Alex Waygood
95294e657c [red-knot] Eagerly normalize type[] types (#15272)
Co-authored-by: Carl Meyer <carl@astral.sh>
2025-01-07 12:53:07 +00:00
InSync
0dc00e63f4 [pyupgrade] Split UP007 to two individual rules for Union and Optional (UP007, UP045) (#15313)
Co-authored-by: Micha Reiser <micha@reiser.io>
2025-01-07 10:22:59 +00:00
David Peter
ce9c4968ae [red-knot] Improve symbol-lookup tracing (#14907)
## Summary

When debugging, I frequently want to know which symbols are being looked
up. `symbol_by_id` adds tracing information, but it only shows the
`ScopedSymbolId`. Since `symbol_by_id` is only called from `symbol`, it
seems reasonable to move the tracing call one level up from
`symbol_by_id` to `symbol`, where we can also show the name of the
symbol.

**Before**:

```
6      └─┐red_knot_python_semantic::types::infer::infer_expression_types{expression=Id(60de), file=/home/shark/tomllib_modified/_parser.py}
6        └─┐red_knot_python_semantic::types::symbol_by_id{symbol=ScopedSymbolId(33)}
6        ┌─┘
6        └─┐red_knot_python_semantic::types::symbol_by_id{symbol=ScopedSymbolId(123)}
6        ┌─┘
6        └─┐red_knot_python_semantic::types::symbol_by_id{symbol=ScopedSymbolId(54)}
6        ┌─┘
6        └─┐red_knot_python_semantic::types::symbol_by_id{symbol=ScopedSymbolId(122)}
6        ┌─┘
6        └─┐red_knot_python_semantic::types::symbol_by_id{symbol=ScopedSymbolId(165)}
6        ┌─┘
6      ┌─┘
6      └─┐red_knot_python_semantic::types::symbol_by_id{symbol=ScopedSymbolId(32)}
6      ┌─┘
6      └─┐red_knot_python_semantic::types::symbol_by_id{symbol=ScopedSymbolId(232)}
6      ┌─┘
6    ┌─┘
6  ┌─┘
6┌─┘
```

**After**:

```
5      └─┐red_knot_python_semantic::types::infer::infer_expression_types{expression=Id(60de), file=/home/shark/tomllib_modified/_parser.py}
5        └─┐red_knot_python_semantic::types::symbol{name="dict"}
5        ┌─┘
5        └─┐red_knot_python_semantic::types::symbol{name="dict"}
5        ┌─┘
5        └─┐red_knot_python_semantic::types::symbol{name="list"}
5        ┌─┘
5        └─┐red_knot_python_semantic::types::symbol{name="list"}
5        ┌─┘
5        └─┐red_knot_python_semantic::types::symbol{name="isinstance"}
5        ┌─┘
5        └─┐red_knot_python_semantic::types::symbol{name="isinstance"}
5        ┌─┘
5      ┌─┘
5      └─┐red_knot_python_semantic::types::symbol{name="ValueError"}
5      ┌─┘
5      └─┐red_knot_python_semantic::types::symbol{name="ValueError"}
5      ┌─┘
5    ┌─┘
5  ┌─┘
5┌─┘
```

## Test Plan

```
cargo run --bin red_knot -- --current-directory path/to/tomllib -vvv
```
2025-01-07 10:41:27 +01:00
Raphael Gaschignard
066239fe5b [red-knot] improve type shrinking coverage in red-knot property tests (#15297)
## Summary

While looking at #14899, I looked at seeing if I could get shrinking on
the examples. It turned out to be straightforward, with a couple of
caveats.

I'm calling `clone` a lot during shrinking. Since by the shrink step
we're already looking at a test failure this feels fine? Unless I
misunderstood `quickcheck`'s core loop

When shrinking `Intersection`s, in order to just rely on `quickcheck`'s
`Vec` shrinking without thinking about it too much, the shrinking
strategy is:
- try to shrink the negative side (keeping the positive side the same)
- try to shrink the positive side (keeping the negative side the same)

This means that you can't shrink from `(A & B & ~C & ~D)` directly to
`(A & ~C)`! You would first need an intermediate failure at `(A & B &
~C)` or `(A & ~C & ~D)`. This feels good enough. Shrinking the negative
side first also has the benefit of trying to strip down negative
elements in these intersections.

## Test Plan
`cargo test -p red_knot_python_semantic -- --ignored
types::property_tests::stable` still fails as it current does on `main`,
but now the errors seem more minimal.
2025-01-07 10:09:18 +01:00
Victor Westerhuis
1e948f739c [flake8-return] Recognize functions returning Never as non-returning (RET503) (#15298)
Co-authored-by: Micha Reiser <micha@reiser.io>
2025-01-07 07:57:34 +00:00
Steve C
78e26cec02 [flake8-bugbear] Implement class-as-data-structure (B903) (#9601)
## Summary

Adds `class-as-data-structure` rule (`B903`). Also compare pylint's `too-few-public-methods` (`PLR0903`).

Took some creative liberty with this by allowing the class to have any
decorators or base classes. There are years-old issues on pylint that
don't approve of the strictness when it comes to these things.

Especially considering that dataclass is a decorator and namedtuple _can
be_ a base class. I feel ignoring those explicitly is redundant all
things considered, but it's not a hill I'm willing to die on!

See: #970 

## Test Plan

`cargo test`

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: dylwil3 <dylwil3@gmail.com>
2025-01-06 21:18:28 -06:00
Charlie Marsh
e7248ee43e Avoid treating newline-separated sections as sub-sections (#15311)
## Summary

Closes https://github.com/astral-sh/ruff/issues/15224.
2025-01-06 22:13:35 -05:00
Charlie Marsh
065274d353 Remove call when removing final argument from format (#15309)
## Summary

Closes https://github.com/astral-sh/ruff/issues/15303.
2025-01-07 02:53:42 +00:00
Charlie Marsh
75a24bbc67 Don't enforce object-without-hash-method in stubs (#15310)
## Summary

Closes https://github.com/astral-sh/ruff/issues/15292.
2025-01-07 02:51:06 +00:00
Douglas Creager
5e9259c96c Don't special-case class instances in binary expression inference (#15161)
Just like in #15045 for unary expressions: In binary expressions, we
were only looking for dunder expressions for `Type::Instance` types. We
had some special cases for coercing the various `Literal` types into
their corresponding `Instance` types before doing the lookup. But we can
side-step all of that by using the existing `Type::to_meta_type` and
`Type::to_instance` methods.
2025-01-06 13:50:20 -05:00
Alex Waygood
d45c1ee44f Upgrade zizmor to the latest version in CI (#15300)
## Summary

This PR upgrades zizmor to the latest release in our CI. zizmor is a
static analyzer checking for security issues in GitHub workflows. The
new release finds some new issues in our workflows; this PR fixes some
of the issues, and adds ignores for some other issues.

The issues fixed in this PR are new cases of zizmor's
[`template-injection`](https://woodruffw.github.io/zizmor/audits/#template-injection)
rule being emitted. The issues I'm ignoring for now are all to do with
the
[`cache-poisoning`](https://woodruffw.github.io/zizmor/audits/#cache-poisoning)
rule. The main reason I'm fixing some but ignoring others is that I'm
confident fixing the template-injection diagnostics won't have any
impact on how our workflows operate in CI, but I'm worried that fixing
the cache-poisoning diagnostics could slow down our CI a fair bit. I
don't mind if somebody else is motivated to try to fix these
diagnostics, but for now I think I'd prefer to just ignore them; it
doesn't seem high-priority enough to try to fix them right now :-)

## Test Plan

- `uvx pre-commit run -a --hook-stage=manual` passes locally
- Let's see if CI passes on this PR...
2025-01-06 15:07:46 +00:00
InSync
636288038f [ruff] Dataclass enums (RUF049) (#15299)
Co-authored-by: Micha Reiser <micha@reiser.io>
2025-01-06 14:44:20 +01:00
InSync
832c0fa04b Better error message when --config is given a table key and a non-inline-table value (#15266)
Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-01-06 13:20:28 +00:00
renovate[bot]
f29c9e48a4 Update pre-commit dependencies (#15289)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-01-06 11:35:02 +00:00
Enric Calabuig
bafe8714a8 Don't fix in ecosystem check (#15267) 2025-01-06 10:21:34 +01:00
renovate[bot]
e5270e2ac2 Update Rust crate itertools to 0.14.0 (#15287)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Micha Reiser <micha@reiser.io>
2025-01-06 07:55:32 +00:00
Avasam
643fd7fe07 Remove accidental empty block at the bottom of split-static-string (SIM905) doc (#15290)
## Summary

Removes the following empty code block

![image](https://github.com/user-attachments/assets/4adbacac-0bd9-4dac-af3a-93da2619a1cb)
2025-01-06 04:24:40 +00:00
renovate[bot]
bd02beec85 Update Rust crate clearscreen to v4 (#15288) 2025-01-05 20:06:13 -05:00
renovate[bot]
7f299fceef Update Rust crate insta to v1.42.0 (#15286) 2025-01-05 20:06:02 -05:00
renovate[bot]
6e2800df85 Update NPM Development dependencies (#15285) 2025-01-05 20:05:51 -05:00
renovate[bot]
391332a835 Update dependency uuid to v11.0.4 (#15284) 2025-01-05 20:05:45 -05:00
renovate[bot]
84e13cea14 Update dependency ruff to v0.8.6 (#15283) 2025-01-05 20:05:36 -05:00
renovate[bot]
bcb5f621c5 Update Rust crate syn to v2.0.95 (#15282) 2025-01-05 20:05:32 -05:00
renovate[bot]
47c8f1ad65 Update Rust crate matchit to v0.8.6 (#15281) 2025-01-05 20:04:02 -05:00
renovate[bot]
a4f8b9311e Update Rust crate bstr to v1.11.3 (#15280) 2025-01-05 20:03:54 -05:00
Alex Waygood
6097fd9bbe [red-knot] Future-proof Type::is_disjoint_from() (#15262) 2025-01-05 22:56:16 +00:00
Alex Waygood
0743838438 [red-knot] Improve Type::is_disjoint_from() for KnownInstanceTypes (#15261) 2025-01-05 22:49:42 +00:00
Alex Waygood
980ce941c7 [red-knot] Minor simplifications and improvements to constraint narrowing logic (#15270) 2025-01-05 21:51:22 +00:00
Shaygan Hooshyari
b26448926a Allow assigning ellipsis literal as parameter default value (#14982)
Resolves #14840

## Summary

Usage of ellipsis literal as default argument is allowed in stub files.

## Test Plan

Added mdtest for both python files and stub files.


---------

Co-authored-by: Carl Meyer <carl@oddbird.net>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-01-05 13:11:32 -06:00
Carl Meyer
2ea63620cf [red-knot] fix control flow for assignment expressions in elif tests (#15274)
## Summary

The test expression in an `elif` clause is evaluated whether or not we
take the branch. Our control flow model for if/elif chains failed to
reflect this, causing wrong inference in cases where an assignment
expression occurs inside an `elif` test expression. Our "no branch taken
yet" snapshot (which is the starting state for every new elif branch)
can't simply be the pre-if state, it must be updated after visiting each
test expression.

Once we do this, it also means we no longer need to track a vector of
narrowing constraints to reapply for each new branch, since our "branch
not taken" state (which is the initial state for each branch) is
continuously updated to include the negative narrowing constraints of
all previous branches.

Fixes #15033.

## Test Plan

Added mdtests.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-01-05 18:35:29 +00:00
InSync
00aa387d9d [refurb] Mark fix as unsafe when the right-hand side is a string (FURB171) (#15273) 2025-01-05 17:54:32 +00:00
Alex Waygood
eb82089551 [red-knot] Type::SubclassOf(SubclassOfType { base: ClassBase::Unknown }).to_instance() should be Unknown, not Any (#15269) 2025-01-05 15:14:01 +00:00
InSync
f144b9684d Add a test for overshadowing redirects (#15259)
Co-authored-by: Micha Reiser <micha@reiser.io>
2025-01-05 09:47:01 +01:00
InSync
df6e5c0293 [ruff] Recode RUF025 to RUF037 (RUF037) (#15258) 2025-01-05 09:35:08 +01:00
Alex Waygood
8f0e01787f [red-knot] Minor cleanup to Type::is_disjoint_from() and Type::is_subtype_of() (#15260) 2025-01-04 17:34:37 +00:00
Micha Reiser
6b907c1305 Ruff 0.8.6 (#15253) 2025-01-04 13:09:26 +01:00
Micha Reiser
f319531632 Make unreachable a test rule for now (#15252) 2025-01-04 12:52:08 +01:00
Micha Reiser
e4d9fe036a Revert "Add all PEP-585 names to UP006 rule" (#15250) 2025-01-04 12:23:53 +01:00
Micha Reiser
baf0d660eb Update salsa (#15243) 2025-01-03 20:04:35 +01:00
Alex Waygood
bde8ecddca [red-knot] Remove unneeded branch in Type::is_equivalent_to() (#15242)
## Summary

We understand `sys.version_info` branches now! As such, I _believe_ this
branch is no longer required; all tests pass without it. I also ran
`QUICKCHECK_TESTS=100000 cargo test -p red_knot_python_semantic --
--ignored types::property_tests::stable`, and no tests failed except for
the known issue with `Type::is_assignable_to()`
(https://github.com/astral-sh/ruff/issues/14899)

## Test Plan

See above
2025-01-03 19:04:01 +00:00
InSync
842f882ef0 [ruff] Avoid reporting when ndigits is possibly negative (RUF057) (#15234) 2025-01-03 19:48:03 +01:00
Douglas Creager
75015b0ed9 Attribute panics to the mdtests that cause them (#15241)
This updates the mdtest harness to catch any panics that occur during
type checking, and to display the panic message as an mdtest failure.
(We don't know which specific line causes the failure, so we attribute
panics to the first line of the test case.)
2025-01-03 13:45:56 -05:00
Dylan
706d87f239 Show errors for attempted fixes only when passed --verbose (#15237)
The default logging level for diagnostics includes logs written using
the `log` crate with level `error`, `warn`, and `info`. An unsuccessful
fix attached to a diagnostic via `try_set_fix` or `try_set_optional_fix`
was logged at level `error`. Note that the user would see these messages
even without passing `--fix`, and possibly also on lines with `noqa`
comments.

This PR changes the logging level here to a `debug`. We also found
ad-hoc instances of error logging in the implementations of several
rules, and have replaced those with either a `debug` or call to
`try_set{_optional}_fix`.

Closes #15229
2025-01-03 08:50:13 -06:00
w0nder1ng
0837cdd931 [RUF] Add rule to detect empty literal in deque call (RUF025) (#15104) 2025-01-03 11:57:13 +01:00
Mike Bernard
0dbfa8d0e0 TD003: remove issue code length restriction (#15175)
Co-authored-by: Micha Reiser <micha@reiser.io>
2025-01-03 10:42:04 +01:00
Micha Reiser
1218bc65ed Preserve multiline implicit concatenated strings in docstring positions (#15126) 2025-01-03 10:27:14 +01:00
InSync
6180f78da4 [pyflakes] Ignore errors in @no_type_check string annotations (F722, F821) (#15215) 2025-01-03 10:05:45 +01:00
Wei Lee
835b453bfd style(AIR302): rename removed_airflow_plugin_extension as check_airflow_plugin_extension (#15233)
## Summary

during the previous refactor, this renaming was missed

## Test Plan

no functionality changed
2025-01-03 10:37:21 +05:30
Auguste Lalande
a3d873ef66 [pylint] Re-implement unreachable (PLW0101) (#10891)
## Summary

This PR re-introduces the control-flow graph implementation which was
first introduced in #5384, and then removed in #9463 due to not being
feature complete. Mainly, it lacked the ability to process
`try`-`except` blocks, along with some more minor bugs.

Closes #8958 and #8959 and #14881.

## Overview of Changes

I will now highlight the major changes implemented in this PR, in order
of implementation.

1. Introduced a post-processing step in loop handling to find any
`continue` or `break` statements within the loop body and redirect them
appropriately.
2. Introduced a loop-continue block which is always placed at the end of
loop blocks, and ensures proper looping regardless of the internal logic
of the block. This resolves #8958.
3. Implemented `try` processing with the following logic (resolves
#8959):
1. In the example below the cfg first encounters a conditional
`ExceptionRaised` forking if an exception was (or will be) raised in the
try block. This is not possible to know (except for trivial cases) so we
assume both paths can be taken unconditionally.
2. Going down the `try` path the cfg goes `try`->`else`->`finally`
unconditionally.
3. Going down the `except` path the cfg will meet several conditional
`ExceptionCaught` which fork depending on the nature of the exception
caught. Again there's no way to know which exceptions may be raised so
both paths are assumed to be taken unconditionally.
4. If none of the exception blocks catch the exception then the cfg
terminates by raising a new exception.
5. A post-processing step is also implemented to redirect any `raises`
or `returns` within the blocks appropriately.
```python
def func():
    try:
        print("try")
    except Exception:
        print("Exception")
    except OtherException as e:
        print("OtherException")
    else:
        print("else")
    finally:
        print("finally")
```
```mermaid
flowchart TD
  start(("Start"))
  return(("End"))
  block0[["`*(empty)*`"]]
  block1["print(#quot;finally#quot;)\n"]
  block2["print(#quot;else#quot;)\n"]
  block3["print(#quot;try#quot;)\n"]
  block4[["Exception raised"]]
  block5["print(#quot;OtherException#quot;)\n"]
  block6["try:
        print(#quot;try#quot;)
    except Exception:
        print(#quot;Exception#quot;)
    except OtherException as e:
        print(#quot;OtherException#quot;)
    else:
        print(#quot;else#quot;)
    finally:
        print(#quot;finally#quot;)\n"]
  block7["print(#quot;Exception#quot;)\n"]
  block8["try:
        print(#quot;try#quot;)
    except Exception:
        print(#quot;Exception#quot;)
    except OtherException as e:
        print(#quot;OtherException#quot;)
    else:
        print(#quot;else#quot;)
    finally:
        print(#quot;finally#quot;)\n"]
  block9["try:
        print(#quot;try#quot;)
    except Exception:
        print(#quot;Exception#quot;)
    except OtherException as e:
        print(#quot;OtherException#quot;)
    else:
        print(#quot;else#quot;)
    finally:
        print(#quot;finally#quot;)\n"]

  start --> block9
  block9 -- "Exception raised" --> block8
  block9 -- "else" --> block3
  block8 -- "Exception" --> block7
  block8 -- "else" --> block6
  block7 --> block1
  block6 -- "OtherException" --> block5
  block6 -- "else" --> block4
  block5 --> block1
  block4 --> return
  block3 --> block2
  block2 --> block1
  block1 --> block0
  block0 --> return
``` 
6. Implemented `with` processing with the following logic:
1. `with` statements have no conditional execution (apart from the
hidden logic handling the enter and exit), so the block is assumed to
execute unconditionally.
2. The one exception is that exceptions raised within the block may
result in control flow resuming at the end of the block. Since it is not
possible know if an exception will be raised, or if it will be handled
by the context manager, we assume that execution always continues after
`with` blocks even if the blocks contain `raise` or `return` statements.
This is handled in a post-processing step.

## Test Plan

Additional test fixtures and control-flow fixtures were added.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: dylwil3 <dylwil3@gmail.com>
2025-01-02 21:54:59 -06:00
Wei Lee
d464ef67cf refactor(AIR303): move duplicate qualified_name.to_string() to Diagnostic argument (#15220)
## Summary

Refactor airflow rule logic like
86bdc2e7b1

## Test Plan

No functionality change. Existing test cases work as it was
2025-01-03 09:10:37 +05:30
Charlie Marsh
2355472d61 Misc. clean up to rounding rules (#15231) 2025-01-02 17:51:35 -05:00
Charlie Marsh
3c3f35a548 Avoid syntax error when removing int over multiple lines (#15230)
## Summary

Closes https://github.com/astral-sh/ruff/issues/15226.
2025-01-02 17:43:15 -05:00
renovate[bot]
2327082c43 Migrate renovate config (#15228)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2025-01-02 21:53:45 +00:00
David Peter
7671a3bbc7 Remove Type::tuple in favor of TupleType::from_elements (#15218)
## Summary

Remove `Type::tuple` in favor of `TupleType::from_elements`, avoid a few
intermediate `Vec`tors. Resolves an old [review
comment](https://github.com/astral-sh/ruff/pull/14744#discussion_r1867493706).

## Test Plan

New regression test for something I ran into while implementing this.
2025-01-02 17:22:32 +01:00
Dhruv Manilawala
11e873eb45 Bump version to 0.8.5 (#15219) 2025-01-02 17:21:21 +05:30
InSync
89ea0371a4 [ruff] Unnecessary rounding (RUF057) (#14828)
Co-authored-by: Micha Reiser <micha@reiser.io>
2025-01-02 10:00:57 +01:00
Wei Lee
f8c9665742 [airflow] Extend names moved from core to provider (AIR303) (#15216)
## Summary

Many core Airflow features have been deprecated and moved to Airflow
Providers since users might need to install an additional package (e.g.,
`apache-airflow-provider-fab==1.0.0`); a separate rule (AIR303) is
created for this.

* `airflow.kubernetes.kubernetes_helper_functions.add_pod_suffix` →
`airflow.providers.cncf.kubernetes.kubernetes_helper_functions.add_pod_suffix`
*
`airflow.kubernetes.kubernetes_helper_functions.annotations_for_logging_task_metadata`
→
`airflow.providers.cncf.kubernetes.kubernetes_helper_functions.annotations_for_logging_task_metadata`
* `airflow.kubernetes.kubernetes_helper_functions.annotations_to_key` →
`airflow.providers.cncf.kubernetes.kubernetes_helper_functions.annotations_to_key`
* `airflow.kubernetes.kubernetes_helper_functions.create_pod_id` →
`airflow.providers.cncf.kubernetes.kubernetes_helper_functions.create_pod_id`
*
`airflow.kubernetes.kubernetes_helper_functions.get_logs_task_metadata`
→
`airflow.providers.cncf.kubernetes.kubernetes_helper_functions.get_logs_task_metadata`
* `airflow.kubernetes.kubernetes_helper_functions.rand_str` →
`airflow.providers.cncf.kubernetes.kubernetes_helper_functions.rand_str`
* `airflow.kubernetes.pod.Port` →
`kubernetes.client.models.V1ContainerPort`
* `airflow.kubernetes.pod.Resources` →
`kubernetes.client.models.V1ResourceRequirements`
* `airflow.kubernetes.pod_launcher.PodLauncher` →
`airflow.providers.cncf.kubernetes.pod_launcher.PodLauncher`
* `airflow.kubernetes.pod_launcher.PodStatus` →
`airflow.providers.cncf.kubernetes.pod_launcher.PodStatus`
* `airflow.kubernetes.pod_launcher_deprecated.PodLauncher` →
`airflow.providers.cncf.kubernetes.pod_launcher_deprecated.PodLauncher`
* `airflow.kubernetes.pod_launcher_deprecated.PodStatus` →
`airflow.providers.cncf.kubernetes.pod_launcher_deprecated.PodStatus`
* `airflow.kubernetes.pod_launcher_deprecated.get_kube_client` →
`airflow.providers.cncf.kubernetes.kube_client.get_kube_client`
* `airflow.kubernetes.pod_launcher_deprecated.PodDefaults` →
`airflow.providers.cncf.kubernetes.pod_generator_deprecated.PodDefaults`
* `airflow.kubernetes.pod_runtime_info_env.PodRuntimeInfoEnv` →
`kubernetes.client.models.V1EnvVar`
* `airflow.kubernetes.volume.Volume` →
`kubernetes.client.models.V1Volume`
* `airflow.kubernetes.volume_mount.VolumeMount` →
`kubernetes.client.models.V1VolumeMount`
* `airflow.kubernetes.k8s_model.K8SModel` →
`airflow.providers.cncf.kubernetes.k8s_model.K8SModel`
* `airflow.kubernetes.k8s_model.append_to_pod` →
`airflow.providers.cncf.kubernetes.k8s_model.append_to_pod`
* `airflow.kubernetes.kube_client._disable_verify_ssl` →
`airflow.kubernetes.airflow.providers.cncf.kubernetes.kube_client._disable_verify_ssl`
* `airflow.kubernetes.kube_client._enable_tcp_keepalive` →
`airflow.kubernetes.airflow.providers.cncf.kubernetes.kube_client._enable_tcp_keepalive`
* `airflow.kubernetes.kube_client.get_kube_client` →
`airflow.kubernetes.airflow.providers.cncf.kubernetes.kube_client.get_kube_client`
* `airflow.kubernetes.pod_generator.datetime_to_label_safe_datestring` →
`airflow.providers.cncf.kubernetes.pod_generator.datetime_to_label_safe_datestring`
* `airflow.kubernetes.pod_generator.extend_object_field` →
`airflow.kubernetes.airflow.providers.cncf.kubernetes.pod_generator.extend_object_field`
* `airflow.kubernetes.pod_generator.label_safe_datestring_to_datetime` →
`airflow.providers.cncf.kubernetes.pod_generator.label_safe_datestring_to_datetime`
* `airflow.kubernetes.pod_generator.make_safe_label_value` →
`airflow.providers.cncf.kubernetes.pod_generator.make_safe_label_value`
* `airflow.kubernetes.pod_generator.merge_objects` →
`airflow.providers.cncf.kubernetes.pod_generator.merge_objects`
* `airflow.kubernetes.pod_generator.PodGenerator` →
`airflow.providers.cncf.kubernetes.pod_generator.PodGenerator`
* `airflow.kubernetes.pod_generator.PodGeneratorDeprecated` →
`airflow.providers.cncf.kubernetes.pod_generator.PodGenerator`
* `airflow.kubernetes.pod_generator.PodDefaults` →
`airflow.providers.cncf.kubernetes.pod_generator_deprecated.PodDefaults`
* `airflow.kubernetes.pod_generator.add_pod_suffix` →
`airflow.providers.cncf.kubernetes.kubernetes_helper_functions.add_pod_suffix`
* `airflow.kubernetes.pod_generator.rand_str` →
`airflow.providers.cncf.kubernetes.kubernetes_helper_functions.rand_str`
* `airflow.kubernetes.pod_generator_deprecated.make_safe_label_value` →
`airflow.providers.cncf.kubernetes.pod_generator_deprecated.make_safe_label_value`
* `airflow.kubernetes.pod_generator_deprecated.PodDefaults` →
`airflow.providers.cncf.kubernetes.pod_generator_deprecated.PodDefaults`
* `airflow.kubernetes.pod_generator_deprecated.PodGenerator` →
`airflow.providers.cncf.kubernetes.pod_generator_deprecated.PodGenerator`
* `airflow.kubernetes.secret.Secret` →
`airflow.providers.cncf.kubernetes.secret.Secret`
* `airflow.kubernetes.secret.K8SModel` →
`airflow.providers.cncf.kubernetes.k8s_model.K8SModel`

## Test Plan

A test fixture has been included for the rule.
2025-01-02 10:30:12 +05:30
InSync
af95f6b577 [pycodestyle] Avoid false positives and negatives related to type parameter default syntax (E225, E251) (#15214) 2025-01-01 11:28:25 +01:00
github-actions[bot]
79682a28b8 Sync vendored typeshed stubs (#15213)
Co-authored-by: typeshedbot <>
2025-01-01 01:14:40 +00:00
David Salvisberg
1ef0f615f1 [flake8-type-checking] Improve flexibility of runtime-evaluated-decorators (#15204)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-12-31 16:28:10 +00:00
Christian Clauss
7ca3f9515c Update references to astral-sh/ruff-action from v2 to v3 (#15212) 2024-12-31 16:43:17 +01:00
Wei Lee
32de5801f7 [airflow]: extend names moved from core to provider (AIR303) (#15196) 2024-12-31 15:16:07 +01:00
InSync
cfd6093579 [pydocstyle] Add setting to ignore missing documentation for*args and **kwargs parameters (D417) (#15210)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-12-31 12:16:55 +01:00
Arnav Gupta
3c9021ffcb [ruff] Implement falsy-dict-get-fallback (RUF056) (#15160)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-12-31 11:40:51 +01:00
Wei Lee
68d2466832 [airflow] Remove additional spaces (AIR302) (#15211)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

During https://github.com/astral-sh/ruff/pull/15209, additional spaces
was accidentally added to the rule
`airflow.operators.latest_only.LatestOnlyOperator`. This PR fixes this
issue

## Test Plan

<!-- How was it tested? -->

A test fixture has been included for the rule.
2024-12-31 13:58:11 +05:30
Avasam
ecf00cdf6a Fix incorrect doc in shebang-not-executable (EXE001) and add git+windows solution to executable bit (#15208)
## Summary


I noticed that the solution mentioned in [shebang-not-executable
(EXE001)](https://docs.astral.sh/ruff/rules/shebang-not-executable/#shebang-not-executable-exe001)
was incorrect and likely copy-pasted from
[shebang-missing-executable-file
(EXE002)](https://docs.astral.sh/ruff/rules/shebang-missing-executable-file/#shebang-missing-executable-file-exe002)

It was telling users to remove the executable bit from a non-executable
file. Which does nothing.

I also noticed locally that:
- `chmod` wouldn't cause any file change to be noticed by git (`EXE` was
also passing locally) under WSL
- Using git allows anyone to fix this lint across OSes, for projects
with CIs using git

So I added a solution using [git update-index
--chmod](https://git-scm.com/docs/git-update-index#Documentation/git-update-index.txt---chmod-x)

## Test Plan

No test plan, doc changes only.
As for running the chmod commands:
https://github.com/python/typeshed/pull/13346
2024-12-31 11:05:44 +05:30
Dhruv Manilawala
86bdc2e7b1 Refactor Airflow removal in 3 code (#15209)
This PR contains a couple of refactors for the Airflow removal in 3
code, nothing major just some minor nits.
2024-12-31 05:27:12 +00:00
Wei Lee
253c274afa [airflow] Extend rule to check class attributes, methods, arguments (AIR302) (#15083)
## Summary

Airflow 3.0 removes various deprecated functions, members, modules, and
other values. They have been deprecated in 2.x, but the removal causes
incompatibilities that we want to detect. This PR add rules for the
following.

* Removed class attribute
* `airflow.providers_manager.ProvidersManager.dataset_factories` →
`airflow.providers_manager.ProvidersManager.asset_factories`
* `airflow.providers_manager.ProvidersManager.dataset_uri_handlers` →
`airflow.providers_manager.ProvidersManager.asset_uri_handlers`
*
`airflow.providers_manager.ProvidersManager.dataset_to_openlineage_converters`
→
`airflow.providers_manager.ProvidersManager.asset_to_openlineage_converters`
* `airflow.lineage.hook.DatasetLineageInfo.dataset` →
`airflow.lineage.hook.AssetLineageInfo.asset`
* Removed class method (subclasses in airflow should also checked)
* `airflow.secrets.base_secrets.BaseSecretsBackend.get_conn_uri` →
`airflow.secrets.base_secrets.BaseSecretsBackend.get_conn_value`
* `airflow.secrets.base_secrets.BaseSecretsBackend.get_connections` →
`airflow.secrets.base_secrets.BaseSecretsBackend.get_connection`
* `airflow.hooks.base.BaseHook.get_connections` → use `get_connection`
* `airflow.datasets.BaseDataset.iter_datasets` →
`airflow.sdk.definitions.asset.BaseAsset.iter_assets`
* `airflow.datasets.BaseDataset.iter_dataset_aliases` →
`airflow.sdk.definitions.asset.BaseAsset.iter_asset_aliases`
* Removed constructor args (subclasses in airflow should also checked)
* argument `filename_template`
in`airflow.utils.log.file_task_handler.FileTaskHandler`
    * in `BaseOperator`
        * `sla`
        * `task_concurrency` → `max_active_tis_per_dag`
    * in `BaseAuthManager`
        * `appbuilder`
* Removed class variable (subclasses anywhere should be checked)
    * in `airflow.plugins_manager.AirflowPlugin`
        * `executors` (from #43289)
        * `hooks`
        * `operators`
        * `sensors`
* Replaced names
	* `airflow.hooks.base_hook.BaseHook` → `airflow.hooks.base.BaseHook`
* `airflow.operators.dagrun_operator.TriggerDagRunLink` →
`airflow.operators.trigger_dagrun.TriggerDagRunLink`
* `airflow.operators.dagrun_operator.TriggerDagRunOperator` →
`airflow.operators.trigger_dagrun.TriggerDagRunOperator`
* `airflow.operators.python_operator.BranchPythonOperator` →
`airflow.operators.python.BranchPythonOperator`
* `airflow.operators.python_operator.PythonOperator` →
`airflow.operators.python.PythonOperator`
* `airflow.operators.python_operator.PythonVirtualenvOperator` →
`airflow.operators.python.PythonVirtualenvOperator`
* `airflow.operators.python_operator.ShortCircuitOperator` →
`airflow.operators.python.ShortCircuitOperator`
* `airflow.operators.latest_only_operator.LatestOnlyOperator` →
`airflow.operators.latest_only.LatestOnlyOperator`


In additional to the changes above, this PR also add utility functions
and improve docstring.


## Test Plan

A test fixture is included in the PR.
2024-12-31 09:49:18 +05:30
InSync
2a1aa29366 [pylint] Detect nested methods correctly (PLW1641) (#15032)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-12-30 16:55:14 +01:00
purajit
42cc67a87c [docs] improve and fix entry for analyze.include-dependencies (#15197)
## Summary

Changes two things about the entry:
* make the example valid TOML - inline tables must be a single line, at
least till v1.1.0 is released,
but also while in the future the toml version used by ruff might handle
it, it would probably be
good to stick to a spec that's readable by the vast majority of other
tools and versions as well,
especially if people are using `pyproject.toml`. The current example
leads to `ruff` failure.
See https://github.com/toml-lang/toml/pull/904
* adds a line about the ability to add non-Python files to the map,
which I think is a specific and
important feature people should know about (in fact, I would assume this
could potentially
become the single biggest use-case for this).

## Test Plan

Ran doc creation as described in the
[contribution](https://docs.astral.sh/ruff/contributing/#mkdocs) guide.

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-12-30 15:45:19 +00:00
InSync
280ba75100 [flake8-pie] Allow cast(SomeType, ...) (PIE796) (#15141) 2024-12-30 15:52:35 +01:00
wookie184
04d538113a Add all PEP-585 names to UP006 rule (#5454)
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
Co-authored-by: dylwil3 <dylwil3@gmail.com>
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-12-30 12:21:42 +01:00
InSync
0b15f17939 [flake8-simplify] More precise inference for dictionaries (SIM300) (#15164)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-12-30 10:41:33 +00:00
Micha Reiser
0caab81d3d @no_type_check support (#15122)
Co-authored-by: Carl Meyer <carl@astral.sh>
2024-12-30 09:42:18 +00:00
InSync
d4ee6abf4a Visit PEP 764 inline TypedDicts' keys as non-type-expressions (#15073)
## Summary

Resolves #10812.

## Test Plan

`cargo nextest run` and `cargo insta test`.
2024-12-30 15:04:55 +05:30
Dhruv Manilawala
8a98d88847 [red-knot] Add diagnostic for invalid unpacking (#15086)
## Summary

Part of #13773 

This PR adds diagnostics when there is a length mismatch during
unpacking between the number of target expressions and the number of
types for the unpack value expression.

There are 3 cases of diagnostics here where the first two occurs when
there isn't a starred expression and the last one occurs when there's a
starred expression:
1. Number of target expressions is **less** than the number of types
that needs to be unpacked
2. Number of target expressions is **greater** then the number of types
that needs to be unpacked
3. When there's a starred expression as one of the target expression and
the number of target expressions is greater than the number of types

Examples for all each of the above cases:
```py
# red-knot: Too many values to unpack (expected 2, got 3) [lint:invalid-assignment]
a, b = (1, 2, 3)

# red-knot: Not enough values to unpack (expected 2, got 1) [lint:invalid-assignment]
a, b = (1,)

# red-knot: Not enough values to unpack (expected 3 or more, got 2) [lint:invalid-assignment]
a, *b, c, d = (1, 2)
```

The (3) case is a bit special because it uses a distinct wording
"expected n or more" instead of "expected n" because of the starred
expression.

### Location

The diagnostic location is the target expression that's being unpacked.
For nested targets, the location will be the nested expression. For
example:

```py
(a, (b, c), d) = (1, (2, 3, 4), 5)
#   ^^^^^^
#   red-knot: Too many values to unpack (expected 2, got 3) [lint:invalid-assignment]
```

For future improvements, it would be useful to show the context for why
this unpacking failed. For example, for why the expected number of
targets is `n`, we can highlight the relevant elements for the value
expression.

In the **ecosystem**, **Pyright** uses the target expressions for
location while **mypy** uses the value expression for the location. For
example:

```py
if 1:
#          mypy: Too many values to unpack (2 expected, 3 provided)  [misc]
#          vvvvvvvvv
	a, b = (1, 2, 3)
#   ^^^^
#   Pyright: Expression with type "tuple[Literal[1], Literal[2], Literal[3]]" cannot be assigned to target tuple
#     Type "tuple[Literal[1], Literal[2], Literal[3]]" is incompatible with target tuple
#       Tuple size mismatch; expected 2 but received 3 [reportAssignmentType]
#   red-knot: Too many values to unpack (expected 2, got 3) [lint:invalid-assignment]
```

## Test Plan

Update existing test cases TODO with the error directives.
2024-12-30 13:10:29 +05:30
InSync
901b7dd8f8 [flake8-use-pathlib] Catch redundant joins in PTH201 and avoid syntax errors (#15177)
## Summary

Resolves #10453, resolves #15165.

## Test Plan

`cargo nextest run` and `cargo insta test`.
2024-12-30 03:31:35 +00:00
renovate[bot]
d3492178e1 Update Rust crate glob to v0.3.2 (#15185) 2024-12-29 21:29:46 -05:00
renovate[bot]
383f6d0967 Update astral-sh/setup-uv action to v5 (#15193) 2024-12-29 21:29:37 -05:00
renovate[bot]
e953ecf42f Update dependency mdformat-mkdocs to v4.1.1 (#15192) 2024-12-29 21:29:33 -05:00
renovate[bot]
e8cf2d3027 Update Rust crate serde_with to v3.12.0 (#15191) 2024-12-29 21:29:27 -05:00
renovate[bot]
0701437104 Update NPM Development dependencies (#15190) 2024-12-29 21:29:23 -05:00
renovate[bot]
db0468bae5 Update pre-commit hook rhysd/actionlint to v1.7.5 (#15189) 2024-12-29 21:29:19 -05:00
renovate[bot]
7e491c4d94 Update Rust crate syn to v2.0.93 (#15188) 2024-12-29 21:29:15 -05:00
renovate[bot]
957df82400 Update Rust crate serde to v1.0.217 (#15187) 2024-12-29 21:29:11 -05:00
renovate[bot]
9ba2fb0a48 Update Rust crate quote to v1.0.38 (#15186) 2024-12-29 21:29:04 -05:00
renovate[bot]
8ff9cb75cd Update Rust crate compact_str to v0.8.1 (#15184) 2024-12-29 21:28:56 -05:00
David Salvisberg
f170932585 [flake8-type-checking] Disable TC006 & TC007 in stub files (#15179)
Fixes: #15176

## Summary

Neither of these rules make any sense in stub files. Technically TC007
should already not have triggered, due to the typing only context of the
binding, but it's better to be explicit.

Keeping TC008 enabled on the other hand makes sense to me, although we
could probably be more aggressive with unquoting in a typing runtime
context.

## Test Plan

`cargo nextest run`
2024-12-29 14:39:16 -05:00
Shantanu
bc3a735d93 Test explicit shadowing involving defs (#15174)
Co-authored-by: Carl Meyer <carl@astral.sh>
2024-12-29 00:47:03 +00:00
Victorien
7ea3a549b2 Fix typo in NameImport.qualified_name docstring (#15170) 2024-12-28 23:44:01 +01:00
Wei Lee
2288cc7478 [airflow]: extend names moved from core to provider (AIR303) (#15159) 2024-12-27 17:04:02 +00:00
Enoch Kan
79816f965c Fix SyntaxError in example replacement snippet in nonlocal-without-binding (#15157) 2024-12-27 12:15:13 +00:00
InSync
8d2d1a73c5 [red-knot] Report classes inheriting from bases with incompatible __slots__ (#15129) 2024-12-27 11:43:48 +00:00
Victorien
2419fdb2ef Fix typo in LogicalLine docstring (#15150) 2024-12-26 18:45:45 +01:00
Micha Reiser
6ed27c3786 Rename the knot|type-ignore mdtest files (#15147) 2024-12-26 10:25:05 +00:00
Wei Lee
5d6aae839e [airflow]: extend moved names (AIR303) (#15145) 2024-12-26 10:55:07 +01:00
sobolevn
8b9c843c72 Fix docs highlight in dict_iter_missing_items.rs (#15140) 2024-12-25 18:52:17 +01:00
Enoch Kan
5bc9d6d3aa Rename rules currently not conforming to naming convention (#15102)
## Summary

This pull request renames 19 rules which currently do not conform to
Ruff's [naming
convention](https://github.com/astral-sh/ruff/blob/main/CONTRIBUTING.md#rule-naming-convention).

## Description

Fixes astral-sh/ruff#15009.
2024-12-23 15:48:45 -06:00
Micha Reiser
97965ff114 Rename --current-directory to --project in Red Knot benchmark script (#15124) 2024-12-23 12:50:35 +00:00
Micha Reiser
8d327087ef Add invalid-ignore-comment rule (#15094) 2024-12-23 10:38:10 +00:00
Micha Reiser
2835d94ec5 Add unknown-rule (#15085)
Co-authored-by: Carl Meyer <carl@astral.sh>
2024-12-23 11:30:54 +01:00
Dhruv Manilawala
68ada05b00 [red-knot] Infer value expr for empty list / tuple target (#15121)
## Summary

This PR resolves
https://github.com/astral-sh/ruff/pull/15058#discussion_r1893868406 by
inferring the value expression even if there are no targets in the list
/ tuple expression.

## Test Plan

Remove TODO from corpus tests, making sure it doesn't panic.
2024-12-23 16:00:35 +05:30
Micha Reiser
2a99c0be02 Add unused-ignore-comment rule (#15084) 2024-12-23 11:15:28 +01:00
Micha Reiser
dcb85b7088 Update benchmark scripts, use uv (#15120) 2024-12-23 11:14:15 +01:00
Harutaka Kawamura
8440f3ea9f [fastapi] Update FAST002 to check keyword-only arguments (#15119)
## Summary

Close #15117. Update `FAST002` to check keyword-only arguments.

## Test Plan

New test case
2024-12-23 15:32:42 +05:30
Micha Reiser
1c3d11e8a8 Support file-level type: ignore comments (#15081) 2024-12-23 09:59:04 +00:00
Micha Reiser
2f85749fa0 type: ignore[codes] and knot: ignore (#15078) 2024-12-23 10:52:43 +01:00
InSync
9eb73cb7e0 [pycodestyle] Preserve original value format (E731) (#15097)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-12-23 09:29:46 +00:00
Dhruv Manilawala
03bb9425df [red-knot] Avoid Ranged for definition target range (#15118)
## Summary

Ref:
3533d7f5b4 (r150651102)

This PR removes the `Ranged` implementation on `DefinitionKind` and
instead uses a method called `target_range` to avoid any confusion about
what range this is for i.e., it's not the range of the node that
represents the definition.
2024-12-23 14:07:09 +05:30
Dhruv Manilawala
113c804a62 [red-knot] Add support for unpacking for target (#15058)
## Summary

Related to #13773 

This PR adds support for unpacking `for` statement targets.

This involves updating the `value` field in the `Unpack` target to use
an enum which specifies the "where did the value expression came from?".
This is because for an iterable expression, we need to unpack the
iterator type while for assignment statement we need to unpack the value
type itself. And, this needs to be done in the unpack query.

### Question

One of the ways unpacking works in `for` statement is by looking at the
union of the types because if the iterable expression is a tuple then
the iterator type will be union of all the types in the tuple. This
means that the test cases that will test the unpacking in `for`
statement will also implicitly test the unpacking union logic. I was
wondering if it makes sense to merge these cases and only add the ones
that are specific to the union unpacking or for statement unpacking
logic.

## Test Plan

Add test cases involving iterating over a tuple type. I've intentionally
left out certain cases for now and I'm curious to know any thoughts on
the above query.
2024-12-23 06:13:49 +00:00
Arnav Gupta
b6c8f5d79e Fix RUF200 doc to have name and email in single object (#15099)
## Summary
Closes #14975 by modifying the docstring of the InvalidPyprojectToml
rule. Previously the docs were incorrectly stating that author name and
emails must be individual items in the authors list, rather than part of
a single object for each respective author.

## Test Plan
This was a docstring change, no tests needed.
2024-12-23 10:30:16 +05:30
renovate[bot]
da8acabc55 Update dependency mdformat to v0.7.21 (#15113)
This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [mdformat](https://redirect.github.com/hukkin/mdformat)
([changelog](https://mdformat.readthedocs.io/en/stable/users/changelog.html))
| `==0.7.19` -> `==0.7.21` |
[![age](https://developer.mend.io/api/mc/badges/age/pypi/mdformat/0.7.21?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/mdformat/0.7.21?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/mdformat/0.7.19/0.7.21?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/mdformat/0.7.19/0.7.21?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|

---

### Release Notes

<details>
<summary>hukkin/mdformat (mdformat)</summary>

###
[`v0.7.21`](https://redirect.github.com/hukkin/mdformat/compare/0.7.20...0.7.21)

[Compare
Source](https://redirect.github.com/hukkin/mdformat/compare/0.7.20...0.7.21)

###
[`v0.7.20`](https://redirect.github.com/hukkin/mdformat/compare/0.7.19...0.7.20)

[Compare
Source](https://redirect.github.com/hukkin/mdformat/compare/0.7.19...0.7.20)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "before 4am on Monday" (UTC),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/astral-sh/ruff).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS44MC4wIiwidXBkYXRlZEluVmVyIjoiMzkuODAuMCIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiaW50ZXJuYWwiXX0=-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-12-23 10:14:06 +05:30
renovate[bot]
fd2b8deddd Update dependency ruff to v0.8.4 (#15114)
This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [ruff](https://docs.astral.sh/ruff)
([source](https://redirect.github.com/astral-sh/ruff),
[changelog](https://redirect.github.com/astral-sh/ruff/blob/main/CHANGELOG.md))
| `==0.8.3` -> `==0.8.4` |
[![age](https://developer.mend.io/api/mc/badges/age/pypi/ruff/0.8.4?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/ruff/0.8.4?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/ruff/0.8.3/0.8.4?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/ruff/0.8.3/0.8.4?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|

---

### Release Notes

<details>
<summary>astral-sh/ruff (ruff)</summary>

###
[`v0.8.4`](https://redirect.github.com/astral-sh/ruff/blob/HEAD/CHANGELOG.md#084)

[Compare
Source](https://redirect.github.com/astral-sh/ruff/compare/0.8.3...0.8.4)

##### Preview features

- \[`airflow`] Extend `AIR302` with additional functions and classes
([#&#8203;15015](https://redirect.github.com/astral-sh/ruff/pull/15015))
- \[`airflow`] Implement `moved-to-provider-in-3` for modules that has
been moved to Airflow providers (`AIR303`)
([#&#8203;14764](https://redirect.github.com/astral-sh/ruff/pull/14764))
- \[`flake8-use-pathlib`] Extend check for invalid path suffix to
include the case `"."` (`PTH210`)
([#&#8203;14902](https://redirect.github.com/astral-sh/ruff/pull/14902))
- \[`perflint`] Fix panic in `PERF401` when list variable is after the
`for` loop
([#&#8203;14971](https://redirect.github.com/astral-sh/ruff/pull/14971))
- \[`perflint`] Simplify finding the loop target in `PERF401`
([#&#8203;15025](https://redirect.github.com/astral-sh/ruff/pull/15025))
- \[`pylint`] Preserve original value format (`PLR6104`)
([#&#8203;14978](https://redirect.github.com/astral-sh/ruff/pull/14978))
- \[`ruff`] Avoid false positives for `RUF027` for typing context
bindings
([#&#8203;15037](https://redirect.github.com/astral-sh/ruff/pull/15037))
- \[`ruff`] Check for ambiguous pattern passed to `pytest.raises()`
(`RUF043`)
([#&#8203;14966](https://redirect.github.com/astral-sh/ruff/pull/14966))

##### Rule changes

- \[`flake8-bandit`] Check `S105` for annotated assignment
([#&#8203;15059](https://redirect.github.com/astral-sh/ruff/pull/15059))
- \[`flake8-pyi`] More autofixes for `redundant-none-literal` (`PYI061`)
([#&#8203;14872](https://redirect.github.com/astral-sh/ruff/pull/14872))
- \[`pydocstyle`] Skip leading whitespace for `D403`
([#&#8203;14963](https://redirect.github.com/astral-sh/ruff/pull/14963))
- \[`ruff`] Skip `SQLModel` base classes for `mutable-class-default`
(`RUF012`)
([#&#8203;14949](https://redirect.github.com/astral-sh/ruff/pull/14949))

##### Bug

- \[`perflint`] Parenthesize walrus expressions in autofix for
`manual-list-comprehension` (`PERF401`)
([#&#8203;15050](https://redirect.github.com/astral-sh/ruff/pull/15050))

##### Server

- Check diagnostic refresh support from client capability which enables
dynamic configuration for various editors
([#&#8203;15014](https://redirect.github.com/astral-sh/ruff/pull/15014))

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "before 4am on Monday" (UTC),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/astral-sh/ruff).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS44MC4wIiwidXBkYXRlZEluVmVyIjoiMzkuODAuMCIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiaW50ZXJuYWwiXX0=-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-12-23 10:12:07 +05:30
renovate[bot]
efbadbd966 Update pre-commit dependencies (#15115)
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
|
[astral-sh/ruff-pre-commit](https://redirect.github.com/astral-sh/ruff-pre-commit)
| repository | patch | `v0.8.3` -> `v0.8.4` |
| [crate-ci/typos](https://redirect.github.com/crate-ci/typos) |
repository | patch | `v1.28.3` -> `v1.28.4` |
|
[executablebooks/mdformat](https://redirect.github.com/executablebooks/mdformat)
| repository | patch | `0.7.19` -> `0.7.21` |
|
[woodruffw/zizmor-pre-commit](https://redirect.github.com/woodruffw/zizmor-pre-commit)
| repository | minor | `v0.9.2` -> `v0.10.0` |

Note: The `pre-commit` manager in Renovate is not supported by the
`pre-commit` maintainers or community. Please do not report any problems
there, instead [create a Discussion in the Renovate
repository](https://redirect.github.com/renovatebot/renovate/discussions/new)
if you have any questions.

---

### Release Notes

<details>
<summary>astral-sh/ruff-pre-commit (astral-sh/ruff-pre-commit)</summary>

###
[`v0.8.4`](https://redirect.github.com/astral-sh/ruff-pre-commit/compare/v0.8.3...v0.8.4)

[Compare
Source](https://redirect.github.com/astral-sh/ruff-pre-commit/compare/v0.8.3...v0.8.4)

</details>

<details>
<summary>crate-ci/typos (crate-ci/typos)</summary>

###
[`v1.28.4`](https://redirect.github.com/crate-ci/typos/releases/tag/v1.28.4)

[Compare
Source](https://redirect.github.com/crate-ci/typos/compare/v1.28.3...v1.28.4)

#### \[1.28.4] - 2024-12-16

##### Features

-   `--format sarif` support

</details>

<details>
<summary>executablebooks/mdformat (executablebooks/mdformat)</summary>

###
[`v0.7.21`](https://redirect.github.com/executablebooks/mdformat/compare/0.7.20...0.7.21)

[Compare
Source](https://redirect.github.com/executablebooks/mdformat/compare/0.7.20...0.7.21)

###
[`v0.7.20`](https://redirect.github.com/executablebooks/mdformat/compare/0.7.19...0.7.20)

[Compare
Source](https://redirect.github.com/executablebooks/mdformat/compare/0.7.19...0.7.20)

</details>

<details>
<summary>woodruffw/zizmor-pre-commit
(woodruffw/zizmor-pre-commit)</summary>

###
[`v0.10.0`](https://redirect.github.com/woodruffw/zizmor-pre-commit/releases/tag/v0.10.0)

[Compare
Source](https://redirect.github.com/woodruffw/zizmor-pre-commit/compare/v0.9.2...v0.10.0)

See: https://github.com/woodruffw/zizmor/releases/tag/v0.10.0

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "before 4am on Monday" (UTC),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

👻 **Immortal**: This PR will be recreated if closed unmerged. Get
[config
help](https://redirect.github.com/renovatebot/renovate/discussions) if
that's undesired.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/astral-sh/ruff).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS44MC4wIiwidXBkYXRlZEluVmVyIjoiMzkuODAuMCIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiaW50ZXJuYWwiXX0=-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-12-23 10:11:52 +05:30
renovate[bot]
130cc24e2c Update Rust crate anyhow to v1.0.95 (#15106)
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [anyhow](https://redirect.github.com/dtolnay/anyhow) |
workspace.dependencies | patch | `1.0.94` -> `1.0.95` |

---

### Release Notes

<details>
<summary>dtolnay/anyhow (anyhow)</summary>

###
[`v1.0.95`](https://redirect.github.com/dtolnay/anyhow/releases/tag/1.0.95)

[Compare
Source](https://redirect.github.com/dtolnay/anyhow/compare/1.0.94...1.0.95)

- Add
[`Error::from_boxed`](https://docs.rs/anyhow/latest/anyhow/struct.Error.html#method.from_boxed)
([#&#8203;401](https://redirect.github.com/dtolnay/anyhow/issues/401),
[#&#8203;402](https://redirect.github.com/dtolnay/anyhow/issues/402))

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "before 4am on Monday" (UTC),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/astral-sh/ruff).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS44MC4wIiwidXBkYXRlZEluVmVyIjoiMzkuODAuMCIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiaW50ZXJuYWwiXX0=-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-12-23 10:07:48 +05:30
renovate[bot]
d555eca729 Update Rust crate env_logger to v0.11.6 (#15107)
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [env_logger](https://redirect.github.com/rust-cli/env_logger) |
workspace.dependencies | patch | `0.11.5` -> `0.11.6` |

---

### Release Notes

<details>
<summary>rust-cli/env_logger (env_logger)</summary>

###
[`v0.11.6`](https://redirect.github.com/rust-cli/env_logger/blob/HEAD/CHANGELOG.md#0116---2024-12-20)

[Compare
Source](https://redirect.github.com/rust-cli/env_logger/compare/v0.11.5...v0.11.6)

##### Features

-   Opt-in file and line rendering

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "before 4am on Monday" (UTC),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/astral-sh/ruff).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS44MC4wIiwidXBkYXRlZEluVmVyIjoiMzkuODAuMCIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiaW50ZXJuYWwiXX0=-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-12-23 10:05:45 +05:30
renovate[bot]
489f4fb27a Update Rust crate libc to v0.2.169 (#15108)
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [libc](https://redirect.github.com/rust-lang/libc) |
workspace.dependencies | patch | `0.2.168` -> `0.2.169` |

---

### Release Notes

<details>
<summary>rust-lang/libc (libc)</summary>

###
[`v0.2.169`](https://redirect.github.com/rust-lang/libc/releases/tag/0.2.169)

[Compare
Source](https://redirect.github.com/rust-lang/libc/compare/0.2.168...0.2.169)

##### Added

- FreeBSD: add more socket TCP stack constants
[#&#8203;4193](https://redirect.github.com/rust-lang/libc/pull/4193)
- Fuchsia: add a `sockaddr_vm` definition
[#&#8203;4194](https://redirect.github.com/rust-lang/libc/pull/4194)

##### Fixed

**Breaking**:
[rust-lang/rust#132975](https://redirect.github.com/rust-lang/rust/pull/132975)
corrected the signedness of `core::ffi::c_char` on various Tier 2 and
Tier 3 platforms (mostly Arm and RISC-V) to match Clang. This release
contains the corresponding changes to `libc`, including the following
specific pull requests:

- ESP-IDF: Replace arch-conditional `c_char` with a reexport
[#&#8203;4195](https://redirect.github.com/rust-lang/libc/pull/4195)
- Fix `c_char` on various targets
[#&#8203;4199](https://redirect.github.com/rust-lang/libc/pull/4199)
- Mirror `c_char` configuration from `rust-lang/rust`
[#&#8203;4198](https://redirect.github.com/rust-lang/libc/pull/4198)

##### Cleanup

- Do not re-export `c_void` in target-specific code
[#&#8203;4200](https://redirect.github.com/rust-lang/libc/pull/4200)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "before 4am on Monday" (UTC),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/astral-sh/ruff).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS44MC4wIiwidXBkYXRlZEluVmVyIjoiMzkuODAuMCIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiaW50ZXJuYWwiXX0=-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-12-23 10:04:20 +05:30
renovate[bot]
429d3d78b8 Update Rust crate lsp-server to v0.7.8 (#15109)
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [lsp-server](https://redirect.github.com/rust-lang/rust-analyzer)
([source](https://redirect.github.com/rust-lang/rust-analyzer/tree/HEAD/lib/lsp-server))
| workspace.dependencies | patch | `0.7.7` -> `0.7.8` |

---

### Configuration

📅 **Schedule**: Branch creation - "before 4am on Monday" (UTC),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/astral-sh/ruff).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS44MC4wIiwidXBkYXRlZEluVmVyIjoiMzkuODAuMCIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiaW50ZXJuYWwiXX0=-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-12-23 09:57:13 +05:30
renovate[bot]
4dae9ee1cf Update Rust crate serde_json to v1.0.134 (#15110)
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [serde_json](https://redirect.github.com/serde-rs/json) |
workspace.dependencies | patch | `1.0.133` -> `1.0.134` |

---

### Release Notes

<details>
<summary>serde-rs/json (serde_json)</summary>

###
[`v1.0.134`](https://redirect.github.com/serde-rs/json/releases/tag/v1.0.134)

[Compare
Source](https://redirect.github.com/serde-rs/json/compare/v1.0.133...v1.0.134)

- Add `RawValue` associated constants for literal `null`, `true`,
`false`
([#&#8203;1221](https://redirect.github.com/serde-rs/json/issues/1221),
thanks [@&#8203;bheylin](https://redirect.github.com/bheylin))

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "before 4am on Monday" (UTC),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/astral-sh/ruff).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS44MC4wIiwidXBkYXRlZEluVmVyIjoiMzkuODAuMCIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiaW50ZXJuYWwiXX0=-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-12-23 09:55:26 +05:30
renovate[bot]
d8b4779a87 Update Rust crate syn to v2.0.91 (#15111)
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [syn](https://redirect.github.com/dtolnay/syn) |
workspace.dependencies | patch | `2.0.90` -> `2.0.91` |

---

### Release Notes

<details>
<summary>dtolnay/syn (syn)</summary>

###
[`v2.0.91`](https://redirect.github.com/dtolnay/syn/releases/tag/2.0.91)

[Compare
Source](https://redirect.github.com/dtolnay/syn/compare/2.0.90...2.0.91)

- Support parsing `Vec<Arm>` using `parse_quote!`
([#&#8203;1796](https://redirect.github.com/dtolnay/syn/issues/1796),
[#&#8203;1797](https://redirect.github.com/dtolnay/syn/issues/1797))

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "before 4am on Monday" (UTC),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/astral-sh/ruff).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS44MC4wIiwidXBkYXRlZEluVmVyIjoiMzkuODAuMCIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiaW50ZXJuYWwiXX0=-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-12-23 09:54:41 +05:30
renovate[bot]
88cdfcfb11 Update Rust crate thiserror to v2.0.9 (#15112)
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [thiserror](https://redirect.github.com/dtolnay/thiserror) |
workspace.dependencies | patch | `2.0.7` -> `2.0.9` |

---

### Release Notes

<details>
<summary>dtolnay/thiserror (thiserror)</summary>

###
[`v2.0.9`](https://redirect.github.com/dtolnay/thiserror/releases/tag/2.0.9)

[Compare
Source](https://redirect.github.com/dtolnay/thiserror/compare/2.0.8...2.0.9)

- Work around `missing_inline_in_public_items` clippy restriction being
triggered in macro-generated code
([#&#8203;404](https://redirect.github.com/dtolnay/thiserror/issues/404))

###
[`v2.0.8`](https://redirect.github.com/dtolnay/thiserror/releases/tag/2.0.8)

[Compare
Source](https://redirect.github.com/dtolnay/thiserror/compare/2.0.7...2.0.8)

- Improve support for macro-generated `derive(Error)` call sites
([#&#8203;399](https://redirect.github.com/dtolnay/thiserror/issues/399))

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "before 4am on Monday" (UTC),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/astral-sh/ruff).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS44MC4wIiwidXBkYXRlZEluVmVyIjoiMzkuODAuMCIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiaW50ZXJuYWwiXX0=-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-12-23 09:53:37 +05:30
InSync
f764f59971 [red-knot] Treat classes as instances of their respective metaclasses in boolean tests (#15105)
## Summary

Follow-up to #15089.

## Test Plan

Markdown tests.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2024-12-23 01:30:51 +00:00
InSync
3b27d5dbad [red-knot] More precise inference for chained boolean expressions (#15089)
## Summary

Resolves #13632.

## Test Plan

Markdown tests.
2024-12-22 10:02:28 -08:00
InSync
60e433c3b5 Rename round_applicability.rs back to unnecessary_cast_to_int.rs (#15095) 2024-12-21 21:32:26 +01:00
TomerBin
2fb6b320d8 Use TypeChecker for detecting fastapi routes (#15093) 2024-12-21 15:45:28 +01:00
Aaron Miller
fd4bea52e5 Fix type subscript on older python versions (#15090) 2024-12-21 15:28:02 +01:00
InSync
bd023c4500 [ruff] Detect more strict-integer expressions (RUF046) (#14833)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-12-21 14:23:26 +00:00
David Peter
000948ad3b [red-knot] Statically known branches (#15019)
## Summary

This changeset adds support for precise type-inference and
boundness-handling of definitions inside control-flow branches with
statically-known conditions, i.e. test-expressions whose truthiness we
can unambiguously infer as *always false* or *always true*.

This branch also includes:
- `sys.platform` support
- statically-known branches handling for Boolean expressions and while
  loops
- new `target-version` requirements in some Markdown tests which were
  now required due to the understanding of `sys.version_info` branches.

closes #12700 
closes #15034 

## Performance

### `tomllib`, -7%, needs to resolve one additional module (sys)

| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|:---|---:|---:|---:|---:|
| `./red_knot_main --project /home/shark/tomllib` | 22.2 ± 1.3 | 19.1 |
25.6 | 1.00 |
| `./red_knot_feature --project /home/shark/tomllib` | 23.8 ± 1.6 | 20.8
| 28.6 | 1.07 ± 0.09 |

### `black`, -6%

| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|:---|---:|---:|---:|---:|
| `./red_knot_main --project /home/shark/black` | 129.3 ± 5.1 | 119.0 |
137.8 | 1.00 |
| `./red_knot_feature --project /home/shark/black` | 136.5 ± 6.8 | 123.8
| 147.5 | 1.06 ± 0.07 |

## Test Plan

- New Markdown tests for the main feature in
  `statically-known-branches.md`
- New Markdown tests for `sys.platform`
- Adapted tests for `EllipsisType`, `Never`, etc
2024-12-21 11:33:10 +01:00
my1e5
d3f51cf3a6 [pydocstyle] Split on first whitespace character (D403) (#15082)
## Summary

This PR fixes an issue where Ruff's `D403` rule
(`first-word-uncapitalized`) was not detecting some single-word edge
cases that are picked up by `pydocstyle`.

The change involves extracting the first word of the docstring by
identifying the first whitespace character. This is consistent with
`pydocstyle` which uses `.split()` - see
8d0cdfc93e/src/pydocstyle/checker.py (L581C13-L581C64)

## Example

Here is a playground example -
https://play.ruff.rs/eab9ea59-92cf-4e44-b1a9-b54b7f69b178

```py
def example1():
    """foo"""

def example2():
    """foo
    
    Hello world!
    """

def example3():
    """foo bar

    Hello world!
    """

def example4():
    """
    foo
    """

def example5():
    """
    foo bar
    """
```

`pydocstyle` detects all five cases:
```bash
$ pydocstyle test.py --select D403
dev/test.py:2 in public function `example1`:
        D403: First word of the first line should be properly capitalized ('Foo', not 'foo')
dev/test.py:5 in public function `example2`:
        D403: First word of the first line should be properly capitalized ('Foo', not 'foo')
dev/test.py:11 in public function `example3`:
        D403: First word of the first line should be properly capitalized ('Foo', not 'foo')
dev/test.py:17 in public function `example4`:
        D403: First word of the first line should be properly capitalized ('Foo', not 'foo')
dev/test.py:22 in public function `example5`:
        D403: First word of the first line should be properly capitalized ('Foo', not 'foo')
```
Ruff (`0.8.4`) fails to catch example2 and example4.

## Test Plan

* Added two new test cases to cover the previously missed single-word
docstring cases.
2024-12-20 12:55:50 -06:00
Dhruv Manilawala
d47fba1e4a [red-knot] Add support for unpacking union types (#15052)
## Summary

Refer:
https://github.com/astral-sh/ruff/issues/13773#issuecomment-2548020368

This PR adds support for unpacking union types. 

Unpacking a union type requires us to first distribute the types for all
the targets that are involved in an unpacking. For example, if there are
two targets and a union type that needs to be unpacked, each target will
get a type from each element in the union type.

For example, if the type is `tuple[int, int] | tuple[int, str]` and the
target has two elements `(a, b)`, then
* The type of `a` will be a union of `int` and `int` which are at index
0 in the first and second tuple respectively which resolves to an `int`.
* Similarly, the type of `b` will be a union of `int` and `str` which
are at index 1 in the first and second tuple respectively which will be
`int | str`.

### Refactors

There are couple of refactors that are added in this PR:
* Add a `debug_assertion` to validate that the unpack target is a list
or a tuple
* Add a separate method to handle starred expression

## Test Plan

Update `unpacking.md` with additional test cases that uses union types.
This is done using parameter type hints style.
2024-12-20 16:31:15 +05:30
David Salvisberg
089a98e904 [ruff] Adds an allowlist for unsafe-markup-use (RUF035) (#15076)
Closes: #14523

## Summary

Adds a whitelist of calls allowed to be used within a
`markupsafe.Markup` call.

## Test Plan

`cargo nextest run`
2024-12-20 09:36:12 +00:00
Micha Reiser
913bce3cd5 Basic support for type: ignore comments (#15046)
## Summary

This PR adds initial support for `type: ignore`. It doesn't do anything
fancy yet like:

* Detecting invalid type ignore comments
* Detecting type ignore comments that are part of another suppression
comment: `# fmt: skip # type: ignore`
* Suppressing specific lints `type: ignore [code]`
* Detecting unsused type ignore comments
* ...

The goal is to add this functionality in separate PRs.

## Test Plan

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-12-20 10:35:09 +01:00
Harutaka Kawamura
6195c026ff [flake8-comprehensions] Skip C416 if comprehension contains unpacking (#14909)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

Fix #11482. Applies
https://github.com/adamchainz/flake8-comprehensions/pull/205 to ruff.
`C416` should be skipped if comprehension contains unpacking. Here's an
example:

```python
list_of_lists = [[1, 2], [3, 4]]

# ruff suggests `list(list_of_lists)` here, but that would change the result.
# `list(list_of_lists)` is not `[(1, 2), (3, 4)]`
a = [(x, y) for x, y in list_of_lists]

# This is equivalent to `list(list_of_lists)`
b = [x for x in list_of_lists]
```

## Test Plan

<!-- How was it tested? -->

Existing checks

---------

Signed-off-by: harupy <hkawamura0130@gmail.com>
2024-12-20 14:35:30 +05:30
KotlinIsland
c0b0491703 Update docs for eq-without-hash (#14885)
## Summary

resolves #14883

This PR removes the known limitation section in the documentation of
`eq-without-hash`. That is not actually a limitation as a subclass
overriding the `__eq__` method would have its `__hash__` set to `None`
implicitly. The user should explicitly inherit the `__hash__` method
from the parent class.

## Test Plan

<img width="619" alt="Screenshot 2024-12-20 at 2 02 47 PM"
src="https://github.com/user-attachments/assets/552defcd-25e1-4153-9ab9-e5b9d5fbe8cc"
/>

---------

Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
2024-12-20 14:03:16 +05:30
Micha Reiser
51863b460b Use &'static str for Replacement (#15075)
## Summary

Smaller nits follow up to https://github.com/astral-sh/ruff/pull/15054
because I wasn't able to push to the branch directly.

## Test Plan

`cargo test`
2024-12-20 07:36:21 +00:00
Wei Lee
14a5a2629e [airflow]: extend removed method calls (AIR302) (#15054)
## Summary


Airflow 3.0 removes various deprecated functions, members, modules, and
other values. They have been deprecated in 2.x, but the removal causes
incompatibilities that we want to detect. This PR deprecates the
following names and add a function for removed methods

* `airflow.datasets.manager.DatasetManager.register_dataset_change` →
`airflow.assets.manager.AssetManager.register_asset_change`
* `airflow.datasets.manager.DatasetManager.create_datasets` →
`airflow.assets.manager.AssetManager.create_assets`
* `airflow.datasets.manager.DatasetManager.notify_dataset_created` →
`airflow.assets.manager.AssetManager.notify_asset_created`
* `airflow.datasets.manager.DatasetManager.notify_dataset_changed` →
`airflow.assets.manager.AssetManager.notify_asset_changed`
* `airflow.datasets.manager.DatasetManager.notify_dataset_alias_created`
→ `airflow.assets.manager.AssetManager.notify_asset_alias_created`
*
`airflow.providers.amazon.auth_manager.aws_auth_manager.AwsAuthManager.is_authorized_dataset`
→
`airflow.providers.amazon.auth_manager.aws_auth_manager.AwsAuthManager.is_authorized_asset`
* `airflow.lineage.hook.HookLineageCollector.create_dataset` →
`airflow.lineage.hook.HookLineageCollector.create_asset`
* `airflow.lineage.hook.HookLineageCollector.add_input_dataset` →
`airflow.lineage.hook.HookLineageCollector.add_input_asset`
* `airflow.lineage.hook.HookLineageCollector.add_output_dataset` →
`airflow.lineage.hook.HookLineageCollector.dd_output_asset`
* `airflow.lineage.hook.HookLineageCollector.collected_datasets` →
`airflow.lineage.hook.HookLineageCollector.collected_assets`
*
`airflow.providers_manager.ProvidersManager.initialize_providers_dataset_uri_resources`
→
`airflow.providers_manager.ProvidersManager.initialize_providers_asset_uri_resources`

## Test Plan

A test fixture is included in the PR.
2024-12-20 08:30:30 +01:00
Alex Waygood
3aed14935d [red-knot] Add support for @final classes (#15070)
Co-authored-by: Carl Meyer <carl@astral.sh>
2024-12-19 21:02:14 +00:00
Alex Waygood
bcec5e615b [red-knot] Rename and rework the CoreStdlibModule enum (#15071) 2024-12-19 20:59:00 +00:00
Alex Waygood
a06099dffe [red-knot] Move attribute access on ModuleLiteral types into a dedicated method (#15067) 2024-12-19 16:02:16 +00:00
Alex Waygood
bb43085939 [red-knot] Reduce TODOs in Type::member() (#15066) 2024-12-19 15:54:01 +00:00
Dylan
c1eaf6ff72 Modify parsing of raise with cause when exception is absent (#15049)
When confronted with `raise from exc` the parser will now create a
`StmtRaise` that has `None` for the exception and `exc` for the cause.

Before, the parser created a `StmtRaise` with `from` for the exception,
no cause, and a spurious expression `exc` afterwards.
2024-12-19 13:36:32 +00:00
Dhruv Manilawala
3bb0dac235 Bump version to 0.8.4 (#15064) 2024-12-19 13:15:42 +00:00
Alex Waygood
40cba5dc8a [red-knot] Cleanup various todo_type!() messages (#15063)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-12-19 13:03:41 +00:00
Dylan
596d80cc8e [perflint] Parenthesize walrus expressions in autofix for manual-list-comprehension (PERF401) (#15050) 2024-12-19 06:56:45 -06:00
Alex Waygood
d8b9a366c8 Disable actionlint hook by default when running pre-commit locally (#15061) 2024-12-19 12:45:17 +00:00
Taras Matsyk
85e71ba91a [flake8-bandit] Check S105 for annotated assignment (#15059)
## Summary

A follow up PR on https://github.com/astral-sh/ruff/issues/14991
Ruff ignores hardcoded passwords for typed variables. Add a rule to
catch passwords in typed code bases

## Test Plan

Includes 2 more test typed variables
2024-12-19 12:26:40 +00:00
Douglas Creager
2802cbde29 Don't special-case class instances in unary expression inference (#15045)
We have a handy `to_meta_type` that does the right thing for class
instances, and also works for all of the other types that are “instances
of” something. Unless I'm missing something, this should let us get rid
of the catch-all clause in one fell swoop.

cf #14548
2024-12-18 14:37:17 -05:00
InSync
ed2bce6ebb [red-knot] Report invalid exceptions (#15042)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-12-18 18:31:24 +00:00
InSync
f0012df686 Fix typos in RUF043.py (#15044)
(Accidentally introduced in #14966.)
2024-12-18 15:39:55 +00:00
Micha Reiser
0fc4e8f795 Introduce InferContext (#14956)
## Summary

I'm currently on the fence about landing the #14760 PR because it's
unclear how we'd support tracking used and unused suppression comments
in a performant way:
* Salsa adds an "untracked" dependency to every query reading
accumulated values. This has the effect that the query re-runs on every
revision. For example, a possible future query
`unused_suppression_comments(db, file)` would re-run on every
incremental change and for every file. I don't expect the operation
itself to be expensive, but it all adds up in a project with 100k+ files
* Salsa collects the accumulated values by traversing the entire query
dependency graph. It can skip over sub-graphs if it is known that they
contain no accumulated values. This makes accumulators a great tool for
when they are rare; diagnostics are a good example. Unfortunately,
suppressions are more common, and they often appear in many different
files, making the "skip over subgraphs" optimization less effective.

Because of that, I want to wait to adopt salsa accumulators for type
check diagnostics (we could start using them for other diagnostics)
until we have very specific reasons that justify regressing incremental
check performance.

This PR does a "small" refactor that brings us closer to what I have in
#14760 but without using accumulators. To emit a diagnostic, a method
needs:

* Access to the db
* Access to the currently checked file

This PR introduces a new `InferContext` that holds on to the db, the
current file, and the reported diagnostics. It replaces the
`TypeCheckDiagnosticsBuilder`. We pass the `InferContext` instead of the
`db` to methods that *might* emit diagnostics. This simplifies some of
the `Outcome` methods, which can now be called with a context instead of
a `db` and the diagnostics builder. Having the `db` and the file on a
single type like this would also be useful when using accumulators.

This PR doesn't solve the issue that the `Outcome` types feel somewhat
complicated nor that it can be annoying when you need to report a
`Diagnostic,` but you don't have access to an `InferContext` (or the
file). However, I also believe that accumulators won't solve these
problems because:

* Even with accumulators, it's necessary to have a reference to the file
that's being checked. The struggle would be to get a reference to that
file rather than getting a reference to `InferContext`.
* Users of the `HasTy` trait (e.g., a linter) don't want to bother
getting the `File` when calling `Type::return_ty` because they aren't
interested in the created diagnostics. They just want to know what
calling the current expression would return (and if it even is a
callable). This is what the different methods of `Outcome` enable today.
I can ask for the return type without needing extra data that's only
relevant for emitting a diagnostic.

A shortcoming of this approach is that it is now a bit confusing when to
pass `db` and when an `InferContext`. An option is that we'd make the
`file` on `InferContext` optional (it won't collect any diagnostics if
`None`) and change all methods on `Type` to take `InferContext` as the
first argument instead of a `db`. I'm interested in your opinion on
this.

Accumulators are definitely harder to use incorrectly because they
remove the need to merge the diagnostics explicitly and there's no risk
that we accidentally merge the diagnostics twice, resulting in
duplicated diagnostics. I still value performance more over making our
life slightly easier.
2024-12-18 12:22:33 +00:00
InSync
ac81c72bf3 [ruff] Ambiguous pattern passed to pytest.raises() (RUF043) (#14966) 2024-12-18 11:53:48 +00:00
David Salvisberg
c0b7c36d43 [ruff] Avoid false positives for RUF027 for typing context bindings. (#15037)
Closes #14000 

## Summary

For typing context bindings we know that they won't be available at
runtime. We shouldn't recommend a fix, that will result in name errors
at runtime.

## Test Plan

`cargo nextest run`
2024-12-18 08:50:49 +01:00
Douglas Creager
e8e461da6a Prioritize attribute in from/import statement (#15041)
This tweaks the new semantics from #15026 a bit when a symbol could be
interpreted both as an attribute and a submodule of a package. For
`from...import`, we should actually prioritize the attribute, because of
how the statement itself is implemented [1].

> 1. check if the imported module has an attribute by that name
> 2. if not, attempt to import a submodule with that name and then check
the imported module again for that attribute

[1] https://docs.python.org/3/reference/simple_stmts.html#the-import-statement
2024-12-17 16:58:23 -05:00
1846 changed files with 64735 additions and 25611 deletions

1
.github/CODEOWNERS vendored
View File

@@ -9,6 +9,7 @@
/crates/ruff_formatter/ @MichaReiser
/crates/ruff_python_formatter/ @MichaReiser
/crates/ruff_python_parser/ @MichaReiser @dhruvmanila
/crates/ruff_annotate_snippets/ @BurntSushi
# flake8-pyi
/crates/ruff_linter/src/rules/flake8_pyi/ @AlexWaygood

View File

@@ -1,12 +0,0 @@
<!--
Thank you for taking the time to report an issue! We're glad to have you involved with Ruff.
If you're filing a bug report, please consider including the following information:
* List of keywords you searched for before creating this issue. Write them down here so that others can find this issue more easily and help provide feedback.
e.g. "RUF001", "unused variable", "Jupyter notebook"
* A minimal code snippet that reproduces the bug.
* The command you invoked (e.g., `ruff /path/to/file.py --fix`), ideally including the `--isolated` flag.
* The current Ruff settings (any relevant sections from your `pyproject.toml`).
* The current Ruff version (`ruff --version`).
-->

View File

@@ -45,7 +45,7 @@
groupName: "Artifact GitHub Actions dependencies",
matchManagers: ["github-actions"],
matchDatasources: ["gitea-tags", "github-tags"],
matchPackagePatterns: ["actions/.*-artifact"],
matchPackageNames: ["actions/.*-artifact"],
description: "Weekly update of artifact-related GitHub Actions dependencies",
},
{
@@ -61,7 +61,7 @@
{
// Disable updates of `zip-rs`; intentionally pinned for now due to ownership change
// See: https://github.com/astral-sh/uv/issues/3642
matchPackagePatterns: ["zip"],
matchPackageNames: ["zip"],
matchManagers: ["cargo"],
enabled: false,
},
@@ -70,7 +70,7 @@
// with `mkdocs-material-insider`.
// See: https://squidfunk.github.io/mkdocs-material/insiders/upgrade/
matchManagers: ["pip_requirements"],
matchPackagePatterns: ["mkdocs-material"],
matchPackageNames: ["mkdocs-material"],
enabled: false,
},
{
@@ -87,13 +87,13 @@
{
groupName: "Monaco",
matchManagers: ["npm"],
matchPackagePatterns: ["monaco"],
matchPackageNames: ["monaco"],
description: "Weekly update of the Monaco editor",
},
{
groupName: "strum",
matchManagers: ["cargo"],
matchPackagePatterns: ["strum"],
matchPackageNames: ["strum"],
description: "Weekly update of strum dependencies",
},
{

View File

@@ -48,11 +48,13 @@ jobs:
- name: Check tag consistency
if: ${{ inputs.plan != '' && !fromJson(inputs.plan).announcement_tag_is_implicit }}
env:
TAG: ${{ inputs.plan != '' && fromJson(inputs.plan).announcement_tag || 'dry-run' }}
run: |
version=$(grep "version = " pyproject.toml | sed -e 's/version = "\(.*\)"/\1/g')
if [ "${{ fromJson(inputs.plan).announcement_tag }}" != "${version}" ]; then
if [ "${TAG}" != "${version}" ]; then
echo "The input tag does not match the version from pyproject.toml:" >&2
echo "${{ fromJson(inputs.plan).announcement_tag }}" >&2
echo "${TAG}" >&2
echo "${version}" >&2
exit 1
else
@@ -175,6 +177,8 @@ jobs:
- name: Generate Dynamic Dockerfile Tags
shell: bash
env:
TAG_VALUE: ${{ fromJson(inputs.plan).announcement_tag }}
run: |
set -euo pipefail
@@ -195,8 +199,8 @@ jobs:
# Loop through all base tags and append its docker metadata pattern to the list
# Order is on purpose such that the label org.opencontainers.image.version has the first pattern with the full version
IFS=','; for TAG in ${BASE_TAGS}; do
TAG_PATTERNS="${TAG_PATTERNS}type=pep440,pattern={{ version }},suffix=-${TAG},value=${{ fromJson(inputs.plan).announcement_tag }}\n"
TAG_PATTERNS="${TAG_PATTERNS}type=pep440,pattern={{ major }}.{{ minor }},suffix=-${TAG},value=${{ fromJson(inputs.plan).announcement_tag }}\n"
TAG_PATTERNS="${TAG_PATTERNS}type=pep440,pattern={{ version }},suffix=-${TAG},value=${TAG_VALUE}\n"
TAG_PATTERNS="${TAG_PATTERNS}type=pep440,pattern={{ major }}.{{ minor }},suffix=-${TAG},value=${TAG_VALUE}\n"
TAG_PATTERNS="${TAG_PATTERNS}type=raw,value=${TAG}\n"
done

View File

@@ -349,7 +349,7 @@ jobs:
- uses: actions/checkout@v4
with:
persist-credentials: false
- uses: astral-sh/setup-uv@v4
- uses: astral-sh/setup-uv@v5
- uses: actions/download-artifact@v4
name: Download Ruff binary to test
id: download-cached-binary
@@ -386,7 +386,7 @@ jobs:
- name: "Install Rust toolchain"
run: rustup component add rustfmt
- uses: Swatinem/rust-cache@v2
- run: ./scripts/add_rule.py --name DoTheThing --prefix PL --code C0999 --linter pylint
- run: ./scripts/add_rule.py --name DoTheThing --prefix F --code 999 --linter pyflakes
- run: cargo check
- run: cargo fmt --all --check
- run: |
@@ -585,7 +585,8 @@ jobs:
run: |
echo '```console' > "$GITHUB_STEP_SUMMARY"
# Enable color output for pre-commit and remove it for the summary
SKIP=cargo-fmt,clippy,dev-generate-all pre-commit run --all-files --show-diff-on-failure --color=always | \
# Use --hook-stage=manual to enable slower pre-commit hooks that are skipped by default
SKIP=cargo-fmt,clippy,dev-generate-all pre-commit run --all-files --show-diff-on-failure --color=always --hook-stage=manual | \
tee >(sed -E 's/\x1B\[([0-9]{1,2}(;[0-9]{1,2})*)?[mGK]//g' >> "$GITHUB_STEP_SUMMARY") >&1
exit_code="${PIPESTATUS[0]}"
echo '```' >> "$GITHUB_STEP_SUMMARY"
@@ -612,7 +613,7 @@ jobs:
- name: "Install Rust toolchain"
run: rustup show
- name: Install uv
uses: astral-sh/setup-uv@v4
uses: astral-sh/setup-uv@v5
- uses: Swatinem/rust-cache@v2
- name: "Install Insiders dependencies"
if: ${{ env.MKDOCS_INSIDERS_SSH_KEY_EXISTS == 'true' }}

View File

@@ -34,7 +34,7 @@ jobs:
- uses: actions/checkout@v4
with:
persist-credentials: false
- uses: astral-sh/setup-uv@v4
- uses: astral-sh/setup-uv@v5
- name: "Install Rust toolchain"
run: rustup show
- name: "Install mold"
@@ -73,6 +73,6 @@ jobs:
owner: "astral-sh",
repo: "ruff",
title: `Daily parser fuzz failed on ${new Date().toDateString()}`,
body: "Runs listed here: https://github.com/astral-sh/ruff/actions/workflows/daily_fuzz.yml",
body: "Run listed here: https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}",
labels: ["bug", "parser", "fuzzer"],
})

View File

@@ -0,0 +1,71 @@
name: Daily property test run
on:
workflow_dispatch:
schedule:
- cron: "0 12 * * *"
pull_request:
paths:
- ".github/workflows/daily_property_tests.yaml"
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: true
env:
CARGO_INCREMENTAL: 0
CARGO_NET_RETRY: 10
CARGO_TERM_COLOR: always
RUSTUP_MAX_RETRIES: 10
FORCE_COLOR: 1
jobs:
property_tests:
name: Property tests
runs-on: ubuntu-latest
timeout-minutes: 20
# Don't run the cron job on forks:
if: ${{ github.repository == 'astral-sh/ruff' || github.event_name != 'schedule' }}
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
- name: "Install Rust toolchain"
run: rustup show
- name: "Install mold"
uses: rui314/setup-mold@v1
- uses: Swatinem/rust-cache@v2
- name: Build Red Knot
# A release build takes longer (2 min vs 1 min), but the property tests run much faster in release
# mode (1.5 min vs 14 min), so the overall time is shorter with a release build.
run: cargo build --locked --release --package red_knot_python_semantic --tests
- name: Run property tests
shell: bash
run: |
export QUICKCHECK_TESTS=100000
for _ in {1..5}; do
cargo test --locked --release --package red_knot_python_semantic -- --ignored types::property_tests::stable
done
create-issue-on-failure:
name: Create an issue if the daily property test run surfaced any bugs
runs-on: ubuntu-latest
needs: property_tests
if: ${{ github.repository == 'astral-sh/ruff' && always() && github.event_name == 'schedule' && needs.property_tests.result == 'failure' }}
permissions:
issues: write
steps:
- uses: actions/github-script@v7
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
await github.rest.issues.create({
owner: "astral-sh",
repo: "ruff",
title: `Daily property test run failed on ${new Date().toDateString()}`,
body: "Run listed here: https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}",
labels: ["bug", "red-knot", "testing"],
})

View File

@@ -33,8 +33,9 @@ jobs:
python-version: 3.12
- name: "Set docs version"
env:
version: ${{ (inputs.plan != '' && fromJson(inputs.plan).announcement_tag) || inputs.ref }}
run: |
version="${{ (inputs.plan != '' && fromJson(inputs.plan).announcement_tag) || inputs.ref }}"
# if version is missing, use 'latest'
if [ -z "$version" ]; then
echo "Using 'latest' as version"

View File

@@ -22,7 +22,7 @@ jobs:
id-token: write
steps:
- name: "Install uv"
uses: astral-sh/setup-uv@v4
uses: astral-sh/setup-uv@v5
- uses: actions/download-artifact@v4
with:
pattern: wheels-*

View File

@@ -78,5 +78,6 @@ jobs:
owner: "astral-sh",
repo: "ruff",
title: `Automated typeshed sync failed on ${new Date().toDateString()}`,
body: "Runs are listed here: https://github.com/astral-sh/ruff/actions/workflows/sync_typeshed.yaml",
body: "Run listed here: https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}",
labels: ["bug", "red-knot"],
})

6
.github/zizmor.yml vendored
View File

@@ -1,6 +1,12 @@
# Configuration for the zizmor static analysis tool, run via pre-commit in CI
# https://woodruffw.github.io/zizmor/configuration/
#
# TODO: can we remove the ignores here so that our workflows are more secure?
rules:
dangerous-triggers:
ignore:
- pr-comment.yaml
cache-poisoning:
ignore:
- build-docker.yml
- publish-playground.yml

View File

@@ -2,6 +2,7 @@ fail_fast: false
exclude: |
(?x)^(
.github/workflows/release.yml|
crates/red_knot_vendored/vendor/.*|
crates/red_knot_workspace/resources/.*|
crates/ruff_linter/resources/.*|
@@ -22,7 +23,7 @@ repos:
- id: validate-pyproject
- repo: https://github.com/executablebooks/mdformat
rev: 0.7.19
rev: 0.7.21
hooks:
- id: mdformat
additional_dependencies:
@@ -58,7 +59,7 @@ repos:
- black==24.10.0
- repo: https://github.com/crate-ci/typos
rev: v1.28.3
rev: v1.29.4
hooks:
- id: typos
@@ -72,7 +73,7 @@ repos:
pass_filenames: false # This makes it a lot faster
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.8.3
rev: v0.9.1
hooks:
- id: ruff-format
- id: ruff
@@ -90,27 +91,26 @@ repos:
# zizmor detects security vulnerabilities in GitHub Actions workflows.
# Additional configuration for the tool is found in `.github/zizmor.yml`
- repo: https://github.com/woodruffw/zizmor-pre-commit
rev: v0.9.2
rev: v1.0.1
hooks:
- id: zizmor
# `release.yml` is autogenerated by `dist`; security issues need to be fixed there
# (https://opensource.axo.dev/cargo-dist/)
exclude: .github/workflows/release.yml
- repo: https://github.com/python-jsonschema/check-jsonschema
rev: 0.30.0
rev: 0.31.0
hooks:
- id: check-github-workflows
# `actionlint` hook, for verifying correct syntax in GitHub Actions workflows.
# Some additional configuration for `actionlint` can be found in `.github/actionlint.yaml`.
- repo: https://github.com/rhysd/actionlint
rev: v1.7.4
rev: v1.7.6
hooks:
- id: actionlint
# `release.yml` is autogenerated by `dist`; issues need to be fixed there
# (https://opensource.axo.dev/cargo-dist/)
exclude: .github/workflows/release.yml
stages:
# This hook is disabled by default, since it's quite slow.
# To run all hooks *including* this hook, use `uvx pre-commit run -a --hook-stage=manual`.
# To run *just* this hook, use `uvx pre-commit run -a actionlint --hook-stage=manual`.
- manual
args:
- "-ignore=SC2129" # ignorable stylistic lint from shellcheck
- "-ignore=SC2016" # another shellcheck lint: seems to have false positives?

View File

@@ -1,5 +1,9 @@
# Breaking Changes
## 0.9.0
Ruff now formats your code according to the 2025 style guide. As a result, your code might now get formatted differently. See the [changelog](./CHANGELOG.md#090) for a detailed list of changes.
## 0.8.0
- **Default to Python 3.9**

View File

@@ -1,5 +1,208 @@
# Changelog
## 0.9.1
### Preview features
- \[`pycodestyle`\] Run `too-many-newlines-at-end-of-file` on each cell in notebooks (`W391`) ([#15308](https://github.com/astral-sh/ruff/pull/15308))
- \[`ruff`\] Omit diagnostic for shadowed private function parameters in `used-dummy-variable` (`RUF052`) ([#15376](https://github.com/astral-sh/ruff/pull/15376))
### Rule changes
- \[`flake8-bugbear`\] Improve `assert-raises-exception` message (`B017`) ([#15389](https://github.com/astral-sh/ruff/pull/15389))
### Formatter
- Preserve trailing end-of line comments for the last string literal in implicitly concatenated strings ([#15378](https://github.com/astral-sh/ruff/pull/15378))
### Server
- Fix a bug where the server and client notebooks were out of sync after reordering cells ([#15398](https://github.com/astral-sh/ruff/pull/15398))
### Bug fixes
- \[`flake8-pie`\] Correctly remove wrapping parentheses (`PIE800`) ([#15394](https://github.com/astral-sh/ruff/pull/15394))
- \[`pyupgrade`\] Handle comments and multiline expressions correctly (`UP037`) ([#15337](https://github.com/astral-sh/ruff/pull/15337))
## 0.9.0
Check out the [blog post](https://astral.sh/blog/ruff-v0.9.0) for a migration guide and overview of the changes!
### Breaking changes
Ruff now formats your code according to the 2025 style guide. As a result, your code might now get formatted differently. See the formatter section for a detailed list of changes.
This release doesnt remove or remap any existing stable rules.
### Stabilization
The following rules have been stabilized and are no longer in preview:
- [`stdlib-module-shadowing`](https://docs.astral.sh/ruff/rules/stdlib-module-shadowing/) (`A005`).
This rule has also been renamed: previously, it was called `builtin-module-shadowing`.
- [`builtin-lambda-argument-shadowing`](https://docs.astral.sh/ruff/rules/builtin-lambda-argument-shadowing/) (`A006`)
- [`slice-to-remove-prefix-or-suffix`](https://docs.astral.sh/ruff/rules/slice-to-remove-prefix-or-suffix/) (`FURB188`)
- [`boolean-chained-comparison`](https://docs.astral.sh/ruff/rules/boolean-chained-comparison/) (`PLR1716`)
- [`decimal-from-float-literal`](https://docs.astral.sh/ruff/rules/decimal-from-float-literal/) (`RUF032`)
- [`post-init-default`](https://docs.astral.sh/ruff/rules/post-init-default/) (`RUF033`)
- [`useless-if-else`](https://docs.astral.sh/ruff/rules/useless-if-else/) (`RUF034`)
The following behaviors have been stabilized:
- [`pytest-parametrize-names-wrong-type`](https://docs.astral.sh/ruff/rules/pytest-parametrize-names-wrong-type/) (`PT006`): Detect [`pytest.parametrize`](https://docs.pytest.org/en/7.1.x/how-to/parametrize.html#parametrize) calls outside decorators and calls with keyword arguments.
- [`module-import-not-at-top-of-file`](https://docs.astral.sh/ruff/rules/module-import-not-at-top-of-file/) (`E402`): Ignore [`pytest.importorskip`](https://docs.pytest.org/en/7.1.x/reference/reference.html#pytest-importorskip) calls between import statements.
- [`mutable-dataclass-default`](https://docs.astral.sh/ruff/rules/mutable-dataclass-default/) (`RUF008`) and [`function-call-in-dataclass-default-argument`](https://docs.astral.sh/ruff/rules/function-call-in-dataclass-default-argument/) (`RUF009`): Add support for [`attrs`](https://www.attrs.org/en/stable/).
- [`bad-version-info-comparison`](https://docs.astral.sh/ruff/rules/bad-version-info-comparison/) (`PYI006`): Extend the rule to check non-stub files.
The following fixes or improvements to fixes have been stabilized:
- [`redundant-numeric-union`](https://docs.astral.sh/ruff/rules/redundant-numeric-union/) (`PYI041`)
- [`duplicate-union-members`](https://docs.astral.sh/ruff/rules/duplicate-union-member/) (`PYI016`)
### Formatter
This release introduces the new 2025 stable style ([#13371](https://github.com/astral-sh/ruff/issues/13371)), stabilizing the following changes:
- Format expressions in f-string elements ([#7594](https://github.com/astral-sh/ruff/issues/7594))
- Alternate quotes for strings inside f-strings ([#13860](https://github.com/astral-sh/ruff/pull/13860))
- Preserve the casing of hex codes in f-string debug expressions ([#14766](https://github.com/astral-sh/ruff/issues/14766))
- Choose the quote style for each string literal in an implicitly concatenated f-string rather than for the entire string ([#13539](https://github.com/astral-sh/ruff/pull/13539))
- Automatically join an implicitly concatenated string into a single string literal if it fits on a single line ([#9457](https://github.com/astral-sh/ruff/issues/9457))
- Remove the [`ISC001`](https://docs.astral.sh/ruff/rules/single-line-implicit-string-concatenation/) incompatibility warning ([#15123](https://github.com/astral-sh/ruff/pull/15123))
- Prefer parenthesizing the `assert` message over breaking the assertion expression ([#9457](https://github.com/astral-sh/ruff/issues/9457))
- Automatically parenthesize over-long `if` guards in `match` `case` clauses ([#13513](https://github.com/astral-sh/ruff/pull/13513))
- More consistent formatting for `match` `case` patterns ([#6933](https://github.com/astral-sh/ruff/issues/6933))
- Avoid unnecessary parentheses around return type annotations ([#13381](https://github.com/astral-sh/ruff/pull/13381))
- Keep the opening parentheses on the same line as the `if` keyword for comprehensions where the condition has a leading comment ([#12282](https://github.com/astral-sh/ruff/pull/12282))
- More consistent formatting for `with` statements with a single context manager for Python 3.8 or older ([#10276](https://github.com/astral-sh/ruff/pull/10276))
- Correctly calculate the line-width for code blocks in docstrings when using `max-doc-code-line-length = "dynamic"` ([#13523](https://github.com/astral-sh/ruff/pull/13523))
### Preview features
- \[`flake8-bugbear`\] Implement `class-as-data-structure` (`B903`) ([#9601](https://github.com/astral-sh/ruff/pull/9601))
- \[`flake8-type-checking`\] Apply `quoted-type-alias` more eagerly in `TYPE_CHECKING` blocks and ignore it in stubs (`TC008`) ([#15180](https://github.com/astral-sh/ruff/pull/15180))
- \[`pylint`\] Ignore `eq-without-hash` in stub files (`PLW1641`) ([#15310](https://github.com/astral-sh/ruff/pull/15310))
- \[`pyupgrade`\] Split `UP007` into two individual rules: `UP007` for `Union` and `UP045` for `Optional` (`UP007`, `UP045`) ([#15313](https://github.com/astral-sh/ruff/pull/15313))
- \[`ruff`\] New rule that detects classes that are both an enum and a `dataclass` (`RUF049`) ([#15299](https://github.com/astral-sh/ruff/pull/15299))
- \[`ruff`\] Recode `RUF025` to `RUF037` (`RUF037`) ([#15258](https://github.com/astral-sh/ruff/pull/15258))
### Rule changes
- \[`flake8-builtins`\] Ignore [`stdlib-module-shadowing`](https://docs.astral.sh/ruff/rules/stdlib-module-shadowing/) in stub files(`A005`) ([#15350](https://github.com/astral-sh/ruff/pull/15350))
- \[`flake8-return`\] Add support for functions returning `typing.Never` (`RET503`) ([#15298](https://github.com/astral-sh/ruff/pull/15298))
### Server
- Improve the observability by removing the need for the ["trace" value](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#traceValue) to turn on or off logging. The server logging is solely controlled using the [`logLevel` server setting](https://docs.astral.sh/ruff/editors/settings/#loglevel)
which defaults to `info`. This addresses the issue where users were notified about an error and told to consult the log, but it didnt contain any messages. ([#15232](https://github.com/astral-sh/ruff/pull/15232))
- Ignore diagnostics from other sources for code action requests ([#15373](https://github.com/astral-sh/ruff/pull/15373))
### CLI
- Improve the error message for `--config key=value` when the `key` is for a table and its a simple `value`
### Bug fixes
- \[`eradicate`\] Ignore metadata blocks directly followed by normal blocks (`ERA001`) ([#15330](https://github.com/astral-sh/ruff/pull/15330))
- \[`flake8-django`\] Recognize other magic methods (`DJ012`) ([#15365](https://github.com/astral-sh/ruff/pull/15365))
- \[`pycodestyle`\] Avoid false positives related to type aliases (`E252`) ([#15356](https://github.com/astral-sh/ruff/pull/15356))
- \[`pydocstyle`\] Avoid treating newline-separated sections as sub-sections (`D405`) ([#15311](https://github.com/astral-sh/ruff/pull/15311))
- \[`pyflakes`\] Remove call when removing final argument from `format` (`F523`) ([#15309](https://github.com/astral-sh/ruff/pull/15309))
- \[`refurb`\] Mark fix as unsafe when the right-hand side is a string (`FURB171`) ([#15273](https://github.com/astral-sh/ruff/pull/15273))
- \[`ruff`\] Treat `)` as a regex metacharacter (`RUF043`, `RUF055`) ([#15318](https://github.com/astral-sh/ruff/pull/15318))
- \[`ruff`\] Parenthesize the `int`-call argument when removing the `int` call would change semantics (`RUF046`) ([#15277](https://github.com/astral-sh/ruff/pull/15277))
## 0.8.6
### Preview features
- \[`format`\]: Preserve multiline implicit concatenated strings in docstring positions ([#15126](https://github.com/astral-sh/ruff/pull/15126))
- \[`ruff`\] Add rule to detect empty literal in deque call (`RUF025`) ([#15104](https://github.com/astral-sh/ruff/pull/15104))
- \[`ruff`\] Avoid reporting when `ndigits` is possibly negative (`RUF057`) ([#15234](https://github.com/astral-sh/ruff/pull/15234))
### Rule changes
- \[`flake8-todos`\] remove issue code length restriction (`TD003`) ([#15175](https://github.com/astral-sh/ruff/pull/15175))
- \[`pyflakes`\] Ignore errors in `@no_type_check` string annotations (`F722`, `F821`) ([#15215](https://github.com/astral-sh/ruff/pull/15215))
### CLI
- Show errors for attempted fixes only when passed `--verbose` ([#15237](https://github.com/astral-sh/ruff/pull/15237))
### Bug fixes
- \[`ruff`\] Avoid syntax error when removing int over multiple lines (`RUF046`) ([#15230](https://github.com/astral-sh/ruff/pull/15230))
- \[`pyupgrade`\] Revert "Add all PEP-585 names to `UP006` rule" ([#15250](https://github.com/astral-sh/ruff/pull/15250))
## 0.8.5
### Preview features
- \[`airflow`\] Extend names moved from core to provider (`AIR303`) ([#15145](https://github.com/astral-sh/ruff/pull/15145), [#15159](https://github.com/astral-sh/ruff/pull/15159), [#15196](https://github.com/astral-sh/ruff/pull/15196), [#15216](https://github.com/astral-sh/ruff/pull/15216))
- \[`airflow`\] Extend rule to check class attributes, methods, arguments (`AIR302`) ([#15054](https://github.com/astral-sh/ruff/pull/15054), [#15083](https://github.com/astral-sh/ruff/pull/15083))
- \[`fastapi`\] Update `FAST002` to check keyword-only arguments ([#15119](https://github.com/astral-sh/ruff/pull/15119))
- \[`flake8-type-checking`\] Disable `TC006` and `TC007` in stub files ([#15179](https://github.com/astral-sh/ruff/pull/15179))
- \[`pylint`\] Detect nested methods correctly (`PLW1641`) ([#15032](https://github.com/astral-sh/ruff/pull/15032))
- \[`ruff`\] Detect more strict-integer expressions (`RUF046`) ([#14833](https://github.com/astral-sh/ruff/pull/14833))
- \[`ruff`\] Implement `falsy-dict-get-fallback` (`RUF056`) ([#15160](https://github.com/astral-sh/ruff/pull/15160))
- \[`ruff`\] Implement `unnecessary-round` (`RUF057`) ([#14828](https://github.com/astral-sh/ruff/pull/14828))
### Rule changes
- Visit PEP 764 inline `TypedDict` keys as non-type-expressions ([#15073](https://github.com/astral-sh/ruff/pull/15073))
- \[`flake8-comprehensions`\] Skip `C416` if comprehension contains unpacking ([#14909](https://github.com/astral-sh/ruff/pull/14909))
- \[`flake8-pie`\] Allow `cast(SomeType, ...)` (`PIE796`) ([#15141](https://github.com/astral-sh/ruff/pull/15141))
- \[`flake8-simplify`\] More precise inference for dictionaries (`SIM300`) ([#15164](https://github.com/astral-sh/ruff/pull/15164))
- \[`flake8-use-pathlib`\] Catch redundant joins in `PTH201` and avoid syntax errors ([#15177](https://github.com/astral-sh/ruff/pull/15177))
- \[`pycodestyle`\] Preserve original value format (`E731`) ([#15097](https://github.com/astral-sh/ruff/pull/15097))
- \[`pydocstyle`\] Split on first whitespace character (`D403`) ([#15082](https://github.com/astral-sh/ruff/pull/15082))
- \[`pyupgrade`\] Add all PEP-585 names to `UP006` rule ([#5454](https://github.com/astral-sh/ruff/pull/5454))
### Configuration
- \[`flake8-type-checking`\] Improve flexibility of `runtime-evaluated-decorators` ([#15204](https://github.com/astral-sh/ruff/pull/15204))
- \[`pydocstyle`\] Add setting to ignore missing documentation for `*args` and `**kwargs` parameters (`D417`) ([#15210](https://github.com/astral-sh/ruff/pull/15210))
- \[`ruff`\] Add an allowlist for `unsafe-markup-use` (`RUF035`) ([#15076](https://github.com/astral-sh/ruff/pull/15076))
### Bug fixes
- Fix type subscript on older python versions ([#15090](https://github.com/astral-sh/ruff/pull/15090))
- Use `TypeChecker` for detecting `fastapi` routes ([#15093](https://github.com/astral-sh/ruff/pull/15093))
- \[`pycodestyle`\] Avoid false positives and negatives related to type parameter default syntax (`E225`, `E251`) ([#15214](https://github.com/astral-sh/ruff/pull/15214))
### Documentation
- Fix incorrect doc in `shebang-not-executable` (`EXE001`) and add git+windows solution to executable bit ([#15208](https://github.com/astral-sh/ruff/pull/15208))
- Rename rules currently not conforming to naming convention ([#15102](https://github.com/astral-sh/ruff/pull/15102))
## 0.8.4
### Preview features
- \[`airflow`\] Extend `AIR302` with additional functions and classes ([#15015](https://github.com/astral-sh/ruff/pull/15015))
- \[`airflow`\] Implement `moved-to-provider-in-3` for modules that has been moved to Airflow providers (`AIR303`) ([#14764](https://github.com/astral-sh/ruff/pull/14764))
- \[`flake8-use-pathlib`\] Extend check for invalid path suffix to include the case `"."` (`PTH210`) ([#14902](https://github.com/astral-sh/ruff/pull/14902))
- \[`perflint`\] Fix panic in `PERF401` when list variable is after the `for` loop ([#14971](https://github.com/astral-sh/ruff/pull/14971))
- \[`perflint`\] Simplify finding the loop target in `PERF401` ([#15025](https://github.com/astral-sh/ruff/pull/15025))
- \[`pylint`\] Preserve original value format (`PLR6104`) ([#14978](https://github.com/astral-sh/ruff/pull/14978))
- \[`ruff`\] Avoid false positives for `RUF027` for typing context bindings ([#15037](https://github.com/astral-sh/ruff/pull/15037))
- \[`ruff`\] Check for ambiguous pattern passed to `pytest.raises()` (`RUF043`) ([#14966](https://github.com/astral-sh/ruff/pull/14966))
### Rule changes
- \[`flake8-bandit`\] Check `S105` for annotated assignment ([#15059](https://github.com/astral-sh/ruff/pull/15059))
- \[`flake8-pyi`\] More autofixes for `redundant-none-literal` (`PYI061`) ([#14872](https://github.com/astral-sh/ruff/pull/14872))
- \[`pydocstyle`\] Skip leading whitespace for `D403` ([#14963](https://github.com/astral-sh/ruff/pull/14963))
- \[`ruff`\] Skip `SQLModel` base classes for `mutable-class-default` (`RUF012`) ([#14949](https://github.com/astral-sh/ruff/pull/14949))
### Bug
- \[`perflint`\] Parenthesize walrus expressions in autofix for `manual-list-comprehension` (`PERF401`) ([#15050](https://github.com/astral-sh/ruff/pull/15050))
### Server
- Check diagnostic refresh support from client capability which enables dynamic configuration for various editors ([#15014](https://github.com/astral-sh/ruff/pull/15014))
## 0.8.3
### Preview features

View File

@@ -467,7 +467,7 @@ cargo build --release && hyperfine --warmup 10 \
"./target/release/ruff check ./crates/ruff_linter/resources/test/cpython/ --no-cache -e --select W505,E501"
```
You can run `poetry install` from `./scripts/benchmarks` to create a working environment for the
You can run `uv venv --project ./scripts/benchmarks`, activate the venv and then run `uv sync --project ./scripts/benchmarks` to create a working environment for the
above. All reported benchmarks were computed using the versions specified by
`./scripts/benchmarks/pyproject.toml` on Python 3.11.

594
Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -13,6 +13,7 @@ license = "MIT"
[workspace.dependencies]
ruff = { path = "crates/ruff" }
ruff_annotate_snippets = { path = "crates/ruff_annotate_snippets" }
ruff_cache = { path = "crates/ruff_cache" }
ruff_db = { path = "crates/ruff_db", default-features = false }
ruff_diagnostics = { path = "crates/ruff_diagnostics" }
@@ -43,7 +44,8 @@ red_knot_test = { path = "crates/red_knot_test" }
red_knot_workspace = { path = "crates/red_knot_workspace", default-features = false }
aho-corasick = { version = "1.1.3" }
annotate-snippets = { version = "0.9.2", features = ["color"] }
anstream = { version = "0.6.18" }
anstyle = { version = "1.0.10" }
anyhow = { version = "1.0.80" }
assert_fs = { version = "1.1.0" }
argfile = { version = "0.2.0" }
@@ -55,9 +57,9 @@ camino = { version = "1.1.7" }
chrono = { version = "0.4.35", default-features = false, features = ["clock"] }
clap = { version = "4.5.3", features = ["derive"] }
clap_complete_command = { version = "0.6.0" }
clearscreen = { version = "3.0.0" }
clearscreen = { version = "4.0.0" }
codspeed-criterion-compat = { version = "2.6.0", default-features = false }
colored = { version = "2.1.0" }
colored = { version = "3.0.0" }
console_error_panic_hook = { version = "0.1.7" }
console_log = { version = "1.0.0" }
countme = { version = "3.0.1" }
@@ -89,7 +91,7 @@ insta = { version = "1.35.1" }
insta-cmd = { version = "0.6.0" }
is-macro = { version = "0.3.5" }
is-wsl = { version = "0.4.0" }
itertools = { version = "0.13.0" }
itertools = { version = "0.14.0" }
js-sys = { version = "0.3.69" }
jod-thread = { version = "0.1.2" }
libc = { version = "0.2.153" }
@@ -119,7 +121,7 @@ rayon = { version = "1.10.0" }
regex = { version = "1.10.2" }
rustc-hash = { version = "2.0.0" }
# When updating salsa, make sure to also update the revision in `fuzz/Cargo.toml`
salsa = { git = "https://github.com/salsa-rs/salsa.git", rev = "3c7f1694c9efba751dbeeacfbc93b227586e316a" }
salsa = { git = "https://github.com/salsa-rs/salsa.git", rev = "88a1d7774d78f048fbd77d40abca9ebd729fd1f0" }
schemars = { version = "0.8.16" }
seahash = { version = "4.1.0" }
serde = { version = "1.0.197", features = ["derive"] }
@@ -132,6 +134,7 @@ serde_with = { version = "3.6.0", default-features = false, features = [
shellexpand = { version = "3.0.0" }
similar = { version = "2.4.0", features = ["inline"] }
smallvec = { version = "1.13.2" }
snapbox = { version = "0.6.0", features = ["diff", "term-svg", "cmd", "examples"] }
static_assertions = "1.1.0"
strum = { version = "0.26.0", features = ["strum_macros"] }
strum_macros = { version = "0.26.0" }
@@ -149,6 +152,7 @@ tracing-subscriber = { version = "0.3.18", default-features = false, features =
"fmt",
] }
tracing-tree = { version = "0.4.0" }
tryfn = { version = "0.2.1" }
typed-arena = { version = "2.0.2" }
unic-ucd-category = { version = "0.9" }
unicode-ident = { version = "1.0.12" }
@@ -211,6 +215,9 @@ redundant_clone = "warn"
debug_assert_with_mut_call = "warn"
unused_peekable = "warn"
# Diagnostics are not actionable: Enable once https://github.com/rust-lang/rust-clippy/issues/13774 is resolved.
large_stack_arrays = "allow"
[profile.release]
# Note that we set these explicitly, and these values
# were chosen based on a trade-off between compile times

View File

@@ -116,12 +116,21 @@ For more, see the [documentation](https://docs.astral.sh/ruff/).
### Installation
Ruff is available as [`ruff`](https://pypi.org/project/ruff/) on PyPI:
Ruff is available as [`ruff`](https://pypi.org/project/ruff/) on PyPI.
Invoke Ruff directly with [`uvx`](https://docs.astral.sh/uv/):
```shell
uvx ruff check # Lint all files in the current directory.
uvx ruff format # Format all files in the current directory.
```
Or install Ruff with `uv` (recommended), `pip`, or `pipx`:
```shell
# With uv.
uv add --dev ruff # to add ruff to your project
uv tool install ruff # to install ruff globally
uv tool install ruff@latest # Install Ruff globally.
uv add --dev ruff # Or add Ruff to your project.
# With pip.
pip install ruff
@@ -140,8 +149,8 @@ curl -LsSf https://astral.sh/ruff/install.sh | sh
powershell -c "irm https://astral.sh/ruff/install.ps1 | iex"
# For a specific version.
curl -LsSf https://astral.sh/ruff/0.8.3/install.sh | sh
powershell -c "irm https://astral.sh/ruff/0.8.3/install.ps1 | iex"
curl -LsSf https://astral.sh/ruff/0.9.1/install.sh | sh
powershell -c "irm https://astral.sh/ruff/0.9.1/install.ps1 | iex"
```
You can also install Ruff via [Homebrew](https://formulae.brew.sh/formula/ruff), [Conda](https://anaconda.org/conda-forge/ruff),
@@ -174,7 +183,7 @@ Ruff can also be used as a [pre-commit](https://pre-commit.com/) hook via [`ruff
```yaml
- repo: https://github.com/astral-sh/ruff-pre-commit
# Ruff version.
rev: v0.8.3
rev: v0.9.1
hooks:
# Run the linter.
- id: ruff
@@ -196,7 +205,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: astral-sh/ruff-action@v1
- uses: astral-sh/ruff-action@v3
```
### Configuration<a id="configuration"></a>

View File

@@ -1,10 +1,9 @@
[files]
# https://github.com/crate-ci/typos/issues/868
extend-exclude = [
"crates/red_knot_vendored/vendor/**/*",
"**/resources/**/*",
"**/snapshots/**/*",
"crates/red_knot_workspace/src/workspace/pyproject/package_name.rs"
"crates/red_knot_vendored/vendor/**/*",
"**/resources/**/*",
"**/snapshots/**/*",
]
[default.extend-words]
@@ -21,7 +20,10 @@ Numer = "Numer" # Library name 'NumerBlox' in "Who's Using Ruff?"
[default]
extend-ignore-re = [
# Line ignore with trailing "spellchecker:disable-line"
"(?Rm)^.*#\\s*spellchecker:disable-line$",
"LICENSEs",
# Line ignore with trailing "spellchecker:disable-line"
"(?Rm)^.*#\\s*spellchecker:disable-line$",
"LICENSEs",
]
[default.extend-identifiers]
"FrIeNdLy" = "FrIeNdLy"

View File

@@ -8,11 +8,11 @@ use crossbeam::channel as crossbeam_channel;
use python_version::PythonVersion;
use red_knot_python_semantic::SitePackages;
use red_knot_server::run_server;
use red_knot_workspace::db::RootDatabase;
use red_knot_workspace::db::ProjectDatabase;
use red_knot_workspace::project::settings::Configuration;
use red_knot_workspace::project::ProjectMetadata;
use red_knot_workspace::watch;
use red_knot_workspace::watch::WorkspaceWatcher;
use red_knot_workspace::workspace::settings::Configuration;
use red_knot_workspace::workspace::WorkspaceMetadata;
use red_knot_workspace::watch::ProjectWatcher;
use ruff_db::diagnostic::Diagnostic;
use ruff_db::system::{OsSystem, System, SystemPath, SystemPathBuf};
use salsa::plumbing::ZalsaDatabase;
@@ -165,7 +165,7 @@ fn run() -> anyhow::Result<ExitStatus> {
let system = OsSystem::new(cwd.clone());
let cli_configuration = args.to_configuration(&cwd);
let workspace_metadata = WorkspaceMetadata::discover(
let workspace_metadata = ProjectMetadata::discover(
system.current_directory(),
&system,
Some(&cli_configuration),
@@ -173,7 +173,7 @@ fn run() -> anyhow::Result<ExitStatus> {
// TODO: Use the `program_settings` to compute the key for the database's persistent
// cache and load the cache if it exists.
let mut db = RootDatabase::new(workspace_metadata, system)?;
let mut db = ProjectDatabase::new(workspace_metadata, system)?;
let (main_loop, main_loop_cancellation_token) = MainLoop::new(cli_configuration);
@@ -226,7 +226,7 @@ struct MainLoop {
receiver: crossbeam_channel::Receiver<MainLoopMessage>,
/// The file system watcher, if running in watch mode.
watcher: Option<WorkspaceWatcher>,
watcher: Option<ProjectWatcher>,
cli_configuration: Configuration,
}
@@ -246,21 +246,21 @@ impl MainLoop {
)
}
fn watch(mut self, db: &mut RootDatabase) -> anyhow::Result<ExitStatus> {
fn watch(mut self, db: &mut ProjectDatabase) -> anyhow::Result<ExitStatus> {
tracing::debug!("Starting watch mode");
let sender = self.sender.clone();
let watcher = watch::directory_watcher(move |event| {
sender.send(MainLoopMessage::ApplyChanges(event)).unwrap();
})?;
self.watcher = Some(WorkspaceWatcher::new(watcher, db));
self.watcher = Some(ProjectWatcher::new(watcher, db));
self.run(db);
Ok(ExitStatus::Success)
}
fn run(mut self, db: &mut RootDatabase) -> ExitStatus {
fn run(mut self, db: &mut ProjectDatabase) -> ExitStatus {
self.sender.send(MainLoopMessage::CheckWorkspace).unwrap();
let result = self.main_loop(db);
@@ -270,7 +270,7 @@ impl MainLoop {
result
}
fn main_loop(&mut self, db: &mut RootDatabase) -> ExitStatus {
fn main_loop(&mut self, db: &mut ProjectDatabase) -> ExitStatus {
// Schedule the first check.
tracing::debug!("Starting main loop");
@@ -282,7 +282,7 @@ impl MainLoop {
let db = db.clone();
let sender = self.sender.clone();
// Spawn a new task that checks the workspace. This needs to be done in a separate thread
// Spawn a new task that checks the project. This needs to be done in a separate thread
// to prevent blocking the main loop here.
rayon::spawn(move || {
if let Ok(result) = db.check() {

View File

@@ -5,18 +5,18 @@ use std::time::{Duration, Instant};
use anyhow::{anyhow, Context};
use red_knot_python_semantic::{resolve_module, ModuleName, Program, PythonVersion, SitePackages};
use red_knot_workspace::db::{Db, RootDatabase};
use red_knot_workspace::watch::{directory_watcher, ChangeEvent, WorkspaceWatcher};
use red_knot_workspace::workspace::settings::{Configuration, SearchPathConfiguration};
use red_knot_workspace::workspace::WorkspaceMetadata;
use red_knot_workspace::db::{Db, ProjectDatabase};
use red_knot_workspace::project::settings::{Configuration, SearchPathConfiguration};
use red_knot_workspace::project::ProjectMetadata;
use red_knot_workspace::watch::{directory_watcher, ChangeEvent, ProjectWatcher};
use ruff_db::files::{system_path_to_file, File, FileError};
use ruff_db::source::source_text;
use ruff_db::system::{OsSystem, SystemPath, SystemPathBuf};
use ruff_db::Upcast;
struct TestCase {
db: RootDatabase,
watcher: Option<WorkspaceWatcher>,
db: ProjectDatabase,
watcher: Option<ProjectWatcher>,
changes_receiver: crossbeam::channel::Receiver<Vec<ChangeEvent>>,
/// The temporary directory that contains the test files.
/// We need to hold on to it in the test case or the temp files get deleted.
@@ -26,15 +26,15 @@ struct TestCase {
}
impl TestCase {
fn workspace_path(&self, relative: impl AsRef<SystemPath>) -> SystemPathBuf {
SystemPath::absolute(relative, self.db.workspace().root(&self.db))
fn project_path(&self, relative: impl AsRef<SystemPath>) -> SystemPathBuf {
SystemPath::absolute(relative, self.db.project().root(&self.db))
}
fn root_path(&self) -> &SystemPath {
&self.root_dir
}
fn db(&self) -> &RootDatabase {
fn db(&self) -> &ProjectDatabase {
&self.db
}
@@ -150,7 +150,7 @@ impl TestCase {
) -> anyhow::Result<()> {
let program = Program::get(self.db());
let new_settings = configuration.to_settings(self.db.workspace().root(&self.db));
let new_settings = configuration.to_settings(self.db.project().root(&self.db));
self.configuration.search_paths = configuration;
program.update_search_paths(&mut self.db, &new_settings)?;
@@ -163,9 +163,8 @@ impl TestCase {
Ok(())
}
fn collect_package_files(&self, path: &SystemPath) -> Vec<File> {
let package = self.db().workspace().package(self.db(), path).unwrap();
let files = package.files(self.db());
fn collect_project_files(&self) -> Vec<File> {
let files = self.db().project().files(self.db());
let mut collected: Vec<_> = files.into_iter().collect();
collected.sort_unstable_by_key(|file| file.path(self.db()).as_system_path().unwrap());
collected
@@ -194,17 +193,17 @@ where
}
trait SetupFiles {
fn setup(self, root_path: &SystemPath, workspace_path: &SystemPath) -> anyhow::Result<()>;
fn setup(self, root_path: &SystemPath, project_path: &SystemPath) -> anyhow::Result<()>;
}
impl<const N: usize, P> SetupFiles for [(P, &'static str); N]
where
P: AsRef<SystemPath>,
{
fn setup(self, _root_path: &SystemPath, workspace_path: &SystemPath) -> anyhow::Result<()> {
fn setup(self, _root_path: &SystemPath, project_path: &SystemPath) -> anyhow::Result<()> {
for (relative_path, content) in self {
let relative_path = relative_path.as_ref();
let absolute_path = workspace_path.join(relative_path);
let absolute_path = project_path.join(relative_path);
if let Some(parent) = absolute_path.parent() {
std::fs::create_dir_all(parent).with_context(|| {
format!("Failed to create parent directory for file `{relative_path}`")
@@ -226,8 +225,8 @@ impl<F> SetupFiles for F
where
F: FnOnce(&SystemPath, &SystemPath) -> anyhow::Result<()>,
{
fn setup(self, root_path: &SystemPath, workspace_path: &SystemPath) -> anyhow::Result<()> {
self(root_path, workspace_path)
fn setup(self, root_path: &SystemPath, project_path: &SystemPath) -> anyhow::Result<()> {
self(root_path, project_path)
}
}
@@ -235,7 +234,7 @@ fn setup<F>(setup_files: F) -> anyhow::Result<TestCase>
where
F: SetupFiles,
{
setup_with_search_paths(setup_files, |_root, _workspace_path| {
setup_with_search_paths(setup_files, |_root, _project_path| {
SearchPathConfiguration::default()
})
}
@@ -265,18 +264,18 @@ where
.simplified()
.to_path_buf();
let workspace_path = root_path.join("workspace");
let project_path = root_path.join("project");
std::fs::create_dir_all(workspace_path.as_std_path())
.with_context(|| format!("Failed to create workspace directory `{workspace_path}`"))?;
std::fs::create_dir_all(project_path.as_std_path())
.with_context(|| format!("Failed to create project directory `{project_path}`"))?;
setup_files
.setup(&root_path, &workspace_path)
.setup(&root_path, &project_path)
.context("Failed to setup test files")?;
let system = OsSystem::new(&workspace_path);
let system = OsSystem::new(&project_path);
let search_paths = create_search_paths(&root_path, &workspace_path);
let search_paths = create_search_paths(&root_path, &project_path);
for path in search_paths
.extra_paths
@@ -300,15 +299,15 @@ where
search_paths,
};
let workspace = WorkspaceMetadata::discover(&workspace_path, &system, Some(&configuration))?;
let project = ProjectMetadata::discover(&project_path, &system, Some(&configuration))?;
let db = RootDatabase::new(workspace, system)?;
let db = ProjectDatabase::new(project, system)?;
let (sender, receiver) = crossbeam::channel::unbounded();
let watcher = directory_watcher(move |events| sender.send(events).unwrap())
.with_context(|| "Failed to create directory watcher")?;
let watcher = WorkspaceWatcher::new(watcher, &db);
let watcher = ProjectWatcher::new(watcher, &db);
assert!(!watcher.has_errored_paths());
let test_case = TestCase {
@@ -359,12 +358,12 @@ fn update_file(path: impl AsRef<SystemPath>, content: &str) -> anyhow::Result<()
#[test]
fn new_file() -> anyhow::Result<()> {
let mut case = setup([("bar.py", "")])?;
let bar_path = case.workspace_path("bar.py");
let bar_path = case.project_path("bar.py");
let bar_file = case.system_file(&bar_path).unwrap();
let foo_path = case.workspace_path("foo.py");
let foo_path = case.project_path("foo.py");
assert_eq!(case.system_file(&foo_path), Err(FileError::NotFound));
assert_eq!(&case.collect_package_files(&bar_path), &[bar_file]);
assert_eq!(&case.collect_project_files(), &[bar_file]);
std::fs::write(foo_path.as_std_path(), "print('Hello')")?;
@@ -374,7 +373,7 @@ fn new_file() -> anyhow::Result<()> {
let foo = case.system_file(&foo_path).expect("foo.py to exist.");
assert_eq!(&case.collect_package_files(&bar_path), &[bar_file, foo]);
assert_eq!(&case.collect_project_files(), &[bar_file, foo]);
Ok(())
}
@@ -382,12 +381,12 @@ fn new_file() -> anyhow::Result<()> {
#[test]
fn new_ignored_file() -> anyhow::Result<()> {
let mut case = setup([("bar.py", ""), (".ignore", "foo.py")])?;
let bar_path = case.workspace_path("bar.py");
let bar_path = case.project_path("bar.py");
let bar_file = case.system_file(&bar_path).unwrap();
let foo_path = case.workspace_path("foo.py");
let foo_path = case.project_path("foo.py");
assert_eq!(case.system_file(&foo_path), Err(FileError::NotFound));
assert_eq!(&case.collect_package_files(&bar_path), &[bar_file]);
assert_eq!(&case.collect_project_files(), &[bar_file]);
std::fs::write(foo_path.as_std_path(), "print('Hello')")?;
@@ -396,7 +395,7 @@ fn new_ignored_file() -> anyhow::Result<()> {
case.apply_changes(changes);
assert!(case.system_file(&foo_path).is_ok());
assert_eq!(&case.collect_package_files(&bar_path), &[bar_file]);
assert_eq!(&case.collect_project_files(), &[bar_file]);
Ok(())
}
@@ -405,11 +404,11 @@ fn new_ignored_file() -> anyhow::Result<()> {
fn changed_file() -> anyhow::Result<()> {
let foo_source = "print('Hello, world!')";
let mut case = setup([("foo.py", foo_source)])?;
let foo_path = case.workspace_path("foo.py");
let foo_path = case.project_path("foo.py");
let foo = case.system_file(&foo_path)?;
assert_eq!(source_text(case.db(), foo).as_str(), foo_source);
assert_eq!(&case.collect_package_files(&foo_path), &[foo]);
assert_eq!(&case.collect_project_files(), &[foo]);
update_file(&foo_path, "print('Version 2')")?;
@@ -420,7 +419,7 @@ fn changed_file() -> anyhow::Result<()> {
case.apply_changes(changes);
assert_eq!(source_text(case.db(), foo).as_str(), "print('Version 2')");
assert_eq!(&case.collect_package_files(&foo_path), &[foo]);
assert_eq!(&case.collect_project_files(), &[foo]);
Ok(())
}
@@ -429,12 +428,12 @@ fn changed_file() -> anyhow::Result<()> {
fn deleted_file() -> anyhow::Result<()> {
let foo_source = "print('Hello, world!')";
let mut case = setup([("foo.py", foo_source)])?;
let foo_path = case.workspace_path("foo.py");
let foo_path = case.project_path("foo.py");
let foo = case.system_file(&foo_path)?;
assert!(foo.exists(case.db()));
assert_eq!(&case.collect_package_files(&foo_path), &[foo]);
assert_eq!(&case.collect_project_files(), &[foo]);
std::fs::remove_file(foo_path.as_std_path())?;
@@ -443,7 +442,7 @@ fn deleted_file() -> anyhow::Result<()> {
case.apply_changes(changes);
assert!(!foo.exists(case.db()));
assert_eq!(&case.collect_package_files(&foo_path), &[] as &[File]);
assert_eq!(&case.collect_project_files(), &[] as &[File]);
Ok(())
}
@@ -455,7 +454,7 @@ fn deleted_file() -> anyhow::Result<()> {
fn move_file_to_trash() -> anyhow::Result<()> {
let foo_source = "print('Hello, world!')";
let mut case = setup([("foo.py", foo_source)])?;
let foo_path = case.workspace_path("foo.py");
let foo_path = case.project_path("foo.py");
let trash_path = case.root_path().join(".trash");
std::fs::create_dir_all(trash_path.as_std_path())?;
@@ -463,7 +462,7 @@ fn move_file_to_trash() -> anyhow::Result<()> {
let foo = case.system_file(&foo_path)?;
assert!(foo.exists(case.db()));
assert_eq!(&case.collect_package_files(&foo_path), &[foo]);
assert_eq!(&case.collect_project_files(), &[foo]);
std::fs::rename(
foo_path.as_std_path(),
@@ -475,58 +474,50 @@ fn move_file_to_trash() -> anyhow::Result<()> {
case.apply_changes(changes);
assert!(!foo.exists(case.db()));
assert_eq!(&case.collect_package_files(&foo_path), &[] as &[File]);
assert_eq!(&case.collect_project_files(), &[] as &[File]);
Ok(())
}
/// Move a file from a non-workspace (non-watched) location into the workspace.
/// Move a file from a non-project (non-watched) location into the project.
#[test]
fn move_file_to_workspace() -> anyhow::Result<()> {
fn move_file_to_project() -> anyhow::Result<()> {
let mut case = setup([("bar.py", "")])?;
let bar_path = case.workspace_path("bar.py");
let bar_path = case.project_path("bar.py");
let bar = case.system_file(&bar_path).unwrap();
let foo_path = case.root_path().join("foo.py");
std::fs::write(foo_path.as_std_path(), "")?;
let foo_in_workspace_path = case.workspace_path("foo.py");
let foo_in_project = case.project_path("foo.py");
assert!(case.system_file(&foo_path).is_ok());
assert_eq!(&case.collect_package_files(&bar_path), &[bar]);
assert!(case
.db()
.workspace()
.package(case.db(), &foo_path)
.is_none());
assert_eq!(&case.collect_project_files(), &[bar]);
std::fs::rename(foo_path.as_std_path(), foo_in_workspace_path.as_std_path())?;
std::fs::rename(foo_path.as_std_path(), foo_in_project.as_std_path())?;
let changes = case.stop_watch(event_for_file("foo.py"));
case.apply_changes(changes);
let foo_in_workspace = case.system_file(&foo_in_workspace_path)?;
let foo_in_project = case.system_file(&foo_in_project)?;
assert!(foo_in_workspace.exists(case.db()));
assert_eq!(
&case.collect_package_files(&foo_in_workspace_path),
&[bar, foo_in_workspace]
);
assert!(foo_in_project.exists(case.db()));
assert_eq!(&case.collect_project_files(), &[bar, foo_in_project]);
Ok(())
}
/// Rename a workspace file.
/// Rename a project file.
#[test]
fn rename_file() -> anyhow::Result<()> {
let mut case = setup([("foo.py", "")])?;
let foo_path = case.workspace_path("foo.py");
let bar_path = case.workspace_path("bar.py");
let foo_path = case.project_path("foo.py");
let bar_path = case.project_path("bar.py");
let foo = case.system_file(&foo_path)?;
assert_eq!(case.collect_package_files(&foo_path), [foo]);
assert_eq!(case.collect_project_files(), [foo]);
std::fs::rename(foo_path.as_std_path(), bar_path.as_std_path())?;
@@ -539,15 +530,15 @@ fn rename_file() -> anyhow::Result<()> {
let bar = case.system_file(&bar_path)?;
assert!(bar.exists(case.db()));
assert_eq!(case.collect_package_files(&foo_path), [bar]);
assert_eq!(case.collect_project_files(), [bar]);
Ok(())
}
#[test]
fn directory_moved_to_workspace() -> anyhow::Result<()> {
fn directory_moved_to_project() -> anyhow::Result<()> {
let mut case = setup([("bar.py", "import sub.a")])?;
let bar = case.system_file(case.workspace_path("bar.py")).unwrap();
let bar = case.system_file(case.project_path("bar.py")).unwrap();
let sub_original_path = case.root_path().join("sub");
let init_original_path = sub_original_path.join("__init__.py");
@@ -565,12 +556,9 @@ fn directory_moved_to_workspace() -> anyhow::Result<()> {
);
assert_eq!(sub_a_module, None);
assert_eq!(
case.collect_package_files(&case.workspace_path("bar.py")),
&[bar]
);
assert_eq!(case.collect_project_files(), &[bar]);
let sub_new_path = case.workspace_path("sub");
let sub_new_path = case.project_path("sub");
std::fs::rename(sub_original_path.as_std_path(), sub_new_path.as_std_path())
.with_context(|| "Failed to move sub directory")?;
@@ -592,10 +580,7 @@ fn directory_moved_to_workspace() -> anyhow::Result<()> {
)
.is_some());
assert_eq!(
case.collect_package_files(&case.workspace_path("bar.py")),
&[bar, init_file, a_file]
);
assert_eq!(case.collect_project_files(), &[bar, init_file, a_file]);
Ok(())
}
@@ -607,7 +592,7 @@ fn directory_moved_to_trash() -> anyhow::Result<()> {
("sub/__init__.py", ""),
("sub/a.py", ""),
])?;
let bar = case.system_file(case.workspace_path("bar.py")).unwrap();
let bar = case.system_file(case.project_path("bar.py")).unwrap();
assert!(resolve_module(
case.db().upcast(),
@@ -615,7 +600,7 @@ fn directory_moved_to_trash() -> anyhow::Result<()> {
)
.is_some());
let sub_path = case.workspace_path("sub");
let sub_path = case.project_path("sub");
let init_file = case
.system_file(sub_path.join("__init__.py"))
.expect("__init__.py to exist");
@@ -623,10 +608,7 @@ fn directory_moved_to_trash() -> anyhow::Result<()> {
.system_file(sub_path.join("a.py"))
.expect("a.py to exist");
assert_eq!(
case.collect_package_files(&case.workspace_path("bar.py")),
&[bar, init_file, a_file]
);
assert_eq!(case.collect_project_files(), &[bar, init_file, a_file]);
std::fs::create_dir(case.root_path().join(".trash").as_std_path())?;
let trashed_sub = case.root_path().join(".trash/sub");
@@ -647,10 +629,7 @@ fn directory_moved_to_trash() -> anyhow::Result<()> {
assert!(!init_file.exists(case.db()));
assert!(!a_file.exists(case.db()));
assert_eq!(
case.collect_package_files(&case.workspace_path("bar.py")),
&[bar]
);
assert_eq!(case.collect_project_files(), &[bar]);
Ok(())
}
@@ -663,7 +642,7 @@ fn directory_renamed() -> anyhow::Result<()> {
("sub/a.py", ""),
])?;
let bar = case.system_file(case.workspace_path("bar.py")).unwrap();
let bar = case.system_file(case.project_path("bar.py")).unwrap();
assert!(resolve_module(
case.db().upcast(),
@@ -676,7 +655,7 @@ fn directory_renamed() -> anyhow::Result<()> {
)
.is_none());
let sub_path = case.workspace_path("sub");
let sub_path = case.project_path("sub");
let sub_init = case
.system_file(sub_path.join("__init__.py"))
.expect("__init__.py to exist");
@@ -684,14 +663,11 @@ fn directory_renamed() -> anyhow::Result<()> {
.system_file(sub_path.join("a.py"))
.expect("a.py to exist");
assert_eq!(
case.collect_package_files(&sub_path),
&[bar, sub_init, sub_a]
);
assert_eq!(case.collect_project_files(), &[bar, sub_init, sub_a]);
let foo_baz = case.workspace_path("foo/baz");
let foo_baz = case.project_path("foo/baz");
std::fs::create_dir(case.workspace_path("foo").as_std_path())?;
std::fs::create_dir(case.project_path("foo").as_std_path())?;
std::fs::rename(sub_path.as_std_path(), foo_baz.as_std_path())
.with_context(|| "Failed to move the sub directory")?;
@@ -730,7 +706,7 @@ fn directory_renamed() -> anyhow::Result<()> {
assert!(foo_baz_a.exists(case.db()));
assert_eq!(
case.collect_package_files(&sub_path),
case.collect_project_files(),
&[bar, foo_baz_init, foo_baz_a]
);
@@ -745,7 +721,7 @@ fn directory_deleted() -> anyhow::Result<()> {
("sub/a.py", ""),
])?;
let bar = case.system_file(case.workspace_path("bar.py")).unwrap();
let bar = case.system_file(case.project_path("bar.py")).unwrap();
assert!(resolve_module(
case.db().upcast(),
@@ -753,7 +729,7 @@ fn directory_deleted() -> anyhow::Result<()> {
)
.is_some());
let sub_path = case.workspace_path("sub");
let sub_path = case.project_path("sub");
let init_file = case
.system_file(sub_path.join("__init__.py"))
@@ -761,10 +737,7 @@ fn directory_deleted() -> anyhow::Result<()> {
let a_file = case
.system_file(sub_path.join("a.py"))
.expect("a.py to exist");
assert_eq!(
case.collect_package_files(&sub_path),
&[bar, init_file, a_file]
);
assert_eq!(case.collect_project_files(), &[bar, init_file, a_file]);
std::fs::remove_dir_all(sub_path.as_std_path())
.with_context(|| "Failed to remove the sub directory")?;
@@ -782,20 +755,20 @@ fn directory_deleted() -> anyhow::Result<()> {
assert!(!init_file.exists(case.db()));
assert!(!a_file.exists(case.db()));
assert_eq!(case.collect_package_files(&sub_path), &[bar]);
assert_eq!(case.collect_project_files(), &[bar]);
Ok(())
}
#[test]
fn search_path() -> anyhow::Result<()> {
let mut case = setup_with_search_paths(
[("bar.py", "import sub.a")],
|root_path, _workspace_path| SearchPathConfiguration {
site_packages: Some(SitePackages::Known(vec![root_path.join("site_packages")])),
..SearchPathConfiguration::default()
},
)?;
let mut case =
setup_with_search_paths([("bar.py", "import sub.a")], |root_path, _project_path| {
SearchPathConfiguration {
site_packages: Some(SitePackages::Known(vec![root_path.join("site_packages")])),
..SearchPathConfiguration::default()
}
})?;
let site_packages = case.root_path().join("site_packages");
@@ -812,8 +785,8 @@ fn search_path() -> anyhow::Result<()> {
assert!(resolve_module(case.db().upcast(), &ModuleName::new_static("a").unwrap()).is_some());
assert_eq!(
case.collect_package_files(&case.workspace_path("bar.py")),
&[case.system_file(case.workspace_path("bar.py")).unwrap()]
case.collect_project_files(),
&[case.system_file(case.project_path("bar.py")).unwrap()]
);
Ok(())
@@ -823,7 +796,7 @@ fn search_path() -> anyhow::Result<()> {
fn add_search_path() -> anyhow::Result<()> {
let mut case = setup([("bar.py", "import sub.a")])?;
let site_packages = case.workspace_path("site_packages");
let site_packages = case.project_path("site_packages");
std::fs::create_dir_all(site_packages.as_std_path())?;
assert!(resolve_module(case.db().upcast(), &ModuleName::new_static("a").unwrap()).is_none());
@@ -848,13 +821,13 @@ fn add_search_path() -> anyhow::Result<()> {
#[test]
fn remove_search_path() -> anyhow::Result<()> {
let mut case = setup_with_search_paths(
[("bar.py", "import sub.a")],
|root_path, _workspace_path| SearchPathConfiguration {
site_packages: Some(SitePackages::Known(vec![root_path.join("site_packages")])),
..SearchPathConfiguration::default()
},
)?;
let mut case =
setup_with_search_paths([("bar.py", "import sub.a")], |root_path, _project_path| {
SearchPathConfiguration {
site_packages: Some(SitePackages::Known(vec![root_path.join("site_packages")])),
..SearchPathConfiguration::default()
}
})?;
// Remove site packages from the search path settings.
let site_packages = case.root_path().join("site_packages");
@@ -876,8 +849,8 @@ fn remove_search_path() -> anyhow::Result<()> {
#[test]
fn changed_versions_file() -> anyhow::Result<()> {
let mut case = setup_with_search_paths(
|root_path: &SystemPath, workspace_path: &SystemPath| {
std::fs::write(workspace_path.join("bar.py").as_std_path(), "import sub.a")?;
|root_path: &SystemPath, project_path: &SystemPath| {
std::fs::write(project_path.join("bar.py").as_std_path(), "import sub.a")?;
std::fs::create_dir_all(root_path.join("typeshed/stdlib").as_std_path())?;
std::fs::write(root_path.join("typeshed/stdlib/VERSIONS").as_std_path(), "")?;
std::fs::write(
@@ -887,7 +860,7 @@ fn changed_versions_file() -> anyhow::Result<()> {
Ok(())
},
|root_path, _workspace_path| SearchPathConfiguration {
|root_path, _project_path| SearchPathConfiguration {
typeshed: Some(root_path.join("typeshed")),
..SearchPathConfiguration::default()
},
@@ -915,11 +888,11 @@ fn changed_versions_file() -> anyhow::Result<()> {
Ok(())
}
/// Watch a workspace that contains two files where one file is a hardlink to another.
/// Watch a project that contains two files where one file is a hardlink to another.
///
/// Setup:
/// ```text
/// - workspace
/// - project
/// |- foo.py
/// |- bar.py (hard link to foo.py)
/// ```
@@ -935,22 +908,22 @@ fn changed_versions_file() -> anyhow::Result<()> {
/// I haven't found any documentation that states the notification behavior on Windows but what
/// we're seeing is that Windows only emits a single event, similar to Linux.
#[test]
fn hard_links_in_workspace() -> anyhow::Result<()> {
let mut case = setup(|_root: &SystemPath, workspace: &SystemPath| {
let foo_path = workspace.join("foo.py");
fn hard_links_in_project() -> anyhow::Result<()> {
let mut case = setup(|_root: &SystemPath, project: &SystemPath| {
let foo_path = project.join("foo.py");
std::fs::write(foo_path.as_std_path(), "print('Version 1')")?;
// Create a hardlink to `foo`
let bar_path = workspace.join("bar.py");
let bar_path = project.join("bar.py");
std::fs::hard_link(foo_path.as_std_path(), bar_path.as_std_path())
.context("Failed to create hard link from foo.py -> bar.py")?;
Ok(())
})?;
let foo_path = case.workspace_path("foo.py");
let foo_path = case.project_path("foo.py");
let foo = case.system_file(&foo_path).unwrap();
let bar_path = case.workspace_path("bar.py");
let bar_path = case.project_path("bar.py");
let bar = case.system_file(&bar_path).unwrap();
assert_eq!(source_text(case.db(), foo).as_str(), "print('Version 1')");
@@ -973,12 +946,12 @@ fn hard_links_in_workspace() -> anyhow::Result<()> {
Ok(())
}
/// Watch a workspace that contains one file that is a hardlink to a file outside the workspace.
/// Watch a project that contains one file that is a hardlink to a file outside the project.
///
/// Setup:
/// ```text
/// - foo.py
/// - workspace
/// - project
/// |- bar.py (hard link to /foo.py)
/// ```
///
@@ -996,7 +969,7 @@ fn hard_links_in_workspace() -> anyhow::Result<()> {
/// [source](https://learn.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-readdirectorychangesw)
///
/// My interpretation of this is that Windows doesn't support observing changes made to
/// hard linked files outside the workspace.
/// hard linked files outside the project.
#[test]
#[cfg_attr(
target_os = "linux",
@@ -1006,13 +979,13 @@ fn hard_links_in_workspace() -> anyhow::Result<()> {
target_os = "windows",
ignore = "windows doesn't support observing changes to hard linked files."
)]
fn hard_links_to_target_outside_workspace() -> anyhow::Result<()> {
let mut case = setup(|root: &SystemPath, workspace: &SystemPath| {
fn hard_links_to_target_outside_project() -> anyhow::Result<()> {
let mut case = setup(|root: &SystemPath, project: &SystemPath| {
let foo_path = root.join("foo.py");
std::fs::write(foo_path.as_std_path(), "print('Version 1')")?;
// Create a hardlink to `foo`
let bar_path = workspace.join("bar.py");
let bar_path = project.join("bar.py");
std::fs::hard_link(foo_path.as_std_path(), bar_path.as_std_path())
.context("Failed to create hard link from foo.py -> bar.py")?;
@@ -1021,7 +994,7 @@ fn hard_links_to_target_outside_workspace() -> anyhow::Result<()> {
let foo_path = case.root_path().join("foo.py");
let foo = case.system_file(&foo_path).unwrap();
let bar_path = case.workspace_path("bar.py");
let bar_path = case.project_path("bar.py");
let bar = case.system_file(&bar_path).unwrap();
assert_eq!(source_text(case.db(), foo).as_str(), "print('Version 1')");
@@ -1044,13 +1017,13 @@ mod unix {
//! Tests that make use of unix specific file-system features.
use super::*;
/// Changes the metadata of the only file in the workspace.
/// Changes the metadata of the only file in the project.
#[test]
fn changed_metadata() -> anyhow::Result<()> {
use std::os::unix::fs::PermissionsExt;
let mut case = setup([("foo.py", "")])?;
let foo_path = case.workspace_path("foo.py");
let foo_path = case.project_path("foo.py");
let foo = case.system_file(&foo_path)?;
assert_eq!(
@@ -1086,14 +1059,14 @@ mod unix {
Ok(())
}
/// A workspace path is a symlink to a file outside the workspace.
/// A project path is a symlink to a file outside the project.
///
/// Setup:
/// ```text
/// - bar
/// |- baz.py
///
/// - workspace
/// - project
/// |- bar -> /bar
/// ```
///
@@ -1115,7 +1088,7 @@ mod unix {
ignore = "FSEvents doesn't emit change events for symlinked directories outside of the watched paths."
)]
fn symlink_target_outside_watched_paths() -> anyhow::Result<()> {
let mut case = setup(|root: &SystemPath, workspace: &SystemPath| {
let mut case = setup(|root: &SystemPath, project: &SystemPath| {
// Set up the symlink target.
let link_target = root.join("bar");
std::fs::create_dir_all(link_target.as_std_path())
@@ -1124,8 +1097,8 @@ mod unix {
std::fs::write(baz_original.as_std_path(), "def baz(): ...")
.context("Failed to write link target file")?;
// Create a symlink inside the workspace
let bar = workspace.join("bar");
// Create a symlink inside the project
let bar = project.join("bar");
std::os::unix::fs::symlink(link_target.as_std_path(), bar.as_std_path())
.context("Failed to create symlink to bar package")?;
@@ -1137,7 +1110,7 @@ mod unix {
&ModuleName::new_static("bar.baz").unwrap(),
)
.expect("Expected bar.baz to exist in site-packages.");
let baz_workspace = case.workspace_path("bar/baz.py");
let baz_project = case.project_path("bar/baz.py");
assert_eq!(
source_text(case.db(), baz.file()).as_str(),
@@ -1145,7 +1118,7 @@ mod unix {
);
assert_eq!(
baz.file().path(case.db()).as_system_path(),
Some(&*baz_workspace)
Some(&*baz_project)
);
let baz_original = case.root_path().join("bar/baz.py");
@@ -1164,7 +1137,7 @@ mod unix {
);
// Write to the symlink source.
update_file(baz_workspace, "def baz(): print('Version 3')")
update_file(baz_project, "def baz(): print('Version 3')")
.context("Failed to update bar/baz.py")?;
let changes = case.stop_watch(event_for_file("baz.py"));
@@ -1179,14 +1152,14 @@ mod unix {
Ok(())
}
/// Workspace contains a symlink to another directory inside the workspace.
/// Project contains a symlink to another directory inside the project.
/// Changes to files in the symlinked directory should be reflected
/// to all files.
///
/// Setup:
/// ```text
/// - workspace
/// | - bar -> /workspace/patched/bar
/// - project
/// | - bar -> /project/patched/bar
/// |
/// | - patched
/// | |-- bar
@@ -1195,10 +1168,10 @@ mod unix {
/// |-- foo.py
/// ```
#[test]
fn symlink_inside_workspace() -> anyhow::Result<()> {
let mut case = setup(|_root: &SystemPath, workspace: &SystemPath| {
fn symlink_inside_project() -> anyhow::Result<()> {
let mut case = setup(|_root: &SystemPath, project: &SystemPath| {
// Set up the symlink target.
let link_target = workspace.join("patched/bar");
let link_target = project.join("patched/bar");
std::fs::create_dir_all(link_target.as_std_path())
.context("Failed to create link target directory")?;
let baz_original = link_target.join("baz.py");
@@ -1206,8 +1179,8 @@ mod unix {
.context("Failed to write link target file")?;
// Create a symlink inside site-packages
let bar_in_workspace = workspace.join("bar");
std::os::unix::fs::symlink(link_target.as_std_path(), bar_in_workspace.as_std_path())
let bar_in_project = project.join("bar");
std::os::unix::fs::symlink(link_target.as_std_path(), bar_in_project.as_std_path())
.context("Failed to create symlink to bar package")?;
Ok(())
@@ -1218,9 +1191,9 @@ mod unix {
&ModuleName::new_static("bar.baz").unwrap(),
)
.expect("Expected bar.baz to exist in site-packages.");
let bar_baz = case.workspace_path("bar/baz.py");
let bar_baz = case.project_path("bar/baz.py");
let patched_bar_baz = case.workspace_path("patched/bar/baz.py");
let patched_bar_baz = case.project_path("patched/bar/baz.py");
let patched_bar_baz_file = case.system_file(&patched_bar_baz).unwrap();
assert_eq!(
@@ -1279,7 +1252,7 @@ mod unix {
/// - site-packages
/// | - bar/baz.py
///
/// - workspace
/// - project
/// |-- .venv/lib/python3.12/site-packages -> /site-packages
/// |
/// |-- foo.py
@@ -1287,7 +1260,7 @@ mod unix {
#[test]
fn symlinked_module_search_path() -> anyhow::Result<()> {
let mut case = setup_with_search_paths(
|root: &SystemPath, workspace: &SystemPath| {
|root: &SystemPath, project: &SystemPath| {
// Set up the symlink target.
let site_packages = root.join("site-packages");
let bar = site_packages.join("bar");
@@ -1298,7 +1271,7 @@ mod unix {
.context("Failed to write baz.py")?;
// Symlink the site packages in the venv to the global site packages
let venv_site_packages = workspace.join(".venv/lib/python3.12/site-packages");
let venv_site_packages = project.join(".venv/lib/python3.12/site-packages");
std::fs::create_dir_all(venv_site_packages.parent().unwrap())
.context("Failed to create .venv directory")?;
std::os::unix::fs::symlink(
@@ -1309,9 +1282,9 @@ mod unix {
Ok(())
},
|_root, workspace| SearchPathConfiguration {
|_root, project| SearchPathConfiguration {
site_packages: Some(SitePackages::Known(vec![
workspace.join(".venv/lib/python3.12/site-packages")
project.join(".venv/lib/python3.12/site-packages")
])),
..SearchPathConfiguration::default()
},
@@ -1323,7 +1296,7 @@ mod unix {
)
.expect("Expected bar.baz to exist in site-packages.");
let baz_site_packages_path =
case.workspace_path(".venv/lib/python3.12/site-packages/bar/baz.py");
case.project_path(".venv/lib/python3.12/site-packages/bar/baz.py");
let baz_site_packages = case.system_file(&baz_site_packages_path).unwrap();
let baz_original = case.root_path().join("site-packages/bar/baz.py");
let baz_original_file = case.system_file(&baz_original).unwrap();
@@ -1372,13 +1345,15 @@ mod unix {
}
#[test]
fn nested_packages_delete_root() -> anyhow::Result<()> {
let mut case = setup(|root: &SystemPath, workspace_root: &SystemPath| {
fn nested_projects_delete_root() -> anyhow::Result<()> {
let mut case = setup(|root: &SystemPath, project_root: &SystemPath| {
std::fs::write(
workspace_root.join("pyproject.toml").as_std_path(),
project_root.join("pyproject.toml").as_std_path(),
r#"
[project]
name = "inner"
[tool.knot]
"#,
)?;
@@ -1387,120 +1362,24 @@ fn nested_packages_delete_root() -> anyhow::Result<()> {
r#"
[project]
name = "outer"
[tool.knot]
"#,
)?;
Ok(())
})?;
assert_eq!(
case.db().workspace().root(case.db()),
&*case.workspace_path("")
);
assert_eq!(case.db().project().root(case.db()), &*case.project_path(""));
std::fs::remove_file(case.workspace_path("pyproject.toml").as_std_path())?;
std::fs::remove_file(case.project_path("pyproject.toml").as_std_path())?;
let changes = case.stop_watch(ChangeEvent::is_deleted);
case.apply_changes(changes);
// It should now pick up the outer workspace.
assert_eq!(case.db().workspace().root(case.db()), case.root_path());
Ok(())
}
#[test]
fn added_package() -> anyhow::Result<()> {
let mut case = setup([
(
"pyproject.toml",
r#"
[project]
name = "inner"
[tool.knot.workspace]
members = ["packages/*"]
"#,
),
(
"packages/a/pyproject.toml",
r#"
[project]
name = "a"
"#,
),
])?;
assert_eq!(case.db().workspace().packages(case.db()).len(), 2);
std::fs::create_dir(case.workspace_path("packages/b").as_std_path())
.context("failed to create folder for package 'b'")?;
// It seems that the file watcher won't pick up on file changes shortly after the folder
// was created... I suspect this is because most file watchers don't support recursive
// file watching. Instead, file-watching libraries manually implement recursive file watching
// by setting a watcher for each directory. But doing this obviously "lags" behind.
case.take_watch_changes();
std::fs::write(
case.workspace_path("packages/b/pyproject.toml")
.as_std_path(),
r#"
[project]
name = "b"
"#,
)
.context("failed to write pyproject.toml for package b")?;
let changes = case.stop_watch(event_for_file("pyproject.toml"));
case.apply_changes(changes);
assert_eq!(case.db().workspace().packages(case.db()).len(), 3);
Ok(())
}
#[test]
fn removed_package() -> anyhow::Result<()> {
let mut case = setup([
(
"pyproject.toml",
r#"
[project]
name = "inner"
[tool.knot.workspace]
members = ["packages/*"]
"#,
),
(
"packages/a/pyproject.toml",
r#"
[project]
name = "a"
"#,
),
(
"packages/b/pyproject.toml",
r#"
[project]
name = "b"
"#,
),
])?;
assert_eq!(case.db().workspace().packages(case.db()).len(), 3);
std::fs::remove_dir_all(case.workspace_path("packages/b").as_std_path())
.context("failed to remove package 'b'")?;
let changes = case.stop_watch(ChangeEvent::is_deleted);
case.apply_changes(changes);
assert_eq!(case.db().workspace().packages(case.db()).len(), 2);
// It should now pick up the outer project.
assert_eq!(case.db().project().root(case.db()), case.root_path());
Ok(())
}

View File

@@ -20,12 +20,14 @@ ruff_python_stdlib = { workspace = true }
ruff_source_file = { workspace = true }
ruff_text_size = { workspace = true }
ruff_python_literal = { workspace = true }
ruff_python_trivia = { workspace = true }
anyhow = { workspace = true }
bitflags = { workspace = true }
camino = { workspace = true }
compact_str = { workspace = true }
countme = { workspace = true }
drop_bomb = { workspace = true }
indexmap = { workspace = true }
itertools = { workspace = true }
ordermap = { workspace = true }
@@ -58,4 +60,3 @@ serde = ["ruff_db/serde", "dep:serde"]
[lints]
workspace = true

View File

@@ -9,8 +9,6 @@ from typing import Literal
from enum import Enum
mode: Literal["w", "r"]
mode2: Literal["w"] | Literal["r"]
union_var: Literal[Literal[Literal[1, 2, 3], "foo"], 5, None]
a1: Literal[26]
a2: Literal[0x1A]
a3: Literal[-4]
@@ -19,7 +17,6 @@ a5: Literal[b"hello world"]
a6: Literal[True]
a7: Literal[None]
a8: Literal[Literal[1]]
a9: Literal[Literal["w"], Literal["r"], Literal[Literal["w+"]]]
class Color(Enum):
RED = 0
@@ -30,9 +27,6 @@ b1: Literal[Color.RED]
def f():
reveal_type(mode) # revealed: Literal["w", "r"]
reveal_type(mode2) # revealed: Literal["w", "r"]
# TODO: should be revealed: Literal[1, 2, 3, "foo", 5] | None
reveal_type(union_var) # revealed: Literal[1, 2, 3, 5] | Literal["foo"] | None
reveal_type(a1) # revealed: Literal[26]
reveal_type(a2) # revealed: Literal[26]
reveal_type(a3) # revealed: Literal[-4]
@@ -41,7 +35,6 @@ def f():
reveal_type(a6) # revealed: Literal[True]
reveal_type(a7) # revealed: None
reveal_type(a8) # revealed: Literal[1]
reveal_type(a9) # revealed: Literal["w", "r", "w+"]
# TODO: This should be Color.RED
reveal_type(b1) # revealed: Literal[0]
@@ -61,6 +54,63 @@ invalid4: Literal[
]
```
## Shortening unions of literals
When a Literal is parameterized with more than one value, its treated as exactly to equivalent to
the union of those types.
```py
from typing import Literal
def x(
a1: Literal[Literal[Literal[1, 2, 3], "foo"], 5, None],
a2: Literal["w"] | Literal["r"],
a3: Literal[Literal["w"], Literal["r"], Literal[Literal["w+"]]],
a4: Literal[True] | Literal[1, 2] | Literal["foo"],
):
reveal_type(a1) # revealed: Literal[1, 2, 3, "foo", 5] | None
reveal_type(a2) # revealed: Literal["w", "r"]
reveal_type(a3) # revealed: Literal["w", "r", "w+"]
reveal_type(a4) # revealed: Literal[True, 1, 2, "foo"]
```
## Display of heterogeneous unions of literals
```py
from typing import Literal, Union
def foo(x: int) -> int:
return x + 1
def bar(s: str) -> str:
return s
class A: ...
class B: ...
def union_example(
x: Union[
# unknown type
# error: [unresolved-reference]
y,
Literal[-1],
Literal["A"],
Literal[b"A"],
Literal[b"\x00"],
Literal[b"\x07"],
Literal[0],
Literal[1],
Literal["B"],
Literal["foo"],
Literal["bar"],
Literal["B"],
Literal[True],
None,
]
):
reveal_type(x) # revealed: Unknown | Literal[-1, "A", b"A", b"\x00", b"\x07", 0, 1, "B", "foo", "bar", True] | None
```
## Detecting Literal outside typing and typing_extensions
Only Literal that is defined in typing and typing_extension modules is detected as the special

View File

@@ -73,12 +73,12 @@ qux = (foo, bar)
reveal_type(qux) # revealed: tuple[Literal["foo"], Literal["bar"]]
# TODO: Infer "LiteralString"
reveal_type(foo.join(qux)) # revealed: @Todo(call todo)
reveal_type(foo.join(qux)) # revealed: @Todo(Attribute access on `StringLiteral` types)
template: LiteralString = "{}, {}"
reveal_type(template) # revealed: Literal["{}, {}"]
# TODO: Infer `LiteralString`
reveal_type(template.format(foo, bar)) # revealed: @Todo(call todo)
reveal_type(template.format(foo, bar)) # revealed: @Todo(Attribute access on `StringLiteral` types)
```
### Assignability
@@ -107,7 +107,7 @@ def _(flag: bool):
qux_2: Literal["qux"] = baz_2 # error: [invalid-assignment]
baz_3 = "foo" if flag else 1
reveal_type(baz_3) # revealed: Literal["foo"] | Literal[1]
reveal_type(baz_3) # revealed: Literal["foo", 1]
qux_3: LiteralString = baz_3 # error: [invalid-assignment]
```

View File

@@ -47,7 +47,9 @@ def f():
## `typing.Never`
`typing.Never` is only available in Python 3.11 and later:
`typing.Never` is only available in Python 3.11 and later.
### Python 3.11
```toml
[environment]
@@ -57,8 +59,17 @@ python-version = "3.11"
```py
from typing import Never
x: Never
def f():
reveal_type(x) # revealed: Never
reveal_type(Never) # revealed: typing.Never
```
### Python 3.10
```toml
[environment]
python-version = "3.10"
```
```py
# error: [unresolved-import]
from typing import Never
```

View File

@@ -105,7 +105,7 @@ def f1(
from typing import Literal
def f(v: Literal["a", r"b", b"c", "d" "e", "\N{LATIN SMALL LETTER F}", "\x67", """h"""]):
reveal_type(v) # revealed: Literal["a", "b", "de", "f", "g", "h"] | Literal[b"c"]
reveal_type(v) # revealed: Literal["a", "b", b"c", "de", "f", "g", "h"]
```
## Class variables
@@ -173,3 +173,40 @@ p: "call()"
r: "[1, 2]"
s: "(1, 2)"
```
## Multi line annotation
Quoted type annotations should be parsed as if surrounded by parentheses.
```py
def valid(
a1: """(
int |
str
)
""",
a2: """
int |
str
""",
):
reveal_type(a1) # revealed: int | str
reveal_type(a2) # revealed: int | str
def invalid(
# error: [invalid-syntax-in-forward-annotation]
a1: """
int |
str)
""",
# error: [invalid-syntax-in-forward-annotation]
a2: """
int) |
str
""",
# error: [invalid-syntax-in-forward-annotation]
a3: """
(int)) """,
):
pass
```

View File

@@ -6,14 +6,11 @@ Several type qualifiers are unsupported by red-knot currently. However, we also
false-positive errors if you use one in an annotation:
```py
from typing_extensions import Final, ClassVar, Required, NotRequired, ReadOnly, TypedDict
from typing_extensions import Final, Required, NotRequired, ReadOnly, TypedDict
X: Final = 42
Y: Final[int] = 42
class Foo:
A: ClassVar[int] = 42
# TODO: `TypedDict` is actually valid as a base
# error: [invalid-base]
class Bar(TypedDict):

View File

@@ -33,8 +33,6 @@ b: tuple[int] = (42,)
c: tuple[str, int] = ("42", 42)
d: tuple[tuple[str, str], tuple[int, int]] = (("foo", "foo"), (42, 42))
e: tuple[str, ...] = ()
# TODO: we should not emit this error
# error: [call-possibly-unbound-method] "Method `__class_getitem__` of type `Literal[tuple]` is possibly unbound"
f: tuple[str, *tuple[int, ...], bytes] = ("42", b"42")
g: tuple[str, Unpack[tuple[int, ...]], bytes] = ("42", b"42")
h: tuple[list[int], list[int]] = ([], [])
@@ -124,3 +122,10 @@ class Foo: ...
x = Foo()
reveal_type(x) # revealed: Foo
```
## Annotated assignments in stub files are inferred correctly
```pyi path=main.pyi
x: int = 1
reveal_type(x) # revealed: Literal[1]
```

View File

@@ -40,9 +40,9 @@ class C:
return 42
x = C()
# error: [invalid-argument-type]
x -= 1
# TODO: should error, once operand type check is implemented
reveal_type(x) # revealed: int
```

View File

@@ -1,4 +1,273 @@
# Class attributes
# Attributes
Tests for attribute access on various kinds of types.
## Class and instance variables
### Pure instance variables
#### Variable only declared/bound in `__init__`
Variables only declared and/or bound in `__init__` are pure instance variables. They cannot be
accessed on the class itself.
```py
class C:
def __init__(self, value2: int, flag: bool = False) -> None:
# bound but not declared
self.pure_instance_variable1 = "value set in __init__"
# bound but not declared - with type inferred from parameter
self.pure_instance_variable2 = value2
# declared but not bound
self.pure_instance_variable3: bytes
# declared and bound
self.pure_instance_variable4: bool = True
# possibly undeclared/unbound
if flag:
self.pure_instance_variable5: str = "possibly set in __init__"
c_instance = C(1)
# TODO: should be `Literal["value set in __init__"]`, or `Unknown | Literal[…]` to allow
# assignments to this unannotated attribute from other scopes.
reveal_type(c_instance.pure_instance_variable1) # revealed: @Todo(instance attributes)
# TODO: should be `int`
reveal_type(c_instance.pure_instance_variable2) # revealed: @Todo(instance attributes)
# TODO: should be `bytes`
reveal_type(c_instance.pure_instance_variable3) # revealed: @Todo(instance attributes)
# TODO: should be `bool`
reveal_type(c_instance.pure_instance_variable4) # revealed: @Todo(instance attributes)
# TODO: should be `str`
# We probably don't want to emit a diagnostic for this being possibly undeclared/unbound.
# mypy and pyright do not show an error here.
reveal_type(c_instance.pure_instance_variable5) # revealed: @Todo(instance attributes)
# TODO: If we choose to infer a precise `Literal[…]` type for the instance attribute (see
# above), this should be an error: incompatible types in assignment. If we choose to infer
# a gradual `Unknown | Literal[…]` type, this assignment is fine.
c_instance.pure_instance_variable1 = "value set on instance"
# TODO: this should be an error (incompatible types in assignment)
c_instance.pure_instance_variable2 = "incompatible"
# TODO: we already show an error here but the message might be improved?
# mypy shows no error here, but pyright raises "reportAttributeAccessIssue"
# error: [unresolved-attribute] "Type `Literal[C]` has no attribute `pure_instance_variable1`"
reveal_type(C.pure_instance_variable1) # revealed: Unknown
# TODO: this should be an error (pure instance variables cannot be accessed on the class)
# mypy shows no error here, but pyright raises "reportAttributeAccessIssue"
C.pure_instance_variable1 = "overwritten on class"
c_instance.pure_instance_variable4 = False
# TODO: After this assignment to the attribute within this scope, we may eventually want to narrow
# the `bool` type (see above) for this instance variable to `Literal[False]` here. This is unsound
# in general (we don't know what else happened to `c_instance` between the assignment and the use
# here), but mypy and pyright support this. In conclusion, this could be `bool` but should probably
# be `Literal[False]`.
reveal_type(c_instance.pure_instance_variable4) # revealed: @Todo(instance attributes)
```
#### Variable declared in class body and declared/bound in `__init__`
The same rule applies even if the variable is *declared* (not bound!) in the class body: it is still
a pure instance variable.
```py
class C:
pure_instance_variable: str
def __init__(self) -> None:
self.pure_instance_variable = "value set in __init__"
c_instance = C()
# TODO: should be `str`
reveal_type(c_instance.pure_instance_variable) # revealed: @Todo(instance attributes)
# TODO: we currently plan to emit a diagnostic here. Note that both mypy
# and pyright show no error in this case! So we may reconsider this in
# the future, if it turns out to produce too many false positives.
reveal_type(C.pure_instance_variable) # revealed: str
# TODO: same as above. We plan to emit a diagnostic here, even if both mypy
# and pyright allow this.
C.pure_instance_variable = "overwritten on class"
# TODO: this should be an error (incompatible types in assignment)
c_instance.pure_instance_variable = 1
```
#### Variable only defined in unrelated method
We also recognize pure instance variables if they are defined in a method that is not `__init__`.
```py
class C:
def set_instance_variable(self) -> None:
self.pure_instance_variable = "value set in method"
c_instance = C()
# Not that we would use this in static analysis, but for a more realistic example, let's actually
# call the method, so that the attribute is bound if this example is actually run.
c_instance.set_instance_variable()
# TODO: should be `Literal["value set in method"]` or `Unknown | Literal[…]` (see above).
reveal_type(c_instance.pure_instance_variable) # revealed: @Todo(instance attributes)
# TODO: We already show an error here, but the message might be improved?
# error: [unresolved-attribute]
reveal_type(C.pure_instance_variable) # revealed: Unknown
# TODO: this should be an error
C.pure_instance_variable = "overwritten on class"
```
#### Variable declared in class body and not bound anywhere
If a variable is declared in the class body but not bound anywhere, we still consider it a pure
instance variable and allow access to it via instances.
```py
class C:
pure_instance_variable: str
c_instance = C()
# TODO: should be 'str'
reveal_type(c_instance.pure_instance_variable) # revealed: @Todo(instance attributes)
# TODO: mypy and pyright do not show an error here, but we plan to emit a diagnostic.
# The type could be changed to 'Unknown' if we decide to emit an error?
reveal_type(C.pure_instance_variable) # revealed: str
# TODO: mypy and pyright do not show an error here, but we plan to emit one.
C.pure_instance_variable = "overwritten on class"
```
### Pure class variables (`ClassVar`)
#### Annotated with `ClassVar` type qualifier
Class variables annotated with the [`typing.ClassVar`] type qualifier are pure class variables. They
cannot be overwritten on instances, but they can be accessed on instances.
For more details, see the [typing spec on `ClassVar`].
```py
from typing import ClassVar
class C:
pure_class_variable1: ClassVar[str] = "value in class body"
pure_class_variable2: ClassVar = 1
reveal_type(C.pure_class_variable1) # revealed: str
# TODO: this should be `Literal[1]`, or `Unknown | Literal[1]`.
reveal_type(C.pure_class_variable2) # revealed: @Todo(Unsupported or invalid type in a type expression)
c_instance = C()
# TODO: This should be `str`. It is okay to access a pure class variable on an instance.
reveal_type(c_instance.pure_class_variable1) # revealed: @Todo(instance attributes)
# TODO: should raise an error. It is not allowed to reassign a pure class variable on an instance.
c_instance.pure_class_variable1 = "value set on instance"
C.pure_class_variable1 = "overwritten on class"
# TODO: should raise an error (incompatible types in assignment)
C.pure_class_variable1 = 1
class Subclass(C):
pure_class_variable1: ClassVar[str] = "overwritten on subclass"
reveal_type(Subclass.pure_class_variable1) # revealed: str
```
#### Variable only mentioned in a class method
We also consider a class variable to be a pure class variable if it is only mentioned in a class
method.
```py
class C:
@classmethod
def class_method(cls):
cls.pure_class_variable = "value set in class method"
# for a more realistic example, let's actually call the method
C.class_method()
# TODO: We currently plan to support this and show no error here.
# mypy shows an error here, pyright does not.
# error: [unresolved-attribute]
reveal_type(C.pure_class_variable) # revealed: Unknown
C.pure_class_variable = "overwritten on class"
# TODO: should be `Literal["overwritten on class"]`
# error: [unresolved-attribute]
reveal_type(C.pure_class_variable) # revealed: Unknown
c_instance = C()
# TODO: should be `Literal["overwritten on class"]`
reveal_type(c_instance.pure_class_variable) # revealed: @Todo(instance attributes)
# TODO: should raise an error.
c_instance.pure_class_variable = "value set on instance"
```
### Instance variables with class-level default values
These are instance attributes, but the fact that we can see that they have a binding (not a
declaration) in the class body means that reading the value from the class directly is also
permitted. This is the only difference for these attributes as opposed to "pure" instance
attributes.
#### Basic
```py
class C:
variable_with_class_default: str = "value in class body"
def instance_method(self):
self.variable_with_class_default = "value set in instance method"
reveal_type(C.variable_with_class_default) # revealed: str
c_instance = C()
# TODO: should be `str`
reveal_type(c_instance.variable_with_class_default) # revealed: @Todo(instance attributes)
c_instance.variable_with_class_default = "value set on instance"
reveal_type(C.variable_with_class_default) # revealed: str
# TODO: Could be Literal["value set on instance"], or still `str` if we choose not to
# narrow the type.
reveal_type(c_instance.variable_with_class_default) # revealed: @Todo(instance attributes)
C.variable_with_class_default = "overwritten on class"
# TODO: Could be `Literal["overwritten on class"]`, or still `str` if we choose not to
# narrow the type.
reveal_type(C.variable_with_class_default) # revealed: str
# TODO: should still be `Literal["value set on instance"]`, or `str`.
reveal_type(c_instance.variable_with_class_default) # revealed: @Todo(instance attributes)
```
## Union of attributes
@@ -22,7 +291,9 @@ def _(flag: bool):
reveal_type(C2.x) # revealed: Literal[3, 4]
```
## Inherited attributes
## Inherited class attributes
### Basic
```py
class A:
@@ -34,7 +305,7 @@ class C(B): ...
reveal_type(C.X) # revealed: Literal["foo"]
```
## Inherited attributes (multiple inheritance)
### Multiple inheritance
```py
class O: ...
@@ -102,7 +373,7 @@ def _(flag: bool, flag1: bool, flag2: bool):
reveal_type(C.x) # revealed: Literal[1, 2, 3]
```
## Unions with all paths unbound
### Unions with all paths unbound
If the symbol is unbound in all elements of the union, we detect that:
@@ -155,3 +426,78 @@ class Foo: ...
reveal_type(Foo.__class__) # revealed: Literal[type]
```
## Literal types
### Function-literal attributes
Most attribute accesses on function-literal types are delegated to `types.FunctionType`, since all
functions are instances of that class:
```py path=a.py
def f(): ...
reveal_type(f.__defaults__) # revealed: @Todo(instance attributes)
reveal_type(f.__kwdefaults__) # revealed: @Todo(instance attributes)
```
Some attributes are special-cased, however:
```py path=b.py
def f(): ...
reveal_type(f.__get__) # revealed: @Todo(`__get__` method on functions)
reveal_type(f.__call__) # revealed: @Todo(`__call__` method on functions)
```
### Int-literal attributes
Most attribute accesses on int-literal types are delegated to `builtins.int`, since all literal
integers are instances of that class:
```py path=a.py
reveal_type((2).bit_length) # revealed: @Todo(instance attributes)
reveal_type((2).denominator) # revealed: @Todo(instance attributes)
```
Some attributes are special-cased, however:
```py path=b.py
reveal_type((2).numerator) # revealed: Literal[2]
reveal_type((2).real) # revealed: Literal[2]
```
### Bool-literal attributes
Most attribute accesses on bool-literal types are delegated to `builtins.bool`, since all literal
bols are instances of that class:
```py path=a.py
reveal_type(True.__and__) # revealed: @Todo(instance attributes)
reveal_type(False.__or__) # revealed: @Todo(instance attributes)
```
Some attributes are special-cased, however:
```py path=b.py
reveal_type(True.numerator) # revealed: Literal[1]
reveal_type(False.real) # revealed: Literal[0]
```
### Bytes-literal attributes
All attribute access on literal `bytes` types is currently delegated to `buitins.bytes`:
```py
reveal_type(b"foo".join) # revealed: @Todo(instance attributes)
reveal_type(b"foo".endswith) # revealed: @Todo(instance attributes)
```
## References
Some of the tests in the *Class and instance variables* section draw inspiration from
[pyright's documentation] on this topic.
[pyright's documentation]: https://microsoft.github.io/pyright/#/type-concepts-advanced?id=class-and-instance-variables
[typing spec on `classvar`]: https://typing.readthedocs.io/en/latest/spec/class-compat.html#classvar
[`typing.classvar`]: https://docs.python.org/3/library/typing.html#typing.ClassVar

View File

@@ -46,3 +46,50 @@ reveal_type(a | b) # revealed: Literal[True]
reveal_type(b | a) # revealed: Literal[True]
reveal_type(b | b) # revealed: Literal[False]
```
## Arithmetic with a variable
```py
a = True
b = False
def lhs_is_int(x: int):
reveal_type(x + a) # revealed: int
reveal_type(x - a) # revealed: int
reveal_type(x * a) # revealed: int
reveal_type(x // a) # revealed: int
reveal_type(x / a) # revealed: float
reveal_type(x % a) # revealed: int
def rhs_is_int(x: int):
reveal_type(a + x) # revealed: int
reveal_type(a - x) # revealed: int
reveal_type(a * x) # revealed: int
reveal_type(a // x) # revealed: int
reveal_type(a / x) # revealed: float
reveal_type(a % x) # revealed: int
def lhs_is_bool(x: bool):
reveal_type(x + a) # revealed: int
reveal_type(x - a) # revealed: int
reveal_type(x * a) # revealed: int
reveal_type(x // a) # revealed: int
reveal_type(x / a) # revealed: float
reveal_type(x % a) # revealed: int
def rhs_is_bool(x: bool):
reveal_type(a + x) # revealed: int
reveal_type(a - x) # revealed: int
reveal_type(a * x) # revealed: int
reveal_type(a // x) # revealed: int
reveal_type(a / x) # revealed: float
reveal_type(a % x) # revealed: int
def both_are_bool(x: bool, y: bool):
reveal_type(x + y) # revealed: int
reveal_type(x - y) # revealed: int
reveal_type(x * y) # revealed: int
reveal_type(x // y) # revealed: int
reveal_type(x / y) # revealed: float
reveal_type(x % y) # revealed: int
```

View File

@@ -0,0 +1,27 @@
# Binary operations on classes
## Union of two classes
Unioning two classes via the `|` operator is only available in Python 3.10 and later.
```toml
[environment]
python-version = "3.10"
```
```py
class A: ...
class B: ...
reveal_type(A | B) # revealed: UnionType
```
## Union of two classes (prior to 3.10)
```py
class A: ...
class B: ...
# error: "Operator `|` is unsupported between objects of type `Literal[A]` and `Literal[B]`"
reveal_type(A | B) # revealed: Unknown
```

View File

@@ -0,0 +1,371 @@
# Custom binary operations
## Class instances
```py
class Yes:
def __add__(self, other) -> Literal["+"]:
return "+"
def __sub__(self, other) -> Literal["-"]:
return "-"
def __mul__(self, other) -> Literal["*"]:
return "*"
def __matmul__(self, other) -> Literal["@"]:
return "@"
def __truediv__(self, other) -> Literal["/"]:
return "/"
def __mod__(self, other) -> Literal["%"]:
return "%"
def __pow__(self, other) -> Literal["**"]:
return "**"
def __lshift__(self, other) -> Literal["<<"]:
return "<<"
def __rshift__(self, other) -> Literal[">>"]:
return ">>"
def __or__(self, other) -> Literal["|"]:
return "|"
def __xor__(self, other) -> Literal["^"]:
return "^"
def __and__(self, other) -> Literal["&"]:
return "&"
def __floordiv__(self, other) -> Literal["//"]:
return "//"
class Sub(Yes): ...
class No: ...
# Yes implements all of the dunder methods.
reveal_type(Yes() + Yes()) # revealed: Literal["+"]
reveal_type(Yes() - Yes()) # revealed: Literal["-"]
reveal_type(Yes() * Yes()) # revealed: Literal["*"]
reveal_type(Yes() @ Yes()) # revealed: Literal["@"]
reveal_type(Yes() / Yes()) # revealed: Literal["/"]
reveal_type(Yes() % Yes()) # revealed: Literal["%"]
reveal_type(Yes() ** Yes()) # revealed: Literal["**"]
reveal_type(Yes() << Yes()) # revealed: Literal["<<"]
reveal_type(Yes() >> Yes()) # revealed: Literal[">>"]
reveal_type(Yes() | Yes()) # revealed: Literal["|"]
reveal_type(Yes() ^ Yes()) # revealed: Literal["^"]
reveal_type(Yes() & Yes()) # revealed: Literal["&"]
reveal_type(Yes() // Yes()) # revealed: Literal["//"]
# Sub inherits Yes's implementation of the dunder methods.
reveal_type(Sub() + Sub()) # revealed: Literal["+"]
reveal_type(Sub() - Sub()) # revealed: Literal["-"]
reveal_type(Sub() * Sub()) # revealed: Literal["*"]
reveal_type(Sub() @ Sub()) # revealed: Literal["@"]
reveal_type(Sub() / Sub()) # revealed: Literal["/"]
reveal_type(Sub() % Sub()) # revealed: Literal["%"]
reveal_type(Sub() ** Sub()) # revealed: Literal["**"]
reveal_type(Sub() << Sub()) # revealed: Literal["<<"]
reveal_type(Sub() >> Sub()) # revealed: Literal[">>"]
reveal_type(Sub() | Sub()) # revealed: Literal["|"]
reveal_type(Sub() ^ Sub()) # revealed: Literal["^"]
reveal_type(Sub() & Sub()) # revealed: Literal["&"]
reveal_type(Sub() // Sub()) # revealed: Literal["//"]
# No does not implement any of the dunder methods.
# error: [unsupported-operator] "Operator `+` is unsupported between objects of type `No` and `No`"
reveal_type(No() + No()) # revealed: Unknown
# error: [unsupported-operator] "Operator `-` is unsupported between objects of type `No` and `No`"
reveal_type(No() - No()) # revealed: Unknown
# error: [unsupported-operator] "Operator `*` is unsupported between objects of type `No` and `No`"
reveal_type(No() * No()) # revealed: Unknown
# error: [unsupported-operator] "Operator `@` is unsupported between objects of type `No` and `No`"
reveal_type(No() @ No()) # revealed: Unknown
# error: [unsupported-operator] "Operator `/` is unsupported between objects of type `No` and `No`"
reveal_type(No() / No()) # revealed: Unknown
# error: [unsupported-operator] "Operator `%` is unsupported between objects of type `No` and `No`"
reveal_type(No() % No()) # revealed: Unknown
# error: [unsupported-operator] "Operator `**` is unsupported between objects of type `No` and `No`"
reveal_type(No() ** No()) # revealed: Unknown
# error: [unsupported-operator] "Operator `<<` is unsupported between objects of type `No` and `No`"
reveal_type(No() << No()) # revealed: Unknown
# error: [unsupported-operator] "Operator `>>` is unsupported between objects of type `No` and `No`"
reveal_type(No() >> No()) # revealed: Unknown
# error: [unsupported-operator] "Operator `|` is unsupported between objects of type `No` and `No`"
reveal_type(No() | No()) # revealed: Unknown
# error: [unsupported-operator] "Operator `^` is unsupported between objects of type `No` and `No`"
reveal_type(No() ^ No()) # revealed: Unknown
# error: [unsupported-operator] "Operator `&` is unsupported between objects of type `No` and `No`"
reveal_type(No() & No()) # revealed: Unknown
# error: [unsupported-operator] "Operator `//` is unsupported between objects of type `No` and `No`"
reveal_type(No() // No()) # revealed: Unknown
# Yes does not implement any of the reflected dunder methods.
# error: [unsupported-operator] "Operator `+` is unsupported between objects of type `No` and `Yes`"
reveal_type(No() + Yes()) # revealed: Unknown
# error: [unsupported-operator] "Operator `-` is unsupported between objects of type `No` and `Yes`"
reveal_type(No() - Yes()) # revealed: Unknown
# error: [unsupported-operator] "Operator `*` is unsupported between objects of type `No` and `Yes`"
reveal_type(No() * Yes()) # revealed: Unknown
# error: [unsupported-operator] "Operator `@` is unsupported between objects of type `No` and `Yes`"
reveal_type(No() @ Yes()) # revealed: Unknown
# error: [unsupported-operator] "Operator `/` is unsupported between objects of type `No` and `Yes`"
reveal_type(No() / Yes()) # revealed: Unknown
# error: [unsupported-operator] "Operator `%` is unsupported between objects of type `No` and `Yes`"
reveal_type(No() % Yes()) # revealed: Unknown
# error: [unsupported-operator] "Operator `**` is unsupported between objects of type `No` and `Yes`"
reveal_type(No() ** Yes()) # revealed: Unknown
# error: [unsupported-operator] "Operator `<<` is unsupported between objects of type `No` and `Yes`"
reveal_type(No() << Yes()) # revealed: Unknown
# error: [unsupported-operator] "Operator `>>` is unsupported between objects of type `No` and `Yes`"
reveal_type(No() >> Yes()) # revealed: Unknown
# error: [unsupported-operator] "Operator `|` is unsupported between objects of type `No` and `Yes`"
reveal_type(No() | Yes()) # revealed: Unknown
# error: [unsupported-operator] "Operator `^` is unsupported between objects of type `No` and `Yes`"
reveal_type(No() ^ Yes()) # revealed: Unknown
# error: [unsupported-operator] "Operator `&` is unsupported between objects of type `No` and `Yes`"
reveal_type(No() & Yes()) # revealed: Unknown
# error: [unsupported-operator] "Operator `//` is unsupported between objects of type `No` and `Yes`"
reveal_type(No() // Yes()) # revealed: Unknown
```
## Subclass reflections override superclass dunders
```py
class Yes:
def __add__(self, other) -> Literal["+"]:
return "+"
def __sub__(self, other) -> Literal["-"]:
return "-"
def __mul__(self, other) -> Literal["*"]:
return "*"
def __matmul__(self, other) -> Literal["@"]:
return "@"
def __truediv__(self, other) -> Literal["/"]:
return "/"
def __mod__(self, other) -> Literal["%"]:
return "%"
def __pow__(self, other) -> Literal["**"]:
return "**"
def __lshift__(self, other) -> Literal["<<"]:
return "<<"
def __rshift__(self, other) -> Literal[">>"]:
return ">>"
def __or__(self, other) -> Literal["|"]:
return "|"
def __xor__(self, other) -> Literal["^"]:
return "^"
def __and__(self, other) -> Literal["&"]:
return "&"
def __floordiv__(self, other) -> Literal["//"]:
return "//"
class Sub(Yes):
def __radd__(self, other) -> Literal["r+"]:
return "r+"
def __rsub__(self, other) -> Literal["r-"]:
return "r-"
def __rmul__(self, other) -> Literal["r*"]:
return "r*"
def __rmatmul__(self, other) -> Literal["r@"]:
return "r@"
def __rtruediv__(self, other) -> Literal["r/"]:
return "r/"
def __rmod__(self, other) -> Literal["r%"]:
return "r%"
def __rpow__(self, other) -> Literal["r**"]:
return "r**"
def __rlshift__(self, other) -> Literal["r<<"]:
return "r<<"
def __rrshift__(self, other) -> Literal["r>>"]:
return "r>>"
def __ror__(self, other) -> Literal["r|"]:
return "r|"
def __rxor__(self, other) -> Literal["r^"]:
return "r^"
def __rand__(self, other) -> Literal["r&"]:
return "r&"
def __rfloordiv__(self, other) -> Literal["r//"]:
return "r//"
class No:
def __radd__(self, other) -> Literal["r+"]:
return "r+"
def __rsub__(self, other) -> Literal["r-"]:
return "r-"
def __rmul__(self, other) -> Literal["r*"]:
return "r*"
def __rmatmul__(self, other) -> Literal["r@"]:
return "r@"
def __rtruediv__(self, other) -> Literal["r/"]:
return "r/"
def __rmod__(self, other) -> Literal["r%"]:
return "r%"
def __rpow__(self, other) -> Literal["r**"]:
return "r**"
def __rlshift__(self, other) -> Literal["r<<"]:
return "r<<"
def __rrshift__(self, other) -> Literal["r>>"]:
return "r>>"
def __ror__(self, other) -> Literal["r|"]:
return "r|"
def __rxor__(self, other) -> Literal["r^"]:
return "r^"
def __rand__(self, other) -> Literal["r&"]:
return "r&"
def __rfloordiv__(self, other) -> Literal["r//"]:
return "r//"
# Subclass reflected dunder methods take precedence over the superclass's regular dunders.
reveal_type(Yes() + Sub()) # revealed: Literal["r+"]
reveal_type(Yes() - Sub()) # revealed: Literal["r-"]
reveal_type(Yes() * Sub()) # revealed: Literal["r*"]
reveal_type(Yes() @ Sub()) # revealed: Literal["r@"]
reveal_type(Yes() / Sub()) # revealed: Literal["r/"]
reveal_type(Yes() % Sub()) # revealed: Literal["r%"]
reveal_type(Yes() ** Sub()) # revealed: Literal["r**"]
reveal_type(Yes() << Sub()) # revealed: Literal["r<<"]
reveal_type(Yes() >> Sub()) # revealed: Literal["r>>"]
reveal_type(Yes() | Sub()) # revealed: Literal["r|"]
reveal_type(Yes() ^ Sub()) # revealed: Literal["r^"]
reveal_type(Yes() & Sub()) # revealed: Literal["r&"]
reveal_type(Yes() // Sub()) # revealed: Literal["r//"]
# But for an unrelated class, the superclass regular dunders are used.
reveal_type(Yes() + No()) # revealed: Literal["+"]
reveal_type(Yes() - No()) # revealed: Literal["-"]
reveal_type(Yes() * No()) # revealed: Literal["*"]
reveal_type(Yes() @ No()) # revealed: Literal["@"]
reveal_type(Yes() / No()) # revealed: Literal["/"]
reveal_type(Yes() % No()) # revealed: Literal["%"]
reveal_type(Yes() ** No()) # revealed: Literal["**"]
reveal_type(Yes() << No()) # revealed: Literal["<<"]
reveal_type(Yes() >> No()) # revealed: Literal[">>"]
reveal_type(Yes() | No()) # revealed: Literal["|"]
reveal_type(Yes() ^ No()) # revealed: Literal["^"]
reveal_type(Yes() & No()) # revealed: Literal["&"]
reveal_type(Yes() // No()) # revealed: Literal["//"]
```
## Classes
Dunder methods defined in a class are available to instances of that class, but not to the class
itself. (For these operators to work on the class itself, they would have to be defined on the
class's type, i.e. `type`.)
```py
class Yes:
def __add__(self, other) -> Literal["+"]:
return "+"
class Sub(Yes): ...
class No: ...
# error: [unsupported-operator] "Operator `+` is unsupported between objects of type `Literal[Yes]` and `Literal[Yes]`"
reveal_type(Yes + Yes) # revealed: Unknown
# error: [unsupported-operator] "Operator `+` is unsupported between objects of type `Literal[Sub]` and `Literal[Sub]`"
reveal_type(Sub + Sub) # revealed: Unknown
# error: [unsupported-operator] "Operator `+` is unsupported between objects of type `Literal[No]` and `Literal[No]`"
reveal_type(No + No) # revealed: Unknown
```
## Subclass
```py
class Yes:
def __add__(self, other) -> Literal["+"]:
return "+"
class Sub(Yes): ...
class No: ...
def yes() -> type[Yes]:
return Yes
def sub() -> type[Sub]:
return Sub
def no() -> type[No]:
return No
# error: [unsupported-operator] "Operator `+` is unsupported between objects of type `type[Yes]` and `type[Yes]`"
reveal_type(yes() + yes()) # revealed: Unknown
# error: [unsupported-operator] "Operator `+` is unsupported between objects of type `type[Sub]` and `type[Sub]`"
reveal_type(sub() + sub()) # revealed: Unknown
# error: [unsupported-operator] "Operator `+` is unsupported between objects of type `type[No]` and `type[No]`"
reveal_type(no() + no()) # revealed: Unknown
```
## Function literals
```py
def f():
pass
# error: [unsupported-operator] "Operator `+` is unsupported between objects of type `Literal[f]` and `Literal[f]`"
reveal_type(f + f) # revealed: Unknown
# error: [unsupported-operator] "Operator `-` is unsupported between objects of type `Literal[f]` and `Literal[f]`"
reveal_type(f - f) # revealed: Unknown
# error: [unsupported-operator] "Operator `*` is unsupported between objects of type `Literal[f]` and `Literal[f]`"
reveal_type(f * f) # revealed: Unknown
# error: [unsupported-operator] "Operator `@` is unsupported between objects of type `Literal[f]` and `Literal[f]`"
reveal_type(f @ f) # revealed: Unknown
# error: [unsupported-operator] "Operator `/` is unsupported between objects of type `Literal[f]` and `Literal[f]`"
reveal_type(f / f) # revealed: Unknown
# error: [unsupported-operator] "Operator `%` is unsupported between objects of type `Literal[f]` and `Literal[f]`"
reveal_type(f % f) # revealed: Unknown
# error: [unsupported-operator] "Operator `**` is unsupported between objects of type `Literal[f]` and `Literal[f]`"
reveal_type(f**f) # revealed: Unknown
# error: [unsupported-operator] "Operator `<<` is unsupported between objects of type `Literal[f]` and `Literal[f]`"
reveal_type(f << f) # revealed: Unknown
# error: [unsupported-operator] "Operator `>>` is unsupported between objects of type `Literal[f]` and `Literal[f]`"
reveal_type(f >> f) # revealed: Unknown
# error: [unsupported-operator] "Operator `|` is unsupported between objects of type `Literal[f]` and `Literal[f]`"
reveal_type(f | f) # revealed: Unknown
# error: [unsupported-operator] "Operator `^` is unsupported between objects of type `Literal[f]` and `Literal[f]`"
reveal_type(f ^ f) # revealed: Unknown
# error: [unsupported-operator] "Operator `&` is unsupported between objects of type `Literal[f]` and `Literal[f]`"
reveal_type(f & f) # revealed: Unknown
# error: [unsupported-operator] "Operator `//` is unsupported between objects of type `Literal[f]` and `Literal[f]`"
reveal_type(f // f) # revealed: Unknown
```

View File

@@ -9,6 +9,34 @@ reveal_type(3 * -1) # revealed: Literal[-3]
reveal_type(-3 // 3) # revealed: Literal[-1]
reveal_type(-3 / 3) # revealed: float
reveal_type(5 % 3) # revealed: Literal[2]
# TODO: We don't currently verify that the actual parameter to int.__add__ matches the declared
# formal parameter type.
reveal_type(2 + "f") # revealed: int
def lhs(x: int):
reveal_type(x + 1) # revealed: int
reveal_type(x - 4) # revealed: int
reveal_type(x * -1) # revealed: int
reveal_type(x // 3) # revealed: int
reveal_type(x / 3) # revealed: float
reveal_type(x % 3) # revealed: int
def rhs(x: int):
reveal_type(2 + x) # revealed: int
reveal_type(3 - x) # revealed: int
reveal_type(3 * x) # revealed: int
reveal_type(-3 // x) # revealed: int
reveal_type(-3 / x) # revealed: float
reveal_type(5 % x) # revealed: int
def both(x: int):
reveal_type(x + x) # revealed: int
reveal_type(x - x) # revealed: int
reveal_type(x * x) # revealed: int
reveal_type(x // x) # revealed: int
reveal_type(x / x) # revealed: float
reveal_type(x % x) # revealed: int
```
## Power
@@ -21,6 +49,11 @@ largest_u32 = 4_294_967_295
reveal_type(2**2) # revealed: Literal[4]
reveal_type(1 ** (largest_u32 + 1)) # revealed: int
reveal_type(2**largest_u32) # revealed: int
def variable(x: int):
reveal_type(x**2) # revealed: @Todo(return type)
reveal_type(2**x) # revealed: @Todo(return type)
reveal_type(x**x) # revealed: @Todo(return type)
```
## Division by Zero

View File

@@ -32,13 +32,10 @@ def _(flag: bool):
```py
if True or (x := 1):
# TODO: infer that the second arm is never executed, and raise `unresolved-reference`.
# error: [possibly-unresolved-reference]
reveal_type(x) # revealed: Literal[1]
# error: [unresolved-reference]
reveal_type(x) # revealed: Unknown
if True and (x := 1):
# TODO: infer that the second arm is always executed, do not raise a diagnostic
# error: [possibly-unresolved-reference]
reveal_type(x) # revealed: Literal[1]
```

View File

@@ -0,0 +1,209 @@
# Boundness and declaredness: public uses
This document demonstrates how type-inference and diagnostics works for *public* uses of a symbol,
that is, a use of a symbol from another scope. If a symbol has a declared type in its local scope
(e.g. `int`), we use that as the symbol's "public type" (the type of the symbol from the perspective
of other scopes) even if there is a more precise local inferred type for the symbol (`Literal[1]`).
We test the whole matrix of possible boundness and declaredness states. The current behavior is
summarized in the following table, while the tests below demonstrate each case. Note that some of
this behavior is questionable and might change in the future. See the TODOs in `symbol_by_id`
(`types.rs`) and [this issue](https://github.com/astral-sh/ruff/issues/14297) for more information.
In particular, we should raise errors in the "possibly-undeclared-and-unbound" as well as the
"undeclared-and-possibly-unbound" cases (marked with a "?").
| **Public type** | declared | possibly-undeclared | undeclared |
| ---------------- | ------------ | -------------------------- | ------------ |
| bound | `T_declared` | `T_declared \| T_inferred` | `T_inferred` |
| possibly-unbound | `T_declared` | `T_declared \| T_inferred` | `T_inferred` |
| unbound | `T_declared` | `T_declared` | `Unknown` |
| **Diagnostic** | declared | possibly-undeclared | undeclared |
| ---------------- | -------- | ------------------------- | ------------------- |
| bound | | | |
| possibly-unbound | | `possibly-unbound-import` | ? |
| unbound | | ? | `unresolved-import` |
## Declared
### Declared and bound
If a symbol has a declared type (`int`), we use that even if there is a more precise inferred type
(`Literal[1]`), or a conflicting inferred type (`Literal[2]`):
```py path=mod.py
x: int = 1
# error: [invalid-assignment]
y: str = 2
```
```py
from mod import x, y
reveal_type(x) # revealed: int
reveal_type(y) # revealed: str
```
### Declared and possibly unbound
If a symbol is declared and *possibly* unbound, we trust that other module and use the declared type
without raising an error.
```py path=mod.py
def flag() -> bool: ...
x: int
y: str
if flag:
x = 1
# error: [invalid-assignment]
y = 2
```
```py
from mod import x, y
reveal_type(x) # revealed: int
reveal_type(y) # revealed: str
```
### Declared and unbound
Similarly, if a symbol is declared but unbound, we do not raise an error. We trust that this symbol
is available somehow and simply use the declared type.
```py path=mod.py
x: int
```
```py
from mod import x
reveal_type(x) # revealed: int
```
## Possibly undeclared
### Possibly undeclared and bound
If a symbol is possibly undeclared but definitely bound, we use the union of the declared and
inferred types:
```py path=mod.py
from typing import Any
def flag() -> bool: ...
x = 1
y = 2
if flag():
x: Any
# error: [invalid-declaration]
y: str
```
```py
from mod import x, y
reveal_type(x) # revealed: Literal[1] | Any
reveal_type(y) # revealed: Literal[2] | Unknown
```
### Possibly undeclared and possibly unbound
If a symbol is possibly undeclared and possibly unbound, we also use the union of the declared and
inferred types. This case is interesting because the "possibly declared" definition might not be the
same as the "possibly bound" definition (symbol `y`). Note that we raise a `possibly-unbound-import`
error for both `x` and `y`:
```py path=mod.py
def flag() -> bool: ...
if flag():
x: Any = 1
y = 2
else:
y: str
```
```py
# error: [possibly-unbound-import]
# error: [possibly-unbound-import]
from mod import x, y
reveal_type(x) # revealed: Literal[1] | Any
reveal_type(y) # revealed: Literal[2] | str
```
### Possibly undeclared and unbound
If a symbol is possibly undeclared and definitely unbound, we currently do not raise an error. This
seems inconsistent when compared to the case just above.
```py path=mod.py
def flag() -> bool: ...
if flag():
x: int
```
```py
# TODO: this should raise an error. Once we fix this, update the section description and the table
# on top of this document.
from mod import x
reveal_type(x) # revealed: int
```
## Undeclared
### Undeclared but bound
We use the inferred type as the public type, if a symbol has no declared type.
```py path=mod.py
x = 1
```
```py
from mod import x
reveal_type(x) # revealed: Literal[1]
```
### Undeclared and possibly unbound
If a symbol is undeclared and *possibly* unbound, we currently do not raise an error. This seems
inconsistent when compared to the "possibly-undeclared-and-possibly-unbound" case.
```py path=mod.py
def flag() -> bool: ...
if flag:
x = 1
```
```py
# TODO: this should raise an error. Once we fix this, update the section description and the table
# on top of this document.
from mod import x
reveal_type(x) # revealed: Literal[1]
```
### Undeclared and unbound
If a symbol is undeclared *and* unbound, we infer `Unknown` and raise an error.
```py path=mod.py
if False:
x: int = 1
```
```py
# error: [unresolved-import]
from mod import x
reveal_type(x) # revealed: Unknown
```

View File

@@ -67,6 +67,35 @@ def _(flag: bool):
def __call__(self) -> int: ...
a = NonCallable()
# error: "Object of type `Literal[__call__] | Literal[1]` is not callable (due to union element `Literal[1]`)"
reveal_type(a()) # revealed: int | Unknown
# error: "Object of type `Literal[1] | Literal[__call__]` is not callable (due to union element `Literal[1]`)"
reveal_type(a()) # revealed: Unknown | int
```
## Call binding errors
### Wrong argument type
```py
class C:
def __call__(self, x: int) -> int:
return 1
c = C()
# error: 15 [invalid-argument-type] "Object of type `Literal["foo"]` cannot be assigned to parameter 2 (`x`) of function `__call__`; expected type `int`"
reveal_type(c("foo")) # revealed: int
```
### Wrong argument type on `self`
```py
class C:
# TODO this definition should also be an error; `C` must be assignable to type of `self`
def __call__(self: int) -> int:
return 1
c = C()
# error: 13 [invalid-argument-type] "Object of type `C` cannot be assigned to parameter 1 (`self`) of function `__call__`; expected type `int`"
reveal_type(c()) # revealed: int
```

View File

@@ -64,3 +64,269 @@ def _(flag: bool):
# error: [possibly-unresolved-reference]
reveal_type(foo()) # revealed: int
```
## Wrong argument type
### Positional argument, positional-or-keyword parameter
```py
def f(x: int) -> int:
return 1
# error: 15 [invalid-argument-type] "Object of type `Literal["foo"]` cannot be assigned to parameter 1 (`x`) of function `f`; expected type `int`"
reveal_type(f("foo")) # revealed: int
```
### Positional argument, positional-only parameter
```py
def f(x: int, /) -> int:
return 1
# error: 15 [invalid-argument-type] "Object of type `Literal["foo"]` cannot be assigned to parameter 1 (`x`) of function `f`; expected type `int`"
reveal_type(f("foo")) # revealed: int
```
### Positional argument, variadic parameter
```py
def f(*args: int) -> int:
return 1
# error: 15 [invalid-argument-type] "Object of type `Literal["foo"]` cannot be assigned to parameter `*args` of function `f`; expected type `int`"
reveal_type(f("foo")) # revealed: int
```
### Keyword argument, positional-or-keyword parameter
```py
def f(x: int) -> int:
return 1
# error: 15 [invalid-argument-type] "Object of type `Literal["foo"]` cannot be assigned to parameter `x` of function `f`; expected type `int`"
reveal_type(f(x="foo")) # revealed: int
```
### Keyword argument, keyword-only parameter
```py
def f(*, x: int) -> int:
return 1
# error: 15 [invalid-argument-type] "Object of type `Literal["foo"]` cannot be assigned to parameter `x` of function `f`; expected type `int`"
reveal_type(f(x="foo")) # revealed: int
```
### Keyword argument, keywords parameter
```py
def f(**kwargs: int) -> int:
return 1
# error: 15 [invalid-argument-type] "Object of type `Literal["foo"]` cannot be assigned to parameter `**kwargs` of function `f`; expected type `int`"
reveal_type(f(x="foo")) # revealed: int
```
### Correctly match keyword out-of-order
```py
def f(x: int = 1, y: str = "foo") -> int:
return 1
# error: 15 [invalid-argument-type] "Object of type `Literal[2]` cannot be assigned to parameter `y` of function `f`; expected type `str`"
# error: 20 [invalid-argument-type] "Object of type `Literal["bar"]` cannot be assigned to parameter `x` of function `f`; expected type `int`"
reveal_type(f(y=2, x="bar")) # revealed: int
```
## Too many positional arguments
### One too many
```py
def f() -> int:
return 1
# error: 15 [too-many-positional-arguments] "Too many positional arguments to function `f`: expected 0, got 1"
reveal_type(f("foo")) # revealed: int
```
### Two too many
```py
def f() -> int:
return 1
# error: 15 [too-many-positional-arguments] "Too many positional arguments to function `f`: expected 0, got 2"
reveal_type(f("foo", "bar")) # revealed: int
```
### No too-many-positional if variadic is taken
```py
def f(*args: int) -> int:
return 1
reveal_type(f(1, 2, 3)) # revealed: int
```
### Multiple keyword arguments map to keyword variadic parameter
```py
def f(**kwargs: int) -> int:
return 1
reveal_type(f(foo=1, bar=2)) # revealed: int
```
## Missing arguments
### No defaults or variadic
```py
def f(x: int) -> int:
return 1
# error: 13 [missing-argument] "No argument provided for required parameter `x` of function `f`"
reveal_type(f()) # revealed: int
```
### With default
```py
def f(x: int, y: str = "foo") -> int:
return 1
# error: 13 [missing-argument] "No argument provided for required parameter `x` of function `f`"
reveal_type(f()) # revealed: int
```
### Defaulted argument is not required
```py
def f(x: int = 1) -> int:
return 1
reveal_type(f()) # revealed: int
```
### With variadic
```py
def f(x: int, *y: str) -> int:
return 1
# error: 13 [missing-argument] "No argument provided for required parameter `x` of function `f`"
reveal_type(f()) # revealed: int
```
### Variadic argument is not required
```py
def f(*args: int) -> int:
return 1
reveal_type(f()) # revealed: int
```
### Keywords argument is not required
```py
def f(**kwargs: int) -> int:
return 1
reveal_type(f()) # revealed: int
```
### Multiple
```py
def f(x: int, y: int) -> int:
return 1
# error: 13 [missing-argument] "No arguments provided for required parameters `x`, `y` of function `f`"
reveal_type(f()) # revealed: int
```
## Unknown argument
```py
def f(x: int) -> int:
return 1
# error: 20 [unknown-argument] "Argument `y` does not match any known parameter of function `f`"
reveal_type(f(x=1, y=2)) # revealed: int
```
## Parameter already assigned
```py
def f(x: int) -> int:
return 1
# error: 18 [parameter-already-assigned] "Multiple values provided for parameter `x` of function `f`"
reveal_type(f(1, x=2)) # revealed: int
```
## Special functions
Some functions require special handling in type inference. Here, we make sure that we still emit
proper diagnostics in case of missing or superfluous arguments.
### `reveal_type`
```py
from typing_extensions import reveal_type
# error: [missing-argument] "No argument provided for required parameter `obj` of function `reveal_type`"
reveal_type() # revealed: Unknown
# error: [too-many-positional-arguments] "Too many positional arguments to function `reveal_type`: expected 1, got 2"
reveal_type(1, 2) # revealed: Literal[1]
```
### `static_assert`
```py
from knot_extensions import static_assert
# error: [missing-argument] "No argument provided for required parameter `condition` of function `static_assert`"
# error: [static-assert-error]
static_assert()
# error: [too-many-positional-arguments] "Too many positional arguments to function `static_assert`: expected 2, got 3"
static_assert(True, 2, 3)
```
### `len`
```py
# error: [missing-argument] "No argument provided for required parameter `obj` of function `len`"
len()
# error: [too-many-positional-arguments] "Too many positional arguments to function `len`: expected 1, got 2"
len([], 1)
```
### Type API predicates
```py
from knot_extensions import is_subtype_of, is_fully_static
# error: [missing-argument]
is_subtype_of()
# error: [missing-argument]
is_subtype_of(int)
# error: [too-many-positional-arguments]
is_subtype_of(int, int, int)
# error: [too-many-positional-arguments]
is_subtype_of(int, int, int, int)
# error: [missing-argument]
is_fully_static()
# error: [too-many-positional-arguments]
is_fully_static(int, int)
```

View File

@@ -0,0 +1,44 @@
# Invalid signatures
## Multiple arguments with the same name
We always map a keyword argument to the first parameter of that name.
```py
# error: [invalid-syntax] "Duplicate parameter "x""
def f(x: int, x: str) -> int:
return 1
# error: 13 [missing-argument] "No argument provided for required parameter `x` of function `f`"
# error: 18 [parameter-already-assigned] "Multiple values provided for parameter `x` of function `f`"
reveal_type(f(1, x=2)) # revealed: int
```
## Positional after non-positional
When parameter kinds are given in an invalid order, we emit a diagnostic and implicitly reorder them
to the valid order:
```py
# error: [invalid-syntax] "Parameter cannot follow var-keyword parameter"
def f(**kw: int, x: str) -> int:
return 1
# error: 15 [invalid-argument-type] "Object of type `Literal[1]` cannot be assigned to parameter 1 (`x`) of function `f`; expected type `str`"
reveal_type(f(1)) # revealed: int
```
## Non-defaulted after defaulted
We emit a syntax diagnostic for this, but it doesn't cause any problems for binding.
```py
# error: [invalid-syntax] "Parameter without a default cannot follow a parameter with a default"
def f(x: int = 1, y: str) -> int:
return 1
reveal_type(f(y="foo")) # revealed: int
# error: [invalid-argument-type] "Object of type `Literal["foo"]` cannot be assigned to parameter 1 (`x`) of function `f`; expected type `int`"
# error: [missing-argument] "No argument provided for required parameter `y` of function `f`"
reveal_type(f("foo")) # revealed: int
```

View File

@@ -56,7 +56,7 @@ def _(flag: bool, flag2: bool):
else:
def f() -> int:
return 1
# error: "Object of type `Literal[1] | Literal["foo"] | Literal[f]` is not callable (due to union elements Literal[1], Literal["foo"])"
# error: "Object of type `Literal[1, "foo"] | Literal[f]` is not callable (due to union elements Literal[1], Literal["foo"])"
# revealed: Unknown | int
reveal_type(f())
```
@@ -72,6 +72,6 @@ def _(flag: bool):
else:
f = "foo"
x = f() # error: "Object of type `Literal[1] | Literal["foo"]` is not callable"
x = f() # error: "Object of type `Literal[1, "foo"]` is not callable"
reveal_type(x) # revealed: Unknown
```

View File

@@ -92,8 +92,7 @@ def _(o: object):
n = None
if o is not None:
reveal_type(o) # revealed: object & ~None
reveal_type(o) # revealed: ~None
reveal_type(o is n) # revealed: Literal[False]
reveal_type(o is not n) # revealed: Literal[True]
```

View File

@@ -31,10 +31,10 @@ class C:
def __lt__(self, other) -> C: ...
x = A() < B() < C()
reveal_type(x) # revealed: A | B
reveal_type(x) # revealed: A & ~AlwaysTruthy | B
y = 0 < 1 < A() < 3
reveal_type(y) # revealed: bool | A
reveal_type(y) # revealed: Literal[False] | A
z = 10 < 0 < A() < B() < C()
reveal_type(z) # revealed: Literal[False]

View File

@@ -22,7 +22,7 @@ def _(flag: bool, flag1: bool, flag2: bool):
reveal_type(d) # revealed: bool
int_literal_or_str_literal = 1 if flag else "foo"
# error: "Operator `in` is not supported for types `Literal[42]` and `Literal[1]`, in comparing `Literal[42]` with `Literal[1] | Literal["foo"]`"
# error: "Operator `in` is not supported for types `Literal[42]` and `Literal[1]`, in comparing `Literal[42]` with `Literal[1, "foo"]`"
e = 42 in int_literal_or_str_literal
reveal_type(e) # revealed: bool

View File

@@ -115,3 +115,35 @@ def _(flag: bool, flag2: bool):
reveal_type(y) # revealed: Literal[2, 3, 4]
```
## if-elif with assignment expressions in tests
```py
def check(x: int) -> bool:
return bool(x)
if check(x := 1):
x = 2
elif check(x := 3):
x = 4
reveal_type(x) # revealed: Literal[2, 3, 4]
```
## constraints apply to later test expressions
```py
def check(x) -> bool:
return bool(x)
def _(flag: bool):
x = 1 if flag else None
y = 0
if x is None:
pass
elif check(y := x):
pass
reveal_type(y) # revealed: Literal[0, 1]
```

View File

@@ -0,0 +1,142 @@
# `assert_type`
## Basic
```py
from typing_extensions import assert_type
def _(x: int):
assert_type(x, int) # fine
assert_type(x, str) # error: [type-assertion-failure]
```
## Narrowing
The asserted type is checked against the inferred type, not the declared type.
```toml
[environment]
python-version = "3.10"
```
```py
from typing_extensions import assert_type
def _(x: int | str):
if isinstance(x, int):
reveal_type(x) # revealed: int
assert_type(x, int) # fine
```
## Equivalence
The actual type must match the asserted type precisely.
```py
from typing import Any, Type, Union
from typing_extensions import assert_type
# Subtype does not count
def _(x: bool):
assert_type(x, int) # error: [type-assertion-failure]
def _(a: type[int], b: type[Any]):
assert_type(a, type[Any]) # error: [type-assertion-failure]
assert_type(b, type[int]) # error: [type-assertion-failure]
# The expression constructing the type is not taken into account
def _(a: type[int]):
assert_type(a, Type[int]) # fine
```
## Gradual types
```py
from typing import Any
from typing_extensions import Literal, assert_type
from knot_extensions import Unknown
# Any and Unknown are considered equivalent
def _(a: Unknown, b: Any):
reveal_type(a) # revealed: Unknown
assert_type(a, Any) # fine
reveal_type(b) # revealed: Any
assert_type(b, Unknown) # fine
def _(a: type[Unknown], b: type[Any]):
reveal_type(a) # revealed: type[Unknown]
assert_type(a, type[Any]) # fine
reveal_type(b) # revealed: type[Any]
assert_type(b, type[Unknown]) # fine
```
## Tuples
Tuple types with the same elements are the same.
```py
from typing_extensions import assert_type
from knot_extensions import Unknown
def _(a: tuple[int, str, bytes]):
assert_type(a, tuple[int, str, bytes]) # fine
assert_type(a, tuple[int, str]) # error: [type-assertion-failure]
assert_type(a, tuple[int, str, bytes, None]) # error: [type-assertion-failure]
assert_type(a, tuple[int, bytes, str]) # error: [type-assertion-failure]
def _(a: tuple[Any, ...], b: tuple[Unknown, ...]):
assert_type(a, tuple[Any, ...]) # fine
assert_type(a, tuple[Unknown, ...]) # fine
assert_type(b, tuple[Unknown, ...]) # fine
assert_type(b, tuple[Any, ...]) # fine
```
## Unions
Unions with the same elements are the same, regardless of order.
```toml
[environment]
python-version = "3.10"
```
```py
from typing_extensions import assert_type
def _(a: str | int):
assert_type(a, str | int) # fine
# TODO: Order-independent union handling in type equivalence
assert_type(a, int | str) # error: [type-assertion-failure]
```
## Intersections
Intersections are the same when their positive and negative parts are respectively the same,
regardless of order.
```py
from typing_extensions import assert_type
from knot_extensions import Intersection, Not
class A: ...
class B: ...
class C: ...
class D: ...
def _(a: A):
if isinstance(a, B) and not isinstance(a, C) and not isinstance(a, D):
reveal_type(a) # revealed: A & B & ~C & ~D
assert_type(a, Intersection[A, B, Not[C], Not[D]]) # fine
# TODO: Order-independent intersection handling in type equivalence
assert_type(a, Intersection[B, A, Not[D], Not[C]]) # error: [type-assertion-failure]
```

View File

@@ -0,0 +1,27 @@
# `cast`
`cast()` takes two arguments, one type and one value, and returns a value of the given type.
The (inferred) type of the value and the given type do not need to have any correlation.
```py
from typing import Literal, cast
reveal_type(True) # revealed: Literal[True]
reveal_type(cast(str, True)) # revealed: str
reveal_type(cast("str", True)) # revealed: str
reveal_type(cast(int | str, 1)) # revealed: int | str
# error: [invalid-type-form]
reveal_type(cast(Literal, True)) # revealed: Unknown
# TODO: These should be errors
cast(1)
cast(str)
cast(str, b"ar", "foo")
# TODO: Either support keyword arguments properly,
# or give a comprehensible error message saying they're unsupported
cast(val="foo", typ=int) # error: [unresolved-reference] "Name `foo` used when not defined"
```

View File

@@ -90,3 +90,83 @@ def foo(
# TODO: should emit a diagnostic here:
reveal_type(g) # revealed: @Todo(full tuple[...] support)
```
## Object raised is not an exception
```py
try:
raise AttributeError() # fine
except:
...
try:
raise FloatingPointError # fine
except:
...
try:
raise 1 # error: [invalid-raise]
except:
...
try:
raise int # error: [invalid-raise]
except:
...
def _(e: Exception | type[Exception]):
raise e # fine
def _(e: Exception | type[Exception] | None):
raise e # error: [invalid-raise]
```
## Exception cause is not an exception
```py
try:
raise EOFError() from GeneratorExit # fine
except:
...
try:
raise StopIteration from MemoryError() # fine
except:
...
try:
raise BufferError() from None # fine
except:
...
try:
raise ZeroDivisionError from False # error: [invalid-raise]
except:
...
try:
raise SystemExit from bool() # error: [invalid-raise]
except:
...
try:
raise
except KeyboardInterrupt as e: # fine
reveal_type(e) # revealed: KeyboardInterrupt
raise LookupError from e # fine
try:
raise
except int as e: # error: [invalid-exception-caught]
reveal_type(e) # revealed: Unknown
raise KeyError from e
def _(e: Exception | type[Exception]):
raise ModuleNotFoundError from e # fine
def _(e: Exception | type[Exception] | None):
raise IndexError from e # fine
def _(e: int | None):
raise IndexError from e # error: [invalid-raise]
```

View File

@@ -17,7 +17,7 @@ def _(flag: bool):
reveal_type(A.always_bound) # revealed: Literal[1]
reveal_type(A.union) # revealed: Literal[1] | Literal["abc"]
reveal_type(A.union) # revealed: Literal[1, "abc"]
# error: [possibly-unbound-attribute] "Attribute `possibly_unbound` on type `Literal[A]` is possibly unbound"
reveal_type(A.possibly_unbound) # revealed: Literal["abc"]

View File

@@ -10,8 +10,8 @@ def _(foo: str):
reveal_type(False or "z") # revealed: Literal["z"]
reveal_type(False or True) # revealed: Literal[True]
reveal_type(False or False) # revealed: Literal[False]
reveal_type(foo or False) # revealed: str | Literal[False]
reveal_type(foo or True) # revealed: str | Literal[True]
reveal_type(foo or False) # revealed: str & ~AlwaysFalsy | Literal[False]
reveal_type(foo or True) # revealed: str & ~AlwaysFalsy | Literal[True]
```
## AND
@@ -20,8 +20,8 @@ def _(foo: str):
def _(foo: str):
reveal_type(True and False) # revealed: Literal[False]
reveal_type(False and True) # revealed: Literal[False]
reveal_type(foo and False) # revealed: str | Literal[False]
reveal_type(foo and True) # revealed: str | Literal[True]
reveal_type(foo and False) # revealed: str & ~AlwaysTruthy | Literal[False]
reveal_type(foo and True) # revealed: str & ~AlwaysTruthy | Literal[True]
reveal_type("x" and "y" and "z") # revealed: Literal["z"]
reveal_type("x" and "y" and "") # revealed: Literal[""]
reveal_type("" and "y") # revealed: Literal[""]

View File

@@ -7,7 +7,7 @@ def _(flag: bool):
reveal_type(1 if flag else 2) # revealed: Literal[1, 2]
```
## Statically known branches
## Statically known conditions in if-expressions
```py
reveal_type(1 if True else 2) # revealed: Literal[1]
@@ -31,9 +31,9 @@ The test inside an if expression should not affect code outside of the expressio
def _(flag: bool):
x: Literal[42, "hello"] = 42 if flag else "hello"
reveal_type(x) # revealed: Literal[42] | Literal["hello"]
reveal_type(x) # revealed: Literal[42, "hello"]
_ = ... if isinstance(x, str) else ...
reveal_type(x) # revealed: Literal[42] | Literal["hello"]
reveal_type(x) # revealed: Literal[42, "hello"]
```

View File

@@ -119,7 +119,7 @@ class ZeroOrStr:
reveal_type(len(Zero())) # revealed: Literal[0]
reveal_type(len(ZeroOrOne())) # revealed: Literal[0, 1]
reveal_type(len(ZeroOrTrue())) # revealed: Literal[0, 1]
reveal_type(len(OneOrFalse())) # revealed: Literal[0, 1]
reveal_type(len(OneOrFalse())) # revealed: Literal[1, 0]
# TODO: Emit a diagnostic
reveal_type(len(OneOrFoo())) # revealed: int

View File

@@ -0,0 +1,31 @@
# Tests for the `@typing(_extensions).final` decorator
## Cannot subclass
Don't do this:
```py
import typing_extensions
from typing import final
@final
class A: ...
class B(A): ... # error: 9 [subclass-of-final-class] "Class `B` cannot inherit from final class `A`"
@typing_extensions.final
class C: ...
class D(C): ... # error: [subclass-of-final-class]
class E: ...
class F: ...
class G: ...
# fmt: off
class H(
E,
F,
A, # error: [subclass-of-final-class]
G,
): ...
```

View File

@@ -0,0 +1,75 @@
# Conflicting attributes and submodules
## Via import
```py
import a.b
reveal_type(a.b) # revealed: <module 'a.b'>
```
```py path=a/__init__.py
b = 42
```
```py path=a/b.py
```
## Via from/import
```py
from a import b
reveal_type(b) # revealed: Literal[42]
```
```py path=a/__init__.py
b = 42
```
```py path=a/b.py
```
## Via both
```py
import a.b
from a import b
reveal_type(b) # revealed: <module 'a.b'>
reveal_type(a.b) # revealed: <module 'a.b'>
```
```py path=a/__init__.py
b = 42
```
```py path=a/b.py
```
## Via both (backwards)
In this test, we infer a different type for `b` than the runtime behavior of the Python interpreter.
The interpreter will not load the submodule `a.b` during the `from a import b` statement, since `a`
contains a non-module attribute named `b`. (See the [definition][from-import] of a `from...import`
statement for details.) However, because our import tracking is flow-insensitive, we will see that
`a.b` is imported somewhere in the file, and therefore assume that the `from...import` statement
sees the submodule as the value of `b` instead of the integer.
```py
from a import b
import a.b
# Python would say `Literal[42]` for `b`
reveal_type(b) # revealed: <module 'a.b'>
reveal_type(a.b) # revealed: <module 'a.b'>
```
```py path=a/__init__.py
b = 42
```
```py path=a/b.py
```
[from-import]: https://docs.python.org/3/reference/simple_stmts.html#the-import-statement

View File

@@ -0,0 +1,848 @@
# Intersection types
## Introduction
This test suite covers certain properties of intersection types and makes sure that we can apply
various simplification strategies. We use `Intersection` (`&`) and `Not` (`~`) to construct
intersection types (note that we display negative contributions at the end; the order does not
matter):
```py
from knot_extensions import Intersection, Not
class P: ...
class Q: ...
def _(
i1: Intersection[P, Q],
i2: Intersection[P, Not[Q]],
i3: Intersection[Not[P], Q],
i4: Intersection[Not[P], Not[Q]],
) -> None:
reveal_type(i1) # revealed: P & Q
reveal_type(i2) # revealed: P & ~Q
reveal_type(i3) # revealed: Q & ~P
reveal_type(i4) # revealed: ~P & ~Q
```
## Notation
Throughout this document, we use the following types as representatives for certain equivalence
classes.
### Non-disjoint types
We use `P`, `Q`, `R`, … to denote types that are non-disjoint:
```py
from knot_extensions import static_assert, is_disjoint_from
class P: ...
class Q: ...
class R: ...
static_assert(not is_disjoint_from(P, Q))
static_assert(not is_disjoint_from(P, R))
static_assert(not is_disjoint_from(Q, R))
```
Although `P` is not a subtype of `Q` and `Q` is not a subtype of `P`, the two types are not disjoint
because it would be possible to create a class `S` that inherits from both `P` and `Q` using
multiple inheritance. An instance of `S` would be a member of the `P` type _and_ the `Q` type.
### Disjoint types
We use `Literal[1]`, `Literal[2]`, … as examples of pairwise-disjoint types, and `int` as a joint
supertype of these:
```py
from knot_extensions import static_assert, is_disjoint_from, is_subtype_of
from typing import Literal
static_assert(is_disjoint_from(Literal[1], Literal[2]))
static_assert(is_disjoint_from(Literal[1], Literal[3]))
static_assert(is_disjoint_from(Literal[2], Literal[3]))
static_assert(is_subtype_of(Literal[1], int))
static_assert(is_subtype_of(Literal[2], int))
static_assert(is_subtype_of(Literal[3], int))
```
### Subtypes
Finally, we use `A <: B <: C` and `A <: B1`, `A <: B2` to denote hierarchies of (proper) subtypes:
```py
from knot_extensions import static_assert, is_subtype_of, is_disjoint_from
class A: ...
class B(A): ...
class C(B): ...
static_assert(is_subtype_of(B, A))
static_assert(is_subtype_of(C, B))
static_assert(is_subtype_of(C, A))
static_assert(not is_subtype_of(A, B))
static_assert(not is_subtype_of(B, C))
static_assert(not is_subtype_of(A, C))
class B1(A): ...
class B2(A): ...
static_assert(is_subtype_of(B1, A))
static_assert(is_subtype_of(B2, A))
static_assert(not is_subtype_of(A, B1))
static_assert(not is_subtype_of(A, B2))
static_assert(not is_subtype_of(B1, B2))
static_assert(not is_subtype_of(B2, B1))
```
## Structural properties
This section covers structural properties of intersection types and documents some decisions on how
to represent mixtures of intersections and unions.
### Single-element intersections
If we have an intersection with a single element, we can simplify to that element. Similarly, we
show an intersection with a single negative contribution as just the negation of that element.
```py
from knot_extensions import Intersection, Not
class P: ...
def _(
i1: Intersection[P],
i2: Intersection[Not[P]],
) -> None:
reveal_type(i1) # revealed: P
reveal_type(i2) # revealed: ~P
```
### Flattening of nested intersections
We eagerly flatten nested intersections types.
```py
from knot_extensions import Intersection, Not
class P: ...
class Q: ...
class R: ...
class S: ...
def positive_contributions(
i1: Intersection[P, Intersection[Q, R]],
i2: Intersection[Intersection[P, Q], R],
) -> None:
reveal_type(i1) # revealed: P & Q & R
reveal_type(i2) # revealed: P & Q & R
def negative_contributions(
i1: Intersection[Not[P], Intersection[Not[Q], Not[R]]],
i2: Intersection[Intersection[Not[P], Not[Q]], Not[R]],
) -> None:
reveal_type(i1) # revealed: ~P & ~Q & ~R
reveal_type(i2) # revealed: ~P & ~Q & ~R
def mixed(
i1: Intersection[P, Intersection[Not[Q], R]],
i2: Intersection[Intersection[P, Not[Q]], R],
i3: Intersection[Not[P], Intersection[Q, Not[R]]],
i4: Intersection[Intersection[Q, Not[R]], Not[P]],
) -> None:
reveal_type(i1) # revealed: P & R & ~Q
reveal_type(i2) # revealed: P & R & ~Q
reveal_type(i3) # revealed: Q & ~P & ~R
reveal_type(i4) # revealed: Q & ~R & ~P
def multiple(
i1: Intersection[Intersection[P, Q], Intersection[R, S]],
):
reveal_type(i1) # revealed: P & Q & R & S
def nested(
i1: Intersection[Intersection[Intersection[P, Q], R], S],
i2: Intersection[P, Intersection[Q, Intersection[R, S]]],
):
reveal_type(i1) # revealed: P & Q & R & S
reveal_type(i2) # revealed: P & Q & R & S
```
### Union of intersections
We always normalize our representation to a _union of intersections_, so when we add a _union to an
intersection_, we distribute the union over the respective elements:
```py
from knot_extensions import Intersection, Not
class P: ...
class Q: ...
class R: ...
class S: ...
def _(
i1: Intersection[P, Q | R | S],
i2: Intersection[P | Q | R, S],
i3: Intersection[P | Q, R | S],
) -> None:
reveal_type(i1) # revealed: P & Q | P & R | P & S
reveal_type(i2) # revealed: P & S | Q & S | R & S
reveal_type(i3) # revealed: P & R | Q & R | P & S | Q & S
def simplifications_for_same_elements(
i1: Intersection[P, Q | P],
i2: Intersection[Q, P | Q],
i3: Intersection[P | Q, Q | R],
i4: Intersection[P | Q, P | Q],
i5: Intersection[P | Q, Q | P],
) -> None:
# P & (Q | P)
# = P & Q | P & P
# = P & Q | P
# = P
# (because P is a supertype of P & Q)
reveal_type(i1) # revealed: P
# similar here:
reveal_type(i2) # revealed: Q
# (P | Q) & (Q | R)
# = P & Q | P & R | Q & Q | Q & R
# = P & Q | P & R | Q | Q & R
# = Q | P & R
# (again, because Q is a supertype of P & Q and of Q & R)
reveal_type(i3) # revealed: Q | P & R
# (P | Q) & (P | Q)
# = P & P | P & Q | Q & P | Q & Q
# = P | P & Q | Q
# = P | Q
reveal_type(i4) # revealed: P | Q
```
### Negation distributes over union
Distribution also applies to a negation operation. This is a manifestation of one of
[De Morgan's laws], namely `~(P | Q) = ~P & ~Q`:
```py
from knot_extensions import Not
from typing import Literal
class P: ...
class Q: ...
class R: ...
def _(i1: Not[P | Q], i2: Not[P | Q | R]) -> None:
reveal_type(i1) # revealed: ~P & ~Q
reveal_type(i2) # revealed: ~P & ~Q & ~R
def example_literals(i: Not[Literal[1, 2]]) -> None:
reveal_type(i) # revealed: ~Literal[1] & ~Literal[2]
```
### Negation of intersections
The other of [De Morgan's laws], `~(P & Q) = ~P | ~Q`, also holds:
```py
from knot_extensions import Intersection, Not
class P: ...
class Q: ...
class R: ...
def _(
i1: Not[Intersection[P, Q]],
i2: Not[Intersection[P, Q, R]],
) -> None:
reveal_type(i1) # revealed: ~P | ~Q
reveal_type(i2) # revealed: ~P | ~Q | ~R
```
### `Never` is dual to `object`
`Never` represents the empty set of values, while `object` represents the set of all values, so
`~Never` is equivalent to `object`, and `~object` is equivalent to `Never`. This is a manifestation
of the [complement laws] of set theory.
```py
from knot_extensions import Intersection, Not
from typing_extensions import Never
def _(
not_never: Not[Never],
not_object: Not[object],
) -> None:
reveal_type(not_never) # revealed: object
reveal_type(not_object) # revealed: Never
```
### `object & ~T` is equivalent to `~T`
A second consequence of the fact that `object` is the top type is that `object` is always redundant
in intersections, and can be eagerly simplified out. `object & P` is equivalent to `P`;
`object & ~P` is equivalent to `~P` for any type `P`.
```py
from knot_extensions import Intersection, Not, is_equivalent_to, static_assert
class P: ...
static_assert(is_equivalent_to(Intersection[object, P], P))
static_assert(is_equivalent_to(Intersection[object, Not[P]], Not[P]))
```
### Intersection of a type and its negation
Continuing with more [complement laws], if we see both `P` and `~P` in an intersection, we can
simplify to `Never`, even in the presence of other types:
```py
from knot_extensions import Intersection, Not
from typing import Any
class P: ...
class Q: ...
def _(
i1: Intersection[P, Not[P]],
i2: Intersection[Not[P], P],
i3: Intersection[P, Q, Not[P]],
i4: Intersection[Not[P], Q, P],
i5: Intersection[P, Any, Not[P]],
i6: Intersection[Not[P], Any, P],
) -> None:
reveal_type(i1) # revealed: Never
reveal_type(i2) # revealed: Never
reveal_type(i3) # revealed: Never
reveal_type(i4) # revealed: Never
reveal_type(i5) # revealed: Never
reveal_type(i6) # revealed: Never
```
### Union of a type and its negation
Similarly, if we have both `P` and `~P` in a _union_, we can simplify that to `object`.
```py
from knot_extensions import Intersection, Not
class P: ...
class Q: ...
def _(
i1: P | Not[P],
i2: Not[P] | P,
i3: P | Q | Not[P],
i4: Not[P] | Q | P,
) -> None:
reveal_type(i1) # revealed: object
reveal_type(i2) # revealed: object
reveal_type(i3) # revealed: object
reveal_type(i4) # revealed: object
```
### Negation is an involution
The final of the [complement laws] states that negating twice is equivalent to not negating at all:
```py
from knot_extensions import Not
class P: ...
def _(
i1: Not[P],
i2: Not[Not[P]],
i3: Not[Not[Not[P]]],
i4: Not[Not[Not[Not[P]]]],
) -> None:
reveal_type(i1) # revealed: ~P
reveal_type(i2) # revealed: P
reveal_type(i3) # revealed: ~P
reveal_type(i4) # revealed: P
```
## Simplification strategies
In this section, we present various simplification strategies that go beyond the structure of the
representation.
### `Never` in intersections
If we intersect with `Never`, we can simplify the whole intersection to `Never`, even if there are
dynamic types involved:
```py
from knot_extensions import Intersection, Not
from typing_extensions import Never, Any
class P: ...
class Q: ...
def _(
i1: Intersection[P, Never],
i2: Intersection[Never, P],
i3: Intersection[Any, Never],
i4: Intersection[Never, Not[Any]],
) -> None:
reveal_type(i1) # revealed: Never
reveal_type(i2) # revealed: Never
reveal_type(i3) # revealed: Never
reveal_type(i4) # revealed: Never
```
### Simplifications using disjointness
#### Positive contributions
If we intersect disjoint types, we can simplify to `Never`, even in the presence of other types:
```py
from knot_extensions import Intersection, Not
from typing import Literal, Any
class P: ...
def _(
i01: Intersection[Literal[1], Literal[2]],
i02: Intersection[Literal[2], Literal[1]],
i03: Intersection[Literal[1], Literal[2], P],
i04: Intersection[Literal[1], P, Literal[2]],
i05: Intersection[P, Literal[1], Literal[2]],
i06: Intersection[Literal[1], Literal[2], Any],
i07: Intersection[Literal[1], Any, Literal[2]],
i08: Intersection[Any, Literal[1], Literal[2]],
) -> None:
reveal_type(i01) # revealed: Never
reveal_type(i02) # revealed: Never
reveal_type(i03) # revealed: Never
reveal_type(i04) # revealed: Never
reveal_type(i05) # revealed: Never
reveal_type(i06) # revealed: Never
reveal_type(i07) # revealed: Never
reveal_type(i08) # revealed: Never
# `bool` is final and can not be subclassed, so `type[bool]` is equivalent to `Literal[bool]`, which
# is disjoint from `type[str]`:
def example_type_bool_type_str(
i: Intersection[type[bool], type[str]],
) -> None:
reveal_type(i) # revealed: Never
```
#### Positive and negative contributions
If we intersect a type `X` with the negation `~Y` of a disjoint type `Y`, we can remove the negative
contribution `~Y`, as `~Y` must fully contain the positive contribution `X` as a subtype:
```py
from knot_extensions import Intersection, Not
from typing import Literal
def _(
i1: Intersection[Literal[1], Not[Literal[2]]],
i2: Intersection[Not[Literal[2]], Literal[1]],
i3: Intersection[Literal[1], Not[Literal[2]], int],
i4: Intersection[Literal[1], int, Not[Literal[2]]],
i5: Intersection[int, Literal[1], Not[Literal[2]]],
) -> None:
reveal_type(i1) # revealed: Literal[1]
reveal_type(i2) # revealed: Literal[1]
reveal_type(i3) # revealed: Literal[1]
reveal_type(i4) # revealed: Literal[1]
reveal_type(i5) # revealed: Literal[1]
# None is disjoint from int, so this simplification applies here
def example_none(
i1: Intersection[int, Not[None]],
i2: Intersection[Not[None], int],
) -> None:
reveal_type(i1) # revealed: int
reveal_type(i2) # revealed: int
```
### Simplifications using subtype relationships
#### Positive type and positive subtype
Subtypes are contained within their supertypes, so we can simplify intersections by removing
superfluous supertypes:
```py
from knot_extensions import Intersection, Not
from typing import Any
class A: ...
class B(A): ...
class C(B): ...
class Unrelated: ...
def _(
i01: Intersection[A, B],
i02: Intersection[B, A],
i03: Intersection[A, C],
i04: Intersection[C, A],
i05: Intersection[B, C],
i06: Intersection[C, B],
i07: Intersection[A, B, C],
i08: Intersection[C, B, A],
i09: Intersection[B, C, A],
i10: Intersection[A, B, Unrelated],
i11: Intersection[B, A, Unrelated],
i12: Intersection[B, Unrelated, A],
i13: Intersection[A, Unrelated, B],
i14: Intersection[Unrelated, A, B],
i15: Intersection[Unrelated, B, A],
i16: Intersection[A, B, Any],
i17: Intersection[B, A, Any],
i18: Intersection[B, Any, A],
i19: Intersection[A, Any, B],
i20: Intersection[Any, A, B],
i21: Intersection[Any, B, A],
) -> None:
reveal_type(i01) # revealed: B
reveal_type(i02) # revealed: B
reveal_type(i03) # revealed: C
reveal_type(i04) # revealed: C
reveal_type(i05) # revealed: C
reveal_type(i06) # revealed: C
reveal_type(i07) # revealed: C
reveal_type(i08) # revealed: C
reveal_type(i09) # revealed: C
reveal_type(i10) # revealed: B & Unrelated
reveal_type(i11) # revealed: B & Unrelated
reveal_type(i12) # revealed: B & Unrelated
reveal_type(i13) # revealed: Unrelated & B
reveal_type(i14) # revealed: Unrelated & B
reveal_type(i15) # revealed: Unrelated & B
reveal_type(i16) # revealed: B & Any
reveal_type(i17) # revealed: B & Any
reveal_type(i18) # revealed: B & Any
reveal_type(i19) # revealed: Any & B
reveal_type(i20) # revealed: Any & B
reveal_type(i21) # revealed: Any & B
```
#### Negative type and negative subtype
For negative contributions, this property is reversed. Here we can remove superfluous _subtypes_:
```py
from knot_extensions import Intersection, Not
from typing import Any
class A: ...
class B(A): ...
class C(B): ...
class Unrelated: ...
def _(
i01: Intersection[Not[B], Not[A]],
i02: Intersection[Not[A], Not[B]],
i03: Intersection[Not[A], Not[C]],
i04: Intersection[Not[C], Not[A]],
i05: Intersection[Not[B], Not[C]],
i06: Intersection[Not[C], Not[B]],
i07: Intersection[Not[A], Not[B], Not[C]],
i08: Intersection[Not[C], Not[B], Not[A]],
i09: Intersection[Not[B], Not[C], Not[A]],
i10: Intersection[Not[B], Not[A], Unrelated],
i11: Intersection[Not[A], Not[B], Unrelated],
i12: Intersection[Not[A], Unrelated, Not[B]],
i13: Intersection[Not[B], Unrelated, Not[A]],
i14: Intersection[Unrelated, Not[A], Not[B]],
i15: Intersection[Unrelated, Not[B], Not[A]],
i16: Intersection[Not[B], Not[A], Any],
i17: Intersection[Not[A], Not[B], Any],
i18: Intersection[Not[A], Any, Not[B]],
i19: Intersection[Not[B], Any, Not[A]],
i20: Intersection[Any, Not[A], Not[B]],
i21: Intersection[Any, Not[B], Not[A]],
) -> None:
reveal_type(i01) # revealed: ~A
reveal_type(i02) # revealed: ~A
reveal_type(i03) # revealed: ~A
reveal_type(i04) # revealed: ~A
reveal_type(i05) # revealed: ~B
reveal_type(i06) # revealed: ~B
reveal_type(i07) # revealed: ~A
reveal_type(i08) # revealed: ~A
reveal_type(i09) # revealed: ~A
reveal_type(i10) # revealed: Unrelated & ~A
reveal_type(i11) # revealed: Unrelated & ~A
reveal_type(i12) # revealed: Unrelated & ~A
reveal_type(i13) # revealed: Unrelated & ~A
reveal_type(i14) # revealed: Unrelated & ~A
reveal_type(i15) # revealed: Unrelated & ~A
reveal_type(i16) # revealed: Any & ~A
reveal_type(i17) # revealed: Any & ~A
reveal_type(i18) # revealed: Any & ~A
reveal_type(i19) # revealed: Any & ~A
reveal_type(i20) # revealed: Any & ~A
reveal_type(i21) # revealed: Any & ~A
```
#### Negative type and multiple negative subtypes
If there are multiple negative subtypes, all of them can be removed:
```py
from knot_extensions import Intersection, Not
class A: ...
class B1(A): ...
class B2(A): ...
def _(
i1: Intersection[Not[A], Not[B1], Not[B2]],
i2: Intersection[Not[A], Not[B2], Not[B1]],
i3: Intersection[Not[B1], Not[A], Not[B2]],
i4: Intersection[Not[B1], Not[B2], Not[A]],
i5: Intersection[Not[B2], Not[A], Not[B1]],
i6: Intersection[Not[B2], Not[B1], Not[A]],
) -> None:
reveal_type(i1) # revealed: ~A
reveal_type(i2) # revealed: ~A
reveal_type(i3) # revealed: ~A
reveal_type(i4) # revealed: ~A
reveal_type(i5) # revealed: ~A
reveal_type(i6) # revealed: ~A
```
#### Negative type and positive subtype
When `A` is a supertype of `B`, its negation `~A` is disjoint from `B`, so we can simplify the
intersection to `Never`:
```py
from knot_extensions import Intersection, Not
from typing import Any
class A: ...
class B(A): ...
class C(B): ...
class Unrelated: ...
def _(
i1: Intersection[Not[A], B],
i2: Intersection[B, Not[A]],
i3: Intersection[Not[A], C],
i4: Intersection[C, Not[A]],
i5: Intersection[Unrelated, Not[A], B],
i6: Intersection[B, Not[A], Not[Unrelated]],
i7: Intersection[Any, Not[A], B],
i8: Intersection[B, Not[A], Not[Any]],
) -> None:
reveal_type(i1) # revealed: Never
reveal_type(i2) # revealed: Never
reveal_type(i3) # revealed: Never
reveal_type(i4) # revealed: Never
reveal_type(i5) # revealed: Never
reveal_type(i6) # revealed: Never
reveal_type(i7) # revealed: Never
reveal_type(i8) # revealed: Never
```
### Simplifications of `bool`, `AlwaysTruthy` and `AlwaysFalsy`
In general, intersections with `AlwaysTruthy` and `AlwaysFalsy` cannot be simplified. Naively, you
might think that `int & AlwaysFalsy` could simplify to `Literal[0]`, but this is not the case: for
example, the `False` constant inhabits the type `int & AlwaysFalsy` (due to the fact that
`False.__class__` is `bool` at runtime, and `bool` subclasses `int`), but `False` does not inhabit
the type `Literal[0]`.
Nonetheless, intersections of `AlwaysFalsy` or `AlwaysTruthy` with `bool` _can_ be simplified, due
to the fact that `bool` is a `@final` class at runtime that cannot be subclassed.
```py
from knot_extensions import Intersection, Not, AlwaysTruthy, AlwaysFalsy
class P: ...
def f(
a: Intersection[bool, AlwaysTruthy],
b: Intersection[bool, AlwaysFalsy],
c: Intersection[bool, Not[AlwaysTruthy]],
d: Intersection[bool, Not[AlwaysFalsy]],
e: Intersection[bool, AlwaysTruthy, P],
f: Intersection[bool, AlwaysFalsy, P],
g: Intersection[bool, Not[AlwaysTruthy], P],
h: Intersection[bool, Not[AlwaysFalsy], P],
):
reveal_type(a) # revealed: Literal[True]
reveal_type(b) # revealed: Literal[False]
reveal_type(c) # revealed: Literal[False]
reveal_type(d) # revealed: Literal[True]
# `bool & AlwaysTruthy & P` -> `Literal[True] & P` -> `Never`
reveal_type(e) # revealed: Never
reveal_type(f) # revealed: Never
reveal_type(g) # revealed: Never
reveal_type(h) # revealed: Never
```
## Simplification of `LiteralString`, `AlwaysTruthy` and `AlwaysFalsy`
Similarly, intersections between `LiteralString`, `AlwaysTruthy` and `AlwaysFalsy` can be
simplified, due to the fact that a `LiteralString` inhabitant is known to have `__class__` set to
exactly `str` (and not a subclass of `str`):
```py
from knot_extensions import Intersection, Not, AlwaysTruthy, AlwaysFalsy, Unknown
from typing_extensions import LiteralString
def f(
a: Intersection[LiteralString, AlwaysTruthy],
b: Intersection[LiteralString, AlwaysFalsy],
c: Intersection[LiteralString, Not[AlwaysTruthy]],
d: Intersection[LiteralString, Not[AlwaysFalsy]],
e: Intersection[AlwaysFalsy, LiteralString],
f: Intersection[Not[AlwaysTruthy], LiteralString],
g: Intersection[AlwaysTruthy, LiteralString],
h: Intersection[Not[AlwaysFalsy], LiteralString],
i: Intersection[Unknown, LiteralString, AlwaysFalsy],
j: Intersection[Not[AlwaysTruthy], Unknown, LiteralString],
):
reveal_type(a) # revealed: LiteralString & ~Literal[""]
reveal_type(b) # revealed: Literal[""]
reveal_type(c) # revealed: Literal[""]
reveal_type(d) # revealed: LiteralString & ~Literal[""]
reveal_type(e) # revealed: Literal[""]
reveal_type(f) # revealed: Literal[""]
reveal_type(g) # revealed: LiteralString & ~Literal[""]
reveal_type(h) # revealed: LiteralString & ~Literal[""]
reveal_type(i) # revealed: Unknown & Literal[""]
reveal_type(j) # revealed: Unknown & Literal[""]
```
## Addition of a type to an intersection with many non-disjoint types
This slightly strange-looking test is a regression test for a mistake that was nearly made in a PR:
<https://github.com/astral-sh/ruff/pull/15475#discussion_r1915041987>.
```py
from knot_extensions import AlwaysFalsy, Intersection, Unknown
from typing_extensions import Literal
def _(x: Intersection[str, Unknown, AlwaysFalsy, Literal[""]]):
reveal_type(x) # revealed: Unknown & Literal[""]
```
## Non fully-static types
### Negation of dynamic types
`Any` represents the dynamic type, an unknown set of runtime values. The negation of that, `~Any`,
is still an unknown set of runtime values, so `~Any` is equivalent to `Any`. We therefore eagerly
simplify `~Any` to `Any` in intersections. The same applies to `Unknown`.
```py
from knot_extensions import Intersection, Not, Unknown
from typing_extensions import Any, Never
class P: ...
def any(
i1: Not[Any],
i2: Intersection[P, Not[Any]],
i3: Intersection[Never, Not[Any]],
) -> None:
reveal_type(i1) # revealed: Any
reveal_type(i2) # revealed: P & Any
reveal_type(i3) # revealed: Never
def unknown(
i1: Not[Unknown],
i2: Intersection[P, Not[Unknown]],
i3: Intersection[Never, Not[Unknown]],
) -> None:
reveal_type(i1) # revealed: Unknown
reveal_type(i2) # revealed: P & Unknown
reveal_type(i3) # revealed: Never
```
### Collapsing of multiple `Any`/`Unknown` contributions
The intersection of an unknown set of runtime values with (another) unknown set of runtime values is
still an unknown set of runtime values:
```py
from knot_extensions import Intersection, Not, Unknown
from typing_extensions import Any
class P: ...
def any(
i1: Intersection[Any, Any],
i2: Intersection[P, Any, Any],
i3: Intersection[Any, P, Any],
i4: Intersection[Any, Any, P],
) -> None:
reveal_type(i1) # revealed: Any
reveal_type(i2) # revealed: P & Any
reveal_type(i3) # revealed: Any & P
reveal_type(i4) # revealed: Any & P
def unknown(
i1: Intersection[Unknown, Unknown],
i2: Intersection[P, Unknown, Unknown],
i3: Intersection[Unknown, P, Unknown],
i4: Intersection[Unknown, Unknown, P],
) -> None:
reveal_type(i1) # revealed: Unknown
reveal_type(i2) # revealed: P & Unknown
reveal_type(i3) # revealed: Unknown & P
reveal_type(i4) # revealed: Unknown & P
```
### No self-cancellation
Dynamic types do not cancel each other out. Intersecting an unknown set of values with the negation
of another unknown set of values is not necessarily empty, so we keep the positive contribution:
```py
from knot_extensions import Intersection, Not, Unknown
def any(
i1: Intersection[Any, Not[Any]],
i2: Intersection[Not[Any], Any],
) -> None:
reveal_type(i1) # revealed: Any
reveal_type(i2) # revealed: Any
def unknown(
i1: Intersection[Unknown, Not[Unknown]],
i2: Intersection[Not[Unknown], Unknown],
) -> None:
reveal_type(i1) # revealed: Unknown
reveal_type(i2) # revealed: Unknown
```
### Mixed dynamic types
We currently do not simplify mixed dynamic types, but might consider doing so in the future:
```py
from knot_extensions import Intersection, Not, Unknown
def mixed(
i1: Intersection[Any, Unknown],
i2: Intersection[Any, Not[Unknown]],
i3: Intersection[Not[Any], Unknown],
i4: Intersection[Not[Any], Not[Unknown]],
) -> None:
reveal_type(i1) # revealed: Any & Unknown
reveal_type(i2) # revealed: Any & Unknown
reveal_type(i3) # revealed: Any & Unknown
reveal_type(i4) # revealed: Any & Unknown
```
[complement laws]: https://en.wikipedia.org/wiki/Complement_(set_theory)
[de morgan's laws]: https://en.wikipedia.org/wiki/De_Morgan%27s_laws

View File

@@ -1,7 +1,23 @@
# Ellipsis literals
## Simple
## Python 3.9
```toml
[environment]
python-version = "3.9"
```
```py
reveal_type(...) # revealed: EllipsisType | ellipsis
reveal_type(...) # revealed: ellipsis
```
## Python 3.10
```toml
[environment]
python-version = "3.10"
```
```py
reveal_type(...) # revealed: EllipsisType
```

View File

@@ -98,7 +98,7 @@ reveal_type(x)
for x in (1, "a", b"foo"):
pass
# revealed: Literal[1] | Literal["a"] | Literal[b"foo"]
# revealed: Literal[1, "a", b"foo"]
# error: [possibly-unresolved-reference]
reveal_type(x)
```

View File

@@ -41,7 +41,7 @@ def _(flag: bool, flag2: bool):
x = 3
reveal_type(x) # revealed: Literal[2, 3]
reveal_type(y) # revealed: Literal[1, 2, 4]
reveal_type(y) # revealed: Literal[4, 1, 2]
```
## Nested `while` loops

View File

@@ -170,8 +170,35 @@ def f(*args, **kwargs) -> int: ...
class A(metaclass=f): ...
# TODO should be `type[int]`
reveal_type(A.__class__) # revealed: @Todo(metaclass not a class)
# TODO: Should be `int`
reveal_type(A) # revealed: Literal[A]
reveal_type(A.__class__) # revealed: type[int]
def _(n: int):
# error: [invalid-metaclass]
class B(metaclass=n): ...
# TODO: Should be `Unknown`
reveal_type(B) # revealed: Literal[B]
reveal_type(B.__class__) # revealed: type[Unknown]
def _(flag: bool):
m = f if flag else 42
# error: [invalid-metaclass]
class C(metaclass=m): ...
# TODO: Should be `int | Unknown`
reveal_type(C) # revealed: Literal[C]
reveal_type(C.__class__) # revealed: type[Unknown]
class SignatureMismatch: ...
# TODO: Emit a diagnostic
class D(metaclass=SignatureMismatch): ...
# TODO: Should be `Unknown`
reveal_type(D) # revealed: Literal[D]
# TODO: Should be `type[Unknown]`
reveal_type(D.__class__) # revealed: Literal[SignatureMismatch]
```
## Cyclic

View File

@@ -56,7 +56,7 @@ def _(x_flag: bool, y_flag: bool):
def _(flag1: bool, flag2: bool):
x = None if flag1 else (1 if flag2 else True)
reveal_type(x) # revealed: None | Literal[1] | Literal[True]
reveal_type(x) # revealed: None | Literal[1, True]
if x is None:
reveal_type(x) # revealed: None
elif x is True:

View File

@@ -17,7 +17,7 @@ def _(flag: bool):
reveal_type(x) # revealed: Never
if isinstance(x, (int, object)):
reveal_type(x) # revealed: Literal[1] | Literal["a"]
reveal_type(x) # revealed: Literal[1, "a"]
```
## `classinfo` is a tuple of types
@@ -30,7 +30,7 @@ def _(flag: bool, flag1: bool, flag2: bool):
x = 1 if flag else "a"
if isinstance(x, (int, str)):
reveal_type(x) # revealed: Literal[1] | Literal["a"]
reveal_type(x) # revealed: Literal[1, "a"]
else:
reveal_type(x) # revealed: Never
@@ -43,19 +43,19 @@ def _(flag: bool, flag1: bool, flag2: bool):
# No narrowing should occur if a larger type is also
# one of the possibilities:
if isinstance(x, (int, object)):
reveal_type(x) # revealed: Literal[1] | Literal["a"]
reveal_type(x) # revealed: Literal[1, "a"]
else:
reveal_type(x) # revealed: Never
y = 1 if flag1 else "a" if flag2 else b"b"
if isinstance(y, (int, str)):
reveal_type(y) # revealed: Literal[1] | Literal["a"]
reveal_type(y) # revealed: Literal[1, "a"]
if isinstance(y, (int, bytes)):
reveal_type(y) # revealed: Literal[1] | Literal[b"b"]
reveal_type(y) # revealed: Literal[1, b"b"]
if isinstance(y, (str, bytes)):
reveal_type(y) # revealed: Literal["a"] | Literal[b"b"]
reveal_type(y) # revealed: Literal["a", b"b"]
```
## `classinfo` is a nested tuple of types
@@ -91,8 +91,7 @@ if isinstance(x, (A, B)):
elif isinstance(x, (A, C)):
reveal_type(x) # revealed: C & ~A & ~B
else:
# TODO: Should be simplified to ~A & ~B & ~C
reveal_type(x) # revealed: object & ~A & ~B & ~C
reveal_type(x) # revealed: ~A & ~B & ~C
```
## No narrowing for instances of `builtins.type`
@@ -107,7 +106,7 @@ def _(flag: bool):
x = 1 if flag else "foo"
if isinstance(x, t):
reveal_type(x) # revealed: Literal[1] | Literal["foo"]
reveal_type(x) # revealed: Literal[1, "foo"]
```
## Do not use custom `isinstance` for narrowing
@@ -119,7 +118,7 @@ def _(flag: bool):
x = 1 if flag else "a"
if isinstance(x, int):
reveal_type(x) # revealed: Literal[1] | Literal["a"]
reveal_type(x) # revealed: Literal[1, "a"]
```
## Do support narrowing if `isinstance` is aliased
@@ -155,12 +154,12 @@ def _(flag: bool):
# TODO: this should cause us to emit a diagnostic during
# type checking
if isinstance(x, "a"):
reveal_type(x) # revealed: Literal[1] | Literal["a"]
reveal_type(x) # revealed: Literal[1, "a"]
# TODO: this should cause us to emit a diagnostic during
# type checking
if isinstance(x, "int"):
reveal_type(x) # revealed: Literal[1] | Literal["a"]
reveal_type(x) # revealed: Literal[1, "a"]
```
## Do not narrow if there are keyword arguments
@@ -169,8 +168,55 @@ def _(flag: bool):
def _(flag: bool):
x = 1 if flag else "a"
# TODO: this should cause us to emit a diagnostic
# (`isinstance` has no `foo` parameter)
# error: [unknown-argument]
if isinstance(x, int, foo="bar"):
reveal_type(x) # revealed: Literal[1] | Literal["a"]
reveal_type(x) # revealed: Literal[1, "a"]
```
## `type[]` types are narrowed as well as class-literal types
```py
def _(x: object, y: type[int]):
if isinstance(x, y):
reveal_type(x) # revealed: int
```
## Adding a disjoint element to an existing intersection
We used to incorrectly infer `Literal` booleans for some of these.
```py
from knot_extensions import Not, Intersection, AlwaysTruthy, AlwaysFalsy
class P: ...
def f(
a: Intersection[P, AlwaysTruthy],
b: Intersection[P, AlwaysFalsy],
c: Intersection[P, Not[AlwaysTruthy]],
d: Intersection[P, Not[AlwaysFalsy]],
):
if isinstance(a, bool):
reveal_type(a) # revealed: Never
else:
# TODO: `bool` is final, so `& ~bool` is redundant here
reveal_type(a) # revealed: P & AlwaysTruthy & ~bool
if isinstance(b, bool):
reveal_type(b) # revealed: Never
else:
# TODO: `bool` is final, so `& ~bool` is redundant here
reveal_type(b) # revealed: P & AlwaysFalsy & ~bool
if isinstance(c, bool):
reveal_type(c) # revealed: Never
else:
# TODO: `bool` is final, so `& ~bool` is redundant here
reveal_type(c) # revealed: P & ~AlwaysTruthy & ~bool
if isinstance(d, bool):
reveal_type(d) # revealed: Never
else:
# TODO: `bool` is final, so `& ~bool` is redundant here
reveal_type(d) # revealed: P & ~AlwaysFalsy & ~bool
```

View File

@@ -90,15 +90,19 @@ def _(t: type[object]):
if issubclass(t, B):
reveal_type(t) # revealed: type[A] & type[B]
else:
reveal_type(t) # revealed: type[object] & ~type[A]
reveal_type(t) # revealed: type & ~type[A]
```
### Handling of `None`
`types.NoneType` is only available in Python 3.10 and later:
```toml
[environment]
python-version = "3.10"
```
```py
# TODO: this error should ideally go away once we (1) understand `sys.version_info` branches,
# and (2) set the target Python version for this test to 3.10.
# error: [possibly-unbound-import] "Member `NoneType` of module `types` is possibly unbound"
from types import NoneType
def _(flag: bool):
@@ -142,7 +146,7 @@ class A: ...
t = object()
# TODO: we should emit a diagnostic here
# error: [invalid-argument-type]
if issubclass(t, A):
reveal_type(t) # revealed: type[A]
```
@@ -156,7 +160,7 @@ branch:
```py
t = 1
# TODO: we should emit a diagnostic here
# error: [invalid-argument-type]
if issubclass(t, int):
reveal_type(t) # revealed: Never
```
@@ -230,8 +234,15 @@ def flag() -> bool: ...
t = int if flag() else str
# TODO: this should cause us to emit a diagnostic
# (`issubclass` has no `foo` parameter)
# error: [unknown-argument]
if issubclass(t, int, foo="bar"):
reveal_type(t) # revealed: Literal[int, str]
```
### `type[]` types are narrowed as well as class-literal types
```py
def _(x: type, y: type[int]):
if issubclass(x, y):
reveal_type(x) # revealed: type[int]
```

View File

@@ -16,3 +16,48 @@ def _(flag: bool):
reveal_type(y) # revealed: Literal[0] | None
```
## Class patterns
```py
def get_object() -> object: ...
class A: ...
class B: ...
x = get_object()
reveal_type(x) # revealed: object
match x:
case A():
reveal_type(x) # revealed: A
case B():
# TODO could be `B & ~A`
reveal_type(x) # revealed: B
reveal_type(x) # revealed: object
```
## Class pattern with guard
```py
def get_object() -> object: ...
class A:
def y() -> int: ...
class B: ...
x = get_object()
reveal_type(x) # revealed: object
match x:
case A() if reveal_type(x): # revealed: A
pass
case B() if reveal_type(x): # revealed: B
pass
reveal_type(x) # revealed: object
```

View File

@@ -9,39 +9,39 @@ def foo() -> Literal[0, -1, True, False, "", "foo", b"", b"bar", None] | tuple[(
x = foo()
if x:
reveal_type(x) # revealed: Literal[-1] | Literal[True] | Literal["foo"] | Literal[b"bar"]
reveal_type(x) # revealed: Literal[-1, True, "foo", b"bar"]
else:
reveal_type(x) # revealed: Literal[0] | Literal[False] | Literal[""] | Literal[b""] | None | tuple[()]
reveal_type(x) # revealed: Literal[0, False, "", b""] | None | tuple[()]
if not x:
reveal_type(x) # revealed: Literal[0] | Literal[False] | Literal[""] | Literal[b""] | None | tuple[()]
reveal_type(x) # revealed: Literal[0, False, "", b""] | None | tuple[()]
else:
reveal_type(x) # revealed: Literal[-1] | Literal[True] | Literal["foo"] | Literal[b"bar"]
reveal_type(x) # revealed: Literal[-1, True, "foo", b"bar"]
if x and not x:
reveal_type(x) # revealed: Never
else:
reveal_type(x) # revealed: Literal[-1, 0] | bool | Literal["", "foo"] | Literal[b"", b"bar"] | None | tuple[()]
reveal_type(x) # revealed: Literal[0, -1, "", "foo", b"", b"bar"] | bool | None | tuple[()]
if not (x and not x):
reveal_type(x) # revealed: Literal[-1, 0] | bool | Literal["", "foo"] | Literal[b"", b"bar"] | None | tuple[()]
reveal_type(x) # revealed: Literal[0, -1, "", "foo", b"", b"bar"] | bool | None | tuple[()]
else:
reveal_type(x) # revealed: Never
if x or not x:
reveal_type(x) # revealed: Literal[-1, 0] | bool | Literal["foo", ""] | Literal[b"bar", b""] | None | tuple[()]
reveal_type(x) # revealed: Literal[0, -1, "", "foo", b"", b"bar"] | bool | None | tuple[()]
else:
reveal_type(x) # revealed: Never
if not (x or not x):
reveal_type(x) # revealed: Never
else:
reveal_type(x) # revealed: Literal[-1, 0] | bool | Literal["foo", ""] | Literal[b"bar", b""] | None | tuple[()]
reveal_type(x) # revealed: Literal[0, -1, "", "foo", b"", b"bar"] | bool | None | tuple[()]
if (isinstance(x, int) or isinstance(x, str)) and x:
reveal_type(x) # revealed: Literal[-1] | Literal[True] | Literal["foo"]
reveal_type(x) # revealed: Literal[-1, True, "foo"]
else:
reveal_type(x) # revealed: Literal[b"", b"bar"] | None | tuple[()] | Literal[0] | Literal[False] | Literal[""]
reveal_type(x) # revealed: Literal[b"", b"bar", 0, False, ""] | None | tuple[()]
```
## Function Literals
@@ -87,10 +87,10 @@ def f(x: A | B):
if x and not x:
reveal_type(x) # revealed: A & ~AlwaysFalsy & ~AlwaysTruthy | B & ~AlwaysFalsy & ~AlwaysTruthy
else:
reveal_type(x) # revealed: A & ~AlwaysTruthy | B & ~AlwaysTruthy | A & ~AlwaysFalsy | B & ~AlwaysFalsy
reveal_type(x) # revealed: A | B
if x or not x:
reveal_type(x) # revealed: A & ~AlwaysFalsy | B & ~AlwaysFalsy | A & ~AlwaysTruthy | B & ~AlwaysTruthy
reveal_type(x) # revealed: A | B
else:
reveal_type(x) # revealed: A & ~AlwaysTruthy & ~AlwaysFalsy | B & ~AlwaysTruthy & ~AlwaysFalsy
```
@@ -166,16 +166,16 @@ y = literals()
if isinstance(x, str) and not isinstance(x, B):
reveal_type(x) # revealed: A & str & ~B
reveal_type(y) # revealed: Literal[0, 42] | Literal["", "hello"]
reveal_type(y) # revealed: Literal[0, 42, "", "hello"]
z = x if flag() else y
reveal_type(z) # revealed: A & str & ~B | Literal[0, 42] | Literal["", "hello"]
reveal_type(z) # revealed: A & str & ~B | Literal[0, 42, "", "hello"]
if z:
reveal_type(z) # revealed: A & str & ~B & ~AlwaysFalsy | Literal[42] | Literal["hello"]
reveal_type(z) # revealed: A & str & ~B & ~AlwaysFalsy | Literal[42, "hello"]
else:
reveal_type(z) # revealed: A & str & ~B & ~AlwaysTruthy | Literal[0] | Literal[""]
reveal_type(z) # revealed: A & str & ~B & ~AlwaysTruthy | Literal[0, ""]
```
## Narrowing Multiple Variables
@@ -199,7 +199,7 @@ def f(x: Literal[0, 1], y: Literal["", "hello"]):
reveal_type(y) # revealed: Literal["", "hello"]
```
## ControlFlow Merging
## Control Flow Merging
After merging control flows, when we take the union of all constraints applied in each branch, we
should return to the original state.
@@ -214,8 +214,118 @@ if x and not x:
reveal_type(y) # revealed: A & ~AlwaysFalsy & ~AlwaysTruthy
else:
y = x
reveal_type(y) # revealed: A & ~AlwaysTruthy | A & ~AlwaysFalsy
reveal_type(y) # revealed: A
# TODO: It should be A. We should improve UnionBuilder or IntersectionBuilder. (issue #15023)
reveal_type(y) # revealed: A & ~AlwaysTruthy | A & ~AlwaysFalsy
reveal_type(y) # revealed: A
```
## Truthiness of classes
```py
class MetaAmbiguous(type):
def __bool__(self) -> bool: ...
class MetaFalsy(type):
def __bool__(self) -> Literal[False]: ...
class MetaTruthy(type):
def __bool__(self) -> Literal[True]: ...
class MetaDeferred(type):
def __bool__(self) -> MetaAmbiguous: ...
class AmbiguousClass(metaclass=MetaAmbiguous): ...
class FalsyClass(metaclass=MetaFalsy): ...
class TruthyClass(metaclass=MetaTruthy): ...
class DeferredClass(metaclass=MetaDeferred): ...
def _(
a: type[AmbiguousClass],
t: type[TruthyClass],
f: type[FalsyClass],
d: type[DeferredClass],
ta: type[TruthyClass | AmbiguousClass],
af: type[AmbiguousClass] | type[FalsyClass],
flag: bool,
):
reveal_type(ta) # revealed: type[TruthyClass] | type[AmbiguousClass]
if ta:
reveal_type(ta) # revealed: type[TruthyClass] | type[AmbiguousClass] & ~AlwaysFalsy
reveal_type(af) # revealed: type[AmbiguousClass] | type[FalsyClass]
if af:
reveal_type(af) # revealed: type[AmbiguousClass] & ~AlwaysFalsy
# TODO: Emit a diagnostic (`d` is not valid in boolean context)
if d:
# TODO: Should be `Unknown`
reveal_type(d) # revealed: type[DeferredClass] & ~AlwaysFalsy
tf = TruthyClass if flag else FalsyClass
reveal_type(tf) # revealed: Literal[TruthyClass, FalsyClass]
if tf:
reveal_type(tf) # revealed: Literal[TruthyClass]
else:
reveal_type(tf) # revealed: Literal[FalsyClass]
```
## Narrowing in chained boolean expressions
```py
from typing import Literal
class A: ...
def _(x: Literal[0, 1]):
reveal_type(x or A()) # revealed: Literal[1] | A
reveal_type(x and A()) # revealed: Literal[0] | A
def _(x: str):
reveal_type(x or A()) # revealed: str & ~AlwaysFalsy | A
reveal_type(x and A()) # revealed: str & ~AlwaysTruthy | A
def _(x: bool | str):
reveal_type(x or A()) # revealed: Literal[True] | str & ~AlwaysFalsy | A
reveal_type(x and A()) # revealed: Literal[False] | str & ~AlwaysTruthy | A
class Falsy:
def __bool__(self) -> Literal[False]: ...
class Truthy:
def __bool__(self) -> Literal[True]: ...
def _(x: Falsy | Truthy):
reveal_type(x or A()) # revealed: Truthy | A
reveal_type(x and A()) # revealed: Falsy | A
class MetaFalsy(type):
def __bool__(self) -> Literal[False]: ...
class MetaTruthy(type):
def __bool__(self) -> Literal[True]: ...
class FalsyClass(metaclass=MetaFalsy): ...
class TruthyClass(metaclass=MetaTruthy): ...
def _(x: type[FalsyClass] | type[TruthyClass]):
reveal_type(x or A()) # revealed: type[TruthyClass] | A
reveal_type(x and A()) # revealed: type[FalsyClass] | A
```
## Truthiness narrowing for `LiteralString`
```py
from typing_extensions import LiteralString
def _(x: LiteralString):
if x:
reveal_type(x) # revealed: LiteralString & ~Literal[""]
else:
reveal_type(x) # revealed: Literal[""]
if not x:
reveal_type(x) # revealed: Literal[""]
else:
reveal_type(x) # revealed: LiteralString & ~Literal[""]
```

View File

@@ -37,7 +37,7 @@ class C:
# error: [possibly-unresolved-reference]
y = x
reveal_type(C.y) # revealed: Literal[1] | Literal["abc"]
reveal_type(C.y) # revealed: Literal[1, "abc"]
```
## Unbound function local

View File

@@ -25,3 +25,29 @@ def f(): ...
f: int = 1
```
## Explicit shadowing involving `def` statements
Since a `def` statement is a declaration, one `def` can shadow another `def`, or shadow a previous
non-`def` declaration, without error.
```py
f = 1
reveal_type(f) # revealed: Literal[1]
def f(): ...
reveal_type(f) # revealed: Literal[f]
def f(x: int) -> int:
raise NotImplementedError
reveal_type(f) # revealed: Literal[f]
f: int = 1
reveal_type(f) # revealed: Literal[1]
def f(): ...
reveal_type(f) # revealed: Literal[f]
```

View File

@@ -0,0 +1,184 @@
# `__slots__`
## Not specified and empty
```py
class A: ...
class B:
__slots__ = ()
class C:
__slots__ = ("lorem", "ipsum")
class AB(A, B): ... # fine
class AC(A, C): ... # fine
class BC(B, C): ... # fine
class ABC(A, B, C): ... # fine
```
## Incompatible tuples
```py
class A:
__slots__ = ("a", "b")
class B:
__slots__ = ("c", "d")
class C(
A, # error: [incompatible-slots]
B, # error: [incompatible-slots]
): ...
```
## Same value
```py
class A:
__slots__ = ("a", "b")
class B:
__slots__ = ("a", "b")
class C(
A, # error: [incompatible-slots]
B, # error: [incompatible-slots]
): ...
```
## Strings
```py
class A:
__slots__ = "abc"
class B:
__slots__ = ("abc",)
class AB(
A, # error: [incompatible-slots]
B, # error: [incompatible-slots]
): ...
```
## Invalid
TODO: Emit diagnostics
```py
class NonString1:
__slots__ = 42
class NonString2:
__slots__ = b"ar"
class NonIdentifier1:
__slots__ = "42"
class NonIdentifier2:
__slots__ = ("lorem", "42")
class NonIdentifier3:
__slots__ = (e for e in ("lorem", "42"))
```
## Inheritance
```py
class A:
__slots__ = ("a", "b")
class B(A): ...
class C:
__slots__ = ("c", "d")
class D(C): ...
class E(
B, # error: [incompatible-slots]
D, # error: [incompatible-slots]
): ...
```
## Single solid base
```py
class A:
__slots__ = ("a", "b")
class B(A): ...
class C(A): ...
class D(B, A): ... # fine
class E(B, C, A): ... # fine
```
## False negatives
### Possibly unbound
```py
def _(flag: bool):
class A:
if flag:
__slots__ = ("a", "b")
class B:
__slots__ = ("c", "d")
# Might or might not be fine at runtime
class C(A, B): ...
```
### Bound but with different types
```py
def _(flag: bool):
class A:
if flag:
__slots__ = ("a", "b")
else:
__slots__ = ()
class B:
__slots__ = ("c", "d")
# Might or might not be fine at runtime
class C(A, B): ...
```
### Non-tuples
```py
class A:
__slots__ = ["a", "b"] # This is treated as "dynamic"
class B:
__slots__ = ("c", "d")
# False negative: [incompatible-slots]
class C(A, B): ...
```
### Post-hoc modifications
```py
class A:
__slots__ = ()
__slots__ += ("a", "b")
reveal_type(A.__slots__) # revealed: @Todo(return type)
class B:
__slots__ = ("c", "d")
# False negative: [incompatible-slots]
class C(A, B): ...
```
### Built-ins with implicit layouts
```py
# False negative: [incompatible-slots]
class A(int, str): ...
```

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,78 @@
# Ellipsis
## Function and methods
The ellipsis literal `...` can be used as a placeholder default value for a function parameter, in a
stub file only, regardless of the type of the parameter.
```py path=test.pyi
def f(x: int = ...) -> None:
reveal_type(x) # revealed: int
def f2(x: str = ...) -> None:
reveal_type(x) # revealed: str
```
## Class and module symbols
The ellipsis literal can be assigned to a class or module symbol, regardless of its declared type,
in a stub file only.
```py path=test.pyi
y: bytes = ...
reveal_type(y) # revealed: bytes
x = ...
reveal_type(x) # revealed: Unknown
class Foo:
y: int = ...
reveal_type(Foo.y) # revealed: int
```
## Unpacking ellipsis literal in assignment
No diagnostic is emitted if an ellipsis literal is "unpacked" in a stub file as part of an
assignment statement:
```py path=test.pyi
x, y = ...
reveal_type(x) # revealed: Unknown
reveal_type(y) # revealed: Unknown
```
## Unpacking ellipsis literal in for loops
Iterating over an ellipsis literal as part of a `for` loop in a stub is invalid, however, and
results in a diagnostic:
```py path=test.pyi
# error: [not-iterable] "Object of type `ellipsis` is not iterable"
for a, b in ...:
reveal_type(a) # revealed: Unknown
reveal_type(b) # revealed: Unknown
```
## Ellipsis usage in non stub file
In a non-stub file, there's no special treatment of ellipsis literals. An ellipsis literal can only
be assigned if `EllipsisType` is actually assignable to the annotated type.
```py
# error: 7 [invalid-parameter-default] "Default value of type `ellipsis` is not assignable to annotated parameter type `int`"
def f(x: int = ...) -> None: ...
# error: 1 [invalid-assignment] "Object of type `ellipsis` is not assignable to `int`"
a: int = ...
b = ...
reveal_type(b) # revealed: ellipsis
```
## Use of `Ellipsis` symbol
There is no special treatment of the builtin name `Ellipsis` in stubs, only of `...` literals.
```py path=test.pyi
# error: 7 [invalid-parameter-default] "Default value of type `ellipsis` is not assignable to annotated parameter type `int`"
def f(x: int = Ellipsis) -> None: ...
```

View File

@@ -81,10 +81,7 @@ python-version = "3.9"
```
```py
# TODO:
# * `tuple.__class_getitem__` is always bound on 3.9 (`sys.version_info`)
# * `tuple[int, str]` is a valid base (generics)
# error: [call-possibly-unbound-method] "Method `__class_getitem__` of type `Literal[tuple]` is possibly unbound"
# TODO: `tuple[int, str]` is a valid base (generics)
# error: [invalid-base] "Invalid class base with type `GenericAlias` (all bases must be a class, `Any`, `Unknown` or `Todo`)"
class A(tuple[int, str]): ...

View File

@@ -0,0 +1,182 @@
# Suppressing errors with `knot: ignore`
Type check errors can be suppressed by a `knot: ignore` comment on the same line as the violation.
## Simple `knot: ignore`
```py
a = 4 + test # knot: ignore
```
## Suppressing a specific code
```py
a = 4 + test # knot: ignore[unresolved-reference]
```
## Unused suppression
```py
test = 10
# error: [unused-ignore-comment] "Unused `knot: ignore` directive: 'possibly-unresolved-reference'"
a = test + 3 # knot: ignore[possibly-unresolved-reference]
```
## Unused suppression if the error codes don't match
```py
# error: [unresolved-reference]
# error: [unused-ignore-comment] "Unused `knot: ignore` directive: 'possibly-unresolved-reference'"
a = test + 3 # knot: ignore[possibly-unresolved-reference]
```
## Suppressed unused comment
```py
# error: [unused-ignore-comment]
a = 10 / 2 # knot: ignore[division-by-zero]
a = 10 / 2 # knot: ignore[division-by-zero, unused-ignore-comment]
a = 10 / 2 # knot: ignore[unused-ignore-comment, division-by-zero]
a = 10 / 2 # knot: ignore[unused-ignore-comment] # type: ignore
a = 10 / 2 # type: ignore # knot: ignore[unused-ignore-comment]
```
## Unused ignore comment
```py
# error: [unused-ignore-comment] "Unused `knot: ignore` directive: 'unused-ignore-comment'"
a = 10 / 0 # knot: ignore[division-by-zero, unused-ignore-comment]
```
## Multiple unused comments
Today, Red Knot emits a diagnostic for every unused code. We might want to group the codes by
comment at some point in the future.
```py
# error: [unused-ignore-comment] "Unused `knot: ignore` directive: 'division-by-zero'"
# error: [unused-ignore-comment] "Unused `knot: ignore` directive: 'unresolved-reference'"
a = 10 / 2 # knot: ignore[division-by-zero, unresolved-reference]
# error: [unused-ignore-comment] "Unused `knot: ignore` directive: 'invalid-assignment'"
# error: [unused-ignore-comment] "Unused `knot: ignore` directive: 'unresolved-reference'"
a = 10 / 0 # knot: ignore[invalid-assignment, division-by-zero, unresolved-reference]
```
## Multiple suppressions
```py
# fmt: off
def test(a: f"f-string type annotation", b: b"byte-string-type-annotation"): ... # knot: ignore[fstring-type-annotation, byte-string-type-annotation]
```
## Can't suppress syntax errors
<!-- blacken-docs:off -->
```py
# error: [invalid-syntax]
# error: [unused-ignore-comment]
def test( # knot: ignore
```
<!-- blacken-docs:on -->
## Can't suppress `revealed-type` diagnostics
```py
a = 10
# revealed: Literal[10]
# error: [unknown-rule] "Unknown rule `revealed-type`"
reveal_type(a) # knot: ignore[revealed-type]
```
## Extra whitespace in type ignore comments is allowed
```py
a = 10 / 0 # knot : ignore
a = 10 / 0 # knot: ignore [ division-by-zero ]
```
## Whitespace is optional
```py
# fmt: off
a = 10 / 0 #knot:ignore[division-by-zero]
```
## Trailing codes comma
Trailing commas in the codes section are allowed:
```py
a = 10 / 0 # knot: ignore[division-by-zero,]
```
## Invalid characters in codes
```py
# error: [division-by-zero]
# error: [invalid-ignore-comment] "Invalid `knot: ignore` comment: expected a alphanumeric character or `-` or `_` as code"
a = 10 / 0 # knot: ignore[*-*]
```
## Trailing whitespace
<!-- blacken-docs:off -->
```py
a = 10 / 0 # knot: ignore[division-by-zero]
# ^^^^^^ trailing whitespace
```
<!-- blacken-docs:on -->
## Missing comma
A missing comma results in an invalid suppression comment. We may want to recover from this in the
future.
```py
# error: [unresolved-reference]
# error: [invalid-ignore-comment] "Invalid `knot: ignore` comment: expected a comma separating the rule codes"
a = x / 0 # knot: ignore[division-by-zero unresolved-reference]
```
## Missing closing bracket
```py
# error: [unresolved-reference] "Name `x` used when not defined"
# error: [invalid-ignore-comment] "Invalid `knot: ignore` comment: expected a comma separating the rule codes"
a = x / 2 # knot: ignore[unresolved-reference
```
## Empty codes
An empty codes array suppresses no-diagnostics and is always useless
```py
# error: [division-by-zero]
# error: [unused-ignore-comment] "Unused `knot: ignore` without a code"
a = 4 / 0 # knot: ignore[]
```
## File-level suppression comments
File level suppression comments are currently intentionally unsupported because we've yet to decide
if they should use a different syntax that also supports enabling rules or changing the rule's
severity: `knot: possibly-undefined-reference=error`
```py
# error: [unused-ignore-comment]
# knot: ignore[division-by-zero]
a = 4 / 0 # error: [division-by-zero]
```
## Unknown rule
```py
# error: [unknown-rule] "Unknown rule `is-equal-14`"
a = 10 + 4 # knot: ignore[is-equal-14]
```

View File

@@ -0,0 +1,118 @@
# `@no_type_check`
> If a type checker supports the `no_type_check` decorator for functions, it should suppress all
> type errors for the def statement and its body including any nested functions or classes. It
> should also ignore all parameter and return type annotations and treat the function as if it were
> unannotated. [source](https://typing.readthedocs.io/en/latest/spec/directives.html#no-type-check)
## Error in the function body
```py
from typing import no_type_check
@no_type_check
def test() -> int:
return a + 5
```
## Error in nested function
```py
from typing import no_type_check
@no_type_check
def test() -> int:
def nested():
return a + 5
```
## Error in nested class
```py
from typing import no_type_check
@no_type_check
def test() -> int:
class Nested:
def inner(self):
return a + 5
```
## Error in preceding decorator
Don't suppress diagnostics for decorators appearing before the `no_type_check` decorator.
```py
from typing import no_type_check
@unknown_decorator # error: [unresolved-reference]
@no_type_check
def test() -> int:
return a + 5
```
## Error in following decorator
Unlike Pyright and mypy, suppress diagnostics appearing after the `no_type_check` decorator. We do
this because it more closely matches Python's runtime semantics of decorators. For more details, see
the discussion on the
[PR adding `@no_type_check` support](https://github.com/astral-sh/ruff/pull/15122#discussion_r1896869411).
```py
from typing import no_type_check
@no_type_check
@unknown_decorator
def test() -> int:
return a + 5
```
## Error in default value
```py
from typing import no_type_check
@no_type_check
def test(a: int = "test"):
return x + 5
```
## Error in return value position
```py
from typing import no_type_check
@no_type_check
def test() -> Undefined:
return x + 5
```
## `no_type_check` on classes isn't supported
Red Knot does not support decorating classes with `no_type_check`. The behaviour of `no_type_check`
when applied to classes is
[not specified currently](https://typing.readthedocs.io/en/latest/spec/directives.html#no-type-check),
and is not supported by Pyright or mypy.
A future improvement might be to emit a diagnostic if a `no_type_check` annotation is applied to a
class.
```py
from typing import no_type_check
@no_type_check
class Test:
def test(self):
return a + 5 # error: [unresolved-reference]
```
## `type: ignore` comments in `@no_type_check` blocks
```py
from typing import no_type_check
@no_type_check
def test():
# error: [unused-ignore-comment] "Unused `knot: ignore` directive: 'unresolved-reference'"
return x + 5 # knot: ignore[unresolved-reference]
```

View File

@@ -0,0 +1,162 @@
# Suppressing errors with `type: ignore`
Type check errors can be suppressed by a `type: ignore` comment on the same line as the violation.
## Simple `type: ignore`
```py
a = 4 + test # type: ignore
```
## Multiline ranges
A diagnostic with a multiline range can be suppressed by a comment on the same line as the
diagnostic's start or end. This is the same behavior as Mypy's.
```py
# fmt: off
y = (
4 / 0 # type: ignore
)
y = (
4 / # type: ignore
0
)
y = (
4 /
0 # type: ignore
)
```
Pyright diverges from this behavior and instead applies a suppression if its range intersects with
the diagnostic range. This can be problematic for nested expressions because a suppression in a
child expression now suppresses errors in the outer expression.
For example, the `type: ignore` comment in this example suppresses the error of adding `2` to
`"test"` and adding `"other"` to the result of the cast.
```py path=nested.py
# fmt: off
from typing import cast
y = (
cast(int, "test" +
# TODO: Remove the expected error after implementing `invalid-operator` for binary expressions
# error: [unused-ignore-comment]
2 # type: ignore
)
+ "other" # TODO: expected-error[invalid-operator]
)
```
Mypy flags the second usage.
## Before opening parenthesis
A suppression that applies to all errors before the opening parenthesis.
```py
a: Test = ( # type: ignore
Test() # error: [unresolved-reference]
) # fmt: skip
```
## Multiline string
```py
a: int = 4
a = """
This is a multiline string and the suppression is at its end
""" # type: ignore
```
## Line continuations
Suppressions after a line continuation apply to all previous lines.
```py
# fmt: off
a = test \
+ 2 # type: ignore
a = test \
+ a \
+ 2 # type: ignore
```
## Codes
Mypy supports `type: ignore[code]`. Red Knot doesn't understand mypy's rule names. Therefore, ignore
the codes and suppress all errors.
```py
a = test # type: ignore[name-defined]
```
## Nested comments
```py
# fmt: off
a = test \
+ 2 # fmt: skip # type: ignore
a = test \
+ 2 # type: ignore # fmt: skip
```
## Misspelled `type: ignore`
```py
# error: [unresolved-reference]
# error: [invalid-ignore-comment]
a = test + 2 # type: ignoree
```
## Invalid - ignore on opening parentheses
`type: ignore` comments after an opening parentheses suppress any type errors inside the parentheses
in Pyright. Neither Ruff, nor mypy support this and neither does Red Knot.
```py
# fmt: off
# error: [unused-ignore-comment]
a = ( # type: ignore
test + 4 # error: [unresolved-reference]
)
```
## File level suppression
```py
# type: ignore
a = 10 / 0
b = a / 0
```
## File level suppression with leading shebang
```py
#!/usr/bin/env/python
# type: ignore
a = 10 / 0
b = a / 0
```
## Invalid own-line suppression
```py
"""
File level suppressions must come before any non-trivia token,
including module docstrings.
"""
# error: [unused-ignore-comment] "Unused blanket `type: ignore` directive"
# type: ignore
a = 10 / 0 # error: [division-by-zero]
b = a / 0 # error: [division-by-zero]
```

View File

@@ -0,0 +1,71 @@
# `sys.platform`
## Default value
When no target platform is specified, we fall back to the type of `sys.platform` declared in
typeshed:
```toml
[environment]
# No python-platform entry
```
```py
import sys
reveal_type(sys.platform) # revealed: LiteralString
```
## Explicit selection of `all` platforms
```toml
[environment]
python-platform = "all"
```
```py
import sys
reveal_type(sys.platform) # revealed: LiteralString
```
## Explicit selection of a specific platform
```toml
[environment]
python-platform = "linux"
```
```py
import sys
reveal_type(sys.platform) # revealed: Literal["linux"]
```
## Testing for a specific platform
### Exact comparison
```toml
[environment]
python-platform = "freebsd8"
```
```py
import sys
reveal_type(sys.platform == "freebsd8") # revealed: Literal[True]
reveal_type(sys.platform == "linux") # revealed: Literal[False]
```
### Substring comparison
It is [recommended](https://docs.python.org/3/library/sys.html#sys.platform) to use
`sys.platform.startswith(...)` for platform checks. This is not yet supported in type inference:
```py
import sys
reveal_type(sys.platform.startswith("freebsd")) # revealed: @Todo(Attribute access on `LiteralString` types)
reveal_type(sys.platform.startswith("linux")) # revealed: @Todo(Attribute access on `LiteralString` types)
```

View File

@@ -0,0 +1,382 @@
# Type API (`knot_extensions`)
This document describes the internal `knot_extensions` API for creating and manipulating types as
well as testing various type system properties.
## Type extensions
The Python language itself allows us to perform a variety of operations on types. For example, we
can build a union of types like `int | None`, or we can use type constructors such as `list[int]`
and `type[int]` to create new types. But some type-level operations that we rely on in Red Knot,
like intersections, cannot yet be expressed in Python. The `knot_extensions` module provides the
`Intersection` and `Not` type constructors (special forms) which allow us to construct these types
directly.
### Negation
```py
from knot_extensions import Not, static_assert
def negate(n1: Not[int], n2: Not[Not[int]], n3: Not[Not[Not[int]]]) -> None:
reveal_type(n1) # revealed: ~int
reveal_type(n2) # revealed: int
reveal_type(n3) # revealed: ~int
def static_truthiness(not_one: Not[Literal[1]]) -> None:
static_assert(not_one != 1)
static_assert(not (not_one == 1))
# error: "Special form `knot_extensions.Not` expected exactly one type parameter"
n: Not[int, str]
```
### Intersection
```py
from knot_extensions import Intersection, Not, is_subtype_of, static_assert
from typing_extensions import Never
class S: ...
class T: ...
def x(x1: Intersection[S, T], x2: Intersection[S, Not[T]]) -> None:
reveal_type(x1) # revealed: S & T
reveal_type(x2) # revealed: S & ~T
def y(y1: Intersection[int, object], y2: Intersection[int, bool], y3: Intersection[int, Never]) -> None:
reveal_type(y1) # revealed: int
reveal_type(y2) # revealed: bool
reveal_type(y3) # revealed: Never
def z(z1: Intersection[int, Not[Literal[1]], Not[Literal[2]]]) -> None:
reveal_type(z1) # revealed: int & ~Literal[1] & ~Literal[2]
class A: ...
class B: ...
class C: ...
type ABC = Intersection[A, B, C]
static_assert(is_subtype_of(ABC, A))
static_assert(is_subtype_of(ABC, B))
static_assert(is_subtype_of(ABC, C))
class D: ...
static_assert(not is_subtype_of(ABC, D))
```
### Unknown type
The `Unknown` type is a special type that we use to represent actually unknown types (no
annotation), as opposed to `Any` which represents an explicitly unknown type.
```py
from knot_extensions import Unknown, static_assert, is_assignable_to, is_fully_static
static_assert(is_assignable_to(Unknown, int))
static_assert(is_assignable_to(int, Unknown))
static_assert(not is_fully_static(Unknown))
def explicit_unknown(x: Unknown, y: tuple[str, Unknown], z: Unknown = 1) -> None:
reveal_type(x) # revealed: Unknown
reveal_type(y) # revealed: tuple[str, Unknown]
reveal_type(z) # revealed: Unknown | Literal[1]
# Unknown can be subclassed, just like Any
class C(Unknown): ...
# revealed: tuple[Literal[C], Unknown, Literal[object]]
reveal_type(C.__mro__)
# error: "Special form `knot_extensions.Unknown` expected no type parameter"
u: Unknown[str]
```
### `AlwaysTruthy` and `AlwaysFalsy`
`AlwaysTruthy` and `AlwaysFalsy` represent the sets of all possible objects whose truthiness is
always truthy or falsy, respectively.
They do not accept any type arguments.
```py
from typing_extensions import Literal
from knot_extensions import AlwaysFalsy, AlwaysTruthy, is_subtype_of, static_assert
static_assert(is_subtype_of(Literal[True], AlwaysTruthy))
static_assert(is_subtype_of(Literal[False], AlwaysFalsy))
static_assert(not is_subtype_of(int, AlwaysFalsy))
static_assert(not is_subtype_of(str, AlwaysFalsy))
def _(t: AlwaysTruthy, f: AlwaysFalsy):
reveal_type(t) # revealed: AlwaysTruthy
reveal_type(f) # revealed: AlwaysFalsy
def f(
a: AlwaysTruthy[int], # error: [invalid-type-form]
b: AlwaysFalsy[str], # error: [invalid-type-form]
):
reveal_type(a) # revealed: Unknown
reveal_type(b) # revealed: Unknown
```
## Static assertions
### Basics
The `knot_extensions` module provides a `static_assert` function that can be used to enforce
properties at type-check time. The function takes an arbitrary expression and raises a type error if
the expression is not of statically known truthiness.
```py
from knot_extensions import static_assert
from typing import TYPE_CHECKING
import sys
static_assert(True)
static_assert(False) # error: "Static assertion error: argument evaluates to `False`"
static_assert(False or True)
static_assert(True and True)
static_assert(False or False) # error: "Static assertion error: argument evaluates to `False`"
static_assert(False and True) # error: "Static assertion error: argument evaluates to `False`"
static_assert(1 + 1 == 2)
static_assert(1 + 1 == 3) # error: "Static assertion error: argument evaluates to `False`"
static_assert("a" in "abc")
static_assert("d" in "abc") # error: "Static assertion error: argument evaluates to `False`"
n = None
static_assert(n is None)
static_assert(TYPE_CHECKING)
static_assert(sys.version_info >= (3, 6))
```
### Narrowing constraints
Static assertions can be used to enforce narrowing constraints:
```py
from knot_extensions import static_assert
def f(x: int) -> None:
if x != 0:
static_assert(x != 0)
else:
# `int` can be subclassed, so we cannot assert that `x == 0` here:
# error: "Static assertion error: argument of type `bool` has an ambiguous static truthiness"
static_assert(x == 0)
```
### Truthy expressions
See also: <https://docs.python.org/3/library/stdtypes.html#truth-value-testing>
```py
from knot_extensions import static_assert
static_assert(True)
static_assert(False) # error: "Static assertion error: argument evaluates to `False`"
static_assert(None) # error: "Static assertion error: argument of type `None` is statically known to be falsy"
static_assert(1)
static_assert(0) # error: "Static assertion error: argument of type `Literal[0]` is statically known to be falsy"
static_assert((0,))
static_assert(()) # error: "Static assertion error: argument of type `tuple[()]` is statically known to be falsy"
static_assert("a")
static_assert("") # error: "Static assertion error: argument of type `Literal[""]` is statically known to be falsy"
static_assert(b"a")
static_assert(b"") # error: "Static assertion error: argument of type `Literal[b""]` is statically known to be falsy"
```
### Error messages
We provide various tailored error messages for wrong argument types to `static_assert`:
```py
from knot_extensions import static_assert
static_assert(2 * 3 == 6)
# error: "Static assertion error: argument evaluates to `False`"
static_assert(2 * 3 == 7)
# error: "Static assertion error: argument of type `bool` has an ambiguous static truthiness"
static_assert(int(2.0 * 3.0) == 6)
class InvalidBoolDunder:
def __bool__(self) -> int:
return 1
# error: "Static assertion error: argument of type `InvalidBoolDunder` has an ambiguous static truthiness"
static_assert(InvalidBoolDunder())
```
### Custom error messages
Alternatively, users can provide custom error messages:
```py
from knot_extensions import static_assert
# error: "Static assertion error: I really want this to be true"
static_assert(1 + 1 == 3, "I really want this to be true")
error_message = "A custom message "
error_message += "constructed from multiple string literals"
# error: "Static assertion error: A custom message constructed from multiple string literals"
static_assert(False, error_message)
# There are limitations to what we can still infer as a string literal. In those cases,
# we simply fall back to the default message.
shouted_message = "A custom message".upper()
# error: "Static assertion error: argument evaluates to `False`"
static_assert(False, shouted_message)
```
## Type predicates
The `knot_extensions` module also provides predicates to test various properties of types. These are
implemented as functions that return `Literal[True]` or `Literal[False]` depending on the result of
the test.
### Equivalence
```py
from knot_extensions import is_equivalent_to, static_assert
from typing_extensions import Never, Union
static_assert(is_equivalent_to(type, type[object]))
static_assert(is_equivalent_to(tuple[int, Never], Never))
static_assert(is_equivalent_to(int | str, Union[int, str]))
static_assert(not is_equivalent_to(int, str))
static_assert(not is_equivalent_to(int | str, int | str | bytes))
```
### Subtyping
```py
from knot_extensions import is_subtype_of, static_assert
static_assert(is_subtype_of(bool, int))
static_assert(not is_subtype_of(str, int))
static_assert(is_subtype_of(bool, int | str))
static_assert(is_subtype_of(str, int | str))
static_assert(not is_subtype_of(bytes, int | str))
class Base: ...
class Derived(Base): ...
class Unrelated: ...
static_assert(is_subtype_of(Derived, Base))
static_assert(not is_subtype_of(Base, Derived))
static_assert(is_subtype_of(Base, Base))
static_assert(not is_subtype_of(Unrelated, Base))
static_assert(not is_subtype_of(Base, Unrelated))
```
### Assignability
```py
from knot_extensions import is_assignable_to, static_assert
from typing import Any
static_assert(is_assignable_to(int, Any))
static_assert(is_assignable_to(Any, str))
static_assert(not is_assignable_to(int, str))
```
### Disjointness
```py
from knot_extensions import is_disjoint_from, static_assert
static_assert(is_disjoint_from(None, int))
static_assert(not is_disjoint_from(Literal[2] | str, int))
```
### Fully static types
```py
from knot_extensions import is_fully_static, static_assert
from typing import Any
static_assert(is_fully_static(int | str))
static_assert(is_fully_static(type[int]))
static_assert(not is_fully_static(int | Any))
static_assert(not is_fully_static(type[Any]))
```
### Singleton types
```py
from knot_extensions import is_singleton, static_assert
static_assert(is_singleton(None))
static_assert(is_singleton(Literal[True]))
static_assert(not is_singleton(int))
static_assert(not is_singleton(Literal["a"]))
```
### Single-valued types
```py
from knot_extensions import is_single_valued, static_assert
static_assert(is_single_valued(None))
static_assert(is_single_valued(Literal[True]))
static_assert(is_single_valued(Literal["a"]))
static_assert(not is_single_valued(int))
static_assert(not is_single_valued(Literal["a"] | Literal["b"]))
```
## `TypeOf`
We use `TypeOf` to get the inferred type of an expression. This is useful when we want to refer to
it in a type expression. For example, if we want to make sure that the class literal type `str` is a
subtype of `type[str]`, we can not use `is_subtype_of(str, type[str])`, as that would test if the
type `str` itself is a subtype of `type[str]`. Instead, we can use `TypeOf[str]` to get the type of
the expression `str`:
```py
from knot_extensions import TypeOf, is_subtype_of, static_assert
# This is incorrect and therefore fails with ...
# error: "Static assertion error: argument evaluates to `False`"
static_assert(is_subtype_of(str, type[str]))
# Correct, returns True:
static_assert(is_subtype_of(TypeOf[str], type[str]))
class Base: ...
class Derived(Base): ...
# `TypeOf` can be used in annotations:
def type_of_annotation() -> None:
t1: TypeOf[Base] = Base
t2: TypeOf[Base] = Derived # error: [invalid-assignment]
# Note how this is different from `type[…]` which includes subclasses:
s1: type[Base] = Base
s2: type[Base] = Derived # no error here
# error: "Special form `knot_extensions.TypeOf` expected exactly one type parameter"
t: TypeOf[int, str, bytes]
```

View File

@@ -142,3 +142,25 @@ class Foo(type[int]): ...
# TODO: should be `tuple[Literal[Foo], Literal[type], Literal[object]]
reveal_type(Foo.__mro__) # revealed: tuple[Literal[Foo], Unknown, Literal[object]]
```
## `@final` classes
`type[]` types are eagerly converted to class-literal types if a class decorated with `@final` is
used as the type argument. This applies to standard-library classes and user-defined classes:
```toml
[environment]
python-version = "3.10"
```
```py
from types import EllipsisType
from typing import final
@final
class Foo: ...
def _(x: type[Foo], y: type[EllipsisType]):
reveal_type(x) # revealed: Literal[Foo]
reveal_type(y) # revealed: Literal[EllipsisType]
```

View File

@@ -47,9 +47,8 @@ x: type = A() # error: [invalid-assignment]
```py
def f(x: type[object]):
reveal_type(x) # revealed: type[object]
# TODO: bound method types
reveal_type(x.__repr__) # revealed: Literal[__repr__]
reveal_type(x) # revealed: type
reveal_type(x.__repr__) # revealed: @Todo(instance attributes)
class A: ...

View File

@@ -0,0 +1,352 @@
# Assignable-to relation
The `is_assignable_to(S, T)` relation below checks if type `S` is assignable to type `T` (target).
This allows us to check if a type `S` can be used in a context where a type `T` is expected
(function arguments, variable assignments). See the [typing documentation] for a precise definition
of this concept.
## Basic types
### Fully static
Fully static types participate in subtyping. If a type `S` is a subtype of `T`, `S` will also be
assignable to `T`. Two equivalent types are subtypes of each other:
```py
from knot_extensions import static_assert, is_assignable_to
class Parent: ...
class Child1(Parent): ...
class Child2(Parent): ...
class Grandchild(Child1, Child2): ...
class Unrelated: ...
static_assert(is_assignable_to(int, int))
static_assert(is_assignable_to(Parent, Parent))
static_assert(is_assignable_to(Child1, Parent))
static_assert(is_assignable_to(Grandchild, Parent))
static_assert(is_assignable_to(Unrelated, Unrelated))
static_assert(not is_assignable_to(str, int))
static_assert(not is_assignable_to(object, int))
static_assert(not is_assignable_to(Parent, Child1))
static_assert(not is_assignable_to(Unrelated, Parent))
static_assert(not is_assignable_to(Child1, Child2))
```
### Gradual types
Gradual types do not participate in subtyping, but can still be assignable to other types (and
static types can be assignable to gradual types):
```py
from knot_extensions import static_assert, is_assignable_to, Unknown
from typing import Any
static_assert(is_assignable_to(Unknown, Literal[1]))
static_assert(is_assignable_to(Any, Literal[1]))
static_assert(is_assignable_to(Literal[1], Unknown))
static_assert(is_assignable_to(Literal[1], Any))
```
## Literal types
### Boolean literals
`Literal[True]` and `Literal[False]` are both subtypes of (and therefore assignable to) `bool`,
which is in turn a subtype of `int`:
```py
from knot_extensions import static_assert, is_assignable_to
from typing import Literal
static_assert(is_assignable_to(Literal[True], Literal[True]))
static_assert(is_assignable_to(Literal[True], bool))
static_assert(is_assignable_to(Literal[True], int))
static_assert(not is_assignable_to(Literal[True], Literal[False]))
static_assert(not is_assignable_to(bool, Literal[True]))
```
### Integer literals
```py
from knot_extensions import static_assert, is_assignable_to
from typing import Literal
static_assert(is_assignable_to(Literal[1], Literal[1]))
static_assert(is_assignable_to(Literal[1], int))
static_assert(not is_assignable_to(Literal[1], Literal[2]))
static_assert(not is_assignable_to(int, Literal[1]))
static_assert(not is_assignable_to(Literal[1], str))
```
### String literals and `LiteralString`
All string-literal types are subtypes of (and therefore assignable to) `LiteralString`, which is in
turn a subtype of `str`:
```py
from knot_extensions import static_assert, is_assignable_to
from typing_extensions import Literal, LiteralString
static_assert(is_assignable_to(Literal["foo"], Literal["foo"]))
static_assert(is_assignable_to(Literal["foo"], LiteralString))
static_assert(is_assignable_to(Literal["foo"], str))
static_assert(is_assignable_to(LiteralString, str))
static_assert(not is_assignable_to(Literal["foo"], Literal["bar"]))
static_assert(not is_assignable_to(str, Literal["foo"]))
static_assert(not is_assignable_to(str, LiteralString))
```
### Byte literals
```py
from knot_extensions import static_assert, is_assignable_to
from typing_extensions import Literal, LiteralString
static_assert(is_assignable_to(Literal[b"foo"], bytes))
static_assert(is_assignable_to(Literal[b"foo"], Literal[b"foo"]))
static_assert(not is_assignable_to(Literal[b"foo"], str))
static_assert(not is_assignable_to(Literal[b"foo"], LiteralString))
static_assert(not is_assignable_to(Literal[b"foo"], Literal[b"bar"]))
static_assert(not is_assignable_to(Literal[b"foo"], Literal["foo"]))
static_assert(not is_assignable_to(Literal["foo"], Literal[b"foo"]))
```
## `type[…]` and class literals
In the following tests, `TypeOf[str]` is a singleton type with a single inhabitant, the class `str`.
This contrasts with `type[str]`, which represents "all possible subclasses of `str`".
Both `TypeOf[str]` and `type[str]` are subtypes of `type` and `type[object]`, which both represent
"all possible instances of `type`"; therefore both `type[str]` and `TypeOf[str]` are assignable to
`type`. `type[Any]`, on the other hand, represents a type of unknown size or inhabitants, but which
is known to be no larger than the set of possible objects represented by `type`.
```py
from knot_extensions import static_assert, is_assignable_to, Unknown, TypeOf
from typing import Any
static_assert(is_assignable_to(type, type))
static_assert(is_assignable_to(type[object], type[object]))
static_assert(is_assignable_to(type, type[object]))
static_assert(is_assignable_to(type[object], type))
static_assert(is_assignable_to(type[str], type[object]))
static_assert(is_assignable_to(TypeOf[str], type[object]))
static_assert(is_assignable_to(type[str], type))
static_assert(is_assignable_to(TypeOf[str], type))
static_assert(is_assignable_to(type[str], type[str]))
static_assert(is_assignable_to(TypeOf[str], type[str]))
static_assert(not is_assignable_to(TypeOf[int], type[str]))
static_assert(not is_assignable_to(type, type[str]))
static_assert(not is_assignable_to(type[object], type[str]))
static_assert(is_assignable_to(type[Any], type[Any]))
static_assert(is_assignable_to(type[Any], type[object]))
static_assert(is_assignable_to(type[object], type[Any]))
static_assert(is_assignable_to(type, type[Any]))
static_assert(is_assignable_to(type[Any], type[str]))
static_assert(is_assignable_to(type[str], type[Any]))
static_assert(is_assignable_to(TypeOf[str], type[Any]))
static_assert(is_assignable_to(type[Unknown], type[Unknown]))
static_assert(is_assignable_to(type[Unknown], type[object]))
static_assert(is_assignable_to(type[object], type[Unknown]))
static_assert(is_assignable_to(type, type[Unknown]))
static_assert(is_assignable_to(type[Unknown], type[str]))
static_assert(is_assignable_to(type[str], type[Unknown]))
static_assert(is_assignable_to(TypeOf[str], type[Unknown]))
static_assert(is_assignable_to(type[Unknown], type[Any]))
static_assert(is_assignable_to(type[Any], type[Unknown]))
static_assert(not is_assignable_to(object, type[Any]))
static_assert(not is_assignable_to(str, type[Any]))
class Meta(type): ...
static_assert(is_assignable_to(type[Any], Meta))
static_assert(is_assignable_to(type[Unknown], Meta))
static_assert(is_assignable_to(Meta, type[Any]))
static_assert(is_assignable_to(Meta, type[Unknown]))
```
## Tuple types
```py
from knot_extensions import static_assert, is_assignable_to
from typing import Literal, Any
static_assert(is_assignable_to(tuple[()], tuple[()]))
static_assert(is_assignable_to(tuple[int], tuple[int]))
static_assert(is_assignable_to(tuple[int], tuple[Any]))
static_assert(is_assignable_to(tuple[Any], tuple[int]))
static_assert(is_assignable_to(tuple[int, str], tuple[int, str]))
static_assert(is_assignable_to(tuple[Literal[1], Literal[2]], tuple[int, int]))
static_assert(is_assignable_to(tuple[Any, Literal[2]], tuple[int, int]))
static_assert(is_assignable_to(tuple[Literal[1], Any], tuple[int, int]))
static_assert(not is_assignable_to(tuple[()], tuple[int]))
static_assert(not is_assignable_to(tuple[int], tuple[str]))
static_assert(not is_assignable_to(tuple[int], tuple[int, str]))
static_assert(not is_assignable_to(tuple[int, str], tuple[int]))
static_assert(not is_assignable_to(tuple[int, int], tuple[Literal[1], int]))
static_assert(not is_assignable_to(tuple[Any, Literal[2]], tuple[int, str]))
```
## Union types
```py
from knot_extensions import static_assert, is_assignable_to, Unknown
from typing import Literal, Any
static_assert(is_assignable_to(int, int | str))
static_assert(is_assignable_to(str, int | str))
static_assert(is_assignable_to(int | str, int | str))
static_assert(is_assignable_to(str | int, int | str))
static_assert(is_assignable_to(Literal[1], int | str))
static_assert(is_assignable_to(Literal[1], Unknown | str))
static_assert(is_assignable_to(Literal[1] | Literal[2], Literal[1] | Literal[2]))
static_assert(is_assignable_to(Literal[1] | Literal[2], int))
static_assert(is_assignable_to(Literal[1] | None, int | None))
static_assert(is_assignable_to(Any, int | str))
static_assert(is_assignable_to(Any | int, int))
static_assert(is_assignable_to(str, int | Any))
static_assert(not is_assignable_to(int | None, int))
static_assert(not is_assignable_to(int | None, str | None))
static_assert(not is_assignable_to(Literal[1] | None, int))
static_assert(not is_assignable_to(Literal[1] | None, str | None))
static_assert(not is_assignable_to(Any | int | str, int))
```
## Intersection types
```py
from knot_extensions import static_assert, is_assignable_to, Intersection, Not
from typing_extensions import Any, Literal
class Parent: ...
class Child1(Parent): ...
class Child2(Parent): ...
class Grandchild(Child1, Child2): ...
class Unrelated: ...
static_assert(is_assignable_to(Intersection[Child1, Child2], Child1))
static_assert(is_assignable_to(Intersection[Child1, Child2], Child2))
static_assert(is_assignable_to(Intersection[Child1, Child2], Parent))
static_assert(is_assignable_to(Intersection[Child1, Parent], Parent))
static_assert(is_assignable_to(Intersection[Parent, Unrelated], Parent))
static_assert(is_assignable_to(Intersection[Child1, Unrelated], Child1))
static_assert(is_assignable_to(Intersection[Child1, Not[Child2]], Child1))
static_assert(is_assignable_to(Intersection[Child1, Not[Child2]], Parent))
static_assert(is_assignable_to(Intersection[Child1, Not[Grandchild]], Parent))
static_assert(is_assignable_to(Intersection[Child1, Child2], Intersection[Child1, Child2]))
static_assert(is_assignable_to(Intersection[Child1, Child2], Intersection[Child2, Child1]))
static_assert(is_assignable_to(Grandchild, Intersection[Child1, Child2]))
static_assert(not is_assignable_to(Parent, Intersection[Parent, Unrelated]))
static_assert(not is_assignable_to(int, Intersection[int, Not[Literal[1]]]))
static_assert(not is_assignable_to(int, Not[int]))
static_assert(not is_assignable_to(int, Not[Literal[1]]))
static_assert(not is_assignable_to(Intersection[Any, Parent], Unrelated))
# TODO: The following assertions should not fail (see https://github.com/astral-sh/ruff/issues/14899)
# error: [static-assert-error]
static_assert(is_assignable_to(Intersection[Any, int], int))
# error: [static-assert-error]
static_assert(is_assignable_to(Intersection[Unrelated, Any], Intersection[Unrelated, Any]))
# error: [static-assert-error]
static_assert(is_assignable_to(Intersection[Unrelated, Any], Intersection[Unrelated, Not[Any]]))
# error: [static-assert-error]
static_assert(is_assignable_to(Intersection[Unrelated, Any], Not[tuple[Unrelated, Any]]))
```
## General properties
See also: our property tests in `property_tests.rs`.
### Everything is assignable to `object`
`object` is Python's top type; the set of all possible objects at runtime:
```py
from knot_extensions import static_assert, is_assignable_to, Unknown
from typing import Literal, Any
static_assert(is_assignable_to(str, object))
static_assert(is_assignable_to(Literal[1], object))
static_assert(is_assignable_to(object, object))
static_assert(is_assignable_to(type, object))
static_assert(is_assignable_to(Any, object))
static_assert(is_assignable_to(Unknown, object))
static_assert(is_assignable_to(type[object], object))
static_assert(is_assignable_to(type[str], object))
static_assert(is_assignable_to(type[Any], object))
```
### Every type is assignable to `Any` / `Unknown`
`Any` and `Unknown` are gradual types. They could materialize to any given type at runtime, and so
any type is assignable to them:
```py
from knot_extensions import static_assert, is_assignable_to, Unknown
from typing import Literal, Any
static_assert(is_assignable_to(str, Any))
static_assert(is_assignable_to(Literal[1], Any))
static_assert(is_assignable_to(object, Any))
static_assert(is_assignable_to(type, Any))
static_assert(is_assignable_to(Any, Any))
static_assert(is_assignable_to(Unknown, Any))
static_assert(is_assignable_to(type[object], Any))
static_assert(is_assignable_to(type[str], Any))
static_assert(is_assignable_to(type[Any], Any))
static_assert(is_assignable_to(str, Unknown))
static_assert(is_assignable_to(Literal[1], Unknown))
static_assert(is_assignable_to(object, Unknown))
static_assert(is_assignable_to(type, Unknown))
static_assert(is_assignable_to(Any, Unknown))
static_assert(is_assignable_to(Unknown, Unknown))
static_assert(is_assignable_to(type[object], Unknown))
static_assert(is_assignable_to(type[str], Unknown))
static_assert(is_assignable_to(type[Any], Unknown))
```
### `Never` is assignable to every type
`Never` is Python's bottom type: the empty set, a type with no inhabitants. It is therefore
assignable to any arbitrary type.
```py
from knot_extensions import static_assert, is_assignable_to, Unknown
from typing_extensions import Never, Any
static_assert(is_assignable_to(Never, str))
static_assert(is_assignable_to(Never, Literal[1]))
static_assert(is_assignable_to(Never, object))
static_assert(is_assignable_to(Never, type))
static_assert(is_assignable_to(Never, Any))
static_assert(is_assignable_to(Never, Unknown))
static_assert(is_assignable_to(Never, type[object]))
static_assert(is_assignable_to(Never, type[str]))
static_assert(is_assignable_to(Never, type[Any]))
```
[typing documentation]: https://typing.readthedocs.io/en/latest/spec/concepts.html#the-assignable-to-or-consistent-subtyping-relation

View File

@@ -0,0 +1,35 @@
# Equivalence relation
`is_equivalent_to` implements [the equivalence relation] for fully static types.
Two types `A` and `B` are equivalent iff `A` is a subtype of `B` and `B` is a subtype of `A`.
## Basic
```py
from typing import Any
from typing_extensions import Literal
from knot_extensions import Unknown, is_equivalent_to, static_assert
static_assert(is_equivalent_to(Literal[1, 2], Literal[1, 2]))
static_assert(is_equivalent_to(type[object], type))
static_assert(not is_equivalent_to(Any, Any))
static_assert(not is_equivalent_to(Unknown, Unknown))
static_assert(not is_equivalent_to(Any, None))
static_assert(not is_equivalent_to(Literal[1, 2], Literal[1, 0]))
static_assert(not is_equivalent_to(Literal[1, 2], Literal[1, 2, 3]))
```
## Equivalence is commutative
```py
from typing_extensions import Literal
from knot_extensions import is_equivalent_to, static_assert
static_assert(is_equivalent_to(type, type[object]))
static_assert(not is_equivalent_to(Literal[1, 0], Literal[1, 2]))
static_assert(not is_equivalent_to(Literal[1, 2, 3], Literal[1, 2]))
```
[the equivalence relation]: https://typing.readthedocs.io/en/latest/spec/glossary.html#term-equivalent

View File

@@ -0,0 +1,453 @@
# Subtype relation
The `is_subtype_of(S, T)` relation below checks if type `S` is a subtype of type `T`.
A fully static type `S` is a subtype of another fully static type `T` iff the set of values
represented by `S` is a subset of the set of values represented by `T`.
See the [typing documentation] for more information.
## Basic builtin types
- `bool` is a subtype of `int`. This is modeled after Python's runtime behavior, where `int` is a
supertype of `bool` (present in `bool`s bases and MRO).
- `int` is not a subtype of `float`/`complex`, even though `float`/`complex` can be used in place of
`int` in some contexts (see [special case for float and complex]).
```py
from knot_extensions import is_subtype_of, static_assert
static_assert(is_subtype_of(bool, bool))
static_assert(is_subtype_of(bool, int))
static_assert(is_subtype_of(bool, object))
static_assert(is_subtype_of(int, int))
static_assert(is_subtype_of(int, object))
static_assert(is_subtype_of(object, object))
static_assert(not is_subtype_of(int, bool))
static_assert(not is_subtype_of(int, str))
static_assert(not is_subtype_of(object, int))
static_assert(not is_subtype_of(int, float))
static_assert(not is_subtype_of(int, complex))
static_assert(is_subtype_of(TypeError, Exception))
static_assert(is_subtype_of(FloatingPointError, Exception))
```
## Class hierarchies
```py
from knot_extensions import is_subtype_of, static_assert
from typing_extensions import Never
class A: ...
class B1(A): ...
class B2(A): ...
class C(B1, B2): ...
static_assert(is_subtype_of(B1, A))
static_assert(not is_subtype_of(A, B1))
static_assert(is_subtype_of(B2, A))
static_assert(not is_subtype_of(A, B2))
static_assert(not is_subtype_of(B1, B2))
static_assert(not is_subtype_of(B2, B1))
static_assert(is_subtype_of(C, B1))
static_assert(is_subtype_of(C, B2))
static_assert(not is_subtype_of(B1, C))
static_assert(not is_subtype_of(B2, C))
static_assert(is_subtype_of(C, A))
static_assert(not is_subtype_of(A, C))
static_assert(is_subtype_of(Never, A))
static_assert(is_subtype_of(Never, B1))
static_assert(is_subtype_of(Never, B2))
static_assert(is_subtype_of(Never, C))
static_assert(is_subtype_of(A, object))
static_assert(is_subtype_of(B1, object))
static_assert(is_subtype_of(B2, object))
static_assert(is_subtype_of(C, object))
```
## Literal types
```py
from typing_extensions import Literal, LiteralString
from knot_extensions import is_subtype_of, static_assert
# Boolean literals
static_assert(is_subtype_of(Literal[True], bool))
static_assert(is_subtype_of(Literal[True], int))
static_assert(is_subtype_of(Literal[True], object))
# Integer literals
static_assert(is_subtype_of(Literal[1], int))
static_assert(is_subtype_of(Literal[1], object))
static_assert(not is_subtype_of(Literal[1], bool))
# See the note above (or link below) concerning int and float/complex
static_assert(not is_subtype_of(Literal[1], float))
# String literals
static_assert(is_subtype_of(Literal["foo"], LiteralString))
static_assert(is_subtype_of(Literal["foo"], str))
static_assert(is_subtype_of(Literal["foo"], object))
static_assert(is_subtype_of(LiteralString, str))
static_assert(is_subtype_of(LiteralString, object))
# Bytes literals
static_assert(is_subtype_of(Literal[b"foo"], bytes))
static_assert(is_subtype_of(Literal[b"foo"], object))
```
## Tuple types
```py
from knot_extensions import is_subtype_of, static_assert
class A1: ...
class B1(A1): ...
class A2: ...
class B2(A2): ...
class Unrelated: ...
static_assert(is_subtype_of(B1, A1))
static_assert(is_subtype_of(B2, A2))
# Zero-element tuples
static_assert(is_subtype_of(tuple[()], tuple[()]))
static_assert(not is_subtype_of(tuple[()], tuple[Unrelated]))
# One-element tuples
static_assert(is_subtype_of(tuple[B1], tuple[A1]))
static_assert(not is_subtype_of(tuple[B1], tuple[Unrelated]))
static_assert(not is_subtype_of(tuple[B1], tuple[()]))
static_assert(not is_subtype_of(tuple[B1], tuple[A1, Unrelated]))
# Two-element tuples
static_assert(is_subtype_of(tuple[B1, B2], tuple[A1, A2]))
static_assert(not is_subtype_of(tuple[B1, B2], tuple[Unrelated, A2]))
static_assert(not is_subtype_of(tuple[B1, B2], tuple[A1, Unrelated]))
static_assert(not is_subtype_of(tuple[B1, B2], tuple[Unrelated, Unrelated]))
static_assert(not is_subtype_of(tuple[B1, B2], tuple[()]))
static_assert(not is_subtype_of(tuple[B1, B2], tuple[A1]))
static_assert(not is_subtype_of(tuple[B1, B2], tuple[A1, A2, Unrelated]))
static_assert(is_subtype_of(tuple[int], tuple))
```
## Union types
```py
from knot_extensions import is_subtype_of, static_assert
class A: ...
class B1(A): ...
class B2(A): ...
class Unrelated1: ...
class Unrelated2: ...
static_assert(is_subtype_of(B1, A))
static_assert(is_subtype_of(B2, A))
# Union on the right hand side
static_assert(is_subtype_of(B1, A | Unrelated1))
static_assert(is_subtype_of(B1, Unrelated1 | A))
static_assert(not is_subtype_of(B1, Unrelated1 | Unrelated2))
# Union on the left hand side
static_assert(is_subtype_of(B1 | B2, A))
static_assert(is_subtype_of(B1 | B2 | A, object))
static_assert(not is_subtype_of(B1 | Unrelated1, A))
static_assert(not is_subtype_of(Unrelated1 | B1, A))
# Union on both sides
static_assert(is_subtype_of(B1 | bool, A | int))
static_assert(is_subtype_of(B1 | bool, int | A))
static_assert(not is_subtype_of(B1 | bool, Unrelated1 | int))
static_assert(not is_subtype_of(B1 | bool, int | Unrelated1))
# Example: Unions of literals
static_assert(is_subtype_of(Literal[1, 2, 3], int))
static_assert(not is_subtype_of(Literal[1, "two", 3], int))
```
## Intersection types
```py
from typing_extensions import Literal, LiteralString
from knot_extensions import Intersection, Not, is_subtype_of, static_assert
class A: ...
class B1(A): ...
class B2(A): ...
class C(B1, B2): ...
class Unrelated: ...
static_assert(is_subtype_of(B1, A))
static_assert(is_subtype_of(B2, A))
static_assert(is_subtype_of(C, A))
static_assert(is_subtype_of(C, B1))
static_assert(is_subtype_of(C, B2))
# For complements, the subtyping relation is reversed:
static_assert(is_subtype_of(Not[A], Not[B1]))
static_assert(is_subtype_of(Not[A], Not[B2]))
static_assert(is_subtype_of(Not[A], Not[C]))
static_assert(is_subtype_of(Not[B1], Not[C]))
static_assert(is_subtype_of(Not[B2], Not[C]))
# The intersection of two types is a subtype of both:
static_assert(is_subtype_of(Intersection[B1, B2], B1))
static_assert(is_subtype_of(Intersection[B1, B2], B2))
# … and of their common supertype:
static_assert(is_subtype_of(Intersection[B1, B2], A))
# A common subtype of two types is a subtype of their intersection:
static_assert(is_subtype_of(C, Intersection[B1, B2]))
# … but not the other way around:
static_assert(not is_subtype_of(Intersection[B1, B2], C))
# "Removing" B1 from A leaves a subtype of A.
static_assert(is_subtype_of(Intersection[A, Not[B1]], A))
static_assert(is_subtype_of(Intersection[A, Not[B1]], Not[B1]))
# B1 and B2 are not disjoint, so this is not true:
static_assert(not is_subtype_of(B2, Intersection[A, Not[B1]]))
# … but for two disjoint subtypes, it is:
static_assert(is_subtype_of(Literal[2], Intersection[int, Not[Literal[1]]]))
# A and Unrelated are not related, so this is not true:
static_assert(not is_subtype_of(Intersection[A, Not[B1]], Not[Unrelated]))
# … but for a disjoint type like `None`, it is:
static_assert(is_subtype_of(Intersection[A, Not[B1]], Not[None]))
# Complements of types are still subtypes of `object`:
static_assert(is_subtype_of(Not[A], object))
# More examples:
static_assert(is_subtype_of(type[str], Not[None]))
static_assert(is_subtype_of(Not[LiteralString], object))
static_assert(not is_subtype_of(Intersection[int, Not[Literal[2]]], Intersection[int, Not[Literal[3]]]))
static_assert(not is_subtype_of(Not[Literal[2]], Not[Literal[3]]))
static_assert(not is_subtype_of(Not[Literal[2]], Not[int]))
static_assert(not is_subtype_of(int, Not[Literal[3]]))
static_assert(not is_subtype_of(Literal[1], Intersection[int, Not[Literal[1]]]))
```
## Special types
### `Never`
`Never` is a subtype of all types.
```py
from typing_extensions import Literal, Never
from knot_extensions import AlwaysTruthy, AlwaysFalsy, is_subtype_of, static_assert
static_assert(is_subtype_of(Never, Never))
static_assert(is_subtype_of(Never, Literal[True]))
static_assert(is_subtype_of(Never, bool))
static_assert(is_subtype_of(Never, int))
static_assert(is_subtype_of(Never, object))
static_assert(is_subtype_of(Never, AlwaysTruthy))
static_assert(is_subtype_of(Never, AlwaysFalsy))
```
### `AlwaysTruthy` and `AlwaysFalsy`
```py
from knot_extensions import AlwaysTruthy, AlwaysFalsy, is_subtype_of, static_assert
static_assert(is_subtype_of(Literal[1], AlwaysTruthy))
static_assert(is_subtype_of(Literal[0], AlwaysFalsy))
static_assert(is_subtype_of(AlwaysTruthy, object))
static_assert(is_subtype_of(AlwaysFalsy, object))
static_assert(not is_subtype_of(Literal[1], AlwaysFalsy))
static_assert(not is_subtype_of(Literal[0], AlwaysTruthy))
static_assert(not is_subtype_of(str, AlwaysTruthy))
static_assert(not is_subtype_of(str, AlwaysFalsy))
```
### Module literals
```py
from types import ModuleType
from knot_extensions import TypeOf, is_subtype_of, static_assert
from typing_extensions import assert_type
import typing
assert_type(typing, TypeOf[typing])
static_assert(is_subtype_of(TypeOf[typing], ModuleType))
```
### Slice literals
```py
from knot_extensions import TypeOf, is_subtype_of, static_assert
static_assert(is_subtype_of(TypeOf[1:2:3], slice))
```
### Special forms
```py
from typing import _SpecialForm
from knot_extensions import TypeOf, is_subtype_of, static_assert
static_assert(is_subtype_of(TypeOf[Literal], _SpecialForm))
static_assert(is_subtype_of(TypeOf[Literal], object))
static_assert(not is_subtype_of(_SpecialForm, TypeOf[Literal]))
```
## Class literal types and `type[…]`
### Basic
```py
from typing import _SpecialForm
from typing_extensions import Literal, assert_type
from knot_extensions import TypeOf, is_subtype_of, static_assert
class Meta(type): ...
class HasCustomMetaclass(metaclass=Meta): ...
type LiteralBool = TypeOf[bool]
type LiteralInt = TypeOf[int]
type LiteralStr = TypeOf[str]
type LiteralObject = TypeOf[object]
assert_type(bool, LiteralBool)
assert_type(int, LiteralInt)
assert_type(str, LiteralStr)
assert_type(object, LiteralObject)
# bool
static_assert(is_subtype_of(LiteralBool, LiteralBool))
static_assert(is_subtype_of(LiteralBool, type[bool]))
static_assert(is_subtype_of(LiteralBool, type[int]))
static_assert(is_subtype_of(LiteralBool, type[object]))
static_assert(is_subtype_of(LiteralBool, type))
static_assert(is_subtype_of(LiteralBool, object))
static_assert(not is_subtype_of(LiteralBool, LiteralInt))
static_assert(not is_subtype_of(LiteralBool, LiteralObject))
static_assert(not is_subtype_of(LiteralBool, bool))
static_assert(not is_subtype_of(type, type[bool]))
# int
static_assert(is_subtype_of(LiteralInt, LiteralInt))
static_assert(is_subtype_of(LiteralInt, type[int]))
static_assert(is_subtype_of(LiteralInt, type[object]))
static_assert(is_subtype_of(LiteralInt, type))
static_assert(is_subtype_of(LiteralInt, object))
static_assert(not is_subtype_of(LiteralInt, LiteralObject))
static_assert(not is_subtype_of(LiteralInt, int))
static_assert(not is_subtype_of(type, type[int]))
# LiteralString
static_assert(is_subtype_of(LiteralStr, type[str]))
static_assert(is_subtype_of(LiteralStr, type))
static_assert(is_subtype_of(LiteralStr, type[object]))
static_assert(not is_subtype_of(type[str], LiteralStr))
# custom meta classes
type LiteralHasCustomMetaclass = TypeOf[HasCustomMetaclass]
static_assert(is_subtype_of(LiteralHasCustomMetaclass, Meta))
static_assert(is_subtype_of(Meta, type[object]))
static_assert(is_subtype_of(Meta, type))
static_assert(not is_subtype_of(Meta, type[type]))
```
### Unions of class literals
```py
from typing_extensions import assert_type
from knot_extensions import TypeOf, is_subtype_of, static_assert
class Base: ...
class Derived(Base): ...
class Unrelated: ...
type LiteralBase = TypeOf[Base]
type LiteralDerived = TypeOf[Derived]
type LiteralUnrelated = TypeOf[Unrelated]
assert_type(Base, LiteralBase)
assert_type(Derived, LiteralDerived)
assert_type(Unrelated, LiteralUnrelated)
static_assert(is_subtype_of(LiteralBase, type))
static_assert(is_subtype_of(LiteralBase, object))
static_assert(is_subtype_of(LiteralBase, type[Base]))
static_assert(is_subtype_of(LiteralDerived, type[Base]))
static_assert(is_subtype_of(LiteralDerived, type[Derived]))
static_assert(not is_subtype_of(LiteralBase, type[Derived]))
static_assert(is_subtype_of(type[Derived], type[Base]))
static_assert(is_subtype_of(LiteralBase | LiteralUnrelated, type))
static_assert(is_subtype_of(LiteralBase | LiteralUnrelated, object))
```
## Non-fully-static types
`Any`, `Unknown`, `Todo` and derivatives thereof do not participate in subtyping.
```py
from knot_extensions import Unknown, is_subtype_of, static_assert, Intersection
from typing_extensions import Any
static_assert(not is_subtype_of(Any, Any))
static_assert(not is_subtype_of(Any, int))
static_assert(not is_subtype_of(int, Any))
static_assert(not is_subtype_of(Any, object))
static_assert(not is_subtype_of(object, Any))
static_assert(not is_subtype_of(int, Any | int))
static_assert(not is_subtype_of(Intersection[Any, int], int))
static_assert(not is_subtype_of(tuple[int, int], tuple[int, Any]))
# The same for `Unknown`:
static_assert(not is_subtype_of(Unknown, Unknown))
static_assert(not is_subtype_of(Unknown, int))
static_assert(not is_subtype_of(int, Unknown))
static_assert(not is_subtype_of(Unknown, object))
static_assert(not is_subtype_of(object, Unknown))
static_assert(not is_subtype_of(int, Unknown | int))
static_assert(not is_subtype_of(Intersection[Unknown, int], int))
static_assert(not is_subtype_of(tuple[int, int], tuple[int, Unknown]))
```
[special case for float and complex]: https://typing.readthedocs.io/en/latest/spec/special-types.html#special-cases-for-float-and-complex
[typing documentation]: https://typing.readthedocs.io/en/latest/spec/concepts.html#subtype-supertype-and-type-equivalence

View File

@@ -0,0 +1,33 @@
# Tuples containing `Never`
A heterogeneous `tuple[…]` type that contains `Never` as a type argument simplifies to `Never`. One
way to think about this is the following: in order to construct a tuple, you need to have an object
of every element type. But since there is no object of type `Never`, you cannot construct the tuple.
Such a tuple type is therefore uninhabited and equivalent to `Never`.
In the language of algebraic data types, a tuple type is a product type and `Never` acts like the
zero element in multiplication, similar to how a Cartesian product with the empty set is the empty
set.
```py
from knot_extensions import static_assert, is_equivalent_to
from typing_extensions import Never, NoReturn
static_assert(is_equivalent_to(Never, tuple[Never]))
static_assert(is_equivalent_to(Never, tuple[Never, int]))
static_assert(is_equivalent_to(Never, tuple[int, Never]))
static_assert(is_equivalent_to(Never, tuple[int, Never, str]))
static_assert(is_equivalent_to(Never, tuple[int, tuple[str, Never]]))
static_assert(is_equivalent_to(Never, tuple[tuple[str, Never], int]))
# The empty tuple is *not* equivalent to Never!
static_assert(not is_equivalent_to(Never, tuple[()]))
# NoReturn is just a different spelling of Never, so the same is true for NoReturn
static_assert(is_equivalent_to(NoReturn, tuple[NoReturn]))
static_assert(is_equivalent_to(NoReturn, tuple[NoReturn, int]))
static_assert(is_equivalent_to(NoReturn, tuple[int, NoReturn]))
static_assert(is_equivalent_to(NoReturn, tuple[int, NoReturn, str]))
static_assert(is_equivalent_to(NoReturn, tuple[int, tuple[str, NoReturn]]))
static_assert(is_equivalent_to(NoReturn, tuple[tuple[str, NoReturn], int]))
```

View File

@@ -0,0 +1,169 @@
# Custom unary operations
## Class instances
```py
class Yes:
def __pos__(self) -> bool:
return False
def __neg__(self) -> str:
return "negative"
def __invert__(self) -> int:
return 17
class Sub(Yes): ...
class No: ...
reveal_type(+Yes()) # revealed: bool
reveal_type(-Yes()) # revealed: str
reveal_type(~Yes()) # revealed: int
reveal_type(+Sub()) # revealed: bool
reveal_type(-Sub()) # revealed: str
reveal_type(~Sub()) # revealed: int
# error: [unsupported-operator] "Unary operator `+` is unsupported for type `No`"
reveal_type(+No()) # revealed: Unknown
# error: [unsupported-operator] "Unary operator `-` is unsupported for type `No`"
reveal_type(-No()) # revealed: Unknown
# error: [unsupported-operator] "Unary operator `~` is unsupported for type `No`"
reveal_type(~No()) # revealed: Unknown
```
## Classes
Dunder methods defined in a class are available to instances of that class, but not to the class
itself. (For these operators to work on the class itself, they would have to be defined on the
class's type, i.e. `type`.)
```py
class Yes:
def __pos__(self) -> bool:
return False
def __neg__(self) -> str:
return "negative"
def __invert__(self) -> int:
return 17
class Sub(Yes): ...
class No: ...
# error: [unsupported-operator] "Unary operator `+` is unsupported for type `Literal[Yes]`"
reveal_type(+Yes) # revealed: Unknown
# error: [unsupported-operator] "Unary operator `-` is unsupported for type `Literal[Yes]`"
reveal_type(-Yes) # revealed: Unknown
# error: [unsupported-operator] "Unary operator `~` is unsupported for type `Literal[Yes]`"
reveal_type(~Yes) # revealed: Unknown
# error: [unsupported-operator] "Unary operator `+` is unsupported for type `Literal[Sub]`"
reveal_type(+Sub) # revealed: Unknown
# error: [unsupported-operator] "Unary operator `-` is unsupported for type `Literal[Sub]`"
reveal_type(-Sub) # revealed: Unknown
# error: [unsupported-operator] "Unary operator `~` is unsupported for type `Literal[Sub]`"
reveal_type(~Sub) # revealed: Unknown
# error: [unsupported-operator] "Unary operator `+` is unsupported for type `Literal[No]`"
reveal_type(+No) # revealed: Unknown
# error: [unsupported-operator] "Unary operator `-` is unsupported for type `Literal[No]`"
reveal_type(-No) # revealed: Unknown
# error: [unsupported-operator] "Unary operator `~` is unsupported for type `Literal[No]`"
reveal_type(~No) # revealed: Unknown
```
## Function literals
```py
def f():
pass
# error: [unsupported-operator] "Unary operator `+` is unsupported for type `Literal[f]`"
reveal_type(+f) # revealed: Unknown
# error: [unsupported-operator] "Unary operator `-` is unsupported for type `Literal[f]`"
reveal_type(-f) # revealed: Unknown
# error: [unsupported-operator] "Unary operator `~` is unsupported for type `Literal[f]`"
reveal_type(~f) # revealed: Unknown
```
## Subclass
```py
class Yes:
def __pos__(self) -> bool:
return False
def __neg__(self) -> str:
return "negative"
def __invert__(self) -> int:
return 17
class Sub(Yes): ...
class No: ...
def yes() -> type[Yes]:
return Yes
def sub() -> type[Sub]:
return Sub
def no() -> type[No]:
return No
# error: [unsupported-operator] "Unary operator `+` is unsupported for type `type[Yes]`"
reveal_type(+yes()) # revealed: Unknown
# error: [unsupported-operator] "Unary operator `-` is unsupported for type `type[Yes]`"
reveal_type(-yes()) # revealed: Unknown
# error: [unsupported-operator] "Unary operator `~` is unsupported for type `type[Yes]`"
reveal_type(~yes()) # revealed: Unknown
# error: [unsupported-operator] "Unary operator `+` is unsupported for type `type[Sub]`"
reveal_type(+sub()) # revealed: Unknown
# error: [unsupported-operator] "Unary operator `-` is unsupported for type `type[Sub]`"
reveal_type(-sub()) # revealed: Unknown
# error: [unsupported-operator] "Unary operator `~` is unsupported for type `type[Sub]`"
reveal_type(~sub()) # revealed: Unknown
# error: [unsupported-operator] "Unary operator `+` is unsupported for type `type[No]`"
reveal_type(+no()) # revealed: Unknown
# error: [unsupported-operator] "Unary operator `-` is unsupported for type `type[No]`"
reveal_type(-no()) # revealed: Unknown
# error: [unsupported-operator] "Unary operator `~` is unsupported for type `type[No]`"
reveal_type(~no()) # revealed: Unknown
```
## Metaclass
```py
class Meta(type):
def __pos__(self) -> bool:
return False
def __neg__(self) -> str:
return "negative"
def __invert__(self) -> int:
return 17
class Yes(metaclass=Meta): ...
class Sub(Yes): ...
class No: ...
reveal_type(+Yes) # revealed: bool
reveal_type(-Yes) # revealed: str
reveal_type(~Yes) # revealed: int
reveal_type(+Sub) # revealed: bool
reveal_type(-Sub) # revealed: str
reveal_type(~Sub) # revealed: int
# error: [unsupported-operator] "Unary operator `+` is unsupported for type `Literal[No]`"
reveal_type(+No) # revealed: Unknown
# error: [unsupported-operator] "Unary operator `-` is unsupported for type `Literal[No]`"
reveal_type(-No) # revealed: Unknown
# error: [unsupported-operator] "Unary operator `~` is unsupported for type `Literal[No]`"
reveal_type(~No) # revealed: Unknown
```

View File

@@ -0,0 +1,143 @@
# Union types
This test suite covers certain basic properties and simplification strategies for union types.
## Basic unions
```py
from typing import Literal
def _(u1: int | str, u2: Literal[0] | Literal[1]) -> None:
reveal_type(u1) # revealed: int | str
reveal_type(u2) # revealed: Literal[0, 1]
```
## Duplicate elements are collapsed
```py
def _(u1: int | int | str, u2: int | str | int) -> None:
reveal_type(u1) # revealed: int | str
reveal_type(u2) # revealed: int | str
```
## `Never` is removed
`Never` is an empty set, a type with no inhabitants. Its presence in a union is always redundant,
and so we eagerly simplify it away. `NoReturn` is equivalent to `Never`.
```py
from typing_extensions import Never, NoReturn
def never(u1: int | Never, u2: int | Never | str) -> None:
reveal_type(u1) # revealed: int
reveal_type(u2) # revealed: int | str
def noreturn(u1: int | NoReturn, u2: int | NoReturn | str) -> None:
reveal_type(u1) # revealed: int
reveal_type(u2) # revealed: int | str
```
## Flattening of nested unions
```py
from typing import Literal
def _(
u1: (int | str) | bytes,
u2: int | (str | bytes),
u3: int | (str | (bytes | complex)),
) -> None:
reveal_type(u1) # revealed: int | str | bytes
reveal_type(u2) # revealed: int | str | bytes
reveal_type(u3) # revealed: int | str | bytes | complex
```
## Simplification using subtyping
The type `S | T` can be simplified to `T` if `S` is a subtype of `T`:
```py
from typing_extensions import Literal, LiteralString
def _(
u1: str | LiteralString, u2: LiteralString | str, u3: Literal["a"] | str | LiteralString, u4: str | bytes | LiteralString
) -> None:
reveal_type(u1) # revealed: str
reveal_type(u2) # revealed: str
reveal_type(u3) # revealed: str
reveal_type(u4) # revealed: str | bytes
```
## Boolean literals
The union `Literal[True] | Literal[False]` is exactly equivalent to `bool`:
```py
from typing import Literal
def _(
u1: Literal[True, False],
u2: bool | Literal[True],
u3: Literal[True] | bool,
u4: Literal[True] | Literal[True, 17],
u5: Literal[True, False, True, 17],
) -> None:
reveal_type(u1) # revealed: bool
reveal_type(u2) # revealed: bool
reveal_type(u3) # revealed: bool
reveal_type(u4) # revealed: Literal[True, 17]
reveal_type(u5) # revealed: bool | Literal[17]
```
## Do not erase `Unknown`
```py
from knot_extensions import Unknown
def _(u1: Unknown | str, u2: str | Unknown) -> None:
reveal_type(u1) # revealed: Unknown | str
reveal_type(u2) # revealed: str | Unknown
```
## Collapse multiple `Unknown`s
Since `Unknown` is a gradual type, it is not a subtype of anything, but multiple `Unknown`s in a
union are still redundant:
```py
from knot_extensions import Unknown
def _(u1: Unknown | Unknown | str, u2: Unknown | str | Unknown, u3: str | Unknown | Unknown) -> None:
reveal_type(u1) # revealed: Unknown | str
reveal_type(u2) # revealed: Unknown | str
reveal_type(u3) # revealed: str | Unknown
```
## Subsume multiple elements
Simplifications still apply when `Unknown` is present.
```py
from knot_extensions import Unknown
def _(u1: str | Unknown | int | object):
reveal_type(u1) # revealed: Unknown | object
```
## Union of intersections
We can simplify unions of intersections:
```py
from knot_extensions import Intersection, Not
class P: ...
class Q: ...
def _(
i1: Intersection[P, Q] | Intersection[P, Q],
i2: Intersection[P, Q] | Intersection[Q, P],
) -> None:
reveal_type(i1) # revealed: P & Q
reveal_type(i2) # revealed: P & Q
```

View File

@@ -61,7 +61,7 @@ reveal_type(c) # revealed: Literal[4]
### Uneven unpacking (1)
```py
# TODO: Add diagnostic (there aren't enough values to unpack)
# error: [invalid-assignment] "Not enough values to unpack (expected 3, got 2)"
(a, b, c) = (1, 2)
reveal_type(a) # revealed: Literal[1]
reveal_type(b) # revealed: Literal[2]
@@ -71,7 +71,7 @@ reveal_type(c) # revealed: Unknown
### Uneven unpacking (2)
```py
# TODO: Add diagnostic (too many values to unpack)
# error: [invalid-assignment] "Too many values to unpack (expected 2, got 3)"
(a, b) = (1, 2, 3)
reveal_type(a) # revealed: Literal[1]
reveal_type(b) # revealed: Literal[2]
@@ -80,7 +80,7 @@ reveal_type(b) # revealed: Literal[2]
### Starred expression (1)
```py
# TODO: Add diagnostic (need more values to unpack)
# error: [invalid-assignment] "Not enough values to unpack (expected 3 or more, got 2)"
[a, *b, c, d] = (1, 2)
reveal_type(a) # revealed: Literal[1]
# TODO: Should be list[Any] once support for assigning to starred expression is added
@@ -133,7 +133,7 @@ reveal_type(c) # revealed: @Todo(starred unpacking)
### Starred expression (6)
```py
# TODO: Add diagnostic (need more values to unpack)
# error: [invalid-assignment] "Not enough values to unpack (expected 5 or more, got 1)"
(a, b, c, *d, e, f) = (1,)
reveal_type(a) # revealed: Literal[1]
reveal_type(b) # revealed: Unknown
@@ -199,7 +199,7 @@ reveal_type(b) # revealed: LiteralString
### Uneven unpacking (1)
```py
# TODO: Add diagnostic (there aren't enough values to unpack)
# error: [invalid-assignment] "Not enough values to unpack (expected 3, got 2)"
a, b, c = "ab"
reveal_type(a) # revealed: LiteralString
reveal_type(b) # revealed: LiteralString
@@ -209,7 +209,7 @@ reveal_type(c) # revealed: Unknown
### Uneven unpacking (2)
```py
# TODO: Add diagnostic (too many values to unpack)
# error: [invalid-assignment] "Too many values to unpack (expected 2, got 3)"
a, b = "abc"
reveal_type(a) # revealed: LiteralString
reveal_type(b) # revealed: LiteralString
@@ -218,7 +218,7 @@ reveal_type(b) # revealed: LiteralString
### Starred expression (1)
```py
# TODO: Add diagnostic (need more values to unpack)
# error: [invalid-assignment] "Not enough values to unpack (expected 3 or more, got 2)"
(a, *b, c, d) = "ab"
reveal_type(a) # revealed: LiteralString
# TODO: Should be list[LiteralString] once support for assigning to starred expression is added
@@ -271,7 +271,7 @@ reveal_type(c) # revealed: @Todo(starred unpacking)
### Unicode
```py
# TODO: Add diagnostic (need more values to unpack)
# error: [invalid-assignment] "Not enough values to unpack (expected 2, got 1)"
(a, b) = "é"
reveal_type(a) # revealed: LiteralString
@@ -281,7 +281,7 @@ reveal_type(b) # revealed: Unknown
### Unicode escape (1)
```py
# TODO: Add diagnostic (need more values to unpack)
# error: [invalid-assignment] "Not enough values to unpack (expected 2, got 1)"
(a, b) = "\u9E6C"
reveal_type(a) # revealed: LiteralString
@@ -291,7 +291,7 @@ reveal_type(b) # revealed: Unknown
### Unicode escape (2)
```py
# TODO: Add diagnostic (need more values to unpack)
# error: [invalid-assignment] "Not enough values to unpack (expected 2, got 1)"
(a, b) = "\U0010FFFF"
reveal_type(a) # revealed: LiteralString
@@ -306,3 +306,273 @@ reveal_type(b) # revealed: Unknown
reveal_type(a) # revealed: LiteralString
reveal_type(b) # revealed: LiteralString
```
## Union
### Same types
Union of two tuples of equal length and each element is of the same type.
```py
def _(arg: tuple[int, int] | tuple[int, int]):
(a, b) = arg
reveal_type(a) # revealed: int
reveal_type(b) # revealed: int
```
### Mixed types (1)
Union of two tuples of equal length and one element differs in its type.
```py
def _(arg: tuple[int, int] | tuple[int, str]):
a, b = arg
reveal_type(a) # revealed: int
reveal_type(b) # revealed: int | str
```
### Mixed types (2)
Union of two tuples of equal length and both the element types are different.
```py
def _(arg: tuple[int, str] | tuple[str, int]):
a, b = arg
reveal_type(a) # revealed: int | str
reveal_type(b) # revealed: str | int
```
### Mixed types (3)
Union of three tuples of equal length and various combination of element types:
1. All same types
1. One different type
1. All different types
```py
def _(arg: tuple[int, int, int] | tuple[int, str, bytes] | tuple[int, int, str]):
a, b, c = arg
reveal_type(a) # revealed: int
reveal_type(b) # revealed: int | str
reveal_type(c) # revealed: int | bytes | str
```
### Nested
```py
def _(arg: tuple[int, tuple[str, bytes]] | tuple[tuple[int, bytes], Literal["ab"]]):
a, (b, c) = arg
reveal_type(a) # revealed: int | tuple[int, bytes]
reveal_type(b) # revealed: str
reveal_type(c) # revealed: bytes | LiteralString
```
### Starred expression
```py
def _(arg: tuple[int, bytes, int] | tuple[int, int, str, int, bytes]):
a, *b, c = arg
reveal_type(a) # revealed: int
# TODO: Should be `list[bytes | int | str]`
reveal_type(b) # revealed: @Todo(starred unpacking)
reveal_type(c) # revealed: int | bytes
```
### Size mismatch (1)
```py
def _(arg: tuple[int, bytes, int] | tuple[int, int, str, int, bytes]):
# error: [invalid-assignment] "Too many values to unpack (expected 2, got 3)"
# error: [invalid-assignment] "Too many values to unpack (expected 2, got 5)"
a, b = arg
reveal_type(a) # revealed: int
reveal_type(b) # revealed: bytes | int
```
### Size mismatch (2)
```py
def _(arg: tuple[int, bytes] | tuple[int, str]):
# error: [invalid-assignment] "Not enough values to unpack (expected 3, got 2)"
# error: [invalid-assignment] "Not enough values to unpack (expected 3, got 2)"
a, b, c = arg
reveal_type(a) # revealed: int
reveal_type(b) # revealed: bytes | str
reveal_type(c) # revealed: Unknown
```
### Same literal types
```py
def _(flag: bool):
if flag:
value = (1, 2)
else:
value = (3, 4)
a, b = value
reveal_type(a) # revealed: Literal[1, 3]
reveal_type(b) # revealed: Literal[2, 4]
```
### Mixed literal types
```py
def _(flag: bool):
if flag:
value = (1, 2)
else:
value = ("a", "b")
a, b = value
reveal_type(a) # revealed: Literal[1, "a"]
reveal_type(b) # revealed: Literal[2, "b"]
```
### Typing literal
```py
from typing import Literal
def _(arg: tuple[int, int] | Literal["ab"]):
a, b = arg
reveal_type(a) # revealed: int | LiteralString
reveal_type(b) # revealed: int | LiteralString
```
### Custom iterator (1)
```py
class Iterator:
def __next__(self) -> tuple[int, int] | tuple[int, str]:
return (1, 2)
class Iterable:
def __iter__(self) -> Iterator:
return Iterator()
((a, b), c) = Iterable()
reveal_type(a) # revealed: int
reveal_type(b) # revealed: int | str
reveal_type(c) # revealed: tuple[int, int] | tuple[int, str]
```
### Custom iterator (2)
```py
class Iterator:
def __next__(self) -> bytes:
return b""
class Iterable:
def __iter__(self) -> Iterator:
return Iterator()
def _(arg: tuple[int, str] | Iterable):
a, b = arg
reveal_type(a) # revealed: int | bytes
reveal_type(b) # revealed: str | bytes
```
## For statement
Unpacking in a `for` statement.
### Same types
```py
def _(arg: tuple[tuple[int, int], tuple[int, int]]):
for a, b in arg:
reveal_type(a) # revealed: int
reveal_type(b) # revealed: int
```
### Mixed types (1)
```py
def _(arg: tuple[tuple[int, int], tuple[int, str]]):
for a, b in arg:
reveal_type(a) # revealed: int
reveal_type(b) # revealed: int | str
```
### Mixed types (2)
```py
def _(arg: tuple[tuple[int, str], tuple[str, int]]):
for a, b in arg:
reveal_type(a) # revealed: int | str
reveal_type(b) # revealed: str | int
```
### Mixed types (3)
```py
def _(arg: tuple[tuple[int, int, int], tuple[int, str, bytes], tuple[int, int, str]]):
for a, b, c in arg:
reveal_type(a) # revealed: int
reveal_type(b) # revealed: int | str
reveal_type(c) # revealed: int | bytes | str
```
### Same literal values
```py
for a, b in ((1, 2), (3, 4)):
reveal_type(a) # revealed: Literal[1, 3]
reveal_type(b) # revealed: Literal[2, 4]
```
### Mixed literal values (1)
```py
for a, b in ((1, 2), ("a", "b")):
reveal_type(a) # revealed: Literal[1, "a"]
reveal_type(b) # revealed: Literal[2, "b"]
```
### Mixed literals values (2)
```py
# error: "Object of type `Literal[1]` is not iterable"
# error: "Object of type `Literal[2]` is not iterable"
# error: "Object of type `Literal[4]` is not iterable"
# error: [invalid-assignment] "Not enough values to unpack (expected 2, got 1)"
for a, b in (1, 2, (3, "a"), 4, (5, "b"), "c"):
reveal_type(a) # revealed: Unknown | Literal[3, 5] | LiteralString
reveal_type(b) # revealed: Unknown | Literal["a", "b"]
```
### Custom iterator (1)
```py
class Iterator:
def __next__(self) -> tuple[int, int]:
return (1, 2)
class Iterable:
def __iter__(self) -> Iterator:
return Iterator()
for a, b in Iterable():
reveal_type(a) # revealed: int
reveal_type(b) # revealed: int
```
### Custom iterator (2)
```py
class Iterator:
def __next__(self) -> bytes:
return b""
class Iterable:
def __iter__(self) -> Iterator:
return Iterator()
def _(arg: tuple[tuple[int, str], Iterable]):
for a, b in arg:
reveal_type(a) # revealed: int | bytes
reveal_type(b) # revealed: str | bytes
```

View File

@@ -17,5 +17,5 @@ class Manager:
async def test():
async with Manager() as f:
reveal_type(f) # revealed: @Todo(async with statement)
reveal_type(f) # revealed: @Todo(async `with` statement)
```

View File

@@ -43,7 +43,7 @@ impl<T> AstNodeRef<T> {
}
/// Returns a reference to the wrapped node.
pub fn node(&self) -> &T {
pub const fn node(&self) -> &T {
// SAFETY: Holding on to `parsed` ensures that the AST to which `node` belongs is still
// alive and not moved.
unsafe { self.node.as_ref() }

View File

@@ -1,4 +1,4 @@
use crate::lint::RuleSelection;
use crate::lint::{LintRegistry, RuleSelection};
use ruff_db::files::File;
use ruff_db::{Db as SourceDb, Upcast};
@@ -8,6 +8,8 @@ pub trait Db: SourceDb + Upcast<dyn SourceDb> {
fn is_file_open(&self, file: File) -> bool;
fn rule_selection(&self) -> &RuleSelection;
fn lint_registry(&self) -> &LintRegistry;
}
#[cfg(test)]
@@ -16,10 +18,10 @@ pub(crate) mod tests {
use crate::program::{Program, SearchPathSettings};
use crate::python_version::PythonVersion;
use crate::{default_lint_registry, ProgramSettings};
use crate::{default_lint_registry, ProgramSettings, PythonPlatform};
use super::Db;
use crate::lint::RuleSelection;
use crate::lint::{LintRegistry, RuleSelection};
use anyhow::Context;
use ruff_db::files::{File, Files};
use ruff_db::system::{DbWithTestSystem, System, SystemPathBuf, TestSystem};
@@ -45,7 +47,7 @@ pub(crate) mod tests {
vendored: red_knot_vendored::file_system().clone(),
events: Arc::default(),
files: Files::default(),
rule_selection: Arc::new(RuleSelection::from_registry(&default_lint_registry())),
rule_selection: Arc::new(RuleSelection::from_registry(default_lint_registry())),
}
}
@@ -112,6 +114,10 @@ pub(crate) mod tests {
fn rule_selection(&self) -> &RuleSelection {
&self.rule_selection
}
fn lint_registry(&self) -> &LintRegistry {
default_lint_registry()
}
}
#[salsa::db]
@@ -127,6 +133,8 @@ pub(crate) mod tests {
pub(crate) struct TestDbBuilder<'a> {
/// Target Python version
python_version: PythonVersion,
/// Target Python platform
python_platform: PythonPlatform,
/// Path to a custom typeshed directory
custom_typeshed: Option<SystemPathBuf>,
/// Path and content pairs for files that should be present
@@ -137,6 +145,7 @@ pub(crate) mod tests {
pub(crate) fn new() -> Self {
Self {
python_version: PythonVersion::default(),
python_platform: PythonPlatform::default(),
custom_typeshed: None,
files: vec![],
}
@@ -171,8 +180,9 @@ pub(crate) mod tests {
Program::from_settings(
&db,
&ProgramSettings {
ProgramSettings {
python_version: self.python_version,
python_platform: self.python_platform,
search_paths,
},
)

View File

@@ -3,10 +3,12 @@ use std::hash::BuildHasherDefault;
use rustc_hash::FxHasher;
use crate::lint::{LintRegistry, LintRegistryBuilder};
use crate::suppression::{INVALID_IGNORE_COMMENT, UNKNOWN_RULE, UNUSED_IGNORE_COMMENT};
pub use db::Db;
pub use module_name::ModuleName;
pub use module_resolver::{resolve_module, system_module_search_paths, Module};
pub use module_resolver::{resolve_module, system_module_search_paths, KnownModule, Module};
pub use program::{Program, ProgramSettings, SearchPathSettings, SitePackages};
pub use python_platform::PythonPlatform;
pub use python_version::PythonVersion;
pub use semantic_model::{HasTy, SemanticModel};
@@ -17,26 +19,36 @@ mod module_name;
mod module_resolver;
mod node_key;
mod program;
mod python_platform;
mod python_version;
pub mod semantic_index;
mod semantic_model;
pub(crate) mod site_packages;
mod stdlib;
mod suppression;
pub(crate) mod symbol;
pub mod types;
mod unpack;
mod util;
mod visibility_constraints;
type FxOrderSet<V> = ordermap::set::OrderSet<V, BuildHasherDefault<FxHasher>>;
/// Creates a new registry with all known semantic lints.
pub fn default_lint_registry() -> LintRegistry {
let mut registry = LintRegistryBuilder::default();
register_lints(&mut registry);
registry.build()
/// Returns the default registry with all known semantic lints.
pub fn default_lint_registry() -> &'static LintRegistry {
static REGISTRY: std::sync::LazyLock<LintRegistry> = std::sync::LazyLock::new(|| {
let mut registry = LintRegistryBuilder::default();
register_lints(&mut registry);
registry.build()
});
&REGISTRY
}
/// Register all known semantic lints.
pub fn register_lints(registry: &mut LintRegistryBuilder) {
types::register_lints(registry);
registry.register_lint(&UNUSED_IGNORE_COMMENT);
registry.register_lint(&UNKNOWN_RULE);
registry.register_lint(&INVALID_IGNORE_COMMENT);
}

View File

@@ -321,7 +321,7 @@ impl LintRegistryBuilder {
}
}
#[derive(Default, Debug)]
#[derive(Default, Debug, Clone)]
pub struct LintRegistry {
lints: Vec<LintId>,
by_name: FxHashMap<&'static str, LintEntry>,
@@ -374,7 +374,7 @@ impl LintRegistry {
}
}
#[derive(Error, Debug, Clone)]
#[derive(Error, Debug, Clone, PartialEq, Eq)]
pub enum GetLintError {
/// The name maps to this removed lint.
#[error("lint {0} has been removed")]
@@ -385,7 +385,7 @@ pub enum GetLintError {
Unknown(String),
}
#[derive(Debug, PartialEq, Eq)]
#[derive(Debug, PartialEq, Eq, Clone, Copy)]
pub enum LintEntry {
/// An existing lint rule. Can be in preview, stable or deprecated.
Lint(LintId),
@@ -444,6 +444,11 @@ impl RuleSelection {
self.lints.get(&lint).copied()
}
/// Returns `true` if the `lint` is enabled.
pub fn is_enabled(&self, lint: LintId) -> bool {
self.severity(lint).is_some()
}
/// Enables `lint` and configures with the given `severity`.
///
/// Overrides any previous configuration for the lint.

View File

@@ -1,6 +1,6 @@
use std::iter::FusedIterator;
pub use module::Module;
pub use module::{KnownModule, Module};
pub use resolver::resolve_module;
pub(crate) use resolver::{file_to_module, SearchPaths};
use ruff_db::system::SystemPath;

View File

@@ -19,12 +19,14 @@ impl Module {
search_path: SearchPath,
file: File,
) -> Self {
let known = KnownModule::try_from_search_path_and_name(&search_path, &name);
Self {
inner: Arc::new(ModuleInner {
name,
kind,
search_path,
file,
known,
}),
}
}
@@ -39,6 +41,16 @@ impl Module {
self.inner.file
}
/// Is this a module that we special-case somehow? If so, which one?
pub fn known(&self) -> Option<KnownModule> {
self.inner.known
}
/// Does this module represent the given known module?
pub fn is_known(&self, known_module: KnownModule) -> bool {
self.known() == Some(known_module)
}
/// The search path from which the module was resolved.
pub(crate) fn search_path(&self) -> &SearchPath {
&self.inner.search_path
@@ -67,6 +79,7 @@ struct ModuleInner {
kind: ModuleKind,
search_path: SearchPath,
file: File,
known: Option<KnownModule>,
}
#[derive(Copy, Clone, Debug, Eq, PartialEq, Hash)]
@@ -83,3 +96,73 @@ impl ModuleKind {
matches!(self, ModuleKind::Package)
}
}
/// Enumeration of various core stdlib modules in which important types are located
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub enum KnownModule {
Builtins,
Types,
Typeshed,
TypingExtensions,
Typing,
Sys,
#[allow(dead_code)]
Abc, // currently only used in tests
Collections,
KnotExtensions,
}
impl KnownModule {
pub const fn as_str(self) -> &'static str {
match self {
Self::Builtins => "builtins",
Self::Types => "types",
Self::Typing => "typing",
Self::Typeshed => "_typeshed",
Self::TypingExtensions => "typing_extensions",
Self::Sys => "sys",
Self::Abc => "abc",
Self::Collections => "collections",
Self::KnotExtensions => "knot_extensions",
}
}
pub fn name(self) -> ModuleName {
let self_as_str = self.as_str();
ModuleName::new_static(self_as_str)
.unwrap_or_else(|| panic!("{self_as_str} should be a valid module name!"))
}
pub(crate) fn try_from_search_path_and_name(
search_path: &SearchPath,
name: &ModuleName,
) -> Option<Self> {
if !search_path.is_standard_library() {
return None;
}
match name.as_str() {
"builtins" => Some(Self::Builtins),
"types" => Some(Self::Types),
"typing" => Some(Self::Typing),
"_typeshed" => Some(Self::Typeshed),
"typing_extensions" => Some(Self::TypingExtensions),
"sys" => Some(Self::Sys),
"abc" => Some(Self::Abc),
"collections" => Some(Self::Collections),
"knot_extensions" => Some(Self::KnotExtensions),
_ => None,
}
}
pub const fn is_builtins(self) -> bool {
matches!(self, Self::Builtins)
}
pub const fn is_typing(self) -> bool {
matches!(self, Self::Typing)
}
pub const fn is_knot_extensions(self) -> bool {
matches!(self, Self::KnotExtensions)
}
}

View File

@@ -721,8 +721,8 @@ mod tests {
use crate::module_name::ModuleName;
use crate::module_resolver::module::ModuleKind;
use crate::module_resolver::testing::{FileSpec, MockedTypeshed, TestCase, TestCaseBuilder};
use crate::ProgramSettings;
use crate::PythonVersion;
use crate::{ProgramSettings, PythonPlatform};
use super::*;
@@ -1262,7 +1262,7 @@ mod tests {
fn symlink() -> anyhow::Result<()> {
use anyhow::Context;
use crate::program::Program;
use crate::{program::Program, PythonPlatform};
use ruff_db::system::{OsSystem, SystemPath};
use crate::db::tests::TestDb;
@@ -1294,8 +1294,9 @@ mod tests {
Program::from_settings(
&db,
&ProgramSettings {
ProgramSettings {
python_version: PythonVersion::PY38,
python_platform: PythonPlatform::default(),
search_paths: SearchPathSettings {
extra_paths: vec![],
src_root: src.clone(),
@@ -1799,8 +1800,9 @@ not_a_directory
Program::from_settings(
&db,
&ProgramSettings {
ProgramSettings {
python_version: PythonVersion::default(),
python_platform: PythonPlatform::default(),
search_paths: SearchPathSettings {
extra_paths: vec![],
src_root: SystemPathBuf::from("/src"),

View File

@@ -4,7 +4,7 @@ use ruff_db::vendored::VendoredPathBuf;
use crate::db::tests::TestDb;
use crate::program::{Program, SearchPathSettings};
use crate::python_version::PythonVersion;
use crate::{ProgramSettings, SitePackages};
use crate::{ProgramSettings, PythonPlatform, SitePackages};
/// A test case for the module resolver.
///
@@ -101,6 +101,7 @@ pub(crate) struct UnspecifiedTypeshed;
pub(crate) struct TestCaseBuilder<T> {
typeshed_option: T,
python_version: PythonVersion,
python_platform: PythonPlatform,
first_party_files: Vec<FileSpec>,
site_packages_files: Vec<FileSpec>,
}
@@ -147,6 +148,7 @@ impl TestCaseBuilder<UnspecifiedTypeshed> {
Self {
typeshed_option: UnspecifiedTypeshed,
python_version: PythonVersion::default(),
python_platform: PythonPlatform::default(),
first_party_files: vec![],
site_packages_files: vec![],
}
@@ -157,12 +159,14 @@ impl TestCaseBuilder<UnspecifiedTypeshed> {
let TestCaseBuilder {
typeshed_option: _,
python_version,
python_platform,
first_party_files,
site_packages_files,
} = self;
TestCaseBuilder {
typeshed_option: VendoredTypeshed,
python_version,
python_platform,
first_party_files,
site_packages_files,
}
@@ -176,6 +180,7 @@ impl TestCaseBuilder<UnspecifiedTypeshed> {
let TestCaseBuilder {
typeshed_option: _,
python_version,
python_platform,
first_party_files,
site_packages_files,
} = self;
@@ -183,6 +188,7 @@ impl TestCaseBuilder<UnspecifiedTypeshed> {
TestCaseBuilder {
typeshed_option: typeshed,
python_version,
python_platform,
first_party_files,
site_packages_files,
}
@@ -212,6 +218,7 @@ impl TestCaseBuilder<MockedTypeshed> {
let TestCaseBuilder {
typeshed_option,
python_version,
python_platform,
first_party_files,
site_packages_files,
} = self;
@@ -225,8 +232,9 @@ impl TestCaseBuilder<MockedTypeshed> {
Program::from_settings(
&db,
&ProgramSettings {
ProgramSettings {
python_version,
python_platform,
search_paths: SearchPathSettings {
extra_paths: vec![],
src_root: src.clone(),
@@ -269,6 +277,7 @@ impl TestCaseBuilder<VendoredTypeshed> {
let TestCaseBuilder {
typeshed_option: VendoredTypeshed,
python_version,
python_platform,
first_party_files,
site_packages_files,
} = self;
@@ -281,8 +290,9 @@ impl TestCaseBuilder<VendoredTypeshed> {
Program::from_settings(
&db,
&ProgramSettings {
ProgramSettings {
python_version,
python_platform,
search_paths: SearchPathSettings {
site_packages: SitePackages::Known(vec![site_packages.clone()]),
..SearchPathSettings::new(src.clone())

View File

@@ -1,36 +1,68 @@
use crate::module_resolver::SearchPaths;
use crate::python_platform::PythonPlatform;
use crate::python_version::PythonVersion;
use crate::Db;
use anyhow::Context;
use ruff_db::system::{SystemPath, SystemPathBuf};
use salsa::Durability;
use salsa::Setter;
use ruff_db::system::{SystemPath, SystemPathBuf};
use crate::module_resolver::SearchPaths;
use crate::Db;
#[salsa::input(singleton)]
pub struct Program {
pub python_version: PythonVersion,
#[return_ref]
pub python_platform: PythonPlatform,
#[return_ref]
pub(crate) search_paths: SearchPaths,
}
impl Program {
pub fn from_settings(db: &dyn Db, settings: &ProgramSettings) -> anyhow::Result<Self> {
pub fn from_settings(db: &dyn Db, settings: ProgramSettings) -> anyhow::Result<Self> {
let ProgramSettings {
python_version,
python_platform,
search_paths,
} = settings;
tracing::info!("Python version: Python {python_version}");
tracing::info!("Python version: Python {python_version}, platform: {python_platform}");
let search_paths = SearchPaths::from_settings(db, search_paths)
let search_paths = SearchPaths::from_settings(db, &search_paths)
.with_context(|| "Invalid search path settings")?;
Ok(Program::builder(settings.python_version, search_paths)
.durability(Durability::HIGH)
.new(db))
Ok(
Program::builder(python_version, python_platform, search_paths)
.durability(Durability::HIGH)
.new(db),
)
}
pub fn update_from_settings(
self,
db: &mut dyn Db,
settings: ProgramSettings,
) -> anyhow::Result<()> {
let ProgramSettings {
python_version,
python_platform,
search_paths,
} = settings;
if &python_platform != self.python_platform(db) {
tracing::debug!("Updating python platform: `{python_platform:?}`");
self.set_python_platform(db).to(python_platform);
}
if python_version != self.python_version(db) {
tracing::debug!("Updating python version: Python {python_version}");
self.set_python_version(db).to(python_version);
}
self.update_search_paths(db, &search_paths)?;
Ok(())
}
pub fn update_search_paths(
@@ -57,6 +89,7 @@ impl Program {
#[cfg_attr(feature = "serde", derive(serde::Serialize))]
pub struct ProgramSettings {
pub python_version: PythonVersion,
pub python_platform: PythonPlatform,
pub search_paths: SearchPathSettings,
}
@@ -69,7 +102,7 @@ pub struct SearchPathSettings {
/// or pyright's stubPath configuration setting.
pub extra_paths: Vec<SystemPathBuf>,
/// The root of the workspace, used for finding first-party modules.
/// The root of the project, used for finding first-party modules.
pub src_root: SystemPathBuf,
/// Optional path to a "custom typeshed" directory on disk for us to use for standard-library types.

View File

@@ -0,0 +1,30 @@
use std::fmt::{Display, Formatter};
/// The target platform to assume when resolving types.
#[derive(Debug, Clone, Default, PartialEq, Eq)]
#[cfg_attr(
feature = "serde",
derive(serde::Serialize, serde::Deserialize),
serde(rename_all = "kebab-case")
)]
pub enum PythonPlatform {
/// Do not make any assumptions about the target platform.
#[default]
All,
/// Assume a specific target platform like `linux`, `darwin` or `win32`.
///
/// We use a string (instead of individual enum variants), as the set of possible platforms
/// may change over time. See <https://docs.python.org/3/library/sys.html#sys.platform> for
/// some known platform identifiers.
#[cfg_attr(feature = "serde", serde(untagged))]
Identifier(String),
}
impl Display for PythonPlatform {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
match self {
PythonPlatform::All => f.write_str("all"),
PythonPlatform::Identifier(name) => f.write_str(name),
}
}
}

View File

@@ -29,7 +29,8 @@ pub mod symbol;
mod use_def;
pub(crate) use self::use_def::{
BindingWithConstraints, BindingWithConstraintsIterator, DeclarationsIterator,
BindingWithConstraints, BindingWithConstraintsIterator, DeclarationWithConstraint,
DeclarationsIterator, ScopedVisibilityConstraintId,
};
type SymbolMap = hashbrown::HashMap<ScopedSymbolId, (), FxBuildHasher>;
@@ -378,14 +379,12 @@ mod tests {
impl UseDefMap<'_> {
fn first_public_binding(&self, symbol: ScopedSymbolId) -> Option<Definition<'_>> {
self.public_bindings(symbol)
.next()
.map(|constrained_binding| constrained_binding.binding)
.find_map(|constrained_binding| constrained_binding.binding)
}
fn first_binding_at_use(&self, use_id: ScopedUseId) -> Option<Definition<'_>> {
self.bindings_at_use(use_id)
.next()
.map(|constrained_binding| constrained_binding.binding)
.find_map(|constrained_binding| constrained_binding.binding)
}
}

View File

@@ -9,12 +9,12 @@ use ruff_index::IndexVec;
use ruff_python_ast as ast;
use ruff_python_ast::name::Name;
use ruff_python_ast::visitor::{walk_expr, walk_pattern, walk_stmt, Visitor};
use ruff_python_ast::{BoolOp, Expr};
use crate::ast_node_ref::AstNodeRef;
use crate::module_name::ModuleName;
use crate::semantic_index::ast_ids::node_key::ExpressionNodeKey;
use crate::semantic_index::ast_ids::AstIdsBuilder;
use crate::semantic_index::constraint::PatternConstraintKind;
use crate::semantic_index::definition::{
AssignmentDefinitionNodeRef, ComprehensionDefinitionNodeRef, Definition, DefinitionNodeKey,
DefinitionNodeRef, ForStmtDefinitionNodeRef, ImportFromDefinitionNodeRef,
@@ -24,9 +24,12 @@ use crate::semantic_index::symbol::{
FileScopeId, NodeWithScopeKey, NodeWithScopeRef, Scope, ScopeId, ScopedSymbolId,
SymbolTableBuilder,
};
use crate::semantic_index::use_def::{FlowSnapshot, UseDefMapBuilder};
use crate::semantic_index::use_def::{
FlowSnapshot, ScopedConstraintId, ScopedVisibilityConstraintId, UseDefMapBuilder,
};
use crate::semantic_index::SemanticIndex;
use crate::unpack::Unpack;
use crate::unpack::{Unpack, UnpackValue};
use crate::visibility_constraints::VisibilityConstraint;
use crate::Db;
use super::constraint::{Constraint, ConstraintNode, PatternConstraint};
@@ -285,11 +288,7 @@ impl<'db> SemanticIndexBuilder<'db> {
constraint
}
fn record_constraint(&mut self, constraint: Constraint<'db>) {
self.current_use_def_map_mut().record_constraint(constraint);
}
fn build_constraint(&mut self, constraint_node: &Expr) -> Constraint<'db> {
fn build_constraint(&mut self, constraint_node: &ast::Expr) -> Constraint<'db> {
let expression = self.add_standalone_expression(constraint_node);
Constraint {
node: ConstraintNode::Expression(expression),
@@ -297,12 +296,89 @@ impl<'db> SemanticIndexBuilder<'db> {
}
}
fn record_negated_constraint(&mut self, constraint: Constraint<'db>) {
/// Adds a new constraint to the list of all constraints, but does not record it. Returns the
/// constraint ID for later recording using [`SemanticIndexBuilder::record_constraint_id`].
fn add_constraint(&mut self, constraint: Constraint<'db>) -> ScopedConstraintId {
self.current_use_def_map_mut().add_constraint(constraint)
}
/// Negates a constraint and adds it to the list of all constraints, does not record it.
fn add_negated_constraint(
&mut self,
constraint: Constraint<'db>,
) -> (Constraint<'db>, ScopedConstraintId) {
let negated = Constraint {
node: constraint.node,
is_positive: false,
};
let id = self.current_use_def_map_mut().add_constraint(negated);
(negated, id)
}
/// Records a previously added constraint by adding it to all live bindings.
fn record_constraint_id(&mut self, constraint: ScopedConstraintId) {
self.current_use_def_map_mut()
.record_constraint(Constraint {
node: constraint.node,
is_positive: false,
});
.record_constraint_id(constraint);
}
/// Adds and records a constraint, i.e. adds it to all live bindings.
fn record_constraint(&mut self, constraint: Constraint<'db>) {
self.current_use_def_map_mut().record_constraint(constraint);
}
/// Negates the given constraint and then adds it to all live bindings.
fn record_negated_constraint(&mut self, constraint: Constraint<'db>) -> ScopedConstraintId {
let (_, id) = self.add_negated_constraint(constraint);
self.record_constraint_id(id);
id
}
/// Adds a new visibility constraint, but does not record it. Returns the constraint ID
/// for later recording using [`SemanticIndexBuilder::record_visibility_constraint_id`].
fn add_visibility_constraint(
&mut self,
constraint: VisibilityConstraint<'db>,
) -> ScopedVisibilityConstraintId {
self.current_use_def_map_mut()
.add_visibility_constraint(constraint)
}
/// Records a previously added visibility constraint by applying it to all live bindings
/// and declarations.
fn record_visibility_constraint_id(&mut self, constraint: ScopedVisibilityConstraintId) {
self.current_use_def_map_mut()
.record_visibility_constraint_id(constraint);
}
/// Negates the given visibility constraint and then adds it to all live bindings and declarations.
fn record_negated_visibility_constraint(
&mut self,
constraint: ScopedVisibilityConstraintId,
) -> ScopedVisibilityConstraintId {
self.current_use_def_map_mut()
.record_visibility_constraint(VisibilityConstraint::VisibleIfNot(constraint))
}
/// Records a visibility constraint by applying it to all live bindings and declarations.
fn record_visibility_constraint(
&mut self,
constraint: Constraint<'db>,
) -> ScopedVisibilityConstraintId {
self.current_use_def_map_mut()
.record_visibility_constraint(VisibilityConstraint::VisibleIf(constraint))
}
/// Records a [`VisibilityConstraint::Ambiguous`] constraint.
fn record_ambiguous_visibility(&mut self) -> ScopedVisibilityConstraintId {
self.current_use_def_map_mut()
.record_visibility_constraint(VisibilityConstraint::Ambiguous)
}
/// Simplifies (resets) visibility constraints on all live bindings and declarations that did
/// not see any new definitions since the given snapshot.
fn simplify_visibility_constraints(&mut self, snapshot: FlowSnapshot) {
self.current_use_def_map_mut()
.simplify_visibility_constraints(snapshot);
}
fn push_assignment(&mut self, assignment: CurrentAssignment<'db>) {
@@ -324,30 +400,52 @@ impl<'db> SemanticIndexBuilder<'db> {
fn add_pattern_constraint(
&mut self,
subject: &ast::Expr,
subject: Expression<'db>,
pattern: &ast::Pattern,
) -> PatternConstraint<'db> {
#[allow(unsafe_code)]
let (subject, pattern) = unsafe {
(
AstNodeRef::new(self.module.clone(), subject),
AstNodeRef::new(self.module.clone(), pattern),
)
guard: Option<&ast::Expr>,
) -> Constraint<'db> {
// This is called for the top-level pattern of each match arm. We need to create a
// standalone expression for each arm of a match statement, since they can introduce
// constraints on the match subject. (Or more accurately, for the match arm's pattern,
// since its the pattern that introduces any constraints, not the body.) Ideally, that
// standalone expression would wrap the match arm's pattern as a whole. But a standalone
// expression can currently only wrap an ast::Expr, which patterns are not. So, we need to
// choose an Expr that can “stand in” for the pattern, which we can wrap in a standalone
// expression.
//
// See the comment in TypeInferenceBuilder::infer_match_pattern for more details.
let guard = guard.map(|guard| self.add_standalone_expression(guard));
let kind = match pattern {
ast::Pattern::MatchValue(pattern) => {
let value = self.add_standalone_expression(&pattern.value);
PatternConstraintKind::Value(value, guard)
}
ast::Pattern::MatchSingleton(singleton) => {
PatternConstraintKind::Singleton(singleton.value, guard)
}
ast::Pattern::MatchClass(pattern) => {
let cls = self.add_standalone_expression(&pattern.cls);
PatternConstraintKind::Class(cls, guard)
}
_ => PatternConstraintKind::Unsupported,
};
let pattern_constraint = PatternConstraint::new(
self.db,
self.file,
self.current_scope(),
subject,
pattern,
kind,
countme::Count::default(),
);
self.current_use_def_map_mut()
.record_constraint(Constraint {
node: ConstraintNode::Pattern(pattern_constraint),
is_positive: true,
});
pattern_constraint
let constraint = Constraint {
node: ConstraintNode::Pattern(pattern_constraint),
is_positive: true,
};
self.current_use_def_map_mut().record_constraint(constraint);
constraint
}
/// Record an expression that needs to be a Salsa ingredient, because we need to infer its type
@@ -726,7 +824,7 @@ where
unsafe {
AstNodeRef::new(self.module.clone(), target)
},
value,
UnpackValue::Assign(value),
countme::Count::default(),
)),
})
@@ -795,10 +893,13 @@ where
}
ast::Stmt::If(node) => {
self.visit_expr(&node.test);
let pre_if = self.flow_snapshot();
let constraint = self.record_expression_constraint(&node.test);
let mut constraints = vec![constraint];
let mut no_branch_taken = self.flow_snapshot();
let mut last_constraint = self.record_expression_constraint(&node.test);
self.visit_body(&node.body);
let visibility_constraint_id = self.record_visibility_constraint(last_constraint);
let mut vis_constraints = vec![visibility_constraint_id];
let mut post_clauses: Vec<FlowSnapshot> = vec![];
let elif_else_clauses = node
.elif_else_clauses
@@ -820,20 +921,37 @@ where
// the state that we merge the other snapshots into
post_clauses.push(self.flow_snapshot());
// we can only take an elif/else branch if none of the previous ones were
// taken, so the block entry state is always `pre_if`
self.flow_restore(pre_if.clone());
for constraint in &constraints {
self.record_negated_constraint(*constraint);
}
if let Some(elif_test) = clause_test {
// taken
self.flow_restore(no_branch_taken.clone());
self.record_negated_constraint(last_constraint);
let elif_constraint = if let Some(elif_test) = clause_test {
self.visit_expr(elif_test);
constraints.push(self.record_expression_constraint(elif_test));
}
// A test expression is evaluated whether the branch is taken or not
no_branch_taken = self.flow_snapshot();
let constraint = self.record_expression_constraint(elif_test);
Some(constraint)
} else {
None
};
self.visit_body(clause_body);
for id in &vis_constraints {
self.record_negated_visibility_constraint(*id);
}
if let Some(elif_constraint) = elif_constraint {
last_constraint = elif_constraint;
let id = self.record_visibility_constraint(elif_constraint);
vis_constraints.push(id);
}
}
for post_clause_state in post_clauses {
self.flow_merge(post_clause_state);
}
self.simplify_visibility_constraints(no_branch_taken);
}
ast::Stmt::While(ast::StmtWhile {
test,
@@ -856,6 +974,8 @@ where
self.visit_body(body);
self.set_inside_loop(outer_loop_state);
let vis_constraint_id = self.record_visibility_constraint(constraint);
// Get the break states from the body of this loop, and restore the saved outer
// ones.
let break_states =
@@ -863,15 +983,21 @@ where
// We may execute the `else` clause without ever executing the body, so merge in
// the pre-loop state before visiting `else`.
self.flow_merge(pre_loop);
self.flow_merge(pre_loop.clone());
self.record_negated_constraint(constraint);
self.visit_body(orelse);
self.record_negated_visibility_constraint(vis_constraint_id);
// Breaking out of a while loop bypasses the `else` clause, so merge in the break
// states after visiting `else`.
for break_state in break_states {
self.flow_merge(break_state);
let snapshot = self.flow_snapshot();
self.flow_restore(break_state);
self.record_visibility_constraint(constraint);
self.flow_merge(snapshot);
}
self.simplify_visibility_constraints(pre_loop);
}
ast::Stmt::With(ast::StmtWith {
items,
@@ -909,16 +1035,47 @@ where
orelse,
},
) => {
self.add_standalone_expression(iter);
debug_assert_eq!(&self.current_assignments, &[]);
let iter_expr = self.add_standalone_expression(iter);
self.visit_expr(iter);
self.record_ambiguous_visibility();
let pre_loop = self.flow_snapshot();
let saved_break_states = std::mem::take(&mut self.loop_break_states);
debug_assert_eq!(&self.current_assignments, &[]);
self.push_assignment(for_stmt.into());
let current_assignment = match &**target {
ast::Expr::List(_) | ast::Expr::Tuple(_) => Some(CurrentAssignment::For {
node: for_stmt,
first: true,
unpack: Some(Unpack::new(
self.db,
self.file,
self.current_scope(),
#[allow(unsafe_code)]
unsafe {
AstNodeRef::new(self.module.clone(), target)
},
UnpackValue::Iterable(iter_expr),
countme::Count::default(),
)),
}),
ast::Expr::Name(_) => Some(CurrentAssignment::For {
node: for_stmt,
unpack: None,
first: false,
}),
_ => None,
};
if let Some(current_assignment) = current_assignment {
self.push_assignment(current_assignment);
}
self.visit_expr(target);
self.pop_assignment();
if current_assignment.is_some() {
self.pop_assignment();
}
// TODO: Definitions created by loop variables
// (and definitions created inside the body)
@@ -947,32 +1104,61 @@ where
cases,
range: _,
}) => {
self.add_standalone_expression(subject);
self.visit_expr(subject);
debug_assert_eq!(self.current_match_case, None);
let after_subject = self.flow_snapshot();
let Some((first, remaining)) = cases.split_first() else {
let subject_expr = self.add_standalone_expression(subject);
self.visit_expr(subject);
if cases.is_empty() {
return;
};
self.add_pattern_constraint(subject, &first.pattern);
self.visit_match_case(first);
let after_subject = self.flow_snapshot();
let mut vis_constraints = vec![];
let mut post_case_snapshots = vec![];
for case in remaining {
post_case_snapshots.push(self.flow_snapshot());
self.flow_restore(after_subject.clone());
self.add_pattern_constraint(subject, &case.pattern);
self.visit_match_case(case);
}
for post_clause_state in post_case_snapshots {
self.flow_merge(post_clause_state);
for (i, case) in cases.iter().enumerate() {
if i != 0 {
post_case_snapshots.push(self.flow_snapshot());
self.flow_restore(after_subject.clone());
}
self.current_match_case = Some(CurrentMatchCase::new(&case.pattern));
self.visit_pattern(&case.pattern);
self.current_match_case = None;
let constraint_id = self.add_pattern_constraint(
subject_expr,
&case.pattern,
case.guard.as_deref(),
);
if let Some(expr) = &case.guard {
self.visit_expr(expr);
}
self.visit_body(&case.body);
for id in &vis_constraints {
self.record_negated_visibility_constraint(*id);
}
let vis_constraint_id = self.record_visibility_constraint(constraint_id);
vis_constraints.push(vis_constraint_id);
}
// If there is no final wildcard match case, pretend there is one. This is similar to how
// we add an implicit `else` block in if-elif chains, in case it's not present.
if !cases
.last()
.is_some_and(|case| case.guard.is_none() && case.pattern.is_wildcard())
{
self.flow_merge(after_subject);
post_case_snapshots.push(self.flow_snapshot());
self.flow_restore(after_subject.clone());
for id in &vis_constraints {
self.record_negated_visibility_constraint(*id);
}
}
for post_clause_state in post_case_snapshots {
self.flow_merge(post_clause_state);
}
self.simplify_visibility_constraints(after_subject);
}
ast::Stmt::Try(ast::StmtTry {
body,
@@ -982,6 +1168,8 @@ where
is_star,
range: _,
}) => {
self.record_ambiguous_visibility();
// Save the state prior to visiting any of the `try` block.
//
// Potentially none of the `try` block could have been executed prior to executing
@@ -1136,12 +1324,18 @@ where
Some(CurrentAssignment::AugAssign(aug_assign)) => {
self.add_definition(symbol, aug_assign);
}
Some(CurrentAssignment::For(node)) => {
Some(CurrentAssignment::For {
node,
first,
unpack,
}) => {
self.add_definition(
symbol,
ForStmtDefinitionNodeRef {
unpack,
first,
iterable: &node.iter,
target: name_node,
name: name_node,
is_async: node.is_async,
},
);
@@ -1177,7 +1371,9 @@ where
}
}
if let Some(CurrentAssignment::Assign { first, .. }) = self.current_assignment_mut()
if let Some(
CurrentAssignment::Assign { first, .. } | CurrentAssignment::For { first, .. },
) = self.current_assignment_mut()
{
*first = false;
}
@@ -1222,19 +1418,19 @@ where
ast::Expr::If(ast::ExprIf {
body, test, orelse, ..
}) => {
// TODO detect statically known truthy or falsy test (via type inference, not naive
// AST inspection, so we can't simplify here, need to record test expression for
// later checking)
self.visit_expr(test);
let pre_if = self.flow_snapshot();
let constraint = self.record_expression_constraint(test);
self.visit_expr(body);
let visibility_constraint = self.record_visibility_constraint(constraint);
let post_body = self.flow_snapshot();
self.flow_restore(pre_if);
self.flow_restore(pre_if.clone());
self.record_negated_constraint(constraint);
self.visit_expr(orelse);
self.record_negated_visibility_constraint(visibility_constraint);
self.flow_merge(post_body);
self.simplify_visibility_constraints(pre_if);
}
ast::Expr::ListComp(
list_comprehension @ ast::ExprListComp {
@@ -1291,27 +1487,55 @@ where
range: _,
op,
}) => {
// TODO detect statically known truthy or falsy values (via type inference, not naive
// AST inspection, so we can't simplify here, need to record test expression for
// later checking)
let pre_op = self.flow_snapshot();
let mut snapshots = vec![];
let mut visibility_constraints = vec![];
for (index, value) in values.iter().enumerate() {
self.visit_expr(value);
// In the last value we don't need to take a snapshot nor add a constraint
for vid in &visibility_constraints {
self.record_visibility_constraint_id(*vid);
}
// For the last value, we don't need to model control flow. There is short-circuiting
// anymore.
if index < values.len() - 1 {
// Snapshot is taken after visiting the expression but before adding the constraint.
snapshots.push(self.flow_snapshot());
let constraint = self.build_constraint(value);
match op {
BoolOp::And => self.record_constraint(constraint),
BoolOp::Or => self.record_negated_constraint(constraint),
let (constraint, constraint_id) = match op {
ast::BoolOp::And => (constraint, self.add_constraint(constraint)),
ast::BoolOp::Or => self.add_negated_constraint(constraint),
};
let visibility_constraint = self
.add_visibility_constraint(VisibilityConstraint::VisibleIf(constraint));
let after_expr = self.flow_snapshot();
// We first model the short-circuiting behavior. We take the short-circuit
// path here if all of the previous short-circuit paths were not taken, so
// we record all previously existing visibility constraints, and negate the
// one for the current expression.
for vid in &visibility_constraints {
self.record_visibility_constraint_id(*vid);
}
self.record_negated_visibility_constraint(visibility_constraint);
snapshots.push(self.flow_snapshot());
// Then we model the non-short-circuiting behavior. Here, we need to delay
// the application of the visibility constraint until after the expression
// has been evaluated, so we only push it onto the stack here.
self.flow_restore(after_expr);
self.record_constraint_id(constraint_id);
visibility_constraints.push(visibility_constraint);
}
}
for snapshot in snapshots {
self.flow_merge(snapshot);
}
self.simplify_visibility_constraints(pre_op);
}
_ => {
walk_expr(self, expr);
@@ -1327,18 +1551,6 @@ where
}
}
fn visit_match_case(&mut self, match_case: &'ast ast::MatchCase) {
debug_assert!(self.current_match_case.is_none());
self.current_match_case = Some(CurrentMatchCase::new(&match_case.pattern));
self.visit_pattern(&match_case.pattern);
self.current_match_case = None;
if let Some(expr) = &match_case.guard {
self.visit_expr(expr);
}
self.visit_body(&match_case.body);
}
fn visit_pattern(&mut self, pattern: &'ast ast::Pattern) {
if let ast::Pattern::MatchStar(ast::PatternMatchStar {
name: Some(name),
@@ -1391,7 +1603,11 @@ enum CurrentAssignment<'a> {
},
AnnAssign(&'a ast::StmtAnnAssign),
AugAssign(&'a ast::StmtAugAssign),
For(&'a ast::StmtFor),
For {
node: &'a ast::StmtFor,
first: bool,
unpack: Option<Unpack<'a>>,
},
Named(&'a ast::ExprNamed),
Comprehension {
node: &'a ast::Comprehension,
@@ -1415,18 +1631,13 @@ impl<'a> From<&'a ast::StmtAugAssign> for CurrentAssignment<'a> {
}
}
impl<'a> From<&'a ast::StmtFor> for CurrentAssignment<'a> {
fn from(value: &'a ast::StmtFor) -> Self {
Self::For(value)
}
}
impl<'a> From<&'a ast::ExprNamed> for CurrentAssignment<'a> {
fn from(value: &'a ast::ExprNamed) -> Self {
Self::Named(value)
}
}
#[derive(Debug, PartialEq)]
struct CurrentMatchCase<'a> {
/// The pattern that's part of the current match case.
pattern: &'a ast::Pattern,

View File

@@ -1,7 +1,6 @@
use ruff_db::files::File;
use ruff_python_ast as ast;
use ruff_python_ast::Singleton;
use crate::ast_node_ref::AstNodeRef;
use crate::db::Db;
use crate::semantic_index::expression::Expression;
use crate::semantic_index::symbol::{FileScopeId, ScopeId};
@@ -18,6 +17,15 @@ pub(crate) enum ConstraintNode<'db> {
Pattern(PatternConstraint<'db>),
}
/// Pattern kinds for which we support type narrowing and/or static visibility analysis.
#[derive(Debug, Clone, PartialEq)]
pub(crate) enum PatternConstraintKind<'db> {
Singleton(Singleton, Option<Expression<'db>>),
Value(Expression<'db>, Option<Expression<'db>>),
Class(Expression<'db>, Option<Expression<'db>>),
Unsupported,
}
#[salsa::tracked]
pub(crate) struct PatternConstraint<'db> {
#[id]
@@ -28,11 +36,11 @@ pub(crate) struct PatternConstraint<'db> {
#[no_eq]
#[return_ref]
pub(crate) subject: AstNodeRef<ast::Expr>,
pub(crate) subject: Expression<'db>,
#[no_eq]
#[return_ref]
pub(crate) pattern: AstNodeRef<ast::Pattern>,
pub(crate) kind: PatternConstraintKind<'db>,
#[no_eq]
count: countme::Count<PatternConstraint<'static>>,

View File

@@ -4,7 +4,6 @@ use ruff_python_ast as ast;
use ruff_text_size::{Ranged, TextRange};
use crate::ast_node_ref::AstNodeRef;
use crate::module_resolver::file_to_module;
use crate::node_key::NodeKey;
use crate::semantic_index::symbol::{FileScopeId, ScopeId, ScopedSymbolId};
use crate::unpack::Unpack;
@@ -61,20 +60,6 @@ impl<'db> Definition<'db> {
pub(crate) fn is_binding(self, db: &'db dyn Db) -> bool {
self.kind(db).category().is_binding()
}
pub(crate) fn is_builtin_definition(self, db: &'db dyn Db) -> bool {
file_to_module(db, self.file(db)).is_some_and(|module| {
module.search_path().is_standard_library() && matches!(&**module.name(), "builtins")
})
}
/// Return true if this symbol was defined in the `typing` or `typing_extensions` modules
pub(crate) fn is_typing_definition(self, db: &'db dyn Db) -> bool {
file_to_module(db, self.file(db)).is_some_and(|module| {
module.search_path().is_standard_library()
&& matches!(&**module.name(), "typing" | "typing_extensions")
})
}
}
#[derive(Copy, Clone, Debug)]
@@ -226,8 +211,10 @@ pub(crate) struct WithItemDefinitionNodeRef<'a> {
#[derive(Copy, Clone, Debug)]
pub(crate) struct ForStmtDefinitionNodeRef<'a> {
pub(crate) unpack: Option<Unpack<'a>>,
pub(crate) iterable: &'a ast::Expr,
pub(crate) target: &'a ast::ExprName,
pub(crate) name: &'a ast::ExprName,
pub(crate) first: bool,
pub(crate) is_async: bool,
}
@@ -299,12 +286,16 @@ impl<'db> DefinitionNodeRef<'db> {
DefinitionKind::AugmentedAssignment(AstNodeRef::new(parsed, augmented_assignment))
}
DefinitionNodeRef::For(ForStmtDefinitionNodeRef {
unpack,
iterable,
target,
name,
first,
is_async,
}) => DefinitionKind::For(ForStmtDefinitionKind {
target: TargetKind::from(unpack),
iterable: AstNodeRef::new(parsed.clone(), iterable),
target: AstNodeRef::new(parsed, target),
name: AstNodeRef::new(parsed, name),
first,
is_async,
}),
DefinitionNodeRef::Comprehension(ComprehensionDefinitionNodeRef {
@@ -383,10 +374,12 @@ impl<'db> DefinitionNodeRef<'db> {
Self::AnnotatedAssignment(node) => node.into(),
Self::AugmentedAssignment(node) => node.into(),
Self::For(ForStmtDefinitionNodeRef {
unpack: _,
iterable: _,
target,
name,
first: _,
is_async: _,
}) => target.into(),
}) => name.into(),
Self::Comprehension(ComprehensionDefinitionNodeRef { target, .. }) => target.into(),
Self::VariadicPositionalParameter(node) => node.into(),
Self::VariadicKeywordParameter(node) => node.into(),
@@ -453,7 +446,7 @@ pub enum DefinitionKind<'db> {
Assignment(AssignmentDefinitionKind<'db>),
AnnotatedAssignment(AstNodeRef<ast::StmtAnnAssign>),
AugmentedAssignment(AstNodeRef<ast::StmtAugAssign>),
For(ForStmtDefinitionKind),
For(ForStmtDefinitionKind<'db>),
Comprehension(ComprehensionDefinitionKind),
VariadicPositionalParameter(AstNodeRef<ast::Parameter>),
VariadicKeywordParameter(AstNodeRef<ast::Parameter>),
@@ -466,8 +459,14 @@ pub enum DefinitionKind<'db> {
TypeVarTuple(AstNodeRef<ast::TypeParamTypeVarTuple>),
}
impl Ranged for DefinitionKind<'_> {
fn range(&self) -> TextRange {
impl DefinitionKind<'_> {
/// Returns the [`TextRange`] of the definition target.
///
/// A definition target would mainly be the node representing the symbol being defined i.e.,
/// [`ast::ExprName`] or [`ast::Identifier`] but could also be other nodes.
///
/// This is mainly used for logging and debugging purposes.
pub(crate) fn target_range(&self) -> TextRange {
match self {
DefinitionKind::Import(alias) => alias.range(),
DefinitionKind::ImportFrom(import) => import.alias().range(),
@@ -478,7 +477,7 @@ impl Ranged for DefinitionKind<'_> {
DefinitionKind::Assignment(assignment) => assignment.name().range(),
DefinitionKind::AnnotatedAssignment(assign) => assign.target.range(),
DefinitionKind::AugmentedAssignment(aug_assign) => aug_assign.target.range(),
DefinitionKind::For(for_stmt) => for_stmt.target().range(),
DefinitionKind::For(for_stmt) => for_stmt.name().range(),
DefinitionKind::Comprehension(comp) => comp.target().range(),
DefinitionKind::VariadicPositionalParameter(parameter) => parameter.name.range(),
DefinitionKind::VariadicKeywordParameter(parameter) => parameter.name.range(),
@@ -491,9 +490,7 @@ impl Ranged for DefinitionKind<'_> {
DefinitionKind::TypeVarTuple(type_var_tuple) => type_var_tuple.name.range(),
}
}
}
impl DefinitionKind<'_> {
pub(crate) fn category(&self) -> DefinitionCategory {
match self {
// functions, classes, and imports always bind, and we consider them declarations
@@ -666,22 +663,32 @@ impl WithItemDefinitionKind {
}
#[derive(Clone, Debug)]
pub struct ForStmtDefinitionKind {
pub struct ForStmtDefinitionKind<'db> {
target: TargetKind<'db>,
iterable: AstNodeRef<ast::Expr>,
target: AstNodeRef<ast::ExprName>,
name: AstNodeRef<ast::ExprName>,
first: bool,
is_async: bool,
}
impl ForStmtDefinitionKind {
impl<'db> ForStmtDefinitionKind<'db> {
pub(crate) fn iterable(&self) -> &ast::Expr {
self.iterable.node()
}
pub(crate) fn target(&self) -> &ast::ExprName {
self.target.node()
pub(crate) fn target(&self) -> TargetKind<'db> {
self.target
}
pub(crate) fn is_async(&self) -> bool {
pub(crate) fn name(&self) -> &ast::ExprName {
self.name.node()
}
pub(crate) const fn is_first(&self) -> bool {
self.first
}
pub(crate) const fn is_async(&self) -> bool {
self.is_async
}
}
@@ -757,12 +764,6 @@ impl From<&ast::StmtAugAssign> for DefinitionNodeKey {
}
}
impl From<&ast::StmtFor> for DefinitionNodeKey {
fn from(value: &ast::StmtFor) -> Self {
Self(NodeKey::from_node(value))
}
}
impl From<&ast::Parameter> for DefinitionNodeKey {
fn from(node: &ast::Parameter) -> Self {
Self(NodeKey::from_node(node))

View File

@@ -463,10 +463,7 @@ impl NodeWithScopeKind {
}
pub fn expect_function(&self) -> &ast::StmtFunctionDef {
match self {
Self::Function(function) => function.node(),
_ => panic!("expected function"),
}
self.as_function().expect("expected function")
}
pub fn expect_type_alias(&self) -> &ast::StmtTypeAlias {
@@ -475,6 +472,13 @@ impl NodeWithScopeKind {
_ => panic!("expected type alias"),
}
}
pub const fn as_function(&self) -> Option<&ast::StmtFunctionDef> {
match self {
Self::Function(function) => Some(function.node()),
_ => None,
}
}
}
#[derive(Copy, Clone, Debug, Eq, PartialEq, Hash)]

View File

@@ -169,17 +169,11 @@
//! indexvecs in the [`UseDefMap`].
//!
//! There is another special kind of possible "definition" for a symbol: there might be a path from
//! the scope entry to a given use in which the symbol is never bound.
//!
//! The simplest way to model "unbound" would be as a "binding" itself: the initial "binding" for
//! each symbol in a scope. But actually modeling it this way would unnecessarily increase the
//! number of [`Definition`]s that Salsa must track. Since "unbound" is special in that all symbols
//! share it, and it doesn't have any additional per-symbol state, and constraints are irrelevant
//! to it, we can represent it more efficiently: we use the `may_be_unbound` boolean on the
//! [`SymbolBindings`] struct. If this flag is `true` for a use of a symbol, it means the symbol
//! has a path to the use in which it is never bound. If this flag is `false`, it means we've
//! eliminated the possibility of unbound: every control flow path to the use includes a binding
//! for this symbol.
//! the scope entry to a given use in which the symbol is never bound. We model this with a special
//! "unbound" definition (a `None` entry at the start of the `all_definitions` vector). If that
//! sentinel definition is present in the live bindings at a given use, it means that there is a
//! possible path through control flow in which that symbol is unbound. Similarly, if that sentinel
//! is present in the live declarations, it means that the symbol is (possibly) undeclared.
//!
//! To build a [`UseDefMap`], the [`UseDefMapBuilder`] is notified of each new use, definition, and
//! constraint as they are encountered by the
@@ -190,11 +184,13 @@
//! end of the scope, it records the state for each symbol as the public definitions of that
//! symbol.
//!
//! Let's walk through the above example. Initially we record for `x` that it has no bindings, and
//! may be unbound. When we see `x = 1`, we record that as the sole live binding of `x`, and flip
//! `may_be_unbound` to `false`. Then we see `x = 2`, and we replace `x = 1` as the sole live
//! binding of `x`. When we get to `y = x`, we record that the live bindings for that use of `x`
//! are just the `x = 2` definition.
//! Let's walk through the above example. Initially we do not have any record of `x`. When we add
//! the new symbol (before we process the first binding), we create a new undefined `SymbolState`
//! which has a single live binding (the "unbound" definition) and a single live declaration (the
//! "undeclared" definition). When we see `x = 1`, we record that as the sole live binding of `x`.
//! The "unbound" binding is no longer visible. Then we see `x = 2`, and we replace `x = 1` as the
//! sole live binding of `x`. When we get to `y = x`, we record that the live bindings for that use
//! of `x` are just the `x = 2` definition.
//!
//! Then we hit the `if` branch. We visit the `test` node (`flag` in this case), since that will
//! happen regardless. Then we take a pre-branch snapshot of the current state for all symbols,
@@ -207,8 +203,8 @@
//! be the pre-if conditions; if we are entering the `else` clause, we know that the `if` test
//! failed and we didn't execute the `if` body. So we first reset the builder to the pre-if state,
//! using the snapshot we took previously (meaning we now have `x = 2` as the sole binding for `x`
//! again), then visit the `else` clause, where `x = 4` replaces `x = 2` as the sole live binding
//! of `x`.
//! again), and record a *negative* `flag` constraint for all live bindings (`x = 2`). We then
//! visit the `else` clause, where `x = 4` replaces `x = 2` as the sole live binding of `x`.
//!
//! Now we reach the end of the if/else, and want to visit the following code. The state here needs
//! to reflect that we might have gone through the `if` branch, or we might have gone through the
@@ -217,18 +213,58 @@
//! snapshot (which has `x = 3` as the only live binding). The result of this merge is that we now
//! have two live bindings of `x`: `x = 3` and `x = 4`.
//!
//! Another piece of information that the `UseDefMap` needs to provide are visibility constraints.
//! These are similar to the narrowing constraints, but apply to bindings and declarations within a
//! control flow path. Consider the following example:
//! ```py
//! x = 1
//! if test:
//! x = 2
//! y = "y"
//! ```
//! In principle, there are two possible control flow paths here. However, if we can statically
//! infer `test` to be always truthy or always falsy (that is, `__bool__` of `test` is of type
//! `Literal[True]` or `Literal[False]`), we can rule out one of the possible paths. To support
//! this feature, we record a visibility constraint of `test` to all live bindings and declarations
//! *after* visiting the body of the `if` statement. And we record a negative visibility constraint
//! `~test` to all live bindings/declarations in the (implicit) `else` branch. For the example
//! above, we would record the following visibility constraints (adding the implicit "unbound"
//! definitions for clarity):
//! ```py
//! x = <unbound> # not live, shadowed by `x = 1`
//! y = <unbound> # visibility constraint: ~test
//!
//! x = 1 # visibility constraint: ~test
//! if test:
//! x = 2 # visibility constraint: test
//! y = "y" # visibility constraint: test
//! ```
//! When we encounter a use of `x` after this `if` statement, we would record two live bindings: `x
//! = 1` with a constraint of `~test`, and `x = 2` with a constraint of `test`. In type inference,
//! when we iterate over all live bindings, we can evaluate these constraints to determine if a
//! particular binding is actually visible. For example, if `test` is always truthy, we only see
//! the `x = 2` binding. If `test` is always falsy, we only see the `x = 1` binding. And if the
//! `__bool__` method of `test` returns type `bool`, we can see both bindings.
//!
//! Note that we also record visibility constraints for the start of the scope. This is important
//! to determine if a symbol is definitely bound, possibly unbound, or definitely unbound. In the
//! example above, The `y = <unbound>` binding is constrained by `~test`, so `y` would only be
//! definitely-bound if `test` is always truthy.
//!
//! The [`UseDefMapBuilder`] itself just exposes methods for taking a snapshot, resetting to a
//! snapshot, and merging a snapshot into the current state. The logic using these methods lives in
//! [`SemanticIndexBuilder`](crate::semantic_index::builder::SemanticIndexBuilder), e.g. where it
//! visits a `StmtIf` node.
use self::symbol_state::{
BindingIdWithConstraintsIterator, ConstraintIdIterator, DeclarationIdIterator,
ScopedConstraintId, ScopedDefinitionId, SymbolBindings, SymbolDeclarations, SymbolState,
ScopedDefinitionId, SymbolBindings, SymbolDeclarations, SymbolState,
};
pub(crate) use self::symbol_state::{ScopedConstraintId, ScopedVisibilityConstraintId};
use crate::semantic_index::ast_ids::ScopedUseId;
use crate::semantic_index::definition::Definition;
use crate::semantic_index::symbol::ScopedSymbolId;
use crate::symbol::Boundness;
use crate::semantic_index::use_def::symbol_state::DeclarationIdWithConstraint;
use crate::visibility_constraints::{VisibilityConstraint, VisibilityConstraints};
use ruff_index::IndexVec;
use rustc_hash::FxHashMap;
@@ -237,14 +273,20 @@ use super::constraint::Constraint;
mod bitset;
mod symbol_state;
type AllConstraints<'db> = IndexVec<ScopedConstraintId, Constraint<'db>>;
/// Applicable definitions and constraints for every use of a name.
#[derive(Debug, PartialEq, Eq)]
pub(crate) struct UseDefMap<'db> {
/// Array of [`Definition`] in this scope.
all_definitions: IndexVec<ScopedDefinitionId, Definition<'db>>,
/// Array of [`Definition`] in this scope. Only the first entry should be `None`;
/// this represents the implicit "unbound"/"undeclared" definition of every symbol.
all_definitions: IndexVec<ScopedDefinitionId, Option<Definition<'db>>>,
/// Array of [`Constraint`] in this scope.
all_constraints: IndexVec<ScopedConstraintId, Constraint<'db>>,
all_constraints: AllConstraints<'db>,
/// Array of [`VisibilityConstraint`]s in this scope.
visibility_constraints: VisibilityConstraints<'db>,
/// [`SymbolBindings`] reaching a [`ScopedUseId`].
bindings_by_use: IndexVec<ScopedUseId, SymbolBindings>,
@@ -275,14 +317,6 @@ impl<'db> UseDefMap<'db> {
self.bindings_iterator(&self.bindings_by_use[use_id])
}
pub(crate) fn use_boundness(&self, use_id: ScopedUseId) -> Boundness {
if self.bindings_by_use[use_id].may_be_unbound() {
Boundness::PossiblyUnbound
} else {
Boundness::Bound
}
}
pub(crate) fn public_bindings(
&self,
symbol: ScopedSymbolId,
@@ -290,14 +324,6 @@ impl<'db> UseDefMap<'db> {
self.bindings_iterator(self.public_symbols[symbol].bindings())
}
pub(crate) fn public_boundness(&self, symbol: ScopedSymbolId) -> Boundness {
if self.public_symbols[symbol].may_be_unbound() {
Boundness::PossiblyUnbound
} else {
Boundness::Bound
}
}
pub(crate) fn bindings_at_declaration(
&self,
declaration: Definition<'db>,
@@ -310,10 +336,10 @@ impl<'db> UseDefMap<'db> {
}
}
pub(crate) fn declarations_at_binding(
&self,
pub(crate) fn declarations_at_binding<'map>(
&'map self,
binding: Definition<'db>,
) -> DeclarationsIterator<'_, 'db> {
) -> DeclarationsIterator<'map, 'db> {
if let SymbolDefinitions::Declarations(declarations) =
&self.definitions_by_definition[&binding]
{
@@ -323,37 +349,34 @@ impl<'db> UseDefMap<'db> {
}
}
pub(crate) fn public_declarations(
&self,
pub(crate) fn public_declarations<'map>(
&'map self,
symbol: ScopedSymbolId,
) -> DeclarationsIterator<'_, 'db> {
) -> DeclarationsIterator<'map, 'db> {
let declarations = self.public_symbols[symbol].declarations();
self.declarations_iterator(declarations)
}
pub(crate) fn has_public_declarations(&self, symbol: ScopedSymbolId) -> bool {
!self.public_symbols[symbol].declarations().is_empty()
}
fn bindings_iterator<'a>(
&'a self,
bindings: &'a SymbolBindings,
) -> BindingWithConstraintsIterator<'a, 'db> {
fn bindings_iterator<'map>(
&'map self,
bindings: &'map SymbolBindings,
) -> BindingWithConstraintsIterator<'map, 'db> {
BindingWithConstraintsIterator {
all_definitions: &self.all_definitions,
all_constraints: &self.all_constraints,
visibility_constraints: &self.visibility_constraints,
inner: bindings.iter(),
}
}
fn declarations_iterator<'a>(
&'a self,
declarations: &'a SymbolDeclarations,
) -> DeclarationsIterator<'a, 'db> {
fn declarations_iterator<'map>(
&'map self,
declarations: &'map SymbolDeclarations,
) -> DeclarationsIterator<'map, 'db> {
DeclarationsIterator {
all_definitions: &self.all_definitions,
visibility_constraints: &self.visibility_constraints,
inner: declarations.iter(),
may_be_undeclared: declarations.may_be_undeclared(),
}
}
}
@@ -367,8 +390,9 @@ enum SymbolDefinitions {
#[derive(Debug)]
pub(crate) struct BindingWithConstraintsIterator<'map, 'db> {
all_definitions: &'map IndexVec<ScopedDefinitionId, Definition<'db>>,
all_constraints: &'map IndexVec<ScopedConstraintId, Constraint<'db>>,
all_definitions: &'map IndexVec<ScopedDefinitionId, Option<Definition<'db>>>,
all_constraints: &'map AllConstraints<'db>,
pub(crate) visibility_constraints: &'map VisibilityConstraints<'db>,
inner: BindingIdWithConstraintsIterator<'map>,
}
@@ -376,14 +400,17 @@ impl<'map, 'db> Iterator for BindingWithConstraintsIterator<'map, 'db> {
type Item = BindingWithConstraints<'map, 'db>;
fn next(&mut self) -> Option<Self::Item> {
let all_constraints = self.all_constraints;
self.inner
.next()
.map(|def_id_with_constraints| BindingWithConstraints {
binding: self.all_definitions[def_id_with_constraints.definition],
.map(|binding_id_with_constraints| BindingWithConstraints {
binding: self.all_definitions[binding_id_with_constraints.definition],
constraints: ConstraintsIterator {
all_constraints: self.all_constraints,
constraint_ids: def_id_with_constraints.constraint_ids,
all_constraints,
constraint_ids: binding_id_with_constraints.constraint_ids,
},
visibility_constraint: binding_id_with_constraints.visibility_constraint,
})
}
}
@@ -391,12 +418,13 @@ impl<'map, 'db> Iterator for BindingWithConstraintsIterator<'map, 'db> {
impl std::iter::FusedIterator for BindingWithConstraintsIterator<'_, '_> {}
pub(crate) struct BindingWithConstraints<'map, 'db> {
pub(crate) binding: Definition<'db>,
pub(crate) binding: Option<Definition<'db>>,
pub(crate) constraints: ConstraintsIterator<'map, 'db>,
pub(crate) visibility_constraint: ScopedVisibilityConstraintId,
}
pub(crate) struct ConstraintsIterator<'map, 'db> {
all_constraints: &'map IndexVec<ScopedConstraintId, Constraint<'db>>,
all_constraints: &'map AllConstraints<'db>,
constraint_ids: ConstraintIdIterator<'map>,
}
@@ -413,22 +441,31 @@ impl<'db> Iterator for ConstraintsIterator<'_, 'db> {
impl std::iter::FusedIterator for ConstraintsIterator<'_, '_> {}
pub(crate) struct DeclarationsIterator<'map, 'db> {
all_definitions: &'map IndexVec<ScopedDefinitionId, Definition<'db>>,
all_definitions: &'map IndexVec<ScopedDefinitionId, Option<Definition<'db>>>,
pub(crate) visibility_constraints: &'map VisibilityConstraints<'db>,
inner: DeclarationIdIterator<'map>,
may_be_undeclared: bool,
}
impl DeclarationsIterator<'_, '_> {
pub(crate) fn may_be_undeclared(&self) -> bool {
self.may_be_undeclared
}
pub(crate) struct DeclarationWithConstraint<'db> {
pub(crate) declaration: Option<Definition<'db>>,
pub(crate) visibility_constraint: ScopedVisibilityConstraintId,
}
impl<'db> Iterator for DeclarationsIterator<'_, 'db> {
type Item = Definition<'db>;
type Item = DeclarationWithConstraint<'db>;
fn next(&mut self) -> Option<Self::Item> {
self.inner.next().map(|def_id| self.all_definitions[def_id])
self.inner.next().map(
|DeclarationIdWithConstraint {
definition,
visibility_constraint,
}| {
DeclarationWithConstraint {
declaration: self.all_definitions[definition],
visibility_constraint,
}
},
)
}
}
@@ -438,15 +475,25 @@ impl std::iter::FusedIterator for DeclarationsIterator<'_, '_> {}
#[derive(Clone, Debug)]
pub(super) struct FlowSnapshot {
symbol_states: IndexVec<ScopedSymbolId, SymbolState>,
scope_start_visibility: ScopedVisibilityConstraintId,
}
#[derive(Debug, Default)]
#[derive(Debug)]
pub(super) struct UseDefMapBuilder<'db> {
/// Append-only array of [`Definition`].
all_definitions: IndexVec<ScopedDefinitionId, Definition<'db>>,
all_definitions: IndexVec<ScopedDefinitionId, Option<Definition<'db>>>,
/// Append-only array of [`Constraint`].
all_constraints: IndexVec<ScopedConstraintId, Constraint<'db>>,
all_constraints: AllConstraints<'db>,
/// Append-only array of [`VisibilityConstraint`].
visibility_constraints: VisibilityConstraints<'db>,
/// A constraint which describes the visibility of the unbound/undeclared state, i.e.
/// whether or not the start of the scope is visible. This is important for cases like
/// `if True: x = 1; use(x)` where we need to hide the implicit "x = unbound" binding
/// in the "else" branch.
scope_start_visibility: ScopedVisibilityConstraintId,
/// Live bindings at each so-far-recorded use.
bindings_by_use: IndexVec<ScopedUseId, SymbolBindings>,
@@ -458,14 +505,30 @@ pub(super) struct UseDefMapBuilder<'db> {
symbol_states: IndexVec<ScopedSymbolId, SymbolState>,
}
impl Default for UseDefMapBuilder<'_> {
fn default() -> Self {
Self {
all_definitions: IndexVec::from_iter([None]),
all_constraints: IndexVec::new(),
visibility_constraints: VisibilityConstraints::default(),
scope_start_visibility: ScopedVisibilityConstraintId::ALWAYS_TRUE,
bindings_by_use: IndexVec::new(),
definitions_by_definition: FxHashMap::default(),
symbol_states: IndexVec::new(),
}
}
}
impl<'db> UseDefMapBuilder<'db> {
pub(super) fn add_symbol(&mut self, symbol: ScopedSymbolId) {
let new_symbol = self.symbol_states.push(SymbolState::undefined());
let new_symbol = self
.symbol_states
.push(SymbolState::undefined(self.scope_start_visibility));
debug_assert_eq!(symbol, new_symbol);
}
pub(super) fn record_binding(&mut self, symbol: ScopedSymbolId, binding: Definition<'db>) {
let def_id = self.all_definitions.push(binding);
let def_id = self.all_definitions.push(Some(binding));
let symbol_state = &mut self.symbol_states[symbol];
self.definitions_by_definition.insert(
binding,
@@ -474,10 +537,82 @@ impl<'db> UseDefMapBuilder<'db> {
symbol_state.record_binding(def_id);
}
pub(super) fn record_constraint(&mut self, constraint: Constraint<'db>) {
let constraint_id = self.all_constraints.push(constraint);
pub(super) fn add_constraint(&mut self, constraint: Constraint<'db>) -> ScopedConstraintId {
self.all_constraints.push(constraint)
}
pub(super) fn record_constraint_id(&mut self, constraint: ScopedConstraintId) {
for state in &mut self.symbol_states {
state.record_constraint(constraint_id);
state.record_constraint(constraint);
}
}
pub(super) fn record_constraint(&mut self, constraint: Constraint<'db>) -> ScopedConstraintId {
let new_constraint_id = self.add_constraint(constraint);
self.record_constraint_id(new_constraint_id);
new_constraint_id
}
pub(super) fn add_visibility_constraint(
&mut self,
constraint: VisibilityConstraint<'db>,
) -> ScopedVisibilityConstraintId {
self.visibility_constraints.add(constraint)
}
pub(super) fn record_visibility_constraint_id(
&mut self,
constraint: ScopedVisibilityConstraintId,
) {
for state in &mut self.symbol_states {
state.record_visibility_constraint(&mut self.visibility_constraints, constraint);
}
self.scope_start_visibility = self
.visibility_constraints
.add_and_constraint(self.scope_start_visibility, constraint);
}
pub(super) fn record_visibility_constraint(
&mut self,
constraint: VisibilityConstraint<'db>,
) -> ScopedVisibilityConstraintId {
let new_constraint_id = self.add_visibility_constraint(constraint);
self.record_visibility_constraint_id(new_constraint_id);
new_constraint_id
}
/// This method resets the visibility constraints for all symbols to a previous state
/// *if* there have been no new declarations or bindings since then. Consider the
/// following example:
/// ```py
/// x = 0
/// y = 0
/// if test_a:
/// y = 1
/// elif test_b:
/// y = 2
/// elif test_c:
/// y = 3
///
/// # RESET
/// ```
/// We build a complex visibility constraint for the `y = 0` binding. We build the same
/// constraint for the `x = 0` binding as well, but at the `RESET` point, we can get rid
/// of it, as the `if`-`elif`-`elif` chain doesn't include any new bindings of `x`.
pub(super) fn simplify_visibility_constraints(&mut self, snapshot: FlowSnapshot) {
debug_assert!(self.symbol_states.len() >= snapshot.symbol_states.len());
self.scope_start_visibility = snapshot.scope_start_visibility;
// Note that this loop terminates when we reach a symbol not present in the snapshot.
// This means we keep visibility constraints for all new symbols, which is intended,
// since these symbols have been introduced in the corresponding branch, which might
// be subject to visibility constraints. We only simplify/reset visibility constraints
// for symbols that have the same bindings and declarations present compared to the
// snapshot.
for (current, snapshot) in self.symbol_states.iter_mut().zip(snapshot.symbol_states) {
current.simplify_visibility_constraints(snapshot);
}
}
@@ -486,7 +621,7 @@ impl<'db> UseDefMapBuilder<'db> {
symbol: ScopedSymbolId,
declaration: Definition<'db>,
) {
let def_id = self.all_definitions.push(declaration);
let def_id = self.all_definitions.push(Some(declaration));
let symbol_state = &mut self.symbol_states[symbol];
self.definitions_by_definition.insert(
declaration,
@@ -501,7 +636,7 @@ impl<'db> UseDefMapBuilder<'db> {
definition: Definition<'db>,
) {
// We don't need to store anything in self.definitions_by_definition.
let def_id = self.all_definitions.push(definition);
let def_id = self.all_definitions.push(Some(definition));
let symbol_state = &mut self.symbol_states[symbol];
symbol_state.record_declaration(def_id);
symbol_state.record_binding(def_id);
@@ -520,6 +655,7 @@ impl<'db> UseDefMapBuilder<'db> {
pub(super) fn snapshot(&self) -> FlowSnapshot {
FlowSnapshot {
symbol_states: self.symbol_states.clone(),
scope_start_visibility: self.scope_start_visibility,
}
}
@@ -533,12 +669,15 @@ impl<'db> UseDefMapBuilder<'db> {
// Restore the current visible-definitions state to the given snapshot.
self.symbol_states = snapshot.symbol_states;
self.scope_start_visibility = snapshot.scope_start_visibility;
// If the snapshot we are restoring is missing some symbols we've recorded since, we need
// to fill them in so the symbol IDs continue to line up. Since they don't exist in the
// snapshot, the correct state to fill them in with is "undefined".
self.symbol_states
.resize(num_symbols, SymbolState::undefined());
self.symbol_states.resize(
num_symbols,
SymbolState::undefined(self.scope_start_visibility),
);
}
/// Merge the given snapshot into the current state, reflecting that we might have taken either
@@ -553,13 +692,19 @@ impl<'db> UseDefMapBuilder<'db> {
let mut snapshot_definitions_iter = snapshot.symbol_states.into_iter();
for current in &mut self.symbol_states {
if let Some(snapshot) = snapshot_definitions_iter.next() {
current.merge(snapshot);
current.merge(snapshot, &mut self.visibility_constraints);
} else {
current.merge(
SymbolState::undefined(snapshot.scope_start_visibility),
&mut self.visibility_constraints,
);
// Symbol not present in snapshot, so it's unbound/undeclared from that path.
current.set_may_be_unbound();
current.set_may_be_undeclared();
}
}
self.scope_start_visibility = self
.visibility_constraints
.add_or_constraint(self.scope_start_visibility, snapshot.scope_start_visibility);
}
pub(super) fn finish(mut self) -> UseDefMap<'db> {
@@ -572,6 +717,7 @@ impl<'db> UseDefMapBuilder<'db> {
UseDefMap {
all_definitions: self.all_definitions,
all_constraints: self.all_constraints,
visibility_constraints: self.visibility_constraints,
bindings_by_use: self.bindings_by_use,
public_symbols: self.symbol_states,
definitions_by_definition: self.definitions_by_definition,

Some files were not shown because too many files have changed in this diff Show More