Compare commits

...

89 Commits

Author SHA1 Message Date
Dhruv Manilawala
2b8c801a79 Make tracing aware that walkdir threads run as part of request 2025-01-16 14:09:16 +05:30
Dhruv Manilawala
79e52c7fdf [pyflakes] Show syntax error message for F722 (#15523)
## Summary

Ref: https://github.com/astral-sh/ruff/pull/15387#discussion_r1917796907

This PR updates `F722` to show syntax error message instead of the
string content.

I think it's more useful to show the syntax error message than the
string content. In the future, when the diagnostics renderer is more
capable, we could even highlight the exact location of the syntax error
along with the annotation string.

This is also in line with how we show the diagnostic in red knot.

## Test Plan

Update existing test snapshots.
2025-01-16 12:44:01 +05:30
Shaygan Hooshyari
cf4ab7cba1 Parse triple quoted string annotations as if parenthesized (#15387)
## Summary

Resolves #9467 

Parse quoted annotations as if the string content is inside parenthesis.
With this logic `x` and `y` in this example are equal:

```python
y: """
   int |
   str
"""

z: """(
    int |
    str
)
"""
```

Also this rule only applies to triple
quotes([link](https://github.com/python/typing-council/issues/9#issuecomment-1890808610)).

This PR is based on the
[comments](https://github.com/astral-sh/ruff/issues/9467#issuecomment-2579180991)
on the issue.

I did one extra change, since we don't want any indentation tokens I am
setting the `State::Other` as the initial state of the Lexer.

Remaining work:

- [x] Add a test case for red-knot.
- [x] Add more tests.

## Test Plan

Added a test which previously failed because quoted annotation contained
indentation.
Added an mdtest for red-knot.
Updated previous test.

Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
Co-authored-by: Micha Reiser <micha@reiser.io>
2025-01-16 11:38:15 +05:30
Dylan
d2656e88a3 [flake8-todos] Allow VSCode GitHub PR extension style links in missing-todo-link (TD003) (#15519)
## Summary
Allow links to issues that appear on the same line as the TODO
directive, if they conform to the format that VSCode's GitHub PR
extension produces.

Revival of #9627 (the branch was stale enough that rebasing was a lot
harder than just making the changes anew). Credit should go to the
author of that PR though.

Closes #8061

Co-authored-by: Martin Bernstorff <martinbernstorff@gmail.com>
2025-01-15 23:47:33 +00:00
Alex Waygood
c53ee608a1 Typeshed-sync workflow: add appropriate labels, link directly to failing run (#15520) 2025-01-15 23:42:35 +00:00
David Peter
c034e280a9 [red-knot] Instance attributes: type inference clarifications (#15512)
## Summary

Some clarifications in the instance-attributes tests, mostly regarding
type inference behavior following this discussion:

https://github.com/astral-sh/ruff/pull/15474#discussion_r1917044566
2025-01-15 21:17:55 +01:00
Alex Waygood
49557a9129 [red-knot] Simplify object out of intersections (#15511) 2025-01-15 20:06:48 +00:00
Andrew Gallant
c9b99e4bee ruff_linter: adjust empty spans after line terminator more generally
Instead of doing this on a lint-by-lint basis, we now just do it right
before rendering. This is more broadly applicable.

Note that this doesn't fix the diagnostic rendering for the Python
parser. But that's using a different path anyway (`annotate-snippets` is
only used in tests).
2025-01-15 13:37:52 -05:00
Andrew Gallant
2ff2a54f56 test: update a few indentation related diagnostics
Previously, these were pointing to the right place, but were missing the
`^`. With the `annotate-snippets` upgrade, the `^` was added, but they
started pointing to the end of the previous line instead of the
beginning of the following line. In this case, we really want it to
point to the beginning of the following line since we're calling out
indentation issues.

As in a prior commit, we fix this by tweaking the offsets emitted by the
lint itself. Instead of an empty range at the beginning of the line, we
point to the first character in the line. This "forces" the renderer to
point to the beginning of the line instead of the end of the preceding
line.

The end effect here is that the rendering is fixed by adding `^` in the
proper location.
2025-01-15 13:37:52 -05:00
Andrew Gallant
17f01a4355 test: add more missing carets
This update includes some missing `^` in the diagnostic annotations.

This update also includes some shifting of "syntax error" annotations to
the end of the preceding line. I believe this is technically a
regression, but fixing them has proven quite difficult. I *think* the
best way to do that might be to tweak the spans generated by the Python
parser errors, but I didn't want to dig into that. (Another approach
would be to change the `annotate-snippets` rendering, but when I tried
that and managed to fix these regressions, I ended up causing a bunch of
other regressions.)

Ref 77d454525e (r1915458616)
2025-01-15 13:37:52 -05:00
Andrew Gallant
5021f32449 test: another update to add back a caret
This change also requires some shuffling to the offsets we generate for
the diagnostic. Previously, we were generating an empty range
immediately *after* the line terminator and immediate before the first
byte of the subsequent line. How this is rendered is somewhat open to
interpretation, but the new version of `annotate-snippets` chooses to
render this at the end of the preceding line instead of the beginning of
the following line.

In this case, we want the diagnostic to point to the beginning of the
following line. So we either need to change `annotate-snippets` to
render such spans at the beginning of the following line, or we need to
change our span to point to the first full character in the following
line. The latter will force `annotate-snippets` to move the caret to the
proper location.

I ended up deciding to change our spans instead of changing how
`annotate-snippets` renders empty spans after a line terminator. While I
didn't investigate it, my guess is that they probably had good reason
for doing so, and it doesn't necessarily strike me as _wrong_.
Furthermore, fixing up our spans seems like a good idea regardless, and
was pretty easy to do.
2025-01-15 13:37:52 -05:00
Andrew Gallant
75b4ed5ad1 codeowners: make BurntSushi owner of ruff_annotate_snippets 2025-01-15 13:37:52 -05:00
Andrew Gallant
e6e610c274 test: tweak in alignment involving unprintable characters
This looks like a bug fix since the caret is now pointing right at the
position of the unprintable character. I'm not sure if this is a result
of an improvement via the `annotate-snippets` upgrade, or because of
more accurate tracking of annotation ranges even after unprintable
characters are replaced. I'm tempted to say the former since in theory
the offsets were never wrong before because they were codepoint offsets.

Regardless, this looks like an improvement.
2025-01-15 13:37:52 -05:00
Andrew Gallant
670fcecd1b test: update snapshots with trimmed lines
This updates snapshots where long lines now get trimmed with
`annotate-snippets`. And an ellipsis is inserted to indicate trimming.

This is a little hokey to test since in tests we don't do any styling.
And I believe this just uses the default "max term width" for rendering.
But in real life, it seems like a big improvement to have long lines
trimmed if they would otherwise wrap in the terminal. So this seems like
an improvement to me.

There are some other fixes here that overlap with previous categories.
2025-01-15 13:37:52 -05:00
Andrew Gallant
84ba4ecaf5 ruff_annotate_snippets: support overriding the "cut indicator"
We do this because `...` is valid Python, which makes it pretty likely
that some line trimming will lead to ambiguous output. So we add support
for overriding the cut indicator. This also requires changing some of
the alignment math, which was previously tightly coupled to `...`.

For Ruff, we go with `…` (`U+2026 HORIZONTAL ELLIPSIS`) for our cut
indicator.

For more details, see the patch sent to upstream:
https://github.com/rust-lang/annotate-snippets-rs/pull/172
2025-01-15 13:37:52 -05:00
Andrew Gallant
a45f4de683 ruff_annotate_snippets: fix false positive line trimming
This fix was sent upstream and the PR description includes more details:
https://github.com/rust-lang/annotate-snippets-rs/pull/170

Without this fix, there was an errant snapshot diff that looked like
this:

  |
1 |   version = "0.1.0"
2 |   # Ensure that the spans from toml handle utf-8 correctly
3 |   authors = [
  |  ___________^
4 | |     { name = "Z͑ͫ̓ͪ̂ͫ̽͏̴̙...A̴̵̜̰͔ͫ͗͢L̠ͨͧͩ͘G̴̻͈͍̑͗̎̅͛́Ǫ̵̹̻̝̳͂̌̌͘", email = 1 }
5 | | ]
  | |_^ RUF200
  |

That ellipsis should _not_ be inserted since the line is not actually
truncated. The handling of line length (in bytes versus actual rendered
length) wasn't quite being handled correctly in all cases.

With this fix, there's (correctly) no snapshot diff.
2025-01-15 13:37:52 -05:00
Andrew Gallant
88df168b63 ruff_annotate_snippets: update snapshot for single ASCII whitespace source
The change to the rendering code is elaborated on in more detail here,
where I attempted to upstream it:
https://github.com/rust-lang/annotate-snippets-rs/pull/169

Otherwise, the snapshot diff also shows a bug fix: a `^` is now rendered
where as it previously was not.
2025-01-15 13:37:52 -05:00
Andrew Gallant
59edee2aca test: update another improperly rendered range
This one almost looks like it fits into the other failure categories,
but without identifying root causes, it's hard to say for sure. The span
here does end after a line terminator, so it feels like it's like the
rest.

I also isolated this change since I found the snapshot diff pretty hard
to read and wanted to look at it more closely. In this case, the before
is:

    E204.py:31:2: E204 [*] Whitespace after decorator
       |
    30 |   # E204
    31 |   @ \
       |  __^
    32 | | foo
       | |_^ E204
    33 |   def baz():
    34 |       print('baz')
       |
       = help: Remove whitespace

And the after is:

    E204.py:31:2: E204 [*] Whitespace after decorator
       |
    30 | # E204
    31 | @ \
       |  ^^ E204
    32 | foo
    33 | def baz():
    34 |     print('baz')
       |
       = help: Remove whitespace

The updated rendering is clearly an improvement, since `foo` itself is
not really the subject of the diagnostic. The whitespace is.

Also, the new rendering matches the span fed to `annotate-snippets`,
where as the old rendering does not.
2025-01-15 13:37:52 -05:00
Andrew Gallant
9fdb1e9bc8 test: update snapshot with fixed annotation but carets include whitespace
I separated out this snapshot update since the string of `^` including
whitespace looked a little odd. I investigated this one specifically,
and indeed, our span in this case is telling `annotate-snippets` to
point at the whitespace. So this is `annotate-snippets` doing what it's
told with a mildly sub-optimal span.

For clarity, the before rendering is:

    skip.py:34:1: I001 [*] Import block is un-sorted or un-formatted
       |
    32 |       import sys; import os  # isort:skip
    33 |       import sys; import os  # isort:skip  # isort:skip
    34 | /     import sys; import os
       |
       = help: Organize imports

And now after is:

    skip.py:34:1: I001 [*] Import block is un-sorted or un-formatted
       |
    32 |     import sys; import os  # isort:skip
    33 |     import sys; import os  # isort:skip  # isort:skip
    34 |     import sys; import os
       | ^^^^^^^^^^^^^^^^^^^^^^^^^ I001
       |
       = help: Organize imports

This is a clear bug fix since it adds in the `I001` annotation, even
though the carets look a little funny by including the whitespace
preceding `import sys; import os`.
2025-01-15 13:37:52 -05:00
Andrew Gallant
eed0595b18 test: another set of updates related to line terminator handling
This group of updates is similar to the last one, but they call out the
fact that while the change is an improvement, it does still seem to be a
little buggy.

As one example, previously we would have this:

       |
     1 | / from __future__ import annotations
     2 | |
     3 | | from typing import Any
     4 | |
     5 | | from requests import Session
     6 | |
     7 | | from my_first_party import my_first_party_object
     8 | |
     9 | | from . import my_local_folder_object
    10 | |
    11 | |
    12 | |
    13 | | class Thing(object):
       | |_^ I001
    14 |     name: str
    15 |     def __init__(self, name: str):
       |
       = help: Organize imports

And now here's what it looks like after:

       |
     1 | / from __future__ import annotations
     2 | |
     3 | | from typing import Any
     4 | |
     5 | | from requests import Session
     6 | |
     7 | | from my_first_party import my_first_party_object
     8 | |
     9 | | from . import my_local_folder_object
    10 | |
    11 | |
    12 | |
       | |__^ Organize imports
    13 |   class Thing(object):
    14 |     name: str
    15 |     def __init__(self, name: str):
       |
       = help: Organize imports

So at least now, the diagnostic is not pointing to a completely
unrelated thing (`class Thing`), but it's still not quite pointing to
the imports directly. And the `^` is a bit offset. After looking at
some examples more closely, I think this is probably more of a bug
with how we're generating offsets, since we are actually pointing to
a location that is a few empty lines _below_ the last import. And
`annotate-snippets` is rendering that part correctly. However, the
offset from the left (the `^` is pointing at `r` instead of `f` or even
at the end of `from . import my_local_folder_object`) appears to be a
problem with `annotate-snippets` itself.

We accept this under the reasoning that it's an improvement, albeit not
perfect.
2025-01-15 13:37:52 -05:00
Andrew Gallant
79e71cbbcd test: another line terminator bug fix
I believe this case is different from the last in that it happens when
the end of a *multi-line* annotation occurs after a line terminator.
Previously, the diagnostic would render on the next line, which is
definitely a bit weird. This new update renders it at the end of the
line the annotation ends on.

In some cases, the annotation was previously rendered to point at source
lines below where the error occurred, which is probably pretty
confusing.
2025-01-15 13:37:52 -05:00
Andrew Gallant
5caef89af3 test: update snapshots with improper end-of-line placement
This looks like a bug fix that occurs when the annotation is a
zero-width span immediately following a line terminator. Previously, the
caret seems to be rendered on the next line, but it should be rendered
at the end of the line the span corresponds to.

I admit that this one is kinda weird. I would somewhat expect that our
spans here are actually incorrect, and that to obtain this sort of
rendering, we should identify a span just immediately _before_ the line
terminator and not after it. But I don't want to dive into that rabbit
hole for now (and given how `annotate-snippets` now renders these
spans, perhaps there is more to it than I see), and this does seem like
a clear improvement given the spans we feed to `annotate-snippets`.
2025-01-15 13:37:52 -05:00
Andrew Gallant
f49cfb6c28 test: update snapshots with missing ^
The previous rendering just seems wrong in that a `^` is omitted. The
new version of `annotate-snippets` seems to get this right. I checked a
pseudo random sample of these, and it seems to only happen when the
position pointed at a line terminator.
2025-01-15 13:37:52 -05:00
Andrew Gallant
f29f58105b test: update formatting of multi-line annotations
It's hard to grok the change from the snapshot diffs alone, so here's
one example. Before:

    PYI021.pyi:15:5: PYI021 [*] Docstrings should not be included in stubs
       |
    14 |   class Baz:
    15 |       """Multiline docstring
       |  _____^
    16 | |
    17 | |     Lorem ipsum dolor sit amet
    18 | |     """
       | |_______^ PYI021
    19 |
    20 |       def __init__(self) -> None: ...
       |
       = help: Remove docstring

And now after:

    PYI021.pyi:15:5: PYI021 [*] Docstrings should not be included in stubs
       |
    14 |   class Baz:
    15 | /     """Multiline docstring
    16 | |
    17 | |     Lorem ipsum dolor sit amet
    18 | |     """
       | |_______^ PYI021
    19 |
    20 |       def __init__(self) -> None: ...
       |
       = help: Remove docstring

I personally think both of these are fine. If we felt strongly, I could
investigate reverting to the old style, but the new style seems okay to
me.

In other words, these updates I believe are just cosmetic and not a bug
fix.
2025-01-15 13:37:52 -05:00
Andrew Gallant
3fa4479c85 test: update snapshots with missing annotations
These updates center around the addition of annotations in the
diagnostic rendering. Previously, the annotation was just not rendered
at all. With the `annotate-snippets` upgrade, it is now rendered. I
examined a pseudo random sample of these, and they all look correct.

As will be true in future batches, some of these snapshots also have
changes to whitespace in them as well.
2025-01-15 13:37:52 -05:00
Andrew Gallant
0de8216a25 test: update snapshots with just whitespace changes
These snapshot changes should *all* only be a result of changes to
trailing whitespace in the output. I checked a psuedo random sample of
these, and the whitespace found in the previous snapshots seems to be an
artifact of the rendering and _not_ of the source data. So this seems
like a strict bug fix to me.

There are other snapshots with whitespace changes, but they also have
other changes that we split out into separate commits. Basically, we're
going to do approximately one commit per category of change.

This represents, by far, the biggest chunk of changes to snapshots as a
result of the `annotate-snippets` upgrade.
2025-01-15 13:37:52 -05:00
Andrew Gallant
2922490cb8 ruff_linter: fix handling of unprintable characters
Previously, we were replacing unprintable ASCII characters with a
printable representation of them via fancier Unicode characters. Since
`annotate-snippets` used to use codepoint offsets, this didn't make our
ranges incorrect: we swapped one codepoint for another.

But now, with the `annotate-snippets` upgrade, we use byte offsets
(which is IMO the correct choice). However, this means our ranges can be
thrown off since an ASCII codepoint is always one byte and a non-ASCII
codepoint is always more than one byte.

Instead of tweaking the `ShowNonprinting` trait and making it more
complicated (which is used in places other than this diagnostic
rendering it seems), we instead change `replace_whitespace` to handle
non-printable characters. This works out because `replace_whitespace`
was already updating the annotation range to account for the tab
replacement. We copy that approach for unprintable characters.
2025-01-15 13:37:52 -05:00
Andrew Gallant
84179aaa96 ruff_linter,ruff_python_parser: migrate to updated annotate-snippets
This is pretty much just moving to the new API and taking care to use
byte offsets. This is *almost* enough. The next commit will fix a bug
involving the handling of unprintable characters as a result of
switching to byte offsets.
2025-01-15 13:37:52 -05:00
Andrew Gallant
1b97677779 ruff_annotate_snippets: make small change to enable omitting header
This is a tiny change that, perhaps slightly shady, permits us to use
the `annotate-snippets` renderer without its mandatory header (which
wasn't there in `annotate-snippets 0.9`). Specifically, we can now do
this:

    Level::None.title("")

The combination of a "none" level and an empty label results in the
`annotate-snippets` header being skipped entirely. (Not even an empty
line is written.)

This is maybe not the right API for upstream `annotate-snippets`, but
it's very easy for us to do and unblocks the upgrade (albeit relying on
a vendored copy).

Ref https://github.com/rust-lang/annotate-snippets-rs/issues/167
2025-01-15 13:37:52 -05:00
Andrew Gallant
9c27c57b5b crates: vendor annotate-snippets crate
This merely adds the crate to our repository. Some cosmetic changes are
made to make it work in our repo and follow our conventions, such as
changing the name to `ruff_annotate_snippets`. We retain the original
license information. We do drop some things, such as benchmarks, but
keep tests and examples.
2025-01-15 13:37:52 -05:00
David Peter
4f3209a3ec [red-knot] More comprehensive 'is_subtype_of' tests (#15490)
## Summary

Make the `is_subtype_of` tests a bit easier to understand and
more comprehensive.
2025-01-15 18:33:29 +00:00
Brent Westbrook
1a77a75935 [FastAPI] Update Annotated fixes (FAST002) (#15462)
## Summary

The initial purpose was to fix #15043, where code like this:
```python
from fastapi import FastAPI, Query

app = FastAPI()

@app.get("/test")
def handler(echo: str = Query("")):
    return echo
```

was being fixed to the invalid code below:

```python
from typing import Annotated
from fastapi import FastAPI, Query

app = FastAPI()

@app.get("/test")
def handler(echo: Annotated[str, Query("")]): # changed
    return echo
```

As @MichaReiser pointed out, the correct fix is:

```python
from typing import Annotated
from fastapi import FastAPI, Query

app = FastAPI()

@app.get("/test")
def handler(echo: Annotated[str, Query()] = ""): # changed
    return echo 
```

After fixing the issue for `Query`, I realized that other classes like
`Path`, `Body`, `Cookie`, `Header`, `File`, and `Form` also looked
susceptible to this issue. The last few commits should handle these too,
which I think means this will also close #12913.

I had to reorder the arguments to the `do_stuff` test case because the
new fix removes some default argument values (eg for `Path`:
`some_path_param: str = Path()` becomes `some_path_param: Annotated[str,
Path()]`).

There's also #14484 related to this rule. I'm happy to take a stab at
that here or in a follow up PR too.

## Test Plan

`cargo test`

I also checked the fixed output with `uv run --with fastapi
FAST002_0.py`, but it required making a bunch of additional changes to
the test file that I wasn't sure we wanted in this PR.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2025-01-15 13:05:53 -05:00
David Peter
48e6541893 [red-knot] Negation reverses subtyping order (#15503)
## Summary

If `S <: T`, then `~T <: ~S`. This test currently fails with example
like:

```
S = tuple[()]
T = ~Literal[True] & ~Literal[False]
```

`T` is equivalent to `~(Literal[True] | Literal[False])` and therefore
equivalent to `~bool`, but the minimal example for a failure is what is
stated above. We correctly recognize that `S <: T`, but fail to see that
`~T <: ~S`, i.e. `bool <: ~tuple[()]`.

This is why the tests goes into the "flaky" section as well.

## Test Plan

```
export QUICKCHECK_TESTS=100000
while cargo test --release -p red_knot_python_semantic -- --ignored types::property_tests::flaky::negation_reverses_subtype_order; do :; done
```
2025-01-15 16:32:21 +01:00
Alex Waygood
55a7f72035 [red-knot] Fix more edge cases for intersection simplification with LiteralString and AlwaysTruthy/AlwaysFalsy (#15496) 2025-01-15 15:02:41 +00:00
David Peter
8712438aec [red-knot] Initial tests for instance attributes (#15474)
## Summary

Adds some initial tests for class and instance attributes, mostly to
document (and discuss) what we want to support eventually. These
tests are not exhaustive yet. The idea is to specify the coarse-grained
behavior first.

Things that we'll eventually want to test:

- Interplay with inheritance
- Support `Final` in addition to `ClassVar`
- Specific tests for `ClassVar`, like making sure that we support things
like `x: Annotated[ClassVar[int], "metadata"]`
- … or making sure that we raise an error here:
  ```py
  class Foo:
      def __init__(self):
          self.x: ClassVar[str] = "x"
  ```
- Add tests for `__new__` in addition to the tests for `__init__`
- Add tests that show that we use the union of types if multiple methods
define the symbol with different types
- Make sure that diagnostics are raised if, e.g., the inferred type of
an assignment within a method does not match the declared type in the
class body.
- https://github.com/astral-sh/ruff/pull/15474#discussion_r1916556284
- Method calls are completely left out for now.
- Same for `@property`
- … and the descriptor protocol

## Test Plan

New Markdown tests

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-01-15 14:43:41 +00:00
Dhruv Manilawala
b5dbb2a1d7 Avoid indexing the same workspace multiple times (#15495)
## Summary

This is not lazy indexing but it should somewhat help with #13686.

Currently, processing the change notifications for config files doesn't
account for the fact that multiple config files could belong to the same
workspace. This means that the server will re-index the same workspace
`n` times where `n` is the number of file events which belongs to the
same workspace. This is evident in the following trace logs:

**Trace logs:**

```
[Trace - 6:21:15 PM] Sending notification 'workspace/didChangeWatchedFiles'.
Params: {
    "changes": [
        {
            "uri": "file:///Users/dhruv/work/astral/parser-checkouts/home-assistant-core/pylint/ruff.toml",
            "type": 1
        },
        {
            "uri": "file:///Users/dhruv/work/astral/parser-checkouts/home-assistant-core/script/ruff.toml",
            "type": 2
        },
        {
            "uri": "file:///Users/dhruv/work/astral/parser-checkouts/home-assistant-core/script/scaffold/templates/ruff.toml",
            "type": 2
        },
        {
            "uri": "file:///Users/dhruv/work/astral/parser-checkouts/home-assistant-core/pyproject.toml",
            "type": 2
        }
    ]
}

...

[Trace - 6:21:19 PM] Sending notification 'workspace/didChangeWatchedFiles'.
Params: {
    "changes": [
        {
            "uri": "file:///Users/dhruv/work/astral/parser-checkouts/home-assistant-core/tests/testing_config/custom_components/ruff.toml",
            "type": 1
        },
        {
            "uri": "file:///Users/dhruv/work/astral/parser-checkouts/home-assistant-core/tests/ruff.toml",
            "type": 2
        }
    ]
}

...
```

**Server logs:**

```
 14.838004208s TRACE     ruff:main notification{method="workspace/didChangeWatchedFiles"}: ruff_server::server::api: enter
  14.838043583s DEBUG     ruff:main notification{method="workspace/didChangeWatchedFiles"}: ruff_server::session::index::ruff_settings: Indexing settings for workspace: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core
  14.854324541s DEBUG ThreadId(55) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/.vscode
  14.854388500s DEBUG ThreadId(55) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/.git
  14.937713291s DEBUG     ruff:main notification{method="workspace/didChangeWatchedFiles"}: ruff_server::session::index::ruff_settings: Indexing settings for workspace: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core
  14.954429833s DEBUG ThreadId(75) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/.git
  14.954675708s DEBUG ThreadId(66) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/.vscode
  15.041465500s DEBUG     ruff:main notification{method="workspace/didChangeWatchedFiles"}: ruff_server::session::index::ruff_settings: Indexing settings for workspace: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core
  15.056731541s DEBUG ThreadId(78) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/.vscode
  15.056796833s DEBUG ThreadId(78) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/.git
  15.117545833s DEBUG     ruff:main notification{method="workspace/didChangeWatchedFiles"}: ruff_server::session::index::ruff_settings: Indexing settings for workspace: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core
  15.133091666s DEBUG ThreadId(90) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/.vscode
  15.133146500s DEBUG ThreadId(90) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/.git
  15.220340666s TRACE ruff:worker:6 request{id=5 method="textDocument/diagnostic"}: ruff_server::server::api: enter
  15.220401458s DEBUG ruff:worker:6 request{id=5 method="textDocument/diagnostic"}: ruff_server::resolve: Included path via `include`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/homeassistant/bootstrap.py
  18.577521250s TRACE     ruff:main notification{method="workspace/didChangeWatchedFiles"}: ruff_server::server::api: enter
  18.577561291s DEBUG     ruff:main notification{method="workspace/didChangeWatchedFiles"}: ruff_server::session::index::ruff_settings: Indexing settings for workspace: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core
  18.616564583s DEBUG ThreadId(102) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/.vscode
  18.616627291s DEBUG ThreadId(102) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/.git
  18.687424250s DEBUG     ruff:main notification{method="workspace/didChangeWatchedFiles"}: ruff_server::session::index::ruff_settings: Indexing settings for workspace: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core
  18.704441416s DEBUG ThreadId(114) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/.vscode
  18.704694958s DEBUG ThreadId(121) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/.git
  18.769627500s TRACE ruff:worker:4 request{id=6 method="textDocument/diagnostic"}: ruff_server::server::api: enter
  18.769696791s DEBUG ruff:worker:4 request{id=6 method="textDocument/diagnostic"}: ruff_server::resolve: Included path via `include`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/homeassistant/bootstrap.py
```

This PR updates the logic to consider all the change events at once
keeping track of the workspace path that have been already indexed.

I want to include this in tomorrow's release to check how this change
would affect the linked issue.

## Test Plan

Run the same scenario as above and check the logs to see that the server
isn't re-indexing the same workspace multiple times:

**Trace logs:**

```
[Trace - 6:04:07 PM] Sending notification 'workspace/didChangeWatchedFiles'.
Params: {
    "changes": [
        {
            "uri": "file:///Users/dhruv/work/astral/parser-checkouts/home-assistant-core/script/ruff.toml",
            "type": 1
        },
        {
            "uri": "file:///Users/dhruv/work/astral/parser-checkouts/home-assistant-core/script/scaffold/templates/ruff.toml",
            "type": 1
        },
        {
            "uri": "file:///Users/dhruv/work/astral/parser-checkouts/home-assistant-core/pylint/ruff.toml",
            "type": 2
        },
        {
            "uri": "file:///Users/dhruv/work/astral/parser-checkouts/home-assistant-core/pyproject.toml",
            "type": 2
        }
    ]
}

...

[Trace - 6:04:11 PM] Sending notification 'workspace/didChangeWatchedFiles'.
Params: {
    "changes": [
        {
            "uri": "file:///Users/dhruv/work/astral/parser-checkouts/home-assistant-core/tests/testing_config/custom_components/ruff.toml",
            "type": 1
        },
        {
            "uri": "file:///Users/dhruv/work/astral/parser-checkouts/home-assistant-core/tests/ruff.toml",
            "type": 2
        }
    ]
}

...
```

**Server logs:**

```
  17.047706750s TRACE     ruff:main notification{method="workspace/didChangeWatchedFiles"}: ruff_server::server::api: enter
  17.047747875s DEBUG     ruff:main notification{method="workspace/didChangeWatchedFiles"}: ruff_server::session::index::ruff_settings: Indexing settings for workspace: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core
  17.080006083s DEBUG ThreadId(54) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/.vscode
  17.080085708s DEBUG ThreadId(54) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/.git
  17.145328791s TRACE ruff:worker:6 request{id=5 method="textDocument/diagnostic"}: ruff_server::server::api: enter
  17.145386166s DEBUG ruff:worker:6 request{id=5 method="textDocument/diagnostic"}: ruff_server::resolve: Included path via `include`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/homeassistant/bootstrap.py
  20.756845958s TRACE     ruff:main notification{method="workspace/didChangeWatchedFiles"}: ruff_server::server::api: enter
  20.756923375s DEBUG     ruff:main notification{method="workspace/didChangeWatchedFiles"}: ruff_server::session::index::ruff_settings: Indexing settings for workspace: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core
  20.781733916s DEBUG ThreadId(66) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/.vscode
  20.781825875s DEBUG ThreadId(75) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/.git
  20.848340750s TRACE ruff:worker:7 request{id=6 method="textDocument/diagnostic"}: ruff_server::server::api: enter
  20.848408041s DEBUG ruff:worker:7 request{id=6 method="textDocument/diagnostic"}: ruff_server::resolve: Included path via `include`: /Users/dhruv/work/astral/parser-checkouts/home-assistant-core/homeassistant/bootstrap.py
```
2025-01-15 18:58:28 +05:30
David Salvisberg
73488e71f8 [flake8-type-checking] Avoid false positives for | in TC008 (#15201) 2025-01-15 14:27:24 +01:00
InSync
8331326cb6 Remove legacy issue template (#15163) 2025-01-15 11:59:22 +01:00
David Peter
3a6238d8c2 [red-knot] Typeshed sync and sys.platform fixes (#15492)
## Summary

The next sync of typeshed would have failed without manual changes
anyway, so I'm doing one manual sync + the required changes in our
`sys.platform` tests (which are necessary because of my tiny typeshed PR
here: https://github.com/python/typeshed/pull/13378).

closes #15485 (the next run of the pipeline in two weeks should be fine
as the bug has been fixed upstream)
2025-01-15 11:21:01 +01:00
David Peter
d4862844f1 [red-knot] 'is_equivalent_to' is an equivalence relation (#15488)
## Summary

Adds two additional tests for `is_equivalent_to` so that we cover all
properties of an [equivalence relation].

## Test Plan

```
while cargo test --release -p red_knot_python_semantic -- --ignored types::property_tests::stable; do :; done
```

[equivalence relation]:
https://en.wikipedia.org/wiki/Equivalence_relation
2025-01-15 09:25:46 +01:00
Micha Reiser
96c2d0996d Fix curly bracket spacing around curly f-string expressions (#15471) 2025-01-15 09:22:47 +01:00
Dhruv Manilawala
6aef4ad008 Fix LSP show message macro to allow format args (#15487)
## Summary

This PR fixes the `show_*_msg` macros to pass all the tokens instead of
just a single token. This allows for using various expressions right in
the macro similar to how it would be in `format_args!`.

## Test Plan

`cargo clippy`
2025-01-15 08:11:49 +00:00
Micha Reiser
18d5dbfb7f Remove workspace support (#15472) 2025-01-15 09:03:38 +01:00
Dhruv Manilawala
bec8441cf5 Use tool specific function to perform exclude checks (#15486)
## Summary

This PR creates separate functions to check whether the document path is
excluded for linting or formatting. The main motivation is to avoid the
double `Option` for the call sites and makes passing the correct
settings simpler.
2025-01-15 13:18:46 +05:30
InSync
aefb607405 [red-knot] Migrate is_equivalent_to unit tests to Markdown tests (#15470)
## Summary

Part of #15397, built on top of #15469.

## Test Plan

Markdown tests.
2025-01-14 18:57:23 +00:00
Alex Waygood
bcf0a715c2 [red-knot] Corrections and improvements to intersection simplification (#15475) 2025-01-14 18:15:38 +00:00
InSync
5ed7b55b15 [red-knot] Migrate is_subtype_of unit tests to Markdown tests (#15469)
## Summary

Part of #15397.

## Test Plan

Markdown tests.

---------

Co-authored-by: David Peter <mail@david-peter.de>
2025-01-14 15:57:24 +01:00
David Peter
8aac69bb2e [red-knot] Add boundness and declaredness tests (#15453)
## Summary

This changeset adds new tests for public uses of symbols,
considering all possible declaredness and boundness states.

Note that this is a mere documentation of the current behavior. There is
still an [open ticket] questioning some of these choices (or unintential
behaviors).

## Test plan

Made sure that the respective test fails if I add the questionable case
again in `symbol_by_id`:

```rs
Symbol::Type(inferred_ty, Boundness::Bound) => {
    Symbol::Type(inferred_ty, Boundness::Bound)
}
```

[open ticket]: https://github.com/astral-sh/ruff/issues/14297
2025-01-14 13:07:16 +01:00
Tom Kuson
9dfc61bf09 [flake8-pytest-style] Tweak documentation and message (#15465) 2025-01-14 08:47:45 +01:00
Tom Kuson
369cbb5424 [flake8-builtins] Improve A005 documentation (#15466) 2025-01-14 08:42:13 +01:00
Garrett Reynolds
dc491e8ade [ruff] Fix false positive on global keyword (RUF052) (#15235) 2025-01-14 08:36:40 +01:00
Wei Lee
a2dc8c93ef [airflow] Replace typo "security_managr" as "security_manager" (AIR303) (#15463)
## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

Replace typo "security_managr" in AIR303 as "security_manager"

## Test Plan

<!-- How was it tested? -->

a test fixture has been updated
2025-01-13 18:38:09 -05:00
Carl Meyer
d54c19b983 [red-knot] remove CallOutcome::Cast variant (#15461)
## Summary

Simplification follow-up to #15413.

There's no need to have a dedicated `CallOutcome` variant for every
known function, it's only necessary if the special-cased behavior of the
known function includes emitting extra diagnostics. For `typing.cast`,
there's no such need; we can use the regular `Callable` outcome variant,
and update the return type according to the cast. (This is the same way
we already handle `len`.)

One reason to avoid proliferating unnecessary `CallOutcome` variants is
that currently we have to explicitly add emitting call-binding
diagnostics, for each outcome variant. So we were previously wrongly
silencing any binding diagnostics on calls to `typing.cast`. Fixing this
revealed a separate bug, that we were emitting a bogus error anytime
more than one keyword argument mapped to a `**kwargs` parameter. So this
PR also adds test and fix for that bug.

## Test Plan

Existing `cast` tests pass unchanged, added new test for `**kwargs` bug.
2025-01-13 10:58:53 -08:00
Micha Reiser
5ad546f187 Change ProgramSettings::python_platform to return a reference (#15457) 2025-01-13 16:23:34 +01:00
InSync
47d0a8ba96 [flake8-pytest-style] Test function parameters with default arguments (PT028) (#15449) 2025-01-13 13:40:54 +01:00
Dhruv Manilawala
56b14454dc Display context for ruff.configuration errors (#15452)
## Summary

I noticed this while trying out
https://github.com/astral-sh/ruff-vscode/issues/665 that we use the
`Display` implementation to show the error which hides the context. This
PR changes it to use the `Debug` implementation and adds the message as
a context.

## Test Plan

**Before:**

```
   0.001228084s ERROR main ruff_server::session::index::ruff_settings: Unable to find editor-specified configuration file: Failed to parse /private/tmp/hatch-test/ruff.toml
```

**After:**

```
   0.002348750s ERROR main ruff_server::session::index::ruff_settings: Unable to load editor-specified configuration file

Caused by:
    0: Failed to parse /private/tmp/hatch-test/ruff.toml
    1: TOML parse error at line 2, column 18
         |
       2 | extend-select = ["ASYNC101"]
         |                  ^^^^^^^^^^
       Unknown rule selector: `ASYNC101`
```
2025-01-13 15:43:20 +05:30
David Peter
eb3cb8d4b2 [red-knot] Use BitSet::union for merging of declarations (#15451)
## Summary

In `SymbolState` merging, use `BitSet::union` instead of inserting
declarations one by one. This used to be the case but was changed in
https://github.com/astral-sh/ruff/pull/15019 because we had to iterate
over declarations anyway.

This is an alternative to https://github.com/astral-sh/ruff/pull/15419
by @MichaReiser. It's similar in performance, but a bit more
declarative and less imperative.
2025-01-13 11:10:42 +01:00
InSync
6f35a4d8d5 [fastapi] Handle parameters with Depends correctly (FAST003) (#15364)
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
Co-authored-by: Micha Reiser <micha@reiser.io>
2025-01-13 08:51:02 +00:00
cake-monotone
82d06a198d [red-knot] Remove duplicate property test (#15450)
## Summary

Follow-up PR from https://github.com/astral-sh/ruff/pull/15415  🥲 

The exact same property test already exists:
`intersection_assignable_to_both` and
`all_type_pairs_can_be_assigned_from_their_intersection`

## Test Plan

`cargo test -p red_knot_python_semantic -- --ignored
types::property_tests::flaky`
2025-01-13 08:18:41 +01:00
InSync
70c3be88b9 [flake8-pie] Reuse parsed tokens (PIE800) (#15438)
## Summary

Follow-up to #15394. See [this review
comment](https://github.com/astral-sh/ruff/pull/15394#discussion_r1910526741).

## Test Plan

`cargo nextest run` and `cargo insta test`.
2025-01-12 21:03:11 -05:00
Tom Kuson
347ab5b47a [flake8-pytest-style] Implement pytest.warns diagnostics (PT029, PT030, PT031) (#15444)
## Summary

Implements upstream diagnostics `PT029`, `PT030`, `PT031` that function
as pytest.warns corollaries of `PT010`, `PT011`, `PT012` respectively.
Most of the implementation and documentation is designed to mirror those
existing diagnostics.

Closes #14239

## Test Plan

Tests for `PT029`, `PT030`, `PT031` largely copied from `PT010`,
`PT011`, `PT012` respectively.

`cargo nextest run`

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2025-01-13 01:46:59 +00:00
renovate[bot]
fa11b08766 Update dependency @types/react to v19.0.6 (#15448) 2025-01-13 01:11:51 +00:00
renovate[bot]
6f3e4e5062 Update NPM Development dependencies to v19.0.5 (#15445) 2025-01-12 20:06:02 -05:00
Charlie Marsh
2454305ef8 [flake8-pathlib] Fix --select for os-path-dirname (PTH120) (#15446)
## Summary

Closes https://github.com/astral-sh/ruff/issues/15439.
2025-01-13 00:55:46 +00:00
InSync
4f37fdeff2 [flake8-bandit] Check for builtins instead of builtin (S102, PTH123) (#15443)
## Summary

Resolves #15442.

## Test Plan

`cargo nextest run` and `cargo insta test`.
2025-01-12 19:45:31 -05:00
InSync
d1666fbbee [red-knot] Add AlwaysTruthy and AlwaysFalsy to knot_extensions (#15437)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-01-12 17:00:57 +00:00
Alex Waygood
06b7f4495e [red-knot] Minor improvements to KnownFunction API (#15441)
A small PR to reduce some of the code duplication between the various
branches, make it a little more readable and move the API closer to what
we already have for `KnownClass`
2025-01-12 16:06:31 +00:00
Alex Waygood
c8795fcb37 [red-knot] Minor improvements to property_tests.rs (#15440) 2025-01-12 13:55:18 +00:00
cake-monotone
ccfde37619 [red-knot] Add Property Tests for Intersection and Union (#15415) 2025-01-12 13:21:29 +00:00
InSync
6ae3e8f8d7 [red-knot] Support cast (#15413)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-01-12 13:05:45 +00:00
renovate[bot]
60d7a464fb Update Rust crate colored to v3 (#15434)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-01-11 17:52:52 +00:00
renovate[bot]
c0259e7bf2 Update dependency ruff to v0.9.1 (#15432)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-01-11 17:18:38 +00:00
renovate[bot]
22edee2353 Update pre-commit dependencies (#15433)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2025-01-11 17:18:13 +00:00
renovate[bot]
7d20277111 Update Rust crate libcst to v1.6.0 (#15431)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-01-11 17:14:44 +00:00
renovate[bot]
bce07f6564 Update Rust crate bitflags to v2.7.0 (#15430)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-01-11 17:14:19 +00:00
renovate[bot]
8ea6605a6d Update NPM Development dependencies (#15428)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-01-11 17:13:32 +00:00
renovate[bot]
d323f2019b Update dependency uuid to v11.0.5 (#15427)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-01-11 17:13:17 +00:00
renovate[bot]
ad883d9b31 Update Rust crate uuid to v1.11.1 (#15426)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-01-11 17:12:54 +00:00
renovate[bot]
7240212d27 Update Rust crate thiserror to v2.0.11 (#15425)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-01-11 17:12:34 +00:00
renovate[bot]
925ee41317 Update Rust crate syn to v2.0.96 (#15424)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-01-11 17:12:23 +00:00
renovate[bot]
78b242fe3f Update Rust crate proc-macro2 to v1.0.93 (#15422)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-01-11 17:12:07 +00:00
renovate[bot]
7ed46d0823 Update Rust crate serde_json to v1.0.135 (#15423)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-01-11 17:11:48 +00:00
renovate[bot]
bff4edb717 Update Rust crate clap to v4.5.26 (#15420)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-01-11 17:09:54 +00:00
Dhruv Manilawala
38f873ba52 Remove flatten to improve deserialization error messages (#15414)
## Summary

Closes: #9719  

## Test Plan

**Before:**

```
ruff failed
  Cause: Failed to parse /Users/dhruv/playground/ruff/pyproject.toml
  Cause: TOML parse error at line 22, column 1
   |
22 | [tool.ruff.lint]
   | ^^^^^^^^^^^^^^^^
invalid type: string "false", expected a boolean
```

**After:**

```
ruff failed
  Cause: Failed to parse /Users/dhruv/playground/ruff/pyproject.toml
  Cause: TOML parse error at line 27, column 20
   |
27 | mypy-init-return = "false"
   |                    ^^^^^^^
invalid type: string "false", expected a boolean
```
2025-01-11 22:08:21 +05:30
Micha Reiser
c39ca8fe6d Upgrade Rust toolchain to 1.84.0 (#15408) 2025-01-11 09:51:58 +01:00
David Peter
2d82445794 [red-knot] Simplify unions of T and ~T (#15400)
## Summary

Simplify unions of `T` and `~T` to `object`.

## Test Plan

Adapted existing tests.
2025-01-10 23:00:52 +01:00
David Peter
398f2e8b0c [red-knot] Minor fixes in intersection-types tests (#15410)
## Summary

Minor fixes in intersection-types tests
2025-01-10 22:53:03 +01:00
InSync
232fbc1300 [red-knot] Understand type[Unknown] (#15409)
## Summary

Follow-up to #15194.

## Test Plan

Markdown tests.
2025-01-10 13:25:59 -08:00
Alex Waygood
c82932e580 [red-knot] Refactor KnownFunction::takes_expression_arguments() (#15406) 2025-01-10 19:09:03 +00:00
1320 changed files with 21540 additions and 11647 deletions

1
.github/CODEOWNERS vendored
View File

@@ -9,6 +9,7 @@
/crates/ruff_formatter/ @MichaReiser
/crates/ruff_python_formatter/ @MichaReiser
/crates/ruff_python_parser/ @MichaReiser @dhruvmanila
/crates/ruff_annotate_snippets/ @BurntSushi
# flake8-pyi
/crates/ruff_linter/src/rules/flake8_pyi/ @AlexWaygood

View File

@@ -1,12 +0,0 @@
<!--
Thank you for taking the time to report an issue! We're glad to have you involved with Ruff.
If you're filing a bug report, please consider including the following information:
* List of keywords you searched for before creating this issue. Write them down here so that others can find this issue more easily and help provide feedback.
e.g. "RUF001", "unused variable", "Jupyter notebook"
* A minimal code snippet that reproduces the bug.
* The command you invoked (e.g., `ruff /path/to/file.py --fix`), ideally including the `--isolated` flag.
* The current Ruff settings (any relevant sections from your `pyproject.toml`).
* The current Ruff version (`ruff --version`).
-->

View File

@@ -78,5 +78,6 @@ jobs:
owner: "astral-sh",
repo: "ruff",
title: `Automated typeshed sync failed on ${new Date().toDateString()}`,
body: "Runs are listed here: https://github.com/astral-sh/ruff/actions/workflows/sync_typeshed.yaml",
body: "Run listed here: https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}",
labels: ["bug", "red-knot"],
})

View File

@@ -73,7 +73,7 @@ repos:
pass_filenames: false # This makes it a lot faster
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.8.6
rev: v0.9.1
hooks:
- id: ruff-format
- id: ruff
@@ -91,12 +91,12 @@ repos:
# zizmor detects security vulnerabilities in GitHub Actions workflows.
# Additional configuration for the tool is found in `.github/zizmor.yml`
- repo: https://github.com/woodruffw/zizmor-pre-commit
rev: v1.0.0
rev: v1.0.1
hooks:
- id: zizmor
- repo: https://github.com/python-jsonschema/check-jsonschema
rev: 0.30.0
rev: 0.31.0
hooks:
- id: check-github-workflows

410
Cargo.lock generated
View File

@@ -57,35 +57,35 @@ version = "0.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c7021ce4924a3f25f802b2cccd1af585e39ea1a363a1aa2e72afe54b67a3a7a7"
[[package]]
name = "annotate-snippets"
version = "0.9.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ccaf7e9dfbb6ab22c82e473cd1a8a7bd313c19a5b7e40970f3d89ef5a5c9e81e"
dependencies = [
"unicode-width 0.1.13",
"yansi-term",
]
[[package]]
name = "anstream"
version = "0.6.13"
version = "0.6.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d96bd03f33fe50a863e394ee9718a706f988b9079b20c3784fb726e7678b62fb"
checksum = "8acc5369981196006228e28809f761875c0327210a891e941f4c683b3a99529b"
dependencies = [
"anstyle",
"anstyle-parse",
"anstyle-query",
"anstyle-wincon",
"colorchoice",
"is_terminal_polyfill",
"utf8parse",
]
[[package]]
name = "anstyle"
version = "1.0.8"
version = "1.0.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1bec1de6f59aedf83baf9ff929c98f2ad654b97c9510f4e70cf6f661d49fd5b1"
checksum = "55cc3b69f167a1ef2e161439aa98aed94e6028e5f9a59be9a6ffb47aef1651f9"
[[package]]
name = "anstyle-lossy"
version = "1.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "934ff8719effd2023a48cf63e69536c1c3ced9d3895068f6f5cc9a4ff845e59b"
dependencies = [
"anstyle",
]
[[package]]
name = "anstyle-parse"
@@ -106,13 +106,26 @@ dependencies = [
]
[[package]]
name = "anstyle-wincon"
version = "3.0.2"
name = "anstyle-svg"
version = "0.1.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1cd54b81ec8d6180e24654d0b371ad22fc3dd083b6ff8ba325b72e00c87660a7"
checksum = "d3607949e9f6de49ea4bafe12f5e4fd73613ebf24795e48587302a8cc0e4bb35"
dependencies = [
"anstream",
"anstyle",
"anstyle-lossy",
"html-escape",
"unicode-width 0.2.0",
]
[[package]]
name = "anstyle-wincon"
version = "3.0.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2109dbce0e72be3ec00bed26e6a7479ca384ad226efdd66db8fa2e3a38c83125"
dependencies = [
"anstyle",
"windows-sys 0.52.0",
"windows-sys 0.59.0",
]
[[package]]
@@ -199,9 +212,9 @@ checksum = "bef38d45163c2f1dde094a7dfd33ccf595c92905c8f8f4fdc18d06fb1037718a"
[[package]]
name = "bitflags"
version = "2.6.0"
version = "2.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b048fb63fd8b5923fc5aa7b340d8e156aec7ec02f0c78fa8a6ddc2613f6f71de"
checksum = "1be3f42a67d6d345ecd59f675f3f012d6974981560836e938c22b424b85ce1be"
[[package]]
name = "block-buffer"
@@ -303,7 +316,7 @@ version = "1.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a5b5db619f3556839cb2223ae86ff3f9a09da2c5013be42bc9af08c9589bf70c"
dependencies = [
"annotate-snippets 0.6.1",
"annotate-snippets",
]
[[package]]
@@ -347,9 +360,9 @@ dependencies = [
[[package]]
name = "clap"
version = "4.5.23"
version = "4.5.26"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3135e7ec2ef7b10c6ed8950f0f792ed96ee093fa088608f1c76e569722700c84"
checksum = "a8eb5e908ef3a6efbe1ed62520fb7287959888c88485abe072543190ecc66783"
dependencies = [
"clap_builder",
"clap_derive",
@@ -357,9 +370,9 @@ dependencies = [
[[package]]
name = "clap_builder"
version = "4.5.23"
version = "4.5.26"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "30582fc632330df2bd26877bde0c1f4470d57c582bbc070376afcd04d8cb4838"
checksum = "96b01801b5fc6a0a232407abc821660c9c6d25a1cafc0d4f85f29fb8d9afc121"
dependencies = [
"anstream",
"anstyle",
@@ -400,14 +413,14 @@ dependencies = [
[[package]]
name = "clap_derive"
version = "4.5.18"
version = "4.5.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4ac6a0c7b1a9e9a5186361f67dfa1b88213572f427fb9ab038efb2bd8c582dab"
checksum = "54b755194d6389280185988721fffba69495eed5ee9feeee9a599b53db80318c"
dependencies = [
"heck",
"proc-macro2",
"quote",
"syn 2.0.95",
"syn 2.0.96",
]
[[package]]
@@ -424,7 +437,7 @@ checksum = "8c41dc435a7b98e4608224bbf65282309f5403719df9113621b30f8b6f74e2f4"
dependencies = [
"nix",
"terminfo",
"thiserror 2.0.9",
"thiserror 2.0.11",
"which",
"windows-sys 0.59.0",
]
@@ -435,7 +448,7 @@ version = "2.7.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "450a0e9df9df1c154156f4344f99d8f6f6e69d0fc4de96ef6e2e68b2ec3bce97"
dependencies = [
"colored",
"colored 2.2.0",
"libc",
"serde_json",
]
@@ -447,7 +460,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8eb1a6cb9c20e177fde58cdef97c1c7c9264eb1424fe45c4fccedc2fb078a569"
dependencies = [
"codspeed",
"colored",
"colored 2.2.0",
"criterion",
]
@@ -464,6 +477,15 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "117725a109d387c937a1533ce01b450cbde6b88abceea8473c4d7a85853cda3c"
dependencies = [
"lazy_static",
"windows-sys 0.48.0",
]
[[package]]
name = "colored"
version = "3.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fde0e0ec90c9dfb3b4b1a0891a7dcd0e2bffde2f7efed5fe7c9bb00e5bfb915e"
dependencies = [
"windows-sys 0.59.0",
]
@@ -687,7 +709,7 @@ dependencies = [
"proc-macro2",
"quote",
"strsim 0.10.0",
"syn 2.0.95",
"syn 2.0.96",
]
[[package]]
@@ -698,7 +720,7 @@ checksum = "a668eda54683121533a393014d8692171709ff57a7d61f187b6e782719f8933f"
dependencies = [
"darling_core",
"quote",
"syn 2.0.95",
"syn 2.0.96",
]
[[package]]
@@ -768,7 +790,7 @@ dependencies = [
"glob",
"proc-macro2",
"quote",
"syn 2.0.95",
"syn 2.0.96",
]
[[package]]
@@ -800,7 +822,7 @@ checksum = "97369cbbc041bc366949bc74d34658d6cda5621039731c6310521892a3a20ae0"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.95",
"syn 2.0.96",
]
[[package]]
@@ -884,6 +906,24 @@ dependencies = [
"windows-sys 0.52.0",
]
[[package]]
name = "escape8259"
version = "0.5.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5692dd7b5a1978a5aeb0ce83b7655c58ca8efdcb79d21036ea249da95afec2c6"
[[package]]
name = "escargot"
version = "0.5.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "05a3ac187a16b5382fef8c69fd1bad123c67b7cf3932240a2d43dcdd32cded88"
dependencies = [
"log",
"once_cell",
"serde",
"serde_json",
]
[[package]]
name = "etcetera"
version = "0.8.0"
@@ -1022,7 +1062,7 @@ version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0bf760ebf69878d9fd8f110c89703d90ce35095324d1f1edcb595c63945ee757"
dependencies = [
"bitflags 2.6.0",
"bitflags 2.7.0",
"ignore",
"walkdir",
]
@@ -1082,6 +1122,15 @@ dependencies = [
"windows-sys 0.52.0",
]
[[package]]
name = "html-escape"
version = "0.2.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6d1ad449764d627e22bfd7cd5e8868264fc9236e07c752972b4080cd351cb476"
dependencies = [
"utf8-width",
]
[[package]]
name = "humantime"
version = "2.1.0"
@@ -1226,7 +1275,7 @@ checksum = "1ec89e9337638ecdc08744df490b221a7399bf8d164eb52a665454e60e075ad6"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.95",
"syn 2.0.96",
]
[[package]]
@@ -1400,7 +1449,7 @@ dependencies = [
"heck",
"proc-macro2",
"quote",
"syn 2.0.95",
"syn 2.0.96",
]
[[package]]
@@ -1424,6 +1473,12 @@ dependencies = [
"once_cell",
]
[[package]]
name = "is_terminal_polyfill"
version = "1.70.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7943c866cc5cd64cbc25b2e01621d07fa8eb2a1a23160ee81ce38704e97b8ecf"
[[package]]
name = "itertools"
version = "0.10.5"
@@ -1516,9 +1571,9 @@ checksum = "b5aba8db14291edd000dfcc4d620c7ebfb122c613afb886ca8803fa4e128a20a"
[[package]]
name = "libcst"
version = "1.5.1"
version = "1.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fa3e60579a8cba3d86aa4a5f7fc98973cc0fd2ac270bf02f85a9bef09700b075"
checksum = "649801a698a649791541a3125d396d5db065ed7cea53faca3652b0179394922a"
dependencies = [
"chic",
"libcst_derive",
@@ -1531,12 +1586,12 @@ dependencies = [
[[package]]
name = "libcst_derive"
version = "1.4.0"
version = "1.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a2ae40017ac09cd2c6a53504cb3c871c7f2b41466eac5bc66ba63f39073b467b"
checksum = "3bf66548c351bcaed792ef3e2b430cc840fbde504e09da6b29ed114ca60dcd4b"
dependencies = [
"quote",
"syn 2.0.95",
"syn 2.0.96",
]
[[package]]
@@ -1555,11 +1610,23 @@ version = "0.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c0ff37bd590ca25063e35af745c343cb7a0271906fb7b37e4813e8f79f00268d"
dependencies = [
"bitflags 2.6.0",
"bitflags 2.7.0",
"libc",
"redox_syscall 0.5.3",
]
[[package]]
name = "libtest-mimic"
version = "0.7.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cc0bda45ed5b3a2904262c1bb91e526127aa70e7ef3758aba2ef93cf896b9b58"
dependencies = [
"clap",
"escape8259",
"termcolor",
"threadpool",
]
[[package]]
name = "linked-hash-map"
version = "0.5.6"
@@ -1714,7 +1781,7 @@ version = "0.29.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "71e2746dc3a24dd78b3cfcb7be93368c6de9963d30f43a6a73998a9cf4b17b46"
dependencies = [
"bitflags 2.6.0",
"bitflags 2.7.0",
"cfg-if",
"cfg_aliases",
"libc",
@@ -1730,13 +1797,19 @@ dependencies = [
"minimal-lexical",
]
[[package]]
name = "normalize-line-endings"
version = "0.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "61807f77802ff30975e01f4f071c8ba10c022052f98b3294119f3e615d13e5be"
[[package]]
name = "notify"
version = "7.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c533b4c39709f9ba5005d8002048266593c1cfaf3c5f0739d5b8ab0c6c504009"
dependencies = [
"bitflags 2.6.0",
"bitflags 2.7.0",
"filetime",
"fsevent-sys",
"inotify",
@@ -1786,6 +1859,16 @@ dependencies = [
"autocfg",
]
[[package]]
name = "num_cpus"
version = "1.16.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4161fcb6d602d4d2081af7c3a45852d875a03dd337a6bfdd6e06407b61342a43"
dependencies = [
"hermit-abi",
"libc",
]
[[package]]
name = "number_prefix"
version = "0.4.0"
@@ -1819,6 +1902,16 @@ dependencies = [
"indexmap",
]
[[package]]
name = "os_pipe"
version = "1.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5ffd2b0a5634335b135d5728d84c5e0fd726954b87111f7506a61c502280d982"
dependencies = [
"libc",
"windows-sys 0.59.0",
]
[[package]]
name = "os_str_bytes"
version = "7.0.0"
@@ -1992,7 +2085,7 @@ dependencies = [
"pest_meta",
"proc-macro2",
"quote",
"syn 2.0.95",
"syn 2.0.96",
]
[[package]]
@@ -2107,9 +2200,9 @@ dependencies = [
[[package]]
name = "proc-macro2"
version = "1.0.92"
version = "1.0.93"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "37d3544b3f2748c54e147655edb5025752e2303145b5aefb3c3ea2c78b973bb0"
checksum = "60946a68e5f9d28b0dc1c21bb8a97ee7d018a8b322fa57838ba31cc878e22d99"
dependencies = [
"unicode-ident",
]
@@ -2139,7 +2232,7 @@ dependencies = [
"newtype-uuid",
"quick-xml",
"strip-ansi-escapes",
"thiserror 2.0.9",
"thiserror 2.0.11",
"uuid",
]
@@ -2238,7 +2331,7 @@ dependencies = [
"anyhow",
"chrono",
"clap",
"colored",
"colored 3.0.0",
"countme",
"crossbeam",
"ctrlc",
@@ -2261,7 +2354,7 @@ name = "red_knot_python_semantic"
version = "0.0.0"
dependencies = [
"anyhow",
"bitflags 2.6.0",
"bitflags 2.7.0",
"camino",
"compact_str",
"countme",
@@ -2294,7 +2387,7 @@ dependencies = [
"static_assertions",
"tempfile",
"test-case",
"thiserror 2.0.9",
"thiserror 2.0.11",
"tracing",
]
@@ -2328,7 +2421,7 @@ version = "0.0.0"
dependencies = [
"anyhow",
"camino",
"colored",
"colored 3.0.0",
"memchr",
"red_knot_python_semantic",
"red_knot_vendored",
@@ -2391,7 +2484,7 @@ dependencies = [
"rustc-hash 2.1.0",
"salsa",
"serde",
"thiserror 2.0.9",
"thiserror 2.0.11",
"toml",
"tracing",
]
@@ -2411,7 +2504,7 @@ version = "0.5.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2a908a6e00f1fdd0dfd9c0eb08ce85126f6d8bbda50017e74bc4a4b7d4a926a4"
dependencies = [
"bitflags 2.6.0",
"bitflags 2.7.0",
]
[[package]]
@@ -2503,13 +2596,13 @@ dependencies = [
"argfile",
"assert_fs",
"bincode",
"bitflags 2.6.0",
"bitflags 2.7.0",
"cachedir",
"chrono",
"clap",
"clap_complete_command",
"clearscreen",
"colored",
"colored 3.0.0",
"filetime",
"globwalk",
"ignore",
@@ -2544,7 +2637,7 @@ dependencies = [
"strum",
"tempfile",
"test-case",
"thiserror 2.0.9",
"thiserror 2.0.11",
"tikv-jemallocator",
"toml",
"tracing",
@@ -2552,6 +2645,21 @@ dependencies = [
"wild",
]
[[package]]
name = "ruff_annotate_snippets"
version = "0.1.0"
dependencies = [
"anstream",
"anstyle",
"memchr",
"ruff_annotate_snippets",
"serde",
"snapbox",
"toml",
"tryfn",
"unicode-width 0.2.0",
]
[[package]]
name = "ruff_benchmark"
version = "0.0.0"
@@ -2614,7 +2722,7 @@ dependencies = [
"salsa",
"serde",
"tempfile",
"thiserror 2.0.9",
"thiserror 2.0.11",
"tracing",
"tracing-subscriber",
"tracing-tree",
@@ -2719,12 +2827,11 @@ name = "ruff_linter"
version = "0.9.1"
dependencies = [
"aho-corasick",
"annotate-snippets 0.9.2",
"anyhow",
"bitflags 2.6.0",
"bitflags 2.7.0",
"chrono",
"clap",
"colored",
"colored 3.0.0",
"fern",
"glob",
"globset",
@@ -2743,6 +2850,7 @@ dependencies = [
"pyproject-toml",
"quick-junit",
"regex",
"ruff_annotate_snippets",
"ruff_cache",
"ruff_diagnostics",
"ruff_index",
@@ -2767,7 +2875,7 @@ dependencies = [
"strum",
"strum_macros",
"test-case",
"thiserror 2.0.9",
"thiserror 2.0.11",
"toml",
"typed-arena",
"unicode-normalization",
@@ -2784,7 +2892,7 @@ dependencies = [
"proc-macro2",
"quote",
"ruff_python_trivia",
"syn 2.0.95",
"syn 2.0.96",
]
[[package]]
@@ -2801,7 +2909,7 @@ dependencies = [
"serde_json",
"serde_with",
"test-case",
"thiserror 2.0.9",
"thiserror 2.0.11",
"uuid",
]
@@ -2810,7 +2918,7 @@ name = "ruff_python_ast"
version = "0.0.0"
dependencies = [
"aho-corasick",
"bitflags 2.6.0",
"bitflags 2.7.0",
"compact_str",
"is-macro",
"itertools 0.14.0",
@@ -2873,7 +2981,7 @@ dependencies = [
"similar",
"smallvec",
"static_assertions",
"thiserror 2.0.9",
"thiserror 2.0.11",
"tracing",
]
@@ -2892,7 +3000,7 @@ dependencies = [
name = "ruff_python_literal"
version = "0.0.0"
dependencies = [
"bitflags 2.6.0",
"bitflags 2.7.0",
"itertools 0.14.0",
"ruff_python_ast",
"unic-ucd-category",
@@ -2902,13 +3010,13 @@ dependencies = [
name = "ruff_python_parser"
version = "0.0.0"
dependencies = [
"annotate-snippets 0.9.2",
"anyhow",
"bitflags 2.6.0",
"bitflags 2.7.0",
"bstr",
"compact_str",
"insta",
"memchr",
"ruff_annotate_snippets",
"ruff_python_ast",
"ruff_python_trivia",
"ruff_source_file",
@@ -2935,7 +3043,7 @@ dependencies = [
name = "ruff_python_semantic"
version = "0.0.0"
dependencies = [
"bitflags 2.6.0",
"bitflags 2.7.0",
"is-macro",
"ruff_cache",
"ruff_index",
@@ -2954,7 +3062,7 @@ dependencies = [
name = "ruff_python_stdlib"
version = "0.0.0"
dependencies = [
"bitflags 2.6.0",
"bitflags 2.7.0",
"unicode-ident",
]
@@ -3007,7 +3115,7 @@ dependencies = [
"serde",
"serde_json",
"shellexpand",
"thiserror 2.0.9",
"thiserror 2.0.11",
"tracing",
"tracing-subscriber",
]
@@ -3061,7 +3169,7 @@ name = "ruff_workspace"
version = "0.0.0"
dependencies = [
"anyhow",
"colored",
"colored 3.0.0",
"etcetera",
"glob",
"globset",
@@ -3121,7 +3229,7 @@ version = "0.38.40"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "99e4ea3e1cdc4b559b8e5650f9c8e5998e3e5c1343b4eaf034565f32318d63c0"
dependencies = [
"bitflags 2.6.0",
"bitflags 2.7.0",
"errno",
"libc",
"linux-raw-sys",
@@ -3205,7 +3313,7 @@ dependencies = [
"heck",
"proc-macro2",
"quote",
"syn 2.0.95",
"syn 2.0.96",
"synstructure",
]
@@ -3239,7 +3347,7 @@ dependencies = [
"proc-macro2",
"quote",
"serde_derive_internals",
"syn 2.0.95",
"syn 2.0.96",
]
[[package]]
@@ -3288,7 +3396,7 @@ checksum = "5a9bf7cf98d04a2b28aead066b7496853d4779c9cc183c440dbac457641e19a0"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.95",
"syn 2.0.96",
]
[[package]]
@@ -3299,14 +3407,14 @@ checksum = "330f01ce65a3a5fe59a60c82f3c9a024b573b8a6e875bd233fe5f934e71d54e3"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.95",
"syn 2.0.96",
]
[[package]]
name = "serde_json"
version = "1.0.134"
version = "1.0.135"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d00f4175c42ee48b15416f6193a959ba3a0d67fc699a0db9ad12df9f83991c7d"
checksum = "2b0d7ba2887406110130a978386c4e1befb98c674b4fba677954e4db976630d9"
dependencies = [
"itoa",
"memchr",
@@ -3322,7 +3430,7 @@ checksum = "6c64451ba24fc7a6a2d60fc75dd9c83c90903b19028d4eff35e88fc1e86564e9"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.95",
"syn 2.0.96",
]
[[package]]
@@ -3363,7 +3471,7 @@ dependencies = [
"darling",
"proc-macro2",
"quote",
"syn 2.0.95",
"syn 2.0.96",
]
[[package]]
@@ -3419,6 +3527,35 @@ version = "1.13.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3c5e1a9a646d36c3599cd173a41282daf47c44583ad367b8e6837255952e5c67"
[[package]]
name = "snapbox"
version = "0.6.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "96dcfc4581e3355d70ac2ee14cfdf81dce3d85c85f1ed9e2c1d3013f53b3436b"
dependencies = [
"anstream",
"anstyle",
"anstyle-svg",
"escargot",
"libc",
"normalize-line-endings",
"os_pipe",
"serde_json",
"similar",
"snapbox-macros",
"wait-timeout",
"windows-sys 0.59.0",
]
[[package]]
name = "snapbox-macros"
version = "0.3.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "16569f53ca23a41bb6f62e0a5084aa1661f4814a67fa33696a79073e03a664af"
dependencies = [
"anstream",
]
[[package]]
name = "spin"
version = "0.9.8"
@@ -3477,7 +3614,7 @@ dependencies = [
"proc-macro2",
"quote",
"rustversion",
"syn 2.0.95",
"syn 2.0.96",
]
[[package]]
@@ -3499,9 +3636,9 @@ dependencies = [
[[package]]
name = "syn"
version = "2.0.95"
version = "2.0.96"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "46f71c0377baf4ef1cc3e3402ded576dccc315800fbc62dfc7fe04b009773b4a"
checksum = "d5d0adab1ae378d7f53bdebc67a39f1f151407ef230f0ce2883572f5d8985c80"
dependencies = [
"proc-macro2",
"quote",
@@ -3516,7 +3653,7 @@ checksum = "c8af7666ab7b6390ab78131fb5b0fce11d6b7a6951602017c35fa82800708971"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.95",
"syn 2.0.96",
]
[[package]]
@@ -3532,6 +3669,15 @@ dependencies = [
"windows-sys 0.59.0",
]
[[package]]
name = "termcolor"
version = "1.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "06794f8f6c5c898b3275aebefa6b8a1cb24cd2c6c79397ab15774837a0bc5755"
dependencies = [
"winapi-util",
]
[[package]]
name = "terminal_size"
version = "0.4.0"
@@ -3578,7 +3724,7 @@ dependencies = [
"cfg-if",
"proc-macro2",
"quote",
"syn 2.0.95",
"syn 2.0.96",
]
[[package]]
@@ -3589,7 +3735,7 @@ checksum = "5c89e72a01ed4c579669add59014b9a524d609c0c88c6a585ce37485879f6ffb"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.95",
"syn 2.0.96",
"test-case-core",
]
@@ -3604,11 +3750,11 @@ dependencies = [
[[package]]
name = "thiserror"
version = "2.0.9"
version = "2.0.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f072643fd0190df67a8bab670c20ef5d8737177d6ac6b2e9a236cb096206b2cc"
checksum = "d452f284b73e6d76dd36758a0c8684b1d5be31f92b89d07fd5822175732206fc"
dependencies = [
"thiserror-impl 2.0.9",
"thiserror-impl 2.0.11",
]
[[package]]
@@ -3619,18 +3765,18 @@ checksum = "b607164372e89797d78b8e23a6d67d5d1038c1c65efd52e1389ef8b77caba2a6"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.95",
"syn 2.0.96",
]
[[package]]
name = "thiserror-impl"
version = "2.0.9"
version = "2.0.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7b50fa271071aae2e6ee85f842e2e28ba8cd2c5fb67f11fcb1fd70b276f9e7d4"
checksum = "26afc1baea8a989337eeb52b6e72a039780ce45c3edfcc9c5b9d112feeb173c2"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.95",
"syn 2.0.96",
]
[[package]]
@@ -3643,6 +3789,15 @@ dependencies = [
"once_cell",
]
[[package]]
name = "threadpool"
version = "1.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d050e60b33d41c19108b32cea32164033a9013fe3b46cbd4457559bfbf77afaa"
dependencies = [
"num_cpus",
]
[[package]]
name = "tikv-jemalloc-sys"
version = "0.6.0+5.3.0-1-ge13ca993e8ccb9ba9847cc330696e02839f328f7"
@@ -3752,7 +3907,7 @@ checksum = "395ae124c09f9e6918a2310af6038fba074bcf474ac352496d5910dd59a2226d"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.95",
"syn 2.0.96",
]
[[package]]
@@ -3829,6 +3984,17 @@ dependencies = [
"tracing-subscriber",
]
[[package]]
name = "tryfn"
version = "0.2.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5fe242ee9e646acec9ab73a5c540e8543ed1b107f0ce42be831e0775d423c396"
dependencies = [
"ignore",
"libtest-mimic",
"snapbox",
]
[[package]]
name = "typed-arena"
version = "2.0.2"
@@ -3990,6 +4156,12 @@ version = "1.0.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c8232dd3cdaed5356e0f716d285e4b40b932ac434100fe9b7e0e8e935b9e6246"
[[package]]
name = "utf8-width"
version = "0.1.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "86bd8d4e895da8537e5315b8254664e6b769c4ff3db18321b297a1e7004392e3"
[[package]]
name = "utf8_iter"
version = "1.0.4"
@@ -4004,9 +4176,9 @@ checksum = "711b9620af191e0cdc7468a8d14e709c3dcdb115b36f838e601583af800a370a"
[[package]]
name = "uuid"
version = "1.11.0"
version = "1.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8c5f0a0af699448548ad1a2fbf920fb4bee257eae39953ba95cb84891a0446a"
checksum = "b913a3b5fe84142e269d63cc62b64319ccaf89b748fc31fe025177f767a756c4"
dependencies = [
"getrandom",
"rand",
@@ -4016,13 +4188,13 @@ dependencies = [
[[package]]
name = "uuid-macro-internal"
version = "1.11.0"
version = "1.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6b91f57fe13a38d0ce9e28a03463d8d3c2468ed03d75375110ec71d93b449a08"
checksum = "c91084647266237a48351d05d55dee65bba9e1b597f555fcf54680f820284a1c"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.95",
"syn 2.0.96",
]
[[package]]
@@ -4079,6 +4251,15 @@ dependencies = [
"quote",
]
[[package]]
name = "wait-timeout"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9f200f5b12eb75f8c1ed65abd4b2db8a6e1b138a20de009dacee265a2498f3f6"
dependencies = [
"libc",
]
[[package]]
name = "walkdir"
version = "2.5.0"
@@ -4117,7 +4298,7 @@ dependencies = [
"once_cell",
"proc-macro2",
"quote",
"syn 2.0.95",
"syn 2.0.96",
"wasm-bindgen-shared",
]
@@ -4152,7 +4333,7 @@ checksum = "98c9ae5a76e46f4deecd0f0255cc223cfa18dc9b261213b8aa0c7b36f61b3f1d"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.95",
"syn 2.0.96",
"wasm-bindgen-backend",
"wasm-bindgen-shared",
]
@@ -4186,7 +4367,7 @@ checksum = "222ebde6ea87fbfa6bdd2e9f1fd8a91d60aee5db68792632176c4e16a74fc7d8"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.95",
"syn 2.0.96",
]
[[package]]
@@ -4460,15 +4641,6 @@ version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cfe53a6657fd280eaa890a3bc59152892ffa3e30101319d168b781ed6529b049"
[[package]]
name = "yansi-term"
version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fe5c30ade05e61656247b2e334a031dfd0cc466fadef865bdcdea8d537951bf1"
dependencies = [
"winapi",
]
[[package]]
name = "yoke"
version = "0.7.4"
@@ -4489,7 +4661,7 @@ checksum = "28cc31741b18cb6f1d5ff12f5b7523e3d6eb0852bbbad19d73905511d9849b95"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.95",
"syn 2.0.96",
"synstructure",
]
@@ -4510,7 +4682,7 @@ checksum = "9ce1b18ccd8e73a9321186f97e46f9f04b778851177567b1975109d26a08d2a6"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.95",
"syn 2.0.96",
]
[[package]]
@@ -4530,7 +4702,7 @@ checksum = "0ea7b4a3637ea8669cedf0f1fd5c286a17f3de97b8dd5a70a6c167a1730e63a5"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.95",
"syn 2.0.96",
"synstructure",
]
@@ -4559,7 +4731,7 @@ checksum = "6eafa6dfb17584ea3e2bd6e76e0cc15ad7af12b09abdd1ca55961bed9b1063c6"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.95",
"syn 2.0.96",
]
[[package]]

View File

@@ -13,6 +13,7 @@ license = "MIT"
[workspace.dependencies]
ruff = { path = "crates/ruff" }
ruff_annotate_snippets = { path = "crates/ruff_annotate_snippets" }
ruff_cache = { path = "crates/ruff_cache" }
ruff_db = { path = "crates/ruff_db", default-features = false }
ruff_diagnostics = { path = "crates/ruff_diagnostics" }
@@ -43,7 +44,8 @@ red_knot_test = { path = "crates/red_knot_test" }
red_knot_workspace = { path = "crates/red_knot_workspace", default-features = false }
aho-corasick = { version = "1.1.3" }
annotate-snippets = { version = "0.9.2", features = ["color"] }
anstream = { version = "0.6.18" }
anstyle = { version = "1.0.10" }
anyhow = { version = "1.0.80" }
assert_fs = { version = "1.1.0" }
argfile = { version = "0.2.0" }
@@ -57,7 +59,7 @@ clap = { version = "4.5.3", features = ["derive"] }
clap_complete_command = { version = "0.6.0" }
clearscreen = { version = "4.0.0" }
codspeed-criterion-compat = { version = "2.6.0", default-features = false }
colored = { version = "2.1.0" }
colored = { version = "3.0.0" }
console_error_panic_hook = { version = "0.1.7" }
console_log = { version = "1.0.0" }
countme = { version = "3.0.1" }
@@ -132,6 +134,7 @@ serde_with = { version = "3.6.0", default-features = false, features = [
shellexpand = { version = "3.0.0" }
similar = { version = "2.4.0", features = ["inline"] }
smallvec = { version = "1.13.2" }
snapbox = { version = "0.6.0", features = ["diff", "term-svg", "cmd", "examples"] }
static_assertions = "1.1.0"
strum = { version = "0.26.0", features = ["strum_macros"] }
strum_macros = { version = "0.26.0" }
@@ -149,6 +152,7 @@ tracing-subscriber = { version = "0.3.18", default-features = false, features =
"fmt",
] }
tracing-tree = { version = "0.4.0" }
tryfn = { version = "0.2.1" }
typed-arena = { version = "2.0.2" }
unic-ucd-category = { version = "0.9" }
unicode-ident = { version = "1.0.12" }
@@ -211,6 +215,9 @@ redundant_clone = "warn"
debug_assert_with_mut_call = "warn"
unused_peekable = "warn"
# Diagnostics are not actionable: Enable once https://github.com/rust-lang/rust-clippy/issues/13774 is resolved.
large_stack_arrays = "allow"
[profile.release]
# Note that we set these explicitly, and these values
# were chosen based on a trade-off between compile times

View File

@@ -1,10 +1,9 @@
[files]
# https://github.com/crate-ci/typos/issues/868
extend-exclude = [
"crates/red_knot_vendored/vendor/**/*",
"**/resources/**/*",
"**/snapshots/**/*",
"crates/red_knot_workspace/src/workspace/pyproject/package_name.rs"
"crates/red_knot_vendored/vendor/**/*",
"**/resources/**/*",
"**/snapshots/**/*",
]
[default.extend-words]
@@ -21,7 +20,10 @@ Numer = "Numer" # Library name 'NumerBlox' in "Who's Using Ruff?"
[default]
extend-ignore-re = [
# Line ignore with trailing "spellchecker:disable-line"
"(?Rm)^.*#\\s*spellchecker:disable-line$",
"LICENSEs",
# Line ignore with trailing "spellchecker:disable-line"
"(?Rm)^.*#\\s*spellchecker:disable-line$",
"LICENSEs",
]
[default.extend-identifiers]
"FrIeNdLy" = "FrIeNdLy"

View File

@@ -8,11 +8,11 @@ use crossbeam::channel as crossbeam_channel;
use python_version::PythonVersion;
use red_knot_python_semantic::SitePackages;
use red_knot_server::run_server;
use red_knot_workspace::db::RootDatabase;
use red_knot_workspace::db::ProjectDatabase;
use red_knot_workspace::project::settings::Configuration;
use red_knot_workspace::project::ProjectMetadata;
use red_knot_workspace::watch;
use red_knot_workspace::watch::WorkspaceWatcher;
use red_knot_workspace::workspace::settings::Configuration;
use red_knot_workspace::workspace::WorkspaceMetadata;
use red_knot_workspace::watch::ProjectWatcher;
use ruff_db::diagnostic::Diagnostic;
use ruff_db::system::{OsSystem, System, SystemPath, SystemPathBuf};
use salsa::plumbing::ZalsaDatabase;
@@ -165,7 +165,7 @@ fn run() -> anyhow::Result<ExitStatus> {
let system = OsSystem::new(cwd.clone());
let cli_configuration = args.to_configuration(&cwd);
let workspace_metadata = WorkspaceMetadata::discover(
let workspace_metadata = ProjectMetadata::discover(
system.current_directory(),
&system,
Some(&cli_configuration),
@@ -173,7 +173,7 @@ fn run() -> anyhow::Result<ExitStatus> {
// TODO: Use the `program_settings` to compute the key for the database's persistent
// cache and load the cache if it exists.
let mut db = RootDatabase::new(workspace_metadata, system)?;
let mut db = ProjectDatabase::new(workspace_metadata, system)?;
let (main_loop, main_loop_cancellation_token) = MainLoop::new(cli_configuration);
@@ -226,7 +226,7 @@ struct MainLoop {
receiver: crossbeam_channel::Receiver<MainLoopMessage>,
/// The file system watcher, if running in watch mode.
watcher: Option<WorkspaceWatcher>,
watcher: Option<ProjectWatcher>,
cli_configuration: Configuration,
}
@@ -246,21 +246,21 @@ impl MainLoop {
)
}
fn watch(mut self, db: &mut RootDatabase) -> anyhow::Result<ExitStatus> {
fn watch(mut self, db: &mut ProjectDatabase) -> anyhow::Result<ExitStatus> {
tracing::debug!("Starting watch mode");
let sender = self.sender.clone();
let watcher = watch::directory_watcher(move |event| {
sender.send(MainLoopMessage::ApplyChanges(event)).unwrap();
})?;
self.watcher = Some(WorkspaceWatcher::new(watcher, db));
self.watcher = Some(ProjectWatcher::new(watcher, db));
self.run(db);
Ok(ExitStatus::Success)
}
fn run(mut self, db: &mut RootDatabase) -> ExitStatus {
fn run(mut self, db: &mut ProjectDatabase) -> ExitStatus {
self.sender.send(MainLoopMessage::CheckWorkspace).unwrap();
let result = self.main_loop(db);
@@ -270,7 +270,7 @@ impl MainLoop {
result
}
fn main_loop(&mut self, db: &mut RootDatabase) -> ExitStatus {
fn main_loop(&mut self, db: &mut ProjectDatabase) -> ExitStatus {
// Schedule the first check.
tracing::debug!("Starting main loop");
@@ -282,7 +282,7 @@ impl MainLoop {
let db = db.clone();
let sender = self.sender.clone();
// Spawn a new task that checks the workspace. This needs to be done in a separate thread
// Spawn a new task that checks the project. This needs to be done in a separate thread
// to prevent blocking the main loop here.
rayon::spawn(move || {
if let Ok(result) = db.check() {

View File

@@ -5,18 +5,18 @@ use std::time::{Duration, Instant};
use anyhow::{anyhow, Context};
use red_knot_python_semantic::{resolve_module, ModuleName, Program, PythonVersion, SitePackages};
use red_knot_workspace::db::{Db, RootDatabase};
use red_knot_workspace::watch::{directory_watcher, ChangeEvent, WorkspaceWatcher};
use red_knot_workspace::workspace::settings::{Configuration, SearchPathConfiguration};
use red_knot_workspace::workspace::WorkspaceMetadata;
use red_knot_workspace::db::{Db, ProjectDatabase};
use red_knot_workspace::project::settings::{Configuration, SearchPathConfiguration};
use red_knot_workspace::project::ProjectMetadata;
use red_knot_workspace::watch::{directory_watcher, ChangeEvent, ProjectWatcher};
use ruff_db::files::{system_path_to_file, File, FileError};
use ruff_db::source::source_text;
use ruff_db::system::{OsSystem, SystemPath, SystemPathBuf};
use ruff_db::Upcast;
struct TestCase {
db: RootDatabase,
watcher: Option<WorkspaceWatcher>,
db: ProjectDatabase,
watcher: Option<ProjectWatcher>,
changes_receiver: crossbeam::channel::Receiver<Vec<ChangeEvent>>,
/// The temporary directory that contains the test files.
/// We need to hold on to it in the test case or the temp files get deleted.
@@ -26,15 +26,15 @@ struct TestCase {
}
impl TestCase {
fn workspace_path(&self, relative: impl AsRef<SystemPath>) -> SystemPathBuf {
SystemPath::absolute(relative, self.db.workspace().root(&self.db))
fn project_path(&self, relative: impl AsRef<SystemPath>) -> SystemPathBuf {
SystemPath::absolute(relative, self.db.project().root(&self.db))
}
fn root_path(&self) -> &SystemPath {
&self.root_dir
}
fn db(&self) -> &RootDatabase {
fn db(&self) -> &ProjectDatabase {
&self.db
}
@@ -150,7 +150,7 @@ impl TestCase {
) -> anyhow::Result<()> {
let program = Program::get(self.db());
let new_settings = configuration.to_settings(self.db.workspace().root(&self.db));
let new_settings = configuration.to_settings(self.db.project().root(&self.db));
self.configuration.search_paths = configuration;
program.update_search_paths(&mut self.db, &new_settings)?;
@@ -163,9 +163,8 @@ impl TestCase {
Ok(())
}
fn collect_package_files(&self, path: &SystemPath) -> Vec<File> {
let package = self.db().workspace().package(self.db(), path).unwrap();
let files = package.files(self.db());
fn collect_project_files(&self) -> Vec<File> {
let files = self.db().project().files(self.db());
let mut collected: Vec<_> = files.into_iter().collect();
collected.sort_unstable_by_key(|file| file.path(self.db()).as_system_path().unwrap());
collected
@@ -194,17 +193,17 @@ where
}
trait SetupFiles {
fn setup(self, root_path: &SystemPath, workspace_path: &SystemPath) -> anyhow::Result<()>;
fn setup(self, root_path: &SystemPath, project_path: &SystemPath) -> anyhow::Result<()>;
}
impl<const N: usize, P> SetupFiles for [(P, &'static str); N]
where
P: AsRef<SystemPath>,
{
fn setup(self, _root_path: &SystemPath, workspace_path: &SystemPath) -> anyhow::Result<()> {
fn setup(self, _root_path: &SystemPath, project_path: &SystemPath) -> anyhow::Result<()> {
for (relative_path, content) in self {
let relative_path = relative_path.as_ref();
let absolute_path = workspace_path.join(relative_path);
let absolute_path = project_path.join(relative_path);
if let Some(parent) = absolute_path.parent() {
std::fs::create_dir_all(parent).with_context(|| {
format!("Failed to create parent directory for file `{relative_path}`")
@@ -226,8 +225,8 @@ impl<F> SetupFiles for F
where
F: FnOnce(&SystemPath, &SystemPath) -> anyhow::Result<()>,
{
fn setup(self, root_path: &SystemPath, workspace_path: &SystemPath) -> anyhow::Result<()> {
self(root_path, workspace_path)
fn setup(self, root_path: &SystemPath, project_path: &SystemPath) -> anyhow::Result<()> {
self(root_path, project_path)
}
}
@@ -235,7 +234,7 @@ fn setup<F>(setup_files: F) -> anyhow::Result<TestCase>
where
F: SetupFiles,
{
setup_with_search_paths(setup_files, |_root, _workspace_path| {
setup_with_search_paths(setup_files, |_root, _project_path| {
SearchPathConfiguration::default()
})
}
@@ -265,18 +264,18 @@ where
.simplified()
.to_path_buf();
let workspace_path = root_path.join("workspace");
let project_path = root_path.join("project");
std::fs::create_dir_all(workspace_path.as_std_path())
.with_context(|| format!("Failed to create workspace directory `{workspace_path}`"))?;
std::fs::create_dir_all(project_path.as_std_path())
.with_context(|| format!("Failed to create project directory `{project_path}`"))?;
setup_files
.setup(&root_path, &workspace_path)
.setup(&root_path, &project_path)
.context("Failed to setup test files")?;
let system = OsSystem::new(&workspace_path);
let system = OsSystem::new(&project_path);
let search_paths = create_search_paths(&root_path, &workspace_path);
let search_paths = create_search_paths(&root_path, &project_path);
for path in search_paths
.extra_paths
@@ -300,15 +299,15 @@ where
search_paths,
};
let workspace = WorkspaceMetadata::discover(&workspace_path, &system, Some(&configuration))?;
let project = ProjectMetadata::discover(&project_path, &system, Some(&configuration))?;
let db = RootDatabase::new(workspace, system)?;
let db = ProjectDatabase::new(project, system)?;
let (sender, receiver) = crossbeam::channel::unbounded();
let watcher = directory_watcher(move |events| sender.send(events).unwrap())
.with_context(|| "Failed to create directory watcher")?;
let watcher = WorkspaceWatcher::new(watcher, &db);
let watcher = ProjectWatcher::new(watcher, &db);
assert!(!watcher.has_errored_paths());
let test_case = TestCase {
@@ -359,12 +358,12 @@ fn update_file(path: impl AsRef<SystemPath>, content: &str) -> anyhow::Result<()
#[test]
fn new_file() -> anyhow::Result<()> {
let mut case = setup([("bar.py", "")])?;
let bar_path = case.workspace_path("bar.py");
let bar_path = case.project_path("bar.py");
let bar_file = case.system_file(&bar_path).unwrap();
let foo_path = case.workspace_path("foo.py");
let foo_path = case.project_path("foo.py");
assert_eq!(case.system_file(&foo_path), Err(FileError::NotFound));
assert_eq!(&case.collect_package_files(&bar_path), &[bar_file]);
assert_eq!(&case.collect_project_files(), &[bar_file]);
std::fs::write(foo_path.as_std_path(), "print('Hello')")?;
@@ -374,7 +373,7 @@ fn new_file() -> anyhow::Result<()> {
let foo = case.system_file(&foo_path).expect("foo.py to exist.");
assert_eq!(&case.collect_package_files(&bar_path), &[bar_file, foo]);
assert_eq!(&case.collect_project_files(), &[bar_file, foo]);
Ok(())
}
@@ -382,12 +381,12 @@ fn new_file() -> anyhow::Result<()> {
#[test]
fn new_ignored_file() -> anyhow::Result<()> {
let mut case = setup([("bar.py", ""), (".ignore", "foo.py")])?;
let bar_path = case.workspace_path("bar.py");
let bar_path = case.project_path("bar.py");
let bar_file = case.system_file(&bar_path).unwrap();
let foo_path = case.workspace_path("foo.py");
let foo_path = case.project_path("foo.py");
assert_eq!(case.system_file(&foo_path), Err(FileError::NotFound));
assert_eq!(&case.collect_package_files(&bar_path), &[bar_file]);
assert_eq!(&case.collect_project_files(), &[bar_file]);
std::fs::write(foo_path.as_std_path(), "print('Hello')")?;
@@ -396,7 +395,7 @@ fn new_ignored_file() -> anyhow::Result<()> {
case.apply_changes(changes);
assert!(case.system_file(&foo_path).is_ok());
assert_eq!(&case.collect_package_files(&bar_path), &[bar_file]);
assert_eq!(&case.collect_project_files(), &[bar_file]);
Ok(())
}
@@ -405,11 +404,11 @@ fn new_ignored_file() -> anyhow::Result<()> {
fn changed_file() -> anyhow::Result<()> {
let foo_source = "print('Hello, world!')";
let mut case = setup([("foo.py", foo_source)])?;
let foo_path = case.workspace_path("foo.py");
let foo_path = case.project_path("foo.py");
let foo = case.system_file(&foo_path)?;
assert_eq!(source_text(case.db(), foo).as_str(), foo_source);
assert_eq!(&case.collect_package_files(&foo_path), &[foo]);
assert_eq!(&case.collect_project_files(), &[foo]);
update_file(&foo_path, "print('Version 2')")?;
@@ -420,7 +419,7 @@ fn changed_file() -> anyhow::Result<()> {
case.apply_changes(changes);
assert_eq!(source_text(case.db(), foo).as_str(), "print('Version 2')");
assert_eq!(&case.collect_package_files(&foo_path), &[foo]);
assert_eq!(&case.collect_project_files(), &[foo]);
Ok(())
}
@@ -429,12 +428,12 @@ fn changed_file() -> anyhow::Result<()> {
fn deleted_file() -> anyhow::Result<()> {
let foo_source = "print('Hello, world!')";
let mut case = setup([("foo.py", foo_source)])?;
let foo_path = case.workspace_path("foo.py");
let foo_path = case.project_path("foo.py");
let foo = case.system_file(&foo_path)?;
assert!(foo.exists(case.db()));
assert_eq!(&case.collect_package_files(&foo_path), &[foo]);
assert_eq!(&case.collect_project_files(), &[foo]);
std::fs::remove_file(foo_path.as_std_path())?;
@@ -443,7 +442,7 @@ fn deleted_file() -> anyhow::Result<()> {
case.apply_changes(changes);
assert!(!foo.exists(case.db()));
assert_eq!(&case.collect_package_files(&foo_path), &[] as &[File]);
assert_eq!(&case.collect_project_files(), &[] as &[File]);
Ok(())
}
@@ -455,7 +454,7 @@ fn deleted_file() -> anyhow::Result<()> {
fn move_file_to_trash() -> anyhow::Result<()> {
let foo_source = "print('Hello, world!')";
let mut case = setup([("foo.py", foo_source)])?;
let foo_path = case.workspace_path("foo.py");
let foo_path = case.project_path("foo.py");
let trash_path = case.root_path().join(".trash");
std::fs::create_dir_all(trash_path.as_std_path())?;
@@ -463,7 +462,7 @@ fn move_file_to_trash() -> anyhow::Result<()> {
let foo = case.system_file(&foo_path)?;
assert!(foo.exists(case.db()));
assert_eq!(&case.collect_package_files(&foo_path), &[foo]);
assert_eq!(&case.collect_project_files(), &[foo]);
std::fs::rename(
foo_path.as_std_path(),
@@ -475,58 +474,50 @@ fn move_file_to_trash() -> anyhow::Result<()> {
case.apply_changes(changes);
assert!(!foo.exists(case.db()));
assert_eq!(&case.collect_package_files(&foo_path), &[] as &[File]);
assert_eq!(&case.collect_project_files(), &[] as &[File]);
Ok(())
}
/// Move a file from a non-workspace (non-watched) location into the workspace.
/// Move a file from a non-project (non-watched) location into the project.
#[test]
fn move_file_to_workspace() -> anyhow::Result<()> {
fn move_file_to_project() -> anyhow::Result<()> {
let mut case = setup([("bar.py", "")])?;
let bar_path = case.workspace_path("bar.py");
let bar_path = case.project_path("bar.py");
let bar = case.system_file(&bar_path).unwrap();
let foo_path = case.root_path().join("foo.py");
std::fs::write(foo_path.as_std_path(), "")?;
let foo_in_workspace_path = case.workspace_path("foo.py");
let foo_in_project = case.project_path("foo.py");
assert!(case.system_file(&foo_path).is_ok());
assert_eq!(&case.collect_package_files(&bar_path), &[bar]);
assert!(case
.db()
.workspace()
.package(case.db(), &foo_path)
.is_none());
assert_eq!(&case.collect_project_files(), &[bar]);
std::fs::rename(foo_path.as_std_path(), foo_in_workspace_path.as_std_path())?;
std::fs::rename(foo_path.as_std_path(), foo_in_project.as_std_path())?;
let changes = case.stop_watch(event_for_file("foo.py"));
case.apply_changes(changes);
let foo_in_workspace = case.system_file(&foo_in_workspace_path)?;
let foo_in_project = case.system_file(&foo_in_project)?;
assert!(foo_in_workspace.exists(case.db()));
assert_eq!(
&case.collect_package_files(&foo_in_workspace_path),
&[bar, foo_in_workspace]
);
assert!(foo_in_project.exists(case.db()));
assert_eq!(&case.collect_project_files(), &[bar, foo_in_project]);
Ok(())
}
/// Rename a workspace file.
/// Rename a project file.
#[test]
fn rename_file() -> anyhow::Result<()> {
let mut case = setup([("foo.py", "")])?;
let foo_path = case.workspace_path("foo.py");
let bar_path = case.workspace_path("bar.py");
let foo_path = case.project_path("foo.py");
let bar_path = case.project_path("bar.py");
let foo = case.system_file(&foo_path)?;
assert_eq!(case.collect_package_files(&foo_path), [foo]);
assert_eq!(case.collect_project_files(), [foo]);
std::fs::rename(foo_path.as_std_path(), bar_path.as_std_path())?;
@@ -539,15 +530,15 @@ fn rename_file() -> anyhow::Result<()> {
let bar = case.system_file(&bar_path)?;
assert!(bar.exists(case.db()));
assert_eq!(case.collect_package_files(&foo_path), [bar]);
assert_eq!(case.collect_project_files(), [bar]);
Ok(())
}
#[test]
fn directory_moved_to_workspace() -> anyhow::Result<()> {
fn directory_moved_to_project() -> anyhow::Result<()> {
let mut case = setup([("bar.py", "import sub.a")])?;
let bar = case.system_file(case.workspace_path("bar.py")).unwrap();
let bar = case.system_file(case.project_path("bar.py")).unwrap();
let sub_original_path = case.root_path().join("sub");
let init_original_path = sub_original_path.join("__init__.py");
@@ -565,12 +556,9 @@ fn directory_moved_to_workspace() -> anyhow::Result<()> {
);
assert_eq!(sub_a_module, None);
assert_eq!(
case.collect_package_files(&case.workspace_path("bar.py")),
&[bar]
);
assert_eq!(case.collect_project_files(), &[bar]);
let sub_new_path = case.workspace_path("sub");
let sub_new_path = case.project_path("sub");
std::fs::rename(sub_original_path.as_std_path(), sub_new_path.as_std_path())
.with_context(|| "Failed to move sub directory")?;
@@ -592,10 +580,7 @@ fn directory_moved_to_workspace() -> anyhow::Result<()> {
)
.is_some());
assert_eq!(
case.collect_package_files(&case.workspace_path("bar.py")),
&[bar, init_file, a_file]
);
assert_eq!(case.collect_project_files(), &[bar, init_file, a_file]);
Ok(())
}
@@ -607,7 +592,7 @@ fn directory_moved_to_trash() -> anyhow::Result<()> {
("sub/__init__.py", ""),
("sub/a.py", ""),
])?;
let bar = case.system_file(case.workspace_path("bar.py")).unwrap();
let bar = case.system_file(case.project_path("bar.py")).unwrap();
assert!(resolve_module(
case.db().upcast(),
@@ -615,7 +600,7 @@ fn directory_moved_to_trash() -> anyhow::Result<()> {
)
.is_some());
let sub_path = case.workspace_path("sub");
let sub_path = case.project_path("sub");
let init_file = case
.system_file(sub_path.join("__init__.py"))
.expect("__init__.py to exist");
@@ -623,10 +608,7 @@ fn directory_moved_to_trash() -> anyhow::Result<()> {
.system_file(sub_path.join("a.py"))
.expect("a.py to exist");
assert_eq!(
case.collect_package_files(&case.workspace_path("bar.py")),
&[bar, init_file, a_file]
);
assert_eq!(case.collect_project_files(), &[bar, init_file, a_file]);
std::fs::create_dir(case.root_path().join(".trash").as_std_path())?;
let trashed_sub = case.root_path().join(".trash/sub");
@@ -647,10 +629,7 @@ fn directory_moved_to_trash() -> anyhow::Result<()> {
assert!(!init_file.exists(case.db()));
assert!(!a_file.exists(case.db()));
assert_eq!(
case.collect_package_files(&case.workspace_path("bar.py")),
&[bar]
);
assert_eq!(case.collect_project_files(), &[bar]);
Ok(())
}
@@ -663,7 +642,7 @@ fn directory_renamed() -> anyhow::Result<()> {
("sub/a.py", ""),
])?;
let bar = case.system_file(case.workspace_path("bar.py")).unwrap();
let bar = case.system_file(case.project_path("bar.py")).unwrap();
assert!(resolve_module(
case.db().upcast(),
@@ -676,7 +655,7 @@ fn directory_renamed() -> anyhow::Result<()> {
)
.is_none());
let sub_path = case.workspace_path("sub");
let sub_path = case.project_path("sub");
let sub_init = case
.system_file(sub_path.join("__init__.py"))
.expect("__init__.py to exist");
@@ -684,14 +663,11 @@ fn directory_renamed() -> anyhow::Result<()> {
.system_file(sub_path.join("a.py"))
.expect("a.py to exist");
assert_eq!(
case.collect_package_files(&sub_path),
&[bar, sub_init, sub_a]
);
assert_eq!(case.collect_project_files(), &[bar, sub_init, sub_a]);
let foo_baz = case.workspace_path("foo/baz");
let foo_baz = case.project_path("foo/baz");
std::fs::create_dir(case.workspace_path("foo").as_std_path())?;
std::fs::create_dir(case.project_path("foo").as_std_path())?;
std::fs::rename(sub_path.as_std_path(), foo_baz.as_std_path())
.with_context(|| "Failed to move the sub directory")?;
@@ -730,7 +706,7 @@ fn directory_renamed() -> anyhow::Result<()> {
assert!(foo_baz_a.exists(case.db()));
assert_eq!(
case.collect_package_files(&sub_path),
case.collect_project_files(),
&[bar, foo_baz_init, foo_baz_a]
);
@@ -745,7 +721,7 @@ fn directory_deleted() -> anyhow::Result<()> {
("sub/a.py", ""),
])?;
let bar = case.system_file(case.workspace_path("bar.py")).unwrap();
let bar = case.system_file(case.project_path("bar.py")).unwrap();
assert!(resolve_module(
case.db().upcast(),
@@ -753,7 +729,7 @@ fn directory_deleted() -> anyhow::Result<()> {
)
.is_some());
let sub_path = case.workspace_path("sub");
let sub_path = case.project_path("sub");
let init_file = case
.system_file(sub_path.join("__init__.py"))
@@ -761,10 +737,7 @@ fn directory_deleted() -> anyhow::Result<()> {
let a_file = case
.system_file(sub_path.join("a.py"))
.expect("a.py to exist");
assert_eq!(
case.collect_package_files(&sub_path),
&[bar, init_file, a_file]
);
assert_eq!(case.collect_project_files(), &[bar, init_file, a_file]);
std::fs::remove_dir_all(sub_path.as_std_path())
.with_context(|| "Failed to remove the sub directory")?;
@@ -782,20 +755,20 @@ fn directory_deleted() -> anyhow::Result<()> {
assert!(!init_file.exists(case.db()));
assert!(!a_file.exists(case.db()));
assert_eq!(case.collect_package_files(&sub_path), &[bar]);
assert_eq!(case.collect_project_files(), &[bar]);
Ok(())
}
#[test]
fn search_path() -> anyhow::Result<()> {
let mut case = setup_with_search_paths(
[("bar.py", "import sub.a")],
|root_path, _workspace_path| SearchPathConfiguration {
site_packages: Some(SitePackages::Known(vec![root_path.join("site_packages")])),
..SearchPathConfiguration::default()
},
)?;
let mut case =
setup_with_search_paths([("bar.py", "import sub.a")], |root_path, _project_path| {
SearchPathConfiguration {
site_packages: Some(SitePackages::Known(vec![root_path.join("site_packages")])),
..SearchPathConfiguration::default()
}
})?;
let site_packages = case.root_path().join("site_packages");
@@ -812,8 +785,8 @@ fn search_path() -> anyhow::Result<()> {
assert!(resolve_module(case.db().upcast(), &ModuleName::new_static("a").unwrap()).is_some());
assert_eq!(
case.collect_package_files(&case.workspace_path("bar.py")),
&[case.system_file(case.workspace_path("bar.py")).unwrap()]
case.collect_project_files(),
&[case.system_file(case.project_path("bar.py")).unwrap()]
);
Ok(())
@@ -823,7 +796,7 @@ fn search_path() -> anyhow::Result<()> {
fn add_search_path() -> anyhow::Result<()> {
let mut case = setup([("bar.py", "import sub.a")])?;
let site_packages = case.workspace_path("site_packages");
let site_packages = case.project_path("site_packages");
std::fs::create_dir_all(site_packages.as_std_path())?;
assert!(resolve_module(case.db().upcast(), &ModuleName::new_static("a").unwrap()).is_none());
@@ -848,13 +821,13 @@ fn add_search_path() -> anyhow::Result<()> {
#[test]
fn remove_search_path() -> anyhow::Result<()> {
let mut case = setup_with_search_paths(
[("bar.py", "import sub.a")],
|root_path, _workspace_path| SearchPathConfiguration {
site_packages: Some(SitePackages::Known(vec![root_path.join("site_packages")])),
..SearchPathConfiguration::default()
},
)?;
let mut case =
setup_with_search_paths([("bar.py", "import sub.a")], |root_path, _project_path| {
SearchPathConfiguration {
site_packages: Some(SitePackages::Known(vec![root_path.join("site_packages")])),
..SearchPathConfiguration::default()
}
})?;
// Remove site packages from the search path settings.
let site_packages = case.root_path().join("site_packages");
@@ -876,8 +849,8 @@ fn remove_search_path() -> anyhow::Result<()> {
#[test]
fn changed_versions_file() -> anyhow::Result<()> {
let mut case = setup_with_search_paths(
|root_path: &SystemPath, workspace_path: &SystemPath| {
std::fs::write(workspace_path.join("bar.py").as_std_path(), "import sub.a")?;
|root_path: &SystemPath, project_path: &SystemPath| {
std::fs::write(project_path.join("bar.py").as_std_path(), "import sub.a")?;
std::fs::create_dir_all(root_path.join("typeshed/stdlib").as_std_path())?;
std::fs::write(root_path.join("typeshed/stdlib/VERSIONS").as_std_path(), "")?;
std::fs::write(
@@ -887,7 +860,7 @@ fn changed_versions_file() -> anyhow::Result<()> {
Ok(())
},
|root_path, _workspace_path| SearchPathConfiguration {
|root_path, _project_path| SearchPathConfiguration {
typeshed: Some(root_path.join("typeshed")),
..SearchPathConfiguration::default()
},
@@ -915,11 +888,11 @@ fn changed_versions_file() -> anyhow::Result<()> {
Ok(())
}
/// Watch a workspace that contains two files where one file is a hardlink to another.
/// Watch a project that contains two files where one file is a hardlink to another.
///
/// Setup:
/// ```text
/// - workspace
/// - project
/// |- foo.py
/// |- bar.py (hard link to foo.py)
/// ```
@@ -935,22 +908,22 @@ fn changed_versions_file() -> anyhow::Result<()> {
/// I haven't found any documentation that states the notification behavior on Windows but what
/// we're seeing is that Windows only emits a single event, similar to Linux.
#[test]
fn hard_links_in_workspace() -> anyhow::Result<()> {
let mut case = setup(|_root: &SystemPath, workspace: &SystemPath| {
let foo_path = workspace.join("foo.py");
fn hard_links_in_project() -> anyhow::Result<()> {
let mut case = setup(|_root: &SystemPath, project: &SystemPath| {
let foo_path = project.join("foo.py");
std::fs::write(foo_path.as_std_path(), "print('Version 1')")?;
// Create a hardlink to `foo`
let bar_path = workspace.join("bar.py");
let bar_path = project.join("bar.py");
std::fs::hard_link(foo_path.as_std_path(), bar_path.as_std_path())
.context("Failed to create hard link from foo.py -> bar.py")?;
Ok(())
})?;
let foo_path = case.workspace_path("foo.py");
let foo_path = case.project_path("foo.py");
let foo = case.system_file(&foo_path).unwrap();
let bar_path = case.workspace_path("bar.py");
let bar_path = case.project_path("bar.py");
let bar = case.system_file(&bar_path).unwrap();
assert_eq!(source_text(case.db(), foo).as_str(), "print('Version 1')");
@@ -973,12 +946,12 @@ fn hard_links_in_workspace() -> anyhow::Result<()> {
Ok(())
}
/// Watch a workspace that contains one file that is a hardlink to a file outside the workspace.
/// Watch a project that contains one file that is a hardlink to a file outside the project.
///
/// Setup:
/// ```text
/// - foo.py
/// - workspace
/// - project
/// |- bar.py (hard link to /foo.py)
/// ```
///
@@ -996,7 +969,7 @@ fn hard_links_in_workspace() -> anyhow::Result<()> {
/// [source](https://learn.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-readdirectorychangesw)
///
/// My interpretation of this is that Windows doesn't support observing changes made to
/// hard linked files outside the workspace.
/// hard linked files outside the project.
#[test]
#[cfg_attr(
target_os = "linux",
@@ -1006,13 +979,13 @@ fn hard_links_in_workspace() -> anyhow::Result<()> {
target_os = "windows",
ignore = "windows doesn't support observing changes to hard linked files."
)]
fn hard_links_to_target_outside_workspace() -> anyhow::Result<()> {
let mut case = setup(|root: &SystemPath, workspace: &SystemPath| {
fn hard_links_to_target_outside_project() -> anyhow::Result<()> {
let mut case = setup(|root: &SystemPath, project: &SystemPath| {
let foo_path = root.join("foo.py");
std::fs::write(foo_path.as_std_path(), "print('Version 1')")?;
// Create a hardlink to `foo`
let bar_path = workspace.join("bar.py");
let bar_path = project.join("bar.py");
std::fs::hard_link(foo_path.as_std_path(), bar_path.as_std_path())
.context("Failed to create hard link from foo.py -> bar.py")?;
@@ -1021,7 +994,7 @@ fn hard_links_to_target_outside_workspace() -> anyhow::Result<()> {
let foo_path = case.root_path().join("foo.py");
let foo = case.system_file(&foo_path).unwrap();
let bar_path = case.workspace_path("bar.py");
let bar_path = case.project_path("bar.py");
let bar = case.system_file(&bar_path).unwrap();
assert_eq!(source_text(case.db(), foo).as_str(), "print('Version 1')");
@@ -1044,13 +1017,13 @@ mod unix {
//! Tests that make use of unix specific file-system features.
use super::*;
/// Changes the metadata of the only file in the workspace.
/// Changes the metadata of the only file in the project.
#[test]
fn changed_metadata() -> anyhow::Result<()> {
use std::os::unix::fs::PermissionsExt;
let mut case = setup([("foo.py", "")])?;
let foo_path = case.workspace_path("foo.py");
let foo_path = case.project_path("foo.py");
let foo = case.system_file(&foo_path)?;
assert_eq!(
@@ -1086,14 +1059,14 @@ mod unix {
Ok(())
}
/// A workspace path is a symlink to a file outside the workspace.
/// A project path is a symlink to a file outside the project.
///
/// Setup:
/// ```text
/// - bar
/// |- baz.py
///
/// - workspace
/// - project
/// |- bar -> /bar
/// ```
///
@@ -1115,7 +1088,7 @@ mod unix {
ignore = "FSEvents doesn't emit change events for symlinked directories outside of the watched paths."
)]
fn symlink_target_outside_watched_paths() -> anyhow::Result<()> {
let mut case = setup(|root: &SystemPath, workspace: &SystemPath| {
let mut case = setup(|root: &SystemPath, project: &SystemPath| {
// Set up the symlink target.
let link_target = root.join("bar");
std::fs::create_dir_all(link_target.as_std_path())
@@ -1124,8 +1097,8 @@ mod unix {
std::fs::write(baz_original.as_std_path(), "def baz(): ...")
.context("Failed to write link target file")?;
// Create a symlink inside the workspace
let bar = workspace.join("bar");
// Create a symlink inside the project
let bar = project.join("bar");
std::os::unix::fs::symlink(link_target.as_std_path(), bar.as_std_path())
.context("Failed to create symlink to bar package")?;
@@ -1137,7 +1110,7 @@ mod unix {
&ModuleName::new_static("bar.baz").unwrap(),
)
.expect("Expected bar.baz to exist in site-packages.");
let baz_workspace = case.workspace_path("bar/baz.py");
let baz_project = case.project_path("bar/baz.py");
assert_eq!(
source_text(case.db(), baz.file()).as_str(),
@@ -1145,7 +1118,7 @@ mod unix {
);
assert_eq!(
baz.file().path(case.db()).as_system_path(),
Some(&*baz_workspace)
Some(&*baz_project)
);
let baz_original = case.root_path().join("bar/baz.py");
@@ -1164,7 +1137,7 @@ mod unix {
);
// Write to the symlink source.
update_file(baz_workspace, "def baz(): print('Version 3')")
update_file(baz_project, "def baz(): print('Version 3')")
.context("Failed to update bar/baz.py")?;
let changes = case.stop_watch(event_for_file("baz.py"));
@@ -1179,14 +1152,14 @@ mod unix {
Ok(())
}
/// Workspace contains a symlink to another directory inside the workspace.
/// Project contains a symlink to another directory inside the project.
/// Changes to files in the symlinked directory should be reflected
/// to all files.
///
/// Setup:
/// ```text
/// - workspace
/// | - bar -> /workspace/patched/bar
/// - project
/// | - bar -> /project/patched/bar
/// |
/// | - patched
/// | |-- bar
@@ -1195,10 +1168,10 @@ mod unix {
/// |-- foo.py
/// ```
#[test]
fn symlink_inside_workspace() -> anyhow::Result<()> {
let mut case = setup(|_root: &SystemPath, workspace: &SystemPath| {
fn symlink_inside_project() -> anyhow::Result<()> {
let mut case = setup(|_root: &SystemPath, project: &SystemPath| {
// Set up the symlink target.
let link_target = workspace.join("patched/bar");
let link_target = project.join("patched/bar");
std::fs::create_dir_all(link_target.as_std_path())
.context("Failed to create link target directory")?;
let baz_original = link_target.join("baz.py");
@@ -1206,8 +1179,8 @@ mod unix {
.context("Failed to write link target file")?;
// Create a symlink inside site-packages
let bar_in_workspace = workspace.join("bar");
std::os::unix::fs::symlink(link_target.as_std_path(), bar_in_workspace.as_std_path())
let bar_in_project = project.join("bar");
std::os::unix::fs::symlink(link_target.as_std_path(), bar_in_project.as_std_path())
.context("Failed to create symlink to bar package")?;
Ok(())
@@ -1218,9 +1191,9 @@ mod unix {
&ModuleName::new_static("bar.baz").unwrap(),
)
.expect("Expected bar.baz to exist in site-packages.");
let bar_baz = case.workspace_path("bar/baz.py");
let bar_baz = case.project_path("bar/baz.py");
let patched_bar_baz = case.workspace_path("patched/bar/baz.py");
let patched_bar_baz = case.project_path("patched/bar/baz.py");
let patched_bar_baz_file = case.system_file(&patched_bar_baz).unwrap();
assert_eq!(
@@ -1279,7 +1252,7 @@ mod unix {
/// - site-packages
/// | - bar/baz.py
///
/// - workspace
/// - project
/// |-- .venv/lib/python3.12/site-packages -> /site-packages
/// |
/// |-- foo.py
@@ -1287,7 +1260,7 @@ mod unix {
#[test]
fn symlinked_module_search_path() -> anyhow::Result<()> {
let mut case = setup_with_search_paths(
|root: &SystemPath, workspace: &SystemPath| {
|root: &SystemPath, project: &SystemPath| {
// Set up the symlink target.
let site_packages = root.join("site-packages");
let bar = site_packages.join("bar");
@@ -1298,7 +1271,7 @@ mod unix {
.context("Failed to write baz.py")?;
// Symlink the site packages in the venv to the global site packages
let venv_site_packages = workspace.join(".venv/lib/python3.12/site-packages");
let venv_site_packages = project.join(".venv/lib/python3.12/site-packages");
std::fs::create_dir_all(venv_site_packages.parent().unwrap())
.context("Failed to create .venv directory")?;
std::os::unix::fs::symlink(
@@ -1309,9 +1282,9 @@ mod unix {
Ok(())
},
|_root, workspace| SearchPathConfiguration {
|_root, project| SearchPathConfiguration {
site_packages: Some(SitePackages::Known(vec![
workspace.join(".venv/lib/python3.12/site-packages")
project.join(".venv/lib/python3.12/site-packages")
])),
..SearchPathConfiguration::default()
},
@@ -1323,7 +1296,7 @@ mod unix {
)
.expect("Expected bar.baz to exist in site-packages.");
let baz_site_packages_path =
case.workspace_path(".venv/lib/python3.12/site-packages/bar/baz.py");
case.project_path(".venv/lib/python3.12/site-packages/bar/baz.py");
let baz_site_packages = case.system_file(&baz_site_packages_path).unwrap();
let baz_original = case.root_path().join("site-packages/bar/baz.py");
let baz_original_file = case.system_file(&baz_original).unwrap();
@@ -1372,13 +1345,15 @@ mod unix {
}
#[test]
fn nested_packages_delete_root() -> anyhow::Result<()> {
let mut case = setup(|root: &SystemPath, workspace_root: &SystemPath| {
fn nested_projects_delete_root() -> anyhow::Result<()> {
let mut case = setup(|root: &SystemPath, project_root: &SystemPath| {
std::fs::write(
workspace_root.join("pyproject.toml").as_std_path(),
project_root.join("pyproject.toml").as_std_path(),
r#"
[project]
name = "inner"
[tool.knot]
"#,
)?;
@@ -1387,120 +1362,24 @@ fn nested_packages_delete_root() -> anyhow::Result<()> {
r#"
[project]
name = "outer"
[tool.knot]
"#,
)?;
Ok(())
})?;
assert_eq!(
case.db().workspace().root(case.db()),
&*case.workspace_path("")
);
assert_eq!(case.db().project().root(case.db()), &*case.project_path(""));
std::fs::remove_file(case.workspace_path("pyproject.toml").as_std_path())?;
std::fs::remove_file(case.project_path("pyproject.toml").as_std_path())?;
let changes = case.stop_watch(ChangeEvent::is_deleted);
case.apply_changes(changes);
// It should now pick up the outer workspace.
assert_eq!(case.db().workspace().root(case.db()), case.root_path());
Ok(())
}
#[test]
fn added_package() -> anyhow::Result<()> {
let mut case = setup([
(
"pyproject.toml",
r#"
[project]
name = "inner"
[tool.knot.workspace]
members = ["packages/*"]
"#,
),
(
"packages/a/pyproject.toml",
r#"
[project]
name = "a"
"#,
),
])?;
assert_eq!(case.db().workspace().packages(case.db()).len(), 2);
std::fs::create_dir(case.workspace_path("packages/b").as_std_path())
.context("failed to create folder for package 'b'")?;
// It seems that the file watcher won't pick up on file changes shortly after the folder
// was created... I suspect this is because most file watchers don't support recursive
// file watching. Instead, file-watching libraries manually implement recursive file watching
// by setting a watcher for each directory. But doing this obviously "lags" behind.
case.take_watch_changes();
std::fs::write(
case.workspace_path("packages/b/pyproject.toml")
.as_std_path(),
r#"
[project]
name = "b"
"#,
)
.context("failed to write pyproject.toml for package b")?;
let changes = case.stop_watch(event_for_file("pyproject.toml"));
case.apply_changes(changes);
assert_eq!(case.db().workspace().packages(case.db()).len(), 3);
Ok(())
}
#[test]
fn removed_package() -> anyhow::Result<()> {
let mut case = setup([
(
"pyproject.toml",
r#"
[project]
name = "inner"
[tool.knot.workspace]
members = ["packages/*"]
"#,
),
(
"packages/a/pyproject.toml",
r#"
[project]
name = "a"
"#,
),
(
"packages/b/pyproject.toml",
r#"
[project]
name = "b"
"#,
),
])?;
assert_eq!(case.db().workspace().packages(case.db()).len(), 3);
std::fs::remove_dir_all(case.workspace_path("packages/b").as_std_path())
.context("failed to remove package 'b'")?;
let changes = case.stop_watch(ChangeEvent::is_deleted);
case.apply_changes(changes);
assert_eq!(case.db().workspace().packages(case.db()).len(), 2);
// It should now pick up the outer project.
assert_eq!(case.db().project().root(case.db()), case.root_path());
Ok(())
}

View File

@@ -173,3 +173,40 @@ p: "call()"
r: "[1, 2]"
s: "(1, 2)"
```
## Multi line annotation
Quoted type annotations should be parsed as if surrounded by parentheses.
```py
def valid(
a1: """(
int |
str
)
""",
a2: """
int |
str
""",
):
reveal_type(a1) # revealed: int | str
reveal_type(a2) # revealed: int | str
def invalid(
# error: [invalid-syntax-in-forward-annotation]
a1: """
int |
str)
""",
# error: [invalid-syntax-in-forward-annotation]
a2: """
int) |
str
""",
# error: [invalid-syntax-in-forward-annotation]
a3: """
(int)) """,
):
pass
```

View File

@@ -6,14 +6,11 @@ Several type qualifiers are unsupported by red-knot currently. However, we also
false-positive errors if you use one in an annotation:
```py
from typing_extensions import Final, ClassVar, Required, NotRequired, ReadOnly, TypedDict
from typing_extensions import Final, Required, NotRequired, ReadOnly, TypedDict
X: Final = 42
Y: Final[int] = 42
class Foo:
A: ClassVar[int] = 42
# TODO: `TypedDict` is actually valid as a base
# error: [invalid-base]
class Bar(TypedDict):

View File

@@ -2,6 +2,273 @@
Tests for attribute access on various kinds of types.
## Class and instance variables
### Pure instance variables
#### Variable only declared/bound in `__init__`
Variables only declared and/or bound in `__init__` are pure instance variables. They cannot be
accessed on the class itself.
```py
class C:
def __init__(self, value2: int, flag: bool = False) -> None:
# bound but not declared
self.pure_instance_variable1 = "value set in __init__"
# bound but not declared - with type inferred from parameter
self.pure_instance_variable2 = value2
# declared but not bound
self.pure_instance_variable3: bytes
# declared and bound
self.pure_instance_variable4: bool = True
# possibly undeclared/unbound
if flag:
self.pure_instance_variable5: str = "possibly set in __init__"
c_instance = C(1)
# TODO: should be `Literal["value set in __init__"]`, or `Unknown | Literal[…]` to allow
# assignments to this unannotated attribute from other scopes.
reveal_type(c_instance.pure_instance_variable1) # revealed: @Todo(instance attributes)
# TODO: should be `int`
reveal_type(c_instance.pure_instance_variable2) # revealed: @Todo(instance attributes)
# TODO: should be `bytes`
reveal_type(c_instance.pure_instance_variable3) # revealed: @Todo(instance attributes)
# TODO: should be `bool`
reveal_type(c_instance.pure_instance_variable4) # revealed: @Todo(instance attributes)
# TODO: should be `str`
# We probably don't want to emit a diagnostic for this being possibly undeclared/unbound.
# mypy and pyright do not show an error here.
reveal_type(c_instance.pure_instance_variable5) # revealed: @Todo(instance attributes)
# TODO: If we choose to infer a precise `Literal[…]` type for the instance attribute (see
# above), this should be an error: incompatible types in assignment. If we choose to infer
# a gradual `Unknown | Literal[…]` type, this assignment is fine.
c_instance.pure_instance_variable1 = "value set on instance"
# TODO: this should be an error (incompatible types in assignment)
c_instance.pure_instance_variable2 = "incompatible"
# TODO: we already show an error here but the message might be improved?
# mypy shows no error here, but pyright raises "reportAttributeAccessIssue"
# error: [unresolved-attribute] "Type `Literal[C]` has no attribute `pure_instance_variable1`"
reveal_type(C.pure_instance_variable1) # revealed: Unknown
# TODO: this should be an error (pure instance variables cannot be accessed on the class)
# mypy shows no error here, but pyright raises "reportAttributeAccessIssue"
C.pure_instance_variable1 = "overwritten on class"
c_instance.pure_instance_variable4 = False
# TODO: After this assignment to the attribute within this scope, we may eventually want to narrow
# the `bool` type (see above) for this instance variable to `Literal[False]` here. This is unsound
# in general (we don't know what else happened to `c_instance` between the assignment and the use
# here), but mypy and pyright support this. In conclusion, this could be `bool` but should probably
# be `Literal[False]`.
reveal_type(c_instance.pure_instance_variable4) # revealed: @Todo(instance attributes)
```
#### Variable declared in class body and declared/bound in `__init__`
The same rule applies even if the variable is *declared* (not bound!) in the class body: it is still
a pure instance variable.
```py
class C:
pure_instance_variable: str
def __init__(self) -> None:
self.pure_instance_variable = "value set in __init__"
c_instance = C()
# TODO: should be `str`
reveal_type(c_instance.pure_instance_variable) # revealed: @Todo(instance attributes)
# TODO: we currently plan to emit a diagnostic here. Note that both mypy
# and pyright show no error in this case! So we may reconsider this in
# the future, if it turns out to produce too many false positives.
reveal_type(C.pure_instance_variable) # revealed: str
# TODO: same as above. We plan to emit a diagnostic here, even if both mypy
# and pyright allow this.
C.pure_instance_variable = "overwritten on class"
# TODO: this should be an error (incompatible types in assignment)
c_instance.pure_instance_variable = 1
```
#### Variable only defined in unrelated method
We also recognize pure instance variables if they are defined in a method that is not `__init__`.
```py
class C:
def set_instance_variable(self) -> None:
self.pure_instance_variable = "value set in method"
c_instance = C()
# Not that we would use this in static analysis, but for a more realistic example, let's actually
# call the method, so that the attribute is bound if this example is actually run.
c_instance.set_instance_variable()
# TODO: should be `Literal["value set in method"]` or `Unknown | Literal[…]` (see above).
reveal_type(c_instance.pure_instance_variable) # revealed: @Todo(instance attributes)
# TODO: We already show an error here, but the message might be improved?
# error: [unresolved-attribute]
reveal_type(C.pure_instance_variable) # revealed: Unknown
# TODO: this should be an error
C.pure_instance_variable = "overwritten on class"
```
#### Variable declared in class body and not bound anywhere
If a variable is declared in the class body but not bound anywhere, we still consider it a pure
instance variable and allow access to it via instances.
```py
class C:
pure_instance_variable: str
c_instance = C()
# TODO: should be 'str'
reveal_type(c_instance.pure_instance_variable) # revealed: @Todo(instance attributes)
# TODO: mypy and pyright do not show an error here, but we plan to emit a diagnostic.
# The type could be changed to 'Unknown' if we decide to emit an error?
reveal_type(C.pure_instance_variable) # revealed: str
# TODO: mypy and pyright do not show an error here, but we plan to emit one.
C.pure_instance_variable = "overwritten on class"
```
### Pure class variables (`ClassVar`)
#### Annotated with `ClassVar` type qualifier
Class variables annotated with the [`typing.ClassVar`] type qualifier are pure class variables. They
cannot be overwritten on instances, but they can be accessed on instances.
For more details, see the [typing spec on `ClassVar`].
```py
from typing import ClassVar
class C:
pure_class_variable1: ClassVar[str] = "value in class body"
pure_class_variable2: ClassVar = 1
reveal_type(C.pure_class_variable1) # revealed: str
# TODO: this should be `Literal[1]`, or `Unknown | Literal[1]`.
reveal_type(C.pure_class_variable2) # revealed: @Todo(Unsupported or invalid type in a type expression)
c_instance = C()
# TODO: This should be `str`. It is okay to access a pure class variable on an instance.
reveal_type(c_instance.pure_class_variable1) # revealed: @Todo(instance attributes)
# TODO: should raise an error. It is not allowed to reassign a pure class variable on an instance.
c_instance.pure_class_variable1 = "value set on instance"
C.pure_class_variable1 = "overwritten on class"
# TODO: should raise an error (incompatible types in assignment)
C.pure_class_variable1 = 1
class Subclass(C):
pure_class_variable1: ClassVar[str] = "overwritten on subclass"
reveal_type(Subclass.pure_class_variable1) # revealed: str
```
#### Variable only mentioned in a class method
We also consider a class variable to be a pure class variable if it is only mentioned in a class
method.
```py
class C:
@classmethod
def class_method(cls):
cls.pure_class_variable = "value set in class method"
# for a more realistic example, let's actually call the method
C.class_method()
# TODO: We currently plan to support this and show no error here.
# mypy shows an error here, pyright does not.
# error: [unresolved-attribute]
reveal_type(C.pure_class_variable) # revealed: Unknown
C.pure_class_variable = "overwritten on class"
# TODO: should be `Literal["overwritten on class"]`
# error: [unresolved-attribute]
reveal_type(C.pure_class_variable) # revealed: Unknown
c_instance = C()
# TODO: should be `Literal["overwritten on class"]`
reveal_type(c_instance.pure_class_variable) # revealed: @Todo(instance attributes)
# TODO: should raise an error.
c_instance.pure_class_variable = "value set on instance"
```
### Instance variables with class-level default values
These are instance attributes, but the fact that we can see that they have a binding (not a
declaration) in the class body means that reading the value from the class directly is also
permitted. This is the only difference for these attributes as opposed to "pure" instance
attributes.
#### Basic
```py
class C:
variable_with_class_default: str = "value in class body"
def instance_method(self):
self.variable_with_class_default = "value set in instance method"
reveal_type(C.variable_with_class_default) # revealed: str
c_instance = C()
# TODO: should be `str`
reveal_type(c_instance.variable_with_class_default) # revealed: @Todo(instance attributes)
c_instance.variable_with_class_default = "value set on instance"
reveal_type(C.variable_with_class_default) # revealed: str
# TODO: Could be Literal["value set on instance"], or still `str` if we choose not to
# narrow the type.
reveal_type(c_instance.variable_with_class_default) # revealed: @Todo(instance attributes)
C.variable_with_class_default = "overwritten on class"
# TODO: Could be `Literal["overwritten on class"]`, or still `str` if we choose not to
# narrow the type.
reveal_type(C.variable_with_class_default) # revealed: str
# TODO: should still be `Literal["value set on instance"]`, or `str`.
reveal_type(c_instance.variable_with_class_default) # revealed: @Todo(instance attributes)
```
## Union of attributes
```py
@@ -24,7 +291,9 @@ def _(flag: bool):
reveal_type(C2.x) # revealed: Literal[3, 4]
```
## Inherited attributes
## Inherited class attributes
### Basic
```py
class A:
@@ -36,7 +305,7 @@ class C(B): ...
reveal_type(C.X) # revealed: Literal["foo"]
```
## Inherited attributes (multiple inheritance)
### Multiple inheritance
```py
class O: ...
@@ -104,7 +373,7 @@ def _(flag: bool, flag1: bool, flag2: bool):
reveal_type(C.x) # revealed: Literal[1, 2, 3]
```
## Unions with all paths unbound
### Unions with all paths unbound
If the symbol is unbound in all elements of the union, we detect that:
@@ -158,7 +427,9 @@ class Foo: ...
reveal_type(Foo.__class__) # revealed: Literal[type]
```
## Function-literal attributes
## Literal types
### Function-literal attributes
Most attribute accesses on function-literal types are delegated to `types.FunctionType`, since all
functions are instances of that class:
@@ -179,7 +450,7 @@ reveal_type(f.__get__) # revealed: @Todo(`__get__` method on functions)
reveal_type(f.__call__) # revealed: @Todo(`__call__` method on functions)
```
## Int-literal attributes
### Int-literal attributes
Most attribute accesses on int-literal types are delegated to `builtins.int`, since all literal
integers are instances of that class:
@@ -196,7 +467,7 @@ reveal_type((2).numerator) # revealed: Literal[2]
reveal_type((2).real) # revealed: Literal[2]
```
## Literal `bool` attributes
### Bool-literal attributes
Most attribute accesses on bool-literal types are delegated to `builtins.bool`, since all literal
bols are instances of that class:
@@ -213,7 +484,7 @@ reveal_type(True.numerator) # revealed: Literal[1]
reveal_type(False.real) # revealed: Literal[0]
```
## Bytes-literal attributes
### Bytes-literal attributes
All attribute access on literal `bytes` types is currently delegated to `buitins.bytes`:
@@ -221,3 +492,12 @@ All attribute access on literal `bytes` types is currently delegated to `buitins
reveal_type(b"foo".join) # revealed: @Todo(instance attributes)
reveal_type(b"foo".endswith) # revealed: @Todo(instance attributes)
```
## References
Some of the tests in the *Class and instance variables* section draw inspiration from
[pyright's documentation] on this topic.
[pyright's documentation]: https://microsoft.github.io/pyright/#/type-concepts-advanced?id=class-and-instance-variables
[typing spec on `classvar`]: https://typing.readthedocs.io/en/latest/spec/class-compat.html#classvar
[`typing.classvar`]: https://docs.python.org/3/library/typing.html#typing.ClassVar

View File

@@ -0,0 +1,209 @@
# Boundness and declaredness: public uses
This document demonstrates how type-inference and diagnostics works for *public* uses of a symbol,
that is, a use of a symbol from another scope. If a symbol has a declared type in its local scope
(e.g. `int`), we use that as the symbol's "public type" (the type of the symbol from the perspective
of other scopes) even if there is a more precise local inferred type for the symbol (`Literal[1]`).
We test the whole matrix of possible boundness and declaredness states. The current behavior is
summarized in the following table, while the tests below demonstrate each case. Note that some of
this behavior is questionable and might change in the future. See the TODOs in `symbol_by_id`
(`types.rs`) and [this issue](https://github.com/astral-sh/ruff/issues/14297) for more information.
In particular, we should raise errors in the "possibly-undeclared-and-unbound" as well as the
"undeclared-and-possibly-unbound" cases (marked with a "?").
| **Public type** | declared | possibly-undeclared | undeclared |
| ---------------- | ------------ | -------------------------- | ------------ |
| bound | `T_declared` | `T_declared \| T_inferred` | `T_inferred` |
| possibly-unbound | `T_declared` | `T_declared \| T_inferred` | `T_inferred` |
| unbound | `T_declared` | `T_declared` | `Unknown` |
| **Diagnostic** | declared | possibly-undeclared | undeclared |
| ---------------- | -------- | ------------------------- | ------------------- |
| bound | | | |
| possibly-unbound | | `possibly-unbound-import` | ? |
| unbound | | ? | `unresolved-import` |
## Declared
### Declared and bound
If a symbol has a declared type (`int`), we use that even if there is a more precise inferred type
(`Literal[1]`), or a conflicting inferred type (`Literal[2]`):
```py path=mod.py
x: int = 1
# error: [invalid-assignment]
y: str = 2
```
```py
from mod import x, y
reveal_type(x) # revealed: int
reveal_type(y) # revealed: str
```
### Declared and possibly unbound
If a symbol is declared and *possibly* unbound, we trust that other module and use the declared type
without raising an error.
```py path=mod.py
def flag() -> bool: ...
x: int
y: str
if flag:
x = 1
# error: [invalid-assignment]
y = 2
```
```py
from mod import x, y
reveal_type(x) # revealed: int
reveal_type(y) # revealed: str
```
### Declared and unbound
Similarly, if a symbol is declared but unbound, we do not raise an error. We trust that this symbol
is available somehow and simply use the declared type.
```py path=mod.py
x: int
```
```py
from mod import x
reveal_type(x) # revealed: int
```
## Possibly undeclared
### Possibly undeclared and bound
If a symbol is possibly undeclared but definitely bound, we use the union of the declared and
inferred types:
```py path=mod.py
from typing import Any
def flag() -> bool: ...
x = 1
y = 2
if flag():
x: Any
# error: [invalid-declaration]
y: str
```
```py
from mod import x, y
reveal_type(x) # revealed: Literal[1] | Any
reveal_type(y) # revealed: Literal[2] | Unknown
```
### Possibly undeclared and possibly unbound
If a symbol is possibly undeclared and possibly unbound, we also use the union of the declared and
inferred types. This case is interesting because the "possibly declared" definition might not be the
same as the "possibly bound" definition (symbol `y`). Note that we raise a `possibly-unbound-import`
error for both `x` and `y`:
```py path=mod.py
def flag() -> bool: ...
if flag():
x: Any = 1
y = 2
else:
y: str
```
```py
# error: [possibly-unbound-import]
# error: [possibly-unbound-import]
from mod import x, y
reveal_type(x) # revealed: Literal[1] | Any
reveal_type(y) # revealed: Literal[2] | str
```
### Possibly undeclared and unbound
If a symbol is possibly undeclared and definitely unbound, we currently do not raise an error. This
seems inconsistent when compared to the case just above.
```py path=mod.py
def flag() -> bool: ...
if flag():
x: int
```
```py
# TODO: this should raise an error. Once we fix this, update the section description and the table
# on top of this document.
from mod import x
reveal_type(x) # revealed: int
```
## Undeclared
### Undeclared but bound
We use the inferred type as the public type, if a symbol has no declared type.
```py path=mod.py
x = 1
```
```py
from mod import x
reveal_type(x) # revealed: Literal[1]
```
### Undeclared and possibly unbound
If a symbol is undeclared and *possibly* unbound, we currently do not raise an error. This seems
inconsistent when compared to the "possibly-undeclared-and-possibly-unbound" case.
```py path=mod.py
def flag() -> bool: ...
if flag:
x = 1
```
```py
# TODO: this should raise an error. Once we fix this, update the section description and the table
# on top of this document.
from mod import x
reveal_type(x) # revealed: Literal[1]
```
### Undeclared and unbound
If a symbol is undeclared *and* unbound, we infer `Unknown` and raise an error.
```py path=mod.py
if False:
x: int = 1
```
```py
# error: [unresolved-import]
from mod import x
reveal_type(x) # revealed: Unknown
```

View File

@@ -169,6 +169,15 @@ def f(*args: int) -> int:
reveal_type(f(1, 2, 3)) # revealed: int
```
### Multiple keyword arguments map to keyword variadic parameter
```py
def f(**kwargs: int) -> int:
return 1
reveal_type(f(foo=1, bar=2)) # revealed: int
```
## Missing arguments
### No defaults or variadic

View File

@@ -92,8 +92,7 @@ def _(o: object):
n = None
if o is not None:
reveal_type(o) # revealed: object & ~None
reveal_type(o) # revealed: ~None
reveal_type(o is n) # revealed: Literal[False]
reveal_type(o is not n) # revealed: Literal[True]
```

View File

@@ -66,14 +66,11 @@ def _(a: Unknown, b: Any):
assert_type(b, Unknown) # fine
def _(a: type[Unknown], b: type[Any]):
# TODO: Should be `type[Unknown]`
reveal_type(a) # revealed: @Todo(unsupported type[X] special form)
# TODO: Should be fine
assert_type(a, type[Any]) # error: [type-assertion-failure]
reveal_type(a) # revealed: type[Unknown]
assert_type(a, type[Any]) # fine
reveal_type(b) # revealed: type[Any]
# TODO: Should be fine
assert_type(b, type[Unknown]) # error: [type-assertion-failure]
assert_type(b, type[Unknown]) # fine
```
## Tuples

View File

@@ -0,0 +1,27 @@
# `cast`
`cast()` takes two arguments, one type and one value, and returns a value of the given type.
The (inferred) type of the value and the given type do not need to have any correlation.
```py
from typing import Literal, cast
reveal_type(True) # revealed: Literal[True]
reveal_type(cast(str, True)) # revealed: str
reveal_type(cast("str", True)) # revealed: str
reveal_type(cast(int | str, 1)) # revealed: int | str
# error: [invalid-type-form]
reveal_type(cast(Literal, True)) # revealed: Unknown
# TODO: These should be errors
cast(1)
cast(str)
cast(str, b"ar", "foo")
# TODO: Either support keyword arguments properly,
# or give a comprehensible error message saying they're unsupported
cast(val="foo", typ=int) # error: [unresolved-reference] "Name `foo` used when not defined"
```

View File

@@ -105,10 +105,10 @@ static_assert(not is_subtype_of(B2, B1))
This section covers structural properties of intersection types and documents some decisions on how
to represent mixtures of intersections and unions.
### Single-element unions
### Single-element intersections
If we have a union of a single element, we can simplify to that element. Similarly, we show an
intersection with a single negative contribution as just the negation of that element.
If we have an intersection with a single element, we can simplify to that element. Similarly, we
show an intersection with a single negative contribution as just the negation of that element.
```py
from knot_extensions import Intersection, Not
@@ -283,6 +283,21 @@ def _(
reveal_type(not_object) # revealed: Never
```
### `object & ~T` is equivalent to `~T`
A second consequence of the fact that `object` is the top type is that `object` is always redundant
in intersections, and can be eagerly simplified out. `object & P` is equivalent to `P`;
`object & ~P` is equivalent to `~P` for any type `P`.
```py
from knot_extensions import Intersection, Not, is_equivalent_to, static_assert
class P: ...
static_assert(is_equivalent_to(Intersection[object, P], P))
static_assert(is_equivalent_to(Intersection[object, Not[P]], Not[P]))
```
### Intersection of a type and its negation
Continuing with more [complement laws], if we see both `P` and `~P` in an intersection, we can
@@ -313,22 +328,24 @@ def _(
### Union of a type and its negation
Similarly, if we have both `P` and `~P` in a _union_, we could simplify that to `object`. However,
this is a rather costly operation which would require us to build the negation of each type that we
add to a union, so this is not implemented at the moment.
Similarly, if we have both `P` and `~P` in a _union_, we can simplify that to `object`.
```py
from knot_extensions import Intersection, Not
class P: ...
class Q: ...
def _(
i1: P | Not[P],
i2: Not[P] | P,
i3: P | Q | Not[P],
i4: Not[P] | Q | P,
) -> None:
# These could be simplified to `object`
reveal_type(i1) # revealed: P | ~P
reveal_type(i2) # revealed: ~P | P
reveal_type(i1) # revealed: object
reveal_type(i2) # revealed: object
reveal_type(i3) # revealed: object
reveal_type(i4) # revealed: object
```
### Negation is an involution
@@ -422,8 +439,8 @@ def example_type_bool_type_str(
#### Positive and negative contributions
If we intersect a type `X` with the negation of a disjoint type `Y`, we can remove the negative
contribution `~Y`, as it necessarily overlaps with the positive contribution `X`:
If we intersect a type `X` with the negation `~Y` of a disjoint type `Y`, we can remove the negative
contribution `~Y`, as `~Y` must fully contain the positive contribution `X` as a subtype:
```py
from knot_extensions import Intersection, Not
@@ -515,8 +532,7 @@ def _(
#### Negative type and negative subtype
For negative contributions, this property is reversed. Here we can get remove superfluous
_subtypes_:
For negative contributions, this property is reversed. Here we can remove superfluous _subtypes_:
```py
from knot_extensions import Intersection, Not
@@ -634,6 +650,91 @@ def _(
reveal_type(i8) # revealed: Never
```
### Simplifications of `bool`, `AlwaysTruthy` and `AlwaysFalsy`
In general, intersections with `AlwaysTruthy` and `AlwaysFalsy` cannot be simplified. Naively, you
might think that `int & AlwaysFalsy` could simplify to `Literal[0]`, but this is not the case: for
example, the `False` constant inhabits the type `int & AlwaysFalsy` (due to the fact that
`False.__class__` is `bool` at runtime, and `bool` subclasses `int`), but `False` does not inhabit
the type `Literal[0]`.
Nonetheless, intersections of `AlwaysFalsy` or `AlwaysTruthy` with `bool` _can_ be simplified, due
to the fact that `bool` is a `@final` class at runtime that cannot be subclassed.
```py
from knot_extensions import Intersection, Not, AlwaysTruthy, AlwaysFalsy
class P: ...
def f(
a: Intersection[bool, AlwaysTruthy],
b: Intersection[bool, AlwaysFalsy],
c: Intersection[bool, Not[AlwaysTruthy]],
d: Intersection[bool, Not[AlwaysFalsy]],
e: Intersection[bool, AlwaysTruthy, P],
f: Intersection[bool, AlwaysFalsy, P],
g: Intersection[bool, Not[AlwaysTruthy], P],
h: Intersection[bool, Not[AlwaysFalsy], P],
):
reveal_type(a) # revealed: Literal[True]
reveal_type(b) # revealed: Literal[False]
reveal_type(c) # revealed: Literal[False]
reveal_type(d) # revealed: Literal[True]
# `bool & AlwaysTruthy & P` -> `Literal[True] & P` -> `Never`
reveal_type(e) # revealed: Never
reveal_type(f) # revealed: Never
reveal_type(g) # revealed: Never
reveal_type(h) # revealed: Never
```
## Simplification of `LiteralString`, `AlwaysTruthy` and `AlwaysFalsy`
Similarly, intersections between `LiteralString`, `AlwaysTruthy` and `AlwaysFalsy` can be
simplified, due to the fact that a `LiteralString` inhabitant is known to have `__class__` set to
exactly `str` (and not a subclass of `str`):
```py
from knot_extensions import Intersection, Not, AlwaysTruthy, AlwaysFalsy, Unknown
from typing_extensions import LiteralString
def f(
a: Intersection[LiteralString, AlwaysTruthy],
b: Intersection[LiteralString, AlwaysFalsy],
c: Intersection[LiteralString, Not[AlwaysTruthy]],
d: Intersection[LiteralString, Not[AlwaysFalsy]],
e: Intersection[AlwaysFalsy, LiteralString],
f: Intersection[Not[AlwaysTruthy], LiteralString],
g: Intersection[AlwaysTruthy, LiteralString],
h: Intersection[Not[AlwaysFalsy], LiteralString],
i: Intersection[Unknown, LiteralString, AlwaysFalsy],
j: Intersection[Not[AlwaysTruthy], Unknown, LiteralString],
):
reveal_type(a) # revealed: LiteralString & ~Literal[""]
reveal_type(b) # revealed: Literal[""]
reveal_type(c) # revealed: Literal[""]
reveal_type(d) # revealed: LiteralString & ~Literal[""]
reveal_type(e) # revealed: Literal[""]
reveal_type(f) # revealed: Literal[""]
reveal_type(g) # revealed: LiteralString & ~Literal[""]
reveal_type(h) # revealed: LiteralString & ~Literal[""]
reveal_type(i) # revealed: Unknown & Literal[""]
reveal_type(j) # revealed: Unknown & Literal[""]
```
## Addition of a type to an intersection with many non-disjoint types
This slightly strange-looking test is a regression test for a mistake that was nearly made in a PR:
<https://github.com/astral-sh/ruff/pull/15475#discussion_r1915041987>.
```py
from knot_extensions import AlwaysFalsy, Intersection, Unknown
from typing_extensions import Literal
def _(x: Intersection[str, Unknown, AlwaysFalsy, Literal[""]]):
reveal_type(x) # revealed: Unknown & Literal[""]
```
## Non fully-static types
### Negation of dynamic types

View File

@@ -91,8 +91,7 @@ if isinstance(x, (A, B)):
elif isinstance(x, (A, C)):
reveal_type(x) # revealed: C & ~A & ~B
else:
# TODO: Should be simplified to ~A & ~B & ~C
reveal_type(x) # revealed: object & ~A & ~B & ~C
reveal_type(x) # revealed: ~A & ~B & ~C
```
## No narrowing for instances of `builtins.type`
@@ -181,3 +180,43 @@ def _(x: object, y: type[int]):
if isinstance(x, y):
reveal_type(x) # revealed: int
```
## Adding a disjoint element to an existing intersection
We used to incorrectly infer `Literal` booleans for some of these.
```py
from knot_extensions import Not, Intersection, AlwaysTruthy, AlwaysFalsy
class P: ...
def f(
a: Intersection[P, AlwaysTruthy],
b: Intersection[P, AlwaysFalsy],
c: Intersection[P, Not[AlwaysTruthy]],
d: Intersection[P, Not[AlwaysFalsy]],
):
if isinstance(a, bool):
reveal_type(a) # revealed: Never
else:
# TODO: `bool` is final, so `& ~bool` is redundant here
reveal_type(a) # revealed: P & AlwaysTruthy & ~bool
if isinstance(b, bool):
reveal_type(b) # revealed: Never
else:
# TODO: `bool` is final, so `& ~bool` is redundant here
reveal_type(b) # revealed: P & AlwaysFalsy & ~bool
if isinstance(c, bool):
reveal_type(c) # revealed: Never
else:
# TODO: `bool` is final, so `& ~bool` is redundant here
reveal_type(c) # revealed: P & ~AlwaysTruthy & ~bool
if isinstance(d, bool):
reveal_type(d) # revealed: Never
else:
# TODO: `bool` is final, so `& ~bool` is redundant here
reveal_type(d) # revealed: P & ~AlwaysFalsy & ~bool
```

View File

@@ -21,22 +21,22 @@ else:
if x and not x:
reveal_type(x) # revealed: Never
else:
reveal_type(x) # revealed: Literal[0, "", b"", -1, "foo", b"bar"] | bool | None | tuple[()]
reveal_type(x) # revealed: Literal[0, -1, "", "foo", b"", b"bar"] | bool | None | tuple[()]
if not (x and not x):
reveal_type(x) # revealed: Literal[0, "", b"", -1, "foo", b"bar"] | bool | None | tuple[()]
reveal_type(x) # revealed: Literal[0, -1, "", "foo", b"", b"bar"] | bool | None | tuple[()]
else:
reveal_type(x) # revealed: Never
if x or not x:
reveal_type(x) # revealed: Literal[-1, "foo", b"bar", 0, "", b""] | bool | None | tuple[()]
reveal_type(x) # revealed: Literal[0, -1, "", "foo", b"", b"bar"] | bool | None | tuple[()]
else:
reveal_type(x) # revealed: Never
if not (x or not x):
reveal_type(x) # revealed: Never
else:
reveal_type(x) # revealed: Literal[-1, "foo", b"bar", 0, "", b""] | bool | None | tuple[()]
reveal_type(x) # revealed: Literal[0, -1, "", "foo", b"", b"bar"] | bool | None | tuple[()]
if (isinstance(x, int) or isinstance(x, str)) and x:
reveal_type(x) # revealed: Literal[-1, True, "foo"]
@@ -87,10 +87,10 @@ def f(x: A | B):
if x and not x:
reveal_type(x) # revealed: A & ~AlwaysFalsy & ~AlwaysTruthy | B & ~AlwaysFalsy & ~AlwaysTruthy
else:
reveal_type(x) # revealed: A & ~AlwaysTruthy | B & ~AlwaysTruthy | A & ~AlwaysFalsy | B & ~AlwaysFalsy
reveal_type(x) # revealed: A | B
if x or not x:
reveal_type(x) # revealed: A & ~AlwaysFalsy | B & ~AlwaysFalsy | A & ~AlwaysTruthy | B & ~AlwaysTruthy
reveal_type(x) # revealed: A | B
else:
reveal_type(x) # revealed: A & ~AlwaysTruthy & ~AlwaysFalsy | B & ~AlwaysTruthy & ~AlwaysFalsy
```
@@ -199,7 +199,7 @@ def f(x: Literal[0, 1], y: Literal["", "hello"]):
reveal_type(y) # revealed: Literal["", "hello"]
```
## ControlFlow Merging
## Control Flow Merging
After merging control flows, when we take the union of all constraints applied in each branch, we
should return to the original state.
@@ -214,10 +214,9 @@ if x and not x:
reveal_type(y) # revealed: A & ~AlwaysFalsy & ~AlwaysTruthy
else:
y = x
reveal_type(y) # revealed: A & ~AlwaysTruthy | A & ~AlwaysFalsy
reveal_type(y) # revealed: A
# TODO: It should be A. We should improve UnionBuilder or IntersectionBuilder. (issue #15023)
reveal_type(y) # revealed: A & ~AlwaysTruthy | A & ~AlwaysFalsy
reveal_type(y) # revealed: A
```
## Truthiness of classes
@@ -313,3 +312,20 @@ def _(x: type[FalsyClass] | type[TruthyClass]):
reveal_type(x or A()) # revealed: type[TruthyClass] | A
reveal_type(x and A()) # revealed: type[FalsyClass] | A
```
## Truthiness narrowing for `LiteralString`
```py
from typing_extensions import LiteralString
def _(x: LiteralString):
if x:
reveal_type(x) # revealed: LiteralString & ~Literal[""]
else:
reveal_type(x) # revealed: Literal[""]
if not x:
reveal_type(x) # revealed: Literal[""]
else:
reveal_type(x) # revealed: LiteralString & ~Literal[""]
```

View File

@@ -13,7 +13,7 @@ typeshed:
```py
import sys
reveal_type(sys.platform) # revealed: str
reveal_type(sys.platform) # revealed: LiteralString
```
## Explicit selection of `all` platforms
@@ -26,7 +26,7 @@ python-platform = "all"
```py
import sys
reveal_type(sys.platform) # revealed: str
reveal_type(sys.platform) # revealed: LiteralString
```
## Explicit selection of a specific platform
@@ -66,6 +66,6 @@ It is [recommended](https://docs.python.org/3/library/sys.html#sys.platform) to
```py
import sys
reveal_type(sys.platform.startswith("freebsd")) # revealed: @Todo(instance attributes)
reveal_type(sys.platform.startswith("linux")) # revealed: @Todo(instance attributes)
reveal_type(sys.platform.startswith("freebsd")) # revealed: @Todo(Attribute access on `LiteralString` types)
reveal_type(sys.platform.startswith("linux")) # revealed: @Todo(Attribute access on `LiteralString` types)
```

View File

@@ -94,6 +94,36 @@ reveal_type(C.__mro__)
u: Unknown[str]
```
### `AlwaysTruthy` and `AlwaysFalsy`
`AlwaysTruthy` and `AlwaysFalsy` represent the sets of all possible objects whose truthiness is
always truthy or falsy, respectively.
They do not accept any type arguments.
```py
from typing_extensions import Literal
from knot_extensions import AlwaysFalsy, AlwaysTruthy, is_subtype_of, static_assert
static_assert(is_subtype_of(Literal[True], AlwaysTruthy))
static_assert(is_subtype_of(Literal[False], AlwaysFalsy))
static_assert(not is_subtype_of(int, AlwaysFalsy))
static_assert(not is_subtype_of(str, AlwaysFalsy))
def _(t: AlwaysTruthy, f: AlwaysFalsy):
reveal_type(t) # revealed: AlwaysTruthy
reveal_type(f) # revealed: AlwaysFalsy
def f(
a: AlwaysTruthy[int], # error: [invalid-type-form]
b: AlwaysFalsy[str], # error: [invalid-type-form]
):
reveal_type(a) # revealed: Unknown
reveal_type(b) # revealed: Unknown
```
## Static assertions
### Basics

View File

@@ -0,0 +1,35 @@
# Equivalence relation
`is_equivalent_to` implements [the equivalence relation] for fully static types.
Two types `A` and `B` are equivalent iff `A` is a subtype of `B` and `B` is a subtype of `A`.
## Basic
```py
from typing import Any
from typing_extensions import Literal
from knot_extensions import Unknown, is_equivalent_to, static_assert
static_assert(is_equivalent_to(Literal[1, 2], Literal[1, 2]))
static_assert(is_equivalent_to(type[object], type))
static_assert(not is_equivalent_to(Any, Any))
static_assert(not is_equivalent_to(Unknown, Unknown))
static_assert(not is_equivalent_to(Any, None))
static_assert(not is_equivalent_to(Literal[1, 2], Literal[1, 0]))
static_assert(not is_equivalent_to(Literal[1, 2], Literal[1, 2, 3]))
```
## Equivalence is commutative
```py
from typing_extensions import Literal
from knot_extensions import is_equivalent_to, static_assert
static_assert(is_equivalent_to(type, type[object]))
static_assert(not is_equivalent_to(Literal[1, 0], Literal[1, 2]))
static_assert(not is_equivalent_to(Literal[1, 2, 3], Literal[1, 2]))
```
[the equivalence relation]: https://typing.readthedocs.io/en/latest/spec/glossary.html#term-equivalent

View File

@@ -0,0 +1,453 @@
# Subtype relation
The `is_subtype_of(S, T)` relation below checks if type `S` is a subtype of type `T`.
A fully static type `S` is a subtype of another fully static type `T` iff the set of values
represented by `S` is a subset of the set of values represented by `T`.
See the [typing documentation] for more information.
## Basic builtin types
- `bool` is a subtype of `int`. This is modeled after Python's runtime behavior, where `int` is a
supertype of `bool` (present in `bool`s bases and MRO).
- `int` is not a subtype of `float`/`complex`, even though `float`/`complex` can be used in place of
`int` in some contexts (see [special case for float and complex]).
```py
from knot_extensions import is_subtype_of, static_assert
static_assert(is_subtype_of(bool, bool))
static_assert(is_subtype_of(bool, int))
static_assert(is_subtype_of(bool, object))
static_assert(is_subtype_of(int, int))
static_assert(is_subtype_of(int, object))
static_assert(is_subtype_of(object, object))
static_assert(not is_subtype_of(int, bool))
static_assert(not is_subtype_of(int, str))
static_assert(not is_subtype_of(object, int))
static_assert(not is_subtype_of(int, float))
static_assert(not is_subtype_of(int, complex))
static_assert(is_subtype_of(TypeError, Exception))
static_assert(is_subtype_of(FloatingPointError, Exception))
```
## Class hierarchies
```py
from knot_extensions import is_subtype_of, static_assert
from typing_extensions import Never
class A: ...
class B1(A): ...
class B2(A): ...
class C(B1, B2): ...
static_assert(is_subtype_of(B1, A))
static_assert(not is_subtype_of(A, B1))
static_assert(is_subtype_of(B2, A))
static_assert(not is_subtype_of(A, B2))
static_assert(not is_subtype_of(B1, B2))
static_assert(not is_subtype_of(B2, B1))
static_assert(is_subtype_of(C, B1))
static_assert(is_subtype_of(C, B2))
static_assert(not is_subtype_of(B1, C))
static_assert(not is_subtype_of(B2, C))
static_assert(is_subtype_of(C, A))
static_assert(not is_subtype_of(A, C))
static_assert(is_subtype_of(Never, A))
static_assert(is_subtype_of(Never, B1))
static_assert(is_subtype_of(Never, B2))
static_assert(is_subtype_of(Never, C))
static_assert(is_subtype_of(A, object))
static_assert(is_subtype_of(B1, object))
static_assert(is_subtype_of(B2, object))
static_assert(is_subtype_of(C, object))
```
## Literal types
```py
from typing_extensions import Literal, LiteralString
from knot_extensions import is_subtype_of, static_assert
# Boolean literals
static_assert(is_subtype_of(Literal[True], bool))
static_assert(is_subtype_of(Literal[True], int))
static_assert(is_subtype_of(Literal[True], object))
# Integer literals
static_assert(is_subtype_of(Literal[1], int))
static_assert(is_subtype_of(Literal[1], object))
static_assert(not is_subtype_of(Literal[1], bool))
# See the note above (or link below) concerning int and float/complex
static_assert(not is_subtype_of(Literal[1], float))
# String literals
static_assert(is_subtype_of(Literal["foo"], LiteralString))
static_assert(is_subtype_of(Literal["foo"], str))
static_assert(is_subtype_of(Literal["foo"], object))
static_assert(is_subtype_of(LiteralString, str))
static_assert(is_subtype_of(LiteralString, object))
# Bytes literals
static_assert(is_subtype_of(Literal[b"foo"], bytes))
static_assert(is_subtype_of(Literal[b"foo"], object))
```
## Tuple types
```py
from knot_extensions import is_subtype_of, static_assert
class A1: ...
class B1(A1): ...
class A2: ...
class B2(A2): ...
class Unrelated: ...
static_assert(is_subtype_of(B1, A1))
static_assert(is_subtype_of(B2, A2))
# Zero-element tuples
static_assert(is_subtype_of(tuple[()], tuple[()]))
static_assert(not is_subtype_of(tuple[()], tuple[Unrelated]))
# One-element tuples
static_assert(is_subtype_of(tuple[B1], tuple[A1]))
static_assert(not is_subtype_of(tuple[B1], tuple[Unrelated]))
static_assert(not is_subtype_of(tuple[B1], tuple[()]))
static_assert(not is_subtype_of(tuple[B1], tuple[A1, Unrelated]))
# Two-element tuples
static_assert(is_subtype_of(tuple[B1, B2], tuple[A1, A2]))
static_assert(not is_subtype_of(tuple[B1, B2], tuple[Unrelated, A2]))
static_assert(not is_subtype_of(tuple[B1, B2], tuple[A1, Unrelated]))
static_assert(not is_subtype_of(tuple[B1, B2], tuple[Unrelated, Unrelated]))
static_assert(not is_subtype_of(tuple[B1, B2], tuple[()]))
static_assert(not is_subtype_of(tuple[B1, B2], tuple[A1]))
static_assert(not is_subtype_of(tuple[B1, B2], tuple[A1, A2, Unrelated]))
static_assert(is_subtype_of(tuple[int], tuple))
```
## Union types
```py
from knot_extensions import is_subtype_of, static_assert
class A: ...
class B1(A): ...
class B2(A): ...
class Unrelated1: ...
class Unrelated2: ...
static_assert(is_subtype_of(B1, A))
static_assert(is_subtype_of(B2, A))
# Union on the right hand side
static_assert(is_subtype_of(B1, A | Unrelated1))
static_assert(is_subtype_of(B1, Unrelated1 | A))
static_assert(not is_subtype_of(B1, Unrelated1 | Unrelated2))
# Union on the left hand side
static_assert(is_subtype_of(B1 | B2, A))
static_assert(is_subtype_of(B1 | B2 | A, object))
static_assert(not is_subtype_of(B1 | Unrelated1, A))
static_assert(not is_subtype_of(Unrelated1 | B1, A))
# Union on both sides
static_assert(is_subtype_of(B1 | bool, A | int))
static_assert(is_subtype_of(B1 | bool, int | A))
static_assert(not is_subtype_of(B1 | bool, Unrelated1 | int))
static_assert(not is_subtype_of(B1 | bool, int | Unrelated1))
# Example: Unions of literals
static_assert(is_subtype_of(Literal[1, 2, 3], int))
static_assert(not is_subtype_of(Literal[1, "two", 3], int))
```
## Intersection types
```py
from typing_extensions import Literal, LiteralString
from knot_extensions import Intersection, Not, is_subtype_of, static_assert
class A: ...
class B1(A): ...
class B2(A): ...
class C(B1, B2): ...
class Unrelated: ...
static_assert(is_subtype_of(B1, A))
static_assert(is_subtype_of(B2, A))
static_assert(is_subtype_of(C, A))
static_assert(is_subtype_of(C, B1))
static_assert(is_subtype_of(C, B2))
# For complements, the subtyping relation is reversed:
static_assert(is_subtype_of(Not[A], Not[B1]))
static_assert(is_subtype_of(Not[A], Not[B2]))
static_assert(is_subtype_of(Not[A], Not[C]))
static_assert(is_subtype_of(Not[B1], Not[C]))
static_assert(is_subtype_of(Not[B2], Not[C]))
# The intersection of two types is a subtype of both:
static_assert(is_subtype_of(Intersection[B1, B2], B1))
static_assert(is_subtype_of(Intersection[B1, B2], B2))
# … and of their common supertype:
static_assert(is_subtype_of(Intersection[B1, B2], A))
# A common subtype of two types is a subtype of their intersection:
static_assert(is_subtype_of(C, Intersection[B1, B2]))
# … but not the other way around:
static_assert(not is_subtype_of(Intersection[B1, B2], C))
# "Removing" B1 from A leaves a subtype of A.
static_assert(is_subtype_of(Intersection[A, Not[B1]], A))
static_assert(is_subtype_of(Intersection[A, Not[B1]], Not[B1]))
# B1 and B2 are not disjoint, so this is not true:
static_assert(not is_subtype_of(B2, Intersection[A, Not[B1]]))
# … but for two disjoint subtypes, it is:
static_assert(is_subtype_of(Literal[2], Intersection[int, Not[Literal[1]]]))
# A and Unrelated are not related, so this is not true:
static_assert(not is_subtype_of(Intersection[A, Not[B1]], Not[Unrelated]))
# … but for a disjoint type like `None`, it is:
static_assert(is_subtype_of(Intersection[A, Not[B1]], Not[None]))
# Complements of types are still subtypes of `object`:
static_assert(is_subtype_of(Not[A], object))
# More examples:
static_assert(is_subtype_of(type[str], Not[None]))
static_assert(is_subtype_of(Not[LiteralString], object))
static_assert(not is_subtype_of(Intersection[int, Not[Literal[2]]], Intersection[int, Not[Literal[3]]]))
static_assert(not is_subtype_of(Not[Literal[2]], Not[Literal[3]]))
static_assert(not is_subtype_of(Not[Literal[2]], Not[int]))
static_assert(not is_subtype_of(int, Not[Literal[3]]))
static_assert(not is_subtype_of(Literal[1], Intersection[int, Not[Literal[1]]]))
```
## Special types
### `Never`
`Never` is a subtype of all types.
```py
from typing_extensions import Literal, Never
from knot_extensions import AlwaysTruthy, AlwaysFalsy, is_subtype_of, static_assert
static_assert(is_subtype_of(Never, Never))
static_assert(is_subtype_of(Never, Literal[True]))
static_assert(is_subtype_of(Never, bool))
static_assert(is_subtype_of(Never, int))
static_assert(is_subtype_of(Never, object))
static_assert(is_subtype_of(Never, AlwaysTruthy))
static_assert(is_subtype_of(Never, AlwaysFalsy))
```
### `AlwaysTruthy` and `AlwaysFalsy`
```py
from knot_extensions import AlwaysTruthy, AlwaysFalsy, is_subtype_of, static_assert
static_assert(is_subtype_of(Literal[1], AlwaysTruthy))
static_assert(is_subtype_of(Literal[0], AlwaysFalsy))
static_assert(is_subtype_of(AlwaysTruthy, object))
static_assert(is_subtype_of(AlwaysFalsy, object))
static_assert(not is_subtype_of(Literal[1], AlwaysFalsy))
static_assert(not is_subtype_of(Literal[0], AlwaysTruthy))
static_assert(not is_subtype_of(str, AlwaysTruthy))
static_assert(not is_subtype_of(str, AlwaysFalsy))
```
### Module literals
```py
from types import ModuleType
from knot_extensions import TypeOf, is_subtype_of, static_assert
from typing_extensions import assert_type
import typing
assert_type(typing, TypeOf[typing])
static_assert(is_subtype_of(TypeOf[typing], ModuleType))
```
### Slice literals
```py
from knot_extensions import TypeOf, is_subtype_of, static_assert
static_assert(is_subtype_of(TypeOf[1:2:3], slice))
```
### Special forms
```py
from typing import _SpecialForm
from knot_extensions import TypeOf, is_subtype_of, static_assert
static_assert(is_subtype_of(TypeOf[Literal], _SpecialForm))
static_assert(is_subtype_of(TypeOf[Literal], object))
static_assert(not is_subtype_of(_SpecialForm, TypeOf[Literal]))
```
## Class literal types and `type[…]`
### Basic
```py
from typing import _SpecialForm
from typing_extensions import Literal, assert_type
from knot_extensions import TypeOf, is_subtype_of, static_assert
class Meta(type): ...
class HasCustomMetaclass(metaclass=Meta): ...
type LiteralBool = TypeOf[bool]
type LiteralInt = TypeOf[int]
type LiteralStr = TypeOf[str]
type LiteralObject = TypeOf[object]
assert_type(bool, LiteralBool)
assert_type(int, LiteralInt)
assert_type(str, LiteralStr)
assert_type(object, LiteralObject)
# bool
static_assert(is_subtype_of(LiteralBool, LiteralBool))
static_assert(is_subtype_of(LiteralBool, type[bool]))
static_assert(is_subtype_of(LiteralBool, type[int]))
static_assert(is_subtype_of(LiteralBool, type[object]))
static_assert(is_subtype_of(LiteralBool, type))
static_assert(is_subtype_of(LiteralBool, object))
static_assert(not is_subtype_of(LiteralBool, LiteralInt))
static_assert(not is_subtype_of(LiteralBool, LiteralObject))
static_assert(not is_subtype_of(LiteralBool, bool))
static_assert(not is_subtype_of(type, type[bool]))
# int
static_assert(is_subtype_of(LiteralInt, LiteralInt))
static_assert(is_subtype_of(LiteralInt, type[int]))
static_assert(is_subtype_of(LiteralInt, type[object]))
static_assert(is_subtype_of(LiteralInt, type))
static_assert(is_subtype_of(LiteralInt, object))
static_assert(not is_subtype_of(LiteralInt, LiteralObject))
static_assert(not is_subtype_of(LiteralInt, int))
static_assert(not is_subtype_of(type, type[int]))
# LiteralString
static_assert(is_subtype_of(LiteralStr, type[str]))
static_assert(is_subtype_of(LiteralStr, type))
static_assert(is_subtype_of(LiteralStr, type[object]))
static_assert(not is_subtype_of(type[str], LiteralStr))
# custom meta classes
type LiteralHasCustomMetaclass = TypeOf[HasCustomMetaclass]
static_assert(is_subtype_of(LiteralHasCustomMetaclass, Meta))
static_assert(is_subtype_of(Meta, type[object]))
static_assert(is_subtype_of(Meta, type))
static_assert(not is_subtype_of(Meta, type[type]))
```
### Unions of class literals
```py
from typing_extensions import assert_type
from knot_extensions import TypeOf, is_subtype_of, static_assert
class Base: ...
class Derived(Base): ...
class Unrelated: ...
type LiteralBase = TypeOf[Base]
type LiteralDerived = TypeOf[Derived]
type LiteralUnrelated = TypeOf[Unrelated]
assert_type(Base, LiteralBase)
assert_type(Derived, LiteralDerived)
assert_type(Unrelated, LiteralUnrelated)
static_assert(is_subtype_of(LiteralBase, type))
static_assert(is_subtype_of(LiteralBase, object))
static_assert(is_subtype_of(LiteralBase, type[Base]))
static_assert(is_subtype_of(LiteralDerived, type[Base]))
static_assert(is_subtype_of(LiteralDerived, type[Derived]))
static_assert(not is_subtype_of(LiteralBase, type[Derived]))
static_assert(is_subtype_of(type[Derived], type[Base]))
static_assert(is_subtype_of(LiteralBase | LiteralUnrelated, type))
static_assert(is_subtype_of(LiteralBase | LiteralUnrelated, object))
```
## Non-fully-static types
`Any`, `Unknown`, `Todo` and derivatives thereof do not participate in subtyping.
```py
from knot_extensions import Unknown, is_subtype_of, static_assert, Intersection
from typing_extensions import Any
static_assert(not is_subtype_of(Any, Any))
static_assert(not is_subtype_of(Any, int))
static_assert(not is_subtype_of(int, Any))
static_assert(not is_subtype_of(Any, object))
static_assert(not is_subtype_of(object, Any))
static_assert(not is_subtype_of(int, Any | int))
static_assert(not is_subtype_of(Intersection[Any, int], int))
static_assert(not is_subtype_of(tuple[int, int], tuple[int, Any]))
# The same for `Unknown`:
static_assert(not is_subtype_of(Unknown, Unknown))
static_assert(not is_subtype_of(Unknown, int))
static_assert(not is_subtype_of(int, Unknown))
static_assert(not is_subtype_of(Unknown, object))
static_assert(not is_subtype_of(object, Unknown))
static_assert(not is_subtype_of(int, Unknown | int))
static_assert(not is_subtype_of(Intersection[Unknown, int], int))
static_assert(not is_subtype_of(tuple[int, int], tuple[int, Unknown]))
```
[special case for float and complex]: https://typing.readthedocs.io/en/latest/spec/special-types.html#special-cases-for-float-and-complex
[typing documentation]: https://typing.readthedocs.io/en/latest/spec/concepts.html#subtype-supertype-and-type-equivalence

View File

@@ -180,7 +180,7 @@ pub(crate) mod tests {
Program::from_settings(
&db,
&ProgramSettings {
ProgramSettings {
python_version: self.python_version,
python_platform: self.python_platform,
search_paths,

View File

@@ -154,6 +154,10 @@ impl KnownModule {
}
}
pub const fn is_builtins(self) -> bool {
matches!(self, Self::Builtins)
}
pub const fn is_typing(self) -> bool {
matches!(self, Self::Typing)
}

View File

@@ -1294,7 +1294,7 @@ mod tests {
Program::from_settings(
&db,
&ProgramSettings {
ProgramSettings {
python_version: PythonVersion::PY38,
python_platform: PythonPlatform::default(),
search_paths: SearchPathSettings {
@@ -1800,7 +1800,7 @@ not_a_directory
Program::from_settings(
&db,
&ProgramSettings {
ProgramSettings {
python_version: PythonVersion::default(),
python_platform: PythonPlatform::default(),
search_paths: SearchPathSettings {

View File

@@ -232,7 +232,7 @@ impl TestCaseBuilder<MockedTypeshed> {
Program::from_settings(
&db,
&ProgramSettings {
ProgramSettings {
python_version,
python_platform,
search_paths: SearchPathSettings {
@@ -290,7 +290,7 @@ impl TestCaseBuilder<VendoredTypeshed> {
Program::from_settings(
&db,
&ProgramSettings {
ProgramSettings {
python_version,
python_platform,
search_paths: SearchPathSettings {

View File

@@ -1,18 +1,18 @@
use crate::module_resolver::SearchPaths;
use crate::python_platform::PythonPlatform;
use crate::python_version::PythonVersion;
use crate::Db;
use anyhow::Context;
use ruff_db::system::{SystemPath, SystemPathBuf};
use salsa::Durability;
use salsa::Setter;
use ruff_db::system::{SystemPath, SystemPathBuf};
use crate::module_resolver::SearchPaths;
use crate::Db;
#[salsa::input(singleton)]
pub struct Program {
pub python_version: PythonVersion,
#[return_ref]
pub python_platform: PythonPlatform,
#[return_ref]
@@ -20,25 +20,51 @@ pub struct Program {
}
impl Program {
pub fn from_settings(db: &dyn Db, settings: &ProgramSettings) -> anyhow::Result<Self> {
pub fn from_settings(db: &dyn Db, settings: ProgramSettings) -> anyhow::Result<Self> {
let ProgramSettings {
python_version,
python_platform,
search_paths,
} = settings;
tracing::info!("Python version: Python {python_version}");
tracing::info!("Python version: Python {python_version}, platform: {python_platform}");
let search_paths = SearchPaths::from_settings(db, search_paths)
let search_paths = SearchPaths::from_settings(db, &search_paths)
.with_context(|| "Invalid search path settings")?;
Ok(
Program::builder(*python_version, python_platform.clone(), search_paths)
Program::builder(python_version, python_platform, search_paths)
.durability(Durability::HIGH)
.new(db),
)
}
pub fn update_from_settings(
self,
db: &mut dyn Db,
settings: ProgramSettings,
) -> anyhow::Result<()> {
let ProgramSettings {
python_version,
python_platform,
search_paths,
} = settings;
if &python_platform != self.python_platform(db) {
tracing::debug!("Updating python platform: `{python_platform:?}`");
self.set_python_platform(db).to(python_platform);
}
if python_version != self.python_version(db) {
tracing::debug!("Updating python version: Python {python_version}");
self.set_python_version(db).to(python_version);
}
self.update_search_paths(db, &search_paths)?;
Ok(())
}
pub fn update_search_paths(
self,
db: &mut dyn Db,
@@ -76,7 +102,7 @@ pub struct SearchPathSettings {
/// or pyright's stubPath configuration setting.
pub extra_paths: Vec<SystemPathBuf>,
/// The root of the workspace, used for finding first-party modules.
/// The root of the project, used for finding first-party modules.
pub src_root: SystemPathBuf,
/// Optional path to a "custom typeshed" directory on disk for us to use for standard-library types.

View File

@@ -1,3 +1,5 @@
use std::fmt::{Display, Formatter};
/// The target platform to assume when resolving types.
#[derive(Debug, Clone, Default, PartialEq, Eq)]
#[cfg_attr(
@@ -17,3 +19,12 @@ pub enum PythonPlatform {
#[cfg_attr(feature = "serde", serde(untagged))]
Identifier(String),
}
impl Display for PythonPlatform {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
match self {
PythonPlatform::All => f.write_str("all"),
PythonPlatform::Identifier(name) => f.write_str(name),
}
}
}

View File

@@ -4,11 +4,10 @@ use ruff_python_ast as ast;
use ruff_text_size::{Ranged, TextRange};
use crate::ast_node_ref::AstNodeRef;
use crate::module_resolver::file_to_module;
use crate::node_key::NodeKey;
use crate::semantic_index::symbol::{FileScopeId, ScopeId, ScopedSymbolId};
use crate::unpack::Unpack;
use crate::{Db, KnownModule};
use crate::Db;
/// A definition of a symbol.
///
@@ -61,24 +60,6 @@ impl<'db> Definition<'db> {
pub(crate) fn is_binding(self, db: &'db dyn Db) -> bool {
self.kind(db).category().is_binding()
}
pub(crate) fn is_builtin_definition(self, db: &'db dyn Db) -> bool {
file_to_module(db, self.file(db))
.is_some_and(|module| module.is_known(KnownModule::Builtins))
}
/// Return true if this symbol was defined in the `typing` or `typing_extensions` modules
pub(crate) fn is_typing_definition(self, db: &'db dyn Db) -> bool {
matches!(
file_to_module(db, self.file(db)).and_then(|module| module.known()),
Some(KnownModule::Typing | KnownModule::TypingExtensions)
)
}
pub(crate) fn is_knot_extensions_definition(self, db: &'db dyn Db) -> bool {
file_to_module(db, self.file(db))
.is_some_and(|module| module.is_known(KnownModule::KnotExtensions))
}
}
#[derive(Copy, Clone, Debug)]

View File

@@ -93,6 +93,19 @@ impl<const B: usize> BitSet<B> {
}
}
/// Union in-place with another [`BitSet`].
pub(super) fn union(&mut self, other: &BitSet<B>) {
let mut max_len = self.blocks().len();
let other_len = other.blocks().len();
if other_len > max_len {
max_len = other_len;
self.resize_blocks(max_len);
}
for (my_block, other_block) in self.blocks_mut().iter_mut().zip(other.blocks()) {
*my_block |= other_block;
}
}
/// Return an iterator over the values (in ascending order) in this [`BitSet`].
pub(super) fn iter(&self) -> BitSetIterator<'_, B> {
let blocks = self.blocks();
@@ -222,6 +235,59 @@ mod tests {
assert_bitset(&b1, &[89]);
}
#[test]
fn union() {
let mut b1 = BitSet::<1>::with(2);
let b2 = BitSet::<1>::with(4);
b1.union(&b2);
assert_bitset(&b1, &[2, 4]);
}
#[test]
fn union_mixed_1() {
let mut b1 = BitSet::<1>::with(4);
let mut b2 = BitSet::<1>::with(4);
b1.insert(89);
b2.insert(5);
b1.union(&b2);
assert_bitset(&b1, &[4, 5, 89]);
}
#[test]
fn union_mixed_2() {
let mut b1 = BitSet::<1>::with(4);
let mut b2 = BitSet::<1>::with(4);
b1.insert(23);
b2.insert(89);
b1.union(&b2);
assert_bitset(&b1, &[4, 23, 89]);
}
#[test]
fn union_heap() {
let mut b1 = BitSet::<1>::with(4);
let mut b2 = BitSet::<1>::with(4);
b1.insert(89);
b2.insert(90);
b1.union(&b2);
assert_bitset(&b1, &[4, 89, 90]);
}
#[test]
fn union_heap_2() {
let mut b1 = BitSet::<1>::with(89);
let mut b2 = BitSet::<1>::with(89);
b1.insert(91);
b2.insert(90);
b1.union(&b2);
assert_bitset(&b1, &[89, 90, 91]);
}
#[test]
fn multiple_blocks() {
let mut b = BitSet::<2>::with(120);

View File

@@ -316,6 +316,9 @@ impl SymbolState {
};
std::mem::swap(&mut a, self);
self.declarations
.live_declarations
.union(&b.declarations.live_declarations);
let mut a_defs_iter = a.bindings.live_bindings.iter();
let mut b_defs_iter = b.bindings.live_bindings.iter();
@@ -449,10 +452,8 @@ impl SymbolState {
let mut opt_a_decl: Option<u32> = a_decls_iter.next();
let mut opt_b_decl: Option<u32> = b_decls_iter.next();
let push = |decl,
vis_constraints_iter: &mut VisibilityConstraintsIntoIterator,
let push = |vis_constraints_iter: &mut VisibilityConstraintsIntoIterator,
merged: &mut Self| {
merged.declarations.live_declarations.insert(decl);
let vis_constraints = vis_constraints_iter
.next()
.expect("declarations and visibility_constraints length mismatch");
@@ -466,15 +467,15 @@ impl SymbolState {
match (opt_a_decl, opt_b_decl) {
(Some(a_decl), Some(b_decl)) => match a_decl.cmp(&b_decl) {
std::cmp::Ordering::Less => {
push(a_decl, &mut a_vis_constraints_iter, self);
push(&mut a_vis_constraints_iter, self);
opt_a_decl = a_decls_iter.next();
}
std::cmp::Ordering::Greater => {
push(b_decl, &mut b_vis_constraints_iter, self);
push(&mut b_vis_constraints_iter, self);
opt_b_decl = b_decls_iter.next();
}
std::cmp::Ordering::Equal => {
push(a_decl, &mut b_vis_constraints_iter, self);
push(&mut b_vis_constraints_iter, self);
let a_vis_constraint = a_vis_constraints_iter
.next()
@@ -487,12 +488,12 @@ impl SymbolState {
opt_b_decl = b_decls_iter.next();
}
},
(Some(a_decl), None) => {
push(a_decl, &mut a_vis_constraints_iter, self);
(Some(_), None) => {
push(&mut a_vis_constraints_iter, self);
opt_a_decl = a_decls_iter.next();
}
(None, Some(b_decl)) => {
push(b_decl, &mut b_vis_constraints_iter, self);
(None, Some(_)) => {
push(&mut b_vis_constraints_iter, self);
opt_b_decl = b_decls_iter.next();
}
(None, None) => break,

View File

@@ -671,6 +671,13 @@ impl<'db> Type<'db> {
.expect("Expected a Type::IntLiteral variant")
}
pub const fn into_instance(self) -> Option<InstanceType<'db>> {
match self {
Type::Instance(instance_type) => Some(instance_type),
_ => None,
}
}
pub const fn into_known_instance(self) -> Option<KnownInstanceType<'db>> {
match self {
Type::KnownInstance(known_instance) => Some(known_instance),
@@ -2014,6 +2021,20 @@ impl<'db> Type<'db> {
CallOutcome::asserted(binding, asserted_ty)
}
Some(KnownFunction::Cast) => {
// TODO: Use `.two_parameter_tys()` exclusively
// when overloads are supported.
if binding.two_parameter_tys().is_none() {
return CallOutcome::callable(binding);
};
if let Some(casted_ty) = arguments.first_argument() {
binding.set_return_ty(casted_ty);
};
CallOutcome::callable(binding)
}
_ => CallOutcome::callable(binding),
}
}
@@ -2278,6 +2299,8 @@ impl<'db> Type<'db> {
fallback_type: Type::unknown(),
}),
Type::KnownInstance(KnownInstanceType::Unknown) => Ok(Type::unknown()),
Type::KnownInstance(KnownInstanceType::AlwaysTruthy) => Ok(Type::AlwaysTruthy),
Type::KnownInstance(KnownInstanceType::AlwaysFalsy) => Ok(Type::AlwaysFalsy),
_ => Ok(todo_type!(
"Unsupported or invalid type in a type expression"
)),
@@ -2541,6 +2564,10 @@ pub enum KnownClass {
}
impl<'db> KnownClass {
pub const fn is_bool(self) -> bool {
matches!(self, Self::Bool)
}
pub const fn as_str(&self) -> &'static str {
match self {
Self::Bool => "bool",
@@ -2813,6 +2840,10 @@ pub enum KnownInstanceType<'db> {
TypeAliasType(TypeAliasType<'db>),
/// The symbol `knot_extensions.Unknown`
Unknown,
/// The symbol `knot_extensions.AlwaysTruthy`
AlwaysTruthy,
/// The symbol `knot_extensions.AlwaysFalsy`
AlwaysFalsy,
/// The symbol `knot_extensions.Not`
Not,
/// The symbol `knot_extensions.Intersection`
@@ -2874,6 +2905,8 @@ impl<'db> KnownInstanceType<'db> {
Self::OrderedDict => "OrderedDict",
Self::ReadOnly => "ReadOnly",
Self::Unknown => "Unknown",
Self::AlwaysTruthy => "AlwaysTruthy",
Self::AlwaysFalsy => "AlwaysFalsy",
Self::Not => "Not",
Self::Intersection => "Intersection",
Self::TypeOf => "TypeOf",
@@ -2917,6 +2950,8 @@ impl<'db> KnownInstanceType<'db> {
| Self::ReadOnly
| Self::TypeAliasType(_)
| Self::Unknown
| Self::AlwaysTruthy
| Self::AlwaysFalsy
| Self::Not
| Self::Intersection
| Self::TypeOf => Truthiness::AlwaysTrue,
@@ -2960,6 +2995,8 @@ impl<'db> KnownInstanceType<'db> {
Self::TypeVar(typevar) => typevar.name(db),
Self::TypeAliasType(_) => "typing.TypeAliasType",
Self::Unknown => "knot_extensions.Unknown",
Self::AlwaysTruthy => "knot_extensions.AlwaysTruthy",
Self::AlwaysFalsy => "knot_extensions.AlwaysFalsy",
Self::Not => "knot_extensions.Not",
Self::Intersection => "knot_extensions.Intersection",
Self::TypeOf => "knot_extensions.TypeOf",
@@ -3006,6 +3043,8 @@ impl<'db> KnownInstanceType<'db> {
Self::Not => KnownClass::SpecialForm,
Self::Intersection => KnownClass::SpecialForm,
Self::Unknown => KnownClass::Object,
Self::AlwaysTruthy => KnownClass::Object,
Self::AlwaysFalsy => KnownClass::Object,
}
}
@@ -3057,6 +3096,8 @@ impl<'db> KnownInstanceType<'db> {
"NotRequired" => Self::NotRequired,
"LiteralString" => Self::LiteralString,
"Unknown" => Self::Unknown,
"AlwaysTruthy" => Self::AlwaysTruthy,
"AlwaysFalsy" => Self::AlwaysFalsy,
"Not" => Self::Not,
"Intersection" => Self::Intersection,
"TypeOf" => Self::TypeOf,
@@ -3109,9 +3150,12 @@ impl<'db> KnownInstanceType<'db> {
| Self::TypeVar(_) => {
matches!(module, KnownModule::Typing | KnownModule::TypingExtensions)
}
Self::Unknown | Self::Not | Self::Intersection | Self::TypeOf => {
module.is_knot_extensions()
}
Self::Unknown
| Self::AlwaysTruthy
| Self::AlwaysFalsy
| Self::Not
| Self::Intersection
| Self::TypeOf => module.is_knot_extensions(),
}
}
@@ -3353,6 +3397,8 @@ pub enum KnownFunction {
/// `typing(_extensions).assert_type`
AssertType,
/// `typing(_extensions).cast`
Cast,
/// `knot_extensions.static_assert`
StaticAssert,
@@ -3385,78 +3431,140 @@ impl KnownFunction {
definition: Definition<'db>,
name: &str,
) -> Option<Self> {
match name {
"reveal_type" if definition.is_typing_definition(db) => Some(KnownFunction::RevealType),
"isinstance" if definition.is_builtin_definition(db) => Some(
KnownFunction::ConstraintFunction(KnownConstraintFunction::IsInstance),
),
"issubclass" if definition.is_builtin_definition(db) => Some(
KnownFunction::ConstraintFunction(KnownConstraintFunction::IsSubclass),
),
"len" if definition.is_builtin_definition(db) => Some(KnownFunction::Len),
"final" if definition.is_typing_definition(db) => Some(KnownFunction::Final),
"no_type_check" if definition.is_typing_definition(db) => {
Some(KnownFunction::NoTypeCheck)
}
"assert_type" if definition.is_typing_definition(db) => Some(KnownFunction::AssertType),
"static_assert" if definition.is_knot_extensions_definition(db) => {
Some(KnownFunction::StaticAssert)
}
"is_subtype_of" if definition.is_knot_extensions_definition(db) => {
Some(KnownFunction::IsSubtypeOf)
}
"is_disjoint_from" if definition.is_knot_extensions_definition(db) => {
Some(KnownFunction::IsDisjointFrom)
}
"is_equivalent_to" if definition.is_knot_extensions_definition(db) => {
Some(KnownFunction::IsEquivalentTo)
}
"is_assignable_to" if definition.is_knot_extensions_definition(db) => {
Some(KnownFunction::IsAssignableTo)
}
"is_fully_static" if definition.is_knot_extensions_definition(db) => {
Some(KnownFunction::IsFullyStatic)
}
"is_singleton" if definition.is_knot_extensions_definition(db) => {
Some(KnownFunction::IsSingleton)
}
"is_single_valued" if definition.is_knot_extensions_definition(db) => {
Some(KnownFunction::IsSingleValued)
}
let candidate = match name {
"isinstance" => Self::ConstraintFunction(KnownConstraintFunction::IsInstance),
"issubclass" => Self::ConstraintFunction(KnownConstraintFunction::IsSubclass),
"reveal_type" => Self::RevealType,
"len" => Self::Len,
"final" => Self::Final,
"no_type_check" => Self::NoTypeCheck,
"assert_type" => Self::AssertType,
"cast" => Self::Cast,
"static_assert" => Self::StaticAssert,
"is_subtype_of" => Self::IsSubtypeOf,
"is_disjoint_from" => Self::IsDisjointFrom,
"is_equivalent_to" => Self::IsEquivalentTo,
"is_assignable_to" => Self::IsAssignableTo,
"is_fully_static" => Self::IsFullyStatic,
"is_singleton" => Self::IsSingleton,
"is_single_valued" => Self::IsSingleValued,
_ => return None,
};
_ => None,
}
candidate
.check_module(file_to_module(db, definition.file(db))?.known()?)
.then_some(candidate)
}
/// Returns a `u32` bitmask specifying whether or not
/// arguments given to a particular function
/// should be interpreted as type expressions or value expressions.
///
/// The argument is treated as a type expression
/// when the corresponding bit is `1`.
/// The least-significant (right-most) bit corresponds to
/// the argument at the index 0 and so on.
///
/// For example, `assert_type()` has the bitmask value of `0b10`.
/// This means the second argument is a type expression and the first a value expression.
const fn takes_type_expression_arguments(self) -> u32 {
const ALL_VALUES: u32 = 0b0;
const SINGLE_TYPE: u32 = 0b1;
const TYPE_TYPE: u32 = 0b11;
const VALUE_TYPE: u32 = 0b10;
/// Return `true` if `self` is defined in `module` at runtime.
const fn check_module(self, module: KnownModule) -> bool {
match self {
KnownFunction::IsEquivalentTo => TYPE_TYPE,
KnownFunction::IsSubtypeOf => TYPE_TYPE,
KnownFunction::IsAssignableTo => TYPE_TYPE,
KnownFunction::IsDisjointFrom => TYPE_TYPE,
KnownFunction::IsFullyStatic => SINGLE_TYPE,
KnownFunction::IsSingleton => SINGLE_TYPE,
KnownFunction::IsSingleValued => SINGLE_TYPE,
KnownFunction::AssertType => VALUE_TYPE,
_ => ALL_VALUES,
Self::ConstraintFunction(constraint_function) => match constraint_function {
KnownConstraintFunction::IsInstance | KnownConstraintFunction::IsSubclass => {
module.is_builtins()
}
},
Self::Len => module.is_builtins(),
Self::AssertType | Self::Cast | Self::RevealType | Self::Final | Self::NoTypeCheck => {
matches!(module, KnownModule::Typing | KnownModule::TypingExtensions)
}
Self::IsAssignableTo
| Self::IsDisjointFrom
| Self::IsEquivalentTo
| Self::IsFullyStatic
| Self::IsSingleValued
| Self::IsSingleton
| Self::IsSubtypeOf
| Self::StaticAssert => module.is_knot_extensions(),
}
}
/// Return the [`ParameterExpectations`] for this function.
const fn parameter_expectations(self) -> ParameterExpectations {
match self {
Self::IsFullyStatic | Self::IsSingleton | Self::IsSingleValued => {
ParameterExpectations::SingleTypeExpression
}
Self::IsEquivalentTo
| Self::IsSubtypeOf
| Self::IsAssignableTo
| Self::IsDisjointFrom => ParameterExpectations::TwoTypeExpressions,
Self::AssertType => ParameterExpectations::ValueExpressionAndTypeExpression,
Self::Cast => ParameterExpectations::TypeExpressionAndValueExpression,
Self::ConstraintFunction(_)
| Self::Len
| Self::Final
| Self::NoTypeCheck
| Self::RevealType
| Self::StaticAssert => ParameterExpectations::AllValueExpressions,
}
}
}
/// Describes whether the parameters in a function expect value expressions or type expressions.
///
/// Whether a specific parameter in the function expects a type expression can be queried
/// using [`ParameterExpectations::expectation_at_index`].
#[derive(Debug, Copy, Clone, PartialEq, Eq, Default)]
enum ParameterExpectations {
/// All parameters in the function expect value expressions
#[default]
AllValueExpressions,
/// The first parameter in the function expects a type expression
SingleTypeExpression,
/// The first two parameters in the function expect type expressions
TwoTypeExpressions,
/// The first parameter in the function expects a value expression,
/// and the second expects a type expression
ValueExpressionAndTypeExpression,
/// The first parameter in the function expects a type expression,
/// and the second expects a value expression
TypeExpressionAndValueExpression,
}
impl ParameterExpectations {
/// Query whether the parameter at `parameter_index` expects a value expression or a type expression
fn expectation_at_index(self, parameter_index: usize) -> ParameterExpectation {
match self {
Self::AllValueExpressions => ParameterExpectation::ValueExpression,
Self::SingleTypeExpression | Self::TypeExpressionAndValueExpression => {
if parameter_index == 0 {
ParameterExpectation::TypeExpression
} else {
ParameterExpectation::ValueExpression
}
}
Self::TwoTypeExpressions => {
if parameter_index < 2 {
ParameterExpectation::TypeExpression
} else {
ParameterExpectation::ValueExpression
}
}
Self::ValueExpressionAndTypeExpression => {
if parameter_index == 1 {
ParameterExpectation::TypeExpression
} else {
ParameterExpectation::ValueExpression
}
}
}
}
}
/// Whether a single parameter in a given function expects a value expression or a [type expression]
///
/// [type expression]: https://typing.readthedocs.io/en/latest/spec/annotations.html#type-and-annotation-expressions
#[derive(Debug, Copy, Clone, PartialEq, Eq, Default)]
enum ParameterExpectation {
/// The parameter expects a value expression
#[default]
ValueExpression,
/// The parameter expects a type expression
TypeExpression,
}
#[salsa::interned]
@@ -4123,7 +4231,7 @@ pub(crate) mod tests {
use super::*;
use crate::db::tests::{setup_db, TestDb, TestDbBuilder};
use crate::stdlib::typing_symbol;
use crate::{resolve_module, PythonVersion};
use crate::PythonVersion;
use ruff_db::files::system_path_to_file;
use ruff_db::parsed::parsed_module;
use ruff_db::system::DbWithTestSystem;
@@ -4147,7 +4255,6 @@ pub(crate) mod tests {
BytesLiteral(&'static str),
// BuiltinInstance("str") corresponds to an instance of the builtin `str` class
BuiltinInstance(&'static str),
TypingInstance(&'static str),
/// Members of the `abc` stdlib module
AbcInstance(&'static str),
AbcClassLiteral(&'static str),
@@ -4164,7 +4271,6 @@ pub(crate) mod tests {
SubclassOfAny,
SubclassOfBuiltinClass(&'static str),
SubclassOfAbcClass(&'static str),
StdlibModule(KnownModule),
SliceLiteral(i32, i32, i32),
AlwaysTruthy,
AlwaysFalsy,
@@ -4190,7 +4296,6 @@ pub(crate) mod tests {
Ty::AbcClassLiteral(s) => {
known_module_symbol(db, KnownModule::Abc, s).expect_type()
}
Ty::TypingInstance(s) => typing_symbol(db, s).expect_type().to_instance(db),
Ty::TypingLiteral => Type::KnownInstance(KnownInstanceType::Literal),
Ty::BuiltinClassLiteral(s) => builtins_symbol(db, s).expect_type(),
Ty::KnownClassInstance(known_class) => known_class.to_instance(db),
@@ -4226,10 +4331,6 @@ pub(crate) mod tests {
.expect_class_literal()
.class,
),
Ty::StdlibModule(module) => {
let module = resolve_module(db, &module.name()).unwrap();
Type::module_literal(db, module.file(), module)
}
Ty::SliceLiteral(start, stop, step) => Type::SliceLiteral(SliceLiteralType::new(
db,
Some(start),
@@ -4242,205 +4343,6 @@ pub(crate) mod tests {
}
}
#[test_case(Ty::BuiltinInstance("str"), Ty::BuiltinInstance("object"))]
#[test_case(Ty::BuiltinInstance("int"), Ty::BuiltinInstance("object"))]
#[test_case(Ty::BuiltinInstance("bool"), Ty::BuiltinInstance("object"))]
#[test_case(Ty::BuiltinInstance("bool"), Ty::BuiltinInstance("int"))]
#[test_case(Ty::Never, Ty::IntLiteral(1))]
#[test_case(Ty::IntLiteral(1), Ty::BuiltinInstance("int"))]
#[test_case(Ty::IntLiteral(1), Ty::BuiltinInstance("object"))]
#[test_case(Ty::BooleanLiteral(true), Ty::BuiltinInstance("bool"))]
#[test_case(Ty::BooleanLiteral(true), Ty::BuiltinInstance("int"))]
#[test_case(Ty::BooleanLiteral(true), Ty::BuiltinInstance("object"))]
#[test_case(Ty::StringLiteral("foo"), Ty::BuiltinInstance("str"))]
#[test_case(Ty::StringLiteral("foo"), Ty::BuiltinInstance("object"))]
#[test_case(Ty::StringLiteral("foo"), Ty::LiteralString)]
#[test_case(Ty::LiteralString, Ty::BuiltinInstance("str"))]
#[test_case(Ty::LiteralString, Ty::BuiltinInstance("object"))]
#[test_case(Ty::BytesLiteral("foo"), Ty::BuiltinInstance("bytes"))]
#[test_case(Ty::BytesLiteral("foo"), Ty::BuiltinInstance("object"))]
#[test_case(Ty::IntLiteral(1), Ty::Union(vec![Ty::BuiltinInstance("int"), Ty::BuiltinInstance("str")]))]
#[test_case(Ty::Union(vec![Ty::BuiltinInstance("str"), Ty::BuiltinInstance("int")]), Ty::BuiltinInstance("object"))]
#[test_case(Ty::Union(vec![Ty::IntLiteral(1), Ty::IntLiteral(2)]), Ty::Union(vec![Ty::IntLiteral(1), Ty::IntLiteral(2), Ty::IntLiteral(3)]))]
#[test_case(Ty::BuiltinInstance("TypeError"), Ty::BuiltinInstance("Exception"))]
#[test_case(Ty::Tuple(vec![]), Ty::Tuple(vec![]))]
#[test_case(Ty::Tuple(vec![Ty::IntLiteral(42)]), Ty::Tuple(vec![Ty::BuiltinInstance("int")]))]
#[test_case(Ty::Tuple(vec![Ty::IntLiteral(42), Ty::StringLiteral("foo")]), Ty::Tuple(vec![Ty::BuiltinInstance("int"), Ty::BuiltinInstance("str")]))]
#[test_case(Ty::Tuple(vec![Ty::BuiltinInstance("int"), Ty::StringLiteral("foo")]), Ty::Tuple(vec![Ty::BuiltinInstance("int"), Ty::BuiltinInstance("str")]))]
#[test_case(Ty::Tuple(vec![Ty::IntLiteral(42), Ty::BuiltinInstance("str")]), Ty::Tuple(vec![Ty::BuiltinInstance("int"), Ty::BuiltinInstance("str")]))]
#[test_case(
Ty::BuiltinInstance("FloatingPointError"),
Ty::BuiltinInstance("Exception")
)]
#[test_case(Ty::Intersection{pos: vec![Ty::BuiltinInstance("int")], neg: vec![Ty::IntLiteral(2)]}, Ty::BuiltinInstance("int"))]
#[test_case(Ty::Intersection{pos: vec![Ty::BuiltinInstance("int")], neg: vec![Ty::IntLiteral(2)]}, Ty::Intersection{pos: vec![], neg: vec![Ty::IntLiteral(2)]})]
#[test_case(Ty::Intersection{pos: vec![], neg: vec![Ty::BuiltinInstance("int")]}, Ty::Intersection{pos: vec![], neg: vec![Ty::IntLiteral(2)]})]
#[test_case(Ty::IntLiteral(1), Ty::Intersection{pos: vec![Ty::BuiltinInstance("int")], neg: vec![Ty::IntLiteral(2)]})]
#[test_case(Ty::Intersection{pos: vec![Ty::BuiltinInstance("str")], neg: vec![Ty::StringLiteral("foo")]}, Ty::Intersection{pos: vec![], neg: vec![Ty::IntLiteral(2)]})]
#[test_case(Ty::BuiltinClassLiteral("int"), Ty::BuiltinClassLiteral("int"))]
#[test_case(Ty::BuiltinClassLiteral("int"), Ty::BuiltinInstance("object"))]
#[test_case(Ty::TypingLiteral, Ty::TypingInstance("_SpecialForm"))]
#[test_case(Ty::TypingLiteral, Ty::BuiltinInstance("object"))]
#[test_case(Ty::AbcClassLiteral("ABC"), Ty::AbcInstance("ABCMeta"))]
#[test_case(Ty::AbcInstance("ABCMeta"), Ty::SubclassOfBuiltinClass("object"))]
#[test_case(Ty::Tuple(vec![Ty::BuiltinInstance("int")]), Ty::BuiltinInstance("tuple"))]
#[test_case(Ty::BuiltinClassLiteral("str"), Ty::BuiltinInstance("type"))]
#[test_case(
Ty::StdlibModule(KnownModule::Typing),
Ty::KnownClassInstance(KnownClass::ModuleType)
)]
#[test_case(Ty::SliceLiteral(1, 2, 3), Ty::BuiltinInstance("slice"))]
#[test_case(Ty::SubclassOfBuiltinClass("str"), Ty::Intersection{pos: vec![], neg: vec![Ty::None]})]
#[test_case(Ty::IntLiteral(1), Ty::AlwaysTruthy)]
#[test_case(Ty::IntLiteral(0), Ty::AlwaysFalsy)]
#[test_case(Ty::AlwaysTruthy, Ty::BuiltinInstance("object"))]
#[test_case(Ty::AlwaysFalsy, Ty::BuiltinInstance("object"))]
#[test_case(Ty::Never, Ty::AlwaysTruthy)]
#[test_case(Ty::Never, Ty::AlwaysFalsy)]
#[test_case(Ty::BuiltinClassLiteral("bool"), Ty::SubclassOfBuiltinClass("int"))]
#[test_case(Ty::Intersection{pos: vec![], neg: vec![Ty::LiteralString]}, Ty::BuiltinInstance("object"))]
fn is_subtype_of(from: Ty, to: Ty) {
let db = setup_db();
assert!(from.into_type(&db).is_subtype_of(&db, to.into_type(&db)));
}
#[test_case(Ty::BuiltinInstance("object"), Ty::BuiltinInstance("int"))]
#[test_case(Ty::Unknown, Ty::Unknown)]
#[test_case(Ty::Unknown, Ty::IntLiteral(1))]
#[test_case(Ty::Any, Ty::Any)]
#[test_case(Ty::Any, Ty::IntLiteral(1))]
#[test_case(Ty::IntLiteral(1), Ty::Unknown)]
#[test_case(Ty::IntLiteral(1), Ty::Any)]
#[test_case(Ty::IntLiteral(1), Ty::Union(vec![Ty::Unknown, Ty::BuiltinInstance("str")]))]
#[test_case(Ty::IntLiteral(1), Ty::BuiltinInstance("str"))]
#[test_case(Ty::Union(vec![Ty::IntLiteral(1), Ty::IntLiteral(2)]), Ty::IntLiteral(1))]
#[test_case(Ty::Union(vec![Ty::IntLiteral(1), Ty::IntLiteral(2)]), Ty::Union(vec![Ty::IntLiteral(1), Ty::IntLiteral(3)]))]
#[test_case(Ty::BuiltinInstance("int"), Ty::BuiltinInstance("str"))]
#[test_case(Ty::BuiltinInstance("int"), Ty::IntLiteral(1))]
#[test_case(Ty::Tuple(vec![]), Ty::Tuple(vec![Ty::IntLiteral(1)]))]
#[test_case(Ty::Tuple(vec![Ty::IntLiteral(42)]), Ty::Tuple(vec![Ty::BuiltinInstance("str")]))]
#[test_case(Ty::Tuple(vec![Ty::Todo]), Ty::Tuple(vec![Ty::IntLiteral(2)]))]
#[test_case(Ty::Tuple(vec![Ty::IntLiteral(2)]), Ty::Tuple(vec![Ty::Todo]))]
#[test_case(Ty::Intersection{pos: vec![Ty::BuiltinInstance("int")], neg: vec![Ty::IntLiteral(2)]}, Ty::Intersection{pos: vec![Ty::BuiltinInstance("int")], neg: vec![Ty::IntLiteral(3)]})]
#[test_case(Ty::Intersection{pos: vec![], neg: vec![Ty::IntLiteral(2)]}, Ty::Intersection{pos: vec![], neg: vec![Ty::IntLiteral(3)]})]
#[test_case(Ty::Intersection{pos: vec![], neg: vec![Ty::IntLiteral(2)]}, Ty::Intersection{pos: vec![], neg: vec![Ty::BuiltinInstance("int")]})]
#[test_case(Ty::BuiltinInstance("int"), Ty::Intersection{pos: vec![], neg: vec![Ty::IntLiteral(3)]})]
#[test_case(Ty::IntLiteral(1), Ty::Intersection{pos: vec![Ty::BuiltinInstance("int")], neg: vec![Ty::IntLiteral(1)]})]
#[test_case(Ty::BuiltinClassLiteral("int"), Ty::BuiltinClassLiteral("object"))]
#[test_case(Ty::BuiltinInstance("int"), Ty::BuiltinClassLiteral("int"))]
#[test_case(Ty::TypingInstance("_SpecialForm"), Ty::TypingLiteral)]
#[test_case(Ty::BuiltinInstance("type"), Ty::SubclassOfBuiltinClass("str"))]
#[test_case(Ty::BuiltinClassLiteral("str"), Ty::SubclassOfAny)]
#[test_case(Ty::AbcInstance("ABCMeta"), Ty::SubclassOfBuiltinClass("type"))]
#[test_case(Ty::SubclassOfBuiltinClass("str"), Ty::BuiltinClassLiteral("str"))]
#[test_case(Ty::IntLiteral(1), Ty::AlwaysFalsy)]
#[test_case(Ty::IntLiteral(0), Ty::AlwaysTruthy)]
#[test_case(Ty::BuiltinInstance("str"), Ty::AlwaysTruthy)]
#[test_case(Ty::BuiltinInstance("str"), Ty::AlwaysFalsy)]
fn is_not_subtype_of(from: Ty, to: Ty) {
let db = setup_db();
assert!(!from.into_type(&db).is_subtype_of(&db, to.into_type(&db)));
}
#[test]
fn is_subtype_of_class_literals() {
let mut db = setup_db();
db.write_dedented(
"/src/module.py",
"
class Base: ...
class Derived(Base): ...
class Unrelated: ...
U = Base if flag else Unrelated
",
)
.unwrap();
let module = ruff_db::files::system_path_to_file(&db, "/src/module.py").unwrap();
// `literal_base` represents `Literal[Base]`.
let literal_base = super::global_symbol(&db, module, "Base").expect_type();
let literal_derived = super::global_symbol(&db, module, "Derived").expect_type();
let u = super::global_symbol(&db, module, "U").expect_type();
assert!(literal_base.is_class_literal());
assert!(literal_base.is_subtype_of(&db, Ty::BuiltinInstance("type").into_type(&db)));
assert!(literal_base.is_subtype_of(&db, Ty::BuiltinInstance("object").into_type(&db)));
assert!(literal_derived.is_class_literal());
// `subclass_of_base` represents `Type[Base]`.
let subclass_of_base = SubclassOfType::from(&db, literal_base.expect_class_literal().class);
assert!(literal_base.is_subtype_of(&db, subclass_of_base));
assert!(literal_derived.is_subtype_of(&db, subclass_of_base));
let subclass_of_derived =
SubclassOfType::from(&db, literal_derived.expect_class_literal().class);
assert!(literal_derived.is_subtype_of(&db, subclass_of_derived));
assert!(!literal_base.is_subtype_of(&db, subclass_of_derived));
// Type[Derived] <: Type[Base]
assert!(subclass_of_derived.is_subtype_of(&db, subclass_of_base));
assert!(u.is_union());
assert!(u.is_subtype_of(&db, Ty::BuiltinInstance("type").into_type(&db)));
assert!(u.is_subtype_of(&db, Ty::BuiltinInstance("object").into_type(&db)));
}
#[test]
fn is_subtype_of_intersection_of_class_instances() {
let mut db = setup_db();
db.write_dedented(
"/src/module.py",
"
class A: ...
a = A()
class B: ...
b = B()
",
)
.unwrap();
let module = ruff_db::files::system_path_to_file(&db, "/src/module.py").unwrap();
let a_ty = super::global_symbol(&db, module, "a").expect_type();
let b_ty = super::global_symbol(&db, module, "b").expect_type();
let intersection = IntersectionBuilder::new(&db)
.add_positive(a_ty)
.add_positive(b_ty)
.build();
assert_eq!(intersection.display(&db).to_string(), "A & B");
assert!(!a_ty.is_subtype_of(&db, b_ty));
assert!(intersection.is_subtype_of(&db, b_ty));
assert!(intersection.is_subtype_of(&db, a_ty));
}
#[test_case(
Ty::Union(vec![Ty::IntLiteral(1), Ty::IntLiteral(2)]),
Ty::Union(vec![Ty::IntLiteral(1), Ty::IntLiteral(2)])
)]
#[test_case(Ty::SubclassOfBuiltinClass("object"), Ty::BuiltinInstance("type"))]
fn is_equivalent_to(from: Ty, to: Ty) {
let db = setup_db();
let from = from.into_type(&db);
let to = to.into_type(&db);
assert!(from.is_equivalent_to(&db, to));
assert!(to.is_equivalent_to(&db, from));
}
#[test_case(Ty::Any, Ty::Any)]
#[test_case(Ty::Any, Ty::None)]
#[test_case(Ty::Unknown, Ty::Unknown)]
#[test_case(Ty::Todo, Ty::Todo)]
#[test_case(Ty::Union(vec![Ty::IntLiteral(1), Ty::IntLiteral(2)]), Ty::Union(vec![Ty::IntLiteral(1), Ty::IntLiteral(0)]))]
#[test_case(Ty::Union(vec![Ty::IntLiteral(1), Ty::IntLiteral(2)]), Ty::Union(vec![Ty::IntLiteral(1), Ty::IntLiteral(2), Ty::IntLiteral(3)]))]
fn is_not_equivalent_to(from: Ty, to: Ty) {
let db = setup_db();
let from = from.into_type(&db);
let to = to.into_type(&db);
assert!(!from.is_equivalent_to(&db, to));
assert!(!to.is_equivalent_to(&db, from));
}
#[test_case(Ty::Never, Ty::Never)]
#[test_case(Ty::Never, Ty::None)]
#[test_case(Ty::Never, Ty::BuiltinInstance("int"))]

View File

@@ -30,8 +30,6 @@ use crate::types::{InstanceType, IntersectionType, KnownClass, Type, UnionType};
use crate::{Db, FxOrderSet};
use smallvec::SmallVec;
use super::Truthiness;
pub(crate) struct UnionBuilder<'db> {
elements: Vec<Type<'db>>,
db: &'db dyn Db,
@@ -65,6 +63,8 @@ impl<'db> UnionBuilder<'db> {
let mut to_add = ty;
let mut to_remove = SmallVec::<[usize; 2]>::new();
let ty_negated = ty.negate(self.db);
for (index, element) in self.elements.iter().enumerate() {
if Some(*element) == bool_pair {
to_add = KnownClass::Bool.to_instance(self.db);
@@ -80,6 +80,17 @@ impl<'db> UnionBuilder<'db> {
return self;
} else if element.is_subtype_of(self.db, ty) {
to_remove.push(index);
} else if ty_negated.is_subtype_of(self.db, *element) {
// We add `ty` to the union. We just checked that `~ty` is a subtype of an existing `element`.
// This also means that `~ty | ty` is a subtype of `element | ty`, because both elements in the
// first union are subtypes of the corresponding elements in the second union. But `~ty | ty` is
// just `object`. Since `object` is a subtype of `element | ty`, we can only conclude that
// `element | ty` must be `object` (object has no other supertypes). This means we can simplify
// the whole union to just `object`, since all other potential elements would also be subtypes of
// `object`.
self.elements.clear();
self.elements.push(KnownClass::Object.to_instance(self.db));
return self;
}
}
match to_remove[..] {
@@ -235,83 +246,164 @@ struct InnerIntersectionBuilder<'db> {
impl<'db> InnerIntersectionBuilder<'db> {
/// Adds a positive type to this intersection.
fn add_positive(&mut self, db: &'db dyn Db, new_positive: Type<'db>) {
if let Type::Intersection(other) = new_positive {
for pos in other.positive(db) {
self.add_positive(db, *pos);
fn add_positive(&mut self, db: &'db dyn Db, mut new_positive: Type<'db>) {
match new_positive {
// `LiteralString & AlwaysTruthy` -> `LiteralString & ~Literal[""]`
Type::AlwaysTruthy if self.positive.contains(&Type::LiteralString) => {
self.add_negative(db, Type::string_literal(db, ""));
}
for neg in other.negative(db) {
self.add_negative(db, *neg);
// `LiteralString & AlwaysFalsy` -> `Literal[""]`
Type::AlwaysFalsy if self.positive.swap_remove(&Type::LiteralString) => {
self.add_positive(db, Type::string_literal(db, ""));
}
} else {
// ~Literal[True] & bool = Literal[False]
// ~AlwaysTruthy & bool = Literal[False]
if let Type::Instance(InstanceType { class }) = new_positive {
if class.is_known(db, KnownClass::Bool) {
if let Some(new_type) = self
.negative
.iter()
.find(|element| {
element.is_boolean_literal()
| matches!(element, Type::AlwaysFalsy | Type::AlwaysTruthy)
})
.map(|element| {
Type::BooleanLiteral(element.bool(db) != Truthiness::AlwaysTrue)
})
// `AlwaysTruthy & LiteralString` -> `LiteralString & ~Literal[""]`
Type::LiteralString if self.positive.swap_remove(&Type::AlwaysTruthy) => {
self.add_positive(db, Type::LiteralString);
self.add_negative(db, Type::string_literal(db, ""));
}
// `AlwaysFalsy & LiteralString` -> `Literal[""]`
Type::LiteralString if self.positive.swap_remove(&Type::AlwaysFalsy) => {
self.add_positive(db, Type::string_literal(db, ""));
}
// `LiteralString & ~AlwaysTruthy` -> `LiteralString & AlwaysFalsy` -> `Literal[""]`
Type::LiteralString if self.negative.swap_remove(&Type::AlwaysTruthy) => {
self.add_positive(db, Type::string_literal(db, ""));
}
// `LiteralString & ~AlwaysFalsy` -> `LiteralString & ~Literal[""]`
Type::LiteralString if self.negative.swap_remove(&Type::AlwaysFalsy) => {
self.add_positive(db, Type::LiteralString);
self.add_negative(db, Type::string_literal(db, ""));
}
// `(A & B & ~C) & (D & E & ~F)` -> `A & B & D & E & ~C & ~F`
Type::Intersection(other) => {
for pos in other.positive(db) {
self.add_positive(db, *pos);
}
for neg in other.negative(db) {
self.add_negative(db, *neg);
}
}
_ => {
let known_instance = new_positive
.into_instance()
.and_then(|instance| instance.class.known(db));
if known_instance == Some(KnownClass::Object) {
// `object & T` -> `T`; it is always redundant to add `object` to an intersection
return;
}
let addition_is_bool_instance = known_instance == Some(KnownClass::Bool);
for (index, existing_positive) in self.positive.iter().enumerate() {
match existing_positive {
// `AlwaysTruthy & bool` -> `Literal[True]`
Type::AlwaysTruthy if addition_is_bool_instance => {
new_positive = Type::BooleanLiteral(true);
}
// `AlwaysFalsy & bool` -> `Literal[False]`
Type::AlwaysFalsy if addition_is_bool_instance => {
new_positive = Type::BooleanLiteral(false);
}
Type::Instance(InstanceType { class })
if class.is_known(db, KnownClass::Bool) =>
{
match new_positive {
// `bool & AlwaysTruthy` -> `Literal[True]`
Type::AlwaysTruthy => {
new_positive = Type::BooleanLiteral(true);
}
// `bool & AlwaysFalsy` -> `Literal[False]`
Type::AlwaysFalsy => {
new_positive = Type::BooleanLiteral(false);
}
_ => continue,
}
}
_ => continue,
}
self.positive.swap_remove_index(index);
break;
}
if addition_is_bool_instance {
for (index, existing_negative) in self.negative.iter().enumerate() {
match existing_negative {
// `bool & ~Literal[False]` -> `Literal[True]`
// `bool & ~Literal[True]` -> `Literal[False]`
Type::BooleanLiteral(bool_value) => {
new_positive = Type::BooleanLiteral(!bool_value);
}
// `bool & ~AlwaysTruthy` -> `Literal[False]`
Type::AlwaysTruthy => {
new_positive = Type::BooleanLiteral(false);
}
// `bool & ~AlwaysFalsy` -> `Literal[True]`
Type::AlwaysFalsy => {
new_positive = Type::BooleanLiteral(true);
}
_ => continue,
}
self.negative.swap_remove_index(index);
break;
}
}
let mut to_remove = SmallVec::<[usize; 1]>::new();
for (index, existing_positive) in self.positive.iter().enumerate() {
// S & T = S if S <: T
if existing_positive.is_subtype_of(db, new_positive)
|| existing_positive.is_same_gradual_form(new_positive)
{
return;
}
// same rule, reverse order
if new_positive.is_subtype_of(db, *existing_positive) {
to_remove.push(index);
}
// A & B = Never if A and B are disjoint
if new_positive.is_disjoint_from(db, *existing_positive) {
*self = Self::default();
self.positive.insert(new_type);
self.positive.insert(Type::Never);
return;
}
}
}
for index in to_remove.into_iter().rev() {
self.positive.swap_remove_index(index);
}
let mut to_remove = SmallVec::<[usize; 1]>::new();
for (index, existing_positive) in self.positive.iter().enumerate() {
// S & T = S if S <: T
if existing_positive.is_subtype_of(db, new_positive)
|| existing_positive.is_same_gradual_form(new_positive)
{
return;
let mut to_remove = SmallVec::<[usize; 1]>::new();
for (index, existing_negative) in self.negative.iter().enumerate() {
// S & ~T = Never if S <: T
if new_positive.is_subtype_of(db, *existing_negative) {
*self = Self::default();
self.positive.insert(Type::Never);
return;
}
// A & ~B = A if A and B are disjoint
if existing_negative.is_disjoint_from(db, new_positive) {
to_remove.push(index);
}
}
// same rule, reverse order
if new_positive.is_subtype_of(db, *existing_positive) {
to_remove.push(index);
for index in to_remove.into_iter().rev() {
self.negative.swap_remove_index(index);
}
// A & B = Never if A and B are disjoint
if new_positive.is_disjoint_from(db, *existing_positive) {
*self = Self::default();
self.positive.insert(Type::Never);
return;
}
}
for index in to_remove.iter().rev() {
self.positive.swap_remove_index(*index);
}
let mut to_remove = SmallVec::<[usize; 1]>::new();
for (index, existing_negative) in self.negative.iter().enumerate() {
// S & ~T = Never if S <: T
if new_positive.is_subtype_of(db, *existing_negative) {
*self = Self::default();
self.positive.insert(Type::Never);
return;
}
// A & ~B = A if A and B are disjoint
if existing_negative.is_disjoint_from(db, new_positive) {
to_remove.push(index);
}
self.positive.insert(new_positive);
}
for index in to_remove.iter().rev() {
self.negative.swap_remove_index(*index);
}
self.positive.insert(new_positive);
}
}
/// Adds a negative type to this intersection.
fn add_negative(&mut self, db: &'db dyn Db, new_negative: Type<'db>) {
let contains_bool = || {
self.positive
.iter()
.filter_map(|ty| ty.into_instance())
.filter_map(|instance| instance.class.known(db))
.any(KnownClass::is_bool)
};
match new_negative {
Type::Intersection(inter) => {
for pos in inter.positive(db) {
@@ -335,15 +427,23 @@ impl<'db> InnerIntersectionBuilder<'db> {
// simplify the representation.
self.add_positive(db, ty);
}
// bool & ~Literal[True] = Literal[False]
// bool & ~AlwaysTruthy = Literal[False]
Type::BooleanLiteral(_) | Type::AlwaysFalsy | Type::AlwaysTruthy
if self.positive.contains(&KnownClass::Bool.to_instance(db)) =>
{
*self = Self::default();
self.positive.insert(Type::BooleanLiteral(
new_negative.bool(db) != Truthiness::AlwaysTrue,
));
// `bool & ~AlwaysTruthy` -> `bool & Literal[False]`
// `bool & ~Literal[True]` -> `bool & Literal[False]`
Type::AlwaysTruthy | Type::BooleanLiteral(true) if contains_bool() => {
self.add_positive(db, Type::BooleanLiteral(false));
}
// `LiteralString & ~AlwaysTruthy` -> `LiteralString & Literal[""]`
Type::AlwaysTruthy if self.positive.contains(&Type::LiteralString) => {
self.add_positive(db, Type::string_literal(db, ""));
}
// `bool & ~AlwaysFalsy` -> `bool & Literal[True]`
// `bool & ~Literal[False]` -> `bool & Literal[True]`
Type::AlwaysFalsy | Type::BooleanLiteral(false) if contains_bool() => {
self.add_positive(db, Type::BooleanLiteral(true));
}
// `LiteralString & ~AlwaysFalsy` -> `LiteralString & ~Literal[""]`
Type::AlwaysFalsy if self.positive.contains(&Type::LiteralString) => {
self.add_negative(db, Type::string_literal(db, ""));
}
_ => {
let mut to_remove = SmallVec::<[usize; 1]>::new();
@@ -357,8 +457,8 @@ impl<'db> InnerIntersectionBuilder<'db> {
return;
}
}
for index in to_remove.iter().rev() {
self.negative.swap_remove_index(*index);
for index in to_remove.into_iter().rev() {
self.negative.swap_remove_index(index);
}
for existing_positive in &self.positive {

View File

@@ -51,7 +51,7 @@ pub(super) enum CallOutcome<'db> {
}
impl<'db> CallOutcome<'db> {
/// Create a new `CallOutcome::Callable` with given return type.
/// Create a new `CallOutcome::Callable` with given binding.
pub(super) fn callable(binding: CallBinding<'db>) -> CallOutcome<'db> {
CallOutcome::Callable { binding }
}
@@ -80,7 +80,7 @@ impl<'db> CallOutcome<'db> {
}
}
/// Create a new `CallOutcome::AssertType` with given revealed and return types.
/// Create a new `CallOutcome::AssertType` with given asserted and return types.
pub(super) fn asserted(binding: CallBinding<'db>, asserted_ty: Type<'db>) -> CallOutcome<'db> {
CallOutcome::AssertType {
binding,
@@ -280,7 +280,7 @@ impl<'db> CallOutcome<'db> {
}),
}
}
CallOutcome::StaticAssertionError {
Self::StaticAssertionError {
binding,
error_kind,
} => {
@@ -325,7 +325,7 @@ impl<'db> CallOutcome<'db> {
Ok(Type::unknown())
}
CallOutcome::AssertType {
Self::AssertType {
binding,
asserted_ty,
} => {

View File

@@ -84,7 +84,7 @@ pub(crate) fn bind_call<'db>(
}
}
if let Some(existing) = parameter_tys[index].replace(*argument_ty) {
if parameter.is_variadic() {
if parameter.is_variadic() || parameter.is_keyword_variadic() {
let union = UnionType::from_elements(db, [existing, *argument_ty]);
parameter_tys[index].replace(union);
} else {

View File

@@ -105,7 +105,9 @@ impl<'db> ClassBase<'db> {
| KnownInstanceType::Optional
| KnownInstanceType::Not
| KnownInstanceType::Intersection
| KnownInstanceType::TypeOf => None,
| KnownInstanceType::TypeOf
| KnownInstanceType::AlwaysTruthy
| KnownInstanceType::AlwaysFalsy => None,
KnownInstanceType::Unknown => Some(Self::unknown()),
KnownInstanceType::Any => Some(Self::any()),
// TODO: Classes inheriting from `typing.Type` et al. also have `Generic` in their MRO

View File

@@ -83,6 +83,7 @@ use super::slots::check_class_slots;
use super::string_annotation::{
parse_string_annotation, BYTE_STRING_TYPE_ANNOTATION, FSTRING_TYPE_ANNOTATION,
};
use super::{ParameterExpectation, ParameterExpectations};
/// Infer all types for a [`ScopeId`], including all definitions and expressions in that scope.
/// Use when checking a scope, or needing to provide a type for an arbitrary expression in the
@@ -956,7 +957,7 @@ impl<'db> TypeInferenceBuilder<'db> {
self.infer_type_parameters(type_params);
if let Some(arguments) = class.arguments.as_deref() {
self.infer_arguments(arguments, 0b0);
self.infer_arguments(arguments, ParameterExpectations::default());
}
}
@@ -2601,18 +2602,15 @@ impl<'db> TypeInferenceBuilder<'db> {
fn infer_arguments<'a>(
&mut self,
arguments: &'a ast::Arguments,
infer_as_type_expressions: u32,
parameter_expectations: ParameterExpectations,
) -> CallArguments<'a, 'db> {
arguments
.arguments_source_order()
.enumerate()
.map(|(index, arg_or_keyword)| {
let infer_argument_type = if index < u32::BITS as usize
&& infer_as_type_expressions & (1 << index) != 0
{
Self::infer_type_expression
} else {
Self::infer_expression
let infer_argument_type = match parameter_expectations.expectation_at_index(index) {
ParameterExpectation::TypeExpression => Self::infer_type_expression,
ParameterExpectation::ValueExpression => Self::infer_expression,
};
match arg_or_keyword {
@@ -3157,13 +3155,13 @@ impl<'db> TypeInferenceBuilder<'db> {
let function_type = self.infer_expression(func);
let infer_arguments_as_type_expressions = function_type
let parameter_expectations = function_type
.into_function_literal()
.and_then(|f| f.known(self.db()))
.map(KnownFunction::takes_type_expression_arguments)
.unwrap_or(0b0);
.map(KnownFunction::parameter_expectations)
.unwrap_or_default();
let call_arguments = self.infer_arguments(arguments, infer_arguments_as_type_expressions);
let call_arguments = self.infer_arguments(arguments, parameter_expectations);
function_type
.call(self.db(), &call_arguments)
.unwrap_with_diagnostic(&self.context, call_expression.into())
@@ -4541,7 +4539,7 @@ impl<'db> TypeInferenceBuilder<'db> {
return dunder_getitem_method
.call(self.db(), &CallArguments::positional([value_ty, slice_ty]))
.return_ty_result( &self.context, value_node.into())
.return_ty_result(&self.context, value_node.into())
.unwrap_or_else(|err| {
self.context.report_lint(
&CALL_NON_CALLABLE,
@@ -5068,6 +5066,9 @@ impl<'db> TypeInferenceBuilder<'db> {
Type::KnownInstance(KnownInstanceType::Any) => {
SubclassOfType::subclass_of_any()
}
Type::KnownInstance(KnownInstanceType::Unknown) => {
SubclassOfType::subclass_of_unknown()
}
_ => todo_type!("unsupported type[X] special form"),
}
}
@@ -5345,8 +5346,8 @@ impl<'db> TypeInferenceBuilder<'db> {
todo_type!("`NotRequired[]` type qualifier")
}
KnownInstanceType::ClassVar => {
self.infer_type_expression(arguments_slice);
todo_type!("`ClassVar[]` type qualifier")
let ty = self.infer_type_expression(arguments_slice);
ty
}
KnownInstanceType::Final => {
self.infer_type_expression(arguments_slice);
@@ -5372,7 +5373,11 @@ impl<'db> TypeInferenceBuilder<'db> {
self.infer_type_expression(arguments_slice);
todo_type!("`Unpack[]` special form")
}
KnownInstanceType::NoReturn | KnownInstanceType::Never | KnownInstanceType::Any => {
KnownInstanceType::NoReturn
| KnownInstanceType::Never
| KnownInstanceType::Any
| KnownInstanceType::AlwaysTruthy
| KnownInstanceType::AlwaysFalsy => {
self.context.report_lint(
&INVALID_TYPE_FORM,
subscript.into(),

View File

@@ -28,7 +28,7 @@ use std::sync::{Arc, Mutex, MutexGuard, OnceLock};
use super::tests::Ty;
use crate::db::tests::{setup_db, TestDb};
use crate::types::KnownClass;
use crate::types::{IntersectionBuilder, KnownClass, Type, UnionType};
use quickcheck::{Arbitrary, Gen};
fn arbitrary_core_type(g: &mut Gen) -> Ty {
@@ -219,16 +219,41 @@ macro_rules! type_property_test {
};
}
fn intersection<'db>(db: &'db TestDb, s: Type<'db>, t: Type<'db>) -> Type<'db> {
IntersectionBuilder::new(db)
.add_positive(s)
.add_positive(t)
.build()
}
fn union<'db>(db: &'db TestDb, s: Type<'db>, t: Type<'db>) -> Type<'db> {
UnionType::from_elements(db, [s, t])
}
mod stable {
use super::union;
use crate::types::{KnownClass, Type};
// `T` is equivalent to itself.
// Reflexivity: `T` is equivalent to itself.
type_property_test!(
equivalent_to_is_reflexive, db,
forall types t. t.is_fully_static(db) => t.is_equivalent_to(db, t)
);
// `T` is a subtype of itself.
// Symmetry: If `S` is equivalent to `T`, then `T` must be equivalent to `S`.
// Note that this (trivially) holds true for gradual types as well.
type_property_test!(
equivalent_to_is_symmetric, db,
forall types s, t. s.is_equivalent_to(db, t) => t.is_equivalent_to(db, s)
);
// Transitivity: If `S` is equivalent to `T` and `T` is equivalent to `U`, then `S` must be equivalent to `U`.
type_property_test!(
equivalent_to_is_transitive, db,
forall types s, t, u. s.is_equivalent_to(db, t) && t.is_equivalent_to(db, u) => s.is_equivalent_to(db, u)
);
// A fully static type `T` is a subtype of itself.
type_property_test!(
subtype_of_is_reflexive, db,
forall types t. t.is_fully_static(db) => t.is_subtype_of(db, t)
@@ -305,6 +330,14 @@ mod stable {
never_subtype_of_every_fully_static_type, db,
forall types t. t.is_fully_static(db) => Type::Never.is_subtype_of(db, t)
);
// For any two fully static types, each type in the pair must be a subtype of their union.
type_property_test!(
all_fully_static_type_pairs_are_subtype_of_their_union, db,
forall types s, t.
s.is_fully_static(db) && t.is_fully_static(db)
=> s.is_subtype_of(db, union(db, s, t)) && t.is_subtype_of(db, union(db, s, t))
);
}
/// This module contains property tests that currently lead to many false positives.
@@ -315,10 +348,7 @@ mod stable {
/// tests to the `stable` section. In the meantime, it can still be useful to run these
/// tests (using [`types::property_tests::flaky`]), to see if there are any new obvious bugs.
mod flaky {
use crate::{
db::tests::TestDb,
types::{IntersectionBuilder, Type},
};
use super::{intersection, union};
// Currently fails due to https://github.com/astral-sh/ruff/issues/14899
// `T` can be assigned to itself.
@@ -327,21 +357,8 @@ mod flaky {
forall types t. t.is_assignable_to(db, t)
);
// Currently fails due to https://github.com/astral-sh/ruff/issues/14899
// An intersection of two types should be assignable to both of them
fn intersection<'db>(db: &'db TestDb, s: Type<'db>, t: Type<'db>) -> Type<'db> {
IntersectionBuilder::new(db)
.add_positive(s)
.add_positive(t)
.build()
}
type_property_test!(
intersection_assignable_to_both, db,
forall types s, t. intersection(db, s, t).is_assignable_to(db, s) && intersection(db, s, t).is_assignable_to(db, t)
);
// `S <: T` and `T <: S` implies that `S` is equivalent to `T`.
// This very often passes now, but occasionally flakes due to https://github.com/astral-sh/ruff/issues/15380
type_property_test!(
subtype_of_is_antisymmetric, db,
forall types s, t. s.is_subtype_of(db, t) && t.is_subtype_of(db, s) => s.is_equivalent_to(db, t)
@@ -352,4 +369,39 @@ mod flaky {
double_negation_is_identity, db,
forall types t. t.negate(db).negate(db).is_equivalent_to(db, t)
);
// ~T should be disjoint from T
type_property_test!(
negation_is_disjoint, db,
forall types t. t.is_fully_static(db) => t.negate(db).is_disjoint_from(db, t)
);
// If `S <: T`, then `~T <: ~S`.
type_property_test!(
negation_reverses_subtype_order, db,
forall types s, t. s.is_subtype_of(db, t) => t.negate(db).is_subtype_of(db, s.negate(db))
);
// For two fully static types, their intersection must be a subtype of each type in the pair.
type_property_test!(
all_fully_static_type_pairs_are_supertypes_of_their_intersection, db,
forall types s, t.
s.is_fully_static(db) && t.is_fully_static(db)
=> intersection(db, s, t).is_subtype_of(db, s) && intersection(db, s, t).is_subtype_of(db, t)
);
// And for non-fully-static types, the intersection of a pair of types
// should be assignable to both types of the pair.
// Currently fails due to https://github.com/astral-sh/ruff/issues/14899
type_property_test!(
all_type_pairs_can_be_assigned_from_their_intersection, db,
forall types s, t. intersection(db, s, t).is_assignable_to(db, s) && intersection(db, s, t).is_assignable_to(db, t)
);
// For *any* pair of types, whether fully static or not,
// each of the pair should be assignable to the union of the two.
type_property_test!(
all_type_pairs_are_assignable_to_their_union, db,
forall types s, t. s.is_assignable_to(db, union(db, s, t)) && t.is_assignable_to(db, union(db, s, t))
);
}

View File

@@ -1,7 +1,7 @@
use ruff_db::source::source_text;
use ruff_python_ast::str::raw_contents;
use ruff_python_ast::{self as ast, ModExpression, StringFlags};
use ruff_python_parser::{parse_expression_range, Parsed};
use ruff_python_ast::{self as ast, ModExpression};
use ruff_python_parser::Parsed;
use ruff_text_size::Ranged;
use crate::declare_lint;
@@ -153,19 +153,7 @@ pub(crate) fn parse_string_annotation(
} else if raw_contents(node_text)
.is_some_and(|raw_contents| raw_contents == string_literal.as_str())
{
let range_excluding_quotes = string_literal
.range()
.add_start(string_literal.flags.opener_len())
.sub_end(string_literal.flags.closer_len());
// TODO: Support multiline strings like:
// ```py
// x: """
// int
// | float
// """ = 1
// ```
match parse_expression_range(source.as_str(), range_excluding_quotes) {
match ruff_python_parser::parse_string_annotation(source.as_str(), string_literal) {
Ok(parsed) => return Some(parsed),
Err(parse_error) => context.report_lint(
&INVALID_SYNTAX_IN_FORWARD_ANNOTATION,

View File

@@ -40,7 +40,7 @@ pub(super) fn try_show_message(
/// Sends an error to the client with a formatted message. The error is sent in a
/// `window/showMessage` notification.
macro_rules! show_err_msg {
($msg:expr$(, $($arg:tt),*)?) => {
crate::message::show_message(::core::format_args!($msg, $($($arg),*)?).to_string(), lsp_types::MessageType::ERROR)
($msg:expr$(, $($arg:tt)*)?) => {
crate::message::show_message(::core::format_args!($msg$(, $($arg)*)?).to_string(), lsp_types::MessageType::ERROR)
};
}

View File

@@ -86,13 +86,11 @@ fn background_request_task<'a, R: traits::BackgroundDocumentRequestHandler>(
return Box::new(|_, _| {});
};
let db = match path {
AnySystemPath::System(path) => {
match session.workspace_db_for_path(path.as_std_path()) {
Some(db) => db.clone(),
None => session.default_workspace_db().clone(),
}
}
AnySystemPath::SystemVirtual(_) => session.default_workspace_db().clone(),
AnySystemPath::System(path) => match session.project_db_for_path(path.as_std_path()) {
Some(db) => db.clone(),
None => session.default_project_db().clone(),
},
AnySystemPath::SystemVirtual(_) => session.default_project_db().clone(),
};
let Some(snapshot) = session.take_snapshot(url) else {

View File

@@ -36,14 +36,14 @@ impl SyncNotificationHandler for DidChangeTextDocumentHandler {
match path {
AnySystemPath::System(path) => {
let db = match session.workspace_db_for_path_mut(path.as_std_path()) {
let db = match session.project_db_for_path_mut(path.as_std_path()) {
Some(db) => db,
None => session.default_workspace_db_mut(),
None => session.default_project_db_mut(),
};
db.apply_changes(vec![ChangeEvent::file_content_changed(path)], None);
}
AnySystemPath::SystemVirtual(virtual_path) => {
let db = session.default_workspace_db_mut();
let db = session.default_project_db_mut();
db.apply_changes(vec![ChangeEvent::ChangedVirtual(virtual_path)], None);
}
}

View File

@@ -34,7 +34,7 @@ impl SyncNotificationHandler for DidCloseTextDocumentHandler {
.with_failure_code(ErrorCode::InternalError)?;
if let AnySystemPath::SystemVirtual(virtual_path) = path {
let db = session.default_workspace_db_mut();
let db = session.default_project_db_mut();
db.apply_changes(vec![ChangeEvent::DeletedVirtual(virtual_path)], None);
}

View File

@@ -33,7 +33,7 @@ impl SyncNotificationHandler for DidCloseNotebookHandler {
.with_failure_code(lsp_server::ErrorCode::InternalError)?;
if let AnySystemPath::SystemVirtual(virtual_path) = path {
let db = session.default_workspace_db_mut();
let db = session.default_project_db_mut();
db.apply_changes(vec![ChangeEvent::DeletedVirtual(virtual_path)], None);
}

View File

@@ -33,14 +33,14 @@ impl SyncNotificationHandler for DidOpenTextDocumentHandler {
match path {
AnySystemPath::System(path) => {
let db = match session.workspace_db_for_path_mut(path.as_std_path()) {
let db = match session.project_db_for_path_mut(path.as_std_path()) {
Some(db) => db,
None => session.default_workspace_db_mut(),
None => session.default_project_db_mut(),
};
db.apply_changes(vec![ChangeEvent::Opened(path)], None);
}
AnySystemPath::SystemVirtual(virtual_path) => {
let db = session.default_workspace_db_mut();
let db = session.default_project_db_mut();
db.files().virtual_file(db, &virtual_path);
}
}

View File

@@ -41,14 +41,14 @@ impl SyncNotificationHandler for DidOpenNotebookHandler {
match path {
AnySystemPath::System(path) => {
let db = match session.workspace_db_for_path_mut(path.as_std_path()) {
let db = match session.project_db_for_path_mut(path.as_std_path()) {
Some(db) => db,
None => session.default_workspace_db_mut(),
None => session.default_project_db_mut(),
};
db.apply_changes(vec![ChangeEvent::Opened(path)], None);
}
AnySystemPath::SystemVirtual(virtual_path) => {
let db = session.default_workspace_db_mut();
let db = session.default_project_db_mut();
db.files().virtual_file(db, &virtual_path);
}
}

View File

@@ -11,7 +11,7 @@ use crate::edit::ToRangeExt;
use crate::server::api::traits::{BackgroundDocumentRequestHandler, RequestHandler};
use crate::server::{client::Notifier, Result};
use crate::session::DocumentSnapshot;
use red_knot_workspace::db::{Db, RootDatabase};
use red_knot_workspace::db::{Db, ProjectDatabase};
use ruff_db::diagnostic::Severity;
use ruff_db::source::{line_index, source_text};
@@ -28,7 +28,7 @@ impl BackgroundDocumentRequestHandler for DocumentDiagnosticRequestHandler {
fn run_with_snapshot(
snapshot: DocumentSnapshot,
db: RootDatabase,
db: ProjectDatabase,
_notifier: Notifier,
_params: DocumentDiagnosticParams,
) -> Result<DocumentDiagnosticReportResult> {
@@ -46,7 +46,7 @@ impl BackgroundDocumentRequestHandler for DocumentDiagnosticRequestHandler {
}
}
fn compute_diagnostics(snapshot: &DocumentSnapshot, db: &RootDatabase) -> Vec<Diagnostic> {
fn compute_diagnostics(snapshot: &DocumentSnapshot, db: &ProjectDatabase) -> Vec<Diagnostic> {
let Some(file) = snapshot.file(db) else {
tracing::info!(
"No file found for snapshot for `{}`",

View File

@@ -5,7 +5,7 @@ use crate::session::{DocumentSnapshot, Session};
use lsp_types::notification::Notification as LSPNotification;
use lsp_types::request::Request;
use red_knot_workspace::db::RootDatabase;
use red_knot_workspace::db::ProjectDatabase;
/// A supertrait for any server request handler.
pub(super) trait RequestHandler {
@@ -34,7 +34,7 @@ pub(super) trait BackgroundDocumentRequestHandler: RequestHandler {
fn run_with_snapshot(
snapshot: DocumentSnapshot,
db: RootDatabase,
db: ProjectDatabase,
notifier: Notifier,
params: <<Self as RequestHandler>::RequestType as Request>::Params,
) -> super::Result<<<Self as RequestHandler>::RequestType as Request>::Result>;

View File

@@ -8,8 +8,8 @@ use std::sync::Arc;
use anyhow::anyhow;
use lsp_types::{ClientCapabilities, TextDocumentContentChangeEvent, Url};
use red_knot_workspace::db::RootDatabase;
use red_knot_workspace::workspace::WorkspaceMetadata;
use red_knot_workspace::db::ProjectDatabase;
use red_knot_workspace::project::ProjectMetadata;
use ruff_db::files::{system_path_to_file, File};
use ruff_db::system::SystemPath;
use ruff_db::Db;
@@ -28,7 +28,7 @@ pub(crate) mod index;
mod settings;
// TODO(dhruvmanila): In general, the server shouldn't use any salsa queries directly and instead
// should use methods on `RootDatabase`.
// should use methods on `ProjectDatabase`.
/// The global state for the LSP
pub struct Session {
@@ -41,8 +41,9 @@ pub struct Session {
/// [`index_mut`]: Session::index_mut
index: Option<Arc<index::Index>>,
/// Maps workspace root paths to their respective databases.
workspaces: BTreeMap<PathBuf, RootDatabase>,
/// Maps workspace folders to their respective project databases.
projects_by_workspace_folder: BTreeMap<PathBuf, ProjectDatabase>,
/// The global position encoding, negotiated during LSP initialization.
position_encoding: PositionEncoding,
/// Tracks what LSP features the client supports and doesn't support.
@@ -68,14 +69,14 @@ impl Session {
let system = LSPSystem::new(index.clone());
// TODO(dhruvmanila): Get the values from the client settings
let metadata = WorkspaceMetadata::discover(system_path, &system, None)?;
let metadata = ProjectMetadata::discover(system_path, &system, None)?;
// TODO(micha): Handle the case where the program settings are incorrect more gracefully.
workspaces.insert(path, RootDatabase::new(metadata, system)?);
workspaces.insert(path, ProjectDatabase::new(metadata, system)?);
}
Ok(Self {
position_encoding,
workspaces,
projects_by_workspace_folder: workspaces,
index: Some(index),
resolved_client_capabilities: Arc::new(ResolvedClientCapabilities::new(
client_capabilities,
@@ -87,38 +88,41 @@ impl Session {
// and `default_workspace_db_mut` but the borrow checker doesn't allow that.
// https://github.com/astral-sh/ruff/pull/13041#discussion_r1726725437
/// Returns a reference to the workspace [`RootDatabase`] corresponding to the given path, if
/// Returns a reference to the project's [`ProjectDatabase`] corresponding to the given path, if
/// any.
pub(crate) fn workspace_db_for_path(&self, path: impl AsRef<Path>) -> Option<&RootDatabase> {
self.workspaces
pub(crate) fn project_db_for_path(&self, path: impl AsRef<Path>) -> Option<&ProjectDatabase> {
self.projects_by_workspace_folder
.range(..=path.as_ref().to_path_buf())
.next_back()
.map(|(_, db)| db)
}
/// Returns a mutable reference to the workspace [`RootDatabase`] corresponding to the given
/// Returns a mutable reference to the project [`ProjectDatabase`] corresponding to the given
/// path, if any.
pub(crate) fn workspace_db_for_path_mut(
pub(crate) fn project_db_for_path_mut(
&mut self,
path: impl AsRef<Path>,
) -> Option<&mut RootDatabase> {
self.workspaces
) -> Option<&mut ProjectDatabase> {
self.projects_by_workspace_folder
.range_mut(..=path.as_ref().to_path_buf())
.next_back()
.map(|(_, db)| db)
}
/// Returns a reference to the default workspace [`RootDatabase`]. The default workspace is the
/// minimum root path in the workspace map.
pub(crate) fn default_workspace_db(&self) -> &RootDatabase {
// SAFETY: Currently, red knot only support a single workspace.
self.workspaces.values().next().unwrap()
/// Returns a reference to the default project [`ProjectDatabase`]. The default project is the
/// minimum root path in the project map.
pub(crate) fn default_project_db(&self) -> &ProjectDatabase {
// SAFETY: Currently, red knot only support a single project.
self.projects_by_workspace_folder.values().next().unwrap()
}
/// Returns a mutable reference to the default workspace [`RootDatabase`].
pub(crate) fn default_workspace_db_mut(&mut self) -> &mut RootDatabase {
// SAFETY: Currently, red knot only support a single workspace.
self.workspaces.values_mut().next().unwrap()
/// Returns a mutable reference to the default project [`ProjectDatabase`].
pub(crate) fn default_project_db_mut(&mut self) -> &mut ProjectDatabase {
// SAFETY: Currently, red knot only support a single project.
self.projects_by_workspace_folder
.values_mut()
.next()
.unwrap()
}
pub fn key_from_url(&self, url: Url) -> DocumentKey {
@@ -187,7 +191,7 @@ impl Session {
fn index_mut(&mut self) -> MutIndexGuard {
let index = self.index.take().unwrap();
for db in self.workspaces.values_mut() {
for db in self.projects_by_workspace_folder.values_mut() {
// Remove the `index` from each database. This drops the count of `Arc<Index>` down to 1
db.system_mut()
.as_any_mut()
@@ -232,7 +236,7 @@ impl Drop for MutIndexGuard<'_> {
fn drop(&mut self) {
if let Some(index) = self.index.take() {
let index = Arc::new(index);
for db in self.session.workspaces.values_mut() {
for db in self.session.projects_by_workspace_folder.values_mut() {
db.system_mut()
.as_any_mut()
.downcast_mut::<LSPSystem>()
@@ -267,7 +271,7 @@ impl DocumentSnapshot {
self.position_encoding
}
pub(crate) fn file(&self, db: &RootDatabase) -> Option<File> {
pub(crate) fn file(&self, db: &ProjectDatabase) -> Option<File> {
match url_to_any_system_path(self.document_ref.file_url()).ok()? {
AnySystemPath::System(path) => system_path_to_file(db, path).ok(),
AnySystemPath::SystemVirtual(virtual_path) => db

View File

@@ -74,7 +74,7 @@ impl Index {
DocumentKey::NotebookCell(url)
} else if Path::new(url.path())
.extension()
.map_or(false, |ext| ext.eq_ignore_ascii_case("ipynb"))
.is_some_and(|ext| ext.eq_ignore_ascii_case("ipynb"))
{
DocumentKey::Notebook(url)
} else {

View File

@@ -38,7 +38,7 @@ impl Db {
Program::from_settings(
&db,
&ProgramSettings {
ProgramSettings {
python_version: PythonVersion::default(),
python_platform: PythonPlatform::default(),
search_paths: SearchPathSettings::new(db.workspace_root.clone()),

View File

@@ -5,6 +5,8 @@ def static_assert(condition: object, msg: LiteralString | None = None) -> None:
# Types
Unknown = object()
AlwaysTruthy = object()
AlwaysFalsy = object()
# Special forms
Not: _SpecialForm

View File

@@ -21,7 +21,7 @@ the project the stubs are for, but instead report them here to typeshed.**
Further documentation on stub files, typeshed, and Python's typing system in
general, can also be found at https://typing.readthedocs.io/en/latest/.
Typeshed supports Python versions 3.8 to 3.13.
Typeshed supports Python versions 3.9 to 3.13.
## Using

View File

@@ -1 +1 @@
2047b820730fdd65d37e6e8efcf29ca9af7ec3e7
101287091cbd71a3305a4fc4a1a8eb5df0e3f6f7

View File

@@ -1,16 +1,19 @@
from typing import Any, SupportsIndex
from typing import Any, Literal, SupportsIndex
from typing_extensions import TypeAlias
_UnboundOp: TypeAlias = Literal[1, 2, 3]
class QueueError(RuntimeError): ...
class QueueNotFoundError(QueueError): ...
def bind(qid: SupportsIndex) -> None: ...
def create(maxsize: SupportsIndex, fmt: SupportsIndex) -> int: ...
def create(maxsize: SupportsIndex, fmt: SupportsIndex, unboundop: _UnboundOp) -> int: ...
def destroy(qid: SupportsIndex) -> None: ...
def get(qid: SupportsIndex) -> tuple[Any, int]: ...
def get(qid: SupportsIndex) -> tuple[Any, int, _UnboundOp | None]: ...
def get_count(qid: SupportsIndex) -> int: ...
def get_maxsize(qid: SupportsIndex) -> int: ...
def get_queue_defaults(qid: SupportsIndex) -> tuple[int]: ...
def get_queue_defaults(qid: SupportsIndex) -> tuple[int, _UnboundOp]: ...
def is_full(qid: SupportsIndex) -> bool: ...
def list_all() -> list[tuple[int, int]]: ...
def put(qid: SupportsIndex, obj: Any, fmt: SupportsIndex) -> None: ...
def list_all() -> list[tuple[int, int, _UnboundOp]]: ...
def put(qid: SupportsIndex, obj: Any, fmt: SupportsIndex, unboundop: _UnboundOp) -> None: ...
def release(qid: SupportsIndex) -> None: ...

View File

@@ -21,7 +21,9 @@ def get_main() -> tuple[int, int]: ...
def is_running(id: SupportsIndex, *, restrict: bool = False) -> bool: ...
def get_config(id: SupportsIndex, *, restrict: bool = False) -> types.SimpleNamespace: ...
def whence(id: SupportsIndex) -> int: ...
def exec(id: SupportsIndex, code: str, shared: bool | None = None, *, restrict: bool = False) -> None: ...
def exec(
id: SupportsIndex, code: str | types.CodeType | Callable[[], object], shared: bool | None = None, *, restrict: bool = False
) -> None | types.SimpleNamespace: ...
def call(
id: SupportsIndex,
callable: Callable[..., object],

View File

@@ -240,9 +240,7 @@ OP_SINGLE_ECDH_USE: int
OP_NO_COMPRESSION: int
OP_ENABLE_MIDDLEBOX_COMPAT: int
OP_NO_RENEGOTIATION: int
if sys.version_info >= (3, 11):
OP_IGNORE_UNEXPECTED_EOF: int
elif sys.version_info >= (3, 8) and sys.platform == "linux":
if sys.version_info >= (3, 11) or sys.platform == "linux":
OP_IGNORE_UNEXPECTED_EOF: int
if sys.version_info >= (3, 12):
OP_LEGACY_SERVER_CONNECT: int

View File

@@ -159,7 +159,14 @@ def ARRAY(typ: _CT, len: int) -> Array[_CT]: ... # Soft Deprecated, no plans to
if sys.platform == "win32":
def DllCanUnloadNow() -> int: ...
def DllGetClassObject(rclsid: Any, riid: Any, ppv: Any) -> int: ... # TODO not documented
def GetLastError() -> int: ...
# Actually just an instance of _NamedFuncPointer (aka _CDLLFuncPointer),
# but we want to set a more specific __call__
@type_check_only
class _GetLastErrorFunctionType(_NamedFuncPointer):
def __call__(self) -> int: ...
GetLastError: _GetLastErrorFunctionType
# Actually just an instance of _CFunctionType, but we want to set a more
# specific __call__.

View File

@@ -1399,7 +1399,7 @@ def create_server(
address: _Address, *, family: int = ..., backlog: int | None = None, reuse_port: bool = False, dualstack_ipv6: bool = False
) -> socket: ...
# the 5th tuple item is an address
# The 5th tuple item is the socket address, for IP4, IP6, or IP6 if Python is compiled with --disable-ipv6, respectively.
def getaddrinfo(
host: bytes | str | None, port: bytes | str | int | None, family: int = 0, type: int = 0, proto: int = 0, flags: int = 0
) -> list[tuple[AddressFamily, SocketKind, int, str, tuple[str, int] | tuple[str, int, int, int]]]: ...
) -> list[tuple[AddressFamily, SocketKind, int, str, tuple[str, int] | tuple[str, int, int, int] | tuple[int, bytes]]]: ...

View File

@@ -6,7 +6,7 @@ from collections.abc import AsyncGenerator, Callable, Sequence
from io import TextIOWrapper
from types import FrameType, ModuleType, TracebackType
from typing import Any, Final, Literal, NoReturn, Protocol, TextIO, TypeVar, final, type_check_only
from typing_extensions import TypeAlias
from typing_extensions import LiteralString, TypeAlias
_T = TypeVar("_T")
@@ -45,7 +45,7 @@ if sys.version_info >= (3, 10):
path: list[str]
path_hooks: list[Callable[[str], PathEntryFinderProtocol]]
path_importer_cache: dict[str, PathEntryFinderProtocol | None]
platform: str
platform: LiteralString
if sys.version_info >= (3, 9):
platlibdir: str
prefix: str
@@ -393,6 +393,10 @@ if sys.platform == "win32":
def getwindowsversion() -> _WinVersion: ...
def intern(string: str, /) -> str: ...
if sys.version_info >= (3, 13):
def _is_gil_enabled() -> bool: ...
def is_finalizing() -> bool: ...
def breakpointhook(*args: Any, **kwargs: Any) -> Any: ...

View File

@@ -123,7 +123,7 @@ def open(
@overload
def open(
name: StrOrBytesPath | None,
mode: Literal["x", "x:", "a", "a:", "w", "w:"],
mode: Literal["x", "x:", "a", "a:", "w", "w:", "w:tar"],
fileobj: _Fileobj | None = None,
bufsize: int = 10240,
*,
@@ -141,7 +141,7 @@ def open(
def open(
name: StrOrBytesPath | None = None,
*,
mode: Literal["x", "x:", "a", "a:", "w", "w:"],
mode: Literal["x", "x:", "a", "a:", "w", "w:", "w:tar"],
fileobj: _Fileobj | None = None,
bufsize: int = 10240,
format: int | None = ...,

View File

@@ -1,5 +1,5 @@
import socket
from collections.abc import Callable, Sequence
from collections.abc import Callable, MutableSequence, Sequence
from re import Match, Pattern
from types import TracebackType
from typing import Any
@@ -114,7 +114,7 @@ class Telnet:
def mt_interact(self) -> None: ...
def listener(self) -> None: ...
def expect(
self, list: Sequence[Pattern[bytes] | bytes], timeout: float | None = None
self, list: MutableSequence[Pattern[bytes] | bytes] | Sequence[Pattern[bytes]], timeout: float | None = None
) -> tuple[int, Match[bytes] | None, bytes]: ...
def __enter__(self) -> Self: ...
def __exit__(

View File

@@ -80,8 +80,8 @@ class Directory(commondialog.Dialog):
# TODO: command kwarg available on macos
def asksaveasfilename(
*,
confirmoverwrite: bool | None = ...,
defaultextension: str | None = ...,
confirmoverwrite: bool | None = True,
defaultextension: str | None = "",
filetypes: Iterable[tuple[str, str | list[str] | tuple[str, ...]]] | None = ...,
initialdir: StrOrBytesPath | None = ...,
initialfile: StrOrBytesPath | None = ...,
@@ -91,7 +91,7 @@ def asksaveasfilename(
) -> str: ... # can be empty string
def askopenfilename(
*,
defaultextension: str | None = ...,
defaultextension: str | None = "",
filetypes: Iterable[tuple[str, str | list[str] | tuple[str, ...]]] | None = ...,
initialdir: StrOrBytesPath | None = ...,
initialfile: StrOrBytesPath | None = ...,
@@ -101,7 +101,7 @@ def askopenfilename(
) -> str: ... # can be empty string
def askopenfilenames(
*,
defaultextension: str | None = ...,
defaultextension: str | None = "",
filetypes: Iterable[tuple[str, str | list[str] | tuple[str, ...]]] | None = ...,
initialdir: StrOrBytesPath | None = ...,
initialfile: StrOrBytesPath | None = ...,
@@ -110,15 +110,15 @@ def askopenfilenames(
typevariable: StringVar | str | None = ...,
) -> Literal[""] | tuple[str, ...]: ...
def askdirectory(
*, initialdir: StrOrBytesPath | None = ..., mustexist: bool | None = ..., parent: Misc | None = ..., title: str | None = ...
*, initialdir: StrOrBytesPath | None = ..., mustexist: bool | None = False, parent: Misc | None = ..., title: str | None = ...
) -> str: ... # can be empty string
# TODO: If someone actually uses these, overload to have the actual return type of open(..., mode)
def asksaveasfile(
mode: str = "w",
*,
confirmoverwrite: bool | None = ...,
defaultextension: str | None = ...,
confirmoverwrite: bool | None = True,
defaultextension: str | None = "",
filetypes: Iterable[tuple[str, str | list[str] | tuple[str, ...]]] | None = ...,
initialdir: StrOrBytesPath | None = ...,
initialfile: StrOrBytesPath | None = ...,
@@ -129,7 +129,7 @@ def asksaveasfile(
def askopenfile(
mode: str = "r",
*,
defaultextension: str | None = ...,
defaultextension: str | None = "",
filetypes: Iterable[tuple[str, str | list[str] | tuple[str, ...]]] | None = ...,
initialdir: StrOrBytesPath | None = ...,
initialfile: StrOrBytesPath | None = ...,
@@ -140,7 +140,7 @@ def askopenfile(
def askopenfiles(
mode: str = "r",
*,
defaultextension: str | None = ...,
defaultextension: str | None = "",
filetypes: Iterable[tuple[str, str | list[str] | tuple[str, ...]]] | None = ...,
initialdir: StrOrBytesPath | None = ...,
initialfile: StrOrBytesPath | None = ...,

View File

@@ -291,8 +291,8 @@ class ReadOnlySequentialNamedNodeMap:
def length(self) -> int: ...
class Identified:
publicId: Incomplete
systemId: Incomplete
publicId: str | None
systemId: str | None
class DocumentType(Identified, Childless, Node):
nodeType: int
@@ -331,7 +331,7 @@ class Notation(Identified, Childless, Node):
class DOMImplementation(DOMImplementationLS):
def hasFeature(self, feature: str, version: str | None) -> bool: ...
def createDocument(self, namespaceURI: str | None, qualifiedName: str | None, doctype: DocumentType | None) -> Document: ...
def createDocumentType(self, qualifiedName: str | None, publicId: str, systemId: str) -> DocumentType: ...
def createDocumentType(self, qualifiedName: str | None, publicId: str | None, systemId: str | None) -> DocumentType: ...
def getInterface(self, feature: str) -> Self | None: ...
class ElementInfo:

View File

@@ -3,9 +3,9 @@ use std::any::Any;
use js_sys::Error;
use wasm_bindgen::prelude::*;
use red_knot_workspace::db::{Db, RootDatabase};
use red_knot_workspace::workspace::settings::Configuration;
use red_knot_workspace::workspace::WorkspaceMetadata;
use red_knot_workspace::db::{Db, ProjectDatabase};
use red_knot_workspace::project::settings::Configuration;
use red_knot_workspace::project::ProjectMetadata;
use ruff_db::diagnostic::Diagnostic;
use ruff_db::files::{system_path_to_file, File};
use ruff_db::system::walk_directory::WalkDirectoryBuilder;
@@ -33,7 +33,7 @@ pub fn run() {
#[wasm_bindgen]
pub struct Workspace {
db: RootDatabase,
db: ProjectDatabase,
system: WasmSystem,
}
@@ -42,7 +42,7 @@ impl Workspace {
#[wasm_bindgen(constructor)]
pub fn new(root: &str, settings: &Settings) -> Result<Workspace, Error> {
let system = WasmSystem::new(SystemPath::new(root));
let workspace = WorkspaceMetadata::discover(
let workspace = ProjectMetadata::discover(
SystemPath::new(root),
&system,
Some(&Configuration {
@@ -52,7 +52,7 @@ impl Workspace {
)
.map_err(into_error)?;
let db = RootDatabase::new(workspace, system.clone()).map_err(into_error)?;
let db = ProjectDatabase::new(workspace, system.clone()).map_err(into_error)?;
Ok(Self { db, system })
}
@@ -67,7 +67,7 @@ impl Workspace {
let file = system_path_to_file(&self.db, path).expect("File to exist");
file.sync(&mut self.db);
self.db.workspace().open_file(&mut self.db, file);
self.db.project().open_file(&mut self.db, file);
Ok(FileHandle {
file,
@@ -95,7 +95,7 @@ impl Workspace {
pub fn close_file(&mut self, file_id: &FileHandle) -> Result<(), Error> {
let file = file_id.file;
self.db.workspace().close_file(&mut self.db, file);
self.db.project().close_file(&mut self.db, file);
self.system
.fs
.remove_file(&file_id.path)

View File

@@ -1,7 +1,7 @@
use std::panic::RefUnwindSafe;
use std::sync::Arc;
use crate::workspace::{check_file, Workspace, WorkspaceMetadata};
use crate::project::{check_file, Project, ProjectMetadata};
use crate::DEFAULT_LINT_REGISTRY;
use red_knot_python_semantic::lint::{LintRegistry, RuleSelection};
use red_knot_python_semantic::{Db as SemanticDb, Program};
@@ -17,28 +17,28 @@ mod changes;
#[salsa::db]
pub trait Db: SemanticDb + Upcast<dyn SemanticDb> {
fn workspace(&self) -> Workspace;
fn project(&self) -> Project;
}
#[salsa::db]
#[derive(Clone)]
pub struct RootDatabase {
workspace: Option<Workspace>,
storage: salsa::Storage<RootDatabase>,
pub struct ProjectDatabase {
project: Option<Project>,
storage: salsa::Storage<ProjectDatabase>,
files: Files,
system: Arc<dyn System + Send + Sync + RefUnwindSafe>,
rule_selection: Arc<RuleSelection>,
}
impl RootDatabase {
pub fn new<S>(workspace: WorkspaceMetadata, system: S) -> anyhow::Result<Self>
impl ProjectDatabase {
pub fn new<S>(project_metadata: ProjectMetadata, system: S) -> anyhow::Result<Self>
where
S: System + 'static + Send + Sync + RefUnwindSafe,
{
let rule_selection = RuleSelection::from_registry(&DEFAULT_LINT_REGISTRY);
let mut db = Self {
workspace: None,
project: None,
storage: salsa::Storage::default(),
files: Files::default(),
system: Arc::new(system),
@@ -46,16 +46,17 @@ impl RootDatabase {
};
// Initialize the `Program` singleton
Program::from_settings(&db, workspace.settings().program())?;
let program_settings = project_metadata.to_program_settings();
Program::from_settings(&db, program_settings)?;
db.workspace = Some(Workspace::from_metadata(&db, workspace));
db.project = Some(Project::from_metadata(&db, project_metadata));
Ok(db)
}
/// Checks all open files in the workspace and its dependencies.
/// Checks all open files in the project and its dependencies.
pub fn check(&self) -> Result<Vec<Box<dyn Diagnostic>>, Cancelled> {
self.with_db(|db| db.workspace().check(db))
self.with_db(|db| db.project().check(db))
}
pub fn check_file(&self, file: File) -> Result<Vec<Box<dyn Diagnostic>>, Cancelled> {
@@ -77,13 +78,13 @@ impl RootDatabase {
pub(crate) fn with_db<F, T>(&self, f: F) -> Result<T, Cancelled>
where
F: FnOnce(&RootDatabase) -> T + std::panic::UnwindSafe,
F: FnOnce(&ProjectDatabase) -> T + std::panic::UnwindSafe,
{
Cancelled::catch(|| f(self))
}
}
impl Upcast<dyn SemanticDb> for RootDatabase {
impl Upcast<dyn SemanticDb> for ProjectDatabase {
fn upcast(&self) -> &(dyn SemanticDb + 'static) {
self
}
@@ -93,7 +94,7 @@ impl Upcast<dyn SemanticDb> for RootDatabase {
}
}
impl Upcast<dyn SourceDb> for RootDatabase {
impl Upcast<dyn SourceDb> for ProjectDatabase {
fn upcast(&self) -> &(dyn SourceDb + 'static) {
self
}
@@ -104,13 +105,13 @@ impl Upcast<dyn SourceDb> for RootDatabase {
}
#[salsa::db]
impl SemanticDb for RootDatabase {
impl SemanticDb for ProjectDatabase {
fn is_file_open(&self, file: File) -> bool {
let Some(workspace) = &self.workspace else {
let Some(project) = &self.project else {
return false;
};
workspace.is_file_open(self, file)
project.is_file_open(self, file)
}
fn rule_selection(&self) -> &RuleSelection {
@@ -123,7 +124,7 @@ impl SemanticDb for RootDatabase {
}
#[salsa::db]
impl SourceDb for RootDatabase {
impl SourceDb for ProjectDatabase {
fn vendored(&self) -> &VendoredFileSystem {
red_knot_vendored::file_system()
}
@@ -138,7 +139,7 @@ impl SourceDb for RootDatabase {
}
#[salsa::db]
impl salsa::Database for RootDatabase {
impl salsa::Database for ProjectDatabase {
fn salsa_event(&self, event: &dyn Fn() -> Event) {
if !tracing::enabled!(tracing::Level::TRACE) {
return;
@@ -154,9 +155,9 @@ impl salsa::Database for RootDatabase {
}
#[salsa::db]
impl Db for RootDatabase {
fn workspace(&self) -> Workspace {
self.workspace.unwrap()
impl Db for ProjectDatabase {
fn project(&self) -> Project {
self.project.unwrap()
}
}
@@ -174,7 +175,7 @@ pub(crate) mod tests {
use ruff_db::{Db as SourceDb, Upcast};
use crate::db::Db;
use crate::workspace::{Workspace, WorkspaceMetadata};
use crate::project::{Project, ProjectMetadata};
use crate::DEFAULT_LINT_REGISTRY;
#[salsa::db]
@@ -186,11 +187,11 @@ pub(crate) mod tests {
system: TestSystem,
vendored: VendoredFileSystem,
rule_selection: RuleSelection,
workspace: Option<Workspace>,
project: Option<Project>,
}
impl TestDb {
pub(crate) fn new(workspace: WorkspaceMetadata) -> Self {
pub(crate) fn new(project: ProjectMetadata) -> Self {
let mut db = Self {
storage: salsa::Storage::default(),
system: TestSystem::default(),
@@ -198,11 +199,11 @@ pub(crate) mod tests {
files: Files::default(),
events: Arc::default(),
rule_selection: RuleSelection::from_registry(&DEFAULT_LINT_REGISTRY),
workspace: None,
project: None,
};
let workspace = Workspace::from_metadata(&db, workspace);
db.workspace = Some(workspace);
let project = Project::from_metadata(&db, project);
db.project = Some(project);
db
}
}
@@ -280,8 +281,8 @@ pub(crate) mod tests {
#[salsa::db]
impl Db for TestDb {
fn workspace(&self) -> Workspace {
self.workspace.unwrap()
fn project(&self) -> Project {
self.project.unwrap()
}
}

View File

@@ -1,8 +1,7 @@
use crate::db::{Db, RootDatabase};
use crate::watch;
use crate::db::{Db, ProjectDatabase};
use crate::project::settings::Configuration;
use crate::project::{Project, ProjectMetadata};
use crate::watch::{ChangeEvent, CreatedKind, DeletedKind};
use crate::workspace::settings::Configuration;
use crate::workspace::{Workspace, WorkspaceMetadata};
use red_knot_python_semantic::Program;
use ruff_db::files::{system_path_to_file, File, Files};
use ruff_db::system::walk_directory::WalkState;
@@ -10,25 +9,24 @@ use ruff_db::system::SystemPath;
use ruff_db::Db as _;
use rustc_hash::FxHashSet;
impl RootDatabase {
impl ProjectDatabase {
#[tracing::instrument(level = "debug", skip(self, changes, base_configuration))]
pub fn apply_changes(
&mut self,
changes: Vec<watch::ChangeEvent>,
changes: Vec<ChangeEvent>,
base_configuration: Option<&Configuration>,
) {
let mut workspace = self.workspace();
let workspace_path = workspace.root(self).to_path_buf();
let mut project = self.project();
let project_path = project.root(self).to_path_buf();
let program = Program::get(self);
let custom_stdlib_versions_path = program
.custom_stdlib_search_path(self)
.map(|path| path.join("VERSIONS"));
let mut workspace_change = false;
// Are there structural changes to the project
let mut project_changed = false;
// Changes to a custom stdlib path's VERSIONS
let mut custom_stdlib_change = false;
// Packages that need reloading
let mut changed_packages = FxHashSet::default();
// Paths that were added
let mut added_paths = FxHashSet::default();
@@ -36,13 +34,13 @@ impl RootDatabase {
let mut synced_files = FxHashSet::default();
let mut synced_recursively = FxHashSet::default();
let mut sync_path = |db: &mut RootDatabase, path: &SystemPath| {
let mut sync_path = |db: &mut ProjectDatabase, path: &SystemPath| {
if synced_files.insert(path.to_path_buf()) {
File::sync_path(db, path);
}
};
let mut sync_recursively = |db: &mut RootDatabase, path: &SystemPath| {
let mut sync_recursively = |db: &mut ProjectDatabase, path: &SystemPath| {
if synced_recursively.insert(path.to_path_buf()) {
Files::sync_recursively(db, path);
}
@@ -54,19 +52,8 @@ impl RootDatabase {
path.file_name(),
Some(".gitignore" | ".ignore" | "ruff.toml" | ".ruff.toml" | "pyproject.toml")
) {
// Changes to ignore files or settings can change the workspace structure or add/remove files
// from packages.
if let Some(package) = workspace.package(self, path) {
if package.root(self) == workspace.root(self)
|| matches!(change, ChangeEvent::Deleted { .. })
{
workspace_change = true;
}
changed_packages.insert(package);
} else {
workspace_change = true;
}
// Changes to ignore files or settings can change the project structure or add/remove files.
project_changed = true;
continue;
}
@@ -77,10 +64,11 @@ impl RootDatabase {
}
match change {
watch::ChangeEvent::Changed { path, kind: _ }
| watch::ChangeEvent::Opened(path) => sync_path(self, &path),
ChangeEvent::Changed { path, kind: _ } | ChangeEvent::Opened(path) => {
sync_path(self, &path);
}
watch::ChangeEvent::Created { kind, path } => {
ChangeEvent::Created { kind, path } => {
match kind {
CreatedKind::File => sync_path(self, &path),
CreatedKind::Directory | CreatedKind::Any => {
@@ -97,7 +85,7 @@ impl RootDatabase {
}
}
watch::ChangeEvent::Deleted { kind, path } => {
ChangeEvent::Deleted { kind, path } => {
let is_file = match kind {
DeletedKind::File => true,
DeletedKind::Directory => {
@@ -113,10 +101,8 @@ impl RootDatabase {
if is_file {
sync_path(self, &path);
if let Some(package) = workspace.package(self, &path) {
if let Some(file) = self.files().try_system(self, &path) {
package.remove_file(self, file);
}
if let Some(file) = self.files().try_system(self, &path) {
project.remove_file(self, file);
}
} else {
sync_recursively(self, &path);
@@ -128,69 +114,68 @@ impl RootDatabase {
custom_stdlib_change = true;
}
if let Some(package) = workspace.package(self, &path) {
changed_packages.insert(package);
} else {
workspace_change = true;
}
// Perform a full-reload in case the deleted directory contained the pyproject.toml.
// We may want to make this more clever in the future, to e.g. iterate over the
// indexed files and remove the once that start with the same path, unless
// the deleted path is the project configuration.
project_changed = true;
}
}
watch::ChangeEvent::CreatedVirtual(path)
| watch::ChangeEvent::ChangedVirtual(path) => {
ChangeEvent::CreatedVirtual(path) | ChangeEvent::ChangedVirtual(path) => {
File::sync_virtual_path(self, &path);
}
watch::ChangeEvent::DeletedVirtual(path) => {
ChangeEvent::DeletedVirtual(path) => {
if let Some(virtual_file) = self.files().try_virtual_file(&path) {
virtual_file.close(self);
}
}
watch::ChangeEvent::Rescan => {
workspace_change = true;
ChangeEvent::Rescan => {
project_changed = true;
Files::sync_all(self);
break;
}
}
}
if workspace_change {
match WorkspaceMetadata::discover(&workspace_path, self.system(), base_configuration) {
if project_changed {
match ProjectMetadata::discover(&project_path, self.system(), base_configuration) {
Ok(metadata) => {
if metadata.root() == workspace.root(self) {
tracing::debug!("Reloading workspace after structural change");
// TODO: Handle changes in the program settings.
workspace.reload(self, metadata);
let program_settings = metadata.to_program_settings();
let program = Program::get(self);
if let Err(error) = program.update_from_settings(self, program_settings) {
tracing::error!("Failed to update the program settings, keeping the old program settings: {error}");
};
if metadata.root() == project.root(self) {
tracing::debug!("Reloading project after structural change");
project.reload(self, metadata);
} else {
tracing::debug!("Replace workspace after structural change");
workspace = Workspace::from_metadata(self, metadata);
self.workspace = Some(workspace);
tracing::debug!("Replace project after structural change");
project = Project::from_metadata(self, metadata);
self.project = Some(project);
}
}
Err(error) => {
tracing::error!(
"Failed to load workspace, keeping old workspace configuration: {error}"
"Failed to load project, keeping old project configuration: {error}"
);
}
}
return;
} else if custom_stdlib_change {
let search_paths = workspace.search_path_settings(self).clone();
let search_paths = project.metadata(self).to_program_settings().search_paths;
if let Err(error) = program.update_search_paths(self, &search_paths) {
tracing::error!("Failed to set the new search paths: {error}");
}
}
let mut added_paths = added_paths.into_iter().filter(|path| {
let Some(package) = workspace.package(self, path) else {
return false;
};
// Skip packages that need reloading
!changed_packages.contains(&package)
});
let mut added_paths = added_paths.into_iter();
// Use directory walking to discover newly added files.
if let Some(path) = added_paths.next() {
@@ -221,18 +206,12 @@ impl RootDatabase {
});
for path in added_paths.into_inner().unwrap() {
let package = workspace.package(self, &path);
let file = system_path_to_file(self, &path);
if let (Some(package), Ok(file)) = (package, file) {
package.add_file(self, file);
if let Ok(file) = file {
project.add_file(self, file);
}
}
}
// Reload
for package in changed_packages {
package.reload_files(self);
}
}
}

View File

@@ -2,8 +2,8 @@ use red_knot_python_semantic::lint::{LintRegistry, LintRegistryBuilder};
use red_knot_python_semantic::register_lints;
pub mod db;
pub mod project;
pub mod watch;
pub mod workspace;
pub static DEFAULT_LINT_REGISTRY: std::sync::LazyLock<LintRegistry> =
std::sync::LazyLock::new(default_lints_registry);

View File

@@ -0,0 +1,459 @@
#![allow(clippy::ref_option)]
use crate::db::Db;
use crate::db::ProjectDatabase;
use crate::project::files::{Index, Indexed, IndexedFiles, IndexedIter};
pub use metadata::{ProjectDiscoveryError, ProjectMetadata};
use red_knot_python_semantic::types::check_types;
use ruff_db::diagnostic::{Diagnostic, DiagnosticId, ParseDiagnostic, Severity};
use ruff_db::parsed::parsed_module;
use ruff_db::source::{source_text, SourceTextError};
use ruff_db::system::FileType;
use ruff_db::{
files::{system_path_to_file, File},
system::{walk_directory::WalkState, SystemPath},
};
use ruff_python_ast::PySourceType;
use ruff_text_size::TextRange;
use rustc_hash::{FxBuildHasher, FxHashSet};
use salsa::{Durability, Setter as _};
use std::borrow::Cow;
use std::sync::Arc;
mod files;
mod metadata;
mod pyproject;
pub mod settings;
/// The project as a Salsa ingredient.
///
/// ## How is a project different from a program?
/// There are two (related) motivations:
///
/// 1. Program is defined in `ruff_db` and it can't reference the settings types for the linter and formatter
/// without introducing a cyclic dependency. The project is defined in a higher level crate
/// where it can reference these setting types.
/// 2. Running `ruff check` with different target versions results in different programs (settings) but
/// it remains the same project. That's why program is a narrowed view of the project only
/// holding on to the most fundamental settings required for checking.
#[salsa::input]
pub struct Project {
/// The files that are open in the project.
///
/// Setting the open files to a non-`None` value changes `check` to only check the
/// open files rather than all files in the project.
#[return_ref]
#[default]
open_fileset: Option<Arc<FxHashSet<File>>>,
/// The first-party files of this project.
#[default]
#[return_ref]
file_set: IndexedFiles,
/// The metadata describing the project, including the unresolved configuration.
#[return_ref]
pub metadata: ProjectMetadata,
}
impl Project {
pub fn from_metadata(db: &dyn Db, metadata: ProjectMetadata) -> Self {
Project::builder(metadata)
.durability(Durability::MEDIUM)
.open_fileset_durability(Durability::LOW)
.file_set_durability(Durability::LOW)
.new(db)
}
pub fn root(self, db: &dyn Db) -> &SystemPath {
self.metadata(db).root()
}
pub fn name(self, db: &dyn Db) -> &str {
self.metadata(db).name()
}
pub fn reload(self, db: &mut dyn Db, metadata: ProjectMetadata) {
tracing::debug!("Reloading project");
assert_eq!(self.root(db), metadata.root());
if &metadata != self.metadata(db) {
self.set_metadata(db).to(metadata);
}
self.reload_files(db);
}
/// Checks all open files in the project and its dependencies.
pub fn check(self, db: &ProjectDatabase) -> Vec<Box<dyn Diagnostic>> {
let project_span = tracing::debug_span!("Project::check");
let _span = project_span.enter();
tracing::debug!("Checking project '{name}'", name = self.name(db));
let result = Arc::new(std::sync::Mutex::new(Vec::new()));
let inner_result = Arc::clone(&result);
let db = db.clone();
let project_span = project_span.clone();
rayon::scope(move |scope| {
let files = ProjectFiles::new(&db, self);
for file in &files {
let result = inner_result.clone();
let db = db.clone();
let project_span = project_span.clone();
scope.spawn(move |_| {
let check_file_span = tracing::debug_span!(parent: &project_span, "check_file", file=%file.path(&db));
let _entered = check_file_span.entered();
let file_diagnostics = check_file(&db, file);
result.lock().unwrap().extend(file_diagnostics);
});
}
});
Arc::into_inner(result).unwrap().into_inner().unwrap()
}
/// Opens a file in the project.
///
/// This changes the behavior of `check` to only check the open files rather than all files in the project.
pub fn open_file(self, db: &mut dyn Db, file: File) {
tracing::debug!("Opening file `{}`", file.path(db));
let mut open_files = self.take_open_files(db);
open_files.insert(file);
self.set_open_files(db, open_files);
}
/// Closes a file in the project.
pub fn close_file(self, db: &mut dyn Db, file: File) -> bool {
tracing::debug!("Closing file `{}`", file.path(db));
let mut open_files = self.take_open_files(db);
let removed = open_files.remove(&file);
if removed {
self.set_open_files(db, open_files);
}
removed
}
/// Returns the open files in the project or `None` if the entire project should be checked.
pub fn open_files(self, db: &dyn Db) -> Option<&FxHashSet<File>> {
self.open_fileset(db).as_deref()
}
/// Sets the open files in the project.
///
/// This changes the behavior of `check` to only check the open files rather than all files in the project.
#[tracing::instrument(level = "debug", skip(self, db))]
pub fn set_open_files(self, db: &mut dyn Db, open_files: FxHashSet<File>) {
tracing::debug!("Set open project files (count: {})", open_files.len());
self.set_open_fileset(db).to(Some(Arc::new(open_files)));
}
/// This takes the open files from the project and returns them.
///
/// This changes the behavior of `check` to check all files in the project instead of just the open files.
fn take_open_files(self, db: &mut dyn Db) -> FxHashSet<File> {
tracing::debug!("Take open project files");
// Salsa will cancel any pending queries and remove its own reference to `open_files`
// so that the reference counter to `open_files` now drops to 1.
let open_files = self.set_open_fileset(db).to(None);
if let Some(open_files) = open_files {
Arc::try_unwrap(open_files).unwrap()
} else {
FxHashSet::default()
}
}
/// Returns `true` if the file is open in the project.
///
/// A file is considered open when:
/// * explicitly set as an open file using [`open_file`](Self::open_file)
/// * It has a [`SystemPath`] and belongs to a package's `src` files
/// * It has a [`SystemVirtualPath`](ruff_db::system::SystemVirtualPath)
pub fn is_file_open(self, db: &dyn Db, file: File) -> bool {
if let Some(open_files) = self.open_files(db) {
open_files.contains(&file)
} else if file.path(db).is_system_path() {
self.contains_file(db, file)
} else {
file.path(db).is_system_virtual_path()
}
}
/// Returns `true` if `file` is a first-party file part of this package.
pub fn contains_file(self, db: &dyn Db, file: File) -> bool {
self.files(db).contains(&file)
}
#[tracing::instrument(level = "debug", skip(db))]
pub fn remove_file(self, db: &mut dyn Db, file: File) {
tracing::debug!(
"Removing file `{}` from project `{}`",
file.path(db),
self.name(db)
);
let Some(mut index) = IndexedFiles::indexed_mut(db, self) else {
return;
};
index.remove(file);
}
pub fn add_file(self, db: &mut dyn Db, file: File) {
tracing::debug!(
"Adding file `{}` to project `{}`",
file.path(db),
self.name(db)
);
let Some(mut index) = IndexedFiles::indexed_mut(db, self) else {
return;
};
index.insert(file);
}
/// Returns the files belonging to this project.
pub fn files(self, db: &dyn Db) -> Indexed<'_> {
let files = self.file_set(db);
let indexed = match files.get() {
Index::Lazy(vacant) => {
let _entered =
tracing::debug_span!("Project::index_files", package = %self.name(db))
.entered();
let files = discover_project_files(db, self);
tracing::info!("Found {} files in project `{}`", files.len(), self.name(db));
vacant.set(files)
}
Index::Indexed(indexed) => indexed,
};
indexed
}
pub fn reload_files(self, db: &mut dyn Db) {
tracing::debug!("Reloading files for project `{}`", self.name(db));
if !self.file_set(db).is_lazy() {
// Force a re-index of the files in the next revision.
self.set_file_set(db).to(IndexedFiles::lazy());
}
}
}
pub(super) fn check_file(db: &dyn Db, file: File) -> Vec<Box<dyn Diagnostic>> {
let mut diagnostics: Vec<Box<dyn Diagnostic>> = Vec::new();
// Abort checking if there are IO errors.
let source = source_text(db.upcast(), file);
if let Some(read_error) = source.read_error() {
diagnostics.push(Box::new(IOErrorDiagnostic {
file,
error: read_error.clone(),
}));
return diagnostics;
}
let parsed = parsed_module(db.upcast(), file);
diagnostics.extend(parsed.errors().iter().map(|error| {
let diagnostic: Box<dyn Diagnostic> = Box::new(ParseDiagnostic::new(file, error.clone()));
diagnostic
}));
diagnostics.extend(check_types(db.upcast(), file).iter().map(|diagnostic| {
let boxed: Box<dyn Diagnostic> = Box::new(diagnostic.clone());
boxed
}));
diagnostics.sort_unstable_by_key(|diagnostic| diagnostic.range().unwrap_or_default().start());
diagnostics
}
fn discover_project_files(db: &dyn Db, project: Project) -> FxHashSet<File> {
let paths = std::sync::Mutex::new(Vec::new());
db.system().walk_directory(project.root(db)).run(|| {
Box::new(|entry| {
match entry {
Ok(entry) => {
// Skip over any non python files to avoid creating too many entries in `Files`.
match entry.file_type() {
FileType::File => {
if entry
.path()
.extension()
.and_then(PySourceType::try_from_extension)
.is_some()
{
let mut paths = paths.lock().unwrap();
paths.push(entry.into_path());
}
}
FileType::Directory | FileType::Symlink => {}
}
}
Err(error) => {
// TODO Handle error
tracing::error!("Failed to walk path: {error}");
}
}
WalkState::Continue
})
});
let paths = paths.into_inner().unwrap();
let mut files = FxHashSet::with_capacity_and_hasher(paths.len(), FxBuildHasher);
for path in paths {
// If this returns `None`, then the file was deleted between the `walk_directory` call and now.
// We can ignore this.
if let Ok(file) = system_path_to_file(db.upcast(), &path) {
files.insert(file);
}
}
files
}
#[derive(Debug)]
enum ProjectFiles<'a> {
OpenFiles(&'a FxHashSet<File>),
Indexed(Indexed<'a>),
}
impl<'a> ProjectFiles<'a> {
fn new(db: &'a dyn Db, project: Project) -> Self {
if let Some(open_files) = project.open_files(db) {
ProjectFiles::OpenFiles(open_files)
} else {
ProjectFiles::Indexed(project.files(db))
}
}
}
impl<'a> IntoIterator for &'a ProjectFiles<'a> {
type Item = File;
type IntoIter = ProjectFilesIter<'a>;
fn into_iter(self) -> Self::IntoIter {
match self {
ProjectFiles::OpenFiles(files) => ProjectFilesIter::OpenFiles(files.iter()),
ProjectFiles::Indexed(indexed) => ProjectFilesIter::Indexed {
files: indexed.into_iter(),
},
}
}
}
enum ProjectFilesIter<'db> {
OpenFiles(std::collections::hash_set::Iter<'db, File>),
Indexed { files: IndexedIter<'db> },
}
impl Iterator for ProjectFilesIter<'_> {
type Item = File;
fn next(&mut self) -> Option<Self::Item> {
match self {
ProjectFilesIter::OpenFiles(files) => files.next().copied(),
ProjectFilesIter::Indexed { files } => files.next(),
}
}
}
#[derive(Debug)]
pub struct IOErrorDiagnostic {
file: File,
error: SourceTextError,
}
impl Diagnostic for IOErrorDiagnostic {
fn id(&self) -> DiagnosticId {
DiagnosticId::Io
}
fn message(&self) -> Cow<str> {
self.error.to_string().into()
}
fn file(&self) -> File {
self.file
}
fn range(&self) -> Option<TextRange> {
None
}
fn severity(&self) -> Severity {
Severity::Error
}
}
#[cfg(test)]
mod tests {
use crate::db::tests::TestDb;
use crate::project::{check_file, ProjectMetadata};
use red_knot_python_semantic::types::check_types;
use ruff_db::diagnostic::Diagnostic;
use ruff_db::files::system_path_to_file;
use ruff_db::source::source_text;
use ruff_db::system::{DbWithTestSystem, SystemPath, SystemPathBuf};
use ruff_db::testing::assert_function_query_was_not_run;
use ruff_python_ast::name::Name;
#[test]
fn check_file_skips_type_checking_when_file_cant_be_read() -> ruff_db::system::Result<()> {
let project = ProjectMetadata::new(Name::new_static("test"), SystemPathBuf::from("/"));
let mut db = TestDb::new(project);
let path = SystemPath::new("test.py");
db.write_file(path, "x = 10")?;
let file = system_path_to_file(&db, path).unwrap();
// Now the file gets deleted before we had a chance to read its source text.
db.memory_file_system().remove_file(path)?;
file.sync(&mut db);
assert_eq!(source_text(&db, file).as_str(), "");
assert_eq!(
check_file(&db, file)
.into_iter()
.map(|diagnostic| diagnostic.message().into_owned())
.collect::<Vec<_>>(),
vec!["Failed to read file: No such file or directory".to_string()]
);
let events = db.take_salsa_events();
assert_function_query_was_not_run(&db, check_types, file, &events);
// The user now creates a new file with an empty text. The source text
// content returned by `source_text` remains unchanged, but the diagnostics should get updated.
db.write_file(path, "").unwrap();
assert_eq!(source_text(&db, file).as_str(), "");
assert_eq!(
check_file(&db, file)
.into_iter()
.map(|diagnostic| diagnostic.message().into_owned())
.collect::<Vec<_>>(),
vec![] as Vec<String>
);
Ok(())
}
}

View File

@@ -8,12 +8,12 @@ use salsa::Setter;
use ruff_db::files::File;
use crate::db::Db;
use crate::workspace::Package;
use crate::project::Project;
/// Cheap cloneable hash set of files.
type FileSet = Arc<FxHashSet<File>>;
/// The indexed files of a package.
/// The indexed files of a project.
///
/// The indexing happens lazily, but the files are then cached for subsequent reads.
///
@@ -24,11 +24,11 @@ type FileSet = Arc<FxHashSet<File>>;
/// the indexed files must go through `IndexedMut`, which uses the Salsa setter `package.set_file_set` to
/// ensure that Salsa always knows when the set of indexed files have changed.
#[derive(Debug)]
pub struct PackageFiles {
pub struct IndexedFiles {
state: std::sync::Mutex<State>,
}
impl PackageFiles {
impl IndexedFiles {
pub fn lazy() -> Self {
Self {
state: std::sync::Mutex::new(State::Lazy),
@@ -60,7 +60,7 @@ impl PackageFiles {
/// Returns a mutable view on the index that allows cheap in-place mutations.
///
/// The changes are automatically written back to the database once the view is dropped.
pub(super) fn indexed_mut(db: &mut dyn Db, package: Package) -> Option<IndexedMut> {
pub(super) fn indexed_mut(db: &mut dyn Db, project: Project) -> Option<IndexedMut> {
// Calling `zalsa_mut` cancels all pending salsa queries. This ensures that there are no pending
// reads to the file set.
// TODO: Use a non-internal API instead https://salsa.zulipchat.com/#narrow/stream/333573-salsa-3.2E0/topic/Expose.20an.20API.20to.20cancel.20other.20queries
@@ -79,7 +79,7 @@ impl PackageFiles {
// all clones must have been dropped at this point and the `Indexed`
// can't outlive the database (constrained by the `db` lifetime).
let state = {
let files = package.file_set(db);
let files = project.file_set(db);
let mut locked = files.state.lock().unwrap();
std::mem::replace(&mut *locked, State::Lazy)
};
@@ -93,14 +93,14 @@ impl PackageFiles {
Some(IndexedMut {
db: Some(db),
package,
project,
files: indexed,
did_change: false,
})
}
}
impl Default for PackageFiles {
impl Default for IndexedFiles {
fn default() -> Self {
Self::lazy()
}
@@ -142,7 +142,7 @@ impl<'db> LazyFiles<'db> {
/// The indexed files of a package.
///
/// Note: This type is intentionally non-cloneable. Making it cloneable requires
/// revisiting the locking behavior in [`PackageFiles::indexed_mut`].
/// revisiting the locking behavior in [`IndexedFiles::indexed_mut`].
#[derive(Debug, PartialEq, Eq)]
pub struct Indexed<'db> {
files: FileSet,
@@ -169,13 +169,13 @@ impl<'a> IntoIterator for &'a Indexed<'_> {
}
}
/// A Mutable view of a package's indexed files.
/// A Mutable view of a project's indexed files.
///
/// Allows in-place mutation of the files without deep cloning the hash set.
/// The changes are written back when the mutable view is dropped or by calling [`Self::set`] manually.
pub(super) struct IndexedMut<'db> {
db: Option<&'db mut dyn Db>,
package: Package,
project: Project,
files: FileSet,
did_change: bool,
}
@@ -212,12 +212,12 @@ impl IndexedMut<'_> {
if self.did_change {
// If there are changes, set the new file_set to trigger a salsa revision change.
self.package
self.project
.set_file_set(db)
.to(PackageFiles::indexed(files));
.to(IndexedFiles::indexed(files));
} else {
// The `indexed_mut` replaced the `state` with Lazy. Restore it back to the indexed state.
*self.package.file_set(db).state.lock().unwrap() = State::Indexed(files);
*self.project.file_set(db).state.lock().unwrap() = State::Indexed(files);
}
}
}
@@ -234,30 +234,24 @@ mod tests {
use crate::db::tests::TestDb;
use crate::db::Db;
use crate::workspace::files::Index;
use crate::workspace::WorkspaceMetadata;
use crate::project::files::Index;
use crate::project::ProjectMetadata;
use ruff_db::files::system_path_to_file;
use ruff_db::system::{DbWithTestSystem, SystemPathBuf};
use ruff_python_ast::name::Name;
#[test]
fn re_entrance() -> anyhow::Result<()> {
let metadata = WorkspaceMetadata::single_package(
Name::new_static("test"),
SystemPathBuf::from("/test"),
);
let metadata = ProjectMetadata::new(Name::new_static("test"), SystemPathBuf::from("/test"));
let mut db = TestDb::new(metadata);
db.write_file("test.py", "")?;
let package = db
.workspace()
.package(&db, "/test")
.expect("test package to exist");
let project = db.project();
let file = system_path_to_file(&db, "test.py").unwrap();
let files = match package.file_set(&db).get() {
let files = match project.file_set(&db).get() {
Index::Lazy(lazy) => lazy.set(FxHashSet::from_iter([file])),
Index::Indexed(files) => files,
};
@@ -265,7 +259,7 @@ mod tests {
// Calling files a second time should not dead-lock.
// This can e.g. happen when `check_file` iterates over all files and
// `is_file_open` queries the open files.
let files_2 = package.file_set(&db).get();
let files_2 = project.file_set(&db).get();
match files_2 {
Index::Lazy(_) => {

View File

@@ -0,0 +1,418 @@
use ruff_db::system::{System, SystemPath, SystemPathBuf};
use ruff_python_ast::name::Name;
use crate::project::pyproject::{PyProject, PyProjectError};
use crate::project::settings::Configuration;
use red_knot_python_semantic::ProgramSettings;
use thiserror::Error;
#[derive(Debug, PartialEq, Eq)]
#[cfg_attr(test, derive(serde::Serialize))]
pub struct ProjectMetadata {
pub(super) name: Name,
pub(super) root: SystemPathBuf,
/// The resolved settings for this project.
pub(super) configuration: Configuration,
}
impl ProjectMetadata {
/// Creates a project with the given name and root that uses the default configuration options.
pub fn new(name: Name, root: SystemPathBuf) -> Self {
Self {
name,
root,
configuration: Configuration::default(),
}
}
/// Loads a project from a `pyproject.toml` file.
pub(crate) fn from_pyproject(
pyproject: PyProject,
root: SystemPathBuf,
base_configuration: Option<&Configuration>,
) -> Self {
let name = pyproject.project.and_then(|project| project.name);
let name = name
.map(|name| Name::new(&*name))
.unwrap_or_else(|| Name::new(root.file_name().unwrap_or("root")));
// TODO: load configuration from pyrpoject.toml
let mut configuration = Configuration::default();
if let Some(base_configuration) = base_configuration {
configuration.extend(base_configuration.clone());
}
Self {
name,
root,
configuration,
}
}
/// Discovers the closest project at `path` and returns its metadata.
///
/// The algorithm traverses upwards in the `path`'s ancestor chain and uses the following precedence
/// the resolve the project's root.
///
/// 1. The closest `pyproject.toml` with a `tool.knot` section.
/// 1. The closest `pyproject.toml`.
/// 1. Fallback to use `path` as the root and use the default settings.
pub fn discover(
path: &SystemPath,
system: &dyn System,
base_configuration: Option<&Configuration>,
) -> Result<ProjectMetadata, ProjectDiscoveryError> {
tracing::debug!("Searching for a project in '{path}'");
if !system.is_directory(path) {
return Err(ProjectDiscoveryError::NotADirectory(path.to_path_buf()));
}
let mut closest_project: Option<ProjectMetadata> = None;
for ancestor in path.ancestors() {
let pyproject_path = ancestor.join("pyproject.toml");
if let Ok(pyproject_str) = system.read_to_string(&pyproject_path) {
let pyproject = PyProject::from_str(&pyproject_str).map_err(|error| {
ProjectDiscoveryError::InvalidPyProject {
path: pyproject_path,
source: Box::new(error),
}
})?;
let has_knot_section = pyproject.knot().is_some();
let metadata = ProjectMetadata::from_pyproject(
pyproject,
ancestor.to_path_buf(),
base_configuration,
);
if has_knot_section {
let project_root = ancestor;
tracing::debug!("Found project at '{}'", project_root);
return Ok(metadata);
}
// Not a project itself, keep looking for an enclosing project.
if closest_project.is_none() {
closest_project = Some(metadata);
}
}
}
// No project found, but maybe a pyproject.toml was found.
let metadata = if let Some(closest_project) = closest_project {
tracing::debug!(
"Project without `tool.knot` section: '{}'",
closest_project.root()
);
closest_project
} else {
tracing::debug!("The ancestor directories contain no `pyproject.toml`. Falling back to a virtual project.");
// Create a package with a default configuration
Self {
name: path.file_name().unwrap_or("root").into(),
root: path.to_path_buf(),
// TODO create the configuration from the pyproject toml
configuration: base_configuration.cloned().unwrap_or_default(),
}
};
Ok(metadata)
}
pub fn root(&self) -> &SystemPath {
&self.root
}
pub fn name(&self) -> &str {
&self.name
}
pub fn configuration(&self) -> &Configuration {
&self.configuration
}
pub fn to_program_settings(&self) -> ProgramSettings {
self.configuration.to_program_settings(self.root())
}
}
#[derive(Debug, Error)]
pub enum ProjectDiscoveryError {
#[error("project path '{0}' is not a directory")]
NotADirectory(SystemPathBuf),
#[error("{path} is not a valid `pyproject.toml`: {source}")]
InvalidPyProject {
source: Box<PyProjectError>,
path: SystemPathBuf,
},
}
#[cfg(test)]
mod tests {
//! Integration tests for project discovery
use crate::snapshot_project;
use anyhow::{anyhow, Context};
use insta::assert_ron_snapshot;
use ruff_db::system::{SystemPathBuf, TestSystem};
use crate::project::{ProjectDiscoveryError, ProjectMetadata};
#[test]
fn project_without_pyproject() -> anyhow::Result<()> {
let system = TestSystem::default();
let root = SystemPathBuf::from("/app");
system
.memory_file_system()
.write_files([(root.join("foo.py"), ""), (root.join("bar.py"), "")])
.context("Failed to write files")?;
let project = ProjectMetadata::discover(&root, &system, None)
.context("Failed to discover project")?;
assert_eq!(project.root(), &*root);
snapshot_project!(project);
Ok(())
}
#[test]
fn project_with_pyproject() -> anyhow::Result<()> {
let system = TestSystem::default();
let root = SystemPathBuf::from("/app");
system
.memory_file_system()
.write_files([
(
root.join("pyproject.toml"),
r#"
[project]
name = "backend"
"#,
),
(root.join("db/__init__.py"), ""),
])
.context("Failed to write files")?;
let project = ProjectMetadata::discover(&root, &system, None)
.context("Failed to discover project")?;
assert_eq!(project.root(), &*root);
snapshot_project!(project);
// Discovering the same package from a subdirectory should give the same result
let from_src = ProjectMetadata::discover(&root.join("db"), &system, None)
.context("Failed to discover project from src sub-directory")?;
assert_eq!(from_src, project);
Ok(())
}
#[test]
fn project_with_invalid_pyproject() -> anyhow::Result<()> {
let system = TestSystem::default();
let root = SystemPathBuf::from("/app");
system
.memory_file_system()
.write_files([
(
root.join("pyproject.toml"),
r#"
[project]
name = "backend"
[tool.knot
"#,
),
(root.join("db/__init__.py"), ""),
])
.context("Failed to write files")?;
let Err(error) = ProjectMetadata::discover(&root, &system, None) else {
return Err(anyhow!("Expected project discovery to fail because of invalid syntax in the pyproject.toml"));
};
assert_error_eq(
&error,
r#"/app/pyproject.toml is not a valid `pyproject.toml`: TOML parse error at line 5, column 31
|
5 | [tool.knot
| ^
invalid table header
expected `.`, `]`
"#,
);
Ok(())
}
#[test]
fn nested_projects_in_sub_project() -> anyhow::Result<()> {
let system = TestSystem::default();
let root = SystemPathBuf::from("/app");
system
.memory_file_system()
.write_files([
(
root.join("pyproject.toml"),
r#"
[project]
name = "project-root"
[tool.knot]
"#,
),
(
root.join("packages/a/pyproject.toml"),
r#"
[project]
name = "nested-project"
[tool.knot]
"#,
),
])
.context("Failed to write files")?;
let sub_project = ProjectMetadata::discover(&root.join("packages/a"), &system, None)?;
snapshot_project!(sub_project);
Ok(())
}
#[test]
fn nested_projects_in_root_project() -> anyhow::Result<()> {
let system = TestSystem::default();
let root = SystemPathBuf::from("/app");
system
.memory_file_system()
.write_files([
(
root.join("pyproject.toml"),
r#"
[project]
name = "project-root"
[tool.knot]
"#,
),
(
root.join("packages/a/pyproject.toml"),
r#"
[project]
name = "nested-project"
[tool.knot]
"#,
),
])
.context("Failed to write files")?;
let root = ProjectMetadata::discover(&root, &system, None)?;
snapshot_project!(root);
Ok(())
}
#[test]
fn nested_projects_without_knot_sections() -> anyhow::Result<()> {
let system = TestSystem::default();
let root = SystemPathBuf::from("/app");
system
.memory_file_system()
.write_files([
(
root.join("pyproject.toml"),
r#"
[project]
name = "project-root"
"#,
),
(
root.join("packages/a/pyproject.toml"),
r#"
[project]
name = "nested-project"
"#,
),
])
.context("Failed to write files")?;
let sub_project = ProjectMetadata::discover(&root.join("packages/a"), &system, None)?;
snapshot_project!(sub_project);
Ok(())
}
#[test]
fn nested_projects_with_outer_knot_section() -> anyhow::Result<()> {
let system = TestSystem::default();
let root = SystemPathBuf::from("/app");
system
.memory_file_system()
.write_files([
(
root.join("pyproject.toml"),
r#"
[project]
name = "project-root"
[tool.knot]
"#,
),
(
root.join("packages/a/pyproject.toml"),
r#"
[project]
name = "nested-project"
"#,
),
])
.context("Failed to write files")?;
let root = ProjectMetadata::discover(&root.join("packages/a"), &system, None)?;
snapshot_project!(root);
Ok(())
}
#[track_caller]
fn assert_error_eq(error: &ProjectDiscoveryError, message: &str) {
assert_eq!(error.to_string().replace('\\', "/"), message);
}
/// Snapshots a project but with all paths using unix separators.
#[macro_export]
macro_rules! snapshot_project {
($project:expr) => {{
assert_ron_snapshot!($project,{
".root" => insta::dynamic_redaction(|content, _content_path| {
content.as_str().unwrap().replace("\\", "/")
}),
});
}};
}
}

View File

@@ -4,9 +4,7 @@ use pep440_rs::{Version, VersionSpecifiers};
use serde::Deserialize;
use thiserror::Error;
use crate::workspace::metadata::WorkspaceDiscoveryError;
pub(crate) use package_name::PackageName;
use ruff_db::system::SystemPath;
/// A `pyproject.toml` as specified in PEP 517.
#[derive(Deserialize, Debug, Default, Clone)]
@@ -19,11 +17,8 @@ pub(crate) struct PyProject {
}
impl PyProject {
pub(crate) fn workspace(&self) -> Option<&Workspace> {
self.tool
.as_ref()
.and_then(|tool| tool.knot.as_ref())
.and_then(|knot| knot.workspace.as_ref())
pub(crate) fn knot(&self) -> Option<&Knot> {
self.tool.as_ref().and_then(|tool| tool.knot.as_ref())
}
}
@@ -62,47 +57,9 @@ pub(crate) struct Tool {
pub knot: Option<Knot>,
}
// TODO(micha): Remove allow once we add knot settings.
// We can't use a unit struct here or deserializing `[tool.knot]` fails.
#[allow(clippy::empty_structs_with_brackets)]
#[derive(Deserialize, Debug, Clone, PartialEq, Eq)]
#[serde(rename_all = "kebab-case", deny_unknown_fields)]
pub(crate) struct Knot {
pub(crate) workspace: Option<Workspace>,
}
#[derive(Deserialize, Debug, Clone, PartialEq, Eq)]
#[serde(rename_all = "kebab-case", deny_unknown_fields)]
pub(crate) struct Workspace {
pub(crate) members: Option<Vec<String>>,
pub(crate) exclude: Option<Vec<String>>,
}
impl Workspace {
pub(crate) fn members(&self) -> &[String] {
self.members.as_deref().unwrap_or_default()
}
pub(crate) fn exclude(&self) -> &[String] {
self.exclude.as_deref().unwrap_or_default()
}
pub(crate) fn is_excluded(
&self,
path: &SystemPath,
workspace_root: &SystemPath,
) -> Result<bool, WorkspaceDiscoveryError> {
for exclude in self.exclude() {
let full_glob =
glob::Pattern::new(workspace_root.join(exclude).as_str()).map_err(|error| {
WorkspaceDiscoveryError::InvalidMembersPattern {
raw_glob: exclude.clone(),
source: error,
}
})?;
if full_glob.matches_path(path.as_std_path()) {
return Ok(true);
}
}
Ok(false)
}
}
pub(crate) struct Knot {}

View File

@@ -1,4 +1,3 @@
use crate::workspace::PackageMetadata;
use red_knot_python_semantic::{
ProgramSettings, PythonPlatform, PythonVersion, SearchPathSettings, SitePackages,
};
@@ -9,17 +8,17 @@ use ruff_db::system::{SystemPath, SystemPathBuf};
/// The main difference to [`Configuration`] is that default values are filled in.
#[derive(Debug, Clone, PartialEq, Eq)]
#[cfg_attr(test, derive(serde::Serialize))]
pub struct WorkspaceSettings {
pub struct ProjectSettings {
pub(super) program: ProgramSettings,
}
impl WorkspaceSettings {
impl ProjectSettings {
pub fn program(&self) -> &ProgramSettings {
&self.program
}
}
/// The configuration for the workspace or a package.
/// The configuration for the project or a package.
#[derive(Debug, Default, Clone, PartialEq, Eq)]
#[cfg_attr(test, derive(serde::Serialize))]
pub struct Configuration {
@@ -34,17 +33,11 @@ impl Configuration {
self.search_paths.extend(with.search_paths);
}
pub fn to_workspace_settings(
&self,
workspace_root: &SystemPath,
_packages: &[PackageMetadata],
) -> WorkspaceSettings {
WorkspaceSettings {
program: ProgramSettings {
python_version: self.python_version.unwrap_or_default(),
python_platform: PythonPlatform::default(),
search_paths: self.search_paths.to_settings(workspace_root),
},
pub(super) fn to_program_settings(&self, first_party_root: &SystemPath) -> ProgramSettings {
ProgramSettings {
python_version: self.python_version.unwrap_or_default(),
python_platform: PythonPlatform::default(),
search_paths: self.search_paths.to_settings(first_party_root),
}
}
}
@@ -57,7 +50,7 @@ pub struct SearchPathConfiguration {
/// or pyright's stubPath configuration setting.
pub extra_paths: Option<Vec<SystemPathBuf>>,
/// The root of the workspace, used for finding first-party modules.
/// The root of the project, used for finding first-party modules.
pub src_root: Option<SystemPathBuf>,
/// Optional path to a "typeshed" directory on disk for us to use for standard-library types.

View File

@@ -0,0 +1,17 @@
---
source: crates/red_knot_workspace/src/project/metadata.rs
expression: root
---
ProjectMetadata(
name: Name("project-root"),
root: "/app",
configuration: Configuration(
python_version: None,
search_paths: SearchPathConfiguration(
extra_paths: None,
src_root: None,
typeshed: None,
site_packages: None,
),
),
)

View File

@@ -0,0 +1,17 @@
---
source: crates/red_knot_workspace/src/project/metadata.rs
expression: sub_project
---
ProjectMetadata(
name: Name("nested-project"),
root: "/app/packages/a",
configuration: Configuration(
python_version: None,
search_paths: SearchPathConfiguration(
extra_paths: None,
src_root: None,
typeshed: None,
site_packages: None,
),
),
)

View File

@@ -0,0 +1,17 @@
---
source: crates/red_knot_workspace/src/project/metadata.rs
expression: root
---
ProjectMetadata(
name: Name("project-root"),
root: "/app",
configuration: Configuration(
python_version: None,
search_paths: SearchPathConfiguration(
extra_paths: None,
src_root: None,
typeshed: None,
site_packages: None,
),
),
)

View File

@@ -0,0 +1,17 @@
---
source: crates/red_knot_workspace/src/project/metadata.rs
expression: sub_project
---
ProjectMetadata(
name: Name("nested-project"),
root: "/app/packages/a",
configuration: Configuration(
python_version: None,
search_paths: SearchPathConfiguration(
extra_paths: None,
src_root: None,
typeshed: None,
site_packages: None,
),
),
)

View File

@@ -0,0 +1,17 @@
---
source: crates/red_knot_workspace/src/project/metadata.rs
expression: project
---
ProjectMetadata(
name: Name("backend"),
root: "/app",
configuration: Configuration(
python_version: None,
search_paths: SearchPathConfiguration(
extra_paths: None,
src_root: None,
typeshed: None,
site_packages: None,
),
),
)

View File

@@ -0,0 +1,17 @@
---
source: crates/red_knot_workspace/src/project/metadata.rs
expression: project
---
ProjectMetadata(
name: Name("app"),
root: "/app",
configuration: Configuration(
python_version: None,
search_paths: SearchPathConfiguration(
extra_paths: None,
src_root: None,
typeshed: None,
site_packages: None,
),
),
)

View File

@@ -1,9 +1,9 @@
pub use project_watcher::ProjectWatcher;
use ruff_db::system::{SystemPath, SystemPathBuf, SystemVirtualPathBuf};
pub use watcher::{directory_watcher, EventHandler, Watcher};
pub use workspace_watcher::WorkspaceWatcher;
mod project_watcher;
mod watcher;
mod workspace_watcher;
/// Classification of a file system change event.
///

View File

@@ -8,11 +8,11 @@ use ruff_cache::{CacheKey, CacheKeyHasher};
use ruff_db::system::{SystemPath, SystemPathBuf};
use ruff_db::{Db as _, Upcast};
use crate::db::{Db, RootDatabase};
use crate::db::{Db, ProjectDatabase};
use crate::watch::Watcher;
/// Wrapper around a [`Watcher`] that watches the relevant paths of a workspace.
pub struct WorkspaceWatcher {
/// Wrapper around a [`Watcher`] that watches the relevant paths of a project.
pub struct ProjectWatcher {
watcher: Watcher,
/// The paths that need to be watched. This includes paths for which setting up file watching failed.
@@ -25,9 +25,9 @@ pub struct WorkspaceWatcher {
cache_key: Option<u64>,
}
impl WorkspaceWatcher {
/// Create a new workspace watcher.
pub fn new(watcher: Watcher, db: &RootDatabase) -> Self {
impl ProjectWatcher {
/// Create a new project watcher.
pub fn new(watcher: Watcher, db: &ProjectDatabase) -> Self {
let mut watcher = Self {
watcher,
watched_paths: Vec::new(),
@@ -40,11 +40,11 @@ impl WorkspaceWatcher {
watcher
}
pub fn update(&mut self, db: &RootDatabase) {
pub fn update(&mut self, db: &ProjectDatabase) {
let search_paths: Vec<_> = system_module_search_paths(db.upcast()).collect();
let workspace_path = db.workspace().root(db).to_path_buf();
let project_path = db.project().root(db).to_path_buf();
let new_cache_key = Self::compute_cache_key(&workspace_path, &search_paths);
let new_cache_key = Self::compute_cache_key(&project_path, &search_paths);
if self.cache_key == Some(new_cache_key) {
return;
@@ -56,7 +56,7 @@ impl WorkspaceWatcher {
// ```text
// - bar
// - baz.py
// - workspace
// - project
// - bar -> /bar
// - foo.py
// ```
@@ -68,23 +68,23 @@ impl WorkspaceWatcher {
self.has_errored_paths = false;
let workspace_path = db
let project_path = db
.system()
.canonicalize_path(&workspace_path)
.unwrap_or(workspace_path);
.canonicalize_path(&project_path)
.unwrap_or(project_path);
// Find the non-overlapping module search paths and filter out paths that are already covered by the workspace.
// Find the non-overlapping module search paths and filter out paths that are already covered by the project.
// Module search paths are already canonicalized.
let unique_module_paths = ruff_db::system::deduplicate_nested_paths(
search_paths
.into_iter()
.filter(|path| !path.starts_with(&workspace_path)),
.filter(|path| !path.starts_with(&project_path)),
)
.map(SystemPath::to_path_buf);
// Now add the new paths, first starting with the workspace path and then
// Now add the new paths, first starting with the project path and then
// adding the library search paths.
for path in std::iter::once(workspace_path).chain(unique_module_paths) {
for path in std::iter::once(project_path).chain(unique_module_paths) {
// Log a warning. It's not worth aborting if registering a single folder fails because
// Ruff otherwise stills works as expected.
if let Err(error) = self.watcher.watch(&path) {
@@ -106,10 +106,10 @@ impl WorkspaceWatcher {
self.cache_key = Some(new_cache_key);
}
fn compute_cache_key(workspace_root: &SystemPath, search_paths: &[&SystemPath]) -> u64 {
fn compute_cache_key(project_root: &SystemPath, search_paths: &[&SystemPath]) -> u64 {
let mut cache_key_hasher = CacheKeyHasher::new();
search_paths.cache_key(&mut cache_key_hasher);
workspace_root.cache_key(&mut cache_key_hasher);
project_root.cache_key(&mut cache_key_hasher);
cache_key_hasher.finish()
}

View File

@@ -1,665 +0,0 @@
#![allow(clippy::ref_option)]
use crate::db::Db;
use crate::db::RootDatabase;
use crate::workspace::files::{Index, Indexed, IndexedIter, PackageFiles};
pub use metadata::{PackageMetadata, WorkspaceDiscoveryError, WorkspaceMetadata};
use red_knot_python_semantic::types::check_types;
use red_knot_python_semantic::SearchPathSettings;
use ruff_db::diagnostic::{Diagnostic, DiagnosticId, ParseDiagnostic, Severity};
use ruff_db::parsed::parsed_module;
use ruff_db::source::{source_text, SourceTextError};
use ruff_db::system::FileType;
use ruff_db::{
files::{system_path_to_file, File},
system::{walk_directory::WalkState, SystemPath, SystemPathBuf},
};
use ruff_python_ast::{name::Name, PySourceType};
use ruff_text_size::TextRange;
use rustc_hash::{FxBuildHasher, FxHashSet};
use salsa::{Durability, Setter as _};
use std::borrow::Cow;
use std::iter::FusedIterator;
use std::{collections::BTreeMap, sync::Arc};
mod files;
mod metadata;
mod pyproject;
pub mod settings;
/// The project workspace as a Salsa ingredient.
///
/// A workspace consists of one or multiple packages. Packages can be nested. A file in a workspace
/// belongs to no or exactly one package (files can't belong to multiple packages).
///
/// How workspaces and packages are discovered is TBD. For now, a workspace can be any directory,
/// and it always contains a single package which has the same root as the workspace.
///
/// ## Examples
///
/// ```text
/// app-1/
/// pyproject.toml
/// src/
/// ... python files
///
/// app-2/
/// pyproject.toml
/// src/
/// ... python files
///
/// shared/
/// pyproject.toml
/// src/
/// ... python files
///
/// pyproject.toml
/// ```
///
/// The above project structure has three packages: `app-1`, `app-2`, and `shared`.
/// Each of the packages can define their own settings in their `pyproject.toml` file, but
/// they must be compatible. For example, each package can define a different `requires-python` range,
/// but the ranges must overlap.
///
/// ## How is a workspace different from a program?
/// There are two (related) motivations:
///
/// 1. Program is defined in `ruff_db` and it can't reference the settings types for the linter and formatter
/// without introducing a cyclic dependency. The workspace is defined in a higher level crate
/// where it can reference these setting types.
/// 2. Running `ruff check` with different target versions results in different programs (settings) but
/// it remains the same workspace. That's why program is a narrowed view of the workspace only
/// holding on to the most fundamental settings required for checking.
#[salsa::input]
pub struct Workspace {
#[return_ref]
root_buf: SystemPathBuf,
/// The files that are open in the workspace.
///
/// Setting the open files to a non-`None` value changes `check` to only check the
/// open files rather than all files in the workspace.
#[return_ref]
#[default]
open_fileset: Option<Arc<FxHashSet<File>>>,
/// The (first-party) packages in this workspace.
#[return_ref]
package_tree: PackageTree,
/// The unresolved search path configuration.
#[return_ref]
pub search_path_settings: SearchPathSettings,
}
/// A first-party package in a workspace.
#[salsa::input]
pub struct Package {
#[return_ref]
pub name: Name,
/// The path to the root directory of the package.
#[return_ref]
root_buf: SystemPathBuf,
/// The files that are part of this package.
#[default]
#[return_ref]
file_set: PackageFiles,
// TODO: Add the loaded settings.
}
impl Workspace {
pub fn from_metadata(db: &dyn Db, metadata: WorkspaceMetadata) -> Self {
let mut packages = BTreeMap::new();
for package in metadata.packages {
packages.insert(package.root.clone(), Package::from_metadata(db, package));
}
let program_settings = metadata.settings.program;
Workspace::builder(
metadata.root,
PackageTree(packages),
program_settings.search_paths,
)
.durability(Durability::MEDIUM)
.open_fileset_durability(Durability::LOW)
.new(db)
}
pub fn root(self, db: &dyn Db) -> &SystemPath {
self.root_buf(db)
}
pub fn reload(self, db: &mut dyn Db, metadata: WorkspaceMetadata) {
tracing::debug!("Reloading workspace");
assert_eq!(self.root(db), metadata.root());
let mut old_packages = self.package_tree(db).0.clone();
let mut new_packages = BTreeMap::new();
for package_metadata in metadata.packages {
let path = package_metadata.root().to_path_buf();
let package = if let Some(old_package) = old_packages.remove(&path) {
old_package.update(db, package_metadata);
old_package
} else {
Package::from_metadata(db, package_metadata)
};
new_packages.insert(path, package);
}
if &metadata.settings.program.search_paths != self.search_path_settings(db) {
self.set_search_path_settings(db)
.to(metadata.settings.program.search_paths);
}
self.set_package_tree(db).to(PackageTree(new_packages));
}
pub fn update_package(self, db: &mut dyn Db, metadata: PackageMetadata) -> anyhow::Result<()> {
let path = metadata.root().to_path_buf();
if let Some(package) = self.package_tree(db).get(&path) {
package.update(db, metadata);
Ok(())
} else {
Err(anyhow::anyhow!("Package {path} not found"))
}
}
pub fn packages(self, db: &dyn Db) -> &PackageTree {
self.package_tree(db)
}
/// Returns the closest package to which the first-party `path` belongs.
///
/// Returns `None` if the `path` is outside of any package or if `file` isn't a first-party file
/// (e.g. third-party dependencies or `excluded`).
pub fn package(self, db: &dyn Db, path: impl AsRef<SystemPath>) -> Option<Package> {
let packages = self.package_tree(db);
packages.get(path.as_ref())
}
/// Checks all open files in the workspace and its dependencies.
pub fn check(self, db: &RootDatabase) -> Vec<Box<dyn Diagnostic>> {
let workspace_span = tracing::debug_span!("check_workspace");
let _span = workspace_span.enter();
tracing::debug!("Checking workspace");
let files = WorkspaceFiles::new(db, self);
let result = Arc::new(std::sync::Mutex::new(Vec::new()));
let inner_result = Arc::clone(&result);
let db = db.clone();
let workspace_span = workspace_span.clone();
rayon::scope(move |scope| {
for file in &files {
let result = inner_result.clone();
let db = db.clone();
let workspace_span = workspace_span.clone();
scope.spawn(move |_| {
let check_file_span = tracing::debug_span!(parent: &workspace_span, "check_file", file=%file.path(&db));
let _entered = check_file_span.entered();
let file_diagnostics = check_file(&db, file);
result.lock().unwrap().extend(file_diagnostics);
});
}
});
Arc::into_inner(result).unwrap().into_inner().unwrap()
}
/// Opens a file in the workspace.
///
/// This changes the behavior of `check` to only check the open files rather than all files in the workspace.
pub fn open_file(self, db: &mut dyn Db, file: File) {
tracing::debug!("Opening file `{}`", file.path(db));
let mut open_files = self.take_open_files(db);
open_files.insert(file);
self.set_open_files(db, open_files);
}
/// Closes a file in the workspace.
pub fn close_file(self, db: &mut dyn Db, file: File) -> bool {
tracing::debug!("Closing file `{}`", file.path(db));
let mut open_files = self.take_open_files(db);
let removed = open_files.remove(&file);
if removed {
self.set_open_files(db, open_files);
}
removed
}
/// Returns the open files in the workspace or `None` if the entire workspace should be checked.
pub fn open_files(self, db: &dyn Db) -> Option<&FxHashSet<File>> {
self.open_fileset(db).as_deref()
}
/// Sets the open files in the workspace.
///
/// This changes the behavior of `check` to only check the open files rather than all files in the workspace.
#[tracing::instrument(level = "debug", skip(self, db))]
pub fn set_open_files(self, db: &mut dyn Db, open_files: FxHashSet<File>) {
tracing::debug!("Set open workspace files (count: {})", open_files.len());
self.set_open_fileset(db).to(Some(Arc::new(open_files)));
}
/// This takes the open files from the workspace and returns them.
///
/// This changes the behavior of `check` to check all files in the workspace instead of just the open files.
pub fn take_open_files(self, db: &mut dyn Db) -> FxHashSet<File> {
tracing::debug!("Take open workspace files");
// Salsa will cancel any pending queries and remove its own reference to `open_files`
// so that the reference counter to `open_files` now drops to 1.
let open_files = self.set_open_fileset(db).to(None);
if let Some(open_files) = open_files {
Arc::try_unwrap(open_files).unwrap()
} else {
FxHashSet::default()
}
}
/// Returns `true` if the file is open in the workspace.
///
/// A file is considered open when:
/// * explicitly set as an open file using [`open_file`](Self::open_file)
/// * It has a [`SystemPath`] and belongs to a package's `src` files
/// * It has a [`SystemVirtualPath`](ruff_db::system::SystemVirtualPath)
pub fn is_file_open(self, db: &dyn Db, file: File) -> bool {
if let Some(open_files) = self.open_files(db) {
open_files.contains(&file)
} else if let Some(system_path) = file.path(db).as_system_path() {
self.package(db, system_path)
.map_or(false, |package| package.contains_file(db, file))
} else {
file.path(db).is_system_virtual_path()
}
}
}
impl Package {
pub fn root(self, db: &dyn Db) -> &SystemPath {
self.root_buf(db)
}
/// Returns `true` if `file` is a first-party file part of this package.
pub fn contains_file(self, db: &dyn Db, file: File) -> bool {
self.files(db).contains(&file)
}
#[tracing::instrument(level = "debug", skip(db))]
pub fn remove_file(self, db: &mut dyn Db, file: File) {
tracing::debug!(
"Removing file `{}` from package `{}`",
file.path(db),
self.name(db)
);
let Some(mut index) = PackageFiles::indexed_mut(db, self) else {
return;
};
index.remove(file);
}
pub fn add_file(self, db: &mut dyn Db, file: File) {
tracing::debug!(
"Adding file `{}` to package `{}`",
file.path(db),
self.name(db)
);
let Some(mut index) = PackageFiles::indexed_mut(db, self) else {
return;
};
index.insert(file);
}
/// Returns the files belonging to this package.
pub fn files(self, db: &dyn Db) -> Indexed<'_> {
let files = self.file_set(db);
let indexed = match files.get() {
Index::Lazy(vacant) => {
let _entered =
tracing::debug_span!("index_package_files", package = %self.name(db)).entered();
let files = discover_package_files(db, self);
tracing::info!("Found {} files in package `{}`", files.len(), self.name(db));
vacant.set(files)
}
Index::Indexed(indexed) => indexed,
};
indexed
}
fn from_metadata(db: &dyn Db, metadata: PackageMetadata) -> Self {
Self::builder(metadata.name, metadata.root)
.durability(Durability::MEDIUM)
.file_set_durability(Durability::LOW)
.new(db)
}
fn update(self, db: &mut dyn Db, metadata: PackageMetadata) {
let root = self.root(db);
assert_eq!(root, metadata.root());
if self.name(db) != metadata.name() {
self.set_name(db).to(metadata.name);
}
}
pub fn reload_files(self, db: &mut dyn Db) {
tracing::debug!("Reloading files for package `{}`", self.name(db));
if !self.file_set(db).is_lazy() {
// Force a re-index of the files in the next revision.
self.set_file_set(db).to(PackageFiles::lazy());
}
}
}
pub(super) fn check_file(db: &dyn Db, file: File) -> Vec<Box<dyn Diagnostic>> {
let mut diagnostics: Vec<Box<dyn Diagnostic>> = Vec::new();
// Abort checking if there are IO errors.
let source = source_text(db.upcast(), file);
if let Some(read_error) = source.read_error() {
diagnostics.push(Box::new(IOErrorDiagnostic {
file,
error: read_error.clone(),
}));
return diagnostics;
}
let parsed = parsed_module(db.upcast(), file);
diagnostics.extend(parsed.errors().iter().map(|error| {
let diagnostic: Box<dyn Diagnostic> = Box::new(ParseDiagnostic::new(file, error.clone()));
diagnostic
}));
diagnostics.extend(check_types(db.upcast(), file).iter().map(|diagnostic| {
let boxed: Box<dyn Diagnostic> = Box::new(diagnostic.clone());
boxed
}));
diagnostics.sort_unstable_by_key(|diagnostic| diagnostic.range().unwrap_or_default().start());
diagnostics
}
fn discover_package_files(db: &dyn Db, package: Package) -> FxHashSet<File> {
let paths = std::sync::Mutex::new(Vec::new());
let packages = db.workspace().packages(db);
db.system().walk_directory(package.root(db)).run(|| {
Box::new(|entry| {
match entry {
Ok(entry) => {
// Skip over any non python files to avoid creating too many entries in `Files`.
match entry.file_type() {
FileType::File => {
if entry
.path()
.extension()
.and_then(PySourceType::try_from_extension)
.is_some()
{
let mut paths = paths.lock().unwrap();
paths.push(entry.into_path());
}
}
FileType::Directory | FileType::Symlink => {
// Don't traverse into nested packages (the workspace-package is an ancestor of all other packages)
if packages.get(entry.path()) != Some(package) {
return WalkState::Skip;
}
}
}
}
Err(error) => {
// TODO Handle error
tracing::error!("Failed to walk path: {error}");
}
}
WalkState::Continue
})
});
let paths = paths.into_inner().unwrap();
let mut files = FxHashSet::with_capacity_and_hasher(paths.len(), FxBuildHasher);
for path in paths {
// If this returns `None`, then the file was deleted between the `walk_directory` call and now.
// We can ignore this.
if let Ok(file) = system_path_to_file(db.upcast(), &path) {
files.insert(file);
}
}
files
}
#[derive(Debug)]
enum WorkspaceFiles<'a> {
OpenFiles(&'a FxHashSet<File>),
PackageFiles(Vec<Indexed<'a>>),
}
impl<'a> WorkspaceFiles<'a> {
fn new(db: &'a dyn Db, workspace: Workspace) -> Self {
if let Some(open_files) = workspace.open_files(db) {
WorkspaceFiles::OpenFiles(open_files)
} else {
WorkspaceFiles::PackageFiles(
workspace
.packages(db)
.iter()
.map(|package| package.files(db))
.collect(),
)
}
}
}
impl<'a> IntoIterator for &'a WorkspaceFiles<'a> {
type Item = File;
type IntoIter = WorkspaceFilesIter<'a>;
fn into_iter(self) -> Self::IntoIter {
match self {
WorkspaceFiles::OpenFiles(files) => WorkspaceFilesIter::OpenFiles(files.iter()),
WorkspaceFiles::PackageFiles(package_files) => {
let mut package_files = package_files.iter();
WorkspaceFilesIter::PackageFiles {
current: package_files.next().map(IntoIterator::into_iter),
package_files,
}
}
}
}
}
enum WorkspaceFilesIter<'db> {
OpenFiles(std::collections::hash_set::Iter<'db, File>),
PackageFiles {
package_files: std::slice::Iter<'db, Indexed<'db>>,
current: Option<IndexedIter<'db>>,
},
}
impl Iterator for WorkspaceFilesIter<'_> {
type Item = File;
fn next(&mut self) -> Option<Self::Item> {
match self {
WorkspaceFilesIter::OpenFiles(files) => files.next().copied(),
WorkspaceFilesIter::PackageFiles {
package_files,
current,
} => loop {
if let Some(file) = current.as_mut().and_then(Iterator::next) {
return Some(file);
}
*current = Some(package_files.next()?.into_iter());
},
}
}
}
#[derive(Debug)]
pub struct IOErrorDiagnostic {
file: File,
error: SourceTextError,
}
impl Diagnostic for IOErrorDiagnostic {
fn id(&self) -> DiagnosticId {
DiagnosticId::Io
}
fn message(&self) -> Cow<str> {
self.error.to_string().into()
}
fn file(&self) -> File {
self.file
}
fn range(&self) -> Option<TextRange> {
None
}
fn severity(&self) -> Severity {
Severity::Error
}
}
#[derive(Debug, Eq, PartialEq, Clone)]
pub struct PackageTree(BTreeMap<SystemPathBuf, Package>);
impl PackageTree {
pub fn get(&self, path: &SystemPath) -> Option<Package> {
let (package_path, package) = self.0.range(..=path.to_path_buf()).next_back()?;
if path.starts_with(package_path) {
Some(*package)
} else {
None
}
}
// The package table should never be empty, that's why `is_empty` makes little sense
#[allow(clippy::len_without_is_empty)]
pub fn len(&self) -> usize {
self.0.len()
}
pub fn iter(&self) -> PackageTreeIter {
PackageTreeIter(self.0.values())
}
}
impl<'a> IntoIterator for &'a PackageTree {
type Item = Package;
type IntoIter = PackageTreeIter<'a>;
fn into_iter(self) -> Self::IntoIter {
self.iter()
}
}
pub struct PackageTreeIter<'a>(std::collections::btree_map::Values<'a, SystemPathBuf, Package>);
impl Iterator for PackageTreeIter<'_> {
type Item = Package;
fn next(&mut self) -> Option<Self::Item> {
self.0.next().copied()
}
fn size_hint(&self) -> (usize, Option<usize>) {
self.0.size_hint()
}
fn last(mut self) -> Option<Self::Item> {
self.0.next_back().copied()
}
}
impl ExactSizeIterator for PackageTreeIter<'_> {}
impl FusedIterator for PackageTreeIter<'_> {}
#[cfg(test)]
mod tests {
use crate::db::tests::TestDb;
use crate::workspace::{check_file, WorkspaceMetadata};
use red_knot_python_semantic::types::check_types;
use ruff_db::diagnostic::Diagnostic;
use ruff_db::files::system_path_to_file;
use ruff_db::source::source_text;
use ruff_db::system::{DbWithTestSystem, SystemPath, SystemPathBuf};
use ruff_db::testing::assert_function_query_was_not_run;
use ruff_python_ast::name::Name;
#[test]
fn check_file_skips_type_checking_when_file_cant_be_read() -> ruff_db::system::Result<()> {
let workspace =
WorkspaceMetadata::single_package(Name::new_static("test"), SystemPathBuf::from("/"));
let mut db = TestDb::new(workspace);
let path = SystemPath::new("test.py");
db.write_file(path, "x = 10")?;
let file = system_path_to_file(&db, path).unwrap();
// Now the file gets deleted before we had a chance to read its source text.
db.memory_file_system().remove_file(path)?;
file.sync(&mut db);
assert_eq!(source_text(&db, file).as_str(), "");
assert_eq!(
check_file(&db, file)
.into_iter()
.map(|diagnostic| diagnostic.message().into_owned())
.collect::<Vec<_>>(),
vec!["Failed to read file: No such file or directory".to_string()]
);
let events = db.take_salsa_events();
assert_function_query_was_not_run(&db, check_types, file, &events);
// The user now creates a new file with an empty text. The source text
// content returned by `source_text` remains unchanged, but the diagnostics should get updated.
db.write_file(path, "").unwrap();
assert_eq!(source_text(&db, file).as_str(), "");
assert_eq!(
check_file(&db, file)
.into_iter()
.map(|diagnostic| diagnostic.message().into_owned())
.collect::<Vec<_>>(),
vec![] as Vec<String>
);
Ok(())
}
}

View File

@@ -1,812 +0,0 @@
use ruff_db::system::{GlobError, System, SystemPath, SystemPathBuf};
use ruff_python_ast::name::Name;
use rustc_hash::{FxBuildHasher, FxHashMap, FxHashSet};
use thiserror::Error;
use crate::workspace::pyproject::{PyProject, PyProjectError, Workspace};
use crate::workspace::settings::{Configuration, WorkspaceSettings};
#[derive(Debug, PartialEq, Eq)]
#[cfg_attr(test, derive(serde::Serialize))]
pub struct WorkspaceMetadata {
pub(super) root: SystemPathBuf,
/// The (first-party) packages in this workspace.
pub(super) packages: Vec<PackageMetadata>,
/// The resolved settings for this workspace.
pub(super) settings: WorkspaceSettings,
}
/// A first-party package in a workspace.
#[derive(Debug, Clone, PartialEq, Eq)]
#[cfg_attr(test, derive(serde::Serialize))]
pub struct PackageMetadata {
pub(super) name: Name,
/// The path to the root directory of the package.
pub(super) root: SystemPathBuf,
pub(super) configuration: Configuration,
}
impl WorkspaceMetadata {
/// Creates a workspace that consists of a single package located at `root`.
pub fn single_package(name: Name, root: SystemPathBuf) -> Self {
let package = PackageMetadata {
name,
root: root.clone(),
configuration: Configuration::default(),
};
let packages = vec![package];
let settings = packages[0]
.configuration
.to_workspace_settings(&root, &packages);
Self {
root,
packages,
settings,
}
}
/// Discovers the closest workspace at `path` and returns its metadata.
///
/// 1. Traverse upwards in the `path`'s ancestor chain and find the first `pyproject.toml`.
/// 1. If the `pyproject.toml` contains no `knot.workspace` table, then keep traversing the `path`'s ancestor
/// chain until we find one or reach the root.
/// 1. If we've found a workspace, then resolve the workspace's members and assert that the closest
/// package (the first found package without a `knot.workspace` table is a member. If not, create
/// a single package workspace for the closest package.
/// 1. If there's no `pyrpoject.toml` with a `knot.workspace` table, then create a single-package workspace.
/// 1. If no ancestor directory contains any `pyproject.toml`, create an ad-hoc workspace for `path`
/// that consists of a single package and uses the default settings.
pub fn discover(
path: &SystemPath,
system: &dyn System,
base_configuration: Option<&Configuration>,
) -> Result<WorkspaceMetadata, WorkspaceDiscoveryError> {
tracing::debug!("Searching for a workspace in '{path}'");
if !system.is_directory(path) {
return Err(WorkspaceDiscoveryError::NotADirectory(path.to_path_buf()));
}
let mut closest_package: Option<PackageMetadata> = None;
for ancestor in path.ancestors() {
let pyproject_path = ancestor.join("pyproject.toml");
if let Ok(pyproject_str) = system.read_to_string(&pyproject_path) {
let pyproject = PyProject::from_str(&pyproject_str).map_err(|error| {
WorkspaceDiscoveryError::InvalidPyProject {
path: pyproject_path,
source: Box::new(error),
}
})?;
let workspace_table = pyproject.workspace().cloned();
let package = PackageMetadata::from_pyproject(
pyproject,
ancestor.to_path_buf(),
base_configuration,
);
if let Some(workspace_table) = workspace_table {
let workspace_root = ancestor;
tracing::debug!("Found workspace at '{}'", workspace_root);
match collect_packages(
package,
&workspace_table,
closest_package,
base_configuration,
system,
)? {
CollectedPackagesOrStandalone::Packages(mut packages) => {
let mut by_name =
FxHashMap::with_capacity_and_hasher(packages.len(), FxBuildHasher);
let mut workspace_package = None;
for package in &packages {
if let Some(conflicting) = by_name.insert(package.name(), package) {
return Err(WorkspaceDiscoveryError::DuplicatePackageNames {
name: package.name().clone(),
first: conflicting.root().to_path_buf(),
second: package.root().to_path_buf(),
});
}
if package.root() == workspace_root {
workspace_package = Some(package);
} else if !package.root().starts_with(workspace_root) {
return Err(WorkspaceDiscoveryError::PackageOutsideWorkspace {
package_name: package.name().clone(),
package_root: package.root().to_path_buf(),
workspace_root: workspace_root.to_path_buf(),
});
}
}
let workspace_package = workspace_package
.expect("workspace package to be part of the workspace's packages");
let settings = workspace_package
.configuration
.to_workspace_settings(workspace_root, &packages);
packages.sort_unstable_by(|a, b| a.root().cmp(b.root()));
return Ok(Self {
root: workspace_root.to_path_buf(),
packages,
settings,
});
}
CollectedPackagesOrStandalone::Standalone(package) => {
closest_package = Some(package);
break;
}
}
}
// Not a workspace itself, keep looking for an enclosing workspace.
if closest_package.is_none() {
closest_package = Some(package);
}
}
}
// No workspace found, but maybe a pyproject.toml was found.
let package = if let Some(enclosing_package) = closest_package {
tracing::debug!("Single package workspace at '{}'", enclosing_package.root());
enclosing_package
} else {
tracing::debug!("The ancestor directories contain no `pyproject.toml`. Falling back to a virtual project.");
// Create a package with a default configuration
PackageMetadata {
name: path.file_name().unwrap_or("root").into(),
root: path.to_path_buf(),
// TODO create the configuration from the pyproject toml
configuration: base_configuration.cloned().unwrap_or_default(),
}
};
let root = package.root().to_path_buf();
let packages = vec![package];
let settings = packages[0]
.configuration
.to_workspace_settings(&root, &packages);
Ok(Self {
root,
packages,
settings,
})
}
pub fn root(&self) -> &SystemPath {
&self.root
}
pub fn packages(&self) -> &[PackageMetadata] {
&self.packages
}
pub fn settings(&self) -> &WorkspaceSettings {
&self.settings
}
}
impl PackageMetadata {
pub(crate) fn from_pyproject(
pyproject: PyProject,
root: SystemPathBuf,
base_configuration: Option<&Configuration>,
) -> Self {
let name = pyproject.project.and_then(|project| project.name);
let name = name
.map(|name| Name::new(&*name))
.unwrap_or_else(|| Name::new(root.file_name().unwrap_or("root")));
// TODO: load configuration from pyrpoject.toml
let mut configuration = Configuration::default();
if let Some(base_configuration) = base_configuration {
configuration.extend(base_configuration.clone());
}
PackageMetadata {
name,
root,
configuration,
}
}
pub fn name(&self) -> &Name {
&self.name
}
pub fn root(&self) -> &SystemPath {
&self.root
}
}
fn collect_packages(
workspace_package: PackageMetadata,
workspace_table: &Workspace,
closest_package: Option<PackageMetadata>,
base_configuration: Option<&Configuration>,
system: &dyn System,
) -> Result<CollectedPackagesOrStandalone, WorkspaceDiscoveryError> {
let workspace_root = workspace_package.root().to_path_buf();
let mut member_paths = FxHashSet::default();
for glob in workspace_table.members() {
let full_glob = workspace_package.root().join(glob);
let matches = system.glob(full_glob.as_str()).map_err(|error| {
WorkspaceDiscoveryError::InvalidMembersPattern {
raw_glob: glob.clone(),
source: error,
}
})?;
for result in matches {
let path = result?;
let normalized = SystemPath::absolute(path, &workspace_root);
// Skip over non-directory entry. E.g.finder might end up creating a `.DS_STORE` file
// that ends up matching `/projects/*`.
if system.is_directory(&normalized) {
member_paths.insert(normalized);
} else {
tracing::debug!("Ignoring non-directory workspace member '{normalized}'");
}
}
}
// The workspace root is always a member. Don't re-add it
let mut packages = vec![workspace_package];
member_paths.remove(&workspace_root);
// Add the package that is closest to the current working directory except
// if that package isn't a workspace member, then fallback to creating a single
// package workspace.
if let Some(closest_package) = closest_package {
// the closest `pyproject.toml` isn't a member of this workspace because it is
// explicitly included or simply not listed.
// Create a standalone workspace.
if !member_paths.remove(closest_package.root())
|| workspace_table.is_excluded(closest_package.root(), &workspace_root)?
{
tracing::debug!(
"Ignoring workspace '{workspace_root}' because package '{package}' is not a member",
package = closest_package.name()
);
return Ok(CollectedPackagesOrStandalone::Standalone(closest_package));
}
tracing::debug!("adding package '{}'", closest_package.name());
packages.push(closest_package);
}
// Add all remaining member paths
for member_path in member_paths {
if workspace_table.is_excluded(&member_path, workspace_root.as_path())? {
tracing::debug!("Ignoring excluded member '{member_path}'");
continue;
}
let pyproject_path = member_path.join("pyproject.toml");
let pyproject_str = match system.read_to_string(&pyproject_path) {
Ok(pyproject_str) => pyproject_str,
Err(error) => {
if error.kind() == std::io::ErrorKind::NotFound
&& member_path
.file_name()
.is_some_and(|name| name.starts_with('.'))
{
tracing::debug!(
"Ignore member '{member_path}' because it has no pyproject.toml and is hidden",
);
continue;
}
return Err(WorkspaceDiscoveryError::MemberFailedToReadPyProject {
package_root: member_path,
source: error,
});
}
};
let pyproject = PyProject::from_str(&pyproject_str).map_err(|error| {
WorkspaceDiscoveryError::InvalidPyProject {
source: Box::new(error),
path: pyproject_path,
}
})?;
if pyproject.workspace().is_some() {
return Err(WorkspaceDiscoveryError::NestedWorkspaces {
package_root: member_path,
});
}
let package = PackageMetadata::from_pyproject(pyproject, member_path, base_configuration);
tracing::debug!(
"Adding package '{}' at '{}'",
package.name(),
package.root()
);
packages.push(package);
}
packages.sort_unstable_by(|a, b| a.root().cmp(b.root()));
Ok(CollectedPackagesOrStandalone::Packages(packages))
}
enum CollectedPackagesOrStandalone {
Packages(Vec<PackageMetadata>),
Standalone(PackageMetadata),
}
#[derive(Debug, Error)]
pub enum WorkspaceDiscoveryError {
#[error("workspace path '{0}' is not a directory")]
NotADirectory(SystemPathBuf),
#[error("nested workspaces aren't supported but the package located at '{package_root}' defines a `knot.workspace` table")]
NestedWorkspaces { package_root: SystemPathBuf },
#[error("the workspace contains two packages named '{name}': '{first}' and '{second}'")]
DuplicatePackageNames {
name: Name,
first: SystemPathBuf,
second: SystemPathBuf,
},
#[error("the package '{package_name}' located at '{package_root}' is outside the workspace's root directory '{workspace_root}'")]
PackageOutsideWorkspace {
workspace_root: SystemPathBuf,
package_name: Name,
package_root: SystemPathBuf,
},
#[error(
"failed to read the `pyproject.toml` for the package located at '{package_root}': {source}"
)]
MemberFailedToReadPyProject {
package_root: SystemPathBuf,
source: std::io::Error,
},
#[error("{path} is not a valid `pyproject.toml`: {source}")]
InvalidPyProject {
source: Box<PyProjectError>,
path: SystemPathBuf,
},
#[error("invalid glob '{raw_glob}' in `tool.knot.workspace.members`: {source}")]
InvalidMembersPattern {
source: glob::PatternError,
raw_glob: String,
},
#[error("failed to match member glob: {error}")]
FailedToMatchGlob {
#[from]
error: GlobError,
},
}
#[cfg(test)]
mod tests {
//! Integration tests for workspace discovery
use crate::snapshot_workspace;
use anyhow::Context;
use insta::assert_ron_snapshot;
use ruff_db::system::{SystemPathBuf, TestSystem};
use crate::workspace::{WorkspaceDiscoveryError, WorkspaceMetadata};
#[test]
fn package_without_pyproject() -> anyhow::Result<()> {
let system = TestSystem::default();
let root = SystemPathBuf::from("/app");
system
.memory_file_system()
.write_files([(root.join("foo.py"), ""), (root.join("bar.py"), "")])
.context("Failed to write files")?;
let workspace = WorkspaceMetadata::discover(&root, &system, None)
.context("Failed to discover workspace")?;
assert_eq!(workspace.root(), &*root);
snapshot_workspace!(workspace);
Ok(())
}
#[test]
fn single_package() -> anyhow::Result<()> {
let system = TestSystem::default();
let root = SystemPathBuf::from("/app");
system
.memory_file_system()
.write_files([
(
root.join("pyproject.toml"),
r#"
[project]
name = "backend"
"#,
),
(root.join("db/__init__.py"), ""),
])
.context("Failed to write files")?;
let workspace = WorkspaceMetadata::discover(&root, &system, None)
.context("Failed to discover workspace")?;
assert_eq!(workspace.root(), &*root);
snapshot_workspace!(workspace);
// Discovering the same package from a subdirectory should give the same result
let from_src = WorkspaceMetadata::discover(&root.join("db"), &system, None)
.context("Failed to discover workspace from src sub-directory")?;
assert_eq!(from_src, workspace);
Ok(())
}
#[test]
fn workspace_members() -> anyhow::Result<()> {
let system = TestSystem::default();
let root = SystemPathBuf::from("/app");
system
.memory_file_system()
.write_files([
(
root.join("pyproject.toml"),
r#"
[project]
name = "workspace-root"
[tool.knot.workspace]
members = ["packages/*"]
exclude = ["packages/excluded"]
"#,
),
(
root.join("packages/a/pyproject.toml"),
r#"
[project]
name = "member-a"
"#,
),
(
root.join("packages/x/pyproject.toml"),
r#"
[project]
name = "member-x"
"#,
),
])
.context("Failed to write files")?;
let workspace = WorkspaceMetadata::discover(&root, &system, None)
.context("Failed to discover workspace")?;
assert_eq!(workspace.root(), &*root);
snapshot_workspace!(workspace);
// Discovering the same package from a member should give the same result
let from_src = WorkspaceMetadata::discover(&root.join("packages/a"), &system, None)
.context("Failed to discover workspace from src sub-directory")?;
assert_eq!(from_src, workspace);
Ok(())
}
#[test]
fn workspace_excluded() -> anyhow::Result<()> {
let system = TestSystem::default();
let root = SystemPathBuf::from("/app");
system
.memory_file_system()
.write_files([
(
root.join("pyproject.toml"),
r#"
[project]
name = "workspace-root"
[tool.knot.workspace]
members = ["packages/*"]
exclude = ["packages/excluded"]
"#,
),
(
root.join("packages/a/pyproject.toml"),
r#"
[project]
name = "member-a"
"#,
),
(
root.join("packages/excluded/pyproject.toml"),
r#"
[project]
name = "member-x"
"#,
),
])
.context("Failed to write files")?;
let workspace = WorkspaceMetadata::discover(&root, &system, None)
.context("Failed to discover workspace")?;
assert_eq!(workspace.root(), &*root);
snapshot_workspace!(workspace);
// Discovering the `workspace` for `excluded` should discover a single-package workspace
let excluded_workspace =
WorkspaceMetadata::discover(&root.join("packages/excluded"), &system, None)
.context("Failed to discover workspace from src sub-directory")?;
assert_ne!(excluded_workspace, workspace);
Ok(())
}
#[test]
fn workspace_non_unique_member_names() -> anyhow::Result<()> {
let system = TestSystem::default();
let root = SystemPathBuf::from("/app");
system
.memory_file_system()
.write_files([
(
root.join("pyproject.toml"),
r#"
[project]
name = "workspace-root"
[tool.knot.workspace]
members = ["packages/*"]
"#,
),
(
root.join("packages/a/pyproject.toml"),
r#"
[project]
name = "a"
"#,
),
(
root.join("packages/b/pyproject.toml"),
r#"
[project]
name = "a"
"#,
),
])
.context("Failed to write files")?;
let error = WorkspaceMetadata::discover(&root, &system, None).expect_err(
"Discovery should error because the workspace contains two packages with the same names.",
);
assert_error_eq(&error, "the workspace contains two packages named 'a': '/app/packages/a' and '/app/packages/b'");
Ok(())
}
#[test]
fn nested_workspaces() -> anyhow::Result<()> {
let system = TestSystem::default();
let root = SystemPathBuf::from("/app");
system
.memory_file_system()
.write_files([
(
root.join("pyproject.toml"),
r#"
[project]
name = "workspace-root"
[tool.knot.workspace]
members = ["packages/*"]
"#,
),
(
root.join("packages/a/pyproject.toml"),
r#"
[project]
name = "nested-workspace"
[tool.knot.workspace]
members = ["packages/*"]
"#,
),
])
.context("Failed to write files")?;
let error = WorkspaceMetadata::discover(&root, &system, None).expect_err(
"Discovery should error because the workspace has a package that itself is a workspace",
);
assert_error_eq(&error, "nested workspaces aren't supported but the package located at '/app/packages/a' defines a `knot.workspace` table");
Ok(())
}
#[test]
fn member_missing_pyproject_toml() -> anyhow::Result<()> {
let system = TestSystem::default();
let root = SystemPathBuf::from("/app");
system
.memory_file_system()
.write_files([
(
root.join("pyproject.toml"),
r#"
[project]
name = "workspace-root"
[tool.knot.workspace]
members = ["packages/*"]
"#,
),
(root.join("packages/a/test.py"), ""),
])
.context("Failed to write files")?;
let error = WorkspaceMetadata::discover(&root, &system, None)
.expect_err("Discovery should error because member `a` has no `pypyroject.toml`");
assert_error_eq(&error, "failed to read the `pyproject.toml` for the package located at '/app/packages/a': No such file or directory");
Ok(())
}
/// Folders that match the members pattern but don't have a pyproject.toml
/// aren't valid members and discovery fails. However, don't fail
/// if the folder name indicates that it is a hidden folder that might
/// have been created by another tool
#[test]
fn member_pattern_matching_hidden_folder() -> anyhow::Result<()> {
let system = TestSystem::default();
let root = SystemPathBuf::from("/app");
system
.memory_file_system()
.write_files([
(
root.join("pyproject.toml"),
r#"
[project]
name = "workspace-root"
[tool.knot.workspace]
members = ["packages/*"]
"#,
),
(root.join("packages/.hidden/a.py"), ""),
])
.context("Failed to write files")?;
let workspace = WorkspaceMetadata::discover(&root, &system, None)?;
snapshot_workspace!(workspace);
Ok(())
}
#[test]
fn member_pattern_matching_file() -> anyhow::Result<()> {
let system = TestSystem::default();
let root = SystemPathBuf::from("/app");
system
.memory_file_system()
.write_files([
(
root.join("pyproject.toml"),
r#"
[project]
name = "workspace-root"
[tool.knot.workspace]
members = ["packages/*"]
"#,
),
(root.join("packages/.DS_STORE"), ""),
])
.context("Failed to write files")?;
let workspace = WorkspaceMetadata::discover(&root, &system, None)?;
snapshot_workspace!(&workspace);
Ok(())
}
#[test]
fn workspace_root_not_an_ancestor_of_member() -> anyhow::Result<()> {
let system = TestSystem::default();
let root = SystemPathBuf::from("/app");
system
.memory_file_system()
.write_files([
(
root.join("pyproject.toml"),
r#"
[project]
name = "workspace-root"
[tool.knot.workspace]
members = ["../packages/*"]
"#,
),
(
root.join("../packages/a/pyproject.toml"),
r#"
[project]
name = "a"
"#,
),
])
.context("Failed to write files")?;
let error = WorkspaceMetadata::discover(&root, &system, None).expect_err(
"Discovery should error because member `a` is outside the workspace's directory`",
);
assert_error_eq(&error, "the package 'a' located at '/packages/a' is outside the workspace's root directory '/app'");
Ok(())
}
#[track_caller]
fn assert_error_eq(error: &WorkspaceDiscoveryError, message: &str) {
assert_eq!(error.to_string().replace('\\', "/"), message);
}
/// Snapshots a workspace but with all paths using unix separators.
#[macro_export]
macro_rules! snapshot_workspace {
($workspace:expr) => {{
assert_ron_snapshot!($workspace,{
".root" => insta::dynamic_redaction(|content, _content_path| {
content.as_str().unwrap().replace("\\", "/")
}),
".packages[].root" => insta::dynamic_redaction(|content, _content_path| {
content.as_str().unwrap().replace("\\", "/")
}),
});
}};
}
}

View File

@@ -1,34 +0,0 @@
---
source: crates/red_knot_workspace/src/workspace/metadata.rs
expression: "&workspace"
---
WorkspaceMetadata(
root: "/app",
packages: [
PackageMetadata(
name: Name("workspace-root"),
root: "/app",
configuration: Configuration(
python_version: None,
search_paths: SearchPathConfiguration(
extra_paths: None,
src_root: None,
typeshed: None,
site_packages: None,
),
),
),
],
settings: WorkspaceSettings(
program: ProgramSettings(
python_version: "3.9",
python_platform: all,
search_paths: SearchPathSettings(
extra_paths: [],
src_root: "/app",
typeshed: None,
site_packages: Known([]),
),
),
),
)

View File

@@ -1,34 +0,0 @@
---
source: crates/red_knot_workspace/src/workspace/metadata.rs
expression: workspace
---
WorkspaceMetadata(
root: "/app",
packages: [
PackageMetadata(
name: Name("workspace-root"),
root: "/app",
configuration: Configuration(
python_version: None,
search_paths: SearchPathConfiguration(
extra_paths: None,
src_root: None,
typeshed: None,
site_packages: None,
),
),
),
],
settings: WorkspaceSettings(
program: ProgramSettings(
python_version: "3.9",
python_platform: all,
search_paths: SearchPathSettings(
extra_paths: [],
src_root: "/app",
typeshed: None,
site_packages: Known([]),
),
),
),
)

View File

@@ -1,34 +0,0 @@
---
source: crates/red_knot_workspace/src/workspace/metadata.rs
expression: workspace
---
WorkspaceMetadata(
root: "/app",
packages: [
PackageMetadata(
name: Name("app"),
root: "/app",
configuration: Configuration(
python_version: None,
search_paths: SearchPathConfiguration(
extra_paths: None,
src_root: None,
typeshed: None,
site_packages: None,
),
),
),
],
settings: WorkspaceSettings(
program: ProgramSettings(
python_version: "3.9",
python_platform: all,
search_paths: SearchPathSettings(
extra_paths: [],
src_root: "/app",
typeshed: None,
site_packages: Known([]),
),
),
),
)

View File

@@ -1,34 +0,0 @@
---
source: crates/red_knot_workspace/src/workspace/metadata.rs
expression: workspace
---
WorkspaceMetadata(
root: "/app",
packages: [
PackageMetadata(
name: Name("backend"),
root: "/app",
configuration: Configuration(
python_version: None,
search_paths: SearchPathConfiguration(
extra_paths: None,
src_root: None,
typeshed: None,
site_packages: None,
),
),
),
],
settings: WorkspaceSettings(
program: ProgramSettings(
python_version: "3.9",
python_platform: all,
search_paths: SearchPathSettings(
extra_paths: [],
src_root: "/app",
typeshed: None,
site_packages: Known([]),
),
),
),
)

View File

@@ -1,47 +0,0 @@
---
source: crates/red_knot_workspace/src/workspace/metadata.rs
expression: workspace
---
WorkspaceMetadata(
root: "/app",
packages: [
PackageMetadata(
name: Name("workspace-root"),
root: "/app",
configuration: Configuration(
python_version: None,
search_paths: SearchPathConfiguration(
extra_paths: None,
src_root: None,
typeshed: None,
site_packages: None,
),
),
),
PackageMetadata(
name: Name("member-a"),
root: "/app/packages/a",
configuration: Configuration(
python_version: None,
search_paths: SearchPathConfiguration(
extra_paths: None,
src_root: None,
typeshed: None,
site_packages: None,
),
),
),
],
settings: WorkspaceSettings(
program: ProgramSettings(
python_version: "3.9",
python_platform: all,
search_paths: SearchPathSettings(
extra_paths: [],
src_root: "/app",
typeshed: None,
site_packages: Known([]),
),
),
),
)

View File

@@ -1,60 +0,0 @@
---
source: crates/red_knot_workspace/src/workspace/metadata.rs
expression: workspace
---
WorkspaceMetadata(
root: "/app",
packages: [
PackageMetadata(
name: Name("workspace-root"),
root: "/app",
configuration: Configuration(
python_version: None,
search_paths: SearchPathConfiguration(
extra_paths: None,
src_root: None,
typeshed: None,
site_packages: None,
),
),
),
PackageMetadata(
name: Name("member-a"),
root: "/app/packages/a",
configuration: Configuration(
python_version: None,
search_paths: SearchPathConfiguration(
extra_paths: None,
src_root: None,
typeshed: None,
site_packages: None,
),
),
),
PackageMetadata(
name: Name("member-x"),
root: "/app/packages/x",
configuration: Configuration(
python_version: None,
search_paths: SearchPathConfiguration(
extra_paths: None,
src_root: None,
typeshed: None,
site_packages: None,
),
),
),
],
settings: WorkspaceSettings(
program: ProgramSettings(
python_version: "3.9",
python_platform: all,
search_paths: SearchPathSettings(
extra_paths: [],
src_root: "/app",
typeshed: None,
site_packages: Known([]),
),
),
),
)

View File

@@ -1,7 +1,7 @@
use anyhow::{anyhow, Context};
use red_knot_python_semantic::{HasTy, SemanticModel};
use red_knot_workspace::db::RootDatabase;
use red_knot_workspace::workspace::WorkspaceMetadata;
use red_knot_workspace::db::ProjectDatabase;
use red_knot_workspace::project::ProjectMetadata;
use ruff_db::files::{system_path_to_file, File};
use ruff_db::parsed::parsed_module;
use ruff_db::system::{SystemPath, SystemPathBuf, TestSystem};
@@ -9,12 +9,12 @@ use ruff_python_ast::visitor::source_order;
use ruff_python_ast::visitor::source_order::SourceOrderVisitor;
use ruff_python_ast::{self as ast, Alias, Expr, Parameter, ParameterWithDefault, Stmt};
fn setup_db(workspace_root: &SystemPath, system: TestSystem) -> anyhow::Result<RootDatabase> {
let workspace = WorkspaceMetadata::discover(workspace_root, &system, None)?;
RootDatabase::new(workspace, system)
fn setup_db(workspace_root: &SystemPath, system: TestSystem) -> anyhow::Result<ProjectDatabase> {
let workspace = ProjectMetadata::discover(workspace_root, &system, None)?;
ProjectDatabase::new(workspace, system)
}
fn get_workspace_root() -> anyhow::Result<SystemPathBuf> {
fn get_cargo_workspace_root() -> anyhow::Result<SystemPathBuf> {
Ok(SystemPathBuf::from(String::from_utf8(
std::process::Command::new("cargo")
.args(["locate-project", "--workspace", "--message-format", "plain"])
@@ -35,7 +35,7 @@ fn corpus_no_panic() -> anyhow::Result<()> {
#[test]
fn parser_no_panic() -> anyhow::Result<()> {
let workspace_root = get_workspace_root()?;
let workspace_root = get_cargo_workspace_root()?;
run_corpus_tests(&format!(
"{workspace_root}/crates/ruff_python_parser/resources/**/*.py"
))
@@ -43,7 +43,7 @@ fn parser_no_panic() -> anyhow::Result<()> {
#[test]
fn linter_af_no_panic() -> anyhow::Result<()> {
let workspace_root = get_workspace_root()?;
let workspace_root = get_cargo_workspace_root()?;
run_corpus_tests(&format!(
"{workspace_root}/crates/ruff_linter/resources/test/fixtures/[a-f]*/**/*.py"
))
@@ -51,7 +51,7 @@ fn linter_af_no_panic() -> anyhow::Result<()> {
#[test]
fn linter_gz_no_panic() -> anyhow::Result<()> {
let workspace_root = get_workspace_root()?;
let workspace_root = get_cargo_workspace_root()?;
run_corpus_tests(&format!(
"{workspace_root}/crates/ruff_linter/resources/test/fixtures/[g-z]*/**/*.py"
))
@@ -60,7 +60,7 @@ fn linter_gz_no_panic() -> anyhow::Result<()> {
#[test]
#[ignore = "Enable running once there are fewer failures"]
fn linter_stubs_no_panic() -> anyhow::Result<()> {
let workspace_root = get_workspace_root()?;
let workspace_root = get_cargo_workspace_root()?;
run_corpus_tests(&format!(
"{workspace_root}/crates/ruff_linter/resources/test/fixtures/**/*.pyi"
))
@@ -69,7 +69,7 @@ fn linter_stubs_no_panic() -> anyhow::Result<()> {
#[test]
#[ignore = "Enable running over typeshed stubs once there are fewer failures"]
fn typeshed_no_panic() -> anyhow::Result<()> {
let workspace_root = get_workspace_root()?;
let workspace_root = get_cargo_workspace_root()?;
run_corpus_tests(&format!(
"{workspace_root}/crates/red_knot_vendored/vendor/typeshed/**/*.pyi"
))
@@ -85,7 +85,7 @@ fn run_corpus_tests(pattern: &str) -> anyhow::Result<()> {
let mut db = setup_db(&root, system.clone())?;
let workspace_root = get_workspace_root()?;
let workspace_root = get_cargo_workspace_root()?;
let workspace_root = workspace_root.to_string();
let corpus = glob::glob(pattern).context("Failed to compile pattern")?;
@@ -163,7 +163,7 @@ fn run_corpus_tests(pattern: &str) -> anyhow::Result<()> {
Ok(())
}
fn pull_types(db: &RootDatabase, file: File) {
fn pull_types(db: &ProjectDatabase, file: File) {
let mut visitor = PullTypesVisitor::new(db, file);
let ast = parsed_module(db, file);
@@ -176,7 +176,7 @@ struct PullTypesVisitor<'db> {
}
impl<'db> PullTypesVisitor<'db> {
fn new(db: &'db RootDatabase, file: File) -> Self {
fn new(db: &'db ProjectDatabase, file: File) -> Self {
Self {
model: SemanticModel::new(db, file),
}

View File

@@ -959,7 +959,7 @@ A `--config` flag must either be a path to a `.toml` configuration file
// We want to display the most helpful error to the user as possible.
if Path::new(value)
.extension()
.map_or(false, |ext| ext.eq_ignore_ascii_case("toml"))
.is_some_and(|ext| ext.eq_ignore_ascii_case("toml"))
{
if !value.contains('=') {
tip.push_str(&format!(

View File

@@ -623,7 +623,7 @@ fn stdin_override_parser_py() {
fn stdin_fix_when_not_fixable_should_still_print_contents() {
let mut cmd = RuffCheck::default().args(["--fix"]).build();
assert_cmd_snapshot!(cmd
.pass_stdin("import os\nimport sys\n\nif (1, 2):\n print(sys.version)\n"), @r"
.pass_stdin("import os\nimport sys\n\nif (1, 2):\n print(sys.version)\n"), @r###"
success: false
exit_code: 1
----- stdout -----
@@ -636,14 +636,14 @@ fn stdin_fix_when_not_fixable_should_still_print_contents() {
-:3:4: F634 If test is a tuple, which is always `True`
|
1 | import sys
2 |
2 |
3 | if (1, 2):
| ^^^^^^ F634
4 | print(sys.version)
|
Found 2 errors (1 fixed, 1 remaining).
");
"###);
}
#[test]

View File

@@ -810,6 +810,8 @@ fn value_given_to_table_key_is_not_inline_table_1() {
- `lint.flake8-pytest-style.raises-require-match-for`
- `lint.flake8-pytest-style.raises-extend-require-match-for`
- `lint.flake8-pytest-style.mark-parentheses`
- `lint.flake8-pytest-style.warns-require-match-for`
- `lint.flake8-pytest-style.warns-extend-require-match-for`
For more information, try '--help'.
"#);
@@ -2070,7 +2072,7 @@ fn flake8_import_convention_invalid_aliases_config_alias_name() -> Result<()> {
.args(STDIN_BASE_OPTIONS)
.arg("--config")
.arg(&ruff_toml)
, @r###"
, @r#"
success: false
exit_code: 2
----- stdout -----
@@ -2078,12 +2080,12 @@ fn flake8_import_convention_invalid_aliases_config_alias_name() -> Result<()> {
----- stderr -----
ruff failed
Cause: Failed to parse [TMP]/ruff.toml
Cause: TOML parse error at line 2, column 2
Cause: TOML parse error at line 3, column 17
|
2 | [lint.flake8-import-conventions.aliases]
| ^^^^
3 | "module.name" = "invalid.alias"
| ^^^^^^^^^^^^^^^
invalid value: string "invalid.alias", expected a Python identifier
"###);});
"#);});
Ok(())
}
@@ -2106,7 +2108,7 @@ fn flake8_import_convention_invalid_aliases_config_extend_alias_name() -> Result
.args(STDIN_BASE_OPTIONS)
.arg("--config")
.arg(&ruff_toml)
, @r###"
, @r#"
success: false
exit_code: 2
----- stdout -----
@@ -2114,12 +2116,12 @@ fn flake8_import_convention_invalid_aliases_config_extend_alias_name() -> Result
----- stderr -----
ruff failed
Cause: Failed to parse [TMP]/ruff.toml
Cause: TOML parse error at line 2, column 2
Cause: TOML parse error at line 3, column 17
|
2 | [lint.flake8-import-conventions.extend-aliases]
| ^^^^
3 | "module.name" = "__debug__"
| ^^^^^^^^^^^
invalid value: string "__debug__", expected an assignable Python identifier
"###);});
"#);});
Ok(())
}
@@ -2142,7 +2144,7 @@ fn flake8_import_convention_invalid_aliases_config_module_name() -> Result<()> {
.args(STDIN_BASE_OPTIONS)
.arg("--config")
.arg(&ruff_toml)
, @r###"
, @r#"
success: false
exit_code: 2
----- stdout -----
@@ -2150,11 +2152,11 @@ fn flake8_import_convention_invalid_aliases_config_module_name() -> Result<()> {
----- stderr -----
ruff failed
Cause: Failed to parse [TMP]/ruff.toml
Cause: TOML parse error at line 2, column 2
Cause: TOML parse error at line 3, column 1
|
2 | [lint.flake8-import-conventions.aliases]
| ^^^^
3 | "module..invalid" = "alias"
| ^^^^^^^^^^^^^^^^^
invalid value: string "module..invalid", expected a sequence of Python identifiers delimited by periods
"###);});
"#);});
Ok(())
}

View File

@@ -0,0 +1,37 @@
[package]
name = "ruff_annotate_snippets"
version = "0.1.0"
publish = false
authors = { workspace = true }
edition = { workspace = true }
rust-version = { workspace = true }
homepage = { workspace = true }
documentation = { workspace = true }
repository = { workspace = true }
license = "MIT OR Apache-2.0"
[lib]
[features]
default = []
testing-colors = []
[dependencies]
anstyle = { workspace = true }
memchr = { workspace = true }
unicode-width = { workspace = true }
[dev-dependencies]
ruff_annotate_snippets = { workspace = true, features = ["testing-colors"] }
anstream = { workspace = true }
serde = { workspace = true, features = ["derive"] }
snapbox = { workspace = true, features = ["diff", "term-svg", "cmd", "examples"] }
toml = { workspace = true }
tryfn = { workspace = true }
[[test]]
name = "fixtures"
harness = false
[lints]
workspace = true

View File

@@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "{}"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright {yyyy} {name of copyright owner}
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

View File

@@ -0,0 +1,19 @@
Copyright (c) Individual contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

View File

@@ -0,0 +1,15 @@
This is a fork of the [`annotate-snippets` crate]. The principle motivation for
this fork, at the time of writing, is [issue #167]. Specifically, we wanted to
upgrade our version of `annotate-snippets`, but do so _without_ changing our
diagnostic message format.
This copy of `annotate-snippets` is basically identical to upstream, but with
an extra `Level::None` variant that permits skipping over a new non-optional
header emitted by `annotate-snippets`.
More generally, it seems plausible that we may want to tweak other aspects of
the output format in the future, so it might make sense to stick with our own
copy so that we can be masters of our own destiny.
[issue #167]: https://github.com/rust-lang/annotate-snippets-rs/issues/167
[`annotate-snippets` crate]: https://github.com/rust-lang/annotate-snippets-rs

Some files were not shown because too many files have changed in this diff Show More