Compare commits

..

51 Commits

Author SHA1 Message Date
Charlie Marsh
f3baec49df Remove from build 2024-09-20 17:28:58 -04:00
Charlie Marsh
e6fde89e26 Disable zstd on powerpc64 2024-09-20 17:26:50 -04:00
Charlie Marsh
7579a792c7 Add test coverage for non-Python globs (#13430) 2024-09-20 20:46:00 +00:00
Charlie Marsh
0bbc138037 Upgrade to latest cargo-dist version (#13416)
## Summary

Follows https://github.com/astral-sh/uv/pull/7092.
2024-09-20 15:59:32 -04:00
Charlie Marsh
ff11db61b4 Add Python version support to ruff analyze CLI (#13426) 2024-09-20 15:40:47 -04:00
Charlie Marsh
2823487bf8 Respect lint.exclude in ruff check --add-noqa (#13427)
## Summary

Closes https://github.com/astral-sh/ruff/issues/13423.
2024-09-20 19:39:36 +00:00
Charlie Marsh
910fac781d Add exclude support to ruff analyze (#13425)
## Summary

Closes https://github.com/astral-sh/ruff/issues/13424.
2024-09-20 15:34:35 -04:00
Carl Meyer
149fb2090e [red-knot] more efficient UnionBuilder::add (#13411)
Avoid quadratic time in subsumed elements when adding a super-type of
existing union elements.

Reserve space in advance when adding multiple elements (from another
union) to a union.

Make union elements a `Box<[Type]>` instead of an `FxOrderSet`; the set
doesn't buy much since the rules of union uniqueness are defined in
terms of supertype/subtype, not in terms of simple type identity.

Move sealed-boolean handling out of a separate `UnionBuilder::simplify`
method and into `UnionBuilder::add`; now that `add` is iterating
existing elements anyway, this is more efficient.

Remove `UnionType::contains`, since it's now `O(n)` and we shouldn't
really need it, generally we care about subtype/supertype, not type
identity. (Right now it's used for `Type::Unbound`, which shouldn't even
be a type.)

Add support for `is_subtype_of` for the `object` type.

Addresses comments on https://github.com/astral-sh/ruff/pull/13401
2024-09-20 10:49:45 -07:00
Carl Meyer
40c65dcfa7 [red-knot] dedicated error message for all-union-elements not callable (#13412)
This was mentioned in an earlier review, and seemed easy enough to just
do it. No need to repeat all the types twice when it gives no additional
information.
2024-09-20 08:08:43 -07:00
yahayaohinoyi
03f3a4e855 [pycodestyle] Fix: Don't autofix if the first line ends in a question mark? (D400) (#13399)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-09-20 11:05:26 +00:00
Micha Reiser
531ebf6dff Fix parentheses around return type annotations (#13381) 2024-09-20 09:23:53 +02:00
Rupert Tombs
7c2011599f Correct Some value is incorrect (#13418) 2024-09-20 08:25:58 +02:00
Charlie Marsh
17e90823da Some minor internal refactors for module graph (#13417) 2024-09-20 00:21:30 -04:00
Charlie Marsh
d01cbf7f8f Bump version to v0.6.6 (#13415) 2024-09-19 23:09:57 -04:00
Charlie Marsh
770b276c21 Cache glob resolutions in import graph (#13413)
## Summary

These are often repeated; caching the resolutions can have a huge
impact.
2024-09-20 02:24:06 +00:00
Charlie Marsh
4e935f7d7d Add a subcommand to generate dependency graphs (#13402)
## Summary

This PR adds an experimental Ruff subcommand to generate dependency
graphs based on module resolution.

A few highlights:

- You can generate either dependency or dependent graphs via the
`--direction` command-line argument.
- Like Pants, we also provide an option to identify imports from string
literals (`--detect-string-imports`).
- Users can also provide additional dependency data via the
`include-dependencies` key under `[tool.ruff.import-map]`. This map uses
file paths as keys, and lists of strings as values. Those strings can be
file paths or globs.

The dependency resolution uses the red-knot module resolver which is
intended to be fully spec compliant, so it's also a chance to expose the
module resolver in a real-world setting.

The CLI is, e.g., `ruff graph build ../autobot`, which will output a
JSON map from file to files it depends on for the `autobot` project.
2024-09-19 21:06:32 -04:00
Carl Meyer
260c2ecd15 [red-knot] visit with-item vars even if not a Name (#13409)
This fixes the last panic on checking pandas.

(Match statement became an `if let` because clippy decided it wanted
that once I added the additional line in the else case?)

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-09-19 10:37:49 -07:00
Dylan
f110d80279 [refurb] Skip slice-to-remove-prefix-or-suffix (FURB188) when nontrivial slice step is present (#13405) 2024-09-19 12:47:17 -04:00
Carl Meyer
a6d3d2fccd [red-knot] support reveal_type as pseudo-builtin (#13403)
Support using `reveal_type` without importing it, as implied by the type
spec and supported by existing type checkers.

We use `typing_extensions.reveal_type` for the implicit built-in; this
way it exists on all Python versions. (It imports from `typing` on newer
Python versions.)

Emits an "undefined name" diagnostic whenever `reveal_type` is
referenced in this way (in addition to the revealed-type diagnostic when
it is called). This follows the mypy example (with `--enable-error-code
unimported-reveal`) and I think provides a good (and easily
understandable) balance for user experience. If you are using
`reveal_type` for quick temporary debugging, the additional
undefined-name diagnostic doesn't hinder that use case. If we make the
revealed-type diagnostic a non-failing one, the undefined-name
diagnostic can still be a failing diagnostic, helping prevent
accidentally leaving it in place. For any use cases where you want to
leave it in place, you can always import it to avoid the undefined-name
diagnostic.

In the future, we can easily provide configuration options to a) turn
off builtin-reveal_type altogether, and/or b) silence the undefined-name
diagnostic when using it, if we have users on either side (loving or
hating pseudo-builtin `reveal_type`) who are dissatisfied with this
compromise.
2024-09-19 07:58:08 -07:00
Micha Reiser
afdb659111 Fix off-by one error in the LineIndex::offset calculation (#13407) 2024-09-19 11:58:45 +00:00
Simon
a8d9104fa3 Fix/#13070 defer annotations when future is active (#13395) 2024-09-19 10:13:37 +02:00
Micha Reiser
d3530ab997 Fix rendering of FURB188 docs (#13406) 2024-09-19 07:29:31 +00:00
Carl Meyer
cf1e91bb59 [red-knot] simplify subtypes from unions (#13401)
Add `Type::is_subtype_of` method, and simplify subtypes out of unions.
2024-09-18 22:06:39 -07:00
Carl Meyer
125eaafae0 [red-knot] inferred type, not Unknown, for undeclared paths (#13400)
After looking at more cases (for example, the case in the added test in
this PR), I realized that our previous rule, "if a symbol has any
declarations, use only declarations for its public type" is not
adequate. Rather than using `Unknown` as fallback if the symbol is not
declared in some paths, we need to use the inferred type as fallback in
that case.

For the paths where the symbol _was_ declared, we know that any bindings
must be assignable to the declared type in that path, so this won't
change the overall declared type in those paths. But for paths where the
symbol wasn't declared, this will give us a better type in place of
`Unknown`.
2024-09-18 21:47:49 -07:00
Carl Meyer
7aae80903c [red-knot] add support for typing_extensions.reveal_type (#13397)
Before `typing.reveal_type` existed, there was
`typing_extensions.reveal_type`. We should support both.

Also adds a test to verify that we can handle aliasing of `reveal_type`
to a different name.

Adds a bit of code to ensure that if we have a union of different
`reveal_type` functions (e.g. a union containing both
`typing_extensions.reveal_type` and `typing.reveal_type`) we still emit
the reveal-type diagnostic only once. This is probably unlikely in
practice, but it doesn't hurt to handle it smoothly. (It comes up now
because we don't support `version_info` checks yet, so
`typing_extensions.reveal_type` is actually that union.)

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-09-18 21:39:03 -07:00
Carl Meyer
4aca9b91ba [red-knot] consider imports to be declarations (#13398)
I noticed that this pattern sometimes occurs in typeshed:
```
if ...:
    from foo import bar
else:
    def bar(): ...
```

If we have the rule that symbols with declarations only use declarations
for the public type, then this ends up resolving as `Unknown |
Literal[bar]`, because we didn't consider the import to be a
declaration.

I think the most straightforward thing here is to also consider imports
as declarations. The same rationale applies as for function and class
definitions: if you shadow an import, you should have to explicitly
shadow with an annotation, rather than just doing it
implicitly/accidentally.

We may also ultimately need to re-evaluate the rule that public type
considers only declarations, if there are declarations.
2024-09-18 20:59:03 -07:00
Hamir Mahal
8b3da1867e refactor: remove unnecessary string hashes (#13250) 2024-09-18 19:08:59 +02:00
Carl Meyer
c173ec5bc7 [red-knot] support for typing.reveal_type (#13384)
Add support for the `typing.reveal_type` function, emitting a diagnostic
revealing the type of its single argument. This is a necessary piece for
the planned testing framework.

This puts the cart slightly in front of the horse, in that we don't yet
have proper support for validating call signatures / argument types. But
it's easy to do just enough to make `reveal_type` work.

This PR includes support for calling union types (this is necessary
because we don't yet support `sys.version_info` checks, so
`typing.reveal_type` itself is a union type), plus some nice
consolidated error messages for calls to unions where some elements are
not callable. This is mostly to demonstrate the flexibility in
diagnostics that we get from the `CallOutcome` enum.
2024-09-18 09:59:51 -07:00
Charlie Marsh
44d916fb4e Respect FastAPI aliases in route definitions (#13394)
## Summary

Closes https://github.com/astral-sh/ruff/issues/13263
2024-09-18 12:06:49 -04:00
Micha Reiser
4eb849aed3 Update the revisions of the formatter stability check projects (#13380) 2024-09-18 08:26:40 +02:00
Micha Reiser
6ac61d7b89 Fix placement of inline parameter comments (#13379) 2024-09-18 08:26:06 +02:00
renovate[bot]
c7b2e336f0 Update dependency vite to v5.4.6 (#13385)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-09-18 08:25:49 +02:00
Charlie Marsh
70748950ae Respect word boundaries when detecting function signature in docs (#13388)
## Summary

Closes https://github.com/astral-sh/ruff/issues/13242.
2024-09-18 00:01:38 -04:00
Carl Meyer
dcfebaa4a8 [red-knot] use declared types in inference/checking (#13335)
Use declared types in inference and checking. This means several things:

* Imports prefer declarations over inference, when declarations are
available.
* When we encounter a binding, we check that the bound value's inferred
type is assignable to the live declarations of the bound symbol, if any.
* When we encounter a declaration, we check that the declared type is
assignable from the inferred type of the symbol from previous bindings,
if any.
* When we encounter a binding+declaration, we check that the inferred
type of the bound value is assignable to the declared type.
2024-09-17 08:11:06 -07:00
Micha Reiser
d86e5ad031 Update Black tests (#13375) 2024-09-17 11:16:50 +02:00
Simon Brugman
bb12fe9d0c DOCS: navigate back to rule overview linter (#13368) 2024-09-16 16:21:26 +00:00
Micha Reiser
3b57faf19b Fix build of ruff_benchmark on NixOS (#13366) 2024-09-16 09:41:46 +02:00
renovate[bot]
c9f7c3d652 Update dependency react-resizable-panels to v2.1.3 (#13360)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-09-16 07:38:21 +00:00
Micha Reiser
489dbbaadc Add diagnostics panel and navigation features to playground (#13357) 2024-09-16 07:34:46 +00:00
renovate[bot]
47e9ea2d5d Update pre-commit hook astral-sh/ruff-pre-commit to v0.6.5 (#13362)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-09-16 09:34:14 +02:00
renovate[bot]
7919a7122a Update NPM Development dependencies (#13363)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-09-16 09:32:56 +02:00
renovate[bot]
a70d693b1c Update dependency ruff to v0.6.5 (#13361)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-09-16 09:32:06 +02:00
github-actions[bot]
1365b0806d Sync vendored typeshed stubs (#13355)
Close and reopen this PR to trigger CI

Co-authored-by: typeshedbot <>
2024-09-14 20:40:42 -04:00
Alex Waygood
f4de49ab37 [red-knot] Clarify how scopes are pushed and popped for comprehensions and generator expressions (#13353) 2024-09-14 13:31:17 -04:00
François-Michel L'Heureux
8b49845537 Fix documentation for editor vim plugin ALE (#13348)
The documented configuration did not work. On failure, ALE suggest to
run `ALEFixSuggest`, into with it documents the working configuration
key

'ruff_format' - Fix python files with the ruff formatter.

Fix an inaccuracy in the documentation, regarding the ALE plugin for the
Vim text editor.
2024-09-13 23:27:17 +05:30
Carl Meyer
d988204b1b [red-knot] add Declarations support to semantic indexing (#13334)
Add support for declared types to the semantic index. This involves a
lot of renaming to clarify the distinction between bindings and
declarations. The Definition (or more specifically, the DefinitionKind)
becomes responsible for determining which definitions are bindings,
which are declarations, and which are both, and the symbol table
building is refactored a bit so that the `IS_BOUND` (renamed from
`IS_DEFINED` for consistent terminology) flag is always set when a
binding is added, rather than being set separately (and requiring us to
ensure it is set properly).

The `SymbolState` is split into two parts, `SymbolBindings` and
`SymbolDeclarations`, because we need to store live bindings for every
declaration and live declarations for every binding; the split lets us
do this without storing more than we need.

The massive doc comment in `use_def.rs` is updated to reflect bindings
vs declarations.

The `UseDefMap` gains some new APIs which are allow-unused for now,
since this PR doesn't yet update type inference to take declarations
into account.
2024-09-13 13:55:22 -04:00
Dhruv Manilawala
8558126df1 Bump version to 0.6.5 (#13346) 2024-09-13 20:12:26 +05:30
Dhruv Manilawala
9bd9981e70 Create insta snapshot for SARIF output (#13345)
## Summary

Follow-up from #13268, this PR updates the test case to use
`assert_snapshot` now that the output is limited to only include the
rules with diagnostics.

## Test Plan

`cargo insta test`
2024-09-13 14:35:45 +00:00
Micha Reiser
21bfab9b69 Playground: Add Copy as pyproject.toml/ruff.toml and paste from TOML (#13328) 2024-09-13 13:44:24 +01:00
Carl Meyer
43a5922f6f [red-knot] add BitSet::is_empty and BitSet::union (#13333)
Add `::is_empty` and `::union` methods to the `BitSet` implementation.

Allowing unused for now, until these methods become used later with the
declared-types implementation.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-09-12 14:25:45 -04:00
Carl Meyer
175d067250 [red-knot] add initial Type::is_equivalent_to and Type::is_assignable_to (#13332)
These are quite incomplete, but I needed to start stubbing them out in
order to build and test declared-types.

Allowing unused for now, until they are used later in the declared-types
PR.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-09-12 14:15:25 -04:00
188 changed files with 9630 additions and 1435 deletions

View File

@@ -1,3 +1,5 @@
# This file was autogenerated by cargo-dist: https://opensource.axo.dev/cargo-dist/
#
# Copyright 2022-2024, axodotdev
# SPDX-License-Identifier: MIT or Apache-2.0
#
@@ -64,7 +66,7 @@ jobs:
# we specify bash to get pipefail; it guards against the `curl` command
# failing. otherwise `sh` won't catch that `curl` returned non-0
shell: bash
run: "curl --proto '=https' --tlsv1.2 -LsSf https://github.com/axodotdev/cargo-dist/releases/download/v0.18.0/cargo-dist-installer.sh | sh"
run: "curl --proto '=https' --tlsv1.2 -LsSf https://github.com/axodotdev/cargo-dist/releases/download/v0.22.1/cargo-dist-installer.sh | sh"
- name: Cache cargo-dist
uses: actions/upload-artifact@v4
with:

View File

@@ -59,7 +59,7 @@ repos:
pass_filenames: false # This makes it a lot faster
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.6.4
rev: v0.6.5
hooks:
- id: ruff-format
- id: ruff

View File

@@ -1,5 +1,61 @@
# Changelog
## 0.6.6
### Preview features
- \[`refurb`\] Skip `slice-to-remove-prefix-or-suffix` (`FURB188`) when non-trivial slice steps are present ([#13405](https://github.com/astral-sh/ruff/pull/13405))
- Add a subcommand to generate dependency graphs ([#13402](https://github.com/astral-sh/ruff/pull/13402))
### Formatter
- Fix placement of inline parameter comments ([#13379](https://github.com/astral-sh/ruff/pull/13379))
### Server
- Fix off-by one error in the `LineIndex::offset` calculation ([#13407](https://github.com/astral-sh/ruff/pull/13407))
### Bug fixes
- \[`fastapi`\] Respect FastAPI aliases in route definitions ([#13394](https://github.com/astral-sh/ruff/pull/13394))
- \[`pydocstyle`\] Respect word boundaries when detecting function signature in docs ([#13388](https://github.com/astral-sh/ruff/pull/13388))
### Documentation
- Add backlinks to rule overview linter ([#13368](https://github.com/astral-sh/ruff/pull/13368))
- Fix documentation for editor vim plugin ALE ([#13348](https://github.com/astral-sh/ruff/pull/13348))
- Fix rendering of `FURB188` docs ([#13406](https://github.com/astral-sh/ruff/pull/13406))
## 0.6.5
### Preview features
- \[`pydoclint`\] Ignore `DOC201` when function name is "**new**" ([#13300](https://github.com/astral-sh/ruff/pull/13300))
- \[`refurb`\] Implement `slice-to-remove-prefix-or-suffix` (`FURB188`) ([#13256](https://github.com/astral-sh/ruff/pull/13256))
### Rule changes
- \[`eradicate`\] Ignore script-comments with multiple end-tags (`ERA001`) ([#13283](https://github.com/astral-sh/ruff/pull/13283))
- \[`pyflakes`\] Improve error message for `UndefinedName` when a builtin was added in a newer version than specified in Ruff config (`F821`) ([#13293](https://github.com/astral-sh/ruff/pull/13293))
### Server
- Add support for extensionless Python files for server ([#13326](https://github.com/astral-sh/ruff/pull/13326))
- Fix configuration inheritance for configurations specified in the LSP settings ([#13285](https://github.com/astral-sh/ruff/pull/13285))
### Bug fixes
- \[`ruff`\] Handle unary operators in `decimal-from-float-literal` (`RUF032`) ([#13275](https://github.com/astral-sh/ruff/pull/13275))
### CLI
- Only include rules with diagnostics in SARIF metadata ([#13268](https://github.com/astral-sh/ruff/pull/13268))
### Playground
- Add "Copy as pyproject.toml/ruff.toml" and "Paste from TOML" ([#13328](https://github.com/astral-sh/ruff/pull/13328))
- Fix errors not shown for restored snippet on page load ([#13262](https://github.com/astral-sh/ruff/pull/13262))
## 0.6.4
### Preview features

222
Cargo.lock generated
View File

@@ -161,6 +161,21 @@ version = "0.7.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "96d30a06541fbafbc7f82ed10c06164cfbd2c401138f6addd8404629c4b16711"
[[package]]
name = "assert_fs"
version = "1.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7efdb1fdb47602827a342857666feb372712cbc64b414172bd6b167a02927674"
dependencies = [
"anstyle",
"doc-comment",
"globwalk",
"predicates",
"predicates-core",
"predicates-tree",
"tempfile",
]
[[package]]
name = "autocfg"
version = "1.2.0"
@@ -194,6 +209,15 @@ version = "2.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b048fb63fd8b5923fc5aa7b340d8e156aec7ec02f0c78fa8a6ddc2613f6f71de"
[[package]]
name = "block-buffer"
version = "0.10.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3078c7629b62d3f0439517fa394996acacc5cbc91c5a20d8c658e77abd503a71"
dependencies = [
"generic-array",
]
[[package]]
name = "bstr"
version = "1.10.0"
@@ -231,6 +255,9 @@ name = "camino"
version = "1.1.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8b96ec4966b5813e2c0507c1f86115c8c5abaadc3980879c3424042a02fd1ad3"
dependencies = [
"serde",
]
[[package]]
name = "cast"
@@ -511,6 +538,15 @@ dependencies = [
"rustc-hash 1.1.0",
]
[[package]]
name = "cpufeatures"
version = "0.2.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "51e852e6dc9a5bed1fae92dd2375037bf2b768725bf3be87811edee3249d09ad"
dependencies = [
"libc",
]
[[package]]
name = "crc32fast"
version = "1.4.0"
@@ -616,6 +652,16 @@ version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7a81dae078cea95a014a339291cec439d2f232ebe854a9d672b796c6afafa9b7"
[[package]]
name = "crypto-common"
version = "0.1.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1bfb12502f3fc46cca1bb51ac28df9d618d813cdc3d2f25b9fe775a34af26bb3"
dependencies = [
"generic-array",
"typenum",
]
[[package]]
name = "ctrlc"
version = "3.4.5"
@@ -694,6 +740,22 @@ version = "0.1.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "56254986775e3233ffa9c4d7d3faaf6d36a2c09d30b20687e9f88bc8bafc16c8"
[[package]]
name = "difflib"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6184e33543162437515c2e2b48714794e37845ec9851711914eec9d308f6ebe8"
[[package]]
name = "digest"
version = "0.10.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9ed9a281f7bc9b7576e61468ba615a66a5c8cfdff42420a70aa82701a3b1e292"
dependencies = [
"block-buffer",
"crypto-common",
]
[[package]]
name = "dirs"
version = "4.0.0"
@@ -735,6 +797,12 @@ dependencies = [
"windows-sys 0.48.0",
]
[[package]]
name = "doc-comment"
version = "0.3.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fea41bba32d969b513997752735605054bc0dfa92b4c56bf1189f2e174be7a10"
[[package]]
name = "drop_bomb"
version = "0.1.5"
@@ -879,6 +947,16 @@ dependencies = [
"libc",
]
[[package]]
name = "generic-array"
version = "0.14.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "85649ca51fd72272d7821adaf274ad91c288277713d9c18820d8499a7ff69e9a"
dependencies = [
"typenum",
"version_check",
]
[[package]]
name = "getopts"
version = "0.2.21"
@@ -920,6 +998,17 @@ dependencies = [
"regex-syntax 0.8.3",
]
[[package]]
name = "globwalk"
version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0bf760ebf69878d9fd8f110c89703d90ce35095324d1f1edcb595c63945ee757"
dependencies = [
"bitflags 2.6.0",
"ignore",
"walkdir",
]
[[package]]
name = "half"
version = "2.4.1"
@@ -1112,6 +1201,8 @@ dependencies = [
"globset",
"lazy_static",
"linked-hash-map",
"pest",
"pest_derive",
"regex",
"serde",
"similar",
@@ -1707,6 +1798,51 @@ version = "2.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e3148f5046208a5d56bcfc03053e3ca6334e51da8dfb19b6cdc8b306fae3283e"
[[package]]
name = "pest"
version = "2.7.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cd53dff83f26735fdc1ca837098ccf133605d794cdae66acfc2bfac3ec809d95"
dependencies = [
"memchr",
"thiserror",
"ucd-trie",
]
[[package]]
name = "pest_derive"
version = "2.7.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2a548d2beca6773b1c244554d36fcf8548a8a58e74156968211567250e48e49a"
dependencies = [
"pest",
"pest_generator",
]
[[package]]
name = "pest_generator"
version = "2.7.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3c93a82e8d145725dcbaf44e5ea887c8a869efdcc28706df2d08c69e17077183"
dependencies = [
"pest",
"pest_meta",
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "pest_meta"
version = "2.7.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a941429fea7e08bedec25e4f6785b6ffaacc6b755da98df5ef3e7dcf4a124c4f"
dependencies = [
"once_cell",
"pest",
"sha2",
]
[[package]]
name = "phf"
version = "0.11.2"
@@ -1769,6 +1905,33 @@ version = "0.2.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5b40af805b3121feab8a3c29f04d8ad262fa8e0561883e7653e024ae4479e6de"
[[package]]
name = "predicates"
version = "3.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7e9086cc7640c29a356d1a29fd134380bee9d8f79a17410aa76e7ad295f42c97"
dependencies = [
"anstyle",
"difflib",
"predicates-core",
]
[[package]]
name = "predicates-core"
version = "1.0.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ae8177bee8e75d6846599c6b9ff679ed51e882816914eec639944d7c9aa11931"
[[package]]
name = "predicates-tree"
version = "1.0.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41b740d195ed3166cd147c8047ec98db0e22ec019eb8eeb76d343b795304fb13"
dependencies = [
"predicates-core",
"termtree",
]
[[package]]
name = "pretty_assertions"
version = "1.4.0"
@@ -2092,10 +2255,11 @@ dependencies = [
[[package]]
name = "ruff"
version = "0.6.4"
version = "0.6.6"
dependencies = [
"anyhow",
"argfile",
"assert_fs",
"bincode",
"bitflags 2.6.0",
"cachedir",
@@ -2105,7 +2269,9 @@ dependencies = [
"clearscreen",
"colored",
"filetime",
"globwalk",
"ignore",
"indoc",
"insta",
"insta-cmd",
"is-macro",
@@ -2117,7 +2283,9 @@ dependencies = [
"rayon",
"regex",
"ruff_cache",
"ruff_db",
"ruff_diagnostics",
"ruff_graph",
"ruff_linter",
"ruff_macros",
"ruff_notebook",
@@ -2200,6 +2368,7 @@ dependencies = [
"ruff_text_size",
"rustc-hash 2.0.0",
"salsa",
"serde",
"tempfile",
"thiserror",
"tracing",
@@ -2275,6 +2444,23 @@ dependencies = [
"unicode-width",
]
[[package]]
name = "ruff_graph"
version = "0.1.0"
dependencies = [
"anyhow",
"clap",
"red_knot_python_semantic",
"ruff_cache",
"ruff_db",
"ruff_linter",
"ruff_macros",
"ruff_python_ast",
"salsa",
"schemars",
"serde",
]
[[package]]
name = "ruff_index"
version = "0.0.0"
@@ -2285,7 +2471,7 @@ dependencies = [
[[package]]
name = "ruff_linter"
version = "0.6.4"
version = "0.6.6"
dependencies = [
"aho-corasick",
"annotate-snippets 0.9.2",
@@ -2605,7 +2791,7 @@ dependencies = [
[[package]]
name = "ruff_wasm"
version = "0.6.4"
version = "0.6.6"
dependencies = [
"console_error_panic_hook",
"console_log",
@@ -2648,6 +2834,7 @@ dependencies = [
"regex",
"ruff_cache",
"ruff_formatter",
"ruff_graph",
"ruff_linter",
"ruff_macros",
"ruff_python_ast",
@@ -2936,6 +3123,17 @@ dependencies = [
"syn",
]
[[package]]
name = "sha2"
version = "0.10.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "793db75ad2bcafc3ffa7c68b215fee268f537982cd901d132f89c6343f3a3dc8"
dependencies = [
"cfg-if",
"cpufeatures",
"digest",
]
[[package]]
name = "sharded-slab"
version = "0.1.7"
@@ -3091,6 +3289,12 @@ dependencies = [
"phf_codegen",
]
[[package]]
name = "termtree"
version = "0.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3369f5ac52d5eb6ab48c6b4ffdc8efbcad6b89c765749064ba298f2c68a16a76"
[[package]]
name = "test-case"
version = "3.3.1"
@@ -3336,6 +3540,18 @@ version = "2.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6af6ae20167a9ece4bcb41af5b80f8a1f1df981f6391189ce00fd257af04126a"
[[package]]
name = "typenum"
version = "1.17.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "42ff0bf0c66b8238c6f3b578df37d0b7848e55df8577b3f74f92a69acceeb825"
[[package]]
name = "ucd-trie"
version = "0.1.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed646292ffc8188ef8ea4d1e0e0150fb15a5c2e12ad9b8fc191ae7a8a7f3c4b9"
[[package]]
name = "unic-char-property"
version = "0.9.0"

View File

@@ -17,6 +17,7 @@ ruff_cache = { path = "crates/ruff_cache" }
ruff_db = { path = "crates/ruff_db" }
ruff_diagnostics = { path = "crates/ruff_diagnostics" }
ruff_formatter = { path = "crates/ruff_formatter" }
ruff_graph = { path = "crates/ruff_graph" }
ruff_index = { path = "crates/ruff_index" }
ruff_linter = { path = "crates/ruff_linter" }
ruff_macros = { path = "crates/ruff_macros" }
@@ -42,6 +43,7 @@ red_knot_workspace = { path = "crates/red_knot_workspace" }
aho-corasick = { version = "1.1.3" }
annotate-snippets = { version = "0.9.2", features = ["color"] }
anyhow = { version = "1.0.80" }
assert_fs = { version = "1.1.0" }
argfile = { version = "0.2.0" }
bincode = { version = "1.3.3" }
bitflags = { version = "2.5.0" }
@@ -68,6 +70,7 @@ fern = { version = "0.6.1" }
filetime = { version = "0.2.23" }
glob = { version = "0.3.1" }
globset = { version = "0.4.14" }
globwalk = { version = "0.9.1" }
hashbrown = "0.14.3"
ignore = { version = "0.4.22" }
imara-diff = { version = "0.1.5" }
@@ -230,9 +233,9 @@ inherits = "release"
# Config for 'cargo dist'
[workspace.metadata.dist]
# The preferred cargo-dist version to use in CI (Cargo.toml SemVer syntax)
cargo-dist-version = "0.18.0"
cargo-dist-version = "0.22.1"
# CI backends to support
ci = ["github"]
ci = "github"
# The installers to generate for each app
installers = ["shell", "powershell"]
# The archive format to use for windows builds (defaults .zip)
@@ -263,11 +266,11 @@ targets = [
auto-includes = false
# Whether cargo-dist should create a GitHub Release or use an existing draft
create-release = true
# Publish jobs to run in CI
# Which actions to run on pull requests
pr-run-mode = "skip"
# Whether CI should trigger releases with dispatches instead of tag pushes
dispatch-releases = true
# The stage during which the GitHub Release should be created
# Which phase cargo-dist should use to create the GitHub release
github-release = "announce"
# Whether CI should include auto-generated code to build local artifacts
build-local-artifacts = false
@@ -275,9 +278,11 @@ build-local-artifacts = false
local-artifacts-jobs = ["./build-binaries", "./build-docker"]
# Publish jobs to run in CI
publish-jobs = ["./publish-pypi", "./publish-wasm"]
# Announcement jobs to run in CI
# Post-announce jobs to run in CI
post-announce-jobs = ["./notify-dependents", "./publish-docs", "./publish-playground"]
# Custom permissions for GitHub Jobs
github-custom-job-permissions = { "build-docker" = { packages = "write", contents = "read" }, "publish-wasm" = { contents = "read", id-token = "write", packages = "write" } }
# Whether to install an updater program
install-updater = false
# Path that installers should place binaries in
install-path = "CARGO_HOME"

View File

@@ -136,8 +136,8 @@ curl -LsSf https://astral.sh/ruff/install.sh | sh
powershell -c "irm https://astral.sh/ruff/install.ps1 | iex"
# For a specific version.
curl -LsSf https://astral.sh/ruff/0.6.4/install.sh | sh
powershell -c "irm https://astral.sh/ruff/0.6.4/install.ps1 | iex"
curl -LsSf https://astral.sh/ruff/0.6.6/install.sh | sh
powershell -c "irm https://astral.sh/ruff/0.6.6/install.ps1 | iex"
```
You can also install Ruff via [Homebrew](https://formulae.brew.sh/formula/ruff), [Conda](https://anaconda.org/conda-forge/ruff),
@@ -170,7 +170,7 @@ Ruff can also be used as a [pre-commit](https://pre-commit.com/) hook via [`ruff
```yaml
- repo: https://github.com/astral-sh/ruff-pre-commit
# Ruff version.
rev: v0.6.4
rev: v0.6.6
hooks:
# Run the linter.
- id: ruff

View File

@@ -38,7 +38,12 @@ test-case = { workspace = true }
[build-dependencies]
path-slash = { workspace = true }
walkdir = { workspace = true }
zip = { workspace = true, features = ["zstd", "deflate"] }
[target.'cfg(not(target_arch = "powerpc64"))'.build-dependencies]
zip = { workspace = true, features = ["deflate", "zstd"] }
[target.'cfg(target_arch = "powerpc64")'.build-dependencies]
zip = { workspace = true, features = ["deflate"] }
[dev-dependencies]
ruff_db = { workspace = true, features = ["os", "testing"] }

View File

@@ -30,10 +30,17 @@ fn zip_dir(directory_path: &str, writer: File) -> ZipResult<File> {
// We can't use `#[cfg(...)]` here because the target-arch in a build script is the
// architecture of the system running the build script and not the architecture of the build-target.
// That's why we use the `TARGET` environment variable here.
let method = if std::env::var("TARGET").unwrap().contains("wasm32") {
CompressionMethod::Deflated
} else {
CompressionMethod::Zstd
#[cfg(target_arch = "powerpc64")]
let method = CompressionMethod::Deflated;
#[cfg(not(target_arch = "powerpc64"))]
let method = {
let target = std::env::var("TARGET").unwrap();
if target.contains("wasm32") || target.contains("powerpc64") {
CompressionMethod::Deflated
} else {
CompressionMethod::Zstd
}
};
let options = FileOptions::default()

View File

@@ -4,7 +4,9 @@ use rustc_hash::FxHasher;
pub use db::Db;
pub use module_name::ModuleName;
pub use module_resolver::{resolve_module, system_module_search_paths, vendored_typeshed_stubs};
pub use module_resolver::{
resolve_module, system_module_search_paths, vendored_typeshed_stubs, Module,
};
pub use program::{Program, ProgramSettings, SearchPathSettings, SitePackages};
pub use python_version::PythonVersion;
pub use semantic_model::{HasTy, SemanticModel};
@@ -23,4 +25,3 @@ mod stdlib;
pub mod types;
type FxOrderSet<V> = ordermap::set::OrderSet<V, BuildHasherDefault<FxHasher>>;
type FxOrderMap<K, V> = ordermap::map::OrderMap<K, V, BuildHasherDefault<FxHasher>>;

View File

@@ -1,6 +1,6 @@
use std::iter::FusedIterator;
pub(crate) use module::Module;
pub use module::Module;
pub use resolver::resolve_module;
pub(crate) use resolver::{file_to_module, SearchPaths};
use ruff_db::system::SystemPath;

View File

@@ -54,6 +54,13 @@ impl TryFrom<(&str, &str)> for PythonVersion {
}
}
impl From<(u8, u8)> for PythonVersion {
fn from(value: (u8, u8)) -> Self {
let (major, minor) = value;
Self { major, minor }
}
}
impl fmt::Display for PythonVersion {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
let PythonVersion { major, minor } = self;

View File

@@ -115,6 +115,9 @@ pub(crate) struct SemanticIndex<'db> {
/// Note: We should not depend on this map when analysing other files or
/// changing a file invalidates all dependents.
ast_ids: IndexVec<FileScopeId, AstIds>,
/// Flags about the global scope (code usage impacting inference)
has_future_annotations: bool,
}
impl<'db> SemanticIndex<'db> {
@@ -215,6 +218,12 @@ impl<'db> SemanticIndex<'db> {
pub(crate) fn node_scope(&self, node: NodeWithScopeRef) -> FileScopeId {
self.scopes_by_node[&node.node_key()]
}
/// Checks if there is an import of `__future__.annotations` in the global scope, which affects
/// the logic for type inference.
pub(super) fn has_future_annotations(&self) -> bool {
self.has_future_annotations
}
}
pub struct AncestorsIter<'a> {
@@ -455,7 +464,7 @@ mod tests {
global_table
.symbol_by_name("foo")
.is_some_and(|symbol| { !symbol.is_bound() && symbol.is_used() }),
"a symbol used but not defined in a scope should have only the used flag"
"a symbol used but not bound in a scope should have only the used flag"
);
let use_def = use_def_map(&db, scope);
let binding = use_def
@@ -1027,7 +1036,7 @@ class C[T]:
}
let TestCase { db, file } = test_case(
r#"
r"
class Test:
def foo():
def bar():
@@ -1036,7 +1045,7 @@ class Test:
pass
def x():
pass"#,
pass",
);
let index = semantic_index(&db, file);

View File

@@ -28,7 +28,8 @@ use crate::Db;
use super::constraint::{Constraint, PatternConstraint};
use super::definition::{
ExceptHandlerDefinitionNodeRef, MatchPatternDefinitionNodeRef, WithItemDefinitionNodeRef,
DefinitionCategory, ExceptHandlerDefinitionNodeRef, MatchPatternDefinitionNodeRef,
WithItemDefinitionNodeRef,
};
pub(super) struct SemanticIndexBuilder<'db> {
@@ -44,6 +45,9 @@ pub(super) struct SemanticIndexBuilder<'db> {
/// Flow states at each `break` in the current loop.
loop_break_states: Vec<FlowSnapshot>,
/// Flags about the file's global scope
has_future_annotations: bool,
// Semantic Index fields
scopes: IndexVec<FileScopeId, Scope>,
scope_ids_by_scope: IndexVec<FileScopeId, ScopeId<'db>>,
@@ -67,6 +71,8 @@ impl<'db> SemanticIndexBuilder<'db> {
current_match_case: None,
loop_break_states: vec![],
has_future_annotations: false,
scopes: IndexVec::new(),
symbol_tables: IndexVec::new(),
ast_ids: IndexVec::new(),
@@ -191,8 +197,9 @@ impl<'db> SemanticIndexBuilder<'db> {
) -> Definition<'db> {
let definition_node: DefinitionNodeRef<'_> = definition_node.into();
#[allow(unsafe_code)]
// SAFETY: `definition_node` is guaranteed to be a child of `self.module`
let kind = unsafe { definition_node.into_owned(self.module.clone()) };
let (is_declaration, is_binding) = (kind.is_declaration(), kind.is_binding());
let category = kind.category();
let definition = Definition::new(
self.db,
self.file,
@@ -207,16 +214,17 @@ impl<'db> SemanticIndexBuilder<'db> {
.insert(definition_node.key(), definition);
debug_assert_eq!(existing_definition, None);
if is_binding {
if category.is_binding() {
self.mark_symbol_bound(symbol);
}
let use_def = self.current_use_def_map_mut();
match (is_declaration, is_binding) {
(true, true) => use_def.record_declaration_and_binding(symbol, definition),
(true, false) => use_def.record_declaration(symbol, definition),
(false, true) => use_def.record_binding(symbol, definition),
(false, false) => unreachable!("definition must be declaration or binding or both"),
match category {
DefinitionCategory::DeclarationAndBinding => {
use_def.record_declaration_and_binding(symbol, definition);
}
DefinitionCategory::Declaration => use_def.record_declaration(symbol, definition),
DefinitionCategory::Binding => use_def.record_binding(symbol, definition),
}
definition
@@ -301,9 +309,11 @@ impl<'db> SemanticIndexBuilder<'db> {
};
let symbol = self.add_symbol(name.id.clone());
// TODO create Definition for PEP 695 typevars
// note that the "bound" on the typevar is a totally different thing than whether
// or not a name is "bound" by a typevar declaration; the latter is always true.
self.mark_symbol_bound(symbol);
if let Some(bound) = bound {
self.visit_expr(bound);
if let Some(bounds) = bound {
self.visit_expr(bounds);
}
if let Some(default) = default {
self.visit_expr(default);
@@ -320,11 +330,23 @@ impl<'db> SemanticIndexBuilder<'db> {
nested_scope
}
/// Visit a list of [`Comprehension`] nodes, assumed to be the "generators" that compose a
/// comprehension (that is, the `for x in y` and `for y in z` parts of `x for x in y for y in z`.)
/// This method does several things:
/// - It pushes a new scope onto the stack for visiting
/// a list/dict/set comprehension or generator expression
/// - Inside that scope, it visits a list of [`Comprehension`] nodes,
/// assumed to be the "generators" that compose a comprehension
/// (that is, the `for x in y` and `for y in z` parts of `x for x in y for y in z`).
/// - Inside that scope, it also calls a closure for visiting the outer `elt`
/// of a list/dict/set comprehension or generator expression
/// - It then pops the new scope off the stack
///
/// [`Comprehension`]: ast::Comprehension
fn visit_generators(&mut self, scope: NodeWithScopeRef, generators: &'db [ast::Comprehension]) {
fn with_generators_scope(
&mut self,
scope: NodeWithScopeRef,
generators: &'db [ast::Comprehension],
visit_outer_elt: impl FnOnce(&mut Self),
) {
let mut generators_iter = generators.iter();
let Some(generator) = generators_iter.next() else {
@@ -363,6 +385,9 @@ impl<'db> SemanticIndexBuilder<'db> {
self.visit_expr(expr);
}
}
visit_outer_elt(self);
self.pop_scope();
}
fn declare_parameter(&mut self, parameter: AnyParameterRef) {
@@ -430,6 +455,7 @@ impl<'db> SemanticIndexBuilder<'db> {
scopes_by_expression: self.scopes_by_expression,
scopes_by_node: self.scopes_by_node,
use_def_maps,
has_future_annotations: self.has_future_annotations,
}
}
}
@@ -523,7 +549,16 @@ where
&alias.name.id
};
// Look for imports `from __future__ import annotations`, ignore `as ...`
// We intentionally don't enforce the rules about location of `__future__`
// imports here, we assume the user's intent was to apply the `__future__`
// import, so we still check using it (and will also emit a diagnostic about a
// miss-placed `__future__` import.)
self.has_future_annotations |= alias.name.id == "annotations"
&& node.module.as_deref() == Some("__future__");
let symbol = self.add_symbol(symbol_name.clone());
self.add_definition(symbol, ImportFromDefinitionNodeRef { node, alias_index });
}
}
@@ -873,6 +908,7 @@ where
}
self.visit_expr(lambda.body.as_ref());
self.pop_scope();
}
ast::Expr::If(ast::ExprIf {
body, test, orelse, ..
@@ -893,30 +929,33 @@ where
elt, generators, ..
},
) => {
self.visit_generators(
self.with_generators_scope(
NodeWithScopeRef::ListComprehension(list_comprehension),
generators,
|builder| builder.visit_expr(elt),
);
self.visit_expr(elt);
}
ast::Expr::SetComp(
set_comprehension @ ast::ExprSetComp {
elt, generators, ..
},
) => {
self.visit_generators(
self.with_generators_scope(
NodeWithScopeRef::SetComprehension(set_comprehension),
generators,
|builder| builder.visit_expr(elt),
);
self.visit_expr(elt);
}
ast::Expr::Generator(
generator @ ast::ExprGenerator {
elt, generators, ..
},
) => {
self.visit_generators(NodeWithScopeRef::GeneratorExpression(generator), generators);
self.visit_expr(elt);
self.with_generators_scope(
NodeWithScopeRef::GeneratorExpression(generator),
generators,
|builder| builder.visit_expr(elt),
);
}
ast::Expr::DictComp(
dict_comprehension @ ast::ExprDictComp {
@@ -926,28 +965,19 @@ where
..
},
) => {
self.visit_generators(
self.with_generators_scope(
NodeWithScopeRef::DictComprehension(dict_comprehension),
generators,
|builder| {
builder.visit_expr(key);
builder.visit_expr(value);
},
);
self.visit_expr(key);
self.visit_expr(value);
}
_ => {
walk_expr(self, expr);
}
}
if matches!(
expr,
ast::Expr::Lambda(_)
| ast::Expr::ListComp(_)
| ast::Expr::SetComp(_)
| ast::Expr::Generator(_)
| ast::Expr::DictComp(_)
) {
self.pop_scope();
}
}
fn visit_parameters(&mut self, parameters: &'ast ast::Parameters) {

View File

@@ -33,6 +33,18 @@ impl<'db> Definition<'db> {
pub(crate) fn scope(self, db: &'db dyn Db) -> ScopeId<'db> {
self.file_scope(db).to_scope_id(db, self.file(db))
}
pub(crate) fn category(self, db: &'db dyn Db) -> DefinitionCategory {
self.kind(db).category()
}
pub(crate) fn is_declaration(self, db: &'db dyn Db) -> bool {
self.kind(db).category().is_declaration()
}
pub(crate) fn is_binding(self, db: &'db dyn Db) -> bool {
self.kind(db).category().is_binding()
}
}
#[derive(Copy, Clone, Debug)]
@@ -302,6 +314,41 @@ impl DefinitionNodeRef<'_> {
}
}
#[derive(Clone, Copy, Debug)]
pub(crate) enum DefinitionCategory {
/// A Definition which binds a value to a name (e.g. `x = 1`).
Binding,
/// A Definition which declares the upper-bound of acceptable types for this name (`x: int`).
Declaration,
/// A Definition which both declares a type and binds a value (e.g. `x: int = 1`).
DeclarationAndBinding,
}
impl DefinitionCategory {
/// True if this definition establishes a "declared type" for the symbol.
///
/// If so, any assignments reached by this definition are in error if they assign a value of a
/// type not assignable to the declared type.
///
/// Annotations establish a declared type. So do function and class definitions, and imports.
pub(crate) fn is_declaration(self) -> bool {
matches!(
self,
DefinitionCategory::Declaration | DefinitionCategory::DeclarationAndBinding
)
}
/// True if this definition assigns a value to the symbol.
///
/// False only for annotated assignments without a RHS.
pub(crate) fn is_binding(self) -> bool {
matches!(
self,
DefinitionCategory::Binding | DefinitionCategory::DeclarationAndBinding
)
}
}
#[derive(Clone, Debug)]
pub enum DefinitionKind {
Import(AstNodeRef<ast::Alias>),
@@ -322,33 +369,46 @@ pub enum DefinitionKind {
}
impl DefinitionKind {
/// True if this definition establishes a "declared type" for the symbol.
///
/// If so, any assignments reached by this definition are in error if they assign a value of a
/// type not assignable to the declared type.
///
/// Annotations establish a declared type. So do function and class definition.
pub(crate) fn is_declaration(&self) -> bool {
pub(crate) fn category(&self) -> DefinitionCategory {
match self {
DefinitionKind::Function(_) => true,
DefinitionKind::Class(_) => true,
DefinitionKind::Parameter(parameter) => parameter.annotation.is_some(),
DefinitionKind::ParameterWithDefault(parameter_with_default) => {
parameter_with_default.parameter.annotation.is_some()
// functions, classes, and imports always bind, and we consider them declarations
DefinitionKind::Function(_)
| DefinitionKind::Class(_)
| DefinitionKind::Import(_)
| DefinitionKind::ImportFrom(_) => DefinitionCategory::DeclarationAndBinding,
// a parameter always binds a value, but is only a declaration if annotated
DefinitionKind::Parameter(parameter) => {
if parameter.annotation.is_some() {
DefinitionCategory::DeclarationAndBinding
} else {
DefinitionCategory::Binding
}
}
DefinitionKind::AnnotatedAssignment(_) => true,
_ => false,
}
}
/// True if this definition assigns a value to the symbol.
///
/// False only for annotated assignments without a RHS.
pub(crate) fn is_binding(&self) -> bool {
if let DefinitionKind::AnnotatedAssignment(ann_assign) = self {
ann_assign.value.is_some()
} else {
true
// presence of a default is irrelevant, same logic as for a no-default parameter
DefinitionKind::ParameterWithDefault(parameter_with_default) => {
if parameter_with_default.parameter.annotation.is_some() {
DefinitionCategory::DeclarationAndBinding
} else {
DefinitionCategory::Binding
}
}
// annotated assignment is always a declaration, only a binding if there is a RHS
DefinitionKind::AnnotatedAssignment(ann_assign) => {
if ann_assign.value.is_some() {
DefinitionCategory::DeclarationAndBinding
} else {
DefinitionCategory::Declaration
}
}
// all of these bind values without declaring a type
DefinitionKind::NamedExpression(_)
| DefinitionKind::Assignment(_)
| DefinitionKind::AugmentedAssignment(_)
| DefinitionKind::For(_)
| DefinitionKind::Comprehension(_)
| DefinitionKind::WithItem(_)
| DefinitionKind::MatchPattern(_)
| DefinitionKind::ExceptHandler(_) => DefinitionCategory::Binding,
}
}
}

View File

@@ -51,7 +51,7 @@ impl Symbol {
bitflags! {
#[derive(Copy, Clone, Debug, Eq, PartialEq)]
pub(super) struct SymbolFlags: u8 {
struct SymbolFlags: u8 {
const IS_USED = 1 << 0;
const IS_BOUND = 1 << 1;
/// TODO: This flag is not yet set by anything

View File

@@ -1,6 +1,6 @@
//! First, some terminology:
//!
//! * a "binding" gives a new value to a variable. This includes many different Python statements
//! * A "binding" gives a new value to a variable. This includes many different Python statements
//! (assignment statements of course, but also imports, `def` and `class` statements, `as`
//! clauses in `with` and `except` statements, match patterns, and others) and even one
//! expression kind (named expressions). It notably does not include annotated assignment
@@ -8,7 +8,7 @@
//! variable. We consider function parameters to be bindings as well, since (from the perspective
//! of the function's internal scope), a function parameter begins the scope bound to a value.
//!
//! * a "declaration" establishes an upper bound type for the values that a variable may be
//! * A "declaration" establishes an upper bound type for the values that a variable may be
//! permitted to take on. Annotated assignment statements (with or without an RHS value) are
//! declarations; annotated function parameters are also declarations. We consider `def` and
//! `class` statements to also be declarations, so as to prohibit accidentally shadowing them.
@@ -23,12 +23,15 @@
//!
//! At any given use of a variable, we can ask about both its "declared type" and its "inferred
//! type". These may be different, but the inferred type must always be assignable to the declared
//! type; that is, the declared type is always wider, and the inferred type may be more precise.
//! type; that is, the declared type is always wider, and the inferred type may be more precise. If
//! we see an invalid assignment, we emit a diagnostic and abandon our inferred type, deferring to
//! the declared type (this allows an explicit annotation to override bad inference, without a
//! cast), maintaining the invariant.
//!
//! The **inferred type** represents the most precise type we believe encompasses all possible
//! values for the variable at a given use. It is based on the bindings which can reach that use
//! through some control flow path, and the narrowing constraints that control flow must have
//! passed through between the binding and the use. For example, in this code:
//! values for the variable at a given use. It is based on a union of the bindings which can reach
//! that use through some control flow path, and the narrowing constraints that control flow must
//! have passed through between the binding and the use. For example, in this code:
//!
//! ```python
//! x = 1 if flag else None
@@ -64,12 +67,13 @@
//! Path(path)`, with the explicit `: Path` annotation, is permitted.
//!
//! The general rule is that whatever declaration(s) can reach a given binding determine the
//! validity of that binding. If multiple declarations can reach a binding, they must be
//! equivalent declarations, or we issue a type error, since we can't reconcile to a single
//! declared type.
//! validity of that binding. If there is a path in which the symbol is not declared, that is a
//! declaration of `Unknown`. If multiple declarations can reach a binding, we union them, but by
//! default we also issue a type error, since this implicit union of declared types may hide an
//! error.
//!
//! To support type inference, we build a map from each use of a symbol to the bindings live at
//! that use, and the type-narrowing constraints that apply to each binding.
//! that use, and the type narrowing constraints that apply to each binding.
//!
//! Let's take this code sample:
//!
@@ -104,7 +108,7 @@
//! all uses (that means a `Name` node with `Load` context) so we have a `ScopedUseId` to
//! efficiently represent each use.
//!
//! We also need to know, for a given definition of a symbol, what type-narrowing constraints apply
//! We also need to know, for a given definition of a symbol, what type narrowing constraints apply
//! to it. For instance, in this code sample:
//!
//! ```python
@@ -142,7 +146,7 @@
//! via a global or nonlocal reference.) But modeling this fully accurately requires whole-program
//! analysis that isn't tractable for an efficient analysis, since it means a given symbol could
//! have a different type every place it's referenced throughout the program, depending on the
//! shape of arbitrarily-sized call/import graphs. So we follow other Python type-checkers in
//! shape of arbitrarily-sized call/import graphs. So we follow other Python type checkers in
//! making the simplifying assumption that usually the scope will finish execution before its
//! symbols are made visible to other scopes; for instance, most imports will import from a
//! complete module, not a partially-executed module. (We may want to get a little smarter than
@@ -169,7 +173,7 @@
//!
//! The simplest way to model "unbound" would be as a "binding" itself: the initial "binding" for
//! each symbol in a scope. But actually modeling it this way would unnecessarily increase the
//! number of [`Definition`] that Salsa must track. Since "unbound" is special in that all symbols
//! number of [`Definition`]s that Salsa must track. Since "unbound" is special in that all symbols
//! share it, and it doesn't have any additional per-symbol state, and constraints are irrelevant
//! to it, we can represent it more efficiently: we use the `may_be_unbound` boolean on the
//! [`SymbolBindings`] struct. If this flag is `true` for a use of a symbol, it means the symbol
@@ -314,7 +318,8 @@ impl<'db> UseDefMap<'db> {
&self,
symbol: ScopedSymbolId,
) -> DeclarationsIterator<'_, 'db> {
self.declarations_iterator(self.public_symbols[symbol].declarations())
let declarations = self.public_symbols[symbol].declarations();
self.declarations_iterator(declarations)
}
pub(crate) fn has_public_declarations(&self, symbol: ScopedSymbolId) -> bool {
@@ -339,6 +344,7 @@ impl<'db> UseDefMap<'db> {
DeclarationsIterator {
all_definitions: &self.all_definitions,
inner: declarations.iter(),
may_be_undeclared: declarations.may_be_undeclared(),
}
}
}
@@ -400,6 +406,13 @@ impl std::iter::FusedIterator for ConstraintsIterator<'_, '_> {}
pub(crate) struct DeclarationsIterator<'map, 'db> {
all_definitions: &'map IndexVec<ScopedDefinitionId, Definition<'db>>,
inner: DeclarationIdIterator<'map>,
may_be_undeclared: bool,
}
impl DeclarationsIterator<'_, '_> {
pub(crate) fn may_be_undeclared(&self) -> bool {
self.may_be_undeclared
}
}
impl<'map, 'db> Iterator for DeclarationsIterator<'map, 'db> {
@@ -537,8 +550,9 @@ impl<'db> UseDefMapBuilder<'db> {
if let Some(snapshot) = snapshot_definitions_iter.next() {
current.merge(snapshot);
} else {
// Symbol not present in snapshot, so it's unbound from that path.
// Symbol not present in snapshot, so it's unbound/undeclared from that path.
current.set_may_be_unbound();
current.set_may_be_undeclared();
}
}
}

View File

@@ -105,9 +105,8 @@ impl<const B: usize> BitSet<B> {
max_len = other_len;
self.resize_blocks(max_len);
}
let other_blocks = other.blocks();
for (i, my_block) in self.blocks_mut().iter_mut().enumerate() {
*my_block |= other_blocks.get(i).unwrap_or(&0);
for (my_block, other_block) in self.blocks_mut().iter_mut().zip(other.blocks()) {
*my_block |= other_block;
}
}

View File

@@ -1,13 +1,13 @@
//! Track visible definitions of a symbol, and applicable constraints per definition.
//! Track live bindings per symbol, applicable constraints per binding, and live declarations.
//!
//! These data structures operate entirely on scope-local newtype-indices for definitions and
//! constraints, referring to their location in the `all_definitions` and `all_constraints`
//! indexvecs in [`super::UseDefMapBuilder`].
//!
//! We need to track arbitrary associations between definitions and constraints, not just a single
//! set of currently dominating constraints (where "dominating" means "control flow must have
//! passed through it to reach this point"), because we can have dominating constraints that apply
//! to some definitions but not others, as in this code:
//! We need to track arbitrary associations between bindings and constraints, not just a single set
//! of currently dominating constraints (where "dominating" means "control flow must have passed
//! through it to reach this point"), because we can have dominating constraints that apply to some
//! bindings but not others, as in this code:
//!
//! ```python
//! x = 1 if flag else None
@@ -18,11 +18,11 @@
//! ```
//!
//! The `x is not None` constraint dominates the final use of `x`, but it applies only to the first
//! definition of `x`, not the second, so `None` is a possible value for `x`.
//! binding of `x`, not the second, so `None` is a possible value for `x`.
//!
//! And we can't just track, for each definition, an index into a list of dominating constraints,
//! either, because we can have definitions which are still visible, but subject to constraints
//! that are no longer dominating, as in this code:
//! And we can't just track, for each binding, an index into a list of dominating constraints,
//! either, because we can have bindings which are still visible, but subject to constraints that
//! are no longer dominating, as in this code:
//!
//! ```python
//! x = 0
@@ -33,13 +33,16 @@
//! ```
//!
//! From the point of view of the final use of `x`, the `x is not None` constraint no longer
//! dominates, but it does dominate the `x = 1 if flag2 else None` definition, so we have to keep
//! dominates, but it does dominate the `x = 1 if flag2 else None` binding, so we have to keep
//! track of that.
//!
//! The data structures used here ([`BitSet`] and [`smallvec::SmallVec`]) optimize for keeping all
//! data inline (avoiding lots of scattered allocations) in small-to-medium cases, and falling back
//! to heap allocation to be able to scale to arbitrary numbers of definitions and constraints when
//! needed.
//! to heap allocation to be able to scale to arbitrary numbers of live bindings and constraints
//! when needed.
//!
//! Tracking live declarations is simpler, since constraints are not involved, but otherwise very
//! similar to tracking live bindings.
use super::bitset::{BitSet, BitSetIterator};
use ruff_index::newtype_index;
use smallvec::SmallVec;
@@ -83,18 +86,28 @@ type ConstraintsIntoIterator = smallvec::IntoIter<InlineConstraintArray>;
pub(super) struct SymbolDeclarations {
/// [`BitSet`]: which declarations (as [`ScopedDefinitionId`]) can reach the current location?
live_declarations: Declarations,
/// Could the symbol be un-declared at this point?
may_be_undeclared: bool,
}
impl SymbolDeclarations {
fn undeclared() -> Self {
Self {
live_declarations: Declarations::default(),
may_be_undeclared: true,
}
}
/// Record a newly-encountered declaration for this symbol.
fn record_declaration(&mut self, declaration_id: ScopedDefinitionId) {
self.live_declarations = Declarations::with(declaration_id.into());
self.may_be_undeclared = false;
}
/// Add undeclared as a possibility for this symbol.
fn set_may_be_undeclared(&mut self) {
self.may_be_undeclared = true;
}
/// Return an iterator over live declarations for this symbol.
@@ -107,6 +120,10 @@ impl SymbolDeclarations {
pub(super) fn is_empty(&self) -> bool {
self.live_declarations.is_empty()
}
pub(super) fn may_be_undeclared(&self) -> bool {
self.may_be_undeclared
}
}
/// Live bindings and narrowing constraints for a single symbol at some point in control flow.
@@ -144,9 +161,8 @@ impl SymbolBindings {
// The new binding replaces all previous live bindings in this path, and has no
// constraints.
self.live_bindings = Bindings::with(binding_id.into());
let mut constraints = Constraints::with_capacity(1);
constraints.push(BitSet::default());
self.constraints = constraints;
self.constraints = Constraints::with_capacity(1);
self.constraints.push(BitSet::default());
self.may_be_unbound = false;
}
@@ -200,6 +216,11 @@ impl SymbolState {
self.bindings.record_constraint(constraint_id);
}
/// Add undeclared as a possibility for this symbol.
pub(super) fn set_may_be_undeclared(&mut self) {
self.declarations.set_may_be_undeclared();
}
/// Record a newly-encountered declaration of this symbol.
pub(super) fn record_declaration(&mut self, declaration_id: ScopedDefinitionId) {
self.declarations.record_declaration(declaration_id);
@@ -213,7 +234,11 @@ impl SymbolState {
constraints: Constraints::default(),
may_be_unbound: self.bindings.may_be_unbound || b.bindings.may_be_unbound,
},
declarations: self.declarations.clone(),
declarations: SymbolDeclarations {
live_declarations: self.declarations.live_declarations.clone(),
may_be_undeclared: self.declarations.may_be_undeclared
|| b.declarations.may_be_undeclared,
},
};
std::mem::swap(&mut a, self);
@@ -384,47 +409,52 @@ impl<'a> Iterator for DeclarationIdIterator<'a> {
}
}
impl std::iter::FusedIterator for DeclarationIdIterator<'_> {}
#[cfg(test)]
mod tests {
use super::{ScopedConstraintId, ScopedDefinitionId, SymbolState};
impl SymbolState {
pub(crate) fn assert_bindings(&self, may_be_unbound: bool, expected: &[&str]) {
assert_eq!(self.may_be_unbound(), may_be_unbound);
let actual = self
.bindings()
.iter()
.map(|def_id_with_constraints| {
format!(
"{}<{}>",
def_id_with_constraints.definition.as_u32(),
def_id_with_constraints
.constraint_ids
.map(ScopedConstraintId::as_u32)
.map(|idx| idx.to_string())
.collect::<Vec<_>>()
.join(", ")
)
})
.collect::<Vec<_>>();
assert_eq!(actual, expected);
}
fn assert_bindings(symbol: &SymbolState, may_be_unbound: bool, expected: &[&str]) {
assert_eq!(symbol.may_be_unbound(), may_be_unbound);
let actual = symbol
.bindings()
.iter()
.map(|def_id_with_constraints| {
format!(
"{}<{}>",
def_id_with_constraints.definition.as_u32(),
def_id_with_constraints
.constraint_ids
.map(ScopedConstraintId::as_u32)
.map(|idx| idx.to_string())
.collect::<Vec<_>>()
.join(", ")
)
})
.collect::<Vec<_>>();
assert_eq!(actual, expected);
}
pub(crate) fn assert_declarations(&self, expected: &[u32]) {
let actual = self
.declarations()
.iter()
.map(ScopedDefinitionId::as_u32)
.collect::<Vec<_>>();
assert_eq!(actual, expected);
}
pub(crate) fn assert_declarations(
symbol: &SymbolState,
may_be_undeclared: bool,
expected: &[u32],
) {
assert_eq!(symbol.declarations.may_be_undeclared(), may_be_undeclared);
let actual = symbol
.declarations()
.iter()
.map(ScopedDefinitionId::as_u32)
.collect::<Vec<_>>();
assert_eq!(actual, expected);
}
#[test]
fn unbound() {
let sym = SymbolState::undefined();
sym.assert_bindings(true, &[]);
assert_bindings(&sym, true, &[]);
}
#[test]
@@ -432,7 +462,7 @@ mod tests {
let mut sym = SymbolState::undefined();
sym.record_binding(ScopedDefinitionId::from_u32(0));
sym.assert_bindings(false, &["0<>"]);
assert_bindings(&sym, false, &["0<>"]);
}
#[test]
@@ -441,7 +471,7 @@ mod tests {
sym.record_binding(ScopedDefinitionId::from_u32(0));
sym.set_may_be_unbound();
sym.assert_bindings(true, &["0<>"]);
assert_bindings(&sym, true, &["0<>"]);
}
#[test]
@@ -450,7 +480,7 @@ mod tests {
sym.record_binding(ScopedDefinitionId::from_u32(0));
sym.record_constraint(ScopedConstraintId::from_u32(0));
sym.assert_bindings(false, &["0<0>"]);
assert_bindings(&sym, false, &["0<0>"]);
}
#[test]
@@ -466,7 +496,7 @@ mod tests {
sym0a.merge(sym0b);
let mut sym0 = sym0a;
sym0.assert_bindings(false, &["0<0>"]);
assert_bindings(&sym0, false, &["0<0>"]);
// merging the same definition with differing constraints drops all constraints
let mut sym1a = SymbolState::undefined();
@@ -479,7 +509,7 @@ mod tests {
sym1a.merge(sym1b);
let sym1 = sym1a;
sym1.assert_bindings(false, &["1<>"]);
assert_bindings(&sym1, false, &["1<>"]);
// merging a constrained definition with unbound keeps both
let mut sym2a = SymbolState::undefined();
@@ -490,12 +520,19 @@ mod tests {
sym2a.merge(sym2b);
let sym2 = sym2a;
sym2.assert_bindings(true, &["2<3>"]);
assert_bindings(&sym2, true, &["2<3>"]);
// merging different definitions keeps them each with their existing constraints
sym0.merge(sym2);
let sym = sym0;
sym.assert_bindings(true, &["0<0>", "2<3>"]);
assert_bindings(&sym, true, &["0<0>", "2<3>"]);
}
#[test]
fn no_declaration() {
let sym = SymbolState::undefined();
assert_declarations(&sym, true, &[]);
}
#[test]
@@ -503,7 +540,7 @@ mod tests {
let mut sym = SymbolState::undefined();
sym.record_declaration(ScopedDefinitionId::from_u32(1));
sym.assert_declarations(&[1]);
assert_declarations(&sym, false, &[1]);
}
#[test]
@@ -512,7 +549,7 @@ mod tests {
sym.record_declaration(ScopedDefinitionId::from_u32(1));
sym.record_declaration(ScopedDefinitionId::from_u32(2));
sym.assert_declarations(&[2]);
assert_declarations(&sym, false, &[2]);
}
#[test]
@@ -525,6 +562,27 @@ mod tests {
sym.merge(sym2);
sym.assert_declarations(&[1, 2]);
assert_declarations(&sym, false, &[1, 2]);
}
#[test]
fn record_declaration_merge_partial_undeclared() {
let mut sym = SymbolState::undefined();
sym.record_declaration(ScopedDefinitionId::from_u32(1));
let sym2 = SymbolState::undefined();
sym.merge(sym2);
assert_declarations(&sym, true, &[1]);
}
#[test]
fn set_may_be_undeclared() {
let mut sym = SymbolState::undefined();
sym.record_declaration(ScopedDefinitionId::from_u32(0));
sym.set_may_be_undeclared();
assert_declarations(&sym, true, &[0]);
}
}

View File

@@ -11,6 +11,7 @@ enum CoreStdlibModule {
Builtins,
Types,
Typeshed,
TypingExtensions,
}
impl CoreStdlibModule {
@@ -19,6 +20,7 @@ impl CoreStdlibModule {
Self::Builtins => "builtins",
Self::Types => "types",
Self::Typeshed => "_typeshed",
Self::TypingExtensions => "typing_extensions",
};
ModuleName::new_static(module_name)
.unwrap_or_else(|| panic!("{module_name} should be a valid module name!"))
@@ -62,6 +64,14 @@ pub(crate) fn typeshed_symbol_ty<'db>(db: &'db dyn Db, symbol: &str) -> Type<'db
core_module_symbol_ty(db, CoreStdlibModule::Typeshed, symbol)
}
/// Lookup the type of `symbol` in the `typing_extensions` module namespace.
///
/// Returns `Unbound` if the `typing_extensions` module isn't available for some reason.
#[inline]
pub(crate) fn typing_extensions_symbol_ty<'db>(db: &'db dyn Db, symbol: &str) -> Type<'db> {
core_module_symbol_ty(db, CoreStdlibModule::TypingExtensions, symbol)
}
/// Get the scope of a core stdlib module.
///
/// Can return `None` if a custom typeshed is used that is missing the core module in question.

View File

@@ -10,12 +10,15 @@ use crate::semantic_index::{
global_scope, semantic_index, symbol_table, use_def_map, BindingWithConstraints,
BindingWithConstraintsIterator, DeclarationsIterator,
};
use crate::stdlib::{builtins_symbol_ty, types_symbol_ty, typeshed_symbol_ty};
use crate::stdlib::{
builtins_symbol_ty, types_symbol_ty, typeshed_symbol_ty, typing_extensions_symbol_ty,
};
use crate::types::narrow::narrowing_constraint;
use crate::{Db, FxOrderSet};
pub(crate) use self::builder::{IntersectionBuilder, UnionBuilder};
pub(crate) use self::diagnostic::TypeCheckDiagnostics;
pub(crate) use self::display::TypeArrayDisplay;
pub(crate) use self::infer::{
infer_deferred_types, infer_definition_types, infer_expression_types, infer_scope_types,
};
@@ -41,17 +44,30 @@ pub fn check_types(db: &dyn Db, file: File) -> TypeCheckDiagnostics {
}
/// Infer the public type of a symbol (its type as seen from outside its scope).
pub(crate) fn symbol_ty_by_id<'db>(
db: &'db dyn Db,
scope: ScopeId<'db>,
symbol: ScopedSymbolId,
) -> Type<'db> {
fn symbol_ty_by_id<'db>(db: &'db dyn Db, scope: ScopeId<'db>, symbol: ScopedSymbolId) -> Type<'db> {
let _span = tracing::trace_span!("symbol_ty_by_id", ?symbol).entered();
let use_def = use_def_map(db, scope);
// If the symbol is declared, the public type is based on declarations; otherwise, it's based
// on inference from bindings.
if use_def.has_public_declarations(symbol) {
declarations_ty(db, use_def.public_declarations(symbol))
let declarations = use_def.public_declarations(symbol);
// If the symbol is undeclared in some paths, include the inferred type in the public type.
let undeclared_ty = if declarations.may_be_undeclared() {
Some(bindings_ty(
db,
use_def.public_bindings(symbol),
use_def
.public_may_be_unbound(symbol)
.then_some(Type::Unknown),
))
} else {
None
};
// Intentionally ignore conflicting declared types; that's not our problem, it's the
// problem of the module we are importing from.
declarations_ty(db, declarations, undeclared_ty).unwrap_or_else(|(ty, _)| ty)
} else {
bindings_ty(
db,
@@ -64,7 +80,7 @@ pub(crate) fn symbol_ty_by_id<'db>(
}
/// Shorthand for `symbol_ty` that takes a symbol name instead of an ID.
pub(crate) fn symbol_ty<'db>(db: &'db dyn Db, scope: ScopeId<'db>, name: &str) -> Type<'db> {
fn symbol_ty<'db>(db: &'db dyn Db, scope: ScopeId<'db>, name: &str) -> Type<'db> {
let table = symbol_table(db, scope);
table
.symbol_id_by_name(name)
@@ -84,7 +100,7 @@ pub(crate) fn binding_ty<'db>(db: &'db dyn Db, definition: Definition<'db>) -> T
}
/// Infer the type of a declaration.
pub(crate) fn declaration_ty<'db>(db: &'db dyn Db, definition: Definition<'db>) -> Type<'db> {
fn declaration_ty<'db>(db: &'db dyn Db, definition: Definition<'db>) -> Type<'db> {
let inference = infer_definition_types(db, definition);
inference.declaration_ty(definition)
}
@@ -93,7 +109,7 @@ pub(crate) fn declaration_ty<'db>(db: &'db dyn Db, definition: Definition<'db>)
///
/// ## Panics
/// If the given expression is not a sub-expression of the given [`Definition`].
pub(crate) fn definition_expression_ty<'db>(
fn definition_expression_ty<'db>(
db: &'db dyn Db,
definition: Definition<'db>,
expression: &ast::Expr,
@@ -107,22 +123,22 @@ pub(crate) fn definition_expression_ty<'db>(
}
}
/// Infer the combined type of an array of [`Definition`]s, plus one optional "unbound type".
/// Infer the combined type of an iterator of bindings, plus one optional "unbound type".
///
/// Will return a union if there is more than one definition, or at least one plus an unbound
/// Will return a union if there is more than one binding, or at least one plus an unbound
/// type.
///
/// The "unbound type" represents the type in case control flow may not have passed through any
/// definitions in this scope. If this isn't possible, then it will be `None`. If it is possible,
/// and the result in that case should be Unbound (e.g. an unbound function local), then it will be
/// bindings in this scope. If this isn't possible, then it will be `None`. If it is possible, and
/// the result in that case should be Unbound (e.g. an unbound function local), then it will be
/// `Some(Type::Unbound)`. If it is possible and the result should be something else (e.g. an
/// implicit global lookup), then `unbound_type` will be `Some(the_global_symbol_type)`.
///
/// # Panics
/// Will panic if called with zero definitions and no `unbound_ty`. This is a logic error,
/// as any symbol with zero visible definitions clearly may be unbound, and the caller should
/// provide an `unbound_ty`.
pub(crate) fn bindings_ty<'db>(
/// Will panic if called with zero bindings and no `unbound_ty`. This is a logic error, as any
/// symbol with zero visible bindings clearly may be unbound, and the caller should provide an
/// `unbound_ty`.
fn bindings_ty<'db>(
db: &'db dyn Db,
bindings_with_constraints: BindingWithConstraintsIterator<'_, 'db>,
unbound_ty: Option<Type<'db>>,
@@ -162,12 +178,56 @@ pub(crate) fn bindings_ty<'db>(
}
}
/// Union an iterable of declared types.
pub(crate) fn declarations_ty<'db>(
/// The result of looking up a declared type from declarations; see [`declarations_ty`].
type DeclaredTypeResult<'db> = Result<Type<'db>, (Type<'db>, Box<[Type<'db>]>)>;
/// Build a declared type from a [`DeclarationsIterator`].
///
/// If there is only one declaration, or all declarations declare the same type, returns
/// `Ok(declared_type)`. If there are conflicting declarations, returns
/// `Err((union_of_declared_types, conflicting_declared_types))`.
///
/// If undeclared is a possibility, `undeclared_ty` type will be part of the return type (and may
/// conflict with other declarations.)
///
/// # Panics
/// Will panic if there are no declarations and no `undeclared_ty` is provided. This is a logic
/// error, as any symbol with zero live declarations clearly must be undeclared, and the caller
/// should provide an `undeclared_ty`.
fn declarations_ty<'db>(
db: &'db dyn Db,
declarations: DeclarationsIterator<'_, 'db>,
) -> Type<'db> {
UnionType::from_elements(db, declarations.map(|decl| declaration_ty(db, decl)))
undeclared_ty: Option<Type<'db>>,
) -> DeclaredTypeResult<'db> {
let decl_types = declarations.map(|declaration| declaration_ty(db, declaration));
let mut all_types = undeclared_ty.into_iter().chain(decl_types);
let first = all_types.next().expect(
"declarations_ty must not be called with zero declarations and no may-be-undeclared.",
);
let mut conflicting: Vec<Type<'db>> = vec![];
let declared_ty = if let Some(second) = all_types.next() {
let mut builder = UnionBuilder::new(db).add(first);
for other in [second].into_iter().chain(all_types) {
if !first.is_equivalent_to(db, other) {
conflicting.push(other);
}
builder = builder.add(other);
}
builder.build()
} else {
first
};
if conflicting.is_empty() {
DeclaredTypeResult::Ok(declared_ty)
} else {
DeclaredTypeResult::Err((
declared_ty,
[first].into_iter().chain(conflicting).collect(),
))
}
}
/// Unique ID for a type.
@@ -187,6 +247,8 @@ pub enum Type<'db> {
None,
/// a specific function object
Function(FunctionType<'db>),
/// The `typing.reveal_type` function, which has special `__call__` behavior.
RevealTypeFunction(FunctionType<'db>),
/// a specific module object
Module(File),
/// a specific class object
@@ -273,14 +335,16 @@ impl<'db> Type<'db> {
pub const fn into_function_type(self) -> Option<FunctionType<'db>> {
match self {
Type::Function(function_type) => Some(function_type),
Type::Function(function_type) | Type::RevealTypeFunction(function_type) => {
Some(function_type)
}
_ => None,
}
}
pub fn expect_function(self) -> FunctionType<'db> {
self.into_function_type()
.expect("Expected a Type::Function variant")
.expect("Expected a variant wrapping a FunctionType")
}
pub const fn into_int_literal_type(self) -> Option<i64> {
@@ -298,7 +362,7 @@ impl<'db> Type<'db> {
pub fn may_be_unbound(&self, db: &'db dyn Db) -> bool {
match self {
Type::Unbound => true,
Type::Union(union) => union.contains(db, Type::Unbound),
Type::Union(union) => union.elements(db).contains(&Type::Unbound),
// Unbound can't appear in an intersection, because an intersection with Unbound
// simplifies to just Unbound.
_ => false,
@@ -316,33 +380,71 @@ impl<'db> Type<'db> {
}
}
/// Return true if this type is assignable to type `other`.
pub(crate) fn is_assignable_to(self, db: &'db dyn Db, other: Type<'db>) -> bool {
if self.is_equivalent_to(db, other) {
pub fn is_stdlib_symbol(&self, db: &'db dyn Db, module_name: &str, name: &str) -> bool {
match self {
Type::Class(class) => class.is_stdlib_symbol(db, module_name, name),
Type::Function(function) | Type::RevealTypeFunction(function) => {
function.is_stdlib_symbol(db, module_name, name)
}
_ => false,
}
}
/// Return true if this type is a [subtype of] type `target`.
///
/// [subtype of]: https://typing.readthedocs.io/en/latest/spec/concepts.html#subtype-supertype-and-type-equivalence
pub(crate) fn is_subtype_of(self, db: &'db dyn Db, target: Type<'db>) -> bool {
if self.is_equivalent_to(db, target) {
return true;
}
match (self, other) {
(Type::Unknown | Type::Any | Type::Never, _) => true,
(_, Type::Unknown | Type::Any) => true,
(Type::IntLiteral(_), Type::Instance(class)) if class.is_builtin_named(db, "int") => {
match (self, target) {
(Type::Unknown | Type::Any, _) => false,
(_, Type::Unknown | Type::Any) => false,
(Type::Never, _) => true,
(_, Type::Never) => false,
(Type::IntLiteral(_), Type::Instance(class))
if class.is_stdlib_symbol(db, "builtins", "int") =>
{
true
}
(Type::StringLiteral(_), Type::LiteralString) => true,
(Type::StringLiteral(_) | Type::LiteralString, Type::Instance(class))
if class.is_builtin_named(db, "str") =>
if class.is_stdlib_symbol(db, "builtins", "str") =>
{
true
}
(Type::BytesLiteral(_), Type::Instance(class))
if class.is_builtin_named(db, "bytes") =>
if class.is_stdlib_symbol(db, "builtins", "bytes") =>
{
true
}
(ty, Type::Union(union)) => union
.elements(db)
.iter()
.any(|&elem_ty| ty.is_subtype_of(db, elem_ty)),
(_, Type::Instance(class)) if class.is_stdlib_symbol(db, "builtins", "object") => true,
(Type::Instance(class), _) if class.is_stdlib_symbol(db, "builtins", "object") => false,
// TODO
_ => false,
}
}
/// Return true if this type is [assignable to] type `target`.
///
/// [assignable to]: https://typing.readthedocs.io/en/latest/spec/concepts.html#the-assignable-to-or-consistent-subtyping-relation
pub(crate) fn is_assignable_to(self, db: &'db dyn Db, target: Type<'db>) -> bool {
match (self, target) {
(Type::Unknown | Type::Any, _) => true,
(_, Type::Unknown | Type::Any) => true,
(ty, Type::Union(union)) => union
.elements(db)
.iter()
.any(|&elem_ty| ty.is_assignable_to(db, elem_ty)),
// TODO other types containing gradual forms (e.g. generics containing Any/Unknown)
_ => self.is_subtype_of(db, target),
}
}
/// Return true if this type is equivalent to type `other`.
pub(crate) fn is_equivalent_to(self, _db: &'db dyn Db, other: Type<'db>) -> bool {
// TODO equivalent but not identical structural types, differently-ordered unions and
@@ -377,7 +479,7 @@ impl<'db> Type<'db> {
// TODO: attribute lookup on None type
Type::Unknown
}
Type::Function(_) => {
Type::Function(_) | Type::RevealTypeFunction(_) => {
// TODO: attribute lookup on function type
Type::Unknown
}
@@ -423,26 +525,39 @@ impl<'db> Type<'db> {
///
/// Returns `None` if `self` is not a callable type.
#[must_use]
pub fn call(&self, db: &'db dyn Db) -> Option<Type<'db>> {
fn call(self, db: &'db dyn Db, arg_types: &[Type<'db>]) -> CallOutcome<'db> {
match self {
Type::Function(function_type) => Some(function_type.return_type(db)),
// TODO validate typed call arguments vs callable signature
Type::Function(function_type) => CallOutcome::callable(function_type.return_type(db)),
Type::RevealTypeFunction(function_type) => CallOutcome::revealed(
function_type.return_type(db),
*arg_types.first().unwrap_or(&Type::Unknown),
),
// TODO annotated return type on `__new__` or metaclass `__call__`
Type::Class(class) => Some(Type::Instance(*class)),
Type::Class(class) => CallOutcome::callable(Type::Instance(class)),
// TODO: handle classes which implement the Callable protocol
Type::Instance(_instance_ty) => Some(Type::Unknown),
// TODO: handle classes which implement the `__call__` protocol
Type::Instance(_instance_ty) => CallOutcome::callable(Type::Unknown),
// `Any` is callable, and its return type is also `Any`.
Type::Any => Some(Type::Any),
Type::Any => CallOutcome::callable(Type::Any),
Type::Unknown => Some(Type::Unknown),
Type::Unknown => CallOutcome::callable(Type::Unknown),
// TODO: union and intersection types, if they reduce to `Callable`
Type::Union(_) => Some(Type::Unknown),
Type::Intersection(_) => Some(Type::Unknown),
Type::Union(union) => CallOutcome::union(
self,
union
.elements(db)
.iter()
.map(|elem| elem.call(db, arg_types))
.collect::<Box<[CallOutcome<'db>]>>(),
),
_ => None,
// TODO: intersection types
Type::Intersection(_) => CallOutcome::callable(Type::Unknown),
_ => CallOutcome::not_callable(self),
}
}
@@ -454,7 +569,7 @@ impl<'db> Type<'db> {
/// for y in x:
/// pass
/// ```
fn iterate(&self, db: &'db dyn Db) -> IterationOutcome<'db> {
fn iterate(self, db: &'db dyn Db) -> IterationOutcome<'db> {
if let Type::Tuple(tuple_type) = self {
return IterationOutcome::Iterable {
element_ty: UnionType::from_elements(db, &**tuple_type.elements(db)),
@@ -467,18 +582,22 @@ impl<'db> Type<'db> {
let dunder_iter_method = iterable_meta_type.member(db, "__iter__");
if !dunder_iter_method.is_unbound() {
let Some(iterator_ty) = dunder_iter_method.call(db) else {
let CallOutcome::Callable {
return_ty: iterator_ty,
} = dunder_iter_method.call(db, &[])
else {
return IterationOutcome::NotIterable {
not_iterable_ty: *self,
not_iterable_ty: self,
};
};
let dunder_next_method = iterator_ty.to_meta_type(db).member(db, "__next__");
return dunder_next_method
.call(db)
.call(db, &[])
.return_ty(db)
.map(|element_ty| IterationOutcome::Iterable { element_ty })
.unwrap_or(IterationOutcome::NotIterable {
not_iterable_ty: *self,
not_iterable_ty: self,
});
}
@@ -491,10 +610,11 @@ impl<'db> Type<'db> {
let dunder_get_item_method = iterable_meta_type.member(db, "__getitem__");
dunder_get_item_method
.call(db)
.call(db, &[])
.return_ty(db)
.map(|element_ty| IterationOutcome::Iterable { element_ty })
.unwrap_or(IterationOutcome::NotIterable {
not_iterable_ty: *self,
not_iterable_ty: self,
})
}
@@ -514,6 +634,7 @@ impl<'db> Type<'db> {
Type::BooleanLiteral(_)
| Type::BytesLiteral(_)
| Type::Function(_)
| Type::RevealTypeFunction(_)
| Type::Instance(_)
| Type::Module(_)
| Type::IntLiteral(_)
@@ -536,7 +657,7 @@ impl<'db> Type<'db> {
Type::BooleanLiteral(_) => builtins_symbol_ty(db, "bool"),
Type::BytesLiteral(_) => builtins_symbol_ty(db, "bytes"),
Type::IntLiteral(_) => builtins_symbol_ty(db, "int"),
Type::Function(_) => types_symbol_ty(db, "FunctionType"),
Type::Function(_) | Type::RevealTypeFunction(_) => types_symbol_ty(db, "FunctionType"),
Type::Module(_) => types_symbol_ty(db, "ModuleType"),
Type::None => typeshed_symbol_ty(db, "NoneType"),
// TODO not accurate if there's a custom metaclass...
@@ -560,6 +681,173 @@ impl<'db> From<&Type<'db>> for Type<'db> {
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
enum CallOutcome<'db> {
Callable {
return_ty: Type<'db>,
},
RevealType {
return_ty: Type<'db>,
revealed_ty: Type<'db>,
},
NotCallable {
not_callable_ty: Type<'db>,
},
Union {
called_ty: Type<'db>,
outcomes: Box<[CallOutcome<'db>]>,
},
}
impl<'db> CallOutcome<'db> {
/// Create a new `CallOutcome::Callable` with given return type.
fn callable(return_ty: Type<'db>) -> CallOutcome {
CallOutcome::Callable { return_ty }
}
/// Create a new `CallOutcome::NotCallable` with given not-callable type.
fn not_callable(not_callable_ty: Type<'db>) -> CallOutcome {
CallOutcome::NotCallable { not_callable_ty }
}
/// Create a new `CallOutcome::RevealType` with given revealed and return types.
fn revealed(return_ty: Type<'db>, revealed_ty: Type<'db>) -> CallOutcome<'db> {
CallOutcome::RevealType {
return_ty,
revealed_ty,
}
}
/// Create a new `CallOutcome::Union` with given wrapped outcomes.
fn union(called_ty: Type<'db>, outcomes: impl Into<Box<[CallOutcome<'db>]>>) -> CallOutcome {
CallOutcome::Union {
called_ty,
outcomes: outcomes.into(),
}
}
/// Get the return type of the call, or `None` if not callable.
fn return_ty(&self, db: &'db dyn Db) -> Option<Type<'db>> {
match self {
Self::Callable { return_ty } => Some(*return_ty),
Self::RevealType {
return_ty,
revealed_ty: _,
} => Some(*return_ty),
Self::NotCallable { not_callable_ty: _ } => None,
Self::Union {
outcomes,
called_ty: _,
} => outcomes
.iter()
// If all outcomes are NotCallable, we return None; if some outcomes are callable
// and some are not, we return a union including Unknown.
.fold(None, |acc, outcome| {
let ty = outcome.return_ty(db);
match (acc, ty) {
(None, None) => None,
(None, Some(ty)) => Some(UnionBuilder::new(db).add(ty)),
(Some(builder), ty) => Some(builder.add(ty.unwrap_or(Type::Unknown))),
}
})
.map(UnionBuilder::build),
}
}
/// Get the return type of the call, emitting diagnostics if needed.
fn unwrap_with_diagnostic<'a>(
&self,
db: &'db dyn Db,
node: ast::AnyNodeRef,
builder: &'a mut TypeInferenceBuilder<'db>,
) -> Type<'db> {
match self {
Self::Callable { return_ty } => *return_ty,
Self::RevealType {
return_ty,
revealed_ty,
} => {
builder.add_diagnostic(
node,
"revealed-type",
format_args!("Revealed type is '{}'.", revealed_ty.display(db)),
);
*return_ty
}
Self::NotCallable { not_callable_ty } => {
builder.add_diagnostic(
node,
"call-non-callable",
format_args!(
"Object of type '{}' is not callable.",
not_callable_ty.display(db)
),
);
Type::Unknown
}
Self::Union {
outcomes,
called_ty,
} => {
let mut not_callable = vec![];
let mut union_builder = UnionBuilder::new(db);
let mut revealed = false;
for outcome in &**outcomes {
let return_ty = match outcome {
Self::NotCallable { not_callable_ty } => {
not_callable.push(*not_callable_ty);
Type::Unknown
}
Self::RevealType {
return_ty,
revealed_ty: _,
} => {
if revealed {
*return_ty
} else {
revealed = true;
outcome.unwrap_with_diagnostic(db, node, builder)
}
}
_ => outcome.unwrap_with_diagnostic(db, node, builder),
};
union_builder = union_builder.add(return_ty);
}
match not_callable[..] {
[] => {}
[elem] => builder.add_diagnostic(
node,
"call-non-callable",
format_args!(
"Object of type '{}' is not callable (due to union element '{}').",
called_ty.display(db),
elem.display(db),
),
),
_ if not_callable.len() == outcomes.len() => builder.add_diagnostic(
node,
"call-non-callable",
format_args!(
"Object of type '{}' is not callable.",
called_ty.display(db)
),
),
_ => builder.add_diagnostic(
node,
"call-non-callable",
format_args!(
"Object of type '{}' is not callable (due to union elements {}).",
called_ty.display(db),
not_callable.display(db),
),
),
}
union_builder.build()
}
}
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum IterationOutcome<'db> {
Iterable { element_ty: Type<'db> },
@@ -591,10 +879,27 @@ pub struct FunctionType<'db> {
definition: Definition<'db>,
/// types of all decorators on this function
decorators: Vec<Type<'db>>,
decorators: Box<[Type<'db>]>,
}
impl<'db> FunctionType<'db> {
/// Return true if this is a standard library function with given module name and name.
pub(crate) fn is_stdlib_symbol(self, db: &'db dyn Db, module_name: &str, name: &str) -> bool {
name == self.name(db)
&& file_to_module(db, self.definition(db).file(db)).is_some_and(|module| {
module.search_path().is_standard_library() && module.name() == module_name
})
}
/// Return true if this is a symbol with given name from `typing` or `typing_extensions`.
pub(crate) fn is_typing_symbol(self, db: &'db dyn Db, name: &str) -> bool {
name == self.name(db)
&& file_to_module(db, self.definition(db).file(db)).is_some_and(|module| {
module.search_path().is_standard_library()
&& matches!(&**module.name(), "typing" | "typing_extensions")
})
}
pub fn has_decorator(self, db: &dyn Db, decorator: Type<'_>) -> bool {
self.decorators(db).contains(&decorator)
}
@@ -642,12 +947,12 @@ pub struct ClassType<'db> {
}
impl<'db> ClassType<'db> {
pub(crate) fn is_builtin_named(self, db: &'db dyn Db, name: &str) -> bool {
name == self.name(db).as_str()
&& file_to_module(db, self.body_scope(db).file(db))
// Builtin module names are special-cased in the resolver, so there can't be a
// module named builtins other than the actual builtins.
.is_some_and(|module| module.name().as_str() == "builtins")
/// Return true if this class is a standard library type with given module name and name.
pub(crate) fn is_stdlib_symbol(self, db: &'db dyn Db, module_name: &str, name: &str) -> bool {
name == self.name(db)
&& file_to_module(db, self.body_scope(db).file(db)).is_some_and(|module| {
module.search_path().is_standard_library() && module.name() == module_name
})
}
/// Return an iterator over the types of this class's bases.
@@ -699,16 +1004,16 @@ impl<'db> ClassType<'db> {
pub struct UnionType<'db> {
/// The union type includes values in any of these types.
#[return_ref]
elements: FxOrderSet<Type<'db>>,
elements_boxed: Box<[Type<'db>]>,
}
impl<'db> UnionType<'db> {
pub fn contains(&self, db: &'db dyn Db, ty: Type<'db>) -> bool {
self.elements(db).contains(&ty)
fn elements(self, db: &'db dyn Db) -> &'db [Type<'db>] {
self.elements_boxed(db)
}
/// Create a union from a list of elements
/// (which may be eagerly simplified into a different variant of [`Type`] altogether)
/// (which may be eagerly simplified into a different variant of [`Type`] altogether).
pub fn from_elements<T: Into<Type<'db>>>(
db: &'db dyn Db,
elements: impl IntoIterator<Item = T>,
@@ -722,13 +1027,13 @@ impl<'db> UnionType<'db> {
}
/// Apply a transformation function to all elements of the union,
/// and create a new union from the resulting set of types
/// and create a new union from the resulting set of types.
pub fn map(
&self,
db: &'db dyn Db,
transform_fn: impl Fn(&Type<'db>) -> Type<'db>,
) -> Type<'db> {
Self::from_elements(db, self.elements(db).into_iter().map(transform_fn))
Self::from_elements(db, self.elements(db).iter().map(transform_fn))
}
}
@@ -767,8 +1072,6 @@ pub struct TupleType<'db> {
#[cfg(test)]
mod tests {
#![allow(clippy::needless_pass_by_value)]
use super::{builtins_symbol_ty, BytesLiteralType, StringLiteralType, Type, UnionType};
use crate::db::tests::TestDb;
use crate::program::{Program, SearchPathSettings};
@@ -809,16 +1112,16 @@ mod tests {
LiteralString,
BytesLiteral(&'static str),
BuiltinInstance(&'static str),
Union(Box<[Ty]>),
Union(Vec<Ty>),
}
impl Ty {
fn to_type<'db>(&self, db: &'db TestDb) -> Type<'db> {
fn into_type(self, db: &TestDb) -> Type<'_> {
match self {
Ty::Never => Type::Never,
Ty::Unknown => Type::Unknown,
Ty::Any => Type::Any,
Ty::IntLiteral(n) => Type::IntLiteral(*n),
Ty::IntLiteral(n) => Type::IntLiteral(n),
Ty::StringLiteral(s) => {
Type::StringLiteral(StringLiteralType::new(db, (*s).into()))
}
@@ -827,11 +1130,15 @@ mod tests {
Type::BytesLiteral(BytesLiteralType::new(db, s.as_bytes().into()))
}
Ty::BuiltinInstance(s) => builtins_symbol_ty(db, s).to_instance(db),
Ty::Union(tys) => UnionType::from_elements(db, tys.iter().map(|ty| ty.to_type(db))),
Ty::Union(tys) => {
UnionType::from_elements(db, tys.into_iter().map(|ty| ty.into_type(db)))
}
}
}
}
#[test_case(Ty::BuiltinInstance("str"), Ty::BuiltinInstance("object"))]
#[test_case(Ty::BuiltinInstance("int"), Ty::BuiltinInstance("object"))]
#[test_case(Ty::Unknown, Ty::IntLiteral(1))]
#[test_case(Ty::Any, Ty::IntLiteral(1))]
#[test_case(Ty::Never, Ty::IntLiteral(1))]
@@ -842,26 +1149,57 @@ mod tests {
#[test_case(Ty::StringLiteral("foo"), Ty::LiteralString)]
#[test_case(Ty::LiteralString, Ty::BuiltinInstance("str"))]
#[test_case(Ty::BytesLiteral("foo"), Ty::BuiltinInstance("bytes"))]
#[test_case(Ty::IntLiteral(1), Ty::Union(vec![Ty::BuiltinInstance("int"), Ty::BuiltinInstance("str")]))]
#[test_case(Ty::IntLiteral(1), Ty::Union(vec![Ty::Unknown, Ty::BuiltinInstance("str")]))]
fn is_assignable_to(from: Ty, to: Ty) {
let db = setup_db();
assert!(from.to_type(&db).is_assignable_to(&db, to.to_type(&db)));
assert!(from.into_type(&db).is_assignable_to(&db, to.into_type(&db)));
}
#[test_case(Ty::BuiltinInstance("object"), Ty::BuiltinInstance("int"))]
#[test_case(Ty::IntLiteral(1), Ty::BuiltinInstance("str"))]
#[test_case(Ty::BuiltinInstance("int"), Ty::BuiltinInstance("str"))]
#[test_case(Ty::BuiltinInstance("int"), Ty::IntLiteral(1))]
fn is_not_assignable_to(from: Ty, to: Ty) {
let db = setup_db();
assert!(!from.to_type(&db).is_assignable_to(&db, to.to_type(&db)));
assert!(!from.into_type(&db).is_assignable_to(&db, to.into_type(&db)));
}
#[test_case(Ty::BuiltinInstance("str"), Ty::BuiltinInstance("object"))]
#[test_case(Ty::BuiltinInstance("int"), Ty::BuiltinInstance("object"))]
#[test_case(Ty::Never, Ty::IntLiteral(1))]
#[test_case(Ty::IntLiteral(1), Ty::BuiltinInstance("int"))]
#[test_case(Ty::StringLiteral("foo"), Ty::BuiltinInstance("str"))]
#[test_case(Ty::StringLiteral("foo"), Ty::LiteralString)]
#[test_case(Ty::LiteralString, Ty::BuiltinInstance("str"))]
#[test_case(Ty::BytesLiteral("foo"), Ty::BuiltinInstance("bytes"))]
#[test_case(Ty::IntLiteral(1), Ty::Union(vec![Ty::BuiltinInstance("int"), Ty::BuiltinInstance("str")]))]
fn is_subtype_of(from: Ty, to: Ty) {
let db = setup_db();
assert!(from.into_type(&db).is_subtype_of(&db, to.into_type(&db)));
}
#[test_case(Ty::BuiltinInstance("object"), Ty::BuiltinInstance("int"))]
#[test_case(Ty::Unknown, Ty::IntLiteral(1))]
#[test_case(Ty::Any, Ty::IntLiteral(1))]
#[test_case(Ty::IntLiteral(1), Ty::Unknown)]
#[test_case(Ty::IntLiteral(1), Ty::Any)]
#[test_case(Ty::IntLiteral(1), Ty::Union(vec![Ty::Unknown, Ty::BuiltinInstance("str")]))]
#[test_case(Ty::IntLiteral(1), Ty::BuiltinInstance("str"))]
#[test_case(Ty::BuiltinInstance("int"), Ty::BuiltinInstance("str"))]
#[test_case(Ty::BuiltinInstance("int"), Ty::IntLiteral(1))]
fn is_not_subtype_of(from: Ty, to: Ty) {
let db = setup_db();
assert!(!from.into_type(&db).is_subtype_of(&db, to.into_type(&db)));
}
#[test_case(
Ty::Union(Box::new([Ty::IntLiteral(1), Ty::IntLiteral(2)])),
Ty::Union(Box::new([Ty::IntLiteral(1), Ty::IntLiteral(2)]))
Ty::Union(vec![Ty::IntLiteral(1), Ty::IntLiteral(2)]),
Ty::Union(vec![Ty::IntLiteral(1), Ty::IntLiteral(2)])
)]
fn is_equivalent_to(from: Ty, to: Ty) {
let db = setup_db();
assert!(from.to_type(&db).is_equivalent_to(&db, to.to_type(&db)));
assert!(from.into_type(&db).is_equivalent_to(&db, to.into_type(&db)));
}
}

View File

@@ -27,10 +27,10 @@
//! * An intersection containing two non-overlapping types should simplify to [`Type::Never`].
use crate::types::{builtins_symbol_ty, IntersectionType, Type, UnionType};
use crate::{Db, FxOrderSet};
use ordermap::set::MutableValues;
use smallvec::SmallVec;
pub(crate) struct UnionBuilder<'db> {
elements: FxOrderSet<Type<'db>>,
elements: Vec<Type<'db>>,
db: &'db dyn Db,
}
@@ -38,7 +38,7 @@ impl<'db> UnionBuilder<'db> {
pub(crate) fn new(db: &'db dyn Db) -> Self {
Self {
db,
elements: FxOrderSet::default(),
elements: vec![],
}
}
@@ -46,47 +46,70 @@ impl<'db> UnionBuilder<'db> {
pub(crate) fn add(mut self, ty: Type<'db>) -> Self {
match ty {
Type::Union(union) => {
self.elements.extend(union.elements(self.db));
let new_elements = union.elements(self.db);
self.elements.reserve(new_elements.len());
for element in new_elements {
self = self.add(*element);
}
}
Type::Never => {}
_ => {
self.elements.insert(ty);
let bool_pair = if let Type::BooleanLiteral(b) = ty {
Some(Type::BooleanLiteral(!b))
} else {
None
};
let mut to_add = ty;
let mut to_remove = SmallVec::<[usize; 2]>::new();
for (index, element) in self.elements.iter().enumerate() {
if Some(*element) == bool_pair {
to_add = builtins_symbol_ty(self.db, "bool");
to_remove.push(index);
// The type we are adding is a BooleanLiteral, which doesn't have any
// subtypes. And we just found that the union already contained our
// mirror-image BooleanLiteral, so it can't also contain bool or any
// supertype of bool. Therefore, we are done.
break;
}
if ty.is_subtype_of(self.db, *element) {
return self;
} else if element.is_subtype_of(self.db, ty) {
to_remove.push(index);
}
}
match to_remove[..] {
[] => self.elements.push(to_add),
[index] => self.elements[index] = to_add,
_ => {
let mut current_index = 0;
let mut to_remove = to_remove.into_iter();
let mut next_to_remove_index = to_remove.next();
self.elements.retain(|_| {
let retain = if Some(current_index) == next_to_remove_index {
next_to_remove_index = to_remove.next();
false
} else {
true
};
current_index += 1;
retain
});
self.elements.push(to_add);
}
}
}
}
self
}
/// Performs the following normalizations:
/// - Replaces `Literal[True,False]` with `bool`.
/// - TODO For enums `E` with members `X1`,...,`Xn`, replaces
/// `Literal[E.X1,...,E.Xn]` with `E`.
fn simplify(&mut self) {
if let Some(true_index) = self.elements.get_index_of(&Type::BooleanLiteral(true)) {
if self.elements.contains(&Type::BooleanLiteral(false)) {
*self.elements.get_index_mut2(true_index).unwrap() =
builtins_symbol_ty(self.db, "bool");
self.elements.remove(&Type::BooleanLiteral(false));
}
}
}
pub(crate) fn build(mut self) -> Type<'db> {
pub(crate) fn build(self) -> Type<'db> {
match self.elements.len() {
0 => Type::Never,
1 => self.elements[0],
_ => {
self.simplify();
match self.elements.len() {
0 => Type::Never,
1 => self.elements[0],
_ => {
self.elements.shrink_to_fit();
Type::Union(UnionType::new(self.db, self.elements))
}
}
}
_ => Type::Union(UnionType::new(self.db, self.elements.into())),
}
}
}
@@ -280,12 +303,6 @@ mod tests {
use crate::ProgramSettings;
use ruff_db::system::{DbWithTestSystem, SystemPathBuf};
impl<'db> UnionType<'db> {
fn elements_vec(self, db: &'db TestDb) -> Vec<Type<'db>> {
self.elements(db).into_iter().copied().collect()
}
}
fn setup_db() -> TestDb {
let db = TestDb::new();
@@ -313,7 +330,7 @@ mod tests {
let t1 = Type::IntLiteral(1);
let union = UnionType::from_elements(&db, [t0, t1]).expect_union();
assert_eq!(union.elements_vec(&db), &[t0, t1]);
assert_eq!(union.elements(&db), &[t0, t1]);
}
#[test]
@@ -350,10 +367,10 @@ mod tests {
let t3 = Type::IntLiteral(17);
let union = UnionType::from_elements(&db, [t0, t1, t3]).expect_union();
assert_eq!(union.elements_vec(&db), &[t0, t3]);
assert_eq!(union.elements(&db), &[t0, t3]);
let union = UnionType::from_elements(&db, [t0, t1, t2, t3]).expect_union();
assert_eq!(union.elements_vec(&db), &[bool_ty, t3]);
assert_eq!(union.elements(&db), &[bool_ty, t3]);
}
#[test]
@@ -365,7 +382,44 @@ mod tests {
let u1 = UnionType::from_elements(&db, [t0, t1]);
let union = UnionType::from_elements(&db, [u1, t2]).expect_union();
assert_eq!(union.elements_vec(&db), &[t0, t1, t2]);
assert_eq!(union.elements(&db), &[t0, t1, t2]);
}
#[test]
fn build_union_simplify_subtype() {
let db = setup_db();
let t0 = builtins_symbol_ty(&db, "str").to_instance(&db);
let t1 = Type::LiteralString;
let u0 = UnionType::from_elements(&db, [t0, t1]);
let u1 = UnionType::from_elements(&db, [t1, t0]);
assert_eq!(u0, t0);
assert_eq!(u1, t0);
}
#[test]
fn build_union_no_simplify_unknown() {
let db = setup_db();
let t0 = builtins_symbol_ty(&db, "str").to_instance(&db);
let t1 = Type::Unknown;
let u0 = UnionType::from_elements(&db, [t0, t1]);
let u1 = UnionType::from_elements(&db, [t1, t0]);
assert_eq!(u0.expect_union().elements(&db), &[t0, t1]);
assert_eq!(u1.expect_union().elements(&db), &[t1, t0]);
}
#[test]
fn build_union_subsume_multiple() {
let db = setup_db();
let str_ty = builtins_symbol_ty(&db, "str").to_instance(&db);
let int_ty = builtins_symbol_ty(&db, "int").to_instance(&db);
let object_ty = builtins_symbol_ty(&db, "object").to_instance(&db);
let unknown_ty = Type::Unknown;
let u0 = UnionType::from_elements(&db, [str_ty, unknown_ty, int_ty, object_ty]);
assert_eq!(u0.expect_union().elements(&db), &[unknown_ty, object_ty]);
}
impl<'db> IntersectionType<'db> {
@@ -446,7 +500,7 @@ mod tests {
.add_positive(u0)
.build()
.expect_union();
let [Type::Intersection(i0), Type::Intersection(i1)] = union.elements_vec(&db)[..] else {
let [Type::Intersection(i0), Type::Intersection(i1)] = union.elements(&db)[..] else {
panic!("expected a union of two intersections");
};
assert_eq!(i0.pos_vec(&db), &[ta, t0]);

View File

@@ -1,19 +1,20 @@
//! Display implementations for types.
use std::fmt::{Display, Formatter};
use std::fmt::{self, Display, Formatter};
use ruff_db::display::FormatterJoinExtension;
use ruff_python_ast::str::Quote;
use ruff_python_literal::escape::AsciiEscape;
use crate::types::{IntersectionType, Type, UnionType};
use crate::{Db, FxOrderMap};
use crate::Db;
use rustc_hash::FxHashMap;
impl<'db> Type<'db> {
pub fn display(&'db self, db: &'db dyn Db) -> DisplayType<'db> {
pub fn display(&self, db: &'db dyn Db) -> DisplayType {
DisplayType { ty: self, db }
}
fn representation(&'db self, db: &'db dyn Db) -> DisplayRepresentation<'db> {
fn representation(self, db: &'db dyn Db) -> DisplayRepresentation<'db> {
DisplayRepresentation { db, ty: self }
}
}
@@ -25,7 +26,7 @@ pub struct DisplayType<'db> {
}
impl Display for DisplayType<'_> {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {
let representation = self.ty.representation(self.db);
if matches!(
self.ty,
@@ -35,6 +36,7 @@ impl Display for DisplayType<'_> {
| Type::BytesLiteral(_)
| Type::Class(_)
| Type::Function(_)
| Type::RevealTypeFunction(_)
) {
write!(f, "Literal[{representation}]",)
} else {
@@ -43,9 +45,9 @@ impl Display for DisplayType<'_> {
}
}
impl std::fmt::Debug for DisplayType<'_> {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
std::fmt::Display::fmt(self, f)
impl fmt::Debug for DisplayType<'_> {
fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {
Display::fmt(self, f)
}
}
@@ -53,12 +55,12 @@ impl std::fmt::Debug for DisplayType<'_> {
/// `Literal[<repr>]` or `Literal[<repr1>, <repr2>]` for literal types or as `<repr>` for
/// non literals
struct DisplayRepresentation<'db> {
ty: &'db Type<'db>,
ty: Type<'db>,
db: &'db dyn Db,
}
impl std::fmt::Display for DisplayRepresentation<'_> {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
impl Display for DisplayRepresentation<'_> {
fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {
match self.ty {
Type::Any => f.write_str("Any"),
Type::Never => f.write_str("Never"),
@@ -71,11 +73,13 @@ impl std::fmt::Display for DisplayRepresentation<'_> {
// TODO functions and classes should display using a fully qualified name
Type::Class(class) => f.write_str(class.name(self.db)),
Type::Instance(class) => f.write_str(class.name(self.db)),
Type::Function(function) => f.write_str(function.name(self.db)),
Type::Function(function) | Type::RevealTypeFunction(function) => {
f.write_str(function.name(self.db))
}
Type::Union(union) => union.display(self.db).fmt(f),
Type::Intersection(intersection) => intersection.display(self.db).fmt(f),
Type::IntLiteral(n) => write!(f, "{n}"),
Type::BooleanLiteral(boolean) => f.write_str(if *boolean { "True" } else { "False" }),
Type::IntLiteral(n) => n.fmt(f),
Type::BooleanLiteral(boolean) => f.write_str(if boolean { "True" } else { "False" }),
Type::StringLiteral(string) => {
write!(f, r#""{}""#, string.value(self.db).replace('"', r#"\""#))
}
@@ -92,14 +96,7 @@ impl std::fmt::Display for DisplayRepresentation<'_> {
if elements.is_empty() {
f.write_str("()")?;
} else {
let mut first = true;
for element in &**elements {
if !first {
f.write_str(", ")?;
}
first = false;
element.display(self.db).fmt(f)?;
}
elements.display(self.db).fmt(f)?;
}
f.write_str("]")
}
@@ -119,11 +116,11 @@ struct DisplayUnionType<'db> {
}
impl Display for DisplayUnionType<'_> {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {
let elements = self.ty.elements(self.db);
// Group literal types by kind.
let mut grouped_literals = FxOrderMap::default();
let mut grouped_literals = FxHashMap::default();
for element in elements {
if let Ok(literal_kind) = LiteralTypeKind::try_from(*element) {
@@ -134,52 +131,51 @@ impl Display for DisplayUnionType<'_> {
}
}
let mut first = true;
let mut join = f.join(" | ");
// Print all types, but write all literals together (while preserving their position).
for ty in elements {
if let Ok(literal_kind) = LiteralTypeKind::try_from(*ty) {
for element in elements {
if let Ok(literal_kind) = LiteralTypeKind::try_from(*element) {
let Some(mut literals) = grouped_literals.remove(&literal_kind) else {
continue;
};
if !first {
f.write_str(" | ")?;
};
f.write_str("Literal[")?;
if literal_kind == LiteralTypeKind::IntLiteral {
literals.sort_unstable_by_key(|ty| ty.expect_int_literal());
}
for (i, literal_ty) in literals.iter().enumerate() {
if i > 0 {
f.write_str(", ")?;
}
literal_ty.representation(self.db).fmt(f)?;
}
f.write_str("]")?;
join.entry(&DisplayLiteralGroup {
literals,
db: self.db,
});
} else {
if !first {
f.write_str(" | ")?;
};
ty.display(self.db).fmt(f)?;
join.entry(&element.display(self.db));
}
first = false;
}
join.finish()?;
debug_assert!(grouped_literals.is_empty());
Ok(())
}
}
impl std::fmt::Debug for DisplayUnionType<'_> {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
std::fmt::Display::fmt(self, f)
impl fmt::Debug for DisplayUnionType<'_> {
fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {
Display::fmt(self, f)
}
}
struct DisplayLiteralGroup<'db> {
literals: Vec<Type<'db>>,
db: &'db dyn Db,
}
impl Display for DisplayLiteralGroup<'_> {
fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {
f.write_str("Literal[")?;
f.join(", ")
.entries(self.literals.iter().map(|ty| ty.representation(self.db)))
.finish()?;
f.write_str("]")
}
}
@@ -198,7 +194,7 @@ impl TryFrom<Type<'_>> for LiteralTypeKind {
fn try_from(value: Type<'_>) -> Result<Self, Self::Error> {
match value {
Type::Class(_) => Ok(Self::Class),
Type::Function(_) => Ok(Self::Function),
Type::Function(_) | Type::RevealTypeFunction(_) => Ok(Self::Function),
Type::IntLiteral(_) => Ok(Self::IntLiteral),
Type::StringLiteral(_) => Ok(Self::StringLiteral),
Type::BytesLiteral(_) => Ok(Self::BytesLiteral),
@@ -219,31 +215,77 @@ struct DisplayIntersectionType<'db> {
}
impl Display for DisplayIntersectionType<'_> {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
let mut first = true;
for (neg, ty) in self
fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {
let tys = self
.ty
.positive(self.db)
.iter()
.map(|ty| (false, ty))
.chain(self.ty.negative(self.db).iter().map(|ty| (true, ty)))
{
if !first {
f.write_str(" & ")?;
};
first = false;
if neg {
f.write_str("~")?;
};
write!(f, "{}", ty.display(self.db))?;
}
Ok(())
.map(|&ty| DisplayMaybeNegatedType {
ty,
db: self.db,
negated: false,
})
.chain(
self.ty
.negative(self.db)
.iter()
.map(|&ty| DisplayMaybeNegatedType {
ty,
db: self.db,
negated: true,
}),
);
f.join(" & ").entries(tys).finish()
}
}
impl std::fmt::Debug for DisplayIntersectionType<'_> {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
std::fmt::Display::fmt(self, f)
impl fmt::Debug for DisplayIntersectionType<'_> {
fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {
Display::fmt(self, f)
}
}
struct DisplayMaybeNegatedType<'db> {
ty: Type<'db>,
db: &'db dyn Db,
negated: bool,
}
impl<'db> Display for DisplayMaybeNegatedType<'db> {
fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {
if self.negated {
f.write_str("~")?;
}
self.ty.display(self.db).fmt(f)
}
}
pub(crate) trait TypeArrayDisplay<'db> {
fn display(&self, db: &'db dyn Db) -> DisplayTypeArray;
}
impl<'db> TypeArrayDisplay<'db> for Box<[Type<'db>]> {
fn display(&self, db: &'db dyn Db) -> DisplayTypeArray {
DisplayTypeArray { types: self, db }
}
}
impl<'db> TypeArrayDisplay<'db> for Vec<Type<'db>> {
fn display(&self, db: &'db dyn Db) -> DisplayTypeArray {
DisplayTypeArray { types: self, db }
}
}
pub(crate) struct DisplayTypeArray<'b, 'db> {
types: &'b [Type<'db>],
db: &'db dyn Db,
}
impl<'db> Display for DisplayTypeArray<'_, 'db> {
fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {
f.join(", ")
.entries(self.types.iter().map(|ty| ty.display(self.db)))
.finish()
}
}

View File

@@ -50,8 +50,9 @@ use crate::semantic_index::SemanticIndex;
use crate::stdlib::builtins_module_scope;
use crate::types::diagnostic::{TypeCheckDiagnostic, TypeCheckDiagnostics};
use crate::types::{
bindings_ty, builtins_symbol_ty, declaration_ty, global_symbol_ty, symbol_ty, BytesLiteralType,
ClassType, FunctionType, StringLiteralType, TupleType, Type, UnionType,
bindings_ty, builtins_symbol_ty, declarations_ty, global_symbol_ty, symbol_ty,
typing_extensions_symbol_ty, BytesLiteralType, ClassType, FunctionType, StringLiteralType,
TupleType, Type, TypeArrayDisplay, UnionType,
};
use crate::Db;
@@ -81,11 +82,11 @@ fn infer_definition_types_cycle_recovery<'db>(
) -> TypeInference<'db> {
tracing::trace!("infer_definition_types_cycle_recovery");
let mut inference = TypeInference::default();
let kind = input.kind(db);
if kind.is_declaration() {
let category = input.category(db);
if category.is_declaration() {
inference.declarations.insert(input, Type::Unknown);
}
if kind.is_binding() {
if category.is_binding() {
inference.bindings.insert(input, Type::Unknown);
}
// TODO we don't fill in expression types for the cycle-participant definitions, which can
@@ -317,9 +318,10 @@ impl<'db> TypeInferenceBuilder<'db> {
self.types.has_deferred |= inference.has_deferred;
}
/// Are we currently inferring types in a stub file?
fn is_stub(&self) -> bool {
self.file.is_stub(self.db.upcast())
/// Are we currently inferring types in file with deferred types?
/// This is true for stub files and files with `__future__.annotations`
fn are_all_types_deferred(&self) -> bool {
self.index.has_future_annotations() || self.file.is_stub(self.db.upcast())
}
/// Are we currently inferring deferred types?
@@ -502,43 +504,42 @@ impl<'db> TypeInferenceBuilder<'db> {
}
fn add_binding(&mut self, node: AnyNodeRef, binding: Definition<'db>, ty: Type<'db>) {
debug_assert!(binding.kind(self.db).is_binding());
debug_assert!(binding.is_binding(self.db));
let use_def = self.index.use_def_map(binding.file_scope(self.db));
let mut declared_tys = use_def
.declarations_at_binding(binding)
.map(|declaration| declaration_ty(self.db, declaration));
let declarations = use_def.declarations_at_binding(binding);
let undeclared_ty = if declarations.may_be_undeclared() {
Some(Type::Unknown)
} else {
None
};
let mut bound_ty = ty;
if let Some(declared_ty) = declared_tys.next() {
let mut check_compat = true;
for other_declared_ty in declared_tys {
if !declared_ty.is_equivalent_to(self.db, other_declared_ty) {
// TODO point out the conflicting declarations in the diagnostic?
let symbol_table = self.index.symbol_table(binding.file_scope(self.db));
let symbol_name = symbol_table.symbol(binding.symbol(self.db)).name();
self.add_diagnostic(
node,
"conflicting-declarations",
format_args!(
"Conflicting declared types for '{symbol_name}': '{}', '{}'.",
declared_ty.display(self.db),
other_declared_ty.display(self.db)
),
);
check_compat = false;
break;
}
}
if check_compat && !bound_ty.is_assignable_to(self.db, declared_ty) {
self.invalid_assignment_diagnostic(node, declared_ty, bound_ty);
bound_ty = declared_ty;
}
let declared_ty = declarations_ty(self.db, declarations, undeclared_ty).unwrap_or_else(
|(ty, conflicting)| {
// TODO point out the conflicting declarations in the diagnostic?
let symbol_table = self.index.symbol_table(binding.file_scope(self.db));
let symbol_name = symbol_table.symbol(binding.symbol(self.db)).name();
self.add_diagnostic(
node,
"conflicting-declarations",
format_args!(
"Conflicting declared types for '{symbol_name}': {}.",
conflicting.display(self.db)
),
);
ty
},
);
if !bound_ty.is_assignable_to(self.db, declared_ty) {
self.invalid_assignment_diagnostic(node, declared_ty, bound_ty);
// allow declarations to override inference in case of invalid assignment
bound_ty = declared_ty;
};
self.types.bindings.insert(binding, bound_ty);
}
fn add_declaration(&mut self, node: AnyNodeRef, declaration: Definition<'db>, ty: Type<'db>) {
debug_assert!(declaration.kind(self.db).is_declaration());
debug_assert!(declaration.is_declaration(self.db));
let use_def = self.index.use_def_map(declaration.file_scope(self.db));
let prior_bindings = use_def.bindings_at_declaration(declaration);
// unbound_ty is Never because for this check we don't care about unbound
@@ -567,8 +568,8 @@ impl<'db> TypeInferenceBuilder<'db> {
declared_ty: Type<'db>,
inferred_ty: Type<'db>,
) {
debug_assert!(definition.kind(self.db).is_binding());
debug_assert!(definition.kind(self.db).is_declaration());
debug_assert!(definition.is_binding(self.db));
debug_assert!(definition.is_declaration(self.db));
let inferred_ty = if inferred_ty.is_assignable_to(self.db, declared_ty) {
inferred_ty
} else {
@@ -703,19 +704,19 @@ impl<'db> TypeInferenceBuilder<'db> {
self.infer_parameters(parameters);
// TODO: this should also be applied to parameter annotations.
if self.is_stub() {
if self.are_all_types_deferred() {
self.types.has_deferred = true;
} else {
self.infer_optional_annotation_expression(returns.as_deref());
}
}
let function_ty = Type::Function(FunctionType::new(
self.db,
name.id.clone(),
definition,
decorator_tys,
));
let function_type = FunctionType::new(self.db, name.id.clone(), definition, decorator_tys);
let function_ty = if function_type.is_typing_symbol(self.db, "reveal_type") {
Type::RevealTypeFunction(function_type)
} else {
Type::Function(function_type)
};
self.add_declaration_with_binding(function.into(), definition, function_ty, function_ty);
}
@@ -831,9 +832,9 @@ impl<'db> TypeInferenceBuilder<'db> {
self.infer_expression(&keyword.value);
}
// inference of bases deferred in stubs
// Inference of bases deferred in stubs
// TODO also defer stringified generic type parameters
if self.is_stub() {
if self.are_all_types_deferred() {
self.types.has_deferred = true;
} else {
for base in class.bases() {
@@ -843,13 +844,11 @@ impl<'db> TypeInferenceBuilder<'db> {
}
fn infer_function_deferred(&mut self, function: &ast::StmtFunctionDef) {
if self.is_stub() {
self.infer_optional_annotation_expression(function.returns.as_deref());
}
self.infer_optional_annotation_expression(function.returns.as_deref());
}
fn infer_class_deferred(&mut self, class: &ast::StmtClassDef) {
if self.is_stub() {
if self.are_all_types_deferred() {
for base in class.bases() {
self.infer_expression(base);
}
@@ -927,14 +926,13 @@ impl<'db> TypeInferenceBuilder<'db> {
} = with_statement;
for item in items {
match item.optional_vars.as_deref() {
Some(ast::Expr::Name(name)) => {
self.infer_definition(name);
}
_ => {
// TODO infer definitions in unpacking assignment
self.infer_expression(&item.context_expr);
}
let target = item.optional_vars.as_deref();
if let Some(ast::Expr::Name(name)) = target {
self.infer_definition(name);
} else {
// TODO infer definitions in unpacking assignment
self.infer_expression(&item.context_expr);
self.infer_optional_expression(target);
}
}
@@ -1247,7 +1245,7 @@ impl<'db> TypeInferenceBuilder<'db> {
node,
"not-iterable",
format_args!(
"Object of type '{}' is not iterable",
"Object of type '{}' is not iterable.",
not_iterable_ty.display(self.db)
),
);
@@ -1322,7 +1320,7 @@ impl<'db> TypeInferenceBuilder<'db> {
Type::Unknown
};
self.add_binding(alias.into(), definition, module_ty);
self.add_declaration_with_binding(alias.into(), definition, module_ty, module_ty);
}
fn infer_import_from_statement(&mut self, import: &ast::StmtImportFrom) {
@@ -1506,11 +1504,9 @@ impl<'db> TypeInferenceBuilder<'db> {
// the runtime error will occur immediately (rather than when the symbol is *used*,
// as would be the case for a symbol with type `Unbound`), so it's appropriate to
// think of the type of the imported symbol as `Unknown` rather than `Unbound`
self.add_binding(
alias.into(),
definition,
member_ty.replace_unbound_with(self.db, Type::Unknown),
);
let ty = member_ty.replace_unbound_with(self.db, Type::Unknown);
self.add_declaration_with_binding(alias.into(), definition, ty, ty);
}
fn infer_return_statement(&mut self, ret: &ast::StmtReturn) {
@@ -2029,19 +2025,12 @@ impl<'db> TypeInferenceBuilder<'db> {
arguments,
} = call_expression;
self.infer_arguments(arguments);
// TODO: proper typed call signature, representing keyword args etc
let arg_types = self.infer_arguments(arguments);
let function_type = self.infer_expression(func);
function_type.call(self.db).unwrap_or_else(|| {
self.add_diagnostic(
func.as_ref().into(),
"call-non-callable",
format_args!(
"Object of type '{}' is not callable",
function_type.display(self.db)
),
);
Type::Unknown
})
function_type
.call(self.db, arg_types.as_slice())
.unwrap_with_diagnostic(self.db, func.as_ref().into(), self)
}
fn infer_starred_expression(&mut self, starred: &ast::ExprStarred) -> Type<'db> {
@@ -2091,7 +2080,8 @@ impl<'db> TypeInferenceBuilder<'db> {
}
/// Look up a name reference that isn't bound in the local scope.
fn lookup_name(&self, name: &ast::name::Name) -> Type<'db> {
fn lookup_name(&mut self, name_node: &ast::ExprName) -> Type<'db> {
let ast::ExprName { id: name, .. } = name_node;
let file_scope_id = self.scope.file_scope_id(self.db);
let is_bound = self
.index
@@ -2136,7 +2126,17 @@ impl<'db> TypeInferenceBuilder<'db> {
};
// Fallback to builtins (without infinite recursion if we're already in builtins.)
if ty.may_be_unbound(self.db) && Some(self.scope) != builtins_module_scope(self.db) {
ty.replace_unbound_with(self.db, builtins_symbol_ty(self.db, name))
let mut builtin_ty = builtins_symbol_ty(self.db, name);
if builtin_ty.is_unbound() && name == "reveal_type" {
self.add_diagnostic(
name_node.into(),
"undefined-reveal",
format_args!(
"'reveal_type' used without importing it; this is allowed for debugging convenience but will fail at runtime."),
);
builtin_ty = typing_extensions_symbol_ty(self.db, name);
}
ty.replace_unbound_with(self.db, builtin_ty)
} else {
ty
}
@@ -2172,7 +2172,7 @@ impl<'db> TypeInferenceBuilder<'db> {
};
let unbound_ty = if may_be_unbound {
Some(self.lookup_name(id))
Some(self.lookup_name(name))
} else {
None
};
@@ -2416,7 +2416,12 @@ impl<'db> TypeInferenceBuilder<'db> {
/// Adds a new diagnostic.
///
/// The diagnostic does not get added if the rule isn't enabled for this file.
fn add_diagnostic(&mut self, node: AnyNodeRef, rule: &str, message: std::fmt::Arguments) {
pub(super) fn add_diagnostic(
&mut self,
node: AnyNodeRef,
rule: &str,
message: std::fmt::Arguments,
) {
if !self.db.is_file_open(self.file) {
return;
}
@@ -2752,6 +2757,87 @@ mod tests {
assert_diagnostic_messages(&diagnostics, expected);
}
#[test]
fn reveal_type() -> anyhow::Result<()> {
let mut db = setup_db();
db.write_dedented(
"/src/a.py",
"
from typing import reveal_type
x = 1
reveal_type(x)
",
)?;
assert_file_diagnostics(&db, "/src/a.py", &["Revealed type is 'Literal[1]'."]);
Ok(())
}
#[test]
fn reveal_type_aliased() -> anyhow::Result<()> {
let mut db = setup_db();
db.write_dedented(
"/src/a.py",
"
from typing import reveal_type as rt
x = 1
rt(x)
",
)?;
assert_file_diagnostics(&db, "/src/a.py", &["Revealed type is 'Literal[1]'."]);
Ok(())
}
#[test]
fn reveal_type_typing_extensions() -> anyhow::Result<()> {
let mut db = setup_db();
db.write_dedented(
"/src/a.py",
"
import typing_extensions
x = 1
typing_extensions.reveal_type(x)
",
)?;
assert_file_diagnostics(&db, "/src/a.py", &["Revealed type is 'Literal[1]'."]);
Ok(())
}
#[test]
fn reveal_type_builtin() -> anyhow::Result<()> {
let mut db = setup_db();
db.write_dedented(
"/src/a.py",
"
x = 1
reveal_type(x)
",
)?;
assert_file_diagnostics(
&db,
"/src/a.py",
&[
"'reveal_type' used without importing it; this is allowed for debugging convenience but will fail at runtime.",
"Revealed type is 'Literal[1]'.",
],
);
Ok(())
}
#[test]
fn follow_import_to_class() -> anyhow::Result<()> {
let mut db = setup_db();
@@ -3234,7 +3320,7 @@ mod tests {
)?;
// TODO: sys.version_info, and need to understand @final and @type_check_only
assert_public_ty(&db, "src/a.py", "x", "EllipsisType | Unknown");
assert_public_ty(&db, "src/a.py", "x", "Unknown | EllipsisType");
Ok(())
}
@@ -3339,6 +3425,129 @@ mod tests {
Ok(())
}
#[test]
fn call_union() -> anyhow::Result<()> {
let mut db = setup_db();
db.write_dedented(
"src/a.py",
"
if flag:
def f() -> int:
return 1
else:
def f() -> str:
return 'foo'
x = f()
",
)?;
assert_public_ty(&db, "src/a.py", "x", "int | str");
Ok(())
}
#[test]
fn call_union_with_unknown() -> anyhow::Result<()> {
let mut db = setup_db();
db.write_dedented(
"src/a.py",
"
from nonexistent import f
if flag:
def f() -> int:
return 1
x = f()
",
)?;
assert_public_ty(&db, "src/a.py", "x", "Unknown | int");
Ok(())
}
#[test]
fn call_union_with_not_callable() -> anyhow::Result<()> {
let mut db = setup_db();
db.write_dedented(
"src/a.py",
"
if flag:
f = 1
else:
def f() -> int:
return 1
x = f()
",
)?;
assert_file_diagnostics(
&db,
"src/a.py",
&["Object of type 'Literal[1] | Literal[f]' is not callable (due to union element 'Literal[1]')."],
);
assert_public_ty(&db, "src/a.py", "x", "Unknown | int");
Ok(())
}
#[test]
fn call_union_with_multiple_not_callable() -> anyhow::Result<()> {
let mut db = setup_db();
db.write_dedented(
"src/a.py",
"
if flag:
f = 1
elif flag2:
f = 'foo'
else:
def f() -> int:
return 1
x = f()
",
)?;
assert_file_diagnostics(
&db,
"src/a.py",
&[
r#"Object of type 'Literal[1] | Literal["foo"] | Literal[f]' is not callable (due to union elements Literal[1], Literal["foo"])."#,
],
);
assert_public_ty(&db, "src/a.py", "x", "Unknown | int");
Ok(())
}
#[test]
fn call_union_with_all_not_callable() -> anyhow::Result<()> {
let mut db = setup_db();
db.write_dedented(
"src/a.py",
"
if flag:
f = 1
else:
f = 'foo'
x = f()
",
)?;
assert_file_diagnostics(
&db,
"src/a.py",
&[r#"Object of type 'Literal[1] | Literal["foo"]' is not callable."#],
);
assert_public_ty(&db, "src/a.py", "x", "Unknown");
Ok(())
}
#[test]
fn invalid_callable() {
let mut db = setup_db();
@@ -3355,7 +3564,7 @@ mod tests {
assert_file_diagnostics(
&db,
"/src/a.py",
&["Object of type 'Literal[123]' is not callable"],
&["Object of type 'Literal[123]' is not callable."],
);
}
@@ -4015,6 +4224,63 @@ mod tests {
Ok(())
}
#[test]
fn deferred_annotation_in_stubs_always_resolve() -> anyhow::Result<()> {
let mut db = setup_db();
// Stub files should always resolve deferred annotations
db.write_dedented(
"/src/stub.pyi",
"
def get_foo() -> Foo: ...
class Foo: ...
foo = get_foo()
",
)?;
assert_public_ty(&db, "/src/stub.pyi", "foo", "Foo");
Ok(())
}
#[test]
fn deferred_annotations_regular_source_fails() -> anyhow::Result<()> {
let mut db = setup_db();
// In (regular) source files, deferred annotations are *not* resolved
// Also tests imports from `__future__` that are not annotations
db.write_dedented(
"/src/source.py",
"
from __future__ import with_statement as annotations
def get_foo() -> Foo: ...
class Foo: ...
foo = get_foo()
",
)?;
assert_public_ty(&db, "/src/source.py", "foo", "Unknown");
Ok(())
}
#[test]
fn deferred_annotation_in_sources_with_future_resolves() -> anyhow::Result<()> {
let mut db = setup_db();
// In source files with `__future__.annotations`, deferred annotations are resolved
db.write_dedented(
"/src/source_with_future.py",
"
from __future__ import annotations
def get_foo() -> Foo: ...
class Foo: ...
foo = get_foo()
",
)?;
assert_public_ty(&db, "/src/source_with_future.py", "foo", "Foo");
Ok(())
}
#[test]
fn narrow_not_none() -> anyhow::Result<()> {
let mut db = setup_db();
@@ -4672,6 +4938,34 @@ mod tests {
Ok(())
}
#[test]
fn for_loop_non_callable_iter() -> anyhow::Result<()> {
let mut db = setup_db();
db.write_dedented(
"src/a.py",
"
class NotIterable:
if flag:
__iter__ = 1
else:
__iter__ = None
for x in NotIterable():
pass
",
)?;
assert_file_diagnostics(
&db,
"src/a.py",
&["Object of type 'NotIterable' is not iterable."],
);
assert_public_ty(&db, "src/a.py", "x", "Unbound | Unknown");
Ok(())
}
#[test]
fn except_handler_single_exception() -> anyhow::Result<()> {
let mut db = setup_db();
@@ -4791,8 +5085,9 @@ mod tests {
assert_file_diagnostics(&db, "src/a.py", &[]);
// TODO: once we support `sys.version_info` branches,
// we should set `--target-version=py311` in this test
assert_public_ty(&db, "src/a.py", "e", "BaseExceptionGroup");
// we can set `--target-version=py311` in this test
// and the inferred type will just be `BaseExceptionGroup` --Alex
assert_public_ty(&db, "src/a.py", "e", "Unknown | BaseExceptionGroup");
Ok(())
}
@@ -4814,10 +5109,11 @@ mod tests {
assert_file_diagnostics(&db, "src/a.py", &[]);
// TODO: once we support `sys.version_info` branches,
// we should set `--target-version=py311` in this test
// we can set `--target-version=py311` in this test
// and the inferred type will just be `BaseExceptionGroup` --Alex
//
// TODO more precise would be `ExceptionGroup[OSError]` --Alex
assert_public_ty(&db, "src/a.py", "e", "BaseExceptionGroup");
assert_public_ty(&db, "src/a.py", "e", "Unknown | BaseExceptionGroup");
Ok(())
}
@@ -4839,10 +5135,11 @@ mod tests {
assert_file_diagnostics(&db, "src/a.py", &[]);
// TODO: once we support `sys.version_info` branches,
// we should set `--target-version=py311` in this test
// we can set `--target-version=py311` in this test
// and the inferred type will just be `BaseExceptionGroup` --Alex
//
// TODO more precise would be `ExceptionGroup[TypeError | AttributeError]` --Alex
assert_public_ty(&db, "src/a.py", "e", "BaseExceptionGroup");
assert_public_ty(&db, "src/a.py", "e", "Unknown | BaseExceptionGroup");
Ok(())
}
@@ -4973,7 +5270,7 @@ mod tests {
assert_file_diagnostics(
&db,
"src/a.py",
&["Object of type 'Unbound' is not iterable"],
&["Object of type 'Unbound' is not iterable."],
);
Ok(())
@@ -5001,7 +5298,7 @@ mod tests {
assert_scope_ty(&db, "src/a.py", &["foo", "<listcomp>"], "x", "int");
assert_scope_ty(&db, "src/a.py", &["foo", "<listcomp>"], "z", "Unknown");
assert_file_diagnostics(&db, "src/a.py", &["Object of type 'int' is not iterable"]);
assert_file_diagnostics(&db, "src/a.py", &["Object of type 'int' is not iterable."]);
Ok(())
}
@@ -5195,7 +5492,7 @@ mod tests {
assert_file_diagnostics(
&db,
"/src/a.py",
&["Object of type 'Literal[123]' is not iterable"],
&["Object of type 'Literal[123]' is not iterable."],
);
}
@@ -5221,7 +5518,7 @@ mod tests {
assert_file_diagnostics(
&db,
"/src/a.py",
&["Object of type 'NotIterable' is not iterable"],
&["Object of type 'NotIterable' is not iterable."],
);
}
@@ -5250,7 +5547,7 @@ mod tests {
assert_file_diagnostics(
&db,
"/src/a.py",
&["Object of type 'NotIterable' is not iterable"],
&["Object of type 'NotIterable' is not iterable."],
);
}
@@ -5280,7 +5577,7 @@ mod tests {
assert_file_diagnostics(
&db,
"/src/a.py",
&["Object of type 'NotIterable' is not iterable"],
&["Object of type 'NotIterable' is not iterable."],
);
}
@@ -5394,7 +5691,75 @@ mod tests {
assert_file_diagnostics(
&db,
"/src/a.py",
&[r"Conflicting declared types for 'x': 'str', 'int'."],
&[r"Conflicting declared types for 'x': str, int."],
);
}
#[test]
fn partial_declarations() {
let mut db = setup_db();
db.write_dedented(
"/src/a.py",
"
if flag:
x: int
x = 1
",
)
.unwrap();
assert_file_diagnostics(
&db,
"/src/a.py",
&[r"Conflicting declared types for 'x': Unknown, int."],
);
}
#[test]
fn incompatible_declarations_bad_assignment() {
let mut db = setup_db();
db.write_dedented(
"/src/a.py",
"
if flag:
x: str
else:
x: int
x = b'foo'
",
)
.unwrap();
assert_file_diagnostics(
&db,
"/src/a.py",
&[
r"Conflicting declared types for 'x': str, int.",
r#"Object of type 'Literal[b"foo"]' is not assignable to 'str | int'."#,
],
);
}
#[test]
fn partial_declarations_questionable_assignment() {
let mut db = setup_db();
db.write_dedented(
"/src/a.py",
"
if flag:
x: int
x = 'foo'
",
)
.unwrap();
assert_file_diagnostics(
&db,
"/src/a.py",
&[r"Conflicting declared types for 'x': Unknown, int."],
);
}
@@ -5489,6 +5854,70 @@ mod tests {
assert_file_diagnostics(&db, "/src/a.py", &[]);
}
#[test]
fn no_implicit_shadow_import() {
let mut db = setup_db();
db.write_dedented(
"/src/a.py",
"
from b import x
x = 'foo'
",
)
.unwrap();
db.write_file("/src/b.py", "x: int").unwrap();
assert_file_diagnostics(
&db,
"/src/a.py",
&[r#"Object of type 'Literal["foo"]' is not assignable to 'int'."#],
);
}
#[test]
fn import_from_conditional_reimport() {
let mut db = setup_db();
db.write_file("/src/a.py", "from b import f").unwrap();
db.write_dedented(
"/src/b.py",
"
if flag:
from c import f
else:
def f(): ...
",
)
.unwrap();
db.write_file("/src/c.py", "def f(): ...").unwrap();
// TODO we should really disambiguate in such cases: Literal[b.f, c.f]
assert_public_ty(&db, "/src/a.py", "f", "Literal[f, f]");
}
#[test]
fn import_from_conditional_reimport_vs_non_declaration() {
let mut db = setup_db();
db.write_file("/src/a.py", "from b import x").unwrap();
db.write_dedented(
"/src/b.py",
"
if flag:
from c import x
else:
x = 1
",
)
.unwrap();
db.write_file("/src/c.pyi", "x: int").unwrap();
assert_public_ty(&db, "/src/a.py", "x", "int");
}
// Incremental inference tests
fn first_public_binding<'db>(db: &'db TestDb, file: File, name: &str) -> Definition<'db> {

View File

@@ -1 +1 @@
23d867efb2df6de5600f64656f1aa8a83e06109e
9e506eb5e8fc2823db8c60ad561b1145ff114947

View File

@@ -41,7 +41,7 @@ _json: 3.0-
_locale: 3.0-
_lsprof: 3.0-
_markupbase: 3.0-
_msi: 3.0-
_msi: 3.0-3.12
_operator: 3.4-
_osx_support: 3.0-
_posixsubprocess: 3.2-

View File

@@ -493,7 +493,7 @@ class _CursesWindow:
def instr(self, y: int, x: int, n: int = ...) -> bytes: ...
def is_linetouched(self, line: int, /) -> bool: ...
def is_wintouched(self) -> bool: ...
def keypad(self, yes: bool) -> None: ...
def keypad(self, yes: bool, /) -> None: ...
def leaveok(self, yes: bool) -> None: ...
def move(self, new_y: int, new_x: int) -> None: ...
def mvderwin(self, y: int, x: int) -> None: ...

View File

@@ -1,17 +1,38 @@
import sys
from _typeshed import StrPath
from collections.abc import Mapping
from typing import Final, Literal, TypedDict, type_check_only
LC_CTYPE: int
LC_COLLATE: int
LC_TIME: int
LC_MONETARY: int
LC_NUMERIC: int
LC_ALL: int
CHAR_MAX: int
@type_check_only
class _LocaleConv(TypedDict):
decimal_point: str
grouping: list[int]
thousands_sep: str
int_curr_symbol: str
currency_symbol: str
p_cs_precedes: Literal[0, 1, 127]
n_cs_precedes: Literal[0, 1, 127]
p_sep_by_space: Literal[0, 1, 127]
n_sep_by_space: Literal[0, 1, 127]
mon_decimal_point: str
frac_digits: int
int_frac_digits: int
mon_thousands_sep: str
mon_grouping: list[int]
positive_sign: str
negative_sign: str
p_sign_posn: Literal[0, 1, 2, 3, 4, 127]
n_sign_posn: Literal[0, 1, 2, 3, 4, 127]
LC_CTYPE: Final[int]
LC_COLLATE: Final[int]
LC_TIME: Final[int]
LC_MONETARY: Final[int]
LC_NUMERIC: Final[int]
LC_ALL: Final[int]
CHAR_MAX: Final = 127
def setlocale(category: int, locale: str | None = None, /) -> str: ...
def localeconv() -> Mapping[str, int | str | list[int]]: ...
def localeconv() -> _LocaleConv: ...
if sys.version_info >= (3, 11):
def getencoding() -> str: ...
@@ -25,67 +46,67 @@ def strxfrm(string: str, /) -> str: ...
if sys.platform != "win32":
LC_MESSAGES: int
ABDAY_1: int
ABDAY_2: int
ABDAY_3: int
ABDAY_4: int
ABDAY_5: int
ABDAY_6: int
ABDAY_7: int
ABDAY_1: Final[int]
ABDAY_2: Final[int]
ABDAY_3: Final[int]
ABDAY_4: Final[int]
ABDAY_5: Final[int]
ABDAY_6: Final[int]
ABDAY_7: Final[int]
ABMON_1: int
ABMON_2: int
ABMON_3: int
ABMON_4: int
ABMON_5: int
ABMON_6: int
ABMON_7: int
ABMON_8: int
ABMON_9: int
ABMON_10: int
ABMON_11: int
ABMON_12: int
ABMON_1: Final[int]
ABMON_2: Final[int]
ABMON_3: Final[int]
ABMON_4: Final[int]
ABMON_5: Final[int]
ABMON_6: Final[int]
ABMON_7: Final[int]
ABMON_8: Final[int]
ABMON_9: Final[int]
ABMON_10: Final[int]
ABMON_11: Final[int]
ABMON_12: Final[int]
DAY_1: int
DAY_2: int
DAY_3: int
DAY_4: int
DAY_5: int
DAY_6: int
DAY_7: int
DAY_1: Final[int]
DAY_2: Final[int]
DAY_3: Final[int]
DAY_4: Final[int]
DAY_5: Final[int]
DAY_6: Final[int]
DAY_7: Final[int]
ERA: int
ERA_D_T_FMT: int
ERA_D_FMT: int
ERA_T_FMT: int
ERA: Final[int]
ERA_D_T_FMT: Final[int]
ERA_D_FMT: Final[int]
ERA_T_FMT: Final[int]
MON_1: int
MON_2: int
MON_3: int
MON_4: int
MON_5: int
MON_6: int
MON_7: int
MON_8: int
MON_9: int
MON_10: int
MON_11: int
MON_12: int
MON_1: Final[int]
MON_2: Final[int]
MON_3: Final[int]
MON_4: Final[int]
MON_5: Final[int]
MON_6: Final[int]
MON_7: Final[int]
MON_8: Final[int]
MON_9: Final[int]
MON_10: Final[int]
MON_11: Final[int]
MON_12: Final[int]
CODESET: int
D_T_FMT: int
D_FMT: int
T_FMT: int
T_FMT_AMPM: int
AM_STR: int
PM_STR: int
CODESET: Final[int]
D_T_FMT: Final[int]
D_FMT: Final[int]
T_FMT: Final[int]
T_FMT_AMPM: Final[int]
AM_STR: Final[int]
PM_STR: Final[int]
RADIXCHAR: int
THOUSEP: int
YESEXPR: int
NOEXPR: int
CRNCYSTR: int
ALT_DIGITS: int
RADIXCHAR: Final[int]
THOUSEP: Final[int]
YESEXPR: Final[int]
NOEXPR: Final[int]
CRNCYSTR: Final[int]
ALT_DIGITS: Final[int]
def nl_langinfo(key: int, /) -> str: ...

View File

@@ -99,6 +99,20 @@ if sys.platform == "win32":
SEC_RESERVE: Final = 0x4000000
SEC_WRITECOMBINE: Final = 0x40000000
if sys.version_info >= (3, 13):
STARTF_FORCEOFFFEEDBACK: Final = 0x80
STARTF_FORCEONFEEDBACK: Final = 0x40
STARTF_PREVENTPINNING: Final = 0x2000
STARTF_RUNFULLSCREEN: Final = 0x20
STARTF_TITLEISAPPID: Final = 0x1000
STARTF_TITLEISLINKNAME: Final = 0x800
STARTF_UNTRUSTEDSOURCE: Final = 0x8000
STARTF_USECOUNTCHARS: Final = 0x8
STARTF_USEFILLATTRIBUTE: Final = 0x10
STARTF_USEHOTKEY: Final = 0x200
STARTF_USEPOSITION: Final = 0x4
STARTF_USESIZE: Final = 0x2
STARTF_USESHOWWINDOW: Final = 0x1
STARTF_USESTDHANDLES: Final = 0x100
@@ -250,6 +264,20 @@ if sys.platform == "win32":
def cancel(self) -> None: ...
def getbuffer(self) -> bytes | None: ...
if sys.version_info >= (3, 13):
def BatchedWaitForMultipleObjects(
handle_seq: Sequence[int], wait_all: bool, milliseconds: int = 0xFFFFFFFF
) -> list[int]: ...
def CreateEventW(security_attributes: int, manual_reset: bool, initial_state: bool, name: str | None) -> int: ...
def CreateMutexW(security_attributes: int, initial_owner: bool, name: str) -> int: ...
def GetLongPathName(path: str) -> str: ...
def GetShortPathName(path: str) -> str: ...
def OpenEventW(desired_access: int, inherit_handle: bool, name: str) -> int: ...
def OpenMutexW(desired_access: int, inherit_handle: bool, name: str) -> int: ...
def ReleaseMutex(mutex: int) -> None: ...
def ResetEvent(event: int) -> None: ...
def SetEvent(event: int) -> None: ...
if sys.version_info >= (3, 12):
def CopyFile2(existing_file_name: str, new_file_name: str, flags: int, progress_routine: int | None = None) -> int: ...
def NeedCurrentDirectoryForExePath(exe_name: str, /) -> bool: ...

View File

@@ -1,3 +1,4 @@
# ruff: noqa: PYI036 # This is the module declaring BaseException
import _ast
import _typeshed
import sys

View File

@@ -80,7 +80,7 @@ class _Encoder(Protocol):
def __call__(self, input: str, errors: str = ..., /) -> tuple[bytes, int]: ... # signature of Codec().encode
class _Decoder(Protocol):
def __call__(self, input: bytes, errors: str = ..., /) -> tuple[str, int]: ... # signature of Codec().decode
def __call__(self, input: ReadableBuffer, errors: str = ..., /) -> tuple[str, int]: ... # signature of Codec().decode
class _StreamReader(Protocol):
def __call__(self, stream: _ReadableStream, errors: str = ..., /) -> StreamReader: ...

View File

@@ -1,16 +1,15 @@
import sys
from typing import Any, Protocol, TypeVar
from typing_extensions import ParamSpec, Self
from typing_extensions import Self
__all__ = ["Error", "copy", "deepcopy"]
_T = TypeVar("_T")
_SR = TypeVar("_SR", bound=_SupportsReplace[Any])
_P = ParamSpec("_P")
_SR = TypeVar("_SR", bound=_SupportsReplace)
class _SupportsReplace(Protocol[_P]):
class _SupportsReplace(Protocol):
# In reality doesn't support args, but there's no other great way to express this.
def __replace__(self, *args: _P.args, **kwargs: _P.kwargs) -> Self: ...
def __replace__(self, *args: Any, **kwargs: Any) -> Self: ...
# None in CPython but non-None in Jython
PyStringMap: Any

View File

@@ -270,7 +270,7 @@ class Distribution:
def has_data_files(self) -> bool: ...
def is_pure(self) -> bool: ...
# Getter methods generated in __init__
# Default getter methods generated in __init__ from self.metadata._METHOD_BASENAMES
def get_name(self) -> str: ...
def get_version(self) -> str: ...
def get_fullname(self) -> str: ...
@@ -292,3 +292,26 @@ class Distribution:
def get_requires(self) -> list[str]: ...
def get_provides(self) -> list[str]: ...
def get_obsoletes(self) -> list[str]: ...
# Default attributes generated in __init__ from self.display_option_names
help_commands: bool | Literal[0]
name: str | Literal[0]
version: str | Literal[0]
fullname: str | Literal[0]
author: str | Literal[0]
author_email: str | Literal[0]
maintainer: str | Literal[0]
maintainer_email: str | Literal[0]
contact: str | Literal[0]
contact_email: str | Literal[0]
url: str | Literal[0]
license: str | Literal[0]
licence: str | Literal[0]
description: str | Literal[0]
long_description: str | Literal[0]
platforms: str | list[str] | Literal[0]
classifiers: str | list[str] | Literal[0]
keywords: str | list[str] | Literal[0]
provides: list[str] | Literal[0]
requires: list[str] | Literal[0]
obsoletes: list[str] | Literal[0]

View File

@@ -1,9 +1,10 @@
import sys
import types
import unittest
from _typeshed import ExcInfo
from collections.abc import Callable
from typing import Any, NamedTuple
from typing_extensions import TypeAlias
from typing import Any, ClassVar, NamedTuple
from typing_extensions import Self, TypeAlias
__all__ = [
"register_optionflag",
@@ -41,9 +42,22 @@ __all__ = [
"debug",
]
class TestResults(NamedTuple):
failed: int
attempted: int
# MyPy errors on conditionals within named tuples.
if sys.version_info >= (3, 13):
class TestResults(NamedTuple):
def __new__(cls, failed: int, attempted: int, *, skipped: int = 0) -> Self: ... # type: ignore[misc]
skipped: int
failed: int
attempted: int
_fields: ClassVar = ("failed", "attempted") # type: ignore[misc]
__match_args__ = ("failed", "attempted") # type: ignore[misc]
__doc__: None # type: ignore[misc]
else:
class TestResults(NamedTuple):
failed: int
attempted: int
OPTIONFLAGS_BY_NAME: dict[str, int]
@@ -134,6 +148,8 @@ class DocTestRunner:
original_optionflags: int
tries: int
failures: int
if sys.version_info >= (3, 13):
skips: int
test: DocTest
def __init__(self, checker: OutputChecker | None = None, verbose: bool | None = None, optionflags: int = 0) -> None: ...
def report_start(self, out: _Out, test: DocTest, example: Example) -> None: ...

View File

@@ -16,6 +16,10 @@ TOKEN_ENDS: Final[set[str]]
ASPECIALS: Final[set[str]]
ATTRIBUTE_ENDS: Final[set[str]]
EXTENDED_ATTRIBUTE_ENDS: Final[set[str]]
# Added in Python 3.8.20, 3.9.20, 3.10.15, 3.11.10, 3.12.5
NLSET: Final[set[str]]
# Added in Python 3.8.20, 3.9.20, 3.10.15, 3.11.10, 3.12.5
SPECIALSNL: Final[set[str]]
def quote_string(value: Any) -> str: ...

View File

@@ -3,12 +3,34 @@ from collections.abc import Callable
from email.errors import MessageDefect
from email.header import Header
from email.message import Message
from typing import Any
from typing_extensions import Self
class _PolicyBase:
def __add__(self, other: Any) -> Self: ...
def clone(self, **kw: Any) -> Self: ...
def __init__(
self,
*,
max_line_length: int | None = 78,
linesep: str = "\n",
cte_type: str = "8bit",
raise_on_defect: bool = False,
mangle_from_: bool = ..., # default depends on sub-class
message_factory: Callable[[Policy], Message] | None = None,
# Added in Python 3.8.20, 3.9.20, 3.10.15, 3.11.10, 3.12.5
verify_generated_headers: bool = True,
) -> None: ...
def clone(
self,
*,
max_line_length: int | None = ...,
linesep: str = ...,
cte_type: str = ...,
raise_on_defect: bool = ...,
mangle_from_: bool = ...,
message_factory: Callable[[Policy], Message] | None = ...,
# Added in Python 3.8.20, 3.9.20, 3.10.15, 3.11.10, 3.12.5
verify_generated_headers: bool = ...,
) -> Self: ...
def __add__(self, other: Policy) -> Self: ...
class Policy(_PolicyBase, metaclass=ABCMeta):
max_line_length: int | None
@@ -17,16 +39,9 @@ class Policy(_PolicyBase, metaclass=ABCMeta):
raise_on_defect: bool
mangle_from_: bool
message_factory: Callable[[Policy], Message] | None
def __init__(
self,
*,
max_line_length: int | None = 78,
linesep: str = "\n",
cte_type: str = "8bit",
raise_on_defect: bool = False,
mangle_from_: bool = False,
message_factory: Callable[[Policy], Message] | None = None,
) -> None: ...
# Added in Python 3.8.20, 3.9.20, 3.10.15, 3.11.10, 3.12.5
verify_generated_headers: bool
def handle_defect(self, obj: Message, defect: MessageDefect) -> None: ...
def register_defect(self, obj: Message, defect: MessageDefect) -> None: ...
def header_max_count(self, name: str) -> int | None: ...

View File

@@ -7,6 +7,9 @@ class BoundaryError(MessageParseError): ...
class MultipartConversionError(MessageError, TypeError): ...
class CharsetError(MessageError): ...
# Added in Python 3.8.20, 3.9.20, 3.10.15, 3.11.10, 3.12.5
class HeaderWriteError(MessageError): ...
class MessageDefect(ValueError):
def __init__(self, line: str | None = None) -> None: ...

View File

@@ -30,20 +30,12 @@ _PDTZ: TypeAlias = tuple[int, int, int, int, int, int, int, int, int, int | None
def quote(str: str) -> str: ...
def unquote(str: str) -> str: ...
if sys.version_info >= (3, 13):
def parseaddr(addr: str | list[str], *, strict: bool = True) -> tuple[str, str]: ...
else:
def parseaddr(addr: str) -> tuple[str, str]: ...
# `strict` parameter added in Python 3.8.20, 3.9.20, 3.10.15, 3.11.10, 3.12.5
def parseaddr(addr: str | list[str], *, strict: bool = True) -> tuple[str, str]: ...
def formataddr(pair: tuple[str | None, str], charset: str | Charset = "utf-8") -> str: ...
if sys.version_info >= (3, 13):
def getaddresses(fieldvalues: Iterable[str], *, strict: bool = True) -> list[tuple[str, str]]: ...
else:
def getaddresses(fieldvalues: Iterable[str]) -> list[tuple[str, str]]: ...
# `strict` parameter added in Python 3.8.20, 3.9.20, 3.10.15, 3.11.10, 3.12.5
def getaddresses(fieldvalues: Iterable[str], *, strict: bool = True) -> list[tuple[str, str]]: ...
@overload
def parsedate(data: None) -> None: ...
@overload

View File

@@ -84,7 +84,6 @@ class RawIOBase(IOBase):
def read(self, size: int = -1, /) -> bytes | None: ...
class BufferedIOBase(IOBase):
raw: RawIOBase # This is not part of the BufferedIOBase API and may not exist on some implementations.
def detach(self) -> RawIOBase: ...
def readinto(self, buffer: WriteableBuffer, /) -> int: ...
def write(self, buffer: ReadableBuffer, /) -> int: ...
@@ -119,11 +118,13 @@ class BytesIO(BufferedIOBase, BinaryIO): # type: ignore[misc] # incompatible d
def read1(self, size: int | None = -1, /) -> bytes: ...
class BufferedReader(BufferedIOBase, BinaryIO): # type: ignore[misc] # incompatible definitions of methods in the base classes
raw: RawIOBase
def __enter__(self) -> Self: ...
def __init__(self, raw: RawIOBase, buffer_size: int = ...) -> None: ...
def peek(self, size: int = 0, /) -> bytes: ...
class BufferedWriter(BufferedIOBase, BinaryIO): # type: ignore[misc] # incompatible definitions of writelines in the base classes
raw: RawIOBase
def __enter__(self) -> Self: ...
def __init__(self, raw: RawIOBase, buffer_size: int = ...) -> None: ...
def write(self, buffer: ReadableBuffer, /) -> int: ...

View File

@@ -2582,6 +2582,11 @@ else:
def list2cmdline(seq: Iterable[StrOrBytesPath]) -> str: ... # undocumented
if sys.platform == "win32":
if sys.version_info >= (3, 13):
from _winapi import STARTF_FORCEOFFFEEDBACK, STARTF_FORCEONFEEDBACK
__all__ += ["STARTF_FORCEOFFFEEDBACK", "STARTF_FORCEONFEEDBACK"]
class STARTUPINFO:
def __init__(
self,

View File

@@ -253,11 +253,11 @@ class _TemporaryFileWrapper(IO[AnyStr]):
def truncate(self, size: int | None = ...) -> int: ...
def writable(self) -> bool: ...
@overload
def write(self: _TemporaryFileWrapper[str], s: str) -> int: ...
def write(self: _TemporaryFileWrapper[str], s: str, /) -> int: ...
@overload
def write(self: _TemporaryFileWrapper[bytes], s: ReadableBuffer) -> int: ...
def write(self: _TemporaryFileWrapper[bytes], s: ReadableBuffer, /) -> int: ...
@overload
def write(self, s: AnyStr) -> int: ...
def write(self, s: AnyStr, /) -> int: ...
@overload
def writelines(self: _TemporaryFileWrapper[str], lines: Iterable[str]) -> None: ...
@overload

View File

@@ -0,0 +1,2 @@
with foo() as self.bar:
pass

View File

@@ -114,22 +114,19 @@ fn lint_maybe_undefined(context: &SemanticLintContext, name: &ast::ExprName) {
return;
}
let semantic = &context.semantic;
match name.ty(semantic) {
Type::Unbound => {
context.push_diagnostic(format_diagnostic(
context,
&format!("Name '{}' used when not defined.", &name.id),
name.start(),
));
}
Type::Union(union) if union.contains(semantic.db(), Type::Unbound) => {
context.push_diagnostic(format_diagnostic(
context,
&format!("Name '{}' used when possibly not defined.", &name.id),
name.start(),
));
}
_ => {}
let ty = name.ty(semantic);
if ty.is_unbound() {
context.push_diagnostic(format_diagnostic(
context,
&format!("Name '{}' used when not defined.", &name.id),
name.start(),
));
} else if ty.may_be_unbound(semantic.db()) {
context.push_diagnostic(format_diagnostic(
context,
&format!("Name '{}' used when possibly not defined.", &name.id),
name.start(),
));
}
}

View File

@@ -1,6 +1,6 @@
[package]
name = "ruff"
version = "0.6.4"
version = "0.6.6"
publish = true
authors = { workspace = true }
edition = { workspace = true }
@@ -14,7 +14,9 @@ default-run = "ruff"
[dependencies]
ruff_cache = { workspace = true }
ruff_db = { workspace = true }
ruff_diagnostics = { workspace = true }
ruff_graph = { workspace = true, features = ["serde", "clap"] }
ruff_linter = { workspace = true, features = ["clap"] }
ruff_macros = { workspace = true }
ruff_notebook = { workspace = true }
@@ -36,6 +38,7 @@ clap_complete_command = { workspace = true }
clearscreen = { workspace = true }
colored = { workspace = true }
filetime = { workspace = true }
globwalk = { workspace = true }
ignore = { workspace = true }
is-macro = { workspace = true }
itertools = { workspace = true }
@@ -59,8 +62,11 @@ wild = { workspace = true }
[dev-dependencies]
# Enable test rules during development
ruff_linter = { workspace = true, features = ["clap", "test-rules"] }
assert_fs = { workspace = true }
# Avoid writing colored snapshots when running tests from the terminal
colored = { workspace = true, features = ["no-color"] }
indoc = { workspace = true }
insta = { workspace = true, features = ["filters", "json"] }
insta-cmd = { workspace = true }
tempfile = { workspace = true }

View File

@@ -7,13 +7,11 @@ use std::sync::Arc;
use anyhow::{anyhow, bail};
use clap::builder::{TypedValueParser, ValueParserFactory};
use clap::{command, Parser};
use clap::{command, Parser, Subcommand};
use colored::Colorize;
use path_absolutize::path_dedot;
use regex::Regex;
use rustc_hash::FxHashMap;
use toml;
use ruff_graph::Direction;
use ruff_linter::line_width::LineLength;
use ruff_linter::logging::LogLevel;
use ruff_linter::registry::Rule;
@@ -27,6 +25,8 @@ use ruff_text_size::TextRange;
use ruff_workspace::configuration::{Configuration, RuleSelection};
use ruff_workspace::options::{Options, PycodestyleOptions};
use ruff_workspace::resolver::ConfigurationTransformer;
use rustc_hash::FxHashMap;
use toml;
/// All configuration options that can be passed "globally",
/// i.e., can be passed to all subcommands
@@ -132,6 +132,9 @@ pub enum Command {
Format(FormatCommand),
/// Run the language server.
Server(ServerCommand),
/// Run analysis over Python source code.
#[clap(subcommand)]
Analyze(AnalyzeCommand),
/// Display Ruff's version
Version {
#[arg(long, value_enum, default_value = "text")]
@@ -139,6 +142,35 @@ pub enum Command {
},
}
#[derive(Debug, Subcommand)]
pub enum AnalyzeCommand {
/// Generate a map of Python file dependencies or dependents.
Graph(AnalyzeGraphCommand),
}
#[derive(Clone, Debug, clap::Parser)]
pub struct AnalyzeGraphCommand {
/// List of files or directories to include.
#[clap(help = "List of files or directories to include [default: .]")]
files: Vec<PathBuf>,
/// The direction of the import map. By default, generates a dependency map, i.e., a map from
/// file to files that it depends on. Use `--direction dependents` to generate a map from file
/// to files that depend on it.
#[clap(long, value_enum, default_value_t)]
direction: Direction,
/// Attempt to detect imports from string literals.
#[clap(long)]
detect_string_imports: bool,
/// Enable preview mode. Use `--no-preview` to disable.
#[arg(long, overrides_with("no_preview"))]
preview: bool,
#[clap(long, overrides_with("preview"), hide = true)]
no_preview: bool,
/// The minimum Python version that should be supported.
#[arg(long, value_enum)]
target_version: Option<PythonVersion>,
}
// The `Parser` derive is for ruff_dev, for ruff `Args` would be sufficient
#[derive(Clone, Debug, clap::Parser)]
#[allow(clippy::struct_excessive_bools)]
@@ -700,6 +732,7 @@ impl CheckCommand {
output_format: resolve_output_format(self.output_format)?,
show_fixes: resolve_bool_arg(self.show_fixes, self.no_show_fixes),
extension: self.extension,
..ExplicitConfigOverrides::default()
};
let config_args = ConfigArguments::from_cli_arguments(global_options, cli_overrides)?;
@@ -732,8 +765,34 @@ impl FormatCommand {
target_version: self.target_version,
cache_dir: self.cache_dir,
extension: self.extension,
..ExplicitConfigOverrides::default()
};
// Unsupported on the formatter CLI, but required on `Overrides`.
let config_args = ConfigArguments::from_cli_arguments(global_options, cli_overrides)?;
Ok((format_arguments, config_args))
}
}
impl AnalyzeGraphCommand {
/// Partition the CLI into command-line arguments and configuration
/// overrides.
pub fn partition(
self,
global_options: GlobalConfigArgs,
) -> anyhow::Result<(AnalyzeGraphArgs, ConfigArguments)> {
let format_arguments = AnalyzeGraphArgs {
files: self.files,
direction: self.direction,
};
let cli_overrides = ExplicitConfigOverrides {
detect_string_imports: if self.detect_string_imports {
Some(true)
} else {
None
},
preview: resolve_bool_arg(self.preview, self.no_preview).map(PreviewMode::from),
target_version: self.target_version,
..ExplicitConfigOverrides::default()
};
@@ -896,7 +955,7 @@ A `--config` flag must either be a path to a `.toml` configuration file
// the user was trying to pass in a path to a configuration file
// or some inline TOML.
// We want to display the most helpful error to the user as possible.
if std::path::Path::new(value)
if Path::new(value)
.extension()
.map_or(false, |ext| ext.eq_ignore_ascii_case("toml"))
{
@@ -1156,6 +1215,13 @@ impl LineColumnParseError {
}
}
/// CLI settings that are distinct from configuration (commands, lists of files, etc.).
#[derive(Clone, Debug)]
pub struct AnalyzeGraphArgs {
pub files: Vec<PathBuf>,
pub direction: Direction,
}
/// Configuration overrides provided via dedicated CLI flags:
/// `--line-length`, `--respect-gitignore`, etc.
#[derive(Clone, Default)]
@@ -1187,6 +1253,7 @@ struct ExplicitConfigOverrides {
output_format: Option<OutputFormat>,
show_fixes: Option<bool>,
extension: Option<Vec<ExtensionPair>>,
detect_string_imports: Option<bool>,
}
impl ConfigurationTransformer for ExplicitConfigOverrides {
@@ -1271,6 +1338,9 @@ impl ConfigurationTransformer for ExplicitConfigOverrides {
if let Some(extension) = &self.extension {
config.extension = Some(extension.iter().cloned().collect());
}
if let Some(detect_string_imports) = &self.detect_string_imports {
config.analyze.detect_string_imports = Some(*detect_string_imports);
}
config
}

View File

@@ -10,7 +10,9 @@ use ruff_linter::linter::add_noqa_to_path;
use ruff_linter::source_kind::SourceKind;
use ruff_linter::warn_user_once;
use ruff_python_ast::{PySourceType, SourceType};
use ruff_workspace::resolver::{python_files_in_path, PyprojectConfig, ResolvedFile};
use ruff_workspace::resolver::{
match_exclusion, python_files_in_path, PyprojectConfig, ResolvedFile,
};
use crate::args::ConfigArguments;
@@ -57,6 +59,15 @@ pub(crate) fn add_noqa(
.and_then(|parent| package_roots.get(parent))
.and_then(|package| *package);
let settings = resolver.resolve(path);
if (settings.file_resolver.force_exclude || !resolved_file.is_root())
&& match_exclusion(
resolved_file.path(),
resolved_file.file_name(),
&settings.linter.exclude,
)
{
return None;
}
let source_kind = match SourceKind::from_path(path, source_type) {
Ok(Some(source_kind)) => source_kind,
Ok(None) => return None,

View File

@@ -0,0 +1,241 @@
use crate::args::{AnalyzeGraphArgs, ConfigArguments};
use crate::resolve::resolve;
use crate::{resolve_default_files, ExitStatus};
use anyhow::Result;
use log::{debug, warn};
use path_absolutize::CWD;
use ruff_db::system::{SystemPath, SystemPathBuf};
use ruff_graph::{Direction, ImportMap, ModuleDb, ModuleImports};
use ruff_linter::{warn_user, warn_user_once};
use ruff_python_ast::{PySourceType, SourceType};
use ruff_workspace::resolver::{match_exclusion, python_files_in_path, ResolvedFile};
use rustc_hash::FxHashMap;
use std::path::{Path, PathBuf};
use std::sync::{Arc, Mutex};
/// Generate an import map.
pub(crate) fn analyze_graph(
args: AnalyzeGraphArgs,
config_arguments: &ConfigArguments,
) -> Result<ExitStatus> {
// Construct the "default" settings. These are used when no `pyproject.toml`
// files are present, or files are injected from outside the hierarchy.
let pyproject_config = resolve(config_arguments, None)?;
if pyproject_config.settings.analyze.preview.is_disabled() {
warn_user!("`ruff analyze graph` is experimental and may change without warning");
}
// Write all paths relative to the current working directory.
let root =
SystemPathBuf::from_path_buf(CWD.clone()).expect("Expected a UTF-8 working directory");
// Find all Python files.
let files = resolve_default_files(args.files, false);
let (paths, resolver) = python_files_in_path(&files, &pyproject_config, config_arguments)?;
if paths.is_empty() {
warn_user_once!("No Python files found under the given path(s)");
return Ok(ExitStatus::Success);
}
// Resolve all package roots.
let package_roots = resolver
.package_roots(
&paths
.iter()
.flatten()
.map(ResolvedFile::path)
.collect::<Vec<_>>(),
)
.into_iter()
.map(|(path, package)| (path.to_path_buf(), package.map(Path::to_path_buf)))
.collect::<FxHashMap<_, _>>();
// Create a database from the source roots.
let db = ModuleDb::from_src_roots(
package_roots
.values()
.filter_map(|package| package.as_deref())
.filter_map(|package| package.parent())
.map(Path::to_path_buf)
.filter_map(|path| SystemPathBuf::from_path_buf(path).ok()),
pyproject_config
.settings
.analyze
.target_version
.as_tuple()
.into(),
)?;
// Create a cache for resolved globs.
let glob_resolver = Arc::new(Mutex::new(GlobResolver::default()));
// Collect and resolve the imports for each file.
let result = Arc::new(Mutex::new(Vec::new()));
let inner_result = Arc::clone(&result);
rayon::scope(move |scope| {
for resolved_file in paths {
let Ok(resolved_file) = resolved_file else {
continue;
};
let path = resolved_file.path();
let package = path
.parent()
.and_then(|parent| package_roots.get(parent))
.and_then(Clone::clone);
// Resolve the per-file settings.
let settings = resolver.resolve(path);
let string_imports = settings.analyze.detect_string_imports;
let include_dependencies = settings.analyze.include_dependencies.get(path).cloned();
// Skip excluded files.
if (settings.file_resolver.force_exclude || !resolved_file.is_root())
&& match_exclusion(
resolved_file.path(),
resolved_file.file_name(),
&settings.analyze.exclude,
)
{
continue;
}
// Ignore non-Python files.
let source_type = match settings.analyze.extension.get(path) {
None => match SourceType::from(&path) {
SourceType::Python(source_type) => source_type,
SourceType::Toml(_) => {
debug!("Ignoring TOML file: {}", path.display());
continue;
}
},
Some(language) => PySourceType::from(language),
};
if matches!(source_type, PySourceType::Ipynb) {
debug!("Ignoring Jupyter notebook: {}", path.display());
continue;
}
// Convert to system paths.
let Ok(package) = package.map(SystemPathBuf::from_path_buf).transpose() else {
warn!("Failed to convert package to system path");
continue;
};
let Ok(path) = SystemPathBuf::from_path_buf(resolved_file.into_path()) else {
warn!("Failed to convert path to system path");
continue;
};
let db = db.snapshot();
let glob_resolver = glob_resolver.clone();
let root = root.clone();
let result = inner_result.clone();
scope.spawn(move |_| {
// Identify any imports via static analysis.
let mut imports =
ModuleImports::detect(&db, &path, package.as_deref(), string_imports)
.unwrap_or_else(|err| {
warn!("Failed to generate import map for {path}: {err}");
ModuleImports::default()
});
debug!("Discovered {} imports for {}", imports.len(), path);
// Append any imports that were statically defined in the configuration.
if let Some((root, globs)) = include_dependencies {
let mut glob_resolver = glob_resolver.lock().unwrap();
imports.extend(glob_resolver.resolve(root, globs));
}
// Convert the path (and imports) to be relative to the working directory.
let path = path
.strip_prefix(&root)
.map(SystemPath::to_path_buf)
.unwrap_or(path);
let imports = imports.relative_to(&root);
result.lock().unwrap().push((path, imports));
});
}
});
// Collect the results.
let imports = Arc::into_inner(result).unwrap().into_inner()?;
// Generate the import map.
let import_map = match args.direction {
Direction::Dependencies => ImportMap::from_iter(imports),
Direction::Dependents => ImportMap::reverse(imports),
};
// Print to JSON.
println!("{}", serde_json::to_string_pretty(&import_map)?);
Ok(ExitStatus::Success)
}
/// A resolver for glob sets.
#[derive(Default, Debug)]
struct GlobResolver {
cache: GlobCache,
}
impl GlobResolver {
/// Resolve a set of globs, anchored at a given root.
fn resolve(&mut self, root: PathBuf, globs: Vec<String>) -> Vec<SystemPathBuf> {
if let Some(cached) = self.cache.get(&root, &globs) {
return cached.clone();
}
let walker = match globwalk::GlobWalkerBuilder::from_patterns(&root, &globs)
.file_type(globwalk::FileType::FILE)
.build()
{
Ok(walker) => walker,
Err(err) => {
warn!("Failed to read glob walker: {err}");
return Vec::new();
}
};
let mut paths = Vec::new();
for entry in walker {
let entry = match entry {
Ok(entry) => entry,
Err(err) => {
warn!("Failed to read glob entry: {err}");
continue;
}
};
let path = match SystemPathBuf::from_path_buf(entry.into_path()) {
Ok(path) => path,
Err(err) => {
warn!("Failed to convert path to system path: {}", err.display());
continue;
}
};
paths.push(path);
}
self.cache.insert(root, globs, paths.clone());
paths
}
}
/// A cache for resolved globs.
#[derive(Default, Debug)]
struct GlobCache(FxHashMap<PathBuf, FxHashMap<Vec<String>, Vec<SystemPathBuf>>>);
impl GlobCache {
/// Insert a resolved glob.
fn insert(&mut self, root: PathBuf, globs: Vec<String>, paths: Vec<SystemPathBuf>) {
self.0.entry(root).or_default().insert(globs, paths);
}
/// Get a resolved glob.
fn get(&self, root: &Path, globs: &[String]) -> Option<&Vec<SystemPathBuf>> {
self.0.get(root).and_then(|map| map.get(globs))
}
}

View File

@@ -1,4 +1,5 @@
pub(crate) mod add_noqa;
pub(crate) mod analyze_graph;
pub(crate) mod check;
pub(crate) mod check_stdin;
pub(crate) mod clean;

View File

@@ -20,7 +20,9 @@ use ruff_linter::settings::types::OutputFormat;
use ruff_linter::{fs, warn_user, warn_user_once};
use ruff_workspace::Settings;
use crate::args::{Args, CheckCommand, Command, FormatCommand};
use crate::args::{
AnalyzeCommand, AnalyzeGraphCommand, Args, CheckCommand, Command, FormatCommand,
};
use crate::printer::{Flags as PrinterFlags, Printer};
pub mod args;
@@ -186,6 +188,7 @@ pub fn run(
Command::Check(args) => check(args, global_options),
Command::Format(args) => format(args, global_options),
Command::Server(args) => server(args),
Command::Analyze(AnalyzeCommand::Graph(args)) => analyze_graph(args, global_options),
}
}
@@ -199,6 +202,15 @@ fn format(args: FormatCommand, global_options: GlobalConfigArgs) -> Result<ExitS
}
}
fn analyze_graph(
args: AnalyzeGraphCommand,
global_options: GlobalConfigArgs,
) -> Result<ExitStatus> {
let (cli, config_arguments) = args.partition(global_options)?;
commands::analyze_graph::analyze_graph(cli, &config_arguments)
}
fn server(args: ServerCommand) -> Result<ExitStatus> {
let four = NonZeroUsize::new(4).unwrap();

View File

@@ -0,0 +1,307 @@
//! Tests the interaction of the `analyze graph` command.
#![cfg(not(target_arch = "wasm32"))]
#![cfg(not(windows))]
use assert_fs::prelude::*;
use std::process::Command;
use std::str;
use anyhow::Result;
use assert_fs::fixture::ChildPath;
use insta_cmd::{assert_cmd_snapshot, get_cargo_bin};
use tempfile::TempDir;
fn command() -> Command {
let mut command = Command::new(get_cargo_bin("ruff"));
command.arg("analyze");
command.arg("graph");
command.arg("--preview");
command
}
const INSTA_FILTERS: &[(&str, &str)] = &[
// Rewrite Windows output to Unix output
(r"\\", "/"),
];
#[test]
fn dependencies() -> Result<()> {
let tempdir = TempDir::new()?;
let root = ChildPath::new(tempdir.path());
root.child("ruff").child("__init__.py").write_str("")?;
root.child("ruff")
.child("a.py")
.write_str(indoc::indoc! {r#"
import ruff.b
"#})?;
root.child("ruff")
.child("b.py")
.write_str(indoc::indoc! {r#"
from ruff import c
"#})?;
root.child("ruff")
.child("c.py")
.write_str(indoc::indoc! {r#"
from . import d
"#})?;
root.child("ruff")
.child("d.py")
.write_str(indoc::indoc! {r#"
from .e import f
"#})?;
root.child("ruff")
.child("e.py")
.write_str(indoc::indoc! {r#"
def f(): pass
"#})?;
insta::with_settings!({
filters => INSTA_FILTERS.to_vec(),
}, {
assert_cmd_snapshot!(command().current_dir(&root), @r###"
success: true
exit_code: 0
----- stdout -----
{
"ruff/__init__.py": [],
"ruff/a.py": [
"ruff/b.py"
],
"ruff/b.py": [
"ruff/c.py"
],
"ruff/c.py": [
"ruff/d.py"
],
"ruff/d.py": [
"ruff/e.py"
],
"ruff/e.py": []
}
----- stderr -----
"###);
});
Ok(())
}
#[test]
fn dependents() -> Result<()> {
let tempdir = TempDir::new()?;
let root = ChildPath::new(tempdir.path());
root.child("ruff").child("__init__.py").write_str("")?;
root.child("ruff")
.child("a.py")
.write_str(indoc::indoc! {r#"
import ruff.b
"#})?;
root.child("ruff")
.child("b.py")
.write_str(indoc::indoc! {r#"
from ruff import c
"#})?;
root.child("ruff")
.child("c.py")
.write_str(indoc::indoc! {r#"
from . import d
"#})?;
root.child("ruff")
.child("d.py")
.write_str(indoc::indoc! {r#"
from .e import f
"#})?;
root.child("ruff")
.child("e.py")
.write_str(indoc::indoc! {r#"
def f(): pass
"#})?;
insta::with_settings!({
filters => INSTA_FILTERS.to_vec(),
}, {
assert_cmd_snapshot!(command().arg("--direction").arg("dependents").current_dir(&root), @r###"
success: true
exit_code: 0
----- stdout -----
{
"ruff/__init__.py": [],
"ruff/a.py": [],
"ruff/b.py": [
"ruff/a.py"
],
"ruff/c.py": [
"ruff/b.py"
],
"ruff/d.py": [
"ruff/c.py"
],
"ruff/e.py": [
"ruff/d.py"
]
}
----- stderr -----
"###);
});
Ok(())
}
#[test]
fn string_detection() -> Result<()> {
let tempdir = TempDir::new()?;
let root = ChildPath::new(tempdir.path());
root.child("ruff").child("__init__.py").write_str("")?;
root.child("ruff")
.child("a.py")
.write_str(indoc::indoc! {r#"
import ruff.b
"#})?;
root.child("ruff")
.child("b.py")
.write_str(indoc::indoc! {r#"
import importlib
importlib.import_module("ruff.c")
"#})?;
root.child("ruff").child("c.py").write_str("")?;
insta::with_settings!({
filters => INSTA_FILTERS.to_vec(),
}, {
assert_cmd_snapshot!(command().current_dir(&root), @r###"
success: true
exit_code: 0
----- stdout -----
{
"ruff/__init__.py": [],
"ruff/a.py": [
"ruff/b.py"
],
"ruff/b.py": [],
"ruff/c.py": []
}
----- stderr -----
"###);
});
insta::with_settings!({
filters => INSTA_FILTERS.to_vec(),
}, {
assert_cmd_snapshot!(command().arg("--detect-string-imports").current_dir(&root), @r###"
success: true
exit_code: 0
----- stdout -----
{
"ruff/__init__.py": [],
"ruff/a.py": [
"ruff/b.py"
],
"ruff/b.py": [
"ruff/c.py"
],
"ruff/c.py": []
}
----- stderr -----
"###);
});
Ok(())
}
#[test]
fn globs() -> Result<()> {
let tempdir = TempDir::new()?;
let root = ChildPath::new(tempdir.path());
root.child("ruff.toml").write_str(indoc::indoc! {r#"
[analyze]
include-dependencies = { "ruff/a.py" = ["ruff/b.py"], "ruff/b.py" = ["ruff/*.py"], "ruff/c.py" = ["*.json"] }
"#})?;
root.child("ruff").child("__init__.py").write_str("")?;
root.child("ruff").child("a.py").write_str("")?;
root.child("ruff").child("b.py").write_str("")?;
root.child("ruff").child("c.py").write_str("")?;
root.child("ruff").child("d.json").write_str("")?;
insta::with_settings!({
filters => INSTA_FILTERS.to_vec(),
}, {
assert_cmd_snapshot!(command().current_dir(&root), @r###"
success: true
exit_code: 0
----- stdout -----
{
"ruff/__init__.py": [],
"ruff/a.py": [
"ruff/b.py"
],
"ruff/b.py": [
"ruff/__init__.py",
"ruff/a.py",
"ruff/b.py",
"ruff/c.py"
],
"ruff/c.py": [
"ruff/d.json"
]
}
----- stderr -----
"###);
});
Ok(())
}
#[test]
fn exclude() -> Result<()> {
let tempdir = TempDir::new()?;
let root = ChildPath::new(tempdir.path());
root.child("ruff.toml").write_str(indoc::indoc! {r#"
[analyze]
exclude = ["ruff/c.py"]
"#})?;
root.child("ruff").child("__init__.py").write_str("")?;
root.child("ruff")
.child("a.py")
.write_str(indoc::indoc! {r#"
import ruff.b
"#})?;
root.child("ruff").child("b.py").write_str("")?;
root.child("ruff").child("c.py").write_str("")?;
insta::with_settings!({
filters => INSTA_FILTERS.to_vec(),
}, {
assert_cmd_snapshot!(command().current_dir(&root), @r###"
success: true
exit_code: 0
----- stdout -----
{
"ruff/__init__.py": [],
"ruff/a.py": [
"ruff/b.py"
],
"ruff/b.py": []
}
----- stderr -----
"###);
});
Ok(())
}

View File

@@ -326,18 +326,18 @@ fn docstring_options() -> Result<()> {
let ruff_toml = tempdir.path().join("ruff.toml");
fs::write(
&ruff_toml,
r#"
r"
[format]
docstring-code-format = true
docstring-code-line-length = 20
"#,
",
)?;
assert_cmd_snapshot!(Command::new(get_cargo_bin(BIN_NAME))
.args(["format", "--config"])
.arg(&ruff_toml)
.arg("-")
.pass_stdin(r#"
.pass_stdin(r"
def f(x):
'''
Something about `f`. And an example:
@@ -357,7 +357,7 @@ def f(x):
>>> foo, bar, quux = this_is_a_long_line(lion, hippo, lemur, bear)
'''
pass
"#), @r###"
"), @r###"
success: true
exit_code: 0
----- stdout -----
@@ -509,9 +509,9 @@ fn syntax_error() -> Result<()> {
fs::write(
tempdir.path().join("main.py"),
r#"
r"
from module import =
"#,
",
)?;
assert_cmd_snapshot!(Command::new(get_cargo_bin(BIN_NAME))
@@ -1945,11 +1945,10 @@ fn range_end_only() {
def foo(arg1, arg2,):
print("Should format this" )
"#), @r###"
"#), @r#"
success: true
exit_code: 0
----- stdout -----
def foo(
arg1,
arg2,
@@ -1958,7 +1957,7 @@ def foo(arg1, arg2,):
----- stderr -----
"###);
"#);
}
#[test]

View File

@@ -158,15 +158,15 @@ fn check_default_files() -> Result<()> {
let tempdir = TempDir::new()?;
fs::write(
tempdir.path().join("foo.py"),
r#"
r"
import foo # unused import
"#,
",
)?;
fs::write(
tempdir.path().join("bar.py"),
r#"
r"
import bar # unused import
"#,
",
)?;
assert_cmd_snapshot!(Command::new(get_cargo_bin(BIN_NAME))
@@ -906,10 +906,10 @@ fn full_output_preview_config() -> Result<()> {
let pyproject_toml = tempdir.path().join("pyproject.toml");
fs::write(
&pyproject_toml,
r#"
r"
[tool.ruff]
preview = true
"#,
",
)?;
let mut cmd = RuffCheck::default().config(&pyproject_toml).build();
assert_cmd_snapshot!(cmd.pass_stdin("l = 1"), @r###"

View File

@@ -1619,6 +1619,58 @@ print(
Ok(())
}
#[test]
fn add_noqa_exclude() -> Result<()> {
let tempdir = TempDir::new()?;
let ruff_toml = tempdir.path().join("ruff.toml");
fs::write(
&ruff_toml,
r#"
[lint]
exclude = ["excluded.py"]
select = ["RUF015"]
"#,
)?;
let test_path = tempdir.path().join("noqa.py");
fs::write(
&test_path,
r#"
def first_square():
return [x * x for x in range(20)][0]
"#,
)?;
let exclude_path = tempdir.path().join("excluded.py");
fs::write(
&exclude_path,
r#"
def first_square():
return [x * x for x in range(20)][0]
"#,
)?;
insta::with_settings!({
filters => vec![(tempdir_filter(&tempdir).as_str(), "[TMP]/")]
}, {
assert_cmd_snapshot!(Command::new(get_cargo_bin(BIN_NAME))
.current_dir(tempdir.path())
.args(STDIN_BASE_OPTIONS)
.args(["--add-noqa"]), @r###"
success: true
exit_code: 0
----- stdout -----
----- stderr -----
Added 1 noqa directive.
"###);
});
Ok(())
}
/// Infer `3.11` from `requires-python` in `pyproject.toml`.
#[test]
fn requires_python() -> Result<()> {

View File

@@ -200,7 +200,7 @@ linter.safety_table.forced_unsafe = []
linter.target_version = Py37
linter.preview = disabled
linter.explicit_preview_rules = false
linter.extension.mapping = {}
linter.extension = ExtensionMapping({})
linter.allowed_confusables = []
linter.builtins = []
linter.dummy_variable_rgx = ^(_+|(_+[a-zA-Z0-9_]*[a-zA-Z0-9]+?))$
@@ -388,4 +388,12 @@ formatter.magic_trailing_comma = respect
formatter.docstring_code_format = disabled
formatter.docstring_code_line_width = dynamic
# Analyze Settings
analyze.exclude = []
analyze.preview = disabled
analyze.target_version = Py37
analyze.detect_string_imports = false
analyze.extension = ExtensionMapping({})
analyze.include_dependencies = {}
----- stderr -----

View File

@@ -66,4 +66,4 @@ codspeed = ["codspeed-criterion-compat"]
mimalloc = { workspace = true }
[target.'cfg(all(not(target_os = "windows"), not(target_os = "openbsd"), any(target_arch = "x86_64", target_arch = "aarch64", target_arch = "powerpc64")))'.dev-dependencies]
tikv-jemallocator = { workspace = true, features = ["unprefixed_malloc_on_supported_platforms"] }
tikv-jemallocator = { workspace = true }

View File

@@ -42,9 +42,9 @@ static GLOBAL: tikv_jemallocator::Jemalloc = tikv_jemallocator::Jemalloc;
)
))]
#[allow(non_upper_case_globals)]
#[export_name = "malloc_conf"]
#[export_name = "_rjem_malloc_conf"]
#[allow(unsafe_code)]
pub static malloc_conf: &[u8] = b"dirty_decay_ms:-1,muzzy_decay_ms:-1\0";
pub static _rjem_malloc_conf: &[u8] = b"dirty_decay_ms:-1,muzzy_decay_ms:-1\0";
fn create_test_cases() -> Result<Vec<TestCase>, TestFileDownloadError> {
Ok(vec![

View File

@@ -23,7 +23,6 @@ const TOMLLIB_312_URL: &str = "https://raw.githubusercontent.com/python/cpython/
// The failed import from 'collections.abc' is due to lack of support for 'import *'.
static EXPECTED_DIAGNOSTICS: &[&str] = &[
"/src/tomllib/_parser.py:5:24: Module '__future__' has no member 'annotations'",
"/src/tomllib/_parser.py:7:29: Module 'collections.abc' has no member 'Iterable'",
"Line 69 is too long (89 characters)",
"Use double quotes for strings",

View File

@@ -26,6 +26,7 @@ filetime = { workspace = true }
ignore = { workspace = true, optional = true }
matchit = { workspace = true }
salsa = { workspace = true }
serde = { workspace = true, optional = true }
path-slash = { workspace = true }
thiserror = { workspace = true }
tracing = { workspace = true }
@@ -33,13 +34,15 @@ tracing-subscriber = { workspace = true, optional = true }
tracing-tree = { workspace = true, optional = true }
rustc-hash = { workspace = true }
[target.'cfg(not(target_arch="wasm32"))'.dependencies]
[target.'cfg(not(any(target_arch = "wasm32", target_arch = "powerpc64")))'.dependencies]
zip = { workspace = true, features = ["zstd"] }
[target.'cfg(target_arch="wasm32")'.dependencies]
web-time = { version = "1.1.0" }
[target.'cfg(any(target_arch = "wasm32", target_arch = "powerpc64"))'.dependencies]
zip = { workspace = true, features = ["deflate"] }
[target.'cfg(target_arch = "wasm32")'.dependencies]
web-time = { version = "1.1.0" }
[dev-dependencies]
insta = { workspace = true }
tempfile = { workspace = true }
@@ -47,5 +50,6 @@ tempfile = { workspace = true }
[features]
cache = ["ruff_cache"]
os = ["ignore"]
serde = ["dep:serde", "camino/serde1"]
# Exposes testing utilities.
testing = ["tracing-subscriber", "tracing-tree"]

View File

@@ -0,0 +1,52 @@
use std::fmt::{self, Display, Formatter};
pub trait FormatterJoinExtension<'b> {
fn join<'a>(&'a mut self, separator: &'static str) -> Join<'a, 'b>;
}
impl<'b> FormatterJoinExtension<'b> for Formatter<'b> {
fn join<'a>(&'a mut self, separator: &'static str) -> Join<'a, 'b> {
Join {
fmt: self,
separator,
result: fmt::Result::Ok(()),
seen_first: false,
}
}
}
pub struct Join<'a, 'b> {
fmt: &'a mut Formatter<'b>,
separator: &'static str,
result: fmt::Result,
seen_first: bool,
}
impl<'a, 'b> Join<'a, 'b> {
pub fn entry(&mut self, item: &dyn Display) -> &mut Self {
if self.seen_first {
self.result = self
.result
.and_then(|()| self.fmt.write_str(self.separator));
} else {
self.seen_first = true;
}
self.result = self.result.and_then(|()| item.fmt(self.fmt));
self
}
pub fn entries<I, F>(&mut self, items: I) -> &mut Self
where
I: IntoIterator<Item = F>,
F: Display,
{
for item in items {
self.entry(&item);
}
self
}
pub fn finish(&mut self) -> fmt::Result {
self.result
}
}

View File

@@ -6,6 +6,7 @@ use crate::files::Files;
use crate::system::System;
use crate::vendored::VendoredFileSystem;
pub mod display;
pub mod file_revision;
pub mod files;
pub mod parsed;

View File

@@ -16,7 +16,7 @@ use super::walk_directory::{
};
/// A system implementation that uses the OS file system.
#[derive(Default, Debug)]
#[derive(Default, Debug, Clone)]
pub struct OsSystem {
inner: Arc<OsSystemInner>,
}

View File

@@ -593,6 +593,27 @@ impl ruff_cache::CacheKey for SystemPathBuf {
}
}
#[cfg(feature = "serde")]
impl serde::Serialize for SystemPath {
fn serialize<S: serde::Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
self.0.serialize(serializer)
}
}
#[cfg(feature = "serde")]
impl serde::Serialize for SystemPathBuf {
fn serialize<S: serde::Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
self.0.serialize(serializer)
}
}
#[cfg(feature = "serde")]
impl<'de> serde::Deserialize<'de> for SystemPathBuf {
fn deserialize<D: serde::Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
Utf8PathBuf::deserialize(deserializer).map(SystemPathBuf)
}
}
/// A slice of a virtual path on [`System`](super::System) (akin to [`str`]).
#[repr(transparent)]
pub struct SystemVirtualPath(str);

View File

@@ -194,6 +194,10 @@ pub(crate) struct Args {
/// Format the files. Without this flag, the python files are not modified
#[arg(long)]
pub(crate) write: bool,
#[arg(long)]
pub(crate) preview: bool,
/// Control the verbosity of the output
#[arg(long, default_value_t, value_enum)]
pub(crate) format: Format,
@@ -235,7 +239,8 @@ pub(crate) fn main(args: &Args) -> anyhow::Result<ExitCode> {
let all_success = if args.multi_project {
format_dev_multi_project(args, error_file)?
} else {
let result = format_dev_project(&args.files, args.stability_check, args.write)?;
let result =
format_dev_project(&args.files, args.stability_check, args.write, args.preview)?;
let error_count = result.error_count();
if result.error_count() > 0 {
@@ -344,7 +349,12 @@ fn format_dev_multi_project(
for project_path in project_paths {
debug!(parent: None, "Starting {}", project_path.display());
match format_dev_project(&[project_path.clone()], args.stability_check, args.write) {
match format_dev_project(
&[project_path.clone()],
args.stability_check,
args.write,
args.preview,
) {
Ok(result) => {
total_errors += result.error_count();
total_files += result.file_count;
@@ -442,6 +452,7 @@ fn format_dev_project(
files: &[PathBuf],
stability_check: bool,
write: bool,
preview: bool,
) -> anyhow::Result<CheckRepoResult> {
let start = Instant::now();
@@ -477,7 +488,14 @@ fn format_dev_project(
#[cfg(feature = "singlethreaded")]
let iter = { paths.into_iter() };
iter.map(|path| {
let result = format_dir_entry(path, stability_check, write, &black_options, &resolver);
let result = format_dir_entry(
path,
stability_check,
write,
preview,
&black_options,
&resolver,
);
pb_span.pb_inc(1);
result
})
@@ -532,6 +550,7 @@ fn format_dir_entry(
resolved_file: Result<ResolvedFile, ignore::Error>,
stability_check: bool,
write: bool,
preview: bool,
options: &BlackOptions,
resolver: &Resolver,
) -> anyhow::Result<(Result<Statistics, CheckFileError>, PathBuf), Error> {
@@ -544,6 +563,10 @@ fn format_dir_entry(
let path = resolved_file.into_path();
let mut options = options.to_py_format_options(&path);
if preview {
options = options.with_preview(PreviewMode::Enabled);
}
let settings = resolver.resolve(&path);
// That's a bad way of doing this but it's not worth doing something better for format_dev
if settings.formatter.line_width != LineWidth::default() {
@@ -551,9 +574,8 @@ fn format_dir_entry(
}
// Handle panics (mostly in `debug_assert!`)
let result = match catch_unwind(|| format_dev_file(&path, stability_check, write, options)) {
Ok(result) => result,
Err(panic) => {
let result = catch_unwind(|| format_dev_file(&path, stability_check, write, options))
.unwrap_or_else(|panic| {
if let Some(message) = panic.downcast_ref::<String>() {
Err(CheckFileError::Panic {
message: message.clone(),
@@ -568,8 +590,7 @@ fn format_dir_entry(
message: "(Panic didn't set a string message)".to_string(),
})
}
}
};
});
Ok((result, path))
}

View File

@@ -6,6 +6,7 @@ use std::fs;
use std::path::PathBuf;
use anyhow::Result;
use itertools::Itertools;
use regex::{Captures, Regex};
use strum::IntoEnumIterator;
@@ -33,7 +34,26 @@ pub(crate) fn main(args: &Args) -> Result<()> {
let (linter, _) = Linter::parse_code(&rule.noqa_code().to_string()).unwrap();
if linter.url().is_some() {
output.push_str(&format!("Derived from the **{}** linter.", linter.name()));
let common_prefix: String = match linter.common_prefix() {
"" => linter
.upstream_categories()
.unwrap()
.iter()
.map(|c| c.prefix)
.join("-"),
prefix => prefix.to_string(),
};
let anchor = format!(
"{}-{}",
linter.name().to_lowercase(),
common_prefix.to_lowercase()
);
output.push_str(&format!(
"Derived from the **[{}](../rules.md#{})** linter.",
linter.name(),
anchor
));
output.push('\n');
output.push('\n');
}

View File

@@ -0,0 +1,31 @@
[package]
name = "ruff_graph"
version = "0.1.0"
edition.workspace = true
rust-version.workspace = true
homepage.workspace = true
documentation.workspace = true
repository.workspace = true
authors.workspace = true
license.workspace = true
[dependencies]
red_knot_python_semantic = { workspace = true }
ruff_cache = { workspace = true }
ruff_db = { workspace = true, features = ["os", "serde"] }
ruff_linter = { workspace = true }
ruff_macros = { workspace = true }
ruff_python_ast = { workspace = true }
anyhow = { workspace = true }
clap = { workspace = true, optional = true }
salsa = { workspace = true }
schemars = { workspace = true, optional = true }
serde = { workspace = true, optional = true }
[lints]
workspace = true
[package.metadata.cargo-shear]
# Used via `CacheKey` macro expansion.
ignored = ["ruff_cache"]

View File

@@ -0,0 +1,111 @@
use red_knot_python_semantic::ModuleName;
use ruff_python_ast::visitor::source_order::{walk_body, walk_expr, walk_stmt, SourceOrderVisitor};
use ruff_python_ast::{self as ast, Expr, ModModule, Stmt};
/// Collect all imports for a given Python file.
#[derive(Default, Debug)]
pub(crate) struct Collector<'a> {
/// The path to the current module.
module_path: Option<&'a [String]>,
/// Whether to detect imports from string literals.
string_imports: bool,
/// The collected imports from the Python AST.
imports: Vec<CollectedImport>,
}
impl<'a> Collector<'a> {
pub(crate) fn new(module_path: Option<&'a [String]>, string_imports: bool) -> Self {
Self {
module_path,
string_imports,
imports: Vec::new(),
}
}
#[must_use]
pub(crate) fn collect(mut self, module: &ModModule) -> Vec<CollectedImport> {
walk_body(&mut self, &module.body);
self.imports
}
}
impl<'ast> SourceOrderVisitor<'ast> for Collector<'_> {
fn visit_stmt(&mut self, stmt: &'ast Stmt) {
match stmt {
Stmt::ImportFrom(ast::StmtImportFrom {
names,
module,
level,
range: _,
}) => {
let module = module.as_deref();
let level = *level;
for alias in names {
let mut components = vec![];
if level > 0 {
// If we're resolving a relative import, we must have a module path.
let Some(module_path) = self.module_path else {
return;
};
// Start with the containing module.
components.extend(module_path.iter().map(String::as_str));
// Remove segments based on the number of dots.
for _ in 0..level {
if components.is_empty() {
return;
}
components.pop();
}
}
// Add the module path.
if let Some(module) = module {
components.extend(module.split('.'));
}
// Add the alias name.
components.push(alias.name.as_str());
if let Some(module_name) = ModuleName::from_components(components) {
self.imports.push(CollectedImport::ImportFrom(module_name));
}
}
}
Stmt::Import(ast::StmtImport { names, range: _ }) => {
for alias in names {
if let Some(module_name) = ModuleName::new(alias.name.as_str()) {
self.imports.push(CollectedImport::Import(module_name));
}
}
}
_ => {
walk_stmt(self, stmt);
}
}
}
fn visit_expr(&mut self, expr: &'ast Expr) {
if self.string_imports {
if let Expr::StringLiteral(ast::ExprStringLiteral { value, range: _ }) = expr {
// Determine whether the string literal "looks like" an import statement: contains
// a dot, and consists solely of valid Python identifiers.
let value = value.to_str();
if let Some(module_name) = ModuleName::new(value) {
self.imports.push(CollectedImport::Import(module_name));
}
}
walk_expr(self, expr);
}
}
}
#[derive(Debug)]
pub(crate) enum CollectedImport {
/// The import was part of an `import` statement.
Import(ModuleName),
/// The import was part of an `import from` statement.
ImportFrom(ModuleName),
}

View File

@@ -0,0 +1,97 @@
use anyhow::Result;
use red_knot_python_semantic::{Db, Program, ProgramSettings, PythonVersion, SearchPathSettings};
use ruff_db::files::{File, Files};
use ruff_db::system::{OsSystem, System, SystemPathBuf};
use ruff_db::vendored::VendoredFileSystem;
use ruff_db::{Db as SourceDb, Upcast};
#[salsa::db]
#[derive(Default)]
pub struct ModuleDb {
storage: salsa::Storage<Self>,
files: Files,
system: OsSystem,
vendored: VendoredFileSystem,
}
impl ModuleDb {
/// Initialize a [`ModuleDb`] from the given source root.
pub fn from_src_roots(
mut src_roots: impl Iterator<Item = SystemPathBuf>,
target_version: PythonVersion,
) -> Result<Self> {
let search_paths = {
// Use the first source root.
let src_root = src_roots
.next()
.ok_or_else(|| anyhow::anyhow!("No source roots provided"))?;
let mut search_paths = SearchPathSettings::new(src_root.to_path_buf());
// Add the remaining source roots as extra paths.
for src_root in src_roots {
search_paths.extra_paths.push(src_root.to_path_buf());
}
search_paths
};
let db = Self::default();
Program::from_settings(
&db,
&ProgramSettings {
target_version,
search_paths,
},
)?;
Ok(db)
}
/// Create a snapshot of the current database.
#[must_use]
pub fn snapshot(&self) -> Self {
Self {
storage: self.storage.clone(),
system: self.system.clone(),
vendored: self.vendored.clone(),
files: self.files.snapshot(),
}
}
}
impl Upcast<dyn SourceDb> for ModuleDb {
fn upcast(&self) -> &(dyn SourceDb + 'static) {
self
}
fn upcast_mut(&mut self) -> &mut (dyn SourceDb + 'static) {
self
}
}
#[salsa::db]
impl SourceDb for ModuleDb {
fn vendored(&self) -> &VendoredFileSystem {
&self.vendored
}
fn system(&self) -> &dyn System {
&self.system
}
fn files(&self) -> &Files {
&self.files
}
}
#[salsa::db]
impl Db for ModuleDb {
fn is_file_open(&self, file: File) -> bool {
!file.path(self).is_vendored_path()
}
}
#[salsa::db]
impl salsa::Database for ModuleDb {
fn salsa_event(&self, _event: &dyn Fn() -> salsa::Event) {}
}

View File

@@ -0,0 +1,126 @@
use crate::collector::Collector;
pub use crate::db::ModuleDb;
use crate::resolver::Resolver;
pub use crate::settings::{AnalyzeSettings, Direction};
use anyhow::Result;
use red_knot_python_semantic::SemanticModel;
use ruff_db::files::system_path_to_file;
use ruff_db::parsed::parsed_module;
use ruff_db::system::{SystemPath, SystemPathBuf};
use ruff_python_ast::helpers::to_module_path;
use serde::{Deserialize, Serialize};
use std::collections::{BTreeMap, BTreeSet};
mod collector;
mod db;
mod resolver;
mod settings;
#[derive(Debug, Default)]
#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
pub struct ModuleImports(BTreeSet<SystemPathBuf>);
impl ModuleImports {
/// Detect the [`ModuleImports`] for a given Python file.
pub fn detect(
db: &ModuleDb,
path: &SystemPath,
package: Option<&SystemPath>,
string_imports: bool,
) -> Result<Self> {
// Read and parse the source code.
let file = system_path_to_file(db, path)?;
let parsed = parsed_module(db, file);
let module_path =
package.and_then(|package| to_module_path(package.as_std_path(), path.as_std_path()));
let model = SemanticModel::new(db, file);
// Collect the imports.
let imports =
Collector::new(module_path.as_deref(), string_imports).collect(parsed.syntax());
// Resolve the imports.
let mut resolved_imports = ModuleImports::default();
for import in imports {
let Some(resolved) = Resolver::new(&model).resolve(import) else {
continue;
};
let Some(path) = resolved.as_system_path() else {
continue;
};
resolved_imports.insert(path.to_path_buf());
}
Ok(resolved_imports)
}
/// Insert a file path into the module imports.
pub fn insert(&mut self, path: SystemPathBuf) {
self.0.insert(path);
}
/// Extend the module imports with additional file paths.
pub fn extend(&mut self, paths: impl IntoIterator<Item = SystemPathBuf>) {
self.0.extend(paths);
}
/// Returns `true` if the module imports are empty.
pub fn is_empty(&self) -> bool {
self.0.is_empty()
}
/// Returns the number of module imports.
pub fn len(&self) -> usize {
self.0.len()
}
/// Convert the file paths to be relative to a given path.
#[must_use]
pub fn relative_to(self, path: &SystemPath) -> Self {
Self(
self.0
.into_iter()
.map(|import| {
import
.strip_prefix(path)
.map(SystemPath::to_path_buf)
.unwrap_or(import)
})
.collect(),
)
}
}
#[derive(Debug, Default)]
#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
pub struct ImportMap(BTreeMap<SystemPathBuf, ModuleImports>);
impl ImportMap {
/// Insert a module's imports into the map.
pub fn insert(&mut self, path: SystemPathBuf, imports: ModuleImports) {
self.0.insert(path, imports);
}
/// Reverse the [`ImportMap`], e.g., to convert from dependencies to dependents.
#[must_use]
pub fn reverse(imports: impl IntoIterator<Item = (SystemPathBuf, ModuleImports)>) -> Self {
let mut reverse = ImportMap::default();
for (path, imports) in imports {
for import in imports.0 {
reverse.0.entry(import).or_default().insert(path.clone());
}
reverse.0.entry(path).or_default();
}
reverse
}
}
impl FromIterator<(SystemPathBuf, ModuleImports)> for ImportMap {
fn from_iter<I: IntoIterator<Item = (SystemPathBuf, ModuleImports)>>(iter: I) -> Self {
let mut map = ImportMap::default();
for (path, imports) in iter {
map.0.entry(path).or_default().0.extend(imports.0);
}
map
}
}

View File

@@ -0,0 +1,39 @@
use red_knot_python_semantic::SemanticModel;
use ruff_db::files::FilePath;
use crate::collector::CollectedImport;
/// Collect all imports for a given Python file.
pub(crate) struct Resolver<'a> {
semantic: &'a SemanticModel<'a>,
}
impl<'a> Resolver<'a> {
/// Initialize a [`Resolver`] with a given [`SemanticModel`].
pub(crate) fn new(semantic: &'a SemanticModel<'a>) -> Self {
Self { semantic }
}
/// Resolve the [`CollectedImport`] into a [`FilePath`].
pub(crate) fn resolve(&self, import: CollectedImport) -> Option<&'a FilePath> {
match import {
CollectedImport::Import(import) => self
.semantic
.resolve_module(import)
.map(|module| module.file().path(self.semantic.db())),
CollectedImport::ImportFrom(import) => {
// Attempt to resolve the member (e.g., given `from foo import bar`, look for `foo.bar`).
let parent = import.parent();
self.semantic
.resolve_module(import)
.map(|module| module.file().path(self.semantic.db()))
.or_else(|| {
// Attempt to resolve the module (e.g., given `from foo import bar`, look for `foo`).
self.semantic
.resolve_module(parent?)
.map(|module| module.file().path(self.semantic.db()))
})
}
}
}
}

View File

@@ -0,0 +1,56 @@
use ruff_linter::display_settings;
use ruff_linter::settings::types::{ExtensionMapping, FilePatternSet, PreviewMode, PythonVersion};
use ruff_macros::CacheKey;
use std::collections::BTreeMap;
use std::fmt;
use std::path::PathBuf;
#[derive(Debug, Default, Clone, CacheKey)]
pub struct AnalyzeSettings {
pub exclude: FilePatternSet,
pub preview: PreviewMode,
pub target_version: PythonVersion,
pub detect_string_imports: bool,
pub include_dependencies: BTreeMap<PathBuf, (PathBuf, Vec<String>)>,
pub extension: ExtensionMapping,
}
impl fmt::Display for AnalyzeSettings {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
writeln!(f, "\n# Analyze Settings")?;
display_settings! {
formatter = f,
namespace = "analyze",
fields = [
self.exclude,
self.preview,
self.target_version | debug,
self.detect_string_imports,
self.extension | debug,
self.include_dependencies | debug,
]
}
Ok(())
}
}
#[derive(Default, Debug, Copy, Clone, PartialEq, Eq, CacheKey)]
#[cfg_attr(feature = "serde", derive(serde::Serialize, serde::Deserialize))]
#[cfg_attr(feature = "schemars", derive(schemars::JsonSchema))]
#[cfg_attr(feature = "clap", derive(clap::ValueEnum))]
pub enum Direction {
/// Construct a map from module to its dependencies (i.e., the modules that it imports).
#[default]
Dependencies,
/// Construct a map from module to its dependents (i.e., the modules that import it).
Dependents,
}
impl fmt::Display for Direction {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Self::Dependencies => write!(f, "\"dependencies\""),
Self::Dependents => write!(f, "\"dependents\""),
}
}
}

View File

@@ -1,6 +1,6 @@
[package]
name = "ruff_linter"
version = "0.6.4"
version = "0.6.6"
publish = false
authors = { workspace = true }
edition = { workspace = true }
@@ -73,7 +73,7 @@ unicode-normalization = { workspace = true }
url = { workspace = true }
[dev-dependencies]
insta = { workspace = true }
insta = { workspace = true, features = ["filters", "json", "redactions"] }
test-case = { workspace = true }
# Disable colored output in tests
colored = { workspace = true, features = ["no-color"] }

View File

@@ -1,4 +1,6 @@
from fastapi import FastAPI
from typing import Annotated
from fastapi import FastAPI, Path
app = FastAPI()
@@ -82,6 +84,11 @@ async def read_thing(
return {"query": query}
@app.get("/books/{name}/{title}")
async def read_thing(*, author: Annotated[str, Path(alias="author_name")], title: str):
return {"author": author, "title": title}
# OK
@app.get("/things/{thing_id}")
async def read_thing(thing_id: int, query: str):
@@ -118,6 +125,11 @@ async def read_thing(*, author: str, title: str):
return {"author": author, "title": title}
@app.get("/books/{name}/{title}")
async def read_thing(*, author: Annotated[str, Path(alias="name")], title: str):
return {"author": author, "title": title}
# Ignored
@app.get("/things/{thing-id}")
async def read_thing(query: str):
@@ -131,4 +143,4 @@ async def read_thing(query: str):
@app.get("/things/{thing_id=}")
async def read_thing(query: str):
return {"query": query}
return {"query": query}

View File

@@ -0,0 +1,20 @@
def f():
"Here's a line ending in a question mark?"
...
def f():
"""Here's a line ending in an exclamation mark!"""
...
def f():
"""Here's a line ending in a colon:"""
...
def f():
"""Here's a line ending in a semi colon;"""
...
def f():
"""Here's a line ending with a whitespace """
...

View File

@@ -0,0 +1,8 @@
def foo():
"""Returns foo()."""
def foo():
""""Use prefix_foo()."""
def foo():
""""Use this function; foo()."""

View File

@@ -151,4 +151,22 @@ def remove_prefix_comparable_literal_expr() -> None:
def shadow_builtins(filename: str, extension: str) -> None:
from builtins import len as builtins_len
return filename[:-builtins_len(extension)] if filename.endswith(extension) else filename
return filename[:-builtins_len(extension)] if filename.endswith(extension) else filename
def okay_steps():
text = "!x!y!z"
if text.startswith("!"):
text = text[1::1]
if text.startswith("!"):
text = text[1::True]
if text.startswith("!"):
text = text[1::None]
print(text)
# this should be skipped
def ignore_step():
text = "!x!y!z"
if text.startswith("!"):
text = text[1::2]
print(text)

View File

@@ -152,6 +152,8 @@ pub fn set_up_logging(level: LogLevel) -> Result<()> {
})
.level(level.level_filter())
.level_for("globset", log::LevelFilter::Warn)
.level_for("red_knot_python_semantic", log::LevelFilter::Warn)
.level_for("salsa", log::LevelFilter::Warn)
.chain(std::io::stderr())
.apply()?;
Ok(())

View File

@@ -186,7 +186,6 @@ impl Serialize for SarifResult {
#[cfg(test)]
mod tests {
use crate::message::tests::{
capture_emitter_output, create_messages, create_syntax_error_messages,
};
@@ -213,30 +212,11 @@ mod tests {
#[test]
fn test_results() {
let content = get_output();
let sarif = serde_json::from_str::<serde_json::Value>(content.as_str()).unwrap();
let rules = sarif["runs"][0]["tool"]["driver"]["rules"]
.as_array()
.unwrap();
let results = sarif["runs"][0]["results"].as_array().unwrap();
assert_eq!(results.len(), 3);
assert_eq!(
results
.iter()
.map(|r| r["message"]["text"].as_str().unwrap())
.collect::<Vec<_>>(),
vec![
"`os` imported but unused",
"Local variable `x` is assigned to but never used",
"Undefined name `a`",
]
);
assert_eq!(rules.len(), 3);
assert_eq!(
rules
.iter()
.map(|r| r["id"].as_str().unwrap())
.collect::<Vec<_>>(),
vec!["F401", "F821", "F841"],
);
let value = serde_json::from_str::<serde_json::Value>(&content).unwrap();
insta::assert_json_snapshot!(value, {
".runs[0].tool.driver.version" => "[VERSION]",
".runs[0].results[].locations[].physicalLocation.artifactLocation.uri" => "[URI]",
});
}
}

View File

@@ -0,0 +1,146 @@
---
source: crates/ruff_linter/src/message/sarif.rs
expression: value
---
{
"$schema": "https://json.schemastore.org/sarif-2.1.0.json",
"runs": [
{
"results": [
{
"level": "error",
"locations": [
{
"physicalLocation": {
"artifactLocation": {
"uri": "[URI]"
},
"region": {
"endColumn": 10,
"endLine": 1,
"startColumn": 8,
"startLine": 1
}
}
}
],
"message": {
"text": "`os` imported but unused"
},
"ruleId": "F401"
},
{
"level": "error",
"locations": [
{
"physicalLocation": {
"artifactLocation": {
"uri": "[URI]"
},
"region": {
"endColumn": 6,
"endLine": 6,
"startColumn": 5,
"startLine": 6
}
}
}
],
"message": {
"text": "Local variable `x` is assigned to but never used"
},
"ruleId": "F841"
},
{
"level": "error",
"locations": [
{
"physicalLocation": {
"artifactLocation": {
"uri": "[URI]"
},
"region": {
"endColumn": 5,
"endLine": 1,
"startColumn": 4,
"startLine": 1
}
}
}
],
"message": {
"text": "Undefined name `a`"
},
"ruleId": "F821"
}
],
"tool": {
"driver": {
"informationUri": "https://github.com/astral-sh/ruff",
"name": "ruff",
"rules": [
{
"fullDescription": {
"text": "## What it does\nChecks for unused imports.\n\n## Why is this bad?\nUnused imports add a performance overhead at runtime, and risk creating\nimport cycles. They also increase the cognitive load of reading the code.\n\nIf an import statement is used to check for the availability or existence\nof a module, consider using `importlib.util.find_spec` instead.\n\nIf an import statement is used to re-export a symbol as part of a module's\npublic interface, consider using a \"redundant\" import alias, which\ninstructs Ruff (and other tools) to respect the re-export, and avoid\nmarking it as unused, as in:\n\n```python\nfrom module import member as member\n```\n\nAlternatively, you can use `__all__` to declare a symbol as part of the module's\ninterface, as in:\n\n```python\n# __init__.py\nimport some_module\n\n__all__ = [\"some_module\"]\n```\n\n## Fix safety\n\nFixes to remove unused imports are safe, except in `__init__.py` files.\n\nApplying fixes to `__init__.py` files is currently in preview. The fix offered depends on the\ntype of the unused import. Ruff will suggest a safe fix to export first-party imports with\neither a redundant alias or, if already present in the file, an `__all__` entry. If multiple\n`__all__` declarations are present, Ruff will not offer a fix. Ruff will suggest an unsafe fix\nto remove third-party and standard library imports -- the fix is unsafe because the module's\ninterface changes.\n\n## Example\n\n```python\nimport numpy as np # unused import\n\n\ndef area(radius):\n return 3.14 * radius**2\n```\n\nUse instead:\n\n```python\ndef area(radius):\n return 3.14 * radius**2\n```\n\nTo check the availability of a module, use `importlib.util.find_spec`:\n\n```python\nfrom importlib.util import find_spec\n\nif find_spec(\"numpy\") is not None:\n print(\"numpy is installed\")\nelse:\n print(\"numpy is not installed\")\n```\n\n## Options\n- `lint.ignore-init-module-imports`\n\n## References\n- [Python documentation: `import`](https://docs.python.org/3/reference/simple_stmts.html#the-import-statement)\n- [Python documentation: `importlib.util.find_spec`](https://docs.python.org/3/library/importlib.html#importlib.util.find_spec)\n- [Typing documentation: interface conventions](https://typing.readthedocs.io/en/latest/source/libraries.html#library-interface-public-and-private-symbols)\n"
},
"help": {
"text": "`{name}` imported but unused; consider using `importlib.util.find_spec` to test for availability"
},
"helpUri": "https://docs.astral.sh/ruff/rules/unused-import",
"id": "F401",
"properties": {
"id": "F401",
"kind": "Pyflakes",
"name": "unused-import",
"problem.severity": "error"
},
"shortDescription": {
"text": "`{name}` imported but unused; consider using `importlib.util.find_spec` to test for availability"
}
},
{
"fullDescription": {
"text": "## What it does\nChecks for uses of undefined names.\n\n## Why is this bad?\nAn undefined name is likely to raise `NameError` at runtime.\n\n## Example\n```python\ndef double():\n return n * 2 # raises `NameError` if `n` is undefined when `double` is called\n```\n\nUse instead:\n```python\ndef double(n):\n return n * 2\n```\n\n## Options\n- [`target-version`]: Can be used to configure which symbols Ruff will understand\n as being available in the `builtins` namespace.\n\n## References\n- [Python documentation: Naming and binding](https://docs.python.org/3/reference/executionmodel.html#naming-and-binding)\n"
},
"help": {
"text": "Undefined name `{name}`. {tip}"
},
"helpUri": "https://docs.astral.sh/ruff/rules/undefined-name",
"id": "F821",
"properties": {
"id": "F821",
"kind": "Pyflakes",
"name": "undefined-name",
"problem.severity": "error"
},
"shortDescription": {
"text": "Undefined name `{name}`. {tip}"
}
},
{
"fullDescription": {
"text": "## What it does\nChecks for the presence of unused variables in function scopes.\n\n## Why is this bad?\nA variable that is defined but not used is likely a mistake, and should\nbe removed to avoid confusion.\n\nIf a variable is intentionally defined-but-not-used, it should be\nprefixed with an underscore, or some other value that adheres to the\n[`lint.dummy-variable-rgx`] pattern.\n\nUnder [preview mode](https://docs.astral.sh/ruff/preview), this rule also\ntriggers on unused unpacked assignments (for example, `x, y = foo()`).\n\n## Example\n```python\ndef foo():\n x = 1\n y = 2\n return x\n```\n\nUse instead:\n```python\ndef foo():\n x = 1\n return x\n```\n\n## Options\n- `lint.dummy-variable-rgx`\n"
},
"help": {
"text": "Local variable `{name}` is assigned to but never used"
},
"helpUri": "https://docs.astral.sh/ruff/rules/unused-variable",
"id": "F841",
"properties": {
"id": "F841",
"kind": "Pyflakes",
"name": "unused-variable",
"problem.severity": "error"
},
"shortDescription": {
"text": "Local variable `{name}` is assigned to but never used"
}
}
],
"version": "[VERSION]"
}
}
}
],
"version": "2.1.0"
}

View File

@@ -6,7 +6,8 @@ use ruff_diagnostics::Fix;
use ruff_diagnostics::{Diagnostic, FixAvailability, Violation};
use ruff_macros::{derive_message_formats, violation};
use ruff_python_ast as ast;
use ruff_python_semantic::Modules;
use ruff_python_ast::{Expr, Parameter, ParameterWithDefault};
use ruff_python_semantic::{Modules, SemanticModel};
use ruff_python_stdlib::identifiers::is_identifier;
use ruff_text_size::{Ranged, TextSize};
@@ -141,7 +142,10 @@ pub(crate) fn fastapi_unused_path_parameter(
.args
.iter()
.chain(function_def.parameters.kwonlyargs.iter())
.map(|arg| arg.parameter.name.as_str())
.map(|ParameterWithDefault { parameter, .. }| {
parameter_alias(parameter, checker.semantic())
.unwrap_or_else(|| parameter.name.as_str())
})
.collect();
// Check if any of the path parameters are not in the function signature.
@@ -190,6 +194,52 @@ pub(crate) fn fastapi_unused_path_parameter(
checker.diagnostics.extend(diagnostics);
}
/// Extract the expected in-route name for a given parameter, if it has an alias.
/// For example, given `document_id: Annotated[str, Path(alias="documentId")]`, returns `"documentId"`.
fn parameter_alias<'a>(parameter: &'a Parameter, semantic: &SemanticModel) -> Option<&'a str> {
let Some(annotation) = &parameter.annotation else {
return None;
};
let Expr::Subscript(subscript) = annotation.as_ref() else {
return None;
};
let Expr::Tuple(tuple) = subscript.slice.as_ref() else {
return None;
};
let Some(Expr::Call(path)) = tuple.elts.get(1) else {
return None;
};
// Find the `alias` keyword argument.
let alias = path
.arguments
.find_keyword("alias")
.map(|alias| &alias.value)?;
// Ensure that it's a literal string.
let Expr::StringLiteral(alias) = alias else {
return None;
};
// Verify that the subscript was a `typing.Annotated`.
if !semantic.match_typing_expr(&subscript.value, "Annotated") {
return None;
}
// Verify that the call was a `fastapi.Path`.
if !semantic
.resolve_qualified_name(&path.func)
.is_some_and(|qualified_name| matches!(qualified_name.segments(), ["fastapi", "Path"]))
{
return None;
}
Some(alias.value.to_str())
}
/// An iterator to extract parameters from FastAPI route paths.
///
/// The iterator yields tuples of the parameter name and the range of the parameter in the input,

View File

@@ -1,323 +1,342 @@
---
source: crates/ruff_linter/src/rules/fastapi/mod.rs
---
FAST003.py:7:19: FAST003 [*] Parameter `thing_id` appears in route path, but not in `read_thing` signature
|
6 | # Errors
7 | @app.get("/things/{thing_id}")
| ^^^^^^^^^^ FAST003
8 | async def read_thing(query: str):
9 | return {"query": query}
|
= help: Add `thing_id` to function signature
FAST003.py:9:19: FAST003 [*] Parameter `thing_id` appears in route path, but not in `read_thing` signature
|
8 | # Errors
9 | @app.get("/things/{thing_id}")
| ^^^^^^^^^^ FAST003
10 | async def read_thing(query: str):
11 | return {"query": query}
|
= help: Add `thing_id` to function signature
Unsafe fix
5 5 |
6 6 | # Errors
7 7 | @app.get("/things/{thing_id}")
8 |-async def read_thing(query: str):
8 |+async def read_thing(query: str, thing_id):
9 9 | return {"query": query}
10 10 |
11 11 |
7 7 |
8 8 | # Errors
9 9 | @app.get("/things/{thing_id}")
10 |-async def read_thing(query: str):
10 |+async def read_thing(query: str, thing_id):
11 11 | return {"query": query}
12 12 |
13 13 |
FAST003.py:12:23: FAST003 [*] Parameter `isbn` appears in route path, but not in `read_thing` signature
FAST003.py:14:23: FAST003 [*] Parameter `isbn` appears in route path, but not in `read_thing` signature
|
12 | @app.get("/books/isbn-{isbn}")
14 | @app.get("/books/isbn-{isbn}")
| ^^^^^^ FAST003
13 | async def read_thing():
14 | ...
15 | async def read_thing():
16 | ...
|
= help: Add `isbn` to function signature
Unsafe fix
10 10 |
11 11 |
12 12 | @app.get("/books/isbn-{isbn}")
13 |-async def read_thing():
13 |+async def read_thing(isbn):
14 14 | ...
15 15 |
16 16 |
12 12 |
13 13 |
14 14 | @app.get("/books/isbn-{isbn}")
15 |-async def read_thing():
15 |+async def read_thing(isbn):
16 16 | ...
17 17 |
18 18 |
FAST003.py:17:19: FAST003 [*] Parameter `thing_id` appears in route path, but not in `read_thing` signature
FAST003.py:19:19: FAST003 [*] Parameter `thing_id` appears in route path, but not in `read_thing` signature
|
17 | @app.get("/things/{thing_id:path}")
19 | @app.get("/things/{thing_id:path}")
| ^^^^^^^^^^^^^^^ FAST003
18 | async def read_thing(query: str):
19 | return {"query": query}
20 | async def read_thing(query: str):
21 | return {"query": query}
|
= help: Add `thing_id` to function signature
Unsafe fix
15 15 |
16 16 |
17 17 | @app.get("/things/{thing_id:path}")
18 |-async def read_thing(query: str):
18 |+async def read_thing(query: str, thing_id):
19 19 | return {"query": query}
20 20 |
21 21 |
17 17 |
18 18 |
19 19 | @app.get("/things/{thing_id:path}")
20 |-async def read_thing(query: str):
20 |+async def read_thing(query: str, thing_id):
21 21 | return {"query": query}
22 22 |
23 23 |
FAST003.py:22:19: FAST003 [*] Parameter `thing_id` appears in route path, but not in `read_thing` signature
FAST003.py:24:19: FAST003 [*] Parameter `thing_id` appears in route path, but not in `read_thing` signature
|
22 | @app.get("/things/{thing_id : path}")
24 | @app.get("/things/{thing_id : path}")
| ^^^^^^^^^^^^^^^^^ FAST003
23 | async def read_thing(query: str):
24 | return {"query": query}
25 | async def read_thing(query: str):
26 | return {"query": query}
|
= help: Add `thing_id` to function signature
Unsafe fix
20 20 |
21 21 |
22 22 | @app.get("/things/{thing_id : path}")
23 |-async def read_thing(query: str):
23 |+async def read_thing(query: str, thing_id):
24 24 | return {"query": query}
25 25 |
26 26 |
22 22 |
23 23 |
24 24 | @app.get("/things/{thing_id : path}")
25 |-async def read_thing(query: str):
25 |+async def read_thing(query: str, thing_id):
26 26 | return {"query": query}
27 27 |
28 28 |
FAST003.py:27:27: FAST003 [*] Parameter `title` appears in route path, but not in `read_thing` signature
FAST003.py:29:27: FAST003 [*] Parameter `title` appears in route path, but not in `read_thing` signature
|
27 | @app.get("/books/{author}/{title}")
29 | @app.get("/books/{author}/{title}")
| ^^^^^^^ FAST003
28 | async def read_thing(author: str):
29 | return {"author": author}
30 | async def read_thing(author: str):
31 | return {"author": author}
|
= help: Add `title` to function signature
Unsafe fix
25 25 |
26 26 |
27 27 | @app.get("/books/{author}/{title}")
28 |-async def read_thing(author: str):
28 |+async def read_thing(author: str, title):
29 29 | return {"author": author}
30 30 |
31 31 |
27 27 |
28 28 |
29 29 | @app.get("/books/{author}/{title}")
30 |-async def read_thing(author: str):
30 |+async def read_thing(author: str, title):
31 31 | return {"author": author}
32 32 |
33 33 |
FAST003.py:32:18: FAST003 [*] Parameter `author_name` appears in route path, but not in `read_thing` signature
FAST003.py:34:18: FAST003 [*] Parameter `author_name` appears in route path, but not in `read_thing` signature
|
32 | @app.get("/books/{author_name}/{title}")
34 | @app.get("/books/{author_name}/{title}")
| ^^^^^^^^^^^^^ FAST003
33 | async def read_thing():
34 | ...
35 | async def read_thing():
36 | ...
|
= help: Add `author_name` to function signature
Unsafe fix
30 30 |
31 31 |
32 32 | @app.get("/books/{author_name}/{title}")
33 |-async def read_thing():
33 |+async def read_thing(author_name):
34 34 | ...
35 35 |
36 36 |
32 32 |
33 33 |
34 34 | @app.get("/books/{author_name}/{title}")
35 |-async def read_thing():
35 |+async def read_thing(author_name):
36 36 | ...
37 37 |
38 38 |
FAST003.py:32:32: FAST003 [*] Parameter `title` appears in route path, but not in `read_thing` signature
FAST003.py:34:32: FAST003 [*] Parameter `title` appears in route path, but not in `read_thing` signature
|
32 | @app.get("/books/{author_name}/{title}")
34 | @app.get("/books/{author_name}/{title}")
| ^^^^^^^ FAST003
33 | async def read_thing():
34 | ...
35 | async def read_thing():
36 | ...
|
= help: Add `title` to function signature
Unsafe fix
30 30 |
31 31 |
32 32 | @app.get("/books/{author_name}/{title}")
33 |-async def read_thing():
33 |+async def read_thing(title):
34 34 | ...
35 35 |
36 36 |
32 32 |
33 33 |
34 34 | @app.get("/books/{author_name}/{title}")
35 |-async def read_thing():
35 |+async def read_thing(title):
36 36 | ...
37 37 |
38 38 |
FAST003.py:37:18: FAST003 Parameter `author` appears in route path, but only as a positional-only argument in `read_thing` signature
FAST003.py:39:18: FAST003 Parameter `author` appears in route path, but only as a positional-only argument in `read_thing` signature
|
37 | @app.get("/books/{author}/{title}")
39 | @app.get("/books/{author}/{title}")
| ^^^^^^^^ FAST003
38 | async def read_thing(author: str, title: str, /):
39 | return {"author": author, "title": title}
40 | async def read_thing(author: str, title: str, /):
41 | return {"author": author, "title": title}
|
FAST003.py:37:27: FAST003 Parameter `title` appears in route path, but only as a positional-only argument in `read_thing` signature
FAST003.py:39:27: FAST003 Parameter `title` appears in route path, but only as a positional-only argument in `read_thing` signature
|
37 | @app.get("/books/{author}/{title}")
39 | @app.get("/books/{author}/{title}")
| ^^^^^^^ FAST003
38 | async def read_thing(author: str, title: str, /):
39 | return {"author": author, "title": title}
40 | async def read_thing(author: str, title: str, /):
41 | return {"author": author, "title": title}
|
FAST003.py:42:27: FAST003 [*] Parameter `title` appears in route path, but not in `read_thing` signature
FAST003.py:44:27: FAST003 [*] Parameter `title` appears in route path, but not in `read_thing` signature
|
42 | @app.get("/books/{author}/{title}/{page}")
44 | @app.get("/books/{author}/{title}/{page}")
| ^^^^^^^ FAST003
43 | async def read_thing(
44 | author: str,
45 | async def read_thing(
46 | author: str,
|
= help: Add `title` to function signature
Unsafe fix
42 42 | @app.get("/books/{author}/{title}/{page}")
43 43 | async def read_thing(
44 44 | author: str,
45 |- query: str,
45 |+ query: str, title,
46 46 | ): ...
47 47 |
48 48 |
44 44 | @app.get("/books/{author}/{title}/{page}")
45 45 | async def read_thing(
46 46 | author: str,
47 |- query: str,
47 |+ query: str, title,
48 48 | ): ...
49 49 |
50 50 |
FAST003.py:42:35: FAST003 [*] Parameter `page` appears in route path, but not in `read_thing` signature
FAST003.py:44:35: FAST003 [*] Parameter `page` appears in route path, but not in `read_thing` signature
|
42 | @app.get("/books/{author}/{title}/{page}")
44 | @app.get("/books/{author}/{title}/{page}")
| ^^^^^^ FAST003
43 | async def read_thing(
44 | author: str,
45 | async def read_thing(
46 | author: str,
|
= help: Add `page` to function signature
Unsafe fix
42 42 | @app.get("/books/{author}/{title}/{page}")
43 43 | async def read_thing(
44 44 | author: str,
45 |- query: str,
45 |+ query: str, page,
46 46 | ): ...
47 47 |
48 48 |
44 44 | @app.get("/books/{author}/{title}/{page}")
45 45 | async def read_thing(
46 46 | author: str,
47 |- query: str,
47 |+ query: str, page,
48 48 | ): ...
49 49 |
50 50 |
FAST003.py:49:18: FAST003 [*] Parameter `author` appears in route path, but not in `read_thing` signature
FAST003.py:51:18: FAST003 [*] Parameter `author` appears in route path, but not in `read_thing` signature
|
49 | @app.get("/books/{author}/{title}")
51 | @app.get("/books/{author}/{title}")
| ^^^^^^^^ FAST003
50 | async def read_thing():
51 | ...
52 | async def read_thing():
53 | ...
|
= help: Add `author` to function signature
Unsafe fix
47 47 |
48 48 |
49 49 | @app.get("/books/{author}/{title}")
50 |-async def read_thing():
50 |+async def read_thing(author):
51 51 | ...
52 52 |
53 53 |
49 49 |
50 50 |
51 51 | @app.get("/books/{author}/{title}")
52 |-async def read_thing():
52 |+async def read_thing(author):
53 53 | ...
54 54 |
55 55 |
FAST003.py:49:27: FAST003 [*] Parameter `title` appears in route path, but not in `read_thing` signature
FAST003.py:51:27: FAST003 [*] Parameter `title` appears in route path, but not in `read_thing` signature
|
49 | @app.get("/books/{author}/{title}")
51 | @app.get("/books/{author}/{title}")
| ^^^^^^^ FAST003
50 | async def read_thing():
51 | ...
52 | async def read_thing():
53 | ...
|
= help: Add `title` to function signature
Unsafe fix
47 47 |
48 48 |
49 49 | @app.get("/books/{author}/{title}")
50 |-async def read_thing():
50 |+async def read_thing(title):
51 51 | ...
52 52 |
53 53 |
49 49 |
50 50 |
51 51 | @app.get("/books/{author}/{title}")
52 |-async def read_thing():
52 |+async def read_thing(title):
53 53 | ...
54 54 |
55 55 |
FAST003.py:54:27: FAST003 [*] Parameter `title` appears in route path, but not in `read_thing` signature
FAST003.py:56:27: FAST003 [*] Parameter `title` appears in route path, but not in `read_thing` signature
|
54 | @app.get("/books/{author}/{title}")
56 | @app.get("/books/{author}/{title}")
| ^^^^^^^ FAST003
55 | async def read_thing(*, author: str):
56 | ...
57 | async def read_thing(*, author: str):
58 | ...
|
= help: Add `title` to function signature
Unsafe fix
52 52 |
53 53 |
54 54 | @app.get("/books/{author}/{title}")
55 |-async def read_thing(*, author: str):
55 |+async def read_thing(title, *, author: str):
56 56 | ...
57 57 |
58 58 |
54 54 |
55 55 |
56 56 | @app.get("/books/{author}/{title}")
57 |-async def read_thing(*, author: str):
57 |+async def read_thing(title, *, author: str):
58 58 | ...
59 59 |
60 60 |
FAST003.py:59:27: FAST003 [*] Parameter `title` appears in route path, but not in `read_thing` signature
FAST003.py:61:27: FAST003 [*] Parameter `title` appears in route path, but not in `read_thing` signature
|
59 | @app.get("/books/{author}/{title}")
61 | @app.get("/books/{author}/{title}")
| ^^^^^^^ FAST003
60 | async def read_thing(hello, /, *, author: str):
61 | ...
62 | async def read_thing(hello, /, *, author: str):
63 | ...
|
= help: Add `title` to function signature
Unsafe fix
57 57 |
58 58 |
59 59 | @app.get("/books/{author}/{title}")
60 |-async def read_thing(hello, /, *, author: str):
60 |+async def read_thing(hello, /, title, *, author: str):
61 61 | ...
62 62 |
63 63 |
59 59 |
60 60 |
61 61 | @app.get("/books/{author}/{title}")
62 |-async def read_thing(hello, /, *, author: str):
62 |+async def read_thing(hello, /, title, *, author: str):
63 63 | ...
64 64 |
65 65 |
FAST003.py:64:19: FAST003 [*] Parameter `thing_id` appears in route path, but not in `read_thing` signature
FAST003.py:66:19: FAST003 [*] Parameter `thing_id` appears in route path, but not in `read_thing` signature
|
64 | @app.get("/things/{thing_id}")
66 | @app.get("/things/{thing_id}")
| ^^^^^^^^^^ FAST003
65 | async def read_thing(
66 | query: str,
67 | async def read_thing(
68 | query: str,
|
= help: Add `thing_id` to function signature
Unsafe fix
63 63 |
64 64 | @app.get("/things/{thing_id}")
65 65 | async def read_thing(
66 |- query: str,
66 |+ query: str, thing_id,
67 67 | ):
68 68 | return {"query": query}
69 69 |
65 65 |
66 66 | @app.get("/things/{thing_id}")
67 67 | async def read_thing(
68 |- query: str,
68 |+ query: str, thing_id,
69 69 | ):
70 70 | return {"query": query}
71 71 |
FAST003.py:71:19: FAST003 [*] Parameter `thing_id` appears in route path, but not in `read_thing` signature
FAST003.py:73:19: FAST003 [*] Parameter `thing_id` appears in route path, but not in `read_thing` signature
|
71 | @app.get("/things/{thing_id}")
73 | @app.get("/things/{thing_id}")
| ^^^^^^^^^^ FAST003
72 | async def read_thing(
73 | query: str = "default",
74 | async def read_thing(
75 | query: str = "default",
|
= help: Add `thing_id` to function signature
Unsafe fix
70 70 |
71 71 | @app.get("/things/{thing_id}")
72 72 | async def read_thing(
73 |- query: str = "default",
73 |+ thing_id, query: str = "default",
74 74 | ):
75 75 | return {"query": query}
76 76 |
72 72 |
73 73 | @app.get("/things/{thing_id}")
74 74 | async def read_thing(
75 |- query: str = "default",
75 |+ thing_id, query: str = "default",
76 76 | ):
77 77 | return {"query": query}
78 78 |
FAST003.py:78:19: FAST003 [*] Parameter `thing_id` appears in route path, but not in `read_thing` signature
FAST003.py:80:19: FAST003 [*] Parameter `thing_id` appears in route path, but not in `read_thing` signature
|
78 | @app.get("/things/{thing_id}")
80 | @app.get("/things/{thing_id}")
| ^^^^^^^^^^ FAST003
79 | async def read_thing(
80 | *, query: str = "default",
81 | async def read_thing(
82 | *, query: str = "default",
|
= help: Add `thing_id` to function signature
Unsafe fix
77 77 |
78 78 | @app.get("/things/{thing_id}")
79 79 | async def read_thing(
80 |- *, query: str = "default",
80 |+ thing_id, *, query: str = "default",
81 81 | ):
82 82 | return {"query": query}
83 83 |
79 79 |
80 80 | @app.get("/things/{thing_id}")
81 81 | async def read_thing(
82 |- *, query: str = "default",
82 |+ thing_id, *, query: str = "default",
83 83 | ):
84 84 | return {"query": query}
85 85 |
FAST003.py:87:18: FAST003 [*] Parameter `name` appears in route path, but not in `read_thing` signature
|
87 | @app.get("/books/{name}/{title}")
| ^^^^^^ FAST003
88 | async def read_thing(*, author: Annotated[str, Path(alias="author_name")], title: str):
89 | return {"author": author, "title": title}
|
= help: Add `name` to function signature
Unsafe fix
85 85 |
86 86 |
87 87 | @app.get("/books/{name}/{title}")
88 |-async def read_thing(*, author: Annotated[str, Path(alias="author_name")], title: str):
88 |+async def read_thing(name, *, author: Annotated[str, Path(alias="author_name")], title: str):
89 89 | return {"author": author, "title": title}
90 90 |
91 91 |

View File

@@ -30,7 +30,7 @@ use crate::registry::Rule;
/// ```console
/// Traceback (most recent call last):
/// File "tmp.py", line 2, in <module>
/// raise RuntimeError("Some value is incorrect")
/// raise RuntimeError("'Some value' is incorrect")
/// RuntimeError: 'Some value' is incorrect
/// ```
///

View File

@@ -29,7 +29,9 @@ mod tests {
#[test_case(Rule::UndocumentedParam, Path::new("sections.py"))]
#[test_case(Rule::EndsInPeriod, Path::new("D.py"))]
#[test_case(Rule::EndsInPeriod, Path::new("D400.py"))]
#[test_case(Rule::EndsInPeriod, Path::new("D400_415.py"))]
#[test_case(Rule::EndsInPunctuation, Path::new("D.py"))]
#[test_case(Rule::EndsInPunctuation, Path::new("D400_415.py"))]
#[test_case(Rule::FirstLineCapitalized, Path::new("D.py"))]
#[test_case(Rule::FirstLineCapitalized, Path::new("D403.py"))]
#[test_case(Rule::FitsOnOneLine, Path::new("D.py"))]
@@ -49,6 +51,7 @@ mod tests {
#[test_case(Rule::OverIndentation, Path::new("D.py"))]
#[test_case(Rule::OverIndentation, Path::new("D208.py"))]
#[test_case(Rule::NoSignature, Path::new("D.py"))]
#[test_case(Rule::NoSignature, Path::new("D402.py"))]
#[test_case(Rule::SurroundingWhitespace, Path::new("D.py"))]
#[test_case(Rule::DocstringStartsWithThis, Path::new("D.py"))]
#[test_case(Rule::UnderIndentation, Path::new("D.py"))]

View File

@@ -1,7 +1,7 @@
use ruff_text_size::TextLen;
use strum::IntoEnumIterator;
use ruff_diagnostics::{AlwaysFixableViolation, Diagnostic, Edit, Fix};
use ruff_diagnostics::{Diagnostic, Edit, Fix, FixAvailability, Violation};
use ruff_macros::{derive_message_formats, violation};
use ruff_source_file::{UniversalNewlineIterator, UniversalNewlines};
use ruff_text_size::Ranged;
@@ -47,14 +47,18 @@ use crate::rules::pydocstyle::helpers::logical_line;
#[violation]
pub struct EndsInPeriod;
impl AlwaysFixableViolation for EndsInPeriod {
impl Violation for EndsInPeriod {
/// `None` in the case a fix is never available or otherwise Some
/// [`FixAvailability`] describing the available fix.
const FIX_AVAILABILITY: FixAvailability = FixAvailability::Sometimes;
#[derive_message_formats]
fn message(&self) -> String {
format!("First line should end with a period")
}
fn fix_title(&self) -> String {
"Add period".to_string()
fn fix_title(&self) -> Option<String> {
Some("Add period".to_string())
}
}
@@ -104,7 +108,7 @@ pub(crate) fn ends_with_period(checker: &mut Checker, docstring: &Docstring) {
if !trimmed.ends_with('.') {
let mut diagnostic = Diagnostic::new(EndsInPeriod, docstring.range());
// Best-effort fix: avoid adding a period after other punctuation marks.
if !trimmed.ends_with([':', ';']) {
if !trimmed.ends_with([':', ';', '?', '!']) {
diagnostic.set_fix(Fix::unsafe_edit(Edit::insertion(
".".to_string(),
line.start() + trimmed.text_len(),

View File

@@ -1,7 +1,7 @@
use ruff_text_size::TextLen;
use strum::IntoEnumIterator;
use ruff_diagnostics::{AlwaysFixableViolation, Diagnostic, Edit, Fix};
use ruff_diagnostics::{Diagnostic, Edit, Fix, FixAvailability, Violation};
use ruff_macros::{derive_message_formats, violation};
use ruff_source_file::{UniversalNewlineIterator, UniversalNewlines};
use ruff_text_size::Ranged;
@@ -46,14 +46,18 @@ use crate::rules::pydocstyle::helpers::logical_line;
#[violation]
pub struct EndsInPunctuation;
impl AlwaysFixableViolation for EndsInPunctuation {
impl Violation for EndsInPunctuation {
/// `None` in the case a fix is never available or otherwise Some
/// [`FixAvailability`] describing the available fix.
const FIX_AVAILABILITY: FixAvailability = FixAvailability::Sometimes;
#[derive_message_formats]
fn message(&self) -> String {
format!("First line should end with a period, question mark, or exclamation point")
}
fn fix_title(&self) -> String {
"Add closing punctuation".to_string()
fn fix_title(&self) -> Option<String> {
Some("Add closing punctuation".to_string())
}
}

View File

@@ -66,7 +66,25 @@ pub(crate) fn no_signature(checker: &mut Checker, docstring: &Docstring) {
// a function named `foo`).
if first_line
.match_indices(function.name.as_str())
.any(|(index, _)| first_line[index + function.name.len()..].starts_with('('))
.any(|(index, _)| {
// The function name must be preceded by a word boundary.
let preceded_by_word_boundary = first_line[..index]
.chars()
.next_back()
.map_or(true, |c| matches!(c, ' ' | '\t' | ';' | ','));
if !preceded_by_word_boundary {
return false;
}
// The function name must be followed by an open parenthesis.
let followed_by_open_parenthesis =
first_line[index + function.name.len()..].starts_with('(');
if !followed_by_open_parenthesis {
return false;
}
true
})
{
checker
.diagnostics

View File

@@ -194,7 +194,7 @@ D.py:487:5: D400 [*] First line should end with a period
489 489 |
490 490 |
D.py:514:5: D400 [*] First line should end with a period
D.py:514:5: D400 First line should end with a period
|
513 | def valid_google_string(): # noqa: D400
514 | """Test a valid something!"""
@@ -202,16 +202,6 @@ D.py:514:5: D400 [*] First line should end with a period
|
= help: Add period
Unsafe fix
511 511 |
512 512 |
513 513 | def valid_google_string(): # noqa: D400
514 |- """Test a valid something!"""
514 |+ """Test a valid something!."""
515 515 |
516 516 |
517 517 | @expect("D415: First line should end with a period, question mark, "
D.py:520:5: D400 [*] First line should end with a period
|
518 | "or exclamation point (not 'g')")
@@ -328,6 +318,4 @@ D.py:664:5: D400 [*] First line should end with a period
665 |+ but continuations shouldn't be considered multi-line."
666 666 |
667 667 |
668 668 |
668 668 |

View File

@@ -0,0 +1,55 @@
---
source: crates/ruff_linter/src/rules/pydocstyle/mod.rs
---
D400_415.py:2:5: D400 First line should end with a period
|
1 | def f():
2 | "Here's a line ending in a question mark?"
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ D400
3 | ...
|
= help: Add period
D400_415.py:7:5: D400 First line should end with a period
|
6 | def f():
7 | """Here's a line ending in an exclamation mark!"""
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ D400
8 | ...
|
= help: Add period
D400_415.py:11:5: D400 First line should end with a period
|
10 | def f():
11 | """Here's a line ending in a colon:"""
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ D400
12 | ...
|
= help: Add period
D400_415.py:15:5: D400 First line should end with a period
|
14 | def f():
15 | """Here's a line ending in a semi colon;"""
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ D400
16 | ...
|
= help: Add period
D400_415.py:19:5: D400 [*] First line should end with a period
|
18 | def f():
19 | """Here's a line ending with a whitespace """
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ D400
20 | ...
|
= help: Add period
Unsafe fix
16 16 | ...
17 17 |
18 18 | def f():
19 |- """Here's a line ending with a whitespace """
19 |+ """Here's a line ending with a whitespace. """
20 20 | ...

View File

@@ -0,0 +1,18 @@
---
source: crates/ruff_linter/src/rules/pydocstyle/mod.rs
---
D402.py:2:5: D402 First line should not be the function's signature
|
1 | def foo():
2 | """Returns foo()."""
| ^^^^^^^^^^^^^^^^^^^^ D402
3 |
4 | def foo():
|
D402.py:8:5: D402 First line should not be the function's signature
|
7 | def foo():
8 | """"Use this function; foo()."""
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ D402
|

View File

@@ -0,0 +1,37 @@
---
source: crates/ruff_linter/src/rules/pydocstyle/mod.rs
---
D400_415.py:11:5: D415 First line should end with a period, question mark, or exclamation point
|
10 | def f():
11 | """Here's a line ending in a colon:"""
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ D415
12 | ...
|
= help: Add closing punctuation
D400_415.py:15:5: D415 First line should end with a period, question mark, or exclamation point
|
14 | def f():
15 | """Here's a line ending in a semi colon;"""
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ D415
16 | ...
|
= help: Add closing punctuation
D400_415.py:19:5: D415 [*] First line should end with a period, question mark, or exclamation point
|
18 | def f():
19 | """Here's a line ending with a whitespace """
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ D415
20 | ...
|
= help: Add closing punctuation
Unsafe fix
16 16 | ...
17 17 |
18 18 | def f():
19 |- """Here's a line ending with a whitespace """
19 |+ """Here's a line ending with a whitespace. """
20 20 | ...

View File

@@ -0,0 +1,37 @@
---
source: crates/ruff_linter/src/rules/pydocstyle/mod.rs
---
D415.py:11:5: D415 First line should end with a period, question mark, or exclamation point
|
10 | def f():
11 | """Here's a line ending in a colon:"""
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ D415
12 | ...
|
= help: Add closing punctuation
D415.py:15:5: D415 First line should end with a period, question mark, or exclamation point
|
14 | def f():
15 | """Here's a line ending in a semi colon;"""
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ D415
16 | ...
|
= help: Add closing punctuation
D415.py:19:5: D415 [*] First line should end with a period, question mark, or exclamation point
|
18 | def f():
19 | """Here's a line ending with a whitespace """
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ D415
20 | ...
|
= help: Add closing punctuation
Unsafe fix
16 16 | ...
17 17 |
18 18 | def f():
19 |- """Here's a line ending with a whitespace """
19 |+ """Here's a line ending with a whitespace. """
20 20 | ...

View File

@@ -11,7 +11,8 @@ use ruff_text_size::{Ranged, TextLen};
/// the string to a slice after checking `.startswith()` or `.endswith()`, respectively.
///
/// ## Why is this bad?
/// The methods [`str.removeprefix`] and [`str.removesuffix`],
/// The methods [`str.removeprefix`](https://docs.python.org/3/library/stdtypes.html#str.removeprefix)
/// and [`str.removesuffix`](https://docs.python.org/3/library/stdtypes.html#str.removesuffix),
/// introduced in Python 3.9, have the same behavior
/// and are more readable and efficient.
///
@@ -33,9 +34,6 @@ use ruff_text_size::{Ranged, TextLen};
/// ```python
/// text = text.removeprefix("pre")
/// ```
///
/// [`str.removeprefix`]: https://docs.python.org/3/library/stdtypes.html#str.removeprefix
/// [`str.removesuffix`]: https://docs.python.org/3/library/stdtypes.html#str.removesuffix
#[violation]
pub struct SliceToRemovePrefixOrSuffix {
string: String,
@@ -248,6 +246,27 @@ fn affix_removal_data<'a>(
return None;
}
let slice = slice.as_slice_expr()?;
// Exit early if slice step is...
if slice
.step
.as_deref()
// present and
.is_some_and(|step| match step {
// not equal to 1
ast::Expr::NumberLiteral(ast::ExprNumberLiteral {
value: ast::Number::Int(x),
..
}) => x.as_u8() != Some(1),
// and not equal to `None` or `True`
ast::Expr::NoneLiteral(_)
| ast::Expr::BooleanLiteral(ast::ExprBooleanLiteral { value: true, .. }) => false,
_ => true,
})
{
return None;
};
let compr_test_expr = ast::comparable::ComparableExpr::from(
&test.as_call_expr()?.func.as_attribute_expr()?.value,
);

View File

@@ -166,6 +166,8 @@ FURB188.py:154:12: FURB188 [*] Prefer `removesuffix` over conditionally replacin
153 |
154 | return filename[:-builtins_len(extension)] if filename.endswith(extension) else filename
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ FURB188
155 |
156 | def okay_steps():
|
= help: Use removesuffix instead of ternary expression conditional upon endswith.
@@ -175,3 +177,77 @@ FURB188.py:154:12: FURB188 [*] Prefer `removesuffix` over conditionally replacin
153 153 |
154 |- return filename[:-builtins_len(extension)] if filename.endswith(extension) else filename
154 |+ return filename.removesuffix(extension)
155 155 |
156 156 | def okay_steps():
157 157 | text = "!x!y!z"
FURB188.py:158:5: FURB188 [*] Prefer `removeprefix` over conditionally replacing with slice.
|
156 | def okay_steps():
157 | text = "!x!y!z"
158 | if text.startswith("!"):
| _____^
159 | | text = text[1::1]
| |_________________________^ FURB188
160 | if text.startswith("!"):
161 | text = text[1::True]
|
= help: Use removeprefix instead of assignment conditional upon startswith.
Safe fix
155 155 |
156 156 | def okay_steps():
157 157 | text = "!x!y!z"
158 |- if text.startswith("!"):
159 |- text = text[1::1]
158 |+ text = text.removeprefix("!")
160 159 | if text.startswith("!"):
161 160 | text = text[1::True]
162 161 | if text.startswith("!"):
FURB188.py:160:5: FURB188 [*] Prefer `removeprefix` over conditionally replacing with slice.
|
158 | if text.startswith("!"):
159 | text = text[1::1]
160 | if text.startswith("!"):
| _____^
161 | | text = text[1::True]
| |____________________________^ FURB188
162 | if text.startswith("!"):
163 | text = text[1::None]
|
= help: Use removeprefix instead of assignment conditional upon startswith.
Safe fix
157 157 | text = "!x!y!z"
158 158 | if text.startswith("!"):
159 159 | text = text[1::1]
160 |- if text.startswith("!"):
161 |- text = text[1::True]
160 |+ text = text.removeprefix("!")
162 161 | if text.startswith("!"):
163 162 | text = text[1::None]
164 163 | print(text)
FURB188.py:162:5: FURB188 [*] Prefer `removeprefix` over conditionally replacing with slice.
|
160 | if text.startswith("!"):
161 | text = text[1::True]
162 | if text.startswith("!"):
| _____^
163 | | text = text[1::None]
| |____________________________^ FURB188
164 | print(text)
|
= help: Use removeprefix instead of assignment conditional upon startswith.
Safe fix
159 159 | text = text[1::1]
160 160 | if text.startswith("!"):
161 161 | text = text[1::True]
162 |- if text.startswith("!"):
163 |- text = text[1::None]
162 |+ text = text.removeprefix("!")
164 163 | print(text)
165 164 |
166 165 |

View File

@@ -61,7 +61,7 @@ pub struct MissingFStringSyntax;
impl AlwaysFixableViolation for MissingFStringSyntax {
#[derive_message_formats]
fn message(&self) -> String {
format!(r#"Possible f-string without an `f` prefix"#)
format!(r"Possible f-string without an `f` prefix")
}
fn fix_title(&self) -> String {

View File

@@ -285,7 +285,7 @@ impl Display for LinterSettings {
self.target_version | debug,
self.preview,
self.explicit_preview_rules,
self.extension | nested,
self.extension | debug,
self.allowed_confusables | array,
self.builtins | array,

View File

@@ -478,46 +478,31 @@ impl From<ExtensionPair> for (String, Language) {
(value.extension, value.language)
}
}
#[derive(Debug, Clone, Default, CacheKey)]
pub struct ExtensionMapping {
mapping: FxHashMap<String, Language>,
}
pub struct ExtensionMapping(FxHashMap<String, Language>);
impl ExtensionMapping {
/// Return the [`Language`] for the given file.
pub fn get(&self, path: &Path) -> Option<Language> {
let ext = path.extension()?.to_str()?;
self.mapping.get(ext).copied()
}
}
impl Display for ExtensionMapping {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
display_settings! {
formatter = f,
namespace = "linter.extension",
fields = [
self.mapping | debug
]
}
Ok(())
self.0.get(ext).copied()
}
}
impl From<FxHashMap<String, Language>> for ExtensionMapping {
fn from(value: FxHashMap<String, Language>) -> Self {
Self { mapping: value }
Self(value)
}
}
impl FromIterator<ExtensionPair> for ExtensionMapping {
fn from_iter<T: IntoIterator<Item = ExtensionPair>>(iter: T) -> Self {
Self {
mapping: iter
.into_iter()
Self(
iter.into_iter()
.map(|pair| (pair.extension, pair.language))
.collect(),
}
)
}
}

View File

@@ -0,0 +1 @@
{"target_version": "py310"}

View File

@@ -0,0 +1,10 @@
class Plotter:
\
pass
class AnotherCase:
\
"""Some
\
Docstring
"""

View File

@@ -0,0 +1,10 @@
class Plotter:
pass
class AnotherCase:
"""Some
\
Docstring
"""

View File

@@ -2,5 +2,7 @@ def bob(): # pylint: disable=W9016
pass
def bobtwo(): # some comment here
def bobtwo():
# some comment here
pass

View File

@@ -67,3 +67,63 @@ async def async_function(self):
@decorated
async def async_function(self):
...
class ClassA:
def f(self):
...
class ClassB:
def f(self):
...
class ClassC:
def f(self):
...
# Comment
class ClassD:
def f(self):# Comment 1
...# Comment 2
# Comment 3
class ClassE:
def f(self):
...
def f2(self):
print(10)
class ClassF:
def f(self):
...# Comment 2
class ClassG:
def f(self):#Comment 1
...# Comment 2
class ClassH:
def f(self):
#Comment
...

View File

@@ -70,3 +70,47 @@ async def async_function(self): ...
@decorated
async def async_function(self): ...
class ClassA:
def f(self): ...
class ClassB:
def f(self): ...
class ClassC:
def f(self):
...
# Comment
class ClassD:
def f(self): # Comment 1
... # Comment 2
# Comment 3
class ClassE:
def f(self): ...
def f2(self):
print(10)
class ClassF:
def f(self): ... # Comment 2
class ClassG:
def f(self): # Comment 1
... # Comment 2
class ClassH:
def f(self):
# Comment
...

Some files were not shown because too many files have changed in this diff Show More