Compare commits

...

216 Commits

Author SHA1 Message Date
konstin
0f2979ed66 Formatter: Show preceding, following and enclosing of comments
**Summary** I used to always add `dbg!` for preceding, following and enclosing. With this change `--print-comments` can do this instead.
```python
import re  # import

def to_camel_case(node: str) -> str:
    """Converts PascalCase to camel_case"""
    return re.sub("([A-Z])", r"_\1", node).lower().lstrip("_")

# a
if True:
    pass  # b
else:
    print()
```
Debug output:
```
11..19 Some((StmtImport, 0..9)) Some((StmtFunctionDef, 22..165)) (ModModule, 0..213) "# import"
168..171 Some((StmtFunctionDef, 22..165)) Some((StmtIf, 172..212)) (ModModule, 0..213) "# a"
191..194 Some((StmtPass, 185..189)) Some((ElifElseClause, 195..212)) (StmtIf, 172..212) "# b"
```

**Test Plan** n/a
2023-08-23 12:55:24 +02:00
David Szotten
fe97a2a302 Fix panic with empty attribute inner comment (#6332)
Fixes https://github.com/astral-sh/ruff/issues/6181
2023-08-04 11:59:55 +02:00
konsti
a48d16e025 Replace Formatter<PyFormatContext<'_>> with PyFormatter (#6330)
This is a refactoring to use the type alias in more places. In the
process, I had to fix and run generate.py. There are no functional
changes.
2023-08-04 10:48:58 +02:00
Charlie Marsh
8a5bc93fdd Make the Nodes vector generic on node type (#6328) 2023-08-04 03:57:15 +00:00
Charlie Marsh
6da527170f Match left-hand side types() call in types-comparison (#6326)
Follow-up to https://github.com/astral-sh/ruff/pull/6325, to avoid false
positives in cases like:

```python
if x == int:
    ...
```

Which is valid, since we don't know that we're comparing the type _of_
something -- we're comparing the type objects directly.
2023-08-03 23:01:23 -04:00
Charlie Marsh
8cddb6c08d Include comparisons to builtin types in type-comparison rule (#6325)
## Summary

Extends `type-comparison` to flag:

```python
if type(obj) is int:
    pass
```

In addition to the existing cases, like:

```python
if type(obj) is type(1):
    pass
```

Closes https://github.com/astral-sh/ruff/issues/6260.
2023-08-04 02:25:19 +00:00
Victor Hugo Gomes
b8ca220eeb [flake8-pyi] Implement PYI055 (#6316) 2023-08-04 01:36:00 +00:00
Charlie Marsh
1d8759d5df Generalize comment-after-bracket handling to lists, sets, etc. (#6320)
## Summary

We already support preserving the end-of-line comment in calls and type
parameters, as in:

```python
foo(  # comment
    bar,
)
```

This PR adds the same behavior for lists, sets, comprehensions, etc.,
such that we preserve:

```python
[  # comment
    1,
    2,
    3,
]
```

And related cases.
2023-08-04 01:28:05 +00:00
Charlie Marsh
d3aa8b4ee0 Add API to chain comment placement operations (#6319)
## Summary

This PR adds an API for chaining comment placement methods based on the
[`then_with`](https://doc.rust-lang.org/std/cmp/enum.Ordering.html#method.then_with)
from `Ordering` in the standard library.

For example, you can now do:

```rust
try_some_case(comment).then_with(|comment| try_some_other_case_if_still_default(comment))
```

This lets us avoid this kind of pattern, which I've seen in
`placement.rs` and used myself before:

```rust
let comment = match handle_own_line_comment_between_branches(comment, preceding, locator) {
    CommentPlacement::Default(comment) => comment,
    placement => return placement,
};
```
2023-08-03 21:08:50 -04:00
Zanie Blue
9ae498595c Upgrade Rust to 1.71 (#6323)
Addresses
[CVE-2023-38497](https://blog.rust-lang.org/2023/08/03/cve-2023-38497.html)

See also the [version release
post](https://blog.rust-lang.org/2023/08/03/Rust-1.71.1.html)
2023-08-03 21:08:39 -04:00
Charlie Marsh
5f225b18ab Generalize bracketed end-of-line comment handling (#6315)
Micha suggested this in
https://github.com/astral-sh/ruff/pull/6274#discussion_r1282774151, and
it allows us to unify the implementations for arguments and type params.
2023-08-03 20:51:03 +00:00
Charlie Marsh
8276b26480 Omit formatter PRs from releases by default (#6317)
I end up removing these manually every time, seems easier to just omit
them for now.
2023-08-03 20:45:50 +00:00
Charlie Marsh
1705fcef36 Mark trailing comments in parenthesized tests (#6287)
## Summary

This ensures that we treat `# comment` as parenthesized in contexts
like:

```python
while (
    True
    # comment
):
    pass
```

The same logic applies equally to `for`, `async for`, `if`, `with`, and
`async with`. The general pattern is that you have an expression which
precedes a colon-separated suite.
2023-08-03 20:45:03 +00:00
konsti
51ff98f9e9 Make formatter ecosystem check failure output better understandable (#6300)
**Summary** Prompted by
https://github.com/astral-sh/ruff/pull/6257#issuecomment-1661308410, it
tried to make the ecosystem script output on failure better
understandable. All log messages are now written to a file, which is
printed on error. Running locally progress is still shown.

Looking through the log output i saw that we currently log syntax errors
in input, which is confusing because they aren't actual errors, but we
don't check that these files don't change due to parser regressions or
improvements. I added `--files-with-errors` to catch that.

**Test Plan** CI
2023-08-03 20:23:25 +02:00
Charlie Marsh
b3f3529499 Improve comments around Arguments handling in classes (#6310)
## Summary

Based on the confusion here:
https://github.com/astral-sh/ruff/pull/6274#discussion_r1282754515.

I looked into moving this logic into `placement.rs`, but I think it's
trickier than it may appear.
2023-08-03 12:34:03 -04:00
Charlie Marsh
2fa508793f Return a slice in StmtClassDef#bases (#6311)
Slices are strictly more flexible, since you can always convert to an
iterator, etc., but not the other way around. Suggested in
https://github.com/astral-sh/ruff/pull/6259#discussion_r1282730994.
2023-08-03 16:21:55 +00:00
Zanie Blue
718e3945e3 Add rule to upgrade type alias annotations to keyword (UP040) (#6289)
Adds rule to convert type aliases defined with annotations i.e. `x:
TypeAlias = int` to the new PEP-695 syntax e.g. `type x = int`.

Does not support using new generic syntax for type variables, will be
addressed in a follow-up.
Added as part of pyupgrade — ~the code 100 as chosen to avoid collision
with real pyupgrade codes~.

Part of #4617 
Builds on #5062
2023-08-03 16:13:06 +00:00
Charlie Marsh
c75e8a8dab Move ExprCall's NeedsParentheses impl into expr_call.rs (#6309)
Accidental move.
2023-08-03 16:01:01 +00:00
Harutaka Kawamura
74e734e962 More precise invalid expression check for UP032 (#6308) 2023-08-03 15:49:02 +00:00
Zanie Blue
0e18abcf95 Add is_ and is_not to excluded functions for FBT003 (#6307)
These methods are commonly used in SQLAlchemy.

See https://github.com/astral-sh/ruff/discussions/6302
2023-08-03 10:41:45 -05:00
Anders Kaseorg
7c8bcede5b Broaden appropriate flake8-pyi rules to check non-stub code too (#6297)
Of the rules that flake8-pyi enforces for `.pyi` type stubs, many of
them equally make sense to check in normal runtime code with type
annotations. Broaden these rules to check all files:

PYI013 ellipsis-in-non-empty-class-body
PYI016 duplicate-union-member
PYI018 unused-private-type-var
PYI019 custom-type-var-return-type
PYI024 collections-named-tuple
PYI025 unaliased-collections-abc-set-import
PYI030 unnecessary-literal-union
PYI032 any-eq-ne-annotation
PYI034 non-self-return-type
PYI036 bad-exit-annotation
PYI041 redundant-numeric-union
PYI042 snake-case-type-alias
PYI043 t-suffixed-type-alias
PYI045 iter-method-return-iterable
PYI046 unused-private-protocol
PYI047 unused-private-type-alias
PYI049 unused-private-typed-dict
PYI050 no-return-argument-annotation-in-stub (Python ≥ 3.11)
PYI051 redundant-literal-union
PYI056 unsupported-method-call-on-all

The other rules are stub-specific and remain enabled only in `.pyi`
files.

PYI001 unprefixed-type-param
PYI002 complex-if-statement-in-stub
PYI003 unrecognized-version-info-check
PYI004 patch-version-comparison
PYI005 wrong-tuple-length-version-comparison (could make sense to
broaden, see
https://github.com/astral-sh/ruff/pull/6297#issuecomment-1663314807)
PYI006 bad-version-info-comparison (same)
PYI007 unrecognized-platform-check
PYI008 unrecognized-platform-name
PYI009 pass-statement-stub-body
PYI010 non-empty-stub-body
PYI011 typed-argument-default-in-stub
PYI012 pass-in-class-body
PYI014 argument-default-in-stub
PYI015 assignment-default-in-stub
PYI017 complex-assignment-in-stub
PYI020 quoted-annotation-in-stub
PYI021 docstring-in-stub
PYI026 type-alias-without-annotation (could make sense to broaden, but
gives many false positives on runtime code as currently implemented)
PYI029 str-or-repr-defined-in-stub
PYI033 type-comment-in-stub
PYI035 unassigned-special-variable-in-stub
PYI044 future-annotations-in-stub
PYI048 stub-body-multiple-statements
PYI052 unannotated-assignment-in-stub
PYI053 string-or-bytes-too-long
PYI054 numeric-literal-too-long

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2023-08-03 11:40:42 -04:00
Harutaka Kawamura
30c2e9430e Update UP032 to support await expressions (#6304)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

In Python >= 3.7, `await` can be included in f-strings. 

https://bugs.python.org/issue28942

## Test Plan

<!-- How was it tested? -->

Existing tests
2023-08-03 09:53:36 -05:00
Harutaka Kawamura
b6f0316d55 Add PT013 and PT015 docs (#6303)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

#2646

## Test Plan

<!-- How was it tested? -->
2023-08-03 09:51:52 -05:00
Charlie Marsh
9f3567dea6 Use range: _ in lieu of range: _range (#6296)
## Summary

`range: _range` is slightly inconvenient because you can't use it
multiple times within a single match, unlike `_`.
2023-08-02 22:11:13 -04:00
Charlie Marsh
9e2bbf4beb Add a simple tooltip to the sidebar (#6295)
## Summary

Not perfect, but IMO helpful:

<img width="1792" alt="Screen Shot 2023-08-02 at 9 29 24 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/e613e918-75cb-475e-9ea4-f833d1a0b5f6">

<img width="1792" alt="Screen Shot 2023-08-02 at 9 29 20 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/bb3efdfe-40e1-45b5-b774-082521b2d214">
2023-08-03 01:41:07 +00:00
Charlie Marsh
d7627c398c Add an icon for FIR (#6292)
It's not a very _good_ icon, but I prefer the consistency. I'm also
going to add tooltips to these.
2023-08-03 01:20:00 +00:00
Charlie Marsh
23e527e386 Increase icon opacity on-hover (#6291)
## Summary

Makes it clearer that these are clickable.
2023-08-03 01:05:38 +00:00
Charlie Marsh
a15b0a9102 Tweak background on theme button (#6290)
## Summary

It's now white on-hover as opposed to yellow, to match the copy button:

<img width="1792" alt="Screen Shot 2023-08-02 at 8 52 10 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/96d5cbf9-ef33-4fba-8888-f2a4af9a6ec4">
2023-08-03 01:00:37 +00:00
qdegraaf
d40597a266 [flake8-pyi] Implement custom_type_var_return_type (PYI019) (#6204)
## Summary

Implements `Y019` from
[flake8-pyi](https://github.com/PyCQA/flake8-pyi).

The rule checks if

-  instance methods that return `self` 
-  class methods that return an instance of `cls`
- `__new__` methods

Return a custom `TypeVar` instead of `typing.Self` and raises a
violation if this is the case. The rule also covers
[PEP-695](https://peps.python.org/pep-0695/) syntax as introduced in
upstream in https://github.com/PyCQA/flake8-pyi/pull/402

## Test Plan

Added fixtures with test cases from upstream implementation (plus
additional one for an excluded edge case, mentioned in upstream
implementation)
2023-08-03 00:42:42 +00:00
Silvano Cerza
82410524d9 [pylint] Implement Pylint bad-format-character (E1300) (#6171)
## Summary

Relates to #970.

Add new `bad-format-character` Pylint rule.

I had to make a change in `crates/ruff_python_literal/src/format.rs` to
get a more detailed error in case the format character is not correct. I
chose to do this since most of the format spec parsing functions are
private. It would have required me reimplementing most of the parsing
logic just to know if the format char was correct.

This PR also doesn't reflect current Pylint functionality in two ways.

It supports new format strings correctly, Pylint as of now doesn't. See
pylint-dev/pylint#6085.

In case there are multiple adjacent string literals delimited by
whitespace the index of the wrong format char will relative to the
single string. Pylint will instead reported it relative to the
concatenated string.

Given this:
```
"%s" "%z" % ("hello", "world")
```

Ruff will report this:
```Unsupported format character 'z' (0x7a) at index 1```

Pylint instead:
```Unsupported format character 'z' (0x7a) at index 3```

I believe it's more sensible to report the index relative to the
individual string.

## Test Plan

Added new snapshot and a small test in
`crates/ruff_python_literal/src/format.rs`.

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2023-08-02 21:32:43 +00:00
Zanie Blue
5b2e973fa5 Add formatting of type alias statements (#6162)
Part of #5062 
Extends https://github.com/astral-sh/ruff/pull/6161
Closes #5929
2023-08-02 20:40:32 +00:00
Zanie Blue
1a60d1e3c6 Add formatting of type parameters in class and function definitions (#6161)
Part of #5062 
Closes https://github.com/astral-sh/ruff/issues/5931

Implements formatting of a sequence of type parameters in a dedicated
struct for reuse by classes, functions, and type aliases (preparing for
#5929). Adds formatting of type parameters in class and function
definitions — previously, they were just elided.
2023-08-02 20:29:28 +00:00
Charlie Marsh
9425ed72a0 Break global and nonlocal statements over continuation lines (#6172)
## Summary

Builds on #6170 to break `global` and `nonlocal` statements, such that
we get:

```python
def f():
    global \
        analyze_featuremap_layer, \
        analyze_featuremapcompression_layer, \
        analyze_latencies_post, \
        analyze_motions_layer, \
        analyze_size_model
```

Instead of:

```python
def f():
    global analyze_featuremap_layer, analyze_featuremapcompression_layer, analyze_latencies_post, analyze_motions_layer, analyze_size_model
```

Notably, we avoid applying this formatting if the statement ends in a
comment. Otherwise, the comment would _need_ to be placed after the last
item, like:

```python
def f():
    global \
        analyze_featuremap_layer, \
        analyze_featuremapcompression_layer, \
        analyze_latencies_post, \
        analyze_motions_layer, \
        analyze_size_model  # noqa
```

To me, this seems wrong (and would break the `# noqa` comment). Ideally,
the items would be parenthesized, and the comment would be on the inner
parenthesis, like:

```python
def f():
    global (  # noqa
        analyze_featuremap_layer,
        analyze_featuremapcompression_layer,
        analyze_latencies_post,
        analyze_motions_layer,
        analyze_size_model
    )
```

But that's not valid syntax.
2023-08-02 19:55:00 +00:00
Victor Hugo Gomes
9f38dbd06e [flake8-pyi] Implement PYI051 (#6215)
## Summary
Checks for the presence of redundant `Literal` types and builtin super
types in an union. See [original
source](2a86db8271/pyi.py (L1261)).

This implementation has a couple of differences from the original. The
first one is, we support the `complex` and `float` builtin types. The
second is, when reporting diagnostic for a `Literal` with multiple
members of the same type, we print the entire `Literal` while `flak8`
only prints the `Literal` with its first member.
For example:
```python
from typing import Literal

x: Literal[1, 2] | int
```  
Ruff will show `Literal[1, 2]` while flake8 only shows `Literal[1]`.

```shell
$ ruff crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:4:18: PYI051 `Literal["foo"]` is redundant in an union with `str`
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:5:37: PYI051 `Literal[b"bar", b"foo"]` is redundant in an union with `bytes`
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:6:37: PYI051 `Literal[5]` is redundant in an union with `int`
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:6:67: PYI051 `Literal["foo"]` is redundant in an union with `str`
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:7:37: PYI051 `Literal[b"str_bytes"]` is redundant in an union with `bytes`
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:7:51: PYI051 `Literal[42]` is redundant in an union with `int`
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:9:31: PYI051 `Literal[1J]` is redundant in an union with `complex`
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:9:53: PYI051 `Literal[3.14]` is redundant in an union with `float`
Found 8 errors.
```

```shell
$ flake8 crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:4:18: Y051 "Literal['foo']" is redundant in a union with "str"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:5:37: Y051 "Literal[b'bar']" is redundant in a union with "bytes"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:6:37: Y051 "Literal[5]" is redundant in a unionwith "int"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:6:67: Y051 "Literal['foo']" is redundant in a union with "str"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:7:37: Y051 "Literal[b'str_bytes']" is redundantin a union with "bytes"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:7:51: Y051 "Literal[42]" is redundant in a union with "int"
```

While implementing this rule, I found a bug in the `is_unchecked_union`
check. This is the new check.


1ab86bad35/crates/ruff/src/checkers/ast/analyze/expression.rs (L85-L102)

The purpose of the check was to prevent rules from navigating through
nested `Union`s, as they already handle nested `Union`s. The way it was
implemented, this was not happening, the rules were getting executed
more than one time and sometimes were receiving expressions that were
not `Union`. For example, with the following code:
 ```python
  typing.Union[Literal[5], int, typing.Union[Literal["foo"], str]]
 ```

The rules were receiving the expressions in the following order:
- `typing.Union[Literal[5], int, typing.Union[Literal["foo"], str]]`
     - `Literal[5]`
     - `typing.Union[Literal["foo"], str]]`

This was causing `PYI030` to report redundant information, for example:
 ```python
typing.Union[Literal[5], int, typing.Union[Literal["foo"],
Literal["bar"]]]
 ```
This is the `PYI030` output for this code:
```shell
PYI030 Multiple literal members in a union. Use a single literal, e.g. `Literal[5, "foo", "bar"]`
YI030 Multiple literal members in a union. Use a single literal, e.g.`Literal[5, "foo"]`
```

If I haven't misinterpreted the rule, that looks incorrect. I didn't
have the time to check the `PYI016` rule.

The last thing is, I couldn't find a reason for the "Why is this bad?"
section for `PYI051`.

Ref: #848 

## Test Plan

Snapshots and manual runs of flake8.
\
2023-08-02 15:37:40 -04:00
Victor Hugo Gomes
7c5791fb77 Fix formatting of lambda star arguments (#6257)
## Summary
Previously, the ruff formatter was removing the star argument of
`lambda` expressions when formatting.

Given the following code snippet
```python
lambda *a: ()
lambda **b: ()
```
it would be formatted to
```python
lambda: ()
lambda: ()
```

We fix this by checking for the presence of `args`, `vararg` or `kwarg`
in the `lambda` expression, before we were only checking for the
presence of `args`.

Fixes #5894

## Test Plan

Add new tests cases.

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2023-08-02 19:31:20 +00:00
Harutaka Kawamura
c362ea7fd4 Add PT025 and PT026 docs (#6264) 2023-08-02 19:00:03 +00:00
Harutaka Kawamura
ec8fad5b02 Extend UP032 to support implicitly concatenated strings (#6263) 2023-08-02 18:56:24 +00:00
Harutaka Kawamura
bcc41ba062 Extend UP032 to support repeated format fields (#6266) 2023-08-02 14:23:25 -04:00
Charlie Marsh
556abf4bd3 Avoid PTH206 with maxsplit (#6283)
## Summary

Avoid suggesting `Path.parts` when a `maxsplit` is specified, since
these behavior differently.

## Test Plan

`cargo test`
2023-08-02 18:16:57 +00:00
Charlie Marsh
23b8fc4366 Move includes_arg_name onto Parameters (#6282)
## Summary

Like #6279, no reason for this to be a standalone method.
2023-08-02 18:05:26 +00:00
Charlie Marsh
fd40864924 Move find_keyword helpers onto Arguments struct (#6280)
## Summary

Similar to #6279, moving some helpers onto the struct in the name of
reducing the number of random undiscoverable utilities we have in
`helpers.rs`.

Most of the churn is migrating rules to take `ast::ExprCall` instead of
the spread call arguments.

## Test Plan

`cargo test`
2023-08-02 13:54:48 -04:00
Charlie Marsh
041946fb64 Remove CallArguments abstraction (#6279)
## Summary

This PR removes a now-unnecessary abstraction from `helper.rs`
(`CallArguments`), in favor of adding methods to `Arguments` directly,
which helps with discoverability.
2023-08-02 13:25:43 -04:00
Charlie Marsh
8a0f844642 Box type params and arguments fields on the class definition node (#6275)
## Summary

This PR boxes the `TypeParams` and `Arguments` fields on the class
definition node. These fields are optional and often emitted, and given
that class definition is our largest enum variant, we pay the cost of
including them for every statement in the AST. Boxing these types
reduces the statement size by 40 bytes, which seems like a good tradeoff
given how infrequently these are accessed.

## Test Plan

Need to benchmark, but no behavior changes.
2023-08-02 16:47:06 +00:00
Charlie Marsh
8c40886f87 Use Arguments node to power remove_argument method (#6278)
## Summary

Internal refactor to take advantage of the new `Arguments` node, to
power our `remove_argument` fix action.

## Test Plan

`cargo test`
2023-08-02 12:38:43 -04:00
Charlie Marsh
4c53bfe896 Add formatter support for call and class definition Arguments (#6274)
## Summary

This PR leverages the `Arguments` AST node introduced in #6259 in the
formatter, which ensures that we correctly handle trailing comments in
calls, like:

```python
f(
  1,
  # comment
)

pass
```

(Previously, this was treated as a leading comment on `pass`.)

This also allows us to unify the argument handling across calls and
class definitions.

## Test Plan

A bunch of new fixture tests, plus improved Black compatibility.
2023-08-02 11:54:22 -04:00
Charlie Marsh
b095b7204b Add a TypeParams node to the AST (#6261)
## Summary

Similar to #6259, this PR adds a `TypeParams` node to the AST, to
capture the list of type parameters with their surrounding brackets.

If a statement lacks type parameters, the `type_params` field will be
`None`.
2023-08-02 14:12:45 +00:00
Charlie Marsh
981e64f82b Introduce an Arguments AST node for function calls and class definitions (#6259)
## Summary

This PR adds a new `Arguments` AST node, which we can use for function
calls and class definitions.

The `Arguments` node spans from the left (open) to right (close)
parentheses inclusive.

In the case of classes, the `Arguments` is an option, to differentiate
between:

```python
# None
class C: ...

# Some, with empty vectors
class C(): ...
```

In this PR, we don't really leverage this change (except that a few
rules get much simpler, since we don't need to lex to find the start and
end ranges of the parentheses, e.g.,
`crates/ruff/src/rules/pyupgrade/rules/lru_cache_without_parameters.rs`,
`crates/ruff/src/rules/pyupgrade/rules/unnecessary_class_parentheses.rs`).

In future PRs, this will be especially helpful for the formatter, since
we can track comments enclosed on the node itself.

## Test Plan

`cargo test`
2023-08-02 10:01:13 -04:00
Ran Benita
0d62ad2480 Permit ClassVar and Final without subscript in RUF012 (#6273)
Fix #6267.
2023-08-02 12:58:44 +00:00
Harutaka Kawamura
b4f224ecea Fix links in docs (#6265)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

Before:

<img width="1031" alt="Screen Shot 2023-08-02 at 15 57 10"
src="https://github.com/astral-sh/ruff/assets/17039389/171a21d5-01a5-4aa5-8079-4e7f8a59ade8">

After:

<img width="1031" alt="Screen Shot 2023-08-02 at 15 58 03"
src="https://github.com/astral-sh/ruff/assets/17039389/afd1b9b7-89e0-4e38-a4a6-e3255b62f021">


## Test Plan

<!-- How was it tested? -->

Manual inspection
2023-08-02 09:42:25 +02:00
Charlie Marsh
7842c82a0a Preserve end-of-line comments on import-from statements (#6216)
## Summary

Ensures that we keep comments at the end-of-line in cases like:

```python
from foo import (  # comment
  bar,
)
```

Closes https://github.com/astral-sh/ruff/issues/6067.
2023-08-01 18:58:05 +00:00
Charlie Marsh
9c708d8fc1 Rename Parameter#arg and ParameterWithDefault#def fields (#6255)
## Summary

This PR renames...

- `Parameter#arg` to `Parameter#name`
- `ParameterWithDefault#def` to `ParameterWithDefault#parameter` (such
that `ParameterWithDefault` has a `default` and a `parameter`)

## Test Plan

`cargo test`
2023-08-01 14:28:34 -04:00
Charlie Marsh
adc8bb7821 Rename Arguments to Parameters in the AST (#6253)
## Summary

This PR renames a few AST nodes for clarity:

- `Arguments` is now `Parameters`
- `Arg` is now `Parameter`
- `ArgWithDefault` is now `ParameterWithDefault`

For now, the attribute names that reference `Parameters` directly are
changed (e.g., on `StmtFunctionDef`), but the attributes on `Parameters`
itself are not (e.g., `vararg`). We may revisit that decision in the
future.

For context, the AST node formerly known as `Arguments` is used in
function definitions. Formally (outside of the Python context),
"arguments" typically refers to "the values passed to a function", while
"parameters" typically refers to "the variables used in a function
definition". E.g., if you Google "arguments vs parameters", you'll get
some explanation like:

> A parameter is a variable in a function definition. It is a
placeholder and hence does not have a concrete value. An argument is a
value passed during function invocation.

We're thus deviating from Python's nomenclature in favor of a scheme
that we find to be more precise.
2023-08-01 13:53:28 -04:00
Charlie Marsh
a82eb9544c Implement Black's rules around newlines before and after class docstrings (#6209)
## Summary

Black allows up to one blank line _before_ a class docstring, and
enforces one blank line _after_ a class docstring. This PR implements
that handling. The cases in
`crates/ruff_python_formatter/resources/test/fixtures/ruff/statement/class_definition.py`
match Black identically.
2023-08-01 13:33:01 -04:00
Zanie Blue
5e41f2fc7d Tweak pre-commit message (#6243)
Follow-up to https://github.com/astral-sh/ruff/pull/6153
2023-08-01 12:32:14 -05:00
konsti
1df7e9831b Replace .map_or(false, $closure) with .is_some_and(closure) (#6244)
**Summary**
[Option::is_some_and](https://doc.rust-lang.org/stable/std/option/enum.Option.html#method.is_some_and)
and
[Result::is_ok_and](https://doc.rust-lang.org/std/result/enum.Result.html#method.is_ok_and)
are new methods is rust 1.70. I find them way more readable than
`.map_or(false, ...)`.

The changes are `s/.map_or(false,/.is_some_and(/g`, then manually
switching to `is_ok_and` where the value is a Result rather than an
Option.

**Test Plan** n/a^
2023-08-01 19:29:42 +02:00
Zanie Blue
2e1754e5fc Update ecosystem checks for bokeh to 3.3 (#6249)
Bokeh 3.3 is planned for release this month
(https://github.com/bokeh/bokeh/issues/13207) and is their default
branch now
2023-08-01 11:56:58 -05:00
Zanie Blue
67b88803d8 Use prettier to format yaml files in pre-commit (#6250)
Prompted by
https://github.com/astral-sh/ruff/pull/6248#discussion_r1280855848
2023-08-01 16:45:08 +00:00
konsti
ed45fcb1f7 Remove old CI comment (#6246)
We don't build abi3 wheels
2023-08-01 11:35:47 -05:00
Zanie Blue
adf227b8a9 Run ecosystem checks on changes to ecosystem test script (#6248)
e.g. https://github.com/astral-sh/ruff/pull/6245 should probably run
checks before merge
2023-08-01 11:35:29 -05:00
Micha Reiser
debfca3a11 Remove Parse trait (#6235) 2023-08-01 18:35:03 +02:00
Charlie Marsh
83fe103d6e Allow generic tuple and list calls in __all__ (#6247)
## Summary

Allows, e.g., `__all__ = list[str]()`.

Closes https://github.com/astral-sh/ruff/issues/6226.
2023-08-01 12:01:48 -04:00
Charlie Marsh
e08f873077 Add Poetry and FastAPI to ecosystem checks (#6245)
Poetry in particular would be useful to avoid issues like
https://github.com/astral-sh/ruff/issues/6233.
2023-08-01 11:48:34 -04:00
Charlie Marsh
928ab63a64 Add empty lines before nested functions and classes (#6206)
## Summary

This PR ensures that if a function or class is the first statement in a
nested suite that _isn't_ a function or class body, we insert a leading
newline.

For example, given:

```python
def f():
    if True:

        def register_type():
            pass
```

We _want_ to preserve the newline, whereas today, we remove it.

Note that this only applies when the function or class doesn't have any
leading comments.

Closes https://github.com/astral-sh/ruff/issues/6066.
2023-08-01 15:30:59 +00:00
Harutaka Kawamura
b68f76f0d9 Add pre-commit install in CONTRIBUTING.md (#6153)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

To enable the git hooks, `pre-commit install` needs to be executed.

## Test Plan

<!-- How was it tested? -->

N/A
2023-08-01 09:28:50 -05:00
Charlie Marsh
1a85953129 Don't require docstrings in .pyi files (#6239)
Closes https://github.com/astral-sh/ruff/issues/6224.
2023-08-01 10:02:57 -04:00
Charlie Marsh
743118ae9a Bump version to 0.0.282 (#6241) 2023-08-01 13:21:33 +00:00
Charlie Marsh
0753017cf1 Revert "Expand scope of quoted-annotation rule (#5766)" (#6237)
This is causing some problems, so we'll just revert for now.

Closes https://github.com/astral-sh/ruff/issues/6189.
2023-08-01 09:03:02 -04:00
Charlie Marsh
29fb655e04 Fix logger-objects documentation (#6238)
Closes https://github.com/astral-sh/ruff/issues/6234.
2023-08-01 12:57:56 +00:00
Micha Reiser
f45e8645d7 Remove unused parser modes
<!--
Thank you for contributing to Ruff! To help us out with reviewing, please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

This PR removes the `Interactive` and `FunctionType` parser modes that are unused by ruff

<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

`cargo test`

<!-- How was it tested? -->
2023-08-01 13:10:07 +02:00
Micha Reiser
7c7231db2e Remove unsupported type_comment field
<!--
Thank you for contributing to Ruff! To help us out with reviewing, please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

This PR removes the `type_comment` field which our parser doesn't support.

<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

`cargo test`

<!-- How was it tested? -->
2023-08-01 12:53:13 +02:00
Micha Reiser
4ad5903ef6 Delete type-ignore node
<!--
Thank you for contributing to Ruff! To help us out with reviewing, please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

This PR removes the type ignore node from the AST because our parser doesn't support it, and just having it around is confusing.

<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

`cargo build`

<!-- How was it tested? -->
2023-08-01 12:34:50 +02:00
konsti
c6986ac95d Consistent CommentPlacement conversion signatures (#6231)
**Summary** Allow passing any node to `CommentPlacement::{leading,
trailing, dangling}` without manually converting. Conversely, Restrict
the comment to the only type we actually pass.

**Test Plan** No changes.
2023-08-01 12:01:17 +02:00
Micha Reiser
ecfdd8d58b Add static assertions to nodes (#6228) 2023-08-01 11:54:49 +02:00
David Szotten
07468f8be9 format ExprJoinedStr (#5932) 2023-08-01 08:26:30 +02:00
David Szotten
ba990b676f add DebugText for self-documenting f-strings (#6167) 2023-08-01 07:55:03 +02:00
Harutaka Kawamura
44a8d1c644 Add PT021, PT022 and PT023 docs (#6143) 2023-08-01 00:41:54 -04:00
Charlie Marsh
88b984e885 Avoid detecting continuations at non-start-of-line (#6219)
## Summary

Previously, given:

```python
a = \
  5;
```

When detecting continuations starting at the offset of the `;`, we'd
flag the previous line as a continuation. We should only flag a
continuation if there isn't leading content prior to the offset.

Closes https://github.com/astral-sh/ruff/issues/6214
2023-08-01 00:20:29 -04:00
Charlie Marsh
bf584c6d74 Remove use of SmallVec in unnecessary-literal-union (#6221)
I prefer to use this on an as-needed basis.
2023-08-01 04:03:58 +00:00
Konrad Listwan-Ciesielski
6ea3c178fd Add DTZ002 documentation (#6146)
## Summary

Adds documentation for DTZ002. Related to
https://github.com/astral-sh/ruff/issues/2646.

## Test Plan

`python scripts/test_docs_formatted.py`
2023-08-01 04:00:50 +00:00
Charlie Marsh
764d35667f Avoid PERF401 false positive on list access in loop (#6220)
Closes https://github.com/astral-sh/ruff/issues/6210.
2023-08-01 03:56:53 +00:00
Charlie Marsh
ff9ebbaa5f Skip trivia when searching for named exception (#6218)
Closes https://github.com/astral-sh/ruff/issues/6213.
2023-08-01 03:42:30 +00:00
Micha Reiser
38b5726948 formatter: WithNodeLevel helper (#6212) 2023-07-31 21:22:17 +00:00
Charlie Marsh
615337a54d Remove newline-insertion logic from JoinNodesBuilder (#6205)
## Summary

This PR moves the "insert empty lines" behavior out of
`JoinNodesBuilder` and into the `Suite` formatter. I find it a little
confusing that the logic is split between those two formatters right
now, and since this is _only_ used in that one place, IMO it is a bit
simpler to just inline it and use a single approach to tracking state
(right now, both are stateful).

The only other place this was used was for decorators. As a side effect,
we now remove blank lines in both of these cases, which is a known but
intentional deviation from Black (which preserves the empty line before
the comment in the first case):

```python
@foo

# Hello
@bar
def baz():
    pass

@foo

@bar
def baz():
    pass
```
2023-07-31 16:58:15 -04:00
Charlie Marsh
6ee5cb37c0 Reset model state when exiting deferred visitors (#6208)
## Summary

Very subtle bug related to the AST traversal. Given:

```python
from __future__ import annotations

from logging import getLogger

__all__ = ("getLogger",)


def foo() -> None:
    pass
```

We end up visiting the `-> None` annotation, then reusing the state
snapshot when we go to visit the `__all__` exports, so when we visit
`"getLogger"`, we think we're inside of a deferred type annotation.

This PR changes all the deferred visitors to snapshot and restore the
state, which is a lot safer -- that way, the visitors avoid modifying
the current visitor state. (Previously, they implicitly left the visitor
state set to the state of the _last_ thing they visited.)

Closes https://github.com/astral-sh/ruff/issues/6207.
2023-07-31 19:46:52 +00:00
konsti
0fddb31235 Use tracing for format_dev (#6177)
## Summary

[tracing](https://github.com/tokio-rs/tracing) is library for logging,
tracing and related features that has a large ecosystem. Using
[tracing-subscriber](https://docs.rs/tracing-subscriber) and
[tracing-indicatif](https://github.com/emersonford/tracing-indicatif),
we get a nice logging output that you can configure with `RUST_LOG`
(e.g. `RUST_LOG=debug`) and a live look into the formatter progress.

Default:
![Screenshot from 2023-07-30
13-59-53](https://github.com/astral-sh/ruff/assets/6826232/6432f835-9ff1-4771-955b-398e54c406dc)

`RUST_LOG=debug`:
![Screenshot from 2023-07-30
14-01-32](https://github.com/astral-sh/ruff/assets/6826232/5f2c87da-0867-4159-82e7-b5757eebb8eb)

It's easy to see in this output which files take a disproportionate
amount of time.

[Peek 2023-07-30
14-35.webm](https://github.com/astral-sh/ruff/assets/6826232/2c92db5c-1354-465b-a6bc-ddfb281d6f9d)

It opens up further integration with the tracing ecosystem,
[tracing-timing](https://docs.rs/tracing-timing/latest/tracing_timing/)
and [tokio-console](https://github.com/tokio-rs/console) can e.g. show
histograms and the json output allows us building better pipelines than
grepping a log file.

One caveat is using `parent: None` for the logging statements because
tracing subscriber does not allow deactivating the span without
reimplementing all the other log message formatting, too, and we don't
need span information, esp. since it would currently show the progress
bar span.

## Test Plan

n/a
2023-07-31 19:14:01 +00:00
konsti
a7aa3caaae Rename formatter_progress to formatter_ecosystem_checks (#6194)
Rename the `scripts/formatter_progress.sh` to
`formatter/formatter_ecosysytem_checks.sh` since it fits the actual task
better.
2023-07-31 18:33:12 +00:00
konsti
e52b636da0 Log configuration in ruff_dev (#6193)
**Summary** This includes two changes:
 * Allow setting `-v` in `ruff_dev`, using the `ruff_cli` implementation
 * `debug!` which ruff configuration strategy was used

This is a byproduct of debugging #6187.

**Test Plan** n/a
2023-07-31 17:52:38 +00:00
konsti
9063f4524d Fix formatting of trailing unescaped quotes in raw triple quoted strings (#6202)
**Summary** This prevents us from turning `r'''\""'''` into
`r"""\"""""`, which is invalid syntax.

This PR fixes CI, which is currently broken on main (in a way that still
passes on linter PRs and allows merging formatter PRs, but it's bad to
have a job be red). Once merged, i'll make the formatted ecosystem
checks a required check.

**Test Plan** Added a regression test.
2023-07-31 19:25:16 +02:00
Charlie Marsh
dbd60b2cf5 Bump version to 0.0.281 (#6195) 2023-07-31 13:21:43 -04:00
Charlie Marsh
7eb2ba47cc Add empty line after import block (#6200)
## Summary

Ensures that, given:

```python
import os
x = 1
```

We format like:

```python
import os

x = 1
```
2023-07-31 12:01:45 -04:00
Dhruv Manilawala
cb34e6d322 Avoid parenthesizing comprehension element (#6198)
## Summary

This PR adds a new precedence level for the comprehension element. This fixes
the generator to not add parentheses around the comprehension element every
time.

The new precedence level is `COMPREHENSION_ELEMENT` and it should occur after
the `NAMED_EXPR` precedence level because named expressions are always parenthesized.

This matches the behavior of Python `ast.unparse` and tested with the
following snippet:

```python
import ast

code = ""
ast.unparse(ast.parse(code))
```

## Test Plan

Add a bunch of test cases for all the valid nodes at that position.

fixes: #5777
2023-07-31 20:56:42 +05:30
Harutaka Kawamura
0274de1fff Preserve backslash in raw string literal (#6152) 2023-07-31 12:48:17 +00:00
konsti
a540933bc9 Print log when formatter ecosystem checks fail (#6187)
**Summary** Print the errors when the formatter ecosystem checks failed.
Im not happy that we current collect the log in the first place, but
this is the less invasive change and we need it to unblock reviewing
#6152.

**Test Plan**
https://github.com/astral-sh/ruff/actions/runs/5713112075/job/15477879403?pr=6188
2023-07-31 14:45:38 +02:00
Micha Reiser
311a1f9ec4 Remove len from JoinCommaSeparatedBuilder (#6185) 2023-07-31 12:19:47 +00:00
Luc Khai Hai
b95fc6d162 Format bytes string (#6166)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

Format bytes string

Closes #6064

## Test Plan

Added a fixture based on string's one
2023-07-31 10:46:40 +02:00
Charlie Marsh
de898c52eb Avoid falsely marking non-submodules as submodule aliases (#6182)
## Summary

We have some code to ensure that if an aliased import is used, any
submodules should be marked as used too. This comment says it best:

```rust
// If the name of a submodule import is the same as an alias of another import, and the
// alias is used, then the submodule import should be marked as used too.
//
// For example, mark `pyarrow.csv` as used in:
//
// ```python
// import pyarrow as pa
// import pyarrow.csv
// print(pa.csv.read_csv("test.csv"))
// ```
```

However, it looks like when we go to look up `pyarrow` (of `import
pyarrow as pa`), we aren't checking to ensure the resolved binding is
_actually_ an import. This was causing us to attribute `print(rm.ANY)`
to `def requests_mock` here:

```python
import requests_mock as rm

def requests_mock(requests_mock: rm.Mocker):
    print(rm.ANY)
```

Closes https://github.com/astral-sh/ruff/issues/6180.
2023-07-30 22:16:25 +00:00
Charlie Marsh
76741cac77 Add global and nonlocal formatting (#6170)
## Summary

Adds `global` and `nonlocal` formatting, without the "deviation from
black" outlined in the linked issue, which I'll do separately.

See: https://github.com/astral-sh/ruff/issues/4798.

## Test Plan

Added a fixture in the Ruff-specific directory since the Black fixtures
don't seem to cover this.
2023-07-29 14:39:42 +00:00
Charlie Marsh
5d9814d84d Remove parentheses around some walrus operators (#6173)
## Summary

Closes https://github.com/astral-sh/ruff/issues/5781

## Test Plan

Added cases to
`crates/ruff_python_formatter/resources/test/fixtures/ruff/expression/named_expr.py`
one-by-one and adjusted the condition as needed.
2023-07-29 10:06:26 -04:00
Micha Reiser
1d7ad30188 CI: Update formatter dependencies (#6168) 2023-07-29 15:24:24 +02:00
Charlie Marsh
4231ed2fc3 Skip partial duplicates when applying multi-edit fixes (#6144)
## Summary

Right now, if we have two fixes that have an overlapping edit, but not
an _identical_ set of edits, they'll conflict, causing us to do another
linter traversal. Here, I've enabled the fixer to support partially
overlapping edits, which (as an example) let's us greatly reduce the
number of iterations required in the test suite.

The most common case here is that in which a bunch of edits need to
import some symbol, and then use that symbol, but in different ways. In
that case, all edits will have a common fix (to import the symbol), but
deviate in some way. With this change, we can do all of those edits in
one pass.

Note that the simplest way to enable this was to store sorted edits on
`Fix`. We don't allow modifying the edits on `Fix` once it's
constructed, so this is an easy change, and allows us to avoid a bunch
of clones and traversals later on.

Closes #5800.
2023-07-29 12:11:57 +00:00
Charlie Marsh
badbfb2d3e Skip BOM when determining Locator's line starts (#6159)
## Summary

If a file has a BOM, the import sorter _always_ reports the imports as
unsorted. The acute issue is that we detect that the line has leading
content (before the imports), which we always consider a violation.
Rather than fixing that one site, this PR instead makes `.line_start`
BOM-aware.

Fixes https://github.com/astral-sh/ruff/issues/6155.
2023-07-29 11:47:13 +00:00
Dhruv Manilawala
44bdf20221 [pep8-naming]: New config option extend-ignore-names (#6169)
## Summary

This PR adds a new config option for `pep8-naming` plugin called
`extend-ignore-names` which is used to extend the default values in
`ignore-names` option.

resolves: #6050
2023-07-29 17:11:04 +05:30
Dhruv Manilawala
3c99fbf808 Implement --diff for Jupyter Notebooks (#6149)
## Summary

Implement `--diff` for Jupyter Notebooks

## Test Plan

1. Use `crates/ruff/resources/test/fixtures/jupyter/isort.ipynb` as a
test case
and add a markdown cell in between the code cells to check that the diff
   outputs the correct cell index.
2. Run the command:
`cargo run --bin ruff --package ruff_cli -- check --no-cache --isolated
--select=ALL crates/ruff/resources/test/fixtures/jupyter/isort.ipynb
--fix --diff`

<details><summary>Example output:</summary>
<p>

```diff
--- /Users/dhruv/playground/ruff/notebooks/test.ipynb:cell 0
+++ /Users/dhruv/playground/ruff/notebooks/test.ipynb:cell 0
@@ -1,3 +0,0 @@
-from pathlib import Path
-import random
-import math
--- /Users/dhruv/playground/ruff/notebooks/test.ipynb:cell 4
+++ /Users/dhruv/playground/ruff/notebooks/test.ipynb:cell 4
@@ -1,5 +1,3 @@
-from typing import Any
-import collections
 # Newline should be added here
 def foo():
     pass

--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 8
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 8
@@ -1,8 +1,7 @@
 import pprint
 import tempfile
 
-from IPython import display
 import matplotlib.pyplot as plt
-
 import tensorflow as tf
-import tensorflow_datasets as tfds
+import tensorflow_datasets as tfds
+from IPython import display
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 10
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 10
@@ -1,5 +1,4 @@
 import tensorflow_models as tfm
 
 # These are not in the tfm public API for v2.9. They will be available in v2.10
-from official.vision.serving import export_saved_model_lib
-import official.core.train_lib
+from official.vision.serving import export_saved_model_lib
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 13
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 13
@@ -1,5 +1,5 @@
-exp_config = tfm.core.exp_factory.get_exp_config('resnet_imagenet')
-tfds_name = 'cifar10'
+exp_config = tfm.core.exp_factory.get_exp_config("resnet_imagenet")
+tfds_name = "cifar10"
 ds,ds_info = tfds.load(
 tfds_name,
 with_info=True)
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 15
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 15
@@ -6,12 +6,12 @@
 # Configure training and testing data
 batch_size = 128
 
-exp_config.task.train_data.input_path = ''
+exp_config.task.train_data.input_path = ""
 exp_config.task.train_data.tfds_name = tfds_name
-exp_config.task.train_data.tfds_split = 'train'
+exp_config.task.train_data.tfds_split = "train"
 exp_config.task.train_data.global_batch_size = batch_size
 
-exp_config.task.validation_data.input_path = ''
+exp_config.task.validation_data.input_path = ""
 exp_config.task.validation_data.tfds_name = tfds_name
-exp_config.task.validation_data.tfds_split = 'test'
+exp_config.task.validation_data.tfds_split = "test"
 exp_config.task.validation_data.global_batch_size = batch_size
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 17
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 17
@@ -1,16 +1,16 @@
 logical_device_names = [logical_device.name for logical_device in tf.config.list_logical_devices()]
 
-if 'GPU' in ''.join(logical_device_names):
-  print('This may be broken in Colab.')
-  device = 'GPU'
-elif 'TPU' in ''.join(logical_device_names):
-  print('This may be broken in Colab.')
-  device = 'TPU'
+if "GPU" in "".join(logical_device_names):
+  print("This may be broken in Colab.")
+  device = "GPU"
+elif "TPU" in "".join(logical_device_names):
+  print("This may be broken in Colab.")
+  device = "TPU"
 else:
-  print('Running on CPU is slow, so only train for a few steps.')
-  device = 'CPU'
+  print("Running on CPU is slow, so only train for a few steps.")
+  device = "CPU"
 
-if device=='CPU':
+if device=="CPU":
   train_steps = 20
   exp_config.trainer.steps_per_loop = 5
 else:
@@ -20,9 +20,9 @@
 exp_config.trainer.summary_interval = 100
 exp_config.trainer.checkpoint_interval = train_steps
 exp_config.trainer.validation_interval = 1000
-exp_config.trainer.validation_steps =  ds_info.splits['test'].num_examples // batch_size
+exp_config.trainer.validation_steps =  ds_info.splits["test"].num_examples // batch_size
 exp_config.trainer.train_steps = train_steps
-exp_config.trainer.optimizer_config.learning_rate.type = 'cosine'
+exp_config.trainer.optimizer_config.learning_rate.type = "cosine"
 exp_config.trainer.optimizer_config.learning_rate.cosine.decay_steps = train_steps
 exp_config.trainer.optimizer_config.learning_rate.cosine.initial_learning_rate = 0.1
 exp_config.trainer.optimizer_config.warmup.linear.warmup_steps = 100
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 21
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 21
@@ -1,14 +1,14 @@
 logical_device_names = [logical_device.name for logical_device in tf.config.list_logical_devices()]
 
 if exp_config.runtime.mixed_precision_dtype == tf.float16:
-    tf.keras.mixed_precision.set_global_policy('mixed_float16')
+    tf.keras.mixed_precision.set_global_policy("mixed_float16")
 
-if 'GPU' in ''.join(logical_device_names):
+if "GPU" in "".join(logical_device_names):
   distribution_strategy = tf.distribute.MirroredStrategy()
-elif 'TPU' in ''.join(logical_device_names):
+elif "TPU" in "".join(logical_device_names):
   tf.tpu.experimental.initialize_tpu_system()
-  tpu = tf.distribute.cluster_resolver.TPUClusterResolver(tpu='/device:TPU_SYSTEM:0')
+  tpu = tf.distribute.cluster_resolver.TPUClusterResolver(tpu="/device:TPU_SYSTEM:0")
   distribution_strategy = tf.distribute.experimental.TPUStrategy(tpu)
 else:
-  print('Warning: this will be really slow.')
+  print("Warning: this will be really slow.")
   distribution_strategy = tf.distribute.OneDeviceStrategy(logical_device_names[0])
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 23
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 23
@@ -1,5 +1,3 @@
 with distribution_strategy.scope():
   model_dir = tempfile.mkdtemp()
   task = tfm.core.task_factory.get_task(exp_config.task, logging_dir=model_dir)
-
-#  tf.keras.utils.plot_model(task.build_model(), show_shapes=True)
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 24
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 24
@@ -1,4 +1,4 @@
 for images, labels in task.build_inputs(exp_config.task.train_data).take(1):
   print()
-  print(f'images.shape: {str(images.shape):16}  images.dtype: {images.dtype!r}')
-  print(f'labels.shape: {str(labels.shape):16}  labels.dtype: {labels.dtype!r}')
+  print(f"images.shape: {images.shape!s:16}  images.dtype: {images.dtype!r}")
+  print(f"labels.shape: {labels.shape!s:16}  labels.dtype: {labels.dtype!r}")
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 27
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 27
@@ -1 +1 @@
-plt.hist(images.numpy().flatten());
+plt.hist(images.numpy().flatten())
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 29
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 29
@@ -1,2 +1,2 @@
-label_info = ds_info.features['label']
+label_info = ds_info.features["label"]
 label_info.int2str(1)
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 31
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 31
@@ -10,9 +10,6 @@
     if predictions is None:
       plt.title(label_info.int2str(labels[i]))
     else:
-      if labels[i] == predictions[i]:
-        color = 'g'
-      else:
-        color = 'r'
+      color = "g" if labels[i] == predictions[i] else "r"
       plt.title(label_info.int2str(predictions[i]), color=color)
     plt.axis("off")
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 35
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 35
@@ -1,3 +1,3 @@
-plt.figure(figsize=(10, 10));
+plt.figure(figsize=(10, 10))
 for images, labels in task.build_inputs(exp_config.task.validation_data).take(1):
   show_batch(images, labels)
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 37
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 37
@@ -1,7 +1,7 @@
 model, eval_logs = tfm.core.train_lib.run_experiment(
     distribution_strategy=distribution_strategy,
     task=task,
-    mode='train_and_eval',
+    mode="train_and_eval",
     params=exp_config,
     model_dir=model_dir,
     run_post_eval=True)
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 38
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 38
@@ -1 +0,0 @@
-#  tf.keras.utils.plot_model(model, show_shapes=True)
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 40
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 40
@@ -1,4 +1,4 @@
 for key, value in eval_logs.items():
     if isinstance(value, tf.Tensor):
       value = value.numpy()
-    print(f'{key:20}: {value:.3f}')
+    print(f"{key:20}: {value:.3f}")
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 42
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 42
@@ -4,5 +4,5 @@
 
 show_batch(images, labels, tf.cast(predictions, tf.int32))
 
-if device=='CPU':
-  plt.suptitle('The model was only trained for a few steps, it is not expected to do well.')
+if device=="CPU":
+  plt.suptitle("The model was only trained for a few steps, it is not expected to do well.")
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 45
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 45
@@ -1,8 +1,8 @@
 # Saving and exporting the trained model
 export_saved_model_lib.export_inference_graph(
-    input_type='image_tensor',
+    input_type="image_tensor",
     batch_size=1,
     input_image_size=[32, 32],
     params=exp_config,
     checkpoint_path=tf.train.latest_checkpoint(model_dir),
-    export_dir='./export/')
+    export_dir="./export/")
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 47
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 47
@@ -1,3 +1,3 @@
 # Importing SavedModel
-imported = tf.saved_model.load('./export/')
-model_fn = imported.signatures['serving_default']
+imported = tf.saved_model.load("./export/")
+model_fn = imported.signatures["serving_default"]
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 49
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 49
@@ -1,10 +1,10 @@
 plt.figure(figsize=(10, 10))
-for data in tfds.load('cifar10', split='test').batch(12).take(1):
+for data in tfds.load("cifar10", split="test").batch(12).take(1):
   predictions = []
-  for image in data['image']:
-    index = tf.argmax(model_fn(image[tf.newaxis, ...])['logits'], axis=1)[0]
+  for image in data["image"]:
+    index = tf.argmax(model_fn(image[tf.newaxis, ...])["logits"], axis=1)[0]
     predictions.append(index)
-  show_batch(data['image'], data['label'], predictions)
+  show_batch(data["image"], data["label"], predictions)
 
-  if device=='CPU':
-    plt.suptitle('The model was only trained for a few steps, it is not expected to do better than random.')
+  if device=="CPU":
+    plt.suptitle("The model was only trained for a few steps, it is not expected to do better than random.")

Would fix 61 errors.
```

</p>
</details> 

resolves: #4727
2023-07-29 04:22:56 +00:00
Charlie Marsh
4802c7c7d8 Avoid key-in-dict violations for self accesses (#6165)
Closes https://github.com/astral-sh/ruff/issues/6163.
2023-07-29 03:35:26 +00:00
Charlie Marsh
646ff6497c Ignore end-of-line file exemption comments (#6160)
## Summary

This PR protects against code like:

```python
from typing import Optional

import bar  # ruff: noqa
import baz

class Foo:
    x: Optional[str] = None
```

In which the user wrote `# ruff: noqa` to ignore a specific error, not
realizing that it was a file-level exemption that thus turned off all
lint rules.

Specifically, if a `# ruff: noqa` directive is not at the start of a
line, we now ignore it and warn, since this is almost certainly a
mistake.
2023-07-29 00:40:32 +00:00
Victor Hugo Gomes
e0d5c7564f [flake8-pyi] Implement PYI049 (#6136)
## Summary

Checks for the presence of unused private `typing.TypedDict`
definitions.

ref #848 

## Test Plan

Snapshots and manual runs of flake8
2023-07-29 00:34:36 +00:00
Victor Hugo Gomes
7838d8c8af Implement PYI047 (#6134)
## Summary

Checks for the presence of unused private `typing.TypeAlias`
definitions.

ref #848 

## Test Plan

Snapshots and manual runs of flake8
2023-07-29 00:21:29 +00:00
Zanie Blue
047c211837 Add semantic analysis of type aliases and parameters (#6109)
Requires https://github.com/astral-sh/RustPython-Parser/pull/42
Related https://github.com/PyCQA/pyflakes/pull/778
[PEP-695](https://peps.python.org/pep-0695)
Part of #5062 

<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->
Adds a scope for type parameters, a type parameter binding kind, and
checker visitation of type parameters in type alias statements, function
definitions, and class definitions.

A few changes were necessary to ensure correctness following the
insertion of a new scope between function and class scopes and their
parent.

## Test Plan

<!-- How was it tested? -->
Undefined name snapshots.

Unused type parameter rule will be added as follow-up.
2023-07-28 17:06:37 -05:00
Charlie Marsh
134d447d4c Avoid refactoring x[:1]-like slices in RUF015 (#6150)
## Summary

Right now, `RUF015` will try to rewrite `x[:1]` as `[next(x)]`. This
isn't equivalent if `x`, for example, is empty, where slicing like
`x[:1]` is forgiving, but `next` raises `StopIteration`. For me this is
a little too much of a deviation to be comfortable with, and most of the
value in this rule is the `x[0]` to `next(x)` conversion anyway.

Closes https://github.com/astral-sh/ruff/issues/6148.
2023-07-28 09:38:13 -04:00
Charlie Marsh
cd4147423c Skip PERF203 violations for multi-statement loops (#6145)
Closes https://github.com/astral-sh/ruff/issues/5858.
2023-07-28 04:55:55 +00:00
Charlie Marsh
d15436458f Only run unused private type rules over finalized bindings (#6142)
## Summary

In #6134 and #6136, we see some false positives for "shadowed" class
definitions. For example, here, the first definition is flagged as
unused, since from the perspective of the semantic model (which doesn't
understand branching), it appears to be immediately shadowed in the
`else`, and thus never used:

```python
if sys.version_info >= (3, 11):
    class _RootLoggerConfiguration(TypedDict, total=False):
        level: _Level
        filters: Sequence[str | _FilterType]
        handlers: Sequence[str]

else:
    class _RootLoggerConfiguration(TypedDict, total=False):
        level: _Level
        filters: Sequence[str]
        handlers: Sequence[str]
```

Instead of looking at _all_ bindings, we should instead look at the
"live" bindings, which is similar to how other rules (like unused
variables detection) is structured. We thus move the rule from
`bindings.rs` (which iterates over _all_ bindings, regardless of whether
they're shadowed) to `deferred_scopes.rs`, which iterates over all
"live" bindings once a scope has been fully analyzed.

## Test Plan

`cargo test`
2023-07-28 02:16:09 +00:00
Charlie Marsh
0bc3edf6c9 Add documentation and test cases for redefinition (#6135) 2023-07-28 00:01:42 +00:00
Aarni Koskela
3d54d31cd9 Implement E241 and E242 (tab/multiple ws after commas) (#6094)
## Summary

This PR implements pycodestyle's E241 (tab after comma) and E242
(multiple whitespace after comma) lints.

These are marked as nursery rules like many other pycodestyle rules.

Refs #2402

## Test Plan

E24.py copied from pycodestyle.
2023-07-27 18:58:41 +00:00
Tom Kuson
1418ee62f8 Add more documentation to the flake8-bandit rules (#6128)
## Summary

Completes the documentation for the ruleset, apart from four rules which
have contradictions, so need to be thought about more regarding how to
document that. Related to #2646.

## Test Plan

`python scripts/test_docs_formatted.py`
2023-07-27 18:57:45 +00:00
Harutaka Kawamura
bf987f80f4 Add PT017 and PT019 docs (#6115) 2023-07-27 18:56:34 +00:00
rembridge
bb08eea5cc missing-whitespace-around-operators comment (#6106)
**Summary**

Updated doc comments for `missing_whitespace_around_operator.rs`. Online
docs also benefit from this update.

**Test Plan**

Checked docs via
[mkdocs](389fe13c93/CONTRIBUTING.md (L267-L296))
2023-07-27 14:52:43 -04:00
Tom Kuson
d16216a2c2 Add documentation to the flynt rules (#6130)
## Summary

Completes the documentation for the one and only (current) rule in the
`flynt` ruleset. Related to #2646.

## Test Plan

`python scripts/test_docs_formatted.py`
2023-07-27 14:32:59 -04:00
Jelle van der Waa
0853004f41 [pylint] Implement eq-without-hash rule (PLW1641) (#5955)
Implement
https://pylint.pycqa.org/en/latest/user_guide/messages/warning/eq-without-hash.html
Issue https://github.com/astral-sh/ruff/issues/970

It's not enabled by default in pylint, so I guess it shouldn't in Ruff
either?
2023-07-27 18:28:44 +00:00
Harutaka Kawamura
fb5bbe30c7 Update SIM115 to cover pathlib.Path.open (#6118) 2023-07-27 14:20:52 -04:00
Charlie Marsh
dd706c7a35 Fix E211 documentation (#6133) 2023-07-27 17:19:33 +00:00
Charlie Marsh
e15b9c5572 Cache name resolutions in the semantic model (#6047)
## Summary

This PR stores the mapping from `ExprName` node to resolved `BindingId`,
which lets us skip scope lookups in `resolve_call_path`. It's enabled by
#6045, since that PR ensures that when we analyze a node (and thus call
`resolve_call_path`), we'll have already visited its `ExprName`
elements.

In more detail: imagine that we're traversing over `foo.bar()`. When we
read `foo`, it will be an `ExprName`, which we'll then resolve to a
binding via `handle_node_load`. With this change, we then store that
binding in a map. Later, if we call `collect_call_path` on `foo.bar`,
we'll identify `foo` (the "head" of the attribute) and grab the resolved
binding in that map. _Almost_ all names are now resolved in advance,
though it's not a strict requirement, and some rules break that pattern
(e.g., if we're analyzing arguments, and they need to inspect their
annotations, which are visited in a deferred manner).

This improves performance by 4-6% on the all-rules benchmark. It looks
like it hurts performance (1-2% drop) in the default-rules benchmark,
presumedly because those rules don't call `resolve_call_path` nearly as
much, and so we're paying for these extra writes.

Here's the benchmark data:

```
linter/default-rules/numpy/globals.py
                        time:   [67.270 µs 67.380 µs 67.489 µs]
                        thrpt:  [43.720 MiB/s 43.792 MiB/s 43.863 MiB/s]
                 change:
                        time:   [+0.4747% +0.7752% +1.0626%] (p = 0.00 < 0.05)
                        thrpt:  [-1.0514% -0.7693% -0.4724%]
                        Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe
linter/default-rules/pydantic/types.py
                        time:   [1.4067 ms 1.4105 ms 1.4146 ms]
                        thrpt:  [18.028 MiB/s 18.081 MiB/s 18.129 MiB/s]
                 change:
                        time:   [+1.3152% +1.6953% +2.0414%] (p = 0.00 < 0.05)
                        thrpt:  [-2.0006% -1.6671% -1.2981%]
                        Performance has regressed.
linter/default-rules/numpy/ctypeslib.py
                        time:   [637.67 µs 638.96 µs 640.28 µs]
                        thrpt:  [26.006 MiB/s 26.060 MiB/s 26.113 MiB/s]
                 change:
                        time:   [+1.5859% +1.8109% +2.0353%] (p = 0.00 < 0.05)
                        thrpt:  [-1.9947% -1.7787% -1.5611%]
                        Performance has regressed.
linter/default-rules/large/dataset.py
                        time:   [3.2289 ms 3.2336 ms 3.2383 ms]
                        thrpt:  [12.563 MiB/s 12.581 MiB/s 12.599 MiB/s]
                 change:
                        time:   [+0.8029% +0.9898% +1.1740%] (p = 0.00 < 0.05)
                        thrpt:  [-1.1604% -0.9801% -0.7965%]
                        Change within noise threshold.

linter/all-rules/numpy/globals.py
                        time:   [134.05 µs 134.15 µs 134.26 µs]
                        thrpt:  [21.977 MiB/s 21.995 MiB/s 22.012 MiB/s]
                 change:
                        time:   [-4.4571% -4.1175% -3.8268%] (p = 0.00 < 0.05)
                        thrpt:  [+3.9791% +4.2943% +4.6651%]
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  2 (2.00%) low mild
  3 (3.00%) high mild
  3 (3.00%) high severe
linter/all-rules/pydantic/types.py
                        time:   [2.5627 ms 2.5669 ms 2.5720 ms]
                        thrpt:  [9.9158 MiB/s 9.9354 MiB/s 9.9516 MiB/s]
                 change:
                        time:   [-5.8304% -5.6374% -5.4452%] (p = 0.00 < 0.05)
                        thrpt:  [+5.7587% +5.9742% +6.1914%]
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  6 (6.00%) high mild
  1 (1.00%) high severe
linter/all-rules/numpy/ctypeslib.py
                        time:   [1.3949 ms 1.3956 ms 1.3964 ms]
                        thrpt:  [11.925 MiB/s 11.931 MiB/s 11.937 MiB/s]
                 change:
                        time:   [-6.2496% -6.0856% -5.9293%] (p = 0.00 < 0.05)
                        thrpt:  [+6.3030% +6.4799% +6.6662%]
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  3 (3.00%) high mild
  4 (4.00%) high severe
linter/all-rules/large/dataset.py
                        time:   [5.5951 ms 5.6019 ms 5.6093 ms]
                        thrpt:  [7.2527 MiB/s 7.2623 MiB/s 7.2711 MiB/s]
                 change:
                        time:   [-5.1781% -4.9783% -4.8070%] (p = 0.00 < 0.05)
                        thrpt:  [+5.0497% +5.2391% +5.4608%]
                        Performance has improved.
```

Still playing with this (the concepts need better names, documentation,
etc.), but opening up for feedback.
2023-07-27 13:01:56 -04:00
qdegraaf
0638a26347 Add AnyExpressionYield to consolidate ExprYield and ExprYieldFrom (#6127)
Co-authored-by: Micha Reiser <micha@reiser.io>
2023-07-27 16:01:16 +00:00
konsti
2a65e6fc38 Explain check_docs_formatted.py error message (#6125)
## Summary

This is an error message only change to lead an implementor of a new
rule that has an unformatted or invalid bad example to the
right code.

## Test Plan

n/a
2023-07-27 10:22:13 -04:00
Charlie Marsh
13af91299d Avoid walking past root when resolving imports (#6126)
## Summary

Noticed in #5954: we walk _past_ the root rather than stopping _at_ the
root when attempting to traverse along the parent path. It's effectively
an off-by-one bug.
2023-07-27 10:22:13 -04:00
konsti
d317af442f Fix windows test warnings (#6124)
See
https://github.com/astral-sh/ruff/actions/runs/5679922286/job/15392998698.
These didn't fail CI because we run clippy on linux only.
2023-07-27 10:22:13 -04:00
Micha Reiser
6bf6646c5d Respect indent when measuring with MeasureMode::AllLines (#6120) 2023-07-27 10:22:13 -04:00
konsti
9574ff3dc7 Unbreak main (#6123)
This fixes main breaking due to two merges.
2023-07-27 10:22:13 -04:00
konsti
06d9ff9577 Don't format trailing comma for lambda arguments (#5946)
**Summary** lambda arguments don't have parentheses, so they shouldn't
get a magic trailing comma either. This fixes some unstable formatting

**Test Plan** Added a regression test.

89 (from previously 145) instances of unstable formatting remaining.

```
$ cargo run --bin ruff_dev --release -- format-dev --stability-check --error-file formatter-ecosystem-errors.txt --multi-project target/checkouts > formatter-ecosystem-progress.txt
$ rg "Unstable formatting" target/formatter-ecosystem-errors.txt | wc -l
89
```

Closes #5892
2023-07-27 10:22:13 -04:00
Micha Reiser
40f54375cb Pull in RustPython parser (#6099) 2023-07-27 09:29:11 +00:00
Victor Hugo Gomes
86539c1fc5 [flake8-pyi] Implement PYI046 (#6098)
## Summary
Checks for the presence of unused private `typing.Protocol` definitions.

ref #848 

## Test Plan

Snapshots and manual runs of flake8.
2023-07-27 02:34:56 +00:00
rembridge
d04367a042 call-datetime-without-tzinfo comment (#6105)
## Summary

Updated doc comment for `call_datetime_without_tzinfo.rs`. Online docs
also benefit from this update.

## Test Plan

Checked docs via
[mkdocs](389fe13c93/CONTRIBUTING.md (L267-L296))
2023-07-26 23:21:03 +00:00
Simon Brugman
ffdd653c54 [flake8-use-pathlib] Implement glob (PTH207) (#5939)
Discovered that the usage of `glob.glob` is
[widespread](https://grep.app/search?current=7&q=glob.glob%28&filter%5Blang%5D%5B0%5D=Python)
when working on the previous lints for `flake8-use-pathlib`.
2023-07-26 23:15:05 +00:00
rembridge
132f07c27b whitespace-before-parameters comment (#6103) 2023-07-26 23:01:47 +00:00
Victor Hugo Gomes
c0dbcb3434 [flake8-pyi] Implement PYI018 (#6018)
## Summary

Check for unused private `TypeVar`. See [original
implementation](2a86db8271/pyi.py (L1958)).

```
$ flake8 --select Y018 crates/ruff/resources/test/fixtures/flake8_pyi/PYI018.pyi

crates/ruff/resources/test/fixtures/flake8_pyi/PYI018.pyi:4:1: Y018 TypeVar "_T" is not used
crates/ruff/resources/test/fixtures/flake8_pyi/PYI018.pyi:5:1: Y018 TypeVar "_P" is not used
```

```
$ ./target/debug/ruff --select PYI018 crates/ruff/resources/test/fixtures/flake8_pyi/PYI018.pyi --no-cache

crates/ruff/resources/test/fixtures/flake8_pyi/PYI018.pyi:4:1: PYI018 TypeVar `_T` is never used
crates/ruff/resources/test/fixtures/flake8_pyi/PYI018.pyi:5:1: PYI018 TypeVar `_P` is never used
Found 2 errors.
```
In the file `unused_private_type_declaration.rs`, I'm planning to add
other rules that are similar to `PYI018` like the `PYI046`, `PYI047` and
`PYI049`.

ref #848

## Test Plan

Snapshots and manual runs of flake8.
2023-07-26 22:56:15 +00:00
Victor Hugo Gomes
788643f718 Add "--select E402" to example snippet in CONTRIBUTING.md (#6108)
## Summary
In Ruff only a subset of rules are enabled by default. This change
change aims to clarify that when adding a new rule, you must explicitly
use the `--select name_of_rule` command to ensure the rule gets
executed.

This was talked about on Discord a while back.

## Test Plan
Checked docs via mkdocs
2023-07-26 22:48:53 +00:00
Charlie Marsh
64a186272f Move utf8-encoding-declaration to token-based rules (#6110)
Closes #5979.
2023-07-26 22:42:37 +00:00
Charlie Marsh
8113615534 Add some additional documentation around import categorization (#6107)
Closes https://github.com/astral-sh/ruff/issues/5529.
2023-07-26 22:39:01 +00:00
konsti
ecf4058e52 Fix cargo test -p ruff (#6104) 2023-07-26 22:44:53 +02:00
Zanie Blue
2d2673f613 Add comment regarding class scope short circuit (#6101) 2023-07-26 14:55:05 -05:00
Harutaka Kawamura
564304eba2 Add PT001 documentation (#6023) 2023-07-26 18:05:25 +00:00
Harutaka Kawamura
5b8fc753ec Add PT024 documentation (#6026) 2023-07-26 13:48:37 -04:00
konsti
13f9a16e33 Rewrite placement logic (#6040)
## Summary
This is a rewrite of the main comment placement logic. `place_comment`
now has three parts:

- place own line comments
  - between branches
  - after a branch
- place end-of-line comments
  - after colon
  - after a branch
- place comments for specific nodes (that include module level comments)

The rewrite fixed three bugs: `class A: # trailing comment` comments now
stay end-of-line, `try: # comment` remains end-of-line and deeply
indented try-else-finally comments remain with the right nested
statement.

It will be much easier to give more alternative branches nodes since
this is abstracted away by `is_node_with_body` and the first/last child
helpers. Adding new node types can now be done by adding an entry to the
`place_comment` match. The code went from 1526 lines before #6033 to
1213 lines now.

It thinks it easier to just read the new `placement.rs` rather than
reviewing the diff.

## Test Plan

The existing fixtures staying the same or improving plus new ones for
the bug fixes.
2023-07-26 16:21:23 +00:00
Micha Reiser
2cf00fee96 Remove parser dependency from ruff-python-ast (#6096) 2023-07-26 17:47:22 +02:00
Harutaka Kawamura
99127243f4 Raise PTH201 for Path("") (#6095) 2023-07-26 09:22:46 -04:00
Harutaka Kawamura
77396c6f92 Fix SIM102 to handle indented elif (#6072)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

The `SIM102` auto-fix fails if `elif` is indented like this:

## Example

```python
def f():
    # SIM102
    if a:
        pass
    elif b:
        if c:
            d
```

```
> cargo run -p ruff_cli -- check --select SIM102 --fix a.py
...
error: Failed to fix nested if: Failed to extract statement from source
a.py:5:5: SIM102 Use a single `if` statement instead of nested `if` statements
Found 1 error.
```

## Test Plan

<!-- How was it tested? -->

New test
2023-07-26 14:37:32 +02:00
Micha Reiser
16e1737d1b Use cursor based lexer (#6012) 2023-07-26 11:32:26 +02:00
Dhruv Manilawala
025fa4eba8 Integrate the new Jupyter AST nodes in Ruff (#6086)
## Summary

This PR adds the implementation for the new Jupyter AST nodes i.e.,
`ExprLineMagic` and `StmtLineMagic`.

## Test Plan

Add test cases for `unparse` containing magic commands

resolves: #6087
2023-07-26 08:20:30 +00:00
Micha Reiser
1fdadee59c playground: Persist source and panel (#6071) 2023-07-26 07:55:59 +02:00
Charlie Marsh
c8ee357613 Remove relative import handling from BindingKind::Import case (#6084)
## Summary

Only `ImportFrom` imports can be relative, this is just unused.
2023-07-26 00:17:41 -04:00
Harutaka Kawamura
96d2ca0bda Allow pytest.raises body to contain a single func or class definition (#6083) 2023-07-25 23:45:57 -04:00
Harutaka Kawamura
62f821daaa Avoid raising PT012 for simple with statements (#6081) 2023-07-26 01:43:31 +00:00
Noah Jenner
9dfe484472 Modify PyPA classifiers and Shields.io badge URLs (#6082)
## Summary

Updated `pyproject.toml` classifiers from `"Development Status :: 4 -
Beta"` to `"Development Status :: 5 - Production/Stable"` to reflect the
transition from Beta to Full Release.
Updated the `README.md` to use `.com/astral-sh/ruff/...` instead of
`.com/charliermarsh/ruff/...` in Shields.io badges to reflect the
transition to a company.

## Test Plan

Utilized the official PyPA classifiers list (located at:
https://pypi.org/classifiers/)
Previewed the markdown file in different browsers on Github to ensure
all badges and logos still render properly.
2023-07-26 01:25:46 +00:00
Tom Kuson
da33c26238 Ignore explicit-string-concatenation on single line (#6028)
## Summary

Ignore `explicit-string-concatenation` on single line.

Closes #5332.

## Test Plan

`cargo test`
2023-07-25 19:20:29 -04:00
rembridge
8c80bfa7da tab indentation comment (#6079)
## Summary

Updated doc comment for `tab_indentation.rs`. Online docs also benefit
from this update.

## Test Plan

Checked docs via
[mkdocs](389fe13c93/CONTRIBUTING.md (L267-L296))
2023-07-25 23:14:43 +00:00
Zanie Blue
389fe13c93 Implement visitation of type aliases and parameters (#5927)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

Part of #5062 
Requires https://github.com/astral-sh/RustPython-Parser/pull/32

Adds visitation of type alias statements and type parameters in class
and function definitions.

Duplicates tests for `PreorderVisitor` into `Visitor` with new
snapshots. Testing required node implementations for the `TypeParam`
enum, which is a chunk of the diff and the reason we need `Ranged`
implementations in
https://github.com/astral-sh/RustPython-Parser/pull/32.

## Test Plan

<!-- How was it tested? -->

Adds unit tests with snapshots.
2023-07-25 17:11:26 +00:00
Zanie Blue
3000a47fe8 Include file permissions in key for cached files (#5901)
Reimplements https://github.com/astral-sh/ruff/pull/3104
Closes https://github.com/astral-sh/ruff/issues/5726

Note that we will generate the hash for a cache key twice in normal
operation. Once to check for the cached item and again to update the
cache. We could optimize this by generating the hash once in
`diagnostics::lint_file` and passing the `u64` into `get` and `update`.
We'd probably want to wrap it in a `CacheKeyHash` enum for type safety.

## Test plan

Unit tests for Windows and Unix.

Manual test with case from issue

```
❯ touch fake.py
❯ chmod +x fake.py
❯ ./target/debug/ruff --select EXE fake.py
fake.py:1:1: EXE002 The file is executable but no shebang is present
Found 1 error.
❯ chmod -x fake.py
❯ ./target/debug/ruff --select EXE fake.py
```
2023-07-25 17:06:47 +00:00
Charlie Marsh
cbf6085375 Fix example in D413 documentation (#6075)
See #6037.
2023-07-25 12:22:11 -04:00
Charlie Marsh
9171bd4c28 Avoid A003 violations for explicitly overridden methods (#6076)
## Summary

If a method is annotated with `@typing_extensions.override`, we should
avoid flagging A003 on it. This isn't part of the standard library yet,
but it's used to explicitly mark methods as overrides.
2023-07-25 16:21:23 +00:00
Chris Pryer
f5c69c1b34 Update ArgumentsParentheses usage (#6070) 2023-07-25 18:03:48 +02:00
Charlie Marsh
5f63b8bfb8 Ignore some common builtin overrides on standard library subclasses (#6074)
## Summary

If a user subclasses `threading.Event`, e.g. with:

```python
from threading import Event


class CustomEvent(Event):
    def set(self) -> None:
        ...
```

They no control over the method name (`set`). This PR allows
`threading.Event#set` and `logging.Filter#filter` overrides, and avoids
flagging A003 in such cases. Ideally, we'd avoid flagging all overridden
methods, but... that's a lot more difficult, and this is at least
_better_ than what we do now.

Closes https://github.com/astral-sh/ruff/issues/6057.

Closes https://github.com/astral-sh/ruff/issues/5956.
2023-07-25 15:54:34 +00:00
Charlie Marsh
c996b614fe Set default max-complexity to 10 for empty McCabe settings (#6073)
Closes https://github.com/astral-sh/ruff/issues/6058.
2023-07-25 15:38:19 +00:00
Ville Skyttä
670db1db4b pycodestyle.max-doc-length doc updates (#6052) 2023-07-25 15:34:26 +00:00
Charlie Marsh
242cbd966d Perform lint rule analysis after subtree traversal (#6045)
## Summary

This PR modifies the order of operations in our AST checker. Previously,
we ran our analysis rules first, then bound names and traversed over the
subtrees. Now, after a series of refactors, we can invert the order: do
the subtree traversal and model-building _first_, then run rules.

The nice thing about this change is that when we go to analyze, e.g., a
function call node, we'll already have traversed any of the constituent
`Expr::Name` nodes... So if we store the resolution of all names when do
the traversal, we can avoid having to do any expensive work in
`resolve_call_path`.

## Test Plan

Clean run of the snapshot tests, and hopefully the ecosystem checks too!
2023-07-25 09:05:44 -04:00
konsti
e7f228f781 Placement refactor (#6034)
## Summary

This PR is a refactoring of placement.rs. The code got more consistent,
some comments were updated and some dead code was removed or replaced
with debug assertions. It also contains a bugfix for the placement of
end-of-branch comments with nested bodies inside try statements that
occurred when refactoring the nested body loop.

## Test Plan

The existing test cases don't change. I added a couple of cases that i
think should be tested but weren't, and a regression test for the bugfix
2023-07-25 11:49:05 +02:00
Paul Mairo
51d8fc1f30 Update contributing.md with where to run ruff from (#6048)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

As of right now, the instructions don't specify where to run ruff from
after cloning the repository this is to address that. Super trivial
change, but helpful for real newbies I think.
2023-07-24 19:44:55 -04:00
Charlie Marsh
ed72c027a3 Replace NoHashHasher usages with FxHashMap (#6049)
## Summary

I had always assumed that `NoHashHasher` would be faster when using
integer keys, but benchmarking shows otherwise:

```
linter/default-rules/numpy/globals.py
                        time:   [66.544 µs 66.606 µs 66.678 µs]
                        thrpt:  [44.253 MiB/s 44.300 MiB/s 44.342 MiB/s]
                 change:
                        time:   [-0.1843% +0.1087% +0.3718%] (p = 0.46 > 0.05)
                        thrpt:  [-0.3704% -0.1086% +0.1847%]
                        No change in performance detected.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild
linter/default-rules/pydantic/types.py
                        time:   [1.3787 ms 1.3811 ms 1.3837 ms]
                        thrpt:  [18.431 MiB/s 18.466 MiB/s 18.498 MiB/s]
                 change:
                        time:   [-0.4827% -0.1074% +0.1927%] (p = 0.56 > 0.05)
                        thrpt:  [-0.1924% +0.1075% +0.4850%]
                        No change in performance detected.
linter/default-rules/numpy/ctypeslib.py
                        time:   [624.82 µs 625.96 µs 627.17 µs]
                        thrpt:  [26.550 MiB/s 26.601 MiB/s 26.650 MiB/s]
                 change:
                        time:   [-0.7071% -0.4908% -0.2736%] (p = 0.00 < 0.05)
                        thrpt:  [+0.2744% +0.4932% +0.7122%]
                        Change within noise threshold.
linter/default-rules/large/dataset.py
                        time:   [3.1585 ms 3.1634 ms 3.1685 ms]
                        thrpt:  [12.840 MiB/s 12.861 MiB/s 12.880 MiB/s]
                 change:
                        time:   [-1.5338% -1.3463% -1.1476%] (p = 0.00 < 0.05)
                        thrpt:  [+1.1610% +1.3647% +1.5577%]
                        Performance has improved.

linter/all-rules/numpy/globals.py
                        time:   [140.17 µs 140.37 µs 140.58 µs]
                        thrpt:  [20.989 MiB/s 21.020 MiB/s 21.051 MiB/s]
                 change:
                        time:   [-0.1066% +0.3140% +0.7479%] (p = 0.14 > 0.05)
                        thrpt:  [-0.7423% -0.3130% +0.1067%]
                        No change in performance detected.
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) high mild
  1 (1.00%) high severe
linter/all-rules/pydantic/types.py
                        time:   [2.7030 ms 2.7069 ms 2.7112 ms]
                        thrpt:  [9.4064 MiB/s 9.4216 MiB/s 9.4351 MiB/s]
                 change:
                        time:   [-0.6721% -0.4874% -0.2974%] (p = 0.00 < 0.05)
                        thrpt:  [+0.2982% +0.4898% +0.6766%]
                        Change within noise threshold.
Found 14 outliers among 100 measurements (14.00%)
  12 (12.00%) high mild
  2 (2.00%) high severe
linter/all-rules/numpy/ctypeslib.py
                        time:   [1.4709 ms 1.4727 ms 1.4749 ms]
                        thrpt:  [11.290 MiB/s 11.306 MiB/s 11.320 MiB/s]
                 change:
                        time:   [-1.1617% -0.9766% -0.8094%] (p = 0.00 < 0.05)
                        thrpt:  [+0.8160% +0.9862% +1.1754%]
                        Change within noise threshold.
Found 12 outliers among 100 measurements (12.00%)
  9 (9.00%) high mild
  3 (3.00%) high severe
linter/all-rules/large/dataset.py
                        time:   [5.8086 ms 5.8163 ms 5.8240 ms]
                        thrpt:  [6.9854 MiB/s 6.9946 MiB/s 7.0038 MiB/s]
                 change:
                        time:   [-1.5651% -1.3536% -1.1584%] (p = 0.00 < 0.05)
                        thrpt:  [+1.1720% +1.3721% +1.5900%]
                        Performance has improved.
```

My guess is that `NoHashHasher` underperforms because the keys are not
randomly distributed...

Anyway, it's a ~1% (significant) performance gain on some of the above,
plus we get to remove a dependency.
2023-07-24 23:41:57 +00:00
Charlie Marsh
b7e7346081 Remove empty newline in deferred_for_loops (#6046)
Trivial change but none of the others have this empty newline.
2023-07-24 21:59:32 +00:00
Charlie Marsh
d35b5248ea Tweak lambda rule to use annotations rather than shadowing (#6044)
## Summary

This PR ensures that we can retain the current behavior even after we
reorder the visitor a bit, by looking for annotated lambdas rather than
"is the name bound to anything?", since if we visit the name before we
run this rule, it'll _always_ be bound. (This check is already a bit
flawed -- in truth, we should probably run this rule deferred so that we
can reliably detect shadowing.)
2023-07-24 21:39:02 +00:00
Charlie Marsh
c535e10fff Move comprehension rules into shared analyze method (#6042) 2023-07-24 21:18:45 +00:00
Charlie Marsh
c3ecdb8783 Fix Arg typo (#6041) 2023-07-24 21:16:28 +00:00
Charlie Marsh
242df67cbf Move lint rules out of checkers/ast/mod.rs (#5957)
## Summary

This PR attempts to draw some basic separation between the `Checker`'s
traversal responsibilities (traversing the AST, building the semantic
model) and its calling-out-to-lint-rule responsibilities. It doesn't try
to introduce any sophisticated API. Instead, it just moves all of the
lint rule calls out of `checkers/ast/mod.rs` and into methods in a new
`analyze` module. (There are four remaining lint rules in `Checker`, but
I'll remove those in future PRs.)

I'm not trying to "solve" our lint rule API here. Instead, I'm trying to
make two improvements:

1. `checkers/ast/mod.rs` has just gotten way too large, and people work
in it all the time. Prior to this PR, it was 5.5k lines, which led to
significant lags in my editor and made it really hard to reason about
the parts that are _actually_ important. (I like big files, but this one
crossed the line for me.) Now, it's < 2,000 lines, and the code is much
more focused.
2. I want to avoid accidentally adding lint rules in the "wrong" parts
of the traversal. By confining lint rule invocations to these "analyze"
calls, we'll avoid (e.g.) putting them in the binding phase.
2023-07-24 19:20:10 +00:00
Charlie Marsh
776d598738 Move flake8-executable rules out of physical lines checker (#6039)
## Summary

These only need the token stream, and we always prefer token-based to
physical line-based rules.

There are a few other changes snuck in here:

- Renaming the rule files to match the diagnostic names (likely an
error).
- The "leading whitespace before shebang" rule now works regardless of
where the comment occurs (i.e., if the shebang is on the second line,
and the first line is blank, we flag and remove that leading
whitespace).
2023-07-24 14:38:05 -04:00
konsti
7f3797185c Fix formatter with-statement after-as own line comment instability (#6033)
**Summary** Fix an instability in with statement formatter when there is
an own line comment as the `as`
```python
with (
    a as
    # bad comment
    b):
```

**Test Plan** Added the comment to the test cases.
2023-07-24 18:12:07 +00:00
konsti
a9f535997d Document formatter progress scripts (#6035)
## Summary

Add documentation to the formatter progress scripts

## Test Plan

n/a
2023-07-24 19:42:20 +02:00
Micha Reiser
fdb3c8852f Prefer breaking the implicit string concatenation over breaking before % (#5947) 2023-07-24 18:30:42 +02:00
Charlie Marsh
42d969f19f Add additional test cases for F823 (#6036)
Making some behavior explicit / codified. See:
https://github.com/astral-sh/ruff/issues/6029.
2023-07-24 15:49:48 +00:00
Charlie Marsh
62ffc773de Avoid treating Literal members as expressions with __future__ (#6032)
Closes https://github.com/astral-sh/ruff/issues/6030.
2023-07-24 15:09:37 +00:00
Charlie Marsh
6feb3fcc1b Ignore end-of-line comments when dirtying if-with-same-arms branches (#6031)
## Summary

Closes https://github.com/astral-sh/ruff/issues/6025 (which contains a
more thorough description of the issue). Previously, the `# noqa` here
was being marked as unused, but removing it raised `SIM114`:

```python
def foo():
    a = True
    b = False
    if a > b:  # noqa: SIM114
        return 3
    elif a == b:
        return 3
```
2023-07-24 10:59:58 -04:00
Chris Pryer
8eadacda33 Update TupleParentheses usage (#5810) 2023-07-24 14:44:36 +00:00
konsti
8a7dcb794b Add formatter progress tracking to CI (#5919)
**Summary** Add a formatter progress testing script to CI. This script
will 1) print the black compability on each run 2) catch regressions wrt
to formatter stability, emitting invalid syntax and other kinds of bugs
(e.g. #5917) before they land on main 3) have an additional layer of
real world tests when implementing new nodes or other new formatter
code.

This is currently a bash script, i'm not sure if we want to keep it that
way, or switch to e.g. the regular ecosystem scripts. The output
separation of `format_dev` could also use some polishing. We should also
consider pinning commits so we don't get spurious regression when they
change their code.

**Test Plan** The script extends CI.
2023-07-24 09:12:42 +00:00
Luc Khai Hai
dfa81b6fe0 Format numeric constants (#5972)
Co-authored-by: Micha Reiser <micha@reiser.io>
2023-07-24 07:04:40 +00:00
Charlie Marsh
33196f1859 Fix logging rules with whitespace around dot (#6022)
## Summary

Attempting to fix, e.g., `logging . warn("Hello World!")` was causing a
syntax error.
2023-07-24 05:14:48 +00:00
Charlie Marsh
0d94337b96 Avoid allocations in SimpleCallArgs (#6021)
## Summary

My intuition is that it's faster to do these checks as-needed rather
than allocation new hash maps and vectors for the arguments. (We
typically only query once anyway.)
2023-07-24 04:55:37 +00:00
Charlie Marsh
f9726af4ef Allow specification of logging.Logger re-exports via logger-objects (#5750)
## Summary

This PR adds a `logger-objects` setting that allows users to mark
specific symbols a `logging.Logger` objects. Currently, if a `logger` is
imported, we only flagged it as a `logging.Logger` if it comes exactly
from the `logging` module or is `flask.current_app.logger`.

This PR allows users to mark specific loggers, like
`logging_setup.logger`, to ensure that they're covered by the
`flake8-logging-format` rules and others.

For example, if you have a module `logging_setup.py` with the following
contents:

```python
import logging

logger = logging.getLogger(__name__)
```

Adding `"logging_setup.logger"` to `logger-objects` will ensure that
`logging_setup.logger` is treated as a `logging.Logger` object when
imported from other modules (e.g., `from logging_setup import logger`).

Closes https://github.com/astral-sh/ruff/issues/5694.
2023-07-24 00:38:20 -04:00
Tom Kuson
727153cf45 [pylint] Impement self-assigning-variable (W0127) (#6015)
## Summary

Implements Pylint rule [`self-assigning-variable`
(`W0127`)](https://pylint.pycqa.org/en/latest/user_guide/messages/warning/self-assigning-variable.html)
as `self-assigning-variable` (`PLW0127`). Includes documentation.
Related to #970.

## Test Plan

`cargo test`
2023-07-24 02:27:09 +00:00
Charlie Marsh
574c0e0105 Use match instead of phf for confusable lookup (#5953)
I don't know whether we want to make this change but here's some data...

Binary size:

- `main`: 30,384
- `charlie/match-phf`: 30,416

llvm-lines:

- `main`: 1,784,148
- `charlie/match-phf`: 1,789,877

llvm-lines and binary size are both unchanged (or, by < 5) when moving
from `u8` to `u32` return types, and even when moving to `char` keys and
values. I didn't expect this, but I'm not very knowledgable on this
topic.

Performance:

```
Confusables/match/src   time:   [4.9102 µs 4.9352 µs 4.9777 µs]
                        change: [+1.7469% +2.2421% +2.8710%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 12 outliers among 100 measurements (12.00%)
  2 (2.00%) low mild
  4 (4.00%) high mild
  6 (6.00%) high severe
Confusables/match-with-skip/src
                        time:   [2.0676 µs 2.0945 µs 2.1317 µs]
                        change: [+0.9384% +1.6000% +2.3920%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 8 outliers among 100 measurements (8.00%)
  3 (3.00%) high mild
  5 (5.00%) high severe
Confusables/phf/src     time:   [31.087 µs 31.188 µs 31.305 µs]
                        change: [+1.9262% +2.2188% +2.5496%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 15 outliers among 100 measurements (15.00%)
  3 (3.00%) low mild
  6 (6.00%) high mild
  6 (6.00%) high severe
Confusables/phf-with-skip/src
                        time:   [2.0470 µs 2.0486 µs 2.0502 µs]
                        change: [-0.3093% -0.1446% +0.0106%] (p = 0.08 > 0.05)
                        No change in performance detected.
Found 4 outliers among 100 measurements (4.00%)
  2 (2.00%) high mild
  2 (2.00%) high severe
```

The `-with-skip` variants add our optimization which first checks
whether the character is ASCII. So `match` is way, way faster than PHF,
but it tends not to matter since almost all source code is ASCII anyway.
2023-07-24 02:23:36 +00:00
Dhruv Manilawala
700c816fd5 Make TRY201 always autofixable (#6008)
## Summary

Make `TRY201` always autofiable.

## Test Plan

1. `cargo test`
2. `cargo insta review`

ref:
https://github.com/astral-sh/ruff/issues/4333#issuecomment-1646359788
2023-07-24 02:23:15 +00:00
Tom Kuson
3b56f6d616 [pylint] Implement subprocess-popen-preexec-fn (W1509) (#5978)
## Summary

Implements Pylint rule [`subprocess-popen-preexec-fn`
(`W1509`)](https://pylint.pycqa.org/en/latest/user_guide/messages/warning/subprocess-popen-preexec-fn.html)
as `subprocess-popen-preexec-fn` (`PLW1509`). Includes documentation.
Related to #970.

## Test Plan

`cargo test`
2023-07-24 02:06:19 +00:00
Harutaka Kawamura
110fa804ff Add PT016 documentation (#6005) 2023-07-23 21:52:48 -04:00
Harutaka Kawamura
2b9c22de0f Add a unit test for python-file-like directory exclusion (#5997) 2023-07-24 01:50:39 +00:00
Harutaka Kawamura
51ebff7e41 Add PT010 doc (#6010) 2023-07-24 01:43:18 +00:00
Dhruv Manilawala
742f615792 Add support for int, float, bool in UP018 (#6013)
## Summary

This pull request add supports for `int`, `float` and `bool` types in
`UP018`
rule to convert empty call to the default value of the type or remove
the call
if a value of the same type is provided as an argument.

## Test Plan

Added tests for `int`, `float` and `bool` types.

Partially resolves #5988
2023-07-23 21:39:43 -04:00
Harutaka Kawamura
95e6258d5d Add PT020 doc (#6011) 2023-07-23 21:37:03 -04:00
Dhruv Manilawala
5dbb4dd823 Update docs for ANN401 (#6009)
Part of #5803
2023-07-23 16:15:04 +00:00
konsti
46f8961292 Formatter: Add EmptyWithDanglingComments helper (#5951)
**Summary** Add a `EmptyWithDanglingComments` format helper that formats
comments inside empty parentheses, brackets or curly braces. Previously,
this was implemented separately, and partially incorrectly, for each use
case.

Empty `()`, `[]` and `{}` are special because there can be dangling
comments, and they can be in
two positions:
```python
x = [  # end-of-line
    # own line
]
```
These comments are dangling because they can't be assigned to any
element inside as they would
in all other cases.

**Test Plan** Added a regression test.

145 (from previously 149) instances of unstable formatting remaining.

```
$ cargo run --bin ruff_dev --release -- format-dev --stability-check --error-file formatter-ecosystem-errors.txt --multi-project target/checkouts > formatter-ecosystem-progress.txt
$ rg "Unstable formatting" target/formatter-ecosystem-errors.txt | wc -l
145
```
2023-07-23 14:32:16 +02:00
Simon Brugman
f886b58c92 [flake8-use-pathlib] Implement os-sep-split (PTH206) (#5936)
Implements
https://github.com/astral-sh/ruff/issues/5905#issuecomment-1644822548

---------

Co-authored-by: konsti <konstin@mailbox.org>
2023-07-23 12:22:26 +02:00
Charlie Marsh
057faabcdd Use Flags::intersects rather than Flags::contains (#6007)
## Summary

This is equivalent for a single flag, but I think it's more likely to be
correct when the bitflags are modified -- the primary reason being that
we sometimes define flags as the union of other flags, e.g.:

```rust
const ANNOTATION = Self::TYPING_ONLY_ANNOTATION.bits() | Self::RUNTIME_ANNOTATION.bits();
```

In this case, `flags.contains(Flag::ANNOTATION)` requires that _both_
flags in the union are set, whereas `flags.intersects(Flag::ANNOTATION)`
requires that _at least one_ flag is set.
2023-07-23 02:59:31 +00:00
Charlie Marsh
0bb175f7f6 Store flags rather than ExecutionContext on references (#6006) 2023-07-23 02:54:39 +00:00
Charlie Marsh
4b2ec7d562 Move runtime execution context into add_reference calls (#6003) 2023-07-23 02:37:51 +00:00
Charlie Marsh
4aac801277 Fix context-to-model references in SemanticModel documentation (#6004) 2023-07-23 02:32:23 +00:00
Charlie Marsh
45a24912a6 Remove extra error! call (#6002) 2023-07-23 02:29:06 +00:00
Simon Brugman
3914fcb7ca Extend SIM118 with not in (#5995)
Closes https://github.com/astral-sh/ruff/issues/5989

Tracking issue https://github.com/astral-sh/ruff/issues/1348
2023-07-23 01:46:21 +00:00
Charlie Marsh
6d58b773b1 Use simple text matching for type: ignore detection (#5999)
Closes #5980.
2023-07-23 01:45:28 +00:00
Tom Kuson
e7f5121922 Extends B002 to detect unary prefix decrement operators (#5998)
## Summary

Extends `B002` to detect unary decrement prefix operators.

Closes #5992.

## Test Plan

`cargo test`
2023-07-23 01:40:49 +00:00
Charlie Marsh
1776cbd2e2 Move blanket noqa and blanket type: ignore rules into token-based checker (#5996)
Closes https://github.com/astral-sh/ruff/issues/5981.
2023-07-22 21:22:48 -04:00
Charlie Marsh
71f1643eda Use memchr for invalid-escape-sequence (#5994) 2023-07-22 20:57:36 -04:00
Tom Kuson
74dc137b30 Use find_keyword helper function in more places (#5993)
## Summary

Use the `find_keyword` helper function instead of reimplementing it.

Follows on from #5983 by doing a different search.

## Test Plan

`cargo test`
2023-07-22 20:27:24 -04:00
Harutaka Kawamura
97e31cad2f Fix F507 false positive (#5986)
## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

F507 should not be raised when the right-hand side value is a non-tuple
object.

```python
'%s' % (1, 2, 3)  # throws
'%s' % [1, 2, 3]  # doesn't throw
'%s' % {1, 2, 3}  # doesn't throw
```
2023-07-22 18:42:44 +00:00
Simon Brugman
ed7d2b8a3d Do not raise SIM105 for non-exceptions (#5985)
Closes https://github.com/astral-sh/ruff/issues/5977

Added a test case from `refurb`
2023-07-22 18:36:46 +00:00
Tom Kuson
c7e4c58181 Use find_keyword helper function (#5983)
## Summary

Use `find_keyword` helper function instead of reimplementing it.

## Test Plan

`cargo test`
2023-07-22 14:09:30 -04:00
Charlie Marsh
6ff566f2c1 Flag [ as an invalid noqa suffix (#5982)
Closes https://github.com/astral-sh/ruff/issues/5960.
2023-07-22 10:16:28 -04:00
Charlie Marsh
32773e8309 Move locator, stylist, and friends better getters (#5968)
## Summary

Rather than exposing these as public fields, use getters, similar to
`semantic()`.
2023-07-22 09:37:24 -04:00
Harutaka Kawamura
050f5953f8 Avoid raising UP032 if format call arguments contain multiline expressions (#5971)
## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

Fix a regression introduced by
https://github.com/astral-sh/ruff/pull/5638. A multiline expression
can't be safely inserted into a format field.

### Example

```
> cat a.py
"{}".format(
    [
        1,
        2,
        3,
    ]
)

> cargo run -p ruff_cli -- check a.py --no-cache --select UP032 --fix
    Finished dev [unoptimized + debuginfo] target(s) in 0.07s
     Running `target/debug/ruff check a.py --no-cache --select UP032 --fix`
error: Autofix introduced a syntax error in `a.py` with rule codes UP032: EOL while scanning string literal at byte offset 5
---
f"{[
        1,
        2,
        3,
    ]}"

---
a.py:1:1: UP032 Use f-string instead of `format` call
Found 1 error.
```


## Test Plan

New test cases
2023-07-22 09:37:08 -04:00
Alex Waygood
aba340a177 Fix typo in PYI056 docs (#5973)
The current "use instead" code would correctly be rejected by any type
checker worth its salt ;)
2023-07-22 09:10:38 -04:00
Victor Hugo Gomes
33657d3a1c [flake8-pyi] Implement PYI056 (#5959)
## Summary

Checks that `append`, `extend` and `remove` methods are not called on
`__all__`. See [original
implementation](2a86db8271/pyi.py (L1133-L1138)).

```
$ flake8 --select Y026 crates/ruff/resources/test/fixtures/flake8_pyi/PYI056.pyi

crates/ruff/resources/test/fixtures/flake8_pyi/PYI056.pyi:3:1: Y056 Calling ".append()" on "__all__" may not be supported by all type checkers (use += instead)
crates/ruff/resources/test/fixtures/flake8_pyi/PYI056.pyi:4:1: Y056 Calling ".extend()" on "__all__" may not be supported by all type checkers (use += instead)
crates/ruff/resources/test/fixtures/flake8_pyi/PYI056.pyi:5:1: Y056 Calling ".remove()" on "__all__" may not be supported by all type checkers (use += instead)
```

```
$ ./target/debug/ruff --select PYI026 crates/ruff/resources/test/fixtures/flake8_pyi/PYI056.pyi --no-cache

crates/ruff/resources/test/fixtures/flake8_pyi/PYI056.pyi:3:1: PYI056 Calling ".append()" on "__all__" may not be supported by all type checkers (use += instead)
crates/ruff/resources/test/fixtures/flake8_pyi/PYI056.pyi:4:1: PYI056 Calling ".extend()" on "__all__" may not be supported by all type checkers (use += instead)
crates/ruff/resources/test/fixtures/flake8_pyi/PYI056.pyi:5:1: PYI056 Calling ".remove()" on "__all__" may not be supported by all type checkers (use += instead)
Found 3 errors.
```

ref #848

## Test Plan

Snapshots and manual runs of flake8.
2023-07-22 04:25:54 +00:00
Charlie Marsh
45318d08b7 Always compute runtime annotations for flake8-type-checking rules (#5967)
## Summary

These are skipped as an optimization, but it feels kind of unnecessary
and makes the code a bit more confusing than is worthwhile.
(non-`strict` is also by far the more popular setting, and the default.)
2023-07-21 23:53:33 -04:00
1214 changed files with 135663 additions and 20594 deletions

1
.github/release.yml vendored
View File

@@ -4,6 +4,7 @@ changelog:
labels:
- internal
- documentation
- formatter
categories:
- title: Breaking Changes
labels:

View File

@@ -3,12 +3,12 @@ name: Benchmark
on:
pull_request:
paths:
- 'Cargo.toml'
- 'Cargo.lock'
- 'rust-toolchain'
- 'crates/**'
- '!crates/ruff_dev'
- '!crates/ruff_shrinking'
- "Cargo.toml"
- "Cargo.lock"
- "rust-toolchain"
- "crates/**"
- "!crates/ruff_dev"
- "!crates/ruff_shrinking"
workflow_dispatch:
@@ -22,7 +22,7 @@ jobs:
name: "Run | ${{ matrix.os }}"
strategy:
matrix:
os: [ ubuntu-latest, windows-latest ]
os: [ubuntu-latest, windows-latest]
runs-on: ${{ matrix.os }}
steps:

View File

@@ -2,7 +2,7 @@ name: CI
on:
push:
branches: [ main ]
branches: [main]
pull_request:
workflow_dispatch:
@@ -16,7 +16,7 @@ env:
CARGO_TERM_COLOR: always
RUSTUP_MAX_RETRIES: 10
PACKAGE_NAME: ruff
PYTHON_VERSION: "3.11" # to build abi3 wheels
PYTHON_VERSION: "3.11"
jobs:
determine_changes:
@@ -42,6 +42,7 @@ jobs:
- "!crates/ruff_formatter/**"
- "!crates/ruff_dev/**"
- "!crates/ruff_shrinking/**"
- scripts/check_ecosystem.py
formatter:
- Cargo.toml
@@ -50,7 +51,11 @@ jobs:
- crates/ruff_formatter/**
- crates/ruff_python_trivia/**
- crates/ruff_python_ast/**
- crates/ruff_source_file/**
- crates/ruff_python_index/**
- crates/ruff_text_size/**
- crates/ruff_python_parser/**
- crates/ruff_dev/**
cargo-fmt:
name: "cargo fmt"
@@ -79,7 +84,7 @@ jobs:
cargo-test:
strategy:
matrix:
os: [ ubuntu-latest, windows-latest ]
os: [ubuntu-latest, windows-latest]
runs-on: ${{ matrix.os }}
name: "cargo test | ${{ matrix.os }}"
steps:
@@ -231,7 +236,6 @@ jobs:
- name: "Run cargo-udeps"
run: cargo +nightly-2023-06-08 udeps
python-package:
name: "python package"
runs-on: ubuntu-latest
@@ -319,8 +323,8 @@ jobs:
if: ${{ env.MKDOCS_INSIDERS_SSH_KEY_EXISTS != 'true' }}
run: mkdocs build --strict -f mkdocs.generated.yml
check-formatter-stability:
name: "Check formatter stability"
check-formatter-ecosystem:
name: "Formatter ecosystem and progress checks"
runs-on: ubuntu-latest
needs: determine_changes
if: needs.determine_changes.outputs.formatter == 'true'
@@ -330,7 +334,12 @@ jobs:
run: rustup show
- name: "Cache rust"
uses: Swatinem/rust-cache@v2
- name: "Formatter progress"
run: scripts/formatter_ecosystem_checks.sh
- name: "Github step summary"
run: grep "similarity index" target/progress_projects_log.txt | sort > $GITHUB_STEP_SUMMARY
# CPython is not black formatted, so we run only the stability check
- name: "Clone CPython 3.10"
run: git clone --branch 3.10 --depth 1 https://github.com/python/cpython.git crates/ruff/resources/test/cpython
- name: "Check stability"
- name: "Check CPython stability"
run: cargo run --bin ruff_dev -- format-dev --stability-check crates/ruff/resources/test/cpython

View File

@@ -3,7 +3,7 @@ name: mkdocs
on:
workflow_dispatch:
release:
types: [ published ]
types: [published]
jobs:
mkdocs:

View File

@@ -66,7 +66,7 @@ jobs:
runs-on: windows-latest
strategy:
matrix:
target: [ x64, x86 ]
target: [x64, x86]
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
@@ -94,7 +94,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
target: [ x86_64, i686 ]
target: [x86_64, i686]
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
@@ -121,7 +121,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
target: [ aarch64, armv7, s390x, ppc64le, ppc64 ]
target: [aarch64, armv7, s390x, ppc64le, ppc64]
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4

View File

@@ -3,7 +3,7 @@ name: "[Playground] Release"
on:
workflow_dispatch:
release:
types: [ published ]
types: [published]
env:
CARGO_INCREMENTAL: 0

View File

@@ -2,8 +2,8 @@ name: PR Check Comment
on:
workflow_run:
workflows: [ CI, Benchmark ]
types: [ completed ]
workflows: [CI, Benchmark]
types: [completed]
workflow_dispatch:
inputs:
workflow_run_id:

View File

@@ -42,13 +42,13 @@ repos:
name: cargo fmt
entry: cargo fmt --
language: system
types: [ rust ]
types: [rust]
pass_filenames: false # This makes it a lot faster
- id: ruff
name: ruff
entry: cargo run --bin ruff -- check --no-cache --force-exclude --fix --exit-non-zero-on-fix
language: system
types_or: [ python, pyi ]
types_or: [python, pyi]
require_serial: true
exclude: |
(?x)^(
@@ -62,5 +62,12 @@ repos:
hooks:
- id: black
# Prettier
- repo: https://github.com/pre-commit/mirrors-prettier
rev: v3.0.0
hooks:
- id: prettier
types: [yaml]
ci:
skip: [ cargo-fmt, dev-generate-all ]
skip: [cargo-fmt, dev-generate-all]

View File

@@ -69,9 +69,16 @@ and pre-commit to run some validation checks:
pipx install pre-commit # or `pip install pre-commit` if you have a virtualenv
```
You can optionally install pre-commit hooks to automatically run the validation checks
when making a commit:
```shell
pre-commit install
```
### Development
After cloning the repository, run Ruff locally with:
After cloning the repository, run Ruff locally from the repository root with:
```shell
cargo run -p ruff_cli -- check /path/to/file.py --no-cache
@@ -122,9 +129,9 @@ At time of writing, the repository includes the following crates:
intermediate representation. The backend for `ruff_python_formatter`.
- `crates/ruff_index`: library crate inspired by `rustc_index`.
- `crates/ruff_macros`: proc macro crate containing macros used by Ruff.
- `crates/ruff_python_ast`: library crate containing Python-specific AST types and utilities. Note
that the AST schema itself is defined in the
[rustpython-ast](https://github.com/astral-sh/RustPython-Parser) crate.
- `crates/ruff_python_ast`: library crate containing Python-specific AST types and utilities.
- `crates/ruff_python_codegen`: library crate containing utilities for generating Python source code.
- `crates/ruff_python_codegen`: library crate containing utilities for generating Python source code.
- `crates/ruff_python_formatter`: library crate implementing the Python formatter. Emits an
intermediate representation for each node, which `ruff_formatter` prints based on the configured
line length.
@@ -135,8 +142,7 @@ At time of writing, the repository includes the following crates:
the names of all built-in exceptions and which standard library types are immutable.
- `crates/ruff_python_trivia`: library crate containing Python-specific trivia utilities (e.g.,
for analyzing indentation, newlines, etc.).
- `crates/ruff_rustpython`: library crate containing `RustPython`-specific utilities.
- `crates/ruff_textwrap`: library crate to indent and dedent Python source code.
- `crates/ruff_python_parser`: library crate containing the Python parser.
- `crates/ruff_wasm`: library crate for exposing Ruff as a WebAssembly module. Powers the
[Ruff Playground](https://play.ruff.rs/).
@@ -156,10 +162,13 @@ At a high level, the steps involved in adding a new lint rule are as follows:
(e.g., `pub(crate) fn assert_false`) based on whatever inputs are required for the rule (e.g.,
an `ast::StmtAssert` node).
1. Define the logic for triggering the violation in `crates/ruff/src/checkers/ast/mod.rs` (for
AST-based checks), `crates/ruff/src/checkers/tokens.rs` (for token-based checks),
`crates/ruff/src/checkers/lines.rs` (for text-based checks), or
`crates/ruff/src/checkers/filesystem.rs` (for filesystem-based checks).
1. Define the logic for invoking the diagnostic in `crates/ruff/src/checkers/ast/analyze` (for
AST-based rules), `crates/ruff/src/checkers/tokens.rs` (for token-based rules),
`crates/ruff/src/checkers/physical_lines.rs` (for text-based rules),
`crates/ruff/src/checkers/filesystem.rs` (for filesystem-based rules), etc. For AST-based rules,
you'll likely want to modify `analyze/statement.rs` (if your rule is based on analyzing
statements, like imports) or `analyze/expression.rs` (if your rule is based on analyzing
expressions, like function calls).
1. Map the violation struct to a rule code in `crates/ruff/src/codes.rs` (e.g., `B011`).
@@ -221,9 +230,12 @@ Once you've completed the code for the rule itself, you can define tests with th
For example, if you're adding a new rule named `E402`, you would run:
```shell
cargo run -p ruff_cli -- check crates/ruff/resources/test/fixtures/pycodestyle/E402.py --no-cache
cargo run -p ruff_cli -- check crates/ruff/resources/test/fixtures/pycodestyle/E402.py --no-cache --select E402
```
**Note:** Only a subset of rules are enabled by default. When testing a new rule, ensure that
you activate it by adding `--select ${rule_code}` to the command.
1. Add the test to the relevant `crates/ruff/src/rules/[linter]/mod.rs` file. If you're contributing
a rule to a pre-existing set, you should be able to find a similar example to pattern-match
against. If you're adding a new linter, you'll need to create a new `mod.rs` file (see,
@@ -310,7 +322,7 @@ even patch releases may contain [non-backwards-compatible changes](https://semve
1. Run the release workflow with the version number (without starting `v`) as input. Make sure
main has your merged PR as last commit
1. The release workflow will do the following:
1. Build all the assets. If this fails (even though we tested in step 4), we havent tagged or
1. Build all the assets. If this fails (even though we tested in step 4), we haven't tagged or
uploaded anything, you can restart after pushing a fix.
1. Upload to PyPI.
1. Create and push the Git tag (as extracted from `pyproject.toml`). We create the Git tag only
@@ -723,9 +735,9 @@ diagnostics, then our current compilation pipeline proceeds as follows:
1. **File discovery**: Given paths like `foo/`, locate all Python files in any specified subdirectories, taking into account our hierarchical settings system and any `exclude` options.
1. **Package resolution**: Determine the package root for every file by traversing over its parent directories and looking for `__init__.py` files.
1. **Package resolution**: Determine the "package root" for every file by traversing over its parent directories and looking for `__init__.py` files.
1. **Cache initialization**: For every package root, initialize an empty cache.
1. **Cache initialization**: For every "package root", initialize an empty cache.
1. **Analysis**: For every file, in parallel:
@@ -733,7 +745,7 @@ diagnostics, then our current compilation pipeline proceeds as follows:
1. **Tokenization**: Run the lexer over the file to generate a token stream.
1. **Indexing**: Extract metadata from the token stream, such as: comment ranges, `# noqa` locations, `# isort: off` locations, doc lines, etc.
1. **Indexing**: Extract metadata from the token stream, such as: comment ranges, `# noqa` locations, `# isort: off` locations, "doc lines", etc.
1. **Token-based rule evaluation**: Run any lint rules that are based on the contents of the token stream (e.g., commented-out code).
@@ -743,9 +755,9 @@ diagnostics, then our current compilation pipeline proceeds as follows:
1. **Parsing**: Run the parser over the token stream to produce an AST. (This consumes the token stream, so anything that relies on the token stream needs to happen before parsing.)
1. **AST-based rule evaluation**: Run any lint rules that are based on the AST. This includes the vast majority of lint rules. As part of this step, we also build the semantic model for the current file as we traverse over the AST. Some lint rules are evaluated eagerly, as we iterate over the AST, while others are evaluated in a deferred manner (e.g., unused imports, since we cant determine whether an import is unused until weve finished analyzing the entire file), after weve finished the initial traversal.
1. **AST-based rule evaluation**: Run any lint rules that are based on the AST. This includes the vast majority of lint rules. As part of this step, we also build the semantic model for the current file as we traverse over the AST. Some lint rules are evaluated eagerly, as we iterate over the AST, while others are evaluated in a deferred manner (e.g., unused imports, since we can't determine whether an import is unused until we've finished analyzing the entire file), after we've finished the initial traversal.
1. **Import-based rule evaluation**: Run any lint rules that are based on the modules imports (e.g., import sorting). These could, in theory, be included in the AST-based rule evaluation phase — theyre just separated for simplicity.
1. **Import-based rule evaluation**: Run any lint rules that are based on the module's imports (e.g., import sorting). These could, in theory, be included in the AST-based rule evaluation phase — they're just separated for simplicity.
1. **Physical line-based rule evaluation**: Run any lint rules that are based on physical lines (e.g., line-length).
@@ -754,3 +766,116 @@ diagnostics, then our current compilation pipeline proceeds as follows:
1. **Cache write**: Write the generated diagnostics to the package cache using the file as a key.
1. **Reporting**: Print diagnostics in the specified format (text, JSON, etc.), to the specified output channel (stdout, a file, etc.).
### Import Categorization
To understand Ruff's import categorization system, we first need to define two concepts:
- "Project root": The directory containing the `pyproject.toml`, `ruff.toml`, or `.ruff.toml` file,
discovered by identifying the "closest" such directory for each Python file. (If you're running
via `ruff --config /path/to/pyproject.toml`, then the current working directory is used as the
"project root".)
- "Package root": The top-most directory defining the Python package that includes a given Python
file. To find the package root for a given Python file, traverse up its parent directories until
you reach a parent directory that doesn't contain an `__init__.py` file (and isn't marked as
a [namespace package](https://beta.ruff.rs/docs/settings/#namespace-packages)); take the directory
just before that, i.e., the first directory in the package.
For example, given:
```text
my_project
├── pyproject.toml
└── src
└── foo
├── __init__.py
└── bar
├── __init__.py
└── baz.py
```
Then when analyzing `baz.py`, the project root would be the top-level directory (`./my_project`),
and the package root would be `./my_project/src/foo`.
#### Project root
The project root does not have a significant impact beyond that all relative paths within the loaded
configuration file are resolved relative to the project root.
For example, to indicate that `bar` above is a namespace package (it isn't, but let's run with it),
the `pyproject.toml` would list `namespace-packages = ["./src/bar"]`, which would resolve
to `my_project/src/bar`.
The same logic applies when providing a configuration file via `--config`. In that case, the
_current working directory_ is used as the project root, and so all paths in that configuration file
are resolved relative to the current working directory. (As a general rule, we want to avoid relying
on the current working directory as much as possible, to ensure that Ruff exhibits the same behavior
regardless of where and how you invoke it — but that's hard to avoid in this case.)
Additionally, if a `pyproject.toml` file _extends_ another configuration file, Ruff will still use
the directory containing that `pyproject.toml` file as the project root. For example, if
`./my_project/pyproject.toml` contains:
```toml
[tool.ruff]
extend = "/path/to/pyproject.toml"
```
Then Ruff will use `./my_project` as the project root, even though the configuration file extends
`/path/to/pyproject.toml`. As such, if the configuration file at `/path/to/pyproject.toml` contains
any relative paths, they will be resolved relative to `./my_project`.
If a project uses nested configuration files, then Ruff would detect multiple project roots, one for
each configuration file.
#### Package root
The package root is used to determine a file's "module path". Consider, again, `baz.py`. In that
case, `./my_project/src/foo` was identified as the package root, so the module path for `baz.py`
would resolve to `foo.bar.baz` — as computed by taking the relative path from the package root
(inclusive of the root itself). The module path can be thought of as "the path you would use to
import the module" (e.g., `import foo.bar.baz`).
The package root and module path are used to, e.g., convert relative to absolute imports, and for
import categorization, as described below.
#### Import categorization
When sorting and formatting import blocks, Ruff categorizes every import into one of five
categories:
1. **"Future"**: the import is a `__future__` import. That's easy: just look at the name of the
imported module!
1. **"Standard library"**: the import comes from the Python standard library (e.g., `import os`).
This is easy too: we include a list of all known standard library modules in Ruff itself, so it's
a simple lookup.
1. **"Local folder"**: the import is a relative import (e.g., `from .foo import bar`). This is easy
too: just check if the import includes a `level` (i.e., a dot-prefix).
1. **"First party"**: the import is part of the current project. (More on this below.)
1. **"Third party"**: everything else.
The real challenge lies in determining whether an import is first-party — everything else is either
trivial, or (as in the case of third-party) merely defined as "not first-party".
There are three ways in which an import can be categorized as "first-party":
1. **Explicit settings**: the import is marked as such via the `known-first-party` setting. (This
should generally be seen as an escape hatch.)
1. **Same-package**: the imported module is in the same package as the current file. This gets back
to the importance of the "package root" and the file's "module path". Imagine that we're
analyzing `baz.py` above. If `baz.py` contains any imports that appear to come from the `foo`
package (e.g., `from foo import bar` or `import foo.bar`), they'll be classified as first-party
automatically. This check is as simple as comparing the first segment of the current file's
module path to the first segment of the import.
1. **Source roots**: Ruff supports a `[src](https://beta.ruff.rs/docs/settings/#src)` setting, which
sets the directories to scan when identifying first-party imports. The algorithm is
straightforward: given an import, like `import foo`, iterate over the directories enumerated in
the `src` setting and, for each directory, check for the existence of a subdirectory `foo` or a
file `foo.py`.
By default, `src` is set to the project root. In the above example, we'd want to set
`src = ["./src"]` to ensure that we locate `./my_project/src/foo` and thus categorize `import foo`
as first-party in `baz.py`. In practice, for this limited example, setting `src = ["./src"]` is
unnecessary, as all imports within `./my_project/src/foo` would be categorized as first-party via
the same-package heuristic; but your project contains multiple packages, you'll want to set `src`
explicitly.

492
Cargo.lock generated
View File

@@ -133,6 +133,21 @@ dependencies = [
"os_str_bytes",
]
[[package]]
name = "arrayvec"
version = "0.7.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "96d30a06541fbafbc7f82ed10c06164cfbd2c401138f6addd8404629c4b16711"
[[package]]
name = "ascii-canvas"
version = "3.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8824ecca2e851cec16968d54a01dd372ef8f95b244fb84b84e70128be347c3c6"
dependencies = [
"term",
]
[[package]]
name = "assert_cmd"
version = "2.0.11"
@@ -169,6 +184,21 @@ dependencies = [
"serde",
]
[[package]]
name = "bit-set"
version = "0.5.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0700ddab506f33b20a03b13996eccd309a48e5ff77d0d95926aa0210fb4e95f1"
dependencies = [
"bit-vec",
]
[[package]]
name = "bit-vec"
version = "0.6.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "349f9b6a179ed607305526ca489b34ad0a41aed5f7980fa90eb03160b69598fb"
[[package]]
name = "bitflags"
version = "1.3.2"
@@ -609,6 +639,16 @@ dependencies = [
"dirs-sys 0.4.1",
]
[[package]]
name = "dirs-next"
version = "2.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b98cf8ebf19c3d1b223e151f99a4f9f0690dca41414773390fc824184ac833e1"
dependencies = [
"cfg-if",
"dirs-sys-next",
]
[[package]]
name = "dirs-sys"
version = "0.3.7"
@@ -632,6 +672,17 @@ dependencies = [
"windows-sys 0.48.0",
]
[[package]]
name = "dirs-sys-next"
version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4ebda144c4fe02d1f7ea1a7d9641b6fc6b580adcfa024ae48797ecdeb6825b4d"
dependencies = [
"libc",
"redox_users",
"winapi",
]
[[package]]
name = "doc-comment"
version = "0.3.3"
@@ -656,6 +707,15 @@ version = "1.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7fcaabb2fef8c910e7f4c7ce9f67a1283a1715879a7c230ca9d6d1ae31f16d91"
[[package]]
name = "ena"
version = "0.14.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c533630cf40e9caa44bd91aadc88a75d75a4c3a12b4cfde353cbed41daa1e1f1"
dependencies = [
"log",
]
[[package]]
name = "encode_unicode"
version = "0.3.6"
@@ -732,9 +792,15 @@ dependencies = [
"windows-sys 0.48.0",
]
[[package]]
name = "fixedbitset"
version = "0.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0ce7134b9999ecaf8bcd65542e436736ef32ddca1b3e06094cb6ec5755203b80"
[[package]]
name = "flake8-to-ruff"
version = "0.0.280"
version = "0.0.282"
dependencies = [
"anyhow",
"clap",
@@ -967,6 +1033,7 @@ dependencies = [
"number_prefix",
"portable-atomic",
"unicode-width",
"vt100",
]
[[package]]
@@ -1099,6 +1166,28 @@ dependencies = [
"libc",
]
[[package]]
name = "lalrpop"
version = "0.20.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "da4081d44f4611b66c6dd725e6de3169f9f63905421e8626fcb86b6a898998b8"
dependencies = [
"ascii-canvas",
"bit-set",
"diff",
"ena",
"is-terminal",
"itertools",
"lalrpop-util",
"petgraph",
"regex",
"regex-syntax 0.7.3",
"string_cache",
"term",
"tiny-keccak",
"unicode-xid",
]
[[package]]
name = "lalrpop-util"
version = "0.20.0"
@@ -1199,6 +1288,16 @@ version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "09fc20d2ca12cb9f044c93e3bd6d32d523e6e2ec3db4f7b2939cd99026ecd3f0"
[[package]]
name = "lock_api"
version = "0.4.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c1cc9717a20b1bb222f333e6a92fd32f7d8a18ddc5a3191a11af45dcbf4dcd16"
dependencies = [
"autocfg",
"scopeguard",
]
[[package]]
name = "log"
version = "0.4.19"
@@ -1277,6 +1376,12 @@ version = "1.0.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "308d96db8debc727c3fd9744aac51751243420e46edf401010908da7f8d5e57c"
[[package]]
name = "new_debug_unreachable"
version = "1.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e4a24736216ec316047a1fc4252e27dabb04218aa4a3f37c6e7ddbf1f9782b54"
[[package]]
name = "nextest-workspace-hack"
version = "0.1.0"
@@ -1295,12 +1400,6 @@ dependencies = [
"static_assertions",
]
[[package]]
name = "nohash-hasher"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2bf50223579dc7cdcfb3bfcacf7069ff68243f8c363f62ffa99cf000a6b9c451"
[[package]]
name = "nom"
version = "7.1.3"
@@ -1427,6 +1526,29 @@ version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b15813163c1d831bf4a13c3610c05c0d03b39feb07f7e09fa234dac9b15aaf39"
[[package]]
name = "parking_lot"
version = "0.12.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3742b2c103b9f06bc9fff0a37ff4912935851bee6d36f3c02bcc755bcfec228f"
dependencies = [
"lock_api",
"parking_lot_core",
]
[[package]]
name = "parking_lot_core"
version = "0.9.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "93f00c865fe7cabf650081affecd3871070f26767e7b2070a3ffae14c654b447"
dependencies = [
"cfg-if",
"libc",
"redox_syscall 0.3.5",
"smallvec",
"windows-targets 0.48.1",
]
[[package]]
name = "paste"
version = "1.0.13"
@@ -1518,14 +1640,23 @@ version = "2.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9b2a4787296e9989611394c33f193f676704af1686e70b8f8033ab5ba9a35a94"
[[package]]
name = "petgraph"
version = "0.6.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4dd7d28ee937e54fe3080c91faa1c3a46c06de6252988a7f4592ba2310ef22a4"
dependencies = [
"fixedbitset",
"indexmap 1.9.3",
]
[[package]]
name = "phf"
version = "0.11.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ade2d8b8f33c7333b51bcf0428d37e217e9f32192ae4772156f65063b8ce03dc"
dependencies = [
"phf_macros",
"phf_shared",
"phf_shared 0.11.2",
]
[[package]]
@@ -1535,7 +1666,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e8d39688d359e6b34654d328e262234662d16cc0f60ec8dcbe5e718709342a5a"
dependencies = [
"phf_generator",
"phf_shared",
"phf_shared 0.11.2",
]
[[package]]
@@ -1544,21 +1675,17 @@ version = "0.11.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "48e4cc64c2ad9ebe670cb8fd69dd50ae301650392e81c05f9bfcb2d5bdbc24b0"
dependencies = [
"phf_shared",
"phf_shared 0.11.2",
"rand",
]
[[package]]
name = "phf_macros"
version = "0.11.2"
name = "phf_shared"
version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3444646e286606587e49f3bcf1679b8cef1dc2c5ecc29ddacaffc305180d464b"
checksum = "b6796ad771acdc0123d2a88dc428b5e38ef24456743ddb1744ed628f9815c096"
dependencies = [
"phf_generator",
"phf_shared",
"proc-macro2",
"quote",
"syn 2.0.23",
"siphasher",
]
[[package]]
@@ -1621,6 +1748,18 @@ version = "1.3.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "767eb9f07d4a5ebcb39bbf2d452058a93c011373abf6832e24194a1c3f004794"
[[package]]
name = "ppv-lite86"
version = "0.2.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5b40af805b3121feab8a3c29f04d8ad262fa8e0561883e7653e024ae4479e6de"
[[package]]
name = "precomputed-hash"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "925383efa346730478fb4838dbe9137d2a47675ad789c546d150a6e1dd4ab31c"
[[package]]
name = "predicates"
version = "3.0.3"
@@ -1745,6 +1884,18 @@ version = "0.8.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "34af8d1a0e25924bc5b7c43c079c942339d8f0a8b57c39049bef581b46327404"
dependencies = [
"libc",
"rand_chacha",
"rand_core",
]
[[package]]
name = "rand_chacha"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6c10a63a0fa32252be49d21e7709d4d4baf8d231c2dbce1eaa8141b9b127d88"
dependencies = [
"ppv-lite86",
"rand_core",
]
@@ -1753,6 +1904,9 @@ name = "rand_core"
version = "0.6.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ec0be4795e2f6a28069bec0b5ff3e2ac9bafc99e6a9a7dc3547996c5c816922c"
dependencies = [
"getrandom",
]
[[package]]
name = "rayon"
@@ -1888,7 +2042,7 @@ dependencies = [
[[package]]
name = "ruff"
version = "0.0.280"
version = "0.0.282"
dependencies = [
"annotate-snippets 0.9.1",
"anyhow",
@@ -1909,14 +2063,12 @@ dependencies = [
"log",
"memchr",
"natord",
"nohash-hasher",
"num-bigint",
"num-traits",
"once_cell",
"path-absolutize",
"pathdiff",
"pep440_rs",
"phf",
"pretty_assertions",
"pyproject-toml",
"quick-junit",
@@ -1927,15 +2079,16 @@ dependencies = [
"ruff_index",
"ruff_macros",
"ruff_python_ast",
"ruff_python_codegen",
"ruff_python_index",
"ruff_python_literal",
"ruff_python_parser",
"ruff_python_semantic",
"ruff_python_stdlib",
"ruff_python_trivia",
"ruff_rustpython",
"ruff_source_file",
"ruff_text_size",
"ruff_textwrap",
"rustc-hash",
"rustpython-format",
"rustpython-parser",
"schemars",
"semver",
"serde",
@@ -1966,7 +2119,7 @@ dependencies = [
"ruff",
"ruff_python_ast",
"ruff_python_formatter",
"rustpython-parser",
"ruff_python_parser",
"serde",
"serde_json",
"tikv-jemallocator",
@@ -1988,7 +2141,7 @@ dependencies = [
[[package]]
name = "ruff_cli"
version = "0.0.280"
version = "0.0.282"
dependencies = [
"annotate-snippets 0.9.1",
"anyhow",
@@ -2017,11 +2170,13 @@ dependencies = [
"ruff",
"ruff_cache",
"ruff_diagnostics",
"ruff_macros",
"ruff_python_ast",
"ruff_python_formatter",
"ruff_python_stdlib",
"ruff_python_trivia",
"ruff_source_file",
"ruff_text_size",
"ruff_textwrap",
"rustc-hash",
"serde",
"serde_json",
@@ -2046,7 +2201,6 @@ dependencies = [
"indoc",
"itertools",
"libcst",
"log",
"once_cell",
"pretty_assertions",
"rayon",
@@ -2055,11 +2209,13 @@ dependencies = [
"ruff_cli",
"ruff_diagnostics",
"ruff_formatter",
"ruff_python_ast",
"ruff_python_codegen",
"ruff_python_formatter",
"ruff_python_literal",
"ruff_python_parser",
"ruff_python_stdlib",
"ruff_textwrap",
"rustpython-format",
"rustpython-parser",
"ruff_python_trivia",
"schemars",
"serde",
"serde_json",
@@ -2068,6 +2224,9 @@ dependencies = [
"strum_macros",
"tempfile",
"toml",
"tracing",
"tracing-indicatif",
"tracing-subscriber",
]
[[package]]
@@ -2110,7 +2269,7 @@ dependencies = [
"itertools",
"proc-macro2",
"quote",
"ruff_textwrap",
"ruff_python_trivia",
"syn 2.0.23",
]
@@ -2118,24 +2277,32 @@ dependencies = [
name = "ruff_python_ast"
version = "0.0.0"
dependencies = [
"anyhow",
"bitflags 2.3.3",
"insta",
"is-macro",
"itertools",
"log",
"memchr",
"num-bigint",
"num-traits",
"once_cell",
"ruff_python_parser",
"ruff_python_trivia",
"ruff_source_file",
"ruff_text_size",
"rustc-hash",
"rustpython-ast",
"rustpython-literal",
"rustpython-parser",
"serde",
"smallvec",
"static_assertions",
]
[[package]]
name = "ruff_python_codegen"
version = "0.0.0"
dependencies = [
"once_cell",
"ruff_python_ast",
"ruff_python_literal",
"ruff_python_parser",
"ruff_source_file",
]
[[package]]
@@ -2152,10 +2319,12 @@ dependencies = [
"once_cell",
"ruff_formatter",
"ruff_python_ast",
"ruff_python_index",
"ruff_python_parser",
"ruff_python_trivia",
"ruff_source_file",
"ruff_text_size",
"rustc-hash",
"rustpython-parser",
"serde",
"serde_json",
"similar",
@@ -2163,6 +2332,55 @@ dependencies = [
"thiserror",
]
[[package]]
name = "ruff_python_index"
version = "0.0.0"
dependencies = [
"itertools",
"ruff_python_ast",
"ruff_python_parser",
"ruff_python_trivia",
"ruff_source_file",
"ruff_text_size",
]
[[package]]
name = "ruff_python_literal"
version = "0.0.0"
dependencies = [
"bitflags 2.3.3",
"hexf-parse",
"is-macro",
"itertools",
"lexical-parse-float",
"num-bigint",
"num-traits",
"rand",
"unic-ucd-category",
]
[[package]]
name = "ruff_python_parser"
version = "0.0.0"
dependencies = [
"anyhow",
"insta",
"is-macro",
"itertools",
"lalrpop",
"lalrpop-util",
"num-bigint",
"num-traits",
"ruff_python_ast",
"ruff_text_size",
"rustc-hash",
"static_assertions",
"tiny-keccak",
"unic-emoji-char",
"unic-ucd-ident",
"unicode_names2",
]
[[package]]
name = "ruff_python_resolver"
version = "0.0.0"
@@ -2179,14 +2397,13 @@ version = "0.0.0"
dependencies = [
"bitflags 2.3.3",
"is-macro",
"nohash-hasher",
"num-traits",
"ruff_index",
"ruff_python_ast",
"ruff_python_stdlib",
"ruff_source_file",
"ruff_text_size",
"rustc-hash",
"rustpython-parser",
"smallvec",
]
@@ -2200,19 +2417,14 @@ version = "0.0.0"
dependencies = [
"insta",
"memchr",
"ruff_python_ast",
"ruff_python_parser",
"ruff_source_file",
"ruff_text_size",
"smallvec",
"unic-ucd-ident",
]
[[package]]
name = "ruff_rustpython"
version = "0.0.0"
dependencies = [
"anyhow",
"rustpython-parser",
]
[[package]]
name = "ruff_shrinking"
version = "0.1.0"
@@ -2222,28 +2434,32 @@ dependencies = [
"fs-err",
"regex",
"ruff_python_ast",
"ruff_rustpython",
"rustpython-ast",
"ruff_python_parser",
"ruff_text_size",
"shlex",
"tracing",
"tracing-subscriber",
]
[[package]]
name = "ruff_text_size"
name = "ruff_source_file"
version = "0.0.0"
source = "git+https://github.com/astral-sh/RustPython-Parser.git?rev=4d03b9b5b212fc869e4cfda151414438186a7779#4d03b9b5b212fc869e4cfda151414438186a7779"
dependencies = [
"schemars",
"insta",
"memchr",
"once_cell",
"ruff_text_size",
"serde",
]
[[package]]
name = "ruff_textwrap"
name = "ruff_text_size"
version = "0.0.0"
dependencies = [
"ruff_python_trivia",
"ruff_text_size",
"schemars",
"serde",
"serde_test",
"static_assertions",
]
[[package]]
@@ -2257,9 +2473,11 @@ dependencies = [
"ruff",
"ruff_diagnostics",
"ruff_python_ast",
"ruff_python_codegen",
"ruff_python_formatter",
"ruff_rustpython",
"rustpython-parser",
"ruff_python_index",
"ruff_python_parser",
"ruff_source_file",
"serde",
"serde-wasm-bindgen",
"wasm-bindgen",
@@ -2331,74 +2549,6 @@ dependencies = [
"untrusted",
]
[[package]]
name = "rustpython-ast"
version = "0.2.0"
source = "git+https://github.com/astral-sh/RustPython-Parser.git?rev=4d03b9b5b212fc869e4cfda151414438186a7779#4d03b9b5b212fc869e4cfda151414438186a7779"
dependencies = [
"is-macro",
"num-bigint",
"rustpython-parser-core",
"static_assertions",
]
[[package]]
name = "rustpython-format"
version = "0.2.0"
source = "git+https://github.com/astral-sh/RustPython-Parser.git?rev=4d03b9b5b212fc869e4cfda151414438186a7779#4d03b9b5b212fc869e4cfda151414438186a7779"
dependencies = [
"bitflags 2.3.3",
"itertools",
"num-bigint",
"num-traits",
"rustpython-literal",
]
[[package]]
name = "rustpython-literal"
version = "0.2.0"
source = "git+https://github.com/astral-sh/RustPython-Parser.git?rev=4d03b9b5b212fc869e4cfda151414438186a7779#4d03b9b5b212fc869e4cfda151414438186a7779"
dependencies = [
"hexf-parse",
"is-macro",
"lexical-parse-float",
"num-traits",
"unic-ucd-category",
]
[[package]]
name = "rustpython-parser"
version = "0.2.0"
source = "git+https://github.com/astral-sh/RustPython-Parser.git?rev=4d03b9b5b212fc869e4cfda151414438186a7779#4d03b9b5b212fc869e4cfda151414438186a7779"
dependencies = [
"anyhow",
"is-macro",
"itertools",
"lalrpop-util",
"log",
"num-bigint",
"num-traits",
"phf",
"phf_codegen",
"rustc-hash",
"rustpython-ast",
"rustpython-parser-core",
"tiny-keccak",
"unic-emoji-char",
"unic-ucd-ident",
"unicode_names2",
]
[[package]]
name = "rustpython-parser-core"
version = "0.2.0"
source = "git+https://github.com/astral-sh/RustPython-Parser.git?rev=4d03b9b5b212fc869e4cfda151414438186a7779#4d03b9b5b212fc869e4cfda151414438186a7779"
dependencies = [
"is-macro",
"memchr",
"ruff_text_size",
]
[[package]]
name = "rustversion"
version = "1.0.13"
@@ -2534,6 +2684,15 @@ dependencies = [
"serde",
]
[[package]]
name = "serde_test"
version = "1.0.176"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5a2f49ace1498612d14f7e0b8245519584db8299541dfe31a06374a828d620ab"
dependencies = [
"serde",
]
[[package]]
name = "serde_with"
version = "3.0.0"
@@ -2616,6 +2775,19 @@ version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a2eb9349b6444b326872e140eb1cf5e7c522154d69e7a0ffb0fb81c06b37543f"
[[package]]
name = "string_cache"
version = "0.8.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f91138e76242f575eb1d3b38b4f1362f10d3a43f47d182a5b359af488a02293b"
dependencies = [
"new_debug_unreachable",
"once_cell",
"parking_lot",
"phf_shared 0.10.0",
"precomputed-hash",
]
[[package]]
name = "strsim"
version = "0.10.0"
@@ -2689,6 +2861,17 @@ dependencies = [
"windows-sys 0.48.0",
]
[[package]]
name = "term"
version = "0.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c59df8ac95d96ff9bede18eb7300b0fda5e5d8d90960e76f8e14ae765eedbf1f"
dependencies = [
"dirs-next",
"rustversion",
"winapi",
]
[[package]]
name = "termcolor"
version = "1.2.0"
@@ -2942,6 +3125,18 @@ dependencies = [
"valuable",
]
[[package]]
name = "tracing-indicatif"
version = "0.3.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b38ed3722d27705c3bd7ca0ccf29acc3d8e1c717b4cd87f97891a2c1834ea1af"
dependencies = [
"indicatif",
"tracing",
"tracing-core",
"tracing-subscriber",
]
[[package]]
name = "tracing-log"
version = "0.1.3"
@@ -3068,6 +3263,12 @@ version = "0.1.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c0edd1e5b14653f783770bce4a4dabb4a5108a5370a5f5d8cfe8710c361f6c8b"
[[package]]
name = "unicode-xid"
version = "0.2.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f962df74c8c05a667b5ee8bcf162993134c104e96440b663c8daa176dc772d8c"
[[package]]
name = "unicode_names2"
version = "0.6.0"
@@ -3134,6 +3335,39 @@ version = "0.9.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "49874b5167b65d7193b8aba1567f5c7d93d001cafc34600cee003eda787e483f"
[[package]]
name = "vt100"
version = "0.15.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "84cd863bf0db7e392ba3bd04994be3473491b31e66340672af5d11943c6274de"
dependencies = [
"itoa",
"log",
"unicode-width",
"vte",
]
[[package]]
name = "vte"
version = "0.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f5022b5fbf9407086c180e9557be968742d839e68346af7792b8592489732197"
dependencies = [
"arrayvec",
"utf8parse",
"vte_generate_state_changes",
]
[[package]]
name = "vte_generate_state_changes"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d257817081c7dffcdbab24b9e62d2def62e2ff7d00b1c20062551e6cccc145ff"
dependencies = [
"proc-macro2",
"quote",
]
[[package]]
name = "wait-timeout"
version = "0.2.0"

View File

@@ -4,7 +4,7 @@ resolver = "2"
[workspace.package]
edition = "2021"
rust-version = "1.70"
rust-version = "1.71"
homepage = "https://beta.ruff.rs/docs"
documentation = "https://beta.ruff.rs/docs"
repository = "https://github.com/astral-sh/ruff"
@@ -26,7 +26,6 @@ is-macro = { version = "0.2.2" }
itertools = { version = "0.10.5" }
log = { version = "0.4.17" }
memchr = "2.5.0"
nohash-hasher = { version = "0.2.0" }
num-bigint = { version = "0.4.3" }
num-traits = { version = "0.2.15" }
once_cell = { version = "1.17.1" }
@@ -47,17 +46,14 @@ syn = { version = "2.0.15" }
test-case = { version = "3.0.0" }
thiserror = { version = "1.0.43" }
toml = { version = "0.7.2" }
tracing = "0.1.37"
tracing-indicatif = "0.3.4"
tracing-subscriber = { version = "0.3.17", features = ["env-filter"] }
wsl = { version = "0.1.0" }
# v1.0.1
libcst = { git = "https://github.com/Instagram/LibCST.git", rev = "3cacca1a1029f05707e50703b49fe3dd860aa839", default-features = false }
ruff_text_size = { git = "https://github.com/astral-sh/RustPython-Parser.git", rev = "4d03b9b5b212fc869e4cfda151414438186a7779" }
rustpython-ast = { git = "https://github.com/astral-sh/RustPython-Parser.git", rev = "4d03b9b5b212fc869e4cfda151414438186a7779" , default-features = false, features = ["num-bigint"]}
rustpython-format = { git = "https://github.com/astral-sh/RustPython-Parser.git", rev = "4d03b9b5b212fc869e4cfda151414438186a7779", default-features = false, features = ["num-bigint"] }
rustpython-literal = { git = "https://github.com/astral-sh/RustPython-Parser.git", rev = "4d03b9b5b212fc869e4cfda151414438186a7779", default-features = false }
rustpython-parser = { git = "https://github.com/astral-sh/RustPython-Parser.git", rev = "4d03b9b5b212fc869e4cfda151414438186a7779" , default-features = false, features = ["full-lexer", "num-bigint"] }
[profile.release]
lto = "fat"
codegen-units = 1
@@ -70,7 +66,7 @@ opt-level = 3
# Reduce complexity of a parser function that would trigger a locals limit in a wasm tool.
# https://github.com/bytecodealliance/wasm-tools/blob/b5c3d98e40590512a3b12470ef358d5c7b983b15/crates/wasmparser/src/limits.rs#L29
[profile.dev.package.rustpython-parser]
[profile.dev.package.ruff_python_parser]
opt-level = 1
# Use the `--profile release-debug` flag to show symbols in release mode.

View File

@@ -2,7 +2,7 @@
# Ruff
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/charliermarsh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
[![image](https://img.shields.io/pypi/v/ruff.svg)](https://pypi.python.org/pypi/ruff)
[![image](https://img.shields.io/pypi/l/ruff.svg)](https://pypi.python.org/pypi/ruff)
[![image](https://img.shields.io/pypi/pyversions/ruff.svg)](https://pypi.python.org/pypi/ruff)
@@ -140,7 +140,7 @@ Ruff can also be used as a [pre-commit](https://pre-commit.com) hook:
```yaml
- repo: https://github.com/astral-sh/ruff-pre-commit
# Ruff version.
rev: v0.0.280
rev: v0.0.282
hooks:
- id: ruff
```
@@ -424,13 +424,13 @@ Ruff is used by a number of major open-source projects and companies, including:
If you're using Ruff, consider adding the Ruff badge to project's `README.md`:
```md
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/charliermarsh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
```
...or `README.rst`:
```rst
.. image:: https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/charliermarsh/ruff/main/assets/badge/v2.json
.. image:: https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json
:target: https://github.com/astral-sh/ruff
:alt: Ruff
```
@@ -438,7 +438,7 @@ If you're using Ruff, consider adding the Ruff badge to project's `README.md`:
...or, as HTML:
```html
<a href="https://github.com/astral-sh/ruff"><img src="https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/charliermarsh/ruff/main/assets/badge/v2.json" alt="Ruff" style="max-width:100%;"></a>
<a href="https://github.com/astral-sh/ruff"><img src="https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json" alt="Ruff" style="max-width:100%;"></a>
```
## License
@@ -447,6 +447,6 @@ MIT
<div align="center">
<a target="_blank" href="https://astral.sh" style="background:none">
<img src="https://raw.githubusercontent.com/charliermarsh/ruff/main/assets/svg/Astral.svg">
<img src="https://raw.githubusercontent.com/astral-sh/ruff/main/assets/svg/Astral.svg">
</a>
</div>

View File

@@ -1,6 +1,6 @@
[package]
name = "flake8-to-ruff"
version = "0.0.280"
version = "0.0.282"
description = """
Convert Flake8 configuration files to Ruff configuration files.
"""

View File

@@ -1,6 +1,6 @@
[package]
name = "ruff"
version = "0.0.280"
version = "0.0.282"
publish = false
authors = { workspace = true }
edition = { workspace = true }
@@ -19,13 +19,16 @@ ruff_cache = { path = "../ruff_cache" }
ruff_diagnostics = { path = "../ruff_diagnostics", features = ["serde"] }
ruff_index = { path = "../ruff_index" }
ruff_macros = { path = "../ruff_macros" }
ruff_python_trivia = { path = "../ruff_python_trivia" }
ruff_python_ast = { path = "../ruff_python_ast", features = ["serde"] }
ruff_python_codegen = { path = "../ruff_python_codegen" }
ruff_python_index = { path = "../ruff_python_index" }
ruff_python_literal = { path = "../ruff_python_literal" }
ruff_python_semantic = { path = "../ruff_python_semantic" }
ruff_python_stdlib = { path = "../ruff_python_stdlib" }
ruff_rustpython = { path = "../ruff_rustpython" }
ruff_text_size = { workspace = true }
ruff_textwrap = { path = "../ruff_textwrap" }
ruff_python_trivia = { path = "../ruff_python_trivia" }
ruff_python_parser = { path = "../ruff_python_parser" }
ruff_source_file = { path = "../ruff_source_file", features = ["serde"] }
ruff_text_size = { path = "../ruff_text_size" }
annotate-snippets = { version = "0.9.1", features = ["color"] }
anyhow = { workspace = true }
@@ -45,7 +48,6 @@ libcst = { workspace = true }
log = { workspace = true }
memchr = { workspace = true }
natord = { version = "1.0.9" }
nohash-hasher = { workspace = true }
num-bigint = { workspace = true }
num-traits = { workspace = true }
once_cell = { workspace = true }
@@ -55,14 +57,13 @@ path-absolutize = { workspace = true, features = [
] }
pathdiff = { version = "0.2.1" }
pep440_rs = { version = "0.3.1", features = ["serde"] }
phf = { version = "0.11", features = ["macros"] }
pyproject-toml = { version = "0.6.0" }
quick-junit = { version = "0.3.2" }
regex = { workspace = true }
result-like = { version = "0.4.6" }
rustc-hash = { workspace = true }
rustpython-format = { workspace = true }
rustpython-parser = { workspace = true }
schemars = { workspace = true, optional = true }
semver = { version = "1.0.16" }
serde = { workspace = true }

View File

@@ -67,7 +67,8 @@ cfg.getboolean("hello", True)
os.set_blocking(0, False)
g_action.set_enabled(True)
settings.set_enable_developer_extras(True)
foo.is_(True)
bar.is_not(False)
class Registry:
def __init__(self) -> None:

View File

@@ -1,6 +1,6 @@
"""
Should emit:
B002 - on lines 15 and 20
B002 - on lines 18, 19, and 24
"""
@@ -8,13 +8,17 @@ def this_is_all_fine(n):
x = n + 1
y = 1 + n
z = +x + y
return +z
a = n - 1
b = 1 - n
c = -a - b
return +z, -c
def this_is_buggy(n):
x = ++n
return x
y = --n
return x, y
def this_is_buggy_too(n):
return ++n
return ++n, --n

View File

@@ -17,3 +17,37 @@ from typing import TypedDict
class MyClass(TypedDict):
id: int
from threading import Event
class CustomEvent(Event):
def set(self) -> None:
...
def str(self) -> None:
...
from logging import Filter, LogRecord
class CustomFilter(Filter):
def filter(self, record: LogRecord) -> bool:
...
def str(self) -> None:
...
from typing_extensions import override
class MyClass:
@override
def str(self):
pass
def int(self):
pass

View File

@@ -0,0 +1,2 @@
#!/usr/bin/env python

View File

@@ -50,3 +50,12 @@ _ = """a""" "b"
_ = 'a' "b"
_ = rf"a" rf"b"
# Single-line explicit concatenation should be ignored.
_ = "abc" + "def" + "ghi"
_ = foo + "abc" + "def"
_ = "abc" + foo + "def"
_ = "abc" + "def" + foo
_ = foo + bar + "abc"
_ = "abc" + foo + bar
_ = foo + "abc" + bar

View File

@@ -1,5 +1,10 @@
import logging
from distutils import log
from logging_setup import logger
logging.warn("Hello World!")
log.warn("Hello world!") # This shouldn't be considered as a logger candidate
logger.warn("Hello world!")
logging . warn("Hello World!")

View File

@@ -0,0 +1,12 @@
import typing
from typing import TypeVar
_T = typing.TypeVar("_T")
_P = TypeVar("_P")
# OK
_UsedTypeVar = TypeVar("_UsedTypeVar")
def func(arg: _UsedTypeVar) -> _UsedTypeVar: ...
_A, _B = TypeVar("_A"), TypeVar("_B")
_C = _D = TypeVar("_C")

View File

@@ -0,0 +1,12 @@
import typing
from typing import TypeVar
_T = typing.TypeVar("_T")
_P = TypeVar("_P")
# OK
_UsedTypeVar = TypeVar("_UsedTypeVar")
def func(arg: _UsedTypeVar) -> _UsedTypeVar: ...
_A, _B = TypeVar("_A"), TypeVar("_B")
_C = _D = TypeVar("_C")

View File

@@ -0,0 +1,42 @@
from typing import TypeVar, Self, Type
_S = TypeVar("_S", bound=BadClass)
_S2 = TypeVar("_S2", BadClass, GoodClass)
class BadClass:
def __new__(cls: type[_S], *args: str, **kwargs: int) -> _S: ... # PYI019
def bad_instance_method(self: _S, arg: bytes) -> _S: ... # PYI019
@classmethod
def bad_class_method(cls: type[_S], arg: int) -> _S: ... # PYI019
@classmethod
def excluded_edge_case(cls: Type[_S], arg: int) -> _S: ... # Ok
class GoodClass:
def __new__(cls: type[Self], *args: list[int], **kwargs: set[str]) -> Self: ...
def good_instance_method_1(self: Self, arg: bytes) -> Self: ...
def good_instance_method_2(self, arg1: _S2, arg2: _S2) -> _S2: ...
@classmethod
def good_cls_method_1(cls: type[Self], arg: int) -> Self: ...
@classmethod
def good_cls_method_2(cls, arg1: _S, arg2: _S) -> _S: ...
@staticmethod
def static_method(arg1: _S) -> _S: ...
# Python > 3.12
class PEP695BadDunderNew[T]:
def __new__[S](cls: type[S], *args: Any, ** kwargs: Any) -> S: ... # PYI019
def generic_instance_method[S](self: S) -> S: ... # PYI019
class PEP695GoodDunderNew[T]:
def __new__(cls, *args: Any, **kwargs: Any) -> Self: ...

View File

@@ -0,0 +1,42 @@
from typing import TypeVar, Self, Type
_S = TypeVar("_S", bound=BadClass)
_S2 = TypeVar("_S2", BadClass, GoodClass)
class BadClass:
def __new__(cls: type[_S], *args: str, **kwargs: int) -> _S: ... # PYI019
def bad_instance_method(self: _S, arg: bytes) -> _S: ... # PYI019
@classmethod
def bad_class_method(cls: type[_S], arg: int) -> _S: ... # PYI019
@classmethod
def excluded_edge_case(cls: Type[_S], arg: int) -> _S: ... # Ok
class GoodClass:
def __new__(cls: type[Self], *args: list[int], **kwargs: set[str]) -> Self: ...
def good_instance_method_1(self: Self, arg: bytes) -> Self: ...
def good_instance_method_2(self, arg1: _S2, arg2: _S2) -> _S2: ...
@classmethod
def good_cls_method_1(cls: type[Self], arg: int) -> Self: ...
@classmethod
def good_cls_method_2(cls, arg1: _S, arg2: _S) -> _S: ...
@staticmethod
def static_method(arg1: _S) -> _S: ...
# Python > 3.12
class PEP695BadDunderNew[T]:
def __new__[S](cls: type[S], *args: Any, ** kwargs: Any) -> S: ... # PYI019
def generic_instance_method[S](self: S) -> S: ... # PYI019
class PEP695GoodDunderNew[T]:
def __new__(cls, *args: Any, **kwargs: Any) -> Self: ...

View File

@@ -1,9 +1,11 @@
import collections
person: collections.namedtuple # OK
person: collections.namedtuple # Y024 Use "typing.NamedTuple" instead of "collections.namedtuple"
from collections import namedtuple
person: namedtuple # OK
person: namedtuple # Y024 Use "typing.NamedTuple" instead of "collections.namedtuple"
person = namedtuple("Person", ["name", "age"]) # OK
person = namedtuple(
"Person", ["name", "age"]
) # Y024 Use "typing.NamedTuple" instead of "collections.namedtuple"

View File

@@ -1,24 +1,38 @@
import typing
import typing_extensions
from typing import Literal
# Shouldn't emit for any cases in the non-stub file for compatibility with flake8-pyi.
# Note that this rule could be applied here in the future.
# Shouldn't affect non-union field types.
field1: Literal[1] # OK
field2: Literal[1] | Literal[2] # OK
def func1(arg1: Literal[1] | Literal[2]): # OK
# Should emit for duplicate field types.
field2: Literal[1] | Literal[2] # Error
# Should emit for union types in arguments.
def func1(arg1: Literal[1] | Literal[2]): # Error
print(arg1)
def func2() -> Literal[1] | Literal[2]: # OK
# Should emit for unions in return types.
def func2() -> Literal[1] | Literal[2]: # Error
return "my Literal[1]ing"
field3: Literal[1] | Literal[2] | str # OK
field4: str | Literal[1] | Literal[2] # OK
field5: Literal[1] | str | Literal[2] # OK
field6: Literal[1] | bool | Literal[2] | str # OK
field7 = Literal[1] | Literal[2] # OK
field8: Literal[1] | (Literal[2] | str) # OK
field9: Literal[1] | (Literal[2] | str) # OK
field10: (Literal[1] | str) | Literal[2] # OK
field11: dict[Literal[1] | Literal[2], str] # OK
# Should emit in longer unions, even if not directly adjacent.
field3: Literal[1] | Literal[2] | str # Error
field4: str | Literal[1] | Literal[2] # Error
field5: Literal[1] | str | Literal[2] # Error
field6: Literal[1] | bool | Literal[2] | str # Error
# Should emit for non-type unions.
field7 = Literal[1] | Literal[2] # Error
# Should emit for parenthesized unions.
field8: Literal[1] | (Literal[2] | str) # Error
# Should handle user parentheses when fixing.
field9: Literal[1] | (Literal[2] | str) # Error
field10: (Literal[1] | str) | Literal[2] # Error
# Should emit for union in generic parent type.
field11: dict[Literal[1] | Literal[2], str] # Error

View File

@@ -3,8 +3,8 @@ import typing
class Bad:
def __eq__(self, other: Any) -> bool: ... # Fine because not a stub file
def __ne__(self, other: typing.Any) -> typing.Any: ... # Fine because not a stub file
def __eq__(self, other: Any) -> bool: ... # Y032
def __ne__(self, other: typing.Any) -> typing.Any: ... # Y032
class Good:

View File

@@ -9,16 +9,16 @@ from typing import (
just_literals_pipe_union: TypeAlias = (
Literal[True] | Literal["idk"]
) # not PYI042 (not a stubfile)
) # PYI042, since not camel case
PublicAliasT: TypeAlias = str | int
PublicAliasT2: TypeAlias = Union[str, bytes]
_ABCDEFGHIJKLMNOPQRST: TypeAlias = typing.Any
_PrivateAliasS: TypeAlias = Literal["I", "guess", "this", "is", "okay"]
_PrivateAliasS2: TypeAlias = Annotated[str, "also okay"]
snake_case_alias1: TypeAlias = str | int # not PYI042 (not a stubfile)
_snake_case_alias2: TypeAlias = Literal["whatever"] # not PYI042 (not a stubfile)
Snake_case_alias: TypeAlias = int | float # not PYI042 (not a stubfile)
snake_case_alias1: TypeAlias = str | int # PYI042, since not camel case
_snake_case_alias2: TypeAlias = Literal["whatever"] # PYI042, since not camel case
Snake_case_alias: TypeAlias = int | float # PYI042, since not camel case
# check that this edge case doesn't crash
_: TypeAlias = str | int

View File

@@ -7,11 +7,11 @@ from typing import (
Literal,
)
_PrivateAliasT: TypeAlias = str | int # not PYI043 (not a stubfile)
_PrivateAliasT2: TypeAlias = typing.Any # not PYI043 (not a stubfile)
_PrivateAliasT: TypeAlias = str | int # PYI043, since this ends in a T
_PrivateAliasT2: TypeAlias = typing.Any # PYI043, since this ends in a T
_PrivateAliasT3: TypeAlias = Literal[
"not", "a", "chance"
] # not PYI043 (not a stubfile)
] # PYI043, since this ends in a T
just_literals_pipe_union: TypeAlias = Literal[True] | Literal["idk"]
PublicAliasT: TypeAlias = str | int
PublicAliasT2: TypeAlias = Union[str, bytes]

View File

@@ -0,0 +1,18 @@
import typing
from typing import Protocol
class _Foo(Protocol):
bar: int
class _Bar(typing.Protocol):
bar: int
# OK
class _UsedPrivateProtocol(Protocol):
bar: int
def uses__UsedPrivateProtocol(arg: _UsedPrivateProtocol) -> None: ...

View File

@@ -0,0 +1,18 @@
import typing
from typing import Protocol
class _Foo(object, Protocol):
bar: int
class _Bar(typing.Protocol):
bar: int
# OK
class _UsedPrivateProtocol(Protocol):
bar: int
def uses__UsedPrivateProtocol(arg: _UsedPrivateProtocol) -> None: ...

View File

@@ -0,0 +1,22 @@
import typing
import sys
from typing import TypeAlias
_UnusedPrivateTypeAlias: TypeAlias = int | None
_T: typing.TypeAlias = str
# OK
_UsedPrivateTypeAlias: TypeAlias = int | None
def func(arg: _UsedPrivateTypeAlias) -> _UsedPrivateTypeAlias:
...
if sys.version_info > (3, 9):
_PrivateTypeAlias: TypeAlias = str | None
else:
_PrivateTypeAlias: TypeAlias = float | None
def func2(arg: _PrivateTypeAlias) -> None: ...

View File

@@ -0,0 +1,22 @@
import typing
import sys
from typing import TypeAlias
_UnusedPrivateTypeAlias: TypeAlias = int | None
_T: typing.TypeAlias = str
# OK
_UsedPrivateTypeAlias: TypeAlias = int | None
def func(arg: _UsedPrivateTypeAlias) -> _UsedPrivateTypeAlias:
...
if sys.version_info > (3, 9):
_PrivateTypeAlias: TypeAlias = str | None
else:
_PrivateTypeAlias: TypeAlias = float | None
def func2(arg: _PrivateTypeAlias) -> None: ...

View File

@@ -0,0 +1,18 @@
import typing
from typing import TypedDict
class _UnusedTypedDict(TypedDict):
foo: str
class _UnusedTypedDict2(typing.TypedDict):
bar: int
class _UsedTypedDict(TypedDict):
foo: bytes
class _CustomClass(_UsedTypedDict):
bar: list[int]

View File

@@ -0,0 +1,32 @@
import sys
import typing
from typing import TypedDict
class _UnusedTypedDict(TypedDict):
foo: str
class _UnusedTypedDict2(typing.TypedDict):
bar: int
# OK
class _UsedTypedDict(TypedDict):
foo: bytes
class _CustomClass(_UsedTypedDict):
bar: list[int]
if sys.version_info >= (3, 10):
class _UsedTypedDict2(TypedDict):
foo: int
else:
class _UsedTypedDict2(TypedDict):
foo: float
class _CustomClass2(_UsedTypedDict2):
bar: list[int]

View File

@@ -0,0 +1,17 @@
import typing
from typing import Literal, TypeAlias, Union
A: str | Literal["foo"]
B: TypeAlias = typing.Union[Literal[b"bar", b"foo"], bytes, str]
C: TypeAlias = typing.Union[Literal[5], int, typing.Union[Literal["foo"], str]]
D: TypeAlias = typing.Union[Literal[b"str_bytes", 42], bytes, int]
def func(x: complex | Literal[1J], y: Union[Literal[3.14], float]): ...
# OK
A: Literal["foo"]
B: TypeAlias = Literal[b"bar", b"foo"]
C: TypeAlias = typing.Union[Literal[5], Literal["foo"]]
D: TypeAlias = Literal[b"str_bytes", 42]
def func(x: Literal[1J], y: Literal[3.14]): ...

View File

@@ -0,0 +1,17 @@
import typing
from typing import Literal, TypeAlias, Union
A: str | Literal["foo"]
B: TypeAlias = typing.Union[Literal[b"bar", b"foo"], bytes, str]
C: TypeAlias = typing.Union[Literal[5], int, typing.Union[Literal["foo"], str]]
D: TypeAlias = typing.Union[Literal[b"str_bytes", 42], bytes, int]
def func(x: complex | Literal[1J], y: Union[Literal[3.14], float]): ...
# OK
A: Literal["foo"]
B: TypeAlias = Literal[b"bar", b"foo"]
C: TypeAlias = typing.Union[Literal[5], Literal["foo"]]
D: TypeAlias = Literal[b"str_bytes", 42]
def func(x: Literal[1J], y: Literal[3.14]): ...

View File

@@ -0,0 +1,20 @@
import builtins
from typing import Union
w: builtins.type[int] | builtins.type[str] | builtins.type[complex]
x: type[int] | type[str] | type[float]
y: builtins.type[int] | type[str] | builtins.type[complex]
z: Union[type[float], type[complex]]
z: Union[type[float, int], type[complex]]
def func(arg: type[int] | str | type[float]) -> None: ...
# OK
x: type[int, str, float]
y: builtins.type[int, str, complex]
z: Union[float, complex]
def func(arg: type[int, float] | str) -> None: ...

View File

@@ -0,0 +1,20 @@
import builtins
from typing import Union
w: builtins.type[int] | builtins.type[str] | builtins.type[complex]
x: type[int] | type[str] | type[float]
y: builtins.type[int] | type[str] | builtins.type[complex]
z: Union[type[float], type[complex]]
z: Union[type[float, int], type[complex]]
def func(arg: type[int] | str | type[float]) -> None: ...
# OK
x: type[int, str, float]
y: builtins.type[int, str, complex]
z: Union[float, complex]
def func(arg: type[int, float] | str) -> None: ...

View File

@@ -0,0 +1,12 @@
__all__ = ["A", "B", "C"]
# Errors
__all__.append("D")
__all__.extend(["E", "Foo"])
__all__.remove("A")
# OK
__all__ += ["D"]
foo = ["Hello"]
foo.append("World")
foo.bar.append("World")

View File

@@ -0,0 +1,12 @@
__all__ = ["A", "B", "C"]
# Errors
__all__.append("D")
__all__.extend(["E", "Foo"])
__all__.remove("A")
# OK
__all__ += ["D"]
foo = ["Hello"]
foo.append("World")
foo.bar.append("World")

View File

@@ -11,6 +11,10 @@ async def test_ok_trivial_with():
with context_manager_under_test():
pass
with pytest.raises(ValueError):
with context_manager_under_test():
raise ValueError
with pytest.raises(AttributeError):
async with context_manager_under_test():
pass
@@ -24,6 +28,16 @@ def test_ok_complex_single_call():
)
def test_ok_func_and_class():
with pytest.raises(AttributeError):
class A:
pass
with pytest.raises(AttributeError):
def f():
pass
def test_error_multiple_statements():
with pytest.raises(AttributeError):
len([])
@@ -47,13 +61,10 @@ async def test_error_complex_statement():
while True:
[].size
with pytest.raises(AttributeError):
with context_manager_under_test():
[].size
with pytest.raises(AttributeError):
async with context_manager_under_test():
[].size
if True:
raise Exception
def test_error_try():

View File

@@ -156,3 +156,12 @@ if False:
if True:
if a:
pass
# SIM102
def f():
if a:
pass
elif b:
if c:
d

View File

@@ -14,6 +14,12 @@ try:
except (ValueError, OSError):
pass
# SIM105
try:
foo()
except (ValueError, OSError) as e:
pass
# SIM105
try:
foo()
@@ -94,3 +100,13 @@ def with_comment():
foo()
except (ValueError, OSError):
pass # Trailing comment.
try:
print()
except ("not", "an", "exception"):
pass
try:
print()
except "not an exception":
pass

View File

@@ -101,3 +101,16 @@ if a:
x = 1
elif c:
x = 1
def foo():
a = True
b = False
if a > b: # end-of-line
return 3
elif a == b:
return 3
elif a < b: # end-of-line
return 4
elif b is None:
return 4

View File

@@ -1,7 +1,15 @@
import contextlib
import pathlib
import pathlib as pl
from pathlib import Path
from pathlib import Path as P
# SIM115
f = open("foo.txt")
f = Path("foo.txt").open()
f = pathlib.Path("foo.txt").open()
f = pl.Path("foo.txt").open()
f = P("foo.txt").open()
data = f.read()
f.close()

View File

@@ -1,11 +1,19 @@
key in obj.keys() # SIM118
key not in obj.keys() # SIM118
foo["bar"] in obj.keys() # SIM118
foo["bar"] not in obj.keys() # SIM118
foo['bar'] in obj.keys() # SIM118
foo['bar'] not in obj.keys() # SIM118
foo() in obj.keys() # SIM118
foo() not in obj.keys() # SIM118
for key in obj.keys(): # SIM118
pass
@@ -22,3 +30,13 @@ for key in list(obj.keys()):
(k for k in obj.keys()) # SIM118
key in (obj or {}).keys() # SIM118
from typing import KeysView
class Foo:
def keys(self) -> KeysView[object]:
...
def __contains__(self, key: object) -> bool:
return key in self.keys() # OK

View File

@@ -0,0 +1,10 @@
"""Regression test: ensure that we don't treat the export entry as a typing-only reference."""
from __future__ import annotations
from logging import getLogger
__all__ = ("getLogger",)
def foo() -> None:
pass

View File

@@ -5,6 +5,7 @@ from pathlib import Path as pth
_ = Path(".")
_ = pth(".")
_ = PurePath(".")
_ = Path("")
# no match
_ = Path()

View File

@@ -0,0 +1,22 @@
import os
from os import sep
file_name = "foo/bar"
# PTH206
"foo/bar/".split(os.sep)
"foo/bar/".split(sep=os.sep)
"foo/bar/".split(os.sep)[-1]
"foo/bar/".split(os.sep)[-2]
"foo/bar/".split(os.sep)[-2:]
"fizz/buzz".split(sep)
"fizz/buzz".split(sep)[-1]
os.path.splitext("path/to/hello_world.py")[0].split(os.sep)[-1]
file_name.split(os.sep)
(os.path.abspath(file_name)).split(os.sep)
# OK
"foo/bar/".split("/")
"foo/bar/".split(os.sep, 1)
"foo/bar/".split(1, sep=os.sep)

View File

@@ -0,0 +1,11 @@
import os
import glob
from glob import glob as search
extensions_dir = "./extensions"
# PTH207
glob.glob(os.path.join(extensions_dir, "ops", "autograd", "*.cpp"))
list(glob.iglob(os.path.join(extensions_dir, "ops", "autograd", "*.cpp")))
search("*.png")

View File

@@ -0,0 +1,2 @@
import bar
import foo

View File

@@ -0,0 +1,2 @@
import foo
import bar

View File

@@ -1 +1 @@
broken "§=($/=(")
broken "§=($/=()

View File

@@ -1,28 +1,23 @@
# PERF203
for i in range(10):
try: # PERF203
try:
print(f"{i}")
except:
print("error")
# OK
try:
for i in range(10):
print(f"{i}")
except:
print("error")
# OK
i = 0
while i < 10: # PERF203
while i < 10:
try:
print(f"{i}")
except:
print("error")
i += 1
try:
i = 0
while i < 10:
print(f"{i}")
i += 1
except:
print("error")

View File

@@ -45,3 +45,18 @@ def f():
for i in items:
if i not in result:
result.append(i) # OK
def f():
fibonacci = [0, 1]
for i in range(20):
fibonacci.append(sum(fibonacci[-2:])) # OK
print(fibonacci)
def f():
foo = object()
foo.fibonacci = [0, 1]
for i in range(20):
foo.fibonacci.append(sum(foo.fibonacci[-2:])) # OK
print(foo.fibonacci)

View File

@@ -0,0 +1,13 @@
#: E241
a = (1, 2)
#: Okay
b = (1, 20)
#: E242
a = (1, 2) # tab before 2
#: Okay
b = (1, 20) # space before 20
#: E241 E241 E241
# issue 135
more_spaces = [a, b,
ef, +h,
c, -d]

View File

@@ -66,3 +66,6 @@ while 1:
#: E703:2:1
0\
;
#: E701:2:3
a = \
5;

View File

@@ -58,3 +58,6 @@ assert type(res) == type(None)
types = StrEnum
if x == types.X:
pass
#: E721
assert type(res) is int

View File

@@ -41,3 +41,5 @@ regex = '\w' # noqa
regex = '''
\w
''' # noqa
regex = '\\\_'

View File

@@ -23,3 +23,5 @@ a = []
'%s %s' % (*a,)
k = {}
'%(k)s' % {**k}
'%s' % [1, 2, 3]
'%s' % {1, 2, 3}

View File

@@ -0,0 +1,11 @@
"""Test case: `Literal` with `__future__` annotations."""
from __future__ import annotations
from typing import Literal, Final
from typing_extensions import assert_type
CONSTANT: Final = "ns"
assert_type(CONSTANT, Literal["ns"])

View File

@@ -0,0 +1,103 @@
"""Test type parameters and aliases"""
# Type parameters in type alias statements
from some_module import Bar
type Foo[T] = T # OK
type Foo[T] = list[T] # OK
type Foo[T: ForwardA] = T # OK
type Foo[*Ts] = Bar[Ts] # OK
type Foo[**P] = Bar[P] # OK
class ForwardA: ...
# Types used in aliased assignment must exist
type Foo = DoesNotExist # F821: Undefined name `DoesNotExist`
type Foo = list[DoesNotExist] # F821: Undefined name `DoesNotExist`
# Type parameters do not escape alias scopes
type Foo[T] = T
T # F821: Undefined name `T` - not accessible afterward alias scope
# Type parameters in functions
def foo[T](t: T) -> T: return t # OK
async def afoo[T](t: T) -> T: return t # OK
def with_forward_ref[T: ForwardB](t: T) -> T: return t # OK
def can_access_inside[T](t: T) -> T: # OK
print(T) # OK
return t # OK
class ForwardB: ...
# Type parameters do not escape function scopes
from some_library import some_decorator
@some_decorator(T) # F821: Undefined name `T` - not accessible in decorators
def foo[T](t: T) -> None: ...
T # F821: Undefined name `T` - not accessible afterward function scope
# Type parameters in classes
class Foo[T](list[T]): ... # OK
class UsesForward[T: ForwardC](list[T]): ... # OK
class ForwardC: ...
class WithinBody[T](list[T]): # OK
t = T # OK
x: T # OK
def foo(self, x: T) -> T: # OK
return x
def foo(self):
T # OK
# Type parameters do not escape class scopes
from some_library import some_decorator
@some_decorator(T) # F821: Undefined name `T` - not accessible in decorators
class Foo[T](list[T]): ...
T # F821: Undefined name `T` - not accessible after class scope
# Types specified in bounds should exist
type Foo[T: DoesNotExist] = T # F821: Undefined name `DoesNotExist`
def foo[T: DoesNotExist](t: T) -> T: return t # F821: Undefined name `DoesNotExist`
class Foo[T: DoesNotExist](list[T]): ... # F821: Undefined name `DoesNotExist`
type Foo[T: (DoesNotExist1, DoesNotExist2)] = T # F821: Undefined name `DoesNotExist1`, Undefined name `DoesNotExist2`
def foo[T: (DoesNotExist1, DoesNotExist2)](t: T) -> T: return t # F821: Undefined name `DoesNotExist1`, Undefined name `DoesNotExist2`
class Foo[T: (DoesNotExist1, DoesNotExist2)](list[T]): ... # F821: Undefined name `DoesNotExist1`, Undefined name `DoesNotExist2`
# Type parameters in nested classes
class Parent[T]:
t = T # OK
def can_use_class_variable(self, x: t) -> t: # OK
return x
class Child:
def can_access_parent_type_parameter(self, x: T) -> T: # OK
T # OK
return x
def cannot_access_parent_variable(self, x: t) -> t: # F821: Undefined name `T`
t # F821: Undefined name `t`
return x
# Type parameters in nested functions
def can_access_inside_nested[T](t: T) -> T: # OK
def bar(x: T) -> T: # OK
T # OK
return x
bar(t)

View File

@@ -39,3 +39,34 @@ class Class:
def f(self):
print(my_var)
my_var = 1
import sys
def main():
print(sys.argv)
try:
3 / 0
except ZeroDivisionError:
import sys
sys.exit(1)
import sys
def main():
print(sys.argv)
for sys in range(5):
pass
import requests_mock as rm
def requests_mock(requests_mock: rm.Mocker):
print(rm.ANY)

View File

@@ -147,3 +147,10 @@ def f() -> None:
global CONSTANT
CONSTANT = 1
CONSTANT = 2
def f() -> None:
try:
print("hello")
except A as e :
print("oh no!")

View File

@@ -1,6 +1,7 @@
x = 1 # type: ignore
x = 1 # type:ignore
x = 1 # type: ignore[attr-defined] # type: ignore
x = 1 # type: ignoreme # type: ignore
x = 1
x = 1 # type ignore

View File

@@ -0,0 +1,29 @@
# pylint: disable=missing-docstring,consider-using-f-string, pointless-statement
## Old style formatting
"%s %z" % ("hello", "world") # [bad-format-character]
"%s" "%z" % ("hello", "world") # [bad-format-character]
"""%s %z""" % ("hello", "world") # [bad-format-character]
"""%s""" """%z""" % ("hello", "world") # [bad-format-character]
## New style formatting
"{:s} {:y}".format("hello", "world") # [bad-format-character]
"{:*^30s}".format("centered")
## f-strings
H, W = "hello", "world"
f"{H} {W}"
f"{H:s} {W:z}" # [bad-format-character]
f"{1:z}" # [bad-format-character]
## False negatives
print(("%" "z") % 1)

View File

@@ -0,0 +1,16 @@
class Person:
def __init__(self):
self.name = "monty"
def __eq__(self, other):
return isinstance(other, Person) and other.name == self.name
class Language:
def __init__(self):
self.name = "python"
def __eq__(self, other):
return isinstance(other, Language) and other.name == self.name
def __hash__(self):
return hash(self.name)

View File

@@ -40,3 +40,4 @@ __all__ = __all__ + ["Hello"]
__all__ = __all__ + multiprocessing.__all__
__all__ = list[str](["Hello", "world"])

View File

@@ -0,0 +1,41 @@
foo = 1
bar = 2
baz = 3
# Errors.
foo = foo
bar = bar
foo, bar = foo, bar
bar, foo = bar, foo
(foo, bar) = (foo, bar)
(bar, foo) = (bar, foo)
foo, (bar, baz) = foo, (bar, baz)
bar, (foo, baz) = bar, (foo, baz)
(foo, bar), baz = (foo, bar), baz
(foo, (bar, baz)) = (foo, (bar, baz))
foo, bar = foo, 1
bar, foo = bar, 1
(foo, bar) = (foo, 1)
(bar, foo) = (bar, 1)
foo, (bar, baz) = foo, (bar, 1)
bar, (foo, baz) = bar, (foo, 1)
(foo, bar), baz = (foo, bar), 1
(foo, (bar, baz)) = (foo, (bar, 1))
foo: int = foo
bar: int = bar
# Non-errors.
foo = bar
bar = foo
foo, bar = bar, foo
foo, bar = bar, foo
(foo, bar) = (bar, foo)
foo, bar = bar, 1
bar, foo = foo, 1
foo: int = bar
bar: int = 1
class Foo:
foo = foo
bar = bar

View File

@@ -0,0 +1,18 @@
import subprocess
def foo():
pass
# Errors.
subprocess.Popen(preexec_fn=foo)
subprocess.Popen(["ls"], preexec_fn=foo)
subprocess.Popen(preexec_fn=lambda: print("Hello, world!"))
subprocess.Popen(["ls"], preexec_fn=lambda: print("Hello, world!"))
# Non-errors.
subprocess.Popen()
subprocess.Popen(["ls"])
subprocess.Popen(preexec_fn=None) # None is the default.
subprocess.Popen(["ls"], preexec_fn=None) # None is the default.

View File

@@ -0,0 +1,4 @@
print('Hello world')
#!/usr/bin/python
# -*- coding: utf-8 -*-

View File

@@ -15,7 +15,22 @@ bytes("foo", **a)
bytes(b"foo"
b"bar")
bytes("foo")
bytes(1)
f"{f'{str()}'}"
int(1.0)
int("1")
int(b"11")
int(10, base=2)
int("10", base=2)
int("10", 2)
float("1.0")
float(b"1.0")
bool(1)
bool(0)
bool("foo")
bool("")
bool(b"")
bool(1.0)
# These become string or byte literals
str()
@@ -27,3 +42,10 @@ bytes(b"foo")
bytes(b"""
foo""")
f"{str()}"
int()
int(1)
float()
float(1.0)
bool()
bool(True)
bool(False)

View File

@@ -6,8 +6,12 @@
"{1} {0}".format(a, b)
"{0} {1} {0}".format(a, b)
"{x.y}".format(x=z)
"{x} {y} {x}".format(x=a, y=b)
"{.x} {.y}".format(a, b)
"{} {}".format(a.b, c.d)
@@ -72,6 +76,58 @@ aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
111111
)
"{a}" "{b}".format(a=1, b=1)
(
"{a}"
"{b}"
).format(a=1, b=1)
(
"{a}"
""
"{b}"
""
).format(a=1, b=1)
(
(
# comment
"{a}"
# comment
"{b}"
)
# comment
.format(a=1, b=1)
)
(
"{a}"
"b"
).format(a=1)
def d(osname, version, release):
return"{}-{}.{}".format(osname, version, release)
def e():
yield"{}".format(1)
assert"{}".format(1)
async def c():
return "{}".format(await 3)
async def c():
return "{}".format(1 + await 3)
"{}".format(1 * 2)
###
# Non-errors
###
@@ -85,8 +141,6 @@ aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
"{} {}".format(*a)
"{0} {0}".format(arg)
"{x} {x}".format(arg)
"{x.y} {x.z}".format(arg)
@@ -103,8 +157,6 @@ b"{} {}".format(a, b)
r'"\N{snowman} {}".format(a)'
"{a}" "{b}".format(a=1, b=1)
"123456789 {}".format(
11111111111111111111111111111111111111111111111111111111111111111111111111,
)
@@ -124,20 +176,25 @@ aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
111111
)
async def c():
return "{}".format(await 3)
"{}".format(
[
1,
2,
3,
]
)
"{a}".format(
a=[
1,
2,
3,
]
)
async def c():
return "{}".format(1 + await 3)
(
"{a}"
"{1 + 2}"
).format(a=1)
def d(osname, version, release):
return"{}-{}.{}".format(osname, version, release)
def e():
yield"{}".format(1)
assert"{}".format(1)
"{}".format(**c)

View File

@@ -1,5 +1,3 @@
"""A mirror of UP037_1.py, with `from __future__ import annotations`."""
from __future__ import annotations
from typing import (

View File

@@ -1,108 +0,0 @@
"""A mirror of UP037_0.py, without `from __future__ import annotations`."""
from typing import (
Annotated,
Callable,
List,
Literal,
NamedTuple,
Tuple,
TypeVar,
TypedDict,
cast,
)
from mypy_extensions import Arg, DefaultArg, DefaultNamedArg, NamedArg, VarArg
def foo(var: "MyClass") -> "MyClass":
x: "MyClass"
def foo(*, inplace: "bool"):
pass
def foo(*args: "str", **kwargs: "int"):
pass
x: Tuple["MyClass"]
x: Callable[["MyClass"], None]
class Foo(NamedTuple):
x: "MyClass"
class D(TypedDict):
E: TypedDict("E", foo="int", total=False)
class D(TypedDict):
E: TypedDict("E", {"foo": "int"})
x: Annotated["str", "metadata"]
x: Arg("str", "name")
x: DefaultArg("str", "name")
x: NamedArg("str", "name")
x: DefaultNamedArg("str", "name")
x: DefaultNamedArg("str", name="name")
x: VarArg("str")
x: List[List[List["MyClass"]]]
x: NamedTuple("X", [("foo", "int"), ("bar", "str")])
x: NamedTuple("X", fields=[("foo", "int"), ("bar", "str")])
x: NamedTuple(typename="X", fields=[("foo", "int")])
X: MyCallable("X")
# OK
class D(TypedDict):
E: TypedDict("E")
x: Annotated[()]
x: DefaultNamedArg(name="name", quox="str")
x: DefaultNamedArg(name="name")
x: NamedTuple("X", [("foo",), ("bar",)])
x: NamedTuple("X", ["foo", "bar"])
x: NamedTuple()
x: Literal["foo", "bar"]
x = cast(x, "str")
def foo(x, *args, **kwargs):
...
def foo(*, inplace):
...
x: Annotated[1:2] = ...
x = TypeVar("x", "str", "int")
x = cast("str", x)
X = List["MyClass"]

View File

@@ -0,0 +1,16 @@
import typing
from typing import TypeAlias
# UP040
x: typing.TypeAlias = int
x: TypeAlias = int
# UP040 with generics (todo)
T = typing.TypeVar["T"]
x: typing.TypeAlias = list[T]
# OK
x: TypeAlias
x: int = 1

View File

@@ -11,6 +11,8 @@ class A:
without_annotation = []
class_variable: ClassVar[list[int]] = []
final_variable: Final[list[int]] = []
class_variable_without_subscript: ClassVar = []
final_variable_without_subscript: Final = []
from dataclasses import dataclass, field

View File

@@ -2,21 +2,9 @@ x = range(10)
# RUF015
list(x)[0]
list(x)[:1]
list(x)[:1:1]
list(x)[:1:2]
tuple(x)[0]
tuple(x)[:1]
tuple(x)[:1:1]
tuple(x)[:1:2]
list(i for i in x)[0]
list(i for i in x)[:1]
list(i for i in x)[:1:1]
list(i for i in x)[:1:2]
[i for i in x][0]
[i for i in x][:1]
[i for i in x][:1:1]
[i for i in x][:1:2]
# OK (not indexing (solely) the first element)
list(x)
@@ -29,6 +17,9 @@ list(x)[::]
[i for i in x]
[i for i in x][1]
[i for i in x][-1]
[i for i in x][:1]
[i for i in x][:1:1]
[i for i in x][:1:2]
[i for i in x][1:]
[i for i in x][:3:2]
[i for i in x][::2]

View File

@@ -0,0 +1,5 @@
import os # ruff: noqa: F401
def f():
x = 1

View File

@@ -4,9 +4,10 @@ use anyhow::{bail, Result};
use libcst_native::{
Codegen, CodegenState, ImportNames, ParenthesizableWhitespace, SmallStatement, Statement,
};
use rustpython_parser::ast::{Ranged, Stmt};
use ruff_python_ast::{Ranged, Stmt};
use ruff_python_ast::source_code::{Locator, Stylist};
use ruff_python_codegen::Stylist;
use ruff_source_file::Locator;
use crate::cst::helpers::compose_module_path;
use crate::cst::matchers::match_statement;

View File

@@ -1,13 +1,15 @@
//! Interface for generating autofix edits from higher-level actions (e.g., "remove an argument").
use anyhow::{bail, Result};
use ruff_text_size::{TextLen, TextRange, TextSize};
use rustpython_parser::ast::{self, ExceptHandler, Expr, Keyword, Ranged, Stmt};
use rustpython_parser::{lexer, Mode};
use ruff_diagnostics::Edit;
use ruff_python_ast::helpers;
use ruff_python_ast::source_code::{Indexer, Locator, Stylist};
use ruff_python_trivia::{is_python_whitespace, NewlineWithTrailingNewline, PythonWhitespace};
use ruff_python_ast::{self as ast, Arguments, ExceptHandler, Expr, Keyword, Ranged, Stmt};
use ruff_python_codegen::Stylist;
use ruff_python_index::Indexer;
use ruff_python_parser::{lexer, Mode};
use ruff_python_trivia::{has_leading_content, is_python_whitespace, PythonWhitespace};
use ruff_source_file::{Locator, NewlineWithTrailingNewline};
use ruff_text_size::{TextLen, TextRange, TextSize};
use crate::autofix::codemods;
@@ -41,11 +43,9 @@ pub(crate) fn delete_stmt(
if let Some(semicolon) = trailing_semicolon(stmt.end(), locator) {
let next = next_stmt_break(semicolon, locator);
Edit::deletion(stmt.start(), next)
} else if helpers::has_leading_content(stmt.start(), locator) {
} else if has_leading_content(stmt.start(), locator) {
Edit::range_deletion(stmt.range())
} else if let Some(start) =
helpers::preceded_by_continuations(stmt.start(), locator, indexer)
{
} else if let Some(start) = indexer.preceded_by_continuations(stmt.start(), locator) {
Edit::range_deletion(TextRange::new(start, stmt.end()))
} else {
let range = locator.full_lines_range(stmt.range());
@@ -69,105 +69,101 @@ pub(crate) fn remove_unused_imports<'a>(
}
}
#[derive(Debug, Copy, Clone)]
pub(crate) enum Parentheses {
/// Remove parentheses, if the removed argument is the only argument left.
Remove,
/// Preserve parentheses, even if the removed argument is the only argument
Preserve,
}
/// Generic function to remove arguments or keyword arguments in function
/// calls and class definitions. (For classes `args` should be considered
/// `bases`)
///
/// Supports the removal of parentheses when this is the only (kw)arg left.
/// For this behavior, set `remove_parentheses` to `true`.
pub(crate) fn remove_argument(
pub(crate) fn remove_argument<T: Ranged>(
argument: &T,
arguments: &Arguments,
parentheses: Parentheses,
locator: &Locator,
call_at: TextSize,
expr_range: TextRange,
args: &[Expr],
keywords: &[Keyword],
remove_parentheses: bool,
) -> Result<Edit> {
// TODO(sbrugman): Preserve trailing comments.
let contents = locator.after(call_at);
if arguments.keywords.len() + arguments.args.len() > 1 {
let mut fix_start = None;
let mut fix_end = None;
let mut fix_start = None;
let mut fix_end = None;
let n_arguments = keywords.len() + args.len();
if n_arguments == 0 {
bail!("No arguments or keywords to remove");
}
if n_arguments == 1 {
// Case 1: there is only one argument.
let mut count = 0u32;
for (tok, range) in lexer::lex_starts_at(contents, Mode::Module, call_at).flatten() {
if tok.is_lpar() {
if count == 0 {
fix_start = Some(if remove_parentheses {
range.start()
} else {
range.start() + TextSize::from(1)
});
}
count = count.saturating_add(1);
}
if tok.is_rpar() {
count = count.saturating_sub(1);
if count == 0 {
fix_end = Some(if remove_parentheses {
if arguments
.args
.iter()
.map(Expr::start)
.chain(arguments.keywords.iter().map(Keyword::start))
.any(|location| location > argument.start())
{
// Case 1: argument or keyword is _not_ the last node, so delete from the start of the
// argument to the end of the subsequent comma.
let mut seen_comma = false;
for (tok, range) in lexer::lex_starts_at(
locator.slice(arguments.range()),
Mode::Module,
arguments.start(),
)
.flatten()
{
if seen_comma {
if tok.is_non_logical_newline() {
// Also delete any non-logical newlines after the comma.
continue;
}
fix_end = Some(if tok.is_newline() {
range.end()
} else {
range.end() - TextSize::from(1)
range.start()
});
break;
}
if range.start() == argument.start() {
fix_start = Some(range.start());
}
if fix_start.is_some() && tok.is_comma() {
seen_comma = true;
}
}
} else {
// Case 2: argument or keyword is the last node, so delete from the start of the
// previous comma to the end of the argument.
for (tok, range) in lexer::lex_starts_at(
locator.slice(arguments.range()),
Mode::Module,
arguments.start(),
)
.flatten()
{
if range.start() == argument.start() {
fix_end = Some(argument.end());
break;
}
if tok.is_comma() {
fix_start = Some(range.start());
}
}
}
} else if args
.iter()
.map(Expr::start)
.chain(keywords.iter().map(Keyword::start))
.any(|location| location > expr_range.start())
{
// Case 2: argument or keyword is _not_ the last node.
let mut seen_comma = false;
for (tok, range) in lexer::lex_starts_at(contents, Mode::Module, call_at).flatten() {
if seen_comma {
if tok.is_non_logical_newline() {
// Also delete any non-logical newlines after the comma.
continue;
}
fix_end = Some(if tok.is_newline() {
range.end()
} else {
range.start()
});
break;
}
if range.start() == expr_range.start() {
fix_start = Some(range.start());
}
if fix_start.is_some() && tok.is_comma() {
seen_comma = true;
match (fix_start, fix_end) {
(Some(start), Some(end)) => Ok(Edit::deletion(start, end)),
_ => {
bail!("No fix could be constructed")
}
}
} else {
// Case 3: argument or keyword is the last node, so we have to find the last
// comma in the stmt.
for (tok, range) in lexer::lex_starts_at(contents, Mode::Module, call_at).flatten() {
if range.start() == expr_range.start() {
fix_end = Some(expr_range.end());
break;
// Only one argument; remove it (but preserve parentheses, if needed).
Ok(match parentheses {
Parentheses::Remove => Edit::deletion(arguments.start(), arguments.end()),
Parentheses::Preserve => {
Edit::replacement("()".to_string(), arguments.start(), arguments.end())
}
if tok.is_comma() {
fix_start = Some(range.start());
}
}
}
match (fix_start, fix_end) {
(Some(start), Some(end)) => Ok(Edit::deletion(start, end)),
_ => {
bail!("No fix could be constructed")
}
})
}
}
@@ -295,24 +291,24 @@ fn next_stmt_break(semicolon: TextSize, locator: &Locator) -> TextSize {
#[cfg(test)]
mod tests {
use anyhow::Result;
use ruff_text_size::TextSize;
use rustpython_parser::ast::{Ranged, Suite};
use rustpython_parser::Parse;
use ruff_python_ast::source_code::Locator;
use ruff_python_ast::Ranged;
use ruff_python_parser::parse_suite;
use ruff_source_file::Locator;
use ruff_text_size::TextSize;
use crate::autofix::edits::{next_stmt_break, trailing_semicolon};
#[test]
fn find_semicolon() -> Result<()> {
let contents = "x = 1";
let program = Suite::parse(contents, "<filename>")?;
let program = parse_suite(contents, "<filename>")?;
let stmt = program.first().unwrap();
let locator = Locator::new(contents);
assert_eq!(trailing_semicolon(stmt.end(), &locator), None);
let contents = "x = 1; y = 1";
let program = Suite::parse(contents, "<filename>")?;
let program = parse_suite(contents, "<filename>")?;
let stmt = program.first().unwrap();
let locator = Locator::new(contents);
assert_eq!(
@@ -321,7 +317,7 @@ mod tests {
);
let contents = "x = 1 ; y = 1";
let program = Suite::parse(contents, "<filename>")?;
let program = parse_suite(contents, "<filename>")?;
let stmt = program.first().unwrap();
let locator = Locator::new(contents);
assert_eq!(
@@ -334,7 +330,7 @@ x = 1 \
; y = 1
"
.trim();
let program = Suite::parse(contents, "<filename>")?;
let program = parse_suite(contents, "<filename>")?;
let stmt = program.first().unwrap();
let locator = Locator::new(contents);
assert_eq!(

View File

@@ -1,12 +1,11 @@
use itertools::Itertools;
use std::collections::BTreeSet;
use itertools::Itertools;
use nohash_hasher::IntSet;
use ruff_text_size::{TextLen, TextRange, TextSize};
use rustc_hash::FxHashMap;
use rustc_hash::{FxHashMap, FxHashSet};
use ruff_diagnostics::{Diagnostic, Edit, Fix, IsolationLevel};
use ruff_python_ast::source_code::Locator;
use ruff_source_file::Locator;
use crate::autofix::source_map::SourceMap;
use crate::linter::FixTable;
@@ -47,7 +46,7 @@ fn apply_fixes<'a>(
let mut output = String::with_capacity(locator.len());
let mut last_pos: Option<TextSize> = None;
let mut applied: BTreeSet<&Edit> = BTreeSet::default();
let mut isolated: IntSet<u32> = IntSet::default();
let mut isolated: FxHashSet<u32> = FxHashSet::default();
let mut fixed = FxHashMap::default();
let mut source_map = SourceMap::default();
@@ -60,35 +59,30 @@ fn apply_fixes<'a>(
})
.sorted_by(|(rule1, fix1), (rule2, fix2)| cmp_fix(*rule1, *rule2, fix1, fix2))
{
// If we already applied an identical fix as part of another correction, skip
// any re-application.
if fix.edits().iter().all(|edit| applied.contains(edit)) {
*fixed.entry(rule).or_default() += 1;
continue;
}
let mut edits = fix
.edits()
.iter()
.filter(|edit| !applied.contains(edit))
.peekable();
// Best-effort approach: if this fix overlaps with a fix we've already applied,
// skip it.
if last_pos.map_or(false, |last_pos| {
fix.min_start()
.map_or(false, |fix_location| last_pos >= fix_location)
}) {
continue;
}
// If the fix contains at least one new edit, enforce isolation and positional requirements.
if let Some(first) = edits.peek() {
// If this fix requires isolation, and we've already applied another fix in the
// same isolation group, skip it.
if let IsolationLevel::Group(id) = fix.isolation() {
if !isolated.insert(id) {
continue;
}
}
// If this fix requires isolation, and we've already applied another fix in the
// same isolation group, skip it.
if let IsolationLevel::Group(id) = fix.isolation() {
if !isolated.insert(id) {
// If this fix overlaps with a fix we've already applied, skip it.
if last_pos.is_some_and(|last_pos| last_pos >= first.start()) {
continue;
}
}
for edit in fix
.edits()
.iter()
.sorted_unstable_by_key(|edit| edit.start())
{
let mut applied_edits = Vec::with_capacity(fix.edits().len());
for edit in edits {
// Add all contents from `last_pos` to `fix.location`.
let slice = locator.slice(TextRange::new(last_pos.unwrap_or_default(), edit.start()));
output.push_str(slice);
@@ -104,9 +98,10 @@ fn apply_fixes<'a>(
// Track that the edit was applied.
last_pos = Some(edit.end());
applied.insert(edit);
applied_edits.push(edit);
}
applied.extend(applied_edits.drain(..));
*fixed.entry(rule).or_default() += 1;
}
@@ -147,7 +142,7 @@ mod tests {
use ruff_diagnostics::Diagnostic;
use ruff_diagnostics::Edit;
use ruff_diagnostics::Fix;
use ruff_python_ast::source_code::Locator;
use ruff_source_file::Locator;
use crate::autofix::source_map::SourceMarker;
use crate::autofix::{apply_fixes, FixResult};

View File

@@ -0,0 +1,67 @@
use ruff_diagnostics::{Diagnostic, Fix};
use crate::checkers::ast::Checker;
use crate::codes::Rule;
use crate::rules::{flake8_import_conventions, flake8_pyi, pyflakes, pylint};
/// Run lint rules over the [`Binding`]s.
pub(crate) fn bindings(checker: &mut Checker) {
if !checker.any_enabled(&[
Rule::InvalidAllFormat,
Rule::InvalidAllObject,
Rule::UnaliasedCollectionsAbcSetImport,
Rule::UnconventionalImportAlias,
Rule::UnusedVariable,
]) {
return;
}
for binding in checker.semantic.bindings.iter() {
if checker.enabled(Rule::UnusedVariable) {
if binding.kind.is_bound_exception() && !binding.is_used() {
let mut diagnostic = Diagnostic::new(
pyflakes::rules::UnusedVariable {
name: binding.name(checker.locator).to_string(),
},
binding.range,
);
if checker.patch(Rule::UnusedVariable) {
diagnostic.try_set_fix(|| {
pyflakes::fixes::remove_exception_handler_assignment(
binding,
checker.locator,
)
.map(Fix::automatic)
});
}
checker.diagnostics.push(diagnostic);
}
}
if checker.enabled(Rule::InvalidAllFormat) {
if let Some(diagnostic) = pylint::rules::invalid_all_format(binding) {
checker.diagnostics.push(diagnostic);
}
}
if checker.enabled(Rule::InvalidAllObject) {
if let Some(diagnostic) = pylint::rules::invalid_all_object(binding) {
checker.diagnostics.push(diagnostic);
}
}
if checker.enabled(Rule::UnconventionalImportAlias) {
if let Some(diagnostic) = flake8_import_conventions::rules::unconventional_import_alias(
checker,
binding,
&checker.settings.flake8_import_conventions.aliases,
) {
checker.diagnostics.push(diagnostic);
}
}
if checker.enabled(Rule::UnaliasedCollectionsAbcSetImport) {
if let Some(diagnostic) =
flake8_pyi::rules::unaliased_collections_abc_set_import(checker, binding)
{
checker.diagnostics.push(diagnostic);
}
}
}
}

View File

@@ -0,0 +1,16 @@
use ruff_python_ast::Comprehension;
use crate::checkers::ast::Checker;
use crate::codes::Rule;
use crate::rules::flake8_simplify;
/// Run lint rules over a [`Comprehension`] syntax nodes.
pub(crate) fn comprehension(comprehension: &Comprehension, checker: &mut Checker) {
if checker.enabled(Rule::InDictKeys) {
flake8_simplify::rules::key_in_dict_for(
checker,
&comprehension.target,
&comprehension.iter,
);
}
}

View File

@@ -0,0 +1,32 @@
use ruff_python_ast::{self as ast, Stmt};
use crate::checkers::ast::Checker;
use crate::codes::Rule;
use crate::rules::{flake8_bugbear, perflint};
/// Run lint rules over all deferred for-loops in the [`SemanticModel`].
pub(crate) fn deferred_for_loops(checker: &mut Checker) {
while !checker.deferred.for_loops.is_empty() {
let for_loops = std::mem::take(&mut checker.deferred.for_loops);
for snapshot in for_loops {
checker.semantic.restore(snapshot);
if let Stmt::For(ast::StmtFor {
target, iter, body, ..
})
| Stmt::AsyncFor(ast::StmtAsyncFor {
target, iter, body, ..
}) = &checker.semantic.stmt()
{
if checker.enabled(Rule::UnusedLoopControlVariable) {
flake8_bugbear::rules::unused_loop_control_variable(checker, target, body);
}
if checker.enabled(Rule::IncorrectDictIterator) {
perflint::rules::incorrect_dict_iterator(checker, target, iter);
}
} else {
unreachable!("Expected Expr::For | Expr::AsyncFor");
}
}
}
}

View File

@@ -0,0 +1,304 @@
use ruff_diagnostics::Diagnostic;
use ruff_python_ast::cast;
use ruff_python_semantic::analyze::{branch_detection, visibility};
use ruff_python_semantic::{Binding, BindingKind, ScopeKind};
use crate::checkers::ast::Checker;
use crate::codes::Rule;
use crate::rules::{flake8_pyi, flake8_type_checking, flake8_unused_arguments, pyflakes, pylint};
/// Run lint rules over all deferred scopes in the [`SemanticModel`].
pub(crate) fn deferred_scopes(checker: &mut Checker) {
if !checker.any_enabled(&[
Rule::GlobalVariableNotAssigned,
Rule::ImportShadowedByLoopVar,
Rule::RedefinedWhileUnused,
Rule::RuntimeImportInTypeCheckingBlock,
Rule::TypingOnlyFirstPartyImport,
Rule::TypingOnlyStandardLibraryImport,
Rule::TypingOnlyThirdPartyImport,
Rule::UndefinedLocal,
Rule::UnusedAnnotation,
Rule::UnusedClassMethodArgument,
Rule::UnusedFunctionArgument,
Rule::UnusedImport,
Rule::UnusedLambdaArgument,
Rule::UnusedMethodArgument,
Rule::UnusedPrivateProtocol,
Rule::UnusedPrivateTypeAlias,
Rule::UnusedPrivateTypeVar,
Rule::UnusedPrivateTypedDict,
Rule::UnusedStaticMethodArgument,
Rule::UnusedVariable,
]) {
return;
}
// Identify any valid runtime imports. If a module is imported at runtime, and
// used at runtime, then by default, we avoid flagging any other
// imports from that model as typing-only.
let enforce_typing_imports = !checker.is_stub
&& checker.any_enabled(&[
Rule::RuntimeImportInTypeCheckingBlock,
Rule::TypingOnlyFirstPartyImport,
Rule::TypingOnlyStandardLibraryImport,
Rule::TypingOnlyThirdPartyImport,
]);
let runtime_imports: Vec<Vec<&Binding>> = if enforce_typing_imports {
checker
.semantic
.scopes
.iter()
.map(|scope| {
scope
.binding_ids()
.map(|binding_id| checker.semantic.binding(binding_id))
.filter(|binding| {
flake8_type_checking::helpers::is_valid_runtime_import(
binding,
&checker.semantic,
)
})
.collect()
})
.collect::<Vec<_>>()
} else {
vec![]
};
let mut diagnostics: Vec<Diagnostic> = vec![];
for scope_id in checker.deferred.scopes.iter().rev().copied() {
let scope = &checker.semantic.scopes[scope_id];
if checker.enabled(Rule::UndefinedLocal) {
pyflakes::rules::undefined_local(checker, scope_id, scope, &mut diagnostics);
}
if checker.enabled(Rule::GlobalVariableNotAssigned) {
for (name, binding_id) in scope.bindings() {
let binding = checker.semantic.binding(binding_id);
if binding.kind.is_global() {
diagnostics.push(Diagnostic::new(
pylint::rules::GlobalVariableNotAssigned {
name: (*name).to_string(),
},
binding.range,
));
}
}
}
if checker.enabled(Rule::ImportShadowedByLoopVar) {
for (name, binding_id) in scope.bindings() {
for shadow in checker.semantic.shadowed_bindings(scope_id, binding_id) {
// If the shadowing binding isn't a loop variable, abort.
let binding = &checker.semantic.bindings[shadow.binding_id()];
if !binding.kind.is_loop_var() {
continue;
}
// If the shadowed binding isn't an import, abort.
let shadowed = &checker.semantic.bindings[shadow.shadowed_id()];
if !matches!(
shadowed.kind,
BindingKind::Import(..)
| BindingKind::FromImport(..)
| BindingKind::SubmoduleImport(..)
| BindingKind::FutureImport
) {
continue;
}
// If the bindings are in different forks, abort.
if shadowed.source.map_or(true, |left| {
binding.source.map_or(true, |right| {
branch_detection::different_forks(left, right, &checker.semantic.stmts)
})
}) {
continue;
}
#[allow(deprecated)]
let line = checker.locator.compute_line_index(shadowed.range.start());
checker.diagnostics.push(Diagnostic::new(
pyflakes::rules::ImportShadowedByLoopVar {
name: name.to_string(),
line,
},
binding.range,
));
}
}
}
if checker.enabled(Rule::RedefinedWhileUnused) {
for (name, binding_id) in scope.bindings() {
for shadow in checker.semantic.shadowed_bindings(scope_id, binding_id) {
// If the shadowing binding is a loop variable, abort, to avoid overlap
// with F402.
let binding = &checker.semantic.bindings[shadow.binding_id()];
if binding.kind.is_loop_var() {
continue;
}
// If the shadowed binding is used, abort.
let shadowed = &checker.semantic.bindings[shadow.shadowed_id()];
if shadowed.is_used() {
continue;
}
// If the shadowing binding isn't considered a "redefinition" of the
// shadowed binding, abort.
if !binding.redefines(shadowed) {
continue;
}
if shadow.same_scope() {
// If the symbol is a dummy variable, abort, unless the shadowed
// binding is an import.
if !matches!(
shadowed.kind,
BindingKind::Import(..)
| BindingKind::FromImport(..)
| BindingKind::SubmoduleImport(..)
| BindingKind::FutureImport
) && checker.settings.dummy_variable_rgx.is_match(name)
{
continue;
}
// If this is an overloaded function, abort.
if shadowed.kind.is_function_definition()
&& visibility::is_overload(
cast::decorator_list(
checker.semantic.stmts[shadowed.source.unwrap()],
),
&checker.semantic,
)
{
continue;
}
} else {
// Only enforce cross-scope shadowing for imports.
if !matches!(
shadowed.kind,
BindingKind::Import(..)
| BindingKind::FromImport(..)
| BindingKind::SubmoduleImport(..)
| BindingKind::FutureImport
) {
continue;
}
}
// If the bindings are in different forks, abort.
if shadowed.source.map_or(true, |left| {
binding.source.map_or(true, |right| {
branch_detection::different_forks(left, right, &checker.semantic.stmts)
})
}) {
continue;
}
#[allow(deprecated)]
let line = checker.locator.compute_line_index(shadowed.range.start());
let mut diagnostic = Diagnostic::new(
pyflakes::rules::RedefinedWhileUnused {
name: (*name).to_string(),
line,
},
binding.range,
);
if let Some(range) = binding.parent_range(&checker.semantic) {
diagnostic.set_parent(range.start());
}
diagnostics.push(diagnostic);
}
}
}
if checker.enabled(Rule::UnusedPrivateTypeVar) {
flake8_pyi::rules::unused_private_type_var(checker, scope, &mut diagnostics);
}
if checker.enabled(Rule::UnusedPrivateProtocol) {
flake8_pyi::rules::unused_private_protocol(checker, scope, &mut diagnostics);
}
if checker.enabled(Rule::UnusedPrivateTypeAlias) {
flake8_pyi::rules::unused_private_type_alias(checker, scope, &mut diagnostics);
}
if checker.enabled(Rule::UnusedPrivateTypedDict) {
flake8_pyi::rules::unused_private_typed_dict(checker, scope, &mut diagnostics);
}
if matches!(
scope.kind,
ScopeKind::Function(_) | ScopeKind::AsyncFunction(_) | ScopeKind::Lambda(_)
) {
if checker.enabled(Rule::UnusedVariable) {
pyflakes::rules::unused_variable(checker, scope, &mut diagnostics);
}
if checker.enabled(Rule::UnusedAnnotation) {
pyflakes::rules::unused_annotation(checker, scope, &mut diagnostics);
}
if !checker.is_stub {
if checker.any_enabled(&[
Rule::UnusedClassMethodArgument,
Rule::UnusedFunctionArgument,
Rule::UnusedLambdaArgument,
Rule::UnusedMethodArgument,
Rule::UnusedStaticMethodArgument,
]) {
flake8_unused_arguments::rules::unused_arguments(
checker,
scope,
&mut diagnostics,
);
}
}
}
if matches!(
scope.kind,
ScopeKind::Function(_) | ScopeKind::AsyncFunction(_) | ScopeKind::Module
) {
if enforce_typing_imports {
let runtime_imports: Vec<&Binding> = checker
.semantic
.scopes
.ancestor_ids(scope_id)
.flat_map(|scope_id| runtime_imports[scope_id.as_usize()].iter())
.copied()
.collect();
if checker.enabled(Rule::RuntimeImportInTypeCheckingBlock) {
flake8_type_checking::rules::runtime_import_in_type_checking_block(
checker,
scope,
&mut diagnostics,
);
}
if checker.any_enabled(&[
Rule::TypingOnlyFirstPartyImport,
Rule::TypingOnlyStandardLibraryImport,
Rule::TypingOnlyThirdPartyImport,
]) {
flake8_type_checking::rules::typing_only_runtime_import(
checker,
scope,
&runtime_imports,
&mut diagnostics,
);
}
}
if checker.enabled(Rule::UnusedImport) {
pyflakes::rules::unused_import(checker, scope, &mut diagnostics);
}
}
}
checker.diagnostics.extend(diagnostics);
}

View File

@@ -0,0 +1,293 @@
use ruff_python_ast::str::raw_contents_range;
use ruff_python_ast::Ranged;
use ruff_text_size::TextRange;
use ruff_python_semantic::{BindingKind, ContextualizedDefinition, Export};
use crate::checkers::ast::Checker;
use crate::codes::Rule;
use crate::docstrings::Docstring;
use crate::fs::relativize_path;
use crate::rules::{flake8_annotations, flake8_pyi, pydocstyle};
use crate::{docstrings, warn_user};
/// Run lint rules over all [`Definition`] nodes in the [`SemanticModel`].
///
/// This phase is expected to run after the AST has been traversed in its entirety; as such,
/// it is expected that all [`Definition`] nodes have been visited by the time, and that this
/// method will not recurse into any other nodes.
pub(crate) fn definitions(checker: &mut Checker) {
let enforce_annotations = checker.any_enabled(&[
Rule::AnyType,
Rule::MissingReturnTypeClassMethod,
Rule::MissingReturnTypePrivateFunction,
Rule::MissingReturnTypeSpecialMethod,
Rule::MissingReturnTypeStaticMethod,
Rule::MissingReturnTypeUndocumentedPublicFunction,
Rule::MissingTypeArgs,
Rule::MissingTypeCls,
Rule::MissingTypeFunctionArgument,
Rule::MissingTypeKwargs,
Rule::MissingTypeSelf,
]);
let enforce_stubs = checker.is_stub && checker.enabled(Rule::DocstringInStub);
let enforce_stubs_and_runtime = checker.enabled(Rule::IterMethodReturnIterable);
let enforce_docstrings = checker.any_enabled(&[
Rule::BlankLineAfterLastSection,
Rule::BlankLineAfterSummary,
Rule::BlankLineBeforeClass,
Rule::BlankLinesBetweenHeaderAndContent,
Rule::CapitalizeSectionName,
Rule::DashedUnderlineAfterSection,
Rule::DocstringStartsWithThis,
Rule::EmptyDocstring,
Rule::EmptyDocstringSection,
Rule::EndsInPeriod,
Rule::EndsInPunctuation,
Rule::EscapeSequenceInDocstring,
Rule::FirstLineCapitalized,
Rule::FitsOnOneLine,
Rule::IndentWithSpaces,
Rule::MultiLineSummaryFirstLine,
Rule::MultiLineSummarySecondLine,
Rule::NewLineAfterLastParagraph,
Rule::NewLineAfterSectionName,
Rule::NoBlankLineAfterFunction,
Rule::NoBlankLineAfterSection,
Rule::NoBlankLineBeforeFunction,
Rule::NoBlankLineBeforeSection,
Rule::NoSignature,
Rule::NonImperativeMood,
Rule::OneBlankLineAfterClass,
Rule::OneBlankLineBeforeClass,
Rule::OverIndentation,
Rule::OverloadWithDocstring,
Rule::SectionNameEndsInColon,
Rule::SectionNotOverIndented,
Rule::SectionUnderlineAfterName,
Rule::SectionUnderlineMatchesSectionLength,
Rule::SectionUnderlineNotOverIndented,
Rule::SurroundingWhitespace,
Rule::TripleSingleQuotes,
Rule::UnderIndentation,
Rule::UndocumentedMagicMethod,
Rule::UndocumentedParam,
Rule::UndocumentedPublicClass,
Rule::UndocumentedPublicFunction,
Rule::UndocumentedPublicInit,
Rule::UndocumentedPublicMethod,
Rule::UndocumentedPublicModule,
Rule::UndocumentedPublicNestedClass,
Rule::UndocumentedPublicPackage,
]);
if !enforce_annotations && !enforce_docstrings && !enforce_stubs && !enforce_stubs_and_runtime {
return;
}
// Compute visibility of all definitions.
let exports: Option<Vec<&str>> = {
checker
.semantic
.global_scope()
.get_all("__all__")
.map(|binding_id| &checker.semantic.bindings[binding_id])
.filter_map(|binding| match &binding.kind {
BindingKind::Export(Export { names }) => Some(names.iter().copied()),
_ => None,
})
.fold(None, |acc, names| {
Some(acc.into_iter().flatten().chain(names).collect())
})
};
let definitions = std::mem::take(&mut checker.semantic.definitions);
let mut overloaded_name: Option<String> = None;
for ContextualizedDefinition {
definition,
visibility,
} in definitions.resolve(exports.as_deref()).iter()
{
let docstring = docstrings::extraction::extract_docstring(definition);
// flake8-annotations
if enforce_annotations {
// TODO(charlie): This should be even stricter, in that an overload
// implementation should come immediately after the overloaded
// interfaces, without any AST nodes in between. Right now, we
// only error when traversing definition boundaries (functions,
// classes, etc.).
if !overloaded_name.is_some_and(|overloaded_name| {
flake8_annotations::helpers::is_overload_impl(
definition,
&overloaded_name,
&checker.semantic,
)
}) {
checker
.diagnostics
.extend(flake8_annotations::rules::definition(
checker,
definition,
*visibility,
));
}
overloaded_name =
flake8_annotations::helpers::overloaded_name(definition, &checker.semantic);
}
// flake8-pyi
if enforce_stubs {
if checker.enabled(Rule::DocstringInStub) {
flake8_pyi::rules::docstring_in_stubs(checker, docstring);
}
}
if enforce_stubs_and_runtime {
if checker.enabled(Rule::IterMethodReturnIterable) {
flake8_pyi::rules::iter_method_return_iterable(checker, definition);
}
}
// pydocstyle
if enforce_docstrings {
if pydocstyle::helpers::should_ignore_definition(
definition,
&checker.settings.pydocstyle.ignore_decorators,
&checker.semantic,
) {
continue;
}
// Extract a `Docstring` from a `Definition`.
let Some(expr) = docstring else {
pydocstyle::rules::not_missing(checker, definition, *visibility);
continue;
};
let contents = checker.locator.slice(expr.range());
let indentation = checker.locator.slice(TextRange::new(
checker.locator.line_start(expr.start()),
expr.start(),
));
if pydocstyle::helpers::should_ignore_docstring(contents) {
#[allow(deprecated)]
let location = checker.locator.compute_source_location(expr.start());
warn_user!(
"Docstring at {}:{}:{} contains implicit string concatenation; ignoring...",
relativize_path(checker.path),
location.row,
location.column
);
continue;
}
// SAFETY: Safe for docstrings that pass `should_ignore_docstring`.
let body_range = raw_contents_range(contents).unwrap();
let docstring = Docstring {
definition,
expr,
contents,
body_range,
indentation,
};
if !pydocstyle::rules::not_empty(checker, &docstring) {
continue;
}
if checker.enabled(Rule::FitsOnOneLine) {
pydocstyle::rules::one_liner(checker, &docstring);
}
if checker.any_enabled(&[
Rule::NoBlankLineAfterFunction,
Rule::NoBlankLineBeforeFunction,
]) {
pydocstyle::rules::blank_before_after_function(checker, &docstring);
}
if checker.any_enabled(&[
Rule::BlankLineBeforeClass,
Rule::OneBlankLineAfterClass,
Rule::OneBlankLineBeforeClass,
]) {
pydocstyle::rules::blank_before_after_class(checker, &docstring);
}
if checker.enabled(Rule::BlankLineAfterSummary) {
pydocstyle::rules::blank_after_summary(checker, &docstring);
}
if checker.any_enabled(&[
Rule::IndentWithSpaces,
Rule::OverIndentation,
Rule::UnderIndentation,
]) {
pydocstyle::rules::indent(checker, &docstring);
}
if checker.enabled(Rule::NewLineAfterLastParagraph) {
pydocstyle::rules::newline_after_last_paragraph(checker, &docstring);
}
if checker.enabled(Rule::SurroundingWhitespace) {
pydocstyle::rules::no_surrounding_whitespace(checker, &docstring);
}
if checker.any_enabled(&[
Rule::MultiLineSummaryFirstLine,
Rule::MultiLineSummarySecondLine,
]) {
pydocstyle::rules::multi_line_summary_start(checker, &docstring);
}
if checker.enabled(Rule::TripleSingleQuotes) {
pydocstyle::rules::triple_quotes(checker, &docstring);
}
if checker.enabled(Rule::EscapeSequenceInDocstring) {
pydocstyle::rules::backslashes(checker, &docstring);
}
if checker.enabled(Rule::EndsInPeriod) {
pydocstyle::rules::ends_with_period(checker, &docstring);
}
if checker.enabled(Rule::NonImperativeMood) {
pydocstyle::rules::non_imperative_mood(
checker,
&docstring,
&checker.settings.pydocstyle.property_decorators,
);
}
if checker.enabled(Rule::NoSignature) {
pydocstyle::rules::no_signature(checker, &docstring);
}
if checker.enabled(Rule::FirstLineCapitalized) {
pydocstyle::rules::capitalized(checker, &docstring);
}
if checker.enabled(Rule::DocstringStartsWithThis) {
pydocstyle::rules::starts_with_this(checker, &docstring);
}
if checker.enabled(Rule::EndsInPunctuation) {
pydocstyle::rules::ends_with_punctuation(checker, &docstring);
}
if checker.enabled(Rule::OverloadWithDocstring) {
pydocstyle::rules::if_needed(checker, &docstring);
}
if checker.any_enabled(&[
Rule::BlankLineAfterLastSection,
Rule::BlankLinesBetweenHeaderAndContent,
Rule::CapitalizeSectionName,
Rule::DashedUnderlineAfterSection,
Rule::EmptyDocstringSection,
Rule::MultiLineSummaryFirstLine,
Rule::NewLineAfterSectionName,
Rule::NoBlankLineAfterSection,
Rule::NoBlankLineBeforeSection,
Rule::SectionNameEndsInColon,
Rule::SectionNotOverIndented,
Rule::SectionUnderlineAfterName,
Rule::SectionUnderlineMatchesSectionLength,
Rule::SectionUnderlineNotOverIndented,
Rule::UndocumentedParam,
]) {
pydocstyle::rules::sections(
checker,
&docstring,
checker.settings.pydocstyle.convention.as_ref(),
);
}
}
}
}

View File

@@ -0,0 +1,88 @@
use ruff_python_ast::{self as ast, ExceptHandler, Ranged};
use crate::checkers::ast::Checker;
use crate::registry::Rule;
use crate::rules::{
flake8_bandit, flake8_blind_except, flake8_bugbear, flake8_builtins, pycodestyle, pylint,
tryceratops,
};
/// Run lint rules over an [`ExceptHandler`] syntax node.
pub(crate) fn except_handler(except_handler: &ExceptHandler, checker: &mut Checker) {
match except_handler {
ExceptHandler::ExceptHandler(ast::ExceptHandlerExceptHandler {
type_,
name,
body,
range: _,
}) => {
if checker.enabled(Rule::BareExcept) {
if let Some(diagnostic) = pycodestyle::rules::bare_except(
type_.as_deref(),
body,
except_handler,
checker.locator,
) {
checker.diagnostics.push(diagnostic);
}
}
if checker.enabled(Rule::RaiseWithoutFromInsideExcept) {
flake8_bugbear::rules::raise_without_from_inside_except(
checker,
name.as_deref(),
body,
);
}
if checker.enabled(Rule::BlindExcept) {
flake8_blind_except::rules::blind_except(
checker,
type_.as_deref(),
name.as_deref(),
body,
);
}
if checker.enabled(Rule::TryExceptPass) {
flake8_bandit::rules::try_except_pass(
checker,
except_handler,
type_.as_deref(),
body,
checker.settings.flake8_bandit.check_typed_exception,
);
}
if checker.enabled(Rule::TryExceptContinue) {
flake8_bandit::rules::try_except_continue(
checker,
except_handler,
type_.as_deref(),
body,
checker.settings.flake8_bandit.check_typed_exception,
);
}
if checker.enabled(Rule::ExceptWithEmptyTuple) {
flake8_bugbear::rules::except_with_empty_tuple(checker, except_handler);
}
if checker.enabled(Rule::ExceptWithNonExceptionClasses) {
flake8_bugbear::rules::except_with_non_exception_classes(checker, except_handler);
}
if checker.enabled(Rule::ReraiseNoCause) {
tryceratops::rules::reraise_no_cause(checker, body);
}
if checker.enabled(Rule::BinaryOpException) {
pylint::rules::binary_op_exception(checker, except_handler);
}
if let Some(name) = name {
if checker.enabled(Rule::AmbiguousVariableName) {
if let Some(diagnostic) =
pycodestyle::rules::ambiguous_variable_name(name.as_str(), name.range())
{
checker.diagnostics.push(diagnostic);
}
}
if checker.enabled(Rule::BuiltinVariableShadowing) {
flake8_builtins::rules::builtin_variable_shadowing(checker, name, name.range());
}
}
}
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,27 @@
pub(super) use bindings::bindings;
pub(super) use comprehension::comprehension;
pub(super) use deferred_for_loops::deferred_for_loops;
pub(super) use deferred_scopes::deferred_scopes;
pub(super) use definitions::definitions;
pub(super) use except_handler::except_handler;
pub(super) use expression::expression;
pub(super) use module::module;
pub(super) use parameter::parameter;
pub(super) use parameters::parameters;
pub(super) use statement::statement;
pub(super) use suite::suite;
pub(super) use unresolved_references::unresolved_references;
mod bindings;
mod comprehension;
mod deferred_for_loops;
mod deferred_scopes;
mod definitions;
mod except_handler;
mod expression;
mod module;
mod parameter;
mod parameters;
mod statement;
mod suite;
mod unresolved_references;

View File

@@ -0,0 +1,12 @@
use ruff_python_ast::Suite;
use crate::checkers::ast::Checker;
use crate::codes::Rule;
use crate::rules::flake8_bugbear;
/// Run lint rules over a module.
pub(crate) fn module(suite: &Suite, checker: &mut Checker) {
if checker.enabled(Rule::FStringDocstring) {
flake8_bugbear::rules::f_string_docstring(checker, suite);
}
}

View File

@@ -0,0 +1,28 @@
use ruff_python_ast::{Parameter, Ranged};
use crate::checkers::ast::Checker;
use crate::codes::Rule;
use crate::rules::{flake8_builtins, pep8_naming, pycodestyle};
/// Run lint rules over a [`Parameter`] syntax node.
pub(crate) fn parameter(parameter: &Parameter, checker: &mut Checker) {
if checker.enabled(Rule::AmbiguousVariableName) {
if let Some(diagnostic) =
pycodestyle::rules::ambiguous_variable_name(&parameter.name, parameter.range())
{
checker.diagnostics.push(diagnostic);
}
}
if checker.enabled(Rule::InvalidArgumentName) {
if let Some(diagnostic) = pep8_naming::rules::invalid_argument_name(
&parameter.name,
parameter,
&checker.settings.pep8_naming.ignore_names,
) {
checker.diagnostics.push(diagnostic);
}
}
if checker.enabled(Rule::BuiltinArgumentShadowing) {
flake8_builtins::rules::builtin_argument_shadowing(checker, parameter);
}
}

View File

@@ -0,0 +1,26 @@
use ruff_python_ast::Parameters;
use crate::checkers::ast::Checker;
use crate::codes::Rule;
use crate::rules::{flake8_bugbear, flake8_pyi, ruff};
/// Run lint rules over a [`Parameters`] syntax node.
pub(crate) fn parameters(parameters: &Parameters, checker: &mut Checker) {
if checker.enabled(Rule::MutableArgumentDefault) {
flake8_bugbear::rules::mutable_argument_default(checker, parameters);
}
if checker.enabled(Rule::FunctionCallInDefaultArgument) {
flake8_bugbear::rules::function_call_in_argument_default(checker, parameters);
}
if checker.settings.rules.enabled(Rule::ImplicitOptional) {
ruff::rules::implicit_optional(checker, parameters);
}
if checker.is_stub {
if checker.enabled(Rule::TypedArgumentDefaultInStub) {
flake8_pyi::rules::typed_argument_simple_defaults(checker, parameters);
}
if checker.enabled(Rule::ArgumentDefaultInStub) {
flake8_pyi::rules::argument_simple_defaults(checker, parameters);
}
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,12 @@
use ruff_python_ast::Stmt;
use crate::checkers::ast::Checker;
use crate::codes::Rule;
use crate::rules::flake8_pie;
/// Run lint rules over a suite of [`Stmt`] syntax nodes.
pub(crate) fn suite(suite: &[Stmt], checker: &mut Checker) {
if checker.enabled(Rule::UnnecessaryPass) {
flake8_pie::rules::no_unnecessary_pass(checker, suite);
}
}

View File

@@ -0,0 +1,47 @@
use ruff_diagnostics::Diagnostic;
use ruff_python_semantic::Exceptions;
use crate::checkers::ast::Checker;
use crate::codes::Rule;
use crate::rules::pyflakes;
/// Run lint rules over all [`UnresolvedReference`] entities in the [`SemanticModel`].
pub(crate) fn unresolved_references(checker: &mut Checker) {
if !checker.any_enabled(&[Rule::UndefinedLocalWithImportStarUsage, Rule::UndefinedName]) {
return;
}
for reference in checker.semantic.unresolved_references() {
if reference.is_wildcard_import() {
if checker.enabled(Rule::UndefinedLocalWithImportStarUsage) {
checker.diagnostics.push(Diagnostic::new(
pyflakes::rules::UndefinedLocalWithImportStarUsage {
name: reference.name(checker.locator).to_string(),
},
reference.range(),
));
}
} else {
if checker.enabled(Rule::UndefinedName) {
// Avoid flagging if `NameError` is handled.
if reference.exceptions().contains(Exceptions::NAME_ERROR) {
continue;
}
// Allow __path__.
if checker.path.ends_with("__init__.py") {
if reference.name(checker.locator) == "__path__" {
continue;
}
}
checker.diagnostics.push(Diagnostic::new(
pyflakes::rules::UndefinedName {
name: reference.name(checker.locator).to_string(),
},
reference.range(),
));
}
}
}
}

View File

@@ -1,7 +1,6 @@
use ruff_text_size::TextRange;
use rustpython_parser::ast::Expr;
use ruff_python_ast::{Expr, TypeParam};
use ruff_python_semantic::{ScopeId, Snapshot};
use ruff_text_size::TextRange;
/// A collection of AST nodes that are deferred for later analysis.
/// Used to, e.g., store functions, whose bodies shouldn't be analyzed until all
@@ -11,6 +10,7 @@ pub(crate) struct Deferred<'a> {
pub(crate) scopes: Vec<ScopeId>,
pub(crate) string_type_definitions: Vec<(TextRange, &'a str, Snapshot)>,
pub(crate) future_type_definitions: Vec<(&'a Expr, Snapshot)>,
pub(crate) type_param_definitions: Vec<(&'a TypeParam, Snapshot)>,
pub(crate) functions: Vec<Snapshot>,
pub(crate) lambdas: Vec<(&'a Expr, Snapshot)>,
pub(crate) for_loops: Vec<Snapshot>,

Some files were not shown because too many files have changed in this diff Show More