Compare commits

..

200 Commits

Author SHA1 Message Date
Zanie
35cc48a64c Add stubs for type params and type aliases 2023-07-17 19:06:16 -05:00
Zanie
0d4f1d86ad Format 2023-07-17 18:06:24 -05:00
Zanie
834910947e Update parser pin in fuzzer; fix lockfiles 2023-07-17 18:04:48 -05:00
Zanie
e34cfeb475 WIP: Add support for TypeAlias and TypeParam 2023-07-17 17:52:59 -05:00
Zanie
bfaa1f9530 Bump RustPython-Parser to include PEP-695
126652b684
2023-07-17 17:52:06 -05:00
David Szotten
52aa2fc875 upgrade rustpython to remove tuple-constants (#5840)
c.f. https://github.com/astral-sh/RustPython-Parser/pull/28

Tests: No snapshots changed

---------

Co-authored-by: Zanie <contact@zanie.dev>
2023-07-17 22:50:31 +00:00
Charlie Marsh
e574a6a769 Add some "Phase" annotations to other visit methods (#5839)
## Summary

Follow-up from #5820.
2023-07-17 14:46:39 -04:00
Charlie Marsh
b9346a4fd6 Draw boundaries between various Checker visitation phases (#5820)
## Summary

This PR does some non-behavior-changing refactoring of the AST checker.
Specifically, it breaks the `Stmt`, `Expr`, and `ExceptHandler` visitors
into four distinct, consistent phases:

1. **Phase 1: Analysis**: Run any lint rules on the node.
2. **Phase 2: Binding**: Bind any symbols declared by the node.
3. **Phase 3: Recursion**: Visit all child nodes.
4. **Phase 4: Clean-up**: Pop scopes, etc.

There are some fuzzy boundaries in the last three phases, but the most
important divide is between the Phase 1 and all the others -- the goal
here is (as much as possible) to disentangle all of the vanilla
lint-rule calls from any other semantic analysis or model building.

Part of the motivation here is that I'm considering re-ordering some of
these phases, and it was just impossible to reason about that change as
long as we had miscellaneous binding-creation and scope-modification
code intermingled with lint rules. However, this could also enable us to
(e.g.) move the entire analysis phase elsewhere, and even with a more
limited API that has read-only access to `Checker` (but can push to a
diagnostics vector).
2023-07-17 13:02:21 -04:00
Charlie Marsh
8001a2f121 Expand convention documentation (#5819) 2023-07-17 14:12:46 +00:00
konsti
7dd30f0270 Read black options in format_dev script (#5827)
## Summary

Comparing repos with black requires that we use the settings as black,
notably line length and magic trailing comma behaviour. Excludes and
preserving quotes (vs. a preference for either quote style) is not yet
implemented because they weren't needed for the test projects.

In the other two commits i fixed the output when the progress bar is
hidden (this way is recommonded in the indicatif docs), added a
`scratch.pyi` file to gitignore because black formats stub files
differently and also updated the ecosystem readme with the projects json
without forks.

## Test Plan

I added a `line-length` vs `line_length` test. Otherwise only my
personal usage atm, a PR to integrate the script into the CI to check
some projects will follow.
2023-07-17 13:29:43 +00:00
Micha Reiser
21063544f7 Fix formatter generate.py (#5829) 2023-07-17 10:41:27 +00:00
Luc Khai Hai
fb336898a5 Format AsyncFor (#5808) 2023-07-17 10:38:59 +02:00
Tom Kuson
f5f8eb31ed Add documentation to the flake8-gettext (INT) rules (#5813)
## Summary

Completes documentation for the `flake8-gettext` (`INT`) ruleset.
Related to #2646.

## Test Plan

`python scripts/check_docs_formatted.py`
2023-07-17 04:09:33 +00:00
Charlie Marsh
be6c744856 Include function name in undocumented-param message (#5818)
Closes #5814.
2023-07-16 22:51:34 -04:00
Charlie Marsh
94998aedef Reduce unnecessary allocations for keyword detection (#5817) 2023-07-17 02:22:30 +00:00
Tom Kuson
1c0376a72d Add documentation to the S5XX rules (#5805)
## Summary

Add documentation to the `S5XX` rules (the `flake8-bandit`
['cryptography'](https://bandit.readthedocs.io/en/latest/plugins/index.html#plugin-id-groupings)
rule group). Related to #2646.

## Test Plan

`python scripts/check_docs_formatted.py`
2023-07-17 02:12:57 +00:00
Simon Brugman
de2a13fcd7 [pandas-vet] series constant series (#5802)
## Summary

Implementation for https://github.com/astral-sh/ruff/issues/5588

Q1: are there any additional semantic helpers that could be used to
guard this rule? Which existing rules should be similar in that respect?
Can we at least check if `pandas` is imported (any pointers welcome)?
Currently, the rule flags:
```python
data = {"a": "b"}
data.nunique() == 1
```

Q2: Any pointers on naming of the rule and selection of the code? It was
proposed, but not replied to/implemented in the upstream. `pandas` did
accept a PR to update their cookbook to reflect this rule though.

## Test Plan

TODO:
- [X] Checking for ecosystem CI results
- [x] Test on selected [real-world
cases](https://github.com/search?q=%22nunique%28%29+%3D%3D+1%22+language%3APython+&type=code)
  - [x] https://github.com/sdv-dev/SDMetrics
  - [x] https://github.com/google-research/robustness_metrics
  - [x] https://github.com/soft-matter/trackpy
  - [x] https://github.com/microsoft/FLAML/
- [ ] Add guarded test cases
2023-07-17 01:55:34 +00:00
Harutaka Kawamura
cfec636046 Do not fix NamedTuple calls containing both a list of fields and keywords (#5799)
## Summary

Fixes #5794

## Test Plan

Existing tests
2023-07-17 01:31:53 +00:00
Tom Kuson
ae431df146 Change pandas-use-of-dot-read-table rule to emit only when read_table is used on CSV data (#5807)
## Summary

Closes #5628 by only emitting if `sep=","`. Includes documentation
(completes the `pandas-vet` ruleset).

Related to #2646.

## Test Plan

`cargo test`
2023-07-17 01:25:13 +00:00
Charlie Marsh
2cd117ba81 Remove TryIdentifier trait (#5816)
## Summary

Last remaining usage here is for patterns, but we now have ranges on
identifiers so it's unnecessary.
2023-07-16 21:24:16 -04:00
Simon Brugman
a956226d95 perf: only compute start offset for overlong lines (#5811)
Moves the computation of the `start_offset` for overlong lines to just
before the result is returned. There is a slight overhead for overlong
lines (double the work for the first `limit` characters).

In practice this results in a speedup on the CPython codebase. Most
lines are not overlong, or are not enforced because the line ends with a
URL, or does not contain whitespace. Nonetheless, the 0.3% of overlong
lines are a lot compared to other violations.

### Before
![selected
before](https://github.com/astral-sh/ruff/assets/9756388/d32047df-7fd2-4ae8-8333-1a3679ce000f)
_Selected W505 and E501_

![all
before](https://github.com/astral-sh/ruff/assets/9756388/98495118-c474-46ff-873c-fb58a78cfe15)
_All rules_

### After
![selected
after](https://github.com/astral-sh/ruff/assets/9756388/e4bd7f10-ff7e-4d52-8267-27cace8c5471)
_Selected W505 and E501_

![all
after](https://github.com/astral-sh/ruff/assets/9756388/573bdbe2-c64f-4f22-9659-c68726ff52c0)
_All rules_

CPython line statistics:
- Number of Python lines: 867.696
- Number of overlong lines: 2.963 (0.3%)

<details>

Benchmark selected:
```shell
cargo build --release && hyperfine --warmup 10 --min-runs 50 \                                                  
  "./target/release/ruff ./crates/ruff/resources/test/cpython/ --no-cache -e --select W505,E501"
```

Benchmark all:
```shell
cargo build --release && hyperfine --warmup 10 --min-runs 50 \                                                  
  "./target/release/ruff ./crates/ruff/resources/test/cpython/ --no-cache -e --select ALL"
```

Overlong lines in CPython

```shell
cargo run -p ruff_cli -- check crates/ruff/resources/test/cpython/Lib --no-cache --select=E501,W505 --statistics
```

Total Python lines:
```shell
find crates/ruff/resources/test/cpython/ -name '*.py' | xargs wc -l
```

</details>

(Performance tested on Mac M1)
2023-07-16 21:05:44 -04:00
Chris Pryer
1dd52ad139 Update generate.py comment (#5809)
## Summary

The generated comment is different from the generate files current
comment.

## Test Plan

None
2023-07-16 11:51:30 -04:00
Charlie Marsh
d692ed0896 Use a match statement for builtin detection (#5798)
## Summary

We've seen speed-ups in the past by converting from slice iteration to
match statements; this just does the same for built-in checks.
2023-07-16 04:57:57 +00:00
Charlie Marsh
01b05fe247 Remove Identifier usages for isolating exception names (#5797)
## Summary

The motivating change here is to remove `let range =
except_handler.try_identifier().unwrap();` and instead just do
`name.range()`, since exception names now have ranges attached to them
by the parse. This also required some refactors (which are improvements)
to the built-in attribute shadowing rules, since at least one invocation
relied on passing in the exception handler and calling
`.try_identifier()`. Now that we have easy access to identifiers, we can
remove the whole `AnyShadowing` abstraction.
2023-07-16 04:49:48 +00:00
Charlie Marsh
59dfd0e793 Move except-handler flag into visit_except_handler (#5796)
## Summary

This is more similar to how these flags work in other contexts (e.g.,
`visit_annotation`), and also ensures that we unset it prior to visit
the `orelse` and `finalbody` (a subtle bug).
2023-07-16 00:35:02 -04:00
Charlie Marsh
c7ff743d30 Use semantic().global() to power global-statement rule (#5795)
## Summary

The intent of this rule is to always flag the `global` declaration, not
the usage. The current implementation does the wrong thing if a global
is assigned multiple times. Using `semantic().global()` is also more
efficient.
2023-07-16 00:34:42 -04:00
konsti
b01a4d8446 Update ruff crate descriptions (#5710)
## Summary

I updated all ruff crate descriptions in the contributing guide

## Test Plan

n/a
2023-07-16 02:41:47 +00:00
Justin Prieto
f012ed2d77 Add autofix for B004 (#5788)
## Summary

Adds autofix for `hasattr` case of B004. I don't think it's safe (or
simple) to implement it for the `getattr` case because, inter alia,
calling `getattr` may have side effects.

Fixes #3545

## Test Plan

Existing tests were sufficient. Updated snapshots
2023-07-16 01:32:21 +00:00
Charlie Marsh
06b5c6c06f Use SmallVec#extend_from_slice in lieu of SmallVec#extend (#5793)
## Summary

There's a note in the docs that suggests this can be faster, and in the
benchmarks it... seems like it is? Might just be noise but held up over
a few runs.

Before:

<img width="1792" alt="Screen Shot 2023-07-15 at 9 10 06 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/973cd955-d4e6-4ae3-898e-90b7eb52ecf2">

After:

<img width="1792" alt="Screen Shot 2023-07-15 at 9 10 09 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/1491b391-d219-48e9-aa47-110bc7dc7f90">
2023-07-15 21:25:12 -04:00
Charlie Marsh
4782675bf9 Remove lexer-based comment range detection (#5785)
## Summary

I'm doing some unrelated profiling, and I noticed that this method is
actually measurable on the CPython benchmark -- it's > 1% of execution
time. We don't need to lex here, we already know the ranges of all
comments, so we can just do a simple binary search for overlap, which
brings the method down to 0%.

## Test Plan

`cargo test`
2023-07-16 01:03:27 +00:00
Charlie Marsh
f2e995f78d Gate runtime-import-in-type-checking-block (TCH004) behind enabled flag (#5789)
Closes #5787.
2023-07-15 20:57:29 +00:00
guillaumeLepape
6824b67f44 Include alias when formatting import-from structs (#5786)
## Summary

When required-imports is set with the syntax from ... import ... as ...,
autofix I002 is failing

## Test Plan

Reuse the same python files as
`crates/ruff/src/rules/isort/mod.rs:required_import` test.
2023-07-15 15:53:21 -04:00
Charlie Marsh
8ccd697020 Expand scope of quoted-annotation rule (#5766)
## Summary

Previously, the `quoted-annotation` rule only removed quotes when `from
__future__ import annotations` was present. However, there are some
other cases in which this is also safe -- for example:

```python
def foo():
    x: "MyClass"
```

We already model these in the semantic model, so this PR just expands
the scope of the rule to handle those.
2023-07-15 15:37:34 -04:00
Charlie Marsh
2de6f30929 Lift Expr::Subscript value visit out of branches (#5783)
Like #5772, but for subscripts.
2023-07-15 15:12:15 -04:00
Micha Reiser
df2efe81c8 Respect magic trailing comma for set expression (#5782)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

This PR uses the `join_comma_separated` builder for formatting set
expressions
to ensure the formatting preserves magic commas, if the setting is
enabled.
<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan
See the fixed black tests

<!-- How was it tested? -->
2023-07-15 16:40:38 +00:00
Chris Pryer
fa4855e6fe Format DictComp expression (#5771)
## Summary

Format `DictComp` like `ListComp` from #5600. It's not 100%, but I
figured maybe it's worth starting to explore.

## Test Plan

Added ruff fixture based on `ListComp`'s.
2023-07-15 17:35:23 +01:00
Micha Reiser
3cda89ecaf Parenthesize with statements (#5758)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

This PR improves the parentheses handling for with items to get closer
to black's formatting.

### Case 1:

```python
# Black / Input
with (
    [
        "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
        "bbbbbbbbbb",
        "cccccccccccccccccccccccccccccccccccccccccc",
        dddddddddddddddddddddddddddddddd,
    ] as example1,
    aaaaaaaaaaaaaaaaaaaaaaaaaa
    + bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
    + cccccccccccccccccccccccccccc
    + ddddddddddddddddd as example2,
    CtxManager2() as example2,
    CtxManager2() as example2,
    CtxManager2() as example2,
):
    ...

# Before
with (
    [
        "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
        "bbbbbbbbbb",
        "cccccccccccccccccccccccccccccccccccccccccc",
        dddddddddddddddddddddddddddddddd,
    ] as example1,
    (
        aaaaaaaaaaaaaaaaaaaaaaaaaa
        + bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
        + cccccccccccccccccccccccccccc
        + ddddddddddddddddd
    ) as example2,
    CtxManager2() as example2,
    CtxManager2() as example2,
    CtxManager2() as example2,
):
    ...
```

Notice how Ruff wraps the binary expression in an extra set of
parentheses


### Case 2:
Black does not expand the with-items if the with has no parentheses:

```python
# Black / Input
with aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb as c:
    ...

# Before
with (
    aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb as c
):
    ...
```

Or 

```python
# Black / Input
with [
    "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
    "bbbbbbbbbb",
    "cccccccccccccccccccccccccccccccccccccccccc",
    dddddddddddddddddddddddddddddddd,
] as example1, aaaaaaaaaaaaaaaaaaaaaaaaaa * bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb * cccccccccccccccccccccccccccc + ddddddddddddddddd as example2, CtxManager222222222222222() as example2:
    ...

# Before (Same as Case 1)
with (
    [
        "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
        "bbbbbbbbbb",
        "cccccccccccccccccccccccccccccccccccccccccc",
        dddddddddddddddddddddddddddddddd,
    ] as example1,
    (
        aaaaaaaaaaaaaaaaaaaaaaaaaa
        * bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
        * cccccccccccccccccccccccccccc
        + ddddddddddddddddd
    ) as example2,
    CtxManager222222222222222() as example2,
):
    ...

```
## Test Plan

I added new snapshot tests

Improves the django similarity index from 0.973 to 0.977
2023-07-15 16:03:09 +01:00
Luc Khai Hai
e1c119fde3 Format SetComp (#5774)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

Format `SetComp` like `ListComp`.

## Test Plan

Derived from `ListComp`'s fixture.
2023-07-15 15:50:47 +01:00
Harutaka Kawamura
daa4b72d5f [B006] Add bytes to immutable types (#5776)
## Summary

`B006` should allow using `bytes(...)` as an argument defaule value.

## Test Plan

A new test case

---------

Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
2023-07-15 13:04:33 +00:00
Charlie Marsh
f029f8b784 Move function visit out of Expr::Call branches (#5772)
## Summary

Non-behavioral change, but this is the same in each branch. Visiting the
`func` first also means we've visited the `func` by the time we try to
resolve it (via `resolve_call_path`), which should be helpful in a
future refactor.
2023-07-15 03:36:19 +00:00
Charlie Marsh
bf248ede93 Handle name nodes prior to running rules (#5770)
## Summary

This is more consistent with other patterns in the Checker. Shouldn't
change behavior at all.
2023-07-15 02:21:55 +00:00
Charlie Marsh
086f8a3c12 Move lambda visitation into recurse phase (#5769)
## Summary

Similar to #5768: when we analyze a lambda, we need to recurse in the
recurse phase, rather than the pre-visit phase.
2023-07-15 02:11:47 +00:00
Charlie Marsh
3dc73395ea Move Literal flag detection into recurse phase (#5768)
## Summary

The AST pass is broken up into three phases: pre-visit (which includes
analysis), recurse (visit all members), and post-visit (clean-up). We're
not supposed to edit semantic model flags in the pre-visit phase, but it
looks like we were for literal detection. This didn't matter in
practice, but I'm looking into some AST refactors for which this _does_
cause issues.

No behavior changes expected.

## Test Plan

Good test coverage on these.
2023-07-15 02:04:15 +00:00
Charlie Marsh
7c32e98d10 Use unused variable detection to power incorrect-dict-iterator (#5763)
## Summary

`PERF102` looks for unused keys or values in `dict.items()` calls, and
suggests instead using `dict.keys()` or `dict.values()`. Previously,
this check determined usage by looking for underscore-prefixed
variables. However, we can use the semantic model to actually detect
whether a variable is used. This has two nice effects:

1. We avoid odd false-positives whereby underscore-prefixed variables
are actually used.
2. We can catch more cases (fewer false-negatives) by detecting unused
loop variables that _aren't_ underscore-prefixed.

Closes #5692.
2023-07-14 15:42:47 -04:00
Charlie Marsh
81b88dcfb9 Misc. minor refactors to incorrect-dict-iterator (#5762)
## Summary

Mostly a no-op: use a single match for key-value, use identifier range
rather than re-lexing, respect our `dummy-variable-rgx` setting.
2023-07-14 17:29:25 +00:00
Micha Reiser
8187bf9f7e Cover Black's is_aritmetic_like formatting (#5738) 2023-07-14 17:54:58 +02:00
Charlie Marsh
513de13c46 Remove B904's lowercase exemption (#5751)
## Summary

It looks like bugbear, [from the
start](https://github.com/PyCQA/flake8-bugbear/pull/181#issuecomment-904314876),
has had an exemption here to exempt `raise lower_case_var`. I looked at
Hypothesis and Trio, which are mentioned in that issue, and Hypothesis
has exactly one case of this, and Trio has none, so IMO it doesn't seem
worth special-casing.

Closes https://github.com/astral-sh/ruff/issues/5664.
2023-07-14 11:46:21 -04:00
Justin Prieto
816f7644a9 Fix nested calls to sorted with differing arguments (#5761)
## Summary

Nested calls to `sorted` can only be collapsed if the calls are
identical (i.e., they have the exact same keyword arguments).
Update C414 to only flag such cases.

Fixes #5712

## Test Plan

Updated snapshots.
Tested against flake8-comprehensions. It incorrectly flags these cases.
2023-07-14 13:43:47 +00:00
konsti
fb46579d30 Add Regression test for #5605, where formatting x[:,] failed. (#5759)
#5605 has been fixed, i added the failing example from the issue as a
regression test.

Closes #5605
2023-07-14 11:55:05 +02:00
Chris Pryer
a961f75e13 Format assert statement (#5168) 2023-07-14 09:01:33 +02:00
Charlie Marsh
5a4516b812 Misc. stylistic changes from flipping through rules late at night (#5757)
## Summary

This is really bad PR hygiene, but a mix of: using `Locator`-based fixes
in a few places (in lieu of `Generator`-based fixes), using match syntax
to avoid `.len() == 1` checks, using common helpers in more places, etc.

## Test Plan

`cargo test`
2023-07-14 05:23:47 +00:00
Charlie Marsh
875e04e369 Avoid removing raw strings in comparison fixes (#5755)
## Summary

Use `Locator`-based verbatim fix rather than a `Generator`-based fix,
which loses trivia (and raw strings).

Closes https://github.com/astral-sh/ruff/issues/4130.
2023-07-14 04:27:46 +00:00
Charlie Marsh
12489d3305 Minor tweaks to playground color scheme (#5754)
## Summary

I kind of hate the light mode theme, but they now use colors from our
actual palette:

<img width="1792" alt="Screen Shot 2023-07-13 at 10 15 14 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/f1da0153-d6ed-4b65-9419-b824f2cad614">
<img width="1792" alt="Screen Shot 2023-07-13 at 10 15 12 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/d9452e10-796b-4b7f-bf3f-7af6e0b14fc0">
<img width="1792" alt="Screen Shot 2023-07-13 at 10 15 10 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/f75e7c1c-3b5a-4a78-8bb8-d8b4d40a337d">
<img width="1792" alt="Screen Shot 2023-07-13 at 10 15 07 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/52c23108-b9c2-4a1f-adf0-e11098dbdc5d">
2023-07-13 22:37:18 -04:00
Charlie Marsh
73228e914c Use Ruff favicon for playground (#5752) 2023-07-14 01:11:44 +00:00
Charlie Marsh
af2a087806 Ignore Enum-and-str subclasses for slots enforcement (#5749)
## Summary

Matches the behavior of the upstream plugin.

Closes #5748.
2023-07-13 20:12:16 +00:00
Charlie Marsh
51a313cca4 Avoid stack overflow for non-BitOr binary types (#5743)
## Summary

Closes #5742.
2023-07-13 14:23:40 -04:00
skykasko
48309cad08 Fix the example for blank-line-before-class (D211) (#5746)
The example for
[D211](https://beta.ruff.rs/docs/rules/blank-line-before-class/) is
currently identical to the example for
[D203](https://beta.ruff.rs/docs/rules/one-blank-line-before-class/). It
should be the opposite, with the incorrect case having a blank line
before the class docstring and the correct case having no blank line.
2023-07-13 17:47:01 +00:00
Charlie Marsh
2c2e5b2704 Add some additional Option links to the docs (#5745) 2023-07-13 13:46:17 -04:00
Dhruv Manilawala
5d135d4e0e Update table of content in CONTRIBUTING.md (#5744) 2023-07-13 17:42:28 +00:00
eggplants
06a04c10e2 Fix Options section of rule docs (#5741)
## Summary

Fix: #5740

A trailing line-break are needed for the anchor.

## Test Plan

http://127.0.0.1:8000/docs/rules/line-too-long/#options

|before|after|
|--|--|

|![image](https://github.com/astral-sh/ruff/assets/42153744/8cb9dcce-aeda-4255-b21e-ab11817ba9e1)|![image](https://github.com/astral-sh/ruff/assets/42153744/b68d4fd7-da5a-4494-bb95-f7792f1a42db)|
2023-07-13 17:25:54 +00:00
Charlie Marsh
fee0f43925 Add an overview of Ruff's compilation pipeline to the docs (#5719)
## Summary

I originally wrote this in Notion but it seems preferable to publish it
publicly in the documentation. Feedback welcome!
2023-07-13 16:50:41 +00:00
Justin Prieto
25e491ad6f [flake8-pyi] Implement PYI041 (#5722)
## Summary

Implements PYI041 from flake8-pyi. See [original
code](2a86db8271/pyi.py (L1283)).

This check only applies to function parameters in order to avoid issues
with mypy. See https://github.com/PyCQA/flake8-pyi/issues/299.

ref: #848

## Test Plan

Snapshots, manual runs of flake8.
2023-07-13 16:48:17 +00:00
Charlie Marsh
e7b059cc5c Fix nested lists in CONTRIBUTING.md (#5721)
## Summary

We have a lot of two-space-indented stuff, but apparently it needs to be
four-space indented to render as expected in MkDocs.
2023-07-13 16:32:59 +00:00
Micha Reiser
5dd5ee0c5b Properly group assignment targets (#5728) 2023-07-13 16:00:49 +02:00
konsti
f48ab2d621 Update scripts/ecosystem_all_check.sh (#5737)
## Summary

These changes make `scripts/ecosystem_all_check.sh --select ALL` work
again, i forgot to update this script to the new directory structure
from #5299 because it's only run manually


## Test Plan

n/a
2023-07-13 15:25:22 +02:00
Dhruv Manilawala
cf48ad7b21 Consider single element subscript expr for implicit optional (#5717)
## Summary

Consider single element subscript expr for implicit optional.

On `main`, the cases where there is only a single element in the
subscript
list was giving false positives such as for the following:

```python
typing.Union[None]
typing.Literal[None]
```

## Test Plan

`cargo test`

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2023-07-13 13:10:07 +00:00
Dhruv Manilawala
f44acc047a Check for Any in other types for ANN401 (#5601)
## Summary

Check for `Any` in other types for `ANN401`. This reuses the logic from
`implicit-optional` rule to resolve the type to `Any`.

Following types are supported:
* `Union[Any, ...]`
* `Any | ...`
* `Optional[Any]`
* `Annotated[<any of the above variant>, ...]`
* Forward references i.e., `"Any | ..."`

## Test Plan

Added test cases for various combinations.

fixes: #5458
2023-07-13 18:19:27 +05:30
Tom Kuson
8420008e79 Avoid checking EXE001 and EXE002 on WSL (#5735)
## Summary

Do not raise `EXE001` and `EXE002` if WSL is detected. Uses the
[`wsl`](https://crates.io/crates/wsl) crate.

Closes #5445.

## Test Plan

`cargo test`

I don't use Windows, so was unable to test on a WSL environment. It
would be good if someone who runs Windows could check the functionality.
2023-07-13 07:36:07 -04:00
Charlie Marsh
932c9a4789 Extend PEP 604 rewrites to support some quoted annotations (#5725)
## Summary

Python doesn't allow `"Foo" | None` if the annotation will be evaluated
at runtime (see the comments in the PR, or the semantic model
documentation for more on what this means and when it is true), but it
_does_ allow it if the annotation is typing-only.

This, for example, is invalid, as Python will evaluate `"Foo" | None` at
runtime in order to
populate the function's `__annotations__`:

```python
def f(x: "Foo" | None): ...
```

This, however, is valid:

```python
def f():
    x: "Foo" | None
```

As is this:

```python
from __future__ import annotations

def f(x: "Foo" | None): ...
```

Closes #5706.
2023-07-13 07:34:04 -04:00
konsti
549173b395 Fix StmtAnnAssign formatting by mirroring StmtAssign (#5732)
## Summary

`StmtAnnAssign` would not insert parentheses when breaking the same way
`StmtAssign` does, causing unstable formatting and likely some syntax
errors.

## Test Plan

I added a regression test.
2023-07-13 10:51:25 +00:00
konsti
b1781abffb Link issue tracker in contributing docs (#5688)
## Summary

This adds links to issue categories that are good for people looking to
implement something and a link to the contributing guide feedback issue
(https://github.com/astral-sh/ruff/issues/5684)

---------

Co-authored-by: Zanie <contact@zanie.dev>
2023-07-13 10:42:09 +00:00
konsti
68e0f97354 Formatter: Better f-string dummy (#5730)
## Summary

The previous dummy was causing instabilities since it turned a string
into a variable.

E.g.
```python
            script_header_dict[
                "slurm_partition_line"
            ] = f"#SBATCH --partition {resources.queue_name}"
```
has an instability as
```python
-            script_header_dict["slurm_partition_line"] = (
-                NOT_YET_IMPLEMENTED_ExprJoinedStr
-            )
+            script_header_dict[
+                "slurm_partition_line"
+            ] = NOT_YET_IMPLEMENTED_ExprJoinedStr
```

## Test Plan

The instability is gone, otherwise it's still a dummy
2023-07-13 09:27:25 +00:00
Dhruv Manilawala
e9771c9c63 Ignore Jupyter Notebooks for --add-noqa (#5727) 2023-07-13 13:26:47 +05:30
Micha Reiser
067b2a6ce6 Pass parent to NeedsParentheses (#5708) 2023-07-13 08:57:29 +02:00
Charlie Marsh
30702c2977 Flatten nested tuples when fixing UP007 violations (#5724)
## Summary

Also upgrading these to "Suggested" from "Manual" (they should've always
been "Suggested", I think), and adding some more test cases.
2023-07-13 04:11:32 +00:00
Charlie Marsh
34b79ead3d Use Locator-based replacement rather than Generator for UP007 (#5723)
## Summary

Locator-based replacement is generally preferable as we get verbatim
fixes.
2023-07-13 03:50:16 +00:00
Justin Prieto
19f475ae1f [flake8-pyi] Implement PYI036 (#5668)
## Summary

Implements PYI036 from `flake8-pyi`. See [original
code](https://github.com/PyCQA/flake8-pyi/blob/main/pyi.py#L1585)

## Test Plan

- Updated snapshots
- Checked against manual runs of flake8

ref: #848
2023-07-13 02:50:00 +00:00
Tom Kuson
2b03bd18f4 Implement Pylint consider-using-in (#5193)
## Summary

Implement Pylint rule [`consider-using-in`
(`R1714`)](https://pylint.pycqa.org/en/latest/user_guide/messages/refactor/consider-using-in.html)
as `repeated-equality-comparison-target` (`PLR1714`). This rule checks
for expressions that can be re-written as a membership test for better
readability and performance.

For example,

```python
foo == "bar" or foo == "baz" or foo == "qux"
```

should be rewritten as

```python
foo in {"bar", "baz", "qux"}
```

Related to #970. Includes documentation.

### Implementation quirks

The implementation does not work with Yoda conditions (e.g., `"a" ==
foo` instead of `foo == "a"`). The Pylint version does. I couldn't find
a way of supporting Yoda-style conditions without it being inefficient,
so didn't (I don't think people write Yoda conditions any way).

## Test Plan

Added fixture.

`cargo test`
2023-07-13 01:32:34 +00:00
Charlie Marsh
c87faca884 Use Cursor for shebang parsing (#5716)
## Summary

Better to leverage the shared functionality we get from `Cursor`. It's
also a little bit faster, which is very cool.
2023-07-12 21:22:09 +00:00
Charlie Marsh
6dbc6d2e59 Use shared Cursor across crates (#5715)
## Summary

We have two `Cursor` implementations. This PR moves the implementation
from the formatter into `ruff_python_whitespace` (kind of a poorly-named
crate now) and uses it for both use-cases.
2023-07-12 21:09:27 +00:00
Charlie Marsh
6ce252f0ed Tweak hierarchy of benchmark docs (#5720)
## Summary

Before:

<img width="309" alt="Screen Shot 2023-07-12 at 4 33 23 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/b4a29dc5-183d-479f-8028-f47157b87e0e">

After:

<img width="281" alt="Screen Shot 2023-07-12 at 4 33 32 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/316859d3-db90-4595-8c07-b4bb6543ac4d">
2023-07-12 17:08:22 -04:00
Charlie Marsh
c029c8b37a Run release testing on PR, not push (#5718)
## Summary

This job runs whenever I put up a PR to bump the version, which is
really useful. But then it also runs again when I merge, and then _that_
job tends to get cancelled immediately, because I run the _actual_
release job, which triggers the cancel-concurrent-runs flow. (See, e.g.,
https://github.com/astral-sh/ruff/actions/runs/5534191373.)

I think it makes sense to run these on PR (when editing `pyproject.toml`
and friends), but not again on merge.
2023-07-12 14:22:29 -04:00
Charlie Marsh
0ead9a16ac Bump version to 0.0.278 (#5714) 2023-07-12 12:39:56 -04:00
Micha Reiser
653429bef9 Handle right parens in join comma builder (#5711) 2023-07-12 18:21:28 +02:00
konsti
f0aa6bd4d3 Document ruff_dev and format_dev (#5648)
## Summary

Document all `ruff_dev` subcommands and document the `format_dev` flags
in the formatter readme.

CC @zanieb please flag everything that isn't clear or missing

## Test Plan

n/a
2023-07-12 16:18:22 +02:00
Zanie
5665968b42 Bump static Python versions in CI from 3.7 to 3.11 (#5700)
Python 3.7 is EOL and we should use the latest stable version for
builds.

Related to https://github.com/astral-sh/ruff-lsp/pull/189

---------

Co-authored-by: konsti <konstin@mailbox.org>
2023-07-12 13:56:22 +00:00
Zanie
33a91773f7 Use permalinks in ecosystem diff references (#5704)
Closes https://github.com/astral-sh/ruff/issues/5702
2023-07-12 01:26:37 -05:00
Zanie
0666added9 Add RUF016: Detection of invalid index types (#5602)
Detects invalid types for tuple, list, bytes, string indices.

For example, the following will raise a `TypeError` at runtime and when
imported Python will display a `SyntaxWarning`

```python
var = [1, 2, 3]["x"]
```

```
example.py:1: SyntaxWarning: list indices must be integers or slices, not str; perhaps you missed a comma?
  var = [1, 2, 3]["x"]
Traceback (most recent call last):
  File "example.py", line 1, in <module>
    var = [1, 2, 3]["x"]
          ~~~~~~~~~^^^^^
TypeError: list indices must be integers or slices, not str
```

Previously, Ruff would not report the invalid syntax but now a violation
will be reported. This does not apply to cases where a variable, call,
or complex expression is used in the index — detection is roughly
limited to static definitions, which matches Python's warnings.

```
❯ ./target/debug/ruff example.py --select RUF015 --show-source --no-cache
example.py:1:17: RUF015 Indexed access to type `list` uses type `str` instead of an integer or slice.
  |
1 | var = [1, 2, 3]["x"]
  |                 ^^^ RUF015
  |
```

Closes https://github.com/astral-sh/ruff/issues/5082
xref
ffff1440d1
2023-07-12 00:23:06 -05:00
qdegraaf
7566ca8ff7 Refactor repeated_keys() to use ComparableExpr (#5696)
## Summary

Replaces `DictionaryKey` enum with the more general `ComparableExpr`
when checking for duplicate keys

## Test Plan

Added test fixture from issue. Can potentially be expanded further
depending on what exactly we want to flag (e.g. do we also want to check
for unhashable types?) and which `ComparableExpr::XYZ` types we consider
literals.

## Issue link

Closes: https://github.com/astral-sh/ruff/issues/5691
2023-07-12 03:46:53 +00:00
Charlie Marsh
5dd9e56748 Misc. tweaks to bandit documentation (#5701) 2023-07-11 23:32:15 -04:00
Tom Kuson
f8173daf4c Add documentation to the S3XX rules (#5592)
## Summary

Add documentation to the `S3XX` rules (the `flake8-bandit`
['blacklists'](https://bandit.readthedocs.io/en/latest/plugins/index.html#plugin-id-groupings)
rule group). Related to #2646 .

Changed the `lxml`-based message to reflect that [`defusedxml` doesn't
support `lxml`](https://github.com/tiran/defusedxml/issues/31).

## Test Plan

`python scripts/check_docs_formatted.py && mkdocs serve`
2023-07-11 18:56:51 -05:00
Charlie Marsh
511ec0d7bc Refactor shebang parsing to remove regex dependency (#5690)
## Summary

Similar to #5567, we can remove the use of regex, plus simplify the
representation (use `Option`), add snapshot tests, etc.

This is about 100x faster than using a regex for cases that match (2.5ns
vs. 250ns). It's obviously not a hot path, but I prefer the consistency
with other similar comment-parsing. I may DRY these up into some common
functionality later on.
2023-07-11 16:30:38 -04:00
Micha Reiser
30bec3fcfa Only omit optinal parens if the expression ends or starts with a parenthesized expression
<!--
Thank you for contributing to Ruff! To help us out with reviewing, please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

This PR matches Black' behavior where it only omits the optional parentheses if the expression starts or ends with a parenthesized expression:

```python
a + [aaa, bbb, cccc] * c # Don't omit
[aaa, bbb, cccc] + a * c # Split
a + c * [aaa, bbb, ccc] # Split 
```

<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

This improves the Jaccard index from 0.945 to 0.946
2023-07-11 17:05:25 +02:00
Micha Reiser
8b9193ab1f Improve comprehension line break beheavior
<!--
Thank you for contributing to Ruff! To help us out with reviewing, please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

This PR improves the Black compatibility when it comes to breaking comprehensions. 

We want to avoid line breaks before the target and `in` whenever possible. Furthermore, `if X is not None` should be grouped together, similar to other binary like expressions

<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

`cargo test`

<!-- How was it tested? -->
2023-07-11 16:51:24 +02:00
konsti
62a24e1028 Format ModExpression (#5689)
## Summary

We don't use `ModExpression` anywhere but it's part of the AST, removes
one `not_implemented_yet` and is a trivial 2-liner, so i implemented
formatting for `ModExpression`.

## Test Plan

None, this kind of node does not occur in file input. Otherwise all the
tests for expressions
2023-07-11 16:41:10 +02:00
Micha Reiser
f1d367655b Format target: annotation = value? expressions (#5661) 2023-07-11 16:40:28 +02:00
konsti
0c8ec80d7b Change lambda dummy to NOT_YET_IMPLEMENTED_lambda (#5687)
This only changes the dummy to be easier to identify.
2023-07-11 13:16:18 +00:00
Micha Reiser
df15ad9696 Print files that are slow to format (#5681)
Co-authored-by: konsti <konstin@mailbox.org>
2023-07-11 13:03:18 +00:00
Micha Reiser
8665a1a19d Pass FormatContext to NeedsParentheses
<!--
Thank you for contributing to Ruff! To help us out with reviewing, please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

I started working on this because I assumed that I would need access to options inside of `NeedsParantheses` but it then turned out that I won't. 
Anyway, it kind of felt nice to pass fewer arguments. So I'm gonna put this out here to get your feedback if you prefer this over passing individual fiels. 

Oh, I sneeked in another change. I renamed `context.contents` to `source`. `contents` is too generic and doesn't tell you anything. 

<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

It compiles
2023-07-11 14:28:50 +02:00
Micha Reiser
9a8ba58b4c Remove mode from BestFitting
<!--
Thank you for contributing to Ruff! To help us out with reviewing, please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

This PR removes the `mode` field from `BestFitting` because it is no longer used (we now use `conditional_group` and `fits_expanded).

<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

`cargo test`

<!-- How was it tested? -->
2023-07-11 14:19:26 +02:00
Micha Reiser
715250a179 Prefer expanding parenthesized expressions before operands
<!--
Thank you for contributing to Ruff! To help us out with reviewing, please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

This PR implements Black's behavior where it first splits off parenthesized expressions before splitting before operands to avoid unnecessary parentheses:

```python
# We want 
if a + [ 
	b,
	c
]: 
	pass

# Rather than
if (
    a
    + [b, c]
): 
	pass
```

This is implemented by using the new IR elements introduced in #5596. 

* We give the group wrapping the optional parentheses an ID (`parentheses_id`)
* We use `conditional_group` for the lower priority groups  (all non-parenthesized expressions) with the condition that the `parentheses_id` group breaks (we want to split before operands only if the parentheses are necessary)
* We use `fits_expanded` to wrap all other parenthesized expressions (lists, dicts, sets), to prevent that expanding e.g. a list expands the `parentheses_id` group. We gate the `fits_expand` to only apply if the `parentheses_id` group fits (because we  prefer `a\n+[b, c]` over expanding `[b, c]` if the whole expression gets parenthesized).

We limit using `fits_expanded` and `conditional_group` only to expressions that themselves are not in parentheses (checking the conditions isn't free)

## Test Plan

It increases the Jaccard index for Django from 0.915 to 0.917

## Incompatibilites

There are two incompatibilities left that I'm aware of (there may be more, I didn't go through all snapshot differences). 

### Long string literals
I  commented on the regression. The issue is that a very long string (or any content without a split point) may not fit when only breaking the right side. The formatter than inserts the optional parentheses. But this is kind of useless because the overlong string will still not fit, because there are no new split points. 

I think we should ignore this incompatibility for now


### Expressions on statement level

I don't fully understand the logic behind this yet, but black doesn't break before the operators for the following example even though the expression exceeds the configured line width

```python
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa < bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb > ccccccccccccccccccccccccccccc == ddddddddddddddddddddd
```

But it would if the expression is used inside of a condition. 

What I understand so far is that Black doesn't insert optional parentheses on the expression statement level (and a few other places) and, therefore, only breaks after opening parentheses. I propose to keep this deviation for now to avoid overlong-lines and use the compatibility report to make a decision if we should implement the same behavior.
2023-07-11 14:07:39 +02:00
Micha Reiser
d30e9125eb Extend formatter IR to support Black's expression formatting (#5596) 2023-07-11 11:20:04 +00:00
konsti
212fd86bf0 Switch from jaccard index to similarity index (#5679)
## Summary

The similarity index, the fraction of unchanged lines, is easier to
understand than the jaccard index, the fraction between intersection and
union.

## Test Plan

I ran this on django and git a 0.945 index, meaning 5.5% of lines are
currently reformatted when compared to black
2023-07-11 13:03:44 +02:00
David Szotten
4b58a9c092 formatter: tidy: list_comp is an expression, not a statement (#5677) 2023-07-11 08:00:10 +00:00
konsti
b7794f855b Format StmtAugAssign (#5655)
## Summary

Format statements such as `tree_depth += 1`. This is a statement that
does not allow any line breaks, the only thing to be mindful of is to
parenthesize the assigned expression

Jaccard index on django: 0.915 -> 0.918

## Test Plan

black tests, and two new tests, a basic one and one that ensures that
the child gets parentheses. I ran the django stability check.
2023-07-11 09:06:23 +02:00
Chris Pryer
15c7b6bcf7 Format delete statement (#5169) 2023-07-11 08:36:26 +02:00
David Szotten
1782fb8c30 format ExprListComp (#5600)
Co-authored-by: Micha Reiser <micha@reiser.io>
2023-07-11 06:35:51 +00:00
Micha Reiser
987111f5fb Format ExpressionStarred nodes (#5654) 2023-07-11 06:08:08 +00:00
Charlie Marsh
9f486fa841 [flake8-bugbear] Implement re-sub-positional-args (B034) (#5669)
## Summary

Needed to do some coding to end the day.

Closes #5665.
2023-07-11 03:52:55 +00:00
Charlie Marsh
4dee49d6fa Run nightly Clippy over the Ruff repo (#5670)
## Summary

This is the result of running `cargo +nightly clippy --workspace
--all-targets --all-features -- -D warnings` and fixing all violations.
Just wanted to see if there were any interesting new checks on nightly
👀
2023-07-10 23:44:38 -04:00
Louis Dispa
e7e2f44440 Format raise statement (#5595)
## Summary

This PR implements the formatting of `raise` statements. I haven't
looked at the black implementation, this is inspired from from the
`return` statements formatting.

## Test Plan

The black differences with insta.

I also compared manually some edge cases with very long string and call
chaining and it seems to do the same formatting as black.

There is one issue:
```python
# input

raise OsError(
    "aksjdhflsakhdflkjsadlfajkslhfdkjsaldajlahflashdfljahlfksajlhfajfjfsaahflakjslhdfkjalhdskjfa"
) from a.aaaaa(aksjdhflsakhdflkjsadlfajkslhfdkjsaldajlahflashdfljahlfksajlhfajfjfsaahflakjslhdfkjalhdskjfa).a(aaaa)


# black

raise OsError(
    "aksjdhflsakhdflkjsadlfajkslhfdkjsaldajlahflashdfljahlfksajlhfajfjfsaahflakjslhdfkjalhdskjfa"
) from a.aaaaa(
    aksjdhflsakhdflkjsadlfajkslhfdkjsaldajlahflashdfljahlfksajlhfajfjfsaahflakjslhdfkjalhdskjfa
).a(
    aaaa
)


# ruff

raise OsError(
    "aksjdhflsakhdflkjsadlfajkslhfdkjsaldajlahflashdfljahlfksajlhfajfjfsaahflakjslhdfkjalhdskjfa"
) from a.aaaaa(
    aksjdhflsakhdflkjsadlfajkslhfdkjsaldajlahflashdfljahlfksajlhfajfjfsaahflakjslhdfkjalhdskjfa
).a(aaaa)
```

But I'm not sure this diff is the raise formatting implementation.

---------

Co-authored-by: Louis Dispa <ldispa@deezer.com>
2023-07-10 21:23:49 +02:00
Dhruv Manilawala
93bfa239b7 Add Jupyter Notebook usage with pre-commit in docs (#5666)
Similar to https://github.com/astral-sh/ruff-pre-commit/pull/45
2023-07-11 00:47:05 +05:30
monosans
14f2158e5d [flake8-self] Ignore _name_ and _value_ (#5663)
## Summary

`Enum._name_` and `Enum._value_` are so named to prevent conflicts. See
<https://docs.python.org/3/library/enum.html#supported-sunder-names>.

## Test Plan

Tests for `ignore-names` already exist.
2023-07-10 14:52:59 -04:00
Tom Kuson
b8a6ce43a2 Properly ignore bivariate types in type-name-incorrect-variance (#5660)
## Summary

#5658 didn't actually ignore bivariate types in some all cases (sorry
about that). This PR fixes that and adds bivariate types to the test
fixture.

## Test Plan

`cargo test`
2023-07-10 14:19:17 -04:00
Tom Kuson
5ab9538573 Improve type-name-incorrect-variance message (#5658)
## Summary

Change the `type-name-incorrect-variance` diagnostic message to include
the detected variance and a name change recommendation. For example,

```
`TypeVar` name "T_co" does not reflect its contravariance; consider renaming it to "T_contra"
```

Related to #5651.

## Test Plan

`cargo test`
2023-07-10 13:33:37 -04:00
Zanie
d19839fe0f Add support for Union declarations without | to PYI016 (#5598)
Previously, PYI016 only supported reporting violations for unions
defined with `|`. Now, union declarations with `typing.Union` are
supported.
2023-07-10 17:11:54 +00:00
dependabot[bot]
8dc06d1035 ci(deps): bump webfactory/ssh-agent from 0.7.0 to 0.8.0 (#5657) 2023-07-10 13:10:25 -04:00
Charlie Marsh
120e9d37f1 Audit some SemanticModel#is_builtin usages (#5659)
## Summary

Non-behavior-changing refactors to delay some `.is_builtin` calls in a
few older rules. Cheaper pre-conditions should always be checked first.
2023-07-10 13:10:08 -04:00
Evan Rittenhouse
28fe2d334a Implement UnnecessaryListAllocationForFirstElement (#5549)
## Summary

Fixes #5503. Ready for final review as the `mkdocs` issue involving SSH
keys is fixed.

Note that this will only throw on a `Name` - it will be refactorable
once we have a type-checker. This means that this is the only sort of
input that will throw.
```python
x = range(10)
list(x)[0]
```

I thought it'd be confusing if we supported direct function results.
Consider this example, assuming we support direct results:
```python
# throws
list(range(10))[0]

def createRange(bound):
    return range(bound)

# "why doesn't this throw, but a direct `range(10)` call does?"
list(createRange(10))[0]
```
If it's necessary, I can go through the list of built-ins and find those
which produce iterables, then add them to the throwing list.

## Test Plan

Added a new fixture, then ran `cargo t`
2023-07-10 16:32:41 +00:00
Tom Kuson
3562d809b2 [pylint] Implement Pylint typevar-name-incorrect-variance (C0105) (#5651)
## Summary

Implement Pylint `typevar-name-incorrect-variance` (`C0105`) as
`type-name-incorrect-variance` (`PLC0105`). Includes documentation.
Related to #970.

The Pylint implementation checks only `TypeVar`, but this PR checks
`ParamSpec` as well.

## Test Plan

Added test fixture.

`cargo test`
2023-07-10 12:28:44 -04:00
Tom Kuson
4cac75bc27 Add documentation to pandas-vet rules (#5629)
## Summary

Completes all the documentation for the `pandas-vet` rules, except for
`pandas-use-of-dot-read-table` as I am unclear of the rule's motivation
(see #5628).

Related to #2646.

## Test Plan

`python scripts/check_docs_formatted.py && mkdocs serve`
2023-07-10 15:45:36 +00:00
Charlie Marsh
ed872145fe Always allow PEP 585 and PEP 604 rewrites in stub files (#5653)
Closes https://github.com/astral-sh/ruff/issues/5640.
2023-07-10 14:51:38 +00:00
Charlie Marsh
35b04c2fab Skip flake8-future-annotations checks in stub files (#5652)
Closes https://github.com/astral-sh/ruff/issues/5649.
2023-07-10 10:49:17 -04:00
Evan Rittenhouse
ae4a7ef0ed Make TRY301 trigger only if a raise throws a caught exception (#5455)
## Summary

Fixes #5246. We generate a hash set of all exception IDs caught by the
`try` statement, then check that the inner `raise` actually raises a
caught exception.

## Test Plan

Added a new test, `cargo t`.
2023-07-10 10:00:43 -04:00
konsti
cab3a507bc Fix find_only_token_in_range with expression parentheses (#5645)
## Summary

Fix an oversight in `find_only_token_in_range` where the following code
would panic due do the closing and opening parentheses being in the
range we scan:
```python
d1 = [
    ("a") if # 1
    ("b") else # 2
    ("c")
]
```
Closing and opening parentheses respectively are now correctly skipped.

## Test Plan

I added a regression test
2023-07-10 15:55:19 +02:00
Harutaka Kawamura
82317ba1fd Support autofix for some multiline str.format calls (#5638)
## Summary

Fixes #5531

## Test Plan

New test cases
2023-07-10 09:49:13 -04:00
Aarni Koskela
24bcbb85a1 Rework upstream categories so we can all_rules() (#5591)
## Summary

This PR reworks the `upstream_categories` mechanism that is only used
for documentation purposes to make it easier to generate docs using
`all_rules()`. The new implementation also relies on "tribal knowledge"
about rule codes, so it's not the best implementation, but gets us
forward.

Another option would be to change the rule-defining proc macros to allow
configuring an optional `RuleCategory`, but that seems more heavy-handed
and possibly unnecessary in the long run...

Draft since this builds on #5439.

cc @charliermarsh :)
2023-07-10 09:41:26 -04:00
Micha Reiser
089a671adb Fix Black compatible snapshot deletion (#5646) 2023-07-10 15:00:18 +02:00
konsti
bd8f65814c Format named expressions (walrus operator) (#5642)
## Summary

Format named expressions (walrus operator) such a `value := f()`. 

Unlike tuples, named expression parentheses are not part of the range
even when mandatory, so mapping optional parentheses to always gives us
decent formatting without implementing all [PEP
572](https://peps.python.org/pep-0572/) rules on when we need
parentheses where other expressions wouldn't. We might want to revisit
this decision later and implement special cases, but for now this gives
us what we need.

## Test Plan

black fixtures, i added some fixtures and checked django and cpython for
stability.

Closes #5613
2023-07-10 12:32:15 +00:00
David Szotten
1e894f328c formatter: multi char tokens in SimpleTokenizer (#5610) 2023-07-10 09:00:59 +01:00
Dhruv Manilawala
52b22ceb6e Add links to ecosystem check result (#5631)
## Summary

Add links for ecosystem check result. This is useful for developers to
quickly check the added/removed violations with a single click.

There are a few downsides of this approach:
* Syntax highlighting is not available for the output
* Content length is increased because of the additional anchor tags

## Test Plan

`python scripts/check_ecosystem.py ./target/debug/ruff ../ruff-test/target/debug/ruff`

<details><summary>Example Output:</summary>

ℹ️ ecosystem check **detected changes**. (+6, -0, 0 error(s))

<details><summary>airflow (+1, -0)</summary>
<p>

<pre>
+ <a
href='https://github.com/apache/airflow/blob/main/dev/breeze/src/airflow_breeze/commands/release_management_commands.py#L654'>dev/breeze/src/airflow_breeze/commands/release_management_commands.py:654:25:</a>
PERF401 Use a list comprehension to create a transformed list
</pre>

</p>
</details>
<details><summary>bokeh (+3, -0)</summary>
<p>

<pre>
+ <a
href='https://github.com/bokeh/bokeh/blob/branch-3.2/src/bokeh/model/model.py#L315'>src/bokeh/model/model.py:315:17:</a>
PERF401 Use a list comprehension to create a transformed list
+ <a
href='https://github.com/bokeh/bokeh/blob/branch-3.2/src/bokeh/resources.py#L470'>src/bokeh/resources.py:470:25:</a>
PERF401 Use a list comprehension to create a transformed list
+ <a
href='https://github.com/bokeh/bokeh/blob/branch-3.2/src/bokeh/sphinxext/bokeh_sampledata_xref.py#L134'>src/bokeh/sphinxext/bokeh_sampledata_xref.py:134:17:</a>
PERF401 Use a list comprehension to create a transformed list
</pre>

</p>
</details>
<details><summary>zulip (+2, -0)</summary>
<p>

<pre>
+ <a
href='https://github.com/zulip/zulip/blob/main/zerver/actions/create_user.py#L197'>zerver/actions/create_user.py:197:17:</a>
PERF401 Use a list comprehension to create a transformed list
+ <a
href='https://github.com/zulip/zulip/blob/main/zerver/lib/markdown/__init__.py#L2412'>zerver/lib/markdown/__init__.py:2412:13:</a>
PERF401 Use a list comprehension to create a transformed list
</pre>

</p>
</details>

</details>

---------

Co-authored-by: konsti <konstin@mailbox.org>
2023-07-10 09:25:26 +05:30
Charlie Marsh
c9d7c0d7d5 Add a link to the nursery; tweak icons (#5637)
## Summary

We now always render the icons, but very faintly if inactive, and always
right-align. This ensures consistent alignment as you scroll down the
page:

<img width="1792" alt="Screen Shot 2023-07-09 at 10 45 50 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/da47ac0e-d646-49e1-bbe1-9f43adf94bb4">
2023-07-10 03:09:08 +00:00
Charlie Marsh
eb69fe37bf Render full-width tables in rules reference (#5636) 2023-07-10 02:39:07 +00:00
Charlie Marsh
27011448ea Fix typo in complex-if-statement-in-stub message (#5635) 2023-07-10 02:35:34 +00:00
Aarni Koskela
b4d6b7c230 docs: show nursery icon for nursery rules (#5439)
## Summary

This changes the docs to show a nursery icon (🌅) for rules in the
nursery.

It currently doesn't do that for the rules that are in sub-categories
(Pylint, Pycodestyle) because there is no `all_rules()` for the
`RuleCodePrefix` that's returned by `UpstreamCategory` iteration (and as
mentioned on Discord, I think `UpstreamCategory` maybe shouldn't be a
thing). (That would be enabled by #5591.)

## Test Plan

Generated docs to see new icons (with the caveat above).
2023-07-09 22:24:57 -04:00
Charlie Marsh
fa1341b0db Improve PERF203 example in docs (#5634)
Closes #5624.
2023-07-10 02:24:46 +00:00
Charlie Marsh
401d172e47 Use a simple match statement for case-insensitive noqa lookup (#5633)
## Summary

It turns out that just doing this match directly without `AhoCorasick`
is much faster, like 2x (and removes one dependency, though we likely
already rely on this transitively).
2023-07-09 22:15:23 -04:00
Dhruv Manilawala
6a4b216362 Avoid PERF401 if conditional depends on list var (#5603)
## Summary

Avoid `PERF401` if conditional depends on list var

## Test Plan

`cargo test`

fixes: #5581
2023-07-09 15:53:27 -04:00
Dhruv Manilawala
9dd05424c4 Update ecosystem script to account for 4 letter code (#5627)
E.g., `PERF`
2023-07-09 15:53:02 -04:00
Tom Kuson
ac2e374a5a Add tkinter import convention (#5626)
## Summary

Adds `import tkinter as tk` to the list of default import conventions.

Closes #5620.

## Test Plan

Added `tkinter` to test fixture.

`cargo test`
2023-07-09 16:26:31 +05:30
Charlie Marsh
38fa305f35 Refactor isort directive skips to use iterators (#5623)
## Summary

We're doing some unsafe accesses to advance these iterators. It's easier
to model these as actual iterators to ensure safety everywhere. Also
added some additional test cases.

Closes #5621.
2023-07-08 19:05:44 +00:00
Charlie Marsh
456273a92e Support individual codes on # flake8: noqa directives (#5618)
## Summary

We now treat `# flake8: noqa: F401` as turning off F401 for the entire
file. (Flake8 treats this as turning off _all rules_ for the entire
file).

This deviates from Flake8, but I think it's a much more user-friendly
deviation than what I introduced in #5571. See
https://github.com/astral-sh/ruff/issues/5617 for an explanation.

Closes https://github.com/astral-sh/ruff/issues/5617.
2023-07-08 16:51:37 +00:00
Charlie Marsh
507961f27d Emit warnings for invalid # noqa directives (#5571)
## Summary

This PR adds a `ParseError` type to the `noqa` parsing system to enable
us to render useful warnings instead of silently failing when parsing
`noqa` codes.

For example, given `foo.py`:

```python
# ruff: noqa: x

# ruff: noqa foo

# flake8: noqa: F401
import os  # noqa: foo-bar
```

We would now output:

```console
warning: Invalid `# noqa` directive on line 2: expected a comma-separated list of codes (e.g., `# noqa: F401, F841`).
warning: Invalid `# noqa` directive on line 4: expected `:` followed by a comma-separated list of codes (e.g., `# noqa: F401, F841`).
warning: Invalid `# noqa` directive on line 6: Flake8's blanket exemption does not support exempting specific codes. To exempt specific codes, use, e.g., `# ruff: noqa: F401, F841` instead.
warning: Invalid `# noqa` directive on line 7: expected a comma-separated list of codes (e.g., `# noqa: F401, F841`).
```

There's one important behavior change here too. Right now, with Flake8,
if you do `# flake8: noqa: F401`, Flake8 treats that as equivalent to `#
flake8: noqa` -- it turns off _all_ diagnostics in the file, not just
`F401`. Historically, we respected this... but, I think it's confusing.
So we now raise a warning, and don't respect it at all. This will lead
to errors in some projects, but I'd argue that right now, those
directives are almost certainly behaving in an unintended way for users
anyway.

Closes https://github.com/astral-sh/ruff/issues/3339.
2023-07-08 16:37:55 +00:00
Charlie Marsh
a1c559eaa4 Only run pyproject.toml lint rules when enabled (#5578)
## Summary

I was testing some changes on Airflow, and I realized that we _always_
run the `pyproject.toml` validation rules, even if they're not enabled.
This PR gates them behind the appropriate enablement flags.

## Test Plan

- Ran: `cargo run -p ruff_cli -- check ../airflow -n`. Verified that no
RUF200 violations were raised.
- Run: `cargo run -p ruff_cli -- check ../airflow -n --select RUF200`.
Verified that two RUF200 violations were raised.
2023-07-08 11:05:05 -04:00
konsti
d0dae7e576 Fix CI by downgrading to cargo insta 1.29.0 (#5589)
Since the (implicit) update to cargo-insta 1.30, CI would pass even when
the tests failed. This downgrades to cargo insta 1.29.0 and CI fails
again when it should (which i can't show here, because CI needs to pass
to merge this PR). I've improved the unreferenced snapshot handling in
the process

See https://github.com/mitsuhiko/insta/issues/392
2023-07-08 14:54:49 +00:00
Dimitri Papadopoulos Orfanos
efe7c393d1 Fix typos found by codespell (#5607)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

Fix typos found by
[codespell](https://github.com/codespell-project/codespell).

I have left out `memoize` for now (see #5606).
<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

CI tests.
<!-- How was it tested? -->
2023-07-08 12:33:18 +02:00
konsti
0b9af031fb Format ExprIfExp (ternary operator) (#5597)
## Summary

Format `ExprIfExp`, also known as the ternary operator or inline `if`.
It can look like
```python
a1 = 1 if True else 2
```
but also
```python
b1 = (
    # We return "a" ...
    "a" # that's our True value
    # ... if this condition matches ...
    if True # that's our test
    # ... otherwise we return "b§
    else "b" # that's our False value
)
```

This also fixes a visitor order bug.

The jaccard index on django goes from 0.911 to 0.915.

## Test Plan

I added fixtures without and with comments in strange places.
2023-07-07 19:11:52 +00:00
konsti
0f9d7283e7 Add format-dev contributor docs (#5594)
## Summary

This adds markdown-level docs for #5492

## Test Plan

n/a

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2023-07-07 16:52:13 +00:00
Zanie
bb7303f867 Implement PYI030: Unnecessary literal union (#5570)
Implements PYI030 as part of
https://github.com/astral-sh/ruff/issues/848

> Union expressions should never have more than one Literal member, as
Literal[1] | Literal[2] is semantically identical to Literal[1, 2].

Note we differ slightly from the flake8-pyi implementation:

- We detect cases where there are parentheses or nested unions
- We detect cases with mixed `Union` and `|` syntax
- We use the same error message for all violations; flake8-pyi has two
different messages
- We retain the user's quoting style when displaying string literals;
flake8-pyi uses single quotes
- We warn on duplicates of the same literal `Literal[1] | Literal[1]`
2023-07-07 16:43:10 +00:00
konsti
60d318ddcf Check formatter stability on CI (#5446)
Check formatter stability on CI using CPython. This should be merged
into the ecosystem checks, but i think this is a good start.
2023-07-07 18:28:36 +02:00
Charlie Marsh
5640c310bb Move file-level rule exemption to lexer-based approach (#5567)
## Summary

In addition to `# noqa` codes, we also support file-level exemptions,
which look like:

- `# flake8: noqa` (ignore all rules in the file, for compatibility)
- `# ruff: noqa` (all rules in the file)
- `# ruff: noqa: F401` (ignore `F401` in the file, Flake8 doesn't
support this)

This PR moves that logic to something that looks a lot more like our `#
noqa` parser. Performance is actually quite a bit _worse_ than the
previous approach (lexing `# flake8: noqa` goes from 2ns to 11ns; lexing
`# ruff: noqa: F401, F841` is about the same`; lexing `# type: ignore #
noqa: E501` fgoes from 4ns to 6ns), but the numbers are very small so
it's... maybe worth it?

The primary benefit here is that we now properly support flexible
whitespace, like: `#flake8:noqa`. Previously, we required exact string
matching, and we also didn't support all case-insensitive variants of
`noqa`.
2023-07-07 15:41:20 +00:00
Charlie Marsh
072358e26b Use Instagram's LibCST rather than our fork (#5593)
## Summary

Historically, we only used a fork to enable building without pyo3. But
pyo3 is an optional feature. I may've just not understood how to
accomplish this way back when.
2023-07-07 10:00:44 -04:00
Peter Attia
aaab9f1597 Bugfix: Remove version numbers from pypi links (#5579)
## Summary

There are two pypi links in the documentation that link to specific
version numbers of other packages. Removing these versioned links allows
users to immediately view the latest version of the package and
maintains consistency with the other links.

## Test Plan

N/A
2023-07-07 09:35:50 -04:00
konsti
b22e6c3d38 Extend ruff_dev formatter script to compute statistics and format a project (#5492)
## Summary

This extends the `ruff_dev` formatter script util. Instead of only doing
stability checks, you can now choose different compatible options on the
CLI and get statistics.

* It adds an option the formats all files that ruff would check to allow
looking at an entire black-formatted repository with `git diff`
* It computes the [Jaccard
index](https://en.wikipedia.org/wiki/Jaccard_index) as a measure of
deviation between input and output, which is useful as single number
metric for assessing our current deviations from black.
* It adds progress bars to both the single projects as well as the
multi-project mode.
* It adds an option to write the multi-project output to a file

Sample usage:

```
$ cargo run --bin ruff_dev -- format-dev --stability-check crates/ruff/resources/test/cpython
$ cargo run --bin ruff_dev -- format-dev --stability-check /home/konsti/projects/django
Syntax error in /home/konsti/projects/django/tests/test_runner_apps/tagged/tests_syntax_error.py: source contains syntax errors (parser error): BaseError { error: UnrecognizedToken(Name { name: "syntax_error" }, None), offset: 131, source_path: "<filename>" }
Found 0 stability errors in 2755 files (jaccard index 0.911) in 9.75s
$ cargo run --bin ruff_dev -- format-dev --write /home/konsti/projects/django
```

Options:

```
Several utils related to the formatter which can be run on one or more repositories. The selected set of files in a repository is the same as for `ruff check`.

* Check formatter stability: Format a repository twice and ensure that it looks that the first and second formatting look the same. * Format: Format the files in a repository to be able to check them with `git diff` * Statistics: The subcommand the Jaccard index between the (assumed to be black formatted) input and the ruff formatted output

Usage: ruff_dev format-dev [OPTIONS] [FILES]...

Arguments:
  [FILES]...
          Like `ruff check`'s files. See `--multi-project` if you want to format an ecosystem checkout

Options:
      --stability-check
          Check stability
          
          We want to ensure that once formatted content stays the same when formatted again, which is known as formatter stability or formatter idempotency, and that the formatter prints syntactically valid code. As our test cases cover only a limited amount of code, this allows checking entire repositories.

      --write
          Format the files. Without this flag, the python files are not modified

      --format <FORMAT>
          Control the verbosity of the output
          
          [default: default]

          Possible values:
          - minimal: Filenames only
          - default: Filenames and reduced diff
          - full:    Full diff and invalid code

  -x, --exit-first-error
          Print only the first error and exit, `-x` is same as pytest

      --multi-project
          Checks each project inside a directory, useful e.g. if you want to check all of the ecosystem checkouts

      --error-file <ERROR_FILE>
          Write all errors to this file in addition to stdout. Only used in multi-project mode
```

## Test Plan

I ran this on django (2755 files, jaccard index 0.911) and discovered a
magic trailing comma problem and that we really needed to implement
import formatting. I ran the script on cpython to identify
https://github.com/astral-sh/ruff/pull/5558.
2023-07-07 11:30:12 +00:00
Micha Reiser
40ddc1604c Introduce parenthesized helper (#5565) 2023-07-07 11:28:25 +02:00
Charlie Marsh
bf4b96c5de Differentiate between runtime and typing-time annotations (#5575)
## Summary

In Python, the annotations on `x` and `y` here have very different
treatment:

```python
def foo(x: int):
  y: int
```

The `int` in `x: int` is a runtime-required annotation, because `x` gets
added to the function's `__annotations__`. You'll notice, for example,
that this fails:

```python
from typing import TYPE_CHECKING

if TYPE_CHECKING:
  from foo import Bar

def f(x: Bar):
  ...
```

Because `Bar` is required to be available at runtime, not just at typing
time. Meanwhile, this succeeds:

```python
from typing import TYPE_CHECKING

if TYPE_CHECKING:
  from foo import Bar

def f():
  x: Bar = 1

f()
```

(Both cases are fine if you use `from __future__ import annotations`.)

Historically, we've tracked those annotations that are _not_
runtime-required via the semantic model's `ANNOTATION` flag. But
annotations that _are_ runtime-required have been treated as "type
definitions" that aren't annotations.

This causes problems for the flake8-future-annotations rules, which try
to detect whether adding `from __future__ import annotations` would
_allow_ you to rewrite a type annotation. We need to know whether we're
in _any_ type annotation, runtime-required or not, since adding `from
__future__ import annotations` will convert any runtime-required
annotation to a typing-only annotation.

This PR adds separate state to track these runtime-required annotations.
The changes in the test fixtures are correct -- these were false
negatives before.

Closes https://github.com/astral-sh/ruff/issues/5574.
2023-07-07 00:21:44 -04:00
Charlie Marsh
b11492e940 Fix remaining Copyright rule references (#5577) 2023-07-07 02:49:19 +00:00
Charlie Marsh
cd4718988a Update JSON schema (#5576)
Confused as to how this got merged, but... oh well.
2023-07-06 22:38:39 -04:00
Tom Kuson
5908b39102 Support globbing in isort options (#5473)
## Summary

Support glob patterns in `isort` options.

Closes #5420.

## Test Plan

Added test.

`cargo test`
2023-07-06 20:37:41 -04:00
Charlie Marsh
edfe76d673 Remove checked-in scratch file (#5573) 2023-07-06 21:46:17 +00:00
konsti
5e5a96ca28 Fix formatter StmtTry test (#5568)
For some reason this didn't turn up on CI before

CC @michareiser this is the fix for the error you had
2023-07-06 18:23:53 +00:00
Tom Kuson
3650aaa8b3 Add documentation to the S1XX rules (#5479)
## Summary

Add documentation to the `S1XX` rules (the `flake8-bandit` ['misc
tests'](https://bandit.readthedocs.io/en/latest/plugins/index.html#plugin-id-groupings)
rule group).

## Test Plan

`python scripts/check_docs_formatted.py && mkdocs serve`
2023-07-06 17:46:16 +00:00
Charlie Marsh
cc822082a7 Refactor noqa directive parsing away from regex-based implementation (#5554)
## Summary

I'll write up a more detailed description tomorrow, but in short, this
PR removes our regex-based implementation in favor of "manual" parsing.

I tried a couple different implementations. In the benchmarks below:

- `Directive/Regex` is our implementation on `main`.
- `Directive/Find` just uses `text.find("noqa")`, which is insufficient,
since it doesn't cover case-insensitive variants like `NOQA`, and
doesn't handle multiple `noqa` matches in a single like, like ` # Here's
a noqa comment # noqa: F401`. But it's kind of a baseline.
- `Directive/Memchr` uses three `memchr` iterative finders (one for
`noqa`, `NOQA`, and `NoQA`).
- `Directive/AhoCorasick` is roughly the variant checked-in here.

The raw results:

```
Directive/Regex/# noqa: F401
                        time:   [273.69 ns 274.71 ns 276.03 ns]
                        change: [+1.4467% +1.8979% +2.4243%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 15 outliers among 100 measurements (15.00%)
  3 (3.00%) low mild
  8 (8.00%) high mild
  4 (4.00%) high severe
Directive/Find/# noqa: F401
                        time:   [66.972 ns 67.048 ns 67.132 ns]
                        change: [+2.8292% +2.9377% +3.0540%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 15 outliers among 100 measurements (15.00%)
  1 (1.00%) low severe
  3 (3.00%) low mild
  8 (8.00%) high mild
  3 (3.00%) high severe
Directive/AhoCorasick/# noqa: F401
                        time:   [76.922 ns 77.189 ns 77.536 ns]
                        change: [+0.4265% +0.6862% +0.9871%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 8 outliers among 100 measurements (8.00%)
  1 (1.00%) low mild
  3 (3.00%) high mild
  4 (4.00%) high severe
Directive/Memchr/# noqa: F401
                        time:   [62.627 ns 62.654 ns 62.679 ns]
                        change: [-0.1780% -0.0887% -0.0120%] (p = 0.03 < 0.05)
                        Change within noise threshold.
Found 11 outliers among 100 measurements (11.00%)
  1 (1.00%) low severe
  5 (5.00%) low mild
  3 (3.00%) high mild
  2 (2.00%) high severe
Directive/Regex/# noqa: F401, F841
                        time:   [321.83 ns 322.39 ns 322.93 ns]
                        change: [+8602.4% +8623.5% +8644.5%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
  1 (1.00%) low severe
  2 (2.00%) low mild
  1 (1.00%) high mild
  1 (1.00%) high severe
Directive/Find/# noqa: F401, F841
                        time:   [78.618 ns 78.758 ns 78.896 ns]
                        change: [+1.6909% +1.8771% +2.0628%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild
Directive/AhoCorasick/# noqa: F401, F841
                        time:   [87.739 ns 88.057 ns 88.468 ns]
                        change: [+0.1843% +0.4685% +0.7854%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 11 outliers among 100 measurements (11.00%)
  5 (5.00%) low mild
  3 (3.00%) high mild
  3 (3.00%) high severe
Directive/Memchr/# noqa: F401, F841
                        time:   [80.674 ns 80.774 ns 80.860 ns]
                        change: [-0.7343% -0.5633% -0.4031%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 14 outliers among 100 measurements (14.00%)
  4 (4.00%) low severe
  9 (9.00%) low mild
  1 (1.00%) high mild
Directive/Regex/# noqa  time:   [194.86 ns 195.93 ns 196.97 ns]
                        change: [+11973% +12039% +12103%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 6 outliers among 100 measurements (6.00%)
  5 (5.00%) low mild
  1 (1.00%) high mild
Directive/Find/# noqa   time:   [25.327 ns 25.354 ns 25.383 ns]
                        change: [+3.8524% +4.0267% +4.1845%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 9 outliers among 100 measurements (9.00%)
  6 (6.00%) high mild
  3 (3.00%) high severe
Directive/AhoCorasick/# noqa
                        time:   [34.267 ns 34.368 ns 34.481 ns]
                        change: [+0.5646% +0.8505% +1.1281%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 5 outliers among 100 measurements (5.00%)
  5 (5.00%) high mild
Directive/Memchr/# noqa time:   [21.770 ns 21.818 ns 21.874 ns]
                        change: [-0.0990% +0.1464% +0.4046%] (p = 0.26 > 0.05)
                        No change in performance detected.
Found 10 outliers among 100 measurements (10.00%)
  4 (4.00%) low mild
  4 (4.00%) high mild
  2 (2.00%) high severe
Directive/Regex/# type: ignore # noqa: E501
                        time:   [278.76 ns 279.69 ns 280.72 ns]
                        change: [+7449.4% +7469.8% +7490.5%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) low mild
  1 (1.00%) high mild
  1 (1.00%) high severe
Directive/Find/# type: ignore # noqa: E501
                        time:   [67.791 ns 67.976 ns 68.184 ns]
                        change: [+2.8321% +3.1735% +3.5418%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 6 outliers among 100 measurements (6.00%)
  5 (5.00%) high mild
  1 (1.00%) high severe
Directive/AhoCorasick/# type: ignore # noqa: E501
                        time:   [75.908 ns 76.055 ns 76.210 ns]
                        change: [+0.9269% +1.1427% +1.3955%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe
Directive/Memchr/# type: ignore # noqa: E501
                        time:   [72.549 ns 72.723 ns 72.957 ns]
                        change: [+1.5881% +1.9660% +2.3974%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 15 outliers among 100 measurements (15.00%)
  10 (10.00%) high mild
  5 (5.00%) high severe
Directive/Regex/# type: ignore # nosec
                        time:   [66.967 ns 67.075 ns 67.207 ns]
                        change: [+1713.0% +1715.8% +1718.9%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 10 outliers among 100 measurements (10.00%)
  1 (1.00%) low severe
  3 (3.00%) low mild
  2 (2.00%) high mild
  4 (4.00%) high severe
Directive/Find/# type: ignore # nosec
                        time:   [18.505 ns 18.548 ns 18.597 ns]
                        change: [+1.3520% +1.6976% +2.0333%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
  4 (4.00%) high mild
Directive/AhoCorasick/# type: ignore # nosec
                        time:   [16.162 ns 16.206 ns 16.252 ns]
                        change: [+1.2919% +1.5587% +1.8430%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
  3 (3.00%) high mild
  1 (1.00%) high severe
Directive/Memchr/# type: ignore # nosec
                        time:   [39.192 ns 39.233 ns 39.276 ns]
                        change: [+0.5164% +0.7456% +0.9790%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 13 outliers among 100 measurements (13.00%)
  2 (2.00%) low severe
  4 (4.00%) low mild
  3 (3.00%) high mild
  4 (4.00%) high severe
Directive/Regex/# some very long comment that # is interspersed with characters but # no directive
                        time:   [81.460 ns 81.578 ns 81.703 ns]
                        change: [+2093.3% +2098.8% +2104.2%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
  2 (2.00%) low mild
  2 (2.00%) high mild
Directive/Find/# some very long comment that # is interspersed with characters but # no directive
                        time:   [26.284 ns 26.331 ns 26.387 ns]
                        change: [+0.7554% +1.1027% +1.3832%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 6 outliers among 100 measurements (6.00%)
  5 (5.00%) high mild
  1 (1.00%) high severe
Directive/AhoCorasick/# some very long comment that # is interspersed with characters but # no direc...
                        time:   [28.643 ns 28.714 ns 28.787 ns]
                        change: [+1.3774% +1.6780% +2.0028%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild
Directive/Memchr/# some very long comment that # is interspersed with characters but # no directive
                        time:   [55.766 ns 55.831 ns 55.897 ns]
                        change: [+1.5802% +1.7476% +1.9021%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) low mild
```

While memchr is faster than aho-corasick in some of the common cases
(like `# noqa: F401`), the latter is way, way faster when there _isn't_
a match (like 2x faster -- see the last two cases). Since most comments
_aren't_ `noqa` comments, this felt like the right tradeoff. Note that
all implementations are significantly faster than the regex version.

(I know I originally reported a 10x speedup, but I ended up improving
the regex version a bit in some prior PRs, so it got unintentionally
faster via some refactors.)

There's also one behavior change in here, which is that we now allow
variable spaces, e.g., `#noqa` or `# noqa`. Previously, we required
exactly one space. This thus closes #5177.
2023-07-06 16:03:10 +00:00
Simon Brugman
87ca6171cf docs: add user (#5563)
## Summary

Adding two repositories at ING Bank using ruff. Demonstrates
corporate/industry adoption, e.g. similar to AstraZeneca.

## Test Plan

Note that the tests failing seems unrelated.
2023-07-06 15:55:27 +00:00
Charlie Marsh
9713ee4b80 Remove ParsedFileExemption::None (#5555)
## Summary

This is more aligned with the other enums in this module. Should've been
changed in a previous refactor, just an oversight.
2023-07-06 11:15:46 -04:00
Charlie Marsh
528bf2df3a Use non-Insiders MkDocs for building in forks (#5562) 2023-07-06 15:02:46 +00:00
konsti
8184235f93 Try statements have a body: Fix formatter instability (#5558)
## Summary

The following code was previously leading to unstable formatting:
```python
try:
    try:
        pass
    finally:
        print(1)  # issue7208
except A:
    pass
```
The comment would be formatted as a trailing comment of `try` which is
unstable as an end-of-line comment gets two extra whitespaces.

This was originally found in
99b00efd5e/Lib/getpass.py (L68-L91)

## Test Plan

I added a regression test
2023-07-06 16:07:47 +02:00
Kar Petrosyan
25981420c4 Add httpx into the Who's Using Ruff? section (#5560) 2023-07-06 13:52:28 +00:00
Charlie Marsh
b56b8915ca Allow MkDocs job to run on forks (#5553)
Conditionally check whether the secret is available -- see:
https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsif.
2023-07-06 05:46:49 +00:00
Charlie Marsh
bf02c77fd7 Replace stat mapping with match statement (#5548) 2023-07-05 23:42:21 +00:00
Charlie Marsh
ba7041b6bf Remove Directive's dependency on Locator (#5547)
## Summary

It's a bit simpler to let the API just take the text itself, plus an
offset (to make the returned `TextRange` absolute, rather than
relative).
2023-07-05 23:33:57 +00:00
Charlie Marsh
5dff3195d4 Refactor tokens-based rules to take an &mut Vec<Diagnostic> (#5525) 2023-07-05 19:21:42 -04:00
Charlie Marsh
23363cafd1 Move Directive fields behind accessor methods (#5546) 2023-07-05 23:13:41 +00:00
Charlie Marsh
e4596ebc35 Remove leading and trailing space length from Directive (#5545)
## Summary

We only need this in one place (when removing the directive), and it
simplifies a lot of details to just compute it there.
2023-07-05 23:03:06 +00:00
Charlie Marsh
c9e02c52a8 Add separate configuration for MkDocs Insiders plugins (#5544)
## Summary

This PR adds a separate configuration file to enable us to turn on
[Insiders-only
plugins](https://squidfunk.github.io/mkdocs-material/insiders/getting-started/#built-in-plugins).

I've turned on the `typeset` plugin which ensures that the settings on
the left-hand navigation pane render as code:

<img width="1792" alt="Screen Shot 2023-07-05 at 6 27 20 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/c93676dd-bb48-417a-9d3b-528bf001e9b7">
2023-07-05 18:40:21 -04:00
Charlie Marsh
d097b49371 Remove Directive::None variant (#5543)
## Summary

This is creating some weird, impossible states. Make impossible states
unrepresentable!
2023-07-05 22:22:21 +00:00
Charlie Marsh
ea270da289 Move some MkDocs responsibilities around (#5542)
## Summary

Note that I've also changed from `mkdocs serve` to `mkdocs serve -f
mkdocs.generated.yml` to be clearer that this is a generated file.
2023-07-05 22:06:01 +00:00
Charlie Marsh
cdb9fda3b8 Add debug-based snapshot tests for noqa directive parsing (#5535)
## Summary

Better tests, helpful for future refactors.
2023-07-05 21:49:07 +00:00
Charlie Marsh
a0c0b74b6d Use structs for noqa Directive variants (#5533)
## Summary

No behavioral changes, just clearer (IMO) and with better documentation.
2023-07-05 21:37:32 +00:00
Charlie Marsh
1a2e444799 Use Insiders version of mkdocs-material (#5540)
## Summary

This PR migrates our `mkdocs-material` version to
[Insiders](https://squidfunk.github.io/mkdocs-material/insiders/), which
we can access now that we're sponsors.

We can't allow public access to the Insiders version, so we instead have
a private fork, which contains a deploy key that I've added as a
read-only Actions secret in this repo. (That is: the deploy key only
lets you read that one repo, and do nothing else.)

In general, non-Astral contributors can use the non-insiders version,
and everything is expected to "work", but without the insiders features
(they're intended to be ignored). See:
https://squidfunk.github.io/mkdocs-material/insiders/#compatibility.
2023-07-05 20:36:26 +00:00
qdegraaf
6f548d9872 [isort] Add --case-sensitive flag (#5539)
## Summary

Adds a `--case-sensitive` setting/flag to isort (default: `false`)
which, when set to `true` sorts imports case sensitively instead of case
insensitively.

Tests and Docs can be improved, can do that if the general idea of the
implementation is in order.

First `isort` edit so any and all feedback is welcomed even more than
usual.

## Test Plan

Added a fixture with an assortment of imports in various cases.

## Issue links

Closes: https://github.com/astral-sh/ruff/issues/5514
2023-07-05 16:10:53 -04:00
Charlie Marsh
5a74a8e5a1 Avoid syntax errors when rewriting str(dict) in f-strings (#5538)
Closes https://github.com/astral-sh/ruff/issues/5530.
2023-07-05 19:22:22 +00:00
Charlie Marsh
c5bfd1e877 Allow descriptor instantiations in dataclass fields (#5537)
## Summary

Per the Python documentation, dataclasses are allowed to instantiate
descriptors, like so:

```python
class IntConversionDescriptor:
  def __init__(self, *, default):
    self._default = default

  def __set_name__(self, owner, name):
    self._name = "_" + name

  def __get__(self, obj, type):
    if obj is None:
      return self._default

    return getattr(obj, self._name, self._default)

  def __set__(self, obj, value):
    setattr(obj, self._name, int(value))

@dataclass
class InventoryItem:
  quantity_on_hand: IntConversionDescriptor = IntConversionDescriptor(default=100)
```

Closes https://github.com/astral-sh/ruff/issues/4451.
2023-07-05 15:19:24 -04:00
Charlie Marsh
9e1039f823 Enable attribute lookups via semantic model (#5536)
## Summary

This PR enables us to resolve attribute accesses within files, at least
for static and class methods. For example, we can now detect that this
is a function access (and avoid a false-positive):

```python
class Class:
    @staticmethod
    def error():
        return ValueError("Something")


# OK
raise Class.error()
```

Closes #5487.

Closes #5416.
2023-07-05 15:19:14 -04:00
Tom Kuson
9478454b96 [pylint] Implement Pylint typevar-double-variance (C0131) (#5517)
## Summary

Implement Pylint `typevar-double-variance` (`C0131`) as
`type-bivariance` (`PLC0131`). Includes documentation. Related to #970.
Renamed the rule to be more clear (it's not immediately obvious what
'double' means, IMO).

The Pylint implementation checks only `TypeVar`, but this PR checks
`ParamSpec` as well.

## Test Plan

Added tests.

`cargo test`
2023-07-05 14:53:41 -04:00
Charlie Marsh
9a8e5f7877 Run cargo update (#5534)
```console
❯ cargo update
    Updating crates.io index
    Updating git repository `https://github.com/charliermarsh/LibCST`
    Updating git repository `https://github.com/astral-sh/RustPython-Parser.git`
    Updating git repository `https://github.com/youknowone/unicode_names2.git`
    Updating bitflags v2.3.2 -> v2.3.3
    Updating bstr v1.5.0 -> v1.6.0
    Updating clap v4.3.8 -> v4.3.11
    Updating clap_builder v4.3.8 -> v4.3.11
    Updating clap_complete v4.3.1 -> v4.3.2
    Updating colored v2.0.0 -> v2.0.4
    Removing hermit-abi v0.2.6
    Removing hermit-abi v0.3.1
      Adding hermit-abi v0.3.2
    Updating is-terminal v0.4.7 -> v0.4.8
    Updating itoa v1.0.6 -> v1.0.8
      Adding linux-raw-sys v0.4.3
    Updating num_cpus v1.15.0 -> v1.16.0
    Updating paste v1.0.12 -> v1.0.13
    Updating pin-project-lite v0.2.9 -> v0.2.10
    Updating quote v1.0.28 -> v1.0.29
    Updating regex v1.8.4 -> v1.9.0
    Updating regex-automata v0.1.10 -> v0.3.0
    Updating regex-syntax v0.7.2 -> v0.7.3
    Removing rustix v0.37.20
      Adding rustix v0.37.23
      Adding rustix v0.38.3
    Updating rustversion v1.0.12 -> v1.0.13
    Updating ryu v1.0.13 -> v1.0.14
    Updating serde v1.0.164 -> v1.0.166
    Updating serde_derive v1.0.164 -> v1.0.166
    Updating serde_json v1.0.99 -> v1.0.100
    Updating syn v2.0.22 -> v2.0.23
    Updating thiserror v1.0.40 -> v1.0.41
    Updating thiserror-impl v1.0.40 -> v1.0.41
    Updating unicode-ident v1.0.9 -> v1.0.10
    Updating uuid v1.3.4 -> v1.4.0
    Updating windows-targets v0.48.0 -> v0.48.1
```
2023-07-05 12:34:15 -04:00
Dhruv Manilawala
6fd71e6f53 Avoid triggering DTZ001-006 when using .astimezone() (#5524)
## Summary

Avoid triggering DTZ001-006 when using `.astimezone()`

## Test Plan

Added test cases to call `.astimezone()` on DTZ001-006

fixes: #5516
2023-07-05 00:18:59 -04:00
Charlie Marsh
dd60a3865c Avoid triggering unnecessary-map (C417) for late-bound lambdas (#5520)
Closes https://github.com/astral-sh/ruff/issues/5502.
2023-07-04 22:11:29 -04:00
Charlie Marsh
0726dc25c2 Add some additional users to the README (#5522) 2023-07-05 02:09:50 +00:00
Charlie Marsh
634ed8975c Add pip to the ecosystem-ci check (#5521) 2023-07-05 02:06:21 +00:00
Evan Rittenhouse
5100c56273 Add rule documentation template to scripts/add_rule.py (#5519) 2023-07-04 21:57:26 -04:00
Charlie Marsh
26a268a3ec Refactor the unnecessary-map (C417) implementation (#5518)
## Summary

No behavioral changes. Just refactors + adding a test for a false
positive, which I'll fix in a downstream PR.
2023-07-04 20:25:54 -04:00
Charlie Marsh
324455f580 Bump version to 0.0.277 (#5515) 2023-07-04 17:31:32 -04:00
Charlie Marsh
da1c320bfa Add .ipynb_checkpoints, .pyenv, .pytest_cache, and .vscode to default excludes (#5513)
## Summary

VS Code extensions are
[recommended](https://code.visualstudio.com/docs/python/settings-reference#_linting-settings)
to exclude `.vscode` and `site-packages`. Black also now omits
`.vscode`, `.pytest_cache`, and `.ipynb_checkpoints` by default.
Omitting `.pyenv` is similar to omitting virtual environments, but
really only matters in the context of VS Code (see:
https://github.com/astral-sh/ruff/discussions/5509).

Closes: #5510.
2023-07-04 20:25:16 +00:00
Charlie Marsh
485d997d35 Tweak prefix match to use .all_rules() (#5512)
## Summary

No behavior change, but I think this is a little cleaner.
2023-07-04 20:02:57 +00:00
Aarni Koskela
d7214e77e6 Add ruff rule --all subcommand (with JSON output) (#5059)
## Summary

This adds a `ruff rule --all` switch that prints out a human-readable
Markdown or a machine-readable JSON document of the lint rules known to
Ruff.

I needed a machine-readable document of the rules [for a
project](https://github.com/astral-sh/ruff/discussions/5078), and
figured it could be useful for other people – or tooling! – to be able
to interrogate Ruff about its arcane knowledge.

The JSON output is an array of the same objects printed by `ruff rule
--format=json`.

## Test Plan

I ran `ruff rule --all --format=json`. I think more might be needed, but
maybe a snapshot test is overkill?
2023-07-04 19:45:38 +00:00
Charlie Marsh
952c623102 Avoid returning first-match for rule prefixes (#5511)
Closes #5495, but there's a TODO here to improve this further. The
current `from_code` implementation feels really indirect.
2023-07-04 19:23:05 +00:00
konsti
0a26201643 Merge clippy and clippy (wasm) jobs on CI (#5447)
## Summary

The clippy wasm job rarely fails if regular clippy doesn't, wasm clippy
still compiles a lot of native dependencies for the proc macro and we
have less CI jobs overall, so i think this an improvement to our CI.

```shell
$ CARGO_TARGET_DIR=target-wasm cargo clippy -p ruff_wasm --target wasm32-unknown-unknown --all-features -j 2 -- -D warnings
$ du -sh target-wasm/*
12K	target-wasm/CACHEDIR.TAG
582M	target-wasm/debug
268M	target-wasm/wasm32-unknown-unknown
```

## Test plan

n/a
2023-07-04 15:22:00 -04:00
Tom Kuson
0e67757edb [pylint] Implement Pylint typevar-name-mismatch (C0132) (#5501)
## Summary

Implement Pylint `typevar-name-mismatch` (`C0132`) as
`type-param-name-mismatch` (`PLC0132`). Includes documentation. Related
to #970.

The Pylint implementation checks only `TypeVar`, but this PR checks
`TypeVarTuple`, `ParamSpec`, and `NewType` as well. This seems to better
represent the Pylint rule's [intended
behaviour](https://github.com/pylint-dev/pylint/issues/5224).

Full disclosure: I am not a fan of the translated name and think it
should probably be different.

## Test Plan

`cargo test`
2023-07-04 18:49:43 +00:00
Charlie Marsh
c395e44bd7 Avoid PERF rules for iteration-dependent assignments (#5508)
## Summary

We need to avoid raising "rewrite as a comprehension" violations in
cases like:

```python
d = defaultdict(list)

for i in [1, 2, 3]:
    d[i].append(i**2)
```

Closes https://github.com/astral-sh/ruff/issues/5494.

Closes https://github.com/astral-sh/ruff/issues/5500.
2023-07-04 18:21:05 +00:00
647 changed files with 23044 additions and 12893 deletions

View File

@@ -16,7 +16,7 @@ env:
CARGO_TERM_COLOR: always
RUSTUP_MAX_RETRIES: 10
PACKAGE_NAME: ruff
PYTHON_VERSION: "3.7" # to build abi3 wheels
PYTHON_VERSION: "3.11" # to build abi3 wheels
jobs:
cargo-fmt:
@@ -31,17 +31,6 @@ jobs:
cargo-clippy:
name: "cargo clippy"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: "Install Rust toolchain"
run: |
rustup component add clippy
- uses: Swatinem/rust-cache@v2
- run: cargo clippy --workspace --all-targets --all-features -- -D warnings
cargo-clippy-wasm:
name: "cargo clippy (wasm)"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: "Install Rust toolchain"
@@ -49,7 +38,10 @@ jobs:
rustup component add clippy
rustup target add wasm32-unknown-unknown
- uses: Swatinem/rust-cache@v2
- run: cargo clippy -p ruff_wasm --target wasm32-unknown-unknown --all-features -- -D warnings
- name: "Clippy"
run: cargo clippy --workspace --all-targets --all-features -- -D warnings
- name: "Clippy (wasm)"
run: cargo clippy -p ruff_wasm --target wasm32-unknown-unknown --all-features -- -D warnings
cargo-test:
strategy:
@@ -62,21 +54,19 @@ jobs:
- name: "Install Rust toolchain"
run: rustup show
- uses: Swatinem/rust-cache@v2
- run: cargo install cargo-insta
# cargo insta 1.30.0 fails for some reason (https://github.com/mitsuhiko/insta/issues/392)
- run: cargo install cargo-insta@=1.29.0
- run: pip install black[d]==23.1.0
- name: "Run tests (Ubuntu)"
if: ${{ matrix.os == 'ubuntu-latest' }}
run: |
cargo insta test --all --all-features --delete-unreferenced-snapshots
git diff --exit-code
run: cargo insta test --all --all-features --unreferenced reject
- name: "Run tests (Windows)"
if: ${{ matrix.os == 'windows-latest' }}
shell: bash
run: |
cargo insta test --all --all-features
git diff --exit-code
# We can't reject unreferenced snapshots on windows because flake8_executable can't run on windows
run: cargo insta test --all --all-features
- run: cargo test --package ruff_cli --test black_compatibility_test -- --ignored
# Skipped as it's currently broken. The resource were moved from the
# TODO: Skipped as it's currently broken. The resource were moved from the
# ruff_cli to ruff crate, but this test was not updated.
if: false
# Check for broken links in the documentation.
@@ -152,7 +142,7 @@ jobs:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: "3.11"
python-version: ${{ env.PYTHON_VERSION }}
- uses: actions/download-artifact@v3
name: Download Ruff binary
@@ -236,7 +226,7 @@ jobs:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: "3.11"
python-version: ${{ env.PYTHON_VERSION }}
- name: "Install Rust toolchain"
run: rustup show
- uses: Swatinem/rust-cache@v2
@@ -260,13 +250,24 @@ jobs:
docs:
name: "mkdocs"
runs-on: ubuntu-latest
env:
MKDOCS_INSIDERS_SSH_KEY_EXISTS: ${{ secrets.MKDOCS_INSIDERS_SSH_KEY != '' }}
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
- name: "Add SSH key"
if: ${{ env.MKDOCS_INSIDERS_SSH_KEY_EXISTS == 'true' }}
uses: webfactory/ssh-agent@v0.8.0
with:
ssh-private-key: ${{ secrets.MKDOCS_INSIDERS_SSH_KEY }}
- name: "Install Rust toolchain"
run: rustup show
- uses: Swatinem/rust-cache@v2
- name: "Install Insiders dependencies"
if: ${{ env.MKDOCS_INSIDERS_SSH_KEY_EXISTS == 'true' }}
run: pip install -r docs/requirements-insiders.txt
- name: "Install dependencies"
if: ${{ env.MKDOCS_INSIDERS_SSH_KEY_EXISTS != 'true' }}
run: pip install -r docs/requirements.txt
- name: "Update README File"
run: python scripts/transform_readme.py --target mkdocs
@@ -274,5 +275,23 @@ jobs:
run: python scripts/generate_mkdocs.py
- name: "Check docs formatting"
run: python scripts/check_docs_formatted.py
- name: "Build Insiders docs"
if: ${{ env.MKDOCS_INSIDERS_SSH_KEY_EXISTS == 'true' }}
run: mkdocs build --strict -f mkdocs.insiders.yml
- name: "Build docs"
run: mkdocs build --strict
if: ${{ env.MKDOCS_INSIDERS_SSH_KEY_EXISTS != 'true' }}
run: mkdocs build --strict -f mkdocs.generated.yml
check-formatter-stability:
name: "Check formatter stability"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: "Install Rust toolchain"
run: rustup show
- name: "Cache rust"
uses: Swatinem/rust-cache@v2
- name: "Clone CPython 3.10"
run: git clone --branch 3.10 --depth 1 https://github.com/python/cpython.git crates/ruff/resources/test/cpython
- name: "Check stability"
run: cargo run --bin ruff_dev -- format-dev --stability-check crates/ruff/resources/test/cpython

View File

@@ -10,20 +10,34 @@ jobs:
runs-on: ubuntu-latest
env:
CF_API_TOKEN_EXISTS: ${{ secrets.CF_API_TOKEN != '' }}
MKDOCS_INSIDERS_SSH_KEY_EXISTS: ${{ secrets.MKDOCS_INSIDERS_SSH_KEY != '' }}
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
- name: "Add SSH key"
if: ${{ env.MKDOCS_INSIDERS_SSH_KEY_EXISTS == 'true' }}
uses: webfactory/ssh-agent@v0.8.0
with:
ssh-private-key: ${{ secrets.MKDOCS_INSIDERS_SSH_KEY }}
- name: "Install Rust toolchain"
run: rustup show
- uses: Swatinem/rust-cache@v2
- name: "Install Insiders dependencies"
if: ${{ env.MKDOCS_INSIDERS_SSH_KEY_EXISTS == 'true' }}
run: pip install -r docs/requirements-insiders.txt
- name: "Install dependencies"
run: |
pip install -r docs/requirements.txt
if: ${{ env.MKDOCS_INSIDERS_SSH_KEY_EXISTS != 'true' }}
run: pip install -r docs/requirements.txt
- name: "Copy README File"
run: |
python scripts/transform_readme.py --target mkdocs
python scripts/generate_mkdocs.py
mkdocs build --strict
- name: "Build Insiders docs"
if: ${{ env.MKDOCS_INSIDERS_SSH_KEY_EXISTS == 'true' }}
run: mkdocs build --strict -f mkdocs.insiders.yml
- name: "Build docs"
if: ${{ env.MKDOCS_INSIDERS_SSH_KEY_EXISTS != 'true' }}
run: mkdocs build --strict -f mkdocs.generated.yml
- name: "Deploy to Cloudflare Pages"
if: ${{ env.CF_API_TOKEN_EXISTS == 'true' }}
uses: cloudflare/wrangler-action@2.0.0

View File

@@ -9,7 +9,7 @@ concurrency:
env:
PACKAGE_NAME: flake8-to-ruff
CRATE_NAME: flake8_to_ruff
PYTHON_VERSION: "3.7" # to build abi3 wheels
PYTHON_VERSION: "3.11"
CARGO_INCREMENTAL: 0
CARGO_NET_RETRY: 10
CARGO_TERM_COLOR: always

View File

@@ -9,7 +9,7 @@ on:
sha:
description: "Optionally, the full sha of the commit to be released"
type: string
push:
pull_request:
paths:
# When we change pyproject.toml, we want to ensure that the maturin builds still work
- pyproject.toml
@@ -20,7 +20,7 @@ concurrency:
env:
PACKAGE_NAME: ruff
PYTHON_VERSION: "3.7" # to build abi3 wheels
PYTHON_VERSION: "3.11"
CARGO_INCREMENTAL: 0
CARGO_NET_RETRY: 10
CARGO_TERM_COLOR: always

5
.gitignore vendored
View File

@@ -1,8 +1,7 @@
# Benchmarking cpython (CONTRIBUTING.md)
crates/ruff/resources/test/cpython
# generate_mkdocs.py
mkdocs.yml
.overrides
mkdocs.generated.yml
# check_ecosystem.py
ruff-old
github_search*.jsonl
@@ -11,7 +10,7 @@ schemastore
# `maturin develop` and ecosystem_all_check.sh
.venv*
# Formatter debugging (crates/ruff_python_formatter/README.md)
scratch.py
scratch.*
# Created by `perf` (CONTRIBUTING.md)
perf.data
perf.data.old

View File

@@ -1,6 +1,10 @@
# default to true for all rules
default: true
# MD007/unordered-list-indent
MD007:
indent: 4
# MD033/no-inline-html
MD033: false
@@ -8,7 +12,4 @@ MD033: false
MD041: false
# MD013/line-length
MD013:
line_length: 100
code_blocks: false
ignore_code_blocks: true
MD013: false

View File

@@ -22,6 +22,7 @@ repos:
hooks:
- id: mdformat
additional_dependencies:
- mdformat-mkdocs
- mdformat-black
- black==23.1.0 # Must be the latest version of Black

View File

@@ -1,5 +1,41 @@
# Breaking Changes
## 0.0.277
### `.ipynb_checkpoints`, `.pyenv`, `.pytest_cache`, and `.vscode` are now excluded by default ([#5513](https://github.com/astral-sh/ruff/pull/5513))
Ruff maintains a list of default exclusions, which now consists of the following patterns:
- `.bzr`
- `.direnv`
- `.eggs`
- `.git`
- `.git-rewrite`
- `.hg`
- `.ipynb_checkpoints`
- `.mypy_cache`
- `.nox`
- `.pants.d`
- `.pyenv`
- `.pytest_cache`
- `.pytype`
- `.ruff_cache`
- `.svn`
- `.tox`
- `.venv`
- `.vscode`
- `__pypackages__`
- `_build`
- `buck-out`
- `build`
- `dist`
- `node_modules`
- `venv`
Previously, the `.ipynb_checkpoints`, `.pyenv`, `.pytest_cache`, and `.vscode` directories were not
excluded by default. This change brings Ruff's default exclusions in line with other tools like
Black.
## 0.0.276
### The `keep-runtime-typing` setting has been reinstated ([#5470](https://github.com/astral-sh/ruff/pull/5470))
@@ -12,12 +48,12 @@ Taking `UP006` (rewrite `List[int]` to `list[int]`) as an example, the setting n
follows:
- On Python 3.7 and Python 3.8, setting `keep-runtime-typing = true` will cause Ruff to ignore
`UP006` violations, even if `from __future__ import annotations` is present in the file.
While such annotations are valid in Python 3.7 and Python 3.8 when combined with
`from __future__ import annotations`, they aren't supported by libraries like Pydantic and
FastAPI, which rely on runtime type checking.
`UP006` violations, even if `from __future__ import annotations` is present in the file.
While such annotations are valid in Python 3.7 and Python 3.8 when combined with
`from __future__ import annotations`, they aren't supported by libraries like Pydantic and
FastAPI, which rely on runtime type checking.
- On Python 3.9 and above, the setting has no effect, as `list[int]` is a valid type annotation,
and libraries like Pydantic and FastAPI support it without issue.
and libraries like Pydantic and FastAPI support it without issue.
In short: `keep-runtime-typing` can be used to ensure that Ruff doesn't introduce type annotations
that are not supported at runtime by the current Python version, which are unsupported by libraries
@@ -167,25 +203,25 @@ This change is largely backwards compatible -- most users should experience
no change in behavior. However, please note the following exceptions:
- Subcommands will now fail when invoked with unsupported arguments, instead
of silently ignoring them. For example, the following will now fail:
of silently ignoring them. For example, the following will now fail:
```console
ruff --clean --respect-gitignore
```
```console
ruff --clean --respect-gitignore
```
(the `clean` command doesn't support `--respect-gitignore`.)
(the `clean` command doesn't support `--respect-gitignore`.)
- The semantics of `ruff <arg>` have changed slightly when `<arg>` is a valid subcommand.
For example, prior to this release, running `ruff rule` would run `ruff` over a file or
directory called `rule`. Now, `ruff rule` would invoke the `rule` subcommand. This should
only impact projects with files or directories named `rule`, `check`, `explain`, `clean`,
or `generate-shell-completion`.
For example, prior to this release, running `ruff rule` would run `ruff` over a file or
directory called `rule`. Now, `ruff rule` would invoke the `rule` subcommand. This should
only impact projects with files or directories named `rule`, `check`, `explain`, `clean`,
or `generate-shell-completion`.
- Scripts that invoke ruff should supply `--` before any positional arguments.
(The semantics of `ruff -- <arg>` have not changed.)
(The semantics of `ruff -- <arg>` have not changed.)
- `--explain` previously treated `--format grouped` as a synonym for `--format text`.
This is no longer supported; instead, use `--format text`.
This is no longer supported; instead, use `--format text`.
## 0.0.226

View File

@@ -6,10 +6,10 @@
- [Scope](#scope)
- [Enforcement](#enforcement)
- [Enforcement Guidelines](#enforcement-guidelines)
- [1. Correction](#1-correction)
- [2. Warning](#2-warning)
- [3. Temporary Ban](#3-temporary-ban)
- [4. Permanent Ban](#4-permanent-ban)
- [1. Correction](#1-correction)
- [2. Warning](#2-warning)
- [3. Temporary Ban](#3-temporary-ban)
- [4. Permanent Ban](#4-permanent-ban)
- [Attribution](#attribution)
## Our Pledge
@@ -33,20 +33,20 @@ community include:
- Being respectful of differing opinions, viewpoints, and experiences
- Giving and gracefully accepting constructive feedback
- Accepting responsibility and apologizing to those affected by our mistakes,
and learning from the experience
and learning from the experience
- Focusing on what is best not just for us as individuals, but for the
overall community
overall community
Examples of unacceptable behavior include:
- The use of sexualized language or imagery, and sexual attention or
advances of any kind
advances of any kind
- Trolling, insulting or derogatory comments, and personal or political attacks
- Public or private harassment
- Publishing others' private information, such as a physical or email
address, without their explicit permission
address, without their explicit permission
- Other conduct which could reasonably be considered inappropriate in a
professional setting
professional setting
## Enforcement Responsibilities

View File

@@ -3,16 +3,29 @@
Welcome! We're happy to have you here. Thank you in advance for your contribution to Ruff.
- [The Basics](#the-basics)
- [Prerequisites](#prerequisites)
- [Development](#development)
- [Project Structure](#project-structure)
- [Example: Adding a new lint rule](#example-adding-a-new-lint-rule)
- [Rule naming convention](#rule-naming-convention)
- [Rule testing: fixtures and snapshots](#rule-testing-fixtures-and-snapshots)
- [Example: Adding a new configuration option](#example-adding-a-new-configuration-option)
- [Prerequisites](#prerequisites)
- [Development](#development)
- [Project Structure](#project-structure)
- [Example: Adding a new lint rule](#example-adding-a-new-lint-rule)
- [Rule naming convention](#rule-naming-convention)
- [Rule testing: fixtures and snapshots](#rule-testing-fixtures-and-snapshots)
- [Example: Adding a new configuration option](#example-adding-a-new-configuration-option)
- [MkDocs](#mkdocs)
- [Release Process](#release-process)
- [Benchmarks](#benchmarking-and-profiling)
- [Creating a new release](#creating-a-new-release)
- [Ecosystem CI](#ecosystem-ci)
- [Benchmarking and Profiling](#benchmarking-and-profiling)
- [CPython Benchmark](#cpython-benchmark)
- [Microbenchmarks](#microbenchmarks)
- [Benchmark-driven Development](#benchmark-driven-development)
- [PR Summary](#pr-summary)
- [Tips](#tips)
- [Profiling Projects](#profiling-projects)
- [Linux](#linux)
- [Mac](#mac)
- [`cargo dev`](#cargo-dev)
- [Subsystems](#subsystems)
- [Compilation Pipeline](#compilation-pipeline)
## The Basics
@@ -23,7 +36,10 @@ For small changes (e.g., bug fixes), feel free to submit a PR.
For larger changes (e.g., new lint rules, new functionality, new configuration options), consider
creating an [**issue**](https://github.com/astral-sh/ruff/issues) outlining your proposed change.
You can also join us on [**Discord**](https://discord.gg/c9MhzV8aU5) to discuss your idea with the
community.
community. We've labeled [beginner-friendly tasks](https://github.com/astral-sh/ruff/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22)
in the issue tracker, along with [bugs](https://github.com/astral-sh/ruff/issues?q=is%3Aissue+is%3Aopen+label%3Abug)
and [improvements](https://github.com/astral-sh/ruff/issues?q=is%3Aissue+is%3Aopen+label%3Aaccepted)
that are ready for contributions.
If you're looking for a place to start, we recommend implementing a new lint rule (see:
[_Adding a new lint rule_](#example-adding-a-new-lint-rule), which will allow you to learn from and
@@ -34,6 +50,8 @@ As a concrete example: consider taking on one of the rules from the [`flake8-pyi
plugin, and looking to the originating [Python source](https://github.com/PyCQA/flake8-pyi) for
guidance.
If you have suggestions on how we might improve the contributing documentation, [let us know](https://github.com/astral-sh/ruff/discussions/5693)!
### Prerequisites
Ruff is written in Rust. You'll need to install the
@@ -92,48 +110,56 @@ The vast majority of the code, including all lint rules, lives in the `ruff` cra
At time of writing, the repository includes the following crates:
- `crates/ruff`: library crate containing all lint rules and the core logic for running them.
If you're working on a rule, this is the crate for you.
- `crates/ruff_benchmark`: binary crate for running micro-benchmarks.
- `crates/ruff_cache`: library crate for caching lint results.
- `crates/ruff_cli`: binary crate containing Ruff's command-line interface.
- `crates/ruff_dev`: binary crate containing utilities used in the development of Ruff itself (e.g.,
`cargo dev generate-all`).
- `crates/ruff_diagnostics`: library crate for the lint diagnostics APIs.
- `crates/ruff_formatter`: library crate for generic code formatting logic based on an intermediate
representation.
`cargo dev generate-all`), see the [`cargo dev`](#cargo-dev) section below.
- `crates/ruff_diagnostics`: library crate for the rule-independent abstractions in the lint
diagnostics APIs.
- `crates/ruff_formatter`: library crate for language agnostic code formatting logic based on an
intermediate representation. The backend for `ruff_python_formatter`.
- `crates/ruff_index`: library crate inspired by `rustc_index`.
- `crates/ruff_macros`: library crate containing macros used by Ruff.
- `crates/ruff_python_ast`: library crate containing Python-specific AST types and utilities.
- `crates/ruff_python_formatter`: library crate containing Python-specific code formatting logic.
- `crates/ruff_macros`: proc macro crate containing macros used by Ruff.
- `crates/ruff_python_ast`: library crate containing Python-specific AST types and utilities. Note
that the AST schema itself is defined in the
[rustpython-ast](https://github.com/astral-sh/RustPython-Parser) crate.
- `crates/ruff_python_formatter`: library crate implementing the Python formatter. Emits an
intermediate representation for each node, which `ruff_formatter` prints based on the configured
line length.
- `crates/ruff_python_semantic`: library crate containing Python-specific semantic analysis logic,
including Ruff's semantic model.
- `crates/ruff_python_stdlib`: library crate containing Python-specific standard library data.
including Ruff's semantic model. Used to resolve queries like "What import does this variable
refer to?"
- `crates/ruff_python_stdlib`: library crate containing Python-specific standard library data, e.g.
the names of all built-in exceptions and which standard library types are immutable.
- `crates/ruff_python_whitespace`: library crate containing Python-specific whitespace analysis
logic.
logic (indentation and newlines).
- `crates/ruff_rustpython`: library crate containing `RustPython`-specific utilities.
- `crates/ruff_testing_macros`: library crate containing macros used for testing Ruff.
- `crates/ruff_textwrap`: library crate to indent and dedent Python source code.
- `crates/ruff_wasm`: library crate for exposing Ruff as a WebAssembly module.
- `crates/ruff_wasm`: library crate for exposing Ruff as a WebAssembly module. Powers the
[Ruff Playground](https://play.ruff.rs/).
### Example: Adding a new lint rule
At a high level, the steps involved in adding a new lint rule are as follows:
1. Determine a name for the new rule as per our [rule naming convention](#rule-naming-convention)
(e.g., `AssertFalse`, as in, "allow `assert False`").
(e.g., `AssertFalse`, as in, "allow `assert False`").
1. Create a file for your rule (e.g., `crates/ruff/src/rules/flake8_bugbear/rules/assert_false.rs`).
1. In that file, define a violation struct (e.g., `pub struct AssertFalse`). You can grep for
`#[violation]` to see examples.
`#[violation]` to see examples.
1. In that file, define a function that adds the violation to the diagnostic list as appropriate
(e.g., `pub(crate) fn assert_false`) based on whatever inputs are required for the rule (e.g.,
an `ast::StmtAssert` node).
(e.g., `pub(crate) fn assert_false`) based on whatever inputs are required for the rule (e.g.,
an `ast::StmtAssert` node).
1. Define the logic for triggering the violation in `crates/ruff/src/checkers/ast/mod.rs` (for
AST-based checks), `crates/ruff/src/checkers/tokens.rs` (for token-based checks),
`crates/ruff/src/checkers/lines.rs` (for text-based checks), or
`crates/ruff/src/checkers/filesystem.rs` (for filesystem-based checks).
AST-based checks), `crates/ruff/src/checkers/tokens.rs` (for token-based checks),
`crates/ruff/src/checkers/lines.rs` (for text-based checks), or
`crates/ruff/src/checkers/filesystem.rs` (for filesystem-based checks).
1. Map the violation struct to a rule code in `crates/ruff/src/codes.rs` (e.g., `B011`).
@@ -166,13 +192,13 @@ suppression comment would be framed as "allow `assert False`".
As such, rule names should...
- Highlight the pattern that is being linted against, rather than the preferred alternative.
For example, `AssertFalse` guards against `assert False` statements.
For example, `AssertFalse` guards against `assert False` statements.
- _Not_ contain instructions on how to fix the violation, which instead belong in the rule
documentation and the `autofix_title`.
documentation and the `autofix_title`.
- _Not_ contain a redundant prefix, like `Disallow` or `Banned`, which are already implied by the
convention.
convention.
When re-implementing rules from other linters, we prioritize adhering to this convention over
preserving the original rule name.
@@ -187,25 +213,25 @@ Ruff's output for each fixture, which you can then commit alongside your changes
Once you've completed the code for the rule itself, you can define tests with the following steps:
1. Add a Python file to `crates/ruff/resources/test/fixtures/[linter]` that contains the code you
want to test. The file name should match the rule name (e.g., `E402.py`), and it should include
examples of both violations and non-violations.
want to test. The file name should match the rule name (e.g., `E402.py`), and it should include
examples of both violations and non-violations.
1. Run Ruff locally against your file and verify the output is as expected. Once you're satisfied
with the output (you see the violations you expect, and no others), proceed to the next step.
For example, if you're adding a new rule named `E402`, you would run:
with the output (you see the violations you expect, and no others), proceed to the next step.
For example, if you're adding a new rule named `E402`, you would run:
```shell
cargo run -p ruff_cli -- check crates/ruff/resources/test/fixtures/pycodestyle/E402.py --no-cache
```
```shell
cargo run -p ruff_cli -- check crates/ruff/resources/test/fixtures/pycodestyle/E402.py --no-cache
```
1. Add the test to the relevant `crates/ruff/src/rules/[linter]/mod.rs` file. If you're contributing
a rule to a pre-existing set, you should be able to find a similar example to pattern-match
against. If you're adding a new linter, you'll need to create a new `mod.rs` file (see,
e.g., `crates/ruff/src/rules/flake8_bugbear/mod.rs`)
a rule to a pre-existing set, you should be able to find a similar example to pattern-match
against. If you're adding a new linter, you'll need to create a new `mod.rs` file (see,
e.g., `crates/ruff/src/rules/flake8_bugbear/mod.rs`)
1. Run `cargo test`. Your test will fail, but you'll be prompted to follow-up
with `cargo insta review`. Run `cargo insta review`, review and accept the generated snapshot,
then commit the snapshot file alongside the rest of your changes.
with `cargo insta review`. Run `cargo insta review`, review and accept the generated snapshot,
then commit the snapshot file alongside the rest of your changes.
1. Run `cargo test` again to ensure that your test passes.
@@ -243,21 +269,25 @@ To preview any changes to the documentation locally:
1. Install MkDocs and Material for MkDocs with:
```shell
pip install -r docs/requirements.txt
```
```shell
pip install -r docs/requirements.txt
```
1. Generate the MkDocs site with:
```shell
python scripts/generate_mkdocs.py
```
```shell
python scripts/generate_mkdocs.py
```
1. Run the development server with:
```shell
mkdocs serve
```
```shell
# For contributors.
mkdocs serve -f mkdocs.generated.yml
# For members of the Astral org, which has access to MkDocs Insiders via sponsorship.
mkdocs serve -f mkdocs.insiders.yml
```
The documentation should then be available locally at
[http://127.0.0.1:8000/docs/](http://127.0.0.1:8000/docs/).
@@ -278,20 +308,19 @@ even patch releases may contain [non-backwards-compatible changes](https://semve
1. Create a PR with the version and `BREAKING_CHANGES.md` updated
1. Merge the PR
1. Run the release workflow with the version number (without starting `v`) as input. Make sure
main has your merged PR as last commit
main has your merged PR as last commit
1. The release workflow will do the following:
1. Build all the assets. If this fails (even though we tested in step 4), we havent tagged or
uploaded anything, you can restart after pushing a fix
1. Upload to pypi
1. Create and push the git tag (from pyproject.toml). We create the git tag only here
because we can't change it ([#4468](https://github.com/charliermarsh/ruff/issues/4468)), so
we want to make sure everything up to and including publishing to pypi worked.
1. Attach artifacts to draft GitHub release
1. Trigger downstream repositories. This can fail without causing fallout, it is possible (if
inconvenient) to trigger the downstream jobs manually
1. Create release notes in GitHub UI and promote from draft to proper release(<https://github.com/charliermarsh/ruff/releases/new>)
1. Build all the assets. If this fails (even though we tested in step 4), we havent tagged or
uploaded anything, you can restart after pushing a fix.
1. Upload to PyPI.
1. Create and push the Git tag (as extracted from `pyproject.toml`). We create the Git tag only
after building the wheels and uploading to PyPI, since we can't delete or modify the tag ([#4468](https://github.com/charliermarsh/ruff/issues/4468)).
1. Attach artifacts to draft GitHub release
1. Trigger downstream repositories. This can fail non-catastrophically, as we can run any
downstream jobs manually if needed.
1. Create release notes in GitHub UI and promote from draft.
1. If needed, [update the schemastore](https://github.com/charliermarsh/ruff/blob/main/scripts/update_schemastore.py)
1. If needed, update ruff-lsp and ruff-vscode
1. If needed, update the `ruff-lsp` and `ruff-vscode` repositories.
## Ecosystem CI
@@ -390,6 +419,13 @@ Summary
159.43 ± 2.48 times faster than 'pycodestyle crates/ruff/resources/test/cpython'
```
To benchmark a subset of rules, e.g. `LineTooLong` and `DocLineTooLong`:
```shell
cargo build --release && hyperfine --warmup 10 \
"./target/release/ruff ./crates/ruff/resources/test/cpython/ --no-cache -e --select W505,E501"
```
You can run `poetry install` from `./scripts/benchmarks` to create a working environment for the
above. All reported benchmarks were computed using the versions specified by
`./scripts/benchmarks/pyproject.toml` on Python 3.11.
@@ -434,7 +470,7 @@ Benchmark 1: find . -type f -name "*.py" | xargs -P 0 pyupgrade --py311-plus
Range (min … max): 29.813 s … 30.356 s 10 runs
```
## Microbenchmarks
### Microbenchmarks
The `ruff_benchmark` crate benchmarks the linter and the formatter on individual files.
@@ -444,7 +480,7 @@ You can run the benchmarks with
cargo benchmark
```
### Benchmark driven Development
#### Benchmark-driven Development
Ruff uses [Criterion.rs](https://bheisler.github.io/criterion.rs/book/) for benchmarks. You can use
`--save-baseline=<name>` to store an initial baseline benchmark (e.g. on `main`) and then use
@@ -459,7 +495,7 @@ cargo benchmark --save-baseline=main
cargo benchmark --baseline=main
```
### PR Summary
#### PR Summary
You can use `--save-baseline` and `critcmp` to get a pretty comparison between two recordings.
This is useful to illustrate the improvements of a PR.
@@ -480,21 +516,21 @@ You must install [`critcmp`](https://github.com/BurntSushi/critcmp) for the comp
cargo install critcmp
```
### Tips
#### Tips
- Use `cargo benchmark <filter>` to only run specific benchmarks. For example: `cargo benchmark linter/pydantic`
to only run the pydantic tests.
to only run the pydantic tests.
- Use `cargo benchmark --quiet` for a more cleaned up output (without statistical relevance)
- Use `cargo benchmark --quick` to get faster results (more prone to noise)
## Profiling Projects
### Profiling Projects
You can either use the microbenchmarks from above or a project directory for benchmarking. There
are a lot of profiling tools out there,
[The Rust Performance Book](https://nnethercote.github.io/perf-book/profiling.html) lists some
examples.
### Linux
#### Linux
Install `perf` and build `ruff_benchmark` with the `release-debug` profile and then run it with perf
@@ -527,7 +563,7 @@ An alternative is to convert the perf data to `flamegraph.svg` using
flamegraph --perfdata perf.data
```
### Mac
#### Mac
Install [`cargo-instruments`](https://crates.io/crates/cargo-instruments):
@@ -542,7 +578,179 @@ cargo instruments -t time --bench linter --profile release-debug -p ruff_benchma
```
- `-t`: Specifies what to profile. Useful options are `time` to profile the wall time and `alloc`
for profiling the allocations.
for profiling the allocations.
- You may want to pass an additional filter to run a single test file
Otherwise, follow the instructions from the linux section.
## `cargo dev`
`cargo dev` is a shortcut for `cargo run --package ruff_dev --bin ruff_dev`. You can run some useful
utils with it:
- `cargo dev print-ast <file>`: Print the AST of a python file using the
[RustPython parser](https://github.com/astral-sh/RustPython-Parser/tree/main/parser) that is
mainly used in Ruff. For `if True: pass # comment`, you can see the syntax tree, the byte offsets
for start and stop of each node and also how the `:` token, the comment and whitespace are not
represented anymore:
```text
[
If(
StmtIf {
range: 0..13,
test: Constant(
ExprConstant {
range: 3..7,
value: Bool(
true,
),
kind: None,
},
),
body: [
Pass(
StmtPass {
range: 9..13,
},
),
],
orelse: [],
},
),
]
```
- `cargo dev print-tokens <file>`: Print the tokens that the AST is built upon. Again for
`if True: pass # comment`:
```text
0 If 2
3 True 7
7 Colon 8
9 Pass 13
14 Comment(
"# comment",
) 23
23 Newline 24
```
- `cargo dev print-cst <file>`: Print the CST of a python file using
[LibCST](https://github.com/Instagram/LibCST), which is used in addition to the RustPython parser
in Ruff. E.g. for `if True: pass # comment` everything including the whitespace is represented:
```text
Module {
body: [
Compound(
If(
If {
test: Name(
Name {
value: "True",
lpar: [],
rpar: [],
},
),
body: SimpleStatementSuite(
SimpleStatementSuite {
body: [
Pass(
Pass {
semicolon: None,
},
),
],
leading_whitespace: SimpleWhitespace(
" ",
),
trailing_whitespace: TrailingWhitespace {
whitespace: SimpleWhitespace(
" ",
),
comment: Some(
Comment(
"# comment",
),
),
newline: Newline(
None,
Real,
),
},
},
),
orelse: None,
leading_lines: [],
whitespace_before_test: SimpleWhitespace(
" ",
),
whitespace_after_test: SimpleWhitespace(
"",
),
is_elif: false,
},
),
),
],
header: [],
footer: [],
default_indent: " ",
default_newline: "\n",
has_trailing_newline: true,
encoding: "utf-8",
}
```
- `cargo dev generate-all`: Update `ruff.schema.json`, `docs/configuration.md` and `docs/rules`.
You can also set `RUFF_UPDATE_SCHEMA=1` to update `ruff.schema.json` during `cargo test`.
- `cargo dev generate-cli-help`, `cargo dev generate-docs` and `cargo dev generate-json-schema`:
Update just `docs/configuration.md`, `docs/rules` and `ruff.schema.json` respectively.
- `cargo dev generate-options`: Generate a markdown-compatible table of all `pyproject.toml`
options. Used for <https://beta.ruff.rs/docs/settings/>
- `cargo dev generate-rules-table`: Generate a markdown-compatible table of all rules. Used for <https://beta.ruff.rs/docs/rules/>
- `cargo dev round-trip <python file or jupyter notebook>`: Read a Python file or Jupyter Notebook,
parse it, serialize the parsed representation and write it back. Used to check how good our
representation is so that fixes don't rewrite irrelevant parts of a file.
- `cargo dev format_dev`: See ruff_python_formatter README.md
## Subsystems
### Compilation Pipeline
If we view Ruff as a compiler, in which the inputs are paths to Python files and the outputs are
diagnostics, then our current compilation pipeline proceeds as follows:
1. **File discovery**: Given paths like `foo/`, locate all Python files in any specified subdirectories, taking into account our hierarchical settings system and any `exclude` options.
1. **Package resolution**: Determine the “package root” for every file by traversing over its parent directories and looking for `__init__.py` files.
1. **Cache initialization**: For every “package root”, initialize an empty cache.
1. **Analysis**: For every file, in parallel:
1. **Cache read**: If the file is cached (i.e., its modification timestamp hasn't changed since it was last analyzed), short-circuit, and return the cached diagnostics.
1. **Tokenization**: Run the lexer over the file to generate a token stream.
1. **Indexing**: Extract metadata from the token stream, such as: comment ranges, `# noqa` locations, `# isort: off` locations, “doc lines”, etc.
1. **Token-based rule evaluation**: Run any lint rules that are based on the contents of the token stream (e.g., commented-out code).
1. **Filesystem-based rule evaluation**: Run any lint rules that are based on the contents of the filesystem (e.g., lack of `__init__.py` file in a package).
1. **Logical line-based rule evaluation**: Run any lint rules that are based on logical lines (e.g., stylistic rules).
1. **Parsing**: Run the parser over the token stream to produce an AST. (This consumes the token stream, so anything that relies on the token stream needs to happen before parsing.)
1. **AST-based rule evaluation**: Run any lint rules that are based on the AST. This includes the vast majority of lint rules. As part of this step, we also build the semantic model for the current file as we traverse over the AST. Some lint rules are evaluated eagerly, as we iterate over the AST, while others are evaluated in a deferred manner (e.g., unused imports, since we cant determine whether an import is unused until weve finished analyzing the entire file), after weve finished the initial traversal.
1. **Import-based rule evaluation**: Run any lint rules that are based on the modules imports (e.g., import sorting). These could, in theory, be included in the AST-based rule evaluation phase — theyre just separated for simplicity.
1. **Physical line-based rule evaluation**: Run any lint rules that are based on physical lines (e.g., line-length).
1. **Suppression enforcement**: Remove any violations that are suppressed via `# noqa` directives or `per-file-ignores`.
1. **Cache write**: Write the generated diagnostics to the package cache using the file as a key.
1. **Reporting**: Print diagnostics in the specified format (text, JSON, etc.), to the specified output channel (stdout, a file, etc.).

448
Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -45,19 +45,21 @@ strum = { version = "0.24.1", features = ["strum_macros"] }
strum_macros = { version = "0.24.3" }
syn = { version = "2.0.15" }
test-case = { version = "3.0.0" }
thiserror = { version = "1.0.43" }
toml = { version = "0.7.2" }
wsl = { version = "0.1.0" }
# v0.0.1
libcst = { git = "https://github.com/charliermarsh/LibCST", rev = "80e4c1399f95e5beb532fdd1e209ad2dbb470438" }
# v1.0.1
libcst = { git = "https://github.com/Instagram/LibCST.git", rev = "3cacca1a1029f05707e50703b49fe3dd860aa839", default-features = false }
# Please tag the RustPython version everytime you update its revision here and in fuzz/Cargo.toml
# Please tag the RustPython version every time you update its revision here and in fuzz/Cargo.toml
# Tagging the version ensures that older ruff versions continue to build from source even when we rebase our RustPython fork.
# Current tag: v0.0.7
ruff_text_size = { git = "https://github.com/astral-sh/RustPython-Parser.git", rev = "c174bbf1f29527edd43d432326327f16f47ab9e0" }
rustpython-ast = { git = "https://github.com/astral-sh/RustPython-Parser.git", rev = "c174bbf1f29527edd43d432326327f16f47ab9e0" , default-features = false, features = ["num-bigint"]}
rustpython-format = { git = "https://github.com/astral-sh/RustPython-Parser.git", rev = "c174bbf1f29527edd43d432326327f16f47ab9e0", default-features = false, features = ["num-bigint"] }
rustpython-literal = { git = "https://github.com/astral-sh/RustPython-Parser.git", rev = "c174bbf1f29527edd43d432326327f16f47ab9e0", default-features = false }
rustpython-parser = { git = "https://github.com/astral-sh/RustPython-Parser.git", rev = "c174bbf1f29527edd43d432326327f16f47ab9e0" , default-features = false, features = ["full-lexer", "num-bigint"] }
# Note: As of tag v0.0.8 we are cherry-picking commits instead of rebasing so the tag is not necessary
ruff_text_size = { git = "https://github.com/astral-sh/RustPython-Parser.git", rev = "126652b684910c29a7bcc32293d4ca0f81454e34" }
rustpython-ast = { git = "https://github.com/astral-sh/RustPython-Parser.git", rev = "126652b684910c29a7bcc32293d4ca0f81454e34" , default-features = false, features = ["num-bigint"]}
rustpython-format = { git = "https://github.com/astral-sh/RustPython-Parser.git", rev = "126652b684910c29a7bcc32293d4ca0f81454e34", default-features = false, features = ["num-bigint"] }
rustpython-literal = { git = "https://github.com/astral-sh/RustPython-Parser.git", rev = "126652b684910c29a7bcc32293d4ca0f81454e34", default-features = false }
rustpython-parser = { git = "https://github.com/astral-sh/RustPython-Parser.git", rev = "126652b684910c29a7bcc32293d4ca0f81454e34" , default-features = false, features = ["full-lexer", "num-bigint"] }
[profile.release]
lto = "fat"

View File

@@ -32,9 +32,10 @@ An extremely fast Python linter, written in Rust.
- 🔧 Autofix support, for automatic error correction (e.g., automatically remove unused imports)
- 📏 Over [500 built-in rules](https://beta.ruff.rs/docs/rules/)
- ⚖️ [Near-parity](https://beta.ruff.rs/docs/faq/#how-does-ruff-compare-to-flake8) with the
built-in Flake8 rule set
built-in Flake8 rule set
- 🔌 Native re-implementations of dozens of Flake8 plugins, like flake8-bugbear
- ⌨️ First-party editor integrations for [VS Code](https://github.com/astral-sh/ruff-vscode) and [more](https://github.com/astral-sh/ruff-lsp)
- ⌨️ First-party [editor integrations](https://beta.ruff.rs/docs/editor-integrations/) for
[VS Code](https://github.com/astral-sh/ruff-vscode) and [more](https://github.com/astral-sh/ruff-lsp)
- 🌎 Monorepo-friendly, with [hierarchical and cascading configuration](https://beta.ruff.rs/docs/configuration/#pyprojecttoml-discovery)
Ruff aims to be orders of magnitude faster than alternative tools while integrating more
@@ -139,7 +140,7 @@ Ruff can also be used as a [pre-commit](https://pre-commit.com) hook:
```yaml
- repo: https://github.com/astral-sh/ruff-pre-commit
# Ruff version.
rev: v0.0.276
rev: v0.0.278
hooks:
- id: ruff
```
@@ -347,6 +348,7 @@ Ruff is released under the MIT license.
Ruff is used by a number of major open-source projects and companies, including:
- Amazon ([AWS SAM](https://github.com/aws/serverless-application-model))
- Anthropic ([Python SDK](https://github.com/anthropics/anthropic-sdk-python))
- [Apache Airflow](https://github.com/apache/airflow)
- AstraZeneca ([Magnus](https://github.com/AstraZeneca/magnus-core))
- Benchling ([Refac](https://github.com/benchling/refac))
@@ -356,26 +358,30 @@ Ruff is used by a number of major open-source projects and companies, including:
- [DVC](https://github.com/iterative/dvc)
- [Dagger](https://github.com/dagger/dagger)
- [Dagster](https://github.com/dagster-io/dagster)
- Databricks ([MLflow](https://github.com/mlflow/mlflow))
- [FastAPI](https://github.com/tiangolo/fastapi)
- [Gradio](https://github.com/gradio-app/gradio)
- [Great Expectations](https://github.com/great-expectations/great_expectations)
- [HTTPX](https://github.com/encode/httpx)
- Hugging Face ([Transformers](https://github.com/huggingface/transformers),
[Datasets](https://github.com/huggingface/datasets),
[Diffusers](https://github.com/huggingface/diffusers))
[Datasets](https://github.com/huggingface/datasets),
[Diffusers](https://github.com/huggingface/diffusers))
- [Hatch](https://github.com/pypa/hatch)
- [Home Assistant](https://github.com/home-assistant/core)
- ING Bank ([popmon](https://github.com/ing-bank/popmon), [probatus](https://github.com/ing-bank/probatus))
- [Ibis](https://github.com/ibis-project/ibis)
- [Jupyter](https://github.com/jupyter-server/jupyter_server)
- [LangChain](https://github.com/hwchase17/langchain)
- [LlamaIndex](https://github.com/jerryjliu/llama_index)
- Matrix ([Synapse](https://github.com/matrix-org/synapse))
- [MegaLinter](https://github.com/oxsecurity/megalinter)
- Meltano ([Meltano CLI](https://github.com/meltano/meltano), [Singer SDK](https://github.com/meltano/sdk))
- Microsoft ([Semantic Kernel](https://github.com/microsoft/semantic-kernel),
[ONNX Runtime](https://github.com/microsoft/onnxruntime),
[LightGBM](https://github.com/microsoft/LightGBM))
- Modern Treasury ([Python SDK](https://github.com/Modern-Treasury/modern-treasury-python-sdk))
- Mozilla ([Firefox](https://github.com/mozilla/gecko-dev))
- [MegaLinter](https://github.com/oxsecurity/megalinter)
- Microsoft ([Semantic Kernel](https://github.com/microsoft/semantic-kernel),
[ONNX Runtime](https://github.com/microsoft/onnxruntime),
[LightGBM](https://github.com/microsoft/LightGBM))
- [Mypy](https://github.com/python/mypy)
- Netflix ([Dispatch](https://github.com/Netflix/dispatch))
- [Neon](https://github.com/neondatabase/neon)
- [ONNX](https://github.com/onnx/onnx)
@@ -411,6 +417,7 @@ Ruff is used by a number of major open-source projects and companies, including:
- [featuretools](https://github.com/alteryx/featuretools)
- [meson-python](https://github.com/mesonbuild/meson-python)
- [nox](https://github.com/wntrblm/nox)
- [pip](https://github.com/pypa/pip)
### Show Your Support

View File

@@ -2,7 +2,6 @@
extend-exclude = ["resources", "snapshots"]
[default.extend-words]
trivias = "trivias"
hel = "hel"
whos = "whos"
spawnve = "spawnve"

View File

@@ -1,6 +1,6 @@
[package]
name = "flake8-to-ruff"
version = "0.0.276"
version = "0.0.278"
description = """
Convert Flake8 configuration files to Ruff configuration files.
"""

View File

@@ -82,12 +82,12 @@ flake8-to-ruff path/to/.flake8 --plugin flake8-builtins --plugin flake8-quotes
## Limitations
1. Ruff only supports a subset of the Flake configuration options. `flake8-to-ruff` will warn on and
ignore unsupported options in the `.flake8` file (or equivalent). (Similarly, Ruff has a few
configuration options that don't exist in Flake8.)
ignore unsupported options in the `.flake8` file (or equivalent). (Similarly, Ruff has a few
configuration options that don't exist in Flake8.)
1. Ruff will omit any rule codes that are unimplemented or unsupported by Ruff, including rule
codes from unsupported plugins. (See the
[documentation](https://beta.ruff.rs/docs/faq/#how-does-ruff-compare-to-flake8) for the complete
list of supported plugins.)
codes from unsupported plugins. (See the
[documentation](https://beta.ruff.rs/docs/faq/#how-does-ruff-compare-to-flake8) for the complete
list of supported plugins.)
## License

View File

@@ -1,6 +1,6 @@
[package]
name = "ruff"
version = "0.0.276"
version = "0.0.278"
publish = false
authors = { workspace = true }
edition = { workspace = true }
@@ -73,11 +73,12 @@ shellexpand = { workspace = true }
smallvec = { workspace = true }
strum = { workspace = true }
strum_macros = { workspace = true }
thiserror = { version = "1.0.38" }
thiserror = { version = "1.0.43" }
toml = { workspace = true }
typed-arena = { version = "2.0.2" }
unicode-width = { version = "0.1.10" }
unicode_names2 = { version = "0.6.0", git = "https://github.com/youknowone/unicode_names2.git", rev = "4ce16aa85cbcdd9cc830410f1a72ef9a235f2fde" }
wsl = { version = "0.1.0" }
[dev-dependencies]
insta = { workspace = true }

View File

@@ -1,4 +1,4 @@
from typing import Any, Type
from typing import Annotated, Any, Optional, Type, Union
from typing_extensions import override
# Error
@@ -95,27 +95,27 @@ class Foo:
def foo(self: "Foo", a: int, *params: str, **options: Any) -> int:
pass
# ANN401
# OK
@override
def foo(self: "Foo", a: Any, *params: str, **options: str) -> int:
pass
# ANN401
# OK
@override
def foo(self: "Foo", a: int, *params: str, **options: str) -> Any:
pass
# ANN401
# OK
@override
def foo(self: "Foo", a: int, *params: Any, **options: Any) -> int:
pass
# ANN401
# OK
@override
def foo(self: "Foo", a: int, *params: Any, **options: str) -> int:
pass
# ANN401
# OK
@override
def foo(self: "Foo", a: int, *params: str, **options: Any) -> int:
pass
@@ -137,3 +137,18 @@ class Foo:
# OK
def f(*args: *tuple[int]) -> None: ...
def f(a: object) -> None: ...
def f(a: str | bytes) -> None: ...
def f(a: Union[str, bytes]) -> None: ...
def f(a: Optional[str]) -> None: ...
def f(a: Annotated[str, ...]) -> None: ...
def f(a: "Union[str, bytes]") -> None: ...
def f(a: int + int) -> None: ...
# ANN401
def f(a: Any | int) -> None: ...
def f(a: int | Any) -> None: ...
def f(a: Union[str, bytes, Any]) -> None: ...
def f(a: Optional[Any]) -> None: ...
def f(a: Annotated[Any, ...]) -> None: ...
def f(a: "Union[str, bytes, Any]") -> None: ...

View File

@@ -177,6 +177,9 @@ def str_okay(value=str("foo")):
def bool_okay(value=bool("bar")):
pass
# Allow immutable bytes() value
def bytes_okay(value=bytes(1)):
pass
# Allow immutable int() value
def int_okay(value=int("12")):

View File

@@ -0,0 +1,27 @@
import re
from re import sub
# B034
re.sub("a", "b", "aaa", re.IGNORECASE)
re.sub("a", "b", "aaa", 5)
re.sub("a", "b", "aaa", 5, re.IGNORECASE)
re.subn("a", "b", "aaa", re.IGNORECASE)
re.subn("a", "b", "aaa", 5)
re.subn("a", "b", "aaa", 5, re.IGNORECASE)
re.split(" ", "a a a a", re.I)
re.split(" ", "a a a a", 2)
re.split(" ", "a a a a", 2, re.I)
sub("a", "b", "aaa", re.IGNORECASE)
# OK
re.sub("a", "b", "aaa")
re.sub("a", "b", "aaa", flags=re.IGNORECASE)
re.sub("a", "b", "aaa", count=5)
re.sub("a", "b", "aaa", count=5, flags=re.IGNORECASE)
re.subn("a", "b", "aaa")
re.subn("a", "b", "aaa", flags=re.IGNORECASE)
re.subn("a", "b", "aaa", count=5)
re.subn("a", "b", "aaa", count=5, flags=re.IGNORECASE)
re.split(" ", "a a a a", flags=re.I)
re.split(" ", "a a a a", maxsplit=2)
re.split(" ", "a a a a", maxsplit=2, flags=re.I)

View File

@@ -14,9 +14,10 @@ except AssertionError:
except Exception as err:
assert err
raise Exception("No cause here...")
except BaseException as base_err:
# Might use this instead of bare raise with the `.with_traceback()` method
raise base_err
except BaseException as err:
raise err
except BaseException as err:
raise some_other_err
finally:
raise Exception("Nothing to chain from, so no warning here")

View File

@@ -12,7 +12,8 @@ set(reversed(x))
sorted(list(x))
sorted(tuple(x))
sorted(sorted(x))
sorted(sorted(x, key=lambda y: y))
sorted(sorted(x, key=foo, reverse=False), reverse=False, key=foo)
sorted(sorted(x, reverse=True), reverse=True)
sorted(reversed(x))
sorted(list(x), key=lambda y: y)
tuple(
@@ -21,3 +22,9 @@ tuple(
"o"]
)
)
# Nested sorts with differing keyword arguments. Not flagged.
sorted(sorted(x, key=lambda y: y))
sorted(sorted(x, key=lambda y: y), key=lambda x: x)
sorted(sorted(x), reverse=True)
sorted(sorted(x, reverse=False), reverse=True)

View File

@@ -25,10 +25,15 @@ map(lambda x=2, y=1: x + y, nums, nums)
set(map(lambda x, y: x, nums, nums))
def myfunc(arg1: int, arg2: int = 4):
def func(arg1: int, arg2: int = 4):
return 2 * arg1 + arg2
list(map(myfunc, nums))
# Non-error: `func` is not a lambda.
list(map(func, nums))
[x for x in nums]
# False positive: need to preserve the late-binding of `x` in the inner lambda.
map(lambda x: lambda: x, range(4))
# Error: the `x` is overridden by the inner lambda.
map(lambda x: lambda x: x, range(4))

View File

@@ -19,3 +19,6 @@ from datetime import datetime
# no args unqualified
datetime(2000, 1, 1, 0, 0, 0)
# uses `astimezone` method
datetime(2000, 1, 1, 0, 0, 0).astimezone()

View File

@@ -7,3 +7,6 @@ from datetime import datetime
# unqualified
datetime.today()
# uses `astimezone` method
datetime.today().astimezone()

View File

@@ -7,3 +7,6 @@ from datetime import datetime
# unqualified
datetime.utcnow()
# uses `astimezone` method
datetime.utcnow().astimezone()

View File

@@ -7,3 +7,6 @@ from datetime import datetime
# unqualified
datetime.utcfromtimestamp(1234)
# uses `astimezone` method
datetime.utcfromtimestamp(1234).astimezone()

View File

@@ -16,3 +16,6 @@ from datetime import datetime
# no args unqualified
datetime.now()
# uses `astimezone` method
datetime.now().astimezone()

View File

@@ -16,3 +16,6 @@ from datetime import datetime
# no args unqualified
datetime.fromtimestamp(1234)
# uses `astimezone` method
datetime.fromtimestamp(1234).astimezone()

View File

@@ -5,15 +5,18 @@ import matplotlib.pyplot # unconventional
import numpy # unconventional
import pandas # unconventional
import seaborn # unconventional
import tkinter # unconventional
import altair as altr # unconventional
import matplotlib.pyplot as plot # unconventional
import numpy as nmp # unconventional
import pandas as pdas # unconventional
import seaborn as sbrn # unconventional
import tkinter as tkr # unconventional
import altair as alt # conventional
import matplotlib.pyplot as plt # conventional
import numpy as np # conventional
import pandas as pd # conventional
import seaborn as sns # conventional
import tkinter as tk # conventional

View File

@@ -1,3 +1,5 @@
import typing
# Shouldn't affect non-union field types.
field1: str
@@ -30,3 +32,45 @@ field10: (str | int) | str # PYI016: Duplicate union member `str`
# Should emit for nested unions.
field11: dict[int | int, str]
# Should emit for unions with more than two cases
field12: int | int | int # Error
field13: int | int | int | int # Error
# Should emit for unions with more than two cases, even if not directly adjacent
field14: int | int | str | int # Error
# Should emit for duplicate literal types; also covered by PYI030
field15: typing.Literal[1] | typing.Literal[1] # Error
# Shouldn't emit if in new parent type
field16: int | dict[int, str] # OK
# Shouldn't emit if not in a union parent
field17: dict[int, int] # OK
# Should emit in cases with newlines
field18: typing.Union[
set[
int # foo
],
set[
int # bar
],
] # Error, newline and comment will not be emitted in message
# Should emit in cases with `typing.Union` instead of `|`
field19: typing.Union[int, int] # Error
# Should emit in cases with nested `typing.Union`
field20: typing.Union[int, typing.Union[int, str]] # Error
# Should emit in cases with mixed `typing.Union` and `|`
field21: typing.Union[int, int | str] # Error
# Should emit only once in cases with multiple nested `typing.Union`
field22: typing.Union[int, typing.Union[int, typing.Union[int, int]]] # Error
# Should emit in cases with newlines
field23: set[ # foo
int] | set[int]

View File

@@ -0,0 +1,24 @@
from typing import Literal
# Shouldn't emit for any cases in the non-stub file for compatibility with flake8-pyi.
# Note that this rule could be applied here in the future.
field1: Literal[1] # OK
field2: Literal[1] | Literal[2] # OK
def func1(arg1: Literal[1] | Literal[2]): # OK
print(arg1)
def func2() -> Literal[1] | Literal[2]: # OK
return "my Literal[1]ing"
field3: Literal[1] | Literal[2] | str # OK
field4: str | Literal[1] | Literal[2] # OK
field5: Literal[1] | str | Literal[2] # OK
field6: Literal[1] | bool | Literal[2] | str # OK
field7 = Literal[1] | Literal[2] # OK
field8: Literal[1] | (Literal[2] | str) # OK
field9: Literal[1] | (Literal[2] | str) # OK
field10: (Literal[1] | str) | Literal[2] # OK
field11: dict[Literal[1] | Literal[2], str] # OK

View File

@@ -0,0 +1,86 @@
import typing
import typing_extensions
from typing import Literal
# Shouldn't affect non-union field types.
field1: Literal[1] # OK
# Should emit for duplicate field types.
field2: Literal[1] | Literal[2] # Error
# Should emit for union types in arguments.
def func1(arg1: Literal[1] | Literal[2]): # Error
print(arg1)
# Should emit for unions in return types.
def func2() -> Literal[1] | Literal[2]: # Error
return "my Literal[1]ing"
# Should emit in longer unions, even if not directly adjacent.
field3: Literal[1] | Literal[2] | str # Error
field4: str | Literal[1] | Literal[2] # Error
field5: Literal[1] | str | Literal[2] # Error
field6: Literal[1] | bool | Literal[2] | str # Error
# Should emit for non-type unions.
field7 = Literal[1] | Literal[2] # Error
# Should emit for parenthesized unions.
field8: Literal[1] | (Literal[2] | str) # Error
# Should handle user parentheses when fixing.
field9: Literal[1] | (Literal[2] | str) # Error
field10: (Literal[1] | str) | Literal[2] # Error
# Should emit for union in generic parent type.
field11: dict[Literal[1] | Literal[2], str] # Error
# Should emit for unions with more than two cases
field12: Literal[1] | Literal[2] | Literal[3] # Error
field13: Literal[1] | Literal[2] | Literal[3] | Literal[4] # Error
# Should emit for unions with more than two cases, even if not directly adjacent
field14: Literal[1] | Literal[2] | str | Literal[3] # Error
# Should emit for unions with mixed literal internal types
field15: Literal[1] | Literal["foo"] | Literal[True] # Error
# Shouldn't emit for duplicate field types with same value; covered by Y016
field16: Literal[1] | Literal[1] # OK
# Shouldn't emit if in new parent type
field17: Literal[1] | dict[Literal[2], str] # OK
# Shouldn't emit if not in a union parent
field18: dict[Literal[1], Literal[2]] # OK
# Should respect name of literal type used
field19: typing.Literal[1] | typing.Literal[2] # Error
# Should emit in cases with newlines
field20: typing.Union[
Literal[
1 # test
],
Literal[2],
] # Error, newline and comment will not be emitted in message
# Should handle multiple unions with multiple members
field21: Literal[1, 2] | Literal[3, 4] # Error
# Should emit in cases with `typing.Union` instead of `|`
field22: typing.Union[Literal[1], Literal[2]] # Error
# Should emit in cases with `typing_extensions.Literal`
field23: typing_extensions.Literal[1] | typing_extensions.Literal[2] # Error
# Should emit in cases with nested `typing.Union`
field24: typing.Union[Literal[1], typing.Union[Literal[2], str]] # Error
# Should emit in cases with mixed `typing.Union` and `|`
field25: typing.Union[Literal[1], Literal[2] | str] # Error
# Should emit only once in cases with multiple nested `typing.Union`
field24: typing.Union[Literal[1], typing.Union[Literal[2], typing.Union[Literal[3], Literal[4]]]] # Error

View File

@@ -0,0 +1,75 @@
import builtins
import types
import typing
from collections.abc import Awaitable
from types import TracebackType
from typing import Any, Type
import _typeshed
import typing_extensions
from _typeshed import Unused
class GoodOne:
def __exit__(self, *args: object) -> None: ...
async def __aexit__(self, *args) -> str: ...
class GoodTwo:
def __exit__(self, typ: type[builtins.BaseException] | None, *args: builtins.object) -> bool | None: ...
async def __aexit__(self, /, typ: Type[BaseException] | None, *args: object, **kwargs) -> bool: ...
class GoodThree:
def __exit__(self, __typ: typing.Type[BaseException] | None, exc: BaseException | None, *args: object) -> None: ...
async def __aexit__(self, typ: typing_extensions.Type[BaseException] | None, __exc: BaseException | None, *args: object) -> None: ...
class GoodFour:
def __exit__(self, typ: type[BaseException] | None, exc: BaseException | None, tb: TracebackType | None) -> None: ...
async def __aexit__(self, typ: type[BaseException] | None, exc: BaseException | None, tb: types.TracebackType | None, *args: list[None]) -> None: ...
class GoodFive:
def __exit__(self, typ: type[BaseException] | None, exc: BaseException | None, tb: TracebackType | None, weird_extra_arg: int = ..., *args: int, **kwargs: str) -> None: ...
async def __aexit__(self, typ: type[BaseException] | None, exc: BaseException | None, tb: TracebackType | None) -> Awaitable[None]: ...
class GoodSix:
def __exit__(self, typ: object, exc: builtins.object, tb: object) -> None: ...
async def __aexit__(self, typ: object, exc: object, tb: builtins.object) -> None: ...
class GoodSeven:
def __exit__(self, *args: Unused) -> bool: ...
async def __aexit__(self, typ: Type[BaseException] | None, *args: _typeshed.Unused) -> Awaitable[None]: ...
class GoodEight:
def __exit__(self, __typ: typing.Type[BaseException] | None, exc: BaseException | None, *args: _typeshed.Unused) -> bool: ...
async def __aexit__(self, typ: type[BaseException] | None, exc: BaseException | None, tb: TracebackType | None, weird_extra_arg: int = ..., *args: Unused, **kwargs: Unused) -> Awaitable[None]: ...
class GoodNine:
def __exit__(self, __typ: typing.Union[typing.Type[BaseException] , None], exc: typing.Union[BaseException , None], *args: _typeshed.Unused) -> bool: ...
async def __aexit__(self, typ: typing.Union[typing.Type[BaseException], None], exc: typing.Union[BaseException , None], tb: typing.Union[TracebackType , None], weird_extra_arg: int = ..., *args: Unused, **kwargs: Unused) -> Awaitable[None]: ...
class GoodTen:
def __exit__(self, __typ: typing.Optional[typing.Type[BaseException]], exc: typing.Optional[BaseException], *args: _typeshed.Unused) -> bool: ...
async def __aexit__(self, typ: typing.Optional[typing.Type[BaseException]], exc: typing.Optional[BaseException], tb: typing.Optional[TracebackType], weird_extra_arg: int = ..., *args: Unused, **kwargs: Unused) -> Awaitable[None]: ...
class BadOne:
def __exit__(self, *args: Any) -> None: ... # PYI036: Bad star-args annotation
async def __aexit__(self) -> None: ... # PYI036: Missing args
class BadTwo:
def __exit__(self, typ, exc, tb, weird_extra_arg) -> None: ... # PYI036: Extra arg must have default
async def __aexit__(self, typ, exc, tb, *, weird_extra_arg) -> None: ...# PYI036: Extra arg must have default
class BadThree:
def __exit__(self, typ: type[BaseException], exc: BaseException | None, tb: TracebackType | None) -> None: ... # PYI036: First arg has bad annotation
async def __aexit__(self, __typ: type[BaseException] | None, __exc: BaseException, __tb: TracebackType) -> bool | None: ... # PYI036: Second arg has bad annotation
class BadFour:
def __exit__(self, typ: typing.Optional[type[BaseException]], exc: typing.Union[BaseException, None], tb: TracebackType) -> None: ... # PYI036: Third arg has bad annotation
async def __aexit__(self, __typ: type[BaseException] | None, __exc: BaseException | None, __tb: typing.Union[TracebackType, None, int]) -> bool | None: ... # PYI036: Third arg has bad annotation
class BadFive:
def __exit__(self, typ: BaseException | None, *args: list[str]) -> bool: ... # PYI036: Bad star-args annotation
async def __aexit__(self, /, typ: type[BaseException] | None, *args: Any) -> Awaitable[None]: ... # PYI036: Bad star-args annotation
class BadSix:
def __exit__(self, typ, exc, tb, weird_extra_arg, extra_arg2 = None) -> None: ... # PYI036: Extra arg must have default
async def __aexit__(self, typ, exc, tb, *, weird_extra_arg) -> None: ... # PYI036: kwargs must have default

View File

@@ -0,0 +1,75 @@
import builtins
import types
import typing
from collections.abc import Awaitable
from types import TracebackType
from typing import Any, Type
import _typeshed
import typing_extensions
from _typeshed import Unused
class GoodOne:
def __exit__(self, *args: object) -> None: ...
async def __aexit__(self, *args) -> str: ...
class GoodTwo:
def __exit__(self, typ: type[builtins.BaseException] | None, *args: builtins.object) -> bool | None: ...
async def __aexit__(self, /, typ: Type[BaseException] | None, *args: object, **kwargs) -> bool: ...
class GoodThree:
def __exit__(self, __typ: typing.Type[BaseException] | None, exc: BaseException | None, *args: object) -> None: ...
async def __aexit__(self, typ: typing_extensions.Type[BaseException] | None, __exc: BaseException | None, *args: object) -> None: ...
class GoodFour:
def __exit__(self, typ: type[BaseException] | None, exc: BaseException | None, tb: TracebackType | None) -> None: ...
async def __aexit__(self, typ: type[BaseException] | None, exc: BaseException | None, tb: types.TracebackType | None, *args: list[None]) -> None: ...
class GoodFive:
def __exit__(self, typ: type[BaseException] | None, exc: BaseException | None, tb: TracebackType | None, weird_extra_arg: int = ..., *args: int, **kwargs: str) -> None: ...
async def __aexit__(self, typ: type[BaseException] | None, exc: BaseException | None, tb: TracebackType | None) -> Awaitable[None]: ...
class GoodSix:
def __exit__(self, typ: object, exc: builtins.object, tb: object) -> None: ...
async def __aexit__(self, typ: object, exc: object, tb: builtins.object) -> None: ...
class GoodSeven:
def __exit__(self, *args: Unused) -> bool: ...
async def __aexit__(self, typ: Type[BaseException] | None, *args: _typeshed.Unused) -> Awaitable[None]: ...
class GoodEight:
def __exit__(self, __typ: typing.Type[BaseException] | None, exc: BaseException | None, *args: _typeshed.Unused) -> bool: ...
async def __aexit__(self, typ: type[BaseException] | None, exc: BaseException | None, tb: TracebackType | None, weird_extra_arg: int = ..., *args: Unused, **kwargs: Unused) -> Awaitable[None]: ...
class GoodNine:
def __exit__(self, __typ: typing.Union[typing.Type[BaseException] , None], exc: typing.Union[BaseException , None], *args: _typeshed.Unused) -> bool: ...
async def __aexit__(self, typ: typing.Union[typing.Type[BaseException], None], exc: typing.Union[BaseException , None], tb: typing.Union[TracebackType , None], weird_extra_arg: int = ..., *args: Unused, **kwargs: Unused) -> Awaitable[None]: ...
class GoodTen:
def __exit__(self, __typ: typing.Optional[typing.Type[BaseException]], exc: typing.Optional[BaseException], *args: _typeshed.Unused) -> bool: ...
async def __aexit__(self, typ: typing.Optional[typing.Type[BaseException]], exc: typing.Optional[BaseException], tb: typing.Optional[TracebackType], weird_extra_arg: int = ..., *args: Unused, **kwargs: Unused) -> Awaitable[None]: ...
class BadOne:
def __exit__(self, *args: Any) -> None: ... # PYI036: Bad star-args annotation
async def __aexit__(self) -> None: ... # PYI036: Missing args
class BadTwo:
def __exit__(self, typ, exc, tb, weird_extra_arg) -> None: ... # PYI036: Extra arg must have default
async def __aexit__(self, typ, exc, tb, *, weird_extra_arg1, weird_extra_arg2) -> None: ...# PYI036: kwargs must have default
class BadThree:
def __exit__(self, typ: type[BaseException], exc: BaseException | None, tb: TracebackType | None) -> None: ... # PYI036: First arg has bad annotation
async def __aexit__(self, __typ: type[BaseException] | None, __exc: BaseException, __tb: TracebackType) -> bool | None: ... # PYI036: Second arg has bad annotation
class BadFour:
def __exit__(self, typ: typing.Optional[type[BaseException]], exc: typing.Union[BaseException, None], tb: TracebackType) -> None: ... # PYI036: Third arg has bad annotation
async def __aexit__(self, __typ: type[BaseException] | None, __exc: BaseException | None, __tb: typing.Union[TracebackType, None, int]) -> bool | None: ... # PYI036: Third arg has bad annotation
class BadFive:
def __exit__(self, typ: BaseException | None, *args: list[str]) -> bool: ... # PYI036: Bad star-args annotation
async def __aexit__(self, /, typ: type[BaseException] | None, *args: Any) -> Awaitable[None]: ... # PYI036: Bad star-args annotation
class BadSix:
def __exit__(self, typ, exc, tb, weird_extra_arg, extra_arg2 = None) -> None: ... # PYI036: Extra arg must have default
async def __aexit__(self, typ, exc, tb, *, weird_extra_arg) -> None: ... # PYI036: kwargs must have default

View File

@@ -0,0 +1,47 @@
from typing import (
Union,
)
from typing_extensions import (
TypeAlias,
)
TA0: TypeAlias = int
TA1: TypeAlias = int | float | bool
TA2: TypeAlias = Union[int, float, bool]
def good1(arg: int) -> int | bool:
...
def good2(arg: int, arg2: int | bool) -> None:
...
def f0(arg1: float | int) -> None:
...
def f1(arg1: float, *, arg2: float | list[str] | type[bool] | complex) -> None:
...
def f2(arg1: int, /, arg2: int | int | float) -> None:
...
def f3(arg1: int, *args: Union[int | int | float]) -> None:
...
async def f4(**kwargs: int | int | float) -> None:
...
class Foo:
def good(self, arg: int) -> None:
...
def bad(self, arg: int | float | complex) -> None:
...

View File

@@ -0,0 +1,39 @@
from typing import (
Union,
)
from typing_extensions import (
TypeAlias,
)
# Type aliases not flagged
TA0: TypeAlias = int
TA1: TypeAlias = int | float | bool
TA2: TypeAlias = Union[int, float, bool]
def good1(arg: int) -> int | bool: ...
def good2(arg: int, arg2: int | bool) -> None: ...
def f0(arg1: float | int) -> None: ... # PYI041
def f1(arg1: float, *, arg2: float | list[str] | type[bool] | complex) -> None: ... # PYI041
def f2(arg1: int, /, arg2: int | int | float) -> None: ... # PYI041
def f3(arg1: int, *args: Union[int | int | float]) -> None: ... # PYI041
async def f4(**kwargs: int | int | float) -> None: ... # PYI041
class Foo:
def good(self, arg: int) -> None: ...
def bad(self, arg: int | float | complex) -> None: ... # PYI041

View File

@@ -29,6 +29,26 @@ raise TypeError(
# Hello, world!
)
# OK
raise AssertionError
# OK
raise AttributeError("test message")
def return_error():
return ValueError("Something")
# OK
raise return_error()
class Class:
@staticmethod
def error():
return ValueError("Something")
# OK
raise Class.error()

View File

@@ -4,3 +4,10 @@ class Bad(str): # SLOT000
class Good(str): # Ok
__slots__ = ["foo"]
from enum import Enum
class Fine(str, Enum): # Ok
__slots__ = ["foo"]

View File

@@ -0,0 +1,9 @@
import A
import B
import b
import C
import d
import E
import f
from g import a, B, c
from h import A, b, C

View File

@@ -26,3 +26,9 @@ def f():
import os # isort:skip
import collections
import abc
def f():
import sys; import os # isort:skip
import sys; import os # isort:skip # isort:skip
import sys; import os

View File

@@ -19,3 +19,13 @@ if True:
import D
import B
import e
import f
# isort: split
# isort: split
import d
import c

View File

@@ -0,0 +1,27 @@
import pandas as pd
data = pd.Series(range(1000))
# PD101
data.nunique() <= 1
data.nunique(dropna=True) <= 1
data.nunique(dropna=False) <= 1
data.nunique() == 1
data.nunique(dropna=True) == 1
data.nunique(dropna=False) == 1
data.nunique() != 1
data.nunique(dropna=True) != 1
data.nunique(dropna=False) != 1
data.nunique() > 1
data.dropna().nunique() == 1
data[data.notnull()].nunique() == 1
# No violation of this rule
data.nunique() == 0 # empty
data.nunique() >= 1 # not-empty
data.nunique() < 1 # empty
data.nunique() == 2 # not constant
data.unique() == 1 # not `nunique`
{"hello": "world"}.nunique() == 1 # no pd.Series

View File

@@ -0,0 +1,20 @@
import pandas as pd
# Errors.
df = pd.read_table("data.csv", sep=",")
df = pd.read_table("data.csv", sep=",", header=0)
filename = "data.csv"
df = pd.read_table(filename, sep=",")
df = pd.read_table(filename, sep=",", header=0)
# Non-errors.
df = pd.read_csv("data.csv")
df = pd.read_table("data.tsv")
df = pd.read_table("data.tsv", sep="\t")
df = pd.read_table("data.tsv", sep=",,")
df = pd.read_table("data.tsv", sep=", ")
df = pd.read_table("data.tsv", sep=" ,")
df = pd.read_table("data.tsv", sep=" , ")
not_pd.read_table("data.csv", sep=",")
data = read_table("data.csv", sep=",")
data = read_table

View File

@@ -1,71 +1,101 @@
some_dict = {"a": 12, "b": 32, "c": 44}
for _, value in some_dict.items(): # PERF102
print(value)
def f():
for _, value in some_dict.items(): # PERF102
print(value)
for key, _ in some_dict.items(): # PERF102
print(key)
def f():
for key, _ in some_dict.items(): # PERF102
print(key)
for weird_arg_name, _ in some_dict.items(): # PERF102
print(weird_arg_name)
def f():
for weird_arg_name, _ in some_dict.items(): # PERF102
print(weird_arg_name)
for name, (_, _) in some_dict.items(): # PERF102
pass
def f():
for name, (_, _) in some_dict.items(): # PERF102
print(name)
for name, (value1, _) in some_dict.items(): # OK
pass
def f():
for name, (value1, _) in some_dict.items(): # OK
print(name, value1)
for (key1, _), (_, _) in some_dict.items(): # PERF102
pass
def f():
for (key1, _), (_, _) in some_dict.items(): # PERF102
print(key1)
for (_, (_, _)), (value, _) in some_dict.items(): # PERF102
pass
def f():
for (_, (_, _)), (value, _) in some_dict.items(): # PERF102
print(value)
for (_, key2), (value1, _) in some_dict.items(): # OK
pass
def f():
for (_, key2), (value1, _) in some_dict.items(): # OK
print(key2, value1)
for ((_, key2), (value1, _)) in some_dict.items(): # OK
pass
def f():
for ((_, key2), (value1, _)) in some_dict.items(): # OK
print(key2, value1)
for ((_, key2), (_, _)) in some_dict.items(): # PERF102
pass
def f():
for ((_, key2), (_, _)) in some_dict.items(): # PERF102
print(key2)
for (_, _, _, variants), (r_language, _, _, _) in some_dict.items(): # OK
pass
def f():
for (_, _, _, variants), (r_language, _, _, _) in some_dict.items(): # OK
print(variants, r_language)
for (_, _, (_, variants)), (_, (_, (r_language, _))) in some_dict.items(): # OK
pass
def f():
for (_, _, (_, variants)), (_, (_, (r_language, _))) in some_dict.items(): # OK
print(variants, r_language)
for key, value in some_dict.items(): # OK
print(key, value)
def f():
for key, value in some_dict.items(): # OK
print(key, value)
for _, value in some_dict.items(12): # OK
print(value)
def f():
for _, value in some_dict.items(12): # OK
print(value)
for key in some_dict.keys(): # OK
print(key)
def f():
for key in some_dict.keys(): # OK
print(key)
for value in some_dict.values(): # OK
print(value)
def f():
for value in some_dict.values(): # OK
print(value)
for name, (_, _) in (some_function()).items(): # PERF102
pass
def f():
for name, (_, _) in (some_function()).items(): # PERF102
print(name)
for name, (_, _) in (some_function().some_attribute).items(): # PERF102
pass
def f():
for name, (_, _) in (some_function().some_attribute).items(): # PERF102
print(name)
def f():
for name, unused_value in some_dict.items(): # PERF102
print(name)
def f():
for unused_name, value in some_dict.items(): # PERF102
print(value)

View File

@@ -30,3 +30,18 @@ def f():
result = []
for i in items:
result.append(i) # OK
def f():
items = [1, 2, 3, 4]
result = {}
for i in items:
result[i].append(i) # OK
def f():
items = [1, 2, 3, 4]
result = []
for i in items:
if i not in result:
result.append(i) # OK

View File

@@ -17,3 +17,10 @@ def f():
result = []
for i in items:
result.append(i * i) # OK
def f():
items = [1, 2, 3, 4]
result = {}
for i in items:
result[i].append(i * i) # OK

View File

@@ -36,3 +36,4 @@ if (True) == TrueElement or x == TrueElement:
assert (not foo) in bar
assert {"x": not foo} in bar
assert [42, not foo] in bar
assert not (re.search(r"^.:\\Users\\[^\\]*\\Downloads\\.*") is None)

View File

@@ -36,3 +36,4 @@ if (True) == TrueElement or x == TrueElement:
assert (not foo) in bar
assert {"x": not foo} in bar
assert [42, not foo] in bar
assert not (re.search(r"^.:\\Users\\[^\\]*\\Downloads\\.*") is None)

View File

@@ -48,3 +48,8 @@ x = {
x = {"a": 1, "a": 1}
x = {"a": 1, "b": 2, "a": 1}
x = {
('a', 'b'): 'asdf',
('a', 'b'): 'qwer',
}

View File

@@ -80,3 +80,8 @@ def multiple_assignment():
global CONSTANT # [global-statement]
CONSTANT = 1
CONSTANT = 2
def no_assignment():
"""Shouldn't warn"""
global CONSTANT

View File

@@ -0,0 +1,34 @@
# Errors.
foo == "a" or foo == "b"
foo != "a" and foo != "b"
foo == "a" or foo == "b" or foo == "c"
foo != "a" and foo != "b" and foo != "c"
foo == a or foo == "b" or foo == 3 # Mixed types.
# False negatives (the current implementation doesn't support Yoda conditions).
"a" == foo or "b" == foo or "c" == foo
"a" != foo and "b" != foo and "c" != foo
"a" == foo or foo == "b" or "c" == foo
# OK
foo == "a" and foo == "b" and foo == "c" # `and` mixed with `==`.
foo != "a" or foo != "b" or foo != "c" # `or` mixed with `!=`.
foo == a or foo == b() or foo == c # Call expression.
foo != a or foo() != b or foo != c # Call expression.
foo in {"a", "b", "c"} # Uses membership test already.
foo not in {"a", "b", "c"} # Uses membership test already.
foo == "a" # Single comparison.
foo != "a" # Single comparison.

View File

@@ -0,0 +1,37 @@
from typing import ParamSpec, TypeVar
# Errors.
T = TypeVar("T", covariant=True, contravariant=True)
T = TypeVar(name="T", covariant=True, contravariant=True)
T = ParamSpec("T", covariant=True, contravariant=True)
T = ParamSpec(name="T", covariant=True, contravariant=True)
# Non-errors.
T = TypeVar("T")
T = TypeVar("T", covariant=False)
T = TypeVar("T", contravariant=False)
T = TypeVar("T", covariant=False, contravariant=False)
T = TypeVar("T", covariant=True)
T = TypeVar("T", covariant=True, contravariant=False)
T = TypeVar(name="T", covariant=True, contravariant=False)
T = TypeVar(name="T", covariant=True)
T = TypeVar("T", contravariant=True)
T = TypeVar("T", covariant=False, contravariant=True)
T = TypeVar(name="T", covariant=False, contravariant=True)
T = TypeVar(name="T", contravariant=True)
T = ParamSpec("T")
T = ParamSpec("T", covariant=False)
T = ParamSpec("T", contravariant=False)
T = ParamSpec("T", covariant=False, contravariant=False)
T = ParamSpec("T", covariant=True)
T = ParamSpec("T", covariant=True, contravariant=False)
T = ParamSpec(name="T", covariant=True, contravariant=False)
T = ParamSpec(name="T", covariant=True)
T = ParamSpec("T", contravariant=True)
T = ParamSpec("T", covariant=False, contravariant=True)
T = ParamSpec(name="T", covariant=False, contravariant=True)
T = ParamSpec(name="T", contravariant=True)

View File

@@ -0,0 +1,68 @@
from typing import ParamSpec, TypeVar
# Errors.
T = TypeVar("T", covariant=True)
T = TypeVar("T", covariant=True, contravariant=False)
T = TypeVar("T", contravariant=True)
T = TypeVar("T", covariant=False, contravariant=True)
P = ParamSpec("P", covariant=True)
P = ParamSpec("P", covariant=True, contravariant=False)
P = ParamSpec("P", contravariant=True)
P = ParamSpec("P", covariant=False, contravariant=True)
T_co = TypeVar("T_co")
T_co = TypeVar("T_co", covariant=False)
T_co = TypeVar("T_co", contravariant=False)
T_co = TypeVar("T_co", covariant=False, contravariant=False)
T_co = TypeVar("T_co", contravariant=True)
T_co = TypeVar("T_co", covariant=False, contravariant=True)
P_co = ParamSpec("P_co")
P_co = ParamSpec("P_co", covariant=False)
P_co = ParamSpec("P_co", contravariant=False)
P_co = ParamSpec("P_co", covariant=False, contravariant=False)
P_co = ParamSpec("P_co", contravariant=True)
P_co = ParamSpec("P_co", covariant=False, contravariant=True)
T_contra = TypeVar("T_contra")
T_contra = TypeVar("T_contra", covariant=False)
T_contra = TypeVar("T_contra", contravariant=False)
T_contra = TypeVar("T_contra", covariant=False, contravariant=False)
T_contra = TypeVar("T_contra", covariant=True)
T_contra = TypeVar("T_contra", covariant=True, contravariant=False)
P_contra = ParamSpec("P_contra")
P_contra = ParamSpec("P_contra", covariant=False)
P_contra = ParamSpec("P_contra", contravariant=False)
P_contra = ParamSpec("P_contra", covariant=False, contravariant=False)
P_contra = ParamSpec("P_contra", covariant=True)
P_contra = ParamSpec("P_contra", covariant=True, contravariant=False)
# Non-errors.
T = TypeVar("T")
T = TypeVar("T", covariant=False)
T = TypeVar("T", contravariant=False)
T = TypeVar("T", covariant=False, contravariant=False)
P = ParamSpec("P")
P = ParamSpec("P", covariant=False)
P = ParamSpec("P", contravariant=False)
P = ParamSpec("P", covariant=False, contravariant=False)
T_co = TypeVar("T_co", covariant=True)
T_co = TypeVar("T_co", covariant=True, contravariant=False)
P_co = ParamSpec("P_co", covariant=True)
P_co = ParamSpec("P_co", covariant=True, contravariant=False)
T_contra = TypeVar("T_contra", contravariant=True)
T_contra = TypeVar("T_contra", covariant=False, contravariant=True)
P_contra = ParamSpec("P_contra", contravariant=True)
P_contra = ParamSpec("P_contra", covariant=False, contravariant=True)
# Bivariate types are errors, but not covered by this check.
T = TypeVar("T", covariant=True, contravariant=True)
P = ParamSpec("P", covariant=True, contravariant=True)
T_co = TypeVar("T_co", covariant=True, contravariant=True)
P_co = ParamSpec("P_co", covariant=True, contravariant=True)
T_contra = TypeVar("T_contra", covariant=True, contravariant=True)
P_contra = ParamSpec("P_contra", covariant=True, contravariant=True)

View File

@@ -0,0 +1,56 @@
from typing import TypeVar, ParamSpec, NewType, TypeVarTuple
# Errors.
X = TypeVar("T")
X = TypeVar(name="T")
Y = ParamSpec("T")
Y = ParamSpec(name="T")
Z = NewType("T", int)
Z = NewType(name="T", tp=int)
Ws = TypeVarTuple("Ts")
Ws = TypeVarTuple(name="Ts")
# Non-errors.
T = TypeVar("T")
T = TypeVar(name="T")
T = ParamSpec("T")
T = ParamSpec(name="T")
T = NewType("T", int)
T = NewType(name="T", tp=int)
Ts = TypeVarTuple("Ts")
Ts = TypeVarTuple(name="Ts")
# Errors, but not covered by this rule.
# Non-string literal name.
T = TypeVar(some_str)
T = TypeVar(name=some_str)
T = TypeVar(1)
T = TypeVar(name=1)
T = ParamSpec(some_str)
T = ParamSpec(name=some_str)
T = ParamSpec(1)
T = ParamSpec(name=1)
T = NewType(some_str, int)
T = NewType(name=some_str, tp=int)
T = NewType(1, int)
T = NewType(name=1, tp=int)
Ts = TypeVarTuple(some_str)
Ts = TypeVarTuple(name=some_str)
Ts = TypeVarTuple(1)
Ts = TypeVarTuple(name=1)
# No names provided.
T = TypeVar()
T = ParamSpec()
T = NewType()
T = NewType(tp=int)
Ts = TypeVarTuple()

View File

@@ -27,6 +27,14 @@ def f(x: typing.Union[(str, int), float]) -> None:
...
def f(x: typing.Union[(int,)]) -> None:
...
def f(x: typing.Union[()]) -> None:
...
def f(x: "Union[str, int, Union[float, bytes]]") -> None:
...

View File

@@ -4,23 +4,9 @@ import typing
# with complex annotations
MyType = NamedTuple("MyType", [("a", int), ("b", tuple[str, ...])])
# with default values as list
MyType = NamedTuple(
"MyType",
[("a", int), ("b", str), ("c", list[bool])],
defaults=["foo", [True]],
)
# with namespace
MyType = typing.NamedTuple("MyType", [("a", int), ("b", str)])
# too many default values (OK)
MyType = NamedTuple(
"MyType",
[("a", int), ("b", str)],
defaults=[1, "bar", "baz"],
)
# invalid identifiers (OK)
MyType = NamedTuple("MyType", [("x-y", int), ("b", tuple[str, ...])])
@@ -29,3 +15,10 @@ MyType = typing.NamedTuple("MyType")
# empty fields
MyType = typing.NamedTuple("MyType", [])
# keywords
MyType = typing.NamedTuple("MyType", a=int, b=tuple[str, ...])
# unfixable
MyType = typing.NamedTuple("MyType", [("a", int)], [("b", str)])
MyType = typing.NamedTuple("MyType", [("a", int)], b=str)

View File

@@ -54,6 +54,14 @@ print("foo {} ".format(x))
'''{[b]}'''.format(a)
"{}".format(
1
)
"123456789 {}".format(
1111111111111111111111111111111111111111111111111111111111111111111111111,
)
###
# Non-errors
###
@@ -87,6 +95,9 @@ r'"\N{snowman} {}".format(a)'
"{a}" "{b}".format(a=1, b=1)
"123456789 {}".format(
11111111111111111111111111111111111111111111111111111111111111111111111111,
)
async def c():
return "{}".format(await 3)

View File

@@ -1,3 +1,5 @@
"""A mirror of UP037_1.py, with `from __future__ import annotations`."""
from __future__ import annotations
from typing import (

View File

@@ -0,0 +1,108 @@
"""A mirror of UP037_0.py, without `from __future__ import annotations`."""
from typing import (
Annotated,
Callable,
List,
Literal,
NamedTuple,
Tuple,
TypeVar,
TypedDict,
cast,
)
from mypy_extensions import Arg, DefaultArg, DefaultNamedArg, NamedArg, VarArg
def foo(var: "MyClass") -> "MyClass":
x: "MyClass"
def foo(*, inplace: "bool"):
pass
def foo(*args: "str", **kwargs: "int"):
pass
x: Tuple["MyClass"]
x: Callable[["MyClass"], None]
class Foo(NamedTuple):
x: "MyClass"
class D(TypedDict):
E: TypedDict("E", foo="int", total=False)
class D(TypedDict):
E: TypedDict("E", {"foo": "int"})
x: Annotated["str", "metadata"]
x: Arg("str", "name")
x: DefaultArg("str", "name")
x: NamedArg("str", "name")
x: DefaultNamedArg("str", "name")
x: DefaultNamedArg("str", name="name")
x: VarArg("str")
x: List[List[List["MyClass"]]]
x: NamedTuple("X", [("foo", "int"), ("bar", "str")])
x: NamedTuple("X", fields=[("foo", "int"), ("bar", "str")])
x: NamedTuple(typename="X", fields=[("foo", "int")])
X: MyCallable("X")
# OK
class D(TypedDict):
E: TypedDict("E")
x: Annotated[()]
x: DefaultNamedArg(name="name", quox="str")
x: DefaultNamedArg(name="name")
x: NamedTuple("X", [("foo",), ("bar",)])
x: NamedTuple("X", ["foo", "bar"])
x: NamedTuple()
x: Literal["foo", "bar"]
x = cast(x, "str")
def foo(x, *args, **kwargs):
...
def foo(*, inplace):
...
x: Annotated[1:2] = ...
x = TypeVar("x", "str", "int")
x = cast("str", x)
X = List["MyClass"]

View File

@@ -6,6 +6,7 @@ from fractions import Fraction
from pathlib import Path
from typing import ClassVar, NamedTuple
def default_function() -> list[int]:
return []
@@ -25,12 +26,13 @@ class A:
fine_timedelta: datetime.timedelta = datetime.timedelta(hours=7)
fine_tuple: tuple[int] = tuple([1])
fine_regex: re.Pattern = re.compile(r".*")
fine_float: float = float('-inf')
fine_float: float = float("-inf")
fine_int: int = int(12)
fine_complex: complex = complex(1, 2)
fine_str: str = str("foo")
fine_bool: bool = bool("foo")
fine_fraction: Fraction = Fraction(1,2)
fine_fraction: Fraction = Fraction(1, 2)
DEFAULT_IMMUTABLETYPE_FOR_ALL_DATACLASSES = ImmutableType(40)
DEFAULT_A_FOR_ALL_DATACLASSES = A([1, 2, 3])
@@ -45,3 +47,25 @@ class B:
okay_variant: A = DEFAULT_A_FOR_ALL_DATACLASSES
fine_dataclass_function: list[int] = field(default_factory=list)
class IntConversionDescriptor:
def __init__(self, *, default):
self._default = default
def __set_name__(self, owner, name):
self._name = "_" + name
def __get__(self, obj, type):
if obj is None:
return self._default
return getattr(obj, self._name, self._default)
def __set__(self, obj, value):
setattr(obj, self._name, int(value))
@dataclass
class InventoryItem:
quantity_on_hand: IntConversionDescriptor = IntConversionDescriptor(default=100)

View File

@@ -34,3 +34,7 @@ f"{ascii(bla)}" # OK
" intermediary content "
f" that flows {repr(obj)} of type {type(obj)}.{additional_message}" # RUF010
)
# OK
f"{str({})}"

View File

@@ -48,6 +48,10 @@ def f(arg: typing.Optional[int] = None):
# Union
def f(arg: Union[None] = None):
pass
def f(arg: Union[None, int] = None):
pass
@@ -68,6 +72,10 @@ def f(arg: Union = None): # RUF013
pass
def f(arg: Union[int] = None): # RUF013
pass
def f(arg: Union[int, str] = None): # RUF013
pass
@@ -106,10 +114,18 @@ def f(arg: None = None):
pass
def f(arg: Literal[None] = None):
pass
def f(arg: Literal[1, 2, None, 3] = None):
pass
def f(arg: Literal[1] = None): # RUF013
pass
def f(arg: Literal[1, "foo"] = None): # RUF013
pass

View File

@@ -0,0 +1,44 @@
x = range(10)
# RUF015
list(x)[0]
list(x)[:1]
list(x)[:1:1]
list(x)[:1:2]
tuple(x)[0]
tuple(x)[:1]
tuple(x)[:1:1]
tuple(x)[:1:2]
list(i for i in x)[0]
list(i for i in x)[:1]
list(i for i in x)[:1:1]
list(i for i in x)[:1:2]
[i for i in x][0]
[i for i in x][:1]
[i for i in x][:1:1]
[i for i in x][:1:2]
# OK (not indexing (solely) the first element)
list(x)
list(x)[1]
list(x)[-1]
list(x)[1:]
list(x)[:3:2]
list(x)[::2]
list(x)[::]
[i for i in x]
[i for i in x][1]
[i for i in x][-1]
[i for i in x][1:]
[i for i in x][:3:2]
[i for i in x][::2]
[i for i in x][::]
# OK (doesn't mirror the underlying list)
[i + 1 for i in x][0]
[i for i in x if i > 5][0]
[(i, i + 1) for i in x][0]
# OK (multiple generators)
y = range(10)
[i + j for i in x for j in y][0]

View File

@@ -0,0 +1,115 @@
# Should not emit for valid access with index
var = "abc"[0]
var = f"abc"[0]
var = [1, 2, 3][0]
var = (1, 2, 3)[0]
var = b"abc"[0]
# Should not emit for valid access with slice
var = "abc"[0:2]
var = f"abc"[0:2]
var = b"abc"[0:2]
var = [1, 2, 3][0:2]
var = (1, 2, 3)[0:2]
var = [1, 2, 3][None:2]
var = [1, 2, 3][0:None]
var = [1, 2, 3][:2]
var = [1, 2, 3][0:]
# Should emit for invalid access on strings
var = "abc"["x"]
var = f"abc"["x"]
# Should emit for invalid access on bytes
var = b"abc"["x"]
# Should emit for invalid access on lists and tuples
var = [1, 2, 3]["x"]
var = (1, 2, 3)["x"]
# Should emit for invalid access on list comprehensions
var = [x for x in range(10)]["x"]
# Should emit for invalid access using tuple
var = "abc"[1, 2]
# Should emit for invalid access using string
var = [1, 2]["x"]
# Should emit for invalid access using float
var = [1, 2][0.25]
# Should emit for invalid access using dict
var = [1, 2][{"x": "y"}]
# Should emit for invalid access using dict comp
var = [1, 2][{x: "y" for x in range(2)}]
# Should emit for invalid access using list
var = [1, 2][2, 3]
# Should emit for invalid access using list comp
var = [1, 2][[x for x in range(2)]]
# Should emit on invalid access using set
var = [1, 2][{"x", "y"}]
# Should emit on invalid access using set comp
var = [1, 2][{x for x in range(2)}]
# Should emit on invalid access using bytes
var = [1, 2][b"x"]
# Should emit for non-integer slice start
var = [1, 2, 3]["x":2]
var = [1, 2, 3][f"x":2]
var = [1, 2, 3][1.2:2]
var = [1, 2, 3][{"x"}:2]
var = [1, 2, 3][{x for x in range(2)}:2]
var = [1, 2, 3][{"x": x for x in range(2)}:2]
var = [1, 2, 3][[x for x in range(2)]:2]
# Should emit for non-integer slice end
var = [1, 2, 3][0:"x"]
var = [1, 2, 3][0:f"x"]
var = [1, 2, 3][0:1.2]
var = [1, 2, 3][0:{"x"}]
var = [1, 2, 3][0:{x for x in range(2)}]
var = [1, 2, 3][0:{"x": x for x in range(2)}]
var = [1, 2, 3][0:[x for x in range(2)]]
# Should emit for non-integer slice step
var = [1, 2, 3][0:1:"x"]
var = [1, 2, 3][0:1:f"x"]
var = [1, 2, 3][0:1:1.2]
var = [1, 2, 3][0:1:{"x"}]
var = [1, 2, 3][0:1:{x for x in range(2)}]
var = [1, 2, 3][0:1:{"x": x for x in range(2)}]
var = [1, 2, 3][0:1:[x for x in range(2)]]
# Should emit for non-integer slice start and end; should emit twice with specific ranges
var = [1, 2, 3]["x":"y"]
# Should emit once for repeated invalid access
var = [1, 2, 3]["x"]["y"]["z"]
# Cannot emit on invalid access using variable in index
x = "x"
var = "abc"[x]
# Cannot emit on invalid access using call
def func():
return 1
var = "abc"[func()]
# Cannot emit on invalid access using a variable in parent
x = [1, 2, 3]
var = x["y"]
# Cannot emit for invalid access on byte array
var = bytearray(b"abc")["x"]
# Cannot emit for slice bound using variable
x = "x"
var = [1, 2, 3][0:x]
var = [1, 2, 3][x:1]

View File

@@ -62,6 +62,5 @@ def fine():
def fine():
try:
raise ValueError("a doesn't exist")
except TypeError: # A different exception is caught
print("A different exception is caught")

View File

@@ -2,7 +2,7 @@
use anyhow::{bail, Result};
use ruff_text_size::{TextLen, TextRange, TextSize};
use rustpython_parser::ast::{self, ExceptHandler, Expr, Keyword, Ranged, Stmt};
use rustpython_parser::{lexer, Mode, Tok};
use rustpython_parser::{lexer, Mode};
use ruff_diagnostics::Edit;
use ruff_python_ast::helpers;
@@ -98,7 +98,7 @@ pub(crate) fn remove_argument(
// Case 1: there is only one argument.
let mut count = 0u32;
for (tok, range) in lexer::lex_starts_at(contents, Mode::Module, call_at).flatten() {
if matches!(tok, Tok::Lpar) {
if tok.is_lpar() {
if count == 0 {
fix_start = Some(if remove_parentheses {
range.start()
@@ -109,7 +109,7 @@ pub(crate) fn remove_argument(
count = count.saturating_add(1);
}
if matches!(tok, Tok::Rpar) {
if tok.is_rpar() {
count = count.saturating_sub(1);
if count == 0 {
fix_end = Some(if remove_parentheses {
@@ -131,11 +131,11 @@ pub(crate) fn remove_argument(
let mut seen_comma = false;
for (tok, range) in lexer::lex_starts_at(contents, Mode::Module, call_at).flatten() {
if seen_comma {
if matches!(tok, Tok::NonLogicalNewline) {
if tok.is_non_logical_newline() {
// Also delete any non-logical newlines after the comma.
continue;
}
fix_end = Some(if matches!(tok, Tok::Newline) {
fix_end = Some(if tok.is_newline() {
range.end()
} else {
range.start()
@@ -145,7 +145,7 @@ pub(crate) fn remove_argument(
if range.start() == expr_range.start() {
fix_start = Some(range.start());
}
if fix_start.is_some() && matches!(tok, Tok::Comma) {
if fix_start.is_some() && tok.is_comma() {
seen_comma = true;
}
}
@@ -157,7 +157,7 @@ pub(crate) fn remove_argument(
fix_end = Some(expr_range.end());
break;
}
if matches!(tok, Tok::Comma) {
if tok.is_comma() {
fix_start = Some(range.start());
}
}
@@ -317,10 +317,10 @@ mod tests {
Some(TextSize::from(6))
);
let contents = r#"
let contents = r"
x = 1 \
; y = 1
"#
"
.trim();
let program = Suite::parse(contents, "<filename>")?;
let stmt = program.first().unwrap();
@@ -349,10 +349,10 @@ x = 1 \
TextSize::from(6)
);
let contents = r#"
let contents = r"
x = 1 \
; y = 1
"#
"
.trim();
let locator = Locator::new(contents);
assert_eq!(

File diff suppressed because it is too large Load Diff

View File

@@ -2,6 +2,7 @@
use itertools::Itertools;
use ruff_text_size::{TextLen, TextRange, TextSize};
use rustpython_parser::ast::Ranged;
use ruff_diagnostics::{Diagnostic, Edit, Fix};
use ruff_python_ast::source_code::Locator;
@@ -22,7 +23,7 @@ pub(crate) fn check_noqa(
settings: &Settings,
) -> Vec<usize> {
// Identify any codes that are globally exempted (within the current file).
let exemption = noqa::file_exemption(locator.contents(), comment_ranges);
let exemption = FileExemption::try_extract(locator.contents(), comment_ranges, locator);
// Extract all `noqa` directives.
let mut noqa_directives = NoqaDirectives::from_commented_ranges(comment_ranges, locator);
@@ -37,19 +38,19 @@ pub(crate) fn check_noqa(
}
match &exemption {
FileExemption::All => {
Some(FileExemption::All) => {
// If the file is exempted, ignore all diagnostics.
ignored_diagnostics.push(index);
continue;
}
FileExemption::Codes(codes) => {
Some(FileExemption::Codes(codes)) => {
// If the diagnostic is ignored by a global exemption, ignore it.
if codes.contains(&diagnostic.kind.rule().noqa_code()) {
ignored_diagnostics.push(index);
continue;
}
}
FileExemption::None => {}
None => {}
}
let noqa_offsets = diagnostic
@@ -63,15 +64,15 @@ pub(crate) fn check_noqa(
if let Some(directive_line) = noqa_directives.find_line_with_directive_mut(noqa_offset)
{
let suppressed = match &directive_line.directive {
Directive::All(..) => {
Directive::All(_) => {
directive_line
.matches
.push(diagnostic.kind.rule().noqa_code());
ignored_diagnostics.push(index);
true
}
Directive::Codes(.., codes, _) => {
if noqa::includes(diagnostic.kind.rule(), codes) {
Directive::Codes(directive) => {
if noqa::includes(diagnostic.kind.rule(), directive.codes()) {
directive_line
.matches
.push(diagnostic.kind.rule().noqa_code());
@@ -81,7 +82,6 @@ pub(crate) fn check_noqa(
false
}
}
Directive::None => unreachable!(),
};
if suppressed {
@@ -95,36 +95,31 @@ pub(crate) fn check_noqa(
if analyze_directives && settings.rules.enabled(Rule::UnusedNOQA) {
for line in noqa_directives.lines() {
match &line.directive {
Directive::All(leading_spaces, noqa_range, trailing_spaces) => {
Directive::All(directive) => {
if line.matches.is_empty() {
let mut diagnostic =
Diagnostic::new(UnusedNOQA { codes: None }, *noqa_range);
Diagnostic::new(UnusedNOQA { codes: None }, directive.range());
if settings.rules.should_fix(diagnostic.kind.rule()) {
#[allow(deprecated)]
diagnostic.set_fix_from_edit(delete_noqa(
*leading_spaces,
*noqa_range,
*trailing_spaces,
locator,
));
diagnostic.set_fix_from_edit(delete_noqa(directive.range(), locator));
}
diagnostics.push(diagnostic);
}
}
Directive::Codes(leading_spaces, range, codes, trailing_spaces) => {
Directive::Codes(directive) => {
let mut disabled_codes = vec![];
let mut unknown_codes = vec![];
let mut unmatched_codes = vec![];
let mut valid_codes = vec![];
let mut self_ignore = false;
for code in codes {
for code in directive.codes() {
let code = get_redirect_target(code).unwrap_or(code);
if Rule::UnusedNOQA.noqa_code() == code {
self_ignore = true;
break;
}
if line.matches.iter().any(|m| *m == code)
if line.matches.iter().any(|match_| *match_ == code)
|| settings.external.contains(code)
{
valid_codes.push(code);
@@ -166,29 +161,24 @@ pub(crate) fn check_noqa(
.collect(),
}),
},
*range,
directive.range(),
);
if settings.rules.should_fix(diagnostic.kind.rule()) {
if valid_codes.is_empty() {
#[allow(deprecated)]
diagnostic.set_fix_from_edit(delete_noqa(
*leading_spaces,
*range,
*trailing_spaces,
locator,
));
diagnostic
.set_fix_from_edit(delete_noqa(directive.range(), locator));
} else {
#[allow(deprecated)]
diagnostic.set_fix(Fix::unspecified(Edit::range_replacement(
format!("# noqa: {}", valid_codes.join(", ")),
*range,
directive.range(),
)));
}
}
diagnostics.push(diagnostic);
}
}
Directive::None => {}
}
}
}
@@ -198,38 +188,46 @@ pub(crate) fn check_noqa(
}
/// Generate a [`Edit`] to delete a `noqa` directive.
fn delete_noqa(
leading_spaces: TextSize,
noqa_range: TextRange,
trailing_spaces: TextSize,
locator: &Locator,
) -> Edit {
let line_range = locator.line_range(noqa_range.start());
fn delete_noqa(range: TextRange, locator: &Locator) -> Edit {
let line_range = locator.line_range(range.start());
// Compute the leading space.
let prefix = locator.slice(TextRange::new(line_range.start(), range.start()));
let leading_space = prefix
.rfind(|c: char| !c.is_whitespace())
.map_or(prefix.len(), |i| prefix.len() - i - 1);
let leading_space_len = TextSize::try_from(leading_space).unwrap();
// Compute the trailing space.
let suffix = locator.slice(TextRange::new(range.end(), line_range.end()));
let trailing_space = suffix
.find(|c: char| !c.is_whitespace())
.map_or(suffix.len(), |i| i);
let trailing_space_len = TextSize::try_from(trailing_space).unwrap();
// Ex) `# noqa`
if line_range
== TextRange::new(
noqa_range.start() - leading_spaces,
noqa_range.end() + trailing_spaces,
range.start() - leading_space_len,
range.end() + trailing_space_len,
)
{
let full_line_end = locator.full_line_end(line_range.end());
Edit::deletion(line_range.start(), full_line_end)
}
// Ex) `x = 1 # noqa`
else if noqa_range.end() + trailing_spaces == line_range.end() {
Edit::deletion(noqa_range.start() - leading_spaces, line_range.end())
else if range.end() + trailing_space_len == line_range.end() {
Edit::deletion(range.start() - leading_space_len, line_range.end())
}
// Ex) `x = 1 # noqa # type: ignore`
else if locator.contents()[usize::from(noqa_range.end() + trailing_spaces)..].starts_with('#')
{
Edit::deletion(noqa_range.start(), noqa_range.end() + trailing_spaces)
else if locator.contents()[usize::from(range.end() + trailing_space_len)..].starts_with('#') {
Edit::deletion(range.start(), range.end() + trailing_space_len)
}
// Ex) `x = 1 # noqa here`
else {
Edit::deletion(
noqa_range.start() + "# ".text_len(),
noqa_range.end() + trailing_spaces,
range.start() + "# ".text_len(),
range.end() + trailing_space_len,
)
}
}

View File

@@ -7,9 +7,9 @@ use ruff_diagnostics::Diagnostic;
use ruff_python_ast::source_code::{Indexer, Locator, Stylist};
use ruff_python_whitespace::UniversalNewlines;
use crate::comments::shebang::ShebangDirective;
use crate::registry::Rule;
use crate::rules::flake8_copyright::rules::missing_copyright_notice;
use crate::rules::flake8_executable::helpers::{extract_shebang, ShebangDirective};
use crate::rules::flake8_executable::rules::{
shebang_missing, shebang_newline, shebang_not_executable, shebang_python, shebang_whitespace,
};
@@ -87,32 +87,33 @@ pub(crate) fn check_physical_lines(
|| enforce_shebang_newline
|| enforce_shebang_python
{
let shebang = extract_shebang(&line);
if enforce_shebang_not_executable {
if let Some(diagnostic) = shebang_not_executable(path, line.range(), &shebang) {
diagnostics.push(diagnostic);
if let Some(shebang) = ShebangDirective::try_extract(&line) {
has_any_shebang = true;
if enforce_shebang_not_executable {
if let Some(diagnostic) =
shebang_not_executable(path, line.range(), &shebang)
{
diagnostics.push(diagnostic);
}
}
}
if enforce_shebang_missing {
if !has_any_shebang && matches!(shebang, ShebangDirective::Match(..)) {
has_any_shebang = true;
if enforce_shebang_whitespace {
if let Some(diagnostic) =
shebang_whitespace(line.range(), &shebang, fix_shebang_whitespace)
{
diagnostics.push(diagnostic);
}
}
}
if enforce_shebang_whitespace {
if let Some(diagnostic) =
shebang_whitespace(line.range(), &shebang, fix_shebang_whitespace)
{
diagnostics.push(diagnostic);
if enforce_shebang_newline {
if let Some(diagnostic) =
shebang_newline(line.range(), &shebang, index == 0)
{
diagnostics.push(diagnostic);
}
}
}
if enforce_shebang_newline {
if let Some(diagnostic) = shebang_newline(line.range(), &shebang, index == 0) {
diagnostics.push(diagnostic);
}
}
if enforce_shebang_python {
if let Some(diagnostic) = shebang_python(line.range(), &shebang) {
diagnostics.push(diagnostic);
if enforce_shebang_python {
if let Some(diagnostic) = shebang_python(line.range(), &shebang) {
diagnostics.push(diagnostic);
}
}
}
}

View File

@@ -3,6 +3,9 @@
use rustpython_parser::lexer::LexResult;
use rustpython_parser::Tok;
use ruff_diagnostics::Diagnostic;
use ruff_python_ast::source_code::{Indexer, Locator};
use crate::directives::TodoComment;
use crate::lex::docstring_detection::StateMachine;
use crate::registry::{AsRule, Rule};
@@ -12,8 +15,6 @@ use crate::rules::{
flake8_todos, pycodestyle, pylint, pyupgrade, ruff,
};
use crate::settings::Settings;
use ruff_diagnostics::Diagnostic;
use ruff_python_ast::source_code::{Indexer, Locator};
pub(crate) fn check_tokens(
locator: &Locator,
@@ -88,10 +89,11 @@ pub(crate) fn check_tokens(
};
if matches!(tok, Tok::String { .. } | Tok::Comment(_)) {
diagnostics.extend(ruff::rules::ambiguous_unicode_character(
ruff::rules::ambiguous_unicode_character(
&mut diagnostics,
locator,
range,
if matches!(tok, Tok::String { .. }) {
if tok.is_string() {
if is_docstring {
Context::Docstring
} else {
@@ -101,93 +103,77 @@ pub(crate) fn check_tokens(
Context::Comment
},
settings,
));
);
}
}
}
// ERA001
if enforce_commented_out_code {
diagnostics.extend(eradicate::rules::commented_out_code(
locator, indexer, settings,
));
eradicate::rules::commented_out_code(&mut diagnostics, locator, indexer, settings);
}
// W605
if enforce_invalid_escape_sequence {
for (tok, range) in tokens.iter().flatten() {
if matches!(tok, Tok::String { .. }) {
diagnostics.extend(pycodestyle::rules::invalid_escape_sequence(
if tok.is_string() {
pycodestyle::rules::invalid_escape_sequence(
&mut diagnostics,
locator,
*range,
settings.rules.should_fix(Rule::InvalidEscapeSequence),
));
);
}
}
}
// PLE2510, PLE2512, PLE2513
if enforce_invalid_string_character {
for (tok, range) in tokens.iter().flatten() {
if matches!(tok, Tok::String { .. }) {
diagnostics.extend(
pylint::rules::invalid_string_characters(locator, *range)
.into_iter()
.filter(|diagnostic| settings.rules.enabled(diagnostic.kind.rule())),
);
if tok.is_string() {
pylint::rules::invalid_string_characters(&mut diagnostics, *range, locator);
}
}
}
// E701, E702, E703
if enforce_compound_statements {
diagnostics.extend(
pycodestyle::rules::compound_statements(tokens, locator, indexer, settings)
.into_iter()
.filter(|diagnostic| settings.rules.enabled(diagnostic.kind.rule())),
pycodestyle::rules::compound_statements(
&mut diagnostics,
tokens,
locator,
indexer,
settings,
);
}
// Q001, Q002, Q003
if enforce_quotes {
diagnostics.extend(
flake8_quotes::rules::from_tokens(tokens, locator, settings)
.into_iter()
.filter(|diagnostic| settings.rules.enabled(diagnostic.kind.rule())),
);
flake8_quotes::rules::from_tokens(&mut diagnostics, tokens, locator, settings);
}
// ISC001, ISC002
if enforce_implicit_string_concatenation {
diagnostics.extend(
flake8_implicit_str_concat::rules::implicit(
tokens,
&settings.flake8_implicit_str_concat,
locator,
)
.into_iter()
.filter(|diagnostic| settings.rules.enabled(diagnostic.kind.rule())),
flake8_implicit_str_concat::rules::implicit(
&mut diagnostics,
tokens,
&settings.flake8_implicit_str_concat,
locator,
);
}
// COM812, COM818, COM819
if enforce_trailing_comma {
diagnostics.extend(
flake8_commas::rules::trailing_commas(tokens, locator, settings)
.into_iter()
.filter(|diagnostic| settings.rules.enabled(diagnostic.kind.rule())),
);
flake8_commas::rules::trailing_commas(&mut diagnostics, tokens, locator, settings);
}
// UP034
if enforce_extraneous_parenthesis {
diagnostics.extend(
pyupgrade::rules::extraneous_parentheses(tokens, locator, settings).into_iter(),
);
pyupgrade::rules::extraneous_parentheses(&mut diagnostics, tokens, locator, settings);
}
// PYI033
if enforce_type_comment_in_stub && is_stub {
diagnostics.extend(flake8_pyi::rules::type_comment_in_stub(locator, indexer));
flake8_pyi::rules::type_comment_in_stub(&mut diagnostics, locator, indexer);
}
// TD001, TD002, TD003, TD004, TD005, TD006, TD007
@@ -203,18 +189,12 @@ pub(crate) fn check_tokens(
})
.collect();
diagnostics.extend(
flake8_todos::rules::todos(&todo_comments, locator, indexer, settings)
.into_iter()
.filter(|diagnostic| settings.rules.enabled(diagnostic.kind.rule())),
);
flake8_todos::rules::todos(&mut diagnostics, &todo_comments, locator, indexer, settings);
diagnostics.extend(
flake8_fixme::rules::todos(&todo_comments)
.into_iter()
.filter(|diagnostic| settings.rules.enabled(diagnostic.kind.rule())),
);
flake8_fixme::rules::todos(&mut diagnostics, &todo_comments);
}
diagnostics.retain(|diagnostic| settings.rules.enabled(diagnostic.kind.rule()));
diagnostics
}

View File

@@ -14,6 +14,18 @@ use crate::rules;
#[derive(PartialEq, Eq, PartialOrd, Ord)]
pub struct NoqaCode(&'static str, &'static str);
impl NoqaCode {
/// Return the prefix for the [`NoqaCode`], e.g., `SIM` for `SIM101`.
pub fn prefix(&self) -> &str {
self.0
}
/// Return the suffix for the [`NoqaCode`], e.g., `101` for `SIM101`.
pub fn suffix(&self) -> &str {
self.1
}
}
impl std::fmt::Debug for NoqaCode {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
std::fmt::Display::fmt(self, f)
@@ -156,6 +168,9 @@ pub fn code_to_rule(linter: Linter, code: &str) -> Option<(RuleGroup, Rule)> {
(Pyflakes, "901") => (RuleGroup::Unspecified, rules::pyflakes::rules::RaiseNotImplemented),
// pylint
(Pylint, "C0105") => (RuleGroup::Unspecified, rules::pylint::rules::TypeNameIncorrectVariance),
(Pylint, "C0131") => (RuleGroup::Unspecified, rules::pylint::rules::TypeBivariance),
(Pylint, "C0132") => (RuleGroup::Unspecified, rules::pylint::rules::TypeParamNameMismatch),
(Pylint, "C0205") => (RuleGroup::Unspecified, rules::pylint::rules::SingleStringSlots),
(Pylint, "C0414") => (RuleGroup::Unspecified, rules::pylint::rules::UselessImportAlias),
(Pylint, "C1901") => (RuleGroup::Nursery, rules::pylint::rules::CompareToEmptyString),
@@ -194,6 +209,7 @@ pub fn code_to_rule(linter: Linter, code: &str) -> Option<(RuleGroup, Rule)> {
(Pylint, "R0915") => (RuleGroup::Unspecified, rules::pylint::rules::TooManyStatements),
(Pylint, "R1701") => (RuleGroup::Unspecified, rules::pylint::rules::RepeatedIsinstanceCalls),
(Pylint, "R1711") => (RuleGroup::Unspecified, rules::pylint::rules::UselessReturn),
(Pylint, "R1714") => (RuleGroup::Unspecified, rules::pylint::rules::RepeatedEqualityComparisonTarget),
(Pylint, "R1722") => (RuleGroup::Unspecified, rules::pylint::rules::SysExitAlias),
(Pylint, "R2004") => (RuleGroup::Unspecified, rules::pylint::rules::MagicValueComparison),
(Pylint, "R5501") => (RuleGroup::Unspecified, rules::pylint::rules::CollapsibleElseIf),
@@ -251,6 +267,7 @@ pub fn code_to_rule(linter: Linter, code: &str) -> Option<(RuleGroup, Rule)> {
(Flake8Bugbear, "031") => (RuleGroup::Unspecified, rules::flake8_bugbear::rules::ReuseOfGroupbyGenerator),
(Flake8Bugbear, "032") => (RuleGroup::Unspecified, rules::flake8_bugbear::rules::UnintentionalTypeAnnotation),
(Flake8Bugbear, "033") => (RuleGroup::Unspecified, rules::flake8_bugbear::rules::DuplicateValue),
(Flake8Bugbear, "034") => (RuleGroup::Unspecified, rules::flake8_bugbear::rules::ReSubPositionalArgs),
(Flake8Bugbear, "904") => (RuleGroup::Unspecified, rules::flake8_bugbear::rules::RaiseWithoutFromInsideExcept),
(Flake8Bugbear, "905") => (RuleGroup::Unspecified, rules::flake8_bugbear::rules::ZipWithoutExplicitStrict),
@@ -375,8 +392,8 @@ pub fn code_to_rule(linter: Linter, code: &str) -> Option<(RuleGroup, Rule)> {
(Flake8Simplify, "401") => (RuleGroup::Unspecified, rules::flake8_simplify::rules::IfElseBlockInsteadOfDictGet),
(Flake8Simplify, "910") => (RuleGroup::Unspecified, rules::flake8_simplify::rules::DictGetWithNoneDefault),
// copyright
(Copyright, "001") => (RuleGroup::Nursery, rules::flake8_copyright::rules::MissingCopyrightNotice),
// flake8-copyright
(Flake8Copyright, "001") => (RuleGroup::Nursery, rules::flake8_copyright::rules::MissingCopyrightNotice),
// pyupgrade
(Pyupgrade, "001") => (RuleGroup::Unspecified, rules::pyupgrade::rules::UselessMetaclassType),
@@ -587,6 +604,7 @@ pub fn code_to_rule(linter: Linter, code: &str) -> Option<(RuleGroup, Rule)> {
(PandasVet, "012") => (RuleGroup::Unspecified, rules::pandas_vet::rules::PandasUseOfDotReadTable),
(PandasVet, "013") => (RuleGroup::Unspecified, rules::pandas_vet::rules::PandasUseOfDotStack),
(PandasVet, "015") => (RuleGroup::Unspecified, rules::pandas_vet::rules::PandasUseOfPdMerge),
(PandasVet, "101") => (RuleGroup::Unspecified, rules::pandas_vet::rules::PandasNuniqueConstantSeriesCheck),
(PandasVet, "901") => (RuleGroup::Unspecified, rules::pandas_vet::rules::PandasDfVariableName),
// flake8-errmsg
@@ -616,10 +634,13 @@ pub fn code_to_rule(linter: Linter, code: &str) -> Option<(RuleGroup, Rule)> {
(Flake8Pyi, "024") => (RuleGroup::Unspecified, rules::flake8_pyi::rules::CollectionsNamedTuple),
(Flake8Pyi, "025") => (RuleGroup::Unspecified, rules::flake8_pyi::rules::UnaliasedCollectionsAbcSetImport),
(Flake8Pyi, "029") => (RuleGroup::Unspecified, rules::flake8_pyi::rules::StrOrReprDefinedInStub),
(Flake8Pyi, "030") => (RuleGroup::Unspecified, rules::flake8_pyi::rules::UnnecessaryLiteralUnion),
(Flake8Pyi, "032") => (RuleGroup::Unspecified, rules::flake8_pyi::rules::AnyEqNeAnnotation),
(Flake8Pyi, "033") => (RuleGroup::Unspecified, rules::flake8_pyi::rules::TypeCommentInStub),
(Flake8Pyi, "034") => (RuleGroup::Unspecified, rules::flake8_pyi::rules::NonSelfReturnType),
(Flake8Pyi, "035") => (RuleGroup::Unspecified, rules::flake8_pyi::rules::UnassignedSpecialVariableInStub),
(Flake8Pyi, "036") => (RuleGroup::Unspecified, rules::flake8_pyi::rules::BadExitAnnotation),
(Flake8Pyi, "041") => (RuleGroup::Unspecified, rules::flake8_pyi::rules::RedundantNumericUnion),
(Flake8Pyi, "042") => (RuleGroup::Unspecified, rules::flake8_pyi::rules::SnakeCaseTypeAlias),
(Flake8Pyi, "043") => (RuleGroup::Unspecified, rules::flake8_pyi::rules::TSuffixedTypeAlias),
(Flake8Pyi, "044") => (RuleGroup::Unspecified, rules::flake8_pyi::rules::FutureAnnotationsInStub),
@@ -763,6 +784,8 @@ pub fn code_to_rule(linter: Linter, code: &str) -> Option<(RuleGroup, Rule)> {
(Ruff, "013") => (RuleGroup::Unspecified, rules::ruff::rules::ImplicitOptional),
#[cfg(feature = "unreachable-code")]
(Ruff, "014") => (RuleGroup::Nursery, rules::ruff::rules::UnreachableCode),
(Ruff, "015") => (RuleGroup::Unspecified, rules::ruff::rules::UnnecessaryIterableAllocationForFirstElement),
(Ruff, "016") => (RuleGroup::Unspecified, rules::ruff::rules::InvalidIndexType),
(Ruff, "100") => (RuleGroup::Unspecified, rules::ruff::rules::UnusedNOQA),
(Ruff, "200") => (RuleGroup::Unspecified, rules::ruff::rules::InvalidPyprojectToml),

View File

@@ -0,0 +1 @@
pub(crate) mod shebang;

View File

@@ -0,0 +1,67 @@
use ruff_python_whitespace::{is_python_whitespace, Cursor};
use ruff_text_size::{TextLen, TextSize};
/// A shebang directive (e.g., `#!/usr/bin/env python3`).
#[derive(Debug, PartialEq, Eq)]
pub(crate) struct ShebangDirective<'a> {
/// The offset of the directive contents (e.g., `/usr/bin/env python3`) from the start of the
/// line.
pub(crate) offset: TextSize,
/// The contents of the directive (e.g., `"/usr/bin/env python3"`).
pub(crate) contents: &'a str,
}
impl<'a> ShebangDirective<'a> {
/// Parse a shebang directive from a line, or return `None` if the line does not contain a
/// shebang directive.
pub(crate) fn try_extract(line: &'a str) -> Option<Self> {
let mut cursor = Cursor::new(line);
// Trim whitespace.
cursor.eat_while(is_python_whitespace);
// Trim the `#!` prefix.
if !cursor.eat_char('#') {
return None;
}
if !cursor.eat_char('!') {
return None;
}
Some(Self {
offset: line.text_len() - cursor.text_len(),
contents: cursor.chars().as_str(),
})
}
}
#[cfg(test)]
mod tests {
use insta::assert_debug_snapshot;
use super::ShebangDirective;
#[test]
fn shebang_non_match() {
let source = "not a match";
assert_debug_snapshot!(ShebangDirective::try_extract(source));
}
#[test]
fn shebang_end_of_line() {
let source = "print('test') #!/usr/bin/python";
assert_debug_snapshot!(ShebangDirective::try_extract(source));
}
#[test]
fn shebang_match() {
let source = "#!/usr/bin/env python";
assert_debug_snapshot!(ShebangDirective::try_extract(source));
}
#[test]
fn shebang_leading_space() {
let source = " #!/usr/bin/env python";
assert_debug_snapshot!(ShebangDirective::try_extract(source));
}
}

View File

@@ -0,0 +1,5 @@
---
source: crates/ruff/src/comments/shebang.rs
expression: "ShebangDirective::try_extract(source)"
---
None

View File

@@ -0,0 +1,10 @@
---
source: crates/ruff/src/comments/shebang.rs
expression: "ShebangDirective::try_extract(source)"
---
Some(
ShebangDirective {
offset: 4,
contents: "/usr/bin/env python",
},
)

View File

@@ -0,0 +1,10 @@
---
source: crates/ruff/src/comments/shebang.rs
expression: "ShebangDirective::try_extract(source)"
---
Some(
ShebangDirective {
offset: 2,
contents: "/usr/bin/env python",
},
)

View File

@@ -0,0 +1,5 @@
---
source: crates/ruff/src/comments/shebang.rs
expression: "ShebangDirective::try_extract(source)"
---
None

View File

@@ -427,22 +427,22 @@ ghi
NoqaMapping::from_iter([TextRange::new(TextSize::from(6), TextSize::from(28))])
);
let contents = r#"x = \
1"#;
let contents = r"x = \
1";
assert_eq!(
noqa_mappings(contents),
NoqaMapping::from_iter([TextRange::new(TextSize::from(0), TextSize::from(6))])
);
let contents = r#"from foo import \
let contents = r"from foo import \
bar as baz, \
qux as quux"#;
qux as quux";
assert_eq!(
noqa_mappings(contents),
NoqaMapping::from_iter([TextRange::new(TextSize::from(0), TextSize::from(36))])
);
let contents = r#"
let contents = r"
# Foo
from foo import \
bar as baz, \
@@ -450,7 +450,7 @@ from foo import \
x = \
1
y = \
2"#;
2";
assert_eq!(
noqa_mappings(contents),
NoqaMapping::from_iter([

View File

@@ -14,6 +14,7 @@ pub const VERSION: &str = env!("CARGO_PKG_VERSION");
mod autofix;
mod checkers;
mod codes;
mod comments;
mod cst;
pub mod directives;
mod doc_lines;
@@ -38,6 +39,7 @@ mod rule_selector;
pub mod rules;
pub mod settings;
pub mod source_kind;
pub mod upstream_categories;
#[cfg(any(test, fuzzing))]
pub mod test;

View File

@@ -51,7 +51,7 @@ mod tests {
#[test]
fn output() {
let mut emitter = AzureEmitter::default();
let mut emitter = AzureEmitter;
let content = capture_emitter_output(&mut emitter, &create_messages());
assert_snapshot!(content);

View File

@@ -66,7 +66,7 @@ mod tests {
#[test]
fn output() {
let mut emitter = GithubEmitter::default();
let mut emitter = GithubEmitter;
let content = capture_emitter_output(&mut emitter, &create_messages());
assert_snapshot!(content);

View File

@@ -108,7 +108,7 @@ mod tests {
#[test]
fn output() {
let mut emitter = JsonEmitter::default();
let mut emitter = JsonEmitter;
let content = capture_emitter_output(&mut emitter, &create_messages());
assert_snapshot!(content);

View File

@@ -24,14 +24,14 @@ impl Emitter for JsonLinesEmitter {
#[cfg(test)]
mod tests {
use crate::message::json_lines::JsonLinesEmitter;
use insta::assert_snapshot;
use crate::message::json_lines::JsonLinesEmitter;
use crate::message::tests::{capture_emitter_output, create_messages};
#[test]
fn output() {
let mut emitter = JsonLinesEmitter::default();
let mut emitter = JsonLinesEmitter;
let content = capture_emitter_output(&mut emitter, &create_messages());
assert_snapshot!(content);

View File

@@ -93,7 +93,7 @@ mod tests {
#[test]
fn output() {
let mut emitter = JunitEmitter::default();
let mut emitter = JunitEmitter;
let content = capture_emitter_output(&mut emitter, &create_messages());
assert_snapshot!(content);

View File

@@ -49,7 +49,7 @@ mod tests {
#[test]
fn output() {
let mut emitter = PylintEmitter::default();
let mut emitter = PylintEmitter;
let content = capture_emitter_output(&mut emitter, &create_messages());
assert_snapshot!(content);

View File

@@ -1,123 +1,188 @@
use std::collections::BTreeMap;
use std::error::Error;
use std::fmt::{Display, Write};
use std::fs;
use std::ops::Add;
use std::path::Path;
use anyhow::Result;
use itertools::Itertools;
use log::warn;
use once_cell::sync::Lazy;
use regex::Regex;
use ruff_text_size::{TextLen, TextRange, TextSize};
use rustpython_parser::ast::Ranged;
use ruff_diagnostics::Diagnostic;
use ruff_python_ast::source_code::Locator;
use ruff_python_whitespace::{LineEnding, PythonWhitespace};
use ruff_python_whitespace::LineEnding;
use crate::codes::NoqaCode;
use crate::registry::{AsRule, Rule, RuleSet};
use crate::rule_redirects::get_redirect_target;
static NOQA_LINE_REGEX: Lazy<Regex> = Lazy::new(|| {
Regex::new(
r"(?P<leading_spaces>\s*)(?P<noqa>(?i:# noqa)(?::\s?(?P<codes>(?:[A-Z]+[0-9]+)(?:[,\s]+[A-Z]+[0-9]+)*))?)(?P<trailing_spaces>\s*)",
)
.unwrap()
});
/// A directive to ignore a set of rules for a given line of Python source code (e.g.,
/// `# noqa: F401, F841`).
#[derive(Debug)]
pub(crate) enum Directive<'a> {
None,
// (leading spaces, noqa_range, trailing_spaces)
All(TextSize, TextRange, TextSize),
// (leading spaces, start_offset, end_offset, codes, trailing_spaces)
Codes(TextSize, TextRange, Vec<&'a str>, TextSize),
/// The `noqa` directive ignores all rules (e.g., `# noqa`).
All(All),
/// The `noqa` directive ignores specific rules (e.g., `# noqa: F401, F841`).
Codes(Codes<'a>),
}
/// Extract the noqa `Directive` from a line of Python source code.
pub(crate) fn extract_noqa_directive<'a>(range: TextRange, locator: &'a Locator) -> Directive<'a> {
let text = &locator.contents()[range];
match NOQA_LINE_REGEX.captures(text) {
Some(caps) => match (
caps.name("leading_spaces"),
caps.name("noqa"),
caps.name("codes"),
caps.name("trailing_spaces"),
) {
(Some(leading_spaces), Some(noqa), Some(codes), Some(trailing_spaces)) => {
let codes = codes
.as_str()
.split(|c: char| c.is_whitespace() || c == ',')
.map(str::trim)
.filter(|code| !code.is_empty())
.collect_vec();
let start = range.start() + TextSize::try_from(noqa.start()).unwrap();
if codes.is_empty() {
#[allow(deprecated)]
let line = locator.compute_line_index(start);
warn!("Expected rule codes on `noqa` directive: \"{line}\"");
}
Directive::Codes(
leading_spaces.as_str().text_len(),
TextRange::at(start, noqa.as_str().text_len()),
codes,
trailing_spaces.as_str().text_len(),
)
impl<'a> Directive<'a> {
/// Extract the noqa `Directive` from a line of Python source code.
pub(crate) fn try_extract(text: &'a str, offset: TextSize) -> Result<Option<Self>, ParseError> {
for (char_index, char) in text.char_indices() {
// Only bother checking for the `noqa` literal if the character is `n` or `N`.
if !matches!(char, 'n' | 'N') {
continue;
}
(Some(leading_spaces), Some(noqa), None, Some(trailing_spaces)) => Directive::All(
leading_spaces.as_str().text_len(),
TextRange::at(
range.start() + TextSize::try_from(noqa.start()).unwrap(),
noqa.as_str().text_len(),
),
trailing_spaces.as_str().text_len(),
),
_ => Directive::None,
},
None => Directive::None,
}
}
enum ParsedExemption<'a> {
None,
All,
Codes(Vec<&'a str>),
}
/// Return a [`ParsedExemption`] for a given comment line.
fn parse_file_exemption(line: &str) -> ParsedExemption {
let line = line.trim_whitespace_start();
if line.starts_with("# flake8: noqa")
|| line.starts_with("# flake8: NOQA")
|| line.starts_with("# flake8: NoQA")
{
return ParsedExemption::All;
}
if let Some(remainder) = line
.strip_prefix("# ruff: noqa")
.or_else(|| line.strip_prefix("# ruff: NOQA"))
.or_else(|| line.strip_prefix("# ruff: NoQA"))
{
if remainder.is_empty() {
return ParsedExemption::All;
} else if let Some(codes) = remainder.strip_prefix(':') {
let codes = codes
.split(|c: char| c.is_whitespace() || c == ',')
.map(str::trim)
.filter(|code| !code.is_empty())
.collect_vec();
if codes.is_empty() {
warn!("Expected rule codes on `noqa` directive: \"{line}\"");
// Determine the start of the `noqa` literal.
if !matches!(
text[char_index..].as_bytes(),
[b'n' | b'N', b'o' | b'O', b'q' | b'Q', b'a' | b'A', ..]
) {
continue;
}
return ParsedExemption::Codes(codes);
let noqa_literal_start = char_index;
let noqa_literal_end = noqa_literal_start + "noqa".len();
// Determine the start of the comment.
let mut comment_start = noqa_literal_start;
// Trim any whitespace between the `#` character and the `noqa` literal.
comment_start = text[..comment_start].trim_end().len();
// The next character has to be the `#` character.
if text[..comment_start]
.chars()
.last()
.map_or(false, |c| c != '#')
{
continue;
}
comment_start -= '#'.len_utf8();
// If the next character is `:`, then it's a list of codes. Otherwise, it's a directive
// to ignore all rules.
return Ok(Some(
if text[noqa_literal_end..]
.chars()
.next()
.map_or(false, |c| c == ':')
{
// E.g., `# noqa: F401, F841`.
let mut codes_start = noqa_literal_end;
// Skip the `:` character.
codes_start += ':'.len_utf8();
// Skip any whitespace between the `:` and the codes.
codes_start += text[codes_start..]
.find(|c: char| !c.is_whitespace())
.unwrap_or(0);
// Extract the comma-separated list of codes.
let mut codes = vec![];
let mut codes_end = codes_start;
let mut leading_space = 0;
while let Some(code) = Self::lex_code(&text[codes_end + leading_space..]) {
codes.push(code);
codes_end += leading_space;
codes_end += code.len();
// Codes can be comma- or whitespace-delimited. Compute the length of the
// delimiter, but only add it in the next iteration, once we find the next
// code.
if let Some(space_between) =
text[codes_end..].find(|c: char| !(c.is_whitespace() || c == ','))
{
leading_space = space_between;
} else {
break;
}
}
// If we didn't identify any codes, warn.
if codes.is_empty() {
return Err(ParseError::MissingCodes);
}
let range = TextRange::new(
TextSize::try_from(comment_start).unwrap(),
TextSize::try_from(codes_end).unwrap(),
);
Self::Codes(Codes {
range: range.add(offset),
codes,
})
} else {
// E.g., `# noqa`.
let range = TextRange::new(
TextSize::try_from(comment_start).unwrap(),
TextSize::try_from(noqa_literal_end).unwrap(),
);
Self::All(All {
range: range.add(offset),
})
},
));
}
warn!("Unexpected suffix on `noqa` directive: \"{line}\"");
Ok(None)
}
ParsedExemption::None
/// Lex an individual rule code (e.g., `F401`).
#[inline]
fn lex_code(line: &str) -> Option<&str> {
// Extract, e.g., the `F` in `F401`.
let prefix = line.chars().take_while(char::is_ascii_uppercase).count();
// Extract, e.g., the `401` in `F401`.
let suffix = line[prefix..]
.chars()
.take_while(char::is_ascii_digit)
.count();
if prefix > 0 && suffix > 0 {
Some(&line[..prefix + suffix])
} else {
None
}
}
}
#[derive(Debug)]
pub(crate) struct All {
range: TextRange,
}
impl Ranged for All {
/// The range of the `noqa` directive.
fn range(&self) -> TextRange {
self.range
}
}
#[derive(Debug)]
pub(crate) struct Codes<'a> {
range: TextRange,
codes: Vec<&'a str>,
}
impl Codes<'_> {
/// The codes that are ignored by the `noqa` directive.
pub(crate) fn codes(&self) -> &[&str] {
&self.codes
}
}
impl Ranged for Codes<'_> {
/// The range of the `noqa` directive.
fn range(&self) -> TextRange {
self.range
}
}
/// Returns `true` if the string list of `codes` includes `code` (or an alias
@@ -138,50 +203,230 @@ pub(crate) fn rule_is_ignored(
) -> bool {
let offset = noqa_line_for.resolve(offset);
let line_range = locator.line_range(offset);
match extract_noqa_directive(line_range, locator) {
Directive::None => false,
Directive::All(..) => true,
Directive::Codes(.., codes, _) => includes(code, &codes),
match Directive::try_extract(locator.slice(line_range), line_range.start()) {
Ok(Some(Directive::All(_))) => true,
Ok(Some(Directive::Codes(Codes { codes, range: _ }))) => includes(code, &codes),
_ => false,
}
}
/// The file-level exemptions extracted from a given Python file.
#[derive(Debug)]
pub(crate) enum FileExemption {
None,
/// The file is exempt from all rules.
All,
/// The file is exempt from the given rules.
Codes(Vec<NoqaCode>),
}
/// Extract the [`FileExemption`] for a given Python source file, enumerating any rules that are
/// globally ignored within the file.
pub(crate) fn file_exemption(contents: &str, comment_ranges: &[TextRange]) -> FileExemption {
let mut exempt_codes: Vec<NoqaCode> = vec![];
impl FileExemption {
/// Extract the [`FileExemption`] for a given Python source file, enumerating any rules that are
/// globally ignored within the file.
pub(crate) fn try_extract(
contents: &str,
comment_ranges: &[TextRange],
locator: &Locator,
) -> Option<Self> {
let mut exempt_codes: Vec<NoqaCode> = vec![];
for range in comment_ranges {
match parse_file_exemption(&contents[*range]) {
ParsedExemption::All => {
return FileExemption::All;
for range in comment_ranges {
match ParsedFileExemption::try_extract(&contents[*range]) {
Err(err) => {
#[allow(deprecated)]
let line = locator.compute_line_index(range.start());
warn!("Invalid `# noqa` directive on line {line}: {err}");
}
Ok(Some(ParsedFileExemption::All)) => {
return Some(Self::All);
}
Ok(Some(ParsedFileExemption::Codes(codes))) => {
exempt_codes.extend(codes.into_iter().filter_map(|code| {
if let Ok(rule) = Rule::from_code(get_redirect_target(code).unwrap_or(code))
{
Some(rule.noqa_code())
} else {
#[allow(deprecated)]
let line = locator.compute_line_index(range.start());
warn!("Invalid code provided to `# ruff: noqa` on line {line}: {code}");
None
}
}));
}
Ok(None) => {}
}
ParsedExemption::Codes(codes) => {
exempt_codes.extend(codes.into_iter().filter_map(|code| {
if let Ok(rule) = Rule::from_code(get_redirect_target(code).unwrap_or(code)) {
Some(rule.noqa_code())
} else {
warn!("Invalid code provided to `# ruff: noqa`: {}", code);
None
}
}));
}
if exempt_codes.is_empty() {
None
} else {
Some(Self::Codes(exempt_codes))
}
}
}
/// An individual file-level exemption (e.g., `# ruff: noqa` or `# ruff: noqa: F401, F841`). Like
/// [`FileExemption`], but only for a single line, as opposed to an aggregated set of exemptions
/// across a source file.
#[derive(Debug)]
enum ParsedFileExemption<'a> {
/// The file-level exemption ignores all rules (e.g., `# ruff: noqa`).
All,
/// The file-level exemption ignores specific rules (e.g., `# ruff: noqa: F401, F841`).
Codes(Vec<&'a str>),
}
impl<'a> ParsedFileExemption<'a> {
/// Return a [`ParsedFileExemption`] for a given comment line.
fn try_extract(line: &'a str) -> Result<Option<Self>, ParseError> {
let line = Self::lex_whitespace(line);
let Some(line) = Self::lex_char(line, '#') else {
return Ok(None);
};
let line = Self::lex_whitespace(line);
let Some(line) = Self::lex_flake8(line).or_else(|| Self::lex_ruff(line)) else {
return Ok(None);
};
let line = Self::lex_whitespace(line);
let Some(line) = Self::lex_char(line, ':') else {
return Ok(None);
};
let line = Self::lex_whitespace(line);
let Some(line) = Self::lex_noqa(line) else {
return Ok(None);
};
let line = Self::lex_whitespace(line);
Ok(Some(if line.is_empty() {
// Ex) `# ruff: noqa`
Self::All
} else {
// Ex) `# ruff: noqa: F401, F841`
let Some(line) = Self::lex_char(line, ':') else {
return Err(ParseError::InvalidSuffix);
};
let line = Self::lex_whitespace(line);
// Extract the codes from the line (e.g., `F401, F841`).
let mut codes = vec![];
let mut line = line;
while let Some(code) = Self::lex_code(line) {
codes.push(code);
line = &line[code.len()..];
// Codes can be comma- or whitespace-delimited.
if let Some(rest) = Self::lex_delimiter(line).map(Self::lex_whitespace) {
line = rest;
} else {
break;
}
}
ParsedExemption::None => {}
// If we didn't identify any codes, warn.
if codes.is_empty() {
return Err(ParseError::MissingCodes);
}
Self::Codes(codes)
}))
}
/// Lex optional leading whitespace.
#[inline]
fn lex_whitespace(line: &str) -> &str {
line.trim_start()
}
/// Lex a specific character, or return `None` if the character is not the first character in
/// the line.
#[inline]
fn lex_char(line: &str, c: char) -> Option<&str> {
let mut chars = line.chars();
if chars.next() == Some(c) {
Some(chars.as_str())
} else {
None
}
}
if exempt_codes.is_empty() {
FileExemption::None
} else {
FileExemption::Codes(exempt_codes)
/// Lex the "flake8" prefix of a `noqa` directive.
#[inline]
fn lex_flake8(line: &str) -> Option<&str> {
line.strip_prefix("flake8")
}
/// Lex the "ruff" prefix of a `noqa` directive.
#[inline]
fn lex_ruff(line: &str) -> Option<&str> {
line.strip_prefix("ruff")
}
/// Lex a `noqa` directive with case-insensitive matching.
#[inline]
fn lex_noqa(line: &str) -> Option<&str> {
match line.as_bytes() {
[b'n' | b'N', b'o' | b'O', b'q' | b'Q', b'a' | b'A', ..] => Some(&line["noqa".len()..]),
_ => None,
}
}
/// Lex a code delimiter, which can either be a comma or whitespace.
#[inline]
fn lex_delimiter(line: &str) -> Option<&str> {
let mut chars = line.chars();
if let Some(c) = chars.next() {
if c == ',' || c.is_whitespace() {
Some(chars.as_str())
} else {
None
}
} else {
None
}
}
/// Lex an individual rule code (e.g., `F401`).
#[inline]
fn lex_code(line: &str) -> Option<&str> {
// Extract, e.g., the `F` in `F401`.
let prefix = line.chars().take_while(char::is_ascii_uppercase).count();
// Extract, e.g., the `401` in `F401`.
let suffix = line[prefix..]
.chars()
.take_while(char::is_ascii_digit)
.count();
if prefix > 0 && suffix > 0 {
Some(&line[..prefix + suffix])
} else {
None
}
}
}
/// The result of an [`Importer::get_or_import_symbol`] call.
#[derive(Debug)]
pub(crate) enum ParseError {
/// The `noqa` directive was missing valid codes (e.g., `# noqa: unused-import` instead of `# noqa: F401`).
MissingCodes,
/// The `noqa` directive used an invalid suffix (e.g., `# noqa; F401` instead of `# noqa: F401`).
InvalidSuffix,
}
impl Display for ParseError {
fn fmt(&self, fmt: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
ParseError::MissingCodes => fmt.write_str("expected a comma-separated list of codes (e.g., `# noqa: F401, F841`)."),
ParseError::InvalidSuffix => {
fmt.write_str("expected `:` followed by a comma-separated list of codes (e.g., `# noqa: F401, F841`).")
}
}
}
}
impl Error for ParseError {}
/// Adds noqa comments to suppress all diagnostics of a file.
pub(crate) fn add_noqa(
path: &Path,
@@ -215,23 +460,23 @@ fn add_noqa_inner(
// Whether the file is exempted from all checks.
// Codes that are globally exempted (within the current file).
let exemption = file_exemption(locator.contents(), commented_ranges);
let exemption = FileExemption::try_extract(locator.contents(), commented_ranges, locator);
let directives = NoqaDirectives::from_commented_ranges(commented_ranges, locator);
// Mark any non-ignored diagnostics.
for diagnostic in diagnostics {
match &exemption {
FileExemption::All => {
Some(FileExemption::All) => {
// If the file is exempted, don't add any noqa directives.
continue;
}
FileExemption::Codes(codes) => {
Some(FileExemption::Codes(codes)) => {
// If the diagnostic is ignored by a global exemption, don't add a noqa directive.
if codes.contains(&diagnostic.kind.rule().noqa_code()) {
continue;
}
}
FileExemption::None => {}
None => {}
}
// Is the violation ignored by a `noqa` directive on the parent line?
@@ -240,28 +485,27 @@ fn add_noqa_inner(
directives.find_line_with_directive(noqa_line_for.resolve(parent))
{
match &directive_line.directive {
Directive::All(..) => {
Directive::All(_) => {
continue;
}
Directive::Codes(.., codes, _) => {
Directive::Codes(Codes { codes, range: _ }) => {
if includes(diagnostic.kind.rule(), codes) {
continue;
}
}
Directive::None => {}
}
}
}
let noqa_offset = noqa_line_for.resolve(diagnostic.start());
// Or ignored by the directive itself
// Or ignored by the directive itself?
if let Some(directive_line) = directives.find_line_with_directive(noqa_offset) {
match &directive_line.directive {
Directive::All(..) => {
Directive::All(_) => {
continue;
}
Directive::Codes(.., codes, _) => {
Directive::Codes(Codes { codes, range: _ }) => {
let rule = diagnostic.kind.rule();
if !includes(rule, codes) {
matches_by_line
@@ -274,7 +518,6 @@ fn add_noqa_inner(
}
continue;
}
Directive::None => {}
}
}
@@ -296,7 +539,7 @@ fn add_noqa_inner(
let line = locator.full_line(offset);
match directive {
None | Some(Directive::None) => {
None => {
// Add existing content.
output.push_str(line.trim_end());
@@ -308,10 +551,10 @@ fn add_noqa_inner(
output.push_str(&line_ending);
count += 1;
}
Some(Directive::All(..)) => {
Some(Directive::All(_)) => {
// Does not get inserted into the map.
}
Some(Directive::Codes(_, noqa_range, existing, _)) => {
Some(Directive::Codes(Codes { range, codes })) => {
// Reconstruct the line based on the preserved rule codes.
// This enables us to tally the number of edits.
let output_start = output.len();
@@ -319,7 +562,7 @@ fn add_noqa_inner(
// Add existing content.
output.push_str(
locator
.slice(TextRange::new(offset, noqa_range.start()))
.slice(TextRange::new(offset, range.start()))
.trim_end(),
);
@@ -331,8 +574,8 @@ fn add_noqa_inner(
&mut output,
rules
.iter()
.map(|r| r.noqa_code().to_string())
.chain(existing.iter().map(ToString::to_string))
.map(|rule| rule.noqa_code().to_string())
.chain(codes.iter().map(ToString::to_string))
.sorted_unstable(),
);
@@ -366,9 +609,11 @@ fn push_codes<I: Display>(str: &mut String, codes: impl Iterator<Item = I>) {
#[derive(Debug)]
pub(crate) struct NoqaDirectiveLine<'a> {
// The range of the text line for which the noqa directive applies.
/// The range of the text line for which the noqa directive applies.
pub(crate) range: TextRange,
/// The noqa directive.
pub(crate) directive: Directive<'a>,
/// The codes that are ignored by the directive.
pub(crate) matches: Vec<NoqaCode>,
}
@@ -384,21 +629,23 @@ impl<'a> NoqaDirectives<'a> {
) -> Self {
let mut directives = Vec::new();
for comment_range in comment_ranges {
let line_range = locator.line_range(comment_range.start());
let directive = match extract_noqa_directive(line_range, locator) {
Directive::None => {
continue;
for range in comment_ranges {
match Directive::try_extract(locator.slice(*range), range.start()) {
Err(err) => {
#[allow(deprecated)]
let line = locator.compute_line_index(range.start());
warn!("Invalid `# noqa` directive on line {line}: {err}");
}
directive @ (Directive::All(..) | Directive::Codes(..)) => directive,
};
// noqa comments are guaranteed to be single line.
directives.push(NoqaDirectiveLine {
range: line_range,
directive,
matches: Vec::new(),
});
Ok(Some(directive)) => {
// noqa comments are guaranteed to be single line.
directives.push(NoqaDirectiveLine {
range: locator.line_range(range.start()),
directive,
matches: Vec::new(),
});
}
Ok(None) => {}
}
}
// Extend a mapping at the end of the file to also include the EOF token.
@@ -460,7 +707,7 @@ impl NoqaMapping {
}
/// Returns the re-mapped position or `position` if no mapping exists.
pub fn resolve(&self, offset: TextSize) -> TextSize {
pub(crate) fn resolve(&self, offset: TextSize) -> TextSize {
let index = self.ranges.binary_search_by(|range| {
if range.end() < offset {
std::cmp::Ordering::Less
@@ -478,7 +725,7 @@ impl NoqaMapping {
}
}
pub fn push_mapping(&mut self, range: TextRange) {
pub(crate) fn push_mapping(&mut self, range: TextRange) {
if let Some(last_range) = self.ranges.last_mut() {
// Strictly sorted insertion
if last_range.end() <= range.start() {
@@ -511,28 +758,190 @@ impl FromIterator<TextRange> for NoqaMapping {
#[cfg(test)]
mod tests {
use insta::assert_debug_snapshot;
use ruff_text_size::{TextRange, TextSize};
use ruff_diagnostics::Diagnostic;
use ruff_python_ast::source_code::Locator;
use ruff_python_whitespace::LineEnding;
use crate::noqa::{add_noqa_inner, NoqaMapping, NOQA_LINE_REGEX};
use crate::noqa::{add_noqa_inner, Directive, NoqaMapping, ParsedFileExemption};
use crate::rules::pycodestyle::rules::AmbiguousVariableName;
use crate::rules::pyflakes;
use crate::rules::pyflakes::rules::UnusedVariable;
#[test]
fn regex() {
assert!(NOQA_LINE_REGEX.is_match("# noqa"));
assert!(NOQA_LINE_REGEX.is_match("# NoQA"));
fn noqa_all() {
let source = "# noqa";
assert_debug_snapshot!(Directive::try_extract(source, TextSize::default()));
}
assert!(NOQA_LINE_REGEX.is_match("# noqa: F401"));
assert!(NOQA_LINE_REGEX.is_match("# NoQA: F401"));
assert!(NOQA_LINE_REGEX.is_match("# noqa: F401, E501"));
#[test]
fn noqa_code() {
let source = "# noqa: F401";
assert_debug_snapshot!(Directive::try_extract(source, TextSize::default()));
}
assert!(NOQA_LINE_REGEX.is_match("# noqa:F401"));
assert!(NOQA_LINE_REGEX.is_match("# NoQA:F401"));
assert!(NOQA_LINE_REGEX.is_match("# noqa:F401, E501"));
#[test]
fn noqa_codes() {
let source = "# noqa: F401, F841";
assert_debug_snapshot!(Directive::try_extract(source, TextSize::default()));
}
#[test]
fn noqa_all_case_insensitive() {
let source = "# NOQA";
assert_debug_snapshot!(Directive::try_extract(source, TextSize::default()));
}
#[test]
fn noqa_code_case_insensitive() {
let source = "# NOQA: F401";
assert_debug_snapshot!(Directive::try_extract(source, TextSize::default()));
}
#[test]
fn noqa_codes_case_insensitive() {
let source = "# NOQA: F401, F841";
assert_debug_snapshot!(Directive::try_extract(source, TextSize::default()));
}
#[test]
fn noqa_leading_space() {
let source = "# # noqa: F401";
assert_debug_snapshot!(Directive::try_extract(source, TextSize::default()));
}
#[test]
fn noqa_trailing_space() {
let source = "# noqa: F401 #";
assert_debug_snapshot!(Directive::try_extract(source, TextSize::default()));
}
#[test]
fn noqa_all_no_space() {
let source = "#noqa";
assert_debug_snapshot!(Directive::try_extract(source, TextSize::default()));
}
#[test]
fn noqa_code_no_space() {
let source = "#noqa:F401";
assert_debug_snapshot!(Directive::try_extract(source, TextSize::default()));
}
#[test]
fn noqa_codes_no_space() {
let source = "#noqa:F401,F841";
assert_debug_snapshot!(Directive::try_extract(source, TextSize::default()));
}
#[test]
fn noqa_all_multi_space() {
let source = "# noqa";
assert_debug_snapshot!(Directive::try_extract(source, TextSize::default()));
}
#[test]
fn noqa_code_multi_space() {
let source = "# noqa: F401";
assert_debug_snapshot!(Directive::try_extract(source, TextSize::default()));
}
#[test]
fn noqa_codes_multi_space() {
let source = "# noqa: F401, F841";
assert_debug_snapshot!(Directive::try_extract(source, TextSize::default()));
}
#[test]
fn noqa_all_leading_comment() {
let source = "# Some comment describing the noqa # noqa";
assert_debug_snapshot!(Directive::try_extract(source, TextSize::default()));
}
#[test]
fn noqa_code_leading_comment() {
let source = "# Some comment describing the noqa # noqa: F401";
assert_debug_snapshot!(Directive::try_extract(source, TextSize::default()));
}
#[test]
fn noqa_codes_leading_comment() {
let source = "# Some comment describing the noqa # noqa: F401, F841";
assert_debug_snapshot!(Directive::try_extract(source, TextSize::default()));
}
#[test]
fn noqa_all_trailing_comment() {
let source = "# noqa # Some comment describing the noqa";
assert_debug_snapshot!(Directive::try_extract(source, TextSize::default()));
}
#[test]
fn noqa_code_trailing_comment() {
let source = "# noqa: F401 # Some comment describing the noqa";
assert_debug_snapshot!(Directive::try_extract(source, TextSize::default()));
}
#[test]
fn noqa_codes_trailing_comment() {
let source = "# noqa: F401, F841 # Some comment describing the noqa";
assert_debug_snapshot!(Directive::try_extract(source, TextSize::default()));
}
#[test]
fn noqa_invalid_codes() {
let source = "# noqa: unused-import, F401, some other code";
assert_debug_snapshot!(Directive::try_extract(source, TextSize::default()));
}
#[test]
fn flake8_exemption_all() {
let source = "# flake8: noqa";
assert_debug_snapshot!(ParsedFileExemption::try_extract(source));
}
#[test]
fn ruff_exemption_all() {
let source = "# ruff: noqa";
assert_debug_snapshot!(ParsedFileExemption::try_extract(source));
}
#[test]
fn flake8_exemption_all_no_space() {
let source = "#flake8:noqa";
assert_debug_snapshot!(ParsedFileExemption::try_extract(source));
}
#[test]
fn ruff_exemption_all_no_space() {
let source = "#ruff:noqa";
assert_debug_snapshot!(ParsedFileExemption::try_extract(source));
}
#[test]
fn flake8_exemption_codes() {
// Note: Flake8 doesn't support this; it's treated as a blanket exemption.
let source = "# flake8: noqa: F401, F841";
assert_debug_snapshot!(ParsedFileExemption::try_extract(source));
}
#[test]
fn ruff_exemption_codes() {
let source = "# ruff: noqa: F401, F841";
assert_debug_snapshot!(ParsedFileExemption::try_extract(source));
}
#[test]
fn flake8_exemption_all_case_insensitive() {
let source = "# flake8: NoQa";
assert_debug_snapshot!(ParsedFileExemption::try_extract(source));
}
#[test]
fn ruff_exemption_all_case_insensitive() {
let source = "# ruff: NoQa";
assert_debug_snapshot!(ParsedFileExemption::try_extract(source));
}
#[test]
@@ -550,7 +959,7 @@ mod tests {
assert_eq!(output, format!("{contents}"));
let diagnostics = [Diagnostic::new(
pyflakes::rules::UnusedVariable {
UnusedVariable {
name: "x".to_string(),
},
TextRange::new(TextSize::from(0), TextSize::from(0)),
@@ -574,7 +983,7 @@ mod tests {
TextRange::new(TextSize::from(0), TextSize::from(0)),
),
Diagnostic::new(
pyflakes::rules::UnusedVariable {
UnusedVariable {
name: "x".to_string(),
},
TextRange::new(TextSize::from(0), TextSize::from(0)),
@@ -598,7 +1007,7 @@ mod tests {
TextRange::new(TextSize::from(0), TextSize::from(0)),
),
Diagnostic::new(
pyflakes::rules::UnusedVariable {
UnusedVariable {
name: "x".to_string(),
},
TextRange::new(TextSize::from(0), TextSize::from(0)),

View File

@@ -7,7 +7,9 @@ use ruff_diagnostics::Diagnostic;
use ruff_python_ast::source_code::SourceFile;
use crate::message::Message;
use crate::registry::Rule;
use crate::rules::ruff::rules::InvalidPyprojectToml;
use crate::settings::Settings;
use crate::IOError;
/// Unlike [`pyproject_toml::PyProjectToml`], in our case `build_system` is also optional
@@ -20,9 +22,11 @@ struct PyProjectToml {
project: Option<Project>,
}
pub fn lint_pyproject_toml(source_file: SourceFile) -> Result<Vec<Message>> {
pub fn lint_pyproject_toml(source_file: SourceFile, settings: &Settings) -> Result<Vec<Message>> {
let mut messages = vec![];
let err = match toml::from_str::<PyProjectToml>(source_file.source_text()) {
Ok(_) => return Ok(Vec::default()),
Ok(_) => return Ok(messages),
Err(err) => err,
};
@@ -32,17 +36,20 @@ pub fn lint_pyproject_toml(source_file: SourceFile) -> Result<Vec<Message>> {
None => TextRange::default(),
Some(range) => {
let Ok(end) = TextSize::try_from(range.end) else {
let diagnostic = Diagnostic::new(
IOError {
message: "pyproject.toml is larger than 4GB".to_string(),
},
TextRange::default(),
);
return Ok(vec![Message::from_diagnostic(
diagnostic,
source_file,
TextSize::default(),
)]);
if settings.rules.enabled(Rule::IOError) {
let diagnostic = Diagnostic::new(
IOError {
message: "pyproject.toml is larger than 4GB".to_string(),
},
TextRange::default(),
);
messages.push(Message::from_diagnostic(
diagnostic,
source_file,
TextSize::default(),
));
}
return Ok(messages);
};
TextRange::new(
// start <= end, so if end < 4GB follows start < 4GB
@@ -52,11 +59,15 @@ pub fn lint_pyproject_toml(source_file: SourceFile) -> Result<Vec<Message>> {
}
};
let toml_err = err.message().to_string();
let diagnostic = Diagnostic::new(InvalidPyprojectToml { message: toml_err }, range);
Ok(vec![Message::from_diagnostic(
diagnostic,
source_file,
TextSize::default(),
)])
if settings.rules.enabled(Rule::InvalidPyprojectToml) {
let toml_err = err.message().to_string();
let diagnostic = Diagnostic::new(InvalidPyprojectToml { message: toml_err }, range);
messages.push(Message::from_diagnostic(
diagnostic,
source_file,
TextSize::default(),
));
}
Ok(messages)
}

View File

@@ -7,7 +7,7 @@ pub use codes::Rule;
use ruff_macros::RuleNamespace;
pub use rule_set::{RuleSet, RuleSetIterator};
use crate::codes::{self, RuleCodePrefix};
use crate::codes::{self};
mod rule_set;
@@ -18,8 +18,10 @@ pub trait AsRule {
impl Rule {
pub fn from_code(code: &str) -> Result<Self, FromCodeError> {
let (linter, code) = Linter::parse_code(code).ok_or(FromCodeError::Unknown)?;
let prefix: RuleCodePrefix = RuleCodePrefix::parse(&linter, code)?;
Ok(prefix.rules().next().unwrap())
linter
.all_rules()
.find(|rule| rule.noqa_code().suffix() == code)
.ok_or(FromCodeError::Unknown)
}
}
@@ -80,9 +82,9 @@ pub enum Linter {
/// [flake8-commas](https://pypi.org/project/flake8-commas/)
#[prefix = "COM"]
Flake8Commas,
/// Copyright-related rules
/// [flake8-copyright](https://pypi.org/project/flake8-copyright/)
#[prefix = "CPY"]
Copyright,
Flake8Copyright,
/// [flake8-comprehensions](https://pypi.org/project/flake8-comprehensions/)
#[prefix = "C4"]
Flake8Comprehensions,
@@ -110,7 +112,7 @@ pub enum Linter {
/// [flake8-import-conventions](https://github.com/joaopalmeiro/flake8-import-conventions)
#[prefix = "ICN"]
Flake8ImportConventions,
/// [flake8-logging-format](https://pypi.org/project/flake8-logging-format/0.9.0/)
/// [flake8-logging-format](https://pypi.org/project/flake8-logging-format/)
#[prefix = "G"]
Flake8LoggingFormat,
/// [flake8-no-pep420](https://pypi.org/project/flake8-no-pep420/)
@@ -179,7 +181,7 @@ pub enum Linter {
/// [Pylint](https://pypi.org/project/pylint/)
#[prefix = "PL"]
Pylint,
/// [tryceratops](https://pypi.org/project/tryceratops/1.1.0/)
/// [tryceratops](https://pypi.org/project/tryceratops/)
#[prefix = "TRY"]
Tryceratops,
/// [flynt](https://pypi.org/project/flynt/)
@@ -216,30 +218,6 @@ pub trait RuleNamespace: Sized {
fn url(&self) -> Option<&'static str>;
}
/// The prefix and name for an upstream linter category.
pub struct UpstreamCategory(pub RuleCodePrefix, pub &'static str);
impl Linter {
pub const fn upstream_categories(&self) -> Option<&'static [UpstreamCategory]> {
match self {
Linter::Pycodestyle => Some(&[
UpstreamCategory(RuleCodePrefix::Pycodestyle(codes::Pycodestyle::E), "Error"),
UpstreamCategory(
RuleCodePrefix::Pycodestyle(codes::Pycodestyle::W),
"Warning",
),
]),
Linter::Pylint => Some(&[
UpstreamCategory(RuleCodePrefix::Pylint(codes::Pylint::C), "Convention"),
UpstreamCategory(RuleCodePrefix::Pylint(codes::Pylint::E), "Error"),
UpstreamCategory(RuleCodePrefix::Pylint(codes::Pylint::R), "Refactor"),
UpstreamCategory(RuleCodePrefix::Pylint(codes::Pylint::W), "Warning"),
]),
_ => None,
}
}
}
#[derive(is_macro::Is, Copy, Clone)]
pub enum LintSource {
Ast,
@@ -250,6 +228,7 @@ pub enum LintSource {
Imports,
Noqa,
Filesystem,
PyprojectToml,
}
impl Rule {
@@ -257,6 +236,7 @@ impl Rule {
/// physical lines).
pub const fn lint_source(&self) -> LintSource {
match self {
Rule::InvalidPyprojectToml => LintSource::PyprojectToml,
Rule::UnusedNOQA => LintSource::Noqa,
Rule::BlanketNOQA
| Rule::BlanketTypeIgnore

View File

@@ -248,8 +248,8 @@ impl Renamer {
| BindingKind::LoopVar
| BindingKind::Global
| BindingKind::Nonlocal(_)
| BindingKind::ClassDefinition
| BindingKind::FunctionDefinition
| BindingKind::ClassDefinition(_)
| BindingKind::FunctionDefinition(_)
| BindingKind::Deletion
| BindingKind::UnboundException(_) => {
Some(Edit::range_replacement(target.to_string(), binding.range))

View File

@@ -51,11 +51,9 @@ pub(crate) fn variable_name_task_id(
value: &Expr,
) -> Option<Diagnostic> {
// If we have more than one target, we can't do anything.
if targets.len() != 1 {
let [target] = targets else {
return None;
}
let target = &targets[0];
};
let Expr::Name(ast::ExprName { id, .. }) = target else {
return None;
};

View File

@@ -48,12 +48,11 @@ fn is_standalone_comment(line: &str) -> bool {
/// ERA001
pub(crate) fn commented_out_code(
diagnostics: &mut Vec<Diagnostic>,
locator: &Locator,
indexer: &Indexer,
settings: &Settings,
) -> Vec<Diagnostic> {
let mut diagnostics = vec![];
) {
for range in indexer.comment_ranges() {
let line = locator.full_lines(*range);
@@ -69,6 +68,4 @@ pub(crate) fn commented_out_code(
diagnostics.push(diagnostic);
}
}
diagnostics
}

View File

@@ -1,4 +1,4 @@
use rustpython_parser::ast::{ArgWithDefault, Expr, Ranged, Stmt};
use rustpython_parser::ast::{self, ArgWithDefault, Constant, Expr, Ranged, Stmt};
use ruff_diagnostics::{AlwaysAutofixableViolation, Diagnostic, Fix, Violation};
use ruff_macros::{derive_message_formats, violation};
@@ -6,12 +6,14 @@ use ruff_python_ast::cast;
use ruff_python_ast::helpers::ReturnStatementVisitor;
use ruff_python_ast::identifier::Identifier;
use ruff_python_ast::statement_visitor::StatementVisitor;
use ruff_python_ast::typing::parse_type_annotation;
use ruff_python_semantic::analyze::visibility;
use ruff_python_semantic::{Definition, Member, MemberKind, SemanticModel};
use ruff_python_semantic::{Definition, Member, MemberKind};
use ruff_python_stdlib::typing::simple_magic_return_type;
use crate::checkers::ast::Checker;
use crate::registry::{AsRule, Rule};
use crate::rules::ruff::typing::type_hint_resolves_to_any;
use super::super::fixes;
use super::super::helpers::match_function_def;
@@ -432,20 +434,46 @@ fn is_none_returning(body: &[Stmt]) -> bool {
/// ANN401
fn check_dynamically_typed<F>(
checker: &Checker,
annotation: &Expr,
func: F,
diagnostics: &mut Vec<Diagnostic>,
is_overridden: bool,
semantic: &SemanticModel,
) where
F: FnOnce() -> String,
{
if !is_overridden && semantic.match_typing_expr(annotation, "Any") {
diagnostics.push(Diagnostic::new(
AnyType { name: func() },
annotation.range(),
));
};
if let Expr::Constant(ast::ExprConstant {
range,
value: Constant::Str(string),
..
}) = annotation
{
// Quoted annotations
if let Ok((parsed_annotation, _)) = parse_type_annotation(string, *range, checker.locator) {
if type_hint_resolves_to_any(
&parsed_annotation,
checker.semantic(),
checker.locator,
checker.settings.target_version.minor(),
) {
diagnostics.push(Diagnostic::new(
AnyType { name: func() },
annotation.range(),
));
}
}
} else {
if type_hint_resolves_to_any(
annotation,
checker.semantic(),
checker.locator,
checker.settings.target_version.minor(),
) {
diagnostics.push(Diagnostic::new(
AnyType { name: func() },
annotation.range(),
));
}
}
}
/// Generate flake8-annotation checks for a given `Definition`.
@@ -500,13 +528,12 @@ pub(crate) fn definition(
// ANN401 for dynamically typed arguments
if let Some(annotation) = &def.annotation {
has_any_typed_arg = true;
if checker.enabled(Rule::AnyType) {
if checker.enabled(Rule::AnyType) && !is_overridden {
check_dynamically_typed(
checker,
annotation,
|| def.arg.to_string(),
&mut diagnostics,
is_overridden,
checker.semantic(),
);
}
} else {
@@ -530,15 +557,9 @@ pub(crate) fn definition(
if let Some(expr) = &arg.annotation {
has_any_typed_arg = true;
if !checker.settings.flake8_annotations.allow_star_arg_any {
if checker.enabled(Rule::AnyType) {
if checker.enabled(Rule::AnyType) && !is_overridden {
let name = &arg.arg;
check_dynamically_typed(
expr,
|| format!("*{name}"),
&mut diagnostics,
is_overridden,
checker.semantic(),
);
check_dynamically_typed(checker, expr, || format!("*{name}"), &mut diagnostics);
}
}
} else {
@@ -562,14 +583,13 @@ pub(crate) fn definition(
if let Some(expr) = &arg.annotation {
has_any_typed_arg = true;
if !checker.settings.flake8_annotations.allow_star_arg_any {
if checker.enabled(Rule::AnyType) {
if checker.enabled(Rule::AnyType) && !is_overridden {
let name = &arg.arg;
check_dynamically_typed(
checker,
expr,
|| format!("**{name}"),
&mut diagnostics,
is_overridden,
checker.semantic(),
);
}
}
@@ -629,14 +649,8 @@ pub(crate) fn definition(
// ANN201, ANN202, ANN401
if let Some(expr) = &returns {
has_typed_return = true;
if checker.enabled(Rule::AnyType) {
check_dynamically_typed(
expr,
|| name.to_string(),
&mut diagnostics,
is_overridden,
checker.semantic(),
);
if checker.enabled(Rule::AnyType) && !is_overridden {
check_dynamically_typed(checker, expr, || name.to_string(), &mut diagnostics);
}
} else if !(
// Allow omission of return annotation if the function only returns `None`

View File

@@ -186,4 +186,60 @@ annotation_presence.py:134:13: ANN101 Missing type annotation for `self` in meth
135 | pass
|
annotation_presence.py:149:10: ANN401 Dynamically typed expressions (typing.Any) are disallowed in `a`
|
148 | # ANN401
149 | def f(a: Any | int) -> None: ...
| ^^^^^^^^^ ANN401
150 | def f(a: int | Any) -> None: ...
151 | def f(a: Union[str, bytes, Any]) -> None: ...
|
annotation_presence.py:150:10: ANN401 Dynamically typed expressions (typing.Any) are disallowed in `a`
|
148 | # ANN401
149 | def f(a: Any | int) -> None: ...
150 | def f(a: int | Any) -> None: ...
| ^^^^^^^^^ ANN401
151 | def f(a: Union[str, bytes, Any]) -> None: ...
152 | def f(a: Optional[Any]) -> None: ...
|
annotation_presence.py:151:10: ANN401 Dynamically typed expressions (typing.Any) are disallowed in `a`
|
149 | def f(a: Any | int) -> None: ...
150 | def f(a: int | Any) -> None: ...
151 | def f(a: Union[str, bytes, Any]) -> None: ...
| ^^^^^^^^^^^^^^^^^^^^^^ ANN401
152 | def f(a: Optional[Any]) -> None: ...
153 | def f(a: Annotated[Any, ...]) -> None: ...
|
annotation_presence.py:152:10: ANN401 Dynamically typed expressions (typing.Any) are disallowed in `a`
|
150 | def f(a: int | Any) -> None: ...
151 | def f(a: Union[str, bytes, Any]) -> None: ...
152 | def f(a: Optional[Any]) -> None: ...
| ^^^^^^^^^^^^^ ANN401
153 | def f(a: Annotated[Any, ...]) -> None: ...
154 | def f(a: "Union[str, bytes, Any]") -> None: ...
|
annotation_presence.py:153:10: ANN401 Dynamically typed expressions (typing.Any) are disallowed in `a`
|
151 | def f(a: Union[str, bytes, Any]) -> None: ...
152 | def f(a: Optional[Any]) -> None: ...
153 | def f(a: Annotated[Any, ...]) -> None: ...
| ^^^^^^^^^^^^^^^^^^^ ANN401
154 | def f(a: "Union[str, bytes, Any]") -> None: ...
|
annotation_presence.py:154:10: ANN401 Dynamically typed expressions (typing.Any) are disallowed in `a`
|
152 | def f(a: Optional[Any]) -> None: ...
153 | def f(a: Annotated[Any, ...]) -> None: ...
154 | def f(a: "Union[str, bytes, Any]") -> None: ...
| ^^^^^^^^^^^^^^^^^^^^^^^^ ANN401
|

View File

@@ -1,15 +1,39 @@
use num_traits::ToPrimitive;
use once_cell::sync::Lazy;
use rustc_hash::FxHashMap;
use rustpython_parser::ast::{self, Constant, Expr, Keyword, Operator, Ranged};
use ruff_diagnostics::{Diagnostic, Violation};
use ruff_macros::{derive_message_formats, violation};
use ruff_python_ast::call_path::compose_call_path;
use ruff_python_ast::call_path::CallPath;
use ruff_python_ast::helpers::SimpleCallArgs;
use ruff_python_semantic::SemanticModel;
use crate::checkers::ast::Checker;
/// ## What it does
/// Checks for files with overly permissive permissions.
///
/// ## Why is this bad?
/// Overly permissive file permissions may allow unintended access and
/// arbitrary code execution.
///
/// ## Example
/// ```python
/// import os
///
/// os.chmod("/etc/secrets.txt", 0o666) # rw-rw-rw-
/// ```
///
/// Use instead:
/// ```python
/// import os
///
/// os.chmod("/etc/secrets.txt", 0o600) # rw-------
/// ```
///
/// ## References
/// - [Python documentation: `os.chmod`](https://docs.python.org/3/library/os.html#os.chmod)
/// - [Python documentation: `stat`](https://docs.python.org/3/library/stat.html)
/// - [Common Weakness Enumeration: CWE-732](https://cwe.mitre.org/data/definitions/732.html)
#[violation]
pub struct BadFilePermissions {
mask: u16,
@@ -19,84 +43,7 @@ impl Violation for BadFilePermissions {
#[derive_message_formats]
fn message(&self) -> String {
let BadFilePermissions { mask } = self;
format!("`os.chmod` setting a permissive mask `{mask:#o}` on file or directory",)
}
}
const WRITE_WORLD: u16 = 0o2;
const EXECUTE_GROUP: u16 = 0o10;
static PYSTAT_MAPPING: Lazy<FxHashMap<&'static str, u16>> = Lazy::new(|| {
FxHashMap::from_iter([
("stat.ST_MODE", 0o0),
("stat.S_IFDOOR", 0o0),
("stat.S_IFPORT", 0o0),
("stat.ST_INO", 0o1),
("stat.S_IXOTH", 0o1),
("stat.UF_NODUMP", 0o1),
("stat.ST_DEV", 0o2),
("stat.S_IWOTH", 0o2),
("stat.UF_IMMUTABLE", 0o2),
("stat.ST_NLINK", 0o3),
("stat.ST_UID", 0o4),
("stat.S_IROTH", 0o4),
("stat.UF_APPEND", 0o4),
("stat.ST_GID", 0o5),
("stat.ST_SIZE", 0o6),
("stat.ST_ATIME", 0o7),
("stat.S_IRWXO", 0o7),
("stat.ST_MTIME", 0o10),
("stat.S_IXGRP", 0o10),
("stat.UF_OPAQUE", 0o10),
("stat.ST_CTIME", 0o11),
("stat.S_IWGRP", 0o20),
("stat.UF_NOUNLINK", 0o20),
("stat.S_IRGRP", 0o40),
("stat.UF_COMPRESSED", 0o40),
("stat.S_IRWXG", 0o70),
("stat.S_IEXEC", 0o100),
("stat.S_IXUSR", 0o100),
("stat.S_IWRITE", 0o200),
("stat.S_IWUSR", 0o200),
("stat.S_IREAD", 0o400),
("stat.S_IRUSR", 0o400),
("stat.S_IRWXU", 0o700),
("stat.S_ISVTX", 0o1000),
("stat.S_ISGID", 0o2000),
("stat.S_ENFMT", 0o2000),
("stat.S_ISUID", 0o4000),
])
});
fn get_int_value(expr: &Expr) -> Option<u16> {
match expr {
Expr::Constant(ast::ExprConstant {
value: Constant::Int(value),
..
}) => value.to_u16(),
Expr::Attribute(_) => {
compose_call_path(expr).and_then(|path| PYSTAT_MAPPING.get(path.as_str()).copied())
}
Expr::BinOp(ast::ExprBinOp {
left,
op,
right,
range: _,
}) => {
if let (Some(left_value), Some(right_value)) =
(get_int_value(left), get_int_value(right))
{
match op {
Operator::BitAnd => Some(left_value & right_value),
Operator::BitOr => Some(left_value | right_value),
Operator::BitXor => Some(left_value ^ right_value),
_ => None,
}
} else {
None
}
}
_ => None,
format!("`os.chmod` setting a permissive mask `{mask:#o}` on file or directory")
}
}
@@ -116,7 +63,7 @@ pub(crate) fn bad_file_permissions(
{
let call_args = SimpleCallArgs::new(args, keywords);
if let Some(mode_arg) = call_args.argument("mode", 1) {
if let Some(int_value) = get_int_value(mode_arg) {
if let Some(int_value) = int_value(mode_arg, checker.semantic()) {
if (int_value & WRITE_WORLD > 0) || (int_value & EXECUTE_GROUP > 0) {
checker.diagnostics.push(Diagnostic::new(
BadFilePermissions { mask: int_value },
@@ -127,3 +74,75 @@ pub(crate) fn bad_file_permissions(
}
}
}
const WRITE_WORLD: u16 = 0o2;
const EXECUTE_GROUP: u16 = 0o10;
fn py_stat(call_path: &CallPath) -> Option<u16> {
match call_path.as_slice() {
["stat", "ST_MODE"] => Some(0o0),
["stat", "S_IFDOOR"] => Some(0o0),
["stat", "S_IFPORT"] => Some(0o0),
["stat", "ST_INO"] => Some(0o1),
["stat", "S_IXOTH"] => Some(0o1),
["stat", "UF_NODUMP"] => Some(0o1),
["stat", "ST_DEV"] => Some(0o2),
["stat", "S_IWOTH"] => Some(0o2),
["stat", "UF_IMMUTABLE"] => Some(0o2),
["stat", "ST_NLINK"] => Some(0o3),
["stat", "ST_UID"] => Some(0o4),
["stat", "S_IROTH"] => Some(0o4),
["stat", "UF_APPEND"] => Some(0o4),
["stat", "ST_GID"] => Some(0o5),
["stat", "ST_SIZE"] => Some(0o6),
["stat", "ST_ATIME"] => Some(0o7),
["stat", "S_IRWXO"] => Some(0o7),
["stat", "ST_MTIME"] => Some(0o10),
["stat", "S_IXGRP"] => Some(0o10),
["stat", "UF_OPAQUE"] => Some(0o10),
["stat", "ST_CTIME"] => Some(0o11),
["stat", "S_IWGRP"] => Some(0o20),
["stat", "UF_NOUNLINK"] => Some(0o20),
["stat", "S_IRGRP"] => Some(0o40),
["stat", "UF_COMPRESSED"] => Some(0o40),
["stat", "S_IRWXG"] => Some(0o70),
["stat", "S_IEXEC"] => Some(0o100),
["stat", "S_IXUSR"] => Some(0o100),
["stat", "S_IWRITE"] => Some(0o200),
["stat", "S_IWUSR"] => Some(0o200),
["stat", "S_IREAD"] => Some(0o400),
["stat", "S_IRUSR"] => Some(0o400),
["stat", "S_IRWXU"] => Some(0o700),
["stat", "S_ISVTX"] => Some(0o1000),
["stat", "S_ISGID"] => Some(0o2000),
["stat", "S_ENFMT"] => Some(0o2000),
["stat", "S_ISUID"] => Some(0o4000),
_ => None,
}
}
fn int_value(expr: &Expr, model: &SemanticModel) -> Option<u16> {
match expr {
Expr::Constant(ast::ExprConstant {
value: Constant::Int(value),
..
}) => value.to_u16(),
Expr::Attribute(_) => model.resolve_call_path(expr).as_ref().and_then(py_stat),
Expr::BinOp(ast::ExprBinOp {
left,
op,
right,
range: _,
}) => {
let left_value = int_value(left, model)?;
let right_value = int_value(right, model)?;
match op {
Operator::BitAnd => Some(left_value & right_value),
Operator::BitOr => Some(left_value | right_value),
Operator::BitXor => Some(left_value ^ right_value),
_ => None,
}
}
_ => None,
}
}

View File

@@ -5,6 +5,21 @@ use ruff_macros::{derive_message_formats, violation};
use crate::checkers::ast::Checker;
/// ## What it does
/// Checks for uses of the builtin `exec` function.
///
/// ## Why is this bad?
/// The `exec()` function is insecure as it allows for arbitrary code
/// execution.
///
/// ## Example
/// ```python
/// exec("print('Hello World')")
/// ```
///
/// ## References
/// - [Python documentation: `exec`](https://docs.python.org/3/library/functions.html#exec)
/// - [Common Weakness Enumeration: CWE-78](https://cwe.mitre.org/data/definitions/78.html)
#[violation]
pub struct ExecBuiltin;

View File

@@ -3,6 +3,27 @@ use ruff_text_size::TextRange;
use ruff_diagnostics::{Diagnostic, Violation};
use ruff_macros::{derive_message_formats, violation};
/// ## What it does
/// Checks for hardcoded bindings to all network interfaces (`0.0.0.0`).
///
/// ## Why is this bad?
/// Binding to all network interfaces is insecure as it allows access from
/// unintended interfaces, which may be poorly secured or unauthorized.
///
/// Instead, bind to specific interfaces.
///
/// ## Example
/// ```python
/// ALLOWED_HOSTS = ["0.0.0.0"]
/// ```
///
/// Use instead:
/// ```python
/// ALLOWED_HOSTS = ["127.0.0.1", "localhost"]
/// ```
///
/// ## References
/// - [Common Weakness Enumeration: CWE-200](https://cwe.mitre.org/data/definitions/200.html)
#[violation]
pub struct HardcodedBindAllInterfaces;

View File

@@ -1,11 +1,42 @@
use rustpython_parser::ast::{Arg, ArgWithDefault, Arguments, Expr, Ranged};
use crate::checkers::ast::Checker;
use ruff_diagnostics::{Diagnostic, Violation};
use ruff_macros::{derive_message_formats, violation};
use crate::checkers::ast::Checker;
use super::super::helpers::{matches_password_name, string_literal};
/// ## What it does
/// Checks for potential uses of hardcoded passwords in function argument
/// defaults.
///
/// ## Why is this bad?
/// Including a hardcoded password in source code is a security risk, as an
/// attacker could discover the password and use it to gain unauthorized
/// access.
///
/// Instead, store passwords and other secrets in configuration files,
/// environment variables, or other sources that are excluded from version
/// control.
///
/// ## Example
/// ```python
/// def connect_to_server(password="hunter2"):
/// ...
/// ```
///
/// Use instead:
/// ```python
/// import os
///
///
/// def connect_to_server(password=os.environ["PASSWORD"]):
/// ...
/// ```
///
/// ## References
/// - [Common Weakness Enumeration: CWE-259](https://cwe.mitre.org/data/definitions/259.html)
#[violation]
pub struct HardcodedPasswordDefault {
name: String,

View File

@@ -1,11 +1,38 @@
use rustpython_parser::ast::{Keyword, Ranged};
use crate::checkers::ast::Checker;
use ruff_diagnostics::{Diagnostic, Violation};
use ruff_macros::{derive_message_formats, violation};
use crate::checkers::ast::Checker;
use super::super::helpers::{matches_password_name, string_literal};
/// ## What it does
/// Checks for potential uses of hardcoded passwords in function calls.
///
/// ## Why is this bad?
/// Including a hardcoded password in source code is a security risk, as an
/// attacker could discover the password and use it to gain unauthorized
/// access.
///
/// Instead, store passwords and other secrets in configuration files,
/// environment variables, or other sources that are excluded from version
/// control.
///
/// ## Example
/// ```python
/// connect_to_server(password="hunter2")
/// ```
///
/// Use instead:
/// ```python
/// import os
///
/// connect_to_server(password=os.environ["PASSWORD"])
/// ```
///
/// ## References
/// - [Common Weakness Enumeration: CWE-259](https://cwe.mitre.org/data/definitions/259.html)
#[violation]
pub struct HardcodedPasswordFuncArg {
name: String,

View File

@@ -7,6 +7,32 @@ use crate::checkers::ast::Checker;
use super::super::helpers::{matches_password_name, string_literal};
/// ## What it does
/// Checks for potential uses of hardcoded passwords in strings.
///
/// ## Why is this bad?
/// Including a hardcoded password in source code is a security risk, as an
/// attacker could discover the password and use it to gain unauthorized
/// access.
///
/// Instead, store passwords and other secrets in configuration files,
/// environment variables, or other sources that are excluded from version
/// control.
///
/// ## Example
/// ```python
/// SECRET_KEY = "hunter2"
/// ```
///
/// Use instead:
/// ```python
/// import os
///
/// SECRET_KEY = os.environ["SECRET_KEY"]
/// ```
///
/// ## References
/// - [Common Weakness Enumeration: CWE-259](https://cwe.mitre.org/data/definitions/259.html)
#[violation]
pub struct HardcodedPasswordString {
name: String,

Some files were not shown because too many files have changed in this diff Show More