Compare commits

..

780 Commits

Author SHA1 Message Date
konstin
de239ace74 Add test cases 2023-09-22 15:50:34 +02:00
konstin
53b5121f30 . 2023-09-21 16:58:54 +02:00
konstin
be93983e8e Basic nested range formatting 2023-09-21 15:23:17 +02:00
konstin
a1239b8f2d Make basic range formatting work in vs code 2023-09-21 13:38:05 +02:00
konstin
d1b12acb3c Add byte offset indexing 2023-09-21 13:37:52 +02:00
konstin
1aabf59f77 Formatter and parser refactoring
I got confused and refactored a bit
2023-09-21 13:28:59 +02:00
dependabot[bot]
ab643017f9 Bump smallvec from 1.11.0 to 1.11.1 (#7563)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-21 11:53:27 +02:00
dependabot[bot]
6eb9a9a633 Bump insta from 1.31.0 to 1.32.0 (#7565)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-21 11:53:14 +02:00
dependabot[bot]
288c07d911 Bump rayon from 1.7.0 to 1.8.0 (#7564)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-21 11:52:41 +02:00
Micha Reiser
f8f1cd5016 Introduce FormatterSettings (#7545) 2023-09-21 08:01:24 +02:00
Charlie Marsh
87a0cd219f Detect asyncio.get_running_loop calls in RUF006 (#7562)
## Summary

We can do a good enough job detecting this with our existing semantic
model.

Closes https://github.com/astral-sh/ruff/issues/3237.
2023-09-21 04:37:38 +00:00
Charlie Marsh
b685ee4749 Enable tab completion for ruff rule (#7560)
## Summary

Writing `ruff rule E1`, then tab, shows:

<img width="724" alt="Screen Shot 2023-09-20 at 9 29 09 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/00297f24-8828-4485-a00e-6af1ab4e7875">

Closes https://github.com/astral-sh/ruff/issues/2812.
2023-09-20 22:05:39 -04:00
Charlie Marsh
ad893f8295 Avoid invalid fix for parenthesized values in F601 (#7559)
Closes https://github.com/astral-sh/ruff/issues/4897.
2023-09-21 01:28:11 +00:00
Charlie Marsh
621bed55c0 Add padding in PERF102 fixes (#7554)
Closes https://github.com/astral-sh/ruff/issues/7097.
2023-09-20 19:33:54 -04:00
Charlie Marsh
86faee1522 Update CONTRIBUTING.md to reflect Settings refactor (#7555)
## Summary

See:
https://github.com/astral-sh/ruff/pull/7544#issuecomment-1728457885.
2023-09-20 19:33:33 -04:00
Charlie Marsh
a0917ec658 Avoid inserting imports directly after continuation (#7553)
## Summary

This is extremely rare in practice, but common in the fuzzer issues so
worth fixing quickly.

Closes https://github.com/astral-sh/ruff/issues/7199.
2023-09-20 21:26:48 +00:00
Mahesh Saripalli
ee9ee005c5 Update flake8-self license to MIT (#7552)
## Summary

This PR updates the flake8-self license from Freely Distributable to the
MIT License to match the upstream license, which was recently updated.


f5fc507515/LICENSE
2023-09-20 17:17:55 -04:00
Charlie Marsh
5df0326bc8 Treat parameters-with-newline as empty in function formatting (#7550)
## Summary

If a function has no parameters (and no comments within the parameters'
`()`), we're supposed to wrap the return annotation _whenever_ it
breaks. However, our `empty_parameters` test didn't properly account for
the case in which the parameters include a newline (but no other
content), like:

```python
def get_dashboards_hierarchy(
) -> Dict[Type['BaseDashboard'], List[Type['BaseDashboard']]]:
    """Get hierarchy of dashboards classes.

    Returns:
        Dict of dashboards classes.
    """
    dashboards_hierarchy = {}
```

This PR fixes that detection. Instead of lexing, it now checks if the
parameters itself is empty (or if it contains comments).

Closes https://github.com/astral-sh/ruff/issues/7457.
2023-09-20 16:20:22 -04:00
Micha Reiser
6540321966 Move Settings and ResolverSettings to ruff_workspace
## Summary

## Stack Summary

This stack splits `Settings` into `FormatterSettings` and `LinterSettings` and moves it into `ruff_workspace`. This change is necessary to add the `FormatterSettings` to `Settings` without adding `ruff_python_formatter` as a dependency to `ruff_linter` (and the linter should not contain the formatter settings). 

A quick overview of our settings struct at play:

* `Options`: 1:1 representation of the options in the `pyproject.toml` or `ruff.toml`.  Used for deserialization.
* `Configuration`: Resolved `Options`, potentially merged from multiple configurations (when using `extend`). The representation is very close if not identical to the `Options`.
* `Settings`: The resolved configuration that uses a data format optimized for reading. Optional fields are initialized with their default values. Initialized by `Configuration::into_settings` .

The goal of this stack is to split `Settings` into tool-specific resolved `Settings` that are independent of each other. This comes at the advantage that the individual crates don't need to know anything about the other tools. The downside is that information gets duplicated between `Settings`. Right now the duplication is minimal (`line-length`, `tab-width`) but we may need to come up with a solution if more expensive data needs sharing.

This stack focuses on `Settings`. Splitting `Configuration` into some smaller structs is something I'll follow up on later. 

## PR Summary

This PR moves the `ResolverSettings` and `Settings` struct to `ruff_workspace`. `LinterSettings` remains in `ruff_linter` because it gets passed to lint rules, the `Checker` etc.

## Test Plan

`cargo test`
2023-09-20 17:24:28 +02:00
Micha Reiser
b34278e0cd Introduce LinterSettings
## Stack Summary

This stack splits `Settings` into `FormatterSettings` and `LinterSettings` and moves it into `ruff_workspace`. This change is necessary to add the `FormatterSettings` to `Settings` without adding `ruff_python_formatter` as a dependency to `ruff_linter` (and the linter should not contain the formatter settings). 

A quick overview of our settings struct at play:

* `Options`: 1:1 representation of the options in the `pyproject.toml` or `ruff.toml`.  Used for deserialization.
* `Configuration`: Resolved `Options`, potentially merged from multiple configurations (when using `extend`). The representation is very close if not identical to the `Options`.
* `Settings`: The resolved configuration that uses a data format optimized for reading. Optional fields are initialized with their default values. Initialized by `Configuration::into_settings` .

The goal of this stack is to split `Settings` into tool-specific resolved `Settings` that are independent of each other. This comes at the advantage that the individual crates don't need to know anything about the other tools. The downside is that information gets duplicated between `Settings`. Right now the duplication is minimal (`line-length`, `tab-width`) but we may need to come up with a solution if more expensive data needs sharing.

This stack focuses on `Settings`. Splitting `Configuration` into some smaller structs is something I'll follow up on later. 

## PR Summary

This PR extracts the linter-specific settings into a new `LinterSettings` struct and adds it as a `linter` field to the `Settings` struct. This is in preparation for moving `Settings` from `ruff_linter` to `ruff_workspace`

## Test Plan

`cargo test`
2023-09-20 17:02:34 +02:00
Micha Reiser
222f1c37b8 Add empty lines between options and move documents (#7547) 2023-09-20 16:40:25 +02:00
Micha Reiser
8f41eab0c7 Extract ResolverSettings
## Stack Summary

This stack splits `Settings` into `FormatterSettings` and `LinterSettings` and moves it into `ruff_workspace`. This change is necessary to add the `FormatterSettings` to `Settings` without adding `ruff_python_formatter` as a dependency to `ruff_linter` (and the linter should not contain the formatter settings). 

A quick overview of our settings struct at play:

* `Options`: 1:1 representation of the options in the `pyproject.toml` or `ruff.toml`.  Used for deserialization.
* `Configuration`: Resolved `Options`, potentially merged from multiple configurations (when using `extend`). The representation is very close if not identical to the `Options`.
* `Settings`: The resolved configuration that uses a data format optimized for reading. Optional fields are initialized with their default values. Initialized by `Configuration::into_settings` .

The goal of this stack is to split `Settings` into tool-specific resolved `Settings` that are independent of each other. This comes at the advantage that the individual crates don't need to know anything about the other tools. The downside is that information gets duplicated between `Settings`. Right now the duplication is minimal (`line-length`, `tab-width`) but we may need to come up with a solution if more expensive data needs sharing.

This stack focuses on `Settings`. Splitting `Configuration` into some smaller structs is something I'll follow up on later. 

## PR Summary

This PR extracts a `ResolverSettings` struct that holds all the resolver-relevant fields (uninteresting for the `Formatter` or `Linter`). This will allow us to move the `ResolverSettings` out of `ruff_linter` further up in the stack.


## Test Plan

`cargo test`

(I'll to more extensive testing at the top of this stack)
2023-09-20 16:37:49 +02:00
Micha Reiser
83daddbeb7 Rename ConfigProcessor to ConfigurationTransformer (#7536) 2023-09-20 14:17:06 +00:00
Micha Reiser
b19eec9b2a Unify Settings and AllSettings (#7532) 2023-09-20 13:56:07 +00:00
Micha Reiser
ca3c15858d Use portable hash function for CacheKey (#7533) 2023-09-20 15:19:59 +02:00
Micha Reiser
bb4f7c681a Rename format option to output-format (#7514) 2023-09-20 15:18:58 +02:00
Dhruv Manilawala
0a167dd20b Use strict sorted and union for NoQA mapping insertion (#7531)
## Summary

This PR fixes the way NoQA range is inserted to the `NoqaMapping`.

Previously, the way the mapping insertion logic worked was as follows:
1. If the range which is to be inserted _touched_ the previous range, meaning
that the end of the previous range was the same as the start of the new
range, then the new range was added in addition to the previous range.
2. Else, if the new range intersected the previous range, then the previous
   range was replaced with the new _intersection_ of the two ranges.

The problem with this logic is that it does not work for the following case:
```python
assert foo, \
    """multi-line
    string"""
```

Now, the comments cannot be added to the same line which ends with a continuation
character. So, the `NoQA` directive has to be added to the next line. But, the
next line is also a triple-quoted string, so the `NoQA` directive for that line
needs to be added to the next line. This creates a **union** pattern instead of an
**intersection** pattern.

But, only union doesn't suffice because (1) means that for the edge case where
the range touch only at the end, the union won't take place.

### Solution

1. Replace '<=' with '<' to have a _strict_ insertion case
2. Use union instead of intersection

## Test Plan

Add a new test case. Run the test suite to ensure that nothing is broken.

### Integration

1. Make a `test.py` file with the following contents:

    ```python
    assert foo, \
        """multi-line
        string"""
    ```

2. Run the following command:

    ```console
	$ cargo run --bin ruff -- check --isolated --no-cache --select=F821 test.py
	/Users/dhruv/playground/ruff/fstring.py:1:8: F821 Undefined name `foo`
    Found 1 error.
    ```

3. Use `--add-noqa`:

    ```console
	$ cargo run --bin ruff -- check --isolated --no-cache --select=F821 --add-noqa test.py
    Added 1 noqa directive.
    ```

4. Check that the NoQA directive was added in the correct position:

    ```python
    assert foo, \
        """multi-line
        string"""  # noqa: F821
    ```

5. Run the `check` command to ensure that the NoQA directive is respected:

    ```console
	$ cargo run --bin ruff -- check --isolated --no-cache --select=F821 test.py
    ```

fixes: #7530
2023-09-20 11:07:19 +00:00
dependabot[bot]
c43542896f Bump unicode-width from 0.1.10 to 0.1.11 (#7534)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-20 10:26:54 +02:00
Micha Reiser
192463c2fb Allow parenthesized content exceed configured line width (#7490) 2023-09-20 08:39:25 +02:00
Charlie Marsh
5849a75223 Rename ruff crate to ruff_linter (#7529) 2023-09-20 08:38:27 +02:00
Mathieu Kniewallner
dcbd8eacd8 feat(rules): implement flake8-bandit S507 (ssh_no_host_key_verification) (#7528)
Part of #1646.

## Summary

Implement `S507`
([`ssh_no_host_key_verification`](https://bandit.readthedocs.io/en/latest/plugins/b507_ssh_no_host_key_verification.html))
rule from `bandit`.

## Test Plan

Snapshot test from
https://github.com/PyCQA/bandit/blob/1.7.5/examples/no_host_key_verification.py,
with several additions to test for more cases (most notably passing the
parameter as a named argument).
2023-09-19 20:31:45 -04:00
Micha Reiser
297ec2c2d2 Use Settings where AllSettings isn't required (#7518) 2023-09-19 23:16:20 +02:00
Tom Kuson
511cc25fc4 [refurb] Implement unnecessary-enumerate (FURB148) (#7454)
## Summary

Implement
[`no-ignored-enumerate-items`](https://github.com/dosisod/refurb/blob/master/refurb/checks/builtin/no_ignored_enumerate.py)
as `unnecessary-enumerate` (`FURB148`).

The auto-fix considers if a `start` argument is passed to the
`enumerate()` function. If only the index is used, then the suggested
fix is to pass the `start` value to the `range()` function. So,

```python
for i, _ in enumerate(xs, 1):
    ...
```

becomes

```python
for i in range(1, len(xs)):
    ...
```

If the index is ignored and only the value is ignored, and if a start
value greater than zero is passed to `enumerate()`, the rule doesn't
produce a suggestion. I couldn't find a unanimously accepted best way to
iterate over a collection whilst skipping the first n elements. The rule
still triggers, however.

Related to #1348.

## Test Plan

`cargo test`

---------

Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2023-09-19 20:19:28 +00:00
Charlie Marsh
4c4eceee36 Add dangling comment handling for lambda expressions (#7493)
## Summary

This PR adds dangling comment handling for `lambda` expressions. In
short, comments around the `lambda` and the `:` are all considered
dangling. Comments that come between the `lambda` and the `:` may be
moved after the colon for simplicity (this is an odd position for a
comment anyway), unless they also precede the lambda parameters, in
which case they're formatted before the parameters.

Closes https://github.com/astral-sh/ruff/issues/7470.

## Test Plan

`cargo test`

No change in similarity.

Before:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 398 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99929 | 648 | 16 |
| zulip | 0.99962 | 1437 | 22 |

After:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 398 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99929 | 648 | 16 |
| zulip | 0.99962 | 1437 | 22 |
2023-09-19 15:23:51 -04:00
Charlie Marsh
e07670ad97 Add dangling comment handling to dictionary key-value pairs (#7495)
## Summary

This PR fixes a formatting instability by changing the comment handling
around the `:` in a dictionary to mirror that of the `:` in a lambda: we
treat comments around the `:` as dangling, then format them after the
`:`.

Closes https://github.com/astral-sh/ruff/issues/7458.

## Test Plan

`cargo test`

No change in similarity.

Before:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1631 |
| django | 0.99983 | 2760 | 36 |
| transformers | 0.99956 | 2587 | 404 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99929 | 648 | 16 |
| zulip | 0.99969 | 1437 | 21 |

After:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1631 |
| django | 0.99983 | 2760 | 36 |
| transformers | 0.99956 | 2587 | 404 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99929 | 648 | 16 |
| zulip | 0.99969 | 1437 | 21 |
2023-09-19 19:17:21 +00:00
Charlie Marsh
b792140579 Ensure that LOG007 only triggers on .exception() calls (#7524)
## Summary

Previously, this rule triggered for `logging.info("...",
exc_info=False)`.
2023-09-19 17:29:54 +00:00
Tom Kuson
36a60bd50e Add documentation to non-empty-stub-body (#7519)
## Summary

Add documentation to `non-empty-stub-body` (`PYI010`) rule. Related to
#2646.

## Test Plan

`python scripts/check_docs_formatted.py`
2023-09-19 13:21:11 -04:00
konsti
4ae463d04b Add a test for stmt assign breaking in preview mode (#7516)
In preview mode, black will consistently break the right side first.
This doesn't work yet, but we'll need the test later.
2023-09-19 16:16:40 +02:00
konsti
6dade5b9ab Tokenizer: Emit only a single bogus token (#7425)
**Summary** Instead of emitting a bogus token per char, we now only emit
on single last bogus token. This leads to much more concise output.

**Test Plan** Updated fixtures
2023-09-19 16:06:03 +02:00
Charlie Marsh
97510c888b Show --no-X variants in CLI help (#7504)
I'd really like this to render as `--preview / --no-preview`, but I
looked for a while in the Clap internals and issue tracker
(https://github.com/clap-rs/clap/issues/815) and I really can't figure
out a way to do it -- this seems like the best we can do? It's also what
they do in Orogene.

Closes https://github.com/astral-sh/ruff/issues/7486.
2023-09-19 12:27:30 +00:00
konsti
94b68f201b Fix stylist indentation with a formfeed (#7489)
**Summary** In python, a formfeed is technically undefined behaviour
(https://docs.python.org/3/reference/lexical_analysis.html#indentation):
> A formfeed character may be present at the start of the line; it will
be ignored for
> the indentation calculations above. Formfeed characters occurring
elsewhere in the
> leading whitespace have an undefined effect (for instance, they may
reset the space
> count to zero).

In practice, they just reset the indentation:


df8b3a46a7/Parser/tokenizer.c (L1819-L1821)
a41bb2733f/crates/ruff_python_parser/src/lexer.rs (L664-L667)

The stylist didn't handle formfeeds previously and would produce invalid
indents. The remedy is to cut everything before a form feed.

Checks box for
https://github.com/astral-sh/ruff/issues/7455#issuecomment-1722458825

**Test Plan** Unit test for the stylist and a regression test for the
rule
2023-09-19 12:01:16 +02:00
konsti
ef34c5cbec Update itertools to 0.11 (#7513)
Preparation for #7469.

Changelog:
https://github.com/rust-itertools/itertools/blob/master/CHANGELOG.md#0110
2023-09-19 09:53:14 +00:00
dependabot[bot]
fdbefd777c Bump syn from 2.0.33 to 2.0.37 (#7512)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-19 11:08:56 +02:00
dependabot[bot]
078547adbb Bump clap from 4.4.3 to 4.4.4 (#7511)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-19 11:08:39 +02:00
dependabot[bot]
42a0bec146 Bump schemars from 0.8.13 to 0.8.15 (#7510)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-19 11:08:22 +02:00
Micha Reiser
37b7d0f921 fix: Compiler warning about unused map_or (#7508) 2023-09-19 08:10:01 +00:00
Micha Reiser
6a4dbd622b Add optimized best_fit_parenthesize IR (#7475) 2023-09-19 06:29:05 +00:00
Charlie Marsh
28b48ab902 Avoid flagging starred expressions in UP007 (#7505)
## Summary

These can't be fixed, because fixing them would lead to invalid syntax.
So flagging them also feels misleading.

Closes https://github.com/astral-sh/ruff/issues/7452.
2023-09-19 03:37:38 +00:00
Valeriy Savchenko
4123d074bd [refurb] Implement reimplemented-starmap rule (FURB140) (#7253)
## Summary

This PR is part of a bigger effort of re-implementing `refurb` rules
#1348. It adds support for
[FURB140](https://github.com/dosisod/refurb/blob/master/refurb/checks/itertools/use_starmap.py)

## Test Plan

I included a new test + checked that all other tests pass.
2023-09-19 02:18:54 +00:00
Mathieu Kniewallner
c6ba7dfbc6 feat(rules): implement flake8-bandit S201 (flask_debug_true) (#7503)
Part of #1646.

## Summary

Implement `S201`
([`flask_debug_true`](https://bandit.readthedocs.io/en/latest/plugins/b201_flask_debug_true.html))
rule from `bandit`.

I am fairly new to Rust and Ruff's codebase, so there might be better
ways to implement the rule or write the code.

## Test Plan

Snapshot test from
https://github.com/PyCQA/bandit/blob/1.7.5/examples/flask_debug.py, with
a few additions in the "unrelated" part to test a bit more cases.
2023-09-19 00:43:28 +00:00
Micael Jarniac
40f6456add Use MkDocs' not_in_nav (#5498)
Closes #5497
Needs MkDocs 1.5 to be released.
- [x] https://github.com/mkdocs/mkdocs/milestone/15

## Summary
Uses MkDocs' `not_in_nav` config to hide spam about files in
`docs/rules/` not being in nav.
2023-09-19 00:01:43 +00:00
Micha Reiser
3e1dffab20 refactor: Use OnceCell in Memoized (#7500) 2023-09-18 19:58:55 +00:00
Micha Reiser
3336d23f48 perf: Reduce best fitting allocations (#7411) 2023-09-18 19:49:44 +00:00
Jonathan Plasse
2421805033 Avoid N802 violations for @overload methods (#7498)
Close #7479

The `@override` was already implemented

## Test Plan

Tested the code in the issue. After removing all the noqa's, only one
occurrence of `BadName()` raised a violation.
Added a fixture
2023-09-18 14:32:40 -04:00
Tom Kuson
359f50e6dc Ignore pass-statement-stub-body documentation formatting (#7497)
## Summary

Fix CI (broken in #7496).

The code snippet was formatted as Black would format a stub file, but
the CI script doesn't know that (it assumes all code snippets are
non-stub files). Easier to ignore.

Sorry for breaking CI!

## Test Plan

`python scripts/check_docs_formatted.py`
2023-09-18 14:27:20 -04:00
qdegraaf
bccba5d73f [flake8-logging] Implement LOG007: ExceptionWithoutExcInfo (#7410)
## Summary

This PR implements a new rule for `flake8-logging` plugin that checks
for uses of `logging.exception()` with `exc_info` set to `False` or a
falsy value. It suggests using `logging.error` in these cases instead.

I am unsure about the name. Open to suggestions there, went with the
most explicit name I could think of in the meantime.

Refer https://github.com/astral-sh/ruff/issues/7248

## Test Plan

Added a new fixture cases and ran `cargo test`
2023-09-18 17:46:12 +00:00
Tom Kuson
0bfdb15ecf Add documentation to pass-statement-stub-body (#7496)
## Summary

Add documentation to `pass-statement-stub-body` (`PYI009`) rule. Related
to #2646.

## Test Plan

`python scripts/check_docs_formatted.py`
2023-09-18 17:33:15 +00:00
dependabot[bot]
a902d14c31 Bump chrono from 0.4.30 to 0.4.31 (#7481) 2023-09-18 13:11:51 -04:00
Charlie Marsh
728539291f Move FormatExprDict to top of expr_dict.rs (#7494)
Put the node itself up top, and internal structs down below.
2023-09-18 11:55:18 -04:00
Dhruv Manilawala
c2bd8af59a Remove triple-quoted string ranges computation (#7476)
## Summary

This is a follow-up PR for #7435 to remove the now unused triple-quoted
string ranges from the indexer.
2023-09-18 20:57:49 +05:30
Jaap Roes
c946bf157e Extend bad-dunder-method-name to permit __html__ (#7492)
## Summary

Fixes #7478

## Test Plan

`cargo test`
2023-09-18 15:16:22 +00:00
Charlie Marsh
8ab2519717 Respect parentheses for precedence in await (#7468)
## Summary

We were using `Parenthesize::IfBreaks` universally for `await`, but
dropping parentheses can change the AST due to precedence. It turns out
that Black's rules aren't _exactly_ the same as operator precedence
(e.g., they leave parentheses around `await ([1, 2, 3])`, although they
aren't strictly required).

Closes https://github.com/astral-sh/ruff/issues/7467.

## Test Plan

`cargo test`

No change in similarity.

Before:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 398 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99929 | 648 | 16 |
| zulip | 0.99962 | 1437 | 22 |

After:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 398 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99929 | 648 | 16 |
| zulip | 0.99962 | 1437 | 22 |
2023-09-18 09:56:41 -04:00
konsti
c4d85d6fb6 Fix ''' ""''' formatting (#7485)
## Summary

`''' ""'''` is an edge case that was previously incorrectly formatted as
`""" """""`.

Fixes #7460

## Test Plan

Added regression test
2023-09-18 10:28:15 +00:00
dependabot[bot]
70ea49bf72 Bump test-case from 3.1.0 to 3.2.1 (#7484)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-18 11:31:41 +02:00
dependabot[bot]
8e255974bc Bump unicode-ident from 1.0.11 to 1.0.12 (#7482)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-18 11:31:06 +02:00
dependabot[bot]
d358604464 Bump serde_json from 1.0.106 to 1.0.107 (#7480)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-18 11:30:06 +02:00
dependabot[bot]
8243db74fe Bump indoc from 2.0.3 to 2.0.4 (#7483)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-18 11:29:50 +02:00
Micha Reiser
0346e781d4 Fix handling of newlines in empty files (#7473) 2023-09-18 06:08:10 +00:00
Tom Kuson
b66bfa6570 Extend bad-dunder-method-name to permit attrs dunders (#7472)
## Summary

Closes #7451.

## Test Plan

`cargo test`
2023-09-17 16:13:33 -04:00
Charlie Marsh
28273eb00b Avoid flagging starred elements in C402 (#7466)
## Summary

Rewriting these is not valid syntax.

Part of https://github.com/astral-sh/ruff/issues/7455.
2023-09-17 15:23:27 +00:00
Charlie Marsh
12acd191e1 Remove parentheses when rewriting assert calls to statements (#7464) 2023-09-17 15:18:56 +00:00
Charlie Marsh
64b929bc29 Add padding to prevent some autofix errors (#7461)
## Summary

We should really be doing this anywhere we _replace_ an expression.

See: https://github.com/astral-sh/ruff/issues/7455.
2023-09-17 15:14:16 +00:00
Micha Reiser
26ae0a6e8d Fix dangling module comments (#7456) 2023-09-17 14:56:41 +00:00
Dhruv Manilawala
959338d39d Refactor tab-indentation as a token-based rule (#7435)
## Summary

This PR updates the `W191` (`tab-indentation`) rule from a line-based to
a token-based rule.

Earlier, the rule used the `triple_quoted_string_ranges` from the
indexer to skip over any lines _inside_ a triple-quoted string. This was the only
use of the ranges. These ranges were extracted through the tokens, so instead
we can directly use the newline tokens to perform the check.

This would also mean that we can remove the `triple_quoted_string_ranges` from
the indexer but I'll hold that off until we have a better idea on #7326
but I don't think it would be a problem to remove it.

This will also fix #7379 once PEP 701 changes are merged.

## Test Plan

`cargo test`
2023-09-16 20:25:20 +00:00
Charlie Marsh
422ff82f4a Avoid extra parentheses in yield expressions (#7444)
## Summary

This is equivalent to https://github.com/astral-sh/ruff/pull/7424, but
for `yield` and `yield from` expressions. Specifically, we want to avoid
adding unnecessary extra parentheses for `yield expr` when `expr` itself
does not require parentheses.

## Test Plan

`cargo test`

No change in any of the similarity metrics.

Before:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 399 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99929 | 648 | 16 |
| zulip | 0.99962 | 1437 | 22 |

After:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 399 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99929 | 648 | 16 |
| zulip | 0.99962 | 1437 | 22 |
2023-09-16 14:46:56 -04:00
Charlie Marsh
8d0a5e01bd Modify comment_ranges slice in BackwardsTokenizer (#7432)
## Summary

I was kinda curious to understand this issue
(https://github.com/astral-sh/ruff/issues/7426) and just ended up
attempting to address it.

## Test Plan

`cargo test`
2023-09-16 14:04:45 -04:00
Charlie Marsh
aae02cf275 Fix broken is_expression_parenthesized call from rebase (#7442) 2023-09-16 17:22:16 +00:00
Charlie Marsh
7e2eba2592 Avoiding grouping comprehension targets separately from conditions (#7429)
## Summary

Black seems to treat comprehension targets and conditions as if they're
in a single group -- so if the comprehension expands, all conditions are
rendered on their own line, etc.

Closes https://github.com/astral-sh/ruff/issues/7421.

## Test Plan

`cargo test`

No change in any of the similarity metrics.

Before:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 399 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99923 | 648 | 18 |
| zulip | 0.99962 | 1437 | 22 |

After:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 399 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99923 | 648 | 18 |
| zulip | 0.99962 | 1437 | 22 |
2023-09-16 17:19:34 +00:00
Charlie Marsh
22770fb4be Avoid extra parentheses in await expressions (#7424)
## Summary

This PR aligns the await parenthesizing with the unary case, which is:
if the value is already parenthesized, avoid parenthesizing; otherwise,
only parenthesize if the _value_ needs parenthesizing.

Closes https://github.com/astral-sh/ruff/issues/7420.

## Test Plan

`cargo test`

No change in similarity.

Before:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 399 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99923 | 648 | 18 |
| zulip | 0.99962 | 1437 | 22 |

After:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 399 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99923 | 648 | 18 |
| zulip | 0.99962 | 1437 | 22 |
2023-09-16 13:10:35 -04:00
Charlie Marsh
1880cceac1 Avoid extra parentheses in unary expressions (#7428)
## Summary

This PR applies a similar fix to unary expressions as in
https://github.com/astral-sh/ruff/pull/7424. Specifically, we only need
to parenthesize the entire operator if the operand itself doesn't have
parentheses, and requires parentheses.

Closes https://github.com/astral-sh/ruff/issues/7423.

## Test Plan

`cargo test`

No change in similarity.

Before:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 399 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99923 | 648 | 18 |
| zulip | 0.99962 | 1437 | 22 |

After:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 399 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99923 | 648 | 18 |
| zulip | 0.99962 | 1437 | 22 |
2023-09-16 13:07:38 -04:00
Dhruv Manilawala
0d1fb823d6 [flake8-logging] Implement LOG002: invalid-get-logger-argument (#7399)
## Summary

This PR implements a new rule for `flake8-logging` plugin that checks
for
`logging.getLogger` calls with either `__file__` or `__cached__` as the
first
argument and generates a suggested fix to use `__name__` instead.

Refer: #7248

## Test Plan

Add test cases and `cargo test`
2023-09-16 12:21:30 -04:00
Micha Reiser
c907317199 Fix build (#7437) 2023-09-16 14:50:36 +00:00
Micha Reiser
916dd5b7fa fix: Use BestFit layout for subscript (#7409) 2023-09-16 16:21:45 +02:00
konsti
2cbe1733c8 Use CommentRanges in backwards lexing (#7360)
## Summary

The tokenizer was split into a forward and a backwards tokenizer. The
backwards tokenizer uses the same names as the forwards ones (e.g.
`next_token`). The backwards tokenizer gets the comment ranges that we
already built to skip comments.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2023-09-16 03:21:45 +00:00
Charlie Marsh
1f6e1485f9 Change playground to use pages deploy (#7430)
Fixes a deprecation warning.
2023-09-16 03:11:34 +00:00
Charlie Marsh
9b43162cc4 Move documentation to docs.astral.sh/ruff (#7419)
## Summary

We're planning to move the documentation from
[https://beta.ruff.rs/docs](https://beta.ruff.rs/docs) to
[https://docs.astral.sh/ruff](https://docs.astral.sh/ruff), for a few
reasons:

1. We want to remove the `beta` from the domain, as Ruff is no longer
considered beta software.
2. We want to migrate to a structure that could accommodate multiple
future tools living under one domain.

The docs are actually already live at
[https://docs.astral.sh/ruff](https://docs.astral.sh/ruff), but later
today, I'll add a permanent redirect from the previous to the new
domain. **All existing links will continue to work, now and in
perpetuity.**

This PR contains the code changes necessary for the updated
documentation. As part of this effort, I moved the playground and
documentation from my personal Cloudflare account to our team Cloudflare
account (hence the new `--project-name` references). After merging, I'll
also update the secrets on this repo.
2023-09-15 22:49:42 -04:00
Charlie Marsh
cc9e84c144 Format trailing operator comments as dangling (#7427)
## Summary

Given a trailing operator comment in a unary expression, like:

```python
if (
  not  # comment
  a):
    ...
```

We were attaching these to the operand (`a`), but formatting them in the
unary operator via special handling. Parents shouldn't format the
comments of their children, so this instead attaches them as dangling
comments on the unary expression. (No intended change in formatting.)
2023-09-15 20:34:09 -04:00
Zanie Blue
f4d50a2aec Fix release validation step (#7417)
Proof of concept at #7416
Fixes `main` branch check added in #7279 (see
[failure](https://github.com/astral-sh/ruff/actions/runs/6201772425/job/16839150669))
Removes the meaningless "SHA consistency" check (since we literally
check out the SHA now)
2023-09-15 20:18:25 +00:00
Zanie Blue
0c030b5bf3 Bump version to 0.0.290 (#7413)
See also:
- https://github.com/astral-sh/astral-sh/pull/41
- https://github.com/astral-sh/ruff-pre-commit/pull/51
2023-09-15 13:51:46 -05:00
Zanie Blue
1b082ce67e Add maximum length for line-length to JSON schema (#7412)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->
Adds the maximum of 320 for the line-length setting to the JSON schema
for better integration with IDEs.

Related https://github.com/astral-sh/ruff/pull/6873

## Test Plan

<!-- How was it tested? -->
2023-09-15 18:10:06 +00:00
dependabot[bot]
f936d319cc Bump toml from 0.7.6 to 0.7.8 (#7405)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-15 09:14:58 +00:00
dependabot[bot]
85d8b6228f Bump chrono from 0.4.28 to 0.4.30 (#7406)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-15 09:00:17 +00:00
dependabot[bot]
7594dadc1d Bump syn from 2.0.29 to 2.0.33 (#7404)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-15 08:49:52 +00:00
dependabot[bot]
de37fbfac9 Bump serde-wasm-bindgen from 0.5.0 to 0.6.0 (#7403)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-15 08:47:42 +00:00
dependabot[bot]
4e2769a16c Bump mimalloc from 0.1.38 to 0.1.39 (#7402)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-15 08:46:55 +00:00
Manuel Jacob
75b5c314e3 Change CWE reference in documentation for S607 rule (#7398)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

The previous reference was “CWE-78: Improper Neutralization of Special
Elements used in an OS Command ('OS Command Injection')”, which
describes another issue. The new reference is “CWE-426: Untrusted Search
Path”, which describes exactly the problem that this rule should warn
about.

## Test Plan

The change was not tested, as it only changes two numbers in the
documentation.
2023-09-14 23:12:54 -05:00
Charlie Marsh
450fb9b99a [flake8-logging] Implement LOG001: direct-logger-instantiation (#7397)
## Summary

See: https://github.com/astral-sh/ruff/issues/7248.
2023-09-14 23:07:10 -04:00
Charlie Marsh
6163c99551 Mark PERF403 as a preview rule (#7396) 2023-09-15 01:57:48 +00:00
qdegraaf
3112202a5b [flake8-logging] Add flake8_logging boilerplate and first rule LOG009 (#7249)
## Summary

Adds `LOG009` from
[flake8-logging](https://github.com/adamchainz/flake8-logging). Also
adds the boilerplate for a new plugin

Checks for usages of undocumented `logging.WARN` constant and suggests
replacement with `logging.WARNING`.

## Test Plan

`cargo test` with fresh fixture

## Issue links

Refers: https://github.com/astral-sh/ruff/issues/7248
2023-09-15 01:41:32 +00:00
Charlie Marsh
64ea00048b Add known problems to UP040 documentation (#7395) 2023-09-15 01:31:20 +00:00
qdegraaf
067a4acd54 [perflint] Add PERF403 (#6132)
## Summary

Adds `PERF403` mirroring `W8403` from
https://github.com/tonybaloney/perflint

## Test Plan

Fixtures were added based on perflint tests

## Issue Links

Refers: https://github.com/astral-sh/ruff/issues/4789
2023-09-15 01:23:51 +00:00
Nathan Whitaker
c88376f468 Add support for bounds, constraints, and explicit variance on generic type variables to UP040 (#6749)
## Summary

Extends UP040 to support moving type variables with
bounds/constraints/variance that are used in type aliases to use PEP-695
syntax.

Part of #4617.

## Test Plan

The existing tests added by #6314 already cover the relevant cases.
2023-09-14 21:11:24 -04:00
Charlie Marsh
9b7c29853d Reflect conversion reason in UP012 messages (#7393)
Closes https://github.com/astral-sh/ruff/issues/7254.
2023-09-14 20:08:58 +00:00
Charlie Marsh
3e21d32b79 Avoid flagging single-quoted docstrings with continuations for multi-line rules (#7392)
Rules like D209 and D205 are only intended to apply to multi-line
docstrings. If a docstring is single-quoted, but extends via a
continuation, it should be excluded (it'll be flagged by another rule
anyway). Closes https://github.com/astral-sh/ruff/issues/7058.
2023-09-14 18:58:39 +00:00
Charlie Marsh
f9e3ea23ba Show rule codes in shell tab completion (#7375)
## Summary

I noticed that we have a custom parser for rule selectors, but it wasn't
actually being used? This PR adds it back to our Clap setup and changes
the parser to only show full categories and individual rules when
tab-completing:

<img width="1792" alt="Screen Shot 2023-09-13 at 9 13 38 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/028b18d2-8c92-49c1-b781-f24c9ae310f7">

<img width="1792" alt="Screen Shot 2023-09-13 at 9 13 40 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/fd598da5-78fb-412d-a69e-2a0963d479cd">

<img width="1792" alt="Screen Shot 2023-09-13 at 9 13 58 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/7c482b90-6e54-425c-ae23-fb50496a177a">

The previous implementation showed all codes, which I found too noisy:

<img width="1792" alt="Screen Shot 2023-09-13 at 8 57 09 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/db370a0e-2a9f-4acd-b1e3-224a1f8e9ce5">
2023-09-14 18:37:23 +00:00
Charlie Marsh
6856d0b44b Use dot references in docs for methods (#7391)
## Summary

This matches the convention used in the Python documentation.
2023-09-14 14:35:34 -04:00
Charlie Marsh
21539f1663 Allow NURSERY in JSON Schema (#7374)
## Summary

At some point, we removed these so that they wouldn't be autocompleted
for users, since we wanted to discourage usage of `ALL`. But given that
they're valid values, I think that was a bad idea -- it leads to an even
more confusing experience whereby JSON Schema validators tell you that
you have an error, when you don't.

Closes https://github.com/astral-sh/ruff/issues/7261.
2023-09-14 18:09:35 +00:00
Zanie Blue
b9bb6bf780 Remove the PREVIEW rule selector (#7389)
The rule selector is not useful because `--select PREVIEW` only targets
Ruff developers and `--ignore PREVIEW` has no effect due to its low
specificity. We may restore it later if useful.
2023-09-14 12:31:59 -05:00
Charlie Marsh
d39eae2713 Extend C416 to catch tuple unpacking (#7363)
Closes https://github.com/astral-sh/ruff/issues/7307.
2023-09-14 15:53:15 +00:00
Charlie Marsh
5d21b9c22e Catch panics in formatter (#7377)
## Summary

This PR ensures that we catch and render panics in the formatter
identically to other kinds of errors. It also improves the consistency
in error rendering throughout and makes a few stylistic changes to the
messages.

Closes https://github.com/astral-sh/ruff/issues/7247.

## Test Plan

I created a file `foo.py` with a syntax error, and a file `bar.py` with
an intentional panic.

<img width="1624" alt="Screen Shot 2023-09-13 at 10 25 22 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/605c2839-ad02-4376-a2e9-d5a593ab660f">

<img width="1624" alt="Screen Shot 2023-09-13 at 10 25 24 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/b1381909-157c-48cb-9630-d0bbfcb1b640">
2023-09-14 11:44:16 -04:00
Charlie Marsh
ec2f229a45 Remove ExprContext from ComparableExpr (#7362)
`ComparableExpr` includes the `ExprContext` field on an expression, so,
e.g., the two tuples in `(a, b) = (a, b)` won't be considered equal.
Similarly, the tuples in `[(a, b) for (a, b) in c]` _also_ wouldn't be
considered equal. I find this behavior surprising, since
`ComparableExpr` is intended to allow you to compare two ASTs, but
`ExprContext` is really encoding information about the broader context
for the expression.
2023-09-14 15:40:02 +00:00
Charlie Marsh
34c1cb7d11 Treat parenthesized power operands as non-simple (#7371)
Closes https://github.com/astral-sh/ruff/issues/7318.
2023-09-14 15:36:21 +00:00
dependabot[bot]
2ddea7c657 Bump regex from 1.9.4 to 1.9.5 (#7384)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-14 14:58:24 +00:00
dependabot[bot]
45eabdd2c3 Bump shlex from 1.1.0 to 1.2.0 (#7381)
Bumps [shlex](https://github.com/comex/rust-shlex) from 1.1.0 to 1.2.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/comex/rust-shlex/blob/master/CHANGELOG.md">shlex's
changelog</a>.</em></p>
<blockquote>
<h1>1.2.0</h1>
<ul>
<li>Adds <code>bytes</code> module to support operating directly on byte
strings.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/comex/rust-shlex/commits">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=shlex&package-manager=cargo&previous-version=1.1.0&new-version=1.2.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Micha Reiser <micha@reiser.io>
2023-09-14 09:40:05 -05:00
Micha Reiser
675c86c175 fix: Group fluent subscript (#7386) 2023-09-14 13:04:14 +02:00
dependabot[bot]
1f8e2b8f14 Bump proc-macro2 from 1.0.66 to 1.0.67 (#7383)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-14 09:49:57 +00:00
Micha Reiser
58b3040342 chore: Upgrade notify (#7338) 2023-09-14 09:47:55 +00:00
dependabot[bot]
6a9b8aede1 Bump thiserror from 1.0.47 to 1.0.48 (#7385)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-14 09:21:47 +00:00
dependabot[bot]
f48126ad00 Bump clap from 4.4.1 to 4.4.3 (#7382)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-14 09:13:12 +00:00
Charlie Marsh
11287f944f Avoid re-parenthesizing call chains whose inner values are parenthesized (#7373)
## Summary

Given a statement like:

```python
result = (
    f(111111111111111111111111111111111111111111111111111111111111111111111111111111111)
    + 1
)()
```

When we go to parenthesize the target of the assignment, we use
`maybe_parenthesize_expression` with `Parenthesize::IfBreaks`. This then
checks if the call on the right-hand side needs to be parenthesized, the
implementation of which looks like:

```rust
impl NeedsParentheses for ExprCall {
    fn needs_parentheses(
        &self,
        _parent: AnyNodeRef,
        context: &PyFormatContext,
    ) -> OptionalParentheses {
        if CallChainLayout::from_expression(self.into(), context.source())
            == CallChainLayout::Fluent
        {
            OptionalParentheses::Multiline
        } else if context.comments().has_dangling(self) {
            OptionalParentheses::Always
        } else {
            self.func.needs_parentheses(self.into(), context)
        }
    }
}
```

Checking for `self.func.needs_parentheses(self.into(), context)` is
problematic, since, as in the example above, `self.func` may _already_
be parenthesized -- in which case, we _don't_ want to parenthesize the
entire expression. If we do, we end up with this non-ideal formatting:

```python
result = (
    (
        f(
            111111111111111111111111111111111111111111111111111111111111111111111111111111111
        )
        + 1
    )()
)
```

This PR modifies the `NeedsParentheses` implementations for call chain
expressions to return `Never` if the inner expression has its own
parentheses, in which case, the formatting implementations for those
expressions will preserve them anyway.

Closes https://github.com/astral-sh/ruff/issues/7370.

## Test Plan

Zulip improves a bit, everything else is unchanged.

Before:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99981 | 2760 | 40 |
| transformers | 0.99944 | 2587 | 413 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99834 | 648 | 20 |
| zulip | 0.99956 | 1437 | 23 |

After:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99981 | 2760 | 40 |
| transformers | 0.99944 | 2587 | 413 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99834 | 648 | 20 |
| **zulip** | **0.99962** | **1437** | **22** |
2023-09-14 05:05:37 -04:00
Micha Reiser
a65efcf459 fix: Don't omit optional parentheses for subscripts (#7380) 2023-09-14 08:43:53 +00:00
Jelle van der Waa
04183b0299 [pylint] Implement too-many-public-methods rule (PLR0904) (#6179)
Implement
https://pylint.pycqa.org/en/latest/user_guide/messages/refactor/too-many-public-methods.html

Confusingly the rule page mentions a max of 7 while in practice it is
20.
https://github.com/search?q=repo%3Apylint-dev%2Fpylint+max-public-methods&type=code

## Summary

Implement pylint's R0904

## Test Plan

Unit tests.
2023-09-14 00:52:26 +00:00
James Braza
36fa1fe359 Docs linking error tutorial with error suppression (#7014)
Documents takeaway from
https://github.com/astral-sh/ruff/discussions/7011#discussioncomment-6869239.
2023-09-13 20:22:18 -04:00
Charlie Marsh
6e625bd93d Invert reverse argument regardless of whether it's a boolean (#7372)
## Summary

When fixing `reversed(sorted(x, reverse=False))`, we rewrite as
`sorted(x, reverse=True)`. However, if the `reverse` argument isn't
`True` or `False`, we leave it as-is, which is incorrect.

Now, given `reversed(sorted(x, reverse=y))`, we rewrite as `sorted(x,
reverse=not y)`.
2023-09-13 20:12:35 -04:00
Zanie Blue
ebd1b296fd Add warnings for nursery and preview rule selection (#7210)
## Summary

Adds warnings for cases where:
- A selector does not include any rules because preview is disabled
- A nursery rule is selected without the preview flag

## Test plan

Add integration tests
2023-09-13 15:29:58 -05:00
Zanie Blue
1373e1c395 Update release workflow to checkout the given sha (#7279) 2023-09-13 14:59:41 -05:00
Zanie Blue
4bff397318 Move FURB145 from nursery to preview (#7364)
Moves the new rule from nursery to preview for the upcoming release.

Adds new test coverage for selection of a single preview rule and fixes
a bug where preview rules were incorrectly selectable with exact codes.
2023-09-13 14:54:28 -05:00
Charlie Marsh
5347df4728 Parenthesize single-generator arguments when adding reverse keyword (#7365)
Closes https://github.com/astral-sh/ruff/issues/7289.
2023-09-13 19:40:45 +00:00
Tom Kuson
ebe9c03545 [refurb] Implement no-slice-copy (FURB145) (#7007)
## Summary

Implement
[`no-slice-copy`](https://github.com/dosisod/refurb/blob/master/refurb/checks/builtin/no_slice_copy.py)
as `slice-copy` (`FURB145`).

Related to #1348.

## Test Plan

`cargo test`
2023-09-13 17:31:15 +00:00
Charlie Marsh
b0cbcd3dfa Update deprecated-import lists based on recent typing-extension release (#7356)
## Summary

Generated by running
f3cff244e3/testing/generate-typing-rewrite-info (L14)
with latest `typing-extensions` and manually applying the changes.

Closes https://github.com/astral-sh/ruff/issues/7324.
2023-09-13 13:21:11 -04:00
Charlie Marsh
4df9e07a79 Add a benchmarking script for the formatter CLI (#7340)
## Summary

This PR adds a benchmarking script for the formatter, which benchmarks
the Ruff formatter against Black, yapf, and autopep8.

Three benchmarks are included:

1. Format everything.
2. Format everything, but use a single thread.
3. Format everything, but `--check` (don't write to disk).

There's some nuance in figuring out the right combination of arguments
to each command, but the _main_ nuance is to ensure that we always run
the given formatter (and modify the target repo in-place) prior to
benchmarking it, so that the formatters aren't disadvantaged by the
existing formatting of the target repo. (E.g.: prior to benchmarking
Black's preview style, we need to make sure we format the target repo
with Black's preview style; otherwise, preview style appears much
slower.)

Part of https://github.com/astral-sh/ruff/issues/7309.
2023-09-13 13:15:48 -04:00
Charlie Marsh
f0f7ea7502 Treat whitespace-only line as blank for D411 (#7351)
This better aligns with the definition of "blank line" that we use
throughout the docstring rules.

Closes https://github.com/astral-sh/ruff/issues/7216.
2023-09-13 16:41:12 +00:00
Micha Reiser
4f26002dd5 chore: Upgrade strum (#7337)
## Summary

This PR upgrades `strum` from 0.24.x to 0.25.x. 

The breaking changes are: 
* strum macros now uses syn2
* The `to_string` behavior changed when using `default`. I did a quick search, we aren't using `strum(default)` 


`strum` now has a `#[derive(EnumIs)]` macro that generates `is_` methods. 

## Test Plan

cargo test
2023-09-13 18:33:27 +02:00
dependabot[bot]
d1a9c198e3 Bump path-absolutize from 3.1.0 to 3.1.1 (#7345)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-13 16:21:51 +00:00
dependabot[bot]
7a4f699fba Bump argfile from 0.1.5 to 0.1.6 (#7344)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-13 16:18:21 +00:00
dependabot[bot]
3fb5418c2c Bump memchr from 2.6.2 to 2.6.3 (#7343)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-13 18:14:42 +02:00
dependabot[bot]
9fcc009a0c Bump serde_json from 1.0.105 to 1.0.106 (#7342)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-13 18:13:24 +02:00
Micha Reiser
bf8e5a167b chore: Upgrade walkdir (#7336)
## Summary

The only commit is [api: add follow_root_links() option to WalkDir](dcc527d832) whicih addsa  new option wheter `walkdir` should follow a root symlink or not. 
The new option defaults to `true` which is the same as before. 

## Test Plan

`cargo test`
2023-09-13 18:07:24 +02:00
Micha Reiser
8a001dfc3d chore: Upgrade pyproject-toml crate (#7335)
## Summary

This PR bumps the pyproject-toml crate to 0.7.0. The only difference is that it now depends on indexmap 2. I reviewed the indexmap 2 changes and they don't seem relevant to us. 

I used this opportunity to remove the default features from `serde_with` which removes our indexmap 1 dependency (and some other unused dependencies)

## Test Plan

`cargo test`
2023-09-13 17:55:03 +02:00
Zanie Blue
0823394525 Display nursery rules as preview in documentation (#7341)
This is broken in the last release
2023-09-13 10:46:43 -05:00
Zanie Blue
e15047815c Update Rust toolchain file to use TOML format (#7339)
Ref https://rust-lang.github.io/rustup/overrides.html#the-toolchain-file
Should resolve Dependabot failure at
https://github.com/astral-sh/ruff/network/updates/721380342
Follows #7034
2023-09-13 15:43:03 +00:00
Micha Reiser
7531bb3b21 Upgrade is-macros to 0.3.0 (#7334) 2023-09-13 15:27:40 +02:00
Micha Reiser
2d9b39871f Introduce IndentWidth (#7301) 2023-09-13 14:52:24 +02:00
Micha Reiser
e122a96d27 playground: Respect line-length and preview configuration (#7330) 2023-09-13 12:14:25 +00:00
konsti
f4c7bff36b Don't reorder parameters in function calls (#7268)
## Summary

In `f(*args, a=b, *args2, **kwargs)` the args (`*args`, `*args2`) and
keywords (`a=b`, `**kwargs`) are interleaved, which we previously didn't
handle.

Fixes #6498

**main**

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| **django** | 0.99966 | 2760 | 58 |
| transformers | 0.99930 | 2587 | 447 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99825 | 648 | 22 |
| zulip | 0.99950 | 1437 | 27 |

**PR**

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| **django** | 0.99967 | 2760 | 53 |
| transformers | 0.99930 | 2587 | 447 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99825 | 648 | 22 |
| zulip | 0.99950 | 1437 | 27 |


## Test Plan

New fixtures
2023-09-13 09:01:49 +00:00
konsti
56440ad835 Introduce ArgOrKeyword to keep call parameter order (#7302)
## Motivation

The `ast::Arguments` for call argument are split into positional
arguments (args) and keywords arguments (keywords). We currently assume
that call consists of first args and then keywords, which is generally
the case, but not always:

```python
f(*args, a=2, *args2, **kwargs)

class A(*args, a=2, *args2, **kwargs):
    pass
```

The consequence is accidentally reordering arguments
(https://github.com/astral-sh/ruff/pull/7268).

## Summary

`Arguments::args_and_keywords` returns an iterator of an `ArgOrKeyword`
enum that yields args and keywords in the correct order. I've fixed the
obvious `args` and `keywords` usages, but there might be some cases with
wrong assumptions remaining.

## Test Plan

The generator got new test cases, otherwise the stacked PR
(https://github.com/astral-sh/ruff/pull/7268) which uncovered this.
2023-09-13 08:45:46 +00:00
Charlie Marsh
179128dc54 Link discussion in formatter README (#7311) 2023-09-12 16:50:22 +00:00
Charlie Marsh
e7a2779402 Bump version to v0.0.289 (#7308) 2023-09-12 12:00:11 -04:00
Zanie Blue
008da95b29 Add preview documentation section (#7281)
Adds a basic documentation section for preview mode based on the FAQ
entry and versioning RFC.
2023-09-12 15:43:31 +00:00
Zanie Blue
5d4dd3e38e Set the target deployment to main during dispatched documentation deployments (#7304)
Closes #7276 by deploying to production when not triggered by a pull
request.
2023-09-12 10:34:05 -05:00
Micha Reiser
e561f5783b Fix(vscode): Respect line length ruff.toml configuration (#7306) 2023-09-12 15:31:47 +00:00
Dhruv Manilawala
ee0f1270cf Add NotebookIndex to the cache (#6863)
## Summary

This PR updates the `FileCache` to include an optional `NotebookIndex`
to support caching for Jupyter Notebooks.

We only require the index to compute the diagnostics and thus we don't
really need to store the entire `Notebook` on the `Diagnostics` struct.
This means we only need the index to be stored in the cache to
reconstruct the `Diagnostics`.

## Test Plan

Update an existing test case to run over the fixtures under
`ruff_notebook` crate where there are multiple Jupyter Notebook.

Locally, the following commands were run in order:
1. Remove the cache: `rm -rf .ruff_cache`
2. Run without cache: `cargo run --bin ruff -- check --isolated
crates/ruff_notebook/resources/test/fixtures/jupyter/unused_variable.ipynb
--no-cache`
3. Run with cache: `cargo run --bin ruff -- check --isolated
crates/ruff_notebook/resources/test/fixtures/jupyter/unused_variable.ipynb`
4. Check whether the `.ruff_cache` directory was created or not
5. Run with cache again and verify: `cargo run --bin ruff -- check
--isolated
crates/ruff_notebook/resources/test/fixtures/jupyter/unused_variable.ipynb`

## Benchmarks

https://github.com/astral-sh/ruff/pull/6863#issuecomment-1715675186

fixes: #6671
2023-09-12 18:29:03 +05:30
Tom Kuson
e7b7e4a18d Add documentation to duplicate-union-member (#7225)
## Summary

Add documentation to `duplicate-union-member` (`PYI016`) rule. Related
to #2646.

## Test Plan

`python scripts/check_docs_formatted.py`
2023-09-12 08:56:33 -04:00
Brendon Happ
b4419c34ea Ignore @override method when enforcing bad-dunder-name rule (#7224)
## Summary

Closes #6958.

If a method has the `override` decorator, there is nothing you can do
about incorrect dunder methods, so they should be ignored.

## Test Plan

Overridden incorrect dunder method was added to the tests to verify ruff
doesn't catch it when evaluating the file. Snapshot changes are all just
line number changes
2023-09-12 11:54:40 +00:00
Micha Reiser
08f19226b9 Fix panic when formatting binary expression with two implicit concatenated string operands (#7287) 2023-09-12 09:49:51 +02:00
Micha Reiser
1e6df19a35 Bool expression comment placement (#7269) 2023-09-12 06:39:57 +00:00
Zanie Blue
c21b960fc7 Display the --preview option in the CLI help menu (#7274)
If we're going to warn on use of NURSERY in #7210 we probably ought to
show the `--preview` option in our help menus.
2023-09-11 18:09:58 -05:00
Zanie Blue
73ad2affa1 Update preview and fix documentation symbols (#7207)
I don't love the sunrise emoji and 🧪 seems nice :)

Requires #7195

---------

Co-authored-by: konsti <konstin@mailbox.org>
2023-09-11 18:08:00 -05:00
Zanie Blue
40c936922e Add "Preview" section to auto-generated release notes (#7280) 2023-09-11 18:07:47 -05:00
Charlie Marsh
874db4fb86 Invert condition for < and <= in outdated version block (#7284)
Closes https://github.com/astral-sh/ruff/issues/7258.
2023-09-11 23:02:23 +00:00
Dhruv Manilawala
a41bb2733f Add range to lexer test snapshots (#7265)
## Summary

This PR updates the lexer test snapshots to include the range value as
well. This is mainly a mechanical refactor.

### Motivation

The main motivation is so that we can verify that the ranges are valid
and do not overlap.

## Test Plan

`cargo test`
2023-09-11 19:12:46 +00:00
Zanie Blue
24b848a4ea Enable preview mode during benchmarks (#7208)
Split out of https://github.com/astral-sh/ruff/pull/7195 so benchmark
changes from enabling additional rules can be reviewed separately.
2023-09-11 14:09:33 -05:00
Zanie Blue
773ba5f816 Update the docs workflow to allow publishing a specific ref (#7278)
Related https://github.com/astral-sh/ruff/issues/7276

Our docs publishing action does not allow targetting a specific commit
when run manually which means we cannot update the documentation to
anything but the latest commit on `main`. This change allows a ref to be
provided.

---------

Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
2023-09-11 18:51:31 +00:00
Dhruv Manilawala
f5701fcc63 Use snapshots for remaining lexer tests (#7264)
## Summary

This PR updates the remaining lexer test cases to use the snapshots.
This is mainly a mechanical refactor.

## Motivation

The main motivation is so that when we add the token range values to the
test case output, it's easier to update the test cases.

The reason they were not using the snapshots before was because of the usage of
`test_case` macro. The macros is mainly used for different EOL test cases. If we
just generate the snapshots directly, then the snapshot name would be suffixed
with `-1`, `-2`, etc. as the test function is still the same. So, we'll create
the snapshot ourselves with the platform name for the respective EOL
test cases.

## Test Plan

`cargo test`
2023-09-12 00:16:38 +05:30
Zanie Blue
ff0feb191c Use pages deploy instead of the deprecated pages publish command to deploy the docs website (#7277)
See https://github.com/cloudflare/workers-sdk/issues/3067

Related #7276
2023-09-11 13:23:47 -05:00
Zanie Blue
6566d00295 Update rule selection to respect preview mode (#7195)
## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

Extends work in #7046 (some relevant discussion there)

Changes:

- All nursery rules are now referred to as preview rules
- Documentation for the nursery is updated to describe preview
- Adds a "PREVIEW" selector for preview rules
- This is primarily to allow `--preview --ignore PREVIEW --extend-select
FOO001,BAR200`
- Using `--preview` enables preview rules that match selectors

Notable decisions:

- Preview rules are not selectable by their rule code without enabling
preview
- Retains the "NURSERY" selector for backwards compatibility
- Nursery rules are selectable by their rule code for backwards
compatiblity

Additional work:

- Selection of preview rules without the "--preview" flag should display
a warning
- Use of deprecated nursery selection behavior should display a warning
- Nursery selection should be removed after some time

## Test Plan

<!-- How was it tested? -->

Manual confirmation (i.e. we don't have an preview rules yet just
nursery rules so I added a preview rule for manual testing)

New unit tests

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2023-09-11 12:28:39 -05:00
Micha Reiser
7c9bbcf4e2 Bump version to 0.0.288 (#7271)
Co-authored-by: Zanie Blue <contact@zanie.dev>
2023-09-11 18:18:11 +02:00
konsti
babf8d718e Fix D204 when there's a semicolon after a docstring (#7174)
## Summary

Another statement on the same line as the docstring would previous make
the D204 (newline after docstring) fix fail:

```python
class StatementOnSameLineAsDocstring:
    "After this docstring there's another statement on the same line separated by a semicolon." ;priorities=1
    def sort_services(self):
        pass
```

The fix handles this case manually:

```python
class StatementOnSameLineAsDocstring:
    "After this docstring there's another statement on the same line separated by a semicolon."

    priorities=1
    def sort_services(self):
        pass
```

Fixes #7088

## Test Plan

Added a new `D` test case
2023-09-11 15:09:47 +02:00
dependabot[bot]
878813f277 Bump tj-actions/changed-files from 38 to 39 (#7267)
Co-authored-by: repo-ranger[bot] <!-- raw HTML omitted --> (<a
Co-authored-by: renovate[bot] <!-- raw HTML omitted --> (<a
Co-authored-by: jackton1 <a
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-11 11:12:24 +02:00
dependabot[bot]
71c8a02ebd Bump actions/checkout from 3 to 4 (#7266)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-11 11:11:25 +02:00
konsti
3a2c3a7398 Format empty lines in stub files like black's preview style (#7206)
## Summary

Fix all but one empty line differences with the black preview style in
typeshed. The remaining differences are breaking with type comments and
trailing commas in function definitions.

I compared the empty line differences with the preview mode of black
since stable has some oddities that would have been hard to replicate
(https://github.com/psf/black/issues/3861). Additionally, it assumes the
style proposed in https://github.com/psf/black/issues/3862.

An edge case that also surfaced with typeshed are newline before
trailing module comments.

**main**

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99966 | 2760 | 58 |
| transformers | 0.99930 | 2587 | 447 |
| twine | 1.00000 | 33 | 0 |
| **typeshed** | 0.99978 | 3496 | **2173** |
| warehouse | 0.99825 | 648 | 22 |
| zulip | 0.99950 | 1437 | 27 |

**PR**
| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99966 | 2760 | 58 |
| transformers | 0.99930 | 2587 | 447 |
| twine | 1.00000 | 33 | 0 |
| **typeshed** | 0.99983 | 3496 | **18** |
| warehouse | 0.99825 | 648 | 22 |
| zulip | 0.99950 | 1437 | 27 |


Closes #6723

## Test Plan

The main driver was the typeshed diff. I added new test cases for all
kinds of possible empty line combinations in stub files, test cases for
newlines before trailing module comments.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2023-09-11 08:03:59 +00:00
Micha Reiser
7440e54ec6 Avoid allocating in lex_decimal (#7252) 2023-09-11 06:37:25 +00:00
Charlie Marsh
0357e801ed Avoid F401 panic with noqa import name (#7260)
Closes https://github.com/astral-sh/ruff/issues/7244.
2023-09-10 16:31:25 +00:00
Ely Ronnen
69d0caabe7 fix D417 error with function docstrings with dashed lines (#7251)
## Summary

Fix https://github.com/astral-sh/ruff/issues/7250, False positive D417
for docstrings with dashed lines.

## Test Plan

Tested on the example in https://github.com/astral-sh/ruff/issues/7250
2023-09-10 17:28:44 +01:00
Micha Reiser
9cb5ce750e Remove IO based lints from linter benchmark (#7240) 2023-09-08 13:39:50 +02:00
Micha Reiser
0a07a2ca62 Extract string part and normalized string (#7219) 2023-09-08 12:56:55 +02:00
Micha Reiser
47a253fb62 Add PreviewMode option to formatter
## Summary

This PR adds the `--preview` and `--no-preview` options to the `format` command (hidden) and passes it through to the formatte. 

## Test Plan

I added the `dbg(f.options().preview())` statement in `FormatNodeRule::fmt` and verified that the option gets correctly passed to the formatter.
2023-09-08 12:04:28 +02:00
Micha Reiser
d9544a2d37 Disable default criterion features (#7241) 2023-09-08 10:03:14 +00:00
Micha Reiser
41f0aad7b3 Add FString support to binary like formatting
## Summary

This is the last part of the string - binary like formatting. It adds support for handling fstrings the same as "regular" strings.


## Test Plan

I added a test for both binary and comparison. 

Small improvements across several projects

**This PR**
| project      | similarity index  | total files       | changed files     |
|--------------|------------------:|------------------:|------------------:|
| cpython      |           0.76083 |              1789 |              1632 |
| django       |           0.99966 |              2760 |                58 |
| **transformers** |           0.99929 |              2587 |               454 |
| twine        |           1.00000 |                33 |                 0 |
| typeshed     |           0.99978 |              3496 |              2173 |
| **warehouse**    |           0.99825 |               648 |                22 |
| **zulip**        |           0.99950 |              1437 |                27 |


**Base**

| project      | similarity index  | total files       | changed files     |
|--------------|------------------:|------------------:|------------------:|
| cpython      |           0.76083 |              1789 |              1632 |
| django       |           0.99966 |              2760 |                58 |
| transformers |           0.99928 |              2587 |               454 |
| twine        |           1.00000 |                33 |                 0 |
| typeshed     |           0.99978 |              3496 |              2173 |
| warehouse    |           0.99824 |               648 |                22 |
| zulip        |           0.99948 |              1437 |                28 |


<!-- How was it tested? -->
2023-09-08 11:48:57 +02:00
qdegraaf
05951dd338 Fix inconsistent expr_lambda formatting (#6318) 2023-09-08 09:40:58 +00:00
Micha Reiser
c260762900 Formatter: Implicit concatenation in compare expressions
## Summary

This PR implements the logic for breaking implicit concatenated strings before compare expressions by building on top of  #7145 

The main change is a new `BinaryLike` enum that has the `BinaryExpression` and `CompareExpression` variants. Supporting both variants requires some downstream changes but doesn't introduce any new concepts. 

## Test Plan

I added a few more tests. The compatibility improvements are minor but we now perfectly match black on twine 🥳 


**PR**

| project      | similarity index  | total files       | changed files     |
|--------------|------------------:|------------------:|------------------:|
| cpython      |           0.76083 |              1789 |              1632 |
| django       |           0.99966 |              2760 |                58 |
| transformers |           0.99928 |              2587 |               454 |
| **twine**        |           1.00000 |                33 |                 0 | <-- improved
| typeshed     |           0.99978 |              3496 |              2173 |
| **warehouse**    |           0.99824 |               648 |                22 | <-- improved
| zulip        |           0.99948 |              1437 |                28 |


**Base**

| project      | similarity index  | total files       | changed files     |
|--------------|------------------:|------------------:|------------------:|
| cpython      |           0.76083 |              1789 |              1633 |
| django       |           0.99966 |              2760 |                58 |
| transformers |           0.99928 |              2587 |               454 |
| twine        |           0.99982 |                33 |                 1 | 
| typeshed     |           0.99978 |              3496 |              2173 |
| warehouse    |           0.99823 |               648 |                23 |
| zulip        |           0.99948 |              1437 |                28 |
2023-09-08 11:32:20 +02:00
konsti
1d5c4b0a14 Show header for formatter comment decoration info (#7228)
Show header for formatter comment decoration info

**Summary** Show a header in the formatter comment decoration debug
output that shows which node is preceding/following/enclosing
(https://github.com/astral-sh/ruff/pull/6813#issuecomment-1708119550). I
kept this intentionally condensed to make it easy to use this is a small
sidebar without vertical scrolling.

```console
$ cargo run --bin ruff_python_formatter -- --emit stdout --print-comments scratch.py
# Comment decoration: Range, Preceding, Following, Enclosing, Comment
17..20, Some((ParameterWithDefault, 6..10)), None, (Parameters, 5..22), "# a"
44..47, Some((StmtExpr, 28..39)), Some((StmtExpr, 52..60)), (StmtFunctionDef, 0..60), "# b"
77..80, None, None, (ExprList, 71..82), "# c"
{
    Node {
        kind: ParameterWithDefault,
        range: 6..10,
        source: `x=[]`,
    }: {
...
```

**Test Plan** It's debug output.
2023-09-08 09:25:06 +00:00
Micha Reiser
a352f2f092 Preserve generator parentheses in single argument call expressions (#7226) 2023-09-08 10:53:34 +02:00
Micha Reiser
e376c3ff7e Split implicit concatenated strings before binary expressions (#7145) 2023-09-08 06:51:26 +00:00
Greger
9671922e40 Do not use code location for Gitlab fingerprints. (#7203) 2023-09-08 08:25:26 +02:00
konsti
45f9fca228 Reuse locator in formatter comments (#7227)
**Summary** The comment visitor used to rebuild the locator for every
comment. Instead, we now keep the locator on the builder. Follow-up to
#6813.

**Test Plan** No formatting changes.
2023-09-07 20:08:28 +02:00
Charlie Marsh
6661be2c30 Use full range for SIM105 fixes (#7221)
Avoids inserting an accidental extra newline after the fix _and_
addresses the case of a trailing semicolon.
2023-09-07 16:16:43 +01:00
Charlie Marsh
97f945651d Change SIM118 to delete .keys() rather than replace expression (#7223)
Also improves the suggestion text. Closes
https://github.com/astral-sh/ruff/issues/7200.
2023-09-07 16:16:24 +01:00
Charlie Marsh
5cea43731e Add required space for FLY002 fixes (#7222)
Closes https://github.com/astral-sh/ruff/issues/7197.
2023-09-07 14:32:43 +00:00
Jaap Roes
7971e0b0ee Add extend-ignore-names for flake8-self (#7194)
## Summary

Add a configuration option to extend the list of names that can be
accessed without triggering SLF001.

Fixes issue #7018

## Test Plan

Manually tested by creating a python file (`test.py`):

```python
def foo(obj):
    obj._meta
```

and a `ruff.toml` file:

```toml
select = ["SLF"]

[flake8-self]
extend-ignore-names = ["_meta"]
```

Then running `cargo run -p ruff_cli -- check test.py --no-cache` (once
with the `extend-ignore-names` line comment out) to see if the
configuration option works.

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2023-09-07 13:41:22 +00:00
Micha Reiser
842ff0212e Add Lexer emoji test case (#7213) 2023-09-07 10:02:50 +00:00
Micha Reiser
f1a4eb9c28 Use the unicode-ident crate (#7212) 2023-09-07 08:19:25 +00:00
Victor Hugo Gomes
041cdb95e0 Update identifier Unicode character validation to match Python spec (#7209)
Co-authored-by: Micha Reiser <micha@reiser.io>
2023-09-07 07:08:42 +00:00
Charlie Marsh
fda48afc23 Add required space to C416 fix (#7204)
Closes https://github.com/astral-sh/ruff/issues/7196.
2023-09-06 16:25:02 +00:00
Charlie Marsh
c1af1c291d Add required space to UP006 and UP007 fixes (#7202)
We're removing a set of brackets here so we need to pad the fix.

Closes https://github.com/astral-sh/ruff/issues/7201.
2023-09-06 16:06:02 +00:00
Micha Reiser
171b66cb43 Lexer: Add skip whitespace fastpath (#7184) 2023-09-06 16:14:01 +02:00
Charlie Marsh
fa0b6f4813 Avoid attempting to fix SIM105 violations with multi-statement lines (#7191)
I may revisit this and fix it "properly", but so rare that it's worth
disabling for now: https://github.com/astral-sh/ruff/issues/7123.
2023-09-06 13:35:46 +00:00
Charlie Marsh
eab85aea1a Use structured types for C417 comprehension target (#7190)
Rather than manually joining the arguments as a comma-separated string,
and treating that comma-separated string as a name.
2023-09-06 13:20:04 +00:00
Charlie Marsh
5b31524920 Parenthesize C417 targets if necessary (#7189)
Closes https://github.com/astral-sh/ruff/issues/7121.
2023-09-06 12:56:47 +00:00
Charlie Marsh
6d0469638c Avoid attempting to fix NPY001 with overridden builtins (#7187) 2023-09-06 12:24:37 +00:00
Charlie Marsh
29ba2bb943 Add required space when fixing C404 (#7185) 2023-09-06 13:05:39 +01:00
Charlie Marsh
f0ea40a68d Restructure signatures of flake8_comprehensions fixers (#7186) 2023-09-06 12:04:50 +00:00
Charlie Marsh
a3a531e0d4 Add alpha instructions to the ruff_python_formatter README (#7064) 2023-09-06 11:55:16 +00:00
Tom Kuson
b3e8eca871 Rename PLR1714 to repeated-equality-comparison (#7182) 2023-09-06 12:46:48 +01:00
konsti
447b7cb0e2 Formatter: Show preceding, following and enclosing nodes of comments, Attempt 2 (#6813) 2023-09-06 12:26:13 +02:00
konsti
e3114a144c Ignore single quote docstrings with newline escape (#7173) 2023-09-06 10:51:50 +02:00
Manuel Martinez
2e58ad437e build: add libcst from crates (#7179)
Co-authored-by: Micha Reiser <micha@reiser.io>
2023-09-06 08:24:28 +00:00
Dhruv Manilawala
04f2842e4f Move ExprConstant::kind to StringConstant::unicode (#7180) 2023-09-06 07:39:25 +00:00
Micha Reiser
31990b8d3f Checker: Remove unnecessary unreachable (#7181) 2023-09-06 07:21:03 +00:00
Micha Reiser
5f59101811 Memoize text width (#6552) 2023-09-06 07:10:13 +00:00
Dhruv Manilawala
fa6bff0078 Add inline documentation for Ipy* AST nodes (#7178) 2023-09-06 12:07:34 +05:30
Dhruv Manilawala
ea7c394817 Copy the starred argument as is for PLW3301 autofix (#7177) 2023-09-06 08:57:05 +05:30
Charlie Marsh
264d9159f8 Add required space when fixing UP024 (#7171) 2023-09-05 17:37:09 +00:00
Charlie Marsh
37dfb205b1 Remove autofix for ambiguous unicode rules (#7168) 2023-09-05 17:22:18 +00:00
Charlie Marsh
f8e4e1d562 Fix named expression precedence in generator (#7170) 2023-09-05 17:06:57 +00:00
Charlie Marsh
89be850b73 Add required space when fixing SIM300 (#7167) 2023-09-05 17:00:07 +00:00
Zanie Blue
d68041ba24 Fix B006 when function docstring is followed by whitespace but no newline (#7160) 2023-09-05 11:10:57 -05:00
Charlie Marsh
b60b37e866 Split within not, rather than outside of not, for PT018 (#7151) 2023-09-05 14:50:16 +00:00
konsti
5a95edab45 Use ruff line-length in format_dev (#6870) 2023-09-05 16:19:17 +02:00
Dhruv Manilawala
1adde24133 Rename parser mode from Jupyter to Ipython (#7153) 2023-09-05 14:12:26 +00:00
konsti
e02d76f070 Use insta_cmd (#6737) 2023-09-05 12:21:27 +00:00
Charlie Marsh
7ead2c17b1 Add required space when fixing C402 (#7152) 2023-09-05 12:19:33 +00:00
Charlie Marsh
e428099e4c Add required space when fixing SIM118 (#7150) 2023-09-05 11:51:34 +00:00
Charlie Marsh
7a83fd9926 Insert required space when fixing B013 (#7148) 2023-09-05 12:49:11 +01:00
Charlie Marsh
e8f78fa2cf Avoid fixing UP022 when capture_output is provided (#7149) 2023-09-05 11:44:17 +00:00
Charlie Marsh
955501f267 Use generator for UP007 autofix (#7137) 2023-09-05 11:41:53 +00:00
Micha Reiser
175b3702c3 Reduce comments.clone calls (#7144) 2023-09-05 11:32:56 +02:00
Nicholas Grisafi
40ee4909b5 Added argfile test and documentation (#7138)
Co-authored-by: konsti <konstin@mailbox.org>
2023-09-05 11:13:58 +02:00
Charlie Marsh
10a8e4a225 Remove output-file and target-version from formatter CLI (#7135) 2023-09-05 09:04:18 +00:00
konsti
0465b03282 Better formatter CLI verbose output (#7129) 2023-09-05 00:25:16 +02:00
Dhruv Manilawala
154fe7bdcc Add lexer benchmark (#7132) 2023-09-04 13:18:36 +00:00
Charlie Marsh
ece30e7c69 Preserve parentheses around partial call chains (#7109) 2023-09-04 10:57:04 +01:00
Charlie Marsh
7be28a38c5 Cache comment lookups in suite.rs (#7092) 2023-09-04 08:45:14 +00:00
Charlie Marsh
5ec73a6137 Avoid triggering N806 on TypeAlias assignments (#7119) 2023-09-04 08:44:28 +00:00
Dhruv Manilawala
1067261a55 Make SourceKind a required parameter (#7013) 2023-09-04 07:45:59 +00:00
Micha Reiser
93ca8ebbc0 Formatter: Detect line endings (#7054) 2023-09-04 08:09:31 +02:00
Charlie Marsh
834566f34f Retain parentheses when fixing SIM210 (#7118) 2023-09-03 22:39:23 +00:00
Charlie Marsh
a56121672c Add parentheses when simplifying conditions in SIM222 (#7117) 2023-09-03 22:35:59 +00:00
Charlie Marsh
32f4a96c64 Fix precedence of annotated assignments in generator (#7115) 2023-09-03 21:41:48 +00:00
Charlie Marsh
c004e03395 Add space after return when inlining number for RET504 (#7116) 2023-09-03 21:33:41 +00:00
Charlie Marsh
b57ddd54d2 Support parenthesized expressions in UP028 (#7114) 2023-09-03 21:20:09 +00:00
dalgarno
af189db5eb Deduplicate information in configuration documentation (#7108) 2023-09-03 22:17:35 +01:00
Charlie Marsh
c7217e34d7 Avoid adding duplicate text keyword to subprocess.run (#7112) 2023-09-03 21:17:04 +00:00
Charlie Marsh
d70247959c Avoid adding duplicate capture_output keyword to subprocess.run (#7113) 2023-09-03 21:14:56 +00:00
Charlie Marsh
911d4f2918 Handle parenthesized calls in PD002 (#7111) 2023-09-03 21:03:55 +00:00
Charlie Marsh
d9cf31f355 Expand F841 fixes to handle parenthesized targets (#7110) 2023-09-03 21:00:44 +00:00
Olivia Crain
7da99cc756 Fix incorrect flake8-copyright link in faq (#7093) 2023-09-03 18:00:25 +00:00
Charlie Marsh
9c3b2c3c3c Parenthesize expressins when converting to B009 (#7091) 2023-09-03 17:50:38 +01:00
Charlie Marsh
37d244d178 Add newline if B006 fix is at end-of-file (#7090) 2023-09-03 17:35:59 +01:00
Charlie Marsh
dbb34804a4 Change Option<Result> to Result<Option> in importer (#7089) 2023-09-03 16:23:42 +00:00
Charlie Marsh
3c7486817b Use symbol import for NPY003 replacement (#7083) 2023-09-03 16:53:28 +01:00
Charlie Marsh
3c3c5b5c57 Support length-2 lists in dictionary comprehension rewrites (#7081) 2023-09-03 13:34:10 +00:00
Charlie Marsh
b0d171ac19 Supported starred exceptions in length-one tuple detection (#7080) 2023-09-03 13:31:13 +00:00
Charlie Marsh
b70dde4a77 Avoid attempting to fix invalid Optional annotations (#7079) 2023-09-03 13:23:36 +00:00
Dhruv Manilawala
4eaa412370 Update LibCST (#7062)
## Summary

This PR updates the revision of `LibCST` dependency to 9c263aa897
inorder to fix https://github.com/astral-sh/ruff/issues/4899

## Test Plan

The test case including the carriage return (`\r`) character was added for
`F504` and then `cargo test`.

fixes: #4899
2023-09-03 09:11:24 +05:30
Charlie Marsh
577280c8be Rename ruff_python_formatter/README.md to CONTRIBUTING.md (#7065) 2023-09-02 16:25:23 +00:00
Charlie Marsh
45680bbb44 Avoid duplicate fixes for multi-import imports in RUF017 (#7063)
If a user has `import collections, functools, operator`, and we try to
import from `functools` and `operator`, we end up adding two identical
synthetic edits to preserve that import statement. We need to dedupe
them.

Closes https://github.com/astral-sh/ruff/issues/7059.
2023-09-02 12:58:18 +01:00
Justin Prieto
71ff6f911d Fix getattr calls on int literals (#7057) 2023-09-02 11:45:35 +00:00
Micha Reiser
c05e4628b1 Introduce Token element (#7048) 2023-09-02 10:05:47 +02:00
Charlie Marsh
2f3a950f6f Bump version to 0.0.287 (#7038) 2023-09-01 17:32:26 +01:00
Charlie Marsh
dea65536e9 Fix placement for comments within f-strings concatenations (#7047)
## Summary

Restores the dangling comment handling for f-strings, which broke with
the parenthesized expression code.

Closes https://github.com/astral-sh/ruff/issues/6898.

## Test Plan

`cargo test`

No change in any of the similarity indexes or changed file counts:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99957 | 2760 | 67 |
| transformers | 0.99927 | 2587 | 468 |
| twine | 0.99982 | 33 | 1 |
| typeshed | 0.99978 | 3496 | 2173 |
| warehouse | 0.99818 | 648 | 24 |
| zulip | 0.99942 | 1437 | 32 |
2023-09-01 16:27:32 +00:00
dependabot[bot]
fbc9b5a604 Bump cloudflare/wrangler-action from 3.1.0 to 3.1.1 (#7045) 2023-09-01 15:00:11 +00:00
Zanie Blue
253a241f5d Add dependabot for cargo dependencies (#7034)
Ideally we shouldn't have to run `cargo update` manually — it requires
us to remember to do so and groups all updates into a single pull
request making it challenging to determine which upgrade introduces
regressions e.g. #6964. Here we add daily checks for cargo dependency
updates.

This pull request also simplifies dependabot configuration for GitHub
Actions versions.
2023-09-01 09:47:40 -05:00
Sergey Chudov
33806b8b7c Fixed panic in missing_copyright_notice (#7029) 2023-09-01 13:58:48 +00:00
Charlie Marsh
afcd00da56 Create ruff_notebook crate (#7039)
## Summary

This PR moves `ruff/jupyter` into its own `ruff_notebook` crate. Beyond
the move itself, there were a few challenges:

1. `ruff_notebook` relies on the source map abstraction. I've moved the
source map into `ruff_diagnostics`, since it doesn't have any
dependencies on its own and is used alongside diagnostics.
2. `ruff_notebook` has a couple tests for end-to-end linting and
autofixing. I had to leave these tests in `ruff` itself.
3. We had code in `ruff/jupyter` that relied on Python lexing, in order
to provide a more targeted error message in the event that a user saves
a `.py` file with a `.ipynb` extension. I removed this in order to avoid
a dependency on the parser, it felt like it wasn't worth retaining just
for that dependency.

## Test Plan

`cargo test`
2023-09-01 13:56:44 +00:00
Charlie Marsh
08e246764f Refactor ruff_cli's run method to return on each branch (#7040)
## Summary

I think the fallthrough here for some branches is a little confusing.
Now each branch either runs a command that returns `Result<ExitStatus>`,
or runs a command that returns `Result<()>` and then explicitly returns
`Ok(ExitStatus::SUCCESS)`.
2023-09-01 14:15:38 +01:00
Chris Pryer
0489bbc54c Match Black's formatting of trailing comments containing NBSP (#7030) 2023-09-01 14:52:59 +02:00
Charlie Marsh
60132da7bb Add a NotebookError type to avoid returning Diagnostics on error (#7035)
## Summary

This PR refactors the error-handling cases around Jupyter notebooks to
use errors rather than `Box<Diagnostics>`, which creates some oddities
in the downstream handling. So, instead of formatting errors as
diagnostics _eagerly_ (in the notebook methods), we now return errors
and convert those errors to diagnostics at the last possible moment (in
`diagnostics.rs`). This is more ergonomic, as errors can be composed and
reported-on in different ways, whereas diagnostics require a `Printer`,
etc.

See, e.g.,
https://github.com/astral-sh/ruff/pull/7013#discussion_r1311136301.

## Test Plan

Ran `cargo run` over a Python file labeled with a `.ipynb` suffix, and
saw:

```
foo.ipynb:1:1: E999 SyntaxError: Expected a Jupyter Notebook, which must be internally stored as JSON, but found a Python source file: expected value at line 1 column 1
```
2023-09-01 11:08:05 +00:00
Chris Pryer
17a44c0078 Exclude pragma comments from measured line width (#7008)
Co-authored-by: Micha Reiser <micha@reiser.io>
2023-09-01 06:34:51 +00:00
Charlie Marsh
376d3caf47 Treat empty-line separated comments as trailing statement comments (#6999)
## Summary

This PR modifies our between-statement comment handling such that
comments that are not separated by a statement by any newlines continue
to be treated as leading comments on the statement, but comments that
_are_ separated are instead formatted as trailing comments on the
preceding statement.

See, e.g., the originating snippet:

```python
DEFAULT_TEMPLATE = "flatpages/default.html"

# This view is called from FlatpageFallbackMiddleware.process_response
# when a 404 is raised, which often means CsrfViewMiddleware.process_view
# has not been called even if CsrfViewMiddleware is installed. So we need
# to use @csrf_protect, in case the template needs {% csrf_token %}.
# However, we can't just wrap this view; if no matching flatpage exists,
# or a redirect is required for authentication, the 404 needs to be returned
# without any CSRF checks. Therefore, we only
# CSRF protect the internal implementation.


def flatpage(request, url):
    pass
```

Here, we need to ensure that the `def flatpage` is precede by two empty
lines. However, we want those two empty lines to be enforced from the
_end_ of the comment block, _unless_ the comments are directly atop the
`def flatpage`.

I played with this a bit, and I think the simplest conceptual model and
implementation is to instead treat those as trailing comments on the
preceding node. The main difficulty with this approach is that, in order
to be fully compatible with Black, we'd sometimes need to insert
newlines _between_ the preceding node and its trailing comments. See,
e.g.:

```python
def func():
    ...
# comment

x = 1
```

In this case, we'd need to insert two blank lines between `def func():
...` and `# comment`, but `# comment` is trailing comment on `def
func(): ...`. So, we'd need to take this case into account in the
various nodes that _require_ newlines after them: functions, classes,
and imports. After some discussion, we've opted _not_ to support this,
and just treat these as trailing comments -- so we won't insert newlines
there. This means our handling is still identical to Black's on
Black-formatted code, but avoids moving such trailing comments on
unformatted code.

I dislike that the empty handling is so complex, and that it's split
between so many different nodes, but this is really tricky. Continuing
to treat these as leading comments is very difficult too, since we'd
need to do similar tricks for the leading comment handling in those
nodes, and influencing leading comments is even harder, since they're
all formatted _before_ the node itself.

Closes https://github.com/astral-sh/ruff/issues/6761.

## Test Plan

`cargo test`

Surprisingly, it doesn't change the similarity at all (apart from a
0.00001 change in CPython), but I manually confirmed that it did fix the
originating issue in Django.

Before:

| project      | similarity index |
|--------------|------------------|
| cpython      | 0.76082          |
| django       | 0.99921          |
| transformers | 0.99854          |
| twine        | 0.99982          |
| typeshed     | 0.99953          |
| warehouse    | 0.99648          |
| zulip        | 0.99928          |


After:

| project      | similarity index |
|--------------|------------------|
| cpython      | 0.76081          |
| django       | 0.99921          |
| transformers | 0.99854          |
| twine        | 0.99982          |
| typeshed     | 0.99953          |
| warehouse    | 0.99648          |
| zulip        | 0.99928          |
2023-08-31 20:55:05 +00:00
Charlie Marsh
51d69b448c Improve compatibility between multi-statement PYI rules (#7024)
## Summary

This PR modifies a few of our rules related to which statements (and how
many) are allowed in function bodies within `.pyi` files, to improve
compatibility with flake8-pyi and improve the interplay dynamics between
them. Each change fixes a deviation from flake8-pyi:

- We now always trigger the multi-statement rule (PYI048) regardless of
whether one of the statements is a docstring.
- We no longer trigger the `...` rule (PYI010) if the single statement
is a docstring or a `pass` (since those are covered by other rules).
- We no longer trigger the `...` rule (PYI010) if the function body
contains multiple statements (since that's covered by PYI048).

Closes https://github.com/astral-sh/ruff/issues/7021.

## Test Plan

`cargo test`
2023-08-31 21:45:26 +01:00
Nicholas Grisafi
f7dca3d958 Add Rippling to who uses ruff (#7032)
## Summary

Adding rippling to who uses ruff section

## Test Plan

We migrated to ruff 🙂
2023-08-31 18:36:48 +00:00
Charlie Marsh
7c1aa98f43 Run cargo update (#6964) 2023-08-31 13:11:11 -05:00
Charlie Marsh
68f605e80a Fix WithItem ranges for parenthesized, non-as items (#6782)
## Summary

This PR attempts to address a problem in the parser related to the
range's of `WithItem` nodes in certain contexts -- specifically,
`WithItem` nodes in parentheses that do not have an `as` token after
them.

For example,
[here](https://play.ruff.rs/71be2d0b-2a04-4c7e-9082-e72bff152679):

```python
with (a, b):
    pass
```

The range of the `WithItem` `a` is set to the range of `(a, b)`, as is
the range of the `WithItem` `b`. In other words, when we have this kind
of sequence, we use the range of the entire parenthesized context,
rather than the ranges of the items themselves.

Note that this also applies to cases
[like](https://play.ruff.rs/c551e8e9-c3db-4b74-8cc6-7c4e3bf3713a):

```python
with (a, b, c as d):
    pass
```

You can see the issue in the parser here:

```rust
#[inline]
WithItemsNoAs: Vec<ast::WithItem> = {
    <location:@L> <all:OneOrMore<Test<"all">>> <end_location:@R> => {
        all.into_iter().map(|context_expr| ast::WithItem { context_expr, optional_vars: None, range: (location..end_location).into() }).collect()
    },
}
```

Fixing this issue is... very tricky. The naive approach is to use the
range of the `context_expr` as the range for the `WithItem`, but that
range will be incorrect when the `context_expr` is itself parenthesized.
For example, _that_ solution would fail here, since the range of the
first `WithItem` would be that of `a`, rather than `(a)`:

```python
with ((a), b):
    pass
```

The `with` parsing in general is highly precarious due to ambiguities in
the grammar. Changing it in _any_ way seems to lead to an ambiguous
grammar that LALRPOP fails to translate. Consensus seems to be that we
don't really understand _why_ the current grammar works (i.e., _how_ it
avoids these ambiguities as-is).

The solution implemented here is to avoid changing the grammar itself,
and instead change the shape of the nodes returned by various rules in
the grammar. Specifically, everywhere that we return `Expr`, we instead
return `ParenthesizedExpr`, which includes a parenthesized range and the
underlying `Expr` itself. (If an `Expr` isn't parenthesized, the ranges
will be equivalent.) In `WithItemsNoAs`, we can then use the
parenthesized range as the range for the `WithItem`.
2023-08-31 16:21:29 +01:00
Zanie Blue
96a9717c1a Add hidden --preview / --no-preview options to ruff check (#7009)
Per discussion at https://github.com/astral-sh/ruff/discussions/6998

<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

Adds a `--preview` and `--no-preview` option to the CLI for `ruff check`
and corresponding settings. The CLI options are hidden for now.

Available in the settings as `preview = true` or `preview = false`.

Does not include environment variable configuration, although we may add
it in the future.

## Test Plan

<!-- How was it tested? -->

`cargo build`

Future work will build on this setting, such as toggling the mode during
a test.
2023-08-31 09:51:59 -05:00
magic-akari
f4ba0ea144 Allow tab_width to be configable (#7016) 2023-08-31 07:40:03 +00:00
Micha Reiser
92143afeee Group binary operators with same precedence only (#7010) 2023-08-31 09:19:45 +02:00
Micha Reiser
eb552da8a9 Avoid parenthesizing multiline strings in binary expressions (#6973) 2023-08-30 16:03:17 +02:00
Charlie Marsh
e2b2b1759f Handle keyword comments between = and value (#6883)
## Summary

This PR adds comment handling for comments between the `=` and the
`value` for keywords, as in the following cases:

```python
func(
    x  # dangling
    =  # dangling
    # dangling
    1,
    **  # dangling
    y
)
```

(Comments after the `**` were already handled in some cases, but I've
unified the handling with the `=` handling.)

Note that, previously, comments between the `**` and its value were
rendered as trailing comments on the value (so they'd appear after `y`).
This struck me as odd since it effectively re-ordered the comment with
respect to its closest AST node (the value). I've made them leading
comments, though I don't know that that's a significant improvement. I
could also imagine us leaving them where they are.
2023-08-30 09:52:51 -04:00
Chris Pryer
a3f4d7745a Use reserved width to include line suffix measurement (#6901)
Co-authored-by: Micha Reiser <micha@reiser.io>
2023-08-30 08:07:11 +00:00
Micha Reiser
edfd888bd6 Add unicode benchmark (#7002) 2023-08-30 09:57:57 +02:00
Micha Reiser
9c382e8291 Show changed files (#7003) 2023-08-30 06:47:45 +00:00
Charlie Marsh
eb2b226142 Unset after_class_docstring state on every iteration (#7001) 2023-08-30 08:20:28 +02:00
Victor Hugo Gomes
31947af6a3 Don't "flatten" nested if expressions when formatting (#6996) 2023-08-30 04:11:58 +00:00
Charlie Marsh
b404e54f33 Remove unnecessary Comment#slice calls (#6997) 2023-08-30 00:44:11 +00:00
qdegraaf
34e8de738e [perflint] Expand PERF401 and PERF402 with type checks (#6994)
## Summary

Attempt at a small improvement to two `perflint` rules using the new
type inference capabilities to only flag `PERF401` and `PERF402` for
values we infer to be lists. This makes the rule more conservative, as
it only flags values that we _know_ to be lists, but it's overall a
desirable change, as it favors false negatives over false positives for
a "nice-to-have" rule.

Closes https://github.com/astral-sh/ruff/issues/6995.

## Test Plan

Add non-list value cases and make sure all old cases are still caught.
2023-08-29 19:15:29 -04:00
qdegraaf
1550a6bfe7 [perflint] Chore: correct PERF401 fixture comments (#6993)
## Summary
Fixes comments in PERF401 fixture to correctly reflect how rule works
2023-08-29 17:16:31 -04:00
Anselm Hahn
dc4db39f78 docs: 📝 Updating the example for assert(S101) (#6986)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

Rewriting the `if`-comparison to focus on the meaning of rule ` assert
S101`.

Fixes #6984

## Test Plan

<!-- How was it tested? -->

---------

Co-authored-by: Zanie Blue <contact@zanie.dev>
2023-08-29 16:08:08 -05:00
qdegraaf
f3aaf84a28 Move refurb/helpers utils to ruff_python_semantic for broader use (#6990)
## Summary

The utils added for `refurb` in its `helpers.rs` file could be useful
for many other plugins. (Such as the PERF4XX codes, see e.g.
https://github.com/astral-sh/ruff/pull/6132 ).

This PR moves them to `ruff_python_semantic::analyzers::typing` as
suggested in
https://github.com/astral-sh/ruff/pull/6132#issuecomment-1697910093

## Test Plan

Confirmed `refurb` and all other tests still work
2023-08-29 14:45:09 -04:00
Charlie Marsh
5de95d7054 Reuse FormatResult and FormatterIterationError in format_stdin.rs (#6985)
## Summary

Ensures that we use the same error types and messages. Also renames
those struct to `FormatCommand*` for consistency, and removes the
`FormatCommandResult::Skipped` variant in favor of skipping in the
iterator directly.
2023-08-29 17:41:53 +00:00
Charlie Marsh
8d1610d960 Implement Display on formatter structs (#6983)
Feedback from
https://github.com/astral-sh/ruff/pull/6948#discussion_r1308260021.
2023-08-29 16:57:26 +00:00
Charlie Marsh
fad23bbe60 Add a --check flag to the formatter CLI (#6982)
## Summary

Returns an exit code of 1 if any files would be reformatted:

```
ruff on  charlie/format-check:main [$?⇡] is 📦 v0.0.286 via 🐍 v3.11.2 via 🦀 v1.72.0
❯ cargo run -p ruff_cli -- format foo.py --check
   Compiling ruff_cli v0.0.286 (/Users/crmarsh/workspace/ruff/crates/ruff_cli)
    Finished dev [unoptimized + debuginfo] target(s) in 1.69s
     Running `target/debug/ruff format foo.py --check`
warning: `ruff format` is a work-in-progress, subject to change at any time, and intended only for experimentation.
1 file would be reformatted
ruff on  charlie/format-check:main [$?⇡] is 📦 v0.0.286 via 🐍 v3.11.2 via 🦀 v1.72.0 took 2s
❯ echo $?
1
```

Closes #6966.
2023-08-29 12:40:00 -04:00
Charlie Marsh
25c374856a Move stdin formatting to its own command file (#6981)
## Summary

This is similar to `commands::check` vs. `commands::check_stdin`, and
gets the logic out of the parent file (`lib.rs`). It also ensures that
we avoid formatting files that should be excluded when `--force-exclude`
is provided.
2023-08-29 16:06:10 +00:00
Charlie Marsh
34221346c1 Rename run.rs command to check.rs (#6980)
The CLI command is called "check", so this is more consistent (and
consistent with the pattern used in other commands).
2023-08-29 15:52:06 +00:00
Chris Pryer
924f10186f Update dynamic_text builder doc comment (#6978) 2023-08-29 16:29:11 +02:00
Charlie Marsh
b7634b6ede Fix typo in banned-from (#6977)
Oops...
2023-08-29 09:39:09 -04:00
Dhruv Manilawala
4d49d5e845 Add eat_char2 for the lexer (#6968)
## Summary

This PR adds a new helper method on the `Cursor` called `eat_char2`
which is similar to `eat_char` but accepts 2 characters instead of 1. It'll
`bump` the cursor twice if both characters are found on lookahead.

## Test Plan

`cargo test`
2023-08-29 17:18:02 +05:30
Micha Reiser
715d86dae9 Remove Comprehension priority (#6947) 2023-08-29 08:30:15 +02:00
Micha Reiser
adb48692d6 Use optional parentheses for tuples in return statements (#6875) 2023-08-29 08:30:05 +02:00
Charlie Marsh
19ccf1d073 Make RUF100 a suggested fix (#6967)
I made this automatic when I removed the deprecated "unspecified"
method, but I think suggested is actually more appropriate.
2023-08-29 04:28:52 +00:00
Charlie Marsh
e3c36465ec Base parameter-find lookup on range, rather than name (#6960)
## Summary

This simplifies the signature a bit and is more reliable in the
(unusual? invalid?) case of duplicate parameters.
2023-08-29 01:39:38 +00:00
Valeriy Savchenko
7e36284684 [refurb] Implement check-and-remove-from-set rule (FURB132) (#6904)
## Summary

This PR is a continuation of #6897 and #6702 and me replicating `refurb`
rules (#1348). It adds support for
[FURB132](https://github.com/dosisod/refurb/blob/master/refurb/checks/builtin/set_discard.py)

## Test Plan

I included a new test + checked that all other tests pass.
2023-08-29 01:17:26 +00:00
Valeriy Savchenko
c448b4086a [refurb] Implement delete-full-slice rule (FURB131) (#6897)
## Summary

This PR is a continuation of #6702 and me replicating `refurb` rules
(#1348). It adds support for
[FURB131](https://github.com/dosisod/refurb/blob/master/refurb/checks/builtin/no_del.py.

## Test Plan

I included a new test + checked that all other tests pass.
2023-08-28 20:52:38 -04:00
Charlie Marsh
3200015c06 Fix banned-from documentation (#6959)
Closes https://github.com/astral-sh/ruff/issues/6839.
2023-08-29 00:51:15 +00:00
Valeriy Savchenko
26d53c56a2 [refurb] Implement repeated-append rule (FURB113) (#6702)
## Summary

As an initial effort with replicating `refurb` rules (#1348 ), this PR
adds support for
[FURB113](https://github.com/dosisod/refurb/blob/master/refurb/checks/builtin/list_extend.py)
and adds a new category of checks.

## Test Plan

I included a new test + checked that all other tests pass.
2023-08-28 22:51:59 +00:00
Charlie Marsh
1439bb592e Report number of changed files on the format CLI (#6948)
## Summary

Very basic summary:

<img width="962" alt="Screen Shot 2023-08-28 at 1 17 37 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/53537aca-7579-44d8-855b-f4553affae50">

If you run with `--verbose`, we'll also show you the timing:

<img width="962" alt="Screen Shot 2023-08-28 at 1 17 58 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/63cbd13e-9462-4e49-b3a3-c6663a7ad41c">
2023-08-28 18:42:31 -04:00
Charlie Marsh
fc47e0dab2 Clean up some misc. code in NamedTuple and TypedDict conversion (#6957)
- Use `Option` instead of `Result` everywhere.
- Use `field` instead of `property` (to match the nomenclature of
`NamedTuple` and `TypedDict`).
- Put the violation function at the top of the file, rather than the
bottom.
2023-08-28 17:12:56 -04:00
Charlie Marsh
87aa5d6b66 Avoid panic when typename is provided as a keyword argument (#6955)
## Summary

The `typename` argument to `NamedTuple` and `TypedDict` is a required
positional argument. We assumed as much, but panicked if it was provided
as a keyword argument or otherwise omitted. This PR handles the case
gracefully.

Closes https://github.com/astral-sh/ruff/issues/6953.
2023-08-28 21:03:18 +00:00
Charlie Marsh
d1ad20c9ea Avoid invalid fix for C417 with separate keys and values (#6954)
Closes https://github.com/astral-sh/ruff/issues/6951.
2023-08-28 16:49:40 -04:00
Charlie Marsh
ecca125f9a Use SourceCodeSnippet for more diagnostic messages (#6950)
## Summary

This ensures that we truncate the messages when they break over multiple
lines, etc.
2023-08-28 16:22:49 -04:00
Charlie Marsh
005e21a139 Add line-length to E501 documentation (#6949)
Closes https://github.com/astral-sh/ruff/issues/6908.
2023-08-28 14:45:30 -04:00
dependabot[bot]
e1db036f90 ci(deps): bump tj-actions/changed-files from 37 to 38 (#6946)
Co-authored-by: jackton1 <a
Co-authored-by: repo-ranger[bot] <!-- raw HTML omitted --> (<a
Co-authored-by: repo-ranger[bot] <!-- raw HTML omitted -->
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-28 16:50:38 +00:00
Zanie Blue
9ad67b0758 Add comment to ecoystem check re. ALL rules (#6943) 2023-08-28 11:28:36 -05:00
Jelle van der Waa
af61abc747 Re-use is_magic where possible (#6945)
## Summary

Use `is_magic` where possible

## Test Plan

Unit tests
2023-08-28 15:35:34 +00:00
Charlie Marsh
ec575188c4 Narrow the supported options on the format CLI (#6944)
## Summary

Ensures that we only show supported options:

<img width="1228" alt="Screen Shot 2023-08-28 at 11 03 16 AM"
src="https://github.com/astral-sh/ruff/assets/1309177/50fb7595-dc30-43d2-a7e4-c0103acc15b9">

For now, I'm not super focused on DRYing up the CLI.
2023-08-28 15:28:22 +00:00
Charlie Marsh
aea7500c1e Allow Locator#slice to take Ranged (#6922)
## Summary

As a small quality-of-life improvement, the locator can now slice like
`locator.slice(stmt)` instead of requiring
`locator.slice(stmt.range())`.

## Test Plan

`cargo test`
2023-08-28 11:08:39 -04:00
Charlie Marsh
58f5f27dc3 Add TOML files to SourceType (#6929)
## Summary

This PR adds a higher-level enum (`SourceType`) around `PySourceType` to
allow us to use the same detection path to handle TOML files. Right now,
we have ad hoc `is_pyproject_toml` checks littered around, and some
codepaths are omitting that logic altogether (like `add_noqa`). Instead,
we should always be required to check the source type and handle TOML
files as appropriate.

This PR will also help with our pre-commit capabilities. If we add
`toml` to pre-commit (to support `pyproject.toml`), pre-commit will
start to pass _other_ files to Ruff (along with `poetry.lock` and
`Pipfile` -- see
[identify](b59996304f/identify/extensions.py (L355))).
By detecting those files and handling those cases, we avoid attempting
to parse them as Python files, which would lead to pre-commit errors.
(We tried to add `toml` to pre-commit here
(https://github.com/astral-sh/ruff-pre-commit/pull/44), but had to
revert here (https://github.com/astral-sh/ruff-pre-commit/pull/45) as it
led to the pre-commit hook attempting to parse `poetry.lock` files as
Python files.)
2023-08-28 15:01:48 +00:00
Dhruv Manilawala
2893a9f6b5 Remove unused f-string error type (#6941) 2023-08-28 18:34:48 +05:30
Micha Reiser
60097bebcd Handle implicit strings in `can_omit_parentheses (#6940) 2023-08-28 12:20:29 +00:00
Dhruv Manilawala
9c98416b96 Avoid lexer infinite loop on invalid input (#6937)
## Summary

This PR fixes a bug which sends the lexer into infinite loop for an invalid input.
The code in question is `[1` where the nesting is never finished. This means
that the lexer will keep emitting the `Err` token forever.

## Test Plan

Add a test case which collects all the tokens from the lexer. This just
makes sure that it doesn't go into infinite loop.
2023-08-28 17:21:38 +05:30
Victor Hugo Gomes
99f4c6886e Format PatternMatchOr (#6905) 2023-08-28 08:09:17 +00:00
Micha Reiser
30ebf7fc86 Truncate ecosystem comment when necessary (#6899) 2023-08-28 07:58:39 +00:00
Chris Pryer
fa25dabf17 Add comments option to playground (#6911)
Co-authored-by: Micha Reiser <micha@reiser.io>
2023-08-28 07:26:23 +00:00
konsti
e615870659 Unify line size settings between ruff and the formatter (#6873) 2023-08-28 06:44:56 +00:00
Micha Reiser
a6aa16630d Move Configuration to ruff_workspace crate (#6920) 2023-08-28 06:21:35 +00:00
Chris Pryer
039694aaed Add LineSuffix reserved width (#6830)
Thanks for working on this.
2023-08-28 07:46:54 +02:00
Charlie Marsh
6bc1ba6d62 Use stdin for formatter when --stdin-filename is provided (#6926)
## Summary

Just making the formatter CLI more consistent with the linter -- e.g.,
we now use stdin on invocations like `cat foo.py | cargo run -p ruff_cli
-- format -- --stdin-filename=foo.py`, instead of _only_ relying on the
`-` file (and use the same helper as the linter to facilitate this).
2023-08-27 20:32:18 +00:00
Charlie Marsh
cd47368ae4 Use consistent formatting for user-facing formatter errors (#6925)
## Summary

This PR changes the alpha formatter CLI to use the same format for
errors as the linter, e.g.:

<img width="868" alt="Screen Shot 2023-08-27 at 4 03 30 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/9f3dea37-593b-4788-a0c0-e64bcf0d0560">
2023-08-27 20:22:06 +00:00
Charlie Marsh
a871714705 Collect result in format CLI (#6924) 2023-08-27 20:02:18 +00:00
Charlie Marsh
292fdd978e Remove unnecessary generic (#6923) 2023-08-27 15:55:23 -04:00
konsti
c2413dcd2c Add prototype of ruff format for projects (#6871)
**Summary** Add recursive formatting based on `ruff check` file
discovery for `ruff format`, as a prototype for the formatter alpha.
This allows e.g. `format ../projects/django/`. It's still lacking
support for any settings except line length.

Note just like the existing `ruff format` this will become part of the
production build, i.e. you'll be able to use it - hidden by default and
with a prominent warning - with `ruff format .` after the next release.

Error handling works in my manual tests (the colors do also work):

```
$  target/debug/ruff format scripts/
warning: `ruff format` is a work-in-progress, subject to change at any time, and intended for internal use only.
```
(the above changes `add_rule.py` where we have the wrong bin op
breaking)

```
$ target/debug/ruff format ../projects/django/
warning: `ruff format` is a work-in-progress, subject to change at any time, and intended for internal use only.
Failed to format /home/konsti/projects/django/tests/test_runner_apps/tagged/tests_syntax_error.py: source contains syntax errors: ParseError { error: UnrecognizedToken(Name { name: "syntax_error" }, None), offset: 131, source_path: "<filename>" }
```

```
$ target/debug/ruff format a
warning: `ruff format` is a work-in-progress, subject to change at any time, and intended for internal use only.
Failed to read /home/konsti/ruff/a/d.py: Permission denied (os error 13)
```

**Test Plan** Missing! I'm not sure if it's worth building tests at this
stage or how they should look like.
2023-08-27 19:12:18 +00:00
Charlie Marsh
059757a8c8 Implement Ranged on more structs (#6921)
Now that it's in `ruff_text_size`, we can use it in a few places that we
couldn't before.
2023-08-27 19:03:08 +00:00
Charlie Marsh
fc89976c24 Move Ranged into ruff_text_size (#6919)
## Summary

The motivation here is that this enables us to implement `Ranged` in
crates that don't depend on `ruff_python_ast`.

Largely a mechanical refactor with a lot of regex, Clippy help, and
manual fixups.

## Test Plan

`cargo test`
2023-08-27 14:12:51 -04:00
Zanie Blue
88c8bece38 Add PrefectHQ/prefect to ecosystem checks (#6918)
At least I will generally be familiar with the patterns over there
2023-08-27 11:07:50 -05:00
Charlie Marsh
d0b051e447 Fix ranges for global usages (#6917)
## Summary

The range of the usage from `Globals` should be the range of the
identifier, not the range of the full `global pandas` statement.

Closes https://github.com/astral-sh/ruff/issues/6914.
2023-08-27 15:27:07 +00:00
Charlie Marsh
381fc5b2a8 Increase ecosystem CI coverage to 40 projects (#6916)
## Summary

Roughly doubles the number of covered projects.
2023-08-27 11:04:02 -04:00
Micha Reiser
7c480236e0 Use dyn dispatch for any_over_* (#6912) 2023-08-27 15:54:01 +02:00
Micha Reiser
3f3494ad44 Implement ConfigProcessor on non-ref type (#6915) 2023-08-27 15:03:11 +02:00
Chris Pryer
f33277a057 Update playground README.md (#6909) 2023-08-27 10:54:42 +02:00
Micha Reiser
eae59cf088 Optional source map generation (#6894) 2023-08-26 18:00:43 +02:00
Charlie Marsh
15b73bdb8a Introduce AST nodes for PatternMatchClass arguments (#6881)
## Summary

This PR introduces two new AST nodes to improve the representation of
`PatternMatchClass`. As a reminder, `PatternMatchClass` looks like this:

```python
case Point2D(0, 0, x=1, y=2):
  ...
```

Historically, this was represented as a vector of patterns (for the `0,
0` portion) and parallel vectors of keyword names (for `x` and `y`) and
values (for `1` and `2`). This introduces a bunch of challenges for the
formatter, but importantly, it's also really different from how we
represent similar nodes, like arguments (`func(0, 0, x=1, y=2)`) or
parameters (`def func(x, y)`).

So, firstly, we now use a single node (`PatternArguments`) for the
entire parenthesized region, making it much more consistent with our
other nodes. So, above, `PatternArguments` would be `(0, 0, x=1, y=2)`.

Secondly, we now have a `PatternKeyword` node for `x=1` and `y=2`. This
is much more similar to the how `Keyword` is represented within
`Arguments` for call expressions.

Closes https://github.com/astral-sh/ruff/issues/6866.

Closes https://github.com/astral-sh/ruff/issues/6880.
2023-08-26 14:45:44 +00:00
Micha Reiser
ed1b4122d0 Use Codspeed for continous benchmarking (#6896) 2023-08-26 16:34:35 +02:00
Micha Reiser
9d77552e18 Add tab width option (#6848) 2023-08-26 12:29:58 +02:00
Charlie Marsh
f91bacbb94 Avoid PEP 604 upgrades that lead to invalid syntax (#6888)
Closes https://github.com/astral-sh/ruff/issues/6843.
2023-08-25 19:47:15 -04:00
Charlie Marsh
2883ae4d46 Insert space to avoid syntax error in RSE fixes (#6886)
Closes https://github.com/astral-sh/ruff/issues/6810.
2023-08-25 18:26:30 -04:00
Charlie Marsh
bd625b1928 Update PT007 docs to mention row-type (#6885)
Closes https://github.com/astral-sh/ruff/issues/6859.
2023-08-25 22:14:39 +00:00
Charlie Marsh
e0a40783ca Update PTH122 documentation to include Path.stem and Path.parent (#6884)
Closes https://github.com/astral-sh/ruff/issues/6846.
2023-08-25 22:11:56 +00:00
konsti
0e79074c31 Update to Rust 1.72 (#6874)
Update to [Rust
1.72](https://blog.rust-lang.org/2023/08/24/Rust-1.72.0.html), fixed the
failing lints.
2023-08-25 17:42:03 -04:00
Charlie Marsh
edb9b0c62a Use the formatter prelude in more files (#6882)
Removes a bunch of imports that are made redundant by the prelude.
2023-08-25 16:51:07 -04:00
Victor Hugo Gomes
91a780c771 Format PatternMatchClass (#6860) 2023-08-25 19:03:37 +00:00
Charlie Marsh
91880b8273 Bump version to 0.0.286 (#6876) 2023-08-25 14:59:26 -04:00
Zanie Blue
100904adb9 Avoid parsing other parts of a format specification if replacements are present (#6858)
Closes #6767
Replaces https://github.com/astral-sh/ruff/pull/6773 (this cherry-picks
some parts from there)
Alternative to the approach introduced in #6616 which added support for
placeholders in format specifications while retaining parsing of other
format specification parts.

The idea is that if there are placeholders in a format specification we
will not attempt to glean semantic meaning from the other parts of the
format specification we'll just extract all of the placeholders ignoring
other characters. The dynamic content of placeholders can drastically
change the meaning of the format specification in ways unknowable by
static analysis. This change prevents false analysis and will ensure
safety if we build other rules on top of this at the cost of missing
detection of some bad specifications.

Minor note: I've use "replacements" and "placeholders" interchangeably
but am trying to go with "placeholder" as I think it's a better term for
the static analysis concept here
2023-08-25 17:42:57 +00:00
Zanie Blue
0bac7bd114 Update RUF100 test to reflect applicability (#6877)
Broken on merge of #6822 due to new test cases conflicting with the
addition of an applicability to RUF100 in #6827
2023-08-25 16:57:59 +00:00
Zanie Blue
f2eb7bcacf Update ERA100 to apply to commented dictionary items with trailing comments (#6822)
Closes https://github.com/astral-sh/ruff/issues/6821

ERA100 was not raising on commented parts of dictionaries if it included
another comment (such as a noqa clause). In cases where this comment was
a noqa clause, RUF100 to be emitted since the noqa would have no effect.
Here, we update ERA100 to raise even when there are trailing comments.
This resolves the linked issue _and_ increases the scope of ERA100. We
could narrow the regular expression to only apply to noqa comments if we
do not want to expand ERA100 however I think this change is within the
spirit of the rule.
2023-08-25 11:02:31 -05:00
Micha Reiser
29a0c1003b Use BestFit layout even for attributes with a short name (#6872) 2023-08-25 17:47:02 +02:00
Micha Reiser
15b7525464 Rename parser goal 'All' to 'all' (#6867) 2023-08-25 12:00:57 +00:00
konsti
0b6dab5e3f Add jupyter notebook cell ids in 4.5+ if missing (#6853)
**Summary** See
https://github.com/astral-sh/ruff/issues/6834#issuecomment-1691202417

**Test Plan** Added a new notebook
2023-08-25 08:34:42 +00:00
David Szotten
1c66bb80b7 fix is_raw_string for multiple prefixes (#6865)
fix `is_raw_string` in the presence of other prefixes (like `rb"foo"`)

fixes #6864
2023-08-25 09:58:26 +02:00
Dhruv Manilawala
d1f07008f7 Rename Notebook related symbols (#6862)
This PR renames the following symbols:

* `PySourceType::Jupyter` -> `PySourceType::Ipynb`
* `SourceKind::Jupyter` -> `SourceKind::IpyNotebook`
* `JupyterIndex` -> `NotebookIndex`
2023-08-25 11:40:54 +05:30
Micha Reiser
61b2ffa8e8 Add assert test cases (#6855) 2023-08-25 07:51:55 +02:00
Charlie Marsh
1044d66c1c Add support for PatternMatchMapping formatting (#6836)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

Adds support for `PatternMatchMapping` -- i.e., cases like:

```python
match foo:
    case {"a": 1, "b": 2, **rest}:
        pass
```

Unfortunately, this node has _three_ kinds of dangling comments:

```python
{  # "open parenthesis comment"
   key: pattern,
   **  # end-of-line "double star comment"
   # own-line "double star comment"
   rest  # end-of-line "after rest comment"
   # own-line "after rest comment"
}
```

Some of the complexity comes from the fact that in `**rest`, `rest` is
an _identifier_, not a node, so we have to handle comments _after_ it as
dangling on the enclosing node, rather than trailing on `**rest`. (We
could change the AST to use `PatternMatchAs` there, which would be more
permissive than the grammar but not totally crazy -- `PatternMatchAs` is
used elsewhere to mean "a single identifier".)

Closes https://github.com/astral-sh/ruff/issues/6644.

## Test Plan

`cargo test`
2023-08-25 04:33:34 +00:00
Charlie Marsh
813d7da7ec Respect own-line leading comments before parenthesized nodes (#6820)
## Summary

This PR ensures that if an expression has an own-line leading comment
_before_ its open parentheses, we render it as such.

For example, given:

```python
[ # foo
    # bar
    ( # baz
        1
    )
]
```

On `main`, we format as:

```python
[  # foo
    (
        # bar
        # baz
        1
    )
]
```

As of this PR, we format as:

```python
[  # foo
    # bar
    (  # baz
        1
    )
]
```

## Test Plan

`cargo test`
2023-08-25 00:18:05 -04:00
Charlie Marsh
59e70896c0 Fix formatting of comments between function and arguments (#6826)
## Summary

We now format comments between a function and its arguments as dangling.
Like with other strange placements, I've biased towards preserving the
existing formatting, rather than attempting to reorder the comments.

Closes https://github.com/astral-sh/ruff/issues/6818.

## Test Plan

`cargo test`

Before:

| project      | similarity index |
|--------------|------------------|
| cpython      | 0.76050          |
| django       | 0.99820          |
| transformers | 0.99800          |
| twine        | 0.99876          |
| typeshed     | 0.99953          |
| warehouse    | 0.99615          |
| zulip        | 0.99729          |

After:

| project      | similarity index |
|--------------|------------------|
| cpython      | 0.76050          |
| django       | 0.99820          |
| transformers | 0.99800          |
| twine        | 0.99876          |
| typeshed     | 0.99953          |
| warehouse    | 0.99615          |
| zulip        | 0.99729          |
2023-08-25 04:06:56 +00:00
Charlie Marsh
f754ad5898 Handle bracketed comments on sequence patterns (#6801)
## Summary

This PR ensures that we handle bracketed comments on sequences, like `#
comment` here:

```python
match x:
    case [ # comment
        1, 2
    ]:
        pass
```

The handling is very similar to other, similar nodes, except that we do
need some special logic to determine whether the sequence is
parenthesized, similar to our logic for tuples.

## Test Plan

`cargo test`
2023-08-25 04:03:27 +00:00
Charlie Marsh
474e8fbcd4 Format all attribute dot comments manually (#6825)
## Summary

This PR modifies our formatting of comments around the `.` in an
attribute. Specifically, the goal here is to avoid _reordering_
comments, and the net effect is that we generally leave comments
where-they-are when dealing with comments between around the dot (which
you can also think of as comments between attributes).

All comments around the dot are now treated as dangling and formatted
manually, with the exception of end-of-line or parenthesized comments on
the value, like those marked as trailing here, which remain trailing:

```python
(
    (
        a # trailing end-of-line
        # trailing own-line
    ) # dangling before dot end-of-line
    .b # trailing end-of-line
)
```

Closes https://github.com/astral-sh/ruff/issues/6823.

## Test Plan

`cargo test`

Before:

| project      | similarity index |
|--------------|------------------|
| cpython      | 0.76050          |
| django       | 0.99820          |
| transformers | 0.99800          |
| twine        | 0.99876          |
| typeshed     | 0.99953          |
| warehouse    | 0.99615          |
| zulip        | 0.99729          |

After:

| project      | similarity index |
|--------------|------------------|
| cpython      | 0.76050          |
| django       | 0.99820   |
| transformers | 0.99800          |
| twine        | 0.99876          |
| typeshed     | 0.99953          |
| warehouse    | 0.99615          |
| zulip        | 0.99729          |
2023-08-25 03:50:56 +00:00
Charlie Marsh
6f23469e00 Handle pattern parentheses in FormatPattern (#6800)
## Summary

This PR fixes the duplicate-parenthesis problem that's visible in the
tests from https://github.com/astral-sh/ruff/pull/6799. The issue is
that we might have parentheses around the entire match-case pattern,
like in `(1)` here:

```python
match foo:
    case (1):
        y = 0
```

In this case, the inner expression (`1`) will _think_ it's
parenthesized, but we'll _also_ detect the parentheses at the case level
-- so they get rendered by the case, then again by the expression.
Instead, if we detect parentheses at the case level, we can force-off
the parentheses for the pattern using a design similar to the way we
handle parentheses on expressions.

Closes https://github.com/astral-sh/ruff/issues/6753.

## Test Plan

`cargo test`
2023-08-25 03:45:49 +00:00
Charlie Marsh
281ce56dc1 Make isort's detect-same-package behavior configurable (#6833)
## Summary

Our first-party import detection uses a heuristic that doesn't exist in
isort: if an import appears to be from within the same package as the
containing file, we mark it as first-party. For example, if you have a
directory `./foo/__init__.py`, and you import `from foo import bar` in
`./foo/baz.py`, we'll mark that as first-party. (See:
https://github.com/astral-sh/ruff/pull/1266.)

This is often unnecessary, and arguably should be removed (though it
does have some important use-cases that are otherwise unserved -- I
believe Dagster uses it to ensure that all packages mark imports from
within the same package as first-party, but not imports _across_
different first-party packages)... but it does exist, and it does help
in cases in which the `src` field is not properly configured.

This PR adds an option to turn off this behavior:

```toml
[tool.ruff.isort]
detect-same-package = false
```

This is being introduced to help codebases migrating over from isort
that may want more consistent behavior with their current sorting.

## Test Plan

`cargo test`
2023-08-24 14:09:26 -04:00
Zanie Blue
3bd199cdf6 Update function-call-in-argument-default (B008) to ignore arguments with immutable annotations (#6784)
Extends #6781 
Part of https://github.com/astral-sh/ruff/issues/3762
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->
Allows calls in argument defaults if the argument is annotated as an
immutable type to avoid false positives.

## Test Plan

<!-- How was it tested? -->

Snapshots
2023-08-24 11:12:59 -05:00
Harutaka Kawamura
948cd29b23 Skip serializing cell ID if it's None (#6851)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

Fix #6834 

## Test Plan

<!-- How was it tested? -->

Need tests?

---------

Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
2023-08-24 13:20:25 +00:00
Micha Reiser
1e7d1968b1 Printer: Slice based queue and stack (#6819) 2023-08-24 14:49:27 +02:00
Micha Reiser
8b46b71038 Fix parenthesizing of implicit strings (#6852) 2023-08-24 12:31:02 +00:00
Micha Reiser
1cd7790a8a Use BestFits for non-fluent attribute chains (#6817) 2023-08-24 14:09:25 +02:00
konsti
d376cb4c2a Improve formatter contributor docs (#6776)
The docs were out of date, and the new version incorporates some
feedback.

I tried to keep the language concise and the information ordered by how
early you need it, so people can get the relevant information quickly
before jumping into the code.

I did some minor format_dev changes for consistency in the docs.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2023-08-24 10:45:08 +00:00
Micha Reiser
04a9a8dd03 Maybe parenthesize long constants and names (#6816) 2023-08-24 09:47:57 +00:00
Micha Reiser
e4c13846e3 Fix Printer group modes (#6815) 2023-08-24 08:42:59 +02:00
Harutaka Kawamura
205d234856 Format PatternMatchStar (#6653) 2023-08-24 01:58:05 +00:00
Charlie Marsh
4889b84338 Document logger-objects setting in flake8-logging-format rules (#6832)
Closes https://github.com/astral-sh/ruff/issues/6764.
2023-08-24 00:18:51 +00:00
Charlie Marsh
847432cacf Avoid attempting to fix PT018 in multi-statement lines (#6829)
## Summary

These fixes will _always_ fail, so we should avoid trying to construct
them in the first place.

Closes https://github.com/astral-sh/ruff/issues/6812.
2023-08-23 19:09:34 -04:00
Charlie Marsh
9b6e008cf1 Remove remaining usages of try_set_fix (#6828)
There are just a few remaining usages of this deprecated method.
2023-08-23 22:36:09 +00:00
Charlie Marsh
39c6665ff9 Remove remaining usage of set_fix_from_edit (#6827)
This method is deprecated; we have one last usage, so removing it.
2023-08-23 18:25:39 -04:00
konsti
19a87c220a Document ecosystem_all_check.sh input json (#6824)
Document the origin of `github_search.jsonl` in `ecosystem_all_check.sh`
2023-08-23 20:08:52 +00:00
Zanie Blue
0688883404 Fix uncessary-coding-comment fix when there's leading content (#6775)
Closes https://github.com/astral-sh/ruff/issues/6756

Including whitespace, code, and continuations.
2023-08-23 12:00:06 -05:00
Zanie Blue
3bb638875f Fix native-literals handling of int literal with attribute access (#6792)
Closes https://github.com/astral-sh/ruff/issues/6788 by special casing
integer literals with attribute access — either retaining parenthesis
for literals with values (e.g. `int(7).denominator` to
`(7).denominator)` or leaving calls without values (e.g.
`int().denominator`) unchanged.
2023-08-23 11:22:05 -05:00
Charlie Marsh
26e63ab137 Remove lexing from flake8-pytest-style (#6795)
## Summary

Another drive-by change to remove unnecessary custom lexing. We just
need to know the parenthesized range, so we can use...
`parenthesized_range`. I've also updated `parenthesized_range` to
support nested parentheses.

## Test Plan

`cargo test`
2023-08-23 15:54:11 +00:00
Zanie Blue
417a1d0717 Update mutable-argument-default (B006) to use extend-immutable-calls when determining if annotations are immutable (#6781)
Part of https://github.com/astral-sh/ruff/issues/3762
2023-08-23 15:44:35 +00:00
Micha Reiser
34b2ae73b4 Extend BestFitting with mode (#6814) 2023-08-23 17:23:45 +02:00
Charlie Marsh
71c25e4f9d Implement FormatPatternMatchValue (#6799)
## Summary

This is effectively #6608, but with additional tests.

We aren't properly handling parenthesized patterns, but that needs to be
dealt with separately as it's somewhat involved.

Closes #6555
2023-08-23 14:01:14 +00:00
Micha Reiser
4bdd99f882 Fix: Re-add missing node start positions (#6780) 2023-08-23 09:59:36 +02:00
Charlie Marsh
1e6d1182bf Improve comment handling around PatternMatchAs (#6797)
## Summary

Follows up on
https://github.com/astral-sh/ruff/pull/6652#discussion_r1300871033 with
some modifications to the `PatternMatchAs` comment handling.
Specifically, any comments between the `as` and the end are now
formatted as dangling, and we now insert some newlines in the
appropriate places.

## Test Plan

`cargo test`
2023-08-23 04:48:20 +00:00
Charlie Marsh
d08f697a04 Remove lexing for colon-matching use cases (#6803)
It's much simpler to just search ahead for the first colon.
2023-08-23 04:44:51 +00:00
Charlie Marsh
4bc5eddf91 Handle open-parenthesis comments on match case (#6798)
## Summary

Ensures that we retain the open-parenthesis comment in cases like:
```python
match pattern_comments:
    case (  # leading
        only_leading
    ):
        ...
```

Previously, this was treated as a leading comment on `only_leading`.

## Test Plan

`cargo test`
2023-08-23 00:40:18 -04:00
Charlie Marsh
5f5de52aba Confine repeated-equality-comparison-target to names and attributes (#6802)
Empirically, Pylint does this, so seems reasonable to follow.
2023-08-23 03:56:37 +00:00
Tom Kuson
1cb1bd731c Extend repeated-equality-comparison-target to check for mixed orderings and Yoda conditions. (#6691) 2023-08-23 03:45:30 +00:00
Dhruv Manilawala
db2e548f4f Simplify ANN204 autofix to use Parameters range (#6793)
## Summary

This PR fixes the bug where the decorator parentheses weren't being considered
when computing the autofix for `ANN204`. The existing logic would only look
for balanced parentheses and not multiple pairs of parentheses.

The solution is to remove the logic to generate the autofix and use the
`Parameters` end range directly which includes the parentheses as well.

## Test Plan

Add test case for `ANN204` with decorator being called

fixes: #6790
2023-08-23 09:05:46 +05:30
Harutaka Kawamura
94f5f18ddb Format PatternMatchSequence (#6676) 2023-08-23 00:44:33 +00:00
Luc Khai Hai
c34a342ab4 Format PatternMatchAs (#6652)
## Summary

Add formatting for `PatternMatchAs`.

This closes #6641.

## Test Plan

Add tests for comments.
2023-08-22 23:58:15 +00:00
Charlie Marsh
42ff833d00 Remove comment lexing from isort (#6794)
## Summary

No need to lex to find comments -- we already know their locations via
`Indexer`.
2023-08-22 21:26:38 +00:00
Konrad Listwan-Ciesielski
e1f4438498 Add docs for DTZ007 (#6757) 2023-08-22 21:12:50 +00:00
Charlie Marsh
1acdec3e29 Add a note on __future__ imports and keep-runtime-typing to pyupgrade rules (#6746)
Closes https://github.com/astral-sh/ruff/issues/6740.
2023-08-22 17:04:00 -04:00
Charlie Marsh
ca2bb20063 Fallback to end-of-file if ends in trailing continuation (#6789)
## Summary

Given:

```python
def end_of_file():
    if False:
        return 1
    x = 2 \

```

Then when searching for the end of the `x = 2` statement, we'd reach a
panic as we'd hit the last line (`\\`) and abort, since the universal
iterator doesn't return trailing newlines. Instead, we should just use
the end of the file as the fallback.

Closes https://github.com/astral-sh/ruff/issues/6787.

## Test Plan

`cargo test`
2023-08-22 15:12:26 -04:00
Dhruv Manilawala
2e00983762 Avoid C417 for lambda with default and variadic parameters (#6752)
## Summary

Avoid `C417` for `lambda` with default and variadic parameters.

## Test Plan

`cargo test` and checking if it generates any autofix errors as test
cases
for `lambda` with default parameters already exists.

fixes: #6715
2023-08-23 00:38:08 +05:30
Dhruv Manilawala
fb7caf43c8 Update lexer tests to use snapshots (#6658)
## Summary

This PR updates the lexer tests to use the snapshot testing framework.
It also
makes the following changes:
* Remove the use of macros in the lexer tests
* Use `test_case` for EOL tests

## Test Plan

```
cargo test --package ruff_python_parser --lib --all-features -- lexer::tests --no-capture
```
2023-08-22 18:23:19 +00:00
konsti
e53bf25616 Update formatter ecosystem checks revisions (#6770)
With https://github.com/django/django/pull/17181 merged, this removes an
odd edge case (tuple expression statements aka bogus trailing commas
after statements that turn them into a tuple without you noticing) that
we don't want to care about because the input code is ~wrong from the
similarity index. I've took this opportunity to update the revisions of
all projects we test.

main

| project      | similarity index |
|--------------|------------------|
| cpython      | 0.75477          |
| django       | 0.99814          |
| transformers | 0.99621          |
| twine        | 0.99876          |
| typeshed     | 0.99953          |
| warehouse    | 0.99601          |
| zulip        | 0.99727          |

this PR

| project      | similarity index |
|--------------|------------------|
| cpython      | 0.75996          |
| django       | 0.99819          |
| transformers | 0.99622          |
| twine        | 0.99876          |
| typeshed     | 0.99953          |
| warehouse    | 0.99607          |
| zulip        | 0.99729          |
2023-08-22 14:19:10 -04:00
Charlie Marsh
214eb707a6 Parenthesize expressions prior to LibCST parsing (#6742)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

This PR adds a utility for transforming expressions via LibCST that
automatically wraps the expression in parentheses, applies a
user-provided transformation, then strips the parentheses from the
generated code. LibCST can't parse arbitrary expression ranges, since
some expressions may require parenthesization in order to be parsed
properly. For example:

```python
option = (
    '{name}={value}'
    .format(nam=name, value=value)
)
```

In this case, the expression range is:

```python
'{name}={value}'
    .format(nam=name, value=value)
```

Which isn't valid on its own. So, instead, we add "fake" parentheses
around the expression.

We were already doing this in a few places, so this is mostly
formalizing and DRYing up that pattern.

Closes https://github.com/astral-sh/ruff/issues/6720.
2023-08-22 17:45:05 +00:00
Zanie Blue
5c1f7fd5dd Add networkx to conventional aliases (#6778)
Closes https://github.com/astral-sh/ruff/issues/6763
2023-08-22 11:49:04 -05:00
Charlie Marsh
cc278c24e2 Allow up to two empty lines after top-level imports (#6777)
## Summary

For imports, we enforce that there's _at least_ one empty line after an
import (assuming the next statement is _not_ an import), but allow up to
two at the module level.

Closes https://github.com/astral-sh/ruff/issues/6760.

## Test Plan

`cargo test`
2023-08-22 12:27:40 -04:00
Charlie Marsh
558b56f8a8 Avoid fixing D200 for docstrings that end in escapes (#6779)
Appease the fuzzers! Closes
https://github.com/astral-sh/ruff/issues/6755.
2023-08-22 16:25:37 +00:00
Charlie Marsh
749da6589a Fix isolation groups for unused imports (#6774)
## Summary

The isolation group for unused imports was relying on
`checker.semantic().current_statement()`, which isn't valid for that
rule, since it runs over the _scope_, not the statement. Instead, we
need to lookup the isolation group based on the `NodeId` of the
statement.

Our tests didn't catch this, because we mostly have cases that look like
this:

```python
if TYPE_CHECKING:
    import shelve
    import importlib
```

In this case, the two fixes to remove the two unused imports are
considered overlapping (since we delete the _full_ line, and the two
_full_ lines touch, and we consider exactly-adjacent fixes to be
overlapping), and so they don't run in a single pass due to the
non-overlapping-fixes requirement. That is: the isolation groups aren't
required for this case. They are, however, required for cases like:

```python
if TYPE_CHECKING:
    import shelve

    import importlib
```

...where the fixes don't overlap.

Closes https://github.com/astral-sh/ruff/issues/6758.

## Test Plan

`cargo test`
2023-08-22 11:55:27 -04:00
Charlie Marsh
d2eace3377 Prefer range_* edit methods (#6751) 2023-08-22 15:46:04 +00:00
Micha Reiser
ccac9681e1 Preserve yield parentheses (#6766) 2023-08-22 10:27:20 +00:00
Micha Reiser
b52cc84df6 Omit tuple parentheses in for statements except when absolutely necessary (#6765) 2023-08-22 12:18:59 +02:00
Micha Reiser
fec6fc2fab Preserve empty lines between try clause headers (#6759) 2023-08-22 11:50:28 +02:00
konsti
ba4c27598a Document IO Error (#6712)
`IOError` is special, it is not actually a lint but an error before
linting. I'm not entirely sure how to document it since it does not
match the general lint rule pattern (`Checks that the file can be read
in its entirety.` is imho worse).

I added the in my experience two most common reasons for io errors on
unix systems and linked two tutorials on how to fix them.

See https://github.com/astral-sh/ruff/issues/2646

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2023-08-22 11:46:18 +02:00
Victor Hugo Gomes
0f9ccfcad9 Format PatternMatchSingleton (#6741) 2023-08-22 08:23:47 +02:00
Charlie Marsh
fa32cd9b6f Truncate some messages in diagnostics (#6748)
## Summary

I noticed this in the ecosystem CI check from
https://github.com/astral-sh/ruff/pull/6742. If we include source code
directly in a diagnostic, we need to be careful to avoid rendering
multi-line diagnostics or even excessively long diagnostics.

## Test Plan

`cargo test`
2023-08-21 23:46:24 -04:00
Victor Hugo Gomes
0aad0c41f6 [pylint] Implement no-self-use (R6301) (#6574) 2023-08-22 03:44:38 +00:00
Charlie Marsh
424b8d4ad2 Use a single node hierarchy to track statements and expressions (#6709)
## Summary

This PR is a follow-up to the suggestion in
https://github.com/astral-sh/ruff/pull/6345#discussion_r1285470953 to
use a single stack to store all statements and expressions, rather than
using separate vectors for each, which gives us something closer to a
full-fidelity chain. (We can then generalize this concept to include all
other AST nodes too.)

This is in part made possible by the removal of the hash map from
`&Stmt` to `StatementId` (#6694), which makes it much cheaper to store
these using a single interface (since doing so no longer introduces the
requirement that we hash all expressions).

I'll follow-up with some profiling, but a few notes on how the data
requirements have changed:

- We now store a `BranchId` for every expression, not just every
statement, so that's an extra `u32`.
- We now store a single `NodeId` on every snapshot, rather than separate
`StatementId` and `ExpressionId` IDs, so that's one fewer `u32` for each
snapshot.
- We're probably doing a few more lookups in general, since any calls to
`current_statement()` etc. now have to iterate up the node hierarchy
until they identify the first statement.

## Test Plan

`cargo test`
2023-08-21 21:32:57 -04:00
Charlie Marsh
abc5065fc7 Avoid E231 if comma is at end-of-line (#6747)
## Summary

I don't know how this could come up in valid Python, but anyway...

Closes https://github.com/astral-sh/ruff/issues/6738.
2023-08-21 20:47:20 -04:00
Victor Hugo Gomes
37f4920e1e Don't trigger eq-without-hash when __hash__ is explicitly set to None (#6739) 2023-08-21 23:51:21 +00:00
Charlie Marsh
c0df99b965 Avoid attempting to fix unconventional submodule imports (#6745)
## Summary

Avoid attempting to rewrite `import matplotlib.pyplot` as `import
matplotlib.pyplot as plt`. We can't support these right now, since we
don't track references at the attribute level (like
`matplotlib.pyplot`).

Closes https://github.com/astral-sh/ruff/issues/6719.
2023-08-21 23:45:32 +00:00
Charlie Marsh
7650c6ee45 Support C419 autofixes for set comprehensions (#6744)
Closes https://github.com/astral-sh/ruff/issues/6713.
2023-08-21 23:41:13 +00:00
Charlie Marsh
7b14d17e39 Ignore star imports when importing symbols in fixes (#6743)
## Summary

Given:

```python
from sys import *

exit(0)
```

We can't add `exit` to `from sys import *`, so we should just ignore it.
Ideally, we'd just resolve `exit` in the first place (since it's
imported from `from sys import *`), but as long as we don't support
wildcard imports, this is more consistent.

Closes https://github.com/astral-sh/ruff/issues/6718.

## Test Plan

`cargo test`
2023-08-21 23:31:30 +00:00
Charlie Marsh
4678f7dafe Remove parenthesis lexing in RSE102 (#6732)
## Summary

Now that we have an `Arguments` node, we can just use the range of the
arguments directly to find the parentheses in `raise Error()`.
2023-08-21 20:59:06 +00:00
konsti
b182368008 Simplify suite formatting (#6722)
Avoid the nesting in a macro by using the new `WithNodeLevel` to
`PyFormatter` deref. No changes otherwise.

I wanted to follow this up with quickly fixing the typeshed empty line
rules but they turned out a lot more complex than i had anticipated.
2023-08-21 21:01:51 +02:00
Charlie Marsh
e032fbd2e7 Remove remove_super_arguments (#6735)
Now that we have an `Arguments` node, we can use it directly to get the
range.
2023-08-21 13:04:07 -04:00
dependabot[bot]
575b77aa52 ci(deps): bump cloudflare/wrangler-action from 3.0.2 to 3.1.0 (#6736)
Bumps
[cloudflare/wrangler-action](https://github.com/cloudflare/wrangler-action)
from 3.0.2 to 3.1.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/cloudflare/wrangler-action/releases">cloudflare/wrangler-action's
releases</a>.</em></p>
<blockquote>
<h2>v3.1.0</h2>
<h3>Minor Changes</h3>
<ul>
<li>
<p><a
href="https://redirect.github.com/cloudflare/wrangler-action/pull/154">#154</a>
<a
href="3f40637a1c"><code>3f40637</code></a>
Thanks <a
href="https://github.com/JacobMGEvans"><code>@​JacobMGEvans</code></a>!
- feat: Quiet mode
Some of the stderr, stdout, info &amp; groupings can be a little noisy
for some users and use cases.
This feature allows for a option to be passed 'quiet: true' this would
significantly reduce the noise.</p>
<p>There will still be output that lets the user know Wrangler Installed
and Wrangler Action completed successfully.
Any failure status will still be output to the user as well, to prevent
silent failures.</p>
<p>resolves <a
href="https://redirect.github.com/cloudflare/wrangler-action/issues/142">#142</a></p>
</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/cloudflare/wrangler-action/blob/main/CHANGELOG.md">cloudflare/wrangler-action's
changelog</a>.</em></p>
<blockquote>
<h2>3.1.0</h2>
<h3>Minor Changes</h3>
<ul>
<li>
<p><a
href="https://redirect.github.com/cloudflare/wrangler-action/pull/154">#154</a>
<a
href="3f40637a1c"><code>3f40637</code></a>
Thanks <a
href="https://github.com/JacobMGEvans"><code>@​JacobMGEvans</code></a>!
- feat: Quiet mode
Some of the stderr, stdout, info &amp; groupings can be a little noisy
for some users and use cases.
This feature allows for a option to be passed 'quiet: true' this would
significantly reduce the noise.</p>
<p>There will still be output that lets the user know Wrangler Installed
and Wrangler Action completed successfully.
Any failure status will still be output to the user as well, to prevent
silent failures.</p>
<p>resolves <a
href="https://redirect.github.com/cloudflare/wrangler-action/issues/142">#142</a></p>
</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="c25aadc965"><code>c25aadc</code></a>
Automatic compilation</li>
<li><a
href="fcf648c789"><code>fcf648c</code></a>
Merge pull request <a
href="https://redirect.github.com/cloudflare/wrangler-action/issues/158">#158</a>
from cloudflare/changeset-release/main</li>
<li><a
href="fcbabec21e"><code>fcbabec</code></a>
Version Packages</li>
<li><a
href="0aa12f0c2b"><code>0aa12f0</code></a>
Merge pull request <a
href="https://redirect.github.com/cloudflare/wrangler-action/issues/154">#154</a>
from cloudflare/jacobmgevans/silence-mode</li>
<li><a
href="ad7441b6ad"><code>ad7441b</code></a>
Merge pull request <a
href="https://redirect.github.com/cloudflare/wrangler-action/issues/157">#157</a>
from EstebanBorai/main</li>
<li><a
href="3f40637a1c"><code>3f40637</code></a>
Quiet feature</li>
<li><a
href="4132892387"><code>4132892</code></a>
fix: use <code>wrangler@3.5.1</code> by default</li>
<li><a
href="62ce9d23a3"><code>62ce9d2</code></a>
Merge pull request <a
href="https://redirect.github.com/cloudflare/wrangler-action/issues/155">#155</a>
from ethanppl/fix-readme</li>
<li><a
href="f089b0a195"><code>f089b0a</code></a>
Update README.md</li>
<li><a
href="4318a2fb97"><code>4318a2f</code></a>
Fix examples in README.md</li>
<li>Additional commits viewable in <a
href="https://github.com/cloudflare/wrangler-action/compare/v3.0.2...v3.1.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=cloudflare/wrangler-action&package-manager=github_actions&previous-version=3.0.2&new-version=3.1.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-21 16:35:59 +00:00
Micha Reiser
17a26e6ff3 Fix fmt:skip for function with return type (#6733) 2023-08-21 17:45:23 +02:00
Charlie Marsh
d5a51b4e45 Allow ctypes.WinError() in flake8-raise (#6731)
Closes https://github.com/astral-sh/ruff/issues/6730.
2023-08-21 14:57:34 +00:00
Charlie Marsh
83f68891e0 Allow next in FBT exclusions (#6729)
Closes https://github.com/astral-sh/ruff/issues/6711.
2023-08-21 14:56:38 +00:00
konsti
aafde6db28 Remove some indexing (#6728)
**Summary** A common pattern in the code used to be
```rust
if statements.len() != 1 {
    return;
}
use_single_entry(statements[0])?;
```
which can be better expressed as
```rust
let [statement] = statements else {
    return;
};
use_single_entry(statements)?;
```

Direct indexing can cause panics if you don't manually take care of
checking the length, while matching (such as if-let or let-else) can
never panic.

This isn't a complete refactor, i've just removed some of the obvious
cases. I've specifically looked for `.len() != 1` and fixed those.

**Test Plan** No functional changes
2023-08-21 16:56:15 +02:00
Charlie Marsh
2405536d03 Remove unnecessary LibCST usage in key-in-dict (#6727)
## Summary

We're using LibCST to ensure that we return the full parenthesized range
of an expression, for display purposes. We can just use
`parenthesized_range` which is more efficient and removes one LibCST
dependency.

## Test Plan

`cargo test`
2023-08-21 10:32:09 -04:00
Micha Reiser
f017555d53 Parenthesize NamedExpr if target breaks (#6714) 2023-08-21 16:29:26 +02:00
Charlie Marsh
be96e0041a Accept empty inner calls in C414 (#6725)
Closes https://github.com/astral-sh/ruff/issues/6716.
2023-08-21 14:05:09 +00:00
Harutaka Kawamura
3c2dd5e42e Remove confusing comment on get_parametrize_name_range (#6724) 2023-08-21 08:52:48 -04:00
Micha Reiser
8b347cdaa9 Simplify IfRequired needs parentheses condition (#6678) 2023-08-21 07:11:31 +00:00
Tom Kuson
2a8d24dd4b Format function and class definitions into a single line if its body is an ellipsis (#6592) 2023-08-21 09:02:23 +02:00
Charlie Marsh
bb5fbb1b5c Use simple lexer for argument removal (#6710) 2023-08-21 04:16:29 +00:00
Harutaka Kawamura
086e11087f [flake8-pytest-style] Autofix PT014 (#6698) 2023-08-21 03:45:12 +00:00
Charlie Marsh
1b7e4a12a9 Refactor remove_unused_variable to take &Binding (#6707) 2023-08-20 15:50:57 +00:00
Charlie Marsh
da1697121e Add BranchId to the model snapshot (#6706)
This _probably_ never matters given the set of rules we support and in
fact I'm having trouble thinking of a test-case for it, but it's
definitely incorrect _not_ to pass on the `BranchId` here.
2023-08-20 15:35:49 +00:00
Harutaka Kawamura
419615f29b Add docs for E275, E231, E251, and E252 (#6700) 2023-08-20 14:51:50 +00:00
Charlie Marsh
a742a562fd Ignore multi-comparisons in repeated-equality-comparison-target (#6705)
Given `foo == "a" == "b" or foo == "c"`, we were suggesting `foo in
{"a", "b", "c"}`.
2023-08-20 14:41:10 +00:00
Harutaka Kawamura
129b19050a Refactor flake8_pytest_style/rules/parametrize.rs (#6703) 2023-08-20 14:30:26 +00:00
Konrad Listwan-Ciesielski
0dc23da1d0 Add docs for DTZ011 and DTZ012 (#6688) 2023-08-20 10:21:10 -04:00
Harutaka Kawamura
c62e544cba Add doc for E999 (#6699) 2023-08-20 14:14:22 +00:00
Charlie Marsh
7e9023b6f8 Use typing_extensions.TypeAlias for PYI026 fixes on pre-3.10 (#6696)
Closes https://github.com/astral-sh/ruff/issues/6695.
2023-08-19 22:16:44 +00:00
Harutaka Kawamura
a489b96a65 [flake8-pie] Implement unnecessary-range-start (PIE808) (#6690) 2023-08-19 21:59:11 +00:00
Charlie Marsh
17af12e57c Add branch detection to the semantic model (#6694)
## Summary

We have a few rules that rely on detecting whether two statements are in
different branches -- for example, different arms of an `if`-`else`.
Historically, the way this was implemented is that, given two statement
IDs, we'd find the common parent (by traversing upwards via our
`Statements` abstraction); then identify branches "manually" by matching
the parents against `try`, `if`, and `match`, and returning iterators
over the arms; then check if there's an arm for which one of the
statements is a child, and the other is not.

This has a few drawbacks:

1. First, the code is generally a bit hard to follow (Konsti mentioned
this too when working on the `ElifElseClause` refactor).

2. Second, this is the only place in the codebase where we need to go
from `&Stmt` to `StatementID` -- _everywhere_ else, we only need to go
in the _other_ direction. Supporting these lookups means we need to
maintain a mapping from `&Stmt` to `StatementID` that includes every
`&Stmt` in the program. (We _also_ end up maintaining a `depth` level
for every statement.) I'd like to get rid of these requirements to
improve efficiency, reduce complexity, and enable us to treat AST modes
more generically in the future. (When I looked at adding the `&Expr` to
our existing statement-tracking infrastructure, maintaining a hash map
with all the statements noticeably hurt performance.)

The solution implemented here instead makes branches a first-class
concept in the semantic model. Like with `Statements`, we now have a
`Branches` abstraction, where each branch points to its optional parent.
When we store statements, we store the `BranchID` alongside each
statement. When we need to detect whether two statements are in the same
branch, we just realize each statement's branch path and compare the
two. (Assuming that the two statements are in the same scope, then
they're on the same branch IFF one branch path is a subset of the other,
starting from the top.) We then add some calls to the visitor to push
and pop branches in the appropriate places, for `if`, `try`, and `match`
statements.

Note that a branch is not 1:1 with a statement; instead, each branch is
closer to a suite, but not _every_ suite is a branch. For example, each
arm in an `if`-`elif`-`else` is a branch, but the `else` in a `for` loop
is not considered a branch.

In addition to being much simpler, this should also be more efficient,
since we've shed the entire `&Stmt` hash map, plus the `depth` that we
track on `StatementWithParent` in favor of a single `Option<BranchID>`
on `StatementWithParent` plus a single vector for all branches. The
lookups should be faster too, since instead of doing a bunch of jumps
around with the hash map + repeated recursive calls to find the common
parents, we instead just do a few simple lookups in the `Branches`
vector to realize and compare the branch paths.

## Test Plan

`cargo test` -- we have a lot of coverage for this, which we inherited
from PyFlakes
2023-08-19 21:28:17 +00:00
Chris Pryer
648333b8b2 ruff_formatter crate doc comment fixes (#6677) 2023-08-19 17:42:02 +01:00
Charlie Marsh
3849fa0cf1 Rewrite yield-in-for-loop to avoid recursing over body (#6692)
## Summary

This is much simpler and avoids (1) multiple passes over the entire
function body, (2) requiring the rule to do its own binding tracking (we
can just use the semantic model), and (3) a usage of `StatementKey`.

In general, where we can, we should try to remove these kinds of custom
visitors that track name references, and instead rely on the semantic
model.

## Test Plan

`cargo test`
2023-08-19 11:25:29 -04:00
Victor Hugo Gomes
59e533047a Fix typo in ruff_python_formatter documentation (#6687)
## Summary

In the documentation was written `Javascript` but we are working with
`Python` here :)

## Test Plan

n/a
2023-08-18 19:16:09 -04:00
Charlie Marsh
053b1145f0 Avoid panic in unused arguments rule for parameter-free lambda (#6679)
## Summary

This was just a mistake in pattern-matching with no test coverage.

## Test Plan

`cargo test`
2023-08-18 18:29:31 +00:00
Charlie Marsh
6a5acde226 Make Parameters an optional field on ExprLambda (#6669)
## Summary

If a lambda doesn't contain any parameters, or any parameter _tokens_
(like `*`), we can use `None` for the parameters. This feels like a
better representation to me, since, e.g., what should the `TextRange` be
for a non-existent set of parameters? It also allows us to remove
several sites where we check if the `Parameters` is empty by seeing if
it contains any arguments, so semantically, we're already trying to
detect and model around this elsewhere.

Changing this also fixes a number of issues with dangling comments in
parameter-less lambdas, since those comments are now automatically
marked as dangling on the lambda. (As-is, we were also doing something
not-great whereby the lambda was responsible for formatting dangling
comments on the parameters, which has been removed.)

Closes https://github.com/astral-sh/ruff/issues/6646.

Closes https://github.com/astral-sh/ruff/issues/6647.

## Test Plan

`cargo test`
2023-08-18 15:34:54 +00:00
Micha Reiser
ea72d5feba Refactor SourceKind to store file content (#6640) 2023-08-18 13:45:38 +00:00
Charlie Marsh
2aeb27334d Avoid cloning source code multiple times (#6629)
## Summary

In working on https://github.com/astral-sh/ruff/pull/6628, I noticed
that we clone the source code contents, potentially multiple times,
prior to linting. The issue is that `SourceKind::Python` takes a
`String`, so we first have to provide it with a `String`. In the stdin
case, that means cloning. However, on top of this, we then have to clone
`source_kind.contents()` because `SourceKind` gets mutated. So for
stdin, we end up cloning twice. For non-stdin, we end up cloning once,
but unnecessarily (since the _contents_ don't get mutated, only the
kind).

This PR removes the `String` from `source_kind`, instead requiring that
we parse it out elsewhere. It reduces the number of clones down to 1 for
Jupyter Notebooks, and zero otherwise.
2023-08-18 09:32:18 -04:00
Micha Reiser
0cea4975fc Rename Comments methods (#6649) 2023-08-18 06:37:01 +00:00
Charlie Marsh
3ceb6fbeb0 Remove some unnecessary ampersands in the formatter (#6667) 2023-08-18 04:18:26 +00:00
Charlie Marsh
8e18f8018f Remove some trailing commas in write calls (#6666) 2023-08-18 00:14:44 -04:00
Charlie Marsh
8228429a70 Convert comment to rustdoc in placement.rs (#6665) 2023-08-18 04:11:38 +00:00
Charlie Marsh
1811312722 Improve with statement comment handling and expression breaking (#6621)
## Summary

The motivating code here was:

```python
with test as (
    # test
foo):
    pass
```

Which we were formatting as:

```python
with test as
# test
(foo):
    pass
```

`with` statements are oddly difficult. This PR makes a bunch of subtle
modifications and adds a more extensive test suite. For example, we now
only preserve parentheses if there's more than one `WithItem` _or_ a
trailing comma; before, we always preserved.

Our formatting is_not_ the same as Black, but here's a diff of our
formatted code vs. Black's for the `with.py` test suite. The primary
difference is that we tend to break parentheses when they contain
comments rather than move them to the end of the life (this is a
consistent difference that we make across the codebase):

```diff
diff --git a/crates/ruff_python_formatter/foo.py b/crates/ruff_python_formatter/foo.py
index 85e761080..31625c876 100644
--- a/crates/ruff_python_formatter/foo.py
+++ b/crates/ruff_python_formatter/foo.py
@@ -1,6 +1,4 @@
-with (
-    aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
-), aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:
+with aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa, aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:
     ...
     # trailing
 
@@ -16,28 +14,33 @@ with (
     # trailing
 
 
-with a, b:  # a  # comma  # c  # colon
+with (
+    a,  # a  # comma
+    b,  # c
+):  # colon
     ...
 
 
 with (
-    a as  # a  # as
-    # own line
-    b,  # b  # comma
+    a as (  # a  # as
+        # own line
+        b
+    ),  # b  # comma
     c,  # c
 ):  # colon
     ...  # body
     # body trailing own
 
-with (
-    a as  # a  # as
+with a as (  # a  # as
     # own line
-    bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb  # b
-):
+    bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
+):  # b
     pass
 
 
-with (a,):  # magic trailing comma
+with (
+    a,
+):  # magic trailing comma
     ...
 
 
@@ -47,6 +50,7 @@ with a:  # should remove brackets
 with aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb as c:
     ...
 
+
 with (
     # leading comment
     a
@@ -74,8 +78,7 @@ with (
 with (
     a  # trailing same line comment
     # trailing own line comment
-    as b
-):
+) as b:
     ...
 
 with (
@@ -87,7 +90,9 @@ with (
 with (
     a
     # trailing own line comment
-) as b:  # trailing as same line comment  # trailing b same line comment
+) as (  # trailing as same line comment
+    b
+):  # trailing b same line comment
     ...
 
 with (
@@ -124,18 +129,24 @@ with (  # comment
     ...
 
 with (  # outer comment
-    CtxManager1() as example1,  # inner comment
+    (  # inner comment
+        CtxManager1()
+    ) as example1,
     CtxManager2() as example2,
     CtxManager3() as example3,
 ):
     ...
 
-with CtxManager() as example:  # outer comment
+with (  # outer comment
+    CtxManager()
+) as example:
     ...
 
 with (  # outer comment
     CtxManager()
-) as example, CtxManager2() as example2:  # inner comment
+) as example, (  # inner comment
+    CtxManager2()
+) as example2:
     ...
 
 with (  # outer comment
@@ -145,7 +156,9 @@ with (  # outer comment
     ...
 
 with (  # outer comment
-    (CtxManager1()),  # inner comment
+    (  # inner comment
+        CtxManager1()
+    ),
     CtxManager2(),
 ) as example:
     ...
@@ -179,7 +192,9 @@ with (
 ):
     pass
 
-with a as (b):  # foo
+with a as (  # foo
+    b
+):
     pass
 
 with f(
@@ -209,17 +224,13 @@ with f(
 ) as b, c as d:
     pass
 
-with (
-    aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
-) as b:
+with aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb as b:
     pass
 
 with aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb as b:
     pass
 
-with (
-    aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
-) as b, c as d:
+with aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb as b, c as d:
     pass
 
 with (
@@ -230,6 +241,8 @@ with (
     pass
 
 with (
-    aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
-) as b, c as d:
+    aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
+    + bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb as b,
+    c as d,
+):
     pass
```

Closes https://github.com/astral-sh/ruff/issues/6600.
## Test Plan

Before:

| project      | similarity index |
|--------------|------------------|
| cpython      | 0.75473          |
| django       | 0.99804          |
| transformers | 0.99618          |
| twine        | 0.99876          |
| typeshed     | 0.74292          |
| warehouse    | 0.99601          |
| zulip        | 0.99727          |

After:

| project      | similarity index |
|--------------|------------------|
| cpython      | 0.75473          |
| django       | 0.99804          |
| transformers | 0.99618          |
| twine        | 0.99876          |
| typeshed     | 0.74292          |
| warehouse    | 0.99601          |
| zulip        | 0.99727          |

`cargo test`
2023-08-18 03:30:38 +00:00
Charlie Marsh
26bba11be6 Manually format comments around := in named expressions (#6634)
## Summary

Attaches comments around the `:=` operator in a named expression as
dangling, and formats them manually in the `named_expr.rs` formatter.

Closes https://github.com/astral-sh/ruff/issues/5695.

## Test Plan

`cargo test`
2023-08-18 03:10:45 +00:00
Shantanu
a128fe5148 Apply RUF017 when start is passed via position (#6664)
As discussed in
https://github.com/astral-sh/ruff/pull/6489#discussion_r1297858919.
Linking https://github.com/astral-sh/ruff/issues/5073
2023-08-17 20:10:07 -04:00
Zanie Blue
5892c691ea Bump version to 0.0.285 (#6660)
Requires
- https://github.com/astral-sh/ruff/pull/6655
- https://github.com/astral-sh/ruff/pull/6657
2023-08-17 15:46:28 -05:00
Zanie Blue
82e0a97b34 Clarify behavior of PLW3201 (#6657)
Otherwise it is unclear that violations will be raised for methods like
`_foo_`
2023-08-17 14:41:55 -05:00
Zanie Blue
a8d7bbae6f Remove experimental label from Jupyter docs (#6655) 2023-08-17 14:40:50 -05:00
Charlie Marsh
1050142a58 Expand expressions to include parentheses in E712 (#6575)
## Summary

This PR exposes our `is_expression_parenthesized` logic such that we can
use it to expand expressions when autofixing to include their
parenthesized ranges.

This solution has a few drawbacks: (1) we need to compute parenthesized
ranges in more places, which also relies on backwards lexing; and (2) we
need to make use of this in any relevant fixes.

However, I still think it's worth pursuing. On (1), the implementation
is very contained, so IMO we can easily swap this out for a more
performant solution in the future if needed. On (2), this improves
correctness and fixes some bad syntax errors detected by fuzzing, which
means it has value even if it's not as robust as an _actual_
`ParenthesizedExpression` node in the AST itself.

Closes https://github.com/astral-sh/ruff/issues/4925.

## Test Plan

`cargo test` with new cases that previously failed the fuzzer.
2023-08-17 15:51:09 +00:00
Charlie Marsh
db1c556508 Implement Ranged on more structs (#6639)
## Summary

I noticed some inconsistencies around uses of `.range.start()`, structs
that have a `TextRange` field but don't implement `Ranged`, etc.

## Test Plan

`cargo test`
2023-08-17 11:22:39 -04:00
Charlie Marsh
a70807e1e1 Expand NamedExpr range to include full range of parenthesized value (#6632)
## Summary

Given:

```python
if (
    x
    :=
    (  # 4
        y # 5
    )  # 6
):
    pass
```

It turns out the parser ended the range of the `NamedExpr` at the end of
`y`, rather than the end of the parenthesis that encloses `y`. This just
seems like a bug -- the range should be from the start of the name on
the left, to the end of the parenthesized node on the right.

## Test Plan

`cargo test`
2023-08-17 14:34:05 +00:00
dependabot[bot]
d9bb51dee4 ci(deps): bump cloudflare/wrangler-action from 3.0.0 to 3.0.2 (#6565)
Bumps
[cloudflare/wrangler-action](https://github.com/cloudflare/wrangler-action)
from 3.0.0 to 3.0.2.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/cloudflare/wrangler-action/releases">cloudflare/wrangler-action's
releases</a>.</em></p>
<blockquote>
<h2>v3.0.2</h2>
<h3>Patch Changes</h3>
<ul>
<li>
<p><a
href="https://redirect.github.com/cloudflare/wrangler-action/pull/147">#147</a>
<a
href="58f274b9f7"><code>58f274b</code></a>
Thanks <a
href="https://github.com/JacobMGEvans"><code>@​JacobMGEvans</code></a>!
- Added more error logging when a command fails to execute
Previously, we prevented any error logs from propagating too far to
prevent leaking of any potentially sensitive information. However, this
made it difficult for developers to debug their code.</p>
<p>In this release, we have updated our error handling to allow for more
error messaging from pre/post and custom commands. We still discourage
the use of these commands for secrets or other sensitive information,
but we believe this change will make it easier for developers to debug
their code.</p>
<p>Relates to <a
href="https://redirect.github.com/cloudflare/wrangler-action/issues/137">#137</a></p>
</li>
<li>
<p><a
href="https://redirect.github.com/cloudflare/wrangler-action/pull/147">#147</a>
<a
href="58f274b9f7"><code>58f274b</code></a>
Thanks <a
href="https://github.com/JacobMGEvans"><code>@​JacobMGEvans</code></a>!
- Adding Changesets</p>
</li>
<li>
<p><a
href="https://github.com/cloudflare/wrangler-action/blob/HEAD/#version-300">Version
3.0.0</a></p>
</li>
<li>
<p><a
href="https://github.com/cloudflare/wrangler-action/blob/HEAD/#version-200">Version
2.0.0</a></p>
</li>
</ul>
<h2>v3.0.1</h2>
<p>Automating Build &amp; Release</p>
<h2>What's Changed</h2>
<ul>
<li>Artifacts are now ESM supported with <code>.mjs</code></li>
<li>Update publish to deploy by <a
href="https://github.com/lrapoport-cf"><code>@​lrapoport-cf</code></a>
in <a
href="https://redirect.github.com/cloudflare/wrangler-action/pull/124">cloudflare/wrangler-action#124</a></li>
<li>Automatically add issues to workers-sdk GH project by <a
href="https://github.com/lrapoport-cf"><code>@​lrapoport-cf</code></a>
in <a
href="https://redirect.github.com/cloudflare/wrangler-action/pull/127">cloudflare/wrangler-action#127</a></li>
<li>Automate Action Release by <a
href="https://github.com/JacobMGEvans"><code>@​JacobMGEvans</code></a>
in <a
href="https://redirect.github.com/cloudflare/wrangler-action/pull/128">cloudflare/wrangler-action#128</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a
href="https://github.com/lrapoport-cf"><code>@​lrapoport-cf</code></a>
made their first contribution in <a
href="https://redirect.github.com/cloudflare/wrangler-action/pull/124">cloudflare/wrangler-action#124</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/cloudflare/wrangler-action/compare/3.0.0...3.0.1">https://github.com/cloudflare/wrangler-action/compare/3.0.0...3.0.1</a></p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/cloudflare/wrangler-action/blob/main/CHANGELOG.md">cloudflare/wrangler-action's
changelog</a>.</em></p>
<blockquote>
<h2>3.0.2</h2>
<h3>Patch Changes</h3>
<ul>
<li>
<p><a
href="https://redirect.github.com/cloudflare/wrangler-action/pull/147">#147</a>
<a
href="58f274b9f7"><code>58f274b</code></a>
Thanks <a
href="https://github.com/JacobMGEvans"><code>@​JacobMGEvans</code></a>!
- Added more error logging when a command fails to execute
Previously, we prevented any error logs from propagating too far to
prevent leaking of any potentially sensitive information. However, this
made it difficult for developers to debug their code.</p>
<p>In this release, we have updated our error handling to allow for more
error messaging from pre/post and custom commands. We still discourage
the use of these commands for secrets or other sensitive information,
but we believe this change will make it easier for developers to debug
their code.</p>
<p>Relates to <a
href="https://redirect.github.com/cloudflare/wrangler-action/issues/137">#137</a></p>
</li>
<li>
<p><a
href="https://redirect.github.com/cloudflare/wrangler-action/pull/147">#147</a>
<a
href="58f274b9f7"><code>58f274b</code></a>
Thanks <a
href="https://github.com/JacobMGEvans"><code>@​JacobMGEvans</code></a>!
- Adding Changesets</p>
</li>
<li>
<p><a
href="https://github.com/cloudflare/wrangler-action/blob/main/#version-300">Version
3.0.0</a></p>
</li>
<li>
<p><a
href="https://github.com/cloudflare/wrangler-action/blob/main/#version-200">Version
2.0.0</a></p>
</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="80501a7b4d"><code>80501a7</code></a>
Automatic compilation</li>
<li><a
href="3252711404"><code>3252711</code></a>
hotfix</li>
<li><a
href="2faabf36a2"><code>2faabf3</code></a>
Merge pull request <a
href="https://redirect.github.com/cloudflare/wrangler-action/issues/150">#150</a>
from cloudflare/jacobmgevans/changesets-tag-cli</li>
<li><a
href="b734d85d74"><code>b734d85</code></a>
Changesets needs a tag created for release</li>
<li><a
href="8ca2ff1612"><code>8ca2ff1</code></a>
Merge pull request <a
href="https://redirect.github.com/cloudflare/wrangler-action/issues/148">#148</a>
from cloudflare/changeset-release/main</li>
<li><a
href="7d08a8657e"><code>7d08a86</code></a>
Merge pull request <a
href="https://redirect.github.com/cloudflare/wrangler-action/issues/149">#149</a>
from cloudflare/jacobmgevans/changesets-needs-publish</li>
<li><a
href="e193627f19"><code>e193627</code></a>
Spoofing publish for Changeset Action</li>
<li><a
href="55b80c5f62"><code>55b80c5</code></a>
Version Packages</li>
<li><a
href="f61dc4d5a9"><code>f61dc4d</code></a>
Merge pull request <a
href="https://redirect.github.com/cloudflare/wrangler-action/issues/147">#147</a>
from cloudflare/revert-146-changeset-release/main</li>
<li><a
href="58f274b9f7"><code>58f274b</code></a>
Revert &quot;Version Packages&quot;</li>
<li>Additional commits viewable in <a
href="https://github.com/cloudflare/wrangler-action/compare/3.0.0...v3.0.2">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=cloudflare/wrangler-action&package-manager=github_actions&previous-version=3.0.0&new-version=3.0.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-17 09:11:04 -05:00
Zanie Blue
d0f2a8e424 Add support for nested replacements inside format specifications (#6616)
Closes https://github.com/astral-sh/ruff/issues/6442

Python string formatting like `"hello {place}".format(place="world")`
supports format specifications for replaced content such as `"hello
{place:>10}".format(place="world")` which will align the text to the
right in a container filled up to ten characters.

Ruff parses formatted strings into `FormatPart`s each of which is either
a `Field` (content in `{...}`) or a `Literal` (the normal content).
Fields are parsed into name and format specifier sections (we'll ignore
conversion specifiers for now).

There are a myriad of specifiers that can be used in a `FormatSpec`.
Unfortunately for linters, the specifier values can be dynamically set.
For example, `"hello {place:{align}{width}}".format(place="world",
align=">", width=10)` and `"hello {place:{fmt}}".format(place="world",
fmt=">10")` will yield the same string as before but variables can be
used to determine the formatting. In this case, when parsing the format
specifier we can't know what _kind_ of specifier is being used as their
meaning is determined by both position and value.

Ruff does not support nested replacements and our current data model
does not support the concept. Here the data model is updated to support
this concept, although linting of specifications with replacements will
be inherently limited. We could split format specifications into two
types, one without any replacements that we can perform lints with and
one with replacements that we cannot inspect. However, it seems
excessive to drop all parsing of format specifiers due to the presence
of a replacement. Instead, I've opted to parse replacements eagerly and
ignore their possible effect on other format specifiers. This will allow
us to retain a simple interface for `FormatSpec` and most syntax checks.
We may need to add some handling to relax errors if a replacement was
seen previously.

It's worth noting that the nested replacement _can_ also include a
format specification although it may fail at runtime if you produce an
invalid outer format specification. For example, `"hello
{place:{fmt:<2}}".format(place="world", fmt=">10")` is valid so we need
to represent each nested replacement as a full `FormatPart`.

## Test plan

Adding unit tests for `FormatSpec` parsing and snapshots for PLE1300
2023-08-17 09:07:30 -05:00
Charlie Marsh
1334232168 Introduce ExpressionRef (#6637)
## Summary

This PR revives the `ExpressionRef` concept introduced in
https://github.com/astral-sh/ruff/pull/5644, motivated by the change we
want to make in https://github.com/astral-sh/ruff/pull/6575 to narrow
the type of the expression that can be passed to `parenthesized_range`.

## Test Plan

`cargo test`
2023-08-17 10:07:16 -04:00
Micha Reiser
fa7442da2f Support fmt: skip on compound statements (#6593) 2023-08-17 06:05:41 +00:00
Micha Reiser
4dc32a00d0 Support fmt: skip for simple-statements and decorators (#6561) 2023-08-17 05:58:19 +00:00
Evan Rittenhouse
e3ecbe660e [ruff] Implement quadratic-list-summation rule (RUF017) (#6489)
## Summary

Adds `RUF017`. Closes #5073 

## Test Plan

`cargo t`
2023-08-16 23:13:05 -04:00
Harutaka Kawamura
8c3a8c4fc6 Support glob patterns for raises_require_match_for and raises_require_match_for (#6635)
## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

Support glob patterns for `raises_require_match_for` and
`raises_require_match_for`. Resolve #6473

## Test Plan

New tests + existing tests
2023-08-17 02:15:50 +00:00
Charlie Marsh
dcc7226685 Make lambda-assignment fix always-manual in class bodies (#6626)
## Summary

Related to https://github.com/astral-sh/ruff/issues/6620 (although that
will only be truly closed once we respect manual fixes on the CLI).
2023-08-16 21:24:48 -04:00
Charlie Marsh
036035bc50 Refactor literal-comparison and not-test rules (#6636)
## Summary

No behavior changes, but these need some refactoring to support
https://github.com/astral-sh/ruff/pull/6575 (namely, they need to take
the `ast::ExprCompare` or similar node instead of the attribute fields),
and I don't want to muddy that PR.

## Test Plan

`cargo test`
2023-08-17 01:02:30 +00:00
Charlie Marsh
97ae9e7433 Don't detect pandas#values for stores, deletes, or class accesses (#6631)
## Summary

Ensures we avoid cases like:

```python
x.values = 1
```

Since Pandas doesn't even expose a setter for that. We also avoid cases
like:

```python
print(self.values)
```

Since it's overwhelming likely to be a false positive.

Closes https://github.com/astral-sh/ruff/issues/6630.

## Test Plan

`cargo test`
2023-08-16 17:13:33 -04:00
Charlie Marsh
98b9f2e705 Respect .ipynb and .pyi sources when linting from stdin (#6628)
## Summary

When running Ruff from stdin, we were always falling back to the default
source type, even if the user specified a path (as is the case when
running from the LSP). This PR wires up the source type inference, which
means we now get the expected result when checking `.pyi` and `.ipynb`
files.

Closes #6627.

## Test Plan

Verified that `cat
crates/ruff/resources/test/fixtures/jupyter/valid.ipynb | cargo run -p
ruff_cli -- --force-exclude --no-cache --no-fix --isolated --select ALL
--stdin-filename foo.ipynb -` yielded the expected results (and differs
from the errors you get if you omit the filename).

Verified that `cat foo.pyi | cargo run -p ruff_cli -- --force-exclude
--no-cache --no-fix --format json --isolated --select TCH
--stdin-filename path/to/foo.pyi -` yielded no errors.
2023-08-16 20:33:59 +00:00
Zanie Blue
6253d8e2c8 Remove unused runtime string formatting logic (#6624)
In https://github.com/astral-sh/ruff/pull/6616 we are adding support for
nested replacements in format specifiers which makes actually formatting
strings infeasible without a great deal of complexity. Since we're not
using these functions (they just exist for runtime use in RustPython),
we can just remove them.
2023-08-16 17:38:33 +00:00
Charlie Marsh
0a5be74be3 Fix transformers checkout in scripts/formatter_ecosystem_checks.sh (#6622)
## Summary

In #6387, we accidentally added `git -C "$dir/django" checkout
95e4d6b81312fdd9f8ebf3385be1c1331168b5cf` as the transformers checkout
(duplicated line from the Django case). This PR fixes the SHA, and
spaces out the cases to make it more visible. I _think_ the net effect
here is that we've been formatting `main` on transformers, rather than
the SHA?
2023-08-16 12:25:46 -05:00
Micha Reiser
fdbb2fbdba Fix unreachable in playground (#6623) 2023-08-16 18:54:42 +02:00
Charlie Marsh
d0b8e4f701 Update Black tests (#6618)
## Summary

Pulls in some tests that we previously couldn't support

## Test Plan

`cargo test`
2023-08-16 15:05:51 +00:00
Charlie Marsh
12f3c4c931 Fix comment formatting for yielded tuples (#6603)
## Summary
Closes https://github.com/astral-sh/ruff/issues/6384, although I think
the issue was fixed already on main, for the most part.

The linked issue is around formatting expressions like:

```python
def test():
    (
        yield 
        #comment 1
        * # comment 2
        # comment 3
        test # comment 4
    )

```

On main, prior to this PR, we now format like:

```python
def test():
    (
        yield (
            # comment 1
            # comment 2
            # comment 3
            *test
        )  # comment 4
    )
```

Which strikes me as reasonable. (We can't test this, since it's a syntax
error after for our parser, despite being a syntax error in both cases
from CPython's perspective.)

Meanwhile, Black does:

```python
def test():
    (
        yield
        # comment 1
        *  # comment 2
        # comment 3
        test  # comment 4
    )
```

So our formatting differs in that we move comments between the star and
the expression above the star.

As of this PR, we also support formatting this input, which is valid:

```python
def test():
    (
        yield 
        #comment 1
        * # comment 2
        # comment 3
        test, # comment 4
        1
    )
```

Like:

```python
def test():
    (
        yield (
            # comment 1
            (
                # comment 2
                # comment 3
                *test,  # comment 4
                1,
            )
        )
    )
```

There were two fixes here: (1) marking starred comments as dangling and
formatting them properly; and (2) supporting parenthesized comments for
tuples that don't contain their own parentheses, as is often the case
for yielded tuples (previously, we hit a debug assert).

Note that this diff

## Test Plan
cargo test
2023-08-16 13:41:07 +00:00
Micha Reiser
7ee2ae8395 Estimate expected VecBuffer size (#6612) 2023-08-16 15:31:31 +02:00
Charlie Marsh
95f78821ad Fix parenthesized detection for tuples (#6599)
## Summary

This PR fixes our code for detecting whether a tuple has its own
parentheses, which is necessary when attempting to preserve parentheses.
As-is, we were getting some cases wrong, like `(a := 1), (b := 3))` --
the detection code inferred that this _was_ parenthesized, and so
wrapped the entire thing in an unnecessary set of parentheses.

## Test Plan

`cargo test`

Before:

| project      | similarity index |
|--------------|------------------|
| cpython      | 0.75472          |
| django       | 0.99804          |
| transformers | 0.99618          |
| twine        | 0.99876          |
| typeshed     | 0.74288          |
| warehouse    | 0.99601          |
| zulip        | 0.99727          |

After:
| project      | similarity index |
|--------------|------------------|
| cpython      | 0.75473          |
| django       | 0.99804 |
| transformers | 0.99618          |
| twine        | 0.99876          |
| typeshed     | 0.74288          |
| warehouse    | 0.99601          |
| zulip        | 0.99727          |
2023-08-16 13:20:48 +00:00
Micha Reiser
daac31d2b9 Make Buffer::write_element non-failable (#6613) 2023-08-16 15:13:07 +02:00
Charlie Marsh
86ccdcc9d9 Add support for multi-character operator tokens to SimpleTokenizer (#6563)
## Summary

Allows for proper lexing of tokens like `->`.

The main challenge is to ensure that our forward and backwards
representations are the same for cases like `===`. Specifically, we want
that to lex as `==` followed by `=` regardless of whether it's a
forwards or backwards lex. To do so, we identify the range of the
sequential characters (the full span of `===`), lex it forwards, then
return the last token.

## Test Plan

`cargo test`
2023-08-16 09:09:19 -04:00
Micha Reiser
e28858bb29 Fast path for ASCII only identifiers start (#6609) 2023-08-16 10:22:44 +02:00
Charlie Marsh
2d86e78bfc Allow top-level await in Jupyter notebooks (#6607)
## Summary

Top-level `await` is allowed in Jupyter notebooks (see:
[autoawait](https://ipython.readthedocs.io/en/stable/interactive/autoawait.html)).

Closes https://github.com/astral-sh/ruff/issues/6584.

## Test Plan

Had to test this manually. Created a notebook, verified that the `yield`
was flagged but the `await` was not.

<img width="868" alt="Screen Shot 2023-08-15 at 11 40 19 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/b2853651-30a6-4dc6-851c-9fe7f694b8e8">
2023-08-15 23:59:05 -04:00
Harutaka Kawamura
d9a81f4fbb [flake8-pytest-style] Implement duplicate parameterized fixture detection (PT014) (#6598) 2023-08-16 03:35:46 +00:00
Micha Reiser
897cce83b3 Call pattern formatting (#6594) 2023-08-16 08:31:25 +05:30
Anton Grouchtchak
9bf6713b76 Change rule count from 500 to 600 (#6605)
Fixes missing change from PR #6579.
2023-08-15 19:17:52 -05:00
Charlie Marsh
3f1658a25b Remove pylint's duplicate_value.rs (#6604)
This was moved to bugbear, but we forgot to delete the file.
2023-08-16 00:10:24 +00:00
Zanie Blue
097db2fcce Fix docs for PLW1508 (#6602) 2023-08-15 15:29:29 -05:00
Charlie Marsh
a3d4f08f29 Add general support for parenthesized comments on expressions (#6485)
## Summary

This PR adds support for parenthesized comments. A parenthesized comment
is a comment that appears within a parenthesis, but not within the range
of the expression enclosed by the parenthesis. For example, the comment
here is a parenthesized comment:

```python
if (
    # comment
    True
):
    ...
```

The parentheses enclose the `True`, but the range of `True` doesn’t
include the `# comment`.

There are at least two problems associated with parenthesized comments:
(1) associating the comment with the correct (i.e., enclosed) node; and
(2) formatting the comment correctly, once it has been associated with
the enclosed node.

The solution proposed here for (1) is to search for parentheses between
preceding and following node, and use open and close parentheses to
break ties, rather than always assigning to the preceding node.

For (2), we handle these special parenthesized comments in `FormatExpr`.
The biggest risk with this approach is that we forget some codepath that
force-disables parenthesization (by passing in `Parentheses::Never`).
I've audited all usages of that enum and added additional handling +
test coverage for such cases.

Closes https://github.com/astral-sh/ruff/issues/6390.

## Test Plan

`cargo test` with new cases.

Before:

| project      | similarity index |
|--------------|------------------|
| build        | 0.75623          |
| cpython      | 0.75472          |
| django       | 0.99804          |
| transformers | 0.99618          |
| typeshed     | 0.74233          |
| warehouse    | 0.99601          |
| zulip        | 0.99727          |

After:

| project      | similarity index |
|--------------|------------------|
| build        | 0.75623          |
| cpython      | 0.75472          |
| django       | 0.99804          |
| transformers | 0.99618          |
| typeshed     | 0.74237          |
| warehouse    | 0.99601          |
| zulip        | 0.99727          |
2023-08-15 18:59:18 +00:00
Micha Reiser
29c0b9f91c Use single lookup for leading, dangling, and trailing comments (#6589) 2023-08-15 17:39:45 +02:00
Harutaka Kawamura
81b1176f99 Fix PT005 doc (#6596) 2023-08-15 12:48:44 +00:00
Charlie Marsh
b1c4c7be69 Add trailing comma for single-element import-from groups (#6583)
## Summary

Unlike other statements, Black always adds a trailing comma if an
import-from statement breaks with a single import member. I believe this
is for compatibility with isort -- see
09f5ee3a19,
https://github.com/psf/black/issues/127, or
66648c528a/src/black/linegen.py (L1452)
for the current version.

## Test Plan

`cargo test`, notice that a big chunk of the compatibility suite is
removed.

Before:

| project      | similarity index |
|--------------|------------------|
| cpython      | 0.75472          |
| django       | 0.99804          |
| transformers | 0.99618          |
| twine        | 0.99876          |
| typeshed     | 0.74233          |
| warehouse    | 0.99601          |
| zulip        | 0.99727          |

After:

| project      | similarity index |
|--------------|------------------|
| cpython      | 0.75472          |
| django       | 0.99804          |
| transformers | 0.99618          |
| twine        | 0.99876          |
| typeshed     | 0.74260          |
| warehouse    | 0.99601          |
| zulip        | 0.99727          |
2023-08-15 07:15:33 -04:00
Tom Kuson
84d178a219 Use one line between top-level items if formatting a stub file (#6501)
Co-authored-by: Micha Reiser <micha@reiser.io>
2023-08-15 09:33:57 +02:00
Micha Reiser
455db84a59 Replace inline(always) with inline (#6590) 2023-08-15 08:58:11 +02:00
Micha Reiser
232b44a8ca Indent statements in suppressed ranges (#6507) 2023-08-15 08:00:35 +02:00
Harutaka Kawamura
e1e213decf Import pytest in flake8-pytest-style docs (#6580) 2023-08-14 23:08:15 -04:00
Charlie Marsh
5f709cd3e0 Bump rule count to 600+ in the docs (#6579)
My informal count yielded 679 rules as of yesterday.
2023-08-15 00:11:32 +00:00
Charlie Marsh
17e7eae2f9 Avoid unused argument rules when functions call locals() (#6578)
Closes https://github.com/astral-sh/ruff/issues/6576.
2023-08-14 19:48:20 -04:00
Charlie Marsh
7f7df852e8 Remove some extraneous newlines in Cargo.toml (#6577) 2023-08-14 23:39:41 +00:00
Nok Lam Chan
9a0d2f5afd Add regular expression example for per-file-ignores (#6573)
## Summary
Hi! This is my first PR to `ruff` and thanks for this amazing project.
While I am working on my project, I need to set different rules for my
`test/` folder and the main `src` package.

It's not immediately obvious that the
[`tool.ruff.per-file-ignores`](https://beta.ruff.rs/docs/settings/#per-file-ignores)
support regular expression. It is useful to set rules on directory
level. The PR add a simple example to make it clear this support regex.
2023-08-14 22:02:40 +00:00
Harutaka Kawamura
ebda5fcd99 Add PT002 ~ PT005 docs (#6521) 2023-08-14 21:29:03 +00:00
Charlie Marsh
b1870b2b16 Add deprecated unittest assertions to PT009 (#6572)
## Summary

This rule was missing `self.failIf` and friends.

## Test Plan

`cargo test`
2023-08-14 21:08:02 +00:00
Harutaka Kawamura
a51d1ac980 Add PT006 and PT007 docs (#6531) 2023-08-14 17:03:42 -04:00
Evan Rittenhouse
1a52b548e7 Ignore PERF203 if try contains loop control flow statements (#6536) 2023-08-14 20:47:37 +00:00
konsti
a3bf6d9cb7 Formatter ecosystem checks: Use twine instead of build (#6559) 2023-08-14 22:40:56 +02:00
Harutaka Kawamura
70696061cd [flake8-pytest-style] Implement pytest-unittest-raises-assertion (PT027) (#6554) 2023-08-14 20:25:23 +00:00
Charlie Marsh
cd634a9489 Expand documentation around flake8-type-checking rules for SQLAlchemy (#6570)
## Summary

Not addressing the root issue as much as improving the documentation.

Closes https://github.com/astral-sh/ruff/issues/6510.
2023-08-14 19:47:10 +00:00
Charlie Marsh
5ddf143cae Clarify FBT documentation and refine rule names (#6567)
Closes https://github.com/astral-sh/ruff/issues/6530.
2023-08-14 15:24:16 -04:00
Charlie Marsh
46862473b9 Omit NotImplementedError from TRY003 (#6568)
Closes https://github.com/astral-sh/ruff/issues/6528.
2023-08-14 18:24:44 +00:00
Charlie Marsh
96d310fbab Remove Stmt::TryStar (#6566)
## Summary

Instead, we set an `is_star` flag on `Stmt::Try`. This is similar to the
pattern we've migrated towards for `Stmt::For` (removing
`Stmt::AsyncFor`) and friends. While these are significant differences
for an interpreter, we tend to handle these cases identically or nearly
identically.

## Test Plan

`cargo test`
2023-08-14 13:39:44 -04:00
Micha Reiser
09c8b17661 fmt: off..on suppression comments (#6477) 2023-08-14 15:57:36 +00:00
qdegraaf
278a4f6e14 Formatter: Fix posonlyargs for expr_lambda (#6562) 2023-08-14 17:38:56 +02:00
Charlie Marsh
c3a9151eb5 Handle comments on open parentheses in with statements (#6515)
## Summary

This PR adds handling for comments on open parentheses in parenthesized
context managers. For example, given:

```python
with (  # comment
    CtxManager1() as example1,
    CtxManager2() as example2,
    CtxManager3() as example3,
):
    ...
```

We want to preserve that formatting. (Black does the same.) On `main`,
we format as:

```python
with (
    # comment
    CtxManager1() as example1,
    CtxManager2() as example2,
    CtxManager3() as example3,
):
    ...
```

It's very similar to how `StmtImportFrom` is handled.

Note that this case _isn't_ covered by the "parenthesized comment"
proposal, since this is a common on the statement that would typically
be attached to the first `WithItem`, and the `WithItem` _itself_ can
have parenthesized comments, like:

```python
with (  # comment
    (
        CtxManager1()  # comment
    ) as example1,
    CtxManager2() as example2,
    CtxManager3() as example3,
):
    ...
```

## Test Plan

`cargo test`

Confirmed no change in similarity score.
2023-08-14 15:11:03 +00:00
Charlie Marsh
3711f8ad59 Expand SimpleTokenizer to all keywords and single-character tokens (#6518)
## Summary

For #6485, I need to be able to use the `SimpleTokenizer` to lex the
space between any two adjacent expressions (i.e., the space between a
preceding and following node). This requires that we support a wider
range of keywords (like `and`, to connect the pieces of `x and y`), and
some additional single-character tokens (like `-` and `>`, to support
`->`). Note that the `SimpleTokenizer` does not support multi-character
tokens, so the `->` in a function signature is lexed as a `-` followed
by a `>` -- but this is fine for our purposes.
2023-08-14 10:35:31 -04:00
Charlie Marsh
a7cf8f0b77 Replace dynamic implicit concatenation detection with parser flag (#6513)
## Summary

In https://github.com/astral-sh/ruff/pull/6512, we added a flag to the
AST to mark implicitly-concatenated string expressions. This PR makes
use of that flag to remove the `is_implicit_concatenation` method.

## Test Plan

`cargo test`
2023-08-14 10:27:17 -04:00
Charlie Marsh
40407dcce5 Avoid marking inner-parenthesized comments as dangling bracket comments (#6517)
## Summary

The bracketed-end-of-line comment rule is meant to assign comments like
this as "immediately following the bracket":

```python
f(  # comment
    1
)
```

However, the logic was such that we treated this equivalently:

```python
f(
    (  # comment
        1
    )
)
```

This PR modifies the placement logic to ensure that we only skip the
opening bracket, and not any nested brackets. The above is now formatted
as:

```python
f(
    (
        # comment
        1
    )
)
```

(But will be corrected once we handle parenthesized comments properly.)

## Test Plan

`cargo test`

Confirmed no change in similarity score.
2023-08-14 09:52:19 -04:00
Charlie Marsh
f16e780e0a Add an implicit concatenation flag to string and bytes constants (#6512)
## Summary

Per the discussion in
https://github.com/astral-sh/ruff/discussions/6183, this PR adds an
`implicit_concatenated` flag to the string and bytes constant variants.
It's not actually _used_ anywhere as of this PR, but it is covered by
the tests.

Specifically, we now use a struct for the string and bytes cases, along
with the `Expr::FString` node. That struct holds the value, plus the
flag:

```rust
#[derive(Clone, Debug, PartialEq, is_macro::Is)]
pub enum Constant {
    Str(StringConstant),
    Bytes(BytesConstant),
    ...
}

#[derive(Clone, Debug, PartialEq, Eq)]
pub struct StringConstant {
    /// The string value as resolved by the parser (i.e., without quotes, or escape sequences, or
    /// implicit concatenations).
    pub value: String,
    /// Whether the string contains multiple string tokens that were implicitly concatenated.
    pub implicit_concatenated: bool,
}

impl Deref for StringConstant {
    type Target = str;
    fn deref(&self) -> &Self::Target {
        self.value.as_str()
    }
}

#[derive(Clone, Debug, PartialEq, Eq)]
pub struct BytesConstant {
    /// The bytes value as resolved by the parser (i.e., without quotes, or escape sequences, or
    /// implicit concatenations).
    pub value: Vec<u8>,
    /// Whether the string contains multiple string tokens that were implicitly concatenated.
    pub implicit_concatenated: bool,
}

impl Deref for BytesConstant {
    type Target = [u8];
    fn deref(&self) -> &Self::Target {
        self.value.as_slice()
    }
}
```

## Test Plan

`cargo test`
2023-08-14 13:46:54 +00:00
Micha Reiser
fc0c9507d0 Override fmt_dangling_comments for frequent nodes (#6551) 2023-08-14 15:29:05 +02:00
Tom Kuson
680d171ae5 Tweak documentation for FBT002 (#6556) 2023-08-14 09:22:48 -04:00
konsti
01eceaf0dc Format docstrings (#6452)
**Summary** Implement docstring formatting

**Test Plan** Matches black's `docstring.py` fixture exactly, added some
new cases for what is hard to debug with black and with what black
doesn't cover.

similarity index:

main:
zulip: 0.99702
django: 0.99784
warehouse: 0.99585
build: 0.75623
transformers: 0.99469
cpython: 0.75989
typeshed: 0.74853

this branch:

zulip: 0.99702
django: 0.99784
warehouse: 0.99585
build: 0.75623
transformers: 0.99464
cpython: 0.75517
typeshed: 0.74853

The regression in transformers is actually an improvement in a file they
don't format with black (they run `black examples tests src utils
setup.py conftest.py`, the difference is in hubconf.py). cpython doesn't
use black.

Closes #6196
2023-08-14 12:28:58 +00:00
Micha Reiser
910dbbd9b6 Printer: Reserve buffer upfront (#6550) 2023-08-14 12:15:36 +00:00
Micha Reiser
9584f613b9 Remove allow(pedantic) from formatter (#6549) 2023-08-14 14:02:06 +02:00
konsti
c39bcbadff Always run check-formatter-ecosystem on main (#6503)
This makes it easier to get the latest similarity numbers from main
2023-08-14 14:01:26 +02:00
Micha Reiser
24f42f0894 Printer: Remove unused state fields (#6548) 2023-08-14 11:08:00 +02:00
Micha Reiser
51ae47ad56 Remove lex and parsing from formatter benchmark (#6547) 2023-08-14 10:25:37 +02:00
Charlie Marsh
1a9536c4e2 Remove SemanticModel#find_binding (#6546)
## Summary

This method is almost never what you actually want, because it doesn't
respect Python's scoping semantics. For example, if you call this within
a class method, it will return class attributes, whereas Python actually
_skips_ symbols in classes unless the load occurs within the class
itself. I also want to move away from these kinds of dynamic lookups and
more towards `resolve_name`, which performs a lookup based on the stored
`BindingId` at the time of symbol resolution, and will make it much
easier for us to separate model building from linting in the near
future.

## Test Plan

`cargo test`
2023-08-14 00:09:05 -04:00
Charlie Marsh
bf4c6473c8 Remove unnecessary expr_name function (#6544) 2023-08-13 23:51:36 -04:00
Charlie Marsh
768686148f Add support for unions to our Python builtins type system (#6541)
## Summary

Fixes some TODOs introduced in
https://github.com/astral-sh/ruff/pull/6538. In short, given an
expression like `1 if x > 0 else "Hello, world!"`, we now return a union
type that says the expression can resolve to either an `int` or a `str`.
The system remains very limited, it only works for obvious primitive
types, and there's no attempt to do inference on any more complex
variables. (If any expression yields `Unknown` or `TypeError`, we
propagate that result throughout and abort on the client's end.)
2023-08-13 18:00:50 -04:00
Charlie Marsh
eb24f5a0b9 Add some additional projects to the ecosystem CI (#6542)
Adding five new projects. Some of these have seen issues filed, the
others, I just tabbed through our dependency pain and looked for some
reasonably-large projects that enabled rules beyond the default rule
set.
2023-08-13 21:15:54 +00:00
Charlie Marsh
446ceed1ad Support IfExp with dual string arms in invalid-envvar-value (#6538)
## Summary

Closes https://github.com/astral-sh/ruff/issues/6537. We need to improve
the `PythonType` algorithm, so this also documents some of its
limitations as TODOs.
2023-08-13 15:52:10 -04:00
Takuma Watanabe
8660e5057c Fix minor document errors (#6533)
## Summary

Fix minor errors in the sample codes of some rules.

## Test Plan

N/A (Just fix document typos.)
2023-08-13 13:35:30 -04:00
Konrad Listwan-Ciesielski
808e09180e Add docs for DTZ005 and DTZ006 (#6529)
Changes:
- Adds docs for `DTZ005`
- Adds docs for `DTZ006`

Related to: https://github.com/astral-sh/ruff/issues/2646
2023-08-12 21:29:32 -04:00
Presley Graham
dbf003fde4 importer: skip whitespace between comments at start of file (#6523)
## Summary

When adding an import, such as when fixing `I002`, ruff doesn't skip
whitespace between comments, but isort does. See this issue for more
detail: https://github.com/astral-sh/ruff/issues/6504

This change would fix that by skipping whitespace between comments in
`Insertion.start_of_file()`.

## Test Plan

I added a new test, `comments_and_newlines`, to verify this behavior. I
also ran `cargo test` and no existing tests broke. That being said, this
is technically a breaking change, as it's possible that someone was
relying on the previous behavior.
2023-08-12 16:37:56 -04:00
Charlie Marsh
010293ddcc Use a unified policy abstraction for the flake8-tidy-imports rules (#6527)
## Summary

Generalizes the abstractions for name matching introduced in
https://github.com/astral-sh/ruff/pull/6378 and applies them to the
existing `banned_api` rule, such that both rules have a uniform API and
implementation.

## Test Plan

`cargo test`
2023-08-12 16:32:09 -04:00
James Braza
4974964ad3 Clarifying target-version in flake8-future-annotations docs (#6520) 2023-08-12 19:01:03 +00:00
Charlie Marsh
b49c80f8c8 Use top-level semantic detection for E402 (#6526)
## Summary

Noticed in https://github.com/astral-sh/ruff/pull/6378. Given `import h;
import i`, we don't consider `import i` to be a "top-level" import for
E402 purposes, which is wrong. Similarly, we _do_ consider `import k` to
be a "top-level" import in:

```python
if __name__ == "__main__":
    import j; \
import k
```

Using the semantic detection, rather than relying on newline position,
fixes both cases.

## Test Plan

`cargo test`
2023-08-12 18:52:44 +00:00
Presley Graham
c03e2acadb [flake8-tidy-imports] Add TID253 (#6378)
## Summary

Add a new rule `TID253` (`banned-module-level-imports`), to ban a
user-specified list of imports from appearing at module level. This rule
doesn't exist in `flake8-tidy-imports`, so it's unique to Ruff. The
implementation is pretty similar to `TID251`.

Briefly discussed
[here](https://github.com/astral-sh/ruff/discussions/6370).

## Test Plan

Added a new test case, checking that inline imports are allowed and that
non-inline imports from the banned list are disallowed.
2023-08-12 18:45:34 +00:00
Charlie Marsh
a1da9da0ef Avoid JSON parse error on playground load (#6519)
## Summary

On page load, the playground very briefly flickers a JSON parse error.
Due to our use of `useDeferredValue`, we attempt to parse the empty JSON
string settings, since after `const initialized = ruffVersion != null;`
returns true, we get one render with the stale deferred value.

This PR refactors the state, such that we start by storing `null` for
the `Source`, and use the `Source` itself to determine initialization
status.

## Test Plan

Set a breakpoint in the `catch` path in `Editor`; verified that it no
longer triggers on load (but did on `main`).
2023-08-12 04:11:44 +00:00
Harutaka Kawamura
c6ad364d8b Add PT008 and PT009 docs (#6479) 2023-08-11 23:44:48 -04:00
Zanie Blue
5b47350c25 Document default behavior of W505 in setting (#6463)
Addresses https://github.com/astral-sh/ruff/discussions/6459
2023-08-11 16:41:31 -05:00
Charlie Marsh
e91caea490 Add test case for walrus operators in return types (#6438)
## Summary

Closes https://github.com/astral-sh/ruff/issues/6437.

## Test Plan

`cargo test`
2023-08-11 18:28:48 +00:00
Charlie Marsh
53246b725e Allow return type annotations to use their own parentheses (#6436)
## Summary

This PR modifies our logic for wrapping return type annotations.
Previously, we _always_ wrapped the annotation in parentheses if it
expanded; however, Black only exhibits this behavior when the function
parameters is empty (i.e., it doesn't and can't break). In other cases,
it uses the normal parenthesization rules, allowing nodes to bring their
own parentheses.

For example, given:

```python
def xxxxxxxxxxxxxxxxxxxxxxxxxxxx() -> Set[
    "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
]:
    ...

def xxxxxxxxxxxxxxxxxxxxxxxxxxxx(x) -> Set[
    "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
]:
    ...
```

Black will format as:

```python
def xxxxxxxxxxxxxxxxxxxxxxxxxxxx() -> (
    Set[
        "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
    ]
):
    ...


def xxxxxxxxxxxxxxxxxxxxxxxxxxxx(
    x,
) -> Set[
    "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
]:
    ...
```

Whereas, prior to this PR, Ruff would format as:

```python
def xxxxxxxxxxxxxxxxxxxxxxxxxxxx() -> (
    Set[
        "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
    ]
):
    ...


def xxxxxxxxxxxxxxxxxxxxxxxxxxxx(
    x,
) -> (
    Set[
        "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
    ]
):
    ...
```

Closes https://github.com/astral-sh/ruff/issues/6431.

## Test Plan

Before:

- `zulip`: 0.99702
- `django`: 0.99784
- `warehouse`: 0.99585
- `build`: 0.75623
- `transformers`: 0.99470
- `cpython`: 0.75988
- `typeshed`: 0.74853

After:

- `zulip`: 0.99724
- `django`: 0.99791
- `warehouse`: 0.99586
- `build`: 0.75623
- `transformers`: 0.99474
- `cpython`: 0.75956
- `typeshed`: 0.74857
2023-08-11 18:19:21 +00:00
Charlie Marsh
d616c9b870 Avoid omitting optional parentheses for argument-less parentheses (#6484)
## Summary

This PR fixes some misformattings around optional parentheses for
expressions.

I first noticed that we were misformatting this:

```python
return (
    unicodedata.normalize("NFKC", s1).casefold()
    == unicodedata.normalize("NFKC", s2).casefold()
)
```

The above is stable Black formatting, but we were doing:
```python
return unicodedata.normalize("NFKC", s1).casefold() == unicodedata.normalize(
    "NFKC", s2
).casefold()
```

Above, the "last" expression is a function call, so our
`can_omit_optional_parentheses` was returning `true`...

However, it turns out that Black treats function calls differently
depending on whether or not they have arguments -- presumedly because
they'll never split empty parentheses, and so they're functionally
non-useful. On further investigation, I believe this applies to all
parenthesized expressions. If Black can't split on the parentheses, it
doesn't leverage them when removing optional parentheses.

## Test Plan

Nice increase in similarity scores.

Before:

- `zulip`: 0.99702
- `django`: 0.99784
- `warehouse`: 0.99585
- `build`: 0.75623
- `transformers`: 0.99470
- `cpython`: 0.75989
- `typeshed`: 0.74853

After:

- `zulip`: 0.99705
- `django`: 0.99795
- `warehouse`: 0.99600
- `build`: 0.75623
- `transformers`: 0.99471
- `cpython`: 0.75989
- `typeshed`: 0.74853
2023-08-11 17:58:42 +00:00
Chris Pryer
7c4aa3948b Fix typo in MeasureMode comment (#6508) 2023-08-11 17:46:59 +00:00
konsti
0c9ded9d84 Use a faster diffing library for the formatter ecosystem checks (#6497)
**Summary** Some files seems notoriously slow in the formatter (secons in debug mode). This time was however almost exclusively spent in the diff algorithm to collect the similarity index, so i replaced that. I kept `similar` for printing actual diff to avoid rewriting that too, with the disadvantage that we now have to diff libraries in format_dev.

I used this PR to remove the spinner from tracing-indicatif and changed `flamegraph --perfdata perf.data` to `flamegraph --perfdata perf.data --no-inline` as the former wouldn't finish for me on release builds with debug info.
2023-08-11 15:51:54 +02:00
Dhruv Manilawala
c434bdd2bd Add formatting for MatchCase (#6360)
## Summary

This PR adds formatting support for `MatchCase` node with subs for the
`Pattern`
nodes.

## Test Plan

Added test cases for case node handling with comments, newlines.

resolves: #6299
2023-08-11 19:20:25 +05:30
konsti
8b24238d19 Show a pretty markdown table in formatter ecosystem checks (#6496)
**Summary** The formatter ecosystem checks will now print a markdown table you can copy&paste into your PR description. 

![image](https://github.com/astral-sh/ruff/assets/6826232/80289ed9-9d2b-400e-a994-de63dca0b065)

copied markdown:

| project      | similarity index |
|--------------|------------------|
| build        | 0.75623          |
| cpython      | 0.75989          |
| django       | 0.99784          |
| transformers | 0.99470          |
| typeshed     | 0.74853          |
| warehouse    | 0.99585          |
| zulip        | 0.99702          |

raw markdown:
```markdown
| project      | similarity index |
|--------------|------------------|
| build        | 0.75623          |
| cpython      | 0.75989          |
| django       | 0.99784          |
| transformers | 0.99470          |
| typeshed     | 0.74853          |
| warehouse    | 0.99585          |
| zulip        | 0.99702          |
```
2023-08-11 15:37:21 +02:00
Charlie Marsh
f2939c678b Avoid breaking call chains unnecessarily (#6488)
## Summary

This PR attempts to fix the formatting of the following expression:

```python
max_message_id = (
    Message.objects.filter(recipient=recipient).order_by("id").reverse()[0].id
)
```

Specifically, Black preserves _that_ formatting, while we do:

```python
max_message_id = (
    Message.objects.filter(recipient=recipient)
    .order_by("id")
    .reverse()[0]
    .id
)
```

The fix here is to add a group around the entire call chain.

## Test Plan

Before:

- `zulip`: 0.99702
- `django`: 0.99784
- `warehouse`: 0.99585
- `build`: 0.75623
- `transformers`: 0.99470
- `cpython`: 0.75989
- `typeshed`: 0.74853

After:

- `zulip`: 0.99703
- `django`: 0.99791
- `warehouse`: 0.99586
- `build`: 0.75623
- `transformers`: 0.99470
- `cpython`: 0.75989
- `typeshed`: 0.74853
2023-08-11 13:33:15 +00:00
Victor Hugo Gomes
b05574babd Fix formatter instability with half-indented comment (#6460)
## Summary
The bug was happening in this
[loop](75f402eb82/crates/ruff_python_formatter/src/comments/placement.rs (L545)).

Basically, In the first iteration of the loop, the `comment_indentation`
is bigger than `child_indentation` (`comment_indentation` is 7 and
`child_indentation` is 4) making the `Ordering::Greater` branch execute.
Inside the `Ordering::Greater` branch, the `if` block gets executed,
resulting in the update of these variables.
```rust
parent_body = current_body;                    
current_body = Some(last_child_in_current_body);
last_child_in_current_body = nested_child;
```
In the second iteration of the loop, `comment_indentation` is smaller
than `child_indentation` (`comment_indentation` is 7 and
`child_indentation` is 8) making the `Ordering::Less` branch execute.
Inside the `Ordering::Less` branch, the `if` block gets executed, this
is where the bug was happening. At this point `parent_body` should be a
`StmtFunctionDef` but it was a `StmtClassDef`. Causing the comment to be
incorrectly formatted.

That happened for the following code:
```python
class A:
    def f():
        pass
       # strangely indented comment

print()
```

There is only one problem that I couldn't figure it out a solution, the
variable `current_body` in this
[line](75f402eb82/crates/ruff_python_formatter/src/comments/placement.rs (L542C5-L542C49))
now gives this warning _"value assigned to `current_body` is never read
maybe it is overwritten before being read?"_
Any tips on how to solve that?

Closes #5337

## Test Plan

Add new test case.

---------

Co-authored-by: konstin <konstin@mailbox.org>
2023-08-11 11:21:16 +00:00
konsti
0ef6af807b Implement DerefMut for WithNodeLevel (#6443)
**Summary** Implement `DerefMut` for `WithNodeLevel` so it can be used
in the same way as `PyFormatter`. I want this for my WIP upstack branch
to enable `.fmt(f)` on `WithNodeLevel` context. We could extend this to
remove the other two method from `WithNodeLevel`.
2023-08-11 10:41:48 +00:00
David Szotten
f091b46497 move comments from expressions in f-strings out (#6481) 2023-08-11 09:22:30 +02:00
Charlie Marsh
2cedb401bd Force parentheses for named expressions in more contexts (#6494)
See:
https://github.com/astral-sh/ruff/pull/6436#issuecomment-1673583888.
2023-08-11 01:54:46 -04:00
Charlie Marsh
2e5c81b202 Ensure that B006 autofix respects docstrings (#6493)
## Summary

Some follow-ups to https://github.com/astral-sh/ruff/pull/6131 to ensure
that fixes are inserted _after_ function docstrings, and that fixes are
robust to a bunch of edge cases.

## Test Plan

`cargo test`
2023-08-11 01:03:56 -04:00
Charlie Marsh
cc151c35a8 Respect dummy-variable-rgx for unused bound exceptions (#6492)
## Summary

This PR respects our unused variable regex when flagging bound
exceptions, so that you no longer get a violation for, e.g.:

```python
def f():
    try:
        pass
    except Exception as _:
        pass
```

This is an odd pattern, but I think it's surprising that the regex
_isn't_ respected here.

Closes https://github.com/astral-sh/ruff/issues/6391

## Test Plan

`cargo test`
2023-08-11 04:02:02 +00:00
Charlie Marsh
563374503f Enable short URLs in the playground (#6383)
## Summary

This PR adds a [Workers
KV](https://developers.cloudflare.com/workers/runtime-apis/kv/)-based
database to the playground, which enables us to associate shared
snippets with a stable ID, which in turn allows us to generate short
URLs, rather than our existing extremely-long URLs.

For now, the URLs are based on UUID, so they look like
https://play.ruff.rs/a1c40d58-f643-4a3e-bc23-15021e16acef. (This URL
isn't expected to work, as the playground isn't deployed; it's only
included as an example.)

There are no visible changes in the UI here -- you still click the
"Share" button, which copies the link to your URL. There's no
user-visible latency either -- KV is very fast.

For context, with Workers KV, we provision a Workers KV store in our
Cloudflare account (`wrangler kv:namespace create "PLAYGROUND"`), and
then create a Cloudflare Worker that's bound to the KV store via the
`wrangler.toml`:

```toml
name = "db"
main = "src/index.ts"
compatibility_date = "2023-08-07"

kv_namespaces = [
  { binding = "PLAYGROUND", id = "672e16c4fb5e4887845973bf0e9f6021", preview_id = "0a96477e116540e5a6e1eab6d6e7523e" }
]
```

The KV store exists in perpetuity, while the Worker can be updated,
deployed, removed, etc. independently of the KV store. The Worker itself
has unfettered access to the KV store. The Worker is exposed publicly,
and just does some basic verification against the request host.
2023-08-11 02:31:09 +00:00
Charlie Marsh
95dea5c868 Respect tab width in line-length heuristic (#6491)
## Summary

In https://github.com/astral-sh/ruff/pull/5811, I suggested that we add
a heuristic to the overlong-lines check such that if the line had fewer
bytes than the character limit, we return early -- the idea being that a
single byte per character was the "worst case". I overlooked that this
isn't true for tabs -- with tabs, the "worst case" scenario is that
every byte is a tab, which can have a width greater than 1.

Closes https://github.com/astral-sh/ruff/issues/6425.

## Test Plan

`cargo test` with a new fixture borrowed from the issue, plus manual
testing.
2023-08-10 22:28:25 -04:00
Victor Hugo Gomes
eb68addf97 [pylint] Implement bad-dunder-name (W3201) (#6486)
## Summary

Checks for any misspelled dunder name method and for any method defined
with `__...__` that's not one of the pre-defined methods.

The pre-defined methods encompass all of Python's standard dunder
methods.

ref: #970

## Test Plan
Snapshots and manual runs of pylint.
2023-08-11 01:31:16 +00:00
Tom Kuson
9ff80a82b4 [pylint] Implement subprocess-run-check (W1510) (#6487)
## Summary

Implements [`subprocess-run-check`
(`W1510`)](https://pylint.readthedocs.io/en/latest/user_guide/messages/warning/subprocess-run-check.html)
as `subprocess-run-without-check` (`PLW1510`). Includes documentation.

Related to #970.

## Test Plan

`cargo test`
2023-08-10 20:54:53 -04:00
Charlie Marsh
84ae00c395 Allow os._exit accesses in SLF001 (#6490)
Closes https://github.com/astral-sh/ruff/issues/6483.
2023-08-11 00:54:38 +00:00
Zanie Blue
1050c4e104 Extend target-version documentation (#6482)
Closes https://github.com/astral-sh/ruff/issues/6462
2023-08-10 12:11:37 -05:00
Charlie Marsh
6706ae4828 Respect scoping rules when identifying builtins (#6468)
## Summary

Our `is_builtin` check did a naive walk over the parent scopes; instead,
it needs to (e.g.) skip symbols in a class scope if being called outside
of the class scope itself.

Closes https://github.com/astral-sh/ruff/issues/6466.

## Test Plan

`cargo test`
2023-08-10 10:20:09 -04:00
magic-akari
dc3275fe7f Improve Ruff Formatter Interoperability (#6472) 2023-08-10 14:39:53 +02:00
qdegraaf
50dab9cea6 [flake8-bugbear] Add autofix for B006 (#6131)
## Summary

Reopening of https://github.com/astral-sh/ruff/pull/4880 

One open TODO as described in:
https://github.com/astral-sh/ruff/pull/4880#discussion_r1265110215

FYI @charliermarsh seeing as you commented you wanted to do final review
and merge. @konstin @dhruvmanila @MichaReiser as previous reviewers.

# Old Description
## Summary

Adds an autofix for B006 turning mutable argument defaults into None and
setting their original value back in the function body if still `None`
at runtime like so:
```python
def before(x=[]):
    pass
    
def after(x=None):
    if x is None:
        x = []
    pass
```

## Test Plan

Added an extra test case to existing fixture with more indentation.
Checked results for all old examples.

NOTE: Also adapted the jupyter notebook test as this checked for B006 as
well.

## Issue link

Closes: https://github.com/charliermarsh/ruff/issues/4693

---------

Co-authored-by: konstin <konstin@mailbox.org>
2023-08-10 11:06:40 +00:00
konsti
4811af0f0b Formatter: Add test cases for comments after opening parentheses (#6420)
**Summary** I collected all examples of end-of-line comments after
opening parentheses that i could think of so we get a comprehensive view
at the state of their formatting (#6390).

This PR intentionally only adds tests cases without any changes in
formatting. We need to decide which exact formatting we want, ideally in
terms of these test files, and implement this in follow-up PRs.

~~One stability check is still deactivated pending
https://github.com/astral-sh/ruff/pull/6386.~~
2023-08-10 08:34:03 +00:00
konsti
39beeb61f7 Track formatting all comments
We currently don't format all comments as match statements are not yet implemented. We can work around this for the top level match statement by setting them manually formatted but the mocked-out top level match doesn't call into its children so they would still have unformatted comments
2023-08-10 09:19:27 +02:00
Micha Reiser
e2f7862404 Preserve dangling f-string comments
<!--
Thank you for contributing to Ruff! To help us out with reviewing, please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

This PR fixes the issue where the FString formatting dropped dangling comments between the string parts.

```python
result_f = (
    f'  File "{__file__}", line {lineno_f+1}, in f\n'
    '    f()\n'
    # XXX: The following line changes depending on whether the tests
    # are run through the interactive interpreter or with -m
    # It also varies depending on the platform (stack size)
    # Fortunately, we don't care about exactness here, so we use regex
    r'  \[Previous line repeated (\d+) more times\]' '\n'
    'RecursionError: maximum recursion depth exceeded\n'
)
```

The solution here isn't ideal because it re-introduces the `enclosing_parent` on `DecoratedComment` but it is the easiest fix that I could come up. 
I didn't spend more time finding another solution becaues I think we have to re-write most of the fstring formatting with the upcoming Python 3.12 support (because lexing the individual parts as we do now will no longer work).

closes #6440

<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

`cargo test`

The child PR testing that all comments are formatted should now pass
2023-08-10 09:11:25 +02:00
Micha Reiser
ac5c8bb3b6 Add AnyNodeRef.visit_preorder
<!--
Thank you for contributing to Ruff! To help us out with reviewing, please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

This PR adds the `AnyNodeRef.visit_preorder` method. I'll need this method to mark all comments of a suppressed node's children as formatted (in debug builds). 

I'm not super happy with this because it now requires a double-dispatch where the `walk_*` methods call into `node.visit_preorder` and the `visit_preorder` then calls back into the visitor. Meaning,
the new implementation now probably results in way more function calls. The other downside is that `AnyNodeRef` now contains code that is difficult to auto-generate. This could be mitigated by extracting the `visit_preorder` method into its own `VisitPreorder` trait. 

Anyway, this approach solves the need and avoids duplicating the visiting code once more. 

<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

`cargo test`

<!-- How was it tested? -->
2023-08-10 08:35:09 +02:00
Micha Reiser
c1bc67686c Use SimpleTokenizer in max_lines (#6451) 2023-08-10 08:13:14 +02:00
Charlie Marsh
7eea0e94a2 Add containers to E721 types (#6469)
Related to https://github.com/astral-sh/ruff/issues/6465.
2023-08-10 02:34:51 +00:00
Charlie Marsh
0252995973 Document FormatSpec fields (#6458) 2023-08-09 18:13:29 -04:00
Charlie Marsh
627f475b91 Avoid applying PYI055 to runtime-evaluated annotations (#6457)
## Summary

The use of `|` as a union operator is not always safe, if a type
annotation is evaluated in a runtime context. For example, this code
errors at runtime:

```python
import httpretty
import requests_mock

item: type[requests_mock.Mocker | httpretty] = requests_mock.Mocker
```

However, it's fine in a `.pyi` file, with `__future__` annotations`, or
if the annotation is in a non-evaluated context, like:

```python
def func():
    item: type[requests_mock.Mocker | httpretty] = requests_mock.Mocker
```

This PR modifies the rule to avoid enforcing in those invalid,
runtime-evaluated contexts.

Closes https://github.com/astral-sh/ruff/issues/6455.
2023-08-09 16:46:41 -04:00
Charlie Marsh
395bb31247 Improve counting of message arguments when msg is provided as a keyword (#6456)
Closes https://github.com/astral-sh/ruff/issues/6454.
2023-08-09 20:39:10 +00:00
Zanie Blue
3ecd263b4d Bump version to 0.0.284 (#6453)
## What's Changed

This release fixes a few bugs, notably the previous release announced a
breaking change where the default target
Python version changed from 3.10 to 3.8 but it was not applied. Thanks
to @rco-ableton for fixing this in
https://github.com/astral-sh/ruff/pull/6444

### Bug Fixes
* Do not trigger `S108` if path is inside `tempfile.*` call by
@dhruvmanila in https://github.com/astral-sh/ruff/pull/6416
* Do not allow on zero tab width by @tjkuson in
https://github.com/astral-sh/ruff/pull/6429
* Fix false-positive in submodule resolution by @charliermarsh in
https://github.com/astral-sh/ruff/pull/6435

## New Contributors
* @rco-ableton made their first contribution in
https://github.com/astral-sh/ruff/pull/6444

**Full Changelog**:
https://github.com/astral-sh/ruff/compare/v0.0.283...v0.0.284
2023-08-09 13:32:33 -05:00
Charlie Marsh
6acf07c5c4 Use latest Python version by default in tests (#6448)
## Summary

Use the same Python version by default for all tests (our
latest-supported version).

## Test Plan

`cargo test`

---------

Co-authored-by: Zanie <contact@zanie.dev>
2023-08-09 15:22:39 +00:00
Charlie Marsh
38b9fb8bbd Set a default on PythonVersion (#6446)
## Summary

I think it makes sense for `PythonVersion::default()` to return our
minimum-supported non-EOL version.

## Test Plan

`cargo test`

---------

Co-authored-by: Zanie <contact@zanie.dev>
2023-08-09 15:19:27 +00:00
dependabot[bot]
e4f57434a2 ci(deps): bump cloudflare/wrangler-action from 2.0.0 to 3.0.0 (#6398)
Bumps
[cloudflare/wrangler-action](https://github.com/cloudflare/wrangler-action)
from 2.0.0 to 3.0.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="089567dec4"><code>089567d</code></a>
feat: rewrite Wrangler Action in TypeScript</li>
<li>See full diff in <a
href="https://github.com/cloudflare/wrangler-action/compare/2.0.0...3.0.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=cloudflare/wrangler-action&package-manager=github_actions&previous-version=2.0.0&new-version=3.0.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-09 10:17:43 -05:00
Dhruv Manilawala
6a64f2289b Rename Magic* to IpyEscape* (#6395)
## Summary

This PR renames the `MagicCommand` token to `IpyEscapeCommand` token and
`MagicKind` to `IpyEscapeKind` type to better reflect the purpose of the
token and type. Similarly, it renames the AST nodes from `LineMagic` to
`IpyEscapeCommand` prefixed with `Stmt`/`Expr` wherever necessary.

It also makes renames from using `jupyter_magic` to
`ipython_escape_commands` in various function names.

The mode value is still `Mode::Jupyter` because the escape commands are
part of the IPython syntax but the lexing/parsing is done for a Jupyter
notebook.

### Motivation behind the rename:
* IPython codebase defines it as "EscapeCommand" / "Escape Sequences":
* Escape Sequences:
292e3a2345/IPython/core/inputtransformer2.py (L329-L333)
* Escape command:
292e3a2345/IPython/core/inputtransformer2.py (L410-L411)
* The word "magic" is used mainly for the actual magic commands i.e.,
the ones starting with `%`/`%%`
(https://ipython.readthedocs.io/en/stable/interactive/reference.html#magic-command-system).
So, this avoids any confusion between the Magic token (`%`, `%%`) and
the escape command itself.
## Test Plan

* `cargo test` to make sure all renames are done correctly.
* `grep` for `jupyter_escape`/`magic` to make sure all renames are done
correctly.
2023-08-09 13:28:18 +00:00
Charlie Marsh
3bf1c66cda Group function definition parameters with return type annotations (#6410)
## Summary

This PR removes the group around function definition parameters, instead
grouping the parameters with the type parameters and return type
annotation.

This increases Zulip's similarity score from 0.99385 to 0.99699, so it's
a meaningful improvement. However, there's at least one stability error
that I'm working on, and I'm really just looking for high-level feedback
at this point, because I'm not happy with the solution.

Closes https://github.com/astral-sh/ruff/issues/6352.

## Test Plan

Before:

- `zulip`: 0.99396
- `django`: 0.99784
- `warehouse`: 0.99578
- `build`: 0.75436
- `transformers`: 0.99407
- `cpython`: 0.75987
- `typeshed`: 0.74432

After:

- `zulip`: 0.99702
- `django`: 0.99784
- `warehouse`: 0.99585
- `build`: 0.75623
- `transformers`: 0.99470
- `cpython`: 0.75988
- `typeshed`: 0.74853
2023-08-09 12:13:58 +00:00
rco-ableton
eaada0345c Set default version to py38 (#6444)
## Summary

In https://github.com/astral-sh/ruff/pull/6397, the documentation was
updated stating that the default target-version is now "py38", but the
actual default value wasn't updated and remained py310. This commit
updates the default value to match what the documentation says.
2023-08-09 12:08:47 +00:00
Micha Reiser
a39dd76d95 Add enter and leave_node methods to Preoder visitor (#6422) 2023-08-09 09:09:00 +00:00
Dhruv Manilawala
e257c5af32 Add support for help end IPython escape commands (#6358)
## Summary

This PR adds support for a stricter version of help end escape
commands[^1] in the parser. By stricter, I mean that the escape tokens
are only at the end of the command and there are no tokens at the start.
This makes it difficult to implement it in the lexer without having to
do a lot of look aheads or keeping track of previous tokens.

Now, as we're adding this in the parser, the lexer needs to recognize
and emit a new token for `?`. So, `Question` token is added which will
be recognized only in `Jupyter` mode.

The conditions applied are the same as the ones in the original
implementation in IPython codebase (which is a regex):
* There can only be either 1 or 2 question mark(s) at the end
* The node before the question mark can be a `Name`, `Attribute`,
`Subscript` (only with integer constants in slice position), or any
combination of the 3 nodes.

## Test Plan

Added test cases for various combination of the possible nodes in the
command value position and update the snapshots.

fixes: #6359
fixes: #5030 (This is the final piece)

[^1]: https://github.com/astral-sh/ruff/pull/6272#issue-1833094281
2023-08-09 10:28:52 +05:30
Dhruv Manilawala
887a47cad9 Avoid S108 if path is inside tempfile.* call (#6416) 2023-08-09 10:22:31 +05:30
Charlie Marsh
a2758513de Fix false-positive in submodule resolution (#6435)
Closes https://github.com/astral-sh/ruff/issues/6433.
2023-08-09 02:36:39 +00:00
Tom Kuson
1b9fed8397 Error on zero tab width (#6429)
## Summary

Error if `tab-size` is set to zero (it is used as a divisor). Closes
#6423.

Also fixes a typo.

## Test Plan

Running ruff with a config

```toml
[tool.ruff]
tab-size = 0
```

returns an error message to the user saying that `tab-size` must be
greater than zero.
2023-08-08 16:51:37 -04:00
Charlie Marsh
55d6fd53cd Treat comments on open parentheses in return annotations as dangling (#6413)
## Summary

Given:

```python
def double(a: int) -> ( # Hello
    int
):
    return 2*a
```

We currently treat `# Hello` as a trailing comment on the parameters
(`(a: int)`). This PR adds a placement method to instead treat it as a
dangling comment on the function definition itself, so that it gets
formatted at the end of the definition, like:

```python
def double(a: int) -> int:  # Hello
    return 2*a
```

The formatting in this case is unchanged, but it's incorrect IMO for
that to be a trailing comment on the parameters, and that placement
leads to an instability after changing the grouping in #6410.

Fixing this led to a _different_ instability related to tuple return
type annotations, like:

```python
def zrevrangebylex(self, name: _Key, max: _Value, min: _Value, start: int | None = None, num: int | None = None) -> (  # type: ignore[override]
):
    ...
```

(This is a real example.)

To fix, I had to special-case tuples in that spot, though I'm not
certain that's correct.
2023-08-08 16:48:38 -04:00
Zanie Blue
d33618062e Improve documentation for PLE1300 (#6430) 2023-08-08 20:16:36 +00:00
Charlie Marsh
c7703e205d Move empty_parenthesized into the parentheses.rs (#6403)
## Summary

This PR moves `empty_parenthesized` such that it's peer to
`parenthesized`, and changes the API to better match that of
`parenthesized` (takes `&str` rather than `StaticText`, has a
`with_dangling_comments` method, etc.).

It may be intentionally _not_ part of `parentheses.rs`, but to me
they're so similar that it makes more sense for them to be in the same
module, with the same API, etc.
2023-08-08 19:17:17 +00:00
Zanie Blue
fe9590f39f Bump version number to 0.0.283 (#6407) 2023-08-08 12:31:30 -05:00
konsti
e769c74899 Check .git in formatter progress checkouts for build (#6387)
From the formatter progress CI logs:
```
2023-08-07T03:49:02.5178602Z + mkdir -p /home/runner/work/ruff/ruff/target/progress_projects
2023-08-07T03:49:02.5193474Z + '[' '!' -d /home/runner/work/ruff/ruff/target/progress_projects/build ']'
2023-08-07T03:49:02.5194228Z + '[' '!' -d /home/runner/work/ruff/ruff/target/progress_projects/django ']'
2023-08-07T03:49:02.5194966Z + git clone --filter=tree:0 https://github.com/django/django /home/runner/work/ruff/ruff/target/progress_projects/django
2023-08-07T03:49:02.5209260Z Cloning into '/home/runner/work/ruff/ruff/target/progress_projects/django'...
```
```
2023-08-07T03:51:17.4726088Z 2023-08-07T03:51:17.472404Z ERROR Failed /home/runner/work/ruff/ruff/target/progress_projects/build: no python files in ["/home/runner/work/ruff/ruff/target/progress_projects/build"]
```

Seems that build exists but is an empty cached folder. These changes
should fix this by a) checking for `.git` instead of just the folder
existing b) running the commit checkout unconditionally. The latter is
also important if we ever want to update the SHAs.
2023-08-08 17:46:46 +02:00
Dhruv Manilawala
d815a25b11 Update StmtMatch formatting snapshots (#6427) 2023-08-08 16:45:02 +02:00
Dhruv Manilawala
001aa486df Add formatting for StmtMatch (#6286)
## Summary

This PR adds support for `StmtMatch` with subs for `MatchCase`.

## Test Plan

Add a few additional test cases around `match` statement, comments, line
breaks.

resolves: #6298
2023-08-08 18:48:49 +05:30
Charlie Marsh
87984e9ac7 Expand parents whenever open-parenthesis comments are present (#6389)
## Summary

This PR modifies our dangling-open-parenthesis handling to _always_
expand the parent expression.

So, for example, given:

```python
a = int(  # type: ignore
    int(  # type: ignore
        int(  # type: ignore
            6
        )
    )
)
```

We now retain that as stable formatting, instead of truncating like:

```python
a = int(int(int(6)))  # comment  # comment  # comment
```

Note that Black _does_ collapse comments like this _unless_ they're `#
type: ignore` comments, and perhaps in some other cases, so this is an
intentional deviation
([playground](https://black.vercel.app/?version=main&state=_Td6WFoAAATm1rRGAgAhARYAAAB0L-Wj4AFEAHpdAD2IimZxl1N_WlOfrjryFgvD4ScVsKPztqdHDGJUg5knO0JCdpUfW1IrWSNmIJPx95s0hP-pRNkCQNH64-eIznIvXjeWBQ5-qax0oNw4yMOuhwr2azvMRZaEB5r8IXVPHmRCJp7fe7y4290u1zzxqK_nAi6q_5sI-jsAAAAA8HgZ9V7hG3QAAZYBxQIAAGnCHXexxGf7AgAAAAAEWVo=)).
2023-08-08 08:45:20 -04:00
Piotr
6aefe71c56 Fix name of rule in example of extend-per-file-ignores in options.rs (#6417)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?

-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

Fix name of rule in example of `extend-per-file-ignores` in `options.rs`
file.

It was `E401` but in configuration example `E402` was listed. Just a
tiny mismatch.

## Test Plan

<!-- How was it tested? -->

Just by my eyes :).
2023-08-08 11:24:41 +02:00
konsti
90ba40c23c Fix zulip unstable formatting with end-of-line comments (#6386)
## Bug

Given
```python
x = () - (#
)
```
the comment is a dangling comment of the empty tuple. This is an
end-of-line comment so it may move after the expression. It still
expands the parent, so the operator breaks:
```python
x = (
    ()
    - ()  #
)
```
In the next formatting pass, the comment is not a trailing tuple but a
trailing bin op comment, so the bin op doesn't break anymore. The
comment again expands the parent, so we still add the superfluous
parentheses
```python
x = (
    () - ()  #
)
```

## Fix

The new formatting is to keep the comment on the empty tuple. This is a
log uglier and again has additional outer parentheses, but it's stable:
```python
x = (
    ()
    - (  #
    )
)
```

## Alternatives

Black formats all the examples above as
```python
x = () - ()  #
```
which i find better. 

I would be happy about any suggestions for better solutions than the
current one. I'd mainly need a workaround for expand parent having an
effect on the bin op instead of first moving the comment to the end and
then applying expand parent to the assign statement.
2023-08-08 09:15:35 +00:00
Micha Reiser
2bd345358f Simplify parenthesized formatting (#6419) 2023-08-08 08:50:57 +00:00
Dhruv Manilawala
289d1e85bf Manually parenthesize tuple expr in B014 autofix (#6415)
## Summary

Manually add the parentheses around tuple expressions for the autofix in
`B014`.
This is also done in various other autofixes as well such as for
[`RUF005`](6df5ab4098/crates/ruff/src/rules/ruff/rules/collection_literal_concatenation.rs (L183-L184)),
[`UP024`](6df5ab4098/crates/ruff/src/rules/pyupgrade/rules/os_error_alias.rs (L137-L137)).

### Alternate Solution

An alternate solution would be to fix this in the `Generator` itself by
checking
if the tuple expression needs to be generated at the top-level or not.
If so,
then always add the parentheses.

```rust
                } else if level == 0 {
                    // Top-level tuples are always parenthesized.
                    self.p("(");
                    let mut first = true;
                    for elt in elts {
                        self.p_delim(&mut first, ", ");
                        self.unparse_expr(elt, precedence::COMMA);
                    }
                    self.p_if(elts.len() == 1, ",");
                    self.p(")");
```

## Test Plan

Add a regression test for this case in `B014`.

fixes: #6412
2023-08-08 09:14:18 +05:30
Anders Kaseorg
6df5ab4098 Remove duplicate line from project structure docs (#6408)
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2023-08-07 20:08:51 -04:00
Zanie Blue
90c9aa2992 Add support for simple generic type variables to UP040 (#6314)
Extends #6289 to support moving type variable usage in type aliases to
use PEP-695.

Does not remove the possibly unused type variable declaration.
Presumably this is handled by other rules, but is not working for me.

Does not handle type variables with bounds or variance declarations yet.

Part of #4617
2023-08-07 16:22:06 -05:00
Charlie Marsh
927cfc9564 Respect file-level # ruff: noqa suppressions for unused-noqa rule (#6405)
## Summary

We weren't respecting `# ruff: noqa: RUF100`, i.e., file-level
suppressions for the `unused-noqa` rule itself.

Closes https://github.com/astral-sh/ruff/issues/6385.
2023-08-07 16:33:01 -04:00
Charlie Marsh
3d06fe743d Change model: &SemanticModel to semantic: &SemanticModel (#6406)
Use the same naming conventions everywhere. See:
https://github.com/astral-sh/ruff/pull/6314/files#r1284457874.
2023-08-07 16:32:55 -04:00
Charlie Marsh
404e334fec Rename ArgumentSeparator to ParameterSeparator (#6404)
To mirror the rename from `Arguments` to `Parameters`.
2023-08-07 15:46:28 -04:00
Charlie Marsh
26098b8d91 Extend nested union detection to handle bitwise or Union expressions (#6399)
## Summary

We have some logic in the expression analyzer method to avoid
re-checking the inner `Union` in `Union[Union[...]]`, since the methods
that analyze `Union` expressions already recurse. Elsewhere, we have
logic to avoid re-checking the inner `|` in `int | (int | str)`, for the
same reason.

This PR unifies that logic into a single method _and_ ensures that, just
as we recurse over both `Union` and `|`, we also detect that we're in
_either_ kind of nested union.

Closes https://github.com/astral-sh/ruff/issues/6285.

## Test Plan

Added some new snapshots.
2023-08-07 15:17:26 -04:00
Charlie Marsh
98d4657961 Avoid attempting to fix .format(...) calls with too-few-arguments (#6401)
## Summary

We can anticipate earlier that this will error, so we should avoid
flagging the error at all. Specifically, we're talking about cases like
`"{1} {0}".format(*args)"`, in which we'd need to reorder the arguments
in order to remove the `1` and `0`, but we _can't_ reorder the arguments
since they're not statically analyzable.

Closes https://github.com/astral-sh/ruff/issues/6388.
2023-08-07 19:13:22 +00:00
Charlie Marsh
8919b6ad9a Add a with_dangling_comments to the parenthesized formatter (#6402)
See: https://github.com/astral-sh/ruff/pull/6376#discussion_r1285514328.
2023-08-07 19:12:12 +00:00
Zanie Blue
bb96647d66 Assume Python 3.8 instead of 3.10 for target version (#6397)
The target version should be the oldest supported version instead of an
arbitary version. Since 3.7 is EOL, we should use 3.8. I would like to
follow this up with more comprehensive default detection based on the
environment.
2023-08-07 13:48:06 -05:00
Charlie Marsh
df1591b3c2 Remove outdated TODO (#6400)
See: https://github.com/astral-sh/ruff/pull/6376#discussion_r1285539278.
2023-08-07 18:33:18 +00:00
Charlie Marsh
a637b8b3a3 Fixup comment handling on opening parenthesis in function definition (#6381)
## Summary

I noticed some deviations in how we treat dangling comments that hug the
opening parenthesis for function definitions.

For example, given:

```python
def f(  # first
    # second
):  # third
    ...
```

We currently format as:

```python
def f(
      # first
    # second
):  # third
    ...
```

This PR adds the proper opening-parenthesis dangling comment handling
for function parameters. Specifically, as with all other parenthesized
nodes, we now detect that dangling comment in `placement.rs` and handle
it in `parameters.rs`. We have to take some care in that file, since we
have multiple "kinds" of dangling comments, but I added a bunch of test
cases that we now format identically to Black.

## Test Plan

`cargo test`

Before:

- `zulip`: 0.99388
- `django`: 0.99784
- `warehouse`: 0.99504
- `transformers`: 0.99404
- `cpython`: 0.75913
- `typeshed`: 0.74364

After:

- `zulip`: 0.99386
- `django`: 0.99784
- `warehouse`: 0.99504
- `transformers`: 0.99404
- `cpython`: 0.75913
- `typeshed`: 0.74409

Meaningful improvement on `typeshed`, minor decrease on `zulip`.
2023-08-07 14:04:56 -04:00
Charlie Marsh
3f0eea6d87 Rename JoinedStr to FString in the AST (#6379)
## Summary

Per the proposal in https://github.com/astral-sh/ruff/discussions/6183,
this PR renames the `JoinedStr` node to `FString`.
2023-08-07 17:33:17 +00:00
Zanie Blue
999d88e773 Fix formatting of chained boolean operations (#6394)
Closes https://github.com/astral-sh/ruff/issues/6068

These commits are kind of a mess as I did some stumbling around here. 

Unrolls formatting of chained boolean operations to prevent nested
grouping which gives us Black-compatible formatting where each boolean
operation is on a new line.
2023-08-07 12:22:33 -05:00
Charlie Marsh
63ffadf0b8 Avoid omitting parentheses for trailing attributes on call expressions (#6322)
## Summary

This PR modifies our `can_omit_optional_parentheses` rules to ensure
that if we see a call followed by an attribute, we treat that as an
attribute access rather than a splittable call expression.

This in turn ensures that we wrap like:

```python
ct_match = aaaaaaaaaaact_id == self.get_content_type(
    obj=rel_obj, using=instance._state.db
)
```

For calls, but:

```python
ct_match = (
    aaaaaaaaaaact_id == self.get_content_type(obj=rel_obj, using=instance._state.db).id
)
```

For calls with trailing attribute accesses.

Closes https://github.com/astral-sh/ruff/issues/6065.

## Test Plan

Similarity index before:

- `zulip`: 0.99436
- `django`: 0.99779
- `warehouse`: 0.99504
- `transformers`: 0.99403
- `cpython`: 0.75912
- `typeshed`: 0.72293

And after:

- `zulip`: 0.99436
- `django`: 0.99780
- `warehouse`: 0.99504
- `transformers`: 0.99404
- `cpython`: 0.75913
- `typeshed`: 0.72293
2023-08-07 13:18:58 -04:00
Charlie Marsh
c439435615 Use dedicated AST nodes on MemberKind (#6374)
## Summary

This PR leverages the unified function definition node to add precise
AST node types to `MemberKind`, which is used to power our docstring
definition tracking (e.g., classes and functions, whether they're
methods or functions or nested functions and so on, whether they have a
docstring, etc.). It was painful to do this in the past because the
function variants needed to support a union anyway, but storing precise
nodes removes like a dozen panics.

No behavior changes -- purely a refactor.

## Test Plan

`cargo test`
2023-08-07 17:17:58 +00:00
Charlie Marsh
daefa74e9a Remove async AST node variants for with, for, and def (#6369)
## Summary

Per the suggestion in
https://github.com/astral-sh/ruff/discussions/6183, this PR removes
`AsyncWith`, `AsyncFor`, and `AsyncFunctionDef`, replacing them with an
`is_async` field on the non-async variants of those structs. Unlike an
interpreter, we _generally_ have identical handling for these nodes, so
separating them into distinct variants adds complexity from which we
don't really benefit. This can be seen below, where we get to remove a
_ton_ of code related to adding generic `Any*` wrappers, and a ton of
duplicate branches for these cases.

## Test Plan

`cargo test` is unchanged, apart from parser snapshots.
2023-08-07 16:36:02 +00:00
Charlie Marsh
c895252aae Remove RefEquality (#6393)
## Summary

See discussion in
https://github.com/astral-sh/ruff/pull/6351#discussion_r1284996979. We
can remove `RefEquality` entirely and instead use a text offset for
statement keys, since no two statements can start at the same text
offset.

## Test Plan

`cargo test`
2023-08-07 16:04:50 +00:00
Charlie Marsh
9328606843 Remove Statements#parent (#6392)
Discussed in
https://github.com/astral-sh/ruff/pull/6351#discussion_r1284997065.
2023-08-07 15:41:02 +00:00
Dhruv Manilawala
e4a4660925 Support help end escape command with priority (#6272)
## Summary

This PR adds support for help end escape command in the lexer.

### What are "help end escape commands"?

First, the escape commands are special IPython syntax which enhances the
functionality for the IPython REPL. There are 9 types of escape kinds
which are recognized by the tokens which are present at the start of the
command (`?`, `??`, `!`, `!!`, etc.).

Here, the help command is using either the `?` or `??` token at the
start (`?str.replace` for example). Those 2 tokens are also supported
when they're at the end of the command (`str.replace?`), but the other
tokens aren't supported in that position.

There are mainly two types of help end escape commands:
1. Ending with either `?` or `??`, but it also starts with one of the
escape tokens (`%matplotlib?`)
2. On the other hand, there's a stricter version for (1) which doesn't
start with any escape tokens (`str.replace?`)

This PR adds support for (1) while (2) will be supported in the parser.

### Priority

Now, if the command starts and ends with an escape token, how do we
decide the kind of this command? This is where priority comes into
picture. This is simple as there's only one priority where `?`/`??` at
the end takes priority over any other escape token and all of the other
tokens are at the same priority. Remember that only `?`/`??` at the end
is considered valid.

This is mainly useful in the case where someone would want to invoke the
help command on the magic command itself. For example, in `%matplotlib?`
the help command takes priority which means that we want help for the
`matplotlib` magic function instead of calling the magic function
itself.

### Specification

Here's where things get a bit tricky. What if there are question mark
tokens at both ends. How do we decide if it's `Help` (`?`) kind or
`Help2` (`??`) kind?

|     | Magic       | Value     | Kind    |
| --- | ---         | ---       | ---     |
| 1   | `?foo?`     | `foo`     | `Help`  |
| 2   | `??foo?`    | `foo`     | `Help`  |
| 3   | `?foo??`    | `foo`     | `Help2` |
| 4   | `??foo??`   | `foo`     | `Help2` |
| 5   | `???foo??`  | `foo`     | `Help2` |
| 6   | `??foo???`  | `foo???`  | `Help2` |
| 7   | `???foo???` | `?foo???` | `Help2` |

Looking at the above table:

- The question mark tokens on the right takes priority over the ones on
the left but only if the number of question mark on the right is 1 or 2.
- If there are more than 2 question mark tokens on the right side, then
the left side is used to determine the same.
- If the right side is used to determine the kind, then all of the
question marks and whitespaces on the left side are ignored in the
`value`, but if it’s the other way around, then all of the extra
question marks are part of the `value`.

### References

- IPython implementation using the regex:
292e3a2345/IPython/core/inputtransformer2.py (L454-L462)
- Priorities:
292e3a2345/IPython/core/inputtransformer2.py (L466-L469)

## Test Plan

Add a bunch of test cases for the lexer and verify that it matches the
behavior of
IPython transformer.

resolves: #6357
2023-08-07 21:01:02 +05:30
Charlie Marsh
b21abe0a57 Use separate structs for expression and statement tracking (#6351)
## Summary

This PR fixes the performance degradation introduced in
https://github.com/astral-sh/ruff/pull/6345. Instead of using the
generic `Nodes` structs, we now use separate `Statement` and
`Expression` structs. Importantly, we can avoid tracking a bunch of
state for expressions that we need for parents: we don't need to track
reference-to-ID pointers (we just have no use-case for this -- I'd
actually like to remove this from statements too, but we need it for
branch detection right now), we don't need to track depth, etc.

In my testing, this entirely removes the regression on all-rules, and
gets us down to 2ms slower on the default rules (as a crude hyperfine
benchmark, so this is within margin of error IMO).

No behavioral changes.
2023-08-07 15:27:42 +00:00
Charlie Marsh
61d3977f95 Make the statement vector private on SemanticModel (#6348)
## Summary

Instead, expose these as methods, now that we can use a reasonable
nomenclature on the API.
2023-08-07 15:02:14 +00:00
Charlie Marsh
bae87fa016 Rename semantic model methods to use current_* prefix (#6347)
## Summary

This PR attempts to draw a clearer divide between "methods that take
(e.g.) an expression or statement as input" and "methods that rely on
the _current_ expression or statement" in the semantic model, by
renaming methods like `stmt()` to `current_statement()`.

This had led to confusion in the past. For example, prior to this PR, we
had `scope()` (which returns the current scope), and `parent_scope`,
which returns the parent _of a scope that's passed in_. Now, the API is
clearer: `current_scope` returns the current scope, and `parent_scope`
takes a scope as argument and returns its parent.

Per above, I also changed `stmt` to `statement` and `expr` to
`expression`.
2023-08-07 14:44:49 +00:00
Charlie Marsh
b763973357 Avoid hard line break after dangling open-parenthesis comments (#6380)
## Summary

Given:

```python
[  # comment
    first,
    second,
    third
]  # another comment
```

We were adding a hard line break as part of the formatting of `#
comment`, which led to the following formatting:

```python
[first, second, third]  # comment
  # another comment
```

Closes https://github.com/astral-sh/ruff/issues/6367.
2023-08-07 14:15:32 +00:00
Charlie Marsh
63692b3798 Use parenthesized_with_dangling_comments in arguments formatter (#6376)
## Summary

Fixes an instability whereby this:

```python
def get_recent_deployments(threshold_days: int) -> Set[str]:
    # Returns a list of deployments not older than threshold days
    # including `/root/zulip` directory if it exists.
    recent = set()
    threshold_date = datetime.datetime.now() - datetime.timedelta(  # noqa: DTZ005
        days=threshold_days
    )
```

Was being formatted as:

```python
def get_recent_deployments(threshold_days: int) -> Set[str]:
    # Returns a list of deployments not older than threshold days
    # including `/root/zulip` directory if it exists.
    recent = set()
    threshold_date = (
        datetime.datetime.now()
        - datetime.timedelta(days=threshold_days)  # noqa: DTZ005
    )
```

Which was in turn being formatted as:

```python
def get_recent_deployments(threshold_days: int) -> Set[str]:
    # Returns a list of deployments not older than threshold days
    # including `/root/zulip` directory if it exists.
    recent = set()
    threshold_date = (
        datetime.datetime.now() - datetime.timedelta(days=threshold_days)  # noqa: DTZ005
    )
```

The second-to-third formattings still differs from Black because we
aren't taking the line suffix into account when splitting
(https://github.com/astral-sh/ruff/issues/6377), but the first
formatting is correct and should be unchanged (i.e., the first-to-second
formattings is incorrect, and fixed here).

## Test Plan

`cargo run --bin ruff_dev -- format-dev --stability-check ../zulip`
2023-08-07 09:43:57 -04:00
Charlie Marsh
89e4e038b0 Store expression hierarchy in semantic model snapshots (#6345)
## Summary

When we iterate over the AST for analysis, we often process nodes in a
"deferred" manner. For example, if we're analyzing a function, we push
the function body onto a deferred stack, along with a snapshot of the
current semantic model state. Later, when we analyze the body, we
restore the semantic model state from the snapshot. This ensures that we
know the correct scope, hierarchy of statement parents, etc., when we go
to analyze the function body.

Historically, we _haven't_ included the _expression_ hierarchy in the
model snapshot -- so we track the current expression parents in the
visitor, but we never save and restore them when processing deferred
nodes. This can lead to subtle bugs, in that methods like
`expr_parent()` aren't guaranteed to be correct, if you're in a deferred
visitor.

This PR migrates expression tracking to mirror statement tracking
exactly. So we push all expressions onto an `IndexVec`, and include the
current expression on the snapshot. This ensures that `expr_parent()`
and related methods are "always correct" rather than "sometimes
correct".

There's a performance cost here, both at runtime and in terms of memory
consumption (we now store an additional pointer for every expression).
In my hyperfine testing, it's about a 1% performance decrease for
all-rules on CPython (up to 533.8ms, from 528.3ms) and a 4% performance
decrease for default-rules on CPython (up to 212ms, from 204ms).
However... I think this is worth it given the incorrectness of our
current approach. In the future, we may want to reconsider how we do
these upward traversals (e.g., with something like a red-green tree).
(**Note**: in https://github.com/astral-sh/ruff/pull/6351, the slowdown
seems to be entirely removed.)
2023-08-07 09:42:04 -04:00
Tom Kuson
5d2a4ebc99 Add documentation to subprocess-with[out]-shell-equals-true rules (#6373) 2023-08-07 03:48:36 +00:00
Harutaka Kawamura
9c3fbcdf4a Add PT011 and PT012 docs (#6362) 2023-08-06 21:28:24 -04:00
Konrad Listwan-Ciesielski
61532e8aad Add DTZ003 and DTZ004 docs (#6223)
Changes:
- Fixes typo and repeated phrase in `DTZ002`
- Adds docs for `DTZ003`
- Adds docs for `DTZ004`
- Adds example for <=Python3.10 in `DTZ001`

Related to: https://github.com/astral-sh/ruff/issues/2646
2023-08-07 01:21:14 +00:00
Charlie Marsh
9171e97d15 Avoid allocation in no-signature (#6375) 2023-08-06 15:27:56 +00:00
Charlie Marsh
a5a29bb8d6 Revert change to require_git(false) in WalkBuilder (#6368)
## Summary

This was changed to fix https://github.com/astral-sh/ruff/issues/5930
(respect `.gitignore` for unzipped source repositories), but led to
undesirable behavior whereby `.gitignore` files in parent directories
are respected regardless of whether you're working in a child git
repository (see: https://github.com/astral-sh/ruff/issues/6335). The
latter is a bigger problem than the former is an important use-case to
support, so pragmatically erring on the side of a revert.

Closes https://github.com/astral-sh/ruff/issues/6335.
2023-08-05 19:45:50 +00:00
Zixuan Li
be657f5e7e Respect typing_extensions imports of Annotated for B006. (#6361)
`typing_extensions.Annotated` should be treated the same way as
`typing.Annotated`.
2023-08-05 17:39:52 +00:00
Charlie Marsh
76148ddb76 Store call paths rather than stringified names (#6102)
## Summary

Historically, we've stored "qualified names" on our
`BindingKind::Import`, `BindingKind::SubmoduleImport`, and
`BindingKind::ImportFrom` structs. In Ruff, a "qualified name" is a
dot-separated path to a symbol. For example, given `import foo.bar`, the
"qualified name" would be `"foo.bar"`; and given `from foo.bar import
baz`, the "qualified name" would be `foo.bar.baz`.

This PR modifies the `BindingKind` structs to instead store _call paths_
rather than qualified names. So in the examples above, we'd store
`["foo", "bar"]` and `["foo", "bar", "baz"]`. It turns out that this
more efficient given our data access patterns. Namely, we frequently
need to convert the qualified name to a call path (whenever we call
`resolve_call_path`), and it turns out that we do this operation enough
that those conversations show up on benchmarks.

There are a few other advantages to using call paths, rather than
qualified names:

1. The size of `BindingKind` is reduced from 32 to 24 bytes, since we no
longer need to store a `String` (only a boxed slice).
2. All three import types are more consistent, since they now all store
a boxed slice, rather than some storing an `&str` and some storing a
`String` (for `BindingKind::ImportFrom`, we needed to allocate a
`String` to create the qualified name, but the call path is a slice of
static elements that don't require that allocation).
3. A lot of code gets simpler, in part because we now do call path
resolution "earlier". Most notably, for relative imports (`from .foo
import bar`), we store the _resolved_ call path rather than the relative
call path, so the semantic model doesn't have to deal with that
resolution. (See that `resolve_call_path` is simpler, fewer branches,
etc.)

In my testing, this change improves the all-rules benchmark by another
4-5% on top of the improvements mentioned in #6047.
2023-08-05 15:21:50 +00:00
Harutaka Kawamura
501f537cb8 Avoid auto-fixing UP031 if there are comments within the right-hand side (#6364) 2023-08-05 11:14:29 -04:00
Dhruv Manilawala
1ac2699b5e Update F841 autofix to not remove line magic expr (#6141)
## Summary

Update `F841` autofix to not remove line magic expr

## Test Plan

Added test case for assignment statement with and without type
annotation

fixes: #6116
2023-08-05 00:45:01 +00:00
Dhruv Manilawala
32fa05765a Use Jupyter mode while parsing Notebook files (#5552)
## Summary

Enable using the new `Mode::Jupyter` for the tokenizer/parser to parse
Jupyter line magic tokens.

The individual call to the lexer i.e., `lex_starts_at` done by various
rules should consider the context of the source code (is this content
from a Jupyter Notebook?). Thus, a new field `source_type` (of type
`PySourceType`) is added to `Checker` which is being passed around as an
argument to the relevant functions. This is then used to determine the
`Mode` for the lexer.

## Test Plan

Add new test cases to make sure that the magic statement is considered
while generating the diagnostic and autofix:
* For `I001`, if there's a magic statement in between two import blocks,
they should be sorted independently

fixes: #6090
2023-08-05 00:32:07 +00:00
Charlie Marsh
d788957ec4 Allow capitalized names for logger candidate heuristic match (#6356)
Closes https://github.com/astral-sh/ruff/issues/6353.
2023-08-04 23:25:34 +00:00
Victor Hugo Gomes
78a370303b [flake8-pyi] Add tests cases for bad imports from PYI027 to PYI022 (UP035) (#6354)
## Summary
As of version
[23.1.0](2a86db8271/CHANGELOG.md (L158-L160)),
`flake8-pyi` remove the rule `Y027`.

The errors that resulted in `PYI027` are now being emitted by `PYI022`
(`UP035`).

ref: #848 

## Test Plan

Add new tests cases.
2023-08-04 19:00:33 -04:00
Charlie Marsh
5e73345a1c Avoid panic with positional-only arguments in PYI019 (#6350)
## Summary

Previously, failed on methods like:

```python
@classmethod
def bad_posonly_class_method(cls: type[_S], /) -> _S: ...  # PYI019
```

Since we check if there are any positional-only or non-positional
arguments, but then do an unsafe access on `parameters.args`.

Closes https://github.com/astral-sh/ruff/issues/6349.

## Test Plan

`cargo test` (verified that `main` panics on the new fixtures)
2023-08-04 18:37:07 +00:00
Charlie Marsh
b8fd69311c Remove ruff_python_ast prefix in fixes.rs (#6346) 2023-08-04 16:48:20 +00:00
Charlie Marsh
fa5c9cced9 Ignore same-line docstrings for lines-before and lines-after rules (#6344)
These rules assume that the docstring is on its own line. pydocstyle
treats them inconsistently, so I'm just going to disable them in this
case.

Closes https://github.com/astral-sh/ruff/issues/6329.
2023-08-04 16:08:36 +00:00
Harutaka Kawamura
08dd87e04d Avoid auto-fixing UP032 if comments are present around format call arguments (#6342) 2023-08-04 15:37:23 +00:00
konsti
9bb21283ca More similarity index digits (#6343)
**Summary** We were at similarity index 0.998 for django, we need more
decimal places, now we're at 0.99779.

**Test Plan** n/a
2023-08-04 17:12:33 +02:00
Charlie Marsh
4d47dfd6c0 Tweak breaking groups for comprehensions (#6321)
## Summary

Fixes some comprehension formatting by avoiding creating the group for
the comprehension itself (so that if it breaks, all parts break on their
own lines, e.g. the `for` and the `if` clauses).

Closes https://github.com/astral-sh/ruff/issues/6063.

## Test Plan

Bunch of new fixtures.
2023-08-04 14:00:54 +00:00
konsti
99baad12d8 Call chain formatting in fluent style (#6151)
Implement fluent style/call chains. See the `call_chains.py` formatting
for examples.

This isn't fully like black because in `raise A from B` they allow `A`
breaking can influence the formatting of `B` even if it is already
multiline.

Similarity index:

| project      | main  | PR    |
|--------------|-------|-------|
| build        | ???   | 0.753 |
| django       | 0.991 | 0.998 |
| transformers | 0.993 | 0.994 |
| typeshed     | 0.723 | 0.723 |
| warehouse    | 0.978 | 0.994 |
| zulip        | 0.992 | 0.994 |

Call chain formatting is affected by
https://github.com/astral-sh/ruff/issues/627, but i'm cutting scope
here.

Closes #5343

**Test Plan**:
 * Added a dedicated call chains test file
 * The ecosystem checks found some bugs
 * I manually check django and zulip formatting

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2023-08-04 13:58:01 +00:00
Charlie Marsh
35bdbe43a8 Flag comparison-with-itself on builtin calls (#6324)
## Summary

Extends `comparison-with-itself` to cover simple function calls on
known-pure functions, like `id`. For example, we now flag `id(x) ==
id(x)`.

Closes https://github.com/astral-sh/ruff/issues/6276.

## Test Plan

`cargo test`
2023-08-04 09:51:41 -04:00
Charlie Marsh
3a985dd71e Rename CommentPlacement#then_with to or_else (#6341)
Per nits in the PR.
2023-08-04 13:50:57 +00:00
Charlie Marsh
1e3fe67ca5 Refactor and rename skip_trailing_trivia (#6312)
Based on feedback here:
https://github.com/astral-sh/ruff/pull/6274#discussion_r1282747964.
2023-08-04 13:30:53 +00:00
Charlie Marsh
38a96c88c1 Add missing enable check for bad-string-format-character (#6340) 2023-08-04 13:27:53 +00:00
Micha Reiser
f4831d5a26 Formatter comment handling nits (#6339) 2023-08-04 13:22:16 +00:00
konsti
1031bb6550 Formatter: Add SourceType to context to enable special formatting for stub files (#6331)
**Summary** This adds the information whether we're in a .py python
source file or in a .pyi stub file to enable people working on #5822 and
related issues.

I'm not completely happy with `Default` for something that depends on
the input.

**Test Plan** None, this is currently unused, i'm leaving this to first
implementation of stub file specific formatting.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2023-08-04 11:52:26 +00:00
David Szotten
fe97a2a302 Fix panic with empty attribute inner comment (#6332)
Fixes https://github.com/astral-sh/ruff/issues/6181
2023-08-04 11:59:55 +02:00
konsti
a48d16e025 Replace Formatter<PyFormatContext<'_>> with PyFormatter (#6330)
This is a refactoring to use the type alias in more places. In the
process, I had to fix and run generate.py. There are no functional
changes.
2023-08-04 10:48:58 +02:00
Charlie Marsh
8a5bc93fdd Make the Nodes vector generic on node type (#6328) 2023-08-04 03:57:15 +00:00
Charlie Marsh
6da527170f Match left-hand side types() call in types-comparison (#6326)
Follow-up to https://github.com/astral-sh/ruff/pull/6325, to avoid false
positives in cases like:

```python
if x == int:
    ...
```

Which is valid, since we don't know that we're comparing the type _of_
something -- we're comparing the type objects directly.
2023-08-03 23:01:23 -04:00
Charlie Marsh
8cddb6c08d Include comparisons to builtin types in type-comparison rule (#6325)
## Summary

Extends `type-comparison` to flag:

```python
if type(obj) is int:
    pass
```

In addition to the existing cases, like:

```python
if type(obj) is type(1):
    pass
```

Closes https://github.com/astral-sh/ruff/issues/6260.
2023-08-04 02:25:19 +00:00
Victor Hugo Gomes
b8ca220eeb [flake8-pyi] Implement PYI055 (#6316) 2023-08-04 01:36:00 +00:00
Charlie Marsh
1d8759d5df Generalize comment-after-bracket handling to lists, sets, etc. (#6320)
## Summary

We already support preserving the end-of-line comment in calls and type
parameters, as in:

```python
foo(  # comment
    bar,
)
```

This PR adds the same behavior for lists, sets, comprehensions, etc.,
such that we preserve:

```python
[  # comment
    1,
    2,
    3,
]
```

And related cases.
2023-08-04 01:28:05 +00:00
Charlie Marsh
d3aa8b4ee0 Add API to chain comment placement operations (#6319)
## Summary

This PR adds an API for chaining comment placement methods based on the
[`then_with`](https://doc.rust-lang.org/std/cmp/enum.Ordering.html#method.then_with)
from `Ordering` in the standard library.

For example, you can now do:

```rust
try_some_case(comment).then_with(|comment| try_some_other_case_if_still_default(comment))
```

This lets us avoid this kind of pattern, which I've seen in
`placement.rs` and used myself before:

```rust
let comment = match handle_own_line_comment_between_branches(comment, preceding, locator) {
    CommentPlacement::Default(comment) => comment,
    placement => return placement,
};
```
2023-08-03 21:08:50 -04:00
Zanie Blue
9ae498595c Upgrade Rust to 1.71 (#6323)
Addresses
[CVE-2023-38497](https://blog.rust-lang.org/2023/08/03/cve-2023-38497.html)

See also the [version release
post](https://blog.rust-lang.org/2023/08/03/Rust-1.71.1.html)
2023-08-03 21:08:39 -04:00
Charlie Marsh
5f225b18ab Generalize bracketed end-of-line comment handling (#6315)
Micha suggested this in
https://github.com/astral-sh/ruff/pull/6274#discussion_r1282774151, and
it allows us to unify the implementations for arguments and type params.
2023-08-03 20:51:03 +00:00
Charlie Marsh
8276b26480 Omit formatter PRs from releases by default (#6317)
I end up removing these manually every time, seems easier to just omit
them for now.
2023-08-03 20:45:50 +00:00
Charlie Marsh
1705fcef36 Mark trailing comments in parenthesized tests (#6287)
## Summary

This ensures that we treat `# comment` as parenthesized in contexts
like:

```python
while (
    True
    # comment
):
    pass
```

The same logic applies equally to `for`, `async for`, `if`, `with`, and
`async with`. The general pattern is that you have an expression which
precedes a colon-separated suite.
2023-08-03 20:45:03 +00:00
konsti
51ff98f9e9 Make formatter ecosystem check failure output better understandable (#6300)
**Summary** Prompted by
https://github.com/astral-sh/ruff/pull/6257#issuecomment-1661308410, it
tried to make the ecosystem script output on failure better
understandable. All log messages are now written to a file, which is
printed on error. Running locally progress is still shown.

Looking through the log output i saw that we currently log syntax errors
in input, which is confusing because they aren't actual errors, but we
don't check that these files don't change due to parser regressions or
improvements. I added `--files-with-errors` to catch that.

**Test Plan** CI
2023-08-03 20:23:25 +02:00
Charlie Marsh
b3f3529499 Improve comments around Arguments handling in classes (#6310)
## Summary

Based on the confusion here:
https://github.com/astral-sh/ruff/pull/6274#discussion_r1282754515.

I looked into moving this logic into `placement.rs`, but I think it's
trickier than it may appear.
2023-08-03 12:34:03 -04:00
Charlie Marsh
2fa508793f Return a slice in StmtClassDef#bases (#6311)
Slices are strictly more flexible, since you can always convert to an
iterator, etc., but not the other way around. Suggested in
https://github.com/astral-sh/ruff/pull/6259#discussion_r1282730994.
2023-08-03 16:21:55 +00:00
Zanie Blue
718e3945e3 Add rule to upgrade type alias annotations to keyword (UP040) (#6289)
Adds rule to convert type aliases defined with annotations i.e. `x:
TypeAlias = int` to the new PEP-695 syntax e.g. `type x = int`.

Does not support using new generic syntax for type variables, will be
addressed in a follow-up.
Added as part of pyupgrade — ~the code 100 as chosen to avoid collision
with real pyupgrade codes~.

Part of #4617 
Builds on #5062
2023-08-03 16:13:06 +00:00
Charlie Marsh
c75e8a8dab Move ExprCall's NeedsParentheses impl into expr_call.rs (#6309)
Accidental move.
2023-08-03 16:01:01 +00:00
Harutaka Kawamura
74e734e962 More precise invalid expression check for UP032 (#6308) 2023-08-03 15:49:02 +00:00
Zanie Blue
0e18abcf95 Add is_ and is_not to excluded functions for FBT003 (#6307)
These methods are commonly used in SQLAlchemy.

See https://github.com/astral-sh/ruff/discussions/6302
2023-08-03 10:41:45 -05:00
Anders Kaseorg
7c8bcede5b Broaden appropriate flake8-pyi rules to check non-stub code too (#6297)
Of the rules that flake8-pyi enforces for `.pyi` type stubs, many of
them equally make sense to check in normal runtime code with type
annotations. Broaden these rules to check all files:

PYI013 ellipsis-in-non-empty-class-body
PYI016 duplicate-union-member
PYI018 unused-private-type-var
PYI019 custom-type-var-return-type
PYI024 collections-named-tuple
PYI025 unaliased-collections-abc-set-import
PYI030 unnecessary-literal-union
PYI032 any-eq-ne-annotation
PYI034 non-self-return-type
PYI036 bad-exit-annotation
PYI041 redundant-numeric-union
PYI042 snake-case-type-alias
PYI043 t-suffixed-type-alias
PYI045 iter-method-return-iterable
PYI046 unused-private-protocol
PYI047 unused-private-type-alias
PYI049 unused-private-typed-dict
PYI050 no-return-argument-annotation-in-stub (Python ≥ 3.11)
PYI051 redundant-literal-union
PYI056 unsupported-method-call-on-all

The other rules are stub-specific and remain enabled only in `.pyi`
files.

PYI001 unprefixed-type-param
PYI002 complex-if-statement-in-stub
PYI003 unrecognized-version-info-check
PYI004 patch-version-comparison
PYI005 wrong-tuple-length-version-comparison (could make sense to
broaden, see
https://github.com/astral-sh/ruff/pull/6297#issuecomment-1663314807)
PYI006 bad-version-info-comparison (same)
PYI007 unrecognized-platform-check
PYI008 unrecognized-platform-name
PYI009 pass-statement-stub-body
PYI010 non-empty-stub-body
PYI011 typed-argument-default-in-stub
PYI012 pass-in-class-body
PYI014 argument-default-in-stub
PYI015 assignment-default-in-stub
PYI017 complex-assignment-in-stub
PYI020 quoted-annotation-in-stub
PYI021 docstring-in-stub
PYI026 type-alias-without-annotation (could make sense to broaden, but
gives many false positives on runtime code as currently implemented)
PYI029 str-or-repr-defined-in-stub
PYI033 type-comment-in-stub
PYI035 unassigned-special-variable-in-stub
PYI044 future-annotations-in-stub
PYI048 stub-body-multiple-statements
PYI052 unannotated-assignment-in-stub
PYI053 string-or-bytes-too-long
PYI054 numeric-literal-too-long

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2023-08-03 11:40:42 -04:00
Harutaka Kawamura
30c2e9430e Update UP032 to support await expressions (#6304)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

In Python >= 3.7, `await` can be included in f-strings. 

https://bugs.python.org/issue28942

## Test Plan

<!-- How was it tested? -->

Existing tests
2023-08-03 09:53:36 -05:00
Harutaka Kawamura
b6f0316d55 Add PT013 and PT015 docs (#6303)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

#2646

## Test Plan

<!-- How was it tested? -->
2023-08-03 09:51:52 -05:00
Charlie Marsh
9f3567dea6 Use range: _ in lieu of range: _range (#6296)
## Summary

`range: _range` is slightly inconvenient because you can't use it
multiple times within a single match, unlike `_`.
2023-08-02 22:11:13 -04:00
Charlie Marsh
9e2bbf4beb Add a simple tooltip to the sidebar (#6295)
## Summary

Not perfect, but IMO helpful:

<img width="1792" alt="Screen Shot 2023-08-02 at 9 29 24 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/e613e918-75cb-475e-9ea4-f833d1a0b5f6">

<img width="1792" alt="Screen Shot 2023-08-02 at 9 29 20 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/bb3efdfe-40e1-45b5-b774-082521b2d214">
2023-08-03 01:41:07 +00:00
Charlie Marsh
d7627c398c Add an icon for FIR (#6292)
It's not a very _good_ icon, but I prefer the consistency. I'm also
going to add tooltips to these.
2023-08-03 01:20:00 +00:00
Charlie Marsh
23e527e386 Increase icon opacity on-hover (#6291)
## Summary

Makes it clearer that these are clickable.
2023-08-03 01:05:38 +00:00
Charlie Marsh
a15b0a9102 Tweak background on theme button (#6290)
## Summary

It's now white on-hover as opposed to yellow, to match the copy button:

<img width="1792" alt="Screen Shot 2023-08-02 at 8 52 10 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/96d5cbf9-ef33-4fba-8888-f2a4af9a6ec4">
2023-08-03 01:00:37 +00:00
qdegraaf
d40597a266 [flake8-pyi] Implement custom_type_var_return_type (PYI019) (#6204)
## Summary

Implements `Y019` from
[flake8-pyi](https://github.com/PyCQA/flake8-pyi).

The rule checks if

-  instance methods that return `self` 
-  class methods that return an instance of `cls`
- `__new__` methods

Return a custom `TypeVar` instead of `typing.Self` and raises a
violation if this is the case. The rule also covers
[PEP-695](https://peps.python.org/pep-0695/) syntax as introduced in
upstream in https://github.com/PyCQA/flake8-pyi/pull/402

## Test Plan

Added fixtures with test cases from upstream implementation (plus
additional one for an excluded edge case, mentioned in upstream
implementation)
2023-08-03 00:42:42 +00:00
Silvano Cerza
82410524d9 [pylint] Implement Pylint bad-format-character (E1300) (#6171)
## Summary

Relates to #970.

Add new `bad-format-character` Pylint rule.

I had to make a change in `crates/ruff_python_literal/src/format.rs` to
get a more detailed error in case the format character is not correct. I
chose to do this since most of the format spec parsing functions are
private. It would have required me reimplementing most of the parsing
logic just to know if the format char was correct.

This PR also doesn't reflect current Pylint functionality in two ways.

It supports new format strings correctly, Pylint as of now doesn't. See
pylint-dev/pylint#6085.

In case there are multiple adjacent string literals delimited by
whitespace the index of the wrong format char will relative to the
single string. Pylint will instead reported it relative to the
concatenated string.

Given this:
```
"%s" "%z" % ("hello", "world")
```

Ruff will report this:
```Unsupported format character 'z' (0x7a) at index 1```

Pylint instead:
```Unsupported format character 'z' (0x7a) at index 3```

I believe it's more sensible to report the index relative to the
individual string.

## Test Plan

Added new snapshot and a small test in
`crates/ruff_python_literal/src/format.rs`.

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2023-08-02 21:32:43 +00:00
Zanie Blue
5b2e973fa5 Add formatting of type alias statements (#6162)
Part of #5062 
Extends https://github.com/astral-sh/ruff/pull/6161
Closes #5929
2023-08-02 20:40:32 +00:00
Zanie Blue
1a60d1e3c6 Add formatting of type parameters in class and function definitions (#6161)
Part of #5062 
Closes https://github.com/astral-sh/ruff/issues/5931

Implements formatting of a sequence of type parameters in a dedicated
struct for reuse by classes, functions, and type aliases (preparing for
#5929). Adds formatting of type parameters in class and function
definitions — previously, they were just elided.
2023-08-02 20:29:28 +00:00
Charlie Marsh
9425ed72a0 Break global and nonlocal statements over continuation lines (#6172)
## Summary

Builds on #6170 to break `global` and `nonlocal` statements, such that
we get:

```python
def f():
    global \
        analyze_featuremap_layer, \
        analyze_featuremapcompression_layer, \
        analyze_latencies_post, \
        analyze_motions_layer, \
        analyze_size_model
```

Instead of:

```python
def f():
    global analyze_featuremap_layer, analyze_featuremapcompression_layer, analyze_latencies_post, analyze_motions_layer, analyze_size_model
```

Notably, we avoid applying this formatting if the statement ends in a
comment. Otherwise, the comment would _need_ to be placed after the last
item, like:

```python
def f():
    global \
        analyze_featuremap_layer, \
        analyze_featuremapcompression_layer, \
        analyze_latencies_post, \
        analyze_motions_layer, \
        analyze_size_model  # noqa
```

To me, this seems wrong (and would break the `# noqa` comment). Ideally,
the items would be parenthesized, and the comment would be on the inner
parenthesis, like:

```python
def f():
    global (  # noqa
        analyze_featuremap_layer,
        analyze_featuremapcompression_layer,
        analyze_latencies_post,
        analyze_motions_layer,
        analyze_size_model
    )
```

But that's not valid syntax.
2023-08-02 19:55:00 +00:00
Victor Hugo Gomes
9f38dbd06e [flake8-pyi] Implement PYI051 (#6215)
## Summary
Checks for the presence of redundant `Literal` types and builtin super
types in an union. See [original
source](2a86db8271/pyi.py (L1261)).

This implementation has a couple of differences from the original. The
first one is, we support the `complex` and `float` builtin types. The
second is, when reporting diagnostic for a `Literal` with multiple
members of the same type, we print the entire `Literal` while `flak8`
only prints the `Literal` with its first member.
For example:
```python
from typing import Literal

x: Literal[1, 2] | int
```  
Ruff will show `Literal[1, 2]` while flake8 only shows `Literal[1]`.

```shell
$ ruff crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:4:18: PYI051 `Literal["foo"]` is redundant in an union with `str`
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:5:37: PYI051 `Literal[b"bar", b"foo"]` is redundant in an union with `bytes`
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:6:37: PYI051 `Literal[5]` is redundant in an union with `int`
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:6:67: PYI051 `Literal["foo"]` is redundant in an union with `str`
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:7:37: PYI051 `Literal[b"str_bytes"]` is redundant in an union with `bytes`
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:7:51: PYI051 `Literal[42]` is redundant in an union with `int`
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:9:31: PYI051 `Literal[1J]` is redundant in an union with `complex`
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:9:53: PYI051 `Literal[3.14]` is redundant in an union with `float`
Found 8 errors.
```

```shell
$ flake8 crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:4:18: Y051 "Literal['foo']" is redundant in a union with "str"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:5:37: Y051 "Literal[b'bar']" is redundant in a union with "bytes"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:6:37: Y051 "Literal[5]" is redundant in a unionwith "int"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:6:67: Y051 "Literal['foo']" is redundant in a union with "str"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:7:37: Y051 "Literal[b'str_bytes']" is redundantin a union with "bytes"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:7:51: Y051 "Literal[42]" is redundant in a union with "int"
```

While implementing this rule, I found a bug in the `is_unchecked_union`
check. This is the new check.


1ab86bad35/crates/ruff/src/checkers/ast/analyze/expression.rs (L85-L102)

The purpose of the check was to prevent rules from navigating through
nested `Union`s, as they already handle nested `Union`s. The way it was
implemented, this was not happening, the rules were getting executed
more than one time and sometimes were receiving expressions that were
not `Union`. For example, with the following code:
 ```python
  typing.Union[Literal[5], int, typing.Union[Literal["foo"], str]]
 ```

The rules were receiving the expressions in the following order:
- `typing.Union[Literal[5], int, typing.Union[Literal["foo"], str]]`
     - `Literal[5]`
     - `typing.Union[Literal["foo"], str]]`

This was causing `PYI030` to report redundant information, for example:
 ```python
typing.Union[Literal[5], int, typing.Union[Literal["foo"],
Literal["bar"]]]
 ```
This is the `PYI030` output for this code:
```shell
PYI030 Multiple literal members in a union. Use a single literal, e.g. `Literal[5, "foo", "bar"]`
YI030 Multiple literal members in a union. Use a single literal, e.g.`Literal[5, "foo"]`
```

If I haven't misinterpreted the rule, that looks incorrect. I didn't
have the time to check the `PYI016` rule.

The last thing is, I couldn't find a reason for the "Why is this bad?"
section for `PYI051`.

Ref: #848 

## Test Plan

Snapshots and manual runs of flake8.
\
2023-08-02 15:37:40 -04:00
Victor Hugo Gomes
7c5791fb77 Fix formatting of lambda star arguments (#6257)
## Summary
Previously, the ruff formatter was removing the star argument of
`lambda` expressions when formatting.

Given the following code snippet
```python
lambda *a: ()
lambda **b: ()
```
it would be formatted to
```python
lambda: ()
lambda: ()
```

We fix this by checking for the presence of `args`, `vararg` or `kwarg`
in the `lambda` expression, before we were only checking for the
presence of `args`.

Fixes #5894

## Test Plan

Add new tests cases.

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2023-08-02 19:31:20 +00:00
Harutaka Kawamura
c362ea7fd4 Add PT025 and PT026 docs (#6264) 2023-08-02 19:00:03 +00:00
Harutaka Kawamura
ec8fad5b02 Extend UP032 to support implicitly concatenated strings (#6263) 2023-08-02 18:56:24 +00:00
Harutaka Kawamura
bcc41ba062 Extend UP032 to support repeated format fields (#6266) 2023-08-02 14:23:25 -04:00
Charlie Marsh
556abf4bd3 Avoid PTH206 with maxsplit (#6283)
## Summary

Avoid suggesting `Path.parts` when a `maxsplit` is specified, since
these behavior differently.

## Test Plan

`cargo test`
2023-08-02 18:16:57 +00:00
Charlie Marsh
23b8fc4366 Move includes_arg_name onto Parameters (#6282)
## Summary

Like #6279, no reason for this to be a standalone method.
2023-08-02 18:05:26 +00:00
Charlie Marsh
fd40864924 Move find_keyword helpers onto Arguments struct (#6280)
## Summary

Similar to #6279, moving some helpers onto the struct in the name of
reducing the number of random undiscoverable utilities we have in
`helpers.rs`.

Most of the churn is migrating rules to take `ast::ExprCall` instead of
the spread call arguments.

## Test Plan

`cargo test`
2023-08-02 13:54:48 -04:00
Charlie Marsh
041946fb64 Remove CallArguments abstraction (#6279)
## Summary

This PR removes a now-unnecessary abstraction from `helper.rs`
(`CallArguments`), in favor of adding methods to `Arguments` directly,
which helps with discoverability.
2023-08-02 13:25:43 -04:00
Charlie Marsh
8a0f844642 Box type params and arguments fields on the class definition node (#6275)
## Summary

This PR boxes the `TypeParams` and `Arguments` fields on the class
definition node. These fields are optional and often emitted, and given
that class definition is our largest enum variant, we pay the cost of
including them for every statement in the AST. Boxing these types
reduces the statement size by 40 bytes, which seems like a good tradeoff
given how infrequently these are accessed.

## Test Plan

Need to benchmark, but no behavior changes.
2023-08-02 16:47:06 +00:00
Charlie Marsh
8c40886f87 Use Arguments node to power remove_argument method (#6278)
## Summary

Internal refactor to take advantage of the new `Arguments` node, to
power our `remove_argument` fix action.

## Test Plan

`cargo test`
2023-08-02 12:38:43 -04:00
Charlie Marsh
4c53bfe896 Add formatter support for call and class definition Arguments (#6274)
## Summary

This PR leverages the `Arguments` AST node introduced in #6259 in the
formatter, which ensures that we correctly handle trailing comments in
calls, like:

```python
f(
  1,
  # comment
)

pass
```

(Previously, this was treated as a leading comment on `pass`.)

This also allows us to unify the argument handling across calls and
class definitions.

## Test Plan

A bunch of new fixture tests, plus improved Black compatibility.
2023-08-02 11:54:22 -04:00
Charlie Marsh
b095b7204b Add a TypeParams node to the AST (#6261)
## Summary

Similar to #6259, this PR adds a `TypeParams` node to the AST, to
capture the list of type parameters with their surrounding brackets.

If a statement lacks type parameters, the `type_params` field will be
`None`.
2023-08-02 14:12:45 +00:00
Charlie Marsh
981e64f82b Introduce an Arguments AST node for function calls and class definitions (#6259)
## Summary

This PR adds a new `Arguments` AST node, which we can use for function
calls and class definitions.

The `Arguments` node spans from the left (open) to right (close)
parentheses inclusive.

In the case of classes, the `Arguments` is an option, to differentiate
between:

```python
# None
class C: ...

# Some, with empty vectors
class C(): ...
```

In this PR, we don't really leverage this change (except that a few
rules get much simpler, since we don't need to lex to find the start and
end ranges of the parentheses, e.g.,
`crates/ruff/src/rules/pyupgrade/rules/lru_cache_without_parameters.rs`,
`crates/ruff/src/rules/pyupgrade/rules/unnecessary_class_parentheses.rs`).

In future PRs, this will be especially helpful for the formatter, since
we can track comments enclosed on the node itself.

## Test Plan

`cargo test`
2023-08-02 10:01:13 -04:00
Ran Benita
0d62ad2480 Permit ClassVar and Final without subscript in RUF012 (#6273)
Fix #6267.
2023-08-02 12:58:44 +00:00
Harutaka Kawamura
b4f224ecea Fix links in docs (#6265)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

Before:

<img width="1031" alt="Screen Shot 2023-08-02 at 15 57 10"
src="https://github.com/astral-sh/ruff/assets/17039389/171a21d5-01a5-4aa5-8079-4e7f8a59ade8">

After:

<img width="1031" alt="Screen Shot 2023-08-02 at 15 58 03"
src="https://github.com/astral-sh/ruff/assets/17039389/afd1b9b7-89e0-4e38-a4a6-e3255b62f021">


## Test Plan

<!-- How was it tested? -->

Manual inspection
2023-08-02 09:42:25 +02:00
Charlie Marsh
7842c82a0a Preserve end-of-line comments on import-from statements (#6216)
## Summary

Ensures that we keep comments at the end-of-line in cases like:

```python
from foo import (  # comment
  bar,
)
```

Closes https://github.com/astral-sh/ruff/issues/6067.
2023-08-01 18:58:05 +00:00
Charlie Marsh
9c708d8fc1 Rename Parameter#arg and ParameterWithDefault#def fields (#6255)
## Summary

This PR renames...

- `Parameter#arg` to `Parameter#name`
- `ParameterWithDefault#def` to `ParameterWithDefault#parameter` (such
that `ParameterWithDefault` has a `default` and a `parameter`)

## Test Plan

`cargo test`
2023-08-01 14:28:34 -04:00
Charlie Marsh
adc8bb7821 Rename Arguments to Parameters in the AST (#6253)
## Summary

This PR renames a few AST nodes for clarity:

- `Arguments` is now `Parameters`
- `Arg` is now `Parameter`
- `ArgWithDefault` is now `ParameterWithDefault`

For now, the attribute names that reference `Parameters` directly are
changed (e.g., on `StmtFunctionDef`), but the attributes on `Parameters`
itself are not (e.g., `vararg`). We may revisit that decision in the
future.

For context, the AST node formerly known as `Arguments` is used in
function definitions. Formally (outside of the Python context),
"arguments" typically refers to "the values passed to a function", while
"parameters" typically refers to "the variables used in a function
definition". E.g., if you Google "arguments vs parameters", you'll get
some explanation like:

> A parameter is a variable in a function definition. It is a
placeholder and hence does not have a concrete value. An argument is a
value passed during function invocation.

We're thus deviating from Python's nomenclature in favor of a scheme
that we find to be more precise.
2023-08-01 13:53:28 -04:00
Charlie Marsh
a82eb9544c Implement Black's rules around newlines before and after class docstrings (#6209)
## Summary

Black allows up to one blank line _before_ a class docstring, and
enforces one blank line _after_ a class docstring. This PR implements
that handling. The cases in
`crates/ruff_python_formatter/resources/test/fixtures/ruff/statement/class_definition.py`
match Black identically.
2023-08-01 13:33:01 -04:00
Zanie Blue
5e41f2fc7d Tweak pre-commit message (#6243)
Follow-up to https://github.com/astral-sh/ruff/pull/6153
2023-08-01 12:32:14 -05:00
konsti
1df7e9831b Replace .map_or(false, $closure) with .is_some_and(closure) (#6244)
**Summary**
[Option::is_some_and](https://doc.rust-lang.org/stable/std/option/enum.Option.html#method.is_some_and)
and
[Result::is_ok_and](https://doc.rust-lang.org/std/result/enum.Result.html#method.is_ok_and)
are new methods is rust 1.70. I find them way more readable than
`.map_or(false, ...)`.

The changes are `s/.map_or(false,/.is_some_and(/g`, then manually
switching to `is_ok_and` where the value is a Result rather than an
Option.

**Test Plan** n/a^
2023-08-01 19:29:42 +02:00
Zanie Blue
2e1754e5fc Update ecosystem checks for bokeh to 3.3 (#6249)
Bokeh 3.3 is planned for release this month
(https://github.com/bokeh/bokeh/issues/13207) and is their default
branch now
2023-08-01 11:56:58 -05:00
Zanie Blue
67b88803d8 Use prettier to format yaml files in pre-commit (#6250)
Prompted by
https://github.com/astral-sh/ruff/pull/6248#discussion_r1280855848
2023-08-01 16:45:08 +00:00
konsti
ed45fcb1f7 Remove old CI comment (#6246)
We don't build abi3 wheels
2023-08-01 11:35:47 -05:00
Zanie Blue
adf227b8a9 Run ecosystem checks on changes to ecosystem test script (#6248)
e.g. https://github.com/astral-sh/ruff/pull/6245 should probably run
checks before merge
2023-08-01 11:35:29 -05:00
Micha Reiser
debfca3a11 Remove Parse trait (#6235) 2023-08-01 18:35:03 +02:00
Charlie Marsh
83fe103d6e Allow generic tuple and list calls in __all__ (#6247)
## Summary

Allows, e.g., `__all__ = list[str]()`.

Closes https://github.com/astral-sh/ruff/issues/6226.
2023-08-01 12:01:48 -04:00
Charlie Marsh
e08f873077 Add Poetry and FastAPI to ecosystem checks (#6245)
Poetry in particular would be useful to avoid issues like
https://github.com/astral-sh/ruff/issues/6233.
2023-08-01 11:48:34 -04:00
Charlie Marsh
928ab63a64 Add empty lines before nested functions and classes (#6206)
## Summary

This PR ensures that if a function or class is the first statement in a
nested suite that _isn't_ a function or class body, we insert a leading
newline.

For example, given:

```python
def f():
    if True:

        def register_type():
            pass
```

We _want_ to preserve the newline, whereas today, we remove it.

Note that this only applies when the function or class doesn't have any
leading comments.

Closes https://github.com/astral-sh/ruff/issues/6066.
2023-08-01 15:30:59 +00:00
Harutaka Kawamura
b68f76f0d9 Add pre-commit install in CONTRIBUTING.md (#6153)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

To enable the git hooks, `pre-commit install` needs to be executed.

## Test Plan

<!-- How was it tested? -->

N/A
2023-08-01 09:28:50 -05:00
Charlie Marsh
1a85953129 Don't require docstrings in .pyi files (#6239)
Closes https://github.com/astral-sh/ruff/issues/6224.
2023-08-01 10:02:57 -04:00
Charlie Marsh
743118ae9a Bump version to 0.0.282 (#6241) 2023-08-01 13:21:33 +00:00
Charlie Marsh
0753017cf1 Revert "Expand scope of quoted-annotation rule (#5766)" (#6237)
This is causing some problems, so we'll just revert for now.

Closes https://github.com/astral-sh/ruff/issues/6189.
2023-08-01 09:03:02 -04:00
Charlie Marsh
29fb655e04 Fix logger-objects documentation (#6238)
Closes https://github.com/astral-sh/ruff/issues/6234.
2023-08-01 12:57:56 +00:00
Micha Reiser
f45e8645d7 Remove unused parser modes
<!--
Thank you for contributing to Ruff! To help us out with reviewing, please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

This PR removes the `Interactive` and `FunctionType` parser modes that are unused by ruff

<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

`cargo test`

<!-- How was it tested? -->
2023-08-01 13:10:07 +02:00
Micha Reiser
7c7231db2e Remove unsupported type_comment field
<!--
Thank you for contributing to Ruff! To help us out with reviewing, please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

This PR removes the `type_comment` field which our parser doesn't support.

<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

`cargo test`

<!-- How was it tested? -->
2023-08-01 12:53:13 +02:00
Micha Reiser
4ad5903ef6 Delete type-ignore node
<!--
Thank you for contributing to Ruff! To help us out with reviewing, please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

This PR removes the type ignore node from the AST because our parser doesn't support it, and just having it around is confusing.

<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

`cargo build`

<!-- How was it tested? -->
2023-08-01 12:34:50 +02:00
konsti
c6986ac95d Consistent CommentPlacement conversion signatures (#6231)
**Summary** Allow passing any node to `CommentPlacement::{leading,
trailing, dangling}` without manually converting. Conversely, Restrict
the comment to the only type we actually pass.

**Test Plan** No changes.
2023-08-01 12:01:17 +02:00
Micha Reiser
ecfdd8d58b Add static assertions to nodes (#6228) 2023-08-01 11:54:49 +02:00
David Szotten
07468f8be9 format ExprJoinedStr (#5932) 2023-08-01 08:26:30 +02:00
David Szotten
ba990b676f add DebugText for self-documenting f-strings (#6167) 2023-08-01 07:55:03 +02:00
Harutaka Kawamura
44a8d1c644 Add PT021, PT022 and PT023 docs (#6143) 2023-08-01 00:41:54 -04:00
Charlie Marsh
88b984e885 Avoid detecting continuations at non-start-of-line (#6219)
## Summary

Previously, given:

```python
a = \
  5;
```

When detecting continuations starting at the offset of the `;`, we'd
flag the previous line as a continuation. We should only flag a
continuation if there isn't leading content prior to the offset.

Closes https://github.com/astral-sh/ruff/issues/6214
2023-08-01 00:20:29 -04:00
Charlie Marsh
bf584c6d74 Remove use of SmallVec in unnecessary-literal-union (#6221)
I prefer to use this on an as-needed basis.
2023-08-01 04:03:58 +00:00
Konrad Listwan-Ciesielski
6ea3c178fd Add DTZ002 documentation (#6146)
## Summary

Adds documentation for DTZ002. Related to
https://github.com/astral-sh/ruff/issues/2646.

## Test Plan

`python scripts/test_docs_formatted.py`
2023-08-01 04:00:50 +00:00
Charlie Marsh
764d35667f Avoid PERF401 false positive on list access in loop (#6220)
Closes https://github.com/astral-sh/ruff/issues/6210.
2023-08-01 03:56:53 +00:00
Charlie Marsh
ff9ebbaa5f Skip trivia when searching for named exception (#6218)
Closes https://github.com/astral-sh/ruff/issues/6213.
2023-08-01 03:42:30 +00:00
Micha Reiser
38b5726948 formatter: WithNodeLevel helper (#6212) 2023-07-31 21:22:17 +00:00
Charlie Marsh
615337a54d Remove newline-insertion logic from JoinNodesBuilder (#6205)
## Summary

This PR moves the "insert empty lines" behavior out of
`JoinNodesBuilder` and into the `Suite` formatter. I find it a little
confusing that the logic is split between those two formatters right
now, and since this is _only_ used in that one place, IMO it is a bit
simpler to just inline it and use a single approach to tracking state
(right now, both are stateful).

The only other place this was used was for decorators. As a side effect,
we now remove blank lines in both of these cases, which is a known but
intentional deviation from Black (which preserves the empty line before
the comment in the first case):

```python
@foo

# Hello
@bar
def baz():
    pass

@foo

@bar
def baz():
    pass
```
2023-07-31 16:58:15 -04:00
Charlie Marsh
6ee5cb37c0 Reset model state when exiting deferred visitors (#6208)
## Summary

Very subtle bug related to the AST traversal. Given:

```python
from __future__ import annotations

from logging import getLogger

__all__ = ("getLogger",)


def foo() -> None:
    pass
```

We end up visiting the `-> None` annotation, then reusing the state
snapshot when we go to visit the `__all__` exports, so when we visit
`"getLogger"`, we think we're inside of a deferred type annotation.

This PR changes all the deferred visitors to snapshot and restore the
state, which is a lot safer -- that way, the visitors avoid modifying
the current visitor state. (Previously, they implicitly left the visitor
state set to the state of the _last_ thing they visited.)

Closes https://github.com/astral-sh/ruff/issues/6207.
2023-07-31 19:46:52 +00:00
konsti
0fddb31235 Use tracing for format_dev (#6177)
## Summary

[tracing](https://github.com/tokio-rs/tracing) is library for logging,
tracing and related features that has a large ecosystem. Using
[tracing-subscriber](https://docs.rs/tracing-subscriber) and
[tracing-indicatif](https://github.com/emersonford/tracing-indicatif),
we get a nice logging output that you can configure with `RUST_LOG`
(e.g. `RUST_LOG=debug`) and a live look into the formatter progress.

Default:
![Screenshot from 2023-07-30
13-59-53](https://github.com/astral-sh/ruff/assets/6826232/6432f835-9ff1-4771-955b-398e54c406dc)

`RUST_LOG=debug`:
![Screenshot from 2023-07-30
14-01-32](https://github.com/astral-sh/ruff/assets/6826232/5f2c87da-0867-4159-82e7-b5757eebb8eb)

It's easy to see in this output which files take a disproportionate
amount of time.

[Peek 2023-07-30
14-35.webm](https://github.com/astral-sh/ruff/assets/6826232/2c92db5c-1354-465b-a6bc-ddfb281d6f9d)

It opens up further integration with the tracing ecosystem,
[tracing-timing](https://docs.rs/tracing-timing/latest/tracing_timing/)
and [tokio-console](https://github.com/tokio-rs/console) can e.g. show
histograms and the json output allows us building better pipelines than
grepping a log file.

One caveat is using `parent: None` for the logging statements because
tracing subscriber does not allow deactivating the span without
reimplementing all the other log message formatting, too, and we don't
need span information, esp. since it would currently show the progress
bar span.

## Test Plan

n/a
2023-07-31 19:14:01 +00:00
konsti
a7aa3caaae Rename formatter_progress to formatter_ecosystem_checks (#6194)
Rename the `scripts/formatter_progress.sh` to
`formatter/formatter_ecosysytem_checks.sh` since it fits the actual task
better.
2023-07-31 18:33:12 +00:00
konsti
e52b636da0 Log configuration in ruff_dev (#6193)
**Summary** This includes two changes:
 * Allow setting `-v` in `ruff_dev`, using the `ruff_cli` implementation
 * `debug!` which ruff configuration strategy was used

This is a byproduct of debugging #6187.

**Test Plan** n/a
2023-07-31 17:52:38 +00:00
konsti
9063f4524d Fix formatting of trailing unescaped quotes in raw triple quoted strings (#6202)
**Summary** This prevents us from turning `r'''\""'''` into
`r"""\"""""`, which is invalid syntax.

This PR fixes CI, which is currently broken on main (in a way that still
passes on linter PRs and allows merging formatter PRs, but it's bad to
have a job be red). Once merged, i'll make the formatted ecosystem
checks a required check.

**Test Plan** Added a regression test.
2023-07-31 19:25:16 +02:00
Charlie Marsh
dbd60b2cf5 Bump version to 0.0.281 (#6195) 2023-07-31 13:21:43 -04:00
Charlie Marsh
7eb2ba47cc Add empty line after import block (#6200)
## Summary

Ensures that, given:

```python
import os
x = 1
```

We format like:

```python
import os

x = 1
```
2023-07-31 12:01:45 -04:00
Dhruv Manilawala
cb34e6d322 Avoid parenthesizing comprehension element (#6198)
## Summary

This PR adds a new precedence level for the comprehension element. This fixes
the generator to not add parentheses around the comprehension element every
time.

The new precedence level is `COMPREHENSION_ELEMENT` and it should occur after
the `NAMED_EXPR` precedence level because named expressions are always parenthesized.

This matches the behavior of Python `ast.unparse` and tested with the
following snippet:

```python
import ast

code = ""
ast.unparse(ast.parse(code))
```

## Test Plan

Add a bunch of test cases for all the valid nodes at that position.

fixes: #5777
2023-07-31 20:56:42 +05:30
Harutaka Kawamura
0274de1fff Preserve backslash in raw string literal (#6152) 2023-07-31 12:48:17 +00:00
konsti
a540933bc9 Print log when formatter ecosystem checks fail (#6187)
**Summary** Print the errors when the formatter ecosystem checks failed.
Im not happy that we current collect the log in the first place, but
this is the less invasive change and we need it to unblock reviewing
#6152.

**Test Plan**
https://github.com/astral-sh/ruff/actions/runs/5713112075/job/15477879403?pr=6188
2023-07-31 14:45:38 +02:00
Micha Reiser
311a1f9ec4 Remove len from JoinCommaSeparatedBuilder (#6185) 2023-07-31 12:19:47 +00:00
Luc Khai Hai
b95fc6d162 Format bytes string (#6166)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

Format bytes string

Closes #6064

## Test Plan

Added a fixture based on string's one
2023-07-31 10:46:40 +02:00
Charlie Marsh
de898c52eb Avoid falsely marking non-submodules as submodule aliases (#6182)
## Summary

We have some code to ensure that if an aliased import is used, any
submodules should be marked as used too. This comment says it best:

```rust
// If the name of a submodule import is the same as an alias of another import, and the
// alias is used, then the submodule import should be marked as used too.
//
// For example, mark `pyarrow.csv` as used in:
//
// ```python
// import pyarrow as pa
// import pyarrow.csv
// print(pa.csv.read_csv("test.csv"))
// ```
```

However, it looks like when we go to look up `pyarrow` (of `import
pyarrow as pa`), we aren't checking to ensure the resolved binding is
_actually_ an import. This was causing us to attribute `print(rm.ANY)`
to `def requests_mock` here:

```python
import requests_mock as rm

def requests_mock(requests_mock: rm.Mocker):
    print(rm.ANY)
```

Closes https://github.com/astral-sh/ruff/issues/6180.
2023-07-30 22:16:25 +00:00
Charlie Marsh
76741cac77 Add global and nonlocal formatting (#6170)
## Summary

Adds `global` and `nonlocal` formatting, without the "deviation from
black" outlined in the linked issue, which I'll do separately.

See: https://github.com/astral-sh/ruff/issues/4798.

## Test Plan

Added a fixture in the Ruff-specific directory since the Black fixtures
don't seem to cover this.
2023-07-29 14:39:42 +00:00
Charlie Marsh
5d9814d84d Remove parentheses around some walrus operators (#6173)
## Summary

Closes https://github.com/astral-sh/ruff/issues/5781

## Test Plan

Added cases to
`crates/ruff_python_formatter/resources/test/fixtures/ruff/expression/named_expr.py`
one-by-one and adjusted the condition as needed.
2023-07-29 10:06:26 -04:00
Micha Reiser
1d7ad30188 CI: Update formatter dependencies (#6168) 2023-07-29 15:24:24 +02:00
Charlie Marsh
4231ed2fc3 Skip partial duplicates when applying multi-edit fixes (#6144)
## Summary

Right now, if we have two fixes that have an overlapping edit, but not
an _identical_ set of edits, they'll conflict, causing us to do another
linter traversal. Here, I've enabled the fixer to support partially
overlapping edits, which (as an example) let's us greatly reduce the
number of iterations required in the test suite.

The most common case here is that in which a bunch of edits need to
import some symbol, and then use that symbol, but in different ways. In
that case, all edits will have a common fix (to import the symbol), but
deviate in some way. With this change, we can do all of those edits in
one pass.

Note that the simplest way to enable this was to store sorted edits on
`Fix`. We don't allow modifying the edits on `Fix` once it's
constructed, so this is an easy change, and allows us to avoid a bunch
of clones and traversals later on.

Closes #5800.
2023-07-29 12:11:57 +00:00
Charlie Marsh
badbfb2d3e Skip BOM when determining Locator's line starts (#6159)
## Summary

If a file has a BOM, the import sorter _always_ reports the imports as
unsorted. The acute issue is that we detect that the line has leading
content (before the imports), which we always consider a violation.
Rather than fixing that one site, this PR instead makes `.line_start`
BOM-aware.

Fixes https://github.com/astral-sh/ruff/issues/6155.
2023-07-29 11:47:13 +00:00
Dhruv Manilawala
44bdf20221 [pep8-naming]: New config option extend-ignore-names (#6169)
## Summary

This PR adds a new config option for `pep8-naming` plugin called
`extend-ignore-names` which is used to extend the default values in
`ignore-names` option.

resolves: #6050
2023-07-29 17:11:04 +05:30
Dhruv Manilawala
3c99fbf808 Implement --diff for Jupyter Notebooks (#6149)
## Summary

Implement `--diff` for Jupyter Notebooks

## Test Plan

1. Use `crates/ruff/resources/test/fixtures/jupyter/isort.ipynb` as a
test case
and add a markdown cell in between the code cells to check that the diff
   outputs the correct cell index.
2. Run the command:
`cargo run --bin ruff --package ruff_cli -- check --no-cache --isolated
--select=ALL crates/ruff/resources/test/fixtures/jupyter/isort.ipynb
--fix --diff`

<details><summary>Example output:</summary>
<p>

```diff
--- /Users/dhruv/playground/ruff/notebooks/test.ipynb:cell 0
+++ /Users/dhruv/playground/ruff/notebooks/test.ipynb:cell 0
@@ -1,3 +0,0 @@
-from pathlib import Path
-import random
-import math
--- /Users/dhruv/playground/ruff/notebooks/test.ipynb:cell 4
+++ /Users/dhruv/playground/ruff/notebooks/test.ipynb:cell 4
@@ -1,5 +1,3 @@
-from typing import Any
-import collections
 # Newline should be added here
 def foo():
     pass

--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 8
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 8
@@ -1,8 +1,7 @@
 import pprint
 import tempfile
 
-from IPython import display
 import matplotlib.pyplot as plt
-
 import tensorflow as tf
-import tensorflow_datasets as tfds
+import tensorflow_datasets as tfds
+from IPython import display
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 10
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 10
@@ -1,5 +1,4 @@
 import tensorflow_models as tfm
 
 # These are not in the tfm public API for v2.9. They will be available in v2.10
-from official.vision.serving import export_saved_model_lib
-import official.core.train_lib
+from official.vision.serving import export_saved_model_lib
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 13
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 13
@@ -1,5 +1,5 @@
-exp_config = tfm.core.exp_factory.get_exp_config('resnet_imagenet')
-tfds_name = 'cifar10'
+exp_config = tfm.core.exp_factory.get_exp_config("resnet_imagenet")
+tfds_name = "cifar10"
 ds,ds_info = tfds.load(
 tfds_name,
 with_info=True)
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 15
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 15
@@ -6,12 +6,12 @@
 # Configure training and testing data
 batch_size = 128
 
-exp_config.task.train_data.input_path = ''
+exp_config.task.train_data.input_path = ""
 exp_config.task.train_data.tfds_name = tfds_name
-exp_config.task.train_data.tfds_split = 'train'
+exp_config.task.train_data.tfds_split = "train"
 exp_config.task.train_data.global_batch_size = batch_size
 
-exp_config.task.validation_data.input_path = ''
+exp_config.task.validation_data.input_path = ""
 exp_config.task.validation_data.tfds_name = tfds_name
-exp_config.task.validation_data.tfds_split = 'test'
+exp_config.task.validation_data.tfds_split = "test"
 exp_config.task.validation_data.global_batch_size = batch_size
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 17
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 17
@@ -1,16 +1,16 @@
 logical_device_names = [logical_device.name for logical_device in tf.config.list_logical_devices()]
 
-if 'GPU' in ''.join(logical_device_names):
-  print('This may be broken in Colab.')
-  device = 'GPU'
-elif 'TPU' in ''.join(logical_device_names):
-  print('This may be broken in Colab.')
-  device = 'TPU'
+if "GPU" in "".join(logical_device_names):
+  print("This may be broken in Colab.")
+  device = "GPU"
+elif "TPU" in "".join(logical_device_names):
+  print("This may be broken in Colab.")
+  device = "TPU"
 else:
-  print('Running on CPU is slow, so only train for a few steps.')
-  device = 'CPU'
+  print("Running on CPU is slow, so only train for a few steps.")
+  device = "CPU"
 
-if device=='CPU':
+if device=="CPU":
   train_steps = 20
   exp_config.trainer.steps_per_loop = 5
 else:
@@ -20,9 +20,9 @@
 exp_config.trainer.summary_interval = 100
 exp_config.trainer.checkpoint_interval = train_steps
 exp_config.trainer.validation_interval = 1000
-exp_config.trainer.validation_steps =  ds_info.splits['test'].num_examples // batch_size
+exp_config.trainer.validation_steps =  ds_info.splits["test"].num_examples // batch_size
 exp_config.trainer.train_steps = train_steps
-exp_config.trainer.optimizer_config.learning_rate.type = 'cosine'
+exp_config.trainer.optimizer_config.learning_rate.type = "cosine"
 exp_config.trainer.optimizer_config.learning_rate.cosine.decay_steps = train_steps
 exp_config.trainer.optimizer_config.learning_rate.cosine.initial_learning_rate = 0.1
 exp_config.trainer.optimizer_config.warmup.linear.warmup_steps = 100
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 21
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 21
@@ -1,14 +1,14 @@
 logical_device_names = [logical_device.name for logical_device in tf.config.list_logical_devices()]
 
 if exp_config.runtime.mixed_precision_dtype == tf.float16:
-    tf.keras.mixed_precision.set_global_policy('mixed_float16')
+    tf.keras.mixed_precision.set_global_policy("mixed_float16")
 
-if 'GPU' in ''.join(logical_device_names):
+if "GPU" in "".join(logical_device_names):
   distribution_strategy = tf.distribute.MirroredStrategy()
-elif 'TPU' in ''.join(logical_device_names):
+elif "TPU" in "".join(logical_device_names):
   tf.tpu.experimental.initialize_tpu_system()
-  tpu = tf.distribute.cluster_resolver.TPUClusterResolver(tpu='/device:TPU_SYSTEM:0')
+  tpu = tf.distribute.cluster_resolver.TPUClusterResolver(tpu="/device:TPU_SYSTEM:0")
   distribution_strategy = tf.distribute.experimental.TPUStrategy(tpu)
 else:
-  print('Warning: this will be really slow.')
+  print("Warning: this will be really slow.")
   distribution_strategy = tf.distribute.OneDeviceStrategy(logical_device_names[0])
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 23
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 23
@@ -1,5 +1,3 @@
 with distribution_strategy.scope():
   model_dir = tempfile.mkdtemp()
   task = tfm.core.task_factory.get_task(exp_config.task, logging_dir=model_dir)
-
-#  tf.keras.utils.plot_model(task.build_model(), show_shapes=True)
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 24
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 24
@@ -1,4 +1,4 @@
 for images, labels in task.build_inputs(exp_config.task.train_data).take(1):
   print()
-  print(f'images.shape: {str(images.shape):16}  images.dtype: {images.dtype!r}')
-  print(f'labels.shape: {str(labels.shape):16}  labels.dtype: {labels.dtype!r}')
+  print(f"images.shape: {images.shape!s:16}  images.dtype: {images.dtype!r}")
+  print(f"labels.shape: {labels.shape!s:16}  labels.dtype: {labels.dtype!r}")
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 27
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 27
@@ -1 +1 @@
-plt.hist(images.numpy().flatten());
+plt.hist(images.numpy().flatten())
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 29
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 29
@@ -1,2 +1,2 @@
-label_info = ds_info.features['label']
+label_info = ds_info.features["label"]
 label_info.int2str(1)
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 31
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 31
@@ -10,9 +10,6 @@
     if predictions is None:
       plt.title(label_info.int2str(labels[i]))
     else:
-      if labels[i] == predictions[i]:
-        color = 'g'
-      else:
-        color = 'r'
+      color = "g" if labels[i] == predictions[i] else "r"
       plt.title(label_info.int2str(predictions[i]), color=color)
     plt.axis("off")
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 35
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 35
@@ -1,3 +1,3 @@
-plt.figure(figsize=(10, 10));
+plt.figure(figsize=(10, 10))
 for images, labels in task.build_inputs(exp_config.task.validation_data).take(1):
   show_batch(images, labels)
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 37
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 37
@@ -1,7 +1,7 @@
 model, eval_logs = tfm.core.train_lib.run_experiment(
     distribution_strategy=distribution_strategy,
     task=task,
-    mode='train_and_eval',
+    mode="train_and_eval",
     params=exp_config,
     model_dir=model_dir,
     run_post_eval=True)
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 38
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 38
@@ -1 +0,0 @@
-#  tf.keras.utils.plot_model(model, show_shapes=True)
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 40
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 40
@@ -1,4 +1,4 @@
 for key, value in eval_logs.items():
     if isinstance(value, tf.Tensor):
       value = value.numpy()
-    print(f'{key:20}: {value:.3f}')
+    print(f"{key:20}: {value:.3f}")
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 42
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 42
@@ -4,5 +4,5 @@
 
 show_batch(images, labels, tf.cast(predictions, tf.int32))
 
-if device=='CPU':
-  plt.suptitle('The model was only trained for a few steps, it is not expected to do well.')
+if device=="CPU":
+  plt.suptitle("The model was only trained for a few steps, it is not expected to do well.")
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 45
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 45
@@ -1,8 +1,8 @@
 # Saving and exporting the trained model
 export_saved_model_lib.export_inference_graph(
-    input_type='image_tensor',
+    input_type="image_tensor",
     batch_size=1,
     input_image_size=[32, 32],
     params=exp_config,
     checkpoint_path=tf.train.latest_checkpoint(model_dir),
-    export_dir='./export/')
+    export_dir="./export/")
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 47
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 47
@@ -1,3 +1,3 @@
 # Importing SavedModel
-imported = tf.saved_model.load('./export/')
-model_fn = imported.signatures['serving_default']
+imported = tf.saved_model.load("./export/")
+model_fn = imported.signatures["serving_default"]
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 49
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 49
@@ -1,10 +1,10 @@
 plt.figure(figsize=(10, 10))
-for data in tfds.load('cifar10', split='test').batch(12).take(1):
+for data in tfds.load("cifar10", split="test").batch(12).take(1):
   predictions = []
-  for image in data['image']:
-    index = tf.argmax(model_fn(image[tf.newaxis, ...])['logits'], axis=1)[0]
+  for image in data["image"]:
+    index = tf.argmax(model_fn(image[tf.newaxis, ...])["logits"], axis=1)[0]
     predictions.append(index)
-  show_batch(data['image'], data['label'], predictions)
+  show_batch(data["image"], data["label"], predictions)
 
-  if device=='CPU':
-    plt.suptitle('The model was only trained for a few steps, it is not expected to do better than random.')
+  if device=="CPU":
+    plt.suptitle("The model was only trained for a few steps, it is not expected to do better than random.")

Would fix 61 errors.
```

</p>
</details> 

resolves: #4727
2023-07-29 04:22:56 +00:00
Charlie Marsh
4802c7c7d8 Avoid key-in-dict violations for self accesses (#6165)
Closes https://github.com/astral-sh/ruff/issues/6163.
2023-07-29 03:35:26 +00:00
Charlie Marsh
646ff6497c Ignore end-of-line file exemption comments (#6160)
## Summary

This PR protects against code like:

```python
from typing import Optional

import bar  # ruff: noqa
import baz

class Foo:
    x: Optional[str] = None
```

In which the user wrote `# ruff: noqa` to ignore a specific error, not
realizing that it was a file-level exemption that thus turned off all
lint rules.

Specifically, if a `# ruff: noqa` directive is not at the start of a
line, we now ignore it and warn, since this is almost certainly a
mistake.
2023-07-29 00:40:32 +00:00
Victor Hugo Gomes
e0d5c7564f [flake8-pyi] Implement PYI049 (#6136)
## Summary

Checks for the presence of unused private `typing.TypedDict`
definitions.

ref #848 

## Test Plan

Snapshots and manual runs of flake8
2023-07-29 00:34:36 +00:00
Victor Hugo Gomes
7838d8c8af Implement PYI047 (#6134)
## Summary

Checks for the presence of unused private `typing.TypeAlias`
definitions.

ref #848 

## Test Plan

Snapshots and manual runs of flake8
2023-07-29 00:21:29 +00:00
Zanie Blue
047c211837 Add semantic analysis of type aliases and parameters (#6109)
Requires https://github.com/astral-sh/RustPython-Parser/pull/42
Related https://github.com/PyCQA/pyflakes/pull/778
[PEP-695](https://peps.python.org/pep-0695)
Part of #5062 

<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->
Adds a scope for type parameters, a type parameter binding kind, and
checker visitation of type parameters in type alias statements, function
definitions, and class definitions.

A few changes were necessary to ensure correctness following the
insertion of a new scope between function and class scopes and their
parent.

## Test Plan

<!-- How was it tested? -->
Undefined name snapshots.

Unused type parameter rule will be added as follow-up.
2023-07-28 17:06:37 -05:00
Charlie Marsh
134d447d4c Avoid refactoring x[:1]-like slices in RUF015 (#6150)
## Summary

Right now, `RUF015` will try to rewrite `x[:1]` as `[next(x)]`. This
isn't equivalent if `x`, for example, is empty, where slicing like
`x[:1]` is forgiving, but `next` raises `StopIteration`. For me this is
a little too much of a deviation to be comfortable with, and most of the
value in this rule is the `x[0]` to `next(x)` conversion anyway.

Closes https://github.com/astral-sh/ruff/issues/6148.
2023-07-28 09:38:13 -04:00
Charlie Marsh
cd4147423c Skip PERF203 violations for multi-statement loops (#6145)
Closes https://github.com/astral-sh/ruff/issues/5858.
2023-07-28 04:55:55 +00:00
Charlie Marsh
d15436458f Only run unused private type rules over finalized bindings (#6142)
## Summary

In #6134 and #6136, we see some false positives for "shadowed" class
definitions. For example, here, the first definition is flagged as
unused, since from the perspective of the semantic model (which doesn't
understand branching), it appears to be immediately shadowed in the
`else`, and thus never used:

```python
if sys.version_info >= (3, 11):
    class _RootLoggerConfiguration(TypedDict, total=False):
        level: _Level
        filters: Sequence[str | _FilterType]
        handlers: Sequence[str]

else:
    class _RootLoggerConfiguration(TypedDict, total=False):
        level: _Level
        filters: Sequence[str]
        handlers: Sequence[str]
```

Instead of looking at _all_ bindings, we should instead look at the
"live" bindings, which is similar to how other rules (like unused
variables detection) is structured. We thus move the rule from
`bindings.rs` (which iterates over _all_ bindings, regardless of whether
they're shadowed) to `deferred_scopes.rs`, which iterates over all
"live" bindings once a scope has been fully analyzed.

## Test Plan

`cargo test`
2023-07-28 02:16:09 +00:00
Charlie Marsh
0bc3edf6c9 Add documentation and test cases for redefinition (#6135) 2023-07-28 00:01:42 +00:00
Aarni Koskela
3d54d31cd9 Implement E241 and E242 (tab/multiple ws after commas) (#6094)
## Summary

This PR implements pycodestyle's E241 (tab after comma) and E242
(multiple whitespace after comma) lints.

These are marked as nursery rules like many other pycodestyle rules.

Refs #2402

## Test Plan

E24.py copied from pycodestyle.
2023-07-27 18:58:41 +00:00
Tom Kuson
1418ee62f8 Add more documentation to the flake8-bandit rules (#6128)
## Summary

Completes the documentation for the ruleset, apart from four rules which
have contradictions, so need to be thought about more regarding how to
document that. Related to #2646.

## Test Plan

`python scripts/test_docs_formatted.py`
2023-07-27 18:57:45 +00:00
Harutaka Kawamura
bf987f80f4 Add PT017 and PT019 docs (#6115) 2023-07-27 18:56:34 +00:00
rembridge
bb08eea5cc missing-whitespace-around-operators comment (#6106)
**Summary**

Updated doc comments for `missing_whitespace_around_operator.rs`. Online
docs also benefit from this update.

**Test Plan**

Checked docs via
[mkdocs](389fe13c93/CONTRIBUTING.md (L267-L296))
2023-07-27 14:52:43 -04:00
Tom Kuson
d16216a2c2 Add documentation to the flynt rules (#6130)
## Summary

Completes the documentation for the one and only (current) rule in the
`flynt` ruleset. Related to #2646.

## Test Plan

`python scripts/test_docs_formatted.py`
2023-07-27 14:32:59 -04:00
Jelle van der Waa
0853004f41 [pylint] Implement eq-without-hash rule (PLW1641) (#5955)
Implement
https://pylint.pycqa.org/en/latest/user_guide/messages/warning/eq-without-hash.html
Issue https://github.com/astral-sh/ruff/issues/970

It's not enabled by default in pylint, so I guess it shouldn't in Ruff
either?
2023-07-27 18:28:44 +00:00
Harutaka Kawamura
fb5bbe30c7 Update SIM115 to cover pathlib.Path.open (#6118) 2023-07-27 14:20:52 -04:00
Charlie Marsh
dd706c7a35 Fix E211 documentation (#6133) 2023-07-27 17:19:33 +00:00
Charlie Marsh
e15b9c5572 Cache name resolutions in the semantic model (#6047)
## Summary

This PR stores the mapping from `ExprName` node to resolved `BindingId`,
which lets us skip scope lookups in `resolve_call_path`. It's enabled by
#6045, since that PR ensures that when we analyze a node (and thus call
`resolve_call_path`), we'll have already visited its `ExprName`
elements.

In more detail: imagine that we're traversing over `foo.bar()`. When we
read `foo`, it will be an `ExprName`, which we'll then resolve to a
binding via `handle_node_load`. With this change, we then store that
binding in a map. Later, if we call `collect_call_path` on `foo.bar`,
we'll identify `foo` (the "head" of the attribute) and grab the resolved
binding in that map. _Almost_ all names are now resolved in advance,
though it's not a strict requirement, and some rules break that pattern
(e.g., if we're analyzing arguments, and they need to inspect their
annotations, which are visited in a deferred manner).

This improves performance by 4-6% on the all-rules benchmark. It looks
like it hurts performance (1-2% drop) in the default-rules benchmark,
presumedly because those rules don't call `resolve_call_path` nearly as
much, and so we're paying for these extra writes.

Here's the benchmark data:

```
linter/default-rules/numpy/globals.py
                        time:   [67.270 µs 67.380 µs 67.489 µs]
                        thrpt:  [43.720 MiB/s 43.792 MiB/s 43.863 MiB/s]
                 change:
                        time:   [+0.4747% +0.7752% +1.0626%] (p = 0.00 < 0.05)
                        thrpt:  [-1.0514% -0.7693% -0.4724%]
                        Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe
linter/default-rules/pydantic/types.py
                        time:   [1.4067 ms 1.4105 ms 1.4146 ms]
                        thrpt:  [18.028 MiB/s 18.081 MiB/s 18.129 MiB/s]
                 change:
                        time:   [+1.3152% +1.6953% +2.0414%] (p = 0.00 < 0.05)
                        thrpt:  [-2.0006% -1.6671% -1.2981%]
                        Performance has regressed.
linter/default-rules/numpy/ctypeslib.py
                        time:   [637.67 µs 638.96 µs 640.28 µs]
                        thrpt:  [26.006 MiB/s 26.060 MiB/s 26.113 MiB/s]
                 change:
                        time:   [+1.5859% +1.8109% +2.0353%] (p = 0.00 < 0.05)
                        thrpt:  [-1.9947% -1.7787% -1.5611%]
                        Performance has regressed.
linter/default-rules/large/dataset.py
                        time:   [3.2289 ms 3.2336 ms 3.2383 ms]
                        thrpt:  [12.563 MiB/s 12.581 MiB/s 12.599 MiB/s]
                 change:
                        time:   [+0.8029% +0.9898% +1.1740%] (p = 0.00 < 0.05)
                        thrpt:  [-1.1604% -0.9801% -0.7965%]
                        Change within noise threshold.

linter/all-rules/numpy/globals.py
                        time:   [134.05 µs 134.15 µs 134.26 µs]
                        thrpt:  [21.977 MiB/s 21.995 MiB/s 22.012 MiB/s]
                 change:
                        time:   [-4.4571% -4.1175% -3.8268%] (p = 0.00 < 0.05)
                        thrpt:  [+3.9791% +4.2943% +4.6651%]
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  2 (2.00%) low mild
  3 (3.00%) high mild
  3 (3.00%) high severe
linter/all-rules/pydantic/types.py
                        time:   [2.5627 ms 2.5669 ms 2.5720 ms]
                        thrpt:  [9.9158 MiB/s 9.9354 MiB/s 9.9516 MiB/s]
                 change:
                        time:   [-5.8304% -5.6374% -5.4452%] (p = 0.00 < 0.05)
                        thrpt:  [+5.7587% +5.9742% +6.1914%]
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  6 (6.00%) high mild
  1 (1.00%) high severe
linter/all-rules/numpy/ctypeslib.py
                        time:   [1.3949 ms 1.3956 ms 1.3964 ms]
                        thrpt:  [11.925 MiB/s 11.931 MiB/s 11.937 MiB/s]
                 change:
                        time:   [-6.2496% -6.0856% -5.9293%] (p = 0.00 < 0.05)
                        thrpt:  [+6.3030% +6.4799% +6.6662%]
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  3 (3.00%) high mild
  4 (4.00%) high severe
linter/all-rules/large/dataset.py
                        time:   [5.5951 ms 5.6019 ms 5.6093 ms]
                        thrpt:  [7.2527 MiB/s 7.2623 MiB/s 7.2711 MiB/s]
                 change:
                        time:   [-5.1781% -4.9783% -4.8070%] (p = 0.00 < 0.05)
                        thrpt:  [+5.0497% +5.2391% +5.4608%]
                        Performance has improved.
```

Still playing with this (the concepts need better names, documentation,
etc.), but opening up for feedback.
2023-07-27 13:01:56 -04:00
qdegraaf
0638a26347 Add AnyExpressionYield to consolidate ExprYield and ExprYieldFrom (#6127)
Co-authored-by: Micha Reiser <micha@reiser.io>
2023-07-27 16:01:16 +00:00
konsti
2a65e6fc38 Explain check_docs_formatted.py error message (#6125)
## Summary

This is an error message only change to lead an implementor of a new
rule that has an unformatted or invalid bad example to the
right code.

## Test Plan

n/a
2023-07-27 10:22:13 -04:00
Charlie Marsh
13af91299d Avoid walking past root when resolving imports (#6126)
## Summary

Noticed in #5954: we walk _past_ the root rather than stopping _at_ the
root when attempting to traverse along the parent path. It's effectively
an off-by-one bug.
2023-07-27 10:22:13 -04:00
konsti
d317af442f Fix windows test warnings (#6124)
See
https://github.com/astral-sh/ruff/actions/runs/5679922286/job/15392998698.
These didn't fail CI because we run clippy on linux only.
2023-07-27 10:22:13 -04:00
Micha Reiser
6bf6646c5d Respect indent when measuring with MeasureMode::AllLines (#6120) 2023-07-27 10:22:13 -04:00
konsti
9574ff3dc7 Unbreak main (#6123)
This fixes main breaking due to two merges.
2023-07-27 10:22:13 -04:00
konsti
06d9ff9577 Don't format trailing comma for lambda arguments (#5946)
**Summary** lambda arguments don't have parentheses, so they shouldn't
get a magic trailing comma either. This fixes some unstable formatting

**Test Plan** Added a regression test.

89 (from previously 145) instances of unstable formatting remaining.

```
$ cargo run --bin ruff_dev --release -- format-dev --stability-check --error-file formatter-ecosystem-errors.txt --multi-project target/checkouts > formatter-ecosystem-progress.txt
$ rg "Unstable formatting" target/formatter-ecosystem-errors.txt | wc -l
89
```

Closes #5892
2023-07-27 10:22:13 -04:00
5453 changed files with 251381 additions and 205127 deletions

View File

@@ -10,7 +10,7 @@ indent_style = space
insert_final_newline = true
indent_size = 2
[*.{rs,py}]
[*.{rs,py,pyi}]
indent_size = 4
[*.snap]

4
.gitattributes vendored
View File

@@ -1,7 +1,7 @@
* text=auto eol=lf
crates/ruff/resources/test/fixtures/isort/line_ending_crlf.py text eol=crlf
crates/ruff/resources/test/fixtures/pycodestyle/W605_1.py text eol=crlf
crates/ruff_linter/resources/test/fixtures/isort/line_ending_crlf.py text eol=crlf
crates/ruff_linter/resources/test/fixtures/pycodestyle/W605_1.py text eol=crlf
ruff.schema.json linguist-generated=true text=auto eol=lf
*.md.snap linguist-language=Markdown

2
.github/CODEOWNERS vendored
View File

@@ -6,4 +6,4 @@
# - Order is important. The last matching pattern has the most precedence.
# Jupyter
/crates/ruff/src/jupyter/ @dhruvmanila
/crates/ruff_linter/src/jupyter/ @dhruvmanila

View File

@@ -4,8 +4,10 @@ updates:
directory: "/"
schedule:
interval: "weekly"
day: "monday"
time: "12:00"
timezone: "America/New_York"
commit-message:
prefix: "ci(deps)"
labels: ["internal"]
- package-ecosystem: "cargo"
directory: "/"
schedule:
interval: "daily"
labels: ["internal"]

4
.github/release.yml vendored
View File

@@ -4,6 +4,7 @@ changelog:
labels:
- internal
- documentation
- formatter
categories:
- title: Breaking Changes
labels:
@@ -19,6 +20,9 @@ changelog:
- title: Bug Fixes
labels:
- bug
- title: Preview
labels:
- preview
- title: Other Changes
labels:
- "*"

View File

@@ -1,136 +0,0 @@
name: Benchmark
on:
pull_request:
paths:
- 'Cargo.toml'
- 'Cargo.lock'
- 'rust-toolchain'
- 'crates/**'
- '!crates/ruff_dev'
- '!crates/ruff_shrinking'
workflow_dispatch:
concurrency:
group: ${{ github.workflow }}-${{ github.ref_name }}-${{ github.event.pull_request.number || github.sha }}
cancel-in-progress: true
jobs:
run-benchmark:
if: github.event_name == 'pull_request'
name: "Run | ${{ matrix.os }}"
strategy:
matrix:
os: [ ubuntu-latest, windows-latest ]
runs-on: ${{ matrix.os }}
steps:
- name: "PR - Checkout Branch"
uses: actions/checkout@v3
with:
ref: ${{ github.event.pull_request.head.sha }}
- name: "PR - Install Rust toolchain"
run: rustup show
- uses: Swatinem/rust-cache@v2
- name: "PR - Build benchmarks"
run: cargo bench -p ruff_benchmark --no-run
- name: "PR - Run benchmarks"
run: cargo benchmark --save-baseline=pr
- name: "Main - Checkout Branch"
uses: actions/checkout@v3
with:
clean: false
ref: main
- name: "Main - Install Rust toolchain"
run: rustup show
- name: "Main - Build benchmarks"
run: cargo bench -p ruff_benchmark --no-run
- name: "Main - Run benchmarks"
run: cargo benchmark --save-baseline=main
- name: "Upload benchmark results"
uses: actions/upload-artifact@v3
with:
name: benchmark-results-${{ matrix.os }}
path: ./target/criterion
# Cleanup
- name: Remove Criterion Artifact
uses: JesseTG/rm@v1.0.3
with:
path: ./target/criterion
benchmark-compare:
if: github.event_name == 'pull_request'
runs-on: ubuntu-latest
name: Compare
needs:
- run-benchmark
steps:
- name: "Install Rust toolchain"
run: rustup show
- name: "Install critcmp"
uses: taiki-e/install-action@v2
with:
tool: critcmp
- name: "Linux | Download PR benchmark results"
uses: actions/download-artifact@v3
with:
name: benchmark-results-ubuntu-latest
path: ./target/criterion
- name: "Linux | Compare benchmark results"
shell: bash
run: |
echo "### Benchmark" >> summary.md
echo "#### Linux" >> summary.md
echo "\`\`\`" >> summary.md
critcmp main pr >> summary.md
echo "\`\`\`" >> summary.md
echo "" >> summary.md
- name: "Linux | Cleanup benchmark results"
run: rm -rf ./target/criterion
- name: "Windows | Download PR benchmark results"
uses: actions/download-artifact@v3
with:
name: benchmark-results-windows-latest
path: ./target/criterion
- name: "Windows | Compare benchmark results"
shell: bash
run: |
echo "#### Windows" >> summary.md
echo "\`\`\`" >> summary.md
critcmp main pr >> summary.md
echo "\`\`\`" >> summary.md
echo "" >> summary.md
echo ${{ github.event.pull_request.number }} > pr-number
cat summary.md > $GITHUB_STEP_SUMMARY
- uses: actions/upload-artifact@v3
name: Upload PR Number
with:
name: pr-number
path: pr-number
- uses: actions/upload-artifact@v3
name: Upload Summary
with:
name: summary
path: summary.md

View File

@@ -2,7 +2,7 @@ name: CI
on:
push:
branches: [ main ]
branches: [main]
pull_request:
workflow_dispatch:
@@ -16,7 +16,7 @@ env:
CARGO_TERM_COLOR: always
RUSTUP_MAX_RETRIES: 10
PACKAGE_NAME: ruff
PYTHON_VERSION: "3.11" # to build abi3 wheels
PYTHON_VERSION: "3.11"
jobs:
determine_changes:
@@ -26,11 +26,11 @@ jobs:
linter: ${{ steps.changed.outputs.linter_any_changed }}
formatter: ${{ steps.changed.outputs.formatter_any_changed }}
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: tj-actions/changed-files@v37
- uses: tj-actions/changed-files@v39
id: changed
with:
files_yaml: |
@@ -42,6 +42,7 @@ jobs:
- "!crates/ruff_formatter/**"
- "!crates/ruff_dev/**"
- "!crates/ruff_shrinking/**"
- scripts/*
formatter:
- Cargo.toml
@@ -50,13 +51,18 @@ jobs:
- crates/ruff_formatter/**
- crates/ruff_python_trivia/**
- crates/ruff_python_ast/**
- crates/ruff_source_file/**
- crates/ruff_python_index/**
- crates/ruff_text_size/**
- crates/ruff_python_parser/**
- crates/ruff_dev/**
- scripts/*
cargo-fmt:
name: "cargo fmt"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: "Install Rust toolchain"
run: rustup component add rustfmt
- run: cargo fmt --all --check
@@ -65,7 +71,7 @@ jobs:
name: "cargo clippy"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: "Install Rust toolchain"
run: |
rustup component add clippy
@@ -79,11 +85,11 @@ jobs:
cargo-test:
strategy:
matrix:
os: [ ubuntu-latest, windows-latest ]
os: [ubuntu-latest, windows-latest]
runs-on: ${{ matrix.os }}
name: "cargo test | ${{ matrix.os }}"
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: "Install Rust toolchain"
run: rustup show
- name: "Install cargo insta"
@@ -119,7 +125,7 @@ jobs:
runs-on: ubuntu-latest
name: "cargo fuzz"
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: "Install Rust toolchain"
run: rustup show
- uses: Swatinem/rust-cache@v2
@@ -135,7 +141,7 @@ jobs:
runs-on: ubuntu-latest
name: "cargo test (wasm)"
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: "Install Rust toolchain"
run: rustup target add wasm32-unknown-unknown
- uses: actions/setup-node@v3
@@ -154,7 +160,7 @@ jobs:
name: "test scripts"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: "Install Rust toolchain"
run: rustup component add rustfmt
- uses: Swatinem/rust-cache@v2
@@ -176,7 +182,7 @@ jobs:
# Only runs on pull requests, since that is the only we way we can find the base version for comparison.
if: github.event_name == 'pull_request' && needs.determine_changes.outputs.linter == 'true'
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
@@ -221,7 +227,7 @@ jobs:
name: "cargo udeps"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: "Install nightly Rust toolchain"
# Only pinned to make caching work, update freely
run: rustup toolchain install nightly-2023-06-08
@@ -231,12 +237,11 @@ jobs:
- name: "Run cargo-udeps"
run: cargo +nightly-2023-06-08 udeps
python-package:
name: "python package"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
@@ -260,7 +265,7 @@ jobs:
name: "pre-commit"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
@@ -290,7 +295,7 @@ jobs:
env:
MKDOCS_INSIDERS_SSH_KEY_EXISTS: ${{ secrets.MKDOCS_INSIDERS_SSH_KEY != '' }}
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
- name: "Add SSH key"
if: ${{ env.MKDOCS_INSIDERS_SSH_KEY_EXISTS == 'true' }}
@@ -323,19 +328,41 @@ jobs:
name: "Formatter ecosystem and progress checks"
runs-on: ubuntu-latest
needs: determine_changes
if: needs.determine_changes.outputs.formatter == 'true'
if: needs.determine_changes.outputs.formatter == 'true' || github.ref == 'refs/heads/main'
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: "Install Rust toolchain"
run: rustup show
- name: "Cache rust"
uses: Swatinem/rust-cache@v2
- name: "Formatter progress"
run: scripts/formatter_progress.sh
run: scripts/formatter_ecosystem_checks.sh
- name: "Github step summary"
run: grep "similarity index" target/progress_projects_report.txt | sort > $GITHUB_STEP_SUMMARY
# CPython is not black formatted, so we run only the stability check
- name: "Clone CPython 3.10"
run: git clone --branch 3.10 --depth 1 https://github.com/python/cpython.git crates/ruff/resources/test/cpython
- name: "Check CPython stability"
run: cargo run --bin ruff_dev -- format-dev --stability-check crates/ruff/resources/test/cpython
run: cat target/progress_projects_stats.txt > $GITHUB_STEP_SUMMARY
- name: "Remove checkouts from cache"
run: rm -r target/progress_projects
benchmarks:
runs-on: ubuntu-latest
steps:
- name: "Checkout Branch"
uses: actions/checkout@v4
- name: "Install Rust toolchain"
run: rustup show
- name: "Install codspeed"
uses: taiki-e/install-action@v2
with:
tool: cargo-codspeed
- uses: Swatinem/rust-cache@v2
- name: "Build benchmarks"
run: cargo codspeed build --features codspeed -p ruff_benchmark
- name: "Run benchmarks"
uses: CodSpeedHQ/action@v1
with:
run: cargo codspeed run
token: ${{ secrets.CODSPEED_TOKEN }}

View File

@@ -2,8 +2,13 @@ name: mkdocs
on:
workflow_dispatch:
inputs:
ref:
description: "The commit SHA, tag, or branch to publish. Uses the default branch if not specified."
default: ""
type: string
release:
types: [ published ]
types: [published]
jobs:
mkdocs:
@@ -12,7 +17,9 @@ jobs:
CF_API_TOKEN_EXISTS: ${{ secrets.CF_API_TOKEN != '' }}
MKDOCS_INSIDERS_SSH_KEY_EXISTS: ${{ secrets.MKDOCS_INSIDERS_SSH_KEY != '' }}
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
ref: ${{ inputs.ref }}
- uses: actions/setup-python@v4
- name: "Add SSH key"
if: ${{ env.MKDOCS_INSIDERS_SSH_KEY_EXISTS == 'true' }}
@@ -40,8 +47,9 @@ jobs:
run: mkdocs build --strict -f mkdocs.generated.yml
- name: "Deploy to Cloudflare Pages"
if: ${{ env.CF_API_TOKEN_EXISTS == 'true' }}
uses: cloudflare/wrangler-action@2.0.0
uses: cloudflare/wrangler-action@v3.1.1
with:
apiToken: ${{ secrets.CF_API_TOKEN }}
accountId: ${{ secrets.CF_ACCOUNT_ID }}
command: pages publish site --project-name=ruff-docs --branch ${GITHUB_HEAD_REF} --commit-hash ${GITHUB_SHA}
# `github.head_ref` is only set during pull requests and for manual runs or tags we use `main` to deploy to production
command: pages deploy site --project-name=astral-docs --branch ${{ github.head_ref || 'main' }} --commit-hash ${GITHUB_SHA}

View File

@@ -19,7 +19,7 @@ jobs:
macos-x86_64:
runs-on: macos-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
@@ -42,7 +42,7 @@ jobs:
macos-universal:
runs-on: macos-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
@@ -66,9 +66,9 @@ jobs:
runs-on: windows-latest
strategy:
matrix:
target: [ x64, x86 ]
target: [x64, x86]
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
@@ -94,9 +94,9 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
target: [ x86_64, i686 ]
target: [x86_64, i686]
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
@@ -121,9 +121,9 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
target: [ aarch64, armv7, s390x, ppc64le, ppc64 ]
target: [aarch64, armv7, s390x, ppc64le, ppc64]
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
@@ -160,7 +160,7 @@ jobs:
- x86_64-unknown-linux-musl
- i686-unknown-linux-musl
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
@@ -196,7 +196,7 @@ jobs:
- target: armv7-unknown-linux-musleabihf
arch: armv7
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}

View File

@@ -3,7 +3,7 @@ name: "[Playground] Release"
on:
workflow_dispatch:
release:
types: [ published ]
types: [published]
env:
CARGO_INCREMENTAL: 0
@@ -17,7 +17,7 @@ jobs:
env:
CF_API_TOKEN_EXISTS: ${{ secrets.CF_API_TOKEN != '' }}
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: "Install Rust toolchain"
run: rustup target add wasm32-unknown-unknown
- uses: actions/setup-node@v3
@@ -40,8 +40,8 @@ jobs:
working-directory: playground
- name: "Deploy to Cloudflare Pages"
if: ${{ env.CF_API_TOKEN_EXISTS == 'true' }}
uses: cloudflare/wrangler-action@2.0.0
uses: cloudflare/wrangler-action@v3.1.1
with:
apiToken: ${{ secrets.CF_API_TOKEN }}
accountId: ${{ secrets.CF_ACCOUNT_ID }}
command: pages publish playground/dist --project-name=ruff --branch ${GITHUB_HEAD_REF} --commit-hash ${GITHUB_SHA}
command: pages deploy playground/dist --project-name=ruff-playground --branch ${GITHUB_HEAD_REF} --commit-hash ${GITHUB_SHA}

View File

@@ -2,8 +2,8 @@ name: PR Check Comment
on:
workflow_run:
workflows: [ CI, Benchmark ]
types: [ completed ]
workflows: [CI]
types: [completed]
workflow_dispatch:
inputs:
workflow_run_id:
@@ -43,42 +43,34 @@ jobs:
path: pr/ecosystem
if_no_artifact_found: ignore
- uses: dawidd6/action-download-artifact@v2
name: "Download Benchmark Result"
id: download-benchmark-result
if: steps.pr-number.outputs.pr-number
with:
name: summary
workflow: benchmark.yaml
pr: ${{ steps.pr-number.outputs.pr-number }}
path: pr/benchmark
if_no_artifact_found: ignore
- name: Generate Comment
id: generate-comment
if: steps.download-ecosystem-result.outputs.found_artifact == 'true' || steps.download-benchmark-result.outputs.found_artifact == 'true'
if: steps.download-ecosystem-result.outputs.found_artifact == 'true'
run: |
echo '## PR Check Results' >> comment.txt
echo "### Ecosystem" >> comment.txt
cat pr/ecosystem/ecosystem-result >> comment.txt
echo "" >> comment.txt
echo 'comment<<EOF' >> $GITHUB_OUTPUT
echo '## PR Check Results' >> $GITHUB_OUTPUT
if [[ -f pr/ecosystem/ecosystem-result ]]
then
echo "### Ecosystem" >> $GITHUB_OUTPUT
cat pr/ecosystem/ecosystem-result >> $GITHUB_OUTPUT
echo "" >> $GITHUB_OUTPUT
fi
if [[ -f pr/benchmark/summary.md ]]
then
cat pr/benchmark/summary.md >> $GITHUB_OUTPUT
fi
cat comment.txt >> $GITHUB_OUTPUT
echo 'EOF' >> $GITHUB_OUTPUT
- name: Create or update comment
if: steps.generate-comment.outputs.comment
uses: thollander/actions-comment-pull-request@v2
- name: Find Comment
uses: peter-evans/find-comment@v2
if: steps.generate-comment.outcome == 'success'
id: find-comment
with:
pr_number: ${{ steps.pr-number.outputs.pr-number }}
message: ${{ steps.generate-comment.outputs.comment }}
comment_tag: PR Check Results
issue-number: ${{ steps.pr-number.outputs.pr-number }}
comment-author: "github-actions[bot]"
body-includes: PR Check Results
- name: Create or update comment
if: steps.find-comment.outcome == 'success'
uses: peter-evans/create-or-update-comment@v3
with:
comment-id: ${{ steps.find-comment.outputs.comment-id }}
issue-number: ${{ steps.pr-number.outputs.pr-number }}
body-path: comment.txt
edit-mode: replace

View File

@@ -7,12 +7,15 @@ on:
description: "The version to tag, without the leading 'v'. If omitted, will initiate a dry run (no uploads)."
type: string
sha:
description: "Optionally, the full sha of the commit to be released"
description: "The full sha of the commit to be released. If omitted, the latest commit on the default branch will be used."
default: ""
type: string
pull_request:
paths:
# When we change pyproject.toml, we want to ensure that the maturin builds still work
- pyproject.toml
# And when we change this workflow itself...
- .github/workflows/release.yaml
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
@@ -30,7 +33,9 @@ jobs:
sdist:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
ref: ${{ inputs.sha }}
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
@@ -56,7 +61,9 @@ jobs:
macos-x86_64:
runs-on: macos-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
ref: ${{ inputs.sha }}
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
@@ -94,7 +101,9 @@ jobs:
macos-universal:
runs-on: macos-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
ref: ${{ inputs.sha }}
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
@@ -140,7 +149,9 @@ jobs:
- target: aarch64-pc-windows-msvc
arch: x64
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
ref: ${{ inputs.sha }}
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
@@ -186,7 +197,9 @@ jobs:
- x86_64-unknown-linux-gnu
- i686-unknown-linux-gnu
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
ref: ${{ inputs.sha }}
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
@@ -243,7 +256,9 @@ jobs:
arch: ppc64
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
ref: ${{ inputs.sha }}
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
@@ -296,7 +311,9 @@ jobs:
- x86_64-unknown-linux-musl
- i686-unknown-linux-musl
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
ref: ${{ inputs.sha }}
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
@@ -350,7 +367,9 @@ jobs:
arch: armv7
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
ref: ${{ inputs.sha }}
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
@@ -398,9 +417,22 @@ jobs:
# If you don't set an input tag, it's a dry run (no uploads).
if: ${{ inputs.tag }}
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
ref: main # We checkout the main branch to check for the commit
- name: Check main branch
if: ${{ inputs.sha }}
run: |
# Fetch the main branch since a shallow checkout is used by default
git fetch origin main --unshallow
if ! git branch --contains ${{ inputs.sha }} | grep -E '(^|\s)main$'; then
echo "The specified sha is not on the main branch" >&2
exit 1
fi
- name: Check tag consistency
run: |
# Switch to the commit we want to release
git checkout ${{ inputs.sha }}
version=$(grep "version = " pyproject.toml | sed -e 's/version = "\(.*\)"/\1/g')
if [ "${{ inputs.tag }}" != "${version}" ]; then
echo "The input tag does not match the version from pyproject.toml:" >&2
@@ -410,18 +442,6 @@ jobs:
else
echo "Releasing ${version}"
fi
- name: Check SHA consistency
if: ${{ inputs.sha }}
run: |
git_sha=$(git rev-parse HEAD)
if [ "${{ inputs.sha }}" != "${git_sha}" ]; then
echo "The specified sha does not match the git checkout" >&2
echo "${{ inputs.sha }}" >&2
echo "${git_sha}" >&2
exit 1
else
echo "Releasing ${git_sha}"
fi
upload-release:
name: Upload to PyPI
@@ -464,7 +484,9 @@ jobs:
# For git tag
contents: write
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
ref: ${{ inputs.sha }}
- name: git tag
run: |
git config user.email "hey@astral.sh"

2
.gitignore vendored
View File

@@ -1,5 +1,5 @@
# Benchmarking cpython (CONTRIBUTING.md)
crates/ruff/resources/test/cpython
crates/ruff_linter/resources/test/cpython
# generate_mkdocs.py
mkdocs.generated.yml
# check_ecosystem.py

View File

@@ -2,8 +2,8 @@ fail_fast: true
exclude: |
(?x)^(
crates/ruff/resources/.*|
crates/ruff/src/rules/.*/snapshots/.*|
crates/ruff_linter/resources/.*|
crates/ruff_linter/src/rules/.*/snapshots/.*|
crates/ruff_cli/resources/.*|
crates/ruff_python_formatter/resources/.*|
crates/ruff_python_formatter/tests/snapshots/.*|
@@ -23,8 +23,6 @@ repos:
- id: mdformat
additional_dependencies:
- mdformat-mkdocs
- mdformat-black
- black==23.1.0 # Must be the latest version of Black
- repo: https://github.com/igorshubovych/markdownlint-cli
rev: v0.33.0
@@ -42,17 +40,17 @@ repos:
name: cargo fmt
entry: cargo fmt --
language: system
types: [ rust ]
types: [rust]
pass_filenames: false # This makes it a lot faster
- id: ruff
name: ruff
entry: cargo run --bin ruff -- check --no-cache --force-exclude --fix --exit-non-zero-on-fix
language: system
types_or: [ python, pyi ]
types_or: [python, pyi]
require_serial: true
exclude: |
(?x)^(
crates/ruff/resources/.*|
crates/ruff_linter/resources/.*|
crates/ruff_python_formatter/resources/.*
)$
@@ -62,5 +60,12 @@ repos:
hooks:
- id: black
# Prettier
- repo: https://github.com/pre-commit/mirrors-prettier
rev: v3.0.0
hooks:
- id: prettier
types: [yaml]
ci:
skip: [ cargo-fmt, dev-generate-all ]
skip: [cargo-fmt, dev-generate-all]

View File

@@ -1,5 +1,30 @@
# Breaking Changes
## 0.0.288
### Remove support for emoji identifiers ([#7212](https://github.com/astral-sh/ruff/pull/7212))
Previously, Ruff supported the non-standard compliant emoji identifiers e.g. `📦 = 1`.
We decided to remove this non-standard language extension, and Ruff now reports syntax errors for emoji identifiers in your code, the same as CPython.
### Improved GitLab fingerprints ([#7203](https://github.com/astral-sh/ruff/pull/7203))
GitLab uses fingerprints to identify new, existing, or fixed violations. Previously, Ruff included the violation's position in the fingerprint. Using the location has the downside that changing any code before the violation causes the fingerprint to change, resulting in GitLab reporting one fixed and one new violation even though it is a pre-existing violation.
Ruff now uses a more stable location-agnostic fingerprint to minimize that existing violations incorrectly get marked as fixed and re-reported as new violations.
Expect GitLab to report each pre-existing violation in your project as fixed and a new violation in your Ruff upgrade PR.
## 0.0.283 / 0.284
### The target Python version now defaults to 3.8 instead of 3.10 ([#6397](https://github.com/astral-sh/ruff/pull/6397))
Previously, when a target Python version was not specified, Ruff would use a default of Python 3.10. However, it is safer to default to an _older_ Python version to avoid assuming the availability of new features. We now default to the oldest supported Python version which is currently Python 3.8.
(We still support Python 3.7 but since [it has reached EOL](https://devguide.python.org/versions/#unsupported-versions) we've decided not to make it the default here.)
Note this change was announced in 0.0.283 but not active until 0.0.284.
## 0.0.277
### `.ipynb_checkpoints`, `.pyenv`, `.pytest_cache`, and `.vscode` are now excluded by default ([#5513](https://github.com/astral-sh/ruff/pull/5513))
@@ -274,4 +299,4 @@ default.
`pyproject.toml` files are now resolved hierarchically, such that for each Python file, we find
the first `pyproject.toml` file in its path, and use that to determine its lint settings.
See the [documentation](https://beta.ruff.rs/docs/configuration/#python-file-discovery) for more.
See the [documentation](https://docs.astral.sh/ruff/configuration/#python-file-discovery) for more.

View File

@@ -69,6 +69,13 @@ and pre-commit to run some validation checks:
pipx install pre-commit # or `pip install pre-commit` if you have a virtualenv
```
You can optionally install pre-commit hooks to automatically run the validation checks
when making a commit:
```shell
pre-commit install
```
### Development
After cloning the repository, run Ruff locally from the repository root with:
@@ -105,11 +112,11 @@ Ruff is structured as a monorepo with a [flat crate structure](https://matklad.g
such that all crates are contained in a flat `crates` directory.
The vast majority of the code, including all lint rules, lives in the `ruff` crate (located at
`crates/ruff`). As a contributor, that's the crate that'll be most relevant to you.
`crates/ruff_linter`). As a contributor, that's the crate that'll be most relevant to you.
At time of writing, the repository includes the following crates:
- `crates/ruff`: library crate containing all lint rules and the core logic for running them.
- `crates/ruff_linter`: library crate containing all lint rules and the core logic for running them.
If you're working on a rule, this is the crate for you.
- `crates/ruff_benchmark`: binary crate for running micro-benchmarks.
- `crates/ruff_cache`: library crate for caching lint results.
@@ -122,9 +129,9 @@ At time of writing, the repository includes the following crates:
intermediate representation. The backend for `ruff_python_formatter`.
- `crates/ruff_index`: library crate inspired by `rustc_index`.
- `crates/ruff_macros`: proc macro crate containing macros used by Ruff.
- `crates/ruff_notebook`: library crate for parsing and manipulating Jupyter notebooks.
- `crates/ruff_python_ast`: library crate containing Python-specific AST types and utilities.
- `crates/ruff_python_codegen`: library crate containing utilities for generating Python source code.
- `crates/ruff_python_codegen`: library crate containing utilities for generating Python source code.
- `crates/ruff_python_formatter`: library crate implementing the Python formatter. Emits an
intermediate representation for each node, which `ruff_formatter` prints based on the configured
line length.
@@ -146,7 +153,7 @@ At a high level, the steps involved in adding a new lint rule are as follows:
1. Determine a name for the new rule as per our [rule naming convention](#rule-naming-convention)
(e.g., `AssertFalse`, as in, "allow `assert False`").
1. Create a file for your rule (e.g., `crates/ruff/src/rules/flake8_bugbear/rules/assert_false.rs`).
1. Create a file for your rule (e.g., `crates/ruff_linter/src/rules/flake8_bugbear/rules/assert_false.rs`).
1. In that file, define a violation struct (e.g., `pub struct AssertFalse`). You can grep for
`#[violation]` to see examples.
@@ -155,21 +162,21 @@ At a high level, the steps involved in adding a new lint rule are as follows:
(e.g., `pub(crate) fn assert_false`) based on whatever inputs are required for the rule (e.g.,
an `ast::StmtAssert` node).
1. Define the logic for invoking the diagnostic in `crates/ruff/src/checkers/ast/analyze` (for
AST-based rules), `crates/ruff/src/checkers/tokens.rs` (for token-based rules),
`crates/ruff/src/checkers/physical_lines.rs` (for text-based rules),
`crates/ruff/src/checkers/filesystem.rs` (for filesystem-based rules), etc. For AST-based rules,
1. Define the logic for invoking the diagnostic in `crates/ruff_linter/src/checkers/ast/analyze` (for
AST-based rules), `crates/ruff_linter/src/checkers/tokens.rs` (for token-based rules),
`crates/ruff_linter/src/checkers/physical_lines.rs` (for text-based rules),
`crates/ruff_linter/src/checkers/filesystem.rs` (for filesystem-based rules), etc. For AST-based rules,
you'll likely want to modify `analyze/statement.rs` (if your rule is based on analyzing
statements, like imports) or `analyze/expression.rs` (if your rule is based on analyzing
expressions, like function calls).
1. Map the violation struct to a rule code in `crates/ruff/src/codes.rs` (e.g., `B011`).
1. Map the violation struct to a rule code in `crates/ruff_linter/src/codes.rs` (e.g., `B011`).
1. Add proper [testing](#rule-testing-fixtures-and-snapshots) for your rule.
1. Update the generated files (documentation and generated code).
To trigger the violation, you'll likely want to augment the logic in `crates/ruff/src/checkers/ast.rs`
To trigger the violation, you'll likely want to augment the logic in `crates/ruff_linter/src/checkers/ast.rs`
to call your new function at the appropriate time and with the appropriate inputs. The `Checker`
defined therein is a Python AST visitor, which iterates over the AST, building up a semantic model,
and calling out to lint rule analyzer functions as it goes.
@@ -214,7 +221,7 @@ Ruff's output for each fixture, which you can then commit alongside your changes
Once you've completed the code for the rule itself, you can define tests with the following steps:
1. Add a Python file to `crates/ruff/resources/test/fixtures/[linter]` that contains the code you
1. Add a Python file to `crates/ruff_linter/resources/test/fixtures/[linter]` that contains the code you
want to test. The file name should match the rule name (e.g., `E402.py`), and it should include
examples of both violations and non-violations.
@@ -223,16 +230,16 @@ Once you've completed the code for the rule itself, you can define tests with th
For example, if you're adding a new rule named `E402`, you would run:
```shell
cargo run -p ruff_cli -- check crates/ruff/resources/test/fixtures/pycodestyle/E402.py --no-cache --select E402
cargo run -p ruff_cli -- check crates/ruff_linter/resources/test/fixtures/pycodestyle/E402.py --no-cache --select E402
```
**Note:** Only a subset of rules are enabled by default. When testing a new rule, ensure that
you activate it by adding `--select ${rule_code}` to the command.
1. Add the test to the relevant `crates/ruff/src/rules/[linter]/mod.rs` file. If you're contributing
1. Add the test to the relevant `crates/ruff_linter/src/rules/[linter]/mod.rs` file. If you're contributing
a rule to a pre-existing set, you should be able to find a similar example to pattern-match
against. If you're adding a new linter, you'll need to create a new `mod.rs` file (see,
e.g., `crates/ruff/src/rules/flake8_bugbear/mod.rs`)
e.g., `crates/ruff_linter/src/rules/flake8_bugbear/mod.rs`)
1. Run `cargo test`. Your test will fail, but you'll be prompted to follow-up
with `cargo insta review`. Run `cargo insta review`, review and accept the generated snapshot,
@@ -244,25 +251,24 @@ Once you've completed the code for the rule itself, you can define tests with th
Ruff's user-facing settings live in a few different places.
First, the command-line options are defined via the `Cli` struct in `crates/ruff/src/cli.rs`.
First, the command-line options are defined via the `Args` struct in `crates/ruff_cli/src/args.rs`.
Second, the `pyproject.toml` options are defined in `crates/ruff/src/settings/options.rs` (via the
`Options` struct), `crates/ruff/src/settings/configuration.rs` (via the `Configuration` struct), and
`crates/ruff/src/settings/mod.rs` (via the `Settings` struct). These represent, respectively: the
schema used to parse the `pyproject.toml` file; an internal, intermediate representation; and the
final, internal representation used to power Ruff.
Second, the `pyproject.toml` options are defined in `crates/ruff_workspace/src/options.rs` (via the
`Options` struct), `crates/ruff_workspace/src/configuration.rs` (via the `Configuration` struct),
and `crates/ruff_workspace/src/settings.rs` (via the `Settings` struct), which then includes
the `LinterSettings` struct as a field.
These represent, respectively: the schema used to parse the `pyproject.toml` file; an internal,
intermediate representation; and the final, internal representation used to power Ruff.
To add a new configuration option, you'll likely want to modify these latter few files (along with
`cli.rs`, if appropriate). If you want to pattern-match against an existing example, grep for
`arg.rs`, if appropriate). If you want to pattern-match against an existing example, grep for
`dummy_variable_rgx`, which defines a regular expression to match against acceptable unused
variables (e.g., `_`).
Note that plugin-specific configuration options are defined in their own modules (e.g.,
`crates/ruff/src/flake8_unused_arguments/settings.rs`).
You may also want to add the new configuration option to the `flake8-to-ruff` tool, which is
responsible for converting `flake8` configuration files to Ruff's TOML format. This logic
lives in `crates/ruff/src/flake8_to_ruff/converter.rs`.
`Settings` in `crates/ruff_linter/src/flake8_unused_arguments/settings.rs` coupled with
`Flake8UnusedArgumentsOptions` in `crates/ruff_workspace/src/options.rs`).
Finally, regenerate the documentation and generated code with `cargo dev generate-all`.
@@ -355,46 +361,46 @@ First, clone [CPython](https://github.com/python/cpython). It's a large and dive
which makes it a good target for benchmarking.
```shell
git clone --branch 3.10 https://github.com/python/cpython.git crates/ruff/resources/test/cpython
git clone --branch 3.10 https://github.com/python/cpython.git crates/ruff_linter/resources/test/cpython
```
To benchmark the release build:
```shell
cargo build --release && hyperfine --warmup 10 \
"./target/release/ruff ./crates/ruff/resources/test/cpython/ --no-cache -e" \
"./target/release/ruff ./crates/ruff/resources/test/cpython/ -e"
"./target/release/ruff ./crates/ruff_linter/resources/test/cpython/ --no-cache -e" \
"./target/release/ruff ./crates/ruff_linter/resources/test/cpython/ -e"
Benchmark 1: ./target/release/ruff ./crates/ruff/resources/test/cpython/ --no-cache
Benchmark 1: ./target/release/ruff ./crates/ruff_linter/resources/test/cpython/ --no-cache
Time (mean ± σ): 293.8 ms ± 3.2 ms [User: 2384.6 ms, System: 90.3 ms]
Range (min … max): 289.9 ms … 301.6 ms 10 runs
Benchmark 2: ./target/release/ruff ./crates/ruff/resources/test/cpython/
Benchmark 2: ./target/release/ruff ./crates/ruff_linter/resources/test/cpython/
Time (mean ± σ): 48.0 ms ± 3.1 ms [User: 65.2 ms, System: 124.7 ms]
Range (min … max): 45.0 ms … 66.7 ms 62 runs
Summary
'./target/release/ruff ./crates/ruff/resources/test/cpython/' ran
6.12 ± 0.41 times faster than './target/release/ruff ./crates/ruff/resources/test/cpython/ --no-cache'
'./target/release/ruff ./crates/ruff_linter/resources/test/cpython/' ran
6.12 ± 0.41 times faster than './target/release/ruff ./crates/ruff_linter/resources/test/cpython/ --no-cache'
```
To benchmark against the ecosystem's existing tools:
```shell
hyperfine --ignore-failure --warmup 5 \
"./target/release/ruff ./crates/ruff/resources/test/cpython/ --no-cache" \
"pyflakes crates/ruff/resources/test/cpython" \
"./target/release/ruff ./crates/ruff_linter/resources/test/cpython/ --no-cache" \
"pyflakes crates/ruff_linter/resources/test/cpython" \
"autoflake --recursive --expand-star-imports --remove-all-unused-imports --remove-unused-variables --remove-duplicate-keys resources/test/cpython" \
"pycodestyle crates/ruff/resources/test/cpython" \
"flake8 crates/ruff/resources/test/cpython"
"pycodestyle crates/ruff_linter/resources/test/cpython" \
"flake8 crates/ruff_linter/resources/test/cpython"
Benchmark 1: ./target/release/ruff ./crates/ruff/resources/test/cpython/ --no-cache
Benchmark 1: ./target/release/ruff ./crates/ruff_linter/resources/test/cpython/ --no-cache
Time (mean ± σ): 294.3 ms ± 3.3 ms [User: 2467.5 ms, System: 89.6 ms]
Range (min … max): 291.1 ms … 302.8 ms 10 runs
Warning: Ignoring non-zero exit code.
Benchmark 2: pyflakes crates/ruff/resources/test/cpython
Benchmark 2: pyflakes crates/ruff_linter/resources/test/cpython
Time (mean ± σ): 15.786 s ± 0.143 s [User: 15.560 s, System: 0.214 s]
Range (min … max): 15.640 s … 16.157 s 10 runs
@@ -404,31 +410,31 @@ Benchmark 3: autoflake --recursive --expand-star-imports --remove-all-unused-imp
Time (mean ± σ): 6.175 s ± 0.169 s [User: 54.102 s, System: 1.057 s]
Range (min … max): 5.950 s … 6.391 s 10 runs
Benchmark 4: pycodestyle crates/ruff/resources/test/cpython
Benchmark 4: pycodestyle crates/ruff_linter/resources/test/cpython
Time (mean ± σ): 46.921 s ± 0.508 s [User: 46.699 s, System: 0.202 s]
Range (min … max): 46.171 s … 47.863 s 10 runs
Warning: Ignoring non-zero exit code.
Benchmark 5: flake8 crates/ruff/resources/test/cpython
Benchmark 5: flake8 crates/ruff_linter/resources/test/cpython
Time (mean ± σ): 12.260 s ± 0.321 s [User: 102.934 s, System: 1.230 s]
Range (min … max): 11.848 s … 12.933 s 10 runs
Warning: Ignoring non-zero exit code.
Summary
'./target/release/ruff ./crates/ruff/resources/test/cpython/ --no-cache' ran
'./target/release/ruff ./crates/ruff_linter/resources/test/cpython/ --no-cache' ran
20.98 ± 0.62 times faster than 'autoflake --recursive --expand-star-imports --remove-all-unused-imports --remove-unused-variables --remove-duplicate-keys resources/test/cpython'
41.66 ± 1.18 times faster than 'flake8 crates/ruff/resources/test/cpython'
53.64 ± 0.77 times faster than 'pyflakes crates/ruff/resources/test/cpython'
159.43 ± 2.48 times faster than 'pycodestyle crates/ruff/resources/test/cpython'
41.66 ± 1.18 times faster than 'flake8 crates/ruff_linter/resources/test/cpython'
53.64 ± 0.77 times faster than 'pyflakes crates/ruff_linter/resources/test/cpython'
159.43 ± 2.48 times faster than 'pycodestyle crates/ruff_linter/resources/test/cpython'
```
To benchmark a subset of rules, e.g. `LineTooLong` and `DocLineTooLong`:
```shell
cargo build --release && hyperfine --warmup 10 \
"./target/release/ruff ./crates/ruff/resources/test/cpython/ --no-cache -e --select W505,E501"
"./target/release/ruff ./crates/ruff_linter/resources/test/cpython/ --no-cache -e --select W505,E501"
```
You can run `poetry install` from `./scripts/benchmarks` to create a working environment for the
@@ -461,10 +467,10 @@ rm Lib/test/bad_coding.py \
Lib/test/test_typing.py
```
Then, from `crates/ruff/resources/test/cpython`, run: `time pylint -j 0 -E $(git ls-files '*.py')`. This
Then, from `crates/ruff_linter/resources/test/cpython`, run: `time pylint -j 0 -E $(git ls-files '*.py')`. This
will execute Pylint with maximum parallelism and only report errors.
To benchmark Pyupgrade, run the following from `crates/ruff/resources/test/cpython`:
To benchmark Pyupgrade, run the following from `crates/ruff_linter/resources/test/cpython`:
```shell
hyperfine --ignore-failure --warmup 5 --prepare "git reset --hard HEAD" \
@@ -565,7 +571,7 @@ An alternative is to convert the perf data to `flamegraph.svg` using
[flamegraph](https://github.com/flamegraph-rs/flamegraph) (`cargo install flamegraph`):
```shell
flamegraph --perfdata perf.data
flamegraph --perfdata perf.data --no-inline
```
#### Mac
@@ -712,8 +718,8 @@ Module {
- `cargo dev generate-cli-help`, `cargo dev generate-docs` and `cargo dev generate-json-schema`:
Update just `docs/configuration.md`, `docs/rules` and `ruff.schema.json` respectively.
- `cargo dev generate-options`: Generate a markdown-compatible table of all `pyproject.toml`
options. Used for <https://beta.ruff.rs/docs/settings/>
- `cargo dev generate-rules-table`: Generate a markdown-compatible table of all rules. Used for <https://beta.ruff.rs/docs/rules/>
options. Used for <https://docs.astral.sh/ruff/settings/>.
- `cargo dev generate-rules-table`: Generate a markdown-compatible table of all rules. Used for <https://docs.astral.sh/ruff/rules/>.
- `cargo dev round-trip <python file or jupyter notebook>`: Read a Python file or Jupyter Notebook,
parse it, serialize the parsed representation and write it back. Used to check how good our
representation is so that fixes don't rewrite irrelevant parts of a file.
@@ -771,7 +777,7 @@ To understand Ruff's import categorization system, we first need to define two c
- "Package root": The top-most directory defining the Python package that includes a given Python
file. To find the package root for a given Python file, traverse up its parent directories until
you reach a parent directory that doesn't contain an `__init__.py` file (and isn't marked as
a [namespace package](https://beta.ruff.rs/docs/settings/#namespace-packages)); take the directory
a [namespace package](https://docs.astral.sh/ruff/settings/#namespace-packages)); take the directory
just before that, i.e., the first directory in the package.
For example, given:
@@ -860,7 +866,7 @@ There are three ways in which an import can be categorized as "first-party":
package (e.g., `from foo import bar` or `import foo.bar`), they'll be classified as first-party
automatically. This check is as simple as comparing the first segment of the current file's
module path to the first segment of the import.
1. **Source roots**: Ruff supports a `[src](https://beta.ruff.rs/docs/settings/#src)` setting, which
1. **Source roots**: Ruff supports a `[src](https://docs.astral.sh/ruff/settings/#src)` setting, which
sets the directories to scan when identifying first-party imports. The algorithm is
straightforward: given an import, like `import foo`, iterate over the directories enumerated in
the `src` setting and, for each directory, check for the existence of a subdirectory `foo` or a

1355
Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -4,9 +4,9 @@ resolver = "2"
[workspace.package]
edition = "2021"
rust-version = "1.70"
homepage = "https://beta.ruff.rs/docs"
documentation = "https://beta.ruff.rs/docs"
rust-version = "1.71"
homepage = "https://docs.astral.sh/ruff"
documentation = "https://docs.astral.sh/ruff"
repository = "https://github.com/astral-sh/ruff"
authors = ["Charlie Marsh <charlie.r.marsh@gmail.com>"]
license = "MIT"
@@ -14,42 +14,49 @@ license = "MIT"
[workspace.dependencies]
anyhow = { version = "1.0.69" }
bitflags = { version = "2.3.1" }
chrono = { version = "0.4.23", default-features = false, features = ["clock"] }
clap = { version = "4.1.8", features = ["derive"] }
chrono = { version = "0.4.31", default-features = false, features = ["clock"] }
clap = { version = "4.4.4", features = ["derive"] }
colored = { version = "2.0.0" }
filetime = { version = "0.2.20" }
glob = { version = "0.3.1" }
globset = { version = "0.4.10" }
ignore = { version = "0.4.20" }
insta = { version = "1.31.0", feature = ["filters", "glob"] }
is-macro = { version = "0.2.2" }
itertools = { version = "0.10.5" }
insta = { version = "1.32.0", feature = ["filters", "glob"] }
is-macro = { version = "0.3.0" }
itertools = { version = "0.11.0" }
log = { version = "0.4.17" }
memchr = "2.5.0"
memchr = "2.6.3"
num-bigint = { version = "0.4.3" }
num-traits = { version = "0.2.15" }
once_cell = { version = "1.17.1" }
path-absolutize = { version = "3.0.14" }
proc-macro2 = { version = "1.0.51" }
path-absolutize = { version = "3.1.1" }
proc-macro2 = { version = "1.0.67" }
quote = { version = "1.0.23" }
regex = { version = "1.7.1" }
regex = { version = "1.9.5" }
rustc-hash = { version = "1.1.0" }
schemars = { version = "0.8.12" }
schemars = { version = "0.8.15" }
serde = { version = "1.0.152", features = ["derive"] }
serde_json = { version = "1.0.93" }
serde_json = { version = "1.0.107" }
shellexpand = { version = "3.0.0" }
similar = { version = "2.2.1", features = ["inline"] }
smallvec = { version = "1.10.0" }
strum = { version = "0.24.1", features = ["strum_macros"] }
strum_macros = { version = "0.24.3" }
syn = { version = "2.0.15" }
test-case = { version = "3.0.0" }
thiserror = { version = "1.0.43" }
toml = { version = "0.7.2" }
smallvec = { version = "1.11.1" }
static_assertions = "1.1.0"
strum = { version = "0.25.0", features = ["strum_macros"] }
strum_macros = { version = "0.25.2" }
syn = { version = "2.0.37" }
test-case = { version = "3.2.1" }
thiserror = { version = "1.0.48" }
toml = { version = "0.7.8" }
tracing = "0.1.37"
tracing-indicatif = "0.3.4"
tracing-subscriber = { version = "0.3.17", features = ["env-filter"] }
unicode-ident = "1.0.12"
unicode-width = "0.1.11"
uuid = { version = "1.4.1", features = ["v4", "fast-rng", "macro-diagnostics", "js"] }
wsl = { version = "0.1.0" }
# v1.0.1
libcst = { git = "https://github.com/Instagram/LibCST.git", rev = "3cacca1a1029f05707e50703b49fe3dd860aa839", default-features = false }
libcst = { version = "0.1.0", default-features = false }
[profile.release]
lto = "fat"

47
LICENSE
View File

@@ -1194,7 +1194,27 @@ are:
- flake8-self, licensed as follows:
"""
Freely Distributable
MIT License
Copyright (c) 2023 Korijn van Golen
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
"""
- flake8-django, licensed under the GPL license.
@@ -1224,6 +1244,31 @@ are:
SOFTWARE.
"""
- flake8-logging, licensed as follows:
"""
MIT License
Copyright (c) 2023 Adam Johnson
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
"""
- Pyright, licensed as follows:
"""
MIT License

View File

@@ -8,7 +8,7 @@
[![image](https://img.shields.io/pypi/pyversions/ruff.svg)](https://pypi.python.org/pypi/ruff)
[![Actions status](https://github.com/astral-sh/ruff/workflows/CI/badge.svg)](https://github.com/astral-sh/ruff/actions)
[**Discord**](https://discord.gg/c9MhzV8aU5) | [**Docs**](https://beta.ruff.rs/docs/) | [**Playground**](https://play.ruff.rs/)
[**Discord**](https://discord.gg/c9MhzV8aU5) | [**Docs**](https://docs.astral.sh/ruff/) | [**Playground**](https://play.ruff.rs/)
An extremely fast Python linter, written in Rust.
@@ -30,13 +30,13 @@ An extremely fast Python linter, written in Rust.
- 🤝 Python 3.11 compatibility
- 📦 Built-in caching, to avoid re-analyzing unchanged files
- 🔧 Autofix support, for automatic error correction (e.g., automatically remove unused imports)
- 📏 Over [500 built-in rules](https://beta.ruff.rs/docs/rules/)
- ⚖️ [Near-parity](https://beta.ruff.rs/docs/faq/#how-does-ruff-compare-to-flake8) with the
- 📏 Over [600 built-in rules](https://docs.astral.sh/ruff/rules/)
- ⚖️ [Near-parity](https://docs.astral.sh/ruff/faq/#how-does-ruff-compare-to-flake8) with the
built-in Flake8 rule set
- 🔌 Native re-implementations of dozens of Flake8 plugins, like flake8-bugbear
- ⌨️ First-party [editor integrations](https://beta.ruff.rs/docs/editor-integrations/) for
- ⌨️ First-party [editor integrations](https://docs.astral.sh/ruff/editor-integrations/) for
[VS Code](https://github.com/astral-sh/ruff-vscode) and [more](https://github.com/astral-sh/ruff-lsp)
- 🌎 Monorepo-friendly, with [hierarchical and cascading configuration](https://beta.ruff.rs/docs/configuration/#pyprojecttoml-discovery)
- 🌎 Monorepo-friendly, with [hierarchical and cascading configuration](https://docs.astral.sh/ruff/configuration/#pyprojecttoml-discovery)
Ruff aims to be orders of magnitude faster than alternative tools while integrating more
functionality behind a single, common interface.
@@ -98,7 +98,7 @@ developer of [Zulip](https://github.com/zulip/zulip):
## Table of Contents
For more, see the [documentation](https://beta.ruff.rs/docs/).
For more, see the [documentation](https://docs.astral.sh/ruff/).
1. [Getting Started](#getting-started)
1. [Configuration](#configuration)
@@ -111,7 +111,7 @@ For more, see the [documentation](https://beta.ruff.rs/docs/).
## Getting Started
For more, see the [documentation](https://beta.ruff.rs/docs/).
For more, see the [documentation](https://docs.astral.sh/ruff/).
### Installation
@@ -122,7 +122,7 @@ pip install ruff
```
You can also install Ruff via [Homebrew](https://formulae.brew.sh/formula/ruff), [Conda](https://anaconda.org/conda-forge/ruff),
and with [a variety of other package managers](https://beta.ruff.rs/docs/installation/).
and with [a variety of other package managers](https://docs.astral.sh/ruff/installation/).
### Usage
@@ -140,7 +140,7 @@ Ruff can also be used as a [pre-commit](https://pre-commit.com) hook:
```yaml
- repo: https://github.com/astral-sh/ruff-pre-commit
# Ruff version.
rev: v0.0.280
rev: v0.0.290
hooks:
- id: ruff
```
@@ -165,7 +165,7 @@ jobs:
### Configuration
Ruff can be configured through a `pyproject.toml`, `ruff.toml`, or `.ruff.toml` file (see:
[_Configuration_](https://beta.ruff.rs/docs/configuration/), or [_Settings_](https://beta.ruff.rs/docs/settings/)
[_Configuration_](https://docs.astral.sh/ruff/configuration/), or [_Settings_](https://docs.astral.sh/ruff/settings/)
for a complete list of all configuration options).
If left unspecified, the default configuration is equivalent to:
@@ -211,8 +211,8 @@ line-length = 88
# Allow unused variables when underscore-prefixed.
dummy-variable-rgx = "^(_+|(_+[a-zA-Z0-9_]*[a-zA-Z0-9]+?))$"
# Assume Python 3.10.
target-version = "py310"
# Assume Python 3.8
target-version = "py38"
[tool.ruff.mccabe]
# Unlike Flake8, default to a complexity level of 10.
@@ -233,12 +233,12 @@ linting command.
<!-- Begin section: Rules -->
**Ruff supports over 500 lint rules**, many of which are inspired by popular tools like Flake8,
**Ruff supports over 600 lint rules**, many of which are inspired by popular tools like Flake8,
isort, pyupgrade, and others. Regardless of the rule's origin, Ruff re-implements every rule in
Rust as a first-party feature.
By default, Ruff enables Flake8's `E` and `F` rules. Ruff supports all rules from the `F` category,
and a [subset](https://beta.ruff.rs/docs/rules/#error-e) of the `E` category, omitting those
and a [subset](https://docs.astral.sh/ruff/rules/#error-e) of the `E` category, omitting those
stylistic rules made obsolete by the use of an autoformatter, like
[Black](https://github.com/psf/black).
@@ -274,6 +274,7 @@ quality tools, including:
- [flake8-gettext](https://pypi.org/project/flake8-gettext/)
- [flake8-implicit-str-concat](https://pypi.org/project/flake8-implicit-str-concat/)
- [flake8-import-conventions](https://github.com/joaopalmeiro/flake8-import-conventions)
- [flake8-logging](https://pypi.org/project/flake8-logging/)
- [flake8-logging-format](https://pypi.org/project/flake8-logging-format/)
- [flake8-no-pep420](https://pypi.org/project/flake8-no-pep420)
- [flake8-pie](https://pypi.org/project/flake8-pie/)
@@ -303,12 +304,12 @@ quality tools, including:
- [tryceratops](https://pypi.org/project/tryceratops/)
- [yesqa](https://pypi.org/project/yesqa/)
For a complete enumeration of the supported rules, see [_Rules_](https://beta.ruff.rs/docs/rules/).
For a complete enumeration of the supported rules, see [_Rules_](https://docs.astral.sh/ruff/rules/).
## Contributing
Contributions are welcome and highly appreciated. To get started, check out the
[**contributing guidelines**](https://beta.ruff.rs/docs/contributing/).
[**contributing guidelines**](https://docs.astral.sh/ruff/contributing/).
You can also join us on [**Discord**](https://discord.gg/c9MhzV8aU5).
@@ -398,6 +399,7 @@ Ruff is used by a number of major open-source projects and companies, including:
- [Pydantic](https://github.com/pydantic/pydantic)
- [Pylint](https://github.com/PyCQA/pylint)
- [Reflex](https://github.com/reflex-dev/reflex)
- [Rippling](https://rippling.com)
- [Robyn](https://github.com/sansyrox/robyn)
- Scale AI ([Launch SDK](https://github.com/scaleapi/launch-python-client))
- Snowflake ([SnowCLI](https://github.com/Snowflake-Labs/snowcli))

View File

@@ -1,6 +1,6 @@
[package]
name = "flake8-to-ruff"
version = "0.0.280"
version = "0.0.290"
description = """
Convert Flake8 configuration files to Ruff configuration files.
"""
@@ -13,13 +13,17 @@ repository = { workspace = true }
license = { workspace = true }
[dependencies]
ruff = { path = "../ruff", default-features = false }
ruff_linter = { path = "../ruff_linter", default-features = false }
ruff_workspace = { path = "../ruff_workspace" }
anyhow = { workspace = true }
clap = { workspace = true }
colored = { workspace = true }
configparser = { version = "3.0.2" }
itertools = { workspace = true }
log = { workspace = true }
once_cell = { workspace = true }
pep440_rs = { version = "0.3.1", features = ["serde"] }
regex = { workspace = true }
rustc-hash = { workspace = true }
serde = { workspace = true }
@@ -27,3 +31,6 @@ serde_json = { workspace = true }
strum = { workspace = true }
strum_macros = { workspace = true }
toml = { workspace = true }
[dev-dependencies]
pretty_assertions = "1.3.0"

View File

@@ -86,7 +86,7 @@ flake8-to-ruff path/to/.flake8 --plugin flake8-builtins --plugin flake8-quotes
configuration options that don't exist in Flake8.)
1. Ruff will omit any rule codes that are unimplemented or unsupported by Ruff, including rule
codes from unsupported plugins. (See the
[documentation](https://beta.ruff.rs/docs/faq/#how-does-ruff-compare-to-flake8) for the complete
[documentation](https://docs.astral.sh/ruff/faq/#how-does-ruff-compare-to-flake8) for the complete
list of supported plugins.)
## License

View File

@@ -0,0 +1,13 @@
//! Extract Black configuration settings from a pyproject.toml.
use ruff_linter::line_width::LineLength;
use ruff_linter::settings::types::PythonVersion;
use serde::{Deserialize, Serialize};
#[derive(Debug, PartialEq, Eq, Serialize, Deserialize, Default)]
pub(crate) struct Black {
#[serde(alias = "line-length", alias = "line_length")]
pub(crate) line_length: Option<LineLength>,
#[serde(alias = "target-version", alias = "target_version")]
pub(crate) target_version: Option<Vec<PythonVersion>>,
}

View File

@@ -1,25 +1,25 @@
use std::collections::{HashMap, HashSet};
use std::str::FromStr;
use anyhow::Result;
use itertools::Itertools;
use crate::line_width::LineLength;
use crate::registry::Linter;
use crate::rule_selector::RuleSelector;
use crate::rules::flake8_pytest_style::types::{
use ruff_linter::line_width::LineLength;
use ruff_linter::registry::Linter;
use ruff_linter::rule_selector::RuleSelector;
use ruff_linter::rules::flake8_pytest_style::types::{
ParametrizeNameType, ParametrizeValuesRowType, ParametrizeValuesType,
};
use crate::rules::flake8_quotes::settings::Quote;
use crate::rules::flake8_tidy_imports::settings::Strictness;
use crate::rules::pydocstyle::settings::Convention;
use crate::rules::{
flake8_annotations, flake8_bugbear, flake8_builtins, flake8_errmsg, flake8_pytest_style,
flake8_quotes, flake8_tidy_imports, mccabe, pep8_naming, pydocstyle,
use ruff_linter::rules::flake8_quotes::settings::Quote;
use ruff_linter::rules::flake8_tidy_imports::settings::Strictness;
use ruff_linter::rules::pydocstyle::settings::Convention;
use ruff_linter::settings::types::PythonVersion;
use ruff_linter::warn_user;
use ruff_workspace::options::{
Flake8AnnotationsOptions, Flake8BugbearOptions, Flake8BuiltinsOptions, Flake8ErrMsgOptions,
Flake8PytestStyleOptions, Flake8QuotesOptions, Flake8TidyImportsOptions, McCabeOptions,
Options, Pep8NamingOptions, PydocstyleOptions,
};
use crate::settings::options::Options;
use crate::settings::pyproject::Pyproject;
use crate::settings::types::PythonVersion;
use crate::warn_user;
use ruff_workspace::pyproject::Pyproject;
use super::external_config::ExternalConfig;
use super::plugin::Plugin;
@@ -30,11 +30,11 @@ const DEFAULT_SELECTORS: &[RuleSelector] = &[
RuleSelector::Linter(Linter::Pycodestyle),
];
pub fn convert(
pub(crate) fn convert(
config: &HashMap<String, HashMap<String, Option<String>>>,
external_config: &ExternalConfig,
plugins: Option<Vec<Plugin>>,
) -> Result<Pyproject> {
) -> Pyproject {
// Extract the Flake8 section.
let flake8 = config
.get("flake8")
@@ -103,16 +103,16 @@ pub fn convert(
// Parse each supported option.
let mut options = Options::default();
let mut flake8_annotations = flake8_annotations::settings::Options::default();
let mut flake8_bugbear = flake8_bugbear::settings::Options::default();
let mut flake8_builtins = flake8_builtins::settings::Options::default();
let mut flake8_errmsg = flake8_errmsg::settings::Options::default();
let mut flake8_pytest_style = flake8_pytest_style::settings::Options::default();
let mut flake8_quotes = flake8_quotes::settings::Options::default();
let mut flake8_tidy_imports = flake8_tidy_imports::options::Options::default();
let mut mccabe = mccabe::settings::Options::default();
let mut pep8_naming = pep8_naming::settings::Options::default();
let mut pydocstyle = pydocstyle::settings::Options::default();
let mut flake8_annotations = Flake8AnnotationsOptions::default();
let mut flake8_bugbear = Flake8BugbearOptions::default();
let mut flake8_builtins = Flake8BuiltinsOptions::default();
let mut flake8_errmsg = Flake8ErrMsgOptions::default();
let mut flake8_pytest_style = Flake8PytestStyleOptions::default();
let mut flake8_quotes = Flake8QuotesOptions::default();
let mut flake8_tidy_imports = Flake8TidyImportsOptions::default();
let mut mccabe = McCabeOptions::default();
let mut pep8_naming = Pep8NamingOptions::default();
let mut pydocstyle = PydocstyleOptions::default();
for (key, value) in flake8 {
if let Some(value) = value {
match key.as_str() {
@@ -120,10 +120,8 @@ pub fn convert(
"builtins" => {
options.builtins = Some(parser::parse_strings(value.as_ref()));
}
"max-line-length" | "max_line_length" => match value.parse::<usize>() {
Ok(line_length) => {
options.line_length = Some(LineLength::from(line_length));
}
"max-line-length" | "max_line_length" => match LineLength::from_str(value) {
Ok(line_length) => options.line_length = Some(line_length),
Err(e) => {
warn_user!("Unable to parse '{key}' property: {e}");
}
@@ -372,41 +370,41 @@ pub fn convert(
.sorted_by_key(RuleSelector::prefix_and_code)
.collect(),
);
if flake8_annotations != flake8_annotations::settings::Options::default() {
if flake8_annotations != Flake8AnnotationsOptions::default() {
options.flake8_annotations = Some(flake8_annotations);
}
if flake8_bugbear != flake8_bugbear::settings::Options::default() {
if flake8_bugbear != Flake8BugbearOptions::default() {
options.flake8_bugbear = Some(flake8_bugbear);
}
if flake8_builtins != flake8_builtins::settings::Options::default() {
if flake8_builtins != Flake8BuiltinsOptions::default() {
options.flake8_builtins = Some(flake8_builtins);
}
if flake8_errmsg != flake8_errmsg::settings::Options::default() {
if flake8_errmsg != Flake8ErrMsgOptions::default() {
options.flake8_errmsg = Some(flake8_errmsg);
}
if flake8_pytest_style != flake8_pytest_style::settings::Options::default() {
if flake8_pytest_style != Flake8PytestStyleOptions::default() {
options.flake8_pytest_style = Some(flake8_pytest_style);
}
if flake8_quotes != flake8_quotes::settings::Options::default() {
if flake8_quotes != Flake8QuotesOptions::default() {
options.flake8_quotes = Some(flake8_quotes);
}
if flake8_tidy_imports != flake8_tidy_imports::options::Options::default() {
if flake8_tidy_imports != Flake8TidyImportsOptions::default() {
options.flake8_tidy_imports = Some(flake8_tidy_imports);
}
if mccabe != mccabe::settings::Options::default() {
if mccabe != McCabeOptions::default() {
options.mccabe = Some(mccabe);
}
if pep8_naming != pep8_naming::settings::Options::default() {
if pep8_naming != Pep8NamingOptions::default() {
options.pep8_naming = Some(pep8_naming);
}
if pydocstyle != pydocstyle::settings::Options::default() {
if pydocstyle != PydocstyleOptions::default() {
options.pydocstyle = Some(pydocstyle);
}
// Extract any settings from the existing `pyproject.toml`.
if let Some(black) = &external_config.black {
if let Some(line_length) = &black.line_length {
options.line_length = Some(LineLength::from(*line_length));
options.line_length = Some(*line_length);
}
if let Some(target_version) = &black.target_version {
@@ -439,7 +437,7 @@ pub fn convert(
}
// Create the pyproject.toml.
Ok(Pyproject::new(options))
Pyproject::new(options)
}
/// Resolve the set of enabled `RuleSelector` values for the given
@@ -458,19 +456,20 @@ mod tests {
use anyhow::Result;
use itertools::Itertools;
use pep440_rs::VersionSpecifiers;
use pretty_assertions::assert_eq;
use crate::flake8_to_ruff::converter::DEFAULT_SELECTORS;
use crate::flake8_to_ruff::pep621::Project;
use crate::flake8_to_ruff::ExternalConfig;
use crate::line_width::LineLength;
use crate::registry::Linter;
use crate::rule_selector::RuleSelector;
use crate::rules::pydocstyle::settings::Convention;
use crate::rules::{flake8_quotes, pydocstyle};
use crate::settings::options::Options;
use crate::settings::pyproject::Pyproject;
use crate::settings::types::PythonVersion;
use pretty_assertions::assert_eq;
use ruff_linter::line_width::LineLength;
use ruff_linter::registry::Linter;
use ruff_linter::rule_selector::RuleSelector;
use ruff_linter::rules::flake8_quotes;
use ruff_linter::rules::pydocstyle::settings::Convention;
use ruff_linter::settings::types::PythonVersion;
use ruff_workspace::options::{Flake8QuotesOptions, Options, PydocstyleOptions};
use ruff_workspace::pyproject::Pyproject;
use crate::converter::DEFAULT_SELECTORS;
use crate::pep621::Project;
use crate::ExternalConfig;
use super::super::plugin::Plugin;
use super::convert;
@@ -491,20 +490,18 @@ mod tests {
}
#[test]
fn it_converts_empty() -> Result<()> {
fn it_converts_empty() {
let actual = convert(
&HashMap::from([("flake8".to_string(), HashMap::default())]),
&ExternalConfig::default(),
None,
)?;
);
let expected = Pyproject::new(default_options([]));
assert_eq!(actual, expected);
Ok(())
}
#[test]
fn it_converts_dashes() -> Result<()> {
fn it_converts_dashes() {
let actual = convert(
&HashMap::from([(
"flake8".to_string(),
@@ -512,18 +509,16 @@ mod tests {
)]),
&ExternalConfig::default(),
Some(vec![]),
)?;
);
let expected = Pyproject::new(Options {
line_length: Some(LineLength::from(100)),
line_length: Some(LineLength::try_from(100).unwrap()),
..default_options([])
});
assert_eq!(actual, expected);
Ok(())
}
#[test]
fn it_converts_underscores() -> Result<()> {
fn it_converts_underscores() {
let actual = convert(
&HashMap::from([(
"flake8".to_string(),
@@ -531,18 +526,16 @@ mod tests {
)]),
&ExternalConfig::default(),
Some(vec![]),
)?;
);
let expected = Pyproject::new(Options {
line_length: Some(LineLength::from(100)),
line_length: Some(LineLength::try_from(100).unwrap()),
..default_options([])
});
assert_eq!(actual, expected);
Ok(())
}
#[test]
fn it_ignores_parse_errors() -> Result<()> {
fn it_ignores_parse_errors() {
let actual = convert(
&HashMap::from([(
"flake8".to_string(),
@@ -550,15 +543,13 @@ mod tests {
)]),
&ExternalConfig::default(),
Some(vec![]),
)?;
);
let expected = Pyproject::new(default_options([]));
assert_eq!(actual, expected);
Ok(())
}
#[test]
fn it_converts_plugin_options() -> Result<()> {
fn it_converts_plugin_options() {
let actual = convert(
&HashMap::from([(
"flake8".to_string(),
@@ -566,9 +557,9 @@ mod tests {
)]),
&ExternalConfig::default(),
Some(vec![]),
)?;
);
let expected = Pyproject::new(Options {
flake8_quotes: Some(flake8_quotes::settings::Options {
flake8_quotes: Some(Flake8QuotesOptions {
inline_quotes: Some(flake8_quotes::settings::Quote::Single),
multiline_quotes: None,
docstring_quotes: None,
@@ -577,12 +568,10 @@ mod tests {
..default_options([])
});
assert_eq!(actual, expected);
Ok(())
}
#[test]
fn it_converts_docstring_conventions() -> Result<()> {
fn it_converts_docstring_conventions() {
let actual = convert(
&HashMap::from([(
"flake8".to_string(),
@@ -593,9 +582,9 @@ mod tests {
)]),
&ExternalConfig::default(),
Some(vec![Plugin::Flake8Docstrings]),
)?;
);
let expected = Pyproject::new(Options {
pydocstyle: Some(pydocstyle::settings::Options {
pydocstyle: Some(PydocstyleOptions {
convention: Some(Convention::Numpy),
ignore_decorators: None,
property_decorators: None,
@@ -603,12 +592,10 @@ mod tests {
..default_options([Linter::Pydocstyle.into()])
});
assert_eq!(actual, expected);
Ok(())
}
#[test]
fn it_infers_plugins_if_omitted() -> Result<()> {
fn it_infers_plugins_if_omitted() {
let actual = convert(
&HashMap::from([(
"flake8".to_string(),
@@ -616,9 +603,9 @@ mod tests {
)]),
&ExternalConfig::default(),
None,
)?;
);
let expected = Pyproject::new(Options {
flake8_quotes: Some(flake8_quotes::settings::Options {
flake8_quotes: Some(Flake8QuotesOptions {
inline_quotes: Some(flake8_quotes::settings::Quote::Single),
multiline_quotes: None,
docstring_quotes: None,
@@ -627,8 +614,6 @@ mod tests {
..default_options([Linter::Flake8Quotes.into()])
});
assert_eq!(actual, expected);
Ok(())
}
#[test]
@@ -642,7 +627,7 @@ mod tests {
..ExternalConfig::default()
},
Some(vec![]),
)?;
);
let expected = Pyproject::new(Options {
target_version: Some(PythonVersion::Py38),
..default_options([])

View File

@@ -0,0 +1,10 @@
use super::black::Black;
use super::isort::Isort;
use super::pep621::Project;
#[derive(Default)]
pub(crate) struct ExternalConfig<'a> {
pub(crate) black: Option<&'a Black>,
pub(crate) isort: Option<&'a Isort>,
pub(crate) project: Option<&'a Project>,
}

View File

@@ -4,7 +4,7 @@ use serde::{Deserialize, Serialize};
/// The [isort configuration](https://pycqa.github.io/isort/docs/configuration/config_files.html).
#[derive(Debug, PartialEq, Eq, Serialize, Deserialize, Default)]
pub struct Isort {
pub(crate) struct Isort {
#[serde(alias = "src-paths", alias = "src_paths")]
pub src_paths: Option<Vec<String>>,
pub(crate) src_paths: Option<Vec<String>>,
}

View File

@@ -1,13 +1,25 @@
//! Utility to generate Ruff's `pyproject.toml` section from a Flake8 INI file.
mod black;
mod converter;
mod external_config;
mod isort;
mod parser;
mod pep621;
mod plugin;
mod pyproject;
use std::path::PathBuf;
use anyhow::Result;
use clap::Parser;
use configparser::ini::Ini;
use ruff::flake8_to_ruff::{self, ExternalConfig};
use ruff::logging::{set_up_logging, LogLevel};
use crate::converter::convert;
use crate::external_config::ExternalConfig;
use crate::plugin::Plugin;
use crate::pyproject::parse;
use ruff_linter::logging::{set_up_logging, LogLevel};
#[derive(Parser)]
#[command(
@@ -25,7 +37,7 @@ struct Args {
pyproject: Option<PathBuf>,
/// List of plugins to enable.
#[arg(long, value_delimiter = ',')]
plugin: Option<Vec<flake8_to_ruff::Plugin>>,
plugin: Option<Vec<Plugin>>,
}
fn main() -> Result<()> {
@@ -39,7 +51,7 @@ fn main() -> Result<()> {
let config = ini.load(args.file).map_err(|msg| anyhow::anyhow!(msg))?;
// Read the pyproject.toml file.
let pyproject = args.pyproject.map(flake8_to_ruff::parse).transpose()?;
let pyproject = args.pyproject.map(parse).transpose()?;
let external_config = pyproject
.as_ref()
.and_then(|pyproject| pyproject.tool.as_ref())
@@ -57,7 +69,7 @@ fn main() -> Result<()> {
};
// Create Ruff's pyproject.toml section.
let pyproject = flake8_to_ruff::convert(&config, &external_config, args.plugin)?;
let pyproject = convert(&config, &external_config, args.plugin);
#[allow(clippy::print_stdout)]
{

View File

@@ -5,9 +5,8 @@ use once_cell::sync::Lazy;
use regex::Regex;
use rustc_hash::FxHashMap;
use crate::rule_selector::RuleSelector;
use crate::settings::types::PatternPrefixPair;
use crate::warn_user;
use ruff_linter::settings::types::PatternPrefixPair;
use ruff_linter::{warn_user, RuleSelector};
static COMMA_SEPARATED_LIST_RE: Lazy<Regex> = Lazy::new(|| Regex::new(r"[,\s]").unwrap());
@@ -195,10 +194,10 @@ pub(crate) fn collect_per_file_ignores(
mod tests {
use anyhow::Result;
use crate::codes;
use crate::registry::Linter;
use crate::rule_selector::RuleSelector;
use crate::settings::types::PatternPrefixPair;
use ruff_linter::codes;
use ruff_linter::registry::Linter;
use ruff_linter::settings::types::PatternPrefixPair;
use ruff_linter::RuleSelector;
use super::{parse_files_to_codes_mapping, parse_prefix_codes, parse_strings};

View File

@@ -4,7 +4,7 @@ use pep440_rs::VersionSpecifiers;
use serde::{Deserialize, Serialize};
#[derive(Debug, PartialEq, Eq, Serialize, Deserialize, Default)]
pub struct Project {
pub(crate) struct Project {
#[serde(alias = "requires-python", alias = "requires_python")]
pub requires_python: Option<VersionSpecifiers>,
pub(crate) requires_python: Option<VersionSpecifiers>,
}

View File

@@ -3,9 +3,9 @@ use std::fmt;
use std::str::FromStr;
use anyhow::anyhow;
use crate::registry::Linter;
use crate::rule_selector::RuleSelector;
use ruff_linter::registry::Linter;
use ruff_linter::settings::types::PreviewMode;
use ruff_linter::RuleSelector;
#[derive(Copy, Clone, Ord, PartialOrd, Eq, PartialEq)]
pub enum Plugin {
@@ -332,7 +332,7 @@ pub(crate) fn infer_plugins_from_codes(selectors: &HashSet<RuleSelector>) -> Vec
.filter(|plugin| {
for selector in selectors {
if selector
.into_iter()
.rules(PreviewMode::Disabled)
.any(|rule| Linter::from(plugin).rules().any(|r| r == rule))
{
return true;

View File

@@ -0,0 +1,26 @@
use std::path::Path;
use anyhow::Result;
use serde::{Deserialize, Serialize};
use super::black::Black;
use super::isort::Isort;
use super::pep621::Project;
#[derive(Debug, PartialEq, Eq, Serialize, Deserialize)]
pub(crate) struct Tools {
pub(crate) black: Option<Black>,
pub(crate) isort: Option<Isort>,
}
#[derive(Debug, PartialEq, Eq, Serialize, Deserialize)]
pub(crate) struct Pyproject {
pub(crate) tool: Option<Tools>,
pub(crate) project: Option<Project>,
}
pub(crate) fn parse<P: AsRef<Path>>(path: P) -> Result<Pyproject> {
let contents = std::fs::read_to_string(path)?;
let pyproject = toml::from_str::<Pyproject>(&contents)?;
Ok(pyproject)
}

View File

@@ -1,96 +0,0 @@
[package]
name = "ruff"
version = "0.0.280"
publish = false
authors = { workspace = true }
edition = { workspace = true }
rust-version = { workspace = true }
homepage = { workspace = true }
documentation = { workspace = true }
repository = { workspace = true }
license = { workspace = true }
readme = "README.md"
[lib]
name = "ruff"
[dependencies]
ruff_cache = { path = "../ruff_cache" }
ruff_diagnostics = { path = "../ruff_diagnostics", features = ["serde"] }
ruff_index = { path = "../ruff_index" }
ruff_macros = { path = "../ruff_macros" }
ruff_python_ast = { path = "../ruff_python_ast", features = ["serde"] }
ruff_python_codegen = { path = "../ruff_python_codegen" }
ruff_python_index = { path = "../ruff_python_index" }
ruff_python_literal = { path = "../ruff_python_literal" }
ruff_python_semantic = { path = "../ruff_python_semantic" }
ruff_python_stdlib = { path = "../ruff_python_stdlib" }
ruff_python_trivia = { path = "../ruff_python_trivia" }
ruff_python_parser = { path = "../ruff_python_parser" }
ruff_source_file = { path = "../ruff_source_file", features = ["serde"] }
ruff_text_size = { path = "../ruff_text_size" }
annotate-snippets = { version = "0.9.1", features = ["color"] }
anyhow = { workspace = true }
bitflags = { workspace = true }
chrono = { workspace = true }
clap = { workspace = true, features = ["derive", "string"], optional = true }
colored = { workspace = true }
dirs = { version = "5.0.0" }
fern = { version = "0.6.1" }
glob = { workspace = true }
globset = { workspace = true }
ignore = { workspace = true }
imperative = { version = "1.0.4" }
is-macro = { workspace = true }
itertools = { workspace = true }
libcst = { workspace = true }
log = { workspace = true }
memchr = { workspace = true }
natord = { version = "1.0.9" }
num-bigint = { workspace = true }
num-traits = { workspace = true }
once_cell = { workspace = true }
path-absolutize = { workspace = true, features = [
"once_cell_cache",
"use_unix_paths_on_wasm",
] }
pathdiff = { version = "0.2.1" }
pep440_rs = { version = "0.3.1", features = ["serde"] }
pyproject-toml = { version = "0.6.0" }
quick-junit = { version = "0.3.2" }
regex = { workspace = true }
result-like = { version = "0.4.6" }
rustc-hash = { workspace = true }
schemars = { workspace = true, optional = true }
semver = { version = "1.0.16" }
serde = { workspace = true }
serde_json = { workspace = true }
serde_with = { version = "3.0.0" }
similar = { workspace = true }
shellexpand = { workspace = true }
smallvec = { workspace = true }
strum = { workspace = true }
strum_macros = { workspace = true }
thiserror = { version = "1.0.43" }
toml = { workspace = true }
typed-arena = { version = "2.0.2" }
unicode-width = { version = "0.1.10" }
unicode_names2 = { version = "0.6.0", git = "https://github.com/youknowone/unicode_names2.git", rev = "4ce16aa85cbcdd9cc830410f1a72ef9a235f2fde" }
wsl = { version = "0.1.0" }
[dev-dependencies]
insta = { workspace = true }
pretty_assertions = "1.3.0"
test-case = { workspace = true }
# Disable colored output in tests
colored = { workspace = true, features = ["no-color"] }
tempfile = "3.6.0"
[features]
default = []
schemars = ["dep:schemars"]
# Enables the UnreachableCode rule
unreachable-code = []

View File

@@ -1,21 +0,0 @@
#import os
# from foo import junk
#a = 3
a = 4
#foo(1, 2, 3)
def foo(x, y, z):
content = 1 # print('hello')
print(x, y, z)
# This is a real comment.
# # This is a (nested) comment.
#return True
return False
#import os # noqa: ERA001
class A():
pass
# b = c

View File

@@ -1,16 +0,0 @@
# ok
with open("/abc/tmp", "w") as f:
f.write("def")
with open("/tmp/abc", "w") as f:
f.write("def")
with open("/var/tmp/123", "w") as f:
f.write("def")
with open("/dev/shm/unit/test", "w") as f:
f.write("def")
# not ok by config
with open("/foo/bar", "w") as f:
f.write("def")

View File

@@ -1,45 +0,0 @@
"""
Should emit:
B009 - Line 19, 20, 21, 22, 23, 24
B010 - Line 40, 41, 42, 43, 44, 45
"""
# Valid getattr usage
getattr(foo, bar)
getattr(foo, "bar", None)
getattr(foo, "bar{foo}".format(foo="a"), None)
getattr(foo, "bar{foo}".format(foo="a"))
getattr(foo, bar, None)
getattr(foo, "123abc")
getattr(foo, r"123\abc")
getattr(foo, "except")
getattr(foo, "__123abc")
# Invalid usage
getattr(foo, "bar")
getattr(foo, "_123abc")
getattr(foo, "__123abc__")
getattr(foo, "abc123")
getattr(foo, r"abc123")
_ = lambda x: getattr(x, "bar")
if getattr(x, "bar"):
pass
# Valid setattr usage
setattr(foo, bar, None)
setattr(foo, "bar{foo}".format(foo="a"), None)
setattr(foo, "123abc", None)
setattr(foo, "__123abc", None)
setattr(foo, r"123\abc", None)
setattr(foo, "except", None)
_ = lambda x: setattr(x, "bar", 1)
if setattr(x, "bar", 1):
pass
# Invalid usage
setattr(foo, "bar", None)
setattr(foo, "_123abc", None)
setattr(foo, "__123abc__", None)
setattr(foo, "abc123", None)
setattr(foo, r"abc123", None)
setattr(foo.bar, r"baz", None)

View File

@@ -1,8 +0,0 @@
try:
pass
except (ValueError,):
pass
except AttributeError:
pass
except (ImportError, TypeError):
pass

View File

@@ -1,18 +0,0 @@
dict((x, x) for x in range(3))
dict(
(x, x) for x in range(3)
)
dict(((x, x) for x in range(3)), z=3)
y = f'{dict((x, x) for x in range(3))}'
print(f'Hello {dict((x, x) for x in range(3))} World')
print(f"Hello {dict((x, x) for x in 'abc')} World")
print(f'Hello {dict((x, x) for x in "abc")} World')
print(f'Hello {dict((x,x) for x in "abc")} World')
f'{dict((x, x) for x in range(3)) | dict((x, x) for x in range(3))}'
f'{ dict((x, x) for x in range(3)) | dict((x, x) for x in range(3)) }'
def f(x):
return x
print(f'Hello {dict((x,f(x)) for x in "abc")} World')

View File

@@ -1,13 +0,0 @@
dict([(i, i) for i in range(3)])
dict([(i, i) for i in range(3)], z=4)
def f(x):
return x
f'{dict([(s,s) for s in "ab"])}'
f"{dict([(s,s) for s in 'ab'])}"
f"{dict([(s, s) for s in 'ab'])}"
f"{dict([(s,f(s)) for s in 'ab'])}"
f'{dict([(s,s) for s in "ab"]) | dict([(s,s) for s in "ab"])}'
f'{ dict([(s,s) for s in "ab"]) | dict([(s,s) for s in "ab"]) }'

View File

@@ -1,15 +0,0 @@
x = [2, 3, 1]
list(x)
list(sorted(x))
reversed(sorted(x))
reversed(sorted(x, key=lambda e: e))
reversed(sorted(x, reverse=True))
reversed(sorted(x, key=lambda e: e, reverse=True))
reversed(sorted(x, reverse=True, key=lambda e: e))
reversed(sorted(x, reverse=False))
def reversed(*args, **kwargs):
return None
reversed(sorted(x, reverse=True))

View File

@@ -1,19 +0,0 @@
x = [1, 2, 3]
y = [("a", 1), ("b", 2), ("c", 3)]
z = [(1,), (2,), (3,)]
d = {"a": 1, "b": 2, "c": 3}
[i for i in x]
{i for i in x}
{k: v for k, v in y}
{k: v for k, v in d.items()}
[i for i, in z]
[i for i, j in y]
[i for i in x if i > 1]
[i for i in x for j in x]
{v: k for k, v in y}
{k.foo: k for k in y}
{k["foo"]: k for k in y}
{k: v if v else None for k, v in y}

View File

@@ -1,39 +0,0 @@
# Errors.
nums = [1, 2, 3]
map(lambda x: x + 1, nums)
map(lambda x: str(x), nums)
list(map(lambda x: x * 2, nums))
set(map(lambda x: x % 2 == 0, nums))
dict(map(lambda v: (v, v**2), nums))
map(lambda: "const", nums)
map(lambda _: 3.0, nums)
_ = "".join(map(lambda x: x in nums and "1" or "0", range(123)))
all(map(lambda v: isinstance(v, dict), nums))
filter(func, map(lambda v: v, nums))
# When inside f-string, then the fix should be surrounded by whitespace
_ = f"{set(map(lambda x: x % 2 == 0, nums))}"
_ = f"{dict(map(lambda v: (v, v**2), nums))}"
# Error, but unfixable.
# For simple expressions, this could be: `(x if x else 1 for x in nums)`.
# For more complex expressions, this would differ: `(x + 2 if x else 3 for x in nums)`.
map(lambda x=1: x, nums)
# False negatives.
map(lambda x=2, y=1: x + y, nums, nums)
set(map(lambda x, y: x, nums, nums))
def func(arg1: int, arg2: int = 4):
return 2 * arg1 + arg2
# Non-error: `func` is not a lambda.
list(map(func, nums))
# False positive: need to preserve the late-binding of `x` in the inner lambda.
map(lambda x: lambda: x, range(4))
# Error: the `x` is overridden by the inner lambda.
map(lambda x: lambda x: x, range(4))

View File

@@ -1,22 +0,0 @@
import math # not checked
import altair # unconventional
import matplotlib.pyplot # unconventional
import numpy # unconventional
import pandas # unconventional
import seaborn # unconventional
import tkinter # unconventional
import altair as altr # unconventional
import matplotlib.pyplot as plot # unconventional
import numpy as nmp # unconventional
import pandas as pdas # unconventional
import seaborn as sbrn # unconventional
import tkinter as tkr # unconventional
import altair as alt # conventional
import matplotlib.pyplot as plt # conventional
import numpy as np # conventional
import pandas as pd # conventional
import seaborn as sns # conventional
import tkinter as tk # conventional

View File

@@ -1,5 +0,0 @@
import logging
name = "world"
logging.info(f"Hello {name}")
logging.log(logging.INFO, f"Hello {name}")

View File

@@ -1,18 +0,0 @@
def bar():
... # OK
def foo():
"""foo""" # OK
def buzz():
print("buzz") # OK, not in stub file
def foo2():
123 # OK, not in a stub file
def bizz():
x = 123 # OK, not in a stub file

View File

@@ -1,12 +0,0 @@
def bar(): ... # OK
def foo():
"""foo""" # OK, strings are handled by another rule
def buzz():
print("buzz") # ERROR PYI010
def foo2():
123 # ERROR PYI010
def bizz():
x = 123 # ERROR PYI010

View File

@@ -1,35 +0,0 @@
# Shouldn't affect non-union field types.
field1: str
# Should emit for duplicate field types.
field2: str | str # PYI016: Duplicate union member `str`
# Should emit for union types in arguments.
def func1(arg1: int | int): # PYI016: Duplicate union member `int`
print(arg1)
# Should emit for unions in return types.
def func2() -> str | str: # PYI016: Duplicate union member `str`
return "my string"
# Should emit in longer unions, even if not directly adjacent.
field3: str | str | int # PYI016: Duplicate union member `str`
field4: int | int | str # PYI016: Duplicate union member `int`
field5: str | int | str # PYI016: Duplicate union member `str`
field6: int | bool | str | int # PYI016: Duplicate union member `int`
# Shouldn't emit for non-type unions.
field7 = str | str
# Should emit for strangely-bracketed unions.
field8: int | (str | int) # PYI016: Duplicate union member `int`
# Should handle user brackets when fixing.
field9: int | (int | str) # PYI016: Duplicate union member `int`
field10: (str | int) | str # PYI016: Duplicate union member `str`
# Should emit for nested unions.
field11: dict[int | int, str]

View File

@@ -1,9 +0,0 @@
import collections
person: collections.namedtuple # OK
from collections import namedtuple
person: namedtuple # OK
person = namedtuple("Person", ["name", "age"]) # OK

View File

@@ -1,24 +0,0 @@
from typing import Literal
# Shouldn't emit for any cases in the non-stub file for compatibility with flake8-pyi.
# Note that this rule could be applied here in the future.
field1: Literal[1] # OK
field2: Literal[1] | Literal[2] # OK
def func1(arg1: Literal[1] | Literal[2]): # OK
print(arg1)
def func2() -> Literal[1] | Literal[2]: # OK
return "my Literal[1]ing"
field3: Literal[1] | Literal[2] | str # OK
field4: str | Literal[1] | Literal[2] # OK
field5: Literal[1] | str | Literal[2] # OK
field6: Literal[1] | bool | Literal[2] | str # OK
field7 = Literal[1] | Literal[2] # OK
field8: Literal[1] | (Literal[2] | str) # OK
field9: Literal[1] | (Literal[2] | str) # OK
field10: (Literal[1] | str) | Literal[2] # OK
field11: dict[Literal[1] | Literal[2], str] # OK

View File

@@ -1,24 +0,0 @@
from typing import Any
import typing
class Bad:
def __eq__(self, other: Any) -> bool: ... # Fine because not a stub file
def __ne__(self, other: typing.Any) -> typing.Any: ... # Fine because not a stub file
class Good:
def __eq__(self, other: object) -> bool: ...
def __ne__(self, obj: object) -> int: ...
class WeirdButFine:
def __eq__(self, other: Any, strange_extra_arg: list[str]) -> Any: ...
def __ne__(self, *, kw_only_other: Any) -> bool: ...
class Unannotated:
def __eq__(self) -> Any: ...
def __ne__(self) -> bool: ...

View File

@@ -1,24 +0,0 @@
import typing
from collections.abc import Mapping
from typing import (
Annotated,
TypeAlias,
Union,
Literal,
)
just_literals_pipe_union: TypeAlias = (
Literal[True] | Literal["idk"]
) # not PYI042 (not a stubfile)
PublicAliasT: TypeAlias = str | int
PublicAliasT2: TypeAlias = Union[str, bytes]
_ABCDEFGHIJKLMNOPQRST: TypeAlias = typing.Any
_PrivateAliasS: TypeAlias = Literal["I", "guess", "this", "is", "okay"]
_PrivateAliasS2: TypeAlias = Annotated[str, "also okay"]
snake_case_alias1: TypeAlias = str | int # not PYI042 (not a stubfile)
_snake_case_alias2: TypeAlias = Literal["whatever"] # not PYI042 (not a stubfile)
Snake_case_alias: TypeAlias = int | float # not PYI042 (not a stubfile)
# check that this edge case doesn't crash
_: TypeAlias = str | int

View File

@@ -1,23 +0,0 @@
import typing
from collections.abc import Mapping
from typing import (
Annotated,
TypeAlias,
Union,
Literal,
)
_PrivateAliasT: TypeAlias = str | int # not PYI043 (not a stubfile)
_PrivateAliasT2: TypeAlias = typing.Any # not PYI043 (not a stubfile)
_PrivateAliasT3: TypeAlias = Literal[
"not", "a", "chance"
] # not PYI043 (not a stubfile)
just_literals_pipe_union: TypeAlias = Literal[True] | Literal["idk"]
PublicAliasT: TypeAlias = str | int
PublicAliasT2: TypeAlias = Union[str, bytes]
_ABCDEFGHIJKLMNOPQRST: TypeAlias = typing.Any
_PrivateAliasS: TypeAlias = Literal["I", "guess", "this", "is", "okay"]
_PrivateAliasS2: TypeAlias = Annotated[str, "also okay"]
# check that this edge case doesn't crash
_: TypeAlias = str | int

View File

@@ -1,19 +0,0 @@
def bar(): # OK
...
def oof(): # OK, docstrings are handled by another rule
"""oof"""
print("foo")
def foo(): # Ok not in Stub file
"""foo"""
print("foo")
print("foo")
def buzz(): # Ok not in Stub file
print("fizz")
print("buzz")
print("test")

View File

@@ -1,20 +0,0 @@
def bar():
... # OK
def oof(): # OK, docstrings are handled by another rule
"""oof"""
print("foo")
def foo(): # ERROR PYI048
"""foo"""
print("foo")
print("foo")
def buzz(): # ERROR PYI048
print("fizz")
print("buzz")
print("test")

View File

@@ -1,45 +0,0 @@
import pytest
def test_ok():
assert something
assert something or something_else
assert something or something_else and something_third
assert not (something and something_else)
assert something, "something message"
assert something or something_else and something_third, "another message"
def test_error():
assert something and something_else
assert something and something_else and something_third
assert something and not something_else
assert something and (something_else or something_third)
assert not something and something_else
assert not (something or something_else)
assert not (something or something_else or something_third)
assert something and something_else == """error
message
"""
assert (
something
and something_else
== """error
message
"""
)
# recursive case
assert not (a or not (b or c))
assert not (a or not (b and c))
# detected, but no autofix for messages
assert something and something_else, "error message"
assert not (something or something_else and something_third), "with message"
# detected, but no autofix for mixed conditions (e.g. `a or b and c`)
assert not (something or something_else and something_third)
assert something # OK
assert something and something_else # Error
assert something and something_else and something_third # Error

View File

@@ -1,54 +0,0 @@
try:
y = 6 + "7"
except TypeError:
# RSE102
raise ValueError()
try:
x = 1 / 0
except ZeroDivisionError:
raise
# RSE102
raise TypeError()
# RSE102
raise TypeError ()
# RSE102
raise TypeError \
()
# RSE102
raise TypeError(
)
# RSE102
raise TypeError(
# Hello, world!
)
# OK
raise AssertionError
# OK
raise AttributeError("test message")
def return_error():
return ValueError("Something")
# OK
raise return_error()
class Class:
@staticmethod
def error():
return ValueError("Something")
# OK
raise Class.error()

View File

@@ -1,32 +0,0 @@
key in obj.keys() # SIM118
key not in obj.keys() # SIM118
foo["bar"] in obj.keys() # SIM118
foo["bar"] not in obj.keys() # SIM118
foo['bar'] in obj.keys() # SIM118
foo['bar'] not in obj.keys() # SIM118
foo() in obj.keys() # SIM118
foo() not in obj.keys() # SIM118
for key in obj.keys(): # SIM118
pass
for key in list(obj.keys()):
if some_property(key):
del obj[key]
[k for k in obj.keys()] # SIM118
{k for k in obj.keys()} # SIM118
{k: k for k in obj.keys()} # SIM118
(k for k in obj.keys()) # SIM118
key in (obj or {}).keys() # SIM118

View File

@@ -1,15 +0,0 @@
a = True if b else False # SIM210
a = True if b != c else False # SIM210
a = True if b + c else False # SIM210
a = False if b else True # OK
def f():
# OK
def bool():
return False
a = True if b else False

View File

@@ -1,15 +0,0 @@
import numpy as np
np.round_(np.random.rand(5, 5), 2)
np.product(np.random.rand(5, 5))
np.cumproduct(np.random.rand(5, 5))
np.sometrue(np.random.rand(5, 5))
np.alltrue(np.random.rand(5, 5))
from numpy import round_, product, cumproduct, sometrue, alltrue
round_(np.random.rand(5, 5), 2)
product(np.random.rand(5, 5))
cumproduct(np.random.rand(5, 5))
sometrue(np.random.rand(5, 5))
alltrue(np.random.rand(5, 5))

View File

@@ -1,40 +0,0 @@
import collections
from collections import namedtuple
from typing import TypeVar
from typing import NewType
from typing import NamedTuple, TypedDict
GLOBAL: str = "foo"
def assign():
global GLOBAL
GLOBAL = "bar"
lower = 0
Camel = 0
CONSTANT = 0
_ = 0
MyObj1 = collections.namedtuple("MyObj1", ["a", "b"])
MyObj2 = namedtuple("MyObj12", ["a", "b"])
T = TypeVar("T")
UserId = NewType("UserId", int)
Employee = NamedTuple('Employee', [('name', str), ('id', int)])
Point2D = TypedDict('Point2D', {'in': int, 'x-y': int})
def aug_assign(rank, world_size):
global CURRENT_PORT
CURRENT_PORT += 1
if CURRENT_PORT > MAX_PORT:
CURRENT_PORT = START_PORT
def loop_assign():
global CURRENT_PORT
for CURRENT_PORT in range(5):
pass

View File

@@ -1,28 +0,0 @@
for i in range(10):
try: # PERF203
print(f"{i}")
except:
print("error")
try:
for i in range(10):
print(f"{i}")
except:
print("error")
i = 0
while i < 10: # PERF203
try:
print(f"{i}")
except:
print("error")
i += 1
try:
i = 0
while i < 10:
print(f"{i}")
i += 1
except:
print("error")

View File

@@ -1,47 +0,0 @@
def f():
items = [1, 2, 3, 4]
result = []
for i in items:
if i % 2:
result.append(i) # PERF401
def f():
items = [1, 2, 3, 4]
result = []
for i in items:
result.append(i * i) # PERF401
def f():
items = [1, 2, 3, 4]
result = []
for i in items:
if i % 2:
result.append(i) # PERF401
elif i % 2:
result.append(i) # PERF401
else:
result.append(i) # PERF401
def f():
items = [1, 2, 3, 4]
result = []
for i in items:
result.append(i) # OK
def f():
items = [1, 2, 3, 4]
result = {}
for i in items:
result[i].append(i) # OK
def f():
items = [1, 2, 3, 4]
result = []
for i in items:
if i not in result:
result.append(i) # OK

View File

@@ -1,26 +0,0 @@
def f():
items = [1, 2, 3, 4]
result = []
for i in items:
result.append(i) # PERF402
def f():
items = [1, 2, 3, 4]
result = []
for i in items:
result.insert(0, i) # PERF402
def f():
items = [1, 2, 3, 4]
result = []
for i in items:
result.append(i * i) # OK
def f():
items = [1, 2, 3, 4]
result = {}
for i in items:
result[i].append(i * i) # OK

View File

@@ -1,11 +0,0 @@
a = """ß9💣24A67ß9💣24A67ß9💣24A67ß9💣24A67ß9💣24A67ß9💣24A67ß9💣24A67ß9💣24A6"""
a = """ß9💣24A67ß9💣24A67ß9💣24A67ß9💣24A67ß9💣24A67ß9💣24A67ß9💣24A67ß9💣24A6"""
b = """ß9💣24A67ß9💣24A67ß9💣24A67ß9💣24A67ß9💣24A67ß9💣24A67ß9💣24A67ß9💣24A6"""
b = """ß9💣24A67ß9💣24A67ß9💣24A67ß9💣24A67ß9💣24A67ß9💣24A67ß9💣24A67ß9💣24A6"""
c = """24A67ß9💣24A67ß9💣24A67ß9💣24A67ß9💣24A67ß9💣24A67ß9💣24A67ß9💣24A6"""
c = """24A67ß9💣24A67ß9💣24A67ß9💣24A67ß9💣24A67ß9💣24A67ß9💣24A67ß9💣24A6"""
d = """💣24A67ß9💣24A67ß9💣24A67ß9💣24A67ß9💣24A67ß9💣24A67ß9💣24A6"""
d = """💣24A67ß9💣24A67ß9💣24A67ß9💣24A67ß9💣24A67ß9💣24A67ß9💣24A6"""

View File

@@ -1,60 +0,0 @@
#: E721
if type(res) == type(42):
pass
#: E721
if type(res) != type(""):
pass
#: E721
import types
if res == types.IntType:
pass
#: E721
import types
if type(res) is not types.ListType:
pass
#: E721
assert type(res) == type(False)
#: E721
assert type(res) == type([])
#: E721
assert type(res) == type(())
#: E721
assert type(res) == type((0,))
#: E721
assert type(res) == type((0))
#: E721
assert type(res) != type((1,))
#: E721
assert type(res) is type((1,))
#: E721
assert type(res) is not type((1,))
#: E211 E721
assert type(res) == type(
[
2,
]
)
#: E201 E201 E202 E721
assert type(res) == type(())
#: E201 E202 E721
assert type(res) == type((0,))
#: Okay
import types
if isinstance(res, int):
pass
if isinstance(res, str):
pass
if isinstance(res, types.MethodType):
pass
if type(a) != type(b) or type(a) == type(ccc):
pass
assert type(res) == type(None)
types = StrEnum
if x == types.X:
pass

View File

@@ -1,45 +0,0 @@
# Errors.
foo == foo
foo != foo
foo > foo
foo >= foo
foo < foo
foo <= foo
foo is foo
foo is not foo
foo in foo
foo not in foo
# Non-errors.
"foo" == "foo" # This is flagged by `comparison-of-constant` instead.
foo == "foo"
foo == bar
foo != bar
foo > bar
foo >= bar
foo < bar
foo <= bar
foo is bar
foo is not bar
foo in bar
foo not in bar

View File

@@ -1,34 +0,0 @@
# Errors.
foo == "a" or foo == "b"
foo != "a" and foo != "b"
foo == "a" or foo == "b" or foo == "c"
foo != "a" and foo != "b" and foo != "c"
foo == a or foo == "b" or foo == 3 # Mixed types.
# False negatives (the current implementation doesn't support Yoda conditions).
"a" == foo or "b" == foo or "c" == foo
"a" != foo and "b" != foo and "c" != foo
"a" == foo or foo == "b" or "c" == foo
# OK
foo == "a" and foo == "b" and foo == "c" # `and` mixed with `==`.
foo != "a" or foo != "b" or foo != "c" # `or` mixed with `!=`.
foo == a or foo == b() or foo == c # Call expression.
foo != a or foo() != b or foo != c # Call expression.
foo in {"a", "b", "c"} # Uses membership test already.
foo not in {"a", "b", "c"} # Uses membership test already.
foo == "a" # Single comparison.
foo != "a" # Single comparison.

View File

@@ -1,15 +0,0 @@
type("")
type(b"")
type(0)
type(0.0)
type(0j)
# OK
type(arg)(" ")
# OK
y = x.dtype.type(0.0)
# OK
type = lambda *args, **kwargs: None
type("")

View File

@@ -1,61 +0,0 @@
import typing
from typing import Optional
from typing import Union
def f(x: Optional[str]) -> None:
...
def f(x: typing.Optional[str]) -> None:
...
def f(x: Union[str, int, Union[float, bytes]]) -> None:
...
def f(x: typing.Union[str, int]) -> None:
...
def f(x: typing.Union[(str, int)]) -> None:
...
def f(x: typing.Union[(str, int), float]) -> None:
...
def f(x: typing.Union[(int,)]) -> None:
...
def f(x: typing.Union[()]) -> None:
...
def f(x: "Union[str, int, Union[float, bytes]]") -> None:
...
def f(x: "typing.Union[str, int]") -> None:
...
def f(x: Union["str", int]) -> None:
...
def f(x: Union[("str", "int"), float]) -> None:
...
def f() -> None:
x: Optional[str]
x = Optional[str]
x = Union[str, int]
x = Union["str", "int"]
x: Union[str, int]
x: Union["str", "int"]

View File

@@ -1,12 +0,0 @@
import subprocess
import subprocess as somename
from subprocess import run
from subprocess import run as anothername
subprocess.run(["foo"], universal_newlines=True, check=True)
somename.run(["foo"], universal_newlines=True)
run(["foo"], universal_newlines=True, check=False)
anothername(["foo"], universal_newlines=True)
subprocess.run(["foo"], check=True)

View File

@@ -1,42 +0,0 @@
from subprocess import run
import subprocess
output = run(["foo"], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
output = subprocess.run(["foo"], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
output = subprocess.run(stdout=subprocess.PIPE, args=["foo"], stderr=subprocess.PIPE)
output = subprocess.run(
["foo"], stdout=subprocess.PIPE, check=True, stderr=subprocess.PIPE
)
output = subprocess.run(
["foo"], stderr=subprocess.PIPE, check=True, stdout=subprocess.PIPE
)
output = subprocess.run(
["foo"],
stdout=subprocess.PIPE,
check=True,
stderr=subprocess.PIPE,
text=True,
encoding="utf-8",
close_fds=True,
)
if output:
output = subprocess.run(
["foo"],
stdout=subprocess.PIPE,
check=True,
stderr=subprocess.PIPE,
text=True,
encoding="utf-8",
)
# Examples that should NOT trigger the rule
from foo import PIPE
subprocess.run(["foo"], stdout=PIPE, stderr=PIPE)
run(["foo"], stdout=None, stderr=PIPE)

View File

@@ -1,34 +0,0 @@
# Invalid calls; errors expected.
"{0}" "{1}" "{2}".format(1, 2, 3)
"a {3} complicated {1} string with {0} {2}".format(
"first", "second", "third", "fourth"
)
'{0}'.format(1)
'{0:x}'.format(30)
x = '{0}'.format(1)
'''{0}\n{1}\n'''.format(1, 2)
x = "foo {0}" \
"bar {1}".format(1, 2)
("{0}").format(1)
"\N{snowman} {0}".format(1)
print(
'foo{0}'
'bar{1}'.format(1, 2)
)
print(
'foo{0}' # ohai\n"
'bar{1}'.format(1, 2)
)
'{' '0}'.format(1)

View File

@@ -1,17 +0,0 @@
# Valid calls; no errors expected.
'{}'.format(1)
x = ('{0} {1}',)
'{0} {0}'.format(1)
'{0:<{1}}'.format(1, 4)
f"{0}".format(a)
f"{0}".format(1)
print(f"{0}".format(1))
''.format(1)

View File

@@ -1,28 +0,0 @@
# These SHOULD change
args = list(range(10))
kwargs = {x: x for x in range(10)}
"{0}".format(*args)
"{0}".format(**kwargs)
"{0}_{1}".format(*args)
"{0}_{1}".format(1, *args)
"{1}_{0}".format(*args)
"{1}_{0}".format(1, *args)
"{0}_{1}".format(1, 2, *args)
"{0}_{1}".format(*args, 1, 2)
"{0}_{1}_{2}".format(1, **kwargs)
"{0}_{1}_{2}".format(1, 2, **kwargs)
"{0}_{1}_{2}".format(1, 2, 3, **kwargs)
"{0}_{1}_{2}".format(1, 2, 3, *args, **kwargs)

View File

@@ -1,59 +0,0 @@
# UP035
from collections import Mapping
from collections import Mapping as MAP
from collections import Mapping, Sequence
from collections import Counter, Mapping
from collections import (Counter, Mapping)
from collections import (Counter,
Mapping)
from collections import Counter, \
Mapping
from collections import Counter, Mapping, Sequence
from collections import Mapping as mapping, Counter
if True:
from collections import Mapping, Counter
if True:
if True:
pass
from collections import Mapping, Counter
if True: from collections import Mapping
import os
from collections import Counter, Mapping
import sys
if True:
from collections import (
Mapping,
Callable,
Bad,
Good,
)
from typing import Callable, Match, Pattern, List, OrderedDict, AbstractSet, ContextManager
if True: from collections import (
Mapping, Counter)
# OK
from a import b
# Ok: `typing_extensions` contains backported improvements.
from typing_extensions import SupportsIndex
# Ok: `typing_extensions` contains backported improvements.
from typing_extensions import NamedTuple
# Ok: `typing_extensions` supports `frozen_default` (backported from 3.12).
from typing_extensions import dataclass_transform

View File

@@ -1,110 +0,0 @@
"""A mirror of UP037_1.py, with `from __future__ import annotations`."""
from __future__ import annotations
from typing import (
Annotated,
Callable,
List,
Literal,
NamedTuple,
Tuple,
TypeVar,
TypedDict,
cast,
)
from mypy_extensions import Arg, DefaultArg, DefaultNamedArg, NamedArg, VarArg
def foo(var: "MyClass") -> "MyClass":
x: "MyClass"
def foo(*, inplace: "bool"):
pass
def foo(*args: "str", **kwargs: "int"):
pass
x: Tuple["MyClass"]
x: Callable[["MyClass"], None]
class Foo(NamedTuple):
x: "MyClass"
class D(TypedDict):
E: TypedDict("E", foo="int", total=False)
class D(TypedDict):
E: TypedDict("E", {"foo": "int"})
x: Annotated["str", "metadata"]
x: Arg("str", "name")
x: DefaultArg("str", "name")
x: NamedArg("str", "name")
x: DefaultNamedArg("str", "name")
x: DefaultNamedArg("str", name="name")
x: VarArg("str")
x: List[List[List["MyClass"]]]
x: NamedTuple("X", [("foo", "int"), ("bar", "str")])
x: NamedTuple("X", fields=[("foo", "int"), ("bar", "str")])
x: NamedTuple(typename="X", fields=[("foo", "int")])
X: MyCallable("X")
# OK
class D(TypedDict):
E: TypedDict("E")
x: Annotated[()]
x: DefaultNamedArg(name="name", quox="str")
x: DefaultNamedArg(name="name")
x: NamedTuple("X", [("foo",), ("bar",)])
x: NamedTuple("X", ["foo", "bar"])
x: NamedTuple()
x: Literal["foo", "bar"]
x = cast(x, "str")
def foo(x, *args, **kwargs):
...
def foo(*, inplace):
...
x: Annotated[1:2] = ...
x = TypeVar("x", "str", "int")
x = cast("str", x)
X = List["MyClass"]

View File

@@ -1,108 +0,0 @@
"""A mirror of UP037_0.py, without `from __future__ import annotations`."""
from typing import (
Annotated,
Callable,
List,
Literal,
NamedTuple,
Tuple,
TypeVar,
TypedDict,
cast,
)
from mypy_extensions import Arg, DefaultArg, DefaultNamedArg, NamedArg, VarArg
def foo(var: "MyClass") -> "MyClass":
x: "MyClass"
def foo(*, inplace: "bool"):
pass
def foo(*args: "str", **kwargs: "int"):
pass
x: Tuple["MyClass"]
x: Callable[["MyClass"], None]
class Foo(NamedTuple):
x: "MyClass"
class D(TypedDict):
E: TypedDict("E", foo="int", total=False)
class D(TypedDict):
E: TypedDict("E", {"foo": "int"})
x: Annotated["str", "metadata"]
x: Arg("str", "name")
x: DefaultArg("str", "name")
x: NamedArg("str", "name")
x: DefaultNamedArg("str", "name")
x: DefaultNamedArg("str", name="name")
x: VarArg("str")
x: List[List[List["MyClass"]]]
x: NamedTuple("X", [("foo", "int"), ("bar", "str")])
x: NamedTuple("X", fields=[("foo", "int"), ("bar", "str")])
x: NamedTuple(typename="X", fields=[("foo", "int")])
X: MyCallable("X")
# OK
class D(TypedDict):
E: TypedDict("E")
x: Annotated[()]
x: DefaultNamedArg(name="name", quox="str")
x: DefaultNamedArg(name="name")
x: NamedTuple("X", [("foo",), ("bar",)])
x: NamedTuple("X", ["foo", "bar"])
x: NamedTuple()
x: Literal["foo", "bar"]
x = cast(x, "str")
def foo(x, *args, **kwargs):
...
def foo(*, inplace):
...
x: Annotated[1:2] = ...
x = TypeVar("x", "str", "int")
x = cast("str", x)
X = List["MyClass"]

View File

@@ -1,96 +0,0 @@
# project
An example multi-package Python project used to test setting resolution and other complex
behaviors.
## Expected behavior
Running from the repo root should pick up and enforce the appropriate settings for each package:
```console
∴ cargo run -p ruff_cli -- check crates/ruff/resources/test/project/
crates/ruff/resources/test/project/examples/.dotfiles/script.py:1:1: I001 [*] Import block is un-sorted or un-formatted
crates/ruff/resources/test/project/examples/.dotfiles/script.py:1:8: F401 [*] `numpy` imported but unused
crates/ruff/resources/test/project/examples/.dotfiles/script.py:2:17: F401 [*] `app.app_file` imported but unused
crates/ruff/resources/test/project/examples/docs/docs/file.py:1:1: I001 [*] Import block is un-sorted or un-formatted
crates/ruff/resources/test/project/examples/docs/docs/file.py:8:5: F841 [*] Local variable `x` is assigned to but never used
crates/ruff/resources/test/project/project/file.py:1:8: F401 [*] `os` imported but unused
crates/ruff/resources/test/project/project/import_file.py:1:1: I001 [*] Import block is un-sorted or un-formatted
Found 7 errors.
[*] 7 potentially fixable with the --fix option.
```
Running from the project directory itself should exhibit the same behavior:
```console
∴ (cd crates/ruff/resources/test/project/ && cargo run -p ruff_cli -- check .)
examples/.dotfiles/script.py:1:1: I001 [*] Import block is un-sorted or un-formatted
examples/.dotfiles/script.py:1:8: F401 [*] `numpy` imported but unused
examples/.dotfiles/script.py:2:17: F401 [*] `app.app_file` imported but unused
examples/docs/docs/file.py:1:1: I001 [*] Import block is un-sorted or un-formatted
examples/docs/docs/file.py:8:5: F841 [*] Local variable `x` is assigned to but never used
project/file.py:1:8: F401 [*] `os` imported but unused
project/import_file.py:1:1: I001 [*] Import block is un-sorted or un-formatted
Found 7 errors.
[*] 7 potentially fixable with the --fix option.
```
Running from the sub-package directory should exhibit the same behavior, but omit the top-level
files:
```console
∴ (cd crates/ruff/resources/test/project/examples/docs && cargo run -p ruff_cli -- check .)
docs/file.py:1:1: I001 [*] Import block is un-sorted or un-formatted
docs/file.py:8:5: F841 [*] Local variable `x` is assigned to but never used
Found 2 errors.
[*] 2 potentially fixable with the --fix option.
```
`--config` should force Ruff to use the specified `pyproject.toml` for all files, and resolve
file paths from the current working directory:
```console
∴ (cargo run -p ruff_cli -- check --config=crates/ruff/resources/test/project/pyproject.toml crates/ruff/resources/test/project/)
crates/ruff/resources/test/project/examples/.dotfiles/script.py:1:8: F401 [*] `numpy` imported but unused
crates/ruff/resources/test/project/examples/.dotfiles/script.py:2:17: F401 [*] `app.app_file` imported but unused
crates/ruff/resources/test/project/examples/docs/docs/concepts/file.py:1:8: F401 [*] `os` imported but unused
crates/ruff/resources/test/project/examples/docs/docs/file.py:1:1: I001 [*] Import block is un-sorted or un-formatted
crates/ruff/resources/test/project/examples/docs/docs/file.py:1:8: F401 [*] `os` imported but unused
crates/ruff/resources/test/project/examples/docs/docs/file.py:3:8: F401 [*] `numpy` imported but unused
crates/ruff/resources/test/project/examples/docs/docs/file.py:4:27: F401 [*] `docs.concepts.file` imported but unused
crates/ruff/resources/test/project/examples/excluded/script.py:1:8: F401 [*] `os` imported but unused
crates/ruff/resources/test/project/project/file.py:1:8: F401 [*] `os` imported but unused
Found 9 errors.
[*] 9 potentially fixable with the --fix option.
```
Running from a parent directory should "ignore" the `exclude` (hence, `concepts/file.py` gets
included in the output):
```console
∴ (cd crates/ruff/resources/test/project/examples && cargo run -p ruff_cli -- check --config=docs/ruff.toml .)
docs/docs/concepts/file.py:5:5: F841 [*] Local variable `x` is assigned to but never used
docs/docs/file.py:1:1: I001 [*] Import block is un-sorted or un-formatted
docs/docs/file.py:8:5: F841 [*] Local variable `x` is assigned to but never used
excluded/script.py:5:5: F841 [*] Local variable `x` is assigned to but never used
Found 4 errors.
[*] 4 potentially fixable with the --fix option.
```
Passing an excluded directory directly should report errors in the contained files:
```console
∴ cargo run -p ruff_cli -- check crates/ruff/resources/test/project/examples/excluded/
crates/ruff/resources/test/project/examples/excluded/script.py:1:8: F401 [*] `os` imported but unused
Found 1 error.
[*] 1 potentially fixable with the --fix option.
```
Unless we `--force-exclude`:
```console
∴ cargo run -p ruff_cli -- check crates/ruff/resources/test/project/examples/excluded/ --force-exclude
warning: No Python files found under the given path(s)
∴ cargo run -p ruff_cli -- check crates/ruff/resources/test/project/examples/excluded/script.py --force-exclude
warning: No Python files found under the given path(s)
```

View File

@@ -1,215 +0,0 @@
//! Interface for editing code snippets. These functions take statements or expressions as input,
//! and return the modified code snippet as output.
use anyhow::{bail, Result};
use libcst_native::{
Codegen, CodegenState, ImportNames, ParenthesizableWhitespace, SmallStatement, Statement,
};
use ruff_python_ast::{Ranged, Stmt};
use ruff_python_codegen::Stylist;
use ruff_source_file::Locator;
use crate::cst::helpers::compose_module_path;
use crate::cst::matchers::match_statement;
/// Glue code to make libcst codegen work with ruff's Stylist
pub(crate) trait CodegenStylist<'a>: Codegen<'a> {
fn codegen_stylist(&self, stylist: &'a Stylist) -> String;
}
impl<'a, T: Codegen<'a>> CodegenStylist<'a> for T {
fn codegen_stylist(&self, stylist: &'a Stylist) -> String {
let mut state = CodegenState {
default_newline: stylist.line_ending().as_str(),
default_indent: stylist.indentation(),
..Default::default()
};
self.codegen(&mut state);
state.to_string()
}
}
/// Given an import statement, remove any imports that are specified in the `imports` iterator.
///
/// Returns `Ok(None)` if the statement is empty after removing the imports.
pub(crate) fn remove_imports<'a>(
imports: impl Iterator<Item = &'a str>,
stmt: &Stmt,
locator: &Locator,
stylist: &Stylist,
) -> Result<Option<String>> {
let module_text = locator.slice(stmt.range());
let mut tree = match_statement(module_text)?;
let Statement::Simple(body) = &mut tree else {
bail!("Expected Statement::Simple");
};
let (aliases, import_module) = match body.body.first_mut() {
Some(SmallStatement::Import(import_body)) => (&mut import_body.names, None),
Some(SmallStatement::ImportFrom(import_body)) => {
if let ImportNames::Aliases(names) = &mut import_body.names {
(
names,
Some((&import_body.relative, import_body.module.as_ref())),
)
} else if let ImportNames::Star(..) = &import_body.names {
// Special-case: if the import is a `from ... import *`, then we delete the
// entire statement.
let mut found_star = false;
for import in imports {
let qualified_name = match import_body.module.as_ref() {
Some(module_name) => format!("{}.*", compose_module_path(module_name)),
None => "*".to_string(),
};
if import == qualified_name {
found_star = true;
} else {
bail!("Expected \"*\" for unused import (got: \"{}\")", import);
}
}
if !found_star {
bail!("Expected \'*\' for unused import");
}
return Ok(None);
} else {
bail!("Expected: ImportNames::Aliases | ImportNames::Star");
}
}
_ => bail!("Expected: SmallStatement::ImportFrom | SmallStatement::Import"),
};
// Preserve the trailing comma (or not) from the last entry.
let trailing_comma = aliases.last().and_then(|alias| alias.comma.clone());
for import in imports {
let alias_index = aliases.iter().position(|alias| {
let qualified_name = match import_module {
Some((relative, module)) => {
let module = module.map(compose_module_path);
let member = compose_module_path(&alias.name);
let mut qualified_name = String::with_capacity(
relative.len() + module.as_ref().map_or(0, String::len) + member.len() + 1,
);
for _ in 0..relative.len() {
qualified_name.push('.');
}
if let Some(module) = module {
qualified_name.push_str(&module);
qualified_name.push('.');
}
qualified_name.push_str(&member);
qualified_name
}
None => compose_module_path(&alias.name),
};
qualified_name == import
});
if let Some(index) = alias_index {
aliases.remove(index);
}
}
// But avoid destroying any trailing comments.
if let Some(alias) = aliases.last_mut() {
let has_comment = if let Some(comma) = &alias.comma {
match &comma.whitespace_after {
ParenthesizableWhitespace::SimpleWhitespace(_) => false,
ParenthesizableWhitespace::ParenthesizedWhitespace(whitespace) => {
whitespace.first_line.comment.is_some()
}
}
} else {
false
};
if !has_comment {
alias.comma = trailing_comma;
}
}
if aliases.is_empty() {
return Ok(None);
}
Ok(Some(tree.codegen_stylist(stylist)))
}
/// Given an import statement, remove any imports that are not specified in the `imports` slice.
///
/// Returns the modified import statement.
pub(crate) fn retain_imports(
imports: &[&str],
stmt: &Stmt,
locator: &Locator,
stylist: &Stylist,
) -> Result<String> {
let module_text = locator.slice(stmt.range());
let mut tree = match_statement(module_text)?;
let Statement::Simple(body) = &mut tree else {
bail!("Expected Statement::Simple");
};
let (aliases, import_module) = match body.body.first_mut() {
Some(SmallStatement::Import(import_body)) => (&mut import_body.names, None),
Some(SmallStatement::ImportFrom(import_body)) => {
if let ImportNames::Aliases(names) = &mut import_body.names {
(
names,
Some((&import_body.relative, import_body.module.as_ref())),
)
} else {
bail!("Expected: ImportNames::Aliases");
}
}
_ => bail!("Expected: SmallStatement::ImportFrom | SmallStatement::Import"),
};
// Preserve the trailing comma (or not) from the last entry.
let trailing_comma = aliases.last().and_then(|alias| alias.comma.clone());
aliases.retain(|alias| {
imports.iter().any(|import| {
let qualified_name = match import_module {
Some((relative, module)) => {
let module = module.map(compose_module_path);
let member = compose_module_path(&alias.name);
let mut qualified_name = String::with_capacity(
relative.len() + module.as_ref().map_or(0, String::len) + member.len() + 1,
);
for _ in 0..relative.len() {
qualified_name.push('.');
}
if let Some(module) = module {
qualified_name.push_str(&module);
qualified_name.push('.');
}
qualified_name.push_str(&member);
qualified_name
}
None => compose_module_path(&alias.name),
};
qualified_name == *import
})
});
// But avoid destroying any trailing comments.
if let Some(alias) = aliases.last_mut() {
let has_comment = if let Some(comma) = &alias.comma {
match &comma.whitespace_after {
ParenthesizableWhitespace::SimpleWhitespace(_) => false,
ParenthesizableWhitespace::ParenthesizedWhitespace(whitespace) => {
whitespace.first_line.comment.is_some()
}
}
} else {
false
};
if !has_comment {
alias.comma = trailing_comma;
}
}
Ok(tree.codegen_stylist(stylist))
}

View File

@@ -1,374 +0,0 @@
//! Interface for generating autofix edits from higher-level actions (e.g., "remove an argument").
use anyhow::{bail, Result};
use ruff_python_ast::{self as ast, ExceptHandler, Expr, Keyword, Ranged, Stmt};
use ruff_python_parser::{lexer, Mode};
use ruff_text_size::{TextLen, TextRange, TextSize};
use ruff_diagnostics::Edit;
use ruff_python_codegen::Stylist;
use ruff_python_index::Indexer;
use ruff_python_trivia::{has_leading_content, is_python_whitespace, PythonWhitespace};
use ruff_source_file::{Locator, NewlineWithTrailingNewline};
use crate::autofix::codemods;
/// Return the `Fix` to use when deleting a `Stmt`.
///
/// In some cases, this is as simple as deleting the `Range` of the `Stmt`
/// itself. However, there are a few exceptions:
/// - If the `Stmt` is _not_ the terminal statement in a multi-statement line,
/// we need to delete up to the start of the next statement (and avoid
/// deleting any content that precedes the statement).
/// - If the `Stmt` is the terminal statement in a multi-statement line, we need
/// to avoid deleting any content that precedes the statement.
/// - If the `Stmt` has no trailing and leading content, then it's convenient to
/// remove the entire start and end lines.
/// - If the `Stmt` is the last statement in its parent body, replace it with a
/// `pass` instead.
pub(crate) fn delete_stmt(
stmt: &Stmt,
parent: Option<&Stmt>,
locator: &Locator,
indexer: &Indexer,
) -> Edit {
if parent
.map(|parent| is_lone_child(stmt, parent))
.unwrap_or_default()
{
// If removing this node would lead to an invalid syntax tree, replace
// it with a `pass`.
Edit::range_replacement("pass".to_string(), stmt.range())
} else {
if let Some(semicolon) = trailing_semicolon(stmt.end(), locator) {
let next = next_stmt_break(semicolon, locator);
Edit::deletion(stmt.start(), next)
} else if has_leading_content(stmt.start(), locator) {
Edit::range_deletion(stmt.range())
} else if let Some(start) = indexer.preceded_by_continuations(stmt.start(), locator) {
Edit::range_deletion(TextRange::new(start, stmt.end()))
} else {
let range = locator.full_lines_range(stmt.range());
Edit::range_deletion(range)
}
}
}
/// Generate a `Fix` to remove the specified imports from an `import` statement.
pub(crate) fn remove_unused_imports<'a>(
unused_imports: impl Iterator<Item = &'a str>,
stmt: &Stmt,
parent: Option<&Stmt>,
locator: &Locator,
stylist: &Stylist,
indexer: &Indexer,
) -> Result<Edit> {
match codemods::remove_imports(unused_imports, stmt, locator, stylist)? {
None => Ok(delete_stmt(stmt, parent, locator, indexer)),
Some(content) => Ok(Edit::range_replacement(content, stmt.range())),
}
}
/// Generic function to remove arguments or keyword arguments in function
/// calls and class definitions. (For classes `args` should be considered
/// `bases`)
///
/// Supports the removal of parentheses when this is the only (kw)arg left.
/// For this behavior, set `remove_parentheses` to `true`.
pub(crate) fn remove_argument(
locator: &Locator,
call_at: TextSize,
expr_range: TextRange,
args: &[Expr],
keywords: &[Keyword],
remove_parentheses: bool,
) -> Result<Edit> {
// TODO(sbrugman): Preserve trailing comments.
let contents = locator.after(call_at);
let mut fix_start = None;
let mut fix_end = None;
let n_arguments = keywords.len() + args.len();
if n_arguments == 0 {
bail!("No arguments or keywords to remove");
}
if n_arguments == 1 {
// Case 1: there is only one argument.
let mut count = 0u32;
for (tok, range) in lexer::lex_starts_at(contents, Mode::Module, call_at).flatten() {
if tok.is_lpar() {
if count == 0 {
fix_start = Some(if remove_parentheses {
range.start()
} else {
range.start() + TextSize::from(1)
});
}
count = count.saturating_add(1);
}
if tok.is_rpar() {
count = count.saturating_sub(1);
if count == 0 {
fix_end = Some(if remove_parentheses {
range.end()
} else {
range.end() - TextSize::from(1)
});
break;
}
}
}
} else if args
.iter()
.map(Expr::start)
.chain(keywords.iter().map(Keyword::start))
.any(|location| location > expr_range.start())
{
// Case 2: argument or keyword is _not_ the last node.
let mut seen_comma = false;
for (tok, range) in lexer::lex_starts_at(contents, Mode::Module, call_at).flatten() {
if seen_comma {
if tok.is_non_logical_newline() {
// Also delete any non-logical newlines after the comma.
continue;
}
fix_end = Some(if tok.is_newline() {
range.end()
} else {
range.start()
});
break;
}
if range.start() == expr_range.start() {
fix_start = Some(range.start());
}
if fix_start.is_some() && tok.is_comma() {
seen_comma = true;
}
}
} else {
// Case 3: argument or keyword is the last node, so we have to find the last
// comma in the stmt.
for (tok, range) in lexer::lex_starts_at(contents, Mode::Module, call_at).flatten() {
if range.start() == expr_range.start() {
fix_end = Some(expr_range.end());
break;
}
if tok.is_comma() {
fix_start = Some(range.start());
}
}
}
match (fix_start, fix_end) {
(Some(start), Some(end)) => Ok(Edit::deletion(start, end)),
_ => {
bail!("No fix could be constructed")
}
}
}
/// Determine if a vector contains only one, specific element.
fn is_only<T: PartialEq>(vec: &[T], value: &T) -> bool {
vec.len() == 1 && vec[0] == *value
}
/// Determine if a child is the only statement in its body.
fn is_lone_child(child: &Stmt, parent: &Stmt) -> bool {
match parent {
Stmt::FunctionDef(ast::StmtFunctionDef { body, .. })
| Stmt::AsyncFunctionDef(ast::StmtAsyncFunctionDef { body, .. })
| Stmt::ClassDef(ast::StmtClassDef { body, .. })
| Stmt::With(ast::StmtWith { body, .. })
| Stmt::AsyncWith(ast::StmtAsyncWith { body, .. }) => {
if is_only(body, child) {
return true;
}
}
Stmt::For(ast::StmtFor { body, orelse, .. })
| Stmt::AsyncFor(ast::StmtAsyncFor { body, orelse, .. })
| Stmt::While(ast::StmtWhile { body, orelse, .. }) => {
if is_only(body, child) || is_only(orelse, child) {
return true;
}
}
Stmt::If(ast::StmtIf {
body,
elif_else_clauses,
..
}) => {
if is_only(body, child)
|| elif_else_clauses
.iter()
.any(|ast::ElifElseClause { body, .. }| is_only(body, child))
{
return true;
}
}
Stmt::Try(ast::StmtTry {
body,
handlers,
orelse,
finalbody,
range: _,
})
| Stmt::TryStar(ast::StmtTryStar {
body,
handlers,
orelse,
finalbody,
range: _,
}) => {
if is_only(body, child)
|| is_only(orelse, child)
|| is_only(finalbody, child)
|| handlers.iter().any(|handler| match handler {
ExceptHandler::ExceptHandler(ast::ExceptHandlerExceptHandler {
body, ..
}) => is_only(body, child),
})
{
return true;
}
}
Stmt::Match(ast::StmtMatch { cases, .. }) => {
if cases.iter().any(|case| is_only(&case.body, child)) {
return true;
}
}
_ => {}
}
false
}
/// Return the location of a trailing semicolon following a `Stmt`, if it's part
/// of a multi-statement line.
fn trailing_semicolon(offset: TextSize, locator: &Locator) -> Option<TextSize> {
let contents = locator.after(offset);
for line in NewlineWithTrailingNewline::from(contents) {
let trimmed = line.trim_whitespace_start();
if trimmed.starts_with(';') {
let colon_offset = line.text_len() - trimmed.text_len();
return Some(offset + line.start() + colon_offset);
}
if !trimmed.starts_with('\\') {
break;
}
}
None
}
/// Find the next valid break for a `Stmt` after a semicolon.
fn next_stmt_break(semicolon: TextSize, locator: &Locator) -> TextSize {
let start_location = semicolon + TextSize::from(1);
let contents = &locator.contents()[usize::from(start_location)..];
for line in NewlineWithTrailingNewline::from(contents) {
let trimmed = line.trim_whitespace();
// Skip past any continuations.
if trimmed.starts_with('\\') {
continue;
}
return start_location
+ if trimmed.is_empty() {
// If the line is empty, then despite the previous statement ending in a
// semicolon, we know that it's not a multi-statement line.
line.start()
} else {
// Otherwise, find the start of the next statement. (Or, anything that isn't
// whitespace.)
let relative_offset = line.find(|c: char| !is_python_whitespace(c)).unwrap();
line.start() + TextSize::try_from(relative_offset).unwrap()
};
}
locator.line_end(start_location)
}
#[cfg(test)]
mod tests {
use anyhow::Result;
use ruff_python_ast::{Ranged, Suite};
use ruff_python_parser::Parse;
use ruff_text_size::TextSize;
use ruff_source_file::Locator;
use crate::autofix::edits::{next_stmt_break, trailing_semicolon};
#[test]
fn find_semicolon() -> Result<()> {
let contents = "x = 1";
let program = Suite::parse(contents, "<filename>")?;
let stmt = program.first().unwrap();
let locator = Locator::new(contents);
assert_eq!(trailing_semicolon(stmt.end(), &locator), None);
let contents = "x = 1; y = 1";
let program = Suite::parse(contents, "<filename>")?;
let stmt = program.first().unwrap();
let locator = Locator::new(contents);
assert_eq!(
trailing_semicolon(stmt.end(), &locator),
Some(TextSize::from(5))
);
let contents = "x = 1 ; y = 1";
let program = Suite::parse(contents, "<filename>")?;
let stmt = program.first().unwrap();
let locator = Locator::new(contents);
assert_eq!(
trailing_semicolon(stmt.end(), &locator),
Some(TextSize::from(6))
);
let contents = r"
x = 1 \
; y = 1
"
.trim();
let program = Suite::parse(contents, "<filename>")?;
let stmt = program.first().unwrap();
let locator = Locator::new(contents);
assert_eq!(
trailing_semicolon(stmt.end(), &locator),
Some(TextSize::from(10))
);
Ok(())
}
#[test]
fn find_next_stmt_break() {
let contents = "x = 1; y = 1";
let locator = Locator::new(contents);
assert_eq!(
next_stmt_break(TextSize::from(4), &locator),
TextSize::from(5)
);
let contents = "x = 1 ; y = 1";
let locator = Locator::new(contents);
assert_eq!(
next_stmt_break(TextSize::from(5), &locator),
TextSize::from(6)
);
let contents = r"
x = 1 \
; y = 1
"
.trim();
let locator = Locator::new(contents);
assert_eq!(
next_stmt_break(TextSize::from(10), &locator),
TextSize::from(12)
);
}
}

View File

@@ -1,401 +0,0 @@
use std::collections::BTreeSet;
use itertools::Itertools;
use ruff_text_size::{TextLen, TextRange, TextSize};
use rustc_hash::{FxHashMap, FxHashSet};
use ruff_diagnostics::{Diagnostic, Edit, Fix, IsolationLevel};
use ruff_source_file::Locator;
use crate::autofix::source_map::SourceMap;
use crate::linter::FixTable;
use crate::registry::{AsRule, Rule};
pub(crate) mod codemods;
pub(crate) mod edits;
pub(crate) mod source_map;
pub(crate) struct FixResult {
/// The resulting source code, after applying all fixes.
pub(crate) code: String,
/// The number of fixes applied for each [`Rule`].
pub(crate) fixes: FixTable,
/// Source map for the fixed source code.
pub(crate) source_map: SourceMap,
}
/// Auto-fix errors in a file, and write the fixed source code to disk.
pub(crate) fn fix_file(diagnostics: &[Diagnostic], locator: &Locator) -> Option<FixResult> {
let mut with_fixes = diagnostics
.iter()
.filter(|diag| diag.fix.is_some())
.peekable();
if with_fixes.peek().is_none() {
None
} else {
Some(apply_fixes(with_fixes, locator))
}
}
/// Apply a series of fixes.
fn apply_fixes<'a>(
diagnostics: impl Iterator<Item = &'a Diagnostic>,
locator: &'a Locator<'a>,
) -> FixResult {
let mut output = String::with_capacity(locator.len());
let mut last_pos: Option<TextSize> = None;
let mut applied: BTreeSet<&Edit> = BTreeSet::default();
let mut isolated: FxHashSet<u32> = FxHashSet::default();
let mut fixed = FxHashMap::default();
let mut source_map = SourceMap::default();
for (rule, fix) in diagnostics
.filter_map(|diagnostic| {
diagnostic
.fix
.as_ref()
.map(|fix| (diagnostic.kind.rule(), fix))
})
.sorted_by(|(rule1, fix1), (rule2, fix2)| cmp_fix(*rule1, *rule2, fix1, fix2))
{
// If we already applied an identical fix as part of another correction, skip
// any re-application.
if fix.edits().iter().all(|edit| applied.contains(edit)) {
*fixed.entry(rule).or_default() += 1;
continue;
}
// Best-effort approach: if this fix overlaps with a fix we've already applied,
// skip it.
if last_pos.map_or(false, |last_pos| {
fix.min_start()
.map_or(false, |fix_location| last_pos >= fix_location)
}) {
continue;
}
// If this fix requires isolation, and we've already applied another fix in the
// same isolation group, skip it.
if let IsolationLevel::Group(id) = fix.isolation() {
if !isolated.insert(id) {
continue;
}
}
for edit in fix
.edits()
.iter()
.sorted_unstable_by_key(|edit| edit.start())
{
// Add all contents from `last_pos` to `fix.location`.
let slice = locator.slice(TextRange::new(last_pos.unwrap_or_default(), edit.start()));
output.push_str(slice);
// Add the start source marker for the patch.
source_map.push_start_marker(edit, output.text_len());
// Add the patch itself.
output.push_str(edit.content().unwrap_or_default());
// Add the end source marker for the added patch.
source_map.push_end_marker(edit, output.text_len());
// Track that the edit was applied.
last_pos = Some(edit.end());
applied.insert(edit);
}
*fixed.entry(rule).or_default() += 1;
}
// Add the remaining content.
let slice = locator.after(last_pos.unwrap_or_default());
output.push_str(slice);
FixResult {
code: output,
fixes: fixed,
source_map,
}
}
/// Compare two fixes.
fn cmp_fix(rule1: Rule, rule2: Rule, fix1: &Fix, fix2: &Fix) -> std::cmp::Ordering {
fix1.min_start()
.cmp(&fix2.min_start())
.then_with(|| match (&rule1, &rule2) {
// Apply `EndsInPeriod` fixes before `NewLineAfterLastParagraph` fixes.
(Rule::EndsInPeriod, Rule::NewLineAfterLastParagraph) => std::cmp::Ordering::Less,
(Rule::NewLineAfterLastParagraph, Rule::EndsInPeriod) => std::cmp::Ordering::Greater,
// Apply `IfElseBlockInsteadOfDictGet` fixes before `IfElseBlockInsteadOfIfExp` fixes.
(Rule::IfElseBlockInsteadOfDictGet, Rule::IfElseBlockInsteadOfIfExp) => {
std::cmp::Ordering::Less
}
(Rule::IfElseBlockInsteadOfIfExp, Rule::IfElseBlockInsteadOfDictGet) => {
std::cmp::Ordering::Greater
}
_ => std::cmp::Ordering::Equal,
})
}
#[cfg(test)]
mod tests {
use ruff_text_size::TextSize;
use ruff_diagnostics::Diagnostic;
use ruff_diagnostics::Edit;
use ruff_diagnostics::Fix;
use ruff_source_file::Locator;
use crate::autofix::source_map::SourceMarker;
use crate::autofix::{apply_fixes, FixResult};
use crate::rules::pycodestyle::rules::MissingNewlineAtEndOfFile;
#[allow(deprecated)]
fn create_diagnostics(edit: impl IntoIterator<Item = Edit>) -> Vec<Diagnostic> {
edit.into_iter()
.map(|edit| Diagnostic {
// The choice of rule here is arbitrary.
kind: MissingNewlineAtEndOfFile.into(),
range: edit.range(),
fix: Some(Fix::unspecified(edit)),
parent: None,
})
.collect()
}
#[test]
fn empty_file() {
let locator = Locator::new(r#""#);
let diagnostics = create_diagnostics([]);
let FixResult {
code,
fixes,
source_map,
} = apply_fixes(diagnostics.iter(), &locator);
assert_eq!(code, "");
assert_eq!(fixes.values().sum::<usize>(), 0);
assert!(source_map.markers().is_empty());
}
#[test]
fn apply_one_insertion() {
let locator = Locator::new(
r#"
import os
print("hello world")
"#
.trim(),
);
let diagnostics = create_diagnostics([Edit::insertion(
"import sys\n".to_string(),
TextSize::new(10),
)]);
let FixResult {
code,
fixes,
source_map,
} = apply_fixes(diagnostics.iter(), &locator);
assert_eq!(
code,
r#"
import os
import sys
print("hello world")
"#
.trim()
);
assert_eq!(fixes.values().sum::<usize>(), 1);
assert_eq!(
source_map.markers(),
&[
SourceMarker {
source: 10.into(),
dest: 10.into(),
},
SourceMarker {
source: 10.into(),
dest: 21.into(),
},
]
);
}
#[test]
fn apply_one_replacement() {
let locator = Locator::new(
r#"
class A(object):
...
"#
.trim(),
);
let diagnostics = create_diagnostics([Edit::replacement(
"Bar".to_string(),
TextSize::new(8),
TextSize::new(14),
)]);
let FixResult {
code,
fixes,
source_map,
} = apply_fixes(diagnostics.iter(), &locator);
assert_eq!(
code,
r#"
class A(Bar):
...
"#
.trim(),
);
assert_eq!(fixes.values().sum::<usize>(), 1);
assert_eq!(
source_map.markers(),
&[
SourceMarker {
source: 8.into(),
dest: 8.into(),
},
SourceMarker {
source: 14.into(),
dest: 11.into(),
},
]
);
}
#[test]
fn apply_one_removal() {
let locator = Locator::new(
r#"
class A(object):
...
"#
.trim(),
);
let diagnostics = create_diagnostics([Edit::deletion(TextSize::new(7), TextSize::new(15))]);
let FixResult {
code,
fixes,
source_map,
} = apply_fixes(diagnostics.iter(), &locator);
assert_eq!(
code,
r#"
class A:
...
"#
.trim()
);
assert_eq!(fixes.values().sum::<usize>(), 1);
assert_eq!(
source_map.markers(),
&[
SourceMarker {
source: 7.into(),
dest: 7.into()
},
SourceMarker {
source: 15.into(),
dest: 7.into()
}
]
);
}
#[test]
fn apply_two_removals() {
let locator = Locator::new(
r#"
class A(object, object, object):
...
"#
.trim(),
);
let diagnostics = create_diagnostics([
Edit::deletion(TextSize::from(8), TextSize::from(16)),
Edit::deletion(TextSize::from(22), TextSize::from(30)),
]);
let FixResult {
code,
fixes,
source_map,
} = apply_fixes(diagnostics.iter(), &locator);
assert_eq!(
code,
r#"
class A(object):
...
"#
.trim()
);
assert_eq!(fixes.values().sum::<usize>(), 2);
assert_eq!(
source_map.markers(),
&[
SourceMarker {
source: 8.into(),
dest: 8.into()
},
SourceMarker {
source: 16.into(),
dest: 8.into()
},
SourceMarker {
source: 22.into(),
dest: 14.into(),
},
SourceMarker {
source: 30.into(),
dest: 14.into(),
}
]
);
}
#[test]
fn ignore_overlapping_fixes() {
let locator = Locator::new(
r#"
class A(object):
...
"#
.trim(),
);
let diagnostics = create_diagnostics([
Edit::deletion(TextSize::from(7), TextSize::from(15)),
Edit::replacement("ignored".to_string(), TextSize::from(9), TextSize::from(11)),
]);
let FixResult {
code,
fixes,
source_map,
} = apply_fixes(diagnostics.iter(), &locator);
assert_eq!(
code,
r#"
class A:
...
"#
.trim(),
);
assert_eq!(fixes.values().sum::<usize>(), 1);
assert_eq!(
source_map.markers(),
&[
SourceMarker {
source: 7.into(),
dest: 7.into(),
},
SourceMarker {
source: 15.into(),
dest: 7.into(),
}
]
);
}
}

View File

@@ -1,59 +0,0 @@
use ruff_text_size::TextSize;
use ruff_diagnostics::Edit;
/// Lightweight sourcemap marker representing the source and destination
/// position for an [`Edit`].
#[derive(Debug, PartialEq, Eq)]
pub(crate) struct SourceMarker {
/// Position of the marker in the original source.
pub(crate) source: TextSize,
/// Position of the marker in the transformed code.
pub(crate) dest: TextSize,
}
/// A collection of [`SourceMarker`].
///
/// Sourcemaps are used to map positions in the original source to positions in
/// the transformed code. Here, only the boundaries of edits are tracked instead
/// of every single character.
#[derive(Default, PartialEq, Eq)]
pub(crate) struct SourceMap(Vec<SourceMarker>);
impl SourceMap {
/// Returns a slice of all the markers in the sourcemap in the order they
/// were added.
pub(crate) fn markers(&self) -> &[SourceMarker] {
&self.0
}
/// Push the start marker for an [`Edit`].
///
/// The `output_length` is the length of the transformed string before the
/// edit is applied.
pub(crate) fn push_start_marker(&mut self, edit: &Edit, output_length: TextSize) {
self.0.push(SourceMarker {
source: edit.start(),
dest: output_length,
});
}
/// Push the end marker for an [`Edit`].
///
/// The `output_length` is the length of the transformed string after the
/// edit has been applied.
pub(crate) fn push_end_marker(&mut self, edit: &Edit, output_length: TextSize) {
if edit.is_insertion() {
self.0.push(SourceMarker {
source: edit.start(),
dest: output_length,
});
} else {
// Deletion or replacement
self.0.push(SourceMarker {
source: edit.end(),
dest: output_length,
});
}
}
}

View File

@@ -1,27 +0,0 @@
use ruff_python_ast::{Arg, Ranged};
use crate::checkers::ast::Checker;
use crate::codes::Rule;
use crate::rules::{flake8_builtins, pep8_naming, pycodestyle};
/// Run lint rules over an [`Arg`] syntax node.
pub(crate) fn argument(arg: &Arg, checker: &mut Checker) {
if checker.enabled(Rule::AmbiguousVariableName) {
if let Some(diagnostic) = pycodestyle::rules::ambiguous_variable_name(&arg.arg, arg.range())
{
checker.diagnostics.push(diagnostic);
}
}
if checker.enabled(Rule::InvalidArgumentName) {
if let Some(diagnostic) = pep8_naming::rules::invalid_argument_name(
&arg.arg,
arg,
&checker.settings.pep8_naming.ignore_names,
) {
checker.diagnostics.push(diagnostic);
}
}
if checker.enabled(Rule::BuiltinArgumentShadowing) {
flake8_builtins::rules::builtin_argument_shadowing(checker, arg);
}
}

View File

@@ -1,26 +0,0 @@
use ruff_python_ast::Arguments;
use crate::checkers::ast::Checker;
use crate::codes::Rule;
use crate::rules::{flake8_bugbear, flake8_pyi, ruff};
/// Run lint rules over a [`Arguments`] syntax node.
pub(crate) fn arguments(arguments: &Arguments, checker: &mut Checker) {
if checker.enabled(Rule::MutableArgumentDefault) {
flake8_bugbear::rules::mutable_argument_default(checker, arguments);
}
if checker.enabled(Rule::FunctionCallInDefaultArgument) {
flake8_bugbear::rules::function_call_in_argument_default(checker, arguments);
}
if checker.settings.rules.enabled(Rule::ImplicitOptional) {
ruff::rules::implicit_optional(checker, arguments);
}
if checker.is_stub {
if checker.enabled(Rule::TypedArgumentDefaultInStub) {
flake8_pyi::rules::typed_argument_simple_defaults(checker, arguments);
}
if checker.enabled(Rule::ArgumentDefaultInStub) {
flake8_pyi::rules::argument_simple_defaults(checker, arguments);
}
}
}

View File

@@ -1,84 +0,0 @@
use crate::checkers::ast::Checker;
use crate::codes::Rule;
use crate::rules::{flake8_import_conventions, flake8_pyi, pyflakes, pylint};
use ruff_diagnostics::{Diagnostic, Fix};
/// Run lint rules over the [`Binding`]s.
pub(crate) fn bindings(checker: &mut Checker) {
if !checker.any_enabled(&[
Rule::InvalidAllFormat,
Rule::InvalidAllObject,
Rule::UnaliasedCollectionsAbcSetImport,
Rule::UnconventionalImportAlias,
Rule::UnusedPrivateTypeVar,
Rule::UnusedVariable,
Rule::UnusedPrivateProtocol,
]) {
return;
}
for binding in checker.semantic.bindings.iter() {
if checker.enabled(Rule::UnusedVariable) {
if binding.kind.is_bound_exception() && !binding.is_used() {
let mut diagnostic = Diagnostic::new(
pyflakes::rules::UnusedVariable {
name: binding.name(checker.locator).to_string(),
},
binding.range,
);
if checker.patch(Rule::UnusedVariable) {
diagnostic.try_set_fix(|| {
pyflakes::fixes::remove_exception_handler_assignment(
binding,
checker.locator,
)
.map(Fix::automatic)
});
}
checker.diagnostics.push(diagnostic);
}
}
if checker.enabled(Rule::InvalidAllFormat) {
if let Some(diagnostic) = pylint::rules::invalid_all_format(binding) {
checker.diagnostics.push(diagnostic);
}
}
if checker.enabled(Rule::InvalidAllObject) {
if let Some(diagnostic) = pylint::rules::invalid_all_object(binding) {
checker.diagnostics.push(diagnostic);
}
}
if checker.enabled(Rule::UnconventionalImportAlias) {
if let Some(diagnostic) = flake8_import_conventions::rules::unconventional_import_alias(
checker,
binding,
&checker.settings.flake8_import_conventions.aliases,
) {
checker.diagnostics.push(diagnostic);
}
}
if checker.is_stub {
if checker.enabled(Rule::UnaliasedCollectionsAbcSetImport) {
if let Some(diagnostic) =
flake8_pyi::rules::unaliased_collections_abc_set_import(checker, binding)
{
checker.diagnostics.push(diagnostic);
}
}
if checker.enabled(Rule::UnusedPrivateTypeVar) {
if let Some(diagnostic) =
flake8_pyi::rules::unused_private_type_var(checker, binding)
{
checker.diagnostics.push(diagnostic);
}
}
if checker.enabled(Rule::UnusedPrivateProtocol) {
if let Some(diagnostic) =
flake8_pyi::rules::unused_private_protocol(checker, binding)
{
checker.diagnostics.push(diagnostic);
}
}
}
}
}

View File

@@ -1,16 +0,0 @@
use ruff_python_ast::Comprehension;
use crate::checkers::ast::Checker;
use crate::codes::Rule;
use crate::rules::flake8_simplify;
/// Run lint rules over a [`Comprehension`] syntax nodes.
pub(crate) fn comprehension(comprehension: &Comprehension, checker: &mut Checker) {
if checker.enabled(Rule::InDictKeys) {
flake8_simplify::rules::key_in_dict_for(
checker,
&comprehension.target,
&comprehension.iter,
);
}
}

View File

@@ -1,32 +0,0 @@
use ruff_python_ast::{self as ast, Stmt};
use crate::checkers::ast::Checker;
use crate::codes::Rule;
use crate::rules::{flake8_bugbear, perflint};
/// Run lint rules over all deferred for-loops in the [`SemanticModel`].
pub(crate) fn deferred_for_loops(checker: &mut Checker) {
while !checker.deferred.for_loops.is_empty() {
let for_loops = std::mem::take(&mut checker.deferred.for_loops);
for snapshot in for_loops {
checker.semantic.restore(snapshot);
if let Stmt::For(ast::StmtFor {
target, iter, body, ..
})
| Stmt::AsyncFor(ast::StmtAsyncFor {
target, iter, body, ..
}) = &checker.semantic.stmt()
{
if checker.enabled(Rule::UnusedLoopControlVariable) {
flake8_bugbear::rules::unused_loop_control_variable(checker, target, body);
}
if checker.enabled(Rule::IncorrectDictIterator) {
perflint::rules::incorrect_dict_iterator(checker, target, iter);
}
} else {
unreachable!("Expected Expr::For | Expr::AsyncFor");
}
}
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -1,27 +0,0 @@
pub(super) use argument::argument;
pub(super) use arguments::arguments;
pub(super) use bindings::bindings;
pub(super) use comprehension::comprehension;
pub(super) use deferred_for_loops::deferred_for_loops;
pub(super) use deferred_scopes::deferred_scopes;
pub(super) use definitions::definitions;
pub(super) use except_handler::except_handler;
pub(super) use expression::expression;
pub(super) use module::module;
pub(super) use statement::statement;
pub(super) use suite::suite;
pub(super) use unresolved_references::unresolved_references;
mod argument;
mod arguments;
mod bindings;
mod comprehension;
mod deferred_for_loops;
mod deferred_scopes;
mod definitions;
mod except_handler;
mod expression;
mod module;
mod statement;
mod suite;
mod unresolved_references;

File diff suppressed because it is too large Load Diff

View File

@@ -1,124 +0,0 @@
//! Lint rules based on import analysis.
use std::borrow::Cow;
use std::path::Path;
use ruff_python_ast::{self as ast, Ranged, Stmt, Suite};
use ruff_diagnostics::Diagnostic;
use ruff_python_ast::helpers::to_module_path;
use ruff_python_ast::imports::{ImportMap, ModuleImport};
use ruff_python_ast::statement_visitor::StatementVisitor;
use ruff_python_codegen::Stylist;
use ruff_python_index::Indexer;
use ruff_python_stdlib::path::is_python_stub_file;
use ruff_source_file::Locator;
use crate::directives::IsortDirectives;
use crate::registry::Rule;
use crate::rules::isort;
use crate::rules::isort::block::{Block, BlockBuilder};
use crate::settings::Settings;
use crate::source_kind::SourceKind;
fn extract_import_map(path: &Path, package: Option<&Path>, blocks: &[&Block]) -> Option<ImportMap> {
let Some(package) = package else {
return None;
};
let Some(module_path) = to_module_path(package, path) else {
return None;
};
let num_imports = blocks.iter().map(|block| block.imports.len()).sum();
let mut module_imports = Vec::with_capacity(num_imports);
for stmt in blocks.iter().flat_map(|block| &block.imports) {
match stmt {
Stmt::Import(ast::StmtImport { names, range: _ }) => {
module_imports.extend(
names
.iter()
.map(|name| ModuleImport::new(name.name.to_string(), stmt.range())),
);
}
Stmt::ImportFrom(ast::StmtImportFrom {
module,
names,
level,
range: _,
}) => {
let level = level.map_or(0, |level| level.to_usize());
let module = if let Some(module) = module {
let module: &String = module.as_ref();
if level == 0 {
Cow::Borrowed(module)
} else {
if module_path.len() <= level {
continue;
}
let prefix = module_path[..module_path.len() - level].join(".");
Cow::Owned(format!("{prefix}.{module}"))
}
} else {
if module_path.len() <= level {
continue;
}
Cow::Owned(module_path[..module_path.len() - level].join("."))
};
module_imports.extend(names.iter().map(|name| {
ModuleImport::new(format!("{}.{}", module, name.name), name.range())
}));
}
_ => panic!("Expected Stmt::Import | Stmt::ImportFrom"),
}
}
let mut import_map = ImportMap::default();
import_map.insert(module_path.join("."), module_imports);
Some(import_map)
}
#[allow(clippy::too_many_arguments)]
pub(crate) fn check_imports(
python_ast: &Suite,
locator: &Locator,
indexer: &Indexer,
directives: &IsortDirectives,
settings: &Settings,
stylist: &Stylist,
path: &Path,
package: Option<&Path>,
source_kind: Option<&SourceKind>,
) -> (Vec<Diagnostic>, Option<ImportMap>) {
let is_stub = is_python_stub_file(path);
// Extract all import blocks from the AST.
let tracker = {
let mut tracker = BlockBuilder::new(locator, directives, is_stub, source_kind);
tracker.visit_body(python_ast);
tracker
};
let blocks: Vec<&Block> = tracker.iter().collect();
// Enforce import rules.
let mut diagnostics = vec![];
if settings.rules.enabled(Rule::UnsortedImports) {
for block in &blocks {
if !block.imports.is_empty() {
if let Some(diagnostic) = isort::rules::organize_imports(
block, locator, stylist, indexer, settings, package,
) {
diagnostics.push(diagnostic);
}
}
}
}
if settings.rules.enabled(Rule::MissingRequiredImport) {
diagnostics.extend(isort::rules::add_required_imports(
python_ast, locator, stylist, settings, is_stub,
));
}
// Extract import map.
let imports = extract_import_map(path, package, &blocks);
(diagnostics, imports)
}

View File

@@ -1,236 +0,0 @@
//! `NoQA` enforcement and validation.
use std::path::Path;
use itertools::Itertools;
use ruff_python_ast::Ranged;
use ruff_text_size::{TextLen, TextRange, TextSize};
use ruff_diagnostics::{Diagnostic, Edit, Fix};
use ruff_source_file::Locator;
use crate::noqa;
use crate::noqa::{Directive, FileExemption, NoqaDirectives, NoqaMapping};
use crate::registry::{AsRule, Rule};
use crate::rule_redirects::get_redirect_target;
use crate::rules::ruff::rules::{UnusedCodes, UnusedNOQA};
use crate::settings::Settings;
pub(crate) fn check_noqa(
diagnostics: &mut Vec<Diagnostic>,
path: &Path,
locator: &Locator,
comment_ranges: &[TextRange],
noqa_line_for: &NoqaMapping,
analyze_directives: bool,
settings: &Settings,
) -> Vec<usize> {
// Identify any codes that are globally exempted (within the current file).
let exemption = FileExemption::try_extract(locator.contents(), comment_ranges, path, locator);
// Extract all `noqa` directives.
let mut noqa_directives = NoqaDirectives::from_commented_ranges(comment_ranges, path, locator);
// Indices of diagnostics that were ignored by a `noqa` directive.
let mut ignored_diagnostics = vec![];
// Remove any ignored diagnostics.
'outer: for (index, diagnostic) in diagnostics.iter().enumerate() {
if matches!(diagnostic.kind.rule(), Rule::BlanketNOQA) {
continue;
}
match &exemption {
Some(FileExemption::All) => {
// If the file is exempted, ignore all diagnostics.
ignored_diagnostics.push(index);
continue;
}
Some(FileExemption::Codes(codes)) => {
// If the diagnostic is ignored by a global exemption, ignore it.
if codes.contains(&diagnostic.kind.rule().noqa_code()) {
ignored_diagnostics.push(index);
continue;
}
}
None => {}
}
let noqa_offsets = diagnostic
.parent
.into_iter()
.chain(std::iter::once(diagnostic.start()))
.map(|position| noqa_line_for.resolve(position))
.unique();
for noqa_offset in noqa_offsets {
if let Some(directive_line) = noqa_directives.find_line_with_directive_mut(noqa_offset)
{
let suppressed = match &directive_line.directive {
Directive::All(_) => {
directive_line
.matches
.push(diagnostic.kind.rule().noqa_code());
ignored_diagnostics.push(index);
true
}
Directive::Codes(directive) => {
if noqa::includes(diagnostic.kind.rule(), directive.codes()) {
directive_line
.matches
.push(diagnostic.kind.rule().noqa_code());
ignored_diagnostics.push(index);
true
} else {
false
}
}
};
if suppressed {
continue 'outer;
}
}
}
}
// Enforce that the noqa directive was actually used (RUF100).
if analyze_directives && settings.rules.enabled(Rule::UnusedNOQA) {
for line in noqa_directives.lines() {
match &line.directive {
Directive::All(directive) => {
if line.matches.is_empty() {
let mut diagnostic =
Diagnostic::new(UnusedNOQA { codes: None }, directive.range());
if settings.rules.should_fix(diagnostic.kind.rule()) {
#[allow(deprecated)]
diagnostic.set_fix_from_edit(delete_noqa(directive.range(), locator));
}
diagnostics.push(diagnostic);
}
}
Directive::Codes(directive) => {
let mut disabled_codes = vec![];
let mut unknown_codes = vec![];
let mut unmatched_codes = vec![];
let mut valid_codes = vec![];
let mut self_ignore = false;
for code in directive.codes() {
let code = get_redirect_target(code).unwrap_or(code);
if Rule::UnusedNOQA.noqa_code() == code {
self_ignore = true;
break;
}
if line.matches.iter().any(|match_| *match_ == code)
|| settings.external.contains(code)
{
valid_codes.push(code);
} else {
if let Ok(rule) = Rule::from_code(code) {
if settings.rules.enabled(rule) {
unmatched_codes.push(code);
} else {
disabled_codes.push(code);
}
} else {
unknown_codes.push(code);
}
}
}
if self_ignore {
continue;
}
if !(disabled_codes.is_empty()
&& unknown_codes.is_empty()
&& unmatched_codes.is_empty())
{
let mut diagnostic = Diagnostic::new(
UnusedNOQA {
codes: Some(UnusedCodes {
disabled: disabled_codes
.iter()
.map(|code| (*code).to_string())
.collect(),
unknown: unknown_codes
.iter()
.map(|code| (*code).to_string())
.collect(),
unmatched: unmatched_codes
.iter()
.map(|code| (*code).to_string())
.collect(),
}),
},
directive.range(),
);
if settings.rules.should_fix(diagnostic.kind.rule()) {
if valid_codes.is_empty() {
#[allow(deprecated)]
diagnostic
.set_fix_from_edit(delete_noqa(directive.range(), locator));
} else {
#[allow(deprecated)]
diagnostic.set_fix(Fix::unspecified(Edit::range_replacement(
format!("# noqa: {}", valid_codes.join(", ")),
directive.range(),
)));
}
}
diagnostics.push(diagnostic);
}
}
}
}
}
ignored_diagnostics.sort_unstable();
ignored_diagnostics
}
/// Generate a [`Edit`] to delete a `noqa` directive.
fn delete_noqa(range: TextRange, locator: &Locator) -> Edit {
let line_range = locator.line_range(range.start());
// Compute the leading space.
let prefix = locator.slice(TextRange::new(line_range.start(), range.start()));
let leading_space = prefix
.rfind(|c: char| !c.is_whitespace())
.map_or(prefix.len(), |i| prefix.len() - i - 1);
let leading_space_len = TextSize::try_from(leading_space).unwrap();
// Compute the trailing space.
let suffix = locator.slice(TextRange::new(range.end(), line_range.end()));
let trailing_space = suffix
.find(|c: char| !c.is_whitespace())
.map_or(suffix.len(), |i| i);
let trailing_space_len = TextSize::try_from(trailing_space).unwrap();
// Ex) `# noqa`
if line_range
== TextRange::new(
range.start() - leading_space_len,
range.end() + trailing_space_len,
)
{
let full_line_end = locator.full_line_end(line_range.end());
Edit::deletion(line_range.start(), full_line_end)
}
// Ex) `x = 1 # noqa`
else if range.end() + trailing_space_len == line_range.end() {
Edit::deletion(range.start() - leading_space_len, line_range.end())
}
// Ex) `x = 1 # noqa # type: ignore`
else if locator.contents()[usize::from(range.end() + trailing_space_len)..].starts_with('#') {
Edit::deletion(range.start(), range.end() + trailing_space_len)
}
// Ex) `x = 1 # noqa here`
else {
Edit::deletion(
range.start() + "# ".text_len(),
range.end() + trailing_space_len,
)
}
}

View File

@@ -1,5 +0,0 @@
---
source: crates/ruff/src/comments/shebang.rs
expression: "ShebangDirective::try_extract(source)"
---
None

View File

@@ -1,5 +0,0 @@
---
source: crates/ruff/src/comments/shebang.rs
expression: "ShebangDirective::try_extract(source)"
---
None

View File

@@ -1,9 +0,0 @@
---
source: crates/ruff/src/comments/shebang.rs
expression: "ShebangDirective::try_extract(source)"
---
Some(
ShebangDirective(
"/usr/bin/env python",
),
)

View File

@@ -1,9 +0,0 @@
---
source: crates/ruff/src/comments/shebang.rs
expression: "ShebangDirective::try_extract(source)"
---
Some(
ShebangDirective(
"/usr/bin/env python # trailing comment",
),
)

View File

@@ -1,5 +0,0 @@
---
source: crates/ruff/src/comments/shebang.rs
expression: "ShebangDirective::try_extract(source)"
---
None

View File

@@ -1,38 +0,0 @@
use libcst_native::{Expression, NameOrAttribute};
fn compose_call_path_inner<'a>(expr: &'a Expression, parts: &mut Vec<&'a str>) {
match expr {
Expression::Call(expr) => {
compose_call_path_inner(&expr.func, parts);
}
Expression::Attribute(expr) => {
compose_call_path_inner(&expr.value, parts);
parts.push(expr.attr.value);
}
Expression::Name(expr) => {
parts.push(expr.value);
}
_ => {}
}
}
pub(crate) fn compose_call_path(expr: &Expression) -> Option<String> {
let mut segments = vec![];
compose_call_path_inner(expr, &mut segments);
if segments.is_empty() {
None
} else {
Some(segments.join("."))
}
}
pub(crate) fn compose_module_path(module: &NameOrAttribute) -> String {
match module {
NameOrAttribute::N(name) => name.value.to_string(),
NameOrAttribute::A(attr) => {
let name = attr.attr.value;
let prefix = compose_call_path(&attr.value);
prefix.map_or_else(|| name.to_string(), |prefix| format!("{prefix}.{name}"))
}
}
}

View File

@@ -1,207 +0,0 @@
use anyhow::{bail, Result};
use libcst_native::{
Arg, Attribute, Call, Comparison, CompoundStatement, Dict, Expression, FunctionDef,
GeneratorExp, If, Import, ImportAlias, ImportFrom, ImportNames, IndentedBlock, Lambda,
ListComp, Module, Name, SmallStatement, Statement, Suite, Tuple, With,
};
pub(crate) fn match_module(module_text: &str) -> Result<Module> {
match libcst_native::parse_module(module_text, None) {
Ok(module) => Ok(module),
Err(_) => bail!("Failed to extract CST from source"),
}
}
pub(crate) fn match_expression(expression_text: &str) -> Result<Expression> {
match libcst_native::parse_expression(expression_text) {
Ok(expression) => Ok(expression),
Err(_) => bail!("Failed to extract expression from source"),
}
}
pub(crate) fn match_statement(statement_text: &str) -> Result<Statement> {
match libcst_native::parse_statement(statement_text) {
Ok(statement) => Ok(statement),
Err(_) => bail!("Failed to extract statement from source"),
}
}
pub(crate) fn match_import<'a, 'b>(statement: &'a mut Statement<'b>) -> Result<&'a mut Import<'b>> {
if let Statement::Simple(expr) = statement {
if let Some(SmallStatement::Import(expr)) = expr.body.first_mut() {
Ok(expr)
} else {
bail!("Expected SmallStatement::Import")
}
} else {
bail!("Expected Statement::Simple")
}
}
pub(crate) fn match_import_from<'a, 'b>(
statement: &'a mut Statement<'b>,
) -> Result<&'a mut ImportFrom<'b>> {
if let Statement::Simple(expr) = statement {
if let Some(SmallStatement::ImportFrom(expr)) = expr.body.first_mut() {
Ok(expr)
} else {
bail!("Expected SmallStatement::ImportFrom")
}
} else {
bail!("Expected Statement::Simple")
}
}
pub(crate) fn match_aliases<'a, 'b>(
import_from: &'a mut ImportFrom<'b>,
) -> Result<&'a mut Vec<ImportAlias<'b>>> {
if let ImportNames::Aliases(aliases) = &mut import_from.names {
Ok(aliases)
} else {
bail!("Expected ImportNames::Aliases")
}
}
pub(crate) fn match_call<'a, 'b>(expression: &'a Expression<'b>) -> Result<&'a Call<'b>> {
if let Expression::Call(call) = expression {
Ok(call)
} else {
bail!("Expected Expression::Call")
}
}
pub(crate) fn match_call_mut<'a, 'b>(
expression: &'a mut Expression<'b>,
) -> Result<&'a mut Call<'b>> {
if let Expression::Call(call) = expression {
Ok(call)
} else {
bail!("Expected Expression::Call")
}
}
pub(crate) fn match_comparison<'a, 'b>(
expression: &'a mut Expression<'b>,
) -> Result<&'a mut Comparison<'b>> {
if let Expression::Comparison(comparison) = expression {
Ok(comparison)
} else {
bail!("Expected Expression::Comparison")
}
}
pub(crate) fn match_dict<'a, 'b>(expression: &'a mut Expression<'b>) -> Result<&'a mut Dict<'b>> {
if let Expression::Dict(dict) = expression {
Ok(dict)
} else {
bail!("Expected Expression::Dict")
}
}
pub(crate) fn match_attribute<'a, 'b>(
expression: &'a mut Expression<'b>,
) -> Result<&'a mut Attribute<'b>> {
if let Expression::Attribute(attribute) = expression {
Ok(attribute)
} else {
bail!("Expected Expression::Attribute")
}
}
pub(crate) fn match_name<'a, 'b>(expression: &'a Expression<'b>) -> Result<&'a Name<'b>> {
if let Expression::Name(name) = expression {
Ok(name)
} else {
bail!("Expected Expression::Name")
}
}
pub(crate) fn match_arg<'a, 'b>(call: &'a Call<'b>) -> Result<&'a Arg<'b>> {
if let Some(arg) = call.args.first() {
Ok(arg)
} else {
bail!("Expected Arg")
}
}
pub(crate) fn match_generator_exp<'a, 'b>(
expression: &'a Expression<'b>,
) -> Result<&'a GeneratorExp<'b>> {
if let Expression::GeneratorExp(generator_exp) = expression {
Ok(generator_exp)
} else {
bail!("Expected Expression::GeneratorExp")
}
}
pub(crate) fn match_tuple<'a, 'b>(expression: &'a Expression<'b>) -> Result<&'a Tuple<'b>> {
if let Expression::Tuple(tuple) = expression {
Ok(tuple)
} else {
bail!("Expected Expression::Tuple")
}
}
pub(crate) fn match_list_comp<'a, 'b>(expression: &'a Expression<'b>) -> Result<&'a ListComp<'b>> {
if let Expression::ListComp(list_comp) = expression {
Ok(list_comp)
} else {
bail!("Expected Expression::ListComp")
}
}
pub(crate) fn match_lambda<'a, 'b>(expression: &'a Expression<'b>) -> Result<&'a Lambda<'b>> {
if let Expression::Lambda(lambda) = expression {
Ok(lambda)
} else {
bail!("Expected Expression::Lambda")
}
}
pub(crate) fn match_function_def<'a, 'b>(
statement: &'a mut Statement<'b>,
) -> Result<&'a mut FunctionDef<'b>> {
if let Statement::Compound(compound) = statement {
if let CompoundStatement::FunctionDef(function_def) = compound {
Ok(function_def)
} else {
bail!("Expected CompoundStatement::FunctionDef")
}
} else {
bail!("Expected Statement::Compound")
}
}
pub(crate) fn match_indented_block<'a, 'b>(
suite: &'a mut Suite<'b>,
) -> Result<&'a mut IndentedBlock<'b>> {
if let Suite::IndentedBlock(indented_block) = suite {
Ok(indented_block)
} else {
bail!("Expected Suite::IndentedBlock")
}
}
pub(crate) fn match_with<'a, 'b>(statement: &'a mut Statement<'b>) -> Result<&'a mut With<'b>> {
if let Statement::Compound(compound) = statement {
if let CompoundStatement::With(with) = compound {
Ok(with)
} else {
bail!("Expected CompoundStatement::With")
}
} else {
bail!("Expected Statement::Compound")
}
}
pub(crate) fn match_if<'a, 'b>(statement: &'a mut Statement<'b>) -> Result<&'a mut If<'b>> {
if let Statement::Compound(compound) = statement {
if let CompoundStatement::If(if_) = compound {
Ok(if_)
} else {
bail!("Expected CompoundStatement::If")
}
} else {
bail!("Expected Statement::Compound")
}
}

View File

@@ -1,91 +0,0 @@
//! Extract docstrings from an AST.
use ruff_python_ast::{self as ast, Constant, Expr, Stmt};
use ruff_python_semantic::{Definition, DefinitionId, Definitions, Member, MemberKind};
/// Extract a docstring from a function or class body.
pub(crate) fn docstring_from(suite: &[Stmt]) -> Option<&Expr> {
let stmt = suite.first()?;
// Require the docstring to be a standalone expression.
let Stmt::Expr(ast::StmtExpr { value, range: _ }) = stmt else {
return None;
};
// Only match strings.
if !matches!(
value.as_ref(),
Expr::Constant(ast::ExprConstant {
value: Constant::Str(_),
..
})
) {
return None;
}
Some(value)
}
/// Extract a docstring from a `Definition`.
pub(crate) fn extract_docstring<'a>(definition: &'a Definition<'a>) -> Option<&'a Expr> {
match definition {
Definition::Module(module) => docstring_from(module.python_ast),
Definition::Member(member) => {
if let Stmt::ClassDef(ast::StmtClassDef { body, .. })
| Stmt::FunctionDef(ast::StmtFunctionDef { body, .. })
| Stmt::AsyncFunctionDef(ast::StmtAsyncFunctionDef { body, .. }) = &member.stmt
{
docstring_from(body)
} else {
None
}
}
}
}
#[derive(Copy, Clone)]
pub(crate) enum ExtractionTarget {
Class,
Function,
}
/// Extract a `Definition` from the AST node defined by a `Stmt`.
pub(crate) fn extract_definition<'a>(
target: ExtractionTarget,
stmt: &'a Stmt,
parent: DefinitionId,
definitions: &Definitions<'a>,
) -> Member<'a> {
match target {
ExtractionTarget::Function => match &definitions[parent] {
Definition::Module(..) => Member {
parent,
kind: MemberKind::Function,
stmt,
},
Definition::Member(Member {
kind: MemberKind::Class | MemberKind::NestedClass,
..
}) => Member {
parent,
kind: MemberKind::Method,
stmt,
},
Definition::Member(..) => Member {
parent,
kind: MemberKind::NestedFunction,
stmt,
},
},
ExtractionTarget::Class => match &definitions[parent] {
Definition::Module(..) => Member {
parent,
kind: MemberKind::Class,
stmt,
},
Definition::Member(..) => Member {
parent,
kind: MemberKind::NestedClass,
stmt,
},
},
}
}

View File

@@ -1,83 +0,0 @@
use std::fmt::{Debug, Formatter};
use std::ops::Deref;
use ruff_python_ast::{Expr, Ranged};
use ruff_text_size::{TextRange, TextSize};
use ruff_python_semantic::Definition;
pub(crate) mod extraction;
pub(crate) mod google;
pub(crate) mod numpy;
pub(crate) mod sections;
pub(crate) mod styles;
#[derive(Debug)]
pub(crate) struct Docstring<'a> {
pub(crate) definition: &'a Definition<'a>,
pub(crate) expr: &'a Expr,
/// The content of the docstring, including the leading and trailing quotes.
pub(crate) contents: &'a str,
/// The range of the docstring body (without the quotes). The range is relative to [`Self::contents`].
pub(crate) body_range: TextRange,
pub(crate) indentation: &'a str,
}
impl<'a> Docstring<'a> {
pub(crate) fn body(&self) -> DocstringBody {
DocstringBody { docstring: self }
}
pub(crate) fn start(&self) -> TextSize {
self.expr.start()
}
pub(crate) fn end(&self) -> TextSize {
self.expr.end()
}
pub(crate) fn range(&self) -> TextRange {
self.expr.range()
}
pub(crate) fn leading_quote(&self) -> &'a str {
&self.contents[TextRange::up_to(self.body_range.start())]
}
}
#[derive(Copy, Clone)]
pub(crate) struct DocstringBody<'a> {
docstring: &'a Docstring<'a>,
}
impl<'a> DocstringBody<'a> {
#[inline]
pub(crate) fn start(self) -> TextSize {
self.range().start()
}
pub(crate) fn range(self) -> TextRange {
self.docstring.body_range + self.docstring.start()
}
pub(crate) fn as_str(self) -> &'a str {
&self.docstring.contents[self.docstring.body_range]
}
}
impl Deref for DocstringBody<'_> {
type Target = str;
fn deref(&self) -> &Self::Target {
self.as_str()
}
}
impl Debug for DocstringBody<'_> {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
f.debug_struct("DocstringBody")
.field("text", &self.as_str())
.field("range", &self.range())
.finish()
}
}

Some files were not shown because too many files have changed in this diff Show More