Compare commits

...

469 Commits

Author SHA1 Message Date
Micha Reiser
5d2513e5a9 POC of Black's string preview style formatting 2023-09-21 12:38:03 +02:00
Micha Reiser
272306bf5a Introduce StringContinuation data structure 2023-09-21 08:01:39 +02:00
Micha Reiser
f8f1cd5016 Introduce FormatterSettings (#7545) 2023-09-21 08:01:24 +02:00
Charlie Marsh
87a0cd219f Detect asyncio.get_running_loop calls in RUF006 (#7562)
## Summary

We can do a good enough job detecting this with our existing semantic
model.

Closes https://github.com/astral-sh/ruff/issues/3237.
2023-09-21 04:37:38 +00:00
Charlie Marsh
b685ee4749 Enable tab completion for ruff rule (#7560)
## Summary

Writing `ruff rule E1`, then tab, shows:

<img width="724" alt="Screen Shot 2023-09-20 at 9 29 09 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/00297f24-8828-4485-a00e-6af1ab4e7875">

Closes https://github.com/astral-sh/ruff/issues/2812.
2023-09-20 22:05:39 -04:00
Charlie Marsh
ad893f8295 Avoid invalid fix for parenthesized values in F601 (#7559)
Closes https://github.com/astral-sh/ruff/issues/4897.
2023-09-21 01:28:11 +00:00
Charlie Marsh
621bed55c0 Add padding in PERF102 fixes (#7554)
Closes https://github.com/astral-sh/ruff/issues/7097.
2023-09-20 19:33:54 -04:00
Charlie Marsh
86faee1522 Update CONTRIBUTING.md to reflect Settings refactor (#7555)
## Summary

See:
https://github.com/astral-sh/ruff/pull/7544#issuecomment-1728457885.
2023-09-20 19:33:33 -04:00
Charlie Marsh
a0917ec658 Avoid inserting imports directly after continuation (#7553)
## Summary

This is extremely rare in practice, but common in the fuzzer issues so
worth fixing quickly.

Closes https://github.com/astral-sh/ruff/issues/7199.
2023-09-20 21:26:48 +00:00
Mahesh Saripalli
ee9ee005c5 Update flake8-self license to MIT (#7552)
## Summary

This PR updates the flake8-self license from Freely Distributable to the
MIT License to match the upstream license, which was recently updated.


f5fc507515/LICENSE
2023-09-20 17:17:55 -04:00
Charlie Marsh
5df0326bc8 Treat parameters-with-newline as empty in function formatting (#7550)
## Summary

If a function has no parameters (and no comments within the parameters'
`()`), we're supposed to wrap the return annotation _whenever_ it
breaks. However, our `empty_parameters` test didn't properly account for
the case in which the parameters include a newline (but no other
content), like:

```python
def get_dashboards_hierarchy(
) -> Dict[Type['BaseDashboard'], List[Type['BaseDashboard']]]:
    """Get hierarchy of dashboards classes.

    Returns:
        Dict of dashboards classes.
    """
    dashboards_hierarchy = {}
```

This PR fixes that detection. Instead of lexing, it now checks if the
parameters itself is empty (or if it contains comments).

Closes https://github.com/astral-sh/ruff/issues/7457.
2023-09-20 16:20:22 -04:00
Micha Reiser
6540321966 Move Settings and ResolverSettings to ruff_workspace
## Summary

## Stack Summary

This stack splits `Settings` into `FormatterSettings` and `LinterSettings` and moves it into `ruff_workspace`. This change is necessary to add the `FormatterSettings` to `Settings` without adding `ruff_python_formatter` as a dependency to `ruff_linter` (and the linter should not contain the formatter settings). 

A quick overview of our settings struct at play:

* `Options`: 1:1 representation of the options in the `pyproject.toml` or `ruff.toml`.  Used for deserialization.
* `Configuration`: Resolved `Options`, potentially merged from multiple configurations (when using `extend`). The representation is very close if not identical to the `Options`.
* `Settings`: The resolved configuration that uses a data format optimized for reading. Optional fields are initialized with their default values. Initialized by `Configuration::into_settings` .

The goal of this stack is to split `Settings` into tool-specific resolved `Settings` that are independent of each other. This comes at the advantage that the individual crates don't need to know anything about the other tools. The downside is that information gets duplicated between `Settings`. Right now the duplication is minimal (`line-length`, `tab-width`) but we may need to come up with a solution if more expensive data needs sharing.

This stack focuses on `Settings`. Splitting `Configuration` into some smaller structs is something I'll follow up on later. 

## PR Summary

This PR moves the `ResolverSettings` and `Settings` struct to `ruff_workspace`. `LinterSettings` remains in `ruff_linter` because it gets passed to lint rules, the `Checker` etc.

## Test Plan

`cargo test`
2023-09-20 17:24:28 +02:00
Micha Reiser
b34278e0cd Introduce LinterSettings
## Stack Summary

This stack splits `Settings` into `FormatterSettings` and `LinterSettings` and moves it into `ruff_workspace`. This change is necessary to add the `FormatterSettings` to `Settings` without adding `ruff_python_formatter` as a dependency to `ruff_linter` (and the linter should not contain the formatter settings). 

A quick overview of our settings struct at play:

* `Options`: 1:1 representation of the options in the `pyproject.toml` or `ruff.toml`.  Used for deserialization.
* `Configuration`: Resolved `Options`, potentially merged from multiple configurations (when using `extend`). The representation is very close if not identical to the `Options`.
* `Settings`: The resolved configuration that uses a data format optimized for reading. Optional fields are initialized with their default values. Initialized by `Configuration::into_settings` .

The goal of this stack is to split `Settings` into tool-specific resolved `Settings` that are independent of each other. This comes at the advantage that the individual crates don't need to know anything about the other tools. The downside is that information gets duplicated between `Settings`. Right now the duplication is minimal (`line-length`, `tab-width`) but we may need to come up with a solution if more expensive data needs sharing.

This stack focuses on `Settings`. Splitting `Configuration` into some smaller structs is something I'll follow up on later. 

## PR Summary

This PR extracts the linter-specific settings into a new `LinterSettings` struct and adds it as a `linter` field to the `Settings` struct. This is in preparation for moving `Settings` from `ruff_linter` to `ruff_workspace`

## Test Plan

`cargo test`
2023-09-20 17:02:34 +02:00
Micha Reiser
222f1c37b8 Add empty lines between options and move documents (#7547) 2023-09-20 16:40:25 +02:00
Micha Reiser
8f41eab0c7 Extract ResolverSettings
## Stack Summary

This stack splits `Settings` into `FormatterSettings` and `LinterSettings` and moves it into `ruff_workspace`. This change is necessary to add the `FormatterSettings` to `Settings` without adding `ruff_python_formatter` as a dependency to `ruff_linter` (and the linter should not contain the formatter settings). 

A quick overview of our settings struct at play:

* `Options`: 1:1 representation of the options in the `pyproject.toml` or `ruff.toml`.  Used for deserialization.
* `Configuration`: Resolved `Options`, potentially merged from multiple configurations (when using `extend`). The representation is very close if not identical to the `Options`.
* `Settings`: The resolved configuration that uses a data format optimized for reading. Optional fields are initialized with their default values. Initialized by `Configuration::into_settings` .

The goal of this stack is to split `Settings` into tool-specific resolved `Settings` that are independent of each other. This comes at the advantage that the individual crates don't need to know anything about the other tools. The downside is that information gets duplicated between `Settings`. Right now the duplication is minimal (`line-length`, `tab-width`) but we may need to come up with a solution if more expensive data needs sharing.

This stack focuses on `Settings`. Splitting `Configuration` into some smaller structs is something I'll follow up on later. 

## PR Summary

This PR extracts a `ResolverSettings` struct that holds all the resolver-relevant fields (uninteresting for the `Formatter` or `Linter`). This will allow us to move the `ResolverSettings` out of `ruff_linter` further up in the stack.


## Test Plan

`cargo test`

(I'll to more extensive testing at the top of this stack)
2023-09-20 16:37:49 +02:00
Micha Reiser
83daddbeb7 Rename ConfigProcessor to ConfigurationTransformer (#7536) 2023-09-20 14:17:06 +00:00
Micha Reiser
b19eec9b2a Unify Settings and AllSettings (#7532) 2023-09-20 13:56:07 +00:00
Micha Reiser
ca3c15858d Use portable hash function for CacheKey (#7533) 2023-09-20 15:19:59 +02:00
Micha Reiser
bb4f7c681a Rename format option to output-format (#7514) 2023-09-20 15:18:58 +02:00
Dhruv Manilawala
0a167dd20b Use strict sorted and union for NoQA mapping insertion (#7531)
## Summary

This PR fixes the way NoQA range is inserted to the `NoqaMapping`.

Previously, the way the mapping insertion logic worked was as follows:
1. If the range which is to be inserted _touched_ the previous range, meaning
that the end of the previous range was the same as the start of the new
range, then the new range was added in addition to the previous range.
2. Else, if the new range intersected the previous range, then the previous
   range was replaced with the new _intersection_ of the two ranges.

The problem with this logic is that it does not work for the following case:
```python
assert foo, \
    """multi-line
    string"""
```

Now, the comments cannot be added to the same line which ends with a continuation
character. So, the `NoQA` directive has to be added to the next line. But, the
next line is also a triple-quoted string, so the `NoQA` directive for that line
needs to be added to the next line. This creates a **union** pattern instead of an
**intersection** pattern.

But, only union doesn't suffice because (1) means that for the edge case where
the range touch only at the end, the union won't take place.

### Solution

1. Replace '<=' with '<' to have a _strict_ insertion case
2. Use union instead of intersection

## Test Plan

Add a new test case. Run the test suite to ensure that nothing is broken.

### Integration

1. Make a `test.py` file with the following contents:

    ```python
    assert foo, \
        """multi-line
        string"""
    ```

2. Run the following command:

    ```console
	$ cargo run --bin ruff -- check --isolated --no-cache --select=F821 test.py
	/Users/dhruv/playground/ruff/fstring.py:1:8: F821 Undefined name `foo`
    Found 1 error.
    ```

3. Use `--add-noqa`:

    ```console
	$ cargo run --bin ruff -- check --isolated --no-cache --select=F821 --add-noqa test.py
    Added 1 noqa directive.
    ```

4. Check that the NoQA directive was added in the correct position:

    ```python
    assert foo, \
        """multi-line
        string"""  # noqa: F821
    ```

5. Run the `check` command to ensure that the NoQA directive is respected:

    ```console
	$ cargo run --bin ruff -- check --isolated --no-cache --select=F821 test.py
    ```

fixes: #7530
2023-09-20 11:07:19 +00:00
dependabot[bot]
c43542896f Bump unicode-width from 0.1.10 to 0.1.11 (#7534)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-20 10:26:54 +02:00
Micha Reiser
192463c2fb Allow parenthesized content exceed configured line width (#7490) 2023-09-20 08:39:25 +02:00
Charlie Marsh
5849a75223 Rename ruff crate to ruff_linter (#7529) 2023-09-20 08:38:27 +02:00
Mathieu Kniewallner
dcbd8eacd8 feat(rules): implement flake8-bandit S507 (ssh_no_host_key_verification) (#7528)
Part of #1646.

## Summary

Implement `S507`
([`ssh_no_host_key_verification`](https://bandit.readthedocs.io/en/latest/plugins/b507_ssh_no_host_key_verification.html))
rule from `bandit`.

## Test Plan

Snapshot test from
https://github.com/PyCQA/bandit/blob/1.7.5/examples/no_host_key_verification.py,
with several additions to test for more cases (most notably passing the
parameter as a named argument).
2023-09-19 20:31:45 -04:00
Micha Reiser
297ec2c2d2 Use Settings where AllSettings isn't required (#7518) 2023-09-19 23:16:20 +02:00
Tom Kuson
511cc25fc4 [refurb] Implement unnecessary-enumerate (FURB148) (#7454)
## Summary

Implement
[`no-ignored-enumerate-items`](https://github.com/dosisod/refurb/blob/master/refurb/checks/builtin/no_ignored_enumerate.py)
as `unnecessary-enumerate` (`FURB148`).

The auto-fix considers if a `start` argument is passed to the
`enumerate()` function. If only the index is used, then the suggested
fix is to pass the `start` value to the `range()` function. So,

```python
for i, _ in enumerate(xs, 1):
    ...
```

becomes

```python
for i in range(1, len(xs)):
    ...
```

If the index is ignored and only the value is ignored, and if a start
value greater than zero is passed to `enumerate()`, the rule doesn't
produce a suggestion. I couldn't find a unanimously accepted best way to
iterate over a collection whilst skipping the first n elements. The rule
still triggers, however.

Related to #1348.

## Test Plan

`cargo test`

---------

Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2023-09-19 20:19:28 +00:00
Charlie Marsh
4c4eceee36 Add dangling comment handling for lambda expressions (#7493)
## Summary

This PR adds dangling comment handling for `lambda` expressions. In
short, comments around the `lambda` and the `:` are all considered
dangling. Comments that come between the `lambda` and the `:` may be
moved after the colon for simplicity (this is an odd position for a
comment anyway), unless they also precede the lambda parameters, in
which case they're formatted before the parameters.

Closes https://github.com/astral-sh/ruff/issues/7470.

## Test Plan

`cargo test`

No change in similarity.

Before:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 398 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99929 | 648 | 16 |
| zulip | 0.99962 | 1437 | 22 |

After:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 398 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99929 | 648 | 16 |
| zulip | 0.99962 | 1437 | 22 |
2023-09-19 15:23:51 -04:00
Charlie Marsh
e07670ad97 Add dangling comment handling to dictionary key-value pairs (#7495)
## Summary

This PR fixes a formatting instability by changing the comment handling
around the `:` in a dictionary to mirror that of the `:` in a lambda: we
treat comments around the `:` as dangling, then format them after the
`:`.

Closes https://github.com/astral-sh/ruff/issues/7458.

## Test Plan

`cargo test`

No change in similarity.

Before:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1631 |
| django | 0.99983 | 2760 | 36 |
| transformers | 0.99956 | 2587 | 404 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99929 | 648 | 16 |
| zulip | 0.99969 | 1437 | 21 |

After:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1631 |
| django | 0.99983 | 2760 | 36 |
| transformers | 0.99956 | 2587 | 404 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99929 | 648 | 16 |
| zulip | 0.99969 | 1437 | 21 |
2023-09-19 19:17:21 +00:00
Charlie Marsh
b792140579 Ensure that LOG007 only triggers on .exception() calls (#7524)
## Summary

Previously, this rule triggered for `logging.info("...",
exc_info=False)`.
2023-09-19 17:29:54 +00:00
Tom Kuson
36a60bd50e Add documentation to non-empty-stub-body (#7519)
## Summary

Add documentation to `non-empty-stub-body` (`PYI010`) rule. Related to
#2646.

## Test Plan

`python scripts/check_docs_formatted.py`
2023-09-19 13:21:11 -04:00
konsti
4ae463d04b Add a test for stmt assign breaking in preview mode (#7516)
In preview mode, black will consistently break the right side first.
This doesn't work yet, but we'll need the test later.
2023-09-19 16:16:40 +02:00
konsti
6dade5b9ab Tokenizer: Emit only a single bogus token (#7425)
**Summary** Instead of emitting a bogus token per char, we now only emit
on single last bogus token. This leads to much more concise output.

**Test Plan** Updated fixtures
2023-09-19 16:06:03 +02:00
Charlie Marsh
97510c888b Show --no-X variants in CLI help (#7504)
I'd really like this to render as `--preview / --no-preview`, but I
looked for a while in the Clap internals and issue tracker
(https://github.com/clap-rs/clap/issues/815) and I really can't figure
out a way to do it -- this seems like the best we can do? It's also what
they do in Orogene.

Closes https://github.com/astral-sh/ruff/issues/7486.
2023-09-19 12:27:30 +00:00
konsti
94b68f201b Fix stylist indentation with a formfeed (#7489)
**Summary** In python, a formfeed is technically undefined behaviour
(https://docs.python.org/3/reference/lexical_analysis.html#indentation):
> A formfeed character may be present at the start of the line; it will
be ignored for
> the indentation calculations above. Formfeed characters occurring
elsewhere in the
> leading whitespace have an undefined effect (for instance, they may
reset the space
> count to zero).

In practice, they just reset the indentation:


df8b3a46a7/Parser/tokenizer.c (L1819-L1821)
a41bb2733f/crates/ruff_python_parser/src/lexer.rs (L664-L667)

The stylist didn't handle formfeeds previously and would produce invalid
indents. The remedy is to cut everything before a form feed.

Checks box for
https://github.com/astral-sh/ruff/issues/7455#issuecomment-1722458825

**Test Plan** Unit test for the stylist and a regression test for the
rule
2023-09-19 12:01:16 +02:00
konsti
ef34c5cbec Update itertools to 0.11 (#7513)
Preparation for #7469.

Changelog:
https://github.com/rust-itertools/itertools/blob/master/CHANGELOG.md#0110
2023-09-19 09:53:14 +00:00
dependabot[bot]
fdbefd777c Bump syn from 2.0.33 to 2.0.37 (#7512)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-19 11:08:56 +02:00
dependabot[bot]
078547adbb Bump clap from 4.4.3 to 4.4.4 (#7511)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-19 11:08:39 +02:00
dependabot[bot]
42a0bec146 Bump schemars from 0.8.13 to 0.8.15 (#7510)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-19 11:08:22 +02:00
Micha Reiser
37b7d0f921 fix: Compiler warning about unused map_or (#7508) 2023-09-19 08:10:01 +00:00
Micha Reiser
6a4dbd622b Add optimized best_fit_parenthesize IR (#7475) 2023-09-19 06:29:05 +00:00
Charlie Marsh
28b48ab902 Avoid flagging starred expressions in UP007 (#7505)
## Summary

These can't be fixed, because fixing them would lead to invalid syntax.
So flagging them also feels misleading.

Closes https://github.com/astral-sh/ruff/issues/7452.
2023-09-19 03:37:38 +00:00
Valeriy Savchenko
4123d074bd [refurb] Implement reimplemented-starmap rule (FURB140) (#7253)
## Summary

This PR is part of a bigger effort of re-implementing `refurb` rules
#1348. It adds support for
[FURB140](https://github.com/dosisod/refurb/blob/master/refurb/checks/itertools/use_starmap.py)

## Test Plan

I included a new test + checked that all other tests pass.
2023-09-19 02:18:54 +00:00
Mathieu Kniewallner
c6ba7dfbc6 feat(rules): implement flake8-bandit S201 (flask_debug_true) (#7503)
Part of #1646.

## Summary

Implement `S201`
([`flask_debug_true`](https://bandit.readthedocs.io/en/latest/plugins/b201_flask_debug_true.html))
rule from `bandit`.

I am fairly new to Rust and Ruff's codebase, so there might be better
ways to implement the rule or write the code.

## Test Plan

Snapshot test from
https://github.com/PyCQA/bandit/blob/1.7.5/examples/flask_debug.py, with
a few additions in the "unrelated" part to test a bit more cases.
2023-09-19 00:43:28 +00:00
Micael Jarniac
40f6456add Use MkDocs' not_in_nav (#5498)
Closes #5497
Needs MkDocs 1.5 to be released.
- [x] https://github.com/mkdocs/mkdocs/milestone/15

## Summary
Uses MkDocs' `not_in_nav` config to hide spam about files in
`docs/rules/` not being in nav.
2023-09-19 00:01:43 +00:00
Micha Reiser
3e1dffab20 refactor: Use OnceCell in Memoized (#7500) 2023-09-18 19:58:55 +00:00
Micha Reiser
3336d23f48 perf: Reduce best fitting allocations (#7411) 2023-09-18 19:49:44 +00:00
Jonathan Plasse
2421805033 Avoid N802 violations for @overload methods (#7498)
Close #7479

The `@override` was already implemented

## Test Plan

Tested the code in the issue. After removing all the noqa's, only one
occurrence of `BadName()` raised a violation.
Added a fixture
2023-09-18 14:32:40 -04:00
Tom Kuson
359f50e6dc Ignore pass-statement-stub-body documentation formatting (#7497)
## Summary

Fix CI (broken in #7496).

The code snippet was formatted as Black would format a stub file, but
the CI script doesn't know that (it assumes all code snippets are
non-stub files). Easier to ignore.

Sorry for breaking CI!

## Test Plan

`python scripts/check_docs_formatted.py`
2023-09-18 14:27:20 -04:00
qdegraaf
bccba5d73f [flake8-logging] Implement LOG007: ExceptionWithoutExcInfo (#7410)
## Summary

This PR implements a new rule for `flake8-logging` plugin that checks
for uses of `logging.exception()` with `exc_info` set to `False` or a
falsy value. It suggests using `logging.error` in these cases instead.

I am unsure about the name. Open to suggestions there, went with the
most explicit name I could think of in the meantime.

Refer https://github.com/astral-sh/ruff/issues/7248

## Test Plan

Added a new fixture cases and ran `cargo test`
2023-09-18 17:46:12 +00:00
Tom Kuson
0bfdb15ecf Add documentation to pass-statement-stub-body (#7496)
## Summary

Add documentation to `pass-statement-stub-body` (`PYI009`) rule. Related
to #2646.

## Test Plan

`python scripts/check_docs_formatted.py`
2023-09-18 17:33:15 +00:00
dependabot[bot]
a902d14c31 Bump chrono from 0.4.30 to 0.4.31 (#7481) 2023-09-18 13:11:51 -04:00
Charlie Marsh
728539291f Move FormatExprDict to top of expr_dict.rs (#7494)
Put the node itself up top, and internal structs down below.
2023-09-18 11:55:18 -04:00
Dhruv Manilawala
c2bd8af59a Remove triple-quoted string ranges computation (#7476)
## Summary

This is a follow-up PR for #7435 to remove the now unused triple-quoted
string ranges from the indexer.
2023-09-18 20:57:49 +05:30
Jaap Roes
c946bf157e Extend bad-dunder-method-name to permit __html__ (#7492)
## Summary

Fixes #7478

## Test Plan

`cargo test`
2023-09-18 15:16:22 +00:00
Charlie Marsh
8ab2519717 Respect parentheses for precedence in await (#7468)
## Summary

We were using `Parenthesize::IfBreaks` universally for `await`, but
dropping parentheses can change the AST due to precedence. It turns out
that Black's rules aren't _exactly_ the same as operator precedence
(e.g., they leave parentheses around `await ([1, 2, 3])`, although they
aren't strictly required).

Closes https://github.com/astral-sh/ruff/issues/7467.

## Test Plan

`cargo test`

No change in similarity.

Before:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 398 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99929 | 648 | 16 |
| zulip | 0.99962 | 1437 | 22 |

After:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 398 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99929 | 648 | 16 |
| zulip | 0.99962 | 1437 | 22 |
2023-09-18 09:56:41 -04:00
konsti
c4d85d6fb6 Fix ''' ""''' formatting (#7485)
## Summary

`''' ""'''` is an edge case that was previously incorrectly formatted as
`""" """""`.

Fixes #7460

## Test Plan

Added regression test
2023-09-18 10:28:15 +00:00
dependabot[bot]
70ea49bf72 Bump test-case from 3.1.0 to 3.2.1 (#7484)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-18 11:31:41 +02:00
dependabot[bot]
8e255974bc Bump unicode-ident from 1.0.11 to 1.0.12 (#7482)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-18 11:31:06 +02:00
dependabot[bot]
d358604464 Bump serde_json from 1.0.106 to 1.0.107 (#7480)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-18 11:30:06 +02:00
dependabot[bot]
8243db74fe Bump indoc from 2.0.3 to 2.0.4 (#7483)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-18 11:29:50 +02:00
Micha Reiser
0346e781d4 Fix handling of newlines in empty files (#7473) 2023-09-18 06:08:10 +00:00
Tom Kuson
b66bfa6570 Extend bad-dunder-method-name to permit attrs dunders (#7472)
## Summary

Closes #7451.

## Test Plan

`cargo test`
2023-09-17 16:13:33 -04:00
Charlie Marsh
28273eb00b Avoid flagging starred elements in C402 (#7466)
## Summary

Rewriting these is not valid syntax.

Part of https://github.com/astral-sh/ruff/issues/7455.
2023-09-17 15:23:27 +00:00
Charlie Marsh
12acd191e1 Remove parentheses when rewriting assert calls to statements (#7464) 2023-09-17 15:18:56 +00:00
Charlie Marsh
64b929bc29 Add padding to prevent some autofix errors (#7461)
## Summary

We should really be doing this anywhere we _replace_ an expression.

See: https://github.com/astral-sh/ruff/issues/7455.
2023-09-17 15:14:16 +00:00
Micha Reiser
26ae0a6e8d Fix dangling module comments (#7456) 2023-09-17 14:56:41 +00:00
Dhruv Manilawala
959338d39d Refactor tab-indentation as a token-based rule (#7435)
## Summary

This PR updates the `W191` (`tab-indentation`) rule from a line-based to
a token-based rule.

Earlier, the rule used the `triple_quoted_string_ranges` from the
indexer to skip over any lines _inside_ a triple-quoted string. This was the only
use of the ranges. These ranges were extracted through the tokens, so instead
we can directly use the newline tokens to perform the check.

This would also mean that we can remove the `triple_quoted_string_ranges` from
the indexer but I'll hold that off until we have a better idea on #7326
but I don't think it would be a problem to remove it.

This will also fix #7379 once PEP 701 changes are merged.

## Test Plan

`cargo test`
2023-09-16 20:25:20 +00:00
Charlie Marsh
422ff82f4a Avoid extra parentheses in yield expressions (#7444)
## Summary

This is equivalent to https://github.com/astral-sh/ruff/pull/7424, but
for `yield` and `yield from` expressions. Specifically, we want to avoid
adding unnecessary extra parentheses for `yield expr` when `expr` itself
does not require parentheses.

## Test Plan

`cargo test`

No change in any of the similarity metrics.

Before:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 399 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99929 | 648 | 16 |
| zulip | 0.99962 | 1437 | 22 |

After:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 399 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99929 | 648 | 16 |
| zulip | 0.99962 | 1437 | 22 |
2023-09-16 14:46:56 -04:00
Charlie Marsh
8d0a5e01bd Modify comment_ranges slice in BackwardsTokenizer (#7432)
## Summary

I was kinda curious to understand this issue
(https://github.com/astral-sh/ruff/issues/7426) and just ended up
attempting to address it.

## Test Plan

`cargo test`
2023-09-16 14:04:45 -04:00
Charlie Marsh
aae02cf275 Fix broken is_expression_parenthesized call from rebase (#7442) 2023-09-16 17:22:16 +00:00
Charlie Marsh
7e2eba2592 Avoiding grouping comprehension targets separately from conditions (#7429)
## Summary

Black seems to treat comprehension targets and conditions as if they're
in a single group -- so if the comprehension expands, all conditions are
rendered on their own line, etc.

Closes https://github.com/astral-sh/ruff/issues/7421.

## Test Plan

`cargo test`

No change in any of the similarity metrics.

Before:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 399 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99923 | 648 | 18 |
| zulip | 0.99962 | 1437 | 22 |

After:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 399 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99923 | 648 | 18 |
| zulip | 0.99962 | 1437 | 22 |
2023-09-16 17:19:34 +00:00
Charlie Marsh
22770fb4be Avoid extra parentheses in await expressions (#7424)
## Summary

This PR aligns the await parenthesizing with the unary case, which is:
if the value is already parenthesized, avoid parenthesizing; otherwise,
only parenthesize if the _value_ needs parenthesizing.

Closes https://github.com/astral-sh/ruff/issues/7420.

## Test Plan

`cargo test`

No change in similarity.

Before:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 399 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99923 | 648 | 18 |
| zulip | 0.99962 | 1437 | 22 |

After:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 399 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99923 | 648 | 18 |
| zulip | 0.99962 | 1437 | 22 |
2023-09-16 13:10:35 -04:00
Charlie Marsh
1880cceac1 Avoid extra parentheses in unary expressions (#7428)
## Summary

This PR applies a similar fix to unary expressions as in
https://github.com/astral-sh/ruff/pull/7424. Specifically, we only need
to parenthesize the entire operator if the operand itself doesn't have
parentheses, and requires parentheses.

Closes https://github.com/astral-sh/ruff/issues/7423.

## Test Plan

`cargo test`

No change in similarity.

Before:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 399 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99923 | 648 | 18 |
| zulip | 0.99962 | 1437 | 22 |

After:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 399 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99923 | 648 | 18 |
| zulip | 0.99962 | 1437 | 22 |
2023-09-16 13:07:38 -04:00
Dhruv Manilawala
0d1fb823d6 [flake8-logging] Implement LOG002: invalid-get-logger-argument (#7399)
## Summary

This PR implements a new rule for `flake8-logging` plugin that checks
for
`logging.getLogger` calls with either `__file__` or `__cached__` as the
first
argument and generates a suggested fix to use `__name__` instead.

Refer: #7248

## Test Plan

Add test cases and `cargo test`
2023-09-16 12:21:30 -04:00
Micha Reiser
c907317199 Fix build (#7437) 2023-09-16 14:50:36 +00:00
Micha Reiser
916dd5b7fa fix: Use BestFit layout for subscript (#7409) 2023-09-16 16:21:45 +02:00
konsti
2cbe1733c8 Use CommentRanges in backwards lexing (#7360)
## Summary

The tokenizer was split into a forward and a backwards tokenizer. The
backwards tokenizer uses the same names as the forwards ones (e.g.
`next_token`). The backwards tokenizer gets the comment ranges that we
already built to skip comments.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2023-09-16 03:21:45 +00:00
Charlie Marsh
1f6e1485f9 Change playground to use pages deploy (#7430)
Fixes a deprecation warning.
2023-09-16 03:11:34 +00:00
Charlie Marsh
9b43162cc4 Move documentation to docs.astral.sh/ruff (#7419)
## Summary

We're planning to move the documentation from
[https://beta.ruff.rs/docs](https://beta.ruff.rs/docs) to
[https://docs.astral.sh/ruff](https://docs.astral.sh/ruff), for a few
reasons:

1. We want to remove the `beta` from the domain, as Ruff is no longer
considered beta software.
2. We want to migrate to a structure that could accommodate multiple
future tools living under one domain.

The docs are actually already live at
[https://docs.astral.sh/ruff](https://docs.astral.sh/ruff), but later
today, I'll add a permanent redirect from the previous to the new
domain. **All existing links will continue to work, now and in
perpetuity.**

This PR contains the code changes necessary for the updated
documentation. As part of this effort, I moved the playground and
documentation from my personal Cloudflare account to our team Cloudflare
account (hence the new `--project-name` references). After merging, I'll
also update the secrets on this repo.
2023-09-15 22:49:42 -04:00
Charlie Marsh
cc9e84c144 Format trailing operator comments as dangling (#7427)
## Summary

Given a trailing operator comment in a unary expression, like:

```python
if (
  not  # comment
  a):
    ...
```

We were attaching these to the operand (`a`), but formatting them in the
unary operator via special handling. Parents shouldn't format the
comments of their children, so this instead attaches them as dangling
comments on the unary expression. (No intended change in formatting.)
2023-09-15 20:34:09 -04:00
Zanie Blue
f4d50a2aec Fix release validation step (#7417)
Proof of concept at #7416
Fixes `main` branch check added in #7279 (see
[failure](https://github.com/astral-sh/ruff/actions/runs/6201772425/job/16839150669))
Removes the meaningless "SHA consistency" check (since we literally
check out the SHA now)
2023-09-15 20:18:25 +00:00
Zanie Blue
0c030b5bf3 Bump version to 0.0.290 (#7413)
See also:
- https://github.com/astral-sh/astral-sh/pull/41
- https://github.com/astral-sh/ruff-pre-commit/pull/51
2023-09-15 13:51:46 -05:00
Zanie Blue
1b082ce67e Add maximum length for line-length to JSON schema (#7412)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->
Adds the maximum of 320 for the line-length setting to the JSON schema
for better integration with IDEs.

Related https://github.com/astral-sh/ruff/pull/6873

## Test Plan

<!-- How was it tested? -->
2023-09-15 18:10:06 +00:00
dependabot[bot]
f936d319cc Bump toml from 0.7.6 to 0.7.8 (#7405)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-15 09:14:58 +00:00
dependabot[bot]
85d8b6228f Bump chrono from 0.4.28 to 0.4.30 (#7406)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-15 09:00:17 +00:00
dependabot[bot]
7594dadc1d Bump syn from 2.0.29 to 2.0.33 (#7404)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-15 08:49:52 +00:00
dependabot[bot]
de37fbfac9 Bump serde-wasm-bindgen from 0.5.0 to 0.6.0 (#7403)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-15 08:47:42 +00:00
dependabot[bot]
4e2769a16c Bump mimalloc from 0.1.38 to 0.1.39 (#7402)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-15 08:46:55 +00:00
Manuel Jacob
75b5c314e3 Change CWE reference in documentation for S607 rule (#7398)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

The previous reference was “CWE-78: Improper Neutralization of Special
Elements used in an OS Command ('OS Command Injection')”, which
describes another issue. The new reference is “CWE-426: Untrusted Search
Path”, which describes exactly the problem that this rule should warn
about.

## Test Plan

The change was not tested, as it only changes two numbers in the
documentation.
2023-09-14 23:12:54 -05:00
Charlie Marsh
450fb9b99a [flake8-logging] Implement LOG001: direct-logger-instantiation (#7397)
## Summary

See: https://github.com/astral-sh/ruff/issues/7248.
2023-09-14 23:07:10 -04:00
Charlie Marsh
6163c99551 Mark PERF403 as a preview rule (#7396) 2023-09-15 01:57:48 +00:00
qdegraaf
3112202a5b [flake8-logging] Add flake8_logging boilerplate and first rule LOG009 (#7249)
## Summary

Adds `LOG009` from
[flake8-logging](https://github.com/adamchainz/flake8-logging). Also
adds the boilerplate for a new plugin

Checks for usages of undocumented `logging.WARN` constant and suggests
replacement with `logging.WARNING`.

## Test Plan

`cargo test` with fresh fixture

## Issue links

Refers: https://github.com/astral-sh/ruff/issues/7248
2023-09-15 01:41:32 +00:00
Charlie Marsh
64ea00048b Add known problems to UP040 documentation (#7395) 2023-09-15 01:31:20 +00:00
qdegraaf
067a4acd54 [perflint] Add PERF403 (#6132)
## Summary

Adds `PERF403` mirroring `W8403` from
https://github.com/tonybaloney/perflint

## Test Plan

Fixtures were added based on perflint tests

## Issue Links

Refers: https://github.com/astral-sh/ruff/issues/4789
2023-09-15 01:23:51 +00:00
Nathan Whitaker
c88376f468 Add support for bounds, constraints, and explicit variance on generic type variables to UP040 (#6749)
## Summary

Extends UP040 to support moving type variables with
bounds/constraints/variance that are used in type aliases to use PEP-695
syntax.

Part of #4617.

## Test Plan

The existing tests added by #6314 already cover the relevant cases.
2023-09-14 21:11:24 -04:00
Charlie Marsh
9b7c29853d Reflect conversion reason in UP012 messages (#7393)
Closes https://github.com/astral-sh/ruff/issues/7254.
2023-09-14 20:08:58 +00:00
Charlie Marsh
3e21d32b79 Avoid flagging single-quoted docstrings with continuations for multi-line rules (#7392)
Rules like D209 and D205 are only intended to apply to multi-line
docstrings. If a docstring is single-quoted, but extends via a
continuation, it should be excluded (it'll be flagged by another rule
anyway). Closes https://github.com/astral-sh/ruff/issues/7058.
2023-09-14 18:58:39 +00:00
Charlie Marsh
f9e3ea23ba Show rule codes in shell tab completion (#7375)
## Summary

I noticed that we have a custom parser for rule selectors, but it wasn't
actually being used? This PR adds it back to our Clap setup and changes
the parser to only show full categories and individual rules when
tab-completing:

<img width="1792" alt="Screen Shot 2023-09-13 at 9 13 38 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/028b18d2-8c92-49c1-b781-f24c9ae310f7">

<img width="1792" alt="Screen Shot 2023-09-13 at 9 13 40 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/fd598da5-78fb-412d-a69e-2a0963d479cd">

<img width="1792" alt="Screen Shot 2023-09-13 at 9 13 58 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/7c482b90-6e54-425c-ae23-fb50496a177a">

The previous implementation showed all codes, which I found too noisy:

<img width="1792" alt="Screen Shot 2023-09-13 at 8 57 09 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/db370a0e-2a9f-4acd-b1e3-224a1f8e9ce5">
2023-09-14 18:37:23 +00:00
Charlie Marsh
6856d0b44b Use dot references in docs for methods (#7391)
## Summary

This matches the convention used in the Python documentation.
2023-09-14 14:35:34 -04:00
Charlie Marsh
21539f1663 Allow NURSERY in JSON Schema (#7374)
## Summary

At some point, we removed these so that they wouldn't be autocompleted
for users, since we wanted to discourage usage of `ALL`. But given that
they're valid values, I think that was a bad idea -- it leads to an even
more confusing experience whereby JSON Schema validators tell you that
you have an error, when you don't.

Closes https://github.com/astral-sh/ruff/issues/7261.
2023-09-14 18:09:35 +00:00
Zanie Blue
b9bb6bf780 Remove the PREVIEW rule selector (#7389)
The rule selector is not useful because `--select PREVIEW` only targets
Ruff developers and `--ignore PREVIEW` has no effect due to its low
specificity. We may restore it later if useful.
2023-09-14 12:31:59 -05:00
Charlie Marsh
d39eae2713 Extend C416 to catch tuple unpacking (#7363)
Closes https://github.com/astral-sh/ruff/issues/7307.
2023-09-14 15:53:15 +00:00
Charlie Marsh
5d21b9c22e Catch panics in formatter (#7377)
## Summary

This PR ensures that we catch and render panics in the formatter
identically to other kinds of errors. It also improves the consistency
in error rendering throughout and makes a few stylistic changes to the
messages.

Closes https://github.com/astral-sh/ruff/issues/7247.

## Test Plan

I created a file `foo.py` with a syntax error, and a file `bar.py` with
an intentional panic.

<img width="1624" alt="Screen Shot 2023-09-13 at 10 25 22 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/605c2839-ad02-4376-a2e9-d5a593ab660f">

<img width="1624" alt="Screen Shot 2023-09-13 at 10 25 24 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/b1381909-157c-48cb-9630-d0bbfcb1b640">
2023-09-14 11:44:16 -04:00
Charlie Marsh
ec2f229a45 Remove ExprContext from ComparableExpr (#7362)
`ComparableExpr` includes the `ExprContext` field on an expression, so,
e.g., the two tuples in `(a, b) = (a, b)` won't be considered equal.
Similarly, the tuples in `[(a, b) for (a, b) in c]` _also_ wouldn't be
considered equal. I find this behavior surprising, since
`ComparableExpr` is intended to allow you to compare two ASTs, but
`ExprContext` is really encoding information about the broader context
for the expression.
2023-09-14 15:40:02 +00:00
Charlie Marsh
34c1cb7d11 Treat parenthesized power operands as non-simple (#7371)
Closes https://github.com/astral-sh/ruff/issues/7318.
2023-09-14 15:36:21 +00:00
dependabot[bot]
2ddea7c657 Bump regex from 1.9.4 to 1.9.5 (#7384)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-14 14:58:24 +00:00
dependabot[bot]
45eabdd2c3 Bump shlex from 1.1.0 to 1.2.0 (#7381)
Bumps [shlex](https://github.com/comex/rust-shlex) from 1.1.0 to 1.2.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/comex/rust-shlex/blob/master/CHANGELOG.md">shlex's
changelog</a>.</em></p>
<blockquote>
<h1>1.2.0</h1>
<ul>
<li>Adds <code>bytes</code> module to support operating directly on byte
strings.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/comex/rust-shlex/commits">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=shlex&package-manager=cargo&previous-version=1.1.0&new-version=1.2.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Micha Reiser <micha@reiser.io>
2023-09-14 09:40:05 -05:00
Micha Reiser
675c86c175 fix: Group fluent subscript (#7386) 2023-09-14 13:04:14 +02:00
dependabot[bot]
1f8e2b8f14 Bump proc-macro2 from 1.0.66 to 1.0.67 (#7383)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-14 09:49:57 +00:00
Micha Reiser
58b3040342 chore: Upgrade notify (#7338) 2023-09-14 09:47:55 +00:00
dependabot[bot]
6a9b8aede1 Bump thiserror from 1.0.47 to 1.0.48 (#7385)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-14 09:21:47 +00:00
dependabot[bot]
f48126ad00 Bump clap from 4.4.1 to 4.4.3 (#7382)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-14 09:13:12 +00:00
Charlie Marsh
11287f944f Avoid re-parenthesizing call chains whose inner values are parenthesized (#7373)
## Summary

Given a statement like:

```python
result = (
    f(111111111111111111111111111111111111111111111111111111111111111111111111111111111)
    + 1
)()
```

When we go to parenthesize the target of the assignment, we use
`maybe_parenthesize_expression` with `Parenthesize::IfBreaks`. This then
checks if the call on the right-hand side needs to be parenthesized, the
implementation of which looks like:

```rust
impl NeedsParentheses for ExprCall {
    fn needs_parentheses(
        &self,
        _parent: AnyNodeRef,
        context: &PyFormatContext,
    ) -> OptionalParentheses {
        if CallChainLayout::from_expression(self.into(), context.source())
            == CallChainLayout::Fluent
        {
            OptionalParentheses::Multiline
        } else if context.comments().has_dangling(self) {
            OptionalParentheses::Always
        } else {
            self.func.needs_parentheses(self.into(), context)
        }
    }
}
```

Checking for `self.func.needs_parentheses(self.into(), context)` is
problematic, since, as in the example above, `self.func` may _already_
be parenthesized -- in which case, we _don't_ want to parenthesize the
entire expression. If we do, we end up with this non-ideal formatting:

```python
result = (
    (
        f(
            111111111111111111111111111111111111111111111111111111111111111111111111111111111
        )
        + 1
    )()
)
```

This PR modifies the `NeedsParentheses` implementations for call chain
expressions to return `Never` if the inner expression has its own
parentheses, in which case, the formatting implementations for those
expressions will preserve them anyway.

Closes https://github.com/astral-sh/ruff/issues/7370.

## Test Plan

Zulip improves a bit, everything else is unchanged.

Before:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99981 | 2760 | 40 |
| transformers | 0.99944 | 2587 | 413 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99834 | 648 | 20 |
| zulip | 0.99956 | 1437 | 23 |

After:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99981 | 2760 | 40 |
| transformers | 0.99944 | 2587 | 413 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99834 | 648 | 20 |
| **zulip** | **0.99962** | **1437** | **22** |
2023-09-14 05:05:37 -04:00
Micha Reiser
a65efcf459 fix: Don't omit optional parentheses for subscripts (#7380) 2023-09-14 08:43:53 +00:00
Jelle van der Waa
04183b0299 [pylint] Implement too-many-public-methods rule (PLR0904) (#6179)
Implement
https://pylint.pycqa.org/en/latest/user_guide/messages/refactor/too-many-public-methods.html

Confusingly the rule page mentions a max of 7 while in practice it is
20.
https://github.com/search?q=repo%3Apylint-dev%2Fpylint+max-public-methods&type=code

## Summary

Implement pylint's R0904

## Test Plan

Unit tests.
2023-09-14 00:52:26 +00:00
James Braza
36fa1fe359 Docs linking error tutorial with error suppression (#7014)
Documents takeaway from
https://github.com/astral-sh/ruff/discussions/7011#discussioncomment-6869239.
2023-09-13 20:22:18 -04:00
Charlie Marsh
6e625bd93d Invert reverse argument regardless of whether it's a boolean (#7372)
## Summary

When fixing `reversed(sorted(x, reverse=False))`, we rewrite as
`sorted(x, reverse=True)`. However, if the `reverse` argument isn't
`True` or `False`, we leave it as-is, which is incorrect.

Now, given `reversed(sorted(x, reverse=y))`, we rewrite as `sorted(x,
reverse=not y)`.
2023-09-13 20:12:35 -04:00
Zanie Blue
ebd1b296fd Add warnings for nursery and preview rule selection (#7210)
## Summary

Adds warnings for cases where:
- A selector does not include any rules because preview is disabled
- A nursery rule is selected without the preview flag

## Test plan

Add integration tests
2023-09-13 15:29:58 -05:00
Zanie Blue
1373e1c395 Update release workflow to checkout the given sha (#7279) 2023-09-13 14:59:41 -05:00
Zanie Blue
4bff397318 Move FURB145 from nursery to preview (#7364)
Moves the new rule from nursery to preview for the upcoming release.

Adds new test coverage for selection of a single preview rule and fixes
a bug where preview rules were incorrectly selectable with exact codes.
2023-09-13 14:54:28 -05:00
Charlie Marsh
5347df4728 Parenthesize single-generator arguments when adding reverse keyword (#7365)
Closes https://github.com/astral-sh/ruff/issues/7289.
2023-09-13 19:40:45 +00:00
Tom Kuson
ebe9c03545 [refurb] Implement no-slice-copy (FURB145) (#7007)
## Summary

Implement
[`no-slice-copy`](https://github.com/dosisod/refurb/blob/master/refurb/checks/builtin/no_slice_copy.py)
as `slice-copy` (`FURB145`).

Related to #1348.

## Test Plan

`cargo test`
2023-09-13 17:31:15 +00:00
Charlie Marsh
b0cbcd3dfa Update deprecated-import lists based on recent typing-extension release (#7356)
## Summary

Generated by running
f3cff244e3/testing/generate-typing-rewrite-info (L14)
with latest `typing-extensions` and manually applying the changes.

Closes https://github.com/astral-sh/ruff/issues/7324.
2023-09-13 13:21:11 -04:00
Charlie Marsh
4df9e07a79 Add a benchmarking script for the formatter CLI (#7340)
## Summary

This PR adds a benchmarking script for the formatter, which benchmarks
the Ruff formatter against Black, yapf, and autopep8.

Three benchmarks are included:

1. Format everything.
2. Format everything, but use a single thread.
3. Format everything, but `--check` (don't write to disk).

There's some nuance in figuring out the right combination of arguments
to each command, but the _main_ nuance is to ensure that we always run
the given formatter (and modify the target repo in-place) prior to
benchmarking it, so that the formatters aren't disadvantaged by the
existing formatting of the target repo. (E.g.: prior to benchmarking
Black's preview style, we need to make sure we format the target repo
with Black's preview style; otherwise, preview style appears much
slower.)

Part of https://github.com/astral-sh/ruff/issues/7309.
2023-09-13 13:15:48 -04:00
Charlie Marsh
f0f7ea7502 Treat whitespace-only line as blank for D411 (#7351)
This better aligns with the definition of "blank line" that we use
throughout the docstring rules.

Closes https://github.com/astral-sh/ruff/issues/7216.
2023-09-13 16:41:12 +00:00
Micha Reiser
4f26002dd5 chore: Upgrade strum (#7337)
## Summary

This PR upgrades `strum` from 0.24.x to 0.25.x. 

The breaking changes are: 
* strum macros now uses syn2
* The `to_string` behavior changed when using `default`. I did a quick search, we aren't using `strum(default)` 


`strum` now has a `#[derive(EnumIs)]` macro that generates `is_` methods. 

## Test Plan

cargo test
2023-09-13 18:33:27 +02:00
dependabot[bot]
d1a9c198e3 Bump path-absolutize from 3.1.0 to 3.1.1 (#7345)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-13 16:21:51 +00:00
dependabot[bot]
7a4f699fba Bump argfile from 0.1.5 to 0.1.6 (#7344)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-13 16:18:21 +00:00
dependabot[bot]
3fb5418c2c Bump memchr from 2.6.2 to 2.6.3 (#7343)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-13 18:14:42 +02:00
dependabot[bot]
9fcc009a0c Bump serde_json from 1.0.105 to 1.0.106 (#7342)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-13 18:13:24 +02:00
Micha Reiser
bf8e5a167b chore: Upgrade walkdir (#7336)
## Summary

The only commit is [api: add follow_root_links() option to WalkDir](dcc527d832) whicih addsa  new option wheter `walkdir` should follow a root symlink or not. 
The new option defaults to `true` which is the same as before. 

## Test Plan

`cargo test`
2023-09-13 18:07:24 +02:00
Micha Reiser
8a001dfc3d chore: Upgrade pyproject-toml crate (#7335)
## Summary

This PR bumps the pyproject-toml crate to 0.7.0. The only difference is that it now depends on indexmap 2. I reviewed the indexmap 2 changes and they don't seem relevant to us. 

I used this opportunity to remove the default features from `serde_with` which removes our indexmap 1 dependency (and some other unused dependencies)

## Test Plan

`cargo test`
2023-09-13 17:55:03 +02:00
Zanie Blue
0823394525 Display nursery rules as preview in documentation (#7341)
This is broken in the last release
2023-09-13 10:46:43 -05:00
Zanie Blue
e15047815c Update Rust toolchain file to use TOML format (#7339)
Ref https://rust-lang.github.io/rustup/overrides.html#the-toolchain-file
Should resolve Dependabot failure at
https://github.com/astral-sh/ruff/network/updates/721380342
Follows #7034
2023-09-13 15:43:03 +00:00
Micha Reiser
7531bb3b21 Upgrade is-macros to 0.3.0 (#7334) 2023-09-13 15:27:40 +02:00
Micha Reiser
2d9b39871f Introduce IndentWidth (#7301) 2023-09-13 14:52:24 +02:00
Micha Reiser
e122a96d27 playground: Respect line-length and preview configuration (#7330) 2023-09-13 12:14:25 +00:00
konsti
f4c7bff36b Don't reorder parameters in function calls (#7268)
## Summary

In `f(*args, a=b, *args2, **kwargs)` the args (`*args`, `*args2`) and
keywords (`a=b`, `**kwargs`) are interleaved, which we previously didn't
handle.

Fixes #6498

**main**

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| **django** | 0.99966 | 2760 | 58 |
| transformers | 0.99930 | 2587 | 447 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99825 | 648 | 22 |
| zulip | 0.99950 | 1437 | 27 |

**PR**

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| **django** | 0.99967 | 2760 | 53 |
| transformers | 0.99930 | 2587 | 447 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99825 | 648 | 22 |
| zulip | 0.99950 | 1437 | 27 |


## Test Plan

New fixtures
2023-09-13 09:01:49 +00:00
konsti
56440ad835 Introduce ArgOrKeyword to keep call parameter order (#7302)
## Motivation

The `ast::Arguments` for call argument are split into positional
arguments (args) and keywords arguments (keywords). We currently assume
that call consists of first args and then keywords, which is generally
the case, but not always:

```python
f(*args, a=2, *args2, **kwargs)

class A(*args, a=2, *args2, **kwargs):
    pass
```

The consequence is accidentally reordering arguments
(https://github.com/astral-sh/ruff/pull/7268).

## Summary

`Arguments::args_and_keywords` returns an iterator of an `ArgOrKeyword`
enum that yields args and keywords in the correct order. I've fixed the
obvious `args` and `keywords` usages, but there might be some cases with
wrong assumptions remaining.

## Test Plan

The generator got new test cases, otherwise the stacked PR
(https://github.com/astral-sh/ruff/pull/7268) which uncovered this.
2023-09-13 08:45:46 +00:00
Charlie Marsh
179128dc54 Link discussion in formatter README (#7311) 2023-09-12 16:50:22 +00:00
Charlie Marsh
e7a2779402 Bump version to v0.0.289 (#7308) 2023-09-12 12:00:11 -04:00
Zanie Blue
008da95b29 Add preview documentation section (#7281)
Adds a basic documentation section for preview mode based on the FAQ
entry and versioning RFC.
2023-09-12 15:43:31 +00:00
Zanie Blue
5d4dd3e38e Set the target deployment to main during dispatched documentation deployments (#7304)
Closes #7276 by deploying to production when not triggered by a pull
request.
2023-09-12 10:34:05 -05:00
Micha Reiser
e561f5783b Fix(vscode): Respect line length ruff.toml configuration (#7306) 2023-09-12 15:31:47 +00:00
Dhruv Manilawala
ee0f1270cf Add NotebookIndex to the cache (#6863)
## Summary

This PR updates the `FileCache` to include an optional `NotebookIndex`
to support caching for Jupyter Notebooks.

We only require the index to compute the diagnostics and thus we don't
really need to store the entire `Notebook` on the `Diagnostics` struct.
This means we only need the index to be stored in the cache to
reconstruct the `Diagnostics`.

## Test Plan

Update an existing test case to run over the fixtures under
`ruff_notebook` crate where there are multiple Jupyter Notebook.

Locally, the following commands were run in order:
1. Remove the cache: `rm -rf .ruff_cache`
2. Run without cache: `cargo run --bin ruff -- check --isolated
crates/ruff_notebook/resources/test/fixtures/jupyter/unused_variable.ipynb
--no-cache`
3. Run with cache: `cargo run --bin ruff -- check --isolated
crates/ruff_notebook/resources/test/fixtures/jupyter/unused_variable.ipynb`
4. Check whether the `.ruff_cache` directory was created or not
5. Run with cache again and verify: `cargo run --bin ruff -- check
--isolated
crates/ruff_notebook/resources/test/fixtures/jupyter/unused_variable.ipynb`

## Benchmarks

https://github.com/astral-sh/ruff/pull/6863#issuecomment-1715675186

fixes: #6671
2023-09-12 18:29:03 +05:30
Tom Kuson
e7b7e4a18d Add documentation to duplicate-union-member (#7225)
## Summary

Add documentation to `duplicate-union-member` (`PYI016`) rule. Related
to #2646.

## Test Plan

`python scripts/check_docs_formatted.py`
2023-09-12 08:56:33 -04:00
Brendon Happ
b4419c34ea Ignore @override method when enforcing bad-dunder-name rule (#7224)
## Summary

Closes #6958.

If a method has the `override` decorator, there is nothing you can do
about incorrect dunder methods, so they should be ignored.

## Test Plan

Overridden incorrect dunder method was added to the tests to verify ruff
doesn't catch it when evaluating the file. Snapshot changes are all just
line number changes
2023-09-12 11:54:40 +00:00
Micha Reiser
08f19226b9 Fix panic when formatting binary expression with two implicit concatenated string operands (#7287) 2023-09-12 09:49:51 +02:00
Micha Reiser
1e6df19a35 Bool expression comment placement (#7269) 2023-09-12 06:39:57 +00:00
Zanie Blue
c21b960fc7 Display the --preview option in the CLI help menu (#7274)
If we're going to warn on use of NURSERY in #7210 we probably ought to
show the `--preview` option in our help menus.
2023-09-11 18:09:58 -05:00
Zanie Blue
73ad2affa1 Update preview and fix documentation symbols (#7207)
I don't love the sunrise emoji and 🧪 seems nice :)

Requires #7195

---------

Co-authored-by: konsti <konstin@mailbox.org>
2023-09-11 18:08:00 -05:00
Zanie Blue
40c936922e Add "Preview" section to auto-generated release notes (#7280) 2023-09-11 18:07:47 -05:00
Charlie Marsh
874db4fb86 Invert condition for < and <= in outdated version block (#7284)
Closes https://github.com/astral-sh/ruff/issues/7258.
2023-09-11 23:02:23 +00:00
Dhruv Manilawala
a41bb2733f Add range to lexer test snapshots (#7265)
## Summary

This PR updates the lexer test snapshots to include the range value as
well. This is mainly a mechanical refactor.

### Motivation

The main motivation is so that we can verify that the ranges are valid
and do not overlap.

## Test Plan

`cargo test`
2023-09-11 19:12:46 +00:00
Zanie Blue
24b848a4ea Enable preview mode during benchmarks (#7208)
Split out of https://github.com/astral-sh/ruff/pull/7195 so benchmark
changes from enabling additional rules can be reviewed separately.
2023-09-11 14:09:33 -05:00
Zanie Blue
773ba5f816 Update the docs workflow to allow publishing a specific ref (#7278)
Related https://github.com/astral-sh/ruff/issues/7276

Our docs publishing action does not allow targetting a specific commit
when run manually which means we cannot update the documentation to
anything but the latest commit on `main`. This change allows a ref to be
provided.

---------

Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
2023-09-11 18:51:31 +00:00
Dhruv Manilawala
f5701fcc63 Use snapshots for remaining lexer tests (#7264)
## Summary

This PR updates the remaining lexer test cases to use the snapshots.
This is mainly a mechanical refactor.

## Motivation

The main motivation is so that when we add the token range values to the
test case output, it's easier to update the test cases.

The reason they were not using the snapshots before was because of the usage of
`test_case` macro. The macros is mainly used for different EOL test cases. If we
just generate the snapshots directly, then the snapshot name would be suffixed
with `-1`, `-2`, etc. as the test function is still the same. So, we'll create
the snapshot ourselves with the platform name for the respective EOL
test cases.

## Test Plan

`cargo test`
2023-09-12 00:16:38 +05:30
Zanie Blue
ff0feb191c Use pages deploy instead of the deprecated pages publish command to deploy the docs website (#7277)
See https://github.com/cloudflare/workers-sdk/issues/3067

Related #7276
2023-09-11 13:23:47 -05:00
Zanie Blue
6566d00295 Update rule selection to respect preview mode (#7195)
## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

Extends work in #7046 (some relevant discussion there)

Changes:

- All nursery rules are now referred to as preview rules
- Documentation for the nursery is updated to describe preview
- Adds a "PREVIEW" selector for preview rules
- This is primarily to allow `--preview --ignore PREVIEW --extend-select
FOO001,BAR200`
- Using `--preview` enables preview rules that match selectors

Notable decisions:

- Preview rules are not selectable by their rule code without enabling
preview
- Retains the "NURSERY" selector for backwards compatibility
- Nursery rules are selectable by their rule code for backwards
compatiblity

Additional work:

- Selection of preview rules without the "--preview" flag should display
a warning
- Use of deprecated nursery selection behavior should display a warning
- Nursery selection should be removed after some time

## Test Plan

<!-- How was it tested? -->

Manual confirmation (i.e. we don't have an preview rules yet just
nursery rules so I added a preview rule for manual testing)

New unit tests

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2023-09-11 12:28:39 -05:00
Micha Reiser
7c9bbcf4e2 Bump version to 0.0.288 (#7271)
Co-authored-by: Zanie Blue <contact@zanie.dev>
2023-09-11 18:18:11 +02:00
konsti
babf8d718e Fix D204 when there's a semicolon after a docstring (#7174)
## Summary

Another statement on the same line as the docstring would previous make
the D204 (newline after docstring) fix fail:

```python
class StatementOnSameLineAsDocstring:
    "After this docstring there's another statement on the same line separated by a semicolon." ;priorities=1
    def sort_services(self):
        pass
```

The fix handles this case manually:

```python
class StatementOnSameLineAsDocstring:
    "After this docstring there's another statement on the same line separated by a semicolon."

    priorities=1
    def sort_services(self):
        pass
```

Fixes #7088

## Test Plan

Added a new `D` test case
2023-09-11 15:09:47 +02:00
dependabot[bot]
878813f277 Bump tj-actions/changed-files from 38 to 39 (#7267)
Co-authored-by: repo-ranger[bot] <!-- raw HTML omitted --> (<a
Co-authored-by: renovate[bot] <!-- raw HTML omitted --> (<a
Co-authored-by: jackton1 <a
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-11 11:12:24 +02:00
dependabot[bot]
71c8a02ebd Bump actions/checkout from 3 to 4 (#7266)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-11 11:11:25 +02:00
konsti
3a2c3a7398 Format empty lines in stub files like black's preview style (#7206)
## Summary

Fix all but one empty line differences with the black preview style in
typeshed. The remaining differences are breaking with type comments and
trailing commas in function definitions.

I compared the empty line differences with the preview mode of black
since stable has some oddities that would have been hard to replicate
(https://github.com/psf/black/issues/3861). Additionally, it assumes the
style proposed in https://github.com/psf/black/issues/3862.

An edge case that also surfaced with typeshed are newline before
trailing module comments.

**main**

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99966 | 2760 | 58 |
| transformers | 0.99930 | 2587 | 447 |
| twine | 1.00000 | 33 | 0 |
| **typeshed** | 0.99978 | 3496 | **2173** |
| warehouse | 0.99825 | 648 | 22 |
| zulip | 0.99950 | 1437 | 27 |

**PR**
| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99966 | 2760 | 58 |
| transformers | 0.99930 | 2587 | 447 |
| twine | 1.00000 | 33 | 0 |
| **typeshed** | 0.99983 | 3496 | **18** |
| warehouse | 0.99825 | 648 | 22 |
| zulip | 0.99950 | 1437 | 27 |


Closes #6723

## Test Plan

The main driver was the typeshed diff. I added new test cases for all
kinds of possible empty line combinations in stub files, test cases for
newlines before trailing module comments.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2023-09-11 08:03:59 +00:00
Micha Reiser
7440e54ec6 Avoid allocating in lex_decimal (#7252) 2023-09-11 06:37:25 +00:00
Charlie Marsh
0357e801ed Avoid F401 panic with noqa import name (#7260)
Closes https://github.com/astral-sh/ruff/issues/7244.
2023-09-10 16:31:25 +00:00
Ely Ronnen
69d0caabe7 fix D417 error with function docstrings with dashed lines (#7251)
## Summary

Fix https://github.com/astral-sh/ruff/issues/7250, False positive D417
for docstrings with dashed lines.

## Test Plan

Tested on the example in https://github.com/astral-sh/ruff/issues/7250
2023-09-10 17:28:44 +01:00
Micha Reiser
9cb5ce750e Remove IO based lints from linter benchmark (#7240) 2023-09-08 13:39:50 +02:00
Micha Reiser
0a07a2ca62 Extract string part and normalized string (#7219) 2023-09-08 12:56:55 +02:00
Micha Reiser
47a253fb62 Add PreviewMode option to formatter
## Summary

This PR adds the `--preview` and `--no-preview` options to the `format` command (hidden) and passes it through to the formatte. 

## Test Plan

I added the `dbg(f.options().preview())` statement in `FormatNodeRule::fmt` and verified that the option gets correctly passed to the formatter.
2023-09-08 12:04:28 +02:00
Micha Reiser
d9544a2d37 Disable default criterion features (#7241) 2023-09-08 10:03:14 +00:00
Micha Reiser
41f0aad7b3 Add FString support to binary like formatting
## Summary

This is the last part of the string - binary like formatting. It adds support for handling fstrings the same as "regular" strings.


## Test Plan

I added a test for both binary and comparison. 

Small improvements across several projects

**This PR**
| project      | similarity index  | total files       | changed files     |
|--------------|------------------:|------------------:|------------------:|
| cpython      |           0.76083 |              1789 |              1632 |
| django       |           0.99966 |              2760 |                58 |
| **transformers** |           0.99929 |              2587 |               454 |
| twine        |           1.00000 |                33 |                 0 |
| typeshed     |           0.99978 |              3496 |              2173 |
| **warehouse**    |           0.99825 |               648 |                22 |
| **zulip**        |           0.99950 |              1437 |                27 |


**Base**

| project      | similarity index  | total files       | changed files     |
|--------------|------------------:|------------------:|------------------:|
| cpython      |           0.76083 |              1789 |              1632 |
| django       |           0.99966 |              2760 |                58 |
| transformers |           0.99928 |              2587 |               454 |
| twine        |           1.00000 |                33 |                 0 |
| typeshed     |           0.99978 |              3496 |              2173 |
| warehouse    |           0.99824 |               648 |                22 |
| zulip        |           0.99948 |              1437 |                28 |


<!-- How was it tested? -->
2023-09-08 11:48:57 +02:00
qdegraaf
05951dd338 Fix inconsistent expr_lambda formatting (#6318) 2023-09-08 09:40:58 +00:00
Micha Reiser
c260762900 Formatter: Implicit concatenation in compare expressions
## Summary

This PR implements the logic for breaking implicit concatenated strings before compare expressions by building on top of  #7145 

The main change is a new `BinaryLike` enum that has the `BinaryExpression` and `CompareExpression` variants. Supporting both variants requires some downstream changes but doesn't introduce any new concepts. 

## Test Plan

I added a few more tests. The compatibility improvements are minor but we now perfectly match black on twine 🥳 


**PR**

| project      | similarity index  | total files       | changed files     |
|--------------|------------------:|------------------:|------------------:|
| cpython      |           0.76083 |              1789 |              1632 |
| django       |           0.99966 |              2760 |                58 |
| transformers |           0.99928 |              2587 |               454 |
| **twine**        |           1.00000 |                33 |                 0 | <-- improved
| typeshed     |           0.99978 |              3496 |              2173 |
| **warehouse**    |           0.99824 |               648 |                22 | <-- improved
| zulip        |           0.99948 |              1437 |                28 |


**Base**

| project      | similarity index  | total files       | changed files     |
|--------------|------------------:|------------------:|------------------:|
| cpython      |           0.76083 |              1789 |              1633 |
| django       |           0.99966 |              2760 |                58 |
| transformers |           0.99928 |              2587 |               454 |
| twine        |           0.99982 |                33 |                 1 | 
| typeshed     |           0.99978 |              3496 |              2173 |
| warehouse    |           0.99823 |               648 |                23 |
| zulip        |           0.99948 |              1437 |                28 |
2023-09-08 11:32:20 +02:00
konsti
1d5c4b0a14 Show header for formatter comment decoration info (#7228)
Show header for formatter comment decoration info

**Summary** Show a header in the formatter comment decoration debug
output that shows which node is preceding/following/enclosing
(https://github.com/astral-sh/ruff/pull/6813#issuecomment-1708119550). I
kept this intentionally condensed to make it easy to use this is a small
sidebar without vertical scrolling.

```console
$ cargo run --bin ruff_python_formatter -- --emit stdout --print-comments scratch.py
# Comment decoration: Range, Preceding, Following, Enclosing, Comment
17..20, Some((ParameterWithDefault, 6..10)), None, (Parameters, 5..22), "# a"
44..47, Some((StmtExpr, 28..39)), Some((StmtExpr, 52..60)), (StmtFunctionDef, 0..60), "# b"
77..80, None, None, (ExprList, 71..82), "# c"
{
    Node {
        kind: ParameterWithDefault,
        range: 6..10,
        source: `x=[]`,
    }: {
...
```

**Test Plan** It's debug output.
2023-09-08 09:25:06 +00:00
Micha Reiser
a352f2f092 Preserve generator parentheses in single argument call expressions (#7226) 2023-09-08 10:53:34 +02:00
Micha Reiser
e376c3ff7e Split implicit concatenated strings before binary expressions (#7145) 2023-09-08 06:51:26 +00:00
Greger
9671922e40 Do not use code location for Gitlab fingerprints. (#7203) 2023-09-08 08:25:26 +02:00
konsti
45f9fca228 Reuse locator in formatter comments (#7227)
**Summary** The comment visitor used to rebuild the locator for every
comment. Instead, we now keep the locator on the builder. Follow-up to
#6813.

**Test Plan** No formatting changes.
2023-09-07 20:08:28 +02:00
Charlie Marsh
6661be2c30 Use full range for SIM105 fixes (#7221)
Avoids inserting an accidental extra newline after the fix _and_
addresses the case of a trailing semicolon.
2023-09-07 16:16:43 +01:00
Charlie Marsh
97f945651d Change SIM118 to delete .keys() rather than replace expression (#7223)
Also improves the suggestion text. Closes
https://github.com/astral-sh/ruff/issues/7200.
2023-09-07 16:16:24 +01:00
Charlie Marsh
5cea43731e Add required space for FLY002 fixes (#7222)
Closes https://github.com/astral-sh/ruff/issues/7197.
2023-09-07 14:32:43 +00:00
Jaap Roes
7971e0b0ee Add extend-ignore-names for flake8-self (#7194)
## Summary

Add a configuration option to extend the list of names that can be
accessed without triggering SLF001.

Fixes issue #7018

## Test Plan

Manually tested by creating a python file (`test.py`):

```python
def foo(obj):
    obj._meta
```

and a `ruff.toml` file:

```toml
select = ["SLF"]

[flake8-self]
extend-ignore-names = ["_meta"]
```

Then running `cargo run -p ruff_cli -- check test.py --no-cache` (once
with the `extend-ignore-names` line comment out) to see if the
configuration option works.

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2023-09-07 13:41:22 +00:00
Micha Reiser
842ff0212e Add Lexer emoji test case (#7213) 2023-09-07 10:02:50 +00:00
Micha Reiser
f1a4eb9c28 Use the unicode-ident crate (#7212) 2023-09-07 08:19:25 +00:00
Victor Hugo Gomes
041cdb95e0 Update identifier Unicode character validation to match Python spec (#7209)
Co-authored-by: Micha Reiser <micha@reiser.io>
2023-09-07 07:08:42 +00:00
Charlie Marsh
fda48afc23 Add required space to C416 fix (#7204)
Closes https://github.com/astral-sh/ruff/issues/7196.
2023-09-06 16:25:02 +00:00
Charlie Marsh
c1af1c291d Add required space to UP006 and UP007 fixes (#7202)
We're removing a set of brackets here so we need to pad the fix.

Closes https://github.com/astral-sh/ruff/issues/7201.
2023-09-06 16:06:02 +00:00
Micha Reiser
171b66cb43 Lexer: Add skip whitespace fastpath (#7184) 2023-09-06 16:14:01 +02:00
Charlie Marsh
fa0b6f4813 Avoid attempting to fix SIM105 violations with multi-statement lines (#7191)
I may revisit this and fix it "properly", but so rare that it's worth
disabling for now: https://github.com/astral-sh/ruff/issues/7123.
2023-09-06 13:35:46 +00:00
Charlie Marsh
eab85aea1a Use structured types for C417 comprehension target (#7190)
Rather than manually joining the arguments as a comma-separated string,
and treating that comma-separated string as a name.
2023-09-06 13:20:04 +00:00
Charlie Marsh
5b31524920 Parenthesize C417 targets if necessary (#7189)
Closes https://github.com/astral-sh/ruff/issues/7121.
2023-09-06 12:56:47 +00:00
Charlie Marsh
6d0469638c Avoid attempting to fix NPY001 with overridden builtins (#7187) 2023-09-06 12:24:37 +00:00
Charlie Marsh
29ba2bb943 Add required space when fixing C404 (#7185) 2023-09-06 13:05:39 +01:00
Charlie Marsh
f0ea40a68d Restructure signatures of flake8_comprehensions fixers (#7186) 2023-09-06 12:04:50 +00:00
Charlie Marsh
a3a531e0d4 Add alpha instructions to the ruff_python_formatter README (#7064) 2023-09-06 11:55:16 +00:00
Tom Kuson
b3e8eca871 Rename PLR1714 to repeated-equality-comparison (#7182) 2023-09-06 12:46:48 +01:00
konsti
447b7cb0e2 Formatter: Show preceding, following and enclosing nodes of comments, Attempt 2 (#6813) 2023-09-06 12:26:13 +02:00
konsti
e3114a144c Ignore single quote docstrings with newline escape (#7173) 2023-09-06 10:51:50 +02:00
Manuel Martinez
2e58ad437e build: add libcst from crates (#7179)
Co-authored-by: Micha Reiser <micha@reiser.io>
2023-09-06 08:24:28 +00:00
Dhruv Manilawala
04f2842e4f Move ExprConstant::kind to StringConstant::unicode (#7180) 2023-09-06 07:39:25 +00:00
Micha Reiser
31990b8d3f Checker: Remove unnecessary unreachable (#7181) 2023-09-06 07:21:03 +00:00
Micha Reiser
5f59101811 Memoize text width (#6552) 2023-09-06 07:10:13 +00:00
Dhruv Manilawala
fa6bff0078 Add inline documentation for Ipy* AST nodes (#7178) 2023-09-06 12:07:34 +05:30
Dhruv Manilawala
ea7c394817 Copy the starred argument as is for PLW3301 autofix (#7177) 2023-09-06 08:57:05 +05:30
Charlie Marsh
264d9159f8 Add required space when fixing UP024 (#7171) 2023-09-05 17:37:09 +00:00
Charlie Marsh
37dfb205b1 Remove autofix for ambiguous unicode rules (#7168) 2023-09-05 17:22:18 +00:00
Charlie Marsh
f8e4e1d562 Fix named expression precedence in generator (#7170) 2023-09-05 17:06:57 +00:00
Charlie Marsh
89be850b73 Add required space when fixing SIM300 (#7167) 2023-09-05 17:00:07 +00:00
Zanie Blue
d68041ba24 Fix B006 when function docstring is followed by whitespace but no newline (#7160) 2023-09-05 11:10:57 -05:00
Charlie Marsh
b60b37e866 Split within not, rather than outside of not, for PT018 (#7151) 2023-09-05 14:50:16 +00:00
konsti
5a95edab45 Use ruff line-length in format_dev (#6870) 2023-09-05 16:19:17 +02:00
Dhruv Manilawala
1adde24133 Rename parser mode from Jupyter to Ipython (#7153) 2023-09-05 14:12:26 +00:00
konsti
e02d76f070 Use insta_cmd (#6737) 2023-09-05 12:21:27 +00:00
Charlie Marsh
7ead2c17b1 Add required space when fixing C402 (#7152) 2023-09-05 12:19:33 +00:00
Charlie Marsh
e428099e4c Add required space when fixing SIM118 (#7150) 2023-09-05 11:51:34 +00:00
Charlie Marsh
7a83fd9926 Insert required space when fixing B013 (#7148) 2023-09-05 12:49:11 +01:00
Charlie Marsh
e8f78fa2cf Avoid fixing UP022 when capture_output is provided (#7149) 2023-09-05 11:44:17 +00:00
Charlie Marsh
955501f267 Use generator for UP007 autofix (#7137) 2023-09-05 11:41:53 +00:00
Micha Reiser
175b3702c3 Reduce comments.clone calls (#7144) 2023-09-05 11:32:56 +02:00
Nicholas Grisafi
40ee4909b5 Added argfile test and documentation (#7138)
Co-authored-by: konsti <konstin@mailbox.org>
2023-09-05 11:13:58 +02:00
Charlie Marsh
10a8e4a225 Remove output-file and target-version from formatter CLI (#7135) 2023-09-05 09:04:18 +00:00
konsti
0465b03282 Better formatter CLI verbose output (#7129) 2023-09-05 00:25:16 +02:00
Dhruv Manilawala
154fe7bdcc Add lexer benchmark (#7132) 2023-09-04 13:18:36 +00:00
Charlie Marsh
ece30e7c69 Preserve parentheses around partial call chains (#7109) 2023-09-04 10:57:04 +01:00
Charlie Marsh
7be28a38c5 Cache comment lookups in suite.rs (#7092) 2023-09-04 08:45:14 +00:00
Charlie Marsh
5ec73a6137 Avoid triggering N806 on TypeAlias assignments (#7119) 2023-09-04 08:44:28 +00:00
Dhruv Manilawala
1067261a55 Make SourceKind a required parameter (#7013) 2023-09-04 07:45:59 +00:00
Micha Reiser
93ca8ebbc0 Formatter: Detect line endings (#7054) 2023-09-04 08:09:31 +02:00
Charlie Marsh
834566f34f Retain parentheses when fixing SIM210 (#7118) 2023-09-03 22:39:23 +00:00
Charlie Marsh
a56121672c Add parentheses when simplifying conditions in SIM222 (#7117) 2023-09-03 22:35:59 +00:00
Charlie Marsh
32f4a96c64 Fix precedence of annotated assignments in generator (#7115) 2023-09-03 21:41:48 +00:00
Charlie Marsh
c004e03395 Add space after return when inlining number for RET504 (#7116) 2023-09-03 21:33:41 +00:00
Charlie Marsh
b57ddd54d2 Support parenthesized expressions in UP028 (#7114) 2023-09-03 21:20:09 +00:00
dalgarno
af189db5eb Deduplicate information in configuration documentation (#7108) 2023-09-03 22:17:35 +01:00
Charlie Marsh
c7217e34d7 Avoid adding duplicate text keyword to subprocess.run (#7112) 2023-09-03 21:17:04 +00:00
Charlie Marsh
d70247959c Avoid adding duplicate capture_output keyword to subprocess.run (#7113) 2023-09-03 21:14:56 +00:00
Charlie Marsh
911d4f2918 Handle parenthesized calls in PD002 (#7111) 2023-09-03 21:03:55 +00:00
Charlie Marsh
d9cf31f355 Expand F841 fixes to handle parenthesized targets (#7110) 2023-09-03 21:00:44 +00:00
Olivia Crain
7da99cc756 Fix incorrect flake8-copyright link in faq (#7093) 2023-09-03 18:00:25 +00:00
Charlie Marsh
9c3b2c3c3c Parenthesize expressins when converting to B009 (#7091) 2023-09-03 17:50:38 +01:00
Charlie Marsh
37d244d178 Add newline if B006 fix is at end-of-file (#7090) 2023-09-03 17:35:59 +01:00
Charlie Marsh
dbb34804a4 Change Option<Result> to Result<Option> in importer (#7089) 2023-09-03 16:23:42 +00:00
Charlie Marsh
3c7486817b Use symbol import for NPY003 replacement (#7083) 2023-09-03 16:53:28 +01:00
Charlie Marsh
3c3c5b5c57 Support length-2 lists in dictionary comprehension rewrites (#7081) 2023-09-03 13:34:10 +00:00
Charlie Marsh
b0d171ac19 Supported starred exceptions in length-one tuple detection (#7080) 2023-09-03 13:31:13 +00:00
Charlie Marsh
b70dde4a77 Avoid attempting to fix invalid Optional annotations (#7079) 2023-09-03 13:23:36 +00:00
Dhruv Manilawala
4eaa412370 Update LibCST (#7062)
## Summary

This PR updates the revision of `LibCST` dependency to 9c263aa897
inorder to fix https://github.com/astral-sh/ruff/issues/4899

## Test Plan

The test case including the carriage return (`\r`) character was added for
`F504` and then `cargo test`.

fixes: #4899
2023-09-03 09:11:24 +05:30
Charlie Marsh
577280c8be Rename ruff_python_formatter/README.md to CONTRIBUTING.md (#7065) 2023-09-02 16:25:23 +00:00
Charlie Marsh
45680bbb44 Avoid duplicate fixes for multi-import imports in RUF017 (#7063)
If a user has `import collections, functools, operator`, and we try to
import from `functools` and `operator`, we end up adding two identical
synthetic edits to preserve that import statement. We need to dedupe
them.

Closes https://github.com/astral-sh/ruff/issues/7059.
2023-09-02 12:58:18 +01:00
Justin Prieto
71ff6f911d Fix getattr calls on int literals (#7057) 2023-09-02 11:45:35 +00:00
Micha Reiser
c05e4628b1 Introduce Token element (#7048) 2023-09-02 10:05:47 +02:00
Charlie Marsh
2f3a950f6f Bump version to 0.0.287 (#7038) 2023-09-01 17:32:26 +01:00
Charlie Marsh
dea65536e9 Fix placement for comments within f-strings concatenations (#7047)
## Summary

Restores the dangling comment handling for f-strings, which broke with
the parenthesized expression code.

Closes https://github.com/astral-sh/ruff/issues/6898.

## Test Plan

`cargo test`

No change in any of the similarity indexes or changed file counts:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99957 | 2760 | 67 |
| transformers | 0.99927 | 2587 | 468 |
| twine | 0.99982 | 33 | 1 |
| typeshed | 0.99978 | 3496 | 2173 |
| warehouse | 0.99818 | 648 | 24 |
| zulip | 0.99942 | 1437 | 32 |
2023-09-01 16:27:32 +00:00
dependabot[bot]
fbc9b5a604 Bump cloudflare/wrangler-action from 3.1.0 to 3.1.1 (#7045) 2023-09-01 15:00:11 +00:00
Zanie Blue
253a241f5d Add dependabot for cargo dependencies (#7034)
Ideally we shouldn't have to run `cargo update` manually — it requires
us to remember to do so and groups all updates into a single pull
request making it challenging to determine which upgrade introduces
regressions e.g. #6964. Here we add daily checks for cargo dependency
updates.

This pull request also simplifies dependabot configuration for GitHub
Actions versions.
2023-09-01 09:47:40 -05:00
Sergey Chudov
33806b8b7c Fixed panic in missing_copyright_notice (#7029) 2023-09-01 13:58:48 +00:00
Charlie Marsh
afcd00da56 Create ruff_notebook crate (#7039)
## Summary

This PR moves `ruff/jupyter` into its own `ruff_notebook` crate. Beyond
the move itself, there were a few challenges:

1. `ruff_notebook` relies on the source map abstraction. I've moved the
source map into `ruff_diagnostics`, since it doesn't have any
dependencies on its own and is used alongside diagnostics.
2. `ruff_notebook` has a couple tests for end-to-end linting and
autofixing. I had to leave these tests in `ruff` itself.
3. We had code in `ruff/jupyter` that relied on Python lexing, in order
to provide a more targeted error message in the event that a user saves
a `.py` file with a `.ipynb` extension. I removed this in order to avoid
a dependency on the parser, it felt like it wasn't worth retaining just
for that dependency.

## Test Plan

`cargo test`
2023-09-01 13:56:44 +00:00
Charlie Marsh
08e246764f Refactor ruff_cli's run method to return on each branch (#7040)
## Summary

I think the fallthrough here for some branches is a little confusing.
Now each branch either runs a command that returns `Result<ExitStatus>`,
or runs a command that returns `Result<()>` and then explicitly returns
`Ok(ExitStatus::SUCCESS)`.
2023-09-01 14:15:38 +01:00
Chris Pryer
0489bbc54c Match Black's formatting of trailing comments containing NBSP (#7030) 2023-09-01 14:52:59 +02:00
Charlie Marsh
60132da7bb Add a NotebookError type to avoid returning Diagnostics on error (#7035)
## Summary

This PR refactors the error-handling cases around Jupyter notebooks to
use errors rather than `Box<Diagnostics>`, which creates some oddities
in the downstream handling. So, instead of formatting errors as
diagnostics _eagerly_ (in the notebook methods), we now return errors
and convert those errors to diagnostics at the last possible moment (in
`diagnostics.rs`). This is more ergonomic, as errors can be composed and
reported-on in different ways, whereas diagnostics require a `Printer`,
etc.

See, e.g.,
https://github.com/astral-sh/ruff/pull/7013#discussion_r1311136301.

## Test Plan

Ran `cargo run` over a Python file labeled with a `.ipynb` suffix, and
saw:

```
foo.ipynb:1:1: E999 SyntaxError: Expected a Jupyter Notebook, which must be internally stored as JSON, but found a Python source file: expected value at line 1 column 1
```
2023-09-01 11:08:05 +00:00
Chris Pryer
17a44c0078 Exclude pragma comments from measured line width (#7008)
Co-authored-by: Micha Reiser <micha@reiser.io>
2023-09-01 06:34:51 +00:00
Charlie Marsh
376d3caf47 Treat empty-line separated comments as trailing statement comments (#6999)
## Summary

This PR modifies our between-statement comment handling such that
comments that are not separated by a statement by any newlines continue
to be treated as leading comments on the statement, but comments that
_are_ separated are instead formatted as trailing comments on the
preceding statement.

See, e.g., the originating snippet:

```python
DEFAULT_TEMPLATE = "flatpages/default.html"

# This view is called from FlatpageFallbackMiddleware.process_response
# when a 404 is raised, which often means CsrfViewMiddleware.process_view
# has not been called even if CsrfViewMiddleware is installed. So we need
# to use @csrf_protect, in case the template needs {% csrf_token %}.
# However, we can't just wrap this view; if no matching flatpage exists,
# or a redirect is required for authentication, the 404 needs to be returned
# without any CSRF checks. Therefore, we only
# CSRF protect the internal implementation.


def flatpage(request, url):
    pass
```

Here, we need to ensure that the `def flatpage` is precede by two empty
lines. However, we want those two empty lines to be enforced from the
_end_ of the comment block, _unless_ the comments are directly atop the
`def flatpage`.

I played with this a bit, and I think the simplest conceptual model and
implementation is to instead treat those as trailing comments on the
preceding node. The main difficulty with this approach is that, in order
to be fully compatible with Black, we'd sometimes need to insert
newlines _between_ the preceding node and its trailing comments. See,
e.g.:

```python
def func():
    ...
# comment

x = 1
```

In this case, we'd need to insert two blank lines between `def func():
...` and `# comment`, but `# comment` is trailing comment on `def
func(): ...`. So, we'd need to take this case into account in the
various nodes that _require_ newlines after them: functions, classes,
and imports. After some discussion, we've opted _not_ to support this,
and just treat these as trailing comments -- so we won't insert newlines
there. This means our handling is still identical to Black's on
Black-formatted code, but avoids moving such trailing comments on
unformatted code.

I dislike that the empty handling is so complex, and that it's split
between so many different nodes, but this is really tricky. Continuing
to treat these as leading comments is very difficult too, since we'd
need to do similar tricks for the leading comment handling in those
nodes, and influencing leading comments is even harder, since they're
all formatted _before_ the node itself.

Closes https://github.com/astral-sh/ruff/issues/6761.

## Test Plan

`cargo test`

Surprisingly, it doesn't change the similarity at all (apart from a
0.00001 change in CPython), but I manually confirmed that it did fix the
originating issue in Django.

Before:

| project      | similarity index |
|--------------|------------------|
| cpython      | 0.76082          |
| django       | 0.99921          |
| transformers | 0.99854          |
| twine        | 0.99982          |
| typeshed     | 0.99953          |
| warehouse    | 0.99648          |
| zulip        | 0.99928          |


After:

| project      | similarity index |
|--------------|------------------|
| cpython      | 0.76081          |
| django       | 0.99921          |
| transformers | 0.99854          |
| twine        | 0.99982          |
| typeshed     | 0.99953          |
| warehouse    | 0.99648          |
| zulip        | 0.99928          |
2023-08-31 20:55:05 +00:00
Charlie Marsh
51d69b448c Improve compatibility between multi-statement PYI rules (#7024)
## Summary

This PR modifies a few of our rules related to which statements (and how
many) are allowed in function bodies within `.pyi` files, to improve
compatibility with flake8-pyi and improve the interplay dynamics between
them. Each change fixes a deviation from flake8-pyi:

- We now always trigger the multi-statement rule (PYI048) regardless of
whether one of the statements is a docstring.
- We no longer trigger the `...` rule (PYI010) if the single statement
is a docstring or a `pass` (since those are covered by other rules).
- We no longer trigger the `...` rule (PYI010) if the function body
contains multiple statements (since that's covered by PYI048).

Closes https://github.com/astral-sh/ruff/issues/7021.

## Test Plan

`cargo test`
2023-08-31 21:45:26 +01:00
Nicholas Grisafi
f7dca3d958 Add Rippling to who uses ruff (#7032)
## Summary

Adding rippling to who uses ruff section

## Test Plan

We migrated to ruff 🙂
2023-08-31 18:36:48 +00:00
Charlie Marsh
7c1aa98f43 Run cargo update (#6964) 2023-08-31 13:11:11 -05:00
Charlie Marsh
68f605e80a Fix WithItem ranges for parenthesized, non-as items (#6782)
## Summary

This PR attempts to address a problem in the parser related to the
range's of `WithItem` nodes in certain contexts -- specifically,
`WithItem` nodes in parentheses that do not have an `as` token after
them.

For example,
[here](https://play.ruff.rs/71be2d0b-2a04-4c7e-9082-e72bff152679):

```python
with (a, b):
    pass
```

The range of the `WithItem` `a` is set to the range of `(a, b)`, as is
the range of the `WithItem` `b`. In other words, when we have this kind
of sequence, we use the range of the entire parenthesized context,
rather than the ranges of the items themselves.

Note that this also applies to cases
[like](https://play.ruff.rs/c551e8e9-c3db-4b74-8cc6-7c4e3bf3713a):

```python
with (a, b, c as d):
    pass
```

You can see the issue in the parser here:

```rust
#[inline]
WithItemsNoAs: Vec<ast::WithItem> = {
    <location:@L> <all:OneOrMore<Test<"all">>> <end_location:@R> => {
        all.into_iter().map(|context_expr| ast::WithItem { context_expr, optional_vars: None, range: (location..end_location).into() }).collect()
    },
}
```

Fixing this issue is... very tricky. The naive approach is to use the
range of the `context_expr` as the range for the `WithItem`, but that
range will be incorrect when the `context_expr` is itself parenthesized.
For example, _that_ solution would fail here, since the range of the
first `WithItem` would be that of `a`, rather than `(a)`:

```python
with ((a), b):
    pass
```

The `with` parsing in general is highly precarious due to ambiguities in
the grammar. Changing it in _any_ way seems to lead to an ambiguous
grammar that LALRPOP fails to translate. Consensus seems to be that we
don't really understand _why_ the current grammar works (i.e., _how_ it
avoids these ambiguities as-is).

The solution implemented here is to avoid changing the grammar itself,
and instead change the shape of the nodes returned by various rules in
the grammar. Specifically, everywhere that we return `Expr`, we instead
return `ParenthesizedExpr`, which includes a parenthesized range and the
underlying `Expr` itself. (If an `Expr` isn't parenthesized, the ranges
will be equivalent.) In `WithItemsNoAs`, we can then use the
parenthesized range as the range for the `WithItem`.
2023-08-31 16:21:29 +01:00
Zanie Blue
96a9717c1a Add hidden --preview / --no-preview options to ruff check (#7009)
Per discussion at https://github.com/astral-sh/ruff/discussions/6998

<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

Adds a `--preview` and `--no-preview` option to the CLI for `ruff check`
and corresponding settings. The CLI options are hidden for now.

Available in the settings as `preview = true` or `preview = false`.

Does not include environment variable configuration, although we may add
it in the future.

## Test Plan

<!-- How was it tested? -->

`cargo build`

Future work will build on this setting, such as toggling the mode during
a test.
2023-08-31 09:51:59 -05:00
magic-akari
f4ba0ea144 Allow tab_width to be configable (#7016) 2023-08-31 07:40:03 +00:00
Micha Reiser
92143afeee Group binary operators with same precedence only (#7010) 2023-08-31 09:19:45 +02:00
Micha Reiser
eb552da8a9 Avoid parenthesizing multiline strings in binary expressions (#6973) 2023-08-30 16:03:17 +02:00
Charlie Marsh
e2b2b1759f Handle keyword comments between = and value (#6883)
## Summary

This PR adds comment handling for comments between the `=` and the
`value` for keywords, as in the following cases:

```python
func(
    x  # dangling
    =  # dangling
    # dangling
    1,
    **  # dangling
    y
)
```

(Comments after the `**` were already handled in some cases, but I've
unified the handling with the `=` handling.)

Note that, previously, comments between the `**` and its value were
rendered as trailing comments on the value (so they'd appear after `y`).
This struck me as odd since it effectively re-ordered the comment with
respect to its closest AST node (the value). I've made them leading
comments, though I don't know that that's a significant improvement. I
could also imagine us leaving them where they are.
2023-08-30 09:52:51 -04:00
Chris Pryer
a3f4d7745a Use reserved width to include line suffix measurement (#6901)
Co-authored-by: Micha Reiser <micha@reiser.io>
2023-08-30 08:07:11 +00:00
Micha Reiser
edfd888bd6 Add unicode benchmark (#7002) 2023-08-30 09:57:57 +02:00
Micha Reiser
9c382e8291 Show changed files (#7003) 2023-08-30 06:47:45 +00:00
Charlie Marsh
eb2b226142 Unset after_class_docstring state on every iteration (#7001) 2023-08-30 08:20:28 +02:00
Victor Hugo Gomes
31947af6a3 Don't "flatten" nested if expressions when formatting (#6996) 2023-08-30 04:11:58 +00:00
Charlie Marsh
b404e54f33 Remove unnecessary Comment#slice calls (#6997) 2023-08-30 00:44:11 +00:00
qdegraaf
34e8de738e [perflint] Expand PERF401 and PERF402 with type checks (#6994)
## Summary

Attempt at a small improvement to two `perflint` rules using the new
type inference capabilities to only flag `PERF401` and `PERF402` for
values we infer to be lists. This makes the rule more conservative, as
it only flags values that we _know_ to be lists, but it's overall a
desirable change, as it favors false negatives over false positives for
a "nice-to-have" rule.

Closes https://github.com/astral-sh/ruff/issues/6995.

## Test Plan

Add non-list value cases and make sure all old cases are still caught.
2023-08-29 19:15:29 -04:00
qdegraaf
1550a6bfe7 [perflint] Chore: correct PERF401 fixture comments (#6993)
## Summary
Fixes comments in PERF401 fixture to correctly reflect how rule works
2023-08-29 17:16:31 -04:00
Anselm Hahn
dc4db39f78 docs: 📝 Updating the example for assert(S101) (#6986)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

Rewriting the `if`-comparison to focus on the meaning of rule ` assert
S101`.

Fixes #6984

## Test Plan

<!-- How was it tested? -->

---------

Co-authored-by: Zanie Blue <contact@zanie.dev>
2023-08-29 16:08:08 -05:00
qdegraaf
f3aaf84a28 Move refurb/helpers utils to ruff_python_semantic for broader use (#6990)
## Summary

The utils added for `refurb` in its `helpers.rs` file could be useful
for many other plugins. (Such as the PERF4XX codes, see e.g.
https://github.com/astral-sh/ruff/pull/6132 ).

This PR moves them to `ruff_python_semantic::analyzers::typing` as
suggested in
https://github.com/astral-sh/ruff/pull/6132#issuecomment-1697910093

## Test Plan

Confirmed `refurb` and all other tests still work
2023-08-29 14:45:09 -04:00
Charlie Marsh
5de95d7054 Reuse FormatResult and FormatterIterationError in format_stdin.rs (#6985)
## Summary

Ensures that we use the same error types and messages. Also renames
those struct to `FormatCommand*` for consistency, and removes the
`FormatCommandResult::Skipped` variant in favor of skipping in the
iterator directly.
2023-08-29 17:41:53 +00:00
Charlie Marsh
8d1610d960 Implement Display on formatter structs (#6983)
Feedback from
https://github.com/astral-sh/ruff/pull/6948#discussion_r1308260021.
2023-08-29 16:57:26 +00:00
Charlie Marsh
fad23bbe60 Add a --check flag to the formatter CLI (#6982)
## Summary

Returns an exit code of 1 if any files would be reformatted:

```
ruff on  charlie/format-check:main [$?⇡] is 📦 v0.0.286 via 🐍 v3.11.2 via 🦀 v1.72.0
❯ cargo run -p ruff_cli -- format foo.py --check
   Compiling ruff_cli v0.0.286 (/Users/crmarsh/workspace/ruff/crates/ruff_cli)
    Finished dev [unoptimized + debuginfo] target(s) in 1.69s
     Running `target/debug/ruff format foo.py --check`
warning: `ruff format` is a work-in-progress, subject to change at any time, and intended only for experimentation.
1 file would be reformatted
ruff on  charlie/format-check:main [$?⇡] is 📦 v0.0.286 via 🐍 v3.11.2 via 🦀 v1.72.0 took 2s
❯ echo $?
1
```

Closes #6966.
2023-08-29 12:40:00 -04:00
Charlie Marsh
25c374856a Move stdin formatting to its own command file (#6981)
## Summary

This is similar to `commands::check` vs. `commands::check_stdin`, and
gets the logic out of the parent file (`lib.rs`). It also ensures that
we avoid formatting files that should be excluded when `--force-exclude`
is provided.
2023-08-29 16:06:10 +00:00
Charlie Marsh
34221346c1 Rename run.rs command to check.rs (#6980)
The CLI command is called "check", so this is more consistent (and
consistent with the pattern used in other commands).
2023-08-29 15:52:06 +00:00
Chris Pryer
924f10186f Update dynamic_text builder doc comment (#6978) 2023-08-29 16:29:11 +02:00
Charlie Marsh
b7634b6ede Fix typo in banned-from (#6977)
Oops...
2023-08-29 09:39:09 -04:00
Dhruv Manilawala
4d49d5e845 Add eat_char2 for the lexer (#6968)
## Summary

This PR adds a new helper method on the `Cursor` called `eat_char2`
which is similar to `eat_char` but accepts 2 characters instead of 1. It'll
`bump` the cursor twice if both characters are found on lookahead.

## Test Plan

`cargo test`
2023-08-29 17:18:02 +05:30
Micha Reiser
715d86dae9 Remove Comprehension priority (#6947) 2023-08-29 08:30:15 +02:00
Micha Reiser
adb48692d6 Use optional parentheses for tuples in return statements (#6875) 2023-08-29 08:30:05 +02:00
Charlie Marsh
19ccf1d073 Make RUF100 a suggested fix (#6967)
I made this automatic when I removed the deprecated "unspecified"
method, but I think suggested is actually more appropriate.
2023-08-29 04:28:52 +00:00
Charlie Marsh
e3c36465ec Base parameter-find lookup on range, rather than name (#6960)
## Summary

This simplifies the signature a bit and is more reliable in the
(unusual? invalid?) case of duplicate parameters.
2023-08-29 01:39:38 +00:00
Valeriy Savchenko
7e36284684 [refurb] Implement check-and-remove-from-set rule (FURB132) (#6904)
## Summary

This PR is a continuation of #6897 and #6702 and me replicating `refurb`
rules (#1348). It adds support for
[FURB132](https://github.com/dosisod/refurb/blob/master/refurb/checks/builtin/set_discard.py)

## Test Plan

I included a new test + checked that all other tests pass.
2023-08-29 01:17:26 +00:00
Valeriy Savchenko
c448b4086a [refurb] Implement delete-full-slice rule (FURB131) (#6897)
## Summary

This PR is a continuation of #6702 and me replicating `refurb` rules
(#1348). It adds support for
[FURB131](https://github.com/dosisod/refurb/blob/master/refurb/checks/builtin/no_del.py.

## Test Plan

I included a new test + checked that all other tests pass.
2023-08-28 20:52:38 -04:00
Charlie Marsh
3200015c06 Fix banned-from documentation (#6959)
Closes https://github.com/astral-sh/ruff/issues/6839.
2023-08-29 00:51:15 +00:00
Valeriy Savchenko
26d53c56a2 [refurb] Implement repeated-append rule (FURB113) (#6702)
## Summary

As an initial effort with replicating `refurb` rules (#1348 ), this PR
adds support for
[FURB113](https://github.com/dosisod/refurb/blob/master/refurb/checks/builtin/list_extend.py)
and adds a new category of checks.

## Test Plan

I included a new test + checked that all other tests pass.
2023-08-28 22:51:59 +00:00
Charlie Marsh
1439bb592e Report number of changed files on the format CLI (#6948)
## Summary

Very basic summary:

<img width="962" alt="Screen Shot 2023-08-28 at 1 17 37 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/53537aca-7579-44d8-855b-f4553affae50">

If you run with `--verbose`, we'll also show you the timing:

<img width="962" alt="Screen Shot 2023-08-28 at 1 17 58 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/63cbd13e-9462-4e49-b3a3-c6663a7ad41c">
2023-08-28 18:42:31 -04:00
Charlie Marsh
fc47e0dab2 Clean up some misc. code in NamedTuple and TypedDict conversion (#6957)
- Use `Option` instead of `Result` everywhere.
- Use `field` instead of `property` (to match the nomenclature of
`NamedTuple` and `TypedDict`).
- Put the violation function at the top of the file, rather than the
bottom.
2023-08-28 17:12:56 -04:00
Charlie Marsh
87aa5d6b66 Avoid panic when typename is provided as a keyword argument (#6955)
## Summary

The `typename` argument to `NamedTuple` and `TypedDict` is a required
positional argument. We assumed as much, but panicked if it was provided
as a keyword argument or otherwise omitted. This PR handles the case
gracefully.

Closes https://github.com/astral-sh/ruff/issues/6953.
2023-08-28 21:03:18 +00:00
Charlie Marsh
d1ad20c9ea Avoid invalid fix for C417 with separate keys and values (#6954)
Closes https://github.com/astral-sh/ruff/issues/6951.
2023-08-28 16:49:40 -04:00
Charlie Marsh
ecca125f9a Use SourceCodeSnippet for more diagnostic messages (#6950)
## Summary

This ensures that we truncate the messages when they break over multiple
lines, etc.
2023-08-28 16:22:49 -04:00
Charlie Marsh
005e21a139 Add line-length to E501 documentation (#6949)
Closes https://github.com/astral-sh/ruff/issues/6908.
2023-08-28 14:45:30 -04:00
dependabot[bot]
e1db036f90 ci(deps): bump tj-actions/changed-files from 37 to 38 (#6946)
Co-authored-by: jackton1 <a
Co-authored-by: repo-ranger[bot] <!-- raw HTML omitted --> (<a
Co-authored-by: repo-ranger[bot] <!-- raw HTML omitted -->
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-28 16:50:38 +00:00
Zanie Blue
9ad67b0758 Add comment to ecoystem check re. ALL rules (#6943) 2023-08-28 11:28:36 -05:00
Jelle van der Waa
af61abc747 Re-use is_magic where possible (#6945)
## Summary

Use `is_magic` where possible

## Test Plan

Unit tests
2023-08-28 15:35:34 +00:00
Charlie Marsh
ec575188c4 Narrow the supported options on the format CLI (#6944)
## Summary

Ensures that we only show supported options:

<img width="1228" alt="Screen Shot 2023-08-28 at 11 03 16 AM"
src="https://github.com/astral-sh/ruff/assets/1309177/50fb7595-dc30-43d2-a7e4-c0103acc15b9">

For now, I'm not super focused on DRYing up the CLI.
2023-08-28 15:28:22 +00:00
Charlie Marsh
aea7500c1e Allow Locator#slice to take Ranged (#6922)
## Summary

As a small quality-of-life improvement, the locator can now slice like
`locator.slice(stmt)` instead of requiring
`locator.slice(stmt.range())`.

## Test Plan

`cargo test`
2023-08-28 11:08:39 -04:00
Charlie Marsh
58f5f27dc3 Add TOML files to SourceType (#6929)
## Summary

This PR adds a higher-level enum (`SourceType`) around `PySourceType` to
allow us to use the same detection path to handle TOML files. Right now,
we have ad hoc `is_pyproject_toml` checks littered around, and some
codepaths are omitting that logic altogether (like `add_noqa`). Instead,
we should always be required to check the source type and handle TOML
files as appropriate.

This PR will also help with our pre-commit capabilities. If we add
`toml` to pre-commit (to support `pyproject.toml`), pre-commit will
start to pass _other_ files to Ruff (along with `poetry.lock` and
`Pipfile` -- see
[identify](b59996304f/identify/extensions.py (L355))).
By detecting those files and handling those cases, we avoid attempting
to parse them as Python files, which would lead to pre-commit errors.
(We tried to add `toml` to pre-commit here
(https://github.com/astral-sh/ruff-pre-commit/pull/44), but had to
revert here (https://github.com/astral-sh/ruff-pre-commit/pull/45) as it
led to the pre-commit hook attempting to parse `poetry.lock` files as
Python files.)
2023-08-28 15:01:48 +00:00
Dhruv Manilawala
2893a9f6b5 Remove unused f-string error type (#6941) 2023-08-28 18:34:48 +05:30
Micha Reiser
60097bebcd Handle implicit strings in `can_omit_parentheses (#6940) 2023-08-28 12:20:29 +00:00
Dhruv Manilawala
9c98416b96 Avoid lexer infinite loop on invalid input (#6937)
## Summary

This PR fixes a bug which sends the lexer into infinite loop for an invalid input.
The code in question is `[1` where the nesting is never finished. This means
that the lexer will keep emitting the `Err` token forever.

## Test Plan

Add a test case which collects all the tokens from the lexer. This just
makes sure that it doesn't go into infinite loop.
2023-08-28 17:21:38 +05:30
Victor Hugo Gomes
99f4c6886e Format PatternMatchOr (#6905) 2023-08-28 08:09:17 +00:00
Micha Reiser
30ebf7fc86 Truncate ecosystem comment when necessary (#6899) 2023-08-28 07:58:39 +00:00
Chris Pryer
fa25dabf17 Add comments option to playground (#6911)
Co-authored-by: Micha Reiser <micha@reiser.io>
2023-08-28 07:26:23 +00:00
konsti
e615870659 Unify line size settings between ruff and the formatter (#6873) 2023-08-28 06:44:56 +00:00
Micha Reiser
a6aa16630d Move Configuration to ruff_workspace crate (#6920) 2023-08-28 06:21:35 +00:00
Chris Pryer
039694aaed Add LineSuffix reserved width (#6830)
Thanks for working on this.
2023-08-28 07:46:54 +02:00
Charlie Marsh
6bc1ba6d62 Use stdin for formatter when --stdin-filename is provided (#6926)
## Summary

Just making the formatter CLI more consistent with the linter -- e.g.,
we now use stdin on invocations like `cat foo.py | cargo run -p ruff_cli
-- format -- --stdin-filename=foo.py`, instead of _only_ relying on the
`-` file (and use the same helper as the linter to facilitate this).
2023-08-27 20:32:18 +00:00
Charlie Marsh
cd47368ae4 Use consistent formatting for user-facing formatter errors (#6925)
## Summary

This PR changes the alpha formatter CLI to use the same format for
errors as the linter, e.g.:

<img width="868" alt="Screen Shot 2023-08-27 at 4 03 30 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/9f3dea37-593b-4788-a0c0-e64bcf0d0560">
2023-08-27 20:22:06 +00:00
Charlie Marsh
a871714705 Collect result in format CLI (#6924) 2023-08-27 20:02:18 +00:00
Charlie Marsh
292fdd978e Remove unnecessary generic (#6923) 2023-08-27 15:55:23 -04:00
konsti
c2413dcd2c Add prototype of ruff format for projects (#6871)
**Summary** Add recursive formatting based on `ruff check` file
discovery for `ruff format`, as a prototype for the formatter alpha.
This allows e.g. `format ../projects/django/`. It's still lacking
support for any settings except line length.

Note just like the existing `ruff format` this will become part of the
production build, i.e. you'll be able to use it - hidden by default and
with a prominent warning - with `ruff format .` after the next release.

Error handling works in my manual tests (the colors do also work):

```
$  target/debug/ruff format scripts/
warning: `ruff format` is a work-in-progress, subject to change at any time, and intended for internal use only.
```
(the above changes `add_rule.py` where we have the wrong bin op
breaking)

```
$ target/debug/ruff format ../projects/django/
warning: `ruff format` is a work-in-progress, subject to change at any time, and intended for internal use only.
Failed to format /home/konsti/projects/django/tests/test_runner_apps/tagged/tests_syntax_error.py: source contains syntax errors: ParseError { error: UnrecognizedToken(Name { name: "syntax_error" }, None), offset: 131, source_path: "<filename>" }
```

```
$ target/debug/ruff format a
warning: `ruff format` is a work-in-progress, subject to change at any time, and intended for internal use only.
Failed to read /home/konsti/ruff/a/d.py: Permission denied (os error 13)
```

**Test Plan** Missing! I'm not sure if it's worth building tests at this
stage or how they should look like.
2023-08-27 19:12:18 +00:00
Charlie Marsh
059757a8c8 Implement Ranged on more structs (#6921)
Now that it's in `ruff_text_size`, we can use it in a few places that we
couldn't before.
2023-08-27 19:03:08 +00:00
Charlie Marsh
fc89976c24 Move Ranged into ruff_text_size (#6919)
## Summary

The motivation here is that this enables us to implement `Ranged` in
crates that don't depend on `ruff_python_ast`.

Largely a mechanical refactor with a lot of regex, Clippy help, and
manual fixups.

## Test Plan

`cargo test`
2023-08-27 14:12:51 -04:00
Zanie Blue
88c8bece38 Add PrefectHQ/prefect to ecosystem checks (#6918)
At least I will generally be familiar with the patterns over there
2023-08-27 11:07:50 -05:00
Charlie Marsh
d0b051e447 Fix ranges for global usages (#6917)
## Summary

The range of the usage from `Globals` should be the range of the
identifier, not the range of the full `global pandas` statement.

Closes https://github.com/astral-sh/ruff/issues/6914.
2023-08-27 15:27:07 +00:00
Charlie Marsh
381fc5b2a8 Increase ecosystem CI coverage to 40 projects (#6916)
## Summary

Roughly doubles the number of covered projects.
2023-08-27 11:04:02 -04:00
Micha Reiser
7c480236e0 Use dyn dispatch for any_over_* (#6912) 2023-08-27 15:54:01 +02:00
Micha Reiser
3f3494ad44 Implement ConfigProcessor on non-ref type (#6915) 2023-08-27 15:03:11 +02:00
Chris Pryer
f33277a057 Update playground README.md (#6909) 2023-08-27 10:54:42 +02:00
Micha Reiser
eae59cf088 Optional source map generation (#6894) 2023-08-26 18:00:43 +02:00
Charlie Marsh
15b73bdb8a Introduce AST nodes for PatternMatchClass arguments (#6881)
## Summary

This PR introduces two new AST nodes to improve the representation of
`PatternMatchClass`. As a reminder, `PatternMatchClass` looks like this:

```python
case Point2D(0, 0, x=1, y=2):
  ...
```

Historically, this was represented as a vector of patterns (for the `0,
0` portion) and parallel vectors of keyword names (for `x` and `y`) and
values (for `1` and `2`). This introduces a bunch of challenges for the
formatter, but importantly, it's also really different from how we
represent similar nodes, like arguments (`func(0, 0, x=1, y=2)`) or
parameters (`def func(x, y)`).

So, firstly, we now use a single node (`PatternArguments`) for the
entire parenthesized region, making it much more consistent with our
other nodes. So, above, `PatternArguments` would be `(0, 0, x=1, y=2)`.

Secondly, we now have a `PatternKeyword` node for `x=1` and `y=2`. This
is much more similar to the how `Keyword` is represented within
`Arguments` for call expressions.

Closes https://github.com/astral-sh/ruff/issues/6866.

Closes https://github.com/astral-sh/ruff/issues/6880.
2023-08-26 14:45:44 +00:00
Micha Reiser
ed1b4122d0 Use Codspeed for continous benchmarking (#6896) 2023-08-26 16:34:35 +02:00
Micha Reiser
9d77552e18 Add tab width option (#6848) 2023-08-26 12:29:58 +02:00
Charlie Marsh
f91bacbb94 Avoid PEP 604 upgrades that lead to invalid syntax (#6888)
Closes https://github.com/astral-sh/ruff/issues/6843.
2023-08-25 19:47:15 -04:00
Charlie Marsh
2883ae4d46 Insert space to avoid syntax error in RSE fixes (#6886)
Closes https://github.com/astral-sh/ruff/issues/6810.
2023-08-25 18:26:30 -04:00
Charlie Marsh
bd625b1928 Update PT007 docs to mention row-type (#6885)
Closes https://github.com/astral-sh/ruff/issues/6859.
2023-08-25 22:14:39 +00:00
Charlie Marsh
e0a40783ca Update PTH122 documentation to include Path.stem and Path.parent (#6884)
Closes https://github.com/astral-sh/ruff/issues/6846.
2023-08-25 22:11:56 +00:00
konsti
0e79074c31 Update to Rust 1.72 (#6874)
Update to [Rust
1.72](https://blog.rust-lang.org/2023/08/24/Rust-1.72.0.html), fixed the
failing lints.
2023-08-25 17:42:03 -04:00
Charlie Marsh
edb9b0c62a Use the formatter prelude in more files (#6882)
Removes a bunch of imports that are made redundant by the prelude.
2023-08-25 16:51:07 -04:00
Victor Hugo Gomes
91a780c771 Format PatternMatchClass (#6860) 2023-08-25 19:03:37 +00:00
Charlie Marsh
91880b8273 Bump version to 0.0.286 (#6876) 2023-08-25 14:59:26 -04:00
Zanie Blue
100904adb9 Avoid parsing other parts of a format specification if replacements are present (#6858)
Closes #6767
Replaces https://github.com/astral-sh/ruff/pull/6773 (this cherry-picks
some parts from there)
Alternative to the approach introduced in #6616 which added support for
placeholders in format specifications while retaining parsing of other
format specification parts.

The idea is that if there are placeholders in a format specification we
will not attempt to glean semantic meaning from the other parts of the
format specification we'll just extract all of the placeholders ignoring
other characters. The dynamic content of placeholders can drastically
change the meaning of the format specification in ways unknowable by
static analysis. This change prevents false analysis and will ensure
safety if we build other rules on top of this at the cost of missing
detection of some bad specifications.

Minor note: I've use "replacements" and "placeholders" interchangeably
but am trying to go with "placeholder" as I think it's a better term for
the static analysis concept here
2023-08-25 17:42:57 +00:00
Zanie Blue
0bac7bd114 Update RUF100 test to reflect applicability (#6877)
Broken on merge of #6822 due to new test cases conflicting with the
addition of an applicability to RUF100 in #6827
2023-08-25 16:57:59 +00:00
Zanie Blue
f2eb7bcacf Update ERA100 to apply to commented dictionary items with trailing comments (#6822)
Closes https://github.com/astral-sh/ruff/issues/6821

ERA100 was not raising on commented parts of dictionaries if it included
another comment (such as a noqa clause). In cases where this comment was
a noqa clause, RUF100 to be emitted since the noqa would have no effect.
Here, we update ERA100 to raise even when there are trailing comments.
This resolves the linked issue _and_ increases the scope of ERA100. We
could narrow the regular expression to only apply to noqa comments if we
do not want to expand ERA100 however I think this change is within the
spirit of the rule.
2023-08-25 11:02:31 -05:00
Micha Reiser
29a0c1003b Use BestFit layout even for attributes with a short name (#6872) 2023-08-25 17:47:02 +02:00
Micha Reiser
15b7525464 Rename parser goal 'All' to 'all' (#6867) 2023-08-25 12:00:57 +00:00
konsti
0b6dab5e3f Add jupyter notebook cell ids in 4.5+ if missing (#6853)
**Summary** See
https://github.com/astral-sh/ruff/issues/6834#issuecomment-1691202417

**Test Plan** Added a new notebook
2023-08-25 08:34:42 +00:00
David Szotten
1c66bb80b7 fix is_raw_string for multiple prefixes (#6865)
fix `is_raw_string` in the presence of other prefixes (like `rb"foo"`)

fixes #6864
2023-08-25 09:58:26 +02:00
Dhruv Manilawala
d1f07008f7 Rename Notebook related symbols (#6862)
This PR renames the following symbols:

* `PySourceType::Jupyter` -> `PySourceType::Ipynb`
* `SourceKind::Jupyter` -> `SourceKind::IpyNotebook`
* `JupyterIndex` -> `NotebookIndex`
2023-08-25 11:40:54 +05:30
Micha Reiser
61b2ffa8e8 Add assert test cases (#6855) 2023-08-25 07:51:55 +02:00
Charlie Marsh
1044d66c1c Add support for PatternMatchMapping formatting (#6836)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

Adds support for `PatternMatchMapping` -- i.e., cases like:

```python
match foo:
    case {"a": 1, "b": 2, **rest}:
        pass
```

Unfortunately, this node has _three_ kinds of dangling comments:

```python
{  # "open parenthesis comment"
   key: pattern,
   **  # end-of-line "double star comment"
   # own-line "double star comment"
   rest  # end-of-line "after rest comment"
   # own-line "after rest comment"
}
```

Some of the complexity comes from the fact that in `**rest`, `rest` is
an _identifier_, not a node, so we have to handle comments _after_ it as
dangling on the enclosing node, rather than trailing on `**rest`. (We
could change the AST to use `PatternMatchAs` there, which would be more
permissive than the grammar but not totally crazy -- `PatternMatchAs` is
used elsewhere to mean "a single identifier".)

Closes https://github.com/astral-sh/ruff/issues/6644.

## Test Plan

`cargo test`
2023-08-25 04:33:34 +00:00
Charlie Marsh
813d7da7ec Respect own-line leading comments before parenthesized nodes (#6820)
## Summary

This PR ensures that if an expression has an own-line leading comment
_before_ its open parentheses, we render it as such.

For example, given:

```python
[ # foo
    # bar
    ( # baz
        1
    )
]
```

On `main`, we format as:

```python
[  # foo
    (
        # bar
        # baz
        1
    )
]
```

As of this PR, we format as:

```python
[  # foo
    # bar
    (  # baz
        1
    )
]
```

## Test Plan

`cargo test`
2023-08-25 00:18:05 -04:00
Charlie Marsh
59e70896c0 Fix formatting of comments between function and arguments (#6826)
## Summary

We now format comments between a function and its arguments as dangling.
Like with other strange placements, I've biased towards preserving the
existing formatting, rather than attempting to reorder the comments.

Closes https://github.com/astral-sh/ruff/issues/6818.

## Test Plan

`cargo test`

Before:

| project      | similarity index |
|--------------|------------------|
| cpython      | 0.76050          |
| django       | 0.99820          |
| transformers | 0.99800          |
| twine        | 0.99876          |
| typeshed     | 0.99953          |
| warehouse    | 0.99615          |
| zulip        | 0.99729          |

After:

| project      | similarity index |
|--------------|------------------|
| cpython      | 0.76050          |
| django       | 0.99820          |
| transformers | 0.99800          |
| twine        | 0.99876          |
| typeshed     | 0.99953          |
| warehouse    | 0.99615          |
| zulip        | 0.99729          |
2023-08-25 04:06:56 +00:00
Charlie Marsh
f754ad5898 Handle bracketed comments on sequence patterns (#6801)
## Summary

This PR ensures that we handle bracketed comments on sequences, like `#
comment` here:

```python
match x:
    case [ # comment
        1, 2
    ]:
        pass
```

The handling is very similar to other, similar nodes, except that we do
need some special logic to determine whether the sequence is
parenthesized, similar to our logic for tuples.

## Test Plan

`cargo test`
2023-08-25 04:03:27 +00:00
Charlie Marsh
474e8fbcd4 Format all attribute dot comments manually (#6825)
## Summary

This PR modifies our formatting of comments around the `.` in an
attribute. Specifically, the goal here is to avoid _reordering_
comments, and the net effect is that we generally leave comments
where-they-are when dealing with comments between around the dot (which
you can also think of as comments between attributes).

All comments around the dot are now treated as dangling and formatted
manually, with the exception of end-of-line or parenthesized comments on
the value, like those marked as trailing here, which remain trailing:

```python
(
    (
        a # trailing end-of-line
        # trailing own-line
    ) # dangling before dot end-of-line
    .b # trailing end-of-line
)
```

Closes https://github.com/astral-sh/ruff/issues/6823.

## Test Plan

`cargo test`

Before:

| project      | similarity index |
|--------------|------------------|
| cpython      | 0.76050          |
| django       | 0.99820          |
| transformers | 0.99800          |
| twine        | 0.99876          |
| typeshed     | 0.99953          |
| warehouse    | 0.99615          |
| zulip        | 0.99729          |

After:

| project      | similarity index |
|--------------|------------------|
| cpython      | 0.76050          |
| django       | 0.99820   |
| transformers | 0.99800          |
| twine        | 0.99876          |
| typeshed     | 0.99953          |
| warehouse    | 0.99615          |
| zulip        | 0.99729          |
2023-08-25 03:50:56 +00:00
Charlie Marsh
6f23469e00 Handle pattern parentheses in FormatPattern (#6800)
## Summary

This PR fixes the duplicate-parenthesis problem that's visible in the
tests from https://github.com/astral-sh/ruff/pull/6799. The issue is
that we might have parentheses around the entire match-case pattern,
like in `(1)` here:

```python
match foo:
    case (1):
        y = 0
```

In this case, the inner expression (`1`) will _think_ it's
parenthesized, but we'll _also_ detect the parentheses at the case level
-- so they get rendered by the case, then again by the expression.
Instead, if we detect parentheses at the case level, we can force-off
the parentheses for the pattern using a design similar to the way we
handle parentheses on expressions.

Closes https://github.com/astral-sh/ruff/issues/6753.

## Test Plan

`cargo test`
2023-08-25 03:45:49 +00:00
Charlie Marsh
281ce56dc1 Make isort's detect-same-package behavior configurable (#6833)
## Summary

Our first-party import detection uses a heuristic that doesn't exist in
isort: if an import appears to be from within the same package as the
containing file, we mark it as first-party. For example, if you have a
directory `./foo/__init__.py`, and you import `from foo import bar` in
`./foo/baz.py`, we'll mark that as first-party. (See:
https://github.com/astral-sh/ruff/pull/1266.)

This is often unnecessary, and arguably should be removed (though it
does have some important use-cases that are otherwise unserved -- I
believe Dagster uses it to ensure that all packages mark imports from
within the same package as first-party, but not imports _across_
different first-party packages)... but it does exist, and it does help
in cases in which the `src` field is not properly configured.

This PR adds an option to turn off this behavior:

```toml
[tool.ruff.isort]
detect-same-package = false
```

This is being introduced to help codebases migrating over from isort
that may want more consistent behavior with their current sorting.

## Test Plan

`cargo test`
2023-08-24 14:09:26 -04:00
Zanie Blue
3bd199cdf6 Update function-call-in-argument-default (B008) to ignore arguments with immutable annotations (#6784)
Extends #6781 
Part of https://github.com/astral-sh/ruff/issues/3762
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->
Allows calls in argument defaults if the argument is annotated as an
immutable type to avoid false positives.

## Test Plan

<!-- How was it tested? -->

Snapshots
2023-08-24 11:12:59 -05:00
Harutaka Kawamura
948cd29b23 Skip serializing cell ID if it's None (#6851)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

Fix #6834 

## Test Plan

<!-- How was it tested? -->

Need tests?

---------

Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
2023-08-24 13:20:25 +00:00
Micha Reiser
1e7d1968b1 Printer: Slice based queue and stack (#6819) 2023-08-24 14:49:27 +02:00
Micha Reiser
8b46b71038 Fix parenthesizing of implicit strings (#6852) 2023-08-24 12:31:02 +00:00
Micha Reiser
1cd7790a8a Use BestFits for non-fluent attribute chains (#6817) 2023-08-24 14:09:25 +02:00
konsti
d376cb4c2a Improve formatter contributor docs (#6776)
The docs were out of date, and the new version incorporates some
feedback.

I tried to keep the language concise and the information ordered by how
early you need it, so people can get the relevant information quickly
before jumping into the code.

I did some minor format_dev changes for consistency in the docs.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2023-08-24 10:45:08 +00:00
Micha Reiser
04a9a8dd03 Maybe parenthesize long constants and names (#6816) 2023-08-24 09:47:57 +00:00
Micha Reiser
e4c13846e3 Fix Printer group modes (#6815) 2023-08-24 08:42:59 +02:00
Harutaka Kawamura
205d234856 Format PatternMatchStar (#6653) 2023-08-24 01:58:05 +00:00
Charlie Marsh
4889b84338 Document logger-objects setting in flake8-logging-format rules (#6832)
Closes https://github.com/astral-sh/ruff/issues/6764.
2023-08-24 00:18:51 +00:00
Charlie Marsh
847432cacf Avoid attempting to fix PT018 in multi-statement lines (#6829)
## Summary

These fixes will _always_ fail, so we should avoid trying to construct
them in the first place.

Closes https://github.com/astral-sh/ruff/issues/6812.
2023-08-23 19:09:34 -04:00
Charlie Marsh
9b6e008cf1 Remove remaining usages of try_set_fix (#6828)
There are just a few remaining usages of this deprecated method.
2023-08-23 22:36:09 +00:00
Charlie Marsh
39c6665ff9 Remove remaining usage of set_fix_from_edit (#6827)
This method is deprecated; we have one last usage, so removing it.
2023-08-23 18:25:39 -04:00
konsti
19a87c220a Document ecosystem_all_check.sh input json (#6824)
Document the origin of `github_search.jsonl` in `ecosystem_all_check.sh`
2023-08-23 20:08:52 +00:00
Zanie Blue
0688883404 Fix uncessary-coding-comment fix when there's leading content (#6775)
Closes https://github.com/astral-sh/ruff/issues/6756

Including whitespace, code, and continuations.
2023-08-23 12:00:06 -05:00
Zanie Blue
3bb638875f Fix native-literals handling of int literal with attribute access (#6792)
Closes https://github.com/astral-sh/ruff/issues/6788 by special casing
integer literals with attribute access — either retaining parenthesis
for literals with values (e.g. `int(7).denominator` to
`(7).denominator)` or leaving calls without values (e.g.
`int().denominator`) unchanged.
2023-08-23 11:22:05 -05:00
Charlie Marsh
26e63ab137 Remove lexing from flake8-pytest-style (#6795)
## Summary

Another drive-by change to remove unnecessary custom lexing. We just
need to know the parenthesized range, so we can use...
`parenthesized_range`. I've also updated `parenthesized_range` to
support nested parentheses.

## Test Plan

`cargo test`
2023-08-23 15:54:11 +00:00
Zanie Blue
417a1d0717 Update mutable-argument-default (B006) to use extend-immutable-calls when determining if annotations are immutable (#6781)
Part of https://github.com/astral-sh/ruff/issues/3762
2023-08-23 15:44:35 +00:00
Micha Reiser
34b2ae73b4 Extend BestFitting with mode (#6814) 2023-08-23 17:23:45 +02:00
Charlie Marsh
71c25e4f9d Implement FormatPatternMatchValue (#6799)
## Summary

This is effectively #6608, but with additional tests.

We aren't properly handling parenthesized patterns, but that needs to be
dealt with separately as it's somewhat involved.

Closes #6555
2023-08-23 14:01:14 +00:00
Micha Reiser
4bdd99f882 Fix: Re-add missing node start positions (#6780) 2023-08-23 09:59:36 +02:00
Charlie Marsh
1e6d1182bf Improve comment handling around PatternMatchAs (#6797)
## Summary

Follows up on
https://github.com/astral-sh/ruff/pull/6652#discussion_r1300871033 with
some modifications to the `PatternMatchAs` comment handling.
Specifically, any comments between the `as` and the end are now
formatted as dangling, and we now insert some newlines in the
appropriate places.

## Test Plan

`cargo test`
2023-08-23 04:48:20 +00:00
Charlie Marsh
d08f697a04 Remove lexing for colon-matching use cases (#6803)
It's much simpler to just search ahead for the first colon.
2023-08-23 04:44:51 +00:00
Charlie Marsh
4bc5eddf91 Handle open-parenthesis comments on match case (#6798)
## Summary

Ensures that we retain the open-parenthesis comment in cases like:
```python
match pattern_comments:
    case (  # leading
        only_leading
    ):
        ...
```

Previously, this was treated as a leading comment on `only_leading`.

## Test Plan

`cargo test`
2023-08-23 00:40:18 -04:00
Charlie Marsh
5f5de52aba Confine repeated-equality-comparison-target to names and attributes (#6802)
Empirically, Pylint does this, so seems reasonable to follow.
2023-08-23 03:56:37 +00:00
Tom Kuson
1cb1bd731c Extend repeated-equality-comparison-target to check for mixed orderings and Yoda conditions. (#6691) 2023-08-23 03:45:30 +00:00
Dhruv Manilawala
db2e548f4f Simplify ANN204 autofix to use Parameters range (#6793)
## Summary

This PR fixes the bug where the decorator parentheses weren't being considered
when computing the autofix for `ANN204`. The existing logic would only look
for balanced parentheses and not multiple pairs of parentheses.

The solution is to remove the logic to generate the autofix and use the
`Parameters` end range directly which includes the parentheses as well.

## Test Plan

Add test case for `ANN204` with decorator being called

fixes: #6790
2023-08-23 09:05:46 +05:30
Harutaka Kawamura
94f5f18ddb Format PatternMatchSequence (#6676) 2023-08-23 00:44:33 +00:00
Luc Khai Hai
c34a342ab4 Format PatternMatchAs (#6652)
## Summary

Add formatting for `PatternMatchAs`.

This closes #6641.

## Test Plan

Add tests for comments.
2023-08-22 23:58:15 +00:00
Charlie Marsh
42ff833d00 Remove comment lexing from isort (#6794)
## Summary

No need to lex to find comments -- we already know their locations via
`Indexer`.
2023-08-22 21:26:38 +00:00
Konrad Listwan-Ciesielski
e1f4438498 Add docs for DTZ007 (#6757) 2023-08-22 21:12:50 +00:00
Charlie Marsh
1acdec3e29 Add a note on __future__ imports and keep-runtime-typing to pyupgrade rules (#6746)
Closes https://github.com/astral-sh/ruff/issues/6740.
2023-08-22 17:04:00 -04:00
Charlie Marsh
ca2bb20063 Fallback to end-of-file if ends in trailing continuation (#6789)
## Summary

Given:

```python
def end_of_file():
    if False:
        return 1
    x = 2 \

```

Then when searching for the end of the `x = 2` statement, we'd reach a
panic as we'd hit the last line (`\\`) and abort, since the universal
iterator doesn't return trailing newlines. Instead, we should just use
the end of the file as the fallback.

Closes https://github.com/astral-sh/ruff/issues/6787.

## Test Plan

`cargo test`
2023-08-22 15:12:26 -04:00
Dhruv Manilawala
2e00983762 Avoid C417 for lambda with default and variadic parameters (#6752)
## Summary

Avoid `C417` for `lambda` with default and variadic parameters.

## Test Plan

`cargo test` and checking if it generates any autofix errors as test
cases
for `lambda` with default parameters already exists.

fixes: #6715
2023-08-23 00:38:08 +05:30
Dhruv Manilawala
fb7caf43c8 Update lexer tests to use snapshots (#6658)
## Summary

This PR updates the lexer tests to use the snapshot testing framework.
It also
makes the following changes:
* Remove the use of macros in the lexer tests
* Use `test_case` for EOL tests

## Test Plan

```
cargo test --package ruff_python_parser --lib --all-features -- lexer::tests --no-capture
```
2023-08-22 18:23:19 +00:00
konsti
e53bf25616 Update formatter ecosystem checks revisions (#6770)
With https://github.com/django/django/pull/17181 merged, this removes an
odd edge case (tuple expression statements aka bogus trailing commas
after statements that turn them into a tuple without you noticing) that
we don't want to care about because the input code is ~wrong from the
similarity index. I've took this opportunity to update the revisions of
all projects we test.

main

| project      | similarity index |
|--------------|------------------|
| cpython      | 0.75477          |
| django       | 0.99814          |
| transformers | 0.99621          |
| twine        | 0.99876          |
| typeshed     | 0.99953          |
| warehouse    | 0.99601          |
| zulip        | 0.99727          |

this PR

| project      | similarity index |
|--------------|------------------|
| cpython      | 0.75996          |
| django       | 0.99819          |
| transformers | 0.99622          |
| twine        | 0.99876          |
| typeshed     | 0.99953          |
| warehouse    | 0.99607          |
| zulip        | 0.99729          |
2023-08-22 14:19:10 -04:00
Charlie Marsh
214eb707a6 Parenthesize expressions prior to LibCST parsing (#6742)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

This PR adds a utility for transforming expressions via LibCST that
automatically wraps the expression in parentheses, applies a
user-provided transformation, then strips the parentheses from the
generated code. LibCST can't parse arbitrary expression ranges, since
some expressions may require parenthesization in order to be parsed
properly. For example:

```python
option = (
    '{name}={value}'
    .format(nam=name, value=value)
)
```

In this case, the expression range is:

```python
'{name}={value}'
    .format(nam=name, value=value)
```

Which isn't valid on its own. So, instead, we add "fake" parentheses
around the expression.

We were already doing this in a few places, so this is mostly
formalizing and DRYing up that pattern.

Closes https://github.com/astral-sh/ruff/issues/6720.
2023-08-22 17:45:05 +00:00
Zanie Blue
5c1f7fd5dd Add networkx to conventional aliases (#6778)
Closes https://github.com/astral-sh/ruff/issues/6763
2023-08-22 11:49:04 -05:00
Charlie Marsh
cc278c24e2 Allow up to two empty lines after top-level imports (#6777)
## Summary

For imports, we enforce that there's _at least_ one empty line after an
import (assuming the next statement is _not_ an import), but allow up to
two at the module level.

Closes https://github.com/astral-sh/ruff/issues/6760.

## Test Plan

`cargo test`
2023-08-22 12:27:40 -04:00
Charlie Marsh
558b56f8a8 Avoid fixing D200 for docstrings that end in escapes (#6779)
Appease the fuzzers! Closes
https://github.com/astral-sh/ruff/issues/6755.
2023-08-22 16:25:37 +00:00
Charlie Marsh
749da6589a Fix isolation groups for unused imports (#6774)
## Summary

The isolation group for unused imports was relying on
`checker.semantic().current_statement()`, which isn't valid for that
rule, since it runs over the _scope_, not the statement. Instead, we
need to lookup the isolation group based on the `NodeId` of the
statement.

Our tests didn't catch this, because we mostly have cases that look like
this:

```python
if TYPE_CHECKING:
    import shelve
    import importlib
```

In this case, the two fixes to remove the two unused imports are
considered overlapping (since we delete the _full_ line, and the two
_full_ lines touch, and we consider exactly-adjacent fixes to be
overlapping), and so they don't run in a single pass due to the
non-overlapping-fixes requirement. That is: the isolation groups aren't
required for this case. They are, however, required for cases like:

```python
if TYPE_CHECKING:
    import shelve

    import importlib
```

...where the fixes don't overlap.

Closes https://github.com/astral-sh/ruff/issues/6758.

## Test Plan

`cargo test`
2023-08-22 11:55:27 -04:00
Charlie Marsh
d2eace3377 Prefer range_* edit methods (#6751) 2023-08-22 15:46:04 +00:00
Micha Reiser
ccac9681e1 Preserve yield parentheses (#6766) 2023-08-22 10:27:20 +00:00
Micha Reiser
b52cc84df6 Omit tuple parentheses in for statements except when absolutely necessary (#6765) 2023-08-22 12:18:59 +02:00
Micha Reiser
fec6fc2fab Preserve empty lines between try clause headers (#6759) 2023-08-22 11:50:28 +02:00
konsti
ba4c27598a Document IO Error (#6712)
`IOError` is special, it is not actually a lint but an error before
linting. I'm not entirely sure how to document it since it does not
match the general lint rule pattern (`Checks that the file can be read
in its entirety.` is imho worse).

I added the in my experience two most common reasons for io errors on
unix systems and linked two tutorials on how to fix them.

See https://github.com/astral-sh/ruff/issues/2646

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2023-08-22 11:46:18 +02:00
Victor Hugo Gomes
0f9ccfcad9 Format PatternMatchSingleton (#6741) 2023-08-22 08:23:47 +02:00
Charlie Marsh
fa32cd9b6f Truncate some messages in diagnostics (#6748)
## Summary

I noticed this in the ecosystem CI check from
https://github.com/astral-sh/ruff/pull/6742. If we include source code
directly in a diagnostic, we need to be careful to avoid rendering
multi-line diagnostics or even excessively long diagnostics.

## Test Plan

`cargo test`
2023-08-21 23:46:24 -04:00
Victor Hugo Gomes
0aad0c41f6 [pylint] Implement no-self-use (R6301) (#6574) 2023-08-22 03:44:38 +00:00
Charlie Marsh
424b8d4ad2 Use a single node hierarchy to track statements and expressions (#6709)
## Summary

This PR is a follow-up to the suggestion in
https://github.com/astral-sh/ruff/pull/6345#discussion_r1285470953 to
use a single stack to store all statements and expressions, rather than
using separate vectors for each, which gives us something closer to a
full-fidelity chain. (We can then generalize this concept to include all
other AST nodes too.)

This is in part made possible by the removal of the hash map from
`&Stmt` to `StatementId` (#6694), which makes it much cheaper to store
these using a single interface (since doing so no longer introduces the
requirement that we hash all expressions).

I'll follow-up with some profiling, but a few notes on how the data
requirements have changed:

- We now store a `BranchId` for every expression, not just every
statement, so that's an extra `u32`.
- We now store a single `NodeId` on every snapshot, rather than separate
`StatementId` and `ExpressionId` IDs, so that's one fewer `u32` for each
snapshot.
- We're probably doing a few more lookups in general, since any calls to
`current_statement()` etc. now have to iterate up the node hierarchy
until they identify the first statement.

## Test Plan

`cargo test`
2023-08-21 21:32:57 -04:00
Charlie Marsh
abc5065fc7 Avoid E231 if comma is at end-of-line (#6747)
## Summary

I don't know how this could come up in valid Python, but anyway...

Closes https://github.com/astral-sh/ruff/issues/6738.
2023-08-21 20:47:20 -04:00
Victor Hugo Gomes
37f4920e1e Don't trigger eq-without-hash when __hash__ is explicitly set to None (#6739) 2023-08-21 23:51:21 +00:00
Charlie Marsh
c0df99b965 Avoid attempting to fix unconventional submodule imports (#6745)
## Summary

Avoid attempting to rewrite `import matplotlib.pyplot` as `import
matplotlib.pyplot as plt`. We can't support these right now, since we
don't track references at the attribute level (like
`matplotlib.pyplot`).

Closes https://github.com/astral-sh/ruff/issues/6719.
2023-08-21 23:45:32 +00:00
Charlie Marsh
7650c6ee45 Support C419 autofixes for set comprehensions (#6744)
Closes https://github.com/astral-sh/ruff/issues/6713.
2023-08-21 23:41:13 +00:00
Charlie Marsh
7b14d17e39 Ignore star imports when importing symbols in fixes (#6743)
## Summary

Given:

```python
from sys import *

exit(0)
```

We can't add `exit` to `from sys import *`, so we should just ignore it.
Ideally, we'd just resolve `exit` in the first place (since it's
imported from `from sys import *`), but as long as we don't support
wildcard imports, this is more consistent.

Closes https://github.com/astral-sh/ruff/issues/6718.

## Test Plan

`cargo test`
2023-08-21 23:31:30 +00:00
Charlie Marsh
4678f7dafe Remove parenthesis lexing in RSE102 (#6732)
## Summary

Now that we have an `Arguments` node, we can just use the range of the
arguments directly to find the parentheses in `raise Error()`.
2023-08-21 20:59:06 +00:00
konsti
b182368008 Simplify suite formatting (#6722)
Avoid the nesting in a macro by using the new `WithNodeLevel` to
`PyFormatter` deref. No changes otherwise.

I wanted to follow this up with quickly fixing the typeshed empty line
rules but they turned out a lot more complex than i had anticipated.
2023-08-21 21:01:51 +02:00
Charlie Marsh
e032fbd2e7 Remove remove_super_arguments (#6735)
Now that we have an `Arguments` node, we can use it directly to get the
range.
2023-08-21 13:04:07 -04:00
dependabot[bot]
575b77aa52 ci(deps): bump cloudflare/wrangler-action from 3.0.2 to 3.1.0 (#6736)
Bumps
[cloudflare/wrangler-action](https://github.com/cloudflare/wrangler-action)
from 3.0.2 to 3.1.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/cloudflare/wrangler-action/releases">cloudflare/wrangler-action's
releases</a>.</em></p>
<blockquote>
<h2>v3.1.0</h2>
<h3>Minor Changes</h3>
<ul>
<li>
<p><a
href="https://redirect.github.com/cloudflare/wrangler-action/pull/154">#154</a>
<a
href="3f40637a1c"><code>3f40637</code></a>
Thanks <a
href="https://github.com/JacobMGEvans"><code>@​JacobMGEvans</code></a>!
- feat: Quiet mode
Some of the stderr, stdout, info &amp; groupings can be a little noisy
for some users and use cases.
This feature allows for a option to be passed 'quiet: true' this would
significantly reduce the noise.</p>
<p>There will still be output that lets the user know Wrangler Installed
and Wrangler Action completed successfully.
Any failure status will still be output to the user as well, to prevent
silent failures.</p>
<p>resolves <a
href="https://redirect.github.com/cloudflare/wrangler-action/issues/142">#142</a></p>
</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/cloudflare/wrangler-action/blob/main/CHANGELOG.md">cloudflare/wrangler-action's
changelog</a>.</em></p>
<blockquote>
<h2>3.1.0</h2>
<h3>Minor Changes</h3>
<ul>
<li>
<p><a
href="https://redirect.github.com/cloudflare/wrangler-action/pull/154">#154</a>
<a
href="3f40637a1c"><code>3f40637</code></a>
Thanks <a
href="https://github.com/JacobMGEvans"><code>@​JacobMGEvans</code></a>!
- feat: Quiet mode
Some of the stderr, stdout, info &amp; groupings can be a little noisy
for some users and use cases.
This feature allows for a option to be passed 'quiet: true' this would
significantly reduce the noise.</p>
<p>There will still be output that lets the user know Wrangler Installed
and Wrangler Action completed successfully.
Any failure status will still be output to the user as well, to prevent
silent failures.</p>
<p>resolves <a
href="https://redirect.github.com/cloudflare/wrangler-action/issues/142">#142</a></p>
</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="c25aadc965"><code>c25aadc</code></a>
Automatic compilation</li>
<li><a
href="fcf648c789"><code>fcf648c</code></a>
Merge pull request <a
href="https://redirect.github.com/cloudflare/wrangler-action/issues/158">#158</a>
from cloudflare/changeset-release/main</li>
<li><a
href="fcbabec21e"><code>fcbabec</code></a>
Version Packages</li>
<li><a
href="0aa12f0c2b"><code>0aa12f0</code></a>
Merge pull request <a
href="https://redirect.github.com/cloudflare/wrangler-action/issues/154">#154</a>
from cloudflare/jacobmgevans/silence-mode</li>
<li><a
href="ad7441b6ad"><code>ad7441b</code></a>
Merge pull request <a
href="https://redirect.github.com/cloudflare/wrangler-action/issues/157">#157</a>
from EstebanBorai/main</li>
<li><a
href="3f40637a1c"><code>3f40637</code></a>
Quiet feature</li>
<li><a
href="4132892387"><code>4132892</code></a>
fix: use <code>wrangler@3.5.1</code> by default</li>
<li><a
href="62ce9d23a3"><code>62ce9d2</code></a>
Merge pull request <a
href="https://redirect.github.com/cloudflare/wrangler-action/issues/155">#155</a>
from ethanppl/fix-readme</li>
<li><a
href="f089b0a195"><code>f089b0a</code></a>
Update README.md</li>
<li><a
href="4318a2fb97"><code>4318a2f</code></a>
Fix examples in README.md</li>
<li>Additional commits viewable in <a
href="https://github.com/cloudflare/wrangler-action/compare/v3.0.2...v3.1.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=cloudflare/wrangler-action&package-manager=github_actions&previous-version=3.0.2&new-version=3.1.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-21 16:35:59 +00:00
Micha Reiser
17a26e6ff3 Fix fmt:skip for function with return type (#6733) 2023-08-21 17:45:23 +02:00
Charlie Marsh
d5a51b4e45 Allow ctypes.WinError() in flake8-raise (#6731)
Closes https://github.com/astral-sh/ruff/issues/6730.
2023-08-21 14:57:34 +00:00
Charlie Marsh
83f68891e0 Allow next in FBT exclusions (#6729)
Closes https://github.com/astral-sh/ruff/issues/6711.
2023-08-21 14:56:38 +00:00
konsti
aafde6db28 Remove some indexing (#6728)
**Summary** A common pattern in the code used to be
```rust
if statements.len() != 1 {
    return;
}
use_single_entry(statements[0])?;
```
which can be better expressed as
```rust
let [statement] = statements else {
    return;
};
use_single_entry(statements)?;
```

Direct indexing can cause panics if you don't manually take care of
checking the length, while matching (such as if-let or let-else) can
never panic.

This isn't a complete refactor, i've just removed some of the obvious
cases. I've specifically looked for `.len() != 1` and fixed those.

**Test Plan** No functional changes
2023-08-21 16:56:15 +02:00
Charlie Marsh
2405536d03 Remove unnecessary LibCST usage in key-in-dict (#6727)
## Summary

We're using LibCST to ensure that we return the full parenthesized range
of an expression, for display purposes. We can just use
`parenthesized_range` which is more efficient and removes one LibCST
dependency.

## Test Plan

`cargo test`
2023-08-21 10:32:09 -04:00
Micha Reiser
f017555d53 Parenthesize NamedExpr if target breaks (#6714) 2023-08-21 16:29:26 +02:00
Charlie Marsh
be96e0041a Accept empty inner calls in C414 (#6725)
Closes https://github.com/astral-sh/ruff/issues/6716.
2023-08-21 14:05:09 +00:00
Harutaka Kawamura
3c2dd5e42e Remove confusing comment on get_parametrize_name_range (#6724) 2023-08-21 08:52:48 -04:00
Micha Reiser
8b347cdaa9 Simplify IfRequired needs parentheses condition (#6678) 2023-08-21 07:11:31 +00:00
Tom Kuson
2a8d24dd4b Format function and class definitions into a single line if its body is an ellipsis (#6592) 2023-08-21 09:02:23 +02:00
Charlie Marsh
bb5fbb1b5c Use simple lexer for argument removal (#6710) 2023-08-21 04:16:29 +00:00
Harutaka Kawamura
086e11087f [flake8-pytest-style] Autofix PT014 (#6698) 2023-08-21 03:45:12 +00:00
Charlie Marsh
1b7e4a12a9 Refactor remove_unused_variable to take &Binding (#6707) 2023-08-20 15:50:57 +00:00
Charlie Marsh
da1697121e Add BranchId to the model snapshot (#6706)
This _probably_ never matters given the set of rules we support and in
fact I'm having trouble thinking of a test-case for it, but it's
definitely incorrect _not_ to pass on the `BranchId` here.
2023-08-20 15:35:49 +00:00
Harutaka Kawamura
419615f29b Add docs for E275, E231, E251, and E252 (#6700) 2023-08-20 14:51:50 +00:00
Charlie Marsh
a742a562fd Ignore multi-comparisons in repeated-equality-comparison-target (#6705)
Given `foo == "a" == "b" or foo == "c"`, we were suggesting `foo in
{"a", "b", "c"}`.
2023-08-20 14:41:10 +00:00
Harutaka Kawamura
129b19050a Refactor flake8_pytest_style/rules/parametrize.rs (#6703) 2023-08-20 14:30:26 +00:00
Konrad Listwan-Ciesielski
0dc23da1d0 Add docs for DTZ011 and DTZ012 (#6688) 2023-08-20 10:21:10 -04:00
Harutaka Kawamura
c62e544cba Add doc for E999 (#6699) 2023-08-20 14:14:22 +00:00
Charlie Marsh
7e9023b6f8 Use typing_extensions.TypeAlias for PYI026 fixes on pre-3.10 (#6696)
Closes https://github.com/astral-sh/ruff/issues/6695.
2023-08-19 22:16:44 +00:00
Harutaka Kawamura
a489b96a65 [flake8-pie] Implement unnecessary-range-start (PIE808) (#6690) 2023-08-19 21:59:11 +00:00
Charlie Marsh
17af12e57c Add branch detection to the semantic model (#6694)
## Summary

We have a few rules that rely on detecting whether two statements are in
different branches -- for example, different arms of an `if`-`else`.
Historically, the way this was implemented is that, given two statement
IDs, we'd find the common parent (by traversing upwards via our
`Statements` abstraction); then identify branches "manually" by matching
the parents against `try`, `if`, and `match`, and returning iterators
over the arms; then check if there's an arm for which one of the
statements is a child, and the other is not.

This has a few drawbacks:

1. First, the code is generally a bit hard to follow (Konsti mentioned
this too when working on the `ElifElseClause` refactor).

2. Second, this is the only place in the codebase where we need to go
from `&Stmt` to `StatementID` -- _everywhere_ else, we only need to go
in the _other_ direction. Supporting these lookups means we need to
maintain a mapping from `&Stmt` to `StatementID` that includes every
`&Stmt` in the program. (We _also_ end up maintaining a `depth` level
for every statement.) I'd like to get rid of these requirements to
improve efficiency, reduce complexity, and enable us to treat AST modes
more generically in the future. (When I looked at adding the `&Expr` to
our existing statement-tracking infrastructure, maintaining a hash map
with all the statements noticeably hurt performance.)

The solution implemented here instead makes branches a first-class
concept in the semantic model. Like with `Statements`, we now have a
`Branches` abstraction, where each branch points to its optional parent.
When we store statements, we store the `BranchID` alongside each
statement. When we need to detect whether two statements are in the same
branch, we just realize each statement's branch path and compare the
two. (Assuming that the two statements are in the same scope, then
they're on the same branch IFF one branch path is a subset of the other,
starting from the top.) We then add some calls to the visitor to push
and pop branches in the appropriate places, for `if`, `try`, and `match`
statements.

Note that a branch is not 1:1 with a statement; instead, each branch is
closer to a suite, but not _every_ suite is a branch. For example, each
arm in an `if`-`elif`-`else` is a branch, but the `else` in a `for` loop
is not considered a branch.

In addition to being much simpler, this should also be more efficient,
since we've shed the entire `&Stmt` hash map, plus the `depth` that we
track on `StatementWithParent` in favor of a single `Option<BranchID>`
on `StatementWithParent` plus a single vector for all branches. The
lookups should be faster too, since instead of doing a bunch of jumps
around with the hash map + repeated recursive calls to find the common
parents, we instead just do a few simple lookups in the `Branches`
vector to realize and compare the branch paths.

## Test Plan

`cargo test` -- we have a lot of coverage for this, which we inherited
from PyFlakes
2023-08-19 21:28:17 +00:00
Chris Pryer
648333b8b2 ruff_formatter crate doc comment fixes (#6677) 2023-08-19 17:42:02 +01:00
Charlie Marsh
3849fa0cf1 Rewrite yield-in-for-loop to avoid recursing over body (#6692)
## Summary

This is much simpler and avoids (1) multiple passes over the entire
function body, (2) requiring the rule to do its own binding tracking (we
can just use the semantic model), and (3) a usage of `StatementKey`.

In general, where we can, we should try to remove these kinds of custom
visitors that track name references, and instead rely on the semantic
model.

## Test Plan

`cargo test`
2023-08-19 11:25:29 -04:00
Victor Hugo Gomes
59e533047a Fix typo in ruff_python_formatter documentation (#6687)
## Summary

In the documentation was written `Javascript` but we are working with
`Python` here :)

## Test Plan

n/a
2023-08-18 19:16:09 -04:00
Charlie Marsh
053b1145f0 Avoid panic in unused arguments rule for parameter-free lambda (#6679)
## Summary

This was just a mistake in pattern-matching with no test coverage.

## Test Plan

`cargo test`
2023-08-18 18:29:31 +00:00
Charlie Marsh
6a5acde226 Make Parameters an optional field on ExprLambda (#6669)
## Summary

If a lambda doesn't contain any parameters, or any parameter _tokens_
(like `*`), we can use `None` for the parameters. This feels like a
better representation to me, since, e.g., what should the `TextRange` be
for a non-existent set of parameters? It also allows us to remove
several sites where we check if the `Parameters` is empty by seeing if
it contains any arguments, so semantically, we're already trying to
detect and model around this elsewhere.

Changing this also fixes a number of issues with dangling comments in
parameter-less lambdas, since those comments are now automatically
marked as dangling on the lambda. (As-is, we were also doing something
not-great whereby the lambda was responsible for formatting dangling
comments on the parameters, which has been removed.)

Closes https://github.com/astral-sh/ruff/issues/6646.

Closes https://github.com/astral-sh/ruff/issues/6647.

## Test Plan

`cargo test`
2023-08-18 15:34:54 +00:00
Micha Reiser
ea72d5feba Refactor SourceKind to store file content (#6640) 2023-08-18 13:45:38 +00:00
Charlie Marsh
2aeb27334d Avoid cloning source code multiple times (#6629)
## Summary

In working on https://github.com/astral-sh/ruff/pull/6628, I noticed
that we clone the source code contents, potentially multiple times,
prior to linting. The issue is that `SourceKind::Python` takes a
`String`, so we first have to provide it with a `String`. In the stdin
case, that means cloning. However, on top of this, we then have to clone
`source_kind.contents()` because `SourceKind` gets mutated. So for
stdin, we end up cloning twice. For non-stdin, we end up cloning once,
but unnecessarily (since the _contents_ don't get mutated, only the
kind).

This PR removes the `String` from `source_kind`, instead requiring that
we parse it out elsewhere. It reduces the number of clones down to 1 for
Jupyter Notebooks, and zero otherwise.
2023-08-18 09:32:18 -04:00
Micha Reiser
0cea4975fc Rename Comments methods (#6649) 2023-08-18 06:37:01 +00:00
Charlie Marsh
3ceb6fbeb0 Remove some unnecessary ampersands in the formatter (#6667) 2023-08-18 04:18:26 +00:00
Charlie Marsh
8e18f8018f Remove some trailing commas in write calls (#6666) 2023-08-18 00:14:44 -04:00
Charlie Marsh
8228429a70 Convert comment to rustdoc in placement.rs (#6665) 2023-08-18 04:11:38 +00:00
Charlie Marsh
1811312722 Improve with statement comment handling and expression breaking (#6621)
## Summary

The motivating code here was:

```python
with test as (
    # test
foo):
    pass
```

Which we were formatting as:

```python
with test as
# test
(foo):
    pass
```

`with` statements are oddly difficult. This PR makes a bunch of subtle
modifications and adds a more extensive test suite. For example, we now
only preserve parentheses if there's more than one `WithItem` _or_ a
trailing comma; before, we always preserved.

Our formatting is_not_ the same as Black, but here's a diff of our
formatted code vs. Black's for the `with.py` test suite. The primary
difference is that we tend to break parentheses when they contain
comments rather than move them to the end of the life (this is a
consistent difference that we make across the codebase):

```diff
diff --git a/crates/ruff_python_formatter/foo.py b/crates/ruff_python_formatter/foo.py
index 85e761080..31625c876 100644
--- a/crates/ruff_python_formatter/foo.py
+++ b/crates/ruff_python_formatter/foo.py
@@ -1,6 +1,4 @@
-with (
-    aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
-), aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:
+with aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa, aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:
     ...
     # trailing
 
@@ -16,28 +14,33 @@ with (
     # trailing
 
 
-with a, b:  # a  # comma  # c  # colon
+with (
+    a,  # a  # comma
+    b,  # c
+):  # colon
     ...
 
 
 with (
-    a as  # a  # as
-    # own line
-    b,  # b  # comma
+    a as (  # a  # as
+        # own line
+        b
+    ),  # b  # comma
     c,  # c
 ):  # colon
     ...  # body
     # body trailing own
 
-with (
-    a as  # a  # as
+with a as (  # a  # as
     # own line
-    bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb  # b
-):
+    bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
+):  # b
     pass
 
 
-with (a,):  # magic trailing comma
+with (
+    a,
+):  # magic trailing comma
     ...
 
 
@@ -47,6 +50,7 @@ with a:  # should remove brackets
 with aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb as c:
     ...
 
+
 with (
     # leading comment
     a
@@ -74,8 +78,7 @@ with (
 with (
     a  # trailing same line comment
     # trailing own line comment
-    as b
-):
+) as b:
     ...
 
 with (
@@ -87,7 +90,9 @@ with (
 with (
     a
     # trailing own line comment
-) as b:  # trailing as same line comment  # trailing b same line comment
+) as (  # trailing as same line comment
+    b
+):  # trailing b same line comment
     ...
 
 with (
@@ -124,18 +129,24 @@ with (  # comment
     ...
 
 with (  # outer comment
-    CtxManager1() as example1,  # inner comment
+    (  # inner comment
+        CtxManager1()
+    ) as example1,
     CtxManager2() as example2,
     CtxManager3() as example3,
 ):
     ...
 
-with CtxManager() as example:  # outer comment
+with (  # outer comment
+    CtxManager()
+) as example:
     ...
 
 with (  # outer comment
     CtxManager()
-) as example, CtxManager2() as example2:  # inner comment
+) as example, (  # inner comment
+    CtxManager2()
+) as example2:
     ...
 
 with (  # outer comment
@@ -145,7 +156,9 @@ with (  # outer comment
     ...
 
 with (  # outer comment
-    (CtxManager1()),  # inner comment
+    (  # inner comment
+        CtxManager1()
+    ),
     CtxManager2(),
 ) as example:
     ...
@@ -179,7 +192,9 @@ with (
 ):
     pass
 
-with a as (b):  # foo
+with a as (  # foo
+    b
+):
     pass
 
 with f(
@@ -209,17 +224,13 @@ with f(
 ) as b, c as d:
     pass
 
-with (
-    aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
-) as b:
+with aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb as b:
     pass
 
 with aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb as b:
     pass
 
-with (
-    aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
-) as b, c as d:
+with aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb as b, c as d:
     pass
 
 with (
@@ -230,6 +241,8 @@ with (
     pass
 
 with (
-    aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
-) as b, c as d:
+    aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
+    + bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb as b,
+    c as d,
+):
     pass
```

Closes https://github.com/astral-sh/ruff/issues/6600.
## Test Plan

Before:

| project      | similarity index |
|--------------|------------------|
| cpython      | 0.75473          |
| django       | 0.99804          |
| transformers | 0.99618          |
| twine        | 0.99876          |
| typeshed     | 0.74292          |
| warehouse    | 0.99601          |
| zulip        | 0.99727          |

After:

| project      | similarity index |
|--------------|------------------|
| cpython      | 0.75473          |
| django       | 0.99804          |
| transformers | 0.99618          |
| twine        | 0.99876          |
| typeshed     | 0.74292          |
| warehouse    | 0.99601          |
| zulip        | 0.99727          |

`cargo test`
2023-08-18 03:30:38 +00:00
Charlie Marsh
26bba11be6 Manually format comments around := in named expressions (#6634)
## Summary

Attaches comments around the `:=` operator in a named expression as
dangling, and formats them manually in the `named_expr.rs` formatter.

Closes https://github.com/astral-sh/ruff/issues/5695.

## Test Plan

`cargo test`
2023-08-18 03:10:45 +00:00
Shantanu
a128fe5148 Apply RUF017 when start is passed via position (#6664)
As discussed in
https://github.com/astral-sh/ruff/pull/6489#discussion_r1297858919.
Linking https://github.com/astral-sh/ruff/issues/5073
2023-08-17 20:10:07 -04:00
Zanie Blue
5892c691ea Bump version to 0.0.285 (#6660)
Requires
- https://github.com/astral-sh/ruff/pull/6655
- https://github.com/astral-sh/ruff/pull/6657
2023-08-17 15:46:28 -05:00
Zanie Blue
82e0a97b34 Clarify behavior of PLW3201 (#6657)
Otherwise it is unclear that violations will be raised for methods like
`_foo_`
2023-08-17 14:41:55 -05:00
Zanie Blue
a8d7bbae6f Remove experimental label from Jupyter docs (#6655) 2023-08-17 14:40:50 -05:00
Charlie Marsh
1050142a58 Expand expressions to include parentheses in E712 (#6575)
## Summary

This PR exposes our `is_expression_parenthesized` logic such that we can
use it to expand expressions when autofixing to include their
parenthesized ranges.

This solution has a few drawbacks: (1) we need to compute parenthesized
ranges in more places, which also relies on backwards lexing; and (2) we
need to make use of this in any relevant fixes.

However, I still think it's worth pursuing. On (1), the implementation
is very contained, so IMO we can easily swap this out for a more
performant solution in the future if needed. On (2), this improves
correctness and fixes some bad syntax errors detected by fuzzing, which
means it has value even if it's not as robust as an _actual_
`ParenthesizedExpression` node in the AST itself.

Closes https://github.com/astral-sh/ruff/issues/4925.

## Test Plan

`cargo test` with new cases that previously failed the fuzzer.
2023-08-17 15:51:09 +00:00
Charlie Marsh
db1c556508 Implement Ranged on more structs (#6639)
## Summary

I noticed some inconsistencies around uses of `.range.start()`, structs
that have a `TextRange` field but don't implement `Ranged`, etc.

## Test Plan

`cargo test`
2023-08-17 11:22:39 -04:00
Charlie Marsh
a70807e1e1 Expand NamedExpr range to include full range of parenthesized value (#6632)
## Summary

Given:

```python
if (
    x
    :=
    (  # 4
        y # 5
    )  # 6
):
    pass
```

It turns out the parser ended the range of the `NamedExpr` at the end of
`y`, rather than the end of the parenthesis that encloses `y`. This just
seems like a bug -- the range should be from the start of the name on
the left, to the end of the parenthesized node on the right.

## Test Plan

`cargo test`
2023-08-17 14:34:05 +00:00
dependabot[bot]
d9bb51dee4 ci(deps): bump cloudflare/wrangler-action from 3.0.0 to 3.0.2 (#6565)
Bumps
[cloudflare/wrangler-action](https://github.com/cloudflare/wrangler-action)
from 3.0.0 to 3.0.2.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/cloudflare/wrangler-action/releases">cloudflare/wrangler-action's
releases</a>.</em></p>
<blockquote>
<h2>v3.0.2</h2>
<h3>Patch Changes</h3>
<ul>
<li>
<p><a
href="https://redirect.github.com/cloudflare/wrangler-action/pull/147">#147</a>
<a
href="58f274b9f7"><code>58f274b</code></a>
Thanks <a
href="https://github.com/JacobMGEvans"><code>@​JacobMGEvans</code></a>!
- Added more error logging when a command fails to execute
Previously, we prevented any error logs from propagating too far to
prevent leaking of any potentially sensitive information. However, this
made it difficult for developers to debug their code.</p>
<p>In this release, we have updated our error handling to allow for more
error messaging from pre/post and custom commands. We still discourage
the use of these commands for secrets or other sensitive information,
but we believe this change will make it easier for developers to debug
their code.</p>
<p>Relates to <a
href="https://redirect.github.com/cloudflare/wrangler-action/issues/137">#137</a></p>
</li>
<li>
<p><a
href="https://redirect.github.com/cloudflare/wrangler-action/pull/147">#147</a>
<a
href="58f274b9f7"><code>58f274b</code></a>
Thanks <a
href="https://github.com/JacobMGEvans"><code>@​JacobMGEvans</code></a>!
- Adding Changesets</p>
</li>
<li>
<p><a
href="https://github.com/cloudflare/wrangler-action/blob/HEAD/#version-300">Version
3.0.0</a></p>
</li>
<li>
<p><a
href="https://github.com/cloudflare/wrangler-action/blob/HEAD/#version-200">Version
2.0.0</a></p>
</li>
</ul>
<h2>v3.0.1</h2>
<p>Automating Build &amp; Release</p>
<h2>What's Changed</h2>
<ul>
<li>Artifacts are now ESM supported with <code>.mjs</code></li>
<li>Update publish to deploy by <a
href="https://github.com/lrapoport-cf"><code>@​lrapoport-cf</code></a>
in <a
href="https://redirect.github.com/cloudflare/wrangler-action/pull/124">cloudflare/wrangler-action#124</a></li>
<li>Automatically add issues to workers-sdk GH project by <a
href="https://github.com/lrapoport-cf"><code>@​lrapoport-cf</code></a>
in <a
href="https://redirect.github.com/cloudflare/wrangler-action/pull/127">cloudflare/wrangler-action#127</a></li>
<li>Automate Action Release by <a
href="https://github.com/JacobMGEvans"><code>@​JacobMGEvans</code></a>
in <a
href="https://redirect.github.com/cloudflare/wrangler-action/pull/128">cloudflare/wrangler-action#128</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a
href="https://github.com/lrapoport-cf"><code>@​lrapoport-cf</code></a>
made their first contribution in <a
href="https://redirect.github.com/cloudflare/wrangler-action/pull/124">cloudflare/wrangler-action#124</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/cloudflare/wrangler-action/compare/3.0.0...3.0.1">https://github.com/cloudflare/wrangler-action/compare/3.0.0...3.0.1</a></p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/cloudflare/wrangler-action/blob/main/CHANGELOG.md">cloudflare/wrangler-action's
changelog</a>.</em></p>
<blockquote>
<h2>3.0.2</h2>
<h3>Patch Changes</h3>
<ul>
<li>
<p><a
href="https://redirect.github.com/cloudflare/wrangler-action/pull/147">#147</a>
<a
href="58f274b9f7"><code>58f274b</code></a>
Thanks <a
href="https://github.com/JacobMGEvans"><code>@​JacobMGEvans</code></a>!
- Added more error logging when a command fails to execute
Previously, we prevented any error logs from propagating too far to
prevent leaking of any potentially sensitive information. However, this
made it difficult for developers to debug their code.</p>
<p>In this release, we have updated our error handling to allow for more
error messaging from pre/post and custom commands. We still discourage
the use of these commands for secrets or other sensitive information,
but we believe this change will make it easier for developers to debug
their code.</p>
<p>Relates to <a
href="https://redirect.github.com/cloudflare/wrangler-action/issues/137">#137</a></p>
</li>
<li>
<p><a
href="https://redirect.github.com/cloudflare/wrangler-action/pull/147">#147</a>
<a
href="58f274b9f7"><code>58f274b</code></a>
Thanks <a
href="https://github.com/JacobMGEvans"><code>@​JacobMGEvans</code></a>!
- Adding Changesets</p>
</li>
<li>
<p><a
href="https://github.com/cloudflare/wrangler-action/blob/main/#version-300">Version
3.0.0</a></p>
</li>
<li>
<p><a
href="https://github.com/cloudflare/wrangler-action/blob/main/#version-200">Version
2.0.0</a></p>
</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="80501a7b4d"><code>80501a7</code></a>
Automatic compilation</li>
<li><a
href="3252711404"><code>3252711</code></a>
hotfix</li>
<li><a
href="2faabf36a2"><code>2faabf3</code></a>
Merge pull request <a
href="https://redirect.github.com/cloudflare/wrangler-action/issues/150">#150</a>
from cloudflare/jacobmgevans/changesets-tag-cli</li>
<li><a
href="b734d85d74"><code>b734d85</code></a>
Changesets needs a tag created for release</li>
<li><a
href="8ca2ff1612"><code>8ca2ff1</code></a>
Merge pull request <a
href="https://redirect.github.com/cloudflare/wrangler-action/issues/148">#148</a>
from cloudflare/changeset-release/main</li>
<li><a
href="7d08a8657e"><code>7d08a86</code></a>
Merge pull request <a
href="https://redirect.github.com/cloudflare/wrangler-action/issues/149">#149</a>
from cloudflare/jacobmgevans/changesets-needs-publish</li>
<li><a
href="e193627f19"><code>e193627</code></a>
Spoofing publish for Changeset Action</li>
<li><a
href="55b80c5f62"><code>55b80c5</code></a>
Version Packages</li>
<li><a
href="f61dc4d5a9"><code>f61dc4d</code></a>
Merge pull request <a
href="https://redirect.github.com/cloudflare/wrangler-action/issues/147">#147</a>
from cloudflare/revert-146-changeset-release/main</li>
<li><a
href="58f274b9f7"><code>58f274b</code></a>
Revert &quot;Version Packages&quot;</li>
<li>Additional commits viewable in <a
href="https://github.com/cloudflare/wrangler-action/compare/3.0.0...v3.0.2">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=cloudflare/wrangler-action&package-manager=github_actions&previous-version=3.0.0&new-version=3.0.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-17 09:11:04 -05:00
Zanie Blue
d0f2a8e424 Add support for nested replacements inside format specifications (#6616)
Closes https://github.com/astral-sh/ruff/issues/6442

Python string formatting like `"hello {place}".format(place="world")`
supports format specifications for replaced content such as `"hello
{place:>10}".format(place="world")` which will align the text to the
right in a container filled up to ten characters.

Ruff parses formatted strings into `FormatPart`s each of which is either
a `Field` (content in `{...}`) or a `Literal` (the normal content).
Fields are parsed into name and format specifier sections (we'll ignore
conversion specifiers for now).

There are a myriad of specifiers that can be used in a `FormatSpec`.
Unfortunately for linters, the specifier values can be dynamically set.
For example, `"hello {place:{align}{width}}".format(place="world",
align=">", width=10)` and `"hello {place:{fmt}}".format(place="world",
fmt=">10")` will yield the same string as before but variables can be
used to determine the formatting. In this case, when parsing the format
specifier we can't know what _kind_ of specifier is being used as their
meaning is determined by both position and value.

Ruff does not support nested replacements and our current data model
does not support the concept. Here the data model is updated to support
this concept, although linting of specifications with replacements will
be inherently limited. We could split format specifications into two
types, one without any replacements that we can perform lints with and
one with replacements that we cannot inspect. However, it seems
excessive to drop all parsing of format specifiers due to the presence
of a replacement. Instead, I've opted to parse replacements eagerly and
ignore their possible effect on other format specifiers. This will allow
us to retain a simple interface for `FormatSpec` and most syntax checks.
We may need to add some handling to relax errors if a replacement was
seen previously.

It's worth noting that the nested replacement _can_ also include a
format specification although it may fail at runtime if you produce an
invalid outer format specification. For example, `"hello
{place:{fmt:<2}}".format(place="world", fmt=">10")` is valid so we need
to represent each nested replacement as a full `FormatPart`.

## Test plan

Adding unit tests for `FormatSpec` parsing and snapshots for PLE1300
2023-08-17 09:07:30 -05:00
Charlie Marsh
1334232168 Introduce ExpressionRef (#6637)
## Summary

This PR revives the `ExpressionRef` concept introduced in
https://github.com/astral-sh/ruff/pull/5644, motivated by the change we
want to make in https://github.com/astral-sh/ruff/pull/6575 to narrow
the type of the expression that can be passed to `parenthesized_range`.

## Test Plan

`cargo test`
2023-08-17 10:07:16 -04:00
Micha Reiser
fa7442da2f Support fmt: skip on compound statements (#6593) 2023-08-17 06:05:41 +00:00
Micha Reiser
4dc32a00d0 Support fmt: skip for simple-statements and decorators (#6561) 2023-08-17 05:58:19 +00:00
Evan Rittenhouse
e3ecbe660e [ruff] Implement quadratic-list-summation rule (RUF017) (#6489)
## Summary

Adds `RUF017`. Closes #5073 

## Test Plan

`cargo t`
2023-08-16 23:13:05 -04:00
Harutaka Kawamura
8c3a8c4fc6 Support glob patterns for raises_require_match_for and raises_require_match_for (#6635)
## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

Support glob patterns for `raises_require_match_for` and
`raises_require_match_for`. Resolve #6473

## Test Plan

New tests + existing tests
2023-08-17 02:15:50 +00:00
Charlie Marsh
dcc7226685 Make lambda-assignment fix always-manual in class bodies (#6626)
## Summary

Related to https://github.com/astral-sh/ruff/issues/6620 (although that
will only be truly closed once we respect manual fixes on the CLI).
2023-08-16 21:24:48 -04:00
5192 changed files with 206296 additions and 185587 deletions

View File

@@ -10,7 +10,7 @@ indent_style = space
insert_final_newline = true
indent_size = 2
[*.{rs,py}]
[*.{rs,py,pyi}]
indent_size = 4
[*.snap]

4
.gitattributes vendored
View File

@@ -1,7 +1,7 @@
* text=auto eol=lf
crates/ruff/resources/test/fixtures/isort/line_ending_crlf.py text eol=crlf
crates/ruff/resources/test/fixtures/pycodestyle/W605_1.py text eol=crlf
crates/ruff_linter/resources/test/fixtures/isort/line_ending_crlf.py text eol=crlf
crates/ruff_linter/resources/test/fixtures/pycodestyle/W605_1.py text eol=crlf
ruff.schema.json linguist-generated=true text=auto eol=lf
*.md.snap linguist-language=Markdown

2
.github/CODEOWNERS vendored
View File

@@ -6,4 +6,4 @@
# - Order is important. The last matching pattern has the most precedence.
# Jupyter
/crates/ruff/src/jupyter/ @dhruvmanila
/crates/ruff_linter/src/jupyter/ @dhruvmanila

View File

@@ -4,8 +4,10 @@ updates:
directory: "/"
schedule:
interval: "weekly"
day: "monday"
time: "12:00"
timezone: "America/New_York"
commit-message:
prefix: "ci(deps)"
labels: ["internal"]
- package-ecosystem: "cargo"
directory: "/"
schedule:
interval: "daily"
labels: ["internal"]

3
.github/release.yml vendored
View File

@@ -20,6 +20,9 @@ changelog:
- title: Bug Fixes
labels:
- bug
- title: Preview
labels:
- preview
- title: Other Changes
labels:
- "*"

View File

@@ -1,136 +0,0 @@
name: Benchmark
on:
pull_request:
paths:
- "Cargo.toml"
- "Cargo.lock"
- "rust-toolchain"
- "crates/**"
- "!crates/ruff_dev"
- "!crates/ruff_shrinking"
workflow_dispatch:
concurrency:
group: ${{ github.workflow }}-${{ github.ref_name }}-${{ github.event.pull_request.number || github.sha }}
cancel-in-progress: true
jobs:
run-benchmark:
if: github.event_name == 'pull_request'
name: "Run | ${{ matrix.os }}"
strategy:
matrix:
os: [ubuntu-latest, windows-latest]
runs-on: ${{ matrix.os }}
steps:
- name: "PR - Checkout Branch"
uses: actions/checkout@v3
with:
ref: ${{ github.event.pull_request.head.sha }}
- name: "PR - Install Rust toolchain"
run: rustup show
- uses: Swatinem/rust-cache@v2
- name: "PR - Build benchmarks"
run: cargo bench -p ruff_benchmark --no-run
- name: "PR - Run benchmarks"
run: cargo benchmark --save-baseline=pr
- name: "Main - Checkout Branch"
uses: actions/checkout@v3
with:
clean: false
ref: main
- name: "Main - Install Rust toolchain"
run: rustup show
- name: "Main - Build benchmarks"
run: cargo bench -p ruff_benchmark --no-run
- name: "Main - Run benchmarks"
run: cargo benchmark --save-baseline=main
- name: "Upload benchmark results"
uses: actions/upload-artifact@v3
with:
name: benchmark-results-${{ matrix.os }}
path: ./target/criterion
# Cleanup
- name: Remove Criterion Artifact
uses: JesseTG/rm@v1.0.3
with:
path: ./target/criterion
benchmark-compare:
if: github.event_name == 'pull_request'
runs-on: ubuntu-latest
name: Compare
needs:
- run-benchmark
steps:
- name: "Install Rust toolchain"
run: rustup show
- name: "Install critcmp"
uses: taiki-e/install-action@v2
with:
tool: critcmp
- name: "Linux | Download PR benchmark results"
uses: actions/download-artifact@v3
with:
name: benchmark-results-ubuntu-latest
path: ./target/criterion
- name: "Linux | Compare benchmark results"
shell: bash
run: |
echo "### Benchmark" >> summary.md
echo "#### Linux" >> summary.md
echo "\`\`\`" >> summary.md
critcmp main pr >> summary.md
echo "\`\`\`" >> summary.md
echo "" >> summary.md
- name: "Linux | Cleanup benchmark results"
run: rm -rf ./target/criterion
- name: "Windows | Download PR benchmark results"
uses: actions/download-artifact@v3
with:
name: benchmark-results-windows-latest
path: ./target/criterion
- name: "Windows | Compare benchmark results"
shell: bash
run: |
echo "#### Windows" >> summary.md
echo "\`\`\`" >> summary.md
critcmp main pr >> summary.md
echo "\`\`\`" >> summary.md
echo "" >> summary.md
echo ${{ github.event.pull_request.number }} > pr-number
cat summary.md > $GITHUB_STEP_SUMMARY
- uses: actions/upload-artifact@v3
name: Upload PR Number
with:
name: pr-number
path: pr-number
- uses: actions/upload-artifact@v3
name: Upload Summary
with:
name: summary
path: summary.md

View File

@@ -26,11 +26,11 @@ jobs:
linter: ${{ steps.changed.outputs.linter_any_changed }}
formatter: ${{ steps.changed.outputs.formatter_any_changed }}
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: tj-actions/changed-files@v37
- uses: tj-actions/changed-files@v39
id: changed
with:
files_yaml: |
@@ -62,7 +62,7 @@ jobs:
name: "cargo fmt"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: "Install Rust toolchain"
run: rustup component add rustfmt
- run: cargo fmt --all --check
@@ -71,7 +71,7 @@ jobs:
name: "cargo clippy"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: "Install Rust toolchain"
run: |
rustup component add clippy
@@ -89,7 +89,7 @@ jobs:
runs-on: ${{ matrix.os }}
name: "cargo test | ${{ matrix.os }}"
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: "Install Rust toolchain"
run: rustup show
- name: "Install cargo insta"
@@ -125,7 +125,7 @@ jobs:
runs-on: ubuntu-latest
name: "cargo fuzz"
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: "Install Rust toolchain"
run: rustup show
- uses: Swatinem/rust-cache@v2
@@ -141,7 +141,7 @@ jobs:
runs-on: ubuntu-latest
name: "cargo test (wasm)"
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: "Install Rust toolchain"
run: rustup target add wasm32-unknown-unknown
- uses: actions/setup-node@v3
@@ -160,7 +160,7 @@ jobs:
name: "test scripts"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: "Install Rust toolchain"
run: rustup component add rustfmt
- uses: Swatinem/rust-cache@v2
@@ -182,7 +182,7 @@ jobs:
# Only runs on pull requests, since that is the only we way we can find the base version for comparison.
if: github.event_name == 'pull_request' && needs.determine_changes.outputs.linter == 'true'
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
@@ -227,7 +227,7 @@ jobs:
name: "cargo udeps"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: "Install nightly Rust toolchain"
# Only pinned to make caching work, update freely
run: rustup toolchain install nightly-2023-06-08
@@ -241,7 +241,7 @@ jobs:
name: "python package"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
@@ -265,7 +265,7 @@ jobs:
name: "pre-commit"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
@@ -295,7 +295,7 @@ jobs:
env:
MKDOCS_INSIDERS_SSH_KEY_EXISTS: ${{ secrets.MKDOCS_INSIDERS_SSH_KEY != '' }}
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
- name: "Add SSH key"
if: ${{ env.MKDOCS_INSIDERS_SSH_KEY_EXISTS == 'true' }}
@@ -330,7 +330,7 @@ jobs:
needs: determine_changes
if: needs.determine_changes.outputs.formatter == 'true' || github.ref == 'refs/heads/main'
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: "Install Rust toolchain"
run: rustup show
- name: "Cache rust"
@@ -341,3 +341,28 @@ jobs:
run: cat target/progress_projects_stats.txt > $GITHUB_STEP_SUMMARY
- name: "Remove checkouts from cache"
run: rm -r target/progress_projects
benchmarks:
runs-on: ubuntu-latest
steps:
- name: "Checkout Branch"
uses: actions/checkout@v4
- name: "Install Rust toolchain"
run: rustup show
- name: "Install codspeed"
uses: taiki-e/install-action@v2
with:
tool: cargo-codspeed
- uses: Swatinem/rust-cache@v2
- name: "Build benchmarks"
run: cargo codspeed build --features codspeed -p ruff_benchmark
- name: "Run benchmarks"
uses: CodSpeedHQ/action@v1
with:
run: cargo codspeed run
token: ${{ secrets.CODSPEED_TOKEN }}

View File

@@ -2,6 +2,11 @@ name: mkdocs
on:
workflow_dispatch:
inputs:
ref:
description: "The commit SHA, tag, or branch to publish. Uses the default branch if not specified."
default: ""
type: string
release:
types: [published]
@@ -12,7 +17,9 @@ jobs:
CF_API_TOKEN_EXISTS: ${{ secrets.CF_API_TOKEN != '' }}
MKDOCS_INSIDERS_SSH_KEY_EXISTS: ${{ secrets.MKDOCS_INSIDERS_SSH_KEY != '' }}
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
ref: ${{ inputs.ref }}
- uses: actions/setup-python@v4
- name: "Add SSH key"
if: ${{ env.MKDOCS_INSIDERS_SSH_KEY_EXISTS == 'true' }}
@@ -40,8 +47,9 @@ jobs:
run: mkdocs build --strict -f mkdocs.generated.yml
- name: "Deploy to Cloudflare Pages"
if: ${{ env.CF_API_TOKEN_EXISTS == 'true' }}
uses: cloudflare/wrangler-action@3.0.0
uses: cloudflare/wrangler-action@v3.1.1
with:
apiToken: ${{ secrets.CF_API_TOKEN }}
accountId: ${{ secrets.CF_ACCOUNT_ID }}
command: pages publish site --project-name=ruff-docs --branch ${GITHUB_HEAD_REF} --commit-hash ${GITHUB_SHA}
# `github.head_ref` is only set during pull requests and for manual runs or tags we use `main` to deploy to production
command: pages deploy site --project-name=astral-docs --branch ${{ github.head_ref || 'main' }} --commit-hash ${GITHUB_SHA}

View File

@@ -19,7 +19,7 @@ jobs:
macos-x86_64:
runs-on: macos-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
@@ -42,7 +42,7 @@ jobs:
macos-universal:
runs-on: macos-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
@@ -68,7 +68,7 @@ jobs:
matrix:
target: [x64, x86]
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
@@ -96,7 +96,7 @@ jobs:
matrix:
target: [x86_64, i686]
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
@@ -123,7 +123,7 @@ jobs:
matrix:
target: [aarch64, armv7, s390x, ppc64le, ppc64]
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
@@ -160,7 +160,7 @@ jobs:
- x86_64-unknown-linux-musl
- i686-unknown-linux-musl
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
@@ -196,7 +196,7 @@ jobs:
- target: armv7-unknown-linux-musleabihf
arch: armv7
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}

View File

@@ -17,7 +17,7 @@ jobs:
env:
CF_API_TOKEN_EXISTS: ${{ secrets.CF_API_TOKEN != '' }}
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: "Install Rust toolchain"
run: rustup target add wasm32-unknown-unknown
- uses: actions/setup-node@v3
@@ -40,8 +40,8 @@ jobs:
working-directory: playground
- name: "Deploy to Cloudflare Pages"
if: ${{ env.CF_API_TOKEN_EXISTS == 'true' }}
uses: cloudflare/wrangler-action@3.0.0
uses: cloudflare/wrangler-action@v3.1.1
with:
apiToken: ${{ secrets.CF_API_TOKEN }}
accountId: ${{ secrets.CF_ACCOUNT_ID }}
command: pages publish playground/dist --project-name=ruff --branch ${GITHUB_HEAD_REF} --commit-hash ${GITHUB_SHA}
command: pages deploy playground/dist --project-name=ruff-playground --branch ${GITHUB_HEAD_REF} --commit-hash ${GITHUB_SHA}

View File

@@ -2,7 +2,7 @@ name: PR Check Comment
on:
workflow_run:
workflows: [CI, Benchmark]
workflows: [CI]
types: [completed]
workflow_dispatch:
inputs:
@@ -43,42 +43,34 @@ jobs:
path: pr/ecosystem
if_no_artifact_found: ignore
- uses: dawidd6/action-download-artifact@v2
name: "Download Benchmark Result"
id: download-benchmark-result
if: steps.pr-number.outputs.pr-number
with:
name: summary
workflow: benchmark.yaml
pr: ${{ steps.pr-number.outputs.pr-number }}
path: pr/benchmark
if_no_artifact_found: ignore
- name: Generate Comment
id: generate-comment
if: steps.download-ecosystem-result.outputs.found_artifact == 'true' || steps.download-benchmark-result.outputs.found_artifact == 'true'
if: steps.download-ecosystem-result.outputs.found_artifact == 'true'
run: |
echo '## PR Check Results' >> comment.txt
echo "### Ecosystem" >> comment.txt
cat pr/ecosystem/ecosystem-result >> comment.txt
echo "" >> comment.txt
echo 'comment<<EOF' >> $GITHUB_OUTPUT
echo '## PR Check Results' >> $GITHUB_OUTPUT
if [[ -f pr/ecosystem/ecosystem-result ]]
then
echo "### Ecosystem" >> $GITHUB_OUTPUT
cat pr/ecosystem/ecosystem-result >> $GITHUB_OUTPUT
echo "" >> $GITHUB_OUTPUT
fi
if [[ -f pr/benchmark/summary.md ]]
then
cat pr/benchmark/summary.md >> $GITHUB_OUTPUT
fi
cat comment.txt >> $GITHUB_OUTPUT
echo 'EOF' >> $GITHUB_OUTPUT
- name: Create or update comment
if: steps.generate-comment.outputs.comment
uses: thollander/actions-comment-pull-request@v2
- name: Find Comment
uses: peter-evans/find-comment@v2
if: steps.generate-comment.outcome == 'success'
id: find-comment
with:
pr_number: ${{ steps.pr-number.outputs.pr-number }}
message: ${{ steps.generate-comment.outputs.comment }}
comment_tag: PR Check Results
issue-number: ${{ steps.pr-number.outputs.pr-number }}
comment-author: "github-actions[bot]"
body-includes: PR Check Results
- name: Create or update comment
if: steps.find-comment.outcome == 'success'
uses: peter-evans/create-or-update-comment@v3
with:
comment-id: ${{ steps.find-comment.outputs.comment-id }}
issue-number: ${{ steps.pr-number.outputs.pr-number }}
body-path: comment.txt
edit-mode: replace

View File

@@ -7,12 +7,15 @@ on:
description: "The version to tag, without the leading 'v'. If omitted, will initiate a dry run (no uploads)."
type: string
sha:
description: "Optionally, the full sha of the commit to be released"
description: "The full sha of the commit to be released. If omitted, the latest commit on the default branch will be used."
default: ""
type: string
pull_request:
paths:
# When we change pyproject.toml, we want to ensure that the maturin builds still work
- pyproject.toml
# And when we change this workflow itself...
- .github/workflows/release.yaml
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
@@ -30,7 +33,9 @@ jobs:
sdist:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
ref: ${{ inputs.sha }}
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
@@ -56,7 +61,9 @@ jobs:
macos-x86_64:
runs-on: macos-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
ref: ${{ inputs.sha }}
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
@@ -94,7 +101,9 @@ jobs:
macos-universal:
runs-on: macos-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
ref: ${{ inputs.sha }}
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
@@ -140,7 +149,9 @@ jobs:
- target: aarch64-pc-windows-msvc
arch: x64
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
ref: ${{ inputs.sha }}
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
@@ -186,7 +197,9 @@ jobs:
- x86_64-unknown-linux-gnu
- i686-unknown-linux-gnu
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
ref: ${{ inputs.sha }}
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
@@ -243,7 +256,9 @@ jobs:
arch: ppc64
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
ref: ${{ inputs.sha }}
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
@@ -296,7 +311,9 @@ jobs:
- x86_64-unknown-linux-musl
- i686-unknown-linux-musl
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
ref: ${{ inputs.sha }}
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
@@ -350,7 +367,9 @@ jobs:
arch: armv7
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
ref: ${{ inputs.sha }}
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
@@ -398,9 +417,22 @@ jobs:
# If you don't set an input tag, it's a dry run (no uploads).
if: ${{ inputs.tag }}
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
ref: main # We checkout the main branch to check for the commit
- name: Check main branch
if: ${{ inputs.sha }}
run: |
# Fetch the main branch since a shallow checkout is used by default
git fetch origin main --unshallow
if ! git branch --contains ${{ inputs.sha }} | grep -E '(^|\s)main$'; then
echo "The specified sha is not on the main branch" >&2
exit 1
fi
- name: Check tag consistency
run: |
# Switch to the commit we want to release
git checkout ${{ inputs.sha }}
version=$(grep "version = " pyproject.toml | sed -e 's/version = "\(.*\)"/\1/g')
if [ "${{ inputs.tag }}" != "${version}" ]; then
echo "The input tag does not match the version from pyproject.toml:" >&2
@@ -410,18 +442,6 @@ jobs:
else
echo "Releasing ${version}"
fi
- name: Check SHA consistency
if: ${{ inputs.sha }}
run: |
git_sha=$(git rev-parse HEAD)
if [ "${{ inputs.sha }}" != "${git_sha}" ]; then
echo "The specified sha does not match the git checkout" >&2
echo "${{ inputs.sha }}" >&2
echo "${git_sha}" >&2
exit 1
else
echo "Releasing ${git_sha}"
fi
upload-release:
name: Upload to PyPI
@@ -464,7 +484,9 @@ jobs:
# For git tag
contents: write
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
ref: ${{ inputs.sha }}
- name: git tag
run: |
git config user.email "hey@astral.sh"

2
.gitignore vendored
View File

@@ -1,5 +1,5 @@
# Benchmarking cpython (CONTRIBUTING.md)
crates/ruff/resources/test/cpython
crates/ruff_linter/resources/test/cpython
# generate_mkdocs.py
mkdocs.generated.yml
# check_ecosystem.py

View File

@@ -2,8 +2,8 @@ fail_fast: true
exclude: |
(?x)^(
crates/ruff/resources/.*|
crates/ruff/src/rules/.*/snapshots/.*|
crates/ruff_linter/resources/.*|
crates/ruff_linter/src/rules/.*/snapshots/.*|
crates/ruff_cli/resources/.*|
crates/ruff_python_formatter/resources/.*|
crates/ruff_python_formatter/tests/snapshots/.*|
@@ -23,8 +23,6 @@ repos:
- id: mdformat
additional_dependencies:
- mdformat-mkdocs
- mdformat-black
- black==23.1.0 # Must be the latest version of Black
- repo: https://github.com/igorshubovych/markdownlint-cli
rev: v0.33.0
@@ -52,7 +50,7 @@ repos:
require_serial: true
exclude: |
(?x)^(
crates/ruff/resources/.*|
crates/ruff_linter/resources/.*|
crates/ruff_python_formatter/resources/.*
)$

View File

@@ -1,5 +1,20 @@
# Breaking Changes
## 0.0.288
### Remove support for emoji identifiers ([#7212](https://github.com/astral-sh/ruff/pull/7212))
Previously, Ruff supported the non-standard compliant emoji identifiers e.g. `📦 = 1`.
We decided to remove this non-standard language extension, and Ruff now reports syntax errors for emoji identifiers in your code, the same as CPython.
### Improved GitLab fingerprints ([#7203](https://github.com/astral-sh/ruff/pull/7203))
GitLab uses fingerprints to identify new, existing, or fixed violations. Previously, Ruff included the violation's position in the fingerprint. Using the location has the downside that changing any code before the violation causes the fingerprint to change, resulting in GitLab reporting one fixed and one new violation even though it is a pre-existing violation.
Ruff now uses a more stable location-agnostic fingerprint to minimize that existing violations incorrectly get marked as fixed and re-reported as new violations.
Expect GitLab to report each pre-existing violation in your project as fixed and a new violation in your Ruff upgrade PR.
## 0.0.283 / 0.284
### The target Python version now defaults to 3.8 instead of 3.10 ([#6397](https://github.com/astral-sh/ruff/pull/6397))
@@ -284,4 +299,4 @@ default.
`pyproject.toml` files are now resolved hierarchically, such that for each Python file, we find
the first `pyproject.toml` file in its path, and use that to determine its lint settings.
See the [documentation](https://beta.ruff.rs/docs/configuration/#python-file-discovery) for more.
See the [documentation](https://docs.astral.sh/ruff/configuration/#python-file-discovery) for more.

View File

@@ -112,11 +112,11 @@ Ruff is structured as a monorepo with a [flat crate structure](https://matklad.g
such that all crates are contained in a flat `crates` directory.
The vast majority of the code, including all lint rules, lives in the `ruff` crate (located at
`crates/ruff`). As a contributor, that's the crate that'll be most relevant to you.
`crates/ruff_linter`). As a contributor, that's the crate that'll be most relevant to you.
At time of writing, the repository includes the following crates:
- `crates/ruff`: library crate containing all lint rules and the core logic for running them.
- `crates/ruff_linter`: library crate containing all lint rules and the core logic for running them.
If you're working on a rule, this is the crate for you.
- `crates/ruff_benchmark`: binary crate for running micro-benchmarks.
- `crates/ruff_cache`: library crate for caching lint results.
@@ -129,6 +129,7 @@ At time of writing, the repository includes the following crates:
intermediate representation. The backend for `ruff_python_formatter`.
- `crates/ruff_index`: library crate inspired by `rustc_index`.
- `crates/ruff_macros`: proc macro crate containing macros used by Ruff.
- `crates/ruff_notebook`: library crate for parsing and manipulating Jupyter notebooks.
- `crates/ruff_python_ast`: library crate containing Python-specific AST types and utilities.
- `crates/ruff_python_codegen`: library crate containing utilities for generating Python source code.
- `crates/ruff_python_formatter`: library crate implementing the Python formatter. Emits an
@@ -152,7 +153,7 @@ At a high level, the steps involved in adding a new lint rule are as follows:
1. Determine a name for the new rule as per our [rule naming convention](#rule-naming-convention)
(e.g., `AssertFalse`, as in, "allow `assert False`").
1. Create a file for your rule (e.g., `crates/ruff/src/rules/flake8_bugbear/rules/assert_false.rs`).
1. Create a file for your rule (e.g., `crates/ruff_linter/src/rules/flake8_bugbear/rules/assert_false.rs`).
1. In that file, define a violation struct (e.g., `pub struct AssertFalse`). You can grep for
`#[violation]` to see examples.
@@ -161,21 +162,21 @@ At a high level, the steps involved in adding a new lint rule are as follows:
(e.g., `pub(crate) fn assert_false`) based on whatever inputs are required for the rule (e.g.,
an `ast::StmtAssert` node).
1. Define the logic for invoking the diagnostic in `crates/ruff/src/checkers/ast/analyze` (for
AST-based rules), `crates/ruff/src/checkers/tokens.rs` (for token-based rules),
`crates/ruff/src/checkers/physical_lines.rs` (for text-based rules),
`crates/ruff/src/checkers/filesystem.rs` (for filesystem-based rules), etc. For AST-based rules,
1. Define the logic for invoking the diagnostic in `crates/ruff_linter/src/checkers/ast/analyze` (for
AST-based rules), `crates/ruff_linter/src/checkers/tokens.rs` (for token-based rules),
`crates/ruff_linter/src/checkers/physical_lines.rs` (for text-based rules),
`crates/ruff_linter/src/checkers/filesystem.rs` (for filesystem-based rules), etc. For AST-based rules,
you'll likely want to modify `analyze/statement.rs` (if your rule is based on analyzing
statements, like imports) or `analyze/expression.rs` (if your rule is based on analyzing
expressions, like function calls).
1. Map the violation struct to a rule code in `crates/ruff/src/codes.rs` (e.g., `B011`).
1. Map the violation struct to a rule code in `crates/ruff_linter/src/codes.rs` (e.g., `B011`).
1. Add proper [testing](#rule-testing-fixtures-and-snapshots) for your rule.
1. Update the generated files (documentation and generated code).
To trigger the violation, you'll likely want to augment the logic in `crates/ruff/src/checkers/ast.rs`
To trigger the violation, you'll likely want to augment the logic in `crates/ruff_linter/src/checkers/ast.rs`
to call your new function at the appropriate time and with the appropriate inputs. The `Checker`
defined therein is a Python AST visitor, which iterates over the AST, building up a semantic model,
and calling out to lint rule analyzer functions as it goes.
@@ -220,7 +221,7 @@ Ruff's output for each fixture, which you can then commit alongside your changes
Once you've completed the code for the rule itself, you can define tests with the following steps:
1. Add a Python file to `crates/ruff/resources/test/fixtures/[linter]` that contains the code you
1. Add a Python file to `crates/ruff_linter/resources/test/fixtures/[linter]` that contains the code you
want to test. The file name should match the rule name (e.g., `E402.py`), and it should include
examples of both violations and non-violations.
@@ -229,16 +230,16 @@ Once you've completed the code for the rule itself, you can define tests with th
For example, if you're adding a new rule named `E402`, you would run:
```shell
cargo run -p ruff_cli -- check crates/ruff/resources/test/fixtures/pycodestyle/E402.py --no-cache --select E402
cargo run -p ruff_cli -- check crates/ruff_linter/resources/test/fixtures/pycodestyle/E402.py --no-cache --select E402
```
**Note:** Only a subset of rules are enabled by default. When testing a new rule, ensure that
you activate it by adding `--select ${rule_code}` to the command.
1. Add the test to the relevant `crates/ruff/src/rules/[linter]/mod.rs` file. If you're contributing
1. Add the test to the relevant `crates/ruff_linter/src/rules/[linter]/mod.rs` file. If you're contributing
a rule to a pre-existing set, you should be able to find a similar example to pattern-match
against. If you're adding a new linter, you'll need to create a new `mod.rs` file (see,
e.g., `crates/ruff/src/rules/flake8_bugbear/mod.rs`)
e.g., `crates/ruff_linter/src/rules/flake8_bugbear/mod.rs`)
1. Run `cargo test`. Your test will fail, but you'll be prompted to follow-up
with `cargo insta review`. Run `cargo insta review`, review and accept the generated snapshot,
@@ -250,25 +251,24 @@ Once you've completed the code for the rule itself, you can define tests with th
Ruff's user-facing settings live in a few different places.
First, the command-line options are defined via the `Cli` struct in `crates/ruff/src/cli.rs`.
First, the command-line options are defined via the `Args` struct in `crates/ruff_cli/src/args.rs`.
Second, the `pyproject.toml` options are defined in `crates/ruff/src/settings/options.rs` (via the
`Options` struct), `crates/ruff/src/settings/configuration.rs` (via the `Configuration` struct), and
`crates/ruff/src/settings/mod.rs` (via the `Settings` struct). These represent, respectively: the
schema used to parse the `pyproject.toml` file; an internal, intermediate representation; and the
final, internal representation used to power Ruff.
Second, the `pyproject.toml` options are defined in `crates/ruff_workspace/src/options.rs` (via the
`Options` struct), `crates/ruff_workspace/src/configuration.rs` (via the `Configuration` struct),
and `crates/ruff_workspace/src/settings.rs` (via the `Settings` struct), which then includes
the `LinterSettings` struct as a field.
These represent, respectively: the schema used to parse the `pyproject.toml` file; an internal,
intermediate representation; and the final, internal representation used to power Ruff.
To add a new configuration option, you'll likely want to modify these latter few files (along with
`cli.rs`, if appropriate). If you want to pattern-match against an existing example, grep for
`arg.rs`, if appropriate). If you want to pattern-match against an existing example, grep for
`dummy_variable_rgx`, which defines a regular expression to match against acceptable unused
variables (e.g., `_`).
Note that plugin-specific configuration options are defined in their own modules (e.g.,
`crates/ruff/src/flake8_unused_arguments/settings.rs`).
You may also want to add the new configuration option to the `flake8-to-ruff` tool, which is
responsible for converting `flake8` configuration files to Ruff's TOML format. This logic
lives in `crates/ruff/src/flake8_to_ruff/converter.rs`.
`Settings` in `crates/ruff_linter/src/flake8_unused_arguments/settings.rs` coupled with
`Flake8UnusedArgumentsOptions` in `crates/ruff_workspace/src/options.rs`).
Finally, regenerate the documentation and generated code with `cargo dev generate-all`.
@@ -361,46 +361,46 @@ First, clone [CPython](https://github.com/python/cpython). It's a large and dive
which makes it a good target for benchmarking.
```shell
git clone --branch 3.10 https://github.com/python/cpython.git crates/ruff/resources/test/cpython
git clone --branch 3.10 https://github.com/python/cpython.git crates/ruff_linter/resources/test/cpython
```
To benchmark the release build:
```shell
cargo build --release && hyperfine --warmup 10 \
"./target/release/ruff ./crates/ruff/resources/test/cpython/ --no-cache -e" \
"./target/release/ruff ./crates/ruff/resources/test/cpython/ -e"
"./target/release/ruff ./crates/ruff_linter/resources/test/cpython/ --no-cache -e" \
"./target/release/ruff ./crates/ruff_linter/resources/test/cpython/ -e"
Benchmark 1: ./target/release/ruff ./crates/ruff/resources/test/cpython/ --no-cache
Benchmark 1: ./target/release/ruff ./crates/ruff_linter/resources/test/cpython/ --no-cache
Time (mean ± σ): 293.8 ms ± 3.2 ms [User: 2384.6 ms, System: 90.3 ms]
Range (min … max): 289.9 ms … 301.6 ms 10 runs
Benchmark 2: ./target/release/ruff ./crates/ruff/resources/test/cpython/
Benchmark 2: ./target/release/ruff ./crates/ruff_linter/resources/test/cpython/
Time (mean ± σ): 48.0 ms ± 3.1 ms [User: 65.2 ms, System: 124.7 ms]
Range (min … max): 45.0 ms … 66.7 ms 62 runs
Summary
'./target/release/ruff ./crates/ruff/resources/test/cpython/' ran
6.12 ± 0.41 times faster than './target/release/ruff ./crates/ruff/resources/test/cpython/ --no-cache'
'./target/release/ruff ./crates/ruff_linter/resources/test/cpython/' ran
6.12 ± 0.41 times faster than './target/release/ruff ./crates/ruff_linter/resources/test/cpython/ --no-cache'
```
To benchmark against the ecosystem's existing tools:
```shell
hyperfine --ignore-failure --warmup 5 \
"./target/release/ruff ./crates/ruff/resources/test/cpython/ --no-cache" \
"pyflakes crates/ruff/resources/test/cpython" \
"./target/release/ruff ./crates/ruff_linter/resources/test/cpython/ --no-cache" \
"pyflakes crates/ruff_linter/resources/test/cpython" \
"autoflake --recursive --expand-star-imports --remove-all-unused-imports --remove-unused-variables --remove-duplicate-keys resources/test/cpython" \
"pycodestyle crates/ruff/resources/test/cpython" \
"flake8 crates/ruff/resources/test/cpython"
"pycodestyle crates/ruff_linter/resources/test/cpython" \
"flake8 crates/ruff_linter/resources/test/cpython"
Benchmark 1: ./target/release/ruff ./crates/ruff/resources/test/cpython/ --no-cache
Benchmark 1: ./target/release/ruff ./crates/ruff_linter/resources/test/cpython/ --no-cache
Time (mean ± σ): 294.3 ms ± 3.3 ms [User: 2467.5 ms, System: 89.6 ms]
Range (min … max): 291.1 ms … 302.8 ms 10 runs
Warning: Ignoring non-zero exit code.
Benchmark 2: pyflakes crates/ruff/resources/test/cpython
Benchmark 2: pyflakes crates/ruff_linter/resources/test/cpython
Time (mean ± σ): 15.786 s ± 0.143 s [User: 15.560 s, System: 0.214 s]
Range (min … max): 15.640 s … 16.157 s 10 runs
@@ -410,31 +410,31 @@ Benchmark 3: autoflake --recursive --expand-star-imports --remove-all-unused-imp
Time (mean ± σ): 6.175 s ± 0.169 s [User: 54.102 s, System: 1.057 s]
Range (min … max): 5.950 s … 6.391 s 10 runs
Benchmark 4: pycodestyle crates/ruff/resources/test/cpython
Benchmark 4: pycodestyle crates/ruff_linter/resources/test/cpython
Time (mean ± σ): 46.921 s ± 0.508 s [User: 46.699 s, System: 0.202 s]
Range (min … max): 46.171 s … 47.863 s 10 runs
Warning: Ignoring non-zero exit code.
Benchmark 5: flake8 crates/ruff/resources/test/cpython
Benchmark 5: flake8 crates/ruff_linter/resources/test/cpython
Time (mean ± σ): 12.260 s ± 0.321 s [User: 102.934 s, System: 1.230 s]
Range (min … max): 11.848 s … 12.933 s 10 runs
Warning: Ignoring non-zero exit code.
Summary
'./target/release/ruff ./crates/ruff/resources/test/cpython/ --no-cache' ran
'./target/release/ruff ./crates/ruff_linter/resources/test/cpython/ --no-cache' ran
20.98 ± 0.62 times faster than 'autoflake --recursive --expand-star-imports --remove-all-unused-imports --remove-unused-variables --remove-duplicate-keys resources/test/cpython'
41.66 ± 1.18 times faster than 'flake8 crates/ruff/resources/test/cpython'
53.64 ± 0.77 times faster than 'pyflakes crates/ruff/resources/test/cpython'
159.43 ± 2.48 times faster than 'pycodestyle crates/ruff/resources/test/cpython'
41.66 ± 1.18 times faster than 'flake8 crates/ruff_linter/resources/test/cpython'
53.64 ± 0.77 times faster than 'pyflakes crates/ruff_linter/resources/test/cpython'
159.43 ± 2.48 times faster than 'pycodestyle crates/ruff_linter/resources/test/cpython'
```
To benchmark a subset of rules, e.g. `LineTooLong` and `DocLineTooLong`:
```shell
cargo build --release && hyperfine --warmup 10 \
"./target/release/ruff ./crates/ruff/resources/test/cpython/ --no-cache -e --select W505,E501"
"./target/release/ruff ./crates/ruff_linter/resources/test/cpython/ --no-cache -e --select W505,E501"
```
You can run `poetry install` from `./scripts/benchmarks` to create a working environment for the
@@ -467,10 +467,10 @@ rm Lib/test/bad_coding.py \
Lib/test/test_typing.py
```
Then, from `crates/ruff/resources/test/cpython`, run: `time pylint -j 0 -E $(git ls-files '*.py')`. This
Then, from `crates/ruff_linter/resources/test/cpython`, run: `time pylint -j 0 -E $(git ls-files '*.py')`. This
will execute Pylint with maximum parallelism and only report errors.
To benchmark Pyupgrade, run the following from `crates/ruff/resources/test/cpython`:
To benchmark Pyupgrade, run the following from `crates/ruff_linter/resources/test/cpython`:
```shell
hyperfine --ignore-failure --warmup 5 --prepare "git reset --hard HEAD" \
@@ -718,8 +718,8 @@ Module {
- `cargo dev generate-cli-help`, `cargo dev generate-docs` and `cargo dev generate-json-schema`:
Update just `docs/configuration.md`, `docs/rules` and `ruff.schema.json` respectively.
- `cargo dev generate-options`: Generate a markdown-compatible table of all `pyproject.toml`
options. Used for <https://beta.ruff.rs/docs/settings/>
- `cargo dev generate-rules-table`: Generate a markdown-compatible table of all rules. Used for <https://beta.ruff.rs/docs/rules/>
options. Used for <https://docs.astral.sh/ruff/settings/>.
- `cargo dev generate-rules-table`: Generate a markdown-compatible table of all rules. Used for <https://docs.astral.sh/ruff/rules/>.
- `cargo dev round-trip <python file or jupyter notebook>`: Read a Python file or Jupyter Notebook,
parse it, serialize the parsed representation and write it back. Used to check how good our
representation is so that fixes don't rewrite irrelevant parts of a file.
@@ -777,7 +777,7 @@ To understand Ruff's import categorization system, we first need to define two c
- "Package root": The top-most directory defining the Python package that includes a given Python
file. To find the package root for a given Python file, traverse up its parent directories until
you reach a parent directory that doesn't contain an `__init__.py` file (and isn't marked as
a [namespace package](https://beta.ruff.rs/docs/settings/#namespace-packages)); take the directory
a [namespace package](https://docs.astral.sh/ruff/settings/#namespace-packages)); take the directory
just before that, i.e., the first directory in the package.
For example, given:
@@ -866,7 +866,7 @@ There are three ways in which an import can be categorized as "first-party":
package (e.g., `from foo import bar` or `import foo.bar`), they'll be classified as first-party
automatically. This check is as simple as comparing the first segment of the current file's
module path to the first segment of the import.
1. **Source roots**: Ruff supports a `[src](https://beta.ruff.rs/docs/settings/#src)` setting, which
1. **Source roots**: Ruff supports a `[src](https://docs.astral.sh/ruff/settings/#src)` setting, which
sets the directories to scan when identifying first-party imports. The algorithm is
straightforward: given an import, like `import foo`, iterate over the directories enumerated in
the `src` setting and, for each directory, check for the existence of a subdirectory `foo` or a

1102
Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -5,8 +5,8 @@ resolver = "2"
[workspace.package]
edition = "2021"
rust-version = "1.71"
homepage = "https://beta.ruff.rs/docs"
documentation = "https://beta.ruff.rs/docs"
homepage = "https://docs.astral.sh/ruff"
documentation = "https://docs.astral.sh/ruff"
repository = "https://github.com/astral-sh/ruff"
authors = ["Charlie Marsh <charlie.r.marsh@gmail.com>"]
license = "MIT"
@@ -14,46 +14,49 @@ license = "MIT"
[workspace.dependencies]
anyhow = { version = "1.0.69" }
bitflags = { version = "2.3.1" }
chrono = { version = "0.4.23", default-features = false, features = ["clock"] }
clap = { version = "4.1.8", features = ["derive"] }
chrono = { version = "0.4.31", default-features = false, features = ["clock"] }
clap = { version = "4.4.4", features = ["derive"] }
colored = { version = "2.0.0" }
filetime = { version = "0.2.20" }
glob = { version = "0.3.1" }
globset = { version = "0.4.10" }
ignore = { version = "0.4.20" }
insta = { version = "1.31.0", feature = ["filters", "glob"] }
is-macro = { version = "0.2.2" }
itertools = { version = "0.10.5" }
is-macro = { version = "0.3.0" }
itertools = { version = "0.11.0" }
log = { version = "0.4.17" }
memchr = "2.5.0"
memchr = "2.6.3"
num-bigint = { version = "0.4.3" }
num-traits = { version = "0.2.15" }
once_cell = { version = "1.17.1" }
path-absolutize = { version = "3.0.14" }
proc-macro2 = { version = "1.0.51" }
path-absolutize = { version = "3.1.1" }
proc-macro2 = { version = "1.0.67" }
quote = { version = "1.0.23" }
regex = { version = "1.7.1" }
regex = { version = "1.9.5" }
rustc-hash = { version = "1.1.0" }
schemars = { version = "0.8.12" }
schemars = { version = "0.8.15" }
serde = { version = "1.0.152", features = ["derive"] }
serde_json = { version = "1.0.93" }
serde_json = { version = "1.0.107" }
shellexpand = { version = "3.0.0" }
similar = { version = "2.2.1", features = ["inline"] }
smallvec = { version = "1.10.0" }
strum = { version = "0.24.1", features = ["strum_macros"] }
strum_macros = { version = "0.24.3" }
syn = { version = "2.0.15" }
test-case = { version = "3.0.0" }
thiserror = { version = "1.0.43" }
toml = { version = "0.7.2" }
static_assertions = "1.1.0"
strum = { version = "0.25.0", features = ["strum_macros"] }
strum_macros = { version = "0.25.2" }
syn = { version = "2.0.37" }
test-case = { version = "3.2.1" }
thiserror = { version = "1.0.48" }
toml = { version = "0.7.8" }
tracing = "0.1.37"
tracing-indicatif = "0.3.4"
tracing-subscriber = { version = "0.3.17", features = ["env-filter"] }
unicode-width = "0.1.10"
unicode-ident = "1.0.12"
unicode-width = "0.1.11"
uuid = { version = "1.4.1", features = ["v4", "fast-rng", "macro-diagnostics", "js"] }
wsl = { version = "0.1.0" }
# v1.0.1
libcst = { git = "https://github.com/Instagram/LibCST.git", rev = "3cacca1a1029f05707e50703b49fe3dd860aa839", default-features = false }
libcst = { version = "0.1.0", default-features = false }
[profile.release]
lto = "fat"

47
LICENSE
View File

@@ -1194,7 +1194,27 @@ are:
- flake8-self, licensed as follows:
"""
Freely Distributable
MIT License
Copyright (c) 2023 Korijn van Golen
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
"""
- flake8-django, licensed under the GPL license.
@@ -1224,6 +1244,31 @@ are:
SOFTWARE.
"""
- flake8-logging, licensed as follows:
"""
MIT License
Copyright (c) 2023 Adam Johnson
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
"""
- Pyright, licensed as follows:
"""
MIT License

View File

@@ -8,7 +8,7 @@
[![image](https://img.shields.io/pypi/pyversions/ruff.svg)](https://pypi.python.org/pypi/ruff)
[![Actions status](https://github.com/astral-sh/ruff/workflows/CI/badge.svg)](https://github.com/astral-sh/ruff/actions)
[**Discord**](https://discord.gg/c9MhzV8aU5) | [**Docs**](https://beta.ruff.rs/docs/) | [**Playground**](https://play.ruff.rs/)
[**Discord**](https://discord.gg/c9MhzV8aU5) | [**Docs**](https://docs.astral.sh/ruff/) | [**Playground**](https://play.ruff.rs/)
An extremely fast Python linter, written in Rust.
@@ -30,13 +30,13 @@ An extremely fast Python linter, written in Rust.
- 🤝 Python 3.11 compatibility
- 📦 Built-in caching, to avoid re-analyzing unchanged files
- 🔧 Autofix support, for automatic error correction (e.g., automatically remove unused imports)
- 📏 Over [600 built-in rules](https://beta.ruff.rs/docs/rules/)
- ⚖️ [Near-parity](https://beta.ruff.rs/docs/faq/#how-does-ruff-compare-to-flake8) with the
- 📏 Over [600 built-in rules](https://docs.astral.sh/ruff/rules/)
- ⚖️ [Near-parity](https://docs.astral.sh/ruff/faq/#how-does-ruff-compare-to-flake8) with the
built-in Flake8 rule set
- 🔌 Native re-implementations of dozens of Flake8 plugins, like flake8-bugbear
- ⌨️ First-party [editor integrations](https://beta.ruff.rs/docs/editor-integrations/) for
- ⌨️ First-party [editor integrations](https://docs.astral.sh/ruff/editor-integrations/) for
[VS Code](https://github.com/astral-sh/ruff-vscode) and [more](https://github.com/astral-sh/ruff-lsp)
- 🌎 Monorepo-friendly, with [hierarchical and cascading configuration](https://beta.ruff.rs/docs/configuration/#pyprojecttoml-discovery)
- 🌎 Monorepo-friendly, with [hierarchical and cascading configuration](https://docs.astral.sh/ruff/configuration/#pyprojecttoml-discovery)
Ruff aims to be orders of magnitude faster than alternative tools while integrating more
functionality behind a single, common interface.
@@ -98,7 +98,7 @@ developer of [Zulip](https://github.com/zulip/zulip):
## Table of Contents
For more, see the [documentation](https://beta.ruff.rs/docs/).
For more, see the [documentation](https://docs.astral.sh/ruff/).
1. [Getting Started](#getting-started)
1. [Configuration](#configuration)
@@ -111,7 +111,7 @@ For more, see the [documentation](https://beta.ruff.rs/docs/).
## Getting Started
For more, see the [documentation](https://beta.ruff.rs/docs/).
For more, see the [documentation](https://docs.astral.sh/ruff/).
### Installation
@@ -122,7 +122,7 @@ pip install ruff
```
You can also install Ruff via [Homebrew](https://formulae.brew.sh/formula/ruff), [Conda](https://anaconda.org/conda-forge/ruff),
and with [a variety of other package managers](https://beta.ruff.rs/docs/installation/).
and with [a variety of other package managers](https://docs.astral.sh/ruff/installation/).
### Usage
@@ -140,7 +140,7 @@ Ruff can also be used as a [pre-commit](https://pre-commit.com) hook:
```yaml
- repo: https://github.com/astral-sh/ruff-pre-commit
# Ruff version.
rev: v0.0.284
rev: v0.0.290
hooks:
- id: ruff
```
@@ -165,7 +165,7 @@ jobs:
### Configuration
Ruff can be configured through a `pyproject.toml`, `ruff.toml`, or `.ruff.toml` file (see:
[_Configuration_](https://beta.ruff.rs/docs/configuration/), or [_Settings_](https://beta.ruff.rs/docs/settings/)
[_Configuration_](https://docs.astral.sh/ruff/configuration/), or [_Settings_](https://docs.astral.sh/ruff/settings/)
for a complete list of all configuration options).
If left unspecified, the default configuration is equivalent to:
@@ -238,7 +238,7 @@ isort, pyupgrade, and others. Regardless of the rule's origin, Ruff re-implement
Rust as a first-party feature.
By default, Ruff enables Flake8's `E` and `F` rules. Ruff supports all rules from the `F` category,
and a [subset](https://beta.ruff.rs/docs/rules/#error-e) of the `E` category, omitting those
and a [subset](https://docs.astral.sh/ruff/rules/#error-e) of the `E` category, omitting those
stylistic rules made obsolete by the use of an autoformatter, like
[Black](https://github.com/psf/black).
@@ -274,6 +274,7 @@ quality tools, including:
- [flake8-gettext](https://pypi.org/project/flake8-gettext/)
- [flake8-implicit-str-concat](https://pypi.org/project/flake8-implicit-str-concat/)
- [flake8-import-conventions](https://github.com/joaopalmeiro/flake8-import-conventions)
- [flake8-logging](https://pypi.org/project/flake8-logging/)
- [flake8-logging-format](https://pypi.org/project/flake8-logging-format/)
- [flake8-no-pep420](https://pypi.org/project/flake8-no-pep420)
- [flake8-pie](https://pypi.org/project/flake8-pie/)
@@ -303,12 +304,12 @@ quality tools, including:
- [tryceratops](https://pypi.org/project/tryceratops/)
- [yesqa](https://pypi.org/project/yesqa/)
For a complete enumeration of the supported rules, see [_Rules_](https://beta.ruff.rs/docs/rules/).
For a complete enumeration of the supported rules, see [_Rules_](https://docs.astral.sh/ruff/rules/).
## Contributing
Contributions are welcome and highly appreciated. To get started, check out the
[**contributing guidelines**](https://beta.ruff.rs/docs/contributing/).
[**contributing guidelines**](https://docs.astral.sh/ruff/contributing/).
You can also join us on [**Discord**](https://discord.gg/c9MhzV8aU5).
@@ -398,6 +399,7 @@ Ruff is used by a number of major open-source projects and companies, including:
- [Pydantic](https://github.com/pydantic/pydantic)
- [Pylint](https://github.com/PyCQA/pylint)
- [Reflex](https://github.com/reflex-dev/reflex)
- [Rippling](https://rippling.com)
- [Robyn](https://github.com/sansyrox/robyn)
- Scale AI ([Launch SDK](https://github.com/scaleapi/launch-python-client))
- Snowflake ([SnowCLI](https://github.com/Snowflake-Labs/snowcli))

View File

@@ -1,6 +1,6 @@
[package]
name = "flake8-to-ruff"
version = "0.0.284"
version = "0.0.290"
description = """
Convert Flake8 configuration files to Ruff configuration files.
"""
@@ -13,13 +13,17 @@ repository = { workspace = true }
license = { workspace = true }
[dependencies]
ruff = { path = "../ruff", default-features = false }
ruff_linter = { path = "../ruff_linter", default-features = false }
ruff_workspace = { path = "../ruff_workspace" }
anyhow = { workspace = true }
clap = { workspace = true }
colored = { workspace = true }
configparser = { version = "3.0.2" }
itertools = { workspace = true }
log = { workspace = true }
once_cell = { workspace = true }
pep440_rs = { version = "0.3.1", features = ["serde"] }
regex = { workspace = true }
rustc-hash = { workspace = true }
serde = { workspace = true }
@@ -27,3 +31,6 @@ serde_json = { workspace = true }
strum = { workspace = true }
strum_macros = { workspace = true }
toml = { workspace = true }
[dev-dependencies]
pretty_assertions = "1.3.0"

View File

@@ -86,7 +86,7 @@ flake8-to-ruff path/to/.flake8 --plugin flake8-builtins --plugin flake8-quotes
configuration options that don't exist in Flake8.)
1. Ruff will omit any rule codes that are unimplemented or unsupported by Ruff, including rule
codes from unsupported plugins. (See the
[documentation](https://beta.ruff.rs/docs/faq/#how-does-ruff-compare-to-flake8) for the complete
[documentation](https://docs.astral.sh/ruff/faq/#how-does-ruff-compare-to-flake8) for the complete
list of supported plugins.)
## License

View File

@@ -0,0 +1,13 @@
//! Extract Black configuration settings from a pyproject.toml.
use ruff_linter::line_width::LineLength;
use ruff_linter::settings::types::PythonVersion;
use serde::{Deserialize, Serialize};
#[derive(Debug, PartialEq, Eq, Serialize, Deserialize, Default)]
pub(crate) struct Black {
#[serde(alias = "line-length", alias = "line_length")]
pub(crate) line_length: Option<LineLength>,
#[serde(alias = "target-version", alias = "target_version")]
pub(crate) target_version: Option<Vec<PythonVersion>>,
}

View File

@@ -1,25 +1,25 @@
use std::collections::{HashMap, HashSet};
use std::str::FromStr;
use anyhow::Result;
use itertools::Itertools;
use crate::line_width::LineLength;
use crate::registry::Linter;
use crate::rule_selector::RuleSelector;
use crate::rules::flake8_pytest_style::types::{
use ruff_linter::line_width::LineLength;
use ruff_linter::registry::Linter;
use ruff_linter::rule_selector::RuleSelector;
use ruff_linter::rules::flake8_pytest_style::types::{
ParametrizeNameType, ParametrizeValuesRowType, ParametrizeValuesType,
};
use crate::rules::flake8_quotes::settings::Quote;
use crate::rules::flake8_tidy_imports::settings::Strictness;
use crate::rules::pydocstyle::settings::Convention;
use crate::rules::{
flake8_annotations, flake8_bugbear, flake8_builtins, flake8_errmsg, flake8_pytest_style,
flake8_quotes, flake8_tidy_imports, mccabe, pep8_naming, pydocstyle,
use ruff_linter::rules::flake8_quotes::settings::Quote;
use ruff_linter::rules::flake8_tidy_imports::settings::Strictness;
use ruff_linter::rules::pydocstyle::settings::Convention;
use ruff_linter::settings::types::PythonVersion;
use ruff_linter::warn_user;
use ruff_workspace::options::{
Flake8AnnotationsOptions, Flake8BugbearOptions, Flake8BuiltinsOptions, Flake8ErrMsgOptions,
Flake8PytestStyleOptions, Flake8QuotesOptions, Flake8TidyImportsOptions, McCabeOptions,
Options, Pep8NamingOptions, PydocstyleOptions,
};
use crate::settings::options::Options;
use crate::settings::pyproject::Pyproject;
use crate::settings::types::PythonVersion;
use crate::warn_user;
use ruff_workspace::pyproject::Pyproject;
use super::external_config::ExternalConfig;
use super::plugin::Plugin;
@@ -30,11 +30,11 @@ const DEFAULT_SELECTORS: &[RuleSelector] = &[
RuleSelector::Linter(Linter::Pycodestyle),
];
pub fn convert(
pub(crate) fn convert(
config: &HashMap<String, HashMap<String, Option<String>>>,
external_config: &ExternalConfig,
plugins: Option<Vec<Plugin>>,
) -> Result<Pyproject> {
) -> Pyproject {
// Extract the Flake8 section.
let flake8 = config
.get("flake8")
@@ -103,16 +103,16 @@ pub fn convert(
// Parse each supported option.
let mut options = Options::default();
let mut flake8_annotations = flake8_annotations::settings::Options::default();
let mut flake8_bugbear = flake8_bugbear::settings::Options::default();
let mut flake8_builtins = flake8_builtins::settings::Options::default();
let mut flake8_errmsg = flake8_errmsg::settings::Options::default();
let mut flake8_pytest_style = flake8_pytest_style::settings::Options::default();
let mut flake8_quotes = flake8_quotes::settings::Options::default();
let mut flake8_tidy_imports = flake8_tidy_imports::options::Options::default();
let mut mccabe = mccabe::settings::Options::default();
let mut pep8_naming = pep8_naming::settings::Options::default();
let mut pydocstyle = pydocstyle::settings::Options::default();
let mut flake8_annotations = Flake8AnnotationsOptions::default();
let mut flake8_bugbear = Flake8BugbearOptions::default();
let mut flake8_builtins = Flake8BuiltinsOptions::default();
let mut flake8_errmsg = Flake8ErrMsgOptions::default();
let mut flake8_pytest_style = Flake8PytestStyleOptions::default();
let mut flake8_quotes = Flake8QuotesOptions::default();
let mut flake8_tidy_imports = Flake8TidyImportsOptions::default();
let mut mccabe = McCabeOptions::default();
let mut pep8_naming = Pep8NamingOptions::default();
let mut pydocstyle = PydocstyleOptions::default();
for (key, value) in flake8 {
if let Some(value) = value {
match key.as_str() {
@@ -120,10 +120,8 @@ pub fn convert(
"builtins" => {
options.builtins = Some(parser::parse_strings(value.as_ref()));
}
"max-line-length" | "max_line_length" => match value.parse::<usize>() {
Ok(line_length) => {
options.line_length = Some(LineLength::from(line_length));
}
"max-line-length" | "max_line_length" => match LineLength::from_str(value) {
Ok(line_length) => options.line_length = Some(line_length),
Err(e) => {
warn_user!("Unable to parse '{key}' property: {e}");
}
@@ -372,41 +370,41 @@ pub fn convert(
.sorted_by_key(RuleSelector::prefix_and_code)
.collect(),
);
if flake8_annotations != flake8_annotations::settings::Options::default() {
if flake8_annotations != Flake8AnnotationsOptions::default() {
options.flake8_annotations = Some(flake8_annotations);
}
if flake8_bugbear != flake8_bugbear::settings::Options::default() {
if flake8_bugbear != Flake8BugbearOptions::default() {
options.flake8_bugbear = Some(flake8_bugbear);
}
if flake8_builtins != flake8_builtins::settings::Options::default() {
if flake8_builtins != Flake8BuiltinsOptions::default() {
options.flake8_builtins = Some(flake8_builtins);
}
if flake8_errmsg != flake8_errmsg::settings::Options::default() {
if flake8_errmsg != Flake8ErrMsgOptions::default() {
options.flake8_errmsg = Some(flake8_errmsg);
}
if flake8_pytest_style != flake8_pytest_style::settings::Options::default() {
if flake8_pytest_style != Flake8PytestStyleOptions::default() {
options.flake8_pytest_style = Some(flake8_pytest_style);
}
if flake8_quotes != flake8_quotes::settings::Options::default() {
if flake8_quotes != Flake8QuotesOptions::default() {
options.flake8_quotes = Some(flake8_quotes);
}
if flake8_tidy_imports != flake8_tidy_imports::options::Options::default() {
if flake8_tidy_imports != Flake8TidyImportsOptions::default() {
options.flake8_tidy_imports = Some(flake8_tidy_imports);
}
if mccabe != mccabe::settings::Options::default() {
if mccabe != McCabeOptions::default() {
options.mccabe = Some(mccabe);
}
if pep8_naming != pep8_naming::settings::Options::default() {
if pep8_naming != Pep8NamingOptions::default() {
options.pep8_naming = Some(pep8_naming);
}
if pydocstyle != pydocstyle::settings::Options::default() {
if pydocstyle != PydocstyleOptions::default() {
options.pydocstyle = Some(pydocstyle);
}
// Extract any settings from the existing `pyproject.toml`.
if let Some(black) = &external_config.black {
if let Some(line_length) = &black.line_length {
options.line_length = Some(LineLength::from(*line_length));
options.line_length = Some(*line_length);
}
if let Some(target_version) = &black.target_version {
@@ -439,7 +437,7 @@ pub fn convert(
}
// Create the pyproject.toml.
Ok(Pyproject::new(options))
Pyproject::new(options)
}
/// Resolve the set of enabled `RuleSelector` values for the given
@@ -458,19 +456,20 @@ mod tests {
use anyhow::Result;
use itertools::Itertools;
use pep440_rs::VersionSpecifiers;
use pretty_assertions::assert_eq;
use crate::flake8_to_ruff::converter::DEFAULT_SELECTORS;
use crate::flake8_to_ruff::pep621::Project;
use crate::flake8_to_ruff::ExternalConfig;
use crate::line_width::LineLength;
use crate::registry::Linter;
use crate::rule_selector::RuleSelector;
use crate::rules::pydocstyle::settings::Convention;
use crate::rules::{flake8_quotes, pydocstyle};
use crate::settings::options::Options;
use crate::settings::pyproject::Pyproject;
use crate::settings::types::PythonVersion;
use pretty_assertions::assert_eq;
use ruff_linter::line_width::LineLength;
use ruff_linter::registry::Linter;
use ruff_linter::rule_selector::RuleSelector;
use ruff_linter::rules::flake8_quotes;
use ruff_linter::rules::pydocstyle::settings::Convention;
use ruff_linter::settings::types::PythonVersion;
use ruff_workspace::options::{Flake8QuotesOptions, Options, PydocstyleOptions};
use ruff_workspace::pyproject::Pyproject;
use crate::converter::DEFAULT_SELECTORS;
use crate::pep621::Project;
use crate::ExternalConfig;
use super::super::plugin::Plugin;
use super::convert;
@@ -491,20 +490,18 @@ mod tests {
}
#[test]
fn it_converts_empty() -> Result<()> {
fn it_converts_empty() {
let actual = convert(
&HashMap::from([("flake8".to_string(), HashMap::default())]),
&ExternalConfig::default(),
None,
)?;
);
let expected = Pyproject::new(default_options([]));
assert_eq!(actual, expected);
Ok(())
}
#[test]
fn it_converts_dashes() -> Result<()> {
fn it_converts_dashes() {
let actual = convert(
&HashMap::from([(
"flake8".to_string(),
@@ -512,18 +509,16 @@ mod tests {
)]),
&ExternalConfig::default(),
Some(vec![]),
)?;
);
let expected = Pyproject::new(Options {
line_length: Some(LineLength::from(100)),
line_length: Some(LineLength::try_from(100).unwrap()),
..default_options([])
});
assert_eq!(actual, expected);
Ok(())
}
#[test]
fn it_converts_underscores() -> Result<()> {
fn it_converts_underscores() {
let actual = convert(
&HashMap::from([(
"flake8".to_string(),
@@ -531,18 +526,16 @@ mod tests {
)]),
&ExternalConfig::default(),
Some(vec![]),
)?;
);
let expected = Pyproject::new(Options {
line_length: Some(LineLength::from(100)),
line_length: Some(LineLength::try_from(100).unwrap()),
..default_options([])
});
assert_eq!(actual, expected);
Ok(())
}
#[test]
fn it_ignores_parse_errors() -> Result<()> {
fn it_ignores_parse_errors() {
let actual = convert(
&HashMap::from([(
"flake8".to_string(),
@@ -550,15 +543,13 @@ mod tests {
)]),
&ExternalConfig::default(),
Some(vec![]),
)?;
);
let expected = Pyproject::new(default_options([]));
assert_eq!(actual, expected);
Ok(())
}
#[test]
fn it_converts_plugin_options() -> Result<()> {
fn it_converts_plugin_options() {
let actual = convert(
&HashMap::from([(
"flake8".to_string(),
@@ -566,9 +557,9 @@ mod tests {
)]),
&ExternalConfig::default(),
Some(vec![]),
)?;
);
let expected = Pyproject::new(Options {
flake8_quotes: Some(flake8_quotes::settings::Options {
flake8_quotes: Some(Flake8QuotesOptions {
inline_quotes: Some(flake8_quotes::settings::Quote::Single),
multiline_quotes: None,
docstring_quotes: None,
@@ -577,12 +568,10 @@ mod tests {
..default_options([])
});
assert_eq!(actual, expected);
Ok(())
}
#[test]
fn it_converts_docstring_conventions() -> Result<()> {
fn it_converts_docstring_conventions() {
let actual = convert(
&HashMap::from([(
"flake8".to_string(),
@@ -593,9 +582,9 @@ mod tests {
)]),
&ExternalConfig::default(),
Some(vec![Plugin::Flake8Docstrings]),
)?;
);
let expected = Pyproject::new(Options {
pydocstyle: Some(pydocstyle::settings::Options {
pydocstyle: Some(PydocstyleOptions {
convention: Some(Convention::Numpy),
ignore_decorators: None,
property_decorators: None,
@@ -603,12 +592,10 @@ mod tests {
..default_options([Linter::Pydocstyle.into()])
});
assert_eq!(actual, expected);
Ok(())
}
#[test]
fn it_infers_plugins_if_omitted() -> Result<()> {
fn it_infers_plugins_if_omitted() {
let actual = convert(
&HashMap::from([(
"flake8".to_string(),
@@ -616,9 +603,9 @@ mod tests {
)]),
&ExternalConfig::default(),
None,
)?;
);
let expected = Pyproject::new(Options {
flake8_quotes: Some(flake8_quotes::settings::Options {
flake8_quotes: Some(Flake8QuotesOptions {
inline_quotes: Some(flake8_quotes::settings::Quote::Single),
multiline_quotes: None,
docstring_quotes: None,
@@ -627,8 +614,6 @@ mod tests {
..default_options([Linter::Flake8Quotes.into()])
});
assert_eq!(actual, expected);
Ok(())
}
#[test]
@@ -642,7 +627,7 @@ mod tests {
..ExternalConfig::default()
},
Some(vec![]),
)?;
);
let expected = Pyproject::new(Options {
target_version: Some(PythonVersion::Py38),
..default_options([])

View File

@@ -0,0 +1,10 @@
use super::black::Black;
use super::isort::Isort;
use super::pep621::Project;
#[derive(Default)]
pub(crate) struct ExternalConfig<'a> {
pub(crate) black: Option<&'a Black>,
pub(crate) isort: Option<&'a Isort>,
pub(crate) project: Option<&'a Project>,
}

View File

@@ -4,7 +4,7 @@ use serde::{Deserialize, Serialize};
/// The [isort configuration](https://pycqa.github.io/isort/docs/configuration/config_files.html).
#[derive(Debug, PartialEq, Eq, Serialize, Deserialize, Default)]
pub struct Isort {
pub(crate) struct Isort {
#[serde(alias = "src-paths", alias = "src_paths")]
pub src_paths: Option<Vec<String>>,
pub(crate) src_paths: Option<Vec<String>>,
}

View File

@@ -1,13 +1,25 @@
//! Utility to generate Ruff's `pyproject.toml` section from a Flake8 INI file.
mod black;
mod converter;
mod external_config;
mod isort;
mod parser;
mod pep621;
mod plugin;
mod pyproject;
use std::path::PathBuf;
use anyhow::Result;
use clap::Parser;
use configparser::ini::Ini;
use ruff::flake8_to_ruff::{self, ExternalConfig};
use ruff::logging::{set_up_logging, LogLevel};
use crate::converter::convert;
use crate::external_config::ExternalConfig;
use crate::plugin::Plugin;
use crate::pyproject::parse;
use ruff_linter::logging::{set_up_logging, LogLevel};
#[derive(Parser)]
#[command(
@@ -25,7 +37,7 @@ struct Args {
pyproject: Option<PathBuf>,
/// List of plugins to enable.
#[arg(long, value_delimiter = ',')]
plugin: Option<Vec<flake8_to_ruff::Plugin>>,
plugin: Option<Vec<Plugin>>,
}
fn main() -> Result<()> {
@@ -39,7 +51,7 @@ fn main() -> Result<()> {
let config = ini.load(args.file).map_err(|msg| anyhow::anyhow!(msg))?;
// Read the pyproject.toml file.
let pyproject = args.pyproject.map(flake8_to_ruff::parse).transpose()?;
let pyproject = args.pyproject.map(parse).transpose()?;
let external_config = pyproject
.as_ref()
.and_then(|pyproject| pyproject.tool.as_ref())
@@ -57,7 +69,7 @@ fn main() -> Result<()> {
};
// Create Ruff's pyproject.toml section.
let pyproject = flake8_to_ruff::convert(&config, &external_config, args.plugin)?;
let pyproject = convert(&config, &external_config, args.plugin);
#[allow(clippy::print_stdout)]
{

View File

@@ -5,9 +5,8 @@ use once_cell::sync::Lazy;
use regex::Regex;
use rustc_hash::FxHashMap;
use crate::rule_selector::RuleSelector;
use crate::settings::types::PatternPrefixPair;
use crate::warn_user;
use ruff_linter::settings::types::PatternPrefixPair;
use ruff_linter::{warn_user, RuleSelector};
static COMMA_SEPARATED_LIST_RE: Lazy<Regex> = Lazy::new(|| Regex::new(r"[,\s]").unwrap());
@@ -195,10 +194,10 @@ pub(crate) fn collect_per_file_ignores(
mod tests {
use anyhow::Result;
use crate::codes;
use crate::registry::Linter;
use crate::rule_selector::RuleSelector;
use crate::settings::types::PatternPrefixPair;
use ruff_linter::codes;
use ruff_linter::registry::Linter;
use ruff_linter::settings::types::PatternPrefixPair;
use ruff_linter::RuleSelector;
use super::{parse_files_to_codes_mapping, parse_prefix_codes, parse_strings};

View File

@@ -4,7 +4,7 @@ use pep440_rs::VersionSpecifiers;
use serde::{Deserialize, Serialize};
#[derive(Debug, PartialEq, Eq, Serialize, Deserialize, Default)]
pub struct Project {
pub(crate) struct Project {
#[serde(alias = "requires-python", alias = "requires_python")]
pub requires_python: Option<VersionSpecifiers>,
pub(crate) requires_python: Option<VersionSpecifiers>,
}

View File

@@ -3,9 +3,9 @@ use std::fmt;
use std::str::FromStr;
use anyhow::anyhow;
use crate::registry::Linter;
use crate::rule_selector::RuleSelector;
use ruff_linter::registry::Linter;
use ruff_linter::settings::types::PreviewMode;
use ruff_linter::RuleSelector;
#[derive(Copy, Clone, Ord, PartialOrd, Eq, PartialEq)]
pub enum Plugin {
@@ -332,7 +332,7 @@ pub(crate) fn infer_plugins_from_codes(selectors: &HashSet<RuleSelector>) -> Vec
.filter(|plugin| {
for selector in selectors {
if selector
.into_iter()
.rules(PreviewMode::Disabled)
.any(|rule| Linter::from(plugin).rules().any(|r| r == rule))
{
return true;

View File

@@ -0,0 +1,26 @@
use std::path::Path;
use anyhow::Result;
use serde::{Deserialize, Serialize};
use super::black::Black;
use super::isort::Isort;
use super::pep621::Project;
#[derive(Debug, PartialEq, Eq, Serialize, Deserialize)]
pub(crate) struct Tools {
pub(crate) black: Option<Black>,
pub(crate) isort: Option<Isort>,
}
#[derive(Debug, PartialEq, Eq, Serialize, Deserialize)]
pub(crate) struct Pyproject {
pub(crate) tool: Option<Tools>,
pub(crate) project: Option<Project>,
}
pub(crate) fn parse<P: AsRef<Path>>(path: P) -> Result<Pyproject> {
let contents = std::fs::read_to_string(path)?;
let pyproject = toml::from_str::<Pyproject>(&contents)?;
Ok(pyproject)
}

View File

@@ -1,94 +0,0 @@
[package]
name = "ruff"
version = "0.0.284"
publish = false
authors = { workspace = true }
edition = { workspace = true }
rust-version = { workspace = true }
homepage = { workspace = true }
documentation = { workspace = true }
repository = { workspace = true }
license = { workspace = true }
readme = "README.md"
[lib]
name = "ruff"
[dependencies]
ruff_cache = { path = "../ruff_cache" }
ruff_diagnostics = { path = "../ruff_diagnostics", features = ["serde"] }
ruff_index = { path = "../ruff_index" }
ruff_macros = { path = "../ruff_macros" }
ruff_python_ast = { path = "../ruff_python_ast", features = ["serde"] }
ruff_python_codegen = { path = "../ruff_python_codegen" }
ruff_python_index = { path = "../ruff_python_index" }
ruff_python_literal = { path = "../ruff_python_literal" }
ruff_python_semantic = { path = "../ruff_python_semantic" }
ruff_python_stdlib = { path = "../ruff_python_stdlib" }
ruff_python_trivia = { path = "../ruff_python_trivia" }
ruff_python_parser = { path = "../ruff_python_parser" }
ruff_source_file = { path = "../ruff_source_file", features = ["serde"] }
ruff_text_size = { path = "../ruff_text_size" }
annotate-snippets = { version = "0.9.1", features = ["color"] }
anyhow = { workspace = true }
bitflags = { workspace = true }
chrono = { workspace = true }
clap = { workspace = true, features = ["derive", "string"], optional = true }
colored = { workspace = true }
dirs = { version = "5.0.0" }
fern = { version = "0.6.1" }
glob = { workspace = true }
globset = { workspace = true }
ignore = { workspace = true }
imperative = { version = "1.0.4" }
is-macro = { workspace = true }
itertools = { workspace = true }
libcst = { workspace = true }
log = { workspace = true }
memchr = { workspace = true }
natord = { version = "1.0.9" }
num-bigint = { workspace = true }
num-traits = { workspace = true }
once_cell = { workspace = true }
path-absolutize = { workspace = true, features = [
"once_cell_cache",
"use_unix_paths_on_wasm",
] }
pathdiff = { version = "0.2.1" }
pep440_rs = { version = "0.3.1", features = ["serde"] }
pyproject-toml = { version = "0.6.0" }
quick-junit = { version = "0.3.2" }
regex = { workspace = true }
result-like = { version = "0.4.6" }
rustc-hash = { workspace = true }
schemars = { workspace = true, optional = true }
semver = { version = "1.0.16" }
serde = { workspace = true }
serde_json = { workspace = true }
serde_with = { version = "3.0.0" }
similar = { workspace = true }
shellexpand = { workspace = true }
smallvec = { workspace = true }
strum = { workspace = true }
strum_macros = { workspace = true }
thiserror = { version = "1.0.43" }
toml = { workspace = true }
typed-arena = { version = "2.0.2" }
unicode-width = { workspace = true }
unicode_names2 = { version = "0.6.0", git = "https://github.com/youknowone/unicode_names2.git", rev = "4ce16aa85cbcdd9cc830410f1a72ef9a235f2fde" }
wsl = { version = "0.1.0" }
[dev-dependencies]
insta = { workspace = true }
pretty_assertions = "1.3.0"
test-case = { workspace = true }
# Disable colored output in tests
colored = { workspace = true, features = ["no-color"] }
tempfile = "3.6.0"
[features]
default = []
schemars = ["dep:schemars"]
# Enables the UnreachableCode rule
unreachable-code = []

View File

@@ -1,21 +0,0 @@
#import os
# from foo import junk
#a = 3
a = 4
#foo(1, 2, 3)
def foo(x, y, z):
content = 1 # print('hello')
print(x, y, z)
# This is a real comment.
# # This is a (nested) comment.
#return True
return False
#import os # noqa: ERA001
class A():
pass
# b = c

View File

@@ -1,45 +0,0 @@
"""
Should emit:
B009 - Line 19, 20, 21, 22, 23, 24
B010 - Line 40, 41, 42, 43, 44, 45
"""
# Valid getattr usage
getattr(foo, bar)
getattr(foo, "bar", None)
getattr(foo, "bar{foo}".format(foo="a"), None)
getattr(foo, "bar{foo}".format(foo="a"))
getattr(foo, bar, None)
getattr(foo, "123abc")
getattr(foo, r"123\abc")
getattr(foo, "except")
getattr(foo, "__123abc")
# Invalid usage
getattr(foo, "bar")
getattr(foo, "_123abc")
getattr(foo, "__123abc__")
getattr(foo, "abc123")
getattr(foo, r"abc123")
_ = lambda x: getattr(x, "bar")
if getattr(x, "bar"):
pass
# Valid setattr usage
setattr(foo, bar, None)
setattr(foo, "bar{foo}".format(foo="a"), None)
setattr(foo, "123abc", None)
setattr(foo, "__123abc", None)
setattr(foo, r"123\abc", None)
setattr(foo, "except", None)
_ = lambda x: setattr(x, "bar", 1)
if setattr(x, "bar", 1):
pass
# Invalid usage
setattr(foo, "bar", None)
setattr(foo, "_123abc", None)
setattr(foo, "__123abc__", None)
setattr(foo, "abc123", None)
setattr(foo, r"abc123", None)
setattr(foo.bar, r"baz", None)

View File

@@ -1,8 +0,0 @@
try:
pass
except (ValueError,):
pass
except AttributeError:
pass
except (ImportError, TypeError):
pass

View File

@@ -1,18 +0,0 @@
dict((x, x) for x in range(3))
dict(
(x, x) for x in range(3)
)
dict(((x, x) for x in range(3)), z=3)
y = f'{dict((x, x) for x in range(3))}'
print(f'Hello {dict((x, x) for x in range(3))} World')
print(f"Hello {dict((x, x) for x in 'abc')} World")
print(f'Hello {dict((x, x) for x in "abc")} World')
print(f'Hello {dict((x,x) for x in "abc")} World')
f'{dict((x, x) for x in range(3)) | dict((x, x) for x in range(3))}'
f'{ dict((x, x) for x in range(3)) | dict((x, x) for x in range(3)) }'
def f(x):
return x
print(f'Hello {dict((x,f(x)) for x in "abc")} World')

View File

@@ -1,13 +0,0 @@
dict([(i, i) for i in range(3)])
dict([(i, i) for i in range(3)], z=4)
def f(x):
return x
f'{dict([(s,s) for s in "ab"])}'
f"{dict([(s,s) for s in 'ab'])}"
f"{dict([(s, s) for s in 'ab'])}"
f"{dict([(s,f(s)) for s in 'ab'])}"
f'{dict([(s,s) for s in "ab"]) | dict([(s,s) for s in "ab"])}'
f'{ dict([(s,s) for s in "ab"]) | dict([(s,s) for s in "ab"]) }'

View File

@@ -1,15 +0,0 @@
x = [2, 3, 1]
list(x)
list(sorted(x))
reversed(sorted(x))
reversed(sorted(x, key=lambda e: e))
reversed(sorted(x, reverse=True))
reversed(sorted(x, key=lambda e: e, reverse=True))
reversed(sorted(x, reverse=True, key=lambda e: e))
reversed(sorted(x, reverse=False))
def reversed(*args, **kwargs):
return None
reversed(sorted(x, reverse=True))

View File

@@ -1,19 +0,0 @@
x = [1, 2, 3]
y = [("a", 1), ("b", 2), ("c", 3)]
z = [(1,), (2,), (3,)]
d = {"a": 1, "b": 2, "c": 3}
[i for i in x]
{i for i in x}
{k: v for k, v in y}
{k: v for k, v in d.items()}
[i for i, in z]
[i for i, j in y]
[i for i in x if i > 1]
[i for i in x for j in x]
{v: k for k, v in y}
{k.foo: k for k in y}
{k["foo"]: k for k in y}
{k: v if v else None for k, v in y}

View File

@@ -1,39 +0,0 @@
# Errors.
nums = [1, 2, 3]
map(lambda x: x + 1, nums)
map(lambda x: str(x), nums)
list(map(lambda x: x * 2, nums))
set(map(lambda x: x % 2 == 0, nums))
dict(map(lambda v: (v, v**2), nums))
map(lambda: "const", nums)
map(lambda _: 3.0, nums)
_ = "".join(map(lambda x: x in nums and "1" or "0", range(123)))
all(map(lambda v: isinstance(v, dict), nums))
filter(func, map(lambda v: v, nums))
# When inside f-string, then the fix should be surrounded by whitespace
_ = f"{set(map(lambda x: x % 2 == 0, nums))}"
_ = f"{dict(map(lambda v: (v, v**2), nums))}"
# Error, but unfixable.
# For simple expressions, this could be: `(x if x else 1 for x in nums)`.
# For more complex expressions, this would differ: `(x + 2 if x else 3 for x in nums)`.
map(lambda x=1: x, nums)
# False negatives.
map(lambda x=2, y=1: x + y, nums, nums)
set(map(lambda x, y: x, nums, nums))
def func(arg1: int, arg2: int = 4):
return 2 * arg1 + arg2
# Non-error: `func` is not a lambda.
list(map(func, nums))
# False positive: need to preserve the late-binding of `x` in the inner lambda.
map(lambda x: lambda: x, range(4))
# Error: the `x` is overridden by the inner lambda.
map(lambda x: lambda x: x, range(4))

View File

@@ -1,22 +0,0 @@
import math # not checked
import altair # unconventional
import matplotlib.pyplot # unconventional
import numpy # unconventional
import pandas # unconventional
import seaborn # unconventional
import tkinter # unconventional
import altair as altr # unconventional
import matplotlib.pyplot as plot # unconventional
import numpy as nmp # unconventional
import pandas as pdas # unconventional
import seaborn as sbrn # unconventional
import tkinter as tkr # unconventional
import altair as alt # conventional
import matplotlib.pyplot as plt # conventional
import numpy as np # conventional
import pandas as pd # conventional
import seaborn as sns # conventional
import tkinter as tk # conventional

View File

@@ -1,18 +0,0 @@
def bar():
... # OK
def foo():
"""foo""" # OK
def buzz():
print("buzz") # OK, not in stub file
def foo2():
123 # OK, not in a stub file
def bizz():
x = 123 # OK, not in a stub file

View File

@@ -1,12 +0,0 @@
def bar(): ... # OK
def foo():
"""foo""" # OK, strings are handled by another rule
def buzz():
print("buzz") # ERROR PYI010
def foo2():
123 # ERROR PYI010
def bizz():
x = 123 # ERROR PYI010

View File

@@ -1,19 +0,0 @@
def bar(): # OK
...
def oof(): # OK, docstrings are handled by another rule
"""oof"""
print("foo")
def foo(): # Ok not in Stub file
"""foo"""
print("foo")
print("foo")
def buzz(): # Ok not in Stub file
print("fizz")
print("buzz")
print("test")

View File

@@ -1,20 +0,0 @@
def bar():
... # OK
def oof(): # OK, docstrings are handled by another rule
"""oof"""
print("foo")
def foo(): # ERROR PYI048
"""foo"""
print("foo")
print("foo")
def buzz(): # ERROR PYI048
print("fizz")
print("buzz")
print("test")

View File

@@ -1,26 +0,0 @@
import pytest
@pytest.mark.parametrize("x", [1, 1, 2])
def test_error_literal(x):
...
a = 1
b = 2
c = 3
@pytest.mark.parametrize("x", [a, a, b, b, b, c])
def test_error_expr_simple(x):
...
@pytest.mark.parametrize("x", [(a, b), (a, b), (b, c)])
def test_error_expr_complex(x):
...
@pytest.mark.parametrize("x", [1, 2])
def test_ok(x):
...

View File

@@ -1,45 +0,0 @@
import pytest
def test_ok():
assert something
assert something or something_else
assert something or something_else and something_third
assert not (something and something_else)
assert something, "something message"
assert something or something_else and something_third, "another message"
def test_error():
assert something and something_else
assert something and something_else and something_third
assert something and not something_else
assert something and (something_else or something_third)
assert not something and something_else
assert not (something or something_else)
assert not (something or something_else or something_third)
assert something and something_else == """error
message
"""
assert (
something
and something_else
== """error
message
"""
)
# recursive case
assert not (a or not (b or c))
assert not (a or not (b and c))
# detected, but no autofix for messages
assert something and something_else, "error message"
assert not (something or something_else and something_third), "with message"
# detected, but no autofix for mixed conditions (e.g. `a or b and c`)
assert not (something or something_else and something_third)
assert something # OK
assert something and something_else # Error
assert something and something_else and something_third # Error

View File

@@ -1,54 +0,0 @@
try:
y = 6 + "7"
except TypeError:
# RSE102
raise ValueError()
try:
x = 1 / 0
except ZeroDivisionError:
raise
# RSE102
raise TypeError()
# RSE102
raise TypeError ()
# RSE102
raise TypeError \
()
# RSE102
raise TypeError(
)
# RSE102
raise TypeError(
# Hello, world!
)
# OK
raise AssertionError
# OK
raise AttributeError("test message")
def return_error():
return ValueError("Something")
# OK
raise return_error()
class Class:
@staticmethod
def error():
return ValueError("Something")
# OK
raise Class.error()

View File

@@ -1,42 +0,0 @@
key in obj.keys() # SIM118
key not in obj.keys() # SIM118
foo["bar"] in obj.keys() # SIM118
foo["bar"] not in obj.keys() # SIM118
foo['bar'] in obj.keys() # SIM118
foo['bar'] not in obj.keys() # SIM118
foo() in obj.keys() # SIM118
foo() not in obj.keys() # SIM118
for key in obj.keys(): # SIM118
pass
for key in list(obj.keys()):
if some_property(key):
del obj[key]
[k for k in obj.keys()] # SIM118
{k for k in obj.keys()} # SIM118
{k: k for k in obj.keys()} # SIM118
(k for k in obj.keys()) # SIM118
key in (obj or {}).keys() # SIM118
from typing import KeysView
class Foo:
def keys(self) -> KeysView[object]:
...
def __contains__(self, key: object) -> bool:
return key in self.keys() # OK

View File

@@ -1,15 +0,0 @@
a = True if b else False # SIM210
a = True if b != c else False # SIM210
a = True if b + c else False # SIM210
a = False if b else True # OK
def f():
# OK
def bool():
return False
a = True if b else False

View File

@@ -1,15 +0,0 @@
import numpy as np
np.round_(np.random.rand(5, 5), 2)
np.product(np.random.rand(5, 5))
np.cumproduct(np.random.rand(5, 5))
np.sometrue(np.random.rand(5, 5))
np.alltrue(np.random.rand(5, 5))
from numpy import round_, product, cumproduct, sometrue, alltrue
round_(np.random.rand(5, 5), 2)
product(np.random.rand(5, 5))
cumproduct(np.random.rand(5, 5))
sometrue(np.random.rand(5, 5))
alltrue(np.random.rand(5, 5))

View File

@@ -1,40 +0,0 @@
import collections
from collections import namedtuple
from typing import TypeVar
from typing import NewType
from typing import NamedTuple, TypedDict
GLOBAL: str = "foo"
def assign():
global GLOBAL
GLOBAL = "bar"
lower = 0
Camel = 0
CONSTANT = 0
_ = 0
MyObj1 = collections.namedtuple("MyObj1", ["a", "b"])
MyObj2 = namedtuple("MyObj12", ["a", "b"])
T = TypeVar("T")
UserId = NewType("UserId", int)
Employee = NamedTuple('Employee', [('name', str), ('id', int)])
Point2D = TypedDict('Point2D', {'in': int, 'x-y': int})
def aug_assign(rank, world_size):
global CURRENT_PORT
CURRENT_PORT += 1
if CURRENT_PORT > MAX_PORT:
CURRENT_PORT = START_PORT
def loop_assign():
global CURRENT_PORT
for CURRENT_PORT in range(5):
pass

View File

@@ -1,26 +0,0 @@
def f():
items = [1, 2, 3, 4]
result = []
for i in items:
result.append(i) # PERF402
def f():
items = [1, 2, 3, 4]
result = []
for i in items:
result.insert(0, i) # PERF402
def f():
items = [1, 2, 3, 4]
result = []
for i in items:
result.append(i * i) # OK
def f():
items = [1, 2, 3, 4]
result = {}
for i in items:
result[i].append(i * i) # OK

View File

@@ -1,46 +0,0 @@
class Apples:
def _init_(self): # [bad-dunder-name]
pass
def __hello__(self): # [bad-dunder-name]
print("hello")
def __init_(self): # [bad-dunder-name]
# author likely unintentionally misspelled the correct init dunder.
pass
def _init_(self): # [bad-dunder-name]
# author likely unintentionally misspelled the correct init dunder.
pass
def ___neg__(self): # [bad-dunder-name]
# author likely accidentally added an additional `_`
pass
def __inv__(self): # [bad-dunder-name]
# author likely meant to call the invert dunder method
pass
def hello(self):
print("hello")
def __init__(self):
pass
def init(self):
# valid name even though someone could accidentally mean __init__
pass
def _protected_method(self):
print("Protected")
def __private_method(self):
print("Private")
@property
def __doc__(self):
return "Docstring"
def __foo_bar__(): # this is not checked by the [bad-dunder-name] rule
...

View File

@@ -1,29 +0,0 @@
# pylint: disable=missing-docstring,consider-using-f-string, pointless-statement
## Old style formatting
"%s %z" % ("hello", "world") # [bad-format-character]
"%s" "%z" % ("hello", "world") # [bad-format-character]
"""%s %z""" % ("hello", "world") # [bad-format-character]
"""%s""" """%z""" % ("hello", "world") # [bad-format-character]
## New style formatting
"{:s} {:y}".format("hello", "world") # [bad-format-character]
"{:*^30s}".format("centered")
## f-strings
H, W = "hello", "world"
f"{H} {W}"
f"{H:s} {W:z}" # [bad-format-character]
f"{1:z}" # [bad-format-character]
## False negatives
print(("%" "z") % 1)

View File

@@ -1,34 +0,0 @@
# Errors.
foo == "a" or foo == "b"
foo != "a" and foo != "b"
foo == "a" or foo == "b" or foo == "c"
foo != "a" and foo != "b" and foo != "c"
foo == a or foo == "b" or foo == 3 # Mixed types.
# False negatives (the current implementation doesn't support Yoda conditions).
"a" == foo or "b" == foo or "c" == foo
"a" != foo and "b" != foo and "c" != foo
"a" == foo or foo == "b" or "c" == foo
# OK
foo == "a" and foo == "b" and foo == "c" # `and` mixed with `==`.
foo != "a" or foo != "b" or foo != "c" # `or` mixed with `!=`.
foo == a or foo == b() or foo == c # Call expression.
foo != a or foo() != b or foo != c # Call expression.
foo in {"a", "b", "c"} # Uses membership test already.
foo not in {"a", "b", "c"} # Uses membership test already.
foo == "a" # Single comparison.
foo != "a" # Single comparison.

View File

@@ -1,15 +0,0 @@
type("")
type(b"")
type(0)
type(0.0)
type(0j)
# OK
type(arg)(" ")
# OK
y = x.dtype.type(0.0)
# OK
type = lambda *args, **kwargs: None
type("")

View File

@@ -1,61 +0,0 @@
import typing
from typing import Optional
from typing import Union
def f(x: Optional[str]) -> None:
...
def f(x: typing.Optional[str]) -> None:
...
def f(x: Union[str, int, Union[float, bytes]]) -> None:
...
def f(x: typing.Union[str, int]) -> None:
...
def f(x: typing.Union[(str, int)]) -> None:
...
def f(x: typing.Union[(str, int), float]) -> None:
...
def f(x: typing.Union[(int,)]) -> None:
...
def f(x: typing.Union[()]) -> None:
...
def f(x: "Union[str, int, Union[float, bytes]]") -> None:
...
def f(x: "typing.Union[str, int]") -> None:
...
def f(x: Union["str", int]) -> None:
...
def f(x: Union[("str", "int"), float]) -> None:
...
def f() -> None:
x: Optional[str]
x = Optional[str]
x = Union[str, int]
x = Union["str", "int"]
x: Union[str, int]
x: Union["str", "int"]

View File

@@ -1,12 +0,0 @@
import subprocess
import subprocess as somename
from subprocess import run
from subprocess import run as anothername
subprocess.run(["foo"], universal_newlines=True, check=True)
somename.run(["foo"], universal_newlines=True)
run(["foo"], universal_newlines=True, check=False)
anothername(["foo"], universal_newlines=True)
subprocess.run(["foo"], check=True)

View File

@@ -1,42 +0,0 @@
from subprocess import run
import subprocess
output = run(["foo"], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
output = subprocess.run(["foo"], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
output = subprocess.run(stdout=subprocess.PIPE, args=["foo"], stderr=subprocess.PIPE)
output = subprocess.run(
["foo"], stdout=subprocess.PIPE, check=True, stderr=subprocess.PIPE
)
output = subprocess.run(
["foo"], stderr=subprocess.PIPE, check=True, stdout=subprocess.PIPE
)
output = subprocess.run(
["foo"],
stdout=subprocess.PIPE,
check=True,
stderr=subprocess.PIPE,
text=True,
encoding="utf-8",
close_fds=True,
)
if output:
output = subprocess.run(
["foo"],
stdout=subprocess.PIPE,
check=True,
stderr=subprocess.PIPE,
text=True,
encoding="utf-8",
)
# Examples that should NOT trigger the rule
from foo import PIPE
subprocess.run(["foo"], stdout=PIPE, stderr=PIPE)
run(["foo"], stdout=None, stderr=PIPE)

View File

@@ -1,96 +0,0 @@
# project
An example multi-package Python project used to test setting resolution and other complex
behaviors.
## Expected behavior
Running from the repo root should pick up and enforce the appropriate settings for each package:
```console
∴ cargo run -p ruff_cli -- check crates/ruff/resources/test/project/
crates/ruff/resources/test/project/examples/.dotfiles/script.py:1:1: I001 [*] Import block is un-sorted or un-formatted
crates/ruff/resources/test/project/examples/.dotfiles/script.py:1:8: F401 [*] `numpy` imported but unused
crates/ruff/resources/test/project/examples/.dotfiles/script.py:2:17: F401 [*] `app.app_file` imported but unused
crates/ruff/resources/test/project/examples/docs/docs/file.py:1:1: I001 [*] Import block is un-sorted or un-formatted
crates/ruff/resources/test/project/examples/docs/docs/file.py:8:5: F841 [*] Local variable `x` is assigned to but never used
crates/ruff/resources/test/project/project/file.py:1:8: F401 [*] `os` imported but unused
crates/ruff/resources/test/project/project/import_file.py:1:1: I001 [*] Import block is un-sorted or un-formatted
Found 7 errors.
[*] 7 potentially fixable with the --fix option.
```
Running from the project directory itself should exhibit the same behavior:
```console
∴ (cd crates/ruff/resources/test/project/ && cargo run -p ruff_cli -- check .)
examples/.dotfiles/script.py:1:1: I001 [*] Import block is un-sorted or un-formatted
examples/.dotfiles/script.py:1:8: F401 [*] `numpy` imported but unused
examples/.dotfiles/script.py:2:17: F401 [*] `app.app_file` imported but unused
examples/docs/docs/file.py:1:1: I001 [*] Import block is un-sorted or un-formatted
examples/docs/docs/file.py:8:5: F841 [*] Local variable `x` is assigned to but never used
project/file.py:1:8: F401 [*] `os` imported but unused
project/import_file.py:1:1: I001 [*] Import block is un-sorted or un-formatted
Found 7 errors.
[*] 7 potentially fixable with the --fix option.
```
Running from the sub-package directory should exhibit the same behavior, but omit the top-level
files:
```console
∴ (cd crates/ruff/resources/test/project/examples/docs && cargo run -p ruff_cli -- check .)
docs/file.py:1:1: I001 [*] Import block is un-sorted or un-formatted
docs/file.py:8:5: F841 [*] Local variable `x` is assigned to but never used
Found 2 errors.
[*] 2 potentially fixable with the --fix option.
```
`--config` should force Ruff to use the specified `pyproject.toml` for all files, and resolve
file paths from the current working directory:
```console
∴ (cargo run -p ruff_cli -- check --config=crates/ruff/resources/test/project/pyproject.toml crates/ruff/resources/test/project/)
crates/ruff/resources/test/project/examples/.dotfiles/script.py:1:8: F401 [*] `numpy` imported but unused
crates/ruff/resources/test/project/examples/.dotfiles/script.py:2:17: F401 [*] `app.app_file` imported but unused
crates/ruff/resources/test/project/examples/docs/docs/concepts/file.py:1:8: F401 [*] `os` imported but unused
crates/ruff/resources/test/project/examples/docs/docs/file.py:1:1: I001 [*] Import block is un-sorted or un-formatted
crates/ruff/resources/test/project/examples/docs/docs/file.py:1:8: F401 [*] `os` imported but unused
crates/ruff/resources/test/project/examples/docs/docs/file.py:3:8: F401 [*] `numpy` imported but unused
crates/ruff/resources/test/project/examples/docs/docs/file.py:4:27: F401 [*] `docs.concepts.file` imported but unused
crates/ruff/resources/test/project/examples/excluded/script.py:1:8: F401 [*] `os` imported but unused
crates/ruff/resources/test/project/project/file.py:1:8: F401 [*] `os` imported but unused
Found 9 errors.
[*] 9 potentially fixable with the --fix option.
```
Running from a parent directory should "ignore" the `exclude` (hence, `concepts/file.py` gets
included in the output):
```console
∴ (cd crates/ruff/resources/test/project/examples && cargo run -p ruff_cli -- check --config=docs/ruff.toml .)
docs/docs/concepts/file.py:5:5: F841 [*] Local variable `x` is assigned to but never used
docs/docs/file.py:1:1: I001 [*] Import block is un-sorted or un-formatted
docs/docs/file.py:8:5: F841 [*] Local variable `x` is assigned to but never used
excluded/script.py:5:5: F841 [*] Local variable `x` is assigned to but never used
Found 4 errors.
[*] 4 potentially fixable with the --fix option.
```
Passing an excluded directory directly should report errors in the contained files:
```console
∴ cargo run -p ruff_cli -- check crates/ruff/resources/test/project/examples/excluded/
crates/ruff/resources/test/project/examples/excluded/script.py:1:8: F401 [*] `os` imported but unused
Found 1 error.
[*] 1 potentially fixable with the --fix option.
```
Unless we `--force-exclude`:
```console
∴ cargo run -p ruff_cli -- check crates/ruff/resources/test/project/examples/excluded/ --force-exclude
warning: No Python files found under the given path(s)
∴ cargo run -p ruff_cli -- check crates/ruff/resources/test/project/examples/excluded/script.py --force-exclude
warning: No Python files found under the given path(s)
```

View File

@@ -1,364 +0,0 @@
//! Interface for generating autofix edits from higher-level actions (e.g., "remove an argument").
use anyhow::{bail, Result};
use ruff_diagnostics::Edit;
use ruff_python_ast::{
self as ast, Arguments, ExceptHandler, Expr, Keyword, PySourceType, Ranged, Stmt,
};
use ruff_python_codegen::Stylist;
use ruff_python_index::Indexer;
use ruff_python_parser::{lexer, AsMode};
use ruff_python_trivia::{has_leading_content, is_python_whitespace, PythonWhitespace};
use ruff_source_file::{Locator, NewlineWithTrailingNewline};
use ruff_text_size::{TextLen, TextRange, TextSize};
use crate::autofix::codemods;
/// Return the `Fix` to use when deleting a `Stmt`.
///
/// In some cases, this is as simple as deleting the `Range` of the `Stmt`
/// itself. However, there are a few exceptions:
/// - If the `Stmt` is _not_ the terminal statement in a multi-statement line,
/// we need to delete up to the start of the next statement (and avoid
/// deleting any content that precedes the statement).
/// - If the `Stmt` is the terminal statement in a multi-statement line, we need
/// to avoid deleting any content that precedes the statement.
/// - If the `Stmt` has no trailing and leading content, then it's convenient to
/// remove the entire start and end lines.
/// - If the `Stmt` is the last statement in its parent body, replace it with a
/// `pass` instead.
pub(crate) fn delete_stmt(
stmt: &Stmt,
parent: Option<&Stmt>,
locator: &Locator,
indexer: &Indexer,
) -> Edit {
if parent
.map(|parent| is_lone_child(stmt, parent))
.unwrap_or_default()
{
// If removing this node would lead to an invalid syntax tree, replace
// it with a `pass`.
Edit::range_replacement("pass".to_string(), stmt.range())
} else {
if let Some(semicolon) = trailing_semicolon(stmt.end(), locator) {
let next = next_stmt_break(semicolon, locator);
Edit::deletion(stmt.start(), next)
} else if has_leading_content(stmt.start(), locator) {
Edit::range_deletion(stmt.range())
} else if let Some(start) = indexer.preceded_by_continuations(stmt.start(), locator) {
Edit::range_deletion(TextRange::new(start, stmt.end()))
} else {
let range = locator.full_lines_range(stmt.range());
Edit::range_deletion(range)
}
}
}
/// Generate a `Fix` to remove the specified imports from an `import` statement.
pub(crate) fn remove_unused_imports<'a>(
member_names: impl Iterator<Item = &'a str>,
stmt: &Stmt,
parent: Option<&Stmt>,
locator: &Locator,
stylist: &Stylist,
indexer: &Indexer,
) -> Result<Edit> {
match codemods::remove_imports(member_names, stmt, locator, stylist)? {
None => Ok(delete_stmt(stmt, parent, locator, indexer)),
Some(content) => Ok(Edit::range_replacement(content, stmt.range())),
}
}
#[derive(Debug, Copy, Clone)]
pub(crate) enum Parentheses {
/// Remove parentheses, if the removed argument is the only argument left.
Remove,
/// Preserve parentheses, even if the removed argument is the only argument
Preserve,
}
/// Generic function to remove arguments or keyword arguments in function
/// calls and class definitions. (For classes `args` should be considered
/// `bases`)
///
/// Supports the removal of parentheses when this is the only (kw)arg left.
/// For this behavior, set `remove_parentheses` to `true`.
pub(crate) fn remove_argument<T: Ranged>(
argument: &T,
arguments: &Arguments,
parentheses: Parentheses,
locator: &Locator,
source_type: PySourceType,
) -> Result<Edit> {
// TODO(sbrugman): Preserve trailing comments.
if arguments.keywords.len() + arguments.args.len() > 1 {
let mut fix_start = None;
let mut fix_end = None;
if arguments
.args
.iter()
.map(Expr::start)
.chain(arguments.keywords.iter().map(Keyword::start))
.any(|location| location > argument.start())
{
// Case 1: argument or keyword is _not_ the last node, so delete from the start of the
// argument to the end of the subsequent comma.
let mut seen_comma = false;
for (tok, range) in lexer::lex_starts_at(
locator.slice(arguments.range()),
source_type.as_mode(),
arguments.start(),
)
.flatten()
{
if seen_comma {
if tok.is_non_logical_newline() {
// Also delete any non-logical newlines after the comma.
continue;
}
fix_end = Some(if tok.is_newline() {
range.end()
} else {
range.start()
});
break;
}
if range.start() == argument.start() {
fix_start = Some(range.start());
}
if fix_start.is_some() && tok.is_comma() {
seen_comma = true;
}
}
} else {
// Case 2: argument or keyword is the last node, so delete from the start of the
// previous comma to the end of the argument.
for (tok, range) in lexer::lex_starts_at(
locator.slice(arguments.range()),
source_type.as_mode(),
arguments.start(),
)
.flatten()
{
if range.start() == argument.start() {
fix_end = Some(argument.end());
break;
}
if tok.is_comma() {
fix_start = Some(range.start());
}
}
}
match (fix_start, fix_end) {
(Some(start), Some(end)) => Ok(Edit::deletion(start, end)),
_ => {
bail!("No fix could be constructed")
}
}
} else {
// Only one argument; remove it (but preserve parentheses, if needed).
Ok(match parentheses {
Parentheses::Remove => Edit::deletion(arguments.start(), arguments.end()),
Parentheses::Preserve => {
Edit::replacement("()".to_string(), arguments.start(), arguments.end())
}
})
}
}
/// Determine if a vector contains only one, specific element.
fn is_only<T: PartialEq>(vec: &[T], value: &T) -> bool {
vec.len() == 1 && vec[0] == *value
}
/// Determine if a child is the only statement in its body.
fn is_lone_child(child: &Stmt, parent: &Stmt) -> bool {
match parent {
Stmt::FunctionDef(ast::StmtFunctionDef { body, .. })
| Stmt::ClassDef(ast::StmtClassDef { body, .. })
| Stmt::With(ast::StmtWith { body, .. }) => {
if is_only(body, child) {
return true;
}
}
Stmt::For(ast::StmtFor { body, orelse, .. })
| Stmt::While(ast::StmtWhile { body, orelse, .. }) => {
if is_only(body, child) || is_only(orelse, child) {
return true;
}
}
Stmt::If(ast::StmtIf {
body,
elif_else_clauses,
..
}) => {
if is_only(body, child)
|| elif_else_clauses
.iter()
.any(|ast::ElifElseClause { body, .. }| is_only(body, child))
{
return true;
}
}
Stmt::Try(ast::StmtTry {
body,
handlers,
orelse,
finalbody,
..
}) => {
if is_only(body, child)
|| is_only(orelse, child)
|| is_only(finalbody, child)
|| handlers.iter().any(|handler| match handler {
ExceptHandler::ExceptHandler(ast::ExceptHandlerExceptHandler {
body, ..
}) => is_only(body, child),
})
{
return true;
}
}
Stmt::Match(ast::StmtMatch { cases, .. }) => {
if cases.iter().any(|case| is_only(&case.body, child)) {
return true;
}
}
_ => {}
}
false
}
/// Return the location of a trailing semicolon following a `Stmt`, if it's part
/// of a multi-statement line.
fn trailing_semicolon(offset: TextSize, locator: &Locator) -> Option<TextSize> {
let contents = locator.after(offset);
for line in NewlineWithTrailingNewline::from(contents) {
let trimmed = line.trim_whitespace_start();
if trimmed.starts_with(';') {
let colon_offset = line.text_len() - trimmed.text_len();
return Some(offset + line.start() + colon_offset);
}
if !trimmed.starts_with('\\') {
break;
}
}
None
}
/// Find the next valid break for a `Stmt` after a semicolon.
fn next_stmt_break(semicolon: TextSize, locator: &Locator) -> TextSize {
let start_location = semicolon + TextSize::from(1);
let contents = &locator.contents()[usize::from(start_location)..];
for line in NewlineWithTrailingNewline::from(contents) {
let trimmed = line.trim_whitespace();
// Skip past any continuations.
if trimmed.starts_with('\\') {
continue;
}
return start_location
+ if trimmed.is_empty() {
// If the line is empty, then despite the previous statement ending in a
// semicolon, we know that it's not a multi-statement line.
line.start()
} else {
// Otherwise, find the start of the next statement. (Or, anything that isn't
// whitespace.)
let relative_offset = line.find(|c: char| !is_python_whitespace(c)).unwrap();
line.start() + TextSize::try_from(relative_offset).unwrap()
};
}
locator.line_end(start_location)
}
#[cfg(test)]
mod tests {
use anyhow::Result;
use ruff_python_ast::Ranged;
use ruff_python_parser::parse_suite;
use ruff_source_file::Locator;
use ruff_text_size::TextSize;
use crate::autofix::edits::{next_stmt_break, trailing_semicolon};
#[test]
fn find_semicolon() -> Result<()> {
let contents = "x = 1";
let program = parse_suite(contents, "<filename>")?;
let stmt = program.first().unwrap();
let locator = Locator::new(contents);
assert_eq!(trailing_semicolon(stmt.end(), &locator), None);
let contents = "x = 1; y = 1";
let program = parse_suite(contents, "<filename>")?;
let stmt = program.first().unwrap();
let locator = Locator::new(contents);
assert_eq!(
trailing_semicolon(stmt.end(), &locator),
Some(TextSize::from(5))
);
let contents = "x = 1 ; y = 1";
let program = parse_suite(contents, "<filename>")?;
let stmt = program.first().unwrap();
let locator = Locator::new(contents);
assert_eq!(
trailing_semicolon(stmt.end(), &locator),
Some(TextSize::from(6))
);
let contents = r"
x = 1 \
; y = 1
"
.trim();
let program = parse_suite(contents, "<filename>")?;
let stmt = program.first().unwrap();
let locator = Locator::new(contents);
assert_eq!(
trailing_semicolon(stmt.end(), &locator),
Some(TextSize::from(10))
);
Ok(())
}
#[test]
fn find_next_stmt_break() {
let contents = "x = 1; y = 1";
let locator = Locator::new(contents);
assert_eq!(
next_stmt_break(TextSize::from(4), &locator),
TextSize::from(5)
);
let contents = "x = 1 ; y = 1";
let locator = Locator::new(contents);
assert_eq!(
next_stmt_break(TextSize::from(5), &locator),
TextSize::from(6)
);
let contents = r"
x = 1 \
; y = 1
"
.trim();
let locator = Locator::new(contents);
assert_eq!(
next_stmt_break(TextSize::from(10), &locator),
TextSize::from(12)
);
}
}

View File

@@ -1,397 +0,0 @@
use itertools::Itertools;
use std::collections::BTreeSet;
use ruff_text_size::{TextLen, TextRange, TextSize};
use rustc_hash::{FxHashMap, FxHashSet};
use ruff_diagnostics::{Diagnostic, Edit, Fix, IsolationLevel};
use ruff_source_file::Locator;
use crate::autofix::source_map::SourceMap;
use crate::linter::FixTable;
use crate::registry::{AsRule, Rule};
pub(crate) mod codemods;
pub(crate) mod edits;
pub(crate) mod source_map;
pub(crate) struct FixResult {
/// The resulting source code, after applying all fixes.
pub(crate) code: String,
/// The number of fixes applied for each [`Rule`].
pub(crate) fixes: FixTable,
/// Source map for the fixed source code.
pub(crate) source_map: SourceMap,
}
/// Auto-fix errors in a file, and write the fixed source code to disk.
pub(crate) fn fix_file(diagnostics: &[Diagnostic], locator: &Locator) -> Option<FixResult> {
let mut with_fixes = diagnostics
.iter()
.filter(|diag| diag.fix.is_some())
.peekable();
if with_fixes.peek().is_none() {
None
} else {
Some(apply_fixes(with_fixes, locator))
}
}
/// Apply a series of fixes.
fn apply_fixes<'a>(
diagnostics: impl Iterator<Item = &'a Diagnostic>,
locator: &'a Locator<'a>,
) -> FixResult {
let mut output = String::with_capacity(locator.len());
let mut last_pos: Option<TextSize> = None;
let mut applied: BTreeSet<&Edit> = BTreeSet::default();
let mut isolated: FxHashSet<u32> = FxHashSet::default();
let mut fixed = FxHashMap::default();
let mut source_map = SourceMap::default();
for (rule, fix) in diagnostics
.filter_map(|diagnostic| {
diagnostic
.fix
.as_ref()
.map(|fix| (diagnostic.kind.rule(), fix))
})
.sorted_by(|(rule1, fix1), (rule2, fix2)| cmp_fix(*rule1, *rule2, fix1, fix2))
{
let mut edits = fix
.edits()
.iter()
.filter(|edit| !applied.contains(edit))
.peekable();
// If the fix contains at least one new edit, enforce isolation and positional requirements.
if let Some(first) = edits.peek() {
// If this fix requires isolation, and we've already applied another fix in the
// same isolation group, skip it.
if let IsolationLevel::Group(id) = fix.isolation() {
if !isolated.insert(id) {
continue;
}
}
// If this fix overlaps with a fix we've already applied, skip it.
if last_pos.is_some_and(|last_pos| last_pos >= first.start()) {
continue;
}
}
let mut applied_edits = Vec::with_capacity(fix.edits().len());
for edit in edits {
// Add all contents from `last_pos` to `fix.location`.
let slice = locator.slice(TextRange::new(last_pos.unwrap_or_default(), edit.start()));
output.push_str(slice);
// Add the start source marker for the patch.
source_map.push_start_marker(edit, output.text_len());
// Add the patch itself.
output.push_str(edit.content().unwrap_or_default());
// Add the end source marker for the added patch.
source_map.push_end_marker(edit, output.text_len());
// Track that the edit was applied.
last_pos = Some(edit.end());
applied_edits.push(edit);
}
applied.extend(applied_edits.drain(..));
*fixed.entry(rule).or_default() += 1;
}
// Add the remaining content.
let slice = locator.after(last_pos.unwrap_or_default());
output.push_str(slice);
FixResult {
code: output,
fixes: fixed,
source_map,
}
}
/// Compare two fixes.
fn cmp_fix(rule1: Rule, rule2: Rule, fix1: &Fix, fix2: &Fix) -> std::cmp::Ordering {
fix1.min_start()
.cmp(&fix2.min_start())
.then_with(|| match (&rule1, &rule2) {
// Apply `EndsInPeriod` fixes before `NewLineAfterLastParagraph` fixes.
(Rule::EndsInPeriod, Rule::NewLineAfterLastParagraph) => std::cmp::Ordering::Less,
(Rule::NewLineAfterLastParagraph, Rule::EndsInPeriod) => std::cmp::Ordering::Greater,
// Apply `IfElseBlockInsteadOfDictGet` fixes before `IfElseBlockInsteadOfIfExp` fixes.
(Rule::IfElseBlockInsteadOfDictGet, Rule::IfElseBlockInsteadOfIfExp) => {
std::cmp::Ordering::Less
}
(Rule::IfElseBlockInsteadOfIfExp, Rule::IfElseBlockInsteadOfDictGet) => {
std::cmp::Ordering::Greater
}
_ => std::cmp::Ordering::Equal,
})
}
#[cfg(test)]
mod tests {
use ruff_text_size::TextSize;
use ruff_diagnostics::Diagnostic;
use ruff_diagnostics::Edit;
use ruff_diagnostics::Fix;
use ruff_source_file::Locator;
use crate::autofix::source_map::SourceMarker;
use crate::autofix::{apply_fixes, FixResult};
use crate::rules::pycodestyle::rules::MissingNewlineAtEndOfFile;
#[allow(deprecated)]
fn create_diagnostics(edit: impl IntoIterator<Item = Edit>) -> Vec<Diagnostic> {
edit.into_iter()
.map(|edit| Diagnostic {
// The choice of rule here is arbitrary.
kind: MissingNewlineAtEndOfFile.into(),
range: edit.range(),
fix: Some(Fix::unspecified(edit)),
parent: None,
})
.collect()
}
#[test]
fn empty_file() {
let locator = Locator::new(r#""#);
let diagnostics = create_diagnostics([]);
let FixResult {
code,
fixes,
source_map,
} = apply_fixes(diagnostics.iter(), &locator);
assert_eq!(code, "");
assert_eq!(fixes.values().sum::<usize>(), 0);
assert!(source_map.markers().is_empty());
}
#[test]
fn apply_one_insertion() {
let locator = Locator::new(
r#"
import os
print("hello world")
"#
.trim(),
);
let diagnostics = create_diagnostics([Edit::insertion(
"import sys\n".to_string(),
TextSize::new(10),
)]);
let FixResult {
code,
fixes,
source_map,
} = apply_fixes(diagnostics.iter(), &locator);
assert_eq!(
code,
r#"
import os
import sys
print("hello world")
"#
.trim()
);
assert_eq!(fixes.values().sum::<usize>(), 1);
assert_eq!(
source_map.markers(),
&[
SourceMarker {
source: 10.into(),
dest: 10.into(),
},
SourceMarker {
source: 10.into(),
dest: 21.into(),
},
]
);
}
#[test]
fn apply_one_replacement() {
let locator = Locator::new(
r#"
class A(object):
...
"#
.trim(),
);
let diagnostics = create_diagnostics([Edit::replacement(
"Bar".to_string(),
TextSize::new(8),
TextSize::new(14),
)]);
let FixResult {
code,
fixes,
source_map,
} = apply_fixes(diagnostics.iter(), &locator);
assert_eq!(
code,
r#"
class A(Bar):
...
"#
.trim(),
);
assert_eq!(fixes.values().sum::<usize>(), 1);
assert_eq!(
source_map.markers(),
&[
SourceMarker {
source: 8.into(),
dest: 8.into(),
},
SourceMarker {
source: 14.into(),
dest: 11.into(),
},
]
);
}
#[test]
fn apply_one_removal() {
let locator = Locator::new(
r#"
class A(object):
...
"#
.trim(),
);
let diagnostics = create_diagnostics([Edit::deletion(TextSize::new(7), TextSize::new(15))]);
let FixResult {
code,
fixes,
source_map,
} = apply_fixes(diagnostics.iter(), &locator);
assert_eq!(
code,
r#"
class A:
...
"#
.trim()
);
assert_eq!(fixes.values().sum::<usize>(), 1);
assert_eq!(
source_map.markers(),
&[
SourceMarker {
source: 7.into(),
dest: 7.into()
},
SourceMarker {
source: 15.into(),
dest: 7.into()
}
]
);
}
#[test]
fn apply_two_removals() {
let locator = Locator::new(
r#"
class A(object, object, object):
...
"#
.trim(),
);
let diagnostics = create_diagnostics([
Edit::deletion(TextSize::from(8), TextSize::from(16)),
Edit::deletion(TextSize::from(22), TextSize::from(30)),
]);
let FixResult {
code,
fixes,
source_map,
} = apply_fixes(diagnostics.iter(), &locator);
assert_eq!(
code,
r#"
class A(object):
...
"#
.trim()
);
assert_eq!(fixes.values().sum::<usize>(), 2);
assert_eq!(
source_map.markers(),
&[
SourceMarker {
source: 8.into(),
dest: 8.into()
},
SourceMarker {
source: 16.into(),
dest: 8.into()
},
SourceMarker {
source: 22.into(),
dest: 14.into(),
},
SourceMarker {
source: 30.into(),
dest: 14.into(),
}
]
);
}
#[test]
fn ignore_overlapping_fixes() {
let locator = Locator::new(
r#"
class A(object):
...
"#
.trim(),
);
let diagnostics = create_diagnostics([
Edit::deletion(TextSize::from(7), TextSize::from(15)),
Edit::replacement("ignored".to_string(), TextSize::from(9), TextSize::from(11)),
]);
let FixResult {
code,
fixes,
source_map,
} = apply_fixes(diagnostics.iter(), &locator);
assert_eq!(
code,
r#"
class A:
...
"#
.trim(),
);
assert_eq!(fixes.values().sum::<usize>(), 1);
assert_eq!(
source_map.markers(),
&[
SourceMarker {
source: 7.into(),
dest: 7.into(),
},
SourceMarker {
source: 15.into(),
dest: 7.into(),
}
]
);
}
}

View File

@@ -1,59 +0,0 @@
use ruff_text_size::TextSize;
use ruff_diagnostics::Edit;
/// Lightweight sourcemap marker representing the source and destination
/// position for an [`Edit`].
#[derive(Debug, PartialEq, Eq)]
pub(crate) struct SourceMarker {
/// Position of the marker in the original source.
pub(crate) source: TextSize,
/// Position of the marker in the transformed code.
pub(crate) dest: TextSize,
}
/// A collection of [`SourceMarker`].
///
/// Sourcemaps are used to map positions in the original source to positions in
/// the transformed code. Here, only the boundaries of edits are tracked instead
/// of every single character.
#[derive(Default, PartialEq, Eq)]
pub(crate) struct SourceMap(Vec<SourceMarker>);
impl SourceMap {
/// Returns a slice of all the markers in the sourcemap in the order they
/// were added.
pub(crate) fn markers(&self) -> &[SourceMarker] {
&self.0
}
/// Push the start marker for an [`Edit`].
///
/// The `output_length` is the length of the transformed string before the
/// edit is applied.
pub(crate) fn push_start_marker(&mut self, edit: &Edit, output_length: TextSize) {
self.0.push(SourceMarker {
source: edit.start(),
dest: output_length,
});
}
/// Push the end marker for an [`Edit`].
///
/// The `output_length` is the length of the transformed string after the
/// edit has been applied.
pub(crate) fn push_end_marker(&mut self, edit: &Edit, output_length: TextSize) {
if edit.is_insertion() {
self.0.push(SourceMarker {
source: edit.start(),
dest: output_length,
});
} else {
// Deletion or replacement
self.0.push(SourceMarker {
source: edit.end(),
dest: output_length,
});
}
}
}

View File

@@ -1,16 +0,0 @@
use ruff_python_ast::Comprehension;
use crate::checkers::ast::Checker;
use crate::codes::Rule;
use crate::rules::flake8_simplify;
/// Run lint rules over a [`Comprehension`] syntax nodes.
pub(crate) fn comprehension(comprehension: &Comprehension, checker: &mut Checker) {
if checker.enabled(Rule::InDictKeys) {
flake8_simplify::rules::key_in_dict_for(
checker,
&comprehension.target,
&comprehension.iter,
);
}
}

View File

@@ -1,29 +0,0 @@
use ruff_python_ast::{self as ast, Stmt};
use crate::checkers::ast::Checker;
use crate::codes::Rule;
use crate::rules::{flake8_bugbear, perflint};
/// Run lint rules over all deferred for-loops in the [`SemanticModel`].
pub(crate) fn deferred_for_loops(checker: &mut Checker) {
while !checker.deferred.for_loops.is_empty() {
let for_loops = std::mem::take(&mut checker.deferred.for_loops);
for snapshot in for_loops {
checker.semantic.restore(snapshot);
let Stmt::For(ast::StmtFor {
target, iter, body, ..
}) = checker.semantic.current_statement()
else {
unreachable!("Expected Stmt::For");
};
if checker.enabled(Rule::UnusedLoopControlVariable) {
flake8_bugbear::rules::unused_loop_control_variable(checker, target, body);
}
if checker.enabled(Rule::IncorrectDictIterator) {
perflint::rules::incorrect_dict_iterator(checker, target, iter);
}
}
}
}

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -1,134 +0,0 @@
//! Lint rules based on import analysis.
use std::borrow::Cow;
use std::path::Path;
use ruff_python_ast::{self as ast, PySourceType, Ranged, Stmt, Suite};
use ruff_diagnostics::Diagnostic;
use ruff_python_ast::helpers::to_module_path;
use ruff_python_ast::imports::{ImportMap, ModuleImport};
use ruff_python_ast::statement_visitor::StatementVisitor;
use ruff_python_codegen::Stylist;
use ruff_python_index::Indexer;
use ruff_source_file::Locator;
use crate::directives::IsortDirectives;
use crate::registry::Rule;
use crate::rules::isort;
use crate::rules::isort::block::{Block, BlockBuilder};
use crate::settings::Settings;
use crate::source_kind::SourceKind;
fn extract_import_map(path: &Path, package: Option<&Path>, blocks: &[&Block]) -> Option<ImportMap> {
let Some(package) = package else {
return None;
};
let Some(module_path) = to_module_path(package, path) else {
return None;
};
let num_imports = blocks.iter().map(|block| block.imports.len()).sum();
let mut module_imports = Vec::with_capacity(num_imports);
for stmt in blocks.iter().flat_map(|block| &block.imports) {
match stmt {
Stmt::Import(ast::StmtImport { names, range: _ }) => {
module_imports.extend(
names
.iter()
.map(|name| ModuleImport::new(name.name.to_string(), stmt.range())),
);
}
Stmt::ImportFrom(ast::StmtImportFrom {
module,
names,
level,
range: _,
}) => {
let level = level.map_or(0, |level| level.to_usize());
let module = if let Some(module) = module {
let module: &String = module.as_ref();
if level == 0 {
Cow::Borrowed(module)
} else {
if module_path.len() <= level {
continue;
}
let prefix = module_path[..module_path.len() - level].join(".");
Cow::Owned(format!("{prefix}.{module}"))
}
} else {
if module_path.len() <= level {
continue;
}
Cow::Owned(module_path[..module_path.len() - level].join("."))
};
module_imports.extend(names.iter().map(|name| {
ModuleImport::new(format!("{}.{}", module, name.name), name.range())
}));
}
_ => panic!("Expected Stmt::Import | Stmt::ImportFrom"),
}
}
let mut import_map = ImportMap::default();
import_map.insert(module_path.join("."), module_imports);
Some(import_map)
}
#[allow(clippy::too_many_arguments)]
pub(crate) fn check_imports(
python_ast: &Suite,
locator: &Locator,
indexer: &Indexer,
directives: &IsortDirectives,
settings: &Settings,
stylist: &Stylist,
path: &Path,
package: Option<&Path>,
source_kind: Option<&SourceKind>,
source_type: PySourceType,
) -> (Vec<Diagnostic>, Option<ImportMap>) {
// Extract all import blocks from the AST.
let tracker = {
let mut tracker =
BlockBuilder::new(locator, directives, source_type.is_stub(), source_kind);
tracker.visit_body(python_ast);
tracker
};
let blocks: Vec<&Block> = tracker.iter().collect();
// Enforce import rules.
let mut diagnostics = vec![];
if settings.rules.enabled(Rule::UnsortedImports) {
for block in &blocks {
if !block.imports.is_empty() {
if let Some(diagnostic) = isort::rules::organize_imports(
block,
locator,
stylist,
indexer,
settings,
package,
source_type,
) {
diagnostics.push(diagnostic);
}
}
}
}
if settings.rules.enabled(Rule::MissingRequiredImport) {
diagnostics.extend(isort::rules::add_required_imports(
python_ast,
locator,
stylist,
settings,
source_type,
));
}
// Extract import map.
let imports = extract_import_map(path, package, &blocks);
(diagnostics, imports)
}

View File

@@ -1,243 +0,0 @@
//! `NoQA` enforcement and validation.
use std::path::Path;
use itertools::Itertools;
use ruff_python_ast::Ranged;
use ruff_text_size::{TextLen, TextRange, TextSize};
use ruff_diagnostics::{Diagnostic, Edit, Fix};
use ruff_source_file::Locator;
use crate::noqa;
use crate::noqa::{Directive, FileExemption, NoqaDirectives, NoqaMapping};
use crate::registry::{AsRule, Rule};
use crate::rule_redirects::get_redirect_target;
use crate::rules::ruff::rules::{UnusedCodes, UnusedNOQA};
use crate::settings::Settings;
pub(crate) fn check_noqa(
diagnostics: &mut Vec<Diagnostic>,
path: &Path,
locator: &Locator,
comment_ranges: &[TextRange],
noqa_line_for: &NoqaMapping,
analyze_directives: bool,
settings: &Settings,
) -> Vec<usize> {
// Identify any codes that are globally exempted (within the current file).
let exemption = FileExemption::try_extract(locator.contents(), comment_ranges, path, locator);
// Extract all `noqa` directives.
let mut noqa_directives = NoqaDirectives::from_commented_ranges(comment_ranges, path, locator);
// Indices of diagnostics that were ignored by a `noqa` directive.
let mut ignored_diagnostics = vec![];
// Remove any ignored diagnostics.
'outer: for (index, diagnostic) in diagnostics.iter().enumerate() {
if matches!(diagnostic.kind.rule(), Rule::BlanketNOQA) {
continue;
}
match &exemption {
Some(FileExemption::All) => {
// If the file is exempted, ignore all diagnostics.
ignored_diagnostics.push(index);
continue;
}
Some(FileExemption::Codes(codes)) => {
// If the diagnostic is ignored by a global exemption, ignore it.
if codes.contains(&diagnostic.kind.rule().noqa_code()) {
ignored_diagnostics.push(index);
continue;
}
}
None => {}
}
let noqa_offsets = diagnostic
.parent
.into_iter()
.chain(std::iter::once(diagnostic.start()))
.map(|position| noqa_line_for.resolve(position))
.unique();
for noqa_offset in noqa_offsets {
if let Some(directive_line) = noqa_directives.find_line_with_directive_mut(noqa_offset)
{
let suppressed = match &directive_line.directive {
Directive::All(_) => {
directive_line
.matches
.push(diagnostic.kind.rule().noqa_code());
ignored_diagnostics.push(index);
true
}
Directive::Codes(directive) => {
if noqa::includes(diagnostic.kind.rule(), directive.codes()) {
directive_line
.matches
.push(diagnostic.kind.rule().noqa_code());
ignored_diagnostics.push(index);
true
} else {
false
}
}
};
if suppressed {
continue 'outer;
}
}
}
}
// Enforce that the noqa directive was actually used (RUF100), unless RUF100 was itself
// suppressed.
if settings.rules.enabled(Rule::UnusedNOQA)
&& analyze_directives
&& !exemption.is_some_and(|exemption| match exemption {
FileExemption::All => true,
FileExemption::Codes(codes) => codes.contains(&Rule::UnusedNOQA.noqa_code()),
})
{
for line in noqa_directives.lines() {
match &line.directive {
Directive::All(directive) => {
if line.matches.is_empty() {
let mut diagnostic =
Diagnostic::new(UnusedNOQA { codes: None }, directive.range());
if settings.rules.should_fix(diagnostic.kind.rule()) {
#[allow(deprecated)]
diagnostic.set_fix_from_edit(delete_noqa(directive.range(), locator));
}
diagnostics.push(diagnostic);
}
}
Directive::Codes(directive) => {
let mut disabled_codes = vec![];
let mut unknown_codes = vec![];
let mut unmatched_codes = vec![];
let mut valid_codes = vec![];
let mut self_ignore = false;
for code in directive.codes() {
let code = get_redirect_target(code).unwrap_or(code);
if Rule::UnusedNOQA.noqa_code() == code {
self_ignore = true;
break;
}
if line.matches.iter().any(|match_| *match_ == code)
|| settings.external.contains(code)
{
valid_codes.push(code);
} else {
if let Ok(rule) = Rule::from_code(code) {
if settings.rules.enabled(rule) {
unmatched_codes.push(code);
} else {
disabled_codes.push(code);
}
} else {
unknown_codes.push(code);
}
}
}
if self_ignore {
continue;
}
if !(disabled_codes.is_empty()
&& unknown_codes.is_empty()
&& unmatched_codes.is_empty())
{
let mut diagnostic = Diagnostic::new(
UnusedNOQA {
codes: Some(UnusedCodes {
disabled: disabled_codes
.iter()
.map(|code| (*code).to_string())
.collect(),
unknown: unknown_codes
.iter()
.map(|code| (*code).to_string())
.collect(),
unmatched: unmatched_codes
.iter()
.map(|code| (*code).to_string())
.collect(),
}),
},
directive.range(),
);
if settings.rules.should_fix(diagnostic.kind.rule()) {
if valid_codes.is_empty() {
#[allow(deprecated)]
diagnostic
.set_fix_from_edit(delete_noqa(directive.range(), locator));
} else {
#[allow(deprecated)]
diagnostic.set_fix(Fix::unspecified(Edit::range_replacement(
format!("# noqa: {}", valid_codes.join(", ")),
directive.range(),
)));
}
}
diagnostics.push(diagnostic);
}
}
}
}
}
ignored_diagnostics.sort_unstable();
ignored_diagnostics
}
/// Generate a [`Edit`] to delete a `noqa` directive.
fn delete_noqa(range: TextRange, locator: &Locator) -> Edit {
let line_range = locator.line_range(range.start());
// Compute the leading space.
let prefix = locator.slice(TextRange::new(line_range.start(), range.start()));
let leading_space = prefix
.rfind(|c: char| !c.is_whitespace())
.map_or(prefix.len(), |i| prefix.len() - i - 1);
let leading_space_len = TextSize::try_from(leading_space).unwrap();
// Compute the trailing space.
let suffix = locator.slice(TextRange::new(range.end(), line_range.end()));
let trailing_space = suffix
.find(|c: char| !c.is_whitespace())
.map_or(suffix.len(), |i| i);
let trailing_space_len = TextSize::try_from(trailing_space).unwrap();
// Ex) `# noqa`
if line_range
== TextRange::new(
range.start() - leading_space_len,
range.end() + trailing_space_len,
)
{
let full_line_end = locator.full_line_end(line_range.end());
Edit::deletion(line_range.start(), full_line_end)
}
// Ex) `x = 1 # noqa`
else if range.end() + trailing_space_len == line_range.end() {
Edit::deletion(range.start() - leading_space_len, line_range.end())
}
// Ex) `x = 1 # noqa # type: ignore`
else if locator.contents()[usize::from(range.end() + trailing_space_len)..].starts_with('#') {
Edit::deletion(range.start(), range.end() + trailing_space_len)
}
// Ex) `x = 1 # noqa here`
else {
Edit::deletion(
range.start() + "# ".text_len(),
range.end() + trailing_space_len,
)
}
}

View File

@@ -1,5 +0,0 @@
---
source: crates/ruff/src/comments/shebang.rs
expression: "ShebangDirective::try_extract(source)"
---
None

View File

@@ -1,5 +0,0 @@
---
source: crates/ruff/src/comments/shebang.rs
expression: "ShebangDirective::try_extract(source)"
---
None

View File

@@ -1,9 +0,0 @@
---
source: crates/ruff/src/comments/shebang.rs
expression: "ShebangDirective::try_extract(source)"
---
Some(
ShebangDirective(
"/usr/bin/env python",
),
)

View File

@@ -1,9 +0,0 @@
---
source: crates/ruff/src/comments/shebang.rs
expression: "ShebangDirective::try_extract(source)"
---
Some(
ShebangDirective(
"/usr/bin/env python # trailing comment",
),
)

View File

@@ -1,5 +0,0 @@
---
source: crates/ruff/src/comments/shebang.rs
expression: "ShebangDirective::try_extract(source)"
---
None

View File

@@ -1,38 +0,0 @@
use libcst_native::{Expression, NameOrAttribute};
fn compose_call_path_inner<'a>(expr: &'a Expression, parts: &mut Vec<&'a str>) {
match expr {
Expression::Call(expr) => {
compose_call_path_inner(&expr.func, parts);
}
Expression::Attribute(expr) => {
compose_call_path_inner(&expr.value, parts);
parts.push(expr.attr.value);
}
Expression::Name(expr) => {
parts.push(expr.value);
}
_ => {}
}
}
pub(crate) fn compose_call_path(expr: &Expression) -> Option<String> {
let mut segments = vec![];
compose_call_path_inner(expr, &mut segments);
if segments.is_empty() {
None
} else {
Some(segments.join("."))
}
}
pub(crate) fn compose_module_path(module: &NameOrAttribute) -> String {
match module {
NameOrAttribute::N(name) => name.value.to_string(),
NameOrAttribute::A(attr) => {
let name = attr.attr.value;
let prefix = compose_call_path(&attr.value);
prefix.map_or_else(|| name.to_string(), |prefix| format!("{prefix}.{name}"))
}
}
}

View File

@@ -1,83 +0,0 @@
use std::fmt::{Debug, Formatter};
use std::ops::Deref;
use ruff_python_ast::{Expr, Ranged};
use ruff_text_size::{TextRange, TextSize};
use ruff_python_semantic::Definition;
pub(crate) mod extraction;
pub(crate) mod google;
pub(crate) mod numpy;
pub(crate) mod sections;
pub(crate) mod styles;
#[derive(Debug)]
pub(crate) struct Docstring<'a> {
pub(crate) definition: &'a Definition<'a>,
pub(crate) expr: &'a Expr,
/// The content of the docstring, including the leading and trailing quotes.
pub(crate) contents: &'a str,
/// The range of the docstring body (without the quotes). The range is relative to [`Self::contents`].
pub(crate) body_range: TextRange,
pub(crate) indentation: &'a str,
}
impl<'a> Docstring<'a> {
pub(crate) fn body(&self) -> DocstringBody {
DocstringBody { docstring: self }
}
pub(crate) fn start(&self) -> TextSize {
self.expr.start()
}
pub(crate) fn end(&self) -> TextSize {
self.expr.end()
}
pub(crate) fn range(&self) -> TextRange {
self.expr.range()
}
pub(crate) fn leading_quote(&self) -> &'a str {
&self.contents[TextRange::up_to(self.body_range.start())]
}
}
#[derive(Copy, Clone)]
pub(crate) struct DocstringBody<'a> {
docstring: &'a Docstring<'a>,
}
impl<'a> DocstringBody<'a> {
#[inline]
pub(crate) fn start(self) -> TextSize {
self.range().start()
}
pub(crate) fn range(self) -> TextRange {
self.docstring.body_range + self.docstring.start()
}
pub(crate) fn as_str(self) -> &'a str {
&self.docstring.contents[self.docstring.body_range]
}
}
impl Deref for DocstringBody<'_> {
type Target = str;
fn deref(&self) -> &Self::Target {
self.as_str()
}
}
impl Debug for DocstringBody<'_> {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
f.debug_struct("DocstringBody")
.field("text", &self.as_str())
.field("range", &self.range())
.finish()
}
}

View File

@@ -1,431 +0,0 @@
use std::fmt::{Debug, Formatter};
use std::iter::FusedIterator;
use ruff_python_ast::docstrings::{leading_space, leading_words};
use ruff_text_size::{TextLen, TextRange, TextSize};
use strum_macros::EnumIter;
use ruff_source_file::{Line, UniversalNewlineIterator, UniversalNewlines};
use crate::docstrings::styles::SectionStyle;
use crate::docstrings::{Docstring, DocstringBody};
#[derive(EnumIter, PartialEq, Eq, Debug, Clone, Copy)]
pub(crate) enum SectionKind {
Args,
Arguments,
Attention,
Attributes,
Caution,
Danger,
Error,
Example,
Examples,
ExtendedSummary,
Hint,
Important,
KeywordArgs,
KeywordArguments,
Methods,
Note,
Notes,
OtherArgs,
OtherArguments,
OtherParams,
OtherParameters,
Parameters,
Raises,
References,
Return,
Returns,
SeeAlso,
ShortSummary,
Tip,
Todo,
Warning,
Warnings,
Warns,
Yield,
Yields,
}
impl SectionKind {
pub(crate) fn from_str(s: &str) -> Option<Self> {
match s.to_ascii_lowercase().as_str() {
"args" => Some(Self::Args),
"arguments" => Some(Self::Arguments),
"attention" => Some(Self::Attention),
"attributes" => Some(Self::Attributes),
"caution" => Some(Self::Caution),
"danger" => Some(Self::Danger),
"error" => Some(Self::Error),
"example" => Some(Self::Example),
"examples" => Some(Self::Examples),
"extended summary" => Some(Self::ExtendedSummary),
"hint" => Some(Self::Hint),
"important" => Some(Self::Important),
"keyword args" => Some(Self::KeywordArgs),
"keyword arguments" => Some(Self::KeywordArguments),
"methods" => Some(Self::Methods),
"note" => Some(Self::Note),
"notes" => Some(Self::Notes),
"other args" => Some(Self::OtherArgs),
"other arguments" => Some(Self::OtherArguments),
"other params" => Some(Self::OtherParams),
"other parameters" => Some(Self::OtherParameters),
"parameters" => Some(Self::Parameters),
"raises" => Some(Self::Raises),
"references" => Some(Self::References),
"return" => Some(Self::Return),
"returns" => Some(Self::Returns),
"see also" => Some(Self::SeeAlso),
"short summary" => Some(Self::ShortSummary),
"tip" => Some(Self::Tip),
"todo" => Some(Self::Todo),
"warning" => Some(Self::Warning),
"warnings" => Some(Self::Warnings),
"warns" => Some(Self::Warns),
"yield" => Some(Self::Yield),
"yields" => Some(Self::Yields),
_ => None,
}
}
pub(crate) fn as_str(self) -> &'static str {
match self {
Self::Args => "Args",
Self::Arguments => "Arguments",
Self::Attention => "Attention",
Self::Attributes => "Attributes",
Self::Caution => "Caution",
Self::Danger => "Danger",
Self::Error => "Error",
Self::Example => "Example",
Self::Examples => "Examples",
Self::ExtendedSummary => "Extended Summary",
Self::Hint => "Hint",
Self::Important => "Important",
Self::KeywordArgs => "Keyword Args",
Self::KeywordArguments => "Keyword Arguments",
Self::Methods => "Methods",
Self::Note => "Note",
Self::Notes => "Notes",
Self::OtherArgs => "Other Args",
Self::OtherArguments => "Other Arguments",
Self::OtherParams => "Other Params",
Self::OtherParameters => "Other Parameters",
Self::Parameters => "Parameters",
Self::Raises => "Raises",
Self::References => "References",
Self::Return => "Return",
Self::Returns => "Returns",
Self::SeeAlso => "See Also",
Self::ShortSummary => "Short Summary",
Self::Tip => "Tip",
Self::Todo => "Todo",
Self::Warning => "Warning",
Self::Warnings => "Warnings",
Self::Warns => "Warns",
Self::Yield => "Yield",
Self::Yields => "Yields",
}
}
}
pub(crate) struct SectionContexts<'a> {
contexts: Vec<SectionContextData>,
docstring: &'a Docstring<'a>,
}
impl<'a> SectionContexts<'a> {
/// Extract all `SectionContext` values from a docstring.
pub(crate) fn from_docstring(docstring: &'a Docstring<'a>, style: SectionStyle) -> Self {
let contents = docstring.body();
let mut contexts = Vec::new();
let mut last: Option<SectionContextData> = None;
let mut lines = contents.universal_newlines().peekable();
// Skip the first line, which is the summary.
let mut previous_line = lines.next();
while let Some(line) = lines.next() {
if let Some(section_kind) = suspected_as_section(&line, style) {
let indent = leading_space(&line);
let section_name = leading_words(&line);
let section_name_range = TextRange::at(indent.text_len(), section_name.text_len());
if is_docstring_section(
&line,
section_name_range,
previous_line.as_ref(),
lines.peek(),
) {
if let Some(mut last) = last.take() {
last.range = TextRange::new(last.range.start(), line.start());
contexts.push(last);
}
last = Some(SectionContextData {
kind: section_kind,
name_range: section_name_range + line.start(),
range: TextRange::empty(line.start()),
summary_full_end: line.full_end(),
});
}
}
previous_line = Some(line);
}
if let Some(mut last) = last.take() {
last.range = TextRange::new(last.range.start(), contents.text_len());
contexts.push(last);
}
Self {
contexts,
docstring,
}
}
pub(crate) fn len(&self) -> usize {
self.contexts.len()
}
pub(crate) fn iter(&self) -> SectionContextsIter {
SectionContextsIter {
docstring_body: self.docstring.body(),
inner: self.contexts.iter(),
}
}
}
impl<'a> IntoIterator for &'a SectionContexts<'a> {
type IntoIter = SectionContextsIter<'a>;
type Item = SectionContext<'a>;
fn into_iter(self) -> Self::IntoIter {
self.iter()
}
}
impl Debug for SectionContexts<'_> {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
f.debug_list().entries(self.iter()).finish()
}
}
pub(crate) struct SectionContextsIter<'a> {
docstring_body: DocstringBody<'a>,
inner: std::slice::Iter<'a, SectionContextData>,
}
impl<'a> Iterator for SectionContextsIter<'a> {
type Item = SectionContext<'a>;
fn next(&mut self) -> Option<Self::Item> {
let next = self.inner.next()?;
Some(SectionContext {
data: next,
docstring_body: self.docstring_body,
})
}
fn size_hint(&self) -> (usize, Option<usize>) {
self.inner.size_hint()
}
}
impl<'a> DoubleEndedIterator for SectionContextsIter<'a> {
fn next_back(&mut self) -> Option<Self::Item> {
let back = self.inner.next_back()?;
Some(SectionContext {
data: back,
docstring_body: self.docstring_body,
})
}
}
impl FusedIterator for SectionContextsIter<'_> {}
impl ExactSizeIterator for SectionContextsIter<'_> {}
#[derive(Debug)]
struct SectionContextData {
kind: SectionKind,
/// Range of the section name, relative to the [`Docstring::body`]
name_range: TextRange,
/// Range from the start to the end of the section, relative to the [`Docstring::body`]
range: TextRange,
/// End of the summary, relative to the [`Docstring::body`]
summary_full_end: TextSize,
}
pub(crate) struct SectionContext<'a> {
data: &'a SectionContextData,
docstring_body: DocstringBody<'a>,
}
impl<'a> SectionContext<'a> {
/// The `kind` of the section, e.g. [`SectionKind::Args`] or [`SectionKind::Returns`].
pub(crate) const fn kind(&self) -> SectionKind {
self.data.kind
}
/// The name of the section as it appears in the docstring, e.g. "Args" or "Returns".
pub(crate) fn section_name(&self) -> &'a str {
&self.docstring_body.as_str()[self.data.name_range]
}
/// Returns the rest of the summary line after the section name.
pub(crate) fn summary_after_section_name(&self) -> &'a str {
&self.summary_line()[usize::from(self.data.name_range.end() - self.data.range.start())..]
}
fn offset(&self) -> TextSize {
self.docstring_body.start()
}
/// The absolute range of the section name
pub(crate) fn section_name_range(&self) -> TextRange {
self.data.name_range + self.offset()
}
/// The absolute range of the summary line, excluding any trailing newline character.
pub(crate) fn summary_range(&self) -> TextRange {
TextRange::at(self.range().start(), self.summary_line().text_len())
}
/// Range of the summary line relative to [`Docstring::body`], including the trailing newline character.
fn summary_full_range_relative(&self) -> TextRange {
TextRange::new(self.range_relative().start(), self.data.summary_full_end)
}
/// Returns the range of this section relative to [`Docstring::body`]
const fn range_relative(&self) -> TextRange {
self.data.range
}
/// The absolute range of the full-section.
pub(crate) fn range(&self) -> TextRange {
self.range_relative() + self.offset()
}
/// Summary line without the trailing newline characters
pub(crate) fn summary_line(&self) -> &'a str {
let full_summary = &self.docstring_body.as_str()[self.summary_full_range_relative()];
let mut bytes = full_summary.bytes().rev();
let newline_width = match bytes.next() {
Some(b'\n') => {
if bytes.next() == Some(b'\r') {
2
} else {
1
}
}
Some(b'\r') => 1,
_ => 0,
};
&full_summary[..full_summary.len() - newline_width]
}
/// Returns the text of the last line of the previous section or an empty string if it is the first section.
pub(crate) fn previous_line(&self) -> Option<&'a str> {
let previous =
&self.docstring_body.as_str()[TextRange::up_to(self.range_relative().start())];
previous.universal_newlines().last().map(|l| l.as_str())
}
/// Returns the lines belonging to this section after the summary line.
pub(crate) fn following_lines(&self) -> UniversalNewlineIterator<'a> {
let lines = self.following_lines_str();
UniversalNewlineIterator::with_offset(lines, self.offset() + self.data.summary_full_end)
}
fn following_lines_str(&self) -> &'a str {
&self.docstring_body.as_str()[self.following_range_relative()]
}
/// Returns the range to the following lines relative to [`Docstring::body`].
const fn following_range_relative(&self) -> TextRange {
TextRange::new(self.data.summary_full_end, self.range_relative().end())
}
/// Returns the absolute range of the following lines.
pub(crate) fn following_range(&self) -> TextRange {
self.following_range_relative() + self.offset()
}
}
impl Debug for SectionContext<'_> {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
f.debug_struct("SectionContext")
.field("kind", &self.kind())
.field("section_name", &self.section_name())
.field("summary_line", &self.summary_line())
.field("following_lines", &&self.following_lines_str())
.finish()
}
}
fn suspected_as_section(line: &str, style: SectionStyle) -> Option<SectionKind> {
if let Some(kind) = SectionKind::from_str(leading_words(line)) {
if style.sections().contains(&kind) {
return Some(kind);
}
}
None
}
/// Check if the suspected context is really a section header.
fn is_docstring_section(
line: &Line,
section_name_range: TextRange,
previous_line: Option<&Line>,
next_line: Option<&Line>,
) -> bool {
// Determine whether the current line looks like a section header, e.g., "Args:".
let section_name_suffix = line[usize::from(section_name_range.end())..].trim();
let this_looks_like_a_section_name =
section_name_suffix == ":" || section_name_suffix.is_empty();
if !this_looks_like_a_section_name {
return false;
}
// Determine whether the next line is an underline, e.g., "-----".
let next_line_is_underline = next_line.is_some_and(|next_line| {
let next_line = next_line.trim();
if next_line.is_empty() {
false
} else {
let next_line_is_underline = next_line.chars().all(|char| matches!(char, '-' | '='));
next_line_is_underline
}
});
if next_line_is_underline {
return true;
}
// Determine whether the previous line looks like the end of a paragraph.
let previous_line_looks_like_end_of_paragraph = previous_line.map_or(true, |previous_line| {
let previous_line = previous_line.trim();
let previous_line_ends_with_punctuation = [',', ';', '.', '-', '\\', '/', ']', '}', ')']
.into_iter()
.any(|char| previous_line.ends_with(char));
previous_line_ends_with_punctuation || previous_line.is_empty()
});
if !previous_line_looks_like_end_of_paragraph {
return false;
}
true
}

View File

@@ -1,13 +0,0 @@
//! Extract Black configuration settings from a pyproject.toml.
use serde::{Deserialize, Serialize};
use crate::settings::types::PythonVersion;
#[derive(Debug, PartialEq, Eq, Serialize, Deserialize, Default)]
pub struct Black {
#[serde(alias = "line-length", alias = "line_length")]
pub line_length: Option<usize>,
#[serde(alias = "target-version", alias = "target_version")]
pub target_version: Option<Vec<PythonVersion>>,
}

View File

@@ -1,10 +0,0 @@
use super::black::Black;
use super::isort::Isort;
use super::pep621::Project;
#[derive(Default)]
pub struct ExternalConfig<'a> {
pub black: Option<&'a Black>,
pub isort: Option<&'a Isort>,
pub project: Option<&'a Project>,
}

View File

@@ -1,13 +0,0 @@
pub use converter::convert;
pub use external_config::ExternalConfig;
pub use plugin::Plugin;
pub use pyproject::parse;
mod black;
mod converter;
mod external_config;
mod isort;
mod parser;
pub mod pep621;
mod plugin;
mod pyproject;

View File

@@ -1,26 +0,0 @@
use std::path::Path;
use anyhow::Result;
use serde::{Deserialize, Serialize};
use super::black::Black;
use super::isort::Isort;
use super::pep621::Project;
#[derive(Debug, PartialEq, Eq, Serialize, Deserialize)]
pub struct Tools {
pub black: Option<Black>,
pub isort: Option<Isort>,
}
#[derive(Debug, PartialEq, Eq, Serialize, Deserialize)]
pub struct Pyproject {
pub tool: Option<Tools>,
pub project: Option<Project>,
}
pub fn parse<P: AsRef<Path>>(path: P) -> Result<Pyproject> {
let contents = std::fs::read_to_string(path)?;
let pyproject = toml::from_str::<Pyproject>(&contents)?;
Ok(pyproject)
}

View File

@@ -1,496 +0,0 @@
//! Code modification struct to add and modify import statements.
//!
//! Enables rules to make module members available (that may be not yet be imported) during fix
//! execution.
use std::error::Error;
use anyhow::Result;
use libcst_native::{ImportAlias, Name, NameOrAttribute};
use ruff_python_ast::{self as ast, PySourceType, Ranged, Stmt, Suite};
use ruff_text_size::TextSize;
use ruff_diagnostics::Edit;
use ruff_python_ast::imports::{AnyImport, Import, ImportFrom};
use ruff_python_codegen::Stylist;
use ruff_python_semantic::SemanticModel;
use ruff_python_trivia::textwrap::indent;
use ruff_source_file::Locator;
use crate::autofix;
use crate::autofix::codemods::CodegenStylist;
use crate::cst::matchers::{match_aliases, match_import_from, match_statement};
use crate::importer::insertion::Insertion;
mod insertion;
pub(crate) struct Importer<'a> {
/// The Python AST to which we are adding imports.
python_ast: &'a Suite,
/// The [`Locator`] for the Python AST.
locator: &'a Locator<'a>,
/// The [`Stylist`] for the Python AST.
stylist: &'a Stylist<'a>,
/// The list of visited, top-level runtime imports in the Python AST.
runtime_imports: Vec<&'a Stmt>,
/// The list of visited, top-level `if TYPE_CHECKING:` blocks in the Python AST.
type_checking_blocks: Vec<&'a Stmt>,
}
impl<'a> Importer<'a> {
pub(crate) fn new(
python_ast: &'a Suite,
locator: &'a Locator<'a>,
stylist: &'a Stylist<'a>,
) -> Self {
Self {
python_ast,
locator,
stylist,
runtime_imports: Vec::default(),
type_checking_blocks: Vec::default(),
}
}
/// Visit a top-level import statement.
pub(crate) fn visit_import(&mut self, import: &'a Stmt) {
self.runtime_imports.push(import);
}
/// Visit a top-level type-checking block.
pub(crate) fn visit_type_checking_block(&mut self, type_checking_block: &'a Stmt) {
self.type_checking_blocks.push(type_checking_block);
}
/// Add an import statement to import the given module.
///
/// If there are no existing imports, the new import will be added at the top
/// of the file. Otherwise, it will be added after the most recent top-level
/// import statement.
pub(crate) fn add_import(&self, import: &AnyImport, at: TextSize) -> Edit {
let required_import = import.to_string();
if let Some(stmt) = self.preceding_import(at) {
// Insert after the last top-level import.
Insertion::end_of_statement(stmt, self.locator, self.stylist)
.into_edit(&required_import)
} else {
// Insert at the start of the file.
Insertion::start_of_file(self.python_ast, self.locator, self.stylist)
.into_edit(&required_import)
}
}
/// Move an existing import to the top-level, thereby making it available at runtime.
///
/// If there are no existing imports, the new import will be added at the top
/// of the file. Otherwise, it will be added after the most recent top-level
/// import statement.
pub(crate) fn runtime_import_edit(
&self,
import: &ImportedMembers,
at: TextSize,
) -> Result<RuntimeImportEdit> {
// Generate the modified import statement.
let content = autofix::codemods::retain_imports(
&import.names,
import.statement,
self.locator,
self.stylist,
)?;
// Add the import to the top-level.
let insertion = if let Some(stmt) = self.preceding_import(at) {
// Insert after the last top-level import.
Insertion::end_of_statement(stmt, self.locator, self.stylist)
} else {
// Insert at the start of the file.
Insertion::start_of_file(self.python_ast, self.locator, self.stylist)
};
let add_import_edit = insertion.into_edit(&content);
Ok(RuntimeImportEdit { add_import_edit })
}
/// Move an existing import into a `TYPE_CHECKING` block.
///
/// If there are no existing `TYPE_CHECKING` blocks, a new one will be added at the top
/// of the file. Otherwise, it will be added after the most recent top-level
/// `TYPE_CHECKING` block.
pub(crate) fn typing_import_edit(
&self,
import: &ImportedMembers,
at: TextSize,
semantic: &SemanticModel,
source_type: PySourceType,
) -> Result<TypingImportEdit> {
// Generate the modified import statement.
let content = autofix::codemods::retain_imports(
&import.names,
import.statement,
self.locator,
self.stylist,
)?;
// Import the `TYPE_CHECKING` symbol from the typing module.
let (type_checking_edit, type_checking) = self.get_or_import_symbol(
&ImportRequest::import_from("typing", "TYPE_CHECKING"),
at,
semantic,
)?;
// Add the import to a `TYPE_CHECKING` block.
let add_import_edit = if let Some(block) = self.preceding_type_checking_block(at) {
// Add the import to the `TYPE_CHECKING` block.
self.add_to_type_checking_block(&content, block.start(), source_type)
} else {
// Add the import to a new `TYPE_CHECKING` block.
self.add_type_checking_block(
&format!(
"{}if {type_checking}:{}{}",
self.stylist.line_ending().as_str(),
self.stylist.line_ending().as_str(),
indent(&content, self.stylist.indentation())
),
at,
)?
};
Ok(TypingImportEdit {
type_checking_edit,
add_import_edit,
})
}
/// Generate an [`Edit`] to reference the given symbol. Returns the [`Edit`] necessary to make
/// the symbol available in the current scope along with the bound name of the symbol.
///
/// Attempts to reuse existing imports when possible.
pub(crate) fn get_or_import_symbol(
&self,
symbol: &ImportRequest,
at: TextSize,
semantic: &SemanticModel,
) -> Result<(Edit, String), ResolutionError> {
match self.get_symbol(symbol, at, semantic) {
Some(result) => result,
None => self.import_symbol(symbol, at, semantic),
}
}
/// Return an [`Edit`] to reference an existing symbol, if it's present in the given [`SemanticModel`].
fn get_symbol(
&self,
symbol: &ImportRequest,
at: TextSize,
semantic: &SemanticModel,
) -> Option<Result<(Edit, String), ResolutionError>> {
// If the symbol is already available in the current scope, use it.
let imported_name = semantic.resolve_qualified_import_name(symbol.module, symbol.member)?;
// If the symbol source (i.e., the import statement) comes after the current location,
// abort. For example, we could be generating an edit within a function, and the import
// could be defined in the module scope, but after the function definition. In this case,
// it's unclear whether we can use the symbol (the function could be called between the
// import and the current location, and thus the symbol would not be available). It's also
// unclear whether should add an import statement at the start of the file, since it could
// be shadowed between the import and the current location.
if imported_name.range().start() > at {
return Some(Err(ResolutionError::ImportAfterUsage));
}
// If the symbol source (i.e., the import statement) is in a typing-only context, but we're
// in a runtime context, abort.
if imported_name.context().is_typing() && semantic.execution_context().is_runtime() {
return Some(Err(ResolutionError::IncompatibleContext));
}
// We also add a no-op edit to force conflicts with any other fixes that might try to
// remove the import. Consider:
//
// ```py
// import sys
//
// quit()
// ```
//
// Assume you omit this no-op edit. If you run Ruff with `unused-imports` and
// `sys-exit-alias` over this snippet, it will generate two fixes: (1) remove the unused
// `sys` import; and (2) replace `quit()` with `sys.exit()`, under the assumption that `sys`
// is already imported and available.
//
// By adding this no-op edit, we force the `unused-imports` fix to conflict with the
// `sys-exit-alias` fix, and thus will avoid applying both fixes in the same pass.
let import_edit = Edit::range_replacement(
self.locator.slice(imported_name.range()).to_string(),
imported_name.range(),
);
Some(Ok((import_edit, imported_name.into_name())))
}
/// Generate an [`Edit`] to reference the given symbol. Returns the [`Edit`] necessary to make
/// the symbol available in the current scope along with the bound name of the symbol.
///
/// For example, assuming `module` is `"functools"` and `member` is `"lru_cache"`, this function
/// could return an [`Edit`] to add `import functools` to the start of the file, alongside with
/// the name on which the `lru_cache` symbol would be made available (`"functools.lru_cache"`).
fn import_symbol(
&self,
symbol: &ImportRequest,
at: TextSize,
semantic: &SemanticModel,
) -> Result<(Edit, String), ResolutionError> {
if let Some(stmt) = self.find_import_from(symbol.module, at) {
// Case 1: `from functools import lru_cache` is in scope, and we're trying to reference
// `functools.cache`; thus, we add `cache` to the import, and return `"cache"` as the
// bound name.
if semantic.is_available(symbol.member) {
let Ok(import_edit) = self.add_member(stmt, symbol.member) else {
return Err(ResolutionError::InvalidEdit);
};
Ok((import_edit, symbol.member.to_string()))
} else {
Err(ResolutionError::ConflictingName(symbol.member.to_string()))
}
} else {
match symbol.style {
ImportStyle::Import => {
// Case 2a: No `functools` import is in scope; thus, we add `import functools`,
// and return `"functools.cache"` as the bound name.
if semantic.is_available(symbol.module) {
let import_edit =
self.add_import(&AnyImport::Import(Import::module(symbol.module)), at);
Ok((
import_edit,
format!(
"{module}.{member}",
module = symbol.module,
member = symbol.member
),
))
} else {
Err(ResolutionError::ConflictingName(symbol.module.to_string()))
}
}
ImportStyle::ImportFrom => {
// Case 2b: No `functools` import is in scope; thus, we add
// `from functools import cache`, and return `"cache"` as the bound name.
if semantic.is_available(symbol.member) {
let import_edit = self.add_import(
&AnyImport::ImportFrom(ImportFrom::member(
symbol.module,
symbol.member,
)),
at,
);
Ok((import_edit, symbol.member.to_string()))
} else {
Err(ResolutionError::ConflictingName(symbol.member.to_string()))
}
}
}
}
}
/// Return the top-level [`Stmt`] that imports the given module using `Stmt::ImportFrom`
/// preceding the given position, if any.
fn find_import_from(&self, module: &str, at: TextSize) -> Option<&Stmt> {
let mut import_from = None;
for stmt in &self.runtime_imports {
if stmt.start() >= at {
break;
}
if let Stmt::ImportFrom(ast::StmtImportFrom {
module: name,
level,
..
}) = stmt
{
if level.map_or(true, |level| level.to_u32() == 0)
&& name.as_ref().is_some_and(|name| name == module)
{
import_from = Some(*stmt);
}
}
}
import_from
}
/// Add the given member to an existing `Stmt::ImportFrom` statement.
fn add_member(&self, stmt: &Stmt, member: &str) -> Result<Edit> {
let mut statement = match_statement(self.locator.slice(stmt.range()))?;
let import_from = match_import_from(&mut statement)?;
let aliases = match_aliases(import_from)?;
aliases.push(ImportAlias {
name: NameOrAttribute::N(Box::new(Name {
value: member,
lpar: vec![],
rpar: vec![],
})),
asname: None,
comma: aliases.last().and_then(|alias| alias.comma.clone()),
});
Ok(Edit::range_replacement(
statement.codegen_stylist(self.stylist),
stmt.range(),
))
}
/// Add a `TYPE_CHECKING` block to the given module.
fn add_type_checking_block(&self, content: &str, at: TextSize) -> Result<Edit> {
let insertion = if let Some(stmt) = self.preceding_import(at) {
// Insert after the last top-level import.
Insertion::end_of_statement(stmt, self.locator, self.stylist)
} else {
// Insert at the start of the file.
Insertion::start_of_file(self.python_ast, self.locator, self.stylist)
};
if insertion.is_inline() {
Err(anyhow::anyhow!(
"Cannot insert `TYPE_CHECKING` block inline"
))
} else {
Ok(insertion.into_edit(content))
}
}
/// Add an import statement to an existing `TYPE_CHECKING` block.
fn add_to_type_checking_block(
&self,
content: &str,
at: TextSize,
source_type: PySourceType,
) -> Edit {
Insertion::start_of_block(at, self.locator, self.stylist, source_type).into_edit(content)
}
/// Return the import statement that precedes the given position, if any.
fn preceding_import(&self, at: TextSize) -> Option<&'a Stmt> {
self.runtime_imports
.partition_point(|stmt| stmt.start() < at)
.checked_sub(1)
.map(|idx| self.runtime_imports[idx])
}
/// Return the `TYPE_CHECKING` block that precedes the given position, if any.
fn preceding_type_checking_block(&self, at: TextSize) -> Option<&'a Stmt> {
let block = self.type_checking_blocks.first()?;
if block.start() <= at {
Some(block)
} else {
None
}
}
}
/// An edit to the top-level of a module, making it available at runtime.
#[derive(Debug)]
pub(crate) struct RuntimeImportEdit {
/// The edit to add the import to the top-level of the module.
add_import_edit: Edit,
}
impl RuntimeImportEdit {
pub(crate) fn into_edits(self) -> Vec<Edit> {
vec![self.add_import_edit]
}
}
/// An edit to an import to a typing-only context.
#[derive(Debug)]
pub(crate) struct TypingImportEdit {
/// The edit to add the `TYPE_CHECKING` symbol to the module.
type_checking_edit: Edit,
/// The edit to add the import to a `TYPE_CHECKING` block.
add_import_edit: Edit,
}
impl TypingImportEdit {
pub(crate) fn into_edits(self) -> Vec<Edit> {
vec![self.type_checking_edit, self.add_import_edit]
}
}
#[derive(Debug)]
enum ImportStyle {
/// Import the symbol using the `import` statement (e.g. `import foo; foo.bar`).
Import,
/// Import the symbol using the `from` statement (e.g. `from foo import bar; bar`).
ImportFrom,
}
#[derive(Debug)]
pub(crate) struct ImportRequest<'a> {
/// The module from which the symbol can be imported (e.g., `foo`, in `from foo import bar`).
module: &'a str,
/// The member to import (e.g., `bar`, in `from foo import bar`).
member: &'a str,
/// The preferred style to use when importing the symbol (e.g., `import foo` or
/// `from foo import bar`), if it's not already in scope.
style: ImportStyle,
}
impl<'a> ImportRequest<'a> {
/// Create a new `ImportRequest` from a module and member. If not present in the scope,
/// the symbol should be imported using the "import" statement.
pub(crate) fn import(module: &'a str, member: &'a str) -> Self {
Self {
module,
member,
style: ImportStyle::Import,
}
}
/// Create a new `ImportRequest` from a module and member. If not present in the scope,
/// the symbol should be imported using the "import from" statement.
pub(crate) fn import_from(module: &'a str, member: &'a str) -> Self {
Self {
module,
member,
style: ImportStyle::ImportFrom,
}
}
}
/// An existing list of module or member imports, located within an import statement.
pub(crate) struct ImportedMembers<'a> {
/// The import statement.
pub(crate) statement: &'a Stmt,
/// The "names" of the imported members.
pub(crate) names: Vec<&'a str>,
}
/// The result of an [`Importer::get_or_import_symbol`] call.
#[derive(Debug)]
pub(crate) enum ResolutionError {
/// The symbol is imported, but the import came after the current location.
ImportAfterUsage,
/// The symbol is imported, but in an incompatible context (e.g., in typing-only context, while
/// we're in a runtime context).
IncompatibleContext,
/// The symbol can't be imported, because another symbol is bound to the same name.
ConflictingName(String),
/// The symbol can't be imported due to an error in editing an existing import statement.
InvalidEdit,
}
impl std::fmt::Display for ResolutionError {
fn fmt(&self, fmt: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
ResolutionError::ImportAfterUsage => {
fmt.write_str("Unable to use existing symbol due to late binding")
}
ResolutionError::IncompatibleContext => {
fmt.write_str("Unable to use existing symbol due to incompatible context")
}
ResolutionError::ConflictingName(binding) => std::write!(
fmt,
"Unable to insert `{binding}` into scope due to name conflict"
),
ResolutionError::InvalidEdit => {
fmt.write_str("Unable to modify existing import statement")
}
}
}
}
impl Error for ResolutionError {}

View File

@@ -1,644 +0,0 @@
use std::cmp::Ordering;
use std::fmt::Display;
use std::fs::File;
use std::io::{BufReader, BufWriter, Cursor, Read, Seek, SeekFrom, Write};
use std::iter;
use std::path::Path;
use itertools::Itertools;
use once_cell::sync::OnceCell;
use serde::Serialize;
use serde_json::error::Category;
use ruff_diagnostics::Diagnostic;
use ruff_python_parser::lexer::lex;
use ruff_python_parser::Mode;
use ruff_source_file::{NewlineWithTrailingNewline, UniversalNewlineIterator};
use ruff_text_size::{TextRange, TextSize};
use crate::autofix::source_map::{SourceMap, SourceMarker};
use crate::jupyter::index::JupyterIndex;
use crate::jupyter::schema::{Cell, RawNotebook, SortAlphabetically, SourceValue};
use crate::rules::pycodestyle::rules::SyntaxError;
use crate::IOError;
pub const JUPYTER_NOTEBOOK_EXT: &str = "ipynb";
/// Run round-trip source code generation on a given Jupyter notebook file path.
pub fn round_trip(path: &Path) -> anyhow::Result<String> {
let mut notebook = Notebook::from_path(path).map_err(|err| {
anyhow::anyhow!(
"Failed to read notebook file `{}`: {:?}",
path.display(),
err
)
})?;
let code = notebook.content().to_string();
notebook.update_cell_content(&code);
let mut writer = Vec::new();
notebook.write_inner(&mut writer)?;
Ok(String::from_utf8(writer)?)
}
impl Display for SourceValue {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
SourceValue::String(string) => f.write_str(string),
SourceValue::StringArray(string_array) => {
for string in string_array {
f.write_str(string)?;
}
Ok(())
}
}
}
}
impl Cell {
/// Return the [`SourceValue`] of the cell.
fn source(&self) -> &SourceValue {
match self {
Cell::Code(cell) => &cell.source,
Cell::Markdown(cell) => &cell.source,
Cell::Raw(cell) => &cell.source,
}
}
/// Update the [`SourceValue`] of the cell.
fn set_source(&mut self, source: SourceValue) {
match self {
Cell::Code(cell) => cell.source = source,
Cell::Markdown(cell) => cell.source = source,
Cell::Raw(cell) => cell.source = source,
}
}
/// Return `true` if it's a valid code cell.
///
/// A valid code cell is a cell where the cell type is [`Cell::Code`] and the
/// source doesn't contain a cell magic.
fn is_valid_code_cell(&self) -> bool {
let source = match self {
Cell::Code(cell) => &cell.source,
_ => return false,
};
// Ignore cells containing cell magic. This is different from line magic
// which is allowed and ignored by the parser.
!match source {
SourceValue::String(string) => string
.lines()
.any(|line| line.trim_start().starts_with("%%")),
SourceValue::StringArray(string_array) => string_array
.iter()
.any(|line| line.trim_start().starts_with("%%")),
}
}
}
#[derive(Clone, Debug, PartialEq)]
pub struct Notebook {
/// Python source code of the notebook.
///
/// This is the concatenation of all valid code cells in the notebook
/// separated by a newline and a trailing newline. The trailing newline
/// is added to make sure that each cell ends with a newline which will
/// be removed when updating the cell content.
content: String,
/// The index of the notebook. This is used to map between the concatenated
/// source code and the original notebook.
index: OnceCell<JupyterIndex>,
/// The raw notebook i.e., the deserialized version of JSON string.
raw: RawNotebook,
/// The offsets of each cell in the concatenated source code. This includes
/// the first and last character offsets as well.
cell_offsets: Vec<TextSize>,
/// The cell index of all valid code cells in the notebook.
valid_code_cells: Vec<u32>,
/// Flag to indicate if the JSON string of the notebook has a trailing newline.
trailing_newline: bool,
}
impl Notebook {
/// Read the Jupyter Notebook from the given [`Path`].
pub fn from_path(path: &Path) -> Result<Self, Box<Diagnostic>> {
Self::from_reader(BufReader::new(File::open(path).map_err(|err| {
Diagnostic::new(
IOError {
message: format!("{err}"),
},
TextRange::default(),
)
})?))
}
/// Read the Jupyter Notebook from its JSON string.
pub fn from_contents(contents: &str) -> Result<Self, Box<Diagnostic>> {
Self::from_reader(Cursor::new(contents))
}
/// Read a Jupyter Notebook from a [`Read`] implementor.
///
/// See also the black implementation
/// <https://github.com/psf/black/blob/69ca0a4c7a365c5f5eea519a90980bab72cab764/src/black/__init__.py#L1017-L1046>
fn from_reader<R>(mut reader: R) -> Result<Self, Box<Diagnostic>>
where
R: Read + Seek,
{
let trailing_newline = reader.seek(SeekFrom::End(-1)).is_ok_and(|_| {
let mut buf = [0; 1];
reader.read_exact(&mut buf).is_ok_and(|_| buf[0] == b'\n')
});
reader.rewind().map_err(|err| {
Diagnostic::new(
IOError {
message: format!("{err}"),
},
TextRange::default(),
)
})?;
let raw_notebook: RawNotebook = match serde_json::from_reader(reader.by_ref()) {
Ok(notebook) => notebook,
Err(err) => {
// Translate the error into a diagnostic
return Err(Box::new({
match err.classify() {
Category::Io => Diagnostic::new(
IOError {
message: format!("{err}"),
},
TextRange::default(),
),
Category::Syntax | Category::Eof => {
// Maybe someone saved the python sources (those with the `# %%` separator)
// as jupyter notebook instead. Let's help them.
let mut contents = String::new();
reader
.rewind()
.and_then(|_| reader.read_to_string(&mut contents))
.map_err(|err| {
Diagnostic::new(
IOError {
message: format!("{err}"),
},
TextRange::default(),
)
})?;
// Check if tokenizing was successful and the file is non-empty
if lex(&contents, Mode::Module).any(|result| result.is_err()) {
Diagnostic::new(
SyntaxError {
message: format!(
"A Jupyter Notebook (.{JUPYTER_NOTEBOOK_EXT}) must internally be JSON, \
but this file isn't valid JSON: {err}"
),
},
TextRange::default(),
)
} else {
Diagnostic::new(
SyntaxError {
message: format!(
"Expected a Jupyter Notebook (.{JUPYTER_NOTEBOOK_EXT}), \
which must be internally stored as JSON, \
but found a Python source file: {err}"
),
},
TextRange::default(),
)
}
}
Category::Data => {
// We could try to read the schema version here but if this fails it's
// a bug anyway
Diagnostic::new(
SyntaxError {
message: format!(
"This file does not match the schema expected of Jupyter Notebooks: {err}"
),
},
TextRange::default(),
)
}
}
}));
}
};
// v4 is what everybody uses
if raw_notebook.nbformat != 4 {
// bail because we should have already failed at the json schema stage
return Err(Box::new(Diagnostic::new(
SyntaxError {
message: format!(
"Expected Jupyter Notebook format 4, found {}",
raw_notebook.nbformat
),
},
TextRange::default(),
)));
}
let valid_code_cells = raw_notebook
.cells
.iter()
.enumerate()
.filter(|(_, cell)| cell.is_valid_code_cell())
.map(|(idx, _)| u32::try_from(idx).unwrap())
.collect::<Vec<_>>();
let mut contents = Vec::with_capacity(valid_code_cells.len());
let mut current_offset = TextSize::from(0);
let mut cell_offsets = Vec::with_capacity(valid_code_cells.len());
cell_offsets.push(TextSize::from(0));
for &idx in &valid_code_cells {
let cell_contents = match &raw_notebook.cells[idx as usize].source() {
SourceValue::String(string) => string.clone(),
SourceValue::StringArray(string_array) => string_array.join(""),
};
current_offset += TextSize::of(&cell_contents) + TextSize::new(1);
contents.push(cell_contents);
cell_offsets.push(current_offset);
}
Ok(Self {
raw: raw_notebook,
index: OnceCell::new(),
// The additional newline at the end is to maintain consistency for
// all cells. These newlines will be removed before updating the
// source code with the transformed content. Refer `update_cell_content`.
content: contents.join("\n") + "\n",
cell_offsets,
valid_code_cells,
trailing_newline,
})
}
/// Update the cell offsets as per the given [`SourceMap`].
fn update_cell_offsets(&mut self, source_map: &SourceMap) {
// When there are multiple cells without any edits, the offsets of those
// cells will be updated using the same marker. So, we can keep track of
// the last marker used to update the offsets and check if it's still
// the closest marker to the current offset.
let mut last_marker: Option<&SourceMarker> = None;
// The first offset is always going to be at 0, so skip it.
for offset in self.cell_offsets.iter_mut().skip(1).rev() {
let closest_marker = match last_marker {
Some(marker) if marker.source <= *offset => marker,
_ => {
let Some(marker) = source_map
.markers()
.iter()
.rev()
.find(|m| m.source <= *offset)
else {
// There are no markers above the current offset, so we can
// stop here.
break;
};
last_marker = Some(marker);
marker
}
};
match closest_marker.source.cmp(&closest_marker.dest) {
Ordering::Less => *offset += closest_marker.dest - closest_marker.source,
Ordering::Greater => *offset -= closest_marker.source - closest_marker.dest,
Ordering::Equal => (),
}
}
}
/// Update the cell contents with the transformed content.
///
/// ## Panics
///
/// Panics if the transformed content is out of bounds for any cell. This
/// can happen only if the cell offsets were not updated before calling
/// this method or the offsets were updated incorrectly.
fn update_cell_content(&mut self, transformed: &str) {
for (&idx, (start, end)) in self
.valid_code_cells
.iter()
.zip(self.cell_offsets.iter().tuple_windows::<(_, _)>())
{
let cell_content = transformed
.get(start.to_usize()..end.to_usize())
.unwrap_or_else(|| {
panic!(
"Transformed content out of bounds ({start:?}..{end:?}) for cell at {idx:?}"
);
});
self.raw.cells[idx as usize].set_source(SourceValue::StringArray(
UniversalNewlineIterator::from(
// We only need to strip the trailing newline which we added
// while concatenating the cell contents.
cell_content.strip_suffix('\n').unwrap_or(cell_content),
)
.map(|line| line.as_full_str().to_string())
.collect::<Vec<_>>(),
));
}
}
/// Build and return the [`JupyterIndex`].
///
/// ## Notes
///
/// Empty cells don't have any newlines, but there's a single visible line
/// in the UI. That single line needs to be accounted for.
///
/// In case of [`SourceValue::StringArray`], newlines are part of the strings.
/// So, to get the actual count of lines, we need to check for any trailing
/// newline for the last line.
///
/// For example, consider the following cell:
/// ```python
/// [
/// "import os\n",
/// "import sys\n",
/// ]
/// ```
///
/// Here, the array suggests that there are two lines, but the actual number
/// of lines visible in the UI is three. The same goes for [`SourceValue::String`]
/// where we need to check for the trailing newline.
///
/// The index building is expensive as it needs to go through the content of
/// every valid code cell.
fn build_index(&self) -> JupyterIndex {
let mut row_to_cell = vec![0];
let mut row_to_row_in_cell = vec![0];
for &idx in &self.valid_code_cells {
let line_count = match &self.raw.cells[idx as usize].source() {
SourceValue::String(string) => {
if string.is_empty() {
1
} else {
u32::try_from(NewlineWithTrailingNewline::from(string).count()).unwrap()
}
}
SourceValue::StringArray(string_array) => {
if string_array.is_empty() {
1
} else {
let trailing_newline =
usize::from(string_array.last().is_some_and(|s| s.ends_with('\n')));
u32::try_from(string_array.len() + trailing_newline).unwrap()
}
}
};
row_to_cell.extend(iter::repeat(idx + 1).take(line_count as usize));
row_to_row_in_cell.extend(1..=line_count);
}
JupyterIndex {
row_to_cell,
row_to_row_in_cell,
}
}
/// Return the notebook content.
///
/// This is the concatenation of all Python code cells.
pub(crate) fn content(&self) -> &str {
&self.content
}
/// Return the Jupyter notebook index.
///
/// The index is built only once when required. This is only used to
/// report diagnostics, so by that time all of the autofixes must have
/// been applied if `--fix` was passed.
pub(crate) fn index(&self) -> &JupyterIndex {
self.index.get_or_init(|| self.build_index())
}
/// Return the cell offsets for the concatenated source code corresponding
/// the Jupyter notebook.
pub(crate) fn cell_offsets(&self) -> &[TextSize] {
&self.cell_offsets
}
/// Update the notebook with the given sourcemap and transformed content.
pub(crate) fn update(&mut self, source_map: &SourceMap, transformed: &str) {
// Cell offsets must be updated before updating the cell content as
// it depends on the offsets to extract the cell content.
self.update_cell_offsets(source_map);
self.update_cell_content(transformed);
self.content = transformed.to_string();
}
/// Return a slice of [`Cell`] in the Jupyter notebook.
pub fn cells(&self) -> &[Cell] {
&self.raw.cells
}
/// Return `true` if the notebook is a Python notebook, `false` otherwise.
pub fn is_python_notebook(&self) -> bool {
self.raw
.metadata
.language_info
.as_ref()
.map_or(true, |language| language.name == "python")
}
fn write_inner(&self, writer: &mut impl Write) -> anyhow::Result<()> {
// https://github.com/psf/black/blob/69ca0a4c7a365c5f5eea519a90980bab72cab764/src/black/__init__.py#LL1041
let formatter = serde_json::ser::PrettyFormatter::with_indent(b" ");
let mut serializer = serde_json::Serializer::with_formatter(writer, formatter);
SortAlphabetically(&self.raw).serialize(&mut serializer)?;
if self.trailing_newline {
writeln!(serializer.into_inner())?;
}
Ok(())
}
/// Write back with an indent of 1, just like black
pub fn write(&self, path: &Path) -> anyhow::Result<()> {
let mut writer = BufWriter::new(File::create(path)?);
self.write_inner(&mut writer)?;
Ok(())
}
}
#[cfg(test)]
mod tests {
use std::path::Path;
use anyhow::Result;
use test_case::test_case;
use crate::jupyter::index::JupyterIndex;
use crate::jupyter::schema::Cell;
use crate::jupyter::Notebook;
use crate::registry::Rule;
use crate::test::{read_jupyter_notebook, test_notebook_path, test_resource_path};
use crate::{assert_messages, settings};
/// Read a Jupyter cell from the `resources/test/fixtures/jupyter/cell` directory.
fn read_jupyter_cell(path: impl AsRef<Path>) -> Result<Cell> {
let path = test_resource_path("fixtures/jupyter/cell").join(path);
let contents = std::fs::read_to_string(path)?;
Ok(serde_json::from_str(&contents)?)
}
#[test]
fn test_valid() {
assert!(read_jupyter_notebook(Path::new("valid.ipynb")).is_ok());
}
#[test]
fn test_r() {
// We can load this, it will be filtered out later
assert!(read_jupyter_notebook(Path::new("R.ipynb")).is_ok());
}
#[test]
fn test_invalid() {
let path = Path::new("resources/test/fixtures/jupyter/invalid_extension.ipynb");
assert_eq!(
Notebook::from_path(path).unwrap_err().kind.body,
"SyntaxError: Expected a Jupyter Notebook (.ipynb), \
which must be internally stored as JSON, \
but found a Python source file: \
expected value at line 1 column 1"
);
let path = Path::new("resources/test/fixtures/jupyter/not_json.ipynb");
assert_eq!(
Notebook::from_path(path).unwrap_err().kind.body,
"SyntaxError: A Jupyter Notebook (.ipynb) must internally be JSON, \
but this file isn't valid JSON: \
expected value at line 1 column 1"
);
let path = Path::new("resources/test/fixtures/jupyter/wrong_schema.ipynb");
assert_eq!(
Notebook::from_path(path).unwrap_err().kind.body,
"SyntaxError: This file does not match the schema expected of Jupyter Notebooks: \
missing field `cells` at line 1 column 2"
);
}
#[test_case(Path::new("markdown.json"), false; "markdown")]
#[test_case(Path::new("only_magic.json"), true; "only_magic")]
#[test_case(Path::new("code_and_magic.json"), true; "code_and_magic")]
#[test_case(Path::new("only_code.json"), true; "only_code")]
#[test_case(Path::new("cell_magic.json"), false; "cell_magic")]
fn test_is_valid_code_cell(path: &Path, expected: bool) -> Result<()> {
assert_eq!(read_jupyter_cell(path)?.is_valid_code_cell(), expected);
Ok(())
}
#[test]
fn test_concat_notebook() -> Result<()> {
let notebook = read_jupyter_notebook(Path::new("valid.ipynb"))?;
assert_eq!(
notebook.content,
r#"def unused_variable():
x = 1
y = 2
print(f"cell one: {y}")
unused_variable()
def mutable_argument(z=set()):
print(f"cell two: {z}")
mutable_argument()
print("after empty cells")
"#
);
assert_eq!(
notebook.index(),
&JupyterIndex {
row_to_cell: vec![0, 1, 1, 1, 1, 1, 1, 3, 3, 3, 3, 3, 5, 7, 7, 8],
row_to_row_in_cell: vec![0, 1, 2, 3, 4, 5, 6, 1, 2, 3, 4, 5, 1, 1, 2, 1],
}
);
assert_eq!(
notebook.cell_offsets(),
&[
0.into(),
90.into(),
168.into(),
169.into(),
171.into(),
198.into()
]
);
Ok(())
}
#[test]
fn test_import_sorting() -> Result<()> {
let path = "isort.ipynb".to_string();
let (diagnostics, source_kind, _) = test_notebook_path(
&path,
Path::new("isort_expected.ipynb"),
&settings::Settings::for_rule(Rule::UnsortedImports),
)?;
assert_messages!(diagnostics, path, source_kind);
Ok(())
}
#[test]
fn test_ipy_escape_command() -> Result<()> {
let path = "ipy_escape_command.ipynb".to_string();
let (diagnostics, source_kind, _) = test_notebook_path(
&path,
Path::new("ipy_escape_command_expected.ipynb"),
&settings::Settings::for_rule(Rule::UnusedImport),
)?;
assert_messages!(diagnostics, path, source_kind);
Ok(())
}
#[test]
fn test_unused_variable() -> Result<()> {
let path = "unused_variable.ipynb".to_string();
let (diagnostics, source_kind, _) = test_notebook_path(
&path,
Path::new("unused_variable_expected.ipynb"),
&settings::Settings::for_rule(Rule::UnusedVariable),
)?;
assert_messages!(diagnostics, path, source_kind);
Ok(())
}
#[test]
fn test_json_consistency() -> Result<()> {
let path = "before_fix.ipynb".to_string();
let (_, _, source_kind) = test_notebook_path(
path,
Path::new("after_fix.ipynb"),
&settings::Settings::for_rule(Rule::UnusedImport),
)?;
let mut writer = Vec::new();
source_kind.expect_jupyter().write_inner(&mut writer)?;
let actual = String::from_utf8(writer)?;
let expected =
std::fs::read_to_string(test_resource_path("fixtures/jupyter/after_fix.ipynb"))?;
assert_eq!(actual, expected);
Ok(())
}
#[test_case(Path::new("before_fix.ipynb"), true; "trailing_newline")]
#[test_case(Path::new("no_trailing_newline.ipynb"), false; "no_trailing_newline")]
fn test_trailing_newline(path: &Path, trailing_newline: bool) -> Result<()> {
let notebook = read_jupyter_notebook(path)?;
assert_eq!(notebook.trailing_newline, trailing_newline);
let mut writer = Vec::new();
notebook.write_inner(&mut writer)?;
let string = String::from_utf8(writer)?;
assert_eq!(string.ends_with('\n'), trailing_newline);
Ok(())
}
}

View File

@@ -1,89 +0,0 @@
---
source: crates/ruff/src/jupyter/notebook.rs
---
isort.ipynb:cell 1:1:1: I001 [*] Import block is un-sorted or un-formatted
|
1 | / from pathlib import Path
2 | | import random
3 | | import math
4 | | from typing import Any
| |_^ I001
5 | import collections
6 | # Newline should be added here
|
= help: Organize imports
Fix
1 |+import math
2 |+import random
1 3 | from pathlib import Path
2 |-import random
3 |-import math
4 4 | from typing import Any
5 5 | import collections
6 6 | # Newline should be added here
isort.ipynb:cell 2:1:1: I001 [*] Import block is un-sorted or un-formatted
|
1 | / from typing import Any
2 | | import collections
3 | | # Newline should be added here
| |_^ I001
4 | def foo():
5 | pass
|
= help: Organize imports
Fix
1 1 | from pathlib import Path
2 2 | import random
3 3 | import math
4 |+import collections
4 5 | from typing import Any
5 |-import collections
6 |+
7 |+
6 8 | # Newline should be added here
7 9 | def foo():
8 10 | pass
isort.ipynb:cell 3:1:1: I001 [*] Import block is un-sorted or un-formatted
|
1 | / from pathlib import Path
2 | | import sys
3 | |
4 | | %matplotlib \
| |_^ I001
5 | --inline
|
= help: Organize imports
Fix
6 6 | # Newline should be added here
7 7 | def foo():
8 8 | pass
9 |+import sys
9 10 | from pathlib import Path
10 |-import sys
11 11 |
12 12 | %matplotlib \
13 13 | --inline
isort.ipynb:cell 3:7:1: I001 [*] Import block is un-sorted or un-formatted
|
5 | --inline
6 |
7 | / import math
8 | | import abc
|
= help: Organize imports
Fix
12 12 | %matplotlib \
13 13 | --inline
14 14 |
15 |+import abc
15 16 | import math
16 |-import abc

View File

@@ -1,23 +0,0 @@
---
source: crates/ruff/src/jupyter/notebook.rs
---
ipy_escape_command.ipynb:cell 1:5:8: F401 [*] `os` imported but unused
|
3 | %matplotlib inline
4 |
5 | import os
| ^^ F401
6 |
7 | _ = math.pi
|
= help: Remove unused import: `os`
Fix
2 2 |
3 3 | %matplotlib inline
4 4 |
5 |-import os
6 5 |
7 6 | _ = math.pi

View File

@@ -1,72 +0,0 @@
---
source: crates/ruff/src/jupyter/notebook.rs
---
unused_variable.ipynb:cell 1:2:5: F841 [*] Local variable `foo1` is assigned to but never used
|
1 | def f():
2 | foo1 = %matplotlib --list
| ^^^^ F841
3 | foo2: list[str] = %matplotlib --list
|
= help: Remove assignment to unused variable `foo1`
Suggested fix
1 1 | def f():
2 |- foo1 = %matplotlib --list
2 |+ %matplotlib --list
3 3 | foo2: list[str] = %matplotlib --list
4 4 | def f():
5 5 | bar1 = !pwd
unused_variable.ipynb:cell 1:3:5: F841 [*] Local variable `foo2` is assigned to but never used
|
1 | def f():
2 | foo1 = %matplotlib --list
3 | foo2: list[str] = %matplotlib --list
| ^^^^ F841
|
= help: Remove assignment to unused variable `foo2`
Suggested fix
1 1 | def f():
2 2 | foo1 = %matplotlib --list
3 |- foo2: list[str] = %matplotlib --list
3 |+ %matplotlib --list
4 4 | def f():
5 5 | bar1 = !pwd
6 6 | bar2: str = !pwd
unused_variable.ipynb:cell 2:2:5: F841 [*] Local variable `bar1` is assigned to but never used
|
1 | def f():
2 | bar1 = !pwd
| ^^^^ F841
3 | bar2: str = !pwd
|
= help: Remove assignment to unused variable `bar1`
Suggested fix
2 2 | foo1 = %matplotlib --list
3 3 | foo2: list[str] = %matplotlib --list
4 4 | def f():
5 |- bar1 = !pwd
5 |+ !pwd
6 6 | bar2: str = !pwd
unused_variable.ipynb:cell 2:3:5: F841 [*] Local variable `bar2` is assigned to but never used
|
1 | def f():
2 | bar1 = !pwd
3 | bar2: str = !pwd
| ^^^^ F841
|
= help: Remove assignment to unused variable `bar2`
Suggested fix
3 3 | foo2: list[str] = %matplotlib --list
4 4 | def f():
5 5 | bar1 = !pwd
6 |- bar2: str = !pwd
6 |+ !pwd

View File

@@ -1,44 +0,0 @@
//! This is the library for the [Ruff] Python linter.
//!
//! **The API is currently completely unstable**
//! and subject to change drastically.
//!
//! [Ruff]: https://github.com/astral-sh/ruff
pub use rule_selector::RuleSelector;
pub use rules::pycodestyle::rules::IOError;
pub const VERSION: &str = env!("CARGO_PKG_VERSION");
mod autofix;
mod checkers;
mod codes;
mod comments;
mod cst;
pub mod directives;
mod doc_lines;
mod docstrings;
pub mod flake8_to_ruff;
pub mod fs;
mod importer;
pub mod jupyter;
mod lex;
pub mod line_width;
pub mod linter;
pub mod logging;
pub mod message;
mod noqa;
pub mod packaging;
pub mod pyproject_toml;
pub mod registry;
mod renamer;
pub mod resolver;
mod rule_redirects;
mod rule_selector;
pub mod rules;
pub mod settings;
pub mod source_kind;
pub mod upstream_categories;
#[cfg(any(test, fuzzing))]
pub mod test;

View File

@@ -1,166 +0,0 @@
use serde::{Deserialize, Serialize};
use std::num::NonZeroU8;
use unicode_width::UnicodeWidthChar;
use ruff_macros::CacheKey;
/// The length of a line of text that is considered too long.
#[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord, Serialize, Deserialize, CacheKey)]
#[cfg_attr(feature = "schemars", derive(schemars::JsonSchema))]
pub struct LineLength(usize);
impl Default for LineLength {
/// The default line length.
fn default() -> Self {
Self(88)
}
}
impl LineLength {
pub const fn get(&self) -> usize {
self.0
}
}
impl From<usize> for LineLength {
fn from(value: usize) -> Self {
Self(value)
}
}
/// A measure of the width of a line of text.
///
/// This is used to determine if a line is too long.
/// It should be compared to a [`LineLength`].
#[derive(Clone, Copy, Debug)]
pub struct LineWidth {
/// The width of the line.
width: usize,
/// The column of the line.
/// This is used to calculate the width of tabs.
column: usize,
/// The tab size to use when calculating the width of tabs.
tab_size: TabSize,
}
impl Default for LineWidth {
fn default() -> Self {
Self::new(TabSize::default())
}
}
impl PartialEq for LineWidth {
fn eq(&self, other: &Self) -> bool {
self.width == other.width
}
}
impl Eq for LineWidth {}
impl PartialOrd for LineWidth {
fn partial_cmp(&self, other: &Self) -> Option<std::cmp::Ordering> {
Some(self.cmp(other))
}
}
impl Ord for LineWidth {
fn cmp(&self, other: &Self) -> std::cmp::Ordering {
self.width.cmp(&other.width)
}
}
impl LineWidth {
pub fn get(&self) -> usize {
self.width
}
/// Creates a new `LineWidth` with the given tab size.
pub fn new(tab_size: TabSize) -> Self {
LineWidth {
width: 0,
column: 0,
tab_size,
}
}
fn update(mut self, chars: impl Iterator<Item = char>) -> Self {
let tab_size: usize = self.tab_size.as_usize();
for c in chars {
match c {
'\t' => {
let tab_offset = tab_size - (self.column % tab_size);
self.width += tab_offset;
self.column += tab_offset;
}
'\n' | '\r' => {
self.width = 0;
self.column = 0;
}
_ => {
self.width += c.width().unwrap_or(0);
self.column += 1;
}
}
}
self
}
/// Adds the given text to the line width.
#[must_use]
pub fn add_str(self, text: &str) -> Self {
self.update(text.chars())
}
/// Adds the given character to the line width.
#[must_use]
pub fn add_char(self, c: char) -> Self {
self.update(std::iter::once(c))
}
/// Adds the given width to the line width.
/// Also adds the given width to the column.
/// It is generally better to use [`LineWidth::add_str`] or [`LineWidth::add_char`].
/// The width and column should be the same for the corresponding text.
/// Currently, this is only used to add spaces.
#[must_use]
pub fn add_width(mut self, width: usize) -> Self {
self.width += width;
self.column += width;
self
}
}
impl PartialEq<LineLength> for LineWidth {
fn eq(&self, other: &LineLength) -> bool {
self.width == other.0
}
}
impl PartialOrd<LineLength> for LineWidth {
fn partial_cmp(&self, other: &LineLength) -> Option<std::cmp::Ordering> {
self.width.partial_cmp(&other.0)
}
}
/// The size of a tab.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, CacheKey)]
#[cfg_attr(feature = "schemars", derive(schemars::JsonSchema))]
pub struct TabSize(NonZeroU8);
impl TabSize {
pub(crate) fn as_usize(self) -> usize {
self.0.get() as usize
}
}
impl Default for TabSize {
fn default() -> Self {
Self(NonZeroU8::new(4).unwrap())
}
}
impl From<NonZeroU8> for TabSize {
fn from(tab_size: NonZeroU8) -> Self {
Self(tab_size)
}
}

View File

@@ -1,613 +0,0 @@
use std::borrow::Cow;
use std::ops::Deref;
use std::path::Path;
use anyhow::{anyhow, Result};
use colored::Colorize;
use itertools::Itertools;
use log::error;
use ruff_python_parser::lexer::LexResult;
use ruff_python_parser::{AsMode, ParseError};
use rustc_hash::FxHashMap;
use ruff_diagnostics::Diagnostic;
use ruff_python_ast::imports::ImportMap;
use ruff_python_ast::PySourceType;
use ruff_python_codegen::Stylist;
use ruff_python_index::Indexer;
use ruff_source_file::{Locator, SourceFileBuilder};
use crate::autofix::{fix_file, FixResult};
use crate::checkers::ast::check_ast;
use crate::checkers::filesystem::check_file_path;
use crate::checkers::imports::check_imports;
use crate::checkers::noqa::check_noqa;
use crate::checkers::physical_lines::check_physical_lines;
use crate::checkers::tokens::check_tokens;
use crate::directives::Directives;
use crate::doc_lines::{doc_lines_from_ast, doc_lines_from_tokens};
use crate::logging::DisplayParseError;
use crate::message::Message;
use crate::noqa::add_noqa;
use crate::registry::{AsRule, Rule};
use crate::rules::pycodestyle;
use crate::settings::{flags, Settings};
use crate::source_kind::SourceKind;
use crate::{directives, fs};
const CARGO_PKG_NAME: &str = env!("CARGO_PKG_NAME");
const CARGO_PKG_REPOSITORY: &str = env!("CARGO_PKG_REPOSITORY");
/// A [`Result`]-like type that returns both data and an error. Used to return
/// diagnostics even in the face of parse errors, since many diagnostics can be
/// generated without a full AST.
pub struct LinterResult<T> {
pub data: T,
pub error: Option<ParseError>,
}
impl<T> LinterResult<T> {
const fn new(data: T, error: Option<ParseError>) -> Self {
Self { data, error }
}
fn map<U, F: FnOnce(T) -> U>(self, f: F) -> LinterResult<U> {
LinterResult::new(f(self.data), self.error)
}
}
pub type FixTable = FxHashMap<Rule, usize>;
pub struct FixerResult<'a> {
/// The result returned by the linter, after applying any fixes.
pub result: LinterResult<(Vec<Message>, Option<ImportMap>)>,
/// The resulting source code, after applying any fixes.
pub transformed: Cow<'a, str>,
/// The number of fixes applied for each [`Rule`].
pub fixed: FixTable,
}
/// Generate `Diagnostic`s from the source code contents at the
/// given `Path`.
#[allow(clippy::too_many_arguments)]
pub fn check_path(
path: &Path,
package: Option<&Path>,
tokens: Vec<LexResult>,
locator: &Locator,
stylist: &Stylist,
indexer: &Indexer,
directives: &Directives,
settings: &Settings,
noqa: flags::Noqa,
source_kind: Option<&SourceKind>,
source_type: PySourceType,
) -> LinterResult<(Vec<Diagnostic>, Option<ImportMap>)> {
// Aggregate all diagnostics.
let mut diagnostics = vec![];
let mut imports = None;
let mut error = None;
// Collect doc lines. This requires a rare mix of tokens (for comments) and AST
// (for docstrings), which demands special-casing at this level.
let use_doc_lines = settings.rules.enabled(Rule::DocLineTooLong);
let mut doc_lines = vec![];
if use_doc_lines {
doc_lines.extend(doc_lines_from_tokens(&tokens));
}
// Run the token-based rules.
if settings
.rules
.iter_enabled()
.any(|rule_code| rule_code.lint_source().is_tokens())
{
diagnostics.extend(check_tokens(
&tokens,
path,
locator,
indexer,
settings,
source_type.is_stub(),
));
}
// Run the filesystem-based rules.
if settings
.rules
.iter_enabled()
.any(|rule_code| rule_code.lint_source().is_filesystem())
{
diagnostics.extend(check_file_path(path, package, settings));
}
// Run the logical line-based rules.
if settings
.rules
.iter_enabled()
.any(|rule_code| rule_code.lint_source().is_logical_lines())
{
diagnostics.extend(crate::checkers::logical_lines::check_logical_lines(
&tokens, locator, stylist, settings,
));
}
// Run the AST-based rules.
let use_ast = settings
.rules
.iter_enabled()
.any(|rule_code| rule_code.lint_source().is_ast());
let use_imports = !directives.isort.skip_file
&& settings
.rules
.iter_enabled()
.any(|rule_code| rule_code.lint_source().is_imports());
if use_ast || use_imports || use_doc_lines {
match ruff_python_parser::parse_program_tokens(
tokens,
&path.to_string_lossy(),
source_type.is_jupyter(),
) {
Ok(python_ast) => {
if use_ast {
diagnostics.extend(check_ast(
&python_ast,
locator,
stylist,
indexer,
&directives.noqa_line_for,
settings,
noqa,
path,
package,
source_type,
));
}
if use_imports {
let (import_diagnostics, module_imports) = check_imports(
&python_ast,
locator,
indexer,
&directives.isort,
settings,
stylist,
path,
package,
source_kind,
source_type,
);
imports = module_imports;
diagnostics.extend(import_diagnostics);
}
if use_doc_lines {
doc_lines.extend(doc_lines_from_ast(&python_ast, locator));
}
}
Err(parse_error) => {
// Always add a diagnostic for the syntax error, regardless of whether
// `Rule::SyntaxError` is enabled. We avoid propagating the syntax error
// if it's disabled via any of the usual mechanisms (e.g., `noqa`,
// `per-file-ignores`), and the easiest way to detect that suppression is
// to see if the diagnostic persists to the end of the function.
pycodestyle::rules::syntax_error(&mut diagnostics, &parse_error, locator);
error = Some(parse_error);
}
}
}
// Deduplicate and reorder any doc lines.
if use_doc_lines {
doc_lines.sort_unstable();
doc_lines.dedup();
}
// Run the lines-based rules.
if settings
.rules
.iter_enabled()
.any(|rule_code| rule_code.lint_source().is_physical_lines())
{
diagnostics.extend(check_physical_lines(
locator, stylist, indexer, &doc_lines, settings,
));
}
// Ignore diagnostics based on per-file-ignores.
if !diagnostics.is_empty() && !settings.per_file_ignores.is_empty() {
let ignores = fs::ignores_from_path(path, &settings.per_file_ignores);
if !ignores.is_empty() {
diagnostics.retain(|diagnostic| !ignores.contains(diagnostic.kind.rule()));
}
};
// Enforce `noqa` directives.
if (noqa.into() && !diagnostics.is_empty())
|| settings
.rules
.iter_enabled()
.any(|rule_code| rule_code.lint_source().is_noqa())
{
let ignored = check_noqa(
&mut diagnostics,
path,
locator,
indexer.comment_ranges(),
&directives.noqa_line_for,
error.is_none(),
settings,
);
if noqa.into() {
for index in ignored.iter().rev() {
diagnostics.swap_remove(*index);
}
}
}
// If there was a syntax error, check if it should be discarded.
if error.is_some() {
// If the syntax error was removed by _any_ of the above disablement methods (e.g., a
// `noqa` directive, or a `per-file-ignore`), discard it.
if !diagnostics
.iter()
.any(|diagnostic| diagnostic.kind.rule() == Rule::SyntaxError)
{
error = None;
}
// If the syntax error _diagnostic_ is disabled, discard the _diagnostic_.
if !settings.rules.enabled(Rule::SyntaxError) {
diagnostics.retain(|diagnostic| diagnostic.kind.rule() != Rule::SyntaxError);
}
}
LinterResult::new((diagnostics, imports), error)
}
const MAX_ITERATIONS: usize = 100;
/// Add any missing `# noqa` pragmas to the source code at the given `Path`.
pub fn add_noqa_to_path(path: &Path, package: Option<&Path>, settings: &Settings) -> Result<usize> {
let source_type = PySourceType::from(path);
// Read the file from disk.
let contents = std::fs::read_to_string(path)?;
// Tokenize once.
let tokens: Vec<LexResult> = ruff_python_parser::tokenize(&contents, source_type.as_mode());
// Map row and column locations to byte slices (lazily).
let locator = Locator::new(&contents);
// Detect the current code style (lazily).
let stylist = Stylist::from_tokens(&tokens, &locator);
// Extra indices from the code.
let indexer = Indexer::from_tokens(&tokens, &locator);
// Extract the `# noqa` and `# isort: skip` directives from the source.
let directives = directives::extract_directives(
&tokens,
directives::Flags::from_settings(settings),
&locator,
&indexer,
);
// Generate diagnostics, ignoring any existing `noqa` directives.
let LinterResult {
data: diagnostics,
error,
} = check_path(
path,
package,
tokens,
&locator,
&stylist,
&indexer,
&directives,
settings,
flags::Noqa::Disabled,
None,
source_type,
);
// Log any parse errors.
if let Some(err) = error {
// TODO(dhruvmanila): This should use `SourceKind`, update when
// `--add-noqa` is supported for Jupyter notebooks.
error!(
"{}",
DisplayParseError::new(err, locator.to_source_code(), None)
);
}
// Add any missing `# noqa` pragmas.
add_noqa(
path,
&diagnostics.0,
&locator,
indexer.comment_ranges(),
&directives.noqa_line_for,
stylist.line_ending(),
)
}
/// Generate a [`Message`] for each [`Diagnostic`] triggered by the given source
/// code.
pub fn lint_only(
contents: &str,
path: &Path,
package: Option<&Path>,
settings: &Settings,
noqa: flags::Noqa,
source_kind: Option<&SourceKind>,
source_type: PySourceType,
) -> LinterResult<(Vec<Message>, Option<ImportMap>)> {
// Tokenize once.
let tokens: Vec<LexResult> = ruff_python_parser::tokenize(contents, source_type.as_mode());
// Map row and column locations to byte slices (lazily).
let locator = Locator::new(contents);
// Detect the current code style (lazily).
let stylist = Stylist::from_tokens(&tokens, &locator);
// Extra indices from the code.
let indexer = Indexer::from_tokens(&tokens, &locator);
// Extract the `# noqa` and `# isort: skip` directives from the source.
let directives = directives::extract_directives(
&tokens,
directives::Flags::from_settings(settings),
&locator,
&indexer,
);
// Generate diagnostics.
let result = check_path(
path,
package,
tokens,
&locator,
&stylist,
&indexer,
&directives,
settings,
noqa,
source_kind,
source_type,
);
result.map(|(diagnostics, imports)| {
(
diagnostics_to_messages(diagnostics, path, &locator, &directives),
imports,
)
})
}
/// Convert from diagnostics to messages.
fn diagnostics_to_messages(
diagnostics: Vec<Diagnostic>,
path: &Path,
locator: &Locator,
directives: &Directives,
) -> Vec<Message> {
let file = once_cell::unsync::Lazy::new(|| {
let mut builder =
SourceFileBuilder::new(path.to_string_lossy().as_ref(), locator.contents());
if let Some(line_index) = locator.line_index() {
builder.set_line_index(line_index.clone());
}
builder.finish()
});
diagnostics
.into_iter()
.map(|diagnostic| {
let noqa_offset = directives.noqa_line_for.resolve(diagnostic.start());
Message::from_diagnostic(diagnostic, file.deref().clone(), noqa_offset)
})
.collect()
}
/// Generate `Diagnostic`s from source code content, iteratively autofixing
/// until stable.
pub fn lint_fix<'a>(
contents: &'a str,
path: &Path,
package: Option<&Path>,
noqa: flags::Noqa,
settings: &Settings,
source_kind: &mut SourceKind,
source_type: PySourceType,
) -> Result<FixerResult<'a>> {
let mut transformed = Cow::Borrowed(contents);
// Track the number of fixed errors across iterations.
let mut fixed = FxHashMap::default();
// As an escape hatch, bail after 100 iterations.
let mut iterations = 0;
// Track whether the _initial_ source code was parseable.
let mut parseable = false;
// Continuously autofix until the source code stabilizes.
loop {
// Tokenize once.
let tokens: Vec<LexResult> =
ruff_python_parser::tokenize(&transformed, source_type.as_mode());
// Map row and column locations to byte slices (lazily).
let locator = Locator::new(&transformed);
// Detect the current code style (lazily).
let stylist = Stylist::from_tokens(&tokens, &locator);
// Extra indices from the code.
let indexer = Indexer::from_tokens(&tokens, &locator);
// Extract the `# noqa` and `# isort: skip` directives from the source.
let directives = directives::extract_directives(
&tokens,
directives::Flags::from_settings(settings),
&locator,
&indexer,
);
// Generate diagnostics.
let result = check_path(
path,
package,
tokens,
&locator,
&stylist,
&indexer,
&directives,
settings,
noqa,
Some(source_kind),
source_type,
);
if iterations == 0 {
parseable = result.error.is_none();
} else {
// If the source code was parseable on the first pass, but is no
// longer parseable on a subsequent pass, then we've introduced a
// syntax error. Return the original code.
if parseable && result.error.is_some() {
report_autofix_syntax_error(
path,
&transformed,
&result.error.unwrap(),
fixed.keys().copied(),
);
return Err(anyhow!("Autofix introduced a syntax error"));
}
}
// Apply autofix.
if let Some(FixResult {
code: fixed_contents,
fixes: applied,
source_map,
}) = fix_file(&result.data.0, &locator)
{
if iterations < MAX_ITERATIONS {
// Count the number of fixed errors.
for (rule, count) in applied {
*fixed.entry(rule).or_default() += count;
}
if let SourceKind::Jupyter(notebook) = source_kind {
notebook.update(&source_map, &fixed_contents);
}
// Store the fixed contents.
transformed = Cow::Owned(fixed_contents);
// Increment the iteration count.
iterations += 1;
// Re-run the linter pass (by avoiding the break).
continue;
}
report_failed_to_converge_error(path, &transformed, &result.data.0);
}
return Ok(FixerResult {
result: result.map(|(diagnostics, imports)| {
(
diagnostics_to_messages(diagnostics, path, &locator, &directives),
imports,
)
}),
transformed,
fixed,
});
}
}
fn collect_rule_codes(rules: impl IntoIterator<Item = Rule>) -> String {
rules
.into_iter()
.map(|rule| rule.noqa_code().to_string())
.sorted_unstable()
.dedup()
.join(", ")
}
#[allow(clippy::print_stderr)]
fn report_failed_to_converge_error(path: &Path, transformed: &str, diagnostics: &[Diagnostic]) {
let codes = collect_rule_codes(diagnostics.iter().map(|diagnostic| diagnostic.kind.rule()));
if cfg!(debug_assertions) {
eprintln!(
"{}: Failed to converge after {} iterations in `{}` with rule codes {}:---\n{}\n---",
"debug error".red().bold(),
MAX_ITERATIONS,
fs::relativize_path(path),
codes,
transformed,
);
} else {
eprintln!(
r#"
{}: Failed to converge after {} iterations.
This indicates a bug in `{}`. If you could open an issue at:
{}/issues/new?title=%5BInfinite%20loop%5D
...quoting the contents of `{}`, the rule codes {}, along with the `pyproject.toml` settings and executed command, we'd be very appreciative!
"#,
"error".red().bold(),
MAX_ITERATIONS,
CARGO_PKG_NAME,
CARGO_PKG_REPOSITORY,
fs::relativize_path(path),
codes
);
}
}
#[allow(clippy::print_stderr)]
fn report_autofix_syntax_error(
path: &Path,
transformed: &str,
error: &ParseError,
rules: impl IntoIterator<Item = Rule>,
) {
let codes = collect_rule_codes(rules);
if cfg!(debug_assertions) {
eprintln!(
"{}: Autofix introduced a syntax error in `{}` with rule codes {}: {}\n---\n{}\n---",
"error".red().bold(),
fs::relativize_path(path),
codes,
error,
transformed,
);
} else {
eprintln!(
r#"
{}: Autofix introduced a syntax error. Reverting all changes.
This indicates a bug in `{}`. If you could open an issue at:
{}/issues/new?title=%5BAutofix%20error%5D
...quoting the contents of `{}`, the rule codes {}, along with the `pyproject.toml` settings and executed command, we'd be very appreciative!
"#,
"error".red().bold(),
CARGO_PKG_NAME,
CARGO_PKG_REPOSITORY,
fs::relativize_path(path),
codes,
);
}
}

View File

@@ -1,296 +0,0 @@
use std::fmt::{Display, Formatter, Write};
use std::path::Path;
use std::sync::Mutex;
use anyhow::Result;
use colored::Colorize;
use fern;
use log::Level;
use once_cell::sync::Lazy;
use ruff_python_parser::{ParseError, ParseErrorType};
use ruff_source_file::{OneIndexed, SourceCode, SourceLocation};
use crate::fs;
use crate::jupyter::Notebook;
use crate::source_kind::SourceKind;
pub(crate) static WARNINGS: Lazy<Mutex<Vec<&'static str>>> = Lazy::new(Mutex::default);
/// Warn a user once, with uniqueness determined by the given ID.
#[macro_export]
macro_rules! warn_user_once_by_id {
($id:expr, $($arg:tt)*) => {
use colored::Colorize;
use log::warn;
if let Ok(mut states) = $crate::logging::WARNINGS.lock() {
if !states.contains(&$id) {
let message = format!("{}", format_args!($($arg)*));
warn!("{}", message.bold());
states.push($id);
}
}
};
}
/// Warn a user once, with uniqueness determined by the calling location itself.
#[macro_export]
macro_rules! warn_user_once {
($($arg:tt)*) => {
use colored::Colorize;
use log::warn;
static WARNED: std::sync::atomic::AtomicBool = std::sync::atomic::AtomicBool::new(false);
if !WARNED.swap(true, std::sync::atomic::Ordering::SeqCst) {
let message = format!("{}", format_args!($($arg)*));
warn!("{}", message.bold());
}
};
}
#[macro_export]
macro_rules! warn_user {
($($arg:tt)*) => {{
use colored::Colorize;
use log::warn;
let message = format!("{}", format_args!($($arg)*));
warn!("{}", message.bold());
}};
}
#[macro_export]
macro_rules! notify_user {
($($arg:tt)*) => {
println!(
"[{}] {}",
chrono::Local::now()
.format("%H:%M:%S %p")
.to_string()
.dimmed(),
format_args!($($arg)*)
)
}
}
#[derive(Debug, Default, PartialOrd, Ord, PartialEq, Eq, Copy, Clone)]
pub enum LogLevel {
/// No output ([`log::LevelFilter::Off`]).
Silent,
/// Only show lint violations, with no decorative output
/// ([`log::LevelFilter::Off`]).
Quiet,
/// All user-facing output ([`log::LevelFilter::Info`]).
#[default]
Default,
/// All user-facing output ([`log::LevelFilter::Debug`]).
Verbose,
}
impl LogLevel {
#[allow(clippy::trivially_copy_pass_by_ref)]
const fn level_filter(&self) -> log::LevelFilter {
match self {
LogLevel::Default => log::LevelFilter::Info,
LogLevel::Verbose => log::LevelFilter::Debug,
LogLevel::Quiet => log::LevelFilter::Off,
LogLevel::Silent => log::LevelFilter::Off,
}
}
}
pub fn set_up_logging(level: &LogLevel) -> Result<()> {
fern::Dispatch::new()
.format(|out, message, record| match record.level() {
Level::Error => {
out.finish(format_args!(
"{}{} {}",
"error".red().bold(),
":".bold(),
message
));
}
Level::Warn => {
out.finish(format_args!(
"{}{} {}",
"warning".yellow().bold(),
":".bold(),
message
));
}
Level::Info | Level::Debug | Level::Trace => {
out.finish(format_args!(
"{}[{}][{}] {}",
chrono::Local::now().format("[%Y-%m-%d][%H:%M:%S]"),
record.target(),
record.level(),
message
));
}
})
.level(level.level_filter())
.level_for("globset", log::LevelFilter::Warn)
.chain(std::io::stderr())
.apply()?;
Ok(())
}
pub struct DisplayParseError<'a> {
error: ParseError,
source_code: SourceCode<'a, 'a>,
source_kind: Option<&'a SourceKind>,
}
impl<'a> DisplayParseError<'a> {
pub fn new(
error: ParseError,
source_code: SourceCode<'a, 'a>,
source_kind: Option<&'a SourceKind>,
) -> Self {
Self {
error,
source_code,
source_kind,
}
}
}
impl Display for DisplayParseError<'_> {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
write!(
f,
"{header} {path}{colon}",
header = "Failed to parse".bold(),
path = fs::relativize_path(Path::new(&self.error.source_path)).bold(),
colon = ":".cyan(),
)?;
let source_location = self.source_code.source_location(self.error.offset);
// If we're working on a Jupyter notebook, translate the positions
// with respect to the cell and row in the cell. This is the same
// format as the `TextEmitter`.
let error_location = if let Some(jupyter_index) = self
.source_kind
.and_then(SourceKind::notebook)
.map(Notebook::index)
{
write!(
f,
"cell {cell}{colon}",
cell = jupyter_index
.cell(source_location.row.get())
.unwrap_or_default(),
colon = ":".cyan(),
)?;
SourceLocation {
row: OneIndexed::new(
jupyter_index
.cell_row(source_location.row.get())
.unwrap_or(1) as usize,
)
.unwrap(),
column: source_location.column,
}
} else {
source_location
};
write!(
f,
"{row}{colon}{column}{colon} {inner}",
row = error_location.row,
column = error_location.column,
colon = ":".cyan(),
inner = &DisplayParseErrorType(&self.error.error)
)
}
}
pub(crate) struct DisplayParseErrorType<'a>(&'a ParseErrorType);
impl<'a> DisplayParseErrorType<'a> {
pub(crate) fn new(error: &'a ParseErrorType) -> Self {
Self(error)
}
}
impl Display for DisplayParseErrorType<'_> {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
match self.0 {
ParseErrorType::Eof => write!(f, "Expected token but reached end of file."),
ParseErrorType::ExtraToken(ref tok) => write!(
f,
"Got extraneous token: {tok}",
tok = TruncateAtNewline(&tok)
),
ParseErrorType::InvalidToken => write!(f, "Got invalid token"),
ParseErrorType::UnrecognizedToken(ref tok, ref expected) => {
if let Some(expected) = expected.as_ref() {
write!(
f,
"Expected '{expected}', but got {tok}",
tok = TruncateAtNewline(&tok)
)
} else {
write!(f, "Unexpected token {tok}", tok = TruncateAtNewline(&tok))
}
}
ParseErrorType::Lexical(ref error) => write!(f, "{error}"),
}
}
}
/// Truncates the display text before the first newline character to avoid line breaks.
struct TruncateAtNewline<'a>(&'a dyn Display);
impl Display for TruncateAtNewline<'_> {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
struct TruncateAdapter<'a> {
inner: &'a mut dyn Write,
after_new_line: bool,
}
impl Write for TruncateAdapter<'_> {
fn write_str(&mut self, s: &str) -> std::fmt::Result {
if self.after_new_line {
Ok(())
} else {
if let Some(end) = s.find(['\n', '\r']) {
self.inner.write_str(&s[..end])?;
self.inner.write_str("\u{23ce}...")?;
self.after_new_line = true;
Ok(())
} else {
self.inner.write_str(s)
}
}
}
}
write!(
TruncateAdapter {
inner: f,
after_new_line: false,
},
"{}",
self.0
)
}
}
#[cfg(test)]
mod tests {
use crate::logging::LogLevel;
#[test]
fn ordering() {
assert!(LogLevel::Default > LogLevel::Silent);
assert!(LogLevel::Default >= LogLevel::Default);
assert!(LogLevel::Quiet > LogLevel::Silent);
assert!(LogLevel::Verbose > LogLevel::Default);
assert!(LogLevel::Verbose > LogLevel::Silent);
}
}

View File

@@ -1,239 +0,0 @@
use std::cmp::Ordering;
use std::collections::BTreeMap;
use std::io::Write;
use std::ops::Deref;
use ruff_text_size::{TextRange, TextSize};
use rustc_hash::FxHashMap;
use crate::source_kind::SourceKind;
pub use azure::AzureEmitter;
pub use github::GithubEmitter;
pub use gitlab::GitlabEmitter;
pub use grouped::GroupedEmitter;
pub use json::JsonEmitter;
pub use json_lines::JsonLinesEmitter;
pub use junit::JunitEmitter;
pub use pylint::PylintEmitter;
use ruff_diagnostics::{Diagnostic, DiagnosticKind, Fix};
use ruff_source_file::{SourceFile, SourceLocation};
pub use text::TextEmitter;
mod azure;
mod diff;
mod github;
mod gitlab;
mod grouped;
mod json;
mod json_lines;
mod junit;
mod pylint;
mod text;
#[derive(Debug, PartialEq, Eq)]
pub struct Message {
pub kind: DiagnosticKind,
pub range: TextRange,
pub fix: Option<Fix>,
pub file: SourceFile,
pub noqa_offset: TextSize,
}
impl Message {
pub fn from_diagnostic(
diagnostic: Diagnostic,
file: SourceFile,
noqa_offset: TextSize,
) -> Self {
Self {
range: diagnostic.range(),
kind: diagnostic.kind,
fix: diagnostic.fix,
file,
noqa_offset,
}
}
pub fn filename(&self) -> &str {
self.file.name()
}
pub fn compute_start_location(&self) -> SourceLocation {
self.file.to_source_code().source_location(self.start())
}
pub fn compute_end_location(&self) -> SourceLocation {
self.file.to_source_code().source_location(self.end())
}
pub const fn start(&self) -> TextSize {
self.range.start()
}
pub const fn end(&self) -> TextSize {
self.range.end()
}
}
impl Ord for Message {
fn cmp(&self, other: &Self) -> Ordering {
(&self.file, self.start()).cmp(&(&other.file, other.start()))
}
}
impl PartialOrd for Message {
fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
Some(self.cmp(other))
}
}
struct MessageWithLocation<'a> {
message: &'a Message,
start_location: SourceLocation,
}
impl Deref for MessageWithLocation<'_> {
type Target = Message;
fn deref(&self) -> &Self::Target {
self.message
}
}
fn group_messages_by_filename(messages: &[Message]) -> BTreeMap<&str, Vec<MessageWithLocation>> {
let mut grouped_messages = BTreeMap::default();
for message in messages {
grouped_messages
.entry(message.filename())
.or_insert_with(Vec::new)
.push(MessageWithLocation {
message,
start_location: message.compute_start_location(),
});
}
grouped_messages
}
/// Display format for a [`Message`]s.
///
/// The emitter serializes a slice of [`Message`]'s and writes them to a [`Write`].
pub trait Emitter {
/// Serializes the `messages` and writes the output to `writer`.
fn emit(
&mut self,
writer: &mut dyn Write,
messages: &[Message],
context: &EmitterContext,
) -> anyhow::Result<()>;
}
/// Context passed to [`Emitter`].
pub struct EmitterContext<'a> {
source_kind: &'a FxHashMap<String, SourceKind>,
}
impl<'a> EmitterContext<'a> {
pub fn new(source_kind: &'a FxHashMap<String, SourceKind>) -> Self {
Self { source_kind }
}
/// Tests if the file with `name` is a jupyter notebook.
pub fn is_jupyter_notebook(&self, name: &str) -> bool {
self.source_kind
.get(name)
.is_some_and(SourceKind::is_jupyter)
}
pub fn source_kind(&self, name: &str) -> Option<&SourceKind> {
self.source_kind.get(name)
}
}
#[cfg(test)]
mod tests {
use ruff_text_size::{TextRange, TextSize};
use rustc_hash::FxHashMap;
use ruff_diagnostics::{Diagnostic, DiagnosticKind, Edit, Fix};
use ruff_source_file::SourceFileBuilder;
use crate::message::{Emitter, EmitterContext, Message};
pub(super) fn create_messages() -> Vec<Message> {
let fib = r#"import os
def fibonacci(n):
"""Compute the nth number in the Fibonacci sequence."""
x = 1
if n == 0:
return 0
elif n == 1:
return 1
else:
return fibonacci(n - 1) + fibonacci(n - 2)
"#;
let unused_import = Diagnostic::new(
DiagnosticKind {
name: "UnusedImport".to_string(),
body: "`os` imported but unused".to_string(),
suggestion: Some("Remove unused import: `os`".to_string()),
},
TextRange::new(TextSize::from(7), TextSize::from(9)),
)
.with_fix(Fix::suggested(Edit::range_deletion(TextRange::new(
TextSize::from(0),
TextSize::from(10),
))));
let fib_source = SourceFileBuilder::new("fib.py", fib).finish();
let unused_variable = Diagnostic::new(
DiagnosticKind {
name: "UnusedVariable".to_string(),
body: "Local variable `x` is assigned to but never used".to_string(),
suggestion: Some("Remove assignment to unused variable `x`".to_string()),
},
TextRange::new(TextSize::from(94), TextSize::from(95)),
)
.with_fix(Fix::suggested(Edit::deletion(
TextSize::from(94),
TextSize::from(99),
)));
let file_2 = r#"if a == 1: pass"#;
let undefined_name = Diagnostic::new(
DiagnosticKind {
name: "UndefinedName".to_string(),
body: "Undefined name `a`".to_string(),
suggestion: None,
},
TextRange::new(TextSize::from(3), TextSize::from(4)),
);
let file_2_source = SourceFileBuilder::new("undef.py", file_2).finish();
let unused_import_start = unused_import.start();
let unused_variable_start = unused_variable.start();
let undefined_name_start = undefined_name.start();
vec![
Message::from_diagnostic(unused_import, fib_source.clone(), unused_import_start),
Message::from_diagnostic(unused_variable, fib_source, unused_variable_start),
Message::from_diagnostic(undefined_name, file_2_source, undefined_name_start),
]
}
pub(super) fn capture_emitter_output(
emitter: &mut dyn Emitter,
messages: &[Message],
) -> String {
let source_kinds = FxHashMap::default();
let context = EmitterContext::new(&source_kinds);
let mut output: Vec<u8> = Vec::new();
emitter.emit(&mut output, messages, &context).unwrap();
String::from_utf8(output).expect("Output to be valid UTF-8")
}
}

View File

@@ -1,8 +0,0 @@
---
source: crates/ruff/src/message/azure.rs
expression: content
---
##vso[task.logissue type=error;sourcepath=fib.py;linenumber=1;columnnumber=8;code=F401;]`os` imported but unused
##vso[task.logissue type=error;sourcepath=fib.py;linenumber=6;columnnumber=5;code=F841;]Local variable `x` is assigned to but never used
##vso[task.logissue type=error;sourcepath=undef.py;linenumber=1;columnnumber=4;code=F821;]Undefined name `a`

View File

@@ -1,8 +0,0 @@
---
source: crates/ruff/src/message/github.rs
expression: content
---
::error title=Ruff (F401),file=fib.py,line=1,col=8,endLine=1,endColumn=10::fib.py:1:8: F401 `os` imported but unused
::error title=Ruff (F841),file=fib.py,line=6,col=5,endLine=6,endColumn=6::fib.py:6:5: F841 Local variable `x` is assigned to but never used
::error title=Ruff (F821),file=undef.py,line=1,col=4,endLine=1,endColumn=5::undef.py:1:4: F821 Undefined name `a`

View File

@@ -1,42 +0,0 @@
---
source: crates/ruff/src/message/gitlab.rs
expression: redact_fingerprint(&content)
---
[
{
"description": "(F401) `os` imported but unused",
"fingerprint": "<redacted>",
"location": {
"lines": {
"begin": 1,
"end": 1
},
"path": "fib.py"
},
"severity": "major"
},
{
"description": "(F841) Local variable `x` is assigned to but never used",
"fingerprint": "<redacted>",
"location": {
"lines": {
"begin": 6,
"end": 6
},
"path": "fib.py"
},
"severity": "major"
},
{
"description": "(F821) Undefined name `a`",
"fingerprint": "<redacted>",
"location": {
"lines": {
"begin": 1,
"end": 1
},
"path": "undef.py"
},
"severity": "major"
}
]

View File

@@ -1,12 +0,0 @@
---
source: crates/ruff/src/message/grouped.rs
expression: content
---
fib.py:
1:8 F401 `os` imported but unused
6:5 F841 Local variable `x` is assigned to but never used
undef.py:
1:4 F821 Undefined name `a`

View File

@@ -1,31 +0,0 @@
---
source: crates/ruff/src/message/grouped.rs
expression: content
---
fib.py:
1:8 F401 [*] `os` imported but unused
|
1 | import os
| ^^ F401
|
= help: Remove unused import: `os`
6:5 F841 [*] Local variable `x` is assigned to but never used
|
4 | def fibonacci(n):
5 | """Compute the nth number in the Fibonacci sequence."""
6 | x = 1
| ^ F841
7 | if n == 0:
8 | return 0
|
= help: Remove assignment to unused variable `x`
undef.py:
1:4 F821 Undefined name `a`
|
1 | if a == 1: pass
| ^ F821
|

View File

@@ -1,31 +0,0 @@
---
source: crates/ruff/src/message/grouped.rs
expression: content
---
fib.py:
1:8 F401 `os` imported but unused
|
1 | import os
| ^^ F401
|
= help: Remove unused import: `os`
6:5 F841 Local variable `x` is assigned to but never used
|
4 | def fibonacci(n):
5 | """Compute the nth number in the Fibonacci sequence."""
6 | x = 1
| ^ F841
7 | if n == 0:
8 | return 0
|
= help: Remove assignment to unused variable `x`
undef.py:
1:4 F821 Undefined name `a`
|
1 | if a == 1: pass
| ^ F821
|

View File

@@ -1,86 +0,0 @@
---
source: crates/ruff/src/message/json.rs
expression: content
---
[
{
"code": "F401",
"end_location": {
"column": 10,
"row": 1
},
"filename": "fib.py",
"fix": {
"applicability": "Suggested",
"edits": [
{
"content": "",
"end_location": {
"column": 1,
"row": 2
},
"location": {
"column": 1,
"row": 1
}
}
],
"message": "Remove unused import: `os`"
},
"location": {
"column": 8,
"row": 1
},
"message": "`os` imported but unused",
"noqa_row": 1,
"url": "https://beta.ruff.rs/docs/rules/unused-import"
},
{
"code": "F841",
"end_location": {
"column": 6,
"row": 6
},
"filename": "fib.py",
"fix": {
"applicability": "Suggested",
"edits": [
{
"content": "",
"end_location": {
"column": 10,
"row": 6
},
"location": {
"column": 5,
"row": 6
}
}
],
"message": "Remove assignment to unused variable `x`"
},
"location": {
"column": 5,
"row": 6
},
"message": "Local variable `x` is assigned to but never used",
"noqa_row": 6,
"url": "https://beta.ruff.rs/docs/rules/unused-variable"
},
{
"code": "F821",
"end_location": {
"column": 5,
"row": 1
},
"filename": "undef.py",
"fix": null,
"location": {
"column": 4,
"row": 1
},
"message": "Undefined name `a`",
"noqa_row": 1,
"url": "https://beta.ruff.rs/docs/rules/undefined-name"
}
]

View File

@@ -1,8 +0,0 @@
---
source: crates/ruff/src/message/json_lines.rs
expression: content
---
{"code":"F401","end_location":{"column":10,"row":1},"filename":"fib.py","fix":{"applicability":"Suggested","edits":[{"content":"","end_location":{"column":1,"row":2},"location":{"column":1,"row":1}}],"message":"Remove unused import: `os`"},"location":{"column":8,"row":1},"message":"`os` imported but unused","noqa_row":1,"url":"https://beta.ruff.rs/docs/rules/unused-import"}
{"code":"F841","end_location":{"column":6,"row":6},"filename":"fib.py","fix":{"applicability":"Suggested","edits":[{"content":"","end_location":{"column":10,"row":6},"location":{"column":5,"row":6}}],"message":"Remove assignment to unused variable `x`"},"location":{"column":5,"row":6},"message":"Local variable `x` is assigned to but never used","noqa_row":6,"url":"https://beta.ruff.rs/docs/rules/unused-variable"}
{"code":"F821","end_location":{"column":5,"row":1},"filename":"undef.py","fix":null,"location":{"column":4,"row":1},"message":"Undefined name `a`","noqa_row":1,"url":"https://beta.ruff.rs/docs/rules/undefined-name"}

Some files were not shown because too many files have changed in this diff Show More