Compare commits

...

216 Commits

Author SHA1 Message Date
Charlie Marsh
d8c9500b37 Rename all of these structs 2023-07-30 19:56:11 -04:00
Charlie Marsh
ac25bd9596 Traits 2023-07-30 19:55:50 -04:00
Charlie Marsh
ac0f51ef4f Some abstractions 2023-07-30 19:55:50 -04:00
Charlie Marsh
3eec7de6e4 Clean up 2023-07-30 19:55:50 -04:00
Charlie Marsh
5dbe4e4653 Remove qualified name 2023-07-30 19:55:49 -04:00
Charlie Marsh
43eb51b15a Handle relative 2023-07-30 19:55:23 -04:00
Charlie Marsh
68a12f72a4 Tweaks 2023-07-30 19:55:23 -04:00
Charlie Marsh
15273c6d95 Store the call path 2023-07-30 19:55:23 -04:00
Charlie Marsh
de898c52eb Avoid falsely marking non-submodules as submodule aliases (#6182)
## Summary

We have some code to ensure that if an aliased import is used, any
submodules should be marked as used too. This comment says it best:

```rust
// If the name of a submodule import is the same as an alias of another import, and the
// alias is used, then the submodule import should be marked as used too.
//
// For example, mark `pyarrow.csv` as used in:
//
// ```python
// import pyarrow as pa
// import pyarrow.csv
// print(pa.csv.read_csv("test.csv"))
// ```
```

However, it looks like when we go to look up `pyarrow` (of `import
pyarrow as pa`), we aren't checking to ensure the resolved binding is
_actually_ an import. This was causing us to attribute `print(rm.ANY)`
to `def requests_mock` here:

```python
import requests_mock as rm

def requests_mock(requests_mock: rm.Mocker):
    print(rm.ANY)
```

Closes https://github.com/astral-sh/ruff/issues/6180.
2023-07-30 22:16:25 +00:00
Charlie Marsh
76741cac77 Add global and nonlocal formatting (#6170)
## Summary

Adds `global` and `nonlocal` formatting, without the "deviation from
black" outlined in the linked issue, which I'll do separately.

See: https://github.com/astral-sh/ruff/issues/4798.

## Test Plan

Added a fixture in the Ruff-specific directory since the Black fixtures
don't seem to cover this.
2023-07-29 14:39:42 +00:00
Charlie Marsh
5d9814d84d Remove parentheses around some walrus operators (#6173)
## Summary

Closes https://github.com/astral-sh/ruff/issues/5781

## Test Plan

Added cases to
`crates/ruff_python_formatter/resources/test/fixtures/ruff/expression/named_expr.py`
one-by-one and adjusted the condition as needed.
2023-07-29 10:06:26 -04:00
Micha Reiser
1d7ad30188 CI: Update formatter dependencies (#6168) 2023-07-29 15:24:24 +02:00
Charlie Marsh
4231ed2fc3 Skip partial duplicates when applying multi-edit fixes (#6144)
## Summary

Right now, if we have two fixes that have an overlapping edit, but not
an _identical_ set of edits, they'll conflict, causing us to do another
linter traversal. Here, I've enabled the fixer to support partially
overlapping edits, which (as an example) let's us greatly reduce the
number of iterations required in the test suite.

The most common case here is that in which a bunch of edits need to
import some symbol, and then use that symbol, but in different ways. In
that case, all edits will have a common fix (to import the symbol), but
deviate in some way. With this change, we can do all of those edits in
one pass.

Note that the simplest way to enable this was to store sorted edits on
`Fix`. We don't allow modifying the edits on `Fix` once it's
constructed, so this is an easy change, and allows us to avoid a bunch
of clones and traversals later on.

Closes #5800.
2023-07-29 12:11:57 +00:00
Charlie Marsh
badbfb2d3e Skip BOM when determining Locator's line starts (#6159)
## Summary

If a file has a BOM, the import sorter _always_ reports the imports as
unsorted. The acute issue is that we detect that the line has leading
content (before the imports), which we always consider a violation.
Rather than fixing that one site, this PR instead makes `.line_start`
BOM-aware.

Fixes https://github.com/astral-sh/ruff/issues/6155.
2023-07-29 11:47:13 +00:00
Dhruv Manilawala
44bdf20221 [pep8-naming]: New config option extend-ignore-names (#6169)
## Summary

This PR adds a new config option for `pep8-naming` plugin called
`extend-ignore-names` which is used to extend the default values in
`ignore-names` option.

resolves: #6050
2023-07-29 17:11:04 +05:30
Dhruv Manilawala
3c99fbf808 Implement --diff for Jupyter Notebooks (#6149)
## Summary

Implement `--diff` for Jupyter Notebooks

## Test Plan

1. Use `crates/ruff/resources/test/fixtures/jupyter/isort.ipynb` as a
test case
and add a markdown cell in between the code cells to check that the diff
   outputs the correct cell index.
2. Run the command:
`cargo run --bin ruff --package ruff_cli -- check --no-cache --isolated
--select=ALL crates/ruff/resources/test/fixtures/jupyter/isort.ipynb
--fix --diff`

<details><summary>Example output:</summary>
<p>

```diff
--- /Users/dhruv/playground/ruff/notebooks/test.ipynb:cell 0
+++ /Users/dhruv/playground/ruff/notebooks/test.ipynb:cell 0
@@ -1,3 +0,0 @@
-from pathlib import Path
-import random
-import math
--- /Users/dhruv/playground/ruff/notebooks/test.ipynb:cell 4
+++ /Users/dhruv/playground/ruff/notebooks/test.ipynb:cell 4
@@ -1,5 +1,3 @@
-from typing import Any
-import collections
 # Newline should be added here
 def foo():
     pass

--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 8
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 8
@@ -1,8 +1,7 @@
 import pprint
 import tempfile
 
-from IPython import display
 import matplotlib.pyplot as plt
-
 import tensorflow as tf
-import tensorflow_datasets as tfds
+import tensorflow_datasets as tfds
+from IPython import display
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 10
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 10
@@ -1,5 +1,4 @@
 import tensorflow_models as tfm
 
 # These are not in the tfm public API for v2.9. They will be available in v2.10
-from official.vision.serving import export_saved_model_lib
-import official.core.train_lib
+from official.vision.serving import export_saved_model_lib
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 13
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 13
@@ -1,5 +1,5 @@
-exp_config = tfm.core.exp_factory.get_exp_config('resnet_imagenet')
-tfds_name = 'cifar10'
+exp_config = tfm.core.exp_factory.get_exp_config("resnet_imagenet")
+tfds_name = "cifar10"
 ds,ds_info = tfds.load(
 tfds_name,
 with_info=True)
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 15
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 15
@@ -6,12 +6,12 @@
 # Configure training and testing data
 batch_size = 128
 
-exp_config.task.train_data.input_path = ''
+exp_config.task.train_data.input_path = ""
 exp_config.task.train_data.tfds_name = tfds_name
-exp_config.task.train_data.tfds_split = 'train'
+exp_config.task.train_data.tfds_split = "train"
 exp_config.task.train_data.global_batch_size = batch_size
 
-exp_config.task.validation_data.input_path = ''
+exp_config.task.validation_data.input_path = ""
 exp_config.task.validation_data.tfds_name = tfds_name
-exp_config.task.validation_data.tfds_split = 'test'
+exp_config.task.validation_data.tfds_split = "test"
 exp_config.task.validation_data.global_batch_size = batch_size
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 17
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 17
@@ -1,16 +1,16 @@
 logical_device_names = [logical_device.name for logical_device in tf.config.list_logical_devices()]
 
-if 'GPU' in ''.join(logical_device_names):
-  print('This may be broken in Colab.')
-  device = 'GPU'
-elif 'TPU' in ''.join(logical_device_names):
-  print('This may be broken in Colab.')
-  device = 'TPU'
+if "GPU" in "".join(logical_device_names):
+  print("This may be broken in Colab.")
+  device = "GPU"
+elif "TPU" in "".join(logical_device_names):
+  print("This may be broken in Colab.")
+  device = "TPU"
 else:
-  print('Running on CPU is slow, so only train for a few steps.')
-  device = 'CPU'
+  print("Running on CPU is slow, so only train for a few steps.")
+  device = "CPU"
 
-if device=='CPU':
+if device=="CPU":
   train_steps = 20
   exp_config.trainer.steps_per_loop = 5
 else:
@@ -20,9 +20,9 @@
 exp_config.trainer.summary_interval = 100
 exp_config.trainer.checkpoint_interval = train_steps
 exp_config.trainer.validation_interval = 1000
-exp_config.trainer.validation_steps =  ds_info.splits['test'].num_examples // batch_size
+exp_config.trainer.validation_steps =  ds_info.splits["test"].num_examples // batch_size
 exp_config.trainer.train_steps = train_steps
-exp_config.trainer.optimizer_config.learning_rate.type = 'cosine'
+exp_config.trainer.optimizer_config.learning_rate.type = "cosine"
 exp_config.trainer.optimizer_config.learning_rate.cosine.decay_steps = train_steps
 exp_config.trainer.optimizer_config.learning_rate.cosine.initial_learning_rate = 0.1
 exp_config.trainer.optimizer_config.warmup.linear.warmup_steps = 100
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 21
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 21
@@ -1,14 +1,14 @@
 logical_device_names = [logical_device.name for logical_device in tf.config.list_logical_devices()]
 
 if exp_config.runtime.mixed_precision_dtype == tf.float16:
-    tf.keras.mixed_precision.set_global_policy('mixed_float16')
+    tf.keras.mixed_precision.set_global_policy("mixed_float16")
 
-if 'GPU' in ''.join(logical_device_names):
+if "GPU" in "".join(logical_device_names):
   distribution_strategy = tf.distribute.MirroredStrategy()
-elif 'TPU' in ''.join(logical_device_names):
+elif "TPU" in "".join(logical_device_names):
   tf.tpu.experimental.initialize_tpu_system()
-  tpu = tf.distribute.cluster_resolver.TPUClusterResolver(tpu='/device:TPU_SYSTEM:0')
+  tpu = tf.distribute.cluster_resolver.TPUClusterResolver(tpu="/device:TPU_SYSTEM:0")
   distribution_strategy = tf.distribute.experimental.TPUStrategy(tpu)
 else:
-  print('Warning: this will be really slow.')
+  print("Warning: this will be really slow.")
   distribution_strategy = tf.distribute.OneDeviceStrategy(logical_device_names[0])
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 23
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 23
@@ -1,5 +1,3 @@
 with distribution_strategy.scope():
   model_dir = tempfile.mkdtemp()
   task = tfm.core.task_factory.get_task(exp_config.task, logging_dir=model_dir)
-
-#  tf.keras.utils.plot_model(task.build_model(), show_shapes=True)
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 24
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 24
@@ -1,4 +1,4 @@
 for images, labels in task.build_inputs(exp_config.task.train_data).take(1):
   print()
-  print(f'images.shape: {str(images.shape):16}  images.dtype: {images.dtype!r}')
-  print(f'labels.shape: {str(labels.shape):16}  labels.dtype: {labels.dtype!r}')
+  print(f"images.shape: {images.shape!s:16}  images.dtype: {images.dtype!r}")
+  print(f"labels.shape: {labels.shape!s:16}  labels.dtype: {labels.dtype!r}")
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 27
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 27
@@ -1 +1 @@
-plt.hist(images.numpy().flatten());
+plt.hist(images.numpy().flatten())
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 29
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 29
@@ -1,2 +1,2 @@
-label_info = ds_info.features['label']
+label_info = ds_info.features["label"]
 label_info.int2str(1)
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 31
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 31
@@ -10,9 +10,6 @@
     if predictions is None:
       plt.title(label_info.int2str(labels[i]))
     else:
-      if labels[i] == predictions[i]:
-        color = 'g'
-      else:
-        color = 'r'
+      color = "g" if labels[i] == predictions[i] else "r"
       plt.title(label_info.int2str(predictions[i]), color=color)
     plt.axis("off")
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 35
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 35
@@ -1,3 +1,3 @@
-plt.figure(figsize=(10, 10));
+plt.figure(figsize=(10, 10))
 for images, labels in task.build_inputs(exp_config.task.validation_data).take(1):
   show_batch(images, labels)
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 37
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 37
@@ -1,7 +1,7 @@
 model, eval_logs = tfm.core.train_lib.run_experiment(
     distribution_strategy=distribution_strategy,
     task=task,
-    mode='train_and_eval',
+    mode="train_and_eval",
     params=exp_config,
     model_dir=model_dir,
     run_post_eval=True)
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 38
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 38
@@ -1 +0,0 @@
-#  tf.keras.utils.plot_model(model, show_shapes=True)
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 40
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 40
@@ -1,4 +1,4 @@
 for key, value in eval_logs.items():
     if isinstance(value, tf.Tensor):
       value = value.numpy()
-    print(f'{key:20}: {value:.3f}')
+    print(f"{key:20}: {value:.3f}")
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 42
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 42
@@ -4,5 +4,5 @@
 
 show_batch(images, labels, tf.cast(predictions, tf.int32))
 
-if device=='CPU':
-  plt.suptitle('The model was only trained for a few steps, it is not expected to do well.')
+if device=="CPU":
+  plt.suptitle("The model was only trained for a few steps, it is not expected to do well.")
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 45
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 45
@@ -1,8 +1,8 @@
 # Saving and exporting the trained model
 export_saved_model_lib.export_inference_graph(
-    input_type='image_tensor',
+    input_type="image_tensor",
     batch_size=1,
     input_image_size=[32, 32],
     params=exp_config,
     checkpoint_path=tf.train.latest_checkpoint(model_dir),
-    export_dir='./export/')
+    export_dir="./export/")
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 47
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 47
@@ -1,3 +1,3 @@
 # Importing SavedModel
-imported = tf.saved_model.load('./export/')
-model_fn = imported.signatures['serving_default']
+imported = tf.saved_model.load("./export/")
+model_fn = imported.signatures["serving_default"]
--- /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 49
+++ /Users/dhruv/playground/ruff/notebooks/image_classification.ipynb:cell 49
@@ -1,10 +1,10 @@
 plt.figure(figsize=(10, 10))
-for data in tfds.load('cifar10', split='test').batch(12).take(1):
+for data in tfds.load("cifar10", split="test").batch(12).take(1):
   predictions = []
-  for image in data['image']:
-    index = tf.argmax(model_fn(image[tf.newaxis, ...])['logits'], axis=1)[0]
+  for image in data["image"]:
+    index = tf.argmax(model_fn(image[tf.newaxis, ...])["logits"], axis=1)[0]
     predictions.append(index)
-  show_batch(data['image'], data['label'], predictions)
+  show_batch(data["image"], data["label"], predictions)
 
-  if device=='CPU':
-    plt.suptitle('The model was only trained for a few steps, it is not expected to do better than random.')
+  if device=="CPU":
+    plt.suptitle("The model was only trained for a few steps, it is not expected to do better than random.")

Would fix 61 errors.
```

</p>
</details> 

resolves: #4727
2023-07-29 04:22:56 +00:00
Charlie Marsh
4802c7c7d8 Avoid key-in-dict violations for self accesses (#6165)
Closes https://github.com/astral-sh/ruff/issues/6163.
2023-07-29 03:35:26 +00:00
Charlie Marsh
646ff6497c Ignore end-of-line file exemption comments (#6160)
## Summary

This PR protects against code like:

```python
from typing import Optional

import bar  # ruff: noqa
import baz

class Foo:
    x: Optional[str] = None
```

In which the user wrote `# ruff: noqa` to ignore a specific error, not
realizing that it was a file-level exemption that thus turned off all
lint rules.

Specifically, if a `# ruff: noqa` directive is not at the start of a
line, we now ignore it and warn, since this is almost certainly a
mistake.
2023-07-29 00:40:32 +00:00
Victor Hugo Gomes
e0d5c7564f [flake8-pyi] Implement PYI049 (#6136)
## Summary

Checks for the presence of unused private `typing.TypedDict`
definitions.

ref #848 

## Test Plan

Snapshots and manual runs of flake8
2023-07-29 00:34:36 +00:00
Victor Hugo Gomes
7838d8c8af Implement PYI047 (#6134)
## Summary

Checks for the presence of unused private `typing.TypeAlias`
definitions.

ref #848 

## Test Plan

Snapshots and manual runs of flake8
2023-07-29 00:21:29 +00:00
Zanie Blue
047c211837 Add semantic analysis of type aliases and parameters (#6109)
Requires https://github.com/astral-sh/RustPython-Parser/pull/42
Related https://github.com/PyCQA/pyflakes/pull/778
[PEP-695](https://peps.python.org/pep-0695)
Part of #5062 

<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->
Adds a scope for type parameters, a type parameter binding kind, and
checker visitation of type parameters in type alias statements, function
definitions, and class definitions.

A few changes were necessary to ensure correctness following the
insertion of a new scope between function and class scopes and their
parent.

## Test Plan

<!-- How was it tested? -->
Undefined name snapshots.

Unused type parameter rule will be added as follow-up.
2023-07-28 17:06:37 -05:00
Charlie Marsh
134d447d4c Avoid refactoring x[:1]-like slices in RUF015 (#6150)
## Summary

Right now, `RUF015` will try to rewrite `x[:1]` as `[next(x)]`. This
isn't equivalent if `x`, for example, is empty, where slicing like
`x[:1]` is forgiving, but `next` raises `StopIteration`. For me this is
a little too much of a deviation to be comfortable with, and most of the
value in this rule is the `x[0]` to `next(x)` conversion anyway.

Closes https://github.com/astral-sh/ruff/issues/6148.
2023-07-28 09:38:13 -04:00
Charlie Marsh
cd4147423c Skip PERF203 violations for multi-statement loops (#6145)
Closes https://github.com/astral-sh/ruff/issues/5858.
2023-07-28 04:55:55 +00:00
Charlie Marsh
d15436458f Only run unused private type rules over finalized bindings (#6142)
## Summary

In #6134 and #6136, we see some false positives for "shadowed" class
definitions. For example, here, the first definition is flagged as
unused, since from the perspective of the semantic model (which doesn't
understand branching), it appears to be immediately shadowed in the
`else`, and thus never used:

```python
if sys.version_info >= (3, 11):
    class _RootLoggerConfiguration(TypedDict, total=False):
        level: _Level
        filters: Sequence[str | _FilterType]
        handlers: Sequence[str]

else:
    class _RootLoggerConfiguration(TypedDict, total=False):
        level: _Level
        filters: Sequence[str]
        handlers: Sequence[str]
```

Instead of looking at _all_ bindings, we should instead look at the
"live" bindings, which is similar to how other rules (like unused
variables detection) is structured. We thus move the rule from
`bindings.rs` (which iterates over _all_ bindings, regardless of whether
they're shadowed) to `deferred_scopes.rs`, which iterates over all
"live" bindings once a scope has been fully analyzed.

## Test Plan

`cargo test`
2023-07-28 02:16:09 +00:00
Charlie Marsh
0bc3edf6c9 Add documentation and test cases for redefinition (#6135) 2023-07-28 00:01:42 +00:00
Aarni Koskela
3d54d31cd9 Implement E241 and E242 (tab/multiple ws after commas) (#6094)
## Summary

This PR implements pycodestyle's E241 (tab after comma) and E242
(multiple whitespace after comma) lints.

These are marked as nursery rules like many other pycodestyle rules.

Refs #2402

## Test Plan

E24.py copied from pycodestyle.
2023-07-27 18:58:41 +00:00
Tom Kuson
1418ee62f8 Add more documentation to the flake8-bandit rules (#6128)
## Summary

Completes the documentation for the ruleset, apart from four rules which
have contradictions, so need to be thought about more regarding how to
document that. Related to #2646.

## Test Plan

`python scripts/test_docs_formatted.py`
2023-07-27 18:57:45 +00:00
Harutaka Kawamura
bf987f80f4 Add PT017 and PT019 docs (#6115) 2023-07-27 18:56:34 +00:00
rembridge
bb08eea5cc missing-whitespace-around-operators comment (#6106)
**Summary**

Updated doc comments for `missing_whitespace_around_operator.rs`. Online
docs also benefit from this update.

**Test Plan**

Checked docs via
[mkdocs](389fe13c93/CONTRIBUTING.md (L267-L296))
2023-07-27 14:52:43 -04:00
Tom Kuson
d16216a2c2 Add documentation to the flynt rules (#6130)
## Summary

Completes the documentation for the one and only (current) rule in the
`flynt` ruleset. Related to #2646.

## Test Plan

`python scripts/test_docs_formatted.py`
2023-07-27 14:32:59 -04:00
Jelle van der Waa
0853004f41 [pylint] Implement eq-without-hash rule (PLW1641) (#5955)
Implement
https://pylint.pycqa.org/en/latest/user_guide/messages/warning/eq-without-hash.html
Issue https://github.com/astral-sh/ruff/issues/970

It's not enabled by default in pylint, so I guess it shouldn't in Ruff
either?
2023-07-27 18:28:44 +00:00
Harutaka Kawamura
fb5bbe30c7 Update SIM115 to cover pathlib.Path.open (#6118) 2023-07-27 14:20:52 -04:00
Charlie Marsh
dd706c7a35 Fix E211 documentation (#6133) 2023-07-27 17:19:33 +00:00
Charlie Marsh
e15b9c5572 Cache name resolutions in the semantic model (#6047)
## Summary

This PR stores the mapping from `ExprName` node to resolved `BindingId`,
which lets us skip scope lookups in `resolve_call_path`. It's enabled by
#6045, since that PR ensures that when we analyze a node (and thus call
`resolve_call_path`), we'll have already visited its `ExprName`
elements.

In more detail: imagine that we're traversing over `foo.bar()`. When we
read `foo`, it will be an `ExprName`, which we'll then resolve to a
binding via `handle_node_load`. With this change, we then store that
binding in a map. Later, if we call `collect_call_path` on `foo.bar`,
we'll identify `foo` (the "head" of the attribute) and grab the resolved
binding in that map. _Almost_ all names are now resolved in advance,
though it's not a strict requirement, and some rules break that pattern
(e.g., if we're analyzing arguments, and they need to inspect their
annotations, which are visited in a deferred manner).

This improves performance by 4-6% on the all-rules benchmark. It looks
like it hurts performance (1-2% drop) in the default-rules benchmark,
presumedly because those rules don't call `resolve_call_path` nearly as
much, and so we're paying for these extra writes.

Here's the benchmark data:

```
linter/default-rules/numpy/globals.py
                        time:   [67.270 µs 67.380 µs 67.489 µs]
                        thrpt:  [43.720 MiB/s 43.792 MiB/s 43.863 MiB/s]
                 change:
                        time:   [+0.4747% +0.7752% +1.0626%] (p = 0.00 < 0.05)
                        thrpt:  [-1.0514% -0.7693% -0.4724%]
                        Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe
linter/default-rules/pydantic/types.py
                        time:   [1.4067 ms 1.4105 ms 1.4146 ms]
                        thrpt:  [18.028 MiB/s 18.081 MiB/s 18.129 MiB/s]
                 change:
                        time:   [+1.3152% +1.6953% +2.0414%] (p = 0.00 < 0.05)
                        thrpt:  [-2.0006% -1.6671% -1.2981%]
                        Performance has regressed.
linter/default-rules/numpy/ctypeslib.py
                        time:   [637.67 µs 638.96 µs 640.28 µs]
                        thrpt:  [26.006 MiB/s 26.060 MiB/s 26.113 MiB/s]
                 change:
                        time:   [+1.5859% +1.8109% +2.0353%] (p = 0.00 < 0.05)
                        thrpt:  [-1.9947% -1.7787% -1.5611%]
                        Performance has regressed.
linter/default-rules/large/dataset.py
                        time:   [3.2289 ms 3.2336 ms 3.2383 ms]
                        thrpt:  [12.563 MiB/s 12.581 MiB/s 12.599 MiB/s]
                 change:
                        time:   [+0.8029% +0.9898% +1.1740%] (p = 0.00 < 0.05)
                        thrpt:  [-1.1604% -0.9801% -0.7965%]
                        Change within noise threshold.

linter/all-rules/numpy/globals.py
                        time:   [134.05 µs 134.15 µs 134.26 µs]
                        thrpt:  [21.977 MiB/s 21.995 MiB/s 22.012 MiB/s]
                 change:
                        time:   [-4.4571% -4.1175% -3.8268%] (p = 0.00 < 0.05)
                        thrpt:  [+3.9791% +4.2943% +4.6651%]
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  2 (2.00%) low mild
  3 (3.00%) high mild
  3 (3.00%) high severe
linter/all-rules/pydantic/types.py
                        time:   [2.5627 ms 2.5669 ms 2.5720 ms]
                        thrpt:  [9.9158 MiB/s 9.9354 MiB/s 9.9516 MiB/s]
                 change:
                        time:   [-5.8304% -5.6374% -5.4452%] (p = 0.00 < 0.05)
                        thrpt:  [+5.7587% +5.9742% +6.1914%]
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  6 (6.00%) high mild
  1 (1.00%) high severe
linter/all-rules/numpy/ctypeslib.py
                        time:   [1.3949 ms 1.3956 ms 1.3964 ms]
                        thrpt:  [11.925 MiB/s 11.931 MiB/s 11.937 MiB/s]
                 change:
                        time:   [-6.2496% -6.0856% -5.9293%] (p = 0.00 < 0.05)
                        thrpt:  [+6.3030% +6.4799% +6.6662%]
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  3 (3.00%) high mild
  4 (4.00%) high severe
linter/all-rules/large/dataset.py
                        time:   [5.5951 ms 5.6019 ms 5.6093 ms]
                        thrpt:  [7.2527 MiB/s 7.2623 MiB/s 7.2711 MiB/s]
                 change:
                        time:   [-5.1781% -4.9783% -4.8070%] (p = 0.00 < 0.05)
                        thrpt:  [+5.0497% +5.2391% +5.4608%]
                        Performance has improved.
```

Still playing with this (the concepts need better names, documentation,
etc.), but opening up for feedback.
2023-07-27 13:01:56 -04:00
qdegraaf
0638a26347 Add AnyExpressionYield to consolidate ExprYield and ExprYieldFrom (#6127)
Co-authored-by: Micha Reiser <micha@reiser.io>
2023-07-27 16:01:16 +00:00
konsti
2a65e6fc38 Explain check_docs_formatted.py error message (#6125)
## Summary

This is an error message only change to lead an implementor of a new
rule that has an unformatted or invalid bad example to the
right code.

## Test Plan

n/a
2023-07-27 10:22:13 -04:00
Charlie Marsh
13af91299d Avoid walking past root when resolving imports (#6126)
## Summary

Noticed in #5954: we walk _past_ the root rather than stopping _at_ the
root when attempting to traverse along the parent path. It's effectively
an off-by-one bug.
2023-07-27 10:22:13 -04:00
konsti
d317af442f Fix windows test warnings (#6124)
See
https://github.com/astral-sh/ruff/actions/runs/5679922286/job/15392998698.
These didn't fail CI because we run clippy on linux only.
2023-07-27 10:22:13 -04:00
Micha Reiser
6bf6646c5d Respect indent when measuring with MeasureMode::AllLines (#6120) 2023-07-27 10:22:13 -04:00
konsti
9574ff3dc7 Unbreak main (#6123)
This fixes main breaking due to two merges.
2023-07-27 10:22:13 -04:00
konsti
06d9ff9577 Don't format trailing comma for lambda arguments (#5946)
**Summary** lambda arguments don't have parentheses, so they shouldn't
get a magic trailing comma either. This fixes some unstable formatting

**Test Plan** Added a regression test.

89 (from previously 145) instances of unstable formatting remaining.

```
$ cargo run --bin ruff_dev --release -- format-dev --stability-check --error-file formatter-ecosystem-errors.txt --multi-project target/checkouts > formatter-ecosystem-progress.txt
$ rg "Unstable formatting" target/formatter-ecosystem-errors.txt | wc -l
89
```

Closes #5892
2023-07-27 10:22:13 -04:00
Micha Reiser
40f54375cb Pull in RustPython parser (#6099) 2023-07-27 09:29:11 +00:00
Victor Hugo Gomes
86539c1fc5 [flake8-pyi] Implement PYI046 (#6098)
## Summary
Checks for the presence of unused private `typing.Protocol` definitions.

ref #848 

## Test Plan

Snapshots and manual runs of flake8.
2023-07-27 02:34:56 +00:00
rembridge
d04367a042 call-datetime-without-tzinfo comment (#6105)
## Summary

Updated doc comment for `call_datetime_without_tzinfo.rs`. Online docs
also benefit from this update.

## Test Plan

Checked docs via
[mkdocs](389fe13c93/CONTRIBUTING.md (L267-L296))
2023-07-26 23:21:03 +00:00
Simon Brugman
ffdd653c54 [flake8-use-pathlib] Implement glob (PTH207) (#5939)
Discovered that the usage of `glob.glob` is
[widespread](https://grep.app/search?current=7&q=glob.glob%28&filter%5Blang%5D%5B0%5D=Python)
when working on the previous lints for `flake8-use-pathlib`.
2023-07-26 23:15:05 +00:00
rembridge
132f07c27b whitespace-before-parameters comment (#6103) 2023-07-26 23:01:47 +00:00
Victor Hugo Gomes
c0dbcb3434 [flake8-pyi] Implement PYI018 (#6018)
## Summary

Check for unused private `TypeVar`. See [original
implementation](2a86db8271/pyi.py (L1958)).

```
$ flake8 --select Y018 crates/ruff/resources/test/fixtures/flake8_pyi/PYI018.pyi

crates/ruff/resources/test/fixtures/flake8_pyi/PYI018.pyi:4:1: Y018 TypeVar "_T" is not used
crates/ruff/resources/test/fixtures/flake8_pyi/PYI018.pyi:5:1: Y018 TypeVar "_P" is not used
```

```
$ ./target/debug/ruff --select PYI018 crates/ruff/resources/test/fixtures/flake8_pyi/PYI018.pyi --no-cache

crates/ruff/resources/test/fixtures/flake8_pyi/PYI018.pyi:4:1: PYI018 TypeVar `_T` is never used
crates/ruff/resources/test/fixtures/flake8_pyi/PYI018.pyi:5:1: PYI018 TypeVar `_P` is never used
Found 2 errors.
```
In the file `unused_private_type_declaration.rs`, I'm planning to add
other rules that are similar to `PYI018` like the `PYI046`, `PYI047` and
`PYI049`.

ref #848

## Test Plan

Snapshots and manual runs of flake8.
2023-07-26 22:56:15 +00:00
Victor Hugo Gomes
788643f718 Add "--select E402" to example snippet in CONTRIBUTING.md (#6108)
## Summary
In Ruff only a subset of rules are enabled by default. This change
change aims to clarify that when adding a new rule, you must explicitly
use the `--select name_of_rule` command to ensure the rule gets
executed.

This was talked about on Discord a while back.

## Test Plan
Checked docs via mkdocs
2023-07-26 22:48:53 +00:00
Charlie Marsh
64a186272f Move utf8-encoding-declaration to token-based rules (#6110)
Closes #5979.
2023-07-26 22:42:37 +00:00
Charlie Marsh
8113615534 Add some additional documentation around import categorization (#6107)
Closes https://github.com/astral-sh/ruff/issues/5529.
2023-07-26 22:39:01 +00:00
konsti
ecf4058e52 Fix cargo test -p ruff (#6104) 2023-07-26 22:44:53 +02:00
Zanie Blue
2d2673f613 Add comment regarding class scope short circuit (#6101) 2023-07-26 14:55:05 -05:00
Harutaka Kawamura
564304eba2 Add PT001 documentation (#6023) 2023-07-26 18:05:25 +00:00
Harutaka Kawamura
5b8fc753ec Add PT024 documentation (#6026) 2023-07-26 13:48:37 -04:00
konsti
13f9a16e33 Rewrite placement logic (#6040)
## Summary
This is a rewrite of the main comment placement logic. `place_comment`
now has three parts:

- place own line comments
  - between branches
  - after a branch
- place end-of-line comments
  - after colon
  - after a branch
- place comments for specific nodes (that include module level comments)

The rewrite fixed three bugs: `class A: # trailing comment` comments now
stay end-of-line, `try: # comment` remains end-of-line and deeply
indented try-else-finally comments remain with the right nested
statement.

It will be much easier to give more alternative branches nodes since
this is abstracted away by `is_node_with_body` and the first/last child
helpers. Adding new node types can now be done by adding an entry to the
`place_comment` match. The code went from 1526 lines before #6033 to
1213 lines now.

It thinks it easier to just read the new `placement.rs` rather than
reviewing the diff.

## Test Plan

The existing fixtures staying the same or improving plus new ones for
the bug fixes.
2023-07-26 16:21:23 +00:00
Micha Reiser
2cf00fee96 Remove parser dependency from ruff-python-ast (#6096) 2023-07-26 17:47:22 +02:00
Harutaka Kawamura
99127243f4 Raise PTH201 for Path("") (#6095) 2023-07-26 09:22:46 -04:00
Harutaka Kawamura
77396c6f92 Fix SIM102 to handle indented elif (#6072)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

The `SIM102` auto-fix fails if `elif` is indented like this:

## Example

```python
def f():
    # SIM102
    if a:
        pass
    elif b:
        if c:
            d
```

```
> cargo run -p ruff_cli -- check --select SIM102 --fix a.py
...
error: Failed to fix nested if: Failed to extract statement from source
a.py:5:5: SIM102 Use a single `if` statement instead of nested `if` statements
Found 1 error.
```

## Test Plan

<!-- How was it tested? -->

New test
2023-07-26 14:37:32 +02:00
Micha Reiser
16e1737d1b Use cursor based lexer (#6012) 2023-07-26 11:32:26 +02:00
Dhruv Manilawala
025fa4eba8 Integrate the new Jupyter AST nodes in Ruff (#6086)
## Summary

This PR adds the implementation for the new Jupyter AST nodes i.e.,
`ExprLineMagic` and `StmtLineMagic`.

## Test Plan

Add test cases for `unparse` containing magic commands

resolves: #6087
2023-07-26 08:20:30 +00:00
Micha Reiser
1fdadee59c playground: Persist source and panel (#6071) 2023-07-26 07:55:59 +02:00
Charlie Marsh
c8ee357613 Remove relative import handling from BindingKind::Import case (#6084)
## Summary

Only `ImportFrom` imports can be relative, this is just unused.
2023-07-26 00:17:41 -04:00
Harutaka Kawamura
96d2ca0bda Allow pytest.raises body to contain a single func or class definition (#6083) 2023-07-25 23:45:57 -04:00
Harutaka Kawamura
62f821daaa Avoid raising PT012 for simple with statements (#6081) 2023-07-26 01:43:31 +00:00
Noah Jenner
9dfe484472 Modify PyPA classifiers and Shields.io badge URLs (#6082)
## Summary

Updated `pyproject.toml` classifiers from `"Development Status :: 4 -
Beta"` to `"Development Status :: 5 - Production/Stable"` to reflect the
transition from Beta to Full Release.
Updated the `README.md` to use `.com/astral-sh/ruff/...` instead of
`.com/charliermarsh/ruff/...` in Shields.io badges to reflect the
transition to a company.

## Test Plan

Utilized the official PyPA classifiers list (located at:
https://pypi.org/classifiers/)
Previewed the markdown file in different browsers on Github to ensure
all badges and logos still render properly.
2023-07-26 01:25:46 +00:00
Tom Kuson
da33c26238 Ignore explicit-string-concatenation on single line (#6028)
## Summary

Ignore `explicit-string-concatenation` on single line.

Closes #5332.

## Test Plan

`cargo test`
2023-07-25 19:20:29 -04:00
rembridge
8c80bfa7da tab indentation comment (#6079)
## Summary

Updated doc comment for `tab_indentation.rs`. Online docs also benefit
from this update.

## Test Plan

Checked docs via
[mkdocs](389fe13c93/CONTRIBUTING.md (L267-L296))
2023-07-25 23:14:43 +00:00
Zanie Blue
389fe13c93 Implement visitation of type aliases and parameters (#5927)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

Part of #5062 
Requires https://github.com/astral-sh/RustPython-Parser/pull/32

Adds visitation of type alias statements and type parameters in class
and function definitions.

Duplicates tests for `PreorderVisitor` into `Visitor` with new
snapshots. Testing required node implementations for the `TypeParam`
enum, which is a chunk of the diff and the reason we need `Ranged`
implementations in
https://github.com/astral-sh/RustPython-Parser/pull/32.

## Test Plan

<!-- How was it tested? -->

Adds unit tests with snapshots.
2023-07-25 17:11:26 +00:00
Zanie Blue
3000a47fe8 Include file permissions in key for cached files (#5901)
Reimplements https://github.com/astral-sh/ruff/pull/3104
Closes https://github.com/astral-sh/ruff/issues/5726

Note that we will generate the hash for a cache key twice in normal
operation. Once to check for the cached item and again to update the
cache. We could optimize this by generating the hash once in
`diagnostics::lint_file` and passing the `u64` into `get` and `update`.
We'd probably want to wrap it in a `CacheKeyHash` enum for type safety.

## Test plan

Unit tests for Windows and Unix.

Manual test with case from issue

```
❯ touch fake.py
❯ chmod +x fake.py
❯ ./target/debug/ruff --select EXE fake.py
fake.py:1:1: EXE002 The file is executable but no shebang is present
Found 1 error.
❯ chmod -x fake.py
❯ ./target/debug/ruff --select EXE fake.py
```
2023-07-25 17:06:47 +00:00
Charlie Marsh
cbf6085375 Fix example in D413 documentation (#6075)
See #6037.
2023-07-25 12:22:11 -04:00
Charlie Marsh
9171bd4c28 Avoid A003 violations for explicitly overridden methods (#6076)
## Summary

If a method is annotated with `@typing_extensions.override`, we should
avoid flagging A003 on it. This isn't part of the standard library yet,
but it's used to explicitly mark methods as overrides.
2023-07-25 16:21:23 +00:00
Chris Pryer
f5c69c1b34 Update ArgumentsParentheses usage (#6070) 2023-07-25 18:03:48 +02:00
Charlie Marsh
5f63b8bfb8 Ignore some common builtin overrides on standard library subclasses (#6074)
## Summary

If a user subclasses `threading.Event`, e.g. with:

```python
from threading import Event


class CustomEvent(Event):
    def set(self) -> None:
        ...
```

They no control over the method name (`set`). This PR allows
`threading.Event#set` and `logging.Filter#filter` overrides, and avoids
flagging A003 in such cases. Ideally, we'd avoid flagging all overridden
methods, but... that's a lot more difficult, and this is at least
_better_ than what we do now.

Closes https://github.com/astral-sh/ruff/issues/6057.

Closes https://github.com/astral-sh/ruff/issues/5956.
2023-07-25 15:54:34 +00:00
Charlie Marsh
c996b614fe Set default max-complexity to 10 for empty McCabe settings (#6073)
Closes https://github.com/astral-sh/ruff/issues/6058.
2023-07-25 15:38:19 +00:00
Ville Skyttä
670db1db4b pycodestyle.max-doc-length doc updates (#6052) 2023-07-25 15:34:26 +00:00
Charlie Marsh
242cbd966d Perform lint rule analysis after subtree traversal (#6045)
## Summary

This PR modifies the order of operations in our AST checker. Previously,
we ran our analysis rules first, then bound names and traversed over the
subtrees. Now, after a series of refactors, we can invert the order: do
the subtree traversal and model-building _first_, then run rules.

The nice thing about this change is that when we go to analyze, e.g., a
function call node, we'll already have traversed any of the constituent
`Expr::Name` nodes... So if we store the resolution of all names when do
the traversal, we can avoid having to do any expensive work in
`resolve_call_path`.

## Test Plan

Clean run of the snapshot tests, and hopefully the ecosystem checks too!
2023-07-25 09:05:44 -04:00
konsti
e7f228f781 Placement refactor (#6034)
## Summary

This PR is a refactoring of placement.rs. The code got more consistent,
some comments were updated and some dead code was removed or replaced
with debug assertions. It also contains a bugfix for the placement of
end-of-branch comments with nested bodies inside try statements that
occurred when refactoring the nested body loop.

## Test Plan

The existing test cases don't change. I added a couple of cases that i
think should be tested but weren't, and a regression test for the bugfix
2023-07-25 11:49:05 +02:00
Paul Mairo
51d8fc1f30 Update contributing.md with where to run ruff from (#6048)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

As of right now, the instructions don't specify where to run ruff from
after cloning the repository this is to address that. Super trivial
change, but helpful for real newbies I think.
2023-07-24 19:44:55 -04:00
Charlie Marsh
ed72c027a3 Replace NoHashHasher usages with FxHashMap (#6049)
## Summary

I had always assumed that `NoHashHasher` would be faster when using
integer keys, but benchmarking shows otherwise:

```
linter/default-rules/numpy/globals.py
                        time:   [66.544 µs 66.606 µs 66.678 µs]
                        thrpt:  [44.253 MiB/s 44.300 MiB/s 44.342 MiB/s]
                 change:
                        time:   [-0.1843% +0.1087% +0.3718%] (p = 0.46 > 0.05)
                        thrpt:  [-0.3704% -0.1086% +0.1847%]
                        No change in performance detected.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild
linter/default-rules/pydantic/types.py
                        time:   [1.3787 ms 1.3811 ms 1.3837 ms]
                        thrpt:  [18.431 MiB/s 18.466 MiB/s 18.498 MiB/s]
                 change:
                        time:   [-0.4827% -0.1074% +0.1927%] (p = 0.56 > 0.05)
                        thrpt:  [-0.1924% +0.1075% +0.4850%]
                        No change in performance detected.
linter/default-rules/numpy/ctypeslib.py
                        time:   [624.82 µs 625.96 µs 627.17 µs]
                        thrpt:  [26.550 MiB/s 26.601 MiB/s 26.650 MiB/s]
                 change:
                        time:   [-0.7071% -0.4908% -0.2736%] (p = 0.00 < 0.05)
                        thrpt:  [+0.2744% +0.4932% +0.7122%]
                        Change within noise threshold.
linter/default-rules/large/dataset.py
                        time:   [3.1585 ms 3.1634 ms 3.1685 ms]
                        thrpt:  [12.840 MiB/s 12.861 MiB/s 12.880 MiB/s]
                 change:
                        time:   [-1.5338% -1.3463% -1.1476%] (p = 0.00 < 0.05)
                        thrpt:  [+1.1610% +1.3647% +1.5577%]
                        Performance has improved.

linter/all-rules/numpy/globals.py
                        time:   [140.17 µs 140.37 µs 140.58 µs]
                        thrpt:  [20.989 MiB/s 21.020 MiB/s 21.051 MiB/s]
                 change:
                        time:   [-0.1066% +0.3140% +0.7479%] (p = 0.14 > 0.05)
                        thrpt:  [-0.7423% -0.3130% +0.1067%]
                        No change in performance detected.
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) high mild
  1 (1.00%) high severe
linter/all-rules/pydantic/types.py
                        time:   [2.7030 ms 2.7069 ms 2.7112 ms]
                        thrpt:  [9.4064 MiB/s 9.4216 MiB/s 9.4351 MiB/s]
                 change:
                        time:   [-0.6721% -0.4874% -0.2974%] (p = 0.00 < 0.05)
                        thrpt:  [+0.2982% +0.4898% +0.6766%]
                        Change within noise threshold.
Found 14 outliers among 100 measurements (14.00%)
  12 (12.00%) high mild
  2 (2.00%) high severe
linter/all-rules/numpy/ctypeslib.py
                        time:   [1.4709 ms 1.4727 ms 1.4749 ms]
                        thrpt:  [11.290 MiB/s 11.306 MiB/s 11.320 MiB/s]
                 change:
                        time:   [-1.1617% -0.9766% -0.8094%] (p = 0.00 < 0.05)
                        thrpt:  [+0.8160% +0.9862% +1.1754%]
                        Change within noise threshold.
Found 12 outliers among 100 measurements (12.00%)
  9 (9.00%) high mild
  3 (3.00%) high severe
linter/all-rules/large/dataset.py
                        time:   [5.8086 ms 5.8163 ms 5.8240 ms]
                        thrpt:  [6.9854 MiB/s 6.9946 MiB/s 7.0038 MiB/s]
                 change:
                        time:   [-1.5651% -1.3536% -1.1584%] (p = 0.00 < 0.05)
                        thrpt:  [+1.1720% +1.3721% +1.5900%]
                        Performance has improved.
```

My guess is that `NoHashHasher` underperforms because the keys are not
randomly distributed...

Anyway, it's a ~1% (significant) performance gain on some of the above,
plus we get to remove a dependency.
2023-07-24 23:41:57 +00:00
Charlie Marsh
b7e7346081 Remove empty newline in deferred_for_loops (#6046)
Trivial change but none of the others have this empty newline.
2023-07-24 21:59:32 +00:00
Charlie Marsh
d35b5248ea Tweak lambda rule to use annotations rather than shadowing (#6044)
## Summary

This PR ensures that we can retain the current behavior even after we
reorder the visitor a bit, by looking for annotated lambdas rather than
"is the name bound to anything?", since if we visit the name before we
run this rule, it'll _always_ be bound. (This check is already a bit
flawed -- in truth, we should probably run this rule deferred so that we
can reliably detect shadowing.)
2023-07-24 21:39:02 +00:00
Charlie Marsh
c535e10fff Move comprehension rules into shared analyze method (#6042) 2023-07-24 21:18:45 +00:00
Charlie Marsh
c3ecdb8783 Fix Arg typo (#6041) 2023-07-24 21:16:28 +00:00
Charlie Marsh
242df67cbf Move lint rules out of checkers/ast/mod.rs (#5957)
## Summary

This PR attempts to draw some basic separation between the `Checker`'s
traversal responsibilities (traversing the AST, building the semantic
model) and its calling-out-to-lint-rule responsibilities. It doesn't try
to introduce any sophisticated API. Instead, it just moves all of the
lint rule calls out of `checkers/ast/mod.rs` and into methods in a new
`analyze` module. (There are four remaining lint rules in `Checker`, but
I'll remove those in future PRs.)

I'm not trying to "solve" our lint rule API here. Instead, I'm trying to
make two improvements:

1. `checkers/ast/mod.rs` has just gotten way too large, and people work
in it all the time. Prior to this PR, it was 5.5k lines, which led to
significant lags in my editor and made it really hard to reason about
the parts that are _actually_ important. (I like big files, but this one
crossed the line for me.) Now, it's < 2,000 lines, and the code is much
more focused.
2. I want to avoid accidentally adding lint rules in the "wrong" parts
of the traversal. By confining lint rule invocations to these "analyze"
calls, we'll avoid (e.g.) putting them in the binding phase.
2023-07-24 19:20:10 +00:00
Charlie Marsh
776d598738 Move flake8-executable rules out of physical lines checker (#6039)
## Summary

These only need the token stream, and we always prefer token-based to
physical line-based rules.

There are a few other changes snuck in here:

- Renaming the rule files to match the diagnostic names (likely an
error).
- The "leading whitespace before shebang" rule now works regardless of
where the comment occurs (i.e., if the shebang is on the second line,
and the first line is blank, we flag and remove that leading
whitespace).
2023-07-24 14:38:05 -04:00
konsti
7f3797185c Fix formatter with-statement after-as own line comment instability (#6033)
**Summary** Fix an instability in with statement formatter when there is
an own line comment as the `as`
```python
with (
    a as
    # bad comment
    b):
```

**Test Plan** Added the comment to the test cases.
2023-07-24 18:12:07 +00:00
konsti
a9f535997d Document formatter progress scripts (#6035)
## Summary

Add documentation to the formatter progress scripts

## Test Plan

n/a
2023-07-24 19:42:20 +02:00
Micha Reiser
fdb3c8852f Prefer breaking the implicit string concatenation over breaking before % (#5947) 2023-07-24 18:30:42 +02:00
Charlie Marsh
42d969f19f Add additional test cases for F823 (#6036)
Making some behavior explicit / codified. See:
https://github.com/astral-sh/ruff/issues/6029.
2023-07-24 15:49:48 +00:00
Charlie Marsh
62ffc773de Avoid treating Literal members as expressions with __future__ (#6032)
Closes https://github.com/astral-sh/ruff/issues/6030.
2023-07-24 15:09:37 +00:00
Charlie Marsh
6feb3fcc1b Ignore end-of-line comments when dirtying if-with-same-arms branches (#6031)
## Summary

Closes https://github.com/astral-sh/ruff/issues/6025 (which contains a
more thorough description of the issue). Previously, the `# noqa` here
was being marked as unused, but removing it raised `SIM114`:

```python
def foo():
    a = True
    b = False
    if a > b:  # noqa: SIM114
        return 3
    elif a == b:
        return 3
```
2023-07-24 10:59:58 -04:00
Chris Pryer
8eadacda33 Update TupleParentheses usage (#5810) 2023-07-24 14:44:36 +00:00
konsti
8a7dcb794b Add formatter progress tracking to CI (#5919)
**Summary** Add a formatter progress testing script to CI. This script
will 1) print the black compability on each run 2) catch regressions wrt
to formatter stability, emitting invalid syntax and other kinds of bugs
(e.g. #5917) before they land on main 3) have an additional layer of
real world tests when implementing new nodes or other new formatter
code.

This is currently a bash script, i'm not sure if we want to keep it that
way, or switch to e.g. the regular ecosystem scripts. The output
separation of `format_dev` could also use some polishing. We should also
consider pinning commits so we don't get spurious regression when they
change their code.

**Test Plan** The script extends CI.
2023-07-24 09:12:42 +00:00
Luc Khai Hai
dfa81b6fe0 Format numeric constants (#5972)
Co-authored-by: Micha Reiser <micha@reiser.io>
2023-07-24 07:04:40 +00:00
Charlie Marsh
33196f1859 Fix logging rules with whitespace around dot (#6022)
## Summary

Attempting to fix, e.g., `logging . warn("Hello World!")` was causing a
syntax error.
2023-07-24 05:14:48 +00:00
Charlie Marsh
0d94337b96 Avoid allocations in SimpleCallArgs (#6021)
## Summary

My intuition is that it's faster to do these checks as-needed rather
than allocation new hash maps and vectors for the arguments. (We
typically only query once anyway.)
2023-07-24 04:55:37 +00:00
Charlie Marsh
f9726af4ef Allow specification of logging.Logger re-exports via logger-objects (#5750)
## Summary

This PR adds a `logger-objects` setting that allows users to mark
specific symbols a `logging.Logger` objects. Currently, if a `logger` is
imported, we only flagged it as a `logging.Logger` if it comes exactly
from the `logging` module or is `flask.current_app.logger`.

This PR allows users to mark specific loggers, like
`logging_setup.logger`, to ensure that they're covered by the
`flake8-logging-format` rules and others.

For example, if you have a module `logging_setup.py` with the following
contents:

```python
import logging

logger = logging.getLogger(__name__)
```

Adding `"logging_setup.logger"` to `logger-objects` will ensure that
`logging_setup.logger` is treated as a `logging.Logger` object when
imported from other modules (e.g., `from logging_setup import logger`).

Closes https://github.com/astral-sh/ruff/issues/5694.
2023-07-24 00:38:20 -04:00
Tom Kuson
727153cf45 [pylint] Impement self-assigning-variable (W0127) (#6015)
## Summary

Implements Pylint rule [`self-assigning-variable`
(`W0127`)](https://pylint.pycqa.org/en/latest/user_guide/messages/warning/self-assigning-variable.html)
as `self-assigning-variable` (`PLW0127`). Includes documentation.
Related to #970.

## Test Plan

`cargo test`
2023-07-24 02:27:09 +00:00
Charlie Marsh
574c0e0105 Use match instead of phf for confusable lookup (#5953)
I don't know whether we want to make this change but here's some data...

Binary size:

- `main`: 30,384
- `charlie/match-phf`: 30,416

llvm-lines:

- `main`: 1,784,148
- `charlie/match-phf`: 1,789,877

llvm-lines and binary size are both unchanged (or, by < 5) when moving
from `u8` to `u32` return types, and even when moving to `char` keys and
values. I didn't expect this, but I'm not very knowledgable on this
topic.

Performance:

```
Confusables/match/src   time:   [4.9102 µs 4.9352 µs 4.9777 µs]
                        change: [+1.7469% +2.2421% +2.8710%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 12 outliers among 100 measurements (12.00%)
  2 (2.00%) low mild
  4 (4.00%) high mild
  6 (6.00%) high severe
Confusables/match-with-skip/src
                        time:   [2.0676 µs 2.0945 µs 2.1317 µs]
                        change: [+0.9384% +1.6000% +2.3920%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 8 outliers among 100 measurements (8.00%)
  3 (3.00%) high mild
  5 (5.00%) high severe
Confusables/phf/src     time:   [31.087 µs 31.188 µs 31.305 µs]
                        change: [+1.9262% +2.2188% +2.5496%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 15 outliers among 100 measurements (15.00%)
  3 (3.00%) low mild
  6 (6.00%) high mild
  6 (6.00%) high severe
Confusables/phf-with-skip/src
                        time:   [2.0470 µs 2.0486 µs 2.0502 µs]
                        change: [-0.3093% -0.1446% +0.0106%] (p = 0.08 > 0.05)
                        No change in performance detected.
Found 4 outliers among 100 measurements (4.00%)
  2 (2.00%) high mild
  2 (2.00%) high severe
```

The `-with-skip` variants add our optimization which first checks
whether the character is ASCII. So `match` is way, way faster than PHF,
but it tends not to matter since almost all source code is ASCII anyway.
2023-07-24 02:23:36 +00:00
Dhruv Manilawala
700c816fd5 Make TRY201 always autofixable (#6008)
## Summary

Make `TRY201` always autofiable.

## Test Plan

1. `cargo test`
2. `cargo insta review`

ref:
https://github.com/astral-sh/ruff/issues/4333#issuecomment-1646359788
2023-07-24 02:23:15 +00:00
Tom Kuson
3b56f6d616 [pylint] Implement subprocess-popen-preexec-fn (W1509) (#5978)
## Summary

Implements Pylint rule [`subprocess-popen-preexec-fn`
(`W1509`)](https://pylint.pycqa.org/en/latest/user_guide/messages/warning/subprocess-popen-preexec-fn.html)
as `subprocess-popen-preexec-fn` (`PLW1509`). Includes documentation.
Related to #970.

## Test Plan

`cargo test`
2023-07-24 02:06:19 +00:00
Harutaka Kawamura
110fa804ff Add PT016 documentation (#6005) 2023-07-23 21:52:48 -04:00
Harutaka Kawamura
2b9c22de0f Add a unit test for python-file-like directory exclusion (#5997) 2023-07-24 01:50:39 +00:00
Harutaka Kawamura
51ebff7e41 Add PT010 doc (#6010) 2023-07-24 01:43:18 +00:00
Dhruv Manilawala
742f615792 Add support for int, float, bool in UP018 (#6013)
## Summary

This pull request add supports for `int`, `float` and `bool` types in
`UP018`
rule to convert empty call to the default value of the type or remove
the call
if a value of the same type is provided as an argument.

## Test Plan

Added tests for `int`, `float` and `bool` types.

Partially resolves #5988
2023-07-23 21:39:43 -04:00
Harutaka Kawamura
95e6258d5d Add PT020 doc (#6011) 2023-07-23 21:37:03 -04:00
Dhruv Manilawala
5dbb4dd823 Update docs for ANN401 (#6009)
Part of #5803
2023-07-23 16:15:04 +00:00
konsti
46f8961292 Formatter: Add EmptyWithDanglingComments helper (#5951)
**Summary** Add a `EmptyWithDanglingComments` format helper that formats
comments inside empty parentheses, brackets or curly braces. Previously,
this was implemented separately, and partially incorrectly, for each use
case.

Empty `()`, `[]` and `{}` are special because there can be dangling
comments, and they can be in
two positions:
```python
x = [  # end-of-line
    # own line
]
```
These comments are dangling because they can't be assigned to any
element inside as they would
in all other cases.

**Test Plan** Added a regression test.

145 (from previously 149) instances of unstable formatting remaining.

```
$ cargo run --bin ruff_dev --release -- format-dev --stability-check --error-file formatter-ecosystem-errors.txt --multi-project target/checkouts > formatter-ecosystem-progress.txt
$ rg "Unstable formatting" target/formatter-ecosystem-errors.txt | wc -l
145
```
2023-07-23 14:32:16 +02:00
Simon Brugman
f886b58c92 [flake8-use-pathlib] Implement os-sep-split (PTH206) (#5936)
Implements
https://github.com/astral-sh/ruff/issues/5905#issuecomment-1644822548

---------

Co-authored-by: konsti <konstin@mailbox.org>
2023-07-23 12:22:26 +02:00
Charlie Marsh
057faabcdd Use Flags::intersects rather than Flags::contains (#6007)
## Summary

This is equivalent for a single flag, but I think it's more likely to be
correct when the bitflags are modified -- the primary reason being that
we sometimes define flags as the union of other flags, e.g.:

```rust
const ANNOTATION = Self::TYPING_ONLY_ANNOTATION.bits() | Self::RUNTIME_ANNOTATION.bits();
```

In this case, `flags.contains(Flag::ANNOTATION)` requires that _both_
flags in the union are set, whereas `flags.intersects(Flag::ANNOTATION)`
requires that _at least one_ flag is set.
2023-07-23 02:59:31 +00:00
Charlie Marsh
0bb175f7f6 Store flags rather than ExecutionContext on references (#6006) 2023-07-23 02:54:39 +00:00
Charlie Marsh
4b2ec7d562 Move runtime execution context into add_reference calls (#6003) 2023-07-23 02:37:51 +00:00
Charlie Marsh
4aac801277 Fix context-to-model references in SemanticModel documentation (#6004) 2023-07-23 02:32:23 +00:00
Charlie Marsh
45a24912a6 Remove extra error! call (#6002) 2023-07-23 02:29:06 +00:00
Simon Brugman
3914fcb7ca Extend SIM118 with not in (#5995)
Closes https://github.com/astral-sh/ruff/issues/5989

Tracking issue https://github.com/astral-sh/ruff/issues/1348
2023-07-23 01:46:21 +00:00
Charlie Marsh
6d58b773b1 Use simple text matching for type: ignore detection (#5999)
Closes #5980.
2023-07-23 01:45:28 +00:00
Tom Kuson
e7f5121922 Extends B002 to detect unary prefix decrement operators (#5998)
## Summary

Extends `B002` to detect unary decrement prefix operators.

Closes #5992.

## Test Plan

`cargo test`
2023-07-23 01:40:49 +00:00
Charlie Marsh
1776cbd2e2 Move blanket noqa and blanket type: ignore rules into token-based checker (#5996)
Closes https://github.com/astral-sh/ruff/issues/5981.
2023-07-22 21:22:48 -04:00
Charlie Marsh
71f1643eda Use memchr for invalid-escape-sequence (#5994) 2023-07-22 20:57:36 -04:00
Tom Kuson
74dc137b30 Use find_keyword helper function in more places (#5993)
## Summary

Use the `find_keyword` helper function instead of reimplementing it.

Follows on from #5983 by doing a different search.

## Test Plan

`cargo test`
2023-07-22 20:27:24 -04:00
Harutaka Kawamura
97e31cad2f Fix F507 false positive (#5986)
## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

F507 should not be raised when the right-hand side value is a non-tuple
object.

```python
'%s' % (1, 2, 3)  # throws
'%s' % [1, 2, 3]  # doesn't throw
'%s' % {1, 2, 3}  # doesn't throw
```
2023-07-22 18:42:44 +00:00
Simon Brugman
ed7d2b8a3d Do not raise SIM105 for non-exceptions (#5985)
Closes https://github.com/astral-sh/ruff/issues/5977

Added a test case from `refurb`
2023-07-22 18:36:46 +00:00
Tom Kuson
c7e4c58181 Use find_keyword helper function (#5983)
## Summary

Use `find_keyword` helper function instead of reimplementing it.

## Test Plan

`cargo test`
2023-07-22 14:09:30 -04:00
Charlie Marsh
6ff566f2c1 Flag [ as an invalid noqa suffix (#5982)
Closes https://github.com/astral-sh/ruff/issues/5960.
2023-07-22 10:16:28 -04:00
Charlie Marsh
32773e8309 Move locator, stylist, and friends better getters (#5968)
## Summary

Rather than exposing these as public fields, use getters, similar to
`semantic()`.
2023-07-22 09:37:24 -04:00
Harutaka Kawamura
050f5953f8 Avoid raising UP032 if format call arguments contain multiline expressions (#5971)
## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

Fix a regression introduced by
https://github.com/astral-sh/ruff/pull/5638. A multiline expression
can't be safely inserted into a format field.

### Example

```
> cat a.py
"{}".format(
    [
        1,
        2,
        3,
    ]
)

> cargo run -p ruff_cli -- check a.py --no-cache --select UP032 --fix
    Finished dev [unoptimized + debuginfo] target(s) in 0.07s
     Running `target/debug/ruff check a.py --no-cache --select UP032 --fix`
error: Autofix introduced a syntax error in `a.py` with rule codes UP032: EOL while scanning string literal at byte offset 5
---
f"{[
        1,
        2,
        3,
    ]}"

---
a.py:1:1: UP032 Use f-string instead of `format` call
Found 1 error.
```


## Test Plan

New test cases
2023-07-22 09:37:08 -04:00
Alex Waygood
aba340a177 Fix typo in PYI056 docs (#5973)
The current "use instead" code would correctly be rejected by any type
checker worth its salt ;)
2023-07-22 09:10:38 -04:00
Victor Hugo Gomes
33657d3a1c [flake8-pyi] Implement PYI056 (#5959)
## Summary

Checks that `append`, `extend` and `remove` methods are not called on
`__all__`. See [original
implementation](2a86db8271/pyi.py (L1133-L1138)).

```
$ flake8 --select Y026 crates/ruff/resources/test/fixtures/flake8_pyi/PYI056.pyi

crates/ruff/resources/test/fixtures/flake8_pyi/PYI056.pyi:3:1: Y056 Calling ".append()" on "__all__" may not be supported by all type checkers (use += instead)
crates/ruff/resources/test/fixtures/flake8_pyi/PYI056.pyi:4:1: Y056 Calling ".extend()" on "__all__" may not be supported by all type checkers (use += instead)
crates/ruff/resources/test/fixtures/flake8_pyi/PYI056.pyi:5:1: Y056 Calling ".remove()" on "__all__" may not be supported by all type checkers (use += instead)
```

```
$ ./target/debug/ruff --select PYI026 crates/ruff/resources/test/fixtures/flake8_pyi/PYI056.pyi --no-cache

crates/ruff/resources/test/fixtures/flake8_pyi/PYI056.pyi:3:1: PYI056 Calling ".append()" on "__all__" may not be supported by all type checkers (use += instead)
crates/ruff/resources/test/fixtures/flake8_pyi/PYI056.pyi:4:1: PYI056 Calling ".extend()" on "__all__" may not be supported by all type checkers (use += instead)
crates/ruff/resources/test/fixtures/flake8_pyi/PYI056.pyi:5:1: PYI056 Calling ".remove()" on "__all__" may not be supported by all type checkers (use += instead)
Found 3 errors.
```

ref #848

## Test Plan

Snapshots and manual runs of flake8.
2023-07-22 04:25:54 +00:00
Charlie Marsh
45318d08b7 Always compute runtime annotations for flake8-type-checking rules (#5967)
## Summary

These are skipped as an optimization, but it feels kind of unnecessary
and makes the code a bit more confusing than is worthwhile.
(non-`strict` is also by far the more popular setting, and the default.)
2023-07-21 23:53:33 -04:00
Charlie Marsh
86b6a3e1ad Remove nested f-string flag (#5966)
## Summary

Not worth taking up a slot in the semantic model flags.
2023-07-21 22:51:37 -04:00
Charlie Marsh
f5a2fb5b5d Bump version to 0.0.280 (#5965) 2023-07-21 22:36:13 -04:00
Charlie Marsh
94a004ee9c Avoid collapsing elif and else branches during import sorting (#5964)
## Summary

I ran into this in the wild. It looks like Ruff will collapse the `else`
and `elif` branches here (i.e., it doesn't recognize that they're too
independent import blocks):

```python
if "sdist" in cmds:
    _sdist = cmds["sdist"]
elif "setuptools" in sys.modules:
    from setuptools.command.sdist import sdist as _sdist
else:
    from setuptools.command.sdist import sdist as _sdist
    from distutils.command.sdist import sdist as _sdist
```

Likely fallout from the `elif_else_branches` refactor.
2023-07-22 02:18:02 +00:00
Tom Kuson
aaf7f362a1 Create snake_case file if linter is Pylint (#5948)
## Summary

The `add_rule.py` script would create a test case that pointed to a file
that didn't exist when the linter is set to `"pylint"`. This PR fixes
that.

## Test Plan

`python scripts/add_rule.py --name DoTheThing --prefix PL --code C0999
--linter pylint`
2023-07-21 22:13:43 -04:00
Charlie Marsh
2dcd9e2e9c Remove unnecessary check_deferred_assignments (#5963)
## Summary

These rules can just be included in the `check_deferred_scopes`.
2023-07-22 02:08:44 +00:00
Charlie Marsh
40e9884353 Move nonlocal-without-binding out of binding step (#5962) 2023-07-22 01:39:27 +00:00
Tom Kuson
9bbb0a5151 Fix typo in documentation (#5961)
## Summary

Close unclosed inline code block that was causing the text not to render
properly.

## Test Plan

`mkdocs serve`
2023-07-22 01:23:30 +00:00
Charlie Marsh
f1f89f2a7e Bump version to 0.0.279 (#5949) 2023-07-21 15:46:53 -04:00
konsti
196cc9b655 Fix RustPython rev to main branch (#5950)
**Summary** I accidentally merged earlier while the RustPython parser
rev was still pointing to the feature branch instead of to the merged
main. This make the rev point to the RustPython parser repo main again
2023-07-21 15:53:14 +00:00
konsti
972f9a9c15 Fix formatting lambda with empty arguments (#5944)
**Summary** Fix implemented in
https://github.com/astral-sh/RustPython-Parser/pull/35: Previously,
empty lambda arguments (e.g. `lambda: 1`) would get the range of the
entire expression, which leads to incorrect comment placement. Now empty
lambda arguments get an empty range between the `lambda` and the `:`
tokens.

**Test Plan** Added a regression test.

149 instances of unstable formatting remaining.

```
$ cargo run --bin ruff_dev --release -- format-dev --stability-check --error-file formatter-ecosystem-errors.txt --multi-project target/checkouts > formatter-ecosystem-progress.txt
$ rg "Unstable formatting" target/formatter-ecosystem-errors.txt | wc -l
149
```
2023-07-21 15:48:45 +02:00
qdegraaf
519dbdffaa Format ExprYield/ExprYieldFrom (#5921)
Co-authored-by: Micha Reiser <micha@reiser.io>
2023-07-21 12:07:51 +00:00
konsti
c3b506fca6 Add script to shrink all formatter errors (#5943)
**Summary** Add script to shrink all formatter errors: This started as a
fun idea and turned out really useful: This script gives us a single
Python file with all formatter stability errors. I want to keep it
around to occasionally update #5828 so I added it to the git.

**Test Plan** None, this is a helper script
2023-07-21 11:32:35 +02:00
konsti
f6b40a021f Document shrinking script (#5942)
**Summary** Document shrinking script: I thinks it's both in a good
enough state and valuable enough to document it's usage.
2023-07-21 11:32:26 +02:00
konsti
b56e8ad696 Document formatter error shrinking (#5915)
## Summary

**Don't minimize files that don't match in the first place** This adds a
sanity check to the minimizer script that the
input matches the condition (e.g. unstable formatting). Otherwise we run
through all checks with the whole file, which is extremely slow. It's
more reasonable for downstream usage to write an empty string to the
output file instead.
2023-07-21 11:32:12 +02:00
Charlie Marsh
03018896de Port over some fixes from #3747 (#5940) 2023-07-21 03:55:01 +00:00
Charlie Marsh
b3d31025b1 Remove some unnecessary lifetime annotations (#5938) 2023-07-21 02:42:17 +00:00
Dhruv Manilawala
29e5e4e0b5 Allow respect_gitignore when not in a git repo (#5937)
## Summary

Allow `respect_gitignore` even when not in a git repo

## Test Plan

Within the Ruff repository:

1. Renamed `.git` to `.hello-world`
2. Added `test.py` in root folder
3. Added `test.py` to `.gitignore`
4. Ran `cargo run --bin ruff -- check --no-cache --isolated --show-files
.` with
   and without `--respect-gitignore` flag

fixes: #5930
2023-07-20 22:35:08 -04:00
Simon Brugman
f7b156523a [flake8-use-pathlib] extend PTH118 with os.sep (#5935)
Closes https://github.com/astral-sh/ruff/issues/5905

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2023-07-21 01:36:02 +00:00
Simon Brugman
d62183b07d Add documentation for the pathlib rules (#5815)
Reviving https://github.com/astral-sh/ruff/pull/2348 step by step

Pt 1: docs

Tracking issue: https://github.com/astral-sh/ruff/issues/2646.
2023-07-21 01:02:22 +00:00
Charlie Marsh
5f2014b0b8 Expand RUF015 to include all expression types (#5767)
## Summary

We now allow RUF015 to fix cases like:

```python
list(range(10))[0]
list(x.y)[0]
list(x["y"])[0]
```

Further, we fix generators like:

```python
[i + 1 for i in x][0]
```

By rewriting to `next(iter(i + 1 for i in x))`.

I've retained the special-case that rewrites `[i for i in x][0]` to
`next(iter(x))`.

Closes https://github.com/astral-sh/ruff/issues/5764.
2023-07-20 20:08:08 -04:00
Tom Kuson
4e681070dc Close unclosed code block in documentation (#5934)
## Summary

Closes an unclosed code block such that the rule documentation renders
properly.

## Test Plan

`mkdocs serve -f mkdocs.generated.yml`
2023-07-20 23:18:16 +00:00
Micha Reiser
4759ffc994 Merge changed steps using files_yaml (#5923) 2023-07-20 23:18:13 +02:00
Charlie Marsh
bcec2f0c4c Move undefined-local into a post-model-building pass (#5928)
## Summary

Similar to #5852 and a bunch of related PRs -- trying to move rules that
rely on point-in-time semantic analysis to _after_ the semantic model
building.
2023-07-20 15:34:22 -04:00
qdegraaf
2cde9b8aa6 [flake8-pyi] Implement PYI017 (#5895)
## Summary

Implements `PYI017` or `Y017` from `flake8-pyi` plug-in. Mirrors
[upstream
implementation](ceab86d16b/pyi.py (L1039-L1048)).
It checks for any assignment with more than 1 target or an assignment to
anything other than a name, and raises a violation for these in stub
files.

Couldn't find a clear and concise explanation for why this is to be
avoided and what is preferred for attribute cases like:

```python
a.b = int
```
So welcome some input there, to learn and to finish up the docs.

## Test Plan

Added test cases from upstream plug-in in a fixture (both `.py` and
`.pyi`). Added a few more.

## Issue link

Refers: https://github.com/astral-sh/ruff/issues/848
2023-07-20 16:35:38 +00:00
Charlie Marsh
c948dcc203 Restore redefined-while-unused violations in classes (#5926)
## Summary

This is a regression from a recent refactor whereby we moved these
checks to a deferred pass.

Closes https://github.com/astral-sh/ruff/issues/5918.
2023-07-20 12:10:26 -04:00
Luc Khai Hai
b866cbb33d Improve slice formatting (#5922)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

- Remove space when start of slice is empty
- Treat unary op except `not` as simple expression

## Test Plan

Add some simple tests for unary op expressions in slice

Closes #5673
2023-07-20 15:05:18 +00:00
Micha Reiser
d351761f5d SimpleTokenizer: Fix infinite loop when lexing empty quotes (#5917) 2023-07-20 15:18:35 +02:00
Tom Kuson
ccc6bd5df0 Fix typo in documentation (#5914) 2023-07-20 13:06:28 +02:00
Micha Reiser
eeb8a5fe0a Avoid line break before for in comprehension if outer expression expands (#5912) 2023-07-20 10:07:22 +00:00
konsti
c2b7b46717 Extend shrinking script to also remove tokens and characters (#5898)
This shrinks a good bit more than previously, which was helpful for all
the formatter bugs. fwiw i treat this as a very ad-hoc script since it's
mainly my ecosystem bug processing companion.
2023-07-20 12:02:00 +02:00
Micha Reiser
6fd8574a0b Only run jobs if relevant files changed (#5908) 2023-07-20 10:01:08 +00:00
Micha Reiser
76e9ce6dc0 Fix SimpleTokenizer's backward lexing of # (#5878) 2023-07-20 11:54:18 +02:00
konsti
8c5f8a8aef Formatter: Small RParen refactoring (#5885)
## Summary

A bit more consistency inspired by
https://github.com/astral-sh/ruff/pull/5882#discussion_r1268182403

## Test Plan

Existing tests (refactoring)
2023-07-20 11:30:39 +02:00
konsti
92f471a666 Handle io errors gracefully (#5611)
## Summary

It can happen that we can't read a file (a python file, a jupyter
notebook or pyproject.toml), which needs to be handled and handled
consistently for all file types. Instead of using `Err` or `error!`, we
emit E602 with the io error as message and continue. This PR makes sure
we handle all three cases consistently, emit E602.

I'm not convinced that it should be possible to disable io errors, but
we now handle the regular case consistently and at least print warning
consistently.

I went with `warn!` but i can change them all to `error!`, too.

It also checks the error case when a pyproject.toml is not readable. The
error message is not very helpful, but it's now a bit clearer that
actually ruff itself failed instead vs this being a diagnostic.

## Examples

This is how an Err of `run` looks now:


![image](https://github.com/astral-sh/ruff/assets/6826232/890f7ab2-2309-4b6f-a4b3-67161947cc83)

With an unreadable file and `IOError` disabled:


![image](https://github.com/astral-sh/ruff/assets/6826232/fd3d6959-fa23-4ddf-b2e5-8d6022df54b1)

(we lint zero files but count files before linting not during so we exit
0)

I'm not sure if it should (or if we should take a different path with
manual ExitStatus), but this currently also triggers when `files` is
empty:


![image](https://github.com/astral-sh/ruff/assets/6826232/f7ede301-41b5-4743-97fd-49149f750337)

## Test Plan

Unix only: Create a temporary directory with files with permissions
`000` (not readable by the owner) and run on that directory. Since this
breaks the assumptions of most of the test code (single file, `ruff`
instead of `ruff_cli`), the test code is rather cumbersome and looks a
bit misplaced; i'm happy about suggestions to fit it in closer with the
other tests or streamline it in other ways. I added another test for
when the entire directory is not readable.
2023-07-20 11:30:14 +02:00
Micha Reiser
029fe05a5f Playground: Fix escaped quotes handling (#5906)
Co-authored-by: konsti <konstin@mailbox.org>
2023-07-20 09:25:27 +00:00
Chris Pryer
9e32585cb1 Use dangling_node_comments in lambda formatting (#5903) 2023-07-20 08:52:32 +02:00
Charlie Marsh
fe7505b738 Move undefined deletions into post-model-building pass (#5904)
## Summary

Similar to #5902, but for undefined names in deletions (e.g., `del x`
where `x` is unbound).
2023-07-20 05:14:46 +00:00
Tom Kuson
266e684192 Add flake8-fixme documentation (#5868)
## Summary

Completes documentation for the `flake8-fixme` (`FIX`) ruleset. Related
to #2646.

Tweaks the violation message. For example,

```
FIX001 Line contains FIXME
```

becomes

```
FIX001 Line contains FIXME, consider resolving the issue
```

This is because the previous message was unclear if it was warning
against the use of FIXME tags per se, or the code the FIXME tag was
annotating.


## Test Plan

`cargo test && python scripts/check_docs_formatted.py`
2023-07-20 02:21:55 +00:00
Simon Brugman
4bba0bcab8 [flake8-use-pathlib] Implement os-path-getsize and os-path-get(a|m|c)-time (PTH202-205) (#5835)
Reviving https://github.com/astral-sh/ruff/pull/2348 step by step

Pt 3. implement detection for:
- `os.path.getsize`
- `os.path.getmtime`
- `os.path.getctime`
- `os.path.getatime`
2023-07-20 02:05:13 +00:00
Simon Brugman
d35cb6942f [flake8-use-pathlib] Implement path-constructor-default-argument (PTH201) (#5833)
Reviving https://github.com/astral-sh/ruff/pull/2348 step by step

Pt 2. PTH201: Path Constructor Default Argument

- rule originates from `refurb`:
https://github.com/charliermarsh/ruff/issues/1348
- Using PTH201 rather than FURBXXX to keep all pathlib logic together
2023-07-20 01:50:54 +00:00
Victor Hugo Gomes
a37d91529b [flake8-pyi] Implement PYI026 (#5844)
## Summary
Checks for `typehint.TypeAlias` annotation in type aliases. See
[original
source](https://github.com/PyCQA/flake8-pyi/blob/main/pyi.py#L1085).
```
$ flake8 --select Y026 crates/ruff/resources/test/fixtures/flake8_pyi/PYI026.pyi
crates/ruff/resources/test/fixtures/flake8_pyi/PYI026.pyi:4:1: Y026 Use typing_extensions.TypeAlias for type aliases, e.g. "NewAny: TypeAlias = Any"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI026.pyi:5:1: Y026 Use typing_extensions.TypeAlias for type aliases, e.g. "OptinalStr: TypeAlias = typing.Optional[str]"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI026.pyi:6:1: Y026 Use typing_extensions.TypeAlias for type aliases, e.g. "Foo: TypeAlias = Literal['foo']"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI026.pyi:7:1: Y026 Use typing_extensions.TypeAlias for type aliases, e.g. "IntOrStr: TypeAlias = int | str"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI026.pyi:8:1: Y026 Use typing_extensions.TypeAlias for type aliases, e.g. "AliasNone: TypeAlias = None"
```

```
$ ./target/debug/ruff --select PYI026 crates/ruff/resources/test/fixtures/flake8_pyi/PYI026.pyi --no-cache
crates/ruff/resources/test/fixtures/flake8_pyi/PYI026.pyi:4:1: PYI026 Use `typing.TypeAlias` for type aliases in `NewAny`, e.g. "NewAny: typing.TypeAlias = Any"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI026.pyi:5:1: PYI026 Use `typing.TypeAlias` for type aliases in `OptinalStr`, e.g. "OptinalStr: typing.TypeAlias = typing.Optional[str]"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI026.pyi:6:1: PYI026 Use `typing.TypeAlias` for type aliases in `Foo`, e.g. "Foo: typing.TypeAlias = Literal["foo"]"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI026.pyi:7:1: PYI026 Use `typing.TypeAlias` for type aliases in `IntOrStr`, e.g. "IntOrStr: typing.TypeAlias = int | str"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI026.pyi:8:1: PYI026 Use `typing.TypeAlias` for type aliases in `AliasNone`, e.g. "AliasNone: typing.TypeAlias = None"
Found 5 errors.
```

ref: #848 

## Test Plan

Snapshots, manual runs of flake8.
2023-07-20 01:39:55 +00:00
Charlie Marsh
963f240e46 Track unresolved references in the semantic model (#5902)
## Summary

As part of my continued quest to separate semantic model-building from
diagnostic emission, this PR moves our unresolved-reference rules to a
deferred pass. So, rather than emitting diagnostics as we encounter
unresolved references, we now track those unresolved references on the
semantic model (just like resolved references), and after traversal,
emit the relevant rules for any unresolved references.
2023-07-19 18:19:55 -04:00
Tom Kuson
23cde4d1f5 Add known problems to compare-to-empty-string documentation (#5879)
## Summary

Add known problems to `compare-to-empty-string` documentation. Related
to #5873.

Tweaked the example in the documentation to be a tad more concise and
correct (that the rule is most applicable when comparing to a `str`
variable).

## Test Plan

`python scripts/check_docs_formatted.py`
2023-07-19 18:12:27 -04:00
Charlie Marsh
9834c69c98 Remove __all__ enforcement rules out of binding phase (#5897)
## Summary

This PR moves two rules (`invalid-all-format` and `invalid-all-object`)
out of the name-binding phase, and into the dedicated pass over all
bindings that occurs at the end of the `Checker`. This is part of my
continued quest to separate the semantic model-building logic from the
actual rule enforcement.
2023-07-19 21:18:47 +00:00
Zanie Blue
b27f0fa433 Implement any_over_expr for type alias and type params (#5866)
Part of https://github.com/astral-sh/ruff/issues/5062
2023-07-19 16:17:06 -05:00
konsti
a459d8ffc7 Filter off-by-default RUF014 out of schema (#5832)
**Summary** Previously, `RUF014` would be part of ruff.schema.json
depending on whether or not the `unreachable-code` feature was active.
This caused problems for contributors who got unrelated RUF014 changes
when updating the schema without the feature active.

An alternative would be to always add `RUF014`.

**Test plan** `cargo dev generate-all` and `cargo run --bin ruff_dev
--features unreachable-code -- generate-all` now have the same effect.
2023-07-19 21:06:10 +00:00
Charlie Marsh
598549d24e Fix incorrect reference in extend-immutable-calls documentation (#5890) 2023-07-19 19:57:05 +00:00
David Cain
e1d76b60cc Add missing backtick to B034 documentation (#5889)
This is a great rule, but the documentation page shows some wonky
formatting due to a missing backtick. Fix a typo too.

Should fix display on
https://beta.ruff.rs/docs/rules/re-sub-positional-args/

<img width="1160" alt="image"
src="https://github.com/astral-sh/ruff/assets/901169/44bd76ec-9eb9-4290-ba7a-7691a7ea21d4">
2023-07-19 17:25:36 +00:00
Pedro
6f96acfd27 Rename Pynecone to Reflex (#5888)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

They just changed the name to `Reflex`

## Test Plan

Nothing
2023-07-19 18:46:49 +02:00
Micha Reiser
5a4317c688 Remove multithreading from check multiproject (#5884) 2023-07-19 16:18:30 +00:00
Charlie Marsh
5f3da9955a Rename ruff_python_whitespace to ruff_python_trivia (#5886)
## Summary

This crate now contains utilities for dealing with trivia more broadly:
whitespace, newlines, "simple" trivia lexing, etc. So renaming it to
reflect its increased responsibilities.

To avoid conflicts, I've also renamed `Token` and `TokenKind` to
`SimpleToken` and `SimpleTokenKind`.
2023-07-19 11:48:27 -04:00
Charlie Marsh
a75a6de577 Use a boxed slice for Export struct (#5887)
## Summary

The vector of names here is immutable -- we never push to it after
initialization. Boxing reduces the size of the variant from 32 bytes to
24 bytes. (See:
https://nnethercote.github.io/perf-book/type-sizes.html#boxed-slices.)
It doesn't make a difference here, since it's not the largest variant,
but it still seems like a prudent change (and I was considering adding
another field to this variant, though I may no longer do so).
2023-07-19 11:45:04 -04:00
konsti
a227775f62 Type alias stub for formatter (#5880)
**Summary** This replaces the `todo!()` with a type alias stub in the
formatter. I added the tests from
704eb40108/parser/src/parser.rs (L901-L936)
as ruff python formatter tests.

**Test Plan** None, testing is part of the actual implementation
2023-07-19 17:28:07 +02:00
konsti
a51606a10a Handle parentheses when formatting slice expressions (#5882)
**Summary** Fix the formatter crash with `x[(1) :: ]` and related code.

**Problem** For assigning comments in slices in subscripts, we need to
find the positions of the colons to assign comments before and after the
colon to the respective lower/upper/step node (or dangling in that
section). Formatting `x[(1) :: ]` was broken because we were looking for
a `:` after the `1` but didn't consider that there could be a `)`
outside the range of the lower node, which contains just the `1` and no
optional parentheses.

**Solution** Use the simple tokenizer directly and skip all closing
parentheses.

**Test Plan** I added regression tests.

Closes #5733
2023-07-19 15:25:25 +00:00
konsti
63ed7a31e8 Add message to formatter SyntaxError (#5881)
**Summary** Add a static string error message to the formatter syntax
error so we can disambiguate where the syntax error came from

**Test Plan** No fixed tests, we don't expect this to occur, but it
helped with transformers syntax error debugging:

```
Error: Failed to format node

Caused by:
    syntax error: slice first colon token was not a colon
```
2023-07-19 17:15:26 +02:00
Micha Reiser
46a17d11f3 playground: Add AST/Tokens/Formatter panels (#5859) 2023-07-19 14:46:08 +00:00
Micha Reiser
9ed7ceeb0a playground: Add left panel and use brand colors (#5838) 2023-07-19 16:33:32 +02:00
Chris Pryer
9fb8d6e999 Omit tuple parentheses inside comprehensions (#5790) 2023-07-19 12:05:38 +00:00
Chris Pryer
38678142ed Format lambda expression (#5806) 2023-07-19 11:47:56 +00:00
David Szotten
5d68ad9008 Format expr generator exp (#5804) 2023-07-19 13:01:58 +02:00
Micha Reiser
cda90d071c Upgrade cargo insta (#5872) 2023-07-19 12:56:32 +02:00
Dhruv Manilawala
7e6b472c5b Make lint_only aware of the source kind (#5876) 2023-07-19 09:29:35 +05:30
Charlie Marsh
1181d25e5a Move a few more candidate rules to the deferred Binding-only pass (#5853)
## Summary

No behavior change, but this is in theory more efficient, since we can
just iterate over the flat `Binding` vector rather than having to
iterate over binding chains via the `Scope`.
2023-07-19 00:59:02 +00:00
Charlie Marsh
626d8dc2cc Use .as_ref() in lieu of &** (#5874)
I find this less opaque (and often more succinct).
2023-07-19 00:49:13 +00:00
Charlie Marsh
7ffcd93afd Move unused deletion tracking to deferred analysis (#5852)
## Summary

This PR moves the "unused exception" rule out of the visitor and into a
deferred check. When we can base rules solely on the semantic model, we
probably should, as it greatly simplifies the `Checker` itself.
2023-07-18 20:43:12 -04:00
Charlie Marsh
2d505e2b04 Remove suite body tracking from SemanticModel (#5848)
## Summary

The `SemanticModel` currently stores the "body" of a given `Suite`,
along with the current statement index. This is used to support "next
sibling" queries, but we only use this in exactly one place -- the rule
that simplifies constructs like this to `any` or `all`:

```python
for x in y:
    if x == 0:
        return True
return False
```

Instead of tracking the state, we can just do a (slightly more
expensive) traversal, by finding the node within its parent and
returning the next node in the body.

Note that we'll only have to do this extremely rarely -- namely, for
functions that contain something like:

```python
for x in y:
    if x == 0:
        return True
```
2023-07-18 18:58:31 -04:00
Zanie Blue
a93254f026 Implement unparse for type aliases and parameters (#5869)
Part of https://github.com/astral-sh/ruff/issues/5062
2023-07-18 16:25:49 -05:00
Micha Reiser
c577045f2e perf(formatter): Use memchar for faster back tokenization (#5823) 2023-07-18 21:05:55 +00:00
Charlie Marsh
4204fc002d Remove exception-handler lexing from unused-bound-exception fix (#5851)
## Summary

The motivation here is that it will make this rule easier to rewrite as
a deferred check. Right now, we can't run this rule in the deferred
phase, because it depends on the `except_handler` to power its autofix.
Instead of lexing the `except_handler`, we can use the `SimpleTokenizer`
from the formatter, and just lex forwards and backwards.

For context, this rule detects the unused `e` in:

```python
try:
  pass
except ValueError as e:
  pass
```
2023-07-18 18:27:46 +00:00
Zanie Blue
41da52a61b Implement TokenKind for type aliases (#5870)
Part of https://github.com/astral-sh/ruff/issues/5062
2023-07-18 18:21:51 +00:00
Zanie Blue
d5c43a45b3 Implement Comparable for type aliases and parameters (#5865)
Part of https://github.com/astral-sh/ruff/issues/5062
2023-07-18 17:18:14 +00:00
Nikita Sobolev
cdfed3d50e Use relativize_path for noqa warnings (#5867)
Refs https://github.com/astral-sh/ruff/pull/5856
2023-07-18 12:44:32 -04:00
Harutaka Kawamura
68097e34e6 Update UP032 to autofix multi-line triple-quoted string (#5862)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

Resolve #5854

## Test Plan

<!-- How was it tested? -->

New test cases

---------

Co-authored-by: konsti <konstin@mailbox.org>
2023-07-18 16:40:37 +00:00
Zanie Blue
f47443014e Remove tag information from RustPython-Parser dependency (#5861)
Following https://github.com/astral-sh/RustPython-Parser/pull/27 we now
cherry-pick commits onto our fork instead of rebasing our fork on top of
the upstream which means we do not overwrite history and a tag is not
necessary to preserve the pinned commit.

In the future, we may rewrite the history in our fork. If we do, we
should return to tagging the commits.
2023-07-18 10:48:51 -05:00
Zanie Blue
0eab4b3c22 Implement AnyNode and AnyNodRef for StmtTypeAlias (#5863)
Part of https://github.com/astral-sh/ruff/issues/5062
2023-07-18 10:44:55 -05:00
Charlie Marsh
c868def374 Unroll collect_call_path to speed up common cases (#5792)
## Summary

This PR just naively unrolls `collect_call_path` to handle attribute
resolutions of up to eight segments. In profiling via Instruments, it
seems to be about 4x faster for a very hot code path (4% of total
execution time on `main`, 1% here).

Profiling by running `RAYON_NUM_THREADS=1 cargo instruments -t time
--profile release-debug --time-limit 10000 -p ruff_cli -o
FromSlice.trace -- check crates/ruff/resources/test/cpython --silent -e
--no-cache --select ALL`, and modifying the linter to loop infinitely up
to the specified time (10 seconds) to increase sample size.

Before:

<img width="1792" alt="Screen Shot 2023-07-15 at 5 13 34 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/4a8b0b45-8b67-43e9-af5e-65b326928a8e">

After:

<img width="1792" alt="Screen Shot 2023-07-15 at 8 38 51 PM"
src="https://github.com/astral-sh/ruff/assets/1309177/d8829159-2c79-4a49-ab3c-9e4e86f5b2b1">
2023-07-18 11:29:59 -04:00
konsti
5d41c832ad Formatter: Run generate.py for ElifElseClauses (#5864)
**Summary** This removes the diff for the next user of `generate.py`.
It's effectively a refactoring.

**Test Plan** No functional changes
2023-07-18 17:17:17 +02:00
Nikita Sobolev
0c7c81aa31 Add filename to noqa warnings (#5856)
## Summary

Before:

```
» ruff litestar tests --fix
warning: Invalid `# noqa` directive on line 19: expected a comma-separated list of codes (e.g., `# noqa: F401, F841`).
warning: Invalid `# noqa` directive on line 65: expected a comma-separated list of codes (e.g., `# noqa: F401, F841`).
warning: Invalid `# noqa` directive on line 74: expected a comma-separated list of codes (e.g., `# noqa: F401, F841`).
warning: Invalid `# noqa` directive on line 22: expected a comma-separated list of codes (e.g., `# noqa: F401, F841`).
warning: Invalid `# noqa` directive on line 66: expected a comma-separated list of codes (e.g., `# noqa: F401, F841`).
warning: Invalid `# noqa` directive on line 75: expected a comma-separated list of codes (e.g., `# noqa: F401, F841`).
```

After:

```
» cargo run --bin ruff ../litestar/litestar ../litestar/tests
    Finished dev [unoptimized + debuginfo] target(s) in 0.15s
     Running `target/debug/ruff ../litestar/litestar ../litestar/tests`
warning: Detected debug build without --no-cache.
warning: Invalid `# noqa` directive on /Users/sobolev/Desktop/litestar/tests/unit/test_contrib/test_sqlalchemy/models_bigint.py:19: expected a comma-separated list of codes (e.g., `# noqa: F401, F841`).
warning: Invalid `# noqa` directive on /Users/sobolev/Desktop/litestar/tests/unit/test_contrib/test_sqlalchemy/models_bigint.py:65: expected a comma-separated list of codes (e.g., `# noqa: F401, F841`).
warning: Invalid `# noqa` directive on /Users/sobolev/Desktop/litestar/tests/unit/test_contrib/test_sqlalchemy/models_bigint.py:74: expected a comma-separated list of codes (e.g., `# noqa: F401, F841`).
warning: Invalid `# noqa` directive on /Users/sobolev/Desktop/litestar/tests/unit/test_contrib/test_sqlalchemy/models_uuid.py:22: expected a comma-separated list of codes (e.g., `# noqa: F401, F841`).
warning: Invalid `# noqa` directive on /Users/sobolev/Desktop/litestar/tests/unit/test_contrib/test_sqlalchemy/models_uuid.py:66: expected a comma-separated list of codes (e.g., `# noqa: F401, F841`).
warning: Invalid `# noqa` directive on /Users/sobolev/Desktop/litestar/tests/unit/test_contrib/test_sqlalchemy/models_uuid.py:75: expected a comma-separated list of codes (e.g., `# noqa: F401, F841`).
```

## Test Plan

I didn't find any existing tests with this warning.

Closes https://github.com/astral-sh/ruff/issues/5855
2023-07-18 14:08:22 +00:00
Micha Reiser
3b32e3a8fe perf(formatter): Improve is_expression_parenthesized performance (#5825) 2023-07-18 15:48:49 +02:00
Charlie Marsh
1aa851796e Add documentation to Checker (#5849)
## Summary

Documents the overall responsibilities along with the various steps in
the data flow.
2023-07-18 07:52:04 -04:00
konsti
730e6b2b4c Refactor StmtIf: Formatter and Linter (#5459)
## Summary

Previously, `StmtIf` was defined recursively as
```rust
pub struct StmtIf {
    pub range: TextRange,
    pub test: Box<Expr>,
    pub body: Vec<Stmt>,
    pub orelse: Vec<Stmt>,
}
```
Every `elif` was represented as an `orelse` with a single `StmtIf`. This
means that this representation couldn't differentiate between
```python
if cond1:
    x = 1
else:
    if cond2:
        x = 2
```
and 
```python
if cond1:
    x = 1
elif cond2:
    x = 2
```
It also makes many checks harder than they need to be because we have to
recurse just to iterate over an entire if-elif-else and because we're
lacking nodes and ranges on the `elif` and `else` branches.

We change the representation to a flat

```rust
pub struct StmtIf {
    pub range: TextRange,
    pub test: Box<Expr>,
    pub body: Vec<Stmt>,
    pub elif_else_clauses: Vec<ElifElseClause>,
}

pub struct ElifElseClause {
    pub range: TextRange,
    pub test: Option<Expr>,
    pub body: Vec<Stmt>,
}
```
where `test: Some(_)` represents an `elif` and `test: None` an else.

This representation is different tradeoff, e.g. we need to allocate the
`Vec<ElifElseClause>`, the `elif`s are now different than the `if`s
(which matters in rules where want to check both `if`s and `elif`s) and
the type system doesn't guarantee that the `test: None` else is actually
last. We're also now a bit more inconsistent since all other `else`,
those from `for`, `while` and `try`, still don't have nodes. With the
new representation some things became easier, e.g. finding the `elif`
token (we can use the start of the `ElifElseClause`) and formatting
comments for if-elif-else (no more dangling comments splitting, we only
have to insert the dangling comment after the colon manually and set
`leading_alternate_branch_comments`, everything else is taken of by
having nodes for each branch and the usual placement.rs fixups).

## Merge Plan

This PR requires coordination between the parser repo and the main ruff
repo. I've split the ruff part, into two stacked PRs which have to be
merged together (only the second one fixes all tests), the first for the
formatter to be reviewed by @michareiser and the second for the linter
to be reviewed by @charliermarsh.

* MH: Review and merge
https://github.com/astral-sh/RustPython-Parser/pull/20
* MH: Review and merge or move later in stack
https://github.com/astral-sh/RustPython-Parser/pull/21
* MH: Review and approve
https://github.com/astral-sh/RustPython-Parser/pull/22
* MH: Review and approve formatter PR
https://github.com/astral-sh/ruff/pull/5459
* CM: Review and approve linter PR
https://github.com/astral-sh/ruff/pull/5460
* Merge linter PR in formatter PR, fix ecosystem checks (ecosystem
checks can't run on the formatter PR and won't run on the linter PR, so
we need to merge them first)
 * Merge https://github.com/astral-sh/RustPython-Parser/pull/22
 * Create tag in the parser, update linter+formatter PR
 * Merge linter+formatter PR https://github.com/astral-sh/ruff/pull/5459

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2023-07-18 13:40:15 +02:00
Chris Pryer
167b9356fa Update from join_with example to join_comma_separated (#5843)
## Summary

Originally `join_with` was used in the formatters README.md. Now it uses

```rs
f.join_comma_separated(item.end())
    .nodes(elts.iter())
    .finish()
```

## Test Plan

None
2023-07-18 11:03:16 +02:00
konsti
d098256c96 Add a tool for shrinking failing examples (#5731)
## Summary

For formatter instabilities, the message we get look something like
this:
```text
Unstable formatting /home/konsti/ruff/target/checkouts/deepmodeling:dpdispatcher/dpdispatcher/slurm.py
@@ -47,9 +47,9 @@
-            script_header_dict["slurm_partition_line"] = (
-                NOT_YET_IMPLEMENTED_ExprJoinedStr
-            )
+            script_header_dict[
+                "slurm_partition_line"
+            ] = NOT_YET_IMPLEMENTED_ExprJoinedStr
Unstable formatting /home/konsti/ruff/target/checkouts/deepmodeling:dpdispatcher/dpdispatcher/pbs.py
@@ -26,9 +26,9 @@
-            pbs_script_header_dict["select_node_line"] += (
-                NOT_YET_IMPLEMENTED_ExprJoinedStr
-            )
+            pbs_script_header_dict[
+                "select_node_line"
+            ] += NOT_YET_IMPLEMENTED_ExprJoinedStr
``` 

For ruff crashes. you don't even get that but just the file that crashed
it. To extract the actual bug, you'd need to manually remove parts of
the file, rerun to see if the bug still occurs (and revert if it
doesn't) until you have a minimal example.

With this script, you run

```shell
cargo run --bin ruff_shrinking -- target/checkouts/deepmodeling:dpdispatcher/dpdispatcher/slurm.py target/minirepo/code.py "Unstable formatting" "target/debug/ruff_dev format-dev --stability-check target/minirepo"
```

and get

```python
class Slurm():
    def gen_script_header(self, job):
        if resources.queue_name != "":
            script_header_dict["slurm_partition_line"] = f"#SBATCH --partition {resources.queue_name}"
```

which is an nice minimal example.

I've been using this script and it would be easier for me if this were
part of main. The main disadvantage to merging is that it adds
additional dependencies.

## Test Plan

I've been using this for a number of minimization. This is an internal
helper script you only run manually. I could add a test that minimizes a
rule violation if required.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2023-07-18 08:03:35 +00:00
Micha Reiser
ef58287c16 playground: Merge Editor state variables (#5831)
<!--
Thank you for contributing to Ruff! To help us out with reviewing, please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

This PR removes state variables that can be derived, merges related variables into a single state, and generally avoids `null` states. 

## Test Plan

I clicked through the playground locally
<!-- How was it tested? -->
2023-07-18 08:08:24 +02:00
Micha Reiser
9ddf40455d Upgrade playground dependencies (#5830)
<!--
Thank you for contributing to Ruff! To help us out with reviewing, please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

This PR upgrades the playground's runtime and dev dependencies

<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

I tested the playground locally

<!-- How was it tested? -->
2023-07-18 08:00:54 +02:00
Harutaka Kawamura
a4e5e3205f Ignore directories when collecting files to lint (#5775)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

Fixes #5739

## Test Plan

<!-- How was it tested? -->

Manually tested:

```sh
$ tree dir
dir
├── dir.py
│   └── file.py
└── file.py

1 directory, 2 files

$ cargo run -p ruff_cli -- check dir --no-cache
    Finished dev [unoptimized + debuginfo] target(s) in 0.08s
     Running `target/debug/ruff check dir --no-cache`
dir/dir.py/file.py:1:7: F821 Undefined name `a`
dir/file.py:1:7: F821 Undefined name `a`
Found 2 errors.
```

Is a unit test needed?
2023-07-17 20:25:43 -05:00
Simon Brugman
17ee80363a refactor: use find_keyword ast helper more (#5847)
Use the ast helper function `find_keyword` where applicable

(found these while working on another feature)
2023-07-17 19:37:23 -04:00
1259 changed files with 141793 additions and 22211 deletions

View File

@@ -2,6 +2,14 @@ name: Benchmark
on:
pull_request:
paths:
- 'Cargo.toml'
- 'Cargo.lock'
- 'rust-toolchain'
- 'crates/**'
- '!crates/ruff_dev'
- '!crates/ruff_shrinking'
workflow_dispatch:
concurrency:

View File

@@ -19,6 +19,43 @@ env:
PYTHON_VERSION: "3.11" # to build abi3 wheels
jobs:
determine_changes:
name: "Determine changes"
runs-on: ubuntu-latest
outputs:
linter: ${{ steps.changed.outputs.linter_any_changed }}
formatter: ${{ steps.changed.outputs.formatter_any_changed }}
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- uses: tj-actions/changed-files@v37
id: changed
with:
files_yaml: |
linter:
- Cargo.toml
- Cargo.lock
- crates/**
- "!crates/ruff_python_formatter/**"
- "!crates/ruff_formatter/**"
- "!crates/ruff_dev/**"
- "!crates/ruff_shrinking/**"
formatter:
- Cargo.toml
- Cargo.lock
- crates/ruff_python_formatter/**
- crates/ruff_formatter/**
- crates/ruff_python_trivia/**
- crates/ruff_python_ast/**
- crates/ruff_source_file/**
- crates/ruff_python_index/**
- crates/ruff_text_size/**
- crates/ruff_python_parser/**
cargo-fmt:
name: "cargo fmt"
runs-on: ubuntu-latest
@@ -53,10 +90,12 @@ jobs:
- uses: actions/checkout@v3
- name: "Install Rust toolchain"
run: rustup show
- uses: Swatinem/rust-cache@v2
# cargo insta 1.30.0 fails for some reason (https://github.com/mitsuhiko/insta/issues/392)
- run: cargo install cargo-insta@=1.29.0
- name: "Install cargo insta"
uses: taiki-e/install-action@v2
with:
tool: cargo-insta
- run: pip install black[d]==23.1.0
- uses: Swatinem/rust-cache@v2
- name: "Run tests (Ubuntu)"
if: ${{ matrix.os == 'ubuntu-latest' }}
run: cargo insta test --all --all-features --unreferenced reject
@@ -135,9 +174,11 @@ jobs:
ecosystem:
name: "ecosystem"
runs-on: ubuntu-latest
needs: cargo-test
needs:
- cargo-test
- determine_changes
# Only runs on pull requests, since that is the only we way we can find the base version for comparison.
if: github.event_name == 'pull_request'
if: github.event_name == 'pull_request' && needs.determine_changes.outputs.linter == 'true'
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
@@ -282,16 +323,23 @@ jobs:
if: ${{ env.MKDOCS_INSIDERS_SSH_KEY_EXISTS != 'true' }}
run: mkdocs build --strict -f mkdocs.generated.yml
check-formatter-stability:
name: "Check formatter stability"
check-formatter-ecosystem:
name: "Formatter ecosystem and progress checks"
runs-on: ubuntu-latest
needs: determine_changes
if: needs.determine_changes.outputs.formatter == 'true'
steps:
- uses: actions/checkout@v3
- name: "Install Rust toolchain"
run: rustup show
- name: "Cache rust"
uses: Swatinem/rust-cache@v2
- name: "Formatter progress"
run: scripts/formatter_progress.sh
- name: "Github step summary"
run: grep "similarity index" target/progress_projects_report.txt | sort > $GITHUB_STEP_SUMMARY
# CPython is not black formatted, so we run only the stability check
- name: "Clone CPython 3.10"
run: git clone --branch 3.10 --depth 1 https://github.com/python/cpython.git crates/ruff/resources/test/cpython
- name: "Check stability"
- name: "Check CPython stability"
run: cargo run --bin ruff_dev -- format-dev --stability-check crates/ruff/resources/test/cpython

View File

@@ -71,7 +71,7 @@ pipx install pre-commit # or `pip install pre-commit` if you have a virtualenv
### Development
After cloning the repository, run Ruff locally with:
After cloning the repository, run Ruff locally from the repository root with:
```shell
cargo run -p ruff_cli -- check /path/to/file.py --no-cache
@@ -122,9 +122,9 @@ At time of writing, the repository includes the following crates:
intermediate representation. The backend for `ruff_python_formatter`.
- `crates/ruff_index`: library crate inspired by `rustc_index`.
- `crates/ruff_macros`: proc macro crate containing macros used by Ruff.
- `crates/ruff_python_ast`: library crate containing Python-specific AST types and utilities. Note
that the AST schema itself is defined in the
[rustpython-ast](https://github.com/astral-sh/RustPython-Parser) crate.
- `crates/ruff_python_ast`: library crate containing Python-specific AST types and utilities.
- `crates/ruff_python_codegen`: library crate containing utilities for generating Python source code.
- `crates/ruff_python_codegen`: library crate containing utilities for generating Python source code.
- `crates/ruff_python_formatter`: library crate implementing the Python formatter. Emits an
intermediate representation for each node, which `ruff_formatter` prints based on the configured
line length.
@@ -133,10 +133,9 @@ At time of writing, the repository includes the following crates:
refer to?"
- `crates/ruff_python_stdlib`: library crate containing Python-specific standard library data, e.g.
the names of all built-in exceptions and which standard library types are immutable.
- `crates/ruff_python_whitespace`: library crate containing Python-specific whitespace analysis
logic (indentation and newlines).
- `crates/ruff_rustpython`: library crate containing `RustPython`-specific utilities.
- `crates/ruff_textwrap`: library crate to indent and dedent Python source code.
- `crates/ruff_python_trivia`: library crate containing Python-specific trivia utilities (e.g.,
for analyzing indentation, newlines, etc.).
- `crates/ruff_python_parser`: library crate containing the Python parser.
- `crates/ruff_wasm`: library crate for exposing Ruff as a WebAssembly module. Powers the
[Ruff Playground](https://play.ruff.rs/).
@@ -156,10 +155,13 @@ At a high level, the steps involved in adding a new lint rule are as follows:
(e.g., `pub(crate) fn assert_false`) based on whatever inputs are required for the rule (e.g.,
an `ast::StmtAssert` node).
1. Define the logic for triggering the violation in `crates/ruff/src/checkers/ast/mod.rs` (for
AST-based checks), `crates/ruff/src/checkers/tokens.rs` (for token-based checks),
`crates/ruff/src/checkers/lines.rs` (for text-based checks), or
`crates/ruff/src/checkers/filesystem.rs` (for filesystem-based checks).
1. Define the logic for invoking the diagnostic in `crates/ruff/src/checkers/ast/analyze` (for
AST-based rules), `crates/ruff/src/checkers/tokens.rs` (for token-based rules),
`crates/ruff/src/checkers/physical_lines.rs` (for text-based rules),
`crates/ruff/src/checkers/filesystem.rs` (for filesystem-based rules), etc. For AST-based rules,
you'll likely want to modify `analyze/statement.rs` (if your rule is based on analyzing
statements, like imports) or `analyze/expression.rs` (if your rule is based on analyzing
expressions, like function calls).
1. Map the violation struct to a rule code in `crates/ruff/src/codes.rs` (e.g., `B011`).
@@ -221,9 +223,12 @@ Once you've completed the code for the rule itself, you can define tests with th
For example, if you're adding a new rule named `E402`, you would run:
```shell
cargo run -p ruff_cli -- check crates/ruff/resources/test/fixtures/pycodestyle/E402.py --no-cache
cargo run -p ruff_cli -- check crates/ruff/resources/test/fixtures/pycodestyle/E402.py --no-cache --select E402
```
**Note:** Only a subset of rules are enabled by default. When testing a new rule, ensure that
you activate it by adding `--select ${rule_code}` to the command.
1. Add the test to the relevant `crates/ruff/src/rules/[linter]/mod.rs` file. If you're contributing
a rule to a pre-existing set, you should be able to find a similar example to pattern-match
against. If you're adding a new linter, you'll need to create a new `mod.rs` file (see,
@@ -310,7 +315,7 @@ even patch releases may contain [non-backwards-compatible changes](https://semve
1. Run the release workflow with the version number (without starting `v`) as input. Make sure
main has your merged PR as last commit
1. The release workflow will do the following:
1. Build all the assets. If this fails (even though we tested in step 4), we havent tagged or
1. Build all the assets. If this fails (even though we tested in step 4), we haven't tagged or
uploaded anything, you can restart after pushing a fix.
1. Upload to PyPI.
1. Create and push the Git tag (as extracted from `pyproject.toml`). We create the Git tag only
@@ -723,9 +728,9 @@ diagnostics, then our current compilation pipeline proceeds as follows:
1. **File discovery**: Given paths like `foo/`, locate all Python files in any specified subdirectories, taking into account our hierarchical settings system and any `exclude` options.
1. **Package resolution**: Determine the package root for every file by traversing over its parent directories and looking for `__init__.py` files.
1. **Package resolution**: Determine the "package root" for every file by traversing over its parent directories and looking for `__init__.py` files.
1. **Cache initialization**: For every package root, initialize an empty cache.
1. **Cache initialization**: For every "package root", initialize an empty cache.
1. **Analysis**: For every file, in parallel:
@@ -733,7 +738,7 @@ diagnostics, then our current compilation pipeline proceeds as follows:
1. **Tokenization**: Run the lexer over the file to generate a token stream.
1. **Indexing**: Extract metadata from the token stream, such as: comment ranges, `# noqa` locations, `# isort: off` locations, doc lines, etc.
1. **Indexing**: Extract metadata from the token stream, such as: comment ranges, `# noqa` locations, `# isort: off` locations, "doc lines", etc.
1. **Token-based rule evaluation**: Run any lint rules that are based on the contents of the token stream (e.g., commented-out code).
@@ -743,9 +748,9 @@ diagnostics, then our current compilation pipeline proceeds as follows:
1. **Parsing**: Run the parser over the token stream to produce an AST. (This consumes the token stream, so anything that relies on the token stream needs to happen before parsing.)
1. **AST-based rule evaluation**: Run any lint rules that are based on the AST. This includes the vast majority of lint rules. As part of this step, we also build the semantic model for the current file as we traverse over the AST. Some lint rules are evaluated eagerly, as we iterate over the AST, while others are evaluated in a deferred manner (e.g., unused imports, since we cant determine whether an import is unused until weve finished analyzing the entire file), after weve finished the initial traversal.
1. **AST-based rule evaluation**: Run any lint rules that are based on the AST. This includes the vast majority of lint rules. As part of this step, we also build the semantic model for the current file as we traverse over the AST. Some lint rules are evaluated eagerly, as we iterate over the AST, while others are evaluated in a deferred manner (e.g., unused imports, since we can't determine whether an import is unused until we've finished analyzing the entire file), after we've finished the initial traversal.
1. **Import-based rule evaluation**: Run any lint rules that are based on the modules imports (e.g., import sorting). These could, in theory, be included in the AST-based rule evaluation phase — theyre just separated for simplicity.
1. **Import-based rule evaluation**: Run any lint rules that are based on the module's imports (e.g., import sorting). These could, in theory, be included in the AST-based rule evaluation phase — they're just separated for simplicity.
1. **Physical line-based rule evaluation**: Run any lint rules that are based on physical lines (e.g., line-length).
@@ -754,3 +759,116 @@ diagnostics, then our current compilation pipeline proceeds as follows:
1. **Cache write**: Write the generated diagnostics to the package cache using the file as a key.
1. **Reporting**: Print diagnostics in the specified format (text, JSON, etc.), to the specified output channel (stdout, a file, etc.).
### Import Categorization
To understand Ruff's import categorization system, we first need to define two concepts:
- "Project root": The directory containing the `pyproject.toml`, `ruff.toml`, or `.ruff.toml` file,
discovered by identifying the "closest" such directory for each Python file. (If you're running
via `ruff --config /path/to/pyproject.toml`, then the current working directory is used as the
"project root".)
- "Package root": The top-most directory defining the Python package that includes a given Python
file. To find the package root for a given Python file, traverse up its parent directories until
you reach a parent directory that doesn't contain an `__init__.py` file (and isn't marked as
a [namespace package](https://beta.ruff.rs/docs/settings/#namespace-packages)); take the directory
just before that, i.e., the first directory in the package.
For example, given:
```text
my_project
├── pyproject.toml
└── src
└── foo
├── __init__.py
└── bar
├── __init__.py
└── baz.py
```
Then when analyzing `baz.py`, the project root would be the top-level directory (`./my_project`),
and the package root would be `./my_project/src/foo`.
#### Project root
The project root does not have a significant impact beyond that all relative paths within the loaded
configuration file are resolved relative to the project root.
For example, to indicate that `bar` above is a namespace package (it isn't, but let's run with it),
the `pyproject.toml` would list `namespace-packages = ["./src/bar"]`, which would resolve
to `my_project/src/bar`.
The same logic applies when providing a configuration file via `--config`. In that case, the
_current working directory_ is used as the project root, and so all paths in that configuration file
are resolved relative to the current working directory. (As a general rule, we want to avoid relying
on the current working directory as much as possible, to ensure that Ruff exhibits the same behavior
regardless of where and how you invoke it — but that's hard to avoid in this case.)
Additionally, if a `pyproject.toml` file _extends_ another configuration file, Ruff will still use
the directory containing that `pyproject.toml` file as the project root. For example, if
`./my_project/pyproject.toml` contains:
```toml
[tool.ruff]
extend = "/path/to/pyproject.toml"
```
Then Ruff will use `./my_project` as the project root, even though the configuration file extends
`/path/to/pyproject.toml`. As such, if the configuration file at `/path/to/pyproject.toml` contains
any relative paths, they will be resolved relative to `./my_project`.
If a project uses nested configuration files, then Ruff would detect multiple project roots, one for
each configuration file.
#### Package root
The package root is used to determine a file's "module path". Consider, again, `baz.py`. In that
case, `./my_project/src/foo` was identified as the package root, so the module path for `baz.py`
would resolve to `foo.bar.baz` — as computed by taking the relative path from the package root
(inclusive of the root itself). The module path can be thought of as "the path you would use to
import the module" (e.g., `import foo.bar.baz`).
The package root and module path are used to, e.g., convert relative to absolute imports, and for
import categorization, as described below.
#### Import categorization
When sorting and formatting import blocks, Ruff categorizes every import into one of five
categories:
1. **"Future"**: the import is a `__future__` import. That's easy: just look at the name of the
imported module!
1. **"Standard library"**: the import comes from the Python standard library (e.g., `import os`).
This is easy too: we include a list of all known standard library modules in Ruff itself, so it's
a simple lookup.
1. **"Local folder"**: the import is a relative import (e.g., `from .foo import bar`). This is easy
too: just check if the import includes a `level` (i.e., a dot-prefix).
1. **"First party"**: the import is part of the current project. (More on this below.)
1. **"Third party"**: everything else.
The real challenge lies in determining whether an import is first-party — everything else is either
trivial, or (as in the case of third-party) merely defined as "not first-party".
There are three ways in which an import can be categorized as "first-party":
1. **Explicit settings**: the import is marked as such via the `known-first-party` setting. (This
should generally be seen as an escape hatch.)
1. **Same-package**: the imported module is in the same package as the current file. This gets back
to the importance of the "package root" and the file's "module path". Imagine that we're
analyzing `baz.py` above. If `baz.py` contains any imports that appear to come from the `foo`
package (e.g., `from foo import bar` or `import foo.bar`), they'll be classified as first-party
automatically. This check is as simple as comparing the first segment of the current file's
module path to the first segment of the import.
1. **Source roots**: Ruff supports a `[src](https://beta.ruff.rs/docs/settings/#src)` setting, which
sets the directories to scan when identifying first-party imports. The algorithm is
straightforward: given an import, like `import foo`, iterate over the directories enumerated in
the `src` setting and, for each directory, check for the existence of a subdirectory `foo` or a
file `foo.py`.
By default, `src` is set to the project root. In the above example, we'd want to set
`src = ["./src"]` to ensure that we locate `./my_project/src/foo` and thus categorize `import foo`
as first-party in `baz.py`. In practice, for this limited example, setting `src = ["./src"]` is
unnecessary, as all imports within `./my_project/src/foo` would be categorized as first-party via
the same-package heuristic; but your project contains multiple packages, you'll want to set `src`
explicitly.

821
Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -21,12 +21,11 @@ filetime = { version = "0.2.20" }
glob = { version = "0.3.1" }
globset = { version = "0.4.10" }
ignore = { version = "0.4.20" }
insta = { version = "1.30.0" }
insta = { version = "1.31.0", feature = ["filters", "glob"] }
is-macro = { version = "0.2.2" }
itertools = { version = "0.10.5" }
log = { version = "0.4.17" }
memchr = "2.5.0"
nohash-hasher = { version = "0.2.0" }
num-bigint = { version = "0.4.3" }
num-traits = { version = "0.2.15" }
once_cell = { version = "1.17.1" }
@@ -52,15 +51,6 @@ wsl = { version = "0.1.0" }
# v1.0.1
libcst = { git = "https://github.com/Instagram/LibCST.git", rev = "3cacca1a1029f05707e50703b49fe3dd860aa839", default-features = false }
# Please tag the RustPython version every time you update its revision here and in fuzz/Cargo.toml
# Tagging the version ensures that older ruff versions continue to build from source even when we rebase our RustPython fork.
# Current tag: v0.0.7
ruff_text_size = { git = "https://github.com/astral-sh/RustPython-Parser.git", rev = "b996b21ffca562ecb2086f632a6a0b05c245c24a" }
rustpython-ast = { git = "https://github.com/astral-sh/RustPython-Parser.git", rev = "b996b21ffca562ecb2086f632a6a0b05c245c24a" , default-features = false, features = ["num-bigint"]}
rustpython-format = { git = "https://github.com/astral-sh/RustPython-Parser.git", rev = "b996b21ffca562ecb2086f632a6a0b05c245c24a", default-features = false, features = ["num-bigint"] }
rustpython-literal = { git = "https://github.com/astral-sh/RustPython-Parser.git", rev = "b996b21ffca562ecb2086f632a6a0b05c245c24a", default-features = false }
rustpython-parser = { git = "https://github.com/astral-sh/RustPython-Parser.git", rev = "b996b21ffca562ecb2086f632a6a0b05c245c24a" , default-features = false, features = ["full-lexer", "num-bigint"] }
[profile.release]
lto = "fat"
codegen-units = 1
@@ -73,7 +63,7 @@ opt-level = 3
# Reduce complexity of a parser function that would trigger a locals limit in a wasm tool.
# https://github.com/bytecodealliance/wasm-tools/blob/b5c3d98e40590512a3b12470ef358d5c7b983b15/crates/wasmparser/src/limits.rs#L29
[profile.dev.package.rustpython-parser]
[profile.dev.package.ruff_python_parser]
opt-level = 1
# Use the `--profile release-debug` flag to show symbols in release mode.

View File

@@ -2,7 +2,7 @@
# Ruff
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/charliermarsh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
[![image](https://img.shields.io/pypi/v/ruff.svg)](https://pypi.python.org/pypi/ruff)
[![image](https://img.shields.io/pypi/l/ruff.svg)](https://pypi.python.org/pypi/ruff)
[![image](https://img.shields.io/pypi/pyversions/ruff.svg)](https://pypi.python.org/pypi/ruff)
@@ -140,7 +140,7 @@ Ruff can also be used as a [pre-commit](https://pre-commit.com) hook:
```yaml
- repo: https://github.com/astral-sh/ruff-pre-commit
# Ruff version.
rev: v0.0.278
rev: v0.0.280
hooks:
- id: ruff
```
@@ -397,7 +397,7 @@ Ruff is used by a number of major open-source projects and companies, including:
- [PyTorch](https://github.com/pytorch/pytorch)
- [Pydantic](https://github.com/pydantic/pydantic)
- [Pylint](https://github.com/PyCQA/pylint)
- [Pynecone](https://github.com/pynecone-io/pynecone)
- [Reflex](https://github.com/reflex-dev/reflex)
- [Robyn](https://github.com/sansyrox/robyn)
- Scale AI ([Launch SDK](https://github.com/scaleapi/launch-python-client))
- Snowflake ([SnowCLI](https://github.com/Snowflake-Labs/snowcli))
@@ -424,13 +424,13 @@ Ruff is used by a number of major open-source projects and companies, including:
If you're using Ruff, consider adding the Ruff badge to project's `README.md`:
```md
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/charliermarsh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
```
...or `README.rst`:
```rst
.. image:: https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/charliermarsh/ruff/main/assets/badge/v2.json
.. image:: https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json
:target: https://github.com/astral-sh/ruff
:alt: Ruff
```
@@ -438,7 +438,7 @@ If you're using Ruff, consider adding the Ruff badge to project's `README.md`:
...or, as HTML:
```html
<a href="https://github.com/astral-sh/ruff"><img src="https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/charliermarsh/ruff/main/assets/badge/v2.json" alt="Ruff" style="max-width:100%;"></a>
<a href="https://github.com/astral-sh/ruff"><img src="https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json" alt="Ruff" style="max-width:100%;"></a>
```
## License
@@ -447,6 +447,6 @@ MIT
<div align="center">
<a target="_blank" href="https://astral.sh" style="background:none">
<img src="https://raw.githubusercontent.com/charliermarsh/ruff/main/assets/svg/Astral.svg">
<img src="https://raw.githubusercontent.com/astral-sh/ruff/main/assets/svg/Astral.svg">
</a>
</div>

View File

@@ -1,6 +1,6 @@
[package]
name = "flake8-to-ruff"
version = "0.0.278"
version = "0.0.280"
description = """
Convert Flake8 configuration files to Ruff configuration files.
"""

View File

@@ -1,6 +1,6 @@
[package]
name = "ruff"
version = "0.0.278"
version = "0.0.280"
publish = false
authors = { workspace = true }
edition = { workspace = true }
@@ -19,13 +19,16 @@ ruff_cache = { path = "../ruff_cache" }
ruff_diagnostics = { path = "../ruff_diagnostics", features = ["serde"] }
ruff_index = { path = "../ruff_index" }
ruff_macros = { path = "../ruff_macros" }
ruff_python_whitespace = { path = "../ruff_python_whitespace" }
ruff_python_ast = { path = "../ruff_python_ast", features = ["serde"] }
ruff_python_codegen = { path = "../ruff_python_codegen" }
ruff_python_index = { path = "../ruff_python_index" }
ruff_python_literal = { path = "../ruff_python_literal" }
ruff_python_semantic = { path = "../ruff_python_semantic" }
ruff_python_stdlib = { path = "../ruff_python_stdlib" }
ruff_rustpython = { path = "../ruff_rustpython" }
ruff_text_size = { workspace = true }
ruff_textwrap = { path = "../ruff_textwrap" }
ruff_python_trivia = { path = "../ruff_python_trivia" }
ruff_python_parser = { path = "../ruff_python_parser" }
ruff_source_file = { path = "../ruff_source_file", features = ["serde"] }
ruff_text_size = { path = "../ruff_text_size" }
annotate-snippets = { version = "0.9.1", features = ["color"] }
anyhow = { workspace = true }
@@ -45,7 +48,6 @@ libcst = { workspace = true }
log = { workspace = true }
memchr = { workspace = true }
natord = { version = "1.0.9" }
nohash-hasher = { workspace = true }
num-bigint = { workspace = true }
num-traits = { workspace = true }
once_cell = { workspace = true }
@@ -55,14 +57,13 @@ path-absolutize = { workspace = true, features = [
] }
pathdiff = { version = "0.2.1" }
pep440_rs = { version = "0.3.1", features = ["serde"] }
phf = { version = "0.11", features = ["macros"] }
pyproject-toml = { version = "0.6.0" }
quick-junit = { version = "0.3.2" }
regex = { workspace = true }
result-like = { version = "0.4.6" }
rustc-hash = { workspace = true }
rustpython-format = { workspace = true }
rustpython-parser = { workspace = true }
schemars = { workspace = true, optional = true }
semver = { version = "1.0.16" }
serde = { workspace = true }
@@ -86,6 +87,7 @@ pretty_assertions = "1.3.0"
test-case = { workspace = true }
# Disable colored output in tests
colored = { workspace = true, features = ["no-color"] }
tempfile = "3.6.0"
[features]
default = []

View File

@@ -1,6 +1,6 @@
"""
Should emit:
B002 - on lines 15 and 20
B002 - on lines 18, 19, and 24
"""
@@ -8,13 +8,17 @@ def this_is_all_fine(n):
x = n + 1
y = 1 + n
z = +x + y
return +z
a = n - 1
b = 1 - n
c = -a - b
return +z, -c
def this_is_buggy(n):
x = ++n
return x
y = --n
return x, y
def this_is_buggy_too(n):
return ++n
return ++n, --n

View File

@@ -97,3 +97,10 @@ def f():
# variable name).
for line_ in range(self.header_lines):
fp.readline()
# Regression test: visitor didn't walk the elif test
for key, value in current_crawler_tags.items():
if key:
pass
elif wanted_tag_value != value:
pass

View File

@@ -17,3 +17,37 @@ from typing import TypedDict
class MyClass(TypedDict):
id: int
from threading import Event
class CustomEvent(Event):
def set(self) -> None:
...
def str(self) -> None:
...
from logging import Filter, LogRecord
class CustomFilter(Filter):
def filter(self, record: LogRecord) -> bool:
...
def str(self) -> None:
...
from typing_extensions import override
class MyClass:
@override
def str(self):
pass
def int(self):
pass

View File

@@ -0,0 +1,2 @@
#!/usr/bin/env python

View File

@@ -50,3 +50,12 @@ _ = """a""" "b"
_ = 'a' "b"
_ = rf"a" rf"b"
# Single-line explicit concatenation should be ignored.
_ = "abc" + "def" + "ghi"
_ = foo + "abc" + "def"
_ = "abc" + foo + "def"
_ = "abc" + "def" + foo
_ = foo + bar + "abc"
_ = "abc" + foo + bar
_ = foo + "abc" + bar

View File

@@ -1,5 +1,10 @@
import logging
from distutils import log
from logging_setup import logger
logging.warn("Hello World!")
log.warn("Hello world!") # This shouldn't be considered as a logger candidate
logger.warn("Hello world!")
logging . warn("Hello World!")

View File

@@ -0,0 +1,14 @@
var: int
a = var # OK
b = c = int # OK
a.b = int # OK
d, e = int, str # OK
f, g, h = int, str, TypeVar("T") # OK
i: TypeAlias = int | str # OK
j: TypeAlias = int # OK

View File

@@ -0,0 +1,14 @@
var: int
a = var # OK
b = c = int # PYI017
a.b = int # PYI017
d, e = int, str # PYI017
f, g, h = int, str, TypeVar("T") # PYI017
i: TypeAlias = int | str # OK
j: TypeAlias = int # OK

View File

@@ -0,0 +1,12 @@
import typing
from typing import TypeVar
_T = typing.TypeVar("_T")
_P = TypeVar("_P")
# OK
_UsedTypeVar = TypeVar("_UsedTypeVar")
def func(arg: _UsedTypeVar) -> _UsedTypeVar: ...
_A, _B = TypeVar("_A"), TypeVar("_B")
_C = _D = TypeVar("_C")

View File

@@ -0,0 +1,12 @@
import typing
from typing import TypeVar
_T = typing.TypeVar("_T")
_P = TypeVar("_P")
# OK
_UsedTypeVar = TypeVar("_UsedTypeVar")
def func(arg: _UsedTypeVar) -> _UsedTypeVar: ...
_A, _B = TypeVar("_A"), TypeVar("_B")
_C = _D = TypeVar("_C")

View File

@@ -0,0 +1,19 @@
import typing
from typing import TypeAlias, Literal, Any
NewAny = Any
OptionalStr = typing.Optional[str]
Foo = Literal["foo"]
IntOrStr = int | str
AliasNone = None
NewAny: typing.TypeAlias = Any
OptionalStr: TypeAlias = typing.Optional[str]
Foo: typing.TypeAlias = Literal["foo"]
IntOrStr: TypeAlias = int | str
IntOrFloat: Foo = int | float
AliasNone: typing.TypeAlias = None
# these are ok
VarAlias = str
AliasFoo = Foo

View File

@@ -0,0 +1,18 @@
from typing import Literal, Any
NewAny = Any
OptionalStr = typing.Optional[str]
Foo = Literal["foo"]
IntOrStr = int | str
AliasNone = None
NewAny: typing.TypeAlias = Any
OptionalStr: TypeAlias = typing.Optional[str]
Foo: typing.TypeAlias = Literal["foo"]
IntOrStr: TypeAlias = int | str
IntOrFloat: Foo = int | float
AliasNone: typing.TypeAlias = None
# these are ok
VarAlias = str
AliasFoo = Foo

View File

@@ -0,0 +1,18 @@
import typing
from typing import Protocol
class _Foo(Protocol):
bar: int
class _Bar(typing.Protocol):
bar: int
# OK
class _UsedPrivateProtocol(Protocol):
bar: int
def uses__UsedPrivateProtocol(arg: _UsedPrivateProtocol) -> None: ...

View File

@@ -0,0 +1,18 @@
import typing
from typing import Protocol
class _Foo(object, Protocol):
bar: int
class _Bar(typing.Protocol):
bar: int
# OK
class _UsedPrivateProtocol(Protocol):
bar: int
def uses__UsedPrivateProtocol(arg: _UsedPrivateProtocol) -> None: ...

View File

@@ -0,0 +1,22 @@
import typing
import sys
from typing import TypeAlias
_UnusedPrivateTypeAlias: TypeAlias = int | None
_T: typing.TypeAlias = str
# OK
_UsedPrivateTypeAlias: TypeAlias = int | None
def func(arg: _UsedPrivateTypeAlias) -> _UsedPrivateTypeAlias:
...
if sys.version_info > (3, 9):
_PrivateTypeAlias: TypeAlias = str | None
else:
_PrivateTypeAlias: TypeAlias = float | None
def func2(arg: _PrivateTypeAlias) -> None: ...

View File

@@ -0,0 +1,22 @@
import typing
import sys
from typing import TypeAlias
_UnusedPrivateTypeAlias: TypeAlias = int | None
_T: typing.TypeAlias = str
# OK
_UsedPrivateTypeAlias: TypeAlias = int | None
def func(arg: _UsedPrivateTypeAlias) -> _UsedPrivateTypeAlias:
...
if sys.version_info > (3, 9):
_PrivateTypeAlias: TypeAlias = str | None
else:
_PrivateTypeAlias: TypeAlias = float | None
def func2(arg: _PrivateTypeAlias) -> None: ...

View File

@@ -0,0 +1,18 @@
import typing
from typing import TypedDict
class _UnusedTypedDict(TypedDict):
foo: str
class _UnusedTypedDict2(typing.TypedDict):
bar: int
class _UsedTypedDict(TypedDict):
foo: bytes
class _CustomClass(_UsedTypedDict):
bar: list[int]

View File

@@ -0,0 +1,32 @@
import sys
import typing
from typing import TypedDict
class _UnusedTypedDict(TypedDict):
foo: str
class _UnusedTypedDict2(typing.TypedDict):
bar: int
# OK
class _UsedTypedDict(TypedDict):
foo: bytes
class _CustomClass(_UsedTypedDict):
bar: list[int]
if sys.version_info >= (3, 10):
class _UsedTypedDict2(TypedDict):
foo: int
else:
class _UsedTypedDict2(TypedDict):
foo: float
class _CustomClass2(_UsedTypedDict2):
bar: list[int]

View File

@@ -0,0 +1,12 @@
__all__ = ["A", "B", "C"]
# Errors
__all__.append("D")
__all__.extend(["E", "Foo"])
__all__.remove("A")
# OK
__all__ += ["D"]
foo = ["Hello"]
foo.append("World")
foo.bar.append("World")

View File

@@ -0,0 +1,12 @@
__all__ = ["A", "B", "C"]
# Errors
__all__.append("D")
__all__.extend(["E", "Foo"])
__all__.remove("A")
# OK
__all__ += ["D"]
foo = ["Hello"]
foo.append("World")
foo.bar.append("World")

View File

@@ -11,6 +11,10 @@ async def test_ok_trivial_with():
with context_manager_under_test():
pass
with pytest.raises(ValueError):
with context_manager_under_test():
raise ValueError
with pytest.raises(AttributeError):
async with context_manager_under_test():
pass
@@ -24,6 +28,16 @@ def test_ok_complex_single_call():
)
def test_ok_func_and_class():
with pytest.raises(AttributeError):
class A:
pass
with pytest.raises(AttributeError):
def f():
pass
def test_error_multiple_statements():
with pytest.raises(AttributeError):
len([])
@@ -47,13 +61,10 @@ async def test_error_complex_statement():
while True:
[].size
with pytest.raises(AttributeError):
with context_manager_under_test():
[].size
with pytest.raises(AttributeError):
async with context_manager_under_test():
[].size
if True:
raise Exception
def test_error_try():

View File

@@ -100,6 +100,14 @@ if node.module0123456789:
):
print("Bad module!")
# SIM102
# Regression test for https://github.com/apache/airflow/blob/145b16caaa43f0c42bffd97344df916c602cddde/airflow/configuration.py#L1161
if a:
if b:
if c:
print("if")
elif d:
print("elif")
# OK
if a:
@@ -148,3 +156,12 @@ if False:
if True:
if a:
pass
# SIM102
def f():
if a:
pass
elif b:
if c:
d

View File

@@ -14,6 +14,12 @@ try:
except (ValueError, OSError):
pass
# SIM105
try:
foo()
except (ValueError, OSError) as e:
pass
# SIM105
try:
foo()
@@ -94,3 +100,13 @@ def with_comment():
foo()
except (ValueError, OSError):
pass # Trailing comment.
try:
print()
except ("not", "an", "exception"):
pass
try:
print()
except "not an exception":
pass

View File

@@ -23,7 +23,7 @@ elif a:
else:
b = 2
# OK (false negative)
# SIM108
if True:
pass
else:

View File

@@ -94,3 +94,23 @@ if result.eofs == "F":
errors = 1
else:
errors = 1
if a:
# Ignore branches with diverging comments because it means we're repeating
# the bodies because we have different reasons for each branch
x = 1
elif c:
x = 1
def foo():
a = True
b = False
if a > b: # end-of-line
return 3
elif a == b:
return 3
elif a < b: # end-of-line
return 4
elif b is None:
return 4

View File

@@ -1,7 +1,15 @@
import contextlib
import pathlib
import pathlib as pl
from pathlib import Path
from pathlib import Path as P
# SIM115
f = open("foo.txt")
f = Path("foo.txt").open()
f = pathlib.Path("foo.txt").open()
f = pl.Path("foo.txt").open()
f = P("foo.txt").open()
data = f.read()
f.close()

View File

@@ -84,3 +84,15 @@ elif func_name == "remove":
return "D"
elif func_name == "move":
return "MV"
# OK
def no_return_in_else(platform):
if platform == "linux":
return "auditwheel repair -w {dest_dir} {wheel}"
elif platform == "macos":
return "delocate-wheel --require-archs {delocate_archs} -w {dest_dir} -v {wheel}"
elif platform == "windows":
return ""
else:
msg = f"Unknown platform: {platform!r}"
raise ValueError(msg)

View File

@@ -1,11 +1,19 @@
key in obj.keys() # SIM118
key not in obj.keys() # SIM118
foo["bar"] in obj.keys() # SIM118
foo["bar"] not in obj.keys() # SIM118
foo['bar'] in obj.keys() # SIM118
foo['bar'] not in obj.keys() # SIM118
foo() in obj.keys() # SIM118
foo() not in obj.keys() # SIM118
for key in obj.keys(): # SIM118
pass
@@ -22,3 +30,13 @@ for key in list(obj.keys()):
(k for k in obj.keys()) # SIM118
key in (obj or {}).keys() # SIM118
from typing import KeysView
class Foo:
def keys(self) -> KeysView[object]:
...
def __contains__(self, key: object) -> bool:
return key in self.keys() # OK

View File

@@ -38,6 +38,15 @@ if key in a_dict:
else:
vars[idx] = "defaultß9💣26789ß9💣26789ß9💣26789ß9💣26789ß9💣26789"
# SIM401
if foo():
pass
else:
if key in a_dict:
vars[idx] = a_dict[key]
else:
vars[idx] = "default"
###
# Negative cases
###
@@ -105,12 +114,3 @@ elif key in a_dict:
vars[idx] = a_dict[key]
else:
vars[idx] = "default"
# OK (false negative for nested else)
if foo():
pass
else:
if key in a_dict:
vars[idx] = a_dict[key]
else:
vars[idx] = "default"

View File

@@ -0,0 +1,15 @@
from pathlib import Path, PurePath
from pathlib import Path as pth
# match
_ = Path(".")
_ = pth(".")
_ = PurePath(".")
_ = Path("")
# no match
_ = Path()
print(".")
Path("file.txt")
Path(".", "folder")
PurePath(".", "folder")

View File

@@ -0,0 +1,14 @@
import os.path
from pathlib import Path
from os.path import getsize
os.path.getsize("filename")
os.path.getsize(b"filename")
os.path.getsize(Path("filename"))
os.path.getsize(__file__)
getsize("filename")
getsize(b"filename")
getsize(Path("filename"))
getsize(__file__)

View File

@@ -0,0 +1,12 @@
import os.path
from pathlib import Path
from os.path import getatime
os.path.getatime("filename")
os.path.getatime(b"filename")
os.path.getatime(Path("filename"))
getatime("filename")
getatime(b"filename")
getatime(Path("filename"))

View File

@@ -0,0 +1,13 @@
import os.path
from pathlib import Path
from os.path import getmtime
os.path.getmtime("filename")
os.path.getmtime(b"filename")
os.path.getmtime(Path("filename"))
getmtime("filename")
getmtime(b"filename")
getmtime(Path("filename"))

View File

@@ -0,0 +1,12 @@
import os.path
from pathlib import Path
from os.path import getctime
os.path.getctime("filename")
os.path.getctime(b"filename")
os.path.getctime(Path("filename"))
getctime("filename")
getctime(b"filename")
getctime(Path("filename"))

View File

@@ -0,0 +1,20 @@
import os
from os import sep
file_name = "foo/bar"
# PTH206
"foo/bar/".split(os.sep)
"foo/bar/".split(sep=os.sep)
"foo/bar/".split(os.sep)[-1]
"foo/bar/".split(os.sep)[-2]
"foo/bar/".split(os.sep)[-2:]
"fizz/buzz".split(sep)
"fizz/buzz".split(sep)[-1]
os.path.splitext("path/to/hello_world.py")[0].split(os.sep)[-1]
file_name.split(os.sep)
(os.path.abspath(file_name)).split(os.sep)
# OK
"foo/bar/".split("/")

View File

@@ -0,0 +1,11 @@
import os
import glob
from glob import glob as search
extensions_dir = "./extensions"
# PTH207
glob.glob(os.path.join(extensions_dir, "ops", "autograd", "*.cpp"))
list(glob.iglob(os.path.join(extensions_dir, "ops", "autograd", "*.cpp")))
search("*.png")

View File

@@ -2,6 +2,7 @@ import os
import os.path
p = "/foo"
q = "bar"
a = os.path.abspath(p)
aa = os.chmod(p)
@@ -21,7 +22,9 @@ bbbbb = os.path.islink(p)
os.readlink(p)
os.stat(p)
os.path.isabs(p)
os.path.join(p)
os.path.join(p, q)
os.sep.join([p, q])
os.sep.join((p, q))
os.path.basename(p)
os.path.dirname(p)
os.path.samefile(p)

View File

@@ -2,6 +2,7 @@ import os as foo
import os.path as foo_p
p = "/foo"
q = "bar"
a = foo_p.abspath(p)
aa = foo.chmod(p)
@@ -21,7 +22,9 @@ bbbbb = foo_p.islink(p)
foo.readlink(p)
foo.stat(p)
foo_p.isabs(p)
foo_p.join(p)
foo_p.join(p, q)
foo.sep.join([p, q])
foo.sep.join((p, q))
foo_p.basename(p)
foo_p.dirname(p)
foo_p.samefile(p)

View File

@@ -1,9 +1,10 @@
from os import chmod, mkdir, makedirs, rename, replace, rmdir
from os import chmod, mkdir, makedirs, rename, replace, rmdir, sep
from os import remove, unlink, getcwd, readlink, stat
from os.path import abspath, exists, expanduser, isdir, isfile, islink
from os.path import isabs, join, basename, dirname, samefile, splitext
p = "/foo"
q = "bar"
a = abspath(p)
aa = chmod(p)
@@ -23,7 +24,9 @@ bbbbb = islink(p)
readlink(p)
stat(p)
isabs(p)
join(p)
join(p, q)
sep.join((p, q))
sep.join([p, q])
basename(p)
dirname(p)
samefile(p)

View File

@@ -1,4 +1,4 @@
from os import chmod as xchmod, mkdir as xmkdir
from os import chmod as xchmod, mkdir as xmkdir, sep as s
from os import makedirs as xmakedirs, rename as xrename, replace as xreplace
from os import rmdir as xrmdir, remove as xremove, unlink as xunlink
from os import getcwd as xgetcwd, readlink as xreadlink, stat as xstat
@@ -9,6 +9,7 @@ from os.path import join as xjoin, basename as xbasename, dirname as xdirname
from os.path import samefile as xsamefile, splitext as xsplitext
p = "/foo"
q = "bar"
a = xabspath(p)
aa = xchmod(p)
@@ -28,7 +29,9 @@ bbbbb = xislink(p)
xreadlink(p)
xstat(p)
xisabs(p)
xjoin(p)
xjoin(p, q)
s.join((p, q))
s.join([p, q])
xbasename(p)
xdirname(p)
xsamefile(p)

View File

@@ -0,0 +1,2 @@
import bar
import foo

View File

@@ -0,0 +1,2 @@
import foo
import bar

View File

@@ -0,0 +1,7 @@
if "sdist" in cmds:
_sdist = cmds["sdist"]
elif "setuptools" in sys.modules:
from setuptools.command.sdist import sdist as _sdist
else:
from setuptools.command.sdist import sdist as _sdist
from distutils.command.sdist import sdist as _sdist

View File

@@ -0,0 +1,7 @@
match 1:
case 1:
import sys
import os
case 2:
import collections
import abc

View File

@@ -1 +1 @@
broken "§=($/=(")
broken "§=($/=()

View File

@@ -1,28 +1,23 @@
# PERF203
for i in range(10):
try: # PERF203
try:
print(f"{i}")
except:
print("error")
# OK
try:
for i in range(10):
print(f"{i}")
except:
print("error")
# OK
i = 0
while i < 10: # PERF203
while i < 10:
try:
print(f"{i}")
except:
print("error")
i += 1
try:
i = 0
while i < 10:
print(f"{i}")
i += 1
except:
print("error")

View File

@@ -0,0 +1,13 @@
#: E241
a = (1, 2)
#: Okay
b = (1, 20)
#: E242
a = (1, 2) # tab before 2
#: Okay
b = (1, 20) # space before 20
#: E241 E241 E241
# issue 135
more_spaces = [a, b,
ef, +h,
c, -d]

View File

@@ -41,3 +41,5 @@ regex = '\w' # noqa
regex = '''
\w
''' # noqa
regex = '\\\_'

View File

@@ -23,3 +23,5 @@ a = []
'%s %s' % (*a,)
k = {}
'%(k)s' % {**k}
'%s' % [1, 2, 3]
'%s' % {1, 2, 3}

View File

@@ -1,6 +1,11 @@
if (1, 2):
pass
if (3, 4):
pass
elif foo:
pass
for _ in range(5):
if True:
pass

View File

@@ -0,0 +1,15 @@
# Regression test for branch detection from
# https://github.com/pypa/build/blob/5800521541e5e749d4429617420d1ef8cdb40b46/src/build/_importlib.py
import sys
if sys.version_info < (3, 8):
import importlib_metadata as metadata
elif sys.version_info < (3, 9, 10) or (3, 10, 0) <= sys.version_info < (3, 10, 2):
try:
import importlib_metadata as metadata
except ModuleNotFoundError:
from importlib import metadata
else:
from importlib import metadata
__all__ = ["metadata"]

View File

@@ -0,0 +1,6 @@
class Class:
def func(self):
pass
def func(self):
pass

View File

@@ -0,0 +1,11 @@
"""Test case: `Literal` with `__future__` annotations."""
from __future__ import annotations
from typing import Literal, Final
from typing_extensions import assert_type
CONSTANT: Final = "ns"
assert_type(CONSTANT, Literal["ns"])

View File

@@ -0,0 +1,103 @@
"""Test type parameters and aliases"""
# Type parameters in type alias statements
from some_module import Bar
type Foo[T] = T # OK
type Foo[T] = list[T] # OK
type Foo[T: ForwardA] = T # OK
type Foo[*Ts] = Bar[Ts] # OK
type Foo[**P] = Bar[P] # OK
class ForwardA: ...
# Types used in aliased assignment must exist
type Foo = DoesNotExist # F821: Undefined name `DoesNotExist`
type Foo = list[DoesNotExist] # F821: Undefined name `DoesNotExist`
# Type parameters do not escape alias scopes
type Foo[T] = T
T # F821: Undefined name `T` - not accessible afterward alias scope
# Type parameters in functions
def foo[T](t: T) -> T: return t # OK
async def afoo[T](t: T) -> T: return t # OK
def with_forward_ref[T: ForwardB](t: T) -> T: return t # OK
def can_access_inside[T](t: T) -> T: # OK
print(T) # OK
return t # OK
class ForwardB: ...
# Type parameters do not escape function scopes
from some_library import some_decorator
@some_decorator(T) # F821: Undefined name `T` - not accessible in decorators
def foo[T](t: T) -> None: ...
T # F821: Undefined name `T` - not accessible afterward function scope
# Type parameters in classes
class Foo[T](list[T]): ... # OK
class UsesForward[T: ForwardC](list[T]): ... # OK
class ForwardC: ...
class WithinBody[T](list[T]): # OK
t = T # OK
x: T # OK
def foo(self, x: T) -> T: # OK
return x
def foo(self):
T # OK
# Type parameters do not escape class scopes
from some_library import some_decorator
@some_decorator(T) # F821: Undefined name `T` - not accessible in decorators
class Foo[T](list[T]): ...
T # F821: Undefined name `T` - not accessible after class scope
# Types specified in bounds should exist
type Foo[T: DoesNotExist] = T # F821: Undefined name `DoesNotExist`
def foo[T: DoesNotExist](t: T) -> T: return t # F821: Undefined name `DoesNotExist`
class Foo[T: DoesNotExist](list[T]): ... # F821: Undefined name `DoesNotExist`
type Foo[T: (DoesNotExist1, DoesNotExist2)] = T # F821: Undefined name `DoesNotExist1`, Undefined name `DoesNotExist2`
def foo[T: (DoesNotExist1, DoesNotExist2)](t: T) -> T: return t # F821: Undefined name `DoesNotExist1`, Undefined name `DoesNotExist2`
class Foo[T: (DoesNotExist1, DoesNotExist2)](list[T]): ... # F821: Undefined name `DoesNotExist1`, Undefined name `DoesNotExist2`
# Type parameters in nested classes
class Parent[T]:
t = T # OK
def can_use_class_variable(self, x: t) -> t: # OK
return x
class Child:
def can_access_parent_type_parameter(self, x: T) -> T: # OK
T # OK
return x
def cannot_access_parent_variable(self, x: t) -> t: # F821: Undefined name `T`
t # F821: Undefined name `t`
return x
# Type parameters in nested functions
def can_access_inside_nested[T](t: T) -> T: # OK
def bar(x: T) -> T: # OK
T # OK
return x
bar(t)

View File

@@ -25,3 +25,48 @@ def dec(x):
def f():
dec = 1
return dec
class Class:
def f(self):
print(my_var)
my_var = 1
class Class:
my_var = 0
def f(self):
print(my_var)
my_var = 1
import sys
def main():
print(sys.argv)
try:
3 / 0
except ZeroDivisionError:
import sys
sys.exit(1)
import sys
def main():
print(sys.argv)
for sys in range(5):
pass
import requests_mock as rm
def requests_mock(requests_mock: rm.Mocker):
print(rm.ANY)

View File

@@ -1,6 +1,7 @@
x = 1 # type: ignore
x = 1 # type:ignore
x = 1 # type: ignore[attr-defined] # type: ignore
x = 1 # type: ignoreme # type: ignore
x = 1
x = 1 # type ignore

View File

@@ -47,3 +47,17 @@ def not_ok1():
pass
else:
pass
# Regression test for https://github.com/apache/airflow/blob/f1e1cdcc3b2826e68ba133f350300b5065bbca33/airflow/models/dag.py#L1737
def not_ok2():
if True:
print(1)
elif True:
print(2)
else:
if True:
print(3)
else:
print(4)

View File

@@ -0,0 +1,16 @@
class Person:
def __init__(self):
self.name = "monty"
def __eq__(self, other):
return isinstance(other, Person) and other.name == self.name
class Language:
def __init__(self):
self.name = "python"
def __eq__(self, other):
return isinstance(other, Language) and other.name == self.name
def __hash__(self):
return hash(self.name)

View File

@@ -0,0 +1,41 @@
foo = 1
bar = 2
baz = 3
# Errors.
foo = foo
bar = bar
foo, bar = foo, bar
bar, foo = bar, foo
(foo, bar) = (foo, bar)
(bar, foo) = (bar, foo)
foo, (bar, baz) = foo, (bar, baz)
bar, (foo, baz) = bar, (foo, baz)
(foo, bar), baz = (foo, bar), baz
(foo, (bar, baz)) = (foo, (bar, baz))
foo, bar = foo, 1
bar, foo = bar, 1
(foo, bar) = (foo, 1)
(bar, foo) = (bar, 1)
foo, (bar, baz) = foo, (bar, 1)
bar, (foo, baz) = bar, (foo, 1)
(foo, bar), baz = (foo, bar), 1
(foo, (bar, baz)) = (foo, (bar, 1))
foo: int = foo
bar: int = bar
# Non-errors.
foo = bar
bar = foo
foo, bar = bar, foo
foo, bar = bar, foo
(foo, bar) = (bar, foo)
foo, bar = bar, 1
bar, foo = foo, 1
foo: int = bar
bar: int = 1
class Foo:
foo = foo
bar = bar

View File

@@ -0,0 +1,18 @@
import subprocess
def foo():
pass
# Errors.
subprocess.Popen(preexec_fn=foo)
subprocess.Popen(["ls"], preexec_fn=foo)
subprocess.Popen(preexec_fn=lambda: print("Hello, world!"))
subprocess.Popen(["ls"], preexec_fn=lambda: print("Hello, world!"))
# Non-errors.
subprocess.Popen()
subprocess.Popen(["ls"])
subprocess.Popen(preexec_fn=None) # None is the default.
subprocess.Popen(["ls"], preexec_fn=None) # None is the default.

View File

@@ -0,0 +1,4 @@
print('Hello world')
#!/usr/bin/python
# -*- coding: utf-8 -*-

View File

@@ -15,7 +15,22 @@ bytes("foo", **a)
bytes(b"foo"
b"bar")
bytes("foo")
bytes(1)
f"{f'{str()}'}"
int(1.0)
int("1")
int(b"11")
int(10, base=2)
int("10", base=2)
int("10", 2)
float("1.0")
float(b"1.0")
bool(1)
bool(0)
bool("foo")
bool("")
bool(b"")
bool(1.0)
# These become string or byte literals
str()
@@ -27,3 +42,10 @@ bytes(b"foo")
bytes(b"""
foo""")
f"{str()}"
int()
int(1)
float()
float(1.0)
bool()
bool(True)
bool(False)

View File

@@ -62,6 +62,16 @@ print("foo {} ".format(x))
1111111111111111111111111111111111111111111111111111111111111111111111111,
)
"""
{}
""".format(1)
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa = """
{}
""".format(
111111
)
###
# Non-errors
###
@@ -99,6 +109,37 @@ r'"\N{snowman} {}".format(a)'
11111111111111111111111111111111111111111111111111111111111111111111111111,
)
"""
{}
{}
{}
""".format(
1,
2,
111111111111111111111111111111111111111111111111111111111111111111111111111111111111111,
)
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa = """{}
""".format(
111111
)
"{}".format(
[
1,
2,
3,
]
)
"{a}".format(
a=[
1,
2,
3,
]
)
async def c():
return "{}".format(await 3)

View File

@@ -7,20 +7,20 @@ if True:
if True:
if foo:
pass
print()
elif sys.version_info < (3, 3):
cmd = [sys.executable, "-m", "test.regrtest"]
if True:
if foo:
pass
print()
elif sys.version_info < (3, 3):
cmd = [sys.executable, "-m", "test.regrtest"]
elif foo:
cmd = [sys.executable, "-m", "test", "-j0"]
if foo:
pass
print()
elif sys.version_info < (3, 3):
cmd = [sys.executable, "-m", "test.regrtest"]
@@ -28,7 +28,7 @@ if True:
cmd = [sys.executable, "-m", "test.regrtest"]
if foo:
pass
print()
elif sys.version_info < (3, 3):
cmd = [sys.executable, "-m", "test.regrtest"]
else:

View File

@@ -2,21 +2,9 @@ x = range(10)
# RUF015
list(x)[0]
list(x)[:1]
list(x)[:1:1]
list(x)[:1:2]
tuple(x)[0]
tuple(x)[:1]
tuple(x)[:1:1]
tuple(x)[:1:2]
list(i for i in x)[0]
list(i for i in x)[:1]
list(i for i in x)[:1:1]
list(i for i in x)[:1:2]
[i for i in x][0]
[i for i in x][:1]
[i for i in x][:1:1]
[i for i in x][:1:2]
# OK (not indexing (solely) the first element)
list(x)
@@ -29,16 +17,31 @@ list(x)[::]
[i for i in x]
[i for i in x][1]
[i for i in x][-1]
[i for i in x][:1]
[i for i in x][:1:1]
[i for i in x][:1:2]
[i for i in x][1:]
[i for i in x][:3:2]
[i for i in x][::2]
[i for i in x][::]
# OK (doesn't mirror the underlying list)
# RUF015 (doesn't mirror the underlying list)
[i + 1 for i in x][0]
[i for i in x if i > 5][0]
[(i, i + 1) for i in x][0]
# OK (multiple generators)
# RUF015 (multiple generators)
y = range(10)
[i + j for i in x for j in y][0]
# RUF015
list(range(10))[0]
list(x.y)[0]
list(x["y"])[0]
# RUF015 (multi-line)
revision_heads_map_ast = [
a
for a in revision_heads_map_ast_obj.body
if isinstance(a, ast.Assign) and a.targets[0].id == "REVISION_HEADS_MAP"
][0]

View File

@@ -0,0 +1,5 @@
import os # ruff: noqa: F401
def f():
x = 1

View File

@@ -230,6 +230,15 @@ def incorrect_multi_conditional(arg1, arg2):
raise Exception("...") # should be typeerror
def multiple_is_instance_checks(some_arg):
if isinstance(some_arg, str):
pass
elif isinstance(some_arg, int):
pass
else:
raise Exception("...") # should be typeerror
class MyCustomTypeValidation(Exception):
pass
@@ -296,6 +305,17 @@ def multiple_ifs(some_args):
pass
def else_body(obj):
if isinstance(obj, datetime.timedelta):
return "TimeDelta"
elif isinstance(obj, relativedelta.relativedelta):
return "RelativeDelta"
elif isinstance(obj, CronExpression):
return "CronExpression"
else:
raise Exception(f"Unknown object type: {obj.__class__.__name__}")
def early_return():
if isinstance(this, some_type):
if x in this:

View File

@@ -4,9 +4,10 @@ use anyhow::{bail, Result};
use libcst_native::{
Codegen, CodegenState, ImportNames, ParenthesizableWhitespace, SmallStatement, Statement,
};
use rustpython_parser::ast::{Ranged, Stmt};
use ruff_python_ast::{Ranged, Stmt};
use ruff_python_ast::source_code::{Locator, Stylist};
use ruff_python_codegen::Stylist;
use ruff_source_file::Locator;
use crate::cst::helpers::compose_module_path;
use crate::cst::matchers::match_statement;
@@ -32,7 +33,7 @@ impl<'a, T: Codegen<'a>> CodegenStylist<'a> for T {
///
/// Returns `Ok(None)` if the statement is empty after removing the imports.
pub(crate) fn remove_imports<'a>(
imports: impl Iterator<Item = &'a str>,
member_names: impl Iterator<Item = &'a str>,
stmt: &Stmt,
locator: &Locator,
stylist: &Stylist,
@@ -44,27 +45,20 @@ pub(crate) fn remove_imports<'a>(
bail!("Expected Statement::Simple");
};
let (aliases, import_module) = match body.body.first_mut() {
Some(SmallStatement::Import(import_body)) => (&mut import_body.names, None),
let aliases = match body.body.first_mut() {
Some(SmallStatement::Import(import_body)) => &mut import_body.names,
Some(SmallStatement::ImportFrom(import_body)) => {
if let ImportNames::Aliases(names) = &mut import_body.names {
(
names,
Some((&import_body.relative, import_body.module.as_ref())),
)
names
} else if let ImportNames::Star(..) = &import_body.names {
// Special-case: if the import is a `from ... import *`, then we delete the
// entire statement.
let mut found_star = false;
for import in imports {
let qualified_name = match import_body.module.as_ref() {
Some(module_name) => format!("{}.*", compose_module_path(module_name)),
None => "*".to_string(),
};
if import == qualified_name {
for member in member_names {
if member == "*" {
found_star = true;
} else {
bail!("Expected \"*\" for unused import (got: \"{}\")", import);
bail!("Expected \"*\" for unused import (got: \"{}\")", member);
}
}
if !found_star {
@@ -81,30 +75,10 @@ pub(crate) fn remove_imports<'a>(
// Preserve the trailing comma (or not) from the last entry.
let trailing_comma = aliases.last().and_then(|alias| alias.comma.clone());
for import in imports {
let alias_index = aliases.iter().position(|alias| {
let qualified_name = match import_module {
Some((relative, module)) => {
let module = module.map(compose_module_path);
let member = compose_module_path(&alias.name);
let mut qualified_name = String::with_capacity(
relative.len() + module.as_ref().map_or(0, String::len) + member.len() + 1,
);
for _ in 0..relative.len() {
qualified_name.push('.');
}
if let Some(module) = module {
qualified_name.push_str(&module);
qualified_name.push('.');
}
qualified_name.push_str(&member);
qualified_name
}
None => compose_module_path(&alias.name),
};
qualified_name == import
});
for member in member_names {
let alias_index = aliases
.iter()
.position(|alias| member == compose_module_path(&alias.name));
if let Some(index) = alias_index {
aliases.remove(index);
}
@@ -138,7 +112,7 @@ pub(crate) fn remove_imports<'a>(
///
/// Returns the modified import statement.
pub(crate) fn retain_imports(
imports: &[&str],
member_names: &[&str],
stmt: &Stmt,
locator: &Locator,
stylist: &Stylist,
@@ -150,14 +124,11 @@ pub(crate) fn retain_imports(
bail!("Expected Statement::Simple");
};
let (aliases, import_module) = match body.body.first_mut() {
Some(SmallStatement::Import(import_body)) => (&mut import_body.names, None),
let aliases = match body.body.first_mut() {
Some(SmallStatement::Import(import_body)) => &mut import_body.names,
Some(SmallStatement::ImportFrom(import_body)) => {
if let ImportNames::Aliases(names) = &mut import_body.names {
(
names,
Some((&import_body.relative, import_body.module.as_ref())),
)
names
} else {
bail!("Expected: ImportNames::Aliases");
}
@@ -169,28 +140,9 @@ pub(crate) fn retain_imports(
let trailing_comma = aliases.last().and_then(|alias| alias.comma.clone());
aliases.retain(|alias| {
imports.iter().any(|import| {
let qualified_name = match import_module {
Some((relative, module)) => {
let module = module.map(compose_module_path);
let member = compose_module_path(&alias.name);
let mut qualified_name = String::with_capacity(
relative.len() + module.as_ref().map_or(0, String::len) + member.len() + 1,
);
for _ in 0..relative.len() {
qualified_name.push('.');
}
if let Some(module) = module {
qualified_name.push_str(&module);
qualified_name.push('.');
}
qualified_name.push_str(&member);
qualified_name
}
None => compose_module_path(&alias.name),
};
qualified_name == *import
})
member_names
.iter()
.any(|member| *member == compose_module_path(&alias.name))
});
// But avoid destroying any trailing comments.

View File

@@ -1,13 +1,14 @@
//! Interface for generating autofix edits from higher-level actions (e.g., "remove an argument").
use anyhow::{bail, Result};
use ruff_python_ast::{self as ast, ExceptHandler, Expr, Keyword, Ranged, Stmt};
use ruff_python_parser::{lexer, Mode};
use ruff_text_size::{TextLen, TextRange, TextSize};
use rustpython_parser::ast::{self, ExceptHandler, Expr, Keyword, Ranged, Stmt};
use rustpython_parser::{lexer, Mode};
use ruff_diagnostics::Edit;
use ruff_python_ast::helpers;
use ruff_python_ast::source_code::{Indexer, Locator, Stylist};
use ruff_python_whitespace::{is_python_whitespace, NewlineWithTrailingNewline, PythonWhitespace};
use ruff_python_codegen::Stylist;
use ruff_python_index::Indexer;
use ruff_python_trivia::{has_leading_content, is_python_whitespace, PythonWhitespace};
use ruff_source_file::{Locator, NewlineWithTrailingNewline};
use crate::autofix::codemods;
@@ -41,11 +42,9 @@ pub(crate) fn delete_stmt(
if let Some(semicolon) = trailing_semicolon(stmt.end(), locator) {
let next = next_stmt_break(semicolon, locator);
Edit::deletion(stmt.start(), next)
} else if helpers::has_leading_content(stmt.start(), locator) {
} else if has_leading_content(stmt.start(), locator) {
Edit::range_deletion(stmt.range())
} else if let Some(start) =
helpers::preceded_by_continuations(stmt.start(), locator, indexer)
{
} else if let Some(start) = indexer.preceded_by_continuations(stmt.start(), locator) {
Edit::range_deletion(TextRange::new(start, stmt.end()))
} else {
let range = locator.full_lines_range(stmt.range());
@@ -56,14 +55,14 @@ pub(crate) fn delete_stmt(
/// Generate a `Fix` to remove the specified imports from an `import` statement.
pub(crate) fn remove_unused_imports<'a>(
unused_imports: impl Iterator<Item = &'a str>,
member_names: impl Iterator<Item = &'a str>,
stmt: &Stmt,
parent: Option<&Stmt>,
locator: &Locator,
stylist: &Stylist,
indexer: &Indexer,
) -> Result<Edit> {
match codemods::remove_imports(unused_imports, stmt, locator, stylist)? {
match codemods::remove_imports(member_names, stmt, locator, stylist)? {
None => Ok(delete_stmt(stmt, parent, locator, indexer)),
Some(content) => Ok(Edit::range_replacement(content, stmt.range())),
}
@@ -190,12 +189,24 @@ fn is_lone_child(child: &Stmt, parent: &Stmt) -> bool {
}
Stmt::For(ast::StmtFor { body, orelse, .. })
| Stmt::AsyncFor(ast::StmtAsyncFor { body, orelse, .. })
| Stmt::While(ast::StmtWhile { body, orelse, .. })
| Stmt::If(ast::StmtIf { body, orelse, .. }) => {
| Stmt::While(ast::StmtWhile { body, orelse, .. }) => {
if is_only(body, child) || is_only(orelse, child) {
return true;
}
}
Stmt::If(ast::StmtIf {
body,
elif_else_clauses,
..
}) => {
if is_only(body, child)
|| elif_else_clauses
.iter()
.any(|ast::ElifElseClause { body, .. }| is_only(body, child))
{
return true;
}
}
Stmt::Try(ast::StmtTry {
body,
handlers,
@@ -283,11 +294,11 @@ fn next_stmt_break(semicolon: TextSize, locator: &Locator) -> TextSize {
#[cfg(test)]
mod tests {
use anyhow::Result;
use ruff_python_ast::{Ranged, Suite};
use ruff_python_parser::Parse;
use ruff_text_size::TextSize;
use rustpython_parser::ast::{Ranged, Suite};
use rustpython_parser::Parse;
use ruff_python_ast::source_code::Locator;
use ruff_source_file::Locator;
use crate::autofix::edits::{next_stmt_break, trailing_semicolon};

View File

@@ -1,12 +1,11 @@
use itertools::Itertools;
use std::collections::BTreeSet;
use itertools::Itertools;
use nohash_hasher::IntSet;
use ruff_text_size::{TextLen, TextRange, TextSize};
use rustc_hash::FxHashMap;
use rustc_hash::{FxHashMap, FxHashSet};
use ruff_diagnostics::{Diagnostic, Edit, Fix, IsolationLevel};
use ruff_python_ast::source_code::Locator;
use ruff_source_file::Locator;
use crate::autofix::source_map::SourceMap;
use crate::linter::FixTable;
@@ -47,7 +46,7 @@ fn apply_fixes<'a>(
let mut output = String::with_capacity(locator.len());
let mut last_pos: Option<TextSize> = None;
let mut applied: BTreeSet<&Edit> = BTreeSet::default();
let mut isolated: IntSet<u32> = IntSet::default();
let mut isolated: FxHashSet<u32> = FxHashSet::default();
let mut fixed = FxHashMap::default();
let mut source_map = SourceMap::default();
@@ -60,35 +59,30 @@ fn apply_fixes<'a>(
})
.sorted_by(|(rule1, fix1), (rule2, fix2)| cmp_fix(*rule1, *rule2, fix1, fix2))
{
// If we already applied an identical fix as part of another correction, skip
// any re-application.
if fix.edits().iter().all(|edit| applied.contains(edit)) {
*fixed.entry(rule).or_default() += 1;
continue;
}
let mut edits = fix
.edits()
.iter()
.filter(|edit| !applied.contains(edit))
.peekable();
// Best-effort approach: if this fix overlaps with a fix we've already applied,
// skip it.
if last_pos.map_or(false, |last_pos| {
fix.min_start()
.map_or(false, |fix_location| last_pos >= fix_location)
}) {
continue;
}
// If the fix contains at least one new edit, enforce isolation and positional requirements.
if let Some(first) = edits.peek() {
// If this fix requires isolation, and we've already applied another fix in the
// same isolation group, skip it.
if let IsolationLevel::Group(id) = fix.isolation() {
if !isolated.insert(id) {
continue;
}
}
// If this fix requires isolation, and we've already applied another fix in the
// same isolation group, skip it.
if let IsolationLevel::Group(id) = fix.isolation() {
if !isolated.insert(id) {
// If this fix overlaps with a fix we've already applied, skip it.
if last_pos.map_or(false, |last_pos| last_pos >= first.start()) {
continue;
}
}
for edit in fix
.edits()
.iter()
.sorted_unstable_by_key(|edit| edit.start())
{
let mut applied_edits = Vec::with_capacity(fix.edits().len());
for edit in edits {
// Add all contents from `last_pos` to `fix.location`.
let slice = locator.slice(TextRange::new(last_pos.unwrap_or_default(), edit.start()));
output.push_str(slice);
@@ -104,9 +98,10 @@ fn apply_fixes<'a>(
// Track that the edit was applied.
last_pos = Some(edit.end());
applied.insert(edit);
applied_edits.push(edit);
}
applied.extend(applied_edits.drain(..));
*fixed.entry(rule).or_default() += 1;
}
@@ -147,7 +142,7 @@ mod tests {
use ruff_diagnostics::Diagnostic;
use ruff_diagnostics::Edit;
use ruff_diagnostics::Fix;
use ruff_python_ast::source_code::Locator;
use ruff_source_file::Locator;
use crate::autofix::source_map::SourceMarker;
use crate::autofix::{apply_fixes, FixResult};

View File

@@ -0,0 +1,27 @@
use ruff_python_ast::{Arg, Ranged};
use crate::checkers::ast::Checker;
use crate::codes::Rule;
use crate::rules::{flake8_builtins, pep8_naming, pycodestyle};
/// Run lint rules over an [`Arg`] syntax node.
pub(crate) fn argument(arg: &Arg, checker: &mut Checker) {
if checker.enabled(Rule::AmbiguousVariableName) {
if let Some(diagnostic) = pycodestyle::rules::ambiguous_variable_name(&arg.arg, arg.range())
{
checker.diagnostics.push(diagnostic);
}
}
if checker.enabled(Rule::InvalidArgumentName) {
if let Some(diagnostic) = pep8_naming::rules::invalid_argument_name(
&arg.arg,
arg,
&checker.settings.pep8_naming.ignore_names,
) {
checker.diagnostics.push(diagnostic);
}
}
if checker.enabled(Rule::BuiltinArgumentShadowing) {
flake8_builtins::rules::builtin_argument_shadowing(checker, arg);
}
}

View File

@@ -0,0 +1,26 @@
use ruff_python_ast::Arguments;
use crate::checkers::ast::Checker;
use crate::codes::Rule;
use crate::rules::{flake8_bugbear, flake8_pyi, ruff};
/// Run lint rules over a [`Arguments`] syntax node.
pub(crate) fn arguments(arguments: &Arguments, checker: &mut Checker) {
if checker.enabled(Rule::MutableArgumentDefault) {
flake8_bugbear::rules::mutable_argument_default(checker, arguments);
}
if checker.enabled(Rule::FunctionCallInDefaultArgument) {
flake8_bugbear::rules::function_call_in_argument_default(checker, arguments);
}
if checker.settings.rules.enabled(Rule::ImplicitOptional) {
ruff::rules::implicit_optional(checker, arguments);
}
if checker.is_stub {
if checker.enabled(Rule::TypedArgumentDefaultInStub) {
flake8_pyi::rules::typed_argument_simple_defaults(checker, arguments);
}
if checker.enabled(Rule::ArgumentDefaultInStub) {
flake8_pyi::rules::argument_simple_defaults(checker, arguments);
}
}
}

View File

@@ -0,0 +1,69 @@
use ruff_diagnostics::{Diagnostic, Fix};
use crate::checkers::ast::Checker;
use crate::codes::Rule;
use crate::rules::{flake8_import_conventions, flake8_pyi, pyflakes, pylint};
/// Run lint rules over the [`Binding`]s.
pub(crate) fn bindings(checker: &mut Checker) {
if !checker.any_enabled(&[
Rule::InvalidAllFormat,
Rule::InvalidAllObject,
Rule::UnaliasedCollectionsAbcSetImport,
Rule::UnconventionalImportAlias,
Rule::UnusedVariable,
]) {
return;
}
for binding in checker.semantic.bindings.iter() {
if checker.enabled(Rule::UnusedVariable) {
if binding.kind.is_bound_exception() && !binding.is_used() {
let mut diagnostic = Diagnostic::new(
pyflakes::rules::UnusedVariable {
name: binding.name(checker.locator).to_string(),
},
binding.range,
);
if checker.patch(Rule::UnusedVariable) {
diagnostic.try_set_fix(|| {
pyflakes::fixes::remove_exception_handler_assignment(
binding,
checker.locator,
)
.map(Fix::automatic)
});
}
checker.diagnostics.push(diagnostic);
}
}
if checker.enabled(Rule::InvalidAllFormat) {
if let Some(diagnostic) = pylint::rules::invalid_all_format(binding) {
checker.diagnostics.push(diagnostic);
}
}
if checker.enabled(Rule::InvalidAllObject) {
if let Some(diagnostic) = pylint::rules::invalid_all_object(binding) {
checker.diagnostics.push(diagnostic);
}
}
if checker.enabled(Rule::UnconventionalImportAlias) {
if let Some(diagnostic) = flake8_import_conventions::rules::unconventional_import_alias(
checker,
binding,
&checker.settings.flake8_import_conventions.aliases,
) {
checker.diagnostics.push(diagnostic);
}
}
if checker.is_stub {
if checker.enabled(Rule::UnaliasedCollectionsAbcSetImport) {
if let Some(diagnostic) =
flake8_pyi::rules::unaliased_collections_abc_set_import(checker, binding)
{
checker.diagnostics.push(diagnostic);
}
}
}
}
}

View File

@@ -0,0 +1,16 @@
use ruff_python_ast::Comprehension;
use crate::checkers::ast::Checker;
use crate::codes::Rule;
use crate::rules::flake8_simplify;
/// Run lint rules over a [`Comprehension`] syntax nodes.
pub(crate) fn comprehension(comprehension: &Comprehension, checker: &mut Checker) {
if checker.enabled(Rule::InDictKeys) {
flake8_simplify::rules::key_in_dict_for(
checker,
&comprehension.target,
&comprehension.iter,
);
}
}

View File

@@ -0,0 +1,32 @@
use ruff_python_ast::{self as ast, Stmt};
use crate::checkers::ast::Checker;
use crate::codes::Rule;
use crate::rules::{flake8_bugbear, perflint};
/// Run lint rules over all deferred for-loops in the [`SemanticModel`].
pub(crate) fn deferred_for_loops(checker: &mut Checker) {
while !checker.deferred.for_loops.is_empty() {
let for_loops = std::mem::take(&mut checker.deferred.for_loops);
for snapshot in for_loops {
checker.semantic.restore(snapshot);
if let Stmt::For(ast::StmtFor {
target, iter, body, ..
})
| Stmt::AsyncFor(ast::StmtAsyncFor {
target, iter, body, ..
}) = &checker.semantic.stmt()
{
if checker.enabled(Rule::UnusedLoopControlVariable) {
flake8_bugbear::rules::unused_loop_control_variable(checker, target, body);
}
if checker.enabled(Rule::IncorrectDictIterator) {
perflint::rules::incorrect_dict_iterator(checker, target, iter);
}
} else {
unreachable!("Expected Expr::For | Expr::AsyncFor");
}
}
}
}

View File

@@ -0,0 +1,306 @@
use ruff_diagnostics::Diagnostic;
use ruff_python_ast::cast;
use ruff_python_semantic::analyze::{branch_detection, visibility};
use ruff_python_semantic::{Binding, BindingKind, ScopeKind};
use crate::checkers::ast::Checker;
use crate::codes::Rule;
use crate::rules::{flake8_pyi, flake8_type_checking, flake8_unused_arguments, pyflakes, pylint};
/// Run lint rules over all deferred scopes in the [`SemanticModel`].
pub(crate) fn deferred_scopes(checker: &mut Checker) {
if !checker.any_enabled(&[
Rule::GlobalVariableNotAssigned,
Rule::ImportShadowedByLoopVar,
Rule::RedefinedWhileUnused,
Rule::RuntimeImportInTypeCheckingBlock,
Rule::TypingOnlyFirstPartyImport,
Rule::TypingOnlyStandardLibraryImport,
Rule::TypingOnlyThirdPartyImport,
Rule::UndefinedLocal,
Rule::UnusedAnnotation,
Rule::UnusedClassMethodArgument,
Rule::UnusedFunctionArgument,
Rule::UnusedImport,
Rule::UnusedLambdaArgument,
Rule::UnusedMethodArgument,
Rule::UnusedPrivateProtocol,
Rule::UnusedPrivateTypeAlias,
Rule::UnusedPrivateTypeVar,
Rule::UnusedPrivateTypedDict,
Rule::UnusedStaticMethodArgument,
Rule::UnusedVariable,
]) {
return;
}
// Identify any valid runtime imports. If a module is imported at runtime, and
// used at runtime, then by default, we avoid flagging any other
// imports from that model as typing-only.
let enforce_typing_imports = !checker.is_stub
&& checker.any_enabled(&[
Rule::RuntimeImportInTypeCheckingBlock,
Rule::TypingOnlyFirstPartyImport,
Rule::TypingOnlyStandardLibraryImport,
Rule::TypingOnlyThirdPartyImport,
]);
let runtime_imports: Vec<Vec<&Binding>> = if enforce_typing_imports {
checker
.semantic
.scopes
.iter()
.map(|scope| {
scope
.binding_ids()
.map(|binding_id| checker.semantic.binding(binding_id))
.filter(|binding| {
flake8_type_checking::helpers::is_valid_runtime_import(
binding,
&checker.semantic,
)
})
.collect()
})
.collect::<Vec<_>>()
} else {
vec![]
};
let mut diagnostics: Vec<Diagnostic> = vec![];
for scope_id in checker.deferred.scopes.iter().rev().copied() {
let scope = &checker.semantic.scopes[scope_id];
if checker.enabled(Rule::UndefinedLocal) {
pyflakes::rules::undefined_local(checker, scope_id, scope, &mut diagnostics);
}
if checker.enabled(Rule::GlobalVariableNotAssigned) {
for (name, binding_id) in scope.bindings() {
let binding = checker.semantic.binding(binding_id);
if binding.kind.is_global() {
diagnostics.push(Diagnostic::new(
pylint::rules::GlobalVariableNotAssigned {
name: (*name).to_string(),
},
binding.range,
));
}
}
}
if checker.enabled(Rule::ImportShadowedByLoopVar) {
for (name, binding_id) in scope.bindings() {
for shadow in checker.semantic.shadowed_bindings(scope_id, binding_id) {
// If the shadowing binding isn't a loop variable, abort.
let binding = &checker.semantic.bindings[shadow.binding_id()];
if !binding.kind.is_loop_var() {
continue;
}
// If the shadowed binding isn't an import, abort.
let shadowed = &checker.semantic.bindings[shadow.shadowed_id()];
if !matches!(
shadowed.kind,
BindingKind::Import(..)
| BindingKind::FromImport(..)
| BindingKind::SubmoduleImport(..)
| BindingKind::FutureImport
) {
continue;
}
// If the bindings are in different forks, abort.
if shadowed.source.map_or(true, |left| {
binding.source.map_or(true, |right| {
branch_detection::different_forks(left, right, &checker.semantic.stmts)
})
}) {
continue;
}
#[allow(deprecated)]
let line = checker.locator.compute_line_index(shadowed.range.start());
checker.diagnostics.push(Diagnostic::new(
pyflakes::rules::ImportShadowedByLoopVar {
name: name.to_string(),
line,
},
binding.range,
));
}
}
}
if checker.enabled(Rule::RedefinedWhileUnused) {
for (name, binding_id) in scope.bindings() {
for shadow in checker.semantic.shadowed_bindings(scope_id, binding_id) {
// If the shadowing binding is a loop variable, abort, to avoid overlap
// with F402.
let binding = &checker.semantic.bindings[shadow.binding_id()];
if binding.kind.is_loop_var() {
continue;
}
// If the shadowed binding is used, abort.
let shadowed = &checker.semantic.bindings[shadow.shadowed_id()];
if shadowed.is_used() {
continue;
}
// If the shadowing binding isn't considered a "redefinition" of the
// shadowed binding, abort.
if !binding.redefines(shadowed) {
continue;
}
if shadow.same_scope() {
// If the symbol is a dummy variable, abort, unless the shadowed
// binding is an import.
if !matches!(
shadowed.kind,
BindingKind::Import(..)
| BindingKind::FromImport(..)
| BindingKind::SubmoduleImport(..)
| BindingKind::FutureImport
) && checker.settings.dummy_variable_rgx.is_match(name)
{
continue;
}
// If this is an overloaded function, abort.
if shadowed.kind.is_function_definition()
&& visibility::is_overload(
cast::decorator_list(
checker.semantic.stmts[shadowed.source.unwrap()],
),
&checker.semantic,
)
{
continue;
}
} else {
// Only enforce cross-scope shadowing for imports.
if !matches!(
shadowed.kind,
BindingKind::Import(..)
| BindingKind::FromImport(..)
| BindingKind::SubmoduleImport(..)
| BindingKind::FutureImport
) {
continue;
}
}
// If the bindings are in different forks, abort.
if shadowed.source.map_or(true, |left| {
binding.source.map_or(true, |right| {
branch_detection::different_forks(left, right, &checker.semantic.stmts)
})
}) {
continue;
}
#[allow(deprecated)]
let line = checker.locator.compute_line_index(shadowed.range.start());
let mut diagnostic = Diagnostic::new(
pyflakes::rules::RedefinedWhileUnused {
name: (*name).to_string(),
line,
},
binding.range,
);
if let Some(range) = binding.parent_range(&checker.semantic) {
diagnostic.set_parent(range.start());
}
diagnostics.push(diagnostic);
}
}
}
if checker.is_stub {
if checker.enabled(Rule::UnusedPrivateTypeVar) {
flake8_pyi::rules::unused_private_type_var(checker, scope, &mut diagnostics);
}
if checker.enabled(Rule::UnusedPrivateProtocol) {
flake8_pyi::rules::unused_private_protocol(checker, scope, &mut diagnostics);
}
if checker.enabled(Rule::UnusedPrivateTypeAlias) {
flake8_pyi::rules::unused_private_type_alias(checker, scope, &mut diagnostics);
}
if checker.enabled(Rule::UnusedPrivateTypedDict) {
flake8_pyi::rules::unused_private_typed_dict(checker, scope, &mut diagnostics);
}
}
if matches!(
scope.kind,
ScopeKind::Function(_) | ScopeKind::AsyncFunction(_) | ScopeKind::Lambda(_)
) {
if checker.enabled(Rule::UnusedVariable) {
pyflakes::rules::unused_variable(checker, scope, &mut diagnostics);
}
if checker.enabled(Rule::UnusedAnnotation) {
pyflakes::rules::unused_annotation(checker, scope, &mut diagnostics);
}
if !checker.is_stub {
if checker.any_enabled(&[
Rule::UnusedClassMethodArgument,
Rule::UnusedFunctionArgument,
Rule::UnusedLambdaArgument,
Rule::UnusedMethodArgument,
Rule::UnusedStaticMethodArgument,
]) {
flake8_unused_arguments::rules::unused_arguments(
checker,
scope,
&mut diagnostics,
);
}
}
}
if matches!(
scope.kind,
ScopeKind::Function(_) | ScopeKind::AsyncFunction(_) | ScopeKind::Module
) {
if enforce_typing_imports {
let runtime_imports: Vec<&Binding> = checker
.semantic
.scopes
.ancestor_ids(scope_id)
.flat_map(|scope_id| runtime_imports[scope_id.as_usize()].iter())
.copied()
.collect();
if checker.enabled(Rule::RuntimeImportInTypeCheckingBlock) {
flake8_type_checking::rules::runtime_import_in_type_checking_block(
checker,
scope,
&mut diagnostics,
);
}
if checker.any_enabled(&[
Rule::TypingOnlyFirstPartyImport,
Rule::TypingOnlyStandardLibraryImport,
Rule::TypingOnlyThirdPartyImport,
]) {
flake8_type_checking::rules::typing_only_runtime_import(
checker,
scope,
&runtime_imports,
&mut diagnostics,
);
}
}
if checker.enabled(Rule::UnusedImport) {
pyflakes::rules::unused_import(checker, scope, &mut diagnostics);
}
}
}
checker.diagnostics.extend(diagnostics);
}

View File

@@ -0,0 +1,291 @@
use ruff_python_ast::str::raw_contents_range;
use ruff_python_ast::Ranged;
use ruff_text_size::TextRange;
use ruff_python_semantic::{BindingKind, ContextualizedDefinition, Export};
use crate::checkers::ast::Checker;
use crate::codes::Rule;
use crate::docstrings::Docstring;
use crate::fs::relativize_path;
use crate::rules::{flake8_annotations, flake8_pyi, pydocstyle};
use crate::{docstrings, warn_user};
/// Run lint rules over all [`Definition`] nodes in the [`SemanticModel`].
///
/// This phase is expected to run after the AST has been traversed in its entirety; as such,
/// it is expected that all [`Definition`] nodes have been visited by the time, and that this
/// method will not recurse into any other nodes.
pub(crate) fn definitions(checker: &mut Checker) {
let enforce_annotations = checker.any_enabled(&[
Rule::AnyType,
Rule::MissingReturnTypeClassMethod,
Rule::MissingReturnTypePrivateFunction,
Rule::MissingReturnTypeSpecialMethod,
Rule::MissingReturnTypeStaticMethod,
Rule::MissingReturnTypeUndocumentedPublicFunction,
Rule::MissingTypeArgs,
Rule::MissingTypeCls,
Rule::MissingTypeFunctionArgument,
Rule::MissingTypeKwargs,
Rule::MissingTypeSelf,
]);
let enforce_stubs = checker.is_stub
&& checker.any_enabled(&[Rule::DocstringInStub, Rule::IterMethodReturnIterable]);
let enforce_docstrings = checker.any_enabled(&[
Rule::BlankLineAfterLastSection,
Rule::BlankLineAfterSummary,
Rule::BlankLineBeforeClass,
Rule::BlankLinesBetweenHeaderAndContent,
Rule::CapitalizeSectionName,
Rule::DashedUnderlineAfterSection,
Rule::DocstringStartsWithThis,
Rule::EmptyDocstring,
Rule::EmptyDocstringSection,
Rule::EndsInPeriod,
Rule::EndsInPunctuation,
Rule::EscapeSequenceInDocstring,
Rule::FirstLineCapitalized,
Rule::FitsOnOneLine,
Rule::IndentWithSpaces,
Rule::MultiLineSummaryFirstLine,
Rule::MultiLineSummarySecondLine,
Rule::NewLineAfterLastParagraph,
Rule::NewLineAfterSectionName,
Rule::NoBlankLineAfterFunction,
Rule::NoBlankLineAfterSection,
Rule::NoBlankLineBeforeFunction,
Rule::NoBlankLineBeforeSection,
Rule::NoSignature,
Rule::NonImperativeMood,
Rule::OneBlankLineAfterClass,
Rule::OneBlankLineBeforeClass,
Rule::OverIndentation,
Rule::OverloadWithDocstring,
Rule::SectionNameEndsInColon,
Rule::SectionNotOverIndented,
Rule::SectionUnderlineAfterName,
Rule::SectionUnderlineMatchesSectionLength,
Rule::SectionUnderlineNotOverIndented,
Rule::SurroundingWhitespace,
Rule::TripleSingleQuotes,
Rule::UnderIndentation,
Rule::UndocumentedMagicMethod,
Rule::UndocumentedParam,
Rule::UndocumentedPublicClass,
Rule::UndocumentedPublicFunction,
Rule::UndocumentedPublicInit,
Rule::UndocumentedPublicMethod,
Rule::UndocumentedPublicModule,
Rule::UndocumentedPublicNestedClass,
Rule::UndocumentedPublicPackage,
]);
if !enforce_annotations && !enforce_docstrings && !enforce_stubs {
return;
}
// Compute visibility of all definitions.
let exports: Option<Vec<&str>> = {
checker
.semantic
.global_scope()
.get_all("__all__")
.map(|binding_id| &checker.semantic.bindings[binding_id])
.filter_map(|binding| match &binding.kind {
BindingKind::Export(Export { names }) => Some(names.iter().copied()),
_ => None,
})
.fold(None, |acc, names| {
Some(acc.into_iter().flatten().chain(names).collect())
})
};
let definitions = std::mem::take(&mut checker.semantic.definitions);
let mut overloaded_name: Option<String> = None;
for ContextualizedDefinition {
definition,
visibility,
} in definitions.resolve(exports.as_deref()).iter()
{
let docstring = docstrings::extraction::extract_docstring(definition);
// flake8-annotations
if enforce_annotations {
// TODO(charlie): This should be even stricter, in that an overload
// implementation should come immediately after the overloaded
// interfaces, without any AST nodes in between. Right now, we
// only error when traversing definition boundaries (functions,
// classes, etc.).
if !overloaded_name.map_or(false, |overloaded_name| {
flake8_annotations::helpers::is_overload_impl(
definition,
&overloaded_name,
&checker.semantic,
)
}) {
checker
.diagnostics
.extend(flake8_annotations::rules::definition(
checker,
definition,
*visibility,
));
}
overloaded_name =
flake8_annotations::helpers::overloaded_name(definition, &checker.semantic);
}
// flake8-pyi
if enforce_stubs {
if checker.enabled(Rule::DocstringInStub) {
flake8_pyi::rules::docstring_in_stubs(checker, docstring);
}
if checker.enabled(Rule::IterMethodReturnIterable) {
flake8_pyi::rules::iter_method_return_iterable(checker, definition);
}
}
// pydocstyle
if enforce_docstrings {
if pydocstyle::helpers::should_ignore_definition(
definition,
&checker.settings.pydocstyle.ignore_decorators,
&checker.semantic,
) {
continue;
}
// Extract a `Docstring` from a `Definition`.
let Some(expr) = docstring else {
pydocstyle::rules::not_missing(checker, definition, *visibility);
continue;
};
let contents = checker.locator.slice(expr.range());
let indentation = checker.locator.slice(TextRange::new(
checker.locator.line_start(expr.start()),
expr.start(),
));
if pydocstyle::helpers::should_ignore_docstring(contents) {
#[allow(deprecated)]
let location = checker.locator.compute_source_location(expr.start());
warn_user!(
"Docstring at {}:{}:{} contains implicit string concatenation; ignoring...",
relativize_path(checker.path),
location.row,
location.column
);
continue;
}
// SAFETY: Safe for docstrings that pass `should_ignore_docstring`.
let body_range = raw_contents_range(contents).unwrap();
let docstring = Docstring {
definition,
expr,
contents,
body_range,
indentation,
};
if !pydocstyle::rules::not_empty(checker, &docstring) {
continue;
}
if checker.enabled(Rule::FitsOnOneLine) {
pydocstyle::rules::one_liner(checker, &docstring);
}
if checker.any_enabled(&[
Rule::NoBlankLineAfterFunction,
Rule::NoBlankLineBeforeFunction,
]) {
pydocstyle::rules::blank_before_after_function(checker, &docstring);
}
if checker.any_enabled(&[
Rule::BlankLineBeforeClass,
Rule::OneBlankLineAfterClass,
Rule::OneBlankLineBeforeClass,
]) {
pydocstyle::rules::blank_before_after_class(checker, &docstring);
}
if checker.enabled(Rule::BlankLineAfterSummary) {
pydocstyle::rules::blank_after_summary(checker, &docstring);
}
if checker.any_enabled(&[
Rule::IndentWithSpaces,
Rule::OverIndentation,
Rule::UnderIndentation,
]) {
pydocstyle::rules::indent(checker, &docstring);
}
if checker.enabled(Rule::NewLineAfterLastParagraph) {
pydocstyle::rules::newline_after_last_paragraph(checker, &docstring);
}
if checker.enabled(Rule::SurroundingWhitespace) {
pydocstyle::rules::no_surrounding_whitespace(checker, &docstring);
}
if checker.any_enabled(&[
Rule::MultiLineSummaryFirstLine,
Rule::MultiLineSummarySecondLine,
]) {
pydocstyle::rules::multi_line_summary_start(checker, &docstring);
}
if checker.enabled(Rule::TripleSingleQuotes) {
pydocstyle::rules::triple_quotes(checker, &docstring);
}
if checker.enabled(Rule::EscapeSequenceInDocstring) {
pydocstyle::rules::backslashes(checker, &docstring);
}
if checker.enabled(Rule::EndsInPeriod) {
pydocstyle::rules::ends_with_period(checker, &docstring);
}
if checker.enabled(Rule::NonImperativeMood) {
pydocstyle::rules::non_imperative_mood(
checker,
&docstring,
&checker.settings.pydocstyle.property_decorators,
);
}
if checker.enabled(Rule::NoSignature) {
pydocstyle::rules::no_signature(checker, &docstring);
}
if checker.enabled(Rule::FirstLineCapitalized) {
pydocstyle::rules::capitalized(checker, &docstring);
}
if checker.enabled(Rule::DocstringStartsWithThis) {
pydocstyle::rules::starts_with_this(checker, &docstring);
}
if checker.enabled(Rule::EndsInPunctuation) {
pydocstyle::rules::ends_with_punctuation(checker, &docstring);
}
if checker.enabled(Rule::OverloadWithDocstring) {
pydocstyle::rules::if_needed(checker, &docstring);
}
if checker.any_enabled(&[
Rule::BlankLineAfterLastSection,
Rule::BlankLinesBetweenHeaderAndContent,
Rule::CapitalizeSectionName,
Rule::DashedUnderlineAfterSection,
Rule::EmptyDocstringSection,
Rule::MultiLineSummaryFirstLine,
Rule::NewLineAfterSectionName,
Rule::NoBlankLineAfterSection,
Rule::NoBlankLineBeforeSection,
Rule::SectionNameEndsInColon,
Rule::SectionNotOverIndented,
Rule::SectionUnderlineAfterName,
Rule::SectionUnderlineMatchesSectionLength,
Rule::SectionUnderlineNotOverIndented,
Rule::UndocumentedParam,
]) {
pydocstyle::rules::sections(
checker,
&docstring,
checker.settings.pydocstyle.convention.as_ref(),
);
}
}
}
}

View File

@@ -0,0 +1,88 @@
use ruff_python_ast::{self as ast, ExceptHandler, Ranged};
use crate::checkers::ast::Checker;
use crate::registry::Rule;
use crate::rules::{
flake8_bandit, flake8_blind_except, flake8_bugbear, flake8_builtins, pycodestyle, pylint,
tryceratops,
};
/// Run lint rules over an [`ExceptHandler`] syntax node.
pub(crate) fn except_handler(except_handler: &ExceptHandler, checker: &mut Checker) {
match except_handler {
ExceptHandler::ExceptHandler(ast::ExceptHandlerExceptHandler {
type_,
name,
body,
range: _,
}) => {
if checker.enabled(Rule::BareExcept) {
if let Some(diagnostic) = pycodestyle::rules::bare_except(
type_.as_deref(),
body,
except_handler,
checker.locator,
) {
checker.diagnostics.push(diagnostic);
}
}
if checker.enabled(Rule::RaiseWithoutFromInsideExcept) {
flake8_bugbear::rules::raise_without_from_inside_except(
checker,
name.as_deref(),
body,
);
}
if checker.enabled(Rule::BlindExcept) {
flake8_blind_except::rules::blind_except(
checker,
type_.as_deref(),
name.as_deref(),
body,
);
}
if checker.enabled(Rule::TryExceptPass) {
flake8_bandit::rules::try_except_pass(
checker,
except_handler,
type_.as_deref(),
body,
checker.settings.flake8_bandit.check_typed_exception,
);
}
if checker.enabled(Rule::TryExceptContinue) {
flake8_bandit::rules::try_except_continue(
checker,
except_handler,
type_.as_deref(),
body,
checker.settings.flake8_bandit.check_typed_exception,
);
}
if checker.enabled(Rule::ExceptWithEmptyTuple) {
flake8_bugbear::rules::except_with_empty_tuple(checker, except_handler);
}
if checker.enabled(Rule::ExceptWithNonExceptionClasses) {
flake8_bugbear::rules::except_with_non_exception_classes(checker, except_handler);
}
if checker.enabled(Rule::ReraiseNoCause) {
tryceratops::rules::reraise_no_cause(checker, body);
}
if checker.enabled(Rule::BinaryOpException) {
pylint::rules::binary_op_exception(checker, except_handler);
}
if let Some(name) = name {
if checker.enabled(Rule::AmbiguousVariableName) {
if let Some(diagnostic) =
pycodestyle::rules::ambiguous_variable_name(name.as_str(), name.range())
{
checker.diagnostics.push(diagnostic);
}
}
if checker.enabled(Rule::BuiltinVariableShadowing) {
flake8_builtins::rules::builtin_variable_shadowing(checker, name, name.range());
}
}
}
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,27 @@
pub(super) use argument::argument;
pub(super) use arguments::arguments;
pub(super) use bindings::bindings;
pub(super) use comprehension::comprehension;
pub(super) use deferred_for_loops::deferred_for_loops;
pub(super) use deferred_scopes::deferred_scopes;
pub(super) use definitions::definitions;
pub(super) use except_handler::except_handler;
pub(super) use expression::expression;
pub(super) use module::module;
pub(super) use statement::statement;
pub(super) use suite::suite;
pub(super) use unresolved_references::unresolved_references;
mod argument;
mod arguments;
mod bindings;
mod comprehension;
mod deferred_for_loops;
mod deferred_scopes;
mod definitions;
mod except_handler;
mod expression;
mod module;
mod statement;
mod suite;
mod unresolved_references;

View File

@@ -0,0 +1,12 @@
use ruff_python_ast::Suite;
use crate::checkers::ast::Checker;
use crate::codes::Rule;
use crate::rules::flake8_bugbear;
/// Run lint rules over a module.
pub(crate) fn module(suite: &Suite, checker: &mut Checker) {
if checker.enabled(Rule::FStringDocstring) {
flake8_bugbear::rules::f_string_docstring(checker, suite);
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,12 @@
use ruff_python_ast::Stmt;
use crate::checkers::ast::Checker;
use crate::codes::Rule;
use crate::rules::flake8_pie;
/// Run lint rules over a suite of [`Stmt`] syntax nodes.
pub(crate) fn suite(suite: &[Stmt], checker: &mut Checker) {
if checker.enabled(Rule::UnnecessaryPass) {
flake8_pie::rules::no_unnecessary_pass(checker, suite);
}
}

Some files were not shown because too many files have changed in this diff Show More