Compare commits

...

9 Commits

Author SHA1 Message Date
Charlie Marsh
4fd6f969e3 Use --lib in uv init in tutorial 2025-01-24 09:48:30 -05:00
David Peter
1feb3cf41a [red-knot] Use Unknown | T_inferred for undeclared public symbols (#15674)
## Summary

Use `Unknown | T_inferred` as the type for *undeclared* public symbols.

## Test Plan

- Updated existing tests
- New test for external `__slots__` modifications.
- New tests for external modifications of public symbols.
2025-01-24 12:47:48 +01:00
Dylan
7778d1d646 [ruff] Parenthesize fix when argument spans multiple lines for unnecessary-round (RUF057) (#15703) 2025-01-24 04:34:56 -06:00
David Peter
fb58a9b610 [red-knot] Rename TestDbBuilder::typeshed to .custom_typeshed (#15712)
## Summary

Correcting a small oversight by me
(https://github.com/astral-sh/ruff/pull/15683#discussion_r1926830914).
2025-01-24 10:25:23 +00:00
Mike Perlov
17a8a55f08 Honor banned top level imports by TID253 in PLC0415. (#15628)
Co-authored-by: Micha Reiser <micha@reiser.io>
2025-01-24 11:07:21 +01:00
Dhruv Manilawala
99d8ec6769 Apply AIR302-context check only in @task function (#15711)
This PR updates `AIR302` to only apply the context keys check in `@task`
decorated function.

Ref: https://github.com/astral-sh/ruff/pull/15144
2025-01-24 07:30:35 +00:00
Ankit Chaurasia
34cc3cab98 [airflow] Update AIR302 to check for deprecated context keys (#15144)
**Summary**

Airflow 3.0 removes a set of deprecated context variables that were
phased out in 2.x. This PR introduces lint rules to detect usage of
these removed variables in various patterns, helping identify
incompatibilities. The removed context variables include:

```
conf
execution_date
next_ds
next_ds_nodash
next_execution_date
prev_ds
prev_ds_nodash
prev_execution_date
prev_execution_date_success
tomorrow_ds
yesterday_ds
yesterday_ds_nodash
```

**Detected Patterns and Examples**

The linter now flags the use of removed context variables in the
following scenarios:

1. **Direct Subscript Access**  
   ```python
   execution_date = context["execution_date"]  # Flagged
   ```
   
2. **`.get("key")` Method Calls**  
   ```python
   print(context.get("execution_date"))  # Flagged
   ```
   
3. **Variables Assigned from `get_current_context()`**  
If a variable is assigned from `get_current_context()` and then used to
access a removed key:
   ```python
   c = get_current_context()
   print(c.get("execution_date"))  # Flagged
   ```
   
4. **Function Parameters in `@task`-Decorated Functions**  
Parameters named after removed context variables in functions decorated
with `@task` are flagged:
   ```python
   from airflow.decorators import task
   
   @task
def my_task(execution_date, **kwargs): # Parameter 'execution_date'
flagged
       pass
   ```
   
5. **Removed Keys in Task Decorator `kwargs` and Other Scenarios**  
Other similar patterns where removed context variables appear (e.g., as
part of `kwargs` in a `@task` function) are also detected.
```
from airflow.decorators import task

@task
def process_with_execution_date(**context):
    execution_date = lambda: context["execution_date"]  # flagged
    print(execution_date)

@task(kwargs={"execution_date": "2021-01-01"})   # flagged
def task_with_kwargs(**context):  
    pass
```

**Test Plan**

Test fixtures covering various patterns of deprecated context usage are
included in this PR. For example:

```python
from airflow.decorators import task, dag, get_current_context
from airflow.models import DAG
from airflow.operators.dummy import DummyOperator
import pendulum
from datetime import datetime

@task
def access_invalid_key_task(**context):
    print(context.get("conf"))  # 'conf' flagged

@task
def print_config(**context):
    execution_date = context["execution_date"]  # Flagged
    prev_ds = context["prev_ds"]                # Flagged

@task
def from_current_context():
    context = get_current_context()
    print(context["execution_date"])            # Flagged

# Usage outside of a task decorated function
c = get_current_context()
print(c.get("execution_date"))                 # Flagged

@task
def some_task(execution_date, **kwargs):
    print("execution date", execution_date)     # Parameter flagged

@dag(
    start_date=pendulum.datetime(2021, 1, 1, tz="UTC")
)
def my_dag():
    task1 = DummyOperator(
        task_id="task1",
        params={
            "execution_date": "{{ execution_date }}",  # Flagged in template context
        },
    )

    access_invalid_key_task()
    print_config()
    from_current_context()
    
dag = my_dag()

class CustomOperator(BaseOperator):
    def execute(self, context):
        execution_date = context.get("execution_date")                      # Flagged
        next_ds = context.get("next_ds")                                               # Flagged
        next_execution_date = context["next_execution_date"]          # Flagged
```

Ruff will emit `AIR302` diagnostics for each deprecated usage, with
suggestions when applicable, aiding in code migration to Airflow 3.0.

related: https://github.com/apache/airflow/issues/44409,
https://github.com/apache/airflow/issues/41641

---------

Co-authored-by: Wei Lee <weilee.rx@gmail.com>
2025-01-24 11:25:05 +05:30
Dhruv Manilawala
9384ba4b91 Remove test rules from JSON schema (#15627)
Closes: #15707
2025-01-24 10:17:59 +05:30
Dylan
2b3550c85f Add two missing commits to changelog (#15701)
Some commits appeared between the creation and merging of the "bump to
v0.9.3" branch.

Also made the same changes to the Releases on githhub.
2025-01-23 15:32:00 -06:00
44 changed files with 1369 additions and 192 deletions

View File

@@ -21,6 +21,7 @@
- \[`flake8-bugbear`\] Do not raise error if keyword argument is present and target-python version is less or equals than 3.9 (`B903`) ([#15549](https://github.com/astral-sh/ruff/pull/15549))
- \[`flake8-comprehensions`\] strip parentheses around generators in `unnecessary-generator-set` (`C401`) ([#15553](https://github.com/astral-sh/ruff/pull/15553))
- \[`flake8-pytest-style`\] Rewrite references to `.exception` (`PT027`) ([#15680](https://github.com/astral-sh/ruff/pull/15680))
- \[`flake8-simplify`\] Mark fixes as unsafe (`SIM201`, `SIM202`) ([#15626](https://github.com/astral-sh/ruff/pull/15626))
- \[`flake8-type-checking`\] Fix some safe fixes being labeled unsafe (`TC006`,`TC008`) ([#15638](https://github.com/astral-sh/ruff/pull/15638))
- \[`isort`\] Omit trailing whitespace in `unsorted-imports` (`I001`) ([#15518](https://github.com/astral-sh/ruff/pull/15518))
@@ -47,11 +48,12 @@
### Bug fixes
- \[`flake8-bandit`\] Add missing single-line/dotall regex flag (`S608`) ([#15654](https://github.com/astral-sh/ruff/pull/15654))
- \[`flake8-import-conventions`\] Fix infinite loop between `ICN001` and `I002` (`ICN001`) ([#15480](https://github.com/astral-sh/ruff/pull/15480))
- \[`flake8-simplify`\] Do not emit diagnostics for expressions inside string type annotations (`SIM222`, `SIM223`) ([#15405](https://github.com/astral-sh/ruff/pull/15405))
- \[`pyflakes`\] Treat arguments passed to the `default=` parameter of `TypeVar` as type expressions (`F821`) ([#15679](https://github.com/astral-sh/ruff/pull/15679))
- \[`pyupgrade`\] Avoid syntax error when the iterable is a non-parenthesized tuple (`UP028`) ([#15543](https://github.com/astral-sh/ruff/pull/15543))
- \[`ruff`\] Exempt `NewType` calls where the original type is immutable (`RUF009`) ([#15588](https://github.com/astral-sh/ruff/pull/15588))
- \[`unconventional-import-alias`\] Fix infinite loop between `ICN001` and `I002` (`ICN001`) ([#15480](https://github.com/astral-sh/ruff/pull/15480))
- Preserve raw string prefix and escapes in all codegen fixes ([#15694](https://github.com/astral-sh/ruff/pull/15694))
### Documentation

View File

@@ -36,7 +36,7 @@ def f():
reveal_type(a7) # revealed: None
reveal_type(a8) # revealed: Literal[1]
# TODO: This should be Color.RED
reveal_type(b1) # revealed: Literal[0]
reveal_type(b1) # revealed: Unknown | Literal[0]
# error: [invalid-type-form]
invalid1: Literal[3 + 4]

View File

@@ -175,7 +175,7 @@ class C:
reveal_type(C.pure_class_variable1) # revealed: str
# TODO: this should be `Literal[1]`, or `Unknown | Literal[1]`.
# TODO: Should be `Unknown | Literal[1]`.
reveal_type(C.pure_class_variable2) # revealed: Unknown
c_instance = C()
@@ -252,8 +252,7 @@ class C:
reveal_type(C.variable_with_class_default1) # revealed: str
# TODO: this should be `Unknown | Literal[1]`.
reveal_type(C.variable_with_class_default2) # revealed: Literal[1]
reveal_type(C.variable_with_class_default2) # revealed: Unknown | Literal[1]
c_instance = C()
@@ -296,8 +295,8 @@ def _(flag: bool):
else:
x = 4
reveal_type(C1.x) # revealed: Literal[1, 2]
reveal_type(C2.x) # revealed: Literal[3, 4]
reveal_type(C1.x) # revealed: Unknown | Literal[1, 2]
reveal_type(C2.x) # revealed: Unknown | Literal[3, 4]
```
## Inherited class attributes
@@ -311,7 +310,7 @@ class A:
class B(A): ...
class C(B): ...
reveal_type(C.X) # revealed: Literal["foo"]
reveal_type(C.X) # revealed: Unknown | Literal["foo"]
```
### Multiple inheritance
@@ -334,7 +333,7 @@ class A(B, C): ...
reveal_type(A.__mro__)
# `E` is earlier in the MRO than `F`, so we should use the type of `E.X`
reveal_type(A.X) # revealed: Literal[42]
reveal_type(A.X) # revealed: Unknown | Literal[42]
```
## Unions with possibly unbound paths
@@ -356,7 +355,7 @@ def _(flag1: bool, flag2: bool):
C = C1 if flag1 else C2 if flag2 else C3
# error: [possibly-unbound-attribute] "Attribute `x` on type `Literal[C1, C2, C3]` is possibly unbound"
reveal_type(C.x) # revealed: Literal[1, 3]
reveal_type(C.x) # revealed: Unknown | Literal[1, 3]
```
### Possibly-unbound within a class
@@ -379,7 +378,7 @@ def _(flag: bool, flag1: bool, flag2: bool):
C = C1 if flag1 else C2 if flag2 else C3
# error: [possibly-unbound-attribute] "Attribute `x` on type `Literal[C1, C2, C3]` is possibly unbound"
reveal_type(C.x) # revealed: Literal[1, 2, 3]
reveal_type(C.x) # revealed: Unknown | Literal[1, 2, 3]
```
### Unions with all paths unbound

View File

@@ -262,7 +262,8 @@ class A:
class B:
__add__ = A()
reveal_type(B() + B()) # revealed: int
# TODO: this could be `int` if we declare `B.__add__` using a `Callable` type
reveal_type(B() + B()) # revealed: Unknown | int
```
## Integration test: numbers from typeshed

View File

@@ -5,6 +5,11 @@ that is, a use of a symbol from another scope. If a symbol has a declared type i
(e.g. `int`), we use that as the symbol's "public type" (the type of the symbol from the perspective
of other scopes) even if there is a more precise local inferred type for the symbol (`Literal[1]`).
If a symbol has no declared type, we use the union of `Unknown` with the inferred type as the public
type. If there is no declaration, then the symbol can be reassigned to any type from another scope;
the union with `Unknown` reflects that its type must at least be as large as the type of the
assigned value, but could be arbitrarily larger.
We test the whole matrix of possible boundness and declaredness states. The current behavior is
summarized in the following table, while the tests below demonstrate each case. Note that some of
this behavior is questionable and might change in the future. See the TODOs in `symbol_by_id`
@@ -12,11 +17,11 @@ this behavior is questionable and might change in the future. See the TODOs in `
In particular, we should raise errors in the "possibly-undeclared-and-unbound" as well as the
"undeclared-and-possibly-unbound" cases (marked with a "?").
| **Public type** | declared | possibly-undeclared | undeclared |
| ---------------- | ------------ | -------------------------- | ------------ |
| bound | `T_declared` | `T_declared \| T_inferred` | `T_inferred` |
| possibly-unbound | `T_declared` | `T_declared \| T_inferred` | `T_inferred` |
| unbound | `T_declared` | `T_declared` | `Unknown` |
| **Public type** | declared | possibly-undeclared | undeclared |
| ---------------- | ------------ | -------------------------- | ----------------------- |
| bound | `T_declared` | `T_declared \| T_inferred` | `Unknown \| T_inferred` |
| possibly-unbound | `T_declared` | `T_declared \| T_inferred` | `Unknown \| T_inferred` |
| unbound | `T_declared` | `T_declared` | `Unknown` |
| **Diagnostic** | declared | possibly-undeclared | undeclared |
| ---------------- | -------- | ------------------------- | ------------------- |
@@ -97,17 +102,24 @@ def flag() -> bool: ...
x = 1
y = 2
z = 3
if flag():
x: Any
x: int
y: Any
# error: [invalid-declaration]
y: str
z: str
```
```py
from mod import x, y
from mod import x, y, z
reveal_type(x) # revealed: Literal[1] | Any
reveal_type(y) # revealed: Literal[2] | Unknown
reveal_type(x) # revealed: int
reveal_type(y) # revealed: Literal[2] | Any
reveal_type(z) # revealed: Literal[3] | Unknown
# External modifications of `x` that violate the declared type are not allowed:
# error: [invalid-assignment]
x = None
```
### Possibly undeclared and possibly unbound
@@ -134,6 +146,10 @@ from mod import x, y
reveal_type(x) # revealed: Literal[1] | Any
reveal_type(y) # revealed: Literal[2] | str
# External modifications of `y` that violate the declared type are not allowed:
# error: [invalid-assignment]
y = None
```
### Possibly undeclared and unbound
@@ -154,14 +170,16 @@ if flag():
from mod import x
reveal_type(x) # revealed: int
# External modifications to `x` that violate the declared type are not allowed:
# error: [invalid-assignment]
x = None
```
## Undeclared
### Undeclared but bound
We use the inferred type as the public type, if a symbol has no declared type.
```py path=mod.py
x = 1
```
@@ -169,7 +187,10 @@ x = 1
```py
from mod import x
reveal_type(x) # revealed: Literal[1]
reveal_type(x) # revealed: Unknown | Literal[1]
# All external modifications of `x` are allowed:
x = None
```
### Undeclared and possibly unbound
@@ -189,7 +210,10 @@ if flag:
# on top of this document.
from mod import x
reveal_type(x) # revealed: Literal[1]
reveal_type(x) # revealed: Unknown | Literal[1]
# All external modifications of `x` are allowed:
x = None
```
### Undeclared and unbound
@@ -206,4 +230,7 @@ if False:
from mod import x
reveal_type(x) # revealed: Unknown
# Modifications allowed in this case:
x = None
```

View File

@@ -52,7 +52,7 @@ class NonCallable:
__call__ = 1
a = NonCallable()
# error: "Object of type `NonCallable` is not callable"
# error: "Object of type `Unknown | Literal[1]` is not callable (due to union element `Literal[1]`)"
reveal_type(a()) # revealed: Unknown
```

View File

@@ -43,7 +43,8 @@ class IntIterable:
def __iter__(self) -> IntIterator:
return IntIterator()
# revealed: tuple[int, int]
# TODO: This could be a `tuple[int, int]` if we model that `y` can not be modified in the outer comprehension scope
# revealed: tuple[int, Unknown | int]
[[reveal_type((x, y)) for x in IntIterable()] for y in IntIterable()]
```
@@ -66,7 +67,8 @@ class IterableOfIterables:
def __iter__(self) -> IteratorOfIterables:
return IteratorOfIterables()
# revealed: tuple[int, IntIterable]
# TODO: This could be a `tuple[int, int]` (see above)
# revealed: tuple[int, Unknown | IntIterable]
[[reveal_type((x, y)) for x in y] for y in IterableOfIterables()]
```

View File

@@ -5,7 +5,7 @@
```py
def _(flag: bool):
class A:
always_bound = 1
always_bound: int = 1
if flag:
union = 1
@@ -13,14 +13,21 @@ def _(flag: bool):
union = "abc"
if flag:
possibly_unbound = "abc"
union_declared: int = 1
else:
union_declared: str = "abc"
reveal_type(A.always_bound) # revealed: Literal[1]
if flag:
possibly_unbound: str = "abc"
reveal_type(A.union) # revealed: Literal[1, "abc"]
reveal_type(A.always_bound) # revealed: int
reveal_type(A.union) # revealed: Unknown | Literal[1, "abc"]
reveal_type(A.union_declared) # revealed: int | str
# error: [possibly-unbound-attribute] "Attribute `possibly_unbound` on type `Literal[A]` is possibly unbound"
reveal_type(A.possibly_unbound) # revealed: Literal["abc"]
reveal_type(A.possibly_unbound) # revealed: str
# error: [unresolved-attribute] "Type `Literal[A]` has no attribute `non_existent`"
reveal_type(A.non_existent) # revealed: Unknown

View File

@@ -55,7 +55,7 @@ reveal_type("x" or "y" and "") # revealed: Literal["x"]
## Evaluates to builtin
```py path=a.py
redefined_builtin_bool = bool
redefined_builtin_bool: type[bool] = bool
def my_bool(x) -> bool:
return True

View File

@@ -172,10 +172,10 @@ class IntUnion:
def __len__(self) -> Literal[SomeEnum.INT, SomeEnum.INT_2]: ...
reveal_type(len(Auto())) # revealed: int
reveal_type(len(Int())) # revealed: Literal[2]
reveal_type(len(Int())) # revealed: int
reveal_type(len(Str())) # revealed: int
reveal_type(len(Tuple())) # revealed: int
reveal_type(len(IntUnion())) # revealed: Literal[2, 32]
reveal_type(len(IntUnion())) # revealed: int
```
### Negative integers

View File

@@ -20,7 +20,7 @@ wrong_innards: MyBox[int] = MyBox("five")
# TODO reveal int, do not leak the typevar
reveal_type(box.data) # revealed: T
reveal_type(MyBox.box_model_number) # revealed: Literal[695]
reveal_type(MyBox.box_model_number) # revealed: Unknown | Literal[695]
```
## Subclassing

View File

@@ -23,8 +23,8 @@ reveal_type(y)
# error: [possibly-unbound-import] "Member `y` of module `maybe_unbound` is possibly unbound"
from maybe_unbound import x, y
reveal_type(x) # revealed: Literal[3]
reveal_type(y) # revealed: Literal[3]
reveal_type(x) # revealed: Unknown | Literal[3]
reveal_type(y) # revealed: Unknown | Literal[3]
```
## Maybe unbound annotated
@@ -52,7 +52,7 @@ Importing an annotated name prefers the declared type over the inferred type:
# error: [possibly-unbound-import] "Member `y` of module `maybe_unbound_annotated` is possibly unbound"
from maybe_unbound_annotated import x, y
reveal_type(x) # revealed: Literal[3]
reveal_type(x) # revealed: Unknown | Literal[3]
reveal_type(y) # revealed: int
```

View File

@@ -109,9 +109,9 @@ reveal_type(x)
def _(flag: bool):
class NotIterable:
if flag:
__iter__ = 1
__iter__: int = 1
else:
__iter__ = None
__iter__: None = None
for x in NotIterable(): # error: "Object of type `NotIterable` is not iterable"
pass
@@ -135,7 +135,7 @@ for x in nonsense: # error: "Object of type `Literal[123]` is not iterable"
class NotIterable:
def __getitem__(self, key: int) -> int:
return 42
__iter__ = None
__iter__: None = None
for x in NotIterable(): # error: "Object of type `NotIterable` is not iterable"
pass

View File

@@ -99,9 +99,9 @@ def _(x: str | int):
class A: ...
class B: ...
alias_for_type = type
def _(x: A | B):
alias_for_type = type
if alias_for_type(x) is A:
reveal_type(x) # revealed: A
```

View File

@@ -6,7 +6,7 @@
def f():
x = 1
def g():
reveal_type(x) # revealed: Literal[1]
reveal_type(x) # revealed: Unknown | Literal[1]
```
## Two levels up
@@ -16,7 +16,7 @@ def f():
x = 1
def g():
def h():
reveal_type(x) # revealed: Literal[1]
reveal_type(x) # revealed: Unknown | Literal[1]
```
## Skips class scope
@@ -28,7 +28,7 @@ def f():
class C:
x = 2
def g():
reveal_type(x) # revealed: Literal[1]
reveal_type(x) # revealed: Unknown | Literal[1]
```
## Skips annotation-only assignment
@@ -41,7 +41,7 @@ def f():
# name is otherwise not defined; maybe should be an error?
x: int
def h():
reveal_type(x) # revealed: Literal[1]
reveal_type(x) # revealed: Unknown | Literal[1]
```
## Implicit global in function
@@ -52,5 +52,5 @@ A name reference to a never-defined symbol in a function is implicitly a global
x = 1
def f():
reveal_type(x) # revealed: Literal[1]
reveal_type(x) # revealed: Unknown | Literal[1]
```

View File

@@ -17,8 +17,8 @@ class C:
x = 2
# error: [possibly-unbound-attribute] "Attribute `x` on type `Literal[C]` is possibly unbound"
reveal_type(C.x) # revealed: Literal[2]
reveal_type(C.y) # revealed: Literal[1]
reveal_type(C.x) # revealed: Unknown | Literal[2]
reveal_type(C.y) # revealed: Unknown | Literal[1]
```
## Possibly unbound in class and global scope
@@ -37,7 +37,7 @@ class C:
# error: [possibly-unresolved-reference]
y = x
reveal_type(C.y) # revealed: Literal[1, "abc"]
reveal_type(C.y) # revealed: Unknown | Literal[1, "abc"]
```
## Unbound function local

View File

@@ -182,3 +182,34 @@ class C(A, B): ...
# False negative: [incompatible-slots]
class A(int, str): ...
```
### Diagnostic if `__slots__` is externally modified
We special-case type inference for `__slots__` and return the pure inferred type, even if the symbol
is not declared — a case in which we union with `Unknown` for other public symbols. The reason for
this is that `__slots__` has a special handling in the runtime. Modifying it externally is actually
allowed, but those changes do not take effect. If you have a class `C` with `__slots__ = ("foo",)`
and externally set `C.__slots__ = ("bar",)`, you still can't access `C.bar`. And you can still
access `C.foo`. We therefore issue a diagnostic for such assignments:
```py
class A:
__slots__ = ("a",)
# Modifying `__slots__` from within the class body is fine:
__slots__ = ("a", "b")
# No `Unknown` here:
reveal_type(A.__slots__) # revealed: tuple[Literal["a"], Literal["b"]]
# But modifying it externally is not:
# error: [invalid-assignment]
A.__slots__ = ("a",)
# error: [invalid-assignment]
A.__slots__ = ("a", "b_new")
# error: [invalid-assignment]
A.__slots__ = ("a", "b", "c")
```

View File

@@ -14,7 +14,8 @@ a = NotSubscriptable()[0] # error: "Cannot subscript object of type `NotSubscri
class NotSubscriptable:
__getitem__ = None
a = NotSubscriptable()[0] # error: "Method `__getitem__` of type `None` is not callable on object of type `NotSubscriptable`"
# error: "Method `__getitem__` of type `Unknown | None` is not callable on object of type `NotSubscriptable`"
a = NotSubscriptable()[0]
```
## Valid getitem

View File

@@ -139,7 +139,9 @@ reveal_type(not AlwaysFalse())
# We don't get into a cycle if someone sets their `__bool__` method to the `bool` builtin:
class BoolIsBool:
__bool__ = bool
# TODO: The `type[bool]` declaration here is a workaround to avoid running into
# https://github.com/astral-sh/ruff/issues/15672
__bool__: type[bool] = bool
# revealed: bool
reveal_type(not BoolIsBool())

View File

@@ -76,11 +76,11 @@ with Manager():
```py
class Manager:
__enter__ = 42
__enter__: int = 42
def __exit__(self, exc_tpe, exc_value, traceback): ...
# error: [invalid-context-manager] "Object of type `Manager` cannot be used with `with` because the method `__enter__` of type `Literal[42]` is not callable"
# error: [invalid-context-manager] "Object of type `Manager` cannot be used with `with` because the method `__enter__` of type `int` is not callable"
with Manager():
...
```
@@ -91,9 +91,9 @@ with Manager():
class Manager:
def __enter__(self) -> Self: ...
__exit__ = 32
__exit__: int = 32
# error: [invalid-context-manager] "Object of type `Manager` cannot be used with `with` because the method `__exit__` of type `Literal[32]` is not callable"
# error: [invalid-context-manager] "Object of type `Manager` cannot be used with `with` because the method `__exit__` of type `int` is not callable"
with Manager():
...
```

View File

@@ -136,7 +136,7 @@ pub(crate) mod tests {
/// Target Python platform
python_platform: PythonPlatform,
/// Path to a custom typeshed directory
typeshed: Option<SystemPathBuf>,
custom_typeshed: Option<SystemPathBuf>,
/// Path and content pairs for files that should be present
files: Vec<(&'a str, &'a str)>,
}
@@ -146,7 +146,7 @@ pub(crate) mod tests {
Self {
python_version: PythonVersion::default(),
python_platform: PythonPlatform::default(),
typeshed: None,
custom_typeshed: None,
files: vec![],
}
}
@@ -171,7 +171,7 @@ pub(crate) mod tests {
.context("Failed to write test files")?;
let mut search_paths = SearchPathSettings::new(vec![src_root]);
search_paths.custom_typeshed = self.typeshed;
search_paths.custom_typeshed = self.custom_typeshed;
Program::from_settings(
&db,

View File

@@ -85,6 +85,14 @@ impl<'db> Symbol<'db> {
Symbol::Unbound => self,
}
}
#[must_use]
pub(crate) fn map_type(self, f: impl FnOnce(Type<'db>) -> Type<'db>) -> Symbol<'db> {
match self {
Symbol::Type(ty, boundness) => Symbol::Type(f(ty), boundness),
Symbol::Unbound => Symbol::Unbound,
}
}
}
#[cfg(test)]

View File

@@ -80,29 +80,54 @@ pub fn check_types(db: &dyn Db, file: File) -> TypeCheckDiagnostics {
diagnostics
}
/// Computes a possibly-widened type `Unknown | T_inferred` from the inferred type `T_inferred`
/// of a symbol, unless the type is a known-instance type (e.g. `typing.Any`) or the symbol is
/// considered non-modifiable (e.g. when the symbol is `@Final`). We need this for public uses
/// of symbols that have no declared type.
fn widen_type_for_undeclared_public_symbol<'db>(
db: &'db dyn Db,
inferred: Symbol<'db>,
is_considered_non_modifiable: bool,
) -> Symbol<'db> {
// We special-case known-instance types here since symbols like `typing.Any` are typically
// not declared in the stubs (e.g. `Any = object()`), but we still want to treat them as
// such.
let is_known_instance = inferred
.ignore_possibly_unbound()
.is_some_and(|ty| matches!(ty, Type::KnownInstance(_)));
if is_considered_non_modifiable || is_known_instance {
inferred
} else {
inferred.map_type(|ty| UnionType::from_elements(db, [Type::unknown(), ty]))
}
}
/// Infer the public type of a symbol (its type as seen from outside its scope).
fn symbol<'db>(db: &'db dyn Db, scope: ScopeId<'db>, name: &str) -> Symbol<'db> {
#[salsa::tracked]
fn symbol_by_id<'db>(
db: &'db dyn Db,
scope: ScopeId<'db>,
symbol: ScopedSymbolId,
is_dunder_slots: bool,
symbol_id: ScopedSymbolId,
) -> Symbol<'db> {
let use_def = use_def_map(db, scope);
// If the symbol is declared, the public type is based on declarations; otherwise, it's based
// on inference from bindings.
let declarations = use_def.public_declarations(symbol);
let declared =
symbol_from_declarations(db, declarations).map(|SymbolAndQualifiers(ty, _)| ty);
let declarations = use_def.public_declarations(symbol_id);
let declared = symbol_from_declarations(db, declarations);
let is_final = declared.as_ref().is_ok_and(SymbolAndQualifiers::is_final);
let declared = declared.map(|SymbolAndQualifiers(symbol, _)| symbol);
match declared {
// Symbol is declared, trust the declared type
Ok(symbol @ Symbol::Type(_, Boundness::Bound)) => symbol,
// Symbol is possibly declared
Ok(Symbol::Type(declared_ty, Boundness::PossiblyUnbound)) => {
let bindings = use_def.public_bindings(symbol);
let bindings = use_def.public_bindings(symbol_id);
let inferred = symbol_from_bindings(db, bindings);
match inferred {
@@ -120,12 +145,14 @@ fn symbol<'db>(db: &'db dyn Db, scope: ScopeId<'db>, name: &str) -> Symbol<'db>
),
}
}
// Symbol is undeclared, return the inferred type
// Symbol is undeclared, return the union of `Unknown` with the inferred type
Ok(Symbol::Unbound) => {
let bindings = use_def.public_bindings(symbol);
symbol_from_bindings(db, bindings)
let bindings = use_def.public_bindings(symbol_id);
let inferred = symbol_from_bindings(db, bindings);
widen_type_for_undeclared_public_symbol(db, inferred, is_dunder_slots || is_final)
}
// Symbol is possibly undeclared
// Symbol has conflicting declared types
Err((declared_ty, _)) => {
// Intentionally ignore conflicting declared types; that's not our problem,
// it's the problem of the module we are importing from.
@@ -177,9 +204,15 @@ fn symbol<'db>(db: &'db dyn Db, scope: ScopeId<'db>, name: &str) -> Symbol<'db>
}
let table = symbol_table(db, scope);
// `__slots__` is a symbol with special behavior in Python's runtime. It can be
// modified externally, but those changes do not take effect. We therefore issue
// a diagnostic if we see it being modified externally. In type inference, we
// can assign a "narrow" type to it even if it is not *declared*. This means, we
// do not have to call [`widen_type_for_undeclared_public_symbol`].
let is_dunder_slots = name == "__slots__";
table
.symbol_id_by_name(name)
.map(|symbol| symbol_by_id(db, scope, symbol))
.map(|symbol| symbol_by_id(db, scope, is_dunder_slots, symbol))
.unwrap_or(Symbol::Unbound)
}
@@ -378,6 +411,10 @@ impl SymbolAndQualifiers<'_> {
fn is_class_var(&self) -> bool {
self.1.contains(TypeQualifiers::CLASS_VAR)
}
fn is_final(&self) -> bool {
self.1.contains(TypeQualifiers::FINAL)
}
}
impl<'db> From<Symbol<'db>> for SymbolAndQualifiers<'db> {
@@ -4076,7 +4113,7 @@ impl<'db> Class<'db> {
/// this class, not on its superclasses.
fn own_instance_member(self, db: &'db dyn Db, name: &str) -> SymbolAndQualifiers<'db> {
// TODO: There are many things that are not yet implemented here:
// - `typing.ClassVar` and `typing.Final`
// - `typing.Final`
// - Proper diagnostics
// - Handling of possibly-undeclared/possibly-unbound attributes
// - The descriptor protocol
@@ -4084,10 +4121,10 @@ impl<'db> Class<'db> {
let body_scope = self.body_scope(db);
let table = symbol_table(db, body_scope);
if let Some(symbol) = table.symbol_id_by_name(name) {
if let Some(symbol_id) = table.symbol_id_by_name(name) {
let use_def = use_def_map(db, body_scope);
let declarations = use_def.public_declarations(symbol);
let declarations = use_def.public_declarations(symbol_id);
match symbol_from_declarations(db, declarations) {
Ok(SymbolAndQualifiers(Symbol::Type(declared_ty, _), qualifiers)) => {
@@ -4104,20 +4141,14 @@ impl<'db> Class<'db> {
SymbolAndQualifiers(Symbol::Type(declared_ty, Boundness::Bound), qualifiers)
}
}
Ok(SymbolAndQualifiers(Symbol::Unbound, qualifiers)) => {
let bindings = use_def.public_bindings(symbol);
Ok(symbol @ SymbolAndQualifiers(Symbol::Unbound, qualifiers)) => {
let bindings = use_def.public_bindings(symbol_id);
let inferred = symbol_from_bindings(db, bindings);
match inferred {
Symbol::Type(ty, _) => SymbolAndQualifiers(
Symbol::Type(
UnionType::from_elements(db, [Type::unknown(), ty]),
Boundness::Bound,
),
qualifiers,
),
Symbol::Unbound => SymbolAndQualifiers(Symbol::Unbound, qualifiers),
}
SymbolAndQualifiers(
widen_type_for_undeclared_public_symbol(db, inferred, symbol.is_final()),
qualifiers,
)
}
Err((declared_ty, _conflicting_declarations)) => {
// Ignore conflicting declarations
@@ -4694,7 +4725,10 @@ pub(crate) mod tests {
let bar = system_path_to_file(&db, "src/bar.py")?;
let a = global_symbol(&db, bar, "a");
assert_eq!(a.expect_type(), KnownClass::Int.to_instance(&db));
assert_eq!(
a.expect_type(),
UnionType::from_elements(&db, [Type::unknown(), KnownClass::Int.to_instance(&db)])
);
// Add a docstring to foo to trigger a re-run.
// The bar-call site of foo should not be re-run because of that
@@ -4710,7 +4744,10 @@ pub(crate) mod tests {
let a = global_symbol(&db, bar, "a");
assert_eq!(a.expect_type(), KnownClass::Int.to_instance(&db));
assert_eq!(
a.expect_type(),
UnionType::from_elements(&db, [Type::unknown(), KnownClass::Int.to_instance(&db)])
);
let events = db.take_salsa_events();
let call = &*parsed_module(&db, bar).syntax().body[1]

View File

@@ -543,7 +543,10 @@ mod tests {
assert_eq!(a_name, "a");
assert_eq!(b_name, "b");
// TODO resolution should not be deferred; we should see A not B
assert_eq!(a_annotated_ty.unwrap().display(&db).to_string(), "B");
assert_eq!(
a_annotated_ty.unwrap().display(&db).to_string(),
"Unknown | B"
);
assert_eq!(b_annotated_ty.unwrap().display(&db).to_string(), "T");
}
@@ -583,7 +586,10 @@ mod tests {
assert_eq!(a_name, "a");
assert_eq!(b_name, "b");
// Parameter resolution deferred; we should see B
assert_eq!(a_annotated_ty.unwrap().display(&db).to_string(), "B");
assert_eq!(
a_annotated_ty.unwrap().display(&db).to_string(),
"Unknown | B"
);
assert_eq!(b_annotated_ty.unwrap().display(&db).to_string(), "T");
}

View File

@@ -43,10 +43,10 @@ static EXPECTED_DIAGNOSTICS: &[&str] = &[
"warning[lint:possibly-unresolved-reference] /src/tomllib/_parser.py:579:12 Name `char` used when possibly not defined",
"warning[lint:possibly-unresolved-reference] /src/tomllib/_parser.py:580:63 Name `char` used when possibly not defined",
// We don't handle intersections in `is_assignable_to` yet
"error[lint:invalid-argument-type] /src/tomllib/_parser.py:626:46 Object of type `@Todo & ~AlwaysFalsy` cannot be assigned to parameter 1 (`match`) of function `match_to_datetime`; expected type `Match`",
"error[lint:invalid-argument-type] /src/tomllib/_parser.py:626:46 Object of type `Unknown & ~AlwaysFalsy | @Todo & ~AlwaysFalsy` cannot be assigned to parameter 1 (`match`) of function `match_to_datetime`; expected type `Match`",
"warning[lint:possibly-unresolved-reference] /src/tomllib/_parser.py:629:38 Name `datetime_obj` used when possibly not defined",
"error[lint:invalid-argument-type] /src/tomllib/_parser.py:632:58 Object of type `@Todo & ~AlwaysFalsy` cannot be assigned to parameter 1 (`match`) of function `match_to_localtime`; expected type `Match`",
"error[lint:invalid-argument-type] /src/tomllib/_parser.py:639:52 Object of type `@Todo & ~AlwaysFalsy` cannot be assigned to parameter 1 (`match`) of function `match_to_number`; expected type `Match`",
"error[lint:invalid-argument-type] /src/tomllib/_parser.py:632:58 Object of type `Unknown & ~AlwaysFalsy | @Todo & ~AlwaysFalsy` cannot be assigned to parameter 1 (`match`) of function `match_to_localtime`; expected type `Match`",
"error[lint:invalid-argument-type] /src/tomllib/_parser.py:639:52 Object of type `Unknown & ~AlwaysFalsy | @Todo & ~AlwaysFalsy` cannot be assigned to parameter 1 (`match`) of function `match_to_number`; expected type `Match`",
"warning[lint:unused-ignore-comment] /src/tomllib/_parser.py:682:31 Unused blanket `type: ignore` directive",
];

View File

@@ -0,0 +1,127 @@
from datetime import datetime
import pendulum
from airflow.decorators import dag, task
from airflow.models import DAG
from airflow.models.baseoperator import BaseOperator
from airflow.operators.dummy import DummyOperator
from airflow.plugins_manager import AirflowPlugin
from airflow.providers.standard.operators.python import PythonOperator
from airflow.utils.context import get_current_context
def access_invalid_key_in_context(**context):
print("access invalid key", context["conf"])
@task
def access_invalid_key_task_out_of_dag(**context):
print("access invalid key", context.get("conf"))
@dag(
schedule=None,
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
catchup=False,
tags=[""],
)
def invalid_dag():
@task()
def access_invalid_key_task(**context):
print("access invalid key", context.get("conf"))
task1 = PythonOperator(
task_id="task1",
python_callable=access_invalid_key_in_context,
)
access_invalid_key_task() >> task1
access_invalid_key_task_out_of_dag()
invalid_dag()
@task
def print_config(**context):
# This should not throw an error as logical_date is part of airflow context.
logical_date = context["logical_date"]
# Removed usage - should trigger violations
execution_date = context["execution_date"]
next_ds = context["next_ds"]
next_ds_nodash = context["next_ds_nodash"]
next_execution_date = context["next_execution_date"]
prev_ds = context["prev_ds"]
prev_ds_nodash = context["prev_ds_nodash"]
prev_execution_date = context["prev_execution_date"]
prev_execution_date_success = context["prev_execution_date_success"]
tomorrow_ds = context["tomorrow_ds"]
yesterday_ds = context["yesterday_ds"]
yesterday_ds_nodash = context["yesterday_ds_nodash"]
with DAG(
dag_id="example_dag",
schedule_interval="@daily",
start_date=datetime(2023, 1, 1),
template_searchpath=["/templates"],
) as dag:
task1 = DummyOperator(
task_id="task1",
params={
# Removed variables in template
"execution_date": "{{ execution_date }}",
"next_ds": "{{ next_ds }}",
"prev_ds": "{{ prev_ds }}"
},
)
class CustomMacrosPlugin(AirflowPlugin):
name = "custom_macros"
macros = {
"execution_date_macro": lambda context: context["execution_date"],
"next_ds_macro": lambda context: context["next_ds"]
}
@task
def print_config():
context = get_current_context()
execution_date = context["execution_date"]
next_ds = context["next_ds"]
next_ds_nodash = context["next_ds_nodash"]
next_execution_date = context["next_execution_date"]
prev_ds = context["prev_ds"]
prev_ds_nodash = context["prev_ds_nodash"]
prev_execution_date = context["prev_execution_date"]
prev_execution_date_success = context["prev_execution_date_success"]
tomorrow_ds = context["tomorrow_ds"]
yesterday_ds = context["yesterday_ds"]
yesterday_ds_nodash = context["yesterday_ds_nodash"]
class CustomOperator(BaseOperator):
def execute(self, context):
execution_date = context["execution_date"]
next_ds = context["next_ds"]
next_ds_nodash = context["next_ds_nodash"]
next_execution_date = context["next_execution_date"]
prev_ds = context["prev_ds"]
prev_ds_nodash = context["prev_ds_nodash"]
prev_execution_date = context["prev_execution_date"]
prev_execution_date_success = context["prev_execution_date_success"]
tomorrow_ds = context["tomorrow_ds"]
yesterday_ds = context["yesterday_ds"]
yesterday_ds_nodash = context["yesterday_ds_nodash"]
@task
def access_invalid_argument_task_out_of_dag(execution_date, tomorrow_ds, logical_date, **context):
print("execution date", execution_date)
print("access invalid key", context.get("conf"))
@task(task_id="print_the_context")
def print_context(ds=None, **kwargs):
"""Print the Airflow context and ds variable from the context."""
print(ds)
print(kwargs.get("tomorrow_ds"))
c = get_current_context()
c.get("execution_date")
class CustomOperatorNew(BaseOperator):
def execute(self, context):
execution_date = context.get("execution_date")
next_ds = context.get("next_ds")

View File

@@ -0,0 +1,40 @@
from typing import TYPE_CHECKING
# Verify that statements nested in conditionals (such as top-level type-checking blocks)
# are still considered top-level
if TYPE_CHECKING:
import string
def import_in_function():
import symtable # [import-outside-toplevel]
import os, sys # [import-outside-toplevel]
import time as thyme # [import-outside-toplevel]
import random as rand, socket as sock # [import-outside-toplevel]
from collections import defaultdict # [import-outside-toplevel]
from math import sin as sign, cos as cosplay # [import-outside-toplevel]
# these should be allowed due to TID253 top-level ban
import foo_banned
import foo_banned as renamed
from pkg import bar_banned
from pkg import bar_banned as renamed
from pkg_banned import one as other, two, three
# this should still trigger an error due to multiple imports
from pkg import foo_allowed, bar_banned # [import-outside-toplevel]
class ClassWithImports:
import tokenize # [import-outside-toplevel]
def __init__(self):
import trace # [import-outside-toplevel]
# these should be allowed due to TID253 top-level ban
import foo_banned
import foo_banned as renamed
from pkg import bar_banned
from pkg import bar_banned as renamed
from pkg_banned import one as other, two, three
# this should still trigger an error due to multiple imports
from pkg import foo_allowed, bar_banned # [import-outside-toplevel]

View File

@@ -64,3 +64,20 @@ round(lorem, -2) # No error
round(lorem, inferred_int) # No error
round(lorem, 3 + 4) # No error
round(lorem, foo) # No error
# Fixes should preserve parentheses when argument
# contains newline.
# See https://github.com/astral-sh/ruff/issues/15598
round(-
1)
round(1
*1
)
# fix should be unsafe if comment is in call range
round(# a comment
17
)
round(
17 # a comment
)

View File

@@ -175,7 +175,9 @@ pub(crate) fn expression(expr: &Expr, checker: &mut Checker) {
if checker.enabled(Rule::NonPEP646Unpack) {
pyupgrade::rules::use_pep646_unpack(checker, subscript);
}
if checker.enabled(Rule::Airflow3Removal) {
airflow::rules::removed_in_3(checker, expr);
}
pandas_vet::rules::subscript(checker, value, expr);
}
Expr::Tuple(ast::ExprTuple {

View File

@@ -376,6 +376,9 @@ pub(crate) fn statement(stmt: &Stmt, checker: &mut Checker) {
if checker.enabled(Rule::PytestParameterWithDefaultArgument) {
flake8_pytest_style::rules::parameter_with_default_argument(checker, function_def);
}
if checker.enabled(Rule::Airflow3Removal) {
airflow::rules::removed_in_3_function_def(checker, function_def);
}
if checker.enabled(Rule::NonPEP695GenericFunction) {
pyupgrade::rules::non_pep695_generic_function(checker, function_def);
}
@@ -605,6 +608,10 @@ pub(crate) fn statement(stmt: &Stmt, checker: &mut Checker) {
flake8_bandit::rules::suspicious_imports(checker, stmt);
}
if checker.enabled(Rule::BannedModuleLevelImports) {
flake8_tidy_imports::rules::banned_module_level_imports(checker, stmt);
}
for alias in names {
if checker.enabled(Rule::NonAsciiImportName) {
pylint::rules::non_ascii_module_import(checker, alias);
@@ -629,18 +636,6 @@ pub(crate) fn statement(stmt: &Stmt, checker: &mut Checker) {
);
}
if checker.enabled(Rule::BannedModuleLevelImports) {
flake8_tidy_imports::rules::banned_module_level_imports(
checker,
&flake8_tidy_imports::matchers::NameMatchPolicy::MatchNameOrParent(
flake8_tidy_imports::matchers::MatchNameOrParent {
module: &alias.name,
},
),
&alias,
);
}
if !checker.source_type.is_stub() {
if checker.enabled(Rule::UselessImportAlias) {
pylint::rules::useless_import_alias(checker, alias);
@@ -845,36 +840,9 @@ pub(crate) fn statement(stmt: &Stmt, checker: &mut Checker) {
}
}
if checker.enabled(Rule::BannedModuleLevelImports) {
if let Some(module) = helpers::resolve_imported_module_path(
level,
module,
checker.module.qualified_name(),
) {
flake8_tidy_imports::rules::banned_module_level_imports(
checker,
&flake8_tidy_imports::matchers::NameMatchPolicy::MatchNameOrParent(
flake8_tidy_imports::matchers::MatchNameOrParent { module: &module },
),
&stmt,
);
for alias in names {
if &alias.name == "*" {
continue;
}
flake8_tidy_imports::rules::banned_module_level_imports(
checker,
&flake8_tidy_imports::matchers::NameMatchPolicy::MatchName(
flake8_tidy_imports::matchers::MatchName {
module: &module,
member: &alias.name,
},
),
&alias,
);
}
}
flake8_tidy_imports::rules::banned_module_level_imports(checker, stmt);
}
if checker.enabled(Rule::PytestIncorrectPytestImport) {
if let Some(diagnostic) =
flake8_pytest_style::rules::import_from(stmt, module, level)

View File

@@ -189,7 +189,7 @@ pub(crate) struct Checker<'a> {
/// The [`Path`] to the package containing the current file.
package: Option<PackageRoot<'a>>,
/// The module representation of the current file (e.g., `foo.bar`).
module: Module<'a>,
pub(crate) module: Module<'a>,
/// The [`PySourceType`] of the current file.
pub(crate) source_type: PySourceType,
/// The [`CellOffsets`] for the current file, if it's a Jupyter notebook.

View File

@@ -316,7 +316,7 @@ mod schema {
// Filter out all test-only rules
#[cfg(any(feature = "test-rules", test))]
#[allow(clippy::used_underscore_binding)]
if _rule.starts_with("RUF9") {
if _rule.starts_with("RUF9") || _rule == "PLW0101" {
return false;
}

View File

@@ -18,6 +18,7 @@ mod tests {
#[test_case(Rule::Airflow3Removal, Path::new("AIR302_names.py"))]
#[test_case(Rule::Airflow3Removal, Path::new("AIR302_class_attribute.py"))]
#[test_case(Rule::Airflow3Removal, Path::new("AIR302_airflow_plugin.py"))]
#[test_case(Rule::Airflow3Removal, Path::new("AIR302_context.py"))]
#[test_case(Rule::Airflow3MovedToProvider, Path::new("AIR303.py"))]
fn rules(rule_code: Rule, path: &Path) -> Result<()> {
let snapshot = format!("{}_{}", rule_code.noqa_code(), path.to_string_lossy());

View File

@@ -1,17 +1,18 @@
use crate::checkers::ast::Checker;
use ruff_diagnostics::{Diagnostic, Edit, Fix, FixAvailability, Violation};
use ruff_macros::{derive_message_formats, ViolationMetadata};
use ruff_python_ast::helpers::map_callable;
use ruff_python_ast::{
name::QualifiedName, Arguments, Expr, ExprAttribute, ExprCall, ExprContext, ExprName,
StmtClassDef,
ExprStringLiteral, ExprSubscript, Stmt, StmtClassDef, StmtFunctionDef,
};
use ruff_python_semantic::analyze::typing;
use ruff_python_semantic::Modules;
use ruff_python_semantic::ScopeKind;
use ruff_python_semantic::SemanticModel;
use ruff_text_size::Ranged;
use ruff_text_size::TextRange;
use crate::checkers::ast::Checker;
/// ## What it does
/// Checks for uses of deprecated Airflow functions and values.
///
@@ -71,6 +72,13 @@ impl Violation for Airflow3Removal {
}
}
#[derive(Debug, Eq, PartialEq)]
enum Replacement {
None,
Name(&'static str),
Message(&'static str),
}
/// AIR302
pub(crate) fn removed_in_3(checker: &mut Checker, expr: &Expr) {
if !checker.semantic().seen_module(Modules::AIRFLOW) {
@@ -83,10 +91,11 @@ pub(crate) fn removed_in_3(checker: &mut Checker, expr: &Expr) {
func, arguments, ..
},
) => {
if let Some(qualname) = checker.semantic().resolve_qualified_name(func) {
check_call_arguments(checker, &qualname, arguments);
if let Some(qualified_name) = checker.semantic().resolve_qualified_name(func) {
check_call_arguments(checker, &qualified_name, arguments);
};
check_method(checker, call_expr);
check_context_key_usage_in_call(checker, call_expr);
}
Expr::Attribute(attribute_expr @ ExprAttribute { attr, .. }) => {
check_name(checker, expr, attr.range());
@@ -100,15 +109,67 @@ pub(crate) fn removed_in_3(checker: &mut Checker, expr: &Expr) {
}
}
}
Expr::Subscript(subscript_expr) => {
check_context_key_usage_in_subscript(checker, subscript_expr);
}
_ => {}
}
}
#[derive(Debug, Eq, PartialEq)]
enum Replacement {
None,
Name(&'static str),
Message(&'static str),
/// AIR302
pub(crate) fn removed_in_3_function_def(checker: &mut Checker, function_def: &StmtFunctionDef) {
if !checker.semantic().seen_module(Modules::AIRFLOW) {
return;
}
check_function_parameters(checker, function_def);
}
const REMOVED_CONTEXT_KEYS: [&str; 12] = [
"conf",
"execution_date",
"next_ds",
"next_ds_nodash",
"next_execution_date",
"prev_ds",
"prev_ds_nodash",
"prev_execution_date",
"prev_execution_date_success",
"tomorrow_ds",
"yesterday_ds",
"yesterday_ds_nodash",
];
/// Check the function parameters for removed context keys.
///
/// For example:
///
/// ```python
/// from airflow.decorators import task
///
/// @task
/// def another_task(execution_date, **kwargs):
/// # ^^^^^^^^^^^^^^
/// # 'execution_date' is removed in Airflow 3.0
/// pass
/// ```
fn check_function_parameters(checker: &mut Checker, function_def: &StmtFunctionDef) {
if !is_airflow_task(function_def, checker.semantic()) {
return;
}
for param in function_def.parameters.iter_non_variadic_params() {
let param_name = param.parameter.name.as_str();
if REMOVED_CONTEXT_KEYS.contains(&param_name) {
checker.diagnostics.push(Diagnostic::new(
Airflow3Removal {
deprecated: param_name.to_string(),
replacement: Replacement::None,
},
param.parameter.name.range(),
));
}
}
}
/// Check whether a removed Airflow argument is passed.
@@ -120,8 +181,12 @@ enum Replacement {
///
/// DAG(schedule_interval="@daily")
/// ```
fn check_call_arguments(checker: &mut Checker, qualname: &QualifiedName, arguments: &Arguments) {
match qualname.segments() {
fn check_call_arguments(
checker: &mut Checker,
qualified_name: &QualifiedName,
arguments: &Arguments,
) {
match qualified_name.segments() {
["airflow", .., "DAG" | "dag"] => {
checker.diagnostics.extend(diagnostic_for_argument(
arguments,
@@ -145,7 +210,7 @@ fn check_call_arguments(checker: &mut Checker, qualname: &QualifiedName, argumen
));
}
_ => {
if is_airflow_auth_manager(qualname.segments()) {
if is_airflow_auth_manager(qualified_name.segments()) {
if !arguments.is_empty() {
checker.diagnostics.push(Diagnostic::new(
Airflow3Removal {
@@ -157,13 +222,13 @@ fn check_call_arguments(checker: &mut Checker, qualname: &QualifiedName, argumen
arguments.range(),
));
}
} else if is_airflow_task_handler(qualname.segments()) {
} else if is_airflow_task_handler(qualified_name.segments()) {
checker.diagnostics.extend(diagnostic_for_argument(
arguments,
"filename_template",
None,
));
} else if is_airflow_operator(qualname.segments()) {
} else if is_airflow_operator(qualified_name.segments()) {
checker
.diagnostics
.extend(diagnostic_for_argument(arguments, "sla", None));
@@ -172,7 +237,7 @@ fn check_call_arguments(checker: &mut Checker, qualname: &QualifiedName, argumen
"task_concurrency",
Some("max_active_tis_per_dag"),
));
match qualname.segments() {
match qualified_name.segments() {
["airflow", .., "operators", "trigger_dagrun", "TriggerDagRunOperator"] => {
checker.diagnostics.extend(diagnostic_for_argument(
arguments,
@@ -252,6 +317,137 @@ fn check_class_attribute(checker: &mut Checker, attribute_expr: &ExprAttribute)
}
}
/// Checks whether an Airflow 3.0removed context key is used in a function decorated with `@task`.
///
/// Specifically, it flags the following two scenarios:
///
/// 1. A removed context key accessed via `context.get("...")` where context is coming from
/// `get_current_context` function.
///
/// ```python
/// from airflow.decorators import task
/// from airflow.utils.context import get_current_context
///
///
/// @task
/// def my_task():
/// context = get_current_context()
/// context.get("conf") # 'conf' is removed in Airflow 3.0
/// ```
///
/// 2. A removed context key accessed via `context.get("...")` where context is a kwarg parameter.
///
/// ```python
/// from airflow.decorators import task
///
///
/// @task
/// def my_task(**context):
/// context.get("conf") # 'conf' is removed in Airflow 3.0
/// ```
fn check_context_key_usage_in_call(checker: &mut Checker, call_expr: &ExprCall) {
if !in_airflow_task_function(checker.semantic()) {
return;
}
let Expr::Attribute(ExprAttribute { value, attr, .. }) = &*call_expr.func else {
return;
};
if attr.as_str() != "get" {
return;
}
let is_kwarg_parameter = value
.as_name_expr()
.is_some_and(|name| is_kwarg_parameter(checker.semantic(), name));
let is_assigned_from_get_current_context =
typing::resolve_assignment(value, checker.semantic()).is_some_and(|qualified_name| {
matches!(
qualified_name.segments(),
["airflow", "utils", "context", "get_current_context"]
)
});
if !(is_kwarg_parameter || is_assigned_from_get_current_context) {
return;
}
for removed_key in REMOVED_CONTEXT_KEYS {
let Some(Expr::StringLiteral(ExprStringLiteral { value, range })) =
call_expr.arguments.find_positional(0)
else {
continue;
};
if value == removed_key {
checker.diagnostics.push(Diagnostic::new(
Airflow3Removal {
deprecated: removed_key.to_string(),
replacement: Replacement::None,
},
*range,
));
}
}
}
/// Check if a subscript expression accesses a removed Airflow context variable.
/// If a removed key is found, push a corresponding diagnostic.
fn check_context_key_usage_in_subscript(checker: &mut Checker, subscript: &ExprSubscript) {
if !in_airflow_task_function(checker.semantic()) {
return;
}
let ExprSubscript { value, slice, .. } = subscript;
let Some(ExprStringLiteral { value: key, .. }) = slice.as_string_literal_expr() else {
return;
};
let is_kwarg_parameter = value
.as_name_expr()
.is_some_and(|name| is_kwarg_parameter(checker.semantic(), name));
let is_assigned_from_get_current_context =
typing::resolve_assignment(value, checker.semantic()).is_some_and(|qualified_name| {
matches!(
qualified_name.segments(),
["airflow", "utils", "context", "get_current_context"]
)
});
if !(is_kwarg_parameter || is_assigned_from_get_current_context) {
return;
}
if REMOVED_CONTEXT_KEYS.contains(&key.to_str()) {
checker.diagnostics.push(Diagnostic::new(
Airflow3Removal {
deprecated: key.to_string(),
replacement: Replacement::None,
},
slice.range(),
));
}
}
/// Finds the parameter definition for a given name expression in a function.
fn is_kwarg_parameter(semantic: &SemanticModel, name: &ExprName) -> bool {
let Some(binding_id) = semantic.only_binding(name) else {
return false;
};
let binding = semantic.binding(binding_id);
let Some(Stmt::FunctionDef(StmtFunctionDef { parameters, .. })) = binding.statement(semantic)
else {
return false;
};
parameters
.kwarg
.as_deref()
.is_some_and(|kwarg| kwarg.name.as_str() == name.id.as_str())
}
/// Check whether a removed Airflow class method is called.
///
/// For example:
@@ -860,3 +1056,23 @@ fn is_airflow_builtin_or_provider(segments: &[&str], module: &str, symbol_suffix
_ => false,
}
}
/// Returns `true` if the current statement hierarchy has a function that's decorated with
/// `@airflow.decorators.task`.
fn in_airflow_task_function(semantic: &SemanticModel) -> bool {
semantic
.current_statements()
.find_map(|stmt| stmt.as_function_def_stmt())
.is_some_and(|function_def| is_airflow_task(function_def, semantic))
}
/// Returns `true` if the given function is decorated with `@airflow.decorators.task`.
fn is_airflow_task(function_def: &StmtFunctionDef, semantic: &SemanticModel) -> bool {
function_def.decorator_list.iter().any(|decorator| {
semantic
.resolve_qualified_name(map_callable(&decorator.expression))
.is_some_and(|qualified_name| {
matches!(qualified_name.segments(), ["airflow", "decorators", "task"])
})
})
}

View File

@@ -0,0 +1,319 @@
---
source: crates/ruff_linter/src/rules/airflow/mod.rs
---
AIR302_context.py:19:45: AIR302 `conf` is removed in Airflow 3.0
|
17 | @task
18 | def access_invalid_key_task_out_of_dag(**context):
19 | print("access invalid key", context.get("conf"))
| ^^^^^^ AIR302
20 |
21 | @dag(
|
AIR302_context.py:30:49: AIR302 `conf` is removed in Airflow 3.0
|
28 | @task()
29 | def access_invalid_key_task(**context):
30 | print("access invalid key", context.get("conf"))
| ^^^^^^ AIR302
31 |
32 | task1 = PythonOperator(
|
AIR302_context.py:47:30: AIR302 `execution_date` is removed in Airflow 3.0
|
46 | # Removed usage - should trigger violations
47 | execution_date = context["execution_date"]
| ^^^^^^^^^^^^^^^^ AIR302
48 | next_ds = context["next_ds"]
49 | next_ds_nodash = context["next_ds_nodash"]
|
AIR302_context.py:48:23: AIR302 `next_ds` is removed in Airflow 3.0
|
46 | # Removed usage - should trigger violations
47 | execution_date = context["execution_date"]
48 | next_ds = context["next_ds"]
| ^^^^^^^^^ AIR302
49 | next_ds_nodash = context["next_ds_nodash"]
50 | next_execution_date = context["next_execution_date"]
|
AIR302_context.py:49:30: AIR302 `next_ds_nodash` is removed in Airflow 3.0
|
47 | execution_date = context["execution_date"]
48 | next_ds = context["next_ds"]
49 | next_ds_nodash = context["next_ds_nodash"]
| ^^^^^^^^^^^^^^^^ AIR302
50 | next_execution_date = context["next_execution_date"]
51 | prev_ds = context["prev_ds"]
|
AIR302_context.py:50:35: AIR302 `next_execution_date` is removed in Airflow 3.0
|
48 | next_ds = context["next_ds"]
49 | next_ds_nodash = context["next_ds_nodash"]
50 | next_execution_date = context["next_execution_date"]
| ^^^^^^^^^^^^^^^^^^^^^ AIR302
51 | prev_ds = context["prev_ds"]
52 | prev_ds_nodash = context["prev_ds_nodash"]
|
AIR302_context.py:51:23: AIR302 `prev_ds` is removed in Airflow 3.0
|
49 | next_ds_nodash = context["next_ds_nodash"]
50 | next_execution_date = context["next_execution_date"]
51 | prev_ds = context["prev_ds"]
| ^^^^^^^^^ AIR302
52 | prev_ds_nodash = context["prev_ds_nodash"]
53 | prev_execution_date = context["prev_execution_date"]
|
AIR302_context.py:52:30: AIR302 `prev_ds_nodash` is removed in Airflow 3.0
|
50 | next_execution_date = context["next_execution_date"]
51 | prev_ds = context["prev_ds"]
52 | prev_ds_nodash = context["prev_ds_nodash"]
| ^^^^^^^^^^^^^^^^ AIR302
53 | prev_execution_date = context["prev_execution_date"]
54 | prev_execution_date_success = context["prev_execution_date_success"]
|
AIR302_context.py:53:35: AIR302 `prev_execution_date` is removed in Airflow 3.0
|
51 | prev_ds = context["prev_ds"]
52 | prev_ds_nodash = context["prev_ds_nodash"]
53 | prev_execution_date = context["prev_execution_date"]
| ^^^^^^^^^^^^^^^^^^^^^ AIR302
54 | prev_execution_date_success = context["prev_execution_date_success"]
55 | tomorrow_ds = context["tomorrow_ds"]
|
AIR302_context.py:54:43: AIR302 `prev_execution_date_success` is removed in Airflow 3.0
|
52 | prev_ds_nodash = context["prev_ds_nodash"]
53 | prev_execution_date = context["prev_execution_date"]
54 | prev_execution_date_success = context["prev_execution_date_success"]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AIR302
55 | tomorrow_ds = context["tomorrow_ds"]
56 | yesterday_ds = context["yesterday_ds"]
|
AIR302_context.py:55:27: AIR302 `tomorrow_ds` is removed in Airflow 3.0
|
53 | prev_execution_date = context["prev_execution_date"]
54 | prev_execution_date_success = context["prev_execution_date_success"]
55 | tomorrow_ds = context["tomorrow_ds"]
| ^^^^^^^^^^^^^ AIR302
56 | yesterday_ds = context["yesterday_ds"]
57 | yesterday_ds_nodash = context["yesterday_ds_nodash"]
|
AIR302_context.py:56:28: AIR302 `yesterday_ds` is removed in Airflow 3.0
|
54 | prev_execution_date_success = context["prev_execution_date_success"]
55 | tomorrow_ds = context["tomorrow_ds"]
56 | yesterday_ds = context["yesterday_ds"]
| ^^^^^^^^^^^^^^ AIR302
57 | yesterday_ds_nodash = context["yesterday_ds_nodash"]
|
AIR302_context.py:57:35: AIR302 `yesterday_ds_nodash` is removed in Airflow 3.0
|
55 | tomorrow_ds = context["tomorrow_ds"]
56 | yesterday_ds = context["yesterday_ds"]
57 | yesterday_ds_nodash = context["yesterday_ds_nodash"]
| ^^^^^^^^^^^^^^^^^^^^^ AIR302
58 |
59 | with DAG(
|
AIR302_context.py:61:5: AIR302 [*] `schedule_interval` is removed in Airflow 3.0
|
59 | with DAG(
60 | dag_id="example_dag",
61 | schedule_interval="@daily",
| ^^^^^^^^^^^^^^^^^ AIR302
62 | start_date=datetime(2023, 1, 1),
63 | template_searchpath=["/templates"],
|
= help: Use `schedule` instead
Safe fix
58 58 |
59 59 | with DAG(
60 60 | dag_id="example_dag",
61 |- schedule_interval="@daily",
61 |+ schedule="@daily",
62 62 | start_date=datetime(2023, 1, 1),
63 63 | template_searchpath=["/templates"],
64 64 | ) as dag:
AIR302_context.py:65:13: AIR302 `airflow.operators.dummy.DummyOperator` is removed in Airflow 3.0
|
63 | template_searchpath=["/templates"],
64 | ) as dag:
65 | task1 = DummyOperator(
| ^^^^^^^^^^^^^ AIR302
66 | task_id="task1",
67 | params={
|
= help: Use `airflow.operators.empty.EmptyOperator` instead
AIR302_context.py:85:30: AIR302 `execution_date` is removed in Airflow 3.0
|
83 | def print_config():
84 | context = get_current_context()
85 | execution_date = context["execution_date"]
| ^^^^^^^^^^^^^^^^ AIR302
86 | next_ds = context["next_ds"]
87 | next_ds_nodash = context["next_ds_nodash"]
|
AIR302_context.py:86:23: AIR302 `next_ds` is removed in Airflow 3.0
|
84 | context = get_current_context()
85 | execution_date = context["execution_date"]
86 | next_ds = context["next_ds"]
| ^^^^^^^^^ AIR302
87 | next_ds_nodash = context["next_ds_nodash"]
88 | next_execution_date = context["next_execution_date"]
|
AIR302_context.py:87:30: AIR302 `next_ds_nodash` is removed in Airflow 3.0
|
85 | execution_date = context["execution_date"]
86 | next_ds = context["next_ds"]
87 | next_ds_nodash = context["next_ds_nodash"]
| ^^^^^^^^^^^^^^^^ AIR302
88 | next_execution_date = context["next_execution_date"]
89 | prev_ds = context["prev_ds"]
|
AIR302_context.py:88:35: AIR302 `next_execution_date` is removed in Airflow 3.0
|
86 | next_ds = context["next_ds"]
87 | next_ds_nodash = context["next_ds_nodash"]
88 | next_execution_date = context["next_execution_date"]
| ^^^^^^^^^^^^^^^^^^^^^ AIR302
89 | prev_ds = context["prev_ds"]
90 | prev_ds_nodash = context["prev_ds_nodash"]
|
AIR302_context.py:89:23: AIR302 `prev_ds` is removed in Airflow 3.0
|
87 | next_ds_nodash = context["next_ds_nodash"]
88 | next_execution_date = context["next_execution_date"]
89 | prev_ds = context["prev_ds"]
| ^^^^^^^^^ AIR302
90 | prev_ds_nodash = context["prev_ds_nodash"]
91 | prev_execution_date = context["prev_execution_date"]
|
AIR302_context.py:90:30: AIR302 `prev_ds_nodash` is removed in Airflow 3.0
|
88 | next_execution_date = context["next_execution_date"]
89 | prev_ds = context["prev_ds"]
90 | prev_ds_nodash = context["prev_ds_nodash"]
| ^^^^^^^^^^^^^^^^ AIR302
91 | prev_execution_date = context["prev_execution_date"]
92 | prev_execution_date_success = context["prev_execution_date_success"]
|
AIR302_context.py:91:35: AIR302 `prev_execution_date` is removed in Airflow 3.0
|
89 | prev_ds = context["prev_ds"]
90 | prev_ds_nodash = context["prev_ds_nodash"]
91 | prev_execution_date = context["prev_execution_date"]
| ^^^^^^^^^^^^^^^^^^^^^ AIR302
92 | prev_execution_date_success = context["prev_execution_date_success"]
93 | tomorrow_ds = context["tomorrow_ds"]
|
AIR302_context.py:92:43: AIR302 `prev_execution_date_success` is removed in Airflow 3.0
|
90 | prev_ds_nodash = context["prev_ds_nodash"]
91 | prev_execution_date = context["prev_execution_date"]
92 | prev_execution_date_success = context["prev_execution_date_success"]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AIR302
93 | tomorrow_ds = context["tomorrow_ds"]
94 | yesterday_ds = context["yesterday_ds"]
|
AIR302_context.py:93:27: AIR302 `tomorrow_ds` is removed in Airflow 3.0
|
91 | prev_execution_date = context["prev_execution_date"]
92 | prev_execution_date_success = context["prev_execution_date_success"]
93 | tomorrow_ds = context["tomorrow_ds"]
| ^^^^^^^^^^^^^ AIR302
94 | yesterday_ds = context["yesterday_ds"]
95 | yesterday_ds_nodash = context["yesterday_ds_nodash"]
|
AIR302_context.py:94:28: AIR302 `yesterday_ds` is removed in Airflow 3.0
|
92 | prev_execution_date_success = context["prev_execution_date_success"]
93 | tomorrow_ds = context["tomorrow_ds"]
94 | yesterday_ds = context["yesterday_ds"]
| ^^^^^^^^^^^^^^ AIR302
95 | yesterday_ds_nodash = context["yesterday_ds_nodash"]
|
AIR302_context.py:95:35: AIR302 `yesterday_ds_nodash` is removed in Airflow 3.0
|
93 | tomorrow_ds = context["tomorrow_ds"]
94 | yesterday_ds = context["yesterday_ds"]
95 | yesterday_ds_nodash = context["yesterday_ds_nodash"]
| ^^^^^^^^^^^^^^^^^^^^^ AIR302
96 |
97 | class CustomOperator(BaseOperator):
|
AIR302_context.py:112:45: AIR302 `execution_date` is removed in Airflow 3.0
|
111 | @task
112 | def access_invalid_argument_task_out_of_dag(execution_date, tomorrow_ds, logical_date, **context):
| ^^^^^^^^^^^^^^ AIR302
113 | print("execution date", execution_date)
114 | print("access invalid key", context.get("conf"))
|
AIR302_context.py:112:61: AIR302 `tomorrow_ds` is removed in Airflow 3.0
|
111 | @task
112 | def access_invalid_argument_task_out_of_dag(execution_date, tomorrow_ds, logical_date, **context):
| ^^^^^^^^^^^ AIR302
113 | print("execution date", execution_date)
114 | print("access invalid key", context.get("conf"))
|
AIR302_context.py:114:45: AIR302 `conf` is removed in Airflow 3.0
|
112 | def access_invalid_argument_task_out_of_dag(execution_date, tomorrow_ds, logical_date, **context):
113 | print("execution date", execution_date)
114 | print("access invalid key", context.get("conf"))
| ^^^^^^ AIR302
115 |
116 | @task(task_id="print_the_context")
|
AIR302_context.py:120:22: AIR302 `tomorrow_ds` is removed in Airflow 3.0
|
118 | """Print the Airflow context and ds variable from the context."""
119 | print(ds)
120 | print(kwargs.get("tomorrow_ds"))
| ^^^^^^^^^^^^^ AIR302
121 | c = get_current_context()
122 | c.get("execution_date")
|
AIR302_context.py:122:11: AIR302 `execution_date` is removed in Airflow 3.0
|
120 | print(kwargs.get("tomorrow_ds"))
121 | c = get_current_context()
122 | c.get("execution_date")
| ^^^^^^^^^^^^^^^^ AIR302
123 |
124 | class CustomOperatorNew(BaseOperator):
|

View File

@@ -1,9 +1,12 @@
use ruff_diagnostics::{Diagnostic, Violation};
use ruff_macros::{derive_message_formats, ViolationMetadata};
use ruff_python_ast::helpers::resolve_imported_module_path;
use ruff_python_ast::{Alias, AnyNodeRef, Stmt, StmtImport, StmtImportFrom};
use ruff_text_size::Ranged;
use std::borrow::Cow;
use crate::checkers::ast::Checker;
use crate::rules::flake8_tidy_imports::matchers::NameMatchPolicy;
use crate::rules::flake8_tidy_imports::matchers::{MatchName, MatchNameOrParent, NameMatchPolicy};
/// ## What it does
/// Checks for module-level imports that should instead be imported lazily
@@ -53,28 +56,131 @@ impl Violation for BannedModuleLevelImports {
}
/// TID253
pub(crate) fn banned_module_level_imports<T: Ranged>(
checker: &mut Checker,
policy: &NameMatchPolicy,
node: &T,
) {
pub(crate) fn banned_module_level_imports(checker: &mut Checker, stmt: &Stmt) {
if !checker.semantic().at_top_level() {
return;
}
if let Some(banned_module) = policy.find(
checker
.settings
.flake8_tidy_imports
.banned_module_level_imports
.iter()
.map(AsRef::as_ref),
) {
checker.diagnostics.push(Diagnostic::new(
BannedModuleLevelImports {
name: banned_module,
},
node.range(),
));
for (policy, node) in &BannedModuleImportPolicies::new(stmt, checker) {
if let Some(banned_module) = policy.find(
checker
.settings
.flake8_tidy_imports
.banned_module_level_imports(),
) {
checker.diagnostics.push(Diagnostic::new(
BannedModuleLevelImports {
name: banned_module,
},
node.range(),
));
}
}
}
pub(crate) enum BannedModuleImportPolicies<'a> {
Import(&'a StmtImport),
ImportFrom {
module: Option<Cow<'a, str>>,
node: &'a StmtImportFrom,
},
NonImport,
}
impl<'a> BannedModuleImportPolicies<'a> {
pub(crate) fn new(stmt: &'a Stmt, checker: &Checker) -> Self {
match stmt {
Stmt::Import(import) => Self::Import(import),
Stmt::ImportFrom(import @ StmtImportFrom { module, level, .. }) => {
let module = resolve_imported_module_path(
*level,
module.as_deref(),
checker.module.qualified_name(),
);
Self::ImportFrom {
module,
node: import,
}
}
_ => Self::NonImport,
}
}
}
impl<'a> IntoIterator for &'a BannedModuleImportPolicies<'a> {
type Item = <Self::IntoIter as Iterator>::Item;
type IntoIter = BannedModuleImportPoliciesIter<'a>;
fn into_iter(self) -> Self::IntoIter {
match self {
BannedModuleImportPolicies::Import(import) => {
BannedModuleImportPoliciesIter::Import(import.names.iter())
}
BannedModuleImportPolicies::ImportFrom { module, node } => {
BannedModuleImportPoliciesIter::ImportFrom {
module: module.as_deref(),
names: node.names.iter(),
import: Some(node),
}
}
BannedModuleImportPolicies::NonImport => BannedModuleImportPoliciesIter::NonImport,
}
}
}
pub(crate) enum BannedModuleImportPoliciesIter<'a> {
Import(std::slice::Iter<'a, Alias>),
ImportFrom {
module: Option<&'a str>,
names: std::slice::Iter<'a, Alias>,
import: Option<&'a StmtImportFrom>,
},
NonImport,
}
impl<'a> Iterator for BannedModuleImportPoliciesIter<'a> {
type Item = (NameMatchPolicy<'a>, AnyNodeRef<'a>);
fn next(&mut self) -> Option<Self::Item> {
match self {
Self::Import(names) => {
let name = names.next()?;
Some((
NameMatchPolicy::MatchNameOrParent(MatchNameOrParent { module: &name.name }),
name.into(),
))
}
Self::ImportFrom {
module,
import,
names,
} => {
let module = module.as_ref()?;
if let Some(import) = import.take() {
return Some((
NameMatchPolicy::MatchNameOrParent(MatchNameOrParent { module }),
import.into(),
));
}
loop {
let alias = names.next()?;
if &alias.name == "*" {
continue;
}
break Some((
NameMatchPolicy::MatchName(MatchName {
module,
member: &alias.name,
}),
alias.into(),
));
}
}
Self::NonImport => None,
}
}
}

View File

@@ -46,6 +46,12 @@ pub struct Settings {
pub banned_module_level_imports: Vec<String>,
}
impl Settings {
pub fn banned_module_level_imports(&self) -> impl Iterator<Item = &str> {
self.banned_module_level_imports.iter().map(AsRef::as_ref)
}
}
impl Display for Settings {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
display_settings! {

View File

@@ -13,7 +13,7 @@ mod tests {
use test_case::test_case;
use crate::registry::Rule;
use crate::rules::pylint;
use crate::rules::{flake8_tidy_imports, pylint};
use crate::settings::types::{PreviewMode, PythonVersion};
use crate::settings::LinterSettings;
@@ -412,6 +412,30 @@ mod tests {
Ok(())
}
#[test]
fn import_outside_top_level_with_banned() -> Result<()> {
let diagnostics = test_path(
Path::new("pylint/import_outside_top_level_with_banned.py"),
&LinterSettings {
preview: PreviewMode::Enabled,
flake8_tidy_imports: flake8_tidy_imports::settings::Settings {
banned_module_level_imports: vec![
"foo_banned".to_string(),
"pkg_banned".to_string(),
"pkg.bar_banned".to_string(),
],
..Default::default()
},
..LinterSettings::for_rules(vec![
Rule::BannedModuleLevelImports,
Rule::ImportOutsideTopLevel,
])
},
)?;
assert_messages!(diagnostics);
Ok(())
}
#[test_case(
Rule::RepeatedEqualityComparison,
Path::new("repeated_equality_comparison.py")

View File

@@ -3,7 +3,10 @@ use ruff_macros::{derive_message_formats, ViolationMetadata};
use ruff_python_ast::Stmt;
use ruff_text_size::Ranged;
use crate::checkers::ast::Checker;
use crate::rules::flake8_tidy_imports::rules::BannedModuleImportPolicies;
use crate::{
checkers::ast::Checker, codes::Rule, rules::flake8_tidy_imports::matchers::NameMatchPolicy,
};
/// ## What it does
/// Checks for `import` statements outside of a module's top-level scope, such
@@ -54,9 +57,45 @@ impl Violation for ImportOutsideTopLevel {
/// C0415
pub(crate) fn import_outside_top_level(checker: &mut Checker, stmt: &Stmt) {
if !checker.semantic().current_scope().kind.is_module() {
checker
.diagnostics
.push(Diagnostic::new(ImportOutsideTopLevel, stmt.range()));
if checker.semantic().current_scope().kind.is_module() {
// "Top-level" imports are allowed
return;
}
// Check if any of the non-top-level imports are banned by TID253
// before emitting the diagnostic to avoid conflicts.
if checker.enabled(Rule::BannedModuleLevelImports) {
let mut all_aliases_banned = true;
let mut has_alias = false;
for (policy, node) in &BannedModuleImportPolicies::new(stmt, checker) {
if node.is_alias() {
has_alias = true;
all_aliases_banned &= is_banned_module_level_import(&policy, checker);
}
// If the entire import is banned
else if is_banned_module_level_import(&policy, checker) {
return;
}
}
if has_alias && all_aliases_banned {
return;
}
}
// Emit the diagnostic
checker
.diagnostics
.push(Diagnostic::new(ImportOutsideTopLevel, stmt.range()));
}
fn is_banned_module_level_import(policy: &NameMatchPolicy, checker: &Checker) -> bool {
policy
.find(
checker
.settings
.flake8_tidy_imports
.banned_module_level_imports(),
)
.is_some()
}

View File

@@ -0,0 +1,94 @@
---
source: crates/ruff_linter/src/rules/pylint/mod.rs
---
import_outside_top_level_with_banned.py:9:5: PLC0415 `import` should be at the top-level of a file
|
8 | def import_in_function():
9 | import symtable # [import-outside-toplevel]
| ^^^^^^^^^^^^^^^ PLC0415
10 | import os, sys # [import-outside-toplevel]
11 | import time as thyme # [import-outside-toplevel]
|
import_outside_top_level_with_banned.py:10:5: PLC0415 `import` should be at the top-level of a file
|
8 | def import_in_function():
9 | import symtable # [import-outside-toplevel]
10 | import os, sys # [import-outside-toplevel]
| ^^^^^^^^^^^^^^ PLC0415
11 | import time as thyme # [import-outside-toplevel]
12 | import random as rand, socket as sock # [import-outside-toplevel]
|
import_outside_top_level_with_banned.py:11:5: PLC0415 `import` should be at the top-level of a file
|
9 | import symtable # [import-outside-toplevel]
10 | import os, sys # [import-outside-toplevel]
11 | import time as thyme # [import-outside-toplevel]
| ^^^^^^^^^^^^^^^^^^^^ PLC0415
12 | import random as rand, socket as sock # [import-outside-toplevel]
13 | from collections import defaultdict # [import-outside-toplevel]
|
import_outside_top_level_with_banned.py:12:5: PLC0415 `import` should be at the top-level of a file
|
10 | import os, sys # [import-outside-toplevel]
11 | import time as thyme # [import-outside-toplevel]
12 | import random as rand, socket as sock # [import-outside-toplevel]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ PLC0415
13 | from collections import defaultdict # [import-outside-toplevel]
14 | from math import sin as sign, cos as cosplay # [import-outside-toplevel]
|
import_outside_top_level_with_banned.py:13:5: PLC0415 `import` should be at the top-level of a file
|
11 | import time as thyme # [import-outside-toplevel]
12 | import random as rand, socket as sock # [import-outside-toplevel]
13 | from collections import defaultdict # [import-outside-toplevel]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ PLC0415
14 | from math import sin as sign, cos as cosplay # [import-outside-toplevel]
|
import_outside_top_level_with_banned.py:14:5: PLC0415 `import` should be at the top-level of a file
|
12 | import random as rand, socket as sock # [import-outside-toplevel]
13 | from collections import defaultdict # [import-outside-toplevel]
14 | from math import sin as sign, cos as cosplay # [import-outside-toplevel]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ PLC0415
15 |
16 | # these should be allowed due to TID253 top-level ban
|
import_outside_top_level_with_banned.py:24:5: PLC0415 `import` should be at the top-level of a file
|
23 | # this should still trigger an error due to multiple imports
24 | from pkg import foo_allowed, bar_banned # [import-outside-toplevel]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ PLC0415
25 |
26 | class ClassWithImports:
|
import_outside_top_level_with_banned.py:27:5: PLC0415 `import` should be at the top-level of a file
|
26 | class ClassWithImports:
27 | import tokenize # [import-outside-toplevel]
| ^^^^^^^^^^^^^^^ PLC0415
28 |
29 | def __init__(self):
|
import_outside_top_level_with_banned.py:30:9: PLC0415 `import` should be at the top-level of a file
|
29 | def __init__(self):
30 | import trace # [import-outside-toplevel]
| ^^^^^^^^^^^^ PLC0415
31 |
32 | # these should be allowed due to TID253 top-level ban
|
import_outside_top_level_with_banned.py:40:9: PLC0415 `import` should be at the top-level of a file
|
39 | # this should still trigger an error due to multiple imports
40 | from pkg import foo_allowed, bar_banned # [import-outside-toplevel]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ PLC0415
|

View File

@@ -6,6 +6,7 @@ use ruff_python_ast::{Arguments, Expr, ExprCall, ExprNumberLiteral, Number};
use ruff_python_semantic::analyze::type_inference::{NumberLike, PythonType, ResolvedPythonType};
use ruff_python_semantic::analyze::typing;
use ruff_python_semantic::SemanticModel;
use ruff_source_file::find_newline;
use ruff_text_size::Ranged;
/// ## What it does
@@ -59,7 +60,7 @@ pub(crate) fn unnecessary_round(checker: &mut Checker, call: &ExprCall) {
return;
}
let applicability = match rounded_value {
let mut applicability = match rounded_value {
// ```python
// some_int: int
//
@@ -86,6 +87,10 @@ pub(crate) fn unnecessary_round(checker: &mut Checker, call: &ExprCall) {
_ => return,
};
if checker.comment_ranges().intersects(call.range()) {
applicability = Applicability::Unsafe;
};
let edit = unwrap_round_call(call, rounded, checker.semantic(), checker.locator());
let fix = Fix::applicable_edit(edit, applicability);
@@ -196,13 +201,13 @@ fn unwrap_round_call(
locator: &Locator,
) -> Edit {
let rounded_expr = locator.slice(rounded.range());
let has_parent_expr = semantic.current_expression_parent().is_some();
let new_content = if has_parent_expr || rounded.is_named_expr() {
format!("({rounded_expr})")
} else {
rounded_expr.to_string()
};
let new_content =
if has_parent_expr || rounded.is_named_expr() || find_newline(rounded_expr).is_some() {
format!("({rounded_expr})")
} else {
rounded_expr.to_string()
};
Edit::range_replacement(new_content, call.range)
}

View File

@@ -1,6 +1,5 @@
---
source: crates/ruff_linter/src/rules/ruff/mod.rs
snapshot_kind: text
---
RUF057.py:6:1: RUF057 [*] Value being rounded is already an integer
|
@@ -181,3 +180,94 @@ RUF057.py:44:1: RUF057 [*] Value being rounded is already an integer
45 45 | round(inferred_int, -2) # No error
46 46 | round(inferred_int, inferred_int) # No error
47 47 | round(inferred_int, 3 + 4) # No error
RUF057.py:71:1: RUF057 [*] Value being rounded is already an integer
|
69 | # contains newline.
70 | # See https://github.com/astral-sh/ruff/issues/15598
71 | / round(-
72 | | 1)
| |__^ RUF057
73 | round(1
74 | *1
|
= help: Remove unnecessary `round` call
Safe fix
68 68 | # Fixes should preserve parentheses when argument
69 69 | # contains newline.
70 70 | # See https://github.com/astral-sh/ruff/issues/15598
71 |-round(-
71 |+(-
72 72 | 1)
73 73 | round(1
74 74 | *1
RUF057.py:73:1: RUF057 [*] Value being rounded is already an integer
|
71 | round(-
72 | 1)
73 | / round(1
74 | | *1
75 | | )
| |_^ RUF057
76 |
77 | # fix should be unsafe if comment is in call range
|
= help: Remove unnecessary `round` call
Safe fix
70 70 | # See https://github.com/astral-sh/ruff/issues/15598
71 71 | round(-
72 72 | 1)
73 |-round(1
74 |-*1
75 |-)
73 |+(1
74 |+*1)
76 75 |
77 76 | # fix should be unsafe if comment is in call range
78 77 | round(# a comment
RUF057.py:78:1: RUF057 [*] Value being rounded is already an integer
|
77 | # fix should be unsafe if comment is in call range
78 | / round(# a comment
79 | | 17
80 | | )
| |_^ RUF057
81 | round(
82 | 17 # a comment
|
= help: Remove unnecessary `round` call
Unsafe fix
75 75 | )
76 76 |
77 77 | # fix should be unsafe if comment is in call range
78 |-round(# a comment
79 78 | 17
80 |-)
81 79 | round(
82 80 | 17 # a comment
83 81 | )
RUF057.py:81:1: RUF057 [*] Value being rounded is already an integer
|
79 | 17
80 | )
81 | / round(
82 | | 17 # a comment
83 | | )
| |_^ RUF057
|
= help: Remove unnecessary `round` call
Unsafe fix
78 78 | round(# a comment
79 79 | 17
80 80 | )
81 |-round(
82 |- 17 # a comment
83 |-)
81 |+17

View File

@@ -7,14 +7,15 @@ your project. For a more detailed overview, see [_Configuring Ruff_](configurati
To start, we'll initialize a project using [uv](https://docs.astral.sh/uv/):
```console
$ uv init numbers
```console/n
$ uv init --lib numbers
```
This command creates a Python project with the following structure:
```text
numbers
├── README.md
├── pyproject.toml
└── src
└── numbers

1
ruff.schema.json generated
View File

@@ -3653,7 +3653,6 @@
"PLW0",
"PLW01",
"PLW010",
"PLW0101",
"PLW0108",
"PLW012",
"PLW0120",