Compare commits

...

19 Commits

Author SHA1 Message Date
Dhruv Manilawala
dc2182aad0 Revert couple of renames 2025-01-27 20:46:32 +05:30
Wei Lee
25653cc839 refactor(AIR302): rename is_airflow_task as is_airflow_task_function_def and in_airflow_task as in_airflow_task_function_def 2025-01-24 23:12:13 +08:00
Wei Lee
4280d113f7 refactor(AIR302): rename removed_in_3 as airflow_3_removal_expr and removed_in_3_function_def as airflow_3_removal_function_def 2025-01-24 23:12:13 +08:00
Wei Lee
d11f865144 Revert "refactor(AIR302): rename check_function_parameters as check_parameters_in_function_def"
This reverts commit f742313316286f0dbc87a7efa3265c25dd86af17.
2025-01-24 23:12:13 +08:00
Wei Lee
b27ec6537b refactor(AIR302): remove unnecessary if else
Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
2025-01-24 23:12:13 +08:00
Wei Lee
d64a952660 refactor(AIR302): rename is_airflow_task as is_decorated_by_airflow_task 2025-01-24 23:12:13 +08:00
Wei Lee
67f65cff4d refactor(AIR302): rename check_function_parameters as check_parameters_in_function_def 2025-01-24 23:12:13 +08:00
Wei Lee
c779cbfa3f refactor(AIR302): rename removed_in_3 as removed_expr_in_3 and removed_in_3_function_def as removed_funciton_def_in_3 2025-01-24 23:12:13 +08:00
Wei Lee
fbfb23ba14 feat(AIR302): add is_execute_method_inherits_from_airflow_operator for checking removed context key in execute method 2025-01-24 23:12:13 +08:00
Wei Lee
fc2dd86d00 test(AIR302): reorganize context handling 2025-01-24 23:12:13 +08:00
Charlie Marsh
ab2e1905c4 Use uv init --lib in tutorial (#15718)
## Summary

Closes https://github.com/astral-sh/uv/issues/10933.
2025-01-24 14:53:20 +00:00
David Peter
1feb3cf41a [red-knot] Use Unknown | T_inferred for undeclared public symbols (#15674)
## Summary

Use `Unknown | T_inferred` as the type for *undeclared* public symbols.

## Test Plan

- Updated existing tests
- New test for external `__slots__` modifications.
- New tests for external modifications of public symbols.
2025-01-24 12:47:48 +01:00
Dylan
7778d1d646 [ruff] Parenthesize fix when argument spans multiple lines for unnecessary-round (RUF057) (#15703) 2025-01-24 04:34:56 -06:00
David Peter
fb58a9b610 [red-knot] Rename TestDbBuilder::typeshed to .custom_typeshed (#15712)
## Summary

Correcting a small oversight by me
(https://github.com/astral-sh/ruff/pull/15683#discussion_r1926830914).
2025-01-24 10:25:23 +00:00
Mike Perlov
17a8a55f08 Honor banned top level imports by TID253 in PLC0415. (#15628)
Co-authored-by: Micha Reiser <micha@reiser.io>
2025-01-24 11:07:21 +01:00
Dhruv Manilawala
99d8ec6769 Apply AIR302-context check only in @task function (#15711)
This PR updates `AIR302` to only apply the context keys check in `@task`
decorated function.

Ref: https://github.com/astral-sh/ruff/pull/15144
2025-01-24 07:30:35 +00:00
Ankit Chaurasia
34cc3cab98 [airflow] Update AIR302 to check for deprecated context keys (#15144)
**Summary**

Airflow 3.0 removes a set of deprecated context variables that were
phased out in 2.x. This PR introduces lint rules to detect usage of
these removed variables in various patterns, helping identify
incompatibilities. The removed context variables include:

```
conf
execution_date
next_ds
next_ds_nodash
next_execution_date
prev_ds
prev_ds_nodash
prev_execution_date
prev_execution_date_success
tomorrow_ds
yesterday_ds
yesterday_ds_nodash
```

**Detected Patterns and Examples**

The linter now flags the use of removed context variables in the
following scenarios:

1. **Direct Subscript Access**  
   ```python
   execution_date = context["execution_date"]  # Flagged
   ```
   
2. **`.get("key")` Method Calls**  
   ```python
   print(context.get("execution_date"))  # Flagged
   ```
   
3. **Variables Assigned from `get_current_context()`**  
If a variable is assigned from `get_current_context()` and then used to
access a removed key:
   ```python
   c = get_current_context()
   print(c.get("execution_date"))  # Flagged
   ```
   
4. **Function Parameters in `@task`-Decorated Functions**  
Parameters named after removed context variables in functions decorated
with `@task` are flagged:
   ```python
   from airflow.decorators import task
   
   @task
def my_task(execution_date, **kwargs): # Parameter 'execution_date'
flagged
       pass
   ```
   
5. **Removed Keys in Task Decorator `kwargs` and Other Scenarios**  
Other similar patterns where removed context variables appear (e.g., as
part of `kwargs` in a `@task` function) are also detected.
```
from airflow.decorators import task

@task
def process_with_execution_date(**context):
    execution_date = lambda: context["execution_date"]  # flagged
    print(execution_date)

@task(kwargs={"execution_date": "2021-01-01"})   # flagged
def task_with_kwargs(**context):  
    pass
```

**Test Plan**

Test fixtures covering various patterns of deprecated context usage are
included in this PR. For example:

```python
from airflow.decorators import task, dag, get_current_context
from airflow.models import DAG
from airflow.operators.dummy import DummyOperator
import pendulum
from datetime import datetime

@task
def access_invalid_key_task(**context):
    print(context.get("conf"))  # 'conf' flagged

@task
def print_config(**context):
    execution_date = context["execution_date"]  # Flagged
    prev_ds = context["prev_ds"]                # Flagged

@task
def from_current_context():
    context = get_current_context()
    print(context["execution_date"])            # Flagged

# Usage outside of a task decorated function
c = get_current_context()
print(c.get("execution_date"))                 # Flagged

@task
def some_task(execution_date, **kwargs):
    print("execution date", execution_date)     # Parameter flagged

@dag(
    start_date=pendulum.datetime(2021, 1, 1, tz="UTC")
)
def my_dag():
    task1 = DummyOperator(
        task_id="task1",
        params={
            "execution_date": "{{ execution_date }}",  # Flagged in template context
        },
    )

    access_invalid_key_task()
    print_config()
    from_current_context()
    
dag = my_dag()

class CustomOperator(BaseOperator):
    def execute(self, context):
        execution_date = context.get("execution_date")                      # Flagged
        next_ds = context.get("next_ds")                                               # Flagged
        next_execution_date = context["next_execution_date"]          # Flagged
```

Ruff will emit `AIR302` diagnostics for each deprecated usage, with
suggestions when applicable, aiding in code migration to Airflow 3.0.

related: https://github.com/apache/airflow/issues/44409,
https://github.com/apache/airflow/issues/41641

---------

Co-authored-by: Wei Lee <weilee.rx@gmail.com>
2025-01-24 11:25:05 +05:30
Dhruv Manilawala
9384ba4b91 Remove test rules from JSON schema (#15627)
Closes: #15707
2025-01-24 10:17:59 +05:30
Dylan
2b3550c85f Add two missing commits to changelog (#15701)
Some commits appeared between the creation and merging of the "bump to
v0.9.3" branch.

Also made the same changes to the Releases on githhub.
2025-01-23 15:32:00 -06:00
44 changed files with 1454 additions and 196 deletions

View File

@@ -21,6 +21,7 @@
- \[`flake8-bugbear`\] Do not raise error if keyword argument is present and target-python version is less or equals than 3.9 (`B903`) ([#15549](https://github.com/astral-sh/ruff/pull/15549))
- \[`flake8-comprehensions`\] strip parentheses around generators in `unnecessary-generator-set` (`C401`) ([#15553](https://github.com/astral-sh/ruff/pull/15553))
- \[`flake8-pytest-style`\] Rewrite references to `.exception` (`PT027`) ([#15680](https://github.com/astral-sh/ruff/pull/15680))
- \[`flake8-simplify`\] Mark fixes as unsafe (`SIM201`, `SIM202`) ([#15626](https://github.com/astral-sh/ruff/pull/15626))
- \[`flake8-type-checking`\] Fix some safe fixes being labeled unsafe (`TC006`,`TC008`) ([#15638](https://github.com/astral-sh/ruff/pull/15638))
- \[`isort`\] Omit trailing whitespace in `unsorted-imports` (`I001`) ([#15518](https://github.com/astral-sh/ruff/pull/15518))
@@ -47,11 +48,12 @@
### Bug fixes
- \[`flake8-bandit`\] Add missing single-line/dotall regex flag (`S608`) ([#15654](https://github.com/astral-sh/ruff/pull/15654))
- \[`flake8-import-conventions`\] Fix infinite loop between `ICN001` and `I002` (`ICN001`) ([#15480](https://github.com/astral-sh/ruff/pull/15480))
- \[`flake8-simplify`\] Do not emit diagnostics for expressions inside string type annotations (`SIM222`, `SIM223`) ([#15405](https://github.com/astral-sh/ruff/pull/15405))
- \[`pyflakes`\] Treat arguments passed to the `default=` parameter of `TypeVar` as type expressions (`F821`) ([#15679](https://github.com/astral-sh/ruff/pull/15679))
- \[`pyupgrade`\] Avoid syntax error when the iterable is a non-parenthesized tuple (`UP028`) ([#15543](https://github.com/astral-sh/ruff/pull/15543))
- \[`ruff`\] Exempt `NewType` calls where the original type is immutable (`RUF009`) ([#15588](https://github.com/astral-sh/ruff/pull/15588))
- \[`unconventional-import-alias`\] Fix infinite loop between `ICN001` and `I002` (`ICN001`) ([#15480](https://github.com/astral-sh/ruff/pull/15480))
- Preserve raw string prefix and escapes in all codegen fixes ([#15694](https://github.com/astral-sh/ruff/pull/15694))
### Documentation

View File

@@ -36,7 +36,7 @@ def f():
reveal_type(a7) # revealed: None
reveal_type(a8) # revealed: Literal[1]
# TODO: This should be Color.RED
reveal_type(b1) # revealed: Literal[0]
reveal_type(b1) # revealed: Unknown | Literal[0]
# error: [invalid-type-form]
invalid1: Literal[3 + 4]

View File

@@ -175,7 +175,7 @@ class C:
reveal_type(C.pure_class_variable1) # revealed: str
# TODO: this should be `Literal[1]`, or `Unknown | Literal[1]`.
# TODO: Should be `Unknown | Literal[1]`.
reveal_type(C.pure_class_variable2) # revealed: Unknown
c_instance = C()
@@ -252,8 +252,7 @@ class C:
reveal_type(C.variable_with_class_default1) # revealed: str
# TODO: this should be `Unknown | Literal[1]`.
reveal_type(C.variable_with_class_default2) # revealed: Literal[1]
reveal_type(C.variable_with_class_default2) # revealed: Unknown | Literal[1]
c_instance = C()
@@ -296,8 +295,8 @@ def _(flag: bool):
else:
x = 4
reveal_type(C1.x) # revealed: Literal[1, 2]
reveal_type(C2.x) # revealed: Literal[3, 4]
reveal_type(C1.x) # revealed: Unknown | Literal[1, 2]
reveal_type(C2.x) # revealed: Unknown | Literal[3, 4]
```
## Inherited class attributes
@@ -311,7 +310,7 @@ class A:
class B(A): ...
class C(B): ...
reveal_type(C.X) # revealed: Literal["foo"]
reveal_type(C.X) # revealed: Unknown | Literal["foo"]
```
### Multiple inheritance
@@ -334,7 +333,7 @@ class A(B, C): ...
reveal_type(A.__mro__)
# `E` is earlier in the MRO than `F`, so we should use the type of `E.X`
reveal_type(A.X) # revealed: Literal[42]
reveal_type(A.X) # revealed: Unknown | Literal[42]
```
## Unions with possibly unbound paths
@@ -356,7 +355,7 @@ def _(flag1: bool, flag2: bool):
C = C1 if flag1 else C2 if flag2 else C3
# error: [possibly-unbound-attribute] "Attribute `x` on type `Literal[C1, C2, C3]` is possibly unbound"
reveal_type(C.x) # revealed: Literal[1, 3]
reveal_type(C.x) # revealed: Unknown | Literal[1, 3]
```
### Possibly-unbound within a class
@@ -379,7 +378,7 @@ def _(flag: bool, flag1: bool, flag2: bool):
C = C1 if flag1 else C2 if flag2 else C3
# error: [possibly-unbound-attribute] "Attribute `x` on type `Literal[C1, C2, C3]` is possibly unbound"
reveal_type(C.x) # revealed: Literal[1, 2, 3]
reveal_type(C.x) # revealed: Unknown | Literal[1, 2, 3]
```
### Unions with all paths unbound

View File

@@ -262,7 +262,8 @@ class A:
class B:
__add__ = A()
reveal_type(B() + B()) # revealed: int
# TODO: this could be `int` if we declare `B.__add__` using a `Callable` type
reveal_type(B() + B()) # revealed: Unknown | int
```
## Integration test: numbers from typeshed

View File

@@ -5,6 +5,11 @@ that is, a use of a symbol from another scope. If a symbol has a declared type i
(e.g. `int`), we use that as the symbol's "public type" (the type of the symbol from the perspective
of other scopes) even if there is a more precise local inferred type for the symbol (`Literal[1]`).
If a symbol has no declared type, we use the union of `Unknown` with the inferred type as the public
type. If there is no declaration, then the symbol can be reassigned to any type from another scope;
the union with `Unknown` reflects that its type must at least be as large as the type of the
assigned value, but could be arbitrarily larger.
We test the whole matrix of possible boundness and declaredness states. The current behavior is
summarized in the following table, while the tests below demonstrate each case. Note that some of
this behavior is questionable and might change in the future. See the TODOs in `symbol_by_id`
@@ -12,11 +17,11 @@ this behavior is questionable and might change in the future. See the TODOs in `
In particular, we should raise errors in the "possibly-undeclared-and-unbound" as well as the
"undeclared-and-possibly-unbound" cases (marked with a "?").
| **Public type** | declared | possibly-undeclared | undeclared |
| ---------------- | ------------ | -------------------------- | ------------ |
| bound | `T_declared` | `T_declared \| T_inferred` | `T_inferred` |
| possibly-unbound | `T_declared` | `T_declared \| T_inferred` | `T_inferred` |
| unbound | `T_declared` | `T_declared` | `Unknown` |
| **Public type** | declared | possibly-undeclared | undeclared |
| ---------------- | ------------ | -------------------------- | ----------------------- |
| bound | `T_declared` | `T_declared \| T_inferred` | `Unknown \| T_inferred` |
| possibly-unbound | `T_declared` | `T_declared \| T_inferred` | `Unknown \| T_inferred` |
| unbound | `T_declared` | `T_declared` | `Unknown` |
| **Diagnostic** | declared | possibly-undeclared | undeclared |
| ---------------- | -------- | ------------------------- | ------------------- |
@@ -97,17 +102,24 @@ def flag() -> bool: ...
x = 1
y = 2
z = 3
if flag():
x: Any
x: int
y: Any
# error: [invalid-declaration]
y: str
z: str
```
```py
from mod import x, y
from mod import x, y, z
reveal_type(x) # revealed: Literal[1] | Any
reveal_type(y) # revealed: Literal[2] | Unknown
reveal_type(x) # revealed: int
reveal_type(y) # revealed: Literal[2] | Any
reveal_type(z) # revealed: Literal[3] | Unknown
# External modifications of `x` that violate the declared type are not allowed:
# error: [invalid-assignment]
x = None
```
### Possibly undeclared and possibly unbound
@@ -134,6 +146,10 @@ from mod import x, y
reveal_type(x) # revealed: Literal[1] | Any
reveal_type(y) # revealed: Literal[2] | str
# External modifications of `y` that violate the declared type are not allowed:
# error: [invalid-assignment]
y = None
```
### Possibly undeclared and unbound
@@ -154,14 +170,16 @@ if flag():
from mod import x
reveal_type(x) # revealed: int
# External modifications to `x` that violate the declared type are not allowed:
# error: [invalid-assignment]
x = None
```
## Undeclared
### Undeclared but bound
We use the inferred type as the public type, if a symbol has no declared type.
```py path=mod.py
x = 1
```
@@ -169,7 +187,10 @@ x = 1
```py
from mod import x
reveal_type(x) # revealed: Literal[1]
reveal_type(x) # revealed: Unknown | Literal[1]
# All external modifications of `x` are allowed:
x = None
```
### Undeclared and possibly unbound
@@ -189,7 +210,10 @@ if flag:
# on top of this document.
from mod import x
reveal_type(x) # revealed: Literal[1]
reveal_type(x) # revealed: Unknown | Literal[1]
# All external modifications of `x` are allowed:
x = None
```
### Undeclared and unbound
@@ -206,4 +230,7 @@ if False:
from mod import x
reveal_type(x) # revealed: Unknown
# Modifications allowed in this case:
x = None
```

View File

@@ -52,7 +52,7 @@ class NonCallable:
__call__ = 1
a = NonCallable()
# error: "Object of type `NonCallable` is not callable"
# error: "Object of type `Unknown | Literal[1]` is not callable (due to union element `Literal[1]`)"
reveal_type(a()) # revealed: Unknown
```

View File

@@ -43,7 +43,8 @@ class IntIterable:
def __iter__(self) -> IntIterator:
return IntIterator()
# revealed: tuple[int, int]
# TODO: This could be a `tuple[int, int]` if we model that `y` can not be modified in the outer comprehension scope
# revealed: tuple[int, Unknown | int]
[[reveal_type((x, y)) for x in IntIterable()] for y in IntIterable()]
```
@@ -66,7 +67,8 @@ class IterableOfIterables:
def __iter__(self) -> IteratorOfIterables:
return IteratorOfIterables()
# revealed: tuple[int, IntIterable]
# TODO: This could be a `tuple[int, int]` (see above)
# revealed: tuple[int, Unknown | IntIterable]
[[reveal_type((x, y)) for x in y] for y in IterableOfIterables()]
```

View File

@@ -5,7 +5,7 @@
```py
def _(flag: bool):
class A:
always_bound = 1
always_bound: int = 1
if flag:
union = 1
@@ -13,14 +13,21 @@ def _(flag: bool):
union = "abc"
if flag:
possibly_unbound = "abc"
union_declared: int = 1
else:
union_declared: str = "abc"
reveal_type(A.always_bound) # revealed: Literal[1]
if flag:
possibly_unbound: str = "abc"
reveal_type(A.union) # revealed: Literal[1, "abc"]
reveal_type(A.always_bound) # revealed: int
reveal_type(A.union) # revealed: Unknown | Literal[1, "abc"]
reveal_type(A.union_declared) # revealed: int | str
# error: [possibly-unbound-attribute] "Attribute `possibly_unbound` on type `Literal[A]` is possibly unbound"
reveal_type(A.possibly_unbound) # revealed: Literal["abc"]
reveal_type(A.possibly_unbound) # revealed: str
# error: [unresolved-attribute] "Type `Literal[A]` has no attribute `non_existent`"
reveal_type(A.non_existent) # revealed: Unknown

View File

@@ -55,7 +55,7 @@ reveal_type("x" or "y" and "") # revealed: Literal["x"]
## Evaluates to builtin
```py path=a.py
redefined_builtin_bool = bool
redefined_builtin_bool: type[bool] = bool
def my_bool(x) -> bool:
return True

View File

@@ -172,10 +172,10 @@ class IntUnion:
def __len__(self) -> Literal[SomeEnum.INT, SomeEnum.INT_2]: ...
reveal_type(len(Auto())) # revealed: int
reveal_type(len(Int())) # revealed: Literal[2]
reveal_type(len(Int())) # revealed: int
reveal_type(len(Str())) # revealed: int
reveal_type(len(Tuple())) # revealed: int
reveal_type(len(IntUnion())) # revealed: Literal[2, 32]
reveal_type(len(IntUnion())) # revealed: int
```
### Negative integers

View File

@@ -20,7 +20,7 @@ wrong_innards: MyBox[int] = MyBox("five")
# TODO reveal int, do not leak the typevar
reveal_type(box.data) # revealed: T
reveal_type(MyBox.box_model_number) # revealed: Literal[695]
reveal_type(MyBox.box_model_number) # revealed: Unknown | Literal[695]
```
## Subclassing

View File

@@ -23,8 +23,8 @@ reveal_type(y)
# error: [possibly-unbound-import] "Member `y` of module `maybe_unbound` is possibly unbound"
from maybe_unbound import x, y
reveal_type(x) # revealed: Literal[3]
reveal_type(y) # revealed: Literal[3]
reveal_type(x) # revealed: Unknown | Literal[3]
reveal_type(y) # revealed: Unknown | Literal[3]
```
## Maybe unbound annotated
@@ -52,7 +52,7 @@ Importing an annotated name prefers the declared type over the inferred type:
# error: [possibly-unbound-import] "Member `y` of module `maybe_unbound_annotated` is possibly unbound"
from maybe_unbound_annotated import x, y
reveal_type(x) # revealed: Literal[3]
reveal_type(x) # revealed: Unknown | Literal[3]
reveal_type(y) # revealed: int
```

View File

@@ -109,9 +109,9 @@ reveal_type(x)
def _(flag: bool):
class NotIterable:
if flag:
__iter__ = 1
__iter__: int = 1
else:
__iter__ = None
__iter__: None = None
for x in NotIterable(): # error: "Object of type `NotIterable` is not iterable"
pass
@@ -135,7 +135,7 @@ for x in nonsense: # error: "Object of type `Literal[123]` is not iterable"
class NotIterable:
def __getitem__(self, key: int) -> int:
return 42
__iter__ = None
__iter__: None = None
for x in NotIterable(): # error: "Object of type `NotIterable` is not iterable"
pass

View File

@@ -99,9 +99,9 @@ def _(x: str | int):
class A: ...
class B: ...
alias_for_type = type
def _(x: A | B):
alias_for_type = type
if alias_for_type(x) is A:
reveal_type(x) # revealed: A
```

View File

@@ -6,7 +6,7 @@
def f():
x = 1
def g():
reveal_type(x) # revealed: Literal[1]
reveal_type(x) # revealed: Unknown | Literal[1]
```
## Two levels up
@@ -16,7 +16,7 @@ def f():
x = 1
def g():
def h():
reveal_type(x) # revealed: Literal[1]
reveal_type(x) # revealed: Unknown | Literal[1]
```
## Skips class scope
@@ -28,7 +28,7 @@ def f():
class C:
x = 2
def g():
reveal_type(x) # revealed: Literal[1]
reveal_type(x) # revealed: Unknown | Literal[1]
```
## Skips annotation-only assignment
@@ -41,7 +41,7 @@ def f():
# name is otherwise not defined; maybe should be an error?
x: int
def h():
reveal_type(x) # revealed: Literal[1]
reveal_type(x) # revealed: Unknown | Literal[1]
```
## Implicit global in function
@@ -52,5 +52,5 @@ A name reference to a never-defined symbol in a function is implicitly a global
x = 1
def f():
reveal_type(x) # revealed: Literal[1]
reveal_type(x) # revealed: Unknown | Literal[1]
```

View File

@@ -17,8 +17,8 @@ class C:
x = 2
# error: [possibly-unbound-attribute] "Attribute `x` on type `Literal[C]` is possibly unbound"
reveal_type(C.x) # revealed: Literal[2]
reveal_type(C.y) # revealed: Literal[1]
reveal_type(C.x) # revealed: Unknown | Literal[2]
reveal_type(C.y) # revealed: Unknown | Literal[1]
```
## Possibly unbound in class and global scope
@@ -37,7 +37,7 @@ class C:
# error: [possibly-unresolved-reference]
y = x
reveal_type(C.y) # revealed: Literal[1, "abc"]
reveal_type(C.y) # revealed: Unknown | Literal[1, "abc"]
```
## Unbound function local

View File

@@ -182,3 +182,34 @@ class C(A, B): ...
# False negative: [incompatible-slots]
class A(int, str): ...
```
### Diagnostic if `__slots__` is externally modified
We special-case type inference for `__slots__` and return the pure inferred type, even if the symbol
is not declared — a case in which we union with `Unknown` for other public symbols. The reason for
this is that `__slots__` has a special handling in the runtime. Modifying it externally is actually
allowed, but those changes do not take effect. If you have a class `C` with `__slots__ = ("foo",)`
and externally set `C.__slots__ = ("bar",)`, you still can't access `C.bar`. And you can still
access `C.foo`. We therefore issue a diagnostic for such assignments:
```py
class A:
__slots__ = ("a",)
# Modifying `__slots__` from within the class body is fine:
__slots__ = ("a", "b")
# No `Unknown` here:
reveal_type(A.__slots__) # revealed: tuple[Literal["a"], Literal["b"]]
# But modifying it externally is not:
# error: [invalid-assignment]
A.__slots__ = ("a",)
# error: [invalid-assignment]
A.__slots__ = ("a", "b_new")
# error: [invalid-assignment]
A.__slots__ = ("a", "b", "c")
```

View File

@@ -14,7 +14,8 @@ a = NotSubscriptable()[0] # error: "Cannot subscript object of type `NotSubscri
class NotSubscriptable:
__getitem__ = None
a = NotSubscriptable()[0] # error: "Method `__getitem__` of type `None` is not callable on object of type `NotSubscriptable`"
# error: "Method `__getitem__` of type `Unknown | None` is not callable on object of type `NotSubscriptable`"
a = NotSubscriptable()[0]
```
## Valid getitem

View File

@@ -139,7 +139,9 @@ reveal_type(not AlwaysFalse())
# We don't get into a cycle if someone sets their `__bool__` method to the `bool` builtin:
class BoolIsBool:
__bool__ = bool
# TODO: The `type[bool]` declaration here is a workaround to avoid running into
# https://github.com/astral-sh/ruff/issues/15672
__bool__: type[bool] = bool
# revealed: bool
reveal_type(not BoolIsBool())

View File

@@ -76,11 +76,11 @@ with Manager():
```py
class Manager:
__enter__ = 42
__enter__: int = 42
def __exit__(self, exc_tpe, exc_value, traceback): ...
# error: [invalid-context-manager] "Object of type `Manager` cannot be used with `with` because the method `__enter__` of type `Literal[42]` is not callable"
# error: [invalid-context-manager] "Object of type `Manager` cannot be used with `with` because the method `__enter__` of type `int` is not callable"
with Manager():
...
```
@@ -91,9 +91,9 @@ with Manager():
class Manager:
def __enter__(self) -> Self: ...
__exit__ = 32
__exit__: int = 32
# error: [invalid-context-manager] "Object of type `Manager` cannot be used with `with` because the method `__exit__` of type `Literal[32]` is not callable"
# error: [invalid-context-manager] "Object of type `Manager` cannot be used with `with` because the method `__exit__` of type `int` is not callable"
with Manager():
...
```

View File

@@ -136,7 +136,7 @@ pub(crate) mod tests {
/// Target Python platform
python_platform: PythonPlatform,
/// Path to a custom typeshed directory
typeshed: Option<SystemPathBuf>,
custom_typeshed: Option<SystemPathBuf>,
/// Path and content pairs for files that should be present
files: Vec<(&'a str, &'a str)>,
}
@@ -146,7 +146,7 @@ pub(crate) mod tests {
Self {
python_version: PythonVersion::default(),
python_platform: PythonPlatform::default(),
typeshed: None,
custom_typeshed: None,
files: vec![],
}
}
@@ -171,7 +171,7 @@ pub(crate) mod tests {
.context("Failed to write test files")?;
let mut search_paths = SearchPathSettings::new(vec![src_root]);
search_paths.custom_typeshed = self.typeshed;
search_paths.custom_typeshed = self.custom_typeshed;
Program::from_settings(
&db,

View File

@@ -85,6 +85,14 @@ impl<'db> Symbol<'db> {
Symbol::Unbound => self,
}
}
#[must_use]
pub(crate) fn map_type(self, f: impl FnOnce(Type<'db>) -> Type<'db>) -> Symbol<'db> {
match self {
Symbol::Type(ty, boundness) => Symbol::Type(f(ty), boundness),
Symbol::Unbound => Symbol::Unbound,
}
}
}
#[cfg(test)]

View File

@@ -80,29 +80,54 @@ pub fn check_types(db: &dyn Db, file: File) -> TypeCheckDiagnostics {
diagnostics
}
/// Computes a possibly-widened type `Unknown | T_inferred` from the inferred type `T_inferred`
/// of a symbol, unless the type is a known-instance type (e.g. `typing.Any`) or the symbol is
/// considered non-modifiable (e.g. when the symbol is `@Final`). We need this for public uses
/// of symbols that have no declared type.
fn widen_type_for_undeclared_public_symbol<'db>(
db: &'db dyn Db,
inferred: Symbol<'db>,
is_considered_non_modifiable: bool,
) -> Symbol<'db> {
// We special-case known-instance types here since symbols like `typing.Any` are typically
// not declared in the stubs (e.g. `Any = object()`), but we still want to treat them as
// such.
let is_known_instance = inferred
.ignore_possibly_unbound()
.is_some_and(|ty| matches!(ty, Type::KnownInstance(_)));
if is_considered_non_modifiable || is_known_instance {
inferred
} else {
inferred.map_type(|ty| UnionType::from_elements(db, [Type::unknown(), ty]))
}
}
/// Infer the public type of a symbol (its type as seen from outside its scope).
fn symbol<'db>(db: &'db dyn Db, scope: ScopeId<'db>, name: &str) -> Symbol<'db> {
#[salsa::tracked]
fn symbol_by_id<'db>(
db: &'db dyn Db,
scope: ScopeId<'db>,
symbol: ScopedSymbolId,
is_dunder_slots: bool,
symbol_id: ScopedSymbolId,
) -> Symbol<'db> {
let use_def = use_def_map(db, scope);
// If the symbol is declared, the public type is based on declarations; otherwise, it's based
// on inference from bindings.
let declarations = use_def.public_declarations(symbol);
let declared =
symbol_from_declarations(db, declarations).map(|SymbolAndQualifiers(ty, _)| ty);
let declarations = use_def.public_declarations(symbol_id);
let declared = symbol_from_declarations(db, declarations);
let is_final = declared.as_ref().is_ok_and(SymbolAndQualifiers::is_final);
let declared = declared.map(|SymbolAndQualifiers(symbol, _)| symbol);
match declared {
// Symbol is declared, trust the declared type
Ok(symbol @ Symbol::Type(_, Boundness::Bound)) => symbol,
// Symbol is possibly declared
Ok(Symbol::Type(declared_ty, Boundness::PossiblyUnbound)) => {
let bindings = use_def.public_bindings(symbol);
let bindings = use_def.public_bindings(symbol_id);
let inferred = symbol_from_bindings(db, bindings);
match inferred {
@@ -120,12 +145,14 @@ fn symbol<'db>(db: &'db dyn Db, scope: ScopeId<'db>, name: &str) -> Symbol<'db>
),
}
}
// Symbol is undeclared, return the inferred type
// Symbol is undeclared, return the union of `Unknown` with the inferred type
Ok(Symbol::Unbound) => {
let bindings = use_def.public_bindings(symbol);
symbol_from_bindings(db, bindings)
let bindings = use_def.public_bindings(symbol_id);
let inferred = symbol_from_bindings(db, bindings);
widen_type_for_undeclared_public_symbol(db, inferred, is_dunder_slots || is_final)
}
// Symbol is possibly undeclared
// Symbol has conflicting declared types
Err((declared_ty, _)) => {
// Intentionally ignore conflicting declared types; that's not our problem,
// it's the problem of the module we are importing from.
@@ -177,9 +204,15 @@ fn symbol<'db>(db: &'db dyn Db, scope: ScopeId<'db>, name: &str) -> Symbol<'db>
}
let table = symbol_table(db, scope);
// `__slots__` is a symbol with special behavior in Python's runtime. It can be
// modified externally, but those changes do not take effect. We therefore issue
// a diagnostic if we see it being modified externally. In type inference, we
// can assign a "narrow" type to it even if it is not *declared*. This means, we
// do not have to call [`widen_type_for_undeclared_public_symbol`].
let is_dunder_slots = name == "__slots__";
table
.symbol_id_by_name(name)
.map(|symbol| symbol_by_id(db, scope, symbol))
.map(|symbol| symbol_by_id(db, scope, is_dunder_slots, symbol))
.unwrap_or(Symbol::Unbound)
}
@@ -378,6 +411,10 @@ impl SymbolAndQualifiers<'_> {
fn is_class_var(&self) -> bool {
self.1.contains(TypeQualifiers::CLASS_VAR)
}
fn is_final(&self) -> bool {
self.1.contains(TypeQualifiers::FINAL)
}
}
impl<'db> From<Symbol<'db>> for SymbolAndQualifiers<'db> {
@@ -4076,7 +4113,7 @@ impl<'db> Class<'db> {
/// this class, not on its superclasses.
fn own_instance_member(self, db: &'db dyn Db, name: &str) -> SymbolAndQualifiers<'db> {
// TODO: There are many things that are not yet implemented here:
// - `typing.ClassVar` and `typing.Final`
// - `typing.Final`
// - Proper diagnostics
// - Handling of possibly-undeclared/possibly-unbound attributes
// - The descriptor protocol
@@ -4084,10 +4121,10 @@ impl<'db> Class<'db> {
let body_scope = self.body_scope(db);
let table = symbol_table(db, body_scope);
if let Some(symbol) = table.symbol_id_by_name(name) {
if let Some(symbol_id) = table.symbol_id_by_name(name) {
let use_def = use_def_map(db, body_scope);
let declarations = use_def.public_declarations(symbol);
let declarations = use_def.public_declarations(symbol_id);
match symbol_from_declarations(db, declarations) {
Ok(SymbolAndQualifiers(Symbol::Type(declared_ty, _), qualifiers)) => {
@@ -4104,20 +4141,14 @@ impl<'db> Class<'db> {
SymbolAndQualifiers(Symbol::Type(declared_ty, Boundness::Bound), qualifiers)
}
}
Ok(SymbolAndQualifiers(Symbol::Unbound, qualifiers)) => {
let bindings = use_def.public_bindings(symbol);
Ok(symbol @ SymbolAndQualifiers(Symbol::Unbound, qualifiers)) => {
let bindings = use_def.public_bindings(symbol_id);
let inferred = symbol_from_bindings(db, bindings);
match inferred {
Symbol::Type(ty, _) => SymbolAndQualifiers(
Symbol::Type(
UnionType::from_elements(db, [Type::unknown(), ty]),
Boundness::Bound,
),
qualifiers,
),
Symbol::Unbound => SymbolAndQualifiers(Symbol::Unbound, qualifiers),
}
SymbolAndQualifiers(
widen_type_for_undeclared_public_symbol(db, inferred, symbol.is_final()),
qualifiers,
)
}
Err((declared_ty, _conflicting_declarations)) => {
// Ignore conflicting declarations
@@ -4694,7 +4725,10 @@ pub(crate) mod tests {
let bar = system_path_to_file(&db, "src/bar.py")?;
let a = global_symbol(&db, bar, "a");
assert_eq!(a.expect_type(), KnownClass::Int.to_instance(&db));
assert_eq!(
a.expect_type(),
UnionType::from_elements(&db, [Type::unknown(), KnownClass::Int.to_instance(&db)])
);
// Add a docstring to foo to trigger a re-run.
// The bar-call site of foo should not be re-run because of that
@@ -4710,7 +4744,10 @@ pub(crate) mod tests {
let a = global_symbol(&db, bar, "a");
assert_eq!(a.expect_type(), KnownClass::Int.to_instance(&db));
assert_eq!(
a.expect_type(),
UnionType::from_elements(&db, [Type::unknown(), KnownClass::Int.to_instance(&db)])
);
let events = db.take_salsa_events();
let call = &*parsed_module(&db, bar).syntax().body[1]

View File

@@ -543,7 +543,10 @@ mod tests {
assert_eq!(a_name, "a");
assert_eq!(b_name, "b");
// TODO resolution should not be deferred; we should see A not B
assert_eq!(a_annotated_ty.unwrap().display(&db).to_string(), "B");
assert_eq!(
a_annotated_ty.unwrap().display(&db).to_string(),
"Unknown | B"
);
assert_eq!(b_annotated_ty.unwrap().display(&db).to_string(), "T");
}
@@ -583,7 +586,10 @@ mod tests {
assert_eq!(a_name, "a");
assert_eq!(b_name, "b");
// Parameter resolution deferred; we should see B
assert_eq!(a_annotated_ty.unwrap().display(&db).to_string(), "B");
assert_eq!(
a_annotated_ty.unwrap().display(&db).to_string(),
"Unknown | B"
);
assert_eq!(b_annotated_ty.unwrap().display(&db).to_string(), "T");
}

View File

@@ -43,10 +43,10 @@ static EXPECTED_DIAGNOSTICS: &[&str] = &[
"warning[lint:possibly-unresolved-reference] /src/tomllib/_parser.py:579:12 Name `char` used when possibly not defined",
"warning[lint:possibly-unresolved-reference] /src/tomllib/_parser.py:580:63 Name `char` used when possibly not defined",
// We don't handle intersections in `is_assignable_to` yet
"error[lint:invalid-argument-type] /src/tomllib/_parser.py:626:46 Object of type `@Todo & ~AlwaysFalsy` cannot be assigned to parameter 1 (`match`) of function `match_to_datetime`; expected type `Match`",
"error[lint:invalid-argument-type] /src/tomllib/_parser.py:626:46 Object of type `Unknown & ~AlwaysFalsy | @Todo & ~AlwaysFalsy` cannot be assigned to parameter 1 (`match`) of function `match_to_datetime`; expected type `Match`",
"warning[lint:possibly-unresolved-reference] /src/tomllib/_parser.py:629:38 Name `datetime_obj` used when possibly not defined",
"error[lint:invalid-argument-type] /src/tomllib/_parser.py:632:58 Object of type `@Todo & ~AlwaysFalsy` cannot be assigned to parameter 1 (`match`) of function `match_to_localtime`; expected type `Match`",
"error[lint:invalid-argument-type] /src/tomllib/_parser.py:639:52 Object of type `@Todo & ~AlwaysFalsy` cannot be assigned to parameter 1 (`match`) of function `match_to_number`; expected type `Match`",
"error[lint:invalid-argument-type] /src/tomllib/_parser.py:632:58 Object of type `Unknown & ~AlwaysFalsy | @Todo & ~AlwaysFalsy` cannot be assigned to parameter 1 (`match`) of function `match_to_localtime`; expected type `Match`",
"error[lint:invalid-argument-type] /src/tomllib/_parser.py:639:52 Object of type `Unknown & ~AlwaysFalsy | @Todo & ~AlwaysFalsy` cannot be assigned to parameter 1 (`match`) of function `match_to_number`; expected type `Match`",
"warning[lint:unused-ignore-comment] /src/tomllib/_parser.py:682:31 Unused blanket `type: ignore` directive",
];

View File

@@ -0,0 +1,152 @@
from __future__ import annotations
from datetime import datetime
import pendulum
from airflow.decorators import dag, task
from airflow.models import DAG
from airflow.models.baseoperator import BaseOperator
from airflow.operators.dummy import DummyOperator
from airflow.plugins_manager import AirflowPlugin
from airflow.providers.standard.operators.python import PythonOperator
from airflow.utils.context import get_current_context
def access_invalid_key_in_context(**context):
print("access invalid key", context["conf"])
print("access invalid key", context.get("conf"))
@task
def access_invalid_key_task_out_of_dag(**context):
print("access invalid key", context["conf"])
print("access invalid key", context.get("conf"))
@task
def access_invalid_argument_task_out_of_dag(
execution_date, tomorrow_ds, logical_date, **context
):
print("execution date", execution_date)
print("access invalid key", context.get("conf"))
@task
def print_config(**context):
# This should not throw an error as logical_date is part of airflow context.
logical_date = context["logical_date"]
# Removed usage - should trigger violations
execution_date = context["execution_date"]
next_ds = context["next_ds"]
next_ds_nodash = context["next_ds_nodash"]
next_execution_date = context["next_execution_date"]
prev_ds = context["prev_ds"]
prev_ds_nodash = context["prev_ds_nodash"]
prev_execution_date = context["prev_execution_date"]
prev_execution_date_success = context["prev_execution_date_success"]
tomorrow_ds = context["tomorrow_ds"]
yesterday_ds = context["yesterday_ds"]
yesterday_ds_nodash = context["yesterday_ds_nodash"]
@task
def print_config_with_get_current_context():
context = get_current_context()
execution_date = context["execution_date"]
next_ds = context["next_ds"]
next_ds_nodash = context["next_ds_nodash"]
next_execution_date = context["next_execution_date"]
prev_ds = context["prev_ds"]
prev_ds_nodash = context["prev_ds_nodash"]
prev_execution_date = context["prev_execution_date"]
prev_execution_date_success = context["prev_execution_date_success"]
tomorrow_ds = context["tomorrow_ds"]
yesterday_ds = context["yesterday_ds"]
yesterday_ds_nodash = context["yesterday_ds_nodash"]
@task(task_id="print_the_context")
def print_context(ds=None, **kwargs):
"""Print the Airflow context and ds variable from the context."""
print(ds)
print(kwargs.get("tomorrow_ds"))
c = get_current_context()
c.get("execution_date")
@dag(
schedule=None,
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
catchup=False,
tags=[""],
)
def invalid_dag():
@task()
def access_invalid_key_task(**context):
print("access invalid key", context.get("conf"))
@task()
def access_invalid_key_explicit_task(execution_date):
print(execution_date)
task1 = PythonOperator(
task_id="task1",
python_callable=access_invalid_key_in_context,
)
access_invalid_key_task() >> task1
access_invalid_key_explicit_task()
access_invalid_argument_task_out_of_dag()
access_invalid_key_task_out_of_dag()
print_config()
print_config_with_get_current_context()
print_context()
invalid_dag()
with DAG(
dag_id="example_dag",
schedule_interval="@daily",
start_date=datetime(2023, 1, 1),
template_searchpath=["/templates"],
) as dag:
task1 = DummyOperator(
task_id="task1",
params={
# Removed variables in template
"execution_date": "{{ execution_date }}",
"next_ds": "{{ next_ds }}",
"prev_ds": "{{ prev_ds }}",
},
)
class CustomMacrosPlugin(AirflowPlugin):
name = "custom_macros"
macros = {
"execution_date_macro": lambda context: context["execution_date"],
"next_ds_macro": lambda context: context["next_ds"],
}
class CustomOperator(BaseOperator):
def execute(self, next_ds, context):
execution_date = context["execution_date"]
next_ds = context["next_ds"]
next_ds_nodash = context["next_ds_nodash"]
next_execution_date = context["next_execution_date"]
prev_ds = context["prev_ds"]
prev_ds_nodash = context["prev_ds_nodash"]
prev_execution_date = context["prev_execution_date"]
prev_execution_date_success = context["prev_execution_date_success"]
tomorrow_ds = context["tomorrow_ds"]
yesterday_ds = context["yesterday_ds"]
yesterday_ds_nodash = context["yesterday_ds_nodash"]
class CustomOperatorNew(BaseOperator):
def execute(self, context):
execution_date = context.get("execution_date")
next_ds = context.get("next_ds")

View File

@@ -0,0 +1,40 @@
from typing import TYPE_CHECKING
# Verify that statements nested in conditionals (such as top-level type-checking blocks)
# are still considered top-level
if TYPE_CHECKING:
import string
def import_in_function():
import symtable # [import-outside-toplevel]
import os, sys # [import-outside-toplevel]
import time as thyme # [import-outside-toplevel]
import random as rand, socket as sock # [import-outside-toplevel]
from collections import defaultdict # [import-outside-toplevel]
from math import sin as sign, cos as cosplay # [import-outside-toplevel]
# these should be allowed due to TID253 top-level ban
import foo_banned
import foo_banned as renamed
from pkg import bar_banned
from pkg import bar_banned as renamed
from pkg_banned import one as other, two, three
# this should still trigger an error due to multiple imports
from pkg import foo_allowed, bar_banned # [import-outside-toplevel]
class ClassWithImports:
import tokenize # [import-outside-toplevel]
def __init__(self):
import trace # [import-outside-toplevel]
# these should be allowed due to TID253 top-level ban
import foo_banned
import foo_banned as renamed
from pkg import bar_banned
from pkg import bar_banned as renamed
from pkg_banned import one as other, two, three
# this should still trigger an error due to multiple imports
from pkg import foo_allowed, bar_banned # [import-outside-toplevel]

View File

@@ -64,3 +64,20 @@ round(lorem, -2) # No error
round(lorem, inferred_int) # No error
round(lorem, 3 + 4) # No error
round(lorem, foo) # No error
# Fixes should preserve parentheses when argument
# contains newline.
# See https://github.com/astral-sh/ruff/issues/15598
round(-
1)
round(1
*1
)
# fix should be unsafe if comment is in call range
round(# a comment
17
)
round(
17 # a comment
)

View File

@@ -175,7 +175,9 @@ pub(crate) fn expression(expr: &Expr, checker: &mut Checker) {
if checker.enabled(Rule::NonPEP646Unpack) {
pyupgrade::rules::use_pep646_unpack(checker, subscript);
}
if checker.enabled(Rule::Airflow3Removal) {
airflow::rules::airflow_3_removal_expr(checker, expr);
}
pandas_vet::rules::subscript(checker, value, expr);
}
Expr::Tuple(ast::ExprTuple {
@@ -225,7 +227,7 @@ pub(crate) fn expression(expr: &Expr, checker: &mut Checker) {
refurb::rules::regex_flag_alias(checker, expr);
}
if checker.enabled(Rule::Airflow3Removal) {
airflow::rules::removed_in_3(checker, expr);
airflow::rules::airflow_3_removal_expr(checker, expr);
}
if checker.enabled(Rule::Airflow3MovedToProvider) {
airflow::rules::moved_to_provider_in_3(checker, expr);
@@ -309,7 +311,7 @@ pub(crate) fn expression(expr: &Expr, checker: &mut Checker) {
}
}
if checker.enabled(Rule::Airflow3Removal) {
airflow::rules::removed_in_3(checker, expr);
airflow::rules::airflow_3_removal_expr(checker, expr);
}
if checker.enabled(Rule::MixedCaseVariableInGlobalScope) {
if matches!(checker.semantic.current_scope().kind, ScopeKind::Module) {
@@ -447,7 +449,7 @@ pub(crate) fn expression(expr: &Expr, checker: &mut Checker) {
flake8_pyi::rules::bytestring_attribute(checker, expr);
}
if checker.enabled(Rule::Airflow3Removal) {
airflow::rules::removed_in_3(checker, expr);
airflow::rules::airflow_3_removal_expr(checker, expr);
}
}
Expr::Call(
@@ -1148,7 +1150,7 @@ pub(crate) fn expression(expr: &Expr, checker: &mut Checker) {
ruff::rules::unnecessary_regular_expression(checker, call);
}
if checker.enabled(Rule::Airflow3Removal) {
airflow::rules::removed_in_3(checker, expr);
airflow::rules::airflow_3_removal_expr(checker, expr);
}
if checker.enabled(Rule::UnnecessaryCastToInt) {
ruff::rules::unnecessary_cast_to_int(checker, call);

View File

@@ -376,6 +376,9 @@ pub(crate) fn statement(stmt: &Stmt, checker: &mut Checker) {
if checker.enabled(Rule::PytestParameterWithDefaultArgument) {
flake8_pytest_style::rules::parameter_with_default_argument(checker, function_def);
}
if checker.enabled(Rule::Airflow3Removal) {
airflow::rules::airflow_3_removal_function_def(checker, function_def);
}
if checker.enabled(Rule::NonPEP695GenericFunction) {
pyupgrade::rules::non_pep695_generic_function(checker, function_def);
}
@@ -605,6 +608,10 @@ pub(crate) fn statement(stmt: &Stmt, checker: &mut Checker) {
flake8_bandit::rules::suspicious_imports(checker, stmt);
}
if checker.enabled(Rule::BannedModuleLevelImports) {
flake8_tidy_imports::rules::banned_module_level_imports(checker, stmt);
}
for alias in names {
if checker.enabled(Rule::NonAsciiImportName) {
pylint::rules::non_ascii_module_import(checker, alias);
@@ -629,18 +636,6 @@ pub(crate) fn statement(stmt: &Stmt, checker: &mut Checker) {
);
}
if checker.enabled(Rule::BannedModuleLevelImports) {
flake8_tidy_imports::rules::banned_module_level_imports(
checker,
&flake8_tidy_imports::matchers::NameMatchPolicy::MatchNameOrParent(
flake8_tidy_imports::matchers::MatchNameOrParent {
module: &alias.name,
},
),
&alias,
);
}
if !checker.source_type.is_stub() {
if checker.enabled(Rule::UselessImportAlias) {
pylint::rules::useless_import_alias(checker, alias);
@@ -845,36 +840,9 @@ pub(crate) fn statement(stmt: &Stmt, checker: &mut Checker) {
}
}
if checker.enabled(Rule::BannedModuleLevelImports) {
if let Some(module) = helpers::resolve_imported_module_path(
level,
module,
checker.module.qualified_name(),
) {
flake8_tidy_imports::rules::banned_module_level_imports(
checker,
&flake8_tidy_imports::matchers::NameMatchPolicy::MatchNameOrParent(
flake8_tidy_imports::matchers::MatchNameOrParent { module: &module },
),
&stmt,
);
for alias in names {
if &alias.name == "*" {
continue;
}
flake8_tidy_imports::rules::banned_module_level_imports(
checker,
&flake8_tidy_imports::matchers::NameMatchPolicy::MatchName(
flake8_tidy_imports::matchers::MatchName {
module: &module,
member: &alias.name,
},
),
&alias,
);
}
}
flake8_tidy_imports::rules::banned_module_level_imports(checker, stmt);
}
if checker.enabled(Rule::PytestIncorrectPytestImport) {
if let Some(diagnostic) =
flake8_pytest_style::rules::import_from(stmt, module, level)

View File

@@ -189,7 +189,7 @@ pub(crate) struct Checker<'a> {
/// The [`Path`] to the package containing the current file.
package: Option<PackageRoot<'a>>,
/// The module representation of the current file (e.g., `foo.bar`).
module: Module<'a>,
pub(crate) module: Module<'a>,
/// The [`PySourceType`] of the current file.
pub(crate) source_type: PySourceType,
/// The [`CellOffsets`] for the current file, if it's a Jupyter notebook.

View File

@@ -316,7 +316,7 @@ mod schema {
// Filter out all test-only rules
#[cfg(any(feature = "test-rules", test))]
#[allow(clippy::used_underscore_binding)]
if _rule.starts_with("RUF9") {
if _rule.starts_with("RUF9") || _rule == "PLW0101" {
return false;
}

View File

@@ -18,6 +18,7 @@ mod tests {
#[test_case(Rule::Airflow3Removal, Path::new("AIR302_names.py"))]
#[test_case(Rule::Airflow3Removal, Path::new("AIR302_class_attribute.py"))]
#[test_case(Rule::Airflow3Removal, Path::new("AIR302_airflow_plugin.py"))]
#[test_case(Rule::Airflow3Removal, Path::new("AIR302_context.py"))]
#[test_case(Rule::Airflow3MovedToProvider, Path::new("AIR303.py"))]
fn rules(rule_code: Rule, path: &Path) -> Result<()> {
let snapshot = format!("{}_{}", rule_code.noqa_code(), path.to_string_lossy());

View File

@@ -1,17 +1,18 @@
use crate::checkers::ast::Checker;
use ruff_diagnostics::{Diagnostic, Edit, Fix, FixAvailability, Violation};
use ruff_macros::{derive_message_formats, ViolationMetadata};
use ruff_python_ast::helpers::map_callable;
use ruff_python_ast::{
name::QualifiedName, Arguments, Expr, ExprAttribute, ExprCall, ExprContext, ExprName,
StmtClassDef,
ExprStringLiteral, ExprSubscript, Stmt, StmtClassDef, StmtFunctionDef,
};
use ruff_python_semantic::analyze::typing;
use ruff_python_semantic::Modules;
use ruff_python_semantic::ScopeKind;
use ruff_python_semantic::SemanticModel;
use ruff_text_size::Ranged;
use ruff_text_size::TextRange;
use crate::checkers::ast::Checker;
/// ## What it does
/// Checks for uses of deprecated Airflow functions and values.
///
@@ -71,8 +72,15 @@ impl Violation for Airflow3Removal {
}
}
#[derive(Debug, Eq, PartialEq)]
enum Replacement {
None,
Name(&'static str),
Message(&'static str),
}
/// AIR302
pub(crate) fn removed_in_3(checker: &mut Checker, expr: &Expr) {
pub(crate) fn airflow_3_removal_expr(checker: &mut Checker, expr: &Expr) {
if !checker.semantic().seen_module(Modules::AIRFLOW) {
return;
}
@@ -83,10 +91,11 @@ pub(crate) fn removed_in_3(checker: &mut Checker, expr: &Expr) {
func, arguments, ..
},
) => {
if let Some(qualname) = checker.semantic().resolve_qualified_name(func) {
check_call_arguments(checker, &qualname, arguments);
if let Some(qualified_name) = checker.semantic().resolve_qualified_name(func) {
check_call_arguments(checker, &qualified_name, arguments);
};
check_method(checker, call_expr);
check_context_key_usage_in_call(checker, call_expr);
}
Expr::Attribute(attribute_expr @ ExprAttribute { attr, .. }) => {
check_name(checker, expr, attr.range());
@@ -100,15 +109,72 @@ pub(crate) fn removed_in_3(checker: &mut Checker, expr: &Expr) {
}
}
}
Expr::Subscript(subscript_expr) => {
check_context_key_usage_in_subscript(checker, subscript_expr);
}
_ => {}
}
}
#[derive(Debug, Eq, PartialEq)]
enum Replacement {
None,
Name(&'static str),
Message(&'static str),
/// AIR302
pub(crate) fn airflow_3_removal_function_def(
checker: &mut Checker,
function_def: &StmtFunctionDef,
) {
if !checker.semantic().seen_module(Modules::AIRFLOW) {
return;
}
check_function_parameters(checker, function_def);
}
const REMOVED_CONTEXT_KEYS: [&str; 12] = [
"conf",
"execution_date",
"next_ds",
"next_ds_nodash",
"next_execution_date",
"prev_ds",
"prev_ds_nodash",
"prev_execution_date",
"prev_execution_date_success",
"tomorrow_ds",
"yesterday_ds",
"yesterday_ds_nodash",
];
/// Check the function parameters for removed context keys.
///
/// For example:
///
/// ```python
/// from airflow.decorators import task
///
/// @task
/// def another_task(execution_date, **kwargs):
/// # ^^^^^^^^^^^^^^
/// # 'execution_date' is removed in Airflow 3.0
/// pass
/// ```
fn check_function_parameters(checker: &mut Checker, function_def: &StmtFunctionDef) {
if !is_airflow_task(function_def, checker.semantic())
&& !is_execute_method_inherits_from_airflow_operator(function_def, checker.semantic())
{
return;
}
for param in function_def.parameters.iter_non_variadic_params() {
let param_name = param.parameter.name.as_str();
if REMOVED_CONTEXT_KEYS.contains(&param_name) {
checker.diagnostics.push(Diagnostic::new(
Airflow3Removal {
deprecated: param_name.to_string(),
replacement: Replacement::None,
},
param.parameter.name.range(),
));
}
}
}
/// Check whether a removed Airflow argument is passed.
@@ -120,8 +186,12 @@ enum Replacement {
///
/// DAG(schedule_interval="@daily")
/// ```
fn check_call_arguments(checker: &mut Checker, qualname: &QualifiedName, arguments: &Arguments) {
match qualname.segments() {
fn check_call_arguments(
checker: &mut Checker,
qualified_name: &QualifiedName,
arguments: &Arguments,
) {
match qualified_name.segments() {
["airflow", .., "DAG" | "dag"] => {
checker.diagnostics.extend(diagnostic_for_argument(
arguments,
@@ -145,7 +215,7 @@ fn check_call_arguments(checker: &mut Checker, qualname: &QualifiedName, argumen
));
}
_ => {
if is_airflow_auth_manager(qualname.segments()) {
if is_airflow_auth_manager(qualified_name.segments()) {
if !arguments.is_empty() {
checker.diagnostics.push(Diagnostic::new(
Airflow3Removal {
@@ -157,13 +227,13 @@ fn check_call_arguments(checker: &mut Checker, qualname: &QualifiedName, argumen
arguments.range(),
));
}
} else if is_airflow_task_handler(qualname.segments()) {
} else if is_airflow_task_handler(qualified_name.segments()) {
checker.diagnostics.extend(diagnostic_for_argument(
arguments,
"filename_template",
None,
));
} else if is_airflow_operator(qualname.segments()) {
} else if is_airflow_operator(qualified_name.segments()) {
checker
.diagnostics
.extend(diagnostic_for_argument(arguments, "sla", None));
@@ -172,7 +242,7 @@ fn check_call_arguments(checker: &mut Checker, qualname: &QualifiedName, argumen
"task_concurrency",
Some("max_active_tis_per_dag"),
));
match qualname.segments() {
match qualified_name.segments() {
["airflow", .., "operators", "trigger_dagrun", "TriggerDagRunOperator"] => {
checker.diagnostics.extend(diagnostic_for_argument(
arguments,
@@ -252,6 +322,137 @@ fn check_class_attribute(checker: &mut Checker, attribute_expr: &ExprAttribute)
}
}
/// Checks whether an Airflow 3.0removed context key is used in a function decorated with `@task`.
///
/// Specifically, it flags the following two scenarios:
///
/// 1. A removed context key accessed via `context.get("...")` where context is coming from
/// `get_current_context` function.
///
/// ```python
/// from airflow.decorators import task
/// from airflow.utils.context import get_current_context
///
///
/// @task
/// def my_task():
/// context = get_current_context()
/// context.get("conf") # 'conf' is removed in Airflow 3.0
/// ```
///
/// 2. A removed context key accessed via `context.get("...")` where context is a kwarg parameter.
///
/// ```python
/// from airflow.decorators import task
///
///
/// @task
/// def my_task(**context):
/// context.get("conf") # 'conf' is removed in Airflow 3.0
/// ```
fn check_context_key_usage_in_call(checker: &mut Checker, call_expr: &ExprCall) {
if !in_airflow_task_function(checker.semantic()) {
return;
}
let Expr::Attribute(ExprAttribute { value, attr, .. }) = &*call_expr.func else {
return;
};
if attr.as_str() != "get" {
return;
}
let is_kwarg_parameter = value
.as_name_expr()
.is_some_and(|name| is_kwarg_parameter(checker.semantic(), name));
let is_assigned_from_get_current_context =
typing::resolve_assignment(value, checker.semantic()).is_some_and(|qualified_name| {
matches!(
qualified_name.segments(),
["airflow", "utils", "context", "get_current_context"]
)
});
if !(is_kwarg_parameter || is_assigned_from_get_current_context) {
return;
}
for removed_key in REMOVED_CONTEXT_KEYS {
let Some(Expr::StringLiteral(ExprStringLiteral { value, range })) =
call_expr.arguments.find_positional(0)
else {
continue;
};
if value == removed_key {
checker.diagnostics.push(Diagnostic::new(
Airflow3Removal {
deprecated: removed_key.to_string(),
replacement: Replacement::None,
},
*range,
));
}
}
}
/// Check if a subscript expression accesses a removed Airflow context variable.
/// If a removed key is found, push a corresponding diagnostic.
fn check_context_key_usage_in_subscript(checker: &mut Checker, subscript: &ExprSubscript) {
if !in_airflow_task_function(checker.semantic()) {
return;
}
let ExprSubscript { value, slice, .. } = subscript;
let Some(ExprStringLiteral { value: key, .. }) = slice.as_string_literal_expr() else {
return;
};
let is_kwarg_parameter = value
.as_name_expr()
.is_some_and(|name| is_kwarg_parameter(checker.semantic(), name));
let is_assigned_from_get_current_context =
typing::resolve_assignment(value, checker.semantic()).is_some_and(|qualified_name| {
matches!(
qualified_name.segments(),
["airflow", "utils", "context", "get_current_context"]
)
});
if !(is_kwarg_parameter || is_assigned_from_get_current_context) {
return;
}
if REMOVED_CONTEXT_KEYS.contains(&key.to_str()) {
checker.diagnostics.push(Diagnostic::new(
Airflow3Removal {
deprecated: key.to_string(),
replacement: Replacement::None,
},
slice.range(),
));
}
}
/// Finds the parameter definition for a given name expression in a function.
fn is_kwarg_parameter(semantic: &SemanticModel, name: &ExprName) -> bool {
let Some(binding_id) = semantic.only_binding(name) else {
return false;
};
let binding = semantic.binding(binding_id);
let Some(Stmt::FunctionDef(StmtFunctionDef { parameters, .. })) = binding.statement(semantic)
else {
return false;
};
parameters
.kwarg
.as_deref()
.is_some_and(|kwarg| kwarg.name.as_str() == name.id.as_str())
}
/// Check whether a removed Airflow class method is called.
///
/// For example:
@@ -860,3 +1061,55 @@ fn is_airflow_builtin_or_provider(segments: &[&str], module: &str, symbol_suffix
_ => false,
}
}
/// Returns `true` if the current statement hierarchy has a function that's decorated with
/// `@airflow.decorators.task`.
fn in_airflow_task_function(semantic: &SemanticModel) -> bool {
semantic
.current_statements()
.find_map(|stmt| stmt.as_function_def_stmt())
.is_some_and(|function_def| is_airflow_task(function_def, semantic))
}
/// Returns `true` if the given function is decorated with `@airflow.decorators.task`.
fn is_airflow_task(function_def: &StmtFunctionDef, semantic: &SemanticModel) -> bool {
function_def.decorator_list.iter().any(|decorator| {
semantic
.resolve_qualified_name(map_callable(&decorator.expression))
.is_some_and(|qualified_name| {
matches!(qualified_name.segments(), ["airflow", "decorators", "task"])
})
})
}
/// Check it's "execute" method inherits from Airflow base operator
///
/// For example:
///
/// ```python
/// from airflow.models.baseoperator import BaseOperator
///
/// class CustomOperator(BaseOperator):
/// def execute(self):
/// pass
/// ```
fn is_execute_method_inherits_from_airflow_operator(
function_def: &StmtFunctionDef,
semantic: &SemanticModel,
) -> bool {
if function_def.name.as_str() != "execute" {
return false;
}
let ScopeKind::Class(class_def) = semantic.current_scope().kind else {
return false;
};
class_def.bases().iter().any(|class_base| {
semantic
.resolve_qualified_name(class_base)
.is_some_and(|qualified_name| {
matches!(qualified_name.segments(), ["airflow", .., "BaseOperator"])
})
})
}

View File

@@ -0,0 +1,338 @@
---
source: crates/ruff_linter/src/rules/airflow/mod.rs
snapshot_kind: text
---
AIR302_context.py:22:41: AIR302 `conf` is removed in Airflow 3.0
|
20 | @task
21 | def access_invalid_key_task_out_of_dag(**context):
22 | print("access invalid key", context["conf"])
| ^^^^^^ AIR302
23 | print("access invalid key", context.get("conf"))
|
AIR302_context.py:23:45: AIR302 `conf` is removed in Airflow 3.0
|
21 | def access_invalid_key_task_out_of_dag(**context):
22 | print("access invalid key", context["conf"])
23 | print("access invalid key", context.get("conf"))
| ^^^^^^ AIR302
|
AIR302_context.py:28:5: AIR302 `execution_date` is removed in Airflow 3.0
|
26 | @task
27 | def access_invalid_argument_task_out_of_dag(
28 | execution_date, tomorrow_ds, logical_date, **context
| ^^^^^^^^^^^^^^ AIR302
29 | ):
30 | print("execution date", execution_date)
|
AIR302_context.py:28:21: AIR302 `tomorrow_ds` is removed in Airflow 3.0
|
26 | @task
27 | def access_invalid_argument_task_out_of_dag(
28 | execution_date, tomorrow_ds, logical_date, **context
| ^^^^^^^^^^^ AIR302
29 | ):
30 | print("execution date", execution_date)
|
AIR302_context.py:31:45: AIR302 `conf` is removed in Airflow 3.0
|
29 | ):
30 | print("execution date", execution_date)
31 | print("access invalid key", context.get("conf"))
| ^^^^^^ AIR302
|
AIR302_context.py:40:30: AIR302 `execution_date` is removed in Airflow 3.0
|
39 | # Removed usage - should trigger violations
40 | execution_date = context["execution_date"]
| ^^^^^^^^^^^^^^^^ AIR302
41 | next_ds = context["next_ds"]
42 | next_ds_nodash = context["next_ds_nodash"]
|
AIR302_context.py:41:23: AIR302 `next_ds` is removed in Airflow 3.0
|
39 | # Removed usage - should trigger violations
40 | execution_date = context["execution_date"]
41 | next_ds = context["next_ds"]
| ^^^^^^^^^ AIR302
42 | next_ds_nodash = context["next_ds_nodash"]
43 | next_execution_date = context["next_execution_date"]
|
AIR302_context.py:42:30: AIR302 `next_ds_nodash` is removed in Airflow 3.0
|
40 | execution_date = context["execution_date"]
41 | next_ds = context["next_ds"]
42 | next_ds_nodash = context["next_ds_nodash"]
| ^^^^^^^^^^^^^^^^ AIR302
43 | next_execution_date = context["next_execution_date"]
44 | prev_ds = context["prev_ds"]
|
AIR302_context.py:43:35: AIR302 `next_execution_date` is removed in Airflow 3.0
|
41 | next_ds = context["next_ds"]
42 | next_ds_nodash = context["next_ds_nodash"]
43 | next_execution_date = context["next_execution_date"]
| ^^^^^^^^^^^^^^^^^^^^^ AIR302
44 | prev_ds = context["prev_ds"]
45 | prev_ds_nodash = context["prev_ds_nodash"]
|
AIR302_context.py:44:23: AIR302 `prev_ds` is removed in Airflow 3.0
|
42 | next_ds_nodash = context["next_ds_nodash"]
43 | next_execution_date = context["next_execution_date"]
44 | prev_ds = context["prev_ds"]
| ^^^^^^^^^ AIR302
45 | prev_ds_nodash = context["prev_ds_nodash"]
46 | prev_execution_date = context["prev_execution_date"]
|
AIR302_context.py:45:30: AIR302 `prev_ds_nodash` is removed in Airflow 3.0
|
43 | next_execution_date = context["next_execution_date"]
44 | prev_ds = context["prev_ds"]
45 | prev_ds_nodash = context["prev_ds_nodash"]
| ^^^^^^^^^^^^^^^^ AIR302
46 | prev_execution_date = context["prev_execution_date"]
47 | prev_execution_date_success = context["prev_execution_date_success"]
|
AIR302_context.py:46:35: AIR302 `prev_execution_date` is removed in Airflow 3.0
|
44 | prev_ds = context["prev_ds"]
45 | prev_ds_nodash = context["prev_ds_nodash"]
46 | prev_execution_date = context["prev_execution_date"]
| ^^^^^^^^^^^^^^^^^^^^^ AIR302
47 | prev_execution_date_success = context["prev_execution_date_success"]
48 | tomorrow_ds = context["tomorrow_ds"]
|
AIR302_context.py:47:43: AIR302 `prev_execution_date_success` is removed in Airflow 3.0
|
45 | prev_ds_nodash = context["prev_ds_nodash"]
46 | prev_execution_date = context["prev_execution_date"]
47 | prev_execution_date_success = context["prev_execution_date_success"]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AIR302
48 | tomorrow_ds = context["tomorrow_ds"]
49 | yesterday_ds = context["yesterday_ds"]
|
AIR302_context.py:48:27: AIR302 `tomorrow_ds` is removed in Airflow 3.0
|
46 | prev_execution_date = context["prev_execution_date"]
47 | prev_execution_date_success = context["prev_execution_date_success"]
48 | tomorrow_ds = context["tomorrow_ds"]
| ^^^^^^^^^^^^^ AIR302
49 | yesterday_ds = context["yesterday_ds"]
50 | yesterday_ds_nodash = context["yesterday_ds_nodash"]
|
AIR302_context.py:49:28: AIR302 `yesterday_ds` is removed in Airflow 3.0
|
47 | prev_execution_date_success = context["prev_execution_date_success"]
48 | tomorrow_ds = context["tomorrow_ds"]
49 | yesterday_ds = context["yesterday_ds"]
| ^^^^^^^^^^^^^^ AIR302
50 | yesterday_ds_nodash = context["yesterday_ds_nodash"]
|
AIR302_context.py:50:35: AIR302 `yesterday_ds_nodash` is removed in Airflow 3.0
|
48 | tomorrow_ds = context["tomorrow_ds"]
49 | yesterday_ds = context["yesterday_ds"]
50 | yesterday_ds_nodash = context["yesterday_ds_nodash"]
| ^^^^^^^^^^^^^^^^^^^^^ AIR302
|
AIR302_context.py:56:30: AIR302 `execution_date` is removed in Airflow 3.0
|
54 | def print_config_with_get_current_context():
55 | context = get_current_context()
56 | execution_date = context["execution_date"]
| ^^^^^^^^^^^^^^^^ AIR302
57 | next_ds = context["next_ds"]
58 | next_ds_nodash = context["next_ds_nodash"]
|
AIR302_context.py:57:23: AIR302 `next_ds` is removed in Airflow 3.0
|
55 | context = get_current_context()
56 | execution_date = context["execution_date"]
57 | next_ds = context["next_ds"]
| ^^^^^^^^^ AIR302
58 | next_ds_nodash = context["next_ds_nodash"]
59 | next_execution_date = context["next_execution_date"]
|
AIR302_context.py:58:30: AIR302 `next_ds_nodash` is removed in Airflow 3.0
|
56 | execution_date = context["execution_date"]
57 | next_ds = context["next_ds"]
58 | next_ds_nodash = context["next_ds_nodash"]
| ^^^^^^^^^^^^^^^^ AIR302
59 | next_execution_date = context["next_execution_date"]
60 | prev_ds = context["prev_ds"]
|
AIR302_context.py:59:35: AIR302 `next_execution_date` is removed in Airflow 3.0
|
57 | next_ds = context["next_ds"]
58 | next_ds_nodash = context["next_ds_nodash"]
59 | next_execution_date = context["next_execution_date"]
| ^^^^^^^^^^^^^^^^^^^^^ AIR302
60 | prev_ds = context["prev_ds"]
61 | prev_ds_nodash = context["prev_ds_nodash"]
|
AIR302_context.py:60:23: AIR302 `prev_ds` is removed in Airflow 3.0
|
58 | next_ds_nodash = context["next_ds_nodash"]
59 | next_execution_date = context["next_execution_date"]
60 | prev_ds = context["prev_ds"]
| ^^^^^^^^^ AIR302
61 | prev_ds_nodash = context["prev_ds_nodash"]
62 | prev_execution_date = context["prev_execution_date"]
|
AIR302_context.py:61:30: AIR302 `prev_ds_nodash` is removed in Airflow 3.0
|
59 | next_execution_date = context["next_execution_date"]
60 | prev_ds = context["prev_ds"]
61 | prev_ds_nodash = context["prev_ds_nodash"]
| ^^^^^^^^^^^^^^^^ AIR302
62 | prev_execution_date = context["prev_execution_date"]
63 | prev_execution_date_success = context["prev_execution_date_success"]
|
AIR302_context.py:62:35: AIR302 `prev_execution_date` is removed in Airflow 3.0
|
60 | prev_ds = context["prev_ds"]
61 | prev_ds_nodash = context["prev_ds_nodash"]
62 | prev_execution_date = context["prev_execution_date"]
| ^^^^^^^^^^^^^^^^^^^^^ AIR302
63 | prev_execution_date_success = context["prev_execution_date_success"]
64 | tomorrow_ds = context["tomorrow_ds"]
|
AIR302_context.py:63:43: AIR302 `prev_execution_date_success` is removed in Airflow 3.0
|
61 | prev_ds_nodash = context["prev_ds_nodash"]
62 | prev_execution_date = context["prev_execution_date"]
63 | prev_execution_date_success = context["prev_execution_date_success"]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AIR302
64 | tomorrow_ds = context["tomorrow_ds"]
65 | yesterday_ds = context["yesterday_ds"]
|
AIR302_context.py:64:27: AIR302 `tomorrow_ds` is removed in Airflow 3.0
|
62 | prev_execution_date = context["prev_execution_date"]
63 | prev_execution_date_success = context["prev_execution_date_success"]
64 | tomorrow_ds = context["tomorrow_ds"]
| ^^^^^^^^^^^^^ AIR302
65 | yesterday_ds = context["yesterday_ds"]
66 | yesterday_ds_nodash = context["yesterday_ds_nodash"]
|
AIR302_context.py:65:28: AIR302 `yesterday_ds` is removed in Airflow 3.0
|
63 | prev_execution_date_success = context["prev_execution_date_success"]
64 | tomorrow_ds = context["tomorrow_ds"]
65 | yesterday_ds = context["yesterday_ds"]
| ^^^^^^^^^^^^^^ AIR302
66 | yesterday_ds_nodash = context["yesterday_ds_nodash"]
|
AIR302_context.py:66:35: AIR302 `yesterday_ds_nodash` is removed in Airflow 3.0
|
64 | tomorrow_ds = context["tomorrow_ds"]
65 | yesterday_ds = context["yesterday_ds"]
66 | yesterday_ds_nodash = context["yesterday_ds_nodash"]
| ^^^^^^^^^^^^^^^^^^^^^ AIR302
|
AIR302_context.py:73:22: AIR302 `tomorrow_ds` is removed in Airflow 3.0
|
71 | """Print the Airflow context and ds variable from the context."""
72 | print(ds)
73 | print(kwargs.get("tomorrow_ds"))
| ^^^^^^^^^^^^^ AIR302
74 | c = get_current_context()
75 | c.get("execution_date")
|
AIR302_context.py:75:11: AIR302 `execution_date` is removed in Airflow 3.0
|
73 | print(kwargs.get("tomorrow_ds"))
74 | c = get_current_context()
75 | c.get("execution_date")
| ^^^^^^^^^^^^^^^^ AIR302
|
AIR302_context.py:87:49: AIR302 `conf` is removed in Airflow 3.0
|
85 | @task()
86 | def access_invalid_key_task(**context):
87 | print("access invalid key", context.get("conf"))
| ^^^^^^ AIR302
88 |
89 | @task()
|
AIR302_context.py:90:42: AIR302 `execution_date` is removed in Airflow 3.0
|
89 | @task()
90 | def access_invalid_key_explicit_task(execution_date):
| ^^^^^^^^^^^^^^ AIR302
91 | print(execution_date)
|
AIR302_context.py:111:5: AIR302 [*] `schedule_interval` is removed in Airflow 3.0
|
109 | with DAG(
110 | dag_id="example_dag",
111 | schedule_interval="@daily",
| ^^^^^^^^^^^^^^^^^ AIR302
112 | start_date=datetime(2023, 1, 1),
113 | template_searchpath=["/templates"],
|
= help: Use `schedule` instead
Safe fix
108 108 |
109 109 | with DAG(
110 110 | dag_id="example_dag",
111 |- schedule_interval="@daily",
111 |+ schedule="@daily",
112 112 | start_date=datetime(2023, 1, 1),
113 113 | template_searchpath=["/templates"],
114 114 | ) as dag:
AIR302_context.py:115:13: AIR302 `airflow.operators.dummy.DummyOperator` is removed in Airflow 3.0
|
113 | template_searchpath=["/templates"],
114 | ) as dag:
115 | task1 = DummyOperator(
| ^^^^^^^^^^^^^ AIR302
116 | task_id="task1",
117 | params={
|
= help: Use `airflow.operators.empty.EmptyOperator` instead
AIR302_context.py:135:23: AIR302 `next_ds` is removed in Airflow 3.0
|
134 | class CustomOperator(BaseOperator):
135 | def execute(self, next_ds, context):
| ^^^^^^^ AIR302
136 | execution_date = context["execution_date"]
137 | next_ds = context["next_ds"]
|

View File

@@ -1,9 +1,12 @@
use ruff_diagnostics::{Diagnostic, Violation};
use ruff_macros::{derive_message_formats, ViolationMetadata};
use ruff_python_ast::helpers::resolve_imported_module_path;
use ruff_python_ast::{Alias, AnyNodeRef, Stmt, StmtImport, StmtImportFrom};
use ruff_text_size::Ranged;
use std::borrow::Cow;
use crate::checkers::ast::Checker;
use crate::rules::flake8_tidy_imports::matchers::NameMatchPolicy;
use crate::rules::flake8_tidy_imports::matchers::{MatchName, MatchNameOrParent, NameMatchPolicy};
/// ## What it does
/// Checks for module-level imports that should instead be imported lazily
@@ -53,28 +56,131 @@ impl Violation for BannedModuleLevelImports {
}
/// TID253
pub(crate) fn banned_module_level_imports<T: Ranged>(
checker: &mut Checker,
policy: &NameMatchPolicy,
node: &T,
) {
pub(crate) fn banned_module_level_imports(checker: &mut Checker, stmt: &Stmt) {
if !checker.semantic().at_top_level() {
return;
}
if let Some(banned_module) = policy.find(
checker
.settings
.flake8_tidy_imports
.banned_module_level_imports
.iter()
.map(AsRef::as_ref),
) {
checker.diagnostics.push(Diagnostic::new(
BannedModuleLevelImports {
name: banned_module,
},
node.range(),
));
for (policy, node) in &BannedModuleImportPolicies::new(stmt, checker) {
if let Some(banned_module) = policy.find(
checker
.settings
.flake8_tidy_imports
.banned_module_level_imports(),
) {
checker.diagnostics.push(Diagnostic::new(
BannedModuleLevelImports {
name: banned_module,
},
node.range(),
));
}
}
}
pub(crate) enum BannedModuleImportPolicies<'a> {
Import(&'a StmtImport),
ImportFrom {
module: Option<Cow<'a, str>>,
node: &'a StmtImportFrom,
},
NonImport,
}
impl<'a> BannedModuleImportPolicies<'a> {
pub(crate) fn new(stmt: &'a Stmt, checker: &Checker) -> Self {
match stmt {
Stmt::Import(import) => Self::Import(import),
Stmt::ImportFrom(import @ StmtImportFrom { module, level, .. }) => {
let module = resolve_imported_module_path(
*level,
module.as_deref(),
checker.module.qualified_name(),
);
Self::ImportFrom {
module,
node: import,
}
}
_ => Self::NonImport,
}
}
}
impl<'a> IntoIterator for &'a BannedModuleImportPolicies<'a> {
type Item = <Self::IntoIter as Iterator>::Item;
type IntoIter = BannedModuleImportPoliciesIter<'a>;
fn into_iter(self) -> Self::IntoIter {
match self {
BannedModuleImportPolicies::Import(import) => {
BannedModuleImportPoliciesIter::Import(import.names.iter())
}
BannedModuleImportPolicies::ImportFrom { module, node } => {
BannedModuleImportPoliciesIter::ImportFrom {
module: module.as_deref(),
names: node.names.iter(),
import: Some(node),
}
}
BannedModuleImportPolicies::NonImport => BannedModuleImportPoliciesIter::NonImport,
}
}
}
pub(crate) enum BannedModuleImportPoliciesIter<'a> {
Import(std::slice::Iter<'a, Alias>),
ImportFrom {
module: Option<&'a str>,
names: std::slice::Iter<'a, Alias>,
import: Option<&'a StmtImportFrom>,
},
NonImport,
}
impl<'a> Iterator for BannedModuleImportPoliciesIter<'a> {
type Item = (NameMatchPolicy<'a>, AnyNodeRef<'a>);
fn next(&mut self) -> Option<Self::Item> {
match self {
Self::Import(names) => {
let name = names.next()?;
Some((
NameMatchPolicy::MatchNameOrParent(MatchNameOrParent { module: &name.name }),
name.into(),
))
}
Self::ImportFrom {
module,
import,
names,
} => {
let module = module.as_ref()?;
if let Some(import) = import.take() {
return Some((
NameMatchPolicy::MatchNameOrParent(MatchNameOrParent { module }),
import.into(),
));
}
loop {
let alias = names.next()?;
if &alias.name == "*" {
continue;
}
break Some((
NameMatchPolicy::MatchName(MatchName {
module,
member: &alias.name,
}),
alias.into(),
));
}
}
Self::NonImport => None,
}
}
}

View File

@@ -46,6 +46,12 @@ pub struct Settings {
pub banned_module_level_imports: Vec<String>,
}
impl Settings {
pub fn banned_module_level_imports(&self) -> impl Iterator<Item = &str> {
self.banned_module_level_imports.iter().map(AsRef::as_ref)
}
}
impl Display for Settings {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
display_settings! {

View File

@@ -13,7 +13,7 @@ mod tests {
use test_case::test_case;
use crate::registry::Rule;
use crate::rules::pylint;
use crate::rules::{flake8_tidy_imports, pylint};
use crate::settings::types::{PreviewMode, PythonVersion};
use crate::settings::LinterSettings;
@@ -412,6 +412,30 @@ mod tests {
Ok(())
}
#[test]
fn import_outside_top_level_with_banned() -> Result<()> {
let diagnostics = test_path(
Path::new("pylint/import_outside_top_level_with_banned.py"),
&LinterSettings {
preview: PreviewMode::Enabled,
flake8_tidy_imports: flake8_tidy_imports::settings::Settings {
banned_module_level_imports: vec![
"foo_banned".to_string(),
"pkg_banned".to_string(),
"pkg.bar_banned".to_string(),
],
..Default::default()
},
..LinterSettings::for_rules(vec![
Rule::BannedModuleLevelImports,
Rule::ImportOutsideTopLevel,
])
},
)?;
assert_messages!(diagnostics);
Ok(())
}
#[test_case(
Rule::RepeatedEqualityComparison,
Path::new("repeated_equality_comparison.py")

View File

@@ -3,7 +3,10 @@ use ruff_macros::{derive_message_formats, ViolationMetadata};
use ruff_python_ast::Stmt;
use ruff_text_size::Ranged;
use crate::checkers::ast::Checker;
use crate::rules::flake8_tidy_imports::rules::BannedModuleImportPolicies;
use crate::{
checkers::ast::Checker, codes::Rule, rules::flake8_tidy_imports::matchers::NameMatchPolicy,
};
/// ## What it does
/// Checks for `import` statements outside of a module's top-level scope, such
@@ -54,9 +57,45 @@ impl Violation for ImportOutsideTopLevel {
/// C0415
pub(crate) fn import_outside_top_level(checker: &mut Checker, stmt: &Stmt) {
if !checker.semantic().current_scope().kind.is_module() {
checker
.diagnostics
.push(Diagnostic::new(ImportOutsideTopLevel, stmt.range()));
if checker.semantic().current_scope().kind.is_module() {
// "Top-level" imports are allowed
return;
}
// Check if any of the non-top-level imports are banned by TID253
// before emitting the diagnostic to avoid conflicts.
if checker.enabled(Rule::BannedModuleLevelImports) {
let mut all_aliases_banned = true;
let mut has_alias = false;
for (policy, node) in &BannedModuleImportPolicies::new(stmt, checker) {
if node.is_alias() {
has_alias = true;
all_aliases_banned &= is_banned_module_level_import(&policy, checker);
}
// If the entire import is banned
else if is_banned_module_level_import(&policy, checker) {
return;
}
}
if has_alias && all_aliases_banned {
return;
}
}
// Emit the diagnostic
checker
.diagnostics
.push(Diagnostic::new(ImportOutsideTopLevel, stmt.range()));
}
fn is_banned_module_level_import(policy: &NameMatchPolicy, checker: &Checker) -> bool {
policy
.find(
checker
.settings
.flake8_tidy_imports
.banned_module_level_imports(),
)
.is_some()
}

View File

@@ -0,0 +1,94 @@
---
source: crates/ruff_linter/src/rules/pylint/mod.rs
---
import_outside_top_level_with_banned.py:9:5: PLC0415 `import` should be at the top-level of a file
|
8 | def import_in_function():
9 | import symtable # [import-outside-toplevel]
| ^^^^^^^^^^^^^^^ PLC0415
10 | import os, sys # [import-outside-toplevel]
11 | import time as thyme # [import-outside-toplevel]
|
import_outside_top_level_with_banned.py:10:5: PLC0415 `import` should be at the top-level of a file
|
8 | def import_in_function():
9 | import symtable # [import-outside-toplevel]
10 | import os, sys # [import-outside-toplevel]
| ^^^^^^^^^^^^^^ PLC0415
11 | import time as thyme # [import-outside-toplevel]
12 | import random as rand, socket as sock # [import-outside-toplevel]
|
import_outside_top_level_with_banned.py:11:5: PLC0415 `import` should be at the top-level of a file
|
9 | import symtable # [import-outside-toplevel]
10 | import os, sys # [import-outside-toplevel]
11 | import time as thyme # [import-outside-toplevel]
| ^^^^^^^^^^^^^^^^^^^^ PLC0415
12 | import random as rand, socket as sock # [import-outside-toplevel]
13 | from collections import defaultdict # [import-outside-toplevel]
|
import_outside_top_level_with_banned.py:12:5: PLC0415 `import` should be at the top-level of a file
|
10 | import os, sys # [import-outside-toplevel]
11 | import time as thyme # [import-outside-toplevel]
12 | import random as rand, socket as sock # [import-outside-toplevel]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ PLC0415
13 | from collections import defaultdict # [import-outside-toplevel]
14 | from math import sin as sign, cos as cosplay # [import-outside-toplevel]
|
import_outside_top_level_with_banned.py:13:5: PLC0415 `import` should be at the top-level of a file
|
11 | import time as thyme # [import-outside-toplevel]
12 | import random as rand, socket as sock # [import-outside-toplevel]
13 | from collections import defaultdict # [import-outside-toplevel]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ PLC0415
14 | from math import sin as sign, cos as cosplay # [import-outside-toplevel]
|
import_outside_top_level_with_banned.py:14:5: PLC0415 `import` should be at the top-level of a file
|
12 | import random as rand, socket as sock # [import-outside-toplevel]
13 | from collections import defaultdict # [import-outside-toplevel]
14 | from math import sin as sign, cos as cosplay # [import-outside-toplevel]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ PLC0415
15 |
16 | # these should be allowed due to TID253 top-level ban
|
import_outside_top_level_with_banned.py:24:5: PLC0415 `import` should be at the top-level of a file
|
23 | # this should still trigger an error due to multiple imports
24 | from pkg import foo_allowed, bar_banned # [import-outside-toplevel]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ PLC0415
25 |
26 | class ClassWithImports:
|
import_outside_top_level_with_banned.py:27:5: PLC0415 `import` should be at the top-level of a file
|
26 | class ClassWithImports:
27 | import tokenize # [import-outside-toplevel]
| ^^^^^^^^^^^^^^^ PLC0415
28 |
29 | def __init__(self):
|
import_outside_top_level_with_banned.py:30:9: PLC0415 `import` should be at the top-level of a file
|
29 | def __init__(self):
30 | import trace # [import-outside-toplevel]
| ^^^^^^^^^^^^ PLC0415
31 |
32 | # these should be allowed due to TID253 top-level ban
|
import_outside_top_level_with_banned.py:40:9: PLC0415 `import` should be at the top-level of a file
|
39 | # this should still trigger an error due to multiple imports
40 | from pkg import foo_allowed, bar_banned # [import-outside-toplevel]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ PLC0415
|

View File

@@ -6,6 +6,7 @@ use ruff_python_ast::{Arguments, Expr, ExprCall, ExprNumberLiteral, Number};
use ruff_python_semantic::analyze::type_inference::{NumberLike, PythonType, ResolvedPythonType};
use ruff_python_semantic::analyze::typing;
use ruff_python_semantic::SemanticModel;
use ruff_source_file::find_newline;
use ruff_text_size::Ranged;
/// ## What it does
@@ -59,7 +60,7 @@ pub(crate) fn unnecessary_round(checker: &mut Checker, call: &ExprCall) {
return;
}
let applicability = match rounded_value {
let mut applicability = match rounded_value {
// ```python
// some_int: int
//
@@ -86,6 +87,10 @@ pub(crate) fn unnecessary_round(checker: &mut Checker, call: &ExprCall) {
_ => return,
};
if checker.comment_ranges().intersects(call.range()) {
applicability = Applicability::Unsafe;
};
let edit = unwrap_round_call(call, rounded, checker.semantic(), checker.locator());
let fix = Fix::applicable_edit(edit, applicability);
@@ -196,13 +201,13 @@ fn unwrap_round_call(
locator: &Locator,
) -> Edit {
let rounded_expr = locator.slice(rounded.range());
let has_parent_expr = semantic.current_expression_parent().is_some();
let new_content = if has_parent_expr || rounded.is_named_expr() {
format!("({rounded_expr})")
} else {
rounded_expr.to_string()
};
let new_content =
if has_parent_expr || rounded.is_named_expr() || find_newline(rounded_expr).is_some() {
format!("({rounded_expr})")
} else {
rounded_expr.to_string()
};
Edit::range_replacement(new_content, call.range)
}

View File

@@ -1,6 +1,5 @@
---
source: crates/ruff_linter/src/rules/ruff/mod.rs
snapshot_kind: text
---
RUF057.py:6:1: RUF057 [*] Value being rounded is already an integer
|
@@ -181,3 +180,94 @@ RUF057.py:44:1: RUF057 [*] Value being rounded is already an integer
45 45 | round(inferred_int, -2) # No error
46 46 | round(inferred_int, inferred_int) # No error
47 47 | round(inferred_int, 3 + 4) # No error
RUF057.py:71:1: RUF057 [*] Value being rounded is already an integer
|
69 | # contains newline.
70 | # See https://github.com/astral-sh/ruff/issues/15598
71 | / round(-
72 | | 1)
| |__^ RUF057
73 | round(1
74 | *1
|
= help: Remove unnecessary `round` call
Safe fix
68 68 | # Fixes should preserve parentheses when argument
69 69 | # contains newline.
70 70 | # See https://github.com/astral-sh/ruff/issues/15598
71 |-round(-
71 |+(-
72 72 | 1)
73 73 | round(1
74 74 | *1
RUF057.py:73:1: RUF057 [*] Value being rounded is already an integer
|
71 | round(-
72 | 1)
73 | / round(1
74 | | *1
75 | | )
| |_^ RUF057
76 |
77 | # fix should be unsafe if comment is in call range
|
= help: Remove unnecessary `round` call
Safe fix
70 70 | # See https://github.com/astral-sh/ruff/issues/15598
71 71 | round(-
72 72 | 1)
73 |-round(1
74 |-*1
75 |-)
73 |+(1
74 |+*1)
76 75 |
77 76 | # fix should be unsafe if comment is in call range
78 77 | round(# a comment
RUF057.py:78:1: RUF057 [*] Value being rounded is already an integer
|
77 | # fix should be unsafe if comment is in call range
78 | / round(# a comment
79 | | 17
80 | | )
| |_^ RUF057
81 | round(
82 | 17 # a comment
|
= help: Remove unnecessary `round` call
Unsafe fix
75 75 | )
76 76 |
77 77 | # fix should be unsafe if comment is in call range
78 |-round(# a comment
79 78 | 17
80 |-)
81 79 | round(
82 80 | 17 # a comment
83 81 | )
RUF057.py:81:1: RUF057 [*] Value being rounded is already an integer
|
79 | 17
80 | )
81 | / round(
82 | | 17 # a comment
83 | | )
| |_^ RUF057
|
= help: Remove unnecessary `round` call
Unsafe fix
78 78 | round(# a comment
79 79 | 17
80 80 | )
81 |-round(
82 |- 17 # a comment
83 |-)
81 |+17

View File

@@ -8,13 +8,14 @@ your project. For a more detailed overview, see [_Configuring Ruff_](configurati
To start, we'll initialize a project using [uv](https://docs.astral.sh/uv/):
```console
$ uv init numbers
$ uv init --lib numbers
```
This command creates a Python project with the following structure:
```text
numbers
├── README.md
├── pyproject.toml
└── src
└── numbers

1
ruff.schema.json generated
View File

@@ -3653,7 +3653,6 @@
"PLW0",
"PLW01",
"PLW010",
"PLW0101",
"PLW0108",
"PLW012",
"PLW0120",