Compare commits

...

23 Commits

Author SHA1 Message Date
Douglas Creager
e5befdf8ff Remove SuppressionsIter 2025-01-24 18:09:01 -05:00
Douglas Creager
a5f5aa4a6b Remove AncestorsIter 2025-01-24 17:48:32 -05:00
Douglas Creager
3dfcf91a1e Remove DescendentsIter and ChildrenIter 2025-01-24 17:44:08 -05:00
Douglas Creager
0adea712ef Remove DeclarationIdIterator and friends 2025-01-24 17:33:02 -05:00
Douglas Creager
ca5a6eef69 Remove ConstraintIdIterator 2025-01-24 17:28:09 -05:00
Douglas Creager
c2060f601f Remove ConstraintsIterator 2025-01-24 17:27:47 -05:00
Douglas Creager
8abc134582 Remove BindingIdWithConstraintsIterator 2025-01-24 17:19:57 -05:00
Douglas Creager
6f5ff25876 Remove BindingWithConstraintsIterator 2025-01-24 17:15:04 -05:00
Douglas Creager
35578672a4 Remove DeclarationsIterator 2025-01-24 17:14:33 -05:00
Zanie Blue
fcd0f349f9 Improve the file watching failure error message (#15728)
I really misunderstood this in
https://github.com/astral-sh/ruff/pull/15664#issuecomment-2613079710
2025-01-24 15:28:30 -06:00
Douglas Creager
5a9d71a5f1 Speed symbol state merging back up (#15731)
This is a follow-up to #15702 that hopefully claws back the 1%
performance regression. Assuming it works, the trick is to iterate over
the constraints vectors via mut reference (aka a single pointer), so
that we're not copying `BitSet`s into and out of the zip tuples as we
iterate. We use `std::mem::take` as a poor-man's move constructor only
at the very end, when we're ready to emplace it into the result. (C++
idioms intended! 😄)

With local testing via hyperfine, I'm seeing this be 1-3% faster than
`main` most of the time — though a small number of runs (1 in 10,
maybe?) are a wash or have `main` faster. Codspeed reports a 2%
gain.
2025-01-24 16:07:31 -05:00
Micha Reiser
9353482a5a Add check command (#15692) 2025-01-24 17:00:30 +01:00
Douglas Creager
716b246cf3 [red-knot] Use itertools to clean up SymbolState::merge (#15702)
[`merge_join_by`](https://docs.rs/itertools/latest/itertools/trait.Itertools.html#method.merge_join_by)
handles the "merge two sorted iterators" bit, and `zip` handles
iterating through the bindings/definitions along with their associated
constraints.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2025-01-24 10:21:29 -05:00
Micha Reiser
4e3982cf95 [red-knot] Add --ignore, --warn, and --error CLI arguments (#15689) 2025-01-24 16:20:15 +01:00
Charlie Marsh
ab2e1905c4 Use uv init --lib in tutorial (#15718)
## Summary

Closes https://github.com/astral-sh/uv/issues/10933.
2025-01-24 14:53:20 +00:00
David Peter
1feb3cf41a [red-knot] Use Unknown | T_inferred for undeclared public symbols (#15674)
## Summary

Use `Unknown | T_inferred` as the type for *undeclared* public symbols.

## Test Plan

- Updated existing tests
- New test for external `__slots__` modifications.
- New tests for external modifications of public symbols.
2025-01-24 12:47:48 +01:00
Dylan
7778d1d646 [ruff] Parenthesize fix when argument spans multiple lines for unnecessary-round (RUF057) (#15703) 2025-01-24 04:34:56 -06:00
David Peter
fb58a9b610 [red-knot] Rename TestDbBuilder::typeshed to .custom_typeshed (#15712)
## Summary

Correcting a small oversight by me
(https://github.com/astral-sh/ruff/pull/15683#discussion_r1926830914).
2025-01-24 10:25:23 +00:00
Mike Perlov
17a8a55f08 Honor banned top level imports by TID253 in PLC0415. (#15628)
Co-authored-by: Micha Reiser <micha@reiser.io>
2025-01-24 11:07:21 +01:00
Dhruv Manilawala
99d8ec6769 Apply AIR302-context check only in @task function (#15711)
This PR updates `AIR302` to only apply the context keys check in `@task`
decorated function.

Ref: https://github.com/astral-sh/ruff/pull/15144
2025-01-24 07:30:35 +00:00
Ankit Chaurasia
34cc3cab98 [airflow] Update AIR302 to check for deprecated context keys (#15144)
**Summary**

Airflow 3.0 removes a set of deprecated context variables that were
phased out in 2.x. This PR introduces lint rules to detect usage of
these removed variables in various patterns, helping identify
incompatibilities. The removed context variables include:

```
conf
execution_date
next_ds
next_ds_nodash
next_execution_date
prev_ds
prev_ds_nodash
prev_execution_date
prev_execution_date_success
tomorrow_ds
yesterday_ds
yesterday_ds_nodash
```

**Detected Patterns and Examples**

The linter now flags the use of removed context variables in the
following scenarios:

1. **Direct Subscript Access**  
   ```python
   execution_date = context["execution_date"]  # Flagged
   ```
   
2. **`.get("key")` Method Calls**  
   ```python
   print(context.get("execution_date"))  # Flagged
   ```
   
3. **Variables Assigned from `get_current_context()`**  
If a variable is assigned from `get_current_context()` and then used to
access a removed key:
   ```python
   c = get_current_context()
   print(c.get("execution_date"))  # Flagged
   ```
   
4. **Function Parameters in `@task`-Decorated Functions**  
Parameters named after removed context variables in functions decorated
with `@task` are flagged:
   ```python
   from airflow.decorators import task
   
   @task
def my_task(execution_date, **kwargs): # Parameter 'execution_date'
flagged
       pass
   ```
   
5. **Removed Keys in Task Decorator `kwargs` and Other Scenarios**  
Other similar patterns where removed context variables appear (e.g., as
part of `kwargs` in a `@task` function) are also detected.
```
from airflow.decorators import task

@task
def process_with_execution_date(**context):
    execution_date = lambda: context["execution_date"]  # flagged
    print(execution_date)

@task(kwargs={"execution_date": "2021-01-01"})   # flagged
def task_with_kwargs(**context):  
    pass
```

**Test Plan**

Test fixtures covering various patterns of deprecated context usage are
included in this PR. For example:

```python
from airflow.decorators import task, dag, get_current_context
from airflow.models import DAG
from airflow.operators.dummy import DummyOperator
import pendulum
from datetime import datetime

@task
def access_invalid_key_task(**context):
    print(context.get("conf"))  # 'conf' flagged

@task
def print_config(**context):
    execution_date = context["execution_date"]  # Flagged
    prev_ds = context["prev_ds"]                # Flagged

@task
def from_current_context():
    context = get_current_context()
    print(context["execution_date"])            # Flagged

# Usage outside of a task decorated function
c = get_current_context()
print(c.get("execution_date"))                 # Flagged

@task
def some_task(execution_date, **kwargs):
    print("execution date", execution_date)     # Parameter flagged

@dag(
    start_date=pendulum.datetime(2021, 1, 1, tz="UTC")
)
def my_dag():
    task1 = DummyOperator(
        task_id="task1",
        params={
            "execution_date": "{{ execution_date }}",  # Flagged in template context
        },
    )

    access_invalid_key_task()
    print_config()
    from_current_context()
    
dag = my_dag()

class CustomOperator(BaseOperator):
    def execute(self, context):
        execution_date = context.get("execution_date")                      # Flagged
        next_ds = context.get("next_ds")                                               # Flagged
        next_execution_date = context["next_execution_date"]          # Flagged
```

Ruff will emit `AIR302` diagnostics for each deprecated usage, with
suggestions when applicable, aiding in code migration to Airflow 3.0.

related: https://github.com/apache/airflow/issues/44409,
https://github.com/apache/airflow/issues/41641

---------

Co-authored-by: Wei Lee <weilee.rx@gmail.com>
2025-01-24 11:25:05 +05:30
Dhruv Manilawala
9384ba4b91 Remove test rules from JSON schema (#15627)
Closes: #15707
2025-01-24 10:17:59 +05:30
Dylan
2b3550c85f Add two missing commits to changelog (#15701)
Some commits appeared between the creation and merging of the "bump to
v0.9.3" branch.

Also made the same changes to the Releases on githhub.
2025-01-23 15:32:00 -06:00
59 changed files with 2075 additions and 897 deletions

2
.gitignore vendored
View File

@@ -30,7 +30,7 @@ tracing-flamechart.svg
tracing-flamegraph.svg
# insta
.rs.pending-snap
*.rs.pending-snap
###

View File

@@ -21,6 +21,7 @@
- \[`flake8-bugbear`\] Do not raise error if keyword argument is present and target-python version is less or equals than 3.9 (`B903`) ([#15549](https://github.com/astral-sh/ruff/pull/15549))
- \[`flake8-comprehensions`\] strip parentheses around generators in `unnecessary-generator-set` (`C401`) ([#15553](https://github.com/astral-sh/ruff/pull/15553))
- \[`flake8-pytest-style`\] Rewrite references to `.exception` (`PT027`) ([#15680](https://github.com/astral-sh/ruff/pull/15680))
- \[`flake8-simplify`\] Mark fixes as unsafe (`SIM201`, `SIM202`) ([#15626](https://github.com/astral-sh/ruff/pull/15626))
- \[`flake8-type-checking`\] Fix some safe fixes being labeled unsafe (`TC006`,`TC008`) ([#15638](https://github.com/astral-sh/ruff/pull/15638))
- \[`isort`\] Omit trailing whitespace in `unsorted-imports` (`I001`) ([#15518](https://github.com/astral-sh/ruff/pull/15518))
@@ -47,11 +48,12 @@
### Bug fixes
- \[`flake8-bandit`\] Add missing single-line/dotall regex flag (`S608`) ([#15654](https://github.com/astral-sh/ruff/pull/15654))
- \[`flake8-import-conventions`\] Fix infinite loop between `ICN001` and `I002` (`ICN001`) ([#15480](https://github.com/astral-sh/ruff/pull/15480))
- \[`flake8-simplify`\] Do not emit diagnostics for expressions inside string type annotations (`SIM222`, `SIM223`) ([#15405](https://github.com/astral-sh/ruff/pull/15405))
- \[`pyflakes`\] Treat arguments passed to the `default=` parameter of `TypeVar` as type expressions (`F821`) ([#15679](https://github.com/astral-sh/ruff/pull/15679))
- \[`pyupgrade`\] Avoid syntax error when the iterable is a non-parenthesized tuple (`UP028`) ([#15543](https://github.com/astral-sh/ruff/pull/15543))
- \[`ruff`\] Exempt `NewType` calls where the original type is immutable (`RUF009`) ([#15588](https://github.com/astral-sh/ruff/pull/15588))
- \[`unconventional-import-alias`\] Fix infinite loop between `ICN001` and `I002` (`ICN001`) ([#15480](https://github.com/astral-sh/ruff/pull/15480))
- Preserve raw string prefix and escapes in all codegen fixes ([#15694](https://github.com/astral-sh/ruff/pull/15694))
### Documentation

190
crates/red_knot/src/args.rs Normal file
View File

@@ -0,0 +1,190 @@
use crate::logging::Verbosity;
use crate::python_version::PythonVersion;
use clap::{ArgAction, ArgMatches, Error, Parser};
use red_knot_project::metadata::options::{EnvironmentOptions, Options};
use red_knot_project::metadata::value::{RangedValue, RelativePathBuf};
use red_knot_python_semantic::lint;
use ruff_db::system::SystemPathBuf;
#[derive(Debug, Parser)]
#[command(
author,
name = "red-knot",
about = "An extremely fast Python type checker."
)]
#[command(version)]
pub(crate) struct Args {
#[command(subcommand)]
pub(crate) command: Command,
}
#[derive(Debug, clap::Subcommand)]
pub(crate) enum Command {
/// Check a project for type errors.
Check(CheckCommand),
/// Start the language server
Server,
}
#[derive(Debug, Parser)]
pub(crate) struct CheckCommand {
/// Run the command within the given project directory.
///
/// All `pyproject.toml` files will be discovered by walking up the directory tree from the given project directory,
/// as will the project's virtual environment (`.venv`) unless the `venv-path` option is set.
///
/// Other command-line arguments (such as relative paths) will be resolved relative to the current working directory.
#[arg(long, value_name = "PROJECT")]
pub(crate) project: Option<SystemPathBuf>,
/// Path to the virtual environment the project uses.
///
/// If provided, red-knot will use the `site-packages` directory of this virtual environment
/// to resolve type information for the project's third-party dependencies.
#[arg(long, value_name = "PATH")]
pub(crate) venv_path: Option<SystemPathBuf>,
/// Custom directory to use for stdlib typeshed stubs.
#[arg(long, value_name = "PATH", alias = "custom-typeshed-dir")]
pub(crate) typeshed: Option<SystemPathBuf>,
/// Additional path to use as a module-resolution source (can be passed multiple times).
#[arg(long, value_name = "PATH")]
pub(crate) extra_search_path: Option<Vec<SystemPathBuf>>,
/// Python version to assume when resolving types.
#[arg(long, value_name = "VERSION", alias = "target-version")]
pub(crate) python_version: Option<PythonVersion>,
#[clap(flatten)]
pub(crate) verbosity: Verbosity,
#[clap(flatten)]
pub(crate) rules: RulesArg,
/// Run in watch mode by re-running whenever files change.
#[arg(long, short = 'W')]
pub(crate) watch: bool,
}
impl CheckCommand {
pub(crate) fn into_options(self) -> Options {
let rules = if self.rules.is_empty() {
None
} else {
Some(
self.rules
.into_iter()
.map(|(rule, level)| (RangedValue::cli(rule), RangedValue::cli(level)))
.collect(),
)
};
Options {
environment: Some(EnvironmentOptions {
python_version: self
.python_version
.map(|version| RangedValue::cli(version.into())),
venv_path: self.venv_path.map(RelativePathBuf::cli),
typeshed: self.typeshed.map(RelativePathBuf::cli),
extra_paths: self.extra_search_path.map(|extra_search_paths| {
extra_search_paths
.into_iter()
.map(RelativePathBuf::cli)
.collect()
}),
..EnvironmentOptions::default()
}),
rules,
..Default::default()
}
}
}
/// A list of rules to enable or disable with a given severity.
///
/// This type is used to parse the `--error`, `--warn`, and `--ignore` arguments
/// while preserving the order in which they were specified (arguments last override previous severities).
#[derive(Debug)]
pub(crate) struct RulesArg(Vec<(String, lint::Level)>);
impl RulesArg {
fn is_empty(&self) -> bool {
self.0.is_empty()
}
fn into_iter(self) -> impl Iterator<Item = (String, lint::Level)> {
self.0.into_iter()
}
}
impl clap::FromArgMatches for RulesArg {
fn from_arg_matches(matches: &ArgMatches) -> Result<Self, Error> {
let mut rules = Vec::new();
for (level, arg_id) in [
(lint::Level::Ignore, "ignore"),
(lint::Level::Warn, "warn"),
(lint::Level::Error, "error"),
] {
let indices = matches.indices_of(arg_id).into_iter().flatten();
let levels = matches.get_many::<String>(arg_id).into_iter().flatten();
rules.extend(
indices
.zip(levels)
.map(|(index, rule)| (index, rule, level)),
);
}
// Sort by their index so that values specified later override earlier ones.
rules.sort_by_key(|(index, _, _)| *index);
Ok(Self(
rules
.into_iter()
.map(|(_, rule, level)| (rule.to_owned(), level))
.collect(),
))
}
fn update_from_arg_matches(&mut self, matches: &ArgMatches) -> Result<(), Error> {
self.0 = Self::from_arg_matches(matches)?.0;
Ok(())
}
}
impl clap::Args for RulesArg {
fn augment_args(cmd: clap::Command) -> clap::Command {
const HELP_HEADING: &str = "Enabling / disabling rules";
cmd.arg(
clap::Arg::new("error")
.long("error")
.action(ArgAction::Append)
.help("Treat the given rule as having severity 'error'. Can be specified multiple times.")
.value_name("RULE")
.help_heading(HELP_HEADING),
)
.arg(
clap::Arg::new("warn")
.long("warn")
.action(ArgAction::Append)
.help("Treat the given rule as having severity 'warn'. Can be specified multiple times.")
.value_name("RULE")
.help_heading(HELP_HEADING),
)
.arg(
clap::Arg::new("ignore")
.long("ignore")
.action(ArgAction::Append)
.help("Disables the rule. Can be specified multiple times.")
.value_name("RULE")
.help_heading(HELP_HEADING),
)
}
fn augment_args_for_update(cmd: clap::Command) -> clap::Command {
Self::augment_args(cmd)
}
}

View File

@@ -1,13 +1,13 @@
use std::process::{ExitCode, Termination};
use std::sync::Mutex;
use crate::args::{Args, CheckCommand, Command};
use crate::logging::setup_tracing;
use anyhow::{anyhow, Context};
use clap::Parser;
use colored::Colorize;
use crossbeam::channel as crossbeam_channel;
use python_version::PythonVersion;
use red_knot_project::metadata::options::{EnvironmentOptions, Options};
use red_knot_project::metadata::value::{RangedValue, RelativePathBuf};
use red_knot_project::metadata::options::Options;
use red_knot_project::watch;
use red_knot_project::watch::ProjectWatcher;
use red_knot_project::{ProjectDatabase, ProjectMetadata};
@@ -16,87 +16,11 @@ use ruff_db::diagnostic::Diagnostic;
use ruff_db::system::{OsSystem, System, SystemPath, SystemPathBuf};
use salsa::plumbing::ZalsaDatabase;
use crate::logging::{setup_tracing, Verbosity};
mod args;
mod logging;
mod python_version;
mod verbosity;
#[derive(Debug, Parser)]
#[command(
author,
name = "red-knot",
about = "An extremely fast Python type checker."
)]
#[command(version)]
struct Args {
#[command(subcommand)]
pub(crate) command: Option<Command>,
/// Run the command within the given project directory.
///
/// All `pyproject.toml` files will be discovered by walking up the directory tree from the given project directory,
/// as will the project's virtual environment (`.venv`) unless the `venv-path` option is set.
///
/// Other command-line arguments (such as relative paths) will be resolved relative to the current working directory.
#[arg(long, value_name = "PROJECT")]
project: Option<SystemPathBuf>,
/// Path to the virtual environment the project uses.
///
/// If provided, red-knot will use the `site-packages` directory of this virtual environment
/// to resolve type information for the project's third-party dependencies.
#[arg(long, value_name = "PATH")]
venv_path: Option<SystemPathBuf>,
/// Custom directory to use for stdlib typeshed stubs.
#[arg(long, value_name = "PATH", alias = "custom-typeshed-dir")]
typeshed: Option<SystemPathBuf>,
/// Additional path to use as a module-resolution source (can be passed multiple times).
#[arg(long, value_name = "PATH")]
extra_search_path: Option<Vec<SystemPathBuf>>,
/// Python version to assume when resolving types.
#[arg(long, value_name = "VERSION", alias = "target-version")]
python_version: Option<PythonVersion>,
#[clap(flatten)]
verbosity: Verbosity,
/// Run in watch mode by re-running whenever files change.
#[arg(long, short = 'W')]
watch: bool,
}
impl Args {
fn to_options(&self) -> Options {
Options {
environment: Some(EnvironmentOptions {
python_version: self
.python_version
.map(|version| RangedValue::cli(version.into())),
venv_path: self.venv_path.as_ref().map(RelativePathBuf::cli),
typeshed: self.typeshed.as_ref().map(RelativePathBuf::cli),
extra_paths: self.extra_search_path.as_ref().map(|extra_search_paths| {
extra_search_paths
.iter()
.map(RelativePathBuf::cli)
.collect()
}),
..EnvironmentOptions::default()
}),
..Default::default()
}
}
}
#[derive(Debug, clap::Subcommand)]
pub enum Command {
/// Start the language server
Server,
}
#[allow(clippy::print_stdout, clippy::unnecessary_wraps, clippy::print_stderr)]
pub fn main() -> ExitStatus {
run().unwrap_or_else(|error| {
@@ -122,10 +46,13 @@ pub fn main() -> ExitStatus {
fn run() -> anyhow::Result<ExitStatus> {
let args = Args::parse_from(std::env::args());
if matches!(args.command, Some(Command::Server)) {
return run_server().map(|()| ExitStatus::Success);
match args.command {
Command::Server => run_server().map(|()| ExitStatus::Success),
Command::Check(check_args) => run_check(check_args),
}
}
fn run_check(args: CheckCommand) -> anyhow::Result<ExitStatus> {
let verbosity = args.verbosity.level();
countme::enable(verbosity.is_trace());
let _guard = setup_tracing(verbosity)?;
@@ -156,7 +83,8 @@ fn run() -> anyhow::Result<ExitStatus> {
.unwrap_or_else(|| cli_base_path.clone());
let system = OsSystem::new(cwd);
let cli_options = args.to_options();
let watch = args.watch;
let cli_options = args.into_options();
let mut workspace_metadata = ProjectMetadata::discover(system.current_directory(), &system)?;
workspace_metadata.apply_cli_options(cli_options.clone());
@@ -174,7 +102,7 @@ fn run() -> anyhow::Result<ExitStatus> {
}
})?;
let exit_status = if args.watch {
let exit_status = if watch {
main_loop.watch(&mut db)?
} else {
main_loop.run(&mut db)

View File

@@ -1,5 +1,5 @@
use anyhow::Context;
use insta::Settings;
use insta::internals::SettingsBindDropGuard;
use insta_cmd::{assert_cmd_snapshot, get_cargo_bin};
use std::path::{Path, PathBuf};
use std::process::Command;
@@ -28,24 +28,22 @@ fn config_override() -> anyhow::Result<()> {
),
])?;
case.insta_settings().bind(|| {
assert_cmd_snapshot!(case.command(), @r"
success: false
exit_code: 1
----- stdout -----
error[lint:unresolved-attribute] <temp_dir>/test.py:5:7 Type `<module 'sys'>` has no attribute `last_exc`
assert_cmd_snapshot!(case.command(), @r"
success: false
exit_code: 1
----- stdout -----
error[lint:unresolved-attribute] <temp_dir>/test.py:5:7 Type `<module 'sys'>` has no attribute `last_exc`
----- stderr -----
");
----- stderr -----
");
assert_cmd_snapshot!(case.command().arg("--python-version").arg("3.12"), @r"
success: true
exit_code: 0
----- stdout -----
assert_cmd_snapshot!(case.command().arg("--python-version").arg("3.12"), @r"
success: true
exit_code: 0
----- stdout -----
----- stderr -----
");
});
----- stderr -----
");
Ok(())
}
@@ -92,25 +90,23 @@ fn cli_arguments_are_relative_to_the_current_directory() -> anyhow::Result<()> {
),
])?;
case.insta_settings().bind(|| {
// Make sure that the CLI fails when the `libs` directory is not in the search path.
assert_cmd_snapshot!(case.command().current_dir(case.project_dir().join("child")), @r#"
success: false
exit_code: 1
----- stdout -----
error[lint:unresolved-import] <temp_dir>/child/test.py:2:1 Cannot resolve import `utils`
// Make sure that the CLI fails when the `libs` directory is not in the search path.
assert_cmd_snapshot!(case.command().current_dir(case.project_dir().join("child")), @r#"
success: false
exit_code: 1
----- stdout -----
error[lint:unresolved-import] <temp_dir>/child/test.py:2:1 Cannot resolve import `utils`
----- stderr -----
"#);
----- stderr -----
"#);
assert_cmd_snapshot!(case.command().current_dir(case.project_dir().join("child")).arg("--extra-search-path").arg("../libs"), @r"
success: true
exit_code: 0
----- stdout -----
assert_cmd_snapshot!(case.command().current_dir(case.project_dir().join("child")).arg("--extra-search-path").arg("../libs"), @r"
success: true
exit_code: 0
----- stdout -----
----- stderr -----
");
});
----- stderr -----
");
Ok(())
}
@@ -156,22 +152,20 @@ fn paths_in_configuration_files_are_relative_to_the_project_root() -> anyhow::Re
),
])?;
case.insta_settings().bind(|| {
assert_cmd_snapshot!(case.command().current_dir(case.project_dir().join("child")), @r"
success: true
exit_code: 0
----- stdout -----
assert_cmd_snapshot!(case.command().current_dir(case.project_dir().join("child")), @r"
success: true
exit_code: 0
----- stdout -----
----- stderr -----
");
});
----- stderr -----
");
Ok(())
}
/// The rule severity can be changed in the configuration file
#[test]
fn rule_severity() -> anyhow::Result<()> {
fn configuration_rule_severity() -> anyhow::Result<()> {
let case = TestCase::with_file(
"test.py",
r#"
@@ -184,41 +178,146 @@ fn rule_severity() -> anyhow::Result<()> {
"#,
)?;
case.insta_settings().bind(|| {
// Assert that there's a possibly unresolved reference diagnostic
// and that division-by-zero has a severity of error by default.
assert_cmd_snapshot!(case.command(), @r"
success: false
exit_code: 1
----- stdout -----
error[lint:division-by-zero] <temp_dir>/test.py:2:5 Cannot divide object of type `Literal[4]` by zero
warning[lint:possibly-unresolved-reference] <temp_dir>/test.py:7:7 Name `x` used when possibly not defined
// Assert that there's a possibly unresolved reference diagnostic
// and that division-by-zero has a severity of error by default.
assert_cmd_snapshot!(case.command(), @r"
success: false
exit_code: 1
----- stdout -----
error[lint:division-by-zero] <temp_dir>/test.py:2:5 Cannot divide object of type `Literal[4]` by zero
warning[lint:possibly-unresolved-reference] <temp_dir>/test.py:7:7 Name `x` used when possibly not defined
----- stderr -----
");
----- stderr -----
");
case.write_file("pyproject.toml", r#"
[tool.knot.rules]
division-by-zero = "warn" # demote to warn
possibly-unresolved-reference = "ignore"
"#)?;
case.write_file(
"pyproject.toml",
r#"
[tool.knot.rules]
division-by-zero = "warn" # demote to warn
possibly-unresolved-reference = "ignore"
"#,
)?;
assert_cmd_snapshot!(case.command(), @r"
success: false
exit_code: 1
----- stdout -----
warning[lint:division-by-zero] <temp_dir>/test.py:2:5 Cannot divide object of type `Literal[4]` by zero
assert_cmd_snapshot!(case.command(), @r"
success: false
exit_code: 1
----- stdout -----
warning[lint:division-by-zero] <temp_dir>/test.py:2:5 Cannot divide object of type `Literal[4]` by zero
----- stderr -----
");
----- stderr -----
");
Ok(())
})
Ok(())
}
/// Red Knot warns about unknown rules
/// The rule severity can be changed using `--ignore`, `--warn`, and `--error`
#[test]
fn unknown_rules() -> anyhow::Result<()> {
fn cli_rule_severity() -> anyhow::Result<()> {
let case = TestCase::with_file(
"test.py",
r#"
import does_not_exit
y = 4 / 0
for a in range(0, y):
x = a
print(x) # possibly-unresolved-reference
"#,
)?;
// Assert that there's a possibly unresolved reference diagnostic
// and that division-by-zero has a severity of error by default.
assert_cmd_snapshot!(case.command(), @r"
success: false
exit_code: 1
----- stdout -----
error[lint:unresolved-import] <temp_dir>/test.py:2:8 Cannot resolve import `does_not_exit`
error[lint:division-by-zero] <temp_dir>/test.py:4:5 Cannot divide object of type `Literal[4]` by zero
warning[lint:possibly-unresolved-reference] <temp_dir>/test.py:9:7 Name `x` used when possibly not defined
----- stderr -----
");
assert_cmd_snapshot!(
case
.command()
.arg("--ignore")
.arg("possibly-unresolved-reference")
.arg("--warn")
.arg("division-by-zero")
.arg("--warn")
.arg("unresolved-import"),
@r"
success: false
exit_code: 1
----- stdout -----
warning[lint:unresolved-import] <temp_dir>/test.py:2:8 Cannot resolve import `does_not_exit`
warning[lint:division-by-zero] <temp_dir>/test.py:4:5 Cannot divide object of type `Literal[4]` by zero
----- stderr -----
"
);
Ok(())
}
/// The rule severity can be changed using `--ignore`, `--warn`, and `--error` and
/// values specified last override previous severities.
#[test]
fn cli_rule_severity_precedence() -> anyhow::Result<()> {
let case = TestCase::with_file(
"test.py",
r#"
y = 4 / 0
for a in range(0, y):
x = a
print(x) # possibly-unresolved-reference
"#,
)?;
// Assert that there's a possibly unresolved reference diagnostic
// and that division-by-zero has a severity of error by default.
assert_cmd_snapshot!(case.command(), @r"
success: false
exit_code: 1
----- stdout -----
error[lint:division-by-zero] <temp_dir>/test.py:2:5 Cannot divide object of type `Literal[4]` by zero
warning[lint:possibly-unresolved-reference] <temp_dir>/test.py:7:7 Name `x` used when possibly not defined
----- stderr -----
");
assert_cmd_snapshot!(
case
.command()
.arg("--error")
.arg("possibly-unresolved-reference")
.arg("--warn")
.arg("division-by-zero")
// Override the error severity with warning
.arg("--ignore")
.arg("possibly-unresolved-reference"),
@r"
success: false
exit_code: 1
----- stdout -----
warning[lint:division-by-zero] <temp_dir>/test.py:2:5 Cannot divide object of type `Literal[4]` by zero
----- stderr -----
"
);
Ok(())
}
/// Red Knot warns about unknown rules specified in a configuration file
#[test]
fn configuration_unknown_rules() -> anyhow::Result<()> {
let case = TestCase::with_files([
(
"pyproject.toml",
@@ -230,22 +329,38 @@ fn unknown_rules() -> anyhow::Result<()> {
("test.py", "print(10)"),
])?;
case.insta_settings().bind(|| {
assert_cmd_snapshot!(case.command(), @r"
success: false
exit_code: 1
----- stdout -----
warning[unknown-rule] <temp_dir>/pyproject.toml:3:1 Unknown lint rule `division-by-zer`
assert_cmd_snapshot!(case.command(), @r"
success: false
exit_code: 1
----- stdout -----
warning[unknown-rule] <temp_dir>/pyproject.toml:3:1 Unknown lint rule `division-by-zer`
----- stderr -----
");
});
----- stderr -----
");
Ok(())
}
/// Red Knot warns about unknown rules specified in a CLI argument
#[test]
fn cli_unknown_rules() -> anyhow::Result<()> {
let case = TestCase::with_file("test.py", "print(10)")?;
assert_cmd_snapshot!(case.command().arg("--ignore").arg("division-by-zer"), @r"
success: false
exit_code: 1
----- stdout -----
warning[unknown-rule] Unknown lint rule `division-by-zer`
----- stderr -----
");
Ok(())
}
struct TestCase {
_temp_dir: TempDir,
_settings_scope: SettingsBindDropGuard,
project_dir: PathBuf,
}
@@ -260,9 +375,16 @@ impl TestCase {
.canonicalize()
.context("Failed to canonicalize project path")?;
let mut settings = insta::Settings::clone_current();
settings.add_filter(&tempdir_filter(&project_dir), "<temp_dir>/");
settings.add_filter(r#"\\(\w\w|\s|\.|")"#, "/$1");
let settings_scope = settings.bind_to_scope();
Ok(Self {
project_dir,
_temp_dir: temp_dir,
_settings_scope: settings_scope,
})
}
@@ -307,17 +429,9 @@ impl TestCase {
&self.project_dir
}
// Returns the insta filters to escape paths in snapshots
fn insta_settings(&self) -> Settings {
let mut settings = insta::Settings::clone_current();
settings.add_filter(&tempdir_filter(&self.project_dir), "<temp_dir>/");
settings.add_filter(r#"\\(\w\w|\s|\.|")"#, "/$1");
settings
}
fn command(&self) -> Command {
let mut command = Command::new(get_cargo_bin("red_knot"));
command.current_dir(&self.project_dir);
command.current_dir(&self.project_dir).arg("check");
command
}
}

View File

@@ -47,7 +47,7 @@ impl TestCase {
#[track_caller]
fn panic_with_formatted_events(events: Vec<ChangeEvent>) -> Vec<ChangeEvent> {
panic!(
"Didn't observe expected change:\n{}",
"Didn't observe the expected event. The following events occurred:\n{}",
events
.into_iter()
.map(|event| format!(" - {event:?}"))

View File

@@ -149,6 +149,16 @@ impl Options {
format!("Unknown lint rule `{rule_name}`"),
Severity::Warning,
),
GetLintError::PrefixedWithCategory { suggestion, .. } => {
OptionDiagnostic::new(
DiagnosticId::UnknownRule,
format!(
"Unknown lint rule `{rule_name}`. Did you mean `{suggestion}`?"
),
Severity::Warning,
)
}
GetLintError::Removed(_) => OptionDiagnostic::new(
DiagnosticId::UnknownRule,
format!("Unknown lint rule `{rule_name}`"),
@@ -206,6 +216,16 @@ pub struct Rules {
inner: FxHashMap<RangedValue<String>, RangedValue<Level>>,
}
impl FromIterator<(RangedValue<String>, RangedValue<Level>)> for Rules {
fn from_iter<T: IntoIterator<Item = (RangedValue<String>, RangedValue<Level>)>>(
iter: T,
) -> Self {
Self {
inner: iter.into_iter().collect(),
}
}
}
#[derive(Error, Debug)]
pub enum KnotTomlError {
#[error(transparent)]

View File

@@ -36,7 +36,7 @@ def f():
reveal_type(a7) # revealed: None
reveal_type(a8) # revealed: Literal[1]
# TODO: This should be Color.RED
reveal_type(b1) # revealed: Literal[0]
reveal_type(b1) # revealed: Unknown | Literal[0]
# error: [invalid-type-form]
invalid1: Literal[3 + 4]

View File

@@ -175,7 +175,7 @@ class C:
reveal_type(C.pure_class_variable1) # revealed: str
# TODO: this should be `Literal[1]`, or `Unknown | Literal[1]`.
# TODO: Should be `Unknown | Literal[1]`.
reveal_type(C.pure_class_variable2) # revealed: Unknown
c_instance = C()
@@ -252,8 +252,7 @@ class C:
reveal_type(C.variable_with_class_default1) # revealed: str
# TODO: this should be `Unknown | Literal[1]`.
reveal_type(C.variable_with_class_default2) # revealed: Literal[1]
reveal_type(C.variable_with_class_default2) # revealed: Unknown | Literal[1]
c_instance = C()
@@ -296,8 +295,8 @@ def _(flag: bool):
else:
x = 4
reveal_type(C1.x) # revealed: Literal[1, 2]
reveal_type(C2.x) # revealed: Literal[3, 4]
reveal_type(C1.x) # revealed: Unknown | Literal[1, 2]
reveal_type(C2.x) # revealed: Unknown | Literal[3, 4]
```
## Inherited class attributes
@@ -311,7 +310,7 @@ class A:
class B(A): ...
class C(B): ...
reveal_type(C.X) # revealed: Literal["foo"]
reveal_type(C.X) # revealed: Unknown | Literal["foo"]
```
### Multiple inheritance
@@ -334,7 +333,7 @@ class A(B, C): ...
reveal_type(A.__mro__)
# `E` is earlier in the MRO than `F`, so we should use the type of `E.X`
reveal_type(A.X) # revealed: Literal[42]
reveal_type(A.X) # revealed: Unknown | Literal[42]
```
## Unions with possibly unbound paths
@@ -356,7 +355,7 @@ def _(flag1: bool, flag2: bool):
C = C1 if flag1 else C2 if flag2 else C3
# error: [possibly-unbound-attribute] "Attribute `x` on type `Literal[C1, C2, C3]` is possibly unbound"
reveal_type(C.x) # revealed: Literal[1, 3]
reveal_type(C.x) # revealed: Unknown | Literal[1, 3]
```
### Possibly-unbound within a class
@@ -379,7 +378,7 @@ def _(flag: bool, flag1: bool, flag2: bool):
C = C1 if flag1 else C2 if flag2 else C3
# error: [possibly-unbound-attribute] "Attribute `x` on type `Literal[C1, C2, C3]` is possibly unbound"
reveal_type(C.x) # revealed: Literal[1, 2, 3]
reveal_type(C.x) # revealed: Unknown | Literal[1, 2, 3]
```
### Unions with all paths unbound

View File

@@ -262,7 +262,8 @@ class A:
class B:
__add__ = A()
reveal_type(B() + B()) # revealed: int
# TODO: this could be `int` if we declare `B.__add__` using a `Callable` type
reveal_type(B() + B()) # revealed: Unknown | int
```
## Integration test: numbers from typeshed

View File

@@ -5,6 +5,11 @@ that is, a use of a symbol from another scope. If a symbol has a declared type i
(e.g. `int`), we use that as the symbol's "public type" (the type of the symbol from the perspective
of other scopes) even if there is a more precise local inferred type for the symbol (`Literal[1]`).
If a symbol has no declared type, we use the union of `Unknown` with the inferred type as the public
type. If there is no declaration, then the symbol can be reassigned to any type from another scope;
the union with `Unknown` reflects that its type must at least be as large as the type of the
assigned value, but could be arbitrarily larger.
We test the whole matrix of possible boundness and declaredness states. The current behavior is
summarized in the following table, while the tests below demonstrate each case. Note that some of
this behavior is questionable and might change in the future. See the TODOs in `symbol_by_id`
@@ -12,11 +17,11 @@ this behavior is questionable and might change in the future. See the TODOs in `
In particular, we should raise errors in the "possibly-undeclared-and-unbound" as well as the
"undeclared-and-possibly-unbound" cases (marked with a "?").
| **Public type** | declared | possibly-undeclared | undeclared |
| ---------------- | ------------ | -------------------------- | ------------ |
| bound | `T_declared` | `T_declared \| T_inferred` | `T_inferred` |
| possibly-unbound | `T_declared` | `T_declared \| T_inferred` | `T_inferred` |
| unbound | `T_declared` | `T_declared` | `Unknown` |
| **Public type** | declared | possibly-undeclared | undeclared |
| ---------------- | ------------ | -------------------------- | ----------------------- |
| bound | `T_declared` | `T_declared \| T_inferred` | `Unknown \| T_inferred` |
| possibly-unbound | `T_declared` | `T_declared \| T_inferred` | `Unknown \| T_inferred` |
| unbound | `T_declared` | `T_declared` | `Unknown` |
| **Diagnostic** | declared | possibly-undeclared | undeclared |
| ---------------- | -------- | ------------------------- | ------------------- |
@@ -97,17 +102,24 @@ def flag() -> bool: ...
x = 1
y = 2
z = 3
if flag():
x: Any
x: int
y: Any
# error: [invalid-declaration]
y: str
z: str
```
```py
from mod import x, y
from mod import x, y, z
reveal_type(x) # revealed: Literal[1] | Any
reveal_type(y) # revealed: Literal[2] | Unknown
reveal_type(x) # revealed: int
reveal_type(y) # revealed: Literal[2] | Any
reveal_type(z) # revealed: Literal[3] | Unknown
# External modifications of `x` that violate the declared type are not allowed:
# error: [invalid-assignment]
x = None
```
### Possibly undeclared and possibly unbound
@@ -134,6 +146,10 @@ from mod import x, y
reveal_type(x) # revealed: Literal[1] | Any
reveal_type(y) # revealed: Literal[2] | str
# External modifications of `y` that violate the declared type are not allowed:
# error: [invalid-assignment]
y = None
```
### Possibly undeclared and unbound
@@ -154,14 +170,16 @@ if flag():
from mod import x
reveal_type(x) # revealed: int
# External modifications to `x` that violate the declared type are not allowed:
# error: [invalid-assignment]
x = None
```
## Undeclared
### Undeclared but bound
We use the inferred type as the public type, if a symbol has no declared type.
```py path=mod.py
x = 1
```
@@ -169,7 +187,10 @@ x = 1
```py
from mod import x
reveal_type(x) # revealed: Literal[1]
reveal_type(x) # revealed: Unknown | Literal[1]
# All external modifications of `x` are allowed:
x = None
```
### Undeclared and possibly unbound
@@ -189,7 +210,10 @@ if flag:
# on top of this document.
from mod import x
reveal_type(x) # revealed: Literal[1]
reveal_type(x) # revealed: Unknown | Literal[1]
# All external modifications of `x` are allowed:
x = None
```
### Undeclared and unbound
@@ -206,4 +230,7 @@ if False:
from mod import x
reveal_type(x) # revealed: Unknown
# Modifications allowed in this case:
x = None
```

View File

@@ -52,7 +52,7 @@ class NonCallable:
__call__ = 1
a = NonCallable()
# error: "Object of type `NonCallable` is not callable"
# error: "Object of type `Unknown | Literal[1]` is not callable (due to union element `Literal[1]`)"
reveal_type(a()) # revealed: Unknown
```

View File

@@ -43,7 +43,8 @@ class IntIterable:
def __iter__(self) -> IntIterator:
return IntIterator()
# revealed: tuple[int, int]
# TODO: This could be a `tuple[int, int]` if we model that `y` can not be modified in the outer comprehension scope
# revealed: tuple[int, Unknown | int]
[[reveal_type((x, y)) for x in IntIterable()] for y in IntIterable()]
```
@@ -66,7 +67,8 @@ class IterableOfIterables:
def __iter__(self) -> IteratorOfIterables:
return IteratorOfIterables()
# revealed: tuple[int, IntIterable]
# TODO: This could be a `tuple[int, int]` (see above)
# revealed: tuple[int, Unknown | IntIterable]
[[reveal_type((x, y)) for x in y] for y in IterableOfIterables()]
```

View File

@@ -5,7 +5,7 @@
```py
def _(flag: bool):
class A:
always_bound = 1
always_bound: int = 1
if flag:
union = 1
@@ -13,14 +13,21 @@ def _(flag: bool):
union = "abc"
if flag:
possibly_unbound = "abc"
union_declared: int = 1
else:
union_declared: str = "abc"
reveal_type(A.always_bound) # revealed: Literal[1]
if flag:
possibly_unbound: str = "abc"
reveal_type(A.union) # revealed: Literal[1, "abc"]
reveal_type(A.always_bound) # revealed: int
reveal_type(A.union) # revealed: Unknown | Literal[1, "abc"]
reveal_type(A.union_declared) # revealed: int | str
# error: [possibly-unbound-attribute] "Attribute `possibly_unbound` on type `Literal[A]` is possibly unbound"
reveal_type(A.possibly_unbound) # revealed: Literal["abc"]
reveal_type(A.possibly_unbound) # revealed: str
# error: [unresolved-attribute] "Type `Literal[A]` has no attribute `non_existent`"
reveal_type(A.non_existent) # revealed: Unknown

View File

@@ -55,7 +55,7 @@ reveal_type("x" or "y" and "") # revealed: Literal["x"]
## Evaluates to builtin
```py path=a.py
redefined_builtin_bool = bool
redefined_builtin_bool: type[bool] = bool
def my_bool(x) -> bool:
return True

View File

@@ -172,10 +172,10 @@ class IntUnion:
def __len__(self) -> Literal[SomeEnum.INT, SomeEnum.INT_2]: ...
reveal_type(len(Auto())) # revealed: int
reveal_type(len(Int())) # revealed: Literal[2]
reveal_type(len(Int())) # revealed: int
reveal_type(len(Str())) # revealed: int
reveal_type(len(Tuple())) # revealed: int
reveal_type(len(IntUnion())) # revealed: Literal[2, 32]
reveal_type(len(IntUnion())) # revealed: int
```
### Negative integers

View File

@@ -20,7 +20,7 @@ wrong_innards: MyBox[int] = MyBox("five")
# TODO reveal int, do not leak the typevar
reveal_type(box.data) # revealed: T
reveal_type(MyBox.box_model_number) # revealed: Literal[695]
reveal_type(MyBox.box_model_number) # revealed: Unknown | Literal[695]
```
## Subclassing

View File

@@ -23,8 +23,8 @@ reveal_type(y)
# error: [possibly-unbound-import] "Member `y` of module `maybe_unbound` is possibly unbound"
from maybe_unbound import x, y
reveal_type(x) # revealed: Literal[3]
reveal_type(y) # revealed: Literal[3]
reveal_type(x) # revealed: Unknown | Literal[3]
reveal_type(y) # revealed: Unknown | Literal[3]
```
## Maybe unbound annotated
@@ -52,7 +52,7 @@ Importing an annotated name prefers the declared type over the inferred type:
# error: [possibly-unbound-import] "Member `y` of module `maybe_unbound_annotated` is possibly unbound"
from maybe_unbound_annotated import x, y
reveal_type(x) # revealed: Literal[3]
reveal_type(x) # revealed: Unknown | Literal[3]
reveal_type(y) # revealed: int
```

View File

@@ -109,9 +109,9 @@ reveal_type(x)
def _(flag: bool):
class NotIterable:
if flag:
__iter__ = 1
__iter__: int = 1
else:
__iter__ = None
__iter__: None = None
for x in NotIterable(): # error: "Object of type `NotIterable` is not iterable"
pass
@@ -135,7 +135,7 @@ for x in nonsense: # error: "Object of type `Literal[123]` is not iterable"
class NotIterable:
def __getitem__(self, key: int) -> int:
return 42
__iter__ = None
__iter__: None = None
for x in NotIterable(): # error: "Object of type `NotIterable` is not iterable"
pass

View File

@@ -99,9 +99,9 @@ def _(x: str | int):
class A: ...
class B: ...
alias_for_type = type
def _(x: A | B):
alias_for_type = type
if alias_for_type(x) is A:
reveal_type(x) # revealed: A
```

View File

@@ -6,7 +6,7 @@
def f():
x = 1
def g():
reveal_type(x) # revealed: Literal[1]
reveal_type(x) # revealed: Unknown | Literal[1]
```
## Two levels up
@@ -16,7 +16,7 @@ def f():
x = 1
def g():
def h():
reveal_type(x) # revealed: Literal[1]
reveal_type(x) # revealed: Unknown | Literal[1]
```
## Skips class scope
@@ -28,7 +28,7 @@ def f():
class C:
x = 2
def g():
reveal_type(x) # revealed: Literal[1]
reveal_type(x) # revealed: Unknown | Literal[1]
```
## Skips annotation-only assignment
@@ -41,7 +41,7 @@ def f():
# name is otherwise not defined; maybe should be an error?
x: int
def h():
reveal_type(x) # revealed: Literal[1]
reveal_type(x) # revealed: Unknown | Literal[1]
```
## Implicit global in function
@@ -52,5 +52,5 @@ A name reference to a never-defined symbol in a function is implicitly a global
x = 1
def f():
reveal_type(x) # revealed: Literal[1]
reveal_type(x) # revealed: Unknown | Literal[1]
```

View File

@@ -17,8 +17,8 @@ class C:
x = 2
# error: [possibly-unbound-attribute] "Attribute `x` on type `Literal[C]` is possibly unbound"
reveal_type(C.x) # revealed: Literal[2]
reveal_type(C.y) # revealed: Literal[1]
reveal_type(C.x) # revealed: Unknown | Literal[2]
reveal_type(C.y) # revealed: Unknown | Literal[1]
```
## Possibly unbound in class and global scope
@@ -37,7 +37,7 @@ class C:
# error: [possibly-unresolved-reference]
y = x
reveal_type(C.y) # revealed: Literal[1, "abc"]
reveal_type(C.y) # revealed: Unknown | Literal[1, "abc"]
```
## Unbound function local

View File

@@ -182,3 +182,34 @@ class C(A, B): ...
# False negative: [incompatible-slots]
class A(int, str): ...
```
### Diagnostic if `__slots__` is externally modified
We special-case type inference for `__slots__` and return the pure inferred type, even if the symbol
is not declared — a case in which we union with `Unknown` for other public symbols. The reason for
this is that `__slots__` has a special handling in the runtime. Modifying it externally is actually
allowed, but those changes do not take effect. If you have a class `C` with `__slots__ = ("foo",)`
and externally set `C.__slots__ = ("bar",)`, you still can't access `C.bar`. And you can still
access `C.foo`. We therefore issue a diagnostic for such assignments:
```py
class A:
__slots__ = ("a",)
# Modifying `__slots__` from within the class body is fine:
__slots__ = ("a", "b")
# No `Unknown` here:
reveal_type(A.__slots__) # revealed: tuple[Literal["a"], Literal["b"]]
# But modifying it externally is not:
# error: [invalid-assignment]
A.__slots__ = ("a",)
# error: [invalid-assignment]
A.__slots__ = ("a", "b_new")
# error: [invalid-assignment]
A.__slots__ = ("a", "b", "c")
```

View File

@@ -14,7 +14,8 @@ a = NotSubscriptable()[0] # error: "Cannot subscript object of type `NotSubscri
class NotSubscriptable:
__getitem__ = None
a = NotSubscriptable()[0] # error: "Method `__getitem__` of type `None` is not callable on object of type `NotSubscriptable`"
# error: "Method `__getitem__` of type `Unknown | None` is not callable on object of type `NotSubscriptable`"
a = NotSubscriptable()[0]
```
## Valid getitem

View File

@@ -180,3 +180,11 @@ a = 4 / 0 # error: [division-by-zero]
# error: [unknown-rule] "Unknown rule `is-equal-14`"
a = 10 + 4 # knot: ignore[is-equal-14]
```
## Code with `lint:` prefix
```py
# error:[unknown-rule] "Unknown rule `lint:division-by-zero`. Did you mean `division-by-zero`?"
# error: [division-by-zero]
a = 10 / 0 # knot: ignore[lint:division-by-zero]
```

View File

@@ -139,7 +139,9 @@ reveal_type(not AlwaysFalse())
# We don't get into a cycle if someone sets their `__bool__` method to the `bool` builtin:
class BoolIsBool:
__bool__ = bool
# TODO: The `type[bool]` declaration here is a workaround to avoid running into
# https://github.com/astral-sh/ruff/issues/15672
__bool__: type[bool] = bool
# revealed: bool
reveal_type(not BoolIsBool())

View File

@@ -76,11 +76,11 @@ with Manager():
```py
class Manager:
__enter__ = 42
__enter__: int = 42
def __exit__(self, exc_tpe, exc_value, traceback): ...
# error: [invalid-context-manager] "Object of type `Manager` cannot be used with `with` because the method `__enter__` of type `Literal[42]` is not callable"
# error: [invalid-context-manager] "Object of type `Manager` cannot be used with `with` because the method `__enter__` of type `int` is not callable"
with Manager():
...
```
@@ -91,9 +91,9 @@ with Manager():
class Manager:
def __enter__(self) -> Self: ...
__exit__ = 32
__exit__: int = 32
# error: [invalid-context-manager] "Object of type `Manager` cannot be used with `with` because the method `__exit__` of type `Literal[32]` is not callable"
# error: [invalid-context-manager] "Object of type `Manager` cannot be used with `with` because the method `__exit__` of type `int` is not callable"
with Manager():
...
```

View File

@@ -136,7 +136,7 @@ pub(crate) mod tests {
/// Target Python platform
python_platform: PythonPlatform,
/// Path to a custom typeshed directory
typeshed: Option<SystemPathBuf>,
custom_typeshed: Option<SystemPathBuf>,
/// Path and content pairs for files that should be present
files: Vec<(&'a str, &'a str)>,
}
@@ -146,7 +146,7 @@ pub(crate) mod tests {
Self {
python_version: PythonVersion::default(),
python_platform: PythonPlatform::default(),
typeshed: None,
custom_typeshed: None,
files: vec![],
}
}
@@ -171,7 +171,7 @@ pub(crate) mod tests {
.context("Failed to write test files")?;
let mut search_paths = SearchPathSettings::new(vec![src_root]);
search_paths.custom_typeshed = self.typeshed;
search_paths.custom_typeshed = self.custom_typeshed;
Program::from_settings(
&db,

View File

@@ -1,5 +1,5 @@
use itertools::Itertools;
use ruff_db::diagnostic::{LintName, Severity};
use ruff_db::diagnostic::{DiagnosticId, LintName, Severity};
use rustc_hash::FxHashMap;
use std::hash::Hasher;
use thiserror::Error;
@@ -345,7 +345,18 @@ impl LintRegistry {
}
}
Some(LintEntry::Removed(lint)) => Err(GetLintError::Removed(lint.name())),
None => Err(GetLintError::Unknown(code.to_string())),
None => {
if let Some(without_prefix) = DiagnosticId::strip_category(code) {
if let Some(entry) = self.by_name.get(without_prefix) {
return Err(GetLintError::PrefixedWithCategory {
prefixed: code.to_string(),
suggestion: entry.id().name.to_string(),
});
}
}
Err(GetLintError::Unknown(code.to_string()))
}
}
}
@@ -382,12 +393,20 @@ impl LintRegistry {
#[derive(Error, Debug, Clone, PartialEq, Eq)]
pub enum GetLintError {
/// The name maps to this removed lint.
#[error("lint {0} has been removed")]
#[error("lint `{0}` has been removed")]
Removed(LintName),
/// No lint with the given name is known.
#[error("unknown lint {0}")]
#[error("unknown lint `{0}`")]
Unknown(String),
/// The name uses the full qualified diagnostic id `lint:<rule>` instead of just `rule`.
/// The String is the name without the `lint:` category prefix.
#[error("unknown lint `{prefixed}`. Did you mean `{suggestion}`?")]
PrefixedWithCategory {
prefixed: String,
suggestion: String,
},
}
#[derive(Debug, PartialEq, Eq, Clone, Copy)]
@@ -399,6 +418,16 @@ pub enum LintEntry {
Alias(LintId),
}
impl LintEntry {
fn id(self) -> LintId {
match self {
LintEntry::Lint(id) => id,
LintEntry::Removed(id) => id,
LintEntry::Alias(id) => id,
}
}
}
impl From<&'static LintMetadata> for LintEntry {
fn from(metadata: &'static LintMetadata) -> Self {
if metadata.status.is_removed() {

View File

@@ -1,4 +1,3 @@
use std::iter::FusedIterator;
use std::sync::Arc;
use rustc_hash::{FxBuildHasher, FxHashMap, FxHashSet};
@@ -6,7 +5,7 @@ use salsa::plumbing::AsId;
use ruff_db::files::File;
use ruff_db::parsed::parsed_module;
use ruff_index::{IndexSlice, IndexVec};
use ruff_index::IndexVec;
use crate::module_name::ModuleName;
use crate::semantic_index::ast_ids::node_key::ExpressionNodeKey;
@@ -17,7 +16,6 @@ use crate::semantic_index::expression::Expression;
use crate::semantic_index::symbol::{
FileScopeId, NodeWithScopeKey, NodeWithScopeRef, Scope, ScopeId, ScopedSymbolId, SymbolTable,
};
use crate::semantic_index::use_def::UseDefMap;
use crate::Db;
pub mod ast_ids;
@@ -29,8 +27,7 @@ pub mod symbol;
mod use_def;
pub(crate) use self::use_def::{
BindingWithConstraints, BindingWithConstraintsIterator, DeclarationWithConstraint,
DeclarationsIterator, ScopedVisibilityConstraintId,
BindingWithConstraints, DeclarationWithConstraint, ScopedVisibilityConstraintId, UseDefMap,
};
type SymbolMap = hashbrown::HashMap<ScopedSymbolId, (), FxBuildHasher>;
@@ -206,20 +203,43 @@ impl<'db> SemanticIndex<'db> {
/// Returns an iterator over the descendent scopes of `scope`.
#[allow(unused)]
pub(crate) fn descendent_scopes(&self, scope: FileScopeId) -> DescendentsIter {
DescendentsIter::new(self, scope)
pub(crate) fn descendent_scopes(
&self,
scope_id: FileScopeId,
) -> impl Iterator<Item = (FileScopeId, &Scope)> + '_ {
let scope = &self.scopes[scope_id];
let scopes = &self.scopes[scope.descendents.clone()];
let mut next_id = scope_id + 1;
scopes.iter().map(move |descendent| {
let result = (next_id, descendent);
next_id = next_id + 1;
result
})
}
/// Returns an iterator over the direct child scopes of `scope`.
#[allow(unused)]
pub(crate) fn child_scopes(&self, scope: FileScopeId) -> ChildrenIter {
ChildrenIter::new(self, scope)
pub(crate) fn child_scopes(
&self,
scope_id: FileScopeId,
) -> impl Iterator<Item = (FileScopeId, &Scope)> + '_ {
self.descendent_scopes(scope_id)
.filter(move |(_, scope)| scope.parent == Some(scope_id))
}
/// Returns an iterator over all ancestors of `scope`, starting with `scope` itself.
#[allow(unused)]
pub(crate) fn ancestor_scopes(&self, scope: FileScopeId) -> AncestorsIter {
AncestorsIter::new(self, scope)
pub(crate) fn ancestor_scopes(
&self,
scope_id: FileScopeId,
) -> impl Iterator<Item = (FileScopeId, &Scope)> + '_ {
let mut next_id = Some(scope_id);
std::iter::from_fn(move || {
let current_id = next_id?;
let current = &self.scopes[current_id];
next_id = current.parent;
Some((current_id, current))
})
}
/// Returns the [`Definition`] salsa ingredient for `definition_key`.
@@ -266,98 +286,6 @@ impl<'db> SemanticIndex<'db> {
}
}
pub struct AncestorsIter<'a> {
scopes: &'a IndexSlice<FileScopeId, Scope>,
next_id: Option<FileScopeId>,
}
impl<'a> AncestorsIter<'a> {
fn new(module_symbol_table: &'a SemanticIndex, start: FileScopeId) -> Self {
Self {
scopes: &module_symbol_table.scopes,
next_id: Some(start),
}
}
}
impl<'a> Iterator for AncestorsIter<'a> {
type Item = (FileScopeId, &'a Scope);
fn next(&mut self) -> Option<Self::Item> {
let current_id = self.next_id?;
let current = &self.scopes[current_id];
self.next_id = current.parent;
Some((current_id, current))
}
}
impl FusedIterator for AncestorsIter<'_> {}
pub struct DescendentsIter<'a> {
next_id: FileScopeId,
descendents: std::slice::Iter<'a, Scope>,
}
impl<'a> DescendentsIter<'a> {
fn new(symbol_table: &'a SemanticIndex, scope_id: FileScopeId) -> Self {
let scope = &symbol_table.scopes[scope_id];
let scopes = &symbol_table.scopes[scope.descendents.clone()];
Self {
next_id: scope_id + 1,
descendents: scopes.iter(),
}
}
}
impl<'a> Iterator for DescendentsIter<'a> {
type Item = (FileScopeId, &'a Scope);
fn next(&mut self) -> Option<Self::Item> {
let descendent = self.descendents.next()?;
let id = self.next_id;
self.next_id = self.next_id + 1;
Some((id, descendent))
}
fn size_hint(&self) -> (usize, Option<usize>) {
self.descendents.size_hint()
}
}
impl FusedIterator for DescendentsIter<'_> {}
impl ExactSizeIterator for DescendentsIter<'_> {}
pub struct ChildrenIter<'a> {
parent: FileScopeId,
descendents: DescendentsIter<'a>,
}
impl<'a> ChildrenIter<'a> {
fn new(module_symbol_table: &'a SemanticIndex, parent: FileScopeId) -> Self {
let descendents = DescendentsIter::new(module_symbol_table, parent);
Self {
parent,
descendents,
}
}
}
impl<'a> Iterator for ChildrenIter<'a> {
type Item = (FileScopeId, &'a Scope);
fn next(&mut self) -> Option<Self::Item> {
self.descendents
.find(|(_, scope)| scope.parent == Some(self.parent))
}
}
impl FusedIterator for ChildrenIter<'_> {}
#[cfg(test)]
mod tests {
use ruff_db::files::{system_path_to_file, File};

View File

@@ -255,11 +255,8 @@
//! snapshot, and merging a snapshot into the current state. The logic using these methods lives in
//! [`SemanticIndexBuilder`](crate::semantic_index::builder::SemanticIndexBuilder), e.g. where it
//! visits a `StmtIf` node.
use self::symbol_state::{
BindingIdWithConstraintsIterator, ConstraintIdIterator, DeclarationIdIterator,
ScopedDefinitionId, SymbolBindings, SymbolDeclarations, SymbolState,
};
pub(crate) use self::symbol_state::{ScopedConstraintId, ScopedVisibilityConstraintId};
use self::symbol_state::{ScopedDefinitionId, SymbolBindings, SymbolDeclarations, SymbolState};
use crate::semantic_index::ast_ids::ScopedUseId;
use crate::semantic_index::definition::Definition;
use crate::semantic_index::symbol::ScopedSymbolId;
@@ -286,7 +283,7 @@ pub(crate) struct UseDefMap<'db> {
all_constraints: AllConstraints<'db>,
/// Array of [`VisibilityConstraint`]s in this scope.
visibility_constraints: VisibilityConstraints<'db>,
pub(crate) visibility_constraints: VisibilityConstraints<'db>,
/// [`SymbolBindings`] reaching a [`ScopedUseId`].
bindings_by_use: IndexVec<ScopedUseId, SymbolBindings>,
@@ -313,21 +310,24 @@ impl<'db> UseDefMap<'db> {
pub(crate) fn bindings_at_use(
&self,
use_id: ScopedUseId,
) -> BindingWithConstraintsIterator<'_, 'db> {
) -> impl Iterator<Item = BindingWithConstraints<'db, impl Iterator<Item = Constraint<'db>> + '_>> + '_
{
self.bindings_iterator(&self.bindings_by_use[use_id])
}
pub(crate) fn public_bindings(
&self,
symbol: ScopedSymbolId,
) -> BindingWithConstraintsIterator<'_, 'db> {
) -> impl Iterator<Item = BindingWithConstraints<'db, impl Iterator<Item = Constraint<'db>> + '_>> + '_
{
self.bindings_iterator(self.public_symbols[symbol].bindings())
}
pub(crate) fn bindings_at_declaration(
&self,
declaration: Definition<'db>,
) -> BindingWithConstraintsIterator<'_, 'db> {
) -> impl Iterator<Item = BindingWithConstraints<'db, impl Iterator<Item = Constraint<'db>> + '_>> + '_
{
if let SymbolDefinitions::Bindings(bindings) = &self.definitions_by_definition[&declaration]
{
self.bindings_iterator(bindings)
@@ -336,10 +336,10 @@ impl<'db> UseDefMap<'db> {
}
}
pub(crate) fn declarations_at_binding<'map>(
&'map self,
pub(crate) fn declarations_at_binding(
&self,
binding: Definition<'db>,
) -> DeclarationsIterator<'map, 'db> {
) -> impl Iterator<Item = DeclarationWithConstraint<'db>> + '_ {
if let SymbolDefinitions::Declarations(declarations) =
&self.definitions_by_definition[&binding]
{
@@ -349,10 +349,10 @@ impl<'db> UseDefMap<'db> {
}
}
pub(crate) fn public_declarations<'map>(
&'map self,
pub(crate) fn public_declarations(
&self,
symbol: ScopedSymbolId,
) -> DeclarationsIterator<'map, 'db> {
) -> impl Iterator<Item = DeclarationWithConstraint<'db>> + '_ {
let declarations = self.public_symbols[symbol].declarations();
self.declarations_iterator(declarations)
}
@@ -360,24 +360,35 @@ impl<'db> UseDefMap<'db> {
fn bindings_iterator<'map>(
&'map self,
bindings: &'map SymbolBindings,
) -> BindingWithConstraintsIterator<'map, 'db> {
BindingWithConstraintsIterator {
all_definitions: &self.all_definitions,
all_constraints: &self.all_constraints,
visibility_constraints: &self.visibility_constraints,
inner: bindings.iter(),
}
) -> impl Iterator<
Item = BindingWithConstraints<'db, impl Iterator<Item = Constraint<'db>> + 'map>,
> + 'map {
bindings
.iter()
.map(|binding_id_with_constraints| BindingWithConstraints {
binding: self.all_definitions[binding_id_with_constraints.definition],
constraints: binding_id_with_constraints
.constraint_ids
.map(|constraint_id| self.all_constraints[constraint_id]),
visibility_constraint: binding_id_with_constraints.visibility_constraint,
})
}
fn declarations_iterator<'map>(
&'map self,
declarations: &'map SymbolDeclarations,
) -> DeclarationsIterator<'map, 'db> {
DeclarationsIterator {
all_definitions: &self.all_definitions,
visibility_constraints: &self.visibility_constraints,
inner: declarations.iter(),
}
) -> impl Iterator<Item = DeclarationWithConstraint<'db>> + 'map {
declarations.iter().map(
move |DeclarationIdWithConstraint {
definition,
visibility_constraint,
}| {
DeclarationWithConstraint {
declaration: self.all_definitions[definition],
visibility_constraint,
}
},
)
}
}
@@ -388,89 +399,17 @@ enum SymbolDefinitions {
Declarations(SymbolDeclarations),
}
#[derive(Debug)]
pub(crate) struct BindingWithConstraintsIterator<'map, 'db> {
all_definitions: &'map IndexVec<ScopedDefinitionId, Option<Definition<'db>>>,
all_constraints: &'map AllConstraints<'db>,
pub(crate) visibility_constraints: &'map VisibilityConstraints<'db>,
inner: BindingIdWithConstraintsIterator<'map>,
}
impl<'map, 'db> Iterator for BindingWithConstraintsIterator<'map, 'db> {
type Item = BindingWithConstraints<'map, 'db>;
fn next(&mut self) -> Option<Self::Item> {
let all_constraints = self.all_constraints;
self.inner
.next()
.map(|binding_id_with_constraints| BindingWithConstraints {
binding: self.all_definitions[binding_id_with_constraints.definition],
constraints: ConstraintsIterator {
all_constraints,
constraint_ids: binding_id_with_constraints.constraint_ids,
},
visibility_constraint: binding_id_with_constraints.visibility_constraint,
})
}
}
impl std::iter::FusedIterator for BindingWithConstraintsIterator<'_, '_> {}
pub(crate) struct BindingWithConstraints<'map, 'db> {
pub(crate) struct BindingWithConstraints<'db, I> {
pub(crate) binding: Option<Definition<'db>>,
pub(crate) constraints: ConstraintsIterator<'map, 'db>,
pub(crate) constraints: I,
pub(crate) visibility_constraint: ScopedVisibilityConstraintId,
}
pub(crate) struct ConstraintsIterator<'map, 'db> {
all_constraints: &'map AllConstraints<'db>,
constraint_ids: ConstraintIdIterator<'map>,
}
impl<'db> Iterator for ConstraintsIterator<'_, 'db> {
type Item = Constraint<'db>;
fn next(&mut self) -> Option<Self::Item> {
self.constraint_ids
.next()
.map(|constraint_id| self.all_constraints[constraint_id])
}
}
impl std::iter::FusedIterator for ConstraintsIterator<'_, '_> {}
pub(crate) struct DeclarationsIterator<'map, 'db> {
all_definitions: &'map IndexVec<ScopedDefinitionId, Option<Definition<'db>>>,
pub(crate) visibility_constraints: &'map VisibilityConstraints<'db>,
inner: DeclarationIdIterator<'map>,
}
pub(crate) struct DeclarationWithConstraint<'db> {
pub(crate) declaration: Option<Definition<'db>>,
pub(crate) visibility_constraint: ScopedVisibilityConstraintId,
}
impl<'db> Iterator for DeclarationsIterator<'_, 'db> {
type Item = DeclarationWithConstraint<'db>;
fn next(&mut self) -> Option<Self::Item> {
self.inner.next().map(
|DeclarationIdWithConstraint {
definition,
visibility_constraint,
}| {
DeclarationWithConstraint {
declaration: self.all_definitions[definition],
visibility_constraint,
}
},
)
}
}
impl std::iter::FusedIterator for DeclarationsIterator<'_, '_> {}
/// A snapshot of the definitions and constraints state at a particular point in control flow.
#[derive(Clone, Debug)]
pub(super) struct FlowSnapshot {

View File

@@ -43,12 +43,14 @@
//!
//! Tracking live declarations is simpler, since constraints are not involved, but otherwise very
//! similar to tracking live bindings.
use crate::semantic_index::use_def::VisibilityConstraints;
use super::bitset::{BitSet, BitSetIterator};
use itertools::{EitherOrBoth, Itertools};
use ruff_index::newtype_index;
use smallvec::SmallVec;
use crate::semantic_index::use_def::bitset::BitSet;
use crate::semantic_index::use_def::VisibilityConstraints;
/// A newtype-index for a definition in a particular scope.
#[newtype_index]
pub(super) struct ScopedDefinitionId;
@@ -71,14 +73,12 @@ const INLINE_BINDING_BLOCKS: usize = 3;
/// A [`BitSet`] of [`ScopedDefinitionId`], representing live bindings of a symbol in a scope.
type Bindings = BitSet<INLINE_BINDING_BLOCKS>;
type BindingsIterator<'a> = BitSetIterator<'a, INLINE_BINDING_BLOCKS>;
/// Can reference this * 64 total declarations inline; more will fall back to the heap.
const INLINE_DECLARATION_BLOCKS: usize = 3;
/// A [`BitSet`] of [`ScopedDefinitionId`], representing live declarations of a symbol in a scope.
type Declarations = BitSet<INLINE_DECLARATION_BLOCKS>;
type DeclarationsIterator<'a> = BitSetIterator<'a, INLINE_DECLARATION_BLOCKS>;
/// Can reference this * 64 total constraints inline; more will fall back to the heap.
const INLINE_CONSTRAINT_BLOCKS: usize = 2;
@@ -94,10 +94,6 @@ type InlineConstraintArray = [Constraints; INLINE_BINDINGS_PER_SYMBOL];
/// One [`BitSet`] of applicable [`ScopedConstraintId`]s per live binding.
type ConstraintsPerBinding = SmallVec<InlineConstraintArray>;
/// Iterate over all constraints for a single binding.
type ConstraintsIterator<'a> = std::slice::Iter<'a, Constraints>;
type ConstraintsIntoIterator = smallvec::IntoIter<InlineConstraintArray>;
/// A newtype-index for a visibility constraint in a particular scope.
#[newtype_index]
pub(crate) struct ScopedVisibilityConstraintId;
@@ -120,16 +116,18 @@ type VisibilityConstraintPerDeclaration = SmallVec<InlineVisibilityConstraintsAr
/// One [`ScopedVisibilityConstraintId`] per live binding.
type VisibilityConstraintPerBinding = SmallVec<InlineVisibilityConstraintsArray>;
/// Iterator over the visibility constraints for all live bindings/declarations.
type VisibilityConstraintsIterator<'a> = std::slice::Iter<'a, ScopedVisibilityConstraintId>;
type VisibilityConstraintsIntoIterator = smallvec::IntoIter<InlineVisibilityConstraintsArray>;
/// Live declarations for a single symbol at some point in control flow, with their
/// corresponding visibility constraints.
#[derive(Clone, Debug, PartialEq, Eq)]
#[derive(Clone, Debug, Default, PartialEq, Eq)]
pub(super) struct SymbolDeclarations {
/// [`BitSet`]: which declarations (as [`ScopedDefinitionId`]) can reach the current location?
///
/// Invariant: Because this is a `BitSet`, it can be viewed as a _sorted_ set of definition
/// IDs. The `visibility_constraints` field stores constraints for each definition. Therefore
/// those fields must always have the same `len()` as `live_declarations`, and the elements
/// must appear in the same order. Effectively, this means that elements must always be added
/// in sorted order, or via a binary search that determines the correct place to insert new
/// constraints.
pub(crate) live_declarations: Declarations,
/// For each live declaration, which visibility constraint applies to it?
@@ -167,19 +165,61 @@ impl SymbolDeclarations {
}
/// Return an iterator over live declarations for this symbol.
pub(super) fn iter(&self) -> DeclarationIdIterator {
DeclarationIdIterator {
declarations: self.live_declarations.iter(),
visibility_constraints: self.visibility_constraints.iter(),
pub(super) fn iter(&self) -> impl Iterator<Item = DeclarationIdWithConstraint> + '_ {
(self.live_declarations.iter())
.zip(self.visibility_constraints.iter())
.map(
|(declaration, &visibility_constraint)| DeclarationIdWithConstraint {
definition: ScopedDefinitionId::from_u32(declaration),
visibility_constraint,
},
)
}
fn merge(&mut self, b: Self, visibility_constraints: &mut VisibilityConstraints) {
let a = std::mem::take(self);
self.live_declarations = a.live_declarations.clone();
self.live_declarations.union(&b.live_declarations);
// Invariant: These zips are well-formed since we maintain an invariant that all of our
// fields are sets/vecs with the same length.
let a = (a.live_declarations.iter()).zip(a.visibility_constraints);
let b = (b.live_declarations.iter()).zip(b.visibility_constraints);
// Invariant: merge_join_by consumes the two iterators in sorted order, which ensures that
// the definition IDs and constraints line up correctly in the merged result. If a
// definition is found in both `a` and `b`, we compose the constraints from the two paths
// in an appropriate way (intersection for narrowing constraints; ternary OR for visibility
// constraints). If a definition is found in only one path, it is used as-is.
for zipped in a.merge_join_by(b, |(a_decl, _), (b_decl, _)| a_decl.cmp(b_decl)) {
match zipped {
EitherOrBoth::Both((_, a_vis_constraint), (_, b_vis_constraint)) => {
let vis_constraint = visibility_constraints
.add_or_constraint(a_vis_constraint, b_vis_constraint);
self.visibility_constraints.push(vis_constraint);
}
EitherOrBoth::Left((_, vis_constraint))
| EitherOrBoth::Right((_, vis_constraint)) => {
self.visibility_constraints.push(vis_constraint);
}
}
}
}
}
/// Live bindings for a single symbol at some point in control flow. Each live binding comes
/// with a set of narrowing constraints and a visibility constraint.
#[derive(Clone, Debug, PartialEq, Eq)]
#[derive(Clone, Debug, Default, PartialEq, Eq)]
pub(super) struct SymbolBindings {
/// [`BitSet`]: which bindings (as [`ScopedDefinitionId`]) can reach the current location?
///
/// Invariant: Because this is a `BitSet`, it can be viewed as a _sorted_ set of definition
/// IDs. The `constraints` and `visibility_constraints` field stores constraints for each
/// definition. Therefore those fields must always have the same `len()` as
/// `live_bindings`, and the elements must appear in the same order. Effectively, this means
/// that elements must always be added in sorted order, or via a binary search that determines
/// the correct place to insert new constraints.
live_bindings: Bindings,
/// For each live binding, which [`ScopedConstraintId`] apply?
@@ -235,11 +275,75 @@ impl SymbolBindings {
}
/// Iterate over currently live bindings for this symbol
pub(super) fn iter(&self) -> BindingIdWithConstraintsIterator {
BindingIdWithConstraintsIterator {
definitions: self.live_bindings.iter(),
constraints: self.constraints.iter(),
visibility_constraints: self.visibility_constraints.iter(),
pub(super) fn iter(
&self,
) -> impl Iterator<Item = BindingIdWithConstraints<impl Iterator<Item = ScopedConstraintId> + '_>> + '_
{
let i = (self.live_bindings.iter())
.zip(self.constraints.iter())
.zip(self.visibility_constraints.iter());
i.map(
|((def, constraints), visibility_constraint_id)| BindingIdWithConstraints {
definition: ScopedDefinitionId::from_u32(def),
constraint_ids: constraints.iter().map(ScopedConstraintId::from_u32),
visibility_constraint: *visibility_constraint_id,
},
)
}
fn merge(&mut self, mut b: Self, visibility_constraints: &mut VisibilityConstraints) {
let mut a = std::mem::take(self);
self.live_bindings = a.live_bindings.clone();
self.live_bindings.union(&b.live_bindings);
// Invariant: These zips are well-formed since we maintain an invariant that all of our
// fields are sets/vecs with the same length.
//
// Performance: We iterate over the `constraints` smallvecs via mut reference, because the
// individual elements are `BitSet`s (currently 24 bytes in size), and we don't want to
// move them by value multiple times during iteration. By iterating by reference, we only
// have to copy single pointers around. In the loop below, the `std::mem::take` calls
// specify precisely where we want to move them into the merged `constraints` smallvec.
//
// We don't need a similar optimization for `visibility_constraints`, since those elements
// are 32-bit IndexVec IDs, and so are already cheap to move/copy.
let a = (a.live_bindings.iter())
.zip(a.constraints.iter_mut())
.zip(a.visibility_constraints);
let b = (b.live_bindings.iter())
.zip(b.constraints.iter_mut())
.zip(b.visibility_constraints);
// Invariant: merge_join_by consumes the two iterators in sorted order, which ensures that
// the definition IDs and constraints line up correctly in the merged result. If a
// definition is found in both `a` and `b`, we compose the constraints from the two paths
// in an appropriate way (intersection for narrowing constraints; ternary OR for visibility
// constraints). If a definition is found in only one path, it is used as-is.
for zipped in a.merge_join_by(b, |((a_def, _), _), ((b_def, _), _)| a_def.cmp(b_def)) {
match zipped {
EitherOrBoth::Both(
((_, a_constraints), a_vis_constraint),
((_, b_constraints), b_vis_constraint),
) => {
// If the same definition is visible through both paths, any constraint
// that applies on only one path is irrelevant to the resulting type from
// unioning the two paths, so we intersect the constraints.
let constraints = a_constraints;
constraints.intersect(b_constraints);
self.constraints.push(std::mem::take(constraints));
// For visibility constraints, we merge them using a ternary OR operation:
let vis_constraint = visibility_constraints
.add_or_constraint(a_vis_constraint, b_vis_constraint);
self.visibility_constraints.push(vis_constraint);
}
EitherOrBoth::Left(((_, constraints), vis_constraint))
| EitherOrBoth::Right(((_, constraints), vis_constraint)) => {
self.constraints.push(std::mem::take(constraints));
self.visibility_constraints.push(vis_constraint);
}
}
}
}
}
@@ -303,202 +407,9 @@ impl SymbolState {
b: SymbolState,
visibility_constraints: &mut VisibilityConstraints,
) {
let mut a = Self {
bindings: SymbolBindings {
live_bindings: Bindings::default(),
constraints: ConstraintsPerBinding::default(),
visibility_constraints: VisibilityConstraintPerBinding::default(),
},
declarations: SymbolDeclarations {
live_declarations: self.declarations.live_declarations.clone(),
visibility_constraints: VisibilityConstraintPerDeclaration::default(),
},
};
std::mem::swap(&mut a, self);
self.bindings.merge(b.bindings, visibility_constraints);
self.declarations
.live_declarations
.union(&b.declarations.live_declarations);
let mut a_defs_iter = a.bindings.live_bindings.iter();
let mut b_defs_iter = b.bindings.live_bindings.iter();
let mut a_constraints_iter = a.bindings.constraints.into_iter();
let mut b_constraints_iter = b.bindings.constraints.into_iter();
let mut a_vis_constraints_iter = a.bindings.visibility_constraints.into_iter();
let mut b_vis_constraints_iter = b.bindings.visibility_constraints.into_iter();
let mut opt_a_def: Option<u32> = a_defs_iter.next();
let mut opt_b_def: Option<u32> = b_defs_iter.next();
// Iterate through the definitions from `a` and `b`, always processing the lower definition
// ID first, and pushing each definition onto the merged `SymbolState` with its
// constraints. If a definition is found in both `a` and `b`, push it with the intersection
// of the constraints from the two paths; a constraint that applies from only one possible
// path is irrelevant.
// Helper to push `def`, with constraints in `constraints_iter`, onto `self`.
let push = |def,
constraints_iter: &mut ConstraintsIntoIterator,
visibility_constraints_iter: &mut VisibilityConstraintsIntoIterator,
merged: &mut Self| {
merged.bindings.live_bindings.insert(def);
// SAFETY: we only ever create SymbolState using [`SymbolState::undefined`], which adds
// one "unbound" definition with corresponding narrowing and visibility constraints, or
// using [`SymbolState::record_binding`] or [`SymbolState::record_declaration`], which
// similarly add one definition with corresponding constraints. [`SymbolState::merge`]
// always pushes one definition and one constraint bitset and one visibility constraint
// together (just below), so the number of definitions and the number of constraints can
// never get out of sync.
// get out of sync.
let constraints = constraints_iter
.next()
.expect("definitions and constraints length mismatch");
let visibility_constraints = visibility_constraints_iter
.next()
.expect("definitions and visibility_constraints length mismatch");
merged.bindings.constraints.push(constraints);
merged
.bindings
.visibility_constraints
.push(visibility_constraints);
};
loop {
match (opt_a_def, opt_b_def) {
(Some(a_def), Some(b_def)) => match a_def.cmp(&b_def) {
std::cmp::Ordering::Less => {
// Next definition ID is only in `a`, push it to `self` and advance `a`.
push(
a_def,
&mut a_constraints_iter,
&mut a_vis_constraints_iter,
self,
);
opt_a_def = a_defs_iter.next();
}
std::cmp::Ordering::Greater => {
// Next definition ID is only in `b`, push it to `self` and advance `b`.
push(
b_def,
&mut b_constraints_iter,
&mut b_vis_constraints_iter,
self,
);
opt_b_def = b_defs_iter.next();
}
std::cmp::Ordering::Equal => {
// Next definition is in both; push to `self` and intersect constraints.
push(
a_def,
&mut b_constraints_iter,
&mut b_vis_constraints_iter,
self,
);
// SAFETY: see comment in `push` above.
let a_constraints = a_constraints_iter
.next()
.expect("definitions and constraints length mismatch");
let current_constraints = self.bindings.constraints.last_mut().unwrap();
// If the same definition is visible through both paths, any constraint
// that applies on only one path is irrelevant to the resulting type from
// unioning the two paths, so we intersect the constraints.
current_constraints.intersect(&a_constraints);
// For visibility constraints, we merge them using a ternary OR operation:
let a_vis_constraint = a_vis_constraints_iter
.next()
.expect("visibility_constraints length mismatch");
let current_vis_constraint =
self.bindings.visibility_constraints.last_mut().unwrap();
*current_vis_constraint = visibility_constraints
.add_or_constraint(*current_vis_constraint, a_vis_constraint);
opt_a_def = a_defs_iter.next();
opt_b_def = b_defs_iter.next();
}
},
(Some(a_def), None) => {
// We've exhausted `b`, just push the def from `a` and move on to the next.
push(
a_def,
&mut a_constraints_iter,
&mut a_vis_constraints_iter,
self,
);
opt_a_def = a_defs_iter.next();
}
(None, Some(b_def)) => {
// We've exhausted `a`, just push the def from `b` and move on to the next.
push(
b_def,
&mut b_constraints_iter,
&mut b_vis_constraints_iter,
self,
);
opt_b_def = b_defs_iter.next();
}
(None, None) => break,
}
}
// Same as above, but for declarations.
let mut a_decls_iter = a.declarations.live_declarations.iter();
let mut b_decls_iter = b.declarations.live_declarations.iter();
let mut a_vis_constraints_iter = a.declarations.visibility_constraints.into_iter();
let mut b_vis_constraints_iter = b.declarations.visibility_constraints.into_iter();
let mut opt_a_decl: Option<u32> = a_decls_iter.next();
let mut opt_b_decl: Option<u32> = b_decls_iter.next();
let push = |vis_constraints_iter: &mut VisibilityConstraintsIntoIterator,
merged: &mut Self| {
let vis_constraints = vis_constraints_iter
.next()
.expect("declarations and visibility_constraints length mismatch");
merged
.declarations
.visibility_constraints
.push(vis_constraints);
};
loop {
match (opt_a_decl, opt_b_decl) {
(Some(a_decl), Some(b_decl)) => match a_decl.cmp(&b_decl) {
std::cmp::Ordering::Less => {
push(&mut a_vis_constraints_iter, self);
opt_a_decl = a_decls_iter.next();
}
std::cmp::Ordering::Greater => {
push(&mut b_vis_constraints_iter, self);
opt_b_decl = b_decls_iter.next();
}
std::cmp::Ordering::Equal => {
push(&mut b_vis_constraints_iter, self);
let a_vis_constraint = a_vis_constraints_iter
.next()
.expect("declarations and visibility_constraints length mismatch");
let current = self.declarations.visibility_constraints.last_mut().unwrap();
*current =
visibility_constraints.add_or_constraint(*current, a_vis_constraint);
opt_a_decl = a_decls_iter.next();
opt_b_decl = b_decls_iter.next();
}
},
(Some(_), None) => {
push(&mut a_vis_constraints_iter, self);
opt_a_decl = a_decls_iter.next();
}
(None, Some(_)) => {
push(&mut b_vis_constraints_iter, self);
opt_b_decl = b_decls_iter.next();
}
(None, None) => break,
}
}
.merge(b.declarations, visibility_constraints);
}
pub(super) fn bindings(&self) -> &SymbolBindings {
@@ -514,61 +425,12 @@ impl SymbolState {
/// narrowing constraints ([`ScopedConstraintId`]) and a corresponding visibility
/// visibility constraint ([`ScopedVisibilityConstraintId`]).
#[derive(Debug)]
pub(super) struct BindingIdWithConstraints<'map> {
pub(super) struct BindingIdWithConstraints<I> {
pub(super) definition: ScopedDefinitionId,
pub(super) constraint_ids: ConstraintIdIterator<'map>,
pub(super) constraint_ids: I,
pub(super) visibility_constraint: ScopedVisibilityConstraintId,
}
#[derive(Debug)]
pub(super) struct BindingIdWithConstraintsIterator<'map> {
definitions: BindingsIterator<'map>,
constraints: ConstraintsIterator<'map>,
visibility_constraints: VisibilityConstraintsIterator<'map>,
}
impl<'map> Iterator for BindingIdWithConstraintsIterator<'map> {
type Item = BindingIdWithConstraints<'map>;
fn next(&mut self) -> Option<Self::Item> {
match (
self.definitions.next(),
self.constraints.next(),
self.visibility_constraints.next(),
) {
(None, None, None) => None,
(Some(def), Some(constraints), Some(visibility_constraint_id)) => {
Some(BindingIdWithConstraints {
definition: ScopedDefinitionId::from_u32(def),
constraint_ids: ConstraintIdIterator {
wrapped: constraints.iter(),
},
visibility_constraint: *visibility_constraint_id,
})
}
// SAFETY: see above.
_ => unreachable!("definitions and constraints length mismatch"),
}
}
}
impl std::iter::FusedIterator for BindingIdWithConstraintsIterator<'_> {}
#[derive(Debug)]
pub(super) struct ConstraintIdIterator<'a> {
wrapped: BitSetIterator<'a, INLINE_CONSTRAINT_BLOCKS>,
}
impl Iterator for ConstraintIdIterator<'_> {
type Item = ScopedConstraintId;
fn next(&mut self) -> Option<Self::Item> {
self.wrapped.next().map(ScopedConstraintId::from_u32)
}
}
impl std::iter::FusedIterator for ConstraintIdIterator<'_> {}
/// A single declaration (as [`ScopedDefinitionId`]) with a corresponding visibility
/// visibility constraint ([`ScopedVisibilityConstraintId`]).
#[derive(Debug)]
@@ -577,31 +439,6 @@ pub(super) struct DeclarationIdWithConstraint {
pub(super) visibility_constraint: ScopedVisibilityConstraintId,
}
pub(super) struct DeclarationIdIterator<'map> {
pub(crate) declarations: DeclarationsIterator<'map>,
pub(crate) visibility_constraints: VisibilityConstraintsIterator<'map>,
}
impl Iterator for DeclarationIdIterator<'_> {
type Item = DeclarationIdWithConstraint;
fn next(&mut self) -> Option<Self::Item> {
match (self.declarations.next(), self.visibility_constraints.next()) {
(None, None) => None,
(Some(declaration), Some(&visibility_constraint)) => {
Some(DeclarationIdWithConstraint {
definition: ScopedDefinitionId::from_u32(declaration),
visibility_constraint,
})
}
// SAFETY: see above.
_ => unreachable!("declarations and visibility_constraints length mismatch"),
}
}
}
impl std::iter::FusedIterator for DeclarationIdIterator<'_> {}
#[cfg(test)]
mod tests {
use super::*;

View File

@@ -163,6 +163,17 @@ fn check_unknown_rule(context: &mut CheckSuppressionsContext) {
format_args!("Unknown rule `{rule}`"),
);
}
GetLintError::PrefixedWithCategory {
prefixed,
suggestion,
} => {
context.report_lint(
&UNKNOWN_RULE,
unknown.range,
format_args!("Unknown rule `{prefixed}`. Did you mean `{suggestion}`?"),
);
}
};
}
}
@@ -203,7 +214,7 @@ fn check_unused_suppressions(context: &mut CheckSuppressionsContext) {
);
// Collect all suppressions that are unused after type-checking.
for suppression in all {
for suppression in all.iter() {
if context.diagnostics.is_used(suppression.id()) {
continue;
}
@@ -390,23 +401,11 @@ impl Suppressions {
})
}
fn iter(&self) -> SuppressionsIter {
fn iter(&self) -> impl Iterator<Item = &Suppression> + '_ {
self.file.iter().chain(&self.line)
}
}
pub(crate) type SuppressionsIter<'a> =
std::iter::Chain<std::slice::Iter<'a, Suppression>, std::slice::Iter<'a, Suppression>>;
impl<'a> IntoIterator for &'a Suppressions {
type Item = &'a Suppression;
type IntoIter = SuppressionsIter<'a>;
fn into_iter(self) -> Self::IntoIter {
self.iter()
}
}
/// A `type: ignore` or `knot: ignore` suppression.
///
/// Suppression comments that suppress multiple codes
@@ -765,8 +764,9 @@ impl<'src> SuppressionParser<'src> {
fn eat_word(&mut self) -> bool {
if self.cursor.eat_if(char::is_alphabetic) {
// Allow `:` for better error recovery when someone uses `lint:code` instead of just `code`.
self.cursor
.eat_while(|c| c.is_alphanumeric() || matches!(c, '_' | '-'));
.eat_while(|c| c.is_alphanumeric() || matches!(c, '_' | '-' | ':'));
true
} else {
false

View File

@@ -85,6 +85,14 @@ impl<'db> Symbol<'db> {
Symbol::Unbound => self,
}
}
#[must_use]
pub(crate) fn map_type(self, f: impl FnOnce(Type<'db>) -> Type<'db>) -> Symbol<'db> {
match self {
Symbol::Type(ty, boundness) => Symbol::Type(f(ty), boundness),
Symbol::Unbound => Symbol::Unbound,
}
}
}
#[cfg(test)]

View File

@@ -23,12 +23,12 @@ pub use self::subclass_of::SubclassOfType;
use crate::module_name::ModuleName;
use crate::module_resolver::{file_to_module, resolve_module, KnownModule};
use crate::semantic_index::ast_ids::HasScopedExpressionId;
use crate::semantic_index::constraint::Constraint;
use crate::semantic_index::definition::Definition;
use crate::semantic_index::symbol::{self as symbol, ScopeId, ScopedSymbolId};
use crate::semantic_index::{
global_scope, imported_modules, semantic_index, symbol_table, use_def_map,
BindingWithConstraints, BindingWithConstraintsIterator, DeclarationWithConstraint,
DeclarationsIterator,
BindingWithConstraints, DeclarationWithConstraint, UseDefMap,
};
use crate::stdlib::{builtins_symbol, known_module_symbol, typing_extensions_symbol};
use crate::suppression::check_suppressions;
@@ -80,30 +80,55 @@ pub fn check_types(db: &dyn Db, file: File) -> TypeCheckDiagnostics {
diagnostics
}
/// Computes a possibly-widened type `Unknown | T_inferred` from the inferred type `T_inferred`
/// of a symbol, unless the type is a known-instance type (e.g. `typing.Any`) or the symbol is
/// considered non-modifiable (e.g. when the symbol is `@Final`). We need this for public uses
/// of symbols that have no declared type.
fn widen_type_for_undeclared_public_symbol<'db>(
db: &'db dyn Db,
inferred: Symbol<'db>,
is_considered_non_modifiable: bool,
) -> Symbol<'db> {
// We special-case known-instance types here since symbols like `typing.Any` are typically
// not declared in the stubs (e.g. `Any = object()`), but we still want to treat them as
// such.
let is_known_instance = inferred
.ignore_possibly_unbound()
.is_some_and(|ty| matches!(ty, Type::KnownInstance(_)));
if is_considered_non_modifiable || is_known_instance {
inferred
} else {
inferred.map_type(|ty| UnionType::from_elements(db, [Type::unknown(), ty]))
}
}
/// Infer the public type of a symbol (its type as seen from outside its scope).
fn symbol<'db>(db: &'db dyn Db, scope: ScopeId<'db>, name: &str) -> Symbol<'db> {
#[salsa::tracked]
fn symbol_by_id<'db>(
db: &'db dyn Db,
scope: ScopeId<'db>,
symbol: ScopedSymbolId,
is_dunder_slots: bool,
symbol_id: ScopedSymbolId,
) -> Symbol<'db> {
let use_def = use_def_map(db, scope);
// If the symbol is declared, the public type is based on declarations; otherwise, it's based
// on inference from bindings.
let declarations = use_def.public_declarations(symbol);
let declared =
symbol_from_declarations(db, declarations).map(|SymbolAndQualifiers(ty, _)| ty);
let declarations = use_def.public_declarations(symbol_id);
let declared = symbol_from_declarations(db, use_def.as_ref(), declarations);
let is_final = declared.as_ref().is_ok_and(SymbolAndQualifiers::is_final);
let declared = declared.map(|SymbolAndQualifiers(symbol, _)| symbol);
match declared {
// Symbol is declared, trust the declared type
Ok(symbol @ Symbol::Type(_, Boundness::Bound)) => symbol,
// Symbol is possibly declared
Ok(Symbol::Type(declared_ty, Boundness::PossiblyUnbound)) => {
let bindings = use_def.public_bindings(symbol);
let inferred = symbol_from_bindings(db, bindings);
let bindings = use_def.public_bindings(symbol_id);
let inferred = symbol_from_bindings(db, use_def.as_ref(), bindings);
match inferred {
// Symbol is possibly undeclared and definitely unbound
@@ -120,12 +145,14 @@ fn symbol<'db>(db: &'db dyn Db, scope: ScopeId<'db>, name: &str) -> Symbol<'db>
),
}
}
// Symbol is undeclared, return the inferred type
// Symbol is undeclared, return the union of `Unknown` with the inferred type
Ok(Symbol::Unbound) => {
let bindings = use_def.public_bindings(symbol);
symbol_from_bindings(db, bindings)
let bindings = use_def.public_bindings(symbol_id);
let inferred = symbol_from_bindings(db, use_def.as_ref(), bindings);
widen_type_for_undeclared_public_symbol(db, inferred, is_dunder_slots || is_final)
}
// Symbol is possibly undeclared
// Symbol has conflicting declared types
Err((declared_ty, _)) => {
// Intentionally ignore conflicting declared types; that's not our problem,
// it's the problem of the module we are importing from.
@@ -177,9 +204,15 @@ fn symbol<'db>(db: &'db dyn Db, scope: ScopeId<'db>, name: &str) -> Symbol<'db>
}
let table = symbol_table(db, scope);
// `__slots__` is a symbol with special behavior in Python's runtime. It can be
// modified externally, but those changes do not take effect. We therefore issue
// a diagnostic if we see it being modified externally. In type inference, we
// can assign a "narrow" type to it even if it is not *declared*. This means, we
// do not have to call [`widen_type_for_undeclared_public_symbol`].
let is_dunder_slots = name == "__slots__";
table
.symbol_id_by_name(name)
.map(|symbol| symbol_by_id(db, scope, symbol))
.map(|symbol| symbol_by_id(db, scope, is_dunder_slots, symbol))
.unwrap_or(Symbol::Unbound)
}
@@ -288,11 +321,14 @@ fn definition_expression_type<'db>(
/// together with boundness information in a [`Symbol`].
///
/// The type will be a union if there are multiple bindings with different types.
fn symbol_from_bindings<'db>(
fn symbol_from_bindings<'map, 'db: 'map>(
db: &'db dyn Db,
bindings_with_constraints: BindingWithConstraintsIterator<'_, 'db>,
use_def: &UseDefMap<'map>,
bindings_with_constraints: impl Iterator<
Item = BindingWithConstraints<'db, impl Iterator<Item = Constraint<'db>>>,
>,
) -> Symbol<'db> {
let visibility_constraints = bindings_with_constraints.visibility_constraints;
let visibility_constraints = &use_def.visibility_constraints;
let mut bindings_with_constraints = bindings_with_constraints.peekable();
let unbound_visibility = if let Some(BindingWithConstraints {
@@ -378,6 +414,10 @@ impl SymbolAndQualifiers<'_> {
fn is_class_var(&self) -> bool {
self.1.contains(TypeQualifiers::CLASS_VAR)
}
fn is_final(&self) -> bool {
self.1.contains(TypeQualifiers::FINAL)
}
}
impl<'db> From<Symbol<'db>> for SymbolAndQualifiers<'db> {
@@ -396,7 +436,7 @@ impl<'db> From<Type<'db>> for SymbolAndQualifiers<'db> {
type SymbolFromDeclarationsResult<'db> =
Result<SymbolAndQualifiers<'db>, (TypeAndQualifiers<'db>, Box<[Type<'db>]>)>;
/// Build a declared type from a [`DeclarationsIterator`].
/// Build a declared type from an iterator of [`DeclarationWithConstraint`]s.
///
/// If there is only one declaration, or all declarations declare the same type, returns
/// `Ok(..)`. If there are conflicting declarations, returns an `Err(..)` variant with
@@ -406,9 +446,10 @@ type SymbolFromDeclarationsResult<'db> =
/// [`TypeQualifiers`] that have been specified on the declaration(s).
fn symbol_from_declarations<'db>(
db: &'db dyn Db,
declarations: DeclarationsIterator<'_, 'db>,
use_def: &UseDefMap,
declarations: impl Iterator<Item = DeclarationWithConstraint<'db>>,
) -> SymbolFromDeclarationsResult<'db> {
let visibility_constraints = declarations.visibility_constraints;
let visibility_constraints = &use_def.visibility_constraints;
let mut declarations = declarations.peekable();
let undeclared_visibility = if let Some(DeclarationWithConstraint {
@@ -4076,7 +4117,7 @@ impl<'db> Class<'db> {
/// this class, not on its superclasses.
fn own_instance_member(self, db: &'db dyn Db, name: &str) -> SymbolAndQualifiers<'db> {
// TODO: There are many things that are not yet implemented here:
// - `typing.ClassVar` and `typing.Final`
// - `typing.Final`
// - Proper diagnostics
// - Handling of possibly-undeclared/possibly-unbound attributes
// - The descriptor protocol
@@ -4084,12 +4125,12 @@ impl<'db> Class<'db> {
let body_scope = self.body_scope(db);
let table = symbol_table(db, body_scope);
if let Some(symbol) = table.symbol_id_by_name(name) {
if let Some(symbol_id) = table.symbol_id_by_name(name) {
let use_def = use_def_map(db, body_scope);
let declarations = use_def.public_declarations(symbol);
let declarations = use_def.public_declarations(symbol_id);
match symbol_from_declarations(db, declarations) {
match symbol_from_declarations(db, use_def.as_ref(), declarations) {
Ok(SymbolAndQualifiers(Symbol::Type(declared_ty, _), qualifiers)) => {
if let Some(function) = declared_ty.into_function_literal() {
// TODO: Eventually, we are going to process all decorators correctly. This is
@@ -4104,20 +4145,14 @@ impl<'db> Class<'db> {
SymbolAndQualifiers(Symbol::Type(declared_ty, Boundness::Bound), qualifiers)
}
}
Ok(SymbolAndQualifiers(Symbol::Unbound, qualifiers)) => {
let bindings = use_def.public_bindings(symbol);
let inferred = symbol_from_bindings(db, bindings);
Ok(symbol @ SymbolAndQualifiers(Symbol::Unbound, qualifiers)) => {
let bindings = use_def.public_bindings(symbol_id);
let inferred = symbol_from_bindings(db, use_def.as_ref(), bindings);
match inferred {
Symbol::Type(ty, _) => SymbolAndQualifiers(
Symbol::Type(
UnionType::from_elements(db, [Type::unknown(), ty]),
Boundness::Bound,
),
qualifiers,
),
Symbol::Unbound => SymbolAndQualifiers(Symbol::Unbound, qualifiers),
}
SymbolAndQualifiers(
widen_type_for_undeclared_public_symbol(db, inferred, symbol.is_final()),
qualifiers,
)
}
Err((declared_ty, _conflicting_declarations)) => {
// Ignore conflicting declarations
@@ -4694,7 +4729,10 @@ pub(crate) mod tests {
let bar = system_path_to_file(&db, "src/bar.py")?;
let a = global_symbol(&db, bar, "a");
assert_eq!(a.expect_type(), KnownClass::Int.to_instance(&db));
assert_eq!(
a.expect_type(),
UnionType::from_elements(&db, [Type::unknown(), KnownClass::Int.to_instance(&db)])
);
// Add a docstring to foo to trigger a re-run.
// The bar-call site of foo should not be re-run because of that
@@ -4710,7 +4748,10 @@ pub(crate) mod tests {
let a = global_symbol(&db, bar, "a");
assert_eq!(a.expect_type(), KnownClass::Int.to_instance(&db));
assert_eq!(
a.expect_type(),
UnionType::from_elements(&db, [Type::unknown(), KnownClass::Int.to_instance(&db)])
);
let events = db.take_salsa_events();
let call = &*parsed_module(&db, bar).syntax().body[1]

View File

@@ -862,7 +862,7 @@ impl<'db> TypeInferenceBuilder<'db> {
let use_def = self.index.use_def_map(binding.file_scope(self.db()));
let declarations = use_def.declarations_at_binding(binding);
let mut bound_ty = ty;
let declared_ty = symbol_from_declarations(self.db(), declarations)
let declared_ty = symbol_from_declarations(self.db(), use_def.as_ref(), declarations)
.map(|SymbolAndQualifiers(s, _)| s.ignore_possibly_unbound().unwrap_or(Type::unknown()))
.unwrap_or_else(|(ty, conflicting)| {
// TODO point out the conflicting declarations in the diagnostic?
@@ -897,7 +897,7 @@ impl<'db> TypeInferenceBuilder<'db> {
let use_def = self.index.use_def_map(declaration.file_scope(self.db()));
let prior_bindings = use_def.bindings_at_declaration(declaration);
// unbound_ty is Never because for this check we don't care about unbound
let inferred_ty = symbol_from_bindings(self.db(), prior_bindings)
let inferred_ty = symbol_from_bindings(self.db(), use_def.as_ref(), prior_bindings)
.ignore_possibly_unbound()
.unwrap_or(Type::Never);
let ty = if inferred_ty.is_assignable_to(self.db(), ty.inner_type()) {
@@ -3362,7 +3362,7 @@ impl<'db> TypeInferenceBuilder<'db> {
// If we're inferring types of deferred expressions, always treat them as public symbols
let inferred = if self.is_deferred() {
if let Some(symbol) = self.index.symbol_table(file_scope_id).symbol_id_by_name(id) {
symbol_from_bindings(self.db(), use_def.public_bindings(symbol))
symbol_from_bindings(self.db(), use_def.as_ref(), use_def.public_bindings(symbol))
} else {
assert!(
self.deferred_state.in_string_annotation(),
@@ -3372,7 +3372,7 @@ impl<'db> TypeInferenceBuilder<'db> {
}
} else {
let use_id = name.scoped_use_id(self.db(), self.scope());
symbol_from_bindings(self.db(), use_def.bindings_at_use(use_id))
symbol_from_bindings(self.db(), use_def.as_ref(), use_def.bindings_at_use(use_id))
};
if let Symbol::Type(ty, Boundness::Bound) = inferred {

View File

@@ -543,7 +543,10 @@ mod tests {
assert_eq!(a_name, "a");
assert_eq!(b_name, "b");
// TODO resolution should not be deferred; we should see A not B
assert_eq!(a_annotated_ty.unwrap().display(&db).to_string(), "B");
assert_eq!(
a_annotated_ty.unwrap().display(&db).to_string(),
"Unknown | B"
);
assert_eq!(b_annotated_ty.unwrap().display(&db).to_string(), "T");
}
@@ -583,7 +586,10 @@ mod tests {
assert_eq!(a_name, "a");
assert_eq!(b_name, "b");
// Parameter resolution deferred; we should see B
assert_eq!(a_annotated_ty.unwrap().display(&db).to_string(), "B");
assert_eq!(
a_annotated_ty.unwrap().display(&db).to_string(),
"Unknown | B"
);
assert_eq!(b_annotated_ty.unwrap().display(&db).to_string(), "T");
}

View File

@@ -43,10 +43,10 @@ static EXPECTED_DIAGNOSTICS: &[&str] = &[
"warning[lint:possibly-unresolved-reference] /src/tomllib/_parser.py:579:12 Name `char` used when possibly not defined",
"warning[lint:possibly-unresolved-reference] /src/tomllib/_parser.py:580:63 Name `char` used when possibly not defined",
// We don't handle intersections in `is_assignable_to` yet
"error[lint:invalid-argument-type] /src/tomllib/_parser.py:626:46 Object of type `@Todo & ~AlwaysFalsy` cannot be assigned to parameter 1 (`match`) of function `match_to_datetime`; expected type `Match`",
"error[lint:invalid-argument-type] /src/tomllib/_parser.py:626:46 Object of type `Unknown & ~AlwaysFalsy | @Todo & ~AlwaysFalsy` cannot be assigned to parameter 1 (`match`) of function `match_to_datetime`; expected type `Match`",
"warning[lint:possibly-unresolved-reference] /src/tomllib/_parser.py:629:38 Name `datetime_obj` used when possibly not defined",
"error[lint:invalid-argument-type] /src/tomllib/_parser.py:632:58 Object of type `@Todo & ~AlwaysFalsy` cannot be assigned to parameter 1 (`match`) of function `match_to_localtime`; expected type `Match`",
"error[lint:invalid-argument-type] /src/tomllib/_parser.py:639:52 Object of type `@Todo & ~AlwaysFalsy` cannot be assigned to parameter 1 (`match`) of function `match_to_number`; expected type `Match`",
"error[lint:invalid-argument-type] /src/tomllib/_parser.py:632:58 Object of type `Unknown & ~AlwaysFalsy | @Todo & ~AlwaysFalsy` cannot be assigned to parameter 1 (`match`) of function `match_to_localtime`; expected type `Match`",
"error[lint:invalid-argument-type] /src/tomllib/_parser.py:639:52 Object of type `Unknown & ~AlwaysFalsy | @Todo & ~AlwaysFalsy` cannot be assigned to parameter 1 (`match`) of function `match_to_number`; expected type `Match`",
"warning[lint:unused-ignore-comment] /src/tomllib/_parser.py:682:31 Unused blanket `type: ignore` directive",
];

View File

@@ -94,6 +94,10 @@ impl DiagnosticId {
matches!(self, DiagnosticId::Lint(self_name) if self_name == name)
}
pub fn strip_category(code: &str) -> Option<&str> {
code.split_once(':').map(|(_, rest)| rest)
}
/// Returns `true` if this `DiagnosticId` matches the given name.
///
/// ## Examples

View File

@@ -0,0 +1,127 @@
from datetime import datetime
import pendulum
from airflow.decorators import dag, task
from airflow.models import DAG
from airflow.models.baseoperator import BaseOperator
from airflow.operators.dummy import DummyOperator
from airflow.plugins_manager import AirflowPlugin
from airflow.providers.standard.operators.python import PythonOperator
from airflow.utils.context import get_current_context
def access_invalid_key_in_context(**context):
print("access invalid key", context["conf"])
@task
def access_invalid_key_task_out_of_dag(**context):
print("access invalid key", context.get("conf"))
@dag(
schedule=None,
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
catchup=False,
tags=[""],
)
def invalid_dag():
@task()
def access_invalid_key_task(**context):
print("access invalid key", context.get("conf"))
task1 = PythonOperator(
task_id="task1",
python_callable=access_invalid_key_in_context,
)
access_invalid_key_task() >> task1
access_invalid_key_task_out_of_dag()
invalid_dag()
@task
def print_config(**context):
# This should not throw an error as logical_date is part of airflow context.
logical_date = context["logical_date"]
# Removed usage - should trigger violations
execution_date = context["execution_date"]
next_ds = context["next_ds"]
next_ds_nodash = context["next_ds_nodash"]
next_execution_date = context["next_execution_date"]
prev_ds = context["prev_ds"]
prev_ds_nodash = context["prev_ds_nodash"]
prev_execution_date = context["prev_execution_date"]
prev_execution_date_success = context["prev_execution_date_success"]
tomorrow_ds = context["tomorrow_ds"]
yesterday_ds = context["yesterday_ds"]
yesterday_ds_nodash = context["yesterday_ds_nodash"]
with DAG(
dag_id="example_dag",
schedule_interval="@daily",
start_date=datetime(2023, 1, 1),
template_searchpath=["/templates"],
) as dag:
task1 = DummyOperator(
task_id="task1",
params={
# Removed variables in template
"execution_date": "{{ execution_date }}",
"next_ds": "{{ next_ds }}",
"prev_ds": "{{ prev_ds }}"
},
)
class CustomMacrosPlugin(AirflowPlugin):
name = "custom_macros"
macros = {
"execution_date_macro": lambda context: context["execution_date"],
"next_ds_macro": lambda context: context["next_ds"]
}
@task
def print_config():
context = get_current_context()
execution_date = context["execution_date"]
next_ds = context["next_ds"]
next_ds_nodash = context["next_ds_nodash"]
next_execution_date = context["next_execution_date"]
prev_ds = context["prev_ds"]
prev_ds_nodash = context["prev_ds_nodash"]
prev_execution_date = context["prev_execution_date"]
prev_execution_date_success = context["prev_execution_date_success"]
tomorrow_ds = context["tomorrow_ds"]
yesterday_ds = context["yesterday_ds"]
yesterday_ds_nodash = context["yesterday_ds_nodash"]
class CustomOperator(BaseOperator):
def execute(self, context):
execution_date = context["execution_date"]
next_ds = context["next_ds"]
next_ds_nodash = context["next_ds_nodash"]
next_execution_date = context["next_execution_date"]
prev_ds = context["prev_ds"]
prev_ds_nodash = context["prev_ds_nodash"]
prev_execution_date = context["prev_execution_date"]
prev_execution_date_success = context["prev_execution_date_success"]
tomorrow_ds = context["tomorrow_ds"]
yesterday_ds = context["yesterday_ds"]
yesterday_ds_nodash = context["yesterday_ds_nodash"]
@task
def access_invalid_argument_task_out_of_dag(execution_date, tomorrow_ds, logical_date, **context):
print("execution date", execution_date)
print("access invalid key", context.get("conf"))
@task(task_id="print_the_context")
def print_context(ds=None, **kwargs):
"""Print the Airflow context and ds variable from the context."""
print(ds)
print(kwargs.get("tomorrow_ds"))
c = get_current_context()
c.get("execution_date")
class CustomOperatorNew(BaseOperator):
def execute(self, context):
execution_date = context.get("execution_date")
next_ds = context.get("next_ds")

View File

@@ -0,0 +1,40 @@
from typing import TYPE_CHECKING
# Verify that statements nested in conditionals (such as top-level type-checking blocks)
# are still considered top-level
if TYPE_CHECKING:
import string
def import_in_function():
import symtable # [import-outside-toplevel]
import os, sys # [import-outside-toplevel]
import time as thyme # [import-outside-toplevel]
import random as rand, socket as sock # [import-outside-toplevel]
from collections import defaultdict # [import-outside-toplevel]
from math import sin as sign, cos as cosplay # [import-outside-toplevel]
# these should be allowed due to TID253 top-level ban
import foo_banned
import foo_banned as renamed
from pkg import bar_banned
from pkg import bar_banned as renamed
from pkg_banned import one as other, two, three
# this should still trigger an error due to multiple imports
from pkg import foo_allowed, bar_banned # [import-outside-toplevel]
class ClassWithImports:
import tokenize # [import-outside-toplevel]
def __init__(self):
import trace # [import-outside-toplevel]
# these should be allowed due to TID253 top-level ban
import foo_banned
import foo_banned as renamed
from pkg import bar_banned
from pkg import bar_banned as renamed
from pkg_banned import one as other, two, three
# this should still trigger an error due to multiple imports
from pkg import foo_allowed, bar_banned # [import-outside-toplevel]

View File

@@ -64,3 +64,20 @@ round(lorem, -2) # No error
round(lorem, inferred_int) # No error
round(lorem, 3 + 4) # No error
round(lorem, foo) # No error
# Fixes should preserve parentheses when argument
# contains newline.
# See https://github.com/astral-sh/ruff/issues/15598
round(-
1)
round(1
*1
)
# fix should be unsafe if comment is in call range
round(# a comment
17
)
round(
17 # a comment
)

View File

@@ -175,7 +175,9 @@ pub(crate) fn expression(expr: &Expr, checker: &mut Checker) {
if checker.enabled(Rule::NonPEP646Unpack) {
pyupgrade::rules::use_pep646_unpack(checker, subscript);
}
if checker.enabled(Rule::Airflow3Removal) {
airflow::rules::removed_in_3(checker, expr);
}
pandas_vet::rules::subscript(checker, value, expr);
}
Expr::Tuple(ast::ExprTuple {

View File

@@ -376,6 +376,9 @@ pub(crate) fn statement(stmt: &Stmt, checker: &mut Checker) {
if checker.enabled(Rule::PytestParameterWithDefaultArgument) {
flake8_pytest_style::rules::parameter_with_default_argument(checker, function_def);
}
if checker.enabled(Rule::Airflow3Removal) {
airflow::rules::removed_in_3_function_def(checker, function_def);
}
if checker.enabled(Rule::NonPEP695GenericFunction) {
pyupgrade::rules::non_pep695_generic_function(checker, function_def);
}
@@ -605,6 +608,10 @@ pub(crate) fn statement(stmt: &Stmt, checker: &mut Checker) {
flake8_bandit::rules::suspicious_imports(checker, stmt);
}
if checker.enabled(Rule::BannedModuleLevelImports) {
flake8_tidy_imports::rules::banned_module_level_imports(checker, stmt);
}
for alias in names {
if checker.enabled(Rule::NonAsciiImportName) {
pylint::rules::non_ascii_module_import(checker, alias);
@@ -629,18 +636,6 @@ pub(crate) fn statement(stmt: &Stmt, checker: &mut Checker) {
);
}
if checker.enabled(Rule::BannedModuleLevelImports) {
flake8_tidy_imports::rules::banned_module_level_imports(
checker,
&flake8_tidy_imports::matchers::NameMatchPolicy::MatchNameOrParent(
flake8_tidy_imports::matchers::MatchNameOrParent {
module: &alias.name,
},
),
&alias,
);
}
if !checker.source_type.is_stub() {
if checker.enabled(Rule::UselessImportAlias) {
pylint::rules::useless_import_alias(checker, alias);
@@ -845,36 +840,9 @@ pub(crate) fn statement(stmt: &Stmt, checker: &mut Checker) {
}
}
if checker.enabled(Rule::BannedModuleLevelImports) {
if let Some(module) = helpers::resolve_imported_module_path(
level,
module,
checker.module.qualified_name(),
) {
flake8_tidy_imports::rules::banned_module_level_imports(
checker,
&flake8_tidy_imports::matchers::NameMatchPolicy::MatchNameOrParent(
flake8_tidy_imports::matchers::MatchNameOrParent { module: &module },
),
&stmt,
);
for alias in names {
if &alias.name == "*" {
continue;
}
flake8_tidy_imports::rules::banned_module_level_imports(
checker,
&flake8_tidy_imports::matchers::NameMatchPolicy::MatchName(
flake8_tidy_imports::matchers::MatchName {
module: &module,
member: &alias.name,
},
),
&alias,
);
}
}
flake8_tidy_imports::rules::banned_module_level_imports(checker, stmt);
}
if checker.enabled(Rule::PytestIncorrectPytestImport) {
if let Some(diagnostic) =
flake8_pytest_style::rules::import_from(stmt, module, level)

View File

@@ -189,7 +189,7 @@ pub(crate) struct Checker<'a> {
/// The [`Path`] to the package containing the current file.
package: Option<PackageRoot<'a>>,
/// The module representation of the current file (e.g., `foo.bar`).
module: Module<'a>,
pub(crate) module: Module<'a>,
/// The [`PySourceType`] of the current file.
pub(crate) source_type: PySourceType,
/// The [`CellOffsets`] for the current file, if it's a Jupyter notebook.

View File

@@ -316,7 +316,7 @@ mod schema {
// Filter out all test-only rules
#[cfg(any(feature = "test-rules", test))]
#[allow(clippy::used_underscore_binding)]
if _rule.starts_with("RUF9") {
if _rule.starts_with("RUF9") || _rule == "PLW0101" {
return false;
}

View File

@@ -18,6 +18,7 @@ mod tests {
#[test_case(Rule::Airflow3Removal, Path::new("AIR302_names.py"))]
#[test_case(Rule::Airflow3Removal, Path::new("AIR302_class_attribute.py"))]
#[test_case(Rule::Airflow3Removal, Path::new("AIR302_airflow_plugin.py"))]
#[test_case(Rule::Airflow3Removal, Path::new("AIR302_context.py"))]
#[test_case(Rule::Airflow3MovedToProvider, Path::new("AIR303.py"))]
fn rules(rule_code: Rule, path: &Path) -> Result<()> {
let snapshot = format!("{}_{}", rule_code.noqa_code(), path.to_string_lossy());

View File

@@ -1,17 +1,18 @@
use crate::checkers::ast::Checker;
use ruff_diagnostics::{Diagnostic, Edit, Fix, FixAvailability, Violation};
use ruff_macros::{derive_message_formats, ViolationMetadata};
use ruff_python_ast::helpers::map_callable;
use ruff_python_ast::{
name::QualifiedName, Arguments, Expr, ExprAttribute, ExprCall, ExprContext, ExprName,
StmtClassDef,
ExprStringLiteral, ExprSubscript, Stmt, StmtClassDef, StmtFunctionDef,
};
use ruff_python_semantic::analyze::typing;
use ruff_python_semantic::Modules;
use ruff_python_semantic::ScopeKind;
use ruff_python_semantic::SemanticModel;
use ruff_text_size::Ranged;
use ruff_text_size::TextRange;
use crate::checkers::ast::Checker;
/// ## What it does
/// Checks for uses of deprecated Airflow functions and values.
///
@@ -71,6 +72,13 @@ impl Violation for Airflow3Removal {
}
}
#[derive(Debug, Eq, PartialEq)]
enum Replacement {
None,
Name(&'static str),
Message(&'static str),
}
/// AIR302
pub(crate) fn removed_in_3(checker: &mut Checker, expr: &Expr) {
if !checker.semantic().seen_module(Modules::AIRFLOW) {
@@ -83,10 +91,11 @@ pub(crate) fn removed_in_3(checker: &mut Checker, expr: &Expr) {
func, arguments, ..
},
) => {
if let Some(qualname) = checker.semantic().resolve_qualified_name(func) {
check_call_arguments(checker, &qualname, arguments);
if let Some(qualified_name) = checker.semantic().resolve_qualified_name(func) {
check_call_arguments(checker, &qualified_name, arguments);
};
check_method(checker, call_expr);
check_context_key_usage_in_call(checker, call_expr);
}
Expr::Attribute(attribute_expr @ ExprAttribute { attr, .. }) => {
check_name(checker, expr, attr.range());
@@ -100,15 +109,67 @@ pub(crate) fn removed_in_3(checker: &mut Checker, expr: &Expr) {
}
}
}
Expr::Subscript(subscript_expr) => {
check_context_key_usage_in_subscript(checker, subscript_expr);
}
_ => {}
}
}
#[derive(Debug, Eq, PartialEq)]
enum Replacement {
None,
Name(&'static str),
Message(&'static str),
/// AIR302
pub(crate) fn removed_in_3_function_def(checker: &mut Checker, function_def: &StmtFunctionDef) {
if !checker.semantic().seen_module(Modules::AIRFLOW) {
return;
}
check_function_parameters(checker, function_def);
}
const REMOVED_CONTEXT_KEYS: [&str; 12] = [
"conf",
"execution_date",
"next_ds",
"next_ds_nodash",
"next_execution_date",
"prev_ds",
"prev_ds_nodash",
"prev_execution_date",
"prev_execution_date_success",
"tomorrow_ds",
"yesterday_ds",
"yesterday_ds_nodash",
];
/// Check the function parameters for removed context keys.
///
/// For example:
///
/// ```python
/// from airflow.decorators import task
///
/// @task
/// def another_task(execution_date, **kwargs):
/// # ^^^^^^^^^^^^^^
/// # 'execution_date' is removed in Airflow 3.0
/// pass
/// ```
fn check_function_parameters(checker: &mut Checker, function_def: &StmtFunctionDef) {
if !is_airflow_task(function_def, checker.semantic()) {
return;
}
for param in function_def.parameters.iter_non_variadic_params() {
let param_name = param.parameter.name.as_str();
if REMOVED_CONTEXT_KEYS.contains(&param_name) {
checker.diagnostics.push(Diagnostic::new(
Airflow3Removal {
deprecated: param_name.to_string(),
replacement: Replacement::None,
},
param.parameter.name.range(),
));
}
}
}
/// Check whether a removed Airflow argument is passed.
@@ -120,8 +181,12 @@ enum Replacement {
///
/// DAG(schedule_interval="@daily")
/// ```
fn check_call_arguments(checker: &mut Checker, qualname: &QualifiedName, arguments: &Arguments) {
match qualname.segments() {
fn check_call_arguments(
checker: &mut Checker,
qualified_name: &QualifiedName,
arguments: &Arguments,
) {
match qualified_name.segments() {
["airflow", .., "DAG" | "dag"] => {
checker.diagnostics.extend(diagnostic_for_argument(
arguments,
@@ -145,7 +210,7 @@ fn check_call_arguments(checker: &mut Checker, qualname: &QualifiedName, argumen
));
}
_ => {
if is_airflow_auth_manager(qualname.segments()) {
if is_airflow_auth_manager(qualified_name.segments()) {
if !arguments.is_empty() {
checker.diagnostics.push(Diagnostic::new(
Airflow3Removal {
@@ -157,13 +222,13 @@ fn check_call_arguments(checker: &mut Checker, qualname: &QualifiedName, argumen
arguments.range(),
));
}
} else if is_airflow_task_handler(qualname.segments()) {
} else if is_airflow_task_handler(qualified_name.segments()) {
checker.diagnostics.extend(diagnostic_for_argument(
arguments,
"filename_template",
None,
));
} else if is_airflow_operator(qualname.segments()) {
} else if is_airflow_operator(qualified_name.segments()) {
checker
.diagnostics
.extend(diagnostic_for_argument(arguments, "sla", None));
@@ -172,7 +237,7 @@ fn check_call_arguments(checker: &mut Checker, qualname: &QualifiedName, argumen
"task_concurrency",
Some("max_active_tis_per_dag"),
));
match qualname.segments() {
match qualified_name.segments() {
["airflow", .., "operators", "trigger_dagrun", "TriggerDagRunOperator"] => {
checker.diagnostics.extend(diagnostic_for_argument(
arguments,
@@ -252,6 +317,137 @@ fn check_class_attribute(checker: &mut Checker, attribute_expr: &ExprAttribute)
}
}
/// Checks whether an Airflow 3.0removed context key is used in a function decorated with `@task`.
///
/// Specifically, it flags the following two scenarios:
///
/// 1. A removed context key accessed via `context.get("...")` where context is coming from
/// `get_current_context` function.
///
/// ```python
/// from airflow.decorators import task
/// from airflow.utils.context import get_current_context
///
///
/// @task
/// def my_task():
/// context = get_current_context()
/// context.get("conf") # 'conf' is removed in Airflow 3.0
/// ```
///
/// 2. A removed context key accessed via `context.get("...")` where context is a kwarg parameter.
///
/// ```python
/// from airflow.decorators import task
///
///
/// @task
/// def my_task(**context):
/// context.get("conf") # 'conf' is removed in Airflow 3.0
/// ```
fn check_context_key_usage_in_call(checker: &mut Checker, call_expr: &ExprCall) {
if !in_airflow_task_function(checker.semantic()) {
return;
}
let Expr::Attribute(ExprAttribute { value, attr, .. }) = &*call_expr.func else {
return;
};
if attr.as_str() != "get" {
return;
}
let is_kwarg_parameter = value
.as_name_expr()
.is_some_and(|name| is_kwarg_parameter(checker.semantic(), name));
let is_assigned_from_get_current_context =
typing::resolve_assignment(value, checker.semantic()).is_some_and(|qualified_name| {
matches!(
qualified_name.segments(),
["airflow", "utils", "context", "get_current_context"]
)
});
if !(is_kwarg_parameter || is_assigned_from_get_current_context) {
return;
}
for removed_key in REMOVED_CONTEXT_KEYS {
let Some(Expr::StringLiteral(ExprStringLiteral { value, range })) =
call_expr.arguments.find_positional(0)
else {
continue;
};
if value == removed_key {
checker.diagnostics.push(Diagnostic::new(
Airflow3Removal {
deprecated: removed_key.to_string(),
replacement: Replacement::None,
},
*range,
));
}
}
}
/// Check if a subscript expression accesses a removed Airflow context variable.
/// If a removed key is found, push a corresponding diagnostic.
fn check_context_key_usage_in_subscript(checker: &mut Checker, subscript: &ExprSubscript) {
if !in_airflow_task_function(checker.semantic()) {
return;
}
let ExprSubscript { value, slice, .. } = subscript;
let Some(ExprStringLiteral { value: key, .. }) = slice.as_string_literal_expr() else {
return;
};
let is_kwarg_parameter = value
.as_name_expr()
.is_some_and(|name| is_kwarg_parameter(checker.semantic(), name));
let is_assigned_from_get_current_context =
typing::resolve_assignment(value, checker.semantic()).is_some_and(|qualified_name| {
matches!(
qualified_name.segments(),
["airflow", "utils", "context", "get_current_context"]
)
});
if !(is_kwarg_parameter || is_assigned_from_get_current_context) {
return;
}
if REMOVED_CONTEXT_KEYS.contains(&key.to_str()) {
checker.diagnostics.push(Diagnostic::new(
Airflow3Removal {
deprecated: key.to_string(),
replacement: Replacement::None,
},
slice.range(),
));
}
}
/// Finds the parameter definition for a given name expression in a function.
fn is_kwarg_parameter(semantic: &SemanticModel, name: &ExprName) -> bool {
let Some(binding_id) = semantic.only_binding(name) else {
return false;
};
let binding = semantic.binding(binding_id);
let Some(Stmt::FunctionDef(StmtFunctionDef { parameters, .. })) = binding.statement(semantic)
else {
return false;
};
parameters
.kwarg
.as_deref()
.is_some_and(|kwarg| kwarg.name.as_str() == name.id.as_str())
}
/// Check whether a removed Airflow class method is called.
///
/// For example:
@@ -860,3 +1056,23 @@ fn is_airflow_builtin_or_provider(segments: &[&str], module: &str, symbol_suffix
_ => false,
}
}
/// Returns `true` if the current statement hierarchy has a function that's decorated with
/// `@airflow.decorators.task`.
fn in_airflow_task_function(semantic: &SemanticModel) -> bool {
semantic
.current_statements()
.find_map(|stmt| stmt.as_function_def_stmt())
.is_some_and(|function_def| is_airflow_task(function_def, semantic))
}
/// Returns `true` if the given function is decorated with `@airflow.decorators.task`.
fn is_airflow_task(function_def: &StmtFunctionDef, semantic: &SemanticModel) -> bool {
function_def.decorator_list.iter().any(|decorator| {
semantic
.resolve_qualified_name(map_callable(&decorator.expression))
.is_some_and(|qualified_name| {
matches!(qualified_name.segments(), ["airflow", "decorators", "task"])
})
})
}

View File

@@ -0,0 +1,319 @@
---
source: crates/ruff_linter/src/rules/airflow/mod.rs
---
AIR302_context.py:19:45: AIR302 `conf` is removed in Airflow 3.0
|
17 | @task
18 | def access_invalid_key_task_out_of_dag(**context):
19 | print("access invalid key", context.get("conf"))
| ^^^^^^ AIR302
20 |
21 | @dag(
|
AIR302_context.py:30:49: AIR302 `conf` is removed in Airflow 3.0
|
28 | @task()
29 | def access_invalid_key_task(**context):
30 | print("access invalid key", context.get("conf"))
| ^^^^^^ AIR302
31 |
32 | task1 = PythonOperator(
|
AIR302_context.py:47:30: AIR302 `execution_date` is removed in Airflow 3.0
|
46 | # Removed usage - should trigger violations
47 | execution_date = context["execution_date"]
| ^^^^^^^^^^^^^^^^ AIR302
48 | next_ds = context["next_ds"]
49 | next_ds_nodash = context["next_ds_nodash"]
|
AIR302_context.py:48:23: AIR302 `next_ds` is removed in Airflow 3.0
|
46 | # Removed usage - should trigger violations
47 | execution_date = context["execution_date"]
48 | next_ds = context["next_ds"]
| ^^^^^^^^^ AIR302
49 | next_ds_nodash = context["next_ds_nodash"]
50 | next_execution_date = context["next_execution_date"]
|
AIR302_context.py:49:30: AIR302 `next_ds_nodash` is removed in Airflow 3.0
|
47 | execution_date = context["execution_date"]
48 | next_ds = context["next_ds"]
49 | next_ds_nodash = context["next_ds_nodash"]
| ^^^^^^^^^^^^^^^^ AIR302
50 | next_execution_date = context["next_execution_date"]
51 | prev_ds = context["prev_ds"]
|
AIR302_context.py:50:35: AIR302 `next_execution_date` is removed in Airflow 3.0
|
48 | next_ds = context["next_ds"]
49 | next_ds_nodash = context["next_ds_nodash"]
50 | next_execution_date = context["next_execution_date"]
| ^^^^^^^^^^^^^^^^^^^^^ AIR302
51 | prev_ds = context["prev_ds"]
52 | prev_ds_nodash = context["prev_ds_nodash"]
|
AIR302_context.py:51:23: AIR302 `prev_ds` is removed in Airflow 3.0
|
49 | next_ds_nodash = context["next_ds_nodash"]
50 | next_execution_date = context["next_execution_date"]
51 | prev_ds = context["prev_ds"]
| ^^^^^^^^^ AIR302
52 | prev_ds_nodash = context["prev_ds_nodash"]
53 | prev_execution_date = context["prev_execution_date"]
|
AIR302_context.py:52:30: AIR302 `prev_ds_nodash` is removed in Airflow 3.0
|
50 | next_execution_date = context["next_execution_date"]
51 | prev_ds = context["prev_ds"]
52 | prev_ds_nodash = context["prev_ds_nodash"]
| ^^^^^^^^^^^^^^^^ AIR302
53 | prev_execution_date = context["prev_execution_date"]
54 | prev_execution_date_success = context["prev_execution_date_success"]
|
AIR302_context.py:53:35: AIR302 `prev_execution_date` is removed in Airflow 3.0
|
51 | prev_ds = context["prev_ds"]
52 | prev_ds_nodash = context["prev_ds_nodash"]
53 | prev_execution_date = context["prev_execution_date"]
| ^^^^^^^^^^^^^^^^^^^^^ AIR302
54 | prev_execution_date_success = context["prev_execution_date_success"]
55 | tomorrow_ds = context["tomorrow_ds"]
|
AIR302_context.py:54:43: AIR302 `prev_execution_date_success` is removed in Airflow 3.0
|
52 | prev_ds_nodash = context["prev_ds_nodash"]
53 | prev_execution_date = context["prev_execution_date"]
54 | prev_execution_date_success = context["prev_execution_date_success"]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AIR302
55 | tomorrow_ds = context["tomorrow_ds"]
56 | yesterday_ds = context["yesterday_ds"]
|
AIR302_context.py:55:27: AIR302 `tomorrow_ds` is removed in Airflow 3.0
|
53 | prev_execution_date = context["prev_execution_date"]
54 | prev_execution_date_success = context["prev_execution_date_success"]
55 | tomorrow_ds = context["tomorrow_ds"]
| ^^^^^^^^^^^^^ AIR302
56 | yesterday_ds = context["yesterday_ds"]
57 | yesterday_ds_nodash = context["yesterday_ds_nodash"]
|
AIR302_context.py:56:28: AIR302 `yesterday_ds` is removed in Airflow 3.0
|
54 | prev_execution_date_success = context["prev_execution_date_success"]
55 | tomorrow_ds = context["tomorrow_ds"]
56 | yesterday_ds = context["yesterday_ds"]
| ^^^^^^^^^^^^^^ AIR302
57 | yesterday_ds_nodash = context["yesterday_ds_nodash"]
|
AIR302_context.py:57:35: AIR302 `yesterday_ds_nodash` is removed in Airflow 3.0
|
55 | tomorrow_ds = context["tomorrow_ds"]
56 | yesterday_ds = context["yesterday_ds"]
57 | yesterday_ds_nodash = context["yesterday_ds_nodash"]
| ^^^^^^^^^^^^^^^^^^^^^ AIR302
58 |
59 | with DAG(
|
AIR302_context.py:61:5: AIR302 [*] `schedule_interval` is removed in Airflow 3.0
|
59 | with DAG(
60 | dag_id="example_dag",
61 | schedule_interval="@daily",
| ^^^^^^^^^^^^^^^^^ AIR302
62 | start_date=datetime(2023, 1, 1),
63 | template_searchpath=["/templates"],
|
= help: Use `schedule` instead
Safe fix
58 58 |
59 59 | with DAG(
60 60 | dag_id="example_dag",
61 |- schedule_interval="@daily",
61 |+ schedule="@daily",
62 62 | start_date=datetime(2023, 1, 1),
63 63 | template_searchpath=["/templates"],
64 64 | ) as dag:
AIR302_context.py:65:13: AIR302 `airflow.operators.dummy.DummyOperator` is removed in Airflow 3.0
|
63 | template_searchpath=["/templates"],
64 | ) as dag:
65 | task1 = DummyOperator(
| ^^^^^^^^^^^^^ AIR302
66 | task_id="task1",
67 | params={
|
= help: Use `airflow.operators.empty.EmptyOperator` instead
AIR302_context.py:85:30: AIR302 `execution_date` is removed in Airflow 3.0
|
83 | def print_config():
84 | context = get_current_context()
85 | execution_date = context["execution_date"]
| ^^^^^^^^^^^^^^^^ AIR302
86 | next_ds = context["next_ds"]
87 | next_ds_nodash = context["next_ds_nodash"]
|
AIR302_context.py:86:23: AIR302 `next_ds` is removed in Airflow 3.0
|
84 | context = get_current_context()
85 | execution_date = context["execution_date"]
86 | next_ds = context["next_ds"]
| ^^^^^^^^^ AIR302
87 | next_ds_nodash = context["next_ds_nodash"]
88 | next_execution_date = context["next_execution_date"]
|
AIR302_context.py:87:30: AIR302 `next_ds_nodash` is removed in Airflow 3.0
|
85 | execution_date = context["execution_date"]
86 | next_ds = context["next_ds"]
87 | next_ds_nodash = context["next_ds_nodash"]
| ^^^^^^^^^^^^^^^^ AIR302
88 | next_execution_date = context["next_execution_date"]
89 | prev_ds = context["prev_ds"]
|
AIR302_context.py:88:35: AIR302 `next_execution_date` is removed in Airflow 3.0
|
86 | next_ds = context["next_ds"]
87 | next_ds_nodash = context["next_ds_nodash"]
88 | next_execution_date = context["next_execution_date"]
| ^^^^^^^^^^^^^^^^^^^^^ AIR302
89 | prev_ds = context["prev_ds"]
90 | prev_ds_nodash = context["prev_ds_nodash"]
|
AIR302_context.py:89:23: AIR302 `prev_ds` is removed in Airflow 3.0
|
87 | next_ds_nodash = context["next_ds_nodash"]
88 | next_execution_date = context["next_execution_date"]
89 | prev_ds = context["prev_ds"]
| ^^^^^^^^^ AIR302
90 | prev_ds_nodash = context["prev_ds_nodash"]
91 | prev_execution_date = context["prev_execution_date"]
|
AIR302_context.py:90:30: AIR302 `prev_ds_nodash` is removed in Airflow 3.0
|
88 | next_execution_date = context["next_execution_date"]
89 | prev_ds = context["prev_ds"]
90 | prev_ds_nodash = context["prev_ds_nodash"]
| ^^^^^^^^^^^^^^^^ AIR302
91 | prev_execution_date = context["prev_execution_date"]
92 | prev_execution_date_success = context["prev_execution_date_success"]
|
AIR302_context.py:91:35: AIR302 `prev_execution_date` is removed in Airflow 3.0
|
89 | prev_ds = context["prev_ds"]
90 | prev_ds_nodash = context["prev_ds_nodash"]
91 | prev_execution_date = context["prev_execution_date"]
| ^^^^^^^^^^^^^^^^^^^^^ AIR302
92 | prev_execution_date_success = context["prev_execution_date_success"]
93 | tomorrow_ds = context["tomorrow_ds"]
|
AIR302_context.py:92:43: AIR302 `prev_execution_date_success` is removed in Airflow 3.0
|
90 | prev_ds_nodash = context["prev_ds_nodash"]
91 | prev_execution_date = context["prev_execution_date"]
92 | prev_execution_date_success = context["prev_execution_date_success"]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AIR302
93 | tomorrow_ds = context["tomorrow_ds"]
94 | yesterday_ds = context["yesterday_ds"]
|
AIR302_context.py:93:27: AIR302 `tomorrow_ds` is removed in Airflow 3.0
|
91 | prev_execution_date = context["prev_execution_date"]
92 | prev_execution_date_success = context["prev_execution_date_success"]
93 | tomorrow_ds = context["tomorrow_ds"]
| ^^^^^^^^^^^^^ AIR302
94 | yesterday_ds = context["yesterday_ds"]
95 | yesterday_ds_nodash = context["yesterday_ds_nodash"]
|
AIR302_context.py:94:28: AIR302 `yesterday_ds` is removed in Airflow 3.0
|
92 | prev_execution_date_success = context["prev_execution_date_success"]
93 | tomorrow_ds = context["tomorrow_ds"]
94 | yesterday_ds = context["yesterday_ds"]
| ^^^^^^^^^^^^^^ AIR302
95 | yesterday_ds_nodash = context["yesterday_ds_nodash"]
|
AIR302_context.py:95:35: AIR302 `yesterday_ds_nodash` is removed in Airflow 3.0
|
93 | tomorrow_ds = context["tomorrow_ds"]
94 | yesterday_ds = context["yesterday_ds"]
95 | yesterday_ds_nodash = context["yesterday_ds_nodash"]
| ^^^^^^^^^^^^^^^^^^^^^ AIR302
96 |
97 | class CustomOperator(BaseOperator):
|
AIR302_context.py:112:45: AIR302 `execution_date` is removed in Airflow 3.0
|
111 | @task
112 | def access_invalid_argument_task_out_of_dag(execution_date, tomorrow_ds, logical_date, **context):
| ^^^^^^^^^^^^^^ AIR302
113 | print("execution date", execution_date)
114 | print("access invalid key", context.get("conf"))
|
AIR302_context.py:112:61: AIR302 `tomorrow_ds` is removed in Airflow 3.0
|
111 | @task
112 | def access_invalid_argument_task_out_of_dag(execution_date, tomorrow_ds, logical_date, **context):
| ^^^^^^^^^^^ AIR302
113 | print("execution date", execution_date)
114 | print("access invalid key", context.get("conf"))
|
AIR302_context.py:114:45: AIR302 `conf` is removed in Airflow 3.0
|
112 | def access_invalid_argument_task_out_of_dag(execution_date, tomorrow_ds, logical_date, **context):
113 | print("execution date", execution_date)
114 | print("access invalid key", context.get("conf"))
| ^^^^^^ AIR302
115 |
116 | @task(task_id="print_the_context")
|
AIR302_context.py:120:22: AIR302 `tomorrow_ds` is removed in Airflow 3.0
|
118 | """Print the Airflow context and ds variable from the context."""
119 | print(ds)
120 | print(kwargs.get("tomorrow_ds"))
| ^^^^^^^^^^^^^ AIR302
121 | c = get_current_context()
122 | c.get("execution_date")
|
AIR302_context.py:122:11: AIR302 `execution_date` is removed in Airflow 3.0
|
120 | print(kwargs.get("tomorrow_ds"))
121 | c = get_current_context()
122 | c.get("execution_date")
| ^^^^^^^^^^^^^^^^ AIR302
123 |
124 | class CustomOperatorNew(BaseOperator):
|

View File

@@ -1,9 +1,12 @@
use ruff_diagnostics::{Diagnostic, Violation};
use ruff_macros::{derive_message_formats, ViolationMetadata};
use ruff_python_ast::helpers::resolve_imported_module_path;
use ruff_python_ast::{Alias, AnyNodeRef, Stmt, StmtImport, StmtImportFrom};
use ruff_text_size::Ranged;
use std::borrow::Cow;
use crate::checkers::ast::Checker;
use crate::rules::flake8_tidy_imports::matchers::NameMatchPolicy;
use crate::rules::flake8_tidy_imports::matchers::{MatchName, MatchNameOrParent, NameMatchPolicy};
/// ## What it does
/// Checks for module-level imports that should instead be imported lazily
@@ -53,28 +56,131 @@ impl Violation for BannedModuleLevelImports {
}
/// TID253
pub(crate) fn banned_module_level_imports<T: Ranged>(
checker: &mut Checker,
policy: &NameMatchPolicy,
node: &T,
) {
pub(crate) fn banned_module_level_imports(checker: &mut Checker, stmt: &Stmt) {
if !checker.semantic().at_top_level() {
return;
}
if let Some(banned_module) = policy.find(
checker
.settings
.flake8_tidy_imports
.banned_module_level_imports
.iter()
.map(AsRef::as_ref),
) {
checker.diagnostics.push(Diagnostic::new(
BannedModuleLevelImports {
name: banned_module,
},
node.range(),
));
for (policy, node) in &BannedModuleImportPolicies::new(stmt, checker) {
if let Some(banned_module) = policy.find(
checker
.settings
.flake8_tidy_imports
.banned_module_level_imports(),
) {
checker.diagnostics.push(Diagnostic::new(
BannedModuleLevelImports {
name: banned_module,
},
node.range(),
));
}
}
}
pub(crate) enum BannedModuleImportPolicies<'a> {
Import(&'a StmtImport),
ImportFrom {
module: Option<Cow<'a, str>>,
node: &'a StmtImportFrom,
},
NonImport,
}
impl<'a> BannedModuleImportPolicies<'a> {
pub(crate) fn new(stmt: &'a Stmt, checker: &Checker) -> Self {
match stmt {
Stmt::Import(import) => Self::Import(import),
Stmt::ImportFrom(import @ StmtImportFrom { module, level, .. }) => {
let module = resolve_imported_module_path(
*level,
module.as_deref(),
checker.module.qualified_name(),
);
Self::ImportFrom {
module,
node: import,
}
}
_ => Self::NonImport,
}
}
}
impl<'a> IntoIterator for &'a BannedModuleImportPolicies<'a> {
type Item = <Self::IntoIter as Iterator>::Item;
type IntoIter = BannedModuleImportPoliciesIter<'a>;
fn into_iter(self) -> Self::IntoIter {
match self {
BannedModuleImportPolicies::Import(import) => {
BannedModuleImportPoliciesIter::Import(import.names.iter())
}
BannedModuleImportPolicies::ImportFrom { module, node } => {
BannedModuleImportPoliciesIter::ImportFrom {
module: module.as_deref(),
names: node.names.iter(),
import: Some(node),
}
}
BannedModuleImportPolicies::NonImport => BannedModuleImportPoliciesIter::NonImport,
}
}
}
pub(crate) enum BannedModuleImportPoliciesIter<'a> {
Import(std::slice::Iter<'a, Alias>),
ImportFrom {
module: Option<&'a str>,
names: std::slice::Iter<'a, Alias>,
import: Option<&'a StmtImportFrom>,
},
NonImport,
}
impl<'a> Iterator for BannedModuleImportPoliciesIter<'a> {
type Item = (NameMatchPolicy<'a>, AnyNodeRef<'a>);
fn next(&mut self) -> Option<Self::Item> {
match self {
Self::Import(names) => {
let name = names.next()?;
Some((
NameMatchPolicy::MatchNameOrParent(MatchNameOrParent { module: &name.name }),
name.into(),
))
}
Self::ImportFrom {
module,
import,
names,
} => {
let module = module.as_ref()?;
if let Some(import) = import.take() {
return Some((
NameMatchPolicy::MatchNameOrParent(MatchNameOrParent { module }),
import.into(),
));
}
loop {
let alias = names.next()?;
if &alias.name == "*" {
continue;
}
break Some((
NameMatchPolicy::MatchName(MatchName {
module,
member: &alias.name,
}),
alias.into(),
));
}
}
Self::NonImport => None,
}
}
}

View File

@@ -46,6 +46,12 @@ pub struct Settings {
pub banned_module_level_imports: Vec<String>,
}
impl Settings {
pub fn banned_module_level_imports(&self) -> impl Iterator<Item = &str> {
self.banned_module_level_imports.iter().map(AsRef::as_ref)
}
}
impl Display for Settings {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
display_settings! {

View File

@@ -13,7 +13,7 @@ mod tests {
use test_case::test_case;
use crate::registry::Rule;
use crate::rules::pylint;
use crate::rules::{flake8_tidy_imports, pylint};
use crate::settings::types::{PreviewMode, PythonVersion};
use crate::settings::LinterSettings;
@@ -412,6 +412,30 @@ mod tests {
Ok(())
}
#[test]
fn import_outside_top_level_with_banned() -> Result<()> {
let diagnostics = test_path(
Path::new("pylint/import_outside_top_level_with_banned.py"),
&LinterSettings {
preview: PreviewMode::Enabled,
flake8_tidy_imports: flake8_tidy_imports::settings::Settings {
banned_module_level_imports: vec![
"foo_banned".to_string(),
"pkg_banned".to_string(),
"pkg.bar_banned".to_string(),
],
..Default::default()
},
..LinterSettings::for_rules(vec![
Rule::BannedModuleLevelImports,
Rule::ImportOutsideTopLevel,
])
},
)?;
assert_messages!(diagnostics);
Ok(())
}
#[test_case(
Rule::RepeatedEqualityComparison,
Path::new("repeated_equality_comparison.py")

View File

@@ -3,7 +3,10 @@ use ruff_macros::{derive_message_formats, ViolationMetadata};
use ruff_python_ast::Stmt;
use ruff_text_size::Ranged;
use crate::checkers::ast::Checker;
use crate::rules::flake8_tidy_imports::rules::BannedModuleImportPolicies;
use crate::{
checkers::ast::Checker, codes::Rule, rules::flake8_tidy_imports::matchers::NameMatchPolicy,
};
/// ## What it does
/// Checks for `import` statements outside of a module's top-level scope, such
@@ -54,9 +57,45 @@ impl Violation for ImportOutsideTopLevel {
/// C0415
pub(crate) fn import_outside_top_level(checker: &mut Checker, stmt: &Stmt) {
if !checker.semantic().current_scope().kind.is_module() {
checker
.diagnostics
.push(Diagnostic::new(ImportOutsideTopLevel, stmt.range()));
if checker.semantic().current_scope().kind.is_module() {
// "Top-level" imports are allowed
return;
}
// Check if any of the non-top-level imports are banned by TID253
// before emitting the diagnostic to avoid conflicts.
if checker.enabled(Rule::BannedModuleLevelImports) {
let mut all_aliases_banned = true;
let mut has_alias = false;
for (policy, node) in &BannedModuleImportPolicies::new(stmt, checker) {
if node.is_alias() {
has_alias = true;
all_aliases_banned &= is_banned_module_level_import(&policy, checker);
}
// If the entire import is banned
else if is_banned_module_level_import(&policy, checker) {
return;
}
}
if has_alias && all_aliases_banned {
return;
}
}
// Emit the diagnostic
checker
.diagnostics
.push(Diagnostic::new(ImportOutsideTopLevel, stmt.range()));
}
fn is_banned_module_level_import(policy: &NameMatchPolicy, checker: &Checker) -> bool {
policy
.find(
checker
.settings
.flake8_tidy_imports
.banned_module_level_imports(),
)
.is_some()
}

View File

@@ -0,0 +1,94 @@
---
source: crates/ruff_linter/src/rules/pylint/mod.rs
---
import_outside_top_level_with_banned.py:9:5: PLC0415 `import` should be at the top-level of a file
|
8 | def import_in_function():
9 | import symtable # [import-outside-toplevel]
| ^^^^^^^^^^^^^^^ PLC0415
10 | import os, sys # [import-outside-toplevel]
11 | import time as thyme # [import-outside-toplevel]
|
import_outside_top_level_with_banned.py:10:5: PLC0415 `import` should be at the top-level of a file
|
8 | def import_in_function():
9 | import symtable # [import-outside-toplevel]
10 | import os, sys # [import-outside-toplevel]
| ^^^^^^^^^^^^^^ PLC0415
11 | import time as thyme # [import-outside-toplevel]
12 | import random as rand, socket as sock # [import-outside-toplevel]
|
import_outside_top_level_with_banned.py:11:5: PLC0415 `import` should be at the top-level of a file
|
9 | import symtable # [import-outside-toplevel]
10 | import os, sys # [import-outside-toplevel]
11 | import time as thyme # [import-outside-toplevel]
| ^^^^^^^^^^^^^^^^^^^^ PLC0415
12 | import random as rand, socket as sock # [import-outside-toplevel]
13 | from collections import defaultdict # [import-outside-toplevel]
|
import_outside_top_level_with_banned.py:12:5: PLC0415 `import` should be at the top-level of a file
|
10 | import os, sys # [import-outside-toplevel]
11 | import time as thyme # [import-outside-toplevel]
12 | import random as rand, socket as sock # [import-outside-toplevel]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ PLC0415
13 | from collections import defaultdict # [import-outside-toplevel]
14 | from math import sin as sign, cos as cosplay # [import-outside-toplevel]
|
import_outside_top_level_with_banned.py:13:5: PLC0415 `import` should be at the top-level of a file
|
11 | import time as thyme # [import-outside-toplevel]
12 | import random as rand, socket as sock # [import-outside-toplevel]
13 | from collections import defaultdict # [import-outside-toplevel]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ PLC0415
14 | from math import sin as sign, cos as cosplay # [import-outside-toplevel]
|
import_outside_top_level_with_banned.py:14:5: PLC0415 `import` should be at the top-level of a file
|
12 | import random as rand, socket as sock # [import-outside-toplevel]
13 | from collections import defaultdict # [import-outside-toplevel]
14 | from math import sin as sign, cos as cosplay # [import-outside-toplevel]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ PLC0415
15 |
16 | # these should be allowed due to TID253 top-level ban
|
import_outside_top_level_with_banned.py:24:5: PLC0415 `import` should be at the top-level of a file
|
23 | # this should still trigger an error due to multiple imports
24 | from pkg import foo_allowed, bar_banned # [import-outside-toplevel]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ PLC0415
25 |
26 | class ClassWithImports:
|
import_outside_top_level_with_banned.py:27:5: PLC0415 `import` should be at the top-level of a file
|
26 | class ClassWithImports:
27 | import tokenize # [import-outside-toplevel]
| ^^^^^^^^^^^^^^^ PLC0415
28 |
29 | def __init__(self):
|
import_outside_top_level_with_banned.py:30:9: PLC0415 `import` should be at the top-level of a file
|
29 | def __init__(self):
30 | import trace # [import-outside-toplevel]
| ^^^^^^^^^^^^ PLC0415
31 |
32 | # these should be allowed due to TID253 top-level ban
|
import_outside_top_level_with_banned.py:40:9: PLC0415 `import` should be at the top-level of a file
|
39 | # this should still trigger an error due to multiple imports
40 | from pkg import foo_allowed, bar_banned # [import-outside-toplevel]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ PLC0415
|

View File

@@ -6,6 +6,7 @@ use ruff_python_ast::{Arguments, Expr, ExprCall, ExprNumberLiteral, Number};
use ruff_python_semantic::analyze::type_inference::{NumberLike, PythonType, ResolvedPythonType};
use ruff_python_semantic::analyze::typing;
use ruff_python_semantic::SemanticModel;
use ruff_source_file::find_newline;
use ruff_text_size::Ranged;
/// ## What it does
@@ -59,7 +60,7 @@ pub(crate) fn unnecessary_round(checker: &mut Checker, call: &ExprCall) {
return;
}
let applicability = match rounded_value {
let mut applicability = match rounded_value {
// ```python
// some_int: int
//
@@ -86,6 +87,10 @@ pub(crate) fn unnecessary_round(checker: &mut Checker, call: &ExprCall) {
_ => return,
};
if checker.comment_ranges().intersects(call.range()) {
applicability = Applicability::Unsafe;
};
let edit = unwrap_round_call(call, rounded, checker.semantic(), checker.locator());
let fix = Fix::applicable_edit(edit, applicability);
@@ -196,13 +201,13 @@ fn unwrap_round_call(
locator: &Locator,
) -> Edit {
let rounded_expr = locator.slice(rounded.range());
let has_parent_expr = semantic.current_expression_parent().is_some();
let new_content = if has_parent_expr || rounded.is_named_expr() {
format!("({rounded_expr})")
} else {
rounded_expr.to_string()
};
let new_content =
if has_parent_expr || rounded.is_named_expr() || find_newline(rounded_expr).is_some() {
format!("({rounded_expr})")
} else {
rounded_expr.to_string()
};
Edit::range_replacement(new_content, call.range)
}

View File

@@ -1,6 +1,5 @@
---
source: crates/ruff_linter/src/rules/ruff/mod.rs
snapshot_kind: text
---
RUF057.py:6:1: RUF057 [*] Value being rounded is already an integer
|
@@ -181,3 +180,94 @@ RUF057.py:44:1: RUF057 [*] Value being rounded is already an integer
45 45 | round(inferred_int, -2) # No error
46 46 | round(inferred_int, inferred_int) # No error
47 47 | round(inferred_int, 3 + 4) # No error
RUF057.py:71:1: RUF057 [*] Value being rounded is already an integer
|
69 | # contains newline.
70 | # See https://github.com/astral-sh/ruff/issues/15598
71 | / round(-
72 | | 1)
| |__^ RUF057
73 | round(1
74 | *1
|
= help: Remove unnecessary `round` call
Safe fix
68 68 | # Fixes should preserve parentheses when argument
69 69 | # contains newline.
70 70 | # See https://github.com/astral-sh/ruff/issues/15598
71 |-round(-
71 |+(-
72 72 | 1)
73 73 | round(1
74 74 | *1
RUF057.py:73:1: RUF057 [*] Value being rounded is already an integer
|
71 | round(-
72 | 1)
73 | / round(1
74 | | *1
75 | | )
| |_^ RUF057
76 |
77 | # fix should be unsafe if comment is in call range
|
= help: Remove unnecessary `round` call
Safe fix
70 70 | # See https://github.com/astral-sh/ruff/issues/15598
71 71 | round(-
72 72 | 1)
73 |-round(1
74 |-*1
75 |-)
73 |+(1
74 |+*1)
76 75 |
77 76 | # fix should be unsafe if comment is in call range
78 77 | round(# a comment
RUF057.py:78:1: RUF057 [*] Value being rounded is already an integer
|
77 | # fix should be unsafe if comment is in call range
78 | / round(# a comment
79 | | 17
80 | | )
| |_^ RUF057
81 | round(
82 | 17 # a comment
|
= help: Remove unnecessary `round` call
Unsafe fix
75 75 | )
76 76 |
77 77 | # fix should be unsafe if comment is in call range
78 |-round(# a comment
79 78 | 17
80 |-)
81 79 | round(
82 80 | 17 # a comment
83 81 | )
RUF057.py:81:1: RUF057 [*] Value being rounded is already an integer
|
79 | 17
80 | )
81 | / round(
82 | | 17 # a comment
83 | | )
| |_^ RUF057
|
= help: Remove unnecessary `round` call
Unsafe fix
78 78 | round(# a comment
79 79 | 17
80 80 | )
81 |-round(
82 |- 17 # a comment
83 |-)
81 |+17

View File

@@ -8,13 +8,14 @@ your project. For a more detailed overview, see [_Configuring Ruff_](configurati
To start, we'll initialize a project using [uv](https://docs.astral.sh/uv/):
```console
$ uv init numbers
$ uv init --lib numbers
```
This command creates a Python project with the following structure:
```text
numbers
├── README.md
├── pyproject.toml
└── src
└── numbers

1
ruff.schema.json generated
View File

@@ -3653,7 +3653,6 @@
"PLW0",
"PLW01",
"PLW010",
"PLW0101",
"PLW0108",
"PLW012",
"PLW0120",

View File

@@ -68,7 +68,7 @@ class Knot(Tool):
)
def cold_command(self, project: Project, venv: Venv) -> Command:
command = [str(self.path), "-v"]
command = [str(self.path), "check", "-v"]
assert len(project.include) < 2, "Knot doesn't support multiple source folders"