Compare commits

...

11 Commits

Author SHA1 Message Date
Charlie Marsh
1099c76fde Recommend running prek against upstream 2026-01-11 13:42:46 -05:00
Charlie Marsh
28cde5d7a6 Set priority for prek to enable concurrent execution 2026-01-11 13:33:20 -05:00
Charlie Marsh
2487233716 Use prek in docuemntation 2026-01-11 13:30:28 -05:00
Charlie Marsh
2c68057c4b [ty] Preserve argument signature in @total_ordering (#22496)
## Summary

Closes https://github.com/astral-sh/ty/issues/2435.
2026-01-10 14:35:58 -05:00
Charlie Marsh
8e29be9c1c Add let chains preference to development guidelines (#22497) 2026-01-10 12:29:07 -05:00
Dylan
880513a013 Respect fmt: skip for multiple statements on same logical line (#22119)
This PR adjusts the logic for skipping formatting so that a `fmt: skip`
can affect multiple statements if they lie on the same line.

Specifically, a `fmt: skip` comment will now suppress all the statements
in the suite in which it appears whose range intersects the line
containing the skip directive. For example:

```python
x=[
'1'
];x=2 # fmt: skip
```

remains unchanged after formatting.

(Note that compound statements are somewhat special and were handled in
a previous PR - see #20633).


Closes #17331 and #11430.

Simplest to review commit by commit - the key diffs of interest are the
commit introducing the core logic, and the diff between the snapshots
introduced in the last commit (compared to the second commit).

# Implementation

On `main` we format a suite of statements by iterating through them. If
we meet a statement with a leading or trailing (own-line)`fmt: off`
comment, then we suppress formatting until we meet a `fmt: on` comment.
Otherwise we format the statement using its own formatting rule.

How are `fmt: skip` comments handled then? They are handled internally
to the formatting of each statement. Specifically, calling `.fmt` on a
statement node will first check to see if there is a trailing,
end-of-line `fmt: skip` (or `fmt: off`/`yapf: off`), and if so then
write the node with suppressed formatting.

In this PR we move the responsibility for handling `fmt: skip` into the
formatting logic of the suite itself. This is done as follows:

- Before beginning to format the suite, we do a pass through the
statements and collect the data of ranges with skipped formatting. More
specifically, we create a map with key given by the _first_ skipped
statement in a block and value a pair consisting of the _last_ skipped
statement and the _range_ to write verbatim.
- We iterate as before, but if we meet a statement that is a key in the
map constructed above, we pause to write the associated range verbatim.
We then advance the iterator to the last statement in the block and
proceed as before.

## Addendum on range formatting

We also had to make some changes to range formatting in order to support
this new behavior. For example, we want to make sure that

```python
<RANGE_START>x=1<RANGE_END>;x=2 # fmt: skip
```

formats verbatim, rather than becoming 

```python
x = 1;x=2 # fmt: skip
```

Recall that range formatting proceeds in two steps:
1. Find the smallest enclosing node containing the range AND that has
enough info to format the range (so it may be larger than you think,
e.g. a docstring has enclosing node given by the suite, not the string
itself.)
2. Carve out the formatted range from the result of formatting that
enclosing node.

We had to modify (1), since the suite knows how to format skipped nodes,
but nodes may not "know" they are skipped. To do this we altered the
`visit_body` bethod of the `FindEnclosingNode` visitor: now we iterate
through the statements and check for skipped ranges intersecting the
format range. If we find them, we return without descending. The result
is to consider the statement containing the suite as the enclosing node
in this case.
2026-01-10 14:56:58 +00:00
Charlie Marsh
046c5a46d8 [ty] Support dataclass_transform as a function call (#22378)
## Summary

Instead of just as a decorator.

Closes https://github.com/astral-sh/ty/issues/2319.
2026-01-10 08:45:45 -05:00
Micha Reiser
cfed34334c Update mypy primer pin (#22490) 2026-01-10 14:00:00 +01:00
Charlie Marsh
11cc324449 [ty] Detect invalid @total_ordering applications in non-decorator contexts (#22486)
## Summary

E.g., `ValidOrderedClass = total_ordering(HasOrderingMethod)`.
2026-01-09 19:37:58 -05:00
Charlie Marsh
c88e1a0663 [ty] Avoid emitting Liskov repeated violations from grandparent to child (#22484)
## Summary

If parent violates LSP against grandparent, and child has the same
violation (but matches parent), we no longer flag the LSP violation on
child, since it can't be fixed without violating parent.

If parent violates LSP against grandparent, and child violates LSP
against both parent and grandparent, we emit two diagnostics (one for
each violation).

If parent violates LSP against grandparent, and child violates LSP
against parent (but not grandparent), we flag it.

Closes https://github.com/astral-sh/ty/issues/2000.
2026-01-09 19:27:57 -05:00
William Woodruff
2c7ac17b1e Remove two old zizmor ignore comments (#22485) 2026-01-09 18:02:44 -05:00
49 changed files with 1892 additions and 598 deletions

View File

@@ -5,4 +5,4 @@ rustup component add clippy rustfmt
cargo install cargo-insta
cargo fetch
pip install maturin pre-commit
pip install maturin prek

View File

@@ -40,12 +40,12 @@ jobs:
- name: Install the latest version of uv
uses: astral-sh/setup-uv@681c641aba71e4a1c380be3ab5e12ad51f415867 # v7.1.6
with:
enable-cache: true # zizmor: ignore[cache-poisoning] acceptable risk for CloudFlare pages artifact
enable-cache: true
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
with:
workspaces: "ruff"
lookup-only: false # zizmor: ignore[cache-poisoning] acceptable risk for CloudFlare pages artifact
lookup-only: false
- name: Install Rust toolchain
run: rustup show

View File

@@ -21,15 +21,62 @@ exclude: |
)$
repos:
# Priority 0: Read-only hooks; hooks that modify disjoint file types.
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v6.0.0
hooks:
- id: check-merge-conflict
priority: 0
- repo: https://github.com/abravalheri/validate-pyproject
rev: v0.24.1
hooks:
- id: validate-pyproject
priority: 0
- repo: https://github.com/crate-ci/typos
rev: v1.40.0
hooks:
- id: typos
priority: 0
- repo: local
hooks:
- id: cargo-fmt
name: cargo fmt
entry: cargo fmt --
language: system
types: [rust]
pass_filenames: false # This makes it a lot faster
priority: 0
# Prettier
- repo: https://github.com/rbubley/mirrors-prettier
rev: v3.7.4
hooks:
- id: prettier
types: [yaml]
priority: 0
# zizmor detects security vulnerabilities in GitHub Actions workflows.
# Additional configuration for the tool is found in `.github/zizmor.yml`
- repo: https://github.com/zizmorcore/zizmor-pre-commit
rev: v1.19.0
hooks:
- id: zizmor
priority: 0
- repo: https://github.com/python-jsonschema/check-jsonschema
rev: 0.36.0
hooks:
- id: check-github-workflows
priority: 0
- repo: https://github.com/shellcheck-py/shellcheck-py
rev: v0.11.0.1
hooks:
- id: shellcheck
priority: 0
- repo: https://github.com/executablebooks/mdformat
rev: 1.0.0
@@ -44,7 +91,20 @@ repos:
docs/formatter/black\.md
| docs/\w+\.md
)$
priority: 0
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.14.10
hooks:
- id: ruff-format
priority: 0
- id: ruff-check
args: [--fix, --exit-non-zero-on-fix]
types_or: [python, pyi]
require_serial: true
priority: 1
# Priority 1: Second-pass fixers (e.g., markdownlint-fix runs after mdformat).
- repo: https://github.com/igorshubovych/markdownlint-cli
rev: v0.47.0
hooks:
@@ -54,7 +114,9 @@ repos:
docs/formatter/black\.md
| docs/\w+\.md
)$
priority: 1
# Priority 2: blacken-docs runs after markdownlint-fix (both modify markdown).
- repo: https://github.com/adamchainz/blacken-docs
rev: 1.20.0
hooks:
@@ -68,48 +130,7 @@ repos:
)$
additional_dependencies:
- black==25.12.0
- repo: https://github.com/crate-ci/typos
rev: v1.40.0
hooks:
- id: typos
- repo: local
hooks:
- id: cargo-fmt
name: cargo fmt
entry: cargo fmt --
language: system
types: [rust]
pass_filenames: false # This makes it a lot faster
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.14.10
hooks:
- id: ruff-format
- id: ruff-check
args: [--fix, --exit-non-zero-on-fix]
types_or: [python, pyi]
require_serial: true
# Prettier
- repo: https://github.com/rbubley/mirrors-prettier
rev: v3.7.4
hooks:
- id: prettier
types: [yaml]
# zizmor detects security vulnerabilities in GitHub Actions workflows.
# Additional configuration for the tool is found in `.github/zizmor.yml`
- repo: https://github.com/zizmorcore/zizmor-pre-commit
rev: v1.19.0
hooks:
- id: zizmor
- repo: https://github.com/python-jsonschema/check-jsonschema
rev: 0.36.0
hooks:
- id: check-github-workflows
priority: 2
# `actionlint` hook, for verifying correct syntax in GitHub Actions workflows.
# Some additional configuration for `actionlint` can be found in `.github/actionlint.yaml`.
@@ -131,8 +152,4 @@ repos:
# and checks these with shellcheck. This is arguably its most useful feature,
# but the integration only works if shellcheck is installed
- "github.com/wasilibs/go-shellcheck/cmd/shellcheck@v0.11.1"
- repo: https://github.com/shellcheck-py/shellcheck-py
rev: v0.11.0.1
hooks:
- id: shellcheck
priority: 0

View File

@@ -65,6 +65,7 @@ When working on ty, PR titles should start with `[ty]` and be tagged with the `t
- All changes must be tested. If you're not testing your changes, you're not done.
- Get your tests to pass. If you didn't run the tests, your code does not work.
- Follow existing code style. Check neighboring files for patterns.
- Always run `uvx pre-commit run -a` at the end of a task.
- Always run `uvx prek run --from-ref @{upstream}` at the end of a task.
- Avoid writing significant amounts of new code. This is often a sign that we're missing an existing method or mechanism that could help solve the problem. Look for existing utilities first.
- Avoid falling back to patterns that require `panic!`, `unreachable!`, or `.unwrap()`. Instead, try to encode those constraints in the type system.
- Prefer let chains (`if let` combined with `&&`) over nested `if let` statements to reduce indentation and improve readability.

View File

@@ -57,8 +57,8 @@ You can optionally install pre-commit hooks to automatically run the validation
when making a commit:
```shell
uv tool install pre-commit
pre-commit install
uv tool install prek
prek install
```
We recommend [nextest](https://nexte.st/) to run Ruff's test suite (via `cargo nextest run`),
@@ -85,7 +85,7 @@ and that it passes both the lint and test validation checks:
```shell
cargo clippy --workspace --all-targets --all-features -- -D warnings # Rust linting
RUFF_UPDATE_SCHEMA=1 cargo test # Rust testing and updating ruff.schema.json
uvx pre-commit run --all-files --show-diff-on-failure # Rust and Python formatting, Markdown and Python linting, etc.
uvx prek run -a # Rust and Python formatting, Markdown and Python linting, etc.
```
These checks will run on GitHub Actions when you open your pull request, but running them locally

View File

@@ -231,7 +231,7 @@ static STATIC_FRAME: Benchmark = Benchmark::new(
max_dep_date: "2025-08-09",
python_version: PythonVersion::PY311,
},
1657,
1700,
);
#[track_caller]

View File

@@ -0,0 +1,133 @@
class Simple:
x=1
x=2 # fmt: skip
x=3
class Semicolon:
x=1
x=2;x=3 # fmt: skip
x=4
class TrailingSemicolon:
x=1
x=2;x=3 ; # fmt: skip
x=4
class SemicolonNewLogicalLine:
x=1;
x=2;x=3 # fmt: skip
x=4
class ManySemicolonOneLine:
x=1
x=2;x=3;x=4 # fmt: skip
x=5
class CompoundInSuite:
x=1
def foo(): y=1 # fmt: skip
x=2
class CompoundInSuiteNewline:
x=1
def foo():
y=1 # fmt: skip
x=2
class MultiLineSkip:
x=1
x = [
'1',
'2',
] # fmt: skip
class MultiLineSemicolon:
x=1
x = [
'1',
'2',
]; x=2 # fmt: skip
class LineContinuationSemicolonAfter:
x=1
x = ['a']\
; y=1 # fmt: skip
class LineContinuationSemicolonBefore:
x=1
x = ['a']; \
y=1 # fmt: skip
class LineContinuationSemicolonAndNewline:
x=1
x = ['a']; \
y=1 # fmt: skip
class LineContinuationSemicolonAndNewlineAndComment:
x=1
x = ['a']; \
# 1
y=1 # fmt: skip
class RepeatedLineContinuation:
x=1
x = ['a']; \
\
\
y=1 # fmt: skip
class MultiLineSemicolonComments:
x=1
# 1
x = [ # 2
'1', # 3
'2',
# 4
]; x=2 # 5 # fmt: skip # 6
class DocstringSkipped:
'''This is a docstring''' # fmt: skip
x=1
class MultilineDocstringSkipped:
'''This is a docstring
''' # fmt: skip
x=1
class FirstStatementNewlines:
x=1 # fmt: skip
class ChainingSemicolons:
x=[
'1',
'2',
'3',
];x=1;x=[
'1',
'2',
'3'
];x=1;x=1 # fmt: skip
class LotsOfComments:
# 1
x=[ # 2
'1', # 3
'2',
'3'
] ;x=2;x=3 # 4 # fmt: skip # 5
# 6
class MixingCompound:
def foo(): bar(); import zoo # fmt: skip
# https://github.com/astral-sh/ruff/issues/17331
def main() -> None:
import ipdb; ipdb.set_trace() # noqa: E402,E702,I001 # fmt: skip
# https://github.com/astral-sh/ruff/issues/11430
print(); print() # noqa # fmt: skip

View File

@@ -0,0 +1,22 @@
# 0
'''Module docstring
multiple lines''' # 1 # fmt: skip # 2
import a;import b; from c import (
x,y,z
); import f # fmt: skip
x=1;x=2;x=3;x=4 # fmt: skip
# 1
x=[ # 2
'1', # 3
'2',
'3'
];x=1;x=1 # 4 # fmt: skip # 5
# 6
def foo(): x=[
'1',
'2',
];x=1 # fmt: skip

View File

@@ -0,0 +1,68 @@
class Simple:
# Range comprises skip range
x=1
<RANGE_START>x=2 <RANGE_END># fmt: skip
x=3
class Semicolon:
# Range is part of skip range
x=1
x=2;<RANGE_START>x=3<RANGE_END> # fmt: skip
x=4
class FormatFirst:
x=1
<RANGE_START>x=2<RANGE_END>;x=3 # fmt: skip
x=4
class FormatMiddle:
x=1
x=2;<RANGE_START>x=3<RANGE_END>;x=4 # fmt: skip
x=5
class SemicolonNewLogicalLine:
# Range overlaps on right side
<RANGE_START>x=1;
x=2<RANGE_END>;x=3 # fmt: skip
x=4
class SemicolonNewLogicalLine:
# Range overlaps on left side
x=1;
x=2;<RANGE_START>x=3 # fmt: skip
x=4<RANGE_END>
class ManySemicolonOneLine:
x=1
x=2;x=3;x=4 # fmt: skip
x=5
class CompoundInSuite:
x=1
<RANGE_START>def foo(): y=1 <RANGE_END># fmt: skip
x=2
class CompoundInSuiteNewline:
x=1
def foo():
y=1 # fmt: skip
x=2
class MultiLineSkip:
# Range inside statement
x=1
x = <RANGE_START>[
'1',
'2',<RANGE_END>
] # fmt: skip
class LotsOfComments:
# 1
x=[ # 2
'1', # 3<RANGE_START>
'2',
'3'
] ;x=2;x=3 # 4<RANGE_END> # fmt: skip # 5
# 6

View File

@@ -24,7 +24,6 @@ pub use crate::options::{
};
use crate::range::is_logical_line;
pub use crate::shared_traits::{AsFormat, FormattedIter, FormattedIterExt, IntoFormat};
use crate::verbatim::suppressed_node;
pub(crate) mod builders;
pub mod cli;
@@ -61,51 +60,39 @@ where
let node_ref = AnyNodeRef::from(node);
let node_comments = comments.leading_dangling_trailing(node_ref);
if self.is_suppressed(node_comments.trailing, f.context()) {
suppressed_node(node_ref).fmt(f)
} else {
leading_comments(node_comments.leading).fmt(f)?;
leading_comments(node_comments.leading).fmt(f)?;
// Emit source map information for nodes that are valid "narrowing" targets
// in range formatting. Never emit source map information if they're disabled
// for performance reasons.
let emit_source_position = (is_logical_line(node_ref) || node_ref.is_mod_module())
&& f.options().source_map_generation().is_enabled();
// Emit source map information for nodes that are valid "narrowing" targets
// in range formatting. Never emit source map information if they're disabled
// for performance reasons.
let emit_source_position = (is_logical_line(node_ref) || node_ref.is_mod_module())
&& f.options().source_map_generation().is_enabled();
emit_source_position
.then_some(source_position(node.start()))
.fmt(f)?;
emit_source_position
.then_some(source_position(node.start()))
.fmt(f)?;
self.fmt_fields(node, f)?;
self.fmt_fields(node, f)?;
debug_assert!(
node_comments
.dangling
.iter()
.all(SourceComment::is_formatted),
"The node has dangling comments that need to be formatted manually. Add the special dangling comments handling to `fmt_fields`."
);
debug_assert!(
node_comments
.dangling
.iter()
.all(SourceComment::is_formatted),
"The node has dangling comments that need to be formatted manually. Add the special dangling comments handling to `fmt_fields`."
);
write!(
f,
[
emit_source_position.then_some(source_position(node.end())),
trailing_comments(node_comments.trailing)
]
)
}
write!(
f,
[
emit_source_position.then_some(source_position(node.end())),
trailing_comments(node_comments.trailing)
]
)
}
/// Formats the node's fields.
fn fmt_fields(&self, item: &N, f: &mut PyFormatter) -> FormatResult<()>;
fn is_suppressed(
&self,
_trailing_comments: &[SourceComment],
_context: &PyFormatContext,
) -> bool {
false
}
}
#[derive(Error, Debug, salsa::Update, PartialEq, Eq)]

View File

@@ -1,9 +1,10 @@
use ruff_formatter::write;
use ruff_python_ast::Decorator;
use ruff_text_size::Ranged;
use crate::comments::SourceComment;
use crate::expression::maybe_parenthesize_expression;
use crate::expression::parentheses::Parenthesize;
use crate::verbatim::verbatim_text;
use crate::{has_skip_comment, prelude::*};
#[derive(Default)]
@@ -11,26 +12,27 @@ pub struct FormatDecorator;
impl FormatNodeRule<Decorator> for FormatDecorator {
fn fmt_fields(&self, item: &Decorator, f: &mut PyFormatter) -> FormatResult<()> {
let Decorator {
expression,
range: _,
node_index: _,
} = item;
let comments = f.context().comments();
let trailing = comments.trailing(item);
write!(
f,
[
token("@"),
maybe_parenthesize_expression(expression, item, Parenthesize::Optional)
]
)
}
if has_skip_comment(trailing, f.context().source()) {
comments.mark_verbatim_node_comments_formatted(item.into());
fn is_suppressed(
&self,
trailing_comments: &[SourceComment],
context: &PyFormatContext,
) -> bool {
has_skip_comment(trailing_comments, context.source())
verbatim_text(item.range()).fmt(f)
} else {
let Decorator {
expression,
range: _,
node_index: _,
} = item;
write!(
f,
[
token("@"),
maybe_parenthesize_expression(expression, item, Parenthesize::Optional)
]
)
}
}
}

View File

@@ -15,7 +15,7 @@ use ruff_text_size::{Ranged, TextLen, TextRange, TextSize};
use crate::comments::Comments;
use crate::context::{IndentLevel, NodeLevel};
use crate::prelude::*;
use crate::statement::suite::DocstringStmt;
use crate::statement::suite::{DocstringStmt, skip_range};
use crate::verbatim::{ends_suppression, starts_suppression};
use crate::{FormatModuleError, PyFormatOptions, format_module_source};
@@ -251,7 +251,29 @@ impl<'ast> SourceOrderVisitor<'ast> for FindEnclosingNode<'_, 'ast> {
// We only visit statements that aren't suppressed that's why we don't need to track the suppression
// state in a stack. Assert that this assumption is safe.
debug_assert!(self.suppressed.is_no());
walk_body(self, body);
let mut iter = body.iter();
while let Some(stmt) = iter.next() {
// If the range intersects a skip range then we need to
// format the entire suite to properly handle the case
// where a `fmt: skip` affects multiple statements.
//
// For example, in the case
//
// ```
// <RANGE_START>x=1<RANGE_END>;x=2 # fmt: skip
// ```
//
// the statement `x=1` does not "know" that it is
// suppressed, but the suite does.
if let Some(verbatim_range) = skip_range(stmt, iter.as_slice(), self.context)
&& verbatim_range.intersect(self.range).is_some()
{
break;
}
self.visit_stmt(stmt);
}
self.suppressed = Suppressed::No;
}
}
@@ -561,7 +583,7 @@ impl NarrowRange<'_> {
}
pub(crate) const fn is_logical_line(node: AnyNodeRef) -> bool {
// Make sure to update [`FormatEnclosingLine`] when changing this.
// Make sure to update [`FormatEnclosingNode`] when changing this.
node.is_statement()
|| node.is_decorator()
|| node.is_except_handler()

View File

@@ -1,14 +1,13 @@
use ruff_formatter::write;
use ruff_python_ast::StmtAnnAssign;
use crate::comments::SourceComment;
use crate::expression::is_splittable_expression;
use crate::expression::parentheses::Parentheses;
use crate::prelude::*;
use crate::statement::stmt_assign::{
AnyAssignmentOperator, AnyBeforeOperator, FormatStatementsLastExpression,
};
use crate::statement::trailing_semicolon;
use crate::{has_skip_comment, prelude::*};
#[derive(Default)]
pub struct FormatStmtAnnAssign;
@@ -84,12 +83,4 @@ impl FormatNodeRule<StmtAnnAssign> for FormatStmtAnnAssign {
Ok(())
}
fn is_suppressed(
&self,
trailing_comments: &[SourceComment],
context: &PyFormatContext,
) -> bool {
has_skip_comment(trailing_comments, context.source())
}
}

View File

@@ -2,10 +2,9 @@ use ruff_formatter::prelude::{space, token};
use ruff_formatter::write;
use ruff_python_ast::StmtAssert;
use crate::comments::SourceComment;
use crate::expression::maybe_parenthesize_expression;
use crate::expression::parentheses::Parenthesize;
use crate::{has_skip_comment, prelude::*};
use crate::prelude::*;
#[derive(Default)]
pub struct FormatStmtAssert;
@@ -41,12 +40,4 @@ impl FormatNodeRule<StmtAssert> for FormatStmtAssert {
Ok(())
}
fn is_suppressed(
&self,
trailing_comments: &[SourceComment],
context: &PyFormatContext,
) -> bool {
has_skip_comment(trailing_comments, context.source())
}
}

View File

@@ -19,6 +19,7 @@ use crate::expression::{
maybe_parenthesize_expression,
};
use crate::other::interpolated_string::InterpolatedStringLayout;
use crate::prelude::*;
use crate::preview::is_parenthesize_lambda_bodies_enabled;
use crate::statement::trailing_semicolon;
use crate::string::StringLikeExtensions;
@@ -26,7 +27,6 @@ use crate::string::implicit::{
FormatImplicitConcatenatedStringExpanded, FormatImplicitConcatenatedStringFlat,
ImplicitConcatenatedLayout,
};
use crate::{has_skip_comment, prelude::*};
#[derive(Default)]
pub struct FormatStmtAssign;
@@ -104,14 +104,6 @@ impl FormatNodeRule<StmtAssign> for FormatStmtAssign {
Ok(())
}
fn is_suppressed(
&self,
trailing_comments: &[SourceComment],
context: &PyFormatContext,
) -> bool {
has_skip_comment(trailing_comments, context.source())
}
}
/// Formats a single target with the equal operator.

View File

@@ -1,15 +1,14 @@
use ruff_formatter::write;
use ruff_python_ast::StmtAugAssign;
use crate::comments::SourceComment;
use crate::expression::parentheses::is_expression_parenthesized;
use crate::prelude::*;
use crate::statement::stmt_assign::{
AnyAssignmentOperator, AnyBeforeOperator, FormatStatementsLastExpression,
has_target_own_parentheses,
};
use crate::statement::trailing_semicolon;
use crate::{AsFormat, FormatNodeRule};
use crate::{has_skip_comment, prelude::*};
#[derive(Default)]
pub struct FormatStmtAugAssign;
@@ -62,12 +61,4 @@ impl FormatNodeRule<StmtAugAssign> for FormatStmtAugAssign {
Ok(())
}
fn is_suppressed(
&self,
trailing_comments: &[SourceComment],
context: &PyFormatContext,
) -> bool {
has_skip_comment(trailing_comments, context.source())
}
}

View File

@@ -1,7 +1,6 @@
use ruff_python_ast::StmtBreak;
use crate::comments::SourceComment;
use crate::{has_skip_comment, prelude::*};
use crate::prelude::*;
#[derive(Default)]
pub struct FormatStmtBreak;
@@ -10,12 +9,4 @@ impl FormatNodeRule<StmtBreak> for FormatStmtBreak {
fn fmt_fields(&self, _item: &StmtBreak, f: &mut PyFormatter) -> FormatResult<()> {
token("break").fmt(f)
}
fn is_suppressed(
&self,
trailing_comments: &[SourceComment],
context: &PyFormatContext,
) -> bool {
has_skip_comment(trailing_comments, context.source())
}
}

View File

@@ -1,7 +1,6 @@
use ruff_python_ast::StmtContinue;
use crate::comments::SourceComment;
use crate::{has_skip_comment, prelude::*};
use crate::prelude::*;
#[derive(Default)]
pub struct FormatStmtContinue;
@@ -10,12 +9,4 @@ impl FormatNodeRule<StmtContinue> for FormatStmtContinue {
fn fmt_fields(&self, _item: &StmtContinue, f: &mut PyFormatter) -> FormatResult<()> {
token("continue").fmt(f)
}
fn is_suppressed(
&self,
trailing_comments: &[SourceComment],
context: &PyFormatContext,
) -> bool {
has_skip_comment(trailing_comments, context.source())
}
}

View File

@@ -3,10 +3,10 @@ use ruff_python_ast::StmtDelete;
use ruff_text_size::Ranged;
use crate::builders::{PyFormatterExtensions, parenthesize_if_expands};
use crate::comments::{SourceComment, dangling_node_comments};
use crate::comments::dangling_node_comments;
use crate::expression::maybe_parenthesize_expression;
use crate::expression::parentheses::Parenthesize;
use crate::{has_skip_comment, prelude::*};
use crate::prelude::*;
#[derive(Default)]
pub struct FormatStmtDelete;
@@ -57,12 +57,4 @@ impl FormatNodeRule<StmtDelete> for FormatStmtDelete {
}
}
}
fn is_suppressed(
&self,
trailing_comments: &[SourceComment],
context: &PyFormatContext,
) -> bool {
has_skip_comment(trailing_comments, context.source())
}
}

View File

@@ -1,12 +1,10 @@
use ruff_python_ast as ast;
use ruff_python_ast::{Expr, Operator, StmtExpr};
use crate::comments::SourceComment;
use crate::expression::maybe_parenthesize_expression;
use crate::expression::parentheses::Parenthesize;
use crate::prelude::*;
use crate::statement::trailing_semicolon;
use crate::{has_skip_comment, prelude::*};
#[derive(Default)]
pub struct FormatStmtExpr;
@@ -30,14 +28,6 @@ impl FormatNodeRule<StmtExpr> for FormatStmtExpr {
Ok(())
}
fn is_suppressed(
&self,
trailing_comments: &[SourceComment],
context: &PyFormatContext,
) -> bool {
has_skip_comment(trailing_comments, context.source())
}
}
const fn is_arithmetic_like(expression: &Expr) -> bool {

View File

@@ -1,8 +1,6 @@
use ruff_formatter::{format_args, write};
use ruff_python_ast::StmtGlobal;
use crate::comments::SourceComment;
use crate::has_skip_comment;
use crate::prelude::*;
#[derive(Default)]
@@ -47,12 +45,4 @@ impl FormatNodeRule<StmtGlobal> for FormatStmtGlobal {
)
}
}
fn is_suppressed(
&self,
trailing_comments: &[SourceComment],
context: &PyFormatContext,
) -> bool {
has_skip_comment(trailing_comments, context.source())
}
}

View File

@@ -1,8 +1,7 @@
use ruff_formatter::{format_args, write};
use ruff_python_ast::StmtImport;
use crate::comments::SourceComment;
use crate::{has_skip_comment, prelude::*};
use crate::prelude::*;
#[derive(Default)]
pub struct FormatStmtImport;
@@ -21,12 +20,4 @@ impl FormatNodeRule<StmtImport> for FormatStmtImport {
});
write!(f, [token("import"), space(), names])
}
fn is_suppressed(
&self,
trailing_comments: &[SourceComment],
context: &PyFormatContext,
) -> bool {
has_skip_comment(trailing_comments, context.source())
}
}

View File

@@ -3,9 +3,7 @@ use ruff_python_ast::StmtImportFrom;
use ruff_text_size::Ranged;
use crate::builders::{PyFormatterExtensions, TrailingComma, parenthesize_if_expands};
use crate::comments::SourceComment;
use crate::expression::parentheses::parenthesized;
use crate::has_skip_comment;
use crate::other::identifier::DotDelimitedIdentifier;
use crate::prelude::*;
@@ -72,12 +70,4 @@ impl FormatNodeRule<StmtImportFrom> for FormatStmtImportFrom {
.fmt(f)
}
}
fn is_suppressed(
&self,
trailing_comments: &[SourceComment],
context: &PyFormatContext,
) -> bool {
has_skip_comment(trailing_comments, context.source())
}
}

View File

@@ -1,8 +1,7 @@
use ruff_python_ast::StmtIpyEscapeCommand;
use ruff_text_size::Ranged;
use crate::comments::SourceComment;
use crate::{has_skip_comment, prelude::*};
use crate::prelude::*;
#[derive(Default)]
pub struct FormatStmtIpyEscapeCommand;
@@ -11,12 +10,4 @@ impl FormatNodeRule<StmtIpyEscapeCommand> for FormatStmtIpyEscapeCommand {
fn fmt_fields(&self, item: &StmtIpyEscapeCommand, f: &mut PyFormatter) -> FormatResult<()> {
source_text_slice(item.range()).fmt(f)
}
fn is_suppressed(
&self,
trailing_comments: &[SourceComment],
context: &PyFormatContext,
) -> bool {
has_skip_comment(trailing_comments, context.source())
}
}

View File

@@ -1,8 +1,6 @@
use ruff_formatter::{format_args, write};
use ruff_python_ast::StmtNonlocal;
use crate::comments::SourceComment;
use crate::has_skip_comment;
use crate::prelude::*;
#[derive(Default)]
@@ -47,12 +45,4 @@ impl FormatNodeRule<StmtNonlocal> for FormatStmtNonlocal {
)
}
}
fn is_suppressed(
&self,
trailing_comments: &[SourceComment],
context: &PyFormatContext,
) -> bool {
has_skip_comment(trailing_comments, context.source())
}
}

View File

@@ -1,7 +1,6 @@
use ruff_python_ast::StmtPass;
use crate::comments::SourceComment;
use crate::{has_skip_comment, prelude::*};
use crate::prelude::*;
#[derive(Default)]
pub struct FormatStmtPass;
@@ -10,12 +9,4 @@ impl FormatNodeRule<StmtPass> for FormatStmtPass {
fn fmt_fields(&self, _item: &StmtPass, f: &mut PyFormatter) -> FormatResult<()> {
token("pass").fmt(f)
}
fn is_suppressed(
&self,
trailing_comments: &[SourceComment],
context: &PyFormatContext,
) -> bool {
has_skip_comment(trailing_comments, context.source())
}
}

View File

@@ -1,10 +1,9 @@
use ruff_formatter::write;
use ruff_python_ast::StmtRaise;
use crate::comments::SourceComment;
use crate::expression::maybe_parenthesize_expression;
use crate::expression::parentheses::Parenthesize;
use crate::{has_skip_comment, prelude::*};
use crate::prelude::*;
#[derive(Default)]
pub struct FormatStmtRaise;
@@ -43,12 +42,4 @@ impl FormatNodeRule<StmtRaise> for FormatStmtRaise {
}
Ok(())
}
fn is_suppressed(
&self,
trailing_comments: &[SourceComment],
context: &PyFormatContext,
) -> bool {
has_skip_comment(trailing_comments, context.source())
}
}

View File

@@ -1,10 +1,9 @@
use ruff_formatter::write;
use ruff_python_ast::{Expr, StmtReturn};
use crate::comments::SourceComment;
use crate::expression::expr_tuple::TupleParentheses;
use crate::prelude::*;
use crate::statement::stmt_assign::FormatStatementsLastExpression;
use crate::{has_skip_comment, prelude::*};
#[derive(Default)]
pub struct FormatStmtReturn;
@@ -43,12 +42,4 @@ impl FormatNodeRule<StmtReturn> for FormatStmtReturn {
None => Ok(()),
}
}
fn is_suppressed(
&self,
trailing_comments: &[SourceComment],
context: &PyFormatContext,
) -> bool {
has_skip_comment(trailing_comments, context.source())
}
}

View File

@@ -1,11 +1,10 @@
use ruff_formatter::write;
use ruff_python_ast::StmtTypeAlias;
use crate::comments::SourceComment;
use crate::prelude::*;
use crate::statement::stmt_assign::{
AnyAssignmentOperator, AnyBeforeOperator, FormatStatementsLastExpression,
};
use crate::{has_skip_comment, prelude::*};
#[derive(Default)]
pub struct FormatStmtTypeAlias;
@@ -42,12 +41,4 @@ impl FormatNodeRule<StmtTypeAlias> for FormatStmtTypeAlias {
]
)
}
fn is_suppressed(
&self,
trailing_comments: &[SourceComment],
context: &PyFormatContext,
) -> bool {
has_skip_comment(trailing_comments, context.source())
}
}

View File

@@ -4,11 +4,15 @@ use ruff_formatter::{
use ruff_python_ast::helpers::is_compound_statement;
use ruff_python_ast::{self as ast, Expr, PySourceType, Stmt, Suite};
use ruff_python_ast::{AnyNodeRef, StmtExpr};
use ruff_python_trivia::{lines_after, lines_after_ignoring_end_of_line_trivia, lines_before};
use ruff_python_trivia::{
SimpleTokenKind, SimpleTokenizer, lines_after, lines_after_ignoring_end_of_line_trivia,
lines_before,
};
use ruff_text_size::{Ranged, TextRange};
use crate::comments::{
Comments, LeadingDanglingTrailingComments, leading_comments, trailing_comments,
Comments, LeadingDanglingTrailingComments, has_skip_comment, leading_comments,
trailing_comments,
};
use crate::context::{NodeLevel, TopLevelStatementPosition, WithIndentLevel, WithNodeLevel};
use crate::other::string_literal::StringLiteralKind;
@@ -16,9 +20,9 @@ use crate::prelude::*;
use crate::preview::{
is_allow_newline_after_block_open_enabled, is_blank_line_before_decorated_class_in_stub_enabled,
};
use crate::statement::stmt_expr::FormatStmtExpr;
use crate::statement::trailing_semicolon;
use crate::verbatim::{
suppressed_node, write_suppressed_statements_starting_with_leading_comment,
write_skipped_statements, write_suppressed_statements_starting_with_leading_comment,
write_suppressed_statements_starting_with_trailing_comment,
};
@@ -152,7 +156,21 @@ impl FormatRule<Suite, PyFormatContext<'_>> for FormatSuite {
let first_comments = comments.leading_dangling_trailing(first);
let (mut preceding, mut empty_line_after_docstring) = if first_comments
let (mut preceding, mut empty_line_after_docstring) = if let Some(verbatim_range) =
skip_range(first.statement(), iter.as_slice(), f.context())
{
let preceding =
write_skipped_statements(first.statement(), &mut iter, verbatim_range, f)?;
// Insert a newline after a module level docstring, but treat
// it as a docstring otherwise. See: https://github.com/psf/black/pull/3932.
let empty_line_after_docstring =
matches!(self.kind, SuiteKind::TopLevel | SuiteKind::Class)
&& DocstringStmt::try_from_statement(preceding, self.kind, f.context())
.is_some();
(preceding, empty_line_after_docstring)
} else if first_comments
.leading
.iter()
.any(|comment| comment.is_suppression_off_comment(source))
@@ -391,7 +409,10 @@ impl FormatRule<Suite, PyFormatContext<'_>> for FormatSuite {
}
}
if following_comments
if let Some(verbatim_range) = skip_range(following, iter.as_slice(), f.context()) {
preceding = write_skipped_statements(following, &mut iter, verbatim_range, f)?;
preceding_comments = comments.leading_dangling_trailing(preceding);
} else if following_comments
.leading
.iter()
.any(|comment| comment.is_suppression_off_comment(source))
@@ -840,61 +861,57 @@ impl Format<PyFormatContext<'_>> for DocstringStmt<'_> {
let comments = f.context().comments().clone();
let node_comments = comments.leading_dangling_trailing(self.docstring);
if FormatStmtExpr.is_suppressed(node_comments.trailing, f.context()) {
suppressed_node(self.docstring).fmt(f)
} else {
// SAFETY: Safe because `DocStringStmt` guarantees that it only ever wraps a `ExprStmt` containing a `ExprStringLiteral`.
let string_literal = self
.docstring
.as_expr_stmt()
.unwrap()
.value
.as_string_literal_expr()
.unwrap();
// SAFETY: Safe because `DocStringStmt` guarantees that it only ever wraps a `ExprStmt` containing a `ExprStringLiteral`.
let string_literal = self
.docstring
.as_expr_stmt()
.unwrap()
.value
.as_string_literal_expr()
.unwrap();
// We format the expression, but the statement carries the comments
write!(
f,
[
leading_comments(node_comments.leading),
f.options()
.source_map_generation()
.is_enabled()
.then_some(source_position(self.docstring.start())),
string_literal
.format()
.with_options(StringLiteralKind::Docstring),
f.options()
.source_map_generation()
.is_enabled()
.then_some(source_position(self.docstring.end())),
]
)?;
// We format the expression, but the statement carries the comments
write!(
f,
[
leading_comments(node_comments.leading),
f.options()
.source_map_generation()
.is_enabled()
.then_some(source_position(self.docstring.start())),
string_literal
.format()
.with_options(StringLiteralKind::Docstring),
f.options()
.source_map_generation()
.is_enabled()
.then_some(source_position(self.docstring.end())),
]
)?;
if self.suite_kind == SuiteKind::Class {
// Comments after class docstrings need a newline between the docstring and the
// comment (https://github.com/astral-sh/ruff/issues/7948).
// ```python
// class ModuleBrowser:
// """Browse module classes and functions in IDLE."""
// # ^ Insert a newline above here
//
// def __init__(self, master, path, *, _htest=False, _utest=False):
// pass
// ```
if let Some(own_line) = node_comments
.trailing
.iter()
.find(|comment| comment.line_position().is_own_line())
{
if lines_before(own_line.start(), f.context().source()) < 2 {
empty_line().fmt(f)?;
}
if self.suite_kind == SuiteKind::Class {
// Comments after class docstrings need a newline between the docstring and the
// comment (https://github.com/astral-sh/ruff/issues/7948).
// ```python
// class ModuleBrowser:
// """Browse module classes and functions in IDLE."""
// # ^ Insert a newline above here
//
// def __init__(self, master, path, *, _htest=False, _utest=False):
// pass
// ```
if let Some(own_line) = node_comments
.trailing
.iter()
.find(|comment| comment.line_position().is_own_line())
{
if lines_before(own_line.start(), f.context().source()) < 2 {
empty_line().fmt(f)?;
}
}
trailing_comments(node_comments.trailing).fmt(f)
}
trailing_comments(node_comments.trailing).fmt(f)
}
}
@@ -938,6 +955,58 @@ impl Format<PyFormatContext<'_>> for SuiteChildStatement<'_> {
}
}
pub(crate) fn skip_range(
first: &Stmt,
statements: &[Stmt],
context: &PyFormatContext,
) -> Option<TextRange> {
let start = first.start();
let mut last_statement = first;
let comments = context.comments();
let source = context.source();
for statement in statements {
if new_logical_line_between_statements(
source,
TextRange::new(last_statement.end(), statement.start()),
) {
break;
}
last_statement = statement;
}
if has_skip_comment(comments.trailing(last_statement), source) {
Some(TextRange::new(
start,
trailing_semicolon(last_statement.into(), source)
.map_or_else(|| last_statement.end(), ruff_text_size::TextRange::end),
))
} else {
None
}
}
fn new_logical_line_between_statements(source: &str, between_statement_range: TextRange) -> bool {
let mut tokenizer = SimpleTokenizer::new(source, between_statement_range).map(|tok| tok.kind());
while let Some(token) = tokenizer.next() {
match token {
SimpleTokenKind::Continuation => {
tokenizer.next();
}
SimpleTokenKind::Newline => {
return true;
}
// Since we are between statements, there are
// no non-trivia tokens, so there is no need to check
// for these and do an early return.
_ => {}
}
}
false
}
#[cfg(test)]
mod tests {
use ruff_formatter::format;

View File

@@ -2,6 +2,7 @@ use std::borrow::Cow;
use std::iter::FusedIterator;
use std::slice::Iter;
use itertools::PeekingNext;
use ruff_formatter::{FormatError, write};
use ruff_python_ast::AnyNodeRef;
use ruff_python_ast::Stmt;
@@ -451,6 +452,40 @@ fn write_suppressed_statements<'a>(
}
}
#[cold]
pub(crate) fn write_skipped_statements<'a>(
first_skipped: &'a Stmt,
statements: &mut std::slice::Iter<'a, Stmt>,
verbatim_range: TextRange,
f: &mut PyFormatter,
) -> FormatResult<&'a Stmt> {
let comments = f.context().comments().clone();
comments.mark_verbatim_node_comments_formatted(first_skipped.into());
let mut preceding = first_skipped;
while let Some(prec) = statements.peeking_next(|next| next.end() <= verbatim_range.end()) {
comments.mark_verbatim_node_comments_formatted(prec.into());
preceding = prec;
}
let first_leading = comments.leading(first_skipped);
let preceding_trailing = comments.trailing(preceding);
// Write the outer comments and format the node as verbatim
write!(
f,
[
leading_comments(first_leading),
source_position(verbatim_range.start()),
verbatim_text(verbatim_range),
source_position(verbatim_range.end()),
trailing_comments(preceding_trailing)
]
)?;
Ok(preceding)
}
#[derive(Copy, Clone, Debug)]
enum InSuppression {
No,
@@ -893,65 +928,6 @@ impl Format<PyFormatContext<'_>> for VerbatimText {
}
}
/// Disables formatting for `node` and instead uses the same formatting as the node has in source.
///
/// The `node` gets indented as any formatted node to avoid syntax errors when the indentation string changes (e.g. from 2 spaces to 4).
/// The `node`s leading and trailing comments are formatted as usual, except if they fall into the suppressed node's range.
#[cold]
pub(crate) fn suppressed_node<'a, N>(node: N) -> FormatSuppressedNode<'a>
where
N: Into<AnyNodeRef<'a>>,
{
FormatSuppressedNode { node: node.into() }
}
pub(crate) struct FormatSuppressedNode<'a> {
node: AnyNodeRef<'a>,
}
impl Format<PyFormatContext<'_>> for FormatSuppressedNode<'_> {
fn fmt(&self, f: &mut Formatter<PyFormatContext<'_>>) -> FormatResult<()> {
let comments = f.context().comments().clone();
let node_comments = comments.leading_dangling_trailing(self.node);
// Mark all comments as formatted that fall into the node range
for comment in node_comments.leading {
if comment.start() > self.node.start() {
comment.mark_formatted();
}
}
for comment in node_comments.trailing {
if comment.start() < self.node.end() {
comment.mark_formatted();
}
}
// Some statements may end with a semicolon. Preserve the semicolon
let semicolon_range = self
.node
.is_statement()
.then(|| trailing_semicolon(self.node, f.context().source()))
.flatten();
let verbatim_range = semicolon_range.map_or(self.node.range(), |semicolon| {
TextRange::new(self.node.start(), semicolon.end())
});
comments.mark_verbatim_node_comments_formatted(self.node);
// Write the outer comments and format the node as verbatim
write!(
f,
[
leading_comments(node_comments.leading),
source_position(verbatim_range.start()),
verbatim_text(verbatim_range),
source_position(verbatim_range.end()),
trailing_comments(node_comments.trailing)
]
)
}
}
#[cold]
pub(crate) fn write_suppressed_clause_header(
header: ClauseHeader,

View File

@@ -0,0 +1,299 @@
---
source: crates/ruff_python_formatter/tests/fixtures.rs
---
## Input
```python
class Simple:
x=1
x=2 # fmt: skip
x=3
class Semicolon:
x=1
x=2;x=3 # fmt: skip
x=4
class TrailingSemicolon:
x=1
x=2;x=3 ; # fmt: skip
x=4
class SemicolonNewLogicalLine:
x=1;
x=2;x=3 # fmt: skip
x=4
class ManySemicolonOneLine:
x=1
x=2;x=3;x=4 # fmt: skip
x=5
class CompoundInSuite:
x=1
def foo(): y=1 # fmt: skip
x=2
class CompoundInSuiteNewline:
x=1
def foo():
y=1 # fmt: skip
x=2
class MultiLineSkip:
x=1
x = [
'1',
'2',
] # fmt: skip
class MultiLineSemicolon:
x=1
x = [
'1',
'2',
]; x=2 # fmt: skip
class LineContinuationSemicolonAfter:
x=1
x = ['a']\
; y=1 # fmt: skip
class LineContinuationSemicolonBefore:
x=1
x = ['a']; \
y=1 # fmt: skip
class LineContinuationSemicolonAndNewline:
x=1
x = ['a']; \
y=1 # fmt: skip
class LineContinuationSemicolonAndNewlineAndComment:
x=1
x = ['a']; \
# 1
y=1 # fmt: skip
class RepeatedLineContinuation:
x=1
x = ['a']; \
\
\
y=1 # fmt: skip
class MultiLineSemicolonComments:
x=1
# 1
x = [ # 2
'1', # 3
'2',
# 4
]; x=2 # 5 # fmt: skip # 6
class DocstringSkipped:
'''This is a docstring''' # fmt: skip
x=1
class MultilineDocstringSkipped:
'''This is a docstring
''' # fmt: skip
x=1
class FirstStatementNewlines:
x=1 # fmt: skip
class ChainingSemicolons:
x=[
'1',
'2',
'3',
];x=1;x=[
'1',
'2',
'3'
];x=1;x=1 # fmt: skip
class LotsOfComments:
# 1
x=[ # 2
'1', # 3
'2',
'3'
] ;x=2;x=3 # 4 # fmt: skip # 5
# 6
class MixingCompound:
def foo(): bar(); import zoo # fmt: skip
# https://github.com/astral-sh/ruff/issues/17331
def main() -> None:
import ipdb; ipdb.set_trace() # noqa: E402,E702,I001 # fmt: skip
# https://github.com/astral-sh/ruff/issues/11430
print(); print() # noqa # fmt: skip
```
## Output
```python
class Simple:
x = 1
x=2 # fmt: skip
x = 3
class Semicolon:
x = 1
x=2;x=3 # fmt: skip
x = 4
class TrailingSemicolon:
x = 1
x=2;x=3 ; # fmt: skip
x = 4
class SemicolonNewLogicalLine:
x = 1
x=2;x=3 # fmt: skip
x = 4
class ManySemicolonOneLine:
x = 1
x=2;x=3;x=4 # fmt: skip
x = 5
class CompoundInSuite:
x = 1
def foo(): y=1 # fmt: skip
x = 2
class CompoundInSuiteNewline:
x = 1
def foo():
y=1 # fmt: skip
x = 2
class MultiLineSkip:
x = 1
x = [
'1',
'2',
] # fmt: skip
class MultiLineSemicolon:
x = 1
x = [
'1',
'2',
]; x=2 # fmt: skip
class LineContinuationSemicolonAfter:
x = 1
x = ['a']\
; y=1 # fmt: skip
class LineContinuationSemicolonBefore:
x = 1
x = ['a']; \
y=1 # fmt: skip
class LineContinuationSemicolonAndNewline:
x = 1
x = ["a"]
y=1 # fmt: skip
class LineContinuationSemicolonAndNewlineAndComment:
x = 1
x = ["a"]
# 1
y=1 # fmt: skip
class RepeatedLineContinuation:
x = 1
x = ['a']; \
\
\
y=1 # fmt: skip
class MultiLineSemicolonComments:
x = 1
# 1
x = [ # 2
'1', # 3
'2',
# 4
]; x=2 # 5 # fmt: skip # 6
class DocstringSkipped:
'''This is a docstring''' # fmt: skip
x = 1
class MultilineDocstringSkipped:
'''This is a docstring
''' # fmt: skip
x = 1
class FirstStatementNewlines:
x=1 # fmt: skip
class ChainingSemicolons:
x=[
'1',
'2',
'3',
];x=1;x=[
'1',
'2',
'3'
];x=1;x=1 # fmt: skip
class LotsOfComments:
# 1
x=[ # 2
'1', # 3
'2',
'3'
] ;x=2;x=3 # 4 # fmt: skip # 5
# 6
class MixingCompound:
def foo(): bar(); import zoo # fmt: skip
# https://github.com/astral-sh/ruff/issues/17331
def main() -> None:
import ipdb; ipdb.set_trace() # noqa: E402,E702,I001 # fmt: skip
# https://github.com/astral-sh/ruff/issues/11430
print(); print() # noqa # fmt: skip
```

View File

@@ -0,0 +1,56 @@
---
source: crates/ruff_python_formatter/tests/fixtures.rs
---
## Input
```python
# 0
'''Module docstring
multiple lines''' # 1 # fmt: skip # 2
import a;import b; from c import (
x,y,z
); import f # fmt: skip
x=1;x=2;x=3;x=4 # fmt: skip
# 1
x=[ # 2
'1', # 3
'2',
'3'
];x=1;x=1 # 4 # fmt: skip # 5
# 6
def foo(): x=[
'1',
'2',
];x=1 # fmt: skip
```
## Output
```python
# 0
'''Module docstring
multiple lines''' # 1 # fmt: skip # 2
import a;import b; from c import (
x,y,z
); import f # fmt: skip
x=1;x=2;x=3;x=4 # fmt: skip
# 1
x=[ # 2
'1', # 3
'2',
'3'
];x=1;x=1 # 4 # fmt: skip # 5
# 6
def foo():
x=[
'1',
'2',
];x=1 # fmt: skip
```

View File

@@ -0,0 +1,147 @@
---
source: crates/ruff_python_formatter/tests/fixtures.rs
---
## Input
```python
class Simple:
# Range comprises skip range
x=1
<RANGE_START>x=2 <RANGE_END># fmt: skip
x=3
class Semicolon:
# Range is part of skip range
x=1
x=2;<RANGE_START>x=3<RANGE_END> # fmt: skip
x=4
class FormatFirst:
x=1
<RANGE_START>x=2<RANGE_END>;x=3 # fmt: skip
x=4
class FormatMiddle:
x=1
x=2;<RANGE_START>x=3<RANGE_END>;x=4 # fmt: skip
x=5
class SemicolonNewLogicalLine:
# Range overlaps on right side
<RANGE_START>x=1;
x=2<RANGE_END>;x=3 # fmt: skip
x=4
class SemicolonNewLogicalLine:
# Range overlaps on left side
x=1;
x=2;<RANGE_START>x=3 # fmt: skip
x=4<RANGE_END>
class ManySemicolonOneLine:
x=1
x=2;x=3;x=4 # fmt: skip
x=5
class CompoundInSuite:
x=1
<RANGE_START>def foo(): y=1 <RANGE_END># fmt: skip
x=2
class CompoundInSuiteNewline:
x=1
def foo():
y=1 # fmt: skip
x=2
class MultiLineSkip:
# Range inside statement
x=1
x = <RANGE_START>[
'1',
'2',<RANGE_END>
] # fmt: skip
class LotsOfComments:
# 1
x=[ # 2
'1', # 3<RANGE_START>
'2',
'3'
] ;x=2;x=3 # 4<RANGE_END> # fmt: skip # 5
# 6
```
## Output
```python
class Simple:
# Range comprises skip range
x=1
x=2 # fmt: skip
x=3
class Semicolon:
# Range is part of skip range
x=1
x=2;x=3 # fmt: skip
x=4
class FormatFirst:
x=1
x=2;x=3 # fmt: skip
x=4
class FormatMiddle:
x=1
x=2;x=3;x=4 # fmt: skip
x=5
class SemicolonNewLogicalLine:
# Range overlaps on right side
x = 1
x=2;x=3 # fmt: skip
x=4
class SemicolonNewLogicalLine:
# Range overlaps on left side
x=1;
x=2;x=3 # fmt: skip
x = 4
class ManySemicolonOneLine:
x=1
x=2;x=3;x=4 # fmt: skip
x=5
class CompoundInSuite:
x=1
def foo(): y=1 # fmt: skip
x=2
class CompoundInSuiteNewline:
x=1
def foo():
y=1 # fmt: skip
x=2
class MultiLineSkip:
# Range inside statement
x=1
x = [
'1',
'2',
] # fmt: skip
class LotsOfComments:
# 1
x=[ # 2
'1', # 3
'2',
'3'
] ;x=2;x=3 # 4 # fmt: skip # 5
# 6
```

View File

@@ -38,8 +38,8 @@ You can optionally install pre-commit hooks to automatically run the validation
when making a commit:
```shell
uv tool install pre-commit
pre-commit install
uv tool install prek
prek install
```
We recommend [nextest](https://nexte.st/) to run ty's test suite (via `cargo nextest run`),
@@ -66,7 +66,7 @@ and that it passes both the lint and test validation checks:
```shell
cargo clippy --workspace --all-targets --all-features -- -D warnings # Rust linting
cargo test # Rust testing
uvx pre-commit run --all-files --show-diff-on-failure # Rust and Python formatting, Markdown and Python linting, etc.
uvx prek run -a # Rust and Python formatting, Markdown and Python linting, etc.
```
These checks will run on GitHub Actions when you open your pull request, but running them locally

View File

@@ -1032,4 +1032,125 @@ reveal_type(asdict(p)) # revealed: dict[str, Any]
reveal_type(replace(p, name="Bob")) # revealed: Person
```
## Calling decorator function directly with a class argument
When a function decorated with `@dataclass_transform()` is called directly with a class argument
(not used as a decorator), it should return the class with the dataclass transformation applied.
### Basic case
```py
from typing_extensions import dataclass_transform
@dataclass_transform()
def my_dataclass[T](cls: type[T]) -> type[T]:
return cls
class A:
x: int
B = my_dataclass(A)
reveal_type(B) # revealed: <class 'A'>
B(1)
```
### Function with additional parameters
```py
from typing_extensions import dataclass_transform
@dataclass_transform()
def my_dataclass[T](cls: type[T], *, order: bool = False) -> type[T]:
return cls
class A:
x: int
B = my_dataclass(A, order=True)
reveal_type(B) # revealed: <class 'A'>
reveal_type(B(1) < B(2)) # revealed: bool
```
### Overloaded decorator function
When the decorator function has overloads (one for direct class application, one for returning a
decorator), calling it with a class should return the class type.
```py
from typing_extensions import dataclass_transform, Callable, overload
@overload
@dataclass_transform()
def my_dataclass[T](cls: type[T]) -> type[T]: ...
@overload
def my_dataclass[T]() -> Callable[[type[T]], type[T]]: ...
def my_dataclass[T](cls: type[T] | None = None) -> type[T] | Callable[[type[T]], type[T]]:
raise NotImplementedError
class A:
x: int
B = my_dataclass(A)
reveal_type(B) # revealed: <class 'A'>
B(1)
```
### Passing a specialized generic class
When calling a `@dataclass_transform()` decorated function with a specialized generic class, the
specialization should be preserved.
```py
from typing_extensions import dataclass_transform
@dataclass_transform()
def my_dataclass[T](cls: type[T]) -> type[T]:
return cls
class A[T]:
x: T
B = my_dataclass(A[int])
reveal_type(B) # revealed: <class 'A[int]'>
B(1)
```
### Decorator factory with class parameter
When a `@dataclass_transform()` decorated function takes a class as a parameter but is used as a
decorator factory (returns a decorator), the dataclass behavior should be applied to the decorated
class, not to the parameter class.
```py
from typing_extensions import dataclass_transform
@dataclass_transform()
def hydrated_dataclass[T](target: type[T], *, frozen: bool = False):
def decorator[U](cls: type[U]) -> type[U]:
return cls
return decorator
class Target:
pass
decorator = hydrated_dataclass(Target)
reveal_type(decorator) # revealed: <decorator produced by dataclass-like function>
@hydrated_dataclass(Target)
class Model:
x: int
# Model should be a dataclass-like class with x as a field
Model(x=1)
reveal_type(Model.__init__) # revealed: (self: Model, x: int) -> None
```
[`typing.dataclass_transform`]: https://docs.python.org/3/library/typing.html#typing.dataclass_transform

View File

@@ -1682,9 +1682,7 @@ def sequence4(cls: type) -> type:
class Foo: ...
ordered_foo = dataclass(order=True)(Foo)
reveal_type(ordered_foo) # revealed: type[Foo] & Any
# TODO: should be `Foo & Any`
reveal_type(ordered_foo()) # revealed: @Todo(Type::Intersection.call)
# TODO: should be `Any`
reveal_type(ordered_foo() < ordered_foo()) # revealed: @Todo(Type::Intersection.call)
reveal_type(ordered_foo) # revealed: <class 'Foo'>
reveal_type(ordered_foo()) # revealed: Foo
reveal_type(ordered_foo() < ordered_foo()) # revealed: bool
```

View File

@@ -38,6 +38,125 @@ reveal_type(s1 > s2) # revealed: bool
reveal_type(s1 >= s2) # revealed: bool
```
## Signature derived from source ordering method
When the source ordering method accepts a broader type (like `object`) for its `other` parameter,
the synthesized comparison methods should use the same signature. This allows comparisons with types
other than the class itself:
```py
from functools import total_ordering
@total_ordering
class Comparable:
def __init__(self, value: int):
self.value = value
def __eq__(self, other: object) -> bool:
if isinstance(other, Comparable):
return self.value == other.value
if isinstance(other, int):
return self.value == other
return NotImplemented
def __lt__(self, other: object) -> bool:
if isinstance(other, Comparable):
return self.value < other.value
if isinstance(other, int):
return self.value < other
return NotImplemented
a = Comparable(10)
b = Comparable(20)
# Comparisons with the same type work.
reveal_type(a <= b) # revealed: bool
reveal_type(a >= b) # revealed: bool
# Comparisons with `int` also work because `__lt__` accepts `object`.
reveal_type(a <= 15) # revealed: bool
reveal_type(a >= 5) # revealed: bool
```
## Multiple ordering methods with different signatures
When multiple ordering methods are defined with different signatures, the decorator selects a "root"
method using the priority order: `__lt__` > `__le__` > `__gt__` > `__ge__`. Synthesized methods use
the signature from the highest-priority method. Methods that are explicitly defined are not
overridden.
```py
from functools import total_ordering
@total_ordering
class MultiSig:
def __init__(self, value: int):
self.value = value
def __eq__(self, other: object) -> bool:
return True
# __lt__ accepts `object` (highest priority, used as root)
def __lt__(self, other: object) -> bool:
return True
# __gt__ only accepts `MultiSig` (not overridden by decorator)
def __gt__(self, other: "MultiSig") -> bool:
return True
a = MultiSig(10)
b = MultiSig(20)
# __le__ and __ge__ are synthesized with __lt__'s signature (accepts `object`)
reveal_type(a <= b) # revealed: bool
reveal_type(a <= 15) # revealed: bool
reveal_type(a >= b) # revealed: bool
reveal_type(a >= 15) # revealed: bool
# __gt__ keeps its original signature (only accepts MultiSig)
reveal_type(a > b) # revealed: bool
a > 15 # error: [unsupported-operator]
```
## Overloaded ordering method
When the source ordering method is overloaded, the synthesized comparison methods should preserve
all overloads:
```py
from functools import total_ordering
from typing import overload
@total_ordering
class Flexible:
def __init__(self, value: int):
self.value = value
def __eq__(self, other: object) -> bool:
return True
@overload
def __lt__(self, other: "Flexible") -> bool: ...
@overload
def __lt__(self, other: int) -> bool: ...
def __lt__(self, other: "Flexible | int") -> bool:
if isinstance(other, Flexible):
return self.value < other.value
return self.value < other
a = Flexible(10)
b = Flexible(20)
# Synthesized __le__ preserves overloads from __lt__
reveal_type(a <= b) # revealed: bool
reveal_type(a <= 15) # revealed: bool
# Synthesized __ge__ also preserves overloads
reveal_type(a >= b) # revealed: bool
reveal_type(a >= 15) # revealed: bool
# But comparison with an unsupported type should still error
a <= "string" # error: [unsupported-operator]
```
## Using `__gt__` as the root comparison method
When a class defines `__eq__` and `__gt__`, the decorator synthesizes `__lt__`, `__le__`, and
@@ -127,6 +246,41 @@ reveal_type(c1 > c2) # revealed: bool
reveal_type(c1 >= c2) # revealed: bool
```
## Method precedence with inheritance
The decorator always prefers `__lt__` > `__le__` > `__gt__` > `__ge__`, regardless of whether the
method is defined locally or inherited. In this example, the inherited `__lt__` takes precedence
over the locally-defined `__gt__`:
```py
from functools import total_ordering
from typing import Literal
class Base:
def __lt__(self, other: "Base") -> Literal[True]:
return True
@total_ordering
class Child(Base):
# __gt__ is defined locally, but __lt__ (inherited) takes precedence
def __gt__(self, other: "Child") -> Literal[False]:
return False
c1 = Child()
c2 = Child()
# __lt__ is inherited from Base
reveal_type(c1 < c2) # revealed: Literal[True]
# __gt__ is defined locally on Child
reveal_type(c1 > c2) # revealed: Literal[False]
# __le__ and __ge__ are synthesized from __lt__ (the highest-priority method),
# even though __gt__ is defined locally on the class itself
reveal_type(c1 <= c2) # revealed: bool
reveal_type(c1 >= c2) # revealed: bool
```
## Explicitly-defined methods are not overridden
When a class explicitly defines multiple comparison methods, the decorator does not override them.
@@ -244,3 +398,109 @@ reveal_type(n1 > n2) # revealed: bool
n1 <= n2 # error: [unsupported-operator]
n1 >= n2 # error: [unsupported-operator]
```
## Non-bool return type
When the root ordering method returns a non-bool type (like `int`), the synthesized methods return a
union of that type and `bool`. This is because `@total_ordering` generates methods like:
```python
def __le__(self, other):
return self < other or self == other
```
If `__lt__` returns `int`, then the synthesized `__le__` could return either `int` (from
`self < other`) or `bool` (from `self == other`). Since `bool` is a subtype of `int`, the union
simplifies to `int`:
```py
from functools import total_ordering
@total_ordering
class IntReturn:
def __init__(self, value: int):
self.value = value
def __eq__(self, other: object) -> bool:
if not isinstance(other, IntReturn):
return NotImplemented
return self.value == other.value
def __lt__(self, other: "IntReturn") -> int:
return self.value - other.value
a = IntReturn(10)
b = IntReturn(20)
# User-defined __lt__ returns int.
reveal_type(a < b) # revealed: int
# Synthesized methods return int (the union int | bool simplifies to int
# because bool is a subtype of int in Python).
reveal_type(a <= b) # revealed: int
reveal_type(a > b) # revealed: int
reveal_type(a >= b) # revealed: int
```
When the root method returns a type that is not a supertype of `bool`, the union is preserved:
```py
from functools import total_ordering
@total_ordering
class StrReturn:
def __init__(self, value: str):
self.value = value
def __eq__(self, other: object) -> bool:
if not isinstance(other, StrReturn):
return NotImplemented
return self.value == other.value
def __lt__(self, other: "StrReturn") -> str:
return self.value
a = StrReturn("a")
b = StrReturn("b")
# User-defined __lt__ returns str.
reveal_type(a < b) # revealed: str
# Synthesized methods return str | bool.
reveal_type(a <= b) # revealed: str | bool
reveal_type(a > b) # revealed: str | bool
reveal_type(a >= b) # revealed: str | bool
```
## Function call form
When `total_ordering` is called as a function (not as a decorator), the same validation is
performed:
```py
from functools import total_ordering
class NoOrderingMethod:
def __eq__(self, other: object) -> bool:
return True
# error: [invalid-total-ordering]
InvalidOrderedClass = total_ordering(NoOrderingMethod)
```
When the class does define an ordering method, no error is emitted:
```py
from functools import total_ordering
class HasOrderingMethod:
def __eq__(self, other: object) -> bool:
return True
def __lt__(self, other: "HasOrderingMethod") -> bool:
return True
# No error (class defines `__lt__`).
ValidOrderedClass = total_ordering(HasOrderingMethod)
reveal_type(ValidOrderedClass) # revealed: type[HasOrderingMethod]
```

View File

@@ -140,7 +140,11 @@ If a child class's method definition is Liskov-compatible with the method defini
class, Liskov compatibility must also nonetheless be checked with respect to the method definition
on its grandparent class. This is because type checkers will treat the child class as a subtype of
the grandparent class just as much as they treat it as a subtype of the parent class, so
substitutability with respect to the grandparent class is just as important:
substitutability with respect to the grandparent class is just as important.
However, if the parent class itself already has an LSP violation with an ancestor, we do not report
the same violation for the child class. This is because the child class cannot fix the violation
without introducing a new, worse violation against its immediate parent's contract.
<!-- snapshot-diagnostics -->
@@ -156,13 +160,31 @@ class Parent(Grandparent):
def method(self, x: str) -> None: ... # error: [invalid-method-override]
class Child(Parent):
# compatible with the signature of `Parent.method`, but not with `Grandparent.method`:
def method(self, x: str) -> None: ... # error: [invalid-method-override]
# compatible with the signature of `Parent.method`, but not with `Grandparent.method`.
# However, since `Parent.method` already violates LSP with `Grandparent.method`,
# we don't report the same violation for `Child` -- it's inherited from `Parent`.
def method(self, x: str) -> None: ...
class OtherChild(Parent):
# compatible with the signature of `Grandparent.method`, but not with `Parent.method`:
def method(self, x: int) -> None: ... # error: [invalid-method-override]
class ChildWithNewViolation(Parent):
# incompatible with BOTH `Parent.method` (str) and `Grandparent.method` (int).
# We report the violation against the immediate parent (`Parent`), not the grandparent.
def method(self, x: bytes) -> None: ... # error: [invalid-method-override]
class GrandparentWithReturnType:
def method(self) -> int: ...
class ParentWithReturnType(GrandparentWithReturnType):
def method(self) -> str: ... # error: [invalid-method-override]
class ChildWithReturnType(ParentWithReturnType):
# Returns `int` again -- compatible with `GrandparentWithReturnType.method`,
# but not with `ParentWithReturnType.method`. We report against the immediate parent.
def method(self) -> int: ... # error: [invalid-method-override]
class GradualParent(Grandparent):
def method(self, x: Any) -> None: ...
@@ -190,8 +212,9 @@ class C(B):
foo = get
class D(C):
# compatible with `C.get` and `B.get`, but not with `A.get`
def get(self, my_default): ... # error: [invalid-method-override]
# compatible with `C.get` and `B.get`, but not with `A.get`.
# Since `B.get` already violates LSP with `A.get`, we don't report for `D`.
def get(self, my_default): ...
```
## Non-generic methods on generic classes work as expected

View File

@@ -22,21 +22,39 @@ mdtest path: crates/ty_python_semantic/resources/mdtest/liskov.md
7 | def method(self, x: str) -> None: ... # error: [invalid-method-override]
8 |
9 | class Child(Parent):
10 | # compatible with the signature of `Parent.method`, but not with `Grandparent.method`:
11 | def method(self, x: str) -> None: ... # error: [invalid-method-override]
12 |
13 | class OtherChild(Parent):
14 | # compatible with the signature of `Grandparent.method`, but not with `Parent.method`:
15 | def method(self, x: int) -> None: ... # error: [invalid-method-override]
16 |
17 | class GradualParent(Grandparent):
18 | def method(self, x: Any) -> None: ...
19 |
20 | class ThirdChild(GradualParent):
21 | # `GradualParent.method` is compatible with the signature of `Grandparent.method`,
22 | # and `ThirdChild.method` is compatible with the signature of `GradualParent.method`,
23 | # but `ThirdChild.method` is not compatible with the signature of `Grandparent.method`
24 | def method(self, x: str) -> None: ... # error: [invalid-method-override]
10 | # compatible with the signature of `Parent.method`, but not with `Grandparent.method`.
11 | # However, since `Parent.method` already violates LSP with `Grandparent.method`,
12 | # we don't report the same violation for `Child` -- it's inherited from `Parent`.
13 | def method(self, x: str) -> None: ...
14 |
15 | class OtherChild(Parent):
16 | # compatible with the signature of `Grandparent.method`, but not with `Parent.method`:
17 | def method(self, x: int) -> None: ... # error: [invalid-method-override]
18 |
19 | class ChildWithNewViolation(Parent):
20 | # incompatible with BOTH `Parent.method` (str) and `Grandparent.method` (int).
21 | # We report the violation against the immediate parent (`Parent`), not the grandparent.
22 | def method(self, x: bytes) -> None: ... # error: [invalid-method-override]
23 |
24 | class GrandparentWithReturnType:
25 | def method(self) -> int: ...
26 |
27 | class ParentWithReturnType(GrandparentWithReturnType):
28 | def method(self) -> str: ... # error: [invalid-method-override]
29 |
30 | class ChildWithReturnType(ParentWithReturnType):
31 | # Returns `int` again -- compatible with `GrandparentWithReturnType.method`,
32 | # but not with `ParentWithReturnType.method`. We report against the immediate parent.
33 | def method(self) -> int: ... # error: [invalid-method-override]
34 |
35 | class GradualParent(Grandparent):
36 | def method(self, x: Any) -> None: ...
37 |
38 | class ThirdChild(GradualParent):
39 | # `GradualParent.method` is compatible with the signature of `Grandparent.method`,
40 | # and `ThirdChild.method` is compatible with the signature of `GradualParent.method`,
41 | # but `ThirdChild.method` is not compatible with the signature of `Grandparent.method`
42 | def method(self, x: str) -> None: ... # error: [invalid-method-override]
```
## other_stub.pyi
@@ -56,8 +74,9 @@ mdtest path: crates/ty_python_semantic/resources/mdtest/liskov.md
12 | foo = get
13 |
14 | class D(C):
15 | # compatible with `C.get` and `B.get`, but not with `A.get`
16 | def get(self, my_default): ... # error: [invalid-method-override]
15 | # compatible with `C.get` and `B.get`, but not with `A.get`.
16 | # Since `B.get` already violates LSP with `A.get`, we don't report for `D`.
17 | def get(self, my_default): ...
```
# Diagnostics
@@ -83,38 +102,14 @@ info: rule `invalid-method-override` is enabled by default
```
error[invalid-method-override]: Invalid override of method `method`
--> src/stub.pyi:11:9
--> src/stub.pyi:17:9
|
9 | class Child(Parent):
10 | # compatible with the signature of `Parent.method`, but not with `Grandparent.method`:
11 | def method(self, x: str) -> None: ... # error: [invalid-method-override]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Definition is incompatible with `Grandparent.method`
12 |
13 | class OtherChild(Parent):
|
::: src/stub.pyi:4:9
|
3 | class Grandparent:
4 | def method(self, x: int) -> None: ...
| ---------------------------- `Grandparent.method` defined here
5 |
6 | class Parent(Grandparent):
|
info: This violates the Liskov Substitution Principle
info: rule `invalid-method-override` is enabled by default
```
```
error[invalid-method-override]: Invalid override of method `method`
--> src/stub.pyi:15:9
|
13 | class OtherChild(Parent):
14 | # compatible with the signature of `Grandparent.method`, but not with `Parent.method`:
15 | def method(self, x: int) -> None: ... # error: [invalid-method-override]
15 | class OtherChild(Parent):
16 | # compatible with the signature of `Grandparent.method`, but not with `Parent.method`:
17 | def method(self, x: int) -> None: ... # error: [invalid-method-override]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Definition is incompatible with `Parent.method`
16 |
17 | class GradualParent(Grandparent):
18 |
19 | class ChildWithNewViolation(Parent):
|
::: src/stub.pyi:7:9
|
@@ -131,11 +126,75 @@ info: rule `invalid-method-override` is enabled by default
```
error[invalid-method-override]: Invalid override of method `method`
--> src/stub.pyi:24:9
--> src/stub.pyi:22:9
|
22 | # and `ThirdChild.method` is compatible with the signature of `GradualParent.method`,
23 | # but `ThirdChild.method` is not compatible with the signature of `Grandparent.method`
24 | def method(self, x: str) -> None: ... # error: [invalid-method-override]
20 | # incompatible with BOTH `Parent.method` (str) and `Grandparent.method` (int).
21 | # We report the violation against the immediate parent (`Parent`), not the grandparent.
22 | def method(self, x: bytes) -> None: ... # error: [invalid-method-override]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Definition is incompatible with `Parent.method`
23 |
24 | class GrandparentWithReturnType:
|
::: src/stub.pyi:7:9
|
6 | class Parent(Grandparent):
7 | def method(self, x: str) -> None: ... # error: [invalid-method-override]
| ---------------------------- `Parent.method` defined here
8 |
9 | class Child(Parent):
|
info: This violates the Liskov Substitution Principle
info: rule `invalid-method-override` is enabled by default
```
```
error[invalid-method-override]: Invalid override of method `method`
--> src/stub.pyi:25:9
|
24 | class GrandparentWithReturnType:
25 | def method(self) -> int: ...
| ------------------- `GrandparentWithReturnType.method` defined here
26 |
27 | class ParentWithReturnType(GrandparentWithReturnType):
28 | def method(self) -> str: ... # error: [invalid-method-override]
| ^^^^^^^^^^^^^^^^^^^ Definition is incompatible with `GrandparentWithReturnType.method`
29 |
30 | class ChildWithReturnType(ParentWithReturnType):
|
info: This violates the Liskov Substitution Principle
info: rule `invalid-method-override` is enabled by default
```
```
error[invalid-method-override]: Invalid override of method `method`
--> src/stub.pyi:28:9
|
27 | class ParentWithReturnType(GrandparentWithReturnType):
28 | def method(self) -> str: ... # error: [invalid-method-override]
| ------------------- `ParentWithReturnType.method` defined here
29 |
30 | class ChildWithReturnType(ParentWithReturnType):
31 | # Returns `int` again -- compatible with `GrandparentWithReturnType.method`,
32 | # but not with `ParentWithReturnType.method`. We report against the immediate parent.
33 | def method(self) -> int: ... # error: [invalid-method-override]
| ^^^^^^^^^^^^^^^^^^^ Definition is incompatible with `ParentWithReturnType.method`
34 |
35 | class GradualParent(Grandparent):
|
info: This violates the Liskov Substitution Principle
info: rule `invalid-method-override` is enabled by default
```
```
error[invalid-method-override]: Invalid override of method `method`
--> src/stub.pyi:42:9
|
40 | # and `ThirdChild.method` is compatible with the signature of `GradualParent.method`,
41 | # but `ThirdChild.method` is not compatible with the signature of `Grandparent.method`
42 | def method(self, x: str) -> None: ... # error: [invalid-method-override]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Definition is incompatible with `Grandparent.method`
|
::: src/stub.pyi:4:9
@@ -169,25 +228,3 @@ info: This violates the Liskov Substitution Principle
info: rule `invalid-method-override` is enabled by default
```
```
error[invalid-method-override]: Invalid override of method `get`
--> src/other_stub.pyi:16:9
|
14 | class D(C):
15 | # compatible with `C.get` and `B.get`, but not with `A.get`
16 | def get(self, my_default): ... # error: [invalid-method-override]
| ^^^^^^^^^^^^^^^^^^^^^ Definition is incompatible with `A.get`
|
::: src/other_stub.pyi:2:9
|
1 | class A:
2 | def get(self, default): ...
| ------------------ `A.get` defined here
3 |
4 | class B(A):
|
info: This violates the Liskov Substitution Principle
info: rule `invalid-method-override` is enabled by default
```

View File

@@ -36,6 +36,7 @@ com2ann
comtypes
core
cpython
cryptography
cwltool
dacite
dd-trace-py
@@ -79,6 +80,7 @@ mypy-protobuf
mypy_primer
nionutils
nox
numpy-stl
openlibrary
operator
optuna

View File

@@ -43,8 +43,8 @@ use crate::types::signatures::{Parameter, ParameterForm, ParameterKind, Paramete
use crate::types::tuple::{TupleLength, TupleSpec, TupleType};
use crate::types::{
BoundMethodType, BoundTypeVarIdentity, BoundTypeVarInstance, CallableSignature, CallableType,
CallableTypeKind, ClassLiteral, DATACLASS_FLAGS, DataclassFlags, DataclassParams,
FieldInstance, KnownBoundMethodType, KnownClass, KnownInstanceType, MemberLookupPolicy,
CallableTypeKind, DATACLASS_FLAGS, DataclassFlags, DataclassParams, FieldInstance,
GenericAlias, KnownBoundMethodType, KnownClass, KnownInstanceType, MemberLookupPolicy,
NominalInstanceType, PropertyInstanceType, SpecialFormType, TrackedConstraintSet,
TypeAliasType, TypeContext, TypeVarVariance, UnionBuilder, UnionType, WrapperDescriptorKind,
enums, list_members, todo_type,
@@ -611,6 +611,25 @@ impl<'db> Bindings<'db> {
}
}
Type::DataclassDecorator(params) => match overload.parameter_types() {
[Some(Type::ClassLiteral(class_literal))] => {
overload.set_return_type(Type::from(
class_literal.with_dataclass_params(db, Some(params)),
));
}
[Some(Type::GenericAlias(generic_alias))] => {
let new_origin = generic_alias
.origin(db)
.with_dataclass_params(db, Some(params));
overload.set_return_type(Type::GenericAlias(GenericAlias::new(
db,
new_origin,
generic_alias.specialization(db),
)));
}
_ => {}
},
Type::BoundMethod(bound_method)
if bound_method.self_instance(db).is_property_instance() =>
{
@@ -1119,17 +1138,9 @@ impl<'db> Bindings<'db> {
overload.parameter_types()
{
let params = DataclassParams::default_params(db);
overload.set_return_type(Type::from(ClassLiteral::new(
db,
class_literal.name(db),
class_literal.body_scope(db),
class_literal.known(db),
class_literal.deprecated(db),
class_literal.type_check_only(db),
Some(params),
class_literal.dataclass_transformer_params(db),
class_literal.total_ordering(db),
)));
overload.set_return_type(Type::from(
class_literal.with_dataclass_params(db, Some(params)),
));
}
}
@@ -1185,44 +1196,102 @@ impl<'db> Bindings<'db> {
// Ideally, either the implementation, or exactly one of the overloads
// of the function can have the dataclass_transform decorator applied.
// However, we do not yet enforce this, and in the case of multiple
// applications of the decorator, we will only consider the last one
// for the return value, since the prior ones will be over-written.
let return_type = function_type
// applications of the decorator, we will only consider the last one.
let transformer_params = function_type
.iter_overloads_and_implementation(db)
.filter_map(|function_overload| {
function_overload.dataclass_transformer_params(db).map(
|params| {
// This is a call to a custom function that was decorated with `@dataclass_transformer`.
// If this function was called with a keyword argument like `order=False`, we extract
// the argument type and overwrite the corresponding flag in `dataclass_params` after
// constructing them from the `dataclass_transformer`-parameter defaults.
.rev()
.find_map(|function_overload| {
function_overload.dataclass_transformer_params(db)
});
let dataclass_params =
DataclassParams::from_transformer_params(
db, params,
);
let mut flags = dataclass_params.flags(db);
if let Some(params) = transformer_params {
// If this function was called with a keyword argument like
// `order=False`, we extract the argument type and overwrite
// the corresponding flag in `dataclass_params`.
let dataclass_params =
DataclassParams::from_transformer_params(db, params);
let mut flags = dataclass_params.flags(db);
for (param, flag) in DATACLASS_FLAGS {
if let Ok(Some(Type::BooleanLiteral(value))) =
overload.parameter_type_by_name(param, false)
{
flags.set(*flag, value);
}
}
for (param, flag) in DATACLASS_FLAGS {
if let Ok(Some(Type::BooleanLiteral(value))) =
overload.parameter_type_by_name(param, false)
{
flags.set(*flag, value);
}
}
Type::DataclassDecorator(DataclassParams::new(
db,
flags,
dataclass_params.field_specifiers(db),
))
},
)
})
.last();
let dataclass_params = DataclassParams::new(
db,
flags,
dataclass_params.field_specifiers(db),
);
if let Some(return_type) = return_type {
overload.set_return_type(return_type);
// The dataclass_transform spec doesn't clarify how to tell whether
// a decorated function is a decorator or a decorator factory. We
// use heuristics based on the number and type of positional arguments:
//
// - Zero positional arguments: assume it's a decorator factory.
// - More than one positional argument: assume it's a decorator factory.
// - Exactly one positional argument that's a class: ambiguous, so check
// the return type to disambiguate (class-like means decorate directly).
let mut positional_args = overload
.signature
.parameters()
.iter()
.zip(overload.parameter_types())
.filter(|(param, ty)| ty.is_some() && !param.is_keyword_only())
.map(|(_, ty)| ty);
let first_positional = positional_args.next();
let has_more = positional_args.next().is_some();
// Only attempt direct decoration if exactly one positional argument.
if !has_more {
// Helper to check if return type is class-like.
let returns_class = || {
matches!(
overload.return_type(),
Type::ClassLiteral(_)
| Type::GenericAlias(_)
| Type::SubclassOf(_)
)
};
match first_positional {
Some(Some(Type::ClassLiteral(class_literal)))
if returns_class() =>
{
overload.set_return_type(Type::from(
class_literal.with_dataclass_params(
db,
Some(dataclass_params),
),
));
continue;
}
Some(Some(Type::GenericAlias(generic_alias)))
if returns_class() =>
{
let new_origin = generic_alias
.origin(db)
.with_dataclass_params(db, Some(dataclass_params));
overload.set_return_type(Type::GenericAlias(
GenericAlias::new(
db,
new_origin,
generic_alias.specialization(db),
),
));
continue;
}
_ => {}
}
}
// Zero or more than one positional argument, or the argument is
// not a class: assume it's a decorator factory.
overload
.set_return_type(Type::DataclassDecorator(dataclass_params));
}
}
},

View File

@@ -604,6 +604,13 @@ impl<'db> ClassType<'db> {
class_literal.is_final(db)
}
/// Returns `true` if any class in this class's MRO (excluding `object`) defines an ordering
/// method (`__lt__`, `__le__`, `__gt__`, `__ge__`). Used by `@total_ordering` validation.
pub(super) fn has_ordering_method_in_mro(self, db: &'db dyn Db) -> bool {
let (class_literal, specialization) = self.class_literal(db);
class_literal.has_ordering_method_in_mro(db, specialization)
}
/// Return `true` if `other` is present in this class's MRO.
pub(super) fn is_subclass_of(self, db: &'db dyn Db, other: ClassType<'db>) -> bool {
self.when_subclass_of(db, other, InferableTypeVars::None)
@@ -1548,6 +1555,25 @@ impl<'db> ClassLiteral<'db> {
self.is_known(db, KnownClass::Tuple)
}
/// Returns a new `ClassLiteral` with the given dataclass params, preserving all other fields.
pub(crate) fn with_dataclass_params(
self,
db: &'db dyn Db,
dataclass_params: Option<DataclassParams<'db>>,
) -> Self {
ClassLiteral::new(
db,
self.name(db).clone(),
self.body_scope(db),
self.known(db),
self.deprecated(db),
self.type_check_only(db),
dataclass_params,
self.dataclass_transformer_params(db),
self.total_ordering(db),
)
}
/// Returns `true` if this class defines any ordering method (`__lt__`, `__le__`, `__gt__`,
/// `__ge__`) in its own body (not inherited). Used by `@total_ordering` to determine if
/// synthesis is valid.
@@ -1559,6 +1585,47 @@ impl<'db> ClassLiteral<'db> {
.any(|method| !class_member(db, body_scope, method).is_undefined())
}
/// Returns `true` if any class in this class's MRO (excluding `object`) defines an ordering
/// method (`__lt__`, `__le__`, `__gt__`, `__ge__`). Used by `@total_ordering` validation.
pub(super) fn has_ordering_method_in_mro(
self,
db: &'db dyn Db,
specialization: Option<Specialization<'db>>,
) -> bool {
self.total_ordering_root_method(db, specialization)
.is_some()
}
/// Returns the type of the ordering method used by `@total_ordering`, if any.
///
/// Following `functools.total_ordering` precedence, we prefer `__lt__` > `__le__` > `__gt__` >
/// `__ge__`, regardless of whether the method is defined locally or inherited.
pub(super) fn total_ordering_root_method(
self,
db: &'db dyn Db,
specialization: Option<Specialization<'db>>,
) -> Option<Type<'db>> {
const ORDERING_METHODS: [&str; 4] = ["__lt__", "__le__", "__gt__", "__ge__"];
for name in ORDERING_METHODS {
for base in self.iter_mro(db, specialization) {
let Some(base_class) = base.into_class() else {
continue;
};
let (base_literal, base_specialization) = base_class.class_literal(db);
if base_literal.is_known(db, KnownClass::Object) {
continue;
}
let member = class_member(db, base_literal.body_scope(db), name);
if let Some(ty) = member.ignore_possibly_undefined() {
return Some(ty.apply_optional_specialization(db, base_specialization));
}
}
}
None
}
pub(crate) fn generic_context(self, db: &'db dyn Db) -> Option<GenericContext<'db>> {
// Several typeshed definitions examine `sys.version_info`. To break cycles, we hard-code
// the knowledge that this class is not generic.
@@ -2408,34 +2475,44 @@ impl<'db> ClassLiteral<'db> {
// ordering method. The decorator requires at least one of __lt__,
// __le__, __gt__, or __ge__ to be defined (either in this class or
// inherited from a superclass, excluding `object`).
if self.total_ordering(db) && matches!(name, "__lt__" | "__le__" | "__gt__" | "__ge__") {
// Check if any class in the MRO (excluding object) defines at least one
// ordering method in its own body (not synthesized).
let has_ordering_method = self
//
// Only synthesize methods that are not already defined in the MRO.
if self.total_ordering(db)
&& matches!(name, "__lt__" | "__le__" | "__gt__" | "__ge__")
&& !self
.iter_mro(db, specialization)
.filter_map(super::class_base::ClassBase::into_class)
.filter_map(ClassBase::into_class)
.filter(|class| !class.class_literal(db).0.is_known(db, KnownClass::Object))
.any(|class| class.class_literal(db).0.has_own_ordering_method(db));
if has_ordering_method {
let instance_ty =
Type::instance(db, self.apply_optional_specialization(db, specialization));
let signature = Signature::new(
Parameters::new(
db,
[
Parameter::positional_or_keyword(Name::new_static("self"))
.with_annotated_type(instance_ty),
Parameter::positional_or_keyword(Name::new_static("other"))
.with_annotated_type(instance_ty),
],
),
KnownClass::Bool.to_instance(db),
.any(|class| {
class_member(db, class.class_literal(db).0.body_scope(db), name)
.ignore_possibly_undefined()
.is_some()
})
&& self.has_ordering_method_in_mro(db, specialization)
&& let Some(root_method_ty) = self.total_ordering_root_method(db, specialization)
&& let Some(callables) = root_method_ty.try_upcast_to_callable(db)
{
let bool_ty = KnownClass::Bool.to_instance(db);
let synthesized_callables = callables.map(|callable| {
let signatures = CallableSignature::from_overloads(
callable.signatures(db).iter().map(|signature| {
// The generated methods return a union of the root method's return type
// and `bool`. This is because `@total_ordering` synthesizes methods like:
// def __gt__(self, other): return not (self == other or self < other)
// If `__lt__` returns `int`, then `__gt__` could return `int | bool`.
let return_ty =
UnionType::from_elements(db, [signature.return_ty, bool_ty]);
Signature::new_generic(
signature.generic_context,
signature.parameters().clone(),
return_ty,
)
}),
);
CallableType::new(db, signatures, CallableTypeKind::FunctionLike)
});
return Some(Type::function_like_callable(db, signature));
}
return Some(synthesized_callables.into_type(db));
}
let field_policy = CodeGeneratorKind::from_class(db, self, specialization)?;

View File

@@ -4684,6 +4684,29 @@ pub(super) fn report_invalid_total_ordering(
diagnostic.info("The decorator will raise `ValueError` at runtime");
}
/// Reports an invalid `total_ordering(cls)` function call where the class
/// does not define any ordering method.
pub(super) fn report_invalid_total_ordering_call(
context: &InferContext<'_, '_>,
class: ClassLiteral<'_>,
call_expression: &ast::ExprCall,
) {
let db = context.db();
let Some(builder) = context.report_lint(&INVALID_TOTAL_ORDERING, call_expression) else {
return;
};
let mut diagnostic = builder.into_diagnostic(
"`@functools.total_ordering` requires at least one ordering method (`__lt__`, `__le__`, `__gt__`, or `__ge__`) to be defined",
);
diagnostic.set_primary_message(format_args!(
"`{}` does not define `__lt__`, `__le__`, `__gt__`, or `__ge__`",
class.name(db)
));
diagnostic.info("The function will raise `ValueError` at runtime");
}
/// This function receives an unresolved `from foo import bar` import,
/// where `foo` can be resolved to a module but that module does not
/// have a `bar` member or submodule.

View File

@@ -70,6 +70,7 @@ use crate::types::context::InferContext;
use crate::types::diagnostic::{
INVALID_ARGUMENT_TYPE, REDUNDANT_CAST, STATIC_ASSERT_ERROR, TYPE_ASSERTION_FAILURE,
report_bad_argument_to_get_protocol_members, report_bad_argument_to_protocol_interface,
report_invalid_total_ordering_call,
report_runtime_check_against_non_runtime_checkable_protocol,
};
use crate::types::display::DisplaySettings;
@@ -747,9 +748,9 @@ impl<'db> FunctionLiteral<'db> {
fn iter_overloads_and_implementation(
self,
db: &'db dyn Db,
) -> impl Iterator<Item = OverloadLiteral<'db>> + 'db {
let (implementation, overloads) = self.overloads_and_implementation(db);
overloads.into_iter().chain(implementation.iter().copied())
) -> impl DoubleEndedIterator<Item = OverloadLiteral<'db>> + 'db {
let (overloads, implementation) = self.overloads_and_implementation(db);
overloads.iter().copied().chain(implementation)
}
/// Typed externally-visible signature for this function.
@@ -1033,7 +1034,7 @@ impl<'db> FunctionType<'db> {
pub(crate) fn iter_overloads_and_implementation(
self,
db: &'db dyn Db,
) -> impl Iterator<Item = OverloadLiteral<'db>> + 'db {
) -> impl DoubleEndedIterator<Item = OverloadLiteral<'db>> + 'db {
self.literal(db).iter_overloads_and_implementation(db)
}
@@ -2005,6 +2006,29 @@ impl KnownFunction {
overload.set_return_type(Type::module_literal(db, file, module));
}
KnownFunction::TotalOrdering => {
// When `total_ordering(cls)` is called as a function (not as a decorator),
// check that the class defines at least one ordering method.
let [Some(class_type)] = parameter_types else {
return;
};
let class = match class_type {
Type::ClassLiteral(class) => ClassType::NonGeneric(*class),
Type::GenericAlias(generic) => ClassType::Generic(*generic),
_ => return,
};
if !class.has_ordering_method_in_mro(db) {
report_invalid_total_ordering_call(
context,
class.class_literal(db).0,
call_expression,
);
}
}
_ => {}
}
}

View File

@@ -854,34 +854,17 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> {
}
// (5) Check that @total_ordering has a valid ordering method in the MRO
if class.total_ordering(self.db()) {
let has_ordering_method = class
.iter_mro(self.db(), None)
.filter_map(super::super::class_base::ClassBase::into_class)
.filter(|base_class| {
!base_class
.class_literal(self.db())
.0
.is_known(self.db(), KnownClass::Object)
})
.any(|base_class| {
base_class
.class_literal(self.db())
.0
.has_own_ordering_method(self.db())
});
if !has_ordering_method {
// Find the @total_ordering decorator to report the diagnostic at its location
if let Some(decorator) = class_node.decorator_list.iter().find(|decorator| {
self.expression_type(&decorator.expression)
.as_function_literal()
.is_some_and(|function| {
function.is_known(self.db(), KnownFunction::TotalOrdering)
})
}) {
report_invalid_total_ordering(&self.context, class, decorator);
}
if class.total_ordering(self.db()) && !class.has_ordering_method_in_mro(self.db(), None)
{
// Find the @total_ordering decorator to report the diagnostic at its location
if let Some(decorator) = class_node.decorator_list.iter().find(|decorator| {
self.expression_type(&decorator.expression)
.as_function_literal()
.is_some_and(|function| {
function.is_known(self.db(), KnownFunction::TotalOrdering)
})
}) {
report_invalid_total_ordering(&self.context, class, decorator);
}
}
@@ -2958,6 +2941,7 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> {
// params of the last seen usage of `@dataclass_transform`
let transformer_params = f
.iter_overloads_and_implementation(self.db())
.rev()
.find_map(|overload| overload.dataclass_transformer_params(self.db()));
if let Some(transformer_params) = transformer_params {
dataclass_params = Some(DataclassParams::from_transformer_params(

View File

@@ -152,6 +152,11 @@ fn check_class_declaration<'db>(
let mut liskov_diagnostic_emitted = false;
let mut overridden_final_method = None;
// Track the first superclass that defines this method (the "immediate parent" for this method).
// We need this to check if parent itself already has an LSP violation with an ancestor.
// If so, we shouldn't report the same violation for the child class.
let mut immediate_parent_method: Option<(ClassType<'db>, Type<'db>)> = None;
for class_base in class.iter_mro(db).skip(1) {
let superclass = match class_base {
ClassBase::Protocol | ClassBase::Generic => continue,
@@ -205,6 +210,11 @@ fn check_class_declaration<'db>(
subclass_overrides_superclass_declaration = true;
// Record the first superclass that defines this method as the "immediate parent method"
if immediate_parent_method.is_none() {
immediate_parent_method = Some((superclass, superclass_type));
}
if configuration.check_final_method_overridden() {
overridden_final_method = overridden_final_method.or_else(|| {
let superclass_symbol_id = superclass_symbol_id?;
@@ -271,11 +281,30 @@ fn check_class_declaration<'db>(
continue;
};
if type_on_subclass_instance.is_assignable_to(db, superclass_type_as_callable.into_type(db))
{
let superclass_type_as_type = superclass_type_as_callable.into_type(db);
if type_on_subclass_instance.is_assignable_to(db, superclass_type_as_type) {
continue;
}
// If this superclass is not the immediate parent for this method,
// check if the immediate parent itself already has an LSP violation with this ancestor.
// If so, don't report the same violation for the child class -- it would be a false positive
// since the child cannot fix the violation without contradicting its immediate parent's contract.
// See: https://github.com/astral-sh/ty/issues/2000
if let Some((immediate_parent, immediate_parent_type)) = immediate_parent_method {
if immediate_parent != superclass {
// The immediate parent already defines this method and is different from the
// current ancestor we're checking. Check if the immediate parent's method
// is also incompatible with this ancestor.
if !immediate_parent_type.is_assignable_to(db, superclass_type_as_type) {
// The immediate parent already has an LSP violation with this ancestor.
// Don't report the same violation for the child.
continue;
}
}
}
report_invalid_method_override(
context,
&member.name,

View File

@@ -265,26 +265,29 @@ Instead, apply the `# fmt: off` comment to the entire statement:
Like Black, Ruff will _also_ recognize [YAPF](https://github.com/google/yapf)'s `# yapf: disable` and `# yapf: enable` pragma
comments, which are treated equivalently to `# fmt: off` and `# fmt: on`, respectively.
`# fmt: skip` comments suppress formatting for a preceding statement, case header, decorator,
function definition, or class definition:
`# fmt: skip` comments suppress formatting for a case header, decorator,
function definition, class definition, or the preceding statements
on the same logical line. The formatter leaves the following unchanged:
```python
if True:
pass
elif False: # fmt: skip
elif False: # fmt: skip
pass
@Test
@Test2 # fmt: skip
@Test2(a,b) # fmt: skip
def test(): ...
a = [1, 2, 3, 4, 5] # fmt: skip
a = [1,2,3,4,5] # fmt: skip
def test(a, b, c, d, e, f) -> int: # fmt: skip
def test(a,b,c,d,e,f) -> int: # fmt: skip
pass
x=1;x=2;x=3 # fmt: skip
```
As such, adding an `# fmt: skip` comment at the end of an expression will have no effect. In
Adding a `# fmt: skip` comment at the end of an expression will have no effect. In
the following example, the list entry `'1'` will be formatted, despite the `# fmt: skip`:
```python

View File

@@ -22,7 +22,7 @@ cd ..
echo "Project selector: ${PRIMER_SELECTOR}"
# Allow the exit code to be 0 or 1, only fail for actual mypy_primer crashes/bugs
uvx \
--from="git+https://github.com/hauntsaninja/mypy_primer@a81360123447a9409ab72f6b4f9684c02a9768e7" \
--from="git+https://github.com/hauntsaninja/mypy_primer@ea7c45364a129b9a556b08a6285f8df0e5aedf0a" \
mypy_primer \
--repo ruff \
--type-checker ty \