Compare commits

...

18 Commits

Author SHA1 Message Date
Micha Reiser
5079d91a3a macos 14 runner, no deployment target 2024-10-17 10:49:34 +02:00
Micha Reiser
3385611795 13 should fail during testing 2024-10-17 10:35:24 +02:00
Micha Reiser
875f1c671e Fix path to ruff executable 2024-10-17 08:44:08 +02:00
Micha Reiser
1affcf3c86 Try version 12.3 2024-10-17 08:35:23 +02:00
Micha Reiser
62a7bca5d8 Try version 13 2024-10-17 08:28:42 +02:00
Micha Reiser
66b3cecc27 Try version 12 2024-10-17 08:21:15 +02:00
Micha Reiser
31a892b1ed Experiment with MACOS_DEPLOYMENT_TARGET 2024-10-17 08:13:32 +02:00
Simon Brugman
6b7a738825 Add explanation of fixable in --statistics command (#13774)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-10-17 08:02:00 +02:00
Santhosh Solomon
4ea4bbb155 [flake8-bandit] Detect patterns from multi line SQL statements (S608) (#13574)
Co-authored-by: Santhosh Solomon <santhosh@advarisk.com>
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-10-17 05:42:03 +00:00
aditya pillai
ed4a0b34ba [red-knot] don't include Unknown in the type for a conditionally-defined import (#13563)
## Summary

Fixes the bug described in #13514 where an unbound public type defaulted
to the type or `Unknown`, whereas it should only be the type if unbound.

## Test Plan

Added a new test case

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2024-10-16 13:46:03 -07:00
Micha Reiser
2095ea8372 Add scope assertion to TypeInference.extend (#13764)
## Summary

This PR adds a debug assertion that asserts that `TypeInference::extend`
is only called on results that have the same scope.
This is critical because `expressions` uses `ScopedExpressionId` that
are local and merging expressions from different
scopes would lead to incorrect expression types.

We could consider storing `scope` only on `TypeInference` for debug
builds. Doing so has the advantage that the `TypeInference` type is
smaller of which we'll have many. However, a `ScopeId` is a `u32`... so
it shouldn't matter that much and it avoids storing the `scope` both on
`TypeInference` and `TypeInferenceBuilder`

## Test Plan

`cargo test`
2024-10-16 08:44:25 -07:00
Alex Waygood
6282402a8c [red-knot] Add control flow for try/except blocks (#13729) 2024-10-16 13:03:59 +00:00
Raphael Gaschignard
d25673f664 [red-knot] Do not panic if named expressions show up in assignment position (#13711)
Co-authored-by: Carl Meyer <carl@astral.sh>
2024-10-16 12:42:39 +00:00
Micha Reiser
a94914dc35 Enable preview mode for 'unstable' black tests (#13776) 2024-10-16 12:25:34 +00:00
cake-monotone
2ffc3fad47 [red-knot] Implement Type::Tuple Comparisons (#13712)
## Summary

This PR implements comparisons for (tuple, tuple).

It will close #13688 and complete an item in #13618 once merged.

## Test Plan

Basic tests are included for (tuple, tuple) comparisons.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2024-10-16 11:39:55 +00:00
Micha Reiser
8f5b2aac9a Refactor: Remove StringPart and AnyStringPart in favor of StringLikePart (#13772) 2024-10-16 12:52:06 +02:00
David Peter
b85be6297e [red knot] Minor follow-up tasks regarding singleton types (#13769)
## Summary

- Do not treat empty tuples as singletons after discussion [1]
- Improve comment regarding intersection types
- Resolve unnecessary TODO in Markdown test

[1]
https://discuss.python.org/t/should-we-specify-in-the-language-reference-that-the-empty-tuple-is-a-singleton/67957

## Test Plan

—
2024-10-16 11:30:03 +02:00
Alex Waygood
fb1d1e3241 [red-knot] Simplify some branches in infer_subscript_expression (#13762)
## Summary

Just a small simplification to remove some unnecessary complexity here.
Rather than using separate branches for subscript expressions involving
boolean literals, we can simply convert them to integer literals and
reuse the logic in the `IntLiteral` branches.

## Test Plan

`cargo test -p red_knot_python_semantic`
2024-10-16 07:58:24 +01:00
65 changed files with 2689 additions and 1211 deletions

76
.github/workflows/macos-12.yml vendored Normal file
View File

@@ -0,0 +1,76 @@
name: "Test macOS 12 (x86_64)"
on:
pull_request
env:
PACKAGE_NAME: ruff
MODULE_NAME: ruff
PYTHON_VERSION: "3.11"
CARGO_INCREMENTAL: 0
CARGO_NET_RETRY: 10
CARGO_TERM_COLOR: always
RUSTUP_MAX_RETRIES: 10
jobs:
macos-x86_64:
runs-on: macos-14
# env:
# MACOSX_DEPLOYMENT_TARGET: 13.7
steps:
- uses: actions/checkout@v4
with:
submodules: recursive
- uses: actions/setup-python@v5
with:
python-version: ${{ env.PYTHON_VERSION }}
architecture: x64
- name: "Prep README.md"
run: python scripts/transform_readme.py --target pypi
- name: "Build wheels - x86_64"
uses: PyO3/maturin-action@v1
with:
target: x86_64
args: --release --locked --out dist
- name: "Upload wheels"
uses: actions/upload-artifact@v4
with:
name: wheels-macos-x86_64
path: dist
- name: "Archive binary"
run: |
TARGET=x86_64-apple-darwin
ARCHIVE_NAME=ruff-$TARGET
ARCHIVE_FILE=$ARCHIVE_NAME.tar.gz
mkdir -p $ARCHIVE_NAME
cp target/$TARGET/release/ruff $ARCHIVE_NAME/ruff
tar czvf $ARCHIVE_FILE $ARCHIVE_NAME
shasum -a 256 $ARCHIVE_FILE > $ARCHIVE_FILE.sha256
- name: "Upload binary"
uses: actions/upload-artifact@v4
with:
name: artifacts-macos-x86_64
path: |
*.tar.gz
*.sha256
test:
needs: macos-x86_64
runs-on: macos-12
steps:
- uses: actions/checkout@v4
with:
submodules: recursive
- uses: actions/setup-python@v5
with:
python-version: ${{ env.PYTHON_VERSION }}
architecture: x64
- name: "Download binaries"
uses: actions/download-artifact@v4
with:
name: artifacts-macos-x86_64
- name: "Extract"
run: tar xzvf ruff-x86_64-apple-darwin.tar.gz
- name: "Run Ruff"
run: ./ruff-x86_64-apple-darwin/ruff --help

View File

@@ -1,24 +1,16 @@
# Unbound
## Maybe unbound
```py
if flag:
y = 3
x = y
reveal_type(x) # revealed: Unbound | Literal[3]
```
## Unbound
```py
x = foo; foo = 1
x = foo
foo = 1
reveal_type(x) # revealed: Unbound
```
## Unbound class variable
Class variables can reference global variables unless overridden within the class scope.
Name lookups within a class scope fall back to globals, but lookups of class attributes don't.
```py
x = 1
@@ -27,6 +19,6 @@ class C:
if flag:
x = 2
reveal_type(C.x) # revealed: Unbound | Literal[2]
reveal_type(C.x) # revealed: Literal[2]
reveal_type(C.y) # revealed: Literal[1]
```

View File

@@ -0,0 +1,205 @@
# Comparison - Tuples
## Heterogeneous
For tuples like `tuple[int, str, Literal[1]]`
### Value Comparisons
"Value Comparisons" refers to the operators: `==`, `!=`, `<`, `<=`, `>`, `>=`
#### Results without Ambiguity
Cases where the result can be definitively inferred as a `BooleanLiteral`.
```py
a = (1, "test", (3, 13), True)
b = (1, "test", (3, 14), False)
reveal_type(a == a) # revealed: Literal[True]
reveal_type(a != a) # revealed: Literal[False]
reveal_type(a < a) # revealed: Literal[False]
reveal_type(a <= a) # revealed: Literal[True]
reveal_type(a > a) # revealed: Literal[False]
reveal_type(a >= a) # revealed: Literal[True]
reveal_type(a == b) # revealed: Literal[False]
reveal_type(a != b) # revealed: Literal[True]
reveal_type(a < b) # revealed: Literal[True]
reveal_type(a <= b) # revealed: Literal[True]
reveal_type(a > b) # revealed: Literal[False]
reveal_type(a >= b) # revealed: Literal[False]
```
Even when tuples have different lengths, comparisons should be handled appropriately.
```py path=different_length.py
a = (1, 2, 3)
b = (1, 2, 3, 4)
reveal_type(a == b) # revealed: Literal[False]
reveal_type(a != b) # revealed: Literal[True]
reveal_type(a < b) # revealed: Literal[True]
reveal_type(a <= b) # revealed: Literal[True]
reveal_type(a > b) # revealed: Literal[False]
reveal_type(a >= b) # revealed: Literal[False]
c = ("a", "b", "c", "d")
d = ("a", "b", "c")
reveal_type(c == d) # revealed: Literal[False]
reveal_type(c != d) # revealed: Literal[True]
reveal_type(c < d) # revealed: Literal[False]
reveal_type(c <= d) # revealed: Literal[False]
reveal_type(c > d) # revealed: Literal[True]
reveal_type(c >= d) # revealed: Literal[True]
```
#### Results with Ambiguity
```py
def bool_instance() -> bool: ...
def int_instance() -> int: ...
a = (bool_instance(),)
b = (int_instance(),)
# TODO: All @Todo should be `bool`
reveal_type(a == a) # revealed: @Todo
reveal_type(a != a) # revealed: @Todo
reveal_type(a < a) # revealed: @Todo
reveal_type(a <= a) # revealed: @Todo
reveal_type(a > a) # revealed: @Todo
reveal_type(a >= a) # revealed: @Todo
reveal_type(a == b) # revealed: @Todo
reveal_type(a != b) # revealed: @Todo
reveal_type(a < b) # revealed: @Todo
reveal_type(a <= b) # revealed: @Todo
reveal_type(a > b) # revealed: @Todo
reveal_type(a >= b) # revealed: @Todo
```
#### Comparison Unsupported
If two tuples contain types that do not support comparison, the result may be `Unknown`.
However, `==` and `!=` are exceptions and can still provide definite results.
```py
a = (1, 2)
b = (1, "hello")
# TODO: should be Literal[False]
reveal_type(a == b) # revealed: @Todo
# TODO: should be Literal[True]
reveal_type(a != b) # revealed: @Todo
# TODO: should be Unknown and add more informative diagnostics
reveal_type(a < b) # revealed: @Todo
reveal_type(a <= b) # revealed: @Todo
reveal_type(a > b) # revealed: @Todo
reveal_type(a >= b) # revealed: @Todo
```
However, if the lexicographic comparison completes without reaching a point where str and int are compared,
Python will still produce a result based on the prior elements.
```py path=short_circuit.py
a = (1, 2)
b = (999999, "hello")
reveal_type(a == b) # revealed: Literal[False]
reveal_type(a != b) # revealed: Literal[True]
reveal_type(a < b) # revealed: Literal[True]
reveal_type(a <= b) # revealed: Literal[True]
reveal_type(a > b) # revealed: Literal[False]
reveal_type(a >= b) # revealed: Literal[False]
```
#### Matryoshka Tuples
```py
a = (1, True, "Hello")
b = (a, a, a)
c = (b, b, b)
reveal_type(c == c) # revealed: Literal[True]
reveal_type(c != c) # revealed: Literal[False]
reveal_type(c < c) # revealed: Literal[False]
reveal_type(c <= c) # revealed: Literal[True]
reveal_type(c > c) # revealed: Literal[False]
reveal_type(c >= c) # revealed: Literal[True]
```
#### Non Boolean Rich Comparisons
```py
class A():
def __eq__(self, o) -> str: ...
def __ne__(self, o) -> int: ...
def __lt__(self, o) -> float: ...
def __le__(self, o) -> object: ...
def __gt__(self, o) -> tuple: ...
def __ge__(self, o) -> list: ...
a = (A(), A())
# TODO: All @Todo should be bool
reveal_type(a == a) # revealed: @Todo
reveal_type(a != a) # revealed: @Todo
reveal_type(a < a) # revealed: @Todo
reveal_type(a <= a) # revealed: @Todo
reveal_type(a > a) # revealed: @Todo
reveal_type(a >= a) # revealed: @Todo
```
### Membership Test Comparisons
"Membership Test Comparisons" refers to the operators `in` and `not in`.
```py
def int_instance() -> int: ...
a = (1, 2)
b = ((3, 4), (1, 2))
c = ((1, 2, 3), (4, 5, 6))
d = ((int_instance(), int_instance()), (int_instance(), int_instance()))
reveal_type(a in b) # revealed: Literal[True]
reveal_type(a not in b) # revealed: Literal[False]
reveal_type(a in c) # revealed: Literal[False]
reveal_type(a not in c) # revealed: Literal[True]
# TODO: All @Todo should be bool
reveal_type(a in d) # revealed: @Todo
reveal_type(a not in d) # revealed: @Todo
```
### Identity Comparisons
"Identity Comparisons" refers to `is` and `is not`.
```py
a = (1, 2)
b = ("a", "b")
c = (1, 2, 3)
reveal_type(a is (1, 2)) # revealed: bool
reveal_type(a is not (1, 2)) # revealed: bool
# TODO: Update to Literal[False] once str == int comparison is implemented
reveal_type(a is b) # revealed: @Todo
# TODO: Update to Literal[True] once str == int comparison is implemented
reveal_type(a is not b) # revealed: @Todo
reveal_type(a is c) # revealed: Literal[False]
reveal_type(a is not c) # revealed: Literal[True]
```
## Homogeneous
For tuples like `tuple[int, ...]`, `tuple[Any, ...]`
// TODO

View File

@@ -28,6 +28,7 @@ else:
y = 5
s = y
x = y
reveal_type(x) # revealed: Literal[3, 4, 5]
reveal_type(r) # revealed: Unbound | Literal[2]
reveal_type(s) # revealed: Unbound | Literal[5]

View File

@@ -0,0 +1,641 @@
# Control flow for exception handlers
These tests assert that we understand the possible "definition states" (which
symbols might or might not be defined) in the various branches of a
`try`/`except`/`else`/`finally` block.
For a full writeup on the semantics of exception handlers,
see [this document][1].
The tests throughout this Markdown document use functions with names starting
with `could_raise_*` to mark definitions that might or might not succeed
(as the function could raise an exception). A type checker must assume that any
arbitrary function call could raise an exception in Python; this is just a
naming convention used in these tests for clarity, and to future-proof the
tests against possible future improvements whereby certain statements or
expressions could potentially be inferred as being incapable of causing an
exception to be raised.
## A single bare `except`
Consider the following `try`/`except` block, with a single bare `except:`.
There are different types for the variable `x` in the two branches of this
block, and we can't determine which branch might have been taken from the
perspective of code following this block. The inferred type after the block's
conclusion is therefore the union of the type at the end of the `try` suite
(`str`) and the type at the end of the `except` suite (`Literal[2]`).
*Within* the `except` suite, we must infer a union of all possible "definition
states" we could have been in at any point during the `try` suite. This is
because control flow could have jumped to the `except` suite without any of the
`try`-suite definitions successfully completing, with only *some* of the
`try`-suite definitions successfully completing, or indeed with *all* of them
successfully completing. The type of `x` at the beginning of the `except` suite
in this example is therefore `Literal[1] | str`, taking into account that we
might have jumped to the `except` suite before the
`x = could_raise_returns_str()` redefinition, but we *also* could have jumped
to the `except` suite *after* that redefinition.
```py path=union_type_inferred.py
def could_raise_returns_str() -> str:
return 'foo'
x = 1
try:
reveal_type(x) # revealed: Literal[1]
x = could_raise_returns_str()
reveal_type(x) # revealed: str
except:
reveal_type(x) # revealed: Literal[1] | str
x = 2
reveal_type(x) # revealed: Literal[2]
reveal_type(x) # revealed: str | Literal[2]
```
If `x` has the same type at the end of both branches, however, the branches
unify and `x` is not inferred as having a union type following the
`try`/`except` block:
```py path=branches_unify_to_non_union_type.py
def could_raise_returns_str() -> str:
return 'foo'
x = 1
try:
x = could_raise_returns_str()
except:
x = could_raise_returns_str()
reveal_type(x) # revealed: str
```
## A non-bare `except`
For simple `try`/`except` blocks, an `except TypeError:` handler has the same
control flow semantics as an `except:` handler. An `except TypeError:` handler
will not catch *all* exceptions: if this is the only handler, it opens up the
possibility that an exception might occur that would not be handled. However,
as described in [the document on exception-handling semantics][1], that would
lead to termination of the scope. It's therefore irrelevant to consider this
possibility when it comes to control-flow analysis.
```py
def could_raise_returns_str() -> str:
return 'foo'
x = 1
try:
reveal_type(x) # revealed: Literal[1]
x = could_raise_returns_str()
reveal_type(x) # revealed: str
except TypeError:
reveal_type(x) # revealed: Literal[1] | str
x = 2
reveal_type(x) # revealed: Literal[2]
reveal_type(x) # revealed: str | Literal[2]
```
## Multiple `except` branches
If the scope reaches the final `reveal_type` call in this example,
either the `try`-block suite of statements was executed in its entirety,
or exactly one `except` suite was executed in its entirety.
The inferred type of `x` at this point is the union of the types at the end of
the three suites:
- At the end of `try`, `type(x) == str`
- At the end of `except TypeError`, `x == 2`
- At the end of `except ValueError`, `x == 3`
```py
def could_raise_returns_str() -> str:
return 'foo'
x = 1
try:
reveal_type(x) # revealed: Literal[1]
x = could_raise_returns_str()
reveal_type(x) # revealed: str
except TypeError:
reveal_type(x) # revealed: Literal[1] | str
x = 2
reveal_type(x) # revealed: Literal[2]
except ValueError:
reveal_type(x) # revealed: Literal[1] | str
x = 3
reveal_type(x) # revealed: Literal[3]
reveal_type(x) # revealed: str | Literal[2, 3]
```
## Exception handlers with `else` branches (but no `finally`)
If we reach the `reveal_type` call at the end of this scope,
either the `try` and `else` suites were both executed in their entireties,
or the `except` suite was executed in its entirety. The type of `x` at this
point is the union of the type at the end of the `else` suite and the type at
the end of the `except` suite:
- At the end of `else`, `x == 3`
- At the end of `except`, `x == 2`
```py path=single_except.py
def could_raise_returns_str() -> str:
return 'foo'
x = 1
try:
reveal_type(x) # revealed: Literal[1]
x = could_raise_returns_str()
reveal_type(x) # revealed: str
except TypeError:
reveal_type(x) # revealed: Literal[1] | str
x = 2
reveal_type(x) # revealed: Literal[2]
else:
reveal_type(x) # revealed: str
x = 3
reveal_type(x) # revealed: Literal[3]
reveal_type(x) # revealed: Literal[2, 3]
```
For a block that has multiple `except` branches and an `else` branch, the same
principle applies. In order to reach the final `reveal_type` call,
either exactly one of the `except` suites must have been executed in its
entirety, or the `try` suite and the `else` suite must both have been executed
in their entireties:
```py
def could_raise_returns_str() -> str:
return 'foo'
x = 1
try:
reveal_type(x) # revealed: Literal[1]
x = could_raise_returns_str()
reveal_type(x) # revealed: str
except TypeError:
reveal_type(x) # revealed: Literal[1] | str
x = 2
reveal_type(x) # revealed: Literal[2]
except ValueError:
reveal_type(x) # revealed: Literal[1] | str
x = 3
reveal_type(x) # revealed: Literal[3]
else:
reveal_type(x) # revealed: str
x = 4
reveal_type(x) # revealed: Literal[4]
reveal_type(x) # revealed: Literal[2, 3, 4]
```
## Exception handlers with `finally` branches (but no `except` branches)
A `finally` suite is *always* executed. As such, if we reach the `reveal_type`
call at the end of this example, we know that `x` *must* have been reassigned
to `2` during the `finally` suite. The type of `x` at the end of the example is
therefore `Literal[2]`:
```py path=redef_in_finally.py
def could_raise_returns_str() -> str:
return 'foo'
x = 1
try:
reveal_type(x) # revealed: Literal[1]
x = could_raise_returns_str()
reveal_type(x) # revealed: str
finally:
x = 2
reveal_type(x) # revealed: Literal[2]
reveal_type(x) # revealed: Literal[2]
```
If `x` was *not* redefined in the `finally` suite, however, things are somewhat
more complicated. If we reach the final `reveal_type` call,
unlike the state when we're visiting the `finally` suite,
we know that the `try`-block suite ran to completion.
This means that there are fewer possible states at this point than there were
when we were inside the `finally` block.
(Our current model does *not* correctly infer the types *inside* `finally`
suites, however; this is still a TODO item for us.)
```py path=no_redef_in_finally.py
def could_raise_returns_str() -> str:
return 'foo'
x = 1
try:
reveal_type(x) # revealed: Literal[1]
x = could_raise_returns_str()
reveal_type(x) # revealed: str
finally:
# TODO: should be Literal[1] | str
reveal_type(x) # revealed: str
reveal_type(x) # revealed: str
```
## Combining an `except` branch with a `finally` branch
As previously stated, we do not yet have accurate inference for types *inside*
`finally` suites. When we do, however, we will have to take account of the
following possibilities inside `finally` suites:
- The `try` suite could have run to completion
- Or we could have jumped from halfway through the `try` suite to an `except`
suite, and the `except` suite ran to completion
- Or we could have jumped from halfway through the `try` suite straight to the
`finally` suite due to an unhandled exception
- Or we could have jumped from halfway through the `try` suite to an
`except` suite, only for an exception raised in the `except` suite to cause
us to jump to the `finally` suite before the `except` suite ran to completion
```py path=redef_in_finally.py
def could_raise_returns_str() -> str:
return 'foo'
def could_raise_returns_bytes() -> bytes:
return b'foo'
def could_raise_returns_bool() -> bool:
return True
x = 1
try:
reveal_type(x) # revealed: Literal[1]
x = could_raise_returns_str()
reveal_type(x) # revealed: str
except TypeError:
reveal_type(x) # revealed: Literal[1] | str
x = could_raise_returns_bytes()
reveal_type(x) # revealed: bytes
x = could_raise_returns_bool()
reveal_type(x) # revealed: bool
finally:
# TODO: should be `Literal[1] | str | bytes | bool`
reveal_type(x) # revealed: str | bool
x = 2
reveal_type(x) # revealed: Literal[2]
reveal_type(x) # revealed: Literal[2]
```
Now for an example without a redefinition in the `finally` suite.
As before, there *should* be fewer possibilities after completion of the
`finally` suite than there were during the `finally` suite itself.
(In some control-flow possibilities, some exceptions were merely *suspended*
during the `finally` suite; these lead to the scope's termination following the
conclusion of the `finally` suite.)
```py path=no_redef_in_finally.py
def could_raise_returns_str() -> str:
return 'foo'
def could_raise_returns_bytes() -> bytes:
return b'foo'
def could_raise_returns_bool() -> bool:
return True
x = 1
try:
reveal_type(x) # revealed: Literal[1]
x = could_raise_returns_str()
reveal_type(x) # revealed: str
except TypeError:
reveal_type(x) # revealed: Literal[1] | str
x = could_raise_returns_bytes()
reveal_type(x) # revealed: bytes
x = could_raise_returns_bool()
reveal_type(x) # revealed: bool
finally:
# TODO: should be `Literal[1] | str | bytes | bool`
reveal_type(x) # revealed: str | bool
reveal_type(x) # revealed: str | bool
```
An example with multiple `except` branches and a `finally` branch:
```py path=multiple_except_branches.py
def could_raise_returns_str() -> str:
return 'foo'
def could_raise_returns_bytes() -> bytes:
return b'foo'
def could_raise_returns_bool() -> bool:
return True
def could_raise_returns_memoryview() -> memoryview:
return memoryview(b"")
def could_raise_returns_float() -> float:
return 3.14
x = 1
try:
reveal_type(x) # revealed: Literal[1]
x = could_raise_returns_str()
reveal_type(x) # revealed: str
except TypeError:
reveal_type(x) # revealed: Literal[1] | str
x = could_raise_returns_bytes()
reveal_type(x) # revealed: bytes
x = could_raise_returns_bool()
reveal_type(x) # revealed: bool
except ValueError:
reveal_type(x) # revealed: Literal[1] | str
x = could_raise_returns_memoryview()
reveal_type(x) # revealed: memoryview
x = could_raise_returns_float()
reveal_type(x) # revealed: float
finally:
# TODO: should be `Literal[1] | str | bytes | bool | memoryview | float`
reveal_type(x) # revealed: str | bool | float
reveal_type(x) # revealed: str | bool | float
```
## Combining `except`, `else` and `finally` branches
If the exception handler has an `else` branch, we must also take into account
the possibility that control flow could have jumped to the `finally` suite from
partway through the `else` suite due to an exception raised *there*.
```py path=single_except_branch.py
def could_raise_returns_str() -> str:
return 'foo'
def could_raise_returns_bytes() -> bytes:
return b'foo'
def could_raise_returns_bool() -> bool:
return True
def could_raise_returns_memoryview() -> memoryview:
return memoryview(b"")
def could_raise_returns_float() -> float:
return 3.14
x = 1
try:
reveal_type(x) # revealed: Literal[1]
x = could_raise_returns_str()
reveal_type(x) # revealed: str
except TypeError:
reveal_type(x) # revealed: Literal[1] | str
x = could_raise_returns_bytes()
reveal_type(x) # revealed: bytes
x = could_raise_returns_bool()
reveal_type(x) # revealed: bool
else:
reveal_type(x) # revealed: str
x = could_raise_returns_memoryview()
reveal_type(x) # revealed: memoryview
x = could_raise_returns_float()
reveal_type(x) # revealed: float
finally:
# TODO: should be `Literal[1] | str | bytes | bool | memoryview | float`
reveal_type(x) # revealed: bool | float
reveal_type(x) # revealed: bool | float
```
The same again, this time with multiple `except` branches:
```py path=multiple_except_branches.py
def could_raise_returns_str() -> str:
return 'foo'
def could_raise_returns_bytes() -> bytes:
return b'foo'
def could_raise_returns_bool() -> bool:
return True
def could_raise_returns_memoryview() -> memoryview:
return memoryview(b"")
def could_raise_returns_float() -> float:
return 3.14
def could_raise_returns_range() -> range:
return range(42)
def could_raise_returns_slice() -> slice:
return slice(None)
x = 1
try:
reveal_type(x) # revealed: Literal[1]
x = could_raise_returns_str()
reveal_type(x) # revealed: str
except TypeError:
reveal_type(x) # revealed: Literal[1] | str
x = could_raise_returns_bytes()
reveal_type(x) # revealed: bytes
x = could_raise_returns_bool()
reveal_type(x) # revealed: bool
except ValueError:
reveal_type(x) # revealed: Literal[1] | str
x = could_raise_returns_memoryview()
reveal_type(x) # revealed: memoryview
x = could_raise_returns_float()
reveal_type(x) # revealed: float
else:
reveal_type(x) # revealed: str
x = could_raise_returns_range()
reveal_type(x) # revealed: range
x = could_raise_returns_slice()
reveal_type(x) # revealed: slice
finally:
# TODO: should be `Literal[1] | str | bytes | bool | memoryview | float | range | slice`
reveal_type(x) # revealed: bool | float | slice
reveal_type(x) # revealed: bool | float | slice
```
## Nested `try`/`except` blocks
It would take advanced analysis, which we are not yet capable of, to be able
to determine that an exception handler always suppresses all exceptions. This
is partly because it is possible for statements in `except`, `else` and
`finally` suites to raise exceptions as well as statements in `try` suites.
This means that if an exception handler is nested inside the `try` statement of
an enclosing exception handler, it should (at least for now) be treated the
same as any other node: as a suite containing statements that could possibly
raise exceptions, which would lead to control flow jumping out of that suite
prior to the suite running to completion.
```py
def could_raise_returns_str() -> str:
return 'foo'
def could_raise_returns_bytes() -> bytes:
return b'foo'
def could_raise_returns_bool() -> bool:
return True
def could_raise_returns_memoryview() -> memoryview:
return memoryview(b"")
def could_raise_returns_float() -> float:
return 3.14
def could_raise_returns_range() -> range:
return range(42)
def could_raise_returns_slice() -> slice:
return slice(None)
def could_raise_returns_complex() -> complex:
return 3j
def could_raise_returns_bytearray() -> bytearray:
return bytearray()
class Foo: ...
class Bar: ...
def could_raise_returns_Foo() -> Foo:
return Foo()
def could_raise_returns_Bar() -> Bar:
return Bar()
x = 1
try:
try:
reveal_type(x) # revealed: Literal[1]
x = could_raise_returns_str()
reveal_type(x) # revealed: str
except TypeError:
reveal_type(x) # revealed: Literal[1] | str
x = could_raise_returns_bytes()
reveal_type(x) # revealed: bytes
x = could_raise_returns_bool()
reveal_type(x) # revealed: bool
except ValueError:
reveal_type(x) # revealed: Literal[1] | str
x = could_raise_returns_memoryview()
reveal_type(x) # revealed: memoryview
x = could_raise_returns_float()
reveal_type(x) # revealed: float
else:
reveal_type(x) # revealed: str
x = could_raise_returns_range()
reveal_type(x) # revealed: range
x = could_raise_returns_slice()
reveal_type(x) # revealed: slice
finally:
# TODO: should be `Literal[1] | str | bytes | bool | memoryview | float | range | slice`
reveal_type(x) # revealed: bool | float | slice
x = 2
reveal_type(x) # revealed: Literal[2]
reveal_type(x) # revealed: Literal[2]
except:
reveal_type(x) # revealed: Literal[1, 2] | str | bytes | bool | memoryview | float | range | slice
x = could_raise_returns_complex()
reveal_type(x) # revealed: complex
x = could_raise_returns_bytearray()
reveal_type(x) # revealed: bytearray
else:
reveal_type(x) # revealed: Literal[2]
x = could_raise_returns_Foo()
reveal_type(x) # revealed: Foo
x = could_raise_returns_Bar()
reveal_type(x) # revealed: Bar
finally:
# TODO: should be `Literal[1, 2] | str | bytes | bool | memoryview | float | range | slice | complex | bytearray | Foo | Bar`
reveal_type(x) # revealed: bytearray | Bar
# Either one `except` branch or the `else`
# must have been taken and completed to get here:
reveal_type(x) # revealed: bytearray | Bar
```
## Nested scopes inside `try` blocks
Shadowing a variable in an inner scope has no effect on type inference of the
variable by that name in the outer scope:
```py
def could_raise_returns_str() -> str:
return 'foo'
def could_raise_returns_bytes() -> bytes:
return b'foo'
def could_raise_returns_range() -> range:
return range(42)
def could_raise_returns_bytearray() -> bytearray:
return bytearray()
def could_raise_returns_float() -> float:
return 3.14
x = 1
try:
def foo(param=could_raise_returns_str()):
x = could_raise_returns_str()
try:
reveal_type(x) # revealed: str
x = could_raise_returns_bytes()
reveal_type(x) # revealed: bytes
except:
reveal_type(x) # revealed: str | bytes
x = could_raise_returns_bytearray()
reveal_type(x) # revealed: bytearray
x = could_raise_returns_float()
reveal_type(x) # revealed: float
finally:
# TODO: should be `str | bytes | bytearray | float`
reveal_type(x) # revealed: bytes | float
reveal_type(x) # revealed: bytes | float
x = foo
reveal_type(x) # revealed: Literal[foo]
except:
reveal_type(x) # revealed: Literal[1] | Literal[foo]
class Bar:
x = could_raise_returns_range()
reveal_type(x) # revealed: range
x = Bar
reveal_type(x) # revealed: Literal[Bar]
finally:
# TODO: should be `Literal[1] | Literal[foo] | Literal[Bar]`
reveal_type(x) # revealed: Literal[foo] | Literal[Bar]
reveal_type(x) # revealed: Literal[foo] | Literal[Bar]
```
[1]: https://astral-sh.notion.site/Exception-handler-control-flow-11348797e1ca80bb8ce1e9aedbbe439d

View File

@@ -1,14 +1,12 @@
# Except star
TODO(Alex): Once we support `sys.version_info` branches, we can set `--target-version=py311` in these tests and the inferred type will just be `BaseExceptionGroup`
## Except\* with BaseException
```py
try:
x
except* BaseException as e:
reveal_type(e) # revealed: Unknown | BaseExceptionGroup
reveal_type(e) # revealed: BaseExceptionGroup
```
## Except\* with specific exception
@@ -18,7 +16,7 @@ try:
x
except* OSError as e:
# TODO(Alex): more precise would be `ExceptionGroup[OSError]`
reveal_type(e) # revealed: Unknown | BaseExceptionGroup
reveal_type(e) # revealed: BaseExceptionGroup
```
## Except\* with multiple exceptions
@@ -28,5 +26,5 @@ try:
x
except* (TypeError, AttributeError) as e:
#TODO(Alex): more precise would be `ExceptionGroup[TypeError | AttributeError]`.
reveal_type(e) # revealed: Unknown | BaseExceptionGroup
reveal_type(e) # revealed: BaseExceptionGroup
```

View File

@@ -1,5 +1,39 @@
# Conditional imports
## Maybe unbound
```py path=maybe_unbound.py
if flag:
y = 3
x = y
reveal_type(x) # revealed: Unbound | Literal[3]
reveal_type(y) # revealed: Unbound | Literal[3]
```
```py
from maybe_unbound import x, y
reveal_type(x) # revealed: Literal[3]
reveal_type(y) # revealed: Literal[3]
```
## Maybe unbound annotated
```py path=maybe_unbound_annotated.py
if flag:
y: int = 3
x = y
reveal_type(x) # revealed: Unbound | Literal[3]
reveal_type(y) # revealed: Unbound | Literal[3]
```
Importing an annotated name prefers the declared type over the inferred type:
```py
from maybe_unbound_annotated import x, y
reveal_type(x) # revealed: Literal[3]
reveal_type(y) # revealed: int
```
## Reimport
```py path=c.py
@@ -14,8 +48,7 @@ else:
```
```py
# TODO we should not emit this error
from b import f # error: [invalid-assignment] "Object of type `Literal[f, f]` is not assignable to `Literal[f, f]`"
from b import f
# TODO: We should disambiguate in such cases, showing `Literal[b.f, c.f]`.
reveal_type(f) # revealed: Literal[f, f]
```

View File

@@ -4,12 +4,14 @@
```py
import bar # error: "Cannot resolve import `bar`"
reveal_type(bar) # revealed: Unknown
```
## Unresolved import from statement
```py
from bar import baz # error: "Cannot resolve import `bar`"
reveal_type(baz) # revealed: Unknown
```
## Unresolved import from resolved module
@@ -19,12 +21,14 @@ from bar import baz # error: "Cannot resolve import `bar`"
```py
from a import thing # error: "Module `a` has no member `thing`"
reveal_type(thing) # revealed: Unknown
```
## Resolved import of symbol from unresolved import
```py path=a.py
import foo as foo # error: "Cannot resolve import `foo`"
reveal_type(foo) # revealed: Unknown
```
Importing the unresolved import into a second file should not trigger an additional "unresolved
@@ -32,9 +36,10 @@ import" violation:
```py
from a import foo
reveal_type(foo) # revealed: Unknown
```
## No implicit shadowing error
## No implicit shadowing
```py path=b.py
x: int
@@ -43,5 +48,5 @@ x: int
```py
from b import x
x = 'foo' # error: "Object of type `Literal["foo"]"
x = 'foo' # error: [invalid-assignment] "Object of type `Literal["foo"]"
```

View File

@@ -126,7 +126,7 @@ reveal_type(y) # revealed: Unknown
```
```py path=package/bar.py
# TODO: submodule imports possibly not supported right now?
# TODO: support submodule imports
from . import foo # error: [unresolved-import]
reveal_type(foo) # revealed: Unknown

View File

@@ -31,10 +31,9 @@ non-singleton class may occupy different addresses in memory even if
they compare equal.
```py
x = [1]
y = [1]
x = 345
y = 345
if x is not y:
# TODO: should include type parameter: list[int]
reveal_type(x) # revealed: list
reveal_type(x) # revealed: Literal[345]
```

View File

@@ -0,0 +1,32 @@
# Builtin scope
## Conditionally global or builtin
If a builtin name is conditionally defined as a global, a name lookup should union the builtin type
with the conditionally-defined type:
```py
def returns_bool() -> bool:
return True
if returns_bool():
copyright = 1
def f():
reveal_type(copyright) # revealed: Literal[copyright] | Literal[1]
```
## Conditionally global or builtin, with annotation
Same is true if the name is annotated:
```py
def returns_bool() -> bool:
return True
if returns_bool():
copyright: int = 1
def f():
reveal_type(copyright) # revealed: Literal[copyright] | int
```

View File

@@ -0,0 +1,33 @@
# List subscripts
## Indexing into lists
A list can be indexed into with:
- numbers
- slices
```py
x = [1, 2, 3]
reveal_type(x) # revealed: list
# TODO reveal int
reveal_type(x[0]) # revealed: @Todo
# TODO reveal list
reveal_type(x[0:1]) # revealed: @Todo
# TODO error
reveal_type(x["a"]) # revealed: @Todo
```
## Assignments within list assignment
In assignment, we might also have a named assignment.
This should also get type checked.
```py
x = [1, 2, 3]
x[0 if (y := 2) else 1] = 5
# TODO error? (indeterminite index type)
x["a" if (y := 2) else 1] = 6
# TODO error (can't index via string)
x["a" if (y := 2) else "b"] = 6
```

View File

@@ -1,5 +1,6 @@
use std::sync::Arc;
use except_handlers::TryNodeContextStackManager;
use rustc_hash::FxHashMap;
use ruff_db::files::File;
@@ -32,18 +33,23 @@ use super::definition::{
MatchPatternDefinitionNodeRef, WithItemDefinitionNodeRef,
};
mod except_handlers;
pub(super) struct SemanticIndexBuilder<'db> {
// Builder state
db: &'db dyn Db,
file: File,
module: &'db ParsedModule,
scope_stack: Vec<FileScopeId>,
/// The assignment we're currently visiting.
current_assignment: Option<CurrentAssignment<'db>>,
/// The assignments we're currently visiting, with
/// the most recent visit at the end of the Vec
current_assignments: Vec<CurrentAssignment<'db>>,
/// The match case we're currently visiting.
current_match_case: Option<CurrentMatchCase<'db>>,
/// Flow states at each `break` in the current loop.
loop_break_states: Vec<FlowSnapshot>,
/// Per-scope contexts regarding nested `try`/`except` statements
try_node_context_stack_manager: TryNodeContextStackManager,
/// Flags about the file's global scope
has_future_annotations: bool,
@@ -67,9 +73,10 @@ impl<'db> SemanticIndexBuilder<'db> {
file,
module: parsed,
scope_stack: Vec::new(),
current_assignment: None,
current_assignments: vec![],
current_match_case: None,
loop_break_states: vec![],
try_node_context_stack_manager: TryNodeContextStackManager::default(),
has_future_annotations: false,
@@ -110,6 +117,7 @@ impl<'db> SemanticIndexBuilder<'db> {
kind: node.scope_kind(),
descendents: children_start..children_start,
};
self.try_node_context_stack_manager.enter_nested_scope();
let file_scope_id = self.scopes.push(scope);
self.symbol_tables.push(SymbolTableBuilder::new());
@@ -139,6 +147,7 @@ impl<'db> SemanticIndexBuilder<'db> {
let children_end = self.scopes.next_index();
let scope = &mut self.scopes[id];
scope.descendents = scope.descendents.start..children_end;
self.try_node_context_stack_manager.exit_scope();
id
}
@@ -227,6 +236,10 @@ impl<'db> SemanticIndexBuilder<'db> {
DefinitionCategory::Binding => use_def.record_binding(symbol, definition),
}
let mut try_node_stack_manager = std::mem::take(&mut self.try_node_context_stack_manager);
try_node_stack_manager.record_definition(self);
self.try_node_context_stack_manager = try_node_stack_manager;
definition
}
@@ -238,6 +251,19 @@ impl<'db> SemanticIndexBuilder<'db> {
expression
}
fn push_assignment(&mut self, assignment: CurrentAssignment<'db>) {
self.current_assignments.push(assignment);
}
fn pop_assignment(&mut self) {
let popped_assignment = self.current_assignments.pop();
debug_assert!(popped_assignment.is_some());
}
fn current_assignment(&self) -> Option<&CurrentAssignment<'db>> {
self.current_assignments.last()
}
fn add_pattern_constraint(
&mut self,
subject: &ast::Expr,
@@ -359,12 +385,12 @@ impl<'db> SemanticIndexBuilder<'db> {
self.visit_expr(&generator.iter);
self.push_scope(scope);
self.current_assignment = Some(CurrentAssignment::Comprehension {
self.push_assignment(CurrentAssignment::Comprehension {
node: generator,
first: true,
});
self.visit_expr(&generator.target);
self.current_assignment = None;
self.pop_assignment();
for expr in &generator.ifs {
self.visit_expr(expr);
@@ -374,12 +400,12 @@ impl<'db> SemanticIndexBuilder<'db> {
self.add_standalone_expression(&generator.iter);
self.visit_expr(&generator.iter);
self.current_assignment = Some(CurrentAssignment::Comprehension {
self.push_assignment(CurrentAssignment::Comprehension {
node: generator,
first: false,
});
self.visit_expr(&generator.target);
self.current_assignment = None;
self.pop_assignment();
for expr in &generator.ifs {
self.visit_expr(expr);
@@ -415,7 +441,7 @@ impl<'db> SemanticIndexBuilder<'db> {
self.pop_scope();
assert!(self.scope_stack.is_empty());
assert!(self.current_assignment.is_none());
assert_eq!(&self.current_assignments, &[]);
let mut symbol_tables: IndexVec<_, _> = self
.symbol_tables
@@ -563,7 +589,7 @@ where
}
}
ast::Stmt::Assign(node) => {
debug_assert!(self.current_assignment.is_none());
debug_assert_eq!(&self.current_assignments, &[]);
self.visit_expr(&node.value);
self.add_standalone_expression(&node.value);
for (target_index, target) in node.targets.iter().enumerate() {
@@ -573,25 +599,28 @@ where
_ => None,
};
if let Some(kind) = kind {
self.current_assignment = Some(CurrentAssignment::Assign {
self.push_assignment(CurrentAssignment::Assign {
assignment: node,
target_index,
kind,
});
}
self.visit_expr(target);
self.current_assignment = None;
if kind.is_some() {
// only need to pop in the case where we pushed something
self.pop_assignment();
}
}
}
ast::Stmt::AnnAssign(node) => {
debug_assert!(self.current_assignment.is_none());
debug_assert_eq!(&self.current_assignments, &[]);
self.visit_expr(&node.annotation);
if let Some(value) = &node.value {
self.visit_expr(value);
}
self.current_assignment = Some(node.into());
self.push_assignment(node.into());
self.visit_expr(&node.target);
self.current_assignment = None;
self.pop_assignment();
}
ast::Stmt::AugAssign(
aug_assign @ ast::StmtAugAssign {
@@ -601,11 +630,11 @@ where
value,
},
) => {
debug_assert!(self.current_assignment.is_none());
debug_assert_eq!(&self.current_assignments, &[]);
self.visit_expr(value);
self.current_assignment = Some(aug_assign.into());
self.push_assignment(aug_assign.into());
self.visit_expr(target);
self.current_assignment = None;
self.pop_assignment();
}
ast::Stmt::If(node) => {
self.visit_expr(&node.test);
@@ -673,9 +702,9 @@ where
self.visit_expr(&item.context_expr);
if let Some(optional_vars) = item.optional_vars.as_deref() {
self.add_standalone_expression(&item.context_expr);
self.current_assignment = Some(item.into());
self.push_assignment(item.into());
self.visit_expr(optional_vars);
self.current_assignment = None;
self.pop_assignment();
}
}
self.visit_body(body);
@@ -700,10 +729,10 @@ where
let pre_loop = self.flow_snapshot();
let saved_break_states = std::mem::take(&mut self.loop_break_states);
debug_assert!(self.current_assignment.is_none());
self.current_assignment = Some(for_stmt.into());
debug_assert_eq!(&self.current_assignments, &[]);
self.push_assignment(for_stmt.into());
self.visit_expr(target);
self.current_assignment = None;
self.pop_assignment();
// TODO: Definitions created by loop variables
// (and definitions created inside the body)
@@ -764,40 +793,104 @@ where
is_star,
range: _,
}) => {
// Save the state prior to visiting any of the `try` block.
//
// Potentially none of the `try` block could have been executed prior to executing
// the `except` block(s) and/or the `finally` block.
// We will merge this state with all of the intermediate
// states during the `try` block before visiting those suites.
let pre_try_block_state = self.flow_snapshot();
self.try_node_context_stack_manager.push_context();
// Visit the `try` block!
self.visit_body(body);
for except_handler in handlers {
let ast::ExceptHandler::ExceptHandler(except_handler) = except_handler;
let ast::ExceptHandlerExceptHandler {
name: symbol_name,
type_: handled_exceptions,
body: handler_body,
range: _,
} = except_handler;
let mut post_except_states = vec![];
if let Some(handled_exceptions) = handled_exceptions {
self.visit_expr(handled_exceptions);
// Take a record also of all the intermediate states we encountered
// while visiting the `try` block
let try_block_snapshots = self.try_node_context_stack_manager.pop_context();
if !handlers.is_empty() {
// Save the state immediately *after* visiting the `try` block
// but *before* we prepare for visiting the `except` block(s).
//
// We will revert to this state prior to visiting the the `else` block,
// as there necessarily must have been 0 `except` blocks executed
// if we hit the `else` block.
let post_try_block_state = self.flow_snapshot();
// Prepare for visiting the `except` block(s)
self.flow_restore(pre_try_block_state);
for state in try_block_snapshots {
self.flow_merge(state);
}
// If `handled_exceptions` above was `None`, it's something like `except as e:`,
// which is invalid syntax. However, it's still pretty obvious here that the user
// *wanted* `e` to be bound, so we should still create a definition here nonetheless.
if let Some(symbol_name) = symbol_name {
let symbol = self.add_symbol(symbol_name.id.clone());
let pre_except_state = self.flow_snapshot();
let num_handlers = handlers.len();
self.add_definition(
symbol,
DefinitionNodeRef::ExceptHandler(ExceptHandlerDefinitionNodeRef {
handler: except_handler,
is_star: *is_star,
}),
);
for (i, except_handler) in handlers.iter().enumerate() {
let ast::ExceptHandler::ExceptHandler(except_handler) = except_handler;
let ast::ExceptHandlerExceptHandler {
name: symbol_name,
type_: handled_exceptions,
body: handler_body,
range: _,
} = except_handler;
if let Some(handled_exceptions) = handled_exceptions {
self.visit_expr(handled_exceptions);
}
// If `handled_exceptions` above was `None`, it's something like `except as e:`,
// which is invalid syntax. However, it's still pretty obvious here that the user
// *wanted* `e` to be bound, so we should still create a definition here nonetheless.
if let Some(symbol_name) = symbol_name {
let symbol = self.add_symbol(symbol_name.id.clone());
self.add_definition(
symbol,
DefinitionNodeRef::ExceptHandler(ExceptHandlerDefinitionNodeRef {
handler: except_handler,
is_star: *is_star,
}),
);
}
self.visit_body(handler_body);
// Each `except` block is mutually exclusive with all other `except` blocks.
post_except_states.push(self.flow_snapshot());
// It's unnecessary to do the `self.flow_restore()` call for the final except handler,
// as we'll immediately call `self.flow_restore()` to a different state
// as soon as this loop over the handlers terminates.
if i < (num_handlers - 1) {
self.flow_restore(pre_except_state.clone());
}
}
self.visit_body(handler_body);
// If we get to the `else` block, we know that 0 of the `except` blocks can have been executed,
// and the entire `try` block must have been executed:
self.flow_restore(post_try_block_state);
}
self.visit_body(orelse);
for post_except_state in post_except_states {
self.flow_merge(post_except_state);
}
// TODO: there's lots of complexity here that isn't yet handled by our model.
// In order to accurately model the semantics of `finally` suites, we in fact need to visit
// the suite twice: once under the (current) assumption that either the `try + else` suite
// ran to completion or exactly one `except` branch ran to completion, and then again under
// the assumption that potentially none of the branches ran to completion and we in fact
// jumped from a `try`, `else` or `except` branch straight into the `finally` branch.
// This requires rethinking some fundamental assumptions semantic indexing makes.
// For more details, see:
// - https://astral-sh.notion.site/Exception-handler-control-flow-11348797e1ca80bb8ce1e9aedbbe439d
// - https://github.com/astral-sh/ruff/pull/13633#discussion_r1788626702
self.visit_body(finalbody);
}
_ => {
@@ -813,7 +906,7 @@ where
match expr {
ast::Expr::Name(name_node @ ast::ExprName { id, ctx, .. }) => {
let (is_use, is_definition) = match (ctx, self.current_assignment) {
let (is_use, is_definition) = match (ctx, self.current_assignment()) {
(ast::ExprContext::Store, Some(CurrentAssignment::AugAssign(_))) => {
// For augmented assignment, the target expression is also used.
(true, true)
@@ -824,8 +917,9 @@ where
(ast::ExprContext::Invalid, _) => (false, false),
};
let symbol = self.add_symbol(id.clone());
if is_definition {
match self.current_assignment {
match self.current_assignment().copied() {
Some(CurrentAssignment::Assign {
assignment,
target_index,
@@ -896,12 +990,11 @@ where
walk_expr(self, expr);
}
ast::Expr::Named(node) => {
debug_assert!(self.current_assignment.is_none());
// TODO walrus in comprehensions is implicitly nonlocal
self.visit_expr(&node.value);
self.current_assignment = Some(node.into());
self.push_assignment(node.into());
self.visit_expr(&node.target);
self.current_assignment = None;
self.pop_assignment();
}
ast::Expr::Lambda(lambda) => {
if let Some(parameters) = &lambda.parameters {
@@ -1060,7 +1153,7 @@ where
}
}
#[derive(Copy, Clone, Debug)]
#[derive(Copy, Clone, Debug, PartialEq)]
enum CurrentAssignment<'a> {
Assign {
assignment: &'a ast::StmtAssign,

View File

@@ -0,0 +1,102 @@
use crate::semantic_index::use_def::FlowSnapshot;
use super::SemanticIndexBuilder;
/// An abstraction over the fact that each scope should have its own [`TryNodeContextStack`]
#[derive(Debug, Default)]
pub(super) struct TryNodeContextStackManager(Vec<TryNodeContextStack>);
impl TryNodeContextStackManager {
/// Push a new [`TryNodeContextStack`] onto the stack of stacks.
///
/// Each [`TryNodeContextStack`] is only valid for a single scope
pub(super) fn enter_nested_scope(&mut self) {
self.0.push(TryNodeContextStack::default());
}
/// Pop a new [`TryNodeContextStack`] off the stack of stacks.
///
/// Each [`TryNodeContextStack`] is only valid for a single scope
pub(super) fn exit_scope(&mut self) {
let popped_context = self.0.pop();
debug_assert!(
popped_context.is_some(),
"exit_scope() should never be called on an empty stack \
(this indicates an unbalanced `enter_nested_scope()`/`exit_scope()` pair of calls)"
);
}
/// Push a [`TryNodeContext`] onto the [`TryNodeContextStack`]
/// at the top of our stack of stacks
pub(super) fn push_context(&mut self) {
self.current_try_context_stack().push_context();
}
/// Pop a [`TryNodeContext`] off the [`TryNodeContextStack`]
/// at the top of our stack of stacks. Return the Vec of [`FlowSnapshot`]s
/// recorded while we were visiting the `try` suite.
pub(super) fn pop_context(&mut self) -> Vec<FlowSnapshot> {
self.current_try_context_stack().pop_context()
}
/// Retrieve the stack that is at the top of our stack of stacks.
/// For each `try` block on that stack, push the snapshot onto the `try` block
pub(super) fn record_definition(&mut self, builder: &SemanticIndexBuilder) {
self.current_try_context_stack().record_definition(builder);
}
/// Retrieve the [`TryNodeContextStack`] that is relevant for the current scope.
fn current_try_context_stack(&mut self) -> &mut TryNodeContextStack {
self.0
.last_mut()
.expect("There should always be at least one `TryBlockContexts` on the stack")
}
}
/// The contexts of nested `try`/`except` blocks for a single scope
#[derive(Debug, Default)]
struct TryNodeContextStack(Vec<TryNodeContext>);
impl TryNodeContextStack {
/// Push a new [`TryNodeContext`] for recording intermediate states
/// while visiting a [`ruff_python_ast::StmtTry`] node that has a `finally` branch.
fn push_context(&mut self) {
self.0.push(TryNodeContext::default());
}
/// Pop a [`TryNodeContext`] off the stack. Return the Vec of [`FlowSnapshot`]s
/// recorded while we were visiting the `try` suite.
fn pop_context(&mut self) -> Vec<FlowSnapshot> {
let TryNodeContext {
try_suite_snapshots,
} = self
.0
.pop()
.expect("Cannot pop a `try` block off an empty `TryBlockContexts` stack");
try_suite_snapshots
}
/// For each `try` block on the stack, push the snapshot onto the `try` block
fn record_definition(&mut self, builder: &SemanticIndexBuilder) {
for context in &mut self.0 {
context.record_definition(builder.flow_snapshot());
}
}
}
/// Context for tracking definitions over the course of a single
/// [`ruff_python_ast::StmtTry`] node
///
/// It will likely be necessary to add more fields to this struct in the future
/// when we add more advanced handling of `finally` branches.
#[derive(Debug, Default)]
struct TryNodeContext {
try_suite_snapshots: Vec<FlowSnapshot>,
}
impl TryNodeContext {
/// Take a record of what the internal state looked like after a definition
fn record_definition(&mut self, snapshot: FlowSnapshot) {
self.try_suite_snapshots.push(snapshot);
}
}

View File

@@ -37,6 +37,13 @@ fn core_module_symbol_ty<'db>(
) -> Type<'db> {
resolve_module(db, &core_module.name())
.map(|module| global_symbol_ty(db, module.file(), symbol))
.map(|ty| {
if ty.is_unbound() {
ty
} else {
ty.replace_unbound_with(db, Type::Never)
}
})
.unwrap_or(Type::Unbound)
}

View File

@@ -60,7 +60,7 @@ fn symbol_ty_by_id<'db>(db: &'db dyn Db, scope: ScopeId<'db>, symbol: ScopedSymb
use_def.public_bindings(symbol),
use_def
.public_may_be_unbound(symbol)
.then_some(Type::Unknown),
.then_some(Type::Unbound),
))
} else {
None
@@ -79,7 +79,7 @@ fn symbol_ty_by_id<'db>(db: &'db dyn Db, scope: ScopeId<'db>, symbol: ScopedSymb
}
}
/// Shorthand for `symbol_ty` that takes a symbol name instead of an ID.
/// Shorthand for `symbol_ty_by_id` that takes a symbol name instead of an ID.
fn symbol_ty<'db>(db: &'db dyn Db, scope: ScopeId<'db>, name: &str) -> Type<'db> {
let table = symbol_table(db, scope);
table
@@ -381,7 +381,7 @@ impl<'db> Type<'db> {
Type::Union(union) => {
union.map(db, |element| element.replace_unbound_with(db, replacement))
}
ty => *ty,
_ => *self,
}
}
@@ -444,6 +444,9 @@ impl<'db> Type<'db> {
///
/// [assignable to]: https://typing.readthedocs.io/en/latest/spec/concepts.html#the-assignable-to-or-consistent-subtyping-relation
pub(crate) fn is_assignable_to(self, db: &'db dyn Db, target: Type<'db>) -> bool {
if self.is_equivalent_to(db, target) {
return true;
}
match (self, target) {
(Type::Unknown | Type::Any | Type::Todo, _) => true,
(_, Type::Unknown | Type::Any | Type::Todo) => true,
@@ -467,7 +470,7 @@ impl<'db> Type<'db> {
///
/// Note: This function aims to have no false positives, but might return `false`
/// for more complicated types that are actually singletons.
pub(crate) fn is_singleton(self, db: &'db dyn Db) -> bool {
pub(crate) fn is_singleton(self) -> bool {
match self {
Type::Any
| Type::Never
@@ -485,14 +488,13 @@ impl<'db> Type<'db> {
false
}
Type::None | Type::BooleanLiteral(_) | Type::Function(..) | Type::Class(..) | Type::Module(..) => true,
Type::Tuple(tuple) => {
// We deliberately deviate from the language specification [1] here and claim
// that the empty tuple type is a singleton type. The reasoning is that `()`
// is often used as a sentinel value in user code. Declaring the empty tuple to
// be of singleton type allows us to narrow types in `is not ()` conditionals.
//
// [1] https://docs.python.org/3/reference/expressions.html#parenthesized-forms
tuple.elements(db).is_empty()
Type::Tuple(..) => {
// The empty tuple is a singleton on CPython and PyPy, but not on other Python
// implementations such as GraalPy. Its *use* as a singleton is discouraged and
// should not be relied on for type narrowing, so we do not treat it as one.
// See:
// https://docs.python.org/3/reference/expressions.html#parenthesized-forms
false
}
Type::Union(..) => {
// A single-element union, where the sole element was a singleton, would itself
@@ -501,13 +503,12 @@ impl<'db> Type<'db> {
false
}
Type::Intersection(..) => {
// Intersection types are hard to analyze. The following types are technically
// all singleton types, but it is not straightforward to compute this. Again,
// we simply return false.
// Here, we assume that all intersection types that are singletons would have
// been reduced to a different form via [`IntersectionBuilder::build`] by now.
// For example:
//
// bool & ~Literal[False]`
// None & (None | int)
// (A | B) & (B | C) with A, B, C disjunct and B a singleton
// bool & ~Literal[False] = Literal[True]
// None & (None | int) = None | None & int = None
//
false
}
@@ -1428,7 +1429,8 @@ impl<'db> ClassType<'db> {
pub fn class_member(self, db: &'db dyn Db, name: &str) -> Type<'db> {
let member = self.own_class_member(db, name);
if !member.is_unbound() {
return member;
// TODO diagnostic if maybe unbound?
return member.replace_unbound_with(db, Type::Never);
}
self.inherited_class_member(db, name)
@@ -1627,6 +1629,7 @@ mod tests {
#[test_case(Ty::BytesLiteral("foo"), Ty::BuiltinInstance("bytes"))]
#[test_case(Ty::IntLiteral(1), Ty::Union(vec![Ty::BuiltinInstance("int"), Ty::BuiltinInstance("str")]))]
#[test_case(Ty::IntLiteral(1), Ty::Union(vec![Ty::Unknown, Ty::BuiltinInstance("str")]))]
#[test_case(Ty::Union(vec![Ty::IntLiteral(1), Ty::IntLiteral(2)]), Ty::Union(vec![Ty::IntLiteral(1), Ty::IntLiteral(2)]))]
fn is_assignable_to(from: Ty, to: Ty) {
let db = setup_db();
assert!(from.into_type(&db).is_assignable_to(&db, to.into_type(&db)));
@@ -1682,23 +1685,23 @@ mod tests {
#[test_case(Ty::None)]
#[test_case(Ty::BoolLiteral(true))]
#[test_case(Ty::BoolLiteral(false))]
#[test_case(Ty::Tuple(vec![]))]
fn is_singleton(from: Ty) {
let db = setup_db();
assert!(from.into_type(&db).is_singleton(&db));
assert!(from.into_type(&db).is_singleton());
}
#[test_case(Ty::Never)]
#[test_case(Ty::IntLiteral(345))]
#[test_case(Ty::BuiltinInstance("str"))]
#[test_case(Ty::Union(vec![Ty::IntLiteral(1), Ty::IntLiteral(2)]))]
#[test_case(Ty::Tuple(vec![]))]
#[test_case(Ty::Tuple(vec![Ty::None]))]
#[test_case(Ty::Tuple(vec![Ty::None, Ty::BoolLiteral(true)]))]
fn is_not_singleton(from: Ty) {
let db = setup_db();
assert!(!from.into_type(&db).is_singleton(&db));
assert!(!from.into_type(&db).is_singleton());
}
#[test_case(Ty::IntLiteral(1); "is_int_literal_truthy")]

View File

@@ -84,7 +84,7 @@ fn infer_definition_types_cycle_recovery<'db>(
input: Definition<'db>,
) -> TypeInference<'db> {
tracing::trace!("infer_definition_types_cycle_recovery");
let mut inference = TypeInference::default();
let mut inference = TypeInference::empty(input.scope(db));
let category = input.category(db);
if category.is_declaration() {
inference.declarations.insert(input, Type::Unknown);
@@ -172,7 +172,7 @@ pub(crate) enum InferenceRegion<'db> {
}
/// The inferred types for a single region.
#[derive(Debug, Eq, PartialEq, Default)]
#[derive(Debug, Eq, PartialEq)]
pub(crate) struct TypeInference<'db> {
/// The types of every expression in this region.
expressions: FxHashMap<ScopedExpressionId, Type<'db>>,
@@ -188,9 +188,23 @@ pub(crate) struct TypeInference<'db> {
/// Are there deferred type expressions in this region?
has_deferred: bool,
/// The scope belong to this region.
scope: ScopeId<'db>,
}
impl<'db> TypeInference<'db> {
pub(crate) fn empty(scope: ScopeId<'db>) -> Self {
Self {
expressions: FxHashMap::default(),
bindings: FxHashMap::default(),
declarations: FxHashMap::default(),
diagnostics: TypeCheckDiagnostics::default(),
has_deferred: false,
scope,
}
}
pub(crate) fn expression_ty(&self, expression: ScopedExpressionId) -> Type<'db> {
self.expressions[&expression]
}
@@ -272,7 +286,6 @@ pub(super) struct TypeInferenceBuilder<'db> {
// Cached lookups
file: File,
scope: ScopeId<'db>,
/// The type inference results
types: TypeInference<'db>,
@@ -305,13 +318,14 @@ impl<'db> TypeInferenceBuilder<'db> {
region,
file,
scope,
types: TypeInference::default(),
types: TypeInference::empty(scope),
}
}
fn extend(&mut self, inference: &TypeInference<'db>) {
debug_assert_eq!(self.types.scope, inference.scope);
self.types.bindings.extend(inference.bindings.iter());
self.types
.declarations
@@ -321,6 +335,10 @@ impl<'db> TypeInferenceBuilder<'db> {
self.types.has_deferred |= inference.has_deferred;
}
fn scope(&self) -> ScopeId<'db> {
self.types.scope
}
/// Are we currently inferring types in file with deferred types?
/// This is true for stub files and files with `__future__.annotations`
fn are_all_types_deferred(&self) -> bool {
@@ -337,7 +355,7 @@ impl<'db> TypeInferenceBuilder<'db> {
/// PANIC if no type has been inferred for this node.
fn expression_ty(&self, expr: &ast::Expr) -> Type<'db> {
self.types
.expression_ty(expr.scoped_ast_id(self.db, self.scope))
.expression_ty(expr.scoped_ast_id(self.db, self.scope()))
}
/// Infers types in the given [`InferenceRegion`].
@@ -799,8 +817,6 @@ impl<'db> TypeInferenceBuilder<'db> {
} = parameter_with_default;
self.infer_optional_expression(parameter.annotation.as_deref());
self.infer_definition(parameter_with_default);
}
fn infer_parameter(&mut self, parameter: &ast::Parameter) {
@@ -811,8 +827,6 @@ impl<'db> TypeInferenceBuilder<'db> {
} = parameter;
self.infer_optional_expression(annotation.as_deref());
self.infer_definition(parameter);
}
fn infer_parameter_with_default_definition(
@@ -1008,7 +1022,7 @@ impl<'db> TypeInferenceBuilder<'db> {
self.types
.expressions
.insert(target.scoped_ast_id(self.db, self.scope), context_expr_ty);
.insert(target.scoped_ast_id(self.db, self.scope()), context_expr_ty);
self.add_binding(target.into(), definition, context_expr_ty);
}
@@ -1199,7 +1213,7 @@ impl<'db> TypeInferenceBuilder<'db> {
self.add_binding(name.into(), definition, target_ty);
self.types
.expressions
.insert(name.scoped_ast_id(self.db, self.scope), target_ty);
.insert(name.scoped_ast_id(self.db, self.scope()), target_ty);
}
fn infer_sequence_unpacking(
@@ -1513,9 +1527,10 @@ impl<'db> TypeInferenceBuilder<'db> {
.unwrap_with_diagnostic(iterable.into(), self)
};
self.types
.expressions
.insert(target.scoped_ast_id(self.db, self.scope), loop_var_value_ty);
self.types.expressions.insert(
target.scoped_ast_id(self.db, self.scope()),
loop_var_value_ty,
);
self.add_binding(target.into(), definition, loop_var_value_ty);
}
@@ -1684,10 +1699,30 @@ impl<'db> TypeInferenceBuilder<'db> {
.ok_or(ModuleNameResolutionError::InvalidSyntax)
};
let module_ty = match module_name {
Ok(name) => {
if let Some(ty) = self.module_ty_from_name(&name) {
ty
let ty = match module_name {
Ok(module_name) => {
if let Some(module_ty) = self.module_ty_from_name(&module_name) {
let ast::Alias {
range: _,
name,
asname: _,
} = alias;
let member_ty = module_ty.member(self.db, &ast::name::Name::new(&name.id));
if member_ty.is_unbound() {
self.add_diagnostic(
AnyNodeRef::Alias(alias),
"unresolved-import",
format_args!("Module `{module_name}` has no member `{name}`",),
);
Type::Unknown
} else {
// For possibly-unbound names, just eliminate Unbound from the type; we
// must be in a bound path. TODO diagnostic for maybe-unbound import?
member_ty.replace_unbound_with(self.db, Type::Never)
}
} else {
self.unresolved_module_diagnostic(import_from, *level, module);
Type::Unknown
@@ -1717,34 +1752,6 @@ impl<'db> TypeInferenceBuilder<'db> {
}
};
let ast::Alias {
range: _,
name,
asname: _,
} = alias;
let member_ty = module_ty.member(self.db, &ast::name::Name::new(&name.id));
// TODO: What if it's a union where one of the elements is `Unbound`?
if member_ty.is_unbound() {
self.add_diagnostic(
AnyNodeRef::Alias(alias),
"unresolved-import",
format_args!(
"Module `{}{}` has no member `{name}`",
".".repeat(*level as usize),
module.unwrap_or_default()
),
);
}
// If a symbol is unbound in the module the symbol was originally defined in,
// when we're trying to import the symbol from that module into "our" module,
// the runtime error will occur immediately (rather than when the symbol is *used*,
// as would be the case for a symbol with type `Unbound`), so it's appropriate to
// think of the type of the imported symbol as `Unknown` rather than `Unbound`
let ty = member_ty.replace_unbound_with(self.db, Type::Unknown);
self.add_declaration_with_binding(alias.into(), definition, ty, ty);
}
@@ -1842,7 +1849,7 @@ impl<'db> TypeInferenceBuilder<'db> {
ast::Expr::IpyEscapeCommand(_) => todo!("Implement Ipy escape command support"),
};
let expr_id = expression.scoped_ast_id(self.db, self.scope);
let expr_id = expression.scoped_ast_id(self.db, self.scope());
let previous = self.types.expressions.insert(expr_id, ty);
assert_eq!(previous, None);
@@ -2161,13 +2168,13 @@ impl<'db> TypeInferenceBuilder<'db> {
let iterable_ty = if is_first {
let lookup_scope = self
.index
.parent_scope_id(self.scope.file_scope_id(self.db))
.parent_scope_id(self.scope().file_scope_id(self.db))
.expect("A comprehension should never be the top-level scope")
.to_scope_id(self.db, self.file);
result.expression_ty(iterable.scoped_ast_id(self.db, lookup_scope))
} else {
self.extend(result);
result.expression_ty(iterable.scoped_ast_id(self.db, self.scope))
result.expression_ty(iterable.scoped_ast_id(self.db, self.scope()))
};
let target_ty = if is_async {
@@ -2181,7 +2188,7 @@ impl<'db> TypeInferenceBuilder<'db> {
self.types
.expressions
.insert(target.scoped_ast_id(self.db, self.scope), target_ty);
.insert(target.scoped_ast_id(self.db, self.scope()), target_ty);
self.add_binding(target.into(), definition, target_ty);
}
@@ -2318,7 +2325,7 @@ impl<'db> TypeInferenceBuilder<'db> {
/// Look up a name reference that isn't bound in the local scope.
fn lookup_name(&mut self, name_node: &ast::ExprName) -> Type<'db> {
let ast::ExprName { id: name, .. } = name_node;
let file_scope_id = self.scope.file_scope_id(self.db);
let file_scope_id = self.scope().file_scope_id(self.db);
let is_bound = self
.index
.symbol_table(file_scope_id)
@@ -2329,7 +2336,7 @@ impl<'db> TypeInferenceBuilder<'db> {
// In function-like scopes, any local variable (symbol that is bound in this scope) can
// only have a definition in this scope, or error; it never references another scope.
// (At runtime, it would use the `LOAD_FAST` opcode.)
if !is_bound || !self.scope.is_function_like(self.db) {
if !is_bound || !self.scope().is_function_like(self.db) {
// Walk up parent scopes looking for a possible enclosing scope that may have a
// definition of this name visible to us (would be `LOAD_DEREF` at runtime.)
for (enclosing_scope_file_id, _) in self.index.ancestor_scopes(file_scope_id) {
@@ -2353,6 +2360,7 @@ impl<'db> TypeInferenceBuilder<'db> {
return symbol_ty(self.db, enclosing_scope_id, name);
}
}
// No nonlocal binding, check module globals. Avoid infinite recursion if `self.scope`
// already is module globals.
let ty = if file_scope_id.is_global() {
@@ -2360,8 +2368,9 @@ impl<'db> TypeInferenceBuilder<'db> {
} else {
global_symbol_ty(self.db, self.file, name)
};
// Fallback to builtins (without infinite recursion if we're already in builtins.)
if ty.may_be_unbound(self.db) && Some(self.scope) != builtins_module_scope(self.db) {
if ty.may_be_unbound(self.db) && Some(self.scope()) != builtins_module_scope(self.db) {
let mut builtin_ty = builtins_symbol_ty(self.db, name);
if builtin_ty.is_unbound() && name == "reveal_type" {
self.add_diagnostic(
@@ -2383,7 +2392,7 @@ impl<'db> TypeInferenceBuilder<'db> {
fn infer_name_expression(&mut self, name: &ast::ExprName) -> Type<'db> {
let ast::ExprName { range: _, id, ctx } = name;
let file_scope_id = self.scope.file_scope_id(self.db);
let file_scope_id = self.scope().file_scope_id(self.db);
match ctx {
ExprContext::Load => {
@@ -2400,7 +2409,7 @@ impl<'db> TypeInferenceBuilder<'db> {
use_def.public_may_be_unbound(symbol),
)
} else {
let use_id = name.scoped_use_id(self.db, self.scope);
let use_id = name.scoped_use_id(self.db, self.scope());
(
use_def.bindings_at_use(use_id),
use_def.use_may_be_unbound(use_id),
@@ -2831,7 +2840,68 @@ impl<'db> TypeInferenceBuilder<'db> {
(_, Type::BytesLiteral(_)) => {
self.infer_binary_type_comparison(left, op, KnownClass::Bytes.to_instance(self.db))
}
(Type::Tuple(lhs), Type::Tuple(rhs)) => {
// Note: This only works on heterogeneous tuple types.
let lhs_elements = lhs.elements(self.db).as_ref();
let rhs_elements = rhs.elements(self.db).as_ref();
let mut lexicographic_type_comparison =
|op| self.infer_lexicographic_type_comparison(lhs_elements, op, rhs_elements);
match op {
ast::CmpOp::Eq => lexicographic_type_comparison(RichCompareOperator::Eq),
ast::CmpOp::NotEq => lexicographic_type_comparison(RichCompareOperator::Ne),
ast::CmpOp::Lt => lexicographic_type_comparison(RichCompareOperator::Lt),
ast::CmpOp::LtE => lexicographic_type_comparison(RichCompareOperator::Le),
ast::CmpOp::Gt => lexicographic_type_comparison(RichCompareOperator::Gt),
ast::CmpOp::GtE => lexicographic_type_comparison(RichCompareOperator::Ge),
ast::CmpOp::In | ast::CmpOp::NotIn => {
let mut eq_count = 0usize;
let mut not_eq_count = 0usize;
for ty in rhs_elements {
let eq_result = self.infer_binary_type_comparison(
Type::Tuple(lhs),
ast::CmpOp::Eq,
*ty,
).expect("infer_binary_type_comparison should never return None for `CmpOp::Eq`");
match eq_result {
Type::Todo => return Some(Type::Todo),
ty => match ty.bool(self.db) {
Truthiness::AlwaysTrue => eq_count += 1,
Truthiness::AlwaysFalse => not_eq_count += 1,
Truthiness::Ambiguous => (),
},
}
}
if eq_count >= 1 {
Some(Type::BooleanLiteral(op.is_in()))
} else if not_eq_count == rhs_elements.len() {
Some(Type::BooleanLiteral(op.is_not_in()))
} else {
Some(KnownClass::Bool.to_instance(self.db))
}
}
ast::CmpOp::Is | ast::CmpOp::IsNot => {
// - `[ast::CmpOp::Is]`: returns `false` if the elements are definitely unequal, otherwise `bool`
// - `[ast::CmpOp::IsNot]`: returns `true` if the elements are definitely unequal, otherwise `bool`
let eq_result = lexicographic_type_comparison(RichCompareOperator::Eq)
.expect(
"infer_binary_type_comparison should never return None for `CmpOp::Eq`",
);
Some(match eq_result {
Type::Todo => Type::Todo,
ty => match ty.bool(self.db) {
Truthiness::AlwaysFalse => Type::BooleanLiteral(op.is_is_not()),
_ => KnownClass::Bool.to_instance(self.db),
},
})
}
}
}
// Lookup the rich comparison `__dunder__` methods on instances
(Type::Instance(left_class_ty), Type::Instance(right_class_ty)) => match op {
ast::CmpOp::Lt => {
@@ -2845,6 +2915,55 @@ impl<'db> TypeInferenceBuilder<'db> {
}
}
/// Performs lexicographic comparison between two slices of types.
///
/// For lexicographic comparison, elements from both slices are compared pairwise using
/// `infer_binary_type_comparison`. If a conclusive result cannot be determined as a `BoolLiteral`,
/// it returns `bool`. Returns `None` if the comparison is not supported.
fn infer_lexicographic_type_comparison(
&mut self,
left: &[Type<'db>],
op: RichCompareOperator,
right: &[Type<'db>],
) -> Option<Type<'db>> {
// Compare paired elements from left and right slices
for (l_ty, r_ty) in left.iter().copied().zip(right.iter().copied()) {
let eq_result = self
.infer_binary_type_comparison(l_ty, ast::CmpOp::Eq, r_ty)
.expect("infer_binary_type_comparison should never return None for `CmpOp::Eq`");
match eq_result {
// If propagation is required, return the result as is
Type::Todo => return Some(Type::Todo),
ty => match ty.bool(self.db) {
// Types are equal, continue to the next pair
Truthiness::AlwaysTrue => continue,
// Types are not equal, perform the specified comparison and return the result
Truthiness::AlwaysFalse => {
return self.infer_binary_type_comparison(l_ty, op.into(), r_ty)
}
// If the intermediate result is ambiguous, we cannot determine the final result as BooleanLiteral.
// In this case, we simply return a bool instance.
Truthiness::Ambiguous => return Some(KnownClass::Bool.to_instance(self.db)),
},
}
}
// At this point, the lengths of the two slices may be different, but the prefix of
// left and right slices is entirely identical.
// We return a comparison of the slice lengths based on the operator.
let (left_len, right_len) = (left.len(), right.len());
Some(Type::BooleanLiteral(match op {
RichCompareOperator::Eq => left_len == right_len,
RichCompareOperator::Ne => left_len != right_len,
RichCompareOperator::Lt => left_len < right_len,
RichCompareOperator::Le => left_len <= right_len,
RichCompareOperator::Gt => left_len > right_len,
RichCompareOperator::Ge => left_len >= right_len,
}))
}
fn infer_subscript_expression(&mut self, subscript: &ast::ExprSubscript) -> Type<'db> {
let ast::ExprSubscript {
range: _,
@@ -2855,7 +2974,15 @@ impl<'db> TypeInferenceBuilder<'db> {
let value_ty = self.infer_expression(value);
let slice_ty = self.infer_expression(slice);
self.infer_subscript_expression_types(value, value_ty, slice_ty)
}
fn infer_subscript_expression_types(
&mut self,
value_node: &ast::Expr,
value_ty: Type<'db>,
slice_ty: Type<'db>,
) -> Type<'db> {
match (value_ty, slice_ty) {
// Ex) Given `("a", "b", "c", "d")[1]`, return `"b"`
(Type::Tuple(tuple_ty), Type::IntLiteral(int)) if int >= 0 => {
@@ -2865,7 +2992,7 @@ impl<'db> TypeInferenceBuilder<'db> {
.and_then(|index| elements.get(index).copied())
.unwrap_or_else(|| {
self.tuple_index_out_of_bounds_diagnostic(
(&**value).into(),
value_node.into(),
value_ty,
elements.len(),
int,
@@ -2882,7 +3009,7 @@ impl<'db> TypeInferenceBuilder<'db> {
.and_then(|index| elements.get(index).copied())
.unwrap_or_else(|| {
self.tuple_index_out_of_bounds_diagnostic(
(&**value).into(),
value_node.into(),
value_ty,
elements.len(),
int,
@@ -2891,19 +3018,11 @@ impl<'db> TypeInferenceBuilder<'db> {
})
}
// Ex) Given `("a", "b", "c", "d")[True]`, return `"b"`
(Type::Tuple(tuple_ty), Type::BooleanLiteral(bool)) => {
let elements = tuple_ty.elements(self.db);
let int = i64::from(bool);
elements.get(usize::from(bool)).copied().unwrap_or_else(|| {
self.tuple_index_out_of_bounds_diagnostic(
(&**value).into(),
value_ty,
elements.len(),
int,
);
Type::Unknown
})
}
(Type::Tuple(_), Type::BooleanLiteral(bool)) => self.infer_subscript_expression_types(
value_node,
value_ty,
Type::IntLiteral(i64::from(bool)),
),
// Ex) Given `"value"[1]`, return `"a"`
(Type::StringLiteral(literal_ty), Type::IntLiteral(int)) if int >= 0 => {
let literal_value = literal_ty.value(self.db);
@@ -2918,7 +3037,7 @@ impl<'db> TypeInferenceBuilder<'db> {
})
.unwrap_or_else(|| {
self.string_index_out_of_bounds_diagnostic(
(&**value).into(),
value_node.into(),
value_ty,
literal_value.chars().count(),
int,
@@ -2941,7 +3060,7 @@ impl<'db> TypeInferenceBuilder<'db> {
})
.unwrap_or_else(|| {
self.string_index_out_of_bounds_diagnostic(
(&**value).into(),
value_node.into(),
value_ty,
literal_value.chars().count(),
int,
@@ -2950,28 +3069,12 @@ impl<'db> TypeInferenceBuilder<'db> {
})
}
// Ex) Given `"value"[True]`, return `"a"`
(Type::StringLiteral(literal_ty), Type::BooleanLiteral(bool)) => {
let literal_value = literal_ty.value(self.db);
let int = i64::from(bool);
literal_value
.chars()
.nth(usize::from(bool))
.map(|ch| {
Type::StringLiteral(StringLiteralType::new(
self.db,
ch.to_string().into_boxed_str(),
))
})
.unwrap_or_else(|| {
self.string_index_out_of_bounds_diagnostic(
(&**value).into(),
value_ty,
literal_value.chars().count(),
int,
);
Type::Unknown
})
}
(Type::StringLiteral(_), Type::BooleanLiteral(bool)) => self
.infer_subscript_expression_types(
value_node,
value_ty,
Type::IntLiteral(i64::from(bool)),
),
(value_ty, slice_ty) => {
// Resolve the value to its class.
let value_meta_ty = value_ty.to_meta_type(self.db);
@@ -2983,10 +3086,10 @@ impl<'db> TypeInferenceBuilder<'db> {
if !dunder_getitem_method.is_unbound() {
return dunder_getitem_method
.call(self.db, &[slice_ty])
.return_ty_result(self.db, value.as_ref().into(), self)
.return_ty_result(self.db, value_node.into(), self)
.unwrap_or_else(|err| {
self.add_diagnostic(
(&**value).into(),
value_node.into(),
"call-non-callable",
format_args!(
"Method `__getitem__` of type `{}` is not callable on object of type `{}`",
@@ -3012,10 +3115,10 @@ impl<'db> TypeInferenceBuilder<'db> {
if !dunder_class_getitem_method.is_unbound() {
return dunder_class_getitem_method
.call(self.db, &[slice_ty])
.return_ty_result(self.db, value.as_ref().into(), self)
.return_ty_result(self.db, value_node.into(), self)
.unwrap_or_else(|err| {
self.add_diagnostic(
(&**value).into(),
value_node.into(),
"call-non-callable",
format_args!(
"Method `__class_getitem__` of type `{}` is not callable on object of type `{}`",
@@ -3033,12 +3136,12 @@ impl<'db> TypeInferenceBuilder<'db> {
}
self.non_subscriptable_diagnostic(
(&**value).into(),
value_node.into(),
value_ty,
"__class_getitem__",
);
} else {
self.non_subscriptable_diagnostic((&**value).into(), value_ty, "__getitem__");
self.non_subscriptable_diagnostic(value_node.into(), value_ty, "__getitem__");
}
Type::Unknown
@@ -3294,7 +3397,7 @@ impl<'db> TypeInferenceBuilder<'db> {
ast::Expr::IpyEscapeCommand(_) => todo!("Implement Ipy escape command support"),
};
let expr_id = expression.scoped_ast_id(self.db, self.scope);
let expr_id = expression.scoped_ast_id(self.db, self.scope());
let previous = self.types.expressions.insert(expr_id, ty);
assert!(previous.is_none());
@@ -3302,6 +3405,29 @@ impl<'db> TypeInferenceBuilder<'db> {
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum RichCompareOperator {
Eq,
Ne,
Gt,
Ge,
Lt,
Le,
}
impl From<RichCompareOperator> for ast::CmpOp {
fn from(value: RichCompareOperator) -> Self {
match value {
RichCompareOperator::Eq => ast::CmpOp::Eq,
RichCompareOperator::Ne => ast::CmpOp::NotEq,
RichCompareOperator::Lt => ast::CmpOp::Lt,
RichCompareOperator::Le => ast::CmpOp::LtE,
RichCompareOperator::Gt => ast::CmpOp::Gt,
RichCompareOperator::Ge => ast::CmpOp::GtE,
}
}
}
fn format_import_from_module(level: u32, module: Option<&str>) -> String {
format!(
"{}{}",
@@ -3745,7 +3871,7 @@ mod tests {
)?;
// TODO: sys.version_info, and need to understand @final and @type_check_only
assert_public_ty(&db, "src/a.py", "x", "Unknown | EllipsisType");
assert_public_ty(&db, "src/a.py", "x", "EllipsisType | Unknown");
Ok(())
}
@@ -3862,38 +3988,6 @@ mod tests {
Ok(())
}
#[test]
fn conditionally_global_or_builtin() -> anyhow::Result<()> {
let mut db = setup_db();
db.write_dedented(
"/src/a.py",
"
if flag:
copyright = 1
def f():
y = copyright
",
)?;
let file = system_path_to_file(&db, "src/a.py").expect("file to exist");
let index = semantic_index(&db, file);
let function_scope = index
.child_scopes(FileScopeId::global())
.next()
.unwrap()
.0
.to_scope_id(&db, file);
let y_ty = symbol_ty(&db, function_scope, "y");
assert_eq!(
y_ty.display(&db).to_string(),
"Literal[copyright] | Literal[1]"
);
Ok(())
}
#[test]
fn local_inference() -> anyhow::Result<()> {
let mut db = setup_db();

View File

@@ -157,7 +157,7 @@ impl<'db> NarrowingConstraintsBuilder<'db> {
let comp_ty = inference.expression_ty(comparator.scoped_ast_id(self.db, scope));
match op {
ast::CmpOp::IsNot => {
if comp_ty.is_singleton(self.db) {
if comp_ty.is_singleton() {
let ty = IntersectionBuilder::new(self.db)
.add_negative(comp_ty)
.build();

View File

@@ -417,6 +417,10 @@ impl Printer {
statistic.name,
)?;
}
if any_fixable {
writeln!(writer, "[*] fixable with `ruff check --fix`",)?;
}
return Ok(());
}
OutputFormat::Json => {

View File

@@ -979,6 +979,7 @@ fn show_statistics() {
exit_code: 1
----- stdout -----
1 F401 [*] unused-import
[*] fixable with `ruff check --fix`
----- stderr -----
"###);

View File

@@ -22,10 +22,15 @@ struct Case {
const TOMLLIB_312_URL: &str = "https://raw.githubusercontent.com/python/cpython/8e8a4baf652f6e1cee7acde9d78c4b6154539748/Lib/tomllib";
// The failed import from 'collections.abc' is due to lack of support for 'import *'.
static EXPECTED_DIAGNOSTICS: &[&str] = &[
// We don't support `ModuleType`-attributes as globals yet:
"/src/tomllib/__init__.py:10:30: Name `__name__` used when not defined",
// We don't support `*` imports yet:
"/src/tomllib/_parser.py:7:29: Module `collections.abc` has no member `Iterable`",
// We don't support terminal statements in control flow yet:
"/src/tomllib/_parser.py:353:5: Method `__getitem__` of type `Unbound | @Todo` is not callable on object of type `Unbound | @Todo`",
"/src/tomllib/_parser.py:455:9: Method `__getitem__` of type `Unbound | @Todo` is not callable on object of type `Unbound | @Todo`",
// True positives!
"Line 69 is too long (89 characters)",
"Use double quotes for strings",
"Use double quotes for strings",
@@ -34,6 +39,25 @@ static EXPECTED_DIAGNOSTICS: &[&str] = &[
"Use double quotes for strings",
"Use double quotes for strings",
"Use double quotes for strings",
// We don't support terminal statements in control flow yet:
"/src/tomllib/_parser.py:66:18: Name `s` used when possibly not defined",
"/src/tomllib/_parser.py:98:12: Name `char` used when possibly not defined",
"/src/tomllib/_parser.py:101:12: Name `char` used when possibly not defined",
"/src/tomllib/_parser.py:104:14: Name `char` used when possibly not defined",
"/src/tomllib/_parser.py:104:14: Name `char` used when possibly not defined",
"/src/tomllib/_parser.py:115:14: Name `char` used when possibly not defined",
"/src/tomllib/_parser.py:115:14: Name `char` used when possibly not defined",
"/src/tomllib/_parser.py:126:12: Name `char` used when possibly not defined",
"/src/tomllib/_parser.py:348:20: Name `nest` used when possibly not defined",
"/src/tomllib/_parser.py:353:5: Name `nest` used when possibly not defined",
"/src/tomllib/_parser.py:453:24: Name `nest` used when possibly not defined",
"/src/tomllib/_parser.py:455:9: Name `nest` used when possibly not defined",
"/src/tomllib/_parser.py:482:16: Name `char` used when possibly not defined",
"/src/tomllib/_parser.py:566:12: Name `char` used when possibly not defined",
"/src/tomllib/_parser.py:573:12: Name `char` used when possibly not defined",
"/src/tomllib/_parser.py:579:12: Name `char` used when possibly not defined",
"/src/tomllib/_parser.py:580:63: Name `char` used when possibly not defined",
"/src/tomllib/_parser.py:629:38: Name `datetime_obj` used when possibly not defined"
];
fn get_test_file(name: &str) -> TestFile {

View File

@@ -95,18 +95,62 @@ cursor.execute('SELECT * FROM table WHERE id = 1')
cursor.executemany('SELECT * FROM table WHERE id = %s', [var, var2])
# # INSERT without INTO (e.g. MySQL and derivatives)
query = "INSERT table VALUES (%s)" % (var,)
query46 = "INSERT table VALUES (%s)" % (var,)
# # REPLACE (e.g. MySQL and derivatives, SQLite)
query = "REPLACE INTO table VALUES (%s)" % (var,)
query = "REPLACE table VALUES (%s)" % (var,)
query47 = "REPLACE INTO table VALUES (%s)" % (var,)
query48 = "REPLACE table VALUES (%s)" % (var,)
query = "Deselect something that is not SQL even though it has a ' from ' somewhere in %s." % "there"
query49 = "Deselect something that is not SQL even though it has a ' from ' somewhere in %s." % "there"
# # pass
["select colA from tableA"] + ["select colB from tableB"]
"SELECT * FROM " + (["table1"] if x > 0 else ["table2"])
# # errors
"SELECT * FROM " + ("table1" if x > 0 else "table2")
"SELECT * FROM " + ("table1" if x > 0 else ["table2"])
"SELECT * FROM " + ("table1" if x > 0 else "table2") # query50
"SELECT * FROM " + ("table1" if x > 0 else ["table2"]) # query51
# test cases from #12044
def query52():
return f"""
SELECT {var}
FROM bar
"""
def query53():
return f"""
SELECT
{var}
FROM bar
"""
def query54():
return f"""
SELECT {var}
FROM
bar
"""
query55 = f"""SELECT * FROM
{var}.table
"""
query56 = f"""SELECT *
FROM {var}.table
"""
query57 = f"""
SELECT *
FROM {var}.table
"""
query57 = f"""
PRESELECT *
FROM {var}.table
"""
# to be handled seperately
# query58 = f"SELECT\
# * FROM {var}.table"

View File

@@ -1,18 +1,17 @@
use once_cell::sync::Lazy;
use regex::Regex;
use ruff_python_ast::{self as ast, Expr, Operator};
use ruff_diagnostics::{Diagnostic, Violation};
use ruff_macros::{derive_message_formats, violation};
use ruff_python_ast::str::raw_contents;
use ruff_python_ast::{self as ast, Expr, Operator};
use ruff_source_file::Locator;
use ruff_text_size::Ranged;
use crate::checkers::ast::Checker;
static SQL_REGEX: Lazy<Regex> = Lazy::new(|| {
Regex::new(r"(?i)\b(select\s.+\sfrom\s|delete\s+from\s|(insert|replace)\s.+\svalues\s|update\s.+\sset\s)")
.unwrap()
Regex::new(r"(?i)\b(select\s+.*\s+from\s|delete\s+from\s|(insert|replace)\s+.*\s+values\s|update\s+.*\s+set\s)")
.unwrap()
});
/// ## What it does
@@ -88,6 +87,7 @@ pub(crate) fn hardcoded_sql_expression(checker: &mut Checker, expr: &Expr) {
};
string.value.to_str().escape_default().to_string()
}
// f"select * from table where val = {val}"
Expr::FString(f_string) => concatenated_f_string(f_string, checker.locator()),
_ => return,
@@ -113,9 +113,7 @@ pub(crate) fn hardcoded_sql_expression(checker: &mut Checker, expr: &Expr) {
fn concatenated_f_string(expr: &ast::ExprFString, locator: &Locator) -> String {
expr.value
.iter()
.filter_map(|part| {
raw_contents(locator.slice(part)).map(|s| s.escape_default().to_string())
})
.filter_map(|part| raw_contents(locator.slice(part)))
.collect()
}

View File

@@ -452,45 +452,128 @@ S608.py:86:30: S608 Possible SQL injection vector through string-based query con
88 | # # pass
|
S608.py:98:9: S608 Possible SQL injection vector through string-based query construction
S608.py:98:11: S608 Possible SQL injection vector through string-based query construction
|
97 | # # INSERT without INTO (e.g. MySQL and derivatives)
98 | query = "INSERT table VALUES (%s)" % (var,)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ S608
98 | query46 = "INSERT table VALUES (%s)" % (var,)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ S608
99 |
100 | # # REPLACE (e.g. MySQL and derivatives, SQLite)
|
S608.py:101:9: S608 Possible SQL injection vector through string-based query construction
S608.py:101:11: S608 Possible SQL injection vector through string-based query construction
|
100 | # # REPLACE (e.g. MySQL and derivatives, SQLite)
101 | query = "REPLACE INTO table VALUES (%s)" % (var,)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ S608
102 | query = "REPLACE table VALUES (%s)" % (var,)
101 | query47 = "REPLACE INTO table VALUES (%s)" % (var,)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ S608
102 | query48 = "REPLACE table VALUES (%s)" % (var,)
|
S608.py:102:9: S608 Possible SQL injection vector through string-based query construction
S608.py:102:11: S608 Possible SQL injection vector through string-based query construction
|
100 | # # REPLACE (e.g. MySQL and derivatives, SQLite)
101 | query = "REPLACE INTO table VALUES (%s)" % (var,)
102 | query = "REPLACE table VALUES (%s)" % (var,)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ S608
101 | query47 = "REPLACE INTO table VALUES (%s)" % (var,)
102 | query48 = "REPLACE table VALUES (%s)" % (var,)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ S608
103 |
104 | query = "Deselect something that is not SQL even though it has a ' from ' somewhere in %s." % "there"
104 | query49 = "Deselect something that is not SQL even though it has a ' from ' somewhere in %s." % "there"
|
S608.py:111:1: S608 Possible SQL injection vector through string-based query construction
|
110 | # # errors
111 | "SELECT * FROM " + ("table1" if x > 0 else "table2")
111 | "SELECT * FROM " + ("table1" if x > 0 else "table2") # query50
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ S608
112 | "SELECT * FROM " + ("table1" if x > 0 else ["table2"])
112 | "SELECT * FROM " + ("table1" if x > 0 else ["table2"]) # query51
|
S608.py:112:1: S608 Possible SQL injection vector through string-based query construction
|
110 | # # errors
111 | "SELECT * FROM " + ("table1" if x > 0 else "table2")
112 | "SELECT * FROM " + ("table1" if x > 0 else ["table2"])
111 | "SELECT * FROM " + ("table1" if x > 0 else "table2") # query50
112 | "SELECT * FROM " + ("table1" if x > 0 else ["table2"]) # query51
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ S608
113 |
114 | # test cases from #12044
|
S608.py:117:12: S608 Possible SQL injection vector through string-based query construction
|
116 | def query52():
117 | return f"""
| ____________^
118 | | SELECT {var}
119 | | FROM bar
120 | | """
| |_______^ S608
121 |
122 | def query53():
|
S608.py:123:12: S608 Possible SQL injection vector through string-based query construction
|
122 | def query53():
123 | return f"""
| ____________^
124 | | SELECT
125 | | {var}
126 | | FROM bar
127 | | """
| |_______^ S608
128 |
129 | def query54():
|
S608.py:130:12: S608 Possible SQL injection vector through string-based query construction
|
129 | def query54():
130 | return f"""
| ____________^
131 | | SELECT {var}
132 | | FROM
133 | | bar
134 | | """
| |_______^ S608
135 |
136 | query55 = f"""SELECT * FROM
|
S608.py:136:11: S608 Possible SQL injection vector through string-based query construction
|
134 | """
135 |
136 | query55 = f"""SELECT * FROM
| ___________^
137 | | {var}.table
138 | | """
| |___^ S608
139 |
140 | query56 = f"""SELECT *
|
S608.py:140:11: S608 Possible SQL injection vector through string-based query construction
|
138 | """
139 |
140 | query56 = f"""SELECT *
| ___________^
141 | | FROM {var}.table
142 | | """
| |___^ S608
143 |
144 | query57 = f"""
|
S608.py:144:11: S608 Possible SQL injection vector through string-based query construction
|
142 | """
143 |
144 | query57 = f"""
| ___________^
145 | | SELECT *
146 | | FROM {var}.table
147 | | """
| |___^ S608
148 |
149 | query57 = f"""
|

View File

@@ -2,7 +2,10 @@ use std::iter::FusedIterator;
use ruff_text_size::{Ranged, TextRange};
use crate::{self as ast, AnyNodeRef, AnyStringFlags, Expr};
use crate::{
self as ast, AnyNodeRef, AnyStringFlags, Expr, ExprBytesLiteral, ExprFString,
ExprStringLiteral, StringFlags,
};
/// Unowned pendant to [`ast::Expr`] that stores a reference instead of a owned value.
#[derive(Copy, Clone, Debug, PartialEq)]
@@ -405,6 +408,10 @@ pub enum StringLike<'a> {
}
impl<'a> StringLike<'a> {
pub const fn is_fstring(self) -> bool {
matches!(self, Self::FString(_))
}
/// Returns an iterator over the [`StringLikePart`] contained in this string-like expression.
pub fn parts(&self) -> StringLikePartIter<'_> {
match self {
@@ -413,6 +420,15 @@ impl<'a> StringLike<'a> {
StringLike::FString(expr) => StringLikePartIter::FString(expr.value.iter()),
}
}
/// Returns `true` if the string is implicitly concatenated.
pub fn is_implicit_concatenated(self) -> bool {
match self {
Self::String(ExprStringLiteral { value, .. }) => value.is_implicit_concatenated(),
Self::Bytes(ExprBytesLiteral { value, .. }) => value.is_implicit_concatenated(),
Self::FString(ExprFString { value, .. }) => value.is_implicit_concatenated(),
}
}
}
impl<'a> From<&'a ast::ExprStringLiteral> for StringLike<'a> {
@@ -433,6 +449,45 @@ impl<'a> From<&'a ast::ExprFString> for StringLike<'a> {
}
}
impl<'a> From<&StringLike<'a>> for ExpressionRef<'a> {
fn from(value: &StringLike<'a>) -> Self {
match value {
StringLike::String(expr) => ExpressionRef::StringLiteral(expr),
StringLike::Bytes(expr) => ExpressionRef::BytesLiteral(expr),
StringLike::FString(expr) => ExpressionRef::FString(expr),
}
}
}
impl<'a> From<StringLike<'a>> for AnyNodeRef<'a> {
fn from(value: StringLike<'a>) -> Self {
AnyNodeRef::from(&value)
}
}
impl<'a> From<&StringLike<'a>> for AnyNodeRef<'a> {
fn from(value: &StringLike<'a>) -> Self {
match value {
StringLike::String(expr) => AnyNodeRef::ExprStringLiteral(expr),
StringLike::Bytes(expr) => AnyNodeRef::ExprBytesLiteral(expr),
StringLike::FString(expr) => AnyNodeRef::ExprFString(expr),
}
}
}
impl<'a> TryFrom<&'a Expr> for StringLike<'a> {
type Error = ();
fn try_from(value: &'a Expr) -> Result<Self, Self::Error> {
match value {
Expr::StringLiteral(value) => Ok(Self::String(value)),
Expr::BytesLiteral(value) => Ok(Self::Bytes(value)),
Expr::FString(value) => Ok(Self::FString(value)),
_ => Err(()),
}
}
}
impl Ranged for StringLike<'_> {
fn range(&self) -> TextRange {
match self {
@@ -460,6 +515,15 @@ impl StringLikePart<'_> {
StringLikePart::FString(f_string) => AnyStringFlags::from(f_string.flags),
}
}
/// Returns the range of the string's content in the source (minus prefix and quotes).
pub fn content_range(self) -> TextRange {
let kind = self.flags();
TextRange::new(
self.start() + kind.opener_len(),
self.end() - kind.closer_len(),
)
}
}
impl<'a> From<&'a ast::StringLiteral> for StringLikePart<'a> {
@@ -480,6 +544,16 @@ impl<'a> From<&'a ast::FString> for StringLikePart<'a> {
}
}
impl<'a> From<&StringLikePart<'a>> for AnyNodeRef<'a> {
fn from(value: &StringLikePart<'a>) -> Self {
match value {
StringLikePart::String(part) => AnyNodeRef::StringLiteral(part),
StringLikePart::Bytes(part) => AnyNodeRef::BytesLiteral(part),
StringLikePart::FString(part) => AnyNodeRef::FString(part),
}
}
}
impl Ranged for StringLikePart<'_> {
fn range(&self) -> TextRange {
match self {

View File

@@ -1 +0,0 @@
{"target_version": "py38"}

View File

@@ -1 +0,0 @@
{"target_version": "py39"}

View File

@@ -1 +0,0 @@
{"target_version": "py38"}

View File

@@ -1 +0,0 @@
{"target_version": "py38"}

View File

@@ -1 +0,0 @@
{"target_version": "py39"}

View File

@@ -0,0 +1 @@
{"preview": "enabled"}

View File

@@ -0,0 +1 @@
{"preview": "enabled"}

View File

@@ -0,0 +1 @@
{"preview": "enabled"}

View File

@@ -0,0 +1 @@
{"preview": "enabled"}

View File

@@ -0,0 +1 @@
{"preview": "enabled"}

View File

@@ -0,0 +1 @@
{"preview": "enabled"}

View File

@@ -1 +0,0 @@
{"target_version": "py37"}

View File

@@ -1 +0,0 @@
{"target_version": "py38"}

View File

@@ -1 +0,0 @@
{"target_version": "py39"}

View File

@@ -39,7 +39,7 @@ def import_fixture(fixture: Path, fixture_set: str):
extension = "py"
if flags:
if "--preview" in flags:
if "--preview" in flags or "--unstable" in flags:
options["preview"] = "enabled"
if "--pyi" in flags:

View File

@@ -5,7 +5,7 @@ use smallvec::SmallVec;
use ruff_formatter::write;
use ruff_python_ast::{
Expr, ExprAttribute, ExprBinOp, ExprBoolOp, ExprCompare, ExprUnaryOp, UnaryOp,
Expr, ExprAttribute, ExprBinOp, ExprBoolOp, ExprCompare, ExprUnaryOp, StringLike, UnaryOp,
};
use ruff_python_trivia::CommentRanges;
use ruff_python_trivia::{SimpleToken, SimpleTokenKind, SimpleTokenizer};
@@ -20,7 +20,7 @@ use crate::expression::parentheses::{
};
use crate::expression::OperatorPrecedence;
use crate::prelude::*;
use crate::string::{AnyString, FormatImplicitConcatenatedString};
use crate::string::FormatImplicitConcatenatedString;
#[derive(Copy, Clone, Debug)]
pub(super) enum BinaryLike<'a> {
@@ -293,7 +293,8 @@ impl Format<PyFormatContext<'_>> for BinaryLike<'_> {
let mut string_operands = flat_binary
.operands()
.filter_map(|(index, operand)| {
AnyString::from_expression(operand.expression())
StringLike::try_from(operand.expression())
.ok()
.filter(|string| {
string.is_implicit_concatenated()
&& !is_expression_parenthesized(

View File

@@ -1,11 +1,11 @@
use ruff_python_ast::AnyNodeRef;
use ruff_python_ast::ExprBinOp;
use ruff_python_ast::{AnyNodeRef, StringLike};
use crate::expression::binary_like::BinaryLike;
use crate::expression::has_parentheses;
use crate::expression::parentheses::{NeedsParentheses, OptionalParentheses};
use crate::prelude::*;
use crate::string::AnyString;
use crate::string::StringLikeExtensions;
#[derive(Default)]
pub struct FormatExprBinOp;
@@ -25,7 +25,7 @@ impl NeedsParentheses for ExprBinOp {
) -> OptionalParentheses {
if parent.is_expr_await() {
OptionalParentheses::Always
} else if let Some(string) = AnyString::from_expression(&self.left) {
} else if let Ok(string) = StringLike::try_from(&*self.left) {
// Multiline strings are guaranteed to never fit, avoid adding unnecessary parentheses
if !string.is_implicit_concatenated()
&& string.is_multiline(context.source())

View File

@@ -1,11 +1,11 @@
use ruff_python_ast::AnyNodeRef;
use ruff_python_ast::ExprBytesLiteral;
use ruff_python_ast::{AnyNodeRef, StringLike};
use crate::expression::parentheses::{
in_parentheses_only_group, NeedsParentheses, OptionalParentheses,
};
use crate::prelude::*;
use crate::string::{AnyString, FormatImplicitConcatenatedString};
use crate::string::{FormatImplicitConcatenatedString, StringLikeExtensions};
#[derive(Default)]
pub struct FormatExprBytesLiteral;
@@ -29,7 +29,7 @@ impl NeedsParentheses for ExprBytesLiteral {
) -> OptionalParentheses {
if self.value.is_implicit_concatenated() {
OptionalParentheses::Multiline
} else if AnyString::Bytes(self).is_multiline(context.source()) {
} else if StringLike::Bytes(self).is_multiline(context.source()) {
OptionalParentheses::Never
} else {
OptionalParentheses::BestFit

View File

@@ -1,12 +1,12 @@
use ruff_formatter::{FormatOwnedWithRule, FormatRefWithRule};
use ruff_python_ast::AnyNodeRef;
use ruff_python_ast::{AnyNodeRef, StringLike};
use ruff_python_ast::{CmpOp, ExprCompare};
use crate::expression::binary_like::BinaryLike;
use crate::expression::has_parentheses;
use crate::expression::parentheses::{NeedsParentheses, OptionalParentheses};
use crate::prelude::*;
use crate::string::AnyString;
use crate::string::StringLikeExtensions;
#[derive(Default)]
pub struct FormatExprCompare;
@@ -26,7 +26,7 @@ impl NeedsParentheses for ExprCompare {
) -> OptionalParentheses {
if parent.is_expr_await() {
OptionalParentheses::Always
} else if let Some(string) = AnyString::from_expression(&self.left) {
} else if let Ok(string) = StringLike::try_from(&*self.left) {
// Multiline strings are guaranteed to never fit, avoid adding unnecessary parentheses
if !string.is_implicit_concatenated()
&& string.is_multiline(context.source())

View File

@@ -1,4 +1,4 @@
use ruff_python_ast::{AnyNodeRef, ExprFString};
use ruff_python_ast::{AnyNodeRef, ExprFString, StringLike};
use ruff_source_file::Locator;
use ruff_text_size::Ranged;
@@ -7,7 +7,7 @@ use crate::expression::parentheses::{
};
use crate::other::f_string_part::FormatFStringPart;
use crate::prelude::*;
use crate::string::{AnyString, FormatImplicitConcatenatedString, Quoting};
use crate::string::{FormatImplicitConcatenatedString, Quoting, StringLikeExtensions};
#[derive(Default)]
pub struct FormatExprFString;
@@ -53,7 +53,7 @@ impl NeedsParentheses for ExprFString {
// ```
// This isn't decided yet, refer to the relevant discussion:
// https://github.com/astral-sh/ruff/discussions/9785
} else if AnyString::FString(self).is_multiline(context.source()) {
} else if StringLike::FString(self).is_multiline(context.source()) {
OptionalParentheses::Never
} else {
OptionalParentheses::BestFit

View File

@@ -1,12 +1,12 @@
use ruff_formatter::FormatRuleWithOptions;
use ruff_python_ast::{AnyNodeRef, ExprStringLiteral};
use ruff_python_ast::{AnyNodeRef, ExprStringLiteral, StringLike};
use crate::expression::parentheses::{
in_parentheses_only_group, NeedsParentheses, OptionalParentheses,
};
use crate::other::string_literal::StringLiteralKind;
use crate::prelude::*;
use crate::string::{AnyString, FormatImplicitConcatenatedString};
use crate::string::{FormatImplicitConcatenatedString, StringLikeExtensions};
#[derive(Default)]
pub struct FormatExprStringLiteral {
@@ -48,7 +48,7 @@ impl NeedsParentheses for ExprStringLiteral {
) -> OptionalParentheses {
if self.value.is_implicit_concatenated() {
OptionalParentheses::Multiline
} else if AnyString::String(self).is_multiline(context.source()) {
} else if StringLike::String(self).is_multiline(context.source()) {
OptionalParentheses::Never
} else {
OptionalParentheses::BestFit

View File

@@ -1,5 +1,5 @@
use ruff_formatter::{write, FormatContext};
use ruff_python_ast::{ArgOrKeyword, Arguments, Expr};
use ruff_python_ast::{ArgOrKeyword, Arguments, Expr, StringLike};
use ruff_python_trivia::{PythonWhitespace, SimpleTokenKind, SimpleTokenizer};
use ruff_text_size::{Ranged, TextLen, TextRange, TextSize};
@@ -8,7 +8,7 @@ use crate::expression::is_expression_huggable;
use crate::expression::parentheses::{empty_parenthesized, parenthesized, Parentheses};
use crate::other::commas;
use crate::prelude::*;
use crate::string::AnyString;
use crate::string::StringLikeExtensions;
#[derive(Default)]
pub struct FormatArguments;
@@ -179,8 +179,8 @@ fn is_arguments_huggable(arguments: &Arguments, context: &PyFormatContext) -> bo
// If the expression itself isn't huggable, then we can't hug it.
if !(is_expression_huggable(arg, context)
|| AnyString::from_expression(arg)
.is_some_and(|string| is_huggable_string_argument(string, arguments, context)))
|| StringLike::try_from(arg)
.is_ok_and(|string| is_huggable_string_argument(string, arguments, context)))
{
return false;
}
@@ -219,7 +219,7 @@ fn is_arguments_huggable(arguments: &Arguments, context: &PyFormatContext) -> bo
/// )
/// ```
fn is_huggable_string_argument(
string: AnyString,
string: StringLike,
arguments: &Arguments,
context: &PyFormatContext,
) -> bool {

View File

@@ -1,202 +0,0 @@
use std::iter::FusedIterator;
use memchr::memchr2;
use ruff_python_ast::{
self as ast, AnyNodeRef, AnyStringFlags, Expr, ExprBytesLiteral, ExprFString,
ExprStringLiteral, ExpressionRef, StringFlags, StringLiteral,
};
use ruff_source_file::Locator;
use ruff_text_size::{Ranged, TextRange};
use crate::expression::expr_f_string::f_string_quoting;
use crate::string::Quoting;
/// Represents any kind of string expression. This could be either a string,
/// bytes or f-string.
#[derive(Copy, Clone, Debug)]
pub(crate) enum AnyString<'a> {
String(&'a ExprStringLiteral),
Bytes(&'a ExprBytesLiteral),
FString(&'a ExprFString),
}
impl<'a> AnyString<'a> {
/// Creates a new [`AnyString`] from the given [`Expr`].
///
/// Returns `None` if the expression is not either a string, bytes or f-string.
pub(crate) fn from_expression(expression: &'a Expr) -> Option<AnyString<'a>> {
match expression {
Expr::StringLiteral(string) => Some(AnyString::String(string)),
Expr::BytesLiteral(bytes) => Some(AnyString::Bytes(bytes)),
Expr::FString(fstring) => Some(AnyString::FString(fstring)),
_ => None,
}
}
/// Returns `true` if the string is implicitly concatenated.
pub(crate) fn is_implicit_concatenated(self) -> bool {
match self {
Self::String(ExprStringLiteral { value, .. }) => value.is_implicit_concatenated(),
Self::Bytes(ExprBytesLiteral { value, .. }) => value.is_implicit_concatenated(),
Self::FString(ExprFString { value, .. }) => value.is_implicit_concatenated(),
}
}
pub(crate) const fn is_fstring(self) -> bool {
matches!(self, Self::FString(_))
}
/// Returns the quoting to be used for this string.
pub(super) fn quoting(self, locator: &Locator<'_>) -> Quoting {
match self {
Self::String(_) | Self::Bytes(_) => Quoting::CanChange,
Self::FString(f_string) => f_string_quoting(f_string, locator),
}
}
/// Returns an iterator over the [`AnyStringPart`]s of this string.
pub(super) fn parts(self) -> AnyStringPartsIter<'a> {
match self {
Self::String(ExprStringLiteral { value, .. }) => {
AnyStringPartsIter::String(value.iter())
}
Self::Bytes(ExprBytesLiteral { value, .. }) => AnyStringPartsIter::Bytes(value.iter()),
Self::FString(ExprFString { value, .. }) => AnyStringPartsIter::FString(value.iter()),
}
}
pub(crate) fn is_multiline(self, source: &str) -> bool {
match self {
AnyString::String(_) | AnyString::Bytes(_) => {
self.parts()
.next()
.is_some_and(|part| part.flags().is_triple_quoted())
&& memchr2(b'\n', b'\r', source[self.range()].as_bytes()).is_some()
}
AnyString::FString(fstring) => {
memchr2(b'\n', b'\r', source[fstring.range].as_bytes()).is_some()
}
}
}
}
impl Ranged for AnyString<'_> {
fn range(&self) -> TextRange {
match self {
Self::String(expr) => expr.range(),
Self::Bytes(expr) => expr.range(),
Self::FString(expr) => expr.range(),
}
}
}
impl<'a> From<&AnyString<'a>> for AnyNodeRef<'a> {
fn from(value: &AnyString<'a>) -> Self {
match value {
AnyString::String(expr) => AnyNodeRef::ExprStringLiteral(expr),
AnyString::Bytes(expr) => AnyNodeRef::ExprBytesLiteral(expr),
AnyString::FString(expr) => AnyNodeRef::ExprFString(expr),
}
}
}
impl<'a> From<AnyString<'a>> for AnyNodeRef<'a> {
fn from(value: AnyString<'a>) -> Self {
AnyNodeRef::from(&value)
}
}
impl<'a> From<&AnyString<'a>> for ExpressionRef<'a> {
fn from(value: &AnyString<'a>) -> Self {
match value {
AnyString::String(expr) => ExpressionRef::StringLiteral(expr),
AnyString::Bytes(expr) => ExpressionRef::BytesLiteral(expr),
AnyString::FString(expr) => ExpressionRef::FString(expr),
}
}
}
impl<'a> From<&'a ExprBytesLiteral> for AnyString<'a> {
fn from(value: &'a ExprBytesLiteral) -> Self {
AnyString::Bytes(value)
}
}
impl<'a> From<&'a ExprStringLiteral> for AnyString<'a> {
fn from(value: &'a ExprStringLiteral) -> Self {
AnyString::String(value)
}
}
impl<'a> From<&'a ExprFString> for AnyString<'a> {
fn from(value: &'a ExprFString) -> Self {
AnyString::FString(value)
}
}
pub(super) enum AnyStringPartsIter<'a> {
String(std::slice::Iter<'a, StringLiteral>),
Bytes(std::slice::Iter<'a, ast::BytesLiteral>),
FString(std::slice::Iter<'a, ast::FStringPart>),
}
impl<'a> Iterator for AnyStringPartsIter<'a> {
type Item = AnyStringPart<'a>;
fn next(&mut self) -> Option<Self::Item> {
let part = match self {
Self::String(inner) => AnyStringPart::String(inner.next()?),
Self::Bytes(inner) => AnyStringPart::Bytes(inner.next()?),
Self::FString(inner) => match inner.next()? {
ast::FStringPart::Literal(string_literal) => AnyStringPart::String(string_literal),
ast::FStringPart::FString(f_string) => AnyStringPart::FString(f_string),
},
};
Some(part)
}
}
impl FusedIterator for AnyStringPartsIter<'_> {}
/// Represents any kind of string which is part of an implicitly concatenated
/// string. This could be either a string, bytes or f-string.
///
/// This is constructed from the [`AnyString::parts`] method on [`AnyString`].
#[derive(Clone, Debug)]
pub(super) enum AnyStringPart<'a> {
String(&'a ast::StringLiteral),
Bytes(&'a ast::BytesLiteral),
FString(&'a ast::FString),
}
impl AnyStringPart<'_> {
fn flags(&self) -> AnyStringFlags {
match self {
Self::String(part) => part.flags.into(),
Self::Bytes(bytes_literal) => bytes_literal.flags.into(),
Self::FString(part) => part.flags.into(),
}
}
}
impl<'a> From<&AnyStringPart<'a>> for AnyNodeRef<'a> {
fn from(value: &AnyStringPart<'a>) -> Self {
match value {
AnyStringPart::String(part) => AnyNodeRef::StringLiteral(part),
AnyStringPart::Bytes(part) => AnyNodeRef::BytesLiteral(part),
AnyStringPart::FString(part) => AnyNodeRef::FString(part),
}
}
}
impl Ranged for AnyStringPart<'_> {
fn range(&self) -> TextRange {
match self {
Self::String(part) => part.range(),
Self::Bytes(part) => part.range(),
Self::FString(part) => part.range(),
}
}
}

View File

@@ -1,23 +1,24 @@
pub(crate) use any::AnyString;
use memchr::memchr2;
pub(crate) use normalize::{normalize_string, NormalizedString, StringNormalizer};
use ruff_formatter::format_args;
use ruff_python_ast::str::Quote;
use ruff_python_ast::{
self as ast,
str_prefix::{AnyStringPrefix, StringLiteralPrefix},
AnyStringFlags, StringFlags,
AnyStringFlags, StringFlags, StringLike, StringLikePart,
};
use ruff_text_size::{Ranged, TextRange};
use ruff_source_file::Locator;
use ruff_text_size::Ranged;
use crate::comments::{leading_comments, trailing_comments};
use crate::expression::expr_f_string::f_string_quoting;
use crate::expression::parentheses::in_parentheses_only_soft_line_break_or_space;
use crate::other::f_string::FormatFString;
use crate::other::string_literal::StringLiteralKind;
use crate::prelude::*;
use crate::string::any::AnyStringPart;
use crate::QuoteStyle;
mod any;
pub(crate) mod docstring;
mod normalize;
@@ -31,11 +32,11 @@ pub(crate) enum Quoting {
/// Formats any implicitly concatenated string. This could be any valid combination
/// of string, bytes or f-string literals.
pub(crate) struct FormatImplicitConcatenatedString<'a> {
string: AnyString<'a>,
string: StringLike<'a>,
}
impl<'a> FormatImplicitConcatenatedString<'a> {
pub(crate) fn new(string: impl Into<AnyString<'a>>) -> Self {
pub(crate) fn new(string: impl Into<StringLike<'a>>) -> Self {
Self {
string: string.into(),
}
@@ -53,7 +54,7 @@ impl Format<PyFormatContext<'_>> for FormatImplicitConcatenatedString<'_> {
let part_comments = comments.leading_dangling_trailing(&part);
let format_part = format_with(|f: &mut PyFormatter| match part {
AnyStringPart::String(part) => {
StringLikePart::String(part) => {
let kind = if self.string.is_fstring() {
#[allow(deprecated)]
StringLiteralKind::InImplicitlyConcatenatedFString(quoting)
@@ -63,8 +64,8 @@ impl Format<PyFormatContext<'_>> for FormatImplicitConcatenatedString<'_> {
part.format().with_options(kind).fmt(f)
}
AnyStringPart::Bytes(bytes_literal) => bytes_literal.format().fmt(f),
AnyStringPart::FString(part) => FormatFString::new(part, quoting).fmt(f),
StringLikePart::Bytes(bytes_literal) => bytes_literal.format().fmt(f),
StringLikePart::FString(part) => FormatFString::new(part, quoting).fmt(f),
});
joiner.entry(&format_args![
@@ -141,57 +142,32 @@ impl From<Quote> for QuoteStyle {
}
}
#[derive(Debug, Clone, Copy)]
pub(crate) struct StringPart {
flags: AnyStringFlags,
range: TextRange,
// Extension trait that adds formatter specific helper methods to `StringLike`.
pub(crate) trait StringLikeExtensions {
fn quoting(&self, locator: &Locator<'_>) -> Quoting;
fn is_multiline(&self, source: &str) -> bool;
}
impl Ranged for StringPart {
fn range(&self) -> TextRange {
self.range
}
}
impl StringPart {
/// Use the `kind()` method to retrieve information about the
fn flags(self) -> AnyStringFlags {
self.flags
impl StringLikeExtensions for ast::StringLike<'_> {
fn quoting(&self, locator: &Locator<'_>) -> Quoting {
match self {
Self::String(_) | Self::Bytes(_) => Quoting::CanChange,
Self::FString(f_string) => f_string_quoting(f_string, locator),
}
}
/// Returns the range of the string's content in the source (minus prefix and quotes).
fn content_range(self) -> TextRange {
let kind = self.flags();
TextRange::new(
self.start() + kind.opener_len(),
self.end() - kind.closer_len(),
)
}
}
impl From<&ast::StringLiteral> for StringPart {
fn from(value: &ast::StringLiteral) -> Self {
Self {
range: value.range,
flags: value.flags.into(),
}
}
}
impl From<&ast::BytesLiteral> for StringPart {
fn from(value: &ast::BytesLiteral) -> Self {
Self {
range: value.range,
flags: value.flags.into(),
}
}
}
impl From<&ast::FString> for StringPart {
fn from(value: &ast::FString) -> Self {
Self {
range: value.range,
flags: value.flags.into(),
fn is_multiline(&self, source: &str) -> bool {
match self {
Self::String(_) | Self::Bytes(_) => {
self.parts()
.next()
.is_some_and(|part| part.flags().is_triple_quoted())
&& memchr2(b'\n', b'\r', source[self.range()].as_bytes()).is_some()
}
Self::FString(fstring) => {
memchr2(b'\n', b'\r', source[fstring.range].as_bytes()).is_some()
}
}
}
}

View File

@@ -3,13 +3,13 @@ use std::cmp::Ordering;
use std::iter::FusedIterator;
use ruff_formatter::FormatContext;
use ruff_python_ast::{str::Quote, AnyStringFlags, StringFlags};
use ruff_python_ast::{str::Quote, AnyStringFlags, StringFlags, StringLikePart};
use ruff_text_size::{Ranged, TextRange};
use crate::context::FStringState;
use crate::prelude::*;
use crate::preview::is_f_string_formatting_enabled;
use crate::string::{Quoting, StringPart, StringQuotes};
use crate::string::{Quoting, StringQuotes};
use crate::QuoteStyle;
pub(crate) struct StringNormalizer<'a, 'src> {
@@ -37,7 +37,7 @@ impl<'a, 'src> StringNormalizer<'a, 'src> {
self
}
fn quoting(&self, string: StringPart) -> Quoting {
fn quoting(&self, string: StringLikePart) -> Quoting {
if let FStringState::InsideExpressionElement(context) = self.context.f_string_state() {
// If we're inside an f-string, we need to make sure to preserve the
// existing quotes unless we're inside a triple-quoted f-string and
@@ -66,7 +66,7 @@ impl<'a, 'src> StringNormalizer<'a, 'src> {
}
/// Computes the strings preferred quotes.
pub(crate) fn choose_quotes(&self, string: StringPart) -> QuoteSelection {
pub(crate) fn choose_quotes(&self, string: StringLikePart) -> QuoteSelection {
let raw_content = self.context.locator().slice(string.content_range());
let first_quote_or_normalized_char_offset = raw_content
.bytes()
@@ -168,7 +168,7 @@ impl<'a, 'src> StringNormalizer<'a, 'src> {
}
/// Computes the strings preferred quotes and normalizes its content.
pub(crate) fn normalize(&self, string: StringPart) -> NormalizedString<'src> {
pub(crate) fn normalize(&self, string: StringLikePart) -> NormalizedString<'src> {
let raw_content = self.context.locator().slice(string.content_range());
let quote_selection = self.choose_quotes(string);

View File

@@ -0,0 +1,340 @@
---
source: crates/ruff_python_formatter/tests/fixtures.rs
input_file: crates/ruff_python_formatter/resources/test/fixtures/black/cases/context_managers_39.py
---
## Input
```python
with \
make_context_manager1() as cm1, \
make_context_manager2() as cm2, \
make_context_manager3() as cm3, \
make_context_manager4() as cm4 \
:
pass
# Leading comment
with \
make_context_manager1() as cm1, \
make_context_manager2(), \
make_context_manager3() as cm3, \
make_context_manager4() \
:
pass
with \
new_new_new1() as cm1, \
new_new_new2() \
:
pass
with (
new_new_new1() as cm1,
new_new_new2()
):
pass
# Leading comment.
with (
# First comment.
new_new_new1() as cm1,
# Second comment.
new_new_new2()
# Last comment.
):
pass
with \
this_is_a_very_long_call(looong_arg1=looong_value1, looong_arg2=looong_value2) as cm1, \
this_is_a_very_long_call(looong_arg1=looong_value1, looong_arg2=looong_value2, looong_arg3=looong_value3, looong_arg4=looong_value4) as cm2 \
:
pass
with mock.patch.object(
self.my_runner, "first_method", autospec=True
) as mock_run_adb, mock.patch.object(
self.my_runner, "second_method", autospec=True, return_value="foo"
):
pass
with xxxxxxxx.some_kind_of_method(
some_argument=[
"first",
"second",
"third",
]
).another_method() as cmd:
pass
async def func():
async with \
make_context_manager1() as cm1, \
make_context_manager2() as cm2, \
make_context_manager3() as cm3, \
make_context_manager4() as cm4 \
:
pass
async with some_function(
argument1, argument2, argument3="some_value"
) as some_cm, some_other_function(
argument1, argument2, argument3="some_value"
):
pass
```
## Black Differences
```diff
--- Black
+++ Ruff
@@ -1,19 +1,9 @@
-with (
- make_context_manager1() as cm1,
- make_context_manager2() as cm2,
- make_context_manager3() as cm3,
- make_context_manager4() as cm4,
-):
+with make_context_manager1() as cm1, make_context_manager2() as cm2, make_context_manager3() as cm3, make_context_manager4() as cm4:
pass
# Leading comment
-with (
- make_context_manager1() as cm1,
- make_context_manager2(),
- make_context_manager3() as cm3,
- make_context_manager4(),
-):
+with make_context_manager1() as cm1, make_context_manager2(), make_context_manager3() as cm3, make_context_manager4():
pass
@@ -36,25 +26,21 @@
pass
-with (
- this_is_a_very_long_call(
- looong_arg1=looong_value1, looong_arg2=looong_value2
- ) as cm1,
- this_is_a_very_long_call(
- looong_arg1=looong_value1,
- looong_arg2=looong_value2,
- looong_arg3=looong_value3,
- looong_arg4=looong_value4,
- ) as cm2,
-):
+with this_is_a_very_long_call(
+ looong_arg1=looong_value1, looong_arg2=looong_value2
+) as cm1, this_is_a_very_long_call(
+ looong_arg1=looong_value1,
+ looong_arg2=looong_value2,
+ looong_arg3=looong_value3,
+ looong_arg4=looong_value4,
+) as cm2:
pass
-with (
- mock.patch.object(self.my_runner, "first_method", autospec=True) as mock_run_adb,
- mock.patch.object(
- self.my_runner, "second_method", autospec=True, return_value="foo"
- ),
+with mock.patch.object(
+ self.my_runner, "first_method", autospec=True
+) as mock_run_adb, mock.patch.object(
+ self.my_runner, "second_method", autospec=True, return_value="foo"
):
pass
@@ -70,16 +56,10 @@
async def func():
- async with (
- make_context_manager1() as cm1,
- make_context_manager2() as cm2,
- make_context_manager3() as cm3,
- make_context_manager4() as cm4,
- ):
+ async with make_context_manager1() as cm1, make_context_manager2() as cm2, make_context_manager3() as cm3, make_context_manager4() as cm4:
pass
- async with (
- some_function(argument1, argument2, argument3="some_value") as some_cm,
- some_other_function(argument1, argument2, argument3="some_value"),
- ):
+ async with some_function(
+ argument1, argument2, argument3="some_value"
+ ) as some_cm, some_other_function(argument1, argument2, argument3="some_value"):
pass
```
## Ruff Output
```python
with make_context_manager1() as cm1, make_context_manager2() as cm2, make_context_manager3() as cm3, make_context_manager4() as cm4:
pass
# Leading comment
with make_context_manager1() as cm1, make_context_manager2(), make_context_manager3() as cm3, make_context_manager4():
pass
with new_new_new1() as cm1, new_new_new2():
pass
with new_new_new1() as cm1, new_new_new2():
pass
# Leading comment.
with (
# First comment.
new_new_new1() as cm1,
# Second comment.
new_new_new2(),
# Last comment.
):
pass
with this_is_a_very_long_call(
looong_arg1=looong_value1, looong_arg2=looong_value2
) as cm1, this_is_a_very_long_call(
looong_arg1=looong_value1,
looong_arg2=looong_value2,
looong_arg3=looong_value3,
looong_arg4=looong_value4,
) as cm2:
pass
with mock.patch.object(
self.my_runner, "first_method", autospec=True
) as mock_run_adb, mock.patch.object(
self.my_runner, "second_method", autospec=True, return_value="foo"
):
pass
with xxxxxxxx.some_kind_of_method(
some_argument=[
"first",
"second",
"third",
]
).another_method() as cmd:
pass
async def func():
async with make_context_manager1() as cm1, make_context_manager2() as cm2, make_context_manager3() as cm3, make_context_manager4() as cm4:
pass
async with some_function(
argument1, argument2, argument3="some_value"
) as some_cm, some_other_function(argument1, argument2, argument3="some_value"):
pass
```
## Black Output
```python
with (
make_context_manager1() as cm1,
make_context_manager2() as cm2,
make_context_manager3() as cm3,
make_context_manager4() as cm4,
):
pass
# Leading comment
with (
make_context_manager1() as cm1,
make_context_manager2(),
make_context_manager3() as cm3,
make_context_manager4(),
):
pass
with new_new_new1() as cm1, new_new_new2():
pass
with new_new_new1() as cm1, new_new_new2():
pass
# Leading comment.
with (
# First comment.
new_new_new1() as cm1,
# Second comment.
new_new_new2(),
# Last comment.
):
pass
with (
this_is_a_very_long_call(
looong_arg1=looong_value1, looong_arg2=looong_value2
) as cm1,
this_is_a_very_long_call(
looong_arg1=looong_value1,
looong_arg2=looong_value2,
looong_arg3=looong_value3,
looong_arg4=looong_value4,
) as cm2,
):
pass
with (
mock.patch.object(self.my_runner, "first_method", autospec=True) as mock_run_adb,
mock.patch.object(
self.my_runner, "second_method", autospec=True, return_value="foo"
),
):
pass
with xxxxxxxx.some_kind_of_method(
some_argument=[
"first",
"second",
"third",
]
).another_method() as cmd:
pass
async def func():
async with (
make_context_manager1() as cm1,
make_context_manager2() as cm2,
make_context_manager3() as cm3,
make_context_manager4() as cm4,
):
pass
async with (
some_function(argument1, argument2, argument3="some_value") as some_cm,
some_other_function(argument1, argument2, argument3="some_value"),
):
pass
```

View File

@@ -0,0 +1,79 @@
---
source: crates/ruff_python_formatter/tests/fixtures.rs
input_file: crates/ruff_python_formatter/resources/test/fixtures/black/cases/context_managers_autodetect_39.py
---
## Input
```python
# This file uses parenthesized context managers introduced in Python 3.9.
with \
make_context_manager1() as cm1, \
make_context_manager2() as cm2, \
make_context_manager3() as cm3, \
make_context_manager4() as cm4 \
:
pass
with (
new_new_new1() as cm1,
new_new_new2()
):
pass
```
## Black Differences
```diff
--- Black
+++ Ruff
@@ -1,12 +1,7 @@
# This file uses parenthesized context managers introduced in Python 3.9.
-with (
- make_context_manager1() as cm1,
- make_context_manager2() as cm2,
- make_context_manager3() as cm3,
- make_context_manager4() as cm4,
-):
+with make_context_manager1() as cm1, make_context_manager2() as cm2, make_context_manager3() as cm3, make_context_manager4() as cm4:
pass
```
## Ruff Output
```python
# This file uses parenthesized context managers introduced in Python 3.9.
with make_context_manager1() as cm1, make_context_manager2() as cm2, make_context_manager3() as cm3, make_context_manager4() as cm4:
pass
with new_new_new1() as cm1, new_new_new2():
pass
```
## Black Output
```python
# This file uses parenthesized context managers introduced in Python 3.9.
with (
make_context_manager1() as cm1,
make_context_manager2() as cm2,
make_context_manager3() as cm3,
make_context_manager4() as cm4,
):
pass
with new_new_new1() as cm1, new_new_new2():
pass
```

View File

@@ -177,94 +177,7 @@ for foo in ["a", "b"]:
```diff
--- Black
+++ Ruff
@@ -1,43 +1,55 @@
def foo_brackets(request):
- return JsonResponse({
- "var_1": foo,
- "var_2": bar,
- })
+ return JsonResponse(
+ {
+ "var_1": foo,
+ "var_2": bar,
+ }
+ )
def foo_square_brackets(request):
- return JsonResponse([
- "var_1",
- "var_2",
- ])
+ return JsonResponse(
+ [
+ "var_1",
+ "var_2",
+ ]
+ )
-func({
- "a": 37,
- "b": 42,
- "c": 927,
- "aaaaaaaaaaaaaaaaaaaaaaaaa": 11111111111111111111111111111111111111111,
-})
+func(
+ {
+ "a": 37,
+ "b": 42,
+ "c": 927,
+ "aaaaaaaaaaaaaaaaaaaaaaaaa": 11111111111111111111111111111111111111111,
+ }
+)
-func([
- "random_string_number_one",
- "random_string_number_two",
- "random_string_number_three",
- "random_string_number_four",
-])
+func(
+ [
+ "random_string_number_one",
+ "random_string_number_two",
+ "random_string_number_three",
+ "random_string_number_four",
+ ]
+)
-func({
- # expand me
- "a": 37,
- "b": 42,
- "c": 927,
-})
+func(
+ {
+ # expand me
+ "a": 37,
+ "b": 42,
+ "c": 927,
+ }
+)
-func([
- "a",
- "b",
- "c",
-])
+func(
+ [
+ "a",
+ "b",
+ "c",
+ ]
+)
func(
[
@@ -47,17 +59,21 @@
@@ -47,17 +47,21 @@
],
)
@@ -296,67 +209,9 @@ for foo in ["a", "b"]:
func(
# preserve me
@@ -68,38 +84,48 @@
]
)
-func([ # preserve me but hug brackets
- "c",
- "d",
- "e",
-])
+func(
+ [ # preserve me but hug brackets
+ "c",
+ "d",
+ "e",
+ ]
+)
-func([
- # preserve me but hug brackets
- "c",
- "d",
- "e",
-])
+func(
+ [
+ # preserve me but hug brackets
+ "c",
+ "d",
+ "e",
+ ]
+)
-func([
- "c",
- # preserve me but hug brackets
- "d",
- "e",
-])
+func(
+ [
+ "c",
+ # preserve me but hug brackets
+ "d",
+ "e",
+ ]
+)
-func([
- "c",
- "d",
- "e",
- # preserve me but hug brackets
-])
+func(
+ [
+ "c",
+ "d",
+ "e",
+ # preserve me but hug brackets
+ ]
+)
@@ -95,11 +99,13 @@
# preserve me but hug brackets
])
-func([
- "c",
@@ -373,28 +228,21 @@ for foo in ["a", "b"]:
func(
[
@@ -114,13 +140,15 @@
func(
[x for x in "long line long line long line long line long line long line long line"]
@@ -111,10 +117,10 @@
)
-func([
- x
- for x in [
+func(
+ [
x
- for x in "long line long line long line long line long line long line long line"
+ for x in [
+ x
+ for x in "long line long line long line long line long line long line long line"
+ ]
]
-])
+)
foooooooooooooooooooo(
[{c: n + 1 for c in range(256)} for n in range(100)] + [{}], {size}
@@ -131,10 +159,12 @@
func([x for x in "short line"])
-func(
- [x for x in "long line long line long line long line long line long line long line"]
-)
func([
+ x for x in "long line long line long line long line long line long line long line"
+])
+func([
x
for x in [
x
@@ -131,10 +137,12 @@
)
nested_mapping = {
@@ -411,135 +259,64 @@ for foo in ["a", "b"]:
}
explicit_exploding = [
[
@@ -144,24 +174,34 @@
],
],
]
-single_item_do_not_explode = Context({
- "version": get_docs_version(),
-})
+single_item_do_not_explode = Context(
+ {
+ "version": get_docs_version(),
+ }
+)
-foo(*[
- str(i) for i in range(100000000000000000000000000000000000000000000000000000000000)
-])
+foo(
+ *[
+ str(i)
+ for i in range(100000000000000000000000000000000000000000000000000000000000)
+ ]
+)
-foo(**{
- "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa": 1,
- "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb": 2,
- "ccccccccccccccccccccccccccccccccc": 3,
- **other,
-})
+foo(
+ **{
+ "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa": 1,
+ "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb": 2,
+ "ccccccccccccccccccccccccccccccccc": 3,
+ **other,
+ }
+)
-foo(**{
- x: y for x, y in enumerate(["long long long long line", "long long long long line"])
-})
+foo(
+ **{
+ x: y
+ for x, y in enumerate(["long long long long line", "long long long long line"])
+ }
+)
@@ -164,9 +172,9 @@
})
# Edge case when deciding whether to hug the brackets without inner content.
very_very_very_long_variable = very_very_very_long_module.VeryVeryVeryVeryLongClassName(
@@ -169,11 +209,13 @@
)
-very_very_very_long_variable = very_very_very_long_module.VeryVeryVeryVeryLongClassName(
- [[]]
-)
+very_very_very_long_variable = very_very_very_long_module.VeryVeryVeryVeryLongClassName([
+ []
+])
for foo in ["a", "b"]:
- output.extend([
- individual
- for
- # Foobar
- container in xs_by_y[foo]
- # Foobar
- for individual in container["nested"]
- ])
+ output.extend(
+ [
+ individual
+ for
+ # Foobar
+ container in xs_by_y[foo]
+ # Foobar
+ for individual in container["nested"]
+ ]
+ )
output.extend([
```
## Ruff Output
```python
def foo_brackets(request):
return JsonResponse(
{
"var_1": foo,
"var_2": bar,
}
)
return JsonResponse({
"var_1": foo,
"var_2": bar,
})
def foo_square_brackets(request):
return JsonResponse(
[
"var_1",
"var_2",
]
)
return JsonResponse([
"var_1",
"var_2",
])
func(
{
"a": 37,
"b": 42,
"c": 927,
"aaaaaaaaaaaaaaaaaaaaaaaaa": 11111111111111111111111111111111111111111,
}
)
func({
"a": 37,
"b": 42,
"c": 927,
"aaaaaaaaaaaaaaaaaaaaaaaaa": 11111111111111111111111111111111111111111,
})
func(
[
"random_string_number_one",
"random_string_number_two",
"random_string_number_three",
"random_string_number_four",
]
)
func([
"random_string_number_one",
"random_string_number_two",
"random_string_number_three",
"random_string_number_four",
])
func(
{
# expand me
"a": 37,
"b": 42,
"c": 927,
}
)
func({
# expand me
"a": 37,
"b": 42,
"c": 927,
})
func(
[
"a",
"b",
"c",
]
)
func([
"a",
"b",
"c",
])
func(
[
@@ -574,40 +351,32 @@ func(
]
)
func(
[ # preserve me but hug brackets
"c",
"d",
"e",
]
)
func([ # preserve me but hug brackets
"c",
"d",
"e",
])
func(
[
# preserve me but hug brackets
"c",
"d",
"e",
]
)
func([
# preserve me but hug brackets
"c",
"d",
"e",
])
func(
[
"c",
# preserve me but hug brackets
"d",
"e",
]
)
func([
"c",
# preserve me but hug brackets
"d",
"e",
])
func(
[
"c",
"d",
"e",
# preserve me but hug brackets
]
)
func([
"c",
"d",
"e",
# preserve me but hug brackets
])
func(
[
@@ -627,18 +396,16 @@ func(
)
func([x for x in "short line"])
func(
[x for x in "long line long line long line long line long line long line long line"]
)
func(
[
func([
x for x in "long line long line long line long line long line long line long line"
])
func([
x
for x in [
x
for x in [
x
for x in "long line long line long line long line long line long line long line"
]
for x in "long line long line long line long line long line long line long line"
]
)
])
foooooooooooooooooooo(
[{c: n + 1 for c in range(256)} for n in range(100)] + [{}], {size}
@@ -664,51 +431,39 @@ explicit_exploding = [
],
],
]
single_item_do_not_explode = Context(
{
"version": get_docs_version(),
}
)
single_item_do_not_explode = Context({
"version": get_docs_version(),
})
foo(
*[
str(i)
for i in range(100000000000000000000000000000000000000000000000000000000000)
]
)
foo(*[
str(i) for i in range(100000000000000000000000000000000000000000000000000000000000)
])
foo(
**{
"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa": 1,
"bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb": 2,
"ccccccccccccccccccccccccccccccccc": 3,
**other,
}
)
foo(**{
"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa": 1,
"bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb": 2,
"ccccccccccccccccccccccccccccccccc": 3,
**other,
})
foo(
**{
x: y
for x, y in enumerate(["long long long long line", "long long long long line"])
}
)
foo(**{
x: y for x, y in enumerate(["long long long long line", "long long long long line"])
})
# Edge case when deciding whether to hug the brackets without inner content.
very_very_very_long_variable = very_very_very_long_module.VeryVeryVeryVeryLongClassName(
[[]]
)
very_very_very_long_variable = very_very_very_long_module.VeryVeryVeryVeryLongClassName([
[]
])
for foo in ["a", "b"]:
output.extend(
[
individual
for
# Foobar
container in xs_by_y[foo]
# Foobar
for individual in container["nested"]
]
)
output.extend([
individual
for
# Foobar
container in xs_by_y[foo]
# Foobar
for individual in container["nested"]
])
```
## Black Output

View File

@@ -35,30 +35,10 @@ nested_array = [[["long line", "long long line", "long long long line", "long lo
```diff
--- Black
+++ Ruff
@@ -1,47 +1,65 @@
# split out from preview_hug_parens_with_brackes_and_square_brackets, as it produces
# different code on the second pass with line-length 1 in many cases.
# Seems to be about whether the last string in a sequence gets wrapped in parens or not.
-foo(*[
- "long long long long long line",
- "long long long long long line",
- "long long long long long line",
-])
+foo(
+ *[
+ "long long long long long line",
+ "long long long long long line",
+ "long long long long long line",
+ ]
+)
func({"short line"})
-func({
- "long line",
- "long long line",
- "long long long line",
- "long long long long line",
- "long long long long long line",
-})
@@ -14,13 +14,15 @@
"long long long long line",
"long long long long long line",
})
-func({{
- "long line",
- "long long line",
@@ -66,20 +46,22 @@ nested_array = [[["long line", "long long line", "long long long line", "long lo
- "long long long long line",
- "long long long long long line",
-}})
-func((
- "long line",
- "long long line",
- "long long long line",
- "long long long long line",
- "long long long long long line",
-))
-func(((
- "long line",
- "long long line",
- "long long long line",
- "long long long long line",
- "long long long long long line",
-)))
+func({
+ {
+ "long line",
+ "long long line",
+ "long long long line",
+ "long long long long line",
+ "long long long long long line",
+ }
+})
func((
"long line",
"long long line",
@@ -35,31 +37,63 @@
"long long long long line",
"long long long long long line",
)))
-func([[
- "long line",
- "long long line",
@@ -87,61 +69,82 @@ nested_array = [[["long line", "long long line", "long long long line", "long lo
- "long long long long line",
- "long long long long long line",
-]])
+func(
+ {
+ "long line",
+ "long long line",
+ "long long long line",
+ "long long long long line",
+ "long long long long long line",
+ }
+)
+func(
+ {
+ {
+ "long line",
+ "long long line",
+ "long long long line",
+ "long long long long line",
+ "long long long long long line",
+ }
+ }
+)
+func(
+ (
+ "long line",
+ "long long line",
+ "long long long line",
+ "long long long long line",
+ "long long long long long line",
+ )
+)
+func(
+ (
+ (
+ "long line",
+ "long long line",
+ "long long long line",
+ "long long long long line",
+ "long long long long long line",
+ )
+ )
+)
+func(
+func([
+ [
+ [
+ "long line",
+ "long long line",
+ "long long long line",
+ "long long long long line",
+ "long long long long long line",
+ ]
+ "long line",
+ "long long line",
+ "long long long line",
+ "long long long long line",
+ "long long long long long line",
+ ]
+)
+])
# Do not hug if the argument fits on a single line.
@@ -70,10 +88,14 @@
-func(
- {"fit line", "fit line", "fit line", "fit line", "fit line", "fit line", "fit line"}
-)
-func(
- ("fit line", "fit line", "fit line", "fit line", "fit line", "fit line", "fit line")
-)
-func(
- ["fit line", "fit line", "fit line", "fit line", "fit line", "fit line", "fit line"]
-)
-func(
- **{"fit line", "fit line", "fit line", "fit line", "fit line", "fit line", "fit---"}
-)
-func(
- *("fit line", "fit line", "fit line", "fit line", "fit line", "fit line", "fit----")
-)
+func({
+ "fit line",
+ "fit line",
+ "fit line",
+ "fit line",
+ "fit line",
+ "fit line",
+ "fit line",
+})
+func((
+ "fit line",
+ "fit line",
+ "fit line",
+ "fit line",
+ "fit line",
+ "fit line",
+ "fit line",
+))
+func([
+ "fit line",
+ "fit line",
+ "fit line",
+ "fit line",
+ "fit line",
+ "fit line",
+ "fit line",
+])
+func(**{
+ "fit line",
+ "fit line",
+ "fit line",
+ "fit line",
+ "fit line",
+ "fit line",
+ "fit---",
+})
+func(*(
+ "fit line",
+ "fit line",
+ "fit line",
+ "fit line",
+ "fit line",
+ "fit line",
+ "fit----",
+))
array = [
{"fit line", "fit line", "fit line", "fit line", "fit line", "fit line", "fit line"}
]
@@ -70,10 +104,14 @@
["fit line", "fit line", "fit line", "fit line", "fit line", "fit line", "fit line"]
]
@@ -171,15 +174,20 @@ nested_array = [[["long line", "long long line", "long long long line", "long lo
# split out from preview_hug_parens_with_brackes_and_square_brackets, as it produces
# different code on the second pass with line-length 1 in many cases.
# Seems to be about whether the last string in a sequence gets wrapped in parens or not.
foo(
*[
"long long long long long line",
"long long long long long line",
"long long long long long line",
]
)
foo(*[
"long long long long long line",
"long long long long long line",
"long long long long long line",
])
func({"short line"})
func(
func({
"long line",
"long long line",
"long long long line",
"long long long long line",
"long long long long long line",
})
func({
{
"long line",
"long long line",
@@ -187,67 +195,78 @@ func(
"long long long long line",
"long long long long long line",
}
)
func(
{
{
"long line",
"long long line",
"long long long line",
"long long long long line",
"long long long long long line",
}
}
)
func(
(
"long line",
"long long line",
"long long long line",
"long long long long line",
"long long long long long line",
)
)
func(
(
(
"long line",
"long long line",
"long long long line",
"long long long long line",
"long long long long long line",
)
)
)
func(
})
func((
"long line",
"long long line",
"long long long line",
"long long long long line",
"long long long long long line",
))
func(((
"long line",
"long long line",
"long long long line",
"long long long long line",
"long long long long long line",
)))
func([
[
[
"long line",
"long long line",
"long long long line",
"long long long long line",
"long long long long long line",
]
"long line",
"long long line",
"long long long line",
"long long long long line",
"long long long long long line",
]
)
])
# Do not hug if the argument fits on a single line.
func(
{"fit line", "fit line", "fit line", "fit line", "fit line", "fit line", "fit line"}
)
func(
("fit line", "fit line", "fit line", "fit line", "fit line", "fit line", "fit line")
)
func(
["fit line", "fit line", "fit line", "fit line", "fit line", "fit line", "fit line"]
)
func(
**{"fit line", "fit line", "fit line", "fit line", "fit line", "fit line", "fit---"}
)
func(
*("fit line", "fit line", "fit line", "fit line", "fit line", "fit line", "fit----")
)
func({
"fit line",
"fit line",
"fit line",
"fit line",
"fit line",
"fit line",
"fit line",
})
func((
"fit line",
"fit line",
"fit line",
"fit line",
"fit line",
"fit line",
"fit line",
))
func([
"fit line",
"fit line",
"fit line",
"fit line",
"fit line",
"fit line",
"fit line",
])
func(**{
"fit line",
"fit line",
"fit line",
"fit line",
"fit line",
"fit line",
"fit---",
})
func(*(
"fit line",
"fit line",
"fit line",
"fit line",
"fit line",
"fit line",
"fit----",
))
array = [
{"fit line", "fit line", "fit line", "fit line", "fit line", "fit line", "fit line"}
]

View File

@@ -114,37 +114,19 @@ class Random:
}
{
@@ -52,16 +46,18 @@
class Random:
def func():
random_service.status.active_states.inactive = (
- make_new_top_level_state_from_dict({
- "topLevelBase": {
- "secondaryBase": {
- "timestamp": 1234,
- "latitude": 1,
- "longitude": 2,
@@ -58,9 +52,9 @@
"timestamp": 1234,
"latitude": 1,
"longitude": 2,
- "actionTimestamp": (
- Timestamp(seconds=1530584000, nanos=0).ToJsonString()
- ),
- }
- },
- })
+ make_new_top_level_state_from_dict(
+ {
+ "topLevelBase": {
+ "secondaryBase": {
+ "timestamp": 1234,
+ "latitude": 1,
+ "longitude": 2,
+ "actionTimestamp": Timestamp(
+ seconds=1530584000, nanos=0
+ ).ToJsonString(),
+ }
+ },
+ }
+ )
)
+ "actionTimestamp": Timestamp(
+ seconds=1530584000, nanos=0
+ ).ToJsonString(),
}
},
})
```
## Ruff Output
@@ -198,20 +180,18 @@ my_dict = {
class Random:
def func():
random_service.status.active_states.inactive = (
make_new_top_level_state_from_dict(
{
"topLevelBase": {
"secondaryBase": {
"timestamp": 1234,
"latitude": 1,
"longitude": 2,
"actionTimestamp": Timestamp(
seconds=1530584000, nanos=0
).ToJsonString(),
}
},
}
)
make_new_top_level_state_from_dict({
"topLevelBase": {
"secondaryBase": {
"timestamp": 1234,
"latitude": 1,
"longitude": 2,
"actionTimestamp": Timestamp(
seconds=1530584000, nanos=0
).ToJsonString(),
}
},
})
)
```

View File

@@ -902,7 +902,7 @@ log.info(f"""Skipping: {'a' == 'b'} {desc['ms_name']} {money=} {dte=} {pos_share
)
dict_with_lambda_values = {
@@ -524,61 +383,54 @@
@@ -524,65 +383,58 @@
# Complex string concatenations with a method call in the middle.
code = (
@@ -941,7 +941,7 @@ log.info(f"""Skipping: {'a' == 'b'} {desc['ms_name']} {money=} {dte=} {pos_share
log.info(
- "Skipping:"
- f" {desc['db_id']} {foo('bar',x=123)} {'foo' != 'bar'} {(x := 'abc=')} {pos_share=} {desc['status']} {desc['exposure_max']}"
+ f'Skipping: {desc["db_id"]} {foo("bar",x=123)} {"foo" != "bar"} {(x := "abc=")} {pos_share=} {desc["status"]} {desc["exposure_max"]}'
+ f'Skipping: {desc["db_id"]} {foo("bar", x=123)} {"foo" != "bar"} {(x := "abc=")} {pos_share=} {desc["status"]} {desc["exposure_max"]}'
)
log.info(
@@ -981,6 +981,18 @@ log.info(f"""Skipping: {'a' == 'b'} {desc['ms_name']} {money=} {dte=} {pos_share
)
log.info(
- f"""Skipping: {"a" == 'b'} {desc["ms_name"]} {money=} {dte=} {pos_share=} {desc["status"]} {desc["exposure_max"]}"""
+ f"""Skipping: {"a" == "b"} {desc["ms_name"]} {money=} {dte=} {pos_share=} {desc["status"]} {desc["exposure_max"]}"""
)
log.info(
@@ -590,5 +442,5 @@
)
log.info(
- f"""Skipping: {'a' == 'b'} {desc['ms_name']} {money=} {dte=} {pos_share=} {desc['status']} {desc['exposure_max']}"""
+ f"""Skipping: {"a" == "b"} {desc["ms_name"]} {money=} {dte=} {pos_share=} {desc["status"]} {desc["exposure_max"]}"""
)
```
## Ruff Output
@@ -1394,7 +1406,7 @@ log.info(
)
log.info(
f'Skipping: {desc["db_id"]} {foo("bar",x=123)} {"foo" != "bar"} {(x := "abc=")} {pos_share=} {desc["status"]} {desc["exposure_max"]}'
f'Skipping: {desc["db_id"]} {foo("bar", x=123)} {"foo" != "bar"} {(x := "abc=")} {pos_share=} {desc["status"]} {desc["exposure_max"]}'
)
log.info(
@@ -1422,7 +1434,7 @@ log.info(
)
log.info(
f"""Skipping: {"a" == 'b'} {desc["ms_name"]} {money=} {dte=} {pos_share=} {desc["status"]} {desc["exposure_max"]}"""
f"""Skipping: {"a" == "b"} {desc["ms_name"]} {money=} {dte=} {pos_share=} {desc["status"]} {desc["exposure_max"]}"""
)
log.info(
@@ -1430,7 +1442,7 @@ log.info(
)
log.info(
f"""Skipping: {'a' == 'b'} {desc['ms_name']} {money=} {dte=} {pos_share=} {desc['status']} {desc['exposure_max']}"""
f"""Skipping: {"a" == "b"} {desc["ms_name"]} {money=} {dte=} {pos_share=} {desc["status"]} {desc["exposure_max"]}"""
)
```

View File

@@ -573,7 +573,7 @@ s = f'Lorem Ipsum is simply dummy text of the printing and typesetting industry:
```diff
--- Black
+++ Ruff
@@ -25,41 +25,42 @@
@@ -25,20 +25,17 @@
"Jaguar",
)
@@ -599,27 +599,7 @@ s = f'Lorem Ipsum is simply dummy text of the printing and typesetting industry:
)
class A:
def foo():
- XXXXXXXXXXXX.append((
- "xxx_xxxxxxxxxx(xxxxx={}, xxxx={}, xxxxx, xxxx_xxxx_xxxxxxxxxx={})".format(
- xxxxx, xxxx, xxxx_xxxx_xxxxxxxxxx
- ),
- my_var,
- my_other_var,
- ))
+ XXXXXXXXXXXX.append(
+ (
+ "xxx_xxxxxxxxxx(xxxxx={}, xxxx={}, xxxxx, xxxx_xxxx_xxxxxxxxxx={})".format(
+ xxxxx, xxxx, xxxx_xxxx_xxxxxxxxxx
+ ),
+ my_var,
+ my_other_var,
+ )
+ )
class A:
@@ -57,9 +54,11 @@
class B:
def foo():
bar(
@@ -634,7 +614,7 @@ s = f'Lorem Ipsum is simply dummy text of the printing and typesetting industry:
),
varX,
varY,
@@ -70,9 +71,10 @@
@@ -70,9 +69,10 @@
def foo(xxxx):
for xxx_xxxx, _xxx_xxx, _xxx_xxxxx, xxx_xxxx in xxxx:
for xxx in xxx_xxxx:
@@ -648,7 +628,7 @@ s = f'Lorem Ipsum is simply dummy text of the printing and typesetting industry:
)
@@ -80,10 +82,11 @@
@@ -80,10 +80,11 @@
def disappearing_comment():
return (
( # xx -x xxxxxxx xx xxx xxxxxxx.
@@ -662,7 +642,7 @@ s = f'Lorem Ipsum is simply dummy text of the printing and typesetting industry:
"--xxxxxxx --xxxxxx=x --xxxxxx-xxxxx=xxxxxx"
" --xxxxxx-xxxx=xxxxxxxxxxx.xxx"
)
@@ -113,18 +116,25 @@
@@ -113,18 +114,25 @@
func_call_where_string_arg_has_method_call_and_bad_parens(
@@ -694,7 +674,7 @@ s = f'Lorem Ipsum is simply dummy text of the printing and typesetting industry:
)
@@ -132,52 +142,60 @@
@@ -132,52 +140,60 @@
def append(self):
if True:
xxxx.xxxxxxx.xxxxx(
@@ -788,7 +768,7 @@ s = f'Lorem Ipsum is simply dummy text of the printing and typesetting industry:
}
@@ -185,10 +203,10 @@
@@ -185,10 +201,10 @@
def foo(self):
if True:
xxxxx_xxxxxxxxxxxx(
@@ -803,7 +783,7 @@ s = f'Lorem Ipsum is simply dummy text of the printing and typesetting industry:
)
@@ -232,39 +250,24 @@
@@ -232,39 +248,24 @@
some_dictionary = {
"xxxxx006": [
@@ -852,7 +832,7 @@ s = f'Lorem Ipsum is simply dummy text of the printing and typesetting industry:
some_commented_string = ( # This comment stays at the top.
"This string is long but not so long that it needs hahahah toooooo be so greatttt"
@@ -279,36 +282,25 @@
@@ -279,37 +280,26 @@
)
lpar_and_rpar_have_comments = func_call( # LPAR Comment
@@ -872,13 +852,13 @@ s = f'Lorem Ipsum is simply dummy text of the printing and typesetting industry:
- f" {'' if ID is None else ID} | perl -nE 'print if /^{field}:/'"
-)
+cmd_fstring = f"sudo -E deluge-console info --detailed --sort-reverse=time_added {'' if ID is None else ID} | perl -nE 'print if /^{field}:/'"
+
+cmd_fstring = f"sudo -E deluge-console info --detailed --sort-reverse=time_added {'{{}}' if ID is None else ID} | perl -nE 'print if /^{field}:/'"
-cmd_fstring = (
- "sudo -E deluge-console info --detailed --sort-reverse=time_added"
- f" {'{{}}' if ID is None else ID} | perl -nE 'print if /^{field}:/'"
-)
+cmd_fstring = f"sudo -E deluge-console info --detailed --sort-reverse=time_added {'{{}}' if ID is None else ID} | perl -nE 'print if /^{field}:/'"
+
+cmd_fstring = f"sudo -E deluge-console info --detailed --sort-reverse=time_added {{'' if ID is None else ID}} | perl -nE 'print if /^{field}:/'"
-cmd_fstring = (
@@ -892,12 +872,13 @@ s = f'Lorem Ipsum is simply dummy text of the printing and typesetting industry:
- f" certainly, absolutely {does}."
+ f"We have to remember to escape {braces}." " Like {these}." f" But not {this}."
)
-
-fstring = f"We have to remember to escape {braces}. Like {{these}}. But not {this}."
-fstring = f"We have to remember to escape {braces}. Like {{these}}. But not {this}."
-
class A:
@@ -364,10 +356,7 @@
class B:
@@ -364,10 +354,7 @@
def foo():
if not hasattr(module, name):
raise ValueError(
@@ -909,7 +890,7 @@ s = f'Lorem Ipsum is simply dummy text of the printing and typesetting industry:
% (name, module_name, get_docs_version())
)
@@ -382,35 +371,33 @@
@@ -382,23 +369,19 @@
class Step(StepBase):
def who(self):
@@ -931,29 +912,16 @@ s = f'Lorem Ipsum is simply dummy text of the printing and typesetting industry:
)
-xxxxxxx_xxxxxx_xxxxxxx = xxx([
- xxxxxxxxxxxx(
- xxxxxx_xxxxxxx=(
xxxxxxx_xxxxxx_xxxxxxx = xxx([
xxxxxxxxxxxx(
xxxxxx_xxxxxxx=(
- '((x.aaaaaaaaa = "xxxxxx.xxxxxxxxxxxxxxxxxxxxx") || (x.xxxxxxxxx ='
- ' "xxxxxxxxxxxx")) && '
- # xxxxx xxxxxxxxxxxx xxxx xxx (xxxxxxxxxxxxxxxx) xx x xxxxxxxxx xx xxxxxx.
- "(x.bbbbbbbbbbbb.xxx != "
- '"xxx:xxx:xxx::cccccccccccc:xxxxxxx-xxxx/xxxxxxxxxxx/xxxxxxxxxxxxxxxxx") && '
+xxxxxxx_xxxxxx_xxxxxxx = xxx(
+ [
+ xxxxxxxxxxxx(
+ xxxxxx_xxxxxxx=(
+ '((x.aaaaaaaaa = "xxxxxx.xxxxxxxxxxxxxxxxxxxxx") || (x.xxxxxxxxx = "xxxxxxxxxxxx")) && '
+ # xxxxx xxxxxxxxxxxx xxxx xxx (xxxxxxxxxxxxxxxx) xx x xxxxxxxxx xx xxxxxx.
+ "(x.bbbbbbbbbbbb.xxx != "
+ '"xxx:xxx:xxx::cccccccccccc:xxxxxxx-xxxx/xxxxxxxxxxx/xxxxxxxxxxxxxxxxx") && '
+ )
)
- )
-])
+ ]
+)
+ '((x.aaaaaaaaa = "xxxxxx.xxxxxxxxxxxxxxxxxxxxx") || (x.xxxxxxxxx = "xxxxxxxxxxxx")) && '
# xxxxx xxxxxxxxxxxx xxxx xxx (xxxxxxxxxxxxxxxx) xx x xxxxxxxxx xx xxxxxx.
"(x.bbbbbbbbbbbb.xxx != "
'"xxx:xxx:xxx::cccccccccccc:xxxxxxx-xxxx/xxxxxxxxxxx/xxxxxxxxxxxxxxxxx") && '
@@ -409,8 +392,8 @@
if __name__ == "__main__":
for i in range(4, 8):
cmd = (
@@ -964,7 +932,7 @@ s = f'Lorem Ipsum is simply dummy text of the printing and typesetting industry:
)
@@ -432,9 +419,7 @@
@@ -432,9 +415,7 @@
assert xxxxxxx_xxxx in [
x.xxxxx.xxxxxx.xxxxx.xxxxxx,
x.xxxxx.xxxxxx.xxxxx.xxxx,
@@ -975,7 +943,7 @@ s = f'Lorem Ipsum is simply dummy text of the printing and typesetting industry:
value.__dict__[key] = (
@@ -449,8 +434,7 @@
@@ -449,8 +430,7 @@
RE_TWO_BACKSLASHES = {
"asdf_hjkl_jkl": re.compile(
@@ -985,23 +953,23 @@ s = f'Lorem Ipsum is simply dummy text of the printing and typesetting industry:
),
}
@@ -462,13 +446,9 @@
@@ -462,13 +442,9 @@
# We do NOT split on f-string expressions.
print(
- "Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam."
- f" {[f'{i}' for i in range(10)]}"
-)
+ f"Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam. {[f'{i}' for i in range(10)]}"
)
-x = (
- "This is a long string which contains an f-expr that should not split"
- f" {{{[i for i in range(5)]}}}."
+ f"Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam. {[f'{i}' for i in range(10)]}"
)
-)
+x = f"This is a long string which contains an f-expr that should not split {{{[i for i in range(5)]}}}."
# The parens should NOT be removed in this case.
(
@@ -478,8 +458,8 @@
@@ -478,8 +454,8 @@
# The parens should NOT be removed in this case.
(
@@ -1012,7 +980,13 @@ s = f'Lorem Ipsum is simply dummy text of the printing and typesetting industry:
)
# The parens should NOT be removed in this case.
@@ -518,88 +498,78 @@
@@ -513,93 +489,83 @@
temp_msg = (
- f"{f'{humanize_number(pos)}.': <{pound_len+2}} "
+ f"{f'{humanize_number(pos)}.': <{pound_len + 2}} "
f"{balance: <{bal_len + 5}} "
f"<<{author.display_name}>>\n"
)
@@ -1136,7 +1110,13 @@ s = f'Lorem Ipsum is simply dummy text of the printing and typesetting industry:
"6. Click on Create Credential at the top."
'7. At the top click the link for "API key".'
"8. No application restrictions are needed. Click Create at the bottom."
@@ -613,55 +583,40 @@
@@ -608,60 +574,45 @@
# It shouldn't matter if the string prefixes are capitalized.
temp_msg = (
- f"{F'{humanize_number(pos)}.': <{pound_len+2}} "
+ f"{f'{humanize_number(pos)}.': <{pound_len + 2}} "
f"{balance: <{bal_len + 5}} "
f"<<{author.display_name}>>\n"
)
@@ -1209,7 +1189,7 @@ s = f'Lorem Ipsum is simply dummy text of the printing and typesetting industry:
)
# Regression test for https://github.com/psf/black/issues/3455.
@@ -672,9 +627,11 @@
@@ -672,9 +623,11 @@
}
# Regression test for https://github.com/psf/black/issues/3506.
@@ -1273,15 +1253,13 @@ class A:
class A:
def foo():
XXXXXXXXXXXX.append(
(
"xxx_xxxxxxxxxx(xxxxx={}, xxxx={}, xxxxx, xxxx_xxxx_xxxxxxxxxx={})".format(
xxxxx, xxxx, xxxx_xxxx_xxxxxxxxxx
),
my_var,
my_other_var,
)
)
XXXXXXXXXXXX.append((
"xxx_xxxxxxxxxx(xxxxx={}, xxxx={}, xxxxx, xxxx_xxxx_xxxxxxxxxx={})".format(
xxxxx, xxxx, xxxx_xxxx_xxxxxxxxxx
),
my_var,
my_other_var,
))
class A:
@@ -1612,18 +1590,16 @@ class Step(StepBase):
)
xxxxxxx_xxxxxx_xxxxxxx = xxx(
[
xxxxxxxxxxxx(
xxxxxx_xxxxxxx=(
'((x.aaaaaaaaa = "xxxxxx.xxxxxxxxxxxxxxxxxxxxx") || (x.xxxxxxxxx = "xxxxxxxxxxxx")) && '
# xxxxx xxxxxxxxxxxx xxxx xxx (xxxxxxxxxxxxxxxx) xx x xxxxxxxxx xx xxxxxx.
"(x.bbbbbbbbbbbb.xxx != "
'"xxx:xxx:xxx::cccccccccccc:xxxxxxx-xxxx/xxxxxxxxxxx/xxxxxxxxxxxxxxxxx") && '
)
xxxxxxx_xxxxxx_xxxxxxx = xxx([
xxxxxxxxxxxx(
xxxxxx_xxxxxxx=(
'((x.aaaaaaaaa = "xxxxxx.xxxxxxxxxxxxxxxxxxxxx") || (x.xxxxxxxxx = "xxxxxxxxxxxx")) && '
# xxxxx xxxxxxxxxxxx xxxx xxx (xxxxxxxxxxxxxxxx) xx x xxxxxxxxx xx xxxxxx.
"(x.bbbbbbbbbbbb.xxx != "
'"xxx:xxx:xxx::cccccccccccc:xxxxxxx-xxxx/xxxxxxxxxxx/xxxxxxxxxxxxxxxxx") && '
)
]
)
)
])
if __name__ == "__main__":
for i in range(4, 8):
@@ -1725,7 +1701,7 @@ class X:
temp_msg = (
f"{f'{humanize_number(pos)}.': <{pound_len+2}} "
f"{f'{humanize_number(pos)}.': <{pound_len + 2}} "
f"{balance: <{bal_len + 5}} "
f"<<{author.display_name}>>\n"
)
@@ -1810,7 +1786,7 @@ message = (
# It shouldn't matter if the string prefixes are capitalized.
temp_msg = (
f"{F'{humanize_number(pos)}.': <{pound_len+2}} "
f"{f'{humanize_number(pos)}.': <{pound_len + 2}} "
f"{balance: <{bal_len + 5}} "
f"<<{author.display_name}>>\n"
)