Compare commits

...

54 Commits

Author SHA1 Message Date
David Peter
f9b1fa25e4 Initial version 2024-11-13 10:25:30 +01:00
Charlie Marsh
147ea399fd Remove extraneous baz.py file (#14299) 2024-11-12 14:01:19 +00:00
David Peter
907047bf4b [red-knot] Add tests for member lookup on union types (#14296)
## Summary

- Write tests for member lookups on union types
- Remove TODO comment

part of: #14022

## Test Plan

New MD tests
2024-11-12 14:11:55 +01:00
InSync
13a1483f1e [flake8-pyi] Add "replace with Self" fix (PYI019) (#14238)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-11-12 11:13:15 +00:00
InSync
be69f61b3e [flake8-simplify] Infer "unknown" truthiness for literal iterables whose items are all unpacks (SIM222) (#14263)
## Summary

Resolves #14237.

## Test Plan

`cargo nextest run` and `cargo insta test`.

---------

Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
2024-11-11 15:23:34 -05:00
David Peter
f1f3bd1cd3 [red-knot] Review remaining 'possibly unbound' call sites (#14284)
## Summary

- Emit diagnostics when looking up (possibly) unbound attributes
- More explicit test assertions for unbound symbols
- Review remaining call sites of `Symbol::ignore_possibly_unbound`. Most
of them are something like `builtins_symbol(self.db,
"Ellipsis").ignore_possibly_unbound().unwrap_or(Type::Unknown)` which
look okay to me, unless we want to emit additional diagnostics. There is
one additional case in enum literal handling, which has a TODO comment
anyway.

part of #14022

## Test Plan

New MD tests for (possibly) unbound attributes.
2024-11-11 20:48:49 +01:00
David Peter
3bef23669f [red-knot] Diagnostic for possibly unbound imports (#14281)
## Summary

This adds a new diagnostic when possibly unbound symbols are imported.
The `TODO` comment had a question mark, do I'm not sure if this is
really something that we want.

This does not touch the un*declared* case, yet.

relates to: #14022

## Test Plan

Updated already existing tests with new diagnostics
2024-11-11 20:26:01 +01:00
David Salvisberg
f82ee8ea59 [flake8-markupsafe] Adds Implementation for MS001 via RUF035 (#14224)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-11-11 18:30:03 +00:00
David Peter
b8a65182dd [red-knot] Symbol API improvements, part 2 (#14276)
## Summary

Apart from one small functional change, this is mostly a refactoring of
the `Symbol` API:

- Rename `as_type` to the more explicit `ignore_possibly_unbound`, no
functional change
- Remove `unwrap_or_unknown` in favor of the more explicit
`.ignore_possibly_unbound().unwrap_or(Type::Unknown)`, no functional
change
- Consistently call it "possibly unbound" (not "may be unbound")
- Rename `replace_unbound_with` to `or_fall_back_to` and properly handle
boundness of the fall back. This is the only functional change (did not
have any impact on existing tests).

relates to: #14022

## Test Plan

New unit tests for `Symbol::or_fall_back_to`

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-11-11 15:24:27 +01:00
Alex Waygood
fc15d8a3bd [red-knot] Infer Literal types from comparisons with sys.version_info (#14244) 2024-11-11 13:58:16 +00:00
Simon Brugman
b3b5c19105 Minor refactoring of some flake-pyi rules (#14275)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-11-11 13:10:48 +00:00
Simon Brugman
f8aae9b1d6 [flake8-pyi] Mark fix as unsafe when type annotation contains comments for duplicate-literal-member (PYI062) (#14268) 2024-11-11 12:48:14 +00:00
Alex Waygood
9180635171 [red-knot] Cleanup some KnownClass APIs (#14269) 2024-11-11 11:54:42 +00:00
Alex Waygood
3ef4b3bf32 [red-knot] Shorten the paths for some mdtest files (#14267) 2024-11-11 11:34:33 +00:00
w0nder1ng
5a3886c8b5 [perflint] implement quick-fix for manual-list-comprehension (PERF401) (#13919)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-11-11 11:17:02 +00:00
Alex Waygood
813ec23ecd [red-knot] Improve mdtest output (#14213) 2024-11-11 11:03:41 +00:00
Dhruv Manilawala
13883414af Add "Notebook behavior" section for F704, PLE1142 (#14266)
## Summary

Move the relevant contents into "Notebook behavior" section similar to
other rules.
2024-11-11 10:54:28 +00:00
Simon Brugman
84d4f114ef Use bitshift consistently for bitflag definitions (#14265) 2024-11-11 10:20:17 +00:00
renovate[bot]
1c586b29e2 Update dependency mkdocs-redirects to v1.2.2 (#14252)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-11-11 10:11:22 +00:00
renovate[bot]
d76a8518c2 Update dependency uuid to v11.0.3 (#14254)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-11-11 10:11:12 +00:00
renovate[bot]
5f0ee2670a Update cloudflare/wrangler-action action to v3.12.1 (#14261)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-11-11 09:56:12 +00:00
renovate[bot]
f8ca6c3316 Update NPM Development dependencies (#14259)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-11-11 09:55:43 +00:00
renovate[bot]
ba7b023f26 Update Rust crate tempfile to v3.14.0 (#14260)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-11-11 09:48:51 +00:00
renovate[bot]
e947d163b2 Update Rust crate thiserror to v2 (#14262)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-11-11 09:46:09 +00:00
renovate[bot]
1cf4d2ff69 Update dependency ruff to v0.7.3 (#14253)
This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [ruff](https://docs.astral.sh/ruff)
([source](https://redirect.github.com/astral-sh/ruff),
[changelog](https://redirect.github.com/astral-sh/ruff/blob/main/CHANGELOG.md))
| `==0.7.2` -> `==0.7.3` |
[![age](https://developer.mend.io/api/mc/badges/age/pypi/ruff/0.7.3?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/ruff/0.7.3?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/ruff/0.7.2/0.7.3?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/ruff/0.7.2/0.7.3?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|

---

### Release Notes

<details>
<summary>astral-sh/ruff (ruff)</summary>

###
[`v0.7.3`](https://redirect.github.com/astral-sh/ruff/blob/HEAD/CHANGELOG.md#073)

[Compare
Source](https://redirect.github.com/astral-sh/ruff/compare/0.7.2...0.7.3)

##### Preview features

- Formatter: Disallow single-line implicit concatenated strings
([#&#8203;13928](https://redirect.github.com/astral-sh/ruff/pull/13928))
- \[`flake8-pyi`] Include all Python file types for `PYI006` and
`PYI066`
([#&#8203;14059](https://redirect.github.com/astral-sh/ruff/pull/14059))
- \[`flake8-simplify`] Implement `split-of-static-string` (`SIM905`)
([#&#8203;14008](https://redirect.github.com/astral-sh/ruff/pull/14008))
- \[`refurb`] Implement `subclass-builtin` (`FURB189`)
([#&#8203;14105](https://redirect.github.com/astral-sh/ruff/pull/14105))
- \[`ruff`] Improve diagnostic messages and docs (`RUF031`, `RUF032`,
`RUF034`)
([#&#8203;14068](https://redirect.github.com/astral-sh/ruff/pull/14068))

##### Rule changes

- Detect items that hash to same value in duplicate sets (`B033`,
`PLC0208`)
([#&#8203;14064](https://redirect.github.com/astral-sh/ruff/pull/14064))
- \[`eradicate`] Better detection of IntelliJ language injection
comments (`ERA001`)
([#&#8203;14094](https://redirect.github.com/astral-sh/ruff/pull/14094))
- \[`flake8-pyi`] Add autofix for `docstring-in-stub` (`PYI021`)
([#&#8203;14150](https://redirect.github.com/astral-sh/ruff/pull/14150))
- \[`flake8-pyi`] Update `duplicate-literal-member` (`PYI062`) to alawys
provide an autofix
([#&#8203;14188](https://redirect.github.com/astral-sh/ruff/pull/14188))
- \[`pyflakes`] Detect items that hash to same value in duplicate
dictionaries (`F601`)
([#&#8203;14065](https://redirect.github.com/astral-sh/ruff/pull/14065))
- \[`ruff`] Fix false positive for decorators (`RUF028`)
([#&#8203;14061](https://redirect.github.com/astral-sh/ruff/pull/14061))

##### Bug fixes

- Avoid parsing joint rule codes as distinct codes in `# noqa`
([#&#8203;12809](https://redirect.github.com/astral-sh/ruff/pull/12809))
- \[`eradicate`] ignore `# language=` in commented-out-code rule
(ERA001)
([#&#8203;14069](https://redirect.github.com/astral-sh/ruff/pull/14069))
- \[`flake8-bugbear`] - do not run `mutable-argument-default` on stubs
(`B006`)
([#&#8203;14058](https://redirect.github.com/astral-sh/ruff/pull/14058))
- \[`flake8-builtins`] Skip lambda expressions in
`builtin-argument-shadowing (A002)`
([#&#8203;14144](https://redirect.github.com/astral-sh/ruff/pull/14144))
- \[`flake8-comprehension`] Also remove trailing comma while fixing
`C409` and `C419`
([#&#8203;14097](https://redirect.github.com/astral-sh/ruff/pull/14097))
- \[`flake8-simplify`] Allow `open` without context manager in `return`
statement (`SIM115`)
([#&#8203;14066](https://redirect.github.com/astral-sh/ruff/pull/14066))
- \[`pylint`] Respect hash-equivalent literals in `iteration-over-set`
(`PLC0208`)
([#&#8203;14063](https://redirect.github.com/astral-sh/ruff/pull/14063))
- \[`pylint`] Update known dunder methods for Python 3.13 (`PLW3201`)
([#&#8203;14146](https://redirect.github.com/astral-sh/ruff/pull/14146))
- \[`pyupgrade`] - ignore kwarg unpacking for `UP044`
([#&#8203;14053](https://redirect.github.com/astral-sh/ruff/pull/14053))
- \[`refurb`] Parse more exotic decimal strings in
`verbose-decimal-constructor` (`FURB157`)
([#&#8203;14098](https://redirect.github.com/astral-sh/ruff/pull/14098))

##### Documentation

- Add links to missing related options within rule documentations
([#&#8203;13971](https://redirect.github.com/astral-sh/ruff/pull/13971))
- Add rule short code to mkdocs tags to allow searching via rule codes
([#&#8203;14040](https://redirect.github.com/astral-sh/ruff/pull/14040))

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "before 4am on Monday" (UTC),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/astral-sh/ruff).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS43LjEiLCJ1cGRhdGVkSW5WZXIiOiIzOS43LjEiLCJ0YXJnZXRCcmFuY2giOiJtYWluIiwibGFiZWxzIjpbImludGVybmFsIl19-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-11-11 08:17:22 +00:00
renovate[bot]
2308522f38 Update pre-commit dependencies (#14256)
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
|
[astral-sh/ruff-pre-commit](https://redirect.github.com/astral-sh/ruff-pre-commit)
| repository | patch | `v0.7.2` -> `v0.7.3` |
| [crate-ci/typos](https://redirect.github.com/crate-ci/typos) |
repository | patch | `v1.27.0` -> `v1.27.3` |

Note: The `pre-commit` manager in Renovate is not supported by the
`pre-commit` maintainers or community. Please do not report any problems
there, instead [create a Discussion in the Renovate
repository](https://redirect.github.com/renovatebot/renovate/discussions/new)
if you have any questions.

---

### Release Notes

<details>
<summary>astral-sh/ruff-pre-commit (astral-sh/ruff-pre-commit)</summary>

###
[`v0.7.3`](https://redirect.github.com/astral-sh/ruff-pre-commit/releases/tag/v0.7.3)

[Compare
Source](https://redirect.github.com/astral-sh/ruff-pre-commit/compare/v0.7.2...v0.7.3)

See: https://github.com/astral-sh/ruff/releases/tag/0.7.3

</details>

<details>
<summary>crate-ci/typos (crate-ci/typos)</summary>

###
[`v1.27.3`](https://redirect.github.com/crate-ci/typos/releases/tag/v1.27.3)

[Compare
Source](https://redirect.github.com/crate-ci/typos/compare/v1.27.2...v1.27.3)

#### \[1.27.3] - 2024-11-08

##### Fixes

-   Don't correct `alloced`
- Don't correct `registor`, a more domain specific variant of `register`

###
[`v1.27.2`](https://redirect.github.com/crate-ci/typos/releases/tag/v1.27.2)

[Compare
Source](https://redirect.github.com/crate-ci/typos/compare/v1.27.1...v1.27.2)

#### \[1.27.2] - 2024-11-06

##### Fixes

-   Correct `fand`

###
[`v1.27.1`](https://redirect.github.com/crate-ci/typos/releases/tag/v1.27.1)

[Compare
Source](https://redirect.github.com/crate-ci/typos/compare/v1.27.0...v1.27.1)

#### \[1.27.1] - 2024-11-06

##### Fixes

-   Correct `alingment` as `alignment`, rather than `alinement`

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "before 4am on Monday" (UTC),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

👻 **Immortal**: This PR will be recreated if closed unmerged. Get
[config
help](https://redirect.github.com/renovatebot/renovate/discussions) if
that's undesired.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/astral-sh/ruff).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS43LjEiLCJ1cGRhdGVkSW5WZXIiOiIzOS43LjEiLCJ0YXJnZXRCcmFuY2giOiJtYWluIiwibGFiZWxzIjpbImludGVybmFsIl19-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-11-11 08:17:07 +00:00
David Peter
438f3d967b [red-knot] is_disjoint_from: tests for function/module literals (#14264)
## Summary

Add unit tests for `is_disjoint_from` for function and module literals
as a follow-up to #14210.

Ref: https://github.com/astral-sh/ruff/pull/14210/files#r1835069885
2024-11-11 09:14:26 +01:00
Charlie Marsh
5bf4759cff Detect permutations in redundant open modes (#14255)
## Summary

Closes https://github.com/astral-sh/ruff/issues/14235.
2024-11-10 22:48:30 -05:00
renovate[bot]
2e9e96338e Update Rust crate url to v2.5.3 (#14251) 2024-11-10 19:47:14 -05:00
renovate[bot]
5fa7ace1f5 Update Rust crate matchit to v0.8.5 (#14250) 2024-11-10 19:47:09 -05:00
renovate[bot]
704868ca83 Update Rust crate libc to v0.2.162 (#14249) 2024-11-10 19:47:02 -05:00
renovate[bot]
dc71c8a484 Update Rust crate hashbrown to v0.15.1 (#14247) 2024-11-10 19:46:55 -05:00
renovate[bot]
2499297392 Update Rust crate is-macro to v0.3.7 (#14248) 2024-11-10 19:46:48 -05:00
renovate[bot]
7b9189bb2c Update Rust crate anyhow to v1.0.93 (#14246) 2024-11-10 19:46:40 -05:00
Harutaka Kawamura
d4cf61d98b Implement shallow-copy-environ / W1507 (#14241)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

Related to #970. Implement [`shallow-copy-environ /
W1507`](https://pylint.readthedocs.io/en/stable/user_guide/messages/warning/shallow-copy-environ.html).

## Test Plan

<!-- How was it tested? -->

Unit test

---------

Co-authored-by: Simon Brugman <sbrugman@users.noreply.github.com>
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-11-10 22:58:02 +00:00
Randolf Scholz
5d91ba0b10 FBT001: exclude boolean operators (#14203)
Fixes #14202

## Summary

Exclude rule FBT001 for boolean operators.

## Test Plan

Updated existing `FBT.py` test.
2024-11-10 22:40:37 +00:00
Carl Meyer
a7e9f0c4b9 [red-knot] follow-ups to typevar types (#14232) 2024-11-09 20:18:32 -08:00
Charlie Marsh
c7d48e10e6 Detect empty implicit namespace packages (#14236)
## Summary

The implicit namespace package rule currently fails to detect cases like
the following:

```text
foo/
├── __init__.py
└── bar/
    └── baz/
        └── __init__.py
```

The problem is that we detect a root at `foo`, and then an independent
root at `baz`. We _would_ detect that `bar` is an implicit namespace
package, but it doesn't contain any files! So we never check it, and
have no place to raise the diagnostic.

This PR adds detection for these kinds of nested packages, and augments
the `INP` rule to flag the `__init__.py` file above with a specialized
message. As a side effect, I've introduced a dedicated `PackageRoot`
struct which we can pass around in lieu of Yet Another `Path`.

For now, I'm only enabling this in preview (and the approach doesn't
affect any other rules). It's a bug fix, but it may end up expanding the
rule.

Closes https://github.com/astral-sh/ruff/issues/13519.
2024-11-09 22:03:34 -05:00
Charlie Marsh
94dee2a36d Avoid applying PEP 646 rewrites in invalid contexts (#14234)
## Summary

Closes https://github.com/astral-sh/ruff/issues/14231.
2024-11-09 15:47:28 -05:00
Charlie Marsh
555a5c9319 [refurb] Avoid triggering hardcoded-string-charset for reordered sets (#14233)
## Summary

It's only safe to enforce the `x in "1234567890"` case if `x` is exactly
one character, since the set on the right has been reordered as compared
to `string.digits`. We can't know if `x` is exactly one character unless
it's a literal. And if it's a literal, well, it's kind of silly code in
the first place?

Closes https://github.com/astral-sh/ruff/issues/13802.
2024-11-09 15:31:26 -05:00
Charlie Marsh
1279c20ee1 Avoid using typing.Self in stub files pre-Python 3.11 (#14230)
## Summary

See:
https://github.com/astral-sh/ruff/pull/14217#discussion_r1835340869.

This means we're recommending `typing_extensions` in non-stubs pre-3.11,
which may not be a valid project dependency, but that's a separate issue
(https://github.com/astral-sh/ruff/issues/9761).
2024-11-09 13:17:36 -05:00
Charlie Marsh
ce3af27f59 Avoid treating lowercase letters as # noqa codes (#14229)
## Summary

An oversight from the original implementation.

Closes https://github.com/astral-sh/ruff/issues/14228.
2024-11-09 12:49:35 -05:00
Harutaka Kawamura
71da1d6df5 Fix await-outside-async to allow await at the top-level scope of a notebook (#14225)
## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

Fix `await-outside-async` to allow `await` at the top-level scope of a
notebook.

```python
# foo.ipynb

await asyncio.sleep(1)  # should be allowed
```

## Test Plan

<!-- How was it tested? -->

A unit test
2024-11-09 12:44:48 -05:00
Alex Waygood
e598240f04 [red-knot] More Type constructors (#14227) 2024-11-09 16:57:11 +00:00
InSync
c9b84e2a85 [ruff] Do not report when Optional has no type arguments (RUF013) (#14181)
## Summary

Resolves #13833.

## Test Plan

`cargo nextest run` and `cargo insta test`.

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-11-09 08:48:56 -05:00
Alex Waygood
d3f1c8e536 [red-knot] Add Type constructors for Instance, ClassLiteral and SubclassOf variants (#14215)
## Summary

Reduces some repetetiveness and verbosity at callsites. Addresses
@carljm's review comments at
https://github.com/astral-sh/ruff/pull/14155/files#r1833252458

## Test Plan

`cargo test -p red_knot_python_semantic`
2024-11-09 09:10:00 +00:00
InSync
eea6b31980 [flake8-pyi] Add "replace with Self" fix (PYI034) (#14217)
## Summary

Resolves #14184.

## Test Plan

`cargo nextest run` and `cargo insta test`.
2024-11-09 02:11:38 +00:00
Dylan
b8dc780bdc [refurb] Further special cases added to verbose-decimal-constructor (FURB157) (#14216)
This PR accounts for further subtleties in `Decimal` parsing:

- Strings which are empty modulo underscores and surrounding whitespace
are skipped
- `Decimal("-0")` is skipped
- `Decimal("{integer literal that is longer than 640 digits}")` are
skipped (see linked issue for explanation)

NB: The snapshot did not need to be updated since the new test cases are
"Ok" instances and added below the diff.

Closes #14204
2024-11-08 21:08:22 -05:00
Charlie Marsh
93fdf7ed36 Fix miscellaneous issues in await-outside-async detection (#14218)
## Summary

Closes https://github.com/astral-sh/ruff/issues/14167.
2024-11-08 21:07:13 -05:00
Michal Čihař
b19f388249 [refurb] Use UserString instead of non-existent UserStr (#14209)
## Summary

The class name is UserString, not a UserStr, see
https://docs.python.org/3.9/library/collections.html#collections.UserString
2024-11-08 20:54:18 -05:00
Alex Waygood
de947deee7 [red-knot] Consolidate detection of cyclically defined classes (#14207) 2024-11-08 22:17:56 +00:00
Carl Meyer
c0c4ae14ac [red-knot] make KnownClass::is_singleton a const fn (#14211)
Follow-up from missed review comment on
https://github.com/astral-sh/ruff/pull/14182
2024-11-08 13:37:25 -08:00
Carl Meyer
645ce7e5ec [red-knot] infer types for PEP695 typevars (#14182)
## Summary

Create definitions and infer types for PEP 695 type variables.

This just gives us the type of the type variable itself (the type of `T`
as a runtime object in the body of `def f[T](): ...`), with special
handling for its attributes `__name__`, `__bound__`, `__constraints__`,
and `__default__`. Mostly the support for these attributes exists
because it is easy to implement and allows testing that we are
internally representing the typevar correctly.

This PR doesn't yet have support for interpreting a typevar as a type
annotation, which is of course the primary use of a typevar. But the
information we store in the typevar's type in this PR gives us
everything we need to handle it correctly in a future PR when the
typevar appears in an annotation.

## Test Plan

Added mdtest.
2024-11-08 21:23:05 +00:00
David Peter
1430f21283 [red-knot] Fix is_disjoint_from for class literals (#14210)
## Summary

`Ty::BuiltinClassLiteral(…)` is a sub~~class~~type of
`Ty::BuiltinInstance("type")`, so it can't be disjoint from it.

## Test Plan

New `is_not_disjoint_from` test case
2024-11-08 20:54:27 +01:00
167 changed files with 6333 additions and 2454 deletions

View File

@@ -47,7 +47,7 @@ jobs:
working-directory: playground
- name: "Deploy to Cloudflare Pages"
if: ${{ env.CF_API_TOKEN_EXISTS == 'true' }}
uses: cloudflare/wrangler-action@v3.11.0
uses: cloudflare/wrangler-action@v3.12.1
with:
apiToken: ${{ secrets.CF_API_TOKEN }}
accountId: ${{ secrets.CF_ACCOUNT_ID }}

View File

@@ -53,13 +53,13 @@ repos:
files: '^crates/.*/resources/mdtest/.*\.md'
exclude: |
(?x)^(
.*?invalid(_.+)_syntax.md
.*?invalid(_.+)*_syntax\.md
)$
additional_dependencies:
- black==24.10.0
- repo: https://github.com/crate-ci/typos
rev: v1.27.0
rev: v1.27.3
hooks:
- id: typos
@@ -73,7 +73,7 @@ repos:
pass_filenames: false # This makes it a lot faster
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.7.2
rev: v0.7.3
hooks:
- id: ruff-format
- id: ruff

360
Cargo.lock generated
View File

@@ -2,12 +2,6 @@
# It is not intended for manual editing.
version = 3
[[package]]
name = "Inflector"
version = "0.11.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fe438c63458706e03479442743baae6c88256498e6431708f6dfc520a26515d3"
[[package]]
name = "adler"
version = "1.0.2"
@@ -123,9 +117,9 @@ dependencies = [
[[package]]
name = "anyhow"
version = "1.0.92"
version = "1.0.93"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "74f37166d7d48a0284b99dd824694c26119c700b53bf0d1540cdb147dbdaaf13"
checksum = "4c95c10ba0b00a02636238b814946408b1322d5ac4760326e6fb8ec956d85775"
[[package]]
name = "append-only-vec"
@@ -424,7 +418,7 @@ checksum = "2f8c93eb5f77c9050c7750e14f13ef1033a40a0aac70c6371535b6763a01438c"
dependencies = [
"nix 0.28.0",
"terminfo",
"thiserror",
"thiserror 1.0.67",
"which",
"winapi",
]
@@ -812,6 +806,17 @@ dependencies = [
"windows-sys 0.48.0",
]
[[package]]
name = "displaydoc"
version = "0.2.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "97369cbbc041bc366949bc74d34658d6cda5621039731c6310521892a3a20ae0"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.87",
]
[[package]]
name = "doc-comment"
version = "0.3.3"
@@ -1045,9 +1050,9 @@ dependencies = [
[[package]]
name = "hashbrown"
version = "0.15.0"
version = "0.15.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e087f84d4f86bf4b218b927129862374b72199ae7d8657835f1e89000eea4fb"
checksum = "3a9bfc1af68b1726ea47d3d5109de126281def866b33970e10fbab11b5dafab3"
[[package]]
name = "hashlink"
@@ -1108,6 +1113,124 @@ dependencies = [
"cc",
]
[[package]]
name = "icu_collections"
version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "db2fa452206ebee18c4b5c2274dbf1de17008e874b4dc4f0aea9d01ca79e4526"
dependencies = [
"displaydoc",
"yoke",
"zerofrom",
"zerovec",
]
[[package]]
name = "icu_locid"
version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "13acbb8371917fc971be86fc8057c41a64b521c184808a698c02acc242dbf637"
dependencies = [
"displaydoc",
"litemap",
"tinystr",
"writeable",
"zerovec",
]
[[package]]
name = "icu_locid_transform"
version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "01d11ac35de8e40fdeda00d9e1e9d92525f3f9d887cdd7aa81d727596788b54e"
dependencies = [
"displaydoc",
"icu_locid",
"icu_locid_transform_data",
"icu_provider",
"tinystr",
"zerovec",
]
[[package]]
name = "icu_locid_transform_data"
version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fdc8ff3388f852bede6b579ad4e978ab004f139284d7b28715f773507b946f6e"
[[package]]
name = "icu_normalizer"
version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "19ce3e0da2ec68599d193c93d088142efd7f9c5d6fc9b803774855747dc6a84f"
dependencies = [
"displaydoc",
"icu_collections",
"icu_normalizer_data",
"icu_properties",
"icu_provider",
"smallvec",
"utf16_iter",
"utf8_iter",
"write16",
"zerovec",
]
[[package]]
name = "icu_normalizer_data"
version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8cafbf7aa791e9b22bec55a167906f9e1215fd475cd22adfcf660e03e989516"
[[package]]
name = "icu_properties"
version = "1.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "93d6020766cfc6302c15dbbc9c8778c37e62c14427cb7f6e601d849e092aeef5"
dependencies = [
"displaydoc",
"icu_collections",
"icu_locid_transform",
"icu_properties_data",
"icu_provider",
"tinystr",
"zerovec",
]
[[package]]
name = "icu_properties_data"
version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "67a8effbc3dd3e4ba1afa8ad918d5684b8868b3b26500753effea8d2eed19569"
[[package]]
name = "icu_provider"
version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6ed421c8a8ef78d3e2dbc98a973be2f3770cb42b606e3ab18d6237c4dfde68d9"
dependencies = [
"displaydoc",
"icu_locid",
"icu_provider_macros",
"stable_deref_trait",
"tinystr",
"writeable",
"yoke",
"zerofrom",
"zerovec",
]
[[package]]
name = "icu_provider_macros"
version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1ec89e9337638ecdc08744df490b221a7399bf8d164eb52a665454e60e075ad6"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.87",
]
[[package]]
name = "ident_case"
version = "1.0.1"
@@ -1116,12 +1239,23 @@ checksum = "b9e0384b61958566e926dc50660321d12159025e767c18e043daf26b70104c39"
[[package]]
name = "idna"
version = "0.5.0"
version = "1.0.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "634d9b1461af396cad843f47fdba5597a4f9e6ddd4bfb6ff5d85028c25cb12f6"
checksum = "686f825264d630750a544639377bae737628043f20d38bbc029e8f29ea968a7e"
dependencies = [
"unicode-bidi",
"unicode-normalization",
"idna_adapter",
"smallvec",
"utf8_iter",
]
[[package]]
name = "idna_adapter"
version = "1.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "daca1df1c957320b2cf139ac61e7bd64fed304c5040df000a745aa1de3b4ef71"
dependencies = [
"icu_normalizer",
"icu_properties",
]
[[package]]
@@ -1167,7 +1301,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "707907fe3c25f5424cce2cb7e1cbcafee6bdbe735ca90ef77c29e84591e5b9da"
dependencies = [
"equivalent",
"hashbrown 0.15.0",
"hashbrown 0.15.1",
"serde",
]
@@ -1260,11 +1394,11 @@ dependencies = [
[[package]]
name = "is-macro"
version = "0.3.6"
version = "0.3.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2069faacbe981460232f880d26bf3c7634e322d49053aa48c27e3ae642f728f1"
checksum = "1d57a3e447e24c22647738e4607f1df1e0ec6f72e16182c4cd199f647cdfb0e4"
dependencies = [
"Inflector",
"heck",
"proc-macro2",
"quote",
"syn 2.0.87",
@@ -1367,9 +1501,9 @@ checksum = "e2abad23fbc42b3700f2f279844dc832adb2b2eb069b2df918f455c4e18cc646"
[[package]]
name = "libc"
version = "0.2.161"
version = "0.2.162"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8e9489c2807c139ffd9c1794f4af0ebe86a828db53ecdc7fea2111d0fed085d1"
checksum = "18d287de67fe55fd7e1581fe933d965a5a9477b38e949cfa9f8574ef01506398"
[[package]]
name = "libcst"
@@ -1383,7 +1517,7 @@ dependencies = [
"paste",
"peg",
"regex",
"thiserror",
"thiserror 1.0.67",
]
[[package]]
@@ -1429,6 +1563,12 @@ version = "0.4.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "78b3ae25bc7c8c38cec158d1f2757ee79e9b3740fbc7ccf0e59e4b08d793fa89"
[[package]]
name = "litemap"
version = "0.7.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "643cb0b8d4fcc284004d5fd0d67ccf61dfffadb7f75e1e71bc420f4688a3a704"
[[package]]
name = "lock_api"
version = "0.4.11"
@@ -1486,9 +1626,9 @@ checksum = "2532096657941c2fea9c289d370a250971c689d4f143798ff67113ec042024a5"
[[package]]
name = "matchit"
version = "0.8.4"
version = "0.8.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "47e1ffaa40ddd1f3ed91f717a33c8c0ee23fff369e3aa8772b9605cc1d22f4c3"
checksum = "bd0aa4b8ca861b08d68afc8702af3250776898c1508b278e1da9d01e01d4b45c"
[[package]]
name = "memchr"
@@ -1815,7 +1955,7 @@ dependencies = [
"pep440_rs 0.4.0",
"regex",
"serde",
"thiserror",
"thiserror 1.0.67",
"tracing",
"unicode-width 0.1.13",
"url",
@@ -1834,7 +1974,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cd53dff83f26735fdc1ca837098ccf133605d794cdae66acfc2bfac3ec809d95"
dependencies = [
"memchr",
"thiserror",
"thiserror 1.0.67",
"ucd-trie",
]
@@ -2004,7 +2144,7 @@ dependencies = [
"newtype-uuid",
"quick-xml",
"strip-ansi-escapes",
"thiserror",
"thiserror 1.0.67",
"uuid",
]
@@ -2111,7 +2251,7 @@ dependencies = [
"compact_str",
"countme",
"dir-test",
"hashbrown 0.15.0",
"hashbrown 0.15.1",
"indexmap",
"insta",
"itertools 0.13.0",
@@ -2133,7 +2273,7 @@ dependencies = [
"static_assertions",
"tempfile",
"test-case",
"thiserror",
"thiserror 2.0.3",
"tracing",
]
@@ -2253,7 +2393,7 @@ checksum = "bd283d9651eeda4b2a83a43c1c91b266c40fd76ecd39a50a8c630ae69dc72891"
dependencies = [
"getrandom",
"libredox",
"thiserror",
"thiserror 1.0.67",
]
[[package]]
@@ -2364,7 +2504,7 @@ dependencies = [
"strum",
"tempfile",
"test-case",
"thiserror",
"thiserror 2.0.3",
"tikv-jemallocator",
"toml",
"tracing",
@@ -2432,7 +2572,7 @@ dependencies = [
"salsa",
"serde",
"tempfile",
"thiserror",
"thiserror 2.0.3",
"tracing",
"tracing-subscriber",
"tracing-tree",
@@ -2584,7 +2724,7 @@ dependencies = [
"strum",
"strum_macros",
"test-case",
"thiserror",
"thiserror 2.0.3",
"toml",
"typed-arena",
"unicode-normalization",
@@ -2618,7 +2758,7 @@ dependencies = [
"serde_json",
"serde_with",
"test-case",
"thiserror",
"thiserror 2.0.3",
"uuid",
]
@@ -2691,7 +2831,7 @@ dependencies = [
"similar",
"smallvec",
"static_assertions",
"thiserror",
"thiserror 2.0.3",
"tracing",
]
@@ -2771,6 +2911,7 @@ dependencies = [
name = "ruff_python_stdlib"
version = "0.0.0"
dependencies = [
"bitflags 2.6.0",
"unicode-ident",
]
@@ -2823,7 +2964,7 @@ dependencies = [
"serde",
"serde_json",
"shellexpand",
"thiserror",
"thiserror 2.0.3",
"tracing",
"tracing-subscriber",
]
@@ -2932,9 +3073,9 @@ checksum = "583034fd73374156e66797ed8e5b0d5690409c9226b22d87cb7f19821c05d152"
[[package]]
name = "rustix"
version = "0.38.37"
version = "0.38.40"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8acb788b847c24f28525660c4d7758620a7210875711f79e7f663cc152726811"
checksum = "99e4ea3e1cdc4b559b8e5650f9c8e5998e3e5c1343b4eaf034565f32318d63c0"
dependencies = [
"bitflags 2.6.0",
"errno",
@@ -3234,6 +3375,12 @@ version = "0.9.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6980e8d7511241f8acf4aebddbb1ff938df5eebe98691418c4468d0b72a96a67"
[[package]]
name = "stable_deref_trait"
version = "1.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a8f112729512f8e442d81f95a8a7ddf2b7c6b8a1a6f509a95864142b30cab2d3"
[[package]]
name = "static_assertions"
version = "1.1.0"
@@ -3324,9 +3471,9 @@ dependencies = [
[[package]]
name = "tempfile"
version = "3.13.0"
version = "3.14.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f0f2c9fc62d0beef6951ccffd757e241266a2c833136efbe35af6cd2567dca5b"
checksum = "28cce251fcbc87fac86a866eeb0d6c2d536fc16d06f184bb61aeae11aa4cee0c"
dependencies = [
"cfg-if",
"fastrand",
@@ -3403,7 +3550,16 @@ version = "1.0.67"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3b3c6efbfc763e64eb85c11c25320f0737cb7364c4b6336db90aa9ebe27a0bbd"
dependencies = [
"thiserror-impl",
"thiserror-impl 1.0.67",
]
[[package]]
name = "thiserror"
version = "2.0.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c006c85c7651b3cf2ada4584faa36773bd07bac24acfb39f3c431b36d7e667aa"
dependencies = [
"thiserror-impl 2.0.3",
]
[[package]]
@@ -3417,6 +3573,17 @@ dependencies = [
"syn 2.0.87",
]
[[package]]
name = "thiserror-impl"
version = "2.0.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f077553d607adc1caf65430528a576c757a71ed73944b66ebb58ef2bbd243568"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.87",
]
[[package]]
name = "thread_local"
version = "1.1.8"
@@ -3447,6 +3614,16 @@ dependencies = [
"tikv-jemalloc-sys",
]
[[package]]
name = "tinystr"
version = "0.7.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9117f5d4db391c1cf6927e7bea3db74b9a1c1add8f7eda9ffd5364f40f57b82f"
dependencies = [
"displaydoc",
"zerovec",
]
[[package]]
name = "tinytemplate"
version = "1.2.1"
@@ -3663,12 +3840,6 @@ dependencies = [
"unic-common",
]
[[package]]
name = "unicode-bidi"
version = "0.3.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "08f95100a766bf4f8f28f90d77e0a5461bbdb219042e7679bebe79004fed8d75"
[[package]]
name = "unicode-ident"
version = "1.0.13"
@@ -3748,9 +3919,9 @@ dependencies = [
[[package]]
name = "url"
version = "2.5.2"
version = "2.5.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "22784dbdf76fdde8af1aeda5622b546b422b6fc585325248a2bf9f5e41e94d6c"
checksum = "8d157f1b96d14500ffdc1f10ba712e780825526c03d9a49b4d0324b0d9113ada"
dependencies = [
"form_urlencoded",
"idna",
@@ -3758,6 +3929,18 @@ dependencies = [
"serde",
]
[[package]]
name = "utf16_iter"
version = "1.0.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c8232dd3cdaed5356e0f716d285e4b40b932ac434100fe9b7e0e8e935b9e6246"
[[package]]
name = "utf8_iter"
version = "1.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b6c140620e7ffbb22c2dee59cafe6084a59b5ffc27a8859a5f0d494b5d52b6be"
[[package]]
name = "utf8parse"
version = "0.2.1"
@@ -4194,6 +4377,18 @@ version = "0.0.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d135d17ab770252ad95e9a872d365cf3090e3be864a34ab46f48555993efc904"
[[package]]
name = "write16"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d1890f4022759daae28ed4fe62859b1236caebfc61ede2f63ed4e695f3f6d936"
[[package]]
name = "writeable"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e9df38ee2d2c3c5948ea468a8406ff0db0b29ae1ffde1bcf20ef305bcc95c51"
[[package]]
name = "yansi"
version = "1.0.1"
@@ -4209,6 +4404,30 @@ dependencies = [
"winapi",
]
[[package]]
name = "yoke"
version = "0.7.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6c5b1314b079b0930c31e3af543d8ee1757b1951ae1e1565ec704403a7240ca5"
dependencies = [
"serde",
"stable_deref_trait",
"yoke-derive",
"zerofrom",
]
[[package]]
name = "yoke-derive"
version = "0.7.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "28cc31741b18cb6f1d5ff12f5b7523e3d6eb0852bbbad19d73905511d9849b95"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.87",
"synstructure",
]
[[package]]
name = "zerocopy"
version = "0.7.32"
@@ -4229,12 +4448,55 @@ dependencies = [
"syn 2.0.87",
]
[[package]]
name = "zerofrom"
version = "0.1.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "91ec111ce797d0e0784a1116d0ddcdbea84322cd79e5d5ad173daeba4f93ab55"
dependencies = [
"zerofrom-derive",
]
[[package]]
name = "zerofrom-derive"
version = "0.1.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0ea7b4a3637ea8669cedf0f1fd5c286a17f3de97b8dd5a70a6c167a1730e63a5"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.87",
"synstructure",
]
[[package]]
name = "zeroize"
version = "1.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "525b4ec142c6b68a2d10f01f7bbf6755599ca3f81ea53b8431b7dd348f5fdb2d"
[[package]]
name = "zerovec"
version = "0.10.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "aa2b893d79df23bfb12d5461018d408ea19dfafe76c2c7ef6d4eba614f8ff079"
dependencies = [
"yoke",
"zerofrom",
"zerovec-derive",
]
[[package]]
name = "zerovec-derive"
version = "0.10.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6eafa6dfb17584ea3e2bd6e76e0cc15ad7af12b09abdd1ca55961bed9b1063c6"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.87",
]
[[package]]
name = "zip"
version = "0.6.6"

View File

@@ -136,7 +136,7 @@ strum_macros = { version = "0.26.0" }
syn = { version = "2.0.55" }
tempfile = { version = "3.9.0" }
test-case = { version = "3.3.1" }
thiserror = { version = "1.0.58" }
thiserror = { version = "2.0.0" }
tikv-jemallocator = { version = "0.6.0" }
toml = { version = "0.8.11" }
tracing = { version = "0.1.40" }

View File

@@ -35,6 +35,7 @@ class C:
if flag:
x = 2
# error: [possibly-unbound-attribute] "Attribute `x` on type `Literal[C]` is possibly unbound"
reveal_type(C.x) # revealed: Literal[2]
reveal_type(C.y) # revealed: Literal[1]
```

View File

@@ -9,14 +9,21 @@ def bool_instance() -> bool:
flag = bool_instance()
if flag:
class C:
class C1:
x = 1
else:
class C:
class C1:
x = 2
reveal_type(C.x) # revealed: Literal[1, 2]
class C2:
if flag:
x = 3
else:
x = 4
reveal_type(C1.x) # revealed: Literal[1, 2]
reveal_type(C2.x) # revealed: Literal[3, 4]
```
## Inherited attributes
@@ -53,3 +60,77 @@ reveal_type(A.__mro__)
# `E` is earlier in the MRO than `F`, so we should use the type of `E.X`
reveal_type(A.X) # revealed: Literal[42]
```
## Unions with possibly unbound paths
### Definite boundness within a class
In this example, the `x` attribute is not defined in the `C2` element of the union:
```py
def bool_instance() -> bool:
return True
class C1:
x = 1
class C2: ...
class C3:
x = 3
flag1 = bool_instance()
flag2 = bool_instance()
C = C1 if flag1 else C2 if flag2 else C3
# error: [possibly-unbound-attribute] "Attribute `x` on type `Literal[C1, C2, C3]` is possibly unbound"
reveal_type(C.x) # revealed: Literal[1, 3]
```
### Possibly-unbound within a class
We raise the same diagnostic if the attribute is possibly-unbound in at least one element of the
union:
```py
def bool_instance() -> bool:
return True
class C1:
x = 1
class C2:
if bool_instance():
x = 2
class C3:
x = 3
flag1 = bool_instance()
flag2 = bool_instance()
C = C1 if flag1 else C2 if flag2 else C3
# error: [possibly-unbound-attribute] "Attribute `x` on type `Literal[C1, C2, C3]` is possibly unbound"
reveal_type(C.x) # revealed: Literal[1, 2, 3]
```
## Unions with all paths unbound
If the symbol is unbound in all elements of the union, we detect that:
```py
def bool_instance() -> bool:
return True
class C1: ...
class C2: ...
flag = bool_instance()
C = C1 if flag else C2
# error: [unresolved-attribute] "Type `Literal[C1, C2]` has no attribute `x`"
reveal_type(C.x) # revealed: Unknown
```

View File

@@ -0,0 +1,28 @@
# Attribute access
## Boundness
```py
def flag() -> bool: ...
class A:
always_bound = 1
if flag():
union = 1
else:
union = "abc"
if flag():
possibly_unbound = "abc"
reveal_type(A.always_bound) # revealed: Literal[1]
reveal_type(A.union) # revealed: Literal[1] | Literal["abc"]
# error: [possibly-unbound-attribute] "Attribute `possibly_unbound` on type `Literal[A]` is possibly unbound"
reveal_type(A.possibly_unbound) # revealed: Literal["abc"]
# error: [unresolved-attribute] "Type `Literal[A]` has no attribute `non_existent`"
reveal_type(A.non_existent) # revealed: Unknown
```

View File

@@ -6,13 +6,9 @@ Basic PEP 695 generics
```py
class MyBox[T]:
# TODO: `T` is defined here
# error: [unresolved-reference] "Name `T` used when not defined"
data: T
box_model_number = 695
# TODO: `T` is defined here
# error: [unresolved-reference] "Name `T` used when not defined"
def __init__(self, data: T):
self.data = data
@@ -31,17 +27,12 @@ reveal_type(MyBox.box_model_number) # revealed: Literal[695]
```py
class MyBox[T]:
# TODO: `T` is defined here
# error: [unresolved-reference] "Name `T` used when not defined"
data: T
# TODO: `T` is defined here
# error: [unresolved-reference] "Name `T` used when not defined"
def __init__(self, data: T):
self.data = data
# TODO not error on the subscripting or the use of type param
# error: [unresolved-reference] "Name `T` used when not defined"
# TODO not error on the subscripting
# error: [non-subscriptable]
class MySecureBox[T](MyBox[T]): ...
@@ -66,3 +57,55 @@ class S[T](Seq[S]): ... # error: [non-subscriptable]
reveal_type(S) # revealed: Literal[S]
```
## Type params
A PEP695 type variable defines a value of type `typing.TypeVar` with attributes `__name__`,
`__bounds__`, `__constraints__`, and `__default__` (the latter three all lazily evaluated):
```py
def f[T, U: A, V: (A, B), W = A, X: A = A1]():
reveal_type(T) # revealed: TypeVar
reveal_type(T.__name__) # revealed: Literal["T"]
reveal_type(T.__bound__) # revealed: None
reveal_type(T.__constraints__) # revealed: tuple[()]
reveal_type(T.__default__) # revealed: NoDefault
reveal_type(U) # revealed: TypeVar
reveal_type(U.__name__) # revealed: Literal["U"]
reveal_type(U.__bound__) # revealed: type[A]
reveal_type(U.__constraints__) # revealed: tuple[()]
reveal_type(U.__default__) # revealed: NoDefault
reveal_type(V) # revealed: TypeVar
reveal_type(V.__name__) # revealed: Literal["V"]
reveal_type(V.__bound__) # revealed: None
reveal_type(V.__constraints__) # revealed: tuple[type[A], type[B]]
reveal_type(V.__default__) # revealed: NoDefault
reveal_type(W) # revealed: TypeVar
reveal_type(W.__name__) # revealed: Literal["W"]
reveal_type(W.__bound__) # revealed: None
reveal_type(W.__constraints__) # revealed: tuple[()]
reveal_type(W.__default__) # revealed: type[A]
reveal_type(X) # revealed: TypeVar
reveal_type(X.__name__) # revealed: Literal["X"]
reveal_type(X.__bound__) # revealed: type[A]
reveal_type(X.__constraints__) # revealed: tuple[()]
reveal_type(X.__default__) # revealed: type[A1]
class A: ...
class B: ...
class A1(A): ...
```
## Minimum two constraints
A typevar with less than two constraints emits a diagnostic and is treated as unconstrained:
```py
# error: [invalid-typevar-constraints] "TypeVar must have at least two constrained types"
def f[T: (int,)]():
reveal_type(T.__constraints__) # revealed: tuple[()]
```

View File

@@ -21,6 +21,7 @@ reveal_type(y)
```
```py
# error: [possibly-unbound-import] "Member `y` of module `maybe_unbound` is possibly unbound"
from maybe_unbound import x, y
reveal_type(x) # revealed: Literal[3]
@@ -50,6 +51,7 @@ reveal_type(y)
Importing an annotated name prefers the declared type over the inferred type:
```py
# error: [possibly-unbound-import] "Member `y` of module `maybe_unbound_annotated` is possibly unbound"
from maybe_unbound_annotated import x, y
reveal_type(x) # revealed: Literal[3]

View File

@@ -102,6 +102,9 @@ else:
### Handling of `None`
```py
# TODO: this error should ideally go away once we (1) understand `sys.version_info` branches,
# and (2) set the target Python version for this test to 3.10.
# error: [possibly-unbound-import] "Member `NoneType` of module `types` is possibly unbound"
from types import NoneType
def flag() -> bool: ...

View File

@@ -74,6 +74,7 @@ we're dealing with:
```py path=__getattr__.py
import typing
# error: [unresolved-attribute]
reveal_type(typing.__getattr__) # revealed: Unknown
```

View File

@@ -0,0 +1,137 @@
# `sys.version_info`
## The type of `sys.version_info`
The type of `sys.version_info` is `sys._version_info`, at least according to typeshed's stubs (which
we treat as the single source of truth for the standard library). This is quite a complicated type
in typeshed, so there are many things we don't fully understand about the type yet; this is the
source of several TODOs in this test file. Many of these TODOs should be naturally fixed as we
implement more type-system features in the future.
```py
import sys
reveal_type(sys.version_info) # revealed: _version_info
```
## Literal types from comparisons
Comparing `sys.version_info` with a 2-element tuple of literal integers always produces a `Literal`
type:
```py
import sys
reveal_type(sys.version_info >= (3, 8)) # revealed: Literal[True]
reveal_type((3, 8) <= sys.version_info) # revealed: Literal[True]
reveal_type(sys.version_info > (3, 8)) # revealed: Literal[True]
reveal_type((3, 8) < sys.version_info) # revealed: Literal[True]
reveal_type(sys.version_info < (3, 8)) # revealed: Literal[False]
reveal_type((3, 8) > sys.version_info) # revealed: Literal[False]
reveal_type(sys.version_info <= (3, 8)) # revealed: Literal[False]
reveal_type((3, 8) >= sys.version_info) # revealed: Literal[False]
reveal_type(sys.version_info == (3, 8)) # revealed: Literal[False]
reveal_type((3, 8) == sys.version_info) # revealed: Literal[False]
reveal_type(sys.version_info != (3, 8)) # revealed: Literal[True]
reveal_type((3, 8) != sys.version_info) # revealed: Literal[True]
```
## Non-literal types from comparisons
Comparing `sys.version_info` with tuples of other lengths will sometimes produce `Literal` types,
sometimes not:
```py
import sys
reveal_type(sys.version_info >= (3, 8, 1)) # revealed: bool
reveal_type(sys.version_info >= (3, 8, 1, "final", 0)) # revealed: bool
# TODO: this is an invalid comparison (`sys.version_info` is a tuple of length 5)
# Should we issue a diagnostic here?
reveal_type(sys.version_info >= (3, 8, 1, "final", 0, 5)) # revealed: bool
# TODO: this should be `Literal[False]`; see #14279
reveal_type(sys.version_info == (3, 8, 1, "finallllll", 0)) # revealed: bool
```
## Imports and aliases
Comparisons with `sys.version_info` still produce literal types, even if the symbol is aliased to
another name:
```py
from sys import version_info
from sys import version_info as foo
reveal_type(version_info >= (3, 8)) # revealed: Literal[True]
reveal_type(foo >= (3, 8)) # revealed: Literal[True]
bar = version_info
reveal_type(bar >= (3, 8)) # revealed: Literal[True]
```
## Non-stdlib modules named `sys`
Only comparisons with the symbol `version_info` from the `sys` module produce literal types:
```py path=package/__init__.py
```
```py path=package/sys.py
version_info: tuple[int, int] = (4, 2)
```
```py path=package/script.py
from .sys import version_info
reveal_type(version_info >= (3, 8)) # revealed: bool
```
## Accessing fields by name
The fields of `sys.version_info` can be accessed by name:
```py path=a.py
import sys
reveal_type(sys.version_info.major >= 3) # revealed: Literal[True]
reveal_type(sys.version_info.minor >= 8) # revealed: Literal[True]
reveal_type(sys.version_info.minor >= 9) # revealed: Literal[False]
```
But the `micro`, `releaselevel` and `serial` fields are inferred as `@Todo` until we support
properties on instance types:
```py path=b.py
import sys
reveal_type(sys.version_info.micro) # revealed: @Todo
reveal_type(sys.version_info.releaselevel) # revealed: @Todo
reveal_type(sys.version_info.serial) # revealed: @Todo
```
## Accessing fields by index/slice
The fields of `sys.version_info` can be accessed by index or by slice:
```py
import sys
reveal_type(sys.version_info[0] < 3) # revealed: Literal[False]
reveal_type(sys.version_info[1] > 8) # revealed: Literal[False]
# revealed: tuple[Literal[3], Literal[8], int, Literal["alpha", "beta", "candidate", "final"], int]
reveal_type(sys.version_info[:5])
reveal_type(sys.version_info[:2] >= (3, 8)) # revealed: Literal[True]
reveal_type(sys.version_info[0:2] >= (3, 9)) # revealed: Literal[False]
reveal_type(sys.version_info[:3] >= (3, 9, 1)) # revealed: Literal[False]
reveal_type(sys.version_info[3] == "final") # revealed: bool
reveal_type(sys.version_info[3] == "finalllllll") # revealed: Literal[False]
```

View File

@@ -10,8 +10,6 @@ reveal_type(not not None) # revealed: Literal[False]
## Function
```py
from typing import reveal_type
def f():
return 1

View File

@@ -373,6 +373,11 @@ impl<'db> SemanticIndexBuilder<'db> {
if let Some(default) = default {
self.visit_expr(default);
}
match type_param {
ast::TypeParam::TypeVar(node) => self.add_definition(symbol, node),
ast::TypeParam::ParamSpec(node) => self.add_definition(symbol, node),
ast::TypeParam::TypeVarTuple(node) => self.add_definition(symbol, node),
};
}
}
@@ -584,6 +589,27 @@ where
},
);
}
ast::Stmt::TypeAlias(type_alias) => {
let symbol = self.add_symbol(
type_alias
.name
.as_name_expr()
.expect("type alias name is a name expr")
.id
.clone(),
);
self.add_definition(symbol, type_alias);
self.with_type_params(
NodeWithScopeRef::TypeAliasTypeParameters(type_alias),
type_alias.type_params.as_ref(),
|builder| {
builder.push_scope(NodeWithScopeRef::TypeAliasTypeParameters(type_alias));
builder.visit_expr(&type_alias.value);
builder.pop_scope()
},
);
}
ast::Stmt::Import(node) => {
for alias in &node.names {
let symbol_name = if let Some(asname) = &alias.asname {

View File

@@ -83,6 +83,7 @@ pub(crate) enum DefinitionNodeRef<'a> {
For(ForStmtDefinitionNodeRef<'a>),
Function(&'a ast::StmtFunctionDef),
Class(&'a ast::StmtClassDef),
TypeAlias(&'a ast::StmtTypeAlias),
NamedExpression(&'a ast::ExprNamed),
Assignment(AssignmentDefinitionNodeRef<'a>),
AnnotatedAssignment(&'a ast::StmtAnnAssign),
@@ -92,6 +93,9 @@ pub(crate) enum DefinitionNodeRef<'a> {
WithItem(WithItemDefinitionNodeRef<'a>),
MatchPattern(MatchPatternDefinitionNodeRef<'a>),
ExceptHandler(ExceptHandlerDefinitionNodeRef<'a>),
TypeVar(&'a ast::TypeParamTypeVar),
ParamSpec(&'a ast::TypeParamParamSpec),
TypeVarTuple(&'a ast::TypeParamTypeVarTuple),
}
impl<'a> From<&'a ast::StmtFunctionDef> for DefinitionNodeRef<'a> {
@@ -106,6 +110,12 @@ impl<'a> From<&'a ast::StmtClassDef> for DefinitionNodeRef<'a> {
}
}
impl<'a> From<&'a ast::StmtTypeAlias> for DefinitionNodeRef<'a> {
fn from(node: &'a ast::StmtTypeAlias) -> Self {
Self::TypeAlias(node)
}
}
impl<'a> From<&'a ast::ExprNamed> for DefinitionNodeRef<'a> {
fn from(node: &'a ast::ExprNamed) -> Self {
Self::NamedExpression(node)
@@ -130,6 +140,24 @@ impl<'a> From<&'a ast::Alias> for DefinitionNodeRef<'a> {
}
}
impl<'a> From<&'a ast::TypeParamTypeVar> for DefinitionNodeRef<'a> {
fn from(value: &'a ast::TypeParamTypeVar) -> Self {
Self::TypeVar(value)
}
}
impl<'a> From<&'a ast::TypeParamParamSpec> for DefinitionNodeRef<'a> {
fn from(value: &'a ast::TypeParamParamSpec) -> Self {
Self::ParamSpec(value)
}
}
impl<'a> From<&'a ast::TypeParamTypeVarTuple> for DefinitionNodeRef<'a> {
fn from(value: &'a ast::TypeParamTypeVarTuple) -> Self {
Self::TypeVarTuple(value)
}
}
impl<'a> From<ImportFromDefinitionNodeRef<'a>> for DefinitionNodeRef<'a> {
fn from(node_ref: ImportFromDefinitionNodeRef<'a>) -> Self {
Self::ImportFrom(node_ref)
@@ -244,6 +272,9 @@ impl<'db> DefinitionNodeRef<'db> {
DefinitionNodeRef::Class(class) => {
DefinitionKind::Class(AstNodeRef::new(parsed, class))
}
DefinitionNodeRef::TypeAlias(type_alias) => {
DefinitionKind::TypeAlias(AstNodeRef::new(parsed, type_alias))
}
DefinitionNodeRef::NamedExpression(named) => {
DefinitionKind::NamedExpression(AstNodeRef::new(parsed, named))
}
@@ -317,6 +348,15 @@ impl<'db> DefinitionNodeRef<'db> {
handler: AstNodeRef::new(parsed, handler),
is_star,
}),
DefinitionNodeRef::TypeVar(node) => {
DefinitionKind::TypeVar(AstNodeRef::new(parsed, node))
}
DefinitionNodeRef::ParamSpec(node) => {
DefinitionKind::ParamSpec(AstNodeRef::new(parsed, node))
}
DefinitionNodeRef::TypeVarTuple(node) => {
DefinitionKind::TypeVarTuple(AstNodeRef::new(parsed, node))
}
}
}
@@ -328,6 +368,7 @@ impl<'db> DefinitionNodeRef<'db> {
}
Self::Function(node) => node.into(),
Self::Class(node) => node.into(),
Self::TypeAlias(node) => node.into(),
Self::NamedExpression(node) => node.into(),
Self::Assignment(AssignmentDefinitionNodeRef {
value: _,
@@ -356,6 +397,9 @@ impl<'db> DefinitionNodeRef<'db> {
identifier.into()
}
Self::ExceptHandler(ExceptHandlerDefinitionNodeRef { handler, .. }) => handler.into(),
Self::TypeVar(node) => node.into(),
Self::ParamSpec(node) => node.into(),
Self::TypeVarTuple(node) => node.into(),
}
}
}
@@ -401,6 +445,7 @@ pub enum DefinitionKind<'db> {
ImportFrom(ImportFromDefinitionKind),
Function(AstNodeRef<ast::StmtFunctionDef>),
Class(AstNodeRef<ast::StmtClassDef>),
TypeAlias(AstNodeRef<ast::StmtTypeAlias>),
NamedExpression(AstNodeRef<ast::ExprNamed>),
Assignment(AssignmentDefinitionKind<'db>),
AnnotatedAssignment(AstNodeRef<ast::StmtAnnAssign>),
@@ -412,6 +457,9 @@ pub enum DefinitionKind<'db> {
WithItem(WithItemDefinitionKind),
MatchPattern(MatchPatternDefinitionKind),
ExceptHandler(ExceptHandlerDefinitionKind),
TypeVar(AstNodeRef<ast::TypeParamTypeVar>),
ParamSpec(AstNodeRef<ast::TypeParamParamSpec>),
TypeVarTuple(AstNodeRef<ast::TypeParamTypeVarTuple>),
}
impl DefinitionKind<'_> {
@@ -420,8 +468,12 @@ impl DefinitionKind<'_> {
// functions, classes, and imports always bind, and we consider them declarations
DefinitionKind::Function(_)
| DefinitionKind::Class(_)
| DefinitionKind::TypeAlias(_)
| DefinitionKind::Import(_)
| DefinitionKind::ImportFrom(_) => DefinitionCategory::DeclarationAndBinding,
| DefinitionKind::ImportFrom(_)
| DefinitionKind::TypeVar(_)
| DefinitionKind::ParamSpec(_)
| DefinitionKind::TypeVarTuple(_) => DefinitionCategory::DeclarationAndBinding,
// a parameter always binds a value, but is only a declaration if annotated
DefinitionKind::Parameter(parameter) => {
if parameter.annotation.is_some() {
@@ -643,6 +695,12 @@ impl From<&ast::StmtClassDef> for DefinitionNodeKey {
}
}
impl From<&ast::StmtTypeAlias> for DefinitionNodeKey {
fn from(node: &ast::StmtTypeAlias) -> Self {
Self(NodeKey::from_node(node))
}
}
impl From<&ast::ExprName> for DefinitionNodeKey {
fn from(node: &ast::ExprName) -> Self {
Self(NodeKey::from_node(node))
@@ -696,3 +754,21 @@ impl From<&ast::ExceptHandlerExceptHandler> for DefinitionNodeKey {
Self(NodeKey::from_node(handler))
}
}
impl From<&ast::TypeParamTypeVar> for DefinitionNodeKey {
fn from(value: &ast::TypeParamTypeVar) -> Self {
Self(NodeKey::from_node(value))
}
}
impl From<&ast::TypeParamParamSpec> for DefinitionNodeKey {
fn from(value: &ast::TypeParamParamSpec) -> Self {
Self(NodeKey::from_node(value))
}
}
impl From<&ast::TypeParamTypeVarTuple> for DefinitionNodeKey {
fn from(value: &ast::TypeParamTypeVarTuple) -> Self {
Self(NodeKey::from_node(value))
}
}

View File

@@ -142,6 +142,11 @@ impl<'db> ScopeId<'db> {
NodeWithScopeKind::Class(class) | NodeWithScopeKind::ClassTypeParameters(class) => {
class.name.as_str()
}
NodeWithScopeKind::TypeAliasTypeParameters(type_alias) => type_alias
.name
.as_name_expr()
.map(|name| name.id.as_str())
.unwrap_or("<type alias>"),
NodeWithScopeKind::Function(function)
| NodeWithScopeKind::FunctionTypeParameters(function) => function.name.as_str(),
NodeWithScopeKind::Lambda(_) => "<lambda>",
@@ -339,6 +344,7 @@ pub(crate) enum NodeWithScopeRef<'a> {
Lambda(&'a ast::ExprLambda),
FunctionTypeParameters(&'a ast::StmtFunctionDef),
ClassTypeParameters(&'a ast::StmtClassDef),
TypeAliasTypeParameters(&'a ast::StmtTypeAlias),
ListComprehension(&'a ast::ExprListComp),
SetComprehension(&'a ast::ExprSetComp),
DictComprehension(&'a ast::ExprDictComp),
@@ -360,6 +366,9 @@ impl NodeWithScopeRef<'_> {
NodeWithScopeRef::Function(function) => {
NodeWithScopeKind::Function(AstNodeRef::new(module, function))
}
NodeWithScopeRef::TypeAliasTypeParameters(type_alias) => {
NodeWithScopeKind::TypeAliasTypeParameters(AstNodeRef::new(module, type_alias))
}
NodeWithScopeRef::Lambda(lambda) => {
NodeWithScopeKind::Lambda(AstNodeRef::new(module, lambda))
}
@@ -400,6 +409,9 @@ impl NodeWithScopeRef<'_> {
NodeWithScopeRef::ClassTypeParameters(class) => {
NodeWithScopeKey::ClassTypeParameters(NodeKey::from_node(class))
}
NodeWithScopeRef::TypeAliasTypeParameters(type_alias) => {
NodeWithScopeKey::TypeAliasTypeParameters(NodeKey::from_node(type_alias))
}
NodeWithScopeRef::ListComprehension(comprehension) => {
NodeWithScopeKey::ListComprehension(NodeKey::from_node(comprehension))
}
@@ -424,6 +436,7 @@ pub enum NodeWithScopeKind {
ClassTypeParameters(AstNodeRef<ast::StmtClassDef>),
Function(AstNodeRef<ast::StmtFunctionDef>),
FunctionTypeParameters(AstNodeRef<ast::StmtFunctionDef>),
TypeAliasTypeParameters(AstNodeRef<ast::StmtTypeAlias>),
Lambda(AstNodeRef<ast::ExprLambda>),
ListComprehension(AstNodeRef<ast::ExprListComp>),
SetComprehension(AstNodeRef<ast::ExprSetComp>),
@@ -438,7 +451,9 @@ impl NodeWithScopeKind {
Self::Class(_) => ScopeKind::Class,
Self::Function(_) => ScopeKind::Function,
Self::Lambda(_) => ScopeKind::Function,
Self::FunctionTypeParameters(_) | Self::ClassTypeParameters(_) => ScopeKind::Annotation,
Self::FunctionTypeParameters(_)
| Self::ClassTypeParameters(_)
| Self::TypeAliasTypeParameters(_) => ScopeKind::Annotation,
Self::ListComprehension(_)
| Self::SetComprehension(_)
| Self::DictComprehension(_)
@@ -468,6 +483,7 @@ pub(crate) enum NodeWithScopeKey {
ClassTypeParameters(NodeKey),
Function(NodeKey),
FunctionTypeParameters(NodeKey),
TypeAliasTypeParameters(NodeKey),
Lambda(NodeKey),
ListComprehension(NodeKey),
SetComprehension(NodeKey),

View File

@@ -277,7 +277,7 @@ impl<'db> UseDefMap<'db> {
pub(crate) fn use_boundness(&self, use_id: ScopedUseId) -> Boundness {
if self.bindings_by_use[use_id].may_be_unbound() {
Boundness::MayBeUnbound
Boundness::PossiblyUnbound
} else {
Boundness::Bound
}
@@ -292,7 +292,7 @@ impl<'db> UseDefMap<'db> {
pub(crate) fn public_boundness(&self, symbol: ScopedSymbolId) -> Boundness {
if self.public_symbols[symbol].may_be_unbound() {
Boundness::MayBeUnbound
Boundness::PossiblyUnbound
} else {
Boundness::Bound
}

View File

@@ -8,32 +8,38 @@ use crate::Db;
/// Enumeration of various core stdlib modules, for which we have dedicated Salsa queries.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum CoreStdlibModule {
pub(crate) enum CoreStdlibModule {
Builtins,
Types,
Typeshed,
TypingExtensions,
Typing,
Sys,
}
impl CoreStdlibModule {
fn name(self) -> ModuleName {
let module_name = match self {
pub(crate) const fn as_str(self) -> &'static str {
match self {
Self::Builtins => "builtins",
Self::Types => "types",
Self::Typing => "typing",
Self::Typeshed => "_typeshed",
Self::TypingExtensions => "typing_extensions",
};
ModuleName::new_static(module_name)
.unwrap_or_else(|| panic!("{module_name} should be a valid module name!"))
Self::Sys => "sys",
}
}
pub(crate) fn name(self) -> ModuleName {
let self_as_str = self.as_str();
ModuleName::new_static(self_as_str)
.unwrap_or_else(|| panic!("{self_as_str} should be a valid module name!"))
}
}
/// Lookup the type of `symbol` in a given core module
///
/// Returns `Symbol::Unbound` if the given core module cannot be resolved for some reason
fn core_module_symbol<'db>(
pub(crate) fn core_module_symbol<'db>(
db: &'db dyn Db,
core_module: CoreStdlibModule,
symbol: &str,
@@ -51,29 +57,14 @@ pub(crate) fn builtins_symbol<'db>(db: &'db dyn Db, symbol: &str) -> Symbol<'db>
core_module_symbol(db, CoreStdlibModule::Builtins, symbol)
}
/// Lookup the type of `symbol` in the `types` module namespace.
///
/// Returns `Symbol::Unbound` if the `types` module isn't available for some reason.
#[inline]
pub(crate) fn types_symbol<'db>(db: &'db dyn Db, symbol: &str) -> Symbol<'db> {
core_module_symbol(db, CoreStdlibModule::Types, symbol)
}
/// Lookup the type of `symbol` in the `typing` module namespace.
///
/// Returns `Symbol::Unbound` if the `typing` module isn't available for some reason.
#[inline]
#[allow(dead_code)] // currently only used in tests
#[cfg(test)]
pub(crate) fn typing_symbol<'db>(db: &'db dyn Db, symbol: &str) -> Symbol<'db> {
core_module_symbol(db, CoreStdlibModule::Typing, symbol)
}
/// Lookup the type of `symbol` in the `_typeshed` module namespace.
///
/// Returns `Symbol::Unbound` if the `_typeshed` module isn't available for some reason.
#[inline]
pub(crate) fn typeshed_symbol<'db>(db: &'db dyn Db, symbol: &str) -> Symbol<'db> {
core_module_symbol(db, CoreStdlibModule::Typeshed, symbol)
}
/// Lookup the type of `symbol` in the `typing_extensions` module namespace.
///

View File

@@ -6,7 +6,16 @@ use crate::{
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub(crate) enum Boundness {
Bound,
MayBeUnbound,
PossiblyUnbound,
}
impl Boundness {
pub(crate) fn or(self, other: Boundness) -> Boundness {
match (self, other) {
(Boundness::Bound, _) | (_, Boundness::Bound) => Boundness::Bound,
(Boundness::PossiblyUnbound, Boundness::PossiblyUnbound) => Boundness::PossiblyUnbound,
}
}
}
/// The result of a symbol lookup, which can either be a (possibly unbound) type
@@ -17,14 +26,14 @@ pub(crate) enum Boundness {
/// bound = 1
///
/// if flag:
/// maybe_unbound = 2
/// possibly_unbound = 2
/// ```
///
/// If we look up symbols in this scope, we would get the following results:
/// ```rs
/// bound: Symbol::Type(Type::IntLiteral(1), Boundness::Bound),
/// maybe_unbound: Symbol::Type(Type::IntLiteral(2), Boundness::MayBeUnbound),
/// non_existent: Symbol::Unbound,
/// bound: Symbol::Type(Type::IntLiteral(1), Boundness::Bound),
/// possibly_unbound: Symbol::Type(Type::IntLiteral(2), Boundness::PossiblyUnbound),
/// non_existent: Symbol::Unbound,
/// ```
#[derive(Debug, Clone, PartialEq)]
pub(crate) enum Symbol<'db> {
@@ -37,21 +46,18 @@ impl<'db> Symbol<'db> {
matches!(self, Symbol::Unbound)
}
pub(crate) fn may_be_unbound(&self) -> bool {
pub(crate) fn possibly_unbound(&self) -> bool {
match self {
Symbol::Type(_, Boundness::MayBeUnbound) | Symbol::Unbound => true,
Symbol::Type(_, Boundness::PossiblyUnbound) | Symbol::Unbound => true,
Symbol::Type(_, Boundness::Bound) => false,
}
}
pub(crate) fn unwrap_or_unknown(&self) -> Type<'db> {
match self {
Symbol::Type(ty, _) => *ty,
Symbol::Unbound => Type::Unknown,
}
}
pub(crate) fn as_type(&self) -> Option<Type<'db>> {
/// Returns the type of the symbol, ignoring possible unboundness.
///
/// If the symbol is *definitely* unbound, this function will return `None`. Otherwise,
/// if there is at least one control-flow path where the symbol is bound, return the type.
pub(crate) fn ignore_possibly_unbound(&self) -> Option<Type<'db>> {
match self {
Symbol::Type(ty, _) => Some(*ty),
Symbol::Unbound => None,
@@ -61,28 +67,80 @@ impl<'db> Symbol<'db> {
#[cfg(test)]
#[track_caller]
pub(crate) fn expect_type(self) -> Type<'db> {
self.as_type()
self.ignore_possibly_unbound()
.expect("Expected a (possibly unbound) type, not an unbound symbol")
}
#[must_use]
pub(crate) fn replace_unbound_with(
self,
db: &'db dyn Db,
replacement: &Symbol<'db>,
) -> Symbol<'db> {
match replacement {
Symbol::Type(replacement, _) => Symbol::Type(
match self {
Symbol::Type(ty, Boundness::Bound) => ty,
Symbol::Type(ty, Boundness::MayBeUnbound) => {
UnionType::from_elements(db, [*replacement, ty])
}
Symbol::Unbound => *replacement,
},
Boundness::Bound,
),
pub(crate) fn or_fall_back_to(self, db: &'db dyn Db, fallback: &Symbol<'db>) -> Symbol<'db> {
match fallback {
Symbol::Type(fallback_ty, fallback_boundness) => match self {
Symbol::Type(_, Boundness::Bound) => self,
Symbol::Type(ty, boundness @ Boundness::PossiblyUnbound) => Symbol::Type(
UnionType::from_elements(db, [*fallback_ty, ty]),
fallback_boundness.or(boundness),
),
Symbol::Unbound => fallback.clone(),
},
Symbol::Unbound => self,
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::types::tests::setup_db;
#[test]
fn test_symbol_or_fall_back_to() {
use Boundness::{Bound, PossiblyUnbound};
let db = setup_db();
let ty1 = Type::IntLiteral(1);
let ty2 = Type::IntLiteral(2);
// Start from an unbound symbol
assert_eq!(
Symbol::Unbound.or_fall_back_to(&db, &Symbol::Unbound),
Symbol::Unbound
);
assert_eq!(
Symbol::Unbound.or_fall_back_to(&db, &Symbol::Type(ty1, PossiblyUnbound)),
Symbol::Type(ty1, PossiblyUnbound)
);
assert_eq!(
Symbol::Unbound.or_fall_back_to(&db, &Symbol::Type(ty1, Bound)),
Symbol::Type(ty1, Bound)
);
// Start from a possibly unbound symbol
assert_eq!(
Symbol::Type(ty1, PossiblyUnbound).or_fall_back_to(&db, &Symbol::Unbound),
Symbol::Type(ty1, PossiblyUnbound)
);
assert_eq!(
Symbol::Type(ty1, PossiblyUnbound)
.or_fall_back_to(&db, &Symbol::Type(ty2, PossiblyUnbound)),
Symbol::Type(UnionType::from_elements(&db, [ty2, ty1]), PossiblyUnbound)
);
assert_eq!(
Symbol::Type(ty1, PossiblyUnbound).or_fall_back_to(&db, &Symbol::Type(ty2, Bound)),
Symbol::Type(UnionType::from_elements(&db, [ty2, ty1]), Bound)
);
// Start from a definitely bound symbol
assert_eq!(
Symbol::Type(ty1, Bound).or_fall_back_to(&db, &Symbol::Unbound),
Symbol::Type(ty1, Bound)
);
assert_eq!(
Symbol::Type(ty1, Bound).or_fall_back_to(&db, &Symbol::Type(ty2, PossiblyUnbound)),
Symbol::Type(ty1, Bound)
);
assert_eq!(
Symbol::Type(ty1, Bound).or_fall_back_to(&db, &Symbol::Type(ty2, Bound)),
Symbol::Type(ty1, Bound)
);
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -383,7 +383,7 @@ mod tests {
use crate::program::{Program, SearchPathSettings};
use crate::python_version::PythonVersion;
use crate::stdlib::typing_symbol;
use crate::types::{global_symbol, KnownClass, StringLiteralType, UnionBuilder};
use crate::types::{global_symbol, KnownClass, UnionBuilder};
use crate::ProgramSettings;
use ruff_db::files::system_path_to_file;
use ruff_db::system::{DbWithTestSystem, SystemPathBuf};
@@ -775,7 +775,7 @@ mod tests {
.build();
assert_eq!(ty, s);
let literal = Type::StringLiteral(StringLiteralType::new(&db, "a"));
let literal = Type::string_literal(&db, "a");
let expected = IntersectionBuilder::new(&db)
.add_positive(s)
.add_negative(literal)
@@ -878,7 +878,7 @@ mod tests {
let ty = IntersectionBuilder::new(&db)
.add_positive(s)
.add_negative(Type::StringLiteral(StringLiteralType::new(&db, "a")))
.add_negative(Type::string_literal(&db, "a"))
.add_negative(t)
.build();
assert_eq!(ty, Type::Never);
@@ -912,7 +912,7 @@ mod tests {
let db = setup_db();
let t_p = KnownClass::Int.to_instance(&db);
let t_n = Type::StringLiteral(StringLiteralType::new(&db, "t_n"));
let t_n = Type::string_literal(&db, "t_n");
let ty = IntersectionBuilder::new(&db)
.add_positive(t_p)

View File

@@ -66,10 +66,13 @@ impl Display for DisplayRepresentation<'_> {
Type::Any => f.write_str("Any"),
Type::Never => f.write_str("Never"),
Type::Unknown => f.write_str("Unknown"),
Type::Instance(InstanceType { class })
if class.is_known(self.db, KnownClass::NoneType) =>
{
f.write_str("None")
Type::Instance(InstanceType { class }) => {
let representation = match class.known(self.db) {
Some(KnownClass::NoneType) => "None",
Some(KnownClass::NoDefaultType) => "NoDefault",
_ => class.name(self.db),
};
f.write_str(representation)
}
// `[Type::Todo]`'s display should be explicit that is not a valid display of
// any other type
@@ -82,7 +85,6 @@ impl Display for DisplayRepresentation<'_> {
Type::SubclassOf(SubclassOfType { class }) => {
write!(f, "type[{}]", class.name(self.db))
}
Type::Instance(InstanceType { class }) => f.write_str(class.name(self.db)),
Type::KnownInstance(known_instance) => f.write_str(known_instance.as_str()),
Type::FunctionLiteral(function) => f.write_str(function.name(self.db)),
Type::Union(union) => union.display(self.db).fmt(f),
@@ -332,9 +334,7 @@ mod tests {
use ruff_db::system::{DbWithTestSystem, SystemPathBuf};
use crate::db::tests::TestDb;
use crate::types::{
global_symbol, BytesLiteralType, SliceLiteralType, StringLiteralType, Type, UnionType,
};
use crate::types::{global_symbol, SliceLiteralType, Type, UnionType};
use crate::{Program, ProgramSettings, PythonVersion, SearchPathSettings};
fn setup_db() -> TestDb {
@@ -380,12 +380,12 @@ mod tests {
Type::Unknown,
Type::IntLiteral(-1),
global_symbol(&db, mod_file, "A").expect_type(),
Type::StringLiteral(StringLiteralType::new(&db, "A")),
Type::BytesLiteral(BytesLiteralType::new(&db, [0u8].as_slice())),
Type::BytesLiteral(BytesLiteralType::new(&db, [7u8].as_slice())),
Type::string_literal(&db, "A"),
Type::bytes_literal(&db, &[0u8]),
Type::bytes_literal(&db, &[7u8]),
Type::IntLiteral(0),
Type::IntLiteral(1),
Type::StringLiteral(StringLiteralType::new(&db, "B")),
Type::string_literal(&db, "B"),
global_symbol(&db, mod_file, "foo").expect_type(),
global_symbol(&db, mod_file, "bar").expect_type(),
global_symbol(&db, mod_file, "B").expect_type(),

View File

@@ -31,7 +31,6 @@ use std::num::NonZeroU32;
use itertools::Itertools;
use ruff_db::files::File;
use ruff_db::parsed::parsed_module;
use ruff_python_ast::name::Name;
use ruff_python_ast::{self as ast, AnyNodeRef, Expr, ExprContext, UnaryOp};
use rustc_hash::FxHashMap;
use salsa;
@@ -56,10 +55,10 @@ use crate::types::mro::MroErrorKind;
use crate::types::unpacker::{UnpackResult, Unpacker};
use crate::types::{
bindings_ty, builtins_symbol, declarations_ty, global_symbol, symbol, typing_extensions_symbol,
Boundness, BytesLiteralType, Class, ClassLiteralType, FunctionType, InstanceType,
IntersectionBuilder, IntersectionType, IterationOutcome, KnownClass, KnownFunction,
KnownInstanceType, MetaclassCandidate, MetaclassErrorKind, SliceLiteralType, StringLiteralType,
Symbol, Truthiness, TupleType, Type, TypeArrayDisplay, UnionBuilder, UnionType,
Boundness, Class, ClassLiteralType, FunctionType, InstanceType, IntersectionBuilder,
IntersectionType, IterationOutcome, KnownClass, KnownFunction, KnownInstanceType,
MetaclassCandidate, MetaclassErrorKind, SliceLiteralType, Symbol, Truthiness, TupleType, Type,
TypeArrayDisplay, TypeVarBoundOrConstraints, TypeVarInstance, UnionBuilder, UnionType,
};
use crate::unpack::Unpack;
use crate::util::subscript::{PyIndex, PySlice};
@@ -426,6 +425,9 @@ impl<'db> TypeInferenceBuilder<'db> {
NodeWithScopeKind::FunctionTypeParameters(function) => {
self.infer_function_type_params(function.node());
}
NodeWithScopeKind::TypeAliasTypeParameters(type_alias) => {
self.infer_type_alias_type_params(type_alias.node());
}
NodeWithScopeKind::ListComprehension(comprehension) => {
self.infer_list_comprehension_expression_scope(comprehension.node());
}
@@ -456,9 +458,13 @@ impl<'db> TypeInferenceBuilder<'db> {
self.check_class_definitions();
}
/// Iterate over all class definitions to check that Python will be able to create a
/// consistent "[method resolution order]" and [metaclass] for each class at runtime. If not,
/// issue a diagnostic.
/// Iterate over all class definitions to check that the definition will not cause an exception
/// to be raised at runtime. This needs to be done after most other types in the scope have been
/// inferred, due to the fact that base classes can be deferred. If it looks like a class
/// definition is invalid in some way, issue a diagnostic.
///
/// Among the things we check for in this method are whether Python will be able to determine a
/// consistent "[method resolution order]" and [metaclass] for each class.
///
/// [method resolution order]: https://docs.python.org/3/glossary.html#term-method-resolution-order
/// [metaclass]: https://docs.python.org/3/reference/datamodel.html#metaclasses
@@ -470,7 +476,25 @@ impl<'db> TypeInferenceBuilder<'db> {
.filter_map(|ty| ty.into_class_literal())
.map(|class_ty| class_ty.class);
// Iterate through all class definitions in this scope.
for class in class_definitions {
// (1) Check that the class does not have a cyclic definition
if class.is_cyclically_defined(self.db) {
self.diagnostics.add(
class.node(self.db).into(),
"cyclic-class-def",
format_args!(
"Cyclic definition of `{}` or bases of `{}` (class cannot inherit from itself)",
class.name(self.db),
class.name(self.db)
),
);
// Attempting to determine the MRO of a class or if the class has a metaclass conflict
// is impossible if the class is cyclically defined; there's nothing more to do here.
continue;
}
// (2) Check that the class's MRO is resolvable
if let Err(mro_error) = class.try_mro(self.db).as_ref() {
match mro_error.reason() {
MroErrorKind::DuplicateBases(duplicates) => {
@@ -483,15 +507,6 @@ impl<'db> TypeInferenceBuilder<'db> {
);
}
}
MroErrorKind::CyclicClassDefinition => self.diagnostics.add(
class.node(self.db).into(),
"cyclic-class-def",
format_args!(
"Cyclic definition of `{}` or bases of `{}` (class cannot inherit from itself)",
class.name(self.db),
class.name(self.db)
),
),
MroErrorKind::InvalidBases(bases) => {
let base_nodes = class.node(self.db).bases();
for (index, base_ty) in bases {
@@ -517,6 +532,7 @@ impl<'db> TypeInferenceBuilder<'db> {
}
}
// (3) Check that the class's metaclass can be determined without error.
if let Err(metaclass_error) = class.try_metaclass(self.db) {
match metaclass_error.reason() {
MetaclassErrorKind::Conflict {
@@ -564,10 +580,6 @@ impl<'db> TypeInferenceBuilder<'db> {
);
}
}
MetaclassErrorKind::CyclicDefinition => {
// Cyclic class definition diagnostic will already have been emitted above
// in MRO calculation.
}
}
}
}
@@ -579,6 +591,9 @@ impl<'db> TypeInferenceBuilder<'db> {
self.infer_function_definition(function.node(), definition);
}
DefinitionKind::Class(class) => self.infer_class_definition(class.node(), definition),
DefinitionKind::TypeAlias(type_alias) => {
self.infer_type_alias_definition(type_alias.node(), definition);
}
DefinitionKind::Import(import) => {
self.infer_import_definition(import.node(), definition);
}
@@ -642,6 +657,15 @@ impl<'db> TypeInferenceBuilder<'db> {
DefinitionKind::ExceptHandler(except_handler_definition) => {
self.infer_except_handler_definition(except_handler_definition, definition);
}
DefinitionKind::TypeVar(node) => {
self.infer_typevar_definition(node, definition);
}
DefinitionKind::ParamSpec(node) => {
self.infer_paramspec_definition(node, definition);
}
DefinitionKind::TypeVarTuple(node) => {
self.infer_typevartuple_definition(node, definition);
}
}
}
@@ -804,6 +828,16 @@ impl<'db> TypeInferenceBuilder<'db> {
self.infer_parameters(&function.parameters);
}
fn infer_type_alias_type_params(&mut self, type_alias: &ast::StmtTypeAlias) {
let type_params = type_alias
.type_params
.as_ref()
.expect("type alias type params scope without type params");
self.infer_type_parameters(type_params);
// self.infer_optional_expression(type_alias.value.as_deref());
}
fn infer_function_body(&mut self, function: &ast::StmtFunctionDef) {
self.infer_body(&function.body);
}
@@ -1019,10 +1053,10 @@ impl<'db> TypeInferenceBuilder<'db> {
let maybe_known_class = file_to_module(self.db, self.file)
.as_ref()
.and_then(|module| KnownClass::maybe_from_module(module, name.as_str()));
.and_then(|module| KnownClass::try_from_module(module, name.as_str()));
let class = Class::new(self.db, &*name.id, body_scope, maybe_known_class);
let class_ty = Type::ClassLiteral(ClassLiteralType { class });
let class_ty = Type::class_literal(class);
self.add_declaration_with_binding(class_node.into(), definition, class_ty, class_ty);
@@ -1057,6 +1091,21 @@ impl<'db> TypeInferenceBuilder<'db> {
}
}
fn infer_type_alias_definition(
&mut self,
type_alias: &ast::StmtTypeAlias,
definition: Definition<'db>,
) {
let type_alias_ty = Type::Todo;
self.add_declaration_with_binding(
type_alias.into(),
definition,
type_alias_ty,
type_alias_ty,
);
}
fn infer_if_statement(&mut self, if_statement: &ast::StmtIf) {
let ast::StmtIf {
range: _,
@@ -1218,7 +1267,7 @@ impl<'db> TypeInferenceBuilder<'db> {
Type::Unknown
}
(Symbol::Type(enter_ty, enter_boundness), exit) => {
if enter_boundness == Boundness::MayBeUnbound {
if enter_boundness == Boundness::PossiblyUnbound {
self.diagnostics.add(
context_expression.into(),
"invalid-context-manager",
@@ -1257,7 +1306,7 @@ impl<'db> TypeInferenceBuilder<'db> {
Symbol::Type(exit_ty, exit_boundness) => {
// TODO: Use the `exit_ty` to determine if any raised exception is suppressed.
if exit_boundness == Boundness::MayBeUnbound {
if exit_boundness == Boundness::PossiblyUnbound {
self.diagnostics.add(
context_expression.into(),
"invalid-context-manager",
@@ -1321,7 +1370,8 @@ impl<'db> TypeInferenceBuilder<'db> {
// TODO should infer `ExceptionGroup` if all caught exceptions
// are subclasses of `Exception` --Alex
builtins_symbol(self.db, "BaseExceptionGroup")
.unwrap_or_unknown()
.ignore_possibly_unbound()
.unwrap_or(Type::Unknown)
.to_instance(self.db)
} else {
// TODO: anything that's a consistent subtype of
@@ -1329,15 +1379,13 @@ impl<'db> TypeInferenceBuilder<'db> {
// anything else is invalid and should lead to a diagnostic being reported --Alex
match node_ty {
Type::Any | Type::Unknown => node_ty,
Type::ClassLiteral(ClassLiteralType { class }) => {
Type::Instance(InstanceType { class })
}
Type::ClassLiteral(ClassLiteralType { class }) => Type::instance(class),
Type::Tuple(tuple) => UnionType::from_elements(
self.db,
tuple.elements(self.db).iter().map(|ty| {
ty.into_class_literal()
.map_or(Type::Todo, |ClassLiteralType { class }| {
Type::Instance(InstanceType { class })
Type::instance(class)
})
}),
),
@@ -1352,6 +1400,82 @@ impl<'db> TypeInferenceBuilder<'db> {
);
}
fn infer_typevar_definition(
&mut self,
node: &ast::TypeParamTypeVar,
definition: Definition<'db>,
) {
let ast::TypeParamTypeVar {
range: _,
name,
bound,
default,
} = node;
let bound_or_constraint = match bound.as_deref() {
Some(expr @ ast::Expr::Tuple(ast::ExprTuple { elts, .. })) => {
if elts.len() < 2 {
self.diagnostics.add(
expr.into(),
"invalid-typevar-constraints",
format_args!("TypeVar must have at least two constrained types"),
);
self.infer_expression(expr);
None
} else {
let tuple = TupleType::new(
self.db,
elts.iter()
.map(|expr| self.infer_type_expression(expr))
.collect::<Box<_>>(),
);
let constraints = TypeVarBoundOrConstraints::Constraints(tuple);
self.store_expression_type(expr, Type::Tuple(tuple));
Some(constraints)
}
}
Some(expr) => Some(TypeVarBoundOrConstraints::UpperBound(
self.infer_type_expression(expr),
)),
None => None,
};
let default_ty = self.infer_optional_type_expression(default.as_deref());
let ty = Type::KnownInstance(KnownInstanceType::TypeVar(TypeVarInstance::new(
self.db,
name.id.clone(),
bound_or_constraint,
default_ty,
)));
self.add_declaration_with_binding(node.into(), definition, ty, ty);
}
fn infer_paramspec_definition(
&mut self,
node: &ast::TypeParamParamSpec,
definition: Definition<'db>,
) {
let ast::TypeParamParamSpec {
range: _,
name: _,
default,
} = node;
self.infer_optional_expression(default.as_deref());
self.add_declaration_with_binding(node.into(), definition, Type::Todo, Type::Todo);
}
fn infer_typevartuple_definition(
&mut self,
node: &ast::TypeParamTypeVarTuple,
definition: Definition<'db>,
) {
let ast::TypeParamTypeVarTuple {
range: _,
name: _,
default,
} = node;
self.infer_optional_expression(default.as_deref());
self.add_declaration_with_binding(node.into(), definition, Type::Todo, Type::Todo);
}
fn infer_match_statement(&mut self, match_statement: &ast::StmtMatch) {
let ast::StmtMatch {
range: _,
@@ -1535,8 +1659,7 @@ impl<'db> TypeInferenceBuilder<'db> {
let mut annotation_ty = self.infer_annotation_expression(annotation);
// If the declared variable is annotated with _SpecialForm class then we treat it differently
// by assigning the known field to the instance.
// Handle various singletons.
if let Type::Instance(InstanceType { class }) = annotation_ty {
if class.is_known(self.db, KnownClass::SpecialForm) {
if let Some(name_expr) = target.as_name_expr() {
@@ -1618,7 +1741,7 @@ impl<'db> TypeInferenceBuilder<'db> {
return match boundness {
Boundness::Bound => augmented_return_ty,
Boundness::MayBeUnbound => {
Boundness::PossiblyUnbound => {
let left_ty = target_type;
let right_ty = value_type;
@@ -1700,18 +1823,19 @@ impl<'db> TypeInferenceBuilder<'db> {
self.infer_augmented_op(assignment, target_type, value_type)
}
fn infer_type_alias_statement(&mut self, type_alias_statement: &ast::StmtTypeAlias) {
let ast::StmtTypeAlias {
range: _,
name,
type_params,
value,
} = type_alias_statement;
self.infer_expression(value);
self.infer_expression(name);
if let Some(type_params) = type_params {
self.infer_type_parameters(type_params);
}
fn infer_type_alias_statement(&mut self, node: &ast::StmtTypeAlias) {
self.infer_definition(node);
// let ast::StmtTypeAlias {
// range: _,
// name,
// type_params,
// value,
// } = type_alias_statement;
// // self.infer_expression(value);
// // self.infer_expression(name);
// if let Some(type_params) = type_params {
// self.infer_type_parameters(type_params);
// }
}
fn infer_for_statement(&mut self, for_statement: &ast::StmtFor) {
@@ -1915,20 +2039,27 @@ impl<'db> TypeInferenceBuilder<'db> {
asname: _,
} = alias;
// For possibly-unbound names, just eliminate Unbound from the type; we
// must be in a bound path. TODO diagnostic for maybe-unbound import?
module_ty
.member(self.db, &ast::name::Name::new(&name.id))
.as_type()
.unwrap_or_else(|| {
match module_ty.member(self.db, &ast::name::Name::new(&name.id)) {
Symbol::Type(ty, boundness) => {
if boundness == Boundness::PossiblyUnbound {
self.diagnostics.add(
AnyNodeRef::Alias(alias),
"possibly-unbound-import",
format_args!("Member `{name}` of module `{module_name}` is possibly unbound",),
);
}
ty
}
Symbol::Unbound => {
self.diagnostics.add(
AnyNodeRef::Alias(alias),
"unresolved-import",
format_args!("Module `{module_name}` has no member `{name}`",),
);
Type::Unknown
})
}
}
} else {
self.diagnostics
.add_unresolved_module(import_from, *level, module);
@@ -2013,13 +2144,6 @@ impl<'db> TypeInferenceBuilder<'db> {
expression.map(|expr| self.infer_expression(expr))
}
fn infer_optional_annotation_expression(
&mut self,
expr: Option<&ast::Expr>,
) -> Option<Type<'db>> {
expr.map(|expr| self.infer_annotation_expression(expr))
}
#[track_caller]
fn infer_expression(&mut self, expression: &ast::Expr) -> Type<'db> {
debug_assert_eq!(
@@ -2101,7 +2225,8 @@ impl<'db> TypeInferenceBuilder<'db> {
.unwrap_or_else(|| KnownClass::Int.to_instance(self.db)),
ast::Number::Float(_) => KnownClass::Float.to_instance(self.db),
ast::Number::Complex { .. } => builtins_symbol(self.db, "complex")
.unwrap_or_unknown()
.ignore_possibly_unbound()
.unwrap_or(Type::Unknown)
.to_instance(self.db),
}
}
@@ -2115,7 +2240,7 @@ impl<'db> TypeInferenceBuilder<'db> {
fn infer_string_literal_expression(&mut self, literal: &ast::ExprStringLiteral) -> Type<'db> {
if literal.value.len() <= Self::MAX_STRING_LITERAL_SIZE {
Type::StringLiteral(StringLiteralType::new(self.db, literal.value.to_str()))
Type::string_literal(self.db, literal.value.to_str())
} else {
Type::LiteralString
}
@@ -2123,10 +2248,8 @@ impl<'db> TypeInferenceBuilder<'db> {
fn infer_bytes_literal_expression(&mut self, literal: &ast::ExprBytesLiteral) -> Type<'db> {
// TODO: ignoring r/R prefixes for now, should normalize bytes values
Type::BytesLiteral(BytesLiteralType::new(
self.db,
literal.value.bytes().collect::<Box<[u8]>>(),
))
let bytes: Vec<u8> = literal.value.bytes().collect();
Type::bytes_literal(self.db, &bytes)
}
fn infer_fstring_expression(&mut self, fstring: &ast::ExprFString) -> Type<'db> {
@@ -2182,7 +2305,9 @@ impl<'db> TypeInferenceBuilder<'db> {
&mut self,
_literal: &ast::ExprEllipsisLiteral,
) -> Type<'db> {
builtins_symbol(self.db, "Ellipsis").unwrap_or_unknown()
builtins_symbol(self.db, "Ellipsis")
.ignore_possibly_unbound()
.unwrap_or(Type::Unknown)
}
fn infer_tuple_expression(&mut self, tuple: &ast::ExprTuple) -> Type<'db> {
@@ -2193,12 +2318,10 @@ impl<'db> TypeInferenceBuilder<'db> {
parenthesized: _,
} = tuple;
let element_types = elts
.iter()
.map(|elt| self.infer_expression(elt))
.collect::<Vec<_>>();
let element_types: Vec<Type<'db>> =
elts.iter().map(|elt| self.infer_expression(elt)).collect();
Type::Tuple(TupleType::new(self.db, element_types.into_boxed_slice()))
Type::tuple(self.db, &element_types)
}
fn infer_list_expression(&mut self, list: &ast::ExprList) -> Type<'db> {
@@ -2607,21 +2730,21 @@ impl<'db> TypeInferenceBuilder<'db> {
};
// Fallback to builtins (without infinite recursion if we're already in builtins.)
if global_symbol.may_be_unbound()
if global_symbol.possibly_unbound()
&& Some(self.scope()) != builtins_module_scope(self.db)
{
let mut symbol = builtins_symbol(self.db, name);
if symbol.is_unbound() && name == "reveal_type" {
let mut builtins_symbol = builtins_symbol(self.db, name);
if builtins_symbol.is_unbound() && name == "reveal_type" {
self.diagnostics.add(
name_node.into(),
"undefined-reveal",
format_args!(
"`reveal_type` used without importing it; this is allowed for debugging convenience but will fail at runtime"),
);
symbol = typing_extensions_symbol(self.db, name);
builtins_symbol = typing_extensions_symbol(self.db, name);
}
global_symbol.replace_unbound_with(self.db, &symbol)
global_symbol.or_fall_back_to(self.db, &builtins_symbol)
} else {
global_symbol
}
@@ -2661,10 +2784,10 @@ impl<'db> TypeInferenceBuilder<'db> {
let bindings_ty = bindings_ty(self.db, definitions);
if boundness == Boundness::MayBeUnbound {
if boundness == Boundness::PossiblyUnbound {
match self.lookup_name(name) {
Symbol::Type(looked_up_ty, looked_up_boundness) => {
if looked_up_boundness == Boundness::MayBeUnbound {
if looked_up_boundness == Boundness::PossiblyUnbound {
self.diagnostics.add_possibly_unresolved_reference(name);
}
@@ -2704,9 +2827,35 @@ impl<'db> TypeInferenceBuilder<'db> {
} = attribute;
let value_ty = self.infer_expression(value);
value_ty
.member(self.db, &Name::new(&attr.id))
.unwrap_or_unknown()
match value_ty.member(self.db, &attr.id) {
Symbol::Type(member_ty, boundness) => {
if boundness == Boundness::PossiblyUnbound {
self.diagnostics.add(
attribute.into(),
"possibly-unbound-attribute",
format_args!(
"Attribute `{}` on type `{}` is possibly unbound",
attr.id,
value_ty.display(self.db),
),
);
}
member_ty
}
Symbol::Unbound => {
self.diagnostics.add(
attribute.into(),
"unresolved-attribute",
format_args!(
"Type `{}` has no attribute `{}`",
value_ty.display(self.db),
attr.id
),
);
Type::Unknown
}
}
}
fn infer_attribute_expression(&mut self, attribute: &ast::ExprAttribute) -> Type<'db> {
@@ -2900,21 +3049,15 @@ impl<'db> TypeInferenceBuilder<'db> {
}
(Type::BytesLiteral(lhs), Type::BytesLiteral(rhs), ast::Operator::Add) => {
Some(Type::BytesLiteral(BytesLiteralType::new(
self.db,
[lhs.value(self.db).as_ref(), rhs.value(self.db).as_ref()]
.concat()
.into_boxed_slice(),
)))
let bytes = [&**lhs.value(self.db), &**rhs.value(self.db)].concat();
Some(Type::bytes_literal(self.db, &bytes))
}
(Type::StringLiteral(lhs), Type::StringLiteral(rhs), ast::Operator::Add) => {
let lhs_value = lhs.value(self.db).to_string();
let rhs_value = rhs.value(self.db).as_ref();
let ty = if lhs_value.len() + rhs_value.len() <= Self::MAX_STRING_LITERAL_SIZE {
Type::StringLiteral(StringLiteralType::new(self.db, {
(lhs_value + rhs_value).into_boxed_str()
}))
Type::string_literal(self.db, &(lhs_value + rhs_value))
} else {
Type::LiteralString
};
@@ -2930,16 +3073,13 @@ impl<'db> TypeInferenceBuilder<'db> {
(Type::StringLiteral(s), Type::IntLiteral(n), ast::Operator::Mult)
| (Type::IntLiteral(n), Type::StringLiteral(s), ast::Operator::Mult) => {
let ty = if n < 1 {
Type::StringLiteral(StringLiteralType::new(self.db, ""))
Type::string_literal(self.db, "")
} else if let Ok(n) = usize::try_from(n) {
if n.checked_mul(s.value(self.db).len())
.is_some_and(|new_length| new_length <= Self::MAX_STRING_LITERAL_SIZE)
{
let new_literal = s.value(self.db).repeat(n);
Type::StringLiteral(StringLiteralType::new(
self.db,
new_literal.into_boxed_str(),
))
Type::string_literal(self.db, &new_literal)
} else {
Type::LiteralString
}
@@ -2952,7 +3092,7 @@ impl<'db> TypeInferenceBuilder<'db> {
(Type::LiteralString, Type::IntLiteral(n), ast::Operator::Mult)
| (Type::IntLiteral(n), Type::LiteralString, ast::Operator::Mult) => {
let ty = if n < 1 {
Type::StringLiteral(StringLiteralType::new(self.db, ""))
Type::string_literal(self.db, "")
} else {
Type::LiteralString
};
@@ -3458,6 +3598,16 @@ impl<'db> TypeInferenceBuilder<'db> {
(_, Type::BytesLiteral(_)) => {
self.infer_binary_type_comparison(left, op, KnownClass::Bytes.to_instance(self.db))
}
(Type::Tuple(_), Type::Instance(InstanceType { class }))
if class.is_known(self.db, KnownClass::VersionInfo) =>
{
self.infer_binary_type_comparison(left, op, Type::version_info_tuple(self.db))
}
(Type::Instance(InstanceType { class }), Type::Tuple(_))
if class.is_known(self.db, KnownClass::VersionInfo) =>
{
self.infer_binary_type_comparison(Type::version_info_tuple(self.db), op, right)
}
(Type::Tuple(lhs), Type::Tuple(rhs)) => {
// Note: This only works on heterogeneous tuple types.
let lhs_elements = lhs.elements(self.db);
@@ -3640,6 +3790,16 @@ impl<'db> TypeInferenceBuilder<'db> {
slice_ty: Type<'db>,
) -> Type<'db> {
match (value_ty, slice_ty) {
(
Type::Instance(InstanceType { class }),
Type::IntLiteral(_) | Type::BooleanLiteral(_) | Type::SliceLiteral(_),
) if class.is_known(self.db, KnownClass::VersionInfo) => self
.infer_subscript_expression_types(
value_node,
Type::version_info_tuple(self.db),
slice_ty,
),
// Ex) Given `("a", "b", "c", "d")[1]`, return `"b"`
(Type::Tuple(tuple_ty), Type::IntLiteral(int)) if i32::try_from(int).is_ok() => {
let elements = tuple_ty.elements(self.db);
@@ -3665,7 +3825,7 @@ impl<'db> TypeInferenceBuilder<'db> {
if let Ok(new_elements) = elements.py_slice(start, stop, step) {
let new_elements: Vec<_> = new_elements.copied().collect();
Type::Tuple(TupleType::new(self.db, new_elements.into_boxed_slice()))
Type::tuple(self.db, &new_elements)
} else {
self.diagnostics.add_slice_step_size_zero(value_node.into());
Type::Unknown
@@ -3679,12 +3839,7 @@ impl<'db> TypeInferenceBuilder<'db> {
literal_value
.chars()
.py_index(i32::try_from(int).expect("checked in branch arm"))
.map(|ch| {
Type::StringLiteral(StringLiteralType::new(
self.db,
ch.to_string().into_boxed_str(),
))
})
.map(|ch| Type::string_literal(self.db, &ch.to_string()))
.unwrap_or_else(|_| {
self.diagnostics.add_index_out_of_bounds(
"string",
@@ -3704,7 +3859,7 @@ impl<'db> TypeInferenceBuilder<'db> {
let chars: Vec<_> = literal_value.chars().collect();
let result = if let Ok(new_chars) = chars.py_slice(start, stop, step) {
let literal: String = new_chars.collect();
Type::StringLiteral(StringLiteralType::new(self.db, literal.into_boxed_str()))
Type::string_literal(self.db, &literal)
} else {
self.diagnostics.add_slice_step_size_zero(value_node.into());
Type::Unknown
@@ -3719,9 +3874,7 @@ impl<'db> TypeInferenceBuilder<'db> {
literal_value
.iter()
.py_index(i32::try_from(int).expect("checked in branch arm"))
.map(|byte| {
Type::BytesLiteral(BytesLiteralType::new(self.db, [*byte].as_slice()))
})
.map(|byte| Type::bytes_literal(self.db, &[*byte]))
.unwrap_or_else(|_| {
self.diagnostics.add_index_out_of_bounds(
"bytes literal",
@@ -3740,7 +3893,7 @@ impl<'db> TypeInferenceBuilder<'db> {
if let Ok(new_bytes) = literal_value.py_slice(start, stop, step) {
let new_bytes: Vec<u8> = new_bytes.copied().collect();
Type::BytesLiteral(BytesLiteralType::new(self.db, new_bytes.into_boxed_slice()))
Type::bytes_literal(self.db, &new_bytes)
} else {
self.diagnostics.add_slice_step_size_zero(value_node.into());
Type::Unknown
@@ -3765,7 +3918,7 @@ impl<'db> TypeInferenceBuilder<'db> {
match value_meta_ty.member(self.db, "__getitem__") {
Symbol::Unbound => {}
Symbol::Type(dunder_getitem_method, boundness) => {
if boundness == Boundness::MayBeUnbound {
if boundness == Boundness::PossiblyUnbound {
self.diagnostics.add(
value_node.into(),
"call-possibly-unbound-method",
@@ -3809,7 +3962,7 @@ impl<'db> TypeInferenceBuilder<'db> {
match dunder_class_getitem_method {
Symbol::Unbound => {}
Symbol::Type(ty, boundness) => {
if boundness == Boundness::MayBeUnbound {
if boundness == Boundness::PossiblyUnbound {
self.diagnostics.add(
value_node.into(),
"call-possibly-unbound-method",
@@ -3912,32 +4065,9 @@ impl<'db> TypeInferenceBuilder<'db> {
} = type_parameters;
for type_param in type_params {
match type_param {
ast::TypeParam::TypeVar(typevar) => {
let ast::TypeParamTypeVar {
range: _,
name: _,
bound,
default,
} = typevar;
self.infer_optional_expression(bound.as_deref());
self.infer_optional_expression(default.as_deref());
}
ast::TypeParam::ParamSpec(param_spec) => {
let ast::TypeParamParamSpec {
range: _,
name: _,
default,
} = param_spec;
self.infer_optional_expression(default.as_deref());
}
ast::TypeParam::TypeVarTuple(typevar_tuple) => {
let ast::TypeParamTypeVarTuple {
range: _,
name: _,
default,
} = typevar_tuple;
self.infer_optional_expression(default.as_deref());
}
ast::TypeParam::TypeVar(node) => self.infer_definition(node),
ast::TypeParam::ParamSpec(node) => self.infer_definition(node),
ast::TypeParam::TypeVarTuple(node) => self.infer_definition(node),
}
}
}
@@ -3971,6 +4101,13 @@ impl<'db> TypeInferenceBuilder<'db> {
self.store_expression_type(expression, annotation_ty);
annotation_ty
}
fn infer_optional_annotation_expression(
&mut self,
expr: Option<&ast::Expr>,
) -> Option<Type<'db>> {
expr.map(|expr| self.infer_annotation_expression(expr))
}
}
/// Type expressions
@@ -4145,6 +4282,13 @@ impl<'db> TypeInferenceBuilder<'db> {
ty
}
fn infer_optional_type_expression(
&mut self,
opt_expression: Option<&ast::Expr>,
) -> Option<Type<'db>> {
opt_expression.map(|expr| self.infer_type_expression(expr))
}
/// Given the slice of a `tuple[]` annotation, return the type that the annotation represents
fn infer_tuple_type_expression(&mut self, tuple_slice: &ast::Expr) -> Type<'db> {
/// In most cases, if a subelement of the tuple is inferred as `Todo`,
@@ -4184,7 +4328,7 @@ impl<'db> TypeInferenceBuilder<'db> {
if return_todo {
Type::Todo
} else {
Type::Tuple(TupleType::new(self.db, element_types.into_boxed_slice()))
Type::tuple(self.db, &element_types)
}
}
single_element => {
@@ -4192,7 +4336,7 @@ impl<'db> TypeInferenceBuilder<'db> {
if element_could_alter_type_of_whole_tuple(single_element, single_element_ty) {
Type::Todo
} else {
Type::Tuple(TupleType::new(self.db, Box::from([single_element_ty])))
Type::tuple(self.db, &[single_element_ty])
}
}
}
@@ -4203,8 +4347,8 @@ impl<'db> TypeInferenceBuilder<'db> {
match slice {
ast::Expr::Name(name) => {
let name_ty = self.infer_name_expression(name);
if let Some(class_literal) = name_ty.into_class_literal() {
Type::SubclassOf(class_literal.to_subclass_of_type())
if let Some(ClassLiteralType { class }) = name_ty.into_class_literal() {
Type::subclass_of(class)
} else {
Type::Todo
}
@@ -4262,6 +4406,7 @@ impl<'db> TypeInferenceBuilder<'db> {
Type::Unknown
}
},
KnownInstanceType::TypeVar(_) => Type::Todo,
}
}
@@ -4306,7 +4451,10 @@ impl<'db> TypeInferenceBuilder<'db> {
ast::Expr::Attribute(ast::ExprAttribute { value, attr, .. }) => {
let value_ty = self.infer_expression(value);
// TODO: Check that value type is enum otherwise return None
value_ty.member(self.db, &attr.id).unwrap_or_unknown()
value_ty
.member(self.db, &attr.id)
.ignore_possibly_unbound()
.unwrap_or(Type::Unknown)
}
ast::Expr::NoneLiteral(_) => Type::none(self.db),
// for negative and positive numbers
@@ -4462,7 +4610,7 @@ impl StringPartsCollector {
if self.expression {
KnownClass::Str.to_instance(db)
} else if let Some(concatenated) = self.concatenated {
Type::StringLiteral(StringLiteralType::new(db, concatenated.into_boxed_str()))
Type::string_literal(db, &concatenated)
} else {
Type::LiteralString
}
@@ -4642,13 +4790,12 @@ mod tests {
}
#[track_caller]
fn assert_scope_ty(
db: &TestDb,
fn get_symbol<'db>(
db: &'db TestDb,
file_name: &str,
scopes: &[&str],
symbol_name: &str,
expected: &str,
) {
) -> Symbol<'db> {
let file = system_path_to_file(db, file_name).expect("file to exist");
let index = semantic_index(db, file);
let mut file_scope_id = FileScopeId::global();
@@ -4663,7 +4810,18 @@ mod tests {
assert_eq!(scope.name(db), *expected_scope_name);
}
let ty = symbol(db, scope, symbol_name).unwrap_or_unknown();
symbol(db, scope, symbol_name)
}
#[track_caller]
fn assert_scope_ty(
db: &TestDb,
file_name: &str,
scopes: &[&str],
symbol_name: &str,
expected: &str,
) {
let ty = get_symbol(db, file_name, scopes, symbol_name).expect_type();
assert_eq!(ty.display(db).to_string(), expected);
}
@@ -5343,9 +5501,10 @@ mod tests {
db.write_dedented("src/a.py", "[z for z in x]")?;
assert_scope_ty(&db, "src/a.py", &["<listcomp>"], "x", "Unknown");
let x = get_symbol(&db, "src/a.py", &["<listcomp>"], "x");
assert!(x.is_unbound());
// Iterating over an `Unbound` yields `Unknown`:
// Iterating over an unbound iterable yields `Unknown`:
assert_scope_ty(&db, "src/a.py", &["<listcomp>"], "z", "Unknown");
assert_file_diagnostics(&db, "src/a.py", &["Name `x` used when not defined"]);
@@ -5479,7 +5638,8 @@ mod tests {
",
)?;
assert_scope_ty(&db, "src/a.py", &["foo", "<listcomp>"], "z", "Unknown");
let z = get_symbol(&db, "src/a.py", &["foo", "<listcomp>"], "z");
assert!(z.is_unbound());
// (There is a diagnostic for invalid syntax that's emitted, but it's not listed by `assert_file_diagnostics`)
assert_file_diagnostics(&db, "src/a.py", &["Name `z` used when not defined"]);

View File

@@ -1,7 +1,6 @@
use std::collections::VecDeque;
use std::ops::Deref;
use indexmap::IndexSet;
use itertools::Either;
use rustc_hash::FxHashSet;
@@ -26,22 +25,30 @@ impl<'db> Mro<'db> {
/// (We emit a diagnostic warning about the runtime `TypeError` in
/// [`super::infer::TypeInferenceBuilder::infer_region_scope`].)
pub(super) fn of_class(db: &'db dyn Db, class: Class<'db>) -> Result<Self, MroError<'db>> {
Self::of_class_impl(db, class).map_err(|error_kind| {
let fallback_mro = Self::from([
ClassBase::Class(class),
ClassBase::Unknown,
ClassBase::object(db),
]);
MroError {
kind: error_kind,
fallback_mro,
}
Self::of_class_impl(db, class).map_err(|error_kind| MroError {
kind: error_kind,
fallback_mro: Self::from_error(db, class),
})
}
pub(super) fn from_error(db: &'db dyn Db, class: Class<'db>) -> Self {
Self::from([
ClassBase::Class(class),
ClassBase::Unknown,
ClassBase::object(db),
])
}
fn of_class_impl(db: &'db dyn Db, class: Class<'db>) -> Result<Self, MroErrorKind<'db>> {
let class_bases = class.explicit_bases(db);
if !class_bases.is_empty() && class.is_cyclically_defined(db) {
// We emit errors for cyclically defined classes elsewhere.
// It's important that we don't even try to infer the MRO for a cyclically defined class,
// or we'll end up in an infinite loop.
return Ok(Mro::from_error(db, class));
}
match class_bases {
// `builtins.object` is the special case:
// the only class in Python that has an MRO with length <2
@@ -77,11 +84,6 @@ impl<'db> Mro<'db> {
Err(MroErrorKind::InvalidBases(bases_info))
},
|single_base| {
if let ClassBase::Class(class_base) = single_base {
if class_is_cyclically_defined(db, class_base) {
return Err(MroErrorKind::CyclicClassDefinition);
}
}
let mro = std::iter::once(ClassBase::Class(class))
.chain(single_base.mro(db))
.collect();
@@ -96,10 +98,6 @@ impl<'db> Mro<'db> {
// what MRO Python will give this class at runtime
// (if an MRO is indeed resolvable at all!)
multiple_bases => {
if class_is_cyclically_defined(db, class) {
return Err(MroErrorKind::CyclicClassDefinition);
}
let mut valid_bases = vec![];
let mut invalid_bases = vec![];
@@ -282,13 +280,6 @@ pub(super) enum MroErrorKind<'db> {
/// Each index is the index of a node representing an invalid base.
InvalidBases(Box<[(usize, Type<'db>)]>),
/// The class inherits from itself!
///
/// This is very unlikely to happen in working real-world code,
/// but it's important to explicitly account for it.
/// If we don't, there's a possibility of an infinite loop and a panic.
CyclicClassDefinition,
/// The class has one or more duplicate bases.
///
/// This variant records the indices and [`Class`]es
@@ -349,7 +340,7 @@ impl<'db> ClassBase<'db> {
/// Return a `ClassBase` representing the class `builtins.object`
fn object(db: &'db dyn Db) -> Self {
KnownClass::Object
.to_class(db)
.to_class_literal(db)
.into_class_literal()
.map_or(Self::Unknown, |ClassLiteralType { class }| {
Self::Class(class)
@@ -381,6 +372,7 @@ impl<'db> ClassBase<'db> {
| Type::SubclassOf(_) => None,
Type::KnownInstance(known_instance) => match known_instance {
KnownInstanceType::Literal => None,
KnownInstanceType::TypeVar(_) => None,
},
}
}
@@ -414,7 +406,7 @@ impl<'db> From<ClassBase<'db>> for Type<'db> {
ClassBase::Any => Type::Any,
ClassBase::Todo => Type::Todo,
ClassBase::Unknown => Type::Unknown,
ClassBase::Class(class) => Type::ClassLiteral(ClassLiteralType { class }),
ClassBase::Class(class) => Type::class_literal(class),
}
}
}
@@ -460,46 +452,3 @@ fn c3_merge(mut sequences: Vec<VecDeque<ClassBase>>) -> Option<Mro> {
}
}
}
/// Return `true` if this class appears to be a cyclic definition,
/// i.e., it inherits either directly or indirectly from itself.
///
/// A class definition like this will fail at runtime,
/// but we must be resilient to it or we could panic.
fn class_is_cyclically_defined(db: &dyn Db, class: Class) -> bool {
fn is_cyclically_defined_recursive<'db>(
db: &'db dyn Db,
class: Class<'db>,
classes_to_watch: &mut IndexSet<Class<'db>>,
) -> bool {
if !classes_to_watch.insert(class) {
return true;
}
for explicit_base_class in class
.explicit_bases(db)
.iter()
.copied()
.filter_map(Type::into_class_literal)
.map(|ClassLiteralType { class }| class)
{
// Each base must be considered in isolation.
// This is due to the fact that if a class uses multiple inheritance,
// there could easily be a situation where two bases have the same class in their MROs;
// that isn't enough to constitute the class being cyclically defined.
let classes_to_watch_len = classes_to_watch.len();
if is_cyclically_defined_recursive(db, explicit_base_class, classes_to_watch) {
return true;
}
classes_to_watch.truncate(classes_to_watch_len);
}
false
}
class
.explicit_bases(db)
.iter()
.copied()
.filter_map(Type::into_class_literal)
.map(|ClassLiteralType { class }| class)
.any(|base_class| is_cyclically_defined_recursive(db, base_class, &mut IndexSet::default()))
}

View File

@@ -5,7 +5,7 @@ use crate::semantic_index::expression::Expression;
use crate::semantic_index::symbol::{ScopeId, ScopedSymbolId, SymbolTable};
use crate::semantic_index::symbol_table;
use crate::types::{
infer_expression_types, ClassLiteralType, InstanceType, IntersectionBuilder, KnownClass,
infer_expression_types, ClassLiteralType, IntersectionBuilder, KnownClass,
KnownConstraintFunction, KnownFunction, Truthiness, Type, UnionBuilder,
};
use crate::Db;
@@ -353,14 +353,12 @@ impl<'db> NarrowingConstraintsBuilder<'db> {
let to_constraint = match function {
KnownConstraintFunction::IsInstance => {
|class_literal: ClassLiteralType<'db>| {
Type::Instance(InstanceType {
class: class_literal.class,
})
Type::instance(class_literal.class)
}
}
KnownConstraintFunction::IsSubclass => {
|class_literal: ClassLiteralType<'db>| {
Type::SubclassOf(class_literal.to_subclass_of_type())
Type::subclass_of(class_literal.class)
}
}
};

View File

@@ -6,7 +6,7 @@ use rustc_hash::FxHashMap;
use crate::semantic_index::ast_ids::{HasScopedAstId, ScopedExpressionId};
use crate::semantic_index::symbol::ScopeId;
use crate::types::{TupleType, Type, TypeCheckDiagnostics, TypeCheckDiagnosticsBuilder};
use crate::types::{Type, TypeCheckDiagnostics, TypeCheckDiagnosticsBuilder};
use crate::Db;
/// Unpacks the value expression type to their respective targets.
@@ -93,11 +93,10 @@ impl<'db> Unpacker<'db> {
// further and deconstruct to an array of `StringLiteral` with each
// individual character, instead of just an array of `LiteralString`, but
// there would be a cost and it's not clear that it's worth it.
let value_ty = Type::Tuple(TupleType::new(
let value_ty = Type::tuple(
self.db,
vec![Type::LiteralString; string_literal_ty.len(self.db)]
.into_boxed_slice(),
));
&vec![Type::LiteralString; string_literal_ty.len(self.db)],
);
self.unpack(target, value_ty, scope);
}
_ => {

View File

@@ -1,7 +1,8 @@
use dir_test::{dir_test, Fixture};
use red_knot_test::run;
use std::ffi::OsStr;
use std::path::Path;
use dir_test::{dir_test, Fixture};
/// See `crates/red_knot_test/README.md` for documentation on these tests.
#[dir_test(
dir: "$CARGO_MANIFEST_DIR/resources/mdtest",
@@ -9,16 +10,16 @@ use std::path::Path;
)]
#[allow(clippy::needless_pass_by_value)]
fn mdtest(fixture: Fixture<&str>) {
let path = fixture.path();
let fixture_path = Path::new(fixture.path());
let crate_dir = Path::new(env!("CARGO_MANIFEST_DIR"));
let workspace_root = crate_dir.parent().and_then(Path::parent).unwrap();
let crate_dir = Path::new(env!("CARGO_MANIFEST_DIR"))
.join("resources/mdtest")
.canonicalize()
let long_title = fixture_path
.strip_prefix(workspace_root)
.unwrap()
.to_str()
.unwrap();
let short_title = fixture_path.file_name().and_then(OsStr::to_str).unwrap();
let relative_path = path
.strip_prefix(crate_dir.to_str().unwrap())
.unwrap_or(path);
run(Path::new(path), relative_path);
red_knot_test::run(fixture_path, long_title, short_title);
}

View File

@@ -19,9 +19,9 @@ mod parser;
///
/// Panic on test failure, and print failure details.
#[allow(clippy::print_stdout)]
pub fn run(path: &Path, title: &str) {
pub fn run(path: &Path, long_title: &str, short_title: &str) {
let source = std::fs::read_to_string(path).unwrap();
let suite = match test_parser::parse(title, &source) {
let suite = match test_parser::parse(short_title, &source) {
Ok(suite) => suite,
Err(err) => {
panic!("Error parsing `{}`: {err}", path.to_str().unwrap())
@@ -49,8 +49,8 @@ pub fn run(path: &Path, title: &str) {
for failure in failures {
let absolute_line_number =
backtick_line.checked_add(relative_line_number).unwrap();
let line_info = format!("{title}:{absolute_line_number}").cyan();
println!(" {line_info} {failure}");
let line_info = format!("{long_title}:{absolute_line_number}").cyan();
println!(" {line_info} {failure}");
}
}
}

View File

@@ -1,2 +1,5 @@
def foo[T: (str, bytes)](x: T) -> T:
...
def bar[T: (str,)](x: T) -> T:
...

View File

@@ -1 +1,2 @@
type foo = int
type ListOrSet[T] = list[T] | set[T]

View File

@@ -20,6 +20,7 @@ use tempfile::NamedTempFile;
use ruff_cache::{CacheKey, CacheKeyHasher};
use ruff_diagnostics::{DiagnosticKind, Fix};
use ruff_linter::message::{DiagnosticMessage, Message};
use ruff_linter::package::PackageRoot;
use ruff_linter::{warn_user, VERSION};
use ruff_macros::CacheKey;
use ruff_notebook::NotebookIndex;
@@ -497,7 +498,7 @@ pub(crate) struct PackageCacheMap<'a>(FxHashMap<&'a Path, Cache>);
impl<'a> PackageCacheMap<'a> {
pub(crate) fn init(
package_roots: &FxHashMap<&'a Path, Option<&'a Path>>,
package_roots: &FxHashMap<&'a Path, Option<PackageRoot<'a>>>,
resolver: &Resolver,
) -> Self {
fn init_cache(path: &Path) {
@@ -513,7 +514,9 @@ impl<'a> PackageCacheMap<'a> {
Self(
package_roots
.iter()
.map(|(package, package_root)| package_root.unwrap_or(package))
.map(|(package, package_root)| {
package_root.map(PackageRoot::path).unwrap_or(package)
})
.unique()
.par_bridge()
.map(|cache_root| {
@@ -587,6 +590,7 @@ mod tests {
use ruff_cache::CACHE_DIR_NAME;
use ruff_linter::message::Message;
use ruff_linter::package::PackageRoot;
use ruff_linter::settings::flags;
use ruff_linter::settings::types::UnsafeFixes;
use ruff_python_ast::PySourceType;
@@ -641,7 +645,7 @@ mod tests {
let diagnostics = lint_path(
&path,
Some(&package_root),
Some(PackageRoot::root(&package_root)),
&settings.linter,
Some(&cache),
flags::Noqa::Enabled,
@@ -683,7 +687,7 @@ mod tests {
for path in paths {
got_diagnostics += lint_path(
&path,
Some(&package_root),
Some(PackageRoot::root(&package_root)),
&settings.linter,
Some(&cache),
flags::Noqa::Enabled,
@@ -1056,7 +1060,7 @@ mod tests {
) -> Result<Diagnostics, anyhow::Error> {
lint_path(
&self.package_root.join(path),
Some(&self.package_root),
Some(PackageRoot::root(&self.package_root)),
&self.settings.linter,
Some(cache),
flags::Noqa::Enabled,

View File

@@ -6,6 +6,7 @@ use log::{debug, warn};
use path_absolutize::CWD;
use ruff_db::system::{SystemPath, SystemPathBuf};
use ruff_graph::{Direction, ImportMap, ModuleDb, ModuleImports};
use ruff_linter::package::PackageRoot;
use ruff_linter::{warn_user, warn_user_once};
use ruff_python_ast::{PySourceType, SourceType};
use ruff_workspace::resolver::{match_exclusion, python_files_in_path, ResolvedFile};
@@ -49,7 +50,12 @@ pub(crate) fn analyze_graph(
.collect::<Vec<_>>(),
)
.into_iter()
.map(|(path, package)| (path.to_path_buf(), package.map(Path::to_path_buf)))
.map(|(path, package)| {
(
path.to_path_buf(),
package.map(PackageRoot::path).map(Path::to_path_buf),
)
})
.collect::<FxHashMap<_, _>>();
// Create a database from the source roots.

View File

@@ -13,6 +13,7 @@ use rustc_hash::FxHashMap;
use ruff_diagnostics::Diagnostic;
use ruff_linter::message::Message;
use ruff_linter::package::PackageRoot;
use ruff_linter::registry::Rule;
use ruff_linter::settings::types::UnsafeFixes;
use ruff_linter::settings::{flags, LinterSettings};
@@ -87,7 +88,9 @@ pub(crate) fn check(
return None;
}
let cache_root = package.unwrap_or_else(|| path.parent().unwrap_or(path));
let cache_root = package
.map(PackageRoot::path)
.unwrap_or_else(|| path.parent().unwrap_or(path));
let cache = caches.get(cache_root);
lint_path(
@@ -181,7 +184,7 @@ pub(crate) fn check(
#[allow(clippy::too_many_arguments)]
fn lint_path(
path: &Path,
package: Option<&Path>,
package: Option<PackageRoot<'_>>,
settings: &LinterSettings,
cache: Option<&Cache>,
noqa: flags::Noqa,

View File

@@ -1,7 +1,7 @@
use std::path::Path;
use anyhow::Result;
use ruff_linter::package::PackageRoot;
use ruff_linter::packaging;
use ruff_linter::settings::flags;
use ruff_workspace::resolver::{match_exclusion, python_file_at_path, PyprojectConfig, Resolver};
@@ -42,6 +42,7 @@ pub(crate) fn check_stdin(
let stdin = read_from_stdin()?;
let package_root = filename.and_then(Path::parent).and_then(|path| {
packaging::detect_package_root(path, &resolver.base_settings().linter.namespace_packages)
.map(PackageRoot::root)
});
let mut diagnostics = lint_stdin(
filename,

View File

@@ -18,6 +18,7 @@ use tracing::debug;
use ruff_diagnostics::SourceMap;
use ruff_linter::fs;
use ruff_linter::logging::{DisplayParseError, LogLevel};
use ruff_linter::package::PackageRoot;
use ruff_linter::registry::Rule;
use ruff_linter::rules::flake8_quotes::settings::Quote;
use ruff_linter::source_kind::{SourceError, SourceKind};
@@ -136,7 +137,9 @@ pub(crate) fn format(
.parent()
.and_then(|parent| package_roots.get(parent).copied())
.flatten();
let cache_root = package.unwrap_or_else(|| path.parent().unwrap_or(path));
let cache_root = package
.map(PackageRoot::path)
.unwrap_or_else(|| path.parent().unwrap_or(path));
let cache = caches.get(cache_root);
Some(

View File

@@ -16,6 +16,7 @@ use ruff_diagnostics::Diagnostic;
use ruff_linter::codes::Rule;
use ruff_linter::linter::{lint_fix, lint_only, FixTable, FixerResult, LinterResult, ParseSource};
use ruff_linter::message::{Message, SyntaxErrorMessage};
use ruff_linter::package::PackageRoot;
use ruff_linter::pyproject_toml::lint_pyproject_toml;
use ruff_linter::settings::types::UnsafeFixes;
use ruff_linter::settings::{flags, LinterSettings};
@@ -180,7 +181,7 @@ impl AddAssign for FixMap {
/// Lint the source code at the given `Path`.
pub(crate) fn lint_path(
path: &Path,
package: Option<&Path>,
package: Option<PackageRoot<'_>>,
settings: &LinterSettings,
cache: Option<&Cache>,
noqa: flags::Noqa,
@@ -373,7 +374,7 @@ pub(crate) fn lint_path(
/// stdin.
pub(crate) fn lint_stdin(
path: Option<&Path>,
package: Option<&Path>,
package: Option<PackageRoot<'_>>,
contents: String,
settings: &Settings,
noqa: flags::Noqa,

View File

@@ -27,11 +27,11 @@ bitflags! {
#[derive(Default, Debug, Copy, Clone)]
pub(crate) struct Flags: u8 {
/// Whether to show violations when emitting diagnostics.
const SHOW_VIOLATIONS = 0b0000_0001;
const SHOW_VIOLATIONS = 1 << 0;
/// Whether to show a summary of the fixed violations when emitting diagnostics.
const SHOW_FIX_SUMMARY = 0b0000_0100;
const SHOW_FIX_SUMMARY = 1 << 1;
/// Whether to show a diff of each fixed violation when emitting diagnostics.
const SHOW_FIX_DIFF = 0b0000_1000;
const SHOW_FIX_DIFF = 1 << 2;
}
}

View File

@@ -8,6 +8,7 @@ use std::process::Command;
use std::str;
use anyhow::Result;
use assert_fs::fixture::{ChildPath, FileTouch, PathChild};
use insta_cmd::{assert_cmd_snapshot, get_cargo_bin};
use tempfile::TempDir;
@@ -1224,10 +1225,7 @@ fn negated_per_file_ignores_absolute() -> Result<()> {
let ignored = tempdir.path().join("ignored.py");
fs::write(ignored, "")?;
insta::with_settings!({filters => vec![
// Replace windows paths
(r"\\", "/"),
]}, {
insta::with_settings!({filters => vec![(r"\\", "/")]}, {
assert_cmd_snapshot!(Command::new(get_cargo_bin(BIN_NAME))
.args(STDIN_BASE_OPTIONS)
.arg("--config")
@@ -1918,3 +1916,58 @@ fn checks_notebooks_in_stable() -> anyhow::Result<()> {
"###);
Ok(())
}
/// Verify that implicit namespace packages are detected even when they are nested.
///
/// See: <https://github.com/astral-sh/ruff/issues/13519>
#[test]
fn nested_implicit_namespace_package() -> Result<()> {
let tempdir = TempDir::new()?;
let root = ChildPath::new(tempdir.path());
root.child("foo").child("__init__.py").touch()?;
root.child("foo")
.child("bar")
.child("baz")
.child("__init__.py")
.touch()?;
root.child("foo")
.child("bar")
.child("baz")
.child("bop.py")
.touch()?;
assert_cmd_snapshot!(Command::new(get_cargo_bin(BIN_NAME))
.args(STDIN_BASE_OPTIONS)
.arg("--select")
.arg("INP")
.current_dir(&tempdir)
, @r###"
success: true
exit_code: 0
----- stdout -----
All checks passed!
----- stderr -----
"###);
insta::with_settings!({filters => vec![(r"\\", "/")]}, {
assert_cmd_snapshot!(Command::new(get_cargo_bin(BIN_NAME))
.args(STDIN_BASE_OPTIONS)
.arg("--select")
.arg("INP")
.arg("--preview")
.current_dir(&tempdir)
, @r###"
success: false
exit_code: 1
----- stdout -----
foo/bar/baz/__init__.py:1:1: INP001 File `foo/bar/baz/__init__.py` declares a package, but is nested under an implicit namespace package. Add an `__init__.py` to `foo/bar`.
Found 1 error.
----- stderr -----
"###);
});
Ok(())
}

View File

@@ -375,6 +375,7 @@ linter.pylint.max_public_methods = 20
linter.pylint.max_locals = 15
linter.pyupgrade.keep_runtime_typing = false
linter.ruff.parenthesize_tuple_in_subscript = false
linter.ruff.extend_markup_names = []
# Formatter Settings
formatter.exclude = []

View File

@@ -94,7 +94,7 @@ class Registry:
object.__setattr__(self, "flag", True)
from typing import Optional, Union
from typing import Optional, Union, Self
def func(x: Union[list, Optional[int | str | float | bool]]):
@@ -154,3 +154,23 @@ from pydantic_settings import BaseSettings
class Settings(BaseSettings):
foo: bool = Field(True, exclude=True)
# https://github.com/astral-sh/ruff/issues/14202
class SupportsXorBool:
def __xor__(self, other: bool) -> Self: ...
# check overload
class CustomFloat:
@overload
def __mul__(self, other: bool) -> Self: ...
@overload
def __mul__(self, other: float) -> Self: ...
@overload
def __mul__(self, other: Self) -> Self: ...
# check union
class BooleanArray:
def __or__(self, other: Self | bool) -> Self: ...
def __ror__(self, other: Self | bool) -> Self: ...
def __ior__(self, other: Self | bool) -> Self: ...

View File

@@ -52,3 +52,38 @@ class CustomClassMethod:
# in the settings for this test:
@foo_classmethod
def foo[S](cls: type[S]) -> S: ... # PYI019
_S695 = TypeVar("_S695", bound="PEP695Fix")
# Only .pyi gets fixes, no fixes for .py
class PEP695Fix:
def __new__[S: PEP695Fix](cls: type[S]) -> S: ...
def __init_subclass__[S](cls: type[S]) -> S: ...
def __neg__[S: PEP695Fix](self: S) -> S: ...
def __pos__[S](self: S) -> S: ...
def __add__[S: PEP695Fix](self: S, other: S) -> S: ...
def __sub__[S](self: S, other: S) -> S: ...
@classmethod
def class_method_bound[S: PEP695Fix](cls: type[S]) -> S: ...
@classmethod
def class_method_unbound[S](cls: type[S]) -> S: ...
def instance_method_bound[S: PEP695Fix](self: S) -> S: ...
def instance_method_unbound[S](self: S) -> S: ...
def instance_method_bound_with_another_parameter[S: PEP695Fix](self: S, other: S) -> S: ...
def instance_method_unbound_with_another_parameter[S](self: S, other: S) -> S: ...
def multiple_type_vars[S, *Ts, T](self: S, other: S, /, *args: *Ts, a: T, b: list[T]) -> S: ...
def mixing_old_and_new_style_type_vars[T](self: _S695, a: T, b: T) -> _S695: ...

View File

@@ -52,3 +52,38 @@ class CustomClassMethod:
# in the settings for this test:
@foo_classmethod
def foo[S](cls: type[S]) -> S: ... # PYI019
_S695 = TypeVar("_S695", bound="PEP695Fix")
# Only .pyi gets fixes, no fixes for .py
class PEP695Fix:
def __new__[S: PEP695Fix](cls: type[S]) -> S: ...
def __init_subclass__[S](cls: type[S]) -> S: ...
def __neg__[S: PEP695Fix](self: S) -> S: ...
def __pos__[S](self: S) -> S: ...
def __add__[S: PEP695Fix](self: S, other: S) -> S: ...
def __sub__[S](self: S, other: S) -> S: ...
@classmethod
def class_method_bound[S: PEP695Fix](cls: type[S]) -> S: ...
@classmethod
def class_method_unbound[S](cls: type[S]) -> S: ...
def instance_method_bound[S: PEP695Fix](self: S) -> S: ...
def instance_method_unbound[S](self: S) -> S: ...
def instance_method_bound_with_another_parameter[S: PEP695Fix](self: S, other: S) -> S: ...
def instance_method_unbound_with_another_parameter[S](self: S, other: S) -> S: ...
def multiple_type_vars[S, *Ts, T](self: S, other: S, /, *args: *Ts, a: T, b: list[T]) -> S: ...
def mixing_old_and_new_style_type_vars[T](self: _S695, a: T, b: T) -> _S695: ...

View File

@@ -14,6 +14,12 @@ Literal[1, Literal[1], Literal[1]] # twice
Literal[1, Literal[2], Literal[2]] # once
t.Literal[1, t.Literal[2, t.Literal[1]]] # once
typing_extensions.Literal[1, 1, 1] # twice
Literal[
1, # comment
Literal[ # another comment
1
]
] # once
# Ensure issue is only raised once, even on nested literals
MyType = Literal["foo", Literal[True, False, True], "bar"] # PYI062

View File

@@ -2,11 +2,11 @@ from typing import Literal
import typing as t
import typing_extensions
x: Literal[True, False, True, False] # PY062 twice here
x: Literal[True, False, True, False] # PYI062 twice here
y: Literal[1, print("hello"), 3, Literal[4, 1]] # PY062 on the last 1
y: Literal[1, print("hello"), 3, Literal[4, 1]] # PYI062 on the last 1
z: Literal[{1, 3, 5}, "foobar", {1,3,5}] # PY062 on the set literal
z: Literal[{1, 3, 5}, "foobar", {1,3,5}] # PYI062 on the set literal
Literal[1, Literal[1]] # once
Literal[1, 2, Literal[1, 2]] # twice
@@ -14,6 +14,12 @@ Literal[1, Literal[1], Literal[1]] # twice
Literal[1, Literal[2], Literal[2]] # once
t.Literal[1, t.Literal[2, t.Literal[1]]] # once
typing_extensions.Literal[1, 1, 1] # twice
Literal[
1, # comment
Literal[ # another comment
1
]
] # once
# Ensure issue is only raised once, even on nested literals
MyType = Literal["foo", Literal[True, False, True], "bar"] # PYI062

View File

@@ -167,3 +167,29 @@ print(f"{a}{b}" or "bar")
print(f"{a}{''}" or "bar")
print(f"{''}{''}" or "bar")
print(f"{1}{''}" or "bar")
# Regression test for: https://github.com/astral-sh/ruff/issues/14237
for x in [*a] or [None]:
pass
for x in {*a} or [None]:
pass
for x in (*a,) or [None]:
pass
for x in {**a} or [None]:
pass
for x in [*a, *b] or [None]:
pass
for x in {*a, *b} or [None]:
pass
for x in (*a, *b) or [None]:
pass
for x in {**a, **b} or [None]:
pass

View File

@@ -87,3 +87,41 @@ def f():
result = []
async for i in items:
result.append(i) # PERF401
def f():
result, _ = [1,2,3,4], ...
for i in range(10):
result.append(i*2) # PERF401
def f():
result = []
if True:
for i in range(10): # single-line comment 1 should be protected
# single-line comment 2 should be protected
if i % 2: # single-line comment 3 should be protected
result.append(i) # PERF401
def f():
result = [] # comment after assignment should be protected
for i in range(10): # single-line comment 1 should be protected
# single-line comment 2 should be protected
if i % 2: # single-line comment 3 should be protected
result.append(i) # PERF401
def f():
result = []
for i in range(10):
"""block comment stops the fix"""
result.append(i*2) # Ok
def f(param):
# PERF401
# make sure the fix does not panic if there is no comments
if param:
new_layers = []
for value in param:
new_layers.append(value * 3)

View File

@@ -0,0 +1,42 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import asyncio\n",
"\n",
"await asyncio.sleep(1) # This is okay\n",
"\n",
"if True:\n",
" await asyncio.sleep(1) # This is okay\n",
"\n",
"def foo():\n",
" await asyncio.sleep(1) # # [await-outside-async]"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "base",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.5"
}
},
"nbformat": 4,
"nbformat_minor": 2
}

View File

@@ -1,13 +1,16 @@
# pylint: disable=missing-docstring,unused-variable
import asyncio
async def nested():
return 42
async def main():
nested()
print(await nested()) # This is okay
def not_async():
print(await nested()) # [await-outside-async]
@@ -15,6 +18,7 @@ def not_async():
async def func(i):
return i**2
async def okay_function():
var = [await func(i) for i in range(5)] # This should be okay
@@ -28,3 +32,43 @@ async def func2():
def outer_func():
async def inner_func():
await asyncio.sleep(1)
def async_for_loop():
async for x in foo():
pass
def async_with():
async with foo():
pass
# See: https://github.com/astral-sh/ruff/issues/14167
def async_for_generator_elt():
(await x for x in foo())
# See: https://github.com/astral-sh/ruff/issues/14167
def async_for_list_comprehension_elt():
[await x for x in foo()]
# See: https://github.com/astral-sh/ruff/issues/14167
def async_for_list_comprehension():
[x async for x in foo()]
# See: https://github.com/astral-sh/ruff/issues/14167
def await_generator_iter():
(x for x in await foo())
# See: https://github.com/astral-sh/ruff/issues/14167
def await_generator_target():
(x async for x in foo())
# See: https://github.com/astral-sh/ruff/issues/14167
def async_for_list_comprehension_target():
[x for x in await foo()]

View File

@@ -0,0 +1,4 @@
import copy
import os
copied_env = copy.copy(os.environ) # [shallow-copy-environ]

View File

@@ -6,6 +6,7 @@ open("foo", "r")
open("foo", "rt")
open("f", "r", encoding="UTF-8")
open("f", "wt")
open("f", "tw")
with open("foo", "U") as f:
pass
@@ -69,19 +70,14 @@ open(file="foo", buffering=-1, encoding=None, errors=None, newline=None, closefd
open(file="foo", buffering=-1, encoding=None, errors=None, mode='Ub', newline=None, closefd=True, opener=None)
open(mode='Ub', file="foo", buffering=-1, encoding=None, errors=None, newline=None, closefd=True, opener=None)
open = 1
open("foo", "U")
open("foo", "Ur")
open("foo", "Ub")
open("foo", "rUb")
open("foo", "r")
open("foo", "rt")
open("f", "r", encoding="UTF-8")
open("f", "wt")
import aiofiles
aiofiles.open("foo", "U")
aiofiles.open("foo", "r")
aiofiles.open("foo", mode="r")
open("foo", "r+")
open("foo", "rb")
open("foo", "r+b")
open("foo", "UU")
open("foo", "wtt")

View File

@@ -1,27 +1,58 @@
from typing import Generic, TypeVarTuple, Unpack
Shape = TypeVarTuple('Shape')
Shape = TypeVarTuple("Shape")
class C(Generic[Unpack[Shape]]):
pass
class D(Generic[Unpack [Shape]]):
class D(Generic[Unpack[Shape]]):
pass
def f(*args: Unpack[tuple[int, ...]]): pass
def f(*args: Unpack[other.Type]): pass
def f(*args: Unpack[tuple[int, ...]]):
pass
# Not valid unpackings but they are valid syntax
def foo(*args: Unpack[int | str]) -> None: pass
def foo(*args: Unpack[int and str]) -> None: pass
def foo(*args: Unpack[int > str]) -> None: pass
def f(*args: Unpack[other.Type]):
pass
def f(*args: Generic[int, Unpack[int]]):
pass
# Valid syntax, but can't be unpacked.
def f(*args: Unpack[int | str]) -> None:
pass
def f(*args: Unpack[int and str]) -> None:
pass
def f(*args: Unpack[int > str]) -> None:
pass
# We do not use the shorthand unpacking syntax in the following cases
from typing import TypedDict
class KwargsDict(TypedDict):
x: int
y: int
def foo(name: str, /, **kwargs: Unpack[KwargsDict]) -> None: pass
# OK
def f(name: str, /, **kwargs: Unpack[KwargsDict]) -> None:
pass
# OK
def f() -> object:
return Unpack[tuple[int, ...]]
# OK
def f(x: Unpack[int]) -> object: ...

View File

@@ -9,8 +9,6 @@ _ = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"
_ = r"""!"#$%&'()*+,-./:;<=>?@[\]^_`{|}~"""
_ = " \t\n\r\v\f"
_ = "" in "1234567890"
_ = "" in "12345670"
_ = '0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ!"#$%&\'()*+,-./:;<=>?@[\\]^_`{|}~ \t\n\r\x0b\x0c'
_ = (
'0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ!"#$%&'
@@ -19,23 +17,6 @@ _ = (
_ = id("0123"
"4567"
"89")
_ = "" in ("123"
"456"
"789"
"0")
_ = "" in ( # comment
"123"
"456"
"789"
"0")
_ = "" in (
"123"
"456" # inline comment
"789"
"0")
_ = (
"0123456789"
@@ -46,8 +27,8 @@ _ = (
# with comment
).capitalize()
# Ok
# OK
_ = "1234567890"
_ = "1234"
_ = "" in "1234"
_ = "12" in "12345670"

View File

@@ -34,4 +34,11 @@ Decimal("1.2")
# Ok: even though this is equal to `Decimal(123)`,
# we assume that a developer would
# only write it this way if they meant to.
Decimal("١٢٣")
Decimal("١٢٣")
# Further subtleties
# https://github.com/astral-sh/ruff/issues/14204
Decimal("-0") # Ok
Decimal("_") # Ok
Decimal(" ") # Ok
Decimal("10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000") # Ok

View File

@@ -0,0 +1,21 @@
# https://github.com/astral-sh/ruff/issues/13833
from typing import Optional
def no_default(arg: Optional): ...
def has_default(arg: Optional = None): ...
def multiple_1(arg1: Optional, arg2: Optional = None): ...
def multiple_2(arg1: Optional, arg2: Optional = None, arg3: int = None): ...
def return_type(arg: Optional = None) -> Optional: ...
def has_type_argument(arg: Optional[int] = None): ...

View File

@@ -0,0 +1,18 @@
import flask
from markupsafe import Markup, escape
content = "<script>alert('Hello, world!')</script>"
Markup(f"unsafe {content}") # RUF035
flask.Markup("unsafe {}".format(content)) # RUF035
Markup("safe {}").format(content)
flask.Markup(b"safe {}", encoding='utf-8').format(content)
escape(content)
Markup(content) # RUF035
flask.Markup("unsafe %s" % content) # RUF035
Markup(object="safe")
Markup(object="unsafe {}".format(content)) # Not currently detected
# NOTE: We may be able to get rid of these false positives with red-knot
# if it includes comprehensive constant expression detection/evaluation.
Markup("*" * 8) # RUF035 (false positive)
flask.Markup("hello {}".format("world")) # RUF035 (false positive)

View File

@@ -0,0 +1,6 @@
from markupsafe import Markup
from webhelpers.html import literal
content = "<script>alert('Hello, world!')</script>"
Markup(f"unsafe {content}") # RUF035
literal(f"unsafe {content}") # RUF035

View File

@@ -0,0 +1,7 @@
from webhelpers.html import literal
# NOTE: This test case exists to make sure our optimization doesn't cause
# additional markup names to be skipped if we don't import either
# markupsafe or flask first.
content = "<script>alert('Hello, world!')</script>"
literal(f"unsafe {content}") # RUF035

View File

@@ -2,7 +2,7 @@ use ruff_python_ast::Comprehension;
use crate::checkers::ast::Checker;
use crate::codes::Rule;
use crate::rules::{flake8_simplify, refurb};
use crate::rules::{flake8_simplify, pylint, refurb};
/// Run lint rules over a [`Comprehension`] syntax nodes.
pub(crate) fn comprehension(comprehension: &Comprehension, checker: &mut Checker) {
@@ -12,4 +12,9 @@ pub(crate) fn comprehension(comprehension: &Comprehension, checker: &mut Checker
if checker.enabled(Rule::ReadlinesInFor) {
refurb::rules::readlines_in_comprehension(checker, comprehension);
}
if comprehension.is_async {
if checker.enabled(Rule::AwaitOutsideAsync) {
pylint::rules::await_outside_async(checker, comprehension);
}
}
}

View File

@@ -145,11 +145,9 @@ pub(crate) fn expression(expr: &Expr, checker: &mut Checker) {
if checker.enabled(Rule::FStringNumberFormat) {
refurb::rules::fstring_number_format(checker, subscript);
}
if checker.enabled(Rule::IncorrectlyParenthesizedTupleInSubscript) {
ruff::rules::subscript_with_parenthesized_tuple(checker, subscript);
}
if checker.enabled(Rule::NonPEP646Unpack) {
pyupgrade::rules::use_pep646_unpack(checker, subscript);
}
@@ -819,6 +817,9 @@ pub(crate) fn expression(expr: &Expr, checker: &mut Checker) {
if checker.enabled(Rule::BadStrStripCall) {
pylint::rules::bad_str_strip_call(checker, func, args);
}
if checker.enabled(Rule::ShallowCopyEnviron) {
pylint::rules::shallow_copy_environ(checker, call);
}
if checker.enabled(Rule::InvalidEnvvarDefault) {
pylint::rules::invalid_envvar_default(checker, call);
}
@@ -1029,6 +1030,9 @@ pub(crate) fn expression(expr: &Expr, checker: &mut Checker) {
if checker.enabled(Rule::IntOnSlicedStr) {
refurb::rules::int_on_sliced_str(checker, call);
}
if checker.enabled(Rule::UnsafeMarkupUse) {
ruff::rules::unsafe_markup_call(checker, call);
}
}
Expr::Dict(dict) => {
if checker.any_enabled(&[
@@ -1371,9 +1375,6 @@ pub(crate) fn expression(expr: &Expr, checker: &mut Checker) {
if checker.enabled(Rule::SingleItemMembershipTest) {
refurb::rules::single_item_membership_test(checker, expr, left, ops, comparators);
}
if checker.enabled(Rule::HardcodedStringCharset) {
refurb::rules::hardcoded_string_charset_comparison(checker, compare);
}
}
Expr::NumberLiteral(number_literal @ ast::ExprNumberLiteral { .. }) => {
if checker.source_type.is_stub() && checker.enabled(Rule::NumericLiteralTooLong) {

View File

@@ -81,7 +81,7 @@ pub(crate) fn statement(stmt: &Stmt, checker: &mut Checker) {
returns,
parameters,
body,
type_params,
type_params: _,
range: _,
},
) => {
@@ -160,14 +160,7 @@ pub(crate) fn statement(stmt: &Stmt, checker: &mut Checker) {
flake8_pyi::rules::bad_generator_return_type(function_def, checker);
}
if checker.enabled(Rule::CustomTypeVarReturnType) {
flake8_pyi::rules::custom_type_var_return_type(
checker,
name,
decorator_list,
returns.as_ref().map(AsRef::as_ref),
parameters,
type_params.as_deref(),
);
flake8_pyi::rules::custom_type_var_return_type(checker, function_def);
}
if checker.source_type.is_stub() {
if checker.enabled(Rule::StrOrReprDefinedInStub) {
@@ -1303,7 +1296,14 @@ pub(crate) fn statement(stmt: &Stmt, checker: &mut Checker) {
ruff::rules::assert_with_print_message(checker, assert_stmt);
}
}
Stmt::With(with_stmt @ ast::StmtWith { items, body, .. }) => {
Stmt::With(
with_stmt @ ast::StmtWith {
items,
body,
is_async,
..
},
) => {
if checker.enabled(Rule::TooManyNestedBlocks) {
pylint::rules::too_many_nested_blocks(checker, stmt);
}
@@ -1335,6 +1335,11 @@ pub(crate) fn statement(stmt: &Stmt, checker: &mut Checker) {
if checker.enabled(Rule::CancelScopeNoCheckpoint) {
flake8_async::rules::cancel_scope_no_checkpoint(checker, with_stmt, items);
}
if *is_async {
if checker.enabled(Rule::AwaitOutsideAsync) {
pylint::rules::await_outside_async(checker, stmt);
}
}
}
Stmt::While(while_stmt @ ast::StmtWhile { body, orelse, .. }) => {
if checker.enabled(Rule::TooManyNestedBlocks) {
@@ -1422,7 +1427,11 @@ pub(crate) fn statement(stmt: &Stmt, checker: &mut Checker) {
if checker.enabled(Rule::ReadlinesInFor) {
refurb::rules::readlines_in_for(checker, for_stmt);
}
if !is_async {
if *is_async {
if checker.enabled(Rule::AwaitOutsideAsync) {
pylint::rules::await_outside_async(checker, stmt);
}
} else {
if checker.enabled(Rule::ReimplementedBuiltin) {
flake8_simplify::rules::convert_for_loop_to_any_all(checker, stmt);
}

View File

@@ -53,9 +53,9 @@ use ruff_python_parser::{Parsed, Tokens};
use ruff_python_semantic::all::{DunderAllDefinition, DunderAllFlags};
use ruff_python_semantic::analyze::{imports, typing};
use ruff_python_semantic::{
BindingFlags, BindingId, BindingKind, Exceptions, Export, FromImport, Globals, Import, Module,
ModuleKind, ModuleSource, NodeId, ScopeId, ScopeKind, SemanticModel, SemanticModelFlags,
StarImport, SubmoduleImport,
BindingFlags, BindingId, BindingKind, Exceptions, Export, FromImport, GeneratorKind, Globals,
Import, Module, ModuleKind, ModuleSource, NodeId, ScopeId, ScopeKind, SemanticModel,
SemanticModelFlags, StarImport, SubmoduleImport,
};
use ruff_python_stdlib::builtins::{python_builtins, MAGIC_GLOBALS};
use ruff_python_trivia::CommentRanges;
@@ -66,6 +66,7 @@ use crate::checkers::ast::annotation::AnnotationContext;
use crate::docstrings::extraction::ExtractionTarget;
use crate::importer::Importer;
use crate::noqa::NoqaMapping;
use crate::package::PackageRoot;
use crate::registry::Rule;
use crate::rules::{flake8_pyi, flake8_type_checking, pyflakes, pyupgrade};
use crate::settings::{flags, LinterSettings};
@@ -186,7 +187,7 @@ pub(crate) struct Checker<'a> {
/// The [`Path`] to the file under analysis.
path: &'a Path,
/// The [`Path`] to the package containing the current file.
package: Option<&'a Path>,
package: Option<PackageRoot<'a>>,
/// The module representation of the current file (e.g., `foo.bar`).
module: Module<'a>,
/// The [`PySourceType`] of the current file.
@@ -238,7 +239,7 @@ impl<'a> Checker<'a> {
noqa_line_for: &'a NoqaMapping,
noqa: flags::Noqa,
path: &'a Path,
package: Option<&'a Path>,
package: Option<PackageRoot<'a>>,
module: Module<'a>,
locator: &'a Locator,
stylist: &'a Stylist,
@@ -247,7 +248,7 @@ impl<'a> Checker<'a> {
cell_offsets: Option<&'a CellOffsets>,
notebook_index: Option<&'a NotebookIndex>,
) -> Checker<'a> {
Checker {
Self {
parsed,
parsed_type_annotation: None,
parsed_annotations_cache: ParsedAnnotationsCache::new(parsed_annotations_arena),
@@ -383,7 +384,7 @@ impl<'a> Checker<'a> {
}
/// The [`Path`] to the package containing the current file.
pub(crate) const fn package(&self) -> Option<&'a Path> {
pub(crate) const fn package(&self) -> Option<PackageRoot<'_>> {
self.package
}
@@ -1137,19 +1138,25 @@ impl<'a> Visitor<'a> for Checker<'a> {
elt,
generators,
range: _,
})
| Expr::SetComp(ast::ExprSetComp {
}) => {
self.visit_generators(GeneratorKind::ListComprehension, generators);
self.visit_expr(elt);
}
Expr::SetComp(ast::ExprSetComp {
elt,
generators,
range: _,
})
| Expr::Generator(ast::ExprGenerator {
}) => {
self.visit_generators(GeneratorKind::SetComprehension, generators);
self.visit_expr(elt);
}
Expr::Generator(ast::ExprGenerator {
elt,
generators,
range: _,
parenthesized: _,
}) => {
self.visit_generators(generators);
self.visit_generators(GeneratorKind::Generator, generators);
self.visit_expr(elt);
}
Expr::DictComp(ast::ExprDictComp {
@@ -1158,7 +1165,7 @@ impl<'a> Visitor<'a> for Checker<'a> {
generators,
range: _,
}) => {
self.visit_generators(generators);
self.visit_generators(GeneratorKind::DictComprehension, generators);
self.visit_expr(key);
self.visit_expr(value);
}
@@ -1748,7 +1755,7 @@ impl<'a> Checker<'a> {
/// Visit a list of [`Comprehension`] nodes, assumed to be the comprehensions that compose a
/// generator expression, like a list or set comprehension.
fn visit_generators(&mut self, generators: &'a [Comprehension]) {
fn visit_generators(&mut self, kind: GeneratorKind, generators: &'a [Comprehension]) {
let mut iterator = generators.iter();
let Some(generator) = iterator.next() else {
@@ -1785,7 +1792,7 @@ impl<'a> Checker<'a> {
// while all subsequent reads and writes are evaluated in the inner scope. In particular,
// `x` is local to `foo`, and the `T` in `y=T` skips the class scope when resolving.
self.visit_expr(&generator.iter);
self.semantic.push_scope(ScopeKind::Generator);
self.semantic.push_scope(ScopeKind::Generator(kind));
self.semantic.flags = flags | SemanticModelFlags::COMPREHENSION_ASSIGNMENT;
self.visit_expr(&generator.target);
@@ -2477,12 +2484,14 @@ pub(crate) fn check_ast(
settings: &LinterSettings,
noqa: flags::Noqa,
path: &Path,
package: Option<&Path>,
package: Option<PackageRoot<'_>>,
source_type: PySourceType,
cell_offsets: Option<&CellOffsets>,
notebook_index: Option<&NotebookIndex>,
) -> Vec<Diagnostic> {
let module_path = package.and_then(|package| to_module_path(package, path));
let module_path = package
.map(PackageRoot::path)
.and_then(|package| to_module_path(package, path));
let module = Module {
kind: if path.ends_with("__init__.py") {
ModuleKind::Package

View File

@@ -3,6 +3,7 @@ use std::path::Path;
use ruff_diagnostics::Diagnostic;
use ruff_python_trivia::CommentRanges;
use crate::package::PackageRoot;
use crate::registry::Rule;
use crate::rules::flake8_builtins::rules::builtin_module_shadowing;
use crate::rules::flake8_no_pep420::rules::implicit_namespace_package;
@@ -12,7 +13,7 @@ use crate::Locator;
pub(crate) fn check_file_path(
path: &Path,
package: Option<&Path>,
package: Option<PackageRoot<'_>>,
locator: &Locator,
comment_ranges: &CommentRanges,
settings: &LinterSettings,
@@ -28,6 +29,7 @@ pub(crate) fn check_file_path(
comment_ranges,
&settings.project_root,
&settings.src,
settings.preview,
) {
diagnostics.push(diagnostic);
}

View File

@@ -1,5 +1,4 @@
//! Lint rules based on import analysis.
use std::path::Path;
use ruff_diagnostics::Diagnostic;
use ruff_notebook::CellOffsets;
@@ -10,6 +9,7 @@ use ruff_python_index::Indexer;
use ruff_python_parser::Parsed;
use crate::directives::IsortDirectives;
use crate::package::PackageRoot;
use crate::registry::Rule;
use crate::rules::isort;
use crate::rules::isort::block::{Block, BlockBuilder};
@@ -24,7 +24,7 @@ pub(crate) fn check_imports(
directives: &IsortDirectives,
settings: &LinterSettings,
stylist: &Stylist,
package: Option<&Path>,
package: Option<PackageRoot<'_>>,
source_type: PySourceType,
cell_offsets: Option<&CellOffsets>,
) -> Vec<Diagnostic> {

View File

@@ -283,6 +283,7 @@ pub fn code_to_rule(linter: Linter, code: &str) -> Option<(RuleGroup, Rule)> {
(Pylint, "W0642") => (RuleGroup::Stable, rules::pylint::rules::SelfOrClsAssignment),
(Pylint, "W0711") => (RuleGroup::Stable, rules::pylint::rules::BinaryOpException),
(Pylint, "W1501") => (RuleGroup::Stable, rules::pylint::rules::BadOpenMode),
(Pylint, "W1507") => (RuleGroup::Preview, rules::pylint::rules::ShallowCopyEnviron),
(Pylint, "W1508") => (RuleGroup::Stable, rules::pylint::rules::InvalidEnvvarDefault),
(Pylint, "W1509") => (RuleGroup::Stable, rules::pylint::rules::SubprocessPopenPreexecFn),
(Pylint, "W1510") => (RuleGroup::Stable, rules::pylint::rules::SubprocessRunWithoutCheck),
@@ -965,6 +966,7 @@ pub fn code_to_rule(linter: Linter, code: &str) -> Option<(RuleGroup, Rule)> {
(Ruff, "032") => (RuleGroup::Preview, rules::ruff::rules::DecimalFromFloatLiteral),
(Ruff, "033") => (RuleGroup::Preview, rules::ruff::rules::PostInitDefault),
(Ruff, "034") => (RuleGroup::Preview, rules::ruff::rules::UselessIfElse),
(Ruff, "035") => (RuleGroup::Preview, rules::ruff::rules::UnsafeMarkupUse),
(Ruff, "100") => (RuleGroup::Stable, rules::ruff::rules::UnusedNOQA),
(Ruff, "101") => (RuleGroup::Stable, rules::ruff::rules::RedirectedNOQA),

View File

@@ -18,8 +18,8 @@ use crate::Locator;
bitflags! {
#[derive(Debug, Copy, Clone)]
pub struct Flags: u8 {
const NOQA = 0b0000_0001;
const ISORT = 0b0000_0010;
const NOQA = 1 << 0;
const ISORT = 1 << 1;
}
}

View File

@@ -32,6 +32,7 @@ mod locator;
pub mod logging;
pub mod message;
mod noqa;
pub mod package;
pub mod packaging;
pub mod pyproject_toml;
pub mod registry;

View File

@@ -28,6 +28,7 @@ use crate::doc_lines::{doc_lines_from_ast, doc_lines_from_tokens};
use crate::fix::{fix_file, FixResult};
use crate::message::Message;
use crate::noqa::add_noqa;
use crate::package::PackageRoot;
use crate::registry::{AsRule, Rule, RuleSet};
#[cfg(any(feature = "test-rules", test))]
use crate::rules::ruff::rules::test_rules::{self, TestRule, TEST_RULES};
@@ -60,7 +61,7 @@ pub struct FixerResult<'a> {
#[allow(clippy::too_many_arguments)]
pub fn check_path(
path: &Path,
package: Option<&Path>,
package: Option<PackageRoot<'_>>,
locator: &Locator,
stylist: &Stylist,
indexer: &Indexer,
@@ -323,7 +324,7 @@ const MAX_ITERATIONS: usize = 100;
/// Add any missing `# noqa` pragmas to the source code at the given `Path`.
pub fn add_noqa_to_path(
path: &Path,
package: Option<&Path>,
package: Option<PackageRoot<'_>>,
source_kind: &SourceKind,
source_type: PySourceType,
settings: &LinterSettings,
@@ -380,7 +381,7 @@ pub fn add_noqa_to_path(
/// code.
pub fn lint_only(
path: &Path,
package: Option<&Path>,
package: Option<PackageRoot<'_>>,
settings: &LinterSettings,
noqa: flags::Noqa,
source_kind: &SourceKind,
@@ -467,7 +468,7 @@ fn diagnostics_to_messages(
#[allow(clippy::too_many_arguments)]
pub fn lint_fix<'a>(
path: &Path,
package: Option<&Path>,
package: Option<PackageRoot<'_>>,
noqa: flags::Noqa,
unsafe_fixes: UnsafeFixes,
settings: &LinterSettings,

View File

@@ -22,11 +22,11 @@ bitflags! {
#[derive(Default)]
struct EmitterFlags: u8 {
/// Whether to show the fix status of a diagnostic.
const SHOW_FIX_STATUS = 0b0000_0001;
const SHOW_FIX_STATUS = 1 << 0;
/// Whether to show the diff of a fix, for diagnostics that have a fix.
const SHOW_FIX_DIFF = 0b0000_0010;
const SHOW_FIX_DIFF = 1 << 1;
/// Whether to show the source code of a diagnostic.
const SHOW_SOURCE = 0b0000_0100;
const SHOW_SOURCE = 1 << 2;
}
}

View File

@@ -183,9 +183,11 @@ impl<'a> Directive<'a> {
// Extract, e.g., the `401` in `F401`.
let suffix = line[prefix..]
.chars()
.take_while(char::is_ascii_alphanumeric)
.take_while(char::is_ascii_digit)
.count();
if prefix > 0 && suffix > 0 {
// SAFETY: we can use `prefix` and `suffix` to index into `line` because we know that
// all characters in `line` are ASCII, i.e., a single byte.
Some(&line[..prefix + suffix])
} else {
None
@@ -1209,6 +1211,12 @@ mod tests {
assert_debug_snapshot!(Directive::try_extract(source, TextSize::default()));
}
#[test]
fn noqa_non_code() {
let source = "# noqa: F401 We're ignoring an import";
assert_debug_snapshot!(Directive::try_extract(source, TextSize::default()));
}
#[test]
fn noqa_invalid_suffix() {
let source = "# noqa[F401]";

View File

@@ -0,0 +1,40 @@
use std::path::Path;
/// The root directory of a Python package.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum PackageRoot<'a> {
/// A normal package root.
Root { path: &'a Path },
/// A nested package root. That is, a package root that's a subdirectory (direct or indirect) of
/// another Python package root.
///
/// For example, `foo/bar/baz` in:
/// ```text
/// foo/
/// ├── __init__.py
/// └── bar/
/// └── baz/
/// └── __init__.py
/// ```
Nested { path: &'a Path },
}
impl<'a> PackageRoot<'a> {
/// Create a [`PackageRoot::Root`] variant.
pub fn root(path: &'a Path) -> Self {
Self::Root { path }
}
/// Create a [`PackageRoot::Nested`] variant.
pub fn nested(path: &'a Path) -> Self {
Self::Nested { path }
}
/// Return the [`Path`] of the package root.
pub fn path(self) -> &'a Path {
match self {
Self::Root { path } => path,
Self::Nested { path } => path,
}
}
}

View File

@@ -66,9 +66,88 @@ pub(super) fn is_user_allowed_func_call(
})
}
/// Returns `true` if a function defines a binary operator.
///
/// This only includes operators, i.e., functions that are usually not called directly.
///
/// See: <https://docs.python.org/3/library/operator.html>
pub(super) fn is_operator_method(name: &str) -> bool {
matches!(
name,
"__contains__" // in
// item access ([])
| "__getitem__" // []
| "__setitem__" // []=
| "__delitem__" // del []
// addition (+)
| "__add__" // +
| "__radd__" // +
| "__iadd__" // +=
// subtraction (-)
| "__sub__" // -
| "__rsub__" // -
| "__isub__" // -=
// multiplication (*)
| "__mul__" // *
| "__rmul__" // *
| "__imul__" // *=
// division (/)
| "__truediv__" // /
| "__rtruediv__" // /
| "__itruediv__" // /=
// floor division (//)
| "__floordiv__" // //
| "__rfloordiv__" // //
| "__ifloordiv__" // //=
// remainder (%)
| "__mod__" // %
| "__rmod__" // %
| "__imod__" // %=
// exponentiation (**)
| "__pow__" // **
| "__rpow__" // **
| "__ipow__" // **=
// left shift (<<)
| "__lshift__" // <<
| "__rlshift__" // <<
| "__ilshift__" // <<=
// right shift (>>)
| "__rshift__" // >>
| "__rrshift__" // >>
| "__irshift__" // >>=
// matrix multiplication (@)
| "__matmul__" // @
| "__rmatmul__" // @
| "__imatmul__" // @=
// meet (&)
| "__and__" // &
| "__rand__" // &
| "__iand__" // &=
// join (|)
| "__or__" // |
| "__ror__" // |
| "__ior__" // |=
// xor (^)
| "__xor__" // ^
| "__rxor__" // ^
| "__ixor__" // ^=
// comparison (>, <, >=, <=, ==, !=)
| "__gt__" // >
| "__lt__" // <
| "__ge__" // >=
| "__le__" // <=
| "__eq__" // ==
| "__ne__" // !=
// unary operators (included for completeness)
| "__pos__" // +
| "__neg__" // -
| "__invert__" // ~
)
}
/// Returns `true` if a function definition is allowed to use a boolean trap.
pub(super) fn is_allowed_func_def(name: &str) -> bool {
matches!(name, "__setitem__" | "__post_init__")
matches!(name, "__post_init__") || is_operator_method(name)
}
/// Returns `true` if an argument is allowed to use a boolean trap. To return

View File

@@ -27,6 +27,9 @@ use crate::rules::flake8_boolean_trap::helpers::is_allowed_func_def;
/// keyword-only argument, to force callers to be explicit when providing
/// the argument.
///
/// Dunder methods that define operators are exempt from this rule, as are
/// setters and `@override` definitions.
///
/// In [preview], this rule will also flag annotations that include boolean
/// variants, like `bool | int`.
///

View File

@@ -1,5 +1,7 @@
use std::path::Path;
use crate::package::PackageRoot;
use crate::settings::types::PythonVersion;
use ruff_diagnostics::{Diagnostic, Violation};
use ruff_macros::{derive_message_formats, violation};
use ruff_python_ast::PySourceType;
@@ -7,8 +9,6 @@ use ruff_python_stdlib::path::is_module_file;
use ruff_python_stdlib::sys::is_known_standard_library;
use ruff_text_size::TextRange;
use crate::settings::types::PythonVersion;
/// ## What it does
/// Checks for modules that use the same names as Python builtin modules.
///
@@ -39,7 +39,7 @@ impl Violation for BuiltinModuleShadowing {
/// A005
pub(crate) fn builtin_module_shadowing(
path: &Path,
package: Option<&Path>,
package: Option<PackageRoot<'_>>,
allowed_modules: &[String],
target_version: PythonVersion,
) -> Option<Diagnostic> {
@@ -49,7 +49,7 @@ pub(crate) fn builtin_module_shadowing(
if let Some(package) = package {
let module_name = if is_module_file(path) {
package.file_name().unwrap().to_string_lossy()
package.path().file_name().unwrap().to_string_lossy()
} else {
path.file_stem().unwrap().to_string_lossy()
};

View File

@@ -8,8 +8,9 @@ mod tests {
use anyhow::Result;
use test_case::test_case;
use crate::assert_messages;
use crate::registry::Rule;
use crate::assert_messages;
use crate::settings::LinterSettings;
use crate::test::{test_path, test_resource_path};
@@ -22,7 +23,7 @@ mod tests {
#[test_case(Path::new("test_pass_pyi"), Path::new("example.pyi"))]
#[test_case(Path::new("test_pass_script"), Path::new("script"))]
#[test_case(Path::new("test_pass_shebang"), Path::new("example.py"))]
fn test_flake8_no_pep420(path: &Path, filename: &Path) -> Result<()> {
fn default(path: &Path, filename: &Path) -> Result<()> {
let snapshot = format!("{}", path.to_string_lossy());
let p = PathBuf::from(format!(
"flake8_no_pep420/{}/{}",

View File

@@ -9,6 +9,8 @@ use ruff_text_size::{TextRange, TextSize};
use crate::comments::shebang::ShebangDirective;
use crate::fs;
use crate::package::PackageRoot;
use crate::settings::types::PreviewMode;
use crate::Locator;
/// ## What it does
@@ -32,24 +34,33 @@ use crate::Locator;
#[violation]
pub struct ImplicitNamespacePackage {
filename: String,
parent: Option<String>,
}
impl Violation for ImplicitNamespacePackage {
#[derive_message_formats]
fn message(&self) -> String {
let ImplicitNamespacePackage { filename } = self;
format!("File `{filename}` is part of an implicit namespace package. Add an `__init__.py`.")
let ImplicitNamespacePackage { filename, parent } = self;
match parent {
None => {
format!("File `{filename}` is part of an implicit namespace package. Add an `__init__.py`.")
}
Some(parent) => {
format!("File `{filename}` declares a package, but is nested under an implicit namespace package. Add an `__init__.py` to `{parent}`.")
}
}
}
}
/// INP001
pub(crate) fn implicit_namespace_package(
path: &Path,
package: Option<&Path>,
package: Option<PackageRoot<'_>>,
locator: &Locator,
comment_ranges: &CommentRanges,
project_root: &Path,
src: &[PathBuf],
preview: PreviewMode,
) -> Option<Diagnostic> {
if package.is_none()
// Ignore non-`.py` files, which don't require an `__init__.py`.
@@ -73,13 +84,39 @@ pub(crate) fn implicit_namespace_package(
let path = path
.to_string_lossy()
.replace(std::path::MAIN_SEPARATOR, "/"); // The snapshot test expects / as the path separator.
Some(Diagnostic::new(
return Some(Diagnostic::new(
ImplicitNamespacePackage {
filename: fs::relativize_path(path),
parent: None,
},
TextRange::default(),
))
} else {
None
));
}
if preview.is_enabled() {
if let Some(PackageRoot::Nested { path: root }) = package.as_ref() {
if path.ends_with("__init__.py") {
// Identify the intermediary package that's missing the `__init__.py` file.
if let Some(parent) = root
.ancestors()
.find(|parent| !parent.join("__init__.py").exists())
{
#[cfg(all(test, windows))]
let path = path
.to_string_lossy()
.replace(std::path::MAIN_SEPARATOR, "/"); // The snapshot test expects / as the path separator.
return Some(Diagnostic::new(
ImplicitNamespacePackage {
filename: fs::relativize_path(path),
parent: Some(fs::relativize_path(parent)),
},
TextRange::default(),
));
}
}
}
}
None
}

View File

@@ -152,6 +152,23 @@ mod tests {
Ok(())
}
#[test]
fn custom_classmethod_rules_preview() -> Result<()> {
let diagnostics = test_path(
Path::new("flake8_pyi/PYI019.pyi"),
&settings::LinterSettings {
pep8_naming: pep8_naming::settings::Settings {
classmethod_decorators: vec!["foo_classmethod".to_string()],
..pep8_naming::settings::Settings::default()
},
preview: PreviewMode::Enabled,
..settings::LinterSettings::for_rule(Rule::CustomTypeVarReturnType)
},
)?;
assert_messages!(diagnostics);
Ok(())
}
#[test_case(Rule::TypeAliasWithoutAnnotation, Path::new("PYI026.py"))]
#[test_case(Rule::TypeAliasWithoutAnnotation, Path::new("PYI026.pyi"))]
fn py38(rule_code: Rule, path: &Path) -> Result<()> {

View File

@@ -1,26 +1,27 @@
use ruff_diagnostics::{Diagnostic, Violation};
use crate::checkers::ast::Checker;
use crate::importer::ImportRequest;
use itertools::Itertools;
use ruff_diagnostics::{Applicability, Diagnostic, Edit, Fix, FixAvailability, Violation};
use ruff_macros::{derive_message_formats, violation};
use ruff_python_ast as ast;
use ruff_python_ast::helpers::map_subscript;
use ruff_python_ast::{Decorator, Expr, Parameters, TypeParam, TypeParams};
use ruff_python_ast::{Expr, Parameters, TypeParam, TypeParams};
use ruff_python_semantic::analyze::function_type::{self, FunctionType};
use ruff_python_semantic::analyze::visibility::{is_abstract, is_overload};
use ruff_text_size::Ranged;
use crate::checkers::ast::Checker;
use ruff_text_size::{Ranged, TextRange};
/// ## What it does
/// Checks for methods that define a custom `TypeVar` for their return type
/// annotation instead of using `typing_extensions.Self`.
/// annotation instead of using `Self`.
///
/// ## Why is this bad?
/// While the semantics are often identical, using `typing_extensions.Self` is
/// more intuitive and succinct (per [PEP 673]) than a custom `TypeVar`. For
/// example, the use of `Self` will typically allow for the omission of type
/// parameters on the `self` and `cls` arguments.
/// While the semantics are often identical, using `Self` is more intuitive
/// and succinct (per [PEP 673]) than a custom `TypeVar`. For example, the
/// use of `Self` will typically allow for the omission of type parameters
/// on the `self` and `cls` arguments.
///
/// This check currently applies to instance methods that return `self`, class
/// methods that return an instance of `cls`, and `__new__` methods.
/// This check currently applies to instance methods that return `self`,
/// class methods that return an instance of `cls`, and `__new__` methods.
///
/// ## Example
///
@@ -44,47 +45,67 @@ use crate::checkers::ast::Checker;
/// def bar(cls, arg: int) -> Self: ...
/// ```
///
/// ## Fix safety
/// The fix is only available in stub files.
/// It will try to remove all usages and declarations of the custom type variable.
/// Pre-[PEP-695]-style declarations will not be removed.
///
/// If a variable's annotation is too complex to handle,
/// the fix will be marked as display only.
/// Otherwise, it will be marked as safe.
///
/// [PEP 673]: https://peps.python.org/pep-0673/#motivation
/// [PEP 695]: https://peps.python.org/pep-0695/
#[violation]
pub struct CustomTypeVarReturnType {
method_name: String,
in_stub: bool,
}
impl Violation for CustomTypeVarReturnType {
const FIX_AVAILABILITY: FixAvailability = FixAvailability::Sometimes;
#[derive_message_formats]
fn message(&self) -> String {
let CustomTypeVarReturnType { method_name } = self;
format!(
"Methods like `{method_name}` should return `typing.Self` instead of a custom `TypeVar`"
)
let method_name = &self.method_name;
format!("Methods like `{method_name}` should return `Self` instead of a custom `TypeVar`")
}
fn fix_title(&self) -> Option<String> {
// See `replace_custom_typevar_with_self`'s doc comment
if self.in_stub {
Some("Replace with `Self`".to_string())
} else {
None
}
}
}
/// PYI019
pub(crate) fn custom_type_var_return_type(
checker: &mut Checker,
name: &str,
decorator_list: &[Decorator],
returns: Option<&Expr>,
args: &Parameters,
type_params: Option<&TypeParams>,
function_def: &ast::StmtFunctionDef,
) {
// Given, e.g., `def foo(self: _S, arg: bytes) -> _T`, extract `_T`.
let Some(returns) = returns else {
let Some(returns) = function_def.returns.as_ref() else {
return;
};
let parameters = &*function_def.parameters;
// Given, e.g., `def foo(self: _S, arg: bytes)`, extract `_S`.
let Some(self_or_cls_annotation) = args
let Some(self_or_cls_annotation) = parameters
.posonlyargs
.iter()
.chain(&args.args)
.chain(&parameters.args)
.next()
.and_then(|parameter_with_default| parameter_with_default.parameter.annotation.as_ref())
else {
return;
};
let decorator_list = &*function_def.decorator_list;
let semantic = checker.semantic();
// Skip any abstract, static, and overloaded methods.
@@ -93,7 +114,7 @@ pub(crate) fn custom_type_var_return_type(
}
let method = match function_type::classify(
name,
&function_def.name,
decorator_list,
semantic.current_scope(),
semantic,
@@ -105,22 +126,17 @@ pub(crate) fn custom_type_var_return_type(
FunctionType::ClassMethod => Method::Class(ClassMethod {
cls_annotation: self_or_cls_annotation,
returns,
type_params,
type_params: function_def.type_params.as_deref(),
}),
FunctionType::Method => Method::Instance(InstanceMethod {
self_annotation: self_or_cls_annotation,
returns,
type_params,
type_params: function_def.type_params.as_deref(),
}),
};
if method.uses_custom_var() {
checker.diagnostics.push(Diagnostic::new(
CustomTypeVarReturnType {
method_name: name.to_string(),
},
returns.range(),
));
add_diagnostic(checker, function_def, returns);
}
}
@@ -147,8 +163,8 @@ struct ClassMethod<'a> {
}
impl<'a> ClassMethod<'a> {
/// Returns `true` if the class method is annotated with a custom `TypeVar` that is likely
/// private.
/// Returns `true` if the class method is annotated with
/// a custom `TypeVar` that is likely private.
fn uses_custom_var(&self) -> bool {
let Expr::Subscript(ast::ExprSubscript { slice, value, .. }) = self.cls_annotation else {
return false;
@@ -188,8 +204,8 @@ struct InstanceMethod<'a> {
}
impl<'a> InstanceMethod<'a> {
/// Returns `true` if the instance method is annotated with a custom `TypeVar` that is likely
/// private.
/// Returns `true` if the instance method is annotated with
/// a custom `TypeVar` that is likely private.
fn uses_custom_var(&self) -> bool {
let Expr::Name(ast::ExprName {
id: first_arg_type, ..
@@ -230,3 +246,181 @@ fn is_likely_private_typevar(type_var_name: &str, type_params: Option<&TypeParam
})
})
}
fn add_diagnostic(checker: &mut Checker, function_def: &ast::StmtFunctionDef, returns: &Expr) {
let in_stub = checker.source_type.is_stub();
let mut diagnostic = Diagnostic::new(
CustomTypeVarReturnType {
method_name: function_def.name.to_string(),
in_stub,
},
returns.range(),
);
// See `replace_custom_typevar_with_self`'s doc comment
if in_stub {
if let Some(fix) = replace_custom_typevar_with_self(checker, function_def, returns) {
diagnostic.set_fix(fix);
}
}
checker.diagnostics.push(diagnostic);
}
/// Add a "Replace with `Self`" fix that does the following:
///
/// * Import `Self` if necessary
/// * Remove the first parameter's annotation
/// * Replace the return annotation with `Self`
/// * Replace other uses of the original type variable elsewhere in the signature with `Self`
/// * Remove that type variable from the PEP 695 type parameter list
///
/// This fix cannot be suggested for non-stubs,
/// as a non-stub fix would have to deal with references in body/at runtime as well,
/// which is substantially harder and requires a type-aware backend.
///
/// The fourth step above has the same problem.
/// This function thus only does replacements for the simplest of cases
/// and will mark the fix as unsafe if an annotation cannot be handled.
fn replace_custom_typevar_with_self(
checker: &Checker,
function_def: &ast::StmtFunctionDef,
returns: &Expr,
) -> Option<Fix> {
if checker.settings.preview.is_disabled() {
return None;
}
// The return annotation is guaranteed to be a name,
// as verified by `uses_custom_var()`.
let typevar_name = returns.as_name_expr().unwrap().id();
let mut all_edits = vec![
replace_return_annotation_with_self(returns),
remove_first_parameter_annotation(&function_def.parameters),
];
let edit = import_self(checker, returns.range())?;
all_edits.push(edit);
if let Some(edit) =
remove_typevar_declaration(function_def.type_params.as_deref(), typevar_name)
{
all_edits.push(edit);
}
let (mut edits, fix_applicability) =
replace_typevar_usages_with_self(&function_def.parameters, typevar_name);
all_edits.append(&mut edits);
let (first, rest) = (all_edits.swap_remove(0), all_edits);
Some(Fix::applicable_edits(first, rest, fix_applicability))
}
fn import_self(checker: &Checker, return_range: TextRange) -> Option<Edit> {
// From PYI034's fix
let target_version = checker.settings.target_version.as_tuple();
let source_module = if target_version >= (3, 11) {
"typing"
} else {
"typing_extensions"
};
let (importer, semantic) = (checker.importer(), checker.semantic());
let request = ImportRequest::import_from(source_module, "Self");
let position = return_range.start();
let (edit, ..) = importer
.get_or_import_symbol(&request, position, semantic)
.ok()?;
Some(edit)
}
fn remove_first_parameter_annotation(parameters: &Parameters) -> Edit {
// The first parameter is guaranteed to be `self`/`cls`,
// as verified by `uses_custom_var()`.
let mut non_variadic_positional = parameters.posonlyargs.iter().chain(&parameters.args);
let first = &non_variadic_positional.next().unwrap().parameter;
let name_end = first.name.range.end();
let annotation_end = first.range.end();
Edit::deletion(name_end, annotation_end)
}
fn replace_return_annotation_with_self(returns: &Expr) -> Edit {
Edit::range_replacement("Self".to_string(), returns.range())
}
fn replace_typevar_usages_with_self(
parameters: &Parameters,
typevar_name: &str,
) -> (Vec<Edit>, Applicability) {
let mut edits = vec![];
let mut could_not_handle_all_usages = false;
for parameter in parameters.iter().skip(1) {
let Some(annotation) = parameter.annotation() else {
continue;
};
let Expr::Name(name) = annotation else {
could_not_handle_all_usages = true;
continue;
};
if name.id.as_str() == typevar_name {
let edit = Edit::range_replacement("Self".to_string(), annotation.range());
edits.push(edit);
} else {
could_not_handle_all_usages = true;
}
}
if could_not_handle_all_usages {
(edits, Applicability::DisplayOnly)
} else {
(edits, Applicability::Safe)
}
}
fn remove_typevar_declaration(type_params: Option<&TypeParams>, name: &str) -> Option<Edit> {
let is_declaration_in_question = |type_param: &&TypeParam| -> bool {
if let TypeParam::TypeVar(typevar) = type_param {
return typevar.name.as_str() == name;
};
false
};
let parameter_list = type_params?;
let parameters = &parameter_list.type_params;
let first = parameters.first()?;
if parameter_list.len() == 1 && is_declaration_in_question(&first) {
return Some(Edit::range_deletion(parameter_list.range));
}
let (index, declaration) = parameters
.iter()
.find_position(is_declaration_in_question)?;
let typevar_range = declaration.range();
let last_index = parameters.len() - 1;
let range = if index < last_index {
// [A, B, C]
// ^^^ Remove this
let next_range = parameters[index + 1].range();
TextRange::new(typevar_range.start(), next_range.start())
} else {
// [A, B, C]
// ^^^ Remove this
let previous_range = parameters[index - 1].range();
TextRange::new(previous_range.end(), typevar_range.start())
};
Some(Edit::range_deletion(range))
}

View File

@@ -2,7 +2,7 @@ use std::collections::HashSet;
use rustc_hash::FxHashSet;
use ruff_diagnostics::{AlwaysFixableViolation, Diagnostic, Edit, Fix};
use ruff_diagnostics::{AlwaysFixableViolation, Applicability, Diagnostic, Edit, Fix};
use ruff_macros::{derive_message_formats, violation};
use ruff_python_ast::comparable::ComparableExpr;
use ruff_python_ast::{self as ast, Expr, ExprContext};
@@ -28,8 +28,10 @@ use crate::checkers::ast::Checker;
/// ```
///
/// ## Fix safety
/// This rule's fix is marked as safe; however, the fix will flatten nested
/// literals into a single top-level literal.
/// This rule's fix is marked as safe, unless the type annotation contains comments.
///
/// Note that the fix will flatten nested literals into a single top-level
/// literal.
///
/// ## References
/// - [Python documentation: `typing.Literal`](https://docs.python.org/3/library/typing.html#typing.Literal)
@@ -73,33 +75,39 @@ pub(crate) fn duplicate_literal_member<'a>(checker: &mut Checker, expr: &'a Expr
// Traverse the literal, collect all diagnostic members.
traverse_literal(&mut check_for_duplicate_members, checker.semantic(), expr);
// If there's at least one diagnostic, create a fix to remove the duplicate members.
if !diagnostics.is_empty() {
if let Expr::Subscript(subscript) = expr {
let subscript = Expr::Subscript(ast::ExprSubscript {
slice: Box::new(if let [elt] = unique_nodes.as_slice() {
(*elt).clone()
} else {
Expr::Tuple(ast::ExprTuple {
elts: unique_nodes.into_iter().cloned().collect(),
range: TextRange::default(),
ctx: ExprContext::Load,
parenthesized: false,
})
}),
value: subscript.value.clone(),
range: TextRange::default(),
ctx: ExprContext::Load,
});
let fix = Fix::safe_edit(Edit::range_replacement(
checker.generator().expr(&subscript),
expr.range(),
));
for diagnostic in &mut diagnostics {
diagnostic.set_fix(fix.clone());
}
}
if diagnostics.is_empty() {
return;
}
// If there's at least one diagnostic, create a fix to remove the duplicate members.
if let Expr::Subscript(subscript) = expr {
let subscript = Expr::Subscript(ast::ExprSubscript {
slice: Box::new(if let [elt] = unique_nodes.as_slice() {
(*elt).clone()
} else {
Expr::Tuple(ast::ExprTuple {
elts: unique_nodes.into_iter().cloned().collect(),
range: TextRange::default(),
ctx: ExprContext::Load,
parenthesized: false,
})
}),
value: subscript.value.clone(),
range: TextRange::default(),
ctx: ExprContext::Load,
});
let fix = Fix::applicable_edit(
Edit::range_replacement(checker.generator().expr(&subscript), expr.range()),
if checker.comment_ranges().intersects(expr.range()) {
Applicability::Unsafe
} else {
Applicability::Safe
},
);
for diagnostic in &mut diagnostics {
diagnostic.set_fix(fix.clone());
}
};
checker.diagnostics.append(&mut diagnostics);
}

View File

@@ -35,34 +35,42 @@ impl Violation for FutureAnnotationsInStub {
/// PYI044
pub(crate) fn from_future_import(checker: &mut Checker, target: &StmtImportFrom) {
if let StmtImportFrom {
let StmtImportFrom {
range,
module: Some(name),
module: Some(module_name),
names,
..
} = target
{
if name == "__future__" && names.iter().any(|alias| &*alias.name == "annotations") {
let mut diagnostic = Diagnostic::new(FutureAnnotationsInStub, *range);
else {
return;
};
if checker.settings.preview.is_enabled() {
let stmt = checker.semantic().current_statement();
if module_name != "__future__" {
return;
};
diagnostic.try_set_fix(|| {
let edit = fix::edits::remove_unused_imports(
std::iter::once("annotations"),
stmt,
None,
checker.locator(),
checker.stylist(),
checker.indexer(),
)?;
Ok(Fix::safe_edit(edit))
});
}
checker.diagnostics.push(diagnostic);
}
if names.iter().all(|alias| &*alias.name != "annotations") {
return;
}
let mut diagnostic = Diagnostic::new(FutureAnnotationsInStub, *range);
if checker.settings.preview.is_enabled() {
let stmt = checker.semantic().current_statement();
diagnostic.try_set_fix(|| {
let edit = fix::edits::remove_unused_imports(
std::iter::once("annotations"),
stmt,
None,
checker.locator(),
checker.stylist(),
checker.indexer(),
)?;
Ok(Fix::safe_edit(edit))
});
}
checker.diagnostics.push(diagnostic);
}

View File

@@ -1,3 +1,5 @@
use std::fmt;
pub(crate) use any_eq_ne_annotation::*;
pub(crate) use bad_generator_return_type::*;
pub(crate) use bad_version_info_comparison::*;
@@ -27,7 +29,6 @@ pub(crate) use redundant_final_literal::*;
pub(crate) use redundant_literal_union::*;
pub(crate) use redundant_numeric_union::*;
pub(crate) use simple_defaults::*;
use std::fmt;
pub(crate) use str_or_repr_defined_in_stub::*;
pub(crate) use string_or_bytes_too_long::*;
pub(crate) use stub_body_multiple_statements::*;

View File

@@ -1,14 +1,15 @@
use ruff_python_ast::{self as ast, Decorator, Expr, Parameters, Stmt};
use ruff_diagnostics::{Diagnostic, Violation};
use crate::checkers::ast::Checker;
use crate::importer::ImportRequest;
use ruff_diagnostics::{Diagnostic, Edit, Fix, FixAvailability, Violation};
use ruff_macros::{derive_message_formats, violation};
use ruff_python_ast::helpers::map_subscript;
use ruff_python_ast::identifier::Identifier;
use ruff_python_semantic::analyze;
use ruff_python_semantic::analyze::visibility::{is_abstract, is_final, is_overload};
use ruff_python_semantic::{ScopeKind, SemanticModel};
use crate::checkers::ast::Checker;
use ruff_text_size::{Ranged, TextRange};
/// ## What it does
/// Checks for methods that are annotated with a fixed return type which
@@ -79,12 +80,15 @@ pub struct NonSelfReturnType {
}
impl Violation for NonSelfReturnType {
const FIX_AVAILABILITY: FixAvailability = FixAvailability::Sometimes;
#[derive_message_formats]
fn message(&self) -> String {
let NonSelfReturnType {
class_name,
method_name,
} = self;
if matches!(class_name.as_str(), "__new__") {
"`__new__` methods usually return `self` at runtime".to_string()
} else {
@@ -93,7 +97,7 @@ impl Violation for NonSelfReturnType {
}
fn fix_title(&self) -> Option<String> {
Some("Consider using `typing_extensions.Self` as return type".to_string())
Some("Use `Self` as return type".to_string())
}
}
@@ -136,13 +140,7 @@ pub(crate) fn non_self_return_type(
&& is_name(returns, &class_def.name)
&& !is_final(&class_def.decorator_list, semantic)
{
checker.diagnostics.push(Diagnostic::new(
NonSelfReturnType {
class_name: class_def.name.to_string(),
method_name: name.to_string(),
},
stmt.identifier(),
));
add_diagnostic(checker, stmt, returns, class_def, name);
}
return;
}
@@ -150,13 +148,7 @@ pub(crate) fn non_self_return_type(
// In-place methods that are expected to return `Self`.
if is_inplace_bin_op(name) {
if !is_self(returns, checker) {
checker.diagnostics.push(Diagnostic::new(
NonSelfReturnType {
class_name: class_def.name.to_string(),
method_name: name.to_string(),
},
stmt.identifier(),
));
add_diagnostic(checker, stmt, returns, class_def, name);
}
return;
}
@@ -164,13 +156,7 @@ pub(crate) fn non_self_return_type(
if is_name(returns, &class_def.name) {
if matches!(name, "__enter__" | "__new__") && !is_final(&class_def.decorator_list, semantic)
{
checker.diagnostics.push(Diagnostic::new(
NonSelfReturnType {
class_name: class_def.name.to_string(),
method_name: name.to_string(),
},
stmt.identifier(),
));
add_diagnostic(checker, stmt, returns, class_def, name);
}
return;
}
@@ -180,32 +166,67 @@ pub(crate) fn non_self_return_type(
if is_iterable_or_iterator(returns, semantic)
&& subclasses_iterator(class_def, semantic)
{
checker.diagnostics.push(Diagnostic::new(
NonSelfReturnType {
class_name: class_def.name.to_string(),
method_name: name.to_string(),
},
stmt.identifier(),
));
add_diagnostic(checker, stmt, returns, class_def, name);
}
}
"__aiter__" => {
if is_async_iterable_or_iterator(returns, semantic)
&& subclasses_async_iterator(class_def, semantic)
{
checker.diagnostics.push(Diagnostic::new(
NonSelfReturnType {
class_name: class_def.name.to_string(),
method_name: name.to_string(),
},
stmt.identifier(),
));
add_diagnostic(checker, stmt, returns, class_def, name);
}
}
_ => {}
}
}
/// Add a diagnostic for the given method.
fn add_diagnostic(
checker: &mut Checker,
stmt: &Stmt,
returns: &Expr,
class_def: &ast::StmtClassDef,
method_name: &str,
) {
/// Return an [`Edit`] that imports `typing.Self` from `typing` or `typing_extensions`.
fn import_self(checker: &Checker, range: TextRange) -> Option<Edit> {
let target_version = checker.settings.target_version.as_tuple();
let source_module = if target_version >= (3, 11) {
"typing"
} else {
"typing_extensions"
};
let (importer, semantic) = (checker.importer(), checker.semantic());
let request = ImportRequest::import_from(source_module, "Self");
let (edit, ..) = importer
.get_or_import_symbol(&request, range.start(), semantic)
.ok()?;
Some(edit)
}
/// Generate a [`Fix`] that replaces the return type with `Self`.
fn replace_with_self(checker: &mut Checker, range: TextRange) -> Option<Fix> {
let import_self = import_self(checker, range)?;
let replace_with_self = Edit::range_replacement("Self".to_string(), range);
Some(Fix::unsafe_edits(import_self, [replace_with_self]))
}
let mut diagnostic = Diagnostic::new(
NonSelfReturnType {
class_name: class_def.name.to_string(),
method_name: method_name.to_string(),
},
stmt.identifier(),
);
if let Some(fix) = replace_with_self(checker, returns.range()) {
diagnostic.set_fix(fix);
}
checker.diagnostics.push(diagnostic);
}
/// Returns `true` if the method is an in-place binary operator.
fn is_inplace_bin_op(name: &str) -> bool {
matches!(

View File

@@ -1,34 +1,35 @@
---
source: crates/ruff_linter/src/rules/flake8_pyi/mod.rs
snapshot_kind: text
---
PYI019.py:7:62: PYI019 Methods like `__new__` should return `typing.Self` instead of a custom `TypeVar`
PYI019.py:7:62: PYI019 Methods like `__new__` should return `Self` instead of a custom `TypeVar`
|
6 | class BadClass:
7 | def __new__(cls: type[_S], *args: str, **kwargs: int) -> _S: ... # PYI019
| ^^ PYI019
|
PYI019.py:10:54: PYI019 Methods like `bad_instance_method` should return `typing.Self` instead of a custom `TypeVar`
PYI019.py:10:54: PYI019 Methods like `bad_instance_method` should return `Self` instead of a custom `TypeVar`
|
10 | def bad_instance_method(self: _S, arg: bytes) -> _S: ... # PYI019
| ^^ PYI019
|
PYI019.py:14:54: PYI019 Methods like `bad_class_method` should return `typing.Self` instead of a custom `TypeVar`
PYI019.py:14:54: PYI019 Methods like `bad_class_method` should return `Self` instead of a custom `TypeVar`
|
13 | @classmethod
14 | def bad_class_method(cls: type[_S], arg: int) -> _S: ... # PYI019
| ^^ PYI019
|
PYI019.py:18:55: PYI019 Methods like `bad_posonly_class_method` should return `typing.Self` instead of a custom `TypeVar`
PYI019.py:18:55: PYI019 Methods like `bad_posonly_class_method` should return `Self` instead of a custom `TypeVar`
|
17 | @classmethod
18 | def bad_posonly_class_method(cls: type[_S], /) -> _S: ... # PYI019
| ^^ PYI019
|
PYI019.py:39:63: PYI019 Methods like `__new__` should return `typing.Self` instead of a custom `TypeVar`
PYI019.py:39:63: PYI019 Methods like `__new__` should return `Self` instead of a custom `TypeVar`
|
37 | # Python > 3.12
38 | class PEP695BadDunderNew[T]:
@@ -36,16 +37,152 @@ PYI019.py:39:63: PYI019 Methods like `__new__` should return `typing.Self` inste
| ^ PYI019
|
PYI019.py:42:46: PYI019 Methods like `generic_instance_method` should return `typing.Self` instead of a custom `TypeVar`
PYI019.py:42:46: PYI019 Methods like `generic_instance_method` should return `Self` instead of a custom `TypeVar`
|
42 | def generic_instance_method[S](self: S) -> S: ... # PYI019
| ^ PYI019
|
PYI019.py:54:32: PYI019 Methods like `foo` should return `typing.Self` instead of a custom `TypeVar`
PYI019.py:54:32: PYI019 Methods like `foo` should return `Self` instead of a custom `TypeVar`
|
52 | # in the settings for this test:
53 | @foo_classmethod
54 | def foo[S](cls: type[S]) -> S: ... # PYI019
| ^ PYI019
|
PYI019.py:61:48: PYI019 Methods like `__new__` should return `Self` instead of a custom `TypeVar`
|
59 | # Only .pyi gets fixes, no fixes for .py
60 | class PEP695Fix:
61 | def __new__[S: PEP695Fix](cls: type[S]) -> S: ...
| ^ PYI019
62 |
63 | def __init_subclass__[S](cls: type[S]) -> S: ...
|
PYI019.py:63:47: PYI019 Methods like `__init_subclass__` should return `Self` instead of a custom `TypeVar`
|
61 | def __new__[S: PEP695Fix](cls: type[S]) -> S: ...
62 |
63 | def __init_subclass__[S](cls: type[S]) -> S: ...
| ^ PYI019
64 |
65 | def __neg__[S: PEP695Fix](self: S) -> S: ...
|
PYI019.py:65:43: PYI019 Methods like `__neg__` should return `Self` instead of a custom `TypeVar`
|
63 | def __init_subclass__[S](cls: type[S]) -> S: ...
64 |
65 | def __neg__[S: PEP695Fix](self: S) -> S: ...
| ^ PYI019
66 |
67 | def __pos__[S](self: S) -> S: ...
|
PYI019.py:67:32: PYI019 Methods like `__pos__` should return `Self` instead of a custom `TypeVar`
|
65 | def __neg__[S: PEP695Fix](self: S) -> S: ...
66 |
67 | def __pos__[S](self: S) -> S: ...
| ^ PYI019
68 |
69 | def __add__[S: PEP695Fix](self: S, other: S) -> S: ...
|
PYI019.py:69:53: PYI019 Methods like `__add__` should return `Self` instead of a custom `TypeVar`
|
67 | def __pos__[S](self: S) -> S: ...
68 |
69 | def __add__[S: PEP695Fix](self: S, other: S) -> S: ...
| ^ PYI019
70 |
71 | def __sub__[S](self: S, other: S) -> S: ...
|
PYI019.py:71:42: PYI019 Methods like `__sub__` should return `Self` instead of a custom `TypeVar`
|
69 | def __add__[S: PEP695Fix](self: S, other: S) -> S: ...
70 |
71 | def __sub__[S](self: S, other: S) -> S: ...
| ^ PYI019
72 |
73 | @classmethod
|
PYI019.py:74:59: PYI019 Methods like `class_method_bound` should return `Self` instead of a custom `TypeVar`
|
73 | @classmethod
74 | def class_method_bound[S: PEP695Fix](cls: type[S]) -> S: ...
| ^ PYI019
75 |
76 | @classmethod
|
PYI019.py:77:50: PYI019 Methods like `class_method_unbound` should return `Self` instead of a custom `TypeVar`
|
76 | @classmethod
77 | def class_method_unbound[S](cls: type[S]) -> S: ...
| ^ PYI019
78 |
79 | def instance_method_bound[S: PEP695Fix](self: S) -> S: ...
|
PYI019.py:79:57: PYI019 Methods like `instance_method_bound` should return `Self` instead of a custom `TypeVar`
|
77 | def class_method_unbound[S](cls: type[S]) -> S: ...
78 |
79 | def instance_method_bound[S: PEP695Fix](self: S) -> S: ...
| ^ PYI019
80 |
81 | def instance_method_unbound[S](self: S) -> S: ...
|
PYI019.py:81:48: PYI019 Methods like `instance_method_unbound` should return `Self` instead of a custom `TypeVar`
|
79 | def instance_method_bound[S: PEP695Fix](self: S) -> S: ...
80 |
81 | def instance_method_unbound[S](self: S) -> S: ...
| ^ PYI019
82 |
83 | def instance_method_bound_with_another_parameter[S: PEP695Fix](self: S, other: S) -> S: ...
|
PYI019.py:83:90: PYI019 Methods like `instance_method_bound_with_another_parameter` should return `Self` instead of a custom `TypeVar`
|
81 | def instance_method_unbound[S](self: S) -> S: ...
82 |
83 | def instance_method_bound_with_another_parameter[S: PEP695Fix](self: S, other: S) -> S: ...
| ^ PYI019
84 |
85 | def instance_method_unbound_with_another_parameter[S](self: S, other: S) -> S: ...
|
PYI019.py:85:81: PYI019 Methods like `instance_method_unbound_with_another_parameter` should return `Self` instead of a custom `TypeVar`
|
83 | def instance_method_bound_with_another_parameter[S: PEP695Fix](self: S, other: S) -> S: ...
84 |
85 | def instance_method_unbound_with_another_parameter[S](self: S, other: S) -> S: ...
| ^ PYI019
86 |
87 | def multiple_type_vars[S, *Ts, T](self: S, other: S, /, *args: *Ts, a: T, b: list[T]) -> S: ...
|
PYI019.py:87:94: PYI019 Methods like `multiple_type_vars` should return `Self` instead of a custom `TypeVar`
|
85 | def instance_method_unbound_with_another_parameter[S](self: S, other: S) -> S: ...
86 |
87 | def multiple_type_vars[S, *Ts, T](self: S, other: S, /, *args: *Ts, a: T, b: list[T]) -> S: ...
| ^ PYI019
88 |
89 | def mixing_old_and_new_style_type_vars[T](self: _S695, a: T, b: T) -> _S695: ...
|
PYI019.py:89:75: PYI019 Methods like `mixing_old_and_new_style_type_vars` should return `Self` instead of a custom `TypeVar`
|
87 | def multiple_type_vars[S, *Ts, T](self: S, other: S, /, *args: *Ts, a: T, b: list[T]) -> S: ...
88 |
89 | def mixing_old_and_new_style_type_vars[T](self: _S695, a: T, b: T) -> _S695: ...
| ^^^^^ PYI019
|

View File

@@ -1,51 +1,208 @@
---
source: crates/ruff_linter/src/rules/flake8_pyi/mod.rs
---
PYI019.pyi:7:62: PYI019 Methods like `__new__` should return `typing.Self` instead of a custom `TypeVar`
PYI019.pyi:7:62: PYI019 Methods like `__new__` should return `Self` instead of a custom `TypeVar`
|
6 | class BadClass:
7 | def __new__(cls: type[_S], *args: str, **kwargs: int) -> _S: ... # PYI019
| ^^ PYI019
|
= help: Replace with `Self`
PYI019.pyi:10:54: PYI019 Methods like `bad_instance_method` should return `typing.Self` instead of a custom `TypeVar`
PYI019.pyi:10:54: PYI019 Methods like `bad_instance_method` should return `Self` instead of a custom `TypeVar`
|
10 | def bad_instance_method(self: _S, arg: bytes) -> _S: ... # PYI019
| ^^ PYI019
|
= help: Replace with `Self`
PYI019.pyi:14:54: PYI019 Methods like `bad_class_method` should return `typing.Self` instead of a custom `TypeVar`
PYI019.pyi:14:54: PYI019 Methods like `bad_class_method` should return `Self` instead of a custom `TypeVar`
|
13 | @classmethod
14 | def bad_class_method(cls: type[_S], arg: int) -> _S: ... # PYI019
| ^^ PYI019
|
= help: Replace with `Self`
PYI019.pyi:18:55: PYI019 Methods like `bad_posonly_class_method` should return `typing.Self` instead of a custom `TypeVar`
PYI019.pyi:18:55: PYI019 Methods like `bad_posonly_class_method` should return `Self` instead of a custom `TypeVar`
|
17 | @classmethod
18 | def bad_posonly_class_method(cls: type[_S], /) -> _S: ... # PYI019
| ^^ PYI019
|
= help: Replace with `Self`
PYI019.pyi:39:63: PYI019 Methods like `__new__` should return `typing.Self` instead of a custom `TypeVar`
PYI019.pyi:39:63: PYI019 Methods like `__new__` should return `Self` instead of a custom `TypeVar`
|
37 | # Python > 3.12
38 | class PEP695BadDunderNew[T]:
39 | def __new__[S](cls: type[S], *args: Any, ** kwargs: Any) -> S: ... # PYI019
| ^ PYI019
|
= help: Replace with `Self`
PYI019.pyi:42:46: PYI019 Methods like `generic_instance_method` should return `typing.Self` instead of a custom `TypeVar`
PYI019.pyi:42:46: PYI019 Methods like `generic_instance_method` should return `Self` instead of a custom `TypeVar`
|
42 | def generic_instance_method[S](self: S) -> S: ... # PYI019
| ^ PYI019
|
= help: Replace with `Self`
PYI019.pyi:54:32: PYI019 Methods like `foo` should return `typing.Self` instead of a custom `TypeVar`
PYI019.pyi:54:32: PYI019 Methods like `foo` should return `Self` instead of a custom `TypeVar`
|
52 | # in the settings for this test:
53 | @foo_classmethod
54 | def foo[S](cls: type[S]) -> S: ... # PYI019
| ^ PYI019
|
= help: Replace with `Self`
PYI019.pyi:61:48: PYI019 Methods like `__new__` should return `Self` instead of a custom `TypeVar`
|
59 | # Only .pyi gets fixes, no fixes for .py
60 | class PEP695Fix:
61 | def __new__[S: PEP695Fix](cls: type[S]) -> S: ...
| ^ PYI019
62 |
63 | def __init_subclass__[S](cls: type[S]) -> S: ...
|
= help: Replace with `Self`
PYI019.pyi:63:47: PYI019 Methods like `__init_subclass__` should return `Self` instead of a custom `TypeVar`
|
61 | def __new__[S: PEP695Fix](cls: type[S]) -> S: ...
62 |
63 | def __init_subclass__[S](cls: type[S]) -> S: ...
| ^ PYI019
64 |
65 | def __neg__[S: PEP695Fix](self: S) -> S: ...
|
= help: Replace with `Self`
PYI019.pyi:65:43: PYI019 Methods like `__neg__` should return `Self` instead of a custom `TypeVar`
|
63 | def __init_subclass__[S](cls: type[S]) -> S: ...
64 |
65 | def __neg__[S: PEP695Fix](self: S) -> S: ...
| ^ PYI019
66 |
67 | def __pos__[S](self: S) -> S: ...
|
= help: Replace with `Self`
PYI019.pyi:67:32: PYI019 Methods like `__pos__` should return `Self` instead of a custom `TypeVar`
|
65 | def __neg__[S: PEP695Fix](self: S) -> S: ...
66 |
67 | def __pos__[S](self: S) -> S: ...
| ^ PYI019
68 |
69 | def __add__[S: PEP695Fix](self: S, other: S) -> S: ...
|
= help: Replace with `Self`
PYI019.pyi:69:53: PYI019 Methods like `__add__` should return `Self` instead of a custom `TypeVar`
|
67 | def __pos__[S](self: S) -> S: ...
68 |
69 | def __add__[S: PEP695Fix](self: S, other: S) -> S: ...
| ^ PYI019
70 |
71 | def __sub__[S](self: S, other: S) -> S: ...
|
= help: Replace with `Self`
PYI019.pyi:71:42: PYI019 Methods like `__sub__` should return `Self` instead of a custom `TypeVar`
|
69 | def __add__[S: PEP695Fix](self: S, other: S) -> S: ...
70 |
71 | def __sub__[S](self: S, other: S) -> S: ...
| ^ PYI019
72 |
73 | @classmethod
|
= help: Replace with `Self`
PYI019.pyi:74:59: PYI019 Methods like `class_method_bound` should return `Self` instead of a custom `TypeVar`
|
73 | @classmethod
74 | def class_method_bound[S: PEP695Fix](cls: type[S]) -> S: ...
| ^ PYI019
75 |
76 | @classmethod
|
= help: Replace with `Self`
PYI019.pyi:77:50: PYI019 Methods like `class_method_unbound` should return `Self` instead of a custom `TypeVar`
|
76 | @classmethod
77 | def class_method_unbound[S](cls: type[S]) -> S: ...
| ^ PYI019
78 |
79 | def instance_method_bound[S: PEP695Fix](self: S) -> S: ...
|
= help: Replace with `Self`
PYI019.pyi:79:57: PYI019 Methods like `instance_method_bound` should return `Self` instead of a custom `TypeVar`
|
77 | def class_method_unbound[S](cls: type[S]) -> S: ...
78 |
79 | def instance_method_bound[S: PEP695Fix](self: S) -> S: ...
| ^ PYI019
80 |
81 | def instance_method_unbound[S](self: S) -> S: ...
|
= help: Replace with `Self`
PYI019.pyi:81:48: PYI019 Methods like `instance_method_unbound` should return `Self` instead of a custom `TypeVar`
|
79 | def instance_method_bound[S: PEP695Fix](self: S) -> S: ...
80 |
81 | def instance_method_unbound[S](self: S) -> S: ...
| ^ PYI019
82 |
83 | def instance_method_bound_with_another_parameter[S: PEP695Fix](self: S, other: S) -> S: ...
|
= help: Replace with `Self`
PYI019.pyi:83:90: PYI019 Methods like `instance_method_bound_with_another_parameter` should return `Self` instead of a custom `TypeVar`
|
81 | def instance_method_unbound[S](self: S) -> S: ...
82 |
83 | def instance_method_bound_with_another_parameter[S: PEP695Fix](self: S, other: S) -> S: ...
| ^ PYI019
84 |
85 | def instance_method_unbound_with_another_parameter[S](self: S, other: S) -> S: ...
|
= help: Replace with `Self`
PYI019.pyi:85:81: PYI019 Methods like `instance_method_unbound_with_another_parameter` should return `Self` instead of a custom `TypeVar`
|
83 | def instance_method_bound_with_another_parameter[S: PEP695Fix](self: S, other: S) -> S: ...
84 |
85 | def instance_method_unbound_with_another_parameter[S](self: S, other: S) -> S: ...
| ^ PYI019
86 |
87 | def multiple_type_vars[S, *Ts, T](self: S, other: S, /, *args: *Ts, a: T, b: list[T]) -> S: ...
|
= help: Replace with `Self`
PYI019.pyi:87:94: PYI019 Methods like `multiple_type_vars` should return `Self` instead of a custom `TypeVar`
|
85 | def instance_method_unbound_with_another_parameter[S](self: S, other: S) -> S: ...
86 |
87 | def multiple_type_vars[S, *Ts, T](self: S, other: S, /, *args: *Ts, a: T, b: list[T]) -> S: ...
| ^ PYI019
88 |
89 | def mixing_old_and_new_style_type_vars[T](self: _S695, a: T, b: T) -> _S695: ...
|
= help: Replace with `Self`
PYI019.pyi:89:75: PYI019 Methods like `mixing_old_and_new_style_type_vars` should return `Self` instead of a custom `TypeVar`
|
87 | def multiple_type_vars[S, *Ts, T](self: S, other: S, /, *args: *Ts, a: T, b: list[T]) -> S: ...
88 |
89 | def mixing_old_and_new_style_type_vars[T](self: _S695, a: T, b: T) -> _S695: ...
| ^^^^^ PYI019
|
= help: Replace with `Self`

Some files were not shown because too many files have changed in this diff Show More