Compare commits

...

124 Commits

Author SHA1 Message Date
Micha Reiser
0c27a95426 Prefer breaking return-type over parameters 2024-11-07 08:38:05 +01:00
Dhruv Manilawala
5b500b838b Update known dunder methods for Python 3.13 (#14146)
## Summary

Closes: #14145
2024-11-07 11:39:00 +05:30
Dylan
cb003ebe22 [flake8-builtins] Skip lambda expressions in builtin-argument-shadowing (A002) (#14144)
Flake8-builtins provides two checks for arguments (really, parameters)
of a function shadowing builtins: A002 checks function definitions, and
A006 checks lambda expressions. This PR ensures that A002 is restricted
to functions rather than lambda expressions.

Closes #14135 .
2024-11-07 05:34:09 +00:00
Carl Meyer
03a5788aa1 [red-knot] a few metaclass cleanups (#14142)
Just cleaning up a few small things I noticed in post-land review.
2024-11-06 22:13:39 +00:00
Charlie Marsh
626f716de6 Add support for resolving metaclasses (#14120)
## Summary

I mirrored some of the idioms that @AlexWaygood used in the MRO work.

Closes https://github.com/astral-sh/ruff/issues/14096.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-11-06 15:41:35 -05:00
InSync
46c5a13103 [eradicate] Better detection of IntelliJ language injection comments (ERA001) (#14094) 2024-11-06 18:24:15 +00:00
Micha Reiser
31681f66c9 Fix duplicate unpack diagnostics (#14125)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-11-06 11:28:29 +00:00
Micha Reiser
a56ee9268e Add mdtest support for files with invalid syntax (#14126) 2024-11-06 12:25:52 +01:00
Dhruv Manilawala
4ece8e5c1e Use "Ruff" instead of "uv" for src setting docs (#14121)
## Summary

From
15aa5a6d57
2024-11-06 03:19:32 +00:00
Dhruv Manilawala
34b6a9b909 Remove unpack field from SemanticIndexBuilder (#14101)
## Summary

Related to
https://github.com/astral-sh/ruff/pull/13979#discussion_r1828305790,
this PR removes the `current_unpack` state field from
`SemanticIndexBuilder` and passes the `Unpack` ingredient via the
`CurrentAssignment` -> `DefinitionNodeRef` conversion to finally store
it on `DefintionNodeKind`.

This involves updating the lifetime of `AnyParameterRef` (parameter to
`declare_parameter`) to use the `'db` lifetime. Currently, all AST nodes
stored on various enums are marked with `'a` lifetime but they're always
utilized using the `'db` lifetime.

This also removes the dedicated `'a` lifetime parameter on
`add_definition` which is currently being used in `DefinitionNodeRef`.
As mentioned, all AST nodes live through the `'db` lifetime so we can
remove the `'a` lifetime parameter from that method and use the `'db`
lifetime instead.
2024-11-06 08:42:58 +05:30
Alex Waygood
eead549254 [red-knot] Introduce a new ClassLiteralType struct (#14108) 2024-11-05 22:16:33 +00:00
Lokejoke
abafeb4bee Fix: Recover boolean test flag after visiting subexpressions (#13909)
Co-authored-by: xbrtnik1 <524841@mail.muni.cz>
2024-11-05 20:55:49 +01:00
Dylan
2b76fa8fa1 [refurb] Parse more exotic decimal strings in verbose-decimal-constructor (FURB157) (#14098)
FURB157 suggests replacing expressions like `Decimal("123")` with
`Decimal(123)`. This PR extends the rule to cover cases where the input
string to `Decimal` can be easily transformed into an integer literal.

For example:

```python
Decimal("1__000")   # fix: `Decimal(1000)`
```

Note: we do not implement the full decimal parsing logic from CPython on
the grounds that certain acceptable string inputs to the `Decimal`
constructor may be presumed purposeful on the part of the developer. For
example, as in the linked issue, `Decimal("١٢٣")` is valid and equal to
`Decimal(123)`, but we do not suggest a replacement in this case.

Closes #13807
2024-11-05 13:33:04 -06:00
David Peter
239cbc6f33 [red-knot] Store starred-expression annotation types (#14106)
## Summary

- Store the expression type for annotations that are starred expressions
(see [discussion
here](https://github.com/astral-sh/ruff/pull/14091#discussion_r1828332857))
- Use `self.store_expression_type(…)` consistently throughout, as it
makes sure that no double-insertion errors occur.

closes #14115

## Test Plan

Added an invalid-syntax example to the corpus which leads to a panic on
`main`. Also added a Markdown test with a valid-syntax example that
would lead to a panic once we implement function parameter inference.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-11-05 20:25:45 +01:00
David Peter
2296627528 [red-knot] Precise inference for identity checks (#14109)
## Summary

Adds more precise type inference for `… is …` and `… is not …` identity
checks in some limited cases where we statically know the answer to be
either `Literal[True]` or `Literal[False]`.

I found this helpful while working on type inference for comparisons
involving intersection types, but I'm not sure if this is at all useful
for real world code (where the answer is most probably *not* statically
known). Note that we already have *type narrowing* for identity tests.
So while we are already able to generate constraints for things like `if
x is None`, we can now — in some limited cases — make an even stronger
conclusion and infer that the test expression itself is `Literal[False]`
(branch never taken) or `Literal[True]` (branch always taken).

## Test Plan

New Markdown tests
2024-11-05 19:48:52 +01:00
Micha Reiser
05687285fe fix double inference of standalone expressions (#14107) 2024-11-05 15:50:31 +01:00
Alex Waygood
05f97bae73 types.rs: remove unused is_stdlib_symbol methods (#14104) 2024-11-05 12:46:17 +00:00
Micha Reiser
4323512a65 Remove AST-node dependency from FunctionType and ClassType (#14087)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-11-05 08:02:38 +00:00
Shaygan Hooshyari
9dddd73c29 [red-knot] Literal special form (#13874)
Handling `Literal` type in annotations.

Resolves: #13672 

## Implementation

Since Literals are not a fully defined type in typeshed. I used a trick
to figure out when a special form is a literal.
When we are inferring assignment types I am checking if the type of that
assignment was resolved to typing.SpecialForm and the name of the target
is `Literal` if that is the case then I am re creating a new instance
type and set the known instance field to `KnownInstance:Literal`.

**Why not defining a new type?**

From this [issue](https://github.com/python/typeshed/issues/6219) I
learned that we want to resolve members to SpecialMethod class. So if we
create a new instance here we can rely on the member resolving in that
already exists.


## Tests


https://typing.readthedocs.io/en/latest/spec/literal.html#equivalence-of-two-literals
Since the type of the value inside Literal is evaluated as a
Literal(LiteralString, LiteralInt, ...) then the equality is only true
when types and value are equal.


https://typing.readthedocs.io/en/latest/spec/literal.html#legal-and-illegal-parameterizations

The illegal parameterizations are mostly implemented I'm currently
checking the slice expression and the slice type to make sure it's
valid.

https://typing.readthedocs.io/en/latest/spec/literal.html#shortening-unions-of-literals

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-11-05 01:45:46 +00:00
TomerBin
6c56a7a868 [red-knot] Implement type narrowing for boolean conditionals (#14037)
## Summary

This PR enables red-knot to support type narrowing based on `and` and
`or` conditionals, including nested combinations and their negation (for
`elif` / `else` blocks and for `not` operator). Part of #13694.

In order to address this properly (hopefully 😅), I had to run
`NarrowingConstraintsBuilder` functions recursively. In the first commit
I introduced a minor refactor - instead of mutating `self.constraints`,
the new constraints are now returned as function return values. I also
modified the constraints map to be optional, preventing unnecessary
hashmap allocations.
Thanks @carljm for your support on this :)

The second commit contains the logic and tests for handling boolean ops,
with improvements to intersections handling in `is_subtype_of` .

As I'm still new to Rust and the internals of type checkers, I’d be more
than happy to hear any insights or suggestions.
Thank you!

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2024-11-04 22:54:35 +00:00
InSync
bb25bd9c6c Also remove trailing comma while fixing C409 and C419 (#14097) 2024-11-04 20:33:30 +00:00
Simon Brugman
b7e32b0a18 Re-enable clippy useless-format (#14095) 2024-11-04 18:25:25 +01:00
Simon Brugman
fb94b71e63 Derive message formats macro support to string (#14093) 2024-11-04 18:06:25 +01:00
Micha Reiser
bc0586d922 Avoid cloning Name when looking up function and class types (#14092) 2024-11-04 15:52:59 +01:00
Simon Brugman
a7a78f939c Replace format! without parameters with .to_string() (#14090)
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2024-11-04 14:09:30 +00:00
David Peter
6dabf045c3 [red-knot] Do not panic when encountering string annotations (#14091)
## Summary

Encountered this while running red-knot benchmarks on the `black`
codebase.

Fixes two of the issues in #13478.

## Test Plan

Added a regression test.
2024-11-04 15:06:54 +01:00
Alex Waygood
df45a0e3f9 [red-knot] Add MRO resolution for classes (#14027) 2024-11-04 13:31:38 +00:00
David Peter
88d9bb191b [red-knot] Remove Type::None (#14024)
## Summary

Removes `Type::None` in favor of `KnownClass::NoneType.to_instance(…)`.

closes #13670

## Performance

There is a -4% performance regression on our red-knot benchmark. This is due to the fact that we now have to import `_typeshed` as a module, and infer types.

## Test Plan

Existing tests pass.
2024-11-04 14:00:05 +01:00
Dhruv Manilawala
e302c2de7c Cached inference of all definitions in an unpacking (#13979)
## Summary

This PR adds a new salsa query and an ingredient to resolve all the
variables involved in an unpacking assignment like `(a, b) = (1, 2)` at
once. Previously, we'd recursively try to match the correct type for
each definition individually which will result in creating duplicate
diagnostics.

This PR still doesn't solve the duplicate diagnostics issue because that
requires a different solution like using salsa accumulator or
de-duplicating the diagnostics manually.

Related: #13773 

## Test Plan

Make sure that all unpack assignment test cases pass, there are no
panics in the corpus tests.

## Todo

- [x] Look at the performance regression
2024-11-04 17:11:57 +05:30
renovate[bot]
012f385f5d Update dependency uuid to v11 (#14084)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-11-04 08:09:18 +01:00
renovate[bot]
a6f7f22b27 Update Rust crate notify to v7 (#14083) 2024-11-04 07:39:06 +01:00
renovate[bot]
8d7dda9fb7 Update cloudflare/wrangler-action action to v3.11.0 (#14080)
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
|
[cloudflare/wrangler-action](https://redirect.github.com/cloudflare/wrangler-action)
| action | minor | `v3.9.0` -> `v3.11.0` |

---

### Release Notes

<details>
<summary>cloudflare/wrangler-action
(cloudflare/wrangler-action)</summary>

###
[`v3.11.0`](https://redirect.github.com/cloudflare/wrangler-action/releases/tag/v3.11.0)

[Compare
Source](https://redirect.github.com/cloudflare/wrangler-action/compare/v3.10.0...v3.11.0)

##### Minor Changes

-
[#&#8203;309](https://redirect.github.com/cloudflare/wrangler-action/pull/309)
[`10d5b9c1c1826adaec0a9ee49fdf5b91113508ef`](10d5b9c1c1)
Thanks [@&#8203;Maximo-Guk](https://redirect.github.com/Maximo-Guk)! -
Revert "Add parity with pages-action for pages deploy outputs"

###
[`v3.10.0`](https://redirect.github.com/cloudflare/wrangler-action/releases/tag/v3.10.0)

[Compare
Source](https://redirect.github.com/cloudflare/wrangler-action/compare/v3.9.0...v3.10.0)

##### Minor Changes

-
[#&#8203;303](https://redirect.github.com/cloudflare/wrangler-action/pull/303)
[`3ec7f8943ef83351f743cfaa8763a9056ef70993`](3ec7f8943e)
Thanks
[@&#8203;courtney-sims](https://redirect.github.com/courtney-sims)! -
Support id, environment, url, and alias outputs for Pages deploys.

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "before 4am on Monday" (UTC),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/astral-sh/ruff).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOC4xNDIuNCIsInVwZGF0ZWRJblZlciI6IjM4LjE0Mi40IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJpbnRlcm5hbCJdfQ==-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-11-04 08:44:06 +05:30
renovate[bot]
fb0881d836 Update dependency mdformat-mkdocs to v3.1.1 (#14081)
This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
|
[mdformat-mkdocs](https://redirect.github.com/kyleking/mdformat-mkdocs)
([changelog](https://redirect.github.com/kyleking/mdformat-mkdocs/releases))
| `==3.0.1` -> `==3.1.1` |
[![age](https://developer.mend.io/api/mc/badges/age/pypi/mdformat-mkdocs/3.1.1?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/mdformat-mkdocs/3.1.1?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/mdformat-mkdocs/3.0.1/3.1.1?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/mdformat-mkdocs/3.0.1/3.1.1?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|

---

### Release Notes

<details>
<summary>kyleking/mdformat-mkdocs (mdformat-mkdocs)</summary>

###
[`v3.1.1`](https://redirect.github.com/KyleKing/mdformat-mkdocs/releases/tag/v3.1.1)

[Compare
Source](https://redirect.github.com/kyleking/mdformat-mkdocs/compare/v3.1.0...v3.1.1)

**Full Changelog**:
https://github.com/KyleKing/mdformat-mkdocs/compare/v3.1.0...v3.1.1

###
[`v3.1.0`](https://redirect.github.com/kyleking/mdformat-mkdocs/compare/v3.0.1...v3.1.0)

[Compare
Source](https://redirect.github.com/kyleking/mdformat-mkdocs/compare/v3.0.1...v3.1.0)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "before 4am on Monday" (UTC),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/astral-sh/ruff).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOC4xNDIuNCIsInVwZGF0ZWRJblZlciI6IjM4LjE0Mi40IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJpbnRlcm5hbCJdfQ==-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-11-04 08:37:53 +05:30
renovate[bot]
ded2b15e05 Update pre-commit dependencies (#14082)
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
|
[abravalheri/validate-pyproject](https://redirect.github.com/abravalheri/validate-pyproject)
| repository | minor | `v0.21` -> `v0.22` |
|
[astral-sh/ruff-pre-commit](https://redirect.github.com/astral-sh/ruff-pre-commit)
| repository | patch | `v0.7.0` -> `v0.7.2` |
| [crate-ci/typos](https://redirect.github.com/crate-ci/typos) |
repository | minor | `v1.26.0` -> `v1.27.0` |

Note: The `pre-commit` manager in Renovate is not supported by the
`pre-commit` maintainers or community. Please do not report any problems
there, instead [create a Discussion in the Renovate
repository](https://redirect.github.com/renovatebot/renovate/discussions/new)
if you have any questions.

---

### Release Notes

<details>
<summary>abravalheri/validate-pyproject
(abravalheri/validate-pyproject)</summary>

###
[`v0.22`](https://redirect.github.com/abravalheri/validate-pyproject/releases/tag/v0.22)

[Compare
Source](https://redirect.github.com/abravalheri/validate-pyproject/compare/v0.21...v0.22)

#### What's Changed

- Prevent injecting defaults and modifying input in-place, by
[@&#8203;henryiii](https://redirect.github.com/henryiii) in
[https://github.com/abravalheri/validate-pyproject/pull/213](https://redirect.github.com/abravalheri/validate-pyproject/pull/213)

**Full Changelog**:
https://github.com/abravalheri/validate-pyproject/compare/v0.21...v0.22

</details>

<details>
<summary>astral-sh/ruff-pre-commit (astral-sh/ruff-pre-commit)</summary>

###
[`v0.7.2`](https://redirect.github.com/astral-sh/ruff-pre-commit/releases/tag/v0.7.2)

[Compare
Source](https://redirect.github.com/astral-sh/ruff-pre-commit/compare/v0.7.1...v0.7.2)

See: https://github.com/astral-sh/ruff/releases/tag/0.7.2

###
[`v0.7.1`](https://redirect.github.com/astral-sh/ruff-pre-commit/releases/tag/v0.7.1)

[Compare
Source](https://redirect.github.com/astral-sh/ruff-pre-commit/compare/v0.7.0...v0.7.1)

See: https://github.com/astral-sh/ruff/releases/tag/0.7.1

</details>

<details>
<summary>crate-ci/typos (crate-ci/typos)</summary>

###
[`v1.27.0`](https://redirect.github.com/crate-ci/typos/releases/tag/v1.27.0)

[Compare
Source](https://redirect.github.com/crate-ci/typos/compare/v1.26.8...v1.27.0)

#### \[1.27.0] - 2024-11-01

##### Features

- Updated the dictionary with the [October
2024](https://redirect.github.com/crate-ci/typos/issues/1106) changes

###
[`v1.26.8`](https://redirect.github.com/crate-ci/typos/releases/tag/v1.26.8)

[Compare
Source](https://redirect.github.com/crate-ci/typos/compare/v1.26.7...v1.26.8)

#### \[1.26.8] - 2024-10-24

###
[`v1.26.7`](https://redirect.github.com/crate-ci/typos/compare/v1.26.6...v1.26.7)

[Compare
Source](https://redirect.github.com/crate-ci/typos/compare/v1.26.6...v1.26.7)

###
[`v1.26.6`](https://redirect.github.com/crate-ci/typos/compare/v1.26.5...v1.26.6)

[Compare
Source](https://redirect.github.com/crate-ci/typos/compare/v1.26.5...v1.26.6)

###
[`v1.26.5`](https://redirect.github.com/crate-ci/typos/compare/v1.26.4...v1.26.5)

[Compare
Source](https://redirect.github.com/crate-ci/typos/compare/v1.26.4...v1.26.5)

###
[`v1.26.4`](https://redirect.github.com/crate-ci/typos/compare/v1.26.3...v1.26.4)

[Compare
Source](https://redirect.github.com/crate-ci/typos/compare/v1.26.3...v1.26.4)

###
[`v1.26.3`](https://redirect.github.com/crate-ci/typos/releases/tag/v1.26.3)

[Compare
Source](https://redirect.github.com/crate-ci/typos/compare/v1.26.2...v1.26.3)

#### \[1.26.3] - 2024-10-24

##### Fixes

-   Accept `additionals`

###
[`v1.26.2`](https://redirect.github.com/crate-ci/typos/releases/tag/v1.26.2)

[Compare
Source](https://redirect.github.com/crate-ci/typos/compare/v1.26.1...v1.26.2)

#### \[1.26.2] - 2024-10-24

##### Fixes

-   Accept `tesselate` variants

###
[`v1.26.1`](https://redirect.github.com/crate-ci/typos/releases/tag/v1.26.1)

[Compare
Source](https://redirect.github.com/crate-ci/typos/compare/v1.26.0...v1.26.1)

#### \[1.26.1] - 2024-10-23

##### Fixes

-   Respect `--force-exclude` for binary files

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "before 4am on Monday" (UTC),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

👻 **Immortal**: This PR will be recreated if closed unmerged. Get
[config
help](https://redirect.github.com/renovatebot/renovate/discussions) if
that's undesired.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/astral-sh/ruff).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOC4xNDIuNCIsInVwZGF0ZWRJblZlciI6IjM4LjE0Mi40IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJpbnRlcm5hbCJdfQ==-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-11-04 08:35:31 +05:30
renovate[bot]
3133964d8c Update dependency ruff to v0.7.2 (#14077) 2024-11-03 21:16:28 -05:00
renovate[bot]
f00039b6f2 Update NPM Development dependencies (#14078) 2024-11-03 21:16:22 -05:00
renovate[bot]
6ccd0f187b Update Rust crate thiserror to v1.0.67 (#14076) 2024-11-03 21:16:13 -05:00
renovate[bot]
de40f6a3ad Update Rust crate syn to v2.0.87 (#14075) 2024-11-03 21:16:08 -05:00
renovate[bot]
dfbd27dc2f Update Rust crate serde to v1.0.214 (#14074) 2024-11-03 21:16:02 -05:00
renovate[bot]
1531ca8a1b Update Rust crate pep440_rs to v0.7.2 (#14073) 2024-11-03 21:15:56 -05:00
renovate[bot]
71702bbd48 Update Rust crate insta to v1.41.1 (#14072) 2024-11-03 21:15:49 -05:00
renovate[bot]
8d9bdb5b92 Update Rust crate anyhow to v1.0.92 (#14071) 2024-11-03 21:15:42 -05:00
Fábio D. Batista
2b73a1c039 [eradicate] ignore # language= in commented-out-code rule (ERA001) (#14069)
## Summary

The `commented-out-code` rule (ERA001) from `eradicate` is currently
flagging a very common idiom that marks Python strings as another
language, to help with syntax highlighting:


![image](https://github.com/user-attachments/assets/d523e83d-95cb-4668-a793-45f01d162234)

This PR adds this idiom to the list of allowed exceptions to the rule.

## Test Plan

I've added some additional test cases.
2024-11-03 16:50:00 -05:00
Charlie Marsh
2b0cdd2338 Improve some rule messages and docs (#14068) 2024-11-03 19:25:43 +00:00
Charlie Marsh
f09dc8b67c Detect items that hash to same value in duplicate dictionaries (#14065)
## Summary

Closes https://github.com/astral-sh/ruff/issues/12772.
2024-11-03 14:16:34 -05:00
Charlie Marsh
71a122f060 Allow open without context manager in return statement (#14066)
## Summary

Closes https://github.com/astral-sh/ruff/issues/13862.
2024-11-03 14:16:27 -05:00
Matt Norton
3ca24785ae Add links to missing related options within rule documentations (#13971)
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-11-03 14:15:57 -05:00
Charlie Marsh
1de36cfe4c Fix wrong-size header in open-file-with-context-handler (#14067) 2024-11-03 19:06:40 +00:00
Charlie Marsh
66872a41fc Detect items that hash to same value in duplicate sets (#14064)
## Summary

Like https://github.com/astral-sh/ruff/pull/14063, but ensures that we
catch cases like `{1, True}` in which the items hash to the same value
despite not being identical.
2024-11-03 18:49:11 +00:00
Charlie Marsh
e00594e8d2 Respect hash-equivalent literals in iteration-over-set (#14063)
## Summary

Closes https://github.com/astral-sh/ruff/issues/14049.
2024-11-03 18:44:52 +00:00
Micha Reiser
443fd3b660 Disallow single-line implicit concatenated strings (#13928) 2024-11-03 11:49:26 +00:00
Steve C
ae9f08d1e5 [ruff] - fix false positive for decorators (RUF028) (#14061) 2024-11-03 11:49:03 +00:00
Steve C
f69712c11d [flake8-pyi] - include all python file types for PYI006 and PYI066 (#14059) 2024-11-03 11:47:36 +00:00
Steve C
be485602de Fix preview link references in 2 rule docs (#14060) 2024-11-03 11:45:35 +00:00
Steve C
bc7615af0e [flake8-bugbear] - do not run mutable-argument-default on stubs (B006) (#14058)
## Summary

Early-exits in `B006` when the file is a stub. Fixes #14026 

## Test Plan

`cargo test`
2024-11-02 22:48:48 -04:00
Charlie Marsh
4a3eeeff86 Remove HashableExpr abstraction (#14057)
## Summary

It looks like `ComparableExpr` now implements `Hash` so we can just
remove this.
2024-11-02 20:28:35 +00:00
Charlie Marsh
35c6dfe481 Avoid parsing joint rule codes as distinct codes in # noqa (#12809)
## Summary

We should enable warnings for unsupported codes, but this at least fixes
the parsing for `# noqa: F401F841`.

Closes https://github.com/astral-sh/ruff/issues/12808.
2024-11-02 20:24:59 +00:00
Simon Brugman
f8374280c0 [flake8-simplify] Implementation for split-of-static-string (SIM905) (#14008)
## Summary

Closes https://github.com/astral-sh/ruff/issues/13944

## Test Plan

Standard snapshot testing

flake8-simplify surprisingly only has a single test case

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-11-02 17:15:36 +00:00
Steve C
0925513529 [pyupgrade] - ignore kwarg unpacking for UP044 (#14053)
## Summary

Fixes #14047 

## Test Plan

`catgo test`

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-11-02 13:10:56 -04:00
Charlie Marsh
70bdde4085 Handle unions in augmented assignments (#14045)
## Summary

Removing more TODOs from the augmented assignment test suite. Now, if
the _target_ is a union, we correctly infer the union of results:

```python
if flag:
    f = Foo()
else:
    f = 42.0
f += 12
```
2024-11-01 19:49:18 +00:00
TomerBin
34a5d7cb7f [red-knot] Infer type of if-expression if test has statically known truthiness (#14048)
## Summary

Detecting statically known truthy or falsy test in if expressions
(ternary).

## Test Plan

new mdtest
2024-11-01 12:23:18 -07:00
Charlie Marsh
487941ea66 Handle maybe-unbound __iadd__-like operators in augmented assignments (#14044)
## Summary

One of the follow-ups from augmented assignment inference, now that
`Type::Unbound` has been removed.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-11-01 13:15:35 -04:00
Harry Reeder
099f077311 [docs] Add rule short code to mkdocs tags (#14040)
## Summary

<!-- What's the purpose of the change? What does it do, and why? -->
This PR updates the metadata in the YAML frontmatter of the mkdocs
documentation to include the rule short code as a tag, so it can be
easily searched.
Ref: #13684

## Test Plan

<!-- How was it tested? -->
This has been tested locally using the documentation provided
[here](https://docs.astral.sh/ruff/contributing/#mkdocs) for generating
docs.

This generates docs that now have the tags section:
```markdown
---
description: Checks for abstract classes without abstract methods.
tags:
- B024
---

# abstract-base-class-without-abstract-method (B024)
... trimmed
```

I've also verified that this gives the ability to get straight to the
page via search when serving mkdocs locally.

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-11-01 15:50:12 +00:00
Micha Reiser
8574751911 Give non-existent files a durability of at least Medium (#14034) 2024-11-01 16:44:30 +01:00
Dhruv Manilawala
ddae741b72 Switch to uv publish (#14042)
## Summary

Ref: https://github.com/astral-sh/uv/pull/8065

## Test Plan

Going to re-release `0.7.2` which failed:
https://github.com/astral-sh/ruff/actions/runs/11630280069
2024-11-01 20:24:29 +05:30
Simon Brugman
5053d2c127 Doc: markdown link fix (#14041)
Typo in `mutable-contextvar-default` in `flake8-bugbear`
2024-11-01 14:19:00 +00:00
Dhruv Manilawala
ef72fd79a7 Bump version to 0.7.2 (#14039) 2024-11-01 19:09:07 +05:30
STACIA
658a51ea10 Fix typo for static method decorator (#14038) 2024-11-01 12:30:50 +00:00
github-actions[bot]
7c2da4f06e Sync vendored typeshed stubs (#14030)
Co-authored-by: typeshedbot <>
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2024-11-01 10:51:56 +00:00
Micha Reiser
48fa839c80 Use named function in incremental red knot benchmark (#14033) 2024-11-01 08:44:38 +00:00
Micha Reiser
cf0f5e1318 Fix formatting of single with-item with trailing comment (#14005) 2024-11-01 09:08:06 +01:00
Micha Reiser
20b8a43017 Fix server panic when undoing an edit (#14010) 2024-11-01 08:16:53 +01:00
Carl Meyer
b8acadd6a2 [red-knot] have mdformat wrap mdtest files to 100 columns (#14020)
This makes it easier to read and edit (and review changes to) these
files as source, even though it doesn't affect the rendering.
2024-10-31 21:00:51 +00:00
David Peter
b372fe7198 [red-knot] Add myself as red-knot codeowner (#14023) 2024-10-31 19:17:37 +00:00
David Peter
53fa32a389 [red-knot] Remove Type::Unbound (#13980)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

- Remove `Type::Unbound`
- Handle (potential) unboundness as a concept orthogonal to the type
system (see new `Symbol` type)
- Improve existing and add new diagnostics related to (potential)
unboundness

closes #13671 

## Test Plan

- Update existing markdown-based tests
- Add new tests for added/modified functionality
2024-10-31 20:05:53 +01:00
Alex Waygood
d1189c20df [red-knot] Add failing tests for iterating over maybe-iterable unions (#14016) 2024-10-31 18:20:21 +00:00
Simon Brugman
9a6b08b557 [flake8-simplify] Include caveats of enabling if-else-block-instead-of-if-exp (SIM108) (#14019) 2024-10-31 17:26:22 +00:00
Micha Reiser
76e4277696 [red-knot] Handle context managers in (sync) with statements (#13998) 2024-10-31 08:18:18 +00:00
Steve C
2d917d72f6 [pyupgrade] - add PEP646 Unpack conversion to * with fix (UP044) (#13988)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-10-31 06:58:34 +00:00
Dhruv Manilawala
2629527559 Fix panic when filling up types vector during unpacking (#14006)
## Summary

This PR fixes a panic which can occur in an unpack assignment when:
* (number of target expressions) - (number of tuple types) > 2
* There's a starred expression

The reason being that the `insert` panics because the index is greater
than the length.

This is an error case and so practically it should occur very rarely.
The solution is to resize the types vector to match the number of
expressions and then insert the starred expression type.

## Test Plan

Add a new test case.
2024-10-30 19:13:57 +00:00
Dhruv Manilawala
bf20061268 Separate type check diagnostics builder (#13978)
## Summary

This PR creates a new `TypeCheckDiagnosticsBuilder` for the
`TypeCheckDiagnostics` struct. The main motivation behind this is to
separate the helpers required to build the diagnostics from the type
inference builder itself. This allows us to use such helpers outside of
the inference builder like for example in the unpacking logic in
https://github.com/astral-sh/ruff/pull/13979.

## Test Plan

`cargo insta test`
2024-10-30 18:50:31 +00:00
Charlie Marsh
eddc8d7644 Add failing tests for augmented assignments with partial binding (#14002)
## Summary

These cases aren't handled correctly yet -- some of them are waiting on
refactors to `Unbound` before fixing. Part of #12699.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-10-30 14:22:34 -04:00
Charlie Marsh
b1ce8a3949 Use Never instead of None for stores (#13984)
## Summary

See:
https://github.com/astral-sh/ruff/pull/13981#issuecomment-2445472433
2024-10-30 12:03:50 -04:00
Charlie Marsh
262c04f297 Use binary semantics when __iadd__ et al are unbound (#13987)
## Summary

I noticed that augmented assignments on floats were yielding "not
supported" diagnostics. If the dunder isn't bound at all, we should use
binary operator semantics, rather than treating it as not-callable.
2024-10-30 13:09:22 +00:00
Charlie Marsh
71536a43db Add remaining augmented assignment dunders (#13985)
## Summary

See: https://github.com/astral-sh/ruff/issues/12699
2024-10-30 13:02:29 +00:00
Alex Waygood
e6dcdf3e49 Switch off the single_match_else Clippy lint (#13994) 2024-10-30 12:24:16 +00:00
Simon Brugman
f426349051 docs: typo in refurb-sorted-min-max (#13993) 2024-10-30 12:07:42 +00:00
Alex Waygood
42c70697d8 [red-knot] Fix bug where union of two iterable types was not recognised as iterable (#13992) 2024-10-30 11:54:16 +00:00
Charlie Marsh
1607d88c22 Use consistent diagnostic messages in augmented assignment inference (#13986) 2024-10-29 22:57:53 -04:00
Charlie Marsh
c6b82151dd Add augmented assignment inference for -= operator (#13981)
## Summary

See: https://github.com/astral-sh/ruff/issues/12699
2024-10-29 22:14:27 -04:00
Alex Waygood
39cf46ecd6 [red-knot] Improve ergonomics for the PySlice trait (#13983) 2024-10-29 20:40:59 +00:00
David Peter
96b3c400fe [red-knot] Minor follow-up on slice expression inference (#13982)
## Summary

Minor follow-up to #13917 — thanks @AlexWaygood for the post-merge
review.

- Add
SliceLiteralType::as_tuple
- Use .expect() instead of SAFETY
comment
- Match on ::try_from
result
- Add TODO comment regarding raising a diagnostic for `"foo"["bar":"baz"]`
2024-10-29 19:40:57 +00:00
jsurany
60a2dc53e7 fix issues in discovering ruff in pip build environments (#13881)
## Summary

<!-- What's the purpose of the change? What does it do, and why? -->
Changes in this PR https://github.com/astral-sh/ruff/pull/13591 did not
allow correct discovery in pip build environments.

```python
# both of these variables are tuple[str, str] (length is 2)
first, second = os.path.split(paths[0]), os.path.split(paths[1])

# so these length checks are guaranteed to fail even for build environment folders
if (
    len(first) >= 3
    and len(second) >= 3 
    ...
)
```

~~Here we instead use `pathlib`, and we check all `pip-build-env-` paths
for the folder that is expected to contain the `ruff` executable.~~

Here we update the logic to more properly split out the path components
that we use for `pip-build-env-` inspection.

## Test Plan

I've checked this manually against a workflow that was failing, I'm not
sure what to do for real tests. The same issues apply as with the
previous PR.

---------

Co-authored-by: Jonathan Surany <jsurany@bloomberg.net>
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-10-29 15:50:29 +00:00
Alex Waygood
8d98aea6c4 [red-knot] Infer attribute expressions in type annotations (#13967) 2024-10-29 11:06:44 +00:00
Alex Waygood
d2c9f5e43c [red-knot] Fallback to attributes on types.ModuleType if a symbol can't be found in locals or globals (#13904) 2024-10-29 10:59:03 +00:00
Alex Waygood
7dd0c7f4bd [red-knot] Infer tuple types from annotations (#13943)
## Summary

This PR adds support for heterogenous `tuple` annotations to red-knot.

It does the following:
- Extends `infer_type_expression` so that it understands tuple
annotations
- Changes `infer_type_expression` so that `ExprStarred` nodes in type
annotations are inferred as `Todo` rather than `Unknown` (they're valid
in PEP-646 tuple annotations)
- Extends `Type::is_subtype_of` to understand when one heterogenous
tuple type can be understood to be a subtype of another (without this
change, the PR would have introduced new false-positive errors to some
existing mdtests).
2024-10-29 10:30:03 +00:00
David Peter
56c796acee [red-knot] Slice expression types & subscript expressions with slices (#13917)
## Summary

- Add a new `Type::SliceLiteral` variant
- Infer `SliceLiteral` types for slice expressions, such as
`<int-literal>:<int-literal>:<int-literal>`.
- Infer "sliced" literal types for subscript expressions using slices,
such as `<string-literal>[<slice-literal>]`.
- Infer types for expressions involving slices of tuples:
`<tuple>[<slice-literal>]`.

closes #13853

## Test Plan

- Unit tests for indexing/slicing utility functions
- Markdown-based tests for
  - Subscript expressions `tuple[slice]`
  - Subscript expressions `string_literal[slice]`
  - Subscript expressions `bytes_literal[slice]`
2024-10-29 10:17:31 +01:00
Raphael Gaschignard
2fe203292a [red-knot] Distribute intersections on negation (#13962)
## Summary

This does two things:
- distribute negated intersections when building up intersections (i.e.
going from `A & ~(B & C)` to `(A & ~B) | (A & ~C)`) (fixing #13931)

## Test Plan

`cargo test`
2024-10-29 02:56:04 +00:00
Charlie Marsh
b6847b371e Skip namespace package enforcement for PEP 723 scripts (#13974)
## Summary

Vendors the PEP 723 parser from
[uv](debe67ffdb/crates/uv-scripts/src/lib.rs (L283)).

Closes https://github.com/astral-sh/ruff/issues/13912.
2024-10-29 02:11:31 +00:00
Charlie Marsh
b19862c64a Rename operator-unsupported to unsupported-operator (#13973)
## Summary

Closes https://github.com/astral-sh/ruff/issues/13959.
2024-10-28 21:34:12 -04:00
TomerBin
9a0dade925 [red-knot] Type narrowing inside boolean expressions (#13970)
## Summary

This PR adds type narrowing in `and` and `or` expressions, for example:

```py
class A: ...

x: A | None = A() if bool_instance() else None

isinstance(x, A) or reveal_type(x)  # revealed: None
``` 

## Test Plan
New mdtests 😍
2024-10-28 18:17:48 -07:00
Dhruv Manilawala
ec6208e51b Treat return type of singledispatch as runtime-required (#13957)
## Summary

fixes: #13955 

## Test Plan

Update existing test case to use a return type hint for which `main`
flags `TCH003`.
2024-10-28 20:33:28 -04:00
TomerBin
74cf66e4c2 [red-knot] Narrowing - Not operator (#13942)
## Summary

After #13918 has landed, narrowing constraint negation became easy, so
adding support for `not` operator.

## Test Plan

Added a new mdtest file for `not` expression.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2024-10-28 20:27:26 +00:00
Carlo Lepelaars
1f19aca632 [DOCS] Add CrowdCent's numerblox to Ruff users. (#13569)
Hi, our open source project
[NumerBlox](https://github.com/crowdcent/numerblox) migrated to `uv` and
`ruff`. Would appreciate the project being included in the list of Ruff
users.

## Summary

Add [NumerBlox](https://github.com/crowdcent/numerblox) to Ruff users in
README.md.
2024-10-28 10:53:37 -04:00
Charlie Marsh
6f52d573ef Support inference for PEP 604 union annotations (#13964)
## Summary

Supports return type inference for, e.g., `def f() -> int | None:`.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-10-28 10:13:01 -04:00
Tim Hatch
c593ccb529 Regenerate known_stdlibs.rs with stdlibs 2024.10.25 (#13963)
## Summary

`stdlibs` has a new release to properly categorize the `_wmi` module
which has been [present since
~2022](https://github.com/python/cpython/issues/89545#issuecomment-1227846806).


## Test Plan

Let CI run, this is only a trivial change to categorization data.
2024-10-28 08:37:54 -04:00
Micha Reiser
9f3a38d408 Extract LineIndex independent methods from Locator (#13938) 2024-10-28 07:53:41 +00:00
renovate[bot]
f8eb547fb4 Update dependency react-resizable-panels to v2.1.6 (#13952) 2024-10-28 07:02:12 +00:00
renovate[bot]
b77de359bc Update NPM Development dependencies (#13954)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-10-28 07:00:44 +00:00
renovate[bot]
41f74512df Update Rust crate insta to v1.41.0 (#13956)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-10-28 06:51:53 +00:00
renovate[bot]
387dc664bd Update Rust crate thiserror to v1.0.65 (#13950) 2024-10-28 06:28:58 +00:00
renovate[bot]
41c9bdbd37 Update Rust crate syn to v2.0.85 (#13949) 2024-10-28 06:28:36 +00:00
renovate[bot]
222a646437 Update Rust crate serde to v1.0.213 (#13948) 2024-10-28 06:27:18 +00:00
renovate[bot]
5b411fe606 Update Rust crate proc-macro2 to v1.0.89 (#13946) 2024-10-28 06:26:53 +00:00
renovate[bot]
47dd83e56f Update Rust crate regex to v1.11.1 (#13947) 2024-10-28 06:26:36 +00:00
renovate[bot]
08e23d78aa Update Rust crate anyhow to v1.0.91 (#13945) 2024-10-28 06:25:44 +00:00
Dhruv Manilawala
5af0966057 Remove unreferenced snapshots (#13958) 2024-10-28 07:16:05 +01:00
renovate[bot]
faf9dfaa9d Update dependency ruff to v0.7.1 (#13953) 2024-10-27 21:13:03 -04:00
renovate[bot]
9d131c8c45 Update dependency mdformat-mkdocs to v3.0.1 (#13951) 2024-10-27 21:12:55 -04:00
Micha Reiser
5a56886414 TCH003: Fix false positive for singledispatchmethod (#13941)
## Summary

Fixes https://github.com/astral-sh/ruff/issues/13924

## Test Plan

Added test
2024-10-27 21:02:45 -04:00
TomerBin
66c3aaa307 [red-knot] - Flow-control for boolean operations (#13940)
## Summary

As python uses short-circuiting boolean operations in runtime, we should
mimic that logic in redknot as well.
For example, we should detect that in the following code `x` might be
undefined inside the block:

```py
if flag or (x := 1):
    print(x) 
```

## Test Plan

Added mdtest suit for boolean expressions.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2024-10-27 03:33:01 +00:00
cake-monotone
b6ffa51c16 [red-knot] Type inference for comparisons between arbitrary instances (#13903)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Carl Meyer <carl@oddbird.net>
2024-10-26 18:19:56 +00:00
TomerBin
35f007f17f [red-knot] Type narrow in else clause (#13918)
## Summary

Add support for type narrowing in elif and else scopes as part of
#13694.

## Test Plan

- mdtest
- builder unit test for union negation.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2024-10-26 16:22:57 +00:00
Jonas Vacek
3006d6da23 Docs: Add GitLab CI/CD to integrations. (#13915) 2024-10-26 18:10:17 +02:00
767 changed files with 15714 additions and 5868 deletions

4
.github/CODEOWNERS vendored
View File

@@ -17,5 +17,5 @@
/scripts/fuzz-parser/ @AlexWaygood
# red-knot
/crates/red_knot* @carljm @MichaReiser @AlexWaygood
/crates/ruff_db/ @carljm @MichaReiser @AlexWaygood
/crates/red_knot* @carljm @MichaReiser @AlexWaygood @sharkdp
/crates/ruff_db/ @carljm @MichaReiser @AlexWaygood @sharkdp

View File

@@ -47,7 +47,7 @@ jobs:
working-directory: playground
- name: "Deploy to Cloudflare Pages"
if: ${{ env.CF_API_TOKEN_EXISTS == 'true' }}
uses: cloudflare/wrangler-action@v3.9.0
uses: cloudflare/wrangler-action@v3.11.0
with:
apiToken: ${{ secrets.CF_API_TOKEN }}
accountId: ${{ secrets.CF_ACCOUNT_ID }}

View File

@@ -21,14 +21,12 @@ jobs:
# For PyPI's trusted publishing.
id-token: write
steps:
- name: "Install uv"
uses: astral-sh/setup-uv@v3
- uses: actions/download-artifact@v4
with:
pattern: wheels-*
path: wheels
merge-multiple: true
- name: Publish to PyPi
uses: pypa/gh-action-pypi-publish@release/v1
with:
skip-existing: true
packages-dir: wheels
verbose: true
run: uv publish -v wheels/*

View File

@@ -17,7 +17,7 @@ exclude: |
repos:
- repo: https://github.com/abravalheri/validate-pyproject
rev: v0.21
rev: v0.22
hooks:
- id: validate-pyproject
@@ -51,11 +51,15 @@ repos:
- id: blacken-docs
args: ["--pyi", "--line-length", "130"]
files: '^crates/.*/resources/mdtest/.*\.md'
exclude: |
(?x)^(
.*?invalid(_.+)_syntax.md
)$
additional_dependencies:
- black==24.10.0
- repo: https://github.com/crate-ci/typos
rev: v1.26.0
rev: v1.27.0
hooks:
- id: typos
@@ -69,7 +73,7 @@ repos:
pass_filenames: false # This makes it a lot faster
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.7.0
rev: v0.7.2
hooks:
- id: ruff-format
- id: ruff

View File

@@ -1,5 +1,31 @@
# Changelog
## 0.7.2
### Preview features
- Fix formatting of single with-item with trailing comment ([#14005](https://github.com/astral-sh/ruff/pull/14005))
- \[`pyupgrade`\] Add PEP 646 `Unpack` conversion to `*` with fix (`UP044`) ([#13988](https://github.com/astral-sh/ruff/pull/13988))
### Rule changes
- Regenerate `known_stdlibs.rs` with stdlibs 2024.10.25 ([#13963](https://github.com/astral-sh/ruff/pull/13963))
- \[`flake8-no-pep420`\] Skip namespace package enforcement for PEP 723 scripts (`INP001`) ([#13974](https://github.com/astral-sh/ruff/pull/13974))
### Server
- Fix server panic when undoing an edit ([#14010](https://github.com/astral-sh/ruff/pull/14010))
### Bug fixes
- Fix issues in discovering ruff in pip build environments ([#13881](https://github.com/astral-sh/ruff/pull/13881))
- \[`flake8-type-checking`\] Fix false positive for `singledispatchmethod` (`TCH003`) ([#13941](https://github.com/astral-sh/ruff/pull/13941))
- \[`flake8-type-checking`\] Treat return type of `singledispatch` as runtime-required (`TCH003`) ([#13957](https://github.com/astral-sh/ruff/pull/13957))
### Documentation
- \[`flake8-simplify`\] Include caveats of enabling `if-else-block-instead-of-if-exp` (`SIM108`) ([#14019](https://github.com/astral-sh/ruff/pull/14019))
## 0.7.1
### Preview features

138
Cargo.lock generated
View File

@@ -123,9 +123,9 @@ dependencies = [
[[package]]
name = "anyhow"
version = "1.0.90"
version = "1.0.92"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "37bf3594c4c988a53154954629820791dde498571819ae4ca50ca811e060cc95"
checksum = "74f37166d7d48a0284b99dd824694c26119c700b53bf0d1540cdb147dbdaaf13"
[[package]]
name = "append-only-vec"
@@ -407,7 +407,7 @@ dependencies = [
"heck",
"proc-macro2",
"quote",
"syn 2.0.82",
"syn 2.0.87",
]
[[package]]
@@ -687,7 +687,7 @@ dependencies = [
"proc-macro2",
"quote",
"strsim 0.10.0",
"syn 2.0.82",
"syn 2.0.87",
]
[[package]]
@@ -698,7 +698,7 @@ checksum = "a668eda54683121533a393014d8692171709ff57a7d61f187b6e782719f8933f"
dependencies = [
"darling_core",
"quote",
"syn 2.0.82",
"syn 2.0.87",
]
[[package]]
@@ -1162,12 +1162,12 @@ dependencies = [
[[package]]
name = "indexmap"
version = "2.5.0"
version = "2.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "68b900aa2f7301e21c36462b170ee99994de34dff39a4a6a528e80e7376d07e5"
checksum = "707907fe3c25f5424cce2cb7e1cbcafee6bdbe735ca90ef77c29e84591e5b9da"
dependencies = [
"equivalent",
"hashbrown 0.14.5",
"hashbrown 0.15.0",
"serde",
]
@@ -1193,9 +1193,9 @@ checksum = "b248f5224d1d606005e02c97f5aa4e88eeb230488bcc03bc9ca4d7991399f2b5"
[[package]]
name = "inotify"
version = "0.9.6"
version = "0.10.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8069d3ec154eb856955c1c0fbffefbf5f3c40a104ec912d4797314c1801abff"
checksum = "fdd168d97690d0b8c412d6b6c10360277f4d7ee495c5d0d5d5fe0854923255cc"
dependencies = [
"bitflags 1.3.2",
"inotify-sys",
@@ -1213,9 +1213,9 @@ dependencies = [
[[package]]
name = "insta"
version = "1.40.0"
version = "1.41.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6593a41c7a73841868772495db7dc1e8ecab43bb5c0b6da2059246c4b506ab60"
checksum = "7e9ffc4d4892617c50a928c52b2961cb5174b6fc6ebf252b2fac9d21955c48b8"
dependencies = [
"console",
"globset",
@@ -1267,7 +1267,7 @@ dependencies = [
"Inflector",
"proc-macro2",
"quote",
"syn 2.0.82",
"syn 2.0.87",
]
[[package]]
@@ -1393,7 +1393,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a2ae40017ac09cd2c6a53504cb3c871c7f2b41466eac5bc66ba63f39073b467b"
dependencies = [
"quote",
"syn 2.0.82",
"syn 2.0.87",
]
[[package]]
@@ -1532,14 +1532,15 @@ dependencies = [
[[package]]
name = "mio"
version = "0.8.11"
version = "1.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a4a650543ca06a924e8b371db273b2756685faae30f8487da1b56505a8f78b0c"
checksum = "80e04d1dcff3aae0704555fe5fee3bcfaf3d1fdf8a7e521d5b9d2b42acb52cec"
dependencies = [
"hermit-abi",
"libc",
"log",
"wasi",
"windows-sys 0.48.0",
"windows-sys 0.52.0",
]
[[package]]
@@ -1593,12 +1594,11 @@ dependencies = [
[[package]]
name = "notify"
version = "6.1.1"
version = "7.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6205bd8bb1e454ad2e27422015fb5e4f2bcc7e08fa8f27058670d208324a4d2d"
checksum = "c533b4c39709f9ba5005d8002048266593c1cfaf3c5f0739d5b8ab0c6c504009"
dependencies = [
"bitflags 2.6.0",
"crossbeam-channel",
"filetime",
"fsevent-sys",
"inotify",
@@ -1606,8 +1606,18 @@ dependencies = [
"libc",
"log",
"mio",
"notify-types",
"walkdir",
"windows-sys 0.48.0",
"windows-sys 0.52.0",
]
[[package]]
name = "notify-types"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7393c226621f817964ffb3dc5704f9509e107a8b024b489cc2c1b217378785df"
dependencies = [
"instant",
]
[[package]]
@@ -1786,9 +1796,9 @@ dependencies = [
[[package]]
name = "pep440_rs"
version = "0.7.1"
version = "0.7.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7c8ee724d21f351f9d47276614ac9710975db827ba9fe2ca5a517ba648193307"
checksum = "0922a442c78611fa8c5ed6065d2d898a820cf12fa90604217fdb2d01675efec7"
dependencies = [
"serde",
"unicode-width 0.2.0",
@@ -1848,7 +1858,7 @@ dependencies = [
"pest_meta",
"proc-macro2",
"quote",
"syn 2.0.82",
"syn 2.0.87",
]
[[package]]
@@ -1963,9 +1973,9 @@ dependencies = [
[[package]]
name = "proc-macro2"
version = "1.0.88"
version = "1.0.89"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7c3a7fc5db1e57d5a779a352c8cdb57b29aa4c40cc69c3a68a7fedc815fbf2f9"
checksum = "f139b0662de085916d1fb67d2b4169d1addddda1919e696f3252b740b629986e"
dependencies = [
"unicode-ident",
]
@@ -2102,6 +2112,7 @@ dependencies = [
"countme",
"dir-test",
"hashbrown 0.15.0",
"indexmap",
"insta",
"itertools 0.13.0",
"memchr",
@@ -2162,6 +2173,7 @@ dependencies = [
"regex",
"ruff_db",
"ruff_index",
"ruff_python_parser",
"ruff_python_trivia",
"ruff_source_file",
"ruff_text_size",
@@ -2247,9 +2259,9 @@ dependencies = [
[[package]]
name = "regex"
version = "1.11.0"
version = "1.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "38200e5ee88914975b69f657f0801b6f6dccafd44fd9326302a4aaeecfacb1d8"
checksum = "b544ef1b4eac5dc2db33ea63606ae9ffcfac26c1416a2806ae0bf5f56b201191"
dependencies = [
"aho-corasick",
"memchr",
@@ -2306,7 +2318,7 @@ dependencies = [
[[package]]
name = "ruff"
version = "0.7.1"
version = "0.7.2"
dependencies = [
"anyhow",
"argfile",
@@ -2523,7 +2535,7 @@ dependencies = [
[[package]]
name = "ruff_linter"
version = "0.7.1"
version = "0.7.2"
dependencies = [
"aho-corasick",
"annotate-snippets 0.9.2",
@@ -2546,7 +2558,7 @@ dependencies = [
"natord",
"path-absolutize",
"pathdiff",
"pep440_rs 0.7.1",
"pep440_rs 0.7.2",
"pyproject-toml",
"quick-junit",
"regex",
@@ -2590,7 +2602,7 @@ dependencies = [
"proc-macro2",
"quote",
"ruff_python_trivia",
"syn 2.0.82",
"syn 2.0.87",
]
[[package]]
@@ -2620,12 +2632,14 @@ dependencies = [
"compact_str",
"is-macro",
"itertools 0.13.0",
"memchr",
"ruff_cache",
"ruff_macros",
"ruff_python_trivia",
"ruff_source_file",
"ruff_text_size",
"rustc-hash 2.0.0",
"salsa",
"schemars",
"serde",
]
@@ -2748,7 +2762,6 @@ dependencies = [
"ruff_python_ast",
"ruff_python_parser",
"ruff_python_stdlib",
"ruff_source_file",
"ruff_text_size",
"rustc-hash 2.0.0",
"schemars",
@@ -2779,7 +2792,6 @@ dependencies = [
"insta",
"ruff_python_parser",
"ruff_python_trivia",
"ruff_source_file",
"ruff_text_size",
]
@@ -2838,7 +2850,7 @@ dependencies = [
[[package]]
name = "ruff_wasm"
version = "0.7.1"
version = "0.7.2"
dependencies = [
"console_error_panic_hook",
"console_log",
@@ -2877,7 +2889,7 @@ dependencies = [
"matchit",
"path-absolutize",
"path-slash",
"pep440_rs 0.7.1",
"pep440_rs 0.7.2",
"regex",
"ruff_cache",
"ruff_formatter",
@@ -3009,7 +3021,7 @@ dependencies = [
"heck",
"proc-macro2",
"quote",
"syn 2.0.82",
"syn 2.0.87",
"synstructure",
]
@@ -3043,7 +3055,7 @@ dependencies = [
"proc-macro2",
"quote",
"serde_derive_internals",
"syn 2.0.82",
"syn 2.0.87",
]
[[package]]
@@ -3066,9 +3078,9 @@ checksum = "1c107b6f4780854c8b126e228ea8869f4d7b71260f962fefb57b996b8959ba6b"
[[package]]
name = "serde"
version = "1.0.210"
version = "1.0.214"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c8e3592472072e6e22e0a54d5904d9febf8508f65fb8552499a1abc7d1078c3a"
checksum = "f55c3193aca71c12ad7890f1785d2b73e1b9f63a0bbc353c08ef26fe03fc56b5"
dependencies = [
"serde_derive",
]
@@ -3086,13 +3098,13 @@ dependencies = [
[[package]]
name = "serde_derive"
version = "1.0.210"
version = "1.0.214"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "243902eda00fad750862fc144cea25caca5e20d615af0a81bee94ca738f1df1f"
checksum = "de523f781f095e28fa605cdce0f8307e451cc0fd14e2eb4cd2e98a355b147766"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.82",
"syn 2.0.87",
]
[[package]]
@@ -3103,7 +3115,7 @@ checksum = "330f01ce65a3a5fe59a60c82f3c9a024b573b8a6e875bd233fe5f934e71d54e3"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.82",
"syn 2.0.87",
]
[[package]]
@@ -3126,7 +3138,7 @@ checksum = "6c64451ba24fc7a6a2d60fc75dd9c83c90903b19028d4eff35e88fc1e86564e9"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.82",
"syn 2.0.87",
]
[[package]]
@@ -3167,7 +3179,7 @@ dependencies = [
"darling",
"proc-macro2",
"quote",
"syn 2.0.82",
"syn 2.0.87",
]
[[package]]
@@ -3269,7 +3281,7 @@ dependencies = [
"proc-macro2",
"quote",
"rustversion",
"syn 2.0.82",
"syn 2.0.87",
]
[[package]]
@@ -3291,9 +3303,9 @@ dependencies = [
[[package]]
name = "syn"
version = "2.0.82"
version = "2.0.87"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "83540f837a8afc019423a8edb95b52a8effe46957ee402287f4292fae35be021"
checksum = "25aa4ce346d03a6dcd68dd8b4010bcb74e54e62c90c573f394c46eae99aba32d"
dependencies = [
"proc-macro2",
"quote",
@@ -3308,7 +3320,7 @@ checksum = "c8af7666ab7b6390ab78131fb5b0fce11d6b7a6951602017c35fa82800708971"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.82",
"syn 2.0.87",
]
[[package]]
@@ -3371,7 +3383,7 @@ dependencies = [
"cfg-if",
"proc-macro2",
"quote",
"syn 2.0.82",
"syn 2.0.87",
]
[[package]]
@@ -3382,28 +3394,28 @@ checksum = "5c89e72a01ed4c579669add59014b9a524d609c0c88c6a585ce37485879f6ffb"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.82",
"syn 2.0.87",
"test-case-core",
]
[[package]]
name = "thiserror"
version = "1.0.64"
version = "1.0.67"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d50af8abc119fb8bb6dbabcfa89656f46f84aa0ac7688088608076ad2b459a84"
checksum = "3b3c6efbfc763e64eb85c11c25320f0737cb7364c4b6336db90aa9ebe27a0bbd"
dependencies = [
"thiserror-impl",
]
[[package]]
name = "thiserror-impl"
version = "1.0.64"
version = "1.0.67"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "08904e7672f5eb876eaaf87e0ce17857500934f4981c4a0ab2b4aa98baac7fc3"
checksum = "b607164372e89797d78b8e23a6d67d5d1038c1c65efd52e1389ef8b77caba2a6"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.82",
"syn 2.0.87",
]
[[package]]
@@ -3515,7 +3527,7 @@ checksum = "34704c8d6ebcbc939824180af020566b01a7c01f80641264eba0999f6c2b6be7"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.82",
"syn 2.0.87",
]
[[package]]
@@ -3773,7 +3785,7 @@ checksum = "6b91f57fe13a38d0ce9e28a03463d8d3c2468ed03d75375110ec71d93b449a08"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.82",
"syn 2.0.87",
]
[[package]]
@@ -3859,7 +3871,7 @@ dependencies = [
"once_cell",
"proc-macro2",
"quote",
"syn 2.0.82",
"syn 2.0.87",
"wasm-bindgen-shared",
]
@@ -3893,7 +3905,7 @@ checksum = "26c6ab57572f7a24a4985830b120de1594465e5d500f24afe89e16b4e833ef68"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.82",
"syn 2.0.87",
"wasm-bindgen-backend",
"wasm-bindgen-shared",
]
@@ -3927,7 +3939,7 @@ checksum = "c97b2ef2c8d627381e51c071c2ab328eac606d3f69dd82bcbca20a9e389d95f0"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.82",
"syn 2.0.87",
]
[[package]]
@@ -4215,7 +4227,7 @@ checksum = "9ce1b18ccd8e73a9321186f97e46f9f04b778851177567b1975109d26a08d2a6"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.82",
"syn 2.0.87",
]
[[package]]

View File

@@ -81,6 +81,7 @@ hashbrown = { version = "0.15.0", default-features = false, features = [
ignore = { version = "0.4.22" }
imara-diff = { version = "0.1.5" }
imperative = { version = "1.0.4" }
indexmap = {version = "2.6.0" }
indicatif = { version = "0.17.8" }
indoc = { version = "2.0.4" }
insta = { version = "1.35.1" }
@@ -101,7 +102,7 @@ matchit = { version = "0.8.1" }
memchr = { version = "2.7.1" }
mimalloc = { version = "0.1.39" }
natord = { version = "1.0.9" }
notify = { version = "6.1.1" }
notify = { version = "7.0.0" }
ordermap = { version = "0.5.0" }
path-absolutize = { version = "3.1.1" }
path-slash = { version = "0.2.1" }
@@ -188,8 +189,9 @@ missing_panics_doc = "allow"
module_name_repetitions = "allow"
must_use_candidate = "allow"
similar_names = "allow"
single_match_else = "allow"
too_many_lines = "allow"
# To allow `#[allow(clippy::all)]` in `crates/ruff_python_parser/src/python.rs`.
# Without the hashes we run into a `rustfmt` bug in some snapshot tests, see #13250
needless_raw_string_hashes = "allow"
# Disallowed restriction lints
print_stdout = "warn"

View File

@@ -136,8 +136,8 @@ curl -LsSf https://astral.sh/ruff/install.sh | sh
powershell -c "irm https://astral.sh/ruff/install.ps1 | iex"
# For a specific version.
curl -LsSf https://astral.sh/ruff/0.7.1/install.sh | sh
powershell -c "irm https://astral.sh/ruff/0.7.1/install.ps1 | iex"
curl -LsSf https://astral.sh/ruff/0.7.2/install.sh | sh
powershell -c "irm https://astral.sh/ruff/0.7.2/install.ps1 | iex"
```
You can also install Ruff via [Homebrew](https://formulae.brew.sh/formula/ruff), [Conda](https://anaconda.org/conda-forge/ruff),
@@ -170,7 +170,7 @@ Ruff can also be used as a [pre-commit](https://pre-commit.com/) hook via [`ruff
```yaml
- repo: https://github.com/astral-sh/ruff-pre-commit
# Ruff version.
rev: v0.7.1
rev: v0.7.2
hooks:
# Run the linter.
- id: ruff
@@ -417,6 +417,7 @@ Ruff is used by a number of major open-source projects and companies, including:
- [Babel](https://github.com/python-babel/babel)
- Benchling ([Refac](https://github.com/benchling/refac))
- [Bokeh](https://github.com/bokeh/bokeh)
- CrowdCent ([NumerBlox](https://github.com/crowdcent/numerblox)) <!-- typos: ignore -->
- [Cryptography (PyCA)](https://github.com/pyca/cryptography)
- CERN ([Indico](https://getindico.io/))
- [DVC](https://github.com/iterative/dvc)

View File

@@ -12,6 +12,7 @@ pn = "pn" # `import panel as pn` is a thing
poit = "poit"
BA = "BA" # acronym for "Bad Allowed", used in testing.
jod = "jod" # e.g., `jod-thread`
Numer = "Numer" # Library name 'NumerBlox' in "Who's Using Ruff?"
[default]
extend-ignore-re = [

View File

@@ -13,7 +13,7 @@ license = { workspace = true }
[dependencies]
ruff_db = { workspace = true }
ruff_index = { workspace = true }
ruff_python_ast = { workspace = true }
ruff_python_ast = { workspace = true, features = ["salsa"] }
ruff_python_stdlib = { workspace = true }
ruff_source_file = { workspace = true }
ruff_text_size = { workspace = true }
@@ -24,7 +24,8 @@ bitflags = { workspace = true }
camino = { workspace = true }
compact_str = { workspace = true }
countme = { workspace = true }
itertools = { workspace = true}
indexmap = { workspace = true }
itertools = { workspace = true }
ordermap = { workspace = true }
salsa = { workspace = true }
thiserror = { workspace = true }
@@ -43,10 +44,9 @@ red_knot_test = { workspace = true }
red_knot_vendored = { workspace = true }
anyhow = { workspace = true }
dir-test = {workspace = true}
dir-test = { workspace = true }
insta = { workspace = true }
tempfile = { workspace = true }
[lints]
workspace = true

View File

@@ -0,0 +1 @@
wrap = 100

View File

@@ -0,0 +1,18 @@
# Starred expression annotations
Type annotations for `*args` can be starred expressions themselves:
```py
from typing_extensions import TypeVarTuple
Ts = TypeVarTuple("Ts")
def append_int(*args: *Ts) -> tuple[*Ts, int]:
# TODO: should show some representation of the variadic generic type
reveal_type(args) # revealed: @Todo
return (*args, 1)
# TODO should be tuple[Literal[True], Literal["a"], int]
reveal_type(append_int(True, "a")) # revealed: @Todo
```

View File

@@ -0,0 +1,9 @@
# String annotations
```py
def f() -> "int":
return 1
# TODO: We do not support string annotations, but we should not panic if we encounter them
reveal_type(f()) # revealed: @Todo
```

View File

@@ -23,12 +23,90 @@ x: int
x = "foo" # error: [invalid-assignment] "Object of type `Literal["foo"]` is not assignable to `int`"
```
## PEP-604 annotations not yet supported
## Tuple annotations are understood
```py path=module.py
from typing_extensions import Unpack
a: tuple[()] = ()
b: tuple[int] = (42,)
c: tuple[str, int] = ("42", 42)
d: tuple[tuple[str, str], tuple[int, int]] = (("foo", "foo"), (42, 42))
e: tuple[str, ...] = ()
# TODO: we should not emit this error
# error: [call-possibly-unbound-method] "Method `__class_getitem__` of type `Literal[tuple]` is possibly unbound"
f: tuple[str, *tuple[int, ...], bytes] = ("42", b"42")
g: tuple[str, Unpack[tuple[int, ...]], bytes] = ("42", b"42")
h: tuple[list[int], list[int]] = ([], [])
i: tuple[str | int, str | int] = (42, 42)
j: tuple[str | int] = (42,)
```
```py path=script.py
from module import a, b, c, d, e, f, g, h, i, j
reveal_type(a) # revealed: tuple[()]
reveal_type(b) # revealed: tuple[int]
reveal_type(c) # revealed: tuple[str, int]
reveal_type(d) # revealed: tuple[tuple[str, str], tuple[int, int]]
# TODO: homogenous tuples, PEP-646 tuples
reveal_type(e) # revealed: @Todo
reveal_type(f) # revealed: @Todo
reveal_type(g) # revealed: @Todo
# TODO: support more kinds of type expressions in annotations
reveal_type(h) # revealed: @Todo
reveal_type(i) # revealed: tuple[str | int, str | int]
reveal_type(j) # revealed: tuple[str | int]
```
## Incorrect tuple assignments are complained about
```py
def f() -> str | None:
# error: [invalid-assignment] "Object of type `tuple[Literal[1], Literal[2]]` is not assignable to `tuple[()]`"
a: tuple[()] = (1, 2)
# error: [invalid-assignment] "Object of type `tuple[Literal["foo"]]` is not assignable to `tuple[int]`"
b: tuple[int] = ("foo",)
# error: [invalid-assignment] "Object of type `tuple[list, Literal["foo"]]` is not assignable to `tuple[str | int, str]`"
c: tuple[str | int, str] = ([], "foo")
```
## PEP-604 annotations are supported
```py
def foo() -> str | int | None:
return None
# TODO: should be `str | None` (but Todo is better than `Unknown`)
reveal_type(f()) # revealed: @Todo
reveal_type(foo()) # revealed: str | int | None
def bar() -> str | str | None:
return None
reveal_type(bar()) # revealed: str | None
def baz() -> str | str:
return "Hello, world!"
reveal_type(baz()) # revealed: str
```
## Attribute expressions in type annotations are understood
```py
import builtins
int = "foo"
a: builtins.int = 42
# error: [invalid-assignment] "Object of type `Literal["bar"]` is not assignable to `int`"
b: builtins.int = "bar"
c: builtins.tuple[builtins.tuple[builtins.int, builtins.int], builtins.int] = ((42, 42), 42)
# error: [invalid-assignment] "Object of type `Literal["foo"]` is not assignable to `tuple[tuple[int, int], int]`"
c: builtins.tuple[builtins.tuple[builtins.int, builtins.int], builtins.int] = "foo"
```

View File

@@ -0,0 +1,182 @@
# Augmented assignment
## Basic
```py
x = 3
x -= 1
reveal_type(x) # revealed: Literal[2]
x = 1.0
x /= 2
reveal_type(x) # revealed: float
```
## Dunder methods
```py
class C:
def __isub__(self, other: int) -> str:
return "Hello, world!"
x = C()
x -= 1
reveal_type(x) # revealed: str
class C:
def __iadd__(self, other: str) -> float:
return 1.0
x = C()
x += "Hello"
reveal_type(x) # revealed: float
```
## Unsupported types
```py
class C:
def __isub__(self, other: str) -> int:
return 42
x = C()
x -= 1
# TODO: should error, once operand type check is implemented
reveal_type(x) # revealed: int
```
## Method union
```py
def bool_instance() -> bool:
return True
flag = bool_instance()
class Foo:
if bool_instance():
def __iadd__(self, other: int) -> str:
return "Hello, world!"
else:
def __iadd__(self, other: int) -> int:
return 42
f = Foo()
f += 12
reveal_type(f) # revealed: str | int
```
## Partially bound `__iadd__`
```py
def bool_instance() -> bool:
return True
class Foo:
if bool_instance():
def __iadd__(self, other: str) -> int:
return 42
f = Foo()
# TODO: We should emit an `unsupported-operator` error here, possibly with the information
# that `Foo.__iadd__` may be unbound as additional context.
f += "Hello, world!"
reveal_type(f) # revealed: int | Unknown
```
## Partially bound with `__add__`
```py
def bool_instance() -> bool:
return True
class Foo:
def __add__(self, other: str) -> str:
return "Hello, world!"
if bool_instance():
def __iadd__(self, other: str) -> int:
return 42
f = Foo()
f += "Hello, world!"
reveal_type(f) # revealed: int | str
```
## Partially bound target union
```py
def bool_instance() -> bool:
return True
class Foo:
def __add__(self, other: int) -> str:
return "Hello, world!"
if bool_instance():
def __iadd__(self, other: int) -> int:
return 42
if bool_instance():
f = Foo()
else:
f = 42.0
f += 12
reveal_type(f) # revealed: int | str | float
```
## Target union
```py
def bool_instance() -> bool:
return True
flag = bool_instance()
class Foo:
def __iadd__(self, other: int) -> str:
return "Hello, world!"
if flag:
f = Foo()
else:
f = 42.0
f += 12
reveal_type(f) # revealed: str | float
```
## Partially bound target union with `__add__`
```py
def bool_instance() -> bool:
return True
flag = bool_instance()
class Foo:
def __add__(self, other: int) -> str:
return "Hello, world!"
if bool_instance():
def __iadd__(self, other: int) -> int:
return 42
class Bar:
def __add__(self, other: int) -> bytes:
return b"Hello, world!"
def __iadd__(self, other: int) -> float:
return 42.0
if flag:
f = Foo()
else:
f = Bar()
f += 12
reveal_type(f) # revealed: int | str | float
```

View File

@@ -6,11 +6,19 @@
x = foo # error: [unresolved-reference] "Name `foo` used when not defined"
foo = 1
# error: [unresolved-reference]
# revealed: Unbound
# No error `unresolved-reference` diagnostic is reported for `x`. This is
# desirable because we would get a lot of cascading errors even though there
# is only one root cause (the unbound variable `foo`).
# revealed: Unknown
reveal_type(x)
```
Note: in this particular example, one could argue that the most likely error would be a wrong order
of the `x`/`foo` definitions, and so it could be desirable to infer `Literal[1]` for the type of
`x`. On the other hand, there might be a variable `fob` a little higher up in this file, and the
actual error might have been just a typo. Inferring `Unknown` thus seems like the safest option.
## Unbound class variable
Name lookups within a class scope fall back to globals, but lookups of class attributes don't.
@@ -30,3 +38,22 @@ class C:
reveal_type(C.x) # revealed: Literal[2]
reveal_type(C.y) # revealed: Literal[1]
```
## Possibly unbound in class and global scope
```py
def bool_instance() -> bool:
return True
if bool_instance():
x = "abc"
class C:
if bool_instance():
x = 1
# error: [possibly-unresolved-reference]
y = x
reveal_type(C.y) # revealed: Literal[1] | Literal["abc"]
```

View File

@@ -18,3 +18,38 @@ else:
reveal_type(C.x) # revealed: Literal[1, 2]
```
## Inherited attributes
```py
class A:
X = "foo"
class B(A): ...
class C(B): ...
reveal_type(C.X) # revealed: Literal["foo"]
```
## Inherited attributes (multiple inheritance)
```py
class O: ...
class F(O):
X = 56
class E(O):
X = 42
class D(O): ...
class C(D, F): ...
class B(E, D): ...
class A(B, C): ...
# revealed: tuple[Literal[A], Literal[B], Literal[E], Literal[C], Literal[D], Literal[F], Literal[O], Literal[object]]
reveal_type(A.__mro__)
# `E` is earlier in the MRO than `F`, so we should use the type of `E.X`
reveal_type(A.X) # revealed: Literal[42]
```

View File

@@ -9,8 +9,8 @@ For references, see:
## Operations
We support inference for all Python's binary operators:
`+`, `-`, `*`, `@`, `/`, `//`, `%`, `**`, `<<`, `>>`, `&`, `^`, and `|`.
We support inference for all Python's binary operators: `+`, `-`, `*`, `@`, `/`, `//`, `%`, `**`,
`<<`, `>>`, `&`, `^`, and `|`.
```py
class A:
@@ -152,9 +152,8 @@ reveal_type(B() - A()) # revealed: int
## Non-reflected precedence in general
In general, if the left-hand side defines `__add__` and the right-hand side
defines `__radd__` and the right-hand side is not a subtype of the left-hand
side, `lhs.__add__` will take precedence:
In general, if the left-hand side defines `__add__` and the right-hand side defines `__radd__` and
the right-hand side is not a subtype of the left-hand side, `lhs.__add__` will take precedence:
```py
class A:
@@ -181,9 +180,8 @@ reveal_type(C() + C()) # revealed: int
## Reflected precedence for subtypes (in some cases)
If the right-hand operand is a subtype of the left-hand operand and has a
different implementation of the reflected method, the reflected method on the
right-hand operand takes precedence.
If the right-hand operand is a subtype of the left-hand operand and has a different implementation
of the reflected method, the reflected method on the right-hand operand takes precedence.
```py
class A:
@@ -204,18 +202,13 @@ reveal_type(A() + B()) # revealed: MyString
# N.B. Still a subtype of `A`, even though `A` does not appear directly in the class's `__bases__`
class C(B): ...
# TODO: we currently only understand direct subclasses as subtypes of the superclass.
# We need to iterate through the full MRO rather than just the class's bases;
# if we do, we'll understand `C` as a subtype of `A`, and correctly understand this as being
# `MyString` rather than `str`
reveal_type(A() + C()) # revealed: str
reveal_type(A() + C()) # revealed: MyString
```
## Reflected precedence 2
If the right-hand operand is a subtype of the left-hand operand, but does not
override the reflected method, the left-hand operand's non-reflected method
still takes precedence:
If the right-hand operand is a subtype of the left-hand operand, but does not override the reflected
method, the left-hand operand's non-reflected method still takes precedence:
```py
class A:
@@ -232,17 +225,15 @@ reveal_type(A() + B()) # revealed: str
## Only reflected supported
For example, at runtime, `(1).__add__(1.2)` is `NotImplemented`, but
`(1.2).__radd__(1) == 2.2`, meaning that `1 + 1.2` succeeds at runtime
(producing `2.2`). The runtime tries the second one only if the first one
returns `NotImplemented` to signal failure.
For example, at runtime, `(1).__add__(1.2)` is `NotImplemented`, but `(1.2).__radd__(1) == 2.2`,
meaning that `1 + 1.2` succeeds at runtime (producing `2.2`). The runtime tries the second one only
if the first one returns `NotImplemented` to signal failure.
Typeshed and other stubs annotate dunder-method calls that would return
`NotImplemented` as being "illegal" calls. `int.__add__` is annotated as only
"accepting" `int`s, even though it strictly-speaking "accepts" any other object
without raising an exception -- it will simply return `NotImplemented`,
allowing the runtime to try the `__radd__` method of the right-hand operand
as well.
Typeshed and other stubs annotate dunder-method calls that would return `NotImplemented` as being
"illegal" calls. `int.__add__` is annotated as only "accepting" `int`s, even though it
strictly-speaking "accepts" any other object without raising an exception -- it will simply return
`NotImplemented`, allowing the runtime to try the `__radd__` method of the right-hand operand as
well.
```py
class A:
@@ -308,8 +299,8 @@ reveal_type(y + 4.12) # revealed: int
## With literal types
When we have a literal type for one operand, we're able to fall back to the
instance handling for its instance super-type.
When we have a literal type for one operand, we're able to fall back to the instance handling for
its instance super-type.
```py
class A:
@@ -348,15 +339,13 @@ reveal_type(literal_string_instance + A()) # revealed: @Todo
## Operations involving instances of classes inheriting from `Any`
`Any` and `Unknown` represent a set of possible runtime objects, wherein the
bounds of the set are unknown. Whether the left-hand operand's dunder or the
right-hand operand's reflected dunder depends on whether the right-hand operand
is an instance of a class that is a subclass of the left-hand operand's class
and overrides the reflected dunder. In the following example, because of the
unknowable nature of `Any`/`Unknown`, we must consider both possibilities:
`Any`/`Unknown` might resolve to an unknown third class that inherits from `X`
and overrides `__radd__`; but it also might not. Thus, the correct answer here
for the `reveal_type` is `int | Unknown`.
`Any` and `Unknown` represent a set of possible runtime objects, wherein the bounds of the set are
unknown. Whether the left-hand operand's dunder or the right-hand operand's reflected dunder depends
on whether the right-hand operand is an instance of a class that is a subclass of the left-hand
operand's class and overrides the reflected dunder. In the following example, because of the
unknowable nature of `Any`/`Unknown`, we must consider both possibilities: `Any`/`Unknown` might
resolve to an unknown third class that inherits from `X` and overrides `__radd__`; but it also might
not. Thus, the correct answer here for the `reveal_type` is `int | Unknown`.
```py
from does_not_exist import Foo # error: [unresolved-import]
@@ -426,10 +415,9 @@ reveal_type(B() + C())
### Reflected dunder is not tried between two objects of the same type
For the specific case where the left-hand operand is the exact same type as the
right-hand operand, the reflected dunder of the right-hand operand is not
tried; the runtime short-circuits after trying the unreflected dunder of the
left-hand operand. For context, see
For the specific case where the left-hand operand is the exact same type as the right-hand operand,
the reflected dunder of the right-hand operand is not tried; the runtime short-circuits after trying
the unreflected dunder of the left-hand operand. For context, see
[this mailing list discussion](https://mail.python.org/archives/list/python-dev@python.org/thread/7NZUCODEAPQFMRFXYRMGJXDSIS3WJYIV/).
```py

View File

@@ -0,0 +1,78 @@
# Short-Circuit Evaluation
## Not all boolean expressions must be evaluated
In `or` expressions, if the left-hand side is truthy, the right-hand side is not evaluated.
Similarly, in `and` expressions, if the left-hand side is falsy, the right-hand side is not
evaluated.
```py
def bool_instance() -> bool:
return True
if bool_instance() or (x := 1):
# error: [possibly-unresolved-reference]
reveal_type(x) # revealed: Literal[1]
if bool_instance() and (x := 1):
# error: [possibly-unresolved-reference]
reveal_type(x) # revealed: Literal[1]
```
## First expression is always evaluated
```py
def bool_instance() -> bool:
return True
if (x := 1) or bool_instance():
reveal_type(x) # revealed: Literal[1]
if (x := 1) and bool_instance():
reveal_type(x) # revealed: Literal[1]
```
## Statically known truthiness
```py
if True or (x := 1):
# TODO: infer that the second arm is never executed, and raise `unresolved-reference`.
# error: [possibly-unresolved-reference]
reveal_type(x) # revealed: Literal[1]
if True and (x := 1):
# TODO: infer that the second arm is always executed, do not raise a diagnostic
# error: [possibly-unresolved-reference]
reveal_type(x) # revealed: Literal[1]
```
## Later expressions can always use variables from earlier expressions
```py
def bool_instance() -> bool:
return True
bool_instance() or (x := 1) or reveal_type(x) # revealed: Literal[1]
# error: [unresolved-reference]
bool_instance() or reveal_type(y) or (y := 1) # revealed: Unknown
```
## Nested expressions
```py
def bool_instance() -> bool:
return True
if bool_instance() or ((x := 1) and bool_instance()):
# error: [possibly-unresolved-reference]
reveal_type(x) # revealed: Literal[1]
if ((y := 1) and bool_instance()) or bool_instance():
reveal_type(y) # revealed: Literal[1]
# error: [possibly-unresolved-reference]
if (bool_instance() and (z := 1)) or reveal_type(z): # revealed: Literal[1]
# error: [possibly-unresolved-reference]
reveal_type(z) # revealed: Literal[1]
```

View File

@@ -1,7 +1,7 @@
# Comparison: Byte literals
These tests assert that we infer precise `Literal` types for comparisons between objects
inferred as having `Literal` bytes types:
These tests assert that we infer precise `Literal` types for comparisons between objects inferred as
having `Literal` bytes types:
```py
reveal_type(b"abc" == b"abc") # revealed: Literal[True]

View File

@@ -0,0 +1,40 @@
# Identity tests
```py
class A: ...
def get_a() -> A: ...
def get_object() -> object: ...
a1 = get_a()
a2 = get_a()
n1 = None
n2 = None
o = get_object()
reveal_type(a1 is a1) # revealed: bool
reveal_type(a1 is a2) # revealed: bool
reveal_type(n1 is n1) # revealed: Literal[True]
reveal_type(n1 is n2) # revealed: Literal[True]
reveal_type(a1 is n1) # revealed: Literal[False]
reveal_type(n1 is a1) # revealed: Literal[False]
reveal_type(a1 is o) # revealed: bool
reveal_type(n1 is o) # revealed: bool
reveal_type(a1 is not a1) # revealed: bool
reveal_type(a1 is not a2) # revealed: bool
reveal_type(n1 is not n1) # revealed: Literal[False]
reveal_type(n1 is not n2) # revealed: Literal[False]
reveal_type(a1 is not n1) # revealed: Literal[True]
reveal_type(n1 is not a1) # revealed: Literal[True]
reveal_type(a1 is not o) # revealed: bool
reveal_type(n1 is not o) # revealed: bool
```

View File

@@ -0,0 +1,160 @@
# Comparison: Membership Test
In Python, the term "membership test operators" refers to the operators `in` and `not in`. To
customize their behavior, classes can implement one of the special methods `__contains__`,
`__iter__`, or `__getitem__`.
For references, see:
- <https://docs.python.org/3/reference/expressions.html#membership-test-details>
- <https://docs.python.org/3/reference/datamodel.html#object.__contains__>
- <https://snarky.ca/unravelling-membership-testing/>
## Implements `__contains__`
Classes can support membership tests by implementing the `__contains__` method:
```py
class A:
def __contains__(self, item: str) -> bool:
return True
reveal_type("hello" in A()) # revealed: bool
reveal_type("hello" not in A()) # revealed: bool
# TODO: should emit diagnostic, need to check arg type, will fail
reveal_type(42 in A()) # revealed: bool
reveal_type(42 not in A()) # revealed: bool
```
## Implements `__iter__`
Classes that don't implement `__contains__`, but do implement `__iter__`, also support containment
checks; the needle will be sought in their iterated items:
```py
class StringIterator:
def __next__(self) -> str:
return "foo"
class A:
def __iter__(self) -> StringIterator:
return StringIterator()
reveal_type("hello" in A()) # revealed: bool
reveal_type("hello" not in A()) # revealed: bool
reveal_type(42 in A()) # revealed: bool
reveal_type(42 not in A()) # revealed: bool
```
## Implements `__getitems__`
The final fallback is to implement `__getitem__` for integer keys. Python will call `__getitem__`
with `0`, `1`, `2`... until either the needle is found (leading the membership test to evaluate to
`True`) or `__getitem__` raises `IndexError` (the raised exception is swallowed, but results in the
membership test evaluating to `False`).
```py
class A:
def __getitem__(self, key: int) -> str:
return "foo"
reveal_type("hello" in A()) # revealed: bool
reveal_type("hello" not in A()) # revealed: bool
reveal_type(42 in A()) # revealed: bool
reveal_type(42 not in A()) # revealed: bool
```
## Wrong Return Type
Python coerces the results of containment checks to `bool`, even if `__contains__` returns a
non-bool:
```py
class A:
def __contains__(self, item: str) -> str:
return "foo"
reveal_type("hello" in A()) # revealed: bool
reveal_type("hello" not in A()) # revealed: bool
```
## Literal Result for `in` and `not in`
`__contains__` with a literal return type may result in a `BooleanLiteral` outcome.
```py
from typing import Literal
class AlwaysTrue:
def __contains__(self, item: int) -> Literal[1]:
return 1
class AlwaysFalse:
def __contains__(self, item: int) -> Literal[""]:
return ""
reveal_type(42 in AlwaysTrue()) # revealed: Literal[True]
reveal_type(42 not in AlwaysTrue()) # revealed: Literal[False]
reveal_type(42 in AlwaysFalse()) # revealed: Literal[False]
reveal_type(42 not in AlwaysFalse()) # revealed: Literal[True]
```
## No Fallback for `__contains__`
If `__contains__` is implemented, checking membership of a type it doesn't accept is an error; it
doesn't result in a fallback to `__iter__` or `__getitem__`:
```py
class CheckContains: ...
class CheckIter: ...
class CheckGetItem: ...
class CheckIterIterator:
def __next__(self) -> CheckIter:
return CheckIter()
class A:
def __contains__(self, item: CheckContains) -> bool:
return True
def __iter__(self) -> CheckIterIterator:
return CheckIterIterator()
def __getitem__(self, key: int) -> CheckGetItem:
return CheckGetItem()
reveal_type(CheckContains() in A()) # revealed: bool
# TODO: should emit diagnostic, need to check arg type,
# should not fall back to __iter__ or __getitem__
reveal_type(CheckIter() in A()) # revealed: bool
reveal_type(CheckGetItem() in A()) # revealed: bool
class B:
def __iter__(self) -> CheckIterIterator:
return CheckIterIterator()
def __getitem__(self, key: int) -> CheckGetItem:
return CheckGetItem()
reveal_type(CheckIter() in B()) # revealed: bool
# Always use `__iter__`, regardless of iterated type; there's no NotImplemented
# in this case, so there's no fallback to `__getitem__`
reveal_type(CheckGetItem() in B()) # revealed: bool
```
## Invalid Old-Style Iteration
If `__getitem__` is implemented but does not accept integer arguments, then the membership test is
not supported and should trigger a diagnostic.
```py
class A:
def __getitem__(self, key: str) -> str:
return "foo"
# TODO should emit a diagnostic
reveal_type(42 in A()) # revealed: bool
reveal_type("hello" in A()) # revealed: bool
```

View File

@@ -0,0 +1,328 @@
# Comparison: Rich Comparison
Rich comparison operations (`==`, `!=`, `<`, `<=`, `>`, `>=`) in Python are implemented through
double-underscore methods that allow customization of comparison behavior.
For references, see:
- <https://docs.python.org/3/reference/datamodel.html#object.__lt__>
- <https://snarky.ca/unravelling-rich-comparison-operators/>
## Rich Comparison Dunder Implementations For Same Class
Classes can support rich comparison by implementing dunder methods like `__eq__`, `__ne__`, etc. The
most common case involves implementing these methods for the same type:
```py
from __future__ import annotations
class A:
def __eq__(self, other: A) -> int:
return 42
def __ne__(self, other: A) -> float:
return 42.0
def __lt__(self, other: A) -> str:
return "42"
def __le__(self, other: A) -> bytes:
return b"42"
def __gt__(self, other: A) -> list:
return [42]
def __ge__(self, other: A) -> set:
return {42}
reveal_type(A() == A()) # revealed: int
reveal_type(A() != A()) # revealed: float
reveal_type(A() < A()) # revealed: str
reveal_type(A() <= A()) # revealed: bytes
reveal_type(A() > A()) # revealed: list
reveal_type(A() >= A()) # revealed: set
```
## Rich Comparison Dunder Implementations for Other Class
In some cases, classes may implement rich comparison dunder methods for comparisons with a different
type:
```py
from __future__ import annotations
class A:
def __eq__(self, other: B) -> int:
return 42
def __ne__(self, other: B) -> float:
return 42.0
def __lt__(self, other: B) -> str:
return "42"
def __le__(self, other: B) -> bytes:
return b"42"
def __gt__(self, other: B) -> list:
return [42]
def __ge__(self, other: B) -> set:
return {42}
class B: ...
reveal_type(A() == B()) # revealed: int
reveal_type(A() != B()) # revealed: float
reveal_type(A() < B()) # revealed: str
reveal_type(A() <= B()) # revealed: bytes
reveal_type(A() > B()) # revealed: list
reveal_type(A() >= B()) # revealed: set
```
## Reflected Comparisons
Fallback to the right-hand sides comparison methods occurs when the left-hand side does not define
them. Note: class `B` has its own `__eq__` and `__ne__` methods to override those of `object`, but
these methods will be ignored here because they require a mismatched operand type.
```py
from __future__ import annotations
class A:
def __eq__(self, other: B) -> int:
return 42
def __ne__(self, other: B) -> float:
return 42.0
def __lt__(self, other: B) -> str:
return "42"
def __le__(self, other: B) -> bytes:
return b"42"
def __gt__(self, other: B) -> list:
return [42]
def __ge__(self, other: B) -> set:
return {42}
class B:
# To override builtins.object.__eq__ and builtins.object.__ne__
# TODO these should emit an invalid override diagnostic
def __eq__(self, other: str) -> B:
return B()
def __ne__(self, other: str) -> B:
return B()
# TODO: should be `int` and `float`.
# Need to check arg type and fall back to `rhs.__eq__` and `rhs.__ne__`.
#
# Because `object.__eq__` and `object.__ne__` accept `object` in typeshed,
# this can only happen with an invalid override of these methods,
# but we still support it.
reveal_type(B() == A()) # revealed: B
reveal_type(B() != A()) # revealed: B
reveal_type(B() < A()) # revealed: list
reveal_type(B() <= A()) # revealed: set
reveal_type(B() > A()) # revealed: str
reveal_type(B() >= A()) # revealed: bytes
class C:
def __gt__(self, other: C) -> int:
return 42
def __ge__(self, other: C) -> float:
return 42.0
reveal_type(C() < C()) # revealed: int
reveal_type(C() <= C()) # revealed: float
```
## Reflected Comparisons with Subclasses
When subclasses override comparison methods, these overridden methods take precedence over those in
the parent class. Class `B` inherits from `A` and redefines comparison methods to return types other
than `A`.
```py
from __future__ import annotations
class A:
def __eq__(self, other: A) -> A:
return A()
def __ne__(self, other: A) -> A:
return A()
def __lt__(self, other: A) -> A:
return A()
def __le__(self, other: A) -> A:
return A()
def __gt__(self, other: A) -> A:
return A()
def __ge__(self, other: A) -> A:
return A()
class B(A):
def __eq__(self, other: A) -> int:
return 42
def __ne__(self, other: A) -> float:
return 42.0
def __lt__(self, other: A) -> str:
return "42"
def __le__(self, other: A) -> bytes:
return b"42"
def __gt__(self, other: A) -> list:
return [42]
def __ge__(self, other: A) -> set:
return {42}
reveal_type(A() == B()) # revealed: int
reveal_type(A() != B()) # revealed: float
reveal_type(A() < B()) # revealed: list
reveal_type(A() <= B()) # revealed: set
reveal_type(A() > B()) # revealed: str
reveal_type(A() >= B()) # revealed: bytes
```
## Reflected Comparisons with Subclass But Falls Back to LHS
In the case of a subclass, the right-hand side has priority. However, if the overridden dunder
method has an mismatched type to operand, the comparison will fall back to the left-hand side.
```py
from __future__ import annotations
class A:
def __lt__(self, other: A) -> A:
return A()
def __gt__(self, other: A) -> A:
return A()
class B(A):
def __lt__(self, other: int) -> B:
return B()
def __gt__(self, other: int) -> B:
return B()
# TODO: should be `A`, need to check argument type and fall back to LHS method
reveal_type(A() < B()) # revealed: B
reveal_type(A() > B()) # revealed: B
```
## Operations involving instances of classes inheriting from `Any`
`Any` and `Unknown` represent a set of possible runtime objects, wherein the bounds of the set are
unknown. Whether the left-hand operand's dunder or the right-hand operand's reflected dunder depends
on whether the right-hand operand is an instance of a class that is a subclass of the left-hand
operand's class and overrides the reflected dunder. In the following example, because of the
unknowable nature of `Any`/`Unknown`, we must consider both possibilities: `Any`/`Unknown` might
resolve to an unknown third class that inherits from `X` and overrides `__gt__`; but it also might
not. Thus, the correct answer here for the `reveal_type` is `int | Unknown`.
(This test is referenced from `mdtest/binary/instances.md`)
```py
from does_not_exist import Foo # error: [unresolved-import]
reveal_type(Foo) # revealed: Unknown
class X:
def __lt__(self, other: object) -> int:
return 42
class Y(Foo): ...
# TODO: Should be `int | Unknown`; see above discussion.
reveal_type(X() < Y()) # revealed: int
```
## Equality and Inequality Fallback
This test confirms that `==` and `!=` comparisons default to identity comparisons (`is`, `is not`)
when argument types do not match the method signature.
Please refer to the [docs](https://docs.python.org/3/reference/datamodel.html#object.__eq__)
```py
from __future__ import annotations
class A:
# TODO both these overrides should emit invalid-override diagnostic
def __eq__(self, other: int) -> A:
return A()
def __ne__(self, other: int) -> A:
return A()
# TODO: it should be `bool`, need to check arg type and fall back to `is` and `is not`
reveal_type(A() == A()) # revealed: A
reveal_type(A() != A()) # revealed: A
```
## Object Comparisons with Typeshed
```py
class A: ...
reveal_type(A() == object()) # revealed: bool
reveal_type(A() != object()) # revealed: bool
reveal_type(object() == A()) # revealed: bool
reveal_type(object() != A()) # revealed: bool
# error: [unsupported-operator] "Operator `<` is not supported for types `A` and `object`"
# revealed: Unknown
reveal_type(A() < object())
```
## Numbers Comparison with typeshed
```py
reveal_type(1 == 1.0) # revealed: bool
reveal_type(1 != 1.0) # revealed: bool
reveal_type(1 < 1.0) # revealed: bool
reveal_type(1 <= 1.0) # revealed: bool
reveal_type(1 > 1.0) # revealed: bool
reveal_type(1 >= 1.0) # revealed: bool
reveal_type(1 == 2j) # revealed: bool
reveal_type(1 != 2j) # revealed: bool
# TODO: should be Unknown and emit diagnostic,
# need to check arg type and should be failed
reveal_type(1 < 2j) # revealed: bool
reveal_type(1 <= 2j) # revealed: bool
reveal_type(1 > 2j) # revealed: bool
reveal_type(1 >= 2j) # revealed: bool
def bool_instance() -> bool:
return True
def int_instance() -> int:
return 42
x = bool_instance()
y = int_instance()
reveal_type(x < y) # revealed: bool
reveal_type(y < x) # revealed: bool
reveal_type(4.2 < x) # revealed: bool
reveal_type(x < 4.2) # revealed: bool
```

View File

@@ -12,16 +12,18 @@ reveal_type(1 is 1) # revealed: bool
reveal_type(1 is not 1) # revealed: bool
reveal_type(1 is 2) # revealed: Literal[False]
reveal_type(1 is not 7) # revealed: Literal[True]
reveal_type(1 <= "" and 0 < 1) # revealed: @Todo | Literal[True]
# TODO: should be Unknown, and emit diagnostic, once we check call argument types
reveal_type(1 <= "" and 0 < 1) # revealed: bool
```
## Integer instance
```py
# TODO: implement lookup of `__eq__` on typeshed `int` stub.
def int_instance() -> int: ...
def int_instance() -> int:
return 42
reveal_type(1 == int_instance()) # revealed: @Todo
reveal_type(1 == int_instance()) # revealed: bool
reveal_type(9 < int_instance()) # revealed: bool
reveal_type(int_instance() < int_instance()) # revealed: bool
```

View File

@@ -5,9 +5,9 @@ Walking through examples:
- `a = A() < B() < C()`
1. `A() < B() and B() < C()` - split in N comparison
1. `A()` and `B()` - evaluate outcome types
1. `bool` and `bool` - evaluate truthiness
1. `A | B` - union of "first true" types
1. `A()` and `B()` - evaluate outcome types
1. `bool` and `bool` - evaluate truthiness
1. `A | B` - union of "first true" types
- `b = 0 < 1 < A() < 3`

View File

@@ -59,51 +59,51 @@ reveal_type(c >= d) # revealed: Literal[True]
```py
def bool_instance() -> bool: ...
def int_instance() -> int: ...
def int_instance() -> int:
return 42
a = (bool_instance(),)
b = (int_instance(),)
# TODO: All @Todo should be `bool`
reveal_type(a == a) # revealed: @Todo
reveal_type(a != a) # revealed: @Todo
reveal_type(a < a) # revealed: @Todo
reveal_type(a <= a) # revealed: @Todo
reveal_type(a > a) # revealed: @Todo
reveal_type(a >= a) # revealed: @Todo
reveal_type(a == a) # revealed: bool
reveal_type(a != a) # revealed: bool
reveal_type(a < a) # revealed: bool
reveal_type(a <= a) # revealed: bool
reveal_type(a > a) # revealed: bool
reveal_type(a >= a) # revealed: bool
reveal_type(a == b) # revealed: @Todo
reveal_type(a != b) # revealed: @Todo
reveal_type(a < b) # revealed: @Todo
reveal_type(a <= b) # revealed: @Todo
reveal_type(a > b) # revealed: @Todo
reveal_type(a >= b) # revealed: @Todo
reveal_type(a == b) # revealed: bool
reveal_type(a != b) # revealed: bool
reveal_type(a < b) # revealed: bool
reveal_type(a <= b) # revealed: bool
reveal_type(a > b) # revealed: bool
reveal_type(a >= b) # revealed: bool
```
#### Comparison Unsupported
If two tuples contain types that do not support comparison, the result may be `Unknown`.
However, `==` and `!=` are exceptions and can still provide definite results.
If two tuples contain types that do not support comparison, the result may be `Unknown`. However,
`==` and `!=` are exceptions and can still provide definite results.
```py
a = (1, 2)
b = (1, "hello")
# TODO: should be Literal[False]
reveal_type(a == b) # revealed: @Todo
# TODO: should be Literal[False], once we implement (in)equality for mismatched literals
reveal_type(a == b) # revealed: bool
# TODO: should be Literal[True]
reveal_type(a != b) # revealed: @Todo
# TODO: should be Literal[True], once we implement (in)equality for mismatched literals
reveal_type(a != b) # revealed: bool
# TODO: should be Unknown and add more informative diagnostics
reveal_type(a < b) # revealed: @Todo
reveal_type(a <= b) # revealed: @Todo
reveal_type(a > b) # revealed: @Todo
reveal_type(a >= b) # revealed: @Todo
reveal_type(a < b) # revealed: bool
reveal_type(a <= b) # revealed: bool
reveal_type(a > b) # revealed: bool
reveal_type(a >= b) # revealed: bool
```
However, if the lexicographic comparison completes without reaching a point where str and int are compared,
Python will still produce a result based on the prior elements.
However, if the lexicographic comparison completes without reaching a point where str and int are
compared, Python will still produce a result based on the prior elements.
```py path=short_circuit.py
a = (1, 2)
@@ -145,13 +145,12 @@ class A:
a = (A(), A())
# TODO: All @Todo should be bool
reveal_type(a == a) # revealed: @Todo
reveal_type(a != a) # revealed: @Todo
reveal_type(a < a) # revealed: @Todo
reveal_type(a <= a) # revealed: @Todo
reveal_type(a > a) # revealed: @Todo
reveal_type(a >= a) # revealed: @Todo
reveal_type(a == a) # revealed: bool
reveal_type(a != a) # revealed: bool
reveal_type(a < a) # revealed: bool
reveal_type(a <= a) # revealed: bool
reveal_type(a > a) # revealed: bool
reveal_type(a >= a) # revealed: bool
```
### Membership Test Comparisons
@@ -159,7 +158,8 @@ reveal_type(a >= a) # revealed: @Todo
"Membership Test Comparisons" refers to the operators `in` and `not in`.
```py
def int_instance() -> int: ...
def int_instance() -> int:
return 42
a = (1, 2)
b = ((3, 4), (1, 2))
@@ -172,9 +172,8 @@ reveal_type(a not in b) # revealed: Literal[False]
reveal_type(a in c) # revealed: Literal[False]
reveal_type(a not in c) # revealed: Literal[True]
# TODO: All @Todo should be bool
reveal_type(a in d) # revealed: @Todo
reveal_type(a not in d) # revealed: @Todo
reveal_type(a in d) # revealed: bool
reveal_type(a not in d) # revealed: bool
```
### Identity Comparisons
@@ -189,10 +188,10 @@ c = (1, 2, 3)
reveal_type(a is (1, 2)) # revealed: bool
reveal_type(a is not (1, 2)) # revealed: bool
# TODO: Update to Literal[False] once str == int comparison is implemented
reveal_type(a is b) # revealed: @Todo
# TODO: Update to Literal[True] once str == int comparison is implemented
reveal_type(a is not b) # revealed: @Todo
# TODO should be Literal[False] once we implement comparison of mismatched literal types
reveal_type(a is b) # revealed: bool
# TODO should be Literal[True] once we implement comparison of mismatched literal types
reveal_type(a is not b) # revealed: bool
reveal_type(a is c) # revealed: Literal[False]
reveal_type(a is not c) # revealed: Literal[True]

View File

@@ -52,8 +52,8 @@ reveal_type(one_or_none is not None) # revealed: bool
## Union on both sides of the comparison
With unions on both sides, we need to consider the full cross product of
options when building the resulting (union) type:
With unions on both sides, we need to consider the full cross product of options when building the
resulting (union) type:
```py
def bool_instance() -> bool:
@@ -72,9 +72,9 @@ reveal_type(small > large) # revealed: Literal[False]
## Unsupported operations
Make sure we emit a diagnostic if *any* of the possible comparisons is
unsupported. For now, we fall back to `bool` for the result type instead of
trying to infer something more precise from the other (supported) variants:
Make sure we emit a diagnostic if *any* of the possible comparisons is unsupported. For now, we fall
back to `bool` for the result type instead of trying to infer something more precise from the other
(supported) variants:
```py
def bool_instance() -> bool:

View File

@@ -10,12 +10,16 @@ reveal_type(a) # revealed: bool
b = 0 not in 10 # error: "Operator `not in` is not supported for types `Literal[0]` and `Literal[10]`"
reveal_type(b) # revealed: bool
c = object() < 5 # error: "Operator `<` is not supported for types `object` and `int`"
reveal_type(c) # revealed: Unknown
# TODO: should error, once operand type check is implemented
# ("Operator `<` is not supported for types `object` and `int`")
c = object() < 5
# TODO: should be Unknown, once operand type check is implemented
reveal_type(c) # revealed: bool
# TODO should error, need to check if __lt__ signature is valid for right operand
# TODO: should error, once operand type check is implemented
# ("Operator `<` is not supported for types `int` and `object`")
d = 5 < object()
# TODO: should be `Unknown`
# TODO: should be Unknown, once operand type check is implemented
reveal_type(d) # revealed: bool
flag = bool_instance()
@@ -27,5 +31,6 @@ reveal_type(e) # revealed: bool
# TODO: should error, need to check if __lt__ signature is valid for right operand
# error may be "Operator `<` is not supported for types `int` and `str`, in comparing `tuple[Literal[1], Literal[2]]` with `tuple[Literal[1], Literal["hello"]]`
f = (1, 2) < (1, "hello")
reveal_type(f) # revealed: @Todo
# TODO: should be Unknown, once operand type check is implemented
reveal_type(f) # revealed: bool
```

View File

@@ -37,11 +37,11 @@ x = y
reveal_type(x) # revealed: Literal[3, 4, 5]
# revealed: Unbound | Literal[2]
# revealed: Literal[2]
# error: [possibly-unresolved-reference]
reveal_type(r)
# revealed: Unbound | Literal[5]
# revealed: Literal[5]
# error: [possibly-unresolved-reference]
reveal_type(s)
```

View File

@@ -21,7 +21,7 @@ match 0:
case 2:
y = 3
# revealed: Unbound | Literal[2, 3]
# revealed: Literal[2, 3]
# error: [possibly-unresolved-reference]
reveal_type(y)
```

View File

@@ -41,7 +41,12 @@ except EXCEPTIONS as f:
## Dynamic exception types
```py
def foo(x: type[AttributeError], y: tuple[type[OSError], type[RuntimeError]], z: tuple[type[BaseException], ...]):
# TODO: we should not emit these `call-possibly-unbound-method` errors for `tuple.__class_getitem__`
def foo(
x: type[AttributeError],
y: tuple[type[OSError], type[RuntimeError]], # error: [call-possibly-unbound-method]
z: tuple[type[BaseException], ...], # error: [call-possibly-unbound-method]
):
try:
help()
except x as e:

View File

@@ -1,40 +1,33 @@
# Control flow for exception handlers
These tests assert that we understand the possible "definition states" (which
symbols might or might not be defined) in the various branches of a
`try`/`except`/`else`/`finally` block.
These tests assert that we understand the possible "definition states" (which symbols might or might
not be defined) in the various branches of a `try`/`except`/`else`/`finally` block.
For a full writeup on the semantics of exception handlers,
see [this document][1].
For a full writeup on the semantics of exception handlers, see [this document][1].
The tests throughout this Markdown document use functions with names starting
with `could_raise_*` to mark definitions that might or might not succeed
(as the function could raise an exception). A type checker must assume that any
arbitrary function call could raise an exception in Python; this is just a
naming convention used in these tests for clarity, and to future-proof the
tests against possible future improvements whereby certain statements or
expressions could potentially be inferred as being incapable of causing an
exception to be raised.
The tests throughout this Markdown document use functions with names starting with `could_raise_*`
to mark definitions that might or might not succeed (as the function could raise an exception). A
type checker must assume that any arbitrary function call could raise an exception in Python; this
is just a naming convention used in these tests for clarity, and to future-proof the tests against
possible future improvements whereby certain statements or expressions could potentially be inferred
as being incapable of causing an exception to be raised.
## A single bare `except`
Consider the following `try`/`except` block, with a single bare `except:`.
There are different types for the variable `x` in the two branches of this
block, and we can't determine which branch might have been taken from the
perspective of code following this block. The inferred type after the block's
conclusion is therefore the union of the type at the end of the `try` suite
(`str`) and the type at the end of the `except` suite (`Literal[2]`).
Consider the following `try`/`except` block, with a single bare `except:`. There are different types
for the variable `x` in the two branches of this block, and we can't determine which branch might
have been taken from the perspective of code following this block. The inferred type after the
block's conclusion is therefore the union of the type at the end of the `try` suite (`str`) and the
type at the end of the `except` suite (`Literal[2]`).
*Within* the `except` suite, we must infer a union of all possible "definition
states" we could have been in at any point during the `try` suite. This is
because control flow could have jumped to the `except` suite without any of the
`try`-suite definitions successfully completing, with only *some* of the
`try`-suite definitions successfully completing, or indeed with *all* of them
successfully completing. The type of `x` at the beginning of the `except` suite
in this example is therefore `Literal[1] | str`, taking into account that we
might have jumped to the `except` suite before the
`x = could_raise_returns_str()` redefinition, but we *also* could have jumped
to the `except` suite *after* that redefinition.
*Within* the `except` suite, we must infer a union of all possible "definition states" we could have
been in at any point during the `try` suite. This is because control flow could have jumped to the
`except` suite without any of the `try`-suite definitions successfully completing, with only *some*
of the `try`-suite definitions successfully completing, or indeed with *all* of them successfully
completing. The type of `x` at the beginning of the `except` suite in this example is therefore
`Literal[1] | str`, taking into account that we might have jumped to the `except` suite before the
`x = could_raise_returns_str()` redefinition, but we *also* could have jumped to the `except` suite
*after* that redefinition.
```py path=union_type_inferred.py
def could_raise_returns_str() -> str:
@@ -54,9 +47,8 @@ except:
reveal_type(x) # revealed: str | Literal[2]
```
If `x` has the same type at the end of both branches, however, the branches
unify and `x` is not inferred as having a union type following the
`try`/`except` block:
If `x` has the same type at the end of both branches, however, the branches unify and `x` is not
inferred as having a union type following the `try`/`except` block:
```py path=branches_unify_to_non_union_type.py
def could_raise_returns_str() -> str:
@@ -74,13 +66,12 @@ reveal_type(x) # revealed: str
## A non-bare `except`
For simple `try`/`except` blocks, an `except TypeError:` handler has the same
control flow semantics as an `except:` handler. An `except TypeError:` handler
will not catch *all* exceptions: if this is the only handler, it opens up the
possibility that an exception might occur that would not be handled. However,
as described in [the document on exception-handling semantics][1], that would
lead to termination of the scope. It's therefore irrelevant to consider this
possibility when it comes to control-flow analysis.
For simple `try`/`except` blocks, an `except TypeError:` handler has the same control flow semantics
as an `except:` handler. An `except TypeError:` handler will not catch *all* exceptions: if this is
the only handler, it opens up the possibility that an exception might occur that would not be
handled. However, as described in [the document on exception-handling semantics][1], that would lead
to termination of the scope. It's therefore irrelevant to consider this possibility when it comes to
control-flow analysis.
```py
def could_raise_returns_str() -> str:
@@ -102,11 +93,9 @@ reveal_type(x) # revealed: str | Literal[2]
## Multiple `except` branches
If the scope reaches the final `reveal_type` call in this example,
either the `try`-block suite of statements was executed in its entirety,
or exactly one `except` suite was executed in its entirety.
The inferred type of `x` at this point is the union of the types at the end of
the three suites:
If the scope reaches the final `reveal_type` call in this example, either the `try`-block suite of
statements was executed in its entirety, or exactly one `except` suite was executed in its entirety.
The inferred type of `x` at this point is the union of the types at the end of the three suites:
- At the end of `try`, `type(x) == str`
- At the end of `except TypeError`, `x == 2`
@@ -136,11 +125,10 @@ reveal_type(x) # revealed: str | Literal[2, 3]
## Exception handlers with `else` branches (but no `finally`)
If we reach the `reveal_type` call at the end of this scope,
either the `try` and `else` suites were both executed in their entireties,
or the `except` suite was executed in its entirety. The type of `x` at this
point is the union of the type at the end of the `else` suite and the type at
the end of the `except` suite:
If we reach the `reveal_type` call at the end of this scope, either the `try` and `else` suites were
both executed in their entireties, or the `except` suite was executed in its entirety. The type of
`x` at this point is the union of the type at the end of the `else` suite and the type at the end of
the `except` suite:
- At the end of `else`, `x == 3`
- At the end of `except`, `x == 2`
@@ -167,10 +155,9 @@ else:
reveal_type(x) # revealed: Literal[2, 3]
```
For a block that has multiple `except` branches and an `else` branch, the same
principle applies. In order to reach the final `reveal_type` call,
either exactly one of the `except` suites must have been executed in its
entirety, or the `try` suite and the `else` suite must both have been executed
For a block that has multiple `except` branches and an `else` branch, the same principle applies. In
order to reach the final `reveal_type` call, either exactly one of the `except` suites must have
been executed in its entirety, or the `try` suite and the `else` suite must both have been executed
in their entireties:
```py
@@ -201,10 +188,9 @@ reveal_type(x) # revealed: Literal[2, 3, 4]
## Exception handlers with `finally` branches (but no `except` branches)
A `finally` suite is *always* executed. As such, if we reach the `reveal_type`
call at the end of this example, we know that `x` *must* have been reassigned
to `2` during the `finally` suite. The type of `x` at the end of the example is
therefore `Literal[2]`:
A `finally` suite is *always* executed. As such, if we reach the `reveal_type` call at the end of
this example, we know that `x` *must* have been reassigned to `2` during the `finally` suite. The
type of `x` at the end of the example is therefore `Literal[2]`:
```py path=redef_in_finally.py
def could_raise_returns_str() -> str:
@@ -223,15 +209,13 @@ finally:
reveal_type(x) # revealed: Literal[2]
```
If `x` was *not* redefined in the `finally` suite, however, things are somewhat
more complicated. If we reach the final `reveal_type` call,
unlike the state when we're visiting the `finally` suite,
we know that the `try`-block suite ran to completion.
This means that there are fewer possible states at this point than there were
when we were inside the `finally` block.
If `x` was *not* redefined in the `finally` suite, however, things are somewhat more complicated. If
we reach the final `reveal_type` call, unlike the state when we're visiting the `finally` suite, we
know that the `try`-block suite ran to completion. This means that there are fewer possible states
at this point than there were when we were inside the `finally` block.
(Our current model does *not* correctly infer the types *inside* `finally`
suites, however; this is still a TODO item for us.)
(Our current model does *not* correctly infer the types *inside* `finally` suites, however; this is
still a TODO item for us.)
```py path=no_redef_in_finally.py
def could_raise_returns_str() -> str:
@@ -252,18 +236,18 @@ reveal_type(x) # revealed: str
## Combining an `except` branch with a `finally` branch
As previously stated, we do not yet have accurate inference for types *inside*
`finally` suites. When we do, however, we will have to take account of the
following possibilities inside `finally` suites:
As previously stated, we do not yet have accurate inference for types *inside* `finally` suites.
When we do, however, we will have to take account of the following possibilities inside `finally`
suites:
- The `try` suite could have run to completion
- Or we could have jumped from halfway through the `try` suite to an `except`
suite, and the `except` suite ran to completion
- Or we could have jumped from halfway through the `try` suite straight to the
`finally` suite due to an unhandled exception
- Or we could have jumped from halfway through the `try` suite to an
`except` suite, only for an exception raised in the `except` suite to cause
us to jump to the `finally` suite before the `except` suite ran to completion
- Or we could have jumped from halfway through the `try` suite to an `except` suite, and the
`except` suite ran to completion
- Or we could have jumped from halfway through the `try` suite straight to the `finally` suite due
to an unhandled exception
- Or we could have jumped from halfway through the `try` suite to an `except` suite, only for an
exception raised in the `except` suite to cause us to jump to the `finally` suite before the
`except` suite ran to completion
```py path=redef_in_finally.py
def could_raise_returns_str() -> str:
@@ -296,12 +280,11 @@ finally:
reveal_type(x) # revealed: Literal[2]
```
Now for an example without a redefinition in the `finally` suite.
As before, there *should* be fewer possibilities after completion of the
`finally` suite than there were during the `finally` suite itself.
(In some control-flow possibilities, some exceptions were merely *suspended*
during the `finally` suite; these lead to the scope's termination following the
conclusion of the `finally` suite.)
Now for an example without a redefinition in the `finally` suite. As before, there *should* be fewer
possibilities after completion of the `finally` suite than there were during the `finally` suite
itself. (In some control-flow possibilities, some exceptions were merely *suspended* during the
`finally` suite; these lead to the scope's termination following the conclusion of the `finally`
suite.)
```py path=no_redef_in_finally.py
def could_raise_returns_str() -> str:
@@ -377,9 +360,9 @@ reveal_type(x) # revealed: str | bool | float
## Combining `except`, `else` and `finally` branches
If the exception handler has an `else` branch, we must also take into account
the possibility that control flow could have jumped to the `finally` suite from
partway through the `else` suite due to an exception raised *there*.
If the exception handler has an `else` branch, we must also take into account the possibility that
control flow could have jumped to the `finally` suite from partway through the `else` suite due to
an exception raised *there*.
```py path=single_except_branch.py
def could_raise_returns_str() -> str:
@@ -479,15 +462,13 @@ reveal_type(x) # revealed: bool | float | slice
## Nested `try`/`except` blocks
It would take advanced analysis, which we are not yet capable of, to be able
to determine that an exception handler always suppresses all exceptions. This
is partly because it is possible for statements in `except`, `else` and
`finally` suites to raise exceptions as well as statements in `try` suites.
This means that if an exception handler is nested inside the `try` statement of
an enclosing exception handler, it should (at least for now) be treated the
same as any other node: as a suite containing statements that could possibly
raise exceptions, which would lead to control flow jumping out of that suite
prior to the suite running to completion.
It would take advanced analysis, which we are not yet capable of, to be able to determine that an
exception handler always suppresses all exceptions. This is partly because it is possible for
statements in `except`, `else` and `finally` suites to raise exceptions as well as statements in
`try` suites. This means that if an exception handler is nested inside the `try` statement of an
enclosing exception handler, it should (at least for now) be treated the same as any other node: as
a suite containing statements that could possibly raise exceptions, which would lead to control flow
jumping out of that suite prior to the suite running to completion.
```py
def could_raise_returns_str() -> str:
@@ -580,8 +561,8 @@ reveal_type(x) # revealed: bytearray | Bar
## Nested scopes inside `try` blocks
Shadowing a variable in an inner scope has no effect on type inference of the
variable by that name in the outer scope:
Shadowing a variable in an inner scope has no effect on type inference of the variable by that name
in the outer scope:
```py
def could_raise_returns_str() -> str:

View File

@@ -0,0 +1,13 @@
# Exception Handling
## Invalid syntax
```py
from typing_extensions import reveal_type
try:
print
except as e: # error: [invalid-syntax]
reveal_type(e) # revealed: Unknown
```

View File

@@ -0,0 +1,24 @@
# If expression
## Union
```py
def bool_instance() -> bool:
return True
reveal_type(1 if bool_instance() else 2) # revealed: Literal[1, 2]
```
## Statically known branches
```py
reveal_type(1 if True else 2) # revealed: Literal[1]
reveal_type(1 if "not empty" else 2) # revealed: Literal[1]
reveal_type(1 if (1,) else 2) # revealed: Literal[1]
reveal_type(1 if 1 else 2) # revealed: Literal[1]
reveal_type(1 if False else 2) # revealed: Literal[2]
reveal_type(1 if None else 2) # revealed: Literal[2]
reveal_type(1 if "" else 2) # revealed: Literal[2]
reveal_type(1 if 0 else 2) # revealed: Literal[2]
```

View File

@@ -16,10 +16,11 @@ class MyBox[T]:
def __init__(self, data: T):
self.data = data
# TODO not error (should be subscriptable)
box: MyBox[int] = MyBox(5) # error: [non-subscriptable]
# TODO error differently (str and int don't unify)
wrong_innards: MyBox[int] = MyBox("five") # error: [non-subscriptable]
box: MyBox[int] = MyBox(5)
# TODO should emit a diagnostic here (str is not assignable to int)
wrong_innards: MyBox[int] = MyBox("five")
# TODO reveal int
reveal_type(box.data) # revealed: @Todo
@@ -52,7 +53,8 @@ reveal_type(secure_box.data) # revealed: @Todo
## Cyclical class definition
In type stubs, classes can reference themselves in their base class definitions. For example, in `typeshed`, we have `class str(Sequence[str]): ...`.
In type stubs, classes can reference themselves in their base class definitions. For example, in
`typeshed`, we have `class str(Sequence[str]): ...`.
This should hold true even with generics at play.

View File

@@ -12,11 +12,10 @@ if flag:
x = y # error: [possibly-unresolved-reference]
# revealed: Unbound | Literal[3]
# error: [possibly-unresolved-reference]
# revealed: Literal[3]
reveal_type(x)
# revealed: Unbound | Literal[3]
# revealed: Literal[3]
# error: [possibly-unresolved-reference]
reveal_type(y)
```
@@ -40,11 +39,10 @@ if flag:
y: int = 3
x = y # error: [possibly-unresolved-reference]
# revealed: Unbound | Literal[3]
# error: [possibly-unresolved-reference]
# revealed: Literal[3]
reveal_type(x)
# revealed: Unbound | Literal[3]
# revealed: Literal[3]
# error: [possibly-unresolved-reference]
reveal_type(y)
```
@@ -58,6 +56,24 @@ reveal_type(x) # revealed: Literal[3]
reveal_type(y) # revealed: int
```
## Maybe undeclared
Importing a possibly undeclared name still gives us its declared type:
```py path=maybe_undeclared.py
def bool_instance() -> bool:
return True
if bool_instance():
x: int
```
```py
from maybe_undeclared import x
reveal_type(x) # revealed: int
```
## Reimport
```py path=c.py

View File

@@ -0,0 +1,91 @@
# Literal
<https://typing.readthedocs.io/en/latest/spec/literal.html#literals>
## Parameterization
```py
from typing import Literal
from enum import Enum
mode: Literal["w", "r"]
mode2: Literal["w"] | Literal["r"]
union_var: Literal[Literal[Literal[1, 2, 3], "foo"], 5, None]
a1: Literal[26]
a2: Literal[0x1A]
a3: Literal[-4]
a4: Literal["hello world"]
a5: Literal[b"hello world"]
a6: Literal[True]
a7: Literal[None]
a8: Literal[Literal[1]]
a9: Literal[Literal["w"], Literal["r"], Literal[Literal["w+"]]]
class Color(Enum):
RED = 0
GREEN = 1
BLUE = 2
b1: Literal[Color.RED]
def f():
reveal_type(mode) # revealed: Literal["w", "r"]
reveal_type(mode2) # revealed: Literal["w", "r"]
# TODO: should be revealed: Literal[1, 2, 3, "foo", 5] | None
reveal_type(union_var) # revealed: Literal[1, 2, 3, 5] | Literal["foo"] | None
reveal_type(a1) # revealed: Literal[26]
reveal_type(a2) # revealed: Literal[26]
reveal_type(a3) # revealed: Literal[-4]
reveal_type(a4) # revealed: Literal["hello world"]
reveal_type(a5) # revealed: Literal[b"hello world"]
reveal_type(a6) # revealed: Literal[True]
reveal_type(a7) # revealed: None
reveal_type(a8) # revealed: Literal[1]
reveal_type(a9) # revealed: Literal["w", "r", "w+"]
# TODO: This should be Color.RED
reveal_type(b1) # revealed: Literal[0]
# error: [invalid-literal-parameter]
invalid1: Literal[3 + 4]
# error: [invalid-literal-parameter]
invalid2: Literal[4 + 3j]
# error: [invalid-literal-parameter]
invalid3: Literal[(3, 4)]
invalid4: Literal[
1 + 2, # error: [invalid-literal-parameter]
"foo",
hello, # error: [invalid-literal-parameter]
(1, 2, 3), # error: [invalid-literal-parameter]
]
```
## Detecting Literal outside typing and typing_extensions
Only Literal that is defined in typing and typing_extension modules is detected as the special
Literal.
```pyi path=other.pyi
from typing import _SpecialForm
Literal: _SpecialForm
```
```py
from other import Literal
a1: Literal[26]
def f():
reveal_type(a1) # revealed: @Todo
```
## Detecting typing_extensions.Literal
```py
from typing_extensions import Literal
a1: Literal[26]
def f():
reveal_type(a1) # revealed: Literal[26]
```

View File

@@ -17,8 +17,8 @@ async def foo():
async for x in Iterator():
pass
# TODO: should reveal `Unbound | Unknown` because `__aiter__` is not defined
# revealed: Unbound | @Todo
# TODO: should reveal `Unknown` because `__aiter__` is not defined
# revealed: @Todo
# error: [possibly-unresolved-reference]
reveal_type(x)
```
@@ -40,6 +40,6 @@ async def foo():
pass
# error: [possibly-unresolved-reference]
# revealed: Unbound | @Todo
# revealed: @Todo
reveal_type(x)
```

View File

@@ -14,7 +14,7 @@ class IntIterable:
for x in IntIterable():
pass
# revealed: Unbound | int
# revealed: int
# error: [possibly-unresolved-reference]
reveal_type(x)
```
@@ -87,7 +87,7 @@ class OldStyleIterable:
for x in OldStyleIterable():
pass
# revealed: Unbound | int
# revealed: int
# error: [possibly-unresolved-reference]
reveal_type(x)
```
@@ -98,7 +98,7 @@ reveal_type(x)
for x in (1, "a", b"foo"):
pass
# revealed: Unbound | Literal[1] | Literal["a"] | Literal[b"foo"]
# revealed: Literal[1] | Literal["a"] | Literal[b"foo"]
# error: [possibly-unresolved-reference]
reveal_type(x)
```
@@ -120,7 +120,7 @@ class NotIterable:
for x in NotIterable(): # error: "Object of type `NotIterable` is not iterable"
pass
# revealed: Unbound | Unknown
# revealed: Unknown
# error: [possibly-unresolved-reference]
reveal_type(x)
```
@@ -144,3 +144,140 @@ class NotIterable:
for x in NotIterable(): # error: "Object of type `NotIterable` is not iterable"
pass
```
## Union type as iterable
```py
class TestIter:
def __next__(self) -> int:
return 42
class Test:
def __iter__(self) -> TestIter:
return TestIter()
class Test2:
def __iter__(self) -> TestIter:
return TestIter()
def bool_instance() -> bool:
return True
flag = bool_instance()
for x in Test() if flag else Test2():
reveal_type(x) # revealed: int
```
## Union type as iterator
```py
class TestIter:
def __next__(self) -> int:
return 42
class TestIter2:
def __next__(self) -> int:
return 42
class Test:
def __iter__(self) -> TestIter | TestIter2:
return TestIter()
for x in Test():
reveal_type(x) # revealed: int
```
## Union type as iterable and union type as iterator
```py
class TestIter:
def __next__(self) -> int | Exception:
return 42
class TestIter2:
def __next__(self) -> str | tuple[int, int]:
return "42"
class TestIter3:
def __next__(self) -> bytes:
return b"42"
class TestIter4:
def __next__(self) -> memoryview:
return memoryview(b"42")
class Test:
def __iter__(self) -> TestIter | TestIter2:
return TestIter()
class Test2:
def __iter__(self) -> TestIter3 | TestIter4:
return TestIter3()
def bool_instance() -> bool:
return True
flag = bool_instance()
for x in Test() if flag else Test2():
reveal_type(x) # revealed: int | Exception | str | tuple[int, int] | bytes | memoryview
```
## Union type as iterable where one union element has no `__iter__` method
```py
class TestIter:
def __next__(self) -> int:
return 42
class Test:
def __iter__(self) -> TestIter:
return TestIter()
def coinflip() -> bool:
return True
# TODO: we should emit a diagnostic here (it might not be iterable)
for x in Test() if coinflip() else 42:
reveal_type(x) # revealed: int
```
## Union type as iterable where one union element has invalid `__iter__` method
```py
class TestIter:
def __next__(self) -> int:
return 42
class Test:
def __iter__(self) -> TestIter:
return TestIter()
class Test2:
def __iter__(self) -> int:
return 42
def coinflip() -> bool:
return True
# error: "Object of type `Test | Test2` is not iterable"
for x in Test() if coinflip() else Test2():
reveal_type(x) # revealed: Unknown
```
## Union type as iterator where one union element has no `__next__` method
```py
class TestIter:
def __next__(self) -> int:
return 42
class Test:
def __iter__(self) -> TestIter | int:
return TestIter()
# error: [not-iterable] "Object of type `Test` is not iterable"
for x in Test():
reveal_type(x) # revealed: Unknown
```

View File

@@ -0,0 +1,196 @@
## Default
```py
class M(type): ...
reveal_type(M.__class__) # revealed: Literal[type]
```
## `object`
```py
reveal_type(object.__class__) # revealed: Literal[type]
```
## `type`
```py
reveal_type(type.__class__) # revealed: Literal[type]
```
## Basic
```py
class M(type): ...
class B(metaclass=M): ...
reveal_type(B.__class__) # revealed: Literal[M]
```
## Invalid metaclass
A class which doesn't inherit `type` (and/or doesn't implement a custom `__new__` accepting the same
arguments as `type.__new__`) isn't a valid metaclass.
```py
class M: ...
class A(metaclass=M): ...
# TODO: emit a diagnostic for the invalid metaclass
reveal_type(A.__class__) # revealed: Literal[M]
```
## Linear inheritance
If a class is a subclass of a class with a custom metaclass, then the subclass will also have that
metaclass.
```py
class M(type): ...
class A(metaclass=M): ...
class B(A): ...
reveal_type(B.__class__) # revealed: Literal[M]
```
## Conflict (1)
The metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its
bases. ("Strict subclass" is a synonym for "proper subclass"; a non-strict subclass can be a
subclass or the class itself.)
```py
class M1(type): ...
class M2(type): ...
class A(metaclass=M1): ...
class B(metaclass=M2): ...
# error: [conflicting-metaclass] "The metaclass of a derived class (`C`) must be a subclass of the metaclasses of all its bases, but `M1` and `M2` have no subclass relationship"
class C(A, B): ...
reveal_type(C.__class__) # revealed: Unknown
```
## Conflict (2)
The metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its
bases. ("Strict subclass" is a synonym for "proper subclass"; a non-strict subclass can be a
subclass or the class itself.)
```py
class M1(type): ...
class M2(type): ...
class A(metaclass=M1): ...
# error: [conflicting-metaclass] "The metaclass of a derived class (`B`) must be a subclass of the metaclasses of all its bases, but `M2` and `M1` have no subclass relationship"
class B(A, metaclass=M2): ...
reveal_type(B.__class__) # revealed: Unknown
```
## Common metaclass
A class has two explicit bases, both of which have the same metaclass.
```py
class M(type): ...
class A(metaclass=M): ...
class B(metaclass=M): ...
class C(A, B): ...
reveal_type(C.__class__) # revealed: Literal[M]
```
## Metaclass metaclass
A class has an explicit base with a custom metaclass. That metaclass itself has a custom metaclass.
```py
class M1(type): ...
class M2(type, metaclass=M1): ...
class M3(M2): ...
class A(metaclass=M3): ...
class B(A): ...
reveal_type(A.__class__) # revealed: Literal[M3]
```
## Diamond inheritance
```py
class M(type): ...
class M1(M): ...
class M2(M): ...
class M12(M1, M2): ...
class A(metaclass=M1): ...
class B(metaclass=M2): ...
class C(metaclass=M12): ...
# error: [conflicting-metaclass] "The metaclass of a derived class (`D`) must be a subclass of the metaclasses of all its bases, but `M1` and `M2` have no subclass relationship"
class D(A, B, C): ...
reveal_type(D.__class__) # revealed: Unknown
```
## Unknown
```py
from nonexistent_module import UnknownClass # error: [unresolved-import]
class C(UnknownClass): ...
# TODO: should be `type[type] & Unknown`
reveal_type(C.__class__) # revealed: Literal[type]
class M(type): ...
class A(metaclass=M): ...
class B(A, UnknownClass): ...
# TODO: should be `type[M] & Unknown`
reveal_type(B.__class__) # revealed: Literal[M]
```
## Duplicate
```py
class M(type): ...
class A(metaclass=M): ...
class B(A, A): ... # error: [duplicate-base] "Duplicate base class `A`"
reveal_type(B.__class__) # revealed: Literal[M]
```
## Non-class
When a class has an explicit `metaclass` that is not a class, but is a callable that accepts
`type.__new__` arguments, we should return the meta type of its return type.
```py
def f(*args, **kwargs) -> int: ...
class A(metaclass=f): ...
# TODO should be `type[int]`
reveal_type(A.__class__) # revealed: @Todo
```
## Cyclic
Retrieving the metaclass of a cyclically defined class should not cause an infinite loop.
```py path=a.pyi
class A(B): ... # error: [cyclic-class-def]
class B(C): ... # error: [cyclic-class-def]
class C(A): ... # error: [cyclic-class-def]
reveal_type(A.__class__) # revealed: Unknown
```
## PEP 695 generic
```py
class M(type): ...
class A[T: str](metaclass=M): ...
reveal_type(A.__class__) # revealed: Literal[M]
```

View File

@@ -0,0 +1,409 @@
# Method Resolution Order tests
Tests that assert that we can infer the correct type for a class's `__mro__` attribute.
This attribute is rarely accessed directly at runtime. However, it's extremely important for *us* to
know the precise possible values of a class's Method Resolution Order, or we won't be able to infer
the correct type of attributes accessed from instances.
For documentation on method resolution orders, see:
- <https://docs.python.org/3/glossary.html#term-method-resolution-order>
- <https://docs.python.org/3/howto/mro.html#python-2-3-mro>
## No bases
```py
class C: ...
reveal_type(C.__mro__) # revealed: tuple[Literal[C], Literal[object]]
```
## The special case: `object` itself
```py
reveal_type(object.__mro__) # revealed: tuple[Literal[object]]
```
## Explicit inheritance from `object`
```py
class C(object): ...
reveal_type(C.__mro__) # revealed: tuple[Literal[C], Literal[object]]
```
## Explicit inheritance from non-`object` single base
```py
class A: ...
class B(A): ...
reveal_type(B.__mro__) # revealed: tuple[Literal[B], Literal[A], Literal[object]]
```
## Linearization of multiple bases
```py
class A: ...
class B: ...
class C(A, B): ...
reveal_type(C.__mro__) # revealed: tuple[Literal[C], Literal[A], Literal[B], Literal[object]]
```
## Complex diamond inheritance (1)
This is "ex_2" from <https://docs.python.org/3/howto/mro.html#the-end>
```py
class O: ...
class X(O): ...
class Y(O): ...
class A(X, Y): ...
class B(Y, X): ...
reveal_type(A.__mro__) # revealed: tuple[Literal[A], Literal[X], Literal[Y], Literal[O], Literal[object]]
reveal_type(B.__mro__) # revealed: tuple[Literal[B], Literal[Y], Literal[X], Literal[O], Literal[object]]
```
## Complex diamond inheritance (2)
This is "ex_5" from <https://docs.python.org/3/howto/mro.html#the-end>
```py
class O: ...
class F(O): ...
class E(O): ...
class D(O): ...
class C(D, F): ...
class B(D, E): ...
class A(B, C): ...
# revealed: tuple[Literal[C], Literal[D], Literal[F], Literal[O], Literal[object]]
reveal_type(C.__mro__)
# revealed: tuple[Literal[B], Literal[D], Literal[E], Literal[O], Literal[object]]
reveal_type(B.__mro__)
# revealed: tuple[Literal[A], Literal[B], Literal[C], Literal[D], Literal[E], Literal[F], Literal[O], Literal[object]]
reveal_type(A.__mro__)
```
## Complex diamond inheritance (3)
This is "ex_6" from <https://docs.python.org/3/howto/mro.html#the-end>
```py
class O: ...
class F(O): ...
class E(O): ...
class D(O): ...
class C(D, F): ...
class B(E, D): ...
class A(B, C): ...
# revealed: tuple[Literal[C], Literal[D], Literal[F], Literal[O], Literal[object]]
reveal_type(C.__mro__)
# revealed: tuple[Literal[B], Literal[E], Literal[D], Literal[O], Literal[object]]
reveal_type(B.__mro__)
# revealed: tuple[Literal[A], Literal[B], Literal[E], Literal[C], Literal[D], Literal[F], Literal[O], Literal[object]]
reveal_type(A.__mro__)
```
## Complex diamond inheritance (4)
This is "ex_9" from <https://docs.python.org/3/howto/mro.html#the-end>
```py
class O: ...
class A(O): ...
class B(O): ...
class C(O): ...
class D(O): ...
class E(O): ...
class K1(A, B, C): ...
class K2(D, B, E): ...
class K3(D, A): ...
class Z(K1, K2, K3): ...
# revealed: tuple[Literal[K1], Literal[A], Literal[B], Literal[C], Literal[O], Literal[object]]
reveal_type(K1.__mro__)
# revealed: tuple[Literal[K2], Literal[D], Literal[B], Literal[E], Literal[O], Literal[object]]
reveal_type(K2.__mro__)
# revealed: tuple[Literal[K3], Literal[D], Literal[A], Literal[O], Literal[object]]
reveal_type(K3.__mro__)
# revealed: tuple[Literal[Z], Literal[K1], Literal[K2], Literal[K3], Literal[D], Literal[A], Literal[B], Literal[C], Literal[E], Literal[O], Literal[object]]
reveal_type(Z.__mro__)
```
## Inheritance from `Unknown`
```py
from does_not_exist import DoesNotExist # error: [unresolved-import]
class A(DoesNotExist): ...
class B: ...
class C: ...
class D(A, B, C): ...
class E(B, C): ...
class F(E, A): ...
reveal_type(A.__mro__) # revealed: tuple[Literal[A], Unknown, Literal[object]]
reveal_type(D.__mro__) # revealed: tuple[Literal[D], Literal[A], Unknown, Literal[B], Literal[C], Literal[object]]
reveal_type(E.__mro__) # revealed: tuple[Literal[E], Literal[B], Literal[C], Literal[object]]
reveal_type(F.__mro__) # revealed: tuple[Literal[F], Literal[E], Literal[B], Literal[C], Literal[A], Unknown, Literal[object]]
```
## `__bases__` lists that cause errors at runtime
If the class's `__bases__` cause an exception to be raised at runtime and therefore the class
creation to fail, we infer the class's `__mro__` as being `[<class>, Unknown, object]`:
```py
# error: [inconsistent-mro] "Cannot create a consistent method resolution order (MRO) for class `Foo` with bases list `[<class 'object'>, <class 'int'>]`"
class Foo(object, int): ...
reveal_type(Foo.__mro__) # revealed: tuple[Literal[Foo], Unknown, Literal[object]]
class Bar(Foo): ...
reveal_type(Bar.__mro__) # revealed: tuple[Literal[Bar], Literal[Foo], Unknown, Literal[object]]
# This is the `TypeError` at the bottom of "ex_2"
# in the examples at <https://docs.python.org/3/howto/mro.html#the-end>
class O: ...
class X(O): ...
class Y(O): ...
class A(X, Y): ...
class B(Y, X): ...
reveal_type(A.__mro__) # revealed: tuple[Literal[A], Literal[X], Literal[Y], Literal[O], Literal[object]]
reveal_type(B.__mro__) # revealed: tuple[Literal[B], Literal[Y], Literal[X], Literal[O], Literal[object]]
# error: [inconsistent-mro] "Cannot create a consistent method resolution order (MRO) for class `Z` with bases list `[<class 'A'>, <class 'B'>]`"
class Z(A, B): ...
reveal_type(Z.__mro__) # revealed: tuple[Literal[Z], Unknown, Literal[object]]
class AA(Z): ...
reveal_type(AA.__mro__) # revealed: tuple[Literal[AA], Literal[Z], Unknown, Literal[object]]
```
## `__bases__` includes a `Union`
We don't support union types in a class's bases; a base must resolve to a single `ClassLiteralType`.
If we find a union type in a class's bases, we infer the class's `__mro__` as being
`[<class>, Unknown, object]`, the same as for MROs that cause errors at runtime.
```py
def returns_bool() -> bool:
return True
class A: ...
class B: ...
if returns_bool():
x = A
else:
x = B
reveal_type(x) # revealed: Literal[A, B]
# error: 11 [invalid-base] "Invalid class base with type `Literal[A, B]` (all bases must be a class, `Any`, `Unknown` or `Todo`)"
class Foo(x): ...
reveal_type(Foo.__mro__) # revealed: tuple[Literal[Foo], Unknown, Literal[object]]
```
## `__bases__` includes multiple `Union`s
```py
def returns_bool() -> bool:
return True
class A: ...
class B: ...
class C: ...
class D: ...
if returns_bool():
x = A
else:
x = B
if returns_bool():
y = C
else:
y = D
reveal_type(x) # revealed: Literal[A, B]
reveal_type(y) # revealed: Literal[C, D]
# error: 11 [invalid-base] "Invalid class base with type `Literal[A, B]` (all bases must be a class, `Any`, `Unknown` or `Todo`)"
# error: 14 [invalid-base] "Invalid class base with type `Literal[C, D]` (all bases must be a class, `Any`, `Unknown` or `Todo`)"
class Foo(x, y): ...
reveal_type(Foo.__mro__) # revealed: tuple[Literal[Foo], Unknown, Literal[object]]
```
## `__bases__` lists that cause errors... now with `Union`s
```py
def returns_bool() -> bool:
return True
class O: ...
class X(O): ...
class Y(O): ...
if bool():
foo = Y
else:
foo = object
# error: 21 [invalid-base] "Invalid class base with type `Literal[Y, object]` (all bases must be a class, `Any`, `Unknown` or `Todo`)"
class PossibleError(foo, X): ...
reveal_type(PossibleError.__mro__) # revealed: tuple[Literal[PossibleError], Unknown, Literal[object]]
class A(X, Y): ...
reveal_type(A.__mro__) # revealed: tuple[Literal[A], Literal[X], Literal[Y], Literal[O], Literal[object]]
if returns_bool():
class B(X, Y): ...
else:
class B(Y, X): ...
# revealed: tuple[Literal[B], Literal[X], Literal[Y], Literal[O], Literal[object]] | tuple[Literal[B], Literal[Y], Literal[X], Literal[O], Literal[object]]
reveal_type(B.__mro__)
# error: 12 [invalid-base] "Invalid class base with type `Literal[B, B]` (all bases must be a class, `Any`, `Unknown` or `Todo`)"
class Z(A, B): ...
reveal_type(Z.__mro__) # revealed: tuple[Literal[Z], Unknown, Literal[object]]
```
## `__bases__` lists with duplicate bases
```py
class Foo(str, str): ... # error: 16 [duplicate-base] "Duplicate base class `str`"
reveal_type(Foo.__mro__) # revealed: tuple[Literal[Foo], Unknown, Literal[object]]
class Spam: ...
class Eggs: ...
class Ham(
Spam,
Eggs,
Spam, # error: [duplicate-base] "Duplicate base class `Spam`"
Eggs, # error: [duplicate-base] "Duplicate base class `Eggs`"
): ...
reveal_type(Ham.__mro__) # revealed: tuple[Literal[Ham], Unknown, Literal[object]]
class Mushrooms: ...
class Omelette(Spam, Eggs, Mushrooms, Mushrooms): ... # error: [duplicate-base]
reveal_type(Omelette.__mro__) # revealed: tuple[Literal[Omelette], Unknown, Literal[object]]
```
## `__bases__` lists with duplicate `Unknown` bases
```py
# error: [unresolved-import]
# error: [unresolved-import]
from does_not_exist import unknown_object_1, unknown_object_2
reveal_type(unknown_object_1) # revealed: Unknown
reveal_type(unknown_object_2) # revealed: Unknown
# We *should* emit an error here to warn the user that we have no idea
# what the MRO of this class should really be.
# However, we don't complain about "duplicate base classes" here,
# even though two classes are both inferred as being `Unknown`.
#
# (TODO: should we revisit this? Does it violate the gradual guarantee?
# Should we just silently infer `[Foo, Unknown, object]` as the MRO here
# without emitting any error at all? Not sure...)
#
# error: [inconsistent-mro] "Cannot create a consistent method resolution order (MRO) for class `Foo` with bases list `[Unknown, Unknown]`"
class Foo(unknown_object_1, unknown_object_2): ...
reveal_type(Foo.__mro__) # revealed: tuple[Literal[Foo], Unknown, Literal[object]]
```
## Unrelated objects inferred as `Any`/`Unknown` do not have special `__mro__` attributes
```py
from does_not_exist import unknown_object # error: [unresolved-import]
reveal_type(unknown_object) # revealed: Unknown
reveal_type(unknown_object.__mro__) # revealed: Unknown
```
## Classes that inherit from themselves
These are invalid, but we need to be able to handle them gracefully without panicking.
```py path=a.pyi
class Foo(Foo): ... # error: [cyclic-class-def]
reveal_type(Foo) # revealed: Literal[Foo]
reveal_type(Foo.__mro__) # revealed: tuple[Literal[Foo], Unknown, Literal[object]]
class Bar: ...
class Baz: ...
class Boz(Bar, Baz, Boz): ... # error: [cyclic-class-def]
reveal_type(Boz) # revealed: Literal[Boz]
reveal_type(Boz.__mro__) # revealed: tuple[Literal[Boz], Unknown, Literal[object]]
```
## Classes with indirect cycles in their MROs
These are similarly unlikely, but we still shouldn't crash:
```py path=a.pyi
class Foo(Bar): ... # error: [cyclic-class-def]
class Bar(Baz): ... # error: [cyclic-class-def]
class Baz(Foo): ... # error: [cyclic-class-def]
reveal_type(Foo.__mro__) # revealed: tuple[Literal[Foo], Unknown, Literal[object]]
reveal_type(Bar.__mro__) # revealed: tuple[Literal[Bar], Unknown, Literal[object]]
reveal_type(Baz.__mro__) # revealed: tuple[Literal[Baz], Unknown, Literal[object]]
```
## Classes with cycles in their MROs, and multiple inheritance
```py path=a.pyi
class Spam: ...
class Foo(Bar): ... # error: [cyclic-class-def]
class Bar(Baz): ... # error: [cyclic-class-def]
class Baz(Foo, Spam): ... # error: [cyclic-class-def]
reveal_type(Foo.__mro__) # revealed: tuple[Literal[Foo], Unknown, Literal[object]]
reveal_type(Bar.__mro__) # revealed: tuple[Literal[Bar], Unknown, Literal[object]]
reveal_type(Baz.__mro__) # revealed: tuple[Literal[Baz], Unknown, Literal[object]]
```
## Classes with cycles in their MRO, and a sub-graph
```py path=a.pyi
class FooCycle(BarCycle): ... # error: [cyclic-class-def]
class Foo: ...
class BarCycle(FooCycle): ... # error: [cyclic-class-def]
class Bar(Foo): ...
# TODO: can we avoid emitting the errors for these?
# The classes have cyclic superclasses,
# but are not themselves cyclic...
class Baz(Bar, BarCycle): ... # error: [cyclic-class-def]
class Spam(Baz): ... # error: [cyclic-class-def]
reveal_type(FooCycle.__mro__) # revealed: tuple[Literal[FooCycle], Unknown, Literal[object]]
reveal_type(BarCycle.__mro__) # revealed: tuple[Literal[BarCycle], Unknown, Literal[object]]
reveal_type(Baz.__mro__) # revealed: tuple[Literal[Baz], Unknown, Literal[object]]
reveal_type(Spam.__mro__) # revealed: tuple[Literal[Spam], Unknown, Literal[object]]
```

View File

@@ -0,0 +1,93 @@
# Narrowing in boolean expressions
In `or` expressions, the right-hand side is evaluated only if the left-hand side is **falsy**. So
when the right-hand side is evaluated, we know the left side has failed.
Similarly, in `and` expressions, the right-hand side is evaluated only if the left-hand side is
**truthy**. So when the right-hand side is evaluated, we know the left side has succeeded.
## Narrowing in `or`
```py
def bool_instance() -> bool:
return True
class A: ...
x: A | None = A() if bool_instance() else None
isinstance(x, A) or reveal_type(x) # revealed: None
x is None or reveal_type(x) # revealed: A
reveal_type(x) # revealed: A | None
```
## Narrowing in `and`
```py
def bool_instance() -> bool:
return True
class A: ...
x: A | None = A() if bool_instance() else None
isinstance(x, A) and reveal_type(x) # revealed: A
x is None and reveal_type(x) # revealed: None
reveal_type(x) # revealed: A | None
```
## Multiple `and` arms
```py
def bool_instance() -> bool:
return True
class A: ...
x: A | None = A() if bool_instance() else None
bool_instance() and isinstance(x, A) and reveal_type(x) # revealed: A
isinstance(x, A) and bool_instance() and reveal_type(x) # revealed: A
reveal_type(x) and isinstance(x, A) and bool_instance() # revealed: A | None
```
## Multiple `or` arms
```py
def bool_instance() -> bool:
return True
class A: ...
x: A | None = A() if bool_instance() else None
bool_instance() or isinstance(x, A) or reveal_type(x) # revealed: None
isinstance(x, A) or bool_instance() or reveal_type(x) # revealed: None
reveal_type(x) or isinstance(x, A) or bool_instance() # revealed: A | None
```
## Multiple predicates
```py
def bool_instance() -> bool:
return True
class A: ...
x: A | None | Literal[1] = A() if bool_instance() else None if bool_instance() else 1
x is None or isinstance(x, A) or reveal_type(x) # revealed: Literal[1]
```
## Mix of `and` and `or`
```py
def bool_instance() -> bool:
return True
class A: ...
x: A | None | Literal[1] = A() if bool_instance() else None if bool_instance() else 1
isinstance(x, A) or x is not None and reveal_type(x) # revealed: Literal[1]
```

View File

@@ -0,0 +1,282 @@
# Narrowing for conditionals with boolean expressions
## Narrowing in `and` conditional
```py
class A: ...
class B: ...
def instance() -> A | B:
return A()
x = instance()
if isinstance(x, A) and isinstance(x, B):
reveal_type(x) # revealed: A & B
else:
reveal_type(x) # revealed: B & ~A | A & ~B
```
## Arms might not add narrowing constraints
```py
class A: ...
class B: ...
def bool_instance() -> bool:
return True
def instance() -> A | B:
return A()
x = instance()
if isinstance(x, A) and bool_instance():
reveal_type(x) # revealed: A
else:
reveal_type(x) # revealed: A | B
if bool_instance() and isinstance(x, A):
reveal_type(x) # revealed: A
else:
reveal_type(x) # revealed: A | B
reveal_type(x) # revealed: A | B
```
## Statically known arms
```py
class A: ...
class B: ...
def instance() -> A | B:
return A()
x = instance()
if isinstance(x, A) and True:
reveal_type(x) # revealed: A
else:
reveal_type(x) # revealed: B & ~A
if True and isinstance(x, A):
reveal_type(x) # revealed: A
else:
reveal_type(x) # revealed: B & ~A
if False and isinstance(x, A):
# TODO: should emit an `unreachable code` diagnostic
reveal_type(x) # revealed: A
else:
reveal_type(x) # revealed: A | B
if False or isinstance(x, A):
reveal_type(x) # revealed: A
else:
reveal_type(x) # revealed: B & ~A
if True or isinstance(x, A):
reveal_type(x) # revealed: A | B
else:
# TODO: should emit an `unreachable code` diagnostic
reveal_type(x) # revealed: B & ~A
reveal_type(x) # revealed: A | B
```
## The type of multiple symbols can be narrowed down
```py
class A: ...
class B: ...
def instance() -> A | B:
return A()
x = instance()
y = instance()
if isinstance(x, A) and isinstance(y, B):
reveal_type(x) # revealed: A
reveal_type(y) # revealed: B
else:
# No narrowing: Only-one or both checks might have failed
reveal_type(x) # revealed: A | B
reveal_type(y) # revealed: A | B
reveal_type(x) # revealed: A | B
reveal_type(y) # revealed: A | B
```
## Narrowing in `or` conditional
```py
class A: ...
class B: ...
class C: ...
def instance() -> A | B | C:
return A()
x = instance()
if isinstance(x, A) or isinstance(x, B):
reveal_type(x) # revealed: A | B
else:
reveal_type(x) # revealed: C & ~A & ~B
```
## In `or`, all arms should add constraint in order to narrow
```py
class A: ...
class B: ...
class C: ...
def instance() -> A | B | C:
return A()
def bool_instance() -> bool:
return True
x = instance()
if isinstance(x, A) or isinstance(x, B) or bool_instance():
reveal_type(x) # revealed: A | B | C
else:
reveal_type(x) # revealed: C & ~A & ~B
```
## in `or`, all arms should narrow the same set of symbols
```py
class A: ...
class B: ...
class C: ...
def instance() -> A | B | C:
return A()
x = instance()
y = instance()
if isinstance(x, A) or isinstance(y, A):
# The predicate might be satisfied by the right side, so the type of `x` cant be narrowed down here.
reveal_type(x) # revealed: A | B | C
# The same for `y`
reveal_type(y) # revealed: A | B | C
else:
reveal_type(x) # revealed: B & ~A | C & ~A
reveal_type(y) # revealed: B & ~A | C & ~A
if (isinstance(x, A) and isinstance(y, A)) or (isinstance(x, B) and isinstance(y, B)):
# Here, types of `x` and `y` can be narrowd since all `or` arms constraint them.
reveal_type(x) # revealed: A | B
reveal_type(y) # revealed: A | B
else:
reveal_type(x) # revealed: A | B | C
reveal_type(y) # revealed: A | B | C
```
## mixing `and` and `not`
```py
class A: ...
class B: ...
class C: ...
def instance() -> A | B | C:
return A()
x = instance()
if isinstance(x, B) and not isinstance(x, C):
reveal_type(x) # revealed: B & ~C
else:
# ~(B & ~C) -> ~B | C -> (A & ~B) | (C & ~B) | C -> (A & ~B) | C
reveal_type(x) # revealed: A & ~B | C
```
## mixing `or` and `not`
```py
class A: ...
class B: ...
class C: ...
def instance() -> A | B | C:
return A()
x = instance()
if isinstance(x, B) or not isinstance(x, C):
reveal_type(x) # revealed: B | A & ~C
else:
reveal_type(x) # revealed: C & ~B
```
## `or` with nested `and`
```py
class A: ...
class B: ...
class C: ...
def instance() -> A | B | C:
return A()
x = instance()
if isinstance(x, A) or (isinstance(x, B) and not isinstance(x, C)):
reveal_type(x) # revealed: A | B & ~C
else:
# ~(A | (B & ~C)) -> ~A & ~(B & ~C) -> ~A & (~B | C) -> (~A & C) | (~A ~ B)
reveal_type(x) # revealed: C & ~A
```
## `and` with nested `or`
```py
class A: ...
class B: ...
class C: ...
def instance() -> A | B | C:
return A()
x = instance()
if isinstance(x, A) and (isinstance(x, B) or not isinstance(x, C)):
# A & (B | ~C) -> (A & B) | (A & ~C)
reveal_type(x) # revealed: A & B | A & ~C
else:
# ~((A & B) | (A & ~C)) ->
# ~(A & B) & ~(A & ~C) ->
# (~A | ~B) & (~A | C) ->
# [(~A | ~B) & ~A] | [(~A | ~B) & C] ->
# ~A | (~A & C) | (~B & C) ->
# ~A | (C & ~B) ->
# ~A | (C & ~B) The positive side of ~A is A | B | C ->
reveal_type(x) # revealed: B & ~A | C & ~A | C & ~B
```
## Boolean expression internal narrowing
```py
def optional_string() -> str | None:
return None
x = optional_string()
y = optional_string()
if x is None and y is not x:
reveal_type(y) # revealed: str
# Neither of the conditions alone is sufficient for narrowing y's type:
if x is None:
reveal_type(y) # revealed: str | None
if y is not x:
reveal_type(y) # revealed: str | None
```

View File

@@ -0,0 +1,57 @@
# Narrowing for conditionals with elif and else
## Positive contributions become negative in elif-else blocks
```py
def int_instance() -> int:
return 42
x = int_instance()
if x == 1:
# cannot narrow; could be a subclass of `int`
reveal_type(x) # revealed: int
elif x == 2:
reveal_type(x) # revealed: int & ~Literal[1]
elif x != 3:
reveal_type(x) # revealed: int & ~Literal[1] & ~Literal[2] & ~Literal[3]
```
## Positive contributions become negative in elif-else blocks, with simplification
```py
def bool_instance() -> bool:
return True
x = 1 if bool_instance() else 2 if bool_instance() else 3
if x == 1:
# TODO should be Literal[1]
reveal_type(x) # revealed: Literal[1, 2, 3]
elif x == 2:
# TODO should be Literal[2]
reveal_type(x) # revealed: Literal[2, 3]
else:
reveal_type(x) # revealed: Literal[3]
```
## Multiple negative contributions using elif, with simplification
```py
def bool_instance() -> bool:
return True
x = 1 if bool_instance() else 2 if bool_instance() else 3
if x != 1:
reveal_type(x) # revealed: Literal[2, 3]
elif x != 2:
# TODO should be `Literal[1]`
reveal_type(x) # revealed: Literal[1, 3]
elif x == 3:
# TODO should be Never
reveal_type(x) # revealed: Literal[1, 2, 3]
else:
# TODO should be Never
reveal_type(x) # revealed: Literal[1, 2]
```

View File

@@ -11,6 +11,8 @@ x = None if flag else 1
if x is None:
reveal_type(x) # revealed: None
else:
reveal_type(x) # revealed: Literal[1]
reveal_type(x) # revealed: None | Literal[1]
```
@@ -30,6 +32,8 @@ y = x if flag else None
if y is x:
reveal_type(y) # revealed: A
else:
reveal_type(y) # revealed: A | None
reveal_type(y) # revealed: A | None
```
@@ -50,4 +54,26 @@ reveal_type(y) # revealed: bool
if y is x is False: # Interpreted as `(y is x) and (x is False)`
reveal_type(x) # revealed: Literal[False]
reveal_type(y) # revealed: bool
else:
# The negation of the clause above is (y is not x) or (x is not False)
# So we can't narrow the type of x or y here, because each arm of the `or` could be true
reveal_type(x) # revealed: bool
reveal_type(y) # revealed: bool
```
## `is` in elif clause
```py
def bool_instance() -> bool:
return True
x = None if bool_instance() else (1 if bool_instance() else True)
reveal_type(x) # revealed: None | Literal[1] | Literal[True]
if x is None:
reveal_type(x) # revealed: None
elif x is True:
reveal_type(x) # revealed: Literal[True]
else:
reveal_type(x) # revealed: Literal[1]
```

View File

@@ -13,6 +13,8 @@ x = None if flag else 1
if x is not None:
reveal_type(x) # revealed: Literal[1]
else:
reveal_type(x) # revealed: None
reveal_type(x) # revealed: None | Literal[1]
```
@@ -29,13 +31,14 @@ reveal_type(x) # revealed: bool
if x is not False:
reveal_type(x) # revealed: Literal[True]
else:
reveal_type(x) # revealed: Literal[False]
```
## `is not` for non-singleton types
Non-singleton types should *not* narrow the type: two instances of a
non-singleton class may occupy different addresses in memory even if
they compare equal.
Non-singleton types should *not* narrow the type: two instances of a non-singleton class may occupy
different addresses in memory even if they compare equal.
```py
x = 345
@@ -43,6 +46,27 @@ y = 345
if x is not y:
reveal_type(x) # revealed: Literal[345]
else:
reveal_type(x) # revealed: Literal[345]
```
## `is not` for other types
```py
def bool_instance() -> bool:
return True
class A: ...
x = A()
y = x if bool_instance() else None
if y is not x:
reveal_type(y) # revealed: A | None
else:
reveal_type(y) # revealed: A
reveal_type(y) # revealed: A | None
```
## `is not` in chained comparisons
@@ -63,4 +87,10 @@ reveal_type(y) # revealed: bool
if y is not x is not False: # Interpreted as `(y is not x) and (x is not False)`
reveal_type(x) # revealed: Literal[True]
reveal_type(y) # revealed: bool
else:
# The negation of the clause above is (y is x) or (x is False)
# So we can't narrow the type of x or y here, because each arm of the `or` could be true
reveal_type(x) # revealed: bool
reveal_type(y) # revealed: bool
```

View File

@@ -3,7 +3,8 @@
## Multiple negative contributions
```py
def int_instance() -> int: ...
def int_instance() -> int:
return 42
x = int_instance()
@@ -27,3 +28,29 @@ if x != 1:
if x != 2:
reveal_type(x) # revealed: Literal[3]
```
## elif-else blocks
```py
def bool_instance() -> bool:
return True
x = 1 if bool_instance() else 2 if bool_instance() else 3
if x != 1:
reveal_type(x) # revealed: Literal[2, 3]
if x == 2:
# TODO should be `Literal[2]`
reveal_type(x) # revealed: Literal[2, 3]
elif x == 3:
reveal_type(x) # revealed: Literal[3]
else:
reveal_type(x) # revealed: Never
elif x != 2:
# TODO should be Literal[1]
reveal_type(x) # revealed: Literal[1, 3]
else:
# TODO should be Never
reveal_type(x) # revealed: Literal[1, 2, 3]
```

View File

@@ -0,0 +1,33 @@
# Narrowing for `not` conditionals
The `not` operator negates a constraint.
## `not is None`
```py
def bool_instance() -> bool:
return True
x = None if bool_instance() else 1
if not x is None:
reveal_type(x) # revealed: Literal[1]
else:
reveal_type(x) # revealed: None
reveal_type(x) # revealed: None | Literal[1]
```
## `not isinstance`
```py
def bool_instance() -> bool:
return True
x = 1 if bool_instance() else "a"
if not isinstance(x, (int)):
reveal_type(x) # revealed: Literal["a"]
else:
reveal_type(x) # revealed: Literal[1]
```

View File

@@ -11,6 +11,9 @@ x = None if flag else 1
if x != None:
reveal_type(x) # revealed: Literal[1]
else:
# TODO should be None
reveal_type(x) # revealed: None | Literal[1]
```
## `!=` for other singleton types
@@ -24,6 +27,9 @@ x = True if flag else False
if x != False:
reveal_type(x) # revealed: Literal[True]
else:
# TODO should be Literal[False]
reveal_type(x) # revealed: bool
```
## `x != y` where `y` is of literal type
@@ -54,6 +60,25 @@ C = A if flag else B
if C != A:
reveal_type(C) # revealed: Literal[B]
else:
# TODO should be Literal[A]
reveal_type(C) # revealed: Literal[A, B]
```
## `x != y` where `y` has multiple single-valued options
```py
def bool_instance() -> bool:
return True
x = 1 if bool_instance() else 2
y = 2 if bool_instance() else 3
if x != y:
reveal_type(x) # revealed: Literal[1, 2]
else:
# TODO should be Literal[2]
reveal_type(x) # revealed: Literal[1, 2]
```
## `!=` for non-single-valued types
@@ -74,3 +99,21 @@ y = int_instance()
if x != y:
reveal_type(x) # revealed: int | None
```
## Mix of single-valued and non-single-valued types
```py
def int_instance() -> int:
return 42
def bool_instance() -> bool:
return True
x = 1 if bool_instance() else 2
y = 2 if bool_instance() else int_instance()
if x != y:
reveal_type(x) # revealed: Literal[1, 2]
else:
reveal_type(x) # revealed: Literal[1, 2]
```

View File

@@ -26,9 +26,8 @@ if isinstance(x, (int, object)):
## `classinfo` is a tuple of types
Note: `isinstance(x, (int, str))` should not be confused with
`isinstance(x, tuple[(int, str)])`. The former is equivalent to
`isinstance(x, int | str)`:
Note: `isinstance(x, (int, str))` should not be confused with `isinstance(x, tuple[(int, str)])`.
The former is equivalent to `isinstance(x, int | str)`:
```py
def bool_instance() -> bool:
@@ -40,6 +39,8 @@ x = 1 if flag else "a"
if isinstance(x, (int, str)):
reveal_type(x) # revealed: Literal[1] | Literal["a"]
else:
reveal_type(x) # revealed: Never
if isinstance(x, (int, bytes)):
reveal_type(x) # revealed: Literal[1]
@@ -51,6 +52,8 @@ if isinstance(x, (bytes, str)):
# one of the possibilities:
if isinstance(x, (int, object)):
reveal_type(x) # revealed: Literal[1] | Literal["a"]
else:
reveal_type(x) # revealed: Never
y = 1 if flag1 else "a" if flag2 else b"b"
if isinstance(y, (int, str)):
@@ -75,6 +78,8 @@ x = 1 if flag else "a"
if isinstance(x, (bool, (bytes, int))):
reveal_type(x) # revealed: Literal[1]
else:
reveal_type(x) # revealed: Literal["a"]
```
## Class types
@@ -82,6 +87,7 @@ if isinstance(x, (bool, (bytes, int))):
```py
class A: ...
class B: ...
class C: ...
def get_object() -> object: ...
@@ -91,6 +97,16 @@ if isinstance(x, A):
reveal_type(x) # revealed: A
if isinstance(x, B):
reveal_type(x) # revealed: A & B
else:
reveal_type(x) # revealed: A & ~B
if isinstance(x, (A, B)):
reveal_type(x) # revealed: A | B
elif isinstance(x, (A, C)):
reveal_type(x) # revealed: C & ~A & ~B
else:
# TODO: Should be simplified to ~A & ~B & ~C
reveal_type(x) # revealed: object & ~A & ~B & ~C
```
## No narrowing for instances of `builtins.type`

View File

@@ -0,0 +1,136 @@
# Implicit globals from `types.ModuleType`
## Implicit `ModuleType` globals
All modules are instances of `types.ModuleType`. If a name can't be found in any local or global
scope, we look it up as an attribute on `types.ModuleType` in typeshed before deciding that the name
is unbound.
```py
reveal_type(__name__) # revealed: str
reveal_type(__file__) # revealed: str | None
reveal_type(__loader__) # revealed: LoaderProtocol | None
reveal_type(__package__) # revealed: str | None
reveal_type(__doc__) # revealed: str | None
# TODO: Should be `ModuleSpec | None`
# (needs support for `*` imports)
reveal_type(__spec__) # revealed: Unknown | None
# TODO: generics
reveal_type(__path__) # revealed: @Todo
class X:
reveal_type(__name__) # revealed: str
def foo():
reveal_type(__name__) # revealed: str
```
However, three attributes on `types.ModuleType` are not present as implicit module globals; these
are excluded:
```py path=unbound_dunders.py
# error: [unresolved-reference]
# revealed: Unknown
reveal_type(__getattr__)
# error: [unresolved-reference]
# revealed: Unknown
reveal_type(__dict__)
# error: [unresolved-reference]
# revealed: Unknown
reveal_type(__init__)
```
## Accessed as attributes
`ModuleType` attributes can also be accessed as attributes on module-literal types. The special
attributes `__dict__` and `__init__`, and all attributes on `builtins.object`, can also be accessed
as attributes on module-literal types, despite the fact that these are inaccessible as globals from
inside the module:
```py
import typing
reveal_type(typing.__name__) # revealed: str
reveal_type(typing.__init__) # revealed: Literal[__init__]
# These come from `builtins.object`, not `types.ModuleType`:
reveal_type(typing.__eq__) # revealed: Literal[__eq__]
reveal_type(typing.__class__) # revealed: Literal[type]
# TODO: needs support for attribute access on instances, properties and generics;
# should be `dict[str, Any]`
reveal_type(typing.__dict__) # revealed: @Todo
```
Typeshed includes a fake `__getattr__` method in the stub for `types.ModuleType` to help out with
dynamic imports; but we ignore that for module-literal types where we know exactly which module
we're dealing with:
```py path=__getattr__.py
import typing
reveal_type(typing.__getattr__) # revealed: Unknown
```
## `types.ModuleType.__dict__` takes precedence over global variable `__dict__`
It's impossible to override the `__dict__` attribute of `types.ModuleType` instances from inside the
module; we should prioritise the attribute in the `types.ModuleType` stub over a variable named
`__dict__` in the module's global namespace:
```py path=foo.py
__dict__ = "foo"
reveal_type(__dict__) # revealed: Literal["foo"]
```
```py path=bar.py
import foo
from foo import __dict__ as foo_dict
# TODO: needs support for attribute access on instances, properties, and generics;
# should be `dict[str, Any]` for both of these:
reveal_type(foo.__dict__) # revealed: @Todo
reveal_type(foo_dict) # revealed: @Todo
```
## Conditionally global or `ModuleType` attribute
Attributes overridden in the module namespace take priority. If a builtin name is conditionally
defined as a global, however, a name lookup should union the `ModuleType` type with the
conditionally defined type:
```py
__file__ = 42
def returns_bool() -> bool:
return True
if returns_bool():
__name__ = 1
reveal_type(__file__) # revealed: Literal[42]
reveal_type(__name__) # revealed: Literal[1] | str
```
## Conditionally global or `ModuleType` attribute, with annotation
The same is true if the name is annotated:
```py
__file__: int = 42
def returns_bool() -> bool:
return True
if returns_bool():
__name__: int = 1
reveal_type(__file__) # revealed: Literal[42]
reveal_type(__name__) # revealed: Literal[1] | str
```

View File

@@ -2,7 +2,8 @@
## Parameter
Parameter `x` of type `str` is shadowed and reassigned with a new `int` value inside the function. No diagnostics should be generated.
Parameter `x` of type `str` is shadowed and reassigned with a new `int` value inside the function.
No diagnostics should be generated.
```py path=a.py
def f(x: str):

View File

@@ -2,10 +2,16 @@
## Cyclical class definition
In type stubs, classes can reference themselves in their base class definitions. For example, in `typeshed`, we have `class str(Sequence[str]): ...`.
In type stubs, classes can reference themselves in their base class definitions. For example, in
`typeshed`, we have `class str(Sequence[str]): ...`.
```py path=a.pyi
class C(C): ...
class Foo[T]: ...
reveal_type(C) # revealed: Literal[C]
# TODO: actually is subscriptable
# error: [non-subscriptable]
class Bar(Foo[Bar]): ...
reveal_type(Bar) # revealed: Literal[Bar]
reveal_type(Bar.__mro__) # revealed: tuple[Literal[Bar], Unknown, Literal[object]]
```

View File

@@ -1,6 +1,6 @@
# Bytes subscript
# Bytes subscripts
## Simple
## Indexing
```py
b = b"\x00abc\xff"
@@ -21,14 +21,37 @@ reveal_type(x) # revealed: Unknown
y = b[-6] # error: [index-out-of-bounds] "Index -6 is out of bounds for bytes literal `Literal[b"\x00abc\xff"]` with length 5"
reveal_type(y) # revealed: Unknown
```
## Function return
```py
def int_instance() -> int: ...
def int_instance() -> int:
return 42
a = b"abcde"[int_instance()]
# TODO: Support overloads... Should be `bytes`
reveal_type(a) # revealed: @Todo
```
## Slices
```py
b = b"\x00abc\xff"
reveal_type(b[0:2]) # revealed: Literal[b"\x00a"]
reveal_type(b[-3:]) # revealed: Literal[b"bc\xff"]
b[0:4:0] # error: [zero-stepsize-in-slice]
b[:4:0] # error: [zero-stepsize-in-slice]
b[0::0] # error: [zero-stepsize-in-slice]
b[::0] # error: [zero-stepsize-in-slice]
def int_instance() -> int: ...
byte_slice1 = b[int_instance() : int_instance()]
# TODO: Support overloads... Should be `bytes`
reveal_type(byte_slice1) # revealed: @Todo
def bytes_instance() -> bytes: ...
byte_slice2 = bytes_instance()[0:5]
# TODO: Support overloads... Should be `bytes`
reveal_type(byte_slice2) # revealed: @Todo
```

View File

@@ -39,7 +39,8 @@ reveal_type(UnionClassGetItem[0]) # revealed: str | int
## Class getitem with class union
```py
flag = True
def bool_instance() -> bool:
return True
class A:
def __class_getitem__(cls, item: int) -> str:
@@ -49,7 +50,7 @@ class B:
def __class_getitem__(cls, item: int) -> int:
return item
x = A if flag else B
x = A if bool_instance() else B
reveal_type(x) # revealed: Literal[A, B]
reveal_type(x[0]) # revealed: str | int
@@ -68,8 +69,8 @@ if flag:
else:
class Spam: ...
# error: [call-non-callable] "Method `__class_getitem__` of type `Literal[__class_getitem__] | Unbound` is not callable on object of type `Literal[Spam, Spam]`"
# revealed: str | Unknown
# error: [call-possibly-unbound-method] "Method `__class_getitem__` of type `Literal[Spam, Spam]` is possibly unbound"
# revealed: str
reveal_type(Spam[42])
```

View File

@@ -23,8 +23,7 @@ reveal_type(x["a"]) # revealed: @Todo
## Assignments within list assignment
In assignment, we might also have a named assignment.
This should also get type checked.
In assignment, we might also have a named assignment. This should also get type checked.
```py
x = [1, 2, 3]

View File

@@ -0,0 +1,13 @@
# Stepsize zero in slices
We raise a `zero-stepsize-in-slice` diagnostic when trying to slice a literal string, bytes, or
tuple with a step size of zero (see tests in `string.md`, `bytes.md` and `tuple.md`). But we don't
want to raise this diagnostic when slicing a custom type:
```py
class MySequence:
def __getitem__(self, s: slice) -> int:
return 0
MySequence()[0:1:0] # No error
```

View File

@@ -1,6 +1,6 @@
# Subscript on strings
# String subscripts
## Simple
## Indexing
```py
s = "abcde"
@@ -18,14 +18,82 @@ reveal_type(a) # revealed: Unknown
b = s[-8] # error: [index-out-of-bounds] "Index -8 is out of bounds for string `Literal["abcde"]` with length 5"
reveal_type(b) # revealed: Unknown
```
## Function return
```py
def int_instance() -> int: ...
a = "abcde"[int_instance()]
# TODO: Support overloads... Should be `str`
reveal_type(a) # revealed: @Todo
```
## Slices
```py
s = "abcde"
reveal_type(s[0:0]) # revealed: Literal[""]
reveal_type(s[0:1]) # revealed: Literal["a"]
reveal_type(s[0:2]) # revealed: Literal["ab"]
reveal_type(s[0:5]) # revealed: Literal["abcde"]
reveal_type(s[0:6]) # revealed: Literal["abcde"]
reveal_type(s[1:3]) # revealed: Literal["bc"]
reveal_type(s[-3:5]) # revealed: Literal["cde"]
reveal_type(s[-4:-2]) # revealed: Literal["bc"]
reveal_type(s[-10:10]) # revealed: Literal["abcde"]
reveal_type(s[0:]) # revealed: Literal["abcde"]
reveal_type(s[2:]) # revealed: Literal["cde"]
reveal_type(s[5:]) # revealed: Literal[""]
reveal_type(s[:2]) # revealed: Literal["ab"]
reveal_type(s[:0]) # revealed: Literal[""]
reveal_type(s[:2]) # revealed: Literal["ab"]
reveal_type(s[:10]) # revealed: Literal["abcde"]
reveal_type(s[:]) # revealed: Literal["abcde"]
reveal_type(s[::-1]) # revealed: Literal["edcba"]
reveal_type(s[::2]) # revealed: Literal["ace"]
reveal_type(s[-2:-5:-1]) # revealed: Literal["dcb"]
reveal_type(s[::-2]) # revealed: Literal["eca"]
reveal_type(s[-1::-3]) # revealed: Literal["eb"]
reveal_type(s[None:2:None]) # revealed: Literal["ab"]
reveal_type(s[1:None:1]) # revealed: Literal["bcde"]
reveal_type(s[None:None:None]) # revealed: Literal["abcde"]
start = 1
stop = None
step = 2
reveal_type(s[start:stop:step]) # revealed: Literal["bd"]
reveal_type(s[False:True]) # revealed: Literal["a"]
reveal_type(s[True:3]) # revealed: Literal["bc"]
s[0:4:0] # error: [zero-stepsize-in-slice]
s[:4:0] # error: [zero-stepsize-in-slice]
s[0::0] # error: [zero-stepsize-in-slice]
s[::0] # error: [zero-stepsize-in-slice]
def int_instance() -> int: ...
substring1 = s[int_instance() : int_instance()]
# TODO: Support overloads... Should be `LiteralString`
reveal_type(substring1) # revealed: @Todo
def str_instance() -> str: ...
substring2 = str_instance()[0:5]
# TODO: Support overloads... Should be `str`
reveal_type(substring2) # revealed: @Todo
```
## Unsupported slice types
```py
# TODO: It would be great if we raised an error here. This can be done once
# we have support for overloads and generics, and once typeshed has a more
# precise annotation for `str.__getitem__`, that makes use of the generic
# `slice[..]` type. We could then infer `slice[str, str]` here and see that
# it doesn't match the signature of `str.__getitem__`.
"foo"["bar":"baz"]
```

View File

@@ -1,6 +1,6 @@
# Tuple subscripts
## Basic
## Indexing
```py
t = (1, "a", "b")
@@ -10,9 +10,66 @@ reveal_type(t[1]) # revealed: Literal["a"]
reveal_type(t[-1]) # revealed: Literal["b"]
reveal_type(t[-2]) # revealed: Literal["a"]
reveal_type(t[False]) # revealed: Literal[1]
reveal_type(t[True]) # revealed: Literal["a"]
a = t[4] # error: [index-out-of-bounds]
reveal_type(a) # revealed: Unknown
b = t[-4] # error: [index-out-of-bounds]
reveal_type(b) # revealed: Unknown
```
## Slices
```py
t = (1, "a", None, b"b")
reveal_type(t[0:0]) # revealed: tuple[()]
reveal_type(t[0:1]) # revealed: tuple[Literal[1]]
reveal_type(t[0:2]) # revealed: tuple[Literal[1], Literal["a"]]
reveal_type(t[0:4]) # revealed: tuple[Literal[1], Literal["a"], None, Literal[b"b"]]
reveal_type(t[0:5]) # revealed: tuple[Literal[1], Literal["a"], None, Literal[b"b"]]
reveal_type(t[1:3]) # revealed: tuple[Literal["a"], None]
reveal_type(t[-2:4]) # revealed: tuple[None, Literal[b"b"]]
reveal_type(t[-3:-1]) # revealed: tuple[Literal["a"], None]
reveal_type(t[-10:10]) # revealed: tuple[Literal[1], Literal["a"], None, Literal[b"b"]]
reveal_type(t[0:]) # revealed: tuple[Literal[1], Literal["a"], None, Literal[b"b"]]
reveal_type(t[2:]) # revealed: tuple[None, Literal[b"b"]]
reveal_type(t[4:]) # revealed: tuple[()]
reveal_type(t[:0]) # revealed: tuple[()]
reveal_type(t[:2]) # revealed: tuple[Literal[1], Literal["a"]]
reveal_type(t[:10]) # revealed: tuple[Literal[1], Literal["a"], None, Literal[b"b"]]
reveal_type(t[:]) # revealed: tuple[Literal[1], Literal["a"], None, Literal[b"b"]]
reveal_type(t[::-1]) # revealed: tuple[Literal[b"b"], None, Literal["a"], Literal[1]]
reveal_type(t[::2]) # revealed: tuple[Literal[1], None]
reveal_type(t[-2:-5:-1]) # revealed: tuple[None, Literal["a"], Literal[1]]
reveal_type(t[::-2]) # revealed: tuple[Literal[b"b"], Literal["a"]]
reveal_type(t[-1::-3]) # revealed: tuple[Literal[b"b"], Literal[1]]
reveal_type(t[None:2:None]) # revealed: tuple[Literal[1], Literal["a"]]
reveal_type(t[1:None:1]) # revealed: tuple[Literal["a"], None, Literal[b"b"]]
reveal_type(t[None:None:None]) # revealed: tuple[Literal[1], Literal["a"], None, Literal[b"b"]]
start = 1
stop = None
step = 2
reveal_type(t[start:stop:step]) # revealed: tuple[Literal["a"], Literal[b"b"]]
reveal_type(t[False:True]) # revealed: tuple[Literal[1]]
reveal_type(t[True:3]) # revealed: tuple[Literal["a"], None]
t[0:4:0] # error: [zero-stepsize-in-slice]
t[:4:0] # error: [zero-stepsize-in-slice]
t[0::0] # error: [zero-stepsize-in-slice]
t[::0] # error: [zero-stepsize-in-slice]
def int_instance() -> int: ...
tuple_slice = t[int_instance() : int_instance()]
# TODO: Support overloads... Should be `tuple[Literal[1, 'a', b"b"] | None, ...]`
reveal_type(tuple_slice) # revealed: @Todo
```

View File

@@ -1,6 +1,8 @@
# Unary Operations
```py
from typing import Literal
class Number:
def __init__(self, value: int):
self.value = 1
@@ -18,7 +20,7 @@ a = Number()
reveal_type(+a) # revealed: int
reveal_type(-a) # revealed: int
reveal_type(~a) # revealed: @Todo
reveal_type(~a) # revealed: Literal[True]
class NoDunder: ...

View File

@@ -81,8 +81,7 @@ reveal_type(b) # revealed: Literal[2]
```py
# TODO: Add diagnostic (need more values to unpack)
# TODO: Remove 'not-iterable' diagnostic
[a, *b, c, d] = (1, 2) # error: "Object of type `None` is not iterable"
[a, *b, c, d] = (1, 2)
reveal_type(a) # revealed: Literal[1]
# TODO: Should be list[Any] once support for assigning to starred expression is added
reveal_type(b) # revealed: @Todo
@@ -93,7 +92,7 @@ reveal_type(d) # revealed: Unknown
### Starred expression (2)
```py
[a, *b, c] = (1, 2) # error: "Object of type `None` is not iterable"
[a, *b, c] = (1, 2)
reveal_type(a) # revealed: Literal[1]
# TODO: Should be list[Any] once support for assigning to starred expression is added
reveal_type(b) # revealed: @Todo
@@ -103,8 +102,7 @@ reveal_type(c) # revealed: Literal[2]
### Starred expression (3)
```py
# TODO: Remove 'not-iterable' diagnostic
[a, *b, c] = (1, 2, 3) # error: "Object of type `None` is not iterable"
[a, *b, c] = (1, 2, 3)
reveal_type(a) # revealed: Literal[1]
# TODO: Should be list[int] once support for assigning to starred expression is added
reveal_type(b) # revealed: @Todo
@@ -114,8 +112,7 @@ reveal_type(c) # revealed: Literal[3]
### Starred expression (4)
```py
# TODO: Remove 'not-iterable' diagnostic
[a, *b, c, d] = (1, 2, 3, 4, 5, 6) # error: "Object of type `None` is not iterable"
[a, *b, c, d] = (1, 2, 3, 4, 5, 6)
reveal_type(a) # revealed: Literal[1]
# TODO: Should be list[int] once support for assigning to starred expression is added
reveal_type(b) # revealed: @Todo
@@ -126,22 +123,29 @@ reveal_type(d) # revealed: Literal[6]
### Starred expression (5)
```py
# TODO: Remove 'not-iterable' diagnostic
[a, b, *c] = (1, 2, 3, 4) # error: "Object of type `None` is not iterable"
[a, b, *c] = (1, 2, 3, 4)
reveal_type(a) # revealed: Literal[1]
reveal_type(b) # revealed: Literal[2]
# TODO: Should be list[int] once support for assigning to starred expression is added
reveal_type(c) # revealed: @Todo
```
### Non-iterable unpacking
TODO: Remove duplicate diagnostics. This is happening because for a sequence-like
assignment target, multiple definitions are created and the inference engine runs
on each of them which results in duplicate diagnostics.
### Starred expression (6)
```py
# TODO: Add diagnostic (need more values to unpack)
(a, b, c, *d, e, f) = (1,)
reveal_type(a) # revealed: Literal[1]
reveal_type(b) # revealed: Unknown
reveal_type(c) # revealed: Unknown
reveal_type(d) # revealed: @Todo
reveal_type(e) # revealed: Unknown
reveal_type(f) # revealed: Unknown
```
### Non-iterable unpacking
```py
# error: "Object of type `Literal[1]` is not iterable"
# error: "Object of type `Literal[1]` is not iterable"
a, b = 1
reveal_type(a) # revealed: Unknown
@@ -215,8 +219,7 @@ reveal_type(b) # revealed: LiteralString
```py
# TODO: Add diagnostic (need more values to unpack)
# TODO: Remove 'not-iterable' diagnostic
(a, *b, c, d) = "ab" # error: "Object of type `None` is not iterable"
(a, *b, c, d) = "ab"
reveal_type(a) # revealed: LiteralString
# TODO: Should be list[LiteralString] once support for assigning to starred expression is added
reveal_type(b) # revealed: @Todo
@@ -227,7 +230,7 @@ reveal_type(d) # revealed: Unknown
### Starred expression (2)
```py
(a, *b, c) = "ab" # error: "Object of type `None` is not iterable"
(a, *b, c) = "ab"
reveal_type(a) # revealed: LiteralString
# TODO: Should be list[Any] once support for assigning to starred expression is added
reveal_type(b) # revealed: @Todo
@@ -237,8 +240,7 @@ reveal_type(c) # revealed: LiteralString
### Starred expression (3)
```py
# TODO: Remove 'not-iterable' diagnostic
(a, *b, c) = "abc" # error: "Object of type `None` is not iterable"
(a, *b, c) = "abc"
reveal_type(a) # revealed: LiteralString
# TODO: Should be list[LiteralString] once support for assigning to starred expression is added
reveal_type(b) # revealed: @Todo
@@ -248,8 +250,7 @@ reveal_type(c) # revealed: LiteralString
### Starred expression (4)
```py
# TODO: Remove 'not-iterable' diagnostic
(a, *b, c, d) = "abcdef" # error: "Object of type `None` is not iterable"
(a, *b, c, d) = "abcdef"
reveal_type(a) # revealed: LiteralString
# TODO: Should be list[LiteralString] once support for assigning to starred expression is added
reveal_type(b) # revealed: @Todo
@@ -260,8 +261,7 @@ reveal_type(d) # revealed: LiteralString
### Starred expression (5)
```py
# TODO: Remove 'not-iterable' diagnostic
(a, b, *c) = "abcd" # error: "Object of type `None` is not iterable"
(a, b, *c) = "abcd"
reveal_type(a) # revealed: LiteralString
reveal_type(b) # revealed: LiteralString
# TODO: Should be list[int] once support for assigning to starred expression is added

View File

@@ -0,0 +1,21 @@
# Async with statements
## Basic `async with` statement
The type of the target variable in a `with` statement should be the return type from the context
manager's `__aenter__` method. However, `async with` statements aren't supported yet. This test
asserts that it doesn't emit any context manager-related errors.
```py
class Target: ...
class Manager:
async def __aenter__(self) -> Target:
return Target()
async def __aexit__(self, exc_type, exc_value, traceback): ...
async def test():
async with Manager() as f:
reveal_type(f) # revealed: @Todo
```

View File

@@ -0,0 +1,141 @@
# With statements
## Basic `with` statement
The type of the target variable in a `with` statement is the return type from the context manager's
`__enter__` method.
```py
class Target: ...
class Manager:
def __enter__(self) -> Target:
return Target()
def __exit__(self, exc_type, exc_value, traceback): ...
with Manager() as f:
reveal_type(f) # revealed: Target
```
## Union context manager
```py
def coinflip() -> bool:
return True
class Manager1:
def __enter__(self) -> str:
return "foo"
def __exit__(self, exc_type, exc_value, traceback): ...
class Manager2:
def __enter__(self) -> int:
return 42
def __exit__(self, exc_type, exc_value, traceback): ...
context_expr = Manager1() if coinflip() else Manager2()
with context_expr as f:
reveal_type(f) # revealed: str | int
```
## Context manager without an `__enter__` or `__exit__` method
```py
class Manager: ...
# error: [invalid-context-manager] "Object of type `Manager` cannot be used with `with` because it doesn't implement `__enter__` and `__exit__`"
with Manager():
...
```
## Context manager without an `__enter__` method
```py
class Manager:
def __exit__(self, exc_tpe, exc_value, traceback): ...
# error: [invalid-context-manager] "Object of type `Manager` cannot be used with `with` because it doesn't implement `__enter__`"
with Manager():
...
```
## Context manager without an `__exit__` method
```py
class Manager:
def __enter__(self): ...
# error: [invalid-context-manager] "Object of type `Manager` cannot be used with `with` because it doesn't implement `__exit__`"
with Manager():
...
```
## Context manager with non-callable `__enter__` attribute
```py
class Manager:
__enter__ = 42
def __exit__(self, exc_tpe, exc_value, traceback): ...
# error: [invalid-context-manager] "Object of type `Manager` cannot be used with `with` because the method `__enter__` of type `Literal[42]` is not callable"
with Manager():
...
```
## Context manager with non-callable `__exit__` attribute
```py
class Manager:
def __enter__(self) -> Self: ...
__exit__ = 32
# error: [invalid-context-manager] "Object of type `Manager` cannot be used with `with` because the method `__exit__` of type `Literal[32]` is not callable"
with Manager():
...
```
## Context expression with possibly-unbound union variants
```py
def coinflip() -> bool:
return True
class Manager1:
def __enter__(self) -> str:
return "foo"
def __exit__(self, exc_type, exc_value, traceback): ...
class NotAContextManager: ...
context_expr = Manager1() if coinflip() else NotAContextManager()
# error: [invalid-context-manager] "Object of type `Manager1 | NotAContextManager` cannot be used with `with` because the method `__enter__` is possibly unbound"
# error: [invalid-context-manager] "Object of type `Manager1 | NotAContextManager` cannot be used with `with` because the method `__exit__` is possibly unbound"
with context_expr as f:
reveal_type(f) # revealed: str
```
## Context expression with "sometimes" callable `__enter__` method
```py
def coinflip() -> bool:
return True
class Manager:
if coinflip():
def __enter__(self) -> str:
return "abcd"
def __exit__(self, *args): ...
# error: [invalid-context-manager] "Object of type `Manager` cannot be used with `with` because the method `__enter__` is possibly unbound"
with Manager() as f:
reveal_type(f) # revealed: str
```

View File

@@ -20,7 +20,9 @@ pub mod semantic_index;
mod semantic_model;
pub(crate) mod site_packages;
mod stdlib;
pub(crate) mod symbol;
pub mod types;
mod unpack;
mod util;
type FxOrderSet<V> = ordermap::set::OrderSet<V, BuildHasherDefault<FxHasher>>;

View File

@@ -125,6 +125,7 @@ impl<'db> SemanticIndex<'db> {
///
/// Use the Salsa cached [`symbol_table()`] query if you only need the
/// symbol table for a single scope.
#[track_caller]
pub(super) fn symbol_table(&self, scope_id: FileScopeId) -> Arc<SymbolTable> {
self.symbol_tables[scope_id].clone()
}
@@ -133,15 +134,18 @@ impl<'db> SemanticIndex<'db> {
///
/// Use the Salsa cached [`use_def_map()`] query if you only need the
/// use-def map for a single scope.
#[track_caller]
pub(super) fn use_def_map(&self, scope_id: FileScopeId) -> Arc<UseDefMap> {
self.use_def_maps[scope_id].clone()
}
#[track_caller]
pub(crate) fn ast_ids(&self, scope_id: FileScopeId) -> &AstIds {
&self.ast_ids[scope_id]
}
/// Returns the ID of the `expression`'s enclosing scope.
#[track_caller]
pub(crate) fn expression_scope_id(
&self,
expression: impl Into<ExpressionNodeKey>,
@@ -151,11 +155,13 @@ impl<'db> SemanticIndex<'db> {
/// Returns the [`Scope`] of the `expression`'s enclosing scope.
#[allow(unused)]
#[track_caller]
pub(crate) fn expression_scope(&self, expression: impl Into<ExpressionNodeKey>) -> &Scope {
&self.scopes[self.expression_scope_id(expression)]
}
/// Returns the [`Scope`] with the given id.
#[track_caller]
pub(crate) fn scope(&self, id: FileScopeId) -> &Scope {
&self.scopes[id]
}
@@ -172,6 +178,7 @@ impl<'db> SemanticIndex<'db> {
/// Returns the parent scope of `scope_id`.
#[allow(unused)]
#[track_caller]
pub(crate) fn parent_scope(&self, scope_id: FileScopeId) -> Option<&Scope> {
Some(&self.scopes[self.parent_scope_id(scope_id)?])
}
@@ -195,6 +202,7 @@ impl<'db> SemanticIndex<'db> {
}
/// Returns the [`Definition`] salsa ingredient for `definition_key`.
#[track_caller]
pub(crate) fn definition(
&self,
definition_key: impl Into<DefinitionNodeKey>,
@@ -206,6 +214,7 @@ impl<'db> SemanticIndex<'db> {
/// Panics if we have no expression ingredient for that node. We can only call this method for
/// standalone-inferable expressions, which we call `add_standalone_expression` for in
/// [`SemanticIndexBuilder`].
#[track_caller]
pub(crate) fn expression(
&self,
expression_key: impl Into<ExpressionNodeKey>,
@@ -213,8 +222,18 @@ impl<'db> SemanticIndex<'db> {
self.expressions_by_node[&expression_key.into()]
}
pub(crate) fn try_expression(
&self,
expression_key: impl Into<ExpressionNodeKey>,
) -> Option<Expression<'db>> {
self.expressions_by_node
.get(&expression_key.into())
.copied()
}
/// Returns the id of the scope that `node` creates. This is different from [`Definition::scope`] which
/// returns the scope in which that definition is defined in.
#[track_caller]
pub(crate) fn node_scope(&self, node: NodeWithScopeRef) -> FileScopeId {
self.scopes_by_node[&node.node_key()]
}

View File

@@ -87,6 +87,14 @@ pub trait HasScopedAstId {
fn scoped_ast_id(&self, db: &dyn Db, scope: ScopeId) -> Self::Id;
}
impl<T: HasScopedAstId> HasScopedAstId for Box<T> {
type Id = <T as HasScopedAstId>::Id;
fn scoped_ast_id(&self, db: &dyn Db, scope: ScopeId) -> Self::Id {
self.as_ref().scoped_ast_id(db, scope)
}
}
/// Uniquely identifies an [`ast::Expr`] in a [`crate::semantic_index::symbol::FileScopeId`].
#[newtype_index]
pub struct ScopedExpressionId;

View File

@@ -9,7 +9,7 @@ use ruff_index::IndexVec;
use ruff_python_ast as ast;
use ruff_python_ast::name::Name;
use ruff_python_ast::visitor::{walk_expr, walk_pattern, walk_stmt, Visitor};
use ruff_python_ast::AnyParameterRef;
use ruff_python_ast::{AnyParameterRef, BoolOp, Expr};
use crate::ast_node_ref::AstNodeRef;
use crate::semantic_index::ast_ids::node_key::ExpressionNodeKey;
@@ -25,12 +25,13 @@ use crate::semantic_index::symbol::{
};
use crate::semantic_index::use_def::{FlowSnapshot, UseDefMapBuilder};
use crate::semantic_index::SemanticIndex;
use crate::unpack::Unpack;
use crate::Db;
use super::constraint::{Constraint, PatternConstraint};
use super::constraint::{Constraint, ConstraintNode, PatternConstraint};
use super::definition::{
AssignmentKind, DefinitionCategory, ExceptHandlerDefinitionNodeRef,
MatchPatternDefinitionNodeRef, WithItemDefinitionNodeRef,
DefinitionCategory, ExceptHandlerDefinitionNodeRef, MatchPatternDefinitionNodeRef,
WithItemDefinitionNodeRef,
};
mod except_handlers;
@@ -46,6 +47,7 @@ pub(super) struct SemanticIndexBuilder<'db> {
current_assignments: Vec<CurrentAssignment<'db>>,
/// The match case we're currently visiting.
current_match_case: Option<CurrentMatchCase<'db>>,
/// Flow states at each `break` in the current loop.
loop_break_states: Vec<FlowSnapshot>,
/// Per-scope contexts regarding nested `try`/`except` statements
@@ -112,9 +114,11 @@ impl<'db> SemanticIndexBuilder<'db> {
fn push_scope_with_parent(&mut self, node: NodeWithScopeRef, parent: Option<FileScopeId>) {
let children_start = self.scopes.next_index() + 1;
#[allow(unsafe_code)]
let scope = Scope {
parent,
kind: node.scope_kind(),
// SAFETY: `node` is guaranteed to be a child of `self.module`
node: unsafe { node.to_kind(self.module.clone()) },
descendents: children_start..children_start,
};
self.try_node_context_stack_manager.enter_nested_scope();
@@ -124,15 +128,7 @@ impl<'db> SemanticIndexBuilder<'db> {
self.use_def_maps.push(UseDefMapBuilder::new());
let ast_id_scope = self.ast_ids.push(AstIdsBuilder::new());
#[allow(unsafe_code)]
// SAFETY: `node` is guaranteed to be a child of `self.module`
let scope_id = ScopeId::new(
self.db,
self.file,
file_scope_id,
unsafe { node.to_kind(self.module.clone()) },
countme::Count::default(),
);
let scope_id = ScopeId::new(self.db, self.file, file_scope_id, countme::Count::default());
self.scope_ids_by_scope.push(scope_id);
self.scopes_by_node.insert(node.node_key(), file_scope_id);
@@ -195,14 +191,18 @@ impl<'db> SemanticIndexBuilder<'db> {
self.current_symbol_table().mark_symbol_bound(id);
}
fn mark_symbol_declared(&mut self, id: ScopedSymbolId) {
self.current_symbol_table().mark_symbol_declared(id);
}
fn mark_symbol_used(&mut self, id: ScopedSymbolId) {
self.current_symbol_table().mark_symbol_used(id);
}
fn add_definition<'a>(
fn add_definition(
&mut self,
symbol: ScopedSymbolId,
definition_node: impl Into<DefinitionNodeRef<'a>>,
definition_node: impl Into<DefinitionNodeRef<'db>>,
) -> Definition<'db> {
let definition_node: DefinitionNodeRef<'_> = definition_node.into();
#[allow(unsafe_code)]
@@ -226,6 +226,9 @@ impl<'db> SemanticIndexBuilder<'db> {
if category.is_binding() {
self.mark_symbol_bound(symbol);
}
if category.is_declaration() {
self.mark_symbol_declared(symbol);
}
let use_def = self.current_use_def_map_mut();
match category {
@@ -243,12 +246,30 @@ impl<'db> SemanticIndexBuilder<'db> {
definition
}
fn add_expression_constraint(&mut self, constraint_node: &ast::Expr) -> Expression<'db> {
let expression = self.add_standalone_expression(constraint_node);
self.current_use_def_map_mut()
.record_constraint(Constraint::Expression(expression));
fn record_expression_constraint(&mut self, constraint_node: &ast::Expr) -> Constraint<'db> {
let constraint = self.build_constraint(constraint_node);
self.record_constraint(constraint);
constraint
}
expression
fn record_constraint(&mut self, constraint: Constraint<'db>) {
self.current_use_def_map_mut().record_constraint(constraint);
}
fn build_constraint(&mut self, constraint_node: &Expr) -> Constraint<'db> {
let expression = self.add_standalone_expression(constraint_node);
Constraint {
node: ConstraintNode::Expression(expression),
is_positive: true,
}
}
fn record_negated_constraint(&mut self, constraint: Constraint<'db>) {
self.current_use_def_map_mut()
.record_constraint(Constraint {
node: constraint.node,
is_positive: false,
});
}
fn push_assignment(&mut self, assignment: CurrentAssignment<'db>) {
@@ -260,8 +281,12 @@ impl<'db> SemanticIndexBuilder<'db> {
debug_assert!(popped_assignment.is_some());
}
fn current_assignment(&self) -> Option<&CurrentAssignment<'db>> {
self.current_assignments.last()
fn current_assignment(&self) -> Option<CurrentAssignment<'db>> {
self.current_assignments.last().copied()
}
fn current_assignment_mut(&mut self) -> Option<&mut CurrentAssignment<'db>> {
self.current_assignments.last_mut()
}
fn add_pattern_constraint(
@@ -285,7 +310,10 @@ impl<'db> SemanticIndexBuilder<'db> {
countme::Count::default(),
);
self.current_use_def_map_mut()
.record_constraint(Constraint::Pattern(pattern_constraint));
.record_constraint(Constraint {
node: ConstraintNode::Pattern(pattern_constraint),
is_positive: true,
});
pattern_constraint
}
@@ -338,6 +366,7 @@ impl<'db> SemanticIndexBuilder<'db> {
// note that the "bound" on the typevar is a totally different thing than whether
// or not a name is "bound" by a typevar declaration; the latter is always true.
self.mark_symbol_bound(symbol);
self.mark_symbol_declared(symbol);
if let Some(bounds) = bound {
self.visit_expr(bounds);
}
@@ -416,7 +445,7 @@ impl<'db> SemanticIndexBuilder<'db> {
self.pop_scope();
}
fn declare_parameter(&mut self, parameter: AnyParameterRef) {
fn declare_parameter(&mut self, parameter: AnyParameterRef<'db>) {
let symbol = self.add_symbol(parameter.name().id().clone());
let definition = self.add_definition(symbol, parameter);
@@ -590,24 +619,48 @@ where
}
ast::Stmt::Assign(node) => {
debug_assert_eq!(&self.current_assignments, &[]);
self.visit_expr(&node.value);
self.add_standalone_expression(&node.value);
for (target_index, target) in node.targets.iter().enumerate() {
let kind = match target {
ast::Expr::List(_) | ast::Expr::Tuple(_) => Some(AssignmentKind::Sequence),
ast::Expr::Name(_) => Some(AssignmentKind::Name),
let value = self.add_standalone_expression(&node.value);
for target in &node.targets {
// We only handle assignments to names and unpackings here, other targets like
// attribute and subscript are handled separately as they don't create a new
// definition.
let current_assignment = match target {
ast::Expr::List(_) | ast::Expr::Tuple(_) => {
Some(CurrentAssignment::Assign {
node,
first: true,
unpack: Some(Unpack::new(
self.db,
self.file,
self.current_scope(),
#[allow(unsafe_code)]
unsafe {
AstNodeRef::new(self.module.clone(), target)
},
value,
countme::Count::default(),
)),
})
}
ast::Expr::Name(_) => Some(CurrentAssignment::Assign {
node,
unpack: None,
first: false,
}),
_ => None,
};
if let Some(kind) = kind {
self.push_assignment(CurrentAssignment::Assign {
assignment: node,
target_index,
kind,
});
if let Some(current_assignment) = current_assignment {
self.push_assignment(current_assignment);
}
self.visit_expr(target);
if kind.is_some() {
// only need to pop in the case where we pushed something
if current_assignment.is_some() {
// Only need to pop in the case where we pushed something
self.pop_assignment();
}
}
@@ -639,7 +692,8 @@ where
ast::Stmt::If(node) => {
self.visit_expr(&node.test);
let pre_if = self.flow_snapshot();
self.add_expression_constraint(&node.test);
let constraint = self.record_expression_constraint(&node.test);
let mut constraints = vec![constraint];
self.visit_body(&node.body);
let mut post_clauses: Vec<FlowSnapshot> = vec![];
for clause in &node.elif_else_clauses {
@@ -649,7 +703,14 @@ where
// we can only take an elif/else branch if none of the previous ones were
// taken, so the block entry state is always `pre_if`
self.flow_restore(pre_if.clone());
self.visit_elif_else_clause(clause);
for constraint in &constraints {
self.record_negated_constraint(*constraint);
}
if let Some(elif_test) = &clause.test {
self.visit_expr(elif_test);
constraints.push(self.record_expression_constraint(elif_test));
}
self.visit_body(&clause.body);
}
for post_clause_state in post_clauses {
self.flow_merge(post_clause_state);
@@ -697,12 +758,20 @@ where
self.flow_merge(break_state);
}
}
ast::Stmt::With(ast::StmtWith { items, body, .. }) => {
ast::Stmt::With(ast::StmtWith {
items,
body,
is_async,
..
}) => {
for item in items {
self.visit_expr(&item.context_expr);
if let Some(optional_vars) = item.optional_vars.as_deref() {
self.add_standalone_expression(&item.context_expr);
self.push_assignment(item.into());
self.push_assignment(CurrentAssignment::WithItem {
item,
is_async: *is_async,
});
self.visit_expr(optional_vars);
self.pop_assignment();
}
@@ -918,20 +987,26 @@ where
};
let symbol = self.add_symbol(id.clone());
if is_use {
self.mark_symbol_used(symbol);
let use_id = self.current_ast_ids().record_use(expr);
self.current_use_def_map_mut().record_use(symbol, use_id);
}
if is_definition {
match self.current_assignment().copied() {
match self.current_assignment() {
Some(CurrentAssignment::Assign {
assignment,
target_index,
kind,
node,
first,
unpack,
}) => {
self.add_definition(
symbol,
AssignmentDefinitionNodeRef {
assignment,
target_index,
unpack,
value: &node.value,
name: name_node,
kind,
first,
},
);
}
@@ -968,12 +1043,13 @@ where
},
);
}
Some(CurrentAssignment::WithItem(with_item)) => {
Some(CurrentAssignment::WithItem { item, is_async }) => {
self.add_definition(
symbol,
WithItemDefinitionNodeRef {
node: with_item,
node: item,
target: name_node,
is_async,
},
);
}
@@ -981,10 +1057,9 @@ where
}
}
if is_use {
self.mark_symbol_used(symbol);
let use_id = self.current_ast_ids().record_use(expr);
self.current_use_def_map_mut().record_use(symbol, use_id);
if let Some(CurrentAssignment::Assign { first, .. }) = self.current_assignment_mut()
{
*first = false;
}
walk_expr(self, expr);
@@ -1027,10 +1102,13 @@ where
// AST inspection, so we can't simplify here, need to record test expression for
// later checking)
self.visit_expr(test);
let constraint = self.record_expression_constraint(test);
let pre_if = self.flow_snapshot();
self.visit_expr(body);
let post_body = self.flow_snapshot();
self.flow_restore(pre_if);
self.record_negated_constraint(constraint);
self.visit_expr(orelse);
self.flow_merge(post_body);
}
@@ -1084,6 +1162,33 @@ where
},
);
}
ast::Expr::BoolOp(ast::ExprBoolOp {
values,
range: _,
op,
}) => {
// TODO detect statically known truthy or falsy values (via type inference, not naive
// AST inspection, so we can't simplify here, need to record test expression for
// later checking)
let mut snapshots = vec![];
for (index, value) in values.iter().enumerate() {
self.visit_expr(value);
// In the last value we don't need to take a snapshot nor add a constraint
if index < values.len() - 1 {
// Snapshot is taken after visiting the expression but before adding the constraint.
snapshots.push(self.flow_snapshot());
let constraint = self.build_constraint(value);
match op {
BoolOp::And => self.record_constraint(constraint),
BoolOp::Or => self.record_negated_constraint(constraint),
}
}
}
for snapshot in snapshots {
self.flow_merge(snapshot);
}
}
_ => {
walk_expr(self, expr);
}
@@ -1156,9 +1261,9 @@ where
#[derive(Copy, Clone, Debug, PartialEq)]
enum CurrentAssignment<'a> {
Assign {
assignment: &'a ast::StmtAssign,
target_index: usize,
kind: AssignmentKind,
node: &'a ast::StmtAssign,
first: bool,
unpack: Option<Unpack<'a>>,
},
AnnAssign(&'a ast::StmtAnnAssign),
AugAssign(&'a ast::StmtAugAssign),
@@ -1168,7 +1273,10 @@ enum CurrentAssignment<'a> {
node: &'a ast::Comprehension,
first: bool,
},
WithItem(&'a ast::WithItem),
WithItem {
item: &'a ast::WithItem,
is_async: bool,
},
}
impl<'a> From<&'a ast::StmtAnnAssign> for CurrentAssignment<'a> {
@@ -1195,12 +1303,6 @@ impl<'a> From<&'a ast::ExprNamed> for CurrentAssignment<'a> {
}
}
impl<'a> From<&'a ast::WithItem> for CurrentAssignment<'a> {
fn from(value: &'a ast::WithItem) -> Self {
Self::WithItem(value)
}
}
struct CurrentMatchCase<'a> {
/// The pattern that's part of the current match case.
pattern: &'a ast::Pattern,

View File

@@ -7,7 +7,13 @@ use crate::semantic_index::expression::Expression;
use crate::semantic_index::symbol::{FileScopeId, ScopeId};
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub(crate) enum Constraint<'db> {
pub(crate) struct Constraint<'db> {
pub(crate) node: ConstraintNode<'db>,
pub(crate) is_positive: bool,
}
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub(crate) enum ConstraintNode<'db> {
Expression(Expression<'db>),
Pattern(PatternConstraint<'db>),
}

View File

@@ -6,8 +6,22 @@ use crate::ast_node_ref::AstNodeRef;
use crate::module_resolver::file_to_module;
use crate::node_key::NodeKey;
use crate::semantic_index::symbol::{FileScopeId, ScopeId, ScopedSymbolId};
use crate::unpack::Unpack;
use crate::Db;
/// A definition of a symbol.
///
/// ## Module-local type
/// This type should not be used as part of any cross-module API because
/// it holds a reference to the AST node. Range-offset changes
/// then propagate through all usages, and deserialization requires
/// reparsing the entire module.
///
/// E.g. don't use this type in:
///
/// * a return type of a cross-module query
/// * a field of a type that is a return type of a cross-module query
/// * an argument of a cross-module query
#[salsa::tracked]
pub struct Definition<'db> {
/// The file in which the definition occurs.
@@ -24,7 +38,7 @@ pub struct Definition<'db> {
#[no_eq]
#[return_ref]
pub(crate) kind: DefinitionKind,
pub(crate) kind: DefinitionKind<'db>,
#[no_eq]
count: countme::Count<Definition<'static>>,
@@ -166,16 +180,17 @@ pub(crate) struct ImportFromDefinitionNodeRef<'a> {
#[derive(Copy, Clone, Debug)]
pub(crate) struct AssignmentDefinitionNodeRef<'a> {
pub(crate) assignment: &'a ast::StmtAssign,
pub(crate) target_index: usize,
pub(crate) unpack: Option<Unpack<'a>>,
pub(crate) value: &'a ast::Expr,
pub(crate) name: &'a ast::ExprName,
pub(crate) kind: AssignmentKind,
pub(crate) first: bool,
}
#[derive(Copy, Clone, Debug)]
pub(crate) struct WithItemDefinitionNodeRef<'a> {
pub(crate) node: &'a ast::WithItem,
pub(crate) target: &'a ast::ExprName,
pub(crate) is_async: bool,
}
#[derive(Copy, Clone, Debug)]
@@ -210,9 +225,9 @@ pub(crate) struct MatchPatternDefinitionNodeRef<'a> {
pub(crate) index: u32,
}
impl DefinitionNodeRef<'_> {
impl<'db> DefinitionNodeRef<'db> {
#[allow(unsafe_code)]
pub(super) unsafe fn into_owned(self, parsed: ParsedModule) -> DefinitionKind {
pub(super) unsafe fn into_owned(self, parsed: ParsedModule) -> DefinitionKind<'db> {
match self {
DefinitionNodeRef::Import(alias) => {
DefinitionKind::Import(AstNodeRef::new(parsed, alias))
@@ -233,15 +248,15 @@ impl DefinitionNodeRef<'_> {
DefinitionKind::NamedExpression(AstNodeRef::new(parsed, named))
}
DefinitionNodeRef::Assignment(AssignmentDefinitionNodeRef {
assignment,
target_index,
unpack,
value,
name,
kind,
first,
}) => DefinitionKind::Assignment(AssignmentDefinitionKind {
assignment: AstNodeRef::new(parsed.clone(), assignment),
target_index,
target: TargetKind::from(unpack),
value: AstNodeRef::new(parsed.clone(), value),
name: AstNodeRef::new(parsed, name),
kind,
first,
}),
DefinitionNodeRef::AnnotatedAssignment(assign) => {
DefinitionKind::AnnotatedAssignment(AstNodeRef::new(parsed, assign))
@@ -277,12 +292,15 @@ impl DefinitionNodeRef<'_> {
DefinitionKind::ParameterWithDefault(AstNodeRef::new(parsed, parameter))
}
},
DefinitionNodeRef::WithItem(WithItemDefinitionNodeRef { node, target }) => {
DefinitionKind::WithItem(WithItemDefinitionKind {
node: AstNodeRef::new(parsed.clone(), node),
target: AstNodeRef::new(parsed, target),
})
}
DefinitionNodeRef::WithItem(WithItemDefinitionNodeRef {
node,
target,
is_async,
}) => DefinitionKind::WithItem(WithItemDefinitionKind {
node: AstNodeRef::new(parsed.clone(), node),
target: AstNodeRef::new(parsed, target),
is_async,
}),
DefinitionNodeRef::MatchPattern(MatchPatternDefinitionNodeRef {
pattern,
identifier,
@@ -312,10 +330,10 @@ impl DefinitionNodeRef<'_> {
Self::Class(node) => node.into(),
Self::NamedExpression(node) => node.into(),
Self::Assignment(AssignmentDefinitionNodeRef {
assignment: _,
target_index: _,
value: _,
unpack: _,
name,
kind: _,
first: _,
}) => name.into(),
Self::AnnotatedAssignment(node) => node.into(),
Self::AugmentedAssignment(node) => node.into(),
@@ -329,7 +347,11 @@ impl DefinitionNodeRef<'_> {
ast::AnyParameterRef::Variadic(parameter) => parameter.into(),
ast::AnyParameterRef::NonVariadic(parameter) => parameter.into(),
},
Self::WithItem(WithItemDefinitionNodeRef { node: _, target }) => target.into(),
Self::WithItem(WithItemDefinitionNodeRef {
node: _,
target,
is_async: _,
}) => target.into(),
Self::MatchPattern(MatchPatternDefinitionNodeRef { identifier, .. }) => {
identifier.into()
}
@@ -374,13 +396,13 @@ impl DefinitionCategory {
}
#[derive(Clone, Debug)]
pub enum DefinitionKind {
pub enum DefinitionKind<'db> {
Import(AstNodeRef<ast::Alias>),
ImportFrom(ImportFromDefinitionKind),
Function(AstNodeRef<ast::StmtFunctionDef>),
Class(AstNodeRef<ast::StmtClassDef>),
NamedExpression(AstNodeRef<ast::ExprNamed>),
Assignment(AssignmentDefinitionKind),
Assignment(AssignmentDefinitionKind<'db>),
AnnotatedAssignment(AstNodeRef<ast::StmtAnnAssign>),
AugmentedAssignment(AstNodeRef<ast::StmtAugAssign>),
For(ForStmtDefinitionKind),
@@ -392,7 +414,7 @@ pub enum DefinitionKind {
ExceptHandler(ExceptHandlerDefinitionKind),
}
impl DefinitionKind {
impl DefinitionKind<'_> {
pub(crate) fn category(&self) -> DefinitionCategory {
match self {
// functions, classes, and imports always bind, and we consider them declarations
@@ -437,6 +459,21 @@ impl DefinitionKind {
}
}
#[derive(Copy, Clone, Debug, PartialEq)]
pub(crate) enum TargetKind<'db> {
Sequence(Unpack<'db>),
Name,
}
impl<'db> From<Option<Unpack<'db>>> for TargetKind<'db> {
fn from(value: Option<Unpack<'db>>) -> Self {
match value {
Some(unpack) => TargetKind::Sequence(unpack),
None => TargetKind::Name,
}
}
}
#[derive(Clone, Debug)]
#[allow(dead_code)]
pub struct MatchPatternDefinitionKind {
@@ -498,42 +535,36 @@ impl ImportFromDefinitionKind {
}
#[derive(Clone, Debug)]
pub struct AssignmentDefinitionKind {
assignment: AstNodeRef<ast::StmtAssign>,
target_index: usize,
pub struct AssignmentDefinitionKind<'db> {
target: TargetKind<'db>,
value: AstNodeRef<ast::Expr>,
name: AstNodeRef<ast::ExprName>,
kind: AssignmentKind,
first: bool,
}
impl AssignmentDefinitionKind {
pub(crate) fn value(&self) -> &ast::Expr {
&self.assignment.node().value
impl<'db> AssignmentDefinitionKind<'db> {
pub(crate) fn target(&self) -> TargetKind<'db> {
self.target
}
pub(crate) fn target(&self) -> &ast::Expr {
&self.assignment.node().targets[self.target_index]
pub(crate) fn value(&self) -> &ast::Expr {
self.value.node()
}
pub(crate) fn name(&self) -> &ast::ExprName {
self.name.node()
}
pub(crate) fn kind(&self) -> AssignmentKind {
self.kind
pub(crate) fn is_first(&self) -> bool {
self.first
}
}
/// The kind of assignment target expression.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum AssignmentKind {
Sequence,
Name,
}
#[derive(Clone, Debug)]
pub struct WithItemDefinitionKind {
node: AstNodeRef<ast::WithItem>,
target: AstNodeRef<ast::ExprName>,
is_async: bool,
}
impl WithItemDefinitionKind {
@@ -544,6 +575,10 @@ impl WithItemDefinitionKind {
pub(crate) fn target(&self) -> &ast::ExprName {
self.target.node()
}
pub(crate) const fn is_async(&self) -> bool {
self.is_async
}
}
#[derive(Clone, Debug)]

View File

@@ -8,6 +8,18 @@ use salsa;
/// An independently type-inferable expression.
///
/// Includes constraint expressions (e.g. if tests) and the RHS of an unpacking assignment.
///
/// ## Module-local type
/// This type should not be used as part of any cross-module API because
/// it holds a reference to the AST node. Range-offset changes
/// then propagate through all usages, and deserialization requires
/// reparsing the entire module.
///
/// E.g. don't use this type in:
///
/// * a return type of a cross-module query
/// * a field of a type that is a return type of a cross-module query
/// * an argument of a cross-module query
#[salsa::tracked]
pub(crate) struct Expression<'db> {
/// The file in which the expression occurs.

View File

@@ -47,17 +47,27 @@ impl Symbol {
pub fn is_bound(&self) -> bool {
self.flags.contains(SymbolFlags::IS_BOUND)
}
/// Is the symbol declared in its containing scope?
pub fn is_declared(&self) -> bool {
self.flags.contains(SymbolFlags::IS_DECLARED)
}
}
bitflags! {
/// Flags that can be queried to obtain information about a symbol in a given scope.
///
/// See the doc-comment at the top of [`super::use_def`] for explanations of what it
/// means for a symbol to be *bound* as opposed to *declared*.
#[derive(Copy, Clone, Debug, Eq, PartialEq)]
struct SymbolFlags: u8 {
const IS_USED = 1 << 0;
const IS_BOUND = 1 << 1;
const IS_BOUND = 1 << 1;
const IS_DECLARED = 1 << 2;
/// TODO: This flag is not yet set by anything
const MARKED_GLOBAL = 1 << 2;
const MARKED_GLOBAL = 1 << 3;
/// TODO: This flag is not yet set by anything
const MARKED_NONLOCAL = 1 << 3;
const MARKED_NONLOCAL = 1 << 4;
}
}
@@ -93,14 +103,10 @@ pub struct ScopedSymbolId;
pub struct ScopeId<'db> {
#[id]
pub file: File,
#[id]
pub file_scope_id: FileScopeId,
/// The node that introduces this scope.
#[no_eq]
#[return_ref]
pub node: NodeWithScopeKind,
#[no_eq]
count: countme::Count<ScopeId<'static>>,
}
@@ -121,6 +127,14 @@ impl<'db> ScopeId<'db> {
)
}
pub(crate) fn node(self, db: &dyn Db) -> &NodeWithScopeKind {
self.scope(db).node()
}
pub(crate) fn scope(self, db: &dyn Db) -> &Scope {
semantic_index(db, self.file(db)).scope(self.file_scope_id(db))
}
#[cfg(test)]
pub(crate) fn name(self, db: &'db dyn Db) -> &'db str {
match self.node(db) {
@@ -159,10 +173,10 @@ impl FileScopeId {
}
}
#[derive(Debug, Eq, PartialEq)]
#[derive(Debug)]
pub struct Scope {
pub(super) parent: Option<FileScopeId>,
pub(super) kind: ScopeKind,
pub(super) node: NodeWithScopeKind,
pub(super) descendents: Range<FileScopeId>,
}
@@ -171,8 +185,12 @@ impl Scope {
self.parent
}
pub fn node(&self) -> &NodeWithScopeKind {
&self.node
}
pub fn kind(&self) -> ScopeKind {
self.kind
self.node().scope_kind()
}
}
@@ -298,6 +316,10 @@ impl SymbolTableBuilder {
self.table.symbols[id].insert_flags(SymbolFlags::IS_BOUND);
}
pub(super) fn mark_symbol_declared(&mut self, id: ScopedSymbolId) {
self.table.symbols[id].insert_flags(SymbolFlags::IS_DECLARED);
}
pub(super) fn mark_symbol_used(&mut self, id: ScopedSymbolId) {
self.table.symbols[id].insert_flags(SymbolFlags::IS_USED);
}
@@ -362,21 +384,6 @@ impl NodeWithScopeRef<'_> {
}
}
pub(super) fn scope_kind(self) -> ScopeKind {
match self {
NodeWithScopeRef::Module => ScopeKind::Module,
NodeWithScopeRef::Class(_) => ScopeKind::Class,
NodeWithScopeRef::Function(_) => ScopeKind::Function,
NodeWithScopeRef::Lambda(_) => ScopeKind::Function,
NodeWithScopeRef::FunctionTypeParameters(_)
| NodeWithScopeRef::ClassTypeParameters(_) => ScopeKind::Annotation,
NodeWithScopeRef::ListComprehension(_)
| NodeWithScopeRef::SetComprehension(_)
| NodeWithScopeRef::DictComprehension(_)
| NodeWithScopeRef::GeneratorExpression(_) => ScopeKind::Comprehension,
}
}
pub(crate) fn node_key(self) -> NodeWithScopeKey {
match self {
NodeWithScopeRef::Module => NodeWithScopeKey::Module,
@@ -424,6 +431,36 @@ pub enum NodeWithScopeKind {
GeneratorExpression(AstNodeRef<ast::ExprGenerator>),
}
impl NodeWithScopeKind {
pub(super) const fn scope_kind(&self) -> ScopeKind {
match self {
Self::Module => ScopeKind::Module,
Self::Class(_) => ScopeKind::Class,
Self::Function(_) => ScopeKind::Function,
Self::Lambda(_) => ScopeKind::Function,
Self::FunctionTypeParameters(_) | Self::ClassTypeParameters(_) => ScopeKind::Annotation,
Self::ListComprehension(_)
| Self::SetComprehension(_)
| Self::DictComprehension(_)
| Self::GeneratorExpression(_) => ScopeKind::Comprehension,
}
}
pub fn expect_class(&self) -> &ast::StmtClassDef {
match self {
Self::Class(class) => class.node(),
_ => panic!("expected class"),
}
}
pub fn expect_function(&self) -> &ast::StmtFunctionDef {
match self {
Self::Function(function) => function.node(),
_ => panic!("expected function"),
}
}
}
#[derive(Copy, Clone, Debug, Eq, PartialEq, Hash)]
pub(crate) enum NodeWithScopeKey {
Module,

View File

@@ -228,6 +228,7 @@ use self::symbol_state::{
use crate::semantic_index::ast_ids::ScopedUseId;
use crate::semantic_index::definition::Definition;
use crate::semantic_index::symbol::ScopedSymbolId;
use crate::symbol::Boundness;
use ruff_index::IndexVec;
use rustc_hash::FxHashMap;
@@ -274,8 +275,12 @@ impl<'db> UseDefMap<'db> {
self.bindings_iterator(&self.bindings_by_use[use_id])
}
pub(crate) fn use_may_be_unbound(&self, use_id: ScopedUseId) -> bool {
self.bindings_by_use[use_id].may_be_unbound()
pub(crate) fn use_boundness(&self, use_id: ScopedUseId) -> Boundness {
if self.bindings_by_use[use_id].may_be_unbound() {
Boundness::MayBeUnbound
} else {
Boundness::Bound
}
}
pub(crate) fn public_bindings(
@@ -285,8 +290,12 @@ impl<'db> UseDefMap<'db> {
self.bindings_iterator(self.public_symbols[symbol].bindings())
}
pub(crate) fn public_may_be_unbound(&self, symbol: ScopedSymbolId) -> bool {
self.public_symbols[symbol].may_be_unbound()
pub(crate) fn public_boundness(&self, symbol: ScopedSymbolId) -> Boundness {
if self.public_symbols[symbol].may_be_unbound() {
Boundness::MayBeUnbound
} else {
Boundness::Bound
}
}
pub(crate) fn bindings_at_declaration(

View File

@@ -8,7 +8,7 @@ use crate::module_name::ModuleName;
use crate::module_resolver::{resolve_module, Module};
use crate::semantic_index::ast_ids::HasScopedAstId;
use crate::semantic_index::semantic_index;
use crate::types::{binding_ty, global_symbol_ty, infer_scope_types, Type};
use crate::types::{binding_ty, infer_scope_types, Type};
use crate::Db;
pub struct SemanticModel<'db> {
@@ -38,10 +38,6 @@ impl<'db> SemanticModel<'db> {
pub fn resolve_module(&self, module_name: &ModuleName) -> Option<Module> {
resolve_module(self.db, module_name)
}
pub fn global_symbol_ty(&self, module: &Module, symbol_name: &str) -> Type<'db> {
global_symbol_ty(self.db, module.file(), symbol_name)
}
}
pub trait HasTy {

View File

@@ -2,7 +2,8 @@ use crate::module_name::ModuleName;
use crate::module_resolver::resolve_module;
use crate::semantic_index::global_scope;
use crate::semantic_index::symbol::ScopeId;
use crate::types::{global_symbol_ty, Type};
use crate::symbol::Symbol;
use crate::types::global_symbol;
use crate::Db;
/// Enumeration of various core stdlib modules, for which we have dedicated Salsa queries.
@@ -12,6 +13,7 @@ enum CoreStdlibModule {
Types,
Typeshed,
TypingExtensions,
Typing,
}
impl CoreStdlibModule {
@@ -19,6 +21,7 @@ impl CoreStdlibModule {
let module_name = match self {
Self::Builtins => "builtins",
Self::Types => "types",
Self::Typing => "typing",
Self::Typeshed => "_typeshed",
Self::TypingExtensions => "typing_extensions",
};
@@ -29,54 +32,55 @@ impl CoreStdlibModule {
/// Lookup the type of `symbol` in a given core module
///
/// Returns `Unbound` if the given core module cannot be resolved for some reason
fn core_module_symbol_ty<'db>(
/// Returns `Symbol::Unbound` if the given core module cannot be resolved for some reason
fn core_module_symbol<'db>(
db: &'db dyn Db,
core_module: CoreStdlibModule,
symbol: &str,
) -> Type<'db> {
) -> Symbol<'db> {
resolve_module(db, &core_module.name())
.map(|module| global_symbol_ty(db, module.file(), symbol))
.map(|ty| {
if ty.is_unbound() {
ty
} else {
ty.replace_unbound_with(db, Type::Never)
}
})
.unwrap_or(Type::Unbound)
.map(|module| global_symbol(db, module.file(), symbol))
.unwrap_or(Symbol::Unbound)
}
/// Lookup the type of `symbol` in the builtins namespace.
///
/// Returns `Unbound` if the `builtins` module isn't available for some reason.
/// Returns `Symbol::Unbound` if the `builtins` module isn't available for some reason.
#[inline]
pub(crate) fn builtins_symbol_ty<'db>(db: &'db dyn Db, symbol: &str) -> Type<'db> {
core_module_symbol_ty(db, CoreStdlibModule::Builtins, symbol)
pub(crate) fn builtins_symbol<'db>(db: &'db dyn Db, symbol: &str) -> Symbol<'db> {
core_module_symbol(db, CoreStdlibModule::Builtins, symbol)
}
/// Lookup the type of `symbol` in the `types` module namespace.
///
/// Returns `Unbound` if the `types` module isn't available for some reason.
/// Returns `Symbol::Unbound` if the `types` module isn't available for some reason.
#[inline]
pub(crate) fn types_symbol_ty<'db>(db: &'db dyn Db, symbol: &str) -> Type<'db> {
core_module_symbol_ty(db, CoreStdlibModule::Types, symbol)
pub(crate) fn types_symbol<'db>(db: &'db dyn Db, symbol: &str) -> Symbol<'db> {
core_module_symbol(db, CoreStdlibModule::Types, symbol)
}
/// Lookup the type of `symbol` in the `typing` module namespace.
///
/// Returns `Symbol::Unbound` if the `typing` module isn't available for some reason.
#[inline]
#[allow(dead_code)] // currently only used in tests
pub(crate) fn typing_symbol<'db>(db: &'db dyn Db, symbol: &str) -> Symbol<'db> {
core_module_symbol(db, CoreStdlibModule::Typing, symbol)
}
/// Lookup the type of `symbol` in the `_typeshed` module namespace.
///
/// Returns `Unbound` if the `_typeshed` module isn't available for some reason.
/// Returns `Symbol::Unbound` if the `_typeshed` module isn't available for some reason.
#[inline]
pub(crate) fn typeshed_symbol_ty<'db>(db: &'db dyn Db, symbol: &str) -> Type<'db> {
core_module_symbol_ty(db, CoreStdlibModule::Typeshed, symbol)
pub(crate) fn typeshed_symbol<'db>(db: &'db dyn Db, symbol: &str) -> Symbol<'db> {
core_module_symbol(db, CoreStdlibModule::Typeshed, symbol)
}
/// Lookup the type of `symbol` in the `typing_extensions` module namespace.
///
/// Returns `Unbound` if the `typing_extensions` module isn't available for some reason.
/// Returns `Symbol::Unbound` if the `typing_extensions` module isn't available for some reason.
#[inline]
pub(crate) fn typing_extensions_symbol_ty<'db>(db: &'db dyn Db, symbol: &str) -> Type<'db> {
core_module_symbol_ty(db, CoreStdlibModule::TypingExtensions, symbol)
pub(crate) fn typing_extensions_symbol<'db>(db: &'db dyn Db, symbol: &str) -> Symbol<'db> {
core_module_symbol(db, CoreStdlibModule::TypingExtensions, symbol)
}
/// Get the scope of a core stdlib module.

View File

@@ -0,0 +1,92 @@
use crate::{
types::{Type, UnionType},
Db,
};
#[derive(Debug, Clone, Copy, PartialEq)]
pub(crate) enum Boundness {
Bound,
MayBeUnbound,
}
/// The result of a symbol lookup, which can either be a (possibly unbound) type
/// or a completely unbound symbol.
///
/// Consider this example:
/// ```py
/// bound = 1
///
/// if flag:
/// maybe_unbound = 2
/// ```
///
/// If we look up symbols in this scope, we would get the following results:
/// ```rs
/// bound: Symbol::Type(Type::IntLiteral(1), Boundness::Bound),
/// maybe_unbound: Symbol::Type(Type::IntLiteral(2), Boundness::MayBeUnbound),
/// non_existent: Symbol::Unbound,
/// ```
#[derive(Debug, Clone, PartialEq)]
pub(crate) enum Symbol<'db> {
Type(Type<'db>, Boundness),
Unbound,
}
impl<'db> Symbol<'db> {
pub(crate) fn is_unbound(&self) -> bool {
matches!(self, Symbol::Unbound)
}
pub(crate) fn may_be_unbound(&self) -> bool {
match self {
Symbol::Type(_, Boundness::MayBeUnbound) | Symbol::Unbound => true,
Symbol::Type(_, Boundness::Bound) => false,
}
}
pub(crate) fn unwrap_or(&self, other: Type<'db>) -> Type<'db> {
match self {
Symbol::Type(ty, _) => *ty,
Symbol::Unbound => other,
}
}
pub(crate) fn unwrap_or_unknown(&self) -> Type<'db> {
self.unwrap_or(Type::Unknown)
}
pub(crate) fn as_type(&self) -> Option<Type<'db>> {
match self {
Symbol::Type(ty, _) => Some(*ty),
Symbol::Unbound => None,
}
}
#[cfg(test)]
#[track_caller]
pub(crate) fn expect_type(self) -> Type<'db> {
self.as_type()
.expect("Expected a (possibly unbound) type, not an unbound symbol")
}
#[must_use]
pub(crate) fn replace_unbound_with(
self,
db: &'db dyn Db,
replacement: &Symbol<'db>,
) -> Symbol<'db> {
match replacement {
Symbol::Type(replacement, _) => Symbol::Type(
match self {
Symbol::Type(ty, Boundness::Bound) => ty,
Symbol::Type(ty, Boundness::MayBeUnbound) => {
UnionType::from_elements(db, [*replacement, ty])
}
Symbol::Unbound => *replacement,
},
Boundness::Bound,
),
Symbol::Unbound => self,
}
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -25,12 +25,11 @@
//! * No type in an intersection can be a supertype of any other type in the intersection (just
//! eliminate the supertype from the intersection).
//! * An intersection containing two non-overlapping types should simplify to [`Type::Never`].
use crate::types::{IntersectionType, Type, UnionType};
use crate::types::{InstanceType, IntersectionType, KnownClass, Type, UnionType};
use crate::{Db, FxOrderSet};
use smallvec::SmallVec;
use super::KnownClass;
pub(crate) struct UnionBuilder<'db> {
elements: Vec<Type<'db>>,
db: &'db dyn Db,
@@ -80,7 +79,6 @@ impl<'db> UnionBuilder<'db> {
to_remove.push(index);
}
}
match to_remove[..] {
[] => self.elements.push(to_add),
[index] => self.elements[index] = to_add,
@@ -103,7 +101,6 @@ impl<'db> UnionBuilder<'db> {
}
}
}
self
}
@@ -173,14 +170,37 @@ impl<'db> IntersectionBuilder<'db> {
pub(crate) fn add_negative(mut self, ty: Type<'db>) -> Self {
// See comments above in `add_positive`; this is just the negated version.
if let Type::Union(union) = ty {
union
.elements(self.db)
for elem in union.elements(self.db) {
self = self.add_negative(*elem);
}
self
} else if let Type::Intersection(intersection) = ty {
// (A | B) & ~(C & ~D)
// -> (A | B) & (~C | D)
// -> ((A | B) & ~C) | ((A | B) & D)
// i.e. if we have an intersection of positive constraints C
// and negative constraints D, then our new intersection
// is (existing & ~C) | (existing & D)
let positive_side = intersection
.positive(self.db)
.iter()
.map(|elem| self.clone().add_negative(*elem))
.fold(IntersectionBuilder::empty(self.db), |mut builder, sub| {
// we negate all the positive constraints while distributing
.map(|elem| self.clone().add_negative(*elem));
let negative_side = intersection
.negative(self.db)
.iter()
// all negative constraints end up becoming positive constraints
.map(|elem| self.clone().add_positive(*elem));
positive_side.chain(negative_side).fold(
IntersectionBuilder::empty(self.db),
|mut builder, sub| {
builder.intersections.extend(sub.intersections);
builder
})
},
)
} else {
for inner in &mut self.intersections {
inner.add_negative(self.db, ty);
@@ -226,8 +246,8 @@ impl<'db> InnerIntersectionBuilder<'db> {
}
} else {
// ~Literal[True] & bool = Literal[False]
if let Type::Instance(class_type) = new_positive {
if class_type.is_known(db, KnownClass::Bool) {
if let Type::Instance(InstanceType { class, .. }) = new_positive {
if class.is_known(db, KnownClass::Bool) {
if let Some(&Type::BooleanLiteral(value)) = self
.negative
.iter()
@@ -293,7 +313,6 @@ impl<'db> InnerIntersectionBuilder<'db> {
self.add_positive(db, *neg);
}
}
Type::Unbound => {}
ty @ (Type::Any | Type::Unknown | Type::Todo) => {
// Adding any of these types to the negative side of an intersection
// is equivalent to adding it to the positive side. We do this to
@@ -344,15 +363,7 @@ impl<'db> InnerIntersectionBuilder<'db> {
}
}
fn simplify_unbound(&mut self) {
if self.positive.contains(&Type::Unbound) {
self.positive.retain(Type::is_unbound);
self.negative.clear();
}
}
fn build(mut self, db: &'db dyn Db) -> Type<'db> {
self.simplify_unbound();
match (self.positive.len(), self.negative.len()) {
(0, 0) => KnownClass::Object.to_instance(db),
(1, 0) => self.positive[0],
@@ -371,8 +382,10 @@ mod tests {
use crate::db::tests::TestDb;
use crate::program::{Program, SearchPathSettings};
use crate::python_version::PythonVersion;
use crate::types::{KnownClass, StringLiteralType, UnionBuilder};
use crate::stdlib::typing_symbol;
use crate::types::{global_symbol, KnownClass, StringLiteralType, UnionBuilder};
use crate::ProgramSettings;
use ruff_db::files::system_path_to_file;
use ruff_db::system::{DbWithTestSystem, SystemPathBuf};
use test_case::test_case;
@@ -561,18 +574,22 @@ mod tests {
let ta = Type::Any;
let t1 = Type::IntLiteral(1);
let t2 = KnownClass::Int.to_instance(&db);
// i0 = Any & ~Literal[1]
let i0 = IntersectionBuilder::new(&db)
.add_positive(ta)
.add_negative(t1)
.build();
let intersection = IntersectionBuilder::new(&db)
// ta_not_i0 = int & ~(Any & ~Literal[1])
// -> int & (~Any | Literal[1])
// (~Any is equivalent to Any)
// -> (int & Any) | (int & Literal[1])
// -> (int & Any) | Literal[1]
let ta_not_i0 = IntersectionBuilder::new(&db)
.add_positive(t2)
.add_negative(i0)
.build()
.expect_intersection();
.build();
assert_eq!(intersection.pos_vec(&db), &[ta, t1]);
assert_eq!(intersection.neg_vec(&db), &[]);
assert_eq!(ta_not_i0.display(&db).to_string(), "int & Any | Literal[1]");
}
#[test]
@@ -595,12 +612,69 @@ mod tests {
assert_eq!(i1.pos_vec(&db), &[ta, t1]);
}
#[test]
fn intersection_negation_distributes_over_union() {
let db = setup_db();
let st = typing_symbol(&db, "Sized").expect_type().to_instance(&db);
let ht = typing_symbol(&db, "Hashable")
.expect_type()
.to_instance(&db);
// sh_t: Sized & Hashable
let sh_t = IntersectionBuilder::new(&db)
.add_positive(st)
.add_positive(ht)
.build()
.expect_intersection();
assert_eq!(sh_t.pos_vec(&db), &[st, ht]);
assert_eq!(sh_t.neg_vec(&db), &[]);
// ~sh_t => ~Sized | ~Hashable
let not_s_h_t = IntersectionBuilder::new(&db)
.add_negative(Type::Intersection(sh_t))
.build()
.expect_union();
// should have as elements: (~Sized),(~Hashable)
let not_st = st.negate(&db);
let not_ht = ht.negate(&db);
assert_eq!(not_s_h_t.elements(&db), &[not_st, not_ht]);
}
#[test]
fn mixed_intersection_negation_distributes_over_union() {
let db = setup_db();
let it = KnownClass::Int.to_instance(&db);
let st = typing_symbol(&db, "Sized").expect_type().to_instance(&db);
let ht = typing_symbol(&db, "Hashable")
.expect_type()
.to_instance(&db);
// s_not_h_t: Sized & ~Hashable
let s_not_h_t = IntersectionBuilder::new(&db)
.add_positive(st)
.add_negative(ht)
.build()
.expect_intersection();
assert_eq!(s_not_h_t.pos_vec(&db), &[st]);
assert_eq!(s_not_h_t.neg_vec(&db), &[ht]);
// let's build int & ~(Sized & ~Hashable)
let tt = IntersectionBuilder::new(&db)
.add_positive(it)
.add_negative(Type::Intersection(s_not_h_t))
.build();
// int & ~(Sized & ~Hashable)
// -> int & (~Sized | Hashable)
// -> (int & ~Sized) | (int & Hashable)
assert_eq!(tt.display(&db).to_string(), "int & ~Sized | int & Hashable");
}
#[test]
fn build_intersection_self_negation() {
let db = setup_db();
let ty = IntersectionBuilder::new(&db)
.add_positive(Type::None)
.add_negative(Type::None)
.add_positive(Type::none(&db))
.add_negative(Type::none(&db))
.build();
assert_eq!(ty, Type::Never);
@@ -610,63 +684,62 @@ mod tests {
fn build_intersection_simplify_negative_never() {
let db = setup_db();
let ty = IntersectionBuilder::new(&db)
.add_positive(Type::None)
.add_positive(Type::none(&db))
.add_negative(Type::Never)
.build();
assert_eq!(ty, Type::None);
assert_eq!(ty, Type::none(&db));
}
#[test]
fn build_intersection_simplify_positive_never() {
let db = setup_db();
let ty = IntersectionBuilder::new(&db)
.add_positive(Type::None)
.add_positive(Type::none(&db))
.add_positive(Type::Never)
.build();
assert_eq!(ty, Type::Never);
}
#[test]
fn build_intersection_simplify_positive_unbound() {
let db = setup_db();
let ty = IntersectionBuilder::new(&db)
.add_positive(Type::Unbound)
.add_positive(Type::IntLiteral(1))
.build();
assert_eq!(ty, Type::Unbound);
}
#[test]
fn build_intersection_simplify_negative_unbound() {
let db = setup_db();
let ty = IntersectionBuilder::new(&db)
.add_negative(Type::Unbound)
.add_positive(Type::IntLiteral(1))
.build();
assert_eq!(ty, Type::IntLiteral(1));
}
#[test]
fn build_intersection_simplify_negative_none() {
let db = setup_db();
let ty = IntersectionBuilder::new(&db)
.add_negative(Type::None)
.add_negative(Type::none(&db))
.add_positive(Type::IntLiteral(1))
.build();
assert_eq!(ty, Type::IntLiteral(1));
let ty = IntersectionBuilder::new(&db)
.add_positive(Type::IntLiteral(1))
.add_negative(Type::None)
.add_negative(Type::none(&db))
.build();
assert_eq!(ty, Type::IntLiteral(1));
}
#[test]
fn build_negative_union_de_morgan() {
let db = setup_db();
let union = UnionBuilder::new(&db)
.add(Type::IntLiteral(1))
.add(Type::IntLiteral(2))
.build();
assert_eq!(union.display(&db).to_string(), "Literal[1, 2]");
let ty = IntersectionBuilder::new(&db).add_negative(union).build();
let expected = IntersectionBuilder::new(&db)
.add_negative(Type::IntLiteral(1))
.add_negative(Type::IntLiteral(2))
.build();
assert_eq!(ty.display(&db).to_string(), "~Literal[1] & ~Literal[2]");
assert_eq!(ty, expected);
}
#[test]
fn build_intersection_simplify_positive_type_and_positive_subtype() {
let db = setup_db();
@@ -800,7 +873,7 @@ mod tests {
let db = setup_db();
let t1 = Type::IntLiteral(1);
let t2 = Type::None;
let t2 = Type::none(&db);
let ty = IntersectionBuilder::new(&db)
.add_positive(t1)
@@ -918,4 +991,66 @@ mod tests {
.build();
assert_eq!(result, ty);
}
#[test]
fn build_intersection_of_two_unions_simplify() {
let mut db = setup_db();
db.write_dedented(
"/src/module.py",
"
class A: ...
class B: ...
a = A()
b = B()
",
)
.unwrap();
let file = system_path_to_file(&db, "src/module.py").expect("file to exist");
let a = global_symbol(&db, file, "a").expect_type();
let b = global_symbol(&db, file, "b").expect_type();
let union = UnionBuilder::new(&db).add(a).add(b).build();
assert_eq!(union.display(&db).to_string(), "A | B");
let reversed_union = UnionBuilder::new(&db).add(b).add(a).build();
assert_eq!(reversed_union.display(&db).to_string(), "B | A");
let intersection = IntersectionBuilder::new(&db)
.add_positive(union)
.add_positive(reversed_union)
.build();
assert_eq!(intersection.display(&db).to_string(), "B | A");
}
#[test]
fn build_union_of_two_intersections_simplify() {
let mut db = setup_db();
db.write_dedented(
"/src/module.py",
"
class A: ...
class B: ...
a = A()
b = B()
",
)
.unwrap();
let file = system_path_to_file(&db, "src/module.py").expect("file to exist");
let a = global_symbol(&db, file, "a").expect_type();
let b = global_symbol(&db, file, "b").expect_type();
let intersection = IntersectionBuilder::new(&db)
.add_positive(a)
.add_positive(b)
.build();
let reversed_intersection = IntersectionBuilder::new(&db)
.add_positive(b)
.add_positive(a)
.build();
let union = UnionBuilder::new(&db)
.add(intersection)
.add(reversed_intersection)
.build();
assert_eq!(union.display(&db).to_string(), "A & B");
}
}

View File

@@ -1,10 +1,14 @@
use ruff_db::files::File;
use ruff_python_ast::{self as ast, AnyNodeRef};
use ruff_text_size::{Ranged, TextRange};
use std::fmt::Formatter;
use std::ops::Deref;
use std::sync::Arc;
#[derive(Debug, Eq, PartialEq)]
use crate::types::{ClassLiteralType, Type};
use crate::Db;
#[derive(Debug, Eq, PartialEq, Clone)]
pub struct TypeCheckDiagnostic {
// TODO: Don't use string keys for rules
pub(super) rule: String,
@@ -109,3 +113,174 @@ impl<'a> IntoIterator for &'a TypeCheckDiagnostics {
self.inner.iter()
}
}
pub(super) struct TypeCheckDiagnosticsBuilder<'db> {
db: &'db dyn Db,
file: File,
diagnostics: TypeCheckDiagnostics,
}
impl<'db> TypeCheckDiagnosticsBuilder<'db> {
pub(super) fn new(db: &'db dyn Db, file: File) -> Self {
Self {
db,
file,
diagnostics: TypeCheckDiagnostics::new(),
}
}
/// Emit a diagnostic declaring that the object represented by `node` is not iterable
pub(super) fn add_not_iterable(&mut self, node: AnyNodeRef, not_iterable_ty: Type<'db>) {
self.add(
node,
"not-iterable",
format_args!(
"Object of type `{}` is not iterable",
not_iterable_ty.display(self.db)
),
);
}
/// Emit a diagnostic declaring that an index is out of bounds for a tuple.
pub(super) fn add_index_out_of_bounds(
&mut self,
kind: &'static str,
node: AnyNodeRef,
tuple_ty: Type<'db>,
length: usize,
index: i64,
) {
self.add(
node,
"index-out-of-bounds",
format_args!(
"Index {index} is out of bounds for {kind} `{}` with length {length}",
tuple_ty.display(self.db)
),
);
}
/// Emit a diagnostic declaring that a type does not support subscripting.
pub(super) fn add_non_subscriptable(
&mut self,
node: AnyNodeRef,
non_subscriptable_ty: Type<'db>,
method: &str,
) {
self.add(
node,
"non-subscriptable",
format_args!(
"Cannot subscript object of type `{}` with no `{method}` method",
non_subscriptable_ty.display(self.db)
),
);
}
pub(super) fn add_unresolved_module(
&mut self,
import_node: impl Into<AnyNodeRef<'db>>,
level: u32,
module: Option<&str>,
) {
self.add(
import_node.into(),
"unresolved-import",
format_args!(
"Cannot resolve import `{}{}`",
".".repeat(level as usize),
module.unwrap_or_default()
),
);
}
pub(super) fn add_slice_step_size_zero(&mut self, node: AnyNodeRef) {
self.add(
node,
"zero-stepsize-in-slice",
format_args!("Slice step size can not be zero"),
);
}
pub(super) fn add_invalid_assignment(
&mut self,
node: AnyNodeRef,
declared_ty: Type<'db>,
assigned_ty: Type<'db>,
) {
match declared_ty {
Type::ClassLiteral(ClassLiteralType { class }) => {
self.add(node, "invalid-assignment", format_args!(
"Implicit shadowing of class `{}`; annotate to make it explicit if this is intentional",
class.name(self.db)));
}
Type::FunctionLiteral(function) => {
self.add(node, "invalid-assignment", format_args!(
"Implicit shadowing of function `{}`; annotate to make it explicit if this is intentional",
function.name(self.db)));
}
_ => {
self.add(
node,
"invalid-assignment",
format_args!(
"Object of type `{}` is not assignable to `{}`",
assigned_ty.display(self.db),
declared_ty.display(self.db),
),
);
}
}
}
pub(super) fn add_possibly_unresolved_reference(&mut self, expr_name_node: &ast::ExprName) {
let ast::ExprName { id, .. } = expr_name_node;
self.add(
expr_name_node.into(),
"possibly-unresolved-reference",
format_args!("Name `{id}` used when possibly not defined"),
);
}
pub(super) fn add_unresolved_reference(&mut self, expr_name_node: &ast::ExprName) {
let ast::ExprName { id, .. } = expr_name_node;
self.add(
expr_name_node.into(),
"unresolved-reference",
format_args!("Name `{id}` used when not defined"),
);
}
/// Adds a new diagnostic.
///
/// The diagnostic does not get added if the rule isn't enabled for this file.
pub(super) fn add(&mut self, node: AnyNodeRef, rule: &str, message: std::fmt::Arguments) {
if !self.db.is_file_open(self.file) {
return;
}
// TODO: Don't emit the diagnostic if:
// * The enclosing node contains any syntax errors
// * The rule is disabled for this file. We probably want to introduce a new query that
// returns a rule selector for a given file that respects the package's settings,
// any global pragma comments in the file, and any per-file-ignores.
self.diagnostics.push(TypeCheckDiagnostic {
file: self.file,
rule: rule.to_string(),
message: message.to_string(),
range: node.range(),
});
}
pub(super) fn extend(&mut self, diagnostics: &TypeCheckDiagnostics) {
self.diagnostics.extend(diagnostics);
}
pub(super) fn finish(mut self) -> TypeCheckDiagnostics {
self.diagnostics.shrink_to_fit();
self.diagnostics
}
}

View File

@@ -6,7 +6,7 @@ use ruff_db::display::FormatterJoinExtension;
use ruff_python_ast::str::Quote;
use ruff_python_literal::escape::AsciiEscape;
use crate::types::{IntersectionType, Type, UnionType};
use crate::types::{ClassLiteralType, InstanceType, IntersectionType, KnownClass, Type, UnionType};
use crate::Db;
use rustc_hash::FxHashMap;
@@ -64,8 +64,11 @@ impl Display for DisplayRepresentation<'_> {
Type::Any => f.write_str("Any"),
Type::Never => f.write_str("Never"),
Type::Unknown => f.write_str("Unknown"),
Type::Unbound => f.write_str("Unbound"),
Type::None => f.write_str("None"),
Type::Instance(InstanceType { class, .. })
if class.is_known(self.db, KnownClass::NoneType) =>
{
f.write_str("None")
}
// `[Type::Todo]`'s display should be explicit that is not a valid display of
// any other type
Type::Todo => f.write_str("@Todo"),
@@ -73,8 +76,11 @@ impl Display for DisplayRepresentation<'_> {
write!(f, "<module '{:?}'>", file.path(self.db))
}
// TODO functions and classes should display using a fully qualified name
Type::ClassLiteral(class) => f.write_str(class.name(self.db)),
Type::Instance(class) => f.write_str(class.name(self.db)),
Type::ClassLiteral(ClassLiteralType { class }) => f.write_str(class.name(self.db)),
Type::Instance(InstanceType { class, known }) => f.write_str(match known {
Some(super::KnownInstance::Literal) => "Literal",
_ => class.name(self.db),
}),
Type::FunctionLiteral(function) => f.write_str(function.name(self.db)),
Type::Union(union) => union.display(self.db).fmt(f),
Type::Intersection(intersection) => intersection.display(self.db).fmt(f),
@@ -90,6 +96,28 @@ impl Display for DisplayRepresentation<'_> {
escape.bytes_repr().write(f)
}
Type::SliceLiteral(slice) => {
f.write_str("slice[")?;
if let Some(start) = slice.start(self.db) {
write!(f, "Literal[{start}]")?;
} else {
f.write_str("None")?;
}
f.write_str(", ")?;
if let Some(stop) = slice.stop(self.db) {
write!(f, "Literal[{stop}]")?;
} else {
f.write_str("None")?;
}
if let Some(step) = slice.step(self.db) {
write!(f, ", Literal[{step}]")?;
}
f.write_str("]")
}
Type::Tuple(tuple) => {
f.write_str("tuple[")?;
let elements = tuple.elements(self.db);
@@ -301,7 +329,9 @@ mod tests {
use ruff_db::system::{DbWithTestSystem, SystemPathBuf};
use crate::db::tests::TestDb;
use crate::types::{global_symbol_ty, BytesLiteralType, StringLiteralType, Type, UnionType};
use crate::types::{
global_symbol, BytesLiteralType, SliceLiteralType, StringLiteralType, Type, UnionType,
};
use crate::{Program, ProgramSettings, PythonVersion, SearchPathSettings};
fn setup_db() -> TestDb {
@@ -346,18 +376,18 @@ mod tests {
let union_elements = &[
Type::Unknown,
Type::IntLiteral(-1),
global_symbol_ty(&db, mod_file, "A"),
global_symbol(&db, mod_file, "A").expect_type(),
Type::StringLiteral(StringLiteralType::new(&db, "A")),
Type::BytesLiteral(BytesLiteralType::new(&db, [0u8].as_slice())),
Type::BytesLiteral(BytesLiteralType::new(&db, [7u8].as_slice())),
Type::IntLiteral(0),
Type::IntLiteral(1),
Type::StringLiteral(StringLiteralType::new(&db, "B")),
global_symbol_ty(&db, mod_file, "foo"),
global_symbol_ty(&db, mod_file, "bar"),
global_symbol_ty(&db, mod_file, "B"),
global_symbol(&db, mod_file, "foo").expect_type(),
global_symbol(&db, mod_file, "bar").expect_type(),
global_symbol(&db, mod_file, "B").expect_type(),
Type::BooleanLiteral(true),
Type::None,
Type::none(&db),
];
let union = UnionType::from_elements(&db, union_elements).expect_union();
let display = format!("{}", union.display(&db));
@@ -376,4 +406,46 @@ mod tests {
);
Ok(())
}
#[test]
fn test_slice_literal_display() {
let db = setup_db();
assert_eq!(
Type::SliceLiteral(SliceLiteralType::new(&db, None, None, None))
.display(&db)
.to_string(),
"slice[None, None]"
);
assert_eq!(
Type::SliceLiteral(SliceLiteralType::new(&db, Some(1), None, None))
.display(&db)
.to_string(),
"slice[Literal[1], None]"
);
assert_eq!(
Type::SliceLiteral(SliceLiteralType::new(&db, None, Some(2), None))
.display(&db)
.to_string(),
"slice[None, Literal[2]]"
);
assert_eq!(
Type::SliceLiteral(SliceLiteralType::new(&db, Some(1), Some(5), None))
.display(&db)
.to_string(),
"slice[Literal[1], Literal[5]]"
);
assert_eq!(
Type::SliceLiteral(SliceLiteralType::new(&db, Some(1), Some(5), Some(2)))
.display(&db)
.to_string(),
"slice[Literal[1], Literal[5], Literal[2]]"
);
assert_eq!(
Type::SliceLiteral(SliceLiteralType::new(&db, None, None, Some(2)))
.display(&db)
.to_string(),
"slice[None, None, Literal[2]]"
);
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,501 @@
use std::collections::VecDeque;
use std::ops::Deref;
use indexmap::IndexSet;
use itertools::Either;
use rustc_hash::FxHashSet;
use super::{Class, ClassLiteralType, KnownClass, Type};
use crate::Db;
/// The inferred method resolution order of a given class.
///
/// See [`Class::iter_mro`] for more details.
#[derive(PartialEq, Eq, Clone, Debug)]
pub(super) struct Mro<'db>(Box<[ClassBase<'db>]>);
impl<'db> Mro<'db> {
/// Attempt to resolve the MRO of a given class
///
/// In the event that a possible list of bases would (or could) lead to a
/// `TypeError` being raised at runtime due to an unresolvable MRO, we infer
/// the MRO of the class as being `[<the class in question>, Unknown, object]`.
/// This seems most likely to reduce the possibility of cascading errors
/// elsewhere.
///
/// (We emit a diagnostic warning about the runtime `TypeError` in
/// [`super::infer::TypeInferenceBuilder::infer_region_scope`].)
pub(super) fn of_class(db: &'db dyn Db, class: Class<'db>) -> Result<Self, MroError<'db>> {
Self::of_class_impl(db, class).map_err(|error_kind| {
let fallback_mro = Self::from([
ClassBase::Class(class),
ClassBase::Unknown,
ClassBase::object(db),
]);
MroError {
kind: error_kind,
fallback_mro,
}
})
}
fn of_class_impl(db: &'db dyn Db, class: Class<'db>) -> Result<Self, MroErrorKind<'db>> {
let class_bases = class.explicit_bases(db);
match class_bases {
// `builtins.object` is the special case:
// the only class in Python that has an MRO with length <2
[] if class.is_known(db, KnownClass::Object) => {
Ok(Self::from([ClassBase::Class(class)]))
}
// All other classes in Python have an MRO with length >=2.
// Even if a class has no explicit base classes,
// it will implicitly inherit from `object` at runtime;
// `object` will appear in the class's `__bases__` list and `__mro__`:
//
// ```pycon
// >>> class Foo: ...
// ...
// >>> Foo.__bases__
// (<class 'object'>,)
// >>> Foo.__mro__
// (<class '__main__.Foo'>, <class 'object'>)
// ```
[] => Ok(Self::from([ClassBase::Class(class), ClassBase::object(db)])),
// Fast path for a class that has only a single explicit base.
//
// This *could* theoretically be handled by the final branch below,
// but it's a common case (i.e., worth optimizing for),
// and the `c3_merge` function requires lots of allocations.
[single_base] => {
let single_base = ClassBase::try_from_ty(*single_base).ok_or(*single_base);
single_base.map_or_else(
|invalid_base_ty| {
let bases_info = Box::from([(0, invalid_base_ty)]);
Err(MroErrorKind::InvalidBases(bases_info))
},
|single_base| {
if let ClassBase::Class(class_base) = single_base {
if class_is_cyclically_defined(db, class_base) {
return Err(MroErrorKind::CyclicClassDefinition);
}
}
let mro = std::iter::once(ClassBase::Class(class))
.chain(single_base.mro(db))
.collect();
Ok(mro)
},
)
}
// The class has multiple explicit bases.
//
// We'll fallback to a full implementation of the C3-merge algorithm to determine
// what MRO Python will give this class at runtime
// (if an MRO is indeed resolvable at all!)
multiple_bases => {
if class_is_cyclically_defined(db, class) {
return Err(MroErrorKind::CyclicClassDefinition);
}
let mut valid_bases = vec![];
let mut invalid_bases = vec![];
for (i, base) in multiple_bases.iter().enumerate() {
match ClassBase::try_from_ty(*base).ok_or(*base) {
Ok(valid_base) => valid_bases.push(valid_base),
Err(invalid_base) => invalid_bases.push((i, invalid_base)),
}
}
if !invalid_bases.is_empty() {
return Err(MroErrorKind::InvalidBases(invalid_bases.into_boxed_slice()));
}
let mut seqs = vec![VecDeque::from([ClassBase::Class(class)])];
for base in &valid_bases {
seqs.push(base.mro(db).collect());
}
seqs.push(valid_bases.iter().copied().collect());
c3_merge(seqs).ok_or_else(|| {
let mut seen_bases = FxHashSet::default();
let mut duplicate_bases = vec![];
for (index, base) in valid_bases
.iter()
.enumerate()
.filter_map(|(index, base)| Some((index, base.into_class_literal_type()?)))
{
if !seen_bases.insert(base) {
duplicate_bases.push((index, base));
}
}
if duplicate_bases.is_empty() {
MroErrorKind::UnresolvableMro {
bases_list: valid_bases.into_boxed_slice(),
}
} else {
MroErrorKind::DuplicateBases(duplicate_bases.into_boxed_slice())
}
})
}
}
}
}
impl<'db, const N: usize> From<[ClassBase<'db>; N]> for Mro<'db> {
fn from(value: [ClassBase<'db>; N]) -> Self {
Self(Box::from(value))
}
}
impl<'db> From<Vec<ClassBase<'db>>> for Mro<'db> {
fn from(value: Vec<ClassBase<'db>>) -> Self {
Self(value.into_boxed_slice())
}
}
impl<'db> Deref for Mro<'db> {
type Target = [ClassBase<'db>];
fn deref(&self) -> &Self::Target {
&self.0
}
}
impl<'db> FromIterator<ClassBase<'db>> for Mro<'db> {
fn from_iter<T: IntoIterator<Item = ClassBase<'db>>>(iter: T) -> Self {
Self(iter.into_iter().collect())
}
}
/// Iterator that yields elements of a class's MRO.
///
/// We avoid materialising the *full* MRO unless it is actually necessary:
/// - Materialising the full MRO is expensive
/// - We need to do it for every class in the code that we're checking, as we need to make sure
/// that there are no class definitions in the code we're checking that would cause an
/// exception to be raised at runtime. But the same does *not* necessarily apply for every class
/// in third-party and stdlib dependencies: we never emit diagnostics about non-first-party code.
/// - However, we *do* need to resolve attribute accesses on classes/instances from
/// third-party and stdlib dependencies. That requires iterating over the MRO of third-party/stdlib
/// classes, but not necessarily the *whole* MRO: often just the first element is enough.
/// Luckily we know that for any class `X`, the first element of `X`'s MRO will always be `X` itself.
/// We can therefore avoid resolving the full MRO for many third-party/stdlib classes while still
/// being faithful to the runtime semantics.
///
/// Even for first-party code, where we will have to resolve the MRO for every class we encounter,
/// loading the cached MRO comes with a certain amount of overhead, so it's best to avoid calling the
/// Salsa-tracked [`Class::try_mro`] method unless it's absolutely necessary.
pub(super) struct MroIterator<'db> {
db: &'db dyn Db,
/// The class whose MRO we're iterating over
class: Class<'db>,
/// Whether or not we've already yielded the first element of the MRO
first_element_yielded: bool,
/// Iterator over all elements of the MRO except the first.
///
/// The full MRO is expensive to materialize, so this field is `None`
/// unless we actually *need* to iterate past the first element of the MRO,
/// at which point it is lazily materialized.
subsequent_elements: Option<std::slice::Iter<'db, ClassBase<'db>>>,
}
impl<'db> MroIterator<'db> {
pub(super) fn new(db: &'db dyn Db, class: Class<'db>) -> Self {
Self {
db,
class,
first_element_yielded: false,
subsequent_elements: None,
}
}
/// Materialize the full MRO of the class.
/// Return an iterator over that MRO which skips the first element of the MRO.
fn full_mro_except_first_element(&mut self) -> impl Iterator<Item = ClassBase<'db>> + '_ {
self.subsequent_elements
.get_or_insert_with(|| {
let mut full_mro_iter = match self.class.try_mro(self.db) {
Ok(mro) => mro.iter(),
Err(error) => error.fallback_mro().iter(),
};
full_mro_iter.next();
full_mro_iter
})
.copied()
}
}
impl<'db> Iterator for MroIterator<'db> {
type Item = ClassBase<'db>;
fn next(&mut self) -> Option<Self::Item> {
if !self.first_element_yielded {
self.first_element_yielded = true;
return Some(ClassBase::Class(self.class));
}
self.full_mro_except_first_element().next()
}
}
impl std::iter::FusedIterator for MroIterator<'_> {}
#[derive(Debug, PartialEq, Eq)]
pub(super) struct MroError<'db> {
kind: MroErrorKind<'db>,
fallback_mro: Mro<'db>,
}
impl<'db> MroError<'db> {
/// Return an [`MroErrorKind`] variant describing why we could not resolve the MRO for this class.
pub(super) fn reason(&self) -> &MroErrorKind<'db> {
&self.kind
}
/// Return the fallback MRO we should infer for this class during type inference
/// (since accurate resolution of its "true" MRO was impossible)
pub(super) fn fallback_mro(&self) -> &Mro<'db> {
&self.fallback_mro
}
}
/// Possible ways in which attempting to resolve the MRO of a class might fail.
#[derive(Debug, PartialEq, Eq)]
pub(super) enum MroErrorKind<'db> {
/// The class inherits from one or more invalid bases.
///
/// To avoid excessive complexity in our implementation,
/// we only permit classes to inherit from class-literal types,
/// `Todo`, `Unknown` or `Any`. Anything else results in us
/// emitting a diagnostic.
///
/// This variant records the indices and types of class bases
/// that we deem to be invalid. The indices are the indices of nodes
/// in the bases list of the class's [`StmtClassDef`](ruff_python_ast::StmtClassDef) node.
/// Each index is the index of a node representing an invalid base.
InvalidBases(Box<[(usize, Type<'db>)]>),
/// The class inherits from itself!
///
/// This is very unlikely to happen in working real-world code,
/// but it's important to explicitly account for it.
/// If we don't, there's a possibility of an infinite loop and a panic.
CyclicClassDefinition,
/// The class has one or more duplicate bases.
///
/// This variant records the indices and [`Class`]es
/// of the duplicate bases. The indices are the indices of nodes
/// in the bases list of the class's [`StmtClassDef`](ruff_python_ast::StmtClassDef) node.
/// Each index is the index of a node representing a duplicate base.
DuplicateBases(Box<[(usize, Class<'db>)]>),
/// The MRO is otherwise unresolvable through the C3-merge algorithm.
///
/// See [`c3_merge`] for more details.
UnresolvableMro { bases_list: Box<[ClassBase<'db>]> },
}
/// Enumeration of the possible kinds of types we allow in class bases.
///
/// This is much more limited than the [`Type`] enum:
/// all types that would be invalid to have as a class base are
/// transformed into [`ClassBase::Unknown`]
#[derive(Debug, Copy, Clone, PartialEq, Eq)]
pub(super) enum ClassBase<'db> {
Any,
Unknown,
Todo,
Class(Class<'db>),
}
impl<'db> ClassBase<'db> {
pub(super) fn display(self, db: &'db dyn Db) -> impl std::fmt::Display + 'db {
struct Display<'db> {
base: ClassBase<'db>,
db: &'db dyn Db,
}
impl std::fmt::Display for Display<'_> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self.base {
ClassBase::Any => f.write_str("Any"),
ClassBase::Todo => f.write_str("Todo"),
ClassBase::Unknown => f.write_str("Unknown"),
ClassBase::Class(class) => write!(f, "<class '{}'>", class.name(self.db)),
}
}
}
Display { base: self, db }
}
#[cfg(test)]
#[track_caller]
pub(super) fn expect_class_base(self) -> Class<'db> {
match self {
ClassBase::Class(class) => class,
_ => panic!("Expected a `ClassBase::Class()` variant"),
}
}
/// Return a `ClassBase` representing the class `builtins.object`
fn object(db: &'db dyn Db) -> Self {
KnownClass::Object
.to_class(db)
.into_class_literal()
.map_or(Self::Unknown, |ClassLiteralType { class }| {
Self::Class(class)
})
}
/// Attempt to resolve `ty` into a `ClassBase`.
///
/// Return `None` if `ty` is not an acceptable type for a class base.
fn try_from_ty(ty: Type<'db>) -> Option<Self> {
match ty {
Type::Any => Some(Self::Any),
Type::Unknown => Some(Self::Unknown),
Type::Todo => Some(Self::Todo),
Type::ClassLiteral(ClassLiteralType { class }) => Some(Self::Class(class)),
Type::Union(_) => None, // TODO -- forces consideration of multiple possible MROs?
Type::Intersection(_) => None, // TODO -- probably incorrect?
Type::Instance(_) => None, // TODO -- handle `__mro_entries__`?
Type::Never
| Type::BooleanLiteral(_)
| Type::FunctionLiteral(_)
| Type::BytesLiteral(_)
| Type::IntLiteral(_)
| Type::StringLiteral(_)
| Type::LiteralString
| Type::Tuple(_)
| Type::SliceLiteral(_)
| Type::ModuleLiteral(_) => None,
}
}
fn into_class_literal_type(self) -> Option<Class<'db>> {
match self {
Self::Class(class) => Some(class),
_ => None,
}
}
/// Iterate over the MRO of this base
fn mro(
self,
db: &'db dyn Db,
) -> Either<impl Iterator<Item = ClassBase<'db>>, impl Iterator<Item = ClassBase<'db>>> {
match self {
ClassBase::Any => Either::Left([ClassBase::Any, ClassBase::object(db)].into_iter()),
ClassBase::Unknown => {
Either::Left([ClassBase::Unknown, ClassBase::object(db)].into_iter())
}
ClassBase::Todo => Either::Left([ClassBase::Todo, ClassBase::object(db)].into_iter()),
ClassBase::Class(class) => Either::Right(class.iter_mro(db)),
}
}
}
impl<'db> From<ClassBase<'db>> for Type<'db> {
fn from(value: ClassBase<'db>) -> Self {
match value {
ClassBase::Any => Type::Any,
ClassBase::Todo => Type::Todo,
ClassBase::Unknown => Type::Unknown,
ClassBase::Class(class) => Type::ClassLiteral(ClassLiteralType { class }),
}
}
}
/// Implementation of the [C3-merge algorithm] for calculating a Python class's
/// [method resolution order].
///
/// [C3-merge algorithm]: https://docs.python.org/3/howto/mro.html#python-2-3-mro
/// [method resolution order]: https://docs.python.org/3/glossary.html#term-method-resolution-order
fn c3_merge(mut sequences: Vec<VecDeque<ClassBase>>) -> Option<Mro> {
// Most MROs aren't that long...
let mut mro = Vec::with_capacity(8);
loop {
sequences.retain(|sequence| !sequence.is_empty());
if sequences.is_empty() {
return Some(Mro::from(mro));
}
// If the candidate exists "deeper down" in the inheritance hierarchy,
// we should refrain from adding it to the MRO for now. Add the first candidate
// for which this does not hold true. If this holds true for all candidates,
// return `None`; it will be impossible to find a consistent MRO for the class
// with the given bases.
let mro_entry = sequences.iter().find_map(|outer_sequence| {
let candidate = outer_sequence[0];
let not_head = sequences
.iter()
.all(|sequence| sequence.iter().skip(1).all(|base| base != &candidate));
not_head.then_some(candidate)
})?;
mro.push(mro_entry);
// Make sure we don't try to add the candidate to the MRO twice:
for sequence in &mut sequences {
if sequence[0] == mro_entry {
sequence.pop_front();
}
}
}
}
/// Return `true` if this class appears to be a cyclic definition,
/// i.e., it inherits either directly or indirectly from itself.
///
/// A class definition like this will fail at runtime,
/// but we must be resilient to it or we could panic.
fn class_is_cyclically_defined(db: &dyn Db, class: Class) -> bool {
fn is_cyclically_defined_recursive<'db>(
db: &'db dyn Db,
class: Class<'db>,
classes_to_watch: &mut IndexSet<Class<'db>>,
) -> bool {
if !classes_to_watch.insert(class) {
return true;
}
for explicit_base_class in class
.explicit_bases(db)
.iter()
.copied()
.filter_map(Type::into_class_literal)
.map(|ClassLiteralType { class }| class)
{
// Each base must be considered in isolation.
// This is due to the fact that if a class uses multiple inheritance,
// there could easily be a situation where two bases have the same class in their MROs;
// that isn't enough to constitute the class being cyclically defined.
let classes_to_watch_len = classes_to_watch.len();
if is_cyclically_defined_recursive(db, explicit_base_class, classes_to_watch) {
return true;
}
classes_to_watch.truncate(classes_to_watch_len);
}
false
}
class
.explicit_bases(db)
.iter()
.copied()
.filter_map(Type::into_class_literal)
.map(|ClassLiteralType { class }| class)
.any(|base_class| is_cyclically_defined_recursive(db, base_class, &mut IndexSet::default()))
}

View File

@@ -1,16 +1,19 @@
use crate::semantic_index::ast_ids::HasScopedAstId;
use crate::semantic_index::constraint::{Constraint, PatternConstraint};
use crate::semantic_index::constraint::{Constraint, ConstraintNode, PatternConstraint};
use crate::semantic_index::definition::Definition;
use crate::semantic_index::expression::Expression;
use crate::semantic_index::symbol::{ScopeId, ScopedSymbolId, SymbolTable};
use crate::semantic_index::symbol_table;
use crate::types::{
infer_expression_types, IntersectionBuilder, KnownFunction, Type, UnionBuilder,
infer_expression_types, ClassLiteralType, IntersectionBuilder, KnownClass, KnownFunction,
Truthiness, Type, UnionBuilder,
};
use crate::Db;
use itertools::Itertools;
use ruff_python_ast as ast;
use ruff_python_ast::{BoolOp, ExprBoolOp};
use rustc_hash::FxHashMap;
use std::collections::hash_map::Entry;
use std::sync::Arc;
/// Return the type constraint that `test` (if true) would place on `definition`, if any.
@@ -34,15 +37,20 @@ pub(crate) fn narrowing_constraint<'db>(
constraint: Constraint<'db>,
definition: Definition<'db>,
) -> Option<Type<'db>> {
match constraint {
Constraint::Expression(expression) => {
all_narrowing_constraints_for_expression(db, expression)
.get(&definition.symbol(db))
.copied()
let constraints = match constraint.node {
ConstraintNode::Expression(expression) => {
if constraint.is_positive {
all_narrowing_constraints_for_expression(db, expression)
} else {
all_negative_narrowing_constraints_for_expression(db, expression)
}
}
Constraint::Pattern(pattern) => all_narrowing_constraints_for_pattern(db, pattern)
.get(&definition.symbol(db))
.copied(),
ConstraintNode::Pattern(pattern) => all_narrowing_constraints_for_pattern(db, pattern),
};
if let Some(constraints) = constraints {
constraints.get(&definition.symbol(db)).copied()
} else {
None
}
}
@@ -50,16 +58,24 @@ pub(crate) fn narrowing_constraint<'db>(
fn all_narrowing_constraints_for_pattern<'db>(
db: &'db dyn Db,
pattern: PatternConstraint<'db>,
) -> NarrowingConstraints<'db> {
NarrowingConstraintsBuilder::new(db, Constraint::Pattern(pattern)).finish()
) -> Option<NarrowingConstraints<'db>> {
NarrowingConstraintsBuilder::new(db, ConstraintNode::Pattern(pattern), true).finish()
}
#[salsa::tracked(return_ref)]
fn all_narrowing_constraints_for_expression<'db>(
db: &'db dyn Db,
expression: Expression<'db>,
) -> NarrowingConstraints<'db> {
NarrowingConstraintsBuilder::new(db, Constraint::Expression(expression)).finish()
) -> Option<NarrowingConstraints<'db>> {
NarrowingConstraintsBuilder::new(db, ConstraintNode::Expression(expression), true).finish()
}
#[salsa::tracked(return_ref)]
fn all_negative_narrowing_constraints_for_expression<'db>(
db: &'db dyn Db,
expression: Expression<'db>,
) -> Option<NarrowingConstraints<'db>> {
NarrowingConstraintsBuilder::new(db, ConstraintNode::Expression(expression), false).finish()
}
/// Generate a constraint from the *type* of the second argument of an `isinstance` call.
@@ -72,7 +88,7 @@ fn generate_isinstance_constraint<'db>(
classinfo: &Type<'db>,
) -> Option<Type<'db>> {
match classinfo {
Type::ClassLiteral(class) => Some(Type::Instance(*class)),
Type::ClassLiteral(ClassLiteralType { class }) => Some(Type::anonymous_instance(*class)),
Type::Tuple(tuple) => {
let mut builder = UnionBuilder::new(db);
for element in tuple.elements(db) {
@@ -86,70 +102,137 @@ fn generate_isinstance_constraint<'db>(
type NarrowingConstraints<'db> = FxHashMap<ScopedSymbolId, Type<'db>>;
fn merge_constraints_and<'db>(
into: &mut NarrowingConstraints<'db>,
from: NarrowingConstraints<'db>,
db: &'db dyn Db,
) {
for (key, value) in from {
match into.entry(key) {
Entry::Occupied(mut entry) => {
*entry.get_mut() = IntersectionBuilder::new(db)
.add_positive(*entry.get())
.add_positive(value)
.build();
}
Entry::Vacant(entry) => {
entry.insert(value);
}
}
}
}
fn merge_constraints_or<'db>(
into: &mut NarrowingConstraints<'db>,
from: &NarrowingConstraints<'db>,
db: &'db dyn Db,
) {
for (key, value) in from {
match into.entry(*key) {
Entry::Occupied(mut entry) => {
*entry.get_mut() = UnionBuilder::new(db).add(*entry.get()).add(*value).build();
}
Entry::Vacant(entry) => {
entry.insert(KnownClass::Object.to_instance(db));
}
}
}
for (key, value) in into.iter_mut() {
if !from.contains_key(key) {
*value = KnownClass::Object.to_instance(db);
}
}
}
struct NarrowingConstraintsBuilder<'db> {
db: &'db dyn Db,
constraint: Constraint<'db>,
constraints: NarrowingConstraints<'db>,
constraint: ConstraintNode<'db>,
is_positive: bool,
}
impl<'db> NarrowingConstraintsBuilder<'db> {
fn new(db: &'db dyn Db, constraint: Constraint<'db>) -> Self {
fn new(db: &'db dyn Db, constraint: ConstraintNode<'db>, is_positive: bool) -> Self {
Self {
db,
constraint,
constraints: NarrowingConstraints::default(),
is_positive,
}
}
fn finish(mut self) -> NarrowingConstraints<'db> {
match self.constraint {
Constraint::Expression(expression) => self.evaluate_expression_constraint(expression),
Constraint::Pattern(pattern) => self.evaluate_pattern_constraint(pattern),
fn finish(mut self) -> Option<NarrowingConstraints<'db>> {
let constraints: Option<NarrowingConstraints<'db>> = match self.constraint {
ConstraintNode::Expression(expression) => {
self.evaluate_expression_constraint(expression, self.is_positive)
}
ConstraintNode::Pattern(pattern) => self.evaluate_pattern_constraint(pattern),
};
if let Some(mut constraints) = constraints {
constraints.shrink_to_fit();
Some(constraints)
} else {
None
}
self.constraints.shrink_to_fit();
self.constraints
}
fn evaluate_expression_constraint(&mut self, expression: Expression<'db>) {
match expression.node_ref(self.db).node() {
fn evaluate_expression_constraint(
&mut self,
expression: Expression<'db>,
is_positive: bool,
) -> Option<NarrowingConstraints<'db>> {
let expression_node = expression.node_ref(self.db).node();
self.evaluate_expression_node_constraint(expression_node, expression, is_positive)
}
fn evaluate_expression_node_constraint(
&mut self,
expression_node: &ruff_python_ast::Expr,
expression: Expression<'db>,
is_positive: bool,
) -> Option<NarrowingConstraints<'db>> {
match expression_node {
ast::Expr::Compare(expr_compare) => {
self.add_expr_compare(expr_compare, expression);
self.evaluate_expr_compare(expr_compare, expression, is_positive)
}
ast::Expr::Call(expr_call) => {
self.add_expr_call(expr_call, expression);
self.evaluate_expr_call(expr_call, expression, is_positive)
}
_ => {} // TODO other test expression kinds
ast::Expr::UnaryOp(unary_op) if unary_op.op == ast::UnaryOp::Not => self
.evaluate_expression_node_constraint(&unary_op.operand, expression, !is_positive),
ast::Expr::BoolOp(bool_op) => self.evaluate_bool_op(bool_op, expression, is_positive),
_ => None, // TODO other test expression kinds
}
}
fn evaluate_pattern_constraint(&mut self, pattern: PatternConstraint<'db>) {
fn evaluate_pattern_constraint(
&mut self,
pattern: PatternConstraint<'db>,
) -> Option<NarrowingConstraints<'db>> {
let subject = pattern.subject(self.db);
match pattern.pattern(self.db).node() {
ast::Pattern::MatchValue(_) => {
// TODO
None // TODO
}
ast::Pattern::MatchSingleton(singleton_pattern) => {
self.add_match_pattern_singleton(subject, singleton_pattern);
self.evaluate_match_pattern_singleton(subject, singleton_pattern)
}
ast::Pattern::MatchSequence(_) => {
// TODO
None // TODO
}
ast::Pattern::MatchMapping(_) => {
// TODO
None // TODO
}
ast::Pattern::MatchClass(_) => {
// TODO
None // TODO
}
ast::Pattern::MatchStar(_) => {
// TODO
None // TODO
}
ast::Pattern::MatchAs(_) => {
// TODO
None // TODO
}
ast::Pattern::MatchOr(_) => {
// TODO
None // TODO
}
}
}
@@ -160,12 +243,17 @@ impl<'db> NarrowingConstraintsBuilder<'db> {
fn scope(&self) -> ScopeId<'db> {
match self.constraint {
Constraint::Expression(expression) => expression.scope(self.db),
Constraint::Pattern(pattern) => pattern.scope(self.db),
ConstraintNode::Expression(expression) => expression.scope(self.db),
ConstraintNode::Pattern(pattern) => pattern.scope(self.db),
}
}
fn add_expr_compare(&mut self, expr_compare: &ast::ExprCompare, expression: Expression<'db>) {
fn evaluate_expr_compare(
&mut self,
expr_compare: &ast::ExprCompare,
expression: Expression<'db>,
is_positive: bool,
) -> Option<NarrowingConstraints<'db>> {
let ast::ExprCompare {
range: _,
left,
@@ -175,7 +263,14 @@ impl<'db> NarrowingConstraintsBuilder<'db> {
if !left.is_name_expr() && comparators.iter().all(|c| !c.is_name_expr()) {
// If none of the comparators are name expressions,
// we have no symbol to narrow down the type of.
return;
return None;
}
if !is_positive && comparators.len() > 1 {
// We can't negate a constraint made by a multi-comparator expression, since we can't
// know which comparison part is the one being negated.
// For example, the negation of `x is 1 is y is 2`, would be `(x is not 1) or (y is not 1) or (y is not 2)`
// and that requires cross-symbol constraints, which we don't support yet.
return None;
}
let scope = self.scope();
let inference = infer_expression_types(self.db, expression);
@@ -183,6 +278,7 @@ impl<'db> NarrowingConstraintsBuilder<'db> {
let comparator_tuples = std::iter::once(&**left)
.chain(comparators)
.tuple_windows::<(&ruff_python_ast::Expr, &ruff_python_ast::Expr)>();
let mut constraints = NarrowingConstraints::default();
for (op, (left, right)) in std::iter::zip(&**ops, comparator_tuples) {
if let ast::Expr::Name(ast::ExprName {
range: _,
@@ -192,27 +288,28 @@ impl<'db> NarrowingConstraintsBuilder<'db> {
{
// SAFETY: we should always have a symbol for every Name node.
let symbol = self.symbols().symbol_id_by_name(id).unwrap();
let comp_ty = inference.expression_ty(right.scoped_ast_id(self.db, scope));
match op {
let rhs_ty = inference.expression_ty(right.scoped_ast_id(self.db, scope));
match if is_positive { *op } else { op.negate() } {
ast::CmpOp::IsNot => {
if comp_ty.is_singleton() {
if rhs_ty.is_singleton(self.db) {
let ty = IntersectionBuilder::new(self.db)
.add_negative(comp_ty)
.add_negative(rhs_ty)
.build();
self.constraints.insert(symbol, ty);
constraints.insert(symbol, ty);
} else {
// Non-singletons cannot be safely narrowed using `is not`
}
}
ast::CmpOp::Is => {
self.constraints.insert(symbol, comp_ty);
constraints.insert(symbol, rhs_ty);
}
ast::CmpOp::NotEq => {
if comp_ty.is_single_valued(self.db) {
if rhs_ty.is_single_valued(self.db) {
let ty = IntersectionBuilder::new(self.db)
.add_negative(comp_ty)
.add_negative(rhs_ty)
.build();
self.constraints.insert(symbol, ty);
constraints.insert(symbol, ty);
}
}
_ => {
@@ -221,15 +318,21 @@ impl<'db> NarrowingConstraintsBuilder<'db> {
}
}
}
Some(constraints)
}
fn add_expr_call(&mut self, expr_call: &ast::ExprCall, expression: Expression<'db>) {
fn evaluate_expr_call(
&mut self,
expr_call: &ast::ExprCall,
expression: Expression<'db>,
is_positive: bool,
) -> Option<NarrowingConstraints<'db>> {
let scope = self.scope();
let inference = infer_expression_types(self.db, expression);
if let Some(func_type) = inference
.expression_ty(expr_call.func.scoped_ast_id(self.db, scope))
.into_function_literal_type()
.into_function_literal()
{
if func_type.is_known(self.db, KnownFunction::IsInstance)
&& expr_call.arguments.keywords.is_empty()
@@ -242,29 +345,93 @@ impl<'db> NarrowingConstraintsBuilder<'db> {
// TODO: add support for PEP 604 union types on the right hand side:
// isinstance(x, str | (int | float))
if let Some(constraint) = generate_isinstance_constraint(self.db, &rhs_type) {
self.constraints.insert(symbol, constraint);
if let Some(mut constraint) = generate_isinstance_constraint(self.db, &rhs_type)
{
if !is_positive {
constraint = constraint.negate(self.db);
}
let mut constraints = NarrowingConstraints::default();
constraints.insert(symbol, constraint);
return Some(constraints);
}
}
}
}
None
}
fn add_match_pattern_singleton(
fn evaluate_match_pattern_singleton(
&mut self,
subject: &ast::Expr,
pattern: &ast::PatternMatchSingleton,
) {
) -> Option<NarrowingConstraints<'db>> {
if let Some(ast::ExprName { id, .. }) = subject.as_name_expr() {
// SAFETY: we should always have a symbol for every Name node.
let symbol = self.symbols().symbol_id_by_name(id).unwrap();
let ty = match pattern.value {
ast::Singleton::None => Type::None,
ast::Singleton::None => Type::none(self.db),
ast::Singleton::True => Type::BooleanLiteral(true),
ast::Singleton::False => Type::BooleanLiteral(false),
};
self.constraints.insert(symbol, ty);
let mut constraints = NarrowingConstraints::default();
constraints.insert(symbol, ty);
Some(constraints)
} else {
None
}
}
fn evaluate_bool_op(
&mut self,
expr_bool_op: &ExprBoolOp,
expression: Expression<'db>,
is_positive: bool,
) -> Option<NarrowingConstraints<'db>> {
let inference = infer_expression_types(self.db, expression);
let scope = self.scope();
let mut sub_constraints = expr_bool_op
.values
.iter()
// filter our arms with statically known truthiness
.filter(|expr| {
inference
.expression_ty(expr.scoped_ast_id(self.db, scope))
.bool(self.db)
!= match expr_bool_op.op {
BoolOp::And => Truthiness::AlwaysTrue,
BoolOp::Or => Truthiness::AlwaysFalse,
}
})
.map(|sub_expr| {
self.evaluate_expression_node_constraint(sub_expr, expression, is_positive)
})
.collect::<Vec<_>>();
match (expr_bool_op.op, is_positive) {
(BoolOp::And, true) | (BoolOp::Or, false) => {
let mut aggregation: Option<NarrowingConstraints> = None;
for sub_constraint in sub_constraints.into_iter().flatten() {
if let Some(ref mut some_aggregation) = aggregation {
merge_constraints_and(some_aggregation, sub_constraint, self.db);
} else {
aggregation = Some(sub_constraint);
}
}
aggregation
}
(BoolOp::Or, true) | (BoolOp::And, false) => {
let (first, rest) = sub_constraints.split_first_mut()?;
if let Some(ref mut first) = first {
for rest_constraint in rest {
if let Some(rest_constraint) = rest_constraint {
merge_constraints_or(first, rest_constraint, self.db);
} else {
return None;
}
}
}
first.clone()
}
}
}
}

View File

@@ -0,0 +1,143 @@
use std::borrow::Cow;
use ruff_db::files::File;
use ruff_python_ast::{self as ast, AnyNodeRef};
use rustc_hash::FxHashMap;
use crate::semantic_index::ast_ids::{HasScopedAstId, ScopedExpressionId};
use crate::semantic_index::symbol::ScopeId;
use crate::types::{TupleType, Type, TypeCheckDiagnostics, TypeCheckDiagnosticsBuilder};
use crate::Db;
/// Unpacks the value expression type to their respective targets.
pub(crate) struct Unpacker<'db> {
db: &'db dyn Db,
targets: FxHashMap<ScopedExpressionId, Type<'db>>,
diagnostics: TypeCheckDiagnosticsBuilder<'db>,
}
impl<'db> Unpacker<'db> {
pub(crate) fn new(db: &'db dyn Db, file: File) -> Self {
Self {
db,
targets: FxHashMap::default(),
diagnostics: TypeCheckDiagnosticsBuilder::new(db, file),
}
}
pub(crate) fn unpack(&mut self, target: &ast::Expr, value_ty: Type<'db>, scope: ScopeId<'db>) {
match target {
ast::Expr::Name(target_name) => {
self.targets
.insert(target_name.scoped_ast_id(self.db, scope), value_ty);
}
ast::Expr::Starred(ast::ExprStarred { value, .. }) => {
self.unpack(value, value_ty, scope);
}
ast::Expr::List(ast::ExprList { elts, .. })
| ast::Expr::Tuple(ast::ExprTuple { elts, .. }) => match value_ty {
Type::Tuple(tuple_ty) => {
let starred_index = elts.iter().position(ast::Expr::is_starred_expr);
let element_types = if let Some(starred_index) = starred_index {
if tuple_ty.len(self.db) >= elts.len() - 1 {
let mut element_types = Vec::with_capacity(elts.len());
element_types.extend_from_slice(
// SAFETY: Safe because of the length check above.
&tuple_ty.elements(self.db)[..starred_index],
);
// E.g., in `(a, *b, c, d) = ...`, the index of starred element `b`
// is 1 and the remaining elements after that are 2.
let remaining = elts.len() - (starred_index + 1);
// This index represents the type of the last element that belongs
// to the starred expression, in an exclusive manner.
let starred_end_index = tuple_ty.len(self.db) - remaining;
// SAFETY: Safe because of the length check above.
let _starred_element_types =
&tuple_ty.elements(self.db)[starred_index..starred_end_index];
// TODO: Combine the types into a list type. If the
// starred_element_types is empty, then it should be `List[Any]`.
// combine_types(starred_element_types);
element_types.push(Type::Todo);
element_types.extend_from_slice(
// SAFETY: Safe because of the length check above.
&tuple_ty.elements(self.db)[starred_end_index..],
);
Cow::Owned(element_types)
} else {
let mut element_types = tuple_ty.elements(self.db).to_vec();
// Subtract 1 to insert the starred expression type at the correct
// index.
element_types.resize(elts.len() - 1, Type::Unknown);
// TODO: This should be `list[Unknown]`
element_types.insert(starred_index, Type::Todo);
Cow::Owned(element_types)
}
} else {
Cow::Borrowed(tuple_ty.elements(self.db).as_ref())
};
for (index, element) in elts.iter().enumerate() {
self.unpack(
element,
element_types.get(index).copied().unwrap_or(Type::Unknown),
scope,
);
}
}
Type::StringLiteral(string_literal_ty) => {
// Deconstruct the string literal to delegate the inference back to the
// tuple type for correct handling of starred expressions. We could go
// further and deconstruct to an array of `StringLiteral` with each
// individual character, instead of just an array of `LiteralString`, but
// there would be a cost and it's not clear that it's worth it.
let value_ty = Type::Tuple(TupleType::new(
self.db,
vec![Type::LiteralString; string_literal_ty.len(self.db)]
.into_boxed_slice(),
));
self.unpack(target, value_ty, scope);
}
_ => {
let value_ty = if value_ty.is_literal_string() {
Type::LiteralString
} else {
value_ty
.iterate(self.db)
.unwrap_with_diagnostic(AnyNodeRef::from(target), &mut self.diagnostics)
};
for element in elts {
self.unpack(element, value_ty, scope);
}
}
},
_ => {}
}
}
pub(crate) fn finish(mut self) -> UnpackResult<'db> {
self.targets.shrink_to_fit();
UnpackResult {
diagnostics: self.diagnostics.finish(),
targets: self.targets,
}
}
}
#[derive(Debug, Default, PartialEq, Eq)]
pub(crate) struct UnpackResult<'db> {
targets: FxHashMap<ScopedExpressionId, Type<'db>>,
diagnostics: TypeCheckDiagnostics,
}
impl<'db> UnpackResult<'db> {
pub(crate) fn get(&self, expr_id: ScopedExpressionId) -> Option<Type<'db>> {
self.targets.get(&expr_id).copied()
}
pub(crate) fn diagnostics(&self) -> &TypeCheckDiagnostics {
&self.diagnostics
}
}

View File

@@ -0,0 +1,55 @@
use ruff_db::files::File;
use ruff_python_ast::{self as ast};
use crate::ast_node_ref::AstNodeRef;
use crate::semantic_index::expression::Expression;
use crate::semantic_index::symbol::{FileScopeId, ScopeId};
use crate::Db;
/// This ingredient represents a single unpacking.
///
/// This is required to make use of salsa to cache the complete unpacking of multiple variables
/// involved. It allows us to:
/// 1. Avoid doing structural match multiple times for each definition
/// 2. Avoid highlighting the same error multiple times
///
/// ## Module-local type
/// This type should not be used as part of any cross-module API because
/// it holds a reference to the AST node. Range-offset changes
/// then propagate through all usages, and deserialization requires
/// reparsing the entire module.
///
/// E.g. don't use this type in:
///
/// * a return type of a cross-module query
/// * a field of a type that is a return type of a cross-module query
/// * an argument of a cross-module query
#[salsa::tracked]
pub(crate) struct Unpack<'db> {
#[id]
pub(crate) file: File,
#[id]
pub(crate) file_scope: FileScopeId,
/// The target expression that is being unpacked. For example, in `(a, b) = (1, 2)`, the target
/// expression is `(a, b)`.
#[no_eq]
#[return_ref]
pub(crate) target: AstNodeRef<ast::Expr>,
/// The ingredient representing the value expression of the unpacking. For example, in
/// `(a, b) = (1, 2)`, the value expression is `(1, 2)`.
#[no_eq]
pub(crate) value: Expression<'db>,
#[no_eq]
count: countme::Count<Unpack<'static>>,
}
impl<'db> Unpack<'db> {
/// Returns the scope where the unpacking is happening.
pub(crate) fn scope(self, db: &'db dyn Db) -> ScopeId<'db> {
self.file_scope(db).to_scope_id(db, self.file(db))
}
}

View File

@@ -1,18 +1,192 @@
pub(crate) trait PythonSubscript {
//! This module provides utility functions for indexing (`PyIndex`) and slicing
//! operations (`PySlice`) on iterators, following the semantics of equivalent
//! operations in Python.
use itertools::Either;
#[derive(Debug, Clone, Copy, PartialEq)]
pub(crate) struct OutOfBoundsError;
pub(crate) trait PyIndex {
type Item;
fn python_subscript(&mut self, index: i64) -> Option<Self::Item>;
fn py_index(&mut self, index: i32) -> Result<Self::Item, OutOfBoundsError>;
}
impl<I, T: DoubleEndedIterator<Item = I>> PythonSubscript for T {
fn from_nonnegative_i32(index: i32) -> usize {
static_assertions::const_assert!(usize::BITS >= 32);
debug_assert!(index >= 0);
usize::try_from(index)
.expect("Should only ever pass a positive integer to `from_nonnegative_i32`")
}
fn from_negative_i32(index: i32) -> usize {
static_assertions::const_assert!(usize::BITS >= 32);
index.checked_neg().map(from_nonnegative_i32).unwrap_or({
// 'checked_neg' only fails for i32::MIN. We can not
// represent -i32::MIN as a i32, but we can represent
// it as a usize, since usize is at least 32 bits.
from_nonnegative_i32(i32::MAX) + 1
})
}
#[derive(Debug, Clone, Copy, PartialEq, PartialOrd)]
enum Position {
BeforeStart,
AtIndex(usize),
AfterEnd,
}
enum Nth {
FromStart(usize),
FromEnd(usize),
}
impl Nth {
fn from_index(index: i32) -> Self {
if index >= 0 {
Nth::FromStart(from_nonnegative_i32(index))
} else {
Nth::FromEnd(from_negative_i32(index) - 1)
}
}
fn to_position(&self, len: usize) -> Position {
debug_assert!(len > 0);
match self {
Nth::FromStart(nth) => {
if *nth < len {
Position::AtIndex(*nth)
} else {
Position::AfterEnd
}
}
Nth::FromEnd(nth_rev) => {
if *nth_rev < len {
Position::AtIndex(len - 1 - *nth_rev)
} else {
Position::BeforeStart
}
}
}
}
}
impl<I, T> PyIndex for T
where
T: DoubleEndedIterator<Item = I>,
{
type Item = I;
fn python_subscript(&mut self, index: i64) -> Option<I> {
if index >= 0 {
self.nth(usize::try_from(index).ok()?)
fn py_index(&mut self, index: i32) -> Result<I, OutOfBoundsError> {
match Nth::from_index(index) {
Nth::FromStart(nth) => self.nth(nth).ok_or(OutOfBoundsError),
Nth::FromEnd(nth_rev) => self.nth_back(nth_rev).ok_or(OutOfBoundsError),
}
}
}
#[derive(Debug, Clone, Copy, PartialEq)]
pub(crate) struct StepSizeZeroError;
pub(crate) trait PySlice {
type Item;
fn py_slice(
&self,
start: Option<i32>,
stop: Option<i32>,
step: Option<i32>,
) -> Result<
Either<impl Iterator<Item = &Self::Item>, impl Iterator<Item = &Self::Item>>,
StepSizeZeroError,
>;
}
impl<T> PySlice for [T] {
type Item = T;
fn py_slice(
&self,
start: Option<i32>,
stop: Option<i32>,
step_int: Option<i32>,
) -> Result<
Either<impl Iterator<Item = &Self::Item>, impl Iterator<Item = &Self::Item>>,
StepSizeZeroError,
> {
let step_int = step_int.unwrap_or(1);
if step_int == 0 {
return Err(StepSizeZeroError);
}
let len = self.len();
if len == 0 {
// The iterator needs to have the same type as the step>0 case below,
// so we need to use `.skip(0)`.
#[allow(clippy::iter_skip_zero)]
return Ok(Either::Left(self.iter().skip(0).take(0).step_by(1)));
}
let to_position = |index| Nth::from_index(index).to_position(len);
if step_int.is_positive() {
let step = from_nonnegative_i32(step_int);
let start = start.map(to_position).unwrap_or(Position::BeforeStart);
let stop = stop.map(to_position).unwrap_or(Position::AfterEnd);
let (skip, take, step) = if start < stop {
let skip = match start {
Position::BeforeStart => 0,
Position::AtIndex(start_index) => start_index,
Position::AfterEnd => len,
};
let take = match stop {
Position::BeforeStart => 0,
Position::AtIndex(stop_index) => stop_index - skip,
Position::AfterEnd => len - skip,
};
(skip, take, step)
} else {
(0, 0, step)
};
Ok(Either::Left(
self.iter().skip(skip).take(take).step_by(step),
))
} else {
let nth_rev = usize::try_from(index.checked_neg()?).ok()?.checked_sub(1)?;
self.rev().nth(nth_rev)
let step = from_negative_i32(step_int);
let start = start.map(to_position).unwrap_or(Position::AfterEnd);
let stop = stop.map(to_position).unwrap_or(Position::BeforeStart);
let (skip, take, step) = if start <= stop {
(0, 0, step)
} else {
let skip = match start {
Position::BeforeStart => len,
Position::AtIndex(start_index) => len - 1 - start_index,
Position::AfterEnd => 0,
};
let take = match stop {
Position::BeforeStart => len - skip,
Position::AtIndex(stop_index) => (len - 1) - skip - stop_index,
Position::AfterEnd => 0,
};
(skip, take, step)
};
Ok(Either::Right(
self.iter().rev().skip(skip).take(take).step_by(step),
))
}
}
}
@@ -20,64 +194,309 @@ impl<I, T: DoubleEndedIterator<Item = I>> PythonSubscript for T {
#[cfg(test)]
#[allow(clippy::redundant_clone)]
mod tests {
use super::PythonSubscript;
use crate::util::subscript::{OutOfBoundsError, StepSizeZeroError};
use super::{PyIndex, PySlice};
use itertools::assert_equal;
#[test]
fn python_subscript_basic() {
let iter = 'a'..='e';
fn py_index_empty() {
let iter = std::iter::empty::<char>();
assert_eq!(iter.clone().python_subscript(0), Some('a'));
assert_eq!(iter.clone().python_subscript(1), Some('b'));
assert_eq!(iter.clone().python_subscript(4), Some('e'));
assert_eq!(iter.clone().python_subscript(5), None);
assert_eq!(iter.clone().python_subscript(-1), Some('e'));
assert_eq!(iter.clone().python_subscript(-2), Some('d'));
assert_eq!(iter.clone().python_subscript(-5), Some('a'));
assert_eq!(iter.clone().python_subscript(-6), None);
assert_eq!(iter.clone().py_index(0), Err(OutOfBoundsError));
assert_eq!(iter.clone().py_index(1), Err(OutOfBoundsError));
assert_eq!(iter.clone().py_index(-1), Err(OutOfBoundsError));
assert_eq!(iter.clone().py_index(i32::MIN), Err(OutOfBoundsError));
assert_eq!(iter.clone().py_index(i32::MAX), Err(OutOfBoundsError));
}
#[test]
fn python_subscript_empty() {
let iter = 'a'..'a';
fn py_index_single_element() {
let iter = ['a'].into_iter();
assert_eq!(iter.clone().python_subscript(0), None);
assert_eq!(iter.clone().python_subscript(1), None);
assert_eq!(iter.clone().python_subscript(-1), None);
assert_eq!(iter.clone().py_index(0), Ok('a'));
assert_eq!(iter.clone().py_index(1), Err(OutOfBoundsError));
assert_eq!(iter.clone().py_index(-1), Ok('a'));
assert_eq!(iter.clone().py_index(-2), Err(OutOfBoundsError));
}
#[test]
fn python_subscript_single_element() {
let iter = 'a'..='a';
fn py_index_more_elements() {
let iter = ['a', 'b', 'c', 'd', 'e'].into_iter();
assert_eq!(iter.clone().python_subscript(0), Some('a'));
assert_eq!(iter.clone().python_subscript(1), None);
assert_eq!(iter.clone().python_subscript(-1), Some('a'));
assert_eq!(iter.clone().python_subscript(-2), None);
assert_eq!(iter.clone().py_index(0), Ok('a'));
assert_eq!(iter.clone().py_index(1), Ok('b'));
assert_eq!(iter.clone().py_index(4), Ok('e'));
assert_eq!(iter.clone().py_index(5), Err(OutOfBoundsError));
assert_eq!(iter.clone().py_index(-1), Ok('e'));
assert_eq!(iter.clone().py_index(-2), Ok('d'));
assert_eq!(iter.clone().py_index(-5), Ok('a'));
assert_eq!(iter.clone().py_index(-6), Err(OutOfBoundsError));
}
#[test]
fn python_subscript_uses_full_index_range() {
let iter = 0..=u64::MAX;
fn py_index_uses_full_index_range() {
let iter = 0..=u32::MAX;
assert_eq!(iter.clone().python_subscript(0), Some(0));
assert_eq!(iter.clone().python_subscript(1), Some(1));
assert_eq!(
iter.clone().python_subscript(i64::MAX),
Some(i64::MAX as u64)
);
// u32::MAX - |i32::MIN| + 1 = 2^32 - 1 - 2^31 + 1 = 2^31
assert_eq!(iter.clone().py_index(i32::MIN), Ok(2u32.pow(31)));
assert_eq!(iter.clone().py_index(-2), Ok(u32::MAX - 2 + 1));
assert_eq!(iter.clone().py_index(-1), Ok(u32::MAX - 1 + 1));
assert_eq!(iter.clone().python_subscript(-1), Some(u64::MAX));
assert_eq!(iter.clone().python_subscript(-2), Some(u64::MAX - 1));
assert_eq!(iter.clone().py_index(0), Ok(0));
assert_eq!(iter.clone().py_index(1), Ok(1));
assert_eq!(iter.clone().py_index(i32::MAX), Ok(i32::MAX as u32));
}
// i64::MIN is not representable as a positive number, so it is not
// a valid index:
assert_eq!(iter.clone().python_subscript(i64::MIN), None);
#[track_caller]
fn assert_eq_slice<const N: usize, const M: usize>(
input: &[char; N],
start: Option<i32>,
stop: Option<i32>,
step: Option<i32>,
expected: &[char; M],
) {
assert_equal(input.py_slice(start, stop, step).unwrap(), expected.iter());
}
// but i64::MIN +1 is:
assert_eq!(
iter.clone().python_subscript(i64::MIN + 1),
Some(2u64.pow(63) + 1)
);
#[test]
fn py_slice_empty_input() {
let input = [];
assert_eq_slice(&input, None, None, None, &[]);
assert_eq_slice(&input, Some(0), None, None, &[]);
assert_eq_slice(&input, None, Some(0), None, &[]);
assert_eq_slice(&input, Some(0), Some(0), None, &[]);
assert_eq_slice(&input, Some(-5), Some(-5), None, &[]);
assert_eq_slice(&input, None, None, Some(-1), &[]);
assert_eq_slice(&input, None, None, Some(2), &[]);
}
#[test]
fn py_slice_single_element_input() {
let input = ['a'];
assert_eq_slice(&input, None, None, None, &['a']);
assert_eq_slice(&input, Some(0), None, None, &['a']);
assert_eq_slice(&input, None, Some(0), None, &[]);
assert_eq_slice(&input, Some(0), Some(0), None, &[]);
assert_eq_slice(&input, Some(0), Some(1), None, &['a']);
assert_eq_slice(&input, Some(0), Some(2), None, &['a']);
assert_eq_slice(&input, Some(-1), None, None, &['a']);
assert_eq_slice(&input, Some(-1), Some(-1), None, &[]);
assert_eq_slice(&input, Some(-1), Some(0), None, &[]);
assert_eq_slice(&input, Some(-1), Some(1), None, &['a']);
assert_eq_slice(&input, Some(-1), Some(2), None, &['a']);
assert_eq_slice(&input, None, Some(-1), None, &[]);
assert_eq_slice(&input, Some(-2), None, None, &['a']);
assert_eq_slice(&input, Some(-2), Some(-1), None, &[]);
assert_eq_slice(&input, Some(-2), Some(0), None, &[]);
assert_eq_slice(&input, Some(-2), Some(1), None, &['a']);
assert_eq_slice(&input, Some(-2), Some(2), None, &['a']);
}
#[test]
fn py_slice_nonnegative_indices() {
let input = ['a', 'b', 'c', 'd', 'e'];
assert_eq_slice(&input, None, Some(0), None, &[]);
assert_eq_slice(&input, None, Some(1), None, &['a']);
assert_eq_slice(&input, None, Some(4), None, &['a', 'b', 'c', 'd']);
assert_eq_slice(&input, None, Some(5), None, &['a', 'b', 'c', 'd', 'e']);
assert_eq_slice(&input, None, Some(6), None, &['a', 'b', 'c', 'd', 'e']);
assert_eq_slice(&input, None, None, None, &['a', 'b', 'c', 'd', 'e']);
assert_eq_slice(&input, Some(0), Some(0), None, &[]);
assert_eq_slice(&input, Some(0), Some(1), None, &['a']);
assert_eq_slice(&input, Some(0), Some(4), None, &['a', 'b', 'c', 'd']);
assert_eq_slice(&input, Some(0), Some(5), None, &['a', 'b', 'c', 'd', 'e']);
assert_eq_slice(&input, Some(0), Some(6), None, &['a', 'b', 'c', 'd', 'e']);
assert_eq_slice(&input, Some(0), None, None, &['a', 'b', 'c', 'd', 'e']);
assert_eq_slice(&input, Some(1), Some(0), None, &[]);
assert_eq_slice(&input, Some(1), Some(1), None, &[]);
assert_eq_slice(&input, Some(1), Some(2), None, &['b']);
assert_eq_slice(&input, Some(1), Some(4), None, &['b', 'c', 'd']);
assert_eq_slice(&input, Some(1), Some(5), None, &['b', 'c', 'd', 'e']);
assert_eq_slice(&input, Some(1), Some(6), None, &['b', 'c', 'd', 'e']);
assert_eq_slice(&input, Some(1), None, None, &['b', 'c', 'd', 'e']);
assert_eq_slice(&input, Some(4), Some(0), None, &[]);
assert_eq_slice(&input, Some(4), Some(4), None, &[]);
assert_eq_slice(&input, Some(4), Some(5), None, &['e']);
assert_eq_slice(&input, Some(4), Some(6), None, &['e']);
assert_eq_slice(&input, Some(4), None, None, &['e']);
assert_eq_slice(&input, Some(5), Some(0), None, &[]);
assert_eq_slice(&input, Some(5), Some(5), None, &[]);
assert_eq_slice(&input, Some(5), Some(6), None, &[]);
assert_eq_slice(&input, Some(5), None, None, &[]);
assert_eq_slice(&input, Some(6), Some(0), None, &[]);
assert_eq_slice(&input, Some(6), Some(6), None, &[]);
assert_eq_slice(&input, Some(6), None, None, &[]);
}
#[test]
fn py_slice_negatice_indices() {
let input = ['a', 'b', 'c', 'd', 'e'];
assert_eq_slice(&input, Some(-6), None, None, &['a', 'b', 'c', 'd', 'e']);
assert_eq_slice(&input, Some(-6), Some(-1), None, &['a', 'b', 'c', 'd']);
assert_eq_slice(&input, Some(-6), Some(-4), None, &['a']);
assert_eq_slice(&input, Some(-6), Some(-5), None, &[]);
assert_eq_slice(&input, Some(-6), Some(-6), None, &[]);
assert_eq_slice(&input, Some(-6), Some(-10), None, &[]);
assert_eq_slice(&input, Some(-5), None, None, &['a', 'b', 'c', 'd', 'e']);
assert_eq_slice(&input, Some(-5), Some(-1), None, &['a', 'b', 'c', 'd']);
assert_eq_slice(&input, Some(-5), Some(-4), None, &['a']);
assert_eq_slice(&input, Some(-5), Some(-5), None, &[]);
assert_eq_slice(&input, Some(-5), Some(-6), None, &[]);
assert_eq_slice(&input, Some(-5), Some(-10), None, &[]);
assert_eq_slice(&input, Some(-4), None, None, &['b', 'c', 'd', 'e']);
assert_eq_slice(&input, Some(-4), Some(-1), None, &['b', 'c', 'd']);
assert_eq_slice(&input, Some(-4), Some(-3), None, &['b']);
assert_eq_slice(&input, Some(-4), Some(-4), None, &[]);
assert_eq_slice(&input, Some(-4), Some(-10), None, &[]);
assert_eq_slice(&input, Some(-1), None, None, &['e']);
assert_eq_slice(&input, Some(-1), Some(-1), None, &[]);
assert_eq_slice(&input, Some(-1), Some(-10), None, &[]);
assert_eq_slice(&input, None, Some(-1), None, &['a', 'b', 'c', 'd']);
assert_eq_slice(&input, None, Some(-4), None, &['a']);
assert_eq_slice(&input, None, Some(-5), None, &[]);
assert_eq_slice(&input, None, Some(-6), None, &[]);
}
#[test]
fn py_slice_mixed_positive_negative_indices() {
let input = ['a', 'b', 'c', 'd', 'e'];
assert_eq_slice(&input, Some(0), Some(-1), None, &['a', 'b', 'c', 'd']);
assert_eq_slice(&input, Some(1), Some(-1), None, &['b', 'c', 'd']);
assert_eq_slice(&input, Some(3), Some(-1), None, &['d']);
assert_eq_slice(&input, Some(4), Some(-1), None, &[]);
assert_eq_slice(&input, Some(5), Some(-1), None, &[]);
assert_eq_slice(&input, Some(0), Some(-4), None, &['a']);
assert_eq_slice(&input, Some(1), Some(-4), None, &[]);
assert_eq_slice(&input, Some(3), Some(-4), None, &[]);
assert_eq_slice(&input, Some(0), Some(-5), None, &[]);
assert_eq_slice(&input, Some(1), Some(-5), None, &[]);
assert_eq_slice(&input, Some(3), Some(-5), None, &[]);
assert_eq_slice(&input, Some(0), Some(-6), None, &[]);
assert_eq_slice(&input, Some(1), Some(-6), None, &[]);
assert_eq_slice(&input, Some(-6), Some(6), None, &['a', 'b', 'c', 'd', 'e']);
assert_eq_slice(&input, Some(-6), Some(5), None, &['a', 'b', 'c', 'd', 'e']);
assert_eq_slice(&input, Some(-6), Some(4), None, &['a', 'b', 'c', 'd']);
assert_eq_slice(&input, Some(-6), Some(1), None, &['a']);
assert_eq_slice(&input, Some(-6), Some(0), None, &[]);
assert_eq_slice(&input, Some(-5), Some(6), None, &['a', 'b', 'c', 'd', 'e']);
assert_eq_slice(&input, Some(-5), Some(5), None, &['a', 'b', 'c', 'd', 'e']);
assert_eq_slice(&input, Some(-5), Some(4), None, &['a', 'b', 'c', 'd']);
assert_eq_slice(&input, Some(-5), Some(1), None, &['a']);
assert_eq_slice(&input, Some(-5), Some(0), None, &[]);
assert_eq_slice(&input, Some(-4), Some(6), None, &['b', 'c', 'd', 'e']);
assert_eq_slice(&input, Some(-4), Some(5), None, &['b', 'c', 'd', 'e']);
assert_eq_slice(&input, Some(-4), Some(4), None, &['b', 'c', 'd']);
assert_eq_slice(&input, Some(-4), Some(2), None, &['b']);
assert_eq_slice(&input, Some(-4), Some(1), None, &[]);
assert_eq_slice(&input, Some(-4), Some(0), None, &[]);
assert_eq_slice(&input, Some(-1), Some(6), None, &['e']);
assert_eq_slice(&input, Some(-1), Some(5), None, &['e']);
assert_eq_slice(&input, Some(-1), Some(4), None, &[]);
assert_eq_slice(&input, Some(-1), Some(1), None, &[]);
}
#[test]
fn py_slice_step_forward() {
// indices: 0 1 2 3 4 5 6
let input = ['a', 'b', 'c', 'd', 'e', 'f', 'g'];
// Step size zero is invalid:
assert!(matches!(
input.py_slice(None, None, Some(0)),
Err(StepSizeZeroError)
));
assert!(matches!(
input.py_slice(Some(0), Some(5), Some(0)),
Err(StepSizeZeroError)
));
assert!(matches!(
input.py_slice(Some(0), Some(0), Some(0)),
Err(StepSizeZeroError)
));
assert_eq_slice(&input, Some(0), Some(8), Some(2), &['a', 'c', 'e', 'g']);
assert_eq_slice(&input, Some(0), Some(7), Some(2), &['a', 'c', 'e', 'g']);
assert_eq_slice(&input, Some(0), Some(6), Some(2), &['a', 'c', 'e']);
assert_eq_slice(&input, Some(0), Some(5), Some(2), &['a', 'c', 'e']);
assert_eq_slice(&input, Some(0), Some(4), Some(2), &['a', 'c']);
assert_eq_slice(&input, Some(0), Some(3), Some(2), &['a', 'c']);
assert_eq_slice(&input, Some(0), Some(2), Some(2), &['a']);
assert_eq_slice(&input, Some(0), Some(1), Some(2), &['a']);
assert_eq_slice(&input, Some(0), Some(0), Some(2), &[]);
assert_eq_slice(&input, Some(1), Some(5), Some(2), &['b', 'd']);
assert_eq_slice(&input, Some(0), Some(7), Some(3), &['a', 'd', 'g']);
assert_eq_slice(&input, Some(0), Some(6), Some(3), &['a', 'd']);
assert_eq_slice(&input, Some(0), None, Some(10), &['a']);
}
#[test]
fn py_slice_step_backward() {
// indices: 0 1 2 3 4 5 6
let input = ['a', 'b', 'c', 'd', 'e', 'f', 'g'];
assert_eq_slice(&input, Some(7), Some(0), Some(-2), &['g', 'e', 'c']);
assert_eq_slice(&input, Some(6), Some(0), Some(-2), &['g', 'e', 'c']);
assert_eq_slice(&input, Some(5), Some(0), Some(-2), &['f', 'd', 'b']);
assert_eq_slice(&input, Some(4), Some(0), Some(-2), &['e', 'c']);
assert_eq_slice(&input, Some(3), Some(0), Some(-2), &['d', 'b']);
assert_eq_slice(&input, Some(2), Some(0), Some(-2), &['c']);
assert_eq_slice(&input, Some(1), Some(0), Some(-2), &['b']);
assert_eq_slice(&input, Some(0), Some(0), Some(-2), &[]);
assert_eq_slice(&input, Some(7), None, Some(-2), &['g', 'e', 'c', 'a']);
assert_eq_slice(&input, None, None, Some(-2), &['g', 'e', 'c', 'a']);
assert_eq_slice(&input, None, Some(0), Some(-2), &['g', 'e', 'c']);
assert_eq_slice(&input, Some(5), Some(1), Some(-2), &['f', 'd']);
assert_eq_slice(&input, Some(5), Some(2), Some(-2), &['f', 'd']);
assert_eq_slice(&input, Some(5), Some(3), Some(-2), &['f']);
assert_eq_slice(&input, Some(5), Some(4), Some(-2), &['f']);
assert_eq_slice(&input, Some(5), Some(5), Some(-2), &[]);
assert_eq_slice(&input, Some(6), None, Some(-3), &['g', 'd', 'a']);
assert_eq_slice(&input, Some(6), Some(0), Some(-3), &['g', 'd']);
assert_eq_slice(&input, Some(7), None, Some(-10), &['g']);
assert_eq_slice(&input, Some(-6), Some(-9), Some(-1), &['b', 'a']);
assert_eq_slice(&input, Some(-6), Some(-8), Some(-1), &['b', 'a']);
assert_eq_slice(&input, Some(-6), Some(-7), Some(-1), &['b']);
assert_eq_slice(&input, Some(-6), Some(-6), Some(-1), &[]);
assert_eq_slice(&input, Some(-7), Some(-9), Some(-1), &['a']);
assert_eq_slice(&input, Some(-8), Some(-9), Some(-1), &[]);
assert_eq_slice(&input, Some(-9), Some(-9), Some(-1), &[]);
assert_eq_slice(&input, Some(-6), Some(-2), Some(-1), &[]);
assert_eq_slice(&input, Some(-9), Some(-6), Some(-1), &[]);
}
}

View File

@@ -15,6 +15,7 @@ red_knot_python_semantic = { workspace = true }
red_knot_vendored = { workspace = true }
ruff_db = { workspace = true }
ruff_index = { workspace = true }
ruff_python_parser = { workspace = true }
ruff_python_trivia = { workspace = true }
ruff_source_file = { workspace = true }
ruff_text_size = { workspace = true }

View File

@@ -40,7 +40,7 @@ use ruff_db::files::File;
use ruff_db::parsed::parsed_module;
use ruff_db::source::{line_index, source_text, SourceText};
use ruff_python_trivia::CommentRanges;
use ruff_source_file::{LineIndex, Locator, OneIndexed};
use ruff_source_file::{LineIndex, OneIndexed};
use ruff_text_size::{Ranged, TextRange};
use smallvec::SmallVec;
use std::ops::Deref;
@@ -67,16 +67,12 @@ impl InlineFileAssertions {
}
}
fn locator(&self) -> Locator {
Locator::with_index(&self.source, self.lines.clone())
}
fn line_number(&self, range: &impl Ranged) -> OneIndexed {
self.lines.line_index(range.start())
}
fn is_own_line_comment(&self, ranged_assertion: &AssertionWithRange) -> bool {
CommentRanges::is_own_line(ranged_assertion.start(), &self.locator())
CommentRanges::is_own_line(ranged_assertion.start(), self.source.as_str())
}
}
@@ -131,10 +127,9 @@ impl<'a> Iterator for AssertionWithRangeIterator<'a> {
type Item = AssertionWithRange<'a>;
fn next(&mut self) -> Option<Self::Item> {
let locator = self.file_assertions.locator();
loop {
let inner_next = self.inner.next()?;
let comment = locator.slice(inner_next);
let comment = &self.file_assertions.source[inner_next];
if let Some(assertion) = Assertion::from_comment(comment) {
return Some(AssertionWithRange(assertion, inner_next));
};

View File

@@ -2,10 +2,63 @@
//!
//! We don't assume that we will get the diagnostics in source order.
use red_knot_python_semantic::types::TypeCheckDiagnostic;
use ruff_python_parser::ParseError;
use ruff_source_file::{LineIndex, OneIndexed};
use ruff_text_size::Ranged;
use ruff_text_size::{Ranged, TextRange};
use std::borrow::Cow;
use std::ops::{Deref, Range};
pub(super) trait Diagnostic: std::fmt::Debug {
fn rule(&self) -> &str;
fn message(&self) -> Cow<str>;
fn range(&self) -> TextRange;
}
impl Diagnostic for TypeCheckDiagnostic {
fn rule(&self) -> &str {
TypeCheckDiagnostic::rule(self)
}
fn message(&self) -> Cow<str> {
TypeCheckDiagnostic::message(self).into()
}
fn range(&self) -> TextRange {
Ranged::range(self)
}
}
impl Diagnostic for ParseError {
fn rule(&self) -> &str {
"invalid-syntax"
}
fn message(&self) -> Cow<str> {
self.error.to_string().into()
}
fn range(&self) -> TextRange {
self.location
}
}
impl Diagnostic for Box<dyn Diagnostic> {
fn rule(&self) -> &str {
(**self).rule()
}
fn message(&self) -> Cow<str> {
(**self).message()
}
fn range(&self) -> TextRange {
(**self).range()
}
}
/// All diagnostics for one embedded Python file, sorted and grouped by start line number.
///
/// The diagnostics are kept in a flat vector, sorted by line number. A separate vector of
@@ -19,13 +72,13 @@ pub(crate) struct SortedDiagnostics<T> {
impl<T> SortedDiagnostics<T>
where
T: Ranged + Clone,
T: Diagnostic,
{
pub(crate) fn new(diagnostics: impl IntoIterator<Item = T>, line_index: &LineIndex) -> Self {
let mut diagnostics: Vec<_> = diagnostics
.into_iter()
.map(|diagnostic| DiagnosticWithLine {
line_number: line_index.line_index(diagnostic.start()),
line_number: line_index.line_index(diagnostic.range().start()),
diagnostic,
})
.collect();
@@ -94,7 +147,7 @@ pub(crate) struct LineDiagnosticsIterator<'a, T> {
impl<'a, T> Iterator for LineDiagnosticsIterator<'a, T>
where
T: Ranged + Clone,
T: Diagnostic,
{
type Item = LineDiagnostics<'a, T>;
@@ -110,7 +163,7 @@ where
}
}
impl<T> std::iter::FusedIterator for LineDiagnosticsIterator<'_, T> where T: Clone + Ranged {}
impl<T> std::iter::FusedIterator for LineDiagnosticsIterator<'_, T> where T: Diagnostic {}
/// All diagnostics that start on a single line of source code in one embedded Python file.
#[derive(Debug)]
@@ -139,11 +192,13 @@ struct DiagnosticWithLine<T> {
#[cfg(test)]
mod tests {
use crate::db::Db;
use crate::diagnostic::Diagnostic;
use ruff_db::files::system_path_to_file;
use ruff_db::source::line_index;
use ruff_db::system::{DbWithTestSystem, SystemPathBuf};
use ruff_source_file::OneIndexed;
use ruff_text_size::{TextRange, TextSize};
use std::borrow::Cow;
#[test]
fn sort_and_group() {
@@ -152,13 +207,18 @@ mod tests {
let file = system_path_to_file(&db, "/src/test.py").unwrap();
let lines = line_index(&db, file);
let ranges = vec![
let ranges = [
TextRange::new(TextSize::new(0), TextSize::new(1)),
TextRange::new(TextSize::new(5), TextSize::new(10)),
TextRange::new(TextSize::new(1), TextSize::new(7)),
];
let sorted = super::SortedDiagnostics::new(&ranges, &lines);
let diagnostics: Vec<_> = ranges
.into_iter()
.map(|range| DummyDiagnostic { range })
.collect();
let sorted = super::SortedDiagnostics::new(diagnostics, &lines);
let grouped = sorted.iter_lines().collect::<Vec<_>>();
let [line1, line2] = &grouped[..] else {
@@ -170,4 +230,23 @@ mod tests {
assert_eq!(line2.line_number, OneIndexed::from_zero_indexed(1));
assert_eq!(line2.diagnostics.len(), 1);
}
#[derive(Debug)]
struct DummyDiagnostic {
range: TextRange,
}
impl Diagnostic for DummyDiagnostic {
fn rule(&self) -> &str {
"dummy"
}
fn message(&self) -> Cow<str> {
"dummy".into()
}
fn range(&self) -> TextRange {
self.range
}
}
}

View File

@@ -1,3 +1,4 @@
use crate::diagnostic::Diagnostic;
use colored::Colorize;
use parser as test_parser;
use red_knot_python_semantic::types::check_types;
@@ -7,6 +8,7 @@ use ruff_db::system::{DbWithTestSystem, SystemPathBuf};
use ruff_source_file::LineIndex;
use ruff_text_size::TextSize;
use std::path::Path;
use std::sync::Arc;
mod assertion;
mod db;
@@ -87,16 +89,23 @@ fn run_test(db: &mut db::Db, test: &parser::MarkdownTest) -> Result<(), Failures
.filter_map(|test_file| {
let parsed = parsed_module(db, test_file.file);
// TODO allow testing against code with syntax errors
assert!(
parsed.errors().is_empty(),
"Python syntax errors in {}, {}: {:?}",
test.name(),
test_file.file.path(db),
parsed.errors()
);
let mut diagnostics: Vec<Box<_>> = parsed
.errors()
.iter()
.cloned()
.map(|error| {
let diagnostic: Box<dyn Diagnostic> = Box::new(error);
diagnostic
})
.collect();
match matcher::match_file(db, test_file.file, check_types(db, test_file.file)) {
let type_diagnostics = check_types(db, test_file.file);
diagnostics.extend(type_diagnostics.into_iter().map(|diagnostic| {
let diagnostic: Box<dyn Diagnostic> = Box::new(Arc::unwrap_or_clone(diagnostic));
diagnostic
}));
match matcher::match_file(db, test_file.file, diagnostics) {
Ok(()) => None,
Err(line_failures) => Some(FileFailures {
backtick_offset: test_file.backtick_offset,

View File

@@ -1,17 +1,14 @@
//! Match [`TypeCheckDiagnostic`]s against [`Assertion`]s and produce test failure messages for any
//! Match [`Diagnostic`]s against [`Assertion`]s and produce test failure messages for any
//! mismatches.
use crate::assertion::{Assertion, ErrorAssertion, InlineFileAssertions};
use crate::db::Db;
use crate::diagnostic::SortedDiagnostics;
use crate::diagnostic::{Diagnostic, SortedDiagnostics};
use colored::Colorize;
use red_knot_python_semantic::types::TypeCheckDiagnostic;
use ruff_db::files::File;
use ruff_db::source::{line_index, source_text, SourceText};
use ruff_source_file::{LineIndex, OneIndexed};
use ruff_text_size::Ranged;
use std::cmp::Ordering;
use std::ops::Range;
use std::sync::Arc;
#[derive(Debug, Default)]
pub(super) struct FailuresByLine {
@@ -55,7 +52,7 @@ pub(super) fn match_file<T>(
diagnostics: impl IntoIterator<Item = T>,
) -> Result<(), FailuresByLine>
where
T: Diagnostic + Clone,
T: Diagnostic,
{
// Parse assertions from comments in the file, and get diagnostics from the file; both
// ordered by line number.
@@ -126,22 +123,6 @@ where
}
}
pub(super) trait Diagnostic: Ranged {
fn rule(&self) -> &str;
fn message(&self) -> &str;
}
impl Diagnostic for Arc<TypeCheckDiagnostic> {
fn rule(&self) -> &str {
self.as_ref().rule()
}
fn message(&self) -> &str {
self.as_ref().message()
}
}
trait Unmatched {
fn unmatched(&self) -> String;
}
@@ -253,9 +234,9 @@ impl Matcher {
}
}
fn column<T: Ranged>(&self, ranged: &T) -> OneIndexed {
fn column<T: Diagnostic>(&self, diagnostic: &T) -> OneIndexed {
self.line_index
.source_location(ranged.start(), &self.source)
.source_location(diagnostic.range().start(), &self.source)
.column
}
@@ -323,11 +304,13 @@ impl Matcher {
#[cfg(test)]
mod tests {
use super::FailuresByLine;
use crate::diagnostic::Diagnostic;
use ruff_db::files::system_path_to_file;
use ruff_db::system::{DbWithTestSystem, SystemPathBuf};
use ruff_python_trivia::textwrap::dedent;
use ruff_source_file::OneIndexed;
use ruff_text_size::{Ranged, TextRange};
use ruff_text_size::TextRange;
use std::borrow::Cow;
#[derive(Clone, Debug)]
struct TestDiagnostic {
@@ -347,18 +330,16 @@ mod tests {
}
}
impl super::Diagnostic for TestDiagnostic {
impl Diagnostic for TestDiagnostic {
fn rule(&self) -> &str {
self.rule
}
fn message(&self) -> &str {
self.message
fn message(&self) -> Cow<str> {
self.message.into()
}
}
impl Ranged for TestDiagnostic {
fn range(&self) -> ruff_text_size::TextRange {
fn range(&self) -> TextRange {
self.range
}
}

View File

@@ -148,7 +148,7 @@ static HEADER_RE: LazyLock<Regex> =
/// Matches a code block fenced by triple backticks, possibly with language and `key=val`
/// configuration items following the opening backticks (in the "tag string" of the code block).
static CODE_RE: LazyLock<Regex> = LazyLock::new(|| {
Regex::new(r"^```(?<lang>(?-u:\w)+)?(?<config>(?: +\S+)*)\s*\n(?<code>(?:.|\n)*?)\n?```\s*\n")
Regex::new(r"^```(?<lang>(?-u:\w)+)?(?<config>(?: +\S+)*)\s*\n(?<code>(?:.|\n)*?)\n?```\s*\n?")
.unwrap()
});
@@ -421,6 +421,31 @@ mod tests {
assert_eq!(file.code, "x = 1");
}
#[test]
fn no_new_line_at_eof() {
let source = dedent(
"
```py
x = 1
```",
);
let mf = super::parse("file.md", &source).unwrap();
let [test] = &mf.tests().collect::<Vec<_>>()[..] else {
panic!("expected one test");
};
assert_eq!(test.name(), "file.md");
let [file] = test.files().collect::<Vec<_>>()[..] else {
panic!("expected one file");
};
assert_eq!(file.path, "test.py");
assert_eq!(file.lang, "py");
assert_eq!(file.code, "x = 1");
}
#[test]
fn multiple_tests() {
let source = dedent(

View File

@@ -1 +1 @@
a871efd90ca2734b3341dde98cffab66f3e08cee
d262beb07502cda412db2179fb406d45d1a9486f

View File

@@ -22,6 +22,7 @@ __main__: 3.0-
_ast: 3.0-
_asyncio: 3.0-
_bisect: 3.0-
_blake2: 3.6-
_bootlocale: 3.4-3.9
_codecs: 3.0-
_collections_abc: 3.3-
@@ -33,6 +34,8 @@ _curses: 3.0-
_decimal: 3.3-
_dummy_thread: 3.0-3.8
_dummy_threading: 3.0-3.8
_frozen_importlib: 3.0-
_frozen_importlib_external: 3.5-
_heapq: 3.0-
_imp: 3.0-
_interpchannels: 3.13-
@@ -160,6 +163,8 @@ imghdr: 3.0-3.12
imp: 3.0-3.11
importlib: 3.0-
importlib._abc: 3.10-
importlib._bootstrap: 3.0-
importlib._bootstrap_external: 3.5-
importlib.metadata: 3.8-
importlib.metadata._meta: 3.10-
importlib.metadata.diagnose: 3.13-

View File

@@ -1,6 +1,6 @@
import sys
from asyncio.events import AbstractEventLoop
from collections.abc import Awaitable, Callable, Coroutine, Generator, Iterable
from collections.abc import Awaitable, Callable, Coroutine, Generator
from contextvars import Context
from types import FrameType
from typing import Any, Literal, TextIO, TypeVar
@@ -13,7 +13,7 @@ _T = TypeVar("_T")
_T_co = TypeVar("_T_co", covariant=True)
_TaskYieldType: TypeAlias = Future[object] | None
class Future(Awaitable[_T], Iterable[_T]):
class Future(Awaitable[_T]):
_state: str
@property
def _exception(self) -> BaseException | None: ...

View File

@@ -0,0 +1,117 @@
import sys
from _typeshed import ReadableBuffer
from typing import ClassVar, final
from typing_extensions import Self
BLAKE2B_MAX_DIGEST_SIZE: int = 64
BLAKE2B_MAX_KEY_SIZE: int = 64
BLAKE2B_PERSON_SIZE: int = 16
BLAKE2B_SALT_SIZE: int = 16
BLAKE2S_MAX_DIGEST_SIZE: int = 32
BLAKE2S_MAX_KEY_SIZE: int = 32
BLAKE2S_PERSON_SIZE: int = 8
BLAKE2S_SALT_SIZE: int = 8
@final
class blake2b:
MAX_DIGEST_SIZE: ClassVar[int] = 64
MAX_KEY_SIZE: ClassVar[int] = 64
PERSON_SIZE: ClassVar[int] = 16
SALT_SIZE: ClassVar[int] = 16
block_size: int
digest_size: int
name: str
if sys.version_info >= (3, 9):
def __init__(
self,
data: ReadableBuffer = b"",
/,
*,
digest_size: int = 64,
key: ReadableBuffer = b"",
salt: ReadableBuffer = b"",
person: ReadableBuffer = b"",
fanout: int = 1,
depth: int = 1,
leaf_size: int = 0,
node_offset: int = 0,
node_depth: int = 0,
inner_size: int = 0,
last_node: bool = False,
usedforsecurity: bool = True,
) -> None: ...
else:
def __init__(
self,
data: ReadableBuffer = b"",
/,
*,
digest_size: int = 64,
key: ReadableBuffer = b"",
salt: ReadableBuffer = b"",
person: ReadableBuffer = b"",
fanout: int = 1,
depth: int = 1,
leaf_size: int = 0,
node_offset: int = 0,
node_depth: int = 0,
inner_size: int = 0,
last_node: bool = False,
) -> None: ...
def copy(self) -> Self: ...
def digest(self) -> bytes: ...
def hexdigest(self) -> str: ...
def update(self, data: ReadableBuffer, /) -> None: ...
@final
class blake2s:
MAX_DIGEST_SIZE: ClassVar[int] = 32
MAX_KEY_SIZE: ClassVar[int] = 32
PERSON_SIZE: ClassVar[int] = 8
SALT_SIZE: ClassVar[int] = 8
block_size: int
digest_size: int
name: str
if sys.version_info >= (3, 9):
def __init__(
self,
data: ReadableBuffer = b"",
/,
*,
digest_size: int = 32,
key: ReadableBuffer = b"",
salt: ReadableBuffer = b"",
person: ReadableBuffer = b"",
fanout: int = 1,
depth: int = 1,
leaf_size: int = 0,
node_offset: int = 0,
node_depth: int = 0,
inner_size: int = 0,
last_node: bool = False,
usedforsecurity: bool = True,
) -> None: ...
else:
def __init__(
self,
data: ReadableBuffer = b"",
/,
*,
digest_size: int = 32,
key: ReadableBuffer = b"",
salt: ReadableBuffer = b"",
person: ReadableBuffer = b"",
fanout: int = 1,
depth: int = 1,
leaf_size: int = 0,
node_offset: int = 0,
node_depth: int = 0,
inner_size: int = 0,
last_node: bool = False,
) -> None: ...
def copy(self) -> Self: ...
def digest(self) -> bytes: ...
def hexdigest(self) -> str: ...
def update(self, data: ReadableBuffer, /) -> None: ...

View File

@@ -1,3 +1,4 @@
import csv
import sys
from _typeshed import SupportsWrite
from collections.abc import Iterable, Iterator
@@ -20,7 +21,7 @@ _QuotingType: TypeAlias = int
class Error(Exception): ...
_DialectLike: TypeAlias = str | Dialect | type[Dialect]
_DialectLike: TypeAlias = str | Dialect | csv.Dialect | type[Dialect | csv.Dialect]
class Dialect:
delimiter: str

View File

@@ -0,0 +1,112 @@
import importlib.abc
import importlib.machinery
import sys
import types
from _typeshed.importlib import LoaderProtocol
from collections.abc import Mapping, Sequence
from types import ModuleType
from typing import Any
# Signature of `builtins.__import__` should be kept identical to `importlib.__import__`
def __import__(
name: str,
globals: Mapping[str, object] | None = None,
locals: Mapping[str, object] | None = None,
fromlist: Sequence[str] = (),
level: int = 0,
) -> ModuleType: ...
def spec_from_loader(
name: str, loader: LoaderProtocol | None, *, origin: str | None = None, is_package: bool | None = None
) -> importlib.machinery.ModuleSpec | None: ...
def module_from_spec(spec: importlib.machinery.ModuleSpec) -> types.ModuleType: ...
def _init_module_attrs(
spec: importlib.machinery.ModuleSpec, module: types.ModuleType, *, override: bool = False
) -> types.ModuleType: ...
class ModuleSpec:
def __init__(
self,
name: str,
loader: importlib.abc.Loader | None,
*,
origin: str | None = None,
loader_state: Any = None,
is_package: bool | None = None,
) -> None: ...
name: str
loader: importlib.abc.Loader | None
origin: str | None
submodule_search_locations: list[str] | None
loader_state: Any
cached: str | None
@property
def parent(self) -> str | None: ...
has_location: bool
def __eq__(self, other: object) -> bool: ...
class BuiltinImporter(importlib.abc.MetaPathFinder, importlib.abc.InspectLoader):
# MetaPathFinder
if sys.version_info < (3, 12):
@classmethod
def find_module(cls, fullname: str, path: Sequence[str] | None = None) -> importlib.abc.Loader | None: ...
@classmethod
def find_spec(
cls, fullname: str, path: Sequence[str] | None = None, target: types.ModuleType | None = None
) -> ModuleSpec | None: ...
# InspectLoader
@classmethod
def is_package(cls, fullname: str) -> bool: ...
@classmethod
def load_module(cls, fullname: str) -> types.ModuleType: ...
@classmethod
def get_code(cls, fullname: str) -> None: ...
@classmethod
def get_source(cls, fullname: str) -> None: ...
# Loader
if sys.version_info < (3, 12):
@staticmethod
def module_repr(module: types.ModuleType) -> str: ...
if sys.version_info >= (3, 10):
@staticmethod
def create_module(spec: ModuleSpec) -> types.ModuleType | None: ...
@staticmethod
def exec_module(module: types.ModuleType) -> None: ...
else:
@classmethod
def create_module(cls, spec: ModuleSpec) -> types.ModuleType | None: ...
@classmethod
def exec_module(cls, module: types.ModuleType) -> None: ...
class FrozenImporter(importlib.abc.MetaPathFinder, importlib.abc.InspectLoader):
# MetaPathFinder
if sys.version_info < (3, 12):
@classmethod
def find_module(cls, fullname: str, path: Sequence[str] | None = None) -> importlib.abc.Loader | None: ...
@classmethod
def find_spec(
cls, fullname: str, path: Sequence[str] | None = None, target: types.ModuleType | None = None
) -> ModuleSpec | None: ...
# InspectLoader
@classmethod
def is_package(cls, fullname: str) -> bool: ...
@classmethod
def load_module(cls, fullname: str) -> types.ModuleType: ...
@classmethod
def get_code(cls, fullname: str) -> None: ...
@classmethod
def get_source(cls, fullname: str) -> None: ...
# Loader
if sys.version_info < (3, 12):
@staticmethod
def module_repr(m: types.ModuleType) -> str: ...
if sys.version_info >= (3, 10):
@staticmethod
def create_module(spec: ModuleSpec) -> types.ModuleType | None: ...
else:
@classmethod
def create_module(cls, spec: ModuleSpec) -> types.ModuleType | None: ...
@staticmethod
def exec_module(module: types.ModuleType) -> None: ...

View File

@@ -0,0 +1,178 @@
import _ast
import _io
import importlib.abc
import importlib.machinery
import sys
import types
from _typeshed import ReadableBuffer, StrOrBytesPath, StrPath
from _typeshed.importlib import LoaderProtocol
from collections.abc import Callable, Iterable, Iterator, Mapping, MutableSequence, Sequence
from importlib.machinery import ModuleSpec
from importlib.metadata import DistributionFinder, PathDistribution
from typing import Any, Literal
from typing_extensions import Self, deprecated
if sys.version_info >= (3, 10):
import importlib.readers
if sys.platform == "win32":
path_separators: Literal["\\/"]
path_sep: Literal["\\"]
path_sep_tuple: tuple[Literal["\\"], Literal["/"]]
else:
path_separators: Literal["/"]
path_sep: Literal["/"]
path_sep_tuple: tuple[Literal["/"]]
MAGIC_NUMBER: bytes
def cache_from_source(path: str, debug_override: bool | None = None, *, optimization: Any | None = None) -> str: ...
def source_from_cache(path: str) -> str: ...
def decode_source(source_bytes: ReadableBuffer) -> str: ...
def spec_from_file_location(
name: str,
location: StrOrBytesPath | None = None,
*,
loader: LoaderProtocol | None = None,
submodule_search_locations: list[str] | None = ...,
) -> importlib.machinery.ModuleSpec | None: ...
class WindowsRegistryFinder(importlib.abc.MetaPathFinder):
if sys.version_info < (3, 12):
@classmethod
def find_module(cls, fullname: str, path: Sequence[str] | None = None) -> importlib.abc.Loader | None: ...
@classmethod
def find_spec(
cls, fullname: str, path: Sequence[str] | None = None, target: types.ModuleType | None = None
) -> ModuleSpec | None: ...
class PathFinder(importlib.abc.MetaPathFinder):
if sys.version_info >= (3, 10):
@staticmethod
def invalidate_caches() -> None: ...
else:
@classmethod
def invalidate_caches(cls) -> None: ...
if sys.version_info >= (3, 10):
@staticmethod
def find_distributions(context: DistributionFinder.Context = ...) -> Iterable[PathDistribution]: ...
else:
@classmethod
def find_distributions(cls, context: DistributionFinder.Context = ...) -> Iterable[PathDistribution]: ...
@classmethod
def find_spec(
cls, fullname: str, path: Sequence[str] | None = None, target: types.ModuleType | None = None
) -> ModuleSpec | None: ...
if sys.version_info < (3, 12):
@classmethod
def find_module(cls, fullname: str, path: Sequence[str] | None = None) -> importlib.abc.Loader | None: ...
SOURCE_SUFFIXES: list[str]
DEBUG_BYTECODE_SUFFIXES: list[str]
OPTIMIZED_BYTECODE_SUFFIXES: list[str]
BYTECODE_SUFFIXES: list[str]
EXTENSION_SUFFIXES: list[str]
class FileFinder(importlib.abc.PathEntryFinder):
path: str
def __init__(self, path: str, *loader_details: tuple[type[importlib.abc.Loader], list[str]]) -> None: ...
@classmethod
def path_hook(
cls, *loader_details: tuple[type[importlib.abc.Loader], list[str]]
) -> Callable[[str], importlib.abc.PathEntryFinder]: ...
class _LoaderBasics:
def is_package(self, fullname: str) -> bool: ...
def create_module(self, spec: ModuleSpec) -> types.ModuleType | None: ...
def exec_module(self, module: types.ModuleType) -> None: ...
def load_module(self, fullname: str) -> types.ModuleType: ...
class SourceLoader(_LoaderBasics):
def path_mtime(self, path: str) -> float: ...
def set_data(self, path: str, data: bytes) -> None: ...
def get_source(self, fullname: str) -> str | None: ...
def path_stats(self, path: str) -> Mapping[str, Any]: ...
def source_to_code(
self, data: ReadableBuffer | str | _ast.Module | _ast.Expression | _ast.Interactive, path: ReadableBuffer | StrPath
) -> types.CodeType: ...
def get_code(self, fullname: str) -> types.CodeType | None: ...
class FileLoader:
name: str
path: str
def __init__(self, fullname: str, path: str) -> None: ...
def get_data(self, path: str) -> bytes: ...
def get_filename(self, name: str | None = None) -> str: ...
def load_module(self, name: str | None = None) -> types.ModuleType: ...
if sys.version_info >= (3, 10):
def get_resource_reader(self, module: types.ModuleType) -> importlib.readers.FileReader: ...
else:
def get_resource_reader(self, module: types.ModuleType) -> Self | None: ...
def open_resource(self, resource: str) -> _io.FileIO: ...
def resource_path(self, resource: str) -> str: ...
def is_resource(self, name: str) -> bool: ...
def contents(self) -> Iterator[str]: ...
class SourceFileLoader(importlib.abc.FileLoader, FileLoader, importlib.abc.SourceLoader, SourceLoader): # type: ignore[misc] # incompatible method arguments in base classes
def set_data(self, path: str, data: ReadableBuffer, *, _mode: int = 0o666) -> None: ...
def path_stats(self, path: str) -> Mapping[str, Any]: ...
class SourcelessFileLoader(importlib.abc.FileLoader, FileLoader, _LoaderBasics):
def get_code(self, fullname: str) -> types.CodeType | None: ...
def get_source(self, fullname: str) -> None: ...
class ExtensionFileLoader(FileLoader, _LoaderBasics, importlib.abc.ExecutionLoader):
def __init__(self, name: str, path: str) -> None: ...
def get_filename(self, name: str | None = None) -> str: ...
def get_source(self, fullname: str) -> None: ...
def create_module(self, spec: ModuleSpec) -> types.ModuleType: ...
def exec_module(self, module: types.ModuleType) -> None: ...
def get_code(self, fullname: str) -> None: ...
def __eq__(self, other: object) -> bool: ...
def __hash__(self) -> int: ...
if sys.version_info >= (3, 11):
class NamespaceLoader(importlib.abc.InspectLoader):
def __init__(
self, name: str, path: MutableSequence[str], path_finder: Callable[[str, tuple[str, ...]], ModuleSpec]
) -> None: ...
def is_package(self, fullname: str) -> Literal[True]: ...
def get_source(self, fullname: str) -> Literal[""]: ...
def get_code(self, fullname: str) -> types.CodeType: ...
def create_module(self, spec: ModuleSpec) -> None: ...
def exec_module(self, module: types.ModuleType) -> None: ...
@deprecated("load_module() is deprecated; use exec_module() instead")
def load_module(self, fullname: str) -> types.ModuleType: ...
def get_resource_reader(self, module: types.ModuleType) -> importlib.readers.NamespaceReader: ...
if sys.version_info < (3, 12):
@staticmethod
@deprecated("module_repr() is deprecated, and has been removed in Python 3.12")
def module_repr(module: types.ModuleType) -> str: ...
_NamespaceLoader = NamespaceLoader
else:
class _NamespaceLoader:
def __init__(
self, name: str, path: MutableSequence[str], path_finder: Callable[[str, tuple[str, ...]], ModuleSpec]
) -> None: ...
def is_package(self, fullname: str) -> Literal[True]: ...
def get_source(self, fullname: str) -> Literal[""]: ...
def get_code(self, fullname: str) -> types.CodeType: ...
def create_module(self, spec: ModuleSpec) -> None: ...
def exec_module(self, module: types.ModuleType) -> None: ...
@deprecated("load_module() is deprecated; use exec_module() instead")
def load_module(self, fullname: str) -> types.ModuleType: ...
if sys.version_info >= (3, 10):
@staticmethod
@deprecated("module_repr() is deprecated, and has been removed in Python 3.12")
def module_repr(module: types.ModuleType) -> str: ...
def get_resource_reader(self, module: types.ModuleType) -> importlib.readers.NamespaceReader: ...
else:
@classmethod
@deprecated("module_repr() is deprecated, and has been removed in Python 3.12")
def module_repr(cls, module: types.ModuleType) -> str: ...
if sys.version_info >= (3, 13):
class AppleFrameworkLoader(ExtensionFileLoader, importlib.abc.ExecutionLoader): ...

View File

@@ -7,7 +7,7 @@ _Configs: TypeAlias = Literal["default", "isolated", "legacy", "empty", ""]
class InterpreterError(Exception): ...
class InterpreterNotFoundError(InterpreterError): ...
class NotShareableError(Exception): ...
class NotShareableError(ValueError): ...
class CrossInterpreterBufferView:
def __buffer__(self, flags: int, /) -> memoryview: ...

View File

@@ -86,19 +86,24 @@ class BytesIO(BufferedIOBase, _BufferedIOBase, BinaryIO): # type: ignore[misc]
class BufferedReader(BufferedIOBase, _BufferedIOBase, BinaryIO): # type: ignore[misc] # incompatible definitions of methods in the base classes
raw: RawIOBase
def __init__(self, raw: RawIOBase, buffer_size: int = ...) -> None: ...
def __init__(self, raw: RawIOBase, buffer_size: int = 8192) -> None: ...
def peek(self, size: int = 0, /) -> bytes: ...
class BufferedWriter(BufferedIOBase, _BufferedIOBase, BinaryIO): # type: ignore[misc] # incompatible definitions of writelines in the base classes
raw: RawIOBase
def __init__(self, raw: RawIOBase, buffer_size: int = ...) -> None: ...
def __init__(self, raw: RawIOBase, buffer_size: int = 8192) -> None: ...
def write(self, buffer: ReadableBuffer, /) -> int: ...
class BufferedRandom(BufferedReader, BufferedWriter, BufferedIOBase, _BufferedIOBase): # type: ignore[misc] # incompatible definitions of methods in the base classes
class BufferedRandom(BufferedIOBase, _BufferedIOBase, BinaryIO): # type: ignore[misc] # incompatible definitions of methods in the base classes
mode: str
name: Any
raw: RawIOBase
def __init__(self, raw: RawIOBase, buffer_size: int = 8192) -> None: ...
def seek(self, target: int, whence: int = 0, /) -> int: ... # stubtest needs this
def peek(self, size: int = 0, /) -> bytes: ...
class BufferedRWPair(BufferedIOBase, _BufferedIOBase):
def __init__(self, reader: RawIOBase, writer: RawIOBase, buffer_size: int = ...) -> None: ...
def __init__(self, reader: RawIOBase, writer: RawIOBase, buffer_size: int = 8192) -> None: ...
def peek(self, size: int = ..., /) -> bytes: ...
class _TextIOBase(_IOBase):
@@ -173,19 +178,23 @@ class TextIOWrapper(TextIOBase, _TextIOBase, TextIO, Generic[_BufferT_co]): # t
# operations.
def seek(self, cookie: int, whence: int = 0, /) -> int: ...
class StringIO(TextIOWrapper, TextIOBase, _TextIOBase): # type: ignore[misc] # incompatible definitions of write in the base classes
class StringIO(TextIOBase, _TextIOBase, TextIO): # type: ignore[misc] # incompatible definitions of write in the base classes
def __init__(self, initial_value: str | None = ..., newline: str | None = ...) -> None: ...
# StringIO does not contain a "name" field. This workaround is necessary
# to allow StringIO sub-classes to add this field, as it is defined
# as a read-only property on IO[].
name: Any
def getvalue(self) -> str: ...
@property
def line_buffering(self) -> bool: ...
class IncrementalNewlineDecoder(codecs.IncrementalDecoder):
class IncrementalNewlineDecoder:
def __init__(self, decoder: codecs.IncrementalDecoder | None, translate: bool, errors: str = ...) -> None: ...
def decode(self, input: ReadableBuffer | str, final: bool = False) -> str: ...
@property
def newlines(self) -> str | tuple[str, ...] | None: ...
def getstate(self) -> tuple[bytes, int]: ...
def reset(self) -> None: ...
def setstate(self, state: tuple[bytes, int], /) -> None: ...
if sys.version_info >= (3, 10):

Some files were not shown because too many files have changed in this diff Show More