Bump version to Ruff 0.9.9 (#16434)

## Summary

This PR updates the ordering of changelog sections to prioritize `bug`
label such that any PRs that has that label is categorized in "Bug
fixes" section in when generating the changelog irrespective of any
other labels present on the PR.

I think this works because I've seen PRs with both `server` and `bug` in
the "Server" section instead of the "Bug fixes" section. For example,
https://github.com/astral-sh/ruff/pull/16262 in
https://github.com/astral-sh/ruff/releases/tag/0.9.7.

On that note, this also changes the ordering such that any PR with both
`server` and `bug` labels are in the "Bug fixes" section instead of the
"Server" section. This is in line with how "Formatter" is done. I think
it makes sense to instead prefix the entries with "Formatter:" and
"Server:" if they're bug fixes. But, I'm happy to change this such that
any PRs with `formatter` and `server` labels are always in their own
section irrespective of other labels.

.github/ISSUE_TEMPLATE/1_bug_report.yaml

@@ -0,0 +1,31 @@
+name: Bug report
+description: Report an error or unexpected behavior
+body:
+  - type: markdown
+    attributes:
+      value: |
+        Thank you for taking the time to report an issue! We're glad to have you involved with Ruff.
+
+        **Before reporting, please make sure to search through [existing issues](https://github.com/astral-sh/ruff/issues?q=is:issue+is:open+label:bug) (including [closed](https://github.com/astral-sh/ruff/issues?q=is:issue%20state:closed%20label:bug)).**
+
+  - type: textarea
+    attributes:
+      label: Summary
+      description: |
+        A clear and concise description of the bug, including a minimal reproducible example.
+
+        Be sure to include the command you invoked (e.g., `ruff check /path/to/file.py --fix`), ideally including the `--isolated` flag and
+        the current Ruff settings (e.g., relevant sections from your `pyproject.toml`).
+
+        If possible, try to include the [playground](https://play.ruff.rs) link that reproduces this issue.
+
+    validations:
+      required: true
+
+  - type: input
+    attributes:
+      label: Version
+      description: What version of ruff are you using? (see `ruff version`)
+      placeholder: e.g., ruff 0.9.3 (90589372d 2025-01-23)
+    validations:
+      required: false

.github/ISSUE_TEMPLATE/2_rule_request.yaml

@@ -0,0 +1,10 @@
+name: Rule request
+description: Anything related to lint rules (proposing new rules, changes to existing rules, auto-fixes, etc.)
+body:
+  - type: textarea
+    attributes:
+      label: Summary
+      description: |
+        A clear and concise description of the relevant request. If applicable, please describe the current behavior as well.
+    validations:
+      required: true

.github/ISSUE_TEMPLATE/3_question.yaml

@@ -0,0 +1,18 @@
+name: Question
+description: Ask a question about Ruff
+labels: ["question"]
+body:
+  - type: textarea
+    attributes:
+      label: Question
+      description: Describe your question in detail.
+    validations:
+      required: true
+
+  - type: input
+    attributes:
+      label: Version
+      description: What version of ruff are you using? (see `ruff version`)
+      placeholder: e.g., ruff 0.9.3 (90589372d 2025-01-23)
+    validations:
+      required: false

.github/ISSUE_TEMPLATE/config.yml

@@ -1,2 +1,8 @@
-# This file cannot use the extension `.yaml`.
-blank_issues_enabled: false
+blank_issues_enabled: true
+contact_links:
+  - name: Documentation
+    url: https://docs.astral.sh/ruff
+    about: Please consult the documentation before creating an issue.
+  - name: Community
+    url: https://discord.com/invite/astral-sh
+    about: Join our Discord community to ask questions and collaborate.

.github/ISSUE_TEMPLATE/issue.yaml

@@ -1,22 +0,0 @@
-name: New issue
-description: A generic issue
-
-body:
-  - type: markdown
-    attributes:
-      value: |
-        Thank you for taking the time to report an issue! We're glad to have you involved with Ruff.
-
-        If you're filing a bug report, please consider including the following information:
-
-        * List of keywords you searched for before creating this issue. Write them down here so that others can find this issue more easily and help provide feedback.
-        e.g. "RUF001", "unused variable", "Jupyter notebook"
-        * A minimal code snippet that reproduces the bug.
-        * The command you invoked (e.g., `ruff /path/to/file.py --fix`), ideally including the `--isolated` flag.
-        * The current Ruff settings (any relevant sections from your `pyproject.toml`).
-        * The current Ruff version (`ruff --version`).
-
-  - type: textarea
-    attributes:
-      label: Description
-      description: A description of the issue

CHANGELOG.md

@@ -1,5 +1,51 @@
 # Changelog
 
+## 0.9.9
+
+### Preview features
+
+- Fix caching of unsupported-syntax errors ([#16425](https://github.com/astral-sh/ruff/pull/16425))
+
+### Bug fixes
+
+- Only show unsupported-syntax errors in editors when preview mode is enabled ([#16429](https://github.com/astral-sh/ruff/pull/16429))
+
+## 0.9.8
+
+### Preview features
+
+- Start detecting version-related syntax errors in the parser ([#16090](https://github.com/astral-sh/ruff/pull/16090))
+
+### Rule changes
+
+- \[`pylint`\] Mark fix unsafe (`PLW1507`) ([#16343](https://github.com/astral-sh/ruff/pull/16343))
+- \[`pylint`\] Catch `case np.nan`/`case math.nan` in `match` statements (`PLW0177`) ([#16378](https://github.com/astral-sh/ruff/pull/16378))
+- \[`ruff`\] Add more Pydantic models variants to the list of default copy semantics (`RUF012`) ([#16291](https://github.com/astral-sh/ruff/pull/16291))
+
+### Server
+
+- Avoid indexing the project if `configurationPreference` is `editorOnly` ([#16381](https://github.com/astral-sh/ruff/pull/16381))
+- Avoid unnecessary info at non-trace server log level ([#16389](https://github.com/astral-sh/ruff/pull/16389))
+- Expand `ruff.configuration` to allow inline config ([#16296](https://github.com/astral-sh/ruff/pull/16296))
+- Notify users for invalid client settings ([#16361](https://github.com/astral-sh/ruff/pull/16361))
+
+### Configuration
+
+- Add `per-file-target-version` option ([#16257](https://github.com/astral-sh/ruff/pull/16257))
+
+### Bug fixes
+
+- \[`refurb`\] Do not consider docstring(s) (`FURB156`) ([#16391](https://github.com/astral-sh/ruff/pull/16391))
+- \[`flake8-self`\] Ignore attribute accesses on instance-like variables (`SLF001`) ([#16149](https://github.com/astral-sh/ruff/pull/16149))
+- \[`pylint`\] Fix false positives, add missing methods, and support positional-only parameters (`PLE0302`) ([#16263](https://github.com/astral-sh/ruff/pull/16263))
+- \[`flake8-pyi`\] Mark `PYI030` fix unsafe when comments are deleted ([#16322](https://github.com/astral-sh/ruff/pull/16322))
+
+### Documentation
+
+- Fix example for `S611` ([#16316](https://github.com/astral-sh/ruff/pull/16316))
+- Normalize inconsistent markdown headings in docstrings ([#16364](https://github.com/astral-sh/ruff/pull/16364))
+- Document MSRV policy ([#16384](https://github.com/astral-sh/ruff/pull/16384))
+
 ## 0.9.7
 
 ### Preview features
@@ -13,16 +59,7 @@
 
 ### Rule changes
 
-- \[`flake8-comprehensions`\]: Handle trailing comma in `C403` fix ([#16110](https://github.com/astral-sh/ruff/pull/16110))
 - \[`flake8-debugger`\] Also flag `sys.breakpointhook` and `sys.__breakpointhook__` (`T100`) ([#16191](https://github.com/astral-sh/ruff/pull/16191))
-- \[`pydocstyle`\] Handle arguments with the same names as sections (`D417`) ([#16011](https://github.com/astral-sh/ruff/pull/16011))
-- \[`pylint`\] Correct ordering of arguments in fix for `if-stmt-min-max` (`PLR1730`) ([#16080](https://github.com/astral-sh/ruff/pull/16080))
-- \[`pylint`\] Do not offer fix for raw strings (`PLE251`) ([#16132](https://github.com/astral-sh/ruff/pull/16132))
-- \[`pyupgrade`\] Do not upgrade functional `TypedDicts` with private field names to the class-based syntax (`UP013`) ([#16219](https://github.com/astral-sh/ruff/pull/16219))
-- \[`pyupgrade`\] Handle micro version numbers correctly (`UP036`) ([#16091](https://github.com/astral-sh/ruff/pull/16091))
-- \[`pyupgrade`\] Unwrap unary expressions correctly (`UP018`) ([#15919](https://github.com/astral-sh/ruff/pull/15919))
-- \[`ruff`\] Skip `RUF001` diagnostics when visiting string type definitions ([#16122](https://github.com/astral-sh/ruff/pull/16122))
-- \[`flake8-pyi`\] Avoid flagging `custom-typevar-for-self` on metaclass methods (`PYI019`) ([#16141](https://github.com/astral-sh/ruff/pull/16141))
 - \[`pycodestyle`\] Exempt `site.addsitedir(...)` calls (`E402`) ([#16251](https://github.com/astral-sh/ruff/pull/16251))
 
 ### Formatter
@@ -43,7 +80,16 @@
 
 ### Bug fixes
 
+- \[`flake8-comprehensions`\] Handle trailing comma in `C403` fix ([#16110](https://github.com/astral-sh/ruff/pull/16110))
+- \[`flake8-pyi`\] Avoid flagging `custom-typevar-for-self` on metaclass methods (`PYI019`) ([#16141](https://github.com/astral-sh/ruff/pull/16141))
+- \[`pydocstyle`\] Handle arguments with the same names as sections (`D417`) ([#16011](https://github.com/astral-sh/ruff/pull/16011))
+- \[`pylint`\] Correct ordering of arguments in fix for `if-stmt-min-max` (`PLR1730`) ([#16080](https://github.com/astral-sh/ruff/pull/16080))
+- \[`pylint`\] Do not offer fix for raw strings (`PLE251`) ([#16132](https://github.com/astral-sh/ruff/pull/16132))
+- \[`pyupgrade`\] Do not upgrade functional `TypedDicts` with private field names to the class-based syntax (`UP013`) ([#16219](https://github.com/astral-sh/ruff/pull/16219))
+- \[`pyupgrade`\] Handle micro version numbers correctly (`UP036`) ([#16091](https://github.com/astral-sh/ruff/pull/16091))
+- \[`pyupgrade`\] Unwrap unary expressions correctly (`UP018`) ([#15919](https://github.com/astral-sh/ruff/pull/15919))
 - \[`refurb`\] Correctly handle lengths of literal strings in `slice-to-remove-prefix-or-suffix` (`FURB188`) ([#16237](https://github.com/astral-sh/ruff/pull/16237))
+- \[`ruff`\] Skip `RUF001` diagnostics when visiting string type definitions ([#16122](https://github.com/astral-sh/ruff/pull/16122))
 
 ### Documentation
 

Cargo.lock

@@ -8,18 +8,6 @@ version = "2.0.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "512761e0bb2578dd7380c6baaa0f4ce03e84f95e960231d1dec8bf4d7d6e2627"
 
-[[package]]
-name = "ahash"
-version = "0.8.11"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "e89da841a80418a9b391ebaea17f5c112ffaaa96f621d2c285b5174da76b9011"
-dependencies = [
- "cfg-if",
- "once_cell",
- "version_check",
- "zerocopy 0.7.35",
-]
-
 [[package]]
 name = "aho-corasick"
 version = "1.1.3"
@@ -136,21 +124,9 @@ dependencies = [
 
 [[package]]
 name = "anyhow"
-version = "1.0.95"
+version = "1.0.96"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "34ac096ce696dc2fcabef30516bb13c0a68a11d30131d3df6f04711467681b04"
-
-[[package]]
-name = "append-only-vec"
-version = "0.1.7"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "7992085ec035cfe96992dd31bfd495a2ebd31969bb95f624471cb6c0b349e571"
-
-[[package]]
-name = "arc-swap"
-version = "1.7.1"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "69f7f8c3906b62b754cd5326047894316021dcfe5a194c8ea52bdd94934a3457"
+checksum = "6b964d184e89d9b6b67dd2715bc8e74cf3107fb2b529990c90cf517326150bf4"
 
 [[package]]
 name = "argfile"
@@ -227,9 +203,12 @@ dependencies = [
 
 [[package]]
 name = "boxcar"
-version = "0.2.8"
+version = "0.2.10"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "2721c3c5a6f0e7f7e607125d963fedeb765f545f67adc9d71ed934693881eb42"
+checksum = "225450ee9328e1e828319b48a89726cffc1b0ad26fd9211ad435de9fa376acae"
+dependencies = [
+ "loom",
+]
 
 [[package]]
 name = "bstr"
@@ -360,9 +339,9 @@ dependencies = [
 
 [[package]]
 name = "clap"
-version = "4.5.29"
+version = "4.5.30"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "8acebd8ad879283633b343856142139f2da2317c96b05b4dd6181c61e2480184"
+checksum = "92b7b18d71fad5313a1e320fa9897994228ce274b60faa4d694fe0ea89cd9e6d"
 dependencies = [
  "clap_builder",
  "clap_derive",
@@ -370,9 +349,9 @@ dependencies = [
 
 [[package]]
 name = "clap_builder"
-version = "4.5.29"
+version = "4.5.30"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "f6ba32cbda51c7e1dfd49acc1457ba1a7dec5b64fe360e828acb13ca8dc9c2f9"
+checksum = "a35db2071778a7344791a4fb4f95308b5673d219dee3ae348b86642574ecc90c"
 dependencies = [
  "anstream",
  "anstyle",
@@ -1013,6 +992,19 @@ dependencies = [
  "libc",
 ]
 
+[[package]]
+name = "generator"
+version = "0.8.4"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "cc6bd114ceda131d3b1d665eba35788690ad37f5916457286b32ab6fd3c438dd"
+dependencies = [
+ "cfg-if",
+ "libc",
+ "log",
+ "rustversion",
+ "windows",
+]
+
 [[package]]
 name = "generic-array"
 version = "0.14.7"
@@ -1102,10 +1094,6 @@ name = "hashbrown"
 version = "0.14.5"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "e5274423e17b7c9fc20b6e7e208532f9b19825d82dfd615708b70edd83df41f1"
-dependencies = [
- "ahash",
- "allocator-api2",
-]
 
 [[package]]
 name = "hashbrown"
@@ -1113,17 +1101,18 @@ version = "0.15.2"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "bf151400ff0baff5465007dd2f3e717f3fe502074ca563069ce3a6629d07b289"
 dependencies = [
+ "allocator-api2",
  "equivalent",
  "foldhash",
 ]
 
 [[package]]
 name = "hashlink"
-version = "0.9.1"
+version = "0.10.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "6ba4ff7128dee98c7dc9794b6a411377e1404dba1c97deb8d1a55297bd25d8af"
+checksum = "7382cf6263419f2d8df38c55d7da83da5c18aef87fc7a7fc1fb1e344edfe14c1"
 dependencies = [
- "hashbrown 0.14.5",
+ "hashbrown 0.15.2",
 ]
 
 [[package]]
@@ -1179,7 +1168,7 @@ dependencies = [
  "iana-time-zone-haiku",
  "js-sys",
  "wasm-bindgen",
- "windows-core",
+ "windows-core 0.52.0",
 ]
 
 [[package]]
@@ -1587,9 +1576,9 @@ checksum = "bbd2bcb4c963f2ddae06a2efc7e9f3591312473c50c6685e1f298068316e66fe"
 
 [[package]]
 name = "libc"
-version = "0.2.169"
+version = "0.2.170"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "b5aba8db14291edd000dfcc4d620c7ebfb122c613afb886ca8803fa4e128a20a"
+checksum = "875b3680cb2f8f71bdcf9a30f38d48282f5d3c95cbf9b3fa57269bb5d5c06828"
 
 [[package]]
 name = "libcst"
@@ -1679,9 +1668,22 @@ dependencies = [
 
 [[package]]
 name = "log"
-version = "0.4.25"
+version = "0.4.26"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "04cbf5b083de1c7e0222a7a51dbfdba1cbe1c6ab0b15e29fff3f6c077fd9cd9f"
+checksum = "30bde2b3dc3671ae49d8e2e9f044c7c005836e7a023ee57cffa25ab82764bb9e"
+
+[[package]]
+name = "loom"
+version = "0.7.2"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "419e0dc8046cb947daa77eb95ae174acfbddb7673b4151f56d1eed8e93fbfaca"
+dependencies = [
+ "cfg-if",
+ "generator",
+ "scoped-tls",
+ "tracing",
+ "tracing-subscriber",
+]
 
 [[package]]
 name = "lsp-server"
@@ -2401,6 +2403,7 @@ name = "red_knot"
 version = "0.0.0"
 dependencies = [
  "anyhow",
+ "argfile",
  "chrono",
  "clap",
  "colored 3.0.0",
@@ -2425,6 +2428,7 @@ dependencies = [
  "tracing-flame",
  "tracing-subscriber",
  "tracing-tree",
+ "wild",
 ]
 
 [[package]]
@@ -2491,6 +2495,8 @@ dependencies = [
  "serde",
  "smallvec",
  "static_assertions",
+ "strum",
+ "strum_macros",
  "tempfile",
  "test-case",
  "thiserror 2.0.11",
@@ -2650,7 +2656,7 @@ dependencies = [
 
 [[package]]
 name = "ruff"
-version = "0.9.7"
+version = "0.9.9"
 dependencies = [
  "anyhow",
  "argfile",
@@ -2884,7 +2890,7 @@ dependencies = [
 
 [[package]]
 name = "ruff_linter"
-version = "0.9.7"
+version = "0.9.9"
 dependencies = [
  "aho-corasick",
  "anyhow",
@@ -3084,6 +3090,8 @@ dependencies = [
  "ruff_source_file",
  "ruff_text_size",
  "rustc-hash 2.1.1",
+ "serde",
+ "serde_json",
  "static_assertions",
  "unicode-ident",
  "unicode-normalization",
@@ -3178,6 +3186,7 @@ dependencies = [
  "serde_json",
  "shellexpand",
  "thiserror 2.0.11",
+ "toml",
  "tracing",
  "tracing-subscriber",
 ]
@@ -3203,7 +3212,7 @@ dependencies = [
 
 [[package]]
 name = "ruff_wasm"
-version = "0.9.7"
+version = "0.9.9"
 dependencies = [
  "console_error_panic_hook",
  "console_log",
@@ -3315,14 +3324,13 @@ checksum = "6ea1a2d0a644769cc99faa24c3ad26b379b786fe7c36fd3c546254801650e6dd"
 [[package]]
 name = "salsa"
 version = "0.18.0"
-source = "git+https://github.com/salsa-rs/salsa.git?rev=351d9cf0037be949d17800d0c7b4838e533c2ed6#351d9cf0037be949d17800d0c7b4838e533c2ed6"
+source = "git+https://github.com/salsa-rs/salsa.git?rev=99be5d9917c3dd88e19735a82ef6bf39ba84bd7e#99be5d9917c3dd88e19735a82ef6bf39ba84bd7e"
 dependencies = [
- "append-only-vec",
- "arc-swap",
+ "boxcar",
  "compact_str",
- "crossbeam",
+ "crossbeam-queue",
  "dashmap 6.1.0",
- "hashbrown 0.14.5",
+ "hashbrown 0.15.2",
  "hashlink",
  "indexmap",
  "parking_lot",
@@ -3337,12 +3345,12 @@ dependencies = [
 [[package]]
 name = "salsa-macro-rules"
 version = "0.1.0"
-source = "git+https://github.com/salsa-rs/salsa.git?rev=351d9cf0037be949d17800d0c7b4838e533c2ed6#351d9cf0037be949d17800d0c7b4838e533c2ed6"
+source = "git+https://github.com/salsa-rs/salsa.git?rev=99be5d9917c3dd88e19735a82ef6bf39ba84bd7e#99be5d9917c3dd88e19735a82ef6bf39ba84bd7e"
 
 [[package]]
 name = "salsa-macros"
 version = "0.18.0"
-source = "git+https://github.com/salsa-rs/salsa.git?rev=351d9cf0037be949d17800d0c7b4838e533c2ed6#351d9cf0037be949d17800d0c7b4838e533c2ed6"
+source = "git+https://github.com/salsa-rs/salsa.git?rev=99be5d9917c3dd88e19735a82ef6bf39ba84bd7e#99be5d9917c3dd88e19735a82ef6bf39ba84bd7e"
 dependencies = [
  "heck",
  "proc-macro2",
@@ -3384,6 +3392,12 @@ dependencies = [
  "syn 2.0.98",
 ]
 
+[[package]]
+name = "scoped-tls"
+version = "1.0.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "e1cf6437eb19a8f4a6cc0f7dca544973b0b78843adbfeb3683d1a94a0024a294"
+
 [[package]]
 name = "scopeguard"
 version = "1.2.0"
@@ -3398,9 +3412,9 @@ checksum = "1c107b6f4780854c8b126e228ea8869f4d7b71260f962fefb57b996b8959ba6b"
 
 [[package]]
 name = "serde"
-version = "1.0.217"
+version = "1.0.218"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "02fc4265df13d6fa1d00ecff087228cc0a2b5f3c0e87e258d8b94a156e984c70"
+checksum = "e8dfc9d19bdbf6d17e22319da49161d5d0108e4188e8b680aef6299eed22df60"
 dependencies = [
  "serde_derive",
 ]
@@ -3418,9 +3432,9 @@ dependencies = [
 
 [[package]]
 name = "serde_derive"
-version = "1.0.217"
+version = "1.0.218"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "5a9bf7cf98d04a2b28aead066b7496853d4779c9cc183c440dbac457641e19a0"
+checksum = "f09503e191f4e797cb8aac08e9a4a4695c5edf6a2e70e376d961ddd5c969f82b"
 dependencies = [
  "proc-macro2",
  "quote",
@@ -3440,9 +3454,9 @@ dependencies = [
 
 [[package]]
 name = "serde_json"
-version = "1.0.138"
+version = "1.0.139"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "d434192e7da787e94a6ea7e9670b26a036d0ca41e0b7efb2676dd32bae872949"
+checksum = "44f86c3acccc9c65b153fe1b85a3be07fe5515274ec9f0653b4a0875731c72a6"
 dependencies = [
  "itoa",
  "memchr",
@@ -3668,9 +3682,9 @@ dependencies = [
 
 [[package]]
 name = "tempfile"
-version = "3.17.0"
+version = "3.17.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "a40f762a77d2afa88c2d919489e390a12bdd261ed568e60cfa7e48d4e20f0d33"
+checksum = "22e5a0acb1f3f55f65cc4a866c361b2fb2a0ff6366785ae6fbb5f85df07ba230"
 dependencies = [
  "cfg-if",
  "fastrand",
@@ -3971,6 +3985,7 @@ version = "0.3.19"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "e8189decb5ac0fa7bc8b96b7cb9b2701d60d48805aca84a238004d665fcc4008"
 dependencies = [
+ "chrono",
  "matchers",
  "nu-ansi-term 0.46.0",
  "once_cell",
@@ -4068,9 +4083,9 @@ dependencies = [
 
 [[package]]
 name = "unicode-ident"
-version = "1.0.16"
+version = "1.0.17"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "a210d160f08b701c8721ba1c726c11662f877ea6b7094007e1ca9a1041945034"
+checksum = "00e2473a93778eb0bad35909dff6a10d28e63f792f16ed15e404fca9d5eeedbe"
 
 [[package]]
 name = "unicode-normalization"
@@ -4451,6 +4466,16 @@ version = "0.4.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f"
 
+[[package]]
+name = "windows"
+version = "0.58.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "dd04d41d93c4992d421894c18c8b43496aa748dd4c081bac0dc93eb0489272b6"
+dependencies = [
+ "windows-core 0.58.0",
+ "windows-targets 0.52.6",
+]
+
 [[package]]
 name = "windows-core"
 version = "0.52.0"
@@ -4460,6 +4485,60 @@ dependencies = [
  "windows-targets 0.52.6",
 ]
 
+[[package]]
+name = "windows-core"
+version = "0.58.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "6ba6d44ec8c2591c134257ce647b7ea6b20335bf6379a27dac5f1641fcf59f99"
+dependencies = [
+ "windows-implement",
+ "windows-interface",
+ "windows-result",
+ "windows-strings",
+ "windows-targets 0.52.6",
+]
+
+[[package]]
+name = "windows-implement"
+version = "0.58.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "2bbd5b46c938e506ecbce286b6628a02171d56153ba733b6c741fc627ec9579b"
+dependencies = [
+ "proc-macro2",
+ "quote",
+ "syn 2.0.98",
+]
+
+[[package]]
+name = "windows-interface"
+version = "0.58.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "053c4c462dc91d3b1504c6fe5a726dd15e216ba718e84a0e46a88fbe5ded3515"
+dependencies = [
+ "proc-macro2",
+ "quote",
+ "syn 2.0.98",
+]
+
+[[package]]
+name = "windows-result"
+version = "0.2.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "1d1043d8214f791817bab27572aaa8af63732e11bf84aa21a45a78d6c317ae0e"
+dependencies = [
+ "windows-targets 0.52.6",
+]
+
+[[package]]
+name = "windows-strings"
+version = "0.1.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "4cd9b125c486025df0eabcb585e62173c6c9eddcec5d117d3b6e8c30e2ee4d10"
+dependencies = [
+ "windows-result",
+ "windows-targets 0.52.6",
+]
+
 [[package]]
 name = "windows-sys"
 version = "0.48.0"

Cargo.toml

@@ -4,7 +4,7 @@ resolver = "2"
 
 [workspace.package]
 edition = "2021"
-rust-version = "1.80"
+rust-version = "1.83"
 homepage = "https://docs.astral.sh/ruff"
 documentation = "https://docs.astral.sh/ruff"
 repository = "https://github.com/astral-sh/ruff"
@@ -123,7 +123,7 @@ rayon = { version = "1.10.0" }
 regex = { version = "1.10.2" }
 rustc-hash = { version = "2.0.0" }
 # When updating salsa, make sure to also update the revision in `fuzz/Cargo.toml`
-salsa = { git = "https://github.com/salsa-rs/salsa.git", rev = "351d9cf0037be949d17800d0c7b4838e533c2ed6" }
+salsa = { git = "https://github.com/salsa-rs/salsa.git", rev = "99be5d9917c3dd88e19735a82ef6bf39ba84bd7e" }
 schemars = { version = "0.8.16" }
 seahash = { version = "4.1.0" }
 serde = { version = "1.0.197", features = ["derive"] }

README.md

@@ -149,8 +149,8 @@ curl -LsSf https://astral.sh/ruff/install.sh | sh
 powershell -c "irm https://astral.sh/ruff/install.ps1 | iex"
 
 # For a specific version.
-curl -LsSf https://astral.sh/ruff/0.9.7/install.sh | sh
-powershell -c "irm https://astral.sh/ruff/0.9.7/install.ps1 | iex"
+curl -LsSf https://astral.sh/ruff/0.9.9/install.sh | sh
+powershell -c "irm https://astral.sh/ruff/0.9.9/install.ps1 | iex"
 ```
 
 You can also install Ruff via [Homebrew](https://formulae.brew.sh/formula/ruff), [Conda](https://anaconda.org/conda-forge/ruff),
@@ -183,7 +183,7 @@ Ruff can also be used as a [pre-commit](https://pre-commit.com/) hook via [`ruff
 ```yaml
 - repo: https://github.com/astral-sh/ruff-pre-commit
   # Ruff version.
-  rev: v0.9.7
+  rev: v0.9.9
   hooks:
     # Run the linter.
     - id: ruff

crates/red_knot/Cargo.toml

@@ -19,6 +19,7 @@ ruff_db = { workspace = true, features = ["os", "cache"] }
 ruff_python_ast = { workspace = true }
 
 anyhow = { workspace = true }
+argfile = { workspace = true }
 chrono = { workspace = true }
 clap = { workspace = true, features = ["wrap_help"] }
 colored = { workspace = true }
@@ -31,6 +32,7 @@ tracing = { workspace = true, features = ["release_max_level_debug"] }
 tracing-subscriber = { workspace = true, features = ["env-filter", "fmt"] }
 tracing-flame = { workspace = true }
 tracing-tree = { workspace = true }
+wild = { workspace = true }
 
 [dev-dependencies]
 ruff_db = { workspace = true, features = ["testing"] }

crates/red_knot/src/args.rs

@@ -41,12 +41,14 @@ pub(crate) struct CheckCommand {
     #[arg(long, value_name = "PROJECT")]
     pub(crate) project: Option<SystemPathBuf>,
 
-    /// Path to the virtual environment the project uses.
+    /// Path to the Python installation from which Red Knot resolves type information and third-party dependencies.
     ///
-    /// If provided, red-knot will use the `site-packages` directory of this virtual environment
-    /// to resolve type information for the project's third-party dependencies.
+    /// Red Knot will search in the path's `site-packages` directories for type information and
+    /// third-party imports.
+    ///
+    /// This option is commonly used to specify the path to a virtual environment.
     #[arg(long, value_name = "PATH")]
-    pub(crate) venv_path: Option<SystemPathBuf>,
+    pub(crate) python: Option<SystemPathBuf>,
 
     /// Custom directory to use for stdlib typeshed stubs.
     #[arg(long, value_name = "PATH", alias = "custom-typeshed-dir")]
@@ -97,7 +99,7 @@ impl CheckCommand {
                 python_version: self
                     .python_version
                     .map(|version| RangedValue::cli(version.into())),
-                venv_path: self.venv_path.map(RelativePathBuf::cli),
+                python: self.python.map(RelativePathBuf::cli),
                 typeshed: self.typeshed.map(RelativePathBuf::cli),
                 extra_paths: self.extra_search_path.map(|extra_search_paths| {
                     extra_search_paths

crates/red_knot/src/main.rs

@@ -1,4 +1,4 @@
-use std::io::{self, BufWriter, Write};
+use std::io::{self, stdout, BufWriter, Write};
 use std::process::{ExitCode, Termination};
 
 use anyhow::Result;
@@ -16,7 +16,7 @@ use red_knot_project::{watch, Db};
 use red_knot_project::{ProjectDatabase, ProjectMetadata};
 use red_knot_server::run_server;
 use ruff_db::diagnostic::{Diagnostic, DisplayDiagnosticConfig, Severity};
-use ruff_db::system::{OsSystem, System, SystemPath, SystemPathBuf};
+use ruff_db::system::{OsSystem, SystemPath, SystemPathBuf};
 use salsa::plumbing::ZalsaDatabase;
 
 mod args;
@@ -39,6 +39,15 @@ pub fn main() -> ExitStatus {
         // the configuration it is help to chain errors ("resolving configuration failed" ->
         // "failed to read file: subdir/pyproject.toml")
         for cause in error.chain() {
+            // Exit "gracefully" on broken pipe errors.
+            //
+            // See: https://github.com/BurntSushi/ripgrep/blob/bf63fe8f258afc09bae6caa48f0ae35eaf115005/crates/core/main.rs#L47C1-L61C14
+            if let Some(ioerr) = cause.downcast_ref::<io::Error>() {
+                if ioerr.kind() == io::ErrorKind::BrokenPipe {
+                    return ExitStatus::Success;
+                }
+            }
+
             writeln!(stderr, "  {} {cause}", "Cause:".bold()).ok();
         }
 
@@ -47,7 +56,10 @@ pub fn main() -> ExitStatus {
 }
 
 fn run() -> anyhow::Result<ExitStatus> {
-    let args = Args::parse_from(std::env::args());
+    let args = wild::args_os();
+    let args = argfile::expand_args_from(args, argfile::parse_fromfile, argfile::PREFIX)
+        .context("Failed to read CLI arguments from file")?;
+    let args = Args::parse_from(args);
 
     match args.command {
         Command::Server => run_server().map(|()| ExitStatus::Success),
@@ -69,7 +81,7 @@ fn run_check(args: CheckCommand) -> anyhow::Result<ExitStatus> {
     let _guard = setup_tracing(verbosity)?;
 
     // The base path to which all CLI arguments are relative to.
-    let cli_base_path = {
+    let cwd = {
         let cwd = std::env::current_dir().context("Failed to get the current working directory")?;
         SystemPathBuf::from_path_buf(cwd)
             .map_err(|path| {
@@ -80,25 +92,27 @@ fn run_check(args: CheckCommand) -> anyhow::Result<ExitStatus> {
             })?
     };
 
-    let cwd = args
+    let project_path = args
         .project
         .as_ref()
-        .map(|cwd| {
-            if cwd.as_std_path().is_dir() {
-                Ok(SystemPath::absolute(cwd, &cli_base_path))
+        .map(|project| {
+            if project.as_std_path().is_dir() {
+                Ok(SystemPath::absolute(project, &cwd))
             } else {
-                Err(anyhow!("Provided project path `{cwd}` is not a directory"))
+                Err(anyhow!(
+                    "Provided project path `{project}` is not a directory"
+                ))
             }
         })
         .transpose()?
-        .unwrap_or_else(|| cli_base_path.clone());
+        .unwrap_or_else(|| cwd.clone());
 
     let system = OsSystem::new(cwd);
     let watch = args.watch;
     let exit_zero = args.exit_zero;
 
     let cli_options = args.into_options();
-    let mut project_metadata = ProjectMetadata::discover(system.current_directory(), &system)?;
+    let mut project_metadata = ProjectMetadata::discover(&project_path, &system)?;
     project_metadata.apply_cli_options(cli_options.clone());
     project_metadata.apply_configuration_files(&system)?;
 
@@ -119,7 +133,7 @@ fn run_check(args: CheckCommand) -> anyhow::Result<ExitStatus> {
     let exit_status = if watch {
         main_loop.watch(&mut db)?
     } else {
-        main_loop.run(&mut db)
+        main_loop.run(&mut db)?
     };
 
     tracing::trace!("Counts for entire CLI run:\n{}", countme::get_all());
@@ -179,7 +193,7 @@ impl MainLoop {
         )
     }
 
-    fn watch(mut self, db: &mut ProjectDatabase) -> anyhow::Result<ExitStatus> {
+    fn watch(mut self, db: &mut ProjectDatabase) -> Result<ExitStatus> {
         tracing::debug!("Starting watch mode");
         let sender = self.sender.clone();
         let watcher = watch::directory_watcher(move |event| {
@@ -188,12 +202,12 @@ impl MainLoop {
 
         self.watcher = Some(ProjectWatcher::new(watcher, db));
 
-        self.run(db);
+        self.run(db)?;
 
         Ok(ExitStatus::Success)
     }
 
-    fn run(mut self, db: &mut ProjectDatabase) -> ExitStatus {
+    fn run(mut self, db: &mut ProjectDatabase) -> Result<ExitStatus> {
         self.sender.send(MainLoopMessage::CheckWorkspace).unwrap();
 
         let result = self.main_loop(db);
@@ -203,7 +217,7 @@ impl MainLoop {
         result
     }
 
-    fn main_loop(&mut self, db: &mut ProjectDatabase) -> ExitStatus {
+    fn main_loop(&mut self, db: &mut ProjectDatabase) -> Result<ExitStatus> {
         // Schedule the first check.
         tracing::debug!("Starting main loop");
 
@@ -246,9 +260,9 @@ impl MainLoop {
                         .any(|diagnostic| diagnostic.severity() >= min_error_severity);
 
                     if check_revision == revision {
-                        #[allow(clippy::print_stdout)]
+                        let mut stdout = stdout().lock();
                         for diagnostic in result {
-                            println!("{}", diagnostic.display(db, &display_config));
+                            writeln!(stdout, "{}", diagnostic.display(db, &display_config))?;
                         }
                     } else {
                         tracing::debug!(
@@ -257,11 +271,11 @@ impl MainLoop {
                     }
 
                     if self.watcher.is_none() {
-                        return if failed {
+                        return Ok(if failed {
                             ExitStatus::Failure
                         } else {
                             ExitStatus::Success
-                        };
+                        });
                     }
 
                     tracing::trace!("Counts after last check:\n{}", countme::get_all());
@@ -281,14 +295,14 @@ impl MainLoop {
                     // TODO: Don't use Salsa internal APIs
                     //  [Zulip-Thread](https://salsa.zulipchat.com/#narrow/stream/333573-salsa-3.2E0/topic/Expose.20an.20API.20to.20cancel.20other.20queries)
                     let _ = db.zalsa_mut();
-                    return ExitStatus::Success;
+                    return Ok(ExitStatus::Success);
                 }
             }
 
             tracing::debug!("Waiting for next main loop message.");
         }
 
-        ExitStatus::Success
+        Ok(ExitStatus::Success)
     }
 }
 

crates/red_knot/tests/file_watching.rs

@@ -462,6 +462,41 @@ fn new_ignored_file() -> anyhow::Result<()> {
     Ok(())
 }
 
+#[test]
+fn new_non_project_file() -> anyhow::Result<()> {
+    let mut case = setup_with_options([("bar.py", "")], |context| {
+        Some(Options {
+            environment: Some(EnvironmentOptions {
+                extra_paths: Some(vec![RelativePathBuf::cli(
+                    context.join_root_path("site_packages"),
+                )]),
+                ..EnvironmentOptions::default()
+            }),
+            ..Options::default()
+        })
+    })?;
+
+    let bar_path = case.project_path("bar.py");
+    let bar_file = case.system_file(&bar_path).unwrap();
+
+    assert_eq!(&case.collect_project_files(), &[bar_file]);
+
+    // Add a file to site packages
+    let black_path = case.root_path().join("site_packages/black.py");
+    std::fs::write(black_path.as_std_path(), "print('Hello')")?;
+
+    let changes = case.stop_watch(event_for_file("black.py"));
+
+    case.apply_changes(changes);
+
+    assert!(case.system_file(&black_path).is_ok());
+
+    // The file should not have been added to the project files
+    assert_eq!(&case.collect_project_files(), &[bar_file]);
+
+    Ok(())
+}
+
 #[test]
 fn changed_file() -> anyhow::Result<()> {
     let foo_source = "print('Hello, world!')";
@@ -1075,6 +1110,7 @@ fn hard_links_in_project() -> anyhow::Result<()> {
 
     assert_eq!(source_text(case.db(), foo).as_str(), "print('Version 1')");
     assert_eq!(source_text(case.db(), bar).as_str(), "print('Version 1')");
+    assert_eq!(case.collect_project_files(), &[bar, foo]);
 
     // Write to the hard link target.
     update_file(foo_path, "print('Version 2')").context("Failed to update foo.py")?;
@@ -1354,6 +1390,8 @@ mod unix {
         );
         assert_eq!(baz.file().path(case.db()).as_system_path(), Some(&*bar_baz));
 
+        assert_eq!(case.collect_project_files(), &[patched_bar_baz_file]);
+
         // Write to the symlink target.
         update_file(&patched_bar_baz, "def baz(): print('Version 2')")
             .context("Failed to update bar/baz.py")?;
@@ -1389,6 +1427,7 @@ mod unix {
             bar_baz_text = bar_baz_text.as_str()
         );
 
+        assert_eq!(case.collect_project_files(), &[patched_bar_baz_file]);
         Ok(())
     }
 
@@ -1469,6 +1508,8 @@ mod unix {
             Some(&*baz_original)
         );
 
+        assert_eq!(case.collect_project_files(), &[]);
+
         // Write to the symlink target.
         update_file(&baz_original, "def baz(): print('Version 2')")
             .context("Failed to update bar/baz.py")?;
@@ -1494,6 +1535,8 @@ mod unix {
             "def baz(): print('Version 2')"
         );
 
+        assert_eq!(case.collect_project_files(), &[]);
+
         Ok(())
     }
 }

crates/red_knot_project/src/combine.rs

@@ -1,6 +1,6 @@
 use std::{collections::HashMap, hash::BuildHasher};
 
-use red_knot_python_semantic::{PythonPlatform, SitePackages};
+use red_knot_python_semantic::{PythonPath, PythonPlatform};
 use ruff_db::system::SystemPathBuf;
 use ruff_python_ast::PythonVersion;
 
@@ -128,7 +128,7 @@ macro_rules! impl_noop_combine {
 
 impl_noop_combine!(SystemPathBuf);
 impl_noop_combine!(PythonPlatform);
-impl_noop_combine!(SitePackages);
+impl_noop_combine!(PythonPath);
 impl_noop_combine!(PythonVersion);
 
 // std types

crates/red_knot_project/src/db/changes.rs

@@ -208,11 +208,12 @@ impl ProjectDatabase {
                         return WalkState::Continue;
                     }
 
-                    if entry
-                        .path()
-                        .extension()
-                        .and_then(PySourceType::try_from_extension)
-                        .is_some()
+                    if entry.path().starts_with(&project_path)
+                        && entry
+                            .path()
+                            .extension()
+                            .and_then(PySourceType::try_from_extension)
+                            .is_some()
                     {
                         let mut paths = added_paths.lock().unwrap();
 

crates/red_knot_project/src/metadata.rs

@@ -77,10 +77,10 @@ impl ProjectMetadata {
         // If the `options` don't specify a python version but the `project.requires-python` field is set,
         // use that as a lower bound instead.
         if let Some(project) = project {
-            if !options
+            if options
                 .environment
                 .as_ref()
-                .is_some_and(|env| env.python_version.is_some())
+                .is_none_or(|env| env.python_version.is_none())
             {
                 if let Some(requires_python) = project.resolve_requires_python_lower_bound()? {
                     let mut environment = options.environment.unwrap_or_default();

crates/red_knot_project/src/metadata/options.rs

@@ -1,7 +1,7 @@
 use crate::metadata::value::{RangedValue, RelativePathBuf, ValueSource, ValueSourceGuard};
 use crate::Db;
 use red_knot_python_semantic::lint::{GetLintError, Level, LintSource, RuleSelection};
-use red_knot_python_semantic::{ProgramSettings, PythonPlatform, SearchPathSettings, SitePackages};
+use red_knot_python_semantic::{ProgramSettings, PythonPath, PythonPlatform, SearchPathSettings};
 use ruff_db::diagnostic::{Diagnostic, DiagnosticId, Severity, Span};
 use ruff_db::files::system_path_to_file;
 use ruff_db::system::{System, SystemPath};
@@ -90,7 +90,7 @@ impl Options {
             .map(|env| {
                 (
                     env.extra_paths.clone(),
-                    env.venv_path.clone(),
+                    env.python.clone(),
                     env.typeshed.clone(),
                 )
             })
@@ -104,11 +104,11 @@ impl Options {
                 .collect(),
             src_roots,
             custom_typeshed: typeshed.map(|path| path.absolute(project_root, system)),
-            site_packages: python
-                .map(|venv_path| SitePackages::Derived {
-                    venv_path: venv_path.absolute(project_root, system),
+            python_path: python
+                .map(|python_path| {
+                    PythonPath::SysPrefix(python_path.absolute(project_root, system))
                 })
-                .unwrap_or(SitePackages::Known(vec![])),
+                .unwrap_or(PythonPath::KnownSitePackages(vec![])),
         }
     }
 
@@ -236,10 +236,14 @@ pub struct EnvironmentOptions {
     #[serde(skip_serializing_if = "Option::is_none")]
     pub typeshed: Option<RelativePathBuf>,
 
-    // TODO: Rename to python, see https://github.com/astral-sh/ruff/issues/15530
-    /// The path to the user's `site-packages` directory, where third-party packages from ``PyPI`` are installed.
+    /// Path to the Python installation from which Red Knot resolves type information and third-party dependencies.
+    ///
+    /// Red Knot will search in the path's `site-packages` directories for type information and
+    /// third-party imports.
+    ///
+    /// This option is commonly used to specify the path to a virtual environment.
     #[serde(skip_serializing_if = "Option::is_none")]
-    pub venv_path: Option<RelativePathBuf>,
+    pub python: Option<RelativePathBuf>,
 }
 
 #[derive(Debug, Default, Clone, Eq, PartialEq, Combine, Serialize, Deserialize)]

crates/red_knot_python_semantic/Cargo.toml

@@ -42,6 +42,8 @@ smallvec = { workspace = true }
 static_assertions = { workspace = true }
 test-case = { workspace = true }
 memchr = { workspace = true }
+strum = { workspace = true}
+strum_macros = { workspace = true}
 
 [dev-dependencies]
 ruff_db = { workspace = true, features = ["testing", "os"] }

crates/red_knot_python_semantic/resources/mdtest/annotations/literal_string.md

@@ -73,12 +73,12 @@ qux = (foo, bar)
 reveal_type(qux)  # revealed: tuple[Literal["foo"], Literal["bar"]]
 
 # TODO: Infer "LiteralString"
-reveal_type(foo.join(qux))  # revealed: @Todo(Attribute access on `StringLiteral` types)
+reveal_type(foo.join(qux))  # revealed: @Todo(overloaded method)
 
 template: LiteralString = "{}, {}"
 reveal_type(template)  # revealed: Literal["{}, {}"]
 # TODO: Infer `LiteralString`
-reveal_type(template.format(foo, bar))  # revealed: @Todo(Attribute access on `StringLiteral` types)
+reveal_type(template.format(foo, bar))  # revealed: @Todo(overloaded method)
 ```
 
 ### Assignability

crates/red_knot_python_semantic/resources/mdtest/annotations/string.md

@@ -116,8 +116,8 @@ MyType = int
 class Aliases:
     MyType = str
 
-    forward: "MyType"
-    not_forward: MyType
+    forward: "MyType" = "value"
+    not_forward: MyType = "value"
 
 reveal_type(Aliases.forward)  # revealed: str
 reveal_type(Aliases.not_forward)  # revealed: str

crates/red_knot_python_semantic/resources/mdtest/attributes.md

@@ -54,13 +54,12 @@ c_instance.declared_and_bound = False
 # error: [invalid-assignment] "Object of type `Literal["incompatible"]` is not assignable to attribute `declared_and_bound` of type `bool`"
 c_instance.declared_and_bound = "incompatible"
 
-# TODO: we already show an error here but the message might be improved?
 # mypy shows no error here, but pyright raises "reportAttributeAccessIssue"
-# error: [unresolved-attribute] "Type `Literal[C]` has no attribute `inferred_from_value`"
+# error: [unresolved-attribute] "Attribute `inferred_from_value` can only be accessed on instances, not on the class object `Literal[C]` itself."
 reveal_type(C.inferred_from_value)  # revealed: Unknown
 
-# TODO: this should be an error (pure instance variables cannot be accessed on the class)
 # mypy shows no error here, but pyright raises "reportAttributeAccessIssue"
+# error: [invalid-attribute-access] "Cannot assign to instance attribute `inferred_from_value` from the class object `Literal[C]`"
 C.inferred_from_value = "overwritten on class"
 
 # This assignment is fine:
@@ -90,13 +89,13 @@ c_instance = C()
 
 reveal_type(c_instance.declared_and_bound)  # revealed: str | None
 
-# TODO: we currently plan to emit a diagnostic here. Note that both mypy
-# and pyright show no error in this case! So we may reconsider this in
-# the future, if it turns out to produce too many false positives.
-reveal_type(C.declared_and_bound)  # revealed: str | None
+# Note that both mypy and pyright show no error in this case! So we may reconsider this in
+# the future, if it turns out to produce too many false positives. We currently emit:
+# error: [unresolved-attribute] "Attribute `declared_and_bound` can only be accessed on instances, not on the class object `Literal[C]` itself."
+reveal_type(C.declared_and_bound)  # revealed: Unknown
 
-# TODO: same as above. We plan to emit a diagnostic here, even if both mypy
-# and pyright allow this.
+# Same as above. Mypy and pyright do not show an error here.
+# error: [invalid-attribute-access] "Cannot assign to instance attribute `declared_and_bound` from the class object `Literal[C]`"
 C.declared_and_bound = "overwritten on class"
 
 # error: [invalid-assignment] "Object of type `Literal[1]` is not assignable to attribute `declared_and_bound` of type `str | None`"
@@ -116,11 +115,11 @@ c_instance = C()
 
 reveal_type(c_instance.only_declared)  # revealed: str
 
-# TODO: mypy and pyright do not show an error here, but we plan to emit a diagnostic.
-# The type could be changed to 'Unknown' if we decide to emit an error?
-reveal_type(C.only_declared)  # revealed: str
+# Mypy and pyright do not show an error here. We treat this as a pure instance variable.
+# error: [unresolved-attribute] "Attribute `only_declared` can only be accessed on instances, not on the class object `Literal[C]` itself."
+reveal_type(C.only_declared)  # revealed: Unknown
 
-# TODO: mypy and pyright do not show an error here, but we plan to emit one.
+# error: [invalid-attribute-access] "Cannot assign to instance attribute `only_declared` from the class object `Literal[C]`"
 C.only_declared = "overwritten on class"
 ```
 
@@ -191,11 +190,10 @@ reveal_type(c_instance.declared_only)  # revealed: bytes
 
 reveal_type(c_instance.declared_and_bound)  # revealed: bool
 
-# TODO: We already show an error here, but the message might be improved?
-# error: [unresolved-attribute]
+# error: [unresolved-attribute] "Attribute `inferred_from_value` can only be accessed on instances, not on the class object `Literal[C]` itself."
 reveal_type(C.inferred_from_value)  # revealed: Unknown
 
-# TODO: this should be an error
+# error: [invalid-attribute-access] "Cannot assign to instance attribute `inferred_from_value` from the class object `Literal[C]`"
 C.inferred_from_value = "overwritten on class"
 ```
 
@@ -598,6 +596,9 @@ C.class_method()
 # error: [unresolved-attribute]
 reveal_type(C.pure_class_variable)  # revealed: Unknown
 
+# TODO: should be no error when descriptor protocol is supported
+# and the assignment is properly attributed to the class method.
+# error: [invalid-attribute-access] "Cannot assign to instance attribute `pure_class_variable` from the class object `Literal[C]`"
 C.pure_class_variable = "overwritten on class"
 
 # TODO: should be  `Unknown | Literal["value set in class method"]` or
@@ -782,6 +783,9 @@ def _(flag1: bool, flag2: bool):
 
     # error: [possibly-unbound-attribute] "Attribute `x` on type `Literal[C1, C2, C3]` is possibly unbound"
     reveal_type(C.x)  # revealed: Unknown | Literal[1, 3]
+
+    # error: [possibly-unbound-attribute] "Attribute `x` on type `C1 | C2 | C3` is possibly unbound"
+    reveal_type(C().x)  # revealed: Unknown | Literal[1, 3]
 ```
 
 ### Possibly-unbound within a class
@@ -805,6 +809,28 @@ def _(flag: bool, flag1: bool, flag2: bool):
 
     # error: [possibly-unbound-attribute] "Attribute `x` on type `Literal[C1, C2, C3]` is possibly unbound"
     reveal_type(C.x)  # revealed: Unknown | Literal[1, 2, 3]
+
+    # Note: we might want to consider ignoring possibly-unbound diagnostics for instance attributes eventually,
+    # see the "Possibly unbound/undeclared instance attribute" section below.
+    # error: [possibly-unbound-attribute] "Attribute `x` on type `C1 | C2 | C3` is possibly unbound"
+    reveal_type(C().x)  # revealed: Unknown | Literal[1, 2, 3]
+```
+
+### Possibly-unbound within gradual types
+
+```py
+from typing import Any
+
+def _(flag: bool):
+    class Base:
+        x: Any
+
+    class Derived(Base):
+        if flag:
+            # Redeclaring `x` with a more static type is okay in terms of LSP.
+            x: int
+
+    reveal_type(Derived().x)  # revealed: int | Any
 ```
 
 ### Attribute possibly unbound on a subclass but not on a superclass
@@ -819,6 +845,8 @@ def _(flag: bool):
             x = 2
 
     reveal_type(Bar.x)  # revealed: Unknown | Literal[2, 1]
+
+    reveal_type(Bar().x)  # revealed: Unknown | Literal[2, 1]
 ```
 
 ### Attribute possibly unbound on a subclass and on a superclass
@@ -835,6 +863,41 @@ def _(flag: bool):
 
     # error: [possibly-unbound-attribute]
     reveal_type(Bar.x)  # revealed: Unknown | Literal[2, 1]
+
+    # error: [possibly-unbound-attribute]
+    reveal_type(Bar().x)  # revealed: Unknown | Literal[2, 1]
+```
+
+### Possibly unbound/undeclared instance attribute
+
+#### Possibly unbound and undeclared
+
+```py
+def _(flag: bool):
+    class Foo:
+        if flag:
+            x: int
+
+        def __init(self):
+            if flag:
+                self.x = 1
+
+    # error: [possibly-unbound-attribute]
+    reveal_type(Foo().x)  # revealed: int
+```
+
+#### Possibly unbound
+
+```py
+def _(flag: bool):
+    class Foo:
+        def __init(self):
+            if flag:
+                self.x = 1
+
+    # Emitting a diagnostic in a case like this is not something we support, and it's unclear
+    # if we ever will (or want to)
+    reveal_type(Foo().x)  # revealed: Unknown | Literal[1]
 ```
 
 ### Attribute access on `Any`
@@ -884,13 +947,18 @@ def _(flag: bool):
 
 ## Objects of all types have a `__class__` method
 
+The type of `x.__class__` is the same as `x`'s meta-type. `x.__class__` is always the same value as
+`type(x)`.
+
 ```py
 import typing_extensions
 
 reveal_type(typing_extensions.__class__)  # revealed: Literal[ModuleType]
+reveal_type(type(typing_extensions))  # revealed: Literal[ModuleType]
 
 a = 42
 reveal_type(a.__class__)  # revealed: Literal[int]
+reveal_type(type(a))  # revealed: Literal[int]
 
 b = "42"
 reveal_type(b.__class__)  # revealed: Literal[str]
@@ -906,8 +974,13 @@ reveal_type(e.__class__)  # revealed: Literal[tuple]
 
 def f(a: int, b: typing_extensions.LiteralString, c: int | str, d: type[str]):
     reveal_type(a.__class__)  # revealed: type[int]
+    reveal_type(type(a))  # revealed: type[int]
+
     reveal_type(b.__class__)  # revealed: Literal[str]
+    reveal_type(type(b))  # revealed: Literal[str]
+
     reveal_type(c.__class__)  # revealed: type[int] | type[str]
+    reveal_type(type(c))  # revealed: type[int] | type[str]
 
     # `type[type]`, a.k.a., either the class `type` or some subclass of `type`.
     # It would be incorrect to infer `Literal[type]` here,
@@ -1005,8 +1078,8 @@ reveal_type(f.__kwdefaults__)  # revealed: @Todo(generics) | None
 Some attributes are special-cased, however:
 
 ```py
-reveal_type(f.__get__)  # revealed: @Todo(`__get__` method on functions)
-reveal_type(f.__call__)  # revealed: @Todo(`__call__` method on functions)
+reveal_type(f.__get__)  # revealed: <method-wrapper `__get__` of `f`>
+reveal_type(f.__call__)  # revealed: <bound method `__call__` of `Literal[f]`>
 ```
 
 ### Int-literal attributes
@@ -1015,7 +1088,7 @@ Most attribute accesses on int-literal types are delegated to `builtins.int`, si
 integers are instances of that class:
 
 ```py
-reveal_type((2).bit_length)  # revealed: @Todo(bound method)
+reveal_type((2).bit_length)  # revealed: <bound method `bit_length` of `Literal[2]`>
 reveal_type((2).denominator)  # revealed: @Todo(@property)
 ```
 
@@ -1029,11 +1102,11 @@ reveal_type((2).real)  # revealed: Literal[2]
 ### Bool-literal attributes
 
 Most attribute accesses on bool-literal types are delegated to `builtins.bool`, since all literal
-bols are instances of that class:
+bools are instances of that class:
 
 ```py
-reveal_type(True.__and__)  # revealed: @Todo(bound method)
-reveal_type(False.__or__)  # revealed: @Todo(bound method)
+reveal_type(True.__and__)  # revealed: @Todo(overloaded method)
+reveal_type(False.__or__)  # revealed: @Todo(overloaded method)
 ```
 
 Some attributes are special-cased, however:
@@ -1045,11 +1118,11 @@ reveal_type(False.real)  # revealed: Literal[0]
 
 ### Bytes-literal attributes
 
-All attribute access on literal `bytes` types is currently delegated to `buitins.bytes`:
+All attribute access on literal `bytes` types is currently delegated to `builtins.bytes`:
 
 ```py
-reveal_type(b"foo".join)  # revealed: @Todo(bound method)
-reveal_type(b"foo".endswith)  # revealed: @Todo(bound method)
+reveal_type(b"foo".join)  # revealed: <bound method `join` of `Literal[b"foo"]`>
+reveal_type(b"foo".endswith)  # revealed: <bound method `endswith` of `Literal[b"foo"]`>
 ```
 
 ## Instance attribute edge cases
@@ -1136,6 +1209,40 @@ class C:
 reveal_type(C().x)  # revealed: Unknown
 ```
 
+### Builtin types attributes
+
+This test can probably be removed eventually, but we currently include it because we do not yet
+understand generic bases and protocols, and we want to make sure that we can still use builtin types
+in our tests in the meantime. See the corresponding TODO in `Type::static_member` for more
+information.
+
+```py
+class C:
+    a_int: int = 1
+    a_str: str = "a"
+    a_bytes: bytes = b"a"
+    a_bool: bool = True
+    a_float: float = 1.0
+    a_complex: complex = 1 + 1j
+    a_tuple: tuple[int] = (1,)
+    a_range: range = range(1)
+    a_slice: slice = slice(1)
+    a_type: type = int
+    a_none: None = None
+
+reveal_type(C.a_int)  # revealed: int
+reveal_type(C.a_str)  # revealed: str
+reveal_type(C.a_bytes)  # revealed: bytes
+reveal_type(C.a_bool)  # revealed: bool
+reveal_type(C.a_float)  # revealed: int | float
+reveal_type(C.a_complex)  # revealed: int | float | complex
+reveal_type(C.a_tuple)  # revealed: tuple[int]
+reveal_type(C.a_range)  # revealed: range
+reveal_type(C.a_slice)  # revealed: slice
+reveal_type(C.a_type)  # revealed: type
+reveal_type(C.a_none)  # revealed: None
+```
+
 ## References
 
 Some of the tests in the *Class and instance variables* section draw inspiration from

crates/red_knot_python_semantic/resources/mdtest/binary/instances.md

@@ -259,11 +259,17 @@ class A:
 class B:
     __add__ = A()
 
-# TODO: this could be `int` if we declare `B.__add__` using a `Callable` type
-# TODO: Should not be an error: `A` instance is not a method descriptor, don't prepend `self` arg.
-#   Revealed type should be `Unknown | int`.
-# error: [unsupported-operator] "Operator `+` is unsupported between objects of type `B` and `B`"
-reveal_type(B() + B())  # revealed: Unknown
+reveal_type(B() + B())  # revealed: Unknown | int
+```
+
+Note that we union with `Unknown` here because `__add__` is not declared. We do infer just `int` if
+the callable is declared:
+
+```py
+class B2:
+    __add__: A = A()
+
+reveal_type(B2() + B2())  # revealed: int
 ```
 
 ## Integration test: numbers from typeshed
@@ -306,7 +312,7 @@ reveal_type(1 + A())  # revealed: A
 reveal_type(A() + "foo")  # revealed: A
 # TODO should be `A` since `str.__add__` doesn't support `A` instances
 # TODO overloads
-reveal_type("foo" + A())  # revealed: @Todo(return type)
+reveal_type("foo" + A())  # revealed: @Todo(return type of decorated function)
 
 reveal_type(A() + b"foo")  # revealed: A
 # TODO should be `A` since `bytes.__add__` doesn't support `A` instances
@@ -314,7 +320,7 @@ reveal_type(b"foo" + A())  # revealed: bytes
 
 reveal_type(A() + ())  # revealed: A
 # TODO this should be `A`, since `tuple.__add__` doesn't support `A` instances
-reveal_type(() + A())  # revealed: @Todo(return type)
+reveal_type(() + A())  # revealed: @Todo(return type of decorated function)
 
 literal_string_instance = "foo" * 1_000_000_000
 # the test is not testing what it's meant to be testing if this isn't a `LiteralString`:
@@ -323,7 +329,7 @@ reveal_type(literal_string_instance)  # revealed: LiteralString
 reveal_type(A() + literal_string_instance)  # revealed: A
 # TODO should be `A` since `str.__add__` doesn't support `A` instances
 # TODO overloads
-reveal_type(literal_string_instance + A())  # revealed: @Todo(return type)
+reveal_type(literal_string_instance + A())  # revealed: @Todo(return type of decorated function)
 ```
 
 ## Operations involving instances of classes inheriting from `Any`
@@ -351,6 +357,20 @@ class Y(Foo): ...
 reveal_type(X() + Y())  # revealed: int
 ```
 
+## Operations involving types with invalid `__bool__` methods
+
+<!-- snapshot-diagnostics -->
+
+```py
+class NotBoolable:
+    __bool__ = 3
+
+a = NotBoolable()
+
+# error: [unsupported-bool-conversion]
+10 and a and True
+```
+
 ## Unsupported
 
 ### Dunder as instance attribute

crates/red_knot_python_semantic/resources/mdtest/binary/integers.md

@@ -51,9 +51,9 @@ reveal_type(1 ** (largest_u32 + 1))  # revealed: int
 reveal_type(2**largest_u32)  # revealed: int
 
 def variable(x: int):
-    reveal_type(x**2)  # revealed: @Todo(return type)
-    reveal_type(2**x)  # revealed: @Todo(return type)
-    reveal_type(x**x)  # revealed: @Todo(return type)
+    reveal_type(x**2)  # revealed: @Todo(return type of decorated function)
+    reveal_type(2**x)  # revealed: @Todo(return type of decorated function)
+    reveal_type(x**x)  # revealed: @Todo(return type of decorated function)
 ```
 
 ## Division by Zero

crates/red_knot_python_semantic/resources/mdtest/call/builtins.md

@@ -0,0 +1,37 @@
+# Calling builtins
+
+## `bool` with incorrect arguments
+
+```py
+class NotBool:
+    __bool__ = None
+
+# TODO: We should emit an `invalid-argument` error here for `2` because `bool` only takes one argument.
+bool(1, 2)
+
+# TODO: We should emit an `unsupported-bool-conversion` error here because the argument doesn't implement `__bool__` correctly.
+bool(NotBool())
+```
+
+## Calls to `type()`
+
+A single-argument call to `type()` returns an object that has the argument's meta-type. (This is
+tested more extensively in `crates/red_knot_python_semantic/resources/mdtest/attributes.md`,
+alongside the tests for the `__class__` attribute.)
+
+```py
+reveal_type(type(1))  # revealed: Literal[int]
+```
+
+But a three-argument call to type creates a dynamic instance of the `type` class:
+
+```py
+reveal_type(type("Foo", (), {}))  # revealed: type
+```
+
+Other numbers of arguments are invalid (TODO -- these should emit a diagnostic)
+
+```py
+type("Foo", ())
+type("Foo", (), {}, weird_other_arg=42)
+```

crates/red_knot_python_semantic/resources/mdtest/call/callable_instance.md

@@ -82,7 +82,7 @@ class C:
 
 c = C()
 
-# error: 15 [invalid-argument-type] "Object of type `Literal["foo"]` cannot be assigned to parameter 2 (`x`) of function `__call__`; expected type `int`"
+# error: 15 [invalid-argument-type] "Object of type `Literal["foo"]` cannot be assigned to parameter 2 (`x`) of bound method `__call__`; expected type `int`"
 reveal_type(c("foo"))  # revealed: int
 ```
 
@@ -96,7 +96,7 @@ class C:
 
 c = C()
 
-# error: 13 [invalid-argument-type] "Object of type `C` cannot be assigned to parameter 1 (`self`) of function `__call__`; expected type `int`"
+# error: 13 [invalid-argument-type] "Object of type `C` cannot be assigned to parameter 1 (`self`) of bound method `__call__`; expected type `int`"
 reveal_type(c())  # revealed: int
 ```
 

crates/red_knot_python_semantic/resources/mdtest/call/dunder.md

@@ -0,0 +1,128 @@
+# Dunder calls
+
+## Introduction
+
+This test suite explains and documents how dunder methods are looked up and called. Throughout the
+document, we use `__getitem__` as an example, but the same principles apply to other dunder methods.
+
+Dunder methods are implicitly called when using certain syntax. For example, the index operator
+`obj[key]` calls the `__getitem__` method under the hood. Exactly *how* a dunder method is looked up
+and called works slightly different from regular methods. Dunder methods are not looked up on `obj`
+directly, but rather on `type(obj)`. But in many ways, they still *act* as if they were called on
+`obj` directly. If the `__getitem__` member of `type(obj)` is a descriptor, it is called with `obj`
+as the `instance` argument to `__get__`. A desugared version of `obj[key]` is roughly equivalent to
+`getitem_desugared(obj, key)` as defined below:
+
+```py
+from typing import Any
+
+def find_name_in_mro(typ: type, name: str) -> Any:
+    # See implementation in https://docs.python.org/3/howto/descriptor.html#invocation-from-an-instance
+    pass
+
+def getitem_desugared(obj: object, key: object) -> object:
+    getitem_callable = find_name_in_mro(type(obj), "__getitem__")
+    if hasattr(getitem_callable, "__get__"):
+        getitem_callable = getitem_callable.__get__(obj, type(obj))
+
+    return getitem_callable(key)
+```
+
+In the following tests, we demonstrate that we implement this behavior correctly.
+
+## Operating on class objects
+
+If we invoke a dunder method on a class, it is looked up on the *meta* class, since any class is an
+instance of its metaclass:
+
+```py
+class Meta(type):
+    def __getitem__(cls, key: int) -> str:
+        return str(key)
+
+class DunderOnMetaClass(metaclass=Meta):
+    pass
+
+reveal_type(DunderOnMetaClass[0])  # revealed: str
+```
+
+## Operating on instances
+
+When invoking a dunder method on an instance of a class, it is looked up on the class:
+
+```py
+class ClassWithNormalDunder:
+    def __getitem__(self, key: int) -> str:
+        return str(key)
+
+class_with_normal_dunder = ClassWithNormalDunder()
+
+reveal_type(class_with_normal_dunder[0])  # revealed: str
+```
+
+Which can be demonstrated by trying to attach a dunder method to an instance, which will not work:
+
+```py
+def external_getitem(instance, key: int) -> str:
+    return str(key)
+
+class ThisFails:
+    def __init__(self):
+        self.__getitem__ = external_getitem
+
+this_fails = ThisFails()
+
+# error: [non-subscriptable] "Cannot subscript object of type `ThisFails` with no `__getitem__` method"
+reveal_type(this_fails[0])  # revealed: Unknown
+```
+
+However, the attached dunder method *can* be called if accessed directly:
+
+```py
+# TODO: `this_fails.__getitem__` is incorrectly treated as a bound method. This
+# should be fixed with https://github.com/astral-sh/ruff/issues/16367
+# error: [too-many-positional-arguments]
+# error: [invalid-argument-type]
+reveal_type(this_fails.__getitem__(this_fails, 0))  # revealed: Unknown | str
+```
+
+## When the dunder is not a method
+
+A dunder can also be a non-method callable:
+
+```py
+class SomeCallable:
+    def __call__(self, key: int) -> str:
+        return str(key)
+
+class ClassWithNonMethodDunder:
+    __getitem__: SomeCallable = SomeCallable()
+
+class_with_callable_dunder = ClassWithNonMethodDunder()
+
+reveal_type(class_with_callable_dunder[0])  # revealed: str
+```
+
+## Dunders are looked up using the descriptor protocol
+
+Here, we demonstrate that the descriptor protocol is invoked when looking up a dunder method. Note
+that the `instance` argument is on object of type `ClassWithDescriptorDunder`:
+
+```py
+from __future__ import annotations
+
+class SomeCallable:
+    def __call__(self, key: int) -> str:
+        return str(key)
+
+class Descriptor:
+    def __get__(self, instance: ClassWithDescriptorDunder, owner: type[ClassWithDescriptorDunder]) -> SomeCallable:
+        return SomeCallable()
+
+class ClassWithDescriptorDunder:
+    __getitem__: Descriptor = Descriptor()
+
+class_with_descriptor_dunder = ClassWithDescriptorDunder()
+
+reveal_type(class_with_descriptor_dunder[0])  # revealed: str
+```

crates/red_knot_python_semantic/resources/mdtest/call/function.md

@@ -44,7 +44,7 @@ def bar() -> str:
     return "bar"
 
 # TODO: should reveal `int`, as the decorator replaces `bar` with `foo`
-reveal_type(bar())  # revealed: @Todo(return type)
+reveal_type(bar())  # revealed: @Todo(return type of decorated function)
 ```
 
 ## Invalid callable

crates/red_knot_python_semantic/resources/mdtest/call/getattr_static.md

@@ -0,0 +1,133 @@
+# `inspect.getattr_static`
+
+## Basic usage
+
+`inspect.getattr_static` is a function that returns attributes of an object without invoking the
+descriptor protocol (for caveats, see the [official documentation]).
+
+Consider the following example:
+
+```py
+import inspect
+
+class Descriptor:
+    def __get__(self, instance, owner) -> str:
+        return 1
+
+class C:
+    normal: int = 1
+    descriptor: Descriptor = Descriptor()
+```
+
+If we access attributes on an instance of `C` as usual, the descriptor protocol is invoked, and we
+get a type of `str` for the `descriptor` attribute:
+
+```py
+c = C()
+
+reveal_type(c.normal)  # revealed: int
+reveal_type(c.descriptor)  # revealed: str
+```
+
+However, if we use `inspect.getattr_static`, we can see the underlying `Descriptor` type:
+
+```py
+reveal_type(inspect.getattr_static(c, "normal"))  # revealed: int
+reveal_type(inspect.getattr_static(c, "descriptor"))  # revealed: Descriptor
+```
+
+For non-existent attributes, a default value can be provided:
+
+```py
+reveal_type(inspect.getattr_static(C, "normal", "default-arg"))  # revealed: int
+reveal_type(inspect.getattr_static(C, "non_existent", "default-arg"))  # revealed: Literal["default-arg"]
+```
+
+When a non-existent attribute is accessed without a default value, the runtime raises an
+`AttributeError`. We could emit a diagnostic for this case, but that is currently not supported:
+
+```py
+# TODO: we could emit a diagnostic here
+reveal_type(inspect.getattr_static(C, "non_existent"))  # revealed: Never
+```
+
+We can access attributes on objects of all kinds:
+
+```py
+import sys
+
+reveal_type(inspect.getattr_static(sys, "platform"))  # revealed: LiteralString
+reveal_type(inspect.getattr_static(inspect, "getattr_static"))  # revealed: Literal[getattr_static]
+
+reveal_type(inspect.getattr_static(1, "real"))  # revealed: Literal[1]
+```
+
+(Implicit) instance attributes can also be accessed through `inspect.getattr_static`:
+
+```py
+class D:
+    def __init__(self) -> None:
+        self.instance_attr: int = 1
+
+reveal_type(inspect.getattr_static(D(), "instance_attr"))  # revealed: int
+```
+
+## Error cases
+
+We can only infer precise types if the attribute is a literal string. In all other cases, we fall
+back to `Any`:
+
+```py
+import inspect
+
+class C:
+    x: int = 1
+
+def _(attr_name: str):
+    reveal_type(inspect.getattr_static(C(), attr_name))  # revealed: Any
+    reveal_type(inspect.getattr_static(C(), attr_name, 1))  # revealed: Any
+```
+
+But we still detect errors in the number or type of arguments:
+
+```py
+# error: [missing-argument] "No arguments provided for required parameters `obj`, `attr` of function `getattr_static`"
+inspect.getattr_static()
+
+# error: [missing-argument] "No argument provided for required parameter `attr`"
+inspect.getattr_static(C())
+
+# error: [invalid-argument-type] "Object of type `Literal[1]` cannot be assigned to parameter 2 (`attr`) of function `getattr_static`; expected type `str`"
+inspect.getattr_static(C(), 1)
+
+# error: [too-many-positional-arguments] "Too many positional arguments to function `getattr_static`: expected 3, got 4"
+inspect.getattr_static(C(), "x", "default-arg", "one too many")
+```
+
+## Possibly unbound attributes
+
+```py
+import inspect
+
+def _(flag: bool):
+    class C:
+        if flag:
+            x: int = 1
+
+    reveal_type(inspect.getattr_static(C, "x", "default"))  # revealed: int | Literal["default"]
+```
+
+## Gradual types
+
+```py
+import inspect
+from typing import Any
+
+def _(a: Any, tuple_of_any: tuple[Any]):
+    reveal_type(inspect.getattr_static(a, "x", "default"))  # revealed: Any | Literal["default"]
+
+    # TODO: Ideally, this would just be `Literal[index]`
+    reveal_type(inspect.getattr_static(tuple_of_any, "index", "default"))  # revealed: Literal[index] | Literal["default"]
+```
+
+[official documentation]: https://docs.python.org/3/library/inspect.html#inspect.getattr_static

crates/red_knot_python_semantic/resources/mdtest/call/methods.md

@@ -0,0 +1,380 @@
+# Methods
+
+## Background: Functions as descriptors
+
+> Note: See also this related section in the descriptor guide: [Functions and methods].
+
+Say we have a simple class `C` with a function definition `f` inside its body:
+
+```py
+class C:
+    def f(self, x: int) -> str:
+        return "a"
+```
+
+Whenever we access the `f` attribute through the class object itself (`C.f`) or through an instance
+(`C().f`), this access happens via the descriptor protocol. Functions are (non-data) descriptors
+because they implement a `__get__` method. This is crucial in making sure that method calls work as
+expected. In general, the signature of the `__get__` method in the descriptor protocol is
+`__get__(self, instance, owner)`. The `self` argument is the descriptor object itself (`f`). The
+passed value for the `instance` argument depends on whether the attribute is accessed from the class
+object (in which case it is `None`), or from an instance (in which case it is the instance of type
+`C`). The `owner` argument is the class itself (`C` of type `Literal[C]`). To summarize:
+
+- `C.f` is equivalent to `getattr_static(C, "f").__get__(None, C)`
+- `C().f` is equivalent to `getattr_static(C, "f").__get__(C(), C)`
+
+Here, `inspect.getattr_static` is used to bypass the descriptor protocol and directly access the
+function attribute. The way the special `__get__` method *on functions* works is as follows. In the
+former case, if the `instance` argument is `None`, `__get__` simply returns the function itself. In
+the latter case, it returns a *bound method* object:
+
+```py
+from inspect import getattr_static
+
+reveal_type(getattr_static(C, "f"))  # revealed: Literal[f]
+
+reveal_type(getattr_static(C, "f").__get__)  # revealed: <method-wrapper `__get__` of `f`>
+
+reveal_type(getattr_static(C, "f").__get__(None, C))  # revealed: Literal[f]
+reveal_type(getattr_static(C, "f").__get__(C(), C))  # revealed: <bound method `f` of `C`>
+```
+
+In conclusion, this is why we see the following two types when accessing the `f` attribute on the
+class object `C` and on an instance `C()`:
+
+```py
+reveal_type(C.f)  # revealed: Literal[f]
+reveal_type(C().f)  # revealed: <bound method `f` of `C`>
+```
+
+A bound method is a callable object that contains a reference to the `instance` that it was called
+on (can be inspected via `__self__`), and the function object that it refers to (can be inspected
+via `__func__`):
+
+```py
+bound_method = C().f
+
+reveal_type(bound_method.__self__)  # revealed: C
+reveal_type(bound_method.__func__)  # revealed: Literal[f]
+```
+
+When we call the bound method, the `instance` is implicitly passed as the first argument (`self`):
+
+```py
+reveal_type(C().f(1))  # revealed: str
+reveal_type(bound_method(1))  # revealed: str
+```
+
+When we call the function object itself, we need to pass the `instance` explicitly:
+
+```py
+C.f(1)  # error: [missing-argument]
+
+reveal_type(C.f(C(), 1))  # revealed: str
+```
+
+When we access methods from derived classes, they will be bound to instances of the derived class:
+
+```py
+class D(C):
+    pass
+
+reveal_type(D().f)  # revealed: <bound method `f` of `D`>
+```
+
+If we access an attribute on a bound method object itself, it will defer to `types.MethodType`:
+
+```py
+reveal_type(bound_method.__hash__)  # revealed: <bound method `__hash__` of `MethodType`>
+```
+
+If an attribute is not available on the bound method object, it will be looked up on the underlying
+function object. We model this explicitly, which means that we can access `__kwdefaults__` on bound
+methods, even though it is not available on `types.MethodType`:
+
+```py
+reveal_type(bound_method.__kwdefaults__)  # revealed: @Todo(generics) | None
+```
+
+## Basic method calls on class objects and instances
+
+```py
+class Base:
+    def method_on_base(self, x: int | None) -> str:
+        return "a"
+
+class Derived(Base):
+    def method_on_derived(self, x: bytes) -> tuple[int, str]:
+        return (1, "a")
+
+reveal_type(Base().method_on_base(1))  # revealed: str
+reveal_type(Base.method_on_base(Base(), 1))  # revealed: str
+
+Base().method_on_base("incorrect")  # error: [invalid-argument-type]
+Base().method_on_base()  # error: [missing-argument]
+Base().method_on_base(1, 2)  # error: [too-many-positional-arguments]
+
+reveal_type(Derived().method_on_base(1))  # revealed: str
+reveal_type(Derived().method_on_derived(b"abc"))  # revealed: tuple[int, str]
+reveal_type(Derived.method_on_base(Derived(), 1))  # revealed: str
+reveal_type(Derived.method_on_derived(Derived(), b"abc"))  # revealed: tuple[int, str]
+```
+
+## Method calls on literals
+
+### Boolean literals
+
+```py
+reveal_type(True.bit_length())  # revealed: int
+reveal_type(True.as_integer_ratio())  # revealed: tuple[int, Literal[1]]
+```
+
+### Integer literals
+
+```py
+reveal_type((42).bit_length())  # revealed: int
+```
+
+### String literals
+
+```py
+reveal_type("abcde".find("abc"))  # revealed: int
+reveal_type("foo".encode(encoding="utf-8"))  # revealed: bytes
+
+"abcde".find(123)  # error: [invalid-argument-type]
+```
+
+### Bytes literals
+
+```py
+reveal_type(b"abcde".startswith(b"abc"))  # revealed: bool
+```
+
+## Method calls on `LiteralString`
+
+```py
+from typing_extensions import LiteralString
+
+def f(s: LiteralString) -> None:
+    reveal_type(s.find("a"))  # revealed: int
+```
+
+## Method calls on `tuple`
+
+```py
+def f(t: tuple[int, str]) -> None:
+    reveal_type(t.index("a"))  # revealed: int
+```
+
+## Method calls on unions
+
+```py
+from typing import Any
+
+class A:
+    def f(self) -> int:
+        return 1
+
+class B:
+    def f(self) -> str:
+        return "a"
+
+def f(a_or_b: A | B, any_or_a: Any | A):
+    reveal_type(a_or_b.f)  # revealed: <bound method `f` of `A`> | <bound method `f` of `B`>
+    reveal_type(a_or_b.f())  # revealed: int | str
+
+    reveal_type(any_or_a.f)  # revealed: Any | <bound method `f` of `A`>
+    reveal_type(any_or_a.f())  # revealed: Any | int
+```
+
+## Method calls on `KnownInstance` types
+
+```toml
+[environment]
+python-version = "3.12"
+```
+
+```py
+type IntOrStr = int | str
+
+reveal_type(IntOrStr.__or__)  # revealed: <bound method `__or__` of `typing.TypeAliasType`>
+```
+
+## Error cases: Calling `__get__` for methods
+
+The `__get__` method on `types.FunctionType` has the following overloaded signature in typeshed:
+
+```py
+from types import FunctionType, MethodType
+from typing import overload
+
+@overload
+def __get__(self, instance: None, owner: type, /) -> FunctionType: ...
+@overload
+def __get__(self, instance: object, owner: type | None = None, /) -> MethodType: ...
+```
+
+Here, we test that this signature is enforced correctly:
+
+```py
+from inspect import getattr_static
+
+class C:
+    def f(self, x: int) -> str:
+        return "a"
+
+method_wrapper = getattr_static(C, "f").__get__
+
+reveal_type(method_wrapper)  # revealed: <method-wrapper `__get__` of `f`>
+
+# All of these are fine:
+method_wrapper(C(), C)
+method_wrapper(C())
+method_wrapper(C(), None)
+method_wrapper(None, C)
+
+# Passing `None` without an `owner` argument is an
+# error: [missing-argument] "No argument provided for required parameter `owner`"
+method_wrapper(None)
+
+# Passing something that is not assignable to `type` as the `owner` argument is an
+# error: [invalid-argument-type] "Object of type `Literal[1]` cannot be assigned to parameter 2 (`owner`) of method wrapper `__get__` of function `f`; expected type `type`"
+method_wrapper(None, 1)
+
+# Passing `None` as the `owner` argument when `instance` is `None` is an
+# error: [invalid-argument-type] "Object of type `None` cannot be assigned to parameter 2 (`owner`) of method wrapper `__get__` of function `f`; expected type `type`"
+method_wrapper(None, None)
+
+# Calling `__get__` without any arguments is an
+# error: [missing-argument] "No argument provided for required parameter `instance`"
+method_wrapper()
+
+# Calling `__get__` with too many positional arguments is an
+# error: [too-many-positional-arguments] "Too many positional arguments to method wrapper `__get__` of function `f`: expected 2, got 3"
+method_wrapper(C(), C, "one too many")
+```
+
+## `@classmethod`
+
+### Basic
+
+When a `@classmethod` attribute is accessed, it returns a bound method object, even when accessed on
+the class object itself:
+
+```py
+from __future__ import annotations
+
+class C:
+    @classmethod
+    def f(cls: type[C], x: int) -> str:
+        return "a"
+
+reveal_type(C.f)  # revealed: <bound method `f` of `Literal[C]`>
+reveal_type(C().f)  # revealed: <bound method `f` of `type[C]`>
+```
+
+The `cls` method argument is then implicitly passed as the first argument when calling the method:
+
+```py
+reveal_type(C.f(1))  # revealed: str
+reveal_type(C().f(1))  # revealed: str
+```
+
+When the class method is called incorrectly, we detect it:
+
+```py
+C.f("incorrect")  # error: [invalid-argument-type]
+C.f()  # error: [missing-argument]
+C.f(1, 2)  # error: [too-many-positional-arguments]
+```
+
+If the `cls` parameter is wrongly annotated, we emit an error at the call site:
+
+```py
+class D:
+    @classmethod
+    def f(cls: D):
+        # This function is wrongly annotated, it should be `type[D]` instead of `D`
+        pass
+
+# error: [invalid-argument-type] "Object of type `Literal[D]` cannot be assigned to parameter 1 (`cls`) of bound method `f`; expected type `D`"
+D.f()
+```
+
+When a class method is accessed on a derived class, it is bound to that derived class:
+
+```py
+class Derived(C):
+    pass
+
+reveal_type(Derived.f)  # revealed: <bound method `f` of `Literal[Derived]`>
+reveal_type(Derived().f)  # revealed: <bound method `f` of `type[Derived]`>
+
+reveal_type(Derived.f(1))  # revealed: str
+reveal_type(Derived().f(1))  # revealed: str
+```
+
+### Accessing the classmethod as a static member
+
+Accessing a `@classmethod`-decorated function at runtime returns a `classmethod` object. We
+currently don't model this explicitly:
+
+```py
+from inspect import getattr_static
+
+class C:
+    @classmethod
+    def f(cls): ...
+
+reveal_type(getattr_static(C, "f"))  # revealed: Literal[f]
+reveal_type(getattr_static(C, "f").__get__)  # revealed: <method-wrapper `__get__` of `f`>
+```
+
+But we correctly model how the `classmethod` descriptor works:
+
+```py
+reveal_type(getattr_static(C, "f").__get__(None, C))  # revealed: <bound method `f` of `Literal[C]`>
+reveal_type(getattr_static(C, "f").__get__(C(), C))  # revealed: <bound method `f` of `Literal[C]`>
+reveal_type(getattr_static(C, "f").__get__(C()))  # revealed: <bound method `f` of `type[C]`>
+```
+
+The `owner` argument takes precedence over the `instance` argument:
+
+```py
+reveal_type(getattr_static(C, "f").__get__("dummy", C))  # revealed: <bound method `f` of `Literal[C]`>
+```
+
+### Classmethods mixed with other decorators
+
+When a `@classmethod` is additionally decorated with another decorator, it is still treated as a
+class method:
+
+```py
+from __future__ import annotations
+
+def does_nothing[T](f: T) -> T:
+    return f
+
+class C:
+    @classmethod
+    @does_nothing
+    def f1(cls: type[C], x: int) -> str:
+        return "a"
+
+    @does_nothing
+    @classmethod
+    def f2(cls: type[C], x: int) -> str:
+        return "a"
+
+# TODO: We do not support decorators yet (only limited special cases). Eventually,
+# these should all return `str`:
+
+reveal_type(C.f1(1))  # revealed: @Todo(return type of decorated function)
+reveal_type(C().f1(1))  # revealed: @Todo(decorated method)
+
+reveal_type(C.f2(1))  # revealed: @Todo(return type of decorated function)
+reveal_type(C().f2(1))  # revealed: @Todo(decorated method)
+```
+
+[functions and methods]: https://docs.python.org/3/howto/descriptor.html#functions-and-methods

crates/red_knot_python_semantic/resources/mdtest/call/union.md

@@ -56,6 +56,7 @@ def _(flag: bool, flag2: bool):
     else:
         def f() -> int:
             return 1
+    # TODO we should mention all non-callable elements of the union
     # error: [call-non-callable] "Object of type `Literal[1]` is not callable"
     # revealed: int | Unknown
     reveal_type(f())
@@ -108,3 +109,38 @@ def _(flag: bool):
     x = f(3)
     reveal_type(x)  # revealed: Unknown
 ```
+
+## Union of binding errors
+
+```py
+def f1(): ...
+def f2(): ...
+def _(flag: bool):
+    if flag:
+        f = f1
+    else:
+        f = f2
+
+    # TODO: we should show all errors from the union, not arbitrarily pick one union element
+    # error: [too-many-positional-arguments] "Too many positional arguments to function `f1`: expected 0, got 1"
+    x = f(3)
+    reveal_type(x)  # revealed: Unknown
+```
+
+## One not-callable, one wrong argument
+
+```py
+class C: ...
+
+def f1(): ...
+def _(flag: bool):
+    if flag:
+        f = f1
+    else:
+        f = C()
+
+    # TODO: we should either show all union errors here, or prioritize the not-callable error
+    # error: [too-many-positional-arguments] "Too many positional arguments to function `f1`: expected 0, got 1"
+    x = f(3)
+    reveal_type(x)  # revealed: Unknown
+```

crates/red_knot_python_semantic/resources/mdtest/comparison/instances/membership_test.md

@@ -160,3 +160,45 @@ reveal_type(42 in A())  # revealed: bool
 # error: [unsupported-operator] "Operator `in` is not supported for types `str` and `A`, in comparing `Literal["hello"]` with `A`"
 reveal_type("hello" in A())  # revealed: bool
 ```
+
+## Return type that doesn't implement `__bool__` correctly
+
+`in` and `not in` operations will fail at runtime if the object on the right-hand side of the
+operation has a `__contains__` method that returns a type which is not convertible to `bool`. This
+is because of the way these operations are handled by the Python interpreter at runtime. If we
+assume that `y` is an object that has a `__contains__` method, the Python expression `x in y`
+desugars to a `contains(y, x)` call, where `contains` looks something like this:
+
+```ignore
+def contains(y, x):
+    return bool(type(y).__contains__(y, x))
+```
+
+where the `bool()` conversion itself implicitly calls `__bool__` under the hood.
+
+TODO: Ideally the message would explain to the user what's wrong. E.g,
+
+```ignore
+error: [operator] cannot use `in` operator on object of type `WithContains`
+    note: This is because the `in` operator implicitly calls `WithContains.__contains__`, but `WithContains.__contains__` is invalidly defined
+    note: `WithContains.__contains__` is invalidly defined because it returns an instance of `NotBoolable`, which cannot be evaluated in a boolean context
+    note: `NotBoolable` cannot be evaluated in a boolean context because its `__bool__` attribute is not callable
+```
+
+It may also be more appropriate to use `unsupported-operator` as the error code.
+
+<!-- snapshot-diagnostics -->
+
+```py
+class NotBoolable:
+    __bool__ = 3
+
+class WithContains:
+    def __contains__(self, item) -> NotBoolable:
+        return NotBoolable()
+
+# error: [unsupported-bool-conversion]
+10 in WithContains()
+# error: [unsupported-bool-conversion]
+10 not in WithContains()
+```

crates/red_knot_python_semantic/resources/mdtest/comparison/instances/rich_comparison.md

@@ -345,3 +345,47 @@ def f(x: bool, y: int):
     reveal_type(4.2 < x)  # revealed: bool
     reveal_type(x < 4.2)  # revealed: bool
 ```
+
+## Chained comparisons with objects that don't implement `__bool__` correctly
+
+<!-- snapshot-diagnostics -->
+
+Python implicitly calls `bool` on the comparison result of preceding elements (but not for the last
+element) of a chained comparison.
+
+```py
+class NotBoolable:
+    __bool__ = 3
+
+class Comparable:
+    def __lt__(self, item) -> NotBoolable:
+        return NotBoolable()
+
+    def __gt__(self, item) -> NotBoolable:
+        return NotBoolable()
+
+# error: [unsupported-bool-conversion]
+10 < Comparable() < 20
+# error: [unsupported-bool-conversion]
+10 < Comparable() < Comparable()
+
+Comparable() < Comparable()  # fine
+```
+
+## Callables as comparison dunders
+
+```py
+from typing import Literal
+
+class AlwaysTrue:
+    def __call__(self, other: object) -> Literal[True]:
+        return True
+
+class A:
+    __eq__: AlwaysTrue = AlwaysTrue()
+    __lt__: AlwaysTrue = AlwaysTrue()
+
+reveal_type(A() == A())  # revealed: Literal[True]
+reveal_type(A() < A())  # revealed: Literal[True]
+reveal_type(A() > A())  # revealed: Literal[True]
+```

crates/red_knot_python_semantic/resources/mdtest/comparison/tuples.md

@@ -334,3 +334,61 @@ reveal_type(a is not c)  # revealed: Literal[True]
 For tuples like `tuple[int, ...]`, `tuple[Any, ...]`
 
 // TODO
+
+## Chained comparisons with elements that incorrectly implement `__bool__`
+
+<!-- snapshot-diagnostics -->
+
+For an operation `A() < A()` to succeed at runtime, the `A.__lt__` method does not necessarily need
+to return an object that is convertible to a `bool`. However, the return type _does_ need to be
+convertible to a `bool` for the operation `A() < A() < A()` (a _chained_ comparison) to succeed.
+This is because `A() < A() < A()` desugars to something like this, which involves several implicit
+conversions to `bool`:
+
+```ignore
+def compute_chained_comparison():
+  a1 = A()
+  a2 = A()
+  first_comparison = a1 < a2
+  return first_comparison and (a2 < A())
+```
+
+```py
+class NotBoolable:
+    __bool__ = 5
+
+class Comparable:
+    def __lt__(self, other) -> NotBoolable:
+        return NotBoolable()
+
+    def __gt__(self, other) -> NotBoolable:
+        return NotBoolable()
+
+a = (1, Comparable())
+b = (1, Comparable())
+
+# error: [unsupported-bool-conversion]
+a < b < b
+
+a < b  # fine
+```
+
+## Equality with elements that incorrectly implement `__bool__`
+
+<!-- snapshot-diagnostics -->
+
+Python does not generally attempt to coerce the result of `==` and `!=` operations between two
+arbitrary objects to a `bool`, but a comparison of tuples will fail if the result of comparing any
+pair of elements at equivalent positions cannot be converted to a `bool`:
+
+```py
+class A:
+    def __eq__(self, other) -> NotBoolable:
+        return NotBoolable()
+
+class NotBoolable:
+    __bool__ = None
+
+# error: [unsupported-bool-conversion]
+(A(),) == (A(),)
+```

crates/red_knot_python_semantic/resources/mdtest/conditional/if_expression.md

@@ -35,3 +35,13 @@ def _(flag: bool):
     x = 1 if flag else None
     reveal_type(x)  # revealed: Literal[1] | None
 ```
+
+## Condition with object that implements `__bool__` incorrectly
+
+```py
+class NotBoolable:
+    __bool__ = 3
+
+# error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable"
+3 if NotBoolable() else 4
+```

crates/red_knot_python_semantic/resources/mdtest/conditional/if_statement.md

@@ -147,3 +147,17 @@ def _(flag: bool):
 
     reveal_type(y)  # revealed: Literal[0, 1]
 ```
+
+## Condition with object that implements `__bool__` incorrectly
+
+```py
+class NotBoolable:
+    __bool__ = 3
+
+# error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable"
+if NotBoolable():
+    ...
+# error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable"
+elif NotBoolable():
+    ...
+```

crates/red_knot_python_semantic/resources/mdtest/conditional/match.md

@@ -43,3 +43,21 @@ def _(target: int):
 
     reveal_type(y)  # revealed: Literal[2, 3, 4]
 ```
+
+## Guard with object that implements `__bool__` incorrectly
+
+```py
+class NotBoolable:
+    __bool__ = 3
+
+def _(target: int, flag: NotBoolable):
+    y = 1
+    match target:
+        # error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable"
+        case 1 if flag:
+            y = 2
+        case 2:
+            y = 3
+
+    reveal_type(y)  # revealed: Literal[1, 2, 3]
+```

crates/red_knot_python_semantic/resources/mdtest/descriptor_protocol.md

@@ -22,22 +22,26 @@ class Ten:
         pass
 
 class C:
-    ten = Ten()
+    ten: Ten = Ten()
 
 c = C()
 
-# TODO: this should be `Literal[10]`
-reveal_type(c.ten)  # revealed: Unknown | Ten
+reveal_type(c.ten)  # revealed: Literal[10]
 
-# TODO: This should `Literal[10]`
-reveal_type(C.ten)  # revealed: Unknown | Ten
+reveal_type(C.ten)  # revealed: Literal[10]
 
 # These are fine:
-c.ten = 10
+# TODO: This should not be an error
+c.ten = 10  # error: [invalid-assignment]
 C.ten = 10
 
-# TODO: Both of these should be errors
+# TODO: This should be an error (as the wrong type is being implicitly passed to `Ten.__set__`),
+# but the error message is misleading.
+# error: [invalid-assignment] "Object of type `Literal[11]` is not assignable to attribute `ten` of type `Ten`"
 c.ten = 11
+
+# TODO: same as above
+# error: [invalid-assignment] "Object of type `Literal[11]` is not assignable to attribute `ten` of type `Literal[10]`"
 C.ten = 11
 ```
 
@@ -57,24 +61,86 @@ class FlexibleInt:
         self._value = int(value)
 
 class C:
-    flexible_int = FlexibleInt()
+    flexible_int: FlexibleInt = FlexibleInt()
 
 c = C()
 
-# TODO: should be `int | None`
-reveal_type(c.flexible_int)  # revealed: Unknown | FlexibleInt
+reveal_type(c.flexible_int)  # revealed: int | None
 
+# TODO: These should not be errors
+# error: [invalid-assignment]
 c.flexible_int = 42  # okay
+# error: [invalid-assignment]
 c.flexible_int = "42"  # also okay!
 
-# TODO: should be `int | None`
-reveal_type(c.flexible_int)  # revealed: Unknown | FlexibleInt
+reveal_type(c.flexible_int)  # revealed: int | None
 
-# TODO: should be an error
+# TODO: This should be an error, but the message needs to be improved.
+# error: [invalid-assignment] "Object of type `None` is not assignable to attribute `flexible_int` of type `FlexibleInt`"
 c.flexible_int = None  # not okay
 
-# TODO: should be `int | None`
-reveal_type(c.flexible_int)  # revealed: Unknown | FlexibleInt
+reveal_type(c.flexible_int)  # revealed: int | None
+```
+
+## Data and non-data descriptors
+
+Descriptors that define `__set__` or `__delete__` are called *data descriptors*. An example\
+of a data descriptor is a `property` with a setter and/or a deleter.\
+Descriptors that only define `__get__`, meanwhile, are called *non-data descriptors*. Examples
+include\
+functions, `classmethod` or `staticmethod`).
+
+The precedence chain for attribute access is (1) data descriptors, (2) instance attributes, and (3)
+non-data descriptors.
+
+```py
+from typing import Literal
+
+class DataDescriptor:
+    def __get__(self, instance: object, owner: type | None = None) -> Literal["data"]:
+        return "data"
+
+    def __set__(self, instance: int, value) -> None:
+        pass
+
+class NonDataDescriptor:
+    def __get__(self, instance: object, owner: type | None = None) -> Literal["non-data"]:
+        return "non-data"
+
+class C:
+    data_descriptor = DataDescriptor()
+    non_data_descriptor = NonDataDescriptor()
+
+    def f(self):
+        # This explains why data descriptors come first in the precedence chain. If
+        # instance attributes would take priority, we would override the descriptor
+        # here. Instead, this calls `DataDescriptor.__set__`, i.e. it does not affect
+        # the type of the `data_descriptor` attribute.
+        self.data_descriptor = 1
+
+        # However, for non-data descriptors, instance attributes do take precedence.
+        # So it is possible to override them.
+        self.non_data_descriptor = 1
+
+c = C()
+
+# TODO: This should ideally be `Unknown | Literal["data"]`.
+#
+#     - Pyright also wrongly shows `int | Literal['data']` here
+#     - Mypy shows Literal["data"] here, but also shows Literal["non-data"] below.
+#
+reveal_type(c.data_descriptor)  # revealed: Unknown | Literal["data", 1]
+
+reveal_type(c.non_data_descriptor)  # revealed: Unknown | Literal["non-data", 1]
+
+reveal_type(C.data_descriptor)  # revealed: Unknown | Literal["data"]
+
+reveal_type(C.non_data_descriptor)  # revealed: Unknown | Literal["non-data"]
+
+# It is possible to override data descriptors via class objects. The following
+# assignment does not call `DataDescriptor.__set__`. For this reason, we infer
+# `Unknown | …` for all (descriptor) attributes.
+C.data_descriptor = "something else"  # This is okay
 ```
 
 ## Built-in `property` descriptor
@@ -101,7 +167,7 @@ c = C()
 reveal_type(c._name)  # revealed: str | None
 
 # Should be `str`
-reveal_type(c.name)  # revealed: @Todo(bound method)
+reveal_type(c.name)  # revealed: @Todo(decorated method)
 
 # Should be `builtins.property`
 reveal_type(C.name)  # revealed: Literal[name]
@@ -135,14 +201,11 @@ class C:
 
 c1 = C.factory("test")  # okay
 
-# TODO: should be `C`
-reveal_type(c1)  # revealed: @Todo(return type)
+reveal_type(c1)  # revealed: C
 
-# TODO: should be `str`
-reveal_type(C.get_name())  # revealed: @Todo(return type)
+reveal_type(C.get_name())  # revealed: str
 
-# TODO: should be `str`
-reveal_type(C("42").get_name())  # revealed: @Todo(bound method)
+reveal_type(C("42").get_name())  # revealed: str
 ```
 
 ## Descriptors only work when used as class variables
@@ -160,9 +223,10 @@ class Ten:
 
 class C:
     def __init__(self):
-        self.ten = Ten()
+        self.ten: Ten = Ten()
 
-reveal_type(C().ten)  # revealed: Unknown | Ten
+# TODO: Should be Ten
+reveal_type(C().ten)  # revealed: Literal[10]
 ```
 
 ## Descriptors distinguishing between class and instance access
@@ -186,13 +250,191 @@ class Descriptor:
             return "called on class object"
 
 class C:
-    d = Descriptor()
+    d: Descriptor = Descriptor()
 
 # TODO: should be `Literal["called on class object"]
-reveal_type(C.d)  # revealed: Unknown | Descriptor
+reveal_type(C.d)  # revealed: LiteralString
 
 # TODO: should be `Literal["called on instance"]
-reveal_type(C().d)  # revealed: Unknown | Descriptor
+reveal_type(C().d)  # revealed: LiteralString
+```
+
+## Undeclared descriptor arguments
+
+If a descriptor attribute is not declared, we union with `Unknown`, just like for regular
+attributes, since that attribute could be overwritten externally. Even a data descriptor with a
+`__set__` method can be overwritten when accessed through a class object.
+
+```py
+class Descriptor:
+    def __get__(self, instance: object, owner: type | None = None) -> int:
+        return 1
+
+    def __set__(self, instance: object, value: int) -> None:
+        pass
+
+class C:
+    descriptor = Descriptor()
+
+C.descriptor = "something else"
+
+# This could also be `Literal["something else"]` if we support narrowing of attribute types based on assignments
+reveal_type(C.descriptor)  # revealed: Unknown | int
+```
+
+## `__get__` is called with correct arguments
+
+```py
+from __future__ import annotations
+
+class TailoredForClassObjectAccess:
+    def __get__(self, instance: None, owner: type[C]) -> int:
+        return 1
+
+class TailoredForInstanceAccess:
+    def __get__(self, instance: C, owner: type[C] | None = None) -> str:
+        return "a"
+
+class C:
+    class_object_access: TailoredForClassObjectAccess = TailoredForClassObjectAccess()
+    instance_access: TailoredForInstanceAccess = TailoredForInstanceAccess()
+
+reveal_type(C.class_object_access)  # revealed: int
+reveal_type(C().instance_access)  # revealed: str
+
+# TODO: These should emit a diagnostic
+reveal_type(C().class_object_access)  # revealed: TailoredForClassObjectAccess
+reveal_type(C.instance_access)  # revealed: TailoredForInstanceAccess
+```
+
+## Descriptors with incorrect `__get__` signature
+
+```py
+class Descriptor:
+    # `__get__` method with missing parameters:
+    def __get__(self) -> int:
+        return 1
+
+class C:
+    descriptor: Descriptor = Descriptor()
+
+# TODO: This should be an error
+reveal_type(C.descriptor)  # revealed: Descriptor
+```
+
+## Possibly-unbound `__get__` method
+
+```py
+def _(flag: bool):
+    class MaybeDescriptor:
+        if flag:
+            def __get__(self, instance: object, owner: type | None = None) -> int:
+                return 1
+
+    class C:
+        descriptor: MaybeDescriptor = MaybeDescriptor()
+
+    # TODO: This should be `MaybeDescriptor | int`
+    reveal_type(C.descriptor)  # revealed: int
+```
+
+## Dunder methods
+
+Dunder methods are looked up on the meta type, but we still need to invoke the descriptor protocol:
+
+```py
+class SomeCallable:
+    def __call__(self, x: int) -> str:
+        return "a"
+
+class Descriptor:
+    def __get__(self, instance: object, owner: type | None = None) -> SomeCallable:
+        return SomeCallable()
+
+class B:
+    __call__: Descriptor = Descriptor()
+
+b_instance = B()
+reveal_type(b_instance(1))  # revealed: str
+
+b_instance("bla")  # error: [invalid-argument-type]
+```
+
+## Functions as descriptors
+
+Functions are descriptors because they implement a `__get__` method. This is crucial in making sure
+that method calls work as expected. See [this test suite](./call/methods.md) for more information.
+Here, we only demonstrate how `__get__` works on functions:
+
+```py
+from inspect import getattr_static
+
+def f(x: object) -> str:
+    return "a"
+
+reveal_type(f)  # revealed: Literal[f]
+reveal_type(f.__get__)  # revealed: <method-wrapper `__get__` of `f`>
+reveal_type(f.__get__(None, type(f)))  # revealed: Literal[f]
+reveal_type(f.__get__(None, type(f))(1))  # revealed: str
+
+wrapper_descriptor = getattr_static(f, "__get__")
+
+reveal_type(wrapper_descriptor)  # revealed: <wrapper-descriptor `__get__` of `function` objects>
+reveal_type(wrapper_descriptor(f, None, type(f)))  # revealed: Literal[f]
+
+# Attribute access on the method-wrapper `f.__get__` falls back to `MethodWrapperType`:
+reveal_type(f.__get__.__hash__)  # revealed: <bound method `__hash__` of `MethodWrapperType`>
+
+# Attribute access on the wrapper-descriptor falls back to `WrapperDescriptorType`:
+reveal_type(wrapper_descriptor.__qualname__)  # revealed: @Todo(@property)
+```
+
+We can also bind the free function `f` to an instance of a class `C`:
+
+```py
+class C: ...
+
+bound_method = wrapper_descriptor(f, C(), C)
+
+reveal_type(bound_method)  # revealed: <bound method `f` of `C`>
+```
+
+We can then call it, and the instance of `C` is implicitly passed to the first parameter of `f`
+(`x`):
+
+```py
+reveal_type(bound_method())  # revealed: str
+```
+
+Finally, we test some error cases for the call to the wrapper descriptor:
+
+```py
+# Calling the wrapper descriptor without any arguments is an
+# error: [missing-argument] "No arguments provided for required parameters `self`, `instance`"
+wrapper_descriptor()
+
+# Calling it without the `instance` argument is an also an
+# error: [missing-argument] "No argument provided for required parameter `instance`"
+wrapper_descriptor(f)
+
+# Calling it without the `owner` argument if `instance` is not `None` is an
+# error: [missing-argument] "No argument provided for required parameter `owner`"
+wrapper_descriptor(f, None)
+
+# But calling it with an instance is fine (in this case, the `owner` argument is optional):
+wrapper_descriptor(f, C())
+
+# Calling it with something that is not a `FunctionType` as the first argument is an
+# error: [invalid-argument-type] "Object of type `Literal[1]` cannot be assigned to parameter 1 (`self`) of wrapper descriptor `FunctionType.__get__`; expected type `FunctionType`"
+wrapper_descriptor(1, None, type(f))
+
+# Calling it with something that is not a `type` as the `owner` argument is an
+# error: [invalid-argument-type] "Object of type `Literal[f]` cannot be assigned to parameter 3 (`owner`) of wrapper descriptor `FunctionType.__get__`; expected type `type`"
+wrapper_descriptor(f, None, f)
+
+# Calling it with too many positional arguments is an
+# error: [too-many-positional-arguments] "Too many positional arguments to wrapper descriptor `FunctionType.__get__`: expected 3, got 4"
+wrapper_descriptor(f, None, type(f), "one too many")
 ```
 
 [descriptors]: https://docs.python.org/3/howto/descriptor.html

crates/red_knot_python_semantic/resources/mdtest/diagnostics/invalid_argument_type.md

@@ -182,3 +182,16 @@ class C:
 c = C()
 c("wrong")  # error: [invalid-argument-type]
 ```
+
+## Calls to methods
+
+Tests that we also see a reference to a function if the callable is a bound method.
+
+```py
+class C:
+    def square(self, x: int) -> int:
+        return x * x
+
+c = C()
+c.square("hello")  # error: [invalid-argument-type]
+```

crates/red_knot_python_semantic/resources/mdtest/expression/assert.md

@@ -0,0 +1,9 @@
+## Condition with object that implements `__bool__` incorrectly
+
+```py
+class NotBoolable:
+    __bool__ = 3
+
+# error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable"
+assert NotBoolable()
+```

crates/red_knot_python_semantic/resources/mdtest/expression/boolean.md

@@ -101,3 +101,55 @@ reveal_type(bool([]))  # revealed: bool
 reveal_type(bool({}))  # revealed: bool
 reveal_type(bool(set()))  # revealed: bool
 ```
+
+## `__bool__` returning `NoReturn`
+
+```py
+from typing import NoReturn
+
+class NotBoolable:
+    def __bool__(self) -> NoReturn:
+        raise NotImplementedError("This object can't be converted to a boolean")
+
+# TODO: This should emit an error that `NotBoolable` can't be converted to a bool but it currently doesn't
+#   because `Never` is assignable to `bool`. This probably requires dead code analysis to fix.
+if NotBoolable():
+    ...
+```
+
+## Not callable `__bool__`
+
+```py
+class NotBoolable:
+    __bool__ = None
+
+# error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable"
+if NotBoolable():
+    ...
+```
+
+## Not-boolable union
+
+```py
+def test(cond: bool):
+    class NotBoolable:
+        __bool__ = None if cond else 3
+
+    # error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `NotBoolable`; it incorrectly implements `__bool__`"
+    if NotBoolable():
+        ...
+```
+
+## Union with some variants implementing `__bool__` incorrectly
+
+```py
+def test(cond: bool):
+    class NotBoolable:
+        __bool__ = None
+
+    a = 10 if cond else NotBoolable()
+
+    # error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `Literal[10] | NotBoolable`; its `__bool__` method isn't callable"
+    if a:
+        ...
+```

crates/red_knot_python_semantic/resources/mdtest/loops/for.md

@@ -105,7 +105,11 @@ reveal_type(x)
 
 ## With non-callable iterator
 
+<!-- snapshot-diagnostics -->
+
 ```py
+from typing_extensions import reveal_type
+
 def _(flag: bool):
     class NotIterable:
         if flag:
@@ -113,7 +117,8 @@ def _(flag: bool):
         else:
             __iter__: None = None
 
-    for x in NotIterable():  # error: "Object of type `NotIterable` is not iterable"
+    # error: [not-iterable]
+    for x in NotIterable():
         pass
 
     # revealed: Unknown
@@ -123,21 +128,25 @@ def _(flag: bool):
 
 ## Invalid iterable
 
+<!-- snapshot-diagnostics -->
+
 ```py
 nonsense = 123
-for x in nonsense:  # error: "Object of type `Literal[123]` is not iterable"
+for x in nonsense:  # error: [not-iterable]
     pass
 ```
 
 ## New over old style iteration protocol
 
+<!-- snapshot-diagnostics -->
+
 ```py
 class NotIterable:
     def __getitem__(self, key: int) -> int:
         return 42
     __iter__: None = None
 
-for x in NotIterable():  # error: "Object of type `NotIterable` is not iterable"
+for x in NotIterable():  # error: [not-iterable]
     pass
 ```
 
@@ -221,7 +230,11 @@ def _(flag: bool):
 
 ## Union type as iterable where one union element has no `__iter__` method
 
+<!-- snapshot-diagnostics -->
+
 ```py
+from typing_extensions import reveal_type
+
 class TestIter:
     def __next__(self) -> int:
         return 42
@@ -231,14 +244,18 @@ class Test:
         return TestIter()
 
 def _(flag: bool):
-    # error: [not-iterable] "Object of type `Test | Literal[42]` is not iterable because its `__iter__` method is possibly unbound"
+    # error: [not-iterable]
     for x in Test() if flag else 42:
         reveal_type(x)  # revealed: int
 ```
 
 ## Union type as iterable where one union element has invalid `__iter__` method
 
+<!-- snapshot-diagnostics -->
+
 ```py
+from typing_extensions import reveal_type
+
 class TestIter:
     def __next__(self) -> int:
         return 42
@@ -253,7 +270,7 @@ class Test2:
 
 def _(flag: bool):
     # TODO: Improve error message to state which union variant isn't iterable (https://github.com/astral-sh/ruff/issues/13989)
-    # error: "Object of type `Test | Test2` is not iterable"
+    # error: [not-iterable]
     for x in Test() if flag else Test2():
         reveal_type(x)  # revealed: int
 ```
@@ -269,7 +286,454 @@ class Test:
     def __iter__(self) -> TestIter | int:
         return TestIter()
 
-# error: [not-iterable] "Object of type `Test` is not iterable"
+# error: [not-iterable] "Object of type `Test` may not be iterable because its `__iter__` method returns an object of type `TestIter | int`, which may not have a `__next__` method"
 for x in Test():
     reveal_type(x)  # revealed: int
 ```
+
+## Possibly-not-callable `__iter__` method
+
+```py
+def _(flag: bool):
+    class Iterator:
+        def __next__(self) -> int:
+            return 42
+
+    class CustomCallable:
+        if flag:
+            def __call__(self, *args, **kwargs) -> Iterator:
+                return Iterator()
+        else:
+            __call__: None = None
+
+    class Iterable1:
+        __iter__: CustomCallable = CustomCallable()
+
+    class Iterable2:
+        if flag:
+            def __iter__(self) -> Iterator:
+                return Iterator()
+        else:
+            __iter__: None = None
+
+    # error: [not-iterable] "Object of type `Iterable1` may not be iterable because its `__iter__` attribute (with type `CustomCallable`) may not be callable"
+    for x in Iterable1():
+        # TODO... `int` might be ideal here?
+        reveal_type(x)  # revealed: int | Unknown
+
+    # error: [not-iterable] "Object of type `Iterable2` may not be iterable because its `__iter__` attribute (with type `<bound method `__iter__` of `Iterable2`> | None`) may not be callable"
+    for y in Iterable2():
+        # TODO... `int` might be ideal here?
+        reveal_type(y)  # revealed: int | Unknown
+```
+
+## `__iter__` method with a bad signature
+
+<!-- snapshot-diagnostics -->
+
+```py
+from typing_extensions import reveal_type
+
+class Iterator:
+    def __next__(self) -> int:
+        return 42
+
+class Iterable:
+    def __iter__(self, extra_arg) -> Iterator:
+        return Iterator()
+
+# error: [not-iterable]
+for x in Iterable():
+    reveal_type(x)  # revealed: int
+```
+
+## `__iter__` does not return an iterator
+
+<!-- snapshot-diagnostics -->
+
+```py
+from typing_extensions import reveal_type
+
+class Bad:
+    def __iter__(self) -> int:
+        return 42
+
+# error: [not-iterable]
+for x in Bad():
+    reveal_type(x)  # revealed: Unknown
+```
+
+## `__iter__` returns an object with a possibly unbound `__next__` method
+
+```py
+def _(flag: bool):
+    class Iterator:
+        if flag:
+            def __next__(self) -> int:
+                return 42
+
+    class Iterable:
+        def __iter__(self) -> Iterator:
+            return Iterator()
+
+    # error: [not-iterable] "Object of type `Iterable` may not be iterable because its `__iter__` method returns an object of type `Iterator`, which may not have a `__next__` method"
+    for x in Iterable():
+        reveal_type(x)  # revealed: int
+```
+
+## `__iter__` returns an iterator with an invalid `__next__` method
+
+<!-- snapshot-diagnostics -->
+
+```py
+from typing_extensions import reveal_type
+
+class Iterator1:
+    def __next__(self, extra_arg) -> int:
+        return 42
+
+class Iterator2:
+    __next__: None = None
+
+class Iterable1:
+    def __iter__(self) -> Iterator1:
+        return Iterator1()
+
+class Iterable2:
+    def __iter__(self) -> Iterator2:
+        return Iterator2()
+
+# error: [not-iterable]
+for x in Iterable1():
+    reveal_type(x)  # revealed: int
+
+# error: [not-iterable]
+for y in Iterable2():
+    reveal_type(y)  # revealed: Unknown
+```
+
+## Possibly unbound `__iter__` and bad `__getitem__` method
+
+<!-- snapshot-diagnostics -->
+
+```py
+from typing_extensions import reveal_type
+
+def _(flag: bool):
+    class Iterator:
+        def __next__(self) -> int:
+            return 42
+
+    class Iterable:
+        if flag:
+            def __iter__(self) -> Iterator:
+                return Iterator()
+        # invalid signature because it only accepts a `str`,
+        # but the old-style iteration protocol will pass it an `int`
+        def __getitem__(self, key: str) -> bytes:
+            return 42
+
+    # error: [not-iterable]
+    for x in Iterable():
+        reveal_type(x)  # revealed: int | bytes
+```
+
+## Possibly unbound `__iter__` and not-callable `__getitem__`
+
+This snippet tests that we infer the element type correctly in the following edge case:
+
+- `__iter__` is a method with the correct parameter spec that returns a valid iterator; BUT
+- `__iter__` is possibly unbound; AND
+- `__getitem__` is set to a non-callable type
+
+It's important that we emit a diagnostic here, but it's also important that we still use the return
+type of the iterator's `__next__` method as the inferred type of `x` in the `for` loop:
+
+```py
+def _(flag: bool):
+    class Iterator:
+        def __next__(self) -> int:
+            return 42
+
+    class Iterable:
+        if flag:
+            def __iter__(self) -> Iterator:
+                return Iterator()
+        __getitem__: None = None
+
+    # error: [not-iterable] "Object of type `Iterable` may not be iterable because it may not have an `__iter__` method and its `__getitem__` attribute has type `None`, which is not callable"
+    for x in Iterable():
+        reveal_type(x)  # revealed: int
+```
+
+## Possibly unbound `__iter__` and possibly unbound `__getitem__`
+
+<!-- snapshot-diagnostics -->
+
+```py
+from typing_extensions import reveal_type
+
+class Iterator:
+    def __next__(self) -> int:
+        return 42
+
+def _(flag1: bool, flag2: bool):
+    class Iterable:
+        if flag1:
+            def __iter__(self) -> Iterator:
+                return Iterator()
+        if flag2:
+            def __getitem__(self, key: int) -> bytes:
+                return 42
+
+    # error: [not-iterable]
+    for x in Iterable():
+        reveal_type(x)  # revealed: int | bytes
+```
+
+## No `__iter__` method and `__getitem__` is not callable
+
+<!-- snapshot-diagnostics -->
+
+```py
+from typing_extensions import reveal_type
+
+class Bad:
+    __getitem__: None = None
+
+# error: [not-iterable]
+for x in Bad():
+    reveal_type(x)  # revealed: Unknown
+```
+
+## Possibly-not-callable `__getitem__` method
+
+<!-- snapshot-diagnostics -->
+
+```py
+from typing_extensions import reveal_type
+
+def _(flag: bool):
+    class CustomCallable:
+        if flag:
+            def __call__(self, *args, **kwargs) -> int:
+                return 42
+        else:
+            __call__: None = None
+
+    class Iterable1:
+        __getitem__: CustomCallable = CustomCallable()
+
+    class Iterable2:
+        if flag:
+            def __getitem__(self, key: int) -> int:
+                return 42
+        else:
+            __getitem__: None = None
+
+    # error: [not-iterable]
+    for x in Iterable1():
+        # TODO... `int` might be ideal here?
+        reveal_type(x)  # revealed: int | Unknown
+
+    # error: [not-iterable]
+    for y in Iterable2():
+        # TODO... `int` might be ideal here?
+        reveal_type(y)  # revealed: int | Unknown
+```
+
+## Bad `__getitem__` method
+
+<!-- snapshot-diagnostics -->
+
+```py
+from typing_extensions import reveal_type
+
+class Iterable:
+    # invalid because it will implicitly be passed an `int`
+    # by the interpreter
+    def __getitem__(self, key: str) -> int:
+        return 42
+
+# error: [not-iterable]
+for x in Iterable():
+    reveal_type(x)  # revealed: int
+```
+
+## Possibly unbound `__iter__` but definitely bound `__getitem__`
+
+Here, we should not emit a diagnostic: if `__iter__` is unbound, we should fallback to
+`__getitem__`:
+
+```py
+class Iterator:
+    def __next__(self) -> str:
+        return "foo"
+
+def _(flag: bool):
+    class Iterable:
+        if flag:
+            def __iter__(self) -> Iterator:
+                return Iterator()
+
+        def __getitem__(self, key: int) -> bytes:
+            return b"foo"
+
+    for x in Iterable():
+        reveal_type(x)  # revealed: str | bytes
+```
+
+## Possibly invalid `__iter__` methods
+
+<!-- snapshot-diagnostics -->
+
+```py
+from typing_extensions import reveal_type
+
+class Iterator:
+    def __next__(self) -> int:
+        return 42
+
+def _(flag: bool):
+    class Iterable1:
+        if flag:
+            def __iter__(self) -> Iterator:
+                return Iterator()
+        else:
+            def __iter__(self, invalid_extra_arg) -> Iterator:
+                return Iterator()
+
+    # error: [not-iterable]
+    for x in Iterable1():
+        reveal_type(x)  # revealed: int
+
+    class Iterable2:
+        if flag:
+            def __iter__(self) -> Iterator:
+                return Iterator()
+        else:
+            __iter__: None = None
+
+    # error: [not-iterable]
+    for x in Iterable2():
+        # TODO: `int` would probably be better here:
+        reveal_type(x)  # revealed: int | Unknown
+```
+
+## Possibly invalid `__next__` method
+
+<!-- snapshot-diagnostics -->
+
+```py
+from typing_extensions import reveal_type
+
+def _(flag: bool):
+    class Iterator1:
+        if flag:
+            def __next__(self) -> int:
+                return 42
+        else:
+            def __next__(self, invalid_extra_arg) -> str:
+                return "foo"
+
+    class Iterator2:
+        if flag:
+            def __next__(self) -> int:
+                return 42
+        else:
+            __next__: None = None
+
+    class Iterable1:
+        def __iter__(self) -> Iterator1:
+            return Iterator1()
+
+    class Iterable2:
+        def __iter__(self) -> Iterator2:
+            return Iterator2()
+
+    # error: [not-iterable]
+    for x in Iterable1():
+        reveal_type(x)  # revealed: int | str
+
+    # error: [not-iterable]
+    for y in Iterable2():
+        # TODO: `int` would probably be better here:
+        reveal_type(y)  # revealed: int | Unknown
+```
+
+## Possibly invalid `__getitem__` methods
+
+<!-- snapshot-diagnostics -->
+
+```py
+from typing_extensions import reveal_type
+
+def _(flag: bool):
+    class Iterable1:
+        if flag:
+            def __getitem__(self, item: int) -> str:
+                return "foo"
+        else:
+            __getitem__: None = None
+
+    class Iterable2:
+        if flag:
+            def __getitem__(self, item: int) -> str:
+                return "foo"
+        else:
+            def __getitem__(self, item: str) -> int:
+                return "foo"
+
+    # error: [not-iterable]
+    for x in Iterable1():
+        # TODO: `str` might be better
+        reveal_type(x)  # revealed: str | Unknown
+
+    # error: [not-iterable]
+    for y in Iterable2():
+        reveal_type(y)  # revealed: str | int
+```
+
+## Possibly unbound `__iter__` and possibly invalid `__getitem__`
+
+<!-- snapshot-diagnostics -->
+
+```py
+from typing_extensions import reveal_type
+
+class Iterator:
+    def __next__(self) -> bytes:
+        return b"foo"
+
+def _(flag: bool, flag2: bool):
+    class Iterable1:
+        if flag:
+            def __getitem__(self, item: int) -> str:
+                return "foo"
+        else:
+            __getitem__: None = None
+
+        if flag2:
+            def __iter__(self) -> Iterator:
+                return Iterator()
+
+    class Iterable2:
+        if flag:
+            def __getitem__(self, item: int) -> str:
+                return "foo"
+        else:
+            def __getitem__(self, item: str) -> int:
+                return "foo"
+        if flag2:
+            def __iter__(self) -> Iterator:
+                return Iterator()
+
+    # error: [not-iterable]
+    for x in Iterable1():
+        # TODO: `bytes | str` might be better
+        reveal_type(x)  # revealed: bytes | str | Unknown
+
+    # error: [not-iterable]
+    for y in Iterable2():
+        reveal_type(y)  # revealed: bytes | str | int
+```

crates/red_knot_python_semantic/resources/mdtest/loops/while_loop.md

@@ -116,3 +116,14 @@ def _(flag: bool, flag2: bool):
     # error: [possibly-unresolved-reference]
     y
 ```
+
+## Condition with object that implements `__bool__` incorrectly
+
+```py
+class NotBoolable:
+    __bool__ = 3
+
+# error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable"
+while NotBoolable():
+    ...
+```

crates/red_knot_python_semantic/resources/mdtest/narrow/isinstance.md

@@ -97,12 +97,7 @@ else:
 ## No narrowing for instances of `builtins.type`
 
 ```py
-def _(flag: bool):
-    t = type("t", (), {})
-
-    # This isn't testing what we want it to test if we infer anything more precise here:
-    reveal_type(t)  # revealed: type
-
+def _(flag: bool, t: type):
     x = 1 if flag else "foo"
 
     if isinstance(x, t):

crates/red_knot_python_semantic/resources/mdtest/narrow/issubclass.md

@@ -112,8 +112,7 @@ def _(flag: bool):
         reveal_type(t)  # revealed: Literal[NoneType]
 
     if issubclass(t, type(None)):
-        # TODO: this should be just `Literal[NoneType]`
-        reveal_type(t)  # revealed: Literal[int, NoneType]
+        reveal_type(t)  # revealed: Literal[NoneType]
 ```
 
 ## `classinfo` contains multiple types

crates/red_knot_python_semantic/resources/mdtest/narrow/truthiness.md

@@ -266,7 +266,7 @@ def _(
     if af:
         reveal_type(af)  # revealed: type[AmbiguousClass] & ~AlwaysFalsy
 
-    # TODO: Emit a diagnostic (`d` is not valid in boolean context)
+    # error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `MetaDeferred`; the return type of its bool method (`MetaAmbiguous`) isn't assignable to `bool"
     if d:
         # TODO: Should be `Unknown`
         reveal_type(d)  # revealed: type[DeferredClass] & ~AlwaysFalsy

crates/red_knot_python_semantic/resources/mdtest/protocols.md

@@ -0,0 +1,15 @@
+# Protocols
+
+We do not support protocols yet, but to avoid false positives, we *partially* support some known
+protocols.
+
+## `typing.SupportsIndex`
+
+```py
+from typing import SupportsIndex, Literal
+
+def _(some_int: int, some_literal_int: Literal[1], some_indexable: SupportsIndex):
+    a: SupportsIndex = some_int
+    b: SupportsIndex = some_literal_int
+    c: SupportsIndex = some_indexable
+```

crates/red_knot_python_semantic/resources/mdtest/scopes/eager.md

@@ -341,10 +341,12 @@ annotation are looked up lazily, even if they occur in an eager scope.
 ### Eager annotations in a Python file
 
 ```py
+from typing import ClassVar
+
 x = int
 
 class C:
-    var: x
+    var: ClassVar[x]
 
 reveal_type(C.var)  # revealed: int
 
@@ -356,10 +358,12 @@ x = str
 ```py
 from __future__ import annotations
 
+from typing import ClassVar
+
 x = int
 
 class C:
-    var: x
+    var: ClassVar[x]
 
 reveal_type(C.var)  # revealed: Unknown | str
 
@@ -369,10 +373,12 @@ x = str
 ### Deferred annotations in a stub file
 
 ```pyi
+from typing import ClassVar
+
 x = int
 
 class C:
-    var: x
+    var: ClassVar[x]
 
 reveal_type(C.var)  # revealed: Unknown | str
 

crates/red_knot_python_semantic/resources/mdtest/scopes/moduletype_attrs.md

@@ -9,7 +9,7 @@ is unbound.
 ```py
 reveal_type(__name__)  # revealed: str
 reveal_type(__file__)  # revealed: str | None
-reveal_type(__loader__)  # revealed: LoaderProtocol | None
+reveal_type(__loader__)  # revealed: @Todo(instance attribute on class with dynamic base) | None
 reveal_type(__package__)  # revealed: str | None
 reveal_type(__doc__)  # revealed: str | None
 
@@ -54,10 +54,10 @@ inside the module:
 import typing
 
 reveal_type(typing.__name__)  # revealed: str
-reveal_type(typing.__init__)  # revealed: @Todo(bound method)
+reveal_type(typing.__init__)  # revealed: <bound method `__init__` of `ModuleType`>
 
 # These come from `builtins.object`, not `types.ModuleType`:
-reveal_type(typing.__eq__)  # revealed: @Todo(bound method)
+reveal_type(typing.__eq__)  # revealed: <bound method `__eq__` of `ModuleType`>
 
 reveal_type(typing.__class__)  # revealed: Literal[ModuleType]
 
@@ -136,3 +136,42 @@ if returns_bool():
 reveal_type(__file__)  # revealed: Literal[42]
 reveal_type(__name__)  # revealed: Literal[1] | str
 ```
+
+## Implicit global attributes in the current module override implicit globals from builtins
+
+Here, we take the type of the implicit global symbol `__name__` from the `types.ModuleType` stub
+(which in this custom typeshed specifies the type as `bytes`). This is because the `main` module has
+an implicit `__name__` global that shadows the builtin `__name__` symbol.
+
+```toml
+[environment]
+typeshed = "/typeshed"
+```
+
+`/typeshed/stdlib/builtins.pyi`:
+
+```pyi
+class int: ...
+class bytes: ...
+
+__name__: int = 42
+```
+
+`/typeshed/stdlib/types.pyi`:
+
+```pyi
+class ModuleType:
+    __name__: bytes
+```
+
+`/typeshed/stdlib/typing_extensions.pyi`:
+
+```pyi
+def reveal_type(obj, /): ...
+```
+
+`main.py`:
+
+```py
+reveal_type(__name__)  # revealed: bytes
+```

crates/red_knot_python_semantic/resources/mdtest/slots.md

@@ -167,7 +167,7 @@ class A:
     __slots__ = ()
     __slots__ += ("a", "b")
 
-reveal_type(A.__slots__)  # revealed: @Todo(return type)
+reveal_type(A.__slots__)  # revealed: @Todo(return type of decorated function)
 
 class B:
     __slots__ = ("c", "d")

crates/red_knot_python_semantic/resources/mdtest/snapshots/for.md_-_For_loops_-_Bad_`__getitem__`_method.snap

@@ -0,0 +1,52 @@
+---
+source: crates/red_knot_test/src/lib.rs
+expression: snapshot
+---
+---
+mdtest name: for.md - For loops - Bad `__getitem__` method
+mdtest path: crates/red_knot_python_semantic/resources/mdtest/loops/for.md
+---
+
+# Python source files
+
+## mdtest_snippet.py
+
+```
+ 1 | from typing_extensions import reveal_type
+ 2 | 
+ 3 | class Iterable:
+ 4 |     # invalid because it will implicitly be passed an `int`
+ 5 |     # by the interpreter
+ 6 |     def __getitem__(self, key: str) -> int:
+ 7 |         return 42
+ 8 | 
+ 9 | # error: [not-iterable]
+10 | for x in Iterable():
+11 |     reveal_type(x)  # revealed: int
+```
+
+# Diagnostics
+
+```
+error: lint:not-iterable
+  --> /src/mdtest_snippet.py:10:10
+   |
+ 9 | # error: [not-iterable]
+10 | for x in Iterable():
+   |          ^^^^^^^^^^ Object of type `Iterable` is not iterable because it has no `__iter__` method and its `__getitem__` method has an incorrect signature for the old-style iteration protocol (expected a signature at least as permissive as `def __getitem__(self, key: int): ...`)
+11 |     reveal_type(x)  # revealed: int
+   |
+
+```
+
+```
+info: revealed-type
+  --> /src/mdtest_snippet.py:11:5
+   |
+ 9 | # error: [not-iterable]
+10 | for x in Iterable():
+11 |     reveal_type(x)  # revealed: int
+   |     -------------- info: Revealed type is `int`
+   |
+
+```

crates/red_knot_python_semantic/resources/mdtest/snapshots/for.md_-_For_loops_-_Invalid_iterable.snap

@@ -0,0 +1,32 @@
+---
+source: crates/red_knot_test/src/lib.rs
+expression: snapshot
+---
+---
+mdtest name: for.md - For loops - Invalid iterable
+mdtest path: crates/red_knot_python_semantic/resources/mdtest/loops/for.md
+---
+
+# Python source files
+
+## mdtest_snippet.py
+
+```
+1 | nonsense = 123
+2 | for x in nonsense:  # error: [not-iterable]
+3 |     pass
+```
+
+# Diagnostics
+
+```
+error: lint:not-iterable
+ --> /src/mdtest_snippet.py:2:10
+  |
+1 | nonsense = 123
+2 | for x in nonsense:  # error: [not-iterable]
+  |          ^^^^^^^^ Object of type `Literal[123]` is not iterable because it doesn't have an `__iter__` method or a `__getitem__` method
+3 |     pass
+  |
+
+```

crates/red_knot_python_semantic/resources/mdtest/snapshots/for.md_-_For_loops_-_New_over_old_style_iteration_protocol.snap

@@ -0,0 +1,37 @@
+---
+source: crates/red_knot_test/src/lib.rs
+expression: snapshot
+---
+---
+mdtest name: for.md - For loops - New over old style iteration protocol
+mdtest path: crates/red_knot_python_semantic/resources/mdtest/loops/for.md
+---
+
+# Python source files
+
+## mdtest_snippet.py
+
+```
+1 | class NotIterable:
+2 |     def __getitem__(self, key: int) -> int:
+3 |         return 42
+4 |     __iter__: None = None
+5 | 
+6 | for x in NotIterable():  # error: [not-iterable]
+7 |     pass
+```
+
+# Diagnostics
+
+```
+error: lint:not-iterable
+ --> /src/mdtest_snippet.py:6:10
+  |
+4 |     __iter__: None = None
+5 |
+6 | for x in NotIterable():  # error: [not-iterable]
+  |          ^^^^^^^^^^^^^ Object of type `NotIterable` is not iterable because its `__iter__` attribute has type `None`, which is not callable
+7 |     pass
+  |
+
+```

crates/red_knot_python_semantic/resources/mdtest/snapshots/for.md_-_For_loops_-_No_`__iter__`_method_and_`__getitem__`_is_not_callable.snap

@@ -0,0 +1,49 @@
+---
+source: crates/red_knot_test/src/lib.rs
+expression: snapshot
+---
+---
+mdtest name: for.md - For loops - No `__iter__` method and `__getitem__` is not callable
+mdtest path: crates/red_knot_python_semantic/resources/mdtest/loops/for.md
+---
+
+# Python source files
+
+## mdtest_snippet.py
+
+```
+1 | from typing_extensions import reveal_type
+2 | 
+3 | class Bad:
+4 |     __getitem__: None = None
+5 | 
+6 | # error: [not-iterable]
+7 | for x in Bad():
+8 |     reveal_type(x)  # revealed: Unknown
+```
+
+# Diagnostics
+
+```
+error: lint:not-iterable
+ --> /src/mdtest_snippet.py:7:10
+  |
+6 | # error: [not-iterable]
+7 | for x in Bad():
+  |          ^^^^^ Object of type `Bad` is not iterable because it has no `__iter__` method and its `__getitem__` attribute has type `None`, which is not callable
+8 |     reveal_type(x)  # revealed: Unknown
+  |
+
+```
+
+```
+info: revealed-type
+ --> /src/mdtest_snippet.py:8:5
+  |
+6 | # error: [not-iterable]
+7 | for x in Bad():
+8 |     reveal_type(x)  # revealed: Unknown
+  |     -------------- info: Revealed type is `Unknown`
+  |
+
+```

crates/red_knot_python_semantic/resources/mdtest/snapshots/for.md_-_For_loops_-_Possibly-not-callable_`__getitem__`_method.snap

@@ -0,0 +1,98 @@
+---
+source: crates/red_knot_test/src/lib.rs
+expression: snapshot
+---
+---
+mdtest name: for.md - For loops - Possibly-not-callable `__getitem__` method
+mdtest path: crates/red_knot_python_semantic/resources/mdtest/loops/for.md
+---
+
+# Python source files
+
+## mdtest_snippet.py
+
+```
+ 1 | from typing_extensions import reveal_type
+ 2 | 
+ 3 | def _(flag: bool):
+ 4 |     class CustomCallable:
+ 5 |         if flag:
+ 6 |             def __call__(self, *args, **kwargs) -> int:
+ 7 |                 return 42
+ 8 |         else:
+ 9 |             __call__: None = None
+10 | 
+11 |     class Iterable1:
+12 |         __getitem__: CustomCallable = CustomCallable()
+13 | 
+14 |     class Iterable2:
+15 |         if flag:
+16 |             def __getitem__(self, key: int) -> int:
+17 |                 return 42
+18 |         else:
+19 |             __getitem__: None = None
+20 | 
+21 |     # error: [not-iterable]
+22 |     for x in Iterable1():
+23 |         # TODO... `int` might be ideal here?
+24 |         reveal_type(x)  # revealed: int | Unknown
+25 | 
+26 |     # error: [not-iterable]
+27 |     for y in Iterable2():
+28 |         # TODO... `int` might be ideal here?
+29 |         reveal_type(y)  # revealed: int | Unknown
+```
+
+# Diagnostics
+
+```
+error: lint:not-iterable
+  --> /src/mdtest_snippet.py:22:14
+   |
+21 |     # error: [not-iterable]
+22 |     for x in Iterable1():
+   |              ^^^^^^^^^^^ Object of type `Iterable1` may not be iterable because it has no `__iter__` method and its `__getitem__` attribute (with type `CustomCallable`) may not be callable
+23 |         # TODO... `int` might be ideal here?
+24 |         reveal_type(x)  # revealed: int | Unknown
+   |
+
+```
+
+```
+info: revealed-type
+  --> /src/mdtest_snippet.py:24:9
+   |
+22 |     for x in Iterable1():
+23 |         # TODO... `int` might be ideal here?
+24 |         reveal_type(x)  # revealed: int | Unknown
+   |         -------------- info: Revealed type is `int | Unknown`
+25 |
+26 |     # error: [not-iterable]
+   |
+
+```
+
+```
+error: lint:not-iterable
+  --> /src/mdtest_snippet.py:27:14
+   |
+26 |     # error: [not-iterable]
+27 |     for y in Iterable2():
+   |              ^^^^^^^^^^^ Object of type `Iterable2` may not be iterable because it has no `__iter__` method and its `__getitem__` attribute (with type `<bound method `__getitem__` of `Iterable2`> | None`) may not be callable
+28 |         # TODO... `int` might be ideal here?
+29 |         reveal_type(y)  # revealed: int | Unknown
+   |
+
+```
+
+```
+info: revealed-type
+  --> /src/mdtest_snippet.py:29:9
+   |
+27 |     for y in Iterable2():
+28 |         # TODO... `int` might be ideal here?
+29 |         reveal_type(y)  # revealed: int | Unknown
+   |         -------------- info: Revealed type is `int | Unknown`
+   |
+
+```

crates/red_knot_python_semantic/resources/mdtest/snapshots/for.md_-_For_loops_-_Possibly_invalid_`__getitem__`_methods.snap

@@ -0,0 +1,94 @@
+---
+source: crates/red_knot_test/src/lib.rs
+expression: snapshot
+---
+---
+mdtest name: for.md - For loops - Possibly invalid `__getitem__` methods
+mdtest path: crates/red_knot_python_semantic/resources/mdtest/loops/for.md
+---
+
+# Python source files
+
+## mdtest_snippet.py
+
+```
+ 1 | from typing_extensions import reveal_type
+ 2 | 
+ 3 | def _(flag: bool):
+ 4 |     class Iterable1:
+ 5 |         if flag:
+ 6 |             def __getitem__(self, item: int) -> str:
+ 7 |                 return "foo"
+ 8 |         else:
+ 9 |             __getitem__: None = None
+10 | 
+11 |     class Iterable2:
+12 |         if flag:
+13 |             def __getitem__(self, item: int) -> str:
+14 |                 return "foo"
+15 |         else:
+16 |             def __getitem__(self, item: str) -> int:
+17 |                 return "foo"
+18 | 
+19 |     # error: [not-iterable]
+20 |     for x in Iterable1():
+21 |         # TODO: `str` might be better
+22 |         reveal_type(x)  # revealed: str | Unknown
+23 | 
+24 |     # error: [not-iterable]
+25 |     for y in Iterable2():
+26 |         reveal_type(y)  # revealed: str | int
+```
+
+# Diagnostics
+
+```
+error: lint:not-iterable
+  --> /src/mdtest_snippet.py:20:14
+   |
+19 |     # error: [not-iterable]
+20 |     for x in Iterable1():
+   |              ^^^^^^^^^^^ Object of type `Iterable1` may not be iterable because it has no `__iter__` method and its `__getitem__` attribute (with type `<bound method `__getitem__` of `Iterable1`> | None`) may not be callable
+21 |         # TODO: `str` might be better
+22 |         reveal_type(x)  # revealed: str | Unknown
+   |
+
+```
+
+```
+info: revealed-type
+  --> /src/mdtest_snippet.py:22:9
+   |
+20 |     for x in Iterable1():
+21 |         # TODO: `str` might be better
+22 |         reveal_type(x)  # revealed: str | Unknown
+   |         -------------- info: Revealed type is `str | Unknown`
+23 |
+24 |     # error: [not-iterable]
+   |
+
+```
+
+```
+error: lint:not-iterable
+  --> /src/mdtest_snippet.py:25:14
+   |
+24 |     # error: [not-iterable]
+25 |     for y in Iterable2():
+   |              ^^^^^^^^^^^ Object of type `Iterable2` may not be iterable because it has no `__iter__` method and its `__getitem__` method (with type `<bound method `__getitem__` of `Iterable2`> | <bound method `__getitem__` of `Iterable2`>`) may have an incorrect signature for the old-style iteration protocol (expected a signature at least as permissive as `def __getitem__(self, key: int): ...`)
+26 |         reveal_type(y)  # revealed: str | int
+   |
+
+```
+
+```
+info: revealed-type
+  --> /src/mdtest_snippet.py:26:9
+   |
+24 |     # error: [not-iterable]
+25 |     for y in Iterable2():
+26 |         reveal_type(y)  # revealed: str | int
+   |         -------------- info: Revealed type is `str | int`
+   |
+
+```

crates/red_knot_python_semantic/resources/mdtest/snapshots/for.md_-_For_loops_-_Possibly_invalid_`__iter__`_methods.snap

@@ -0,0 +1,98 @@
+---
+source: crates/red_knot_test/src/lib.rs
+expression: snapshot
+---
+---
+mdtest name: for.md - For loops - Possibly invalid `__iter__` methods
+mdtest path: crates/red_knot_python_semantic/resources/mdtest/loops/for.md
+---
+
+# Python source files
+
+## mdtest_snippet.py
+
+```
+ 1 | from typing_extensions import reveal_type
+ 2 | 
+ 3 | class Iterator:
+ 4 |     def __next__(self) -> int:
+ 5 |         return 42
+ 6 | 
+ 7 | def _(flag: bool):
+ 8 |     class Iterable1:
+ 9 |         if flag:
+10 |             def __iter__(self) -> Iterator:
+11 |                 return Iterator()
+12 |         else:
+13 |             def __iter__(self, invalid_extra_arg) -> Iterator:
+14 |                 return Iterator()
+15 | 
+16 |     # error: [not-iterable]
+17 |     for x in Iterable1():
+18 |         reveal_type(x)  # revealed: int
+19 | 
+20 |     class Iterable2:
+21 |         if flag:
+22 |             def __iter__(self) -> Iterator:
+23 |                 return Iterator()
+24 |         else:
+25 |             __iter__: None = None
+26 | 
+27 |     # error: [not-iterable]
+28 |     for x in Iterable2():
+29 |         # TODO: `int` would probably be better here:
+30 |         reveal_type(x)  # revealed: int | Unknown
+```
+
+# Diagnostics
+
+```
+error: lint:not-iterable
+  --> /src/mdtest_snippet.py:17:14
+   |
+16 |     # error: [not-iterable]
+17 |     for x in Iterable1():
+   |              ^^^^^^^^^^^ Object of type `Iterable1` may not be iterable because its `__iter__` method (with type `<bound method `__iter__` of `Iterable1`> | <bound method `__iter__` of `Iterable1`>`) may have an invalid signature (expected `def __iter__(self): ...`)
+18 |         reveal_type(x)  # revealed: int
+   |
+
+```
+
+```
+info: revealed-type
+  --> /src/mdtest_snippet.py:18:9
+   |
+16 |     # error: [not-iterable]
+17 |     for x in Iterable1():
+18 |         reveal_type(x)  # revealed: int
+   |         -------------- info: Revealed type is `int`
+19 |
+20 |     class Iterable2:
+   |
+
+```
+
+```
+error: lint:not-iterable
+  --> /src/mdtest_snippet.py:28:14
+   |
+27 |     # error: [not-iterable]
+28 |     for x in Iterable2():
+   |              ^^^^^^^^^^^ Object of type `Iterable2` may not be iterable because its `__iter__` attribute (with type `<bound method `__iter__` of `Iterable2`> | None`) may not be callable
+29 |         # TODO: `int` would probably be better here:
+30 |         reveal_type(x)  # revealed: int | Unknown
+   |
+
+```
+
+```
+info: revealed-type
+  --> /src/mdtest_snippet.py:30:9
+   |
+28 |     for x in Iterable2():
+29 |         # TODO: `int` would probably be better here:
+30 |         reveal_type(x)  # revealed: int | Unknown
+   |         -------------- info: Revealed type is `int | Unknown`
+   |
+
+```

crates/red_knot_python_semantic/resources/mdtest/snapshots/for.md_-_For_loops_-_Possibly_invalid_`__next__`_method.snap

@@ -0,0 +1,102 @@
+---
+source: crates/red_knot_test/src/lib.rs
+expression: snapshot
+---
+---
+mdtest name: for.md - For loops - Possibly invalid `__next__` method
+mdtest path: crates/red_knot_python_semantic/resources/mdtest/loops/for.md
+---
+
+# Python source files
+
+## mdtest_snippet.py
+
+```
+ 1 | from typing_extensions import reveal_type
+ 2 | 
+ 3 | def _(flag: bool):
+ 4 |     class Iterator1:
+ 5 |         if flag:
+ 6 |             def __next__(self) -> int:
+ 7 |                 return 42
+ 8 |         else:
+ 9 |             def __next__(self, invalid_extra_arg) -> str:
+10 |                 return "foo"
+11 | 
+12 |     class Iterator2:
+13 |         if flag:
+14 |             def __next__(self) -> int:
+15 |                 return 42
+16 |         else:
+17 |             __next__: None = None
+18 | 
+19 |     class Iterable1:
+20 |         def __iter__(self) -> Iterator1:
+21 |             return Iterator1()
+22 | 
+23 |     class Iterable2:
+24 |         def __iter__(self) -> Iterator2:
+25 |             return Iterator2()
+26 | 
+27 |     # error: [not-iterable]
+28 |     for x in Iterable1():
+29 |         reveal_type(x)  # revealed: int | str
+30 | 
+31 |     # error: [not-iterable]
+32 |     for y in Iterable2():
+33 |         # TODO: `int` would probably be better here:
+34 |         reveal_type(y)  # revealed: int | Unknown
+```
+
+# Diagnostics
+
+```
+error: lint:not-iterable
+  --> /src/mdtest_snippet.py:28:14
+   |
+27 |     # error: [not-iterable]
+28 |     for x in Iterable1():
+   |              ^^^^^^^^^^^ Object of type `Iterable1` may not be iterable because its `__iter__` method returns an object of type `Iterator1`, which may have an invalid `__next__` method (expected `def __next__(self): ...`)
+29 |         reveal_type(x)  # revealed: int | str
+   |
+
+```
+
+```
+info: revealed-type
+  --> /src/mdtest_snippet.py:29:9
+   |
+27 |     # error: [not-iterable]
+28 |     for x in Iterable1():
+29 |         reveal_type(x)  # revealed: int | str
+   |         -------------- info: Revealed type is `int | str`
+30 |
+31 |     # error: [not-iterable]
+   |
+
+```
+
+```
+error: lint:not-iterable
+  --> /src/mdtest_snippet.py:32:14
+   |
+31 |     # error: [not-iterable]
+32 |     for y in Iterable2():
+   |              ^^^^^^^^^^^ Object of type `Iterable2` may not be iterable because its `__iter__` method returns an object of type `Iterator2`, which has a `__next__` attribute that may not be callable
+33 |         # TODO: `int` would probably be better here:
+34 |         reveal_type(y)  # revealed: int | Unknown
+   |
+
+```
+
+```
+info: revealed-type
+  --> /src/mdtest_snippet.py:34:9
+   |
+32 |     for y in Iterable2():
+33 |         # TODO: `int` would probably be better here:
+34 |         reveal_type(y)  # revealed: int | Unknown
+   |         -------------- info: Revealed type is `int | Unknown`
+   |
+
+```

crates/red_knot_python_semantic/resources/mdtest/snapshots/for.md_-_For_loops_-_Possibly_unbound_`__iter__`_and_bad_`__getitem__`_method.snap

@@ -0,0 +1,60 @@
+---
+source: crates/red_knot_test/src/lib.rs
+expression: snapshot
+---
+---
+mdtest name: for.md - For loops - Possibly unbound `__iter__` and bad `__getitem__` method
+mdtest path: crates/red_knot_python_semantic/resources/mdtest/loops/for.md
+---
+
+# Python source files
+
+## mdtest_snippet.py
+
+```
+ 1 | from typing_extensions import reveal_type
+ 2 | 
+ 3 | def _(flag: bool):
+ 4 |     class Iterator:
+ 5 |         def __next__(self) -> int:
+ 6 |             return 42
+ 7 | 
+ 8 |     class Iterable:
+ 9 |         if flag:
+10 |             def __iter__(self) -> Iterator:
+11 |                 return Iterator()
+12 |         # invalid signature because it only accepts a `str`,
+13 |         # but the old-style iteration protocol will pass it an `int`
+14 |         def __getitem__(self, key: str) -> bytes:
+15 |             return 42
+16 | 
+17 |     # error: [not-iterable]
+18 |     for x in Iterable():
+19 |         reveal_type(x)  # revealed: int | bytes
+```
+
+# Diagnostics
+
+```
+error: lint:not-iterable
+  --> /src/mdtest_snippet.py:18:14
+   |
+17 |     # error: [not-iterable]
+18 |     for x in Iterable():
+   |              ^^^^^^^^^^ Object of type `Iterable` may not be iterable because it may not have an `__iter__` method and its `__getitem__` method has an incorrect signature for the old-style iteration protocol (expected a signature at least as permissive as `def __getitem__(self, key: int): ...`)
+19 |         reveal_type(x)  # revealed: int | bytes
+   |
+
+```
+
+```
+info: revealed-type
+  --> /src/mdtest_snippet.py:19:9
+   |
+17 |     # error: [not-iterable]
+18 |     for x in Iterable():
+19 |         reveal_type(x)  # revealed: int | bytes
+   |         -------------- info: Revealed type is `int | bytes`
+   |
+
+```

crates/red_knot_python_semantic/resources/mdtest/snapshots/for.md_-_For_loops_-_Possibly_unbound_`__iter__`_and_possibly_invalid_`__getitem__`.snap

@@ -0,0 +1,106 @@
+---
+source: crates/red_knot_test/src/lib.rs
+expression: snapshot
+---
+---
+mdtest name: for.md - For loops - Possibly unbound `__iter__` and possibly invalid `__getitem__`
+mdtest path: crates/red_knot_python_semantic/resources/mdtest/loops/for.md
+---
+
+# Python source files
+
+## mdtest_snippet.py
+
+```
+ 1 | from typing_extensions import reveal_type
+ 2 | 
+ 3 | class Iterator:
+ 4 |     def __next__(self) -> bytes:
+ 5 |         return b"foo"
+ 6 | 
+ 7 | def _(flag: bool, flag2: bool):
+ 8 |     class Iterable1:
+ 9 |         if flag:
+10 |             def __getitem__(self, item: int) -> str:
+11 |                 return "foo"
+12 |         else:
+13 |             __getitem__: None = None
+14 | 
+15 |         if flag2:
+16 |             def __iter__(self) -> Iterator:
+17 |                 return Iterator()
+18 | 
+19 |     class Iterable2:
+20 |         if flag:
+21 |             def __getitem__(self, item: int) -> str:
+22 |                 return "foo"
+23 |         else:
+24 |             def __getitem__(self, item: str) -> int:
+25 |                 return "foo"
+26 |         if flag2:
+27 |             def __iter__(self) -> Iterator:
+28 |                 return Iterator()
+29 | 
+30 |     # error: [not-iterable]
+31 |     for x in Iterable1():
+32 |         # TODO: `bytes | str` might be better
+33 |         reveal_type(x)  # revealed: bytes | str | Unknown
+34 | 
+35 |     # error: [not-iterable]
+36 |     for y in Iterable2():
+37 |         reveal_type(y)  # revealed: bytes | str | int
+```
+
+# Diagnostics
+
+```
+error: lint:not-iterable
+  --> /src/mdtest_snippet.py:31:14
+   |
+30 |     # error: [not-iterable]
+31 |     for x in Iterable1():
+   |              ^^^^^^^^^^^ Object of type `Iterable1` may not be iterable because it may not have an `__iter__` method and its `__getitem__` attribute (with type `<bound method `__getitem__` of `Iterable1`> | None`) may not be callable
+32 |         # TODO: `bytes | str` might be better
+33 |         reveal_type(x)  # revealed: bytes | str | Unknown
+   |
+
+```
+
+```
+info: revealed-type
+  --> /src/mdtest_snippet.py:33:9
+   |
+31 |     for x in Iterable1():
+32 |         # TODO: `bytes | str` might be better
+33 |         reveal_type(x)  # revealed: bytes | str | Unknown
+   |         -------------- info: Revealed type is `bytes | str | Unknown`
+34 |
+35 |     # error: [not-iterable]
+   |
+
+```
+
+```
+error: lint:not-iterable
+  --> /src/mdtest_snippet.py:36:14
+   |
+35 |     # error: [not-iterable]
+36 |     for y in Iterable2():
+   |              ^^^^^^^^^^^ Object of type `Iterable2` may not be iterable because it may not have an `__iter__` method and its `__getitem__` method (with type `<bound method `__getitem__` of `Iterable2`> | <bound method `__getitem__` of `Iterable2`>`)
+                            may have an incorrect signature for the old-style iteration protocol (expected a signature at least as permissive as `def __getitem__(self, key: int): ...`)
+37 |         reveal_type(y)  # revealed: bytes | str | int
+   |
+
+```
+
+```
+info: revealed-type
+  --> /src/mdtest_snippet.py:37:9
+   |
+35 |     # error: [not-iterable]
+36 |     for y in Iterable2():
+37 |         reveal_type(y)  # revealed: bytes | str | int
+   |         -------------- info: Revealed type is `bytes | str | int`
+   |
+
+```

crates/red_knot_python_semantic/resources/mdtest/snapshots/for.md_-_For_loops_-_Possibly_unbound_`__iter__`_and_possibly_unbound_`__getitem__`.snap

@@ -0,0 +1,59 @@
+---
+source: crates/red_knot_test/src/lib.rs
+expression: snapshot
+---
+---
+mdtest name: for.md - For loops - Possibly unbound `__iter__` and possibly unbound `__getitem__`
+mdtest path: crates/red_knot_python_semantic/resources/mdtest/loops/for.md
+---
+
+# Python source files
+
+## mdtest_snippet.py
+
+```
+ 1 | from typing_extensions import reveal_type
+ 2 | 
+ 3 | class Iterator:
+ 4 |     def __next__(self) -> int:
+ 5 |         return 42
+ 6 | 
+ 7 | def _(flag1: bool, flag2: bool):
+ 8 |     class Iterable:
+ 9 |         if flag1:
+10 |             def __iter__(self) -> Iterator:
+11 |                 return Iterator()
+12 |         if flag2:
+13 |             def __getitem__(self, key: int) -> bytes:
+14 |                 return 42
+15 | 
+16 |     # error: [not-iterable]
+17 |     for x in Iterable():
+18 |         reveal_type(x)  # revealed: int | bytes
+```
+
+# Diagnostics
+
+```
+error: lint:not-iterable
+  --> /src/mdtest_snippet.py:17:14
+   |
+16 |     # error: [not-iterable]
+17 |     for x in Iterable():
+   |              ^^^^^^^^^^ Object of type `Iterable` may not be iterable because it may not have an `__iter__` method or a `__getitem__` method
+18 |         reveal_type(x)  # revealed: int | bytes
+   |
+
+```
+
+```
+info: revealed-type
+  --> /src/mdtest_snippet.py:18:9
+   |
+16 |     # error: [not-iterable]
+17 |     for x in Iterable():
+18 |         reveal_type(x)  # revealed: int | bytes
+   |         -------------- info: Revealed type is `int | bytes`
+   |
+
+```

crates/red_knot_python_semantic/resources/mdtest/snapshots/for.md_-_For_loops_-_Union_type_as_iterable_where_one_union_element_has_invalid_`__iter__`_method.snap

@@ -0,0 +1,61 @@
+---
+source: crates/red_knot_test/src/lib.rs
+expression: snapshot
+---
+---
+mdtest name: for.md - For loops - Union type as iterable where one union element has invalid `__iter__` method
+mdtest path: crates/red_knot_python_semantic/resources/mdtest/loops/for.md
+---
+
+# Python source files
+
+## mdtest_snippet.py
+
+```
+ 1 | from typing_extensions import reveal_type
+ 2 | 
+ 3 | class TestIter:
+ 4 |     def __next__(self) -> int:
+ 5 |         return 42
+ 6 | 
+ 7 | class Test:
+ 8 |     def __iter__(self) -> TestIter:
+ 9 |         return TestIter()
+10 | 
+11 | class Test2:
+12 |     def __iter__(self) -> int:
+13 |         return 42
+14 | 
+15 | def _(flag: bool):
+16 |     # TODO: Improve error message to state which union variant isn't iterable (https://github.com/astral-sh/ruff/issues/13989)
+17 |     # error: [not-iterable]
+18 |     for x in Test() if flag else Test2():
+19 |         reveal_type(x)  # revealed: int
+```
+
+# Diagnostics
+
+```
+error: lint:not-iterable
+  --> /src/mdtest_snippet.py:18:14
+   |
+16 |     # TODO: Improve error message to state which union variant isn't iterable (https://github.com/astral-sh/ruff/issues/13989)
+17 |     # error: [not-iterable]
+18 |     for x in Test() if flag else Test2():
+   |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Object of type `Test | Test2` may not be iterable because its `__iter__` method returns an object of type `TestIter | int`, which may not have a `__next__` method
+19 |         reveal_type(x)  # revealed: int
+   |
+
+```
+
+```
+info: revealed-type
+  --> /src/mdtest_snippet.py:19:9
+   |
+17 |     # error: [not-iterable]
+18 |     for x in Test() if flag else Test2():
+19 |         reveal_type(x)  # revealed: int
+   |         -------------- info: Revealed type is `int`
+   |
+
+```

crates/red_knot_python_semantic/resources/mdtest/snapshots/for.md_-_For_loops_-_Union_type_as_iterable_where_one_union_element_has_no_`__iter__`_method.snap

@@ -0,0 +1,56 @@
+---
+source: crates/red_knot_test/src/lib.rs
+expression: snapshot
+---
+---
+mdtest name: for.md - For loops - Union type as iterable where one union element has no `__iter__` method
+mdtest path: crates/red_knot_python_semantic/resources/mdtest/loops/for.md
+---
+
+# Python source files
+
+## mdtest_snippet.py
+
+```
+ 1 | from typing_extensions import reveal_type
+ 2 | 
+ 3 | class TestIter:
+ 4 |     def __next__(self) -> int:
+ 5 |         return 42
+ 6 | 
+ 7 | class Test:
+ 8 |     def __iter__(self) -> TestIter:
+ 9 |         return TestIter()
+10 | 
+11 | def _(flag: bool):
+12 |     # error: [not-iterable]
+13 |     for x in Test() if flag else 42:
+14 |         reveal_type(x)  # revealed: int
+```
+
+# Diagnostics
+
+```
+error: lint:not-iterable
+  --> /src/mdtest_snippet.py:13:14
+   |
+11 | def _(flag: bool):
+12 |     # error: [not-iterable]
+13 |     for x in Test() if flag else 42:
+   |              ^^^^^^^^^^^^^^^^^^^^^^ Object of type `Test | Literal[42]` may not be iterable because it may not have an `__iter__` method and it doesn't have a `__getitem__` method
+14 |         reveal_type(x)  # revealed: int
+   |
+
+```
+
+```
+info: revealed-type
+  --> /src/mdtest_snippet.py:14:9
+   |
+12 |     # error: [not-iterable]
+13 |     for x in Test() if flag else 42:
+14 |         reveal_type(x)  # revealed: int
+   |         -------------- info: Revealed type is `int`
+   |
+
+```

crates/red_knot_python_semantic/resources/mdtest/snapshots/for.md_-_For_loops_-_With_non-callable_iterator.snap

@@ -0,0 +1,69 @@
+---
+source: crates/red_knot_test/src/lib.rs
+expression: snapshot
+---
+---
+mdtest name: for.md - For loops - With non-callable iterator
+mdtest path: crates/red_knot_python_semantic/resources/mdtest/loops/for.md
+---
+
+# Python source files
+
+## mdtest_snippet.py
+
+```
+ 1 | from typing_extensions import reveal_type
+ 2 | 
+ 3 | def _(flag: bool):
+ 4 |     class NotIterable:
+ 5 |         if flag:
+ 6 |             __iter__: int = 1
+ 7 |         else:
+ 8 |             __iter__: None = None
+ 9 | 
+10 |     # error: [not-iterable]
+11 |     for x in NotIterable():
+12 |         pass
+13 | 
+14 |     # revealed: Unknown
+15 |     # error: [possibly-unresolved-reference]
+16 |     reveal_type(x)
+```
+
+# Diagnostics
+
+```
+error: lint:not-iterable
+  --> /src/mdtest_snippet.py:11:14
+   |
+10 |     # error: [not-iterable]
+11 |     for x in NotIterable():
+   |              ^^^^^^^^^^^^^ Object of type `NotIterable` is not iterable because its `__iter__` attribute has type `int | None`, which is not callable
+12 |         pass
+   |
+
+```
+
+```
+warning: lint:possibly-unresolved-reference
+  --> /src/mdtest_snippet.py:16:17
+   |
+14 |     # revealed: Unknown
+15 |     # error: [possibly-unresolved-reference]
+16 |     reveal_type(x)
+   |                 - Name `x` used when possibly not defined
+   |
+
+```
+
+```
+info: revealed-type
+  --> /src/mdtest_snippet.py:16:5
+   |
+14 |     # revealed: Unknown
+15 |     # error: [possibly-unresolved-reference]
+16 |     reveal_type(x)
+   |     -------------- info: Revealed type is `Unknown`
+   |
+
+```

crates/red_knot_python_semantic/resources/mdtest/snapshots/for.md_-_For_loops_-_`__iter__`_does_not_return_an_iterator.snap

@@ -0,0 +1,50 @@
+---
+source: crates/red_knot_test/src/lib.rs
+expression: snapshot
+---
+---
+mdtest name: for.md - For loops - `__iter__` does not return an iterator
+mdtest path: crates/red_knot_python_semantic/resources/mdtest/loops/for.md
+---
+
+# Python source files
+
+## mdtest_snippet.py
+
+```
+1 | from typing_extensions import reveal_type
+2 | 
+3 | class Bad:
+4 |     def __iter__(self) -> int:
+5 |         return 42
+6 | 
+7 | # error: [not-iterable]
+8 | for x in Bad():
+9 |     reveal_type(x)  # revealed: Unknown
+```
+
+# Diagnostics
+
+```
+error: lint:not-iterable
+ --> /src/mdtest_snippet.py:8:10
+  |
+7 | # error: [not-iterable]
+8 | for x in Bad():
+  |          ^^^^^ Object of type `Bad` is not iterable because its `__iter__` method returns an object of type `int`, which has no `__next__` method
+9 |     reveal_type(x)  # revealed: Unknown
+  |
+
+```
+
+```
+info: revealed-type
+ --> /src/mdtest_snippet.py:9:5
+  |
+7 | # error: [not-iterable]
+8 | for x in Bad():
+9 |     reveal_type(x)  # revealed: Unknown
+  |     -------------- info: Revealed type is `Unknown`
+  |
+
+```

crates/red_knot_python_semantic/resources/mdtest/snapshots/for.md_-_For_loops_-_`__iter__`_method_with_a_bad_signature.snap

@@ -0,0 +1,54 @@
+---
+source: crates/red_knot_test/src/lib.rs
+expression: snapshot
+---
+---
+mdtest name: for.md - For loops - `__iter__` method with a bad signature
+mdtest path: crates/red_knot_python_semantic/resources/mdtest/loops/for.md
+---
+
+# Python source files
+
+## mdtest_snippet.py
+
+```
+ 1 | from typing_extensions import reveal_type
+ 2 | 
+ 3 | class Iterator:
+ 4 |     def __next__(self) -> int:
+ 5 |         return 42
+ 6 | 
+ 7 | class Iterable:
+ 8 |     def __iter__(self, extra_arg) -> Iterator:
+ 9 |         return Iterator()
+10 | 
+11 | # error: [not-iterable]
+12 | for x in Iterable():
+13 |     reveal_type(x)  # revealed: int
+```
+
+# Diagnostics
+
+```
+error: lint:not-iterable
+  --> /src/mdtest_snippet.py:12:10
+   |
+11 | # error: [not-iterable]
+12 | for x in Iterable():
+   |          ^^^^^^^^^^ Object of type `Iterable` is not iterable because its `__iter__` method has an invalid signature (expected `def __iter__(self): ...`)
+13 |     reveal_type(x)  # revealed: int
+   |
+
+```
+
+```
+info: revealed-type
+  --> /src/mdtest_snippet.py:13:5
+   |
+11 | # error: [not-iterable]
+12 | for x in Iterable():
+13 |     reveal_type(x)  # revealed: int
+   |     -------------- info: Revealed type is `int`
+   |
+
+```

crates/red_knot_python_semantic/resources/mdtest/snapshots/for.md_-_For_loops_-_`__iter__`_returns_an_iterator_with_an_invalid_`__next__`_method.snap

@@ -0,0 +1,91 @@
+---
+source: crates/red_knot_test/src/lib.rs
+expression: snapshot
+---
+---
+mdtest name: for.md - For loops - `__iter__` returns an iterator with an invalid `__next__` method
+mdtest path: crates/red_knot_python_semantic/resources/mdtest/loops/for.md
+---
+
+# Python source files
+
+## mdtest_snippet.py
+
+```
+ 1 | from typing_extensions import reveal_type
+ 2 | 
+ 3 | class Iterator1:
+ 4 |     def __next__(self, extra_arg) -> int:
+ 5 |         return 42
+ 6 | 
+ 7 | class Iterator2:
+ 8 |     __next__: None = None
+ 9 | 
+10 | class Iterable1:
+11 |     def __iter__(self) -> Iterator1:
+12 |         return Iterator1()
+13 | 
+14 | class Iterable2:
+15 |     def __iter__(self) -> Iterator2:
+16 |         return Iterator2()
+17 | 
+18 | # error: [not-iterable]
+19 | for x in Iterable1():
+20 |     reveal_type(x)  # revealed: int
+21 | 
+22 | # error: [not-iterable]
+23 | for y in Iterable2():
+24 |     reveal_type(y)  # revealed: Unknown
+```
+
+# Diagnostics
+
+```
+error: lint:not-iterable
+  --> /src/mdtest_snippet.py:19:10
+   |
+18 | # error: [not-iterable]
+19 | for x in Iterable1():
+   |          ^^^^^^^^^^^ Object of type `Iterable1` is not iterable because its `__iter__` method returns an object of type `Iterator1`, which has an invalid `__next__` method (expected `def __next__(self): ...`)
+20 |     reveal_type(x)  # revealed: int
+   |
+
+```
+
+```
+info: revealed-type
+  --> /src/mdtest_snippet.py:20:5
+   |
+18 | # error: [not-iterable]
+19 | for x in Iterable1():
+20 |     reveal_type(x)  # revealed: int
+   |     -------------- info: Revealed type is `int`
+21 |
+22 | # error: [not-iterable]
+   |
+
+```
+
+```
+error: lint:not-iterable
+  --> /src/mdtest_snippet.py:23:10
+   |
+22 | # error: [not-iterable]
+23 | for y in Iterable2():
+   |          ^^^^^^^^^^^ Object of type `Iterable2` is not iterable because its `__iter__` method returns an object of type `Iterator2`, which has a `__next__` attribute that is not callable
+24 |     reveal_type(y)  # revealed: Unknown
+   |
+
+```
+
+```
+info: revealed-type
+  --> /src/mdtest_snippet.py:24:5
+   |
+22 | # error: [not-iterable]
+23 | for y in Iterable2():
+24 |     reveal_type(y)  # revealed: Unknown
+   |     -------------- info: Revealed type is `Unknown`
+   |
+
+```

crates/red_knot_python_semantic/resources/mdtest/snapshots/instances.md_-_Binary_operations_on_instances_-_Operations_involving_types_with_invalid_`__bool__`_methods.snap

@@ -0,0 +1,35 @@
+---
+source: crates/red_knot_test/src/lib.rs
+expression: snapshot
+---
+---
+mdtest name: instances.md - Binary operations on instances - Operations involving types with invalid `__bool__` methods
+mdtest path: crates/red_knot_python_semantic/resources/mdtest/binary/instances.md
+---
+
+# Python source files
+
+## mdtest_snippet.py
+
+```
+1 | class NotBoolable:
+2 |     __bool__ = 3
+3 | 
+4 | a = NotBoolable()
+5 | 
+6 | # error: [unsupported-bool-conversion]
+7 | 10 and a and True
+```
+
+# Diagnostics
+
+```
+error: lint:unsupported-bool-conversion
+ --> /src/mdtest_snippet.py:7:8
+  |
+6 | # error: [unsupported-bool-conversion]
+7 | 10 and a and True
+  |        ^ Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable
+  |
+
+```

crates/red_knot_python_semantic/resources/mdtest/snapshots/invalid_argument_type.md_-_Invalid_argument_type_diagnostics_-_Calls_to_methods.snap

@@ -0,0 +1,41 @@
+---
+source: crates/red_knot_test/src/lib.rs
+expression: snapshot
+---
+---
+mdtest name: invalid_argument_type.md - Invalid argument type diagnostics - Calls to methods
+mdtest path: crates/red_knot_python_semantic/resources/mdtest/diagnostics/invalid_argument_type.md
+---
+
+# Python source files
+
+## mdtest_snippet.py
+
+```
+1 | class C:
+2 |     def square(self, x: int) -> int:
+3 |         return x * x
+4 | 
+5 | c = C()
+6 | c.square("hello")  # error: [invalid-argument-type]
+```
+
+# Diagnostics
+
+```
+error: lint:invalid-argument-type
+ --> /src/mdtest_snippet.py:6:10
+  |
+5 | c = C()
+6 | c.square("hello")  # error: [invalid-argument-type]
+  |          ^^^^^^^ Object of type `Literal["hello"]` cannot be assigned to parameter 2 (`x`) of bound method `square`; expected type `int`
+  |
+ ::: /src/mdtest_snippet.py:2:22
+  |
+1 | class C:
+2 |     def square(self, x: int) -> int:
+  |                      ------ info: parameter declared in function definition here
+3 |         return x * x
+  |
+
+```

crates/red_knot_python_semantic/resources/mdtest/snapshots/invalid_argument_type.md_-_Invalid_argument_type_diagnostics_-_Tests_for_a_variety_of_argument_types_-_Synthetic_arguments.snap

@@ -28,7 +28,7 @@ error: lint:invalid-argument-type
   |
 5 | c = C()
 6 | c("wrong")  # error: [invalid-argument-type]
-  |   ^^^^^^^ Object of type `Literal["wrong"]` cannot be assigned to parameter 2 (`x`) of function `__call__`; expected type `int`
+  |   ^^^^^^^ Object of type `Literal["wrong"]` cannot be assigned to parameter 2 (`x`) of bound method `__call__`; expected type `int`
   |
  ::: /src/mdtest_snippet.py:2:24
   |

crates/red_knot_python_semantic/resources/mdtest/snapshots/membership_test.md_-_Comparison___Membership_Test_-_Return_type_that_doesn't_implement_`__bool__`_correctly.snap

@@ -0,0 +1,53 @@
+---
+source: crates/red_knot_test/src/lib.rs
+expression: snapshot
+---
+---
+mdtest name: membership_test.md - Comparison: Membership Test - Return type that doesn't implement `__bool__` correctly
+mdtest path: crates/red_knot_python_semantic/resources/mdtest/comparison/instances/membership_test.md
+---
+
+# Python source files
+
+## mdtest_snippet.py
+
+```
+ 1 | class NotBoolable:
+ 2 |     __bool__ = 3
+ 3 | 
+ 4 | class WithContains:
+ 5 |     def __contains__(self, item) -> NotBoolable:
+ 6 |         return NotBoolable()
+ 7 | 
+ 8 | # error: [unsupported-bool-conversion]
+ 9 | 10 in WithContains()
+10 | # error: [unsupported-bool-conversion]
+11 | 10 not in WithContains()
+```
+
+# Diagnostics
+
+```
+error: lint:unsupported-bool-conversion
+  --> /src/mdtest_snippet.py:9:1
+   |
+ 8 | # error: [unsupported-bool-conversion]
+ 9 | 10 in WithContains()
+   | ^^^^^^^^^^^^^^^^^^^^ Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable
+10 | # error: [unsupported-bool-conversion]
+11 | 10 not in WithContains()
+   |
+
+```
+
+```
+error: lint:unsupported-bool-conversion
+  --> /src/mdtest_snippet.py:11:1
+   |
+ 9 | 10 in WithContains()
+10 | # error: [unsupported-bool-conversion]
+11 | 10 not in WithContains()
+   | ^^^^^^^^^^^^^^^^^^^^^^^^ Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable
+   |
+
+```

crates/red_knot_python_semantic/resources/mdtest/snapshots/not.md_-_Unary_not_-_Object_that_implements_`__bool__`_incorrectly.snap

@@ -0,0 +1,33 @@
+---
+source: crates/red_knot_test/src/lib.rs
+expression: snapshot
+---
+---
+mdtest name: not.md - Unary not - Object that implements `__bool__` incorrectly
+mdtest path: crates/red_knot_python_semantic/resources/mdtest/unary/not.md
+---
+
+# Python source files
+
+## mdtest_snippet.py
+
+```
+1 | class NotBoolable:
+2 |     __bool__ = 3
+3 | 
+4 | # error: [unsupported-bool-conversion]
+5 | not NotBoolable()
+```
+
+# Diagnostics
+
+```
+error: lint:unsupported-bool-conversion
+ --> /src/mdtest_snippet.py:5:1
+  |
+4 | # error: [unsupported-bool-conversion]
+5 | not NotBoolable()
+  | ^^^^^^^^^^^^^^^^^ Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable
+  |
+
+```

crates/red_knot_python_semantic/resources/mdtest/snapshots/rich_comparison.md_-_Comparison___Rich_Comparison_-_Chained_comparisons_with_objects_that_don't_implement_`__bool__`_correctly.snap

@@ -0,0 +1,60 @@
+---
+source: crates/red_knot_test/src/lib.rs
+expression: snapshot
+---
+---
+mdtest name: rich_comparison.md - Comparison: Rich Comparison - Chained comparisons with objects that don't implement `__bool__` correctly
+mdtest path: crates/red_knot_python_semantic/resources/mdtest/comparison/instances/rich_comparison.md
+---
+
+# Python source files
+
+## mdtest_snippet.py
+
+```
+ 1 | class NotBoolable:
+ 2 |     __bool__ = 3
+ 3 | 
+ 4 | class Comparable:
+ 5 |     def __lt__(self, item) -> NotBoolable:
+ 6 |         return NotBoolable()
+ 7 | 
+ 8 |     def __gt__(self, item) -> NotBoolable:
+ 9 |         return NotBoolable()
+10 | 
+11 | # error: [unsupported-bool-conversion]
+12 | 10 < Comparable() < 20
+13 | # error: [unsupported-bool-conversion]
+14 | 10 < Comparable() < Comparable()
+15 | 
+16 | Comparable() < Comparable()  # fine
+```
+
+# Diagnostics
+
+```
+error: lint:unsupported-bool-conversion
+  --> /src/mdtest_snippet.py:12:1
+   |
+11 | # error: [unsupported-bool-conversion]
+12 | 10 < Comparable() < 20
+   | ^^^^^^^^^^^^^^^^^ Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable
+13 | # error: [unsupported-bool-conversion]
+14 | 10 < Comparable() < Comparable()
+   |
+
+```
+
+```
+error: lint:unsupported-bool-conversion
+  --> /src/mdtest_snippet.py:14:1
+   |
+12 | 10 < Comparable() < 20
+13 | # error: [unsupported-bool-conversion]
+14 | 10 < Comparable() < Comparable()
+   | ^^^^^^^^^^^^^^^^^ Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable
+15 |
+16 | Comparable() < Comparable()  # fine
+   |
+
+```

crates/red_knot_python_semantic/resources/mdtest/snapshots/tuples.md_-_Comparison___Tuples_-_Chained_comparisons_with_elements_that_incorrectly_implement_`__bool__`.snap

@@ -0,0 +1,47 @@
+---
+source: crates/red_knot_test/src/lib.rs
+expression: snapshot
+---
+---
+mdtest name: tuples.md - Comparison: Tuples - Chained comparisons with elements that incorrectly implement `__bool__`
+mdtest path: crates/red_knot_python_semantic/resources/mdtest/comparison/tuples.md
+---
+
+# Python source files
+
+## mdtest_snippet.py
+
+```
+ 1 | class NotBoolable:
+ 2 |     __bool__ = 5
+ 3 | 
+ 4 | class Comparable:
+ 5 |     def __lt__(self, other) -> NotBoolable:
+ 6 |         return NotBoolable()
+ 7 | 
+ 8 |     def __gt__(self, other) -> NotBoolable:
+ 9 |         return NotBoolable()
+10 | 
+11 | a = (1, Comparable())
+12 | b = (1, Comparable())
+13 | 
+14 | # error: [unsupported-bool-conversion]
+15 | a < b < b
+16 | 
+17 | a < b  # fine
+```
+
+# Diagnostics
+
+```
+error: lint:unsupported-bool-conversion
+  --> /src/mdtest_snippet.py:15:1
+   |
+14 | # error: [unsupported-bool-conversion]
+15 | a < b < b
+   | ^^^^^ Boolean conversion is unsupported for type `NotBoolable | Literal[False]`; its `__bool__` method isn't callable
+16 |
+17 | a < b  # fine
+   |
+
+```

crates/red_knot_python_semantic/resources/mdtest/snapshots/tuples.md_-_Comparison___Tuples_-_Equality_with_elements_that_incorrectly_implement_`__bool__`.snap

@@ -0,0 +1,37 @@
+---
+source: crates/red_knot_test/src/lib.rs
+expression: snapshot
+---
+---
+mdtest name: tuples.md - Comparison: Tuples - Equality with elements that incorrectly implement `__bool__`
+mdtest path: crates/red_knot_python_semantic/resources/mdtest/comparison/tuples.md
+---
+
+# Python source files
+
+## mdtest_snippet.py
+
+```
+1 | class A:
+2 |     def __eq__(self, other) -> NotBoolable:
+3 |         return NotBoolable()
+4 | 
+5 | class NotBoolable:
+6 |     __bool__ = None
+7 | 
+8 | # error: [unsupported-bool-conversion]
+9 | (A(),) == (A(),)
+```
+
+# Diagnostics
+
+```
+error: lint:unsupported-bool-conversion
+ --> /src/mdtest_snippet.py:9:1
+  |
+8 | # error: [unsupported-bool-conversion]
+9 | (A(),) == (A(),)
+  | ^^^^^^^^^^^^^^^^ Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable
+  |
+
+```

crates/red_knot_python_semantic/resources/mdtest/snapshots/unpacking.md_-_Unpacking_-_Right_hand_side_not_iterable.snap

@@ -22,7 +22,7 @@ error: lint:not-iterable
  --> /src/mdtest_snippet.py:1:8
   |
 1 | a, b = 1  # error: [not-iterable]
-  |        ^ Object of type `Literal[1]` is not iterable
+  |        ^ Object of type `Literal[1]` is not iterable because it doesn't have an `__iter__` method or a `__getitem__` method
   |
 
 ```

crates/red_knot_python_semantic/resources/mdtest/subscript/bytes.md

@@ -25,7 +25,7 @@ reveal_type(y)  # revealed: Unknown
 def _(n: int):
     a = b"abcde"[n]
     # TODO: Support overloads... Should be `bytes`
-    reveal_type(a)  # revealed: @Todo(return type)
+    reveal_type(a)  # revealed: @Todo(return type of decorated function)
 ```
 
 ## Slices
@@ -44,10 +44,10 @@ b[::0]  # error: [zero-stepsize-in-slice]
 def _(m: int, n: int):
     byte_slice1 = b[m:n]
     # TODO: Support overloads... Should be `bytes`
-    reveal_type(byte_slice1)  # revealed: @Todo(return type)
+    reveal_type(byte_slice1)  # revealed: @Todo(return type of decorated function)
 
 def _(s: bytes) -> bytes:
     byte_slice2 = s[0:5]
     # TODO: Support overloads... Should be `bytes`
-    reveal_type(byte_slice2)  # revealed: @Todo(return type)
+    reveal_type(byte_slice2)  # revealed: @Todo(return type of decorated function)
 ```

crates/red_knot_python_semantic/resources/mdtest/subscript/lists.md

@@ -12,13 +12,13 @@ x = [1, 2, 3]
 reveal_type(x)  # revealed: list
 
 # TODO reveal int
-reveal_type(x[0])  # revealed: @Todo(return type)
+reveal_type(x[0])  # revealed: @Todo(return type of decorated function)
 
 # TODO reveal list
-reveal_type(x[0:1])  # revealed: @Todo(return type)
+reveal_type(x[0:1])  # revealed: @Todo(return type of decorated function)
 
 # TODO error
-reveal_type(x["a"])  # revealed: @Todo(return type)
+reveal_type(x["a"])  # revealed: @Todo(return type of decorated function)
 ```
 
 ## Assignments within list assignment

crates/red_knot_python_semantic/resources/mdtest/subscript/string.md

@@ -22,7 +22,7 @@ reveal_type(b)  # revealed: Unknown
 def _(n: int):
     a = "abcde"[n]
     # TODO: Support overloads... Should be `str`
-    reveal_type(a)  # revealed: @Todo(return type)
+    reveal_type(a)  # revealed: @Todo(return type of decorated function)
 ```
 
 ## Slices
@@ -76,11 +76,11 @@ def _(m: int, n: int, s2: str):
 
     substring1 = s[m:n]
     # TODO: Support overloads... Should be `LiteralString`
-    reveal_type(substring1)  # revealed: @Todo(return type)
+    reveal_type(substring1)  # revealed: @Todo(return type of decorated function)
 
     substring2 = s2[0:5]
     # TODO: Support overloads... Should be `str`
-    reveal_type(substring2)  # revealed: @Todo(return type)
+    reveal_type(substring2)  # revealed: @Todo(return type of decorated function)
 ```
 
 ## Unsupported slice types

crates/red_knot_python_semantic/resources/mdtest/subscript/tuple.md

@@ -70,7 +70,7 @@ def _(m: int, n: int):
 
     tuple_slice = t[m:n]
     # TODO: Support overloads... Should be `tuple[Literal[1, 'a', b"b"] | None, ...]`
-    reveal_type(tuple_slice)  # revealed: @Todo(return type)
+    reveal_type(tuple_slice)  # revealed: @Todo(return type of decorated function)
 ```
 
 ## Inheritance

crates/red_knot_python_semantic/resources/mdtest/sys_platform.md

@@ -66,6 +66,6 @@ It is [recommended](https://docs.python.org/3/library/sys.html#sys.platform) to
 ```py
 import sys
 
-reveal_type(sys.platform.startswith("freebsd"))  # revealed: @Todo(Attribute access on `LiteralString` types)
-reveal_type(sys.platform.startswith("linux"))  # revealed: @Todo(Attribute access on `LiteralString` types)
+reveal_type(sys.platform.startswith("freebsd"))  # revealed: bool
+reveal_type(sys.platform.startswith("linux"))  # revealed: bool
 ```

crates/red_knot_python_semantic/resources/mdtest/type_api.md

@@ -223,7 +223,7 @@ class InvalidBoolDunder:
     def __bool__(self) -> int:
         return 1
 
-# error: "Static assertion error: argument of type `InvalidBoolDunder` has an ambiguous static truthiness"
+# error: [unsupported-bool-conversion]  "Boolean conversion is unsupported for type `InvalidBoolDunder`; the return type of its bool method (`int`) isn't assignable to `bool"
 static_assert(InvalidBoolDunder())
 ```
 

crates/red_knot_python_semantic/resources/mdtest/type_of/dynamic.md

@@ -35,7 +35,7 @@ in strict mode.
 ```py
 def f(x: type):
     reveal_type(x)  # revealed: type
-    reveal_type(x.__repr__)  # revealed: @Todo(bound method)
+    reveal_type(x.__repr__)  # revealed: <bound method `__repr__` of `type`>
 
 class A: ...
 
@@ -50,7 +50,7 @@ x: type = A()  # error: [invalid-assignment]
 ```py
 def f(x: type[object]):
     reveal_type(x)  # revealed: type
-    reveal_type(x.__repr__)  # revealed: @Todo(bound method)
+    reveal_type(x.__repr__)  # revealed: <bound method `__repr__` of `type`>
 
 class A: ...
 

crates/red_knot_python_semantic/resources/mdtest/type_properties/truthiness.md

@@ -75,3 +75,48 @@ class Boom:
 
 reveal_type(bool(Boom()))  # revealed: bool
 ```
+
+### Possibly unbound __bool__ method
+
+```py
+from typing import Literal
+
+def flag() -> bool:
+    return True
+
+class PossiblyUnboundTrue:
+    if flag():
+        def __bool__(self) -> Literal[True]:
+            return True
+
+reveal_type(bool(PossiblyUnboundTrue()))  # revealed: bool
+```
+
+### Special-cased classes
+
+Some special-cased `@final` classes are known by red-knot to have instances that are either always
+truthy or always falsy.
+
+```toml
+[environment]
+python-version = "3.12"
+```
+
+```py
+import types
+import typing
+import sys
+from knot_extensions import AlwaysTruthy, static_assert, is_subtype_of
+from typing_extensions import _NoDefaultType
+
+static_assert(is_subtype_of(sys.version_info.__class__, AlwaysTruthy))
+static_assert(is_subtype_of(types.EllipsisType, AlwaysTruthy))
+static_assert(is_subtype_of(_NoDefaultType, AlwaysTruthy))
+static_assert(is_subtype_of(slice, AlwaysTruthy))
+static_assert(is_subtype_of(types.FunctionType, AlwaysTruthy))
+static_assert(is_subtype_of(types.MethodType, AlwaysTruthy))
+static_assert(is_subtype_of(typing.TypeVar, AlwaysTruthy))
+static_assert(is_subtype_of(typing.TypeAliasType, AlwaysTruthy))
+static_assert(is_subtype_of(types.MethodWrapperType, AlwaysTruthy))
+static_assert(is_subtype_of(types.WrapperDescriptorType, AlwaysTruthy))
+```

crates/red_knot_python_semantic/resources/mdtest/type_qualifiers/classvar.md

@@ -64,6 +64,24 @@ c = C()
 c.a = 2
 ```
 
+and similarly here:
+
+```py
+class Base:
+    a: ClassVar[int] = 1
+
+class Derived(Base):
+    if flag():
+        a: int
+
+reveal_type(Derived.a)  # revealed: int
+
+d = Derived()
+
+# error: [invalid-attribute-access]
+d.a = 2
+```
+
 ## Too many arguments
 
 ```py

crates/red_knot_python_semantic/resources/mdtest/unary/not.md

@@ -183,12 +183,11 @@ class WithBothLenAndBool2:
 # revealed: Literal[False]
 reveal_type(not WithBothLenAndBool2())
 
-# TODO: raise diagnostic when __bool__ method is not valid: [unsupported-operator] "Method __bool__ for type `MethodBoolInvalid` should return `bool`, returned type `int`"
-# https://docs.python.org/3/reference/datamodel.html#object.__bool__
 class MethodBoolInvalid:
     def __bool__(self) -> int:
         return 0
 
+# error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `MethodBoolInvalid`; the return type of its bool method (`int`) isn't assignable to `bool"
 # revealed: bool
 reveal_type(not MethodBoolInvalid())
 
@@ -204,3 +203,15 @@ class PossiblyUnboundBool:
 # revealed: bool
 reveal_type(not PossiblyUnboundBool())
 ```
+
+## Object that implements `__bool__` incorrectly
+
+<!-- snapshot-diagnostics -->
+
+```py
+class NotBoolable:
+    __bool__ = 3
+
+# error: [unsupported-bool-conversion]
+not NotBoolable()
+```

crates/red_knot_python_semantic/src/lib.rs

@@ -7,7 +7,7 @@ use crate::suppression::{INVALID_IGNORE_COMMENT, UNKNOWN_RULE, UNUSED_IGNORE_COM
 pub use db::Db;
 pub use module_name::ModuleName;
 pub use module_resolver::{resolve_module, system_module_search_paths, KnownModule, Module};
-pub use program::{Program, ProgramSettings, SearchPathSettings, SitePackages};
+pub use program::{Program, ProgramSettings, PythonPath, SearchPathSettings};
 pub use python_platform::PythonPlatform;
 pub use semantic_model::{HasType, SemanticModel};
 
@@ -27,7 +27,6 @@ pub(crate) mod symbol;
 pub mod types;
 mod unpack;
 mod util;
-mod visibility_constraints;
 
 type FxOrderSet<V> = ordermap::set::OrderSet<V, BuildHasherDefault<FxHasher>>;
 

crates/red_knot_python_semantic/src/module_resolver/module.rs

@@ -1,4 +1,5 @@
 use std::fmt::Formatter;
+use std::str::FromStr;
 use std::sync::Arc;
 
 use ruff_db::files::File;
@@ -98,10 +99,13 @@ impl ModuleKind {
 }
 
 /// Enumeration of various core stdlib modules in which important types are located
-#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, strum_macros::EnumString)]
+#[cfg_attr(test, derive(strum_macros::EnumIter))]
+#[strum(serialize_all = "snake_case")]
 pub enum KnownModule {
     Builtins,
     Types,
+    #[strum(serialize = "_typeshed")]
     Typeshed,
     TypingExtensions,
     Typing,
@@ -109,6 +113,7 @@ pub enum KnownModule {
     #[allow(dead_code)]
     Abc, // currently only used in tests
     Collections,
+    Inspect,
     KnotExtensions,
 }
 
@@ -123,6 +128,7 @@ impl KnownModule {
             Self::Sys => "sys",
             Self::Abc => "abc",
             Self::Collections => "collections",
+            Self::Inspect => "inspect",
             Self::KnotExtensions => "knot_extensions",
         }
     }
@@ -137,20 +143,10 @@ impl KnownModule {
         search_path: &SearchPath,
         name: &ModuleName,
     ) -> Option<Self> {
-        if !search_path.is_standard_library() {
-            return None;
-        }
-        match name.as_str() {
-            "builtins" => Some(Self::Builtins),
-            "types" => Some(Self::Types),
-            "typing" => Some(Self::Typing),
-            "_typeshed" => Some(Self::Typeshed),
-            "typing_extensions" => Some(Self::TypingExtensions),
-            "sys" => Some(Self::Sys),
-            "abc" => Some(Self::Abc),
-            "collections" => Some(Self::Collections),
-            "knot_extensions" => Some(Self::KnotExtensions),
-            _ => None,
+        if search_path.is_standard_library() {
+            Self::from_str(name.as_str()).ok()
+        } else {
+            None
         }
     }
 
@@ -165,4 +161,29 @@ impl KnownModule {
     pub const fn is_knot_extensions(self) -> bool {
         matches!(self, Self::KnotExtensions)
     }
+
+    pub const fn is_inspect(self) -> bool {
+        matches!(self, Self::Inspect)
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    use strum::IntoEnumIterator;
+
+    #[test]
+    fn known_module_roundtrip_from_str() {
+        let stdlib_search_path = SearchPath::vendored_stdlib();
+
+        for module in KnownModule::iter() {
+            let module_name = module.name();
+
+            assert_eq!(
+                KnownModule::try_from_search_path_and_name(&stdlib_search_path, &module_name),
+                Some(module),
+                "The strum `EnumString` implementation appears to be incorrect for `{module_name}`"
+            );
+        }
+    }
 }

crates/red_knot_python_semantic/src/module_resolver/resolver.rs

@@ -12,7 +12,7 @@ use crate::db::Db;
 use crate::module_name::ModuleName;
 use crate::module_resolver::typeshed::{vendored_typeshed_versions, TypeshedVersions};
 use crate::site_packages::VirtualEnvironment;
-use crate::{Program, SearchPathSettings, SitePackages};
+use crate::{Program, PythonPath, SearchPathSettings};
 
 use super::module::{Module, ModuleKind};
 use super::path::{ModulePath, SearchPath, SearchPathValidationError};
@@ -171,7 +171,7 @@ impl SearchPaths {
             extra_paths,
             src_roots,
             custom_typeshed: typeshed,
-            site_packages: site_packages_paths,
+            python_path,
         } = settings;
 
         let system = db.system();
@@ -222,16 +222,16 @@ impl SearchPaths {
 
         static_paths.push(stdlib_path);
 
-        let site_packages_paths = match site_packages_paths {
-            SitePackages::Derived { venv_path } => {
+        let site_packages_paths = match python_path {
+            PythonPath::SysPrefix(sys_prefix) => {
                 // TODO: We may want to warn here if the venv's python version is older
                 //  than the one resolved in the program settings because it indicates
                 //  that the `target-version` is incorrectly configured or that the
                 //  venv is out of date.
-                VirtualEnvironment::new(venv_path, system)
+                VirtualEnvironment::new(sys_prefix, system)
                     .and_then(|venv| venv.site_packages_directories(system))?
             }
-            SitePackages::Known(paths) => paths
+            PythonPath::KnownSitePackages(paths) => paths
                 .iter()
                 .map(|path| canonicalize(path, system))
                 .collect(),
@@ -1310,7 +1310,7 @@ mod tests {
                     extra_paths: vec![],
                     src_roots: vec![src.clone()],
                     custom_typeshed: Some(custom_typeshed),
-                    site_packages: SitePackages::Known(vec![site_packages]),
+                    python_path: PythonPath::KnownSitePackages(vec![site_packages]),
                 },
             },
         )
@@ -1816,7 +1816,7 @@ not_a_directory
                     extra_paths: vec![],
                     src_roots: vec![SystemPathBuf::from("/src")],
                     custom_typeshed: None,
-                    site_packages: SitePackages::Known(vec![
+                    python_path: PythonPath::KnownSitePackages(vec![
                         venv_site_packages,
                         system_site_packages,
                     ]),

crates/red_knot_python_semantic/src/module_resolver/testing.rs

@@ -4,7 +4,7 @@ use ruff_python_ast::PythonVersion;
 
 use crate::db::tests::TestDb;
 use crate::program::{Program, SearchPathSettings};
-use crate::{ProgramSettings, PythonPlatform, SitePackages};
+use crate::{ProgramSettings, PythonPath, PythonPlatform};
 
 /// A test case for the module resolver.
 ///
@@ -239,7 +239,7 @@ impl TestCaseBuilder<MockedTypeshed> {
                     extra_paths: vec![],
                     src_roots: vec![src.clone()],
                     custom_typeshed: Some(typeshed.clone()),
-                    site_packages: SitePackages::Known(vec![site_packages.clone()]),
+                    python_path: PythonPath::KnownSitePackages(vec![site_packages.clone()]),
                 },
             },
         )
@@ -294,7 +294,7 @@ impl TestCaseBuilder<VendoredTypeshed> {
                 python_version,
                 python_platform,
                 search_paths: SearchPathSettings {
-                    site_packages: SitePackages::Known(vec![site_packages.clone()]),
+                    python_path: PythonPath::KnownSitePackages(vec![site_packages.clone()]),
                     ..SearchPathSettings::new(vec![src.clone()])
                 },
             },

crates/red_knot_python_semantic/src/program.rs

@@ -110,8 +110,9 @@ pub struct SearchPathSettings {
     /// bundled as a zip file in the binary
     pub custom_typeshed: Option<SystemPathBuf>,
 
-    /// The path to the user's `site-packages` directory, where third-party packages from ``PyPI`` are installed.
-    pub site_packages: SitePackages,
+    /// Path to the Python installation from which Red Knot resolves third party dependencies
+    /// and their type information.
+    pub python_path: PythonPath,
 }
 
 impl SearchPathSettings {
@@ -120,17 +121,32 @@ impl SearchPathSettings {
             src_roots,
             extra_paths: vec![],
             custom_typeshed: None,
-            site_packages: SitePackages::Known(vec![]),
+            python_path: PythonPath::KnownSitePackages(vec![]),
         }
     }
 }
 
 #[derive(Debug, Clone, Eq, PartialEq)]
 #[cfg_attr(feature = "serde", derive(serde::Serialize, serde::Deserialize))]
-pub enum SitePackages {
-    Derived {
-        venv_path: SystemPathBuf,
-    },
-    /// Resolved site packages paths
-    Known(Vec<SystemPathBuf>),
+pub enum PythonPath {
+    /// A path that represents the value of [`sys.prefix`] at runtime in Python
+    /// for a given Python executable.
+    ///
+    /// For the case of a virtual environment, where a
+    /// Python binary is at `/.venv/bin/python`, `sys.prefix` is the path to
+    /// the virtual environment the Python binary lies inside, i.e. `/.venv`,
+    /// and `site-packages` will be at `.venv/lib/python3.X/site-packages`.
+    /// System Python installations generally work the same way: if a system
+    /// Python installation lies at `/opt/homebrew/bin/python`, `sys.prefix`
+    /// will be `/opt/homebrew`, and `site-packages` will be at
+    /// `/opt/homebrew/lib/python3.X/site-packages`.
+    ///
+    /// [`sys.prefix`]: https://docs.python.org/3/library/sys.html#sys.prefix
+    SysPrefix(SystemPathBuf),
+
+    /// Resolved site packages paths.
+    ///
+    /// This variant is mainly intended for testing where we want to skip resolving `site-packages`
+    /// because it would unnecessarily complicate the test setup.
+    KnownSitePackages(Vec<SystemPathBuf>),
 }

crates/red_knot_python_semantic/src/semantic_index.rs

@@ -25,11 +25,13 @@ use crate::Db;
 pub mod ast_ids;
 pub mod attribute_assignment;
 mod builder;
-pub(crate) mod constraint;
 pub mod definition;
 pub mod expression;
+mod narrowing_constraints;
+pub(crate) mod predicate;
 pub mod symbol;
 mod use_def;
+mod visibility_constraints;
 
 pub(crate) use self::use_def::{
     BindingWithConstraints, BindingWithConstraintsIterator, DeclarationWithConstraint,

crates/red_knot_python_semantic/src/semantic_index/builder.rs

@@ -15,48 +15,48 @@ use crate::module_name::ModuleName;
 use crate::semantic_index::ast_ids::node_key::ExpressionNodeKey;
 use crate::semantic_index::ast_ids::AstIdsBuilder;
 use crate::semantic_index::attribute_assignment::{AttributeAssignment, AttributeAssignments};
-use crate::semantic_index::constraint::PatternConstraintKind;
 use crate::semantic_index::definition::{
-    AssignmentDefinitionNodeRef, ComprehensionDefinitionNodeRef, Definition, DefinitionNodeKey,
-    DefinitionNodeRef, ForStmtDefinitionNodeRef, ImportFromDefinitionNodeRef,
+    AssignmentDefinitionNodeRef, ComprehensionDefinitionNodeRef, Definition, DefinitionCategory,
+    DefinitionNodeKey, DefinitionNodeRef, ExceptHandlerDefinitionNodeRef, ForStmtDefinitionNodeRef,
+    ImportDefinitionNodeRef, ImportFromDefinitionNodeRef, MatchPatternDefinitionNodeRef,
+    WithItemDefinitionNodeRef,
 };
 use crate::semantic_index::expression::{Expression, ExpressionKind};
+use crate::semantic_index::predicate::{
+    PatternPredicate, PatternPredicateKind, Predicate, PredicateNode, ScopedPredicateId,
+};
 use crate::semantic_index::symbol::{
     FileScopeId, NodeWithScopeKey, NodeWithScopeRef, Scope, ScopeId, ScopeKind, ScopedSymbolId,
     SymbolTableBuilder,
 };
 use crate::semantic_index::use_def::{
-    EagerBindingsKey, FlowSnapshot, ScopedConstraintId, ScopedEagerBindingsId, UseDefMapBuilder,
+    EagerBindingsKey, FlowSnapshot, ScopedEagerBindingsId, UseDefMapBuilder,
+};
+use crate::semantic_index::visibility_constraints::{
+    ScopedVisibilityConstraintId, VisibilityConstraintsBuilder,
 };
 use crate::semantic_index::SemanticIndex;
 use crate::unpack::{Unpack, UnpackValue};
-use crate::visibility_constraints::{ScopedVisibilityConstraintId, VisibilityConstraintsBuilder};
 use crate::Db;
 
-use super::constraint::{Constraint, ConstraintNode, PatternConstraint};
-use super::definition::{
-    DefinitionCategory, ExceptHandlerDefinitionNodeRef, ImportDefinitionNodeRef,
-    MatchPatternDefinitionNodeRef, WithItemDefinitionNodeRef,
-};
-
 mod except_handlers;
 
-/// Are we in a state where a `break` statement is allowed?
-#[derive(Clone, Copy, Debug)]
-enum LoopState {
-    InLoop,
-    NotInLoop,
+#[derive(Clone, Debug, Default)]
+struct Loop {
+    /// Flow states at each `break` in the current loop.
+    break_states: Vec<FlowSnapshot>,
 }
 
-impl LoopState {
-    fn is_inside(self) -> bool {
-        matches!(self, LoopState::InLoop)
+impl Loop {
+    fn push_break(&mut self, state: FlowSnapshot) {
+        self.break_states.push(state);
     }
 }
 
 struct ScopeInfo {
     file_scope_id: FileScopeId,
-    loop_state: LoopState,
+    /// Current loop state; None if we are not currently visiting a loop
+    current_loop: Option<Loop>,
 }
 
 pub(super) struct SemanticIndexBuilder<'db> {
@@ -73,8 +73,6 @@ pub(super) struct SemanticIndexBuilder<'db> {
     /// The name of the first function parameter of the innermost function that we're currently visiting.
     current_first_parameter_name: Option<&'db str>,
 
-    /// Flow states at each `break` in the current loop.
-    loop_break_states: Vec<FlowSnapshot>,
     /// Per-scope contexts regarding nested `try`/`except` statements
     try_node_context_stack_manager: TryNodeContextStackManager,
 
@@ -106,7 +104,6 @@ impl<'db> SemanticIndexBuilder<'db> {
             current_assignments: vec![],
             current_match_case: None,
             current_first_parameter_name: None,
-            loop_break_states: vec![],
             try_node_context_stack_manager: TryNodeContextStackManager::default(),
 
             has_future_annotations: false,
@@ -134,19 +131,20 @@ impl<'db> SemanticIndexBuilder<'db> {
         builder
     }
 
-    fn current_scope(&self) -> FileScopeId {
-        *self
-            .scope_stack
-            .last()
-            .map(|ScopeInfo { file_scope_id, .. }| file_scope_id)
-            .expect("SemanticIndexBuilder should have created a root scope")
-    }
-
-    fn loop_state(&self) -> LoopState {
+    fn current_scope_info(&self) -> &ScopeInfo {
         self.scope_stack
             .last()
             .expect("SemanticIndexBuilder should have created a root scope")
-            .loop_state
+    }
+
+    fn current_scope_info_mut(&mut self) -> &mut ScopeInfo {
+        self.scope_stack
+            .last_mut()
+            .expect("SemanticIndexBuilder should have created a root scope")
+    }
+
+    fn current_scope(&self) -> FileScopeId {
+        self.current_scope_info().file_scope_id
     }
 
     /// Returns the scope ID of the surrounding class body scope if the current scope
@@ -167,11 +165,21 @@ impl<'db> SemanticIndexBuilder<'db> {
         }
     }
 
-    fn set_inside_loop(&mut self, state: LoopState) {
-        self.scope_stack
-            .last_mut()
-            .expect("Always to have a root scope")
-            .loop_state = state;
+    /// Push a new loop, returning the outer loop, if any.
+    fn push_loop(&mut self) -> Option<Loop> {
+        self.current_scope_info_mut()
+            .current_loop
+            .replace(Loop::default())
+    }
+
+    /// Pop a loop, replacing with the previous saved outer loop, if any.
+    fn pop_loop(&mut self, outer_loop: Option<Loop>) -> Loop {
+        std::mem::replace(&mut self.current_scope_info_mut().current_loop, outer_loop)
+            .expect("pop_loop() should not be called without a prior push_loop()")
+    }
+
+    fn current_loop_mut(&mut self) -> Option<&mut Loop> {
+        self.current_scope_info_mut().current_loop.as_mut()
     }
 
     fn push_scope(&mut self, node: NodeWithScopeRef) {
@@ -204,7 +212,7 @@ impl<'db> SemanticIndexBuilder<'db> {
 
         self.scope_stack.push(ScopeInfo {
             file_scope_id,
-            loop_state: LoopState::NotInLoop,
+            current_loop: None,
         });
     }
 
@@ -294,7 +302,7 @@ impl<'db> SemanticIndexBuilder<'db> {
         &self.use_def_maps[scope_id]
     }
 
-    fn current_visibility_constraints_mut(&mut self) -> &mut VisibilityConstraintsBuilder<'db> {
+    fn current_visibility_constraints_mut(&mut self) -> &mut VisibilityConstraintsBuilder {
         let scope_id = self.current_scope();
         &mut self.use_def_maps[scope_id].visibility_constraints
     }
@@ -346,12 +354,14 @@ impl<'db> SemanticIndexBuilder<'db> {
         // SAFETY: `definition_node` is guaranteed to be a child of `self.module`
         let kind = unsafe { definition_node.into_owned(self.module.clone()) };
         let category = kind.category();
+        let is_reexported = kind.is_reexported();
         let definition = Definition::new(
             self.db,
             self.file,
             self.current_scope(),
             symbol,
             kind,
+            is_reexported,
             countme::Count::default(),
         );
 
@@ -383,54 +393,60 @@ impl<'db> SemanticIndexBuilder<'db> {
         definition
     }
 
-    fn record_expression_constraint(&mut self, constraint_node: &ast::Expr) -> Constraint<'db> {
-        let constraint = self.build_constraint(constraint_node);
-        self.record_constraint(constraint);
-        constraint
+    fn record_expression_narrowing_constraint(
+        &mut self,
+        precide_node: &ast::Expr,
+    ) -> Predicate<'db> {
+        let predicate = self.build_predicate(precide_node);
+        self.record_narrowing_constraint(predicate);
+        predicate
     }
 
-    fn build_constraint(&mut self, constraint_node: &ast::Expr) -> Constraint<'db> {
-        let expression = self.add_standalone_expression(constraint_node);
-        Constraint {
-            node: ConstraintNode::Expression(expression),
+    fn build_predicate(&mut self, predicate_node: &ast::Expr) -> Predicate<'db> {
+        let expression = self.add_standalone_expression(predicate_node);
+        Predicate {
+            node: PredicateNode::Expression(expression),
             is_positive: true,
         }
     }
 
-    /// Adds a new constraint to the list of all constraints, but does not record it. Returns the
-    /// constraint ID for later recording using [`SemanticIndexBuilder::record_constraint_id`].
-    fn add_constraint(&mut self, constraint: Constraint<'db>) -> ScopedConstraintId {
-        self.current_use_def_map_mut().add_constraint(constraint)
+    /// Adds a new predicate to the list of all predicates, but does not record it. Returns the
+    /// predicate ID for later recording using
+    /// [`SemanticIndexBuilder::record_narrowing_constraint_id`].
+    fn add_predicate(&mut self, predicate: Predicate<'db>) -> ScopedPredicateId {
+        self.current_use_def_map_mut().add_predicate(predicate)
     }
 
-    /// Negates a constraint and adds it to the list of all constraints, does not record it.
-    fn add_negated_constraint(
-        &mut self,
-        constraint: Constraint<'db>,
-    ) -> (Constraint<'db>, ScopedConstraintId) {
-        let negated = Constraint {
-            node: constraint.node,
+    /// Negates a predicate and adds it to the list of all predicates, does not record it.
+    fn add_negated_predicate(&mut self, predicate: Predicate<'db>) -> ScopedPredicateId {
+        let negated = Predicate {
+            node: predicate.node,
             is_positive: false,
         };
-        let id = self.current_use_def_map_mut().add_constraint(negated);
-        (negated, id)
+        self.current_use_def_map_mut().add_predicate(negated)
     }
 
-    /// Records a previously added constraint by adding it to all live bindings.
-    fn record_constraint_id(&mut self, constraint: ScopedConstraintId) {
+    /// Records a previously added narrowing constraint by adding it to all live bindings.
+    fn record_narrowing_constraint_id(&mut self, predicate: ScopedPredicateId) {
         self.current_use_def_map_mut()
-            .record_constraint_id(constraint);
+            .record_narrowing_constraint(predicate);
     }
 
-    /// Adds and records a constraint, i.e. adds it to all live bindings.
-    fn record_constraint(&mut self, constraint: Constraint<'db>) {
-        self.current_use_def_map_mut().record_constraint(constraint);
+    /// Adds and records a narrowing constraint, i.e. adds it to all live bindings.
+    fn record_narrowing_constraint(&mut self, predicate: Predicate<'db>) {
+        let use_def = self.current_use_def_map_mut();
+        let predicate_id = use_def.add_predicate(predicate);
+        use_def.record_narrowing_constraint(predicate_id);
     }
 
-    /// Negates the given constraint and then adds it to all live bindings.
-    fn record_negated_constraint(&mut self, constraint: Constraint<'db>) -> ScopedConstraintId {
-        let (_, id) = self.add_negated_constraint(constraint);
-        self.record_constraint_id(id);
+    /// Negates the given predicate and then adds it as a narrowing constraint to all live
+    /// bindings.
+    fn record_negated_narrowing_constraint(
+        &mut self,
+        predicate: Predicate<'db>,
+    ) -> ScopedPredicateId {
+        let id = self.add_negated_predicate(predicate);
+        self.record_narrowing_constraint_id(id);
         id
     }
 
@@ -456,11 +472,12 @@ impl<'db> SemanticIndexBuilder<'db> {
     /// Records a visibility constraint by applying it to all live bindings and declarations.
     fn record_visibility_constraint(
         &mut self,
-        constraint: Constraint<'db>,
+        predicate: Predicate<'db>,
     ) -> ScopedVisibilityConstraintId {
+        let predicate_id = self.current_use_def_map_mut().add_predicate(predicate);
         let id = self
             .current_visibility_constraints_mut()
-            .add_atom(constraint, 0);
+            .add_atom(predicate_id);
         self.record_visibility_constraint_id(id);
         id
     }
@@ -527,12 +544,12 @@ impl<'db> SemanticIndexBuilder<'db> {
         }
     }
 
-    fn add_pattern_constraint(
+    fn add_pattern_narrowing_constraint(
         &mut self,
         subject: Expression<'db>,
         pattern: &ast::Pattern,
         guard: Option<&ast::Expr>,
-    ) -> Constraint<'db> {
+    ) -> Predicate<'db> {
         // This is called for the top-level pattern of each match arm. We need to create a
         // standalone expression for each arm of a match statement, since they can introduce
         // constraints on the match subject. (Or more accurately, for the match arm's pattern,
@@ -549,19 +566,19 @@ impl<'db> SemanticIndexBuilder<'db> {
         let kind = match pattern {
             ast::Pattern::MatchValue(pattern) => {
                 let value = self.add_standalone_expression(&pattern.value);
-                PatternConstraintKind::Value(value, guard)
+                PatternPredicateKind::Value(value, guard)
             }
             ast::Pattern::MatchSingleton(singleton) => {
-                PatternConstraintKind::Singleton(singleton.value, guard)
+                PatternPredicateKind::Singleton(singleton.value, guard)
             }
             ast::Pattern::MatchClass(pattern) => {
                 let cls = self.add_standalone_expression(&pattern.cls);
-                PatternConstraintKind::Class(cls, guard)
+                PatternPredicateKind::Class(cls, guard)
             }
-            _ => PatternConstraintKind::Unsupported,
+            _ => PatternPredicateKind::Unsupported,
         };
 
-        let pattern_constraint = PatternConstraint::new(
+        let pattern_predicate = PatternPredicate::new(
             self.db,
             self.file,
             self.current_scope(),
@@ -569,12 +586,12 @@ impl<'db> SemanticIndexBuilder<'db> {
             kind,
             countme::Count::default(),
         );
-        let constraint = Constraint {
-            node: ConstraintNode::Pattern(pattern_constraint),
+        let predicate = Predicate {
+            node: PredicateNode::Pattern(pattern_predicate),
             is_positive: true,
         };
-        self.current_use_def_map_mut().record_constraint(constraint);
-        constraint
+        self.record_narrowing_constraint(predicate);
+        predicate
     }
 
     /// Record an expression that needs to be a Salsa ingredient, because we need to infer its type
@@ -1117,10 +1134,10 @@ where
             ast::Stmt::If(node) => {
                 self.visit_expr(&node.test);
                 let mut no_branch_taken = self.flow_snapshot();
-                let mut last_constraint = self.record_expression_constraint(&node.test);
+                let mut last_predicate = self.record_expression_narrowing_constraint(&node.test);
                 self.visit_body(&node.body);
 
-                let visibility_constraint_id = self.record_visibility_constraint(last_constraint);
+                let visibility_constraint_id = self.record_visibility_constraint(last_predicate);
                 let mut vis_constraints = vec![visibility_constraint_id];
 
                 let mut post_clauses: Vec<FlowSnapshot> = vec![];
@@ -1146,14 +1163,14 @@ where
                     // we can only take an elif/else branch if none of the previous ones were
                     // taken
                     self.flow_restore(no_branch_taken.clone());
-                    self.record_negated_constraint(last_constraint);
+                    self.record_negated_narrowing_constraint(last_predicate);
 
-                    let elif_constraint = if let Some(elif_test) = clause_test {
+                    let elif_predicate = if let Some(elif_test) = clause_test {
                         self.visit_expr(elif_test);
                         // A test expression is evaluated whether the branch is taken or not
                         no_branch_taken = self.flow_snapshot();
-                        let constraint = self.record_expression_constraint(elif_test);
-                        Some(constraint)
+                        let predicate = self.record_expression_narrowing_constraint(elif_test);
+                        Some(predicate)
                     } else {
                         None
                     };
@@ -1163,9 +1180,9 @@ where
                     for id in &vis_constraints {
                         self.record_negated_visibility_constraint(*id);
                     }
-                    if let Some(elif_constraint) = elif_constraint {
-                        last_constraint = elif_constraint;
-                        let id = self.record_visibility_constraint(elif_constraint);
+                    if let Some(elif_predicate) = elif_predicate {
+                        last_predicate = elif_predicate;
+                        let id = self.record_visibility_constraint(elif_predicate);
                         vis_constraints.push(id);
                     }
                 }
@@ -1185,27 +1202,23 @@ where
                 self.visit_expr(test);
 
                 let pre_loop = self.flow_snapshot();
-                let constraint = self.record_expression_constraint(test);
+                let predicate = self.record_expression_narrowing_constraint(test);
 
                 // We need multiple copies of the visibility constraint for the while condition,
                 // since we need to model situations where the first evaluation of the condition
                 // returns True, but a later evaluation returns False.
+                let first_predicate_id = self.current_use_def_map_mut().add_predicate(predicate);
+                let later_predicate_id = self.current_use_def_map_mut().add_predicate(predicate);
                 let first_vis_constraint_id = self
                     .current_visibility_constraints_mut()
-                    .add_atom(constraint, 0);
+                    .add_atom(first_predicate_id);
                 let later_vis_constraint_id = self
                     .current_visibility_constraints_mut()
-                    .add_atom(constraint, 1);
+                    .add_atom(later_predicate_id);
 
-                // Save aside any break states from an outer loop
-                let saved_break_states = std::mem::take(&mut self.loop_break_states);
-
-                // TODO: definitions created inside the body should be fully visible
-                // to other statements/expressions inside the body --Alex/Carl
-                let outer_loop_state = self.loop_state();
-                self.set_inside_loop(LoopState::InLoop);
+                let outer_loop = self.push_loop();
                 self.visit_body(body);
-                self.set_inside_loop(outer_loop_state);
+                let this_loop = self.pop_loop(outer_loop);
 
                 // If the body is executed, we know that we've evaluated the condition at least
                 // once, and that the first evaluation was True. We might not have evaluated the
@@ -1214,11 +1227,6 @@ where
                 let body_vis_constraint_id = first_vis_constraint_id;
                 self.record_visibility_constraint_id(body_vis_constraint_id);
 
-                // Get the break states from the body of this loop, and restore the saved outer
-                // ones.
-                let break_states =
-                    std::mem::replace(&mut self.loop_break_states, saved_break_states);
-
                 // We execute the `else` once the condition evaluates to false. This could happen
                 // without ever executing the body, if the condition is false the first time it's
                 // tested. So the starting flow state of the `else` clause is the union of:
@@ -1233,13 +1241,13 @@ where
                 self.flow_restore(pre_loop.clone());
                 self.record_negated_visibility_constraint(first_vis_constraint_id);
                 self.flow_merge(post_body);
-                self.record_negated_constraint(constraint);
+                self.record_negated_narrowing_constraint(predicate);
                 self.visit_body(orelse);
                 self.record_negated_visibility_constraint(later_vis_constraint_id);
 
                 // Breaking out of a while loop bypasses the `else` clause, so merge in the break
                 // states after visiting `else`.
-                for break_state in break_states {
+                for break_state in this_loop.break_states {
                     let snapshot = self.flow_snapshot();
                     self.flow_restore(break_state);
                     self.record_visibility_constraint_id(body_vis_constraint_id);
@@ -1287,7 +1295,6 @@ where
                 self.record_ambiguous_visibility();
 
                 let pre_loop = self.flow_snapshot();
-                let saved_break_states = std::mem::take(&mut self.loop_break_states);
 
                 let current_assignment = match &**target {
                     ast::Expr::List(_) | ast::Expr::Tuple(_) => Some(CurrentAssignment::For {
@@ -1335,16 +1342,9 @@ where
                     self.pop_assignment();
                 }
 
-                // TODO: Definitions created by loop variables
-                // (and definitions created inside the body)
-                // are fully visible to other statements/expressions inside the body --Alex/Carl
-                let outer_loop_state = self.loop_state();
-                self.set_inside_loop(LoopState::InLoop);
+                let outer_loop = self.push_loop();
                 self.visit_body(body);
-                self.set_inside_loop(outer_loop_state);
-
-                let break_states =
-                    std::mem::replace(&mut self.loop_break_states, saved_break_states);
+                let this_loop = self.pop_loop(outer_loop);
 
                 // We may execute the `else` clause without ever executing the body, so merge in
                 // the pre-loop state before visiting `else`.
@@ -1353,7 +1353,7 @@ where
 
                 // Breaking out of a `for` loop bypasses the `else` clause, so merge in the break
                 // states after visiting `else`.
-                for break_state in break_states {
+                for break_state in this_loop.break_states {
                     self.flow_merge(break_state);
                 }
             }
@@ -1382,7 +1382,7 @@ where
                     self.current_match_case = Some(CurrentMatchCase::new(&case.pattern));
                     self.visit_pattern(&case.pattern);
                     self.current_match_case = None;
-                    let constraint_id = self.add_pattern_constraint(
+                    let predicate = self.add_pattern_narrowing_constraint(
                         subject_expr,
                         &case.pattern,
                         case.guard.as_deref(),
@@ -1394,7 +1394,7 @@ where
                     for id in &vis_constraints {
                         self.record_negated_visibility_constraint(*id);
                     }
-                    let vis_constraint_id = self.record_visibility_constraint(constraint_id);
+                    let vis_constraint_id = self.record_visibility_constraint(predicate);
                     vis_constraints.push(vis_constraint_id);
                 }
 
@@ -1536,8 +1536,9 @@ where
             }
 
             ast::Stmt::Break(_) => {
-                if self.loop_state().is_inside() {
-                    self.loop_break_states.push(self.flow_snapshot());
+                let snapshot = self.flow_snapshot();
+                if let Some(current_loop) = self.current_loop_mut() {
+                    current_loop.push_break(snapshot);
                 }
                 // Everything in the current block after a terminal statement is unreachable.
                 self.mark_unreachable();
@@ -1693,13 +1694,13 @@ where
             }) => {
                 self.visit_expr(test);
                 let pre_if = self.flow_snapshot();
-                let constraint = self.record_expression_constraint(test);
+                let predicate = self.record_expression_narrowing_constraint(test);
                 self.visit_expr(body);
-                let visibility_constraint = self.record_visibility_constraint(constraint);
+                let visibility_constraint = self.record_visibility_constraint(predicate);
                 let post_body = self.flow_snapshot();
                 self.flow_restore(pre_if.clone());
 
-                self.record_negated_constraint(constraint);
+                self.record_negated_narrowing_constraint(predicate);
                 self.visit_expr(orelse);
                 self.record_negated_visibility_constraint(visibility_constraint);
                 self.flow_merge(post_body);
@@ -1775,14 +1776,14 @@ where
                     // For the last value, we don't need to model control flow. There is short-circuiting
                     // anymore.
                     if index < values.len() - 1 {
-                        let constraint = self.build_constraint(value);
-                        let (constraint, constraint_id) = match op {
-                            ast::BoolOp::And => (constraint, self.add_constraint(constraint)),
-                            ast::BoolOp::Or => self.add_negated_constraint(constraint),
+                        let predicate = self.build_predicate(value);
+                        let predicate_id = match op {
+                            ast::BoolOp::And => self.add_predicate(predicate),
+                            ast::BoolOp::Or => self.add_negated_predicate(predicate),
                         };
                         let visibility_constraint = self
                             .current_visibility_constraints_mut()
-                            .add_atom(constraint, 0);
+                            .add_atom(predicate_id);
 
                         let after_expr = self.flow_snapshot();
 
@@ -1800,7 +1801,7 @@ where
                         // the application of the visibility constraint until after the expression
                         // has been evaluated, so we only push it onto the stack here.
                         self.flow_restore(after_expr);
-                        self.record_constraint_id(constraint_id);
+                        self.record_narrowing_constraint_id(predicate_id);
                         visibility_constraints.push(visibility_constraint);
                     }
                 }

crates/red_knot_python_semantic/src/semantic_index/constraint.rs

@@ -1,47 +0,0 @@
-use ruff_db::files::File;
-use ruff_python_ast::Singleton;
-
-use crate::db::Db;
-use crate::semantic_index::expression::Expression;
-use crate::semantic_index::symbol::{FileScopeId, ScopeId};
-
-#[derive(Clone, Copy, Debug, Hash, PartialEq, Eq, salsa::Update)]
-pub(crate) struct Constraint<'db> {
-    pub(crate) node: ConstraintNode<'db>,
-    pub(crate) is_positive: bool,
-}
-
-#[derive(Clone, Copy, Debug, Hash, PartialEq, Eq, salsa::Update)]
-pub(crate) enum ConstraintNode<'db> {
-    Expression(Expression<'db>),
-    Pattern(PatternConstraint<'db>),
-}
-
-/// Pattern kinds for which we support type narrowing and/or static visibility analysis.
-#[derive(Debug, Clone, Hash, PartialEq, salsa::Update)]
-pub(crate) enum PatternConstraintKind<'db> {
-    Singleton(Singleton, Option<Expression<'db>>),
-    Value(Expression<'db>, Option<Expression<'db>>),
-    Class(Expression<'db>, Option<Expression<'db>>),
-    Unsupported,
-}
-
-#[salsa::tracked]
-pub(crate) struct PatternConstraint<'db> {
-    pub(crate) file: File,
-
-    pub(crate) file_scope: FileScopeId,
-
-    pub(crate) subject: Expression<'db>,
-
-    #[return_ref]
-    pub(crate) kind: PatternConstraintKind<'db>,
-
-    count: countme::Count<PatternConstraint<'static>>,
-}
-
-impl<'db> PatternConstraint<'db> {
-    pub(crate) fn scope(self, db: &'db dyn Db) -> ScopeId<'db> {
-        self.file_scope(db).to_scope_id(db, self.file(db))
-    }
-}

crates/red_knot_python_semantic/src/semantic_index/definition.rs

@@ -33,11 +33,16 @@ pub struct Definition<'db> {
     /// The symbol defined.
     pub(crate) symbol: ScopedSymbolId,
 
+    /// WARNING: Only access this field when doing type inference for the same
+    /// file as where `Definition` is defined to avoid cross-file query dependencies.
     #[no_eq]
     #[return_ref]
     #[tracked]
     pub(crate) kind: DefinitionKind<'db>,
 
+    /// This is a dedicated field to avoid accessing `kind` to compute this value.
+    pub(crate) is_reexported: bool,
+
     count: countme::Count<Definition<'static>>,
 }
 
@@ -45,22 +50,6 @@ impl<'db> Definition<'db> {
     pub(crate) fn scope(self, db: &'db dyn Db) -> ScopeId<'db> {
         self.file_scope(db).to_scope_id(db, self.file(db))
     }
-
-    pub(crate) fn category(self, db: &'db dyn Db) -> DefinitionCategory {
-        self.kind(db).category()
-    }
-
-    pub(crate) fn is_declaration(self, db: &'db dyn Db) -> bool {
-        self.kind(db).category().is_declaration()
-    }
-
-    pub(crate) fn is_binding(self, db: &'db dyn Db) -> bool {
-        self.kind(db).category().is_binding()
-    }
-
-    pub(crate) fn is_reexported(self, db: &'db dyn Db) -> bool {
-        self.kind(db).is_reexported()
-    }
 }
 
 #[derive(Copy, Clone, Debug)]

crates/red_knot_python_semantic/src/semantic_index/narrowing_constraints.rs

@@ -0,0 +1,151 @@
+//! # Narrowing constraints
+//!
+//! When building a semantic index for a file, we associate each binding with a _narrowing
+//! constraint_, which constrains the type of the binding's symbol. Note that a binding can be
+//! associated with a different narrowing constraint at different points in a file. See the
+//! [`use_def`][crate::semantic_index::use_def] module for more details.
+//!
+//! This module defines how narrowing constraints are stored internally.
+//!
+//! A _narrowing constraint_ consists of a list of _predicates_, each of which corresponds with an
+//! expression in the source file (represented by a [`Predicate`]). We need to support the
+//! following operations on narrowing constraints:
+//!
+//! - Adding a new predicate to an existing constraint
+//! - Merging two constraints together, which produces the _intersection_ of their predicates
+//! - Iterating through the predicates in a constraint
+//!
+//! In particular, note that we do not need random access to the predicates in a constraint. That
+//! means that we can use a simple [_sorted association list_][ruff_index::list] as our data
+//! structure. That lets us use a single 32-bit integer to store each narrowing constraint, no
+//! matter how many predicates it contains. It also makes merging two narrowing constraints fast,
+//! since alists support fast intersection.
+//!
+//! Because we visit the contents of each scope in source-file order, and assign scoped IDs in
+//! source-file order, that means that we will tend to visit narrowing constraints in order by
+//! their predicate IDs. This is exactly how to get the best performance from our alist
+//! implementation.
+//!
+//! [`Predicate`]: crate::semantic_index::predicate::Predicate
+
+use ruff_index::list::{ListBuilder, ListSetReverseIterator, ListStorage};
+use ruff_index::newtype_index;
+
+use crate::semantic_index::predicate::ScopedPredicateId;
+
+/// A narrowing constraint associated with a live binding.
+///
+/// A constraint is a list of [`Predicate`]s that each constrain the type of the binding's symbol.
+///
+/// An instance of this type represents a _non-empty_ narrowing constraint. You will often wrap
+/// this in `Option` and use `None` to represent an empty narrowing constraint.
+///
+/// [`Predicate`]: crate::semantic_index::predicate::Predicate
+#[newtype_index]
+pub(crate) struct ScopedNarrowingConstraintId;
+
+/// One of the [`Predicate`]s in a narrowing constraint, which constraints the type of the
+/// binding's symbol.
+///
+/// Note that those [`Predicate`]s are stored in [their own per-scope
+/// arena][crate::semantic_index::predicate::Predicates], so internally we use a
+/// [`ScopedPredicateId`] to refer to the underlying predicate.
+///
+/// [`Predicate`]: crate::semantic_index::predicate::Predicate
+#[derive(Clone, Copy, Debug, Eq, Ord, PartialEq, PartialOrd)]
+pub(crate) struct ScopedNarrowingConstraintPredicate(ScopedPredicateId);
+
+impl ScopedNarrowingConstraintPredicate {
+    /// Returns (the ID of) the `Predicate`
+    pub(crate) fn predicate(self) -> ScopedPredicateId {
+        self.0
+    }
+}
+
+impl From<ScopedPredicateId> for ScopedNarrowingConstraintPredicate {
+    fn from(predicate: ScopedPredicateId) -> ScopedNarrowingConstraintPredicate {
+        ScopedNarrowingConstraintPredicate(predicate)
+    }
+}
+
+/// A collection of narrowing constraints for a given scope.
+#[derive(Debug, Eq, PartialEq)]
+pub(crate) struct NarrowingConstraints {
+    lists: ListStorage<ScopedNarrowingConstraintId, ScopedNarrowingConstraintPredicate>,
+}
+
+// Building constraints
+// --------------------
+
+/// A builder for creating narrowing constraints.
+#[derive(Debug, Default, Eq, PartialEq)]
+pub(crate) struct NarrowingConstraintsBuilder {
+    lists: ListBuilder<ScopedNarrowingConstraintId, ScopedNarrowingConstraintPredicate>,
+}
+
+impl NarrowingConstraintsBuilder {
+    pub(crate) fn build(self) -> NarrowingConstraints {
+        NarrowingConstraints {
+            lists: self.lists.build(),
+        }
+    }
+
+    /// Adds a predicate to an existing narrowing constraint.
+    pub(crate) fn add_predicate_to_constraint(
+        &mut self,
+        constraint: Option<ScopedNarrowingConstraintId>,
+        predicate: ScopedNarrowingConstraintPredicate,
+    ) -> Option<ScopedNarrowingConstraintId> {
+        self.lists.insert(constraint, predicate)
+    }
+
+    /// Returns the intersection of two narrowing constraints. The result contains the predicates
+    /// that appear in both inputs.
+    pub(crate) fn intersect_constraints(
+        &mut self,
+        a: Option<ScopedNarrowingConstraintId>,
+        b: Option<ScopedNarrowingConstraintId>,
+    ) -> Option<ScopedNarrowingConstraintId> {
+        self.lists.intersect(a, b)
+    }
+}
+
+// Iteration
+// ---------
+
+pub(crate) type NarrowingConstraintsIterator<'a> = std::iter::Copied<
+    ListSetReverseIterator<'a, ScopedNarrowingConstraintId, ScopedNarrowingConstraintPredicate>,
+>;
+
+impl NarrowingConstraints {
+    /// Iterates over the predicates in a narrowing constraint.
+    pub(crate) fn iter_predicates(
+        &self,
+        set: Option<ScopedNarrowingConstraintId>,
+    ) -> NarrowingConstraintsIterator<'_> {
+        self.lists.iter_set_reverse(set).copied()
+    }
+}
+
+// Test support
+// ------------
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    impl ScopedNarrowingConstraintPredicate {
+        pub(crate) fn as_u32(self) -> u32 {
+            self.0.as_u32()
+        }
+    }
+
+    impl NarrowingConstraintsBuilder {
+        pub(crate) fn iter_predicates(
+            &self,
+            set: Option<ScopedNarrowingConstraintId>,
+        ) -> NarrowingConstraintsIterator<'_> {
+            self.lists.iter_set_reverse(set).copied()
+        }
+    }
+}

crates/red_knot_python_semantic/src/semantic_index/predicate.rs

@@ -0,0 +1,84 @@
+//! _Predicates_ are Python expressions whose runtime values can affect type inference.
+//!
+//! We currently use predicates in two places:
+//!
+//! - [_Narrowing constraints_][crate::semantic_index::narrowing_constraints] constrain the type of
+//!   a binding that is visible at a particular use.
+//! - [_Visibility constraints_][crate::semantic_index::visibility_constraints] determine the
+//!   static visibility of a binding, and the reachability of a statement.
+
+use ruff_db::files::File;
+use ruff_index::{newtype_index, IndexVec};
+use ruff_python_ast::Singleton;
+
+use crate::db::Db;
+use crate::semantic_index::expression::Expression;
+use crate::semantic_index::symbol::{FileScopeId, ScopeId};
+
+// A scoped identifier for each `Predicate` in a scope.
+#[newtype_index]
+#[derive(Ord, PartialOrd)]
+pub(crate) struct ScopedPredicateId;
+
+// A collection of predicates for a given scope.
+pub(crate) type Predicates<'db> = IndexVec<ScopedPredicateId, Predicate<'db>>;
+
+#[derive(Debug, Default)]
+pub(crate) struct PredicatesBuilder<'db> {
+    predicates: IndexVec<ScopedPredicateId, Predicate<'db>>,
+}
+
+impl<'db> PredicatesBuilder<'db> {
+    /// Adds a predicate. Note that we do not deduplicate predicates. If you add a `Predicate`
+    /// more than once, you will get distinct `ScopedPredicateId`s for each one. (This lets you
+    /// model predicates that might evaluate to different values at different points of execution.)
+    pub(crate) fn add_predicate(&mut self, predicate: Predicate<'db>) -> ScopedPredicateId {
+        self.predicates.push(predicate)
+    }
+
+    pub(crate) fn build(mut self) -> Predicates<'db> {
+        self.predicates.shrink_to_fit();
+        self.predicates
+    }
+}
+
+#[derive(Clone, Copy, Debug, Hash, PartialEq, Eq, salsa::Update)]
+pub(crate) struct Predicate<'db> {
+    pub(crate) node: PredicateNode<'db>,
+    pub(crate) is_positive: bool,
+}
+
+#[derive(Clone, Copy, Debug, Hash, PartialEq, Eq, salsa::Update)]
+pub(crate) enum PredicateNode<'db> {
+    Expression(Expression<'db>),
+    Pattern(PatternPredicate<'db>),
+}
+
+/// Pattern kinds for which we support type narrowing and/or static visibility analysis.
+#[derive(Debug, Clone, Hash, PartialEq, salsa::Update)]
+pub(crate) enum PatternPredicateKind<'db> {
+    Singleton(Singleton, Option<Expression<'db>>),
+    Value(Expression<'db>, Option<Expression<'db>>),
+    Class(Expression<'db>, Option<Expression<'db>>),
+    Unsupported,
+}
+
+#[salsa::tracked]
+pub(crate) struct PatternPredicate<'db> {
+    pub(crate) file: File,
+
+    pub(crate) file_scope: FileScopeId,
+
+    pub(crate) subject: Expression<'db>,
+
+    #[return_ref]
+    pub(crate) kind: PatternPredicateKind<'db>,
+
+    count: countme::Count<PatternPredicate<'static>>,
+}
+
+impl<'db> PatternPredicate<'db> {
+    pub(crate) fn scope(self, db: &'db dyn Db) -> ScopeId<'db> {
+        self.file_scope(db).to_scope_id(db, self.file(db))
+    }
+}

crates/red_knot_python_semantic/src/semantic_index/use_def.rs

@@ -165,7 +165,7 @@
 //! don't actually store these "list of visible definitions" as a vector of [`Definition`].
 //! Instead, [`SymbolBindings`] and [`SymbolDeclarations`] are structs which use bit-sets to track
 //! definitions (and constraints, in the case of bindings) in terms of [`ScopedDefinitionId`] and
-//! [`ScopedConstraintId`], which are indices into the `all_definitions` and `all_constraints`
+//! [`ScopedPredicateId`], which are indices into the `all_definitions` and `predicates`
 //! indexvecs in the [`UseDefMap`].
 //!
 //! There is another special kind of possible "definition" for a symbol: there might be a path from
@@ -255,28 +255,29 @@
 //! snapshot, and merging a snapshot into the current state. The logic using these methods lives in
 //! [`SemanticIndexBuilder`](crate::semantic_index::builder::SemanticIndexBuilder), e.g. where it
 //! visits a `StmtIf` node.
-pub(crate) use self::symbol_state::ScopedConstraintId;
-use self::symbol_state::{
-    BindingIdWithConstraintsIterator, ConstraintIdIterator, DeclarationIdIterator,
-    ScopedDefinitionId, SymbolBindings, SymbolDeclarations, SymbolState,
-};
-use crate::semantic_index::ast_ids::ScopedUseId;
-use crate::semantic_index::definition::Definition;
-use crate::semantic_index::symbol::{FileScopeId, ScopedSymbolId};
-use crate::semantic_index::use_def::symbol_state::DeclarationIdWithConstraint;
-use crate::visibility_constraints::{
-    ScopedVisibilityConstraintId, VisibilityConstraints, VisibilityConstraintsBuilder,
-};
+
 use ruff_index::{newtype_index, IndexVec};
 use rustc_hash::FxHashMap;
 
-use super::constraint::Constraint;
+use self::symbol_state::{
+    LiveBindingsIterator, LiveDeclaration, LiveDeclarationsIterator, ScopedDefinitionId,
+    SymbolBindings, SymbolDeclarations, SymbolState,
+};
+use crate::semantic_index::ast_ids::ScopedUseId;
+use crate::semantic_index::definition::Definition;
+use crate::semantic_index::narrowing_constraints::{
+    NarrowingConstraints, NarrowingConstraintsBuilder, NarrowingConstraintsIterator,
+};
+use crate::semantic_index::predicate::{
+    Predicate, Predicates, PredicatesBuilder, ScopedPredicateId,
+};
+use crate::semantic_index::symbol::{FileScopeId, ScopedSymbolId};
+use crate::semantic_index::visibility_constraints::{
+    ScopedVisibilityConstraintId, VisibilityConstraints, VisibilityConstraintsBuilder,
+};
 
-mod bitset;
 mod symbol_state;
 
-type AllConstraints<'db> = IndexVec<ScopedConstraintId, Constraint<'db>>;
-
 /// Applicable definitions and constraints for every use of a name.
 #[derive(Debug, PartialEq, Eq, salsa::Update)]
 pub(crate) struct UseDefMap<'db> {
@@ -284,11 +285,14 @@ pub(crate) struct UseDefMap<'db> {
     /// this represents the implicit "unbound"/"undeclared" definition of every symbol.
     all_definitions: IndexVec<ScopedDefinitionId, Option<Definition<'db>>>,
 
-    /// Array of [`Constraint`] in this scope.
-    all_constraints: AllConstraints<'db>,
+    /// Array of predicates in this scope.
+    predicates: Predicates<'db>,
+
+    /// Array of narrowing constraints in this scope.
+    narrowing_constraints: NarrowingConstraints,
 
     /// Array of visibility constraints in this scope.
-    visibility_constraints: VisibilityConstraints<'db>,
+    visibility_constraints: VisibilityConstraints,
 
     /// [`SymbolBindings`] reaching a [`ScopedUseId`].
     bindings_by_use: IndexVec<ScopedUseId, SymbolBindings>,
@@ -370,7 +374,8 @@ impl<'db> UseDefMap<'db> {
     ) -> BindingWithConstraintsIterator<'map, 'db> {
         BindingWithConstraintsIterator {
             all_definitions: &self.all_definitions,
-            all_constraints: &self.all_constraints,
+            predicates: &self.predicates,
+            narrowing_constraints: &self.narrowing_constraints,
             visibility_constraints: &self.visibility_constraints,
             inner: bindings.iter(),
         }
@@ -382,6 +387,7 @@ impl<'db> UseDefMap<'db> {
     ) -> DeclarationsIterator<'map, 'db> {
         DeclarationsIterator {
             all_definitions: &self.all_definitions,
+            predicates: &self.predicates,
             visibility_constraints: &self.visibility_constraints,
             inner: declarations.iter(),
         }
@@ -415,26 +421,29 @@ type EagerBindings = IndexVec<ScopedEagerBindingsId, SymbolBindings>;
 #[derive(Debug)]
 pub(crate) struct BindingWithConstraintsIterator<'map, 'db> {
     all_definitions: &'map IndexVec<ScopedDefinitionId, Option<Definition<'db>>>,
-    all_constraints: &'map AllConstraints<'db>,
-    pub(crate) visibility_constraints: &'map VisibilityConstraints<'db>,
-    inner: BindingIdWithConstraintsIterator<'map>,
+    pub(crate) predicates: &'map Predicates<'db>,
+    pub(crate) narrowing_constraints: &'map NarrowingConstraints,
+    pub(crate) visibility_constraints: &'map VisibilityConstraints,
+    inner: LiveBindingsIterator<'map>,
 }
 
 impl<'map, 'db> Iterator for BindingWithConstraintsIterator<'map, 'db> {
     type Item = BindingWithConstraints<'map, 'db>;
 
     fn next(&mut self) -> Option<Self::Item> {
-        let all_constraints = self.all_constraints;
+        let predicates = self.predicates;
+        let narrowing_constraints = self.narrowing_constraints;
 
         self.inner
             .next()
-            .map(|binding_id_with_constraints| BindingWithConstraints {
-                binding: self.all_definitions[binding_id_with_constraints.definition],
-                constraints: ConstraintsIterator {
-                    all_constraints,
-                    constraint_ids: binding_id_with_constraints.constraint_ids,
+            .map(|live_binding| BindingWithConstraints {
+                binding: self.all_definitions[live_binding.binding],
+                narrowing_constraint: ConstraintsIterator {
+                    predicates,
+                    constraint_ids: narrowing_constraints
+                        .iter_predicates(live_binding.narrowing_constraint),
                 },
-                visibility_constraint: binding_id_with_constraints.visibility_constraint,
+                visibility_constraint: live_binding.visibility_constraint,
             })
     }
 }
@@ -443,22 +452,22 @@ impl std::iter::FusedIterator for BindingWithConstraintsIterator<'_, '_> {}
 
 pub(crate) struct BindingWithConstraints<'map, 'db> {
     pub(crate) binding: Option<Definition<'db>>,
-    pub(crate) constraints: ConstraintsIterator<'map, 'db>,
+    pub(crate) narrowing_constraint: ConstraintsIterator<'map, 'db>,
     pub(crate) visibility_constraint: ScopedVisibilityConstraintId,
 }
 
 pub(crate) struct ConstraintsIterator<'map, 'db> {
-    all_constraints: &'map AllConstraints<'db>,
-    constraint_ids: ConstraintIdIterator<'map>,
+    predicates: &'map Predicates<'db>,
+    constraint_ids: NarrowingConstraintsIterator<'map>,
 }
 
 impl<'db> Iterator for ConstraintsIterator<'_, 'db> {
-    type Item = Constraint<'db>;
+    type Item = Predicate<'db>;
 
     fn next(&mut self) -> Option<Self::Item> {
         self.constraint_ids
             .next()
-            .map(|constraint_id| self.all_constraints[constraint_id])
+            .map(|narrowing_constraint| self.predicates[narrowing_constraint.predicate()])
     }
 }
 
@@ -466,8 +475,9 @@ impl std::iter::FusedIterator for ConstraintsIterator<'_, '_> {}
 
 pub(crate) struct DeclarationsIterator<'map, 'db> {
     all_definitions: &'map IndexVec<ScopedDefinitionId, Option<Definition<'db>>>,
-    pub(crate) visibility_constraints: &'map VisibilityConstraints<'db>,
-    inner: DeclarationIdIterator<'map>,
+    pub(crate) predicates: &'map Predicates<'db>,
+    pub(crate) visibility_constraints: &'map VisibilityConstraints,
+    inner: LiveDeclarationsIterator<'map>,
 }
 
 pub(crate) struct DeclarationWithConstraint<'db> {
@@ -480,13 +490,13 @@ impl<'db> Iterator for DeclarationsIterator<'_, 'db> {
 
     fn next(&mut self) -> Option<Self::Item> {
         self.inner.next().map(
-            |DeclarationIdWithConstraint {
-                 definition,
+            |LiveDeclaration {
+                 declaration,
                  visibility_constraint,
              }| {
                 DeclarationWithConstraint {
-                    declaration: self.all_definitions[definition],
-                    visibility_constraint,
+                    declaration: self.all_definitions[*declaration],
+                    visibility_constraint: *visibility_constraint,
                 }
             },
         )
@@ -507,11 +517,14 @@ pub(super) struct UseDefMapBuilder<'db> {
     /// Append-only array of [`Definition`].
     all_definitions: IndexVec<ScopedDefinitionId, Option<Definition<'db>>>,
 
-    /// Append-only array of [`Constraint`].
-    all_constraints: AllConstraints<'db>,
+    /// Builder of predicates.
+    pub(super) predicates: PredicatesBuilder<'db>,
+
+    /// Builder of narrowing constraints.
+    pub(super) narrowing_constraints: NarrowingConstraintsBuilder,
 
     /// Builder of visibility constraints.
-    pub(super) visibility_constraints: VisibilityConstraintsBuilder<'db>,
+    pub(super) visibility_constraints: VisibilityConstraintsBuilder,
 
     /// A constraint which describes the visibility of the unbound/undeclared state, i.e.
     /// whether or not the start of the scope is visible. This is important for cases like
@@ -540,7 +553,8 @@ impl Default for UseDefMapBuilder<'_> {
     fn default() -> Self {
         Self {
             all_definitions: IndexVec::from_iter([None]),
-            all_constraints: IndexVec::new(),
+            predicates: PredicatesBuilder::default(),
+            narrowing_constraints: NarrowingConstraintsBuilder::default(),
             visibility_constraints: VisibilityConstraintsBuilder::default(),
             scope_start_visibility: ScopedVisibilityConstraintId::ALWAYS_TRUE,
             bindings_by_use: IndexVec::new(),
@@ -572,22 +586,18 @@ impl<'db> UseDefMapBuilder<'db> {
         symbol_state.record_binding(def_id, self.scope_start_visibility);
     }
 
-    pub(super) fn add_constraint(&mut self, constraint: Constraint<'db>) -> ScopedConstraintId {
-        self.all_constraints.push(constraint)
+    pub(super) fn add_predicate(&mut self, predicate: Predicate<'db>) -> ScopedPredicateId {
+        self.predicates.add_predicate(predicate)
     }
 
-    pub(super) fn record_constraint_id(&mut self, constraint: ScopedConstraintId) {
+    pub(super) fn record_narrowing_constraint(&mut self, predicate: ScopedPredicateId) {
+        let narrowing_constraint = predicate.into();
         for state in &mut self.symbol_states {
-            state.record_constraint(constraint);
+            state
+                .record_narrowing_constraint(&mut self.narrowing_constraints, narrowing_constraint);
         }
     }
 
-    pub(super) fn record_constraint(&mut self, constraint: Constraint<'db>) -> ScopedConstraintId {
-        let new_constraint_id = self.add_constraint(constraint);
-        self.record_constraint_id(new_constraint_id);
-        new_constraint_id
-    }
-
     pub(super) fn record_visibility_constraint(
         &mut self,
         constraint: ScopedVisibilityConstraintId,
@@ -736,10 +746,15 @@ impl<'db> UseDefMapBuilder<'db> {
         let mut snapshot_definitions_iter = snapshot.symbol_states.into_iter();
         for current in &mut self.symbol_states {
             if let Some(snapshot) = snapshot_definitions_iter.next() {
-                current.merge(snapshot, &mut self.visibility_constraints);
+                current.merge(
+                    snapshot,
+                    &mut self.narrowing_constraints,
+                    &mut self.visibility_constraints,
+                );
             } else {
                 current.merge(
                     SymbolState::undefined(snapshot.scope_start_visibility),
+                    &mut self.narrowing_constraints,
                     &mut self.visibility_constraints,
                 );
                 // Symbol not present in snapshot, so it's unbound/undeclared from that path.
@@ -753,7 +768,6 @@ impl<'db> UseDefMapBuilder<'db> {
 
     pub(super) fn finish(mut self) -> UseDefMap<'db> {
         self.all_definitions.shrink_to_fit();
-        self.all_constraints.shrink_to_fit();
         self.symbol_states.shrink_to_fit();
         self.bindings_by_use.shrink_to_fit();
         self.declarations_by_binding.shrink_to_fit();
@@ -762,7 +776,8 @@ impl<'db> UseDefMapBuilder<'db> {
 
         UseDefMap {
             all_definitions: self.all_definitions,
-            all_constraints: self.all_constraints,
+            predicates: self.predicates.build(),
+            narrowing_constraints: self.narrowing_constraints.build(),
             visibility_constraints: self.visibility_constraints.build(),
             bindings_by_use: self.bindings_by_use,
             public_symbols: self.symbol_states,

crates/red_knot_python_semantic/src/semantic_index/use_def/bitset.rs

@@ -1,298 +0,0 @@
-/// Ordered set of `u32`.
-///
-/// Uses an inline bit-set for small values (up to 64 * B), falls back to heap allocated vector of
-/// blocks for larger values.
-#[derive(Debug, Clone, PartialEq, Eq)]
-pub(super) enum BitSet<const B: usize> {
-    /// Bit-set (in 64-bit blocks) for the first 64 * B entries.
-    Inline([u64; B]),
-
-    /// Overflow beyond 64 * B.
-    Heap(Vec<u64>),
-}
-
-impl<const B: usize> Default for BitSet<B> {
-    fn default() -> Self {
-        // B * 64 must fit in a u32, or else we have unusable bits; this assertion makes the
-        // truncating casts to u32 below safe. This would be better as a const assertion, but
-        // that's not possible on stable with const generic params. (B should never really be
-        // anywhere close to this large.)
-        assert!(B * 64 < (u32::MAX as usize));
-        // This implementation requires usize >= 32 bits.
-        static_assertions::const_assert!(usize::BITS >= 32);
-        Self::Inline([0; B])
-    }
-}
-
-impl<const B: usize> BitSet<B> {
-    /// Create and return a new [`BitSet`] with a single `value` inserted.
-    pub(super) fn with(value: u32) -> Self {
-        let mut bitset = Self::default();
-        bitset.insert(value);
-        bitset
-    }
-
-    /// Convert from Inline to Heap, if needed, and resize the Heap vector, if needed.
-    fn resize(&mut self, value: u32) {
-        let num_blocks_needed = (value / 64) + 1;
-        self.resize_blocks(num_blocks_needed as usize);
-    }
-
-    fn resize_blocks(&mut self, num_blocks_needed: usize) {
-        match self {
-            Self::Inline(blocks) => {
-                let mut vec = blocks.to_vec();
-                vec.resize(num_blocks_needed, 0);
-                *self = Self::Heap(vec);
-            }
-            Self::Heap(vec) => {
-                vec.resize(num_blocks_needed, 0);
-            }
-        }
-    }
-
-    fn blocks_mut(&mut self) -> &mut [u64] {
-        match self {
-            Self::Inline(blocks) => blocks.as_mut_slice(),
-            Self::Heap(blocks) => blocks.as_mut_slice(),
-        }
-    }
-
-    fn blocks(&self) -> &[u64] {
-        match self {
-            Self::Inline(blocks) => blocks.as_slice(),
-            Self::Heap(blocks) => blocks.as_slice(),
-        }
-    }
-
-    /// Insert a value into the [`BitSet`].
-    ///
-    /// Return true if the value was newly inserted, false if already present.
-    pub(super) fn insert(&mut self, value: u32) -> bool {
-        let value_usize = value as usize;
-        let (block, index) = (value_usize / 64, value_usize % 64);
-        if block >= self.blocks().len() {
-            self.resize(value);
-        }
-        let blocks = self.blocks_mut();
-        let missing = blocks[block] & (1 << index) == 0;
-        blocks[block] |= 1 << index;
-        missing
-    }
-
-    /// Intersect in-place with another [`BitSet`].
-    pub(super) fn intersect(&mut self, other: &BitSet<B>) {
-        let my_blocks = self.blocks_mut();
-        let other_blocks = other.blocks();
-        let min_len = my_blocks.len().min(other_blocks.len());
-        for i in 0..min_len {
-            my_blocks[i] &= other_blocks[i];
-        }
-        for block in my_blocks.iter_mut().skip(min_len) {
-            *block = 0;
-        }
-    }
-
-    /// Union in-place with another [`BitSet`].
-    pub(super) fn union(&mut self, other: &BitSet<B>) {
-        let mut max_len = self.blocks().len();
-        let other_len = other.blocks().len();
-        if other_len > max_len {
-            max_len = other_len;
-            self.resize_blocks(max_len);
-        }
-        for (my_block, other_block) in self.blocks_mut().iter_mut().zip(other.blocks()) {
-            *my_block |= other_block;
-        }
-    }
-
-    /// Return an iterator over the values (in ascending order) in this [`BitSet`].
-    pub(super) fn iter(&self) -> BitSetIterator<'_, B> {
-        let blocks = self.blocks();
-        BitSetIterator {
-            blocks,
-            current_block_index: 0,
-            current_block: blocks[0],
-        }
-    }
-}
-
-/// Iterator over values in a [`BitSet`].
-#[derive(Debug)]
-pub(super) struct BitSetIterator<'a, const B: usize> {
-    /// The blocks we are iterating over.
-    blocks: &'a [u64],
-
-    /// The index of the block we are currently iterating through.
-    current_block_index: usize,
-
-    /// The block we are currently iterating through (and zeroing as we go.)
-    current_block: u64,
-}
-
-impl<const B: usize> Iterator for BitSetIterator<'_, B> {
-    type Item = u32;
-
-    fn next(&mut self) -> Option<Self::Item> {
-        while self.current_block == 0 {
-            if self.current_block_index + 1 >= self.blocks.len() {
-                return None;
-            }
-            self.current_block_index += 1;
-            self.current_block = self.blocks[self.current_block_index];
-        }
-        let lowest_bit_set = self.current_block.trailing_zeros();
-        // reset the lowest set bit, without a data dependency on `lowest_bit_set`
-        self.current_block &= self.current_block.wrapping_sub(1);
-        // SAFETY: `lowest_bit_set` cannot be more than 64, `current_block_index` cannot be more
-        // than `B - 1`, and we check above that `B * 64 < u32::MAX`. So both `64 *
-        // current_block_index` and the final value here must fit in u32.
-        #[allow(clippy::cast_possible_truncation)]
-        Some(lowest_bit_set + (64 * self.current_block_index) as u32)
-    }
-}
-
-impl<const B: usize> std::iter::FusedIterator for BitSetIterator<'_, B> {}
-
-#[cfg(test)]
-mod tests {
-    use super::BitSet;
-
-    fn assert_bitset<const B: usize>(bitset: &BitSet<B>, contents: &[u32]) {
-        assert_eq!(bitset.iter().collect::<Vec<_>>(), contents);
-    }
-
-    #[test]
-    fn iter() {
-        let mut b = BitSet::<1>::with(3);
-        b.insert(27);
-        b.insert(6);
-        assert!(matches!(b, BitSet::Inline(_)));
-        assert_bitset(&b, &[3, 6, 27]);
-    }
-
-    #[test]
-    fn iter_overflow() {
-        let mut b = BitSet::<1>::with(140);
-        b.insert(100);
-        b.insert(129);
-        assert!(matches!(b, BitSet::Heap(_)));
-        assert_bitset(&b, &[100, 129, 140]);
-    }
-
-    #[test]
-    fn intersect() {
-        let mut b1 = BitSet::<1>::with(4);
-        let mut b2 = BitSet::<1>::with(4);
-        b1.insert(23);
-        b2.insert(5);
-
-        b1.intersect(&b2);
-        assert_bitset(&b1, &[4]);
-    }
-
-    #[test]
-    fn intersect_mixed_1() {
-        let mut b1 = BitSet::<1>::with(4);
-        let mut b2 = BitSet::<1>::with(4);
-        b1.insert(89);
-        b2.insert(5);
-
-        b1.intersect(&b2);
-        assert_bitset(&b1, &[4]);
-    }
-
-    #[test]
-    fn intersect_mixed_2() {
-        let mut b1 = BitSet::<1>::with(4);
-        let mut b2 = BitSet::<1>::with(4);
-        b1.insert(23);
-        b2.insert(89);
-
-        b1.intersect(&b2);
-        assert_bitset(&b1, &[4]);
-    }
-
-    #[test]
-    fn intersect_heap() {
-        let mut b1 = BitSet::<1>::with(4);
-        let mut b2 = BitSet::<1>::with(4);
-        b1.insert(89);
-        b2.insert(90);
-
-        b1.intersect(&b2);
-        assert_bitset(&b1, &[4]);
-    }
-
-    #[test]
-    fn intersect_heap_2() {
-        let mut b1 = BitSet::<1>::with(89);
-        let mut b2 = BitSet::<1>::with(89);
-        b1.insert(91);
-        b2.insert(90);
-
-        b1.intersect(&b2);
-        assert_bitset(&b1, &[89]);
-    }
-
-    #[test]
-    fn union() {
-        let mut b1 = BitSet::<1>::with(2);
-        let b2 = BitSet::<1>::with(4);
-
-        b1.union(&b2);
-        assert_bitset(&b1, &[2, 4]);
-    }
-
-    #[test]
-    fn union_mixed_1() {
-        let mut b1 = BitSet::<1>::with(4);
-        let mut b2 = BitSet::<1>::with(4);
-        b1.insert(89);
-        b2.insert(5);
-
-        b1.union(&b2);
-        assert_bitset(&b1, &[4, 5, 89]);
-    }
-
-    #[test]
-    fn union_mixed_2() {
-        let mut b1 = BitSet::<1>::with(4);
-        let mut b2 = BitSet::<1>::with(4);
-        b1.insert(23);
-        b2.insert(89);
-
-        b1.union(&b2);
-        assert_bitset(&b1, &[4, 23, 89]);
-    }
-
-    #[test]
-    fn union_heap() {
-        let mut b1 = BitSet::<1>::with(4);
-        let mut b2 = BitSet::<1>::with(4);
-        b1.insert(89);
-        b2.insert(90);
-
-        b1.union(&b2);
-        assert_bitset(&b1, &[4, 89, 90]);
-    }
-
-    #[test]
-    fn union_heap_2() {
-        let mut b1 = BitSet::<1>::with(89);
-        let mut b2 = BitSet::<1>::with(89);
-        b1.insert(91);
-        b2.insert(90);
-
-        b1.union(&b2);
-        assert_bitset(&b1, &[89, 90, 91]);
-    }
-
-    #[test]
-    fn multiple_blocks() {
-        let mut b = BitSet::<2>::with(120);
-        b.insert(45);
-        assert!(matches!(b, BitSet::Inline(_)));
-        assert_bitset(&b, &[45, 120]);
-    }
-}

crates/red_knot_python_semantic/src/semantic_index/use_def/symbol_state.rs

@@ -36,24 +36,26 @@
 //! dominates, but it does dominate the `x = 1 if flag2 else None` binding, so we have to keep
 //! track of that.
 //!
-//! The data structures used here ([`BitSet`] and [`smallvec::SmallVec`]) optimize for keeping all
-//! data inline (avoiding lots of scattered allocations) in small-to-medium cases, and falling back
-//! to heap allocation to be able to scale to arbitrary numbers of live bindings and constraints
-//! when needed.
+//! The data structures use `IndexVec` arenas to store all data compactly and contiguously, while
+//! supporting very cheap clones.
 //!
 //! Tracking live declarations is simpler, since constraints are not involved, but otherwise very
 //! similar to tracking live bindings.
 
 use itertools::{EitherOrBoth, Itertools};
 use ruff_index::newtype_index;
-use smallvec::SmallVec;
+use smallvec::{smallvec, SmallVec};
 
-use crate::semantic_index::use_def::bitset::{BitSet, BitSetIterator};
-use crate::semantic_index::use_def::VisibilityConstraintsBuilder;
-use crate::visibility_constraints::ScopedVisibilityConstraintId;
+use crate::semantic_index::narrowing_constraints::{
+    NarrowingConstraintsBuilder, ScopedNarrowingConstraintId, ScopedNarrowingConstraintPredicate,
+};
+use crate::semantic_index::visibility_constraints::{
+    ScopedVisibilityConstraintId, VisibilityConstraintsBuilder,
+};
 
 /// A newtype-index for a definition in a particular scope.
 #[newtype_index]
+#[derive(Ord, PartialOrd)]
 pub(super) struct ScopedDefinitionId;
 
 impl ScopedDefinitionId {
@@ -65,89 +67,46 @@ impl ScopedDefinitionId {
     pub(super) const UNBOUND: ScopedDefinitionId = ScopedDefinitionId::from_u32(0);
 }
 
-/// A newtype-index for a constraint expression in a particular scope.
-#[newtype_index]
-pub(crate) struct ScopedConstraintId;
-
-/// Can reference this * 64 total definitions inline; more will fall back to the heap.
-const INLINE_BINDING_BLOCKS: usize = 3;
-
-/// A [`BitSet`] of [`ScopedDefinitionId`], representing live bindings of a symbol in a scope.
-type Bindings = BitSet<INLINE_BINDING_BLOCKS>;
-type BindingsIterator<'a> = BitSetIterator<'a, INLINE_BINDING_BLOCKS>;
-
-/// Can reference this * 64 total declarations inline; more will fall back to the heap.
-const INLINE_DECLARATION_BLOCKS: usize = 3;
-
-/// A [`BitSet`] of [`ScopedDefinitionId`], representing live declarations of a symbol in a scope.
-type Declarations = BitSet<INLINE_DECLARATION_BLOCKS>;
-type DeclarationsIterator<'a> = BitSetIterator<'a, INLINE_DECLARATION_BLOCKS>;
-
-/// Can reference this * 64 total constraints inline; more will fall back to the heap.
-const INLINE_CONSTRAINT_BLOCKS: usize = 2;
-
-/// Can keep inline this many live bindings per symbol at a given time; more will go to heap.
-const INLINE_BINDINGS_PER_SYMBOL: usize = 4;
-
-/// Which constraints apply to a given binding?
-type Constraints = BitSet<INLINE_CONSTRAINT_BLOCKS>;
-
-type InlineConstraintArray = [Constraints; INLINE_BINDINGS_PER_SYMBOL];
-
-/// One [`BitSet`] of applicable [`ScopedConstraintId`]s per live binding.
-type ConstraintsPerBinding = SmallVec<InlineConstraintArray>;
-
-/// Iterate over all constraints for a single binding.
-type ConstraintsIterator<'a> = std::slice::Iter<'a, Constraints>;
-
-const INLINE_VISIBILITY_CONSTRAINTS: usize = 4;
-type InlineVisibilityConstraintsArray =
-    [ScopedVisibilityConstraintId; INLINE_VISIBILITY_CONSTRAINTS];
-
-/// One [`ScopedVisibilityConstraintId`] per live declaration.
-type VisibilityConstraintPerDeclaration = SmallVec<InlineVisibilityConstraintsArray>;
-
-/// One [`ScopedVisibilityConstraintId`] per live binding.
-type VisibilityConstraintPerBinding = SmallVec<InlineVisibilityConstraintsArray>;
-
-/// Iterator over the visibility constraints for all live bindings/declarations.
-type VisibilityConstraintsIterator<'a> = std::slice::Iter<'a, ScopedVisibilityConstraintId>;
+/// Can keep inline this many live bindings or declarations per symbol at a given time; more will
+/// go to heap.
+const INLINE_DEFINITIONS_PER_SYMBOL: usize = 4;
 
 /// Live declarations for a single symbol at some point in control flow, with their
 /// corresponding visibility constraints.
 #[derive(Clone, Debug, Default, PartialEq, Eq, salsa::Update)]
 pub(super) struct SymbolDeclarations {
-    /// [`BitSet`]: which declarations (as [`ScopedDefinitionId`]) can reach the current location?
-    ///
-    /// Invariant: Because this is a `BitSet`, it can be viewed as a _sorted_ set of definition
-    /// IDs. The `visibility_constraints` field stores constraints for each definition. Therefore
-    /// those fields must always have the same `len()` as `live_declarations`, and the elements
-    /// must appear in the same order.  Effectively, this means that elements must always be added
-    /// in sorted order, or via a binary search that determines the correct place to insert new
-    /// constraints.
-    pub(crate) live_declarations: Declarations,
-
-    /// For each live declaration, which visibility constraint applies to it?
-    pub(crate) visibility_constraints: VisibilityConstraintPerDeclaration,
+    /// A list of live declarations for this symbol, sorted by their `ScopedDefinitionId`
+    live_declarations: SmallVec<[LiveDeclaration; INLINE_DEFINITIONS_PER_SYMBOL]>,
 }
 
+/// One of the live declarations for a single symbol at some point in control flow.
+#[derive(Clone, Debug, PartialEq, Eq)]
+pub(super) struct LiveDeclaration {
+    pub(super) declaration: ScopedDefinitionId,
+    pub(super) visibility_constraint: ScopedVisibilityConstraintId,
+}
+
+pub(super) type LiveDeclarationsIterator<'a> = std::slice::Iter<'a, LiveDeclaration>;
+
 impl SymbolDeclarations {
     fn undeclared(scope_start_visibility: ScopedVisibilityConstraintId) -> Self {
+        let initial_declaration = LiveDeclaration {
+            declaration: ScopedDefinitionId::UNBOUND,
+            visibility_constraint: scope_start_visibility,
+        };
         Self {
-            live_declarations: Declarations::with(0),
-            visibility_constraints: VisibilityConstraintPerDeclaration::from_iter([
-                scope_start_visibility,
-            ]),
+            live_declarations: smallvec![initial_declaration],
         }
     }
 
     /// Record a newly-encountered declaration for this symbol.
-    fn record_declaration(&mut self, declaration_id: ScopedDefinitionId) {
-        self.live_declarations = Declarations::with(declaration_id.into());
-
-        self.visibility_constraints = VisibilityConstraintPerDeclaration::with_capacity(1);
-        self.visibility_constraints
-            .push(ScopedVisibilityConstraintId::ALWAYS_TRUE);
+    fn record_declaration(&mut self, declaration: ScopedDefinitionId) {
+        // The new declaration replaces all previous live declaration in this path.
+        self.live_declarations.clear();
+        self.live_declarations.push(LiveDeclaration {
+            declaration,
+            visibility_constraint: ScopedVisibilityConstraintId::ALWAYS_TRUE,
+        });
     }
 
     /// Add given visibility constraint to all live declarations.
@@ -156,45 +115,62 @@ impl SymbolDeclarations {
         visibility_constraints: &mut VisibilityConstraintsBuilder,
         constraint: ScopedVisibilityConstraintId,
     ) {
-        for existing in &mut self.visibility_constraints {
-            *existing = visibility_constraints.add_and_constraint(*existing, constraint);
+        for declaration in &mut self.live_declarations {
+            declaration.visibility_constraint = visibility_constraints
+                .add_and_constraint(declaration.visibility_constraint, constraint);
         }
     }
 
     /// Return an iterator over live declarations for this symbol.
-    pub(super) fn iter(&self) -> DeclarationIdIterator {
-        DeclarationIdIterator {
-            declarations: self.live_declarations.iter(),
-            visibility_constraints: self.visibility_constraints.iter(),
+    pub(super) fn iter(&self) -> LiveDeclarationsIterator<'_> {
+        self.live_declarations.iter()
+    }
+
+    /// Iterate over the IDs of each currently live declaration for this symbol
+    fn iter_declarations(&self) -> impl Iterator<Item = ScopedDefinitionId> + '_ {
+        self.iter().map(|lb| lb.declaration)
+    }
+
+    fn simplify_visibility_constraints(&mut self, other: SymbolDeclarations) {
+        // If the set of live declarations hasn't changed, don't simplify.
+        if self.live_declarations.len() != other.live_declarations.len()
+            || !self.iter_declarations().eq(other.iter_declarations())
+        {
+            return;
+        }
+
+        for (declaration, other_declaration) in self
+            .live_declarations
+            .iter_mut()
+            .zip(other.live_declarations)
+        {
+            declaration.visibility_constraint = other_declaration.visibility_constraint;
         }
     }
 
     fn merge(&mut self, b: Self, visibility_constraints: &mut VisibilityConstraintsBuilder) {
         let a = std::mem::take(self);
-        self.live_declarations = a.live_declarations.clone();
-        self.live_declarations.union(&b.live_declarations);
-
-        // Invariant: These zips are well-formed since we maintain an invariant that all of our
-        // fields are sets/vecs with the same length.
-        let a = (a.live_declarations.iter()).zip(a.visibility_constraints);
-        let b = (b.live_declarations.iter()).zip(b.visibility_constraints);
 
         // Invariant: merge_join_by consumes the two iterators in sorted order, which ensures that
-        // the definition IDs and constraints line up correctly in the merged result. If a
-        // definition is found in both `a` and `b`, we compose the constraints from the two paths
-        // in an appropriate way (intersection for narrowing constraints; ternary OR for visibility
-        // constraints). If a definition is found in only one path, it is used as-is.
-        for zipped in a.merge_join_by(b, |(a_decl, _), (b_decl, _)| a_decl.cmp(b_decl)) {
+        // the merged `live_declarations` vec remains sorted. If a definition is found in both `a`
+        // and `b`, we compose the constraints from the two paths in an appropriate way
+        // (intersection for narrowing constraints; ternary OR for visibility constraints). If a
+        // definition is found in only one path, it is used as-is.
+        let a = a.live_declarations.into_iter();
+        let b = b.live_declarations.into_iter();
+        for zipped in a.merge_join_by(b, |a, b| a.declaration.cmp(&b.declaration)) {
             match zipped {
-                EitherOrBoth::Both((_, a_vis_constraint), (_, b_vis_constraint)) => {
-                    let vis_constraint = visibility_constraints
-                        .add_or_constraint(a_vis_constraint, b_vis_constraint);
-                    self.visibility_constraints.push(vis_constraint);
+                EitherOrBoth::Both(a, b) => {
+                    let visibility_constraint = visibility_constraints
+                        .add_or_constraint(a.visibility_constraint, b.visibility_constraint);
+                    self.live_declarations.push(LiveDeclaration {
+                        declaration: a.declaration,
+                        visibility_constraint,
+                    });
                 }
 
-                EitherOrBoth::Left((_, vis_constraint))
-                | EitherOrBoth::Right((_, vis_constraint)) => {
-                    self.visibility_constraints.push(vis_constraint);
+                EitherOrBoth::Left(declaration) | EitherOrBoth::Right(declaration) => {
+                    self.live_declarations.push(declaration);
                 }
             }
         }
@@ -205,57 +181,57 @@ impl SymbolDeclarations {
 /// with a set of narrowing constraints and a visibility constraint.
 #[derive(Clone, Debug, Default, PartialEq, Eq, salsa::Update)]
 pub(super) struct SymbolBindings {
-    /// [`BitSet`]: which bindings (as [`ScopedDefinitionId`]) can reach the current location?
-    ///
-    /// Invariant: Because this is a `BitSet`, it can be viewed as a _sorted_ set of definition
-    /// IDs. The `constraints` and `visibility_constraints` field stores constraints for each
-    /// definition. Therefore those fields must always have the same `len()` as
-    /// `live_bindings`, and the elements must appear in the same order.  Effectively, this means
-    /// that elements must always be added in sorted order, or via a binary search that determines
-    /// the correct place to insert new constraints.
-    live_bindings: Bindings,
-
-    /// For each live binding, which [`ScopedConstraintId`] apply?
-    ///
-    /// This is a [`smallvec::SmallVec`] which should always have one [`BitSet`] of constraints per
-    /// binding in `live_bindings`.
-    constraints: ConstraintsPerBinding,
-
-    /// For each live binding, which visibility constraint applies to it?
-    visibility_constraints: VisibilityConstraintPerBinding,
+    /// A list of live bindings for this symbol, sorted by their `ScopedDefinitionId`
+    live_bindings: SmallVec<[LiveBinding; INLINE_DEFINITIONS_PER_SYMBOL]>,
 }
 
+/// One of the live bindings for a single symbol at some point in control flow.
+#[derive(Clone, Debug, PartialEq, Eq)]
+pub(super) struct LiveBinding {
+    pub(super) binding: ScopedDefinitionId,
+    pub(super) narrowing_constraint: Option<ScopedNarrowingConstraintId>,
+    pub(super) visibility_constraint: ScopedVisibilityConstraintId,
+}
+
+pub(super) type LiveBindingsIterator<'a> = std::slice::Iter<'a, LiveBinding>;
+
 impl SymbolBindings {
     fn unbound(scope_start_visibility: ScopedVisibilityConstraintId) -> Self {
+        let initial_binding = LiveBinding {
+            binding: ScopedDefinitionId::UNBOUND,
+            narrowing_constraint: None,
+            visibility_constraint: scope_start_visibility,
+        };
         Self {
-            live_bindings: Bindings::with(ScopedDefinitionId::UNBOUND.as_u32()),
-            constraints: ConstraintsPerBinding::from_iter([Constraints::default()]),
-            visibility_constraints: VisibilityConstraintPerBinding::from_iter([
-                scope_start_visibility,
-            ]),
+            live_bindings: smallvec![initial_binding],
         }
     }
 
     /// Record a newly-encountered binding for this symbol.
     pub(super) fn record_binding(
         &mut self,
-        binding_id: ScopedDefinitionId,
+        binding: ScopedDefinitionId,
         visibility_constraint: ScopedVisibilityConstraintId,
     ) {
         // The new binding replaces all previous live bindings in this path, and has no
         // constraints.
-        self.live_bindings = Bindings::with(binding_id.into());
-        self.constraints = ConstraintsPerBinding::with_capacity(1);
-        self.constraints.push(Constraints::default());
-
-        self.visibility_constraints = VisibilityConstraintPerBinding::with_capacity(1);
-        self.visibility_constraints.push(visibility_constraint);
+        self.live_bindings.clear();
+        self.live_bindings.push(LiveBinding {
+            binding,
+            narrowing_constraint: None,
+            visibility_constraint,
+        });
     }
 
     /// Add given constraint to all live bindings.
-    pub(super) fn record_constraint(&mut self, constraint_id: ScopedConstraintId) {
-        for bitset in &mut self.constraints {
-            bitset.insert(constraint_id.into());
+    pub(super) fn record_narrowing_constraint(
+        &mut self,
+        narrowing_constraints: &mut NarrowingConstraintsBuilder,
+        predicate: ScopedNarrowingConstraintPredicate,
+    ) {
+        for binding in &mut self.live_bindings {
+            binding.narrowing_constraint = narrowing_constraints
+                .add_predicate_to_constraint(binding.narrowing_constraint, predicate);
         }
     }
 
@@ -265,71 +241,72 @@ impl SymbolBindings {
         visibility_constraints: &mut VisibilityConstraintsBuilder,
         constraint: ScopedVisibilityConstraintId,
     ) {
-        for existing in &mut self.visibility_constraints {
-            *existing = visibility_constraints.add_and_constraint(*existing, constraint);
+        for binding in &mut self.live_bindings {
+            binding.visibility_constraint = visibility_constraints
+                .add_and_constraint(binding.visibility_constraint, constraint);
         }
     }
 
     /// Iterate over currently live bindings for this symbol
-    pub(super) fn iter(&self) -> BindingIdWithConstraintsIterator {
-        BindingIdWithConstraintsIterator {
-            definitions: self.live_bindings.iter(),
-            constraints: self.constraints.iter(),
-            visibility_constraints: self.visibility_constraints.iter(),
+    pub(super) fn iter(&self) -> LiveBindingsIterator<'_> {
+        self.live_bindings.iter()
+    }
+
+    /// Iterate over the IDs of each currently live binding for this symbol
+    fn iter_bindings(&self) -> impl Iterator<Item = ScopedDefinitionId> + '_ {
+        self.iter().map(|lb| lb.binding)
+    }
+
+    fn simplify_visibility_constraints(&mut self, other: SymbolBindings) {
+        // If the set of live bindings hasn't changed, don't simplify.
+        if self.live_bindings.len() != other.live_bindings.len()
+            || !self.iter_bindings().eq(other.iter_bindings())
+        {
+            return;
+        }
+
+        for (binding, other_binding) in self.live_bindings.iter_mut().zip(other.live_bindings) {
+            binding.visibility_constraint = other_binding.visibility_constraint;
         }
     }
 
-    fn merge(&mut self, mut b: Self, visibility_constraints: &mut VisibilityConstraintsBuilder) {
-        let mut a = std::mem::take(self);
-        self.live_bindings = a.live_bindings.clone();
-        self.live_bindings.union(&b.live_bindings);
-
-        // Invariant: These zips are well-formed since we maintain an invariant that all of our
-        // fields are sets/vecs with the same length.
-        //
-        // Performance: We iterate over the `constraints` smallvecs via mut reference, because the
-        // individual elements are `BitSet`s (currently 24 bytes in size), and we don't want to
-        // move them by value multiple times during iteration. By iterating by reference, we only
-        // have to copy single pointers around.  In the loop below, the `std::mem::take` calls
-        // specify precisely where we want to move them into the merged `constraints` smallvec.
-        //
-        // We don't need a similar optimization for `visibility_constraints`, since those elements
-        // are 32-bit IndexVec IDs, and so are already cheap to move/copy.
-        let a = (a.live_bindings.iter())
-            .zip(a.constraints.iter_mut())
-            .zip(a.visibility_constraints);
-        let b = (b.live_bindings.iter())
-            .zip(b.constraints.iter_mut())
-            .zip(b.visibility_constraints);
+    fn merge(
+        &mut self,
+        b: Self,
+        narrowing_constraints: &mut NarrowingConstraintsBuilder,
+        visibility_constraints: &mut VisibilityConstraintsBuilder,
+    ) {
+        let a = std::mem::take(self);
 
         // Invariant: merge_join_by consumes the two iterators in sorted order, which ensures that
-        // the definition IDs and constraints line up correctly in the merged result. If a
-        // definition is found in both `a` and `b`, we compose the constraints from the two paths
-        // in an appropriate way (intersection for narrowing constraints; ternary OR for visibility
-        // constraints). If a definition is found in only one path, it is used as-is.
-        for zipped in a.merge_join_by(b, |((a_def, _), _), ((b_def, _), _)| a_def.cmp(b_def)) {
+        // the merged `live_bindings` vec remains sorted. If a definition is found in both `a` and
+        // `b`, we compose the constraints from the two paths in an appropriate way (intersection
+        // for narrowing constraints; ternary OR for visibility constraints). If a definition is
+        // found in only one path, it is used as-is.
+        let a = a.live_bindings.into_iter();
+        let b = b.live_bindings.into_iter();
+        for zipped in a.merge_join_by(b, |a, b| a.binding.cmp(&b.binding)) {
             match zipped {
-                EitherOrBoth::Both(
-                    ((_, a_constraints), a_vis_constraint),
-                    ((_, b_constraints), b_vis_constraint),
-                ) => {
+                EitherOrBoth::Both(a, b) => {
                     // If the same definition is visible through both paths, any constraint
                     // that applies on only one path is irrelevant to the resulting type from
                     // unioning the two paths, so we intersect the constraints.
-                    let constraints = a_constraints;
-                    constraints.intersect(b_constraints);
-                    self.constraints.push(std::mem::take(constraints));
+                    let narrowing_constraint = narrowing_constraints
+                        .intersect_constraints(a.narrowing_constraint, b.narrowing_constraint);
 
                     // For visibility constraints, we merge them using a ternary OR operation:
-                    let vis_constraint = visibility_constraints
-                        .add_or_constraint(a_vis_constraint, b_vis_constraint);
-                    self.visibility_constraints.push(vis_constraint);
+                    let visibility_constraint = visibility_constraints
+                        .add_or_constraint(a.visibility_constraint, b.visibility_constraint);
+
+                    self.live_bindings.push(LiveBinding {
+                        binding: a.binding,
+                        narrowing_constraint,
+                        visibility_constraint,
+                    });
                 }
 
-                EitherOrBoth::Left(((_, constraints), vis_constraint))
-                | EitherOrBoth::Right(((_, constraints), vis_constraint)) => {
-                    self.constraints.push(std::mem::take(constraints));
-                    self.visibility_constraints.push(vis_constraint);
+                EitherOrBoth::Left(binding) | EitherOrBoth::Right(binding) => {
+                    self.live_bindings.push(binding);
                 }
             }
         }
@@ -363,8 +340,13 @@ impl SymbolState {
     }
 
     /// Add given constraint to all live bindings.
-    pub(super) fn record_constraint(&mut self, constraint_id: ScopedConstraintId) {
-        self.bindings.record_constraint(constraint_id);
+    pub(super) fn record_narrowing_constraint(
+        &mut self,
+        narrowing_constraints: &mut NarrowingConstraintsBuilder,
+        constraint: ScopedNarrowingConstraintPredicate,
+    ) {
+        self.bindings
+            .record_narrowing_constraint(narrowing_constraints, constraint);
     }
 
     /// Add given visibility constraint to all live bindings.
@@ -379,14 +361,14 @@ impl SymbolState {
             .record_visibility_constraint(visibility_constraints, constraint);
     }
 
+    /// Simplifies this snapshot to have the same visibility constraints as a previous point in the
+    /// control flow, but only if the set of live bindings or declarations for this symbol hasn't
+    /// changed.
     pub(super) fn simplify_visibility_constraints(&mut self, snapshot_state: SymbolState) {
-        if self.bindings.live_bindings == snapshot_state.bindings.live_bindings {
-            self.bindings.visibility_constraints = snapshot_state.bindings.visibility_constraints;
-        }
-        if self.declarations.live_declarations == snapshot_state.declarations.live_declarations {
-            self.declarations.visibility_constraints =
-                snapshot_state.declarations.visibility_constraints;
-        }
+        self.bindings
+            .simplify_visibility_constraints(snapshot_state.bindings);
+        self.declarations
+            .simplify_visibility_constraints(snapshot_state.declarations);
     }
 
     /// Record a newly-encountered declaration of this symbol.
@@ -398,9 +380,11 @@ impl SymbolState {
     pub(super) fn merge(
         &mut self,
         b: SymbolState,
+        narrowing_constraints: &mut NarrowingConstraintsBuilder,
         visibility_constraints: &mut VisibilityConstraintsBuilder,
     ) {
-        self.bindings.merge(b.bindings, visibility_constraints);
+        self.bindings
+            .merge(b.bindings, narrowing_constraints, visibility_constraints);
         self.declarations
             .merge(b.declarations, visibility_constraints);
     }
@@ -414,121 +398,34 @@ impl SymbolState {
     }
 }
 
-/// A single binding (as [`ScopedDefinitionId`]) with an iterator of its applicable
-/// narrowing constraints ([`ScopedConstraintId`]) and a corresponding visibility
-/// visibility constraint ([`ScopedVisibilityConstraintId`]).
-#[derive(Debug)]
-pub(super) struct BindingIdWithConstraints<'map> {
-    pub(super) definition: ScopedDefinitionId,
-    pub(super) constraint_ids: ConstraintIdIterator<'map>,
-    pub(super) visibility_constraint: ScopedVisibilityConstraintId,
-}
-
-#[derive(Debug)]
-pub(super) struct BindingIdWithConstraintsIterator<'map> {
-    definitions: BindingsIterator<'map>,
-    constraints: ConstraintsIterator<'map>,
-    visibility_constraints: VisibilityConstraintsIterator<'map>,
-}
-
-impl<'map> Iterator for BindingIdWithConstraintsIterator<'map> {
-    type Item = BindingIdWithConstraints<'map>;
-
-    fn next(&mut self) -> Option<Self::Item> {
-        match (
-            self.definitions.next(),
-            self.constraints.next(),
-            self.visibility_constraints.next(),
-        ) {
-            (None, None, None) => None,
-            (Some(def), Some(constraints), Some(visibility_constraint_id)) => {
-                Some(BindingIdWithConstraints {
-                    definition: ScopedDefinitionId::from_u32(def),
-                    constraint_ids: ConstraintIdIterator {
-                        wrapped: constraints.iter(),
-                    },
-                    visibility_constraint: *visibility_constraint_id,
-                })
-            }
-            // SAFETY: see above.
-            _ => unreachable!("definitions and constraints length mismatch"),
-        }
-    }
-}
-
-impl std::iter::FusedIterator for BindingIdWithConstraintsIterator<'_> {}
-
-#[derive(Debug)]
-pub(super) struct ConstraintIdIterator<'a> {
-    wrapped: BitSetIterator<'a, INLINE_CONSTRAINT_BLOCKS>,
-}
-
-impl Iterator for ConstraintIdIterator<'_> {
-    type Item = ScopedConstraintId;
-
-    fn next(&mut self) -> Option<Self::Item> {
-        self.wrapped.next().map(ScopedConstraintId::from_u32)
-    }
-}
-
-impl std::iter::FusedIterator for ConstraintIdIterator<'_> {}
-
-/// A single declaration (as [`ScopedDefinitionId`]) with a corresponding visibility
-/// visibility constraint ([`ScopedVisibilityConstraintId`]).
-#[derive(Debug)]
-pub(super) struct DeclarationIdWithConstraint {
-    pub(super) definition: ScopedDefinitionId,
-    pub(super) visibility_constraint: ScopedVisibilityConstraintId,
-}
-
-pub(super) struct DeclarationIdIterator<'map> {
-    pub(crate) declarations: DeclarationsIterator<'map>,
-    pub(crate) visibility_constraints: VisibilityConstraintsIterator<'map>,
-}
-
-impl Iterator for DeclarationIdIterator<'_> {
-    type Item = DeclarationIdWithConstraint;
-
-    fn next(&mut self) -> Option<Self::Item> {
-        match (self.declarations.next(), self.visibility_constraints.next()) {
-            (None, None) => None,
-            (Some(declaration), Some(&visibility_constraint)) => {
-                Some(DeclarationIdWithConstraint {
-                    definition: ScopedDefinitionId::from_u32(declaration),
-                    visibility_constraint,
-                })
-            }
-            // SAFETY: see above.
-            _ => unreachable!("declarations and visibility_constraints length mismatch"),
-        }
-    }
-}
-
-impl std::iter::FusedIterator for DeclarationIdIterator<'_> {}
-
 #[cfg(test)]
 mod tests {
     use super::*;
 
+    use crate::semantic_index::predicate::ScopedPredicateId;
+
     #[track_caller]
-    fn assert_bindings(symbol: &SymbolState, expected: &[&str]) {
+    fn assert_bindings(
+        narrowing_constraints: &NarrowingConstraintsBuilder,
+        symbol: &SymbolState,
+        expected: &[&str],
+    ) {
         let actual = symbol
             .bindings()
             .iter()
-            .map(|def_id_with_constraints| {
-                let def_id = def_id_with_constraints.definition;
+            .map(|live_binding| {
+                let def_id = live_binding.binding;
                 let def = if def_id == ScopedDefinitionId::UNBOUND {
                     "unbound".into()
                 } else {
                     def_id.as_u32().to_string()
                 };
-                let constraints = def_id_with_constraints
-                    .constraint_ids
-                    .map(ScopedConstraintId::as_u32)
-                    .map(|idx| idx.to_string())
+                let predicates = narrowing_constraints
+                    .iter_predicates(live_binding.narrowing_constraint)
+                    .map(|idx| idx.as_u32().to_string())
                     .collect::<Vec<_>>()
                     .join(", ");
-                format!("{def}<{constraints}>")
+                format!("{def}<{predicates}>")
             })
             .collect::<Vec<_>>();
         assert_eq!(actual, expected);
@@ -540,14 +437,14 @@ mod tests {
             .declarations()
             .iter()
             .map(
-                |DeclarationIdWithConstraint {
-                     definition,
+                |LiveDeclaration {
+                     declaration,
                      visibility_constraint: _,
                  }| {
-                    if definition == ScopedDefinitionId::UNBOUND {
+                    if *declaration == ScopedDefinitionId::UNBOUND {
                         "undeclared".into()
                     } else {
-                        definition.as_u32().to_string()
+                        declaration.as_u32().to_string()
                     }
                 },
             )
@@ -557,36 +454,41 @@ mod tests {
 
     #[test]
     fn unbound() {
+        let narrowing_constraints = NarrowingConstraintsBuilder::default();
         let sym = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
 
-        assert_bindings(&sym, &["unbound<>"]);
+        assert_bindings(&narrowing_constraints, &sym, &["unbound<>"]);
     }
 
     #[test]
     fn with() {
+        let narrowing_constraints = NarrowingConstraintsBuilder::default();
         let mut sym = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
         sym.record_binding(
             ScopedDefinitionId::from_u32(1),
             ScopedVisibilityConstraintId::ALWAYS_TRUE,
         );
 
-        assert_bindings(&sym, &["1<>"]);
+        assert_bindings(&narrowing_constraints, &sym, &["1<>"]);
     }
 
     #[test]
     fn record_constraint() {
+        let mut narrowing_constraints = NarrowingConstraintsBuilder::default();
         let mut sym = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
         sym.record_binding(
             ScopedDefinitionId::from_u32(1),
             ScopedVisibilityConstraintId::ALWAYS_TRUE,
         );
-        sym.record_constraint(ScopedConstraintId::from_u32(0));
+        let predicate = ScopedPredicateId::from_u32(0).into();
+        sym.record_narrowing_constraint(&mut narrowing_constraints, predicate);
 
-        assert_bindings(&sym, &["1<0>"]);
+        assert_bindings(&narrowing_constraints, &sym, &["1<0>"]);
     }
 
     #[test]
     fn merge() {
+        let mut narrowing_constraints = NarrowingConstraintsBuilder::default();
         let mut visibility_constraints = VisibilityConstraintsBuilder::default();
 
         // merging the same definition with the same constraint keeps the constraint
@@ -595,18 +497,24 @@ mod tests {
             ScopedDefinitionId::from_u32(1),
             ScopedVisibilityConstraintId::ALWAYS_TRUE,
         );
-        sym1a.record_constraint(ScopedConstraintId::from_u32(0));
+        let predicate = ScopedPredicateId::from_u32(0).into();
+        sym1a.record_narrowing_constraint(&mut narrowing_constraints, predicate);
 
         let mut sym1b = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
         sym1b.record_binding(
             ScopedDefinitionId::from_u32(1),
             ScopedVisibilityConstraintId::ALWAYS_TRUE,
         );
-        sym1b.record_constraint(ScopedConstraintId::from_u32(0));
+        let predicate = ScopedPredicateId::from_u32(0).into();
+        sym1b.record_narrowing_constraint(&mut narrowing_constraints, predicate);
 
-        sym1a.merge(sym1b, &mut visibility_constraints);
+        sym1a.merge(
+            sym1b,
+            &mut narrowing_constraints,
+            &mut visibility_constraints,
+        );
         let mut sym1 = sym1a;
-        assert_bindings(&sym1, &["1<0>"]);
+        assert_bindings(&narrowing_constraints, &sym1, &["1<0>"]);
 
         // merging the same definition with differing constraints drops all constraints
         let mut sym2a = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
@@ -614,18 +522,24 @@ mod tests {
             ScopedDefinitionId::from_u32(2),
             ScopedVisibilityConstraintId::ALWAYS_TRUE,
         );
-        sym2a.record_constraint(ScopedConstraintId::from_u32(1));
+        let predicate = ScopedPredicateId::from_u32(1).into();
+        sym2a.record_narrowing_constraint(&mut narrowing_constraints, predicate);
 
         let mut sym1b = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
         sym1b.record_binding(
             ScopedDefinitionId::from_u32(2),
             ScopedVisibilityConstraintId::ALWAYS_TRUE,
         );
-        sym1b.record_constraint(ScopedConstraintId::from_u32(2));
+        let predicate = ScopedPredicateId::from_u32(2).into();
+        sym1b.record_narrowing_constraint(&mut narrowing_constraints, predicate);
 
-        sym2a.merge(sym1b, &mut visibility_constraints);
+        sym2a.merge(
+            sym1b,
+            &mut narrowing_constraints,
+            &mut visibility_constraints,
+        );
         let sym2 = sym2a;
-        assert_bindings(&sym2, &["2<>"]);
+        assert_bindings(&narrowing_constraints, &sym2, &["2<>"]);
 
         // merging a constrained definition with unbound keeps both
         let mut sym3a = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
@@ -633,18 +547,27 @@ mod tests {
             ScopedDefinitionId::from_u32(3),
             ScopedVisibilityConstraintId::ALWAYS_TRUE,
         );
-        sym3a.record_constraint(ScopedConstraintId::from_u32(3));
+        let predicate = ScopedPredicateId::from_u32(3).into();
+        sym3a.record_narrowing_constraint(&mut narrowing_constraints, predicate);
 
         let sym2b = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
 
-        sym3a.merge(sym2b, &mut visibility_constraints);
+        sym3a.merge(
+            sym2b,
+            &mut narrowing_constraints,
+            &mut visibility_constraints,
+        );
         let sym3 = sym3a;
-        assert_bindings(&sym3, &["unbound<>", "3<3>"]);
+        assert_bindings(&narrowing_constraints, &sym3, &["unbound<>", "3<3>"]);
 
         // merging different definitions keeps them each with their existing constraints
-        sym1.merge(sym3, &mut visibility_constraints);
+        sym1.merge(
+            sym3,
+            &mut narrowing_constraints,
+            &mut visibility_constraints,
+        );
         let sym = sym1;
-        assert_bindings(&sym, &["unbound<>", "1<0>", "3<3>"]);
+        assert_bindings(&narrowing_constraints, &sym, &["unbound<>", "1<0>", "3<3>"]);
     }
 
     #[test]
@@ -673,6 +596,7 @@ mod tests {
 
     #[test]
     fn record_declaration_merge() {
+        let mut narrowing_constraints = NarrowingConstraintsBuilder::default();
         let mut visibility_constraints = VisibilityConstraintsBuilder::default();
         let mut sym = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
         sym.record_declaration(ScopedDefinitionId::from_u32(1));
@@ -680,20 +604,29 @@ mod tests {
         let mut sym2 = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
         sym2.record_declaration(ScopedDefinitionId::from_u32(2));
 
-        sym.merge(sym2, &mut visibility_constraints);
+        sym.merge(
+            sym2,
+            &mut narrowing_constraints,
+            &mut visibility_constraints,
+        );
 
         assert_declarations(&sym, &["1", "2"]);
     }
 
     #[test]
     fn record_declaration_merge_partial_undeclared() {
+        let mut narrowing_constraints = NarrowingConstraintsBuilder::default();
         let mut visibility_constraints = VisibilityConstraintsBuilder::default();
         let mut sym = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
         sym.record_declaration(ScopedDefinitionId::from_u32(1));
 
         let sym2 = SymbolState::undefined(ScopedVisibilityConstraintId::ALWAYS_TRUE);
 
-        sym.merge(sym2, &mut visibility_constraints);
+        sym.merge(
+            sym2,
+            &mut narrowing_constraints,
+            &mut visibility_constraints,
+        );
 
         assert_declarations(&sym, &["undeclared", "1"]);
     }

crates/red_knot_python_semantic/src/semantic_index/visibility_constraints.rs

@@ -178,18 +178,17 @@ use std::cmp::Ordering;
 use ruff_index::{Idx, IndexVec};
 use rustc_hash::FxHashMap;
 
-use crate::semantic_index::{
-    ast_ids::HasScopedExpressionId,
-    constraint::{Constraint, ConstraintNode, PatternConstraintKind},
+use crate::semantic_index::predicate::{
+    PatternPredicateKind, Predicate, PredicateNode, Predicates, ScopedPredicateId,
 };
-use crate::types::{infer_expression_types, Truthiness};
+use crate::types::{infer_expression_type, Truthiness};
 use crate::Db;
 
 /// A ternary formula that defines under what conditions a binding is visible. (A ternary formula
 /// is just like a boolean formula, but with `Ambiguous` as a third potential result. See the
 /// module documentation for more details.)
 ///
-/// The primitive atoms of the formula are [`Constraint`]s, which express some property of the
+/// The primitive atoms of the formula are [`Predicate`]s, which express some property of the
 /// runtime state of the code that we are analyzing.
 ///
 /// We assume that each atom has a stable value each time that the formula is evaluated. An atom
@@ -198,7 +197,7 @@ use crate::Db;
 /// allows us to perform simplifications like `A ∨ !A → true` and `A ∧ !A → false`.
 ///
 /// That means that when you are constructing a formula, you might need to create distinct atoms
-/// for a particular [`Constraint`], if your formula needs to consider how a particular runtime
+/// for a particular [`Predicate`], if your formula needs to consider how a particular runtime
 /// property might be different at different points in the execution of the program.
 ///
 /// Visibility constraints are normalized, so equivalent constraints are guaranteed to have equal
@@ -226,7 +225,7 @@ impl std::fmt::Debug for ScopedVisibilityConstraintId {
 //
 // There are 3 terminals, with hard-coded constraint IDs: true, ambiguous, and false.
 //
-// _Atoms_ are the underlying Constraints, which are the variables that are evaluated by the
+// _Atoms_ are the underlying Predicates, which are the variables that are evaluated by the
 // ternary function.
 //
 // _Interior nodes_ provide the TDD structure for the formula. Interior nodes are stored in an
@@ -234,69 +233,15 @@ impl std::fmt::Debug for ScopedVisibilityConstraintId {
 
 #[derive(Clone, Copy, Debug, Eq, Hash, PartialEq)]
 struct InteriorNode {
-    atom: Atom,
+    /// A "variable" that is evaluated as part of a TDD ternary function. For visibility
+    /// constraints, this is a `Predicate` that represents some runtime property of the Python
+    /// code that we are evaluating.
+    atom: ScopedPredicateId,
     if_true: ScopedVisibilityConstraintId,
     if_ambiguous: ScopedVisibilityConstraintId,
     if_false: ScopedVisibilityConstraintId,
 }
 
-/// A "variable" that is evaluated as part of a TDD ternary function. For visibility constraints,
-/// this is a `Constraint` that represents some runtime property of the Python code that we are
-/// evaluating. We intern these constraints in an arena ([`VisibilityConstraints::constraints`]).
-/// An atom is then an index into this arena.
-///
-/// By using a 32-bit index, we would typically allow 4 billion distinct constraints within a
-/// scope. However, we sometimes have to model how a `Constraint` can have a different runtime
-/// value at different points in the execution of the program. To handle this, we reserve the top
-/// byte of an atom to represent a "copy number". This is just an opaque value that allows
-/// different `Atom`s to evaluate the same `Constraint`. This yields a maximum of 16 million
-/// distinct `Constraint`s in a scope, and 256 possible copies of each of those constraints.
-#[derive(Clone, Copy, Eq, Hash, Ord, PartialEq, PartialOrd)]
-struct Atom(u32);
-
-impl Atom {
-    /// Deconstruct an atom into a constraint index and a copy number.
-    #[inline]
-    fn into_index_and_copy(self) -> (u32, u8) {
-        let copy = self.0 >> 24;
-        let index = self.0 & 0x00ff_ffff;
-        (index, copy as u8)
-    }
-
-    #[inline]
-    fn copy_of(mut self, copy: u8) -> Self {
-        // Clear out the previous copy number
-        self.0 &= 0x00ff_ffff;
-        // OR in the new one
-        self.0 |= u32::from(copy) << 24;
-        self
-    }
-}
-
-// A custom Debug implementation that prints out the constraint index and copy number as distinct
-// fields.
-impl std::fmt::Debug for Atom {
-    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
-        let (index, copy) = self.into_index_and_copy();
-        f.debug_tuple("Atom").field(&index).field(&copy).finish()
-    }
-}
-
-impl Idx for Atom {
-    #[inline]
-    fn new(value: usize) -> Self {
-        assert!(value <= 0x00ff_ffff);
-        #[allow(clippy::cast_possible_truncation)]
-        Self(value as u32)
-    }
-
-    #[inline]
-    fn index(self) -> usize {
-        let (index, _) = self.into_index_and_copy();
-        index as usize
-    }
-}
-
 impl ScopedVisibilityConstraintId {
     /// A special ID that is used for an "always true" / "always visible" constraint.
     pub(crate) const ALWAYS_TRUE: ScopedVisibilityConstraintId =
@@ -336,19 +281,15 @@ const AMBIGUOUS: ScopedVisibilityConstraintId = ScopedVisibilityConstraintId::AM
 const ALWAYS_FALSE: ScopedVisibilityConstraintId = ScopedVisibilityConstraintId::ALWAYS_FALSE;
 const SMALLEST_TERMINAL: ScopedVisibilityConstraintId = ALWAYS_FALSE;
 
-/// A collection of visibility constraints. This is currently stored in `UseDefMap`, which means we
-/// maintain a separate set of visibility constraints for each scope in file.
+/// A collection of visibility constraints for a given scope.
 #[derive(Debug, PartialEq, Eq, salsa::Update)]
-pub(crate) struct VisibilityConstraints<'db> {
-    constraints: IndexVec<Atom, Constraint<'db>>,
+pub(crate) struct VisibilityConstraints {
     interiors: IndexVec<ScopedVisibilityConstraintId, InteriorNode>,
 }
 
 #[derive(Debug, Default, PartialEq, Eq)]
-pub(crate) struct VisibilityConstraintsBuilder<'db> {
-    constraints: IndexVec<Atom, Constraint<'db>>,
+pub(crate) struct VisibilityConstraintsBuilder {
     interiors: IndexVec<ScopedVisibilityConstraintId, InteriorNode>,
-    constraint_cache: FxHashMap<Constraint<'db>, Atom>,
     interior_cache: FxHashMap<InteriorNode, ScopedVisibilityConstraintId>,
     not_cache: FxHashMap<ScopedVisibilityConstraintId, ScopedVisibilityConstraintId>,
     and_cache: FxHashMap<
@@ -361,10 +302,9 @@ pub(crate) struct VisibilityConstraintsBuilder<'db> {
     >,
 }
 
-impl<'db> VisibilityConstraintsBuilder<'db> {
-    pub(crate) fn build(self) -> VisibilityConstraints<'db> {
+impl VisibilityConstraintsBuilder {
+    pub(crate) fn build(self) -> VisibilityConstraints {
         VisibilityConstraints {
-            constraints: self.constraints,
             interiors: self.interiors,
         }
     }
@@ -388,14 +328,6 @@ impl<'db> VisibilityConstraintsBuilder<'db> {
         }
     }
 
-    /// Adds a constraint, ensuring that we only store any particular constraint once.
-    fn add_constraint(&mut self, constraint: Constraint<'db>, copy: u8) -> Atom {
-        self.constraint_cache
-            .entry(constraint)
-            .or_insert_with(|| self.constraints.push(constraint))
-            .copy_of(copy)
-    }
-
     /// Adds an interior node, ensuring that we always use the same visibility constraint ID for
     /// equal nodes.
     fn add_interior(&mut self, node: InteriorNode) -> ScopedVisibilityConstraintId {
@@ -411,17 +343,23 @@ impl<'db> VisibilityConstraintsBuilder<'db> {
             .or_insert_with(|| self.interiors.push(node))
     }
 
-    /// Adds a new visibility constraint that checks a single [`Constraint`]. Provide different
-    /// values for `copy` if you need to model that the constraint can evaluate to different
-    /// results at different points in the execution of the program being modeled.
+    /// Adds a new visibility constraint that checks a single [`Predicate`].
+    ///
+    /// [`ScopedPredicateId`]s are the “variables” that are evaluated by a TDD. A TDD variable has
+    /// the same value no matter how many times it appears in the ternary formula that the TDD
+    /// represents.
+    ///
+    /// However, we sometimes have to model how a `Predicate` can have a different runtime
+    /// value at different points in the execution of the program. To handle this, you can take
+    /// advantage of the fact that the [`Predicates`] arena does not deduplicate `Predicate`s.
+    /// You can add a `Predicate` multiple times, yielding different `ScopedPredicateId`s, which
+    /// you can then create separate TDD atoms for.
     pub(crate) fn add_atom(
         &mut self,
-        constraint: Constraint<'db>,
-        copy: u8,
+        predicate: ScopedPredicateId,
     ) -> ScopedVisibilityConstraintId {
-        let atom = self.add_constraint(constraint, copy);
         self.add_interior(InteriorNode {
-            atom,
+            atom: predicate,
             if_true: ALWAYS_TRUE,
             if_ambiguous: AMBIGUOUS,
             if_false: ALWAYS_FALSE,
@@ -591,11 +529,12 @@ impl<'db> VisibilityConstraintsBuilder<'db> {
     }
 }
 
-impl<'db> VisibilityConstraints<'db> {
+impl VisibilityConstraints {
     /// Analyze the statically known visibility for a given visibility constraint.
-    pub(crate) fn evaluate(
+    pub(crate) fn evaluate<'db>(
         &self,
         db: &'db dyn Db,
+        predicates: &Predicates<'db>,
         mut id: ScopedVisibilityConstraintId,
     ) -> Truthiness {
         loop {
@@ -605,8 +544,8 @@ impl<'db> VisibilityConstraints<'db> {
                 ALWAYS_FALSE => return Truthiness::AlwaysFalse,
                 _ => self.interiors[id],
             };
-            let constraint = &self.constraints[node.atom];
-            match Self::analyze_single(db, constraint) {
+            let predicate = &predicates[node.atom];
+            match Self::analyze_single(db, predicate) {
                 Truthiness::AlwaysTrue => id = node.if_true,
                 Truthiness::Ambiguous => id = node.if_ambiguous,
                 Truthiness::AlwaysFalse => id = node.if_false,
@@ -614,31 +553,17 @@ impl<'db> VisibilityConstraints<'db> {
         }
     }
 
-    fn analyze_single(db: &dyn Db, constraint: &Constraint) -> Truthiness {
-        match constraint.node {
-            ConstraintNode::Expression(test_expr) => {
-                let inference = infer_expression_types(db, test_expr);
-                let scope = test_expr.scope(db);
-                let ty = inference
-                    .expression_type(test_expr.node_ref(db).scoped_expression_id(db, scope));
-
-                ty.bool(db).negate_if(!constraint.is_positive)
+    fn analyze_single(db: &dyn Db, predicate: &Predicate) -> Truthiness {
+        match predicate.node {
+            PredicateNode::Expression(test_expr) => {
+                let ty = infer_expression_type(db, test_expr);
+                ty.bool(db).negate_if(!predicate.is_positive)
             }
-            ConstraintNode::Pattern(inner) => match inner.kind(db) {
-                PatternConstraintKind::Value(value, guard) => {
+            PredicateNode::Pattern(inner) => match inner.kind(db) {
+                PatternPredicateKind::Value(value, guard) => {
                     let subject_expression = inner.subject(db);
-                    let inference = infer_expression_types(db, subject_expression);
-                    let scope = subject_expression.scope(db);
-                    let subject_ty = inference.expression_type(
-                        subject_expression
-                            .node_ref(db)
-                            .scoped_expression_id(db, scope),
-                    );
-
-                    let inference = infer_expression_types(db, *value);
-                    let scope = value.scope(db);
-                    let value_ty = inference
-                        .expression_type(value.node_ref(db).scoped_expression_id(db, scope));
+                    let subject_ty = infer_expression_type(db, subject_expression);
+                    let value_ty = infer_expression_type(db, *value);
 
                     if subject_ty.is_single_valued(db) {
                         let truthiness =
@@ -654,9 +579,9 @@ impl<'db> VisibilityConstraints<'db> {
                         Truthiness::Ambiguous
                     }
                 }
-                PatternConstraintKind::Singleton(..)
-                | PatternConstraintKind::Class(..)
-                | PatternConstraintKind::Unsupported => Truthiness::Ambiguous,
+                PatternPredicateKind::Singleton(..)
+                | PatternPredicateKind::Class(..)
+                | PatternPredicateKind::Unsupported => Truthiness::Ambiguous,
             },
         }
     }