Ruff v0.10 Release (#16708)

Co-authored-by: dylwil3 <dylwil3@gmail.com>
Co-authored-by: Brent Westbrook <36778786+ntBre@users.noreply.github.com>

.github/ISSUE_TEMPLATE/1_bug_report.yaml

@@ -0,0 +1,31 @@
+name: Bug report
+description: Report an error or unexpected behavior
+body:
+  - type: markdown
+    attributes:
+      value: |
+        Thank you for taking the time to report an issue! We're glad to have you involved with Ruff.
+
+        **Before reporting, please make sure to search through [existing issues](https://github.com/astral-sh/ruff/issues?q=is:issue+is:open+label:bug) (including [closed](https://github.com/astral-sh/ruff/issues?q=is:issue%20state:closed%20label:bug)).**
+
+  - type: textarea
+    attributes:
+      label: Summary
+      description: |
+        A clear and concise description of the bug, including a minimal reproducible example.
+
+        Be sure to include the command you invoked (e.g., `ruff check /path/to/file.py --fix`), ideally including the `--isolated` flag and
+        the current Ruff settings (e.g., relevant sections from your `pyproject.toml`).
+
+        If possible, try to include the [playground](https://play.ruff.rs) link that reproduces this issue.
+
+    validations:
+      required: true
+
+  - type: input
+    attributes:
+      label: Version
+      description: What version of ruff are you using? (see `ruff version`)
+      placeholder: e.g., ruff 0.9.3 (90589372d 2025-01-23)
+    validations:
+      required: false

.github/ISSUE_TEMPLATE/2_rule_request.yaml

@@ -0,0 +1,10 @@
+name: Rule request
+description: Anything related to lint rules (proposing new rules, changes to existing rules, auto-fixes, etc.)
+body:
+  - type: textarea
+    attributes:
+      label: Summary
+      description: |
+        A clear and concise description of the relevant request. If applicable, please describe the current behavior as well.
+    validations:
+      required: true

.github/ISSUE_TEMPLATE/3_question.yaml

@@ -0,0 +1,18 @@
+name: Question
+description: Ask a question about Ruff
+labels: ["question"]
+body:
+  - type: textarea
+    attributes:
+      label: Question
+      description: Describe your question in detail.
+    validations:
+      required: true
+
+  - type: input
+    attributes:
+      label: Version
+      description: What version of ruff are you using? (see `ruff version`)
+      placeholder: e.g., ruff 0.9.3 (90589372d 2025-01-23)
+    validations:
+      required: false

.github/ISSUE_TEMPLATE/config.yml

@@ -1,2 +1,8 @@
-# This file cannot use the extension `.yaml`.
-blank_issues_enabled: false
+blank_issues_enabled: true
+contact_links:
+  - name: Documentation
+    url: https://docs.astral.sh/ruff
+    about: Please consult the documentation before creating an issue.
+  - name: Community
+    url: https://discord.com/invite/astral-sh
+    about: Join our Discord community to ask questions and collaborate.

.github/ISSUE_TEMPLATE/issue.yaml

@@ -1,22 +0,0 @@
-name: New issue
-description: A generic issue
-
-body:
-  - type: markdown
-    attributes:
-      value: |
-        Thank you for taking the time to report an issue! We're glad to have you involved with Ruff.
-
-        If you're filing a bug report, please consider including the following information:
-
-        * List of keywords you searched for before creating this issue. Write them down here so that others can find this issue more easily and help provide feedback.
-        e.g. "RUF001", "unused variable", "Jupyter notebook"
-        * A minimal code snippet that reproduces the bug.
-        * The command you invoked (e.g., `ruff /path/to/file.py --fix`), ideally including the `--isolated` flag.
-        * The current Ruff settings (any relevant sections from your `pyproject.toml`).
-        * The current Ruff version (`ruff --version`).
-
-  - type: textarea
-    attributes:
-      label: Description
-      description: A description of the issue

.github/renovate.json5

@@ -58,12 +58,6 @@
       description: "Disable PRs updating GitHub runners (e.g. 'runs-on: macos-14')",
       enabled: false,
     },
-    {
-      // TODO: Remove this once the codebase is upgrade to v4 (https://github.com/astral-sh/ruff/pull/16069)
-      matchPackageNames: ["tailwindcss"],
-      matchManagers: ["npm"],
-      enabled: false,
-    },
     {
       // Disable updates of `zip-rs`; intentionally pinned for now due to ownership change
       // See: https://github.com/astral-sh/uv/issues/3642
@@ -101,14 +95,7 @@
       matchManagers: ["cargo"],
       matchPackageNames: ["strum"],
       description: "Weekly update of strum dependencies",
-    },
-    {
-      groupName: "ESLint",
-      matchManagers: ["npm"],
-      matchPackageNames: ["eslint"],
-      allowedVersions: "<9",
-      description: "Constraint ESLint to version 8 until TypeScript-eslint supports ESLint 9", // https://github.com/typescript-eslint/typescript-eslint/issues/8211
-    },
+    }
   ],
   vulnerabilityAlerts: {
     commitMessageSuffix: "",

.github/workflows/build-docker.yml

@@ -163,7 +163,7 @@ jobs:
         # Mapping of base image followed by a comma followed by one or more base tags (comma separated)
         # Note, org.opencontainers.image.version label will use the first base tag (use the most specific tag first)
         image-mapping:
-          - alpine:3.20,alpine3.20,alpine
+          - alpine:3.21,alpine3.21,alpine
           - debian:bookworm-slim,bookworm-slim,debian-slim
           - buildpack-deps:bookworm,bookworm,debian
     steps:

.github/workflows/daily_property_tests.yaml

@@ -47,6 +47,7 @@ jobs:
         run: |
           export QUICKCHECK_TESTS=100000
           for _ in {1..5}; do
+            cargo test --locked --release --package red_knot_python_semantic -- --ignored list::property_tests
             cargo test --locked --release --package red_knot_python_semantic -- --ignored types::property_tests::stable
           done
 

.github/workflows/mypy_primer.yaml

@@ -0,0 +1,93 @@
+name: Run mypy_primer
+
+permissions: {}
+
+on:
+  pull_request:
+    paths:
+      - "crates/red_knot*/**"
+      - "crates/ruff_db"
+      - "crates/ruff_python_ast"
+      - "crates/ruff_python_parser"
+      - ".github/workflows/mypy_primer.yaml"
+      - ".github/workflows/mypy_primer_comment.yaml"
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref_name }}-${{ github.event.pull_request.number || github.sha }}
+  cancel-in-progress: true
+
+env:
+  CARGO_INCREMENTAL: 0
+  CARGO_NET_RETRY: 10
+  CARGO_TERM_COLOR: always
+  RUSTUP_MAX_RETRIES: 10
+
+jobs:
+  mypy_primer:
+    name: Run mypy_primer
+    runs-on: ubuntu-24.04
+    timeout-minutes: 20
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          path: ruff
+          fetch-depth: 0
+          persist-credentials: false
+
+      - name: Install the latest version of uv
+        uses: astral-sh/setup-uv@v5
+
+      - uses: Swatinem/rust-cache@v2
+        with:
+          workspaces: "ruff"
+      - name: Install Rust toolchain
+        run: rustup show
+
+      - name: Install mypy_primer
+        run: |
+          uv tool install "git+https://github.com/astral-sh/mypy_primer.git@add-red-knot-support"
+
+      - name: Run mypy_primer
+        shell: bash
+        run: |
+          cd ruff
+
+          echo "new commit"
+          git rev-list --format=%s --max-count=1 "$GITHUB_SHA"
+
+          MERGE_BASE="$(git merge-base "$GITHUB_SHA" "origin/$GITHUB_BASE_REF")"
+          git checkout -b base_commit "$MERGE_BASE"
+          echo "base commit"
+          git rev-list --format=%s --max-count=1 base_commit
+
+          cd ..
+
+          # Allow the exit code to be 0 or 1, only fail for actual mypy_primer crashes/bugs
+          uvx mypy_primer \
+            --repo ruff \
+            --type-checker knot \
+            --old base_commit \
+            --new "$GITHUB_SHA" \
+            --project-selector '/(mypy_primer|black|pyp|git-revise|zipp|arrow)$' \
+            --output concise \
+            --debug > mypy_primer.diff || [ $? -eq 1 ]
+
+          # Output diff with ANSI color codes
+          cat mypy_primer.diff
+
+          # Remove ANSI color codes before uploading
+          sed -ie 's/\x1b\[[0-9;]*m//g' mypy_primer.diff
+
+          echo ${{ github.event.number }} > pr-number
+
+      - name: Upload diff
+        uses: actions/upload-artifact@v4
+        with:
+          name: mypy_primer_diff
+          path: mypy_primer.diff
+
+      - name: Upload pr-number
+        uses: actions/upload-artifact@v4
+        with:
+          name: pr-number
+          path: pr-number

.github/workflows/mypy_primer_comment.yaml

@@ -0,0 +1,97 @@
+name: PR comment (mypy_primer)
+
+on: # zizmor: ignore[dangerous-triggers]
+  workflow_run:
+    workflows: [Run mypy_primer]
+    types: [completed]
+  workflow_dispatch:
+    inputs:
+      workflow_run_id:
+        description: The mypy_primer workflow that triggers the workflow run
+        required: true
+
+jobs:
+  comment:
+    runs-on: ubuntu-24.04
+    permissions:
+      pull-requests: write
+    steps:
+      - uses: dawidd6/action-download-artifact@v8
+        name: Download PR number
+        with:
+          name: pr-number
+          run_id: ${{ github.event.workflow_run.id ||  github.event.inputs.workflow_run_id }}
+          if_no_artifact_found: ignore
+          allow_forks: true
+
+      - name: Parse pull request number
+        id: pr-number
+        run: |
+          if [[ -f pr-number ]]
+          then
+            echo "pr-number=$(<pr-number)" >> "$GITHUB_OUTPUT"
+          fi
+
+      - uses: dawidd6/action-download-artifact@v8
+        name: "Download mypy_primer results"
+        id: download-mypy_primer_diff
+        if: steps.pr-number.outputs.pr-number
+        with:
+          name: mypy_primer_diff
+          workflow: mypy_primer.yaml
+          pr: ${{ steps.pr-number.outputs.pr-number }}
+          path: pr/mypy_primer_diff
+          workflow_conclusion: completed
+          if_no_artifact_found: ignore
+          allow_forks: true
+
+      - name: Generate comment content
+        id: generate-comment
+        if: steps.download-mypy_primer_diff.outputs.found_artifact == 'true'
+        run: |
+          # Guard against malicious mypy_primer results that symlink to a secret
+          # file on this runner
+          if [[ -L pr/mypy_primer_diff/mypy_primer.diff ]]
+          then
+              echo "Error: mypy_primer.diff cannot be a symlink"
+              exit 1
+          fi
+
+          # Note this identifier is used to find the comment to update on
+          # subsequent runs
+          echo '<!-- generated-comment mypy_primer -->' >> comment.txt
+
+          echo '## `mypy_primer` results' >> comment.txt
+          if [ -s "pr/mypy_primer_diff/mypy_primer.diff" ]; then
+            echo '<details>' >> comment.txt
+            echo '<summary>Changes were detected when running on open source projects</summary>' >> comment.txt
+            echo '' >> comment.txt
+            echo '```diff' >> comment.txt
+            cat pr/mypy_primer_diff/mypy_primer.diff >> comment.txt
+            echo '```' >> comment.txt
+            echo '</details>' >> comment.txt
+          else
+            echo 'No ecosystem changes detected ✅' >> comment.txt
+          fi
+
+          echo 'comment<<EOF' >> "$GITHUB_OUTPUT"
+          cat comment.txt >> "$GITHUB_OUTPUT"
+          echo 'EOF' >> "$GITHUB_OUTPUT"
+
+      - name: Find existing comment
+        uses: peter-evans/find-comment@v3
+        if: steps.generate-comment.outcome == 'success'
+        id: find-comment
+        with:
+          issue-number: ${{ steps.pr-number.outputs.pr-number }}
+          comment-author: "github-actions[bot]"
+          body-includes: "<!-- generated-comment mypy_primer -->"
+
+      - name: Create or update comment
+        if: steps.find-comment.outcome == 'success'
+        uses: peter-evans/create-or-update-comment@v4
+        with:
+          comment-id: ${{ steps.find-comment.outputs.comment-id }}
+          issue-number: ${{ steps.pr-number.outputs.pr-number }}
+          body-path: comment.txt
+          edit-mode: replace

.pre-commit-config.yaml

@@ -60,7 +60,7 @@ repos:
           - black==25.1.0
 
   - repo: https://github.com/crate-ci/typos
-    rev: v1.29.7
+    rev: v1.30.0
     hooks:
       - id: typos
 
@@ -74,7 +74,7 @@ repos:
         pass_filenames: false # This makes it a lot faster
 
   - repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: v0.9.6
+    rev: v0.9.9
     hooks:
       - id: ruff-format
       - id: ruff
@@ -84,7 +84,7 @@ repos:
 
   # Prettier
   - repo: https://github.com/rbubley/mirrors-prettier
-    rev: v3.5.1
+    rev: v3.5.2
     hooks:
       - id: prettier
         types: [yaml]
@@ -92,12 +92,12 @@ repos:
   # zizmor detects security vulnerabilities in GitHub Actions workflows.
   # Additional configuration for the tool is found in `.github/zizmor.yml`
   - repo: https://github.com/woodruffw/zizmor-pre-commit
-    rev: v1.3.1
+    rev: v1.4.1
     hooks:
       - id: zizmor
 
   - repo: https://github.com/python-jsonschema/check-jsonschema
-    rev: 0.31.1
+    rev: 0.31.2
     hooks:
       - id: check-github-workflows
 

BREAKING_CHANGES.md

@@ -1,5 +1,46 @@
 # Breaking Changes
 
+## 0.10.0
+
+- **Changes to how the Python version is inferred when a `target-version` is not specified** ([#16319](https://github.com/astral-sh/ruff/pull/16319))
+
+    In previous versions of Ruff, you could specify your Python version with:
+
+    - The `target-version` option in a `ruff.toml` file or the `[tool.ruff]` section of a pyproject.toml file.
+    - The `project.requires-python` field in a `pyproject.toml` file with a `[tool.ruff]` section.
+
+    These options worked well in most cases, and are still recommended for fine control of the Python version. However, because of the way Ruff discovers config files, `pyproject.toml` files without a `[tool.ruff]` section would be ignored, including the `requires-python` setting. Ruff would then use the default Python version (3.9 as of this writing) instead, which is surprising when you've attempted to request another version.
+
+    In v0.10, config discovery has been updated to address this issue:
+
+    - If Ruff finds a `ruff.toml` file without a `target-version`, it will check
+        for a `pyproject.toml` file in the same directory and respect its
+        `requires-python` version, even if it does not contain a `[tool.ruff]`
+        section.
+    - If Ruff finds a user-level configuration, the `requires-python` field of the closest `pyproject.toml` in a parent directory will take precedence.
+    - If there is no config file (`ruff.toml`or `pyproject.toml` with a
+        `[tool.ruff]` section) in the directory of the file being checked, Ruff will
+        search for the closest `pyproject.toml` in the parent directories and use its
+        `requires-python` setting.
+
+- **Updated `TYPE_CHECKING` behavior** ([#16669](https://github.com/astral-sh/ruff/pull/16669))
+
+    Previously, Ruff only recognized typechecking blocks that tested the `typing.TYPE_CHECKING` symbol. Now, Ruff recognizes any local variable named `TYPE_CHECKING`. This release also removes support for the legacy `if 0:` and `if False:` typechecking checks. Use a local `TYPE_CHECKING` variable instead.
+
+- **More robust noqa parsing** ([#16483](https://github.com/astral-sh/ruff/pull/16483))
+
+    The syntax for both file-level and in-line suppression comments has been unified and made more robust to certain errors. In most cases, this will result in more suppression comments being read by Ruff, but there are a few instances where previously read comments will now log an error to the user instead. Please refer to the documentation on [_Error suppression_](https://docs.astral.sh/ruff/linter/#error-suppression) for the full specification.
+
+- **Avoid unnecessary parentheses around with statements with a single context manager and a trailing comment** ([#14005](https://github.com/astral-sh/ruff/pull/14005))
+
+    This change fixes a bug in the formatter where it introduced unnecessary parentheses around with statements with a single context manager and a trailing comment. This change may result in a change in formatting for some users.
+
+- **Bump alpine default tag to 3.21 for derived Docker images** ([#16456](https://github.com/astral-sh/ruff/pull/16456))
+
+    Alpine 3.21 was released in Dec 2024 and is used in the official Alpine-based Python images. Now the ruff:alpine image will use 3.21 instead of 3.20 and ruff:alpine3.20 will no longer be updated.
+
+- **\[`unsafe-markup-use`\]: `RUF035` has been recoded to `S704`** ([#15957](https://github.com/astral-sh/ruff/pull/15957))
+
 ## 0.9.0
 
 Ruff now formats your code according to the 2025 style guide. As a result, your code might now get formatted differently. See the [changelog](./CHANGELOG.md#090) for a detailed list of changes.

CHANGELOG.md

@@ -1,5 +1,200 @@
 # Changelog
 
+## 0.10.0
+
+Check out the [blog post](https://astral.sh/blog/ruff-v0.10.0) for a migration guide and overview of the changes!
+
+### Breaking changes
+
+See also, the "Remapped rules" section which may result in disabled rules.
+
+- **Changes to how the Python version is inferred when a `target-version` is not specified** ([#16319](https://github.com/astral-sh/ruff/pull/16319))
+
+    In previous versions of Ruff, you could specify your Python version with:
+
+    - The `target-version` option in a `ruff.toml` file or the `[tool.ruff]` section of a pyproject.toml file.
+    - The `project.requires-python` field in a `pyproject.toml` file with a `[tool.ruff]` section.
+
+    These options worked well in most cases, and are still recommended for fine control of the Python version. However, because of the way Ruff discovers config files, `pyproject.toml` files without a `[tool.ruff]` section would be ignored, including the `requires-python` setting. Ruff would then use the default Python version (3.9 as of this writing) instead, which is surprising when you've attempted to request another version.
+
+    In v0.10, config discovery has been updated to address this issue:
+
+    - If Ruff finds a `ruff.toml` file without a `target-version`, it will check
+        for a `pyproject.toml` file in the same directory and respect its
+        `requires-python` version, even if it does not contain a `[tool.ruff]`
+        section.
+    - If Ruff finds a user-level configuration, the `requires-python` field of the closest `pyproject.toml` in a parent directory will take precedence.
+    - If there is no config file (`ruff.toml`or `pyproject.toml` with a
+        `[tool.ruff]` section) in the directory of the file being checked, Ruff will
+        search for the closest `pyproject.toml` in the parent directories and use its
+        `requires-python` setting.
+
+- **Updated `TYPE_CHECKING` behavior** ([#16669](https://github.com/astral-sh/ruff/pull/16669))
+
+    Previously, Ruff only recognized typechecking blocks that tested the `typing.TYPE_CHECKING` symbol. Now, Ruff recognizes any local variable named `TYPE_CHECKING`. This release also removes support for the legacy `if 0:` and `if False:` typechecking checks. Use a local `TYPE_CHECKING` variable instead.
+
+- **More robust noqa parsing** ([#16483](https://github.com/astral-sh/ruff/pull/16483))
+
+    The syntax for both file-level and in-line suppression comments has been unified and made more robust to certain errors. In most cases, this will result in more suppression comments being read by Ruff, but there are a few instances where previously read comments will now log an error to the user instead. Please refer to the documentation on [_Error suppression_](https://docs.astral.sh/ruff/linter/#error-suppression) for the full specification.
+
+- **Avoid unnecessary parentheses around with statements with a single context manager and a trailing comment** ([#14005](https://github.com/astral-sh/ruff/pull/14005))
+
+    This change fixes a bug in the formatter where it introduced unnecessary parentheses around with statements with a single context manager and a trailing comment. This change may result in a change in formatting for some users.
+
+- **Bump alpine default tag to 3.21 for derived Docker images** ([#16456](https://github.com/astral-sh/ruff/pull/16456))
+
+    Alpine 3.21 was released in Dec 2024 and is used in the official Alpine-based Python images. Now the ruff:alpine image will use 3.21 instead of 3.20 and ruff:alpine3.20 will no longer be updated.
+
+### Deprecated Rules
+
+The following rules have been deprecated:
+
+- [`non-pep604-isinstance`](https://docs.astral.sh/ruff/rules/non-pep604-isinstance/) (`UP038`)
+- [`suspicious-xmle-tree-usage`](https://docs.astral.sh/ruff/rules/suspicious-xmle-tree-usage/) (`S320`)
+
+### Remapped rules
+
+The following rules have been remapped to new rule codes:
+
+- \[`unsafe-markup-use`\]: `RUF035` to `S704`
+
+### Stabilization
+
+The following rules have been stabilized and are no longer in preview:
+
+- [`batched-without-explicit-strict`](https://docs.astral.sh/ruff/rules/batched-without-explicit-strict) (`B911`)
+- [`unnecessary-dict-comprehension-for-iterable`](https://docs.astral.sh/ruff/rules/unnecessary-dict-comprehension-for-iterable) (`C420`)
+- [`datetime-min-max`](https://docs.astral.sh/ruff/rules/datetime-min-max) (`DTZ901`)
+- [`fast-api-unused-path-parameter`](https://docs.astral.sh/ruff/rules/fast-api-unused-path-parameter) (`FAST003`)
+- [`root-logger-call`](https://docs.astral.sh/ruff/rules/root-logger-call) (`LOG015`)
+- [`len-test`](https://docs.astral.sh/ruff/rules/len-test) (`PLC1802`)
+- [`shallow-copy-environ`](https://docs.astral.sh/ruff/rules/shallow-copy-environ) (`PLW1507`)
+- [`os-listdir`](https://docs.astral.sh/ruff/rules/os-listdir) (`PTH208`)
+- [`invalid-pathlib-with-suffix`](https://docs.astral.sh/ruff/rules/invalid-pathlib-with-suffix) (`PTH210`)
+- [`invalid-assert-message-literal-argument`](https://docs.astral.sh/ruff/rules/invalid-assert-message-literal-argument) (`RUF040`)
+- [`unnecessary-nested-literal`](https://docs.astral.sh/ruff/rules/unnecessary-nested-literal) (`RUF041`)
+- [`unnecessary-cast-to-int`](https://docs.astral.sh/ruff/rules/unnecessary-cast-to-int) (`RUF046`)
+- [`map-int-version-parsing`](https://docs.astral.sh/ruff/rules/map-int-version-parsing) (`RUF048`)
+- [`if-key-in-dict-del`](https://docs.astral.sh/ruff/rules/if-key-in-dict-del) (`RUF051`)
+- [`unsafe-markup-use`](https://docs.astral.sh/ruff/rules/unsafe-markup-use) (`S704`). This rule has also been renamed from `RUF035`.
+- [`split-static-string`](https://docs.astral.sh/ruff/rules/split-static-string) (`SIM905`)
+- [`runtime-cast-value`](https://docs.astral.sh/ruff/rules/runtime-cast-value) (`TC006`)
+- [`unquoted-type-alias`](https://docs.astral.sh/ruff/rules/unquoted-type-alias) (`TC007`)
+- [`non-pep646-unpack`](https://docs.astral.sh/ruff/rules/non-pep646-unpack) (`UP044`)
+
+The following behaviors have been stabilized:
+
+- [`bad-staticmethod-argument`](https://docs.astral.sh/ruff/rules/bad-staticmethod-argument/) (`PLW0211`) [`invalid-first-argument-name-for-class-method`](https://docs.astral.sh/ruff/rules/invalid-first-argument-name-for-class-method/) (`N804`): `__new__` methods are now no longer flagged by `invalid-first-argument-name-for-class-method` (`N804`) but instead by `bad-staticmethod-argument` (`PLW0211`)
+- [`bad-str-strip-call`](https://docs.astral.sh/ruff/rules/bad-str-strip-call/) (`PLE1310`): The rule now applies to objects which are known to have type `str` or `bytes`.
+- [`blanket-noqa`](https://docs.astral.sh/ruff/rules/blanket-noqa/) (`PGH004`): Also detect blanked file-level noqa comments (and not just line level comments).
+- [`custom-type-var-for-self`](https://docs.astral.sh/ruff/rules/custom-type-var-for-self/) (`PYI019`): More accurate detection of custom `TypeVars` replaceable by `Self`. The range of the diagnostic is now the full function header rather than just the return annotation.
+- [`invalid-argument-name`](https://docs.astral.sh/ruff/rules/invalid-argument-name/) (`N803`): Ignore argument names of functions decorated with `typing.override`
+- [`invalid-envvar-default`](https://docs.astral.sh/ruff/rules/invalid-envvar-default/) (`PLW1508`): Detect default value arguments to `os.environ.get` with invalid type.
+- [`pytest-raises-with-multiple-statements`](https://docs.astral.sh/ruff/rules/pytest-raises-with-multiple-statements/) (`PT012`) [`pytest-warns-with-multiple-statements`](https://docs.astral.sh/ruff/rules/pytest-warns-with-multiple-statements/) (`PT031`): Allow `for` statements with an empty body in `pytest.raises` and `pytest.warns` `with` statements.
+- [`redundant-open-modes`](https://docs.astral.sh/ruff/rules/redundant-open-modes/) (`UP015`): The diagnostic range is now the range of the redundant mode argument where it previously was the range of the entire open call. You may have to replace your `noqa` comments when suppressing `UP015`.
+- [`stdlib-module-shadowing`](https://docs.astral.sh/ruff/rules/stdlib-module-shadowing/) (`A005`): Changes the default value of `lint.flake8-builtins.strict-checking` from `true` to `false`.
+- [`type-none-comparison`](https://docs.astral.sh/ruff/rules/type-none-comparison/) (`FURB169`): Now also recognizes `type(expr) is type(None)` comparisons where `expr` isn't a name expression.
+
+The following fixes or improvements to fixes have been stabilized:
+
+- [`repeated-equality-comparison`](https://docs.astral.sh/ruff/rules/repeated-equality-comparison/) (`PLR1714`) ([#16685](https://github.com/astral-sh/ruff/pull/16685))
+- [`needless-bool`](https://docs.astral.sh/ruff/rules/needless-bool/) (`SIM103`) ([#16684](https://github.com/astral-sh/ruff/pull/16684))
+- [`unused-private-type-var`](https://docs.astral.sh/ruff/rules/unused-private-type-var/) (`PYI018`) ([#16682](https://github.com/astral-sh/ruff/pull/16682))
+
+### Server
+
+- Remove logging output for `ruff.printDebugInformation` ([#16617](https://github.com/astral-sh/ruff/pull/16617))
+
+### Configuration
+
+- \[`flake8-builtins`\] Deprecate the `builtins-` prefixed options in favor of the unprefixed options (e.g. `builtins-allowed-modules` is now deprecated in favor of `allowed-modules`) ([#16092](https://github.com/astral-sh/ruff/pull/16092))
+
+### Bug fixes
+
+- [flake8-bandit] Fix mixed-case hash algorithm names (S324) ([#16552](https://github.com/astral-sh/ruff/pull/16552))
+
+### CLI
+
+- [ruff] Fix `last_tag`/`commits_since_last_tag` for `version` command ([#16686](https://github.com/astral-sh/ruff/pull/16686))
+
+## 0.9.10
+
+### Preview features
+
+- \[`ruff`\] Add new rule `RUF059`: Unused unpacked assignment ([#16449](https://github.com/astral-sh/ruff/pull/16449))
+- \[`syntax-errors`\] Detect assignment expressions before Python 3.8 ([#16383](https://github.com/astral-sh/ruff/pull/16383))
+- \[`syntax-errors`\] Named expressions in decorators before Python 3.9 ([#16386](https://github.com/astral-sh/ruff/pull/16386))
+- \[`syntax-errors`\] Parenthesized keyword argument names after Python 3.8 ([#16482](https://github.com/astral-sh/ruff/pull/16482))
+- \[`syntax-errors`\] Positional-only parameters before Python 3.8 ([#16481](https://github.com/astral-sh/ruff/pull/16481))
+- \[`syntax-errors`\] Tuple unpacking in `return` and `yield` before Python 3.8 ([#16485](https://github.com/astral-sh/ruff/pull/16485))
+- \[`syntax-errors`\] Type parameter defaults before Python 3.13 ([#16447](https://github.com/astral-sh/ruff/pull/16447))
+- \[`syntax-errors`\] Type parameter lists before Python 3.12 ([#16479](https://github.com/astral-sh/ruff/pull/16479))
+- \[`syntax-errors`\] `except*` before Python 3.11 ([#16446](https://github.com/astral-sh/ruff/pull/16446))
+- \[`syntax-errors`\] `type` statements before Python 3.12 ([#16478](https://github.com/astral-sh/ruff/pull/16478))
+
+### Bug fixes
+
+- Escape template filenames in glob patterns in configuration ([#16407](https://github.com/astral-sh/ruff/pull/16407))
+- \[`flake8-simplify`\] Exempt unittest context methods for `SIM115` rule ([#16439](https://github.com/astral-sh/ruff/pull/16439))
+- Formatter: Fix syntax error location in notebooks ([#16499](https://github.com/astral-sh/ruff/pull/16499))
+- \[`pyupgrade`\] Do not offer fix when at least one target is `global`/`nonlocal` (`UP028`) ([#16451](https://github.com/astral-sh/ruff/pull/16451))
+- \[`flake8-builtins`\] Ignore variables matching module attribute names (`A001`) ([#16454](https://github.com/astral-sh/ruff/pull/16454))
+- \[`pylint`\] Convert `code` keyword argument to a positional argument in fix for (`PLR1722`) ([#16424](https://github.com/astral-sh/ruff/pull/16424))
+
+### CLI
+
+- Move rule code from `description` to `check_name` in GitLab output serializer ([#16437](https://github.com/astral-sh/ruff/pull/16437))
+
+### Documentation
+
+- \[`pydocstyle`\] Clarify that `D417` only checks docstrings with an arguments section ([#16494](https://github.com/astral-sh/ruff/pull/16494))
+
+## 0.9.9
+
+### Preview features
+
+- Fix caching of unsupported-syntax errors ([#16425](https://github.com/astral-sh/ruff/pull/16425))
+
+### Bug fixes
+
+- Only show unsupported-syntax errors in editors when preview mode is enabled ([#16429](https://github.com/astral-sh/ruff/pull/16429))
+
+## 0.9.8
+
+### Preview features
+
+- Start detecting version-related syntax errors in the parser ([#16090](https://github.com/astral-sh/ruff/pull/16090))
+
+### Rule changes
+
+- \[`pylint`\] Mark fix unsafe (`PLW1507`) ([#16343](https://github.com/astral-sh/ruff/pull/16343))
+- \[`pylint`\] Catch `case np.nan`/`case math.nan` in `match` statements (`PLW0177`) ([#16378](https://github.com/astral-sh/ruff/pull/16378))
+- \[`ruff`\] Add more Pydantic models variants to the list of default copy semantics (`RUF012`) ([#16291](https://github.com/astral-sh/ruff/pull/16291))
+
+### Server
+
+- Avoid indexing the project if `configurationPreference` is `editorOnly` ([#16381](https://github.com/astral-sh/ruff/pull/16381))
+- Avoid unnecessary info at non-trace server log level ([#16389](https://github.com/astral-sh/ruff/pull/16389))
+- Expand `ruff.configuration` to allow inline config ([#16296](https://github.com/astral-sh/ruff/pull/16296))
+- Notify users for invalid client settings ([#16361](https://github.com/astral-sh/ruff/pull/16361))
+
+### Configuration
+
+- Add `per-file-target-version` option ([#16257](https://github.com/astral-sh/ruff/pull/16257))
+
+### Bug fixes
+
+- \[`refurb`\] Do not consider docstring(s) (`FURB156`) ([#16391](https://github.com/astral-sh/ruff/pull/16391))
+- \[`flake8-self`\] Ignore attribute accesses on instance-like variables (`SLF001`) ([#16149](https://github.com/astral-sh/ruff/pull/16149))
+- \[`pylint`\] Fix false positives, add missing methods, and support positional-only parameters (`PLE0302`) ([#16263](https://github.com/astral-sh/ruff/pull/16263))
+- \[`flake8-pyi`\] Mark `PYI030` fix unsafe when comments are deleted ([#16322](https://github.com/astral-sh/ruff/pull/16322))
+
+### Documentation
+
+- Fix example for `S611` ([#16316](https://github.com/astral-sh/ruff/pull/16316))
+- Normalize inconsistent markdown headings in docstrings ([#16364](https://github.com/astral-sh/ruff/pull/16364))
+- Document MSRV policy ([#16384](https://github.com/astral-sh/ruff/pull/16384))
+
 ## 0.9.7
 
 ### Preview features
@@ -13,16 +208,7 @@
 
 ### Rule changes
 
-- \[`flake8-comprehensions`\]: Handle trailing comma in `C403` fix ([#16110](https://github.com/astral-sh/ruff/pull/16110))
 - \[`flake8-debugger`\] Also flag `sys.breakpointhook` and `sys.__breakpointhook__` (`T100`) ([#16191](https://github.com/astral-sh/ruff/pull/16191))
-- \[`pydocstyle`\] Handle arguments with the same names as sections (`D417`) ([#16011](https://github.com/astral-sh/ruff/pull/16011))
-- \[`pylint`\] Correct ordering of arguments in fix for `if-stmt-min-max` (`PLR1730`) ([#16080](https://github.com/astral-sh/ruff/pull/16080))
-- \[`pylint`\] Do not offer fix for raw strings (`PLE251`) ([#16132](https://github.com/astral-sh/ruff/pull/16132))
-- \[`pyupgrade`\] Do not upgrade functional `TypedDicts` with private field names to the class-based syntax (`UP013`) ([#16219](https://github.com/astral-sh/ruff/pull/16219))
-- \[`pyupgrade`\] Handle micro version numbers correctly (`UP036`) ([#16091](https://github.com/astral-sh/ruff/pull/16091))
-- \[`pyupgrade`\] Unwrap unary expressions correctly (`UP018`) ([#15919](https://github.com/astral-sh/ruff/pull/15919))
-- \[`ruff`\] Skip `RUF001` diagnostics when visiting string type definitions ([#16122](https://github.com/astral-sh/ruff/pull/16122))
-- \[`flake8-pyi`\] Avoid flagging `custom-typevar-for-self` on metaclass methods (`PYI019`) ([#16141](https://github.com/astral-sh/ruff/pull/16141))
 - \[`pycodestyle`\] Exempt `site.addsitedir(...)` calls (`E402`) ([#16251](https://github.com/astral-sh/ruff/pull/16251))
 
 ### Formatter
@@ -43,7 +229,16 @@
 
 ### Bug fixes
 
+- \[`flake8-comprehensions`\] Handle trailing comma in `C403` fix ([#16110](https://github.com/astral-sh/ruff/pull/16110))
+- \[`flake8-pyi`\] Avoid flagging `custom-typevar-for-self` on metaclass methods (`PYI019`) ([#16141](https://github.com/astral-sh/ruff/pull/16141))
+- \[`pydocstyle`\] Handle arguments with the same names as sections (`D417`) ([#16011](https://github.com/astral-sh/ruff/pull/16011))
+- \[`pylint`\] Correct ordering of arguments in fix for `if-stmt-min-max` (`PLR1730`) ([#16080](https://github.com/astral-sh/ruff/pull/16080))
+- \[`pylint`\] Do not offer fix for raw strings (`PLE251`) ([#16132](https://github.com/astral-sh/ruff/pull/16132))
+- \[`pyupgrade`\] Do not upgrade functional `TypedDicts` with private field names to the class-based syntax (`UP013`) ([#16219](https://github.com/astral-sh/ruff/pull/16219))
+- \[`pyupgrade`\] Handle micro version numbers correctly (`UP036`) ([#16091](https://github.com/astral-sh/ruff/pull/16091))
+- \[`pyupgrade`\] Unwrap unary expressions correctly (`UP018`) ([#15919](https://github.com/astral-sh/ruff/pull/15919))
 - \[`refurb`\] Correctly handle lengths of literal strings in `slice-to-remove-prefix-or-suffix` (`FURB188`) ([#16237](https://github.com/astral-sh/ruff/pull/16237))
+- \[`ruff`\] Skip `RUF001` diagnostics when visiting string type definitions ([#16122](https://github.com/astral-sh/ruff/pull/16122))
 
 ### Documentation
 
@@ -1253,7 +1448,7 @@ The following fixes have been stabilized:
 
 ## 0.5.6
 
-Ruff 0.5.6 automatically enables linting and formatting of notebooks in *preview mode*.
+Ruff 0.5.6 automatically enables linting and formatting of notebooks in _preview mode_.
 You can opt-out of this behavior by adding `*.ipynb` to the `extend-exclude` setting.
 
 ```toml
@@ -2006,7 +2201,7 @@ To setup `ruff server` with your editor, refer to the [README.md](https://github
 
 ### Server
 
-*This section is devoted to updates for our new language server, written in Rust.*
+_This section is devoted to updates for our new language server, written in Rust._
 
 - Enable ruff-specific source actions ([#10916](https://github.com/astral-sh/ruff/pull/10916))
 - Refreshes diagnostics for open files when file configuration is changed ([#10988](https://github.com/astral-sh/ruff/pull/10988))
@@ -3413,7 +3608,7 @@ Read Ruff's new [versioning policy](https://docs.astral.sh/ruff/versioning/).
 - \[`refurb`\] Add `single-item-membership-test` (`FURB171`) ([#7815](https://github.com/astral-sh/ruff/pull/7815))
 - \[`pylint`\] Add `and-or-ternary` (`R1706`) ([#7811](https://github.com/astral-sh/ruff/pull/7811))
 
-*New rules are added in [preview](https://docs.astral.sh/ruff/preview/).*
+_New rules are added in [preview](https://docs.astral.sh/ruff/preview/)._
 
 ### Configuration
 

Cargo.toml

@@ -4,7 +4,7 @@ resolver = "2"
 
 [workspace.package]
 edition = "2021"
-rust-version = "1.80"
+rust-version = "1.83"
 homepage = "https://docs.astral.sh/ruff"
 documentation = "https://docs.astral.sh/ruff"
 repository = "https://github.com/astral-sh/ruff"
@@ -123,7 +123,7 @@ rayon = { version = "1.10.0" }
 regex = { version = "1.10.2" }
 rustc-hash = { version = "2.0.0" }
 # When updating salsa, make sure to also update the revision in `fuzz/Cargo.toml`
-salsa = { git = "https://github.com/salsa-rs/salsa.git", rev = "351d9cf0037be949d17800d0c7b4838e533c2ed6" }
+salsa = { git = "https://github.com/salsa-rs/salsa.git", rev = "095d8b2b8115c3cf8bf31914dd9ea74648bb7cf9" }
 schemars = { version = "0.8.16" }
 seahash = { version = "4.1.0" }
 serde = { version = "1.0.197", features = ["derive"] }

README.md

@@ -149,8 +149,8 @@ curl -LsSf https://astral.sh/ruff/install.sh | sh
 powershell -c "irm https://astral.sh/ruff/install.ps1 | iex"
 
 # For a specific version.
-curl -LsSf https://astral.sh/ruff/0.9.7/install.sh | sh
-powershell -c "irm https://astral.sh/ruff/0.9.7/install.ps1 | iex"
+curl -LsSf https://astral.sh/ruff/0.10.0/install.sh | sh
+powershell -c "irm https://astral.sh/ruff/0.10.0/install.ps1 | iex"
 ```
 
 You can also install Ruff via [Homebrew](https://formulae.brew.sh/formula/ruff), [Conda](https://anaconda.org/conda-forge/ruff),
@@ -183,7 +183,7 @@ Ruff can also be used as a [pre-commit](https://pre-commit.com/) hook via [`ruff
 ```yaml
 - repo: https://github.com/astral-sh/ruff-pre-commit
   # Ruff version.
-  rev: v0.9.7
+  rev: v0.10.0
   hooks:
     # Run the linter.
     - id: ruff

_typos.toml

@@ -23,6 +23,10 @@ extend-ignore-re = [
     # Line ignore with trailing "spellchecker:disable-line"
     "(?Rm)^.*#\\s*spellchecker:disable-line$",
     "LICENSEs",
+    # Various third party dependencies uses `typ` as struct field names (e.g., lsp_types::LogMessageParams)
+    "typ",
+    # TODO: Remove this once the `TYP` redirects are removed from `rule_redirects.rs`
+    "TYP",
 ]
 
 [default.extend-identifiers]

crates/red_knot/Cargo.toml

@@ -19,6 +19,7 @@ ruff_db = { workspace = true, features = ["os", "cache"] }
 ruff_python_ast = { workspace = true }
 
 anyhow = { workspace = true }
+argfile = { workspace = true }
 chrono = { workspace = true }
 clap = { workspace = true, features = ["wrap_help"] }
 colored = { workspace = true }
@@ -31,6 +32,7 @@ tracing = { workspace = true, features = ["release_max_level_debug"] }
 tracing-subscriber = { workspace = true, features = ["env-filter", "fmt"] }
 tracing-flame = { workspace = true }
 tracing-tree = { workspace = true }
+wild = { workspace = true }
 
 [dev-dependencies]
 ruff_db = { workspace = true, features = ["testing"] }

crates/red_knot/docs/mypy_primer.md

@@ -0,0 +1,61 @@
+# Running `mypy_primer`
+
+## Basics
+
+For now, we use our own [fork of mypy primer]. It can be run using `uvx --from "…" mypy_primer`. For example, to see the help message, run:
+
+```sh
+uvx --from "git+https://github.com/astral-sh/mypy_primer.git@add-red-knot-support" mypy_primer -h
+```
+
+Alternatively, you can install the forked version of `mypy_primer` using:
+
+```sh
+uv tool install "git+https://github.com/astral-sh/mypy_primer.git@add-red-knot-support"
+```
+
+and then run it using `uvx mypy_primer` or just `mypy_primer`, if your `PATH` is set up accordingly (see: [Tool executables]).
+
+## Showing the diagnostics diff between two Git revisions
+
+To show the diagnostics diff between two Git revisions (e.g. your feature branch and `main`), run:
+
+```sh
+mypy_primer \
+    --type-checker knot \
+    --old origin/main \
+    --new my/feature \
+    --debug \
+    --output concise \
+    --project-selector '/black$'
+```
+
+This will show the diagnostics diff for the `black` project between the `main` branch and your `my/feature` branch. To run the
+diff for all projects, you currently need to copy the project-selector regex from the CI pipeline in `.github/workflows/mypy_primer.yaml`.
+
+You can also take a look at the [full list of ecosystem projects]. Note that some of them might still need a `knot_paths` configuration
+option to work correctly.
+
+## Avoiding recompilation
+
+If you want to run `mypy_primer` repeatedly, e.g. for different projects, but for the same combination of `--old` and `--new`, you
+can use set the `MYPY_PRIMER_NO_REBUILD` environment variable to avoid recompilation of Red Knot:
+
+```sh
+MYPY_PRIMER_NO_REBUILD=1 mypy_primer …
+```
+
+## Running from a local copy of the repository
+
+If you are working on a local branch, you can use `mypy_primer`'s `--repo` option to specify the path to your local copy of the `ruff` repository.
+This allows `mypy_primer` to check out local branches:
+
+```sh
+mypy_primer --repo /path/to/ruff --old origin/main --new my/local-branch …
+```
+
+Note that you might need to clean up `/tmp/mypy_primer` in order for this to work correctly.
+
+[fork of mypy primer]: https://github.com/astral-sh/mypy_primer/tree/add-red-knot-support
+[full list of ecosystem projects]: https://github.com/astral-sh/mypy_primer/blob/add-red-knot-support/mypy_primer/projects.py
+[tool executables]: https://docs.astral.sh/uv/concepts/tools/#tool-executables

crates/red_knot/src/args.rs

@@ -32,6 +32,13 @@ pub(crate) enum Command {
 
 #[derive(Debug, Parser)]
 pub(crate) struct CheckCommand {
+    /// List of files or directories to check.
+    #[clap(
+        help = "List of files or directories to check [default: the project root]",
+        value_name = "PATH"
+    )]
+    pub paths: Vec<SystemPathBuf>,
+
     /// Run the command within the given project directory.
     ///
     /// All `pyproject.toml` files will be discovered by walking up the directory tree from the given project directory,
@@ -41,12 +48,14 @@ pub(crate) struct CheckCommand {
     #[arg(long, value_name = "PROJECT")]
     pub(crate) project: Option<SystemPathBuf>,
 
-    /// Path to the virtual environment the project uses.
+    /// Path to the Python installation from which Red Knot resolves type information and third-party dependencies.
     ///
-    /// If provided, red-knot will use the `site-packages` directory of this virtual environment
-    /// to resolve type information for the project's third-party dependencies.
+    /// Red Knot will search in the path's `site-packages` directories for type information and
+    /// third-party imports.
+    ///
+    /// This option is commonly used to specify the path to a virtual environment.
     #[arg(long, value_name = "PATH")]
-    pub(crate) venv_path: Option<SystemPathBuf>,
+    pub(crate) python: Option<SystemPathBuf>,
 
     /// Custom directory to use for stdlib typeshed stubs.
     #[arg(long, value_name = "PATH", alias = "custom-typeshed-dir")]
@@ -74,7 +83,7 @@ pub(crate) struct CheckCommand {
     #[arg(long)]
     pub(crate) exit_zero: bool,
 
-    /// Run in watch mode by re-running whenever files change.
+    /// Watch files for changes and recheck files related to the changed files.
     #[arg(long, short = 'W')]
     pub(crate) watch: bool,
 }
@@ -97,7 +106,7 @@ impl CheckCommand {
                 python_version: self
                     .python_version
                     .map(|version| RangedValue::cli(version.into())),
-                venv_path: self.venv_path.map(RelativePathBuf::cli),
+                python: self.python.map(RelativePathBuf::cli),
                 typeshed: self.typeshed.map(RelativePathBuf::cli),
                 extra_paths: self.extra_search_path.map(|extra_search_paths| {
                     extra_search_paths

crates/red_knot/src/main.rs

@@ -1,4 +1,4 @@
-use std::io::{self, BufWriter, Write};
+use std::io::{self, stdout, BufWriter, Write};
 use std::process::{ExitCode, Termination};
 
 use anyhow::Result;
@@ -15,8 +15,8 @@ use red_knot_project::watch::ProjectWatcher;
 use red_knot_project::{watch, Db};
 use red_knot_project::{ProjectDatabase, ProjectMetadata};
 use red_knot_server::run_server;
-use ruff_db::diagnostic::{Diagnostic, DisplayDiagnosticConfig, Severity};
-use ruff_db::system::{OsSystem, System, SystemPath, SystemPathBuf};
+use ruff_db::diagnostic::{DisplayDiagnosticConfig, OldDiagnosticTrait, Severity};
+use ruff_db::system::{OsSystem, SystemPath, SystemPathBuf};
 use salsa::plumbing::ZalsaDatabase;
 
 mod args;
@@ -39,6 +39,15 @@ pub fn main() -> ExitStatus {
         // the configuration it is help to chain errors ("resolving configuration failed" ->
         // "failed to read file: subdir/pyproject.toml")
         for cause in error.chain() {
+            // Exit "gracefully" on broken pipe errors.
+            //
+            // See: https://github.com/BurntSushi/ripgrep/blob/bf63fe8f258afc09bae6caa48f0ae35eaf115005/crates/core/main.rs#L47C1-L61C14
+            if let Some(ioerr) = cause.downcast_ref::<io::Error>() {
+                if ioerr.kind() == io::ErrorKind::BrokenPipe {
+                    return ExitStatus::Success;
+                }
+            }
+
             writeln!(stderr, "  {} {cause}", "Cause:".bold()).ok();
         }
 
@@ -47,7 +56,10 @@ pub fn main() -> ExitStatus {
 }
 
 fn run() -> anyhow::Result<ExitStatus> {
-    let args = Args::parse_from(std::env::args());
+    let args = wild::args_os();
+    let args = argfile::expand_args_from(args, argfile::parse_fromfile, argfile::PREFIX)
+        .context("Failed to read CLI arguments from file")?;
+    let args = Args::parse_from(args);
 
     match args.command {
         Command::Server => run_server().map(|()| ExitStatus::Success),
@@ -69,7 +81,7 @@ fn run_check(args: CheckCommand) -> anyhow::Result<ExitStatus> {
     let _guard = setup_tracing(verbosity)?;
 
     // The base path to which all CLI arguments are relative to.
-    let cli_base_path = {
+    let cwd = {
         let cwd = std::env::current_dir().context("Failed to get the current working directory")?;
         SystemPathBuf::from_path_buf(cwd)
             .map_err(|path| {
@@ -80,30 +92,42 @@ fn run_check(args: CheckCommand) -> anyhow::Result<ExitStatus> {
             })?
     };
 
-    let cwd = args
+    let project_path = args
         .project
         .as_ref()
-        .map(|cwd| {
-            if cwd.as_std_path().is_dir() {
-                Ok(SystemPath::absolute(cwd, &cli_base_path))
+        .map(|project| {
+            if project.as_std_path().is_dir() {
+                Ok(SystemPath::absolute(project, &cwd))
             } else {
-                Err(anyhow!("Provided project path `{cwd}` is not a directory"))
+                Err(anyhow!(
+                    "Provided project path `{project}` is not a directory"
+                ))
             }
         })
         .transpose()?
-        .unwrap_or_else(|| cli_base_path.clone());
+        .unwrap_or_else(|| cwd.clone());
+
+    let check_paths: Vec<_> = args
+        .paths
+        .iter()
+        .map(|path| SystemPath::absolute(path, &cwd))
+        .collect();
 
     let system = OsSystem::new(cwd);
     let watch = args.watch;
     let exit_zero = args.exit_zero;
 
     let cli_options = args.into_options();
-    let mut project_metadata = ProjectMetadata::discover(system.current_directory(), &system)?;
+    let mut project_metadata = ProjectMetadata::discover(&project_path, &system)?;
     project_metadata.apply_cli_options(cli_options.clone());
     project_metadata.apply_configuration_files(&system)?;
 
     let mut db = ProjectDatabase::new(project_metadata, system)?;
 
+    if !check_paths.is_empty() {
+        db.project().set_included_paths(&mut db, check_paths);
+    }
+
     let (main_loop, main_loop_cancellation_token) = MainLoop::new(cli_options);
 
     // Listen to Ctrl+C and abort the watch mode.
@@ -119,7 +143,7 @@ fn run_check(args: CheckCommand) -> anyhow::Result<ExitStatus> {
     let exit_status = if watch {
         main_loop.watch(&mut db)?
     } else {
-        main_loop.run(&mut db)
+        main_loop.run(&mut db)?
     };
 
     tracing::trace!("Counts for entire CLI run:\n{}", countme::get_all());
@@ -179,7 +203,7 @@ impl MainLoop {
         )
     }
 
-    fn watch(mut self, db: &mut ProjectDatabase) -> anyhow::Result<ExitStatus> {
+    fn watch(mut self, db: &mut ProjectDatabase) -> Result<ExitStatus> {
         tracing::debug!("Starting watch mode");
         let sender = self.sender.clone();
         let watcher = watch::directory_watcher(move |event| {
@@ -188,12 +212,12 @@ impl MainLoop {
 
         self.watcher = Some(ProjectWatcher::new(watcher, db));
 
-        self.run(db);
+        self.run(db)?;
 
         Ok(ExitStatus::Success)
     }
 
-    fn run(mut self, db: &mut ProjectDatabase) -> ExitStatus {
+    fn run(mut self, db: &mut ProjectDatabase) -> Result<ExitStatus> {
         self.sender.send(MainLoopMessage::CheckWorkspace).unwrap();
 
         let result = self.main_loop(db);
@@ -203,7 +227,7 @@ impl MainLoop {
         result
     }
 
-    fn main_loop(&mut self, db: &mut ProjectDatabase) -> ExitStatus {
+    fn main_loop(&mut self, db: &mut ProjectDatabase) -> Result<ExitStatus> {
         // Schedule the first check.
         tracing::debug!("Starting main loop");
 
@@ -241,14 +265,43 @@ impl MainLoop {
                             Severity::Error
                         };
 
-                    let failed = result
-                        .iter()
-                        .any(|diagnostic| diagnostic.severity() >= min_error_severity);
-
                     if check_revision == revision {
-                        #[allow(clippy::print_stdout)]
-                        for diagnostic in result {
-                            println!("{}", diagnostic.display(db, &display_config));
+                        if db.project().files(db).is_empty() {
+                            tracing::warn!("No python files found under the given path(s)");
+                        }
+
+                        let mut stdout = stdout().lock();
+
+                        if result.is_empty() {
+                            writeln!(stdout, "All checks passed!")?;
+
+                            if self.watcher.is_none() {
+                                return Ok(ExitStatus::Success);
+                            }
+                        } else {
+                            let mut failed = false;
+                            let diagnostics_count = result.len();
+
+                            for diagnostic in result {
+                                writeln!(stdout, "{}", diagnostic.display(db, &display_config))?;
+
+                                failed |= diagnostic.severity() >= min_error_severity;
+                            }
+
+                            writeln!(
+                                stdout,
+                                "Found {} diagnostic{}",
+                                diagnostics_count,
+                                if diagnostics_count > 1 { "s" } else { "" }
+                            )?;
+
+                            if self.watcher.is_none() {
+                                return Ok(if failed {
+                                    ExitStatus::Failure
+                                } else {
+                                    ExitStatus::Success
+                                });
+                            }
                         }
                     } else {
                         tracing::debug!(
@@ -256,14 +309,6 @@ impl MainLoop {
                         );
                     }
 
-                    if self.watcher.is_none() {
-                        return if failed {
-                            ExitStatus::Failure
-                        } else {
-                            ExitStatus::Success
-                        };
-                    }
-
                     tracing::trace!("Counts after last check:\n{}", countme::get_all());
                 }
 
@@ -281,14 +326,14 @@ impl MainLoop {
                     // TODO: Don't use Salsa internal APIs
                     //  [Zulip-Thread](https://salsa.zulipchat.com/#narrow/stream/333573-salsa-3.2E0/topic/Expose.20an.20API.20to.20cancel.20other.20queries)
                     let _ = db.zalsa_mut();
-                    return ExitStatus::Success;
+                    return Ok(ExitStatus::Success);
                 }
             }
 
             tracing::debug!("Waiting for next main loop message.");
         }
 
-        ExitStatus::Success
+        Ok(ExitStatus::Success)
     }
 }
 
@@ -308,7 +353,8 @@ impl MainLoopCancellationToken {
 enum MainLoopMessage {
     CheckWorkspace,
     CheckCompleted {
-        result: Vec<Box<dyn Diagnostic>>,
+        /// The diagnostics that were found during the check.
+        result: Vec<Box<dyn OldDiagnosticTrait>>,
         revision: u64,
     },
     ApplyChanges(Vec<watch::ChangeEvent>),

crates/red_knot/tests/cli.rs

@@ -28,7 +28,7 @@ fn config_override() -> anyhow::Result<()> {
         ),
     ])?;
 
-    assert_cmd_snapshot!(case.command(), @r###"
+    assert_cmd_snapshot!(case.command(), @r"
     success: false
     exit_code: 1
     ----- stdout -----
@@ -40,16 +40,18 @@ fn config_override() -> anyhow::Result<()> {
       |       ^^^^^^^^^^^^ Type `<module 'sys'>` has no attribute `last_exc`
       |
 
+    Found 1 diagnostic
 
     ----- stderr -----
-    "###);
+    ");
 
     assert_cmd_snapshot!(case.command().arg("--python-version").arg("3.12"), @r"
-        success: true
-        exit_code: 0
-        ----- stdout -----
+    success: true
+    exit_code: 0
+    ----- stdout -----
+    All checks passed!
 
-        ----- stderr -----
+    ----- stderr -----
     ");
 
     Ok(())
@@ -84,7 +86,7 @@ fn cli_arguments_are_relative_to_the_current_directory() -> anyhow::Result<()> {
             "libs/utils.py",
             r#"
             def add(a: int, b: int) -> int:
-                a + b
+                return a + b
             "#,
         ),
         (
@@ -98,7 +100,7 @@ fn cli_arguments_are_relative_to_the_current_directory() -> anyhow::Result<()> {
     ])?;
 
     // Make sure that the CLI fails when the `libs` directory is not in the search path.
-    assert_cmd_snapshot!(case.command().current_dir(case.root().join("child")), @r###"
+    assert_cmd_snapshot!(case.command().current_dir(case.root().join("child")), @r"
     success: false
     exit_code: 1
     ----- stdout -----
@@ -111,16 +113,18 @@ fn cli_arguments_are_relative_to_the_current_directory() -> anyhow::Result<()> {
     4 | stat = add(10, 15)
       |
 
+    Found 1 diagnostic
 
     ----- stderr -----
-    "###);
+    ");
 
     assert_cmd_snapshot!(case.command().current_dir(case.root().join("child")).arg("--extra-search-path").arg("../libs"), @r"
-        success: true
-        exit_code: 0
-        ----- stdout -----
+    success: true
+    exit_code: 0
+    ----- stdout -----
+    All checks passed!
 
-        ----- stderr -----
+    ----- stderr -----
     ");
 
     Ok(())
@@ -154,7 +158,7 @@ fn paths_in_configuration_files_are_relative_to_the_project_root() -> anyhow::Re
             "libs/utils.py",
             r#"
             def add(a: int, b: int) -> int:
-                a + b
+                return a + b
             "#,
         ),
         (
@@ -168,11 +172,12 @@ fn paths_in_configuration_files_are_relative_to_the_project_root() -> anyhow::Re
     ])?;
 
     assert_cmd_snapshot!(case.command().current_dir(case.root().join("child")), @r"
-        success: true
-        exit_code: 0
-        ----- stdout -----
+    success: true
+    exit_code: 0
+    ----- stdout -----
+    All checks passed!
 
-        ----- stderr -----
+    ----- stderr -----
     ");
 
     Ok(())
@@ -195,7 +200,7 @@ fn configuration_rule_severity() -> anyhow::Result<()> {
 
     // Assert that there's a possibly unresolved reference diagnostic
     // and that division-by-zero has a severity of error by default.
-    assert_cmd_snapshot!(case.command(), @r###"
+    assert_cmd_snapshot!(case.command(), @r"
     success: false
     exit_code: 1
     ----- stdout -----
@@ -217,9 +222,10 @@ fn configuration_rule_severity() -> anyhow::Result<()> {
       |       - Name `x` used when possibly not defined
       |
 
+    Found 2 diagnostics
 
     ----- stderr -----
-    "###);
+    ");
 
     case.write_file(
         "pyproject.toml",
@@ -230,7 +236,7 @@ fn configuration_rule_severity() -> anyhow::Result<()> {
     "#,
     )?;
 
-    assert_cmd_snapshot!(case.command(), @r###"
+    assert_cmd_snapshot!(case.command(), @r"
     success: true
     exit_code: 0
     ----- stdout -----
@@ -243,9 +249,10 @@ fn configuration_rule_severity() -> anyhow::Result<()> {
     4 | for a in range(0, y):
       |
 
+    Found 1 diagnostic
 
     ----- stderr -----
-    "###);
+    ");
 
     Ok(())
 }
@@ -269,7 +276,7 @@ fn cli_rule_severity() -> anyhow::Result<()> {
 
     // Assert that there's a possibly unresolved reference diagnostic
     // and that division-by-zero has a severity of error by default.
-    assert_cmd_snapshot!(case.command(), @r###"
+    assert_cmd_snapshot!(case.command(), @r"
     success: false
     exit_code: 1
     ----- stdout -----
@@ -302,9 +309,10 @@ fn cli_rule_severity() -> anyhow::Result<()> {
       |       - Name `x` used when possibly not defined
       |
 
+    Found 3 diagnostics
 
     ----- stderr -----
-    "###);
+    ");
 
     assert_cmd_snapshot!(
         case
@@ -315,7 +323,7 @@ fn cli_rule_severity() -> anyhow::Result<()> {
             .arg("division-by-zero")
             .arg("--warn")
             .arg("unresolved-import"),
-        @r###"
+        @r"
     success: true
     exit_code: 0
     ----- stdout -----
@@ -339,9 +347,10 @@ fn cli_rule_severity() -> anyhow::Result<()> {
     6 | for a in range(0, y):
       |
 
+    Found 2 diagnostics
 
     ----- stderr -----
-    "###
+    "
     );
 
     Ok(())
@@ -365,7 +374,7 @@ fn cli_rule_severity_precedence() -> anyhow::Result<()> {
 
     // Assert that there's a possibly unresolved reference diagnostic
     // and that division-by-zero has a severity of error by default.
-    assert_cmd_snapshot!(case.command(), @r###"
+    assert_cmd_snapshot!(case.command(), @r"
     success: false
     exit_code: 1
     ----- stdout -----
@@ -387,9 +396,10 @@ fn cli_rule_severity_precedence() -> anyhow::Result<()> {
       |       - Name `x` used when possibly not defined
       |
 
+    Found 2 diagnostics
 
     ----- stderr -----
-    "###);
+    ");
 
     assert_cmd_snapshot!(
         case
@@ -401,7 +411,7 @@ fn cli_rule_severity_precedence() -> anyhow::Result<()> {
             // Override the error severity with warning
             .arg("--ignore")
             .arg("possibly-unresolved-reference"),
-        @r###"
+        @r"
     success: true
     exit_code: 0
     ----- stdout -----
@@ -414,9 +424,10 @@ fn cli_rule_severity_precedence() -> anyhow::Result<()> {
     4 | for a in range(0, y):
       |
 
+    Found 1 diagnostic
 
     ----- stderr -----
-    "###
+    "
     );
 
     Ok(())
@@ -436,7 +447,7 @@ fn configuration_unknown_rules() -> anyhow::Result<()> {
         ("test.py", "print(10)"),
     ])?;
 
-    assert_cmd_snapshot!(case.command(), @r###"
+    assert_cmd_snapshot!(case.command(), @r#"
     success: true
     exit_code: 0
     ----- stdout -----
@@ -448,9 +459,10 @@ fn configuration_unknown_rules() -> anyhow::Result<()> {
       | --------------- Unknown lint rule `division-by-zer`
       |
 
+    Found 1 diagnostic
 
     ----- stderr -----
-    "###);
+    "#);
 
     Ok(())
 }
@@ -460,15 +472,16 @@ fn configuration_unknown_rules() -> anyhow::Result<()> {
 fn cli_unknown_rules() -> anyhow::Result<()> {
     let case = TestCase::with_file("test.py", "print(10)")?;
 
-    assert_cmd_snapshot!(case.command().arg("--ignore").arg("division-by-zer"), @r###"
+    assert_cmd_snapshot!(case.command().arg("--ignore").arg("division-by-zer"), @r"
     success: true
     exit_code: 0
     ----- stdout -----
     warning: unknown-rule: Unknown lint rule `division-by-zer`
 
+    Found 1 diagnostic
 
     ----- stderr -----
-    "###);
+    ");
 
     Ok(())
 }
@@ -477,7 +490,7 @@ fn cli_unknown_rules() -> anyhow::Result<()> {
 fn exit_code_only_warnings() -> anyhow::Result<()> {
     let case = TestCase::with_file("test.py", r"print(x)  # [unresolved-reference]")?;
 
-    assert_cmd_snapshot!(case.command(), @r###"
+    assert_cmd_snapshot!(case.command(), @r"
     success: true
     exit_code: 0
     ----- stdout -----
@@ -488,9 +501,10 @@ fn exit_code_only_warnings() -> anyhow::Result<()> {
       |       - Name `x` used when not defined
       |
 
+    Found 1 diagnostic
 
     ----- stderr -----
-    "###);
+    ");
 
     Ok(())
 }
@@ -505,7 +519,7 @@ fn exit_code_only_info() -> anyhow::Result<()> {
         "#,
     )?;
 
-    assert_cmd_snapshot!(case.command(), @r###"
+    assert_cmd_snapshot!(case.command(), @r"
     success: true
     exit_code: 0
     ----- stdout -----
@@ -517,9 +531,10 @@ fn exit_code_only_info() -> anyhow::Result<()> {
       | -------------- info: Revealed type is `Literal[1]`
       |
 
+    Found 1 diagnostic
 
     ----- stderr -----
-    "###);
+    ");
 
     Ok(())
 }
@@ -534,7 +549,7 @@ fn exit_code_only_info_and_error_on_warning_is_true() -> anyhow::Result<()> {
         "#,
     )?;
 
-    assert_cmd_snapshot!(case.command().arg("--error-on-warning"), @r###"
+    assert_cmd_snapshot!(case.command().arg("--error-on-warning"), @r"
     success: true
     exit_code: 0
     ----- stdout -----
@@ -546,9 +561,10 @@ fn exit_code_only_info_and_error_on_warning_is_true() -> anyhow::Result<()> {
       | -------------- info: Revealed type is `Literal[1]`
       |
 
+    Found 1 diagnostic
 
     ----- stderr -----
-    "###);
+    ");
 
     Ok(())
 }
@@ -557,7 +573,7 @@ fn exit_code_only_info_and_error_on_warning_is_true() -> anyhow::Result<()> {
 fn exit_code_no_errors_but_error_on_warning_is_true() -> anyhow::Result<()> {
     let case = TestCase::with_file("test.py", r"print(x)  # [unresolved-reference]")?;
 
-    assert_cmd_snapshot!(case.command().arg("--error-on-warning"), @r###"
+    assert_cmd_snapshot!(case.command().arg("--error-on-warning"), @r"
     success: false
     exit_code: 1
     ----- stdout -----
@@ -568,9 +584,10 @@ fn exit_code_no_errors_but_error_on_warning_is_true() -> anyhow::Result<()> {
       |       - Name `x` used when not defined
       |
 
+    Found 1 diagnostic
 
     ----- stderr -----
-    "###);
+    ");
 
     Ok(())
 }
@@ -588,7 +605,7 @@ fn exit_code_no_errors_but_error_on_warning_is_enabled_in_configuration() -> any
         ),
     ])?;
 
-    assert_cmd_snapshot!(case.command(), @r###"
+    assert_cmd_snapshot!(case.command(), @r"
     success: false
     exit_code: 1
     ----- stdout -----
@@ -599,9 +616,10 @@ fn exit_code_no_errors_but_error_on_warning_is_enabled_in_configuration() -> any
       |       - Name `x` used when not defined
       |
 
+    Found 1 diagnostic
 
     ----- stderr -----
-    "###);
+    ");
 
     Ok(())
 }
@@ -616,7 +634,7 @@ fn exit_code_both_warnings_and_errors() -> anyhow::Result<()> {
         "#,
     )?;
 
-    assert_cmd_snapshot!(case.command(), @r###"
+    assert_cmd_snapshot!(case.command(), @r"
     success: false
     exit_code: 1
     ----- stdout -----
@@ -636,9 +654,10 @@ fn exit_code_both_warnings_and_errors() -> anyhow::Result<()> {
       |       ^ Cannot subscript object of type `Literal[4]` with no `__getitem__` method
       |
 
+    Found 2 diagnostics
 
     ----- stderr -----
-    "###);
+    ");
 
     Ok(())
 }
@@ -653,7 +672,7 @@ fn exit_code_both_warnings_and_errors_and_error_on_warning_is_true() -> anyhow::
         "###,
     )?;
 
-    assert_cmd_snapshot!(case.command().arg("--error-on-warning"), @r###"
+    assert_cmd_snapshot!(case.command().arg("--error-on-warning"), @r"
     success: false
     exit_code: 1
     ----- stdout -----
@@ -673,9 +692,10 @@ fn exit_code_both_warnings_and_errors_and_error_on_warning_is_true() -> anyhow::
       |       ^ Cannot subscript object of type `Literal[4]` with no `__getitem__` method
       |
 
+    Found 2 diagnostics
 
     ----- stderr -----
-    "###);
+    ");
 
     Ok(())
 }
@@ -690,7 +710,7 @@ fn exit_code_exit_zero_is_true() -> anyhow::Result<()> {
         "#,
     )?;
 
-    assert_cmd_snapshot!(case.command().arg("--exit-zero"), @r###"
+    assert_cmd_snapshot!(case.command().arg("--exit-zero"), @r"
     success: true
     exit_code: 0
     ----- stdout -----
@@ -710,9 +730,10 @@ fn exit_code_exit_zero_is_true() -> anyhow::Result<()> {
       |       ^ Cannot subscript object of type `Literal[4]` with no `__getitem__` method
       |
 
+    Found 2 diagnostics
 
     ----- stderr -----
-    "###);
+    ");
 
     Ok(())
 }
@@ -749,7 +770,7 @@ fn user_configuration() -> anyhow::Result<()> {
 
     assert_cmd_snapshot!(
         case.command().current_dir(case.root().join("project")).env(config_env_var, config_directory.as_os_str()),
-        @r###"
+        @r"
     success: true
     exit_code: 0
     ----- stdout -----
@@ -771,9 +792,10 @@ fn user_configuration() -> anyhow::Result<()> {
       |       - Name `x` used when possibly not defined
       |
 
+    Found 2 diagnostics
 
     ----- stderr -----
-    "###
+    "
     );
 
     // The user-level configuration promotes `possibly-unresolved-reference` to an error.
@@ -790,7 +812,7 @@ fn user_configuration() -> anyhow::Result<()> {
 
     assert_cmd_snapshot!(
         case.command().current_dir(case.root().join("project")).env(config_env_var, config_directory.as_os_str()),
-        @r###"
+        @r"
     success: false
     exit_code: 1
     ----- stdout -----
@@ -812,9 +834,134 @@ fn user_configuration() -> anyhow::Result<()> {
       |       ^ Name `x` used when possibly not defined
       |
 
+    Found 2 diagnostics
 
     ----- stderr -----
-    "###
+    "
+    );
+
+    Ok(())
+}
+
+#[test]
+fn check_specific_paths() -> anyhow::Result<()> {
+    let case = TestCase::with_files([
+        (
+            "project/main.py",
+            r#"
+            y = 4 / 0  # error: division-by-zero
+            "#,
+        ),
+        (
+            "project/tests/test_main.py",
+            r#"
+            import does_not_exist  # error: unresolved-import
+            "#,
+        ),
+        (
+            "project/other.py",
+            r#"
+            from main2 import z  # error: unresolved-import
+
+            print(z)
+            "#,
+        ),
+    ])?;
+
+    assert_cmd_snapshot!(
+        case.command(),
+        @r"
+    success: false
+    exit_code: 1
+    ----- stdout -----
+    error: lint:unresolved-import
+     --> <temp_dir>/project/tests/test_main.py:2:8
+      |
+    2 | import does_not_exist  # error: unresolved-import
+      |        ^^^^^^^^^^^^^^ Cannot resolve import `does_not_exist`
+      |
+
+    error: lint:division-by-zero
+     --> <temp_dir>/project/main.py:2:5
+      |
+    2 | y = 4 / 0  # error: division-by-zero
+      |     ^^^^^ Cannot divide object of type `Literal[4]` by zero
+      |
+
+    error: lint:unresolved-import
+     --> <temp_dir>/project/other.py:2:6
+      |
+    2 | from main2 import z  # error: unresolved-import
+      |      ^^^^^ Cannot resolve import `main2`
+    3 |
+    4 | print(z)
+      |
+
+    Found 3 diagnostics
+
+    ----- stderr -----
+    "
+    );
+
+    // Now check only the `tests` and `other.py` files.
+    // We should no longer see any diagnostics related to `main.py`.
+    assert_cmd_snapshot!(
+        case.command().arg("project/tests").arg("project/other.py"),
+        @r"
+    success: false
+    exit_code: 1
+    ----- stdout -----
+    error: lint:unresolved-import
+     --> <temp_dir>/project/tests/test_main.py:2:8
+      |
+    2 | import does_not_exist  # error: unresolved-import
+      |        ^^^^^^^^^^^^^^ Cannot resolve import `does_not_exist`
+      |
+
+    error: lint:unresolved-import
+     --> <temp_dir>/project/other.py:2:6
+      |
+    2 | from main2 import z  # error: unresolved-import
+      |      ^^^^^ Cannot resolve import `main2`
+    3 |
+    4 | print(z)
+      |
+
+    Found 2 diagnostics
+
+    ----- stderr -----
+    "
+    );
+
+    Ok(())
+}
+
+#[test]
+fn check_non_existing_path() -> anyhow::Result<()> {
+    let case = TestCase::with_files([])?;
+
+    let mut settings = insta::Settings::clone_current();
+    settings.add_filter(
+        &regex::escape("The system cannot find the path specified. (os error 3)"),
+        "No such file or directory (os error 2)",
+    );
+    let _s = settings.bind_to_scope();
+
+    assert_cmd_snapshot!(
+        case.command().arg("project/main.py").arg("project/tests"),
+        @r"
+    success: false
+    exit_code: 1
+    ----- stdout -----
+    error: io: `<temp_dir>/project/main.py`: No such file or directory (os error 2)
+
+    error: io: `<temp_dir>/project/tests`: No such file or directory (os error 2)
+
+    Found 2 diagnostics
+
+    ----- stderr -----
+    WARN No python files found under the given path(s)
+    "
     );
 
     Ok(())

crates/red_knot/tests/file_watching.rs

@@ -1,5 +1,6 @@
 #![allow(clippy::disallowed_names)]
 
+use std::collections::HashSet;
 use std::io::Write;
 use std::time::{Duration, Instant};
 
@@ -193,11 +194,29 @@ impl TestCase {
         Ok(())
     }
 
-    fn collect_project_files(&self) -> Vec<File> {
-        let files = self.db().project().files(self.db());
-        let mut collected: Vec<_> = files.into_iter().collect();
-        collected.sort_unstable_by_key(|file| file.path(self.db()).as_system_path().unwrap());
-        collected
+    #[track_caller]
+    fn assert_indexed_project_files(&self, expected: impl IntoIterator<Item = File>) {
+        let mut expected: HashSet<_> = expected.into_iter().collect();
+
+        let actual = self.db().project().files(self.db());
+        for file in &actual {
+            assert!(
+                expected.remove(&file),
+                "Indexed project files contains '{}' which was not expected.",
+                file.path(self.db())
+            );
+        }
+
+        if !expected.is_empty() {
+            let paths: Vec<_> = expected
+                .iter()
+                .map(|file| file.path(self.db()).as_str())
+                .collect();
+            panic!(
+                "Indexed project files are missing the following files: {:?}",
+                paths.join(", ")
+            );
+        }
     }
 
     fn system_file(&self, path: impl AsRef<SystemPath>) -> Result<File, FileError> {
@@ -222,13 +241,15 @@ where
     }
 }
 
-trait SetupFiles {
-    fn setup(self, context: &SetupContext) -> anyhow::Result<()>;
+trait Setup {
+    fn setup(self, context: &mut SetupContext) -> anyhow::Result<()>;
 }
 
 struct SetupContext<'a> {
     system: &'a OsSystem,
     root_path: &'a SystemPath,
+    options: Option<Options>,
+    included_paths: Option<Vec<SystemPathBuf>>,
 }
 
 impl<'a> SetupContext<'a> {
@@ -251,55 +272,77 @@ impl<'a> SetupContext<'a> {
     fn join_root_path(&self, relative: impl AsRef<SystemPath>) -> SystemPathBuf {
         self.root_path().join(relative)
     }
+
+    fn write_project_file(
+        &self,
+        relative_path: impl AsRef<SystemPath>,
+        content: &str,
+    ) -> anyhow::Result<()> {
+        let relative_path = relative_path.as_ref();
+        let absolute_path = self.join_project_path(relative_path);
+        Self::write_file_impl(absolute_path, content)
+    }
+
+    fn write_file(
+        &self,
+        relative_path: impl AsRef<SystemPath>,
+        content: &str,
+    ) -> anyhow::Result<()> {
+        let relative_path = relative_path.as_ref();
+        let absolute_path = self.join_root_path(relative_path);
+        Self::write_file_impl(absolute_path, content)
+    }
+
+    fn write_file_impl(path: impl AsRef<SystemPath>, content: &str) -> anyhow::Result<()> {
+        let path = path.as_ref();
+        if let Some(parent) = path.parent() {
+            std::fs::create_dir_all(parent)
+                .with_context(|| format!("Failed to create parent directory for file `{path}`"))?;
+        }
+
+        let mut file = std::fs::File::create(path.as_std_path())
+            .with_context(|| format!("Failed to open file `{path}`"))?;
+        file.write_all(content.as_bytes())
+            .with_context(|| format!("Failed to write to file `{path}`"))?;
+        file.sync_data()?;
+
+        Ok(())
+    }
+
+    fn set_options(&mut self, options: Options) {
+        self.options = Some(options);
+    }
+
+    fn set_included_paths(&mut self, paths: Vec<SystemPathBuf>) {
+        self.included_paths = Some(paths);
+    }
 }
 
-impl<const N: usize, P> SetupFiles for [(P, &'static str); N]
+impl<const N: usize, P> Setup for [(P, &'static str); N]
 where
     P: AsRef<SystemPath>,
 {
-    fn setup(self, context: &SetupContext) -> anyhow::Result<()> {
+    fn setup(self, context: &mut SetupContext) -> anyhow::Result<()> {
         for (relative_path, content) in self {
-            let relative_path = relative_path.as_ref();
-            let absolute_path = context.join_project_path(relative_path);
-            if let Some(parent) = absolute_path.parent() {
-                std::fs::create_dir_all(parent).with_context(|| {
-                    format!("Failed to create parent directory for file `{relative_path}`")
-                })?;
-            }
-
-            let mut file = std::fs::File::create(absolute_path.as_std_path())
-                .with_context(|| format!("Failed to open file `{relative_path}`"))?;
-            file.write_all(content.as_bytes())
-                .with_context(|| format!("Failed to write to file `{relative_path}`"))?;
-            file.sync_data()?;
+            context.write_project_file(relative_path, content)?;
         }
 
         Ok(())
     }
 }
 
-impl<F> SetupFiles for F
+impl<F> Setup for F
 where
-    F: FnOnce(&SetupContext) -> anyhow::Result<()>,
+    F: FnOnce(&mut SetupContext) -> anyhow::Result<()>,
 {
-    fn setup(self, context: &SetupContext) -> anyhow::Result<()> {
+    fn setup(self, context: &mut SetupContext) -> anyhow::Result<()> {
         self(context)
     }
 }
 
 fn setup<F>(setup_files: F) -> anyhow::Result<TestCase>
 where
-    F: SetupFiles,
-{
-    setup_with_options(setup_files, |_context| None)
-}
-
-fn setup_with_options<F>(
-    setup_files: F,
-    create_options: impl FnOnce(&SetupContext) -> Option<Options>,
-) -> anyhow::Result<TestCase>
-where
-    F: SetupFiles,
+    F: Setup,
 {
     let temp_dir = tempfile::tempdir()?;
 
@@ -325,16 +368,18 @@ where
         .with_context(|| format!("Failed to create project directory `{project_path}`"))?;
 
     let system = OsSystem::new(&project_path);
-    let setup_context = SetupContext {
+    let mut setup_context = SetupContext {
         system: &system,
         root_path: &root_path,
+        options: None,
+        included_paths: None,
     };
 
     setup_files
-        .setup(&setup_context)
+        .setup(&mut setup_context)
         .context("Failed to setup test files")?;
 
-    if let Some(options) = create_options(&setup_context) {
+    if let Some(options) = setup_context.options {
         std::fs::write(
             project_path.join("pyproject.toml").as_std_path(),
             toml::to_string(&PyProject {
@@ -348,6 +393,8 @@ where
         .context("Failed to write configuration")?;
     }
 
+    let included_paths = setup_context.included_paths;
+
     let mut project = ProjectMetadata::discover(&project_path, &system)?;
     project.apply_configuration_files(&system)?;
 
@@ -363,7 +410,11 @@ where
             .with_context(|| format!("Failed to create search path `{path}`"))?;
     }
 
-    let db = ProjectDatabase::new(project, system)?;
+    let mut db = ProjectDatabase::new(project, system)?;
+
+    if let Some(included_paths) = included_paths {
+        db.project().set_included_paths(&mut db, included_paths);
+    }
 
     let (sender, receiver) = crossbeam::channel::unbounded();
     let watcher = directory_watcher(move |events| sender.send(events).unwrap())
@@ -425,7 +476,7 @@ fn new_file() -> anyhow::Result<()> {
     let foo_path = case.project_path("foo.py");
 
     assert_eq!(case.system_file(&foo_path), Err(FileError::NotFound));
-    assert_eq!(&case.collect_project_files(), &[bar_file]);
+    case.assert_indexed_project_files([bar_file]);
 
     std::fs::write(foo_path.as_std_path(), "print('Hello')")?;
 
@@ -435,7 +486,7 @@ fn new_file() -> anyhow::Result<()> {
 
     let foo = case.system_file(&foo_path).expect("foo.py to exist.");
 
-    assert_eq!(&case.collect_project_files(), &[bar_file, foo]);
+    case.assert_indexed_project_files([bar_file, foo]);
 
     Ok(())
 }
@@ -448,7 +499,7 @@ fn new_ignored_file() -> anyhow::Result<()> {
     let foo_path = case.project_path("foo.py");
 
     assert_eq!(case.system_file(&foo_path), Err(FileError::NotFound));
-    assert_eq!(&case.collect_project_files(), &[bar_file]);
+    case.assert_indexed_project_files([bar_file]);
 
     std::fs::write(foo_path.as_std_path(), "print('Hello')")?;
 
@@ -457,7 +508,132 @@ fn new_ignored_file() -> anyhow::Result<()> {
     case.apply_changes(changes);
 
     assert!(case.system_file(&foo_path).is_ok());
-    assert_eq!(&case.collect_project_files(), &[bar_file]);
+    case.assert_indexed_project_files([bar_file]);
+
+    Ok(())
+}
+
+#[test]
+fn new_non_project_file() -> anyhow::Result<()> {
+    let mut case = setup(|context: &mut SetupContext| {
+        context.write_project_file("bar.py", "")?;
+        context.set_options(Options {
+            environment: Some(EnvironmentOptions {
+                extra_paths: Some(vec![RelativePathBuf::cli(
+                    context.join_root_path("site_packages"),
+                )]),
+                ..EnvironmentOptions::default()
+            }),
+            ..Options::default()
+        });
+
+        Ok(())
+    })?;
+
+    let bar_path = case.project_path("bar.py");
+    let bar_file = case.system_file(&bar_path).unwrap();
+
+    case.assert_indexed_project_files([bar_file]);
+
+    // Add a file to site packages
+    let black_path = case.root_path().join("site_packages/black.py");
+    std::fs::write(black_path.as_std_path(), "print('Hello')")?;
+
+    let changes = case.stop_watch(event_for_file("black.py"));
+
+    case.apply_changes(changes);
+
+    assert!(case.system_file(&black_path).is_ok());
+
+    // The file should not have been added to the project files
+    case.assert_indexed_project_files([bar_file]);
+
+    Ok(())
+}
+
+#[test]
+fn new_files_with_explicit_included_paths() -> anyhow::Result<()> {
+    let mut case = setup(|context: &mut SetupContext| {
+        context.write_project_file("src/main.py", "")?;
+        context.write_project_file("src/sub/__init__.py", "")?;
+        context.write_project_file("src/test.py", "")?;
+        context.set_included_paths(vec![
+            context.join_project_path("src/main.py"),
+            context.join_project_path("src/sub"),
+        ]);
+        Ok(())
+    })?;
+
+    let main_path = case.project_path("src/main.py");
+    let main_file = case.system_file(&main_path).unwrap();
+
+    let sub_init_path = case.project_path("src/sub/__init__.py");
+    let sub_init = case.system_file(&sub_init_path).unwrap();
+
+    case.assert_indexed_project_files([main_file, sub_init]);
+
+    // Write a new file to `sub` which is an included path
+    let sub_a_path = case.project_path("src/sub/a.py");
+    std::fs::write(sub_a_path.as_std_path(), "print('Hello')")?;
+
+    // and write a second file in the root directory -- this should not be included
+    let test2_path = case.project_path("src/test2.py");
+    std::fs::write(test2_path.as_std_path(), "print('Hello')")?;
+
+    let changes = case.stop_watch(event_for_file("test2.py"));
+
+    case.apply_changes(changes);
+
+    let sub_a_file = case.system_file(&sub_a_path).expect("sub/a.py to exist");
+
+    case.assert_indexed_project_files([main_file, sub_init, sub_a_file]);
+
+    Ok(())
+}
+
+#[test]
+fn new_file_in_included_out_of_project_directory() -> anyhow::Result<()> {
+    let mut case = setup(|context: &mut SetupContext| {
+        context.write_project_file("src/main.py", "")?;
+        context.write_project_file("script.py", "")?;
+        context.write_file("outside_project/a.py", "")?;
+
+        context.set_included_paths(vec![
+            context.join_root_path("outside_project"),
+            context.join_project_path("src"),
+        ]);
+        Ok(())
+    })?;
+
+    let main_path = case.project_path("src/main.py");
+    let main_file = case.system_file(&main_path).unwrap();
+
+    let outside_a_path = case.root_path().join("outside_project/a.py");
+    let outside_a = case.system_file(&outside_a_path).unwrap();
+
+    case.assert_indexed_project_files([outside_a, main_file]);
+
+    // Write a new file to `src` which should be watched
+    let src_a = case.project_path("src/a.py");
+    std::fs::write(src_a.as_std_path(), "print('Hello')")?;
+
+    // and write a second file to `outside_project` which should be watched too
+    let outside_b_path = case.root_path().join("outside_project/b.py");
+    std::fs::write(outside_b_path.as_std_path(), "print('Hello')")?;
+
+    // and a third file in the project's root that should not be included
+    let script2_path = case.project_path("script2.py");
+    std::fs::write(script2_path.as_std_path(), "print('Hello')")?;
+
+    let changes = case.stop_watch(event_for_file("script2.py"));
+
+    case.apply_changes(changes);
+
+    let src_a_file = case.system_file(&src_a).unwrap();
+    let outside_b_file = case.system_file(&outside_b_path).unwrap();
+
+    // The file should not have been added to the project files
+    case.assert_indexed_project_files([main_file, outside_a, outside_b_file, src_a_file]);
 
     Ok(())
 }
@@ -470,7 +646,7 @@ fn changed_file() -> anyhow::Result<()> {
 
     let foo = case.system_file(&foo_path)?;
     assert_eq!(source_text(case.db(), foo).as_str(), foo_source);
-    assert_eq!(&case.collect_project_files(), &[foo]);
+    case.assert_indexed_project_files([foo]);
 
     update_file(&foo_path, "print('Version 2')")?;
 
@@ -481,7 +657,7 @@ fn changed_file() -> anyhow::Result<()> {
     case.apply_changes(changes);
 
     assert_eq!(source_text(case.db(), foo).as_str(), "print('Version 2')");
-    assert_eq!(&case.collect_project_files(), &[foo]);
+    case.assert_indexed_project_files([foo]);
 
     Ok(())
 }
@@ -495,7 +671,7 @@ fn deleted_file() -> anyhow::Result<()> {
     let foo = case.system_file(&foo_path)?;
 
     assert!(foo.exists(case.db()));
-    assert_eq!(&case.collect_project_files(), &[foo]);
+    case.assert_indexed_project_files([foo]);
 
     std::fs::remove_file(foo_path.as_std_path())?;
 
@@ -504,7 +680,7 @@ fn deleted_file() -> anyhow::Result<()> {
     case.apply_changes(changes);
 
     assert!(!foo.exists(case.db()));
-    assert_eq!(&case.collect_project_files(), &[] as &[File]);
+    case.assert_indexed_project_files([]);
 
     Ok(())
 }
@@ -524,7 +700,7 @@ fn move_file_to_trash() -> anyhow::Result<()> {
     let foo = case.system_file(&foo_path)?;
 
     assert!(foo.exists(case.db()));
-    assert_eq!(&case.collect_project_files(), &[foo]);
+    case.assert_indexed_project_files([foo]);
 
     std::fs::rename(
         foo_path.as_std_path(),
@@ -536,7 +712,7 @@ fn move_file_to_trash() -> anyhow::Result<()> {
     case.apply_changes(changes);
 
     assert!(!foo.exists(case.db()));
-    assert_eq!(&case.collect_project_files(), &[] as &[File]);
+    case.assert_indexed_project_files([]);
 
     Ok(())
 }
@@ -554,7 +730,7 @@ fn move_file_to_project() -> anyhow::Result<()> {
     let foo_in_project = case.project_path("foo.py");
 
     assert!(case.system_file(&foo_path).is_ok());
-    assert_eq!(&case.collect_project_files(), &[bar]);
+    case.assert_indexed_project_files([bar]);
 
     std::fs::rename(foo_path.as_std_path(), foo_in_project.as_std_path())?;
 
@@ -565,7 +741,7 @@ fn move_file_to_project() -> anyhow::Result<()> {
     let foo_in_project = case.system_file(&foo_in_project)?;
 
     assert!(foo_in_project.exists(case.db()));
-    assert_eq!(&case.collect_project_files(), &[bar, foo_in_project]);
+    case.assert_indexed_project_files([bar, foo_in_project]);
 
     Ok(())
 }
@@ -579,7 +755,7 @@ fn rename_file() -> anyhow::Result<()> {
 
     let foo = case.system_file(&foo_path)?;
 
-    assert_eq!(case.collect_project_files(), [foo]);
+    case.assert_indexed_project_files([foo]);
 
     std::fs::rename(foo_path.as_std_path(), bar_path.as_std_path())?;
 
@@ -592,7 +768,7 @@ fn rename_file() -> anyhow::Result<()> {
     let bar = case.system_file(&bar_path)?;
 
     assert!(bar.exists(case.db()));
-    assert_eq!(case.collect_project_files(), [bar]);
+    case.assert_indexed_project_files([bar]);
 
     Ok(())
 }
@@ -618,7 +794,7 @@ fn directory_moved_to_project() -> anyhow::Result<()> {
     );
 
     assert_eq!(sub_a_module, None);
-    assert_eq!(case.collect_project_files(), &[bar]);
+    case.assert_indexed_project_files([bar]);
 
     let sub_new_path = case.project_path("sub");
     std::fs::rename(sub_original_path.as_std_path(), sub_new_path.as_std_path())
@@ -642,7 +818,7 @@ fn directory_moved_to_project() -> anyhow::Result<()> {
     )
     .is_some());
 
-    assert_eq!(case.collect_project_files(), &[bar, init_file, a_file]);
+    case.assert_indexed_project_files([bar, init_file, a_file]);
 
     Ok(())
 }
@@ -670,7 +846,7 @@ fn directory_moved_to_trash() -> anyhow::Result<()> {
         .system_file(sub_path.join("a.py"))
         .expect("a.py to exist");
 
-    assert_eq!(case.collect_project_files(), &[bar, init_file, a_file]);
+    case.assert_indexed_project_files([bar, init_file, a_file]);
 
     std::fs::create_dir(case.root_path().join(".trash").as_std_path())?;
     let trashed_sub = case.root_path().join(".trash/sub");
@@ -691,7 +867,7 @@ fn directory_moved_to_trash() -> anyhow::Result<()> {
     assert!(!init_file.exists(case.db()));
     assert!(!a_file.exists(case.db()));
 
-    assert_eq!(case.collect_project_files(), &[bar]);
+    case.assert_indexed_project_files([bar]);
 
     Ok(())
 }
@@ -725,7 +901,7 @@ fn directory_renamed() -> anyhow::Result<()> {
         .system_file(sub_path.join("a.py"))
         .expect("a.py to exist");
 
-    assert_eq!(case.collect_project_files(), &[bar, sub_init, sub_a]);
+    case.assert_indexed_project_files([bar, sub_init, sub_a]);
 
     let foo_baz = case.project_path("foo/baz");
 
@@ -767,10 +943,7 @@ fn directory_renamed() -> anyhow::Result<()> {
     assert!(foo_baz_init.exists(case.db()));
     assert!(foo_baz_a.exists(case.db()));
 
-    assert_eq!(
-        case.collect_project_files(),
-        &[bar, foo_baz_init, foo_baz_a]
-    );
+    case.assert_indexed_project_files([bar, foo_baz_init, foo_baz_a]);
 
     Ok(())
 }
@@ -799,7 +972,7 @@ fn directory_deleted() -> anyhow::Result<()> {
     let a_file = case
         .system_file(sub_path.join("a.py"))
         .expect("a.py to exist");
-    assert_eq!(case.collect_project_files(), &[bar, init_file, a_file]);
+    case.assert_indexed_project_files([bar, init_file, a_file]);
 
     std::fs::remove_dir_all(sub_path.as_std_path())
         .with_context(|| "Failed to remove the sub directory")?;
@@ -817,15 +990,17 @@ fn directory_deleted() -> anyhow::Result<()> {
 
     assert!(!init_file.exists(case.db()));
     assert!(!a_file.exists(case.db()));
-    assert_eq!(case.collect_project_files(), &[bar]);
+    case.assert_indexed_project_files([bar]);
 
     Ok(())
 }
 
 #[test]
 fn search_path() -> anyhow::Result<()> {
-    let mut case = setup_with_options([("bar.py", "import sub.a")], |context| {
-        Some(Options {
+    let mut case = setup(|context: &mut SetupContext| {
+        context.write_project_file("bar.py", "import sub.a")?;
+
+        context.set_options(Options {
             environment: Some(EnvironmentOptions {
                 extra_paths: Some(vec![RelativePathBuf::cli(
                     context.join_root_path("site_packages"),
@@ -833,7 +1008,8 @@ fn search_path() -> anyhow::Result<()> {
                 ..EnvironmentOptions::default()
             }),
             ..Options::default()
-        })
+        });
+        Ok(())
     })?;
 
     let site_packages = case.root_path().join("site_packages");
@@ -850,10 +1026,7 @@ fn search_path() -> anyhow::Result<()> {
     case.apply_changes(changes);
 
     assert!(resolve_module(case.db().upcast(), &ModuleName::new_static("a").unwrap()).is_some());
-    assert_eq!(
-        case.collect_project_files(),
-        &[case.system_file(case.project_path("bar.py")).unwrap()]
-    );
+    case.assert_indexed_project_files([case.system_file(case.project_path("bar.py")).unwrap()]);
 
     Ok(())
 }
@@ -890,8 +1063,9 @@ fn add_search_path() -> anyhow::Result<()> {
 
 #[test]
 fn remove_search_path() -> anyhow::Result<()> {
-    let mut case = setup_with_options([("bar.py", "import sub.a")], |context| {
-        Some(Options {
+    let mut case = setup(|context: &mut SetupContext| {
+        context.write_project_file("bar.py", "import sub.a")?;
+        context.set_options(Options {
             environment: Some(EnvironmentOptions {
                 extra_paths: Some(vec![RelativePathBuf::cli(
                     context.join_root_path("site_packages"),
@@ -899,7 +1073,9 @@ fn remove_search_path() -> anyhow::Result<()> {
                 ..EnvironmentOptions::default()
             }),
             ..Options::default()
-        })
+        });
+
+        Ok(())
     })?;
 
     // Remove site packages from the search path settings.
@@ -922,30 +1098,30 @@ fn remove_search_path() -> anyhow::Result<()> {
 
 #[test]
 fn change_python_version_and_platform() -> anyhow::Result<()> {
-    let mut case = setup_with_options(
+    let mut case = setup(|context: &mut SetupContext| {
         // `sys.last_exc` is a Python 3.12 only feature
         // `os.getegid()` is Unix only
-        [(
+        context.write_project_file(
             "bar.py",
             r#"
 import sys
 import os
 print(sys.last_exc, os.getegid())
 "#,
-        )],
-        |_context| {
-            Some(Options {
-                environment: Some(EnvironmentOptions {
-                    python_version: Some(RangedValue::cli(PythonVersion::PY311)),
-                    python_platform: Some(RangedValue::cli(PythonPlatform::Identifier(
-                        "win32".to_string(),
-                    ))),
-                    ..EnvironmentOptions::default()
-                }),
-                ..Options::default()
-            })
-        },
-    )?;
+        )?;
+        context.set_options(Options {
+            environment: Some(EnvironmentOptions {
+                python_version: Some(RangedValue::cli(PythonVersion::PY311)),
+                python_platform: Some(RangedValue::cli(PythonPlatform::Identifier(
+                    "win32".to_string(),
+                ))),
+                ..EnvironmentOptions::default()
+            }),
+            ..Options::default()
+        });
+
+        Ok(())
+    })?;
 
     let diagnostics = case.db.check().context("Failed to check project.")?;
 
@@ -980,38 +1156,35 @@ print(sys.last_exc, os.getegid())
 
 #[test]
 fn changed_versions_file() -> anyhow::Result<()> {
-    let mut case = setup_with_options(
-        |context: &SetupContext| {
-            std::fs::write(
-                context.join_project_path("bar.py").as_std_path(),
-                "import sub.a",
-            )?;
-            std::fs::create_dir_all(context.join_root_path("typeshed/stdlib").as_std_path())?;
-            std::fs::write(
-                context
-                    .join_root_path("typeshed/stdlib/VERSIONS")
-                    .as_std_path(),
-                "",
-            )?;
-            std::fs::write(
-                context
-                    .join_root_path("typeshed/stdlib/os.pyi")
-                    .as_std_path(),
-                "# not important",
-            )?;
+    let mut case = setup(|context: &mut SetupContext| {
+        std::fs::write(
+            context.join_project_path("bar.py").as_std_path(),
+            "import sub.a",
+        )?;
+        std::fs::create_dir_all(context.join_root_path("typeshed/stdlib").as_std_path())?;
+        std::fs::write(
+            context
+                .join_root_path("typeshed/stdlib/VERSIONS")
+                .as_std_path(),
+            "",
+        )?;
+        std::fs::write(
+            context
+                .join_root_path("typeshed/stdlib/os.pyi")
+                .as_std_path(),
+            "# not important",
+        )?;
 
-            Ok(())
-        },
-        |context| {
-            Some(Options {
-                environment: Some(EnvironmentOptions {
-                    typeshed: Some(RelativePathBuf::cli(context.join_root_path("typeshed"))),
-                    ..EnvironmentOptions::default()
-                }),
-                ..Options::default()
-            })
-        },
-    )?;
+        context.set_options(Options {
+            environment: Some(EnvironmentOptions {
+                typeshed: Some(RelativePathBuf::cli(context.join_root_path("typeshed"))),
+                ..EnvironmentOptions::default()
+            }),
+            ..Options::default()
+        });
+
+        Ok(())
+    })?;
 
     // Unset the custom typeshed directory.
     assert_eq!(
@@ -1056,7 +1229,7 @@ fn changed_versions_file() -> anyhow::Result<()> {
 /// we're seeing is that Windows only emits a single event, similar to Linux.
 #[test]
 fn hard_links_in_project() -> anyhow::Result<()> {
-    let mut case = setup(|context: &SetupContext| {
+    let mut case = setup(|context: &mut SetupContext| {
         let foo_path = context.join_project_path("foo.py");
         std::fs::write(foo_path.as_std_path(), "print('Version 1')")?;
 
@@ -1075,6 +1248,7 @@ fn hard_links_in_project() -> anyhow::Result<()> {
 
     assert_eq!(source_text(case.db(), foo).as_str(), "print('Version 1')");
     assert_eq!(source_text(case.db(), bar).as_str(), "print('Version 1')");
+    case.assert_indexed_project_files([bar, foo]);
 
     // Write to the hard link target.
     update_file(foo_path, "print('Version 2')").context("Failed to update foo.py")?;
@@ -1127,7 +1301,7 @@ fn hard_links_in_project() -> anyhow::Result<()> {
     ignore = "windows doesn't support observing changes to hard linked files."
 )]
 fn hard_links_to_target_outside_project() -> anyhow::Result<()> {
-    let mut case = setup(|context: &SetupContext| {
+    let mut case = setup(|context: &mut SetupContext| {
         let foo_path = context.join_root_path("foo.py");
         std::fs::write(foo_path.as_std_path(), "print('Version 1')")?;
 
@@ -1235,7 +1409,7 @@ mod unix {
         ignore = "FSEvents doesn't emit change events for symlinked directories outside of the watched paths."
     )]
     fn symlink_target_outside_watched_paths() -> anyhow::Result<()> {
-        let mut case = setup(|context: &SetupContext| {
+        let mut case = setup(|context: &mut SetupContext| {
             // Set up the symlink target.
             let link_target = context.join_root_path("bar");
             std::fs::create_dir_all(link_target.as_std_path())
@@ -1316,7 +1490,7 @@ mod unix {
     /// ```
     #[test]
     fn symlink_inside_project() -> anyhow::Result<()> {
-        let mut case = setup(|context: &SetupContext| {
+        let mut case = setup(|context: &mut SetupContext| {
             // Set up the symlink target.
             let link_target = context.join_project_path("patched/bar");
             std::fs::create_dir_all(link_target.as_std_path())
@@ -1354,6 +1528,8 @@ mod unix {
         );
         assert_eq!(baz.file().path(case.db()).as_system_path(), Some(&*bar_baz));
 
+        case.assert_indexed_project_files([patched_bar_baz_file]);
+
         // Write to the symlink target.
         update_file(&patched_bar_baz, "def baz(): print('Version 2')")
             .context("Failed to update bar/baz.py")?;
@@ -1389,6 +1565,7 @@ mod unix {
             bar_baz_text = bar_baz_text.as_str()
         );
 
+        case.assert_indexed_project_files([patched_bar_baz_file]);
         Ok(())
     }
 
@@ -1406,43 +1583,39 @@ mod unix {
     /// ```
     #[test]
     fn symlinked_module_search_path() -> anyhow::Result<()> {
-        let mut case = setup_with_options(
-            |context: &SetupContext| {
-                // Set up the symlink target.
-                let site_packages = context.join_root_path("site-packages");
-                let bar = site_packages.join("bar");
-                std::fs::create_dir_all(bar.as_std_path())
-                    .context("Failed to create bar directory")?;
-                let baz_original = bar.join("baz.py");
-                std::fs::write(baz_original.as_std_path(), "def baz(): ...")
-                    .context("Failed to write baz.py")?;
+        let mut case = setup(|context: &mut SetupContext| {
+            // Set up the symlink target.
+            let site_packages = context.join_root_path("site-packages");
+            let bar = site_packages.join("bar");
+            std::fs::create_dir_all(bar.as_std_path()).context("Failed to create bar directory")?;
+            let baz_original = bar.join("baz.py");
+            std::fs::write(baz_original.as_std_path(), "def baz(): ...")
+                .context("Failed to write baz.py")?;
 
-                // Symlink the site packages in the venv to the global site packages
-                let venv_site_packages =
-                    context.join_project_path(".venv/lib/python3.12/site-packages");
-                std::fs::create_dir_all(venv_site_packages.parent().unwrap())
-                    .context("Failed to create .venv directory")?;
-                std::os::unix::fs::symlink(
-                    site_packages.as_std_path(),
-                    venv_site_packages.as_std_path(),
-                )
-                .context("Failed to create symlink to site-packages")?;
+            // Symlink the site packages in the venv to the global site packages
+            let venv_site_packages =
+                context.join_project_path(".venv/lib/python3.12/site-packages");
+            std::fs::create_dir_all(venv_site_packages.parent().unwrap())
+                .context("Failed to create .venv directory")?;
+            std::os::unix::fs::symlink(
+                site_packages.as_std_path(),
+                venv_site_packages.as_std_path(),
+            )
+            .context("Failed to create symlink to site-packages")?;
 
-                Ok(())
-            },
-            |_context| {
-                Some(Options {
-                    environment: Some(EnvironmentOptions {
-                        extra_paths: Some(vec![RelativePathBuf::cli(
-                            ".venv/lib/python3.12/site-packages",
-                        )]),
-                        python_version: Some(RangedValue::cli(PythonVersion::PY312)),
-                        ..EnvironmentOptions::default()
-                    }),
-                    ..Options::default()
-                })
-            },
-        )?;
+            context.set_options(Options {
+                environment: Some(EnvironmentOptions {
+                    extra_paths: Some(vec![RelativePathBuf::cli(
+                        ".venv/lib/python3.12/site-packages",
+                    )]),
+                    python_version: Some(RangedValue::cli(PythonVersion::PY312)),
+                    ..EnvironmentOptions::default()
+                }),
+                ..Options::default()
+            });
+
+            Ok(())
+        })?;
 
         let baz = resolve_module(
             case.db().upcast(),
@@ -1469,6 +1642,8 @@ mod unix {
             Some(&*baz_original)
         );
 
+        case.assert_indexed_project_files([]);
+
         // Write to the symlink target.
         update_file(&baz_original, "def baz(): print('Version 2')")
             .context("Failed to update bar/baz.py")?;
@@ -1494,13 +1669,15 @@ mod unix {
             "def baz(): print('Version 2')"
         );
 
+        case.assert_indexed_project_files([]);
+
         Ok(())
     }
 }
 
 #[test]
 fn nested_projects_delete_root() -> anyhow::Result<()> {
-    let mut case = setup(|context: &SetupContext| {
+    let mut case = setup(|context: &mut SetupContext| {
         std::fs::write(
             context.join_project_path("pyproject.toml").as_std_path(),
             r#"
@@ -1542,7 +1719,7 @@ fn nested_projects_delete_root() -> anyhow::Result<()> {
 fn changes_to_user_configuration() -> anyhow::Result<()> {
     let mut _config_dir_override: Option<UserConfigDirectoryOverrideGuard> = None;
 
-    let mut case = setup(|context: &SetupContext| {
+    let mut case = setup(|context: &mut SetupContext| {
         std::fs::write(
             context.join_project_path("pyproject.toml").as_std_path(),
             r#"

crates/red_knot_project/src/combine.rs

@@ -1,6 +1,6 @@
 use std::{collections::HashMap, hash::BuildHasher};
 
-use red_knot_python_semantic::{PythonPlatform, SitePackages};
+use red_knot_python_semantic::{PythonPath, PythonPlatform};
 use ruff_db::system::SystemPathBuf;
 use ruff_python_ast::PythonVersion;
 
@@ -128,7 +128,7 @@ macro_rules! impl_noop_combine {
 
 impl_noop_combine!(SystemPathBuf);
 impl_noop_combine!(PythonPlatform);
-impl_noop_combine!(SitePackages);
+impl_noop_combine!(PythonPath);
 impl_noop_combine!(PythonVersion);
 
 // std types

crates/red_knot_project/src/db.rs

@@ -5,7 +5,7 @@ use crate::DEFAULT_LINT_REGISTRY;
 use crate::{Project, ProjectMetadata};
 use red_knot_python_semantic::lint::{LintRegistry, RuleSelection};
 use red_knot_python_semantic::{Db as SemanticDb, Program};
-use ruff_db::diagnostic::Diagnostic;
+use ruff_db::diagnostic::OldDiagnosticTrait;
 use ruff_db::files::{File, Files};
 use ruff_db::system::System;
 use ruff_db::vendored::VendoredFileSystem;
@@ -55,11 +55,11 @@ impl ProjectDatabase {
     }
 
     /// Checks all open files in the project and its dependencies.
-    pub fn check(&self) -> Result<Vec<Box<dyn Diagnostic>>, Cancelled> {
+    pub fn check(&self) -> Result<Vec<Box<dyn OldDiagnosticTrait>>, Cancelled> {
         self.with_db(|db| db.project().check(db))
     }
 
-    pub fn check_file(&self, file: File) -> Result<Vec<Box<dyn Diagnostic>>, Cancelled> {
+    pub fn check_file(&self, file: File) -> Result<Vec<Box<dyn OldDiagnosticTrait>>, Cancelled> {
         let _span = tracing::debug_span!("check_file", file=%file.path(self)).entered();
 
         self.with_db(|db| self.project().check_file(db, file))

crates/red_knot_project/src/db/changes.rs

@@ -2,20 +2,20 @@ use crate::db::{Db, ProjectDatabase};
 use crate::metadata::options::Options;
 use crate::watch::{ChangeEvent, CreatedKind, DeletedKind};
 use crate::{Project, ProjectMetadata};
+use std::collections::BTreeSet;
 
+use crate::walk::ProjectFilesWalker;
 use red_knot_python_semantic::Program;
-use ruff_db::files::{system_path_to_file, File, Files};
-use ruff_db::system::walk_directory::WalkState;
+use ruff_db::files::{File, Files};
 use ruff_db::system::SystemPath;
 use ruff_db::Db as _;
-use ruff_python_ast::PySourceType;
 use rustc_hash::FxHashSet;
 
 impl ProjectDatabase {
     #[tracing::instrument(level = "debug", skip(self, changes, cli_options))]
     pub fn apply_changes(&mut self, changes: Vec<ChangeEvent>, cli_options: Option<&Options>) {
         let mut project = self.project();
-        let project_path = project.root(self).to_path_buf();
+        let project_root = project.root(self).to_path_buf();
         let program = Program::get(self);
         let custom_stdlib_versions_path = program
             .custom_stdlib_search_path(self)
@@ -30,7 +30,7 @@ impl ProjectDatabase {
 
         // Deduplicate the `sync` calls. Many file watchers emit multiple events for the same path.
         let mut synced_files = FxHashSet::default();
-        let mut synced_recursively = FxHashSet::default();
+        let mut sync_recursively = BTreeSet::default();
 
         let mut sync_path = |db: &mut ProjectDatabase, path: &SystemPath| {
             if synced_files.insert(path.to_path_buf()) {
@@ -38,13 +38,9 @@ impl ProjectDatabase {
             }
         };
 
-        let mut sync_recursively = |db: &mut ProjectDatabase, path: &SystemPath| {
-            if synced_recursively.insert(path.to_path_buf()) {
-                Files::sync_recursively(db, path);
-            }
-        };
-
         for change in changes {
+            tracing::trace!("Handle change: {:?}", change);
+
             if let Some(path) = change.system_path() {
                 if matches!(
                     path.file_name(),
@@ -70,16 +66,27 @@ impl ProjectDatabase {
                     match kind {
                         CreatedKind::File => sync_path(self, &path),
                         CreatedKind::Directory | CreatedKind::Any => {
-                            sync_recursively(self, &path);
+                            sync_recursively.insert(path.clone());
                         }
                     }
 
-                    if self.system().is_file(&path) {
-                        // Add the parent directory because `walkdir` always visits explicitly passed files
-                        // even if they match an exclude filter.
-                        added_paths.insert(path.parent().unwrap().to_path_buf());
-                    } else {
-                        added_paths.insert(path);
+                    // Unlike other files, it's not only important to update the status of existing
+                    // and known `File`s (`sync_recursively`), it's also important to discover new files
+                    // that were added in the project's root (or any of the paths included for checking).
+                    //
+                    // This is important because `Project::check` iterates over all included files.
+                    // The code below walks the `added_paths` and adds all files that
+                    // should be included in the project. We can skip this check for
+                    // paths that aren't part of the project or shouldn't be included
+                    // when checking the project.
+                    if project.is_path_included(self, &path) {
+                        if self.system().is_file(&path) {
+                            // Add the parent directory because `walkdir` always visits explicitly passed files
+                            // even if they match an exclude filter.
+                            added_paths.insert(path.parent().unwrap().to_path_buf());
+                        } else {
+                            added_paths.insert(path);
+                        }
                     }
                 }
 
@@ -103,7 +110,7 @@ impl ProjectDatabase {
                             project.remove_file(self, file);
                         }
                     } else {
-                        sync_recursively(self, &path);
+                        sync_recursively.insert(path.clone());
 
                         if custom_stdlib_versions_path
                             .as_ref()
@@ -112,11 +119,19 @@ impl ProjectDatabase {
                             custom_stdlib_change = true;
                         }
 
-                        // Perform a full-reload in case the deleted directory contained the pyproject.toml.
-                        // We may want to make this more clever in the future, to e.g. iterate over the
-                        // indexed files and remove the once that start with the same path, unless
-                        // the deleted path is the project configuration.
-                        project_changed = true;
+                        if project.is_path_included(self, &path) || path == project_root {
+                            // TODO: Shouldn't it be enough to simply traverse the project files and remove all
+                            // that start with the given path?
+                            tracing::debug!(
+                                "Reload project because of a path that could have been a directory."
+                            );
+
+                            // Perform a full-reload in case the deleted directory contained the pyproject.toml.
+                            // We may want to make this more clever in the future, to e.g. iterate over the
+                            // indexed files and remove the once that start with the same path, unless
+                            // the deleted path is the project configuration.
+                            project_changed = true;
+                        }
                     }
                 }
 
@@ -133,13 +148,29 @@ impl ProjectDatabase {
                 ChangeEvent::Rescan => {
                     project_changed = true;
                     Files::sync_all(self);
+                    sync_recursively.clear();
                     break;
                 }
             }
         }
 
+        let sync_recursively = sync_recursively.into_iter();
+        let mut last = None;
+
+        for path in sync_recursively {
+            // Avoid re-syncing paths that are sub-paths of each other.
+            if let Some(last) = &last {
+                if path.starts_with(last) {
+                    continue;
+                }
+            }
+
+            Files::sync_recursively(self, &path);
+            last = Some(path);
+        }
+
         if project_changed {
-            match ProjectMetadata::discover(&project_path, self.system()) {
+            match ProjectMetadata::discover(&project_root, self.system()) {
                 Ok(mut metadata) => {
                     if let Some(cli_options) = cli_options {
                         metadata.apply_cli_options(cli_options.clone());
@@ -186,50 +217,24 @@ impl ProjectDatabase {
             }
         }
 
-        let mut added_paths = added_paths.into_iter();
+        let diagnostics = if let Some(walker) = ProjectFilesWalker::incremental(self, added_paths) {
+            // Use directory walking to discover newly added files.
+            let (files, diagnostics) = walker.collect_vec(self);
 
-        // Use directory walking to discover newly added files.
-        if let Some(path) = added_paths.next() {
-            let mut walker = self.system().walk_directory(&path);
-
-            for extra_path in added_paths {
-                walker = walker.add(&extra_path);
+            for file in files {
+                project.add_file(self, file);
             }
 
-            let added_paths = std::sync::Mutex::new(Vec::default());
+            diagnostics
+        } else {
+            Vec::new()
+        };
 
-            walker.run(|| {
-                Box::new(|entry| {
-                    let Ok(entry) = entry else {
-                        return WalkState::Continue;
-                    };
-
-                    if !entry.file_type().is_file() {
-                        return WalkState::Continue;
-                    }
-
-                    if entry
-                        .path()
-                        .extension()
-                        .and_then(PySourceType::try_from_extension)
-                        .is_some()
-                    {
-                        let mut paths = added_paths.lock().unwrap();
-
-                        paths.push(entry.into_path());
-                    }
-
-                    WalkState::Continue
-                })
-            });
-
-            for path in added_paths.into_inner().unwrap() {
-                let file = system_path_to_file(self, &path);
-
-                if let Ok(file) = file {
-                    project.add_file(self, file);
-                }
-            }
-        }
+        // Note: We simply replace all IO related diagnostics here. This isn't ideal, because
+        // it removes IO errors that may still be relevant. However, tracking IO errors correctly
+        // across revisions doesn't feel essential, considering that they're rare. However, we could
+        // implement a `BTreeMap` or similar and only prune the diagnostics from paths that we've
+        // re-scanned (or that were removed etc).
+        project.replace_index_diagnostics(self, diagnostics);
     }
 }

crates/red_knot_project/src/files.rs

@@ -8,10 +8,7 @@ use salsa::Setter;
 use ruff_db::files::File;
 
 use crate::db::Db;
-use crate::Project;
-
-/// Cheap cloneable hash set of files.
-type FileSet = Arc<FxHashSet<File>>;
+use crate::{IOErrorDiagnostic, Project};
 
 /// The indexed files of a project.
 ///
@@ -35,9 +32,9 @@ impl IndexedFiles {
         }
     }
 
-    fn indexed(files: FileSet) -> Self {
+    fn indexed(inner: Arc<IndexedInner>) -> Self {
         Self {
-            state: std::sync::Mutex::new(State::Indexed(files)),
+            state: std::sync::Mutex::new(State::Indexed(inner)),
         }
     }
 
@@ -46,8 +43,8 @@ impl IndexedFiles {
 
         match &*state {
             State::Lazy => Index::Lazy(LazyFiles { files: state }),
-            State::Indexed(files) => Index::Indexed(Indexed {
-                files: Arc::clone(files),
+            State::Indexed(inner) => Index::Indexed(Indexed {
+                inner: Arc::clone(inner),
                 _lifetime: PhantomData,
             }),
         }
@@ -94,7 +91,7 @@ impl IndexedFiles {
         Some(IndexedMut {
             db: Some(db),
             project,
-            files: indexed,
+            indexed,
             did_change: false,
         })
     }
@@ -112,7 +109,7 @@ enum State {
     Lazy,
 
     /// The files are indexed. Stores the known files of a package.
-    Indexed(FileSet),
+    Indexed(Arc<IndexedInner>),
 }
 
 pub(super) enum Index<'db> {
@@ -129,32 +126,48 @@ pub(super) struct LazyFiles<'db> {
 
 impl<'db> LazyFiles<'db> {
     /// Sets the indexed files of a package to `files`.
-    pub(super) fn set(mut self, files: FxHashSet<File>) -> Indexed<'db> {
+    pub(super) fn set(
+        mut self,
+        files: FxHashSet<File>,
+        diagnostics: Vec<IOErrorDiagnostic>,
+    ) -> Indexed<'db> {
         let files = Indexed {
-            files: Arc::new(files),
+            inner: Arc::new(IndexedInner { files, diagnostics }),
             _lifetime: PhantomData,
         };
-        *self.files = State::Indexed(Arc::clone(&files.files));
+        *self.files = State::Indexed(Arc::clone(&files.inner));
         files
     }
 }
 
-/// The indexed files of a package.
+/// The indexed files of the project.
 ///
 /// Note: This type is intentionally non-cloneable. Making it cloneable requires
 /// revisiting the locking behavior in [`IndexedFiles::indexed_mut`].
-#[derive(Debug, PartialEq, Eq)]
+#[derive(Debug)]
 pub struct Indexed<'db> {
-    files: FileSet,
+    inner: Arc<IndexedInner>,
     // Preserve the lifetime of `PackageFiles`.
     _lifetime: PhantomData<&'db ()>,
 }
 
+#[derive(Debug)]
+struct IndexedInner {
+    files: FxHashSet<File>,
+    diagnostics: Vec<IOErrorDiagnostic>,
+}
+
+impl Indexed<'_> {
+    pub(super) fn diagnostics(&self) -> &[IOErrorDiagnostic] {
+        &self.inner.diagnostics
+    }
+}
+
 impl Deref for Indexed<'_> {
     type Target = FxHashSet<File>;
 
     fn deref(&self) -> &Self::Target {
-        &self.files
+        &self.inner.files
     }
 }
 
@@ -165,7 +178,7 @@ impl<'a> IntoIterator for &'a Indexed<'_> {
     type IntoIter = IndexedIter<'a>;
 
     fn into_iter(self) -> Self::IntoIter {
-        self.files.iter().copied()
+        self.inner.files.iter().copied()
     }
 }
 
@@ -176,13 +189,13 @@ impl<'a> IntoIterator for &'a Indexed<'_> {
 pub(super) struct IndexedMut<'db> {
     db: Option<&'db mut dyn Db>,
     project: Project,
-    files: FileSet,
+    indexed: Arc<IndexedInner>,
     did_change: bool,
 }
 
 impl IndexedMut<'_> {
     pub(super) fn insert(&mut self, file: File) -> bool {
-        if self.files_mut().insert(file) {
+        if self.inner_mut().files.insert(file) {
             self.did_change = true;
             true
         } else {
@@ -191,7 +204,7 @@ impl IndexedMut<'_> {
     }
 
     pub(super) fn remove(&mut self, file: File) -> bool {
-        if self.files_mut().remove(&file) {
+        if self.inner_mut().files.remove(&file) {
             self.did_change = true;
             true
         } else {
@@ -199,8 +212,13 @@ impl IndexedMut<'_> {
         }
     }
 
-    fn files_mut(&mut self) -> &mut FxHashSet<File> {
-        Arc::get_mut(&mut self.files).expect("All references to `FilesSet` to have been dropped")
+    pub(super) fn set_diagnostics(&mut self, diagnostics: Vec<IOErrorDiagnostic>) {
+        self.inner_mut().diagnostics = diagnostics;
+    }
+
+    fn inner_mut(&mut self) -> &mut IndexedInner {
+        Arc::get_mut(&mut self.indexed)
+            .expect("All references to `FilesSet` should have been dropped")
     }
 
     fn set_impl(&mut self) {
@@ -208,16 +226,16 @@ impl IndexedMut<'_> {
             return;
         };
 
-        let files = Arc::clone(&self.files);
+        let indexed = Arc::clone(&self.indexed);
 
         if self.did_change {
             // If there are changes, set the new file_set to trigger a salsa revision change.
             self.project
                 .set_file_set(db)
-                .to(IndexedFiles::indexed(files));
+                .to(IndexedFiles::indexed(indexed));
         } else {
             // The `indexed_mut` replaced the `state` with Lazy. Restore it back to the indexed state.
-            *self.project.file_set(db).state.lock().unwrap() = State::Indexed(files);
+            *self.project.file_set(db).state.lock().unwrap() = State::Indexed(indexed);
         }
     }
 }
@@ -237,7 +255,7 @@ mod tests {
     use crate::files::Index;
     use crate::ProjectMetadata;
     use ruff_db::files::system_path_to_file;
-    use ruff_db::system::{DbWithTestSystem, SystemPathBuf};
+    use ruff_db::system::{DbWithWritableSystem as _, SystemPathBuf};
     use ruff_python_ast::name::Name;
 
     #[test]
@@ -252,7 +270,7 @@ mod tests {
         let file = system_path_to_file(&db, "test.py").unwrap();
 
         let files = match project.file_set(&db).get() {
-            Index::Lazy(lazy) => lazy.set(FxHashSet::from_iter([file])),
+            Index::Lazy(lazy) => lazy.set(FxHashSet::from_iter([file]), Vec::new()),
             Index::Indexed(files) => files,
         };
 

crates/red_knot_project/src/lib.rs

@@ -1,6 +1,7 @@
 #![allow(clippy::ref_option)]
 
 use crate::metadata::options::OptionDiagnostic;
+use crate::walk::{ProjectFilesFilter, ProjectFilesWalker};
 pub use db::{Db, ProjectDatabase};
 use files::{Index, Indexed, IndexedFiles};
 use metadata::settings::Settings;
@@ -8,24 +9,24 @@ pub use metadata::{ProjectDiscoveryError, ProjectMetadata};
 use red_knot_python_semantic::lint::{LintRegistry, LintRegistryBuilder, RuleSelection};
 use red_knot_python_semantic::register_lints;
 use red_knot_python_semantic::types::check_types;
-use ruff_db::diagnostic::{Diagnostic, DiagnosticId, ParseDiagnostic, Severity, Span};
-use ruff_db::files::{system_path_to_file, File};
+use ruff_db::diagnostic::{DiagnosticId, OldDiagnosticTrait, OldParseDiagnostic, Severity, Span};
+use ruff_db::files::File;
 use ruff_db::parsed::parsed_module;
 use ruff_db::source::{source_text, SourceTextError};
-use ruff_db::system::walk_directory::WalkState;
-use ruff_db::system::{FileType, SystemPath};
-use ruff_python_ast::PySourceType;
-use rustc_hash::{FxBuildHasher, FxHashSet};
+use ruff_db::system::{SystemPath, SystemPathBuf};
+use rustc_hash::FxHashSet;
 use salsa::Durability;
 use salsa::Setter;
 use std::borrow::Cow;
 use std::sync::Arc;
+use thiserror::Error;
 
 pub mod combine;
 
 mod db;
 mod files;
 pub mod metadata;
+mod walk;
 pub mod watch;
 
 pub static DEFAULT_LINT_REGISTRY: std::sync::LazyLock<LintRegistry> =
@@ -71,6 +72,30 @@ pub struct Project {
     #[return_ref]
     pub settings: Settings,
 
+    /// The paths that should be included when checking this project.
+    ///
+    /// The default (when this list is empty) is to include all files in the project root
+    /// (that satisfy the configured include and exclude patterns).
+    /// However, it's sometimes desired to only check a subset of the project, e.g. to see
+    /// the diagnostics for a single file or a folder.
+    ///
+    /// This list gets initialized by the paths passed to `knot check <paths>`
+    ///
+    /// ## How is this different from `open_files`?
+    ///
+    /// The `included_paths` is closely related to `open_files`. The only difference is that
+    /// `open_files` is already a resolved set of files whereas `included_paths` is only a list of paths
+    /// that are resolved to files by indexing them. The other difference is that
+    /// new files added to any directory in `included_paths` will be indexed and added to the project
+    /// whereas `open_files` needs to be updated manually (e.g. by the IDE).
+    ///
+    /// In short, `open_files` is cheaper in contexts where the set of files is known, like
+    /// in an IDE when the user only wants to check the open tabs. This could be modeled
+    /// with `included_paths` too but it would require an explicit walk dir step that's simply unnecessary.
+    #[default]
+    #[return_ref]
+    included_paths_list: Vec<SystemPathBuf>,
+
     /// Diagnostics that were generated when resolving the project settings.
     #[return_ref]
     settings_diagnostics: Vec<OptionDiagnostic>,
@@ -106,6 +131,16 @@ impl Project {
         self.settings(db).to_rules()
     }
 
+    /// Returns `true` if `path` is both part of the project and included (see `included_paths_list`).
+    ///
+    /// Unlike [Self::files], this method does not respect `.gitignore` files. It only checks
+    /// the project's include and exclude settings as well as the paths that were passed to `knot check <paths>`.
+    /// This means, that this method is an over-approximation of `Self::files` and may return `true` for paths
+    /// that won't be included when checking the project because they're ignored in a `.gitignore` file.
+    pub fn is_path_included(self, db: &dyn Db, path: &SystemPath) -> bool {
+        ProjectFilesFilter::from_project(db, self).is_included(path)
+    }
+
     pub fn reload(self, db: &mut dyn Db, metadata: ProjectMetadata) {
         tracing::debug!("Reloading project");
         assert_eq!(self.root(db), metadata.root());
@@ -128,15 +163,22 @@ impl Project {
     }
 
     /// Checks all open files in the project and its dependencies.
-    pub(crate) fn check(self, db: &ProjectDatabase) -> Vec<Box<dyn Diagnostic>> {
+    pub(crate) fn check(self, db: &ProjectDatabase) -> Vec<Box<dyn OldDiagnosticTrait>> {
         let project_span = tracing::debug_span!("Project::check");
         let _span = project_span.enter();
 
         tracing::debug!("Checking project '{name}'", name = self.name(db));
 
-        let mut diagnostics: Vec<Box<dyn Diagnostic>> = Vec::new();
+        let mut diagnostics: Vec<Box<dyn OldDiagnosticTrait>> = Vec::new();
         diagnostics.extend(self.settings_diagnostics(db).iter().map(|diagnostic| {
-            let diagnostic: Box<dyn Diagnostic> = Box::new(diagnostic.clone());
+            let diagnostic: Box<dyn OldDiagnosticTrait> = Box::new(diagnostic.clone());
+            diagnostic
+        }));
+
+        let files = ProjectFiles::new(db, self);
+
+        diagnostics.extend(files.diagnostics().iter().cloned().map(|diagnostic| {
+            let diagnostic: Box<dyn OldDiagnosticTrait> = Box::new(diagnostic);
             diagnostic
         }));
 
@@ -147,7 +189,6 @@ impl Project {
         let project_span = project_span.clone();
 
         rayon::scope(move |scope| {
-            let files = ProjectFiles::new(&db, self);
             for file in &files {
                 let result = inner_result.clone();
                 let db = db.clone();
@@ -166,12 +207,12 @@ impl Project {
         Arc::into_inner(result).unwrap().into_inner().unwrap()
     }
 
-    pub(crate) fn check_file(self, db: &dyn Db, file: File) -> Vec<Box<dyn Diagnostic>> {
+    pub(crate) fn check_file(self, db: &dyn Db, file: File) -> Vec<Box<dyn OldDiagnosticTrait>> {
         let mut file_diagnostics: Vec<_> = self
             .settings_diagnostics(db)
             .iter()
             .map(|diagnostic| {
-                let diagnostic: Box<dyn Diagnostic> = Box::new(diagnostic.clone());
+                let diagnostic: Box<dyn OldDiagnosticTrait> = Box::new(diagnostic.clone());
                 diagnostic
             })
             .collect();
@@ -207,6 +248,30 @@ impl Project {
         removed
     }
 
+    pub fn set_included_paths(self, db: &mut dyn Db, paths: Vec<SystemPathBuf>) {
+        tracing::debug!("Setting included paths: {paths}", paths = paths.len());
+
+        self.set_included_paths_list(db).to(paths);
+        self.reload_files(db);
+    }
+
+    /// Returns the paths that should be checked.
+    ///
+    /// The default is to check the entire project in which case this method returns
+    /// the project root. However, users can specify to only check specific sub-folders or
+    /// even files of a project by using `knot check <paths>`. In that case, this method
+    /// returns the provided absolute paths.
+    ///
+    /// Note: The CLI doesn't prohibit users from specifying paths outside the project root.
+    /// This can be useful to check arbitrary files, but it isn't something we recommend.
+    /// We should try to support this use case but it's okay if there are some limitations around it.
+    fn included_paths_or_root(self, db: &dyn Db) -> &[SystemPathBuf] {
+        match &**self.included_paths_list(db) {
+            [] => std::slice::from_ref(&self.metadata(db).root),
+            paths => paths,
+        }
+    }
+
     /// Returns the open files in the project or `None` if the entire project should be checked.
     pub fn open_files(self, db: &dyn Db) -> Option<&FxHashSet<File>> {
         self.open_fileset(db).as_deref()
@@ -289,6 +354,17 @@ impl Project {
         index.insert(file);
     }
 
+    /// Replaces the diagnostics from indexing the project files with `diagnostics`.
+    ///
+    /// This is a no-op if the project files haven't been indexed yet.
+    pub fn replace_index_diagnostics(self, db: &mut dyn Db, diagnostics: Vec<IOErrorDiagnostic>) {
+        let Some(mut index) = IndexedFiles::indexed_mut(db, self) else {
+            return;
+        };
+
+        index.set_diagnostics(diagnostics);
+    }
+
     /// Returns the files belonging to this project.
     pub fn files(self, db: &dyn Db) -> Indexed<'_> {
         let files = self.file_set(db);
@@ -296,12 +372,14 @@ impl Project {
         let indexed = match files.get() {
             Index::Lazy(vacant) => {
                 let _entered =
-                    tracing::debug_span!("Project::index_files", package = %self.name(db))
+                    tracing::debug_span!("Project::index_files", project = %self.name(db))
                         .entered();
 
-                let files = discover_project_files(db, self);
-                tracing::info!("Found {} files in project `{}`", files.len(), self.name(db));
-                vacant.set(files)
+                let walker = ProjectFilesWalker::new(db);
+                let (files, diagnostics) = walker.collect_set(db);
+
+                tracing::info!("Indexed {} file(s)", files.len());
+                vacant.set(files, diagnostics)
             }
             Index::Indexed(indexed) => indexed,
         };
@@ -319,28 +397,29 @@ impl Project {
     }
 }
 
-fn check_file_impl(db: &dyn Db, file: File) -> Vec<Box<dyn Diagnostic>> {
-    let mut diagnostics: Vec<Box<dyn Diagnostic>> = Vec::new();
+fn check_file_impl(db: &dyn Db, file: File) -> Vec<Box<dyn OldDiagnosticTrait>> {
+    let mut diagnostics: Vec<Box<dyn OldDiagnosticTrait>> = Vec::new();
 
     // Abort checking if there are IO errors.
     let source = source_text(db.upcast(), file);
 
     if let Some(read_error) = source.read_error() {
         diagnostics.push(Box::new(IOErrorDiagnostic {
-            file,
-            error: read_error.clone(),
+            file: Some(file),
+            error: read_error.clone().into(),
         }));
         return diagnostics;
     }
 
     let parsed = parsed_module(db.upcast(), file);
     diagnostics.extend(parsed.errors().iter().map(|error| {
-        let diagnostic: Box<dyn Diagnostic> = Box::new(ParseDiagnostic::new(file, error.clone()));
+        let diagnostic: Box<dyn OldDiagnosticTrait> =
+            Box::new(OldParseDiagnostic::new(file, error.clone()));
         diagnostic
     }));
 
     diagnostics.extend(check_types(db.upcast(), file).iter().map(|diagnostic| {
-        let boxed: Box<dyn Diagnostic> = Box::new(diagnostic.clone());
+        let boxed: Box<dyn OldDiagnosticTrait> = Box::new(diagnostic.clone());
         boxed
     }));
 
@@ -355,53 +434,6 @@ fn check_file_impl(db: &dyn Db, file: File) -> Vec<Box<dyn Diagnostic>> {
     diagnostics
 }
 
-fn discover_project_files(db: &dyn Db, project: Project) -> FxHashSet<File> {
-    let paths = std::sync::Mutex::new(Vec::new());
-
-    db.system().walk_directory(project.root(db)).run(|| {
-        Box::new(|entry| {
-            match entry {
-                Ok(entry) => {
-                    // Skip over any non python files to avoid creating too many entries in `Files`.
-                    match entry.file_type() {
-                        FileType::File => {
-                            if entry
-                                .path()
-                                .extension()
-                                .and_then(PySourceType::try_from_extension)
-                                .is_some()
-                            {
-                                let mut paths = paths.lock().unwrap();
-                                paths.push(entry.into_path());
-                            }
-                        }
-                        FileType::Directory | FileType::Symlink => {}
-                    }
-                }
-                Err(error) => {
-                    // TODO Handle error
-                    tracing::error!("Failed to walk path: {error}");
-                }
-            }
-
-            WalkState::Continue
-        })
-    });
-
-    let paths = paths.into_inner().unwrap();
-    let mut files = FxHashSet::with_capacity_and_hasher(paths.len(), FxBuildHasher);
-
-    for path in paths {
-        // If this returns `None`, then the file was deleted between the `walk_directory` call and now.
-        // We can ignore this.
-        if let Ok(file) = system_path_to_file(db.upcast(), &path) {
-            files.insert(file);
-        }
-    }
-
-    files
-}
-
 #[derive(Debug)]
 enum ProjectFiles<'a> {
     OpenFiles(&'a FxHashSet<File>),
@@ -416,6 +448,13 @@ impl<'a> ProjectFiles<'a> {
             ProjectFiles::Indexed(project.files(db))
         }
     }
+
+    fn diagnostics(&self) -> &[IOErrorDiagnostic] {
+        match self {
+            ProjectFiles::OpenFiles(_) => &[],
+            ProjectFiles::Indexed(indexed) => indexed.diagnostics(),
+        }
+    }
 }
 
 impl<'a> IntoIterator for &'a ProjectFiles<'a> {
@@ -448,13 +487,13 @@ impl Iterator for ProjectFilesIter<'_> {
     }
 }
 
-#[derive(Debug)]
+#[derive(Debug, Clone)]
 pub struct IOErrorDiagnostic {
-    file: File,
-    error: SourceTextError,
+    file: Option<File>,
+    error: IOErrorKind,
 }
 
-impl Diagnostic for IOErrorDiagnostic {
+impl OldDiagnosticTrait for IOErrorDiagnostic {
     fn id(&self) -> DiagnosticId {
         DiagnosticId::Io
     }
@@ -464,7 +503,7 @@ impl Diagnostic for IOErrorDiagnostic {
     }
 
     fn span(&self) -> Option<Span> {
-        Some(Span::from(self.file))
+        self.file.map(Span::from)
     }
 
     fn severity(&self) -> Severity {
@@ -472,15 +511,24 @@ impl Diagnostic for IOErrorDiagnostic {
     }
 }
 
+#[derive(Error, Debug, Clone)]
+enum IOErrorKind {
+    #[error(transparent)]
+    Walk(#[from] walk::WalkError),
+
+    #[error(transparent)]
+    SourceText(#[from] SourceTextError),
+}
+
 #[cfg(test)]
 mod tests {
     use crate::db::tests::TestDb;
     use crate::{check_file_impl, ProjectMetadata};
     use red_knot_python_semantic::types::check_types;
-    use ruff_db::diagnostic::Diagnostic;
+    use ruff_db::diagnostic::OldDiagnosticTrait;
     use ruff_db::files::system_path_to_file;
     use ruff_db::source::source_text;
-    use ruff_db::system::{DbWithTestSystem, SystemPath, SystemPathBuf};
+    use ruff_db::system::{DbWithTestSystem, DbWithWritableSystem as _, SystemPath, SystemPathBuf};
     use ruff_db::testing::assert_function_query_was_not_run;
     use ruff_python_ast::name::Name;
 

crates/red_knot_project/src/metadata.rs

@@ -77,10 +77,10 @@ impl ProjectMetadata {
         // If the `options` don't specify a python version but the `project.requires-python` field is set,
         // use that as a lower bound instead.
         if let Some(project) = project {
-            if !options
+            if options
                 .environment
                 .as_ref()
-                .is_some_and(|env| env.python_version.is_some())
+                .is_none_or(|env| env.python_version.is_none())
             {
                 if let Some(requires_python) = project.resolve_requires_python_lower_bound()? {
                     let mut environment = options.environment.unwrap_or_default();
@@ -321,7 +321,7 @@ mod tests {
 
         system
             .memory_file_system()
-            .write_files([(root.join("foo.py"), ""), (root.join("bar.py"), "")])
+            .write_files_all([(root.join("foo.py"), ""), (root.join("bar.py"), "")])
             .context("Failed to write files")?;
 
         let project =
@@ -349,7 +349,7 @@ mod tests {
 
         system
             .memory_file_system()
-            .write_files([
+            .write_files_all([
                 (
                     root.join("pyproject.toml"),
                     r#"
@@ -393,7 +393,7 @@ mod tests {
 
         system
             .memory_file_system()
-            .write_files([
+            .write_files_all([
                 (
                     root.join("pyproject.toml"),
                     r#"
@@ -432,7 +432,7 @@ expected `.`, `]`
 
         system
             .memory_file_system()
-            .write_files([
+            .write_files_all([
                 (
                     root.join("pyproject.toml"),
                     r#"
@@ -482,7 +482,7 @@ expected `.`, `]`
 
         system
             .memory_file_system()
-            .write_files([
+            .write_files_all([
                 (
                     root.join("pyproject.toml"),
                     r#"
@@ -532,7 +532,7 @@ expected `.`, `]`
 
         system
             .memory_file_system()
-            .write_files([
+            .write_files_all([
                 (
                     root.join("pyproject.toml"),
                     r#"
@@ -572,7 +572,7 @@ expected `.`, `]`
 
         system
             .memory_file_system()
-            .write_files([
+            .write_files_all([
                 (
                     root.join("pyproject.toml"),
                     r#"
@@ -623,7 +623,7 @@ expected `.`, `]`
 
         system
             .memory_file_system()
-            .write_files([
+            .write_files_all([
                 (
                     root.join("pyproject.toml"),
                     r#"
@@ -673,7 +673,7 @@ expected `.`, `]`
 
         system
             .memory_file_system()
-            .write_file(
+            .write_file_all(
                 root.join("pyproject.toml"),
                 r#"
                 [project]
@@ -703,7 +703,7 @@ expected `.`, `]`
 
         system
             .memory_file_system()
-            .write_file(
+            .write_file_all(
                 root.join("pyproject.toml"),
                 r#"
                 [project]
@@ -735,7 +735,7 @@ expected `.`, `]`
 
         system
             .memory_file_system()
-            .write_file(
+            .write_file_all(
                 root.join("pyproject.toml"),
                 r#"
                 [project]
@@ -765,7 +765,7 @@ expected `.`, `]`
 
         system
             .memory_file_system()
-            .write_file(
+            .write_file_all(
                 root.join("pyproject.toml"),
                 r#"
                 [project]
@@ -795,7 +795,7 @@ expected `.`, `]`
 
         system
             .memory_file_system()
-            .write_file(
+            .write_file_all(
                 root.join("pyproject.toml"),
                 r#"
                 [project]
@@ -828,7 +828,7 @@ expected `.`, `]`
 
         system
             .memory_file_system()
-            .write_file(
+            .write_file_all(
                 root.join("pyproject.toml"),
                 r#"
                 [project]
@@ -861,7 +861,7 @@ expected `.`, `]`
 
         system
             .memory_file_system()
-            .write_file(
+            .write_file_all(
                 root.join("pyproject.toml"),
                 r#"
                 [project]
@@ -886,7 +886,7 @@ expected `.`, `]`
 
         system
             .memory_file_system()
-            .write_file(
+            .write_file_all(
                 root.join("pyproject.toml"),
                 r#"
                 [project]
@@ -911,7 +911,7 @@ expected `.`, `]`
 
         system
             .memory_file_system()
-            .write_file(
+            .write_file_all(
                 root.join("pyproject.toml"),
                 r#"
                 [project]

crates/red_knot_project/src/metadata/options.rs

@@ -1,8 +1,8 @@
 use crate::metadata::value::{RangedValue, RelativePathBuf, ValueSource, ValueSourceGuard};
 use crate::Db;
 use red_knot_python_semantic::lint::{GetLintError, Level, LintSource, RuleSelection};
-use red_knot_python_semantic::{ProgramSettings, PythonPlatform, SearchPathSettings, SitePackages};
-use ruff_db::diagnostic::{Diagnostic, DiagnosticId, Severity, Span};
+use red_knot_python_semantic::{ProgramSettings, PythonPath, PythonPlatform, SearchPathSettings};
+use ruff_db::diagnostic::{DiagnosticId, OldDiagnosticTrait, Severity, Span};
 use ruff_db::files::system_path_to_file;
 use ruff_db::system::{System, SystemPath};
 use ruff_macros::Combine;
@@ -90,7 +90,7 @@ impl Options {
             .map(|env| {
                 (
                     env.extra_paths.clone(),
-                    env.venv_path.clone(),
+                    env.python.clone(),
                     env.typeshed.clone(),
                 )
             })
@@ -104,11 +104,11 @@ impl Options {
                 .collect(),
             src_roots,
             custom_typeshed: typeshed.map(|path| path.absolute(project_root, system)),
-            site_packages: python
-                .map(|venv_path| SitePackages::Derived {
-                    venv_path: venv_path.absolute(project_root, system),
+            python_path: python
+                .map(|python_path| {
+                    PythonPath::SysPrefix(python_path.absolute(project_root, system))
                 })
-                .unwrap_or(SitePackages::Known(vec![])),
+                .unwrap_or(PythonPath::KnownSitePackages(vec![])),
         }
     }
 
@@ -236,10 +236,14 @@ pub struct EnvironmentOptions {
     #[serde(skip_serializing_if = "Option::is_none")]
     pub typeshed: Option<RelativePathBuf>,
 
-    // TODO: Rename to python, see https://github.com/astral-sh/ruff/issues/15530
-    /// The path to the user's `site-packages` directory, where third-party packages from ``PyPI`` are installed.
+    /// Path to the Python installation from which Red Knot resolves type information and third-party dependencies.
+    ///
+    /// Red Knot will search in the path's `site-packages` directories for type information and
+    /// third-party imports.
+    ///
+    /// This option is commonly used to specify the path to a virtual environment.
     #[serde(skip_serializing_if = "Option::is_none")]
-    pub venv_path: Option<RelativePathBuf>,
+    pub python: Option<RelativePathBuf>,
 }
 
 #[derive(Debug, Default, Clone, Eq, PartialEq, Combine, Serialize, Deserialize)]
@@ -372,7 +376,7 @@ impl OptionDiagnostic {
     }
 }
 
-impl Diagnostic for OptionDiagnostic {
+impl OldDiagnosticTrait for OptionDiagnostic {
     fn id(&self) -> DiagnosticId {
         self.id
     }

crates/red_knot_project/src/walk.rs

@@ -0,0 +1,256 @@
+use crate::{Db, IOErrorDiagnostic, IOErrorKind, Project};
+use ruff_db::files::{system_path_to_file, File};
+use ruff_db::system::walk_directory::{ErrorKind, WalkDirectoryBuilder, WalkState};
+use ruff_db::system::{FileType, SystemPath, SystemPathBuf};
+use ruff_python_ast::PySourceType;
+use rustc_hash::{FxBuildHasher, FxHashSet};
+use std::path::PathBuf;
+use thiserror::Error;
+
+/// Filter that decides which files are included in the project.
+///
+/// In the future, this will hold a reference to the `include` and `exclude` pattern.
+///
+/// This struct mainly exists because `dyn Db` isn't `Send` or `Sync`, making it impossible
+/// to access fields from within the walker.
+#[derive(Default, Debug)]
+pub(crate) struct ProjectFilesFilter<'a> {
+    /// The same as [`Project::included_paths_or_root`].
+    included_paths: &'a [SystemPathBuf],
+
+    /// The filter skips checking if the path is in `included_paths` if set to `true`.
+    ///
+    /// Skipping this check is useful when the walker only walks over `included_paths`.
+    skip_included_paths: bool,
+}
+
+impl<'a> ProjectFilesFilter<'a> {
+    pub(crate) fn from_project(db: &'a dyn Db, project: Project) -> Self {
+        Self {
+            included_paths: project.included_paths_or_root(db),
+            skip_included_paths: false,
+        }
+    }
+
+    /// Returns `true` if a file is part of the project and included in the paths to check.
+    ///
+    /// A file is included in the checked files if it is a sub path of the project's root
+    /// (when no CLI path arguments are specified) or if it is a sub path of any path provided on the CLI (`knot check <paths>`) AND:
+    ///
+    /// * It matches a positive `include` pattern and isn't excluded by a later negative `include` pattern.
+    /// * It doesn't match a positive `exclude` pattern or is re-included by a later negative `exclude` pattern.
+    ///
+    /// ## Note
+    ///
+    /// This method may return `true` for files that don't end up being included when walking the
+    /// project tree because it doesn't consider `.gitignore` and other ignore files when deciding
+    /// if a file's included.
+    pub(crate) fn is_included(&self, path: &SystemPath) -> bool {
+        #[derive(Copy, Clone, Debug, PartialEq, Eq, PartialOrd, Ord)]
+        enum CheckPathMatch {
+            /// The path is a partial match of the checked path (it's a sub path)
+            Partial,
+
+            /// The path matches a check path exactly.
+            Full,
+        }
+
+        let m = if self.skip_included_paths {
+            Some(CheckPathMatch::Partial)
+        } else {
+            self.included_paths
+                .iter()
+                .filter_map(|included_path| {
+                    if let Ok(relative_path) = path.strip_prefix(included_path) {
+                        // Exact matches are always included
+                        if relative_path.as_str().is_empty() {
+                            Some(CheckPathMatch::Full)
+                        } else {
+                            Some(CheckPathMatch::Partial)
+                        }
+                    } else {
+                        None
+                    }
+                })
+                .max()
+        };
+
+        match m {
+            None => false,
+            Some(CheckPathMatch::Partial) => {
+                // TODO: For partial matches, only include the file if it is included by the project's include/exclude settings.
+                true
+            }
+            Some(CheckPathMatch::Full) => true,
+        }
+    }
+}
+
+pub(crate) struct ProjectFilesWalker<'a> {
+    walker: WalkDirectoryBuilder,
+
+    filter: ProjectFilesFilter<'a>,
+}
+
+impl<'a> ProjectFilesWalker<'a> {
+    pub(crate) fn new(db: &'a dyn Db) -> Self {
+        let project = db.project();
+
+        let mut filter = ProjectFilesFilter::from_project(db, project);
+        // It's unnecessary to filter on included paths because it only iterates over those to start with.
+        filter.skip_included_paths = true;
+
+        Self::from_paths(db, project.included_paths_or_root(db), filter)
+            .expect("included_paths_or_root to never return an empty iterator")
+    }
+
+    /// Creates a walker for indexing the project files incrementally.
+    ///
+    /// The main difference to a full project walk is that `paths` may contain paths
+    /// that aren't part of the included files.
+    pub(crate) fn incremental<P>(db: &'a dyn Db, paths: impl IntoIterator<Item = P>) -> Option<Self>
+    where
+        P: AsRef<SystemPath>,
+    {
+        let project = db.project();
+
+        let filter = ProjectFilesFilter::from_project(db, project);
+
+        Self::from_paths(db, paths, filter)
+    }
+
+    fn from_paths<P>(
+        db: &'a dyn Db,
+        paths: impl IntoIterator<Item = P>,
+        filter: ProjectFilesFilter<'a>,
+    ) -> Option<Self>
+    where
+        P: AsRef<SystemPath>,
+    {
+        let mut paths = paths.into_iter();
+
+        let mut walker = db.system().walk_directory(paths.next()?.as_ref());
+
+        for path in paths {
+            walker = walker.add(path);
+        }
+
+        Some(Self { walker, filter })
+    }
+
+    /// Walks the project paths and collects the paths of all files that
+    /// are included in the project.
+    pub(crate) fn walk_paths(self) -> (Vec<SystemPathBuf>, Vec<IOErrorDiagnostic>) {
+        let paths = std::sync::Mutex::new(Vec::new());
+        let diagnostics = std::sync::Mutex::new(Vec::new());
+
+        self.walker.run(|| {
+            Box::new(|entry| {
+                match entry {
+                    Ok(entry) => {
+                        if !self.filter.is_included(entry.path()) {
+                            tracing::debug!("Ignoring not-included path: {}", entry.path());
+                            return WalkState::Skip;
+                        }
+
+                        // Skip over any non python files to avoid creating too many entries in `Files`.
+                        match entry.file_type() {
+                            FileType::File => {
+                                if entry
+                                    .path()
+                                    .extension()
+                                    .and_then(PySourceType::try_from_extension)
+                                    .is_some()
+                                {
+                                    let mut paths = paths.lock().unwrap();
+                                    paths.push(entry.into_path());
+                                }
+                            }
+                            FileType::Directory | FileType::Symlink => {}
+                        }
+                    }
+                    Err(error) => match error.kind() {
+                        ErrorKind::Loop { .. } => {
+                            unreachable!("Loops shouldn't be possible without following symlinks.")
+                        }
+                        ErrorKind::Io { path, err } => {
+                            let mut diagnostics = diagnostics.lock().unwrap();
+                            let error = if let Some(path) = path {
+                                WalkError::IOPathError {
+                                    path: path.clone(),
+                                    error: err.to_string(),
+                                }
+                            } else {
+                                WalkError::IOError {
+                                    error: err.to_string(),
+                                }
+                            };
+
+                            diagnostics.push(IOErrorDiagnostic {
+                                file: None,
+                                error: IOErrorKind::Walk(error),
+                            });
+                        }
+                        ErrorKind::NonUtf8Path { path } => {
+                            diagnostics.lock().unwrap().push(IOErrorDiagnostic {
+                                file: None,
+                                error: IOErrorKind::Walk(WalkError::NonUtf8Path {
+                                    path: path.clone(),
+                                }),
+                            });
+                        }
+                    },
+                }
+
+                WalkState::Continue
+            })
+        });
+
+        (
+            paths.into_inner().unwrap(),
+            diagnostics.into_inner().unwrap(),
+        )
+    }
+
+    pub(crate) fn collect_vec(self, db: &dyn Db) -> (Vec<File>, Vec<IOErrorDiagnostic>) {
+        let (paths, diagnostics) = self.walk_paths();
+
+        (
+            paths
+                .into_iter()
+                .filter_map(move |path| {
+                    // If this returns `None`, then the file was deleted between the `walk_directory` call and now.
+                    // We can ignore this.
+                    system_path_to_file(db.upcast(), &path).ok()
+                })
+                .collect(),
+            diagnostics,
+        )
+    }
+
+    pub(crate) fn collect_set(self, db: &dyn Db) -> (FxHashSet<File>, Vec<IOErrorDiagnostic>) {
+        let (paths, diagnostics) = self.walk_paths();
+
+        let mut files = FxHashSet::with_capacity_and_hasher(paths.len(), FxBuildHasher);
+
+        for path in paths {
+            if let Ok(file) = system_path_to_file(db.upcast(), &path) {
+                files.insert(file);
+            }
+        }
+
+        (files, diagnostics)
+    }
+}
+
+#[derive(Error, Debug, Clone)]
+pub(crate) enum WalkError {
+    #[error("`{path}`: {error}")]
+    IOPathError { path: SystemPathBuf, error: String },
+
+    #[error("Failed to walk project directory: {error}")]
+    IOError { error: String },
+
+    #[error("`{path}` is not a valid UTF-8 path")]
+    NonUtf8Path { path: PathBuf },
+}

crates/red_knot_project/src/watch/project_watcher.rs

@@ -6,7 +6,7 @@ use tracing::info;
 use red_knot_python_semantic::system_module_search_paths;
 use ruff_cache::{CacheKey, CacheKeyHasher};
 use ruff_db::system::{SystemPath, SystemPathBuf};
-use ruff_db::{Db as _, Upcast};
+use ruff_db::Upcast;
 
 use crate::db::{Db, ProjectDatabase};
 use crate::watch::Watcher;
@@ -42,9 +42,9 @@ impl ProjectWatcher {
 
     pub fn update(&mut self, db: &ProjectDatabase) {
         let search_paths: Vec<_> = system_module_search_paths(db.upcast()).collect();
-        let project_path = db.project().root(db).to_path_buf();
+        let project_path = db.project().root(db);
 
-        let new_cache_key = Self::compute_cache_key(&project_path, &search_paths);
+        let new_cache_key = Self::compute_cache_key(project_path, &search_paths);
 
         if self.cache_key == Some(new_cache_key) {
             return;
@@ -68,41 +68,47 @@ impl ProjectWatcher {
 
         self.has_errored_paths = false;
 
-        let project_path = db
-            .system()
-            .canonicalize_path(&project_path)
-            .unwrap_or(project_path);
-
         let config_paths = db
             .project()
             .metadata(db)
             .extra_configuration_paths()
             .iter()
-            .cloned();
+            .map(SystemPathBuf::as_path);
+
+        // Watch both the project root and any paths provided by the user on the CLI (removing any redundant nested paths).
+        // This is necessary to observe changes to files that are outside the project root.
+        // We always need to watch the project root to observe changes to its configuration.
+        let included_paths = ruff_db::system::deduplicate_nested_paths(
+            std::iter::once(project_path).chain(
+                db.project()
+                    .included_paths_list(db)
+                    .iter()
+                    .map(SystemPathBuf::as_path),
+            ),
+        );
 
         // Find the non-overlapping module search paths and filter out paths that are already covered by the project.
         // Module search paths are already canonicalized.
         let unique_module_paths = ruff_db::system::deduplicate_nested_paths(
             search_paths
                 .into_iter()
-                .filter(|path| !path.starts_with(&project_path)),
-        )
-        .map(SystemPath::to_path_buf);
+                .filter(|path| !path.starts_with(project_path)),
+        );
 
         // Now add the new paths, first starting with the project path and then
         // adding the library search paths, and finally the paths for configurations.
-        for path in std::iter::once(project_path)
+        for path in included_paths
             .chain(unique_module_paths)
             .chain(config_paths)
         {
             // Log a warning. It's not worth aborting if registering a single folder fails because
             // Ruff otherwise stills works as expected.
-            if let Err(error) = self.watcher.watch(&path) {
+            if let Err(error) = self.watcher.watch(path) {
                 // TODO: Log a user-facing warning.
                 tracing::warn!("Failed to setup watcher for path `{path}`: {error}. You have to restart Ruff after making changes to files under this path or you might see stale results.");
                 self.has_errored_paths = true;
             } else {
-                self.watched_paths.push(path);
+                self.watched_paths.push(path.to_path_buf());
             }
         }
 

crates/red_knot_project/tests/check.rs

@@ -117,7 +117,7 @@ fn run_corpus_tests(pattern: &str) -> anyhow::Result<()> {
         let code = std::fs::read_to_string(source)?;
 
         let mut check_with_file_name = |path: &SystemPath| {
-            memory_fs.write_file(path, &code).unwrap();
+            memory_fs.write_file_all(path, &code).unwrap();
             File::sync_path(&mut db, path);
 
             // this test is only asserting that we can pull every expression type without a panic
@@ -216,6 +216,17 @@ impl SourceOrderVisitor<'_> for PullTypesVisitor<'_> {
                 self.visit_body(&for_stmt.orelse);
                 return;
             }
+            Stmt::With(with_stmt) => {
+                for item in &with_stmt.items {
+                    if let Some(target) = &item.optional_vars {
+                        self.visit_target(target);
+                    }
+                    self.visit_expr(&item.context_expr);
+                }
+
+                self.visit_body(&with_stmt.body);
+                return;
+            }
             Stmt::AnnAssign(_)
             | Stmt::Return(_)
             | Stmt::Delete(_)
@@ -223,7 +234,6 @@ impl SourceOrderVisitor<'_> for PullTypesVisitor<'_> {
             | Stmt::TypeAlias(_)
             | Stmt::While(_)
             | Stmt::If(_)
-            | Stmt::With(_)
             | Stmt::Match(_)
             | Stmt::Raise(_)
             | Stmt::Try(_)
@@ -269,18 +279,4 @@ impl SourceOrderVisitor<'_> for PullTypesVisitor<'_> {
 
 /// Whether or not the .py/.pyi version of this file is expected to fail
 #[rustfmt::skip]
-const KNOWN_FAILURES: &[(&str, bool, bool)] = &[
-    // related to circular references in nested functions
-    ("crates/ruff_linter/resources/test/fixtures/flake8_return/RET503.py", false, true),
-    // related to circular references in class definitions
-    ("crates/ruff_linter/resources/test/fixtures/pyflakes/F821_26.py", true, true),
-    ("crates/ruff_linter/resources/test/fixtures/pyflakes/F821_27.py", true, true),
-    ("crates/ruff_linter/resources/test/fixtures/pyflakes/F811_19.py", true, false),
-    ("crates/ruff_linter/resources/test/fixtures/pyupgrade/UP039.py", true, false),
-    // related to circular references in type aliases (salsa cycle panic):
-    ("crates/ruff_python_parser/resources/inline/err/type_alias_invalid_value_expr.py", true, true),
-    ("crates/ruff_linter/resources/test/fixtures/flake8_type_checking/TC008.py", true, true),
-    // related to circular references in f-string annotations (invalid syntax)
-    ("crates/ruff_linter/resources/test/fixtures/pyflakes/F821_15.py", true, true),
-    ("crates/ruff_linter/resources/test/fixtures/pyflakes/F821_14.py", false, true),
-];
+const KNOWN_FAILURES: &[(&str, bool, bool)] = &[];

crates/red_knot_python_semantic/Cargo.toml

@@ -42,6 +42,8 @@ smallvec = { workspace = true }
 static_assertions = { workspace = true }
 test-case = { workspace = true }
 memchr = { workspace = true }
+strum = { workspace = true}
+strum_macros = { workspace = true}
 
 [dev-dependencies]
 ruff_db = { workspace = true, features = ["testing", "os"] }

crates/red_knot_python_semantic/resources/mdtest/annotations/callable.md

@@ -0,0 +1,211 @@
+# Callable
+
+References:
+
+- <https://typing.readthedocs.io/en/latest/spec/callables.html#callable>
+
+TODO: Use `collections.abc` as importing from `typing` is deprecated but this requires support for
+`*` imports. See: <https://docs.python.org/3/library/typing.html#deprecated-aliases>.
+
+## Invalid forms
+
+The `Callable` special form requires _exactly_ two arguments where the first argument is either a
+parameter type list, parameter specification, `typing.Concatenate`, or `...` and the second argument
+is the return type. Here, we explore various invalid forms.
+
+### Empty
+
+A bare `Callable` without any type arguments:
+
+```py
+from typing import Callable
+
+def _(c: Callable):
+    reveal_type(c)  # revealed: (...) -> Unknown
+```
+
+### Invalid parameter type argument
+
+When it's not a list:
+
+```py
+from typing import Callable
+
+# error: [invalid-type-form] "The first argument to `Callable` must be either a list of types, ParamSpec, Concatenate, or `...`"
+def _(c: Callable[int, str]):
+    reveal_type(c)  # revealed: (...) -> Unknown
+```
+
+Or, when it's a literal type:
+
+```py
+# error: [invalid-type-form] "The first argument to `Callable` must be either a list of types, ParamSpec, Concatenate, or `...`"
+def _(c: Callable[42, str]):
+    reveal_type(c)  # revealed: (...) -> Unknown
+```
+
+Or, when one of the parameter type is invalid in the list:
+
+```py
+def _(c: Callable[[int, 42, str, False], None]):
+    # revealed: (int, @Todo(number literal in type expression), str, @Todo(boolean literal in type expression), /) -> None
+    reveal_type(c)
+```
+
+### Missing return type
+
+Using a parameter list:
+
+```py
+from typing import Callable
+
+# error: [invalid-type-form] "Special form `typing.Callable` expected exactly two arguments (parameter types and return type)"
+def _(c: Callable[[int, str]]):
+    reveal_type(c)  # revealed: (int, str, /) -> Unknown
+```
+
+Or, an ellipsis:
+
+```py
+# error: [invalid-type-form] "Special form `typing.Callable` expected exactly two arguments (parameter types and return type)"
+def _(c: Callable[...]):
+    reveal_type(c)  # revealed: (...) -> Unknown
+```
+
+### More than two arguments
+
+We can't reliably infer the callable type if there are more then 2 arguments because we don't know
+which argument corresponds to either the parameters or the return type.
+
+```py
+from typing import Callable
+
+# error: [invalid-type-form] "Special form `typing.Callable` expected exactly two arguments (parameter types and return type)"
+def _(c: Callable[[int], str, str]):
+    reveal_type(c)  # revealed: (...) -> Unknown
+```
+
+## Simple
+
+A simple `Callable` with multiple parameters and a return type:
+
+```py
+from typing import Callable
+
+def _(c: Callable[[int, str], int]):
+    reveal_type(c)  # revealed: (int, str, /) -> int
+```
+
+## Nested
+
+A nested `Callable` as one of the parameter types:
+
+```py
+from typing import Callable
+
+def _(c: Callable[[Callable[[int], str]], int]):
+    reveal_type(c)  # revealed: ((int, /) -> str, /) -> int
+```
+
+And, as the return type:
+
+```py
+def _(c: Callable[[int, str], Callable[[int], int]]):
+    reveal_type(c)  # revealed: (int, str, /) -> (int, /) -> int
+```
+
+## Gradual form
+
+The `Callable` special form supports the use of `...` in place of the list of parameter types. This
+is a [gradual form] indicating that the type is consistent with any input signature:
+
+```py
+from typing import Callable
+
+def gradual_form(c: Callable[..., str]):
+    reveal_type(c)  # revealed: (...) -> str
+```
+
+## Using `typing.Concatenate`
+
+Using `Concatenate` as the first argument to `Callable`:
+
+```py
+from typing_extensions import Callable, Concatenate
+
+def _(c: Callable[Concatenate[int, str, ...], int]):
+    reveal_type(c)  # revealed: (*args: @Todo(todo signature *args), **kwargs: @Todo(todo signature **kwargs)) -> int
+```
+
+And, as one of the parameter types:
+
+```py
+def _(c: Callable[[Concatenate[int, str, ...], int], int]):
+    reveal_type(c)  # revealed: (*args: @Todo(todo signature *args), **kwargs: @Todo(todo signature **kwargs)) -> int
+```
+
+## Using `typing.ParamSpec`
+
+Using a `ParamSpec` in a `Callable` annotation:
+
+```py
+from typing_extensions import Callable
+
+# TODO: Not an error; remove once `ParamSpec` is supported
+# error: [invalid-type-form]
+def _[**P1](c: Callable[P1, int]):
+    reveal_type(c)  # revealed: (...) -> Unknown
+```
+
+And, using the legacy syntax:
+
+```py
+from typing_extensions import ParamSpec
+
+P2 = ParamSpec("P2")
+
+# TODO: Not an error; remove once `ParamSpec` is supported
+# error: [invalid-type-form]
+def _(c: Callable[P2, int]):
+    reveal_type(c)  # revealed: (...) -> Unknown
+```
+
+## Using `typing.Unpack`
+
+Using the unpack operator (`*`):
+
+```py
+from typing_extensions import Callable, TypeVarTuple
+
+Ts = TypeVarTuple("Ts")
+
+def _(c: Callable[[int, *Ts], int]):
+    reveal_type(c)  # revealed: (*args: @Todo(todo signature *args), **kwargs: @Todo(todo signature **kwargs)) -> int
+```
+
+And, using the legacy syntax using `Unpack`:
+
+```py
+from typing_extensions import Unpack
+
+def _(c: Callable[[int, Unpack[Ts]], int]):
+    reveal_type(c)  # revealed: (*args: @Todo(todo signature *args), **kwargs: @Todo(todo signature **kwargs)) -> int
+```
+
+## Member lookup
+
+```py
+from typing import Callable
+
+def _(c: Callable[[int], int]):
+    reveal_type(c.__init__)  # revealed: Literal[__init__]
+    reveal_type(c.__class__)  # revealed: type
+
+    # TODO: The member lookup for `Callable` uses `object` which does not have a `__call__`
+    # attribute. We could special case `__call__` in this context. Refer to
+    # https://github.com/astral-sh/ruff/pull/16493#discussion_r1985098508 for more details.
+    # error: [unresolved-attribute] "Type `(int, /) -> int` has no attribute `__call__`"
+    reveal_type(c.__call__)  # revealed: Unknown
+```
+
+[gradual form]: https://typing.readthedocs.io/en/latest/spec/glossary.html#term-gradual-form

crates/red_knot_python_semantic/resources/mdtest/annotations/deferred.md

@@ -38,7 +38,8 @@ If `__future__.annotations` is imported, annotations *are* deferred.
 ```py
 from __future__ import annotations
 
-def get_foo() -> Foo: ...
+def get_foo() -> Foo:
+    return Foo()
 
 class Foo: ...
 

crates/red_knot_python_semantic/resources/mdtest/annotations/invalid.md

@@ -0,0 +1,45 @@
+# Tests for invalid types in type expressions
+
+## Invalid types are rejected
+
+Many types are illegal in the context of a type expression:
+
+```py
+import typing
+from knot_extensions import AlwaysTruthy, AlwaysFalsy
+from typing_extensions import Literal, Never
+
+def _(
+    a: type[int],
+    b: AlwaysTruthy,
+    c: AlwaysFalsy,
+    d: Literal[True],
+    e: Literal["bar"],
+    f: Literal[b"foo"],
+    g: tuple[int, str],
+    h: Never,
+):
+    def foo(): ...
+    def invalid(
+        i: a,  # error: [invalid-type-form] "Variable of type `type[int]` is not allowed in a type expression"
+        j: b,  # error: [invalid-type-form]
+        k: c,  # error: [invalid-type-form]
+        l: d,  # error: [invalid-type-form]
+        m: e,  # error: [invalid-type-form]
+        n: f,  # error: [invalid-type-form]
+        o: g,  # error: [invalid-type-form]
+        p: h,  # error: [invalid-type-form]
+        q: typing,  # error: [invalid-type-form]
+        r: foo,  # error: [invalid-type-form]
+    ):
+        reveal_type(i)  # revealed: Unknown
+        reveal_type(j)  # revealed: Unknown
+        reveal_type(k)  # revealed: Unknown
+        reveal_type(l)  # revealed: Unknown
+        reveal_type(m)  # revealed: Unknown
+        reveal_type(n)  # revealed: Unknown
+        reveal_type(o)  # revealed: Unknown
+        reveal_type(p)  # revealed: Unknown
+        reveal_type(q)  # revealed: Unknown
+        reveal_type(r)  # revealed: Unknown
+```

crates/red_knot_python_semantic/resources/mdtest/annotations/literal_string.md

@@ -73,12 +73,12 @@ qux = (foo, bar)
 reveal_type(qux)  # revealed: tuple[Literal["foo"], Literal["bar"]]
 
 # TODO: Infer "LiteralString"
-reveal_type(foo.join(qux))  # revealed: @Todo(decorated method)
+reveal_type(foo.join(qux))  # revealed: @Todo(return type of decorated function)
 
 template: LiteralString = "{}, {}"
 reveal_type(template)  # revealed: Literal["{}, {}"]
 # TODO: Infer `LiteralString`
-reveal_type(template.format(foo, bar))  # revealed: @Todo(decorated method)
+reveal_type(template.format(foo, bar))  # revealed: @Todo(return type of decorated function)
 ```
 
 ### Assignability

crates/red_knot_python_semantic/resources/mdtest/annotations/stdlib_typing_aliases.md

@@ -70,8 +70,7 @@ import typing
 
 class ListSubclass(typing.List): ...
 
-# TODO: should have `Generic`, should not have `Unknown`
-# revealed: tuple[Literal[ListSubclass], Literal[list], Unknown, Literal[object]]
+# revealed: tuple[Literal[ListSubclass], Literal[list], Literal[MutableSequence], Literal[Sequence], Literal[Reversible], Literal[Collection], Literal[Iterable], Literal[Container], @Todo(protocol), Literal[object]]
 reveal_type(ListSubclass.__mro__)
 
 class DictSubclass(typing.Dict): ...

crates/red_knot_python_semantic/resources/mdtest/annotations/string.md

@@ -116,8 +116,8 @@ MyType = int
 class Aliases:
     MyType = str
 
-    forward: "MyType"
-    not_forward: MyType
+    forward: "MyType" = "value"
+    not_forward: MyType = "value"
 
 reveal_type(Aliases.forward)  # revealed: str
 reveal_type(Aliases.not_forward)  # revealed: str

crates/red_knot_python_semantic/resources/mdtest/annotations/unsupported_special_forms.md

@@ -18,7 +18,7 @@ def f(*args: Unpack[Ts]) -> tuple[Unpack[Ts]]:
     # TODO: should understand the annotation
     reveal_type(args)  # revealed: tuple
 
-    reveal_type(Alias)  # revealed: @Todo(Unsupported or invalid type in a type expression)
+    reveal_type(Alias)  # revealed: @Todo(Invalid or unsupported `KnownInstanceType` in `Type::to_type_expression`)
 
 def g() -> TypeGuard[int]: ...
 def h() -> TypeIs[int]: ...
@@ -29,11 +29,13 @@ def i(callback: Callable[Concatenate[int, P], R_co], *args: P.args, **kwargs: P.
     # TODO: should understand the annotation
     reveal_type(kwargs)  # revealed: dict
 
+    # TODO: not an error; remove once `call` is implemented for `Callable`
+    # error: [call-non-callable]
     return callback(42, *args, **kwargs)
 
 class Foo:
     def method(self, x: Self):
-        reveal_type(x)  # revealed: @Todo(Unsupported or invalid type in a type expression)
+        reveal_type(x)  # revealed: @Todo(Invalid or unsupported `KnownInstanceType` in `Type::to_type_expression`)
 ```
 
 ## Inheritance

crates/red_knot_python_semantic/resources/mdtest/assignment/augmented.md

@@ -75,8 +75,7 @@ def _(flag: bool):
 
     f = Foo()
 
-    # TODO: We should emit an `unsupported-operator` error here, possibly with the information
-    # that `Foo.__iadd__` may be unbound as additional context.
+    # error: [unsupported-operator] "Operator `+=` is unsupported between objects of type `Foo` and `Literal["Hello, world!"]`"
     f += "Hello, world!"
 
     reveal_type(f)  # revealed: int | Unknown

crates/red_knot_python_semantic/resources/mdtest/attributes.md

@@ -54,13 +54,12 @@ c_instance.declared_and_bound = False
 # error: [invalid-assignment] "Object of type `Literal["incompatible"]` is not assignable to attribute `declared_and_bound` of type `bool`"
 c_instance.declared_and_bound = "incompatible"
 
-# TODO: we already show an error here but the message might be improved?
 # mypy shows no error here, but pyright raises "reportAttributeAccessIssue"
-# error: [unresolved-attribute] "Type `Literal[C]` has no attribute `inferred_from_value`"
+# error: [unresolved-attribute] "Attribute `inferred_from_value` can only be accessed on instances, not on the class object `Literal[C]` itself."
 reveal_type(C.inferred_from_value)  # revealed: Unknown
 
-# TODO: this should be an error (pure instance variables cannot be accessed on the class)
 # mypy shows no error here, but pyright raises "reportAttributeAccessIssue"
+# error: [invalid-attribute-access] "Cannot assign to instance attribute `inferred_from_value` from the class object `Literal[C]`"
 C.inferred_from_value = "overwritten on class"
 
 # This assignment is fine:
@@ -90,13 +89,13 @@ c_instance = C()
 
 reveal_type(c_instance.declared_and_bound)  # revealed: str | None
 
-# TODO: we currently plan to emit a diagnostic here. Note that both mypy
-# and pyright show no error in this case! So we may reconsider this in
-# the future, if it turns out to produce too many false positives.
-reveal_type(C.declared_and_bound)  # revealed: str | None
+# Note that both mypy and pyright show no error in this case! So we may reconsider this in
+# the future, if it turns out to produce too many false positives. We currently emit:
+# error: [unresolved-attribute] "Attribute `declared_and_bound` can only be accessed on instances, not on the class object `Literal[C]` itself."
+reveal_type(C.declared_and_bound)  # revealed: Unknown
 
-# TODO: same as above. We plan to emit a diagnostic here, even if both mypy
-# and pyright allow this.
+# Same as above. Mypy and pyright do not show an error here.
+# error: [invalid-attribute-access] "Cannot assign to instance attribute `declared_and_bound` from the class object `Literal[C]`"
 C.declared_and_bound = "overwritten on class"
 
 # error: [invalid-assignment] "Object of type `Literal[1]` is not assignable to attribute `declared_and_bound` of type `str | None`"
@@ -116,11 +115,11 @@ c_instance = C()
 
 reveal_type(c_instance.only_declared)  # revealed: str
 
-# TODO: mypy and pyright do not show an error here, but we plan to emit a diagnostic.
-# The type could be changed to 'Unknown' if we decide to emit an error?
-reveal_type(C.only_declared)  # revealed: str
+# Mypy and pyright do not show an error here. We treat this as a pure instance variable.
+# error: [unresolved-attribute] "Attribute `only_declared` can only be accessed on instances, not on the class object `Literal[C]` itself."
+reveal_type(C.only_declared)  # revealed: Unknown
 
-# TODO: mypy and pyright do not show an error here, but we plan to emit one.
+# error: [invalid-attribute-access] "Cannot assign to instance attribute `only_declared` from the class object `Literal[C]`"
 C.only_declared = "overwritten on class"
 ```
 
@@ -156,7 +155,9 @@ reveal_type(c_instance.declared_in_body_and_init)  # revealed: str | None
 
 reveal_type(c_instance.declared_in_body_defined_in_init)  # revealed: str | None
 
-reveal_type(c_instance.bound_in_body_declared_in_init)  # revealed: str | None
+# TODO: This should be `str | None`. Fixing this requires an overhaul of the `Symbol` API,
+# which is planned in https://github.com/astral-sh/ruff/issues/14297
+reveal_type(c_instance.bound_in_body_declared_in_init)  # revealed: Unknown | str | None
 
 reveal_type(c_instance.bound_in_body_and_init)  # revealed: Unknown | None | Literal["a"]
 ```
@@ -191,11 +192,10 @@ reveal_type(c_instance.declared_only)  # revealed: bytes
 
 reveal_type(c_instance.declared_and_bound)  # revealed: bool
 
-# TODO: We already show an error here, but the message might be improved?
-# error: [unresolved-attribute]
+# error: [unresolved-attribute] "Attribute `inferred_from_value` can only be accessed on instances, not on the class object `Literal[C]` itself."
 reveal_type(C.inferred_from_value)  # revealed: Unknown
 
-# TODO: this should be an error
+# error: [invalid-attribute-access] "Cannot assign to instance attribute `inferred_from_value` from the class object `Literal[C]`"
 C.inferred_from_value = "overwritten on class"
 ```
 
@@ -348,8 +348,11 @@ reveal_type(C().y)  # revealed: Unknown | str
 
 ```py
 class ContextManager:
-    def __enter__(self) -> int | None: ...
-    def __exit__(self, exc_type, exc_value, traceback) -> None: ...
+    def __enter__(self) -> int | None:
+        return 1
+
+    def __exit__(self, exc_type, exc_value, traceback) -> None:
+        pass
 
 class C:
     def __init__(self) -> None:
@@ -358,9 +361,28 @@ class C:
 
 c_instance = C()
 
-# TODO: Should be `Unknown | int | None`
-# error: [unresolved-attribute]
-reveal_type(c_instance.x)  # revealed: Unknown
+reveal_type(c_instance.x)  # revealed: Unknown | int | None
+```
+
+#### Attributes defined in `with` statements, but with unpacking
+
+```py
+class ContextManager:
+    def __enter__(self) -> tuple[int | None, int]:
+        return 1, 2
+
+    def __exit__(self, exc_type, exc_value, traceback) -> None:
+        pass
+
+class C:
+    def __init__(self) -> None:
+        with ContextManager() as (self.x, self.y):
+            pass
+
+c_instance = C()
+
+reveal_type(c_instance.x)  # revealed: Unknown | int | None
+reveal_type(c_instance.y)  # revealed: Unknown | int
 ```
 
 #### Attributes defined in comprehensions
@@ -598,6 +620,9 @@ C.class_method()
 # error: [unresolved-attribute]
 reveal_type(C.pure_class_variable)  # revealed: Unknown
 
+# TODO: should be no error when descriptor protocol is supported
+# and the assignment is properly attributed to the class method.
+# error: [invalid-attribute-access] "Cannot assign to instance attribute `pure_class_variable` from the class object `Literal[C]`"
 C.pure_class_variable = "overwritten on class"
 
 # TODO: should be  `Unknown | Literal["value set in class method"]` or
@@ -703,7 +728,90 @@ reveal_type(Derived().declared_in_body)  # revealed: int | None
 reveal_type(Derived().defined_in_init)  # revealed: str | None
 ```
 
-## Union of attributes
+## Accessing attributes on class objects
+
+When accessing attributes on class objects, they are always looked up on the type of the class
+object first, i.e. on the metaclass:
+
+```py
+from typing import Literal
+
+class Meta1:
+    attr: Literal["metaclass value"] = "metaclass value"
+
+class C1(metaclass=Meta1): ...
+
+reveal_type(C1.attr)  # revealed: Literal["metaclass value"]
+```
+
+However, the metaclass attribute only takes precedence over a class-level attribute if it is a data
+descriptor. If it is a non-data descriptor or a normal attribute, the class-level attribute is used
+instead (see the [descriptor protocol tests] for data/non-data descriptor attributes):
+
+```py
+class Meta2:
+    attr: str = "metaclass value"
+
+class C2(metaclass=Meta2):
+    attr: Literal["class value"] = "class value"
+
+reveal_type(C2.attr)  # revealed: Literal["class value"]
+```
+
+If the class-level attribute is only partially defined, we union the metaclass attribute with the
+class-level attribute:
+
+```py
+def _(flag: bool):
+    class Meta3:
+        attr1 = "metaclass value"
+        attr2: Literal["metaclass value"] = "metaclass value"
+
+    class C3(metaclass=Meta3):
+        if flag:
+            attr1 = "class value"
+            # TODO: Neither mypy nor pyright show an error here, but we could consider emitting a conflicting-declaration diagnostic here.
+            attr2: Literal["class value"] = "class value"
+
+    reveal_type(C3.attr1)  # revealed: Unknown | Literal["metaclass value", "class value"]
+    reveal_type(C3.attr2)  # revealed: Literal["metaclass value", "class value"]
+```
+
+If the *metaclass* attribute is only partially defined, we emit a `possibly-unbound-attribute`
+diagnostic:
+
+```py
+def _(flag: bool):
+    class Meta4:
+        if flag:
+            attr1: str = "metaclass value"
+
+    class C4(metaclass=Meta4): ...
+    # error: [possibly-unbound-attribute]
+    reveal_type(C4.attr1)  # revealed: str
+```
+
+Finally, if both the metaclass attribute and the class-level attribute are only partially defined,
+we union them and emit a `possibly-unbound-attribute` diagnostic:
+
+```py
+def _(flag1: bool, flag2: bool):
+    class Meta5:
+        if flag1:
+            attr1 = "metaclass value"
+
+    class C5(metaclass=Meta5):
+        if flag2:
+            attr1 = "class value"
+
+    # error: [possibly-unbound-attribute]
+    reveal_type(C5.attr1)  # revealed: Unknown | Literal["metaclass value", "class value"]
+```
+
+## Unions of attributes
+
+If the (meta)class is a union type or if the attribute on the (meta) class has a union type, we
+infer those union types accordingly:
 
 ```py
 def _(flag: bool):
@@ -715,14 +823,186 @@ def _(flag: bool):
         class C1:
             x = 2
 
+    reveal_type(C1.x)  # revealed: Unknown | Literal[1, 2]
+
     class C2:
         if flag:
             x = 3
         else:
             x = 4
 
-    reveal_type(C1.x)  # revealed: Unknown | Literal[1, 2]
     reveal_type(C2.x)  # revealed: Unknown | Literal[3, 4]
+
+    if flag:
+        class Meta3(type):
+            x = 5
+
+    else:
+        class Meta3(type):
+            x = 6
+
+    class C3(metaclass=Meta3): ...
+    reveal_type(C3.x)  # revealed: Unknown | Literal[5, 6]
+
+    class Meta4(type):
+        if flag:
+            x = 7
+        else:
+            x = 8
+
+    class C4(metaclass=Meta4): ...
+    reveal_type(C4.x)  # revealed: Unknown | Literal[7, 8]
+```
+
+## Unions with possibly unbound paths
+
+### Definite boundness within a class
+
+In this example, the `x` attribute is not defined in the `C2` element of the union:
+
+```py
+def _(flag1: bool, flag2: bool):
+    class C1:
+        x = 1
+
+    class C2: ...
+
+    class C3:
+        x = 3
+
+    C = C1 if flag1 else C2 if flag2 else C3
+
+    # error: [possibly-unbound-attribute] "Attribute `x` on type `Literal[C1, C2, C3]` is possibly unbound"
+    reveal_type(C.x)  # revealed: Unknown | Literal[1, 3]
+
+    # error: [possibly-unbound-attribute] "Attribute `x` on type `C1 | C2 | C3` is possibly unbound"
+    reveal_type(C().x)  # revealed: Unknown | Literal[1, 3]
+```
+
+### Possibly-unbound within a class
+
+We raise the same diagnostic if the attribute is possibly-unbound in at least one element of the
+union:
+
+```py
+def _(flag: bool, flag1: bool, flag2: bool):
+    class C1:
+        x = 1
+
+    class C2:
+        if flag:
+            x = 2
+
+    class C3:
+        x = 3
+
+    C = C1 if flag1 else C2 if flag2 else C3
+
+    # error: [possibly-unbound-attribute] "Attribute `x` on type `Literal[C1, C2, C3]` is possibly unbound"
+    reveal_type(C.x)  # revealed: Unknown | Literal[1, 2, 3]
+
+    # Note: we might want to consider ignoring possibly-unbound diagnostics for instance attributes eventually,
+    # see the "Possibly unbound/undeclared instance attribute" section below.
+    # error: [possibly-unbound-attribute] "Attribute `x` on type `C1 | C2 | C3` is possibly unbound"
+    reveal_type(C().x)  # revealed: Unknown | Literal[1, 2, 3]
+```
+
+### Possibly-unbound within gradual types
+
+```py
+from typing import Any
+
+def _(flag: bool):
+    class Base:
+        x: Any
+
+    class Derived(Base):
+        if flag:
+            # Redeclaring `x` with a more static type is okay in terms of LSP.
+            x: int
+
+    reveal_type(Derived().x)  # revealed: int | Any
+```
+
+### Attribute possibly unbound on a subclass but not on a superclass
+
+```py
+def _(flag: bool):
+    class Foo:
+        x = 1
+
+    class Bar(Foo):
+        if flag:
+            x = 2
+
+    reveal_type(Bar.x)  # revealed: Unknown | Literal[2, 1]
+
+    reveal_type(Bar().x)  # revealed: Unknown | Literal[2, 1]
+```
+
+### Attribute possibly unbound on a subclass and on a superclass
+
+```py
+def _(flag: bool):
+    class Foo:
+        if flag:
+            x = 1
+
+    class Bar(Foo):
+        if flag:
+            x = 2
+
+    # error: [possibly-unbound-attribute]
+    reveal_type(Bar.x)  # revealed: Unknown | Literal[2, 1]
+
+    # error: [possibly-unbound-attribute]
+    reveal_type(Bar().x)  # revealed: Unknown | Literal[2, 1]
+```
+
+### Possibly unbound/undeclared instance attribute
+
+#### Possibly unbound and undeclared
+
+```py
+def _(flag: bool):
+    class Foo:
+        if flag:
+            x: int
+
+        def __init(self):
+            if flag:
+                self.x = 1
+
+    # error: [possibly-unbound-attribute]
+    reveal_type(Foo().x)  # revealed: int | Unknown
+```
+
+#### Possibly unbound
+
+```py
+def _(flag: bool):
+    class Foo:
+        def __init(self):
+            if flag:
+                self.x = 1
+
+    # Emitting a diagnostic in a case like this is not something we support, and it's unclear
+    # if we ever will (or want to)
+    reveal_type(Foo().x)  # revealed: Unknown | Literal[1]
+```
+
+### Unions with all paths unbound
+
+If the symbol is unbound in all elements of the union, we detect that:
+
+```py
+def _(flag: bool):
+    class C1: ...
+    class C2: ...
+    C = C1 if flag else C2
+
+    # error: [unresolved-attribute] "Type `Literal[C1, C2]` has no attribute `x`"
+    reveal_type(C.x)  # revealed: Unknown
 ```
 
 ## Inherited class attributes
@@ -762,82 +1042,133 @@ reveal_type(A.__mro__)
 reveal_type(A.X)  # revealed: Unknown | Literal[42]
 ```
 
-## Unions with possibly unbound paths
+## Intersections of attributes
 
-### Definite boundness within a class
-
-In this example, the `x` attribute is not defined in the `C2` element of the union:
+### Attribute only available on one element
 
 ```py
-def _(flag1: bool, flag2: bool):
-    class C1:
-        x = 1
+from knot_extensions import Intersection
 
-    class C2: ...
+class A:
+    x: int = 1
 
-    class C3:
-        x = 3
+class B: ...
 
-    C = C1 if flag1 else C2 if flag2 else C3
+def _(a_and_b: Intersection[A, B]):
+    reveal_type(a_and_b.x)  # revealed: int
 
-    # error: [possibly-unbound-attribute] "Attribute `x` on type `Literal[C1, C2, C3]` is possibly unbound"
-    reveal_type(C.x)  # revealed: Unknown | Literal[1, 3]
+# Same for class objects
+def _(a_and_b: Intersection[type[A], type[B]]):
+    reveal_type(a_and_b.x)  # revealed: int
 ```
 
-### Possibly-unbound within a class
-
-We raise the same diagnostic if the attribute is possibly-unbound in at least one element of the
-union:
+### Attribute available on both elements
 
 ```py
-def _(flag: bool, flag1: bool, flag2: bool):
-    class C1:
-        x = 1
+from knot_extensions import Intersection
 
-    class C2:
-        if flag:
-            x = 2
+class P: ...
+class Q: ...
 
-    class C3:
-        x = 3
+class A:
+    x: P = P()
 
-    C = C1 if flag1 else C2 if flag2 else C3
+class B:
+    x: Q = Q()
 
-    # error: [possibly-unbound-attribute] "Attribute `x` on type `Literal[C1, C2, C3]` is possibly unbound"
-    reveal_type(C.x)  # revealed: Unknown | Literal[1, 2, 3]
+def _(a_and_b: Intersection[A, B]):
+    reveal_type(a_and_b.x)  # revealed: P & Q
+
+# Same for class objects
+def _(a_and_b: Intersection[type[A], type[B]]):
+    reveal_type(a_and_b.x)  # revealed: P & Q
 ```
 
-### Attribute possibly unbound on a subclass but not on a superclass
+### Possible unboundness
 
 ```py
+from knot_extensions import Intersection
+
+class P: ...
+class Q: ...
+
 def _(flag: bool):
-    class Foo:
-        x = 1
-
-    class Bar(Foo):
+    class A1:
         if flag:
-            x = 2
+            x: P = P()
 
-    reveal_type(Bar.x)  # revealed: Unknown | Literal[2, 1]
+    class B1: ...
+
+    def inner1(a_and_b: Intersection[A1, B1]):
+        # error: [possibly-unbound-attribute]
+        reveal_type(a_and_b.x)  # revealed: P
+    # Same for class objects
+    def inner1_class(a_and_b: Intersection[type[A1], type[B1]]):
+        # error: [possibly-unbound-attribute]
+        reveal_type(a_and_b.x)  # revealed: P
+
+    class A2:
+        if flag:
+            x: P = P()
+
+    class B1:
+        x: Q = Q()
+
+    def inner2(a_and_b: Intersection[A2, B1]):
+        reveal_type(a_and_b.x)  # revealed: P & Q
+    # Same for class objects
+    def inner2_class(a_and_b: Intersection[type[A2], type[B1]]):
+        reveal_type(a_and_b.x)  # revealed: P & Q
+
+    class A3:
+        if flag:
+            x: P = P()
+
+    class B3:
+        if flag:
+            x: Q = Q()
+
+    def inner3(a_and_b: Intersection[A3, B3]):
+        # error: [possibly-unbound-attribute]
+        reveal_type(a_and_b.x)  # revealed: P & Q
+    # Same for class objects
+    def inner3_class(a_and_b: Intersection[type[A3], type[B3]]):
+        # error: [possibly-unbound-attribute]
+        reveal_type(a_and_b.x)  # revealed: P & Q
+
+    class A4: ...
+    class B4: ...
+
+    def inner4(a_and_b: Intersection[A4, B4]):
+        # error: [unresolved-attribute]
+        reveal_type(a_and_b.x)  # revealed: Unknown
+    # Same for class objects
+    def inner4_class(a_and_b: Intersection[type[A4], type[B4]]):
+        # error: [unresolved-attribute]
+        reveal_type(a_and_b.x)  # revealed: Unknown
 ```
 
-### Attribute possibly unbound on a subclass and on a superclass
+### Intersection of implicit instance attributes
 
 ```py
-def _(flag: bool):
-    class Foo:
-        if flag:
-            x = 1
+from knot_extensions import Intersection
 
-    class Bar(Foo):
-        if flag:
-            x = 2
+class P: ...
+class Q: ...
 
-    # error: [possibly-unbound-attribute]
-    reveal_type(Bar.x)  # revealed: Unknown | Literal[2, 1]
+class A:
+    def __init__(self):
+        self.x: P = P()
+
+class B:
+    def __init__(self):
+        self.x: Q = Q()
+
+def _(a_and_b: Intersection[A, B]):
+    reveal_type(a_and_b.x)  # revealed: P & Q
 ```
 
-### Attribute access on `Any`
+## Attribute access on `Any`
 
 The union of the set of types that `Any` could materialise to is equivalent to `object`. It follows
 from this that attribute access on `Any` resolves to `Any` if the attribute does not exist on
@@ -868,29 +1199,112 @@ reveal_type(C.__mro__)  # revealed: tuple[Literal[C], Literal[B], Any, Literal[A
 reveal_type(C.x)  # revealed: Literal[1] & Any
 ```
 
-### Unions with all paths unbound
+## Classes with custom `__getattr__` methods
 
-If the symbol is unbound in all elements of the union, we detect that:
+### Basic
+
+If a type provides a custom `__getattr__` method, we use the return type of that method as the type
+for unknown attributes. Consider the following `CustomGetAttr` class:
 
 ```py
-def _(flag: bool):
-    class C1: ...
-    class C2: ...
-    C = C1 if flag else C2
+from typing import Literal
 
-    # error: [unresolved-attribute] "Type `Literal[C1, C2]` has no attribute `x`"
-    reveal_type(C.x)  # revealed: Unknown
+def flag() -> bool:
+    return True
+
+class GetAttrReturnType: ...
+
+class CustomGetAttr:
+    class_attr: int = 1
+
+    if flag():
+        possibly_unbound: bytes = b"a"
+
+    def __init__(self) -> None:
+        self.instance_attr: str = "a"
+
+    def __getattr__(self, name: str) -> GetAttrReturnType:
+        return GetAttrReturnType()
+```
+
+We can access arbitrary attributes on instances of this class, and the type of the attribute will be
+`GetAttrReturnType`:
+
+```py
+c = CustomGetAttr()
+
+reveal_type(c.whatever)  # revealed: GetAttrReturnType
+```
+
+If an attribute is defined on the class, it takes precedence over the `__getattr__` method:
+
+```py
+reveal_type(c.class_attr)  # revealed: int
+```
+
+If the class attribute is possibly unbound, we union the type of the attribute with the fallback
+type of the `__getattr__` method:
+
+```py
+reveal_type(c.possibly_unbound)  # revealed: bytes | GetAttrReturnType
+```
+
+Instance attributes also take precedence over the `__getattr__` method:
+
+```py
+# Note: we could attempt to union with the fallback type of `__getattr__` here, as we currently do not
+# attempt to determine if instance attributes are always bound or not. Neither mypy nor pyright do this,
+# so it's not a priority.
+reveal_type(c.instance_attr)  # revealed: str
+```
+
+### Type of the `name` parameter
+
+If the `name` parameter of the `__getattr__` method is annotated with a (union of) literal type(s),
+we only consider the attribute access to be valid if the accessed attribute is one of them:
+
+```py
+from typing import Literal
+
+class Date:
+    def __getattr__(self, name: Literal["day", "month", "year"]) -> int:
+        return 0
+
+date = Date()
+
+reveal_type(date.day)  # revealed: int
+reveal_type(date.month)  # revealed: int
+reveal_type(date.year)  # revealed: int
+
+# error: [unresolved-attribute] "Type `Date` has no attribute `century`"
+reveal_type(date.century)  # revealed: Unknown
+```
+
+### `argparse.Namespace`
+
+A standard library example of a class with a custom `__getattr__` method is `argparse.Namespace`:
+
+```py
+import argparse
+
+def _(ns: argparse.Namespace):
+    reveal_type(ns.whatever)  # revealed: Any
 ```
 
 ## Objects of all types have a `__class__` method
 
+The type of `x.__class__` is the same as `x`'s meta-type. `x.__class__` is always the same value as
+`type(x)`.
+
 ```py
 import typing_extensions
 
 reveal_type(typing_extensions.__class__)  # revealed: Literal[ModuleType]
+reveal_type(type(typing_extensions))  # revealed: Literal[ModuleType]
 
 a = 42
 reveal_type(a.__class__)  # revealed: Literal[int]
+reveal_type(type(a))  # revealed: Literal[int]
 
 b = "42"
 reveal_type(b.__class__)  # revealed: Literal[str]
@@ -906,8 +1320,13 @@ reveal_type(e.__class__)  # revealed: Literal[tuple]
 
 def f(a: int, b: typing_extensions.LiteralString, c: int | str, d: type[str]):
     reveal_type(a.__class__)  # revealed: type[int]
+    reveal_type(type(a))  # revealed: type[int]
+
     reveal_type(b.__class__)  # revealed: Literal[str]
+    reveal_type(type(b))  # revealed: Literal[str]
+
     reveal_type(c.__class__)  # revealed: type[int] | type[str]
+    reveal_type(type(c))  # revealed: type[int] | type[str]
 
     # `type[type]`, a.k.a., either the class `type` or some subclass of `type`.
     # It would be incorrect to infer `Literal[type]` here,
@@ -924,6 +1343,8 @@ reveal_type(Foo.__class__)  # revealed: Literal[type]
 
 ## Module attributes
 
+### Basic
+
 `mod.py`:
 
 ```py
@@ -958,7 +1379,7 @@ for mod.global_symbol in IntIterable():
     pass
 ```
 
-## Nested attributes
+### Nested module attributes
 
 `outer/__init__.py`:
 
@@ -1032,8 +1453,8 @@ Most attribute accesses on bool-literal types are delegated to `builtins.bool`,
 bools are instances of that class:
 
 ```py
-reveal_type(True.__and__)  # revealed: @Todo(decorated method)
-reveal_type(False.__or__)  # revealed: @Todo(decorated method)
+reveal_type(True.__and__)  # revealed: <bound method `__and__` of `Literal[True]`>
+reveal_type(False.__or__)  # revealed: <bound method `__or__` of `Literal[False]`>
 ```
 
 Some attributes are special-cased, however:
@@ -1136,6 +1557,20 @@ class C:
 reveal_type(C().x)  # revealed: Unknown
 ```
 
+### Accessing attributes on `Never`
+
+Arbitrary attributes can be accessed on `Never` without emitting any errors:
+
+```py
+from typing_extensions import Never
+
+def f(never: Never):
+    reveal_type(never.arbitrary_attribute)  # revealed: Never
+
+    # Assigning `Never` to an attribute on `Never` is also allowed:
+    never.another_attribute = never
+```
+
 ### Builtin types attributes
 
 This test can probably be removed eventually, but we currently include it because we do not yet
@@ -1175,6 +1610,7 @@ reveal_type(C.a_none)  # revealed: None
 Some of the tests in the *Class and instance variables* section draw inspiration from
 [pyright's documentation] on this topic.
 
+[descriptor protocol tests]: descriptor_protocol.md
 [pyright's documentation]: https://microsoft.github.io/pyright/#/type-concepts-advanced?id=class-and-instance-variables
 [typing spec on `classvar`]: https://typing.readthedocs.io/en/latest/spec/class-compat.html#classvar
 [`typing.classvar`]: https://docs.python.org/3/library/typing.html#typing.ClassVar

crates/red_knot_python_semantic/resources/mdtest/binary/instances.md

@@ -259,11 +259,17 @@ class A:
 class B:
     __add__ = A()
 
-# TODO: this could be `int` if we declare `B.__add__` using a `Callable` type
-# TODO: Should not be an error: `A` instance is not a method descriptor, don't prepend `self` arg.
-#   Revealed type should be `Unknown | int`.
-# error: [unsupported-operator] "Operator `+` is unsupported between objects of type `B` and `B`"
-reveal_type(B() + B())  # revealed: Unknown
+reveal_type(B() + B())  # revealed: Unknown | int
+```
+
+Note that we union with `Unknown` here because `__add__` is not declared. We do infer just `int` if
+the callable is declared:
+
+```py
+class B2:
+    __add__: A = A()
+
+reveal_type(B2() + B2())  # revealed: int
 ```
 
 ## Integration test: numbers from typeshed
@@ -306,7 +312,7 @@ reveal_type(1 + A())  # revealed: A
 reveal_type(A() + "foo")  # revealed: A
 # TODO should be `A` since `str.__add__` doesn't support `A` instances
 # TODO overloads
-reveal_type("foo" + A())  # revealed: @Todo(return type)
+reveal_type("foo" + A())  # revealed: @Todo(return type of decorated function)
 
 reveal_type(A() + b"foo")  # revealed: A
 # TODO should be `A` since `bytes.__add__` doesn't support `A` instances
@@ -314,7 +320,7 @@ reveal_type(b"foo" + A())  # revealed: bytes
 
 reveal_type(A() + ())  # revealed: A
 # TODO this should be `A`, since `tuple.__add__` doesn't support `A` instances
-reveal_type(() + A())  # revealed: @Todo(return type)
+reveal_type(() + A())  # revealed: @Todo(return type of decorated function)
 
 literal_string_instance = "foo" * 1_000_000_000
 # the test is not testing what it's meant to be testing if this isn't a `LiteralString`:
@@ -323,7 +329,7 @@ reveal_type(literal_string_instance)  # revealed: LiteralString
 reveal_type(A() + literal_string_instance)  # revealed: A
 # TODO should be `A` since `str.__add__` doesn't support `A` instances
 # TODO overloads
-reveal_type(literal_string_instance + A())  # revealed: @Todo(return type)
+reveal_type(literal_string_instance + A())  # revealed: @Todo(return type of decorated function)
 ```
 
 ## Operations involving instances of classes inheriting from `Any`
@@ -351,6 +357,20 @@ class Y(Foo): ...
 reveal_type(X() + Y())  # revealed: int
 ```
 
+## Operations involving types with invalid `__bool__` methods
+
+<!-- snapshot-diagnostics -->
+
+```py
+class NotBoolable:
+    __bool__ = 3
+
+a = NotBoolable()
+
+# error: [unsupported-bool-conversion]
+10 and a and True
+```
+
 ## Unsupported
 
 ### Dunder as instance attribute
@@ -386,10 +406,12 @@ A left-hand dunder method doesn't apply for the right-hand operand, or vice vers
 
 ```py
 class A:
-    def __add__(self, other) -> int: ...
+    def __add__(self, other) -> int:
+        return 1
 
 class B:
-    def __radd__(self, other) -> int: ...
+    def __radd__(self, other) -> int:
+        return 1
 
 class C: ...
 

crates/red_knot_python_semantic/resources/mdtest/binary/integers.md

@@ -10,8 +10,7 @@ reveal_type(-3 // 3)  # revealed: Literal[-1]
 reveal_type(-3 / 3)  # revealed: float
 reveal_type(5 % 3)  # revealed: Literal[2]
 
-# TODO: Should emit `unsupported-operator` but we don't understand the bases of `str`, so we think
-#       it inherits `Unknown`, so we think `str.__radd__` is `Unknown` instead of nonexistent.
+# error: [unsupported-operator] "Operator `+` is unsupported between objects of type `Literal[2]` and `Literal["f"]`"
 reveal_type(2 + "f")  # revealed: Unknown
 
 def lhs(x: int):
@@ -51,9 +50,9 @@ reveal_type(1 ** (largest_u32 + 1))  # revealed: int
 reveal_type(2**largest_u32)  # revealed: int
 
 def variable(x: int):
-    reveal_type(x**2)  # revealed: @Todo(return type)
-    reveal_type(2**x)  # revealed: @Todo(return type)
-    reveal_type(x**x)  # revealed: @Todo(return type)
+    reveal_type(x**2)  # revealed: @Todo(return type of decorated function)
+    reveal_type(2**x)  # revealed: @Todo(return type of decorated function)
+    reveal_type(x**x)  # revealed: @Todo(return type of decorated function)
 ```
 
 ## Division by Zero

crates/red_knot_python_semantic/resources/mdtest/binary/unions.md

@@ -0,0 +1,51 @@
+# Binary operations on union types
+
+Binary operations on union types are only available if they are supported for all possible
+combinations of types:
+
+```py
+def f1(i: int, u: int | None):
+    # error: [unsupported-operator] "Operator `+` is unsupported between objects of type `int` and `int | None`"
+    reveal_type(i + u)  # revealed: Unknown
+    # error: [unsupported-operator] "Operator `+` is unsupported between objects of type `int | None` and `int`"
+    reveal_type(u + i)  # revealed: Unknown
+```
+
+`int` can be added to `int`, and `str` can be added to `str`, but expressions of type `int | str`
+cannot be added, because that would require addition of `int` and `str` or vice versa:
+
+```py
+def f2(i: int, s: str, int_or_str: int | str):
+    i + i
+    s + s
+    # error: [unsupported-operator] "Operator `+` is unsupported between objects of type `int | str` and `int | str`"
+    reveal_type(int_or_str + int_or_str)  # revealed: Unknown
+```
+
+However, if an operation is supported for all possible combinations, the result will be a union of
+the possible outcomes:
+
+```py
+from typing import Literal
+
+def f3(two_or_three: Literal[2, 3], a_or_b: Literal["a", "b"]):
+    reveal_type(two_or_three + two_or_three)  # revealed: Literal[4, 5, 6]
+    reveal_type(two_or_three**two_or_three)  # revealed: Literal[4, 8, 9, 27]
+
+    reveal_type(a_or_b + a_or_b)  # revealed: Literal["aa", "ab", "ba", "bb"]
+
+    reveal_type(two_or_three * a_or_b)  # revealed: Literal["aa", "bb", "aaa", "bbb"]
+```
+
+We treat a type annotation of `float` as a union of `int` and `float`, so union handling is relevant
+here:
+
+```py
+def f4(x: float, y: float):
+    reveal_type(x + y)  # revealed: int | float
+    reveal_type(x - y)  # revealed: int | float
+    reveal_type(x * y)  # revealed: int | float
+    reveal_type(x / y)  # revealed: int | float
+    reveal_type(x // y)  # revealed: int | float
+    reveal_type(x % y)  # revealed: int | float
+```

crates/red_knot_python_semantic/resources/mdtest/boundness_declaredness/public.md

@@ -69,7 +69,8 @@ without raising an error.
 from typing import Any
 
 def any() -> Any: ...
-def flag() -> bool: ...
+def flag() -> bool:
+    return True
 
 a: int
 b: str
@@ -126,7 +127,8 @@ inferred types:
 from typing import Any
 
 def any() -> Any: ...
-def flag() -> bool: ...
+def flag() -> bool:
+    return True
 
 a = 1
 b = 2
@@ -164,7 +166,8 @@ error for both `a` and `b`:
 ```py
 from typing import Any
 
-def flag() -> bool: ...
+def flag() -> bool:
+    return True
 
 if flag():
     a: Any = 1
@@ -194,7 +197,8 @@ seems inconsistent when compared to the case just above.
 `mod.py`:
 
 ```py
-def flag() -> bool: ...
+def flag() -> bool:
+    return True
 
 if flag():
     a: int
@@ -248,7 +252,8 @@ inconsistent when compared to the "possibly-undeclared-and-possibly-unbound" cas
 `mod.py`:
 
 ```py
-def flag() -> bool: ...
+def flag() -> bool:
+    return True
 
 if flag:
     a = 1

crates/red_knot_python_semantic/resources/mdtest/call/builtins.md

@@ -0,0 +1,40 @@
+# Calling builtins
+
+## `bool` with incorrect arguments
+
+```py
+class NotBool:
+    __bool__ = None
+
+# error: [too-many-positional-arguments] "Too many positional arguments to class `bool`: expected 1, got 2"
+bool(1, 2)
+
+# TODO: We should emit an `unsupported-bool-conversion` error here because the argument doesn't implement `__bool__` correctly.
+bool(NotBool())
+```
+
+## Calls to `type()`
+
+A single-argument call to `type()` returns an object that has the argument's meta-type. (This is
+tested more extensively in `crates/red_knot_python_semantic/resources/mdtest/attributes.md`,
+alongside the tests for the `__class__` attribute.)
+
+```py
+reveal_type(type(1))  # revealed: Literal[int]
+```
+
+But a three-argument call to type creates a dynamic instance of the `type` class:
+
+```py
+reveal_type(type("Foo", (), {}))  # revealed: type
+```
+
+Other numbers of arguments are invalid
+
+```py
+# error: [no-matching-overload] "No overload of class `type` matches arguments"
+type("Foo", ())
+
+# error: [no-matching-overload] "No overload of class `type` matches arguments"
+type("Foo", (), {}, weird_other_arg=42)
+```

crates/red_knot_python_semantic/resources/mdtest/call/callable_instance.md

@@ -25,7 +25,8 @@ reveal_type(b)  # revealed: Unknown
 def _(flag: bool):
     class PossiblyNotCallable:
         if flag:
-            def __call__(self) -> int: ...
+            def __call__(self) -> int:
+                return 1
 
     a = PossiblyNotCallable()
     result = a()  # error: "Object of type `PossiblyNotCallable` is not callable (possibly unbound `__call__` method)"
@@ -38,7 +39,8 @@ def _(flag: bool):
 def _(flag: bool):
     if flag:
         class PossiblyUnbound:
-            def __call__(self) -> int: ...
+            def __call__(self) -> int:
+                return 1
 
     # error: [possibly-unresolved-reference]
     a = PossiblyUnbound()
@@ -64,7 +66,8 @@ def _(flag: bool):
         if flag:
             __call__ = 1
         else:
-            def __call__(self) -> int: ...
+            def __call__(self) -> int:
+                return 1
 
     a = NonCallable()
     # error: [call-non-callable] "Object of type `Literal[1]` is not callable"
@@ -82,7 +85,7 @@ class C:
 
 c = C()
 
-# error: 15 [invalid-argument-type] "Object of type `Literal["foo"]` cannot be assigned to parameter 2 (`x`) of function `__call__`; expected type `int`"
+# error: 15 [invalid-argument-type] "Object of type `Literal["foo"]` cannot be assigned to parameter 2 (`x`) of bound method `__call__`; expected type `int`"
 reveal_type(c("foo"))  # revealed: int
 ```
 
@@ -96,7 +99,7 @@ class C:
 
 c = C()
 
-# error: 13 [invalid-argument-type] "Object of type `C` cannot be assigned to parameter 1 (`self`) of function `__call__`; expected type `int`"
+# error: 13 [invalid-argument-type] "Object of type `C` cannot be assigned to parameter 1 (`self`) of bound method `__call__`; expected type `int`"
 reveal_type(c())  # revealed: int
 ```
 

crates/red_knot_python_semantic/resources/mdtest/call/dunder.md

@@ -0,0 +1,219 @@
+# Dunder calls
+
+## Introduction
+
+This test suite explains and documents how dunder methods are looked up and called. Throughout the
+document, we use `__getitem__` as an example, but the same principles apply to other dunder methods.
+
+Dunder methods are implicitly called when using certain syntax. For example, the index operator
+`obj[key]` calls the `__getitem__` method under the hood. Exactly *how* a dunder method is looked up
+and called works slightly different from regular methods. Dunder methods are not looked up on `obj`
+directly, but rather on `type(obj)`. But in many ways, they still *act* as if they were called on
+`obj` directly. If the `__getitem__` member of `type(obj)` is a descriptor, it is called with `obj`
+as the `instance` argument to `__get__`. A desugared version of `obj[key]` is roughly equivalent to
+`getitem_desugared(obj, key)` as defined below:
+
+```py
+from typing import Any
+
+def find_name_in_mro(typ: type, name: str) -> Any:
+    # See implementation in https://docs.python.org/3/howto/descriptor.html#invocation-from-an-instance
+    pass
+
+def getitem_desugared(obj: object, key: object) -> object:
+    getitem_callable = find_name_in_mro(type(obj), "__getitem__")
+    if hasattr(getitem_callable, "__get__"):
+        getitem_callable = getitem_callable.__get__(obj, type(obj))
+
+    return getitem_callable(key)
+```
+
+In the following tests, we demonstrate that we implement this behavior correctly.
+
+## Operating on class objects
+
+If we invoke a dunder method on a class, it is looked up on the *meta* class, since any class is an
+instance of its metaclass:
+
+```py
+class Meta(type):
+    def __getitem__(cls, key: int) -> str:
+        return str(key)
+
+class DunderOnMetaclass(metaclass=Meta):
+    pass
+
+reveal_type(DunderOnMetaclass[0])  # revealed: str
+```
+
+If the dunder method is only present on the class itself, it will not be called:
+
+```py
+class ClassWithNormalDunder:
+    def __getitem__(self, key: int) -> str:
+        return str(key)
+
+# error: [non-subscriptable]
+ClassWithNormalDunder[0]
+```
+
+## Operating on instances
+
+When invoking a dunder method on an instance of a class, it is looked up on the class:
+
+```py
+class ClassWithNormalDunder:
+    def __getitem__(self, key: int) -> str:
+        return str(key)
+
+class_with_normal_dunder = ClassWithNormalDunder()
+
+reveal_type(class_with_normal_dunder[0])  # revealed: str
+```
+
+Which can be demonstrated by trying to attach a dunder method to an instance, which will not work:
+
+```py
+def external_getitem(instance, key: int) -> str:
+    return str(key)
+
+class ThisFails:
+    def __init__(self):
+        self.__getitem__ = external_getitem
+
+this_fails = ThisFails()
+
+# error: [non-subscriptable] "Cannot subscript object of type `ThisFails` with no `__getitem__` method"
+reveal_type(this_fails[0])  # revealed: Unknown
+```
+
+However, the attached dunder method *can* be called if accessed directly:
+
+```py
+reveal_type(this_fails.__getitem__(this_fails, 0))  # revealed: Unknown | str
+```
+
+The instance-level method is also not called when the class-level method is present:
+
+```py
+def external_getitem1(instance, key) -> str:
+    return "a"
+
+def external_getitem2(key) -> int:
+    return 1
+
+def _(flag: bool):
+    class ThisFails:
+        if flag:
+            __getitem__ = external_getitem1
+
+        def __init__(self):
+            self.__getitem__ = external_getitem2
+
+    this_fails = ThisFails()
+
+    # error: [call-possibly-unbound-method]
+    reveal_type(this_fails[0])  # revealed: Unknown | str
+```
+
+## When the dunder is not a method
+
+A dunder can also be a non-method callable:
+
+```py
+class SomeCallable:
+    def __call__(self, key: int) -> str:
+        return str(key)
+
+class ClassWithNonMethodDunder:
+    __getitem__: SomeCallable = SomeCallable()
+
+class_with_callable_dunder = ClassWithNonMethodDunder()
+
+reveal_type(class_with_callable_dunder[0])  # revealed: str
+```
+
+## Dunders are looked up using the descriptor protocol
+
+Here, we demonstrate that the descriptor protocol is invoked when looking up a dunder method. Note
+that the `instance` argument is on object of type `ClassWithDescriptorDunder`:
+
+```py
+from __future__ import annotations
+
+class SomeCallable:
+    def __call__(self, key: int) -> str:
+        return str(key)
+
+class Descriptor:
+    def __get__(self, instance: ClassWithDescriptorDunder, owner: type[ClassWithDescriptorDunder]) -> SomeCallable:
+        return SomeCallable()
+
+class ClassWithDescriptorDunder:
+    __getitem__: Descriptor = Descriptor()
+
+class_with_descriptor_dunder = ClassWithDescriptorDunder()
+
+reveal_type(class_with_descriptor_dunder[0])  # revealed: str
+```
+
+## Dunders can not be overwritten on instances
+
+If we attempt to overwrite a dunder method on an instance, it does not affect the behavior of
+implicit dunder calls:
+
+```py
+class C:
+    def __getitem__(self, key: int) -> str:
+        return str(key)
+
+    def f(self):
+        # TODO: This should emit an `invalid-assignment` diagnostic once we understand the type of `self`
+        self.__getitem__ = None
+
+# This is still fine, and simply calls the `__getitem__` method on the class
+reveal_type(C()[0])  # revealed: str
+```
+
+## Calling a union of dunder methods
+
+```py
+def _(flag: bool):
+    class C:
+        if flag:
+            def __getitem__(self, key: int) -> str:
+                return str(key)
+        else:
+            def __getitem__(self, key: int) -> bytes:
+                return bytes()
+
+    c = C()
+    reveal_type(c[0])  # revealed: str | bytes
+
+    if flag:
+        class D:
+            def __getitem__(self, key: int) -> str:
+                return str(key)
+
+    else:
+        class D:
+            def __getitem__(self, key: int) -> bytes:
+                return bytes()
+
+    d = D()
+    reveal_type(d[0])  # revealed: str | bytes
+```
+
+## Calling a possibly-unbound dunder method
+
+```py
+def _(flag: bool):
+    class C:
+        if flag:
+            def __getitem__(self, key: int) -> str:
+                return str(key)
+
+    c = C()
+    # error: [call-possibly-unbound-method]
+    reveal_type(c[0])  # revealed: str
+```

crates/red_knot_python_semantic/resources/mdtest/call/function.md

@@ -37,6 +37,8 @@ def foo() -> int:
     return 42
 
 def decorator(func) -> Callable[[], int]:
+    # TODO: no error
+    # error: [invalid-return-type]
     return foo
 
 @decorator
@@ -44,7 +46,7 @@ def bar() -> str:
     return "bar"
 
 # TODO: should reveal `int`, as the decorator replaces `bar` with `foo`
-reveal_type(bar())  # revealed: @Todo(return type)
+reveal_type(bar())  # revealed: @Todo(return type of decorated function)
 ```
 
 ## Invalid callable

crates/red_knot_python_semantic/resources/mdtest/call/getattr_static.md

@@ -12,7 +12,7 @@ import inspect
 
 class Descriptor:
     def __get__(self, instance, owner) -> str:
-        return 1
+        return "a"
 
 class C:
     normal: int = 1
@@ -59,7 +59,7 @@ import sys
 reveal_type(inspect.getattr_static(sys, "platform"))  # revealed: LiteralString
 reveal_type(inspect.getattr_static(inspect, "getattr_static"))  # revealed: Literal[getattr_static]
 
-reveal_type(inspect.getattr_static(1, "real"))  # revealed: Literal[1]
+reveal_type(inspect.getattr_static(1, "real"))  # revealed: Literal[real]
 ```
 
 (Implicit) instance attributes can also be accessed through `inspect.getattr_static`:
@@ -72,6 +72,23 @@ class D:
 reveal_type(inspect.getattr_static(D(), "instance_attr"))  # revealed: int
 ```
 
+And attributes on metaclasses can be accessed when probing the class:
+
+```py
+class Meta(type):
+    attr: int = 1
+
+class E(metaclass=Meta): ...
+
+reveal_type(inspect.getattr_static(E, "attr"))  # revealed: int
+```
+
+Metaclass attributes can not be added when probing an instance of the class:
+
+```py
+reveal_type(inspect.getattr_static(E(), "attr", "non_existent"))  # revealed: Literal["non_existent"]
+```
+
 ## Error cases
 
 We can only infer precise types if the attribute is a literal string. In all other cases, we fall

crates/red_knot_python_semantic/resources/mdtest/call/methods.md

@@ -235,24 +235,198 @@ method_wrapper(C(), None)
 method_wrapper(None, C)
 
 # Passing `None` without an `owner` argument is an
-# error: [missing-argument] "No argument provided for required parameter `owner`"
+# error: [no-matching-overload] "No overload of method wrapper `__get__` of function `f` matches arguments"
 method_wrapper(None)
 
 # Passing something that is not assignable to `type` as the `owner` argument is an
-# error: [invalid-argument-type] "Object of type `Literal[1]` cannot be assigned to parameter 2 (`owner`); expected type `type`"
+# error: [no-matching-overload] "No overload of method wrapper `__get__` of function `f` matches arguments"
 method_wrapper(None, 1)
 
 # Passing `None` as the `owner` argument when `instance` is `None` is an
-# error: [invalid-argument-type] "Object of type `None` cannot be assigned to parameter 2 (`owner`); expected type `type`"
+# error: [no-matching-overload] "No overload of method wrapper `__get__` of function `f` matches arguments"
 method_wrapper(None, None)
 
 # Calling `__get__` without any arguments is an
-# error: [missing-argument] "No argument provided for required parameter `instance`"
+# error: [no-matching-overload] "No overload of method wrapper `__get__` of function `f` matches arguments"
 method_wrapper()
 
 # Calling `__get__` with too many positional arguments is an
-# error: [too-many-positional-arguments] "Too many positional arguments: expected 2, got 3"
+# error: [no-matching-overload] "No overload of method wrapper `__get__` of function `f` matches arguments"
 method_wrapper(C(), C, "one too many")
 ```
 
+## Fallback to metaclass
+
+When a method is accessed on a class object, it is looked up on the metaclass if it is not found on
+the class itself. This also creates a bound method that is bound to the class object itself:
+
+```py
+from __future__ import annotations
+
+class Meta(type):
+    def f(cls, arg: int) -> str:
+        return "a"
+
+class C(metaclass=Meta):
+    pass
+
+reveal_type(C.f)  # revealed: <bound method `f` of `Literal[C]`>
+reveal_type(C.f(1))  # revealed: str
+```
+
+The method `f` can not be accessed from an instance of the class:
+
+```py
+# error: [unresolved-attribute] "Type `C` has no attribute `f`"
+C().f
+```
+
+A metaclass function can be shadowed by a method on the class:
+
+```py
+from typing import Any, Literal
+
+class D(metaclass=Meta):
+    def f(arg: int) -> Literal["a"]:
+        return "a"
+
+reveal_type(D.f(1))  # revealed: Literal["a"]
+```
+
+If the class method is possibly unbound, we union the return types:
+
+```py
+def flag() -> bool:
+    return True
+
+class E(metaclass=Meta):
+    if flag():
+        def f(arg: int) -> Any:
+            return "a"
+
+reveal_type(E.f(1))  # revealed: str | Any
+```
+
+## `@classmethod`
+
+### Basic
+
+When a `@classmethod` attribute is accessed, it returns a bound method object, even when accessed on
+the class object itself:
+
+```py
+from __future__ import annotations
+
+class C:
+    @classmethod
+    def f(cls: type[C], x: int) -> str:
+        return "a"
+
+reveal_type(C.f)  # revealed: <bound method `f` of `Literal[C]`>
+reveal_type(C().f)  # revealed: <bound method `f` of `type[C]`>
+```
+
+The `cls` method argument is then implicitly passed as the first argument when calling the method:
+
+```py
+reveal_type(C.f(1))  # revealed: str
+reveal_type(C().f(1))  # revealed: str
+```
+
+When the class method is called incorrectly, we detect it:
+
+```py
+C.f("incorrect")  # error: [invalid-argument-type]
+C.f()  # error: [missing-argument]
+C.f(1, 2)  # error: [too-many-positional-arguments]
+```
+
+If the `cls` parameter is wrongly annotated, we emit an error at the call site:
+
+```py
+class D:
+    @classmethod
+    def f(cls: D):
+        # This function is wrongly annotated, it should be `type[D]` instead of `D`
+        pass
+
+# error: [invalid-argument-type] "Object of type `Literal[D]` cannot be assigned to parameter 1 (`cls`) of bound method `f`; expected type `D`"
+D.f()
+```
+
+When a class method is accessed on a derived class, it is bound to that derived class:
+
+```py
+class Derived(C):
+    pass
+
+reveal_type(Derived.f)  # revealed: <bound method `f` of `Literal[Derived]`>
+reveal_type(Derived().f)  # revealed: <bound method `f` of `type[Derived]`>
+
+reveal_type(Derived.f(1))  # revealed: str
+reveal_type(Derived().f(1))  # revealed: str
+```
+
+### Accessing the classmethod as a static member
+
+Accessing a `@classmethod`-decorated function at runtime returns a `classmethod` object. We
+currently don't model this explicitly:
+
+```py
+from inspect import getattr_static
+
+class C:
+    @classmethod
+    def f(cls): ...
+
+reveal_type(getattr_static(C, "f"))  # revealed: Literal[f]
+reveal_type(getattr_static(C, "f").__get__)  # revealed: <method-wrapper `__get__` of `f`>
+```
+
+But we correctly model how the `classmethod` descriptor works:
+
+```py
+reveal_type(getattr_static(C, "f").__get__(None, C))  # revealed: <bound method `f` of `Literal[C]`>
+reveal_type(getattr_static(C, "f").__get__(C(), C))  # revealed: <bound method `f` of `Literal[C]`>
+reveal_type(getattr_static(C, "f").__get__(C()))  # revealed: <bound method `f` of `type[C]`>
+```
+
+The `owner` argument takes precedence over the `instance` argument:
+
+```py
+reveal_type(getattr_static(C, "f").__get__("dummy", C))  # revealed: <bound method `f` of `Literal[C]`>
+```
+
+### Classmethods mixed with other decorators
+
+When a `@classmethod` is additionally decorated with another decorator, it is still treated as a
+class method:
+
+```py
+from __future__ import annotations
+
+def does_nothing[T](f: T) -> T:
+    return f
+
+class C:
+    @classmethod
+    @does_nothing
+    def f1(cls: type[C], x: int) -> str:
+        return "a"
+
+    @does_nothing
+    @classmethod
+    def f2(cls: type[C], x: int) -> str:
+        return "a"
+
+# TODO: We do not support decorators yet (only limited special cases). Eventually,
+# these should all return `str`:
+
+reveal_type(C.f1(1))  # revealed: @Todo(return type of decorated function)
+reveal_type(C().f1(1))  # revealed: @Todo(return type of decorated function)
+
+reveal_type(C.f2(1))  # revealed: @Todo(return type of decorated function)
+reveal_type(C().f2(1))  # revealed: @Todo(return type of decorated function)
+```
+
 [functions and methods]: https://docs.python.org/3/howto/descriptor.html#functions-and-methods

crates/red_knot_python_semantic/resources/mdtest/call/never.md

@@ -0,0 +1,12 @@
+# Never is callable
+
+The type `Never` is callable with an arbitrary set of arguments. The result is always `Never`.
+
+```py
+from typing_extensions import Never
+
+def f(never: Never):
+    reveal_type(never())  # revealed: Never
+    reveal_type(never(1))  # revealed: Never
+    reveal_type(never(1, "a", never, x=None))  # revealed: Never
+```

crates/red_knot_python_semantic/resources/mdtest/call/subclass_of.md

@@ -0,0 +1,50 @@
+# Call `type[...]`
+
+## Single class
+
+### Trivial constructor
+
+```py
+class C: ...
+
+def _(subclass_of_c: type[C]):
+    reveal_type(subclass_of_c())  # revealed: C
+```
+
+### Non-trivial constructor
+
+```py
+class C:
+    def __init__(self, x: int): ...
+
+def _(subclass_of_c: type[C]):
+    reveal_type(subclass_of_c(1))  # revealed: C
+
+    # TODO: Those should all be errors
+    reveal_type(subclass_of_c("a"))  # revealed: C
+    reveal_type(subclass_of_c())  # revealed: C
+    reveal_type(subclass_of_c(1, 2))  # revealed: C
+```
+
+## Dynamic base
+
+```py
+from typing import Any
+from knot_extensions import Unknown
+
+def _(subclass_of_any: type[Any], subclass_of_unknown: type[Unknown]):
+    reveal_type(subclass_of_any())  # revealed: Any
+    reveal_type(subclass_of_any("any", "args", 1, 2))  # revealed: Any
+    reveal_type(subclass_of_unknown())  # revealed: Unknown
+    reveal_type(subclass_of_unknown("any", "args", 1, 2))  # revealed: Unknown
+```
+
+## Unions of classes
+
+```py
+class A: ...
+class B: ...
+
+def _(subclass_of_ab: type[A | B]):
+    reveal_type(subclass_of_ab())  # revealed: A | B
+```

crates/red_knot_python_semantic/resources/mdtest/call/union.md

@@ -56,6 +56,7 @@ def _(flag: bool, flag2: bool):
     else:
         def f() -> int:
             return 1
+    # TODO we should mention all non-callable elements of the union
     # error: [call-non-callable] "Object of type `Literal[1]` is not callable"
     # revealed: int | Unknown
     reveal_type(f())
@@ -81,8 +82,12 @@ def _(flag: bool):
 Calling a union where the arguments don't match the signature of all variants.
 
 ```py
-def f1(a: int) -> int: ...
-def f2(a: str) -> str: ...
+def f1(a: int) -> int:
+    return a
+
+def f2(a: str) -> str:
+    return a
+
 def _(flag: bool):
     if flag:
         f = f1
@@ -108,3 +113,38 @@ def _(flag: bool):
     x = f(3)
     reveal_type(x)  # revealed: Unknown
 ```
+
+## Union of binding errors
+
+```py
+def f1(): ...
+def f2(): ...
+def _(flag: bool):
+    if flag:
+        f = f1
+    else:
+        f = f2
+
+    # TODO: we should show all errors from the union, not arbitrarily pick one union element
+    # error: [too-many-positional-arguments] "Too many positional arguments to function `f1`: expected 0, got 1"
+    x = f(3)
+    reveal_type(x)  # revealed: Unknown
+```
+
+## One not-callable, one wrong argument
+
+```py
+class C: ...
+
+def f1(): ...
+def _(flag: bool):
+    if flag:
+        f = f1
+    else:
+        f = C()
+
+    # TODO: we should either show all union errors here, or prioritize the not-callable error
+    # error: [too-many-positional-arguments] "Too many positional arguments to function `f1`: expected 0, got 1"
+    x = f(3)
+    reveal_type(x)  # revealed: Unknown
+```

crates/red_knot_python_semantic/resources/mdtest/comparison/instances/membership_test.md

@@ -160,3 +160,45 @@ reveal_type(42 in A())  # revealed: bool
 # error: [unsupported-operator] "Operator `in` is not supported for types `str` and `A`, in comparing `Literal["hello"]` with `A`"
 reveal_type("hello" in A())  # revealed: bool
 ```
+
+## Return type that doesn't implement `__bool__` correctly
+
+`in` and `not in` operations will fail at runtime if the object on the right-hand side of the
+operation has a `__contains__` method that returns a type which is not convertible to `bool`. This
+is because of the way these operations are handled by the Python interpreter at runtime. If we
+assume that `y` is an object that has a `__contains__` method, the Python expression `x in y`
+desugars to a `contains(y, x)` call, where `contains` looks something like this:
+
+```ignore
+def contains(y, x):
+    return bool(type(y).__contains__(y, x))
+```
+
+where the `bool()` conversion itself implicitly calls `__bool__` under the hood.
+
+TODO: Ideally the message would explain to the user what's wrong. E.g,
+
+```ignore
+error: [operator] cannot use `in` operator on object of type `WithContains`
+    note: This is because the `in` operator implicitly calls `WithContains.__contains__`, but `WithContains.__contains__` is invalidly defined
+    note: `WithContains.__contains__` is invalidly defined because it returns an instance of `NotBoolable`, which cannot be evaluated in a boolean context
+    note: `NotBoolable` cannot be evaluated in a boolean context because its `__bool__` attribute is not callable
+```
+
+It may also be more appropriate to use `unsupported-operator` as the error code.
+
+<!-- snapshot-diagnostics -->
+
+```py
+class NotBoolable:
+    __bool__ = 3
+
+class WithContains:
+    def __contains__(self, item) -> NotBoolable:
+        return NotBoolable()
+
+# error: [unsupported-bool-conversion]
+10 in WithContains()
+# error: [unsupported-bool-conversion]
+10 not in WithContains()
+```

crates/red_knot_python_semantic/resources/mdtest/comparison/instances/rich_comparison.md

@@ -154,7 +154,7 @@ reveal_type(B() >= A())  # revealed: LeReturnType
 
 class C:
     def __gt__(self, other: C) -> EqReturnType:
-        return 42
+        return EqReturnType()
 
     def __ge__(self, other: C) -> NeReturnType:
         return NeReturnType()
@@ -345,3 +345,47 @@ def f(x: bool, y: int):
     reveal_type(4.2 < x)  # revealed: bool
     reveal_type(x < 4.2)  # revealed: bool
 ```
+
+## Chained comparisons with objects that don't implement `__bool__` correctly
+
+<!-- snapshot-diagnostics -->
+
+Python implicitly calls `bool` on the comparison result of preceding elements (but not for the last
+element) of a chained comparison.
+
+```py
+class NotBoolable:
+    __bool__ = 3
+
+class Comparable:
+    def __lt__(self, item) -> NotBoolable:
+        return NotBoolable()
+
+    def __gt__(self, item) -> NotBoolable:
+        return NotBoolable()
+
+# error: [unsupported-bool-conversion]
+10 < Comparable() < 20
+# error: [unsupported-bool-conversion]
+10 < Comparable() < Comparable()
+
+Comparable() < Comparable()  # fine
+```
+
+## Callables as comparison dunders
+
+```py
+from typing import Literal
+
+class AlwaysTrue:
+    def __call__(self, other: object) -> Literal[True]:
+        return True
+
+class A:
+    __eq__: AlwaysTrue = AlwaysTrue()
+    __lt__: AlwaysTrue = AlwaysTrue()
+
+reveal_type(A() == A())  # revealed: Literal[True]
+reveal_type(A() < A())  # revealed: Literal[True]
+reveal_type(A() > A())  # revealed: Literal[True]
+```

crates/red_knot_python_semantic/resources/mdtest/comparison/intersections.md

@@ -110,7 +110,8 @@ given operator:
 
 ```py
 class Container:
-    def __contains__(self, x) -> bool: ...
+    def __contains__(self, x) -> bool:
+        return False
 
 class NonContainer: ...
 
@@ -130,7 +131,8 @@ unsupported for the given operator:
 
 ```py
 class Container:
-    def __contains__(self, x) -> bool: ...
+    def __contains__(self, x) -> bool:
+        return False
 
 class NonContainer: ...
 

crates/red_knot_python_semantic/resources/mdtest/comparison/non_bool_returns.md

@@ -22,14 +22,19 @@ Walking through examples:
 from __future__ import annotations
 
 class A:
-    def __lt__(self, other) -> A: ...
-    def __gt__(self, other) -> bool: ...
+    def __lt__(self, other) -> A:
+        return self
+
+    def __gt__(self, other) -> bool:
+        return False
 
 class B:
-    def __lt__(self, other) -> B: ...
+    def __lt__(self, other) -> B:
+        return self
 
 class C:
-    def __lt__(self, other) -> C: ...
+    def __lt__(self, other) -> C:
+        return self
 
 x = A() < B() < C()
 reveal_type(x)  # revealed: A & ~AlwaysTruthy | B

crates/red_knot_python_semantic/resources/mdtest/comparison/tuples.md

@@ -197,7 +197,7 @@ class LtReturnTypeOnB: ...
 
 class B:
     def __lt__(self, o: B) -> LtReturnTypeOnB:
-        return set()
+        return LtReturnTypeOnB()
 
 reveal_type((A(), B()) < (A(), B()))  # revealed: LtReturnType | LtReturnTypeOnB | Literal[False]
 ```
@@ -334,3 +334,61 @@ reveal_type(a is not c)  # revealed: Literal[True]
 For tuples like `tuple[int, ...]`, `tuple[Any, ...]`
 
 // TODO
+
+## Chained comparisons with elements that incorrectly implement `__bool__`
+
+<!-- snapshot-diagnostics -->
+
+For an operation `A() < A()` to succeed at runtime, the `A.__lt__` method does not necessarily need
+to return an object that is convertible to a `bool`. However, the return type _does_ need to be
+convertible to a `bool` for the operation `A() < A() < A()` (a _chained_ comparison) to succeed.
+This is because `A() < A() < A()` desugars to something like this, which involves several implicit
+conversions to `bool`:
+
+```ignore
+def compute_chained_comparison():
+  a1 = A()
+  a2 = A()
+  first_comparison = a1 < a2
+  return first_comparison and (a2 < A())
+```
+
+```py
+class NotBoolable:
+    __bool__ = 5
+
+class Comparable:
+    def __lt__(self, other) -> NotBoolable:
+        return NotBoolable()
+
+    def __gt__(self, other) -> NotBoolable:
+        return NotBoolable()
+
+a = (1, Comparable())
+b = (1, Comparable())
+
+# error: [unsupported-bool-conversion]
+a < b < b
+
+a < b  # fine
+```
+
+## Equality with elements that incorrectly implement `__bool__`
+
+<!-- snapshot-diagnostics -->
+
+Python does not generally attempt to coerce the result of `==` and `!=` operations between two
+arbitrary objects to a `bool`, but a comparison of tuples will fail if the result of comparing any
+pair of elements at equivalent positions cannot be converted to a `bool`:
+
+```py
+class A:
+    def __eq__(self, other) -> NotBoolable:
+        return NotBoolable()
+
+class NotBoolable:
+    __bool__ = None
+
+# error: [unsupported-bool-conversion]
+(A(),) == (A(),)
+```

crates/red_knot_python_semantic/resources/mdtest/comprehensions/basic.md

@@ -104,7 +104,8 @@ class Iterator:
         return 42
 
 class Iterable:
-    def __iter__(self) -> Iterator: ...
+    def __iter__(self) -> Iterator:
+        return Iterator()
 
 # This is fine:
 x = [*Iterable()]

crates/red_knot_python_semantic/resources/mdtest/conditional/if_expression.md

@@ -35,3 +35,13 @@ def _(flag: bool):
     x = 1 if flag else None
     reveal_type(x)  # revealed: Literal[1] | None
 ```
+
+## Condition with object that implements `__bool__` incorrectly
+
+```py
+class NotBoolable:
+    __bool__ = 3
+
+# error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable"
+3 if NotBoolable() else 4
+```

crates/red_knot_python_semantic/resources/mdtest/conditional/if_statement.md

@@ -147,3 +147,17 @@ def _(flag: bool):
 
     reveal_type(y)  # revealed: Literal[0, 1]
 ```
+
+## Condition with object that implements `__bool__` incorrectly
+
+```py
+class NotBoolable:
+    __bool__ = 3
+
+# error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable"
+if NotBoolable():
+    ...
+# error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable"
+elif NotBoolable():
+    ...
+```

crates/red_knot_python_semantic/resources/mdtest/conditional/match.md

@@ -43,3 +43,21 @@ def _(target: int):
 
     reveal_type(y)  # revealed: Literal[2, 3, 4]
 ```
+
+## Guard with object that implements `__bool__` incorrectly
+
+```py
+class NotBoolable:
+    __bool__ = 3
+
+def _(target: int, flag: NotBoolable):
+    y = 1
+    match target:
+        # error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable"
+        case 1 if flag:
+            y = 2
+        case 2:
+            y = 3
+
+    reveal_type(y)  # revealed: Literal[1, 2, 3]
+```

crates/red_knot_python_semantic/resources/mdtest/descriptor_protocol.md

@@ -6,7 +6,9 @@ A descriptor is an attribute value that has one of the methods in the descriptor
 methods are `__get__()`, `__set__()`, and `__delete__()`. If any of those methods are defined for an
 attribute, it is said to be a descriptor.
 
-## Basic example
+## Basic properties
+
+### Example
 
 An introductory example, modeled after a [simple example] in the primer on descriptors, involving a
 descriptor that returns a constant value:
@@ -31,21 +33,17 @@ reveal_type(c.ten)  # revealed: Literal[10]
 reveal_type(C.ten)  # revealed: Literal[10]
 
 # These are fine:
-# TODO: This should not be an error
-c.ten = 10  # error: [invalid-assignment]
+c.ten = 10
 C.ten = 10
 
-# TODO: This should be an error (as the wrong type is being implicitly passed to `Ten.__set__`),
-# but the error message is misleading.
-# error: [invalid-assignment] "Object of type `Literal[11]` is not assignable to attribute `ten` of type `Ten`"
+# error: [invalid-assignment] "Object of type `Literal[11]` is not assignable to attribute `ten` of type `Literal[10]`"
 c.ten = 11
 
-# TODO: same as above
 # error: [invalid-assignment] "Object of type `Literal[11]` is not assignable to attribute `ten` of type `Literal[10]`"
 C.ten = 11
 ```
 
-## Different types for `__get__` and `__set__`
+### Different types for `__get__` and `__set__`
 
 The return type of `__get__` and the value type of `__set__` can be different:
 
@@ -67,28 +65,25 @@ c = C()
 
 reveal_type(c.flexible_int)  # revealed: int | None
 
-# TODO: These should not be errors
-# error: [invalid-assignment]
 c.flexible_int = 42  # okay
+# TODO: This should not be an error
 # error: [invalid-assignment]
 c.flexible_int = "42"  # also okay!
 
 reveal_type(c.flexible_int)  # revealed: int | None
 
-# TODO: This should be an error, but the message needs to be improved.
-# error: [invalid-assignment] "Object of type `None` is not assignable to attribute `flexible_int` of type `FlexibleInt`"
+# TODO: This should be an error
 c.flexible_int = None  # not okay
 
 reveal_type(c.flexible_int)  # revealed: int | None
 ```
 
-## Data and non-data descriptors
+### Data and non-data descriptors
 
-Descriptors that define `__set__` or `__delete__` are called *data descriptors*. An example\
-of a data descriptor is a `property` with a setter and/or a deleter.\
-Descriptors that only define `__get__`, meanwhile, are called *non-data descriptors*. Examples
-include\
-functions, `classmethod` or `staticmethod`).
+Descriptors that define `__set__` or `__delete__` are called *data descriptors*. An example of a
+data descriptor is a `property` with a setter and/or a deleter. Descriptors that only define
+`__get__`, meanwhile, are called *non-data descriptors*. Examples include functions, `classmethod`
+or `staticmethod`.
 
 The precedence chain for attribute access is (1) data descriptors, (2) instance attributes, and (3)
 non-data descriptors.
@@ -100,7 +95,7 @@ class DataDescriptor:
     def __get__(self, instance: object, owner: type | None = None) -> Literal["data"]:
         return "data"
 
-    def __set__(self, instance: int, value) -> None:
+    def __set__(self, instance: object, value: int) -> None:
         pass
 
 class NonDataDescriptor:
@@ -124,12 +119,7 @@ class C:
 
 c = C()
 
-# TODO: This should ideally be `Unknown | Literal["data"]`.
-#
-#     - Pyright also wrongly shows `int | Literal['data']` here
-#     - Mypy shows Literal["data"] here, but also shows Literal["non-data"] below.
-#
-reveal_type(c.data_descriptor)  # revealed: Unknown | Literal["data", 1]
+reveal_type(c.data_descriptor)  # revealed: Unknown | Literal["data"]
 
 reveal_type(c.non_data_descriptor)  # revealed: Unknown | Literal["non-data", 1]
 
@@ -143,79 +133,58 @@ reveal_type(C.non_data_descriptor)  # revealed: Unknown | Literal["non-data"]
 C.data_descriptor = "something else"  # This is okay
 ```
 
-## Built-in `property` descriptor
+### Partial fall back
 
-The built-in `property` decorator creates a descriptor. The names for attribute reads/writes are
-determined by the return type of the `name` method and the parameter type of the setter,
-respectively.
+Our implementation of the descriptor protocol takes into account that symbols can be possibly
+unbound. In those cases, we fall back to lower precedence steps of the descriptor protocol and union
+all possible results accordingly. We start by defining a data and a non-data descriptor:
 
 ```py
-class C:
-    _name: str | None = None
+from typing import Literal
 
-    @property
-    def name(self) -> str:
-        return self._name or "Unset"
-    # TODO: No diagnostic should be emitted here
-    # error: [unresolved-attribute] "Type `Literal[name]` has no attribute `setter`"
-    @name.setter
-    def name(self, value: str | None) -> None:
-        self._value = value
+class DataDescriptor:
+    def __get__(self, instance: object, owner: type | None = None) -> Literal["data"]:
+        return "data"
 
-c = C()
+    def __set__(self, instance: object, value: int) -> None:
+        pass
 
-reveal_type(c._name)  # revealed: str | None
-
-# Should be `str`
-reveal_type(c.name)  # revealed: @Todo(decorated method)
-
-# Should be `builtins.property`
-reveal_type(C.name)  # revealed: Literal[name]
-
-# This is fine:
-c.name = "new"
-
-c.name = None
-
-# TODO: this should be an error
-c.name = 42
+class NonDataDescriptor:
+    def __get__(self, instance: object, owner: type | None = None) -> Literal["non-data"]:
+        return "non-data"
 ```
 
-## Built-in `classmethod` descriptor
-
-Similarly to `property`, `classmethod` decorator creates an implicit descriptor that binds the first
-argument to the class instead of the instance.
+Then, we demonstrate that we fall back to an instance attribute if a data descriptor is possibly
+unbound:
 
 ```py
-class C:
-    def __init__(self, value: str) -> None:
-        self._name: str = value
+def f1(flag: bool):
+    class C1:
+        if flag:
+            attr = DataDescriptor()
 
-    @classmethod
-    def factory(cls, value: str) -> "C":
-        return cls(value)
+        def f(self):
+            self.attr = "normal"
 
-    @classmethod
-    def get_name(cls) -> str:
-        return cls.__name__
-
-c1 = C.factory("test")  # okay
-
-# TODO: should be `C`
-reveal_type(c1)  # revealed: @Todo(return type)
-
-# TODO: should be `str`
-reveal_type(C.get_name())  # revealed: @Todo(return type)
-
-# TODO: should be `str`
-reveal_type(C("42").get_name())  # revealed: @Todo(decorated method)
+    reveal_type(C1().attr)  # revealed: Unknown | Literal["data", "normal"]
 ```
 
-## Descriptors only work when used as class variables
+We never treat implicit instance attributes as definitely bound, so we fall back to the non-data
+descriptor here:
 
-From the descriptor guide:
+```py
+def f2(flag: bool):
+    class C2:
+        def f(self):
+            self.attr = "normal"
+        attr = NonDataDescriptor()
 
-> Descriptors only work when used as class variables. When put in instances, they have no effect.
+    reveal_type(C2().attr)  # revealed: Unknown | Literal["non-data", "normal"]
+```
+
+### Descriptors only work when used as class variables
+
+Descriptors only work when used as class variables. When put in instances, they have no effect.
 
 ```py
 from typing import Literal
@@ -228,8 +197,182 @@ class C:
     def __init__(self):
         self.ten: Ten = Ten()
 
-# TODO: Should be Ten
-reveal_type(C().ten)  # revealed: Literal[10]
+reveal_type(C().ten)  # revealed: Ten
+```
+
+## Descriptor protocol for class objects
+
+When attributes are accessed on a class object, the following [precedence chain] is used:
+
+- Data descriptor on the metaclass
+- Data or non-data descriptor on the class
+- Class attribute
+- Non-data descriptor on the metaclass
+- Metaclass attribute
+
+To verify this, we define a data and a non-data descriptor:
+
+```py
+from typing import Literal, Any
+
+class DataDescriptor:
+    def __get__(self, instance: object, owner: type | None = None) -> Literal["data"]:
+        return "data"
+
+    def __set__(self, instance: object, value: str) -> None:
+        pass
+
+class NonDataDescriptor:
+    def __get__(self, instance: object, owner: type | None = None) -> Literal["non-data"]:
+        return "non-data"
+```
+
+First, we make sure that the descriptors are correctly accessed when defined on the metaclass or the
+class:
+
+```py
+class Meta1(type):
+    meta_data_descriptor: DataDescriptor = DataDescriptor()
+    meta_non_data_descriptor: NonDataDescriptor = NonDataDescriptor()
+
+class C1(metaclass=Meta1):
+    class_data_descriptor: DataDescriptor = DataDescriptor()
+    class_non_data_descriptor: NonDataDescriptor = NonDataDescriptor()
+
+reveal_type(C1.meta_data_descriptor)  # revealed: Literal["data"]
+reveal_type(C1.meta_non_data_descriptor)  # revealed: Literal["non-data"]
+
+reveal_type(C1.class_data_descriptor)  # revealed: Literal["data"]
+reveal_type(C1.class_non_data_descriptor)  # revealed: Literal["non-data"]
+```
+
+Next, we demonstrate that a *metaclass data descriptor* takes precedence over all class-level
+attributes:
+
+```py
+class Meta2(type):
+    meta_data_descriptor1: DataDescriptor = DataDescriptor()
+    meta_data_descriptor2: DataDescriptor = DataDescriptor()
+
+class ClassLevelDataDescriptor:
+    def __get__(self, instance: object, owner: type | None = None) -> Literal["class level data descriptor"]:
+        return "class level data descriptor"
+
+    def __set__(self, instance: object, value: str) -> None:
+        pass
+
+class C2(metaclass=Meta2):
+    meta_data_descriptor1: Literal["value on class"] = "value on class"
+    meta_data_descriptor2: ClassLevelDataDescriptor = ClassLevelDataDescriptor()
+
+reveal_type(C2.meta_data_descriptor1)  # revealed: Literal["data"]
+reveal_type(C2.meta_data_descriptor2)  # revealed: Literal["data"]
+```
+
+On the other hand, normal metaclass attributes and metaclass non-data descriptors are shadowed by
+class-level attributes (descriptor or not):
+
+```py
+class Meta3(type):
+    meta_attribute1: Literal["value on metaclass"] = "value on metaclass"
+    meta_attribute2: Literal["value on metaclass"] = "value on metaclass"
+    meta_non_data_descriptor1: NonDataDescriptor = NonDataDescriptor()
+    meta_non_data_descriptor2: NonDataDescriptor = NonDataDescriptor()
+
+class C3(metaclass=Meta3):
+    meta_attribute1: Literal["value on class"] = "value on class"
+    meta_attribute2: ClassLevelDataDescriptor = ClassLevelDataDescriptor()
+    meta_non_data_descriptor1: Literal["value on class"] = "value on class"
+    meta_non_data_descriptor2: ClassLevelDataDescriptor = ClassLevelDataDescriptor()
+
+reveal_type(C3.meta_attribute1)  # revealed: Literal["value on class"]
+reveal_type(C3.meta_attribute2)  # revealed: Literal["class level data descriptor"]
+reveal_type(C3.meta_non_data_descriptor1)  # revealed: Literal["value on class"]
+reveal_type(C3.meta_non_data_descriptor2)  # revealed: Literal["class level data descriptor"]
+```
+
+Finally, metaclass attributes and metaclass non-data descriptors are only accessible when they are
+not shadowed by class-level attributes:
+
+```py
+class Meta4(type):
+    meta_attribute: Literal["value on metaclass"] = "value on metaclass"
+    meta_non_data_descriptor: NonDataDescriptor = NonDataDescriptor()
+
+class C4(metaclass=Meta4): ...
+
+reveal_type(C4.meta_attribute)  # revealed: Literal["value on metaclass"]
+reveal_type(C4.meta_non_data_descriptor)  # revealed: Literal["non-data"]
+```
+
+When a metaclass data descriptor is possibly unbound, we union the result type of its `__get__`
+method with an underlying class level attribute, if present:
+
+```py
+def _(flag: bool):
+    class Meta5(type):
+        if flag:
+            meta_data_descriptor1: DataDescriptor = DataDescriptor()
+            meta_data_descriptor2: DataDescriptor = DataDescriptor()
+
+    class C5(metaclass=Meta5):
+        meta_data_descriptor1: Literal["value on class"] = "value on class"
+
+    reveal_type(C5.meta_data_descriptor1)  # revealed: Literal["data", "value on class"]
+    # error: [possibly-unbound-attribute]
+    reveal_type(C5.meta_data_descriptor2)  # revealed: Literal["data"]
+```
+
+When a class-level attribute is possibly unbound, we union its (descriptor protocol) type with the
+metaclass attribute (unless it's a data descriptor, which always takes precedence):
+
+```py
+from typing import Any
+
+def _(flag: bool):
+    class Meta6(type):
+        attribute1: DataDescriptor = DataDescriptor()
+        attribute2: NonDataDescriptor = NonDataDescriptor()
+        attribute3: Literal["value on metaclass"] = "value on metaclass"
+
+    class C6(metaclass=Meta6):
+        if flag:
+            attribute1: Literal["value on class"] = "value on class"
+            attribute2: Literal["value on class"] = "value on class"
+            attribute3: Literal["value on class"] = "value on class"
+            attribute4: Literal["value on class"] = "value on class"
+
+    reveal_type(C6.attribute1)  # revealed: Literal["data"]
+    reveal_type(C6.attribute2)  # revealed: Literal["non-data", "value on class"]
+    reveal_type(C6.attribute3)  # revealed: Literal["value on metaclass", "value on class"]
+    # error: [possibly-unbound-attribute]
+    reveal_type(C6.attribute4)  # revealed: Literal["value on class"]
+```
+
+Finally, we can also have unions of various types of attributes:
+
+```py
+def _(flag: bool):
+    class Meta7(type):
+        if flag:
+            union_of_metaclass_attributes: Literal[1] = 1
+            union_of_metaclass_data_descriptor_and_attribute: DataDescriptor = DataDescriptor()
+        else:
+            union_of_metaclass_attributes: Literal[2] = 2
+            union_of_metaclass_data_descriptor_and_attribute: Literal[2] = 2
+
+    class C7(metaclass=Meta7):
+        if flag:
+            union_of_class_attributes: Literal[1] = 1
+            union_of_class_data_descriptor_and_attribute: DataDescriptor = DataDescriptor()
+        else:
+            union_of_class_attributes: Literal[2] = 2
+            union_of_class_data_descriptor_and_attribute: Literal[2] = 2
+
+    reveal_type(C7.union_of_metaclass_attributes)  # revealed: Literal[1, 2]
+    reveal_type(C7.union_of_metaclass_data_descriptor_and_attribute)  # revealed: Literal["data", 2]
+    reveal_type(C7.union_of_class_attributes)  # revealed: Literal[1, 2]
+    reveal_type(C7.union_of_class_data_descriptor_and_attribute)  # revealed: Literal["data", 2]
 ```
 
 ## Descriptors distinguishing between class and instance access
@@ -262,63 +405,12 @@ reveal_type(C.d)  # revealed: LiteralString
 reveal_type(C().d)  # revealed: LiteralString
 ```
 
-## Undeclared descriptor arguments
+## Descriptor protocol for dunder methods
 
-If a descriptor attribute is not declared, we union with `Unknown`, just like for regular
-attributes, since that attribute could be overwritten externally. Even a data descriptor with a
-`__set__` method can be overwritten when accessed through a class object.
-
-```py
-class Descriptor:
-    def __get__(self, instance: object, owner: type | None = None) -> int:
-        return 1
-
-    def __set__(self, instance: object, value: int) -> None:
-        pass
-
-class C:
-    descriptor = Descriptor()
-
-C.descriptor = "something else"
-
-# This could also be `Literal["something else"]` if we support narrowing of attribute types based on assignments
-reveal_type(C.descriptor)  # revealed: Unknown | int
-```
-
-## Descriptors with incorrect `__get__` signature
-
-```py
-class Descriptor:
-    # `__get__` method with missing parameters:
-    def __get__(self) -> int:
-        return 1
-
-class C:
-    descriptor: Descriptor = Descriptor()
-
-# TODO: This should be an error
-reveal_type(C.descriptor)  # revealed: Descriptor
-```
-
-## Possibly-unbound `__get__` method
-
-```py
-def _(flag: bool):
-    class MaybeDescriptor:
-        if flag:
-            def __get__(self, instance: object, owner: type | None = None) -> int:
-                return 1
-
-    class C:
-        descriptor: MaybeDescriptor = MaybeDescriptor()
-
-    # TODO: This should be `MaybeDescriptor | int`
-    reveal_type(C.descriptor)  # revealed: int
-```
-
-## Dunder methods
-
-Dunder methods are looked up on the meta type, but we still need to invoke the descriptor protocol:
+Dunder methods are always looked up on the meta-type. There is no instance fallback. This means that
+an implicit dunder call on an instance-like object will not only look up the dunder method on the
+class object, without considering instance attributes. And an implicit dunder call on a class object
+will look up the dunder method on the metaclass, without considering class attributes.
 
 ```py
 class SomeCallable:
@@ -338,7 +430,77 @@ reveal_type(b_instance(1))  # revealed: str
 b_instance("bla")  # error: [invalid-argument-type]
 ```
 
-## Functions as descriptors
+## Special descriptors
+
+### Built-in `property` descriptor
+
+The built-in `property` decorator creates a descriptor. The names for attribute reads/writes are
+determined by the return type of the `name` method and the parameter type of the setter,
+respectively.
+
+```py
+class C:
+    _name: str | None = None
+
+    @property
+    def name(self) -> str:
+        return self._name or "Unset"
+    # TODO: No diagnostic should be emitted here
+    # error: [unresolved-attribute] "Type `Literal[name]` has no attribute `setter`"
+    @name.setter
+    def name(self, value: str | None) -> None:
+        self._value = value
+
+c = C()
+
+reveal_type(c._name)  # revealed: str | None
+
+# TODO: Should be `str`
+reveal_type(c.name)  # revealed: <bound method `name` of `C`>
+
+# Should be `builtins.property`
+reveal_type(C.name)  # revealed: Literal[name]
+
+# TODO: These should not emit errors
+# error: [invalid-assignment]
+c.name = "new"
+
+# error: [invalid-assignment]
+c.name = None
+
+# TODO: this should be an error, but with a proper error message
+# error: [invalid-assignment] "Object of type `Literal[42]` is not assignable to attribute `name` of type `<bound method `name` of `C`>`"
+c.name = 42
+```
+
+### Built-in `classmethod` descriptor
+
+Similarly to `property`, `classmethod` decorator creates an implicit descriptor that binds the first
+argument to the class instead of the instance.
+
+```py
+class C:
+    def __init__(self, value: str) -> None:
+        self._name: str = value
+
+    @classmethod
+    def factory(cls, value: str) -> "C":
+        return cls(value)
+
+    @classmethod
+    def get_name(cls) -> str:
+        return cls.__name__
+
+c1 = C.factory("test")  # okay
+
+reveal_type(c1)  # revealed: C
+
+reveal_type(C.get_name())  # revealed: str
+
+reveal_type(C("42").get_name())  # revealed: str
+```
+
+### Functions as descriptors
 
 Functions are descriptors because they implement a `__get__` method. This is crucial in making sure
 that method calls work as expected. See [this test suite](./call/methods.md) for more information.
@@ -388,32 +550,202 @@ Finally, we test some error cases for the call to the wrapper descriptor:
 
 ```py
 # Calling the wrapper descriptor without any arguments is an
-# error: [missing-argument] "No arguments provided for required parameters `self`, `instance`"
+# error: [no-matching-overload] "No overload of wrapper descriptor `FunctionType.__get__` matches arguments"
 wrapper_descriptor()
 
 # Calling it without the `instance` argument is an also an
-# error: [missing-argument] "No argument provided for required parameter `instance`"
+# error: [no-matching-overload] "No overload of wrapper descriptor `FunctionType.__get__` matches arguments"
 wrapper_descriptor(f)
 
 # Calling it without the `owner` argument if `instance` is not `None` is an
-# error: [missing-argument] "No argument provided for required parameter `owner`"
+# error: [no-matching-overload] "No overload of wrapper descriptor `FunctionType.__get__` matches arguments"
 wrapper_descriptor(f, None)
 
 # But calling it with an instance is fine (in this case, the `owner` argument is optional):
 wrapper_descriptor(f, C())
 
 # Calling it with something that is not a `FunctionType` as the first argument is an
-# error: [invalid-argument-type] "Object of type `Literal[1]` cannot be assigned to parameter 1 (`self`); expected type `FunctionType`"
+# error: [no-matching-overload] "No overload of wrapper descriptor `FunctionType.__get__` matches arguments"
 wrapper_descriptor(1, None, type(f))
 
 # Calling it with something that is not a `type` as the `owner` argument is an
-# error: [invalid-argument-type] "Object of type `Literal[f]` cannot be assigned to parameter 3 (`owner`); expected type `type`"
+# error: [no-matching-overload] "No overload of wrapper descriptor `FunctionType.__get__` matches arguments"
 wrapper_descriptor(f, None, f)
 
 # Calling it with too many positional arguments is an
-# error: [too-many-positional-arguments] "Too many positional arguments: expected 3, got 4"
+# error: [no-matching-overload] "No overload of wrapper descriptor `FunctionType.__get__` matches arguments"
 wrapper_descriptor(f, None, type(f), "one too many")
 ```
 
+## Error handling and edge cases
+
+### `__get__` is called with correct arguments
+
+This test makes sure that we call `__get__` with the right argument types for various scenarios:
+
+```py
+from __future__ import annotations
+
+class TailoredForClassObjectAccess:
+    def __get__(self, instance: None, owner: type[C]) -> int:
+        return 1
+
+class TailoredForInstanceAccess:
+    def __get__(self, instance: C, owner: type[C] | None = None) -> str:
+        return "a"
+
+class TailoredForMetaclassAccess:
+    def __get__(self, instance: type[C], owner: type[Meta]) -> bytes:
+        return b"a"
+
+class Meta(type):
+    metaclass_access: TailoredForMetaclassAccess = TailoredForMetaclassAccess()
+
+class C(metaclass=Meta):
+    class_object_access: TailoredForClassObjectAccess = TailoredForClassObjectAccess()
+    instance_access: TailoredForInstanceAccess = TailoredForInstanceAccess()
+
+reveal_type(C.class_object_access)  # revealed: int
+reveal_type(C().instance_access)  # revealed: str
+reveal_type(C.metaclass_access)  # revealed: bytes
+
+# TODO: These should emit a diagnostic
+reveal_type(C().class_object_access)  # revealed: TailoredForClassObjectAccess
+reveal_type(C.instance_access)  # revealed: TailoredForInstanceAccess
+```
+
+### Descriptors with incorrect `__get__` signature
+
+```py
+class Descriptor:
+    # `__get__` method with missing parameters:
+    def __get__(self) -> int:
+        return 1
+
+class C:
+    descriptor: Descriptor = Descriptor()
+
+# TODO: This should be an error
+reveal_type(C.descriptor)  # revealed: Descriptor
+
+# TODO: This should be an error
+reveal_type(C().descriptor)  # revealed: Descriptor
+```
+
+### Undeclared descriptor arguments
+
+If a descriptor attribute is not declared, we union with `Unknown`, just like for regular
+attributes, since that attribute could be overwritten externally. Even a data descriptor with a
+`__set__` method can be overwritten when accessed through a class object.
+
+```py
+class Descriptor:
+    def __get__(self, instance: object, owner: type | None = None) -> int:
+        return 1
+
+    def __set__(self, instance: object, value: int) -> None:
+        pass
+
+class C:
+    descriptor = Descriptor()
+
+C.descriptor = "something else"
+
+# This could also be `Literal["something else"]` if we support narrowing of attribute types based on assignments
+reveal_type(C.descriptor)  # revealed: Unknown | int
+```
+
+### Possibly unbound descriptor attributes
+
+```py
+class DataDescriptor:
+    def __get__(self, instance: object, owner: type | None = None) -> int:
+        return 1
+
+    def __set__(self, instance: int, value) -> None:
+        pass
+
+class NonDataDescriptor:
+    def __get__(self, instance: object, owner: type | None = None) -> int:
+        return 1
+
+def _(flag: bool):
+    class PossiblyUnbound:
+        if flag:
+            non_data: NonDataDescriptor = NonDataDescriptor()
+            data: DataDescriptor = DataDescriptor()
+
+    # error: [possibly-unbound-attribute] "Attribute `non_data` on type `Literal[PossiblyUnbound]` is possibly unbound"
+    reveal_type(PossiblyUnbound.non_data)  # revealed: int
+
+    # error: [possibly-unbound-attribute] "Attribute `non_data` on type `PossiblyUnbound` is possibly unbound"
+    reveal_type(PossiblyUnbound().non_data)  # revealed: int
+
+    # error: [possibly-unbound-attribute] "Attribute `data` on type `Literal[PossiblyUnbound]` is possibly unbound"
+    reveal_type(PossiblyUnbound.data)  # revealed: int
+
+    # error: [possibly-unbound-attribute] "Attribute `data` on type `PossiblyUnbound` is possibly unbound"
+    reveal_type(PossiblyUnbound().data)  # revealed: int
+```
+
+### Possibly-unbound `__get__` method
+
+```py
+def _(flag: bool):
+    class MaybeDescriptor:
+        if flag:
+            def __get__(self, instance: object, owner: type | None = None) -> int:
+                return 1
+
+    class C:
+        descriptor: MaybeDescriptor = MaybeDescriptor()
+
+    reveal_type(C.descriptor)  # revealed: int | MaybeDescriptor
+
+    reveal_type(C().descriptor)  # revealed: int | MaybeDescriptor
+```
+
+### Descriptors with non-function `__get__` callables that are descriptors themselves
+
+The descriptor protocol is recursive, i.e. looking up `__get__` can involve triggering the
+descriptor protocol on the callable's `__call__` method:
+
+```py
+from __future__ import annotations
+
+class ReturnedCallable2:
+    def __call__(self, descriptor: Descriptor1, instance: None, owner: type[C]) -> int:
+        return 1
+
+class ReturnedCallable1:
+    def __call__(self, descriptor: Descriptor2, instance: Callable1, owner: type[Callable1]) -> ReturnedCallable2:
+        return ReturnedCallable2()
+
+class Callable3:
+    def __call__(self, descriptor: Descriptor3, instance: Callable2, owner: type[Callable2]) -> ReturnedCallable1:
+        return ReturnedCallable1()
+
+class Descriptor3:
+    __get__: Callable3 = Callable3()
+
+class Callable2:
+    __call__: Descriptor3 = Descriptor3()
+
+class Descriptor2:
+    __get__: Callable2 = Callable2()
+
+class Callable1:
+    __call__: Descriptor2 = Descriptor2()
+
+class Descriptor1:
+    __get__: Callable1 = Callable1()
+
+class C:
+    d: Descriptor1 = Descriptor1()
+
+reveal_type(C.d)  # revealed: int
+```
+
 [descriptors]: https://docs.python.org/3/howto/descriptor.html
+[precedence chain]: https://github.com/python/cpython/blob/3.13/Objects/typeobject.c#L5393-L5481
 [simple example]: https://docs.python.org/3/howto/descriptor.html#simple-example-a-descriptor-that-returns-a-constant

crates/red_knot_python_semantic/resources/mdtest/diagnostics/invalid_argument_type.md

@@ -182,3 +182,16 @@ class C:
 c = C()
 c("wrong")  # error: [invalid-argument-type]
 ```
+
+## Calls to methods
+
+Tests that we also see a reference to a function if the callable is a bound method.
+
+```py
+class C:
+    def square(self, x: int) -> int:
+        return x * x
+
+c = C()
+c.square("hello")  # error: [invalid-argument-type]
+```

crates/red_knot_python_semantic/resources/mdtest/diagnostics/no_matching_overload.md

@@ -0,0 +1,14 @@
+# No matching overload diagnostics
+
+<!-- snapshot-diagnostics -->
+
+## Calls to overloaded functions
+
+TODO: Note that we do not yet support the `@overload` decorator to define overloaded functions in
+real Python code. We are instead testing a special-cased function where we create an overloaded
+signature internally. Update this to an `@overload` function in the Python snippet itself once we
+can.
+
+```py
+type("Foo", ())  # error: [no-matching-overload]
+```

crates/red_knot_python_semantic/resources/mdtest/expression/assert.md

@@ -0,0 +1,9 @@
+## Condition with object that implements `__bool__` incorrectly
+
+```py
+class NotBoolable:
+    __bool__ = 3
+
+# error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable"
+assert NotBoolable()
+```

crates/red_knot_python_semantic/resources/mdtest/expression/boolean.md

@@ -101,3 +101,55 @@ reveal_type(bool([]))  # revealed: bool
 reveal_type(bool({}))  # revealed: bool
 reveal_type(bool(set()))  # revealed: bool
 ```
+
+## `__bool__` returning `NoReturn`
+
+```py
+from typing import NoReturn
+
+class NotBoolable:
+    def __bool__(self) -> NoReturn:
+        raise NotImplementedError("This object can't be converted to a boolean")
+
+# TODO: This should emit an error that `NotBoolable` can't be converted to a bool but it currently doesn't
+#   because `Never` is assignable to `bool`. This probably requires dead code analysis to fix.
+if NotBoolable():
+    ...
+```
+
+## Not callable `__bool__`
+
+```py
+class NotBoolable:
+    __bool__ = None
+
+# error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable"
+if NotBoolable():
+    ...
+```
+
+## Not-boolable union
+
+```py
+def test(cond: bool):
+    class NotBoolable:
+        __bool__ = None if cond else 3
+
+    # error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `NotBoolable`; it incorrectly implements `__bool__`"
+    if NotBoolable():
+        ...
+```
+
+## Union with some variants implementing `__bool__` incorrectly
+
+```py
+def test(cond: bool):
+    class NotBoolable:
+        __bool__ = None
+
+    a = 10 if cond else NotBoolable()
+
+    # error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `Literal[10] | NotBoolable`; its `__bool__` method isn't callable"
+    if a:
+        ...
+```

crates/red_knot_python_semantic/resources/mdtest/expression/lambda.md

@@ -0,0 +1,100 @@
+# `lambda` expression
+
+## No parameters
+
+`lambda` expressions can be defined without any parameters.
+
+```py
+reveal_type(lambda: 1)  # revealed: () -> @Todo(lambda return type)
+
+# error: [unresolved-reference]
+reveal_type(lambda: a)  # revealed: () -> @Todo(lambda return type)
+```
+
+## With parameters
+
+Unlike parameters in function definition, the parameters in a `lambda` expression cannot be
+annotated.
+
+```py
+reveal_type(lambda a: a)  # revealed: (a) -> @Todo(lambda return type)
+reveal_type(lambda a, b: a + b)  # revealed: (a, b) -> @Todo(lambda return type)
+```
+
+But, it can have default values:
+
+```py
+reveal_type(lambda a=1: a)  # revealed: (a=Literal[1]) -> @Todo(lambda return type)
+reveal_type(lambda a, b=2: a)  # revealed: (a, b=Literal[2]) -> @Todo(lambda return type)
+```
+
+And, positional-only parameters:
+
+```py
+reveal_type(lambda a, b, /, c: c)  # revealed: (a, b, /, c) -> @Todo(lambda return type)
+```
+
+And, keyword-only parameters:
+
+```py
+reveal_type(lambda a, *, b=2, c: b)  # revealed: (a, *, b=Literal[2], c) -> @Todo(lambda return type)
+```
+
+And, variadic parameter:
+
+```py
+reveal_type(lambda *args: args)  # revealed: (*args) -> @Todo(lambda return type)
+```
+
+And, keyword-varidic parameter:
+
+```py
+reveal_type(lambda **kwargs: kwargs)  # revealed: (**kwargs) -> @Todo(lambda return type)
+```
+
+Mixing all of them together:
+
+```py
+# revealed: (a, b, /, c=Literal[True], *args, *, d=Literal["default"], e=Literal[5], **kwargs) -> @Todo(lambda return type)
+reveal_type(lambda a, b, /, c=True, *args, d="default", e=5, **kwargs: None)
+```
+
+## Parameter type
+
+In addition to correctly inferring the `lambda` expression, the parameters should also be inferred
+correctly.
+
+Using a parameter with no default value:
+
+```py
+lambda x: reveal_type(x)  # revealed: Unknown
+```
+
+Using a parameter with default value:
+
+```py
+lambda x=1: reveal_type(x)  # revealed: Unknown | Literal[1]
+```
+
+Using a variadic paramter:
+
+```py
+# TODO: should be `tuple[Unknown, ...]` (needs generics)
+lambda *args: reveal_type(args)  # revealed: tuple
+```
+
+Using a keyword-varidic parameter:
+
+```py
+# TODO: should be `dict[str, Unknown]` (needs generics)
+lambda **kwargs: reveal_type(kwargs)  # revealed: dict
+```
+
+## Nested `lambda` expressions
+
+Here, a `lambda` expression is used as the default value for a parameter in another `lambda`
+expression.
+
+```py
+reveal_type(lambda a=lambda x, y: 0: 2)  # revealed: (a=(x, y) -> @Todo(lambda return type)) -> @Todo(lambda return type)
+```

crates/red_knot_python_semantic/resources/mdtest/expression/len.md

@@ -99,22 +99,28 @@ The returned value of `__len__` is implicitly and recursively converted to `int`
 from typing import Literal
 
 class Zero:
-    def __len__(self) -> Literal[0]: ...
+    def __len__(self) -> Literal[0]:
+        return 0
 
 class ZeroOrOne:
-    def __len__(self) -> Literal[0, 1]: ...
+    def __len__(self) -> Literal[0, 1]:
+        return 0
 
 class ZeroOrTrue:
-    def __len__(self) -> Literal[0, True]: ...
+    def __len__(self) -> Literal[0, True]:
+        return 0
 
 class OneOrFalse:
-    def __len__(self) -> Literal[1] | Literal[False]: ...
+    def __len__(self) -> Literal[1] | Literal[False]:
+        return 1
 
 class OneOrFoo:
-    def __len__(self) -> Literal[1, "foo"]: ...
+    def __len__(self) -> Literal[1, "foo"]:
+        return 1
 
 class ZeroOrStr:
-    def __len__(self) -> Literal[0] | str: ...
+    def __len__(self) -> Literal[0] | str:
+        return 0
 
 reveal_type(len(Zero()))  # revealed: Literal[0]
 reveal_type(len(ZeroOrOne()))  # revealed: Literal[0, 1]
@@ -134,10 +140,12 @@ reveal_type(len(ZeroOrStr()))  # revealed: int
 from typing import Literal
 
 class LiteralTrue:
-    def __len__(self) -> Literal[True]: ...
+    def __len__(self) -> Literal[True]:
+        return True
 
 class LiteralFalse:
-    def __len__(self) -> Literal[False]: ...
+    def __len__(self) -> Literal[False]:
+        return False
 
 reveal_type(len(LiteralTrue()))  # revealed: Literal[1]
 reveal_type(len(LiteralFalse()))  # revealed: Literal[0]
@@ -157,19 +165,24 @@ class SomeEnum(Enum):
     INT_2 = 3_2
 
 class Auto:
-    def __len__(self) -> Literal[SomeEnum.AUTO]: ...
+    def __len__(self) -> Literal[SomeEnum.AUTO]:
+        return SomeEnum.AUTO
 
 class Int:
-    def __len__(self) -> Literal[SomeEnum.INT]: ...
+    def __len__(self) -> Literal[SomeEnum.INT]:
+        return SomeEnum.INT
 
 class Str:
-    def __len__(self) -> Literal[SomeEnum.STR]: ...
+    def __len__(self) -> Literal[SomeEnum.STR]:
+        return SomeEnum.STR
 
 class Tuple:
-    def __len__(self) -> Literal[SomeEnum.TUPLE]: ...
+    def __len__(self) -> Literal[SomeEnum.TUPLE]:
+        return SomeEnum.TUPLE
 
 class IntUnion:
-    def __len__(self) -> Literal[SomeEnum.INT, SomeEnum.INT_2]: ...
+    def __len__(self) -> Literal[SomeEnum.INT, SomeEnum.INT_2]:
+        return SomeEnum.INT
 
 reveal_type(len(Auto()))  # revealed: int
 reveal_type(len(Int()))  # revealed: int
@@ -184,7 +197,8 @@ reveal_type(len(IntUnion()))  # revealed: int
 from typing import Literal
 
 class Negative:
-    def __len__(self) -> Literal[-1]: ...
+    def __len__(self) -> Literal[-1]:
+        return -1
 
 # TODO: Emit a diagnostic
 reveal_type(len(Negative()))  # revealed: int
@@ -196,10 +210,12 @@ reveal_type(len(Negative()))  # revealed: int
 from typing import Literal
 
 class SecondOptionalArgument:
-    def __len__(self, v: int = 0) -> Literal[0]: ...
+    def __len__(self, v: int = 0) -> Literal[0]:
+        return 0
 
 class SecondRequiredArgument:
-    def __len__(self, v: int) -> Literal[1]: ...
+    def __len__(self, v: int) -> Literal[1]:
+        return 1
 
 # TODO: Emit a diagnostic
 reveal_type(len(SecondOptionalArgument()))  # revealed: Literal[0]

crates/red_knot_python_semantic/resources/mdtest/function/return_type.md

@@ -0,0 +1,245 @@
+# Function return type
+
+When a function's return type is annotated, all return statements are checked to ensure that the
+type of the returned value is assignable to the annotated return type.
+
+## Basic examples
+
+A return value assignable to the annotated return type is valid.
+
+```py
+def f() -> int:
+    return 1
+```
+
+The type of the value obtained by calling a function is the annotated return type, not the inferred
+return type.
+
+```py
+reveal_type(f())  # revealed: int
+```
+
+A `raise` is equivalent to a return of `Never`, which is assignable to any annotated return type.
+
+```py
+def f() -> str:
+    raise ValueError()
+
+reveal_type(f())  # revealed: str
+```
+
+## Stub functions
+
+"Stub" function definitions (that is, function definitions with an empty body) are permissible in
+stub files, or in a few other locations: Protocol method definitions, abstract methods, and
+overloads. In this case the function body is considered to be omitted (thus no return type checking
+is performed on it), not assumed to implicitly return `None`.
+
+A stub function's "empty" body may contain only an optional docstring, followed (optionally) by an
+ellipsis (`...`) or `pass`.
+
+### In stub file
+
+```pyi
+def f() -> int: ...
+
+def f() -> int:
+    pass
+
+def f() -> int:
+    """Some docstring"""
+
+def f() -> int:
+    """Some docstring"""
+    ...
+```
+
+### In Protocol
+
+```py
+from typing import Protocol
+
+class Bar(Protocol):
+    # TODO: no error
+    # error: [invalid-return-type]
+    def f(self) -> int: ...
+```
+
+### In abstract method
+
+```py
+from abc import ABC, abstractmethod
+
+class Foo(ABC):
+    @abstractmethod
+    # TODO: no error
+    # error: [invalid-return-type]
+    def f(self) -> int: ...
+    @abstractmethod
+    # error: [invalid-return-type]
+    def g[T](self, x: T) -> T: ...
+```
+
+### In overload
+
+```py
+from typing import overload
+
+@overload
+def f(x: int) -> int: ...
+@overload
+def f(x: str) -> str: ...
+def f(x: int | str):
+    return x
+```
+
+## Conditional return type
+
+```py
+def f(cond: bool) -> int:
+    if cond:
+        return 1
+    else:
+        return 2
+
+def f(cond: bool) -> int | None:
+    if cond:
+        return 1
+    else:
+        return
+
+def f(cond: bool) -> int:
+    if cond:
+        return 1
+    else:
+        raise ValueError()
+
+def f(cond: bool) -> str | int:
+    if cond:
+        return "a"
+    else:
+        return 1
+```
+
+## Implicit return type
+
+```py
+def f(cond: bool) -> int | None:
+    if cond:
+        return 1
+
+# no implicit return
+def f() -> int:
+    if True:
+        return 1
+
+# no implicit return
+def f(cond: bool) -> int:
+    cond = True
+    if cond:
+        return 1
+
+def f(cond: bool) -> int:
+    if cond:
+        cond = True
+    else:
+        return 1
+    if cond:
+        return 2
+```
+
+## Invalid return type
+
+<!-- snapshot-diagnostics -->
+
+```py
+# error: [invalid-return-type]
+def f() -> int:
+    1
+
+def f() -> str:
+    # error: [invalid-return-type]
+    return 1
+
+def f() -> int:
+    # error: [invalid-return-type]
+    return
+
+from typing import TypeVar
+
+T = TypeVar("T")
+
+# TODO: `invalid-return-type` error should be emitted
+def m(x: T) -> T: ...
+```
+
+## Invalid return type in stub file
+
+<!-- snapshot-diagnostics -->
+
+```pyi
+def f() -> int:
+    # error: [invalid-return-type]
+    return ...
+
+# error: [invalid-return-type]
+def foo() -> int:
+    print("...")
+    ...
+
+# error: [invalid-return-type]
+def foo() -> int:
+    f"""{foo} is a function that ..."""
+    ...
+```
+
+## Invalid conditional return type
+
+<!-- snapshot-diagnostics -->
+
+```py
+def f(cond: bool) -> str:
+    if cond:
+        return "a"
+    else:
+        # error: [invalid-return-type]
+        return 1
+
+def f(cond: bool) -> str:
+    if cond:
+        # error: [invalid-return-type]
+        return 1
+    else:
+        # error: [invalid-return-type]
+        return 2
+```
+
+## Invalid implicit return type
+
+<!-- snapshot-diagnostics -->
+
+```py
+def f() -> None:
+    if False:
+        # error: [invalid-return-type]
+        return 1
+
+# error: [invalid-return-type]
+def f(cond: bool) -> int:
+    if cond:
+        return 1
+
+# error: [invalid-return-type]
+def f(cond: bool) -> int:
+    if cond:
+        raise ValueError()
+
+# error: [invalid-return-type]
+def f(cond: bool) -> int:
+    if cond:
+        cond = False
+    else:
+        return 1
+    if cond:
+        return 2
+```

crates/red_knot_python_semantic/resources/mdtest/generics.md

@@ -1,81 +0,0 @@
-# PEP 695 Generics
-
-## Class Declarations
-
-Basic PEP 695 generics
-
-```py
-class MyBox[T]:
-    data: T
-    box_model_number = 695
-
-    def __init__(self, data: T):
-        self.data = data
-
-box: MyBox[int] = MyBox(5)
-
-# TODO should emit a diagnostic here (str is not assignable to int)
-wrong_innards: MyBox[int] = MyBox("five")
-
-# TODO reveal int, do not leak the typevar
-reveal_type(box.data)  # revealed: T
-
-reveal_type(MyBox.box_model_number)  # revealed: Unknown | Literal[695]
-```
-
-## Subclassing
-
-```py
-class MyBox[T]:
-    data: T
-
-    def __init__(self, data: T):
-        self.data = data
-
-# TODO not error on the subscripting
-# error: [non-subscriptable]
-class MySecureBox[T](MyBox[T]): ...
-
-secure_box: MySecureBox[int] = MySecureBox(5)
-reveal_type(secure_box)  # revealed: MySecureBox
-# TODO reveal int
-# The @Todo(…) is misleading here. We currently treat `MyBox[T]` as a dynamic base class because we
-# don't understand generics and therefore infer `Unknown` for the `MyBox[T]` base of `MySecureBox[T]`.
-reveal_type(secure_box.data)  # revealed: @Todo(instance attribute on class with dynamic base)
-```
-
-## Cyclical class definition
-
-In type stubs, classes can reference themselves in their base class definitions. For example, in
-`typeshed`, we have `class str(Sequence[str]): ...`.
-
-This should hold true even with generics at play.
-
-```pyi
-class Seq[T]: ...
-
-# TODO not error on the subscripting
-class S[T](Seq[S]): ...  # error: [non-subscriptable]
-
-reveal_type(S)  # revealed: Literal[S]
-```
-
-## Type params
-
-A PEP695 type variable defines a value of type `typing.TypeVar`.
-
-```py
-def f[T]():
-    reveal_type(T)  # revealed: T
-    reveal_type(T.__name__)  # revealed: Literal["T"]
-```
-
-## Minimum two constraints
-
-A typevar with less than two constraints emits a diagnostic:
-
-```py
-# error: [invalid-type-variable-constraints] "TypeVar must have at least two constrained types"
-def f[T: (int,)]():
-    pass
-```

crates/red_knot_python_semantic/resources/mdtest/generics/classes.md

@@ -0,0 +1,200 @@
+# Generic classes
+
+## PEP 695 syntax
+
+TODO: Add a `red_knot_extension` function that asserts whether a function or class is generic.
+
+This is a generic class defined using PEP 695 syntax:
+
+```py
+class C[T]: ...
+```
+
+A class that inherits from a generic class, and fills its type parameters with typevars, is generic:
+
+```py
+# TODO: no error
+# error: [non-subscriptable]
+class D[U](C[U]): ...
+```
+
+A class that inherits from a generic class, but fills its type parameters with concrete types, is
+_not_ generic:
+
+```py
+# TODO: no error
+# error: [non-subscriptable]
+class E(C[int]): ...
+```
+
+A class that inherits from a generic class, and doesn't fill its type parameters at all, implicitly
+uses the default value for the typevar. In this case, that default type is `Unknown`, so `F`
+inherits from `C[Unknown]` and is not itself generic.
+
+```py
+class F(C): ...
+```
+
+## Legacy syntax
+
+This is a generic class defined using the legacy syntax:
+
+```py
+from typing import Generic, TypeVar
+
+T = TypeVar("T")
+
+# TODO: no error
+# error: [invalid-base]
+class C(Generic[T]): ...
+```
+
+A class that inherits from a generic class, and fills its type parameters with typevars, is generic.
+
+```py
+class D(C[T]): ...
+```
+
+(Examples `E` and `F` from above do not have analogues in the legacy syntax.)
+
+## Inferring generic class parameters
+
+The type parameter can be specified explicitly:
+
+```py
+class C[T]:
+    x: T
+
+# TODO: no error
+# TODO: revealed: C[int]
+# error: [non-subscriptable]
+reveal_type(C[int]())  # revealed: C
+```
+
+We can infer the type parameter from a type context:
+
+```py
+c: C[int] = C()
+# TODO: revealed: C[int]
+reveal_type(c)  # revealed: C
+```
+
+The typevars of a fully specialized generic class should no longer be visible:
+
+```py
+# TODO: revealed: int
+reveal_type(c.x)  # revealed: T
+```
+
+If the type parameter is not specified explicitly, and there are no constraints that let us infer a
+specific type, we infer the typevar's default type:
+
+```py
+class D[T = int]: ...
+
+# TODO: revealed: D[int]
+reveal_type(D())  # revealed: D
+```
+
+If a typevar does not provide a default, we use `Unknown`:
+
+```py
+# TODO: revealed: C[Unknown]
+reveal_type(C())  # revealed: C
+```
+
+If the type of a constructor parameter is a class typevar, we can use that to infer the type
+parameter:
+
+```py
+class E[T]:
+    def __init__(self, x: T) -> None: ...
+
+# TODO: revealed: E[int] or E[Literal[1]]
+reveal_type(E(1))  # revealed: E
+```
+
+The types inferred from a type context and from a constructor parameter must be consistent with each
+other:
+
+```py
+# TODO: error
+wrong_innards: E[int] = E("five")
+```
+
+## Generic subclass
+
+When a generic subclass fills its superclass's type parameter with one of its own, the actual types
+propagate through:
+
+```py
+class Base[T]:
+    x: T | None = None
+
+# TODO: no error
+# error: [non-subscriptable]
+class Sub[U](Base[U]): ...
+
+# TODO: no error
+# TODO: revealed: int | None
+# error: [non-subscriptable]
+reveal_type(Base[int].x)  # revealed: T | None
+# TODO: revealed: int | None
+# error: [non-subscriptable]
+reveal_type(Sub[int].x)  # revealed: T | None
+```
+
+## Cyclic class definition
+
+A class can use itself as the type parameter of one of its superclasses. (This is also known as the
+[curiously recurring template pattern][crtp] or [F-bounded quantification][f-bound].)
+
+Here, `Sub` is not a generic class, since it fills its superclass's type parameter (with itself).
+
+`stub.pyi`:
+
+```pyi
+class Base[T]: ...
+# TODO: no error
+# error: [non-subscriptable]
+class Sub(Base[Sub]): ...
+
+reveal_type(Sub)  # revealed: Literal[Sub]
+```
+
+A similar case can work in a non-stub file, if forward references are stringified:
+
+`string_annotation.py`:
+
+```py
+class Base[T]: ...
+
+# TODO: no error
+# error: [non-subscriptable]
+class Sub(Base["Sub"]): ...
+
+reveal_type(Sub)  # revealed: Literal[Sub]
+```
+
+In a non-stub file, without stringified forward references, this raises a `NameError`:
+
+`bare_annotation.py`:
+
+```py
+class Base[T]: ...
+
+# TODO: error: [unresolved-reference]
+# error: [non-subscriptable]
+class Sub(Base[Sub]): ...
+```
+
+## Another cyclic case
+
+```pyi
+# TODO no error (generics)
+# error: [invalid-base]
+class Derived[T](list[Derived[T]]): ...
+```
+
+[crtp]: https://en.wikipedia.org/wiki/Curiously_recurring_template_pattern
+[f-bound]: https://en.wikipedia.org/wiki/Bounded_quantification#F-bounded_quantification

crates/red_knot_python_semantic/resources/mdtest/generics/functions.md

@@ -0,0 +1,246 @@
+# Generic functions
+
+## Typevar must be used at least twice
+
+If you're only using a typevar for a single parameter, you don't need the typevar — just use
+`object` (or the typevar's upper bound):
+
+```py
+# TODO: error, should be (x: object)
+def typevar_not_needed[T](x: T) -> None:
+    pass
+
+# TODO: error, should be (x: int)
+def bounded_typevar_not_needed[T: int](x: T) -> None:
+    pass
+```
+
+Typevars are only needed if you use them more than once. For instance, to specify that two
+parameters must both have the same type:
+
+```py
+def two_params[T](x: T, y: T) -> T:
+    return x
+```
+
+or to specify that a return value is the same as a parameter:
+
+```py
+def return_value[T](x: T) -> T:
+    return x
+```
+
+Each typevar must also appear _somewhere_ in the parameter list:
+
+```py
+def absurd[T]() -> T:
+    # There's no way to construct a T!
+    raise ValueError("absurd")
+```
+
+## Inferring generic function parameter types
+
+If the type of a generic function parameter is a typevar, then we can infer what type that typevar
+is bound to at each call site.
+
+TODO: Note that some of the TODO revealed types have two options, since we haven't decided yet
+whether we want to infer a more specific `Literal` type where possible, or use heuristics to weaken
+the inferred type to e.g. `int`.
+
+```py
+def f[T](x: T) -> T:
+    return x
+
+# TODO: no error
+# TODO: revealed: int or Literal[1]
+# error: [invalid-argument-type]
+reveal_type(f(1))  # revealed: T
+
+# TODO: no error
+# TODO: revealed: float
+# error: [invalid-argument-type]
+reveal_type(f(1.0))  # revealed: T
+
+# TODO: no error
+# TODO: revealed: bool or Literal[true]
+# error: [invalid-argument-type]
+reveal_type(f(True))  # revealed: T
+
+# TODO: no error
+# TODO: revealed: str or Literal["string"]
+# error: [invalid-argument-type]
+reveal_type(f("string"))  # revealed: T
+```
+
+## Inferring “deep” generic parameter types
+
+The matching up of call arguments and discovery of constraints on typevars can be a recursive
+process for arbitrarily-nested generic types in parameters.
+
+```py
+def f[T](x: list[T]) -> T:
+    return x[0]
+
+# TODO: revealed: float
+reveal_type(f([1.0, 2.0]))  # revealed: T
+```
+
+## Typevar constraints
+
+If a type parameter has an upper bound, that upper bound constrains which types can be used for that
+typevar. This effectively adds the upper bound as an intersection to every appearance of the typevar
+in the function.
+
+```py
+def good_param[T: int](x: T) -> None:
+    # TODO: revealed: T & int
+    reveal_type(x)  # revealed: T
+```
+
+If the function is annotated as returning the typevar, this means that the upper bound is _not_
+assignable to that typevar, since return types are contravariant. In `bad`, we can infer that
+`x + 1` has type `int`. But `T` might be instantiated with a narrower type than `int`, and so the
+return value is not guaranteed to be compatible for all `T: int`.
+
+```py
+def good_return[T: int](x: T) -> T:
+    return x
+
+def bad_return[T: int](x: T) -> T:
+    # TODO: error: int is not assignable to T
+    # error: [unsupported-operator] "Operator `+` is unsupported between objects of type `T` and `Literal[1]`"
+    return x + 1
+```
+
+## All occurrences of the same typevar have the same type
+
+If a typevar appears multiple times in a function signature, all occurrences have the same type.
+
+```py
+def different_types[T, S](cond: bool, t: T, s: S) -> T:
+    if cond:
+        return t
+    else:
+        # error: [invalid-return-type] "Object of type `S` is not assignable to return type `T`"
+        return s
+
+def same_types[T](cond: bool, t1: T, t2: T) -> T:
+    if cond:
+        return t1
+    else:
+        return t2
+```
+
+## All occurrences of the same constrained typevar have the same type
+
+The above is true even when the typevars are constrained. Here, both `int` and `str` have `__add__`
+methods that are compatible with the return type, so the `return` expression is always well-typed:
+
+```py
+def same_constrained_types[T: (int, str)](t1: T, t2: T) -> T:
+    # TODO: no error
+    # error: [unsupported-operator] "Operator `+` is unsupported between objects of type `T` and `T`"
+    return t1 + t2
+```
+
+This is _not_ the same as a union type, because of this additional constraint that the two
+occurrences have the same type. In `unions_are_different`, `t1` and `t2` might have different types,
+and an `int` and a `str` cannot be added together:
+
+```py
+def unions_are_different(t1: int | str, t2: int | str) -> int | str:
+    # error: [unsupported-operator] "Operator `+` is unsupported between objects of type `int | str` and `int | str`"
+    return t1 + t2
+```
+
+## Typevar inference is a unification problem
+
+When inferring typevar assignments in a generic function call, we cannot simply solve constraints
+eagerly for each parameter in turn. We must solve a unification problem involving all of the
+parameters simultaneously.
+
+```py
+def two_params[T](x: T, y: T) -> T:
+    return x
+
+# TODO: no error
+# TODO: revealed: str
+# error: [invalid-argument-type]
+# error: [invalid-argument-type]
+reveal_type(two_params("a", "b"))  # revealed: T
+
+# TODO: no error
+# TODO: revealed: str | int
+# error: [invalid-argument-type]
+# error: [invalid-argument-type]
+reveal_type(two_params("a", 1))  # revealed: T
+```
+
+```py
+def param_with_union[T](x: T | int, y: T) -> T:
+    return y
+
+# TODO: no error
+# TODO: revealed: str
+# error: [invalid-argument-type]
+reveal_type(param_with_union(1, "a"))  # revealed: T
+
+# TODO: no error
+# TODO: revealed: str
+# error: [invalid-argument-type]
+# error: [invalid-argument-type]
+reveal_type(param_with_union("a", "a"))  # revealed: T
+
+# TODO: no error
+# TODO: revealed: int
+# error: [invalid-argument-type]
+reveal_type(param_with_union(1, 1))  # revealed: T
+
+# TODO: no error
+# TODO: revealed: str | int
+# error: [invalid-argument-type]
+# error: [invalid-argument-type]
+reveal_type(param_with_union("a", 1))  # revealed: T
+```
+
+```py
+def tuple_param[T, S](x: T | S, y: tuple[T, S]) -> tuple[T, S]:
+    return y
+
+# TODO: no error
+# TODO: revealed: tuple[str, int]
+# error: [invalid-argument-type]
+# error: [invalid-argument-type]
+reveal_type(tuple_param("a", ("a", 1)))  # revealed: tuple[T, S]
+
+# TODO: no error
+# TODO: revealed: tuple[str, int]
+# error: [invalid-argument-type]
+# error: [invalid-argument-type]
+reveal_type(tuple_param(1, ("a", 1)))  # revealed: tuple[T, S]
+```
+
+## Inferring nested generic function calls
+
+We can infer type assignments in nested calls to multiple generic functions. If they use the same
+type variable, we do not confuse the two; `T@f` and `T@g` have separate types in each example below.
+
+```py
+def f[T](x: T) -> tuple[T, int]:
+    return (x, 1)
+
+def g[T](x: T) -> T | None:
+    return x
+
+# TODO: no error
+# TODO: revealed: tuple[str | None, int]
+# error: [invalid-argument-type]
+# error: [invalid-argument-type]
+reveal_type(f(g("a")))  # revealed: tuple[T, int]
+
+# TODO: no error
+# TODO: revealed: tuple[str, int] | None
+# error: [invalid-argument-type]
+# error: [invalid-argument-type]
+reveal_type(g(f("a")))  # revealed: T | None
+```

crates/red_knot_python_semantic/resources/mdtest/generics/legacy.md

@@ -0,0 +1,72 @@
+# Legacy type variables
+
+The tests in this file focus on how type variables are defined using the legacy notation. Most
+_uses_ of type variables are tested in other files in this directory; we do not duplicate every test
+for both type variable syntaxes.
+
+Unless otherwise specified, all quotations come from the [Generics] section of the typing spec.
+
+## Type variables
+
+### Defining legacy type variables
+
+> Generics can be parameterized by using a factory available in `typing` called `TypeVar`.
+
+This was the only way to create type variables prior to PEP 695/Python 3.12. It is still available
+in newer Python releases.
+
+```py
+from typing import TypeVar
+
+T = TypeVar("T")
+```
+
+### Directly assigned to a variable
+
+> A `TypeVar()` expression must always directly be assigned to a variable (it should not be used as
+> part of a larger expression).
+
+```py
+from typing import TypeVar
+
+# TODO: error
+TestList = list[TypeVar("W")]
+```
+
+### `TypeVar` parameter must match variable name
+
+> The argument to `TypeVar()` must be a string equal to the variable name to which it is assigned.
+
+```py
+from typing import TypeVar
+
+# TODO: error
+T = TypeVar("Q")
+```
+
+### No redefinition
+
+> Type variables must not be redefined.
+
+```py
+from typing import TypeVar
+
+T = TypeVar("T")
+
+# TODO: error
+T = TypeVar("T")
+```
+
+### Cannot have only one constraint
+
+> `TypeVar` supports constraining parametric types to a fixed set of possible types...There should
+> be at least two constraints, if any; specifying a single constraint is disallowed.
+
+```py
+from typing import TypeVar
+
+# TODO: error: [invalid-type-variable-constraints]
+T = TypeVar("T", int)
+```
+
+[generics]: https://typing.readthedocs.io/en/latest/spec/generics.html

crates/red_knot_python_semantic/resources/mdtest/generics/pep695.md

@@ -0,0 +1,51 @@
+# PEP 695 Generics
+
+[PEP 695] and Python 3.12 introduced new, more ergonomic syntax for type variables.
+
+## Type variables
+
+### Defining PEP 695 type variables
+
+PEP 695 introduces a new syntax for defining type variables. The resulting type variables are
+instances of `typing.TypeVar`, just like legacy type variables.
+
+```py
+def f[T]():
+    reveal_type(type(T))  # revealed: Literal[TypeVar]
+    reveal_type(T)  # revealed: T
+    reveal_type(T.__name__)  # revealed: Literal["T"]
+```
+
+### Cannot have only one constraint
+
+> `TypeVar` supports constraining parametric types to a fixed set of possible types...There should
+> be at least two constraints, if any; specifying a single constraint is disallowed.
+
+```py
+# error: [invalid-type-variable-constraints] "TypeVar must have at least two constrained types"
+def f[T: (int,)]():
+    pass
+```
+
+## Invalid uses
+
+Note that many of the invalid uses of legacy typevars do not apply to PEP 695 typevars, since the
+PEP 695 syntax is only allowed places where typevars are allowed.
+
+## Displaying typevars
+
+We use a suffix when displaying the typevars of a generic function or class. This helps distinguish
+different uses of the same typevar.
+
+```py
+def f[T](x: T, y: T) -> None:
+    # TODO: revealed: T@f
+    reveal_type(x)  # revealed: T
+
+class C[T]:
+    def m(self, x: T) -> None:
+        # TODO: revealed: T@c
+        reveal_type(x)  # revealed: T
+```
+
+[pep 695]: https://peps.python.org/pep-0695/

crates/red_knot_python_semantic/resources/mdtest/generics/scoping.md

@@ -0,0 +1,270 @@
+# Scoping rules for type variables
+
+Most of these tests come from the [Scoping rules for type variables][scoping] section of the typing
+spec.
+
+## Typevar used outside of generic function or class
+
+Typevars may only be used in generic function or class definitions.
+
+```py
+from typing import TypeVar
+
+T = TypeVar("T")
+
+# TODO: error
+x: T
+
+class C:
+    # TODO: error
+    x: T
+
+def f() -> None:
+    # TODO: error
+    x: T
+```
+
+## Legacy typevar used multiple times
+
+> A type variable used in a generic function could be inferred to represent different types in the
+> same code block.
+
+This only applies to typevars defined using the legacy syntax, since the PEP 695 syntax creates a
+new distinct typevar for each occurrence.
+
+```py
+from typing import TypeVar
+
+T = TypeVar("T")
+
+def f1(x: T) -> T:
+    return x
+
+def f2(x: T) -> T:
+    return x
+
+f1(1)
+f2("a")
+```
+
+## Typevar inferred multiple times
+
+> A type variable used in a generic function could be inferred to represent different types in the
+> same code block.
+
+This also applies to a single generic function being used multiple times, instantiating the typevar
+to a different type each time.
+
+```py
+def f[T](x: T) -> T:
+    return x
+
+# TODO: no error
+# TODO: revealed: int or Literal[1]
+# error: [invalid-argument-type]
+reveal_type(f(1))  # revealed: T
+# TODO: no error
+# TODO: revealed: str or Literal["a"]
+# error: [invalid-argument-type]
+reveal_type(f("a"))  # revealed: T
+```
+
+## Methods can mention class typevars
+
+> A type variable used in a method of a generic class that coincides with one of the variables that
+> parameterize this class is always bound to that variable.
+
+```py
+class C[T]:
+    def m1(self, x: T) -> T:
+        return x
+
+    def m2(self, x: T) -> T:
+        return x
+
+c: C[int] = C()
+# TODO: no error
+# error: [invalid-argument-type]
+c.m1(1)
+# TODO: no error
+# error: [invalid-argument-type]
+c.m2(1)
+# TODO: expected type `int`
+# error: [invalid-argument-type] "Object of type `Literal["string"]` cannot be assigned to parameter 2 (`x`) of bound method `m2`; expected type `T`"
+c.m2("string")
+```
+
+## Methods can mention other typevars
+
+> A type variable used in a method that does not match any of the variables that parameterize the
+> class makes this method a generic function in that variable.
+
+```py
+from typing import TypeVar, Generic
+
+T = TypeVar("T")
+S = TypeVar("S")
+
+# TODO: no error
+# error: [invalid-base]
+class Legacy(Generic[T]):
+    def m(self, x: T, y: S) -> S:
+        return y
+
+legacy: Legacy[int] = Legacy()
+# TODO: revealed: str
+reveal_type(legacy.m(1, "string"))  # revealed: @Todo(Invalid or unsupported `Instance` in `Type::to_type_expression`)
+```
+
+With PEP 695 syntax, it is clearer that the method uses a separate typevar:
+
+```py
+class C[T]:
+    def m[S](self, x: T, y: S) -> S:
+        return y
+
+c: C[int] = C()
+# TODO: no errors
+# TODO: revealed: str
+# error: [invalid-argument-type]
+# error: [invalid-argument-type]
+reveal_type(c.m(1, "string"))  # revealed: S
+```
+
+## Unbound typevars
+
+> Unbound type variables should not appear in the bodies of generic functions, or in the class
+> bodies apart from method definitions.
+
+This is true with the legacy syntax:
+
+```py
+from typing import TypeVar, Generic
+
+T = TypeVar("T")
+S = TypeVar("S")
+
+def f(x: T) -> None:
+    x: list[T] = []
+    # TODO: error
+    y: list[S] = []
+
+# TODO: no error
+# error: [invalid-base]
+class C(Generic[T]):
+    # TODO: error
+    x: list[S] = []
+
+    # This is not an error, as shown in the previous test
+    def m(self, x: S) -> S:
+        return x
+```
+
+This is true with PEP 695 syntax, as well, though we must use the legacy syntax to define the
+unbound typevars:
+
+`pep695.py`:
+
+```py
+from typing import TypeVar
+
+S = TypeVar("S")
+
+def f[T](x: T) -> None:
+    x: list[T] = []
+    # TODO: error
+    y: list[S] = []
+
+class C[T]:
+    # TODO: error
+    x: list[S] = []
+
+    def m1(self, x: S) -> S:
+        return x
+
+    def m2[S](self, x: S) -> S:
+        return x
+```
+
+## Nested formal typevars must be distinct
+
+Generic functions and classes can be nested in each other, but it is an error for the same typevar
+to be used in nested generic definitions.
+
+Note that the typing spec only mentions two specific versions of this rule:
+
+> A generic class definition that appears inside a generic function should not use type variables
+> that parameterize the generic function.
+
+and
+
+> A generic class nested in another generic class cannot use the same type variables.
+
+We assume that the more general form holds.
+
+### Generic function within generic function
+
+```py
+def f[T](x: T, y: T) -> None:
+    def ok[S](a: S, b: S) -> None: ...
+
+    # TODO: error
+    def bad[T](a: T, b: T) -> None: ...
+```
+
+### Generic method within generic class
+
+```py
+class C[T]:
+    def ok[S](self, a: S, b: S) -> None: ...
+
+    # TODO: error
+    def bad[T](self, a: T, b: T) -> None: ...
+```
+
+### Generic class within generic function
+
+```py
+from typing import Iterable
+
+def f[T](x: T, y: T) -> None:
+    class Ok[S]: ...
+    # TODO: error for reuse of typevar
+    class Bad1[T]: ...
+    # TODO: no non-subscriptable error, error for reuse of typevar
+    # error: [non-subscriptable]
+    class Bad2(Iterable[T]): ...
+```
+
+### Generic class within generic class
+
+```py
+from typing import Iterable
+
+class C[T]:
+    class Ok1[S]: ...
+    # TODO: error for reuse of typevar
+    class Bad1[T]: ...
+    # TODO: no non-subscriptable error, error for reuse of typevar
+    # error: [non-subscriptable]
+    class Bad2(Iterable[T]): ...
+```
+
+## Class scopes do not cover inner scopes
+
+Just like regular symbols, the typevars of a generic class are only available in that class's scope,
+and are not available in nested scopes.
+
+```py
+class C[T]:
+    ok1: list[T] = []
+
+    class Bad:
+        # TODO: error
+        bad: list[T] = []
+
+    class Inner[S]: ...
+    ok2: Inner[T]
+```
+
+[scoping]: https://typing.readthedocs.io/en/latest/spec/generics.html#scoping-rules-for-type-variables

crates/red_knot_python_semantic/resources/mdtest/import/case_sensitive.md

@@ -0,0 +1,131 @@
+# Case Sensitive Imports
+
+```toml
+# TODO: This test should use the real file system instead of the memory file system.
+# but we can't change the file system yet because the tests would then start failing for 
+# case-insensitive file systems.
+#system = "os"
+```
+
+Python's import system is case-sensitive even on case-insensitive file system. This means, importing
+a module `a` should fail if the file in the search paths is named `A.py`. See
+[PEP 235](https://peps.python.org/pep-0235/).
+
+## Correct casing
+
+Importing a module where the name matches the file name's casing should succeed.
+
+`a.py`:
+
+```py
+class Foo:
+    x: int = 1
+```
+
+```python
+from a import Foo
+
+reveal_type(Foo().x)  # revealed: int
+```
+
+## Incorrect casing
+
+Importing a module where the name does not match the file name's casing should fail.
+
+`A.py`:
+
+```py
+class Foo:
+    x: int = 1
+```
+
+```python
+# error: [unresolved-import]
+from a import Foo
+```
+
+## Multiple search paths with different cased modules
+
+The resolved module is the first matching the file name's casing but Python falls back to later
+search paths if the file name's casing does not match.
+
+```toml
+[environment]
+extra-paths = ["/search-1", "/search-2"]
+```
+
+`/search-1/A.py`:
+
+```py
+class Foo:
+    x: int = 1
+```
+
+`/search-2/a.py`:
+
+```py
+class Bar:
+    x: str = "test"
+```
+
+```python
+from A import Foo
+from a import Bar
+
+reveal_type(Foo().x)  # revealed: int
+reveal_type(Bar().x)  # revealed: str
+```
+
+## Intermediate segments
+
+`db/__init__.py`:
+
+```py
+```
+
+`db/a.py`:
+
+```py
+class Foo:
+    x: int = 1
+```
+
+`correctly_cased.py`:
+
+```python
+from db.a import Foo
+
+reveal_type(Foo().x)  # revealed: int
+```
+
+Imports where some segments are incorrectly cased should fail.
+
+`incorrectly_cased.py`:
+
+```python
+# error: [unresolved-import]
+from DB.a import Foo
+
+# error: [unresolved-import]
+from DB.A import Foo
+
+# error: [unresolved-import]
+from db.A import Foo
+```
+
+## Incorrect extension casing
+
+The extension of imported python modules must be `.py` or `.pyi` but not `.PY` or `Py` or any
+variant where some characters are uppercase.
+
+`a.PY`:
+
+```py
+class Foo:
+    x: int = 1
+```
+
+```python
+# error: [unresolved-import]
+from a import Foo
+```

crates/red_knot_python_semantic/resources/mdtest/invalid_syntax.md

@@ -91,3 +91,16 @@ match while:
 for x in foo.pass:
     pass
 ```
+
+## Invalid annotation
+
+### `typing.Callable`
+
+```py
+from typing import Callable
+
+# error: [invalid-syntax] "Expected index or slice expression"
+# error: [invalid-type-form] "Special form `typing.Callable` expected exactly two arguments (parameter types and return type)"
+def _(c: Callable[]):
+    reveal_type(c)  # revealed: (...) -> Unknown
+```

crates/red_knot_python_semantic/resources/mdtest/known_constants.md

@@ -26,23 +26,6 @@ from typing import TYPE_CHECKING as TC
 reveal_type(TC)  # revealed: Literal[True]
 ```
 
-### Must originate from `typing`
-
-Make sure we only use our special handling for `typing.TYPE_CHECKING` and not for other constants
-with the same name:
-
-`constants.py`:
-
-```py
-TYPE_CHECKING: bool = False
-```
-
-```py
-from constants import TYPE_CHECKING
-
-reveal_type(TYPE_CHECKING)  # revealed: bool
-```
-
 ### `typing_extensions` re-export
 
 This should behave in the same way as `typing.TYPE_CHECKING`:
@@ -52,3 +35,117 @@ from typing_extensions import TYPE_CHECKING
 
 reveal_type(TYPE_CHECKING)  # revealed: Literal[True]
 ```
+
+## User-defined `TYPE_CHECKING`
+
+If we set `TYPE_CHECKING = False` directly instead of importing it from the `typing` module, it will
+still be treated as `True` during type checking. This behavior is for compatibility with other major
+type checkers, e.g. mypy and pyright.
+
+### With no type annotation
+
+```py
+TYPE_CHECKING = False
+reveal_type(TYPE_CHECKING)  # revealed: Literal[True]
+if TYPE_CHECKING:
+    type_checking = True
+if not TYPE_CHECKING:
+    runtime = True
+
+# type_checking is treated as unconditionally assigned.
+reveal_type(type_checking)  # revealed: Literal[True]
+# error: [unresolved-reference]
+reveal_type(runtime)  # revealed: Unknown
+```
+
+### With a type annotation
+
+We can also define `TYPE_CHECKING` with a type annotation. The type must be one to which `bool` can
+be assigned. Even in this case, the type of `TYPE_CHECKING` is still inferred to be `Literal[True]`.
+
+```py
+TYPE_CHECKING: bool = False
+reveal_type(TYPE_CHECKING)  # revealed: Literal[True]
+if TYPE_CHECKING:
+    type_checking = True
+if not TYPE_CHECKING:
+    runtime = True
+
+reveal_type(type_checking)  # revealed: Literal[True]
+# error: [unresolved-reference]
+reveal_type(runtime)  # revealed: Unknown
+```
+
+### Importing user-defined `TYPE_CHECKING`
+
+`constants.py`:
+
+```py
+TYPE_CHECKING = False
+```
+
+`stub.pyi`:
+
+```pyi
+TYPE_CHECKING: bool
+# or
+TYPE_CHECKING: bool = ...
+```
+
+```py
+from constants import TYPE_CHECKING
+
+reveal_type(TYPE_CHECKING)  # revealed: Literal[True]
+
+from stub import TYPE_CHECKING
+
+reveal_type(TYPE_CHECKING)  # revealed: Literal[True]
+```
+
+### Invalid assignment to `TYPE_CHECKING`
+
+Only `False` can be assigned to `TYPE_CHECKING`; any assignment other than `False` will result in an
+error. A type annotation to which `bool` is not assignable is also an error.
+
+```py
+from typing import Literal
+
+# error: [invalid-type-checking-constant]
+TYPE_CHECKING = True
+
+# error: [invalid-type-checking-constant]
+TYPE_CHECKING: bool = True
+
+# error: [invalid-type-checking-constant]
+TYPE_CHECKING: int = 1
+
+# error: [invalid-type-checking-constant]
+TYPE_CHECKING: str = "str"
+
+# error: [invalid-type-checking-constant]
+TYPE_CHECKING: str = False
+
+# error: [invalid-type-checking-constant]
+TYPE_CHECKING: Literal[False] = False
+
+# error: [invalid-type-checking-constant]
+TYPE_CHECKING: Literal[True] = False
+```
+
+The same rules apply in a stub file:
+
+```pyi
+from typing import Literal
+
+# error: [invalid-type-checking-constant]
+TYPE_CHECKING: str
+
+# error: [invalid-type-checking-constant]
+TYPE_CHECKING: str = False
+
+# error: [invalid-type-checking-constant]
+TYPE_CHECKING: Literal[False] = ...
+
+# error: [invalid-type-checking-constant]
+TYPE_CHECKING: object = "str"
+```

crates/red_knot_python_semantic/resources/mdtest/loops/for.md

@@ -105,7 +105,11 @@ reveal_type(x)
 
 ## With non-callable iterator
 
+<!-- snapshot-diagnostics -->
+
 ```py
+from typing_extensions import reveal_type
+
 def _(flag: bool):
     class NotIterable:
         if flag:
@@ -113,7 +117,8 @@ def _(flag: bool):
         else:
             __iter__: None = None
 
-    for x in NotIterable():  # error: "Object of type `NotIterable` is not iterable"
+    # error: [not-iterable]
+    for x in NotIterable():
         pass
 
     # revealed: Unknown
@@ -123,21 +128,25 @@ def _(flag: bool):
 
 ## Invalid iterable
 
+<!-- snapshot-diagnostics -->
+
 ```py
 nonsense = 123
-for x in nonsense:  # error: "Object of type `Literal[123]` is not iterable"
+for x in nonsense:  # error: [not-iterable]
     pass
 ```
 
 ## New over old style iteration protocol
 
+<!-- snapshot-diagnostics -->
+
 ```py
 class NotIterable:
     def __getitem__(self, key: int) -> int:
         return 42
     __iter__: None = None
 
-for x in NotIterable():  # error: "Object of type `NotIterable` is not iterable"
+for x in NotIterable():  # error: [not-iterable]
     pass
 ```
 
@@ -221,7 +230,11 @@ def _(flag: bool):
 
 ## Union type as iterable where one union element has no `__iter__` method
 
+<!-- snapshot-diagnostics -->
+
 ```py
+from typing_extensions import reveal_type
+
 class TestIter:
     def __next__(self) -> int:
         return 42
@@ -231,14 +244,18 @@ class Test:
         return TestIter()
 
 def _(flag: bool):
-    # error: [not-iterable] "Object of type `Test | Literal[42]` is not iterable because its `__iter__` method is possibly unbound"
+    # error: [not-iterable]
     for x in Test() if flag else 42:
         reveal_type(x)  # revealed: int
 ```
 
 ## Union type as iterable where one union element has invalid `__iter__` method
 
+<!-- snapshot-diagnostics -->
+
 ```py
+from typing_extensions import reveal_type
+
 class TestIter:
     def __next__(self) -> int:
         return 42
@@ -253,7 +270,7 @@ class Test2:
 
 def _(flag: bool):
     # TODO: Improve error message to state which union variant isn't iterable (https://github.com/astral-sh/ruff/issues/13989)
-    # error: "Object of type `Test | Test2` is not iterable"
+    # error: [not-iterable]
     for x in Test() if flag else Test2():
         reveal_type(x)  # revealed: int
 ```
@@ -269,7 +286,464 @@ class Test:
     def __iter__(self) -> TestIter | int:
         return TestIter()
 
-# error: [not-iterable] "Object of type `Test` is not iterable"
+# error: [not-iterable] "Object of type `Test` may not be iterable because its `__iter__` method returns an object of type `TestIter | int`, which may not have a `__next__` method"
 for x in Test():
     reveal_type(x)  # revealed: int
 ```
+
+## Possibly-not-callable `__iter__` method
+
+```py
+def _(flag: bool):
+    class Iterator:
+        def __next__(self) -> int:
+            return 42
+
+    class CustomCallable:
+        if flag:
+            def __call__(self, *args, **kwargs) -> Iterator:
+                return Iterator()
+        else:
+            __call__: None = None
+
+    class Iterable1:
+        __iter__: CustomCallable = CustomCallable()
+
+    class Iterable2:
+        if flag:
+            def __iter__(self) -> Iterator:
+                return Iterator()
+        else:
+            __iter__: None = None
+
+    # error: [not-iterable] "Object of type `Iterable1` may not be iterable because its `__iter__` attribute (with type `CustomCallable`) may not be callable"
+    for x in Iterable1():
+        # TODO... `int` might be ideal here?
+        reveal_type(x)  # revealed: int | Unknown
+
+    # error: [not-iterable] "Object of type `Iterable2` may not be iterable because its `__iter__` attribute (with type `<bound method `__iter__` of `Iterable2`> | None`) may not be callable"
+    for y in Iterable2():
+        # TODO... `int` might be ideal here?
+        reveal_type(y)  # revealed: int | Unknown
+```
+
+## `__iter__` method with a bad signature
+
+<!-- snapshot-diagnostics -->
+
+```py
+from typing_extensions import reveal_type
+
+class Iterator:
+    def __next__(self) -> int:
+        return 42
+
+class Iterable:
+    def __iter__(self, extra_arg) -> Iterator:
+        return Iterator()
+
+# error: [not-iterable]
+for x in Iterable():
+    reveal_type(x)  # revealed: int
+```
+
+## `__iter__` does not return an iterator
+
+<!-- snapshot-diagnostics -->
+
+```py
+from typing_extensions import reveal_type
+
+class Bad:
+    def __iter__(self) -> int:
+        return 42
+
+# error: [not-iterable]
+for x in Bad():
+    reveal_type(x)  # revealed: Unknown
+```
+
+## `__iter__` returns an object with a possibly unbound `__next__` method
+
+```py
+def _(flag: bool):
+    class Iterator:
+        if flag:
+            def __next__(self) -> int:
+                return 42
+
+    class Iterable:
+        def __iter__(self) -> Iterator:
+            return Iterator()
+
+    # error: [not-iterable] "Object of type `Iterable` may not be iterable because its `__iter__` method returns an object of type `Iterator`, which may not have a `__next__` method"
+    for x in Iterable():
+        reveal_type(x)  # revealed: int
+```
+
+## `__iter__` returns an iterator with an invalid `__next__` method
+
+<!-- snapshot-diagnostics -->
+
+```py
+from typing_extensions import reveal_type
+
+class Iterator1:
+    def __next__(self, extra_arg) -> int:
+        return 42
+
+class Iterator2:
+    __next__: None = None
+
+class Iterable1:
+    def __iter__(self) -> Iterator1:
+        return Iterator1()
+
+class Iterable2:
+    def __iter__(self) -> Iterator2:
+        return Iterator2()
+
+# error: [not-iterable]
+for x in Iterable1():
+    reveal_type(x)  # revealed: int
+
+# error: [not-iterable]
+for y in Iterable2():
+    reveal_type(y)  # revealed: Unknown
+```
+
+## Possibly unbound `__iter__` and bad `__getitem__` method
+
+<!-- snapshot-diagnostics -->
+
+```py
+from typing_extensions import reveal_type
+
+def _(flag: bool):
+    class Iterator:
+        def __next__(self) -> int:
+            return 42
+
+    class Iterable:
+        if flag:
+            def __iter__(self) -> Iterator:
+                return Iterator()
+        # invalid signature because it only accepts a `str`,
+        # but the old-style iteration protocol will pass it an `int`
+        def __getitem__(self, key: str) -> bytes:
+            return bytes()
+
+    # error: [not-iterable]
+    for x in Iterable():
+        reveal_type(x)  # revealed: int | bytes
+```
+
+## Possibly unbound `__iter__` and not-callable `__getitem__`
+
+This snippet tests that we infer the element type correctly in the following edge case:
+
+- `__iter__` is a method with the correct parameter spec that returns a valid iterator; BUT
+- `__iter__` is possibly unbound; AND
+- `__getitem__` is set to a non-callable type
+
+It's important that we emit a diagnostic here, but it's also important that we still use the return
+type of the iterator's `__next__` method as the inferred type of `x` in the `for` loop:
+
+```py
+def _(flag: bool):
+    class Iterator:
+        def __next__(self) -> int:
+            return 42
+
+    class Iterable:
+        if flag:
+            def __iter__(self) -> Iterator:
+                return Iterator()
+        __getitem__: None = None
+
+    # error: [not-iterable] "Object of type `Iterable` may not be iterable because it may not have an `__iter__` method and its `__getitem__` attribute has type `None`, which is not callable"
+    for x in Iterable():
+        reveal_type(x)  # revealed: int
+```
+
+## Possibly unbound `__iter__` and possibly unbound `__getitem__`
+
+<!-- snapshot-diagnostics -->
+
+```py
+from typing_extensions import reveal_type
+
+class Iterator:
+    def __next__(self) -> int:
+        return 42
+
+def _(flag1: bool, flag2: bool):
+    class Iterable:
+        if flag1:
+            def __iter__(self) -> Iterator:
+                return Iterator()
+        if flag2:
+            def __getitem__(self, key: int) -> bytes:
+                return bytes()
+
+    # error: [not-iterable]
+    for x in Iterable():
+        reveal_type(x)  # revealed: int | bytes
+```
+
+## No `__iter__` method and `__getitem__` is not callable
+
+<!-- snapshot-diagnostics -->
+
+```py
+from typing_extensions import reveal_type
+
+class Bad:
+    __getitem__: None = None
+
+# error: [not-iterable]
+for x in Bad():
+    reveal_type(x)  # revealed: Unknown
+```
+
+## Possibly-not-callable `__getitem__` method
+
+<!-- snapshot-diagnostics -->
+
+```py
+from typing_extensions import reveal_type
+
+def _(flag: bool):
+    class CustomCallable:
+        if flag:
+            def __call__(self, *args, **kwargs) -> int:
+                return 42
+        else:
+            __call__: None = None
+
+    class Iterable1:
+        __getitem__: CustomCallable = CustomCallable()
+
+    class Iterable2:
+        if flag:
+            def __getitem__(self, key: int) -> int:
+                return 42
+        else:
+            __getitem__: None = None
+
+    # error: [not-iterable]
+    for x in Iterable1():
+        # TODO... `int` might be ideal here?
+        reveal_type(x)  # revealed: int | Unknown
+
+    # error: [not-iterable]
+    for y in Iterable2():
+        # TODO... `int` might be ideal here?
+        reveal_type(y)  # revealed: int | Unknown
+```
+
+## Bad `__getitem__` method
+
+<!-- snapshot-diagnostics -->
+
+```py
+from typing_extensions import reveal_type
+
+class Iterable:
+    # invalid because it will implicitly be passed an `int`
+    # by the interpreter
+    def __getitem__(self, key: str) -> int:
+        return 42
+
+# error: [not-iterable]
+for x in Iterable():
+    reveal_type(x)  # revealed: int
+```
+
+## Possibly unbound `__iter__` but definitely bound `__getitem__`
+
+Here, we should not emit a diagnostic: if `__iter__` is unbound, we should fallback to
+`__getitem__`:
+
+```py
+class Iterator:
+    def __next__(self) -> str:
+        return "foo"
+
+def _(flag: bool):
+    class Iterable:
+        if flag:
+            def __iter__(self) -> Iterator:
+                return Iterator()
+
+        def __getitem__(self, key: int) -> bytes:
+            return b"foo"
+
+    for x in Iterable():
+        reveal_type(x)  # revealed: str | bytes
+```
+
+## Possibly invalid `__iter__` methods
+
+<!-- snapshot-diagnostics -->
+
+```py
+from typing_extensions import reveal_type
+
+class Iterator:
+    def __next__(self) -> int:
+        return 42
+
+def _(flag: bool):
+    class Iterable1:
+        if flag:
+            def __iter__(self) -> Iterator:
+                return Iterator()
+        else:
+            def __iter__(self, invalid_extra_arg) -> Iterator:
+                return Iterator()
+
+    # error: [not-iterable]
+    for x in Iterable1():
+        reveal_type(x)  # revealed: int
+
+    class Iterable2:
+        if flag:
+            def __iter__(self) -> Iterator:
+                return Iterator()
+        else:
+            __iter__: None = None
+
+    # error: [not-iterable]
+    for x in Iterable2():
+        # TODO: `int` would probably be better here:
+        reveal_type(x)  # revealed: int | Unknown
+```
+
+## Possibly invalid `__next__` method
+
+<!-- snapshot-diagnostics -->
+
+```py
+from typing_extensions import reveal_type
+
+def _(flag: bool):
+    class Iterator1:
+        if flag:
+            def __next__(self) -> int:
+                return 42
+        else:
+            def __next__(self, invalid_extra_arg) -> str:
+                return "foo"
+
+    class Iterator2:
+        if flag:
+            def __next__(self) -> int:
+                return 42
+        else:
+            __next__: None = None
+
+    class Iterable1:
+        def __iter__(self) -> Iterator1:
+            return Iterator1()
+
+    class Iterable2:
+        def __iter__(self) -> Iterator2:
+            return Iterator2()
+
+    # error: [not-iterable]
+    for x in Iterable1():
+        reveal_type(x)  # revealed: int | str
+
+    # error: [not-iterable]
+    for y in Iterable2():
+        # TODO: `int` would probably be better here:
+        reveal_type(y)  # revealed: int | Unknown
+```
+
+## Possibly invalid `__getitem__` methods
+
+<!-- snapshot-diagnostics -->
+
+```py
+from typing_extensions import reveal_type
+
+def _(flag: bool):
+    class Iterable1:
+        if flag:
+            def __getitem__(self, item: int) -> str:
+                return "foo"
+        else:
+            __getitem__: None = None
+
+    class Iterable2:
+        if flag:
+            def __getitem__(self, item: int) -> str:
+                return "foo"
+        else:
+            def __getitem__(self, item: str) -> int:
+                return 42
+
+    # error: [not-iterable]
+    for x in Iterable1():
+        # TODO: `str` might be better
+        reveal_type(x)  # revealed: str | Unknown
+
+    # error: [not-iterable]
+    for y in Iterable2():
+        reveal_type(y)  # revealed: str | int
+```
+
+## Possibly unbound `__iter__` and possibly invalid `__getitem__`
+
+<!-- snapshot-diagnostics -->
+
+```py
+from typing_extensions import reveal_type
+
+class Iterator:
+    def __next__(self) -> bytes:
+        return b"foo"
+
+def _(flag: bool, flag2: bool):
+    class Iterable1:
+        if flag:
+            def __getitem__(self, item: int) -> str:
+                return "foo"
+        else:
+            __getitem__: None = None
+
+        if flag2:
+            def __iter__(self) -> Iterator:
+                return Iterator()
+
+    class Iterable2:
+        if flag:
+            def __getitem__(self, item: int) -> str:
+                return "foo"
+        else:
+            def __getitem__(self, item: str) -> int:
+                return 42
+        if flag2:
+            def __iter__(self) -> Iterator:
+                return Iterator()
+
+    # error: [not-iterable]
+    for x in Iterable1():
+        # TODO: `bytes | str` might be better
+        reveal_type(x)  # revealed: bytes | str | Unknown
+
+    # error: [not-iterable]
+    for y in Iterable2():
+        reveal_type(y)  # revealed: bytes | str | int
+```
+
+## Never is iterable
+
+```py
+from typing_extensions import Never
+
+def f(never: Never):
+    for x in never:
+        reveal_type(x)  # revealed: Never
+```

crates/red_knot_python_semantic/resources/mdtest/loops/iterators.md

@@ -10,7 +10,8 @@ class Iterator:
         return 42
 
 class Iterable:
-    def __iter__(self) -> Iterator: ...
+    def __iter__(self) -> Iterator:
+        return Iterator()
 
 def generator_function():
     yield from Iterable()

crates/red_knot_python_semantic/resources/mdtest/loops/while_loop.md

@@ -116,3 +116,14 @@ def _(flag: bool, flag2: bool):
     # error: [possibly-unresolved-reference]
     y
 ```
+
+## Condition with object that implements `__bool__` incorrectly
+
+```py
+class NotBoolable:
+    __bool__ = 3
+
+# error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable"
+while NotBoolable():
+    ...
+```

crates/red_knot_python_semantic/resources/mdtest/metaclass.md

@@ -163,10 +163,11 @@ reveal_type(B.__class__)  # revealed: Literal[M]
 ## Non-class
 
 When a class has an explicit `metaclass` that is not a class, but is a callable that accepts
-`type.__new__` arguments, we should return the meta type of its return type.
+`type.__new__` arguments, we should return the meta-type of its return type.
 
 ```py
-def f(*args, **kwargs) -> int: ...
+def f(*args, **kwargs) -> int:
+    return 1
 
 class A(metaclass=f): ...
 

crates/red_knot_python_semantic/resources/mdtest/narrow/bool-call.md

@@ -22,11 +22,13 @@ def _(flag: bool):
 
     # invalid invocation, too many positional args
     reveal_type(x)  # revealed: Literal[1] | None
-    if bool(x is not None, 5):  # TODO diagnostic
+    # error: [too-many-positional-arguments] "Too many positional arguments to class `bool`: expected 1, got 2"
+    if bool(x is not None, 5):
         reveal_type(x)  # revealed: Literal[1] | None
 
     # invalid invocation, too many kwargs
     reveal_type(x)  # revealed: Literal[1] | None
-    if bool(x is not None, y=5):  # TODO diagnostic
+    # error: [unknown-argument] "Argument `y` does not match any known parameter of class `bool`"
+    if bool(x is not None, y=5):
         reveal_type(x)  # revealed: Literal[1] | None
 ```

crates/red_knot_python_semantic/resources/mdtest/narrow/isinstance.md

@@ -97,12 +97,7 @@ else:
 ## No narrowing for instances of `builtins.type`
 
 ```py
-def _(flag: bool):
-    t = type("t", (), {})
-
-    # This isn't testing what we want it to test if we infer anything more precise here:
-    reveal_type(t)  # revealed: type
-
+def _(flag: bool, t: type):
     x = 1 if flag else "foo"
 
     if isinstance(x, t):

crates/red_knot_python_semantic/resources/mdtest/narrow/issubclass.md

@@ -112,8 +112,7 @@ def _(flag: bool):
         reveal_type(t)  # revealed: Literal[NoneType]
 
     if issubclass(t, type(None)):
-        # TODO: this should be just `Literal[NoneType]`
-        reveal_type(t)  # revealed: Literal[int, NoneType]
+        reveal_type(t)  # revealed: Literal[NoneType]
 ```
 
 ## `classinfo` contains multiple types
@@ -171,7 +170,8 @@ if issubclass(t, int):
 def issubclass(c, ci):
     return True
 
-def flag() -> bool: ...
+def flag() -> bool:
+    return True
 
 t = int if flag() else str
 if issubclass(t, int):
@@ -183,7 +183,8 @@ if issubclass(t, int):
 ```py
 issubclass_alias = issubclass
 
-def flag() -> bool: ...
+def flag() -> bool:
+    return True
 
 t = int if flag() else str
 if issubclass_alias(t, int):
@@ -195,7 +196,8 @@ if issubclass_alias(t, int):
 ```py
 from builtins import issubclass as imported_issubclass
 
-def flag() -> bool: ...
+def flag() -> bool:
+    return True
 
 t = int if flag() else str
 if imported_issubclass(t, int):
@@ -207,7 +209,8 @@ if imported_issubclass(t, int):
 ```py
 from typing import Any
 
-def flag() -> bool: ...
+def flag() -> bool:
+    return True
 
 t = int if flag() else str
 
@@ -230,7 +233,8 @@ if issubclass(t, Any):
 ### Do not narrow if there are keyword arguments
 
 ```py
-def flag() -> bool: ...
+def flag() -> bool:
+    return True
 
 t = int if flag() else str
 

crates/red_knot_python_semantic/resources/mdtest/narrow/match.md

@@ -20,7 +20,8 @@ def _(flag: bool):
 ## Class patterns
 
 ```py
-def get_object() -> object: ...
+def get_object() -> object:
+    return object()
 
 class A: ...
 class B: ...
@@ -42,10 +43,12 @@ reveal_type(x)  # revealed: object
 ## Class pattern with guard
 
 ```py
-def get_object() -> object: ...
+def get_object() -> object:
+    return object()
 
 class A:
-    def y() -> int: ...
+    def y() -> int:
+        return 1
 
 class B: ...
 

crates/red_knot_python_semantic/resources/mdtest/narrow/truthiness.md

@@ -233,16 +233,20 @@ reveal_type(y)  # revealed: A
 from typing import Literal
 
 class MetaAmbiguous(type):
-    def __bool__(self) -> bool: ...
+    def __bool__(self) -> bool:
+        return True
 
 class MetaFalsy(type):
-    def __bool__(self) -> Literal[False]: ...
+    def __bool__(self) -> Literal[False]:
+        return False
 
 class MetaTruthy(type):
-    def __bool__(self) -> Literal[True]: ...
+    def __bool__(self) -> Literal[True]:
+        return True
 
 class MetaDeferred(type):
-    def __bool__(self) -> MetaAmbiguous: ...
+    def __bool__(self) -> MetaAmbiguous:
+        return MetaAmbiguous()
 
 class AmbiguousClass(metaclass=MetaAmbiguous): ...
 class FalsyClass(metaclass=MetaFalsy): ...
@@ -266,7 +270,7 @@ def _(
     if af:
         reveal_type(af)  # revealed: type[AmbiguousClass] & ~AlwaysFalsy
 
-    # TODO: Emit a diagnostic (`d` is not valid in boolean context)
+    # error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `MetaDeferred`; the return type of its bool method (`MetaAmbiguous`) isn't assignable to `bool"
     if d:
         # TODO: Should be `Unknown`
         reveal_type(d)  # revealed: type[DeferredClass] & ~AlwaysFalsy
@@ -300,20 +304,24 @@ def _(x: bool | str):
     reveal_type(x and A())  # revealed: Literal[False] | str & ~AlwaysTruthy | A
 
 class Falsy:
-    def __bool__(self) -> Literal[False]: ...
+    def __bool__(self) -> Literal[False]:
+        return False
 
 class Truthy:
-    def __bool__(self) -> Literal[True]: ...
+    def __bool__(self) -> Literal[True]:
+        return True
 
 def _(x: Falsy | Truthy):
     reveal_type(x or A())  # revealed: Truthy | A
     reveal_type(x and A())  # revealed: Falsy | A
 
 class MetaFalsy(type):
-    def __bool__(self) -> Literal[False]: ...
+    def __bool__(self) -> Literal[False]:
+        return False
 
 class MetaTruthy(type):
-    def __bool__(self) -> Literal[True]: ...
+    def __bool__(self) -> Literal[True]:
+        return True
 
 class FalsyClass(metaclass=MetaFalsy): ...
 class TruthyClass(metaclass=MetaTruthy): ...

crates/red_knot_python_semantic/resources/mdtest/narrow/type.md

@@ -88,7 +88,7 @@ def _(x: str | int):
 
 ```py
 def _(x: str | int):
-    # TODO: we could issue a diagnostic here
+    # error: [no-matching-overload] "No overload of class `type` matches arguments"
     if type(object=x) is str:
         reveal_type(x)  # revealed: str | int
 ```

crates/red_knot_python_semantic/resources/mdtest/narrow/while.md

@@ -9,7 +9,8 @@ is retained after the loop.
 ## Basic `while` loop
 
 ```py
-def next_item() -> int | None: ...
+def next_item() -> int | None:
+    return 1
 
 x = next_item()
 
@@ -23,7 +24,8 @@ reveal_type(x)  # revealed: None
 ## `while` loop with `else`
 
 ```py
-def next_item() -> int | None: ...
+def next_item() -> int | None:
+    return 1
 
 x = next_item()
 
@@ -41,7 +43,8 @@ reveal_type(x)  # revealed: None
 ```py
 from typing import Literal
 
-def next_item() -> Literal[1, 2, 3]: ...
+def next_item() -> Literal[1, 2, 3]:
+    raise NotImplementedError
 
 x = next_item()
 

crates/red_knot_python_semantic/resources/mdtest/scopes/eager.md

@@ -341,10 +341,12 @@ annotation are looked up lazily, even if they occur in an eager scope.
 ### Eager annotations in a Python file
 
 ```py
+from typing import ClassVar
+
 x = int
 
 class C:
-    var: x
+    var: ClassVar[x]
 
 reveal_type(C.var)  # revealed: int
 
@@ -356,10 +358,12 @@ x = str
 ```py
 from __future__ import annotations
 
+from typing import ClassVar
+
 x = int
 
 class C:
-    var: x
+    var: ClassVar[x]
 
 reveal_type(C.var)  # revealed: Unknown | str
 
@@ -369,10 +373,12 @@ x = str
 ### Deferred annotations in a stub file
 
 ```pyi
+from typing import ClassVar
+
 x = int
 
 class C:
-    var: x
+    var: ClassVar[x]
 
 reveal_type(C.var)  # revealed: Unknown | str
 

crates/red_knot_python_semantic/resources/mdtest/scopes/moduletype_attrs.md

@@ -9,7 +9,7 @@ is unbound.
 ```py
 reveal_type(__name__)  # revealed: str
 reveal_type(__file__)  # revealed: str | None
-reveal_type(__loader__)  # revealed: @Todo(instance attribute on class with dynamic base) | None
+reveal_type(__loader__)  # revealed: LoaderProtocol | None
 reveal_type(__package__)  # revealed: str | None
 reveal_type(__doc__)  # revealed: str | None
 
@@ -136,3 +136,43 @@ if returns_bool():
 reveal_type(__file__)  # revealed: Literal[42]
 reveal_type(__name__)  # revealed: Literal[1] | str
 ```
+
+## Implicit global attributes in the current module override implicit globals from builtins
+
+Here, we take the type of the implicit global symbol `__name__` from the `types.ModuleType` stub
+(which in this custom typeshed specifies the type as `bytes`). This is because the `main` module has
+an implicit `__name__` global that shadows the builtin `__name__` symbol.
+
+```toml
+[environment]
+typeshed = "/typeshed"
+```
+
+`/typeshed/stdlib/builtins.pyi`:
+
+```pyi
+class object: ...
+class int: ...
+class bytes: ...
+
+__name__: int = 42
+```
+
+`/typeshed/stdlib/types.pyi`:
+
+```pyi
+class ModuleType:
+    __name__: bytes
+```
+
+`/typeshed/stdlib/typing_extensions.pyi`:
+
+```pyi
+def reveal_type(obj, /): ...
+```
+
+`main.py`:
+
+```py
+reveal_type(__name__)  # revealed: bytes
+```

crates/red_knot_python_semantic/resources/mdtest/slots.md

@@ -167,7 +167,7 @@ class A:
     __slots__ = ()
     __slots__ += ("a", "b")
 
-reveal_type(A.__slots__)  # revealed: @Todo(return type)
+reveal_type(A.__slots__)  # revealed: @Todo(return type of decorated function)
 
 class B:
     __slots__ = ("c", "d")

crates/red_knot_python_semantic/resources/mdtest/snapshots/for.md_-_For_loops_-_Bad_`__getitem__`_method.snap

@@ -0,0 +1,52 @@
+---
+source: crates/red_knot_test/src/lib.rs
+expression: snapshot
+---
+---
+mdtest name: for.md - For loops - Bad `__getitem__` method
+mdtest path: crates/red_knot_python_semantic/resources/mdtest/loops/for.md
+---
+
+# Python source files
+
+## mdtest_snippet.py
+
+```
+ 1 | from typing_extensions import reveal_type
+ 2 | 
+ 3 | class Iterable:
+ 4 |     # invalid because it will implicitly be passed an `int`
+ 5 |     # by the interpreter
+ 6 |     def __getitem__(self, key: str) -> int:
+ 7 |         return 42
+ 8 | 
+ 9 | # error: [not-iterable]
+10 | for x in Iterable():
+11 |     reveal_type(x)  # revealed: int
+```
+
+# Diagnostics
+
+```
+error: lint:not-iterable
+  --> /src/mdtest_snippet.py:10:10
+   |
+ 9 | # error: [not-iterable]
+10 | for x in Iterable():
+   |          ^^^^^^^^^^ Object of type `Iterable` is not iterable because it has no `__iter__` method and its `__getitem__` method has an incorrect signature for the old-style iteration protocol (expected a signature at least as permissive as `def __getitem__(self, key: int): ...`)
+11 |     reveal_type(x)  # revealed: int
+   |
+
+```
+
+```
+info: revealed-type
+  --> /src/mdtest_snippet.py:11:5
+   |
+ 9 | # error: [not-iterable]
+10 | for x in Iterable():
+11 |     reveal_type(x)  # revealed: int
+   |     -------------- info: Revealed type is `int`
+   |
+
+```

crates/red_knot_python_semantic/resources/mdtest/snapshots/for.md_-_For_loops_-_Invalid_iterable.snap

@@ -0,0 +1,32 @@
+---
+source: crates/red_knot_test/src/lib.rs
+expression: snapshot
+---
+---
+mdtest name: for.md - For loops - Invalid iterable
+mdtest path: crates/red_knot_python_semantic/resources/mdtest/loops/for.md
+---
+
+# Python source files
+
+## mdtest_snippet.py
+
+```
+1 | nonsense = 123
+2 | for x in nonsense:  # error: [not-iterable]
+3 |     pass
+```
+
+# Diagnostics
+
+```
+error: lint:not-iterable
+ --> /src/mdtest_snippet.py:2:10
+  |
+1 | nonsense = 123
+2 | for x in nonsense:  # error: [not-iterable]
+  |          ^^^^^^^^ Object of type `Literal[123]` is not iterable because it doesn't have an `__iter__` method or a `__getitem__` method
+3 |     pass
+  |
+
+```

crates/red_knot_python_semantic/resources/mdtest/snapshots/for.md_-_For_loops_-_New_over_old_style_iteration_protocol.snap

@@ -0,0 +1,37 @@
+---
+source: crates/red_knot_test/src/lib.rs
+expression: snapshot
+---
+---
+mdtest name: for.md - For loops - New over old style iteration protocol
+mdtest path: crates/red_knot_python_semantic/resources/mdtest/loops/for.md
+---
+
+# Python source files
+
+## mdtest_snippet.py
+
+```
+1 | class NotIterable:
+2 |     def __getitem__(self, key: int) -> int:
+3 |         return 42
+4 |     __iter__: None = None
+5 | 
+6 | for x in NotIterable():  # error: [not-iterable]
+7 |     pass
+```
+
+# Diagnostics
+
+```
+error: lint:not-iterable
+ --> /src/mdtest_snippet.py:6:10
+  |
+4 |     __iter__: None = None
+5 |
+6 | for x in NotIterable():  # error: [not-iterable]
+  |          ^^^^^^^^^^^^^ Object of type `NotIterable` is not iterable because its `__iter__` attribute has type `None`, which is not callable
+7 |     pass
+  |
+
+```

crates/red_knot_python_semantic/resources/mdtest/snapshots/for.md_-_For_loops_-_No_`__iter__`_method_and_`__getitem__`_is_not_callable.snap

@@ -0,0 +1,49 @@
+---
+source: crates/red_knot_test/src/lib.rs
+expression: snapshot
+---
+---
+mdtest name: for.md - For loops - No `__iter__` method and `__getitem__` is not callable
+mdtest path: crates/red_knot_python_semantic/resources/mdtest/loops/for.md
+---
+
+# Python source files
+
+## mdtest_snippet.py
+
+```
+1 | from typing_extensions import reveal_type
+2 | 
+3 | class Bad:
+4 |     __getitem__: None = None
+5 | 
+6 | # error: [not-iterable]
+7 | for x in Bad():
+8 |     reveal_type(x)  # revealed: Unknown
+```
+
+# Diagnostics
+
+```
+error: lint:not-iterable
+ --> /src/mdtest_snippet.py:7:10
+  |
+6 | # error: [not-iterable]
+7 | for x in Bad():
+  |          ^^^^^ Object of type `Bad` is not iterable because it has no `__iter__` method and its `__getitem__` attribute has type `None`, which is not callable
+8 |     reveal_type(x)  # revealed: Unknown
+  |
+
+```
+
+```
+info: revealed-type
+ --> /src/mdtest_snippet.py:8:5
+  |
+6 | # error: [not-iterable]
+7 | for x in Bad():
+8 |     reveal_type(x)  # revealed: Unknown
+  |     -------------- info: Revealed type is `Unknown`
+  |
+
+```