[red-knot] mypy_primer: split installation and execution

## Summary

- Add comment to explain `sed` command
- Fix double reporting of diff
- Hide (large) diffs in `<details>`

.github/ISSUE_TEMPLATE/1_bug_report.yaml

@@ -0,0 +1,31 @@
+name: Bug report
+description: Report an error or unexpected behavior
+body:
+  - type: markdown
+    attributes:
+      value: |
+        Thank you for taking the time to report an issue! We're glad to have you involved with Ruff.
+
+        **Before reporting, please make sure to search through [existing issues](https://github.com/astral-sh/ruff/issues?q=is:issue+is:open+label:bug) (including [closed](https://github.com/astral-sh/ruff/issues?q=is:issue%20state:closed%20label:bug)).**
+
+  - type: textarea
+    attributes:
+      label: Summary
+      description: |
+        A clear and concise description of the bug, including a minimal reproducible example.
+
+        Be sure to include the command you invoked (e.g., `ruff check /path/to/file.py --fix`), ideally including the `--isolated` flag and
+        the current Ruff settings (e.g., relevant sections from your `pyproject.toml`).
+
+        If possible, try to include the [playground](https://play.ruff.rs) link that reproduces this issue.
+
+    validations:
+      required: true
+
+  - type: input
+    attributes:
+      label: Version
+      description: What version of ruff are you using? (see `ruff version`)
+      placeholder: e.g., ruff 0.9.3 (90589372d 2025-01-23)
+    validations:
+      required: false

.github/ISSUE_TEMPLATE/2_rule_request.yaml

@@ -0,0 +1,10 @@
+name: Rule request
+description: Anything related to lint rules (proposing new rules, changes to existing rules, auto-fixes, etc.)
+body:
+  - type: textarea
+    attributes:
+      label: Summary
+      description: |
+        A clear and concise description of the relevant request. If applicable, please describe the current behavior as well.
+    validations:
+      required: true

.github/ISSUE_TEMPLATE/3_question.yaml

@@ -0,0 +1,18 @@
+name: Question
+description: Ask a question about Ruff
+labels: ["question"]
+body:
+  - type: textarea
+    attributes:
+      label: Question
+      description: Describe your question in detail.
+    validations:
+      required: true
+
+  - type: input
+    attributes:
+      label: Version
+      description: What version of ruff are you using? (see `ruff version`)
+      placeholder: e.g., ruff 0.9.3 (90589372d 2025-01-23)
+    validations:
+      required: false

.github/ISSUE_TEMPLATE/config.yml

@@ -1,2 +1,8 @@
-# This file cannot use the extension `.yaml`.
-blank_issues_enabled: false
+blank_issues_enabled: true
+contact_links:
+  - name: Documentation
+    url: https://docs.astral.sh/ruff
+    about: Please consult the documentation before creating an issue.
+  - name: Community
+    url: https://discord.com/invite/astral-sh
+    about: Join our Discord community to ask questions and collaborate.

.github/ISSUE_TEMPLATE/issue.yaml

@@ -1,22 +0,0 @@
-name: New issue
-description: A generic issue
-
-body:
-  - type: markdown
-    attributes:
-      value: |
-        Thank you for taking the time to report an issue! We're glad to have you involved with Ruff.
-
-        If you're filing a bug report, please consider including the following information:
-
-        * List of keywords you searched for before creating this issue. Write them down here so that others can find this issue more easily and help provide feedback.
-        e.g. "RUF001", "unused variable", "Jupyter notebook"
-        * A minimal code snippet that reproduces the bug.
-        * The command you invoked (e.g., `ruff /path/to/file.py --fix`), ideally including the `--isolated` flag.
-        * The current Ruff settings (any relevant sections from your `pyproject.toml`).
-        * The current Ruff version (`ruff --version`).
-
-  - type: textarea
-    attributes:
-      label: Description
-      description: A description of the issue

.github/renovate.json5

@@ -95,14 +95,7 @@
       matchManagers: ["cargo"],
       matchPackageNames: ["strum"],
       description: "Weekly update of strum dependencies",
-    },
-    {
-      groupName: "ESLint",
-      matchManagers: ["npm"],
-      matchPackageNames: ["eslint"],
-      allowedVersions: "<9",
-      description: "Constraint ESLint to version 8 until TypeScript-eslint supports ESLint 9", // https://github.com/typescript-eslint/typescript-eslint/issues/8211
-    },
+    }
   ],
   vulnerabilityAlerts: {
     commitMessageSuffix: "",

.github/workflows/ci.yaml

@@ -712,7 +712,7 @@ jobs:
           just test
 
   benchmarks:
-    runs-on: ubuntu-22.04
+    runs-on: ubuntu-24.04
     needs: determine_changes
     if: ${{ github.repository == 'astral-sh/ruff' && !contains(github.event.pull_request.labels.*.name, 'no-test') && (needs.determine_changes.outputs.code == 'true' || github.ref == 'refs/heads/main') }}
     timeout-minutes: 20

.github/workflows/daily_property_tests.yaml

@@ -47,6 +47,7 @@ jobs:
         run: |
           export QUICKCHECK_TESTS=100000
           for _ in {1..5}; do
+            cargo test --locked --release --package red_knot_python_semantic -- --ignored list::property_tests
             cargo test --locked --release --package red_knot_python_semantic -- --ignored types::property_tests::stable
           done
 

.github/workflows/mypy_primer.yaml

@@ -0,0 +1,93 @@
+name: Run mypy_primer
+
+permissions: {}
+
+on:
+  pull_request:
+    paths:
+      - "crates/red_knot*/**"
+      - "crates/ruff_db"
+      - "crates/ruff_python_ast"
+      - "crates/ruff_python_parser"
+      - ".github/workflows/mypy_primer.yaml"
+      - ".github/workflows/mypy_primer_comment.yaml"
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref_name }}-${{ github.event.pull_request.number || github.sha }}
+  cancel-in-progress: true
+
+env:
+  CARGO_INCREMENTAL: 0
+  CARGO_NET_RETRY: 10
+  CARGO_TERM_COLOR: always
+  RUSTUP_MAX_RETRIES: 10
+
+jobs:
+  mypy_primer:
+    name: Run mypy_primer
+    runs-on: ubuntu-24.04
+    timeout-minutes: 20
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          path: ruff
+          fetch-depth: 0
+          persist-credentials: false
+
+      - name: Install the latest version of uv
+        uses: astral-sh/setup-uv@v5
+
+      - uses: Swatinem/rust-cache@v2
+        with:
+          workspaces: "ruff"
+      - name: Install Rust toolchain
+        run: rustup show
+
+      - name: Install mypy_primer
+        run: |
+          uv tool install "git+https://github.com/astral-sh/mypy_primer.git@add-red-knot-support"
+
+      - name: Run mypy_primer
+        shell: bash
+        run: |
+          cd ruff
+
+          echo "new commit"
+          git rev-list --format=%s --max-count=1 "$GITHUB_SHA"
+
+          MERGE_BASE="$(git merge-base "$GITHUB_SHA" "origin/$GITHUB_BASE_REF")"
+          git checkout -b base_commit "$MERGE_BASE"
+          echo "base commit"
+          git rev-list --format=%s --max-count=1 base_commit
+
+          cd ..
+
+          # Allow the exit code to be 0 or 1, only fail for actual mypy_primer crashes/bugs
+          uvx mypy_primer \
+            --repo ruff \
+            --type-checker knot \
+            --old base_commit \
+            --new "$GITHUB_SHA" \
+            --project-selector '/(mypy_primer|black|pyp|git-revise|zipp|arrow)$' \
+            --output concise \
+            --debug > mypy_primer.diff || [ $? -eq 1 ]
+
+          # Output diff with ANSI color codes
+          cat mypy_primer.diff
+
+          # Remove ANSI color codes before uploading
+          sed -ie 's/\x1b\[[0-9;]*m//g' mypy_primer.diff
+
+          echo ${{ github.event.number }} > pr-number
+
+      - name: Upload diff
+        uses: actions/upload-artifact@v4
+        with:
+          name: mypy_primer_diff
+          path: mypy_primer.diff
+
+      - name: Upload pr-number
+        uses: actions/upload-artifact@v4
+        with:
+          name: pr-number
+          path: pr-number

.github/workflows/mypy_primer_comment.yaml

@@ -0,0 +1,97 @@
+name: PR comment (mypy_primer)
+
+on: # zizmor: ignore[dangerous-triggers]
+  workflow_run:
+    workflows: [Run mypy_primer]
+    types: [completed]
+  workflow_dispatch:
+    inputs:
+      workflow_run_id:
+        description: The mypy_primer workflow that triggers the workflow run
+        required: true
+
+jobs:
+  comment:
+    runs-on: ubuntu-24.04
+    permissions:
+      pull-requests: write
+    steps:
+      - uses: dawidd6/action-download-artifact@v8
+        name: Download PR number
+        with:
+          name: pr-number
+          run_id: ${{ github.event.workflow_run.id ||  github.event.inputs.workflow_run_id }}
+          if_no_artifact_found: ignore
+          allow_forks: true
+
+      - name: Parse pull request number
+        id: pr-number
+        run: |
+          if [[ -f pr-number ]]
+          then
+            echo "pr-number=$(<pr-number)" >> "$GITHUB_OUTPUT"
+          fi
+
+      - uses: dawidd6/action-download-artifact@v8
+        name: "Download mypy_primer results"
+        id: download-mypy_primer_diff
+        if: steps.pr-number.outputs.pr-number
+        with:
+          name: mypy_primer_diff
+          workflow: mypy_primer.yaml
+          pr: ${{ steps.pr-number.outputs.pr-number }}
+          path: pr/mypy_primer_diff
+          workflow_conclusion: completed
+          if_no_artifact_found: ignore
+          allow_forks: true
+
+      - name: Generate comment content
+        id: generate-comment
+        if: steps.download-mypy_primer_diff.outputs.found_artifact == 'true'
+        run: |
+          # Guard against malicious mypy_primer results that symlink to a secret
+          # file on this runner
+          if [[ -L pr/mypy_primer_diff/mypy_primer.diff ]]
+          then
+              echo "Error: mypy_primer.diff cannot be a symlink"
+              exit 1
+          fi
+
+          # Note this identifier is used to find the comment to update on
+          # subsequent runs
+          echo '<!-- generated-comment mypy_primer -->' >> comment.txt
+
+          echo '## `mypy_primer` results' >> comment.txt
+          if [ -s "pr/mypy_primer_diff/mypy_primer.diff" ]; then
+            echo '<details>' >> comment.txt
+            echo '<summary>Changes were detected when running on open source projects</summary>' >> comment.txt
+            echo '' >> comment.txt
+            echo '```diff' >> comment.txt
+            cat pr/mypy_primer_diff/mypy_primer.diff >> comment.txt
+            echo '```' >> comment.txt
+            echo '</details>' >> comment.txt
+          else
+            echo 'No ecosystem changes detected ✅' >> comment.txt
+          fi
+
+          echo 'comment<<EOF' >> "$GITHUB_OUTPUT"
+          cat comment.txt >> "$GITHUB_OUTPUT"
+          echo 'EOF' >> "$GITHUB_OUTPUT"
+
+      - name: Find existing comment
+        uses: peter-evans/find-comment@v3
+        if: steps.generate-comment.outcome == 'success'
+        id: find-comment
+        with:
+          issue-number: ${{ steps.pr-number.outputs.pr-number }}
+          comment-author: "github-actions[bot]"
+          body-includes: "<!-- generated-comment mypy_primer -->"
+
+      - name: Create or update comment
+        if: steps.find-comment.outcome == 'success'
+        uses: peter-evans/create-or-update-comment@v4
+        with:
+          comment-id: ${{ steps.find-comment.outputs.comment-id }}
+          issue-number: ${{ steps.pr-number.outputs.pr-number }}
+          body-path: comment.txt
+          edit-mode: replace

.github/workflows/publish-playground.yml

@@ -35,6 +35,8 @@ jobs:
           cache: "npm"
           cache-dependency-path: playground/package-lock.json
       - uses: jetli/wasm-pack-action@v0.4.0
+        with:
+          version: v0.13.1
       - uses: jetli/wasm-bindgen-action@v0.2.0
       - name: "Run wasm-pack"
         run: wasm-pack build --target web --out-dir ../../playground/src/pkg crates/ruff_wasm
@@ -49,7 +51,7 @@ jobs:
         working-directory: playground
       - name: "Deploy to Cloudflare Pages"
         if: ${{ env.CF_API_TOKEN_EXISTS == 'true' }}
-        uses: cloudflare/wrangler-action@v3.13.1
+        uses: cloudflare/wrangler-action@v3.14.0
         with:
           apiToken: ${{ secrets.CF_API_TOKEN }}
           accountId: ${{ secrets.CF_ACCOUNT_ID }}

.github/workflows/publish-wasm.yml

@@ -35,6 +35,8 @@ jobs:
       - name: "Install Rust toolchain"
         run: rustup target add wasm32-unknown-unknown
       - uses: jetli/wasm-pack-action@v0.4.0
+        with:
+          version: v0.13.1
       - uses: jetli/wasm-bindgen-action@v0.2.0
       - name: "Run wasm-pack build"
         run: wasm-pack build --target ${{ matrix.target }} crates/ruff_wasm

.pre-commit-config.yaml

@@ -60,7 +60,7 @@ repos:
           - black==25.1.0
 
   - repo: https://github.com/crate-ci/typos
-    rev: v1.29.5
+    rev: v1.30.0
     hooks:
       - id: typos
 
@@ -74,7 +74,7 @@ repos:
         pass_filenames: false # This makes it a lot faster
 
   - repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: v0.9.4
+    rev: v0.9.9
     hooks:
       - id: ruff-format
       - id: ruff
@@ -84,7 +84,7 @@ repos:
 
   # Prettier
   - repo: https://github.com/rbubley/mirrors-prettier
-    rev: v3.4.2
+    rev: v3.5.2
     hooks:
       - id: prettier
         types: [yaml]
@@ -92,12 +92,12 @@ repos:
   # zizmor detects security vulnerabilities in GitHub Actions workflows.
   # Additional configuration for the tool is found in `.github/zizmor.yml`
   - repo: https://github.com/woodruffw/zizmor-pre-commit
-    rev: v1.3.0
+    rev: v1.4.1
     hooks:
       - id: zizmor
 
   - repo: https://github.com/python-jsonschema/check-jsonschema
-    rev: 0.31.1
+    rev: 0.31.2
     hooks:
       - id: check-github-workflows
 

BREAKING_CHANGES.md

@@ -209,8 +209,8 @@ This change only affects those using Ruff under its default rule set. Users that
 
 ### Remove support for emoji identifiers ([#7212](https://github.com/astral-sh/ruff/pull/7212))
 
-Previously, Ruff supported the non-standard compliant emoji identifiers e.g. `📦 = 1`.
-We decided to remove this non-standard language extension, and Ruff now reports syntax errors for emoji identifiers in your code, the same as CPython.
+Previously, Ruff supported non-standards-compliant emoji identifiers such as `📦 = 1`.
+We decided to remove this non-standard language extension. Ruff now reports syntax errors for invalid emoji identifiers in your code, the same as CPython.
 
 ### Improved GitLab fingerprints ([#7203](https://github.com/astral-sh/ruff/pull/7203))
 

CHANGELOG.md

@@ -1,5 +1,172 @@
 # Changelog
 
+## 0.9.10
+
+### Preview features
+
+- \[`ruff`\] Add new rule `RUF059`: Unused unpacked assignment ([#16449](https://github.com/astral-sh/ruff/pull/16449))
+- \[`syntax-errors`\] Detect assignment expressions before Python 3.8 ([#16383](https://github.com/astral-sh/ruff/pull/16383))
+- \[`syntax-errors`\] Named expressions in decorators before Python 3.9 ([#16386](https://github.com/astral-sh/ruff/pull/16386))
+- \[`syntax-errors`\] Parenthesized keyword argument names after Python 3.8 ([#16482](https://github.com/astral-sh/ruff/pull/16482))
+- \[`syntax-errors`\] Positional-only parameters before Python 3.8 ([#16481](https://github.com/astral-sh/ruff/pull/16481))
+- \[`syntax-errors`\] Tuple unpacking in `return` and `yield` before Python 3.8 ([#16485](https://github.com/astral-sh/ruff/pull/16485))
+- \[`syntax-errors`\] Type parameter defaults before Python 3.13 ([#16447](https://github.com/astral-sh/ruff/pull/16447))
+- \[`syntax-errors`\] Type parameter lists before Python 3.12 ([#16479](https://github.com/astral-sh/ruff/pull/16479))
+- \[`syntax-errors`\] `except*` before Python 3.11 ([#16446](https://github.com/astral-sh/ruff/pull/16446))
+- \[`syntax-errors`\] `type` statements before Python 3.12 ([#16478](https://github.com/astral-sh/ruff/pull/16478))
+
+### Bug fixes
+
+- Escape template filenames in glob patterns in configuration ([#16407](https://github.com/astral-sh/ruff/pull/16407))
+- \[`flake8-simplify`\] Exempt unittest context methods for `SIM115` rule ([#16439](https://github.com/astral-sh/ruff/pull/16439))
+- Formatter: Fix syntax error location in notebooks ([#16499](https://github.com/astral-sh/ruff/pull/16499))
+- \[`pyupgrade`\] Do not offer fix when at least one target is `global`/`nonlocal` (`UP028`) ([#16451](https://github.com/astral-sh/ruff/pull/16451))
+- \[`flake8-builtins`\] Ignore variables matching module attribute names (`A001`) ([#16454](https://github.com/astral-sh/ruff/pull/16454))
+- \[`pylint`\] Convert `code` keyword argument to a positional argument in fix for (`PLR1722`) ([#16424](https://github.com/astral-sh/ruff/pull/16424))
+
+### CLI
+
+- Move rule code from `description` to `check_name` in GitLab output serializer ([#16437](https://github.com/astral-sh/ruff/pull/16437))
+
+### Documentation
+
+- \[`pydocstyle`\] Clarify that `D417` only checks docstrings with an arguments section ([#16494](https://github.com/astral-sh/ruff/pull/16494))
+
+## 0.9.9
+
+### Preview features
+
+- Fix caching of unsupported-syntax errors ([#16425](https://github.com/astral-sh/ruff/pull/16425))
+
+### Bug fixes
+
+- Only show unsupported-syntax errors in editors when preview mode is enabled ([#16429](https://github.com/astral-sh/ruff/pull/16429))
+
+## 0.9.8
+
+### Preview features
+
+- Start detecting version-related syntax errors in the parser ([#16090](https://github.com/astral-sh/ruff/pull/16090))
+
+### Rule changes
+
+- \[`pylint`\] Mark fix unsafe (`PLW1507`) ([#16343](https://github.com/astral-sh/ruff/pull/16343))
+- \[`pylint`\] Catch `case np.nan`/`case math.nan` in `match` statements (`PLW0177`) ([#16378](https://github.com/astral-sh/ruff/pull/16378))
+- \[`ruff`\] Add more Pydantic models variants to the list of default copy semantics (`RUF012`) ([#16291](https://github.com/astral-sh/ruff/pull/16291))
+
+### Server
+
+- Avoid indexing the project if `configurationPreference` is `editorOnly` ([#16381](https://github.com/astral-sh/ruff/pull/16381))
+- Avoid unnecessary info at non-trace server log level ([#16389](https://github.com/astral-sh/ruff/pull/16389))
+- Expand `ruff.configuration` to allow inline config ([#16296](https://github.com/astral-sh/ruff/pull/16296))
+- Notify users for invalid client settings ([#16361](https://github.com/astral-sh/ruff/pull/16361))
+
+### Configuration
+
+- Add `per-file-target-version` option ([#16257](https://github.com/astral-sh/ruff/pull/16257))
+
+### Bug fixes
+
+- \[`refurb`\] Do not consider docstring(s) (`FURB156`) ([#16391](https://github.com/astral-sh/ruff/pull/16391))
+- \[`flake8-self`\] Ignore attribute accesses on instance-like variables (`SLF001`) ([#16149](https://github.com/astral-sh/ruff/pull/16149))
+- \[`pylint`\] Fix false positives, add missing methods, and support positional-only parameters (`PLE0302`) ([#16263](https://github.com/astral-sh/ruff/pull/16263))
+- \[`flake8-pyi`\] Mark `PYI030` fix unsafe when comments are deleted ([#16322](https://github.com/astral-sh/ruff/pull/16322))
+
+### Documentation
+
+- Fix example for `S611` ([#16316](https://github.com/astral-sh/ruff/pull/16316))
+- Normalize inconsistent markdown headings in docstrings ([#16364](https://github.com/astral-sh/ruff/pull/16364))
+- Document MSRV policy ([#16384](https://github.com/astral-sh/ruff/pull/16384))
+
+## 0.9.7
+
+### Preview features
+
+- Consider `__new__` methods as special function type for enforcing class method or static method rules ([#13305](https://github.com/astral-sh/ruff/pull/13305))
+- \[`airflow`\] Improve the internal logic to differentiate deprecated symbols (`AIR303`) ([#16013](https://github.com/astral-sh/ruff/pull/16013))
+- \[`refurb`\] Manual timezone monkeypatching (`FURB162`) ([#16113](https://github.com/astral-sh/ruff/pull/16113))
+- \[`ruff`\] Implicit class variable in dataclass (`RUF045`) ([#14349](https://github.com/astral-sh/ruff/pull/14349))
+- \[`ruff`\] Skip singleton starred expressions for `incorrectly-parenthesized-tuple-in-subscript` (`RUF031`) ([#16083](https://github.com/astral-sh/ruff/pull/16083))
+- \[`refurb`\] Check for subclasses includes subscript expressions (`FURB189`) ([#16155](https://github.com/astral-sh/ruff/pull/16155))
+
+### Rule changes
+
+- \[`flake8-debugger`\] Also flag `sys.breakpointhook` and `sys.__breakpointhook__` (`T100`) ([#16191](https://github.com/astral-sh/ruff/pull/16191))
+- \[`pycodestyle`\] Exempt `site.addsitedir(...)` calls (`E402`) ([#16251](https://github.com/astral-sh/ruff/pull/16251))
+
+### Formatter
+
+- Fix unstable formatting of trailing end-of-line comments of parenthesized attribute values ([#16187](https://github.com/astral-sh/ruff/pull/16187))
+
+### Server
+
+- Fix handling of requests received after shutdown message ([#16262](https://github.com/astral-sh/ruff/pull/16262))
+- Ignore `source.organizeImports.ruff` and `source.fixAll.ruff` code actions for a notebook cell ([#16154](https://github.com/astral-sh/ruff/pull/16154))
+- Include document specific debug info for `ruff.printDebugInformation` ([#16215](https://github.com/astral-sh/ruff/pull/16215))
+- Update server to return the debug info as string with `ruff.printDebugInformation` ([#16214](https://github.com/astral-sh/ruff/pull/16214))
+
+### CLI
+
+- Warn on invalid `noqa` even when there are no diagnostics ([#16178](https://github.com/astral-sh/ruff/pull/16178))
+- Better error messages while loading configuration `extend`s ([#15658](https://github.com/astral-sh/ruff/pull/15658))
+
+### Bug fixes
+
+- \[`flake8-comprehensions`\] Handle trailing comma in `C403` fix ([#16110](https://github.com/astral-sh/ruff/pull/16110))
+- \[`flake8-pyi`\] Avoid flagging `custom-typevar-for-self` on metaclass methods (`PYI019`) ([#16141](https://github.com/astral-sh/ruff/pull/16141))
+- \[`pydocstyle`\] Handle arguments with the same names as sections (`D417`) ([#16011](https://github.com/astral-sh/ruff/pull/16011))
+- \[`pylint`\] Correct ordering of arguments in fix for `if-stmt-min-max` (`PLR1730`) ([#16080](https://github.com/astral-sh/ruff/pull/16080))
+- \[`pylint`\] Do not offer fix for raw strings (`PLE251`) ([#16132](https://github.com/astral-sh/ruff/pull/16132))
+- \[`pyupgrade`\] Do not upgrade functional `TypedDicts` with private field names to the class-based syntax (`UP013`) ([#16219](https://github.com/astral-sh/ruff/pull/16219))
+- \[`pyupgrade`\] Handle micro version numbers correctly (`UP036`) ([#16091](https://github.com/astral-sh/ruff/pull/16091))
+- \[`pyupgrade`\] Unwrap unary expressions correctly (`UP018`) ([#15919](https://github.com/astral-sh/ruff/pull/15919))
+- \[`refurb`\] Correctly handle lengths of literal strings in `slice-to-remove-prefix-or-suffix` (`FURB188`) ([#16237](https://github.com/astral-sh/ruff/pull/16237))
+- \[`ruff`\] Skip `RUF001` diagnostics when visiting string type definitions ([#16122](https://github.com/astral-sh/ruff/pull/16122))
+
+### Documentation
+
+- Add FAQ entry for `source.*` code actions in Notebook ([#16212](https://github.com/astral-sh/ruff/pull/16212))
+- Add `SECURITY.md` ([#16224](https://github.com/astral-sh/ruff/pull/16224))
+
+## 0.9.6
+
+### Preview features
+
+- \[`airflow`\] Add `external_task.{ExternalTaskMarker, ExternalTaskSensor}` for `AIR302` ([#16014](https://github.com/astral-sh/ruff/pull/16014))
+- \[`flake8-builtins`\] Make strict module name comparison optional (`A005`) ([#15951](https://github.com/astral-sh/ruff/pull/15951))
+- \[`flake8-pyi`\] Extend fix to Python \<= 3.9 for `redundant-none-literal` (`PYI061`) ([#16044](https://github.com/astral-sh/ruff/pull/16044))
+- \[`pylint`\] Also report when the object isn't a literal (`PLE1310`) ([#15985](https://github.com/astral-sh/ruff/pull/15985))
+- \[`ruff`\] Implement `indented-form-feed` (`RUF054`) ([#16049](https://github.com/astral-sh/ruff/pull/16049))
+- \[`ruff`\] Skip type definitions for `missing-f-string-syntax` (`RUF027`) ([#16054](https://github.com/astral-sh/ruff/pull/16054))
+
+### Rule changes
+
+- \[`flake8-annotations`\] Correct syntax for `typing.Union` in suggested return type fixes for `ANN20x` rules ([#16025](https://github.com/astral-sh/ruff/pull/16025))
+- \[`flake8-builtins`\] Match upstream module name comparison (`A005`) ([#16006](https://github.com/astral-sh/ruff/pull/16006))
+- \[`flake8-comprehensions`\] Detect overshadowed `list`/`set`/`dict`, ignore variadics and named expressions (`C417`) ([#15955](https://github.com/astral-sh/ruff/pull/15955))
+- \[`flake8-pie`\] Remove following comma correctly when the unpacked dictionary is empty (`PIE800`) ([#16008](https://github.com/astral-sh/ruff/pull/16008))
+- \[`flake8-simplify`\] Only trigger `SIM401` on known dictionaries ([#15995](https://github.com/astral-sh/ruff/pull/15995))
+- \[`pylint`\] Do not report calls when object type and argument type mismatch, remove custom escape handling logic (`PLE1310`) ([#15984](https://github.com/astral-sh/ruff/pull/15984))
+- \[`pyupgrade`\] Comments within parenthesized value ranges should not affect applicability (`UP040`) ([#16027](https://github.com/astral-sh/ruff/pull/16027))
+- \[`pyupgrade`\] Don't introduce invalid syntax when upgrading old-style type aliases with parenthesized multiline values (`UP040`) ([#16026](https://github.com/astral-sh/ruff/pull/16026))
+- \[`pyupgrade`\] Ensure we do not rename two type parameters to the same name (`UP049`) ([#16038](https://github.com/astral-sh/ruff/pull/16038))
+- \[`pyupgrade`\] \[`ruff`\] Don't apply renamings if the new name is shadowed in a scope of one of the references to the binding (`UP049`, `RUF052`) ([#16032](https://github.com/astral-sh/ruff/pull/16032))
+- \[`ruff`\] Update `RUF009` to behave similar to `B008` and ignore attributes with immutable types ([#16048](https://github.com/astral-sh/ruff/pull/16048))
+
+### Server
+
+- Root exclusions in the server to project root ([#16043](https://github.com/astral-sh/ruff/pull/16043))
+
+### Bug fixes
+
+- \[`flake8-datetime`\] Ignore `.replace()` calls while looking for `.astimezone` ([#16050](https://github.com/astral-sh/ruff/pull/16050))
+- \[`flake8-type-checking`\] Avoid `TC004` false positive where the runtime definition is provided by `__getattr__` ([#16052](https://github.com/astral-sh/ruff/pull/16052))
+
+### Documentation
+
+- Improve `ruff-lsp` migration document ([#16072](https://github.com/astral-sh/ruff/pull/16072))
+- Undeprecate `ruff.nativeServer` ([#16039](https://github.com/astral-sh/ruff/pull/16039))
+
 ## 0.9.5
 
 ### Preview features

CONTRIBUTING.md

@@ -526,7 +526,7 @@ cargo benchmark
 #### Benchmark-driven Development
 
 Ruff uses [Criterion.rs](https://bheisler.github.io/criterion.rs/book/) for benchmarks. You can use
-`--save-baseline=<name>` to store an initial baseline benchmark (e.g. on `main`) and then use
+`--save-baseline=<name>` to store an initial baseline benchmark (e.g., on `main`) and then use
 `--benchmark=<name>` to compare against that benchmark. Criterion will print a message telling you
 if the benchmark improved/regressed compared to that baseline.
 
@@ -678,9 +678,9 @@ utils with it:
 23 Newline 24
 ```
 
-- `cargo dev print-cst <file>`: Print the CST of a python file using
+- `cargo dev print-cst <file>`: Print the CST of a Python file using
     [LibCST](https://github.com/Instagram/LibCST), which is used in addition to the RustPython parser
-    in Ruff. E.g. for `if True: pass # comment` everything including the whitespace is represented:
+    in Ruff. For example, for `if True: pass # comment`, everything, including the whitespace, is represented:
 
 ```text
 Module {

Cargo.toml

@@ -4,7 +4,7 @@ resolver = "2"
 
 [workspace.package]
 edition = "2021"
-rust-version = "1.80"
+rust-version = "1.83"
 homepage = "https://docs.astral.sh/ruff"
 documentation = "https://docs.astral.sh/ruff"
 repository = "https://github.com/astral-sh/ruff"
@@ -123,7 +123,7 @@ rayon = { version = "1.10.0" }
 regex = { version = "1.10.2" }
 rustc-hash = { version = "2.0.0" }
 # When updating salsa, make sure to also update the revision in `fuzz/Cargo.toml`
-salsa = { git = "https://github.com/salsa-rs/salsa.git", rev = "88a1d7774d78f048fbd77d40abca9ebd729fd1f0" }
+salsa = { git = "https://github.com/salsa-rs/salsa.git", rev = "99be5d9917c3dd88e19735a82ef6bf39ba84bd7e" }
 schemars = { version = "0.8.16" }
 seahash = { version = "4.1.0" }
 serde = { version = "1.0.197", features = ["derive"] }
@@ -143,8 +143,8 @@ snapbox = { version = "0.6.0", features = [
     "examples",
 ] }
 static_assertions = "1.1.0"
-strum = { version = "0.26.0", features = ["strum_macros"] }
-strum_macros = { version = "0.26.0" }
+strum = { version = "0.27.0", features = ["strum_macros"] }
+strum_macros = { version = "0.27.0" }
 syn = { version = "2.0.55" }
 tempfile = { version = "3.9.0" }
 test-case = { version = "3.3.1" }

README.md

@@ -149,8 +149,8 @@ curl -LsSf https://astral.sh/ruff/install.sh | sh
 powershell -c "irm https://astral.sh/ruff/install.ps1 | iex"
 
 # For a specific version.
-curl -LsSf https://astral.sh/ruff/0.9.5/install.sh | sh
-powershell -c "irm https://astral.sh/ruff/0.9.5/install.ps1 | iex"
+curl -LsSf https://astral.sh/ruff/0.9.10/install.sh | sh
+powershell -c "irm https://astral.sh/ruff/0.9.10/install.ps1 | iex"
 ```
 
 You can also install Ruff via [Homebrew](https://formulae.brew.sh/formula/ruff), [Conda](https://anaconda.org/conda-forge/ruff),
@@ -183,7 +183,7 @@ Ruff can also be used as a [pre-commit](https://pre-commit.com/) hook via [`ruff
 ```yaml
 - repo: https://github.com/astral-sh/ruff-pre-commit
   # Ruff version.
-  rev: v0.9.5
+  rev: v0.9.10
   hooks:
     # Run the linter.
     - id: ruff

SECURITY.md

@@ -0,0 +1,15 @@
+# Security policy
+
+## Reporting a vulnerability
+
+If you have found a possible vulnerability, please email `security at astral dot sh`.
+
+## Bug bounties
+
+While we sincerely appreciate and encourage reports of suspected security problems, please note that
+Astral does not currently run any bug bounty programs.
+
+## Vulnerability disclosures
+
+Critical vulnerabilities will be disclosed via GitHub's
+[security advisory](https://github.com/astral-sh/ruff/security) system.

_typos.toml

@@ -23,6 +23,10 @@ extend-ignore-re = [
     # Line ignore with trailing "spellchecker:disable-line"
     "(?Rm)^.*#\\s*spellchecker:disable-line$",
     "LICENSEs",
+    # Various third party dependencies uses `typ` as struct field names (e.g., lsp_types::LogMessageParams)
+    "typ",
+    # TODO: Remove this once the `TYP` redirects are removed from `rule_redirects.rs`
+    "TYP",
 ]
 
 [default.extend-identifiers]

crates/red_knot/Cargo.toml

@@ -16,8 +16,10 @@ red_knot_python_semantic = { workspace = true }
 red_knot_project = { workspace = true, features = ["zstd"] }
 red_knot_server = { workspace = true }
 ruff_db = { workspace = true, features = ["os", "cache"] }
+ruff_python_ast = { workspace = true }
 
 anyhow = { workspace = true }
+argfile = { workspace = true }
 chrono = { workspace = true }
 clap = { workspace = true, features = ["wrap_help"] }
 colored = { workspace = true }
@@ -30,6 +32,7 @@ tracing = { workspace = true, features = ["release_max_level_debug"] }
 tracing-subscriber = { workspace = true, features = ["env-filter", "fmt"] }
 tracing-flame = { workspace = true }
 tracing-tree = { workspace = true }
+wild = { workspace = true }
 
 [dev-dependencies]
 ruff_db = { workspace = true, features = ["testing"] }

crates/red_knot/README.md

@@ -0,0 +1,25 @@
+# Red Knot
+
+Red Knot is an extremely fast type checker.
+Currently, it is a work-in-progress and not ready for user testing.
+
+Red Knot is designed to prioritize good type inference, even in unannotated code,
+and aims to avoid false positives.
+
+While Red Knot will produce similar results to mypy and pyright on many codebases,
+100% compatibility with these tools is a non-goal.
+On some codebases, Red Knot's design decisions lead to different outcomes
+than you would get from running one of these more established tools.
+
+## Contributing
+
+Core type checking tests are written as Markdown code blocks.
+They can be found in [`red_knot_python_semantic/resources/mdtest`][resources-mdtest].
+See [`red_knot_test/README.md`][mdtest-readme] for more information
+on the test framework itself.
+
+The list of open issues can be found [here][open-issues].
+
+[mdtest-readme]: ../red_knot_test/README.md
+[open-issues]: https://github.com/astral-sh/ruff/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20label%3Ared-knot
+[resources-mdtest]: ../red_knot_python_semantic/resources/mdtest

crates/red_knot/src/args.rs

@@ -1,7 +1,7 @@
 use crate::logging::Verbosity;
 use crate::python_version::PythonVersion;
 use clap::{ArgAction, ArgMatches, Error, Parser};
-use red_knot_project::metadata::options::{EnvironmentOptions, Options};
+use red_knot_project::metadata::options::{EnvironmentOptions, Options, TerminalOptions};
 use red_knot_project::metadata::value::{RangedValue, RelativePathBuf};
 use red_knot_python_semantic::lint;
 use ruff_db::system::SystemPathBuf;
@@ -32,6 +32,13 @@ pub(crate) enum Command {
 
 #[derive(Debug, Parser)]
 pub(crate) struct CheckCommand {
+    /// List of files or directories to check.
+    #[clap(
+        help = "List of files or directories to check [default: the project root]",
+        value_name = "PATH"
+    )]
+    pub paths: Vec<SystemPathBuf>,
+
     /// Run the command within the given project directory.
     ///
     /// All `pyproject.toml` files will be discovered by walking up the directory tree from the given project directory,
@@ -41,12 +48,14 @@ pub(crate) struct CheckCommand {
     #[arg(long, value_name = "PROJECT")]
     pub(crate) project: Option<SystemPathBuf>,
 
-    /// Path to the virtual environment the project uses.
+    /// Path to the Python installation from which Red Knot resolves type information and third-party dependencies.
     ///
-    /// If provided, red-knot will use the `site-packages` directory of this virtual environment
-    /// to resolve type information for the project's third-party dependencies.
+    /// Red Knot will search in the path's `site-packages` directories for type information and
+    /// third-party imports.
+    ///
+    /// This option is commonly used to specify the path to a virtual environment.
     #[arg(long, value_name = "PATH")]
-    pub(crate) venv_path: Option<SystemPathBuf>,
+    pub(crate) python: Option<SystemPathBuf>,
 
     /// Custom directory to use for stdlib typeshed stubs.
     #[arg(long, value_name = "PATH", alias = "custom-typeshed-dir")]
@@ -67,14 +76,14 @@ pub(crate) struct CheckCommand {
     pub(crate) rules: RulesArg,
 
     /// Use exit code 1 if there are any warning-level diagnostics.
-    #[arg(long, conflicts_with = "exit_zero")]
-    pub(crate) error_on_warning: bool,
+    #[arg(long, conflicts_with = "exit_zero", default_missing_value = "true", num_args=0..1)]
+    pub(crate) error_on_warning: Option<bool>,
 
     /// Always use exit code 0, even when there are error-level diagnostics.
     #[arg(long)]
     pub(crate) exit_zero: bool,
 
-    /// Run in watch mode by re-running whenever files change.
+    /// Watch files for changes and recheck files related to the changed files.
     #[arg(long, short = 'W')]
     pub(crate) watch: bool,
 }
@@ -97,7 +106,7 @@ impl CheckCommand {
                 python_version: self
                     .python_version
                     .map(|version| RangedValue::cli(version.into())),
-                venv_path: self.venv_path.map(RelativePathBuf::cli),
+                python: self.python.map(RelativePathBuf::cli),
                 typeshed: self.typeshed.map(RelativePathBuf::cli),
                 extra_paths: self.extra_search_path.map(|extra_search_paths| {
                     extra_search_paths
@@ -107,6 +116,9 @@ impl CheckCommand {
                 }),
                 ..EnvironmentOptions::default()
             }),
+            terminal: Some(TerminalOptions {
+                error_on_warning: self.error_on_warning,
+            }),
             rules,
             ..Default::default()
         }

crates/red_knot/src/main.rs

@@ -1,4 +1,4 @@
-use std::io::{self, BufWriter, Write};
+use std::io::{self, stdout, BufWriter, Write};
 use std::process::{ExitCode, Termination};
 
 use anyhow::Result;
@@ -11,18 +11,17 @@ use clap::Parser;
 use colored::Colorize;
 use crossbeam::channel as crossbeam_channel;
 use red_knot_project::metadata::options::Options;
-use red_knot_project::watch;
 use red_knot_project::watch::ProjectWatcher;
+use red_knot_project::{watch, Db};
 use red_knot_project::{ProjectDatabase, ProjectMetadata};
 use red_knot_server::run_server;
-use ruff_db::diagnostic::{Diagnostic, Severity};
-use ruff_db::system::{OsSystem, System, SystemPath, SystemPathBuf};
+use ruff_db::diagnostic::{DisplayDiagnosticConfig, OldDiagnosticTrait, Severity};
+use ruff_db::system::{OsSystem, SystemPath, SystemPathBuf};
 use salsa::plumbing::ZalsaDatabase;
 
 mod args;
 mod logging;
 mod python_version;
-mod verbosity;
 mod version;
 
 #[allow(clippy::print_stdout, clippy::unnecessary_wraps, clippy::print_stderr)]
@@ -40,6 +39,15 @@ pub fn main() -> ExitStatus {
         // the configuration it is help to chain errors ("resolving configuration failed" ->
         // "failed to read file: subdir/pyproject.toml")
         for cause in error.chain() {
+            // Exit "gracefully" on broken pipe errors.
+            //
+            // See: https://github.com/BurntSushi/ripgrep/blob/bf63fe8f258afc09bae6caa48f0ae35eaf115005/crates/core/main.rs#L47C1-L61C14
+            if let Some(ioerr) = cause.downcast_ref::<io::Error>() {
+                if ioerr.kind() == io::ErrorKind::BrokenPipe {
+                    return ExitStatus::Success;
+                }
+            }
+
             writeln!(stderr, "  {} {cause}", "Cause:".bold()).ok();
         }
 
@@ -48,7 +56,10 @@ pub fn main() -> ExitStatus {
 }
 
 fn run() -> anyhow::Result<ExitStatus> {
-    let args = Args::parse_from(std::env::args());
+    let args = wild::args_os();
+    let args = argfile::expand_args_from(args, argfile::parse_fromfile, argfile::PREFIX)
+        .context("Failed to read CLI arguments from file")?;
+    let args = Args::parse_from(args);
 
     match args.command {
         Command::Server => run_server().map(|()| ExitStatus::Success),
@@ -70,7 +81,7 @@ fn run_check(args: CheckCommand) -> anyhow::Result<ExitStatus> {
     let _guard = setup_tracing(verbosity)?;
 
     // The base path to which all CLI arguments are relative to.
-    let cli_base_path = {
+    let cwd = {
         let cwd = std::env::current_dir().context("Failed to get the current working directory")?;
         SystemPathBuf::from_path_buf(cwd)
             .map_err(|path| {
@@ -81,35 +92,43 @@ fn run_check(args: CheckCommand) -> anyhow::Result<ExitStatus> {
             })?
     };
 
-    let cwd = args
+    let project_path = args
         .project
         .as_ref()
-        .map(|cwd| {
-            if cwd.as_std_path().is_dir() {
-                Ok(SystemPath::absolute(cwd, &cli_base_path))
+        .map(|project| {
+            if project.as_std_path().is_dir() {
+                Ok(SystemPath::absolute(project, &cwd))
             } else {
-                Err(anyhow!("Provided project path `{cwd}` is not a directory"))
+                Err(anyhow!(
+                    "Provided project path `{project}` is not a directory"
+                ))
             }
         })
         .transpose()?
-        .unwrap_or_else(|| cli_base_path.clone());
+        .unwrap_or_else(|| cwd.clone());
+
+    let check_paths: Vec<_> = args
+        .paths
+        .iter()
+        .map(|path| SystemPath::absolute(path, &cwd))
+        .collect();
 
     let system = OsSystem::new(cwd);
     let watch = args.watch;
     let exit_zero = args.exit_zero;
-    let min_error_severity = if args.error_on_warning {
-        Severity::Warning
-    } else {
-        Severity::Error
-    };
 
     let cli_options = args.into_options();
-    let mut workspace_metadata = ProjectMetadata::discover(system.current_directory(), &system)?;
-    workspace_metadata.apply_cli_options(cli_options.clone());
+    let mut project_metadata = ProjectMetadata::discover(&project_path, &system)?;
+    project_metadata.apply_cli_options(cli_options.clone());
+    project_metadata.apply_configuration_files(&system)?;
 
-    let mut db = ProjectDatabase::new(workspace_metadata, system)?;
+    let mut db = ProjectDatabase::new(project_metadata, system)?;
 
-    let (main_loop, main_loop_cancellation_token) = MainLoop::new(cli_options, min_error_severity);
+    if !check_paths.is_empty() {
+        db.project().set_included_paths(&mut db, check_paths);
+    }
+
+    let (main_loop, main_loop_cancellation_token) = MainLoop::new(cli_options);
 
     // Listen to Ctrl+C and abort the watch mode.
     let main_loop_cancellation_token = Mutex::new(Some(main_loop_cancellation_token));
@@ -124,7 +143,7 @@ fn run_check(args: CheckCommand) -> anyhow::Result<ExitStatus> {
     let exit_status = if watch {
         main_loop.watch(&mut db)?
     } else {
-        main_loop.run(&mut db)
+        main_loop.run(&mut db)?
     };
 
     tracing::trace!("Counts for entire CLI run:\n{}", countme::get_all());
@@ -167,18 +186,10 @@ struct MainLoop {
     watcher: Option<ProjectWatcher>,
 
     cli_options: Options,
-
-    /// The minimum severity to consider an error when deciding the exit status.
-    ///
-    /// TODO(micha): Get from the terminal settings.
-    min_error_severity: Severity,
 }
 
 impl MainLoop {
-    fn new(
-        cli_options: Options,
-        min_error_severity: Severity,
-    ) -> (Self, MainLoopCancellationToken) {
+    fn new(cli_options: Options) -> (Self, MainLoopCancellationToken) {
         let (sender, receiver) = crossbeam_channel::bounded(10);
 
         (
@@ -187,13 +198,12 @@ impl MainLoop {
                 receiver,
                 watcher: None,
                 cli_options,
-                min_error_severity,
             },
             MainLoopCancellationToken { sender },
         )
     }
 
-    fn watch(mut self, db: &mut ProjectDatabase) -> anyhow::Result<ExitStatus> {
+    fn watch(mut self, db: &mut ProjectDatabase) -> Result<ExitStatus> {
         tracing::debug!("Starting watch mode");
         let sender = self.sender.clone();
         let watcher = watch::directory_watcher(move |event| {
@@ -202,12 +212,12 @@ impl MainLoop {
 
         self.watcher = Some(ProjectWatcher::new(watcher, db));
 
-        self.run(db);
+        self.run(db)?;
 
         Ok(ExitStatus::Success)
     }
 
-    fn run(mut self, db: &mut ProjectDatabase) -> ExitStatus {
+    fn run(mut self, db: &mut ProjectDatabase) -> Result<ExitStatus> {
         self.sender.send(MainLoopMessage::CheckWorkspace).unwrap();
 
         let result = self.main_loop(db);
@@ -217,7 +227,7 @@ impl MainLoop {
         result
     }
 
-    fn main_loop(&mut self, db: &mut ProjectDatabase) -> ExitStatus {
+    fn main_loop(&mut self, db: &mut ProjectDatabase) -> Result<ExitStatus> {
         // Schedule the first check.
         tracing::debug!("Starting main loop");
 
@@ -245,14 +255,53 @@ impl MainLoop {
                     result,
                     revision: check_revision,
                 } => {
-                    let failed = result
-                        .iter()
-                        .any(|diagnostic| diagnostic.severity() >= self.min_error_severity);
+                    let display_config = DisplayDiagnosticConfig::default()
+                        .color(colored::control::SHOULD_COLORIZE.should_colorize());
+
+                    let min_error_severity =
+                        if db.project().settings(db).terminal().error_on_warning {
+                            Severity::Warning
+                        } else {
+                            Severity::Error
+                        };
 
                     if check_revision == revision {
-                        #[allow(clippy::print_stdout)]
-                        for diagnostic in result {
-                            println!("{}", diagnostic.display(db));
+                        if db.project().files(db).is_empty() {
+                            tracing::warn!("No python files found under the given path(s)");
+                        }
+
+                        let mut stdout = stdout().lock();
+
+                        if result.is_empty() {
+                            writeln!(stdout, "All checks passed!")?;
+
+                            if self.watcher.is_none() {
+                                return Ok(ExitStatus::Success);
+                            }
+                        } else {
+                            let mut failed = false;
+                            let diagnostics_count = result.len();
+
+                            for diagnostic in result {
+                                writeln!(stdout, "{}", diagnostic.display(db, &display_config))?;
+
+                                failed |= diagnostic.severity() >= min_error_severity;
+                            }
+
+                            writeln!(
+                                stdout,
+                                "Found {} diagnostic{}",
+                                diagnostics_count,
+                                if diagnostics_count > 1 { "s" } else { "" }
+                            )?;
+
+                            if self.watcher.is_none() {
+                                return Ok(if failed {
+                                    ExitStatus::Failure
+                                } else {
+                                    ExitStatus::Success
+                                });
+                            }
                         }
                     } else {
                         tracing::debug!(
@@ -260,14 +309,6 @@ impl MainLoop {
                         );
                     }
 
-                    if self.watcher.is_none() {
-                        return if failed {
-                            ExitStatus::Failure
-                        } else {
-                            ExitStatus::Success
-                        };
-                    }
-
                     tracing::trace!("Counts after last check:\n{}", countme::get_all());
                 }
 
@@ -285,14 +326,14 @@ impl MainLoop {
                     // TODO: Don't use Salsa internal APIs
                     //  [Zulip-Thread](https://salsa.zulipchat.com/#narrow/stream/333573-salsa-3.2E0/topic/Expose.20an.20API.20to.20cancel.20other.20queries)
                     let _ = db.zalsa_mut();
-                    return ExitStatus::Success;
+                    return Ok(ExitStatus::Success);
                 }
             }
 
             tracing::debug!("Waiting for next main loop message.");
         }
 
-        ExitStatus::Success
+        Ok(ExitStatus::Success)
     }
 }
 
@@ -312,7 +353,8 @@ impl MainLoopCancellationToken {
 enum MainLoopMessage {
     CheckWorkspace,
     CheckCompleted {
-        result: Vec<Box<dyn Diagnostic>>,
+        /// The diagnostics that were found during the check.
+        result: Vec<Box<dyn OldDiagnosticTrait>>,
         revision: u64,
     },
     ApplyChanges(Vec<watch::ChangeEvent>),

crates/red_knot/src/python_version.rs

@@ -40,7 +40,7 @@ impl std::fmt::Display for PythonVersion {
     }
 }
 
-impl From<PythonVersion> for red_knot_python_semantic::PythonVersion {
+impl From<PythonVersion> for ruff_python_ast::PythonVersion {
     fn from(value: PythonVersion) -> Self {
         match value {
             PythonVersion::Py37 => Self::PY37,
@@ -61,8 +61,8 @@ mod tests {
     #[test]
     fn same_default_as_python_version() {
         assert_eq!(
-            red_knot_python_semantic::PythonVersion::from(PythonVersion::default()),
-            red_knot_python_semantic::PythonVersion::default()
+            ruff_python_ast::PythonVersion::from(PythonVersion::default()),
+            ruff_python_ast::PythonVersion::default()
         );
     }
 }

crates/red_knot/src/verbosity.rs

@@ -1 +0,0 @@
-

crates/red_knot/tests/cli.rs

@@ -28,7 +28,7 @@ fn config_override() -> anyhow::Result<()> {
         ),
     ])?;
 
-    assert_cmd_snapshot!(case.command(), @r###"
+    assert_cmd_snapshot!(case.command(), @r"
     success: false
     exit_code: 1
     ----- stdout -----
@@ -40,16 +40,18 @@ fn config_override() -> anyhow::Result<()> {
       |       ^^^^^^^^^^^^ Type `<module 'sys'>` has no attribute `last_exc`
       |
 
+    Found 1 diagnostic
 
     ----- stderr -----
-    "###);
+    ");
 
     assert_cmd_snapshot!(case.command().arg("--python-version").arg("3.12"), @r"
-        success: true
-        exit_code: 0
-        ----- stdout -----
+    success: true
+    exit_code: 0
+    ----- stdout -----
+    All checks passed!
 
-        ----- stderr -----
+    ----- stderr -----
     ");
 
     Ok(())
@@ -98,7 +100,7 @@ fn cli_arguments_are_relative_to_the_current_directory() -> anyhow::Result<()> {
     ])?;
 
     // Make sure that the CLI fails when the `libs` directory is not in the search path.
-    assert_cmd_snapshot!(case.command().current_dir(case.project_dir().join("child")), @r###"
+    assert_cmd_snapshot!(case.command().current_dir(case.root().join("child")), @r"
     success: false
     exit_code: 1
     ----- stdout -----
@@ -111,16 +113,18 @@ fn cli_arguments_are_relative_to_the_current_directory() -> anyhow::Result<()> {
     4 | stat = add(10, 15)
       |
 
+    Found 1 diagnostic
 
     ----- stderr -----
-    "###);
+    ");
 
-    assert_cmd_snapshot!(case.command().current_dir(case.project_dir().join("child")).arg("--extra-search-path").arg("../libs"), @r"
-        success: true
-        exit_code: 0
-        ----- stdout -----
+    assert_cmd_snapshot!(case.command().current_dir(case.root().join("child")).arg("--extra-search-path").arg("../libs"), @r"
+    success: true
+    exit_code: 0
+    ----- stdout -----
+    All checks passed!
 
-        ----- stderr -----
+    ----- stderr -----
     ");
 
     Ok(())
@@ -167,12 +171,13 @@ fn paths_in_configuration_files_are_relative_to_the_project_root() -> anyhow::Re
         ),
     ])?;
 
-    assert_cmd_snapshot!(case.command().current_dir(case.project_dir().join("child")), @r"
-        success: true
-        exit_code: 0
-        ----- stdout -----
+    assert_cmd_snapshot!(case.command().current_dir(case.root().join("child")), @r"
+    success: true
+    exit_code: 0
+    ----- stdout -----
+    All checks passed!
 
-        ----- stderr -----
+    ----- stderr -----
     ");
 
     Ok(())
@@ -195,7 +200,7 @@ fn configuration_rule_severity() -> anyhow::Result<()> {
 
     // Assert that there's a possibly unresolved reference diagnostic
     // and that division-by-zero has a severity of error by default.
-    assert_cmd_snapshot!(case.command(), @r###"
+    assert_cmd_snapshot!(case.command(), @r"
     success: false
     exit_code: 1
     ----- stdout -----
@@ -217,9 +222,10 @@ fn configuration_rule_severity() -> anyhow::Result<()> {
       |       - Name `x` used when possibly not defined
       |
 
+    Found 2 diagnostics
 
     ----- stderr -----
-    "###);
+    ");
 
     case.write_file(
         "pyproject.toml",
@@ -230,7 +236,7 @@ fn configuration_rule_severity() -> anyhow::Result<()> {
     "#,
     )?;
 
-    assert_cmd_snapshot!(case.command(), @r###"
+    assert_cmd_snapshot!(case.command(), @r"
     success: true
     exit_code: 0
     ----- stdout -----
@@ -243,9 +249,10 @@ fn configuration_rule_severity() -> anyhow::Result<()> {
     4 | for a in range(0, y):
       |
 
+    Found 1 diagnostic
 
     ----- stderr -----
-    "###);
+    ");
 
     Ok(())
 }
@@ -269,7 +276,7 @@ fn cli_rule_severity() -> anyhow::Result<()> {
 
     // Assert that there's a possibly unresolved reference diagnostic
     // and that division-by-zero has a severity of error by default.
-    assert_cmd_snapshot!(case.command(), @r###"
+    assert_cmd_snapshot!(case.command(), @r"
     success: false
     exit_code: 1
     ----- stdout -----
@@ -302,9 +309,10 @@ fn cli_rule_severity() -> anyhow::Result<()> {
       |       - Name `x` used when possibly not defined
       |
 
+    Found 3 diagnostics
 
     ----- stderr -----
-    "###);
+    ");
 
     assert_cmd_snapshot!(
         case
@@ -315,7 +323,7 @@ fn cli_rule_severity() -> anyhow::Result<()> {
             .arg("division-by-zero")
             .arg("--warn")
             .arg("unresolved-import"),
-        @r###"
+        @r"
     success: true
     exit_code: 0
     ----- stdout -----
@@ -339,9 +347,10 @@ fn cli_rule_severity() -> anyhow::Result<()> {
     6 | for a in range(0, y):
       |
 
+    Found 2 diagnostics
 
     ----- stderr -----
-    "###
+    "
     );
 
     Ok(())
@@ -365,7 +374,7 @@ fn cli_rule_severity_precedence() -> anyhow::Result<()> {
 
     // Assert that there's a possibly unresolved reference diagnostic
     // and that division-by-zero has a severity of error by default.
-    assert_cmd_snapshot!(case.command(), @r###"
+    assert_cmd_snapshot!(case.command(), @r"
     success: false
     exit_code: 1
     ----- stdout -----
@@ -387,9 +396,10 @@ fn cli_rule_severity_precedence() -> anyhow::Result<()> {
       |       - Name `x` used when possibly not defined
       |
 
+    Found 2 diagnostics
 
     ----- stderr -----
-    "###);
+    ");
 
     assert_cmd_snapshot!(
         case
@@ -401,7 +411,7 @@ fn cli_rule_severity_precedence() -> anyhow::Result<()> {
             // Override the error severity with warning
             .arg("--ignore")
             .arg("possibly-unresolved-reference"),
-        @r###"
+        @r"
     success: true
     exit_code: 0
     ----- stdout -----
@@ -414,9 +424,10 @@ fn cli_rule_severity_precedence() -> anyhow::Result<()> {
     4 | for a in range(0, y):
       |
 
+    Found 1 diagnostic
 
     ----- stderr -----
-    "###
+    "
     );
 
     Ok(())
@@ -436,7 +447,7 @@ fn configuration_unknown_rules() -> anyhow::Result<()> {
         ("test.py", "print(10)"),
     ])?;
 
-    assert_cmd_snapshot!(case.command(), @r###"
+    assert_cmd_snapshot!(case.command(), @r#"
     success: true
     exit_code: 0
     ----- stdout -----
@@ -448,9 +459,10 @@ fn configuration_unknown_rules() -> anyhow::Result<()> {
       | --------------- Unknown lint rule `division-by-zer`
       |
 
+    Found 1 diagnostic
 
     ----- stderr -----
-    "###);
+    "#);
 
     Ok(())
 }
@@ -460,15 +472,16 @@ fn configuration_unknown_rules() -> anyhow::Result<()> {
 fn cli_unknown_rules() -> anyhow::Result<()> {
     let case = TestCase::with_file("test.py", "print(10)")?;
 
-    assert_cmd_snapshot!(case.command().arg("--ignore").arg("division-by-zer"), @r###"
+    assert_cmd_snapshot!(case.command().arg("--ignore").arg("division-by-zer"), @r"
     success: true
     exit_code: 0
     ----- stdout -----
     warning: unknown-rule: Unknown lint rule `division-by-zer`
 
+    Found 1 diagnostic
 
     ----- stderr -----
-    "###);
+    ");
 
     Ok(())
 }
@@ -477,7 +490,7 @@ fn cli_unknown_rules() -> anyhow::Result<()> {
 fn exit_code_only_warnings() -> anyhow::Result<()> {
     let case = TestCase::with_file("test.py", r"print(x)  # [unresolved-reference]")?;
 
-    assert_cmd_snapshot!(case.command(), @r###"
+    assert_cmd_snapshot!(case.command(), @r"
     success: true
     exit_code: 0
     ----- stdout -----
@@ -488,9 +501,10 @@ fn exit_code_only_warnings() -> anyhow::Result<()> {
       |       - Name `x` used when not defined
       |
 
+    Found 1 diagnostic
 
     ----- stderr -----
-    "###);
+    ");
 
     Ok(())
 }
@@ -505,7 +519,7 @@ fn exit_code_only_info() -> anyhow::Result<()> {
         "#,
     )?;
 
-    assert_cmd_snapshot!(case.command(), @r###"
+    assert_cmd_snapshot!(case.command(), @r"
     success: true
     exit_code: 0
     ----- stdout -----
@@ -517,9 +531,10 @@ fn exit_code_only_info() -> anyhow::Result<()> {
       | -------------- info: Revealed type is `Literal[1]`
       |
 
+    Found 1 diagnostic
 
     ----- stderr -----
-    "###);
+    ");
 
     Ok(())
 }
@@ -534,7 +549,7 @@ fn exit_code_only_info_and_error_on_warning_is_true() -> anyhow::Result<()> {
         "#,
     )?;
 
-    assert_cmd_snapshot!(case.command().arg("--error-on-warning"), @r###"
+    assert_cmd_snapshot!(case.command().arg("--error-on-warning"), @r"
     success: true
     exit_code: 0
     ----- stdout -----
@@ -546,9 +561,10 @@ fn exit_code_only_info_and_error_on_warning_is_true() -> anyhow::Result<()> {
       | -------------- info: Revealed type is `Literal[1]`
       |
 
+    Found 1 diagnostic
 
     ----- stderr -----
-    "###);
+    ");
 
     Ok(())
 }
@@ -557,7 +573,7 @@ fn exit_code_only_info_and_error_on_warning_is_true() -> anyhow::Result<()> {
 fn exit_code_no_errors_but_error_on_warning_is_true() -> anyhow::Result<()> {
     let case = TestCase::with_file("test.py", r"print(x)  # [unresolved-reference]")?;
 
-    assert_cmd_snapshot!(case.command().arg("--error-on-warning"), @r###"
+    assert_cmd_snapshot!(case.command().arg("--error-on-warning"), @r"
     success: false
     exit_code: 1
     ----- stdout -----
@@ -568,9 +584,42 @@ fn exit_code_no_errors_but_error_on_warning_is_true() -> anyhow::Result<()> {
       |       - Name `x` used when not defined
       |
 
+    Found 1 diagnostic
 
     ----- stderr -----
-    "###);
+    ");
+
+    Ok(())
+}
+
+#[test]
+fn exit_code_no_errors_but_error_on_warning_is_enabled_in_configuration() -> anyhow::Result<()> {
+    let case = TestCase::with_files([
+        ("test.py", r"print(x)  # [unresolved-reference]"),
+        (
+            "knot.toml",
+            r#"
+            [terminal]
+            error-on-warning = true
+        "#,
+        ),
+    ])?;
+
+    assert_cmd_snapshot!(case.command(), @r"
+    success: false
+    exit_code: 1
+    ----- stdout -----
+    warning: lint:unresolved-reference
+     --> <temp_dir>/test.py:1:7
+      |
+    1 | print(x)  # [unresolved-reference]
+      |       - Name `x` used when not defined
+      |
+
+    Found 1 diagnostic
+
+    ----- stderr -----
+    ");
 
     Ok(())
 }
@@ -585,7 +634,7 @@ fn exit_code_both_warnings_and_errors() -> anyhow::Result<()> {
         "#,
     )?;
 
-    assert_cmd_snapshot!(case.command(), @r###"
+    assert_cmd_snapshot!(case.command(), @r"
     success: false
     exit_code: 1
     ----- stdout -----
@@ -605,9 +654,10 @@ fn exit_code_both_warnings_and_errors() -> anyhow::Result<()> {
       |       ^ Cannot subscript object of type `Literal[4]` with no `__getitem__` method
       |
 
+    Found 2 diagnostics
 
     ----- stderr -----
-    "###);
+    ");
 
     Ok(())
 }
@@ -622,7 +672,7 @@ fn exit_code_both_warnings_and_errors_and_error_on_warning_is_true() -> anyhow::
         "###,
     )?;
 
-    assert_cmd_snapshot!(case.command().arg("--error-on-warning"), @r###"
+    assert_cmd_snapshot!(case.command().arg("--error-on-warning"), @r"
     success: false
     exit_code: 1
     ----- stdout -----
@@ -642,9 +692,10 @@ fn exit_code_both_warnings_and_errors_and_error_on_warning_is_true() -> anyhow::
       |       ^ Cannot subscript object of type `Literal[4]` with no `__getitem__` method
       |
 
+    Found 2 diagnostics
 
     ----- stderr -----
-    "###);
+    ");
 
     Ok(())
 }
@@ -659,7 +710,7 @@ fn exit_code_exit_zero_is_true() -> anyhow::Result<()> {
         "#,
     )?;
 
-    assert_cmd_snapshot!(case.command().arg("--exit-zero"), @r###"
+    assert_cmd_snapshot!(case.command().arg("--exit-zero"), @r"
     success: true
     exit_code: 0
     ----- stdout -----
@@ -679,9 +730,239 @@ fn exit_code_exit_zero_is_true() -> anyhow::Result<()> {
       |       ^ Cannot subscript object of type `Literal[4]` with no `__getitem__` method
       |
 
+    Found 2 diagnostics
 
     ----- stderr -----
-    "###);
+    ");
+
+    Ok(())
+}
+
+#[test]
+fn user_configuration() -> anyhow::Result<()> {
+    let case = TestCase::with_files([
+        (
+            "project/knot.toml",
+            r#"
+            [rules]
+            division-by-zero = "warn"
+            "#,
+        ),
+        (
+            "project/main.py",
+            r#"
+            y = 4 / 0
+
+            for a in range(0, y):
+                x = a
+
+            print(x)
+            "#,
+        ),
+    ])?;
+
+    let config_directory = case.root().join("home/.config");
+    let config_env_var = if cfg!(windows) {
+        "APPDATA"
+    } else {
+        "XDG_CONFIG_HOME"
+    };
+
+    assert_cmd_snapshot!(
+        case.command().current_dir(case.root().join("project")).env(config_env_var, config_directory.as_os_str()),
+        @r"
+    success: true
+    exit_code: 0
+    ----- stdout -----
+    warning: lint:division-by-zero
+     --> <temp_dir>/project/main.py:2:5
+      |
+    2 | y = 4 / 0
+      |     ----- Cannot divide object of type `Literal[4]` by zero
+    3 |
+    4 | for a in range(0, y):
+      |
+
+    warning: lint:possibly-unresolved-reference
+     --> <temp_dir>/project/main.py:7:7
+      |
+    5 |     x = a
+    6 |
+    7 | print(x)
+      |       - Name `x` used when possibly not defined
+      |
+
+    Found 2 diagnostics
+
+    ----- stderr -----
+    "
+    );
+
+    // The user-level configuration promotes `possibly-unresolved-reference` to an error.
+    // Changing the level for `division-by-zero` has no effect, because the project-level configuration
+    // has higher precedence.
+    case.write_file(
+        config_directory.join("knot/knot.toml"),
+        r#"
+        [rules]
+        division-by-zero = "error"
+        possibly-unresolved-reference = "error"
+        "#,
+    )?;
+
+    assert_cmd_snapshot!(
+        case.command().current_dir(case.root().join("project")).env(config_env_var, config_directory.as_os_str()),
+        @r"
+    success: false
+    exit_code: 1
+    ----- stdout -----
+    warning: lint:division-by-zero
+     --> <temp_dir>/project/main.py:2:5
+      |
+    2 | y = 4 / 0
+      |     ----- Cannot divide object of type `Literal[4]` by zero
+    3 |
+    4 | for a in range(0, y):
+      |
+
+    error: lint:possibly-unresolved-reference
+     --> <temp_dir>/project/main.py:7:7
+      |
+    5 |     x = a
+    6 |
+    7 | print(x)
+      |       ^ Name `x` used when possibly not defined
+      |
+
+    Found 2 diagnostics
+
+    ----- stderr -----
+    "
+    );
+
+    Ok(())
+}
+
+#[test]
+fn check_specific_paths() -> anyhow::Result<()> {
+    let case = TestCase::with_files([
+        (
+            "project/main.py",
+            r#"
+            y = 4 / 0  # error: division-by-zero
+            "#,
+        ),
+        (
+            "project/tests/test_main.py",
+            r#"
+            import does_not_exist  # error: unresolved-import
+            "#,
+        ),
+        (
+            "project/other.py",
+            r#"
+            from main2 import z  # error: unresolved-import
+
+            print(z)
+            "#,
+        ),
+    ])?;
+
+    assert_cmd_snapshot!(
+        case.command(),
+        @r"
+    success: false
+    exit_code: 1
+    ----- stdout -----
+    error: lint:unresolved-import
+     --> <temp_dir>/project/tests/test_main.py:2:8
+      |
+    2 | import does_not_exist  # error: unresolved-import
+      |        ^^^^^^^^^^^^^^ Cannot resolve import `does_not_exist`
+      |
+
+    error: lint:division-by-zero
+     --> <temp_dir>/project/main.py:2:5
+      |
+    2 | y = 4 / 0  # error: division-by-zero
+      |     ^^^^^ Cannot divide object of type `Literal[4]` by zero
+      |
+
+    error: lint:unresolved-import
+     --> <temp_dir>/project/other.py:2:6
+      |
+    2 | from main2 import z  # error: unresolved-import
+      |      ^^^^^ Cannot resolve import `main2`
+    3 |
+    4 | print(z)
+      |
+
+    Found 3 diagnostics
+
+    ----- stderr -----
+    "
+    );
+
+    // Now check only the `tests` and `other.py` files.
+    // We should no longer see any diagnostics related to `main.py`.
+    assert_cmd_snapshot!(
+        case.command().arg("project/tests").arg("project/other.py"),
+        @r"
+    success: false
+    exit_code: 1
+    ----- stdout -----
+    error: lint:unresolved-import
+     --> <temp_dir>/project/tests/test_main.py:2:8
+      |
+    2 | import does_not_exist  # error: unresolved-import
+      |        ^^^^^^^^^^^^^^ Cannot resolve import `does_not_exist`
+      |
+
+    error: lint:unresolved-import
+     --> <temp_dir>/project/other.py:2:6
+      |
+    2 | from main2 import z  # error: unresolved-import
+      |      ^^^^^ Cannot resolve import `main2`
+    3 |
+    4 | print(z)
+      |
+
+    Found 2 diagnostics
+
+    ----- stderr -----
+    "
+    );
+
+    Ok(())
+}
+
+#[test]
+fn check_non_existing_path() -> anyhow::Result<()> {
+    let case = TestCase::with_files([])?;
+
+    let mut settings = insta::Settings::clone_current();
+    settings.add_filter(
+        &regex::escape("The system cannot find the path specified. (os error 3)"),
+        "No such file or directory (os error 2)",
+    );
+    let _s = settings.bind_to_scope();
+
+    assert_cmd_snapshot!(
+        case.command().arg("project/main.py").arg("project/tests"),
+        @r"
+    success: false
+    exit_code: 1
+    ----- stdout -----
+    error: io: `<temp_dir>/project/main.py`: No such file or directory (os error 2)
+
+    error: io: `<temp_dir>/project/tests`: No such file or directory (os error 2)
+
+    Found 2 diagnostics
+
+    ----- stderr -----
+    WARN No python files found under the given path(s)
+    "
+    );
 
     Ok(())
 }
@@ -753,7 +1034,7 @@ impl TestCase {
         Ok(())
     }
 
-    fn project_dir(&self) -> &Path {
+    fn root(&self) -> &Path {
         &self.project_dir
     }
 

crates/red_knot/tests/file_watching.rs

@@ -1,5 +1,6 @@
 #![allow(clippy::disallowed_names)]
 
+use std::collections::HashSet;
 use std::io::Write;
 use std::time::{Duration, Instant};
 
@@ -9,11 +10,14 @@ use red_knot_project::metadata::pyproject::{PyProject, Tool};
 use red_knot_project::metadata::value::{RangedValue, RelativePathBuf};
 use red_knot_project::watch::{directory_watcher, ChangeEvent, ProjectWatcher};
 use red_knot_project::{Db, ProjectDatabase, ProjectMetadata};
-use red_knot_python_semantic::{resolve_module, ModuleName, PythonPlatform, PythonVersion};
+use red_knot_python_semantic::{resolve_module, ModuleName, PythonPlatform};
 use ruff_db::files::{system_path_to_file, File, FileError};
 use ruff_db::source::source_text;
-use ruff_db::system::{OsSystem, SystemPath, SystemPathBuf};
+use ruff_db::system::{
+    OsSystem, System, SystemPath, SystemPathBuf, UserConfigDirectoryOverrideGuard,
+};
 use ruff_db::Upcast;
+use ruff_python_ast::PythonVersion;
 
 struct TestCase {
     db: ProjectDatabase,
@@ -190,11 +194,29 @@ impl TestCase {
         Ok(())
     }
 
-    fn collect_project_files(&self) -> Vec<File> {
-        let files = self.db().project().files(self.db());
-        let mut collected: Vec<_> = files.into_iter().collect();
-        collected.sort_unstable_by_key(|file| file.path(self.db()).as_system_path().unwrap());
-        collected
+    #[track_caller]
+    fn assert_indexed_project_files(&self, expected: impl IntoIterator<Item = File>) {
+        let mut expected: HashSet<_> = expected.into_iter().collect();
+
+        let actual = self.db().project().files(self.db());
+        for file in &actual {
+            assert!(
+                expected.remove(&file),
+                "Indexed project files contains '{}' which was not expected.",
+                file.path(self.db())
+            );
+        }
+
+        if !expected.is_empty() {
+            let paths: Vec<_> = expected
+                .iter()
+                .map(|file| file.path(self.db()).as_str())
+                .collect();
+            panic!(
+                "Indexed project files are missing the following files: {:?}",
+                paths.join(", ")
+            );
+        }
     }
 
     fn system_file(&self, path: impl AsRef<SystemPath>) -> Result<File, FileError> {
@@ -219,58 +241,108 @@ where
     }
 }
 
-trait SetupFiles {
-    fn setup(self, root_path: &SystemPath, project_path: &SystemPath) -> anyhow::Result<()>;
+trait Setup {
+    fn setup(self, context: &mut SetupContext) -> anyhow::Result<()>;
 }
 
-impl<const N: usize, P> SetupFiles for [(P, &'static str); N]
+struct SetupContext<'a> {
+    system: &'a OsSystem,
+    root_path: &'a SystemPath,
+    options: Option<Options>,
+    included_paths: Option<Vec<SystemPathBuf>>,
+}
+
+impl<'a> SetupContext<'a> {
+    fn system(&self) -> &'a OsSystem {
+        self.system
+    }
+
+    fn join_project_path(&self, relative: impl AsRef<SystemPath>) -> SystemPathBuf {
+        self.project_path().join(relative)
+    }
+
+    fn project_path(&self) -> &SystemPath {
+        self.system.current_directory()
+    }
+
+    fn root_path(&self) -> &'a SystemPath {
+        self.root_path
+    }
+
+    fn join_root_path(&self, relative: impl AsRef<SystemPath>) -> SystemPathBuf {
+        self.root_path().join(relative)
+    }
+
+    fn write_project_file(
+        &self,
+        relative_path: impl AsRef<SystemPath>,
+        content: &str,
+    ) -> anyhow::Result<()> {
+        let relative_path = relative_path.as_ref();
+        let absolute_path = self.join_project_path(relative_path);
+        Self::write_file_impl(absolute_path, content)
+    }
+
+    fn write_file(
+        &self,
+        relative_path: impl AsRef<SystemPath>,
+        content: &str,
+    ) -> anyhow::Result<()> {
+        let relative_path = relative_path.as_ref();
+        let absolute_path = self.join_root_path(relative_path);
+        Self::write_file_impl(absolute_path, content)
+    }
+
+    fn write_file_impl(path: impl AsRef<SystemPath>, content: &str) -> anyhow::Result<()> {
+        let path = path.as_ref();
+        if let Some(parent) = path.parent() {
+            std::fs::create_dir_all(parent)
+                .with_context(|| format!("Failed to create parent directory for file `{path}`"))?;
+        }
+
+        let mut file = std::fs::File::create(path.as_std_path())
+            .with_context(|| format!("Failed to open file `{path}`"))?;
+        file.write_all(content.as_bytes())
+            .with_context(|| format!("Failed to write to file `{path}`"))?;
+        file.sync_data()?;
+
+        Ok(())
+    }
+
+    fn set_options(&mut self, options: Options) {
+        self.options = Some(options);
+    }
+
+    fn set_included_paths(&mut self, paths: Vec<SystemPathBuf>) {
+        self.included_paths = Some(paths);
+    }
+}
+
+impl<const N: usize, P> Setup for [(P, &'static str); N]
 where
     P: AsRef<SystemPath>,
 {
-    fn setup(self, _root_path: &SystemPath, project_path: &SystemPath) -> anyhow::Result<()> {
+    fn setup(self, context: &mut SetupContext) -> anyhow::Result<()> {
         for (relative_path, content) in self {
-            let relative_path = relative_path.as_ref();
-            let absolute_path = project_path.join(relative_path);
-            if let Some(parent) = absolute_path.parent() {
-                std::fs::create_dir_all(parent).with_context(|| {
-                    format!("Failed to create parent directory for file `{relative_path}`")
-                })?;
-            }
-
-            let mut file = std::fs::File::create(absolute_path.as_std_path())
-                .with_context(|| format!("Failed to open file `{relative_path}`"))?;
-            file.write_all(content.as_bytes())
-                .with_context(|| format!("Failed to write to file `{relative_path}`"))?;
-            file.sync_data()?;
+            context.write_project_file(relative_path, content)?;
         }
 
         Ok(())
     }
 }
 
-impl<F> SetupFiles for F
+impl<F> Setup for F
 where
-    F: FnOnce(&SystemPath, &SystemPath) -> anyhow::Result<()>,
+    F: FnOnce(&mut SetupContext) -> anyhow::Result<()>,
 {
-    fn setup(self, root_path: &SystemPath, project_path: &SystemPath) -> anyhow::Result<()> {
-        self(root_path, project_path)
+    fn setup(self, context: &mut SetupContext) -> anyhow::Result<()> {
+        self(context)
     }
 }
 
 fn setup<F>(setup_files: F) -> anyhow::Result<TestCase>
 where
-    F: SetupFiles,
-{
-    setup_with_options(setup_files, |_root, _project_path| None)
-}
-
-// TODO: Replace with configuration?
-fn setup_with_options<F>(
-    setup_files: F,
-    create_options: impl FnOnce(&SystemPath, &SystemPath) -> Option<Options>,
-) -> anyhow::Result<TestCase>
-where
-    F: SetupFiles,
+    F: Setup,
 {
     let temp_dir = tempfile::tempdir()?;
 
@@ -295,13 +367,19 @@ where
     std::fs::create_dir_all(project_path.as_std_path())
         .with_context(|| format!("Failed to create project directory `{project_path}`"))?;
 
+    let system = OsSystem::new(&project_path);
+    let mut setup_context = SetupContext {
+        system: &system,
+        root_path: &root_path,
+        options: None,
+        included_paths: None,
+    };
+
     setup_files
-        .setup(&root_path, &project_path)
+        .setup(&mut setup_context)
         .context("Failed to setup test files")?;
 
-    let system = OsSystem::new(&project_path);
-
-    if let Some(options) = create_options(&root_path, &project_path) {
+    if let Some(options) = setup_context.options {
         std::fs::write(
             project_path.join("pyproject.toml").as_std_path(),
             toml::to_string(&PyProject {
@@ -315,7 +393,11 @@ where
         .context("Failed to write configuration")?;
     }
 
-    let project = ProjectMetadata::discover(&project_path, &system)?;
+    let included_paths = setup_context.included_paths;
+
+    let mut project = ProjectMetadata::discover(&project_path, &system)?;
+    project.apply_configuration_files(&system)?;
+
     let program_settings = project.to_program_settings(&system);
 
     for path in program_settings
@@ -328,7 +410,11 @@ where
             .with_context(|| format!("Failed to create search path `{path}`"))?;
     }
 
-    let db = ProjectDatabase::new(project, system)?;
+    let mut db = ProjectDatabase::new(project, system)?;
+
+    if let Some(included_paths) = included_paths {
+        db.project().set_included_paths(&mut db, included_paths);
+    }
 
     let (sender, receiver) = crossbeam::channel::unbounded();
     let watcher = directory_watcher(move |events| sender.send(events).unwrap())
@@ -390,7 +476,7 @@ fn new_file() -> anyhow::Result<()> {
     let foo_path = case.project_path("foo.py");
 
     assert_eq!(case.system_file(&foo_path), Err(FileError::NotFound));
-    assert_eq!(&case.collect_project_files(), &[bar_file]);
+    case.assert_indexed_project_files([bar_file]);
 
     std::fs::write(foo_path.as_std_path(), "print('Hello')")?;
 
@@ -400,7 +486,7 @@ fn new_file() -> anyhow::Result<()> {
 
     let foo = case.system_file(&foo_path).expect("foo.py to exist.");
 
-    assert_eq!(&case.collect_project_files(), &[bar_file, foo]);
+    case.assert_indexed_project_files([bar_file, foo]);
 
     Ok(())
 }
@@ -413,7 +499,7 @@ fn new_ignored_file() -> anyhow::Result<()> {
     let foo_path = case.project_path("foo.py");
 
     assert_eq!(case.system_file(&foo_path), Err(FileError::NotFound));
-    assert_eq!(&case.collect_project_files(), &[bar_file]);
+    case.assert_indexed_project_files([bar_file]);
 
     std::fs::write(foo_path.as_std_path(), "print('Hello')")?;
 
@@ -422,7 +508,132 @@ fn new_ignored_file() -> anyhow::Result<()> {
     case.apply_changes(changes);
 
     assert!(case.system_file(&foo_path).is_ok());
-    assert_eq!(&case.collect_project_files(), &[bar_file]);
+    case.assert_indexed_project_files([bar_file]);
+
+    Ok(())
+}
+
+#[test]
+fn new_non_project_file() -> anyhow::Result<()> {
+    let mut case = setup(|context: &mut SetupContext| {
+        context.write_project_file("bar.py", "")?;
+        context.set_options(Options {
+            environment: Some(EnvironmentOptions {
+                extra_paths: Some(vec![RelativePathBuf::cli(
+                    context.join_root_path("site_packages"),
+                )]),
+                ..EnvironmentOptions::default()
+            }),
+            ..Options::default()
+        });
+
+        Ok(())
+    })?;
+
+    let bar_path = case.project_path("bar.py");
+    let bar_file = case.system_file(&bar_path).unwrap();
+
+    case.assert_indexed_project_files([bar_file]);
+
+    // Add a file to site packages
+    let black_path = case.root_path().join("site_packages/black.py");
+    std::fs::write(black_path.as_std_path(), "print('Hello')")?;
+
+    let changes = case.stop_watch(event_for_file("black.py"));
+
+    case.apply_changes(changes);
+
+    assert!(case.system_file(&black_path).is_ok());
+
+    // The file should not have been added to the project files
+    case.assert_indexed_project_files([bar_file]);
+
+    Ok(())
+}
+
+#[test]
+fn new_files_with_explicit_included_paths() -> anyhow::Result<()> {
+    let mut case = setup(|context: &mut SetupContext| {
+        context.write_project_file("src/main.py", "")?;
+        context.write_project_file("src/sub/__init__.py", "")?;
+        context.write_project_file("src/test.py", "")?;
+        context.set_included_paths(vec![
+            context.join_project_path("src/main.py"),
+            context.join_project_path("src/sub"),
+        ]);
+        Ok(())
+    })?;
+
+    let main_path = case.project_path("src/main.py");
+    let main_file = case.system_file(&main_path).unwrap();
+
+    let sub_init_path = case.project_path("src/sub/__init__.py");
+    let sub_init = case.system_file(&sub_init_path).unwrap();
+
+    case.assert_indexed_project_files([main_file, sub_init]);
+
+    // Write a new file to `sub` which is an included path
+    let sub_a_path = case.project_path("src/sub/a.py");
+    std::fs::write(sub_a_path.as_std_path(), "print('Hello')")?;
+
+    // and write a second file in the root directory -- this should not be included
+    let test2_path = case.project_path("src/test2.py");
+    std::fs::write(test2_path.as_std_path(), "print('Hello')")?;
+
+    let changes = case.stop_watch(event_for_file("test2.py"));
+
+    case.apply_changes(changes);
+
+    let sub_a_file = case.system_file(&sub_a_path).expect("sub/a.py to exist");
+
+    case.assert_indexed_project_files([main_file, sub_init, sub_a_file]);
+
+    Ok(())
+}
+
+#[test]
+fn new_file_in_included_out_of_project_directory() -> anyhow::Result<()> {
+    let mut case = setup(|context: &mut SetupContext| {
+        context.write_project_file("src/main.py", "")?;
+        context.write_project_file("script.py", "")?;
+        context.write_file("outside_project/a.py", "")?;
+
+        context.set_included_paths(vec![
+            context.join_root_path("outside_project"),
+            context.join_project_path("src"),
+        ]);
+        Ok(())
+    })?;
+
+    let main_path = case.project_path("src/main.py");
+    let main_file = case.system_file(&main_path).unwrap();
+
+    let outside_a_path = case.root_path().join("outside_project/a.py");
+    let outside_a = case.system_file(&outside_a_path).unwrap();
+
+    case.assert_indexed_project_files([outside_a, main_file]);
+
+    // Write a new file to `src` which should be watched
+    let src_a = case.project_path("src/a.py");
+    std::fs::write(src_a.as_std_path(), "print('Hello')")?;
+
+    // and write a second file to `outside_project` which should be watched too
+    let outside_b_path = case.root_path().join("outside_project/b.py");
+    std::fs::write(outside_b_path.as_std_path(), "print('Hello')")?;
+
+    // and a third file in the project's root that should not be included
+    let script2_path = case.project_path("script2.py");
+    std::fs::write(script2_path.as_std_path(), "print('Hello')")?;
+
+    let changes = case.stop_watch(event_for_file("script2.py"));
+
+    case.apply_changes(changes);
+
+    let src_a_file = case.system_file(&src_a).unwrap();
+    let outside_b_file = case.system_file(&outside_b_path).unwrap();
+
+    // The file should not have been added to the project files
+    case.assert_indexed_project_files([main_file, outside_a, outside_b_file, src_a_file]);
 
     Ok(())
 }
@@ -435,7 +646,7 @@ fn changed_file() -> anyhow::Result<()> {
 
     let foo = case.system_file(&foo_path)?;
     assert_eq!(source_text(case.db(), foo).as_str(), foo_source);
-    assert_eq!(&case.collect_project_files(), &[foo]);
+    case.assert_indexed_project_files([foo]);
 
     update_file(&foo_path, "print('Version 2')")?;
 
@@ -446,7 +657,7 @@ fn changed_file() -> anyhow::Result<()> {
     case.apply_changes(changes);
 
     assert_eq!(source_text(case.db(), foo).as_str(), "print('Version 2')");
-    assert_eq!(&case.collect_project_files(), &[foo]);
+    case.assert_indexed_project_files([foo]);
 
     Ok(())
 }
@@ -460,7 +671,7 @@ fn deleted_file() -> anyhow::Result<()> {
     let foo = case.system_file(&foo_path)?;
 
     assert!(foo.exists(case.db()));
-    assert_eq!(&case.collect_project_files(), &[foo]);
+    case.assert_indexed_project_files([foo]);
 
     std::fs::remove_file(foo_path.as_std_path())?;
 
@@ -469,7 +680,7 @@ fn deleted_file() -> anyhow::Result<()> {
     case.apply_changes(changes);
 
     assert!(!foo.exists(case.db()));
-    assert_eq!(&case.collect_project_files(), &[] as &[File]);
+    case.assert_indexed_project_files([]);
 
     Ok(())
 }
@@ -489,7 +700,7 @@ fn move_file_to_trash() -> anyhow::Result<()> {
     let foo = case.system_file(&foo_path)?;
 
     assert!(foo.exists(case.db()));
-    assert_eq!(&case.collect_project_files(), &[foo]);
+    case.assert_indexed_project_files([foo]);
 
     std::fs::rename(
         foo_path.as_std_path(),
@@ -501,7 +712,7 @@ fn move_file_to_trash() -> anyhow::Result<()> {
     case.apply_changes(changes);
 
     assert!(!foo.exists(case.db()));
-    assert_eq!(&case.collect_project_files(), &[] as &[File]);
+    case.assert_indexed_project_files([]);
 
     Ok(())
 }
@@ -519,7 +730,7 @@ fn move_file_to_project() -> anyhow::Result<()> {
     let foo_in_project = case.project_path("foo.py");
 
     assert!(case.system_file(&foo_path).is_ok());
-    assert_eq!(&case.collect_project_files(), &[bar]);
+    case.assert_indexed_project_files([bar]);
 
     std::fs::rename(foo_path.as_std_path(), foo_in_project.as_std_path())?;
 
@@ -530,7 +741,7 @@ fn move_file_to_project() -> anyhow::Result<()> {
     let foo_in_project = case.system_file(&foo_in_project)?;
 
     assert!(foo_in_project.exists(case.db()));
-    assert_eq!(&case.collect_project_files(), &[bar, foo_in_project]);
+    case.assert_indexed_project_files([bar, foo_in_project]);
 
     Ok(())
 }
@@ -544,7 +755,7 @@ fn rename_file() -> anyhow::Result<()> {
 
     let foo = case.system_file(&foo_path)?;
 
-    assert_eq!(case.collect_project_files(), [foo]);
+    case.assert_indexed_project_files([foo]);
 
     std::fs::rename(foo_path.as_std_path(), bar_path.as_std_path())?;
 
@@ -557,7 +768,7 @@ fn rename_file() -> anyhow::Result<()> {
     let bar = case.system_file(&bar_path)?;
 
     assert!(bar.exists(case.db()));
-    assert_eq!(case.collect_project_files(), [bar]);
+    case.assert_indexed_project_files([bar]);
 
     Ok(())
 }
@@ -583,7 +794,7 @@ fn directory_moved_to_project() -> anyhow::Result<()> {
     );
 
     assert_eq!(sub_a_module, None);
-    assert_eq!(case.collect_project_files(), &[bar]);
+    case.assert_indexed_project_files([bar]);
 
     let sub_new_path = case.project_path("sub");
     std::fs::rename(sub_original_path.as_std_path(), sub_new_path.as_std_path())
@@ -607,7 +818,7 @@ fn directory_moved_to_project() -> anyhow::Result<()> {
     )
     .is_some());
 
-    assert_eq!(case.collect_project_files(), &[bar, init_file, a_file]);
+    case.assert_indexed_project_files([bar, init_file, a_file]);
 
     Ok(())
 }
@@ -635,7 +846,7 @@ fn directory_moved_to_trash() -> anyhow::Result<()> {
         .system_file(sub_path.join("a.py"))
         .expect("a.py to exist");
 
-    assert_eq!(case.collect_project_files(), &[bar, init_file, a_file]);
+    case.assert_indexed_project_files([bar, init_file, a_file]);
 
     std::fs::create_dir(case.root_path().join(".trash").as_std_path())?;
     let trashed_sub = case.root_path().join(".trash/sub");
@@ -656,7 +867,7 @@ fn directory_moved_to_trash() -> anyhow::Result<()> {
     assert!(!init_file.exists(case.db()));
     assert!(!a_file.exists(case.db()));
 
-    assert_eq!(case.collect_project_files(), &[bar]);
+    case.assert_indexed_project_files([bar]);
 
     Ok(())
 }
@@ -690,7 +901,7 @@ fn directory_renamed() -> anyhow::Result<()> {
         .system_file(sub_path.join("a.py"))
         .expect("a.py to exist");
 
-    assert_eq!(case.collect_project_files(), &[bar, sub_init, sub_a]);
+    case.assert_indexed_project_files([bar, sub_init, sub_a]);
 
     let foo_baz = case.project_path("foo/baz");
 
@@ -732,10 +943,7 @@ fn directory_renamed() -> anyhow::Result<()> {
     assert!(foo_baz_init.exists(case.db()));
     assert!(foo_baz_a.exists(case.db()));
 
-    assert_eq!(
-        case.collect_project_files(),
-        &[bar, foo_baz_init, foo_baz_a]
-    );
+    case.assert_indexed_project_files([bar, foo_baz_init, foo_baz_a]);
 
     Ok(())
 }
@@ -764,7 +972,7 @@ fn directory_deleted() -> anyhow::Result<()> {
     let a_file = case
         .system_file(sub_path.join("a.py"))
         .expect("a.py to exist");
-    assert_eq!(case.collect_project_files(), &[bar, init_file, a_file]);
+    case.assert_indexed_project_files([bar, init_file, a_file]);
 
     std::fs::remove_dir_all(sub_path.as_std_path())
         .with_context(|| "Failed to remove the sub directory")?;
@@ -782,21 +990,26 @@ fn directory_deleted() -> anyhow::Result<()> {
 
     assert!(!init_file.exists(case.db()));
     assert!(!a_file.exists(case.db()));
-    assert_eq!(case.collect_project_files(), &[bar]);
+    case.assert_indexed_project_files([bar]);
 
     Ok(())
 }
 
 #[test]
 fn search_path() -> anyhow::Result<()> {
-    let mut case = setup_with_options([("bar.py", "import sub.a")], |root_path, _project_path| {
-        Some(Options {
+    let mut case = setup(|context: &mut SetupContext| {
+        context.write_project_file("bar.py", "import sub.a")?;
+
+        context.set_options(Options {
             environment: Some(EnvironmentOptions {
-                extra_paths: Some(vec![RelativePathBuf::cli(root_path.join("site_packages"))]),
+                extra_paths: Some(vec![RelativePathBuf::cli(
+                    context.join_root_path("site_packages"),
+                )]),
                 ..EnvironmentOptions::default()
             }),
             ..Options::default()
-        })
+        });
+        Ok(())
     })?;
 
     let site_packages = case.root_path().join("site_packages");
@@ -813,10 +1026,7 @@ fn search_path() -> anyhow::Result<()> {
     case.apply_changes(changes);
 
     assert!(resolve_module(case.db().upcast(), &ModuleName::new_static("a").unwrap()).is_some());
-    assert_eq!(
-        case.collect_project_files(),
-        &[case.system_file(case.project_path("bar.py")).unwrap()]
-    );
+    case.assert_indexed_project_files([case.system_file(case.project_path("bar.py")).unwrap()]);
 
     Ok(())
 }
@@ -853,14 +1063,19 @@ fn add_search_path() -> anyhow::Result<()> {
 
 #[test]
 fn remove_search_path() -> anyhow::Result<()> {
-    let mut case = setup_with_options([("bar.py", "import sub.a")], |root_path, _project_path| {
-        Some(Options {
+    let mut case = setup(|context: &mut SetupContext| {
+        context.write_project_file("bar.py", "import sub.a")?;
+        context.set_options(Options {
             environment: Some(EnvironmentOptions {
-                extra_paths: Some(vec![RelativePathBuf::cli(root_path.join("site_packages"))]),
+                extra_paths: Some(vec![RelativePathBuf::cli(
+                    context.join_root_path("site_packages"),
+                )]),
                 ..EnvironmentOptions::default()
             }),
             ..Options::default()
-        })
+        });
+
+        Ok(())
     })?;
 
     // Remove site packages from the search path settings.
@@ -883,30 +1098,30 @@ fn remove_search_path() -> anyhow::Result<()> {
 
 #[test]
 fn change_python_version_and_platform() -> anyhow::Result<()> {
-    let mut case = setup_with_options(
+    let mut case = setup(|context: &mut SetupContext| {
         // `sys.last_exc` is a Python 3.12 only feature
         // `os.getegid()` is Unix only
-        [(
+        context.write_project_file(
             "bar.py",
             r#"
 import sys
 import os
 print(sys.last_exc, os.getegid())
 "#,
-        )],
-        |_root_path, _project_path| {
-            Some(Options {
-                environment: Some(EnvironmentOptions {
-                    python_version: Some(RangedValue::cli(PythonVersion::PY311)),
-                    python_platform: Some(RangedValue::cli(PythonPlatform::Identifier(
-                        "win32".to_string(),
-                    ))),
-                    ..EnvironmentOptions::default()
-                }),
-                ..Options::default()
-            })
-        },
-    )?;
+        )?;
+        context.set_options(Options {
+            environment: Some(EnvironmentOptions {
+                python_version: Some(RangedValue::cli(PythonVersion::PY311)),
+                python_platform: Some(RangedValue::cli(PythonPlatform::Identifier(
+                    "win32".to_string(),
+                ))),
+                ..EnvironmentOptions::default()
+            }),
+            ..Options::default()
+        });
+
+        Ok(())
+    })?;
 
     let diagnostics = case.db.check().context("Failed to check project.")?;
 
@@ -941,28 +1156,35 @@ print(sys.last_exc, os.getegid())
 
 #[test]
 fn changed_versions_file() -> anyhow::Result<()> {
-    let mut case = setup_with_options(
-        |root_path: &SystemPath, project_path: &SystemPath| {
-            std::fs::write(project_path.join("bar.py").as_std_path(), "import sub.a")?;
-            std::fs::create_dir_all(root_path.join("typeshed/stdlib").as_std_path())?;
-            std::fs::write(root_path.join("typeshed/stdlib/VERSIONS").as_std_path(), "")?;
-            std::fs::write(
-                root_path.join("typeshed/stdlib/os.pyi").as_std_path(),
-                "# not important",
-            )?;
+    let mut case = setup(|context: &mut SetupContext| {
+        std::fs::write(
+            context.join_project_path("bar.py").as_std_path(),
+            "import sub.a",
+        )?;
+        std::fs::create_dir_all(context.join_root_path("typeshed/stdlib").as_std_path())?;
+        std::fs::write(
+            context
+                .join_root_path("typeshed/stdlib/VERSIONS")
+                .as_std_path(),
+            "",
+        )?;
+        std::fs::write(
+            context
+                .join_root_path("typeshed/stdlib/os.pyi")
+                .as_std_path(),
+            "# not important",
+        )?;
 
-            Ok(())
-        },
-        |root_path, _project_path| {
-            Some(Options {
-                environment: Some(EnvironmentOptions {
-                    typeshed: Some(RelativePathBuf::cli(root_path.join("typeshed"))),
-                    ..EnvironmentOptions::default()
-                }),
-                ..Options::default()
-            })
-        },
-    )?;
+        context.set_options(Options {
+            environment: Some(EnvironmentOptions {
+                typeshed: Some(RelativePathBuf::cli(context.join_root_path("typeshed"))),
+                ..EnvironmentOptions::default()
+            }),
+            ..Options::default()
+        });
+
+        Ok(())
+    })?;
 
     // Unset the custom typeshed directory.
     assert_eq!(
@@ -1007,12 +1229,12 @@ fn changed_versions_file() -> anyhow::Result<()> {
 /// we're seeing is that Windows only emits a single event, similar to Linux.
 #[test]
 fn hard_links_in_project() -> anyhow::Result<()> {
-    let mut case = setup(|_root: &SystemPath, project: &SystemPath| {
-        let foo_path = project.join("foo.py");
+    let mut case = setup(|context: &mut SetupContext| {
+        let foo_path = context.join_project_path("foo.py");
         std::fs::write(foo_path.as_std_path(), "print('Version 1')")?;
 
         // Create a hardlink to `foo`
-        let bar_path = project.join("bar.py");
+        let bar_path = context.join_project_path("bar.py");
         std::fs::hard_link(foo_path.as_std_path(), bar_path.as_std_path())
             .context("Failed to create hard link from foo.py -> bar.py")?;
 
@@ -1026,6 +1248,7 @@ fn hard_links_in_project() -> anyhow::Result<()> {
 
     assert_eq!(source_text(case.db(), foo).as_str(), "print('Version 1')");
     assert_eq!(source_text(case.db(), bar).as_str(), "print('Version 1')");
+    case.assert_indexed_project_files([bar, foo]);
 
     // Write to the hard link target.
     update_file(foo_path, "print('Version 2')").context("Failed to update foo.py")?;
@@ -1078,12 +1301,12 @@ fn hard_links_in_project() -> anyhow::Result<()> {
     ignore = "windows doesn't support observing changes to hard linked files."
 )]
 fn hard_links_to_target_outside_project() -> anyhow::Result<()> {
-    let mut case = setup(|root: &SystemPath, project: &SystemPath| {
-        let foo_path = root.join("foo.py");
+    let mut case = setup(|context: &mut SetupContext| {
+        let foo_path = context.join_root_path("foo.py");
         std::fs::write(foo_path.as_std_path(), "print('Version 1')")?;
 
         // Create a hardlink to `foo`
-        let bar_path = project.join("bar.py");
+        let bar_path = context.join_project_path("bar.py");
         std::fs::hard_link(foo_path.as_std_path(), bar_path.as_std_path())
             .context("Failed to create hard link from foo.py -> bar.py")?;
 
@@ -1186,9 +1409,9 @@ mod unix {
         ignore = "FSEvents doesn't emit change events for symlinked directories outside of the watched paths."
     )]
     fn symlink_target_outside_watched_paths() -> anyhow::Result<()> {
-        let mut case = setup(|root: &SystemPath, project: &SystemPath| {
+        let mut case = setup(|context: &mut SetupContext| {
             // Set up the symlink target.
-            let link_target = root.join("bar");
+            let link_target = context.join_root_path("bar");
             std::fs::create_dir_all(link_target.as_std_path())
                 .context("Failed to create link target directory")?;
             let baz_original = link_target.join("baz.py");
@@ -1196,7 +1419,7 @@ mod unix {
                 .context("Failed to write link target file")?;
 
             // Create a symlink inside the project
-            let bar = project.join("bar");
+            let bar = context.join_project_path("bar");
             std::os::unix::fs::symlink(link_target.as_std_path(), bar.as_std_path())
                 .context("Failed to create symlink to bar package")?;
 
@@ -1267,9 +1490,9 @@ mod unix {
     /// ```
     #[test]
     fn symlink_inside_project() -> anyhow::Result<()> {
-        let mut case = setup(|_root: &SystemPath, project: &SystemPath| {
+        let mut case = setup(|context: &mut SetupContext| {
             // Set up the symlink target.
-            let link_target = project.join("patched/bar");
+            let link_target = context.join_project_path("patched/bar");
             std::fs::create_dir_all(link_target.as_std_path())
                 .context("Failed to create link target directory")?;
             let baz_original = link_target.join("baz.py");
@@ -1277,7 +1500,7 @@ mod unix {
                 .context("Failed to write link target file")?;
 
             // Create a symlink inside site-packages
-            let bar_in_project = project.join("bar");
+            let bar_in_project = context.join_project_path("bar");
             std::os::unix::fs::symlink(link_target.as_std_path(), bar_in_project.as_std_path())
                 .context("Failed to create symlink to bar package")?;
 
@@ -1305,6 +1528,8 @@ mod unix {
         );
         assert_eq!(baz.file().path(case.db()).as_system_path(), Some(&*bar_baz));
 
+        case.assert_indexed_project_files([patched_bar_baz_file]);
+
         // Write to the symlink target.
         update_file(&patched_bar_baz, "def baz(): print('Version 2')")
             .context("Failed to update bar/baz.py")?;
@@ -1340,6 +1565,7 @@ mod unix {
             bar_baz_text = bar_baz_text.as_str()
         );
 
+        case.assert_indexed_project_files([patched_bar_baz_file]);
         Ok(())
     }
 
@@ -1357,42 +1583,39 @@ mod unix {
     /// ```
     #[test]
     fn symlinked_module_search_path() -> anyhow::Result<()> {
-        let mut case = setup_with_options(
-            |root: &SystemPath, project: &SystemPath| {
-                // Set up the symlink target.
-                let site_packages = root.join("site-packages");
-                let bar = site_packages.join("bar");
-                std::fs::create_dir_all(bar.as_std_path())
-                    .context("Failed to create bar directory")?;
-                let baz_original = bar.join("baz.py");
-                std::fs::write(baz_original.as_std_path(), "def baz(): ...")
-                    .context("Failed to write baz.py")?;
+        let mut case = setup(|context: &mut SetupContext| {
+            // Set up the symlink target.
+            let site_packages = context.join_root_path("site-packages");
+            let bar = site_packages.join("bar");
+            std::fs::create_dir_all(bar.as_std_path()).context("Failed to create bar directory")?;
+            let baz_original = bar.join("baz.py");
+            std::fs::write(baz_original.as_std_path(), "def baz(): ...")
+                .context("Failed to write baz.py")?;
 
-                // Symlink the site packages in the venv to the global site packages
-                let venv_site_packages = project.join(".venv/lib/python3.12/site-packages");
-                std::fs::create_dir_all(venv_site_packages.parent().unwrap())
-                    .context("Failed to create .venv directory")?;
-                std::os::unix::fs::symlink(
-                    site_packages.as_std_path(),
-                    venv_site_packages.as_std_path(),
-                )
-                .context("Failed to create symlink to site-packages")?;
+            // Symlink the site packages in the venv to the global site packages
+            let venv_site_packages =
+                context.join_project_path(".venv/lib/python3.12/site-packages");
+            std::fs::create_dir_all(venv_site_packages.parent().unwrap())
+                .context("Failed to create .venv directory")?;
+            std::os::unix::fs::symlink(
+                site_packages.as_std_path(),
+                venv_site_packages.as_std_path(),
+            )
+            .context("Failed to create symlink to site-packages")?;
 
-                Ok(())
-            },
-            |_root, _project| {
-                Some(Options {
-                    environment: Some(EnvironmentOptions {
-                        extra_paths: Some(vec![RelativePathBuf::cli(
-                            ".venv/lib/python3.12/site-packages",
-                        )]),
-                        python_version: Some(RangedValue::cli(PythonVersion::PY312)),
-                        ..EnvironmentOptions::default()
-                    }),
-                    ..Options::default()
-                })
-            },
-        )?;
+            context.set_options(Options {
+                environment: Some(EnvironmentOptions {
+                    extra_paths: Some(vec![RelativePathBuf::cli(
+                        ".venv/lib/python3.12/site-packages",
+                    )]),
+                    python_version: Some(RangedValue::cli(PythonVersion::PY312)),
+                    ..EnvironmentOptions::default()
+                }),
+                ..Options::default()
+            });
+
+            Ok(())
+        })?;
 
         let baz = resolve_module(
             case.db().upcast(),
@@ -1419,6 +1642,8 @@ mod unix {
             Some(&*baz_original)
         );
 
+        case.assert_indexed_project_files([]);
+
         // Write to the symlink target.
         update_file(&baz_original, "def baz(): print('Version 2')")
             .context("Failed to update bar/baz.py")?;
@@ -1444,15 +1669,17 @@ mod unix {
             "def baz(): print('Version 2')"
         );
 
+        case.assert_indexed_project_files([]);
+
         Ok(())
     }
 }
 
 #[test]
 fn nested_projects_delete_root() -> anyhow::Result<()> {
-    let mut case = setup(|root: &SystemPath, project_root: &SystemPath| {
+    let mut case = setup(|context: &mut SetupContext| {
         std::fs::write(
-            project_root.join("pyproject.toml").as_std_path(),
+            context.join_project_path("pyproject.toml").as_std_path(),
             r#"
             [project]
             name = "inner"
@@ -1462,7 +1689,7 @@ fn nested_projects_delete_root() -> anyhow::Result<()> {
         )?;
 
         std::fs::write(
-            root.join("pyproject.toml").as_std_path(),
+            context.join_root_path("pyproject.toml").as_std_path(),
             r#"
             [project]
             name = "outer"
@@ -1487,3 +1714,79 @@ fn nested_projects_delete_root() -> anyhow::Result<()> {
 
     Ok(())
 }
+
+#[test]
+fn changes_to_user_configuration() -> anyhow::Result<()> {
+    let mut _config_dir_override: Option<UserConfigDirectoryOverrideGuard> = None;
+
+    let mut case = setup(|context: &mut SetupContext| {
+        std::fs::write(
+            context.join_project_path("pyproject.toml").as_std_path(),
+            r#"
+            [project]
+            name = "test"
+            "#,
+        )?;
+
+        std::fs::write(
+            context.join_project_path("foo.py").as_std_path(),
+            "a = 10 / 0",
+        )?;
+
+        let config_directory = context.join_root_path("home/.config");
+        std::fs::create_dir_all(config_directory.join("knot").as_std_path())?;
+        std::fs::write(
+            config_directory.join("knot/knot.toml").as_std_path(),
+            r#"
+            [rules]
+            division-by-zero = "ignore"
+            "#,
+        )?;
+
+        _config_dir_override = Some(
+            context
+                .system()
+                .with_user_config_directory(Some(config_directory)),
+        );
+
+        Ok(())
+    })?;
+
+    let foo = case
+        .system_file(case.project_path("foo.py"))
+        .expect("foo.py to exist");
+    let diagnostics = case
+        .db()
+        .check_file(foo)
+        .context("Failed to check project.")?;
+
+    assert!(
+        diagnostics.is_empty(),
+        "Expected no diagnostics but got: {diagnostics:#?}"
+    );
+
+    // Enable division-by-zero in the user configuration with warning severity
+    update_file(
+        case.root_path().join("home/.config/knot/knot.toml"),
+        r#"
+        [rules]
+        division-by-zero = "warn"
+        "#,
+    )?;
+
+    let changes = case.stop_watch(event_for_file("knot.toml"));
+
+    case.apply_changes(changes);
+
+    let diagnostics = case
+        .db()
+        .check_file(foo)
+        .context("Failed to check project.")?;
+
+    assert!(
+        diagnostics.len() == 1,
+        "Expected exactly one diagnostic but got: {diagnostics:#?}"
+    );
+
+    Ok(())
+}

crates/red_knot_project/Cargo.toml

@@ -13,7 +13,7 @@ license.workspace = true
 
 [dependencies]
 ruff_cache = { workspace = true }
-ruff_db = { workspace = true, features = ["os", "cache", "serde"] }
+ruff_db = { workspace = true, features = ["cache", "serde"] }
 ruff_macros = { workspace = true }
 ruff_python_ast = { workspace = true, features = ["serde"] }
 ruff_text_size = { workspace = true }
@@ -24,7 +24,7 @@ anyhow = { workspace = true }
 crossbeam = { workspace = true }
 glob = { workspace = true }
 notify = { workspace = true }
-pep440_rs = { workspace = true }
+pep440_rs = { workspace = true, features = ["version-ranges"] }
 rayon = { workspace = true }
 rustc-hash = { workspace = true }
 salsa = { workspace = true }

crates/red_knot_project/src/combine.rs

@@ -1,7 +1,8 @@
 use std::{collections::HashMap, hash::BuildHasher};
 
-use red_knot_python_semantic::{PythonPlatform, PythonVersion, SitePackages};
+use red_knot_python_semantic::{PythonPath, PythonPlatform};
 use ruff_db::system::SystemPathBuf;
+use ruff_python_ast::PythonVersion;
 
 /// Combine two values, preferring the values in `self`.
 ///
@@ -127,7 +128,7 @@ macro_rules! impl_noop_combine {
 
 impl_noop_combine!(SystemPathBuf);
 impl_noop_combine!(PythonPlatform);
-impl_noop_combine!(SitePackages);
+impl_noop_combine!(PythonPath);
 impl_noop_combine!(PythonVersion);
 
 // std types

crates/red_knot_project/src/db.rs

@@ -5,7 +5,7 @@ use crate::DEFAULT_LINT_REGISTRY;
 use crate::{Project, ProjectMetadata};
 use red_knot_python_semantic::lint::{LintRegistry, RuleSelection};
 use red_knot_python_semantic::{Db as SemanticDb, Program};
-use ruff_db::diagnostic::Diagnostic;
+use ruff_db::diagnostic::OldDiagnosticTrait;
 use ruff_db::files::{File, Files};
 use ruff_db::system::System;
 use ruff_db::vendored::VendoredFileSystem;
@@ -55,11 +55,11 @@ impl ProjectDatabase {
     }
 
     /// Checks all open files in the project and its dependencies.
-    pub fn check(&self) -> Result<Vec<Box<dyn Diagnostic>>, Cancelled> {
+    pub fn check(&self) -> Result<Vec<Box<dyn OldDiagnosticTrait>>, Cancelled> {
         self.with_db(|db| db.project().check(db))
     }
 
-    pub fn check_file(&self, file: File) -> Result<Vec<Box<dyn Diagnostic>>, Cancelled> {
+    pub fn check_file(&self, file: File) -> Result<Vec<Box<dyn OldDiagnosticTrait>>, Cancelled> {
         let _span = tracing::debug_span!("check_file", file=%file.path(self)).entered();
 
         self.with_db(|db| self.project().check_file(db, file))
@@ -114,8 +114,8 @@ impl SemanticDb for ProjectDatabase {
         project.is_file_open(self, file)
     }
 
-    fn rule_selection(&self) -> &RuleSelection {
-        self.project().rule_selection(self)
+    fn rule_selection(&self) -> Arc<RuleSelection> {
+        self.project().rules(self)
     }
 
     fn lint_registry(&self) -> &LintRegistry {
@@ -186,7 +186,6 @@ pub(crate) mod tests {
         files: Files,
         system: TestSystem,
         vendored: VendoredFileSystem,
-        rule_selection: RuleSelection,
         project: Option<Project>,
     }
 
@@ -198,7 +197,6 @@ pub(crate) mod tests {
                 vendored: red_knot_vendored::file_system().clone(),
                 files: Files::default(),
                 events: Arc::default(),
-                rule_selection: RuleSelection::from_registry(&DEFAULT_LINT_REGISTRY),
                 project: None,
             };
 
@@ -270,8 +268,8 @@ pub(crate) mod tests {
             !file.path(self).is_vendored_path()
         }
 
-        fn rule_selection(&self) -> &RuleSelection {
-            &self.rule_selection
+        fn rule_selection(&self) -> Arc<RuleSelection> {
+            self.project().rules(self)
         }
 
         fn lint_registry(&self) -> &LintRegistry {

crates/red_knot_project/src/db/changes.rs

@@ -2,10 +2,11 @@ use crate::db::{Db, ProjectDatabase};
 use crate::metadata::options::Options;
 use crate::watch::{ChangeEvent, CreatedKind, DeletedKind};
 use crate::{Project, ProjectMetadata};
+use std::collections::BTreeSet;
 
+use crate::walk::ProjectFilesWalker;
 use red_knot_python_semantic::Program;
-use ruff_db::files::{system_path_to_file, File, Files};
-use ruff_db::system::walk_directory::WalkState;
+use ruff_db::files::{File, Files};
 use ruff_db::system::SystemPath;
 use ruff_db::Db as _;
 use rustc_hash::FxHashSet;
@@ -14,7 +15,7 @@ impl ProjectDatabase {
     #[tracing::instrument(level = "debug", skip(self, changes, cli_options))]
     pub fn apply_changes(&mut self, changes: Vec<ChangeEvent>, cli_options: Option<&Options>) {
         let mut project = self.project();
-        let project_path = project.root(self).to_path_buf();
+        let project_root = project.root(self).to_path_buf();
         let program = Program::get(self);
         let custom_stdlib_versions_path = program
             .custom_stdlib_search_path(self)
@@ -29,7 +30,7 @@ impl ProjectDatabase {
 
         // Deduplicate the `sync` calls. Many file watchers emit multiple events for the same path.
         let mut synced_files = FxHashSet::default();
-        let mut synced_recursively = FxHashSet::default();
+        let mut sync_recursively = BTreeSet::default();
 
         let mut sync_path = |db: &mut ProjectDatabase, path: &SystemPath| {
             if synced_files.insert(path.to_path_buf()) {
@@ -37,17 +38,13 @@ impl ProjectDatabase {
             }
         };
 
-        let mut sync_recursively = |db: &mut ProjectDatabase, path: &SystemPath| {
-            if synced_recursively.insert(path.to_path_buf()) {
-                Files::sync_recursively(db, path);
-            }
-        };
-
         for change in changes {
+            tracing::trace!("Handle change: {:?}", change);
+
             if let Some(path) = change.system_path() {
                 if matches!(
                     path.file_name(),
-                    Some(".gitignore" | ".ignore" | "ruff.toml" | ".ruff.toml" | "pyproject.toml")
+                    Some(".gitignore" | ".ignore" | "knot.toml" | "pyproject.toml")
                 ) {
                     // Changes to ignore files or settings can change the project structure or add/remove files.
                     project_changed = true;
@@ -69,16 +66,27 @@ impl ProjectDatabase {
                     match kind {
                         CreatedKind::File => sync_path(self, &path),
                         CreatedKind::Directory | CreatedKind::Any => {
-                            sync_recursively(self, &path);
+                            sync_recursively.insert(path.clone());
                         }
                     }
 
-                    if self.system().is_file(&path) {
-                        // Add the parent directory because `walkdir` always visits explicitly passed files
-                        // even if they match an exclude filter.
-                        added_paths.insert(path.parent().unwrap().to_path_buf());
-                    } else {
-                        added_paths.insert(path);
+                    // Unlike other files, it's not only important to update the status of existing
+                    // and known `File`s (`sync_recursively`), it's also important to discover new files
+                    // that were added in the project's root (or any of the paths included for checking).
+                    //
+                    // This is important because `Project::check` iterates over all included files.
+                    // The code below walks the `added_paths` and adds all files that
+                    // should be included in the project. We can skip this check for
+                    // paths that aren't part of the project or shouldn't be included
+                    // when checking the project.
+                    if project.is_path_included(self, &path) {
+                        if self.system().is_file(&path) {
+                            // Add the parent directory because `walkdir` always visits explicitly passed files
+                            // even if they match an exclude filter.
+                            added_paths.insert(path.parent().unwrap().to_path_buf());
+                        } else {
+                            added_paths.insert(path);
+                        }
                     }
                 }
 
@@ -102,7 +110,7 @@ impl ProjectDatabase {
                             project.remove_file(self, file);
                         }
                     } else {
-                        sync_recursively(self, &path);
+                        sync_recursively.insert(path.clone());
 
                         if custom_stdlib_versions_path
                             .as_ref()
@@ -111,11 +119,19 @@ impl ProjectDatabase {
                             custom_stdlib_change = true;
                         }
 
-                        // Perform a full-reload in case the deleted directory contained the pyproject.toml.
-                        // We may want to make this more clever in the future, to e.g. iterate over the
-                        // indexed files and remove the once that start with the same path, unless
-                        // the deleted path is the project configuration.
-                        project_changed = true;
+                        if project.is_path_included(self, &path) || path == project_root {
+                            // TODO: Shouldn't it be enough to simply traverse the project files and remove all
+                            // that start with the given path?
+                            tracing::debug!(
+                                "Reload project because of a path that could have been a directory."
+                            );
+
+                            // Perform a full-reload in case the deleted directory contained the pyproject.toml.
+                            // We may want to make this more clever in the future, to e.g. iterate over the
+                            // indexed files and remove the once that start with the same path, unless
+                            // the deleted path is the project configuration.
+                            project_changed = true;
+                        }
                     }
                 }
 
@@ -132,18 +148,40 @@ impl ProjectDatabase {
                 ChangeEvent::Rescan => {
                     project_changed = true;
                     Files::sync_all(self);
+                    sync_recursively.clear();
                     break;
                 }
             }
         }
 
+        let sync_recursively = sync_recursively.into_iter();
+        let mut last = None;
+
+        for path in sync_recursively {
+            // Avoid re-syncing paths that are sub-paths of each other.
+            if let Some(last) = &last {
+                if path.starts_with(last) {
+                    continue;
+                }
+            }
+
+            Files::sync_recursively(self, &path);
+            last = Some(path);
+        }
+
         if project_changed {
-            match ProjectMetadata::discover(&project_path, self.system()) {
+            match ProjectMetadata::discover(&project_root, self.system()) {
                 Ok(mut metadata) => {
                     if let Some(cli_options) = cli_options {
                         metadata.apply_cli_options(cli_options.clone());
                     }
 
+                    if let Err(error) = metadata.apply_configuration_files(self.system()) {
+                        tracing::error!(
+                            "Failed to apply configuration files, continuing without applying them: {error}"
+                        );
+                    }
+
                     let program_settings = metadata.to_program_settings(self.system());
 
                     let program = Program::get(self);
@@ -179,43 +217,24 @@ impl ProjectDatabase {
             }
         }
 
-        let mut added_paths = added_paths.into_iter();
+        let diagnostics = if let Some(walker) = ProjectFilesWalker::incremental(self, added_paths) {
+            // Use directory walking to discover newly added files.
+            let (files, diagnostics) = walker.collect_vec(self);
 
-        // Use directory walking to discover newly added files.
-        if let Some(path) = added_paths.next() {
-            let mut walker = self.system().walk_directory(&path);
-
-            for extra_path in added_paths {
-                walker = walker.add(&extra_path);
+            for file in files {
+                project.add_file(self, file);
             }
 
-            let added_paths = std::sync::Mutex::new(Vec::default());
+            diagnostics
+        } else {
+            Vec::new()
+        };
 
-            walker.run(|| {
-                Box::new(|entry| {
-                    let Ok(entry) = entry else {
-                        return WalkState::Continue;
-                    };
-
-                    if !entry.file_type().is_file() {
-                        return WalkState::Continue;
-                    }
-
-                    let mut paths = added_paths.lock().unwrap();
-
-                    paths.push(entry.into_path());
-
-                    WalkState::Continue
-                })
-            });
-
-            for path in added_paths.into_inner().unwrap() {
-                let file = system_path_to_file(self, &path);
-
-                if let Ok(file) = file {
-                    project.add_file(self, file);
-                }
-            }
-        }
+        // Note: We simply replace all IO related diagnostics here. This isn't ideal, because
+        // it removes IO errors that may still be relevant. However, tracking IO errors correctly
+        // across revisions doesn't feel essential, considering that they're rare. However, we could
+        // implement a `BTreeMap` or similar and only prune the diagnostics from paths that we've
+        // re-scanned (or that were removed etc).
+        project.replace_index_diagnostics(self, diagnostics);
     }
 }

crates/red_knot_project/src/files.rs

@@ -8,10 +8,7 @@ use salsa::Setter;
 use ruff_db::files::File;
 
 use crate::db::Db;
-use crate::Project;
-
-/// Cheap cloneable hash set of files.
-type FileSet = Arc<FxHashSet<File>>;
+use crate::{IOErrorDiagnostic, Project};
 
 /// The indexed files of a project.
 ///
@@ -35,9 +32,9 @@ impl IndexedFiles {
         }
     }
 
-    fn indexed(files: FileSet) -> Self {
+    fn indexed(inner: Arc<IndexedInner>) -> Self {
         Self {
-            state: std::sync::Mutex::new(State::Indexed(files)),
+            state: std::sync::Mutex::new(State::Indexed(inner)),
         }
     }
 
@@ -46,8 +43,8 @@ impl IndexedFiles {
 
         match &*state {
             State::Lazy => Index::Lazy(LazyFiles { files: state }),
-            State::Indexed(files) => Index::Indexed(Indexed {
-                files: Arc::clone(files),
+            State::Indexed(inner) => Index::Indexed(Indexed {
+                inner: Arc::clone(inner),
                 _lifetime: PhantomData,
             }),
         }
@@ -94,7 +91,7 @@ impl IndexedFiles {
         Some(IndexedMut {
             db: Some(db),
             project,
-            files: indexed,
+            indexed,
             did_change: false,
         })
     }
@@ -112,7 +109,7 @@ enum State {
     Lazy,
 
     /// The files are indexed. Stores the known files of a package.
-    Indexed(FileSet),
+    Indexed(Arc<IndexedInner>),
 }
 
 pub(super) enum Index<'db> {
@@ -129,32 +126,48 @@ pub(super) struct LazyFiles<'db> {
 
 impl<'db> LazyFiles<'db> {
     /// Sets the indexed files of a package to `files`.
-    pub(super) fn set(mut self, files: FxHashSet<File>) -> Indexed<'db> {
+    pub(super) fn set(
+        mut self,
+        files: FxHashSet<File>,
+        diagnostics: Vec<IOErrorDiagnostic>,
+    ) -> Indexed<'db> {
         let files = Indexed {
-            files: Arc::new(files),
+            inner: Arc::new(IndexedInner { files, diagnostics }),
             _lifetime: PhantomData,
         };
-        *self.files = State::Indexed(Arc::clone(&files.files));
+        *self.files = State::Indexed(Arc::clone(&files.inner));
         files
     }
 }
 
-/// The indexed files of a package.
+/// The indexed files of the project.
 ///
 /// Note: This type is intentionally non-cloneable. Making it cloneable requires
 /// revisiting the locking behavior in [`IndexedFiles::indexed_mut`].
-#[derive(Debug, PartialEq, Eq)]
+#[derive(Debug)]
 pub struct Indexed<'db> {
-    files: FileSet,
+    inner: Arc<IndexedInner>,
     // Preserve the lifetime of `PackageFiles`.
     _lifetime: PhantomData<&'db ()>,
 }
 
+#[derive(Debug)]
+struct IndexedInner {
+    files: FxHashSet<File>,
+    diagnostics: Vec<IOErrorDiagnostic>,
+}
+
+impl Indexed<'_> {
+    pub(super) fn diagnostics(&self) -> &[IOErrorDiagnostic] {
+        &self.inner.diagnostics
+    }
+}
+
 impl Deref for Indexed<'_> {
     type Target = FxHashSet<File>;
 
     fn deref(&self) -> &Self::Target {
-        &self.files
+        &self.inner.files
     }
 }
 
@@ -165,7 +178,7 @@ impl<'a> IntoIterator for &'a Indexed<'_> {
     type IntoIter = IndexedIter<'a>;
 
     fn into_iter(self) -> Self::IntoIter {
-        self.files.iter().copied()
+        self.inner.files.iter().copied()
     }
 }
 
@@ -176,13 +189,13 @@ impl<'a> IntoIterator for &'a Indexed<'_> {
 pub(super) struct IndexedMut<'db> {
     db: Option<&'db mut dyn Db>,
     project: Project,
-    files: FileSet,
+    indexed: Arc<IndexedInner>,
     did_change: bool,
 }
 
 impl IndexedMut<'_> {
     pub(super) fn insert(&mut self, file: File) -> bool {
-        if self.files_mut().insert(file) {
+        if self.inner_mut().files.insert(file) {
             self.did_change = true;
             true
         } else {
@@ -191,7 +204,7 @@ impl IndexedMut<'_> {
     }
 
     pub(super) fn remove(&mut self, file: File) -> bool {
-        if self.files_mut().remove(&file) {
+        if self.inner_mut().files.remove(&file) {
             self.did_change = true;
             true
         } else {
@@ -199,8 +212,13 @@ impl IndexedMut<'_> {
         }
     }
 
-    fn files_mut(&mut self) -> &mut FxHashSet<File> {
-        Arc::get_mut(&mut self.files).expect("All references to `FilesSet` to have been dropped")
+    pub(super) fn set_diagnostics(&mut self, diagnostics: Vec<IOErrorDiagnostic>) {
+        self.inner_mut().diagnostics = diagnostics;
+    }
+
+    fn inner_mut(&mut self) -> &mut IndexedInner {
+        Arc::get_mut(&mut self.indexed)
+            .expect("All references to `FilesSet` should have been dropped")
     }
 
     fn set_impl(&mut self) {
@@ -208,16 +226,16 @@ impl IndexedMut<'_> {
             return;
         };
 
-        let files = Arc::clone(&self.files);
+        let indexed = Arc::clone(&self.indexed);
 
         if self.did_change {
             // If there are changes, set the new file_set to trigger a salsa revision change.
             self.project
                 .set_file_set(db)
-                .to(IndexedFiles::indexed(files));
+                .to(IndexedFiles::indexed(indexed));
         } else {
             // The `indexed_mut` replaced the `state` with Lazy. Restore it back to the indexed state.
-            *self.project.file_set(db).state.lock().unwrap() = State::Indexed(files);
+            *self.project.file_set(db).state.lock().unwrap() = State::Indexed(indexed);
         }
     }
 }
@@ -237,7 +255,7 @@ mod tests {
     use crate::files::Index;
     use crate::ProjectMetadata;
     use ruff_db::files::system_path_to_file;
-    use ruff_db::system::{DbWithTestSystem, SystemPathBuf};
+    use ruff_db::system::{DbWithWritableSystem as _, SystemPathBuf};
     use ruff_python_ast::name::Name;
 
     #[test]
@@ -252,7 +270,7 @@ mod tests {
         let file = system_path_to_file(&db, "test.py").unwrap();
 
         let files = match project.file_set(&db).get() {
-            Index::Lazy(lazy) => lazy.set(FxHashSet::from_iter([file])),
+            Index::Lazy(lazy) => lazy.set(FxHashSet::from_iter([file]), Vec::new()),
             Index::Indexed(files) => files,
         };
 

crates/red_knot_project/src/lib.rs

@@ -1,31 +1,32 @@
 #![allow(clippy::ref_option)]
 
 use crate::metadata::options::OptionDiagnostic;
+use crate::walk::{ProjectFilesFilter, ProjectFilesWalker};
 pub use db::{Db, ProjectDatabase};
 use files::{Index, Indexed, IndexedFiles};
+use metadata::settings::Settings;
 pub use metadata::{ProjectDiscoveryError, ProjectMetadata};
 use red_knot_python_semantic::lint::{LintRegistry, LintRegistryBuilder, RuleSelection};
 use red_knot_python_semantic::register_lints;
 use red_knot_python_semantic::types::check_types;
-use ruff_db::diagnostic::{Diagnostic, DiagnosticId, ParseDiagnostic, Severity};
-use ruff_db::files::{system_path_to_file, File};
+use ruff_db::diagnostic::{DiagnosticId, OldDiagnosticTrait, OldParseDiagnostic, Severity, Span};
+use ruff_db::files::File;
 use ruff_db::parsed::parsed_module;
 use ruff_db::source::{source_text, SourceTextError};
-use ruff_db::system::walk_directory::WalkState;
-use ruff_db::system::{FileType, SystemPath};
-use ruff_python_ast::PySourceType;
-use ruff_text_size::TextRange;
-use rustc_hash::{FxBuildHasher, FxHashSet};
+use ruff_db::system::{SystemPath, SystemPathBuf};
+use rustc_hash::FxHashSet;
 use salsa::Durability;
 use salsa::Setter;
 use std::borrow::Cow;
 use std::sync::Arc;
+use thiserror::Error;
 
 pub mod combine;
 
 mod db;
 mod files;
 pub mod metadata;
+mod walk;
 pub mod watch;
 
 pub static DEFAULT_LINT_REGISTRY: std::sync::LazyLock<LintRegistry> =
@@ -66,12 +67,46 @@ pub struct Project {
     /// The metadata describing the project, including the unresolved options.
     #[return_ref]
     pub metadata: ProjectMetadata,
+
+    /// The resolved project settings.
+    #[return_ref]
+    pub settings: Settings,
+
+    /// The paths that should be included when checking this project.
+    ///
+    /// The default (when this list is empty) is to include all files in the project root
+    /// (that satisfy the configured include and exclude patterns).
+    /// However, it's sometimes desired to only check a subset of the project, e.g. to see
+    /// the diagnostics for a single file or a folder.
+    ///
+    /// This list gets initialized by the paths passed to `knot check <paths>`
+    ///
+    /// ## How is this different from `open_files`?
+    ///
+    /// The `included_paths` is closely related to `open_files`. The only difference is that
+    /// `open_files` is already a resolved set of files whereas `included_paths` is only a list of paths
+    /// that are resolved to files by indexing them. The other difference is that
+    /// new files added to any directory in `included_paths` will be indexed and added to the project
+    /// whereas `open_files` needs to be updated manually (e.g. by the IDE).
+    ///
+    /// In short, `open_files` is cheaper in contexts where the set of files is known, like
+    /// in an IDE when the user only wants to check the open tabs. This could be modeled
+    /// with `included_paths` too but it would require an explicit walk dir step that's simply unnecessary.
+    #[default]
+    #[return_ref]
+    included_paths_list: Vec<SystemPathBuf>,
+
+    /// Diagnostics that were generated when resolving the project settings.
+    #[return_ref]
+    settings_diagnostics: Vec<OptionDiagnostic>,
 }
 
 #[salsa::tracked]
 impl Project {
     pub fn from_metadata(db: &dyn Db, metadata: ProjectMetadata) -> Self {
-        Project::builder(metadata)
+        let (settings, settings_diagnostics) = metadata.options().to_settings(db);
+
+        Project::builder(metadata, settings, settings_diagnostics)
             .durability(Durability::MEDIUM)
             .open_fileset_durability(Durability::LOW)
             .file_set_durability(Durability::LOW)
@@ -86,41 +121,64 @@ impl Project {
         self.metadata(db).name()
     }
 
+    /// Returns the resolved linter rules for the project.
+    ///
+    /// This is a salsa query to prevent re-computing queries if other, unrelated
+    /// settings change. For example, we don't want that changing the terminal settings
+    /// invalidates any type checking queries.
+    #[salsa::tracked]
+    pub fn rules(self, db: &dyn Db) -> Arc<RuleSelection> {
+        self.settings(db).to_rules()
+    }
+
+    /// Returns `true` if `path` is both part of the project and included (see `included_paths_list`).
+    ///
+    /// Unlike [Self::files], this method does not respect `.gitignore` files. It only checks
+    /// the project's include and exclude settings as well as the paths that were passed to `knot check <paths>`.
+    /// This means, that this method is an over-approximation of `Self::files` and may return `true` for paths
+    /// that won't be included when checking the project because they're ignored in a `.gitignore` file.
+    pub fn is_path_included(self, db: &dyn Db, path: &SystemPath) -> bool {
+        ProjectFilesFilter::from_project(db, self).is_included(path)
+    }
+
     pub fn reload(self, db: &mut dyn Db, metadata: ProjectMetadata) {
         tracing::debug!("Reloading project");
         assert_eq!(self.root(db), metadata.root());
 
         if &metadata != self.metadata(db) {
+            let (settings, settings_diagnostics) = metadata.options().to_settings(db);
+
+            if self.settings(db) != &settings {
+                self.set_settings(db).to(settings);
+            }
+
+            if self.settings_diagnostics(db) != &settings_diagnostics {
+                self.set_settings_diagnostics(db).to(settings_diagnostics);
+            }
+
             self.set_metadata(db).to(metadata);
         }
 
         self.reload_files(db);
     }
 
-    pub fn rule_selection(self, db: &dyn Db) -> &RuleSelection {
-        let (selection, _) = self.rule_selection_with_diagnostics(db);
-        selection
-    }
-
-    #[salsa::tracked(return_ref)]
-    fn rule_selection_with_diagnostics(
-        self,
-        db: &dyn Db,
-    ) -> (RuleSelection, Vec<OptionDiagnostic>) {
-        self.metadata(db).options().to_rule_selection(db)
-    }
-
     /// Checks all open files in the project and its dependencies.
-    pub(crate) fn check(self, db: &ProjectDatabase) -> Vec<Box<dyn Diagnostic>> {
+    pub(crate) fn check(self, db: &ProjectDatabase) -> Vec<Box<dyn OldDiagnosticTrait>> {
         let project_span = tracing::debug_span!("Project::check");
         let _span = project_span.enter();
 
         tracing::debug!("Checking project '{name}'", name = self.name(db));
 
-        let mut diagnostics: Vec<Box<dyn Diagnostic>> = Vec::new();
-        let (_, options_diagnostics) = self.rule_selection_with_diagnostics(db);
-        diagnostics.extend(options_diagnostics.iter().map(|diagnostic| {
-            let diagnostic: Box<dyn Diagnostic> = Box::new(diagnostic.clone());
+        let mut diagnostics: Vec<Box<dyn OldDiagnosticTrait>> = Vec::new();
+        diagnostics.extend(self.settings_diagnostics(db).iter().map(|diagnostic| {
+            let diagnostic: Box<dyn OldDiagnosticTrait> = Box::new(diagnostic.clone());
+            diagnostic
+        }));
+
+        let files = ProjectFiles::new(db, self);
+
+        diagnostics.extend(files.diagnostics().iter().cloned().map(|diagnostic| {
+            let diagnostic: Box<dyn OldDiagnosticTrait> = Box::new(diagnostic);
             diagnostic
         }));
 
@@ -131,7 +189,6 @@ impl Project {
         let project_span = project_span.clone();
 
         rayon::scope(move |scope| {
-            let files = ProjectFiles::new(&db, self);
             for file in &files {
                 let result = inner_result.clone();
                 let db = db.clone();
@@ -150,13 +207,12 @@ impl Project {
         Arc::into_inner(result).unwrap().into_inner().unwrap()
     }
 
-    pub(crate) fn check_file(self, db: &dyn Db, file: File) -> Vec<Box<dyn Diagnostic>> {
-        let (_, options_diagnostics) = self.rule_selection_with_diagnostics(db);
-
-        let mut file_diagnostics: Vec<_> = options_diagnostics
+    pub(crate) fn check_file(self, db: &dyn Db, file: File) -> Vec<Box<dyn OldDiagnosticTrait>> {
+        let mut file_diagnostics: Vec<_> = self
+            .settings_diagnostics(db)
             .iter()
             .map(|diagnostic| {
-                let diagnostic: Box<dyn Diagnostic> = Box::new(diagnostic.clone());
+                let diagnostic: Box<dyn OldDiagnosticTrait> = Box::new(diagnostic.clone());
                 diagnostic
             })
             .collect();
@@ -192,6 +248,30 @@ impl Project {
         removed
     }
 
+    pub fn set_included_paths(self, db: &mut dyn Db, paths: Vec<SystemPathBuf>) {
+        tracing::debug!("Setting included paths: {paths}", paths = paths.len());
+
+        self.set_included_paths_list(db).to(paths);
+        self.reload_files(db);
+    }
+
+    /// Returns the paths that should be checked.
+    ///
+    /// The default is to check the entire project in which case this method returns
+    /// the project root. However, users can specify to only check specific sub-folders or
+    /// even files of a project by using `knot check <paths>`. In that case, this method
+    /// returns the provided absolute paths.
+    ///
+    /// Note: The CLI doesn't prohibit users from specifying paths outside the project root.
+    /// This can be useful to check arbitrary files, but it isn't something we recommend.
+    /// We should try to support this use case but it's okay if there are some limitations around it.
+    fn included_paths_or_root(self, db: &dyn Db) -> &[SystemPathBuf] {
+        match &**self.included_paths_list(db) {
+            [] => std::slice::from_ref(&self.metadata(db).root),
+            paths => paths,
+        }
+    }
+
     /// Returns the open files in the project or `None` if the entire project should be checked.
     pub fn open_files(self, db: &dyn Db) -> Option<&FxHashSet<File>> {
         self.open_fileset(db).as_deref()
@@ -274,6 +354,17 @@ impl Project {
         index.insert(file);
     }
 
+    /// Replaces the diagnostics from indexing the project files with `diagnostics`.
+    ///
+    /// This is a no-op if the project files haven't been indexed yet.
+    pub fn replace_index_diagnostics(self, db: &mut dyn Db, diagnostics: Vec<IOErrorDiagnostic>) {
+        let Some(mut index) = IndexedFiles::indexed_mut(db, self) else {
+            return;
+        };
+
+        index.set_diagnostics(diagnostics);
+    }
+
     /// Returns the files belonging to this project.
     pub fn files(self, db: &dyn Db) -> Indexed<'_> {
         let files = self.file_set(db);
@@ -281,12 +372,14 @@ impl Project {
         let indexed = match files.get() {
             Index::Lazy(vacant) => {
                 let _entered =
-                    tracing::debug_span!("Project::index_files", package = %self.name(db))
+                    tracing::debug_span!("Project::index_files", project = %self.name(db))
                         .entered();
 
-                let files = discover_project_files(db, self);
-                tracing::info!("Found {} files in project `{}`", files.len(), self.name(db));
-                vacant.set(files)
+                let walker = ProjectFilesWalker::new(db);
+                let (files, diagnostics) = walker.collect_set(db);
+
+                tracing::info!("Indexed {} file(s)", files.len());
+                vacant.set(files, diagnostics)
             }
             Index::Indexed(indexed) => indexed,
         };
@@ -304,81 +397,41 @@ impl Project {
     }
 }
 
-fn check_file_impl(db: &dyn Db, file: File) -> Vec<Box<dyn Diagnostic>> {
-    let mut diagnostics: Vec<Box<dyn Diagnostic>> = Vec::new();
+fn check_file_impl(db: &dyn Db, file: File) -> Vec<Box<dyn OldDiagnosticTrait>> {
+    let mut diagnostics: Vec<Box<dyn OldDiagnosticTrait>> = Vec::new();
 
     // Abort checking if there are IO errors.
     let source = source_text(db.upcast(), file);
 
     if let Some(read_error) = source.read_error() {
         diagnostics.push(Box::new(IOErrorDiagnostic {
-            file,
-            error: read_error.clone(),
+            file: Some(file),
+            error: read_error.clone().into(),
         }));
         return diagnostics;
     }
 
     let parsed = parsed_module(db.upcast(), file);
     diagnostics.extend(parsed.errors().iter().map(|error| {
-        let diagnostic: Box<dyn Diagnostic> = Box::new(ParseDiagnostic::new(file, error.clone()));
+        let diagnostic: Box<dyn OldDiagnosticTrait> =
+            Box::new(OldParseDiagnostic::new(file, error.clone()));
         diagnostic
     }));
 
     diagnostics.extend(check_types(db.upcast(), file).iter().map(|diagnostic| {
-        let boxed: Box<dyn Diagnostic> = Box::new(diagnostic.clone());
+        let boxed: Box<dyn OldDiagnosticTrait> = Box::new(diagnostic.clone());
         boxed
     }));
 
-    diagnostics.sort_unstable_by_key(|diagnostic| diagnostic.range().unwrap_or_default().start());
-
-    diagnostics
-}
-
-fn discover_project_files(db: &dyn Db, project: Project) -> FxHashSet<File> {
-    let paths = std::sync::Mutex::new(Vec::new());
-
-    db.system().walk_directory(project.root(db)).run(|| {
-        Box::new(|entry| {
-            match entry {
-                Ok(entry) => {
-                    // Skip over any non python files to avoid creating too many entries in `Files`.
-                    match entry.file_type() {
-                        FileType::File => {
-                            if entry
-                                .path()
-                                .extension()
-                                .and_then(PySourceType::try_from_extension)
-                                .is_some()
-                            {
-                                let mut paths = paths.lock().unwrap();
-                                paths.push(entry.into_path());
-                            }
-                        }
-                        FileType::Directory | FileType::Symlink => {}
-                    }
-                }
-                Err(error) => {
-                    // TODO Handle error
-                    tracing::error!("Failed to walk path: {error}");
-                }
-            }
-
-            WalkState::Continue
-        })
+    diagnostics.sort_unstable_by_key(|diagnostic| {
+        diagnostic
+            .span()
+            .and_then(|span| span.range())
+            .unwrap_or_default()
+            .start()
     });
 
-    let paths = paths.into_inner().unwrap();
-    let mut files = FxHashSet::with_capacity_and_hasher(paths.len(), FxBuildHasher);
-
-    for path in paths {
-        // If this returns `None`, then the file was deleted between the `walk_directory` call and now.
-        // We can ignore this.
-        if let Ok(file) = system_path_to_file(db.upcast(), &path) {
-            files.insert(file);
-        }
-    }
-
-    files
+    diagnostics
 }
 
 #[derive(Debug)]
@@ -395,6 +448,13 @@ impl<'a> ProjectFiles<'a> {
             ProjectFiles::Indexed(project.files(db))
         }
     }
+
+    fn diagnostics(&self) -> &[IOErrorDiagnostic] {
+        match self {
+            ProjectFiles::OpenFiles(_) => &[],
+            ProjectFiles::Indexed(indexed) => indexed.diagnostics(),
+        }
+    }
 }
 
 impl<'a> IntoIterator for &'a ProjectFiles<'a> {
@@ -427,13 +487,13 @@ impl Iterator for ProjectFilesIter<'_> {
     }
 }
 
-#[derive(Debug)]
+#[derive(Debug, Clone)]
 pub struct IOErrorDiagnostic {
-    file: File,
-    error: SourceTextError,
+    file: Option<File>,
+    error: IOErrorKind,
 }
 
-impl Diagnostic for IOErrorDiagnostic {
+impl OldDiagnosticTrait for IOErrorDiagnostic {
     fn id(&self) -> DiagnosticId {
         DiagnosticId::Io
     }
@@ -442,12 +502,8 @@ impl Diagnostic for IOErrorDiagnostic {
         self.error.to_string().into()
     }
 
-    fn file(&self) -> Option<File> {
-        Some(self.file)
-    }
-
-    fn range(&self) -> Option<TextRange> {
-        None
+    fn span(&self) -> Option<Span> {
+        self.file.map(Span::from)
     }
 
     fn severity(&self) -> Severity {
@@ -455,15 +511,24 @@ impl Diagnostic for IOErrorDiagnostic {
     }
 }
 
+#[derive(Error, Debug, Clone)]
+enum IOErrorKind {
+    #[error(transparent)]
+    Walk(#[from] walk::WalkError),
+
+    #[error(transparent)]
+    SourceText(#[from] SourceTextError),
+}
+
 #[cfg(test)]
 mod tests {
     use crate::db::tests::TestDb;
     use crate::{check_file_impl, ProjectMetadata};
     use red_knot_python_semantic::types::check_types;
-    use ruff_db::diagnostic::Diagnostic;
+    use ruff_db::diagnostic::OldDiagnosticTrait;
     use ruff_db::files::system_path_to_file;
     use ruff_db::source::source_text;
-    use ruff_db::system::{DbWithTestSystem, SystemPath, SystemPathBuf};
+    use ruff_db::system::{DbWithTestSystem, DbWithWritableSystem as _, SystemPath, SystemPathBuf};
     use ruff_db::testing::assert_function_query_was_not_run;
     use ruff_python_ast::name::Name;
 

crates/red_knot_project/src/metadata.rs

@@ -1,3 +1,4 @@
+use configuration_file::{ConfigurationFile, ConfigurationFileError};
 use red_knot_python_semantic::ProgramSettings;
 use ruff_db::system::{System, SystemPath, SystemPathBuf};
 use ruff_python_ast::name::Name;
@@ -5,13 +6,15 @@ use std::sync::Arc;
 use thiserror::Error;
 
 use crate::combine::Combine;
-use crate::metadata::pyproject::{Project, PyProject, PyProjectError};
+use crate::metadata::pyproject::{Project, PyProject, PyProjectError, ResolveRequiresPythonError};
 use crate::metadata::value::ValueSource;
 use options::KnotTomlError;
 use options::Options;
 
+mod configuration_file;
 pub mod options;
 pub mod pyproject;
+pub mod settings;
 pub mod value;
 
 #[derive(Debug, PartialEq, Eq)]
@@ -23,6 +26,15 @@ pub struct ProjectMetadata {
 
     /// The raw options
     pub(super) options: Options,
+
+    /// Paths of configurations other than the project's configuration that were combined into [`Self::options`].
+    ///
+    /// This field stores the paths of the configuration files, mainly for
+    /// knowing which files to watch for changes.
+    ///
+    /// The path ordering doesn't imply precedence.
+    #[cfg_attr(test, serde(skip_serializing_if = "Vec::is_empty"))]
+    pub(super) extra_configuration_paths: Vec<SystemPathBuf>,
 }
 
 impl ProjectMetadata {
@@ -31,12 +43,16 @@ impl ProjectMetadata {
         Self {
             name,
             root,
+            extra_configuration_paths: Vec::default(),
             options: Options::default(),
         }
     }
 
     /// Loads a project from a `pyproject.toml` file.
-    pub(crate) fn from_pyproject(pyproject: PyProject, root: SystemPathBuf) -> Self {
+    pub(crate) fn from_pyproject(
+        pyproject: PyProject,
+        root: SystemPathBuf,
+    ) -> Result<Self, ResolveRequiresPythonError> {
         Self::from_options(
             pyproject
                 .tool
@@ -49,21 +65,37 @@ impl ProjectMetadata {
 
     /// Loads a project from a set of options with an optional pyproject-project table.
     pub(crate) fn from_options(
-        options: Options,
+        mut options: Options,
         root: SystemPathBuf,
         project: Option<&Project>,
-    ) -> Self {
+    ) -> Result<Self, ResolveRequiresPythonError> {
         let name = project
-            .and_then(|project| project.name.as_ref())
-            .map(|name| Name::new(&***name))
+            .and_then(|project| project.name.as_deref())
+            .map(|name| Name::new(&**name))
             .unwrap_or_else(|| Name::new(root.file_name().unwrap_or("root")));
 
-        // TODO(https://github.com/astral-sh/ruff/issues/15491): Respect requires-python
-        Self {
+        // If the `options` don't specify a python version but the `project.requires-python` field is set,
+        // use that as a lower bound instead.
+        if let Some(project) = project {
+            if options
+                .environment
+                .as_ref()
+                .is_none_or(|env| env.python_version.is_none())
+            {
+                if let Some(requires_python) = project.resolve_requires_python_lower_bound()? {
+                    let mut environment = options.environment.unwrap_or_default();
+                    environment.python_version = Some(requires_python);
+                    options.environment = Some(environment);
+                }
+            }
+        }
+
+        Ok(Self {
             name,
             root,
             options,
-        }
+            extra_configuration_paths: Vec::new(),
+        })
     }
 
     /// Discovers the closest project at `path` and returns its metadata.
@@ -131,19 +163,34 @@ impl ProjectMetadata {
                 }
 
                 tracing::debug!("Found project at '{}'", project_root);
-                return Ok(ProjectMetadata::from_options(
+
+                let metadata = ProjectMetadata::from_options(
                     options,
                     project_root.to_path_buf(),
                     pyproject
                         .as_ref()
                         .and_then(|pyproject| pyproject.project.as_ref()),
-                ));
+                )
+                .map_err(|err| {
+                    ProjectDiscoveryError::InvalidRequiresPythonConstraint {
+                        source: err,
+                        path: pyproject_path,
+                    }
+                })?;
+
+                return Ok(metadata);
             }
 
             if let Some(pyproject) = pyproject {
                 let has_knot_section = pyproject.knot().is_some();
                 let metadata =
-                    ProjectMetadata::from_pyproject(pyproject, project_root.to_path_buf());
+                    ProjectMetadata::from_pyproject(pyproject, project_root.to_path_buf())
+                        .map_err(
+                            |err| ProjectDiscoveryError::InvalidRequiresPythonConstraint {
+                                source: err,
+                                path: pyproject_path,
+                            },
+                        )?;
 
                 if has_knot_section {
                     tracing::debug!("Found project at '{}'", project_root);
@@ -191,6 +238,10 @@ impl ProjectMetadata {
         &self.options
     }
 
+    pub fn extra_configuration_paths(&self) -> &[SystemPathBuf] {
+        &self.extra_configuration_paths
+    }
+
     pub fn to_program_settings(&self, system: &dyn System) -> ProgramSettings {
         self.options.to_program_settings(self.root(), system)
     }
@@ -200,9 +251,31 @@ impl ProjectMetadata {
         self.options = options.combine(std::mem::take(&mut self.options));
     }
 
-    /// Combine the project options with the user options where project options take precedence.
-    pub fn apply_user_options(&mut self, options: Options) {
-        self.options.combine_with(options);
+    /// Applies the options from the configuration files to the project's options.
+    ///
+    /// This includes:
+    ///
+    /// * The user-level configuration
+    pub fn apply_configuration_files(
+        &mut self,
+        system: &dyn System,
+    ) -> Result<(), ConfigurationFileError> {
+        if let Some(user) = ConfigurationFile::user(system)? {
+            tracing::debug!(
+                "Applying user-level configuration loaded from `{path}`.",
+                path = user.path()
+            );
+            self.apply_configuration_file(user);
+        }
+
+        Ok(())
+    }
+
+    /// Applies a lower-precedence configuration files to the project's options.
+    fn apply_configuration_file(&mut self, options: ConfigurationFile) {
+        self.extra_configuration_paths
+            .push(options.path().to_owned());
+        self.options.combine_with(options.into_options());
     }
 }
 
@@ -222,16 +295,22 @@ pub enum ProjectDiscoveryError {
         source: Box<KnotTomlError>,
         path: SystemPathBuf,
     },
+
+    #[error("Invalid `requires-python` version specifier (`{path}`): {source}")]
+    InvalidRequiresPythonConstraint {
+        source: ResolveRequiresPythonError,
+        path: SystemPathBuf,
+    },
 }
 
 #[cfg(test)]
 mod tests {
     //! Integration tests for project discovery
 
-    use crate::snapshot_project;
     use anyhow::{anyhow, Context};
     use insta::assert_ron_snapshot;
     use ruff_db::system::{SystemPathBuf, TestSystem};
+    use ruff_python_ast::PythonVersion;
 
     use crate::{ProjectDiscoveryError, ProjectMetadata};
 
@@ -242,7 +321,7 @@ mod tests {
 
         system
             .memory_file_system()
-            .write_files([(root.join("foo.py"), ""), (root.join("bar.py"), "")])
+            .write_files_all([(root.join("foo.py"), ""), (root.join("bar.py"), "")])
             .context("Failed to write files")?;
 
         let project =
@@ -250,7 +329,15 @@ mod tests {
 
         assert_eq!(project.root(), &*root);
 
-        snapshot_project!(project);
+        with_escaped_paths(|| {
+            assert_ron_snapshot!(&project, @r#"
+                ProjectMetadata(
+                  name: Name("app"),
+                  root: "/app",
+                  options: Options(),
+                )
+            "#);
+        });
 
         Ok(())
     }
@@ -262,7 +349,7 @@ mod tests {
 
         system
             .memory_file_system()
-            .write_files([
+            .write_files_all([
                 (
                     root.join("pyproject.toml"),
                     r#"
@@ -279,7 +366,16 @@ mod tests {
             ProjectMetadata::discover(&root, &system).context("Failed to discover project")?;
 
         assert_eq!(project.root(), &*root);
-        snapshot_project!(project);
+
+        with_escaped_paths(|| {
+            assert_ron_snapshot!(&project, @r#"
+                ProjectMetadata(
+                  name: Name("backend"),
+                  root: "/app",
+                  options: Options(),
+                )
+            "#);
+        });
 
         // Discovering the same package from a subdirectory should give the same result
         let from_src = ProjectMetadata::discover(&root.join("db"), &system)
@@ -297,7 +393,7 @@ mod tests {
 
         system
             .memory_file_system()
-            .write_files([
+            .write_files_all([
                 (
                     root.join("pyproject.toml"),
                     r#"
@@ -336,7 +432,7 @@ expected `.`, `]`
 
         system
             .memory_file_system()
-            .write_files([
+            .write_files_all([
                 (
                     root.join("pyproject.toml"),
                     r#"
@@ -362,7 +458,19 @@ expected `.`, `]`
 
         let sub_project = ProjectMetadata::discover(&root.join("packages/a"), &system)?;
 
-        snapshot_project!(sub_project);
+        with_escaped_paths(|| {
+            assert_ron_snapshot!(sub_project, @r#"
+            ProjectMetadata(
+              name: Name("nested-project"),
+              root: "/app/packages/a",
+              options: Options(
+                src: Some(SrcOptions(
+                  root: Some("src"),
+                )),
+              ),
+            )
+            "#);
+        });
 
         Ok(())
     }
@@ -374,7 +482,7 @@ expected `.`, `]`
 
         system
             .memory_file_system()
-            .write_files([
+            .write_files_all([
                 (
                     root.join("pyproject.toml"),
                     r#"
@@ -400,7 +508,19 @@ expected `.`, `]`
 
         let root = ProjectMetadata::discover(&root, &system)?;
 
-        snapshot_project!(root);
+        with_escaped_paths(|| {
+            assert_ron_snapshot!(root, @r#"
+                              ProjectMetadata(
+                                name: Name("project-root"),
+                                root: "/app",
+                                options: Options(
+                                  src: Some(SrcOptions(
+                                    root: Some("src"),
+                                  )),
+                                ),
+                              )
+                              "#);
+        });
 
         Ok(())
     }
@@ -412,7 +532,7 @@ expected `.`, `]`
 
         system
             .memory_file_system()
-            .write_files([
+            .write_files_all([
                 (
                     root.join("pyproject.toml"),
                     r#"
@@ -432,7 +552,15 @@ expected `.`, `]`
 
         let sub_project = ProjectMetadata::discover(&root.join("packages/a"), &system)?;
 
-        snapshot_project!(sub_project);
+        with_escaped_paths(|| {
+            assert_ron_snapshot!(sub_project, @r#"
+            ProjectMetadata(
+              name: Name("nested-project"),
+              root: "/app/packages/a",
+              options: Options(),
+            )
+            "#);
+        });
 
         Ok(())
     }
@@ -444,7 +572,7 @@ expected `.`, `]`
 
         system
             .memory_file_system()
-            .write_files([
+            .write_files_all([
                 (
                     root.join("pyproject.toml"),
                     r#"
@@ -467,7 +595,19 @@ expected `.`, `]`
 
         let root = ProjectMetadata::discover(&root.join("packages/a"), &system)?;
 
-        snapshot_project!(root);
+        with_escaped_paths(|| {
+            assert_ron_snapshot!(root, @r#"
+            ProjectMetadata(
+              name: Name("project-root"),
+              root: "/app",
+              options: Options(
+                environment: Some(EnvironmentOptions(
+                  r#python-version: Some("3.10"),
+                )),
+              ),
+            )
+            "#);
+        });
 
         Ok(())
     }
@@ -483,31 +623,308 @@ expected `.`, `]`
 
         system
             .memory_file_system()
-            .write_files([
+            .write_files_all([
                 (
                     root.join("pyproject.toml"),
                     r#"
-                        [project]
-                        name = "super-app"
-                        requires-python = ">=3.12"
+                    [project]
+                    name = "super-app"
+                    requires-python = ">=3.12"
 
-                        [tool.knot.src]
-                        root = "this_option_is_ignored"
-                        "#,
+                    [tool.knot.src]
+                    root = "this_option_is_ignored"
+                    "#,
                 ),
                 (
                     root.join("knot.toml"),
                     r#"
-                        [src]
-                        root = "src"
-                        "#,
+                    [src]
+                    root = "src"
+                    "#,
                 ),
             ])
             .context("Failed to write files")?;
 
         let root = ProjectMetadata::discover(&root, &system)?;
 
-        snapshot_project!(root);
+        with_escaped_paths(|| {
+            assert_ron_snapshot!(root, @r#"
+            ProjectMetadata(
+              name: Name("super-app"),
+              root: "/app",
+              options: Options(
+                environment: Some(EnvironmentOptions(
+                  r#python-version: Some("3.12"),
+                )),
+                src: Some(SrcOptions(
+                  root: Some("src"),
+                )),
+              ),
+            )
+            "#);
+        });
+
+        Ok(())
+    }
+    #[test]
+    fn requires_python_major_minor() -> anyhow::Result<()> {
+        let system = TestSystem::default();
+        let root = SystemPathBuf::from("/app");
+
+        system
+            .memory_file_system()
+            .write_file_all(
+                root.join("pyproject.toml"),
+                r#"
+                [project]
+                requires-python = ">=3.12"
+                "#,
+            )
+            .context("Failed to write file")?;
+
+        let root = ProjectMetadata::discover(&root, &system)?;
+
+        assert_eq!(
+            root.options
+                .environment
+                .unwrap_or_default()
+                .python_version
+                .as_deref(),
+            Some(&PythonVersion::PY312)
+        );
+
+        Ok(())
+    }
+
+    #[test]
+    fn requires_python_major_only() -> anyhow::Result<()> {
+        let system = TestSystem::default();
+        let root = SystemPathBuf::from("/app");
+
+        system
+            .memory_file_system()
+            .write_file_all(
+                root.join("pyproject.toml"),
+                r#"
+                [project]
+                requires-python = ">=3"
+                "#,
+            )
+            .context("Failed to write file")?;
+
+        let root = ProjectMetadata::discover(&root, &system)?;
+
+        assert_eq!(
+            root.options
+                .environment
+                .unwrap_or_default()
+                .python_version
+                .as_deref(),
+            Some(&PythonVersion::from((3, 0)))
+        );
+
+        Ok(())
+    }
+
+    /// A `requires-python` constraint with major, minor and patch can be simplified
+    /// to major and minor (e.g. 3.12.1 -> 3.12).
+    #[test]
+    fn requires_python_major_minor_patch() -> anyhow::Result<()> {
+        let system = TestSystem::default();
+        let root = SystemPathBuf::from("/app");
+
+        system
+            .memory_file_system()
+            .write_file_all(
+                root.join("pyproject.toml"),
+                r#"
+                [project]
+                requires-python = ">=3.12.8"
+                "#,
+            )
+            .context("Failed to write file")?;
+
+        let root = ProjectMetadata::discover(&root, &system)?;
+
+        assert_eq!(
+            root.options
+                .environment
+                .unwrap_or_default()
+                .python_version
+                .as_deref(),
+            Some(&PythonVersion::PY312)
+        );
+
+        Ok(())
+    }
+
+    #[test]
+    fn requires_python_beta_version() -> anyhow::Result<()> {
+        let system = TestSystem::default();
+        let root = SystemPathBuf::from("/app");
+
+        system
+            .memory_file_system()
+            .write_file_all(
+                root.join("pyproject.toml"),
+                r#"
+                [project]
+                requires-python = ">= 3.13.0b0"
+                "#,
+            )
+            .context("Failed to write file")?;
+
+        let root = ProjectMetadata::discover(&root, &system)?;
+
+        assert_eq!(
+            root.options
+                .environment
+                .unwrap_or_default()
+                .python_version
+                .as_deref(),
+            Some(&PythonVersion::PY313)
+        );
+
+        Ok(())
+    }
+
+    #[test]
+    fn requires_python_greater_than_major_minor() -> anyhow::Result<()> {
+        let system = TestSystem::default();
+        let root = SystemPathBuf::from("/app");
+
+        system
+            .memory_file_system()
+            .write_file_all(
+                root.join("pyproject.toml"),
+                r#"
+                [project]
+                # This is somewhat nonsensical because 3.12.1 > 3.12 is true.
+                # That's why simplifying the constraint to >= 3.12 is correct
+                requires-python = ">3.12"
+                "#,
+            )
+            .context("Failed to write file")?;
+
+        let root = ProjectMetadata::discover(&root, &system)?;
+
+        assert_eq!(
+            root.options
+                .environment
+                .unwrap_or_default()
+                .python_version
+                .as_deref(),
+            Some(&PythonVersion::PY312)
+        );
+
+        Ok(())
+    }
+
+    /// `python-version` takes precedence if both `requires-python` and `python-version` are configured.
+    #[test]
+    fn requires_python_and_python_version() -> anyhow::Result<()> {
+        let system = TestSystem::default();
+        let root = SystemPathBuf::from("/app");
+
+        system
+            .memory_file_system()
+            .write_file_all(
+                root.join("pyproject.toml"),
+                r#"
+                [project]
+                requires-python = ">=3.12"
+
+                [tool.knot.environment]
+                python-version = "3.10"
+                "#,
+            )
+            .context("Failed to write file")?;
+
+        let root = ProjectMetadata::discover(&root, &system)?;
+
+        assert_eq!(
+            root.options
+                .environment
+                .unwrap_or_default()
+                .python_version
+                .as_deref(),
+            Some(&PythonVersion::PY310)
+        );
+
+        Ok(())
+    }
+
+    #[test]
+    fn requires_python_less_than() -> anyhow::Result<()> {
+        let system = TestSystem::default();
+        let root = SystemPathBuf::from("/app");
+
+        system
+            .memory_file_system()
+            .write_file_all(
+                root.join("pyproject.toml"),
+                r#"
+                [project]
+                requires-python = "<3.12"
+                "#,
+            )
+            .context("Failed to write file")?;
+
+        let Err(error) = ProjectMetadata::discover(&root, &system) else {
+            return Err(anyhow!("Expected project discovery to fail because the `requires-python` doesn't specify a lower bound (it only specifies an upper bound)."));
+        };
+
+        assert_error_eq(&error, "Invalid `requires-python` version specifier (`/app/pyproject.toml`): value `<3.12` does not contain a lower bound. Add a lower bound to indicate the minimum compatible Python version (e.g., `>=3.13`) or specify a version in `environment.python-version`.");
+
+        Ok(())
+    }
+
+    #[test]
+    fn requires_python_no_specifiers() -> anyhow::Result<()> {
+        let system = TestSystem::default();
+        let root = SystemPathBuf::from("/app");
+
+        system
+            .memory_file_system()
+            .write_file_all(
+                root.join("pyproject.toml"),
+                r#"
+                [project]
+                requires-python = ""
+                "#,
+            )
+            .context("Failed to write file")?;
+
+        let Err(error) = ProjectMetadata::discover(&root, &system) else {
+            return Err(anyhow!("Expected project discovery to fail because the `requires-python` specifiers are empty and don't define a lower bound."));
+        };
+
+        assert_error_eq(&error, "Invalid `requires-python` version specifier (`/app/pyproject.toml`): value `` does not contain a lower bound. Add a lower bound to indicate the minimum compatible Python version (e.g., `>=3.13`) or specify a version in `environment.python-version`.");
+
+        Ok(())
+    }
+
+    #[test]
+    fn requires_python_too_large_major_version() -> anyhow::Result<()> {
+        let system = TestSystem::default();
+        let root = SystemPathBuf::from("/app");
+
+        system
+            .memory_file_system()
+            .write_file_all(
+                root.join("pyproject.toml"),
+                r#"
+                [project]
+                requires-python = ">=999.0"
+                "#,
+            )
+            .context("Failed to write file")?;
+
+        let Err(error) = ProjectMetadata::discover(&root, &system) else {
+            return Err(anyhow!("Expected project discovery to fail because of the requires-python major version that is larger than 255."));
+        };
+
+        assert_error_eq(&error, "Invalid `requires-python` version specifier (`/app/pyproject.toml`): The major version `999` is larger than the maximum supported value 255");
 
         Ok(())
     }
@@ -517,15 +934,12 @@ expected `.`, `]`
         assert_eq!(error.to_string().replace('\\', "/"), message);
     }
 
-    /// Snapshots a project but with all paths using unix separators.
-    #[macro_export]
-    macro_rules! snapshot_project {
-    ($project:expr) => {{
-        assert_ron_snapshot!($project,{
-            ".root" => insta::dynamic_redaction(|content, _content_path| {
-                content.as_str().unwrap().replace("\\", "/")
-            }),
+    fn with_escaped_paths<R>(f: impl FnOnce() -> R) -> R {
+        let mut settings = insta::Settings::clone_current();
+        settings.add_dynamic_redaction(".root", |content, _path| {
+            content.as_str().unwrap().replace('\\', "/")
         });
-    }};
-}
+
+        settings.bind(f)
+    }
 }

crates/red_knot_project/src/metadata/configuration_file.rs

@@ -0,0 +1,69 @@
+use std::sync::Arc;
+
+use ruff_db::system::{System, SystemPath, SystemPathBuf};
+use thiserror::Error;
+
+use crate::metadata::value::ValueSource;
+
+use super::options::{KnotTomlError, Options};
+
+/// A `knot.toml` configuration file with the options it contains.
+pub(crate) struct ConfigurationFile {
+    path: SystemPathBuf,
+    options: Options,
+}
+
+impl ConfigurationFile {
+    /// Loads the user-level configuration file if it exists.
+    ///
+    /// Returns `None` if the file does not exist or if the concept of user-level configurations
+    /// doesn't exist on `system`.
+    pub(crate) fn user(system: &dyn System) -> Result<Option<Self>, ConfigurationFileError> {
+        let Some(configuration_directory) = system.user_config_directory() else {
+            return Ok(None);
+        };
+
+        let knot_toml_path = configuration_directory.join("knot").join("knot.toml");
+
+        tracing::debug!(
+            "Searching for a user-level configuration at `{path}`",
+            path = &knot_toml_path
+        );
+
+        let Ok(knot_toml_str) = system.read_to_string(&knot_toml_path) else {
+            return Ok(None);
+        };
+
+        match Options::from_toml_str(
+            &knot_toml_str,
+            ValueSource::File(Arc::new(knot_toml_path.clone())),
+        ) {
+            Ok(options) => Ok(Some(Self {
+                path: knot_toml_path,
+                options,
+            })),
+            Err(error) => Err(ConfigurationFileError::InvalidKnotToml {
+                source: Box::new(error),
+                path: knot_toml_path,
+            }),
+        }
+    }
+
+    /// Returns the path to the configuration file.
+    pub(crate) fn path(&self) -> &SystemPath {
+        &self.path
+    }
+
+    pub(crate) fn into_options(self) -> Options {
+        self.options
+    }
+}
+
+#[derive(Debug, Error)]
+pub enum ConfigurationFileError {
+    #[error("{path} is not a valid `knot.toml`: {source}")]
+    InvalidKnotToml {
+        source: Box<KnotTomlError>,
+        path: SystemPathBuf,
+    },
+}

crates/red_knot_project/src/metadata/options.rs

@@ -1,20 +1,20 @@
 use crate::metadata::value::{RangedValue, RelativePathBuf, ValueSource, ValueSourceGuard};
 use crate::Db;
 use red_knot_python_semantic::lint::{GetLintError, Level, LintSource, RuleSelection};
-use red_knot_python_semantic::{
-    ProgramSettings, PythonPlatform, PythonVersion, SearchPathSettings, SitePackages,
-};
-use ruff_db::diagnostic::{Diagnostic, DiagnosticId, Severity};
-use ruff_db::files::{system_path_to_file, File};
+use red_knot_python_semantic::{ProgramSettings, PythonPath, PythonPlatform, SearchPathSettings};
+use ruff_db::diagnostic::{DiagnosticId, OldDiagnosticTrait, Severity, Span};
+use ruff_db::files::system_path_to_file;
 use ruff_db::system::{System, SystemPath};
 use ruff_macros::Combine;
-use ruff_text_size::TextRange;
+use ruff_python_ast::PythonVersion;
 use rustc_hash::FxHashMap;
 use serde::{Deserialize, Serialize};
 use std::borrow::Cow;
 use std::fmt::Debug;
 use thiserror::Error;
 
+use super::settings::{Settings, TerminalSettings};
+
 /// The options for the project.
 #[derive(Debug, Default, Clone, PartialEq, Eq, Combine, Serialize, Deserialize)]
 #[serde(rename_all = "kebab-case", deny_unknown_fields)]
@@ -30,6 +30,9 @@ pub struct Options {
     /// Configures the enabled lints and their severity.
     #[serde(skip_serializing_if = "Option::is_none")]
     pub rules: Option<Rules>,
+
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub terminal: Option<TerminalOptions>,
 }
 
 impl Options {
@@ -87,7 +90,7 @@ impl Options {
             .map(|env| {
                 (
                     env.extra_paths.clone(),
-                    env.venv_path.clone(),
+                    env.python.clone(),
                     env.typeshed.clone(),
                 )
             })
@@ -101,16 +104,31 @@ impl Options {
                 .collect(),
             src_roots,
             custom_typeshed: typeshed.map(|path| path.absolute(project_root, system)),
-            site_packages: python
-                .map(|venv_path| SitePackages::Derived {
-                    venv_path: venv_path.absolute(project_root, system),
+            python_path: python
+                .map(|python_path| {
+                    PythonPath::SysPrefix(python_path.absolute(project_root, system))
                 })
-                .unwrap_or(SitePackages::Known(vec![])),
+                .unwrap_or(PythonPath::KnownSitePackages(vec![])),
         }
     }
 
     #[must_use]
-    pub(crate) fn to_rule_selection(&self, db: &dyn Db) -> (RuleSelection, Vec<OptionDiagnostic>) {
+    pub(crate) fn to_settings(&self, db: &dyn Db) -> (Settings, Vec<OptionDiagnostic>) {
+        let (rules, diagnostics) = self.to_rule_selection(db);
+
+        let mut settings = Settings::new(rules);
+
+        if let Some(terminal) = self.terminal.as_ref() {
+            settings.set_terminal(TerminalSettings {
+                error_on_warning: terminal.error_on_warning.unwrap_or_default(),
+            });
+        }
+
+        (settings, diagnostics)
+    }
+
+    #[must_use]
+    fn to_rule_selection(&self, db: &dyn Db) -> (RuleSelection, Vec<OptionDiagnostic>) {
         let registry = db.lint_registry();
         let mut diagnostics = Vec::new();
 
@@ -169,7 +187,14 @@ impl Options {
                         ),
                     };
 
-                    diagnostics.push(diagnostic.with_file(file).with_range(rule_name.range()));
+                    let span = file.map(Span::from).map(|span| {
+                        if let Some(range) = rule_name.range() {
+                            span.with_range(range)
+                        } else {
+                            span
+                        }
+                    });
+                    diagnostics.push(diagnostic.with_span(span));
                 }
             }
         }
@@ -211,10 +236,14 @@ pub struct EnvironmentOptions {
     #[serde(skip_serializing_if = "Option::is_none")]
     pub typeshed: Option<RelativePathBuf>,
 
-    // TODO: Rename to python, see https://github.com/astral-sh/ruff/issues/15530
-    /// The path to the user's `site-packages` directory, where third-party packages from ``PyPI`` are installed.
+    /// Path to the Python installation from which Red Knot resolves type information and third-party dependencies.
+    ///
+    /// Red Knot will search in the path's `site-packages` directories for type information and
+    /// third-party imports.
+    ///
+    /// This option is commonly used to specify the path to a virtual environment.
     #[serde(skip_serializing_if = "Option::is_none")]
-    pub venv_path: Option<RelativePathBuf>,
+    pub python: Option<RelativePathBuf>,
 }
 
 #[derive(Debug, Default, Clone, Eq, PartialEq, Combine, Serialize, Deserialize)]
@@ -244,6 +273,16 @@ impl FromIterator<(RangedValue<String>, RangedValue<Level>)> for Rules {
     }
 }
 
+#[derive(Debug, Default, Clone, Eq, PartialEq, Combine, Serialize, Deserialize)]
+#[serde(rename_all = "kebab-case", deny_unknown_fields)]
+#[cfg_attr(feature = "schemars", derive(schemars::JsonSchema))]
+pub struct TerminalOptions {
+    /// Use exit code 1 if there are any warning-level diagnostics.
+    ///
+    /// Defaults to `false`.
+    pub error_on_warning: Option<bool>,
+}
+
 #[cfg(feature = "schemars")]
 mod schema {
     use crate::DEFAULT_LINT_REGISTRY;
@@ -318,8 +357,7 @@ pub struct OptionDiagnostic {
     id: DiagnosticId,
     message: String,
     severity: Severity,
-    file: Option<File>,
-    range: Option<TextRange>,
+    span: Option<Span>,
 }
 
 impl OptionDiagnostic {
@@ -328,25 +366,17 @@ impl OptionDiagnostic {
             id,
             message,
             severity,
-            file: None,
-            range: None,
+            span: None,
         }
     }
 
     #[must_use]
-    fn with_file(mut self, file: Option<File>) -> Self {
-        self.file = file;
-        self
-    }
-
-    #[must_use]
-    fn with_range(mut self, range: Option<TextRange>) -> Self {
-        self.range = range;
-        self
+    fn with_span(self, span: Option<Span>) -> Self {
+        OptionDiagnostic { span, ..self }
     }
 }
 
-impl Diagnostic for OptionDiagnostic {
+impl OldDiagnosticTrait for OptionDiagnostic {
     fn id(&self) -> DiagnosticId {
         self.id
     }
@@ -355,12 +385,8 @@ impl Diagnostic for OptionDiagnostic {
         Cow::Borrowed(&self.message)
     }
 
-    fn file(&self) -> Option<File> {
-        self.file
-    }
-
-    fn range(&self) -> Option<TextRange> {
-        self.range
+    fn span(&self) -> Option<Span> {
+        self.span.clone()
     }
 
     fn severity(&self) -> Severity {

crates/red_knot_project/src/metadata/pyproject.rs

@@ -1,10 +1,11 @@
-use pep440_rs::{Version, VersionSpecifiers};
-use serde::{Deserialize, Deserializer, Serialize};
-use std::ops::Deref;
-use thiserror::Error;
-
 use crate::metadata::options::Options;
 use crate::metadata::value::{RangedValue, ValueSource, ValueSourceGuard};
+use pep440_rs::{release_specifiers_to_ranges, Version, VersionSpecifiers};
+use ruff_python_ast::PythonVersion;
+use serde::{Deserialize, Deserializer, Serialize};
+use std::collections::Bound;
+use std::ops::Deref;
+use thiserror::Error;
 
 /// A `pyproject.toml` as specified in PEP 517.
 #[derive(Deserialize, Serialize, Debug, Default, Clone)]
@@ -55,6 +56,73 @@ pub struct Project {
     pub requires_python: Option<RangedValue<VersionSpecifiers>>,
 }
 
+impl Project {
+    pub(super) fn resolve_requires_python_lower_bound(
+        &self,
+    ) -> Result<Option<RangedValue<PythonVersion>>, ResolveRequiresPythonError> {
+        let Some(requires_python) = self.requires_python.as_ref() else {
+            return Ok(None);
+        };
+
+        tracing::debug!("Resolving requires-python constraint: `{requires_python}`");
+
+        let ranges = release_specifiers_to_ranges((**requires_python).clone());
+        let Some((lower, _)) = ranges.bounding_range() else {
+            return Ok(None);
+        };
+
+        let version = match lower {
+            // Ex) `>=3.10.1` -> `>=3.10`
+            Bound::Included(version) => version,
+
+            // Ex) `>3.10.1` -> `>=3.10` or `>3.10` -> `>=3.10`
+            // The second example looks obscure at first but it is required because
+            // `3.10.1 > 3.10` is true but we only have two digits here. So including 3.10 is the
+            // right move. Overall, using `>` without a patch release is most likely bogus.
+            Bound::Excluded(version) => version,
+
+            // Ex) `<3.10` or ``
+            Bound::Unbounded => {
+                return Err(ResolveRequiresPythonError::NoLowerBound(
+                    requires_python.to_string(),
+                ))
+            }
+        };
+
+        // Take the major and minor version
+        let mut versions = version.release().iter().take(2);
+
+        let Some(major) = versions.next().copied() else {
+            return Ok(None);
+        };
+
+        let minor = versions.next().copied().unwrap_or_default();
+
+        tracing::debug!("Resolved requires-python constraint to: {major}.{minor}");
+
+        let major =
+            u8::try_from(major).map_err(|_| ResolveRequiresPythonError::TooLargeMajor(major))?;
+        let minor =
+            u8::try_from(minor).map_err(|_| ResolveRequiresPythonError::TooLargeMajor(minor))?;
+
+        Ok(Some(
+            requires_python
+                .clone()
+                .map_value(|_| PythonVersion::from((major, minor))),
+        ))
+    }
+}
+
+#[derive(Debug, Error)]
+pub enum ResolveRequiresPythonError {
+    #[error("The major version `{0}` is larger than the maximum supported value 255")]
+    TooLargeMajor(u64),
+    #[error("The minor version `{0}` is larger than the maximum supported value 255")]
+    TooLargeMinor(u64),
+    #[error("value `{0}` does not contain a lower bound. Add a lower bound to indicate the minimum compatible Python version (e.g., `>=3.13`) or specify a version in `environment.python-version`.")]
+    NoLowerBound(String),
+}
+
 #[derive(Deserialize, Serialize, Debug, Clone, PartialEq, Eq)]
 #[serde(rename_all = "kebab-case")]
 pub struct Tool {

crates/red_knot_project/src/metadata/settings.rs

@@ -0,0 +1,53 @@
+use std::sync::Arc;
+
+use red_knot_python_semantic::lint::RuleSelection;
+
+/// The resolved [`super::Options`] for the project.
+///
+/// Unlike [`super::Options`], the struct has default values filled in and
+/// uses representations that are optimized for reads (instead of preserving the source representation).
+/// It's also not required that this structure precisely resembles the TOML schema, although
+/// it's encouraged to use a similar structure.
+///
+/// It's worth considering to adding a salsa query for specific settings to
+/// limit the blast radius when only some settings change. For example,
+/// changing the terminal settings shouldn't invalidate any core type-checking queries.
+/// This can be achieved by adding a salsa query for the type checking specific settings.
+///
+/// Settings that are part of [`red_knot_python_semantic::ProgramSettings`] are not included here.
+#[derive(Clone, Debug, Eq, PartialEq)]
+pub struct Settings {
+    rules: Arc<RuleSelection>,
+
+    terminal: TerminalSettings,
+}
+
+impl Settings {
+    pub fn new(rules: RuleSelection) -> Self {
+        Self {
+            rules: Arc::new(rules),
+            terminal: TerminalSettings::default(),
+        }
+    }
+
+    pub fn rules(&self) -> &RuleSelection {
+        &self.rules
+    }
+
+    pub fn to_rules(&self) -> Arc<RuleSelection> {
+        self.rules.clone()
+    }
+
+    pub fn terminal(&self) -> &TerminalSettings {
+        &self.terminal
+    }
+
+    pub fn set_terminal(&mut self, terminal: TerminalSettings) {
+        self.terminal = terminal;
+    }
+}
+
+#[derive(Debug, Clone, PartialEq, Eq, Default)]
+pub struct TerminalSettings {
+    pub error_on_warning: bool,
+}

crates/red_knot_project/src/metadata/value.rs

@@ -118,6 +118,15 @@ impl<T> RangedValue<T> {
         self
     }
 
+    #[must_use]
+    pub fn map_value<R>(self, f: impl FnOnce(T) -> R) -> RangedValue<R> {
+        RangedValue {
+            value: f(self.value),
+            source: self.source,
+            range: self.range,
+        }
+    }
+
     pub fn into_inner(self) -> T {
         self.value
     }

crates/red_knot_project/src/snapshots/red_knot_project__metadata__tests__nested_projects_in_root_project.snap

@@ -1,13 +0,0 @@
----
-source: crates/red_knot_project/src/metadata.rs
-expression: root
----
-ProjectMetadata(
-  name: Name("project-root"),
-  root: "/app",
-  options: Options(
-    src: Some(SrcOptions(
-      root: Some("src"),
-    )),
-  ),
-)

crates/red_knot_project/src/snapshots/red_knot_project__metadata__tests__nested_projects_in_sub_project.snap

@@ -1,13 +0,0 @@
----
-source: crates/red_knot_project/src/metadata.rs
-expression: sub_project
----
-ProjectMetadata(
-  name: Name("nested-project"),
-  root: "/app/packages/a",
-  options: Options(
-    src: Some(SrcOptions(
-      root: Some("src"),
-    )),
-  ),
-)

crates/red_knot_project/src/snapshots/red_knot_project__metadata__tests__nested_projects_with_outer_knot_section.snap

@@ -1,13 +0,0 @@
----
-source: crates/red_knot_project/src/metadata.rs
-expression: root
----
-ProjectMetadata(
-  name: Name("project-root"),
-  root: "/app",
-  options: Options(
-    environment: Some(EnvironmentOptions(
-      r#python-version: Some("3.10"),
-    )),
-  ),
-)

crates/red_knot_project/src/snapshots/red_knot_project__metadata__tests__nested_projects_without_knot_sections.snap

@@ -1,9 +0,0 @@
----
-source: crates/red_knot_project/src/metadata.rs
-expression: sub_project
----
-ProjectMetadata(
-  name: Name("nested-project"),
-  root: "/app/packages/a",
-  options: Options(),
-)

crates/red_knot_project/src/snapshots/red_knot_project__metadata__tests__project_with_knot_and_pyproject_toml.snap

@@ -1,13 +0,0 @@
----
-source: crates/red_knot_project/src/metadata.rs
-expression: root
----
-ProjectMetadata(
-  name: Name("super-app"),
-  root: "/app",
-  options: Options(
-    src: Some(SrcOptions(
-      root: Some("src"),
-    )),
-  ),
-)

crates/red_knot_project/src/snapshots/red_knot_project__metadata__tests__project_with_pyproject.snap

@@ -1,9 +0,0 @@
----
-source: crates/red_knot_project/src/metadata.rs
-expression: project
----
-ProjectMetadata(
-  name: Name("backend"),
-  root: "/app",
-  options: Options(),
-)

crates/red_knot_project/src/snapshots/red_knot_project__metadata__tests__project_without_pyproject.snap

@@ -1,9 +0,0 @@
----
-source: crates/red_knot_project/src/metadata.rs
-expression: project
----
-ProjectMetadata(
-  name: Name("app"),
-  root: "/app",
-  options: Options(),
-)

crates/red_knot_project/src/walk.rs

@@ -0,0 +1,256 @@
+use crate::{Db, IOErrorDiagnostic, IOErrorKind, Project};
+use ruff_db::files::{system_path_to_file, File};
+use ruff_db::system::walk_directory::{ErrorKind, WalkDirectoryBuilder, WalkState};
+use ruff_db::system::{FileType, SystemPath, SystemPathBuf};
+use ruff_python_ast::PySourceType;
+use rustc_hash::{FxBuildHasher, FxHashSet};
+use std::path::PathBuf;
+use thiserror::Error;
+
+/// Filter that decides which files are included in the project.
+///
+/// In the future, this will hold a reference to the `include` and `exclude` pattern.
+///
+/// This struct mainly exists because `dyn Db` isn't `Send` or `Sync`, making it impossible
+/// to access fields from within the walker.
+#[derive(Default, Debug)]
+pub(crate) struct ProjectFilesFilter<'a> {
+    /// The same as [`Project::included_paths_or_root`].
+    included_paths: &'a [SystemPathBuf],
+
+    /// The filter skips checking if the path is in `included_paths` if set to `true`.
+    ///
+    /// Skipping this check is useful when the walker only walks over `included_paths`.
+    skip_included_paths: bool,
+}
+
+impl<'a> ProjectFilesFilter<'a> {
+    pub(crate) fn from_project(db: &'a dyn Db, project: Project) -> Self {
+        Self {
+            included_paths: project.included_paths_or_root(db),
+            skip_included_paths: false,
+        }
+    }
+
+    /// Returns `true` if a file is part of the project and included in the paths to check.
+    ///
+    /// A file is included in the checked files if it is a sub path of the project's root
+    /// (when no CLI path arguments are specified) or if it is a sub path of any path provided on the CLI (`knot check <paths>`) AND:
+    ///
+    /// * It matches a positive `include` pattern and isn't excluded by a later negative `include` pattern.
+    /// * It doesn't match a positive `exclude` pattern or is re-included by a later negative `exclude` pattern.
+    ///
+    /// ## Note
+    ///
+    /// This method may return `true` for files that don't end up being included when walking the
+    /// project tree because it doesn't consider `.gitignore` and other ignore files when deciding
+    /// if a file's included.
+    pub(crate) fn is_included(&self, path: &SystemPath) -> bool {
+        #[derive(Copy, Clone, Debug, PartialEq, Eq, PartialOrd, Ord)]
+        enum CheckPathMatch {
+            /// The path is a partial match of the checked path (it's a sub path)
+            Partial,
+
+            /// The path matches a check path exactly.
+            Full,
+        }
+
+        let m = if self.skip_included_paths {
+            Some(CheckPathMatch::Partial)
+        } else {
+            self.included_paths
+                .iter()
+                .filter_map(|included_path| {
+                    if let Ok(relative_path) = path.strip_prefix(included_path) {
+                        // Exact matches are always included
+                        if relative_path.as_str().is_empty() {
+                            Some(CheckPathMatch::Full)
+                        } else {
+                            Some(CheckPathMatch::Partial)
+                        }
+                    } else {
+                        None
+                    }
+                })
+                .max()
+        };
+
+        match m {
+            None => false,
+            Some(CheckPathMatch::Partial) => {
+                // TODO: For partial matches, only include the file if it is included by the project's include/exclude settings.
+                true
+            }
+            Some(CheckPathMatch::Full) => true,
+        }
+    }
+}
+
+pub(crate) struct ProjectFilesWalker<'a> {
+    walker: WalkDirectoryBuilder,
+
+    filter: ProjectFilesFilter<'a>,
+}
+
+impl<'a> ProjectFilesWalker<'a> {
+    pub(crate) fn new(db: &'a dyn Db) -> Self {
+        let project = db.project();
+
+        let mut filter = ProjectFilesFilter::from_project(db, project);
+        // It's unnecessary to filter on included paths because it only iterates over those to start with.
+        filter.skip_included_paths = true;
+
+        Self::from_paths(db, project.included_paths_or_root(db), filter)
+            .expect("included_paths_or_root to never return an empty iterator")
+    }
+
+    /// Creates a walker for indexing the project files incrementally.
+    ///
+    /// The main difference to a full project walk is that `paths` may contain paths
+    /// that aren't part of the included files.
+    pub(crate) fn incremental<P>(db: &'a dyn Db, paths: impl IntoIterator<Item = P>) -> Option<Self>
+    where
+        P: AsRef<SystemPath>,
+    {
+        let project = db.project();
+
+        let filter = ProjectFilesFilter::from_project(db, project);
+
+        Self::from_paths(db, paths, filter)
+    }
+
+    fn from_paths<P>(
+        db: &'a dyn Db,
+        paths: impl IntoIterator<Item = P>,
+        filter: ProjectFilesFilter<'a>,
+    ) -> Option<Self>
+    where
+        P: AsRef<SystemPath>,
+    {
+        let mut paths = paths.into_iter();
+
+        let mut walker = db.system().walk_directory(paths.next()?.as_ref());
+
+        for path in paths {
+            walker = walker.add(path);
+        }
+
+        Some(Self { walker, filter })
+    }
+
+    /// Walks the project paths and collects the paths of all files that
+    /// are included in the project.
+    pub(crate) fn walk_paths(self) -> (Vec<SystemPathBuf>, Vec<IOErrorDiagnostic>) {
+        let paths = std::sync::Mutex::new(Vec::new());
+        let diagnostics = std::sync::Mutex::new(Vec::new());
+
+        self.walker.run(|| {
+            Box::new(|entry| {
+                match entry {
+                    Ok(entry) => {
+                        if !self.filter.is_included(entry.path()) {
+                            tracing::debug!("Ignoring not-included path: {}", entry.path());
+                            return WalkState::Skip;
+                        }
+
+                        // Skip over any non python files to avoid creating too many entries in `Files`.
+                        match entry.file_type() {
+                            FileType::File => {
+                                if entry
+                                    .path()
+                                    .extension()
+                                    .and_then(PySourceType::try_from_extension)
+                                    .is_some()
+                                {
+                                    let mut paths = paths.lock().unwrap();
+                                    paths.push(entry.into_path());
+                                }
+                            }
+                            FileType::Directory | FileType::Symlink => {}
+                        }
+                    }
+                    Err(error) => match error.kind() {
+                        ErrorKind::Loop { .. } => {
+                            unreachable!("Loops shouldn't be possible without following symlinks.")
+                        }
+                        ErrorKind::Io { path, err } => {
+                            let mut diagnostics = diagnostics.lock().unwrap();
+                            let error = if let Some(path) = path {
+                                WalkError::IOPathError {
+                                    path: path.clone(),
+                                    error: err.to_string(),
+                                }
+                            } else {
+                                WalkError::IOError {
+                                    error: err.to_string(),
+                                }
+                            };
+
+                            diagnostics.push(IOErrorDiagnostic {
+                                file: None,
+                                error: IOErrorKind::Walk(error),
+                            });
+                        }
+                        ErrorKind::NonUtf8Path { path } => {
+                            diagnostics.lock().unwrap().push(IOErrorDiagnostic {
+                                file: None,
+                                error: IOErrorKind::Walk(WalkError::NonUtf8Path {
+                                    path: path.clone(),
+                                }),
+                            });
+                        }
+                    },
+                }
+
+                WalkState::Continue
+            })
+        });
+
+        (
+            paths.into_inner().unwrap(),
+            diagnostics.into_inner().unwrap(),
+        )
+    }
+
+    pub(crate) fn collect_vec(self, db: &dyn Db) -> (Vec<File>, Vec<IOErrorDiagnostic>) {
+        let (paths, diagnostics) = self.walk_paths();
+
+        (
+            paths
+                .into_iter()
+                .filter_map(move |path| {
+                    // If this returns `None`, then the file was deleted between the `walk_directory` call and now.
+                    // We can ignore this.
+                    system_path_to_file(db.upcast(), &path).ok()
+                })
+                .collect(),
+            diagnostics,
+        )
+    }
+
+    pub(crate) fn collect_set(self, db: &dyn Db) -> (FxHashSet<File>, Vec<IOErrorDiagnostic>) {
+        let (paths, diagnostics) = self.walk_paths();
+
+        let mut files = FxHashSet::with_capacity_and_hasher(paths.len(), FxBuildHasher);
+
+        for path in paths {
+            if let Ok(file) = system_path_to_file(db.upcast(), &path) {
+                files.insert(file);
+            }
+        }
+
+        (files, diagnostics)
+    }
+}
+
+#[derive(Error, Debug, Clone)]
+pub(crate) enum WalkError {
+    #[error("`{path}`: {error}")]
+    IOPathError { path: SystemPathBuf, error: String },
+
+    #[error("Failed to walk project directory: {error}")]
+    IOError { error: String },
+
+    #[error("`{path}` is not a valid UTF-8 path")]
+    NonUtf8Path { path: PathBuf },
+}

crates/red_knot_project/src/watch/project_watcher.rs

@@ -6,7 +6,7 @@ use tracing::info;
 use red_knot_python_semantic::system_module_search_paths;
 use ruff_cache::{CacheKey, CacheKeyHasher};
 use ruff_db::system::{SystemPath, SystemPathBuf};
-use ruff_db::{Db as _, Upcast};
+use ruff_db::Upcast;
 
 use crate::db::{Db, ProjectDatabase};
 use crate::watch::Watcher;
@@ -42,9 +42,9 @@ impl ProjectWatcher {
 
     pub fn update(&mut self, db: &ProjectDatabase) {
         let search_paths: Vec<_> = system_module_search_paths(db.upcast()).collect();
-        let project_path = db.project().root(db).to_path_buf();
+        let project_path = db.project().root(db);
 
-        let new_cache_key = Self::compute_cache_key(&project_path, &search_paths);
+        let new_cache_key = Self::compute_cache_key(project_path, &search_paths);
 
         if self.cache_key == Some(new_cache_key) {
             return;
@@ -68,31 +68,47 @@ impl ProjectWatcher {
 
         self.has_errored_paths = false;
 
-        let project_path = db
-            .system()
-            .canonicalize_path(&project_path)
-            .unwrap_or(project_path);
+        let config_paths = db
+            .project()
+            .metadata(db)
+            .extra_configuration_paths()
+            .iter()
+            .map(SystemPathBuf::as_path);
+
+        // Watch both the project root and any paths provided by the user on the CLI (removing any redundant nested paths).
+        // This is necessary to observe changes to files that are outside the project root.
+        // We always need to watch the project root to observe changes to its configuration.
+        let included_paths = ruff_db::system::deduplicate_nested_paths(
+            std::iter::once(project_path).chain(
+                db.project()
+                    .included_paths_list(db)
+                    .iter()
+                    .map(SystemPathBuf::as_path),
+            ),
+        );
 
         // Find the non-overlapping module search paths and filter out paths that are already covered by the project.
         // Module search paths are already canonicalized.
         let unique_module_paths = ruff_db::system::deduplicate_nested_paths(
             search_paths
                 .into_iter()
-                .filter(|path| !path.starts_with(&project_path)),
-        )
-        .map(SystemPath::to_path_buf);
+                .filter(|path| !path.starts_with(project_path)),
+        );
 
         // Now add the new paths, first starting with the project path and then
-        // adding the library search paths.
-        for path in std::iter::once(project_path).chain(unique_module_paths) {
+        // adding the library search paths, and finally the paths for configurations.
+        for path in included_paths
+            .chain(unique_module_paths)
+            .chain(config_paths)
+        {
             // Log a warning. It's not worth aborting if registering a single folder fails because
             // Ruff otherwise stills works as expected.
-            if let Err(error) = self.watcher.watch(&path) {
+            if let Err(error) = self.watcher.watch(path) {
                 // TODO: Log a user-facing warning.
                 tracing::warn!("Failed to setup watcher for path `{path}`: {error}. You have to restart Ruff after making changes to files under this path or you might see stale results.");
                 self.has_errored_paths = true;
             } else {
-                self.watched_paths.push(path);
+                self.watched_paths.push(path.to_path_buf());
             }
         }
 

crates/red_knot_project/tests/check.rs

@@ -117,7 +117,7 @@ fn run_corpus_tests(pattern: &str) -> anyhow::Result<()> {
         let code = std::fs::read_to_string(source)?;
 
         let mut check_with_file_name = |path: &SystemPath| {
-            memory_fs.write_file(path, &code).unwrap();
+            memory_fs.write_file_all(path, &code).unwrap();
             File::sync_path(&mut db, path);
 
             // this test is only asserting that we can pull every expression type without a panic
@@ -216,6 +216,17 @@ impl SourceOrderVisitor<'_> for PullTypesVisitor<'_> {
                 self.visit_body(&for_stmt.orelse);
                 return;
             }
+            Stmt::With(with_stmt) => {
+                for item in &with_stmt.items {
+                    if let Some(target) = &item.optional_vars {
+                        self.visit_target(target);
+                    }
+                    self.visit_expr(&item.context_expr);
+                }
+
+                self.visit_body(&with_stmt.body);
+                return;
+            }
             Stmt::AnnAssign(_)
             | Stmt::Return(_)
             | Stmt::Delete(_)
@@ -223,7 +234,6 @@ impl SourceOrderVisitor<'_> for PullTypesVisitor<'_> {
             | Stmt::TypeAlias(_)
             | Stmt::While(_)
             | Stmt::If(_)
-            | Stmt::With(_)
             | Stmt::Match(_)
             | Stmt::Raise(_)
             | Stmt::Try(_)
@@ -283,4 +293,9 @@ const KNOWN_FAILURES: &[(&str, bool, bool)] = &[
     // related to circular references in f-string annotations (invalid syntax)
     ("crates/ruff_linter/resources/test/fixtures/pyflakes/F821_15.py", true, true),
     ("crates/ruff_linter/resources/test/fixtures/pyflakes/F821_14.py", false, true),
+    // related to circular references in stub type annotations (salsa cycle panic):
+    ("crates/ruff_linter/resources/test/fixtures/pycodestyle/E501_4.py", false, true),
+    ("crates/ruff_linter/resources/test/fixtures/pyflakes/F401_0.py", false, true),
+    ("crates/ruff_linter/resources/test/fixtures/pyflakes/F401_12.py", false, true),
+    ("crates/ruff_linter/resources/test/fixtures/pyflakes/F401_14.py", false, true),
 ];

crates/red_knot_python_semantic/Cargo.toml

@@ -12,9 +12,9 @@ license = { workspace = true }
 
 [dependencies]
 ruff_db = { workspace = true }
-ruff_index = { workspace = true }
+ruff_index = { workspace = true, features = ["salsa"] }
 ruff_macros = { workspace = true }
-ruff_python_ast = { workspace = true }
+ruff_python_ast = { workspace = true, features = ["salsa"] }
 ruff_python_parser = { workspace = true }
 ruff_python_stdlib = { workspace = true }
 ruff_source_file = { workspace = true }
@@ -31,7 +31,7 @@ drop_bomb = { workspace = true }
 indexmap = { workspace = true }
 itertools = { workspace = true }
 ordermap = { workspace = true }
-salsa = { workspace = true }
+salsa = { workspace = true, features = ["compact_str"] }
 thiserror = { workspace = true }
 tracing = { workspace = true }
 rustc-hash = { workspace = true }
@@ -42,9 +42,11 @@ smallvec = { workspace = true }
 static_assertions = { workspace = true }
 test-case = { workspace = true }
 memchr = { workspace = true }
+strum = { workspace = true}
+strum_macros = { workspace = true}
 
 [dev-dependencies]
-ruff_db = { workspace = true, features = ["os", "testing"] }
+ruff_db = { workspace = true, features = ["testing", "os"] }
 ruff_python_parser = { workspace = true }
 red_knot_test = { workspace = true }
 red_knot_vendored = { workspace = true }
@@ -57,7 +59,7 @@ quickcheck = { version = "1.0.3", default-features = false }
 quickcheck_macros = { version = "1.0.0" }
 
 [features]
-serde = ["ruff_db/serde", "dep:serde"]
+serde = ["ruff_db/serde", "dep:serde", "ruff_python_ast/serde"]
 
 [lints]
 workspace = true

crates/red_knot_python_semantic/resources/mdtest/annotations/callable.md

@@ -0,0 +1,195 @@
+# Callable
+
+References:
+
+- <https://typing.readthedocs.io/en/latest/spec/callables.html#callable>
+
+TODO: Use `collections.abc` as importing from `typing` is deprecated but this requires support for
+`*` imports. See: <https://docs.python.org/3/library/typing.html#deprecated-aliases>.
+
+## Invalid forms
+
+The `Callable` special form requires _exactly_ two arguments where the first argument is either a
+parameter type list, parameter specification, `typing.Concatenate`, or `...` and the second argument
+is the return type. Here, we explore various invalid forms.
+
+### Empty
+
+A bare `Callable` without any type arguments:
+
+```py
+from typing import Callable
+
+def _(c: Callable):
+    reveal_type(c)  # revealed: (...) -> Unknown
+```
+
+### Invalid parameter type argument
+
+When it's not a list:
+
+```py
+from typing import Callable
+
+# error: [invalid-type-form] "The first argument to `Callable` must be either a list of types, ParamSpec, Concatenate, or `...`"
+def _(c: Callable[int, str]):
+    reveal_type(c)  # revealed: (...) -> Unknown
+```
+
+Or, when it's a literal type:
+
+```py
+# error: [invalid-type-form] "The first argument to `Callable` must be either a list of types, ParamSpec, Concatenate, or `...`"
+def _(c: Callable[42, str]):
+    reveal_type(c)  # revealed: (...) -> Unknown
+```
+
+Or, when one of the parameter type is invalid in the list:
+
+```py
+def _(c: Callable[[int, 42, str, False], None]):
+    # revealed: (int, @Todo(number literal in type expression), str, @Todo(boolean literal in type expression), /) -> None
+    reveal_type(c)
+```
+
+### Missing return type
+
+Using a parameter list:
+
+```py
+from typing import Callable
+
+# error: [invalid-type-form] "Special form `typing.Callable` expected exactly two arguments (parameter types and return type)"
+def _(c: Callable[[int, str]]):
+    reveal_type(c)  # revealed: (int, str, /) -> Unknown
+```
+
+Or, an ellipsis:
+
+```py
+# error: [invalid-type-form] "Special form `typing.Callable` expected exactly two arguments (parameter types and return type)"
+def _(c: Callable[...]):
+    reveal_type(c)  # revealed: (...) -> Unknown
+```
+
+### More than two arguments
+
+We can't reliably infer the callable type if there are more then 2 arguments because we don't know
+which argument corresponds to either the parameters or the return type.
+
+```py
+from typing import Callable
+
+# error: [invalid-type-form] "Special form `typing.Callable` expected exactly two arguments (parameter types and return type)"
+def _(c: Callable[[int], str, str]):
+    reveal_type(c)  # revealed: (...) -> Unknown
+```
+
+## Simple
+
+A simple `Callable` with multiple parameters and a return type:
+
+```py
+from typing import Callable
+
+def _(c: Callable[[int, str], int]):
+    reveal_type(c)  # revealed: (int, str, /) -> int
+```
+
+## Nested
+
+A nested `Callable` as one of the parameter types:
+
+```py
+from typing import Callable
+
+def _(c: Callable[[Callable[[int], str]], int]):
+    reveal_type(c)  # revealed: ((int, /) -> str, /) -> int
+```
+
+And, as the return type:
+
+```py
+def _(c: Callable[[int, str], Callable[[int], int]]):
+    reveal_type(c)  # revealed: (int, str, /) -> (int, /) -> int
+```
+
+## Gradual form
+
+The `Callable` special form supports the use of `...` in place of the list of parameter types. This
+is a [gradual form] indicating that the type is consistent with any input signature:
+
+```py
+from typing import Callable
+
+def gradual_form(c: Callable[..., str]):
+    reveal_type(c)  # revealed: (...) -> str
+```
+
+## Using `typing.Concatenate`
+
+Using `Concatenate` as the first argument to `Callable`:
+
+```py
+from typing_extensions import Callable, Concatenate
+
+def _(c: Callable[Concatenate[int, str, ...], int]):
+    reveal_type(c)  # revealed: (*args: @Todo(todo signature *args), **kwargs: @Todo(todo signature **kwargs)) -> int
+```
+
+And, as one of the parameter types:
+
+```py
+def _(c: Callable[[Concatenate[int, str, ...], int], int]):
+    reveal_type(c)  # revealed: (*args: @Todo(todo signature *args), **kwargs: @Todo(todo signature **kwargs)) -> int
+```
+
+## Using `typing.ParamSpec`
+
+Using a `ParamSpec` in a `Callable` annotation:
+
+```py
+from typing_extensions import Callable
+
+# TODO: Not an error; remove once `ParamSpec` is supported
+# error: [invalid-type-form]
+def _[**P1](c: Callable[P1, int]):
+    reveal_type(c)  # revealed: (...) -> Unknown
+```
+
+And, using the legacy syntax:
+
+```py
+from typing_extensions import ParamSpec
+
+P2 = ParamSpec("P2")
+
+# TODO: Not an error; remove once `ParamSpec` is supported
+# error: [invalid-type-form]
+def _(c: Callable[P2, int]):
+    reveal_type(c)  # revealed: (...) -> Unknown
+```
+
+## Using `typing.Unpack`
+
+Using the unpack operator (`*`):
+
+```py
+from typing_extensions import Callable, TypeVarTuple
+
+Ts = TypeVarTuple("Ts")
+
+def _(c: Callable[[int, *Ts], int]):
+    reveal_type(c)  # revealed: (*args: @Todo(todo signature *args), **kwargs: @Todo(todo signature **kwargs)) -> int
+```
+
+And, using the legacy syntax using `Unpack`:
+
+```py
+from typing_extensions import Unpack
+
+def _(c: Callable[[int, Unpack[Ts]], int]):
+    reveal_type(c)  # revealed: (*args: @Todo(todo signature *args), **kwargs: @Todo(todo signature **kwargs)) -> int
+```
+
+[gradual form]: https://typing.readthedocs.io/en/latest/spec/glossary.html#term-gradual-form

crates/red_knot_python_semantic/resources/mdtest/annotations/int_float_complex.md

@@ -0,0 +1,90 @@
+# Special cases for int/float/complex in annotations
+
+In order to support common use cases, an annotation of `float` actually means `int | float`, and an
+annotation of `complex` actually means `int | float | complex`. See
+[the specification](https://typing.readthedocs.io/en/latest/spec/special-types.html#special-cases-for-float-and-complex)
+
+## float
+
+An annotation of `float` means `int | float`, so `int` is assignable to it:
+
+```py
+def takes_float(x: float):
+    pass
+
+def passes_int_to_float(x: int):
+    # no error!
+    takes_float(x)
+```
+
+It also applies to variable annotations:
+
+```py
+def assigns_int_to_float(x: int):
+    # no error!
+    y: float = x
+```
+
+It doesn't work the other way around:
+
+```py
+def takes_int(x: int):
+    pass
+
+def passes_float_to_int(x: float):
+    # error: [invalid-argument-type]
+    takes_int(x)
+
+def assigns_float_to_int(x: float):
+    # error: [invalid-assignment]
+    y: int = x
+```
+
+Unlike other type checkers, we choose not to obfuscate this special case by displaying `int | float`
+as just `float`; we display the actual type:
+
+```py
+def f(x: float):
+    reveal_type(x)  # revealed: int | float
+```
+
+## complex
+
+An annotation of `complex` means `int | float | complex`, so `int` and `float` are both assignable
+to it (but not the other way around):
+
+```py
+def takes_complex(x: complex):
+    pass
+
+def passes_to_complex(x: float, y: int):
+    # no errors!
+    takes_complex(x)
+    takes_complex(y)
+
+def assigns_to_complex(x: float, y: int):
+    # no errors!
+    a: complex = x
+    b: complex = y
+
+def takes_int(x: int):
+    pass
+
+def takes_float(x: float):
+    pass
+
+def passes_complex(x: complex):
+    # error: [invalid-argument-type]
+    takes_int(x)
+    # error: [invalid-argument-type]
+    takes_float(x)
+
+def assigns_complex(x: complex):
+    # error: [invalid-assignment]
+    y: int = x
+    # error: [invalid-assignment]
+    z: float = x
+
+def f(x: complex):
+    reveal_type(x)  # revealed: int | float | complex
+```

crates/red_knot_python_semantic/resources/mdtest/annotations/invalid.md

@@ -0,0 +1,45 @@
+# Tests for invalid types in type expressions
+
+## Invalid types are rejected
+
+Many types are illegal in the context of a type expression:
+
+```py
+import typing
+from knot_extensions import AlwaysTruthy, AlwaysFalsy
+from typing_extensions import Literal, Never
+
+def _(
+    a: type[int],
+    b: AlwaysTruthy,
+    c: AlwaysFalsy,
+    d: Literal[True],
+    e: Literal["bar"],
+    f: Literal[b"foo"],
+    g: tuple[int, str],
+    h: Never,
+):
+    def foo(): ...
+    def invalid(
+        i: a,  # error: [invalid-type-form] "Variable of type `type[int]` is not allowed in a type expression"
+        j: b,  # error: [invalid-type-form]
+        k: c,  # error: [invalid-type-form]
+        l: d,  # error: [invalid-type-form]
+        m: e,  # error: [invalid-type-form]
+        n: f,  # error: [invalid-type-form]
+        o: g,  # error: [invalid-type-form]
+        p: h,  # error: [invalid-type-form]
+        q: typing,  # error: [invalid-type-form]
+        r: foo,  # error: [invalid-type-form]
+    ):
+        reveal_type(i)  # revealed: Unknown
+        reveal_type(j)  # revealed: Unknown
+        reveal_type(k)  # revealed: Unknown
+        reveal_type(l)  # revealed: Unknown
+        reveal_type(m)  # revealed: Unknown
+        reveal_type(n)  # revealed: Unknown
+        reveal_type(o)  # revealed: Unknown
+        reveal_type(p)  # revealed: Unknown
+        reveal_type(q)  # revealed: Unknown
+        reveal_type(r)  # revealed: Unknown
+```

crates/red_knot_python_semantic/resources/mdtest/annotations/literal_string.md

@@ -73,12 +73,12 @@ qux = (foo, bar)
 reveal_type(qux)  # revealed: tuple[Literal["foo"], Literal["bar"]]
 
 # TODO: Infer "LiteralString"
-reveal_type(foo.join(qux))  # revealed: @Todo(Attribute access on `StringLiteral` types)
+reveal_type(foo.join(qux))  # revealed: @Todo(return type of decorated function)
 
 template: LiteralString = "{}, {}"
 reveal_type(template)  # revealed: Literal["{}, {}"]
 # TODO: Infer `LiteralString`
-reveal_type(template.format(foo, bar))  # revealed: @Todo(Attribute access on `StringLiteral` types)
+reveal_type(template.format(foo, bar))  # revealed: @Todo(return type of decorated function)
 ```
 
 ### Assignability

crates/red_knot_python_semantic/resources/mdtest/annotations/stdlib_typing_aliases.md

@@ -70,8 +70,7 @@ import typing
 
 class ListSubclass(typing.List): ...
 
-# TODO: should have `Generic`, should not have `Unknown`
-# revealed: tuple[Literal[ListSubclass], Literal[list], Unknown, Literal[object]]
+# revealed: tuple[Literal[ListSubclass], Literal[list], Literal[MutableSequence], Literal[Sequence], Literal[Reversible], Literal[Collection], Literal[Iterable], Literal[Container], @Todo(protocol), Literal[object]]
 reveal_type(ListSubclass.__mro__)
 
 class DictSubclass(typing.Dict): ...

crates/red_knot_python_semantic/resources/mdtest/annotations/string.md

@@ -116,8 +116,8 @@ MyType = int
 class Aliases:
     MyType = str
 
-    forward: "MyType"
-    not_forward: MyType
+    forward: "MyType" = "value"
+    not_forward: MyType = "value"
 
 reveal_type(Aliases.forward)  # revealed: str
 reveal_type(Aliases.not_forward)  # revealed: str

crates/red_knot_python_semantic/resources/mdtest/annotations/union.md

@@ -9,9 +9,9 @@ from typing import Union
 
 a: Union[int, str]
 a1: Union[int, bool]
-a2: Union[int, Union[float, str]]
+a2: Union[int, Union[bytes, str]]
 a3: Union[int, None]
-a4: Union[Union[float, str]]
+a4: Union[Union[bytes, str]]
 a5: Union[int]
 a6: Union[()]
 
@@ -21,11 +21,11 @@ def f():
     # Since bool is a subtype of int we simplify to int here. But we do allow assigning boolean values (see below).
     # revealed: int
     reveal_type(a1)
-    # revealed: int | float | str
+    # revealed: int | bytes | str
     reveal_type(a2)
     # revealed: int | None
     reveal_type(a3)
-    # revealed: float | str
+    # revealed: bytes | str
     reveal_type(a4)
     # revealed: int
     reveal_type(a5)

crates/red_knot_python_semantic/resources/mdtest/annotations/unsupported_special_forms.md

@@ -18,7 +18,7 @@ def f(*args: Unpack[Ts]) -> tuple[Unpack[Ts]]:
     # TODO: should understand the annotation
     reveal_type(args)  # revealed: tuple
 
-    reveal_type(Alias)  # revealed: @Todo(Unsupported or invalid type in a type expression)
+    reveal_type(Alias)  # revealed: @Todo(Invalid or unsupported `KnownInstanceType` in `Type::to_type_expression`)
 
 def g() -> TypeGuard[int]: ...
 def h() -> TypeIs[int]: ...
@@ -29,11 +29,13 @@ def i(callback: Callable[Concatenate[int, P], R_co], *args: P.args, **kwargs: P.
     # TODO: should understand the annotation
     reveal_type(kwargs)  # revealed: dict
 
+    # TODO: not an error; remove once `call` is implemented for `Callable`
+    # error: [call-non-callable]
     return callback(42, *args, **kwargs)
 
 class Foo:
     def method(self, x: Self):
-        reveal_type(x)  # revealed: @Todo(Unsupported or invalid type in a type expression)
+        reveal_type(x)  # revealed: @Todo(Invalid or unsupported `KnownInstanceType` in `Type::to_type_expression`)
 ```
 
 ## Inheritance

crates/red_knot_python_semantic/resources/mdtest/assignment/augmented.md

@@ -9,7 +9,7 @@ reveal_type(x)  # revealed: Literal[2]
 
 x = 1.0
 x /= 2
-reveal_type(x)  # revealed: float
+reveal_type(x)  # revealed: int | float
 ```
 
 ## Dunder methods
@@ -24,12 +24,12 @@ x -= 1
 reveal_type(x)  # revealed: str
 
 class C:
-    def __iadd__(self, other: str) -> float:
-        return 1.0
+    def __iadd__(self, other: str) -> int:
+        return 1
 
 x = C()
 x += "Hello"
-reveal_type(x)  # revealed: float
+reveal_type(x)  # revealed: int
 ```
 
 ## Unsupported types
@@ -40,7 +40,7 @@ class C:
         return 42
 
 x = C()
-# error: [invalid-argument-type]
+# error: [unsupported-operator] "Operator `-=` is unsupported between objects of type `C` and `Literal[1]`"
 x -= 1
 
 reveal_type(x)  # revealed: int
@@ -75,8 +75,7 @@ def _(flag: bool):
 
     f = Foo()
 
-    # TODO: We should emit an `unsupported-operator` error here, possibly with the information
-    # that `Foo.__iadd__` may be unbound as additional context.
+    # error: [unsupported-operator] "Operator `+=` is unsupported between objects of type `Foo` and `Literal["Hello, world!"]`"
     f += "Hello, world!"
 
     reveal_type(f)  # revealed: int | Unknown
@@ -130,10 +129,10 @@ def _(flag: bool):
     if flag:
         f = Foo()
     else:
-        f = 42.0
+        f = 42
     f += 12
 
-    reveal_type(f)  # revealed: str | float
+    reveal_type(f)  # revealed: str | Literal[54]
 ```
 
 ## Partially bound target union with `__add__`

crates/red_knot_python_semantic/resources/mdtest/attributes.md

@@ -30,7 +30,11 @@ reveal_type(c_instance.inferred_from_value)  # revealed: Unknown | Literal[1, "a
 # TODO: Same here. This should be `Unknown | Literal[1, "a"]`
 reveal_type(c_instance.inferred_from_other_attribute)  # revealed: Unknown
 
-# TODO: should be `int | None`
+# There is no special handling of attributes that are (directly) assigned to a declared parameter,
+# which means we union with `Unknown` here, since the attribute itself is not declared. This is
+# something that we might want to change in the future.
+#
+# See https://github.com/astral-sh/ruff/issues/15960 for a related discussion.
 reveal_type(c_instance.inferred_from_param)  # revealed: Unknown | int | None
 
 reveal_type(c_instance.declared_only)  # revealed: bytes
@@ -45,18 +49,17 @@ reveal_type(c_instance.possibly_undeclared_unbound)  # revealed: str
 c_instance.inferred_from_value = "value set on instance"
 
 # This assignment is also fine:
-c_instance.inferred_from_param = None
+c_instance.declared_and_bound = False
 
-# TODO: this should be an error (incompatible types in assignment)
-c_instance.inferred_from_param = "incompatible"
+# error: [invalid-assignment] "Object of type `Literal["incompatible"]` is not assignable to attribute `declared_and_bound` of type `bool`"
+c_instance.declared_and_bound = "incompatible"
 
-# TODO: we already show an error here but the message might be improved?
 # mypy shows no error here, but pyright raises "reportAttributeAccessIssue"
-# error: [unresolved-attribute] "Type `Literal[C]` has no attribute `inferred_from_value`"
+# error: [unresolved-attribute] "Attribute `inferred_from_value` can only be accessed on instances, not on the class object `Literal[C]` itself."
 reveal_type(C.inferred_from_value)  # revealed: Unknown
 
-# TODO: this should be an error (pure instance variables cannot be accessed on the class)
 # mypy shows no error here, but pyright raises "reportAttributeAccessIssue"
+# error: [invalid-attribute-access] "Cannot assign to instance attribute `inferred_from_value` from the class object `Literal[C]`"
 C.inferred_from_value = "overwritten on class"
 
 # This assignment is fine:
@@ -86,13 +89,13 @@ c_instance = C()
 
 reveal_type(c_instance.declared_and_bound)  # revealed: str | None
 
-# TODO: we currently plan to emit a diagnostic here. Note that both mypy
-# and pyright show no error in this case! So we may reconsider this in
-# the future, if it turns out to produce too many false positives.
-reveal_type(C.declared_and_bound)  # revealed: str | None
+# Note that both mypy and pyright show no error in this case! So we may reconsider this in
+# the future, if it turns out to produce too many false positives. We currently emit:
+# error: [unresolved-attribute] "Attribute `declared_and_bound` can only be accessed on instances, not on the class object `Literal[C]` itself."
+reveal_type(C.declared_and_bound)  # revealed: Unknown
 
-# TODO: same as above. We plan to emit a diagnostic here, even if both mypy
-# and pyright allow this.
+# Same as above. Mypy and pyright do not show an error here.
+# error: [invalid-attribute-access] "Cannot assign to instance attribute `declared_and_bound` from the class object `Literal[C]`"
 C.declared_and_bound = "overwritten on class"
 
 # error: [invalid-assignment] "Object of type `Literal[1]` is not assignable to attribute `declared_and_bound` of type `str | None`"
@@ -112,11 +115,11 @@ c_instance = C()
 
 reveal_type(c_instance.only_declared)  # revealed: str
 
-# TODO: mypy and pyright do not show an error here, but we plan to emit a diagnostic.
-# The type could be changed to 'Unknown' if we decide to emit an error?
-reveal_type(C.only_declared)  # revealed: str
+# Mypy and pyright do not show an error here. We treat this as a pure instance variable.
+# error: [unresolved-attribute] "Attribute `only_declared` can only be accessed on instances, not on the class object `Literal[C]` itself."
+reveal_type(C.only_declared)  # revealed: Unknown
 
-# TODO: mypy and pyright do not show an error here, but we plan to emit one.
+# error: [invalid-attribute-access] "Cannot assign to instance attribute `only_declared` from the class object `Literal[C]`"
 C.only_declared = "overwritten on class"
 ```
 
@@ -152,7 +155,9 @@ reveal_type(c_instance.declared_in_body_and_init)  # revealed: str | None
 
 reveal_type(c_instance.declared_in_body_defined_in_init)  # revealed: str | None
 
-reveal_type(c_instance.bound_in_body_declared_in_init)  # revealed: str | None
+# TODO: This should be `str | None`. Fixing this requires an overhaul of the `Symbol` API,
+# which is planned in https://github.com/astral-sh/ruff/issues/14297
+reveal_type(c_instance.bound_in_body_declared_in_init)  # revealed: Unknown | str | None
 
 reveal_type(c_instance.bound_in_body_and_init)  # revealed: Unknown | None | Literal["a"]
 ```
@@ -181,18 +186,16 @@ reveal_type(c_instance.inferred_from_value)  # revealed: Unknown | Literal[1, "a
 # TODO: Should be `Unknown | Literal[1, "a"]`
 reveal_type(c_instance.inferred_from_other_attribute)  # revealed: Unknown
 
-# TODO: Should be `int | None`
 reveal_type(c_instance.inferred_from_param)  # revealed: Unknown | int | None
 
 reveal_type(c_instance.declared_only)  # revealed: bytes
 
 reveal_type(c_instance.declared_and_bound)  # revealed: bool
 
-# TODO: We already show an error here, but the message might be improved?
-# error: [unresolved-attribute]
+# error: [unresolved-attribute] "Attribute `inferred_from_value` can only be accessed on instances, not on the class object `Literal[C]` itself."
 reveal_type(C.inferred_from_value)  # revealed: Unknown
 
-# TODO: this should be an error
+# error: [invalid-attribute-access] "Cannot assign to instance attribute `inferred_from_value` from the class object `Literal[C]`"
 C.inferred_from_value = "overwritten on class"
 ```
 
@@ -355,9 +358,25 @@ class C:
 
 c_instance = C()
 
-# TODO: Should be `Unknown | int | None`
-# error: [unresolved-attribute]
-reveal_type(c_instance.x)  # revealed: Unknown
+reveal_type(c_instance.x)  # revealed: Unknown | int | None
+```
+
+#### Attributes defined in `with` statements, but with unpacking
+
+```py
+class ContextManager:
+    def __enter__(self) -> tuple[int | None, int]: ...
+    def __exit__(self, exc_type, exc_value, traceback) -> None: ...
+
+class C:
+    def __init__(self) -> None:
+        with ContextManager() as (self.x, self.y):
+            pass
+
+c_instance = C()
+
+reveal_type(c_instance.x)  # revealed: Unknown | int | None
+reveal_type(c_instance.y)  # revealed: Unknown | int
 ```
 
 #### Attributes defined in comprehensions
@@ -595,6 +614,9 @@ C.class_method()
 # error: [unresolved-attribute]
 reveal_type(C.pure_class_variable)  # revealed: Unknown
 
+# TODO: should be no error when descriptor protocol is supported
+# and the assignment is properly attributed to the class method.
+# error: [invalid-attribute-access] "Cannot assign to instance attribute `pure_class_variable` from the class object `Literal[C]`"
 C.pure_class_variable = "overwritten on class"
 
 # TODO: should be  `Unknown | Literal["value set in class method"]` or
@@ -700,8 +722,91 @@ reveal_type(Derived().declared_in_body)  # revealed: int | None
 reveal_type(Derived().defined_in_init)  # revealed: str | None
 ```
 
+## Accessing attributes on class objects
+
+When accessing attributes on class objects, they are always looked up on the type of the class
+object first, i.e. on the metaclass:
+
+```py
+from typing import Literal
+
+class Meta1:
+    attr: Literal["metaclass value"] = "metaclass value"
+
+class C1(metaclass=Meta1): ...
+
+reveal_type(C1.attr)  # revealed: Literal["metaclass value"]
+```
+
+However, the metaclass attribute only takes precedence over a class-level attribute if it is a data
+descriptor. If it is a non-data descriptor or a normal attribute, the class-level attribute is used
+instead (see the [descriptor protocol tests] for data/non-data descriptor attributes):
+
+```py
+class Meta2:
+    attr: str = "metaclass value"
+
+class C2(metaclass=Meta2):
+    attr: Literal["class value"] = "class value"
+
+reveal_type(C2.attr)  # revealed: Literal["class value"]
+```
+
+If the class-level attribute is only partially defined, we union the metaclass attribute with the
+class-level attribute:
+
+```py
+def _(flag: bool):
+    class Meta3:
+        attr1 = "metaclass value"
+        attr2: Literal["metaclass value"] = "metaclass value"
+
+    class C3(metaclass=Meta3):
+        if flag:
+            attr1 = "class value"
+            # TODO: Neither mypy nor pyright show an error here, but we could consider emitting a conflicting-declaration diagnostic here.
+            attr2: Literal["class value"] = "class value"
+
+    reveal_type(C3.attr1)  # revealed: Unknown | Literal["metaclass value", "class value"]
+    reveal_type(C3.attr2)  # revealed: Literal["metaclass value", "class value"]
+```
+
+If the *metaclass* attribute is only partially defined, we emit a `possibly-unbound-attribute`
+diagnostic:
+
+```py
+def _(flag: bool):
+    class Meta4:
+        if flag:
+            attr1: str = "metaclass value"
+
+    class C4(metaclass=Meta4): ...
+    # error: [possibly-unbound-attribute]
+    reveal_type(C4.attr1)  # revealed: str
+```
+
+Finally, if both the metaclass attribute and the class-level attribute are only partially defined,
+we union them and emit a `possibly-unbound-attribute` diagnostic:
+
+```py
+def _(flag1: bool, flag2: bool):
+    class Meta5:
+        if flag1:
+            attr1 = "metaclass value"
+
+    class C5(metaclass=Meta5):
+        if flag2:
+            attr1 = "class value"
+
+    # error: [possibly-unbound-attribute]
+    reveal_type(C5.attr1)  # revealed: Unknown | Literal["metaclass value", "class value"]
+```
+
 ## Union of attributes
 
+If the (meta)class is a union type or if the attribute on the (meta) class has a union type, we
+infer those union types accordingly:
+
 ```py
 def _(flag: bool):
     if flag:
@@ -712,14 +817,35 @@ def _(flag: bool):
         class C1:
             x = 2
 
+    reveal_type(C1.x)  # revealed: Unknown | Literal[1, 2]
+
     class C2:
         if flag:
             x = 3
         else:
             x = 4
 
-    reveal_type(C1.x)  # revealed: Unknown | Literal[1, 2]
     reveal_type(C2.x)  # revealed: Unknown | Literal[3, 4]
+
+    if flag:
+        class Meta3(type):
+            x = 5
+
+    else:
+        class Meta3(type):
+            x = 6
+
+    class C3(metaclass=Meta3): ...
+    reveal_type(C3.x)  # revealed: Unknown | Literal[5, 6]
+
+    class Meta4(type):
+        if flag:
+            x = 7
+        else:
+            x = 8
+
+    class C4(metaclass=Meta4): ...
+    reveal_type(C4.x)  # revealed: Unknown | Literal[7, 8]
 ```
 
 ## Inherited class attributes
@@ -779,6 +905,9 @@ def _(flag1: bool, flag2: bool):
 
     # error: [possibly-unbound-attribute] "Attribute `x` on type `Literal[C1, C2, C3]` is possibly unbound"
     reveal_type(C.x)  # revealed: Unknown | Literal[1, 3]
+
+    # error: [possibly-unbound-attribute] "Attribute `x` on type `C1 | C2 | C3` is possibly unbound"
+    reveal_type(C().x)  # revealed: Unknown | Literal[1, 3]
 ```
 
 ### Possibly-unbound within a class
@@ -802,6 +931,126 @@ def _(flag: bool, flag1: bool, flag2: bool):
 
     # error: [possibly-unbound-attribute] "Attribute `x` on type `Literal[C1, C2, C3]` is possibly unbound"
     reveal_type(C.x)  # revealed: Unknown | Literal[1, 2, 3]
+
+    # Note: we might want to consider ignoring possibly-unbound diagnostics for instance attributes eventually,
+    # see the "Possibly unbound/undeclared instance attribute" section below.
+    # error: [possibly-unbound-attribute] "Attribute `x` on type `C1 | C2 | C3` is possibly unbound"
+    reveal_type(C().x)  # revealed: Unknown | Literal[1, 2, 3]
+```
+
+### Possibly-unbound within gradual types
+
+```py
+from typing import Any
+
+def _(flag: bool):
+    class Base:
+        x: Any
+
+    class Derived(Base):
+        if flag:
+            # Redeclaring `x` with a more static type is okay in terms of LSP.
+            x: int
+
+    reveal_type(Derived().x)  # revealed: int | Any
+```
+
+### Attribute possibly unbound on a subclass but not on a superclass
+
+```py
+def _(flag: bool):
+    class Foo:
+        x = 1
+
+    class Bar(Foo):
+        if flag:
+            x = 2
+
+    reveal_type(Bar.x)  # revealed: Unknown | Literal[2, 1]
+
+    reveal_type(Bar().x)  # revealed: Unknown | Literal[2, 1]
+```
+
+### Attribute possibly unbound on a subclass and on a superclass
+
+```py
+def _(flag: bool):
+    class Foo:
+        if flag:
+            x = 1
+
+    class Bar(Foo):
+        if flag:
+            x = 2
+
+    # error: [possibly-unbound-attribute]
+    reveal_type(Bar.x)  # revealed: Unknown | Literal[2, 1]
+
+    # error: [possibly-unbound-attribute]
+    reveal_type(Bar().x)  # revealed: Unknown | Literal[2, 1]
+```
+
+### Possibly unbound/undeclared instance attribute
+
+#### Possibly unbound and undeclared
+
+```py
+def _(flag: bool):
+    class Foo:
+        if flag:
+            x: int
+
+        def __init(self):
+            if flag:
+                self.x = 1
+
+    # error: [possibly-unbound-attribute]
+    reveal_type(Foo().x)  # revealed: int | Unknown
+```
+
+#### Possibly unbound
+
+```py
+def _(flag: bool):
+    class Foo:
+        def __init(self):
+            if flag:
+                self.x = 1
+
+    # Emitting a diagnostic in a case like this is not something we support, and it's unclear
+    # if we ever will (or want to)
+    reveal_type(Foo().x)  # revealed: Unknown | Literal[1]
+```
+
+### Attribute access on `Any`
+
+The union of the set of types that `Any` could materialise to is equivalent to `object`. It follows
+from this that attribute access on `Any` resolves to `Any` if the attribute does not exist on
+`object` -- but if the attribute *does* exist on `object`, the type of the attribute is
+`<type as it exists on object> & Any`.
+
+```py
+from typing import Any
+
+class Foo(Any): ...
+
+reveal_type(Foo.bar)  # revealed: Any
+reveal_type(Foo.__repr__)  # revealed: Literal[__repr__] & Any
+```
+
+Similar principles apply if `Any` appears in the middle of an inheritance hierarchy:
+
+```py
+from typing import ClassVar, Literal
+
+class A:
+    x: ClassVar[Literal[1]] = 1
+
+class B(Any): ...
+class C(B, A): ...
+
+reveal_type(C.__mro__)  # revealed: tuple[Literal[C], Literal[B], Any, Literal[A], Literal[object]]
+reveal_type(C.x)  # revealed: Literal[1] & Any
 ```
 
 ### Unions with all paths unbound
@@ -820,13 +1069,18 @@ def _(flag: bool):
 
 ## Objects of all types have a `__class__` method
 
+The type of `x.__class__` is the same as `x`'s meta-type. `x.__class__` is always the same value as
+`type(x)`.
+
 ```py
 import typing_extensions
 
 reveal_type(typing_extensions.__class__)  # revealed: Literal[ModuleType]
+reveal_type(type(typing_extensions))  # revealed: Literal[ModuleType]
 
 a = 42
 reveal_type(a.__class__)  # revealed: Literal[int]
+reveal_type(type(a))  # revealed: Literal[int]
 
 b = "42"
 reveal_type(b.__class__)  # revealed: Literal[str]
@@ -842,8 +1096,13 @@ reveal_type(e.__class__)  # revealed: Literal[tuple]
 
 def f(a: int, b: typing_extensions.LiteralString, c: int | str, d: type[str]):
     reveal_type(a.__class__)  # revealed: type[int]
+    reveal_type(type(a))  # revealed: type[int]
+
     reveal_type(b.__class__)  # revealed: Literal[str]
+    reveal_type(type(b))  # revealed: Literal[str]
+
     reveal_type(c.__class__)  # revealed: type[int] | type[str]
+    reveal_type(type(c))  # revealed: type[int] | type[str]
 
     # `type[type]`, a.k.a., either the class `type` or some subclass of `type`.
     # It would be incorrect to infer `Literal[type]` here,
@@ -941,8 +1200,8 @@ reveal_type(f.__kwdefaults__)  # revealed: @Todo(generics) | None
 Some attributes are special-cased, however:
 
 ```py
-reveal_type(f.__get__)  # revealed: @Todo(`__get__` method on functions)
-reveal_type(f.__call__)  # revealed: @Todo(`__call__` method on functions)
+reveal_type(f.__get__)  # revealed: <method-wrapper `__get__` of `f`>
+reveal_type(f.__call__)  # revealed: <bound method `__call__` of `Literal[f]`>
 ```
 
 ### Int-literal attributes
@@ -951,7 +1210,7 @@ Most attribute accesses on int-literal types are delegated to `builtins.int`, si
 integers are instances of that class:
 
 ```py
-reveal_type((2).bit_length)  # revealed: @Todo(bound method)
+reveal_type((2).bit_length)  # revealed: <bound method `bit_length` of `Literal[2]`>
 reveal_type((2).denominator)  # revealed: @Todo(@property)
 ```
 
@@ -965,11 +1224,11 @@ reveal_type((2).real)  # revealed: Literal[2]
 ### Bool-literal attributes
 
 Most attribute accesses on bool-literal types are delegated to `builtins.bool`, since all literal
-bols are instances of that class:
+bools are instances of that class:
 
 ```py
-reveal_type(True.__and__)  # revealed: @Todo(bound method)
-reveal_type(False.__or__)  # revealed: @Todo(bound method)
+reveal_type(True.__and__)  # revealed: <bound method `__and__` of `Literal[True]`>
+reveal_type(False.__or__)  # revealed: <bound method `__or__` of `Literal[False]`>
 ```
 
 Some attributes are special-cased, however:
@@ -981,11 +1240,11 @@ reveal_type(False.real)  # revealed: Literal[0]
 
 ### Bytes-literal attributes
 
-All attribute access on literal `bytes` types is currently delegated to `buitins.bytes`:
+All attribute access on literal `bytes` types is currently delegated to `builtins.bytes`:
 
 ```py
-reveal_type(b"foo".join)  # revealed: @Todo(bound method)
-reveal_type(b"foo".endswith)  # revealed: @Todo(bound method)
+reveal_type(b"foo".join)  # revealed: <bound method `join` of `Literal[b"foo"]`>
+reveal_type(b"foo".endswith)  # revealed: <bound method `endswith` of `Literal[b"foo"]`>
 ```
 
 ## Instance attribute edge cases
@@ -1072,11 +1331,60 @@ class C:
 reveal_type(C().x)  # revealed: Unknown
 ```
 
+### Accessing attributes on `Never`
+
+Arbitrary attributes can be accessed on `Never` without emitting any errors:
+
+```py
+from typing_extensions import Never
+
+def f(never: Never):
+    reveal_type(never.arbitrary_attribute)  # revealed: Never
+
+    # Assigning `Never` to an attribute on `Never` is also allowed:
+    never.another_attribute = never
+```
+
+### Builtin types attributes
+
+This test can probably be removed eventually, but we currently include it because we do not yet
+understand generic bases and protocols, and we want to make sure that we can still use builtin types
+in our tests in the meantime. See the corresponding TODO in `Type::static_member` for more
+information.
+
+```py
+class C:
+    a_int: int = 1
+    a_str: str = "a"
+    a_bytes: bytes = b"a"
+    a_bool: bool = True
+    a_float: float = 1.0
+    a_complex: complex = 1 + 1j
+    a_tuple: tuple[int] = (1,)
+    a_range: range = range(1)
+    a_slice: slice = slice(1)
+    a_type: type = int
+    a_none: None = None
+
+reveal_type(C.a_int)  # revealed: int
+reveal_type(C.a_str)  # revealed: str
+reveal_type(C.a_bytes)  # revealed: bytes
+reveal_type(C.a_bool)  # revealed: bool
+reveal_type(C.a_float)  # revealed: int | float
+reveal_type(C.a_complex)  # revealed: int | float | complex
+reveal_type(C.a_tuple)  # revealed: tuple[int]
+reveal_type(C.a_range)  # revealed: range
+reveal_type(C.a_slice)  # revealed: slice
+reveal_type(C.a_type)  # revealed: type
+reveal_type(C.a_none)  # revealed: None
+```
+
 ## References
 
 Some of the tests in the *Class and instance variables* section draw inspiration from
 [pyright's documentation] on this topic.
 
+[descriptor protocol tests]: descriptor_protocol.md
 [pyright's documentation]: https://microsoft.github.io/pyright/#/type-concepts-advanced?id=class-and-instance-variables
 [typing spec on `classvar`]: https://typing.readthedocs.io/en/latest/spec/class-compat.html#classvar
 [`typing.classvar`]: https://docs.python.org/3/library/typing.html#typing.ClassVar

crates/red_knot_python_semantic/resources/mdtest/binary/booleans.md

@@ -56,7 +56,7 @@ def _(a: bool):
         reveal_type(x - a)  # revealed: int
         reveal_type(x * a)  # revealed: int
         reveal_type(x // a)  # revealed: int
-        reveal_type(x / a)  # revealed: float
+        reveal_type(x / a)  # revealed: int | float
         reveal_type(x % a)  # revealed: int
 
     def rhs_is_int(x: int):
@@ -64,7 +64,7 @@ def _(a: bool):
         reveal_type(a - x)  # revealed: int
         reveal_type(a * x)  # revealed: int
         reveal_type(a // x)  # revealed: int
-        reveal_type(a / x)  # revealed: float
+        reveal_type(a / x)  # revealed: int | float
         reveal_type(a % x)  # revealed: int
 
     def lhs_is_bool(x: bool):
@@ -72,7 +72,7 @@ def _(a: bool):
         reveal_type(x - a)  # revealed: int
         reveal_type(x * a)  # revealed: int
         reveal_type(x // a)  # revealed: int
-        reveal_type(x / a)  # revealed: float
+        reveal_type(x / a)  # revealed: int | float
         reveal_type(x % a)  # revealed: int
 
     def rhs_is_bool(x: bool):
@@ -80,7 +80,7 @@ def _(a: bool):
         reveal_type(a - x)  # revealed: int
         reveal_type(a * x)  # revealed: int
         reveal_type(a // x)  # revealed: int
-        reveal_type(a / x)  # revealed: float
+        reveal_type(a / x)  # revealed: int | float
         reveal_type(a % x)  # revealed: int
 
     def both_are_bool(x: bool, y: bool):
@@ -88,6 +88,6 @@ def _(a: bool):
         reveal_type(x - y)  # revealed: int
         reveal_type(x * y)  # revealed: int
         reveal_type(x // y)  # revealed: int
-        reveal_type(x / y)  # revealed: float
+        reveal_type(x / y)  # revealed: int | float
         reveal_type(x % y)  # revealed: int
 ```

crates/red_knot_python_semantic/resources/mdtest/binary/instances.md

@@ -244,10 +244,7 @@ class B:
     def __rsub__(self, other: A) -> B:
         return B()
 
-# TODO: this should be `B` (the return annotation of `B.__rsub__`),
-# because `A.__sub__` is annotated as only accepting `A`,
-# but `B.__rsub__` will accept `A`.
-reveal_type(A() - B())  # revealed: A
+reveal_type(A() - B())  # revealed: B
 ```
 
 ## Callable instances as dunders
@@ -262,32 +259,38 @@ class A:
 class B:
     __add__ = A()
 
-# TODO: this could be `int` if we declare `B.__add__` using a `Callable` type
 reveal_type(B() + B())  # revealed: Unknown | int
 ```
 
+Note that we union with `Unknown` here because `__add__` is not declared. We do infer just `int` if
+the callable is declared:
+
+```py
+class B2:
+    __add__: A = A()
+
+reveal_type(B2() + B2())  # revealed: int
+```
+
 ## Integration test: numbers from typeshed
 
+We get less precise results from binary operations on float/complex literals due to the special case
+for annotations of `float` or `complex`, which applies also to return annotations for typeshed
+dunder methods. Perhaps we could have a special-case on the special-case, to exclude these typeshed
+return annotations from the widening, and preserve a bit more precision here?
+
 ```py
-reveal_type(3j + 3.14)  # revealed: complex
-reveal_type(4.2 + 42)  # revealed: float
-reveal_type(3j + 3)  # revealed: complex
-
-# TODO should be complex, need to check arg type and fall back to `rhs.__radd__`
-reveal_type(3.14 + 3j)  # revealed: float
-
-# TODO should be float, need to check arg type and fall back to `rhs.__radd__`
-reveal_type(42 + 4.2)  # revealed: int
-
-# TODO should be complex, need to check arg type and fall back to `rhs.__radd__`
-reveal_type(3 + 3j)  # revealed: int
+reveal_type(3j + 3.14)  # revealed: int | float | complex
+reveal_type(4.2 + 42)  # revealed: int | float
+reveal_type(3j + 3)  # revealed: int | float | complex
+reveal_type(3.14 + 3j)  # revealed: int | float | complex
+reveal_type(42 + 4.2)  # revealed: int | float
+reveal_type(3 + 3j)  # revealed: int | float | complex
 
 def _(x: bool, y: int):
     reveal_type(x + y)  # revealed: int
-    reveal_type(4.2 + x)  # revealed: float
-
-    # TODO should be float, need to check arg type and fall back to `rhs.__radd__`
-    reveal_type(y + 4.12)  # revealed: int
+    reveal_type(4.2 + x)  # revealed: int | float
+    reveal_type(y + 4.12)  # revealed: int | float
 ```
 
 ## With literal types
@@ -304,13 +307,12 @@ class A:
         return self
 
 reveal_type(A() + 1)  # revealed: A
-# TODO should be `A` since `int.__add__` doesn't support `A` instances
-reveal_type(1 + A())  # revealed: int
+reveal_type(1 + A())  # revealed: A
 
 reveal_type(A() + "foo")  # revealed: A
 # TODO should be `A` since `str.__add__` doesn't support `A` instances
 # TODO overloads
-reveal_type("foo" + A())  # revealed: @Todo(return type)
+reveal_type("foo" + A())  # revealed: @Todo(return type of decorated function)
 
 reveal_type(A() + b"foo")  # revealed: A
 # TODO should be `A` since `bytes.__add__` doesn't support `A` instances
@@ -318,7 +320,7 @@ reveal_type(b"foo" + A())  # revealed: bytes
 
 reveal_type(A() + ())  # revealed: A
 # TODO this should be `A`, since `tuple.__add__` doesn't support `A` instances
-reveal_type(() + A())  # revealed: @Todo(return type)
+reveal_type(() + A())  # revealed: @Todo(return type of decorated function)
 
 literal_string_instance = "foo" * 1_000_000_000
 # the test is not testing what it's meant to be testing if this isn't a `LiteralString`:
@@ -327,7 +329,7 @@ reveal_type(literal_string_instance)  # revealed: LiteralString
 reveal_type(A() + literal_string_instance)  # revealed: A
 # TODO should be `A` since `str.__add__` doesn't support `A` instances
 # TODO overloads
-reveal_type(literal_string_instance + A())  # revealed: @Todo(return type)
+reveal_type(literal_string_instance + A())  # revealed: @Todo(return type of decorated function)
 ```
 
 ## Operations involving instances of classes inheriting from `Any`
@@ -355,6 +357,20 @@ class Y(Foo): ...
 reveal_type(X() + Y())  # revealed: int
 ```
 
+## Operations involving types with invalid `__bool__` methods
+
+<!-- snapshot-diagnostics -->
+
+```py
+class NotBoolable:
+    __bool__ = 3
+
+a = NotBoolable()
+
+# error: [unsupported-bool-conversion]
+10 and a and True
+```
+
 ## Unsupported
 
 ### Dunder as instance attribute

crates/red_knot_python_semantic/resources/mdtest/binary/integers.md

@@ -10,16 +10,15 @@ reveal_type(-3 // 3)  # revealed: Literal[-1]
 reveal_type(-3 / 3)  # revealed: float
 reveal_type(5 % 3)  # revealed: Literal[2]
 
-# TODO: We don't currently verify that the actual parameter to int.__add__ matches the declared
-# formal parameter type.
-reveal_type(2 + "f")  # revealed: int
+# error: [unsupported-operator] "Operator `+` is unsupported between objects of type `Literal[2]` and `Literal["f"]`"
+reveal_type(2 + "f")  # revealed: Unknown
 
 def lhs(x: int):
     reveal_type(x + 1)  # revealed: int
     reveal_type(x - 4)  # revealed: int
     reveal_type(x * -1)  # revealed: int
     reveal_type(x // 3)  # revealed: int
-    reveal_type(x / 3)  # revealed: float
+    reveal_type(x / 3)  # revealed: int | float
     reveal_type(x % 3)  # revealed: int
 
 def rhs(x: int):
@@ -27,7 +26,7 @@ def rhs(x: int):
     reveal_type(3 - x)  # revealed: int
     reveal_type(3 * x)  # revealed: int
     reveal_type(-3 // x)  # revealed: int
-    reveal_type(-3 / x)  # revealed: float
+    reveal_type(-3 / x)  # revealed: int | float
     reveal_type(5 % x)  # revealed: int
 
 def both(x: int):
@@ -35,7 +34,7 @@ def both(x: int):
     reveal_type(x - x)  # revealed: int
     reveal_type(x * x)  # revealed: int
     reveal_type(x // x)  # revealed: int
-    reveal_type(x / x)  # revealed: float
+    reveal_type(x / x)  # revealed: int | float
     reveal_type(x % x)  # revealed: int
 ```
 
@@ -51,9 +50,9 @@ reveal_type(1 ** (largest_u32 + 1))  # revealed: int
 reveal_type(2**largest_u32)  # revealed: int
 
 def variable(x: int):
-    reveal_type(x**2)  # revealed: @Todo(return type)
-    reveal_type(2**x)  # revealed: @Todo(return type)
-    reveal_type(x**x)  # revealed: @Todo(return type)
+    reveal_type(x**2)  # revealed: @Todo(return type of decorated function)
+    reveal_type(2**x)  # revealed: @Todo(return type of decorated function)
+    reveal_type(x**x)  # revealed: @Todo(return type of decorated function)
 ```
 
 ## Division by Zero
@@ -80,24 +79,20 @@ c = 3 % 0  # error: "Cannot reduce object of type `Literal[3]` modulo zero"
 reveal_type(c)  # revealed: int
 
 # error: "Cannot divide object of type `int` by zero"
-# revealed: float
-reveal_type(int() / 0)
+reveal_type(int() / 0)  # revealed: int | float
 
 # error: "Cannot divide object of type `Literal[1]` by zero"
-# revealed: float
-reveal_type(1 / False)
+reveal_type(1 / False)  # revealed: float
 # error: [division-by-zero] "Cannot divide object of type `Literal[True]` by zero"
 True / False
 # error: [division-by-zero] "Cannot divide object of type `Literal[True]` by zero"
 bool(1) / False
 
 # error: "Cannot divide object of type `float` by zero"
-# revealed: float
-reveal_type(1.0 / 0)
+reveal_type(1.0 / 0)  # revealed: int | float
 
 class MyInt(int): ...
 
 # No error for a subclass of int
-# revealed: float
-reveal_type(MyInt(3) / 0)
+reveal_type(MyInt(3) / 0)  # revealed: int | float
 ```

crates/red_knot_python_semantic/resources/mdtest/call/builtins.md

@@ -0,0 +1,37 @@
+# Calling builtins
+
+## `bool` with incorrect arguments
+
+```py
+class NotBool:
+    __bool__ = None
+
+# TODO: We should emit an `invalid-argument` error here for `2` because `bool` only takes one argument.
+bool(1, 2)
+
+# TODO: We should emit an `unsupported-bool-conversion` error here because the argument doesn't implement `__bool__` correctly.
+bool(NotBool())
+```
+
+## Calls to `type()`
+
+A single-argument call to `type()` returns an object that has the argument's meta-type. (This is
+tested more extensively in `crates/red_knot_python_semantic/resources/mdtest/attributes.md`,
+alongside the tests for the `__class__` attribute.)
+
+```py
+reveal_type(type(1))  # revealed: Literal[int]
+```
+
+But a three-argument call to type creates a dynamic instance of the `type` class:
+
+```py
+reveal_type(type("Foo", (), {}))  # revealed: type
+```
+
+Other numbers of arguments are invalid (TODO -- these should emit a diagnostic)
+
+```py
+type("Foo", ())
+type("Foo", (), {}, weird_other_arg=42)
+```

crates/red_knot_python_semantic/resources/mdtest/call/callable_instance.md

@@ -4,14 +4,14 @@
 
 ```py
 class Multiplier:
-    def __init__(self, factor: float):
+    def __init__(self, factor: int):
         self.factor = factor
 
-    def __call__(self, number: float) -> float:
+    def __call__(self, number: int) -> int:
         return number * self.factor
 
-a = Multiplier(2.0)(3.0)
-reveal_type(a)  # revealed: float
+a = Multiplier(2)(3)
+reveal_type(a)  # revealed: int
 
 class Unit: ...
 
@@ -52,7 +52,7 @@ class NonCallable:
     __call__ = 1
 
 a = NonCallable()
-# error: "Object of type `Unknown | Literal[1]` is not callable (due to union element `Literal[1]`)"
+# error: [call-non-callable] "Object of type `Literal[1]` is not callable"
 reveal_type(a())  # revealed: Unknown
 ```
 
@@ -67,8 +67,8 @@ def _(flag: bool):
             def __call__(self) -> int: ...
 
     a = NonCallable()
-    # error: "Object of type `Literal[1] | Literal[__call__]` is not callable (due to union element `Literal[1]`)"
-    reveal_type(a())  # revealed: Unknown | int
+    # error: [call-non-callable] "Object of type `Literal[1]` is not callable"
+    reveal_type(a())  # revealed: int | Unknown
 ```
 
 ## Call binding errors
@@ -82,7 +82,7 @@ class C:
 
 c = C()
 
-# error: 15 [invalid-argument-type] "Object of type `Literal["foo"]` cannot be assigned to parameter 2 (`x`) of function `__call__`; expected type `int`"
+# error: 15 [invalid-argument-type] "Object of type `Literal["foo"]` cannot be assigned to parameter 2 (`x`) of bound method `__call__`; expected type `int`"
 reveal_type(c("foo"))  # revealed: int
 ```
 
@@ -96,6 +96,29 @@ class C:
 
 c = C()
 
-# error: 13 [invalid-argument-type] "Object of type `C` cannot be assigned to parameter 1 (`self`) of function `__call__`; expected type `int`"
+# error: 13 [invalid-argument-type] "Object of type `C` cannot be assigned to parameter 1 (`self`) of bound method `__call__`; expected type `int`"
 reveal_type(c())  # revealed: int
 ```
+
+## Union over callables
+
+### Possibly unbound `__call__`
+
+```py
+def outer(cond1: bool):
+    class Test:
+        if cond1:
+            def __call__(self): ...
+
+    class Other:
+        def __call__(self): ...
+
+    def inner(cond2: bool):
+        if cond2:
+            a = Test()
+        else:
+            a = Other()
+
+            # error: [call-non-callable] "Object of type `Test` is not callable (possibly unbound `__call__` method)"
+        a()
+```

crates/red_knot_python_semantic/resources/mdtest/call/dunder.md

@@ -0,0 +1,219 @@
+# Dunder calls
+
+## Introduction
+
+This test suite explains and documents how dunder methods are looked up and called. Throughout the
+document, we use `__getitem__` as an example, but the same principles apply to other dunder methods.
+
+Dunder methods are implicitly called when using certain syntax. For example, the index operator
+`obj[key]` calls the `__getitem__` method under the hood. Exactly *how* a dunder method is looked up
+and called works slightly different from regular methods. Dunder methods are not looked up on `obj`
+directly, but rather on `type(obj)`. But in many ways, they still *act* as if they were called on
+`obj` directly. If the `__getitem__` member of `type(obj)` is a descriptor, it is called with `obj`
+as the `instance` argument to `__get__`. A desugared version of `obj[key]` is roughly equivalent to
+`getitem_desugared(obj, key)` as defined below:
+
+```py
+from typing import Any
+
+def find_name_in_mro(typ: type, name: str) -> Any:
+    # See implementation in https://docs.python.org/3/howto/descriptor.html#invocation-from-an-instance
+    pass
+
+def getitem_desugared(obj: object, key: object) -> object:
+    getitem_callable = find_name_in_mro(type(obj), "__getitem__")
+    if hasattr(getitem_callable, "__get__"):
+        getitem_callable = getitem_callable.__get__(obj, type(obj))
+
+    return getitem_callable(key)
+```
+
+In the following tests, we demonstrate that we implement this behavior correctly.
+
+## Operating on class objects
+
+If we invoke a dunder method on a class, it is looked up on the *meta* class, since any class is an
+instance of its metaclass:
+
+```py
+class Meta(type):
+    def __getitem__(cls, key: int) -> str:
+        return str(key)
+
+class DunderOnMetaclass(metaclass=Meta):
+    pass
+
+reveal_type(DunderOnMetaclass[0])  # revealed: str
+```
+
+If the dunder method is only present on the class itself, it will not be called:
+
+```py
+class ClassWithNormalDunder:
+    def __getitem__(self, key: int) -> str:
+        return str(key)
+
+# error: [non-subscriptable]
+ClassWithNormalDunder[0]
+```
+
+## Operating on instances
+
+When invoking a dunder method on an instance of a class, it is looked up on the class:
+
+```py
+class ClassWithNormalDunder:
+    def __getitem__(self, key: int) -> str:
+        return str(key)
+
+class_with_normal_dunder = ClassWithNormalDunder()
+
+reveal_type(class_with_normal_dunder[0])  # revealed: str
+```
+
+Which can be demonstrated by trying to attach a dunder method to an instance, which will not work:
+
+```py
+def external_getitem(instance, key: int) -> str:
+    return str(key)
+
+class ThisFails:
+    def __init__(self):
+        self.__getitem__ = external_getitem
+
+this_fails = ThisFails()
+
+# error: [non-subscriptable] "Cannot subscript object of type `ThisFails` with no `__getitem__` method"
+reveal_type(this_fails[0])  # revealed: Unknown
+```
+
+However, the attached dunder method *can* be called if accessed directly:
+
+```py
+reveal_type(this_fails.__getitem__(this_fails, 0))  # revealed: Unknown | str
+```
+
+The instance-level method is also not called when the class-level method is present:
+
+```py
+def external_getitem1(instance, key) -> str:
+    return "a"
+
+def external_getitem2(key) -> int:
+    return 1
+
+def _(flag: bool):
+    class ThisFails:
+        if flag:
+            __getitem__ = external_getitem1
+
+        def __init__(self):
+            self.__getitem__ = external_getitem2
+
+    this_fails = ThisFails()
+
+    # error: [call-possibly-unbound-method]
+    reveal_type(this_fails[0])  # revealed: Unknown | str
+```
+
+## When the dunder is not a method
+
+A dunder can also be a non-method callable:
+
+```py
+class SomeCallable:
+    def __call__(self, key: int) -> str:
+        return str(key)
+
+class ClassWithNonMethodDunder:
+    __getitem__: SomeCallable = SomeCallable()
+
+class_with_callable_dunder = ClassWithNonMethodDunder()
+
+reveal_type(class_with_callable_dunder[0])  # revealed: str
+```
+
+## Dunders are looked up using the descriptor protocol
+
+Here, we demonstrate that the descriptor protocol is invoked when looking up a dunder method. Note
+that the `instance` argument is on object of type `ClassWithDescriptorDunder`:
+
+```py
+from __future__ import annotations
+
+class SomeCallable:
+    def __call__(self, key: int) -> str:
+        return str(key)
+
+class Descriptor:
+    def __get__(self, instance: ClassWithDescriptorDunder, owner: type[ClassWithDescriptorDunder]) -> SomeCallable:
+        return SomeCallable()
+
+class ClassWithDescriptorDunder:
+    __getitem__: Descriptor = Descriptor()
+
+class_with_descriptor_dunder = ClassWithDescriptorDunder()
+
+reveal_type(class_with_descriptor_dunder[0])  # revealed: str
+```
+
+## Dunders can not be overwritten on instances
+
+If we attempt to overwrite a dunder method on an instance, it does not affect the behavior of
+implicit dunder calls:
+
+```py
+class C:
+    def __getitem__(self, key: int) -> str:
+        return str(key)
+
+    def f(self):
+        # TODO: This should emit an `invalid-assignment` diagnostic once we understand the type of `self`
+        self.__getitem__ = None
+
+# This is still fine, and simply calls the `__getitem__` method on the class
+reveal_type(C()[0])  # revealed: str
+```
+
+## Calling a union of dunder methods
+
+```py
+def _(flag: bool):
+    class C:
+        if flag:
+            def __getitem__(self, key: int) -> str:
+                return str(key)
+        else:
+            def __getitem__(self, key: int) -> bytes:
+                return key
+
+    c = C()
+    reveal_type(c[0])  # revealed: str | bytes
+
+    if flag:
+        class D:
+            def __getitem__(self, key: int) -> str:
+                return str(key)
+
+    else:
+        class D:
+            def __getitem__(self, key: int) -> bytes:
+                return key
+
+    d = D()
+    reveal_type(d[0])  # revealed: str | bytes
+```
+
+## Calling a possibly-unbound dunder method
+
+```py
+def _(flag: bool):
+    class C:
+        if flag:
+            def __getitem__(self, key: int) -> str:
+                return str(key)
+
+    c = C()
+    # error: [call-possibly-unbound-method]
+    reveal_type(c[0])  # revealed: str
+```

crates/red_knot_python_semantic/resources/mdtest/call/function.md

@@ -44,7 +44,7 @@ def bar() -> str:
     return "bar"
 
 # TODO: should reveal `int`, as the decorator replaces `bar` with `foo`
-reveal_type(bar())  # revealed: @Todo(return type)
+reveal_type(bar())  # revealed: @Todo(return type of decorated function)
 ```
 
 ## Invalid callable
@@ -278,10 +278,10 @@ proper diagnostics in case of missing or superfluous arguments.
 from typing_extensions import reveal_type
 
 # error: [missing-argument] "No argument provided for required parameter `obj` of function `reveal_type`"
-reveal_type()  # revealed: Unknown
+reveal_type()
 
 # error: [too-many-positional-arguments] "Too many positional arguments to function `reveal_type`: expected 1, got 2"
-reveal_type(1, 2)  # revealed: Literal[1]
+reveal_type(1, 2)
 ```
 
 ### `static_assert`
@@ -290,7 +290,6 @@ reveal_type(1, 2)  # revealed: Literal[1]
 from knot_extensions import static_assert
 
 # error: [missing-argument] "No argument provided for required parameter `condition` of function `static_assert`"
-# error: [static-assert-error]
 static_assert()
 
 # error: [too-many-positional-arguments] "Too many positional arguments to function `static_assert`: expected 2, got 3"

crates/red_knot_python_semantic/resources/mdtest/call/getattr_static.md

@@ -0,0 +1,150 @@
+# `inspect.getattr_static`
+
+## Basic usage
+
+`inspect.getattr_static` is a function that returns attributes of an object without invoking the
+descriptor protocol (for caveats, see the [official documentation]).
+
+Consider the following example:
+
+```py
+import inspect
+
+class Descriptor:
+    def __get__(self, instance, owner) -> str:
+        return "a"
+
+class C:
+    normal: int = 1
+    descriptor: Descriptor = Descriptor()
+```
+
+If we access attributes on an instance of `C` as usual, the descriptor protocol is invoked, and we
+get a type of `str` for the `descriptor` attribute:
+
+```py
+c = C()
+
+reveal_type(c.normal)  # revealed: int
+reveal_type(c.descriptor)  # revealed: str
+```
+
+However, if we use `inspect.getattr_static`, we can see the underlying `Descriptor` type:
+
+```py
+reveal_type(inspect.getattr_static(c, "normal"))  # revealed: int
+reveal_type(inspect.getattr_static(c, "descriptor"))  # revealed: Descriptor
+```
+
+For non-existent attributes, a default value can be provided:
+
+```py
+reveal_type(inspect.getattr_static(C, "normal", "default-arg"))  # revealed: int
+reveal_type(inspect.getattr_static(C, "non_existent", "default-arg"))  # revealed: Literal["default-arg"]
+```
+
+When a non-existent attribute is accessed without a default value, the runtime raises an
+`AttributeError`. We could emit a diagnostic for this case, but that is currently not supported:
+
+```py
+# TODO: we could emit a diagnostic here
+reveal_type(inspect.getattr_static(C, "non_existent"))  # revealed: Never
+```
+
+We can access attributes on objects of all kinds:
+
+```py
+import sys
+
+reveal_type(inspect.getattr_static(sys, "platform"))  # revealed: LiteralString
+reveal_type(inspect.getattr_static(inspect, "getattr_static"))  # revealed: Literal[getattr_static]
+
+reveal_type(inspect.getattr_static(1, "real"))  # revealed: Literal[real]
+```
+
+(Implicit) instance attributes can also be accessed through `inspect.getattr_static`:
+
+```py
+class D:
+    def __init__(self) -> None:
+        self.instance_attr: int = 1
+
+reveal_type(inspect.getattr_static(D(), "instance_attr"))  # revealed: int
+```
+
+And attributes on metaclasses can be accessed when probing the class:
+
+```py
+class Meta(type):
+    attr: int = 1
+
+class E(metaclass=Meta): ...
+
+reveal_type(inspect.getattr_static(E, "attr"))  # revealed: int
+```
+
+Metaclass attributes can not be added when probing an instance of the class:
+
+```py
+reveal_type(inspect.getattr_static(E(), "attr", "non_existent"))  # revealed: Literal["non_existent"]
+```
+
+## Error cases
+
+We can only infer precise types if the attribute is a literal string. In all other cases, we fall
+back to `Any`:
+
+```py
+import inspect
+
+class C:
+    x: int = 1
+
+def _(attr_name: str):
+    reveal_type(inspect.getattr_static(C(), attr_name))  # revealed: Any
+    reveal_type(inspect.getattr_static(C(), attr_name, 1))  # revealed: Any
+```
+
+But we still detect errors in the number or type of arguments:
+
+```py
+# error: [missing-argument] "No arguments provided for required parameters `obj`, `attr` of function `getattr_static`"
+inspect.getattr_static()
+
+# error: [missing-argument] "No argument provided for required parameter `attr`"
+inspect.getattr_static(C())
+
+# error: [invalid-argument-type] "Object of type `Literal[1]` cannot be assigned to parameter 2 (`attr`) of function `getattr_static`; expected type `str`"
+inspect.getattr_static(C(), 1)
+
+# error: [too-many-positional-arguments] "Too many positional arguments to function `getattr_static`: expected 3, got 4"
+inspect.getattr_static(C(), "x", "default-arg", "one too many")
+```
+
+## Possibly unbound attributes
+
+```py
+import inspect
+
+def _(flag: bool):
+    class C:
+        if flag:
+            x: int = 1
+
+    reveal_type(inspect.getattr_static(C, "x", "default"))  # revealed: int | Literal["default"]
+```
+
+## Gradual types
+
+```py
+import inspect
+from typing import Any
+
+def _(a: Any, tuple_of_any: tuple[Any]):
+    reveal_type(inspect.getattr_static(a, "x", "default"))  # revealed: Any | Literal["default"]
+
+    # TODO: Ideally, this would just be `Literal[index]`
+    reveal_type(inspect.getattr_static(tuple_of_any, "index", "default"))  # revealed: Literal[index] | Literal["default"]
+```
+
+[official documentation]: https://docs.python.org/3/library/inspect.html#inspect.getattr_static

crates/red_knot_python_semantic/resources/mdtest/call/methods.md

@@ -0,0 +1,432 @@
+# Methods
+
+## Background: Functions as descriptors
+
+> Note: See also this related section in the descriptor guide: [Functions and methods].
+
+Say we have a simple class `C` with a function definition `f` inside its body:
+
+```py
+class C:
+    def f(self, x: int) -> str:
+        return "a"
+```
+
+Whenever we access the `f` attribute through the class object itself (`C.f`) or through an instance
+(`C().f`), this access happens via the descriptor protocol. Functions are (non-data) descriptors
+because they implement a `__get__` method. This is crucial in making sure that method calls work as
+expected. In general, the signature of the `__get__` method in the descriptor protocol is
+`__get__(self, instance, owner)`. The `self` argument is the descriptor object itself (`f`). The
+passed value for the `instance` argument depends on whether the attribute is accessed from the class
+object (in which case it is `None`), or from an instance (in which case it is the instance of type
+`C`). The `owner` argument is the class itself (`C` of type `Literal[C]`). To summarize:
+
+- `C.f` is equivalent to `getattr_static(C, "f").__get__(None, C)`
+- `C().f` is equivalent to `getattr_static(C, "f").__get__(C(), C)`
+
+Here, `inspect.getattr_static` is used to bypass the descriptor protocol and directly access the
+function attribute. The way the special `__get__` method *on functions* works is as follows. In the
+former case, if the `instance` argument is `None`, `__get__` simply returns the function itself. In
+the latter case, it returns a *bound method* object:
+
+```py
+from inspect import getattr_static
+
+reveal_type(getattr_static(C, "f"))  # revealed: Literal[f]
+
+reveal_type(getattr_static(C, "f").__get__)  # revealed: <method-wrapper `__get__` of `f`>
+
+reveal_type(getattr_static(C, "f").__get__(None, C))  # revealed: Literal[f]
+reveal_type(getattr_static(C, "f").__get__(C(), C))  # revealed: <bound method `f` of `C`>
+```
+
+In conclusion, this is why we see the following two types when accessing the `f` attribute on the
+class object `C` and on an instance `C()`:
+
+```py
+reveal_type(C.f)  # revealed: Literal[f]
+reveal_type(C().f)  # revealed: <bound method `f` of `C`>
+```
+
+A bound method is a callable object that contains a reference to the `instance` that it was called
+on (can be inspected via `__self__`), and the function object that it refers to (can be inspected
+via `__func__`):
+
+```py
+bound_method = C().f
+
+reveal_type(bound_method.__self__)  # revealed: C
+reveal_type(bound_method.__func__)  # revealed: Literal[f]
+```
+
+When we call the bound method, the `instance` is implicitly passed as the first argument (`self`):
+
+```py
+reveal_type(C().f(1))  # revealed: str
+reveal_type(bound_method(1))  # revealed: str
+```
+
+When we call the function object itself, we need to pass the `instance` explicitly:
+
+```py
+C.f(1)  # error: [missing-argument]
+
+reveal_type(C.f(C(), 1))  # revealed: str
+```
+
+When we access methods from derived classes, they will be bound to instances of the derived class:
+
+```py
+class D(C):
+    pass
+
+reveal_type(D().f)  # revealed: <bound method `f` of `D`>
+```
+
+If we access an attribute on a bound method object itself, it will defer to `types.MethodType`:
+
+```py
+reveal_type(bound_method.__hash__)  # revealed: <bound method `__hash__` of `MethodType`>
+```
+
+If an attribute is not available on the bound method object, it will be looked up on the underlying
+function object. We model this explicitly, which means that we can access `__kwdefaults__` on bound
+methods, even though it is not available on `types.MethodType`:
+
+```py
+reveal_type(bound_method.__kwdefaults__)  # revealed: @Todo(generics) | None
+```
+
+## Basic method calls on class objects and instances
+
+```py
+class Base:
+    def method_on_base(self, x: int | None) -> str:
+        return "a"
+
+class Derived(Base):
+    def method_on_derived(self, x: bytes) -> tuple[int, str]:
+        return (1, "a")
+
+reveal_type(Base().method_on_base(1))  # revealed: str
+reveal_type(Base.method_on_base(Base(), 1))  # revealed: str
+
+Base().method_on_base("incorrect")  # error: [invalid-argument-type]
+Base().method_on_base()  # error: [missing-argument]
+Base().method_on_base(1, 2)  # error: [too-many-positional-arguments]
+
+reveal_type(Derived().method_on_base(1))  # revealed: str
+reveal_type(Derived().method_on_derived(b"abc"))  # revealed: tuple[int, str]
+reveal_type(Derived.method_on_base(Derived(), 1))  # revealed: str
+reveal_type(Derived.method_on_derived(Derived(), b"abc"))  # revealed: tuple[int, str]
+```
+
+## Method calls on literals
+
+### Boolean literals
+
+```py
+reveal_type(True.bit_length())  # revealed: int
+reveal_type(True.as_integer_ratio())  # revealed: tuple[int, Literal[1]]
+```
+
+### Integer literals
+
+```py
+reveal_type((42).bit_length())  # revealed: int
+```
+
+### String literals
+
+```py
+reveal_type("abcde".find("abc"))  # revealed: int
+reveal_type("foo".encode(encoding="utf-8"))  # revealed: bytes
+
+"abcde".find(123)  # error: [invalid-argument-type]
+```
+
+### Bytes literals
+
+```py
+reveal_type(b"abcde".startswith(b"abc"))  # revealed: bool
+```
+
+## Method calls on `LiteralString`
+
+```py
+from typing_extensions import LiteralString
+
+def f(s: LiteralString) -> None:
+    reveal_type(s.find("a"))  # revealed: int
+```
+
+## Method calls on `tuple`
+
+```py
+def f(t: tuple[int, str]) -> None:
+    reveal_type(t.index("a"))  # revealed: int
+```
+
+## Method calls on unions
+
+```py
+from typing import Any
+
+class A:
+    def f(self) -> int:
+        return 1
+
+class B:
+    def f(self) -> str:
+        return "a"
+
+def f(a_or_b: A | B, any_or_a: Any | A):
+    reveal_type(a_or_b.f)  # revealed: <bound method `f` of `A`> | <bound method `f` of `B`>
+    reveal_type(a_or_b.f())  # revealed: int | str
+
+    reveal_type(any_or_a.f)  # revealed: Any | <bound method `f` of `A`>
+    reveal_type(any_or_a.f())  # revealed: Any | int
+```
+
+## Method calls on `KnownInstance` types
+
+```toml
+[environment]
+python-version = "3.12"
+```
+
+```py
+type IntOrStr = int | str
+
+reveal_type(IntOrStr.__or__)  # revealed: <bound method `__or__` of `typing.TypeAliasType`>
+```
+
+## Error cases: Calling `__get__` for methods
+
+The `__get__` method on `types.FunctionType` has the following overloaded signature in typeshed:
+
+```py
+from types import FunctionType, MethodType
+from typing import overload
+
+@overload
+def __get__(self, instance: None, owner: type, /) -> FunctionType: ...
+@overload
+def __get__(self, instance: object, owner: type | None = None, /) -> MethodType: ...
+```
+
+Here, we test that this signature is enforced correctly:
+
+```py
+from inspect import getattr_static
+
+class C:
+    def f(self, x: int) -> str:
+        return "a"
+
+method_wrapper = getattr_static(C, "f").__get__
+
+reveal_type(method_wrapper)  # revealed: <method-wrapper `__get__` of `f`>
+
+# All of these are fine:
+method_wrapper(C(), C)
+method_wrapper(C())
+method_wrapper(C(), None)
+method_wrapper(None, C)
+
+# Passing `None` without an `owner` argument is an
+# error: [missing-argument] "No argument provided for required parameter `owner`"
+method_wrapper(None)
+
+# Passing something that is not assignable to `type` as the `owner` argument is an
+# error: [invalid-argument-type] "Object of type `Literal[1]` cannot be assigned to parameter 2 (`owner`) of method wrapper `__get__` of function `f`; expected type `type`"
+method_wrapper(None, 1)
+
+# Passing `None` as the `owner` argument when `instance` is `None` is an
+# error: [invalid-argument-type] "Object of type `None` cannot be assigned to parameter 2 (`owner`) of method wrapper `__get__` of function `f`; expected type `type`"
+method_wrapper(None, None)
+
+# Calling `__get__` without any arguments is an
+# error: [missing-argument] "No argument provided for required parameter `instance`"
+method_wrapper()
+
+# Calling `__get__` with too many positional arguments is an
+# error: [too-many-positional-arguments] "Too many positional arguments to method wrapper `__get__` of function `f`: expected 2, got 3"
+method_wrapper(C(), C, "one too many")
+```
+
+## Fallback to metaclass
+
+When a method is accessed on a class object, it is looked up on the metaclass if it is not found on
+the class itself. This also creates a bound method that is bound to the class object itself:
+
+```py
+from __future__ import annotations
+
+class Meta(type):
+    def f(cls, arg: int) -> str:
+        return "a"
+
+class C(metaclass=Meta):
+    pass
+
+reveal_type(C.f)  # revealed: <bound method `f` of `Literal[C]`>
+reveal_type(C.f(1))  # revealed: str
+```
+
+The method `f` can not be accessed from an instance of the class:
+
+```py
+# error: [unresolved-attribute] "Type `C` has no attribute `f`"
+C().f
+```
+
+A metaclass function can be shadowed by a method on the class:
+
+```py
+from typing import Any, Literal
+
+class D(metaclass=Meta):
+    def f(arg: int) -> Literal["a"]:
+        return "a"
+
+reveal_type(D.f(1))  # revealed: Literal["a"]
+```
+
+If the class method is possibly unbound, we union the return types:
+
+```py
+def flag() -> bool:
+    return True
+
+class E(metaclass=Meta):
+    if flag():
+        def f(arg: int) -> Any:
+            return "a"
+
+reveal_type(E.f(1))  # revealed: str | Any
+```
+
+## `@classmethod`
+
+### Basic
+
+When a `@classmethod` attribute is accessed, it returns a bound method object, even when accessed on
+the class object itself:
+
+```py
+from __future__ import annotations
+
+class C:
+    @classmethod
+    def f(cls: type[C], x: int) -> str:
+        return "a"
+
+reveal_type(C.f)  # revealed: <bound method `f` of `Literal[C]`>
+reveal_type(C().f)  # revealed: <bound method `f` of `type[C]`>
+```
+
+The `cls` method argument is then implicitly passed as the first argument when calling the method:
+
+```py
+reveal_type(C.f(1))  # revealed: str
+reveal_type(C().f(1))  # revealed: str
+```
+
+When the class method is called incorrectly, we detect it:
+
+```py
+C.f("incorrect")  # error: [invalid-argument-type]
+C.f()  # error: [missing-argument]
+C.f(1, 2)  # error: [too-many-positional-arguments]
+```
+
+If the `cls` parameter is wrongly annotated, we emit an error at the call site:
+
+```py
+class D:
+    @classmethod
+    def f(cls: D):
+        # This function is wrongly annotated, it should be `type[D]` instead of `D`
+        pass
+
+# error: [invalid-argument-type] "Object of type `Literal[D]` cannot be assigned to parameter 1 (`cls`) of bound method `f`; expected type `D`"
+D.f()
+```
+
+When a class method is accessed on a derived class, it is bound to that derived class:
+
+```py
+class Derived(C):
+    pass
+
+reveal_type(Derived.f)  # revealed: <bound method `f` of `Literal[Derived]`>
+reveal_type(Derived().f)  # revealed: <bound method `f` of `type[Derived]`>
+
+reveal_type(Derived.f(1))  # revealed: str
+reveal_type(Derived().f(1))  # revealed: str
+```
+
+### Accessing the classmethod as a static member
+
+Accessing a `@classmethod`-decorated function at runtime returns a `classmethod` object. We
+currently don't model this explicitly:
+
+```py
+from inspect import getattr_static
+
+class C:
+    @classmethod
+    def f(cls): ...
+
+reveal_type(getattr_static(C, "f"))  # revealed: Literal[f]
+reveal_type(getattr_static(C, "f").__get__)  # revealed: <method-wrapper `__get__` of `f`>
+```
+
+But we correctly model how the `classmethod` descriptor works:
+
+```py
+reveal_type(getattr_static(C, "f").__get__(None, C))  # revealed: <bound method `f` of `Literal[C]`>
+reveal_type(getattr_static(C, "f").__get__(C(), C))  # revealed: <bound method `f` of `Literal[C]`>
+reveal_type(getattr_static(C, "f").__get__(C()))  # revealed: <bound method `f` of `type[C]`>
+```
+
+The `owner` argument takes precedence over the `instance` argument:
+
+```py
+reveal_type(getattr_static(C, "f").__get__("dummy", C))  # revealed: <bound method `f` of `Literal[C]`>
+```
+
+### Classmethods mixed with other decorators
+
+When a `@classmethod` is additionally decorated with another decorator, it is still treated as a
+class method:
+
+```py
+from __future__ import annotations
+
+def does_nothing[T](f: T) -> T:
+    return f
+
+class C:
+    @classmethod
+    @does_nothing
+    def f1(cls: type[C], x: int) -> str:
+        return "a"
+
+    @does_nothing
+    @classmethod
+    def f2(cls: type[C], x: int) -> str:
+        return "a"
+
+# TODO: We do not support decorators yet (only limited special cases). Eventually,
+# these should all return `str`:
+
+reveal_type(C.f1(1))  # revealed: @Todo(return type of decorated function)
+reveal_type(C().f1(1))  # revealed: @Todo(return type of decorated function)
+
+reveal_type(C.f2(1))  # revealed: @Todo(return type of decorated function)
+reveal_type(C().f2(1))  # revealed: @Todo(return type of decorated function)
+```
+
+[functions and methods]: https://docs.python.org/3/howto/descriptor.html#functions-and-methods

crates/red_knot_python_semantic/resources/mdtest/call/never.md

@@ -0,0 +1,12 @@
+# Never is callable
+
+The type `Never` is callable with an arbitrary set of arguments. The result is always `Never`.
+
+```py
+from typing_extensions import Never
+
+def f(never: Never):
+    reveal_type(never())  # revealed: Never
+    reveal_type(never(1))  # revealed: Never
+    reveal_type(never(1, "a", never, x=None))  # revealed: Never
+```

crates/red_knot_python_semantic/resources/mdtest/call/subclass_of.md

@@ -0,0 +1,50 @@
+# Call `type[...]`
+
+## Single class
+
+### Trivial constructor
+
+```py
+class C: ...
+
+def _(subclass_of_c: type[C]):
+    reveal_type(subclass_of_c())  # revealed: C
+```
+
+### Non-trivial constructor
+
+```py
+class C:
+    def __init__(self, x: int): ...
+
+def _(subclass_of_c: type[C]):
+    reveal_type(subclass_of_c(1))  # revealed: C
+
+    # TODO: Those should all be errors
+    reveal_type(subclass_of_c("a"))  # revealed: C
+    reveal_type(subclass_of_c())  # revealed: C
+    reveal_type(subclass_of_c(1, 2))  # revealed: C
+```
+
+## Dynamic base
+
+```py
+from typing import Any
+from knot_extensions import Unknown
+
+def _(subclass_of_any: type[Any], subclass_of_unknown: type[Unknown]):
+    reveal_type(subclass_of_any())  # revealed: Any
+    reveal_type(subclass_of_any("any", "args", 1, 2))  # revealed: Any
+    reveal_type(subclass_of_unknown())  # revealed: Unknown
+    reveal_type(subclass_of_unknown("any", "args", 1, 2))  # revealed: Unknown
+```
+
+## Unions of classes
+
+```py
+class A: ...
+class B: ...
+
+def _(subclass_of_ab: type[A | B]):
+    reveal_type(subclass_of_ab())  # revealed: A | B
+```

crates/red_knot_python_semantic/resources/mdtest/call/union.md

@@ -39,8 +39,8 @@ def _(flag: bool):
     else:
         def f() -> int:
             return 1
-    x = f()  # error: "Object of type `Literal[1] | Literal[f]` is not callable (due to union element `Literal[1]`)"
-    reveal_type(x)  # revealed: Unknown | int
+    x = f()  # error: [call-non-callable] "Object of type `Literal[1]` is not callable"
+    reveal_type(x)  # revealed: int | Unknown
 ```
 
 ## Multiple non-callable elements in a union
@@ -56,8 +56,9 @@ def _(flag: bool, flag2: bool):
     else:
         def f() -> int:
             return 1
-    # error: "Object of type `Literal[1, "foo"] | Literal[f]` is not callable (due to union elements Literal[1], Literal["foo"])"
-    # revealed: Unknown | int
+    # TODO we should mention all non-callable elements of the union
+    # error: [call-non-callable] "Object of type `Literal[1]` is not callable"
+    # revealed: int | Unknown
     reveal_type(f())
 ```
 
@@ -72,6 +73,74 @@ def _(flag: bool):
     else:
         f = "foo"
 
-    x = f()  # error: "Object of type `Literal[1, "foo"]` is not callable"
+    x = f()  # error: [call-non-callable] "Object of type `Literal[1, "foo"]` is not callable"
+    reveal_type(x)  # revealed: Unknown
+```
+
+## Mismatching signatures
+
+Calling a union where the arguments don't match the signature of all variants.
+
+```py
+def f1(a: int) -> int: ...
+def f2(a: str) -> str: ...
+def _(flag: bool):
+    if flag:
+        f = f1
+    else:
+        f = f2
+
+    # error: [invalid-argument-type] "Object of type `Literal[3]` cannot be assigned to parameter 1 (`a`) of function `f2`; expected type `str`"
+    x = f(3)
+    reveal_type(x)  # revealed: int | str
+```
+
+## Any non-callable variant
+
+```py
+def f1(a: int): ...
+def _(flag: bool):
+    if flag:
+        f = f1
+    else:
+        f = "This is a string literal"
+
+    # error: [call-non-callable] "Object of type `Literal["This is a string literal"]` is not callable"
+    x = f(3)
+    reveal_type(x)  # revealed: Unknown
+```
+
+## Union of binding errors
+
+```py
+def f1(): ...
+def f2(): ...
+def _(flag: bool):
+    if flag:
+        f = f1
+    else:
+        f = f2
+
+    # TODO: we should show all errors from the union, not arbitrarily pick one union element
+    # error: [too-many-positional-arguments] "Too many positional arguments to function `f1`: expected 0, got 1"
+    x = f(3)
+    reveal_type(x)  # revealed: Unknown
+```
+
+## One not-callable, one wrong argument
+
+```py
+class C: ...
+
+def f1(): ...
+def _(flag: bool):
+    if flag:
+        f = f1
+    else:
+        f = C()
+
+    # TODO: we should either show all union errors here, or prioritize the not-callable error
+    # error: [too-many-positional-arguments] "Too many positional arguments to function `f1`: expected 0, got 1"
+    x = f(3)
     reveal_type(x)  # revealed: Unknown
 ```

crates/red_knot_python_semantic/resources/mdtest/comparison/instances/membership_test.md

@@ -21,8 +21,9 @@ class A:
 
 reveal_type("hello" in A())  # revealed: bool
 reveal_type("hello" not in A())  # revealed: bool
-# TODO: should emit diagnostic, need to check arg type, will fail
+# error: [unsupported-operator] "Operator `in` is not supported for types `int` and `A`, in comparing `Literal[42]` with `A`"
 reveal_type(42 in A())  # revealed: bool
+# error: [unsupported-operator] "Operator `not in` is not supported for types `int` and `A`, in comparing `Literal[42]` with `A`"
 reveal_type(42 not in A())  # revealed: bool
 ```
 
@@ -126,9 +127,9 @@ class A:
 
 reveal_type(CheckContains() in A())  # revealed: bool
 
-# TODO: should emit diagnostic, need to check arg type,
-# should not fall back to __iter__ or __getitem__
+# error: [unsupported-operator] "Operator `in` is not supported for types `CheckIter` and `A`"
 reveal_type(CheckIter() in A())  # revealed: bool
+# error: [unsupported-operator] "Operator `in` is not supported for types `CheckGetItem` and `A`"
 reveal_type(CheckGetItem() in A())  # revealed: bool
 
 class B:
@@ -154,7 +155,50 @@ class A:
     def __getitem__(self, key: str) -> str:
         return "foo"
 
-# TODO should emit a diagnostic
+# error: [unsupported-operator] "Operator `in` is not supported for types `int` and `A`, in comparing `Literal[42]` with `A`"
 reveal_type(42 in A())  # revealed: bool
+# error: [unsupported-operator] "Operator `in` is not supported for types `str` and `A`, in comparing `Literal["hello"]` with `A`"
 reveal_type("hello" in A())  # revealed: bool
 ```
+
+## Return type that doesn't implement `__bool__` correctly
+
+`in` and `not in` operations will fail at runtime if the object on the right-hand side of the
+operation has a `__contains__` method that returns a type which is not convertible to `bool`. This
+is because of the way these operations are handled by the Python interpreter at runtime. If we
+assume that `y` is an object that has a `__contains__` method, the Python expression `x in y`
+desugars to a `contains(y, x)` call, where `contains` looks something like this:
+
+```ignore
+def contains(y, x):
+    return bool(type(y).__contains__(y, x))
+```
+
+where the `bool()` conversion itself implicitly calls `__bool__` under the hood.
+
+TODO: Ideally the message would explain to the user what's wrong. E.g,
+
+```ignore
+error: [operator] cannot use `in` operator on object of type `WithContains`
+    note: This is because the `in` operator implicitly calls `WithContains.__contains__`, but `WithContains.__contains__` is invalidly defined
+    note: `WithContains.__contains__` is invalidly defined because it returns an instance of `NotBoolable`, which cannot be evaluated in a boolean context
+    note: `NotBoolable` cannot be evaluated in a boolean context because its `__bool__` attribute is not callable
+```
+
+It may also be more appropriate to use `unsupported-operator` as the error code.
+
+<!-- snapshot-diagnostics -->
+
+```py
+class NotBoolable:
+    __bool__ = 3
+
+class WithContains:
+    def __contains__(self, item) -> NotBoolable:
+        return NotBoolable()
+
+# error: [unsupported-bool-conversion]
+10 in WithContains()
+# error: [unsupported-bool-conversion]
+10 not in WithContains()
+```

crates/red_knot_python_semantic/resources/mdtest/comparison/instances/rich_comparison.md

@@ -16,31 +16,38 @@ most common case involves implementing these methods for the same type:
 ```py
 from __future__ import annotations
 
+class EqReturnType: ...
+class NeReturnType: ...
+class LtReturnType: ...
+class LeReturnType: ...
+class GtReturnType: ...
+class GeReturnType: ...
+
 class A:
-    def __eq__(self, other: A) -> int:
-        return 42
+    def __eq__(self, other: A) -> EqReturnType:
+        return EqReturnType()
 
-    def __ne__(self, other: A) -> float:
-        return 42.0
+    def __ne__(self, other: A) -> NeReturnType:
+        return NeReturnType()
 
-    def __lt__(self, other: A) -> str:
-        return "42"
+    def __lt__(self, other: A) -> LtReturnType:
+        return LtReturnType()
 
-    def __le__(self, other: A) -> bytes:
-        return b"42"
+    def __le__(self, other: A) -> LeReturnType:
+        return LeReturnType()
 
-    def __gt__(self, other: A) -> list:
-        return [42]
+    def __gt__(self, other: A) -> GtReturnType:
+        return GtReturnType()
 
-    def __ge__(self, other: A) -> set:
-        return {42}
+    def __ge__(self, other: A) -> GeReturnType:
+        return GeReturnType()
 
-reveal_type(A() == A())  # revealed: int
-reveal_type(A() != A())  # revealed: float
-reveal_type(A() < A())  # revealed: str
-reveal_type(A() <= A())  # revealed: bytes
-reveal_type(A() > A())  # revealed: list
-reveal_type(A() >= A())  # revealed: set
+reveal_type(A() == A())  # revealed: EqReturnType
+reveal_type(A() != A())  # revealed: NeReturnType
+reveal_type(A() < A())  # revealed: LtReturnType
+reveal_type(A() <= A())  # revealed: LeReturnType
+reveal_type(A() > A())  # revealed: GtReturnType
+reveal_type(A() >= A())  # revealed: GeReturnType
 ```
 
 ## Rich Comparison Dunder Implementations for Other Class
@@ -51,33 +58,40 @@ type:
 ```py
 from __future__ import annotations
 
+class EqReturnType: ...
+class NeReturnType: ...
+class LtReturnType: ...
+class LeReturnType: ...
+class GtReturnType: ...
+class GeReturnType: ...
+
 class A:
-    def __eq__(self, other: B) -> int:
-        return 42
+    def __eq__(self, other: B) -> EqReturnType:
+        return EqReturnType()
 
-    def __ne__(self, other: B) -> float:
-        return 42.0
+    def __ne__(self, other: B) -> NeReturnType:
+        return NeReturnType()
 
-    def __lt__(self, other: B) -> str:
-        return "42"
+    def __lt__(self, other: B) -> LtReturnType:
+        return LtReturnType()
 
-    def __le__(self, other: B) -> bytes:
-        return b"42"
+    def __le__(self, other: B) -> LeReturnType:
+        return LeReturnType()
 
-    def __gt__(self, other: B) -> list:
-        return [42]
+    def __gt__(self, other: B) -> GtReturnType:
+        return GtReturnType()
 
-    def __ge__(self, other: B) -> set:
-        return {42}
+    def __ge__(self, other: B) -> GeReturnType:
+        return GeReturnType()
 
 class B: ...
 
-reveal_type(A() == B())  # revealed: int
-reveal_type(A() != B())  # revealed: float
-reveal_type(A() < B())  # revealed: str
-reveal_type(A() <= B())  # revealed: bytes
-reveal_type(A() > B())  # revealed: list
-reveal_type(A() >= B())  # revealed: set
+reveal_type(A() == B())  # revealed: EqReturnType
+reveal_type(A() != B())  # revealed: NeReturnType
+reveal_type(A() < B())  # revealed: LtReturnType
+reveal_type(A() <= B())  # revealed: LeReturnType
+reveal_type(A() > B())  # revealed: GtReturnType
+reveal_type(A() >= B())  # revealed: GeReturnType
 ```
 
 ## Reflected Comparisons
@@ -89,58 +103,64 @@ these methods will be ignored here because they require a mismatched operand typ
 ```py
 from __future__ import annotations
 
+class EqReturnType: ...
+class NeReturnType: ...
+class LtReturnType: ...
+class LeReturnType: ...
+class GtReturnType: ...
+class GeReturnType: ...
+
 class A:
-    def __eq__(self, other: B) -> int:
-        return 42
+    def __eq__(self, other: B) -> EqReturnType:
+        return EqReturnType()
 
-    def __ne__(self, other: B) -> float:
-        return 42.0
+    def __ne__(self, other: B) -> NeReturnType:
+        return NeReturnType()
 
-    def __lt__(self, other: B) -> str:
-        return "42"
+    def __lt__(self, other: B) -> LtReturnType:
+        return LtReturnType()
 
-    def __le__(self, other: B) -> bytes:
-        return b"42"
+    def __le__(self, other: B) -> LeReturnType:
+        return LeReturnType()
 
-    def __gt__(self, other: B) -> list:
-        return [42]
+    def __gt__(self, other: B) -> GtReturnType:
+        return GtReturnType()
 
-    def __ge__(self, other: B) -> set:
-        return {42}
+    def __ge__(self, other: B) -> GeReturnType:
+        return GeReturnType()
+
+class Unrelated: ...
 
 class B:
     # To override builtins.object.__eq__ and builtins.object.__ne__
     # TODO these should emit an invalid override diagnostic
-    def __eq__(self, other: str) -> B:
+    def __eq__(self, other: Unrelated) -> B:
         return B()
 
-    def __ne__(self, other: str) -> B:
+    def __ne__(self, other: Unrelated) -> B:
         return B()
 
-# TODO: should be `int` and `float`.
-# Need to check arg type and fall back to `rhs.__eq__` and `rhs.__ne__`.
-#
 # Because `object.__eq__` and `object.__ne__` accept `object` in typeshed,
 # this can only happen with an invalid override of these methods,
 # but we still support it.
-reveal_type(B() == A())  # revealed: B
-reveal_type(B() != A())  # revealed: B
+reveal_type(B() == A())  # revealed: EqReturnType
+reveal_type(B() != A())  # revealed: NeReturnType
 
-reveal_type(B() < A())  # revealed: list
-reveal_type(B() <= A())  # revealed: set
+reveal_type(B() < A())  # revealed: GtReturnType
+reveal_type(B() <= A())  # revealed: GeReturnType
 
-reveal_type(B() > A())  # revealed: str
-reveal_type(B() >= A())  # revealed: bytes
+reveal_type(B() > A())  # revealed: LtReturnType
+reveal_type(B() >= A())  # revealed: LeReturnType
 
 class C:
-    def __gt__(self, other: C) -> int:
+    def __gt__(self, other: C) -> EqReturnType:
         return 42
 
-    def __ge__(self, other: C) -> float:
-        return 42.0
+    def __ge__(self, other: C) -> NeReturnType:
+        return NeReturnType()
 
-reveal_type(C() < C())  # revealed: int
-reveal_type(C() <= C())  # revealed: float
+reveal_type(C() < C())  # revealed: EqReturnType
+reveal_type(C() <= C())  # revealed: NeReturnType
 ```
 
 ## Reflected Comparisons with Subclasses
@@ -152,6 +172,13 @@ than `A`.
 ```py
 from __future__ import annotations
 
+class EqReturnType: ...
+class NeReturnType: ...
+class LtReturnType: ...
+class LeReturnType: ...
+class GtReturnType: ...
+class GeReturnType: ...
+
 class A:
     def __eq__(self, other: A) -> A:
         return A()
@@ -172,32 +199,32 @@ class A:
         return A()
 
 class B(A):
-    def __eq__(self, other: A) -> int:
-        return 42
+    def __eq__(self, other: A) -> EqReturnType:
+        return EqReturnType()
 
-    def __ne__(self, other: A) -> float:
-        return 42.0
+    def __ne__(self, other: A) -> NeReturnType:
+        return NeReturnType()
 
-    def __lt__(self, other: A) -> str:
-        return "42"
+    def __lt__(self, other: A) -> LtReturnType:
+        return LtReturnType()
 
-    def __le__(self, other: A) -> bytes:
-        return b"42"
+    def __le__(self, other: A) -> LeReturnType:
+        return LeReturnType()
 
-    def __gt__(self, other: A) -> list:
-        return [42]
+    def __gt__(self, other: A) -> GtReturnType:
+        return GtReturnType()
 
-    def __ge__(self, other: A) -> set:
-        return {42}
+    def __ge__(self, other: A) -> GeReturnType:
+        return GeReturnType()
 
-reveal_type(A() == B())  # revealed: int
-reveal_type(A() != B())  # revealed: float
+reveal_type(A() == B())  # revealed: EqReturnType
+reveal_type(A() != B())  # revealed: NeReturnType
 
-reveal_type(A() < B())  # revealed: list
-reveal_type(A() <= B())  # revealed: set
+reveal_type(A() < B())  # revealed: GtReturnType
+reveal_type(A() <= B())  # revealed: GeReturnType
 
-reveal_type(A() > B())  # revealed: str
-reveal_type(A() >= B())  # revealed: bytes
+reveal_type(A() > B())  # revealed: LtReturnType
+reveal_type(A() >= B())  # revealed: LeReturnType
 ```
 
 ## Reflected Comparisons with Subclass But Falls Back to LHS
@@ -222,9 +249,8 @@ class B(A):
     def __gt__(self, other: int) -> B:
         return B()
 
-# TODO: should be `A`, need to check argument type and fall back to LHS method
-reveal_type(A() < B())  # revealed: B
-reveal_type(A() > B())  # revealed: B
+reveal_type(A() < B())  # revealed: A
+reveal_type(A() > B())  # revealed: A
 ```
 
 ## Operations involving instances of classes inheriting from `Any`
@@ -272,9 +298,8 @@ class A:
     def __ne__(self, other: int) -> A:
         return A()
 
-# TODO: it should be `bool`, need to check arg type and fall back to `is` and `is not`
-reveal_type(A() == A())  # revealed: A
-reveal_type(A() != A())  # revealed: A
+reveal_type(A() == A())  # revealed: bool
+reveal_type(A() != A())  # revealed: bool
 ```
 
 ## Object Comparisons with Typeshed
@@ -305,12 +330,14 @@ reveal_type(1 >= 1.0)  # revealed: bool
 reveal_type(1 == 2j)  # revealed: bool
 reveal_type(1 != 2j)  # revealed: bool
 
-# TODO: should be Unknown and emit diagnostic,
-# need to check arg type and should be failed
-reveal_type(1 < 2j)  # revealed: bool
-reveal_type(1 <= 2j)  # revealed: bool
-reveal_type(1 > 2j)  # revealed: bool
-reveal_type(1 >= 2j)  # revealed: bool
+# error: [unsupported-operator] "Operator `<` is not supported for types `int` and `complex`, in comparing `Literal[1]` with `complex`"
+reveal_type(1 < 2j)  # revealed: Unknown
+# error: [unsupported-operator] "Operator `<=` is not supported for types `int` and `complex`, in comparing `Literal[1]` with `complex`"
+reveal_type(1 <= 2j)  # revealed: Unknown
+# error: [unsupported-operator] "Operator `>` is not supported for types `int` and `complex`, in comparing `Literal[1]` with `complex`"
+reveal_type(1 > 2j)  # revealed: Unknown
+# error: [unsupported-operator] "Operator `>=` is not supported for types `int` and `complex`, in comparing `Literal[1]` with `complex`"
+reveal_type(1 >= 2j)  # revealed: Unknown
 
 def f(x: bool, y: int):
     reveal_type(x < y)  # revealed: bool
@@ -318,3 +345,47 @@ def f(x: bool, y: int):
     reveal_type(4.2 < x)  # revealed: bool
     reveal_type(x < 4.2)  # revealed: bool
 ```
+
+## Chained comparisons with objects that don't implement `__bool__` correctly
+
+<!-- snapshot-diagnostics -->
+
+Python implicitly calls `bool` on the comparison result of preceding elements (but not for the last
+element) of a chained comparison.
+
+```py
+class NotBoolable:
+    __bool__ = 3
+
+class Comparable:
+    def __lt__(self, item) -> NotBoolable:
+        return NotBoolable()
+
+    def __gt__(self, item) -> NotBoolable:
+        return NotBoolable()
+
+# error: [unsupported-bool-conversion]
+10 < Comparable() < 20
+# error: [unsupported-bool-conversion]
+10 < Comparable() < Comparable()
+
+Comparable() < Comparable()  # fine
+```
+
+## Callables as comparison dunders
+
+```py
+from typing import Literal
+
+class AlwaysTrue:
+    def __call__(self, other: object) -> Literal[True]:
+        return True
+
+class A:
+    __eq__: AlwaysTrue = AlwaysTrue()
+    __lt__: AlwaysTrue = AlwaysTrue()
+
+reveal_type(A() == A())  # revealed: Literal[True]
+reveal_type(A() < A())  # revealed: Literal[True]
+reveal_type(A() > A())  # revealed: Literal[True]
+```

crates/red_knot_python_semantic/resources/mdtest/comparison/integers.md

@@ -12,8 +12,8 @@ reveal_type(1 is 1)  # revealed: bool
 reveal_type(1 is not 1)  # revealed: bool
 reveal_type(1 is 2)  # revealed: Literal[False]
 reveal_type(1 is not 7)  # revealed: Literal[True]
-# TODO: should be Unknown, and emit diagnostic, once we check call argument types
-reveal_type(1 <= "" and 0 < 1)  # revealed: bool
+# error: [unsupported-operator] "Operator `<=` is not supported for types `int` and `str`, in comparing `Literal[1]` with `Literal[""]`"
+reveal_type(1 <= "" and 0 < 1)  # revealed: Unknown & ~AlwaysTruthy | Literal[True]
 ```
 
 ## Integer instance

crates/red_knot_python_semantic/resources/mdtest/comparison/intersections.md

@@ -8,7 +8,9 @@ types, we can infer that the result for the intersection type is also true/false
 ```py
 from typing import Literal
 
-class Base: ...
+class Base:
+    def __gt__(self, other) -> bool:
+        return False
 
 class Child1(Base):
     def __eq__(self, other) -> Literal[True]:

crates/red_knot_python_semantic/resources/mdtest/comparison/non_bool_returns.md

@@ -23,6 +23,7 @@ from __future__ import annotations
 
 class A:
     def __lt__(self, other) -> A: ...
+    def __gt__(self, other) -> bool: ...
 
 class B:
     def __lt__(self, other) -> B: ...

crates/red_knot_python_semantic/resources/mdtest/comparison/tuples.md

@@ -92,11 +92,14 @@ reveal_type(a == b)  # revealed: bool
 # TODO: should be Literal[True], once we implement (in)equality for mismatched literals
 reveal_type(a != b)  # revealed: bool
 
-# TODO: should be Unknown and add more informative diagnostics
-reveal_type(a < b)  # revealed: bool
-reveal_type(a <= b)  # revealed: bool
-reveal_type(a > b)  # revealed: bool
-reveal_type(a >= b)  # revealed: bool
+# error: [unsupported-operator] "Operator `<` is not supported for types `int` and `str`, in comparing `tuple[Literal[1], Literal[2]]` with `tuple[Literal[1], Literal["hello"]]`"
+reveal_type(a < b)  # revealed: Unknown
+# error: [unsupported-operator] "Operator `<=` is not supported for types `int` and `str`, in comparing `tuple[Literal[1], Literal[2]]` with `tuple[Literal[1], Literal["hello"]]`"
+reveal_type(a <= b)  # revealed: Unknown
+# error: [unsupported-operator] "Operator `>` is not supported for types `int` and `str`, in comparing `tuple[Literal[1], Literal[2]]` with `tuple[Literal[1], Literal["hello"]]`"
+reveal_type(a > b)  # revealed: Unknown
+# error: [unsupported-operator] "Operator `>=` is not supported for types `int` and `str`, in comparing `tuple[Literal[1], Literal[2]]` with `tuple[Literal[1], Literal["hello"]]`"
+reveal_type(a >= b)  # revealed: Unknown
 ```
 
 However, if the lexicographic comparison completes without reaching a point where str and int are
@@ -144,33 +147,40 @@ of the dunder methods.)
 ```py
 from __future__ import annotations
 
+class EqReturnType: ...
+class NeReturnType: ...
+class LtReturnType: ...
+class LeReturnType: ...
+class GtReturnType: ...
+class GeReturnType: ...
+
 class A:
-    def __eq__(self, o: object) -> str:
-        return "hello"
+    def __eq__(self, o: object) -> EqReturnType:
+        return EqReturnType()
 
-    def __ne__(self, o: object) -> bytes:
-        return b"world"
+    def __ne__(self, o: object) -> NeReturnType:
+        return NeReturnType()
 
-    def __lt__(self, o: A) -> float:
-        return 3.14
+    def __lt__(self, o: A) -> LtReturnType:
+        return LtReturnType()
 
-    def __le__(self, o: A) -> complex:
-        return complex(0.5, -0.5)
+    def __le__(self, o: A) -> LeReturnType:
+        return LeReturnType()
 
-    def __gt__(self, o: A) -> tuple:
-        return (1, 2, 3)
+    def __gt__(self, o: A) -> GtReturnType:
+        return GtReturnType()
 
-    def __ge__(self, o: A) -> list:
-        return [1, 2, 3]
+    def __ge__(self, o: A) -> GeReturnType:
+        return GeReturnType()
 
 a = (A(), A())
 
 reveal_type(a == a)  # revealed: bool
 reveal_type(a != a)  # revealed: bool
-reveal_type(a < a)  # revealed: float | Literal[False]
-reveal_type(a <= a)  # revealed: complex | Literal[True]
-reveal_type(a > a)  # revealed: tuple | Literal[False]
-reveal_type(a >= a)  # revealed: list | Literal[True]
+reveal_type(a < a)  # revealed: LtReturnType | Literal[False]
+reveal_type(a <= a)  # revealed: LeReturnType | Literal[True]
+reveal_type(a > a)  # revealed: GtReturnType | Literal[False]
+reveal_type(a >= a)  # revealed: GeReturnType | Literal[True]
 
 # If lexicographic comparison is finished before comparing A()
 b = ("1_foo", A())
@@ -183,11 +193,13 @@ reveal_type(b <= c)  # revealed: Literal[True]
 reveal_type(b > c)  # revealed: Literal[False]
 reveal_type(b >= c)  # revealed: Literal[False]
 
+class LtReturnTypeOnB: ...
+
 class B:
-    def __lt__(self, o: B) -> set:
+    def __lt__(self, o: B) -> LtReturnTypeOnB:
         return set()
 
-reveal_type((A(), B()) < (A(), B()))  # revealed: float | set | Literal[False]
+reveal_type((A(), B()) < (A(), B()))  # revealed: LtReturnType | LtReturnTypeOnB | Literal[False]
 ```
 
 #### Special Handling of Eq and NotEq in Lexicographic Comparisons
@@ -322,3 +334,61 @@ reveal_type(a is not c)  # revealed: Literal[True]
 For tuples like `tuple[int, ...]`, `tuple[Any, ...]`
 
 // TODO
+
+## Chained comparisons with elements that incorrectly implement `__bool__`
+
+<!-- snapshot-diagnostics -->
+
+For an operation `A() < A()` to succeed at runtime, the `A.__lt__` method does not necessarily need
+to return an object that is convertible to a `bool`. However, the return type _does_ need to be
+convertible to a `bool` for the operation `A() < A() < A()` (a _chained_ comparison) to succeed.
+This is because `A() < A() < A()` desugars to something like this, which involves several implicit
+conversions to `bool`:
+
+```ignore
+def compute_chained_comparison():
+  a1 = A()
+  a2 = A()
+  first_comparison = a1 < a2
+  return first_comparison and (a2 < A())
+```
+
+```py
+class NotBoolable:
+    __bool__ = 5
+
+class Comparable:
+    def __lt__(self, other) -> NotBoolable:
+        return NotBoolable()
+
+    def __gt__(self, other) -> NotBoolable:
+        return NotBoolable()
+
+a = (1, Comparable())
+b = (1, Comparable())
+
+# error: [unsupported-bool-conversion]
+a < b < b
+
+a < b  # fine
+```
+
+## Equality with elements that incorrectly implement `__bool__`
+
+<!-- snapshot-diagnostics -->
+
+Python does not generally attempt to coerce the result of `==` and `!=` operations between two
+arbitrary objects to a `bool`, but a comparison of tuples will fail if the result of comparing any
+pair of elements at equivalent positions cannot be converted to a `bool`:
+
+```py
+class A:
+    def __eq__(self, other) -> NotBoolable:
+        return NotBoolable()
+
+class NotBoolable:
+    __bool__ = None
+
+# error: [unsupported-bool-conversion]
+(A(),) == (A(),)
+```

crates/red_knot_python_semantic/resources/mdtest/comparison/unsupported.md

@@ -9,28 +9,22 @@ def _(flag: bool, flag1: bool, flag2: bool):
     b = 0 not in 10  # error: "Operator `not in` is not supported for types `Literal[0]` and `Literal[10]`"
     reveal_type(b)  # revealed: bool
 
-    # TODO: should error, once operand type check is implemented
-    # ("Operator `<` is not supported for types `object` and `int`")
+    # error: [unsupported-operator] "Operator `<` is not supported for types `object` and `int`, in comparing `object` with `Literal[5]`"
     c = object() < 5
-    # TODO: should be Unknown, once operand type check is implemented
-    reveal_type(c)  # revealed: bool
+    reveal_type(c)  # revealed: Unknown
 
-    # TODO: should error, once operand type check is implemented
-    # ("Operator `<` is not supported for types `int` and `object`")
+    # error: [unsupported-operator] "Operator `<` is not supported for types `int` and `object`, in comparing `Literal[5]` with `object`"
     d = 5 < object()
-    # TODO: should be Unknown, once operand type check is implemented
-    reveal_type(d)  # revealed: bool
+    reveal_type(d)  # revealed: Unknown
 
     int_literal_or_str_literal = 1 if flag else "foo"
     # error: "Operator `in` is not supported for types `Literal[42]` and `Literal[1]`, in comparing `Literal[42]` with `Literal[1, "foo"]`"
     e = 42 in int_literal_or_str_literal
     reveal_type(e)  # revealed: bool
 
-    # TODO: should error, need to check if __lt__ signature is valid for right operand
-    # error may be "Operator `<` is not supported for types `int` and `str`, in comparing `tuple[Literal[1], Literal[2]]` with `tuple[Literal[1], Literal["hello"]]`
+    # error: [unsupported-operator] "Operator `<` is not supported for types `int` and `str`, in comparing `tuple[Literal[1], Literal[2]]` with `tuple[Literal[1], Literal["hello"]]`"
     f = (1, 2) < (1, "hello")
-    # TODO: should be Unknown, once operand type check is implemented
-    reveal_type(f)  # revealed: bool
+    reveal_type(f)  # revealed: Unknown
 
     # error: [unsupported-operator] "Operator `<` is not supported for types `A` and `A`, in comparing `tuple[bool, A]` with `tuple[bool, A]`"
     g = (flag1, A()) < (flag2, A())

crates/red_knot_python_semantic/resources/mdtest/comprehensions/basic.md

@@ -43,8 +43,7 @@ class IntIterable:
     def __iter__(self) -> IntIterator:
         return IntIterator()
 
-# TODO: This could be a `tuple[int, int]` if we model that `y` can not be modified in the outer comprehension scope
-# revealed: tuple[int, Unknown | int]
+# revealed: tuple[int, int]
 [[reveal_type((x, y)) for x in IntIterable()] for y in IntIterable()]
 ```
 
@@ -67,8 +66,7 @@ class IterableOfIterables:
     def __iter__(self) -> IteratorOfIterables:
         return IteratorOfIterables()
 
-# TODO: This could be a `tuple[int, int]` (see above)
-# revealed: tuple[int, Unknown | IntIterable]
+# revealed: tuple[int, IntIterable]
 [[reveal_type((x, y)) for x in y] for y in IterableOfIterables()]
 ```
 

crates/red_knot_python_semantic/resources/mdtest/conditional/if_expression.md

@@ -35,3 +35,13 @@ def _(flag: bool):
     x = 1 if flag else None
     reveal_type(x)  # revealed: Literal[1] | None
 ```
+
+## Condition with object that implements `__bool__` incorrectly
+
+```py
+class NotBoolable:
+    __bool__ = 3
+
+# error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable"
+3 if NotBoolable() else 4
+```

crates/red_knot_python_semantic/resources/mdtest/conditional/if_statement.md

@@ -147,3 +147,17 @@ def _(flag: bool):
 
     reveal_type(y)  # revealed: Literal[0, 1]
 ```
+
+## Condition with object that implements `__bool__` incorrectly
+
+```py
+class NotBoolable:
+    __bool__ = 3
+
+# error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable"
+if NotBoolable():
+    ...
+# error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable"
+elif NotBoolable():
+    ...
+```

crates/red_knot_python_semantic/resources/mdtest/conditional/match.md

@@ -43,3 +43,21 @@ def _(target: int):
 
     reveal_type(y)  # revealed: Literal[2, 3, 4]
 ```
+
+## Guard with object that implements `__bool__` incorrectly
+
+```py
+class NotBoolable:
+    __bool__ = 3
+
+def _(target: int, flag: NotBoolable):
+    y = 1
+    match target:
+        # error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable"
+        case 1 if flag:
+            y = 2
+        case 2:
+            y = 3
+
+    reveal_type(y)  # revealed: Literal[1, 2, 3]
+```

crates/red_knot_python_semantic/resources/mdtest/descriptor_protocol.md

@@ -22,22 +22,22 @@ class Ten:
         pass
 
 class C:
-    ten = Ten()
+    ten: Ten = Ten()
 
 c = C()
 
-# TODO: this should be `Literal[10]`
-reveal_type(c.ten)  # revealed: Unknown | Ten
+reveal_type(c.ten)  # revealed: Literal[10]
 
-# TODO: This should `Literal[10]`
-reveal_type(C.ten)  # revealed: Unknown | Ten
+reveal_type(C.ten)  # revealed: Literal[10]
 
 # These are fine:
 c.ten = 10
 C.ten = 10
 
-# TODO: Both of these should be errors
+# error: [invalid-assignment] "Object of type `Literal[11]` is not assignable to attribute `ten` of type `Literal[10]`"
 c.ten = 11
+
+# error: [invalid-assignment] "Object of type `Literal[11]` is not assignable to attribute `ten` of type `Literal[10]`"
 C.ten = 11
 ```
 
@@ -57,24 +57,302 @@ class FlexibleInt:
         self._value = int(value)
 
 class C:
-    flexible_int = FlexibleInt()
+    flexible_int: FlexibleInt = FlexibleInt()
 
 c = C()
 
-# TODO: should be `int | None`
-reveal_type(c.flexible_int)  # revealed: Unknown | FlexibleInt
+reveal_type(c.flexible_int)  # revealed: int | None
 
 c.flexible_int = 42  # okay
+# TODO: This should not be an error
+# error: [invalid-assignment]
 c.flexible_int = "42"  # also okay!
 
-# TODO: should be `int | None`
-reveal_type(c.flexible_int)  # revealed: Unknown | FlexibleInt
+reveal_type(c.flexible_int)  # revealed: int | None
 
-# TODO: should be an error
+# TODO: This should be an error
 c.flexible_int = None  # not okay
 
-# TODO: should be `int | None`
-reveal_type(c.flexible_int)  # revealed: Unknown | FlexibleInt
+reveal_type(c.flexible_int)  # revealed: int | None
+```
+
+## Data and non-data descriptors
+
+Descriptors that define `__set__` or `__delete__` are called *data descriptors*. An example of a
+data descriptor is a `property` with a setter and/or a deleter. Descriptors that only define
+`__get__`, meanwhile, are called *non-data descriptors*. Examples include functions, `classmethod`
+or `staticmethod`.
+
+The precedence chain for attribute access is (1) data descriptors, (2) instance attributes, and (3)
+non-data descriptors.
+
+```py
+from typing import Literal
+
+class DataDescriptor:
+    def __get__(self, instance: object, owner: type | None = None) -> Literal["data"]:
+        return "data"
+
+    def __set__(self, instance: object, value: int) -> None:
+        pass
+
+class NonDataDescriptor:
+    def __get__(self, instance: object, owner: type | None = None) -> Literal["non-data"]:
+        return "non-data"
+
+class C:
+    data_descriptor = DataDescriptor()
+    non_data_descriptor = NonDataDescriptor()
+
+    def f(self):
+        # This explains why data descriptors come first in the precedence chain. If
+        # instance attributes would take priority, we would override the descriptor
+        # here. Instead, this calls `DataDescriptor.__set__`, i.e. it does not affect
+        # the type of the `data_descriptor` attribute.
+        self.data_descriptor = 1
+
+        # However, for non-data descriptors, instance attributes do take precedence.
+        # So it is possible to override them.
+        self.non_data_descriptor = 1
+
+c = C()
+
+reveal_type(c.data_descriptor)  # revealed: Unknown | Literal["data"]
+
+reveal_type(c.non_data_descriptor)  # revealed: Unknown | Literal["non-data", 1]
+
+reveal_type(C.data_descriptor)  # revealed: Unknown | Literal["data"]
+
+reveal_type(C.non_data_descriptor)  # revealed: Unknown | Literal["non-data"]
+
+# It is possible to override data descriptors via class objects. The following
+# assignment does not call `DataDescriptor.__set__`. For this reason, we infer
+# `Unknown | …` for all (descriptor) attributes.
+C.data_descriptor = "something else"  # This is okay
+```
+
+## Descriptor protocol for class objects
+
+When attributes are accessed on a class object, the following [precedence chain] is used:
+
+- Data descriptor on the metaclass
+- Data or non-data descriptor on the class
+- Class attribute
+- Non-data descriptor on the metaclass
+- Metaclass attribute
+
+To verify this, we define a data and a non-data descriptor:
+
+```py
+from typing import Literal, Any
+
+class DataDescriptor:
+    def __get__(self, instance: object, owner: type | None = None) -> Literal["data"]:
+        return "data"
+
+    def __set__(self, instance: object, value: str) -> None:
+        pass
+
+class NonDataDescriptor:
+    def __get__(self, instance: object, owner: type | None = None) -> Literal["non-data"]:
+        return "non-data"
+```
+
+First, we make sure that the descriptors are correctly accessed when defined on the metaclass or the
+class:
+
+```py
+class Meta1(type):
+    meta_data_descriptor: DataDescriptor = DataDescriptor()
+    meta_non_data_descriptor: NonDataDescriptor = NonDataDescriptor()
+
+class C1(metaclass=Meta1):
+    class_data_descriptor: DataDescriptor = DataDescriptor()
+    class_non_data_descriptor: NonDataDescriptor = NonDataDescriptor()
+
+reveal_type(C1.meta_data_descriptor)  # revealed: Literal["data"]
+reveal_type(C1.meta_non_data_descriptor)  # revealed: Literal["non-data"]
+
+reveal_type(C1.class_data_descriptor)  # revealed: Literal["data"]
+reveal_type(C1.class_non_data_descriptor)  # revealed: Literal["non-data"]
+```
+
+Next, we demonstrate that a *metaclass data descriptor* takes precedence over all class-level
+attributes:
+
+```py
+class Meta2(type):
+    meta_data_descriptor1: DataDescriptor = DataDescriptor()
+    meta_data_descriptor2: DataDescriptor = DataDescriptor()
+
+class ClassLevelDataDescriptor:
+    def __get__(self, instance: object, owner: type | None = None) -> Literal["class level data descriptor"]:
+        return "class level data descriptor"
+
+    def __set__(self, instance: object, value: str) -> None:
+        pass
+
+class C2(metaclass=Meta2):
+    meta_data_descriptor1: Literal["value on class"] = "value on class"
+    meta_data_descriptor2: ClassLevelDataDescriptor = ClassLevelDataDescriptor()
+
+reveal_type(C2.meta_data_descriptor1)  # revealed: Literal["data"]
+reveal_type(C2.meta_data_descriptor2)  # revealed: Literal["data"]
+```
+
+On the other hand, normal metaclass attributes and metaclass non-data descriptors are shadowed by
+class-level attributes (descriptor or not):
+
+```py
+class Meta3(type):
+    meta_attribute1: Literal["value on metaclass"] = "value on metaclass"
+    meta_attribute2: Literal["value on metaclass"] = "value on metaclass"
+    meta_non_data_descriptor1: NonDataDescriptor = NonDataDescriptor()
+    meta_non_data_descriptor2: NonDataDescriptor = NonDataDescriptor()
+
+class C3(metaclass=Meta3):
+    meta_attribute1: Literal["value on class"] = "value on class"
+    meta_attribute2: ClassLevelDataDescriptor = ClassLevelDataDescriptor()
+    meta_non_data_descriptor1: Literal["value on class"] = "value on class"
+    meta_non_data_descriptor2: ClassLevelDataDescriptor = ClassLevelDataDescriptor()
+
+reveal_type(C3.meta_attribute1)  # revealed: Literal["value on class"]
+reveal_type(C3.meta_attribute2)  # revealed: Literal["class level data descriptor"]
+reveal_type(C3.meta_non_data_descriptor1)  # revealed: Literal["value on class"]
+reveal_type(C3.meta_non_data_descriptor2)  # revealed: Literal["class level data descriptor"]
+```
+
+Finally, metaclass attributes and metaclass non-data descriptors are only accessible when they are
+not shadowed by class-level attributes:
+
+```py
+class Meta4(type):
+    meta_attribute: Literal["value on metaclass"] = "value on metaclass"
+    meta_non_data_descriptor: NonDataDescriptor = NonDataDescriptor()
+
+class C4(metaclass=Meta4): ...
+
+reveal_type(C4.meta_attribute)  # revealed: Literal["value on metaclass"]
+reveal_type(C4.meta_non_data_descriptor)  # revealed: Literal["non-data"]
+```
+
+When a metaclass data descriptor is possibly unbound, we union the result type of its `__get__`
+method with an underlying class level attribute, if present:
+
+```py
+def _(flag: bool):
+    class Meta5(type):
+        if flag:
+            meta_data_descriptor1: DataDescriptor = DataDescriptor()
+            meta_data_descriptor2: DataDescriptor = DataDescriptor()
+
+    class C5(metaclass=Meta5):
+        meta_data_descriptor1: Literal["value on class"] = "value on class"
+
+    reveal_type(C5.meta_data_descriptor1)  # revealed: Literal["data", "value on class"]
+    # error: [possibly-unbound-attribute]
+    reveal_type(C5.meta_data_descriptor2)  # revealed: Literal["data"]
+```
+
+When a class-level attribute is possibly unbound, we union its (descriptor protocol) type with the
+metaclass attribute (unless it's a data descriptor, which always takes precedence):
+
+```py
+from typing import Any
+
+def _(flag: bool):
+    class Meta6(type):
+        attribute1: DataDescriptor = DataDescriptor()
+        attribute2: NonDataDescriptor = NonDataDescriptor()
+        attribute3: Literal["value on metaclass"] = "value on metaclass"
+
+    class C6(metaclass=Meta6):
+        if flag:
+            attribute1: Literal["value on class"] = "value on class"
+            attribute2: Literal["value on class"] = "value on class"
+            attribute3: Literal["value on class"] = "value on class"
+            attribute4: Literal["value on class"] = "value on class"
+
+    reveal_type(C6.attribute1)  # revealed: Literal["data"]
+    reveal_type(C6.attribute2)  # revealed: Literal["non-data", "value on class"]
+    reveal_type(C6.attribute3)  # revealed: Literal["value on metaclass", "value on class"]
+    # error: [possibly-unbound-attribute]
+    reveal_type(C6.attribute4)  # revealed: Literal["value on class"]
+```
+
+Finally, we can also have unions of various types of attributes:
+
+```py
+def _(flag: bool):
+    class Meta7(type):
+        if flag:
+            union_of_metaclass_attributes: Literal[1] = 1
+            union_of_metaclass_data_descriptor_and_attribute: DataDescriptor = DataDescriptor()
+        else:
+            union_of_metaclass_attributes: Literal[2] = 2
+            union_of_metaclass_data_descriptor_and_attribute: Literal[2] = 2
+
+    class C7(metaclass=Meta7):
+        if flag:
+            union_of_class_attributes: Literal[1] = 1
+            union_of_class_data_descriptor_and_attribute: DataDescriptor = DataDescriptor()
+        else:
+            union_of_class_attributes: Literal[2] = 2
+            union_of_class_data_descriptor_and_attribute: Literal[2] = 2
+
+    reveal_type(C7.union_of_metaclass_attributes)  # revealed: Literal[1, 2]
+    reveal_type(C7.union_of_metaclass_data_descriptor_and_attribute)  # revealed: Literal["data", 2]
+    reveal_type(C7.union_of_class_attributes)  # revealed: Literal[1, 2]
+    reveal_type(C7.union_of_class_data_descriptor_and_attribute)  # revealed: Literal["data", 2]
+```
+
+## Partial fall back
+
+Our implementation of the descriptor protocol takes into account that symbols can be possibly
+unbound. In those cases, we fall back to lower precedence steps of the descriptor protocol and union
+all possible results accordingly. We start by defining a data and a non-data descriptor:
+
+```py
+from typing import Literal
+
+class DataDescriptor:
+    def __get__(self, instance: object, owner: type | None = None) -> Literal["data"]:
+        return "data"
+
+    def __set__(self, instance: object, value: int) -> None:
+        pass
+
+class NonDataDescriptor:
+    def __get__(self, instance: object, owner: type | None = None) -> Literal["non-data"]:
+        return "non-data"
+```
+
+Then, we demonstrate that we fall back to an instance attribute if a data descriptor is possibly
+unbound:
+
+```py
+def f1(flag: bool):
+    class C1:
+        if flag:
+            attr = DataDescriptor()
+
+        def f(self):
+            self.attr = "normal"
+
+    reveal_type(C1().attr)  # revealed: Unknown | Literal["data", "normal"]
+```
+
+We never treat implicit instance attributes as definitely bound, so we fall back to the non-data
+descriptor here:
+
+```py
+def f2(flag: bool):
+    class C2:
+        def f(self):
+            self.attr = "normal"
+        attr = NonDataDescriptor()
+
+    reveal_type(C2().attr)  # revealed: Unknown | Literal["non-data", "normal"]
 ```
 
 ## Built-in `property` descriptor
@@ -100,18 +378,21 @@ c = C()
 
 reveal_type(c._name)  # revealed: str | None
 
-# Should be `str`
-reveal_type(c.name)  # revealed: @Todo(bound method)
+# TODO: Should be `str`
+reveal_type(c.name)  # revealed: <bound method `name` of `C`>
 
 # Should be `builtins.property`
 reveal_type(C.name)  # revealed: Literal[name]
 
-# This is fine:
+# TODO: These should not emit errors
+# error: [invalid-assignment]
 c.name = "new"
 
+# error: [invalid-assignment]
 c.name = None
 
-# TODO: this should be an error
+# TODO: this should be an error, but with a proper error message
+# error: [invalid-assignment] "Object of type `Literal[42]` is not assignable to attribute `name` of type `<bound method `name` of `C`>`"
 c.name = 42
 ```
 
@@ -135,14 +416,11 @@ class C:
 
 c1 = C.factory("test")  # okay
 
-# TODO: should be `C`
-reveal_type(c1)  # revealed: @Todo(return type)
+reveal_type(c1)  # revealed: C
 
-# TODO: should be `str`
-reveal_type(C.get_name())  # revealed: @Todo(return type)
+reveal_type(C.get_name())  # revealed: str
 
-# TODO: should be `str`
-reveal_type(C("42").get_name())  # revealed: @Todo(bound method)
+reveal_type(C("42").get_name())  # revealed: str
 ```
 
 ## Descriptors only work when used as class variables
@@ -160,9 +438,9 @@ class Ten:
 
 class C:
     def __init__(self):
-        self.ten = Ten()
+        self.ten: Ten = Ten()
 
-reveal_type(C().ten)  # revealed: Unknown | Ten
+reveal_type(C().ten)  # revealed: Ten
 ```
 
 ## Descriptors distinguishing between class and instance access
@@ -186,14 +464,279 @@ class Descriptor:
             return "called on class object"
 
 class C:
-    d = Descriptor()
+    d: Descriptor = Descriptor()
 
 # TODO: should be `Literal["called on class object"]
-reveal_type(C.d)  # revealed: Unknown | Descriptor
+reveal_type(C.d)  # revealed: LiteralString
 
 # TODO: should be `Literal["called on instance"]
-reveal_type(C().d)  # revealed: Unknown | Descriptor
+reveal_type(C().d)  # revealed: LiteralString
+```
+
+## Undeclared descriptor arguments
+
+If a descriptor attribute is not declared, we union with `Unknown`, just like for regular
+attributes, since that attribute could be overwritten externally. Even a data descriptor with a
+`__set__` method can be overwritten when accessed through a class object.
+
+```py
+class Descriptor:
+    def __get__(self, instance: object, owner: type | None = None) -> int:
+        return 1
+
+    def __set__(self, instance: object, value: int) -> None:
+        pass
+
+class C:
+    descriptor = Descriptor()
+
+C.descriptor = "something else"
+
+# This could also be `Literal["something else"]` if we support narrowing of attribute types based on assignments
+reveal_type(C.descriptor)  # revealed: Unknown | int
+```
+
+## `__get__` is called with correct arguments
+
+```py
+from __future__ import annotations
+
+class TailoredForClassObjectAccess:
+    def __get__(self, instance: None, owner: type[C]) -> int:
+        return 1
+
+class TailoredForInstanceAccess:
+    def __get__(self, instance: C, owner: type[C] | None = None) -> str:
+        return "a"
+
+class TailoredForMetaclassAccess:
+    def __get__(self, instance: type[C], owner: type[Meta]) -> bytes:
+        return b"a"
+
+class Meta(type):
+    metaclass_access: TailoredForMetaclassAccess = TailoredForMetaclassAccess()
+
+class C(metaclass=Meta):
+    class_object_access: TailoredForClassObjectAccess = TailoredForClassObjectAccess()
+    instance_access: TailoredForInstanceAccess = TailoredForInstanceAccess()
+
+reveal_type(C.class_object_access)  # revealed: int
+reveal_type(C().instance_access)  # revealed: str
+reveal_type(C.metaclass_access)  # revealed: bytes
+
+# TODO: These should emit a diagnostic
+reveal_type(C().class_object_access)  # revealed: TailoredForClassObjectAccess
+reveal_type(C.instance_access)  # revealed: TailoredForInstanceAccess
+```
+
+## Descriptors with incorrect `__get__` signature
+
+```py
+class Descriptor:
+    # `__get__` method with missing parameters:
+    def __get__(self) -> int:
+        return 1
+
+class C:
+    descriptor: Descriptor = Descriptor()
+
+# TODO: This should be an error
+reveal_type(C.descriptor)  # revealed: Descriptor
+
+# TODO: This should be an error
+reveal_type(C().descriptor)  # revealed: Descriptor
+```
+
+## Possibly unbound descriptor attributes
+
+```py
+class DataDescriptor:
+    def __get__(self, instance: object, owner: type | None = None) -> int:
+        return 1
+
+    def __set__(self, instance: int, value) -> None:
+        pass
+
+class NonDataDescriptor:
+    def __get__(self, instance: object, owner: type | None = None) -> int:
+        return 1
+
+def _(flag: bool):
+    class PossiblyUnbound:
+        if flag:
+            non_data: NonDataDescriptor = NonDataDescriptor()
+            data: DataDescriptor = DataDescriptor()
+
+    # error: [possibly-unbound-attribute] "Attribute `non_data` on type `Literal[PossiblyUnbound]` is possibly unbound"
+    reveal_type(PossiblyUnbound.non_data)  # revealed: int
+
+    # error: [possibly-unbound-attribute] "Attribute `non_data` on type `PossiblyUnbound` is possibly unbound"
+    reveal_type(PossiblyUnbound().non_data)  # revealed: int
+
+    # error: [possibly-unbound-attribute] "Attribute `data` on type `Literal[PossiblyUnbound]` is possibly unbound"
+    reveal_type(PossiblyUnbound.data)  # revealed: int
+
+    # error: [possibly-unbound-attribute] "Attribute `data` on type `PossiblyUnbound` is possibly unbound"
+    reveal_type(PossiblyUnbound().data)  # revealed: int
+```
+
+## Possibly-unbound `__get__` method
+
+```py
+def _(flag: bool):
+    class MaybeDescriptor:
+        if flag:
+            def __get__(self, instance: object, owner: type | None = None) -> int:
+                return 1
+
+    class C:
+        descriptor: MaybeDescriptor = MaybeDescriptor()
+
+    reveal_type(C.descriptor)  # revealed: int | MaybeDescriptor
+
+    reveal_type(C().descriptor)  # revealed: int | MaybeDescriptor
+```
+
+## Descriptors with non-function `__get__` callables that are descriptors themselves
+
+The descriptor protocol is recursive, i.e. looking up `__get__` can involve triggering the
+descriptor protocol on the callable's `__call__` method:
+
+```py
+from __future__ import annotations
+
+class ReturnedCallable2:
+    def __call__(self, descriptor: Descriptor1, instance: None, owner: type[C]) -> int:
+        return 1
+
+class ReturnedCallable1:
+    def __call__(self, descriptor: Descriptor2, instance: Callable1, owner: type[Callable1]) -> ReturnedCallable2:
+        return ReturnedCallable2()
+
+class Callable3:
+    def __call__(self, descriptor: Descriptor3, instance: Callable2, owner: type[Callable2]) -> ReturnedCallable1:
+        return ReturnedCallable1()
+
+class Descriptor3:
+    __get__: Callable3 = Callable3()
+
+class Callable2:
+    __call__: Descriptor3 = Descriptor3()
+
+class Descriptor2:
+    __get__: Callable2 = Callable2()
+
+class Callable1:
+    __call__: Descriptor2 = Descriptor2()
+
+class Descriptor1:
+    __get__: Callable1 = Callable1()
+
+class C:
+    d: Descriptor1 = Descriptor1()
+
+reveal_type(C.d)  # revealed: int
+```
+
+## Dunder methods
+
+Dunder methods are looked up on the meta-type, but we still need to invoke the descriptor protocol:
+
+```py
+class SomeCallable:
+    def __call__(self, x: int) -> str:
+        return "a"
+
+class Descriptor:
+    def __get__(self, instance: object, owner: type | None = None) -> SomeCallable:
+        return SomeCallable()
+
+class B:
+    __call__: Descriptor = Descriptor()
+
+b_instance = B()
+reveal_type(b_instance(1))  # revealed: str
+
+b_instance("bla")  # error: [invalid-argument-type]
+```
+
+## Functions as descriptors
+
+Functions are descriptors because they implement a `__get__` method. This is crucial in making sure
+that method calls work as expected. See [this test suite](./call/methods.md) for more information.
+Here, we only demonstrate how `__get__` works on functions:
+
+```py
+from inspect import getattr_static
+
+def f(x: object) -> str:
+    return "a"
+
+reveal_type(f)  # revealed: Literal[f]
+reveal_type(f.__get__)  # revealed: <method-wrapper `__get__` of `f`>
+reveal_type(f.__get__(None, type(f)))  # revealed: Literal[f]
+reveal_type(f.__get__(None, type(f))(1))  # revealed: str
+
+wrapper_descriptor = getattr_static(f, "__get__")
+
+reveal_type(wrapper_descriptor)  # revealed: <wrapper-descriptor `__get__` of `function` objects>
+reveal_type(wrapper_descriptor(f, None, type(f)))  # revealed: Literal[f]
+
+# Attribute access on the method-wrapper `f.__get__` falls back to `MethodWrapperType`:
+reveal_type(f.__get__.__hash__)  # revealed: <bound method `__hash__` of `MethodWrapperType`>
+
+# Attribute access on the wrapper-descriptor falls back to `WrapperDescriptorType`:
+reveal_type(wrapper_descriptor.__qualname__)  # revealed: @Todo(@property)
+```
+
+We can also bind the free function `f` to an instance of a class `C`:
+
+```py
+class C: ...
+
+bound_method = wrapper_descriptor(f, C(), C)
+
+reveal_type(bound_method)  # revealed: <bound method `f` of `C`>
+```
+
+We can then call it, and the instance of `C` is implicitly passed to the first parameter of `f`
+(`x`):
+
+```py
+reveal_type(bound_method())  # revealed: str
+```
+
+Finally, we test some error cases for the call to the wrapper descriptor:
+
+```py
+# Calling the wrapper descriptor without any arguments is an
+# error: [missing-argument] "No arguments provided for required parameters `self`, `instance`"
+wrapper_descriptor()
+
+# Calling it without the `instance` argument is an also an
+# error: [missing-argument] "No argument provided for required parameter `instance`"
+wrapper_descriptor(f)
+
+# Calling it without the `owner` argument if `instance` is not `None` is an
+# error: [missing-argument] "No argument provided for required parameter `owner`"
+wrapper_descriptor(f, None)
+
+# But calling it with an instance is fine (in this case, the `owner` argument is optional):
+wrapper_descriptor(f, C())
+
+# Calling it with something that is not a `FunctionType` as the first argument is an
+# error: [invalid-argument-type] "Object of type `Literal[1]` cannot be assigned to parameter 1 (`self`) of wrapper descriptor `FunctionType.__get__`; expected type `FunctionType`"
+wrapper_descriptor(1, None, type(f))
+
+# Calling it with something that is not a `type` as the `owner` argument is an
+# error: [invalid-argument-type] "Object of type `Literal[f]` cannot be assigned to parameter 3 (`owner`) of wrapper descriptor `FunctionType.__get__`; expected type `type`"
+wrapper_descriptor(f, None, f)
+
+# Calling it with too many positional arguments is an
+# error: [too-many-positional-arguments] "Too many positional arguments to wrapper descriptor `FunctionType.__get__`: expected 3, got 4"
+wrapper_descriptor(f, None, type(f), "one too many")
 ```
 
 [descriptors]: https://docs.python.org/3/howto/descriptor.html
+[precedence chain]: https://github.com/python/cpython/blob/3.13/Objects/typeobject.c#L5393-L5481
 [simple example]: https://docs.python.org/3/howto/descriptor.html#simple-example-a-descriptor-that-returns-a-constant

crates/red_knot_python_semantic/resources/mdtest/diagnostics/invalid_argument_type.md

@@ -0,0 +1,197 @@
+# Invalid argument type diagnostics
+
+<!-- snapshot-diagnostics -->
+
+## Basic
+
+This is a basic test demonstrating that a diagnostic points to the function definition corresponding
+to the invalid argument.
+
+```py
+def foo(x: int) -> int:
+    return x * x
+
+foo("hello")  # error: [invalid-argument-type]
+```
+
+## Different source order
+
+This is like the basic test, except we put the call site above the function definition.
+
+```py
+def bar():
+    foo("hello")  # error: [invalid-argument-type]
+
+def foo(x: int) -> int:
+    return x * x
+```
+
+## Different files
+
+This tests that a diagnostic can point to a function definition in a different file in which an
+invalid call site was found.
+
+`package.py`:
+
+```py
+def foo(x: int) -> int:
+    return x * x
+```
+
+```py
+import package
+
+package.foo("hello")  # error: [invalid-argument-type]
+```
+
+## Many parameters
+
+This checks that a diagnostic renders reasonably when there are multiple parameters.
+
+```py
+def foo(x: int, y: int, z: int) -> int:
+    return x * y * z
+
+foo(1, "hello", 3)  # error: [invalid-argument-type]
+```
+
+## Many parameters across multiple lines
+
+This checks that a diagnostic renders reasonably when there are multiple parameters spread out
+across multiple lines.
+
+```py
+def foo(
+    x: int,
+    y: int,
+    z: int,
+) -> int:
+    return x * y * z
+
+foo(1, "hello", 3)  # error: [invalid-argument-type]
+```
+
+## Many parameters with multiple invalid arguments
+
+This checks that a diagnostic renders reasonably when there are multiple parameters and multiple
+invalid argument types.
+
+```py
+def foo(x: int, y: int, z: int) -> int:
+    return x * y * z
+
+# error: [invalid-argument-type]
+# error: [invalid-argument-type]
+# error: [invalid-argument-type]
+foo("a", "b", "c")
+```
+
+At present (2025-02-18), this renders three different diagnostic messages. But arguably, these could
+all be folded into one diagnostic. Fixing this requires at least better support for multi-spans in
+the diagnostic model and possibly also how diagnostics are emitted by the type checker itself.
+
+## Test calling a function whose type is vendored from `typeshed`
+
+This tests that diagnostic rendering is reasonable when the function being called is from the
+standard library.
+
+```py
+import json
+
+json.loads(5)  # error: [invalid-argument-type]
+```
+
+## Tests for a variety of argument types
+
+These tests check that diagnostic output is reasonable regardless of the kinds of arguments used in
+a function definition.
+
+### Only positional
+
+Tests a function definition with only positional parameters.
+
+```py
+def foo(x: int, y: int, z: int, /) -> int:
+    return x * y * z
+
+foo(1, "hello", 3)  # error: [invalid-argument-type]
+```
+
+### Variadic arguments
+
+Tests a function definition with variadic arguments.
+
+```py
+def foo(*numbers: int) -> int:
+    return len(numbers)
+
+foo(1, 2, 3, "hello", 5)  # error: [invalid-argument-type]
+```
+
+### Keyword only arguments
+
+Tests a function definition with keyword-only arguments.
+
+```py
+def foo(x: int, y: int, *, z: int = 0) -> int:
+    return x * y * z
+
+foo(1, 2, z="hello")  # error: [invalid-argument-type]
+```
+
+### One keyword argument
+
+Tests a function definition with keyword-only arguments.
+
+```py
+def foo(x: int, y: int, z: int = 0) -> int:
+    return x * y * z
+
+foo(1, 2, "hello")  # error: [invalid-argument-type]
+```
+
+### Variadic keyword arguments
+
+```py
+def foo(**numbers: int) -> int:
+    return len(numbers)
+
+foo(a=1, b=2, c=3, d="hello", e=5)  # error: [invalid-argument-type]
+```
+
+### Mix of arguments
+
+Tests a function definition with multiple different kinds of arguments.
+
+```py
+def foo(x: int, /, y: int, *, z: int = 0) -> int:
+    return x * y * z
+
+foo(1, 2, z="hello")  # error: [invalid-argument-type]
+```
+
+### Synthetic arguments
+
+Tests a function call with synthetic arguments.
+
+```py
+class C:
+    def __call__(self, x: int) -> int:
+        return 1
+
+c = C()
+c("wrong")  # error: [invalid-argument-type]
+```
+
+## Calls to methods
+
+Tests that we also see a reference to a function if the callable is a bound method.
+
+```py
+class C:
+    def square(self, x: int) -> int:
+        return x * x
+
+c = C()
+c.square("hello")  # error: [invalid-argument-type]
+```

crates/red_knot_python_semantic/resources/mdtest/doc/README.md

@@ -0,0 +1,2 @@
+This directory contains user-facing documentation, but also doubles as an extended test suite that
+makes sure that our documentation stays up to date.

crates/red_knot_python_semantic/resources/mdtest/doc/public_type_undeclared_symbols.md

@@ -0,0 +1,125 @@
+# Public type of undeclared symbols
+
+## Summary
+
+One major deviation from the behavior of existing Python type checkers is our handling of 'public'
+types for undeclared symbols. This is best illustrated with an example:
+
+```py
+class Wrapper:
+    value = None
+
+wrapper = Wrapper()
+
+reveal_type(wrapper.value)  # revealed: Unknown | None
+
+wrapper.value = 1
+```
+
+Mypy and Pyright both infer a type of `None` for the type of `wrapper.value`. Consequently, both
+tools emit an error when trying to assign `1` to `wrapper.value`. But there is nothing wrong with
+this program. Emitting an error here violates the [gradual guarantee] which states that *"Removing
+type annotations (making the program more dynamic) should not result in additional static type
+errors."*: If `value` were annotated with `int | None` here, Mypy and Pyright would not emit any
+errors.
+
+By inferring `Unknown | None` instead, we allow arbitrary values to be assigned to `wrapper.value`.
+This is a deliberate choice to prevent false positive errors on untyped code.
+
+More generally, we infer `Unknown | T_inferred` for undeclared symbols, where `T_inferred` is the
+inferred type of the right-hand side of the assignment. This gradual type represents an *unknown*
+fully-static type that is *at least as large as* `T_inferred`. It accurately describes our static
+knowledge about this type. In the example above, we don't know what values `wrapper.value` could
+possibly contain, but we *do know* that `None` is a possibility. This allows us to catch errors
+where `wrapper.value` is used in a way that is incompatible with `None`:
+
+```py
+def accepts_int(i: int) -> None:
+    pass
+
+def f(w: Wrapper) -> None:
+    # This is fine
+    v: int | None = w.value
+
+    # This function call is incorrect, because `w.value` could be `None`. We therefore emit the following
+    # error: "`Unknown | None` cannot be assigned to parameter 1 (`i`) of function `accepts_int`; expected type `int`"
+    c = accepts_int(w.value)
+```
+
+## Explicit lack of knowledge
+
+The following example demonstrates how Mypy and Pyright's type inference of fully-static types in
+these situations can lead to false-negatives, even though everything appears to be (statically)
+typed. To make this a bit more realistic, imagine that `OptionalInt` is imported from an external,
+untyped module:
+
+`optional_int.py`:
+
+```py
+class OptionalInt:
+    value = 10
+
+def reset(o):
+    o.value = None
+```
+
+It is then used like this:
+
+```py
+from optional_int import OptionalInt, reset
+
+o = OptionalInt()
+reset(o)  # Oh no...
+
+# Mypy and Pyright infer a fully-static type of `int` here, which appears to make the
+# subsequent division operation safe -- but it is not. We infer the following type:
+reveal_type(o.value)  # revealed: Unknown | Literal[10]
+
+print(o.value // 2)  # Runtime error!
+```
+
+We do not catch this mistake either, but we accurately reflect our lack of knowledge about
+`o.value`. Together with a possible future type-checker mode that would detect the prevalence of
+dynamic types, this could help developers catch such mistakes.
+
+## Stricter behavior
+
+Users can always opt in to stricter behavior by adding type annotations. For the `OptionalInt`
+class, this would probably be:
+
+```py
+class OptionalInt:
+    value: int | None = 10
+
+o = OptionalInt()
+
+# The following public type is now
+# revealed: int | None
+reveal_type(o.value)
+
+# Incompatible assignments are now caught:
+# error: "Object of type `Literal["a"]` is not assignable to attribute `value` of type `int | None`"
+o.value = "a"
+```
+
+## What is meant by 'public' type?
+
+We apply different semantics depending on whether a symbol is accessed from the same scope in which
+it was originally defined, or whether it is accessed from an external scope. External scopes will
+see the symbol's "public type", which has been discussed above. But within the same scope the symbol
+was defined in, we use a narrower type of `T_inferred` for undeclared symbols. This is because, from
+the perspective of this scope, there is no way that the value of the symbol could have been
+reassigned from external scopes. For example:
+
+```py
+class Wrapper:
+    value = None
+
+    # Type as seen from the same scope:
+    reveal_type(value)  # revealed: None
+
+# Type as seen from another scope:
+reveal_type(Wrapper.value)  # revealed: Unknown | None
+```
+
+[gradual guarantee]: https://typing.readthedocs.io/en/latest/spec/concepts.html#the-gradual-guarantee

crates/red_knot_python_semantic/resources/mdtest/exception/control_flow.md

@@ -241,30 +241,34 @@ suites:
     `except` suite ran to completion
 
 ```py
-def could_raise_returns_str() -> str:
-    return "foo"
+class A: ...
+class B: ...
+class C: ...
 
-def could_raise_returns_bytes() -> bytes:
-    return b"foo"
+def could_raise_returns_A() -> A:
+    return A()
 
-def could_raise_returns_bool() -> bool:
-    return True
+def could_raise_returns_B() -> B:
+    return B()
+
+def could_raise_returns_C() -> C:
+    return C()
 
 x = 1
 
 try:
     reveal_type(x)  # revealed: Literal[1]
-    x = could_raise_returns_str()
-    reveal_type(x)  # revealed: str
+    x = could_raise_returns_A()
+    reveal_type(x)  # revealed: A
 except TypeError:
-    reveal_type(x)  # revealed: Literal[1] | str
-    x = could_raise_returns_bytes()
-    reveal_type(x)  # revealed: bytes
-    x = could_raise_returns_bool()
-    reveal_type(x)  # revealed: bool
+    reveal_type(x)  # revealed: Literal[1] | A
+    x = could_raise_returns_B()
+    reveal_type(x)  # revealed: B
+    x = could_raise_returns_C()
+    reveal_type(x)  # revealed: C
 finally:
-    # TODO: should be `Literal[1] | str | bytes | bool`
-    reveal_type(x)  # revealed: str | bool
+    # TODO: should be `Literal[1] | A | B | C`
+    reveal_type(x)  # revealed: A | C
     x = 2
     reveal_type(x)  # revealed: Literal[2]
 
@@ -282,53 +286,56 @@ x = 1
 
 try:
     reveal_type(x)  # revealed: Literal[1]
-    x = could_raise_returns_str()
-    reveal_type(x)  # revealed: str
+    x = could_raise_returns_A()
+    reveal_type(x)  # revealed: A
 except TypeError:
-    reveal_type(x)  # revealed: Literal[1] | str
-    x = could_raise_returns_bytes()
-    reveal_type(x)  # revealed: bytes
-    x = could_raise_returns_bool()
-    reveal_type(x)  # revealed: bool
+    reveal_type(x)  # revealed: Literal[1] | A
+    x = could_raise_returns_B()
+    reveal_type(x)  # revealed: B
+    x = could_raise_returns_C()
+    reveal_type(x)  # revealed: C
 finally:
-    # TODO: should be `Literal[1] | str | bytes | bool`
-    reveal_type(x)  # revealed: str | bool
+    # TODO: should be `Literal[1] | A | B | C`
+    reveal_type(x)  # revealed: A | C
 
-reveal_type(x)  # revealed: str | bool
+reveal_type(x)  # revealed: A | C
 ```
 
 An example with multiple `except` branches and a `finally` branch:
 
 ```py
-def could_raise_returns_memoryview() -> memoryview:
-    return memoryview(b"")
+class D: ...
+class E: ...
 
-def could_raise_returns_float() -> float:
-    return 3.14
+def could_raise_returns_D() -> D:
+    return D()
+
+def could_raise_returns_E() -> E:
+    return E()
 
 x = 1
 
 try:
     reveal_type(x)  # revealed: Literal[1]
-    x = could_raise_returns_str()
-    reveal_type(x)  # revealed: str
+    x = could_raise_returns_A()
+    reveal_type(x)  # revealed: A
 except TypeError:
-    reveal_type(x)  # revealed: Literal[1] | str
-    x = could_raise_returns_bytes()
-    reveal_type(x)  # revealed: bytes
-    x = could_raise_returns_bool()
-    reveal_type(x)  # revealed: bool
+    reveal_type(x)  # revealed: Literal[1] | A
+    x = could_raise_returns_B()
+    reveal_type(x)  # revealed: B
+    x = could_raise_returns_C()
+    reveal_type(x)  # revealed: C
 except ValueError:
-    reveal_type(x)  # revealed: Literal[1] | str
-    x = could_raise_returns_memoryview()
-    reveal_type(x)  # revealed: memoryview
-    x = could_raise_returns_float()
-    reveal_type(x)  # revealed: float
+    reveal_type(x)  # revealed: Literal[1] | A
+    x = could_raise_returns_D()
+    reveal_type(x)  # revealed: D
+    x = could_raise_returns_E()
+    reveal_type(x)  # revealed: E
 finally:
-    # TODO: should be `Literal[1] | str | bytes | bool | memoryview | float`
-    reveal_type(x)  # revealed: str | bool | float
+    # TODO: should be `Literal[1] | A | B | C | D | E`
+    reveal_type(x)  # revealed: A | C | E
 
-reveal_type(x)  # revealed: str | bool | float
+reveal_type(x)  # revealed: A | C | E
 ```
 
 ## Combining `except`, `else` and `finally` branches
@@ -338,84 +345,93 @@ control flow could have jumped to the `finally` suite from partway through the `
 an exception raised *there*.
 
 ```py
-def could_raise_returns_str() -> str:
-    return "foo"
+class A: ...
+class B: ...
+class C: ...
+class D: ...
+class E: ...
 
-def could_raise_returns_bytes() -> bytes:
-    return b"foo"
+def could_raise_returns_A() -> A:
+    return A()
 
-def could_raise_returns_bool() -> bool:
-    return True
+def could_raise_returns_B() -> B:
+    return B()
 
-def could_raise_returns_memoryview() -> memoryview:
-    return memoryview(b"")
+def could_raise_returns_C() -> C:
+    return C()
 
-def could_raise_returns_float() -> float:
-    return 3.14
+def could_raise_returns_D() -> D:
+    return D()
+
+def could_raise_returns_E() -> E:
+    return E()
 
 x = 1
 
 try:
     reveal_type(x)  # revealed: Literal[1]
-    x = could_raise_returns_str()
-    reveal_type(x)  # revealed: str
+    x = could_raise_returns_A()
+    reveal_type(x)  # revealed: A
 except TypeError:
-    reveal_type(x)  # revealed: Literal[1] | str
-    x = could_raise_returns_bytes()
-    reveal_type(x)  # revealed: bytes
-    x = could_raise_returns_bool()
-    reveal_type(x)  # revealed: bool
+    reveal_type(x)  # revealed: Literal[1] | A
+    x = could_raise_returns_B()
+    reveal_type(x)  # revealed: B
+    x = could_raise_returns_C()
+    reveal_type(x)  # revealed: C
 else:
-    reveal_type(x)  # revealed: str
-    x = could_raise_returns_memoryview()
-    reveal_type(x)  # revealed: memoryview
-    x = could_raise_returns_float()
-    reveal_type(x)  # revealed: float
+    reveal_type(x)  # revealed: A
+    x = could_raise_returns_D()
+    reveal_type(x)  # revealed: D
+    x = could_raise_returns_E()
+    reveal_type(x)  # revealed: E
 finally:
-    # TODO: should be `Literal[1] | str | bytes | bool | memoryview | float`
-    reveal_type(x)  # revealed: bool | float
+    # TODO: should be `Literal[1] | A | B | C | D | E`
+    reveal_type(x)  # revealed: C | E
 
-reveal_type(x)  # revealed: bool | float
+reveal_type(x)  # revealed: C | E
 ```
 
 The same again, this time with multiple `except` branches:
 
 ```py
-def could_raise_returns_range() -> range:
-    return range(42)
+class F: ...
+class G: ...
 
-def could_raise_returns_slice() -> slice:
-    return slice(None)
+def could_raise_returns_F() -> F:
+    return F()
+
+def could_raise_returns_G() -> G:
+    return G()
 
 x = 1
 
 try:
     reveal_type(x)  # revealed: Literal[1]
-    x = could_raise_returns_str()
-    reveal_type(x)  # revealed: str
+    x = could_raise_returns_A()
+    reveal_type(x)  # revealed: A
 except TypeError:
-    reveal_type(x)  # revealed: Literal[1] | str
-    x = could_raise_returns_bytes()
-    reveal_type(x)  # revealed: bytes
-    x = could_raise_returns_bool()
-    reveal_type(x)  # revealed: bool
+    reveal_type(x)  # revealed: Literal[1] | A
+    x = could_raise_returns_B()
+    reveal_type(x)  # revealed: B
+    x = could_raise_returns_C()
+    reveal_type(x)  # revealed: C
 except ValueError:
-    reveal_type(x)  # revealed: Literal[1] | str
-    x = could_raise_returns_memoryview()
-    reveal_type(x)  # revealed: memoryview
-    x = could_raise_returns_float()
-    reveal_type(x)  # revealed: float
+    reveal_type(x)  # revealed: Literal[1] | A
+    x = could_raise_returns_D()
+    reveal_type(x)  # revealed: D
+    x = could_raise_returns_E()
+    reveal_type(x)  # revealed: E
 else:
-    reveal_type(x)  # revealed: str
-    x = could_raise_returns_range()
-    reveal_type(x)  # revealed: range
-    x = could_raise_returns_slice()
-    reveal_type(x)  # revealed: slice
+    reveal_type(x)  # revealed: A
+    x = could_raise_returns_F()
+    reveal_type(x)  # revealed: F
+    x = could_raise_returns_G()
+    reveal_type(x)  # revealed: G
 finally:
-    # TODO: should be `Literal[1] | str | bytes | bool | memoryview | float | range | slice`
-    reveal_type(x)  # revealed: bool | float | slice
+    # TODO: should be `Literal[1] | A | B | C | D | E | F | G`
+    reveal_type(x)  # revealed: C | E | G
 
-reveal_type(x)  # revealed: bool | float | slice
+reveal_type(x)  # revealed: C | E | G
 ```
 
 ## Nested `try`/`except` blocks
@@ -429,92 +445,101 @@ a suite containing statements that could possibly raise exceptions, which would
 jumping out of that suite prior to the suite running to completion.
 
 ```py
-def could_raise_returns_str() -> str:
-    return "foo"
+class A: ...
+class B: ...
+class C: ...
+class D: ...
+class E: ...
+class F: ...
+class G: ...
+class H: ...
+class I: ...
+class J: ...
+class K: ...
 
-def could_raise_returns_bytes() -> bytes:
-    return b"foo"
+def could_raise_returns_A() -> A:
+    return A()
 
-def could_raise_returns_bool() -> bool:
-    return True
+def could_raise_returns_B() -> B:
+    return B()
 
-def could_raise_returns_memoryview() -> memoryview:
-    return memoryview(b"")
+def could_raise_returns_C() -> C:
+    return C()
 
-def could_raise_returns_float() -> float:
-    return 3.14
+def could_raise_returns_D() -> D:
+    return D()
 
-def could_raise_returns_range() -> range:
-    return range(42)
+def could_raise_returns_E() -> E:
+    return E()
 
-def could_raise_returns_slice() -> slice:
-    return slice(None)
+def could_raise_returns_F() -> F:
+    return F()
 
-def could_raise_returns_complex() -> complex:
-    return 3j
+def could_raise_returns_G() -> G:
+    return G()
 
-def could_raise_returns_bytearray() -> bytearray:
-    return bytearray()
+def could_raise_returns_H() -> H:
+    return H()
 
-class Foo: ...
-class Bar: ...
+def could_raise_returns_I() -> I:
+    return I()
 
-def could_raise_returns_Foo() -> Foo:
-    return Foo()
+def could_raise_returns_J() -> J:
+    return J()
 
-def could_raise_returns_Bar() -> Bar:
-    return Bar()
+def could_raise_returns_K() -> K:
+    return K()
 
 x = 1
 
 try:
     try:
         reveal_type(x)  # revealed: Literal[1]
-        x = could_raise_returns_str()
-        reveal_type(x)  # revealed: str
+        x = could_raise_returns_A()
+        reveal_type(x)  # revealed: A
     except TypeError:
-        reveal_type(x)  # revealed: Literal[1] | str
-        x = could_raise_returns_bytes()
-        reveal_type(x)  # revealed: bytes
-        x = could_raise_returns_bool()
-        reveal_type(x)  # revealed: bool
+        reveal_type(x)  # revealed: Literal[1] | A
+        x = could_raise_returns_B()
+        reveal_type(x)  # revealed: B
+        x = could_raise_returns_C()
+        reveal_type(x)  # revealed: C
     except ValueError:
-        reveal_type(x)  # revealed: Literal[1] | str
-        x = could_raise_returns_memoryview()
-        reveal_type(x)  # revealed: memoryview
-        x = could_raise_returns_float()
-        reveal_type(x)  # revealed: float
+        reveal_type(x)  # revealed: Literal[1] | A
+        x = could_raise_returns_D()
+        reveal_type(x)  # revealed: D
+        x = could_raise_returns_E()
+        reveal_type(x)  # revealed: E
     else:
-        reveal_type(x)  # revealed: str
-        x = could_raise_returns_range()
-        reveal_type(x)  # revealed: range
-        x = could_raise_returns_slice()
-        reveal_type(x)  # revealed: slice
+        reveal_type(x)  # revealed: A
+        x = could_raise_returns_F()
+        reveal_type(x)  # revealed: F
+        x = could_raise_returns_G()
+        reveal_type(x)  # revealed: G
     finally:
-        # TODO: should be `Literal[1] | str | bytes | bool | memoryview | float | range | slice`
-        reveal_type(x)  # revealed: bool | float | slice
+        # TODO: should be `Literal[1] | A | B | C | D | E | F | G`
+        reveal_type(x)  # revealed: C | E | G
         x = 2
         reveal_type(x)  # revealed: Literal[2]
     reveal_type(x)  # revealed: Literal[2]
 except:
-    reveal_type(x)  # revealed: Literal[1, 2] | str | bytes | bool | memoryview | float | range | slice
-    x = could_raise_returns_complex()
-    reveal_type(x)  # revealed: complex
-    x = could_raise_returns_bytearray()
-    reveal_type(x)  # revealed: bytearray
+    reveal_type(x)  # revealed: Literal[1, 2] | A | B | C | D | E | F | G
+    x = could_raise_returns_H()
+    reveal_type(x)  # revealed: H
+    x = could_raise_returns_I()
+    reveal_type(x)  # revealed: I
 else:
     reveal_type(x)  # revealed: Literal[2]
-    x = could_raise_returns_Foo()
-    reveal_type(x)  # revealed: Foo
-    x = could_raise_returns_Bar()
-    reveal_type(x)  # revealed: Bar
+    x = could_raise_returns_J()
+    reveal_type(x)  # revealed: J
+    x = could_raise_returns_K()
+    reveal_type(x)  # revealed: K
 finally:
-    # TODO: should be `Literal[1, 2] | str | bytes | bool | memoryview | float | range | slice | complex | bytearray | Foo | Bar`
-    reveal_type(x)  # revealed: bytearray | Bar
+    # TODO: should be `Literal[1, 2] | A | B | C | D | E | F | G | H | I | J | K`
+    reveal_type(x)  # revealed: I | K
 
 # Either one `except` branch or the `else`
 # must have been taken and completed to get here:
-reveal_type(x)  # revealed: bytearray | Bar
+reveal_type(x)  # revealed: I | K
 ```
 
 ## Nested scopes inside `try` blocks
@@ -523,50 +548,56 @@ Shadowing a variable in an inner scope has no effect on type inference of the va
 in the outer scope:
 
 ```py
-def could_raise_returns_str() -> str:
-    return "foo"
+class A: ...
+class B: ...
+class C: ...
+class D: ...
+class E: ...
 
-def could_raise_returns_bytes() -> bytes:
-    return b"foo"
+def could_raise_returns_A() -> A:
+    return A()
 
-def could_raise_returns_range() -> range:
-    return range(42)
+def could_raise_returns_B() -> B:
+    return B()
 
-def could_raise_returns_bytearray() -> bytearray:
-    return bytearray()
+def could_raise_returns_C() -> C:
+    return C()
 
-def could_raise_returns_float() -> float:
-    return 3.14
+def could_raise_returns_D() -> D:
+    return D()
+
+def could_raise_returns_E() -> E:
+    return E()
 
 x = 1
 
 try:
 
-    def foo(param=could_raise_returns_str()):
-        x = could_raise_returns_str()
+    def foo(param=could_raise_returns_A()):
+        x = could_raise_returns_A()
 
         try:
-            reveal_type(x)  # revealed: str
-            x = could_raise_returns_bytes()
-            reveal_type(x)  # revealed: bytes
+            reveal_type(x)  # revealed: A
+            x = could_raise_returns_B()
+            reveal_type(x)  # revealed: B
         except:
-            reveal_type(x)  # revealed: str | bytes
-            x = could_raise_returns_bytearray()
-            reveal_type(x)  # revealed: bytearray
-            x = could_raise_returns_float()
-            reveal_type(x)  # revealed: float
+            reveal_type(x)  # revealed: A | B
+            x = could_raise_returns_C()
+            reveal_type(x)  # revealed: C
+            x = could_raise_returns_D()
+            reveal_type(x)  # revealed: D
         finally:
-            # TODO: should be `str | bytes | bytearray | float`
-            reveal_type(x)  # revealed: bytes | float
-        reveal_type(x)  # revealed: bytes | float
+            # TODO: should be `A | B | C | D`
+            reveal_type(x)  # revealed: B | D
+        reveal_type(x)  # revealed: B | D
     x = foo
     reveal_type(x)  # revealed: Literal[foo]
 except:
     reveal_type(x)  # revealed: Literal[1] | Literal[foo]
 
     class Bar:
-        x = could_raise_returns_range()
-        reveal_type(x)  # revealed: range
+        x = could_raise_returns_E()
+        reveal_type(x)  # revealed: E
 
     x = Bar
     reveal_type(x)  # revealed: Literal[Bar]

crates/red_knot_python_semantic/resources/mdtest/expression/assert.md

@@ -0,0 +1,9 @@
+## Condition with object that implements `__bool__` incorrectly
+
+```py
+class NotBoolable:
+    __bool__ = 3
+
+# error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable"
+assert NotBoolable()
+```

crates/red_knot_python_semantic/resources/mdtest/expression/boolean.md

@@ -101,3 +101,55 @@ reveal_type(bool([]))  # revealed: bool
 reveal_type(bool({}))  # revealed: bool
 reveal_type(bool(set()))  # revealed: bool
 ```
+
+## `__bool__` returning `NoReturn`
+
+```py
+from typing import NoReturn
+
+class NotBoolable:
+    def __bool__(self) -> NoReturn:
+        raise NotImplementedError("This object can't be converted to a boolean")
+
+# TODO: This should emit an error that `NotBoolable` can't be converted to a bool but it currently doesn't
+#   because `Never` is assignable to `bool`. This probably requires dead code analysis to fix.
+if NotBoolable():
+    ...
+```
+
+## Not callable `__bool__`
+
+```py
+class NotBoolable:
+    __bool__ = None
+
+# error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `NotBoolable`; its `__bool__` method isn't callable"
+if NotBoolable():
+    ...
+```
+
+## Not-boolable union
+
+```py
+def test(cond: bool):
+    class NotBoolable:
+        __bool__ = None if cond else 3
+
+    # error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `NotBoolable`; it incorrectly implements `__bool__`"
+    if NotBoolable():
+        ...
+```
+
+## Union with some variants implementing `__bool__` incorrectly
+
+```py
+def test(cond: bool):
+    class NotBoolable:
+        __bool__ = None
+
+    a = 10 if cond else NotBoolable()
+
+    # error: [unsupported-bool-conversion] "Boolean conversion is unsupported for type `Literal[10] | NotBoolable`; its `__bool__` method isn't callable"
+    if a:
+        ...
+```

crates/red_knot_python_semantic/resources/mdtest/expression/lambda.md

@@ -0,0 +1,100 @@
+# `lambda` expression
+
+## No parameters
+
+`lambda` expressions can be defined without any parameters.
+
+```py
+reveal_type(lambda: 1)  # revealed: () -> @Todo(lambda return type)
+
+# error: [unresolved-reference]
+reveal_type(lambda: a)  # revealed: () -> @Todo(lambda return type)
+```
+
+## With parameters
+
+Unlike parameters in function definition, the parameters in a `lambda` expression cannot be
+annotated.
+
+```py
+reveal_type(lambda a: a)  # revealed: (a) -> @Todo(lambda return type)
+reveal_type(lambda a, b: a + b)  # revealed: (a, b) -> @Todo(lambda return type)
+```
+
+But, it can have default values:
+
+```py
+reveal_type(lambda a=1: a)  # revealed: (a=Literal[1]) -> @Todo(lambda return type)
+reveal_type(lambda a, b=2: a)  # revealed: (a, b=Literal[2]) -> @Todo(lambda return type)
+```
+
+And, positional-only parameters:
+
+```py
+reveal_type(lambda a, b, /, c: c)  # revealed: (a, b, /, c) -> @Todo(lambda return type)
+```
+
+And, keyword-only parameters:
+
+```py
+reveal_type(lambda a, *, b=2, c: b)  # revealed: (a, *, b=Literal[2], c) -> @Todo(lambda return type)
+```
+
+And, variadic parameter:
+
+```py
+reveal_type(lambda *args: args)  # revealed: (*args) -> @Todo(lambda return type)
+```
+
+And, keyword-varidic parameter:
+
+```py
+reveal_type(lambda **kwargs: kwargs)  # revealed: (**kwargs) -> @Todo(lambda return type)
+```
+
+Mixing all of them together:
+
+```py
+# revealed: (a, b, /, c=Literal[True], *args, *, d=Literal["default"], e=Literal[5], **kwargs) -> @Todo(lambda return type)
+reveal_type(lambda a, b, /, c=True, *args, d="default", e=5, **kwargs: None)
+```
+
+## Parameter type
+
+In addition to correctly inferring the `lambda` expression, the parameters should also be inferred
+correctly.
+
+Using a parameter with no default value:
+
+```py
+lambda x: reveal_type(x)  # revealed: Unknown
+```
+
+Using a parameter with default value:
+
+```py
+lambda x=1: reveal_type(x)  # revealed: Unknown | Literal[1]
+```
+
+Using a variadic paramter:
+
+```py
+# TODO: should be `tuple[Unknown, ...]` (needs generics)
+lambda *args: reveal_type(args)  # revealed: tuple
+```
+
+Using a keyword-varidic parameter:
+
+```py
+# TODO: should be `dict[str, Unknown]` (needs generics)
+lambda **kwargs: reveal_type(kwargs)  # revealed: dict
+```
+
+## Nested `lambda` expressions
+
+Here, a `lambda` expression is used as the default value for a parameter in another `lambda`
+expression.
+
+```py
+reveal_type(lambda a=lambda x, y: 0: 2)  # revealed: (a=(x, y) -> @Todo(lambda return type)) -> @Todo(lambda return type)
+```

crates/red_knot_python_semantic/resources/mdtest/generics.md

@@ -1,81 +0,0 @@
-# PEP 695 Generics
-
-## Class Declarations
-
-Basic PEP 695 generics
-
-```py
-class MyBox[T]:
-    data: T
-    box_model_number = 695
-
-    def __init__(self, data: T):
-        self.data = data
-
-box: MyBox[int] = MyBox(5)
-
-# TODO should emit a diagnostic here (str is not assignable to int)
-wrong_innards: MyBox[int] = MyBox("five")
-
-# TODO reveal int, do not leak the typevar
-reveal_type(box.data)  # revealed: T
-
-reveal_type(MyBox.box_model_number)  # revealed: Unknown | Literal[695]
-```
-
-## Subclassing
-
-```py
-class MyBox[T]:
-    data: T
-
-    def __init__(self, data: T):
-        self.data = data
-
-# TODO not error on the subscripting
-# error: [non-subscriptable]
-class MySecureBox[T](MyBox[T]): ...
-
-secure_box: MySecureBox[int] = MySecureBox(5)
-reveal_type(secure_box)  # revealed: MySecureBox
-# TODO reveal int
-# The @Todo(…) is misleading here. We currently treat `MyBox[T]` as a dynamic base class because we
-# don't understand generics and therefore infer `Unknown` for the `MyBox[T]` base of `MySecureBox[T]`.
-reveal_type(secure_box.data)  # revealed: @Todo(instance attribute on class with dynamic base)
-```
-
-## Cyclical class definition
-
-In type stubs, classes can reference themselves in their base class definitions. For example, in
-`typeshed`, we have `class str(Sequence[str]): ...`.
-
-This should hold true even with generics at play.
-
-```pyi
-class Seq[T]: ...
-
-# TODO not error on the subscripting
-class S[T](Seq[S]): ...  # error: [non-subscriptable]
-
-reveal_type(S)  # revealed: Literal[S]
-```
-
-## Type params
-
-A PEP695 type variable defines a value of type `typing.TypeVar`.
-
-```py
-def f[T]():
-    reveal_type(T)  # revealed: T
-    reveal_type(T.__name__)  # revealed: Literal["T"]
-```
-
-## Minimum two constraints
-
-A typevar with less than two constraints emits a diagnostic:
-
-```py
-# error: [invalid-type-variable-constraints] "TypeVar must have at least two constrained types"
-def f[T: (int,)]():
-    pass
-```

crates/red_knot_python_semantic/resources/mdtest/generics/classes.md

@@ -0,0 +1,187 @@
+# Generic classes
+
+## PEP 695 syntax
+
+TODO: Add a `red_knot_extension` function that asserts whether a function or class is generic.
+
+This is a generic class defined using PEP 695 syntax:
+
+```py
+class C[T]: ...
+```
+
+A class that inherits from a generic class, and fills its type parameters with typevars, is generic:
+
+```py
+# TODO: no error
+# error: [non-subscriptable]
+class D[U](C[U]): ...
+```
+
+A class that inherits from a generic class, but fills its type parameters with concrete types, is
+_not_ generic:
+
+```py
+# TODO: no error
+# error: [non-subscriptable]
+class E(C[int]): ...
+```
+
+A class that inherits from a generic class, and doesn't fill its type parameters at all, implicitly
+uses the default value for the typevar. In this case, that default type is `Unknown`, so `F`
+inherits from `C[Unknown]` and is not itself generic.
+
+```py
+class F(C): ...
+```
+
+## Legacy syntax
+
+This is a generic class defined using the legacy syntax:
+
+```py
+from typing import Generic, TypeVar
+
+T = TypeVar("T")
+
+# TODO: no error
+# error: [invalid-base]
+class C(Generic[T]): ...
+```
+
+A class that inherits from a generic class, and fills its type parameters with typevars, is generic.
+
+```py
+class D(C[T]): ...
+```
+
+(Examples `E` and `F` from above do not have analogues in the legacy syntax.)
+
+## Inferring generic class parameters
+
+The type parameter can be specified explicitly:
+
+```py
+class C[T]:
+    x: T
+
+# TODO: no error
+# TODO: revealed: C[int]
+# error: [non-subscriptable]
+reveal_type(C[int]())  # revealed: C
+```
+
+We can infer the type parameter from a type context:
+
+```py
+c: C[int] = C()
+# TODO: revealed: C[int]
+reveal_type(c)  # revealed: C
+```
+
+The typevars of a fully specialized generic class should no longer be visible:
+
+```py
+# TODO: revealed: int
+reveal_type(c.x)  # revealed: T
+```
+
+If the type parameter is not specified explicitly, and there are no constraints that let us infer a
+specific type, we infer the typevar's default type:
+
+```py
+class D[T = int]: ...
+
+# TODO: revealed: D[int]
+reveal_type(D())  # revealed: D
+```
+
+If a typevar does not provide a default, we use `Unknown`:
+
+```py
+# TODO: revealed: C[Unknown]
+reveal_type(C())  # revealed: C
+```
+
+If the type of a constructor parameter is a class typevar, we can use that to infer the type
+parameter:
+
+```py
+class E[T]:
+    def __init__(self, x: T) -> None: ...
+
+# TODO: revealed: E[int] or E[Literal[1]]
+reveal_type(E(1))  # revealed: E
+```
+
+The types inferred from a type context and from a constructor parameter must be consistent with each
+other:
+
+```py
+# TODO: error
+wrong_innards: E[int] = E("five")
+```
+
+## Generic subclass
+
+When a generic subclass fills its superclass's type parameter with one of its own, the actual types
+propagate through:
+
+```py
+class Base[T]:
+    x: T | None = None
+
+# TODO: no error
+# error: [non-subscriptable]
+class Sub[U](Base[U]): ...
+
+# TODO: no error
+# TODO: revealed: int | None
+# error: [non-subscriptable]
+reveal_type(Base[int].x)  # revealed: T | None
+# TODO: revealed: int | None
+# error: [non-subscriptable]
+reveal_type(Sub[int].x)  # revealed: T | None
+```
+
+## Cyclic class definition
+
+A class can use itself as the type parameter of one of its superclasses. (This is also known as the
+[curiously recurring template pattern][crtp] or [F-bounded quantification][f-bound].)
+
+Here, `Sub` is not a generic class, since it fills its superclass's type parameter (with itself).
+
+`stub.pyi`:
+
+```pyi
+class Base[T]: ...
+# TODO: no error
+# error: [non-subscriptable]
+class Sub(Base[Sub]): ...
+
+reveal_type(Sub)  # revealed: Literal[Sub]
+```
+
+`string_annotation.py`:
+
+```py
+class Base[T]: ...
+
+# TODO: no error
+# error: [non-subscriptable]
+class Sub(Base["Sub"]): ...
+
+reveal_type(Sub)  # revealed: Literal[Sub]
+```
+
+`bare_annotation.py`:
+
+```py
+class Base[T]: ...
+
+# TODO: error: [unresolved-reference]
+class Sub(Base[Sub]): ...
+```
+
+[crtp]: https://en.wikipedia.org/wiki/Curiously_recurring_template_pattern
+[f-bound]: https://en.wikipedia.org/wiki/Bounded_quantification#F-bounded_quantification

crates/red_knot_python_semantic/resources/mdtest/generics/functions.md

@@ -0,0 +1,244 @@
+# Generic functions
+
+## Typevar must be used at least twice
+
+If you're only using a typevar for a single parameter, you don't need the typevar — just use
+`object` (or the typevar's upper bound):
+
+```py
+# TODO: error, should be (x: object)
+def typevar_not_needed[T](x: T) -> None:
+    pass
+
+# TODO: error, should be (x: int)
+def bounded_typevar_not_needed[T: int](x: T) -> None:
+    pass
+```
+
+Typevars are only needed if you use them more than once. For instance, to specify that two
+parameters must both have the same type:
+
+```py
+def two_params[T](x: T, y: T) -> T:
+    return x
+```
+
+or to specify that a return value is the same as a parameter:
+
+```py
+def return_value[T](x: T) -> T:
+    return x
+```
+
+Each typevar must also appear _somewhere_ in the parameter list:
+
+```py
+def absurd[T]() -> T:
+    # There's no way to construct a T!
+    ...
+```
+
+## Inferring generic function parameter types
+
+If the type of a generic function parameter is a typevar, then we can infer what type that typevar
+is bound to at each call site.
+
+TODO: Note that some of the TODO revealed types have two options, since we haven't decided yet
+whether we want to infer a more specific `Literal` type where possible, or use heuristics to weaken
+the inferred type to e.g. `int`.
+
+```py
+def f[T](x: T) -> T: ...
+
+# TODO: no error
+# TODO: revealed: int or Literal[1]
+# error: [invalid-argument-type]
+reveal_type(f(1))  # revealed: T
+
+# TODO: no error
+# TODO: revealed: float
+# error: [invalid-argument-type]
+reveal_type(f(1.0))  # revealed: T
+
+# TODO: no error
+# TODO: revealed: bool or Literal[true]
+# error: [invalid-argument-type]
+reveal_type(f(True))  # revealed: T
+
+# TODO: no error
+# TODO: revealed: str or Literal["string"]
+# error: [invalid-argument-type]
+reveal_type(f("string"))  # revealed: T
+```
+
+## Inferring “deep” generic parameter types
+
+The matching up of call arguments and discovery of constraints on typevars can be a recursive
+process for arbitrarily-nested generic types in parameters.
+
+```py
+def f[T](x: list[T]) -> T: ...
+
+# TODO: revealed: float
+reveal_type(f([1.0, 2.0]))  # revealed: T
+```
+
+## Typevar constraints
+
+If a type parameter has an upper bound, that upper bound constrains which types can be used for that
+typevar. This effectively adds the upper bound as an intersection to every appearance of the typevar
+in the function.
+
+```py
+def good_param[T: int](x: T) -> None:
+    # TODO: revealed: T & int
+    reveal_type(x)  # revealed: T
+```
+
+If the function is annotated as returning the typevar, this means that the upper bound is _not_
+assignable to that typevar, since return types are contravariant. In `bad`, we can infer that
+`x + 1` has type `int`. But `T` might be instantiated with a narrower type than `int`, and so the
+return value is not guaranteed to be compatible for all `T: int`.
+
+```py
+def good_return[T: int](x: T) -> T:
+    return x
+
+def bad_return[T: int](x: T) -> T:
+    # TODO: error: int is not assignable to T
+    # error: [unsupported-operator] "Operator `+` is unsupported between objects of type `T` and `Literal[1]`"
+    return x + 1
+```
+
+## All occurrences of the same typevar have the same type
+
+If a typevar appears multiple times in a function signature, all occurrences have the same type.
+
+```py
+def different_types[T, S](cond: bool, t: T, s: S) -> T:
+    if cond:
+        return t
+    else:
+        # TODO: error: S is not assignable to T
+        return s
+
+def same_types[T](cond: bool, t1: T, t2: T) -> T:
+    if cond:
+        return t1
+    else:
+        return t2
+```
+
+## All occurrences of the same constrained typevar have the same type
+
+The above is true even when the typevars are constrained. Here, both `int` and `str` have `__add__`
+methods that are compatible with the return type, so the `return` expression is always well-typed:
+
+```py
+def same_constrained_types[T: (int, str)](t1: T, t2: T) -> T:
+    # TODO: no error
+    # error: [unsupported-operator] "Operator `+` is unsupported between objects of type `T` and `T`"
+    return t1 + t2
+```
+
+This is _not_ the same as a union type, because of this additional constraint that the two
+occurrences have the same type. In `unions_are_different`, `t1` and `t2` might have different types,
+and an `int` and a `str` cannot be added together:
+
+```py
+def unions_are_different(t1: int | str, t2: int | str) -> int | str:
+    # error: [unsupported-operator] "Operator `+` is unsupported between objects of type `int | str` and `int | str`"
+    return t1 + t2
+```
+
+## Typevar inference is a unification problem
+
+When inferring typevar assignments in a generic function call, we cannot simply solve constraints
+eagerly for each parameter in turn. We must solve a unification problem involving all of the
+parameters simultaneously.
+
+```py
+def two_params[T](x: T, y: T) -> T:
+    return x
+
+# TODO: no error
+# TODO: revealed: str
+# error: [invalid-argument-type]
+# error: [invalid-argument-type]
+reveal_type(two_params("a", "b"))  # revealed: T
+
+# TODO: no error
+# TODO: revealed: str | int
+# error: [invalid-argument-type]
+# error: [invalid-argument-type]
+reveal_type(two_params("a", 1))  # revealed: T
+```
+
+```py
+def param_with_union[T](x: T | int, y: T) -> T:
+    return y
+
+# TODO: no error
+# TODO: revealed: str
+# error: [invalid-argument-type]
+reveal_type(param_with_union(1, "a"))  # revealed: T
+
+# TODO: no error
+# TODO: revealed: str
+# error: [invalid-argument-type]
+# error: [invalid-argument-type]
+reveal_type(param_with_union("a", "a"))  # revealed: T
+
+# TODO: no error
+# TODO: revealed: int
+# error: [invalid-argument-type]
+reveal_type(param_with_union(1, 1))  # revealed: T
+
+# TODO: no error
+# TODO: revealed: str | int
+# error: [invalid-argument-type]
+# error: [invalid-argument-type]
+reveal_type(param_with_union("a", 1))  # revealed: T
+```
+
+```py
+def tuple_param[T, S](x: T | S, y: tuple[T, S]) -> tuple[T, S]:
+    return y
+
+# TODO: no error
+# TODO: revealed: tuple[str, int]
+# error: [invalid-argument-type]
+# error: [invalid-argument-type]
+reveal_type(tuple_param("a", ("a", 1)))  # revealed: tuple[T, S]
+
+# TODO: no error
+# TODO: revealed: tuple[str, int]
+# error: [invalid-argument-type]
+# error: [invalid-argument-type]
+reveal_type(tuple_param(1, ("a", 1)))  # revealed: tuple[T, S]
+```
+
+## Inferring nested generic function calls
+
+We can infer type assignments in nested calls to multiple generic functions. If they use the same
+type variable, we do not confuse the two; `T@f` and `T@g` have separate types in each example below.
+
+```py
+def f[T](x: T) -> tuple[T, int]:
+    return (x, 1)
+
+def g[T](x: T) -> T | None:
+    return x
+
+# TODO: no error
+# TODO: revealed: tuple[str | None, int]
+# error: [invalid-argument-type]
+# error: [invalid-argument-type]
+reveal_type(f(g("a")))  # revealed: tuple[T, int]
+
+# TODO: no error
+# TODO: revealed: tuple[str, int] | None
+# error: [invalid-argument-type]
+# error: [invalid-argument-type]
+reveal_type(g(f("a")))  # revealed: T | None
+```

crates/red_knot_python_semantic/resources/mdtest/generics/legacy.md

@@ -0,0 +1,72 @@
+# Legacy type variables
+
+The tests in this file focus on how type variables are defined using the legacy notation. Most
+_uses_ of type variables are tested in other files in this directory; we do not duplicate every test
+for both type variable syntaxes.
+
+Unless otherwise specified, all quotations come from the [Generics] section of the typing spec.
+
+## Type variables
+
+### Defining legacy type variables
+
+> Generics can be parameterized by using a factory available in `typing` called `TypeVar`.
+
+This was the only way to create type variables prior to PEP 695/Python 3.12. It is still available
+in newer Python releases.
+
+```py
+from typing import TypeVar
+
+T = TypeVar("T")
+```
+
+### Directly assigned to a variable
+
+> A `TypeVar()` expression must always directly be assigned to a variable (it should not be used as
+> part of a larger expression).
+
+```py
+from typing import TypeVar
+
+# TODO: error
+TestList = list[TypeVar("W")]
+```
+
+### `TypeVar` parameter must match variable name
+
+> The argument to `TypeVar()` must be a string equal to the variable name to which it is assigned.
+
+```py
+from typing import TypeVar
+
+# TODO: error
+T = TypeVar("Q")
+```
+
+### No redefinition
+
+> Type variables must not be redefined.
+
+```py
+from typing import TypeVar
+
+T = TypeVar("T")
+
+# TODO: error
+T = TypeVar("T")
+```
+
+### Cannot have only one constraint
+
+> `TypeVar` supports constraining parametric types to a fixed set of possible types...There should
+> be at least two constraints, if any; specifying a single constraint is disallowed.
+
+```py
+from typing import TypeVar
+
+# TODO: error: [invalid-type-variable-constraints]
+T = TypeVar("T", int)
+```
+
+[generics]: https://typing.readthedocs.io/en/latest/spec/generics.html

crates/red_knot_python_semantic/resources/mdtest/generics/pep695.md

@@ -0,0 +1,51 @@
+# PEP 695 Generics
+
+[PEP 695] and Python 3.12 introduced new, more ergonomic syntax for type variables.
+
+## Type variables
+
+### Defining PEP 695 type variables
+
+PEP 695 introduces a new syntax for defining type variables. The resulting type variables are
+instances of `typing.TypeVar`, just like legacy type variables.
+
+```py
+def f[T]():
+    reveal_type(type(T))  # revealed: Literal[TypeVar]
+    reveal_type(T)  # revealed: T
+    reveal_type(T.__name__)  # revealed: Literal["T"]
+```
+
+### Cannot have only one constraint
+
+> `TypeVar` supports constraining parametric types to a fixed set of possible types...There should
+> be at least two constraints, if any; specifying a single constraint is disallowed.
+
+```py
+# error: [invalid-type-variable-constraints] "TypeVar must have at least two constrained types"
+def f[T: (int,)]():
+    pass
+```
+
+## Invalid uses
+
+Note that many of the invalid uses of legacy typevars do not apply to PEP 695 typevars, since the
+PEP 695 syntax is only allowed places where typevars are allowed.
+
+## Displaying typevars
+
+We use a suffix when displaying the typevars of a generic function or class. This helps distinguish
+different uses of the same typevar.
+
+```py
+def f[T](x: T, y: T) -> None:
+    # TODO: revealed: T@f
+    reveal_type(x)  # revealed: T
+
+class C[T]:
+    def m(self, x: T) -> None:
+        # TODO: revealed: T@c
+        reveal_type(x)  # revealed: T
+```
+
+[pep 695]: https://peps.python.org/pep-0695/

crates/red_knot_python_semantic/resources/mdtest/generics/scoping.md

@@ -0,0 +1,257 @@
+# Scoping rules for type variables
+
+Most of these tests come from the [Scoping rules for type variables][scoping] section of the typing
+spec.
+
+## Typevar used outside of generic function or class
+
+Typevars may only be used in generic function or class definitions.
+
+```py
+from typing import TypeVar
+
+T = TypeVar("T")
+
+# TODO: error
+x: T
+
+class C:
+    # TODO: error
+    x: T
+
+def f() -> None:
+    # TODO: error
+    x: T
+```
+
+## Legacy typevar used multiple times
+
+> A type variable used in a generic function could be inferred to represent different types in the
+> same code block.
+
+This only applies to typevars defined using the legacy syntax, since the PEP 695 syntax creates a
+new distinct typevar for each occurrence.
+
+```py
+from typing import TypeVar
+
+T = TypeVar("T")
+
+def f1(x: T) -> T: ...
+def f2(x: T) -> T: ...
+
+f1(1)
+f2("a")
+```
+
+## Typevar inferred multiple times
+
+> A type variable used in a generic function could be inferred to represent different types in the
+> same code block.
+
+This also applies to a single generic function being used multiple times, instantiating the typevar
+to a different type each time.
+
+```py
+def f[T](x: T) -> T: ...
+
+# TODO: no error
+# TODO: revealed: int or Literal[1]
+# error: [invalid-argument-type]
+reveal_type(f(1))  # revealed: T
+# TODO: no error
+# TODO: revealed: str or Literal["a"]
+# error: [invalid-argument-type]
+reveal_type(f("a"))  # revealed: T
+```
+
+## Methods can mention class typevars
+
+> A type variable used in a method of a generic class that coincides with one of the variables that
+> parameterize this class is always bound to that variable.
+
+```py
+class C[T]:
+    def m1(self, x: T) -> T: ...
+    def m2(self, x: T) -> T: ...
+
+c: C[int] = C()
+# TODO: no error
+# error: [invalid-argument-type]
+c.m1(1)
+# TODO: no error
+# error: [invalid-argument-type]
+c.m2(1)
+# TODO: expected type `int`
+# error: [invalid-argument-type] "Object of type `Literal["string"]` cannot be assigned to parameter 2 (`x`) of bound method `m2`; expected type `T`"
+c.m2("string")
+```
+
+## Methods can mention other typevars
+
+> A type variable used in a method that does not match any of the variables that parameterize the
+> class makes this method a generic function in that variable.
+
+```py
+from typing import TypeVar, Generic
+
+T = TypeVar("T")
+S = TypeVar("S")
+
+# TODO: no error
+# error: [invalid-base]
+class Legacy(Generic[T]):
+    def m(self, x: T, y: S) -> S: ...
+
+legacy: Legacy[int] = Legacy()
+# TODO: revealed: str
+reveal_type(legacy.m(1, "string"))  # revealed: @Todo(Invalid or unsupported `Instance` in `Type::to_type_expression`)
+```
+
+With PEP 695 syntax, it is clearer that the method uses a separate typevar:
+
+```py
+class C[T]:
+    def m[S](self, x: T, y: S) -> S: ...
+
+c: C[int] = C()
+# TODO: no errors
+# TODO: revealed: str
+# error: [invalid-argument-type]
+# error: [invalid-argument-type]
+reveal_type(c.m(1, "string"))  # revealed: S
+```
+
+## Unbound typevars
+
+> Unbound type variables should not appear in the bodies of generic functions, or in the class
+> bodies apart from method definitions.
+
+This is true with the legacy syntax:
+
+```py
+from typing import TypeVar, Generic
+
+T = TypeVar("T")
+S = TypeVar("S")
+
+def f(x: T) -> None:
+    x: list[T] = []
+    # TODO: error
+    y: list[S] = []
+
+# TODO: no error
+# error: [invalid-base]
+class C(Generic[T]):
+    # TODO: error
+    x: list[S] = []
+
+    # This is not an error, as shown in the previous test
+    def m(self, x: S) -> S: ...
+```
+
+This is true with PEP 695 syntax, as well, though we must use the legacy syntax to define the
+unbound typevars:
+
+`pep695.py`:
+
+```py
+from typing import TypeVar
+
+S = TypeVar("S")
+
+def f[T](x: T) -> None:
+    x: list[T] = []
+    # TODO: error
+    y: list[S] = []
+
+class C[T]:
+    # TODO: error
+    x: list[S] = []
+
+    def m1(self, x: S) -> S: ...
+    def m2[S](self, x: S) -> S: ...
+```
+
+## Nested formal typevars must be distinct
+
+Generic functions and classes can be nested in each other, but it is an error for the same typevar
+to be used in nested generic definitions.
+
+Note that the typing spec only mentions two specific versions of this rule:
+
+> A generic class definition that appears inside a generic function should not use type variables
+> that parameterize the generic function.
+
+and
+
+> A generic class nested in another generic class cannot use the same type variables.
+
+We assume that the more general form holds.
+
+### Generic function within generic function
+
+```py
+def f[T](x: T, y: T) -> None:
+    def ok[S](a: S, b: S) -> None: ...
+
+    # TODO: error
+    def bad[T](a: T, b: T) -> None: ...
+```
+
+### Generic method within generic class
+
+```py
+class C[T]:
+    def ok[S](self, a: S, b: S) -> None: ...
+
+    # TODO: error
+    def bad[T](self, a: T, b: T) -> None: ...
+```
+
+### Generic class within generic function
+
+```py
+from typing import Iterable
+
+def f[T](x: T, y: T) -> None:
+    class Ok[S]: ...
+    # TODO: error for reuse of typevar
+    class Bad1[T]: ...
+    # TODO: no non-subscriptable error, error for reuse of typevar
+    # error: [non-subscriptable]
+    class Bad2(Iterable[T]): ...
+```
+
+### Generic class within generic class
+
+```py
+from typing import Iterable
+
+class C[T]:
+    class Ok1[S]: ...
+    # TODO: error for reuse of typevar
+    class Bad1[T]: ...
+    # TODO: no non-subscriptable error, error for reuse of typevar
+    # error: [non-subscriptable]
+    class Bad2(Iterable[T]): ...
+```
+
+## Class scopes do not cover inner scopes
+
+Just like regular symbols, the typevars of a generic class are only available in that class's scope,
+and are not available in nested scopes.
+
+```py
+class C[T]:
+    ok1: list[T] = []
+
+    class Bad:
+        # TODO: error
+        bad: list[T] = []
+
+    class Inner[S]: ...
+    ok2: Inner[T]
+```
+
+[scoping]: https://typing.readthedocs.io/en/latest/spec/generics.html#scoping-rules-for-type-variables

crates/red_knot_python_semantic/resources/mdtest/import/case_sensitive.md

@@ -0,0 +1,131 @@
+# Case Sensitive Imports
+
+```toml
+# TODO: This test should use the real file system instead of the memory file system.
+# but we can't change the file system yet because the tests would then start failing for 
+# case-insensitive file systems.
+#system = "os"
+```
+
+Python's import system is case-sensitive even on case-insensitive file system. This means, importing
+a module `a` should fail if the file in the search paths is named `A.py`. See
+[PEP 235](https://peps.python.org/pep-0235/).
+
+## Correct casing
+
+Importing a module where the name matches the file name's casing should succeed.
+
+`a.py`:
+
+```py
+class Foo:
+    x: int = 1
+```
+
+```python
+from a import Foo
+
+reveal_type(Foo().x)  # revealed: int
+```
+
+## Incorrect casing
+
+Importing a module where the name does not match the file name's casing should fail.
+
+`A.py`:
+
+```py
+class Foo:
+    x: int = 1
+```
+
+```python
+# error: [unresolved-import]
+from a import Foo
+```
+
+## Multiple search paths with different cased modules
+
+The resolved module is the first matching the file name's casing but Python falls back to later
+search paths if the file name's casing does not match.
+
+```toml
+[environment]
+extra-paths = ["/search-1", "/search-2"]
+```
+
+`/search-1/A.py`:
+
+```py
+class Foo:
+    x: int = 1
+```
+
+`/search-2/a.py`:
+
+```py
+class Bar:
+    x: str = "test"
+```
+
+```python
+from A import Foo
+from a import Bar
+
+reveal_type(Foo().x)  # revealed: int
+reveal_type(Bar().x)  # revealed: str
+```
+
+## Intermediate segments
+
+`db/__init__.py`:
+
+```py
+```
+
+`db/a.py`:
+
+```py
+class Foo:
+    x: int = 1
+```
+
+`correctly_cased.py`:
+
+```python
+from db.a import Foo
+
+reveal_type(Foo().x)  # revealed: int
+```
+
+Imports where some segments are incorrectly cased should fail.
+
+`incorrectly_cased.py`:
+
+```python
+# error: [unresolved-import]
+from DB.a import Foo
+
+# error: [unresolved-import]
+from DB.A import Foo
+
+# error: [unresolved-import]
+from db.A import Foo
+```
+
+## Incorrect extension casing
+
+The extension of imported python modules must be `.py` or `.pyi` but not `.PY` or `Py` or any
+variant where some characters are uppercase.
+
+`a.PY`:
+
+```py
+class Foo:
+    x: int = 1
+```
+
+```python
+# error: [unresolved-import]
+from a import Foo
+```