[ty] mdtests with external dependencies (#20904)

## Summary

This PR adds the possibility to write mdtests that specify external
dependencies in a `project` section of TOML blocks. For example, here is
a test that makes sure that we understand Pydantic's dataclass-transform
setup:

````markdown
```toml
[environment]
python-version = "3.12"
python-platform = "linux"

[project]
dependencies = ["pydantic==2.12.2"]
```

```py
from pydantic import BaseModel

class User(BaseModel):
    id: int
    name: str

user = User(id=1, name="Alice")
reveal_type(user.id)  # revealed: int
reveal_type(user.name)  # revealed: str

# error: [missing-argument] "No argument provided for required parameter
`name`"
invalid_user = User(id=2)
```
````

## How?

Using the `python-version` and the `dependencies` fields from the
Markdown section, we generate a `pyproject.toml` file, write it to a
temporary directory, and use `uv sync` to install the dependencies into
a virtual environment. We then copy the Python source files from that
venv's `site-packages` folder to a corresponding directory structure in
the in-memory filesystem. Finally, we configure the search paths
accordingly, and run the mdtest as usual.

I fully understand that there are valid concerns here:
* Doesn't this require network access? (yes, it does)
* Is this fast enough? (`uv` caching makes this almost unnoticeable,
actually)
* Is this deterministic? ~~(probably not, package resolution can depend
on the platform you're on)~~ (yes, hopefully)

For this reason, this first version is opt-in, locally. ~~We don't even
run these tests in CI (even though they worked fine in a previous
iteration of this PR).~~ You need to set `MDTEST_EXTERNAL=1`, or use the
new `-e/--enable-external` command line option of the `mdtest.py`
runner. For example:
```bash
# Skip mdtests with external dependencies (default):
uv run crates/ty_python_semantic/mdtest.py

# Run all mdtests, including those with external dependencies:
uv run crates/ty_python_semantic/mdtest.py -e

# Only run the `pydantic` tests. Use `-e` to make sure it is not skipped:
uv run crates/ty_python_semantic/mdtest.py -e pydantic
```

## Why?

I believe that this can be a useful addition to our testing strategy,
which lies somewhere between ecosystem tests and normal mdtests.
Ecosystem tests cover much more code, but they have the disadvantage
that we only see second- or third-order effects via diagnostic diffs. If
we unexpectedly gain or lose type coverage somewhere, we might not even
notice (assuming the gradual guarantee holds, and ecosystem code is
mostly correct). Another disadvantage of ecosystem checks is that they
only test checked-in code that is usually correct. However, we also want
to test what happens on wrong code, like the code that is momentarily
written in an editor, before fixing it. On the other end of the spectrum
we have normal mdtests, which have the disadvantage that they do not
reflect the reality of complex real-world code. We experience this
whenever we're surprised by an ecosystem report on a PR.

That said, these tests should not be seen as a replacement for either of
these things. For example, we should still strive to write detailed
self-contained mdtests for user-reported issues. But we might use this
new layer for regression tests, or simply as a debugging tool. It can
also serve as a tool to document our support for popular third-party
libraries.

## Test Plan

* I've been locally using this for a couple of weeks now.
* `uv run crates/ty_python_semantic/mdtest.py -e`

crates/ty_python_semantic/resources/mdtest/external/README.md

@@ -0,0 +1,4 @@
+# mdtests with external dependencies
+
+This directory contains mdtests that make use of external packages. See the mdtest `README.md` for
+more information.

crates/ty_python_semantic/resources/mdtest/external/attrs.md

@@ -0,0 +1,78 @@
+# attrs
+
+```toml
+[environment]
+python-version = "3.13"
+python-platform = "linux"
+
+[project]
+dependencies = ["attrs==25.4.0"]
+```
+
+## Basic class (`attr`)
+
+```py
+import attr
+
+@attr.s
+class User:
+    id: int = attr.ib()
+    name: str = attr.ib()
+
+user = User(id=1, name="John Doe")
+
+reveal_type(user.id)  # revealed: int
+reveal_type(user.name)  # revealed: str
+```
+
+## Basic class (`define`)
+
+```py
+from attrs import define, field
+
+@define
+class User:
+    id: int = field()
+    internal_name: str = field(alias="name")
+
+user = User(id=1, name="John Doe")
+reveal_type(user.id)  # revealed: int
+reveal_type(user.internal_name)  # revealed: str
+```
+
+## Usage of `field` parameters
+
+```py
+from attrs import define, field
+
+@define
+class Product:
+    id: int = field(init=False)
+    name: str = field()
+    price_cent: int = field(kw_only=True)
+
+reveal_type(Product.__init__)  # revealed: (self: Product, name: str, *, price_cent: int) -> None
+```
+
+## Dedicated support for the `default` decorator?
+
+We currently do not support this:
+
+```py
+from attrs import define, field
+
+@define
+class Person:
+    id: int = field()
+    name: str = field()
+
+    # error: [call-non-callable] "Object of type `_MISSING_TYPE` is not callable"
+    @id.default
+    def _default_id(self) -> int:
+        raise NotImplementedError
+
+# error: [missing-argument] "No argument provided for required parameter `id`"
+person = Person(name="Alice")
+reveal_type(person.id)  # revealed: int
+reveal_type(person.name)  # revealed: str
+```

crates/ty_python_semantic/resources/mdtest/external/numpy.md

@@ -0,0 +1,23 @@
+# numpy
+
+```toml
+[environment]
+python-version = "3.13"
+python-platform = "linux"
+
+[project]
+dependencies = ["numpy==2.3.0"]
+```
+
+## Basic usage
+
+```py
+import numpy as np
+
+xs = np.array([1, 2, 3])
+reveal_type(xs)  # revealed: ndarray[tuple[Any, ...], dtype[Any]]
+
+xs = np.array([1.0, 2.0, 3.0], dtype=np.float64)
+# TODO: should be `ndarray[tuple[Any, ...], dtype[float64]]`
+reveal_type(xs)  # revealed: ndarray[tuple[Any, ...], dtype[Unknown]]
+```

crates/ty_python_semantic/resources/mdtest/external/pydantic.md

@@ -0,0 +1,48 @@
+# Pydantic
+
+```toml
+[environment]
+python-version = "3.12"
+python-platform = "linux"
+
+[project]
+dependencies = ["pydantic==2.12.2"]
+```
+
+## Basic model
+
+```py
+from pydantic import BaseModel
+
+class User(BaseModel):
+    id: int
+    name: str
+
+reveal_type(User.__init__)  # revealed: (self: User, *, id: int, name: str) -> None
+
+user = User(id=1, name="John Doe")
+reveal_type(user.id)  # revealed: int
+reveal_type(user.name)  # revealed: str
+
+# error: [missing-argument] "No argument provided for required parameter `name`"
+invalid_user = User(id=2)
+```
+
+## Usage of `Field`
+
+```py
+from pydantic import BaseModel, Field
+
+class Product(BaseModel):
+    id: int = Field(init=False)
+    name: str = Field(..., kw_only=False, min_length=1)
+    internal_price_cent: int = Field(..., gt=0, alias="price_cent")
+
+reveal_type(Product.__init__)  # revealed: (self: Product, name: str = Any, *, price_cent: int = Any) -> None
+
+product = Product("Laptop", price_cent=999_00)
+
+reveal_type(product.id)  # revealed: int
+reveal_type(product.name)  # revealed: str
+reveal_type(product.internal_price_cent)  # revealed: int
+```

crates/ty_python_semantic/resources/mdtest/external/pytest.md

@@ -0,0 +1,27 @@
+# pytest
+
+```toml
+[environment]
+python-version = "3.13"
+python-platform = "linux"
+
+[project]
+dependencies = ["pytest==9.0.1"]
+```
+
+## `pytest.fail`
+
+Make sure that we recognize `pytest.fail` calls as terminal:
+
+```py
+import pytest
+
+def some_runtime_condition() -> bool:
+    return True
+
+def test_something():
+    if not some_runtime_condition():
+        pytest.fail("Runtime condition failed")
+
+        no_error_here_this_is_unreachable
+```

crates/ty_python_semantic/resources/mdtest/external/sqlalchemy.md

@@ -0,0 +1,124 @@
+# SQLAlchemy
+
+```toml
+[environment]
+python-version = "3.13"
+python-platform = "linux"
+
+[project]
+dependencies = ["SQLAlchemy==2.0.44"]
+```
+
+## Basic model
+
+Here, we mostly make sure that ty understands SQLAlchemy's dataclass-transformer setup:
+
+```py
+from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column
+
+class Base(DeclarativeBase):
+    pass
+
+class User(Base):
+    __tablename__ = "user"
+
+    id: Mapped[int] = mapped_column(primary_key=True, init=False)
+    internal_name: Mapped[str] = mapped_column(alias="name")
+
+user = User(name="John Doe")
+reveal_type(user.id)  # revealed: int
+reveal_type(user.internal_name)  # revealed: str
+```
+
+Unfortunately, SQLAlchemy overrides `__init__` and explicitly accepts all combinations of keyword
+arguments. This is why we currently cannot flag invalid constructor calls:
+
+```py
+reveal_type(User.__init__)  # revealed: def __init__(self, **kw: Any) -> Unknown
+
+# TODO: this should ideally be an error
+invalid_user = User(invalid_arg=42)
+```
+
+## Queries
+
+First, the basic setup:
+
+```py
+from datetime import datetime
+
+from sqlalchemy import select, Integer, Text, Boolean, DateTime
+from sqlalchemy.orm import Session
+from sqlalchemy.orm import DeclarativeBase
+from sqlalchemy.orm import Mapped, mapped_column
+from sqlalchemy import create_engine
+
+engine = create_engine("sqlite://example.db")
+session = Session(engine)
+```
+
+Now we can declare a simple model:
+
+```py
+class Base(DeclarativeBase):
+    pass
+
+class User(Base):
+    __tablename__ = "users"
+
+    id: Mapped[int] = mapped_column(Integer, primary_key=True)
+    name: Mapped[str] = mapped_column(Text)
+    is_admin: Mapped[bool] = mapped_column(Boolean, default=False)
+```
+
+And perform simple queries:
+
+```py
+stmt = select(User)
+reveal_type(stmt)  # revealed: Select[tuple[User]]
+
+users = session.scalars(stmt).all()
+reveal_type(users)  # revealed: Sequence[User]
+
+for row in session.execute(stmt):
+    reveal_type(row)  # revealed: Row[tuple[User]]
+
+stmt = select(User).where(User.name == "Alice")
+alice = session.scalars(stmt).first()
+reveal_type(alice)  # revealed: User | None
+
+stmt = select(User).where(User.is_admin == True).order_by(User.name).limit(10)
+admin_users = session.scalars(stmt).all()
+reveal_type(admin_users)  # revealed: Sequence[User]
+```
+
+This also works with the legacy `query` API:
+
+```py
+users_legacy = session.query(User).all()
+reveal_type(users_legacy)  # revealed: list[User]
+```
+
+We can also specify particular columns to select:
+
+```py
+stmt = select(User.id, User.name)
+# TODO: should be `Select[tuple[int, str]]`
+reveal_type(stmt)  # revealed: Select[tuple[Unknown, Unknown]]
+
+for row in session.execute(stmt):
+    # TODO: should be `Row[Tuple[int, str]]`
+    reveal_type(row)  # revealed: Row[tuple[Unknown, Unknown]]
+```
+
+And similarly with the legacy `query` API:
+
+```py
+query = session.query(User.id, User.name)
+# TODO: should be `RowReturningQuery[tuple[int, str]]`
+reveal_type(query)  # revealed: RowReturningQuery[tuple[Unknown, Unknown]]
+
+for row in query.all():
+    # TODO: should be `Row[Tuple[int, str]]`
+    reveal_type(row)  # revealed: Row[tuple[Unknown, Unknown]]
+```

crates/ty_python_semantic/resources/mdtest/external/sqlmodel.md

@@ -0,0 +1,30 @@
+# SQLModel
+
+```toml
+[environment]
+python-version = "3.13"
+python-platform = "linux"
+
+[project]
+dependencies = ["sqlmodel==0.0.27"]
+```
+
+## Basic model
+
+```py
+from sqlmodel import SQLModel
+
+class User(SQLModel):
+    id: int
+    name: str
+
+user = User(id=1, name="John Doe")
+reveal_type(user.id)  # revealed: int
+reveal_type(user.name)  # revealed: str
+
+# TODO: this should not mention `__pydantic_self__`, and have proper parameters defined by the fields
+reveal_type(User.__init__)  # revealed: def __init__(__pydantic_self__, **data: Any) -> None
+
+# TODO: this should be an error
+User()
+```

crates/ty_python_semantic/resources/mdtest/external/strawberry.md

@@ -0,0 +1,27 @@
+# Strawberry GraphQL
+
+```toml
+[environment]
+python-version = "3.13"
+python-platform = "linux"
+
+[project]
+dependencies = ["strawberry-graphql==0.283.3"]
+```
+
+## Basic model
+
+```py
+import strawberry
+
+@strawberry.type
+class User:
+    id: int
+    role: str = strawberry.field(default="user")
+
+reveal_type(User.__init__)  # revealed: (self: User, *, id: int, role: str = Any) -> None
+
+user = User(id=1)
+reveal_type(user.id)  # revealed: int
+reveal_type(user.role)  # revealed: str
+```