[ty] Lockfiles for mdtests with external dependencies (#22077)
## Summary Add lockfiles for all mdtests which make use of external dependencies. When running tests normally, we use this lockfile when creating the temporary venv using `uv sync --locked`. A new `MDTEST_UPGRADE_LOCKFILES` environment variable is used to switch to a mode in which those lockfiles can be updated or regenerated. When using the Python mdtest runner, this environment variable is automatically set (because we use this command while developing, not to simulate exactly what happens in CI). A command-line flag is provided to opt out of this. ## Test Plan ### Using the mdtest runner #### Adding a new test (no lockfile yet) * Removed `attrs.lock` to simulate this * Ran `uv run crates/ty_python_semantic/mdtest.py -e external/`. The lockfile is generated and the test succeeds. #### Upgrading/downgrading a dependency * Changed pydantic requirement from `pydantic==2.12.2` to `pydantic==2.12.5` (also tested with `2.12.0`) * Ran `uv run crates/ty_python_semantic/mdtest.py -e external/`. The lockfile is updated and the test succeeds. ### Using cargo #### Adding a new test (no lockfile yet) * Removed `attrs.lock` to simulate this * Ran `MDTEST_EXTERNAL=1 cargo test -p ty_python_semantic --test mdtest mdtest__external` "naively", which outputs: > Failed to setup in-memory virtual environment with dependencies: Lockfile not found at '/home/shark/ruff/crates/ty_python_semantic/resources/mdtest/external/attrs.lock'. Run with `MDTEST_UPGRADE_LOCKFILES=1` to generate it. * Ran `MDTEST_UPGRADE_LOCKFILES=1 MDTEST_EXTERNAL=1 cargo test -p ty_python_semantic --test mdtest mdtest__external`. The lockfile is updated and the test succeeds. #### Upgrading/downgrading a dependency * Changed pydantic requirement from `pydantic==2.12.2` to `pydantic==2.12.5` (also tested with `2.12.0`) * Ran `MDTEST_EXTERNAL=1 cargo test -p ty_python_semantic --test mdtest mdtest__external` "naively", which outputs a similar error message as above. * Ran the command suggested in the error message (`MDTEST_EXTERNAL=1 MDTEST_UPGRADE_LOCKFILES=1 cargo test -p ty_python_semantic --test mdtest mdtest__external`). The lockfile is updated and the test succeeds.
This commit is contained in:
@@ -38,8 +38,14 @@ class MDTestRunner:
|
||||
console: Console
|
||||
filters: list[str]
|
||||
enable_external: bool
|
||||
upgrade_lockfiles: bool
|
||||
|
||||
def __init__(self, filters: list[str] | None, enable_external: bool) -> None:
|
||||
def __init__(
|
||||
self,
|
||||
filters: list[str] | None,
|
||||
enable_external: bool,
|
||||
upgrade_lockfiles: bool,
|
||||
) -> None:
|
||||
self.mdtest_executable = None
|
||||
self.console = Console()
|
||||
self.filters = [
|
||||
@@ -47,6 +53,7 @@ class MDTestRunner:
|
||||
for f in (filters or [])
|
||||
]
|
||||
self.enable_external = enable_external
|
||||
self.upgrade_lockfiles = upgrade_lockfiles
|
||||
|
||||
def _run_cargo_test(self, *, message_format: Literal["human", "json"]) -> str:
|
||||
return subprocess.check_output(
|
||||
@@ -123,6 +130,7 @@ class MDTestRunner:
|
||||
INSTA_FORCE_PASS="1",
|
||||
INSTA_OUTPUT="none",
|
||||
MDTEST_EXTERNAL="1" if self.enable_external else "0",
|
||||
MDTEST_UPGRADE_LOCKFILES="1" if self.upgrade_lockfiles else "0",
|
||||
),
|
||||
capture_output=capture_output,
|
||||
text=True,
|
||||
@@ -260,12 +268,20 @@ def main() -> None:
|
||||
action="store_true",
|
||||
help="Enable tests with external dependencies",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--no-lockfile-upgrades",
|
||||
action="store_true",
|
||||
help="By default, lockfiles will be upgraded when dependency requirements in the Markdown test change."
|
||||
+ " Set this flag to never upgrade any lockfiles.",
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
try:
|
||||
runner = MDTestRunner(
|
||||
filters=args.filters, enable_external=args.enable_external
|
||||
filters=args.filters,
|
||||
enable_external=args.enable_external,
|
||||
upgrade_lockfiles=not args.no_lockfile_upgrades,
|
||||
)
|
||||
runner.watch()
|
||||
except KeyboardInterrupt:
|
||||
|
||||
Reference in New Issue
Block a user