Cyclic0007/ruff
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[ty] Splat variadic arguments into parameter list (#18996)

Browse Source 
This PR updates our call binding logic to handle splatted arguments.

Complicating matters is that we have separated call bind analysis into
two phases: parameter matching and type checking. Parameter matching
looks at the arity of the function signature and call site, and assigns
arguments to parameters. Importantly, we don't yet know the type of each
argument! This is needed so that we can decide whether to infer the type
of each argument as a type form or value form, depending on the
requirements of the parameter that the argument was matched to.

This is an issue when splatting an argument, since we need to know how
many elements the splatted argument contains to know how many positional
parameters to match it against. And to know how many elements the
splatted argument has, we need to know its type.

To get around this, we now make the assumption that splatted arguments
can only be used with value-form parameters. (If you end up splatting an
argument into a type-form parameter, we will silently pass in its
value-form type instead.) That allows us to preemptively infer the
(value-form) type of any splatted argument, so that we have its arity
available during parameter matching. We defer inference of non-splatted
arguments until after parameter matching has finished, as before.

We reuse a lot of the new tuple machinery to make this happen — in
particular resizing the tuple spec representing the number of arguments
passed in with the tuple length representing the number of parameters
the splat was matched with.

This work also shows that we might need to change how we are performing
argument expansion during overload resolution. At the moment, when we
expand parameters, we assume that each argument will still be matched to
the same parameters as before, and only retry the type-checking phase.
With splatted arguments, this is no longer the case, since the inferred
arity of each union element might be different than the arity of the
union as a whole, which can affect how many parameters the splatted
argument is matched to. See the regression test case in
`mdtest/call/function.md` for more details.
This commit is contained in:
Douglas Creager
2025-07-22 14:33:08 -04:00
committed by GitHub
parent 9d5ecacdc5
commit 7673d46b71
10 changed files with 694 additions and 130 deletions
Download Patch File Download Diff File Expand all files Collapse all files

240
crates/ty_python_semantic/resources/mdtest/call/function.md
View File

@@ -69,6 +69,246 @@ def _(flag: bool):
reveal_type(foo()) # revealed: int
```
## Splatted arguments
### Unknown argument length
```py
def takes_zero() -> None: ...
def takes_one(x: int) -> None: ...
def takes_two(x: int, y: int) -> None: ...
def takes_two_positional_only(x: int, y: int, /) -> None: ...
def takes_two_different(x: int, y: str) -> None: ...
def takes_two_different_positional_only(x: int, y: str, /) -> None: ...
def takes_at_least_zero(*args) -> None: ...
def takes_at_least_one(x: int, *args) -> None: ...
def takes_at_least_two(x: int, y: int, *args) -> None: ...
def takes_at_least_two_positional_only(x: int, y: int, /, *args) -> None: ...
# Test all of the above with a number of different splatted argument types
def _(args: list[int]) -> None:
takes_zero(*args)
takes_one(*args)
takes_two(*args)
takes_two_positional_only(*args)
takes_two_different(*args) # error: [invalid-argument-type]
takes_two_different_positional_only(*args) # error: [invalid-argument-type]
takes_at_least_zero(*args)
takes_at_least_one(*args)
takes_at_least_two(*args)
takes_at_least_two_positional_only(*args)
def _(args: tuple[int, ...]) -> None:
takes_zero(*args)
takes_one(*args)
takes_two(*args)
takes_two_positional_only(*args)
takes_two_different(*args) # error: [invalid-argument-type]
takes_two_different_positional_only(*args) # error: [invalid-argument-type]
takes_at_least_zero(*args)
takes_at_least_one(*args)
takes_at_least_two(*args)
takes_at_least_two_positional_only(*args)
```
### Fixed-length tuple argument
```py
def takes_zero() -> None: ...
def takes_one(x: int) -> None: ...
def takes_two(x: int, y: int) -> None: ...
def takes_two_positional_only(x: int, y: int, /) -> None: ...
def takes_two_different(x: int, y: str) -> None: ...
def takes_two_different_positional_only(x: int, y: str, /) -> None: ...
def takes_at_least_zero(*args) -> None: ...
def takes_at_least_one(x: int, *args) -> None: ...
def takes_at_least_two(x: int, y: int, *args) -> None: ...
def takes_at_least_two_positional_only(x: int, y: int, /, *args) -> None: ...
# Test all of the above with a number of different splatted argument types
def _(args: tuple[int]) -> None:
takes_zero(*args) # error: [too-many-positional-arguments]
takes_one(*args)
takes_two(*args) # error: [missing-argument]
takes_two_positional_only(*args) # error: [missing-argument]
takes_two_different(*args) # error: [missing-argument]
takes_two_different_positional_only(*args) # error: [missing-argument]
takes_at_least_zero(*args)
takes_at_least_one(*args)
takes_at_least_two(*args) # error: [missing-argument]
takes_at_least_two_positional_only(*args) # error: [missing-argument]
def _(args: tuple[int, int]) -> None:
takes_zero(*args) # error: [too-many-positional-arguments]
takes_one(*args) # error: [too-many-positional-arguments]
takes_two(*args)
takes_two_positional_only(*args)
takes_two_different(*args) # error: [invalid-argument-type]
takes_two_different_positional_only(*args) # error: [invalid-argument-type]
takes_at_least_zero(*args)
takes_at_least_one(*args)
takes_at_least_two(*args)
takes_at_least_two_positional_only(*args)
def _(args: tuple[int, str]) -> None:
takes_zero(*args) # error: [too-many-positional-arguments]
takes_one(*args) # error: [too-many-positional-arguments]
takes_two(*args) # error: [invalid-argument-type]
takes_two_positional_only(*args) # error: [invalid-argument-type]
takes_two_different(*args)
takes_two_different_positional_only(*args)
takes_at_least_zero(*args)
takes_at_least_one(*args)
takes_at_least_two(*args) # error: [invalid-argument-type]
takes_at_least_two_positional_only(*args) # error: [invalid-argument-type]
```
### Mixed tuple argument
```toml
[environment]
python-version = "3.11"
```
```py
def takes_zero() -> None: ...
def takes_one(x: int) -> None: ...
def takes_two(x: int, y: int) -> None: ...
def takes_two_positional_only(x: int, y: int, /) -> None: ...
def takes_two_different(x: int, y: str) -> None: ...
def takes_two_different_positional_only(x: int, y: str, /) -> None: ...
def takes_at_least_zero(*args) -> None: ...
def takes_at_least_one(x: int, *args) -> None: ...
def takes_at_least_two(x: int, y: int, *args) -> None: ...
def takes_at_least_two_positional_only(x: int, y: int, /, *args) -> None: ...
# Test all of the above with a number of different splatted argument types
def _(args: tuple[int, *tuple[int, ...]]) -> None:
takes_zero(*args) # error: [too-many-positional-arguments]
takes_one(*args)
takes_two(*args)
takes_two_positional_only(*args)
takes_two_different(*args) # error: [invalid-argument-type]
takes_two_different_positional_only(*args) # error: [invalid-argument-type]
takes_at_least_zero(*args)
takes_at_least_one(*args)
takes_at_least_two(*args)
takes_at_least_two_positional_only(*args)
def _(args: tuple[int, *tuple[str, ...]]) -> None:
takes_zero(*args) # error: [too-many-positional-arguments]
takes_one(*args)
takes_two(*args) # error: [invalid-argument-type]
takes_two_positional_only(*args) # error: [invalid-argument-type]
takes_two_different(*args)
takes_two_different_positional_only(*args)
takes_at_least_zero(*args)
takes_at_least_one(*args)
takes_at_least_two(*args) # error: [invalid-argument-type]
takes_at_least_two_positional_only(*args) # error: [invalid-argument-type]
def _(args: tuple[int, int, *tuple[int, ...]]) -> None:
takes_zero(*args) # error: [too-many-positional-arguments]
takes_one(*args) # error: [too-many-positional-arguments]
takes_two(*args)
takes_two_positional_only(*args)
takes_two_different(*args) # error: [invalid-argument-type]
takes_two_different_positional_only(*args) # error: [invalid-argument-type]
takes_at_least_zero(*args)
takes_at_least_one(*args)
takes_at_least_two(*args)
takes_at_least_two_positional_only(*args)
def _(args: tuple[int, int, *tuple[str, ...]]) -> None:
takes_zero(*args) # error: [too-many-positional-arguments]
takes_one(*args) # error: [too-many-positional-arguments]
takes_two(*args)
takes_two_positional_only(*args)
takes_two_different(*args) # error: [invalid-argument-type]
takes_two_different_positional_only(*args) # error: [invalid-argument-type]
takes_at_least_zero(*args)
takes_at_least_one(*args)
takes_at_least_two(*args)
takes_at_least_two_positional_only(*args)
def _(args: tuple[int, *tuple[int, ...], int]) -> None:
takes_zero(*args) # error: [too-many-positional-arguments]
takes_one(*args) # error: [too-many-positional-arguments]
takes_two(*args)
takes_two_positional_only(*args)
takes_two_different(*args) # error: [invalid-argument-type]
takes_two_different_positional_only(*args) # error: [invalid-argument-type]
takes_at_least_zero(*args)
takes_at_least_one(*args)
takes_at_least_two(*args)
takes_at_least_two_positional_only(*args)
def _(args: tuple[int, *tuple[str, ...], int]) -> None:
takes_zero(*args) # error: [too-many-positional-arguments]
takes_one(*args) # error: [too-many-positional-arguments]
takes_two(*args) # error: [invalid-argument-type]
takes_two_positional_only(*args) # error: [invalid-argument-type]
takes_two_different(*args)
takes_two_different_positional_only(*args)
takes_at_least_zero(*args)
takes_at_least_one(*args)
takes_at_least_two(*args) # error: [invalid-argument-type]
takes_at_least_two_positional_only(*args) # error: [invalid-argument-type]
```
### Argument expansion regression
This is a regression that was highlighted by the ecosystem check, which shows that we might need to
rethink how we perform argument expansion during overload resolution. In particular, we might need
to retry both `match_parameters` _and_ `check_types` for each expansion. Currently we only retry
`check_types`.
The issue is that argument expansion might produce a splatted value with a different arity than what
we originally inferred for the unexpanded value, and that in turn can affect which parameters the
splatted value is matched with.
The first example correctly produces an error. The `tuple[int, str]` union element has a precise
arity of two, and so parameter matching chooses the first overload. The second element of the tuple
does not match the second parameter type, which yielding an `invalid-argument-type` error.
The third example should produce the same error. However, because we have a union, we do not see the
precise arity of each union element during parameter matching. Instead, we infer an arity of "zero
or more" for the union as a whole, and use that less precise arity when matching parameters. We
therefore consider the second overload to still be a potential candidate for the `tuple[int, str]`
union element. During type checking, we have to force the arity of each union element to match the
inferred arity of the union as a whole (turning `tuple[int, str]` into `tuple[int | str, ...]`).
That less precise tuple type-checks successfully against the second overload, making us incorrectly
think that `tuple[int, str]` is a valid splatted call.
If we update argument expansion to retry parameter matching with the precise arity of each union
element, we will correctly rule out the second overload for `tuple[int, str]`, just like we do when
splatting that tuple directly (instead of as part of a union).
```py
from typing import overload
@overload
def f(x: int, y: int) -> None: ...
@overload
def f(x: int, y: str, z: int) -> None: ...
def f(*args): ...
# Test all of the above with a number of different splatted argument types
def _(t: tuple[int, str]) -> None:
f(*t) # error: [invalid-argument-type]
def _(t: tuple[int, str, int]) -> None:
f(*t)
def _(t: tuple[int, str] | tuple[int, str, int]) -> None:
# TODO: error: [invalid-argument-type]
f(*t)
```
## Wrong argument type
### Positional argument, positional-or-keyword parameter