Files

Carl Meyer 6d5fb09e92 [ty] fix and simplify callable type materializations (#22213 )

## Summary

A couple things I noticed when taking another look at the callable type
materializations.

1) Previously we wrongly ignored the return type when
bottom-materializing a callable with gradual signature, and always
changed it to `Never`.
2) We weren't correctly handling overloads that included a gradual
signature. Rather than separately materializing each overload, we would
just mark the entire callable as "top" or replace the entire callable
with the bottom signature.

Really, "top parameters" is something that belongs on the `Parameters`,
not on the entire `CallableType`. Conveniently, we already have
`ParametersKind` where we can track this, right next to where we already
track `ParametersKind::Gradual`. This saves a bit of memory, fixes the
two bugs above, and simplifies the implementation considerably (net
removal of 100+ LOC, a bunch of places that shouldn't need to care about
topness of a callable no longer need to.)

One user-visible change from this is that I now display the "top
callable" as `(Top[...]) -> object` instead of `Top[(...) -> object]`. I
think this is a (minor) improvement, because it wraps exactly the part
in `Top` that needs to be, rather than misleadingly wrapping the entire
callable type, including the return type (which has already been
separately materialized). I think the prior display would be
particularly confusing if the return type also has its own `Top` in it:
previously we could have e.g. `Top[(...) -> Top[list[Unknown]]]`, which
I think is less clear than the new `(Top[...]) -> Top[list[Unknown]]`.

## Test Plan

Added mdtests that failed before this PR and pass after it.

### Ecosystem

The changed diagnostics are all either the change to `Top` display, or
else known non-deterministic output. The added diagnostics are all true
positives:

The added diagnostic at
aa35ca1965/torchvision/transforms/v2/_utils.py (L149)
is a true positive that wasn't caught by the previous version. `str` is
not assignable to `Callable[[Any], Any]` (strings are not callable), nor
is the top callable (top callable includes callables that do not take a
single required positional argument.)

The added diagnostic at
081535ad9b/starlette/routing.py (L67)
is also a (pedantic) true positive. It's the same case as #1567 -- the
code assumes that it is impossible for a subclass of `Response` to
implement `__await__` (yielding something other than a `Response`).

The pytest added diagnostics are also both similar true positives: they
make the assumption that an object cannot simultaneously be a `Sequence`
and callable, or an `Iterable` and callable.

2025-12-27 10:45:07 -08:00

2.9 KiB

Raw Blame History

Narrowing for `callable()`

Basic narrowing

The callable() builtin returns TypeIs[Callable[..., object]], which narrows the type to the intersection with Top[Callable[..., object]]. The Top[...] wrapper indicates this is a fully static type representing the top materialization of a gradual callable.

Since all callable types are subtypes of Top[Callable[..., object]], intersections with Top[...] simplify to just the original callable type.

from typing import Any, Callable

def f(x: Callable[..., Any] | None):
    if callable(x):
        # The intersection simplifies because `(...) -> Any` is a subtype of
        # `Top[(...) -> object]` - all callables are subtypes of the top materialization.
        reveal_type(x)  # revealed: (...) -> Any
    else:
        # Since `(...) -> Any` is a subtype of `Top[(...) -> object]`, the intersection
        # with the negation is empty (Never), leaving just None.
        reveal_type(x)  # revealed: None

Narrowing with other callable types

from typing import Any, Callable

def g(x: Callable[[int], str] | None):
    if callable(x):
        # All callables are subtypes of `Top[(...) -> object]`, so the intersection simplifies.
        reveal_type(x)  # revealed: (int, /) -> str
    else:
        reveal_type(x)  # revealed: None

def h(x: Callable[..., int] | None):
    if callable(x):
        reveal_type(x)  # revealed: (...) -> int
    else:
        reveal_type(x)  # revealed: None

Narrowing from object

def f(x: object):
    if callable(x):
        reveal_type(x)  # revealed: Top[(...) -> object]
    else:
        reveal_type(x)  # revealed: ~Top[(...) -> object]

Calling narrowed callables

The narrowed type Top[Callable[..., object]] represents the set of all possible callable types (including, e.g., functions that take no arguments and functions that require arguments). While such objects are callable (they pass callable()), no specific set of arguments can be guaranteed to be valid.

import typing as t

def call_with_args(y: object, a: int, b: str) -> object:
    if isinstance(y, t.Callable):
        # error: [call-top-callable]
        return y(a, b)
    return None

Assignability of narrowed callables

A narrowed callable Top[Callable[..., object]] should be assignable to Callable[..., Any]. This is important for decorators and other patterns where we need to pass the narrowed callable to functions expecting gradual callables.

from typing import Any, Callable, TypeVar
from ty_extensions import static_assert, Top, is_assignable_to

static_assert(is_assignable_to(Top[Callable[..., bool]], Callable[..., int]))

F = TypeVar("F", bound=Callable[..., Any])

def wrap(f: F) -> F:
    return f

def f(x: object):
    if callable(x):
        # x has type `Top[(...) -> object]`, which should be assignable to `Callable[..., Any]`
        wrap(x)

2.9 KiB Raw Blame History

Narrowing for callable()