Compare commits

..

176 Commits

Author SHA1 Message Date
Micha Reiser
05854142e7 Test Salsa #851 2025-05-08 10:53:51 +02:00
David Peter
33eb008fb6 Disabled event handler if tracing is not enabled 2025-05-08 10:28:59 +02:00
David Peter
04d4febfc4 [ty] Update salsa 2025-05-08 10:17:32 +02:00
Shaygan Hooshyari
d566636ca5 Support typing.Self in methods (#17689)
## Summary

Fixes: astral-sh/ty#159 

This PR adds support for using `Self` in methods.
When the type of an annotation is `TypingSelf` it is converted to a type
var based on:
https://typing.python.org/en/latest/spec/generics.html#self

I just skipped Protocols because it had more problems and the tests was
not useful.
Also I need to create a follow up PR that implicitly assumes `self`
argument has type `Self`.

In order to infer the type in the `in_type_expression` method I needed
to have scope id and semantic index available. I used the idea from
[this PR](https://github.com/astral-sh/ruff/pull/17589/files) to pass
additional context to this method.
Also I think in all places that `in_type_expression` is called we need
to have this context because `Self` can be there so I didn't split the
method into one version with context and one without.

## Test Plan

Added new tests from spec.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Carl Meyer <carl@astral.sh>
2025-05-07 15:58:00 -07:00
Alex Waygood
51cef5a72b [ty] Recognise functions containing yield from expressions as being generator functions (#17930) 2025-05-07 23:29:44 +01:00
Douglas Creager
2cf5cba7ff [ty] Check base classes when determining subtyping etc for generic aliases (#17927)
#17897 added variance handling for legacy typevars — but they were only
being considered when checking generic aliases of the same class:

```py
class A: ...
class B(A): ...

class C[T]: ...

static_assert(is_subtype_of(C[B], C[A]))
```

and not for generic subclasses:

```py
class D[U](C[U]): ...

static_assert(is_subtype_of(D[B], C[A]))
```

Now we check those too!

Closes https://github.com/astral-sh/ty/issues/101
2025-05-07 15:21:11 -04:00
yunchi
ce0800fccf [pylint] add fix safety section (PLC2801) (#17825)
parent: #15584 
fix was introduced at: #9587 
reasoning: #9572
2025-05-07 14:34:34 -04:00
Micha Reiser
d03a7069ad Add instructions on how to upgrade to a newer Rust version (#17928) 2025-05-07 20:11:58 +02:00
Abhijeet Prasad Bodas
f5096f2050 [parser] Flag single unparenthesized generator expr with trailing comma in arguments. (#17893)
Fixes #17867

## Summary

The CPython parser does not allow generator expressions which are the
sole arguments in an argument list to have a trailing comma.
With this change, we start flagging such instances.

## Test Plan

Added new inline tests.
2025-05-07 14:11:35 -04:00
Alex Waygood
895b6161a6 [ty] Ensure that T is disjoint from ~T even when T is a TypeVar (#17922)
Same as https://github.com/astral-sh/ruff/pull/17910 but for
disjointness
2025-05-07 10:59:16 -07:00
Alex Waygood
74fe7982ba [ty] Sort collected diagnostics before snapshotting them in mdtest (#17926) 2025-05-07 18:23:22 +01:00
Micha Reiser
51386b3c7a [ty] Add basic file watching to server (#17912) 2025-05-07 19:03:30 +02:00
Charlie Marsh
51e2effd2d Make completions an opt-in LSP feature (#17921)
## Summary

We now expect the client to send initialization options to opt-in to
experimental (but LSP-standardized) features, like completion support.
Specifically, the client should set `"experimental.completions.enable":
true`.

Closes https://github.com/astral-sh/ty/issues/74.
2025-05-07 16:39:35 +00:00
Dhruv Manilawala
82d31a6014 Add link to ty issue tracker (#17924)
Co-authored-by: Micha Reiser <micha@reiser.io>
2025-05-07 16:33:27 +00:00
Dhruv Manilawala
78054824c0 [ty] Add support for __all__ (#17856)
## Summary

This PR adds support for the `__all__` module variable.

Reference spec:
https://typing.python.org/en/latest/spec/distributing.html#library-interface-public-and-private-symbols

This PR adds a new `dunder_all_names` query that returns a set of
`Name`s defined in the `__all__` variable of the given `File`. The query
works by implementing the `StatementVisitor` and collects all the names
by recognizing the supported idioms as mentioned in the spec. Any idiom
that's not recognized are ignored.

The current implementation is minimum to what's required for us to
remove all the false positives that this is causing. Refer to the
"Follow-ups" section below to see what we can do next. I'll a open
separate issue to keep track of them.

Closes: astral-sh/ty#106 
Closes: astral-sh/ty#199

### Follow-ups

* Diagnostics:
* Add warning diagnostics for unrecognized `__all__` idioms, `__all__`
containing non-string element
* Add an error diagnostic for elements that are present in `__all__` but
not defined in the module. This could lead to runtime error
* Maybe we should return `<type>` instead of `Unknown | <type>` for
`module.__all__`. For example:
https://playknot.ruff.rs/2a6fe5d7-4e16-45b1-8ec3-d79f2d4ca894
* Mark a symbol that's mentioned in `__all__` as used otherwise it could
raise (possibly in the future) "unused-name" diagnostic

Supporting diagnostics will require that we update the return type of
the query to be something other than `Option<FxHashSet<Name>>`,
something that behaves like a result and provides a way to check whether
a name exists in `__all__`, loop over elements in `__all__`, loop over
the invalid elements, etc.

## Ecosystem analysis

The following are the maximum amount of diagnostics **removed** in the
ecosystem:

* "Type <module '...'> has no attribute ..."
    * `collections.abc` - 14
    * `numpy` - 35534
    * `numpy.ma` - 296
    * `numpy.char` - 37
    * `numpy.testing` - 175
    * `hashlib` - 311
    * `scipy.fft` - 2
    * `scipy.stats` - 38
* "Module '...' has no member ..."
    * `collections.abc` - 85
    * `numpy` - 508
    * `numpy.testing` - 741
    * `hashlib` - 36
    * `scipy.stats` - 68
    * `scipy.interpolate` - 7
    * `scipy.signal` - 5

The following modules have dynamic `__all__` definition, so `ty` assumes
that `__all__` doesn't exists in that module:
* `scipy.stats`
(95a5d6ea8b/scipy/stats/__init__.py (L665))
* `scipy.interpolate`
(95a5d6ea8b/scipy/interpolate/__init__.py (L221))
* `scipy.signal` (indirectly via
95a5d6ea8b/scipy/signal/_signal_api.py (L30))
* `numpy.testing`
(de784cd6ee/numpy/testing/__init__.py (L16-L18))

~There's this one category of **false positives** that have been added:~
Fixed the false positives by also ignoring `__all__` from a module that
uses unrecognized idioms.

<details><summary>Details about the false postivie:</summary>
<p>

The `scipy.stats` module has dynamic `__all__` and it imports a bunch of
symbols via star imports. Some of those modules have a mix of valid and
invalid `__all__` idioms. For example, in
95a5d6ea8b/scipy/stats/distributions.py (L18-L24),
2 out of 4 `__all__` idioms are invalid but currently `ty` recognizes
two of them and says that the module has a `__all__` with 5 values. This
leads to around **2055** newly added false positives of the form:
```
Type <module 'scipy.stats'> has no attribute ...
```

I think the fix here is to completely ignore `__all__`, not only if
there are invalid elements in it, but also if there are unrecognized
idioms used in the module.

</p>
</details> 

## Test Plan

Add a bunch of test cases using the new `ty_extensions.dunder_all_names`
function to extract a module's `__all__` names.

Update various test cases to remove false positives around `*` imports
and re-export convention.

Add new test cases for named import behavior as `*` imports covers all
of it already (thanks Alex!).
2025-05-07 21:42:42 +05:30
Alex Waygood
c6f4929cdc [ty] fix assigning a typevar to a union with itself (#17910)
Co-authored-by: Carl Meyer <carl@astral.sh>
2025-05-07 15:50:22 +00:00
Alex Waygood
2ec0d7e072 [ty] Improve UX for [duplicate-base] diagnostics (#17914) 2025-05-07 15:27:37 +00:00
Charlie Marsh
ad658f4d68 Clean up some Ruff references in the ty server (#17920)
## Summary

Anything user-facing, etc.
2025-05-07 10:55:16 -04:00
Abhijeet Prasad Bodas
3dedd70a92 [ty] Detect overloads decorated with @dataclass_transform (#17835)
## Summary

Fixes #17541

Before this change, in the case of overloaded functions,
`@dataclass_transform` was detected only when applied to the
implementation, not the overloads.
However, the spec also allows this decorator to be applied to any of the
overloads as well.
With this PR, we start handling `@dataclass_transform`s applied to
overloads.

## Test Plan

Fixed existing TODOs in the test suite.
2025-05-07 15:51:13 +02:00
David Peter
fab862c8cd [ty] Ecosystem checks: activate running on 'manticore' (#17916)
## Summary

This is sort of an anticlimactic resolution to #17863, but now that we
understand what the root cause for the stack overflows was, I think it's
fine to enable running on this project. See the linked ticket for the
full analysis.

closes #17863

## Test Plan

Ran lots of times locally and never observed a crash at worker thread
stack sizes > 8 MiB.
2025-05-07 06:27:36 -07:00
Douglas Creager
0d9b6a0975 [ty] Handle explicit variance in legacy typevars (#17897)
We now track the variance of each typevar, and obey the `covariant` and
`contravariant` parameters to the legacy `TypeVar` constructor. We still
don't yet infer variance for PEP-695 typevars or for the
`infer_variance` legacy constructor parameter.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Carl Meyer <carl@astral.sh>
2025-05-07 08:44:51 -04:00
Micha Reiser
c5e299e796 Update salsa (#17895) 2025-05-07 09:51:15 +02:00
Victor Hugo Gomes
c504001b32 [pyupgrade] Add spaces between tokens as necessary to avoid syntax errors in UP018 autofix (#17648)
Co-authored-by: Micha Reiser <micha@reiser.io>
2025-05-07 09:34:08 +02:00
David Peter
04457f99b6 [ty] Protocols: Fixpoint iteration for fully-static check (#17880)
## Summary

A recursive protocol like the following would previously lead to stack
overflows when attempting to create the union type for the `P | None`
member, because `UnionBuilder` checks if element types are fully static,
and the fully-static check on `P` would in turn list all members and
check whether all of them were fully static, leading to a cycle.

```py
from __future__ import annotations

from typing import Protocol

class P(Protocol):
    parent: P | None
```

Here, we make the fully-static check on protocols a salsa query and add
fixpoint iteration, starting with `true` as the initial value (assume
that the recursive protocol is fully-static). If the recursive protocol
has any non-fully-static members, we still return `false` when
re-executing the query (see newly added tests).

closes #17861

## Test Plan

Added regression test
2025-05-07 08:55:21 +02:00
InSync
a33d0d4bf4 [ty] Support generate-shell-completion (#17879)
## Summary

Resolves #15502.

`ty generate-shell-completion` now works in a similar manner to `ruff
generate-shell-completion`.

## Test Plan

Manually:

<details>

```shell
$ cargo run --package ty generate-shell-completion nushell
module completions {

  # An extremely fast Python type checker.
  export extern ty [
    --help(-h)                # Print help
    --version(-V)             # Print version
  ]
  
  # ...

}

export use completions *
```
</details>
2025-05-06 18:04:57 -07:00
Charlie Marsh
443f62e98d Remove condensed display type enum (#17902)
## Summary

See: https://github.com/astral-sh/ruff/pull/17889#discussion_r2076556002
2025-05-06 18:04:03 -07:00
Charlie Marsh
a2e9a7732a Update class literal display to use <class 'Foo'> style (#17889)
## Summary

Closes https://github.com/astral-sh/ruff/issues/17238.
2025-05-06 20:11:25 -04:00
Micha Reiser
b2de749c32 Add a note to diagnostics why the rule is enabled (#17854) 2025-05-06 20:29:03 +02:00
Douglas Creager
9085f18353 [ty] Propagate specializations to ancestor base classes (#17892)
@AlexWaygood discovered that even though we've been propagating
specializations to _parent_ base classes correctly, we haven't been
passing them on to _grandparent_ base classes:
https://github.com/astral-sh/ruff/pull/17832#issuecomment-2854360969

```py
class Bar[T]:
    x: T

class Baz[T](Bar[T]): ...
class Spam[T](Baz[T]): ...

reveal_type(Spam[int]().x) # revealed: `T`, but should be `int`
```

This PR updates the MRO machinery to apply the current specialization
when starting to iterate the MRO of each base class.
2025-05-06 14:25:21 -04:00
Dylan
8152ba7cb7 [ty] Add minimal docs for a few lints (#17874)
Just the bare minimum to remove a few TODOs - omitted examples, and only
did 9 but I will check back tomorrow and try to knock out a few more!
2025-05-06 10:36:47 -07:00
Aria Desires
3d01d3be3e update the repository for ty (#17891)
This metadata is used by cargo-dist for artifact URLs (in curl-sh
expressions)
2025-05-06 12:03:38 -04:00
Brent Westbrook
4510a236d3 Default to latest supported Python version for version-related syntax errors (#17529)
## Summary

This PR partially addresses #16418 via the following:

- `LinterSettings::unresolved_python_version` is now a `TargetVersion`,
which is a thin wrapper around an `Option<PythonVersion>`
- `Checker::target_version` now calls `TargetVersion::linter_version`
internally, which in turn uses `unwrap_or_default` to preserve the
current default behavior
- Calls to the parser now call `TargetVersion::parser_version`, which
calls `unwrap_or_else(PythonVersion::latest)`
- The `Checker`'s implementation of
`SemanticSyntaxContext::python_version` also uses
`TargetVersion::parser_version` to use `PythonVersion::latest` for
semantic errors

In short, all lint rule behavior should be unchanged, but we default to
the latest Python version for the new syntax errors, which should
minimize confusing version-related syntax errors for users without a
version configured.

## Test Plan

Existing tests, which showed no changes (except for printing default
settings).
2025-05-06 10:19:13 -04:00
Marcus Näslund
76b6d53d8b Add new rule InEmptyCollection (#16480)
## Summary

Introducing a new rule based on discussions in #15732 and #15729 that
checks for unnecessary in with empty collections.

I called it in_empty_collection and gave the rule number RUF060.

Rule is in preview group.
2025-05-06 13:52:07 +00:00
Zanie Blue
f82b72882b Display ty version for ty --version and ty -V (#17888)
e.g.,

```
❯ uv run -q -- ty -V
ty 0.0.0-alpha.4 (08881edba 2025-05-05)
❯ uv run -q -- ty --version
ty 0.0.0-alpha.4 (08881edba 2025-05-05)
```

Previously, this just displayed `ty 0.0.0` because it didn't use our
custom version implementation. We no longer have a short version —
matching the interface in uv. We could add a variant for it, if it seems
important to people. However, I think we found it more confusing than
not over there and didn't get any complaints about the change.

Closes https://github.com/astral-sh/ty/issues/54
2025-05-06 08:06:41 -05:00
Zanie Blue
d07eefc408 Parse dist-workspace.toml for version (#17868)
Extends https://github.com/astral-sh/ruff/pull/17866, using
`dist-workspace.toml` as a source of truth for versions to enable
version retrieval in distributions that are not Git repositories (i.e.,
Python source distributions and source tarballs consumed by Linux
distros).

I retain the Git tag lookup from
https://github.com/astral-sh/ruff/pull/17866 as a fallback — it seems
harmless, but we could drop it to simplify things here.

I confirmed this works from the repository as well as Python source and
binary distributions:

```
❯ uv run --refresh-package ty --reinstall-package ty -q --  ty version
ty 0.0.1-alpha.1+5 (2eadc9e61 2025-05-05)
❯ uv build
...
❯ uvx --from ty@dist/ty-0.0.0a1.tar.gz --no-cache -q -- ty version
ty 0.0.1-alpha.1
❯ uvx --from ty@dist/ty-0.0.0a1-py3-none-macosx_11_0_arm64.whl -q -- ty version
ty 0.0.1-alpha.1
```

Requires https://github.com/astral-sh/ty/pull/36

cc @Gankra and @MichaReiser for review.
2025-05-06 12:18:17 +00:00
Zanie Blue
f7237e3b69 Update ty version to use parent Git repository information (#17866)
Currently, `ty version` pulls its information from the Ruff repository —
but we want this to pull from the repository in the directory _above_
when Ruff is a submodule.

I tested this in the `ty` repository after tagging an arbitrary commit:

```
❯ uv run --refresh-package ty --reinstall-package ty ty version
      Built ty @ file:///Users/zb/workspace/ty
Uninstalled 1 package in 2ms
Installed 1 package in 1ms
ty 0.0.0+3 (34253b1d4 2025-05-05)
```

We also use the last Git tag as the source of truth for the version,
instead of the crate version. However, we'll need a way to set the
version for releases still, as the tag is published _after_ the build.
We can either tag early (without pushing the tag to the remote), or add
another environment variable. (**Note, this approach is changed in a
follow-up. See https://github.com/astral-sh/ruff/pull/17868**)

From this repository, the version will be `unknown`:

```
❯ cargo run -q --bin ty -- version
ty unknown
```

We could add special handling like... `ty unknown (ruff@...)` but I see
that as a secondary goal.

Closes https://github.com/astral-sh/ty/issues/5

The reviewer situation in this repository is unhinged, cc @Gankra and
@MichaReiser for review.
2025-05-06 07:14:30 -05:00
Alex Waygood
2f9992b6ef [ty] Fix duplicate diagnostics for unresolved module when an import from statement imports multiple members (#17886) 2025-05-06 12:37:10 +01:00
Alex Waygood
457ec4dddd Generalize special-casing for enums constructed with the functional syntax (#17885) 2025-05-06 11:02:55 +01:00
Micha Reiser
aa0614509b Update salsa to pull in fixpoint fixes (#17847) 2025-05-06 11:27:51 +02:00
Micha Reiser
9000eb3bfd Update favicon and URL for playground (#17860) 2025-05-06 10:51:45 +02:00
Micha Reiser
7f50b503cf Allowlist play.ty.dev (#17857) 2025-05-06 10:16:17 +02:00
Micha Reiser
24d3fc27fb Fixup wording of fatal error warning (#17881) 2025-05-06 09:44:23 +02:00
Micha Reiser
6f821ac846 Show a warning at the end of the diagnostic list if there are any fatal warnings (#17855) 2025-05-06 07:14:21 +00:00
Charlie Marsh
d410d12bc5 Update code of conduct email address (#17875)
## Summary

For consistency with the `pyproject.toml`, etc.
2025-05-05 21:34:45 -04:00
Alex Waygood
89424cce5f [ty] Do not emit errors if enums or NamedTuples constructed using functional syntax are used in type expressions (#17873)
## Summary

This fixes some false positives that showed up in the primer diff for
https://github.com/astral-sh/ruff/pull/17832

## Test Plan

new mdtests added that fail with false-positive diagnostics on `main`
2025-05-06 00:37:24 +01:00
Shunsuke Shibayama
fd76d70a31 [red-knot] fix narrowing in nested scopes (#17630)
## Summary

This PR fixes #17595.

## Test Plan

New test cases are added to `mdtest/narrow/conditionals/nested.md`.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2025-05-05 16:28:42 -07:00
yunchi
a4c8e43c5f [pylint] Add fix safety section (PLR1722) (#17826)
parent: #15584 
fix introduced at: #816
2025-05-05 19:13:04 -04:00
Douglas Creager
ada4c4cb1f [ty] Don't require default typevars when specializing (#17872)
If a typevar is declared as having a default, we shouldn't require a
type to be specified for that typevar when explicitly specializing a
generic class:

```py
class WithDefault[T, U = int]: ...

reveal_type(WithDefault[str]())  # revealed: WithDefault[str, int]
```

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-05-05 18:29:30 -04:00
Alex Waygood
bb6c7cad07 [ty] Fix false-positive [invalid-return-type] diagnostics on generator functions (#17871) 2025-05-05 21:44:59 +00:00
Douglas Creager
47e3aa40b3 [ty] Specialize bound methods and nominal instances (#17865)
Fixes
https://github.com/astral-sh/ruff/pull/17832#issuecomment-2851224968. We
had a comment that we did not need to apply specializations to generic
aliases, or to the bound `self` of a bound method, because they were
already specialized. But they might be specialized with a type variable,
which _does_ need to be specialized, in the case of a "multi-step"
specialization, such as:

```py
class LinkedList[T]: ...

class C[U]:
    def method(self) -> LinkedList[U]:
        return LinkedList[U]()
```

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-05-05 17:17:36 -04:00
David Peter
9a6633da0b [ty] ecosystem: activate running on 'sympy' (#17870)
## Summary

Following #17869, we can now run `ty` on `sympy`.
2025-05-05 21:50:13 +02:00
David Peter
de78da5ee6 [ty] Increase worker-thread stack size (#17869)
## Summary

closes #17472 

This is obviously just a band-aid solution to this problem (in that you
can always make your [pathological
inputs](28994edd82/sympy/polys/numberfields/resolvent_lookup.py)
bigger and it will still crash), but I think this is not an unreasonable
change — even if we add more sophisticated solutions later. I tried
using `stacker` as suggested by @MichaReiser, and it works. But it's
unclear where exactly would be the right place to put it, and even for
the `sympy` problem, we would need to add it both in the semantic index
builder AST traversal and in type inference. Increasing the default
stack size for worker threads, as proposed here, doesn't solve the
underlying problem (that there is a hard limit), but it is more
universal in the sense that it is not specific to large binary-operator
expression chains.

To determine a reasonable stack size, I created files that look like

*right associative*:
```py
from typing import reveal_type
total = (1 + (1 + (1 + (1 + (… + 1)))))
reveal_type(total)
```

*left associative*
```py
from typing import reveal_type
total = 1 + 1 + 1 + 1 + … + 1
reveal_type(total)
```

with a variable amount of operands (`N`). I then chose the stack size
large enough to still be able to handle cases that existing type
checkers can not:

```
right

  N = 20: mypy takes ~ 1min
  N = 350: pyright crashes with a stack overflow (mypy fails with "too many nested parentheses")
  N = 800: ty(main) infers Literal[800] instantly
  N = 1000: ty(main) crashes with "thread '<unknown>' has overflowed its stack"

  N = 7000: ty(this branch) infers Literal[7000] instantly
  N = 8000+: ty(this branch) crashes


left

  N = 300: pyright emits "Maximum parse depth exceeded; break expression into smaller sub-expressions"
           total is inferred as Unknown
  N = 5500: mypy crashes with "INTERNAL ERROR"
  N = 2500: ty(main) infers Literal[2500] instantly
  N = 3000: ty(main) crashes with "thread '<unknown>' has overflowed its stack"

  N = 22000: ty(this branch) infers Literal[22000] instantly
  N = 23000+: ty(this branch) crashes
```

## Test Plan

New regression test.
2025-05-05 21:31:55 +02:00
Carl Meyer
20d64b9c85 [ty] add passing projects to primer (#17834)
Add projects to primer that now pass, with
https://github.com/astral-sh/ruff/pull/17833
2025-05-05 12:21:06 -07:00
Carl Meyer
4850c187ea [ty] add cycle handling for FunctionType::signature query (#17833)
This fixes cycle panics in several ecosystem projects (moved to
`good.txt` in a following PR
https://github.com/astral-sh/ruff/pull/17834 because our mypy-primer job
doesn't handle it well if we move projects to `good.txt` in the same PR
that fixes `ty` to handle them), as well as in the minimal case in the
added mdtest. It also fixes a number of panicking fuzzer seeds. It
doesn't appear to cause any regression in any ecosystem project or any
fuzzer seed.
2025-05-05 12:12:38 -07:00
Vasco Schiavo
3f32446e16 [ruff] add fix safety section (RUF013) (#17759)
The PR add the fix safety section for rule `RUF013`
(https://github.com/astral-sh/ruff/issues/15584 )
The fix was introduced here #4831

The rule as a lot of False Negative (as it is explained in the docs of
the rule).

The main reason because the fix is unsafe is that it could change code
generation tools behaviour, as in the example here:

```python
def generate_api_docs(func):
    hints = get_type_hints(func)
    for param, hint in hints.items():
        if is_optional_type(hint):
            print(f"Parameter '{param}' is optional")
        else:
            print(f"Parameter '{param}' is required")

# Before fix
def create_user(name: str, roles: list[str] = None):
    pass

# After fix
def create_user(name: str, roles: Optional[list[str]] = None):
    pass

# Generated docs would change from "roles is required" to "roles is optional"
```
2025-05-05 13:47:56 -05:00
Aria Desires
784daae497 migrate to dist-workspace.toml, use new workspace.packages config (#17864) 2025-05-05 14:07:46 -04:00
Max Mynter
178c882740 [semantic-syntax-tests] Add test fixtures for AwaitOutsideAsyncFunction (#17785)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
Re: #17526 
## Summary
Add test fixtures for `AwaitOutsideAsync` and
`AsyncComprehensionOutsideAsyncFunction` errors.

<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan
This is a test. 

<!-- How was it tested? -->
2025-05-05 14:02:06 -04:00
Max Mynter
101e1a5ddd [semantic-syntax-tests] for for InvalidStarExpression, DuplicateMatchKey, and DuplicateMatchClassAttribute (#17754)
Re: #17526 

## Summary
Add integration tests for Python Semantic Syntax for
`InvalidStarExpression`, `DuplicateMatchKey`, and
`DuplicateMatchClassAttribute`.

## Note
- Red knot integration tests for `DuplicateMatchKey` exist already in
line 89-101.
<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan
This is a test.
<!-- How was it tested? -->
2025-05-05 17:30:16 +00:00
Dylan
965a4dd731 [isort] Check full module path against project root(s) when categorizing first-party (#16565)
When attempting to determine whether `import foo.bar.baz` is a known
first-party import relative to [user-provided source
paths](https://docs.astral.sh/ruff/settings/#src), when `preview` is
enabled we now check that `SRC/foo/bar/baz` is a directory or
`SRC/foo/bar/baz.py` or `SRC/foo/bar/baz.pyi` exist.

Previously, we just checked the analogous thing for `SRC/foo`, but this
can be misleading in situations with disjoint namespace packages that
share a common base name (e.g. we may be working inside the namespace
package `foo.buzz` and importing `foo.bar` from elsewhere).

Supersedes #12987 
Closes #12984
2025-05-05 11:40:01 -05:00
Victor Hugo Gomes
5e2c818417 [flake8-bandit] Mark tuples of string literals as trusted input in S603 (#17801)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

Fixes #17798
<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

Snapshot tests
<!-- How was it tested? -->
2025-05-05 10:50:44 -04:00
David Peter
90c12f4177 [ty] ecosystem: Activate running on 'materialize' (#17862) 2025-05-05 16:34:23 +02:00
Wei Lee
6e9fb9af38 [airflow] Skip attribute check in try catch block (AIR301) (#17790)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

Skip attribute check in try catch block (`AIR301`)

## Test Plan

<!-- How was it tested? -->
update
`crates/ruff_linter/resources/test/fixtures/airflow/AIR301_names_try.py`
2025-05-05 10:01:05 -04:00
Wei Lee
a507c1b8b3 [airflow] Remove airflow.utils.dag_parsing_context.get_parsing_context (AIR301) (#17852)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->
Remove `airflow.utils.dag_parsing_context.get_parsing_context` from
AIR301 as it has been moved to AIR311

## Test Plan

<!-- How was it tested? -->

the test fixture was updated in the previous PR
2025-05-05 09:31:57 -04:00
David Peter
5a91badb8b [ty] Move 'scipy' to list of 'good' projects (#17850)
## Summary

Adds `scipy` as a new project to `good.txt` as a follow-up to #17849.
2025-05-05 13:52:57 +02:00
David Peter
1945bfdb84 [ty] Fix standalone expression type retrieval in presence of cycles (#17849)
## Summary

When entering an `infer_expression_types` cycle from
`TypeInferenceBuilder::infer_standalone_expression`, we might get back a
`TypeInference::cycle_fallback(…)` that doesn't actually contain any new
types, but instead it contains a `cycle_fallback_type` which is set to
`Some(Type::Never)`. When calling `self.extend(…)`, we therefore don't
really pull in a type for the expression we're interested in. This
caused us to panic if we tried to call `self.expression_type(…)` after
`self.extend(…)`.

The proposed fix here is to retrieve that type from the nested
`TypeInferenceBuilder` directly, which will correctly fall back to
`cycle_fallback_type`.

## Details

I minimized the second example from #17792 a bit further and used this
example for debugging:

```py
from __future__ import annotations

class C: ...

def f(arg: C):
    pass

x, _ = f(1)

assert x
```

This is self-referential because when we check the assignment statement
`x, _ = f(1)`, we need to look up the signature of `f`. Since evaluation
of annotations is deferred, we look up the public type of `C` for the
`arg` parameter. The public use of `C` is visibility-constraint by "`x`"
via the `assert` statement. While evaluating this constraint, we need to
look up the type of `x`, which in turn leads us back to the `x, _ =
f(1)` definition.

The reason why this only showed up in the relatively peculiar case with
unpack assignments is the code here:


78b4c3ccf1/crates/ty_python_semantic/src/types/infer.rs (L2709-L2718)

For a non-unpack assignment like `x = f(1)`, we would not try to infer
the right-hand side eagerly. Instead, we would enter a
`infer_definition_types` cycle that handles the situation correctly. For
unpack assignments, however, we try to infer the type of `value`
(`f(1)`) and therefore enter the cycle via `standalone_expression_type
=> infer_expression_type`.

closes #17792 

## Test Plan

* New regression test
* Made sure that we can now run successfully on scipy => see #17850
2025-05-05 13:52:08 +02:00
Dylan
a95c73d5d0 Implement deferred annotations for Python 3.14 (#17658)
This PR updates the semantic model for Python 3.14 by essentially
equating "run using Python 3.14" with "uses `from __future__ import
annotations`".

While this is not technically correct under the hood, it appears to be
correct for the purposes of our semantic model. That is: from the point
of view of deciding when to parse, bind, etc. annotations, these two
contexts behave the same. More generally these contexts behave the same
unless you are performing some kind of introspection like the following:


Without future import:
```pycon
>>> from annotationlib import get_annotations,Format
>>> def foo()->Bar:...
...
>>> get_annotations(foo,format=Format.FORWARDREF)
{'return': ForwardRef('Bar')}
>>> get_annotations(foo,format=Format.STRING)
{'return': 'Bar'}
>>> get_annotations(foo,format=Format.VALUE)
Traceback (most recent call last):
[...]
NameError: name 'Bar' is not defined
>>> get_annotations(foo)
Traceback (most recent call last):
[...]
NameError: name 'Bar' is not defined
```

With future import:
```
>>> from __future__ import annotations
>>> from annotationlib import get_annotations,Format
>>> def foo()->Bar:...
...
>>> get_annotations(foo,format=Format.FORWARDREF)
{'return': 'Bar'}
>>> get_annotations(foo,format=Format.STRING)
{'return': 'Bar'}
>>> get_annotations(foo,format=Format.VALUE)
{'return': 'Bar'}
>>> get_annotations(foo)
{'return': 'Bar'}
```

(Note: the result of the last call to `get_annotations` in these
examples relies on the fact that, as of this writing, the default value
for `format` is `Format.VALUE`).

If one day we support lint rules targeting code that introspects using
the new `annotationlib`, then it is possible we will need to revisit our
approximation.

Closes #15100
2025-05-05 06:40:36 -05:00
David Peter
78b4c3ccf1 [ty] Minor typo in environment variable name (#17848) 2025-05-05 10:25:48 +00:00
renovate[bot]
2485afe640 Update pre-commit dependencies (#17840)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Micha Reiser <micha@reiser.io>
2025-05-05 07:36:09 +00:00
renovate[bot]
b8ed729f59 Update taiki-e/install-action digest to 86c23ee (#17838)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-05-05 07:43:54 +02:00
renovate[bot]
108c470348 Update actions/download-artifact action to v4.3.0 (#17845)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-05-05 07:43:33 +02:00
renovate[bot]
87c64c9eab Update actions/setup-python action to v5.6.0 (#17846)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-05-05 07:43:01 +02:00
renovate[bot]
a10606dda2 Update Rust crate jiff to v0.2.12 (#17843) 2025-05-05 07:34:09 +02:00
renovate[bot]
d1c6dd9ac1 Update Rust crate toml to v0.8.22 (#17844) 2025-05-05 07:32:50 +02:00
renovate[bot]
073b993ab0 Update Rust crate assert_fs to v1.1.3 (#17841) 2025-05-05 07:31:02 +02:00
renovate[bot]
6a36cd6f02 Update Rust crate hashbrown to v0.15.3 (#17842) 2025-05-05 07:30:46 +02:00
renovate[bot]
3b15af6d4f Update dependency ruff to v0.11.8 (#17839) 2025-05-05 07:29:36 +02:00
Micha Reiser
e95130ad80 Introduce TY_MAX_PARALLELISM environment variable (#17830) 2025-05-04 16:27:15 +02:00
Micha Reiser
68e32c103f Ignore PRs labeled with ty for Ruff changelog (#17831) 2025-05-04 14:42:10 +01:00
Brent Westbrook
fe4051b2e6 Fix missing combine call for lint.typing-extensions setting (#17823)
## Summary

Fixes #17821.

## Test Plan

New CLI test. This might be overkill for such a simple fix, but it made
me feel better to add a test.
2025-05-03 18:19:19 -04:00
Micha Reiser
fa628018b2 Use #[expect(lint)] over #[allow(lint)] where possible (#17822) 2025-05-03 21:20:31 +02:00
Eric Botti
8535af8516 [red-knot] Add support for the LSP diagnostic tag (#17657)
Co-authored-by: Micha Reiser <micha@reiser.io>
2025-05-03 20:35:03 +02:00
Micha Reiser
b51c4f82ea Rename Red Knot (#17820) 2025-05-03 19:49:15 +02:00
Alex Waygood
e6a798b962 [red-knot] Recurse into the types of protocol members when normalizing a protocol's interface (#17808)
## Summary

Currently red-knot does not understand `Foo` and `Bar` here as being
equivalent:

```py
from typing import Protocol

class A: ...
class B: ...
class C: ...

class Foo(Protocol):
    x: A | B | C

class Bar(Protocol):
    x: B | A | C
```

Nor does it understand `A | B | Foo` as being equivalent to `Bar | B |
A`. This PR fixes that.

## Test Plan

new mdtest assertions added that fail on `main`
2025-05-03 16:43:37 +01:00
Alex Waygood
52b0470870 [red-knot] Synthesize a __call__ attribute for Callable types (#17809)
## Summary

Currently this assertion fails on `main`, because we do not synthesize a
`__call__` attribute for Callable types:

```py
from typing import Protocol, Callable
from knot_extensions import static_assert, is_assignable_to

class Foo(Protocol):
    def __call__(self, x: int, /) -> str: ...

static_assert(is_assignable_to(Callable[[int], str], Foo))
```

This PR fixes that.

See previous discussion about this in
https://github.com/astral-sh/ruff/pull/16493#discussion_r1985098508 and
https://github.com/astral-sh/ruff/pull/17682#issuecomment-2839527750

## Test Plan

Existing mdtests updated; a couple of new ones added.
2025-05-03 16:43:18 +01:00
Brent Westbrook
c4a08782cc Add regression test for parent noqa (#17783)
Summary
--

Adds a regression test for #2253 after I tried to delete the fix from
#2464.
2025-05-03 11:38:31 -04:00
Alex Waygood
91481a8be7 [red-knot] Minor simplifications to types/display.rs (#17813) 2025-05-03 15:29:05 +01:00
Victor Hugo Gomes
097af060c9 [refurb] Fix false positive for float and complex numbers in FURB116 (#17661) 2025-05-03 15:59:46 +02:00
Alex Waygood
b7d0b3f9e5 [red-knot] Add tests asserting that subclasses of Any are assignable to arbitrary protocol types (#17810) 2025-05-03 12:41:55 +00:00
Alex Waygood
084352f72c [red-knot] Distinguish fully static protocols from non-fully-static protocols (#17795) 2025-05-03 11:12:23 +01:00
Carl Meyer
78d4356301 [red-knot] add tracing of salsa events in mdtests (#17803) 2025-05-03 09:00:11 +02:00
Douglas Creager
96697c98f3 [red-knot] Legacy generic classes (#17721)
This adds support for legacy generic classes, which use a
`typing.Generic` base class, or which inherit from another generic class
that has been specialized with legacy typevars.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2025-05-02 20:34:20 -04:00
Micha Reiser
f7cae4ffb5 [red-knot] Don't panic when primary-span is missing while panicking (#17799) 2025-05-02 21:19:03 +01:00
Douglas Creager
675a5af89a [red-knot] Use Vec in CallArguments; reuse self when we can (#17793)
Quick follow-on to #17788. If there is no bound `self` parameter, we can
reuse the existing `CallArgument{,Type}s`, and we can use a straight
`Vec` instead of a `VecDeque`.
2025-05-02 12:00:02 -04:00
David Peter
ea3f4ac059 [red-knot] Refactor: no mutability in call APIs (#17788)
## Summary

Remove mutability in parameter types for a few functions such as
`with_self` and `try_call`. I tried the `Rc`-approach with cheap cloning
[suggest
here](https://github.com/astral-sh/ruff/pull/17733#discussion_r2068722860)
first, but it turns out we need a whole stack of prepended arguments
(there can be [both `self` *and*
`cls`](3cf44e401a/crates/red_knot_python_semantic/resources/mdtest/call/constructor.md (L113))),
and we would need the same construct not just for `CallArguments` but
also for `CallArgumentTypes`. At that point we're cloning `VecDeque`s
anyway, so the overhead of cloning the whole `VecDeque` with all
arguments didn't seem to justify the additional code complexity.

## Benchmarks

Benchmarks on tomllib, black, jinja, isort seem neutral.
2025-05-02 13:53:19 +02:00
David Peter
6d2c10cca2 [red-knot] Fix panic for tuple[x[y]] string annotation (#17787)
## Summary

closes #17775

## Test Plan

Added corpus regression test
2025-05-02 12:11:47 +02:00
David Peter
3cf44e401a [red-knot] Implicit instance attributes in generic methods (#17769)
## Summary

Add the ability to detect instance attribute assignments in class
methods that are generic.

This does not address the code duplication mentioned in #16928. I can
open a ticket for this after this has been merged.

closes #16928

## Test Plan

Added regression test.
2025-05-02 08:20:37 +00:00
Micha Reiser
17050e2ec5 doc: Add link to check-typed-exception from S110 and S112 (#17786) 2025-05-02 09:25:58 +02:00
Micha Reiser
a6dc04f96e Fix module name in ASYNC110, 115, and 116 fixes (#17774) 2025-05-01 23:37:09 +02:00
David Peter
e515899141 [red-knot] More informative hover-types for assignments (#17762)
## Summary

closes https://github.com/astral-sh/ruff/issues/17122

## Test Plan

* New hover tests
* Opened the playground locally and saw that new hover-types are shown
as expected.
2025-05-01 20:33:51 +02:00
Abhijeet Prasad Bodas
0c80c56afc [syntax-errors] Use consistent message for bad starred expression usage. (#17772) 2025-05-01 20:18:35 +02:00
Andrew Gallant
b7ce694162 red_knot_server: add auto-completion MVP
This PR does the wiring necessary to respond to completion requests from
LSP clients.

As far as the actual completion results go, they are nearly about the
dumbest and simplest thing we can do: we simply return a de-duplicated
list of all identifiers from the current module.
2025-05-01 12:08:10 -04:00
Brent Westbrook
163d526407 Allow passing a virtual environment to ruff analyze graph (#17743)
Summary
--

Fixes #16598 by adding the `--python` flag to `ruff analyze graph`,
which adds a `PythonPath` to the `SearchPathSettings` for module
resolution. For the [albatross-virtual-workspace] example from the uv
repo, this updates the output from the initial issue:

```shell
> ruff analyze graph packages/albatross
{
  "packages/albatross/check_installed_albatross.py": [
    "packages/albatross/src/albatross/__init__.py"
  ],
  "packages/albatross/src/albatross/__init__.py": []
}
```

To include both the the workspace `bird_feeder` import _and_ the
third-party `tqdm` import in the output:

```shell
> myruff analyze graph packages/albatross --python .venv
{
  "packages/albatross/check_installed_albatross.py": [
    "packages/albatross/src/albatross/__init__.py"
  ],
  "packages/albatross/src/albatross/__init__.py": [
    ".venv/lib/python3.12/site-packages/tqdm/__init__.py",
    "packages/bird-feeder/src/bird_feeder/__init__.py"
  ]
}
```

Note the hash in the uv link! I was temporarily very confused why my
local tests were showing an `iniconfig` import instead of `tqdm` until I
realized that the example has been updated on the uv main branch, which
I had locally.

Test Plan
--

A new integration test with a stripped down venv based on the
`albatross` example.

[albatross-virtual-workspace]:
aa629c4a54/scripts/workspaces/albatross-virtual-workspace
2025-05-01 11:29:52 -04:00
Brent Westbrook
75effb8ed7 Bump 0.11.8 (#17766) 2025-05-01 10:19:58 -04:00
Victor Hugo Gomes
3353d07938 [flake8-use-pathlib] Fix PTH104false positive when rename is passed a file descriptor (#17712)
## Summary
Contains the same changes to the semantic type inference as
https://github.com/astral-sh/ruff/pull/17705.

Fixes #17694
<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

<!-- How was it tested? -->
Snapshot tests.

---------

Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
Co-authored-by: Brent Westbrook <brentrwestbrook@gmail.com>
2025-05-01 10:01:17 -04:00
Alex Waygood
41f3f21629 Improve messages outputted by py-fuzzer (#17764) 2025-05-01 12:32:45 +00:00
Hans
76ec64d535 [red-knot] Allow subclasses of Any to be assignable to Callable types (#17717)
## Summary

Fixes #17701.

## Test plan

New Markdown test.

---------

Co-authored-by: David Peter <mail@david-peter.de>
2025-05-01 10:18:12 +02:00
Micha Reiser
b7e69ecbfc [red-knot] Increase durability of read-only File fields (#17757) 2025-05-01 09:25:48 +02:00
Micha Reiser
9c57862262 [red-knot] Cache source type during semanic index building (#17756)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-05-01 08:51:53 +02:00
Victor Hugo Gomes
67ef370733 [flake8-use-pathlib] Fix PTH116 false positive when stat is passed a file descriptor (#17709)
Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
2025-05-01 08:16:28 +02:00
github-actions[bot]
e17e1e860b Sync vendored typeshed stubs (#17753)
Co-authored-by: typeshedbot <>
2025-05-01 07:57:03 +02:00
David Peter
03d8679adf [red-knot] Preliminary NamedTuple support (#17738)
## Summary

Adds preliminary support for `NamedTuple`s, including:
* No false positives when constructing a `NamedTuple` object
* Correct signature for the synthesized `__new__` method, i.e. proper
checking of constructor calls
* A patched MRO (`NamedTuple` => `tuple`), mainly to make type inference
of named attributes possible, but also to better reflect the runtime
MRO.

All of this works:
```py
from typing import NamedTuple

class Person(NamedTuple):
    id: int
    name: str
    age: int | None = None

alice = Person(1, "Alice", 42)
alice = Person(id=1, name="Alice", age=42)

reveal_type(alice.id)  # revealed: int
reveal_type(alice.name)  # revealed: str
reveal_type(alice.age)  # revealed: int | None

# error: [missing-argument]
Person(3)

# error: [too-many-positional-arguments]
Person(3, "Eve", 99, "extra")

# error: [invalid-argument-type]
Person(id="3", name="Eve")
```

Not included:
* type inference for index-based access.
* support for the functional `MyTuple = NamedTuple("MyTuple", […])`
syntax

## Test Plan

New Markdown tests

## Ecosystem analysis

```
                          Diagnostic Analysis Report                           
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━┓
┃ Diagnostic ID                     ┃ Severity ┃ Removed ┃ Added ┃ Net Change ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━┩
│ lint:call-non-callable            │ error    │       0 │     3 │         +3 │
│ lint:call-possibly-unbound-method │ warning  │       0 │     4 │         +4 │
│ lint:invalid-argument-type        │ error    │       0 │    72 │        +72 │
│ lint:invalid-context-manager      │ error    │       0 │     2 │         +2 │
│ lint:invalid-return-type          │ error    │       0 │     2 │         +2 │
│ lint:missing-argument             │ error    │       0 │    46 │        +46 │
│ lint:no-matching-overload         │ error    │   19121 │     0 │     -19121 │
│ lint:not-iterable                 │ error    │       0 │     6 │         +6 │
│ lint:possibly-unbound-attribute   │ warning  │      13 │    32 │        +19 │
│ lint:redundant-cast               │ warning  │       0 │     1 │         +1 │
│ lint:unresolved-attribute         │ error    │       0 │    10 │        +10 │
│ lint:unsupported-operator         │ error    │       3 │     9 │         +6 │
│ lint:unused-ignore-comment        │ warning  │      15 │     4 │        -11 │
├───────────────────────────────────┼──────────┼─────────┼───────┼────────────┤
│ TOTAL                             │          │   19152 │   191 │     -18961 │
└───────────────────────────────────┴──────────┴─────────┴───────┴────────────┘

Analysis complete. Found 13 unique diagnostic IDs.
Total diagnostics removed: 19152
Total diagnostics added: 191
Net change: -18961
```

I uploaded the ecosystem full diff (ignoring the 19k
`no-matching-overload` diagnostics)
[here](https://shark.fish/diff-namedtuple.html).

* There are some new `missing-argument` false positives which come from
the fact that named tuples are often created using unpacking as in
`MyNamedTuple(*fields)`, which we do not understand yet.
* There are some new `unresolved-attribute` false positives, because
methods like `_replace` are not available.
* Lots of the `invalid-argument-type` diagnostics look like true
positives

---------

Co-authored-by: Douglas Creager <dcreager@dcreager.net>
2025-04-30 22:52:04 +02:00
Victor Hugo Gomes
d33a503686 [red-knot] Add tests for classes that have incompatible __new__ and __init__ methods (#17747)
Closes #17737
2025-04-30 20:40:16 +00:00
renovate[bot]
650cbdd296 Update dependency vite to v6.2.7 (#17746) 2025-04-30 22:12:03 +02:00
Dhruv Manilawala
d2a238dfad [red-knot] Update call binding to return all matching overloads (#17618)
## Summary

This PR updates the existing overload matching methods to return an
iterator of all the matched overloads instead.

This would be useful once the overload call evaluation algorithm is
implemented which should provide an accurate picture of all the matched
overloads. The return type would then be picked from either the only
matched overload or the first overload from the ones that are matched.

In an earlier version of this PR, it tried to check if using an
intersection of return types from the matched overload would help reduce
the false positives but that's not enough. [This
comment](https://github.com/astral-sh/ruff/pull/17618#issuecomment-2842891696)
keep the ecosystem analysis for that change for prosperity.

> [!NOTE]
>
> The best way to review this PR is by hiding the whitespace changes
because there are two instances where a large match expression is
indented to be inside a loop over matching overlods
>
> <img width="1207" alt="Screenshot 2025-04-28 at 15 12 16"
src="https://github.com/user-attachments/assets/e06cbfa4-04fa-435f-84ef-4e5c3c5626d1"
/>

## Test Plan

Make sure existing test cases are unaffected and no ecosystem changes.
2025-05-01 01:33:21 +05:30
Wei Lee
6e765b4527 [airflow] apply Replacement::AutoImport to AIR312 (#17570)
## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

This is not yet fixing anything as the names are not changed, but it
lays down the foundation for fixing.

## Test Plan

<!-- How was it tested? -->

the existing test fixture should already cover this change
2025-04-30 15:53:10 -04:00
Vasco Schiavo
c5e41c278c [ruff] Add fix safety section (RUF028) (#17722)
The PR add the fix safety section for rule `RUF028`
(https://github.com/astral-sh/ruff/issues/15584 )

See also
[here](https://github.com/astral-sh/ruff/issues/15584#issuecomment-2820424485)
for the reason behind the _unsafe_ of the fix.
2025-04-30 15:06:25 -04:00
Abhijeet Prasad Bodas
0eeb02c0c1 [syntax-errors] Detect single starred expression assignment x = *y (#17624)
## Summary

Part of #17412

Starred expressions cannot be used as values in assignment expressions.
Add a new semantic syntax error to catch such instances.
Note that we already have
`ParseErrorType::InvalidStarredExpressionUsage` to catch some starred
expression errors during parsing, but that does not cover top level
assignment expressions.

## Test Plan

- Added new inline tests for the new rule
- Found some examples marked as "valid" in existing tests (`_ = *data`),
which are not really valid (per this new rule) and updated them
- There was an existing inline test - `assign_stmt_invalid_value_expr`
which had instances of `*` expression which would be deemed invalid by
this new rule. Converted these to tuples, so that they do not trigger
this new rule.
2025-04-30 15:04:00 -04:00
Alex Waygood
f31b1c695c py-fuzzer: fix minimization logic when --only-new-bugs is passed (#17739) 2025-04-30 18:48:31 +01:00
Brendan Cooley
5679bf00bc Fix example syntax for pydocstyle ignore_var_parameters option (#17740)
Co-authored-by: Brendan Cooley <brendanc@ladodgers.com>
2025-04-30 18:19:41 +02:00
Micha Reiser
a7c358ab5c [red-knot] Update salsa to prevent panic in custom panic-handler (#17742) 2025-04-30 18:19:07 +02:00
Alex Waygood
b6de01b9a5 [red-knot] Ban direct instantiation of generic protocols as well as non-generic ones (#17741) 2025-04-30 16:01:28 +00:00
David Peter
18bac94226 [red-knot] Lookup of __new__ (#17733)
## Summary

Model the lookup of `__new__` without going through
`Type::try_call_dunder`. The `__new__` method is only looked up on the
constructed type itself, not on the meta-type.

This now removes ~930 false positives across the ecosystem (vs 255 for
https://github.com/astral-sh/ruff/pull/17662). It introduces 30 new
false positives related to the construction of enums via something like
`Color = enum.Enum("Color", ["RED", "GREEN"])`. This is expected,
because we don't handle custom metaclass `__call__` methods. The fact
that we previously didn't emit diagnostics there was a coincidence (we
incorrectly called `EnumMeta.__new__`, and since we don't fully
understand its signature, that happened to work with `str`, `list`
arguments).

closes #17462

## Test Plan

Regression test
2025-04-30 17:27:09 +02:00
Dhruv Manilawala
7568eeb7a5 [red-knot] Check decorator consistency on overloads (#17684)
## Summary

Part of #15383.

As per the spec
(https://typing.python.org/en/latest/spec/overload.html#invalid-overload-definitions):

For `@staticmethod` and `@classmethod`:

> If one overload signature is decorated with `@staticmethod` or
`@classmethod`, all overload signatures must be similarly decorated. The
implementation, if present, must also have a consistent decorator. Type
checkers should report an error if these conditions are not met.

For `@final` and `@override`:

> If a `@final` or `@override` decorator is supplied for a function with
overloads, the decorator should be applied only to the overload
implementation if it is present. If an overload implementation isn’t
present (for example, in a stub file), the `@final` or `@override`
decorator should be applied only to the first overload. Type checkers
should enforce these rules and generate an error when they are violated.
If a `@final` or `@override` decorator follows these rules, a type
checker should treat the decorator as if it is present on all overloads.

## Test Plan

Update existing tests; add snapshots.
2025-04-30 20:34:21 +05:30
Hans
0e85cbdd91 [flake8-use-pathlib] Avoid suggesting Path.iterdir() for os.listdir with file descriptor (PTH208) (#17715)
## Summary

Fixes: #17695

---------

Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
2025-04-30 20:08:57 +05:30
Dhruv Manilawala
7825975972 [red-knot] Check overloads without an implementation (#17681)
## Summary

As mentioned in the spec
(https://typing.python.org/en/latest/spec/overload.html#invalid-overload-definitions),
part of #15383:

> The `@overload`-decorated definitions must be followed by an overload
implementation, which does not include an `@overload` decorator. Type
checkers should report an error or warning if an implementation is
missing. Overload definitions within stub files, protocols, and on
abstract methods within abstract base classes are exempt from this
check.

## Test Plan

Remove TODOs from the test; create one diagnostic snapshot.
2025-04-30 19:54:21 +05:30
Max Mynter
f584b66824 Expand Semantic Syntax Coverage (#17725)
Re: #17526 

## Summary
Adds tests to red knot and `linter.rs` for the semantic syntax. 

Specifically add tests for `ReboundComprehensionVariable`,
`DuplicateTypeParameter`, and `MultipleCaseAssignment`.

Refactor the `test_async_comprehension_in_sync_comprehension` →
`test_semantic_error` to be more general for all semantic syntax test
cases.

## Test Plan
This is a test.

## Question
I'm happy to contribute more tests the coming days. 

Should that happen here or should we merge this PR such that the
refactor `test_async_comprehension_in_sync_comprehension` →
`test_semantic_error` is available on main and others can chime in, too?
2025-04-30 10:14:08 -04:00
Dhruv Manilawala
ad1a8da4d1 [red-knot] Check for invalid overload usages (#17609)
## Summary

Part of #15383, this PR adds the core infrastructure to check for
invalid overloads and adds a diagnostic to raise if there are < 2
overloads for a given definition.

### Design notes

The requirements to check the overloads are:
* Requires `FunctionType` which has the `to_overloaded` method
* The `FunctionType` **should** be for the function that is either the
implementation or the last overload if the implementation doesn't exists
* Avoid checking any `FunctionType` that are part of an overload chain
* Consider visibility constraints

This required a couple of iteration to make sure all of the above
requirements are fulfilled.

#### 1. Use a set to deduplicate

The logic would first collect all the `FunctionType` that are part of
the overload chain except for the implementation or the last overload if
the implementation doesn't exists. Then, when iterating over all the
function declarations within the scope, we'd avoid checking these
functions. But, this approach would fail to consider visibility
constraints as certain overloads _can_ be behind a version check. Those
aren't part of the overload chain but those aren't a separate overload
chain either.

<details><summary>Implementation:</summary>
<p>

```rs
fn check_overloaded_functions(&mut self) {
    let function_definitions = || {
        self.types
            .declarations
            .iter()
            .filter_map(|(definition, ty)| {
                // Filter out function literals that result from anything other than a function
                // definition e.g., imports.
                if let DefinitionKind::Function(function) = definition.kind(self.db()) {
                    ty.inner_type()
                        .into_function_literal()
                        .map(|ty| (ty, definition.symbol(self.db()), function.node()))
                } else {
                    None
                }
            })
    };

    // A set of all the functions that are part of an overloaded function definition except for
    // the implementation function and the last overload in case the implementation doesn't
    // exists. This allows us to collect all the function definitions that needs to be skipped
    // when checking for invalid overload usages.
    let mut overloads: HashSet<FunctionType<'db>> = HashSet::default();

    for (function, _) in function_definitions() {
        let Some(overloaded) = function.to_overloaded(self.db()) else {
            continue;
        };
        if overloaded.implementation.is_some() {
            overloads.extend(overloaded.overloads.iter().copied());
        } else if let Some((_, previous_overloads)) = overloaded.overloads.split_last() {
            overloads.extend(previous_overloads.iter().copied());
        }
    }

    for (function, function_node) in function_definitions() {
        let Some(overloaded) = function.to_overloaded(self.db()) else {
            continue;
        };
        if overloads.contains(&function) {
            continue;
        }

        // At this point, the `function` variable is either the implementation function or the
        // last overloaded function if the implementation doesn't exists.

        if overloaded.overloads.len() < 2 {
            if let Some(builder) = self
                .context
                .report_lint(&INVALID_OVERLOAD, &function_node.name)
            {
                let mut diagnostic = builder.into_diagnostic(format_args!(
                    "Function `{}` requires at least two overloads",
                    &function_node.name
                ));
                if let Some(first_overload) = overloaded.overloads.first() {
                    diagnostic.annotate(
                        self.context
                            .secondary(first_overload.focus_range(self.db()))
                            .message(format_args!("Only one overload defined here")),
                    );
                }
            }
        }
    }
 }
```

</p>
</details> 

#### 2. Define a `predecessor` query

The `predecessor` query would return the previous `FunctionType` for the
given `FunctionType` i.e., the current logic would be extracted to be a
query instead. This could then be used to make sure that we're checking
the entire overload chain once. The way this would've been implemented
is to have a `to_overloaded` implementation which would take the root of
the overload chain instead of the leaf. But, this would require updates
to the use-def map to somehow be able to return the _following_
functions for a given definition.

#### 3. Create a successor link

This is what Pyrefly uses, we'd create a forward link between two
functions that are involved in an overload chain. This means that for a
given function, we can get the successor function. This could be used to
find the _leaf_ of the overload chain which can then be used with the
`to_overloaded` method to get the entire overload chain. But, this would
also require updating the use-def map to be able to "see" the
_following_ function.

### Implementation 

This leads us to the final implementation that this PR implements which
is to consider the overloaded functions using:
* Collect all the **function symbols** that are defined **and** called
within the same file. This could potentially be an overloaded function
* Use the public bindings to get the leaf of the overload chain and use
that to get the entire overload chain via `to_overloaded` and perform
the check

This has a limitation that in case a function redefines an overload,
then that overload will not be checked. For example:

```py
from typing import overload

@overload
def f() -> None: ...
@overload
def f(x: int) -> int: ...

# The above overload will not be checked as the below function with the same name
# shadows it

def f(*args: int) -> int: ...
```

## Test Plan

Update existing mdtest and add snapshot diagnostics.
2025-04-30 19:37:42 +05:30
Micha Reiser
0861ecfa55 [red-knot] Use 'full' salsa backtrace output that includes durability and revisions (#17735) 2025-04-30 11:04:06 +00:00
Alex Waygood
d1f359afbb [red-knot] Initial support for protocol types (#17682) 2025-04-30 11:03:10 +00:00
Alex Waygood
b84b58760e [red-knot] Computing a type ordering for two non-normalized types is meaningless (#17734) 2025-04-30 11:58:55 +01:00
Micha Reiser
d94be0e780 [red-knot] Include salsa backtrace in check and mdtest panic messages (#17732)
Co-authored-by: David Peter <sharkdp@users.noreply.github.com>
2025-04-30 10:26:40 +02:00
Alex Waygood
8a6787b39e [red-knot] Fix control flow for assert statements (#17702)
## Summary

@sharkdp and I realised in our 1:1 this morning that our control flow
for `assert` statements isn't quite accurate at the moment. Namely, for
something like this:

```py
def _(x: int | None):
    assert x is None, reveal_type(x)
```

we currently reveal `None` for `x` here, but this is incorrect. In
actual fact, the `msg` expression of an `assert` statement (the
expression after the comma) will only be evaluated if the test (`x is
None`) evaluates to `False`. As such, we should be adding a constraint
of `~None` to `x` in the `msg` expression, which should simplify the
inferred type of `x` to `int` in that context (`(int | None) & ~None` ->
`int`).

## Test Plan

Mdtests added.

---------

Co-authored-by: David Peter <mail@david-peter.de>
2025-04-30 09:57:49 +02:00
David Peter
4a621c2c12 [red-knot] Fix recording of negative visibility constraints (#17731)
## Summary

We were previously recording wrong reachability constraints for negative
branches. Instead of `[cond] AND (NOT [True])` below, we were recording
`[cond] AND (NOT ([cond] AND [True]))`, i.e. we were negating not just
the last predicate, but the `AND`-ed reachability constraint from last
clause. With this fix, we now record the correct constraints for the
example from #17723:

```py
def _(cond: bool):
    if cond:
        # reachability: [cond]
        if True:
            # reachability: [cond] AND [True]
            pass
        else:
            # reachability: [cond] AND (NOT [True])
            x
```

closes #17723 

## Test Plan

* Regression test.
* Verified the ecosystem changes
2025-04-30 09:32:13 +02:00
Micha Reiser
2bb99df394 [red-knot] Update salsa (#17730) 2025-04-30 08:58:31 +02:00
Dhruv Manilawala
f11d9cb509 [red-knot] Support overloads for callable equivalence (#17698)
## Summary

Part of #15383, this PR adds `is_equivalent_to` support for overloaded
callables.

This is mainly done by delegating it to the subtyping check in that two
types A and B are considered equivalent if A is a subtype of B and B is
a subtype of A.

## Test Plan

Add test cases for overloaded callables in `is_equivalent_to.md`
2025-04-30 02:53:59 +05:30
Alex Waygood
549ab74bd6 [red-knot] Run py-fuzzer in CI to check for new panics (#17719) 2025-04-29 21:19:29 +00:00
Alex Waygood
81fc7d7d3a Upload red-knot binaries in CI on completion of linux tests (#17720) 2025-04-29 22:15:26 +01:00
Victor Hugo Gomes
8c68d30c3a [flake8-use-pathlib] Fix PTH123 false positive when open is passed a file descriptor from a function call (#17705)
## Summary
Includes minor changes to the semantic type inference to help detect the
return type of function call.

Fixes #17691

## Test Plan

Snapshot tests
2025-04-29 16:51:38 -04:00
Alex Waygood
93d6a3567b [red-knot] mdtest.py: Watch for changes in red_knot_vendored and red_knot_test as well as in red_knot_python_semantic (#17718) 2025-04-29 18:27:49 +00:00
Micha Reiser
1d788981cd [red-knot] Capture backtrace in "check-failed" diagnostic (#17641)
Co-authored-by: David Peter <sharkdp@users.noreply.github.com>
2025-04-29 16:58:58 +00:00
Hans
7d46579808 [docs] fix duplicated 'are' in comment for PTH123 rule (#17714) 2025-04-29 17:58:39 +02:00
Alex Waygood
c9a6b1a9d0 [red-knot] Make Type::signatures() exhaustive (#17706) 2025-04-29 15:14:08 +01:00
Hans
9b9d16c3ba [red-knot] colorize concise output diagnostics (#17232) (#17479)
Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Andrew Gallant <andrew@astral.sh>
2025-04-29 16:07:16 +02:00
David Peter
79f8473e51 [red-knot] Assignability of class literals to Callables (#17704)
## Summary

Subtyping was already modeled, but assignability also needs an explicit
branch. Removes 921 ecosystem false positives.

## Test Plan

New Markdown tests.
2025-04-29 15:04:22 +02:00
Douglas Creager
ca4fdf452d Create TypeVarInstance type for legacy typevars (#16538)
We are currently representing type variables using a `KnownInstance`
variant, which wraps a `TypeVarInstance` that contains the information
about the typevar (name, bounds, constraints, default type). We were
previously only constructing that type for PEP 695 typevars. This PR
constructs that type for legacy typevars as well.

It also detects functions that are generic because they use legacy
typevars in their parameter list. With the existing logic for inferring
specializations of function calls (#17301), that means that we are
correctly detecting that the definition of `reveal_type` in the typeshed
is generic, and inferring the correct specialization of `_T` for each
call site.

This does not yet handle legacy generic classes; that will come in a
follow-on PR.
2025-04-29 09:03:06 -04:00
Dylan
3c460a7b9a Make syntax error for unparenthesized except tuples version specific to before 3.14 (#17660)
What it says on the tin 😄
2025-04-29 07:55:30 -05:00
Alex Waygood
31e6576971 [red-knot] micro-optimise ClassLiteral::is_protocol (#17703) 2025-04-29 12:35:53 +00:00
Micha Reiser
c953e7d143 [red-knot] Improve log message for default python platform (#17700) 2025-04-29 08:26:41 +00:00
Vasco Schiavo
5096824793 [ruff] add fix safety section (RUF017) (#17480)
The PR add the `fix safety` section for rule `RUF017` (#15584 )
2025-04-28 22:07:22 +00:00
Dylan
ae7691b026 Add Python 3.14 to configuration options (#17647)
A small PR that just updates the various settings/configurations to
allow Python 3.14. At the moment selecting that target version will
have no impact compared to Python 3.13 - except that a warning
is emitted if the user does so with `preview` disabled.
2025-04-28 16:29:00 -05:00
Wei Lee
504fa20057 [airflow] Apply auto fixes to cases where the names have changed in Airflow 3 (AIR302) (#17553)
## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

Apply auto fixes to cases where the names have changed in Airflow 3 in
AIR302 and split the huge test cases into different test cases based on
proivder

## Test Plan

<!-- How was it tested? -->

the test cases has been split into multiple for easier checking
2025-04-28 16:35:17 -04:00
David Peter
f0868ac0c9 [red-knot] Revert blanket clippy::too_many_arguments allow (#17688)
## Summary

Now that https://github.com/salsa-rs/salsa/issues/808 has been fixed, we
can revert this global change in `Cargo.toml`.
2025-04-28 21:21:53 +02:00
Brent Westbrook
01a31c08f5 Add config option to disable typing_extensions imports (#17611)
Summary
--

This PR resolves https://github.com/astral-sh/ruff/issues/9761 by adding
a linter configuration option to disable
`typing_extensions` imports. As mentioned [here], it would be ideal if
we could
detect whether or not `typing_extensions` is available as a dependency
automatically, but this seems like a much easier fix in the meantime.

The default for the new option, `typing-extensions`, is `true`,
preserving the current behavior. Setting it to `false` will bail out of
the new
`Checker::typing_importer` method, which has been refactored from the 
`Checker::import_from_typing` method in
https://github.com/astral-sh/ruff/pull/17340),
with `None`, which is then handled specially by each rule that calls it.

I considered some alternatives to a config option, such as checking if
`typing_extensions` has been imported or checking for a `TYPE_CHECKING`
block we could use, but I think defaulting to allowing
`typing_extensions` imports and allowing the user to disable this with
an option is both simple to implement and pretty intuitive.

[here]:
https://github.com/astral-sh/ruff/issues/9761#issuecomment-2790492853

Test Plan
--

New linter tests exercising several combinations of Python versions and
the new config option for PYI019. I also added tests for the other
affected rules, but only in the case where the new config option is
enabled. The rules' existing tests also cover the default case.
2025-04-28 14:57:36 -04:00
Andrew Gallant
405878a128 ruff_db: render file paths in diagnostics as relative paths if possible
This is done in what appears to be the same way as Ruff: we get the CWD,
strip the prefix from the path if possible, and use that. If stripping
the prefix fails, then we print the full path as-is.

Fixes #17233
2025-04-28 14:32:34 -04:00
Alex Waygood
80103a179d Bump mypy_primer pin (#17685) 2025-04-28 16:13:07 +00:00
Andrew Gallant
9a8f3cf247 red_knot_python_semantic: improve not-iterable diagnostic
This cleans up one particular TODO by splitting the "because" part of
the `not-iterable` diagnostic out into an info sub-diagnostic.
2025-04-28 11:03:41 -04:00
David Peter
07718f4788 [red-knot] Allow all callables to be assignable to @Todo-signatures (#17680)
## Summary

Removes ~850 diagnostics related to assignability of callable types,
where the callable-being-assigned-to has a "Todo signature", which
should probably accept any left hand side callable/signature.
2025-04-28 16:40:35 +02:00
Dylan
1e8881f9af [refurb] Mark fix as safe for readlines-in-for (FURB129) (#17644)
This PR promotes the fix applicability of [readlines-in-for
(FURB129)](https://docs.astral.sh/ruff/rules/readlines-in-for/#readlines-in-for-furb129)
to always safe.

In the original PR (https://github.com/astral-sh/ruff/pull/9880), the
author marked the rule as unsafe because Ruff's type inference couldn't
quite guarantee that we had an `IOBase` object in hand. Some false
positives were recorded in the test fixture. However, before the PR was
merged, Charlie added the necessary type inference and the false
positives went away.

According to the [Python
documentation](https://docs.python.org/3/library/io.html#io.IOBase), I
believe this fix is safe for any proper implementation of `IOBase`:

>[IOBase](https://docs.python.org/3/library/io.html#io.IOBase) (and its
subclasses) supports the iterator protocol, meaning that an
[IOBase](https://docs.python.org/3/library/io.html#io.IOBase) object can
be iterated over yielding the lines in a stream. Lines are defined
slightly differently depending on whether the stream is a binary stream
(yielding bytes), or a text stream (yielding character strings). See
[readline()](https://docs.python.org/3/library/io.html#io.IOBase.readline)
below.

and then in the [documentation for
`readlines`](https://docs.python.org/3/library/io.html#io.IOBase.readlines):

>Read and return a list of lines from the stream. hint can be specified
to control the number of lines read: no more lines will be read if the
total size (in bytes/characters) of all lines so far exceeds hint. [...]
>Note that it’s already possible to iterate on file objects using for
line in file: ... without calling file.readlines().

I believe that a careful reading of our [versioning
policy](https://docs.astral.sh/ruff/versioning/#version-changes)
requires that this change be deferred to a minor release - but please
correct me if I'm wrong!
2025-04-28 09:39:55 -05:00
Dylan
152a0b6585 Collect preview lint behaviors in separate module (#17646)
This PR collects all behavior gated under preview into a new module
`ruff_linter::preview` that exposes functions like
`is_my_new_feature_enabled` - just as is done in the formatter crate.
2025-04-28 09:12:24 -05:00
Alex Waygood
1ad5015e19 Upgrade Salsa to a more recent commit (#17678) 2025-04-28 13:32:19 +01:00
David Peter
92f95ff494 [red-knot] TypedDict: No errors for introspection dunder attributes (#17677)
## Summary

Do not emit errors when accessing introspection dunder attributes such
as `__required_keys__` on `TypedDict`s.
2025-04-28 13:28:43 +02:00
Victor Hugo Gomes
ceb2bf1168 [flake8-pyi] Ensure Literal[None,] | Literal[None,] is not autofixed to None | None (PYI061) (#17659)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-04-28 12:23:29 +01:00
David Peter
f521358033 [red-knot] No errors for definitions of TypedDicts (#17674)
## Summary

Do not emit errors when defining `TypedDict`s:

```py
from typing_extensions import TypedDict

# No error here
class Person(TypedDict):
    name: str
    age: int | None

# No error for this alternative syntax
Message = TypedDict("Message", {"id": int, "content": str})
```

## Ecosystem analysis

* Removes ~ 450 false positives for `TypedDict` definitions.
* Changes a few diagnostic messages.
* Adds a few (< 10) false positives, for example:
  ```diff
+ error[lint:unresolved-attribute]
/tmp/mypy_primer/projects/hydra-zen/src/hydra_zen/structured_configs/_utils.py:262:5:
Type `Literal[DataclassOptions]` has no attribute `__required_keys__`
+ error[lint:unresolved-attribute]
/tmp/mypy_primer/projects/hydra-zen/src/hydra_zen/structured_configs/_utils.py:262:42:
Type `Literal[DataclassOptions]` has no attribute `__optional_keys__`
  ```
* New true positive

4f8263cd7f/corporate/lib/remote_billing_util.py (L155-L157)
  ```diff
+ error[lint:invalid-assignment]
/tmp/mypy_primer/projects/zulip/corporate/lib/remote_billing_util.py:155:5:
Object of type `RemoteBillingIdentityDict | LegacyServerIdentityDict |
None` is not assignable to `LegacyServerIdentityDict | None`
  ```

## Test Plan

New Markdown tests
2025-04-28 13:13:28 +02:00
renovate[bot]
74081032d9 Update actions/download-artifact digest to d3f86a1 (#17664)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-04-28 10:51:59 +00:00
Micha Reiser
dbc137c951 [red-knot] Use 101 exit code when there's at least one diagnostic with severity 'fatal' (#17640) 2025-04-28 10:03:14 +02:00
Victor Hugo Gomes
826b2c9ff3 [pycodestyle] Fix duplicated diagnostic in E712 (#17651) 2025-04-28 08:31:16 +01:00
jie211
a3e55cfd8f [airflow] fix typos AIR312 (#17673) 2025-04-28 08:31:41 +02:00
justin
d2246278e6 [red-knot] Don't ignore hidden files by default (#17655) 2025-04-28 08:21:11 +02:00
renovate[bot]
6bd1863bf0 Update pre-commit hook astral-sh/ruff-pre-commit to v0.11.7 (#17670)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-04-28 08:16:10 +02:00
renovate[bot]
97dc58fc77 Update docker/build-push-action digest to 14487ce (#17665)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-04-28 08:15:54 +02:00
renovate[bot]
53a9448fb5 Update taiki-e/install-action digest to ab3728c (#17666)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-04-28 08:15:01 +02:00
renovate[bot]
516291b693 Update dependency react-resizable-panels to v2.1.9 (#17667)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-04-28 08:10:24 +02:00
renovate[bot]
b09f00a4ef Update dependency ruff to v0.11.7 (#17668)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-04-28 08:09:50 +02:00
renovate[bot]
03065c245c Update dependency smol-toml to v1.3.4 (#17669)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-04-28 08:09:33 +02:00
renovate[bot]
b45598389d Update Rust crate jiff to v0.2.10 (#17671)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-04-28 08:09:10 +02:00
renovate[bot]
4729ff2bc8 Update Rust crate syn to v2.0.101 (#17672)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-04-28 08:08:41 +02:00
1943 changed files with 95834 additions and 10796 deletions

10
.github/CODEOWNERS vendored
View File

@@ -14,11 +14,11 @@
# flake8-pyi
/crates/ruff_linter/src/rules/flake8_pyi/ @AlexWaygood
# Script for fuzzing the parser/red-knot etc.
# Script for fuzzing the parser/ty etc.
/python/py-fuzzer/ @AlexWaygood
# red-knot
/crates/red_knot* @carljm @MichaReiser @AlexWaygood @sharkdp @dcreager
# ty
/crates/ty* @carljm @MichaReiser @AlexWaygood @sharkdp @dcreager
/crates/ruff_db/ @carljm @MichaReiser @AlexWaygood @sharkdp @dcreager
/scripts/knot_benchmark/ @carljm @MichaReiser @AlexWaygood @sharkdp @dcreager
/crates/red_knot_python_semantic @carljm @AlexWaygood @sharkdp @dcreager
/scripts/ty_benchmark/ @carljm @MichaReiser @AlexWaygood @sharkdp @dcreager
/crates/ty_python_semantic @carljm @AlexWaygood @sharkdp @dcreager

View File

@@ -1,5 +1,8 @@
blank_issues_enabled: true
contact_links:
- name: Report an issue with ty
url: https://github.com/astral-sh/ty/issues/new/choose
about: Please report issues for our type checker ty in the ty repository.
- name: Documentation
url: https://docs.astral.sh/ruff
about: Please consult the documentation before creating an issue.

View File

@@ -43,7 +43,7 @@ jobs:
with:
submodules: recursive
persist-credentials: false
- uses: actions/setup-python@8d9ed9ac5c53483de85588cdf95a591a75ab9f55 # v5.5.0
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: "Prep README.md"
@@ -72,7 +72,7 @@ jobs:
with:
submodules: recursive
persist-credentials: false
- uses: actions/setup-python@8d9ed9ac5c53483de85588cdf95a591a75ab9f55 # v5.5.0
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: ${{ env.PYTHON_VERSION }}
architecture: x64
@@ -114,7 +114,7 @@ jobs:
with:
submodules: recursive
persist-credentials: false
- uses: actions/setup-python@8d9ed9ac5c53483de85588cdf95a591a75ab9f55 # v5.5.0
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: ${{ env.PYTHON_VERSION }}
architecture: arm64
@@ -170,7 +170,7 @@ jobs:
with:
submodules: recursive
persist-credentials: false
- uses: actions/setup-python@8d9ed9ac5c53483de85588cdf95a591a75ab9f55 # v5.5.0
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: ${{ env.PYTHON_VERSION }}
architecture: ${{ matrix.platform.arch }}
@@ -223,7 +223,7 @@ jobs:
with:
submodules: recursive
persist-credentials: false
- uses: actions/setup-python@8d9ed9ac5c53483de85588cdf95a591a75ab9f55 # v5.5.0
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: ${{ env.PYTHON_VERSION }}
architecture: x64
@@ -298,7 +298,7 @@ jobs:
with:
submodules: recursive
persist-credentials: false
- uses: actions/setup-python@8d9ed9ac5c53483de85588cdf95a591a75ab9f55 # v5.5.0
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: "Prep README.md"
@@ -363,7 +363,7 @@ jobs:
with:
submodules: recursive
persist-credentials: false
- uses: actions/setup-python@8d9ed9ac5c53483de85588cdf95a591a75ab9f55 # v5.5.0
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: ${{ env.PYTHON_VERSION }}
architecture: x64
@@ -429,7 +429,7 @@ jobs:
with:
submodules: recursive
persist-credentials: false
- uses: actions/setup-python@8d9ed9ac5c53483de85588cdf95a591a75ab9f55 # v5.5.0
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: "Prep README.md"

View File

@@ -79,7 +79,7 @@ jobs:
# Adapted from https://docs.docker.com/build/ci/github-actions/multi-platform/
- name: Build and push by digest
id: build
uses: docker/build-push-action@471d1dc4e07e5cdedd4c2171150001c434f0b7a4 # v6
uses: docker/build-push-action@14487ce63c7a62a4a324b0bfb37086795e31c6c1 # v6
with:
context: .
platforms: ${{ matrix.platform }}
@@ -113,7 +113,7 @@ jobs:
if: ${{ inputs.plan != '' && !fromJson(inputs.plan).announcement_tag_is_implicit }}
steps:
- name: Download digests
uses: actions/download-artifact@95815c38cf2ff2164869cbab79da8d1f422bc89e # v4.2.1
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
path: /tmp/digests
pattern: digests-*
@@ -231,7 +231,7 @@ jobs:
${{ env.TAG_PATTERNS }}
- name: Build and push
uses: docker/build-push-action@471d1dc4e07e5cdedd4c2171150001c434f0b7a4 # v6
uses: docker/build-push-action@14487ce63c7a62a4a324b0bfb37086795e31c6c1 # v6
with:
context: .
platforms: linux/amd64,linux/arm64
@@ -256,7 +256,7 @@ jobs:
if: ${{ inputs.plan != '' && !fromJson(inputs.plan).announcement_tag_is_implicit }}
steps:
- name: Download digests
uses: actions/download-artifact@95815c38cf2ff2164869cbab79da8d1f422bc89e # v4.2.1
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
path: /tmp/digests
pattern: digests-*

View File

@@ -36,8 +36,8 @@ jobs:
code: ${{ steps.check_code.outputs.changed }}
# Flag that is raised when any code that affects the fuzzer is changed
fuzz: ${{ steps.check_fuzzer.outputs.changed }}
# Flag that is set to "true" when code related to red-knot changes.
red_knot: ${{ steps.check_red_knot.outputs.changed }}
# Flag that is set to "true" when code related to ty changes.
ty: ${{ steps.check_ty.outputs.changed }}
# Flag that is set to "true" when code related to the playground changes.
playground: ${{ steps.check_playground.outputs.changed }}
@@ -84,7 +84,7 @@ jobs:
if git diff --quiet "${MERGE_BASE}...HEAD" -- ':Cargo.toml' \
':Cargo.lock' \
':crates/**' \
':!crates/red_knot*/**' \
':!crates/ty*/**' \
':!crates/ruff_python_formatter/**' \
':!crates/ruff_formatter/**' \
':!crates/ruff_dev/**' \
@@ -145,7 +145,7 @@ jobs:
run: |
if git diff --quiet "${MERGE_BASE}...HEAD" -- ':**' \
':!**/*.md' \
':crates/red_knot_python_semantic/resources/mdtest/**/*.md' \
':crates/ty_python_semantic/resources/mdtest/**/*.md' \
':!docs/**' \
':!assets/**' \
':.github/workflows/ci.yaml' \
@@ -168,15 +168,15 @@ jobs:
echo "changed=true" >> "$GITHUB_OUTPUT"
fi
- name: Check if the red-knot code changed
id: check_red_knot
- name: Check if the ty code changed
id: check_ty
env:
MERGE_BASE: ${{ steps.merge_base.outputs.sha }}
run: |
if git diff --quiet "${MERGE_BASE}...HEAD" -- \
':Cargo.toml' \
':Cargo.lock' \
':crates/red_knot*/**' \
':crates/ty*/**' \
':crates/ruff_db/**' \
':crates/ruff_annotate_snippets/**' \
':crates/ruff_python_ast/**' \
@@ -221,7 +221,7 @@ jobs:
- name: "Clippy"
run: cargo clippy --workspace --all-targets --all-features --locked -- -D warnings
- name: "Clippy (wasm)"
run: cargo clippy -p ruff_wasm -p red_knot_wasm --target wasm32-unknown-unknown --all-features --locked -- -D warnings
run: cargo clippy -p ruff_wasm -p ty_wasm --target wasm32-unknown-unknown --all-features --locked -- -D warnings
cargo-test-linux:
name: "cargo test (linux)"
@@ -239,21 +239,21 @@ jobs:
- name: "Install mold"
uses: rui314/setup-mold@e16410e7f8d9e167b74ad5697a9089a35126eb50 # v1
- name: "Install cargo nextest"
uses: taiki-e/install-action@09dc018eee06ae1c9e0409786563f534210ceb83 # v2
uses: taiki-e/install-action@86c23eed46c17b80677df6d8151545ce3e236c61 # v2
with:
tool: cargo-nextest
- name: "Install cargo insta"
uses: taiki-e/install-action@09dc018eee06ae1c9e0409786563f534210ceb83 # v2
uses: taiki-e/install-action@86c23eed46c17b80677df6d8151545ce3e236c61 # v2
with:
tool: cargo-insta
- name: Red-knot mdtests (GitHub annotations)
if: ${{ needs.determine_changes.outputs.red_knot == 'true' }}
- name: ty mdtests (GitHub annotations)
if: ${{ needs.determine_changes.outputs.ty == 'true' }}
env:
NO_COLOR: 1
MDTEST_GITHUB_ANNOTATIONS_FORMAT: 1
# Ignore errors if this step fails; we want to continue to later steps in the workflow anyway.
# This step is just to get nice GitHub annotations on the PR diff in the files-changed tab.
run: cargo test -p red_knot_python_semantic --test mdtest || true
run: cargo test -p ty_python_semantic --test mdtest || true
- name: "Run tests"
shell: bash
env:
@@ -268,7 +268,7 @@ jobs:
# sync, not just public items. Eventually we should do this for all
# crates; for now add crates here as they are warning-clean to prevent
# regression.
- run: cargo doc --no-deps -p red_knot_python_semantic -p red_knot -p red_knot_test -p ruff_db --document-private-items
- run: cargo doc --no-deps -p ty_python_semantic -p ty -p ty_test -p ruff_db --document-private-items
env:
# Setting RUSTDOCFLAGS because `cargo doc --check` isn't yet implemented (https://github.com/rust-lang/cargo/issues/10025).
RUSTDOCFLAGS: "-D warnings"
@@ -276,6 +276,10 @@ jobs:
with:
name: ruff
path: target/debug/ruff
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: ty
path: target/debug/ty
cargo-test-linux-release:
name: "cargo test (linux, release)"
@@ -293,11 +297,11 @@ jobs:
- name: "Install mold"
uses: rui314/setup-mold@e16410e7f8d9e167b74ad5697a9089a35126eb50 # v1
- name: "Install cargo nextest"
uses: taiki-e/install-action@09dc018eee06ae1c9e0409786563f534210ceb83 # v2
uses: taiki-e/install-action@86c23eed46c17b80677df6d8151545ce3e236c61 # v2
with:
tool: cargo-nextest
- name: "Install cargo insta"
uses: taiki-e/install-action@09dc018eee06ae1c9e0409786563f534210ceb83 # v2
uses: taiki-e/install-action@86c23eed46c17b80677df6d8151545ce3e236c61 # v2
with:
tool: cargo-insta
- name: "Run tests"
@@ -320,7 +324,7 @@ jobs:
- name: "Install Rust toolchain"
run: rustup show
- name: "Install cargo nextest"
uses: taiki-e/install-action@09dc018eee06ae1c9e0409786563f534210ceb83 # v2
uses: taiki-e/install-action@86c23eed46c17b80677df6d8151545ce3e236c61 # v2
with:
tool: cargo-nextest
- name: "Run tests"
@@ -358,9 +362,9 @@ jobs:
run: |
cd crates/ruff_wasm
wasm-pack test --node
- name: "Test red_knot_wasm"
- name: "Test ty_wasm"
run: |
cd crates/red_knot_wasm
cd crates/ty_wasm
wasm-pack test --node
cargo-build-release:
@@ -403,11 +407,11 @@ jobs:
- name: "Install mold"
uses: rui314/setup-mold@e16410e7f8d9e167b74ad5697a9089a35126eb50 # v1
- name: "Install cargo nextest"
uses: taiki-e/install-action@09dc018eee06ae1c9e0409786563f534210ceb83 # v2
uses: taiki-e/install-action@86c23eed46c17b80677df6d8151545ce3e236c61 # v2
with:
tool: cargo-nextest
- name: "Install cargo insta"
uses: taiki-e/install-action@09dc018eee06ae1c9e0409786563f534210ceb83 # v2
uses: taiki-e/install-action@86c23eed46c17b80677df6d8151545ce3e236c61 # v2
with:
tool: cargo-insta
- name: "Run tests"
@@ -456,7 +460,7 @@ jobs:
with:
persist-credentials: false
- uses: astral-sh/setup-uv@d4b2f3b6ecc6e67c4457f6d3e41ec42d3d0fcb86 # v5.4.2
- uses: actions/download-artifact@95815c38cf2ff2164869cbab79da8d1f422bc89e # v4.2.1
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
name: Download Ruff binary to test
id: download-cached-binary
with:
@@ -521,11 +525,11 @@ jobs:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- uses: actions/setup-python@8d9ed9ac5c53483de85588cdf95a591a75ab9f55 # v5.5.0
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: ${{ env.PYTHON_VERSION }}
- uses: actions/download-artifact@95815c38cf2ff2164869cbab79da8d1f422bc89e # v4.2.1
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
name: Download comparison Ruff binary
id: ruff-target
with:
@@ -632,6 +636,53 @@ jobs:
name: ecosystem-result
path: ecosystem-result
fuzz-ty:
name: "Fuzz for new ty panics"
runs-on: depot-ubuntu-22.04-16
needs:
- cargo-test-linux
- determine_changes
# Only runs on pull requests, since that is the only we way we can find the base version for comparison.
if: ${{ !contains(github.event.pull_request.labels.*.name, 'no-test') && github.event_name == 'pull_request' && needs.determine_changes.outputs.ty == 'true' }}
timeout-minutes: 20
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
name: Download new ty binary
id: ty-new
with:
name: ty
path: target/debug
- uses: dawidd6/action-download-artifact@20319c5641d495c8a52e688b7dc5fada6c3a9fbc # v8
name: Download baseline ty binary
with:
name: ty
branch: ${{ github.event.pull_request.base.ref }}
workflow: "ci.yaml"
check_artifacts: true
- uses: astral-sh/setup-uv@d4b2f3b6ecc6e67c4457f6d3e41ec42d3d0fcb86 # v5.4.2
- name: Fuzz
env:
FORCE_COLOR: 1
NEW_TY: ${{ steps.ty-new.outputs.download-path }}
run: |
# Make executable, since artifact download doesn't preserve this
chmod +x "${PWD}/ty" "${NEW_TY}/ty"
(
uvx \
--python="${PYTHON_VERSION}" \
--from=./python/py-fuzzer \
fuzz \
--test-executable="${NEW_TY}/ty" \
--baseline-executable="${PWD}/ty" \
--only-new-bugs \
--bin=ty \
0-500
)
cargo-shear:
name: "cargo shear"
runs-on: ubuntu-latest
@@ -654,7 +705,7 @@ jobs:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- uses: actions/setup-python@8d9ed9ac5c53483de85588cdf95a591a75ab9f55 # v5.5.0
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: ${{ env.PYTHON_VERSION }}
architecture: x64
@@ -708,7 +759,7 @@ jobs:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- uses: actions/setup-python@8d9ed9ac5c53483de85588cdf95a591a75ab9f55 # v5.5.0
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: "3.13"
- uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
@@ -779,12 +830,12 @@ jobs:
persist-credentials: false
repository: "astral-sh/ruff-lsp"
- uses: actions/setup-python@8d9ed9ac5c53483de85588cdf95a591a75ab9f55 # v5.5.0
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
# installation fails on 3.13 and newer
python-version: "3.12"
- uses: actions/download-artifact@95815c38cf2ff2164869cbab79da8d1f422bc89e # v4.2.1
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
name: Download development ruff binary
id: ruff-target
with:
@@ -857,7 +908,7 @@ jobs:
run: rustup show
- name: "Install codspeed"
uses: taiki-e/install-action@09dc018eee06ae1c9e0409786563f534210ceb83 # v2
uses: taiki-e/install-action@86c23eed46c17b80677df6d8151545ce3e236c61 # v2
with:
tool: cargo-codspeed

View File

@@ -38,17 +38,17 @@ jobs:
- name: "Install mold"
uses: rui314/setup-mold@e16410e7f8d9e167b74ad5697a9089a35126eb50 # v1
- uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
- name: Build Red Knot
- name: Build ty
# A release build takes longer (2 min vs 1 min), but the property tests run much faster in release
# mode (1.5 min vs 14 min), so the overall time is shorter with a release build.
run: cargo build --locked --release --package red_knot_python_semantic --tests
run: cargo build --locked --release --package ty_python_semantic --tests
- name: Run property tests
shell: bash
run: |
export QUICKCHECK_TESTS=100000
for _ in {1..5}; do
cargo test --locked --release --package red_knot_python_semantic -- --ignored list::property_tests
cargo test --locked --release --package red_knot_python_semantic -- --ignored types::property_tests::stable
cargo test --locked --release --package ty_python_semantic -- --ignored list::property_tests
cargo test --locked --release --package ty_python_semantic -- --ignored types::property_tests::stable
done
create-issue-on-failure:
@@ -68,5 +68,5 @@ jobs:
repo: "ruff",
title: `Daily property test run failed on ${new Date().toDateString()}`,
body: "Run listed here: https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}",
labels: ["bug", "red-knot", "testing"],
labels: ["bug", "ty", "testing"],
})

View File

@@ -5,7 +5,7 @@ permissions: {}
on:
pull_request:
paths:
- "crates/red_knot*/**"
- "crates/ty*/**"
- "crates/ruff_db"
- "crates/ruff_python_ast"
- "crates/ruff_python_parser"
@@ -41,19 +41,16 @@ jobs:
- uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
with:
workspaces: "ruff"
- name: Install Rust toolchain
run: rustup show
- name: Install mypy_primer
run: |
uv tool install "git+https://github.com/hauntsaninja/mypy_primer@4c22d192a456e27badf85b3ea0f830707375d2b7"
- name: Run mypy_primer
shell: bash
run: |
cd ruff
PRIMER_SELECTOR="$(paste -s -d'|' crates/red_knot_python_semantic/resources/primer/good.txt)"
PRIMER_SELECTOR="$(paste -s -d'|' crates/ty_python_semantic/resources/primer/good.txt)"
echo "new commit"
git rev-list --format=%s --max-count=1 "$GITHUB_SHA"
@@ -67,9 +64,11 @@ jobs:
echo "Project selector: $PRIMER_SELECTOR"
# Allow the exit code to be 0 or 1, only fail for actual mypy_primer crashes/bugs
uvx mypy_primer \
uvx \
--from="git+https://github.com/hauntsaninja/mypy_primer@4b15cf3b07db69db67bbfaebfffb2a8a28040933" \
mypy_primer \
--repo ruff \
--type-checker knot \
--type-checker ty \
--old base_commit \
--new "$GITHUB_SHA" \
--project-selector "/($PRIMER_SELECTOR)\$" \

View File

@@ -28,7 +28,7 @@ jobs:
ref: ${{ inputs.ref }}
persist-credentials: true
- uses: actions/setup-python@8d9ed9ac5c53483de85588cdf95a591a75ab9f55 # v5.5.0
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: 3.12

View File

@@ -23,7 +23,7 @@ jobs:
steps:
- name: "Install uv"
uses: astral-sh/setup-uv@d4b2f3b6ecc6e67c4457f6d3e41ec42d3d0fcb86 # v5.4.2
- uses: actions/download-artifact@95815c38cf2ff2164869cbab79da8d1f422bc89e # v4.2.1
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
pattern: wheels-*
path: wheels

View File

@@ -1,5 +1,5 @@
# Publish the Red Knot playground.
name: "[Knot Playground] Release"
# Publish the ty playground.
name: "[ty Playground] Release"
permissions: {}
@@ -7,12 +7,12 @@ on:
push:
branches: [main]
paths:
- "crates/red_knot*/**"
- "crates/ty*/**"
- "crates/ruff_db/**"
- "crates/ruff_python_ast/**"
- "crates/ruff_python_parser/**"
- "playground/**"
- ".github/workflows/publish-knot-playground.yml"
- ".github/workflows/publish-ty-playground.yml"
concurrency:
group: ${{ github.workflow }}-${{ github.ref_name }}
@@ -45,8 +45,8 @@ jobs:
- name: "Run TypeScript checks"
run: npm run check
working-directory: playground
- name: "Build Knot playground"
run: npm run build --workspace knot-playground
- name: "Build ty playground"
run: npm run build --workspace ty-playground
working-directory: playground
- name: "Deploy to Cloudflare Pages"
if: ${{ env.CF_API_TOKEN_EXISTS == 'true' }}
@@ -55,4 +55,4 @@ jobs:
apiToken: ${{ secrets.CF_API_TOKEN }}
accountId: ${{ secrets.CF_ACCOUNT_ID }}
# `github.head_ref` is only set during pull requests and for manual runs or tags we use `main` to deploy to production
command: pages deploy playground/knot/dist --project-name=knot-playground --branch ${{ github.head_ref || 'main' }} --commit-hash ${GITHUB_SHA}
command: pages deploy playground/ty/dist --project-name=ty-playground --branch ${{ github.head_ref || 'main' }} --commit-hash ${GITHUB_SHA}

View File

@@ -69,7 +69,7 @@ jobs:
# we specify bash to get pipefail; it guards against the `curl` command
# failing. otherwise `sh` won't catch that `curl` returned non-0
shell: bash
run: "curl --proto '=https' --tlsv1.2 -LsSf https://github.com/astral-sh/cargo-dist/releases/download/v0.28.4/cargo-dist-installer.sh | sh"
run: "curl --proto '=https' --tlsv1.2 -LsSf https://github.com/astral-sh/cargo-dist/releases/download/v0.28.5-prerelease.1/cargo-dist-installer.sh | sh"
- name: Cache dist
uses: actions/upload-artifact@6027e3dd177782cd8ab9af838c04fd81a07f1d47
with:
@@ -129,14 +129,14 @@ jobs:
persist-credentials: false
submodules: recursive
- name: Install cached dist
uses: actions/download-artifact@95815c38cf2ff2164869cbab79da8d1f422bc89e
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093
with:
name: cargo-dist-cache
path: ~/.cargo/bin/
- run: chmod +x ~/.cargo/bin/dist
# Get all the local artifacts for the global tasks to use (for e.g. checksums)
- name: Fetch local artifacts
uses: actions/download-artifact@95815c38cf2ff2164869cbab79da8d1f422bc89e
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093
with:
pattern: artifacts-*
path: target/distrib/
@@ -180,14 +180,14 @@ jobs:
persist-credentials: false
submodules: recursive
- name: Install cached dist
uses: actions/download-artifact@95815c38cf2ff2164869cbab79da8d1f422bc89e
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093
with:
name: cargo-dist-cache
path: ~/.cargo/bin/
- run: chmod +x ~/.cargo/bin/dist
# Fetch artifacts from scratch-storage
- name: Fetch artifacts
uses: actions/download-artifact@95815c38cf2ff2164869cbab79da8d1f422bc89e
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093
with:
pattern: artifacts-*
path: target/distrib/
@@ -257,7 +257,7 @@ jobs:
submodules: recursive
# Create a GitHub Release while uploading all files to it
- name: "Download GitHub Artifacts"
uses: actions/download-artifact@95815c38cf2ff2164869cbab79da8d1f422bc89e
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093
with:
pattern: artifacts-*
path: artifacts

View File

@@ -39,13 +39,13 @@ jobs:
- name: Sync typeshed
id: sync
run: |
rm -rf ruff/crates/red_knot_vendored/vendor/typeshed
mkdir ruff/crates/red_knot_vendored/vendor/typeshed
cp typeshed/README.md ruff/crates/red_knot_vendored/vendor/typeshed
cp typeshed/LICENSE ruff/crates/red_knot_vendored/vendor/typeshed
cp -r typeshed/stdlib ruff/crates/red_knot_vendored/vendor/typeshed/stdlib
rm -rf ruff/crates/red_knot_vendored/vendor/typeshed/stdlib/@tests
git -C typeshed rev-parse HEAD > ruff/crates/red_knot_vendored/vendor/typeshed/source_commit.txt
rm -rf ruff/crates/ty_vendored/vendor/typeshed
mkdir ruff/crates/ty_vendored/vendor/typeshed
cp typeshed/README.md ruff/crates/ty_vendored/vendor/typeshed
cp typeshed/LICENSE ruff/crates/ty_vendored/vendor/typeshed
cp -r typeshed/stdlib ruff/crates/ty_vendored/vendor/typeshed/stdlib
rm -rf ruff/crates/ty_vendored/vendor/typeshed/stdlib/@tests
git -C typeshed rev-parse HEAD > ruff/crates/ty_vendored/vendor/typeshed/source_commit.txt
- name: Commit the changes
id: commit
if: ${{ steps.sync.outcome == 'success' }}
@@ -79,5 +79,5 @@ jobs:
repo: "ruff",
title: `Automated typeshed sync failed on ${new Date().toDateString()}`,
body: "Run listed here: https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}",
labels: ["bug", "red-knot"],
labels: ["bug", "ty"],
})

View File

@@ -3,8 +3,8 @@ fail_fast: false
exclude: |
(?x)^(
.github/workflows/release.yml|
crates/red_knot_vendored/vendor/.*|
crates/red_knot_project/resources/.*|
crates/ty_vendored/vendor/.*|
crates/ty_project/resources/.*|
crates/ruff_benchmark/resources/.*|
crates/ruff_linter/resources/.*|
crates/ruff_linter/src/rules/.*/snapshots/.*|
@@ -65,7 +65,7 @@ repos:
- black==25.1.0
- repo: https://github.com/crate-ci/typos
rev: v1.31.1
rev: v1.32.0
hooks:
- id: typos
@@ -79,7 +79,7 @@ repos:
pass_filenames: false # This makes it a lot faster
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.11.6
rev: v0.11.8
hooks:
- id: ruff-format
- id: ruff

View File

@@ -1,5 +1,41 @@
# Changelog
## 0.11.8
### Preview features
- \[`airflow`\] Apply auto fixes to cases where the names have changed in Airflow 3 (`AIR302`, `AIR311`) ([#17553](https://github.com/astral-sh/ruff/pull/17553), [#17570](https://github.com/astral-sh/ruff/pull/17570), [#17571](https://github.com/astral-sh/ruff/pull/17571))
- \[`airflow`\] Extend `AIR301` rule ([#17598](https://github.com/astral-sh/ruff/pull/17598))
- \[`airflow`\] Update existing `AIR302` rules with better suggestions ([#17542](https://github.com/astral-sh/ruff/pull/17542))
- \[`refurb`\] Mark fix as safe for `readlines-in-for` (`FURB129`) ([#17644](https://github.com/astral-sh/ruff/pull/17644))
- [syntax-errors] `nonlocal` declaration at module level ([#17559](https://github.com/astral-sh/ruff/pull/17559))
- [syntax-errors] Detect single starred expression assignment `x = *y` ([#17624](https://github.com/astral-sh/ruff/pull/17624))
### Bug fixes
- \[`flake8-pyi`\] Ensure `Literal[None,] | Literal[None,]` is not autofixed to `None | None` (`PYI061`) ([#17659](https://github.com/astral-sh/ruff/pull/17659))
- \[`flake8-use-pathlib`\] Avoid suggesting `Path.iterdir()` for `os.listdir` with file descriptor (`PTH208`) ([#17715](https://github.com/astral-sh/ruff/pull/17715))
- \[`flake8-use-pathlib`\] Fix `PTH104` false positive when `rename` is passed a file descriptor ([#17712](https://github.com/astral-sh/ruff/pull/17712))
- \[`flake8-use-pathlib`\] Fix `PTH116` false positive when `stat` is passed a file descriptor ([#17709](https://github.com/astral-sh/ruff/pull/17709))
- \[`flake8-use-pathlib`\] Fix `PTH123` false positive when `open` is passed a file descriptor from a function call ([#17705](https://github.com/astral-sh/ruff/pull/17705))
- \[`pycodestyle`\] Fix duplicated diagnostic in `E712` ([#17651](https://github.com/astral-sh/ruff/pull/17651))
- \[`pylint`\] Detect `global` declarations in module scope (`PLE0118`) ([#17411](https://github.com/astral-sh/ruff/pull/17411))
- [syntax-errors] Make `async-comprehension-in-sync-comprehension` more specific ([#17460](https://github.com/astral-sh/ruff/pull/17460))
### Configuration
- Add option to disable `typing_extensions` imports ([#17611](https://github.com/astral-sh/ruff/pull/17611))
### Documentation
- Fix example syntax for the `lint.pydocstyle.ignore-var-parameters` option ([#17740](https://github.com/astral-sh/ruff/pull/17740))
- Add fix safety sections (`ASYNC116`, `FLY002`, `D200`, `RUF005`, `RUF017`, `RUF027`, `RUF028`, `RUF057`) ([#17497](https://github.com/astral-sh/ruff/pull/17497), [#17496](https://github.com/astral-sh/ruff/pull/17496), [#17502](https://github.com/astral-sh/ruff/pull/17502), [#17484](https://github.com/astral-sh/ruff/pull/17484), [#17480](https://github.com/astral-sh/ruff/pull/17480), [#17485](https://github.com/astral-sh/ruff/pull/17485), [#17722](https://github.com/astral-sh/ruff/pull/17722), [#17483](https://github.com/astral-sh/ruff/pull/17483))
### Other changes
- Add Python 3.14 to configuration options ([#17647](https://github.com/astral-sh/ruff/pull/17647))
- Make syntax error for unparenthesized except tuples version specific to before 3.14 ([#17660](https://github.com/astral-sh/ruff/pull/17660))
## 0.11.7
### Preview features

View File

@@ -71,8 +71,7 @@ representative at an online or offline event.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported to the community leaders responsible for enforcement at
<charlie.r.marsh@gmail.com>.
reported to the community leaders responsible for enforcement at <hey@astral.sh>.
All complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the

View File

@@ -366,6 +366,15 @@ uvx --from ./python/ruff-ecosystem ruff-ecosystem format ruff "./target/debug/ru
See the [ruff-ecosystem package](https://github.com/astral-sh/ruff/tree/main/python/ruff-ecosystem) for more details.
## Upgrading Rust
1. Change the `channel` in `./rust-toolchain.toml` to the new Rust version (`<latest>`)
1. Change the `rust-version` in the `./Cargo.toml` to `<latest> - 2` (e.g. 1.84 if the latest is 1.86)
1. Run `cargo clippy --fix --allow-dirty --allow-staged` to fix new clippy warnings
1. Create and merge the PR
1. Bump the Rust version in Ruff's conda forge recipe. See [this PR](https://github.com/conda-forge/ruff-feedstock/pull/266) for an example.
1. Enjoy the new Rust version!
## Benchmarking and Profiling
We have several ways of benchmarking and profiling Ruff:

603
Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -35,14 +35,14 @@ ruff_python_trivia = { path = "crates/ruff_python_trivia" }
ruff_server = { path = "crates/ruff_server" }
ruff_source_file = { path = "crates/ruff_source_file" }
ruff_text_size = { path = "crates/ruff_text_size" }
red_knot_vendored = { path = "crates/red_knot_vendored" }
ruff_workspace = { path = "crates/ruff_workspace" }
red_knot_ide = { path = "crates/red_knot_ide" }
red_knot_project = { path = "crates/red_knot_project", default-features = false }
red_knot_python_semantic = { path = "crates/red_knot_python_semantic" }
red_knot_server = { path = "crates/red_knot_server" }
red_knot_test = { path = "crates/red_knot_test" }
ty_ide = { path = "crates/ty_ide" }
ty_project = { path = "crates/ty_project", default-features = false }
ty_python_semantic = { path = "crates/ty_python_semantic" }
ty_server = { path = "crates/ty_server" }
ty_test = { path = "crates/ty_test" }
ty_vendored = { path = "crates/ty_vendored" }
aho-corasick = { version = "1.1.3" }
anstream = { version = "0.6.18" }
@@ -124,7 +124,7 @@ rayon = { version = "1.10.0" }
regex = { version = "1.10.2" }
rustc-hash = { version = "2.0.0" }
# When updating salsa, make sure to also update the revision in `fuzz/Cargo.toml`
salsa = { git = "https://github.com/salsa-rs/salsa.git", rev = "87bf6b6c2d5f6479741271da73bd9d30c2580c26" }
salsa = { git = "https://github.com/salsa-rs/salsa.git", rev = "ef93d3830ffb8dc621fef1ba7b234ccef9dc3639" }
schemars = { version = "0.8.16" }
seahash = { version = "4.1.0" }
serde = { version = "1.0.197", features = ["derive"] }
@@ -231,10 +231,6 @@ unused_peekable = "warn"
# Diagnostics are not actionable: Enable once https://github.com/rust-lang/rust-clippy/issues/13774 is resolved.
large_stack_arrays = "allow"
# Salsa generates functions with parameters for each field of a `salsa::interned` struct.
# If we don't allow this, we get warnings for structs with too many fields.
too_many_arguments = "allow"
[profile.release]
# Note that we set these explicitly, and these values
# were chosen based on a trade-off between compile times
@@ -272,75 +268,3 @@ debug = 1
# The profile that 'cargo dist' will build with.
[profile.dist]
inherits = "release"
# Config for 'dist'
[workspace.metadata.dist]
# The preferred dist version to use in CI (Cargo.toml SemVer syntax)
cargo-dist-version = "0.28.4"
# Make distability of apps opt-in instead of opt-out
dist = false
# CI backends to support
ci = "github"
# The installers to generate for each app
installers = ["shell", "powershell"]
# The archive format to use for windows builds (defaults .zip)
windows-archive = ".zip"
# The archive format to use for non-windows builds (defaults .tar.xz)
unix-archive = ".tar.gz"
# Target platforms to build apps for (Rust target-triple syntax)
targets = [
"aarch64-apple-darwin",
"aarch64-pc-windows-msvc",
"aarch64-unknown-linux-gnu",
"aarch64-unknown-linux-musl",
"arm-unknown-linux-musleabihf",
"armv7-unknown-linux-gnueabihf",
"armv7-unknown-linux-musleabihf",
"i686-pc-windows-msvc",
"i686-unknown-linux-gnu",
"i686-unknown-linux-musl",
"powerpc64-unknown-linux-gnu",
"powerpc64le-unknown-linux-gnu",
"s390x-unknown-linux-gnu",
"x86_64-apple-darwin",
"x86_64-pc-windows-msvc",
"x86_64-unknown-linux-gnu",
"x86_64-unknown-linux-musl",
]
# Whether to auto-include files like READMEs, LICENSEs, and CHANGELOGs (default true)
auto-includes = false
# Whether dist should create a Github Release or use an existing draft
create-release = true
# Which actions to run on pull requests
pr-run-mode = "plan"
# Whether CI should trigger releases with dispatches instead of tag pushes
dispatch-releases = true
# Which phase dist should use to create the GitHub release
github-release = "announce"
# Whether CI should include auto-generated code to build local artifacts
build-local-artifacts = false
# Local artifacts jobs to run in CI
local-artifacts-jobs = ["./build-binaries", "./build-docker"]
# Publish jobs to run in CI
publish-jobs = ["./publish-pypi", "./publish-wasm"]
# Post-announce jobs to run in CI
post-announce-jobs = [
"./notify-dependents",
"./publish-docs",
"./publish-playground",
]
# Custom permissions for GitHub Jobs
github-custom-job-permissions = { "build-docker" = { packages = "write", contents = "read" }, "publish-wasm" = { contents = "read", id-token = "write", packages = "write" } }
# Whether to install an updater program
install-updater = false
# Path that installers should place binaries in
install-path = ["$XDG_BIN_HOME/", "$XDG_DATA_HOME/../bin", "~/.local/bin"]
[workspace.metadata.dist.github-custom-runners]
global = "depot-ubuntu-latest-4"
[workspace.metadata.dist.github-action-commits]
"actions/checkout" = "85e6279cec87321a52edac9c87bce653a07cf6c2" # v4
"actions/upload-artifact" = "6027e3dd177782cd8ab9af838c04fd81a07f1d47" # v4.6.2
"actions/download-artifact" = "95815c38cf2ff2164869cbab79da8d1f422bc89e" # v4.2.1
"actions/attest-build-provenance" = "c074443f1aee8d4aeeae555aebba3282517141b2" #v2.2.3

View File

@@ -149,8 +149,8 @@ curl -LsSf https://astral.sh/ruff/install.sh | sh
powershell -c "irm https://astral.sh/ruff/install.ps1 | iex"
# For a specific version.
curl -LsSf https://astral.sh/ruff/0.11.7/install.sh | sh
powershell -c "irm https://astral.sh/ruff/0.11.7/install.ps1 | iex"
curl -LsSf https://astral.sh/ruff/0.11.8/install.sh | sh
powershell -c "irm https://astral.sh/ruff/0.11.8/install.ps1 | iex"
```
You can also install Ruff via [Homebrew](https://formulae.brew.sh/formula/ruff), [Conda](https://anaconda.org/conda-forge/ruff),
@@ -183,7 +183,7 @@ Ruff can also be used as a [pre-commit](https://pre-commit.com/) hook via [`ruff
```yaml
- repo: https://github.com/astral-sh/ruff-pre-commit
# Ruff version.
rev: v0.11.7
rev: v0.11.8
hooks:
# Run the linter.
- id: ruff

View File

@@ -1,7 +1,7 @@
[files]
# https://github.com/crate-ci/typos/issues/868
extend-exclude = [
"crates/red_knot_vendored/vendor/**/*",
"crates/ty_vendored/vendor/**/*",
"**/resources/**/*",
"**/snapshots/**/*",
]

View File

@@ -1,25 +0,0 @@
# Red Knot
Red Knot is an extremely fast type checker.
Currently, it is a work-in-progress and not ready for user testing.
Red Knot is designed to prioritize good type inference, even in unannotated code,
and aims to avoid false positives.
While Red Knot will produce similar results to mypy and pyright on many codebases,
100% compatibility with these tools is a non-goal.
On some codebases, Red Knot's design decisions lead to different outcomes
than you would get from running one of these more established tools.
## Contributing
Core type checking tests are written as Markdown code blocks.
They can be found in [`red_knot_python_semantic/resources/mdtest`][resources-mdtest].
See [`red_knot_test/README.md`][mdtest-readme] for more information
on the test framework itself.
The list of open issues can be found [here][open-issues].
[mdtest-readme]: ../red_knot_test/README.md
[open-issues]: https://github.com/astral-sh/ruff/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20label%3Ared-knot
[resources-mdtest]: ../red_knot_python_semantic/resources/mdtest

View File

@@ -1,102 +0,0 @@
# Any
## Annotation
`typing.Any` is a way to name the Any type.
```py
from typing import Any
x: Any = 1
x = "foo"
def f():
reveal_type(x) # revealed: Any
```
## Aliased to a different name
If you alias `typing.Any` to another name, we still recognize that as a spelling of the Any type.
```py
from typing import Any as RenamedAny
x: RenamedAny = 1
x = "foo"
def f():
reveal_type(x) # revealed: Any
```
## Shadowed class
If you define your own class named `Any`, using that in a type expression refers to your class, and
isn't a spelling of the Any type.
```py
class Any: ...
x: Any
def f():
reveal_type(x) # revealed: Any
# This verifies that we're not accidentally seeing typing.Any, since str is assignable
# to that but not to our locally defined class.
y: Any = "not an Any" # error: [invalid-assignment]
```
## Subclass
The spec allows you to define subclasses of `Any`.
`Subclass` has an unknown superclass, which might be `int`. The assignment to `x` should not be
allowed, even when the unknown superclass is `int`. The assignment to `y` should be allowed, since
`Subclass` might have `int` as a superclass, and is therefore assignable to `int`.
```py
from typing import Any
class Subclass(Any): ...
reveal_type(Subclass.__mro__) # revealed: tuple[Literal[Subclass], Any, Literal[object]]
x: Subclass = 1 # error: [invalid-assignment]
y: int = Subclass()
def _(s: Subclass):
reveal_type(s) # revealed: Subclass
```
`Subclass` should not be assignable to a final class though, because `Subclass` could not possibly
be a subclass of `FinalClass`:
```py
from typing import final
@final
class FinalClass: ...
f: FinalClass = Subclass() # error: [invalid-assignment]
```
A use case where this comes up is with mocking libraries, where the mock object should be assignable
to any type:
```py
from unittest.mock import MagicMock
x: int = MagicMock()
```
## Invalid
`Any` cannot be parameterized:
```py
from typing import Any
# error: [invalid-type-form] "Type `typing.Any` expected no type parameter"
def f(x: Any[int]):
reveal_type(x) # revealed: Unknown
```

View File

@@ -1,129 +0,0 @@
# Typing-module aliases to other stdlib classes
The `typing` module has various aliases to other stdlib classes. These are a legacy feature, but
still need to be supported by a type checker.
## Correspondence
All of the following symbols can be mapped one-to-one with the actual type:
```py
import typing
def f(
list_bare: typing.List,
list_parametrized: typing.List[int],
dict_bare: typing.Dict,
dict_parametrized: typing.Dict[int, str],
set_bare: typing.Set,
set_parametrized: typing.Set[int],
frozen_set_bare: typing.FrozenSet,
frozen_set_parametrized: typing.FrozenSet[str],
chain_map_bare: typing.ChainMap,
chain_map_parametrized: typing.ChainMap[int],
counter_bare: typing.Counter,
counter_parametrized: typing.Counter[int],
default_dict_bare: typing.DefaultDict,
default_dict_parametrized: typing.DefaultDict[str, int],
deque_bare: typing.Deque,
deque_parametrized: typing.Deque[str],
ordered_dict_bare: typing.OrderedDict,
ordered_dict_parametrized: typing.OrderedDict[int, str],
):
reveal_type(list_bare) # revealed: list
reveal_type(list_parametrized) # revealed: list
reveal_type(dict_bare) # revealed: dict
reveal_type(dict_parametrized) # revealed: dict
reveal_type(set_bare) # revealed: set
reveal_type(set_parametrized) # revealed: set
reveal_type(frozen_set_bare) # revealed: frozenset
reveal_type(frozen_set_parametrized) # revealed: frozenset
reveal_type(chain_map_bare) # revealed: ChainMap
reveal_type(chain_map_parametrized) # revealed: ChainMap
reveal_type(counter_bare) # revealed: Counter
reveal_type(counter_parametrized) # revealed: Counter
reveal_type(default_dict_bare) # revealed: defaultdict
reveal_type(default_dict_parametrized) # revealed: defaultdict
reveal_type(deque_bare) # revealed: deque
reveal_type(deque_parametrized) # revealed: deque
reveal_type(ordered_dict_bare) # revealed: OrderedDict
reveal_type(ordered_dict_parametrized) # revealed: OrderedDict
```
## Inheritance
The aliases can be inherited from. Some of these are still partially or wholly TODOs.
```py
import typing
####################
### Built-ins
####################
class ListSubclass(typing.List): ...
# TODO: generic protocols
# revealed: tuple[Literal[ListSubclass], Literal[list], Literal[MutableSequence], Literal[Sequence], Literal[Reversible], Literal[Collection], Literal[Iterable], Literal[Container], @Todo(`Protocol[]` subscript), @Todo(`Generic[]` subscript), Literal[object]]
reveal_type(ListSubclass.__mro__)
class DictSubclass(typing.Dict): ...
# TODO: generic protocols
# revealed: tuple[Literal[DictSubclass], Literal[dict], Literal[MutableMapping], Literal[Mapping], Literal[Collection], Literal[Iterable], Literal[Container], @Todo(`Protocol[]` subscript), @Todo(`Generic[]` subscript), Literal[object]]
reveal_type(DictSubclass.__mro__)
class SetSubclass(typing.Set): ...
# TODO: generic protocols
# revealed: tuple[Literal[SetSubclass], Literal[set], Literal[MutableSet], Literal[AbstractSet], Literal[Collection], Literal[Iterable], Literal[Container], @Todo(`Protocol[]` subscript), @Todo(`Generic[]` subscript), Literal[object]]
reveal_type(SetSubclass.__mro__)
class FrozenSetSubclass(typing.FrozenSet): ...
# TODO: should have `Generic`, should not have `Unknown`
# revealed: tuple[Literal[FrozenSetSubclass], Literal[frozenset], Unknown, Literal[object]]
reveal_type(FrozenSetSubclass.__mro__)
####################
### `collections`
####################
class ChainMapSubclass(typing.ChainMap): ...
# TODO: generic protocols
# revealed: tuple[Literal[ChainMapSubclass], Literal[ChainMap], Literal[MutableMapping], Literal[Mapping], Literal[Collection], Literal[Iterable], Literal[Container], @Todo(`Protocol[]` subscript), @Todo(`Generic[]` subscript), Literal[object]]
reveal_type(ChainMapSubclass.__mro__)
class CounterSubclass(typing.Counter): ...
# TODO: Should be (CounterSubclass, Counter, dict, MutableMapping, Mapping, Collection, Sized, Iterable, Container, Generic, object)
# revealed: tuple[Literal[CounterSubclass], Literal[Counter], @Todo(GenericAlias instance), @Todo(`Generic[]` subscript), Literal[object]]
reveal_type(CounterSubclass.__mro__)
class DefaultDictSubclass(typing.DefaultDict): ...
# TODO: Should be (DefaultDictSubclass, defaultdict, dict, MutableMapping, Mapping, Collection, Sized, Iterable, Container, Generic, object)
# revealed: tuple[Literal[DefaultDictSubclass], Literal[defaultdict], @Todo(GenericAlias instance), Literal[object]]
reveal_type(DefaultDictSubclass.__mro__)
class DequeSubclass(typing.Deque): ...
# TODO: generic protocols
# revealed: tuple[Literal[DequeSubclass], Literal[deque], Literal[MutableSequence], Literal[Sequence], Literal[Reversible], Literal[Collection], Literal[Iterable], Literal[Container], @Todo(`Protocol[]` subscript), @Todo(`Generic[]` subscript), Literal[object]]
reveal_type(DequeSubclass.__mro__)
class OrderedDictSubclass(typing.OrderedDict): ...
# TODO: Should be (OrderedDictSubclass, OrderedDict, dict, MutableMapping, Mapping, Collection, Sized, Iterable, Container, Generic, object)
# revealed: tuple[Literal[OrderedDictSubclass], Literal[OrderedDict], @Todo(GenericAlias instance), Literal[object]]
reveal_type(OrderedDictSubclass.__mro__)
```

View File

@@ -1,72 +0,0 @@
# Legacy type variables
The tests in this file focus on how type variables are defined using the legacy notation. Most
_uses_ of type variables are tested in other files in this directory; we do not duplicate every test
for both type variable syntaxes.
Unless otherwise specified, all quotations come from the [Generics] section of the typing spec.
## Type variables
### Defining legacy type variables
> Generics can be parameterized by using a factory available in `typing` called `TypeVar`.
This was the only way to create type variables prior to PEP 695/Python 3.12. It is still available
in newer Python releases.
```py
from typing import TypeVar
T = TypeVar("T")
```
### Directly assigned to a variable
> A `TypeVar()` expression must always directly be assigned to a variable (it should not be used as
> part of a larger expression).
```py
from typing import TypeVar
# TODO: error
TestList = list[TypeVar("W")]
```
### `TypeVar` parameter must match variable name
> The argument to `TypeVar()` must be a string equal to the variable name to which it is assigned.
```py
from typing import TypeVar
# TODO: error
T = TypeVar("Q")
```
### No redefinition
> Type variables must not be redefined.
```py
from typing import TypeVar
T = TypeVar("T")
# TODO: error
T = TypeVar("T")
```
### Cannot have only one constraint
> `TypeVar` supports constraining parametric types to a fixed set of possible types...There should
> be at least two constraints, if any; specifying a single constraint is disallowed.
```py
from typing import TypeVar
# TODO: error: [invalid-type-variable-constraints]
T = TypeVar("T", int)
```
[generics]: https://typing.python.org/en/latest/spec/generics.html

View File

@@ -1,7 +0,0 @@
# Dictionaries
## Empty dictionary
```py
reveal_type({}) # revealed: dict
```

View File

@@ -1,408 +0,0 @@
# Method Resolution Order tests
Tests that assert that we can infer the correct type for a class's `__mro__` attribute.
This attribute is rarely accessed directly at runtime. However, it's extremely important for *us* to
know the precise possible values of a class's Method Resolution Order, or we won't be able to infer
the correct type of attributes accessed from instances.
For documentation on method resolution orders, see:
- <https://docs.python.org/3/glossary.html#term-method-resolution-order>
- <https://docs.python.org/3/howto/mro.html#python-2-3-mro>
## No bases
```py
class C: ...
reveal_type(C.__mro__) # revealed: tuple[Literal[C], Literal[object]]
```
## The special case: `object` itself
```py
reveal_type(object.__mro__) # revealed: tuple[Literal[object]]
```
## Explicit inheritance from `object`
```py
class C(object): ...
reveal_type(C.__mro__) # revealed: tuple[Literal[C], Literal[object]]
```
## Explicit inheritance from non-`object` single base
```py
class A: ...
class B(A): ...
reveal_type(B.__mro__) # revealed: tuple[Literal[B], Literal[A], Literal[object]]
```
## Linearization of multiple bases
```py
class A: ...
class B: ...
class C(A, B): ...
reveal_type(C.__mro__) # revealed: tuple[Literal[C], Literal[A], Literal[B], Literal[object]]
```
## Complex diamond inheritance (1)
This is "ex_2" from <https://docs.python.org/3/howto/mro.html#the-end>
```py
class O: ...
class X(O): ...
class Y(O): ...
class A(X, Y): ...
class B(Y, X): ...
reveal_type(A.__mro__) # revealed: tuple[Literal[A], Literal[X], Literal[Y], Literal[O], Literal[object]]
reveal_type(B.__mro__) # revealed: tuple[Literal[B], Literal[Y], Literal[X], Literal[O], Literal[object]]
```
## Complex diamond inheritance (2)
This is "ex_5" from <https://docs.python.org/3/howto/mro.html#the-end>
```py
class O: ...
class F(O): ...
class E(O): ...
class D(O): ...
class C(D, F): ...
class B(D, E): ...
class A(B, C): ...
# revealed: tuple[Literal[C], Literal[D], Literal[F], Literal[O], Literal[object]]
reveal_type(C.__mro__)
# revealed: tuple[Literal[B], Literal[D], Literal[E], Literal[O], Literal[object]]
reveal_type(B.__mro__)
# revealed: tuple[Literal[A], Literal[B], Literal[C], Literal[D], Literal[E], Literal[F], Literal[O], Literal[object]]
reveal_type(A.__mro__)
```
## Complex diamond inheritance (3)
This is "ex_6" from <https://docs.python.org/3/howto/mro.html#the-end>
```py
class O: ...
class F(O): ...
class E(O): ...
class D(O): ...
class C(D, F): ...
class B(E, D): ...
class A(B, C): ...
# revealed: tuple[Literal[C], Literal[D], Literal[F], Literal[O], Literal[object]]
reveal_type(C.__mro__)
# revealed: tuple[Literal[B], Literal[E], Literal[D], Literal[O], Literal[object]]
reveal_type(B.__mro__)
# revealed: tuple[Literal[A], Literal[B], Literal[E], Literal[C], Literal[D], Literal[F], Literal[O], Literal[object]]
reveal_type(A.__mro__)
```
## Complex diamond inheritance (4)
This is "ex_9" from <https://docs.python.org/3/howto/mro.html#the-end>
```py
class O: ...
class A(O): ...
class B(O): ...
class C(O): ...
class D(O): ...
class E(O): ...
class K1(A, B, C): ...
class K2(D, B, E): ...
class K3(D, A): ...
class Z(K1, K2, K3): ...
# revealed: tuple[Literal[K1], Literal[A], Literal[B], Literal[C], Literal[O], Literal[object]]
reveal_type(K1.__mro__)
# revealed: tuple[Literal[K2], Literal[D], Literal[B], Literal[E], Literal[O], Literal[object]]
reveal_type(K2.__mro__)
# revealed: tuple[Literal[K3], Literal[D], Literal[A], Literal[O], Literal[object]]
reveal_type(K3.__mro__)
# revealed: tuple[Literal[Z], Literal[K1], Literal[K2], Literal[K3], Literal[D], Literal[A], Literal[B], Literal[C], Literal[E], Literal[O], Literal[object]]
reveal_type(Z.__mro__)
```
## Inheritance from `Unknown`
```py
from does_not_exist import DoesNotExist # error: [unresolved-import]
class A(DoesNotExist): ...
class B: ...
class C: ...
class D(A, B, C): ...
class E(B, C): ...
class F(E, A): ...
reveal_type(A.__mro__) # revealed: tuple[Literal[A], Unknown, Literal[object]]
reveal_type(D.__mro__) # revealed: tuple[Literal[D], Literal[A], Unknown, Literal[B], Literal[C], Literal[object]]
reveal_type(E.__mro__) # revealed: tuple[Literal[E], Literal[B], Literal[C], Literal[object]]
reveal_type(F.__mro__) # revealed: tuple[Literal[F], Literal[E], Literal[B], Literal[C], Literal[A], Unknown, Literal[object]]
```
## `__bases__` lists that cause errors at runtime
If the class's `__bases__` cause an exception to be raised at runtime and therefore the class
creation to fail, we infer the class's `__mro__` as being `[<class>, Unknown, object]`:
```py
# error: [inconsistent-mro] "Cannot create a consistent method resolution order (MRO) for class `Foo` with bases list `[<class 'object'>, <class 'int'>]`"
class Foo(object, int): ...
reveal_type(Foo.__mro__) # revealed: tuple[Literal[Foo], Unknown, Literal[object]]
class Bar(Foo): ...
reveal_type(Bar.__mro__) # revealed: tuple[Literal[Bar], Literal[Foo], Unknown, Literal[object]]
# This is the `TypeError` at the bottom of "ex_2"
# in the examples at <https://docs.python.org/3/howto/mro.html#the-end>
class O: ...
class X(O): ...
class Y(O): ...
class A(X, Y): ...
class B(Y, X): ...
reveal_type(A.__mro__) # revealed: tuple[Literal[A], Literal[X], Literal[Y], Literal[O], Literal[object]]
reveal_type(B.__mro__) # revealed: tuple[Literal[B], Literal[Y], Literal[X], Literal[O], Literal[object]]
# error: [inconsistent-mro] "Cannot create a consistent method resolution order (MRO) for class `Z` with bases list `[<class 'A'>, <class 'B'>]`"
class Z(A, B): ...
reveal_type(Z.__mro__) # revealed: tuple[Literal[Z], Unknown, Literal[object]]
class AA(Z): ...
reveal_type(AA.__mro__) # revealed: tuple[Literal[AA], Literal[Z], Unknown, Literal[object]]
```
## `__bases__` includes a `Union`
We don't support union types in a class's bases; a base must resolve to a single `ClassType`. If we
find a union type in a class's bases, we infer the class's `__mro__` as being
`[<class>, Unknown, object]`, the same as for MROs that cause errors at runtime.
```py
def returns_bool() -> bool:
return True
class A: ...
class B: ...
if returns_bool():
x = A
else:
x = B
reveal_type(x) # revealed: Literal[A, B]
# error: 11 [invalid-base] "Invalid class base with type `Literal[A, B]` (all bases must be a class, `Any`, `Unknown` or `Todo`)"
class Foo(x): ...
reveal_type(Foo.__mro__) # revealed: tuple[Literal[Foo], Unknown, Literal[object]]
```
## `__bases__` includes multiple `Union`s
```py
def returns_bool() -> bool:
return True
class A: ...
class B: ...
class C: ...
class D: ...
if returns_bool():
x = A
else:
x = B
if returns_bool():
y = C
else:
y = D
reveal_type(x) # revealed: Literal[A, B]
reveal_type(y) # revealed: Literal[C, D]
# error: 11 [invalid-base] "Invalid class base with type `Literal[A, B]` (all bases must be a class, `Any`, `Unknown` or `Todo`)"
# error: 14 [invalid-base] "Invalid class base with type `Literal[C, D]` (all bases must be a class, `Any`, `Unknown` or `Todo`)"
class Foo(x, y): ...
reveal_type(Foo.__mro__) # revealed: tuple[Literal[Foo], Unknown, Literal[object]]
```
## `__bases__` lists that cause errors... now with `Union`s
```py
def returns_bool() -> bool:
return True
class O: ...
class X(O): ...
class Y(O): ...
if returns_bool():
foo = Y
else:
foo = object
# error: 21 [invalid-base] "Invalid class base with type `Literal[Y, object]` (all bases must be a class, `Any`, `Unknown` or `Todo`)"
class PossibleError(foo, X): ...
reveal_type(PossibleError.__mro__) # revealed: tuple[Literal[PossibleError], Unknown, Literal[object]]
class A(X, Y): ...
reveal_type(A.__mro__) # revealed: tuple[Literal[A], Literal[X], Literal[Y], Literal[O], Literal[object]]
if returns_bool():
class B(X, Y): ...
else:
class B(Y, X): ...
# revealed: tuple[Literal[B], Literal[X], Literal[Y], Literal[O], Literal[object]] | tuple[Literal[B], Literal[Y], Literal[X], Literal[O], Literal[object]]
reveal_type(B.__mro__)
# error: 12 [invalid-base] "Invalid class base with type `Literal[B, B]` (all bases must be a class, `Any`, `Unknown` or `Todo`)"
class Z(A, B): ...
reveal_type(Z.__mro__) # revealed: tuple[Literal[Z], Unknown, Literal[object]]
```
## `__bases__` lists with duplicate bases
```py
class Foo(str, str): ... # error: 16 [duplicate-base] "Duplicate base class `str`"
reveal_type(Foo.__mro__) # revealed: tuple[Literal[Foo], Unknown, Literal[object]]
class Spam: ...
class Eggs: ...
class Ham(
Spam,
Eggs,
Spam, # error: [duplicate-base] "Duplicate base class `Spam`"
Eggs, # error: [duplicate-base] "Duplicate base class `Eggs`"
): ...
reveal_type(Ham.__mro__) # revealed: tuple[Literal[Ham], Unknown, Literal[object]]
class Mushrooms: ...
class Omelette(Spam, Eggs, Mushrooms, Mushrooms): ... # error: [duplicate-base]
reveal_type(Omelette.__mro__) # revealed: tuple[Literal[Omelette], Unknown, Literal[object]]
```
## `__bases__` lists with duplicate `Unknown` bases
```py
# error: [unresolved-import]
# error: [unresolved-import]
from does_not_exist import unknown_object_1, unknown_object_2
reveal_type(unknown_object_1) # revealed: Unknown
reveal_type(unknown_object_2) # revealed: Unknown
# We *should* emit an error here to warn the user that we have no idea
# what the MRO of this class should really be.
# However, we don't complain about "duplicate base classes" here,
# even though two classes are both inferred as being `Unknown`.
#
# (TODO: should we revisit this? Does it violate the gradual guarantee?
# Should we just silently infer `[Foo, Unknown, object]` as the MRO here
# without emitting any error at all? Not sure...)
#
# error: [inconsistent-mro] "Cannot create a consistent method resolution order (MRO) for class `Foo` with bases list `[Unknown, Unknown]`"
class Foo(unknown_object_1, unknown_object_2): ...
reveal_type(Foo.__mro__) # revealed: tuple[Literal[Foo], Unknown, Literal[object]]
```
## Unrelated objects inferred as `Any`/`Unknown` do not have special `__mro__` attributes
```py
from does_not_exist import unknown_object # error: [unresolved-import]
reveal_type(unknown_object) # revealed: Unknown
reveal_type(unknown_object.__mro__) # revealed: Unknown
```
## Classes that inherit from themselves
These are invalid, but we need to be able to handle them gracefully without panicking.
```pyi
class Foo(Foo): ... # error: [cyclic-class-definition]
reveal_type(Foo) # revealed: Literal[Foo]
reveal_type(Foo.__mro__) # revealed: tuple[Literal[Foo], Unknown, Literal[object]]
class Bar: ...
class Baz: ...
class Boz(Bar, Baz, Boz): ... # error: [cyclic-class-definition]
reveal_type(Boz) # revealed: Literal[Boz]
reveal_type(Boz.__mro__) # revealed: tuple[Literal[Boz], Unknown, Literal[object]]
```
## Classes with indirect cycles in their MROs
These are similarly unlikely, but we still shouldn't crash:
```pyi
class Foo(Bar): ... # error: [cyclic-class-definition]
class Bar(Baz): ... # error: [cyclic-class-definition]
class Baz(Foo): ... # error: [cyclic-class-definition]
reveal_type(Foo.__mro__) # revealed: tuple[Literal[Foo], Unknown, Literal[object]]
reveal_type(Bar.__mro__) # revealed: tuple[Literal[Bar], Unknown, Literal[object]]
reveal_type(Baz.__mro__) # revealed: tuple[Literal[Baz], Unknown, Literal[object]]
```
## Classes with cycles in their MROs, and multiple inheritance
```pyi
class Spam: ...
class Foo(Bar): ... # error: [cyclic-class-definition]
class Bar(Baz): ... # error: [cyclic-class-definition]
class Baz(Foo, Spam): ... # error: [cyclic-class-definition]
reveal_type(Foo.__mro__) # revealed: tuple[Literal[Foo], Unknown, Literal[object]]
reveal_type(Bar.__mro__) # revealed: tuple[Literal[Bar], Unknown, Literal[object]]
reveal_type(Baz.__mro__) # revealed: tuple[Literal[Baz], Unknown, Literal[object]]
```
## Classes with cycles in their MRO, and a sub-graph
```pyi
class FooCycle(BarCycle): ... # error: [cyclic-class-definition]
class Foo: ...
class BarCycle(FooCycle): ... # error: [cyclic-class-definition]
class Bar(Foo): ...
# Avoid emitting the errors for these. The classes have cyclic superclasses,
# but are not themselves cyclic...
class Baz(Bar, BarCycle): ...
class Spam(Baz): ...
reveal_type(FooCycle.__mro__) # revealed: tuple[Literal[FooCycle], Unknown, Literal[object]]
reveal_type(BarCycle.__mro__) # revealed: tuple[Literal[BarCycle], Unknown, Literal[object]]
reveal_type(Baz.__mro__) # revealed: tuple[Literal[Baz], Unknown, Literal[object]]
reveal_type(Spam.__mro__) # revealed: tuple[Literal[Spam], Unknown, Literal[object]]
```

View File

@@ -1,53 +0,0 @@
# Narrowing with assert statements
## `assert` a value `is None` or `is not None`
```py
def _(x: str | None, y: str | None):
assert x is not None
reveal_type(x) # revealed: str
assert y is None
reveal_type(y) # revealed: None
```
## `assert` a value is truthy or falsy
```py
def _(x: bool, y: bool):
assert x
reveal_type(x) # revealed: Literal[True]
assert not y
reveal_type(y) # revealed: Literal[False]
```
## `assert` with `is` and `==` for literals
```py
from typing import Literal
def _(x: Literal[1, 2, 3], y: Literal[1, 2, 3]):
assert x is 2
reveal_type(x) # revealed: Literal[2]
assert y == 2
reveal_type(y) # revealed: Literal[2]
```
## `assert` with `isinstance`
```py
def _(x: int | str):
assert isinstance(x, int)
reveal_type(x) # revealed: int
```
## `assert` a value `in` a tuple
```py
from typing import Literal
def _(x: Literal[1, 2, 3], y: Literal[1, 2, 3]):
assert x in (1, 2)
reveal_type(x) # revealed: Literal[1, 2]
assert y not in (1, 2)
reveal_type(y) # revealed: Literal[3]
```

View File

@@ -1,44 +0,0 @@
# Narrowing for nested conditionals
## Multiple negative contributions
```py
def _(x: int):
if x != 1:
if x != 2:
if x != 3:
reveal_type(x) # revealed: int & ~Literal[1] & ~Literal[2] & ~Literal[3]
```
## Multiple negative contributions with simplification
```py
def _(flag1: bool, flag2: bool):
x = 1 if flag1 else 2 if flag2 else 3
if x != 1:
reveal_type(x) # revealed: Literal[2, 3]
if x != 2:
reveal_type(x) # revealed: Literal[3]
```
## elif-else blocks
```py
def _(flag1: bool, flag2: bool):
x = 1 if flag1 else 2 if flag2 else 3
if x != 1:
reveal_type(x) # revealed: Literal[2, 3]
if x == 2:
reveal_type(x) # revealed: Literal[2]
elif x == 3:
reveal_type(x) # revealed: Literal[3]
else:
reveal_type(x) # revealed: Never
elif x != 2:
reveal_type(x) # revealed: Literal[1]
else:
reveal_type(x) # revealed: Never
```

View File

@@ -1,28 +0,0 @@
---
source: crates/red_knot_test/src/lib.rs
expression: snapshot
---
---
mdtest name: basic.md - Structures - Unresolvable module import
mdtest path: crates/red_knot_python_semantic/resources/mdtest/import/basic.md
---
# Python source files
## mdtest_snippet.py
```
1 | import zqzqzqzqzqzqzq # error: [unresolved-import] "Cannot resolve import `zqzqzqzqzqzqzq`"
```
# Diagnostics
```
error: lint:unresolved-import: Cannot resolve import `zqzqzqzqzqzqzq`
--> /src/mdtest_snippet.py:1:8
|
1 | import zqzqzqzqzqzqzq # error: [unresolved-import] "Cannot resolve import `zqzqzqzqzqzqzq`"
| ^^^^^^^^^^^^^^
|
```

View File

@@ -1,117 +0,0 @@
---
source: crates/red_knot_test/src/lib.rs
expression: snapshot
---
---
mdtest name: protocols.md - Protocols - Calls to protocol classes
mdtest path: crates/red_knot_python_semantic/resources/mdtest/protocols.md
---
# Python source files
## mdtest_snippet.py
```
1 | from typing_extensions import Protocol, reveal_type
2 |
3 | # error: [call-non-callable]
4 | reveal_type(Protocol()) # revealed: Unknown
5 |
6 | class MyProtocol(Protocol):
7 | x: int
8 |
9 | # error: [call-non-callable] "Cannot instantiate class `MyProtocol`"
10 | reveal_type(MyProtocol()) # revealed: MyProtocol
11 | class SubclassOfMyProtocol(MyProtocol): ...
12 |
13 | reveal_type(SubclassOfMyProtocol()) # revealed: SubclassOfMyProtocol
14 | def f(x: type[MyProtocol]):
15 | reveal_type(x()) # revealed: MyProtocol
```
# Diagnostics
```
error: lint:call-non-callable: Object of type `typing.Protocol` is not callable
--> /src/mdtest_snippet.py:4:13
|
3 | # error: [call-non-callable]
4 | reveal_type(Protocol()) # revealed: Unknown
| ^^^^^^^^^^
5 |
6 | class MyProtocol(Protocol):
|
```
```
info: revealed-type: Revealed type
--> /src/mdtest_snippet.py:4:1
|
3 | # error: [call-non-callable]
4 | reveal_type(Protocol()) # revealed: Unknown
| ^^^^^^^^^^^^^^^^^^^^^^^ `Unknown`
5 |
6 | class MyProtocol(Protocol):
|
```
```
error: lint:call-non-callable: Cannot instantiate class `MyProtocol`
--> /src/mdtest_snippet.py:10:13
|
9 | # error: [call-non-callable] "Cannot instantiate class `MyProtocol`"
10 | reveal_type(MyProtocol()) # revealed: MyProtocol
| ^^^^^^^^^^^^ This call will raise `TypeError` at runtime
11 | class SubclassOfMyProtocol(MyProtocol): ...
|
info: Protocol classes cannot be instantiated
--> /src/mdtest_snippet.py:6:7
|
4 | reveal_type(Protocol()) # revealed: Unknown
5 |
6 | class MyProtocol(Protocol):
| ^^^^^^^^^^^^^^^^^^^^ `MyProtocol` declared as a protocol here
7 | x: int
|
```
```
info: revealed-type: Revealed type
--> /src/mdtest_snippet.py:10:1
|
9 | # error: [call-non-callable] "Cannot instantiate class `MyProtocol`"
10 | reveal_type(MyProtocol()) # revealed: MyProtocol
| ^^^^^^^^^^^^^^^^^^^^^^^^^ `MyProtocol`
11 | class SubclassOfMyProtocol(MyProtocol): ...
|
```
```
info: revealed-type: Revealed type
--> /src/mdtest_snippet.py:13:1
|
11 | class SubclassOfMyProtocol(MyProtocol): ...
12 |
13 | reveal_type(SubclassOfMyProtocol()) # revealed: SubclassOfMyProtocol
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `SubclassOfMyProtocol`
14 | def f(x: type[MyProtocol]):
15 | reveal_type(x()) # revealed: MyProtocol
|
```
```
info: revealed-type: Revealed type
--> /src/mdtest_snippet.py:15:5
|
13 | reveal_type(SubclassOfMyProtocol()) # revealed: SubclassOfMyProtocol
14 | def f(x: type[MyProtocol]):
15 | reveal_type(x()) # revealed: MyProtocol
| ^^^^^^^^^^^^^^^^ `MyProtocol`
|
```

View File

@@ -1,191 +0,0 @@
# Suppressing errors with `knot: ignore`
Type check errors can be suppressed by a `knot: ignore` comment on the same line as the violation.
## Simple `knot: ignore`
```py
a = 4 + test # knot: ignore
```
## Suppressing a specific code
```py
a = 4 + test # knot: ignore[unresolved-reference]
```
## Unused suppression
```py
test = 10
# error: [unused-ignore-comment] "Unused `knot: ignore` directive: 'possibly-unresolved-reference'"
a = test + 3 # knot: ignore[possibly-unresolved-reference]
```
## Unused suppression if the error codes don't match
```py
# error: [unresolved-reference]
# error: [unused-ignore-comment] "Unused `knot: ignore` directive: 'possibly-unresolved-reference'"
a = test + 3 # knot: ignore[possibly-unresolved-reference]
```
## Suppressed unused comment
```py
# error: [unused-ignore-comment]
a = 10 / 2 # knot: ignore[division-by-zero]
a = 10 / 2 # knot: ignore[division-by-zero, unused-ignore-comment]
a = 10 / 2 # knot: ignore[unused-ignore-comment, division-by-zero]
a = 10 / 2 # knot: ignore[unused-ignore-comment] # type: ignore
a = 10 / 2 # type: ignore # knot: ignore[unused-ignore-comment]
```
## Unused ignore comment
```py
# error: [unused-ignore-comment] "Unused `knot: ignore` directive: 'unused-ignore-comment'"
a = 10 / 0 # knot: ignore[division-by-zero, unused-ignore-comment]
```
## Multiple unused comments
Today, Red Knot emits a diagnostic for every unused code. We might want to group the codes by
comment at some point in the future.
```py
# error: [unused-ignore-comment] "Unused `knot: ignore` directive: 'division-by-zero'"
# error: [unused-ignore-comment] "Unused `knot: ignore` directive: 'unresolved-reference'"
a = 10 / 2 # knot: ignore[division-by-zero, unresolved-reference]
# error: [unused-ignore-comment] "Unused `knot: ignore` directive: 'invalid-assignment'"
# error: [unused-ignore-comment] "Unused `knot: ignore` directive: 'unresolved-reference'"
a = 10 / 0 # knot: ignore[invalid-assignment, division-by-zero, unresolved-reference]
```
## Multiple suppressions
```py
# fmt: off
def test(a: f"f-string type annotation", b: b"byte-string-type-annotation"): ... # knot: ignore[fstring-type-annotation, byte-string-type-annotation]
```
## Can't suppress syntax errors
<!-- blacken-docs:off -->
```py
# error: [invalid-syntax]
# error: [unused-ignore-comment]
def test($): # knot: ignore
pass
```
<!-- blacken-docs:on -->
## Can't suppress `revealed-type` diagnostics
```py
a = 10
# revealed: Literal[10]
# error: [unknown-rule] "Unknown rule `revealed-type`"
reveal_type(a) # knot: ignore[revealed-type]
```
## Extra whitespace in type ignore comments is allowed
```py
a = 10 / 0 # knot : ignore
a = 10 / 0 # knot: ignore [ division-by-zero ]
```
## Whitespace is optional
```py
# fmt: off
a = 10 / 0 #knot:ignore[division-by-zero]
```
## Trailing codes comma
Trailing commas in the codes section are allowed:
```py
a = 10 / 0 # knot: ignore[division-by-zero,]
```
## Invalid characters in codes
```py
# error: [division-by-zero]
# error: [invalid-ignore-comment] "Invalid `knot: ignore` comment: expected a alphanumeric character or `-` or `_` as code"
a = 10 / 0 # knot: ignore[*-*]
```
## Trailing whitespace
<!-- blacken-docs:off -->
```py
a = 10 / 0 # knot: ignore[division-by-zero]
# ^^^^^^ trailing whitespace
```
<!-- blacken-docs:on -->
## Missing comma
A missing comma results in an invalid suppression comment. We may want to recover from this in the
future.
```py
# error: [unresolved-reference]
# error: [invalid-ignore-comment] "Invalid `knot: ignore` comment: expected a comma separating the rule codes"
a = x / 0 # knot: ignore[division-by-zero unresolved-reference]
```
## Missing closing bracket
```py
# error: [unresolved-reference] "Name `x` used when not defined"
# error: [invalid-ignore-comment] "Invalid `knot: ignore` comment: expected a comma separating the rule codes"
a = x / 2 # knot: ignore[unresolved-reference
```
## Empty codes
An empty codes array suppresses no-diagnostics and is always useless
```py
# error: [division-by-zero]
# error: [unused-ignore-comment] "Unused `knot: ignore` without a code"
a = 4 / 0 # knot: ignore[]
```
## File-level suppression comments
File level suppression comments are currently intentionally unsupported because we've yet to decide
if they should use a different syntax that also supports enabling rules or changing the rule's
severity: `knot: possibly-undefined-reference=error`
```py
# error: [unused-ignore-comment]
# knot: ignore[division-by-zero]
a = 4 / 0 # error: [division-by-zero]
```
## Unknown rule
```py
# error: [unknown-rule] "Unknown rule `is-equal-14`"
a = 10 + 4 # knot: ignore[is-equal-14]
```
## Code with `lint:` prefix
```py
# error:[unknown-rule] "Unknown rule `lint:division-by-zero`. Did you mean `division-by-zero`?"
# error: [division-by-zero]
a = 10 / 0 # knot: ignore[lint:division-by-zero]
```

View File

@@ -1,3 +0,0 @@
The `knot_extensions.pyi` file in this directory will be symlinked into
the `vendor/typeshed/stdlib` directory every time we sync our `typeshed`
stubs (see `.github/workflows/sync_typeshed.yaml`).

View File

@@ -1 +0,0 @@
f65bdc1acde54fda93c802459280da74518d2eef

View File

@@ -1,43 +0,0 @@
from typing import Any, Iterable, SupportsIndex, Protocol, TypeVar
class range:
def __init__(self, x): ...
def __iter__(self): ...
class object:
def __new__(cls): ...
def __init__(self) -> None: ...
@property
def __class__(self): ...
@property.setter
def __class__(self, value) -> None: ...
def __str__(self) -> str: ...
class int:
def __init__(self, x): ...
def __index__(self) -> int: ...
def __pow__(self, other: int, /) -> int: ...
class bool(int): ...
T = TypeVar("T")
class Iterable(Protocol[T]):
def __iter__(self): ...
class str(Iterable[str]): ...
class tuple:
def __len__(self) -> int: ...
class bytes: ...
class memoryview: ...
class type: ...
class list:
def __len__(self) -> int: ...
class property:
def setter(self, fset, /) -> property: ...
def isinstance(a, b) -> bool: ...
def issubclass(a, b) -> bool: ...

View File

@@ -1 +0,0 @@
def dataclass(func): ...

View File

@@ -1,3 +0,0 @@
class _version_info: ...
version_info: _version_info

View File

@@ -1,3 +0,0 @@
@final
class FunctionType: ...
class ModuleType: ...

View File

@@ -1,32 +0,0 @@
class Any: ...
def final(f: _T) -> _T: ...
class TypeVar:
def __new__(cls, name: str): ...
class _SpecialForm: ...
Generic: _SpecialForm
Protocol: _SpecialForm
ClassVar: _SpecialForm
Final: _SpecialForm
Literal: _SpecialForm
Callable: _SpecialForm
def runtime_checkable(cls: type) -> type: ...
@runtime_checkable
class SupportsIndex(Protocol):
def __index__(self) -> int: ...
@runtime_checkable
class Sized(Protocol):
def __len__(self) -> int: ...
def reveal_type(obj): ...
class NamedTuple: ...
def is_protocol(tp: type, /) -> bool: ...
def get_protocol_members(tp: type, /): ...

View File

@@ -1,4 +0,0 @@
from typing import *
from typing import _SpecialForm
TypeAlias: _SpecialForm

View File

@@ -1,6 +1,6 @@
[package]
name = "ruff"
version = "0.11.7"
version = "0.11.8"
publish = true
authors = { workspace = true }
edition = { workspace = true }

View File

@@ -13,7 +13,6 @@ fn main() {
commit_info(&workspace_root);
#[allow(clippy::disallowed_methods)]
let target = std::env::var("TARGET").unwrap();
println!("cargo::rustc-env=RUST_HOST_TARGET={target}");
}

View File

@@ -93,7 +93,7 @@ pub struct Args {
pub(crate) global_options: GlobalConfigArgs,
}
#[allow(clippy::large_enum_variant)]
#[expect(clippy::large_enum_variant)]
#[derive(Debug, clap::Subcommand)]
pub enum Command {
/// Run Ruff on the given files or directories.
@@ -177,11 +177,14 @@ pub struct AnalyzeGraphCommand {
/// The minimum Python version that should be supported.
#[arg(long, value_enum)]
target_version: Option<PythonVersion>,
/// Path to a virtual environment to use for resolving additional dependencies
#[arg(long)]
python: Option<PathBuf>,
}
// The `Parser` derive is for ruff_dev, for ruff `Args` would be sufficient
#[derive(Clone, Debug, clap::Parser)]
#[allow(clippy::struct_excessive_bools)]
#[expect(clippy::struct_excessive_bools)]
pub struct CheckCommand {
/// List of files or directories to check.
#[clap(help = "List of files or directories to check [default: .]")]
@@ -443,7 +446,7 @@ pub struct CheckCommand {
}
#[derive(Clone, Debug, clap::Parser)]
#[allow(clippy::struct_excessive_bools)]
#[expect(clippy::struct_excessive_bools)]
pub struct FormatCommand {
/// List of files or directories to format.
#[clap(help = "List of files or directories to format [default: .]")]
@@ -557,7 +560,7 @@ pub enum HelpFormat {
Json,
}
#[allow(clippy::module_name_repetitions)]
#[expect(clippy::module_name_repetitions)]
#[derive(Debug, Default, Clone, clap::Args)]
pub struct LogLevelArgs {
/// Enable verbose logging.
@@ -796,6 +799,7 @@ impl AnalyzeGraphCommand {
let format_arguments = AnalyzeGraphArgs {
files: self.files,
direction: self.direction,
python: self.python,
};
let cli_overrides = ExplicitConfigOverrides {
@@ -1027,7 +1031,7 @@ Possible choices:
/// CLI settings that are distinct from configuration (commands, lists of files,
/// etc.).
#[allow(clippy::struct_excessive_bools)]
#[expect(clippy::struct_excessive_bools)]
pub struct CheckArguments {
pub add_noqa: bool,
pub diff: bool,
@@ -1046,7 +1050,7 @@ pub struct CheckArguments {
/// CLI settings that are distinct from configuration (commands, lists of files,
/// etc.).
#[allow(clippy::struct_excessive_bools)]
#[expect(clippy::struct_excessive_bools)]
pub struct FormatArguments {
pub check: bool,
pub no_cache: bool,
@@ -1261,12 +1265,12 @@ impl LineColumnParseError {
pub struct AnalyzeGraphArgs {
pub files: Vec<PathBuf>,
pub direction: Direction,
pub python: Option<PathBuf>,
}
/// Configuration overrides provided via dedicated CLI flags:
/// `--line-length`, `--respect-gitignore`, etc.
#[derive(Clone, Default)]
#[allow(clippy::struct_excessive_bools)]
struct ExplicitConfigOverrides {
dummy_variable_rgx: Option<Regex>,
exclude: Option<Vec<FilePattern>>,

View File

@@ -86,7 +86,7 @@ pub(crate) struct Cache {
changes: Mutex<Vec<Change>>,
/// The "current" timestamp used as cache for the updates of
/// [`FileCache::last_seen`]
#[allow(clippy::struct_field_names)]
#[expect(clippy::struct_field_names)]
last_seen_cache: u64,
}
@@ -146,7 +146,7 @@ impl Cache {
Cache::new(path, package)
}
#[allow(clippy::cast_possible_truncation)]
#[expect(clippy::cast_possible_truncation)]
fn new(path: PathBuf, package: PackageCache) -> Self {
Cache {
path,
@@ -204,7 +204,7 @@ impl Cache {
}
/// Applies the pending changes without storing the cache to disk.
#[allow(clippy::cast_possible_truncation)]
#[expect(clippy::cast_possible_truncation)]
pub(crate) fn save(&mut self) -> bool {
/// Maximum duration for which we keep a file in cache that hasn't been seen.
const MAX_LAST_SEEN: Duration = Duration::from_secs(30 * 24 * 60 * 60); // 30 days.
@@ -616,7 +616,7 @@ mod tests {
let settings = Settings {
cache_dir,
linter: LinterSettings {
unresolved_target_version: PythonVersion::latest(),
unresolved_target_version: PythonVersion::latest().into(),
..Default::default()
},
..Settings::default()
@@ -834,7 +834,6 @@ mod tests {
// Regression test for issue #3086.
#[cfg(unix)]
#[allow(clippy::items_after_statements)]
fn flip_execute_permission_bit(path: &Path) -> io::Result<()> {
use std::os::unix::fs::PermissionsExt;
let file = fs::OpenOptions::new().write(true).open(path)?;
@@ -843,7 +842,6 @@ mod tests {
}
#[cfg(windows)]
#[allow(clippy::items_after_statements)]
fn flip_read_only_permission(path: &Path) -> io::Result<()> {
let file = fs::OpenOptions::new().write(true).open(path)?;
let mut perms = file.metadata()?.permissions();

View File

@@ -75,6 +75,8 @@ pub(crate) fn analyze_graph(
.target_version
.as_tuple()
.into(),
args.python
.and_then(|python| SystemPathBuf::from_path_buf(python).ok()),
)?;
let imports = {

View File

@@ -30,7 +30,6 @@ use crate::cache::{Cache, PackageCacheMap, PackageCaches};
use crate::diagnostics::Diagnostics;
/// Run the linter over a collection of files.
#[allow(clippy::too_many_arguments)]
pub(crate) fn check(
files: &[PathBuf],
pyproject_config: &PyprojectConfig,
@@ -181,7 +180,6 @@ pub(crate) fn check(
/// Wraps [`lint_path`](crate::diagnostics::lint_path) in a [`catch_unwind`](std::panic::catch_unwind) and emits
/// a diagnostic if the linting the file panics.
#[allow(clippy::too_many_arguments)]
fn lint_path(
path: &Path,
package: Option<PackageRoot<'_>>,

View File

@@ -5,7 +5,7 @@ use crate::args::HelpFormat;
use ruff_workspace::options::Options;
use ruff_workspace::options_base::OptionsMetadata;
#[allow(clippy::print_stdout)]
#[expect(clippy::print_stdout)]
pub(crate) fn config(key: Option<&str>, format: HelpFormat) -> Result<()> {
match key {
None => {

View File

@@ -160,7 +160,7 @@ pub(crate) fn format(
}),
Err(error) => Err(FormatCommandError::Panic(
Some(resolved_file.path().to_path_buf()),
error,
Box::new(error),
)),
},
)
@@ -362,7 +362,7 @@ pub(crate) fn format_source(
})
} else {
// Using `Printed::into_code` requires adding `ruff_formatter` as a direct dependency, and I suspect that Rust can optimize the closure away regardless.
#[allow(clippy::redundant_closure_for_method_calls)]
#[expect(clippy::redundant_closure_for_method_calls)]
format_module_source(unformatted, options).map(|formatted| formatted.into_code())
};
@@ -635,7 +635,7 @@ impl<'a> FormatResults<'a> {
pub(crate) enum FormatCommandError {
Ignore(#[from] ignore::Error),
Parse(#[from] DisplayParseError),
Panic(Option<PathBuf>, PanicError),
Panic(Option<PathBuf>, Box<PanicError>),
Read(Option<PathBuf>, SourceError),
Format(Option<PathBuf>, FormatModuleError),
Write(Option<PathBuf>, SourceError),

View File

@@ -19,7 +19,7 @@ struct Explanation<'a> {
summary: &'a str,
message_formats: &'a [&'a str],
fix: String,
#[allow(clippy::struct_field_names)]
#[expect(clippy::struct_field_names)]
explanation: Option<&'a str>,
preview: bool,
}

View File

@@ -134,7 +134,7 @@ pub fn run(
{
let default_panic_hook = std::panic::take_hook();
std::panic::set_hook(Box::new(move |info| {
#[allow(clippy::print_stderr)]
#[expect(clippy::print_stderr)]
{
eprintln!(
r#"
@@ -326,7 +326,7 @@ pub fn check(args: CheckCommand, global_options: GlobalConfigArgs) -> Result<Exi
commands::add_noqa::add_noqa(&files, &pyproject_config, &config_arguments)?;
if modifications > 0 && config_arguments.log_level >= LogLevel::Default {
let s = if modifications == 1 { "" } else { "s" };
#[allow(clippy::print_stderr)]
#[expect(clippy::print_stderr)]
{
eprintln!("Added {modifications} noqa directive{s}.");
}

View File

@@ -241,7 +241,6 @@ impl Printer {
}
if !self.flags.intersects(Flags::SHOW_VIOLATIONS) {
#[allow(deprecated)]
if matches!(
self.format,
OutputFormat::Full | OutputFormat::Concise | OutputFormat::Grouped

View File

@@ -422,3 +422,153 @@ fn nested_imports() -> Result<()> {
Ok(())
}
/// Test for venv resolution with the `--python` flag.
///
/// Based on the [albatross-virtual-workspace] example from the uv repo and the report in [#16598].
///
/// [albatross-virtual-workspace]: https://github.com/astral-sh/uv/tree/aa629c4a/scripts/workspaces/albatross-virtual-workspace
/// [#16598]: https://github.com/astral-sh/ruff/issues/16598
#[test]
fn venv() -> Result<()> {
let tempdir = TempDir::new()?;
let root = ChildPath::new(tempdir.path());
// packages
// ├── albatross
// │ ├── check_installed_albatross.py
// │ ├── pyproject.toml
// │ └── src
// │ └── albatross
// │ └── __init__.py
// └── bird-feeder
// ├── check_installed_bird_feeder.py
// ├── pyproject.toml
// └── src
// └── bird_feeder
// └── __init__.py
let packages = root.child("packages");
let albatross = packages.child("albatross");
albatross
.child("check_installed_albatross.py")
.write_str("from albatross import fly")?;
albatross
.child("pyproject.toml")
.write_str(indoc::indoc! {r#"
[project]
name = "albatross"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = ["bird-feeder", "tqdm>=4,<5"]
[tool.uv.sources]
bird-feeder = { workspace = true }
"#})?;
albatross
.child("src")
.child("albatross")
.child("__init__.py")
.write_str("import tqdm; from bird_feeder import use")?;
let bird_feeder = packages.child("bird-feeder");
bird_feeder
.child("check_installed_bird_feeder.py")
.write_str("from bird_feeder import use; from albatross import fly")?;
bird_feeder
.child("pyproject.toml")
.write_str(indoc::indoc! {r#"
[project]
name = "bird-feeder"
version = "1.0.0"
requires-python = ">=3.12"
dependencies = ["anyio>=4.3.0,<5"]
"#})?;
bird_feeder
.child("src")
.child("bird_feeder")
.child("__init__.py")
.write_str("import anyio")?;
let venv = root.child(".venv");
let bin = venv.child("bin");
bin.child("python").touch()?;
let home = format!("home = {}", bin.to_string_lossy());
venv.child("pyvenv.cfg").write_str(&home)?;
let site_packages = venv.child("lib").child("python3.12").child("site-packages");
site_packages
.child("_albatross.pth")
.write_str(&albatross.join("src").to_string_lossy())?;
site_packages
.child("_bird_feeder.pth")
.write_str(&bird_feeder.join("src").to_string_lossy())?;
site_packages.child("tqdm").child("__init__.py").touch()?;
// without `--python .venv`, the result should only include dependencies within the albatross
// package
insta::with_settings!({
filters => INSTA_FILTERS.to_vec(),
}, {
assert_cmd_snapshot!(
command().arg("packages/albatross").current_dir(&root),
@r#"
success: true
exit_code: 0
----- stdout -----
{
"packages/albatross/check_installed_albatross.py": [
"packages/albatross/src/albatross/__init__.py"
],
"packages/albatross/src/albatross/__init__.py": []
}
----- stderr -----
"#);
});
// with `--python .venv` both workspace and third-party dependencies are included
insta::with_settings!({
filters => INSTA_FILTERS.to_vec(),
}, {
assert_cmd_snapshot!(
command().args(["--python", ".venv"]).arg("packages/albatross").current_dir(&root),
@r#"
success: true
exit_code: 0
----- stdout -----
{
"packages/albatross/check_installed_albatross.py": [
"packages/albatross/src/albatross/__init__.py"
],
"packages/albatross/src/albatross/__init__.py": [
".venv/lib/python3.12/site-packages/tqdm/__init__.py",
"packages/bird-feeder/src/bird_feeder/__init__.py"
]
}
----- stderr -----
"#);
});
// test the error message for a non-existent venv. it's important that the `ruff analyze graph`
// flag matches the ty flag used to generate the error message (`--python`)
insta::with_settings!({
filters => INSTA_FILTERS.to_vec(),
}, {
assert_cmd_snapshot!(
command().args(["--python", "none"]).arg("packages/albatross").current_dir(&root),
@r"
success: false
exit_code: 2
----- stdout -----
----- stderr -----
ruff failed
Cause: Invalid search path settings
Cause: Failed to discover the site-packages directory: Invalid `--python` argument: `none` could not be canonicalized
");
});
Ok(())
}

View File

@@ -994,6 +994,7 @@ fn value_given_to_table_key_is_not_inline_table_2() {
- `lint.extend-per-file-ignores`
- `lint.exclude`
- `lint.preview`
- `lint.typing-extensions`
For more information, try '--help'.
");
@@ -1899,6 +1900,40 @@ def first_square():
Ok(())
}
/// Regression test for <https://github.com/astral-sh/ruff/issues/2253>
#[test]
fn add_noqa_parent() -> Result<()> {
let tempdir = TempDir::new()?;
let test_path = tempdir.path().join("noqa.py");
fs::write(
&test_path,
r#"
from foo import ( # noqa: F401
bar
)
"#,
)?;
insta::with_settings!({
filters => vec![(tempdir_filter(&tempdir).as_str(), "[TMP]/")]
}, {
assert_cmd_snapshot!(Command::new(get_cargo_bin(BIN_NAME))
.args(STDIN_BASE_OPTIONS)
.arg("--add-noqa")
.arg("--select=F401")
.arg("noqa.py")
.current_dir(&tempdir), @r"
success: true
exit_code: 0
----- stdout -----
----- stderr -----
");
});
Ok(())
}
/// Infer `3.11` from `requires-python` in `pyproject.toml`.
#[test]
fn requires_python() -> Result<()> {
@@ -2117,7 +2152,7 @@ requires-python = ">= 3.11"
.arg("test.py")
.arg("-")
.current_dir(project_dir)
, @r###"
, @r#"
success: true
exit_code: 0
----- stdout -----
@@ -2207,6 +2242,7 @@ requires-python = ">= 3.11"
XXX,
]
linter.typing_modules = []
linter.typing_extensions = true
# Linter Plugins
linter.flake8_annotations.mypy_init_return = false
@@ -2390,7 +2426,7 @@ requires-python = ">= 3.11"
analyze.include_dependencies = {}
----- stderr -----
"###);
"#);
});
Ok(())
}
@@ -2428,7 +2464,7 @@ requires-python = ">= 3.11"
.arg("test.py")
.arg("-")
.current_dir(project_dir)
, @r###"
, @r#"
success: true
exit_code: 0
----- stdout -----
@@ -2518,6 +2554,7 @@ requires-python = ">= 3.11"
XXX,
]
linter.typing_modules = []
linter.typing_extensions = true
# Linter Plugins
linter.flake8_annotations.mypy_init_return = false
@@ -2701,7 +2738,7 @@ requires-python = ">= 3.11"
analyze.include_dependencies = {}
----- stderr -----
"###);
"#);
});
Ok(())
}
@@ -2790,7 +2827,7 @@ from typing import Union;foo: Union[int, str] = 1
.args(STDIN_BASE_OPTIONS)
.arg("test.py")
.arg("--show-settings")
.current_dir(project_dir), @r###"
.current_dir(project_dir), @r#"
success: true
exit_code: 0
----- stdout -----
@@ -2881,6 +2918,7 @@ from typing import Union;foo: Union[int, str] = 1
XXX,
]
linter.typing_modules = []
linter.typing_extensions = true
# Linter Plugins
linter.flake8_annotations.mypy_init_return = false
@@ -3064,7 +3102,7 @@ from typing import Union;foo: Union[int, str] = 1
analyze.include_dependencies = {}
----- stderr -----
"###);
"#);
});
Ok(())
}
@@ -3170,7 +3208,7 @@ from typing import Union;foo: Union[int, str] = 1
.arg("--show-settings")
.args(["--select","UP007"])
.arg("foo/test.py")
.current_dir(&project_dir), @r###"
.current_dir(&project_dir), @r#"
success: true
exit_code: 0
----- stdout -----
@@ -3260,6 +3298,7 @@ from typing import Union;foo: Union[int, str] = 1
XXX,
]
linter.typing_modules = []
linter.typing_extensions = true
# Linter Plugins
linter.flake8_annotations.mypy_init_return = false
@@ -3443,7 +3482,7 @@ from typing import Union;foo: Union[int, str] = 1
analyze.include_dependencies = {}
----- stderr -----
"###);
"#);
});
Ok(())
}
@@ -3475,7 +3514,7 @@ requires-python = ">= 3.11"
&inner_pyproject,
r#"
[tool.ruff]
target-version = "py310"
target-version = "py310"
"#,
)?;
@@ -3497,7 +3536,7 @@ from typing import Union;foo: Union[int, str] = 1
.arg("--show-settings")
.args(["--select","UP007"])
.arg("foo/test.py")
.current_dir(&project_dir), @r###"
.current_dir(&project_dir), @r#"
success: true
exit_code: 0
----- stdout -----
@@ -3587,6 +3626,7 @@ from typing import Union;foo: Union[int, str] = 1
XXX,
]
linter.typing_modules = []
linter.typing_extensions = true
# Linter Plugins
linter.flake8_annotations.mypy_init_return = false
@@ -3770,7 +3810,7 @@ from typing import Union;foo: Union[int, str] = 1
analyze.include_dependencies = {}
----- stderr -----
"###);
"#);
});
Ok(())
}
@@ -3823,7 +3863,7 @@ from typing import Union;foo: Union[int, str] = 1
.args(STDIN_BASE_OPTIONS)
.arg("--show-settings")
.arg("foo/test.py")
.current_dir(&project_dir), @r###"
.current_dir(&project_dir), @r#"
success: true
exit_code: 0
----- stdout -----
@@ -3890,7 +3930,7 @@ from typing import Union;foo: Union[int, str] = 1
linter.per_file_ignores = {}
linter.safety_table.forced_safe = []
linter.safety_table.forced_unsafe = []
linter.unresolved_target_version = 3.9
linter.unresolved_target_version = none
linter.per_file_target_version = {}
linter.preview = disabled
linter.explicit_preview_rules = false
@@ -3914,6 +3954,7 @@ from typing import Union;foo: Union[int, str] = 1
XXX,
]
linter.typing_modules = []
linter.typing_extensions = true
# Linter Plugins
linter.flake8_annotations.mypy_init_return = false
@@ -4097,7 +4138,7 @@ from typing import Union;foo: Union[int, str] = 1
analyze.include_dependencies = {}
----- stderr -----
"###);
"#);
});
insta::with_settings!({
@@ -4107,7 +4148,7 @@ from typing import Union;foo: Union[int, str] = 1
.args(STDIN_BASE_OPTIONS)
.arg("--show-settings")
.arg("test.py")
.current_dir(project_dir.join("foo")), @r###"
.current_dir(project_dir.join("foo")), @r#"
success: true
exit_code: 0
----- stdout -----
@@ -4174,7 +4215,7 @@ from typing import Union;foo: Union[int, str] = 1
linter.per_file_ignores = {}
linter.safety_table.forced_safe = []
linter.safety_table.forced_unsafe = []
linter.unresolved_target_version = 3.9
linter.unresolved_target_version = none
linter.per_file_target_version = {}
linter.preview = disabled
linter.explicit_preview_rules = false
@@ -4198,6 +4239,7 @@ from typing import Union;foo: Union[int, str] = 1
XXX,
]
linter.typing_modules = []
linter.typing_extensions = true
# Linter Plugins
linter.flake8_annotations.mypy_init_return = false
@@ -4381,7 +4423,7 @@ from typing import Union;foo: Union[int, str] = 1
analyze.include_dependencies = {}
----- stderr -----
"###);
"#);
});
Ok(())
}
@@ -4444,7 +4486,7 @@ from typing import Union;foo: Union[int, str] = 1
.args(STDIN_BASE_OPTIONS)
.arg("--show-settings")
.arg("test.py")
.current_dir(&project_dir), @r###"
.current_dir(&project_dir), @r#"
success: true
exit_code: 0
----- stdout -----
@@ -4535,6 +4577,7 @@ from typing import Union;foo: Union[int, str] = 1
XXX,
]
linter.typing_modules = []
linter.typing_extensions = true
# Linter Plugins
linter.flake8_annotations.mypy_init_return = false
@@ -4718,7 +4761,7 @@ from typing import Union;foo: Union[int, str] = 1
analyze.include_dependencies = {}
----- stderr -----
"###);
"#);
});
Ok(())
@@ -4980,6 +5023,53 @@ fn flake8_import_convention_unused_aliased_import_no_conflict() {
.pass_stdin("1"));
}
// See: https://github.com/astral-sh/ruff/issues/16177
#[test]
fn flake8_pyi_redundant_none_literal() {
let snippet = r#"
from typing import Literal
# For each of these expressions, Ruff provides a fix for one of the `Literal[None]` elements
# but not both, as if both were autofixed it would result in `None | None`,
# which leads to a `TypeError` at runtime.
a: Literal[None,] | Literal[None,]
b: Literal[None] | Literal[None]
c: Literal[None] | Literal[None,]
d: Literal[None,] | Literal[None]
"#;
assert_cmd_snapshot!(Command::new(get_cargo_bin(BIN_NAME))
.args(STDIN_BASE_OPTIONS)
.args(["--select", "PYI061"])
.args(["--stdin-filename", "test.py"])
.arg("--preview")
.arg("--diff")
.arg("-")
.pass_stdin(snippet), @r"
success: false
exit_code: 1
----- stdout -----
--- test.py
+++ test.py
@@ -4,7 +4,7 @@
# For each of these expressions, Ruff provides a fix for one of the `Literal[None]` elements
# but not both, as if both were autofixed it would result in `None | None`,
# which leads to a `TypeError` at runtime.
-a: Literal[None,] | Literal[None,]
-b: Literal[None] | Literal[None]
-c: Literal[None] | Literal[None,]
-d: Literal[None,] | Literal[None]
+a: None | Literal[None,]
+b: None | Literal[None]
+c: None | Literal[None,]
+d: None | Literal[None]
----- stderr -----
Would fix 4 errors.
");
}
/// Test that private, old-style `TypeVar` generics
/// 1. Get replaced with PEP 695 type parameters (UP046, UP047)
/// 2. Get renamed to remove leading underscores (UP049)
@@ -5564,3 +5654,34 @@ fn semantic_syntax_errors() -> Result<()> {
Ok(())
}
/// Regression test for <https://github.com/astral-sh/ruff/issues/17821>.
///
/// `lint.typing-extensions = false` with Python 3.9 should disable the PYI019 lint because it would
/// try to import `Self` from `typing_extensions`
#[test]
fn combine_typing_extensions_config() {
let contents = "
from typing import TypeVar
T = TypeVar('T')
class Foo:
def f(self: T) -> T: ...
";
assert_cmd_snapshot!(
Command::new(get_cargo_bin(BIN_NAME))
.args(STDIN_BASE_OPTIONS)
.args(["--config", "lint.typing-extensions = false"])
.arg("--select=PYI019")
.arg("--target-version=py39")
.arg("-")
.pass_stdin(contents),
@r"
success: true
exit_code: 0
----- stdout -----
All checks passed!
----- stderr -----
"
);
}

View File

@@ -5,7 +5,6 @@ info:
args:
- rule
- F401
snapshot_kind: text
---
success: true
exit_code: 0
@@ -84,6 +83,11 @@ else:
print("numpy is not installed")
```
## Preview
When [preview](https://docs.astral.sh/ruff/preview/) is enabled,
the criterion for determining whether an import is first-party
is stricter, which could affect the suggested fix. See [this FAQ section](https://docs.astral.sh/ruff/faq/#how-does-ruff-determine-which-of-my-imports-are-first-party-third-party-etc) for more details.
## Options
- `lint.ignore-init-module-imports`
- `lint.pyflakes.allowed-unused-imports`

View File

@@ -213,6 +213,7 @@ linter.task_tags = [
XXX,
]
linter.typing_modules = []
linter.typing_extensions = true
# Linter Plugins
linter.flake8_annotations.mypy_init_return = false

View File

@@ -33,7 +33,7 @@ name = "formatter"
harness = false
[[bench]]
name = "red_knot"
name = "ty"
harness = false
[dependencies]
@@ -49,7 +49,7 @@ ruff_python_ast = { workspace = true }
ruff_python_formatter = { workspace = true }
ruff_python_parser = { workspace = true }
ruff_python_trivia = { workspace = true }
red_knot_project = { workspace = true }
ty_project = { workspace = true }
[lints]
workspace = true

View File

@@ -45,9 +45,9 @@ static GLOBAL: tikv_jemallocator::Jemalloc = tikv_jemallocator::Jemalloc;
target_arch = "powerpc64"
)
))]
#[allow(non_upper_case_globals)]
#[expect(non_upper_case_globals)]
#[export_name = "_rjem_malloc_conf"]
#[allow(unsafe_code)]
#[expect(unsafe_code)]
pub static _rjem_malloc_conf: &[u8] = b"dirty_decay_ms:-1,muzzy_decay_ms:-1\0";
fn create_test_cases() -> Vec<TestCase> {

View File

@@ -7,16 +7,16 @@ use criterion::{criterion_group, criterion_main, BatchSize, Criterion};
use rayon::ThreadPoolBuilder;
use rustc_hash::FxHashSet;
use red_knot_project::metadata::options::{EnvironmentOptions, Options};
use red_knot_project::metadata::value::RangedValue;
use red_knot_project::watch::{ChangeEvent, ChangedKind};
use red_knot_project::{Db, ProjectDatabase, ProjectMetadata};
use ruff_benchmark::TestFile;
use ruff_db::diagnostic::{Diagnostic, DiagnosticId, Severity};
use ruff_db::files::{system_path_to_file, File};
use ruff_db::source::source_text;
use ruff_db::system::{MemoryFileSystem, SystemPath, SystemPathBuf, TestSystem};
use ruff_python_ast::PythonVersion;
use ty_project::metadata::options::{EnvironmentOptions, Options};
use ty_project::metadata::value::RangedValue;
use ty_project::watch::{ChangeEvent, ChangedKind};
use ty_project::{Db, ProjectDatabase, ProjectMetadata};
struct Case {
db: ProjectDatabase,
@@ -122,7 +122,7 @@ static RAYON_INITIALIZED: std::sync::Once = std::sync::Once::new();
fn setup_rayon() {
// Initialize the rayon thread pool outside the benchmark because it has a significant cost.
// We limit the thread pool to only one (the current thread) because we're focused on
// where red knot spends time and less about how well the code runs concurrently.
// where ty spends time and less about how well the code runs concurrently.
// We might want to add a benchmark focusing on concurrency to detect congestion in the future.
RAYON_INITIALIZED.call_once(|| {
ThreadPoolBuilder::new()
@@ -172,7 +172,7 @@ fn benchmark_incremental(criterion: &mut Criterion) {
setup_rayon();
criterion.bench_function("red_knot_check_file[incremental]", |b| {
criterion.bench_function("ty_check_file[incremental]", |b| {
b.iter_batched_ref(setup, incremental, BatchSize::SmallInput);
});
}
@@ -180,7 +180,7 @@ fn benchmark_incremental(criterion: &mut Criterion) {
fn benchmark_cold(criterion: &mut Criterion) {
setup_rayon();
criterion.bench_function("red_knot_check_file[cold]", |b| {
criterion.bench_function("ty_check_file[cold]", |b| {
b.iter_batched_ref(
setup_tomllib_case,
|case| {
@@ -257,7 +257,7 @@ fn setup_micro_case(code: &str) -> Case {
fn benchmark_many_string_assignments(criterion: &mut Criterion) {
setup_rayon();
criterion.bench_function("red_knot_micro[many_string_assignments]", |b| {
criterion.bench_function("ty_micro[many_string_assignments]", |b| {
b.iter_batched_ref(
|| {
// This is a micro benchmark, but it is effectively identical to a code sample

View File

@@ -213,7 +213,7 @@ macro_rules! impl_cache_key_tuple {
( $($name:ident)+) => (
impl<$($name: CacheKey),+> CacheKey for ($($name,)+) where last_type!($($name,)+): ?Sized {
#[allow(non_snake_case)]
#[expect(non_snake_case)]
#[inline]
fn cache_key(&self, state: &mut CacheKeyHasher) {
let ($(ref $name,)+) = *self;

View File

@@ -47,7 +47,7 @@ fn struct_ignored_fields() {
struct NamedFieldsStruct {
a: String,
#[cache_key(ignore)]
#[allow(unused)]
#[expect(unused)]
b: String,
}

View File

@@ -20,6 +20,7 @@ ruff_python_trivia = { workspace = true }
ruff_source_file = { workspace = true }
ruff_text_size = { workspace = true }
anstyle = { workspace = true }
camino = { workspace = true }
countme = { workspace = true }
dashmap = { workspace = true }

View File

@@ -11,6 +11,7 @@ use crate::Db;
use self::render::FileResolver;
mod render;
mod stylesheet;
/// A collection of information that can be rendered into a diagnostic.
///
@@ -133,20 +134,20 @@ impl Diagnostic {
/// NOTE: At present, this routine will return the first primary
/// annotation's message as the primary message when the main diagnostic
/// message is empty. This is meant to facilitate an incremental migration
/// in Red Knot over to the new diagnostic data model. (The old data model
/// in ty over to the new diagnostic data model. (The old data model
/// didn't distinguish between messages on the entire diagnostic and
/// messages attached to a particular span.)
pub fn primary_message(&self) -> &str {
if !self.inner.message.as_str().is_empty() {
return self.inner.message.as_str();
}
// FIXME: As a special case, while we're migrating Red Knot
// FIXME: As a special case, while we're migrating ty
// to the new diagnostic data model, we'll look for a primary
// message from the primary annotation. This is because most
// Red Knot diagnostics are created with an empty diagnostic
// ty diagnostics are created with an empty diagnostic
// message and instead attach the message to the annotation.
// Fixing this will require touching basically every diagnostic
// in Red Knot, so we do it this way for now to match the old
// in ty, so we do it this way for now to match the old
// semantics. ---AG
self.primary_annotation()
.and_then(|ann| ann.get_message())
@@ -164,7 +165,7 @@ impl Diagnostic {
///
/// The reason why we don't just always return both the main diagnostic
/// message and the primary annotation message is because this was written
/// in the midst of an incremental migration of Red Knot over to the new
/// in the midst of an incremental migration of ty over to the new
/// diagnostic data model. At time of writing, diagnostics were still
/// constructed in the old model where the main diagnostic message and the
/// primary annotation message were not distinguished from each other. So
@@ -226,6 +227,20 @@ impl Diagnostic {
pub fn primary_span(&self) -> Option<Span> {
self.primary_annotation().map(|ann| ann.span.clone())
}
/// Returns the tags from the primary annotation of this diagnostic if it exists.
pub fn primary_tags(&self) -> Option<&[DiagnosticTag]> {
self.primary_annotation().map(|ann| ann.tags.as_slice())
}
/// Returns a key that can be used to sort two diagnostics into the canonical order
/// in which they should appear when rendered.
pub fn rendering_sort_key<'a>(&'a self, db: &'a dyn Db) -> impl Ord + 'a {
RenderingSortKey {
db,
diagnostic: self,
}
}
}
#[derive(Debug, Clone, Eq, PartialEq)]
@@ -237,6 +252,64 @@ struct DiagnosticInner {
subs: Vec<SubDiagnostic>,
}
struct RenderingSortKey<'a> {
db: &'a dyn Db,
diagnostic: &'a Diagnostic,
}
impl Ord for RenderingSortKey<'_> {
// We sort diagnostics in a way that keeps them in source order
// and grouped by file. After that, we fall back to severity
// (with fatal messages sorting before info messages) and then
// finally the diagnostic ID.
fn cmp(&self, other: &Self) -> std::cmp::Ordering {
if let (Some(span1), Some(span2)) = (
self.diagnostic.primary_span(),
other.diagnostic.primary_span(),
) {
let order = span1
.file()
.path(self.db)
.as_str()
.cmp(span2.file().path(self.db).as_str());
if order.is_ne() {
return order;
}
if let (Some(range1), Some(range2)) = (span1.range(), span2.range()) {
let order = range1.start().cmp(&range2.start());
if order.is_ne() {
return order;
}
}
}
// Reverse so that, e.g., Fatal sorts before Info.
let order = self
.diagnostic
.severity()
.cmp(&other.diagnostic.severity())
.reverse();
if order.is_ne() {
return order;
}
self.diagnostic.id().cmp(&other.diagnostic.id())
}
}
impl PartialOrd for RenderingSortKey<'_> {
fn partial_cmp(&self, other: &Self) -> Option<std::cmp::Ordering> {
Some(self.cmp(other))
}
}
impl PartialEq for RenderingSortKey<'_> {
fn eq(&self, other: &Self) -> bool {
self.cmp(other).is_eq()
}
}
impl Eq for RenderingSortKey<'_> {}
/// A collection of information subservient to a diagnostic.
///
/// A sub-diagnostic is always rendered after the parent diagnostic it is
@@ -337,6 +410,8 @@ pub struct Annotation {
/// Whether this annotation is "primary" or not. When it isn't primary, an
/// annotation is said to be "secondary."
is_primary: bool,
/// The diagnostic tags associated with this annotation.
tags: Vec<DiagnosticTag>,
}
impl Annotation {
@@ -354,6 +429,7 @@ impl Annotation {
span,
message: None,
is_primary: true,
tags: Vec::new(),
}
}
@@ -369,6 +445,7 @@ impl Annotation {
span,
message: None,
is_primary: false,
tags: Vec::new(),
}
}
@@ -411,6 +488,36 @@ impl Annotation {
pub fn get_span(&self) -> &Span {
&self.span
}
/// Returns the tags associated with this annotation.
pub fn get_tags(&self) -> &[DiagnosticTag] {
&self.tags
}
/// Attaches this tag to this annotation.
///
/// It will not replace any existing tags.
pub fn tag(mut self, tag: DiagnosticTag) -> Annotation {
self.tags.push(tag);
self
}
/// Attaches an additional tag to this annotation.
pub fn push_tag(&mut self, tag: DiagnosticTag) {
self.tags.push(tag);
}
}
/// Tags that can be associated with an annotation.
///
/// These tags are used to provide additional information about the annotation.
/// and are passed through to the language server protocol.
#[derive(Debug, Clone, Eq, PartialEq)]
pub enum DiagnosticTag {
/// Unused or unnecessary code. Used for unused parameters, unreachable code, etc.
Unnecessary,
/// Deprecated or obsolete code.
Deprecated,
}
/// A string identifier for a lint rule.
@@ -634,6 +741,10 @@ impl Severity {
Severity::Fatal => AnnotateLevel::Error,
}
}
pub const fn is_fatal(self) -> bool {
matches!(self, Severity::Fatal)
}
}
/// Configuration for rendering diagnostics.

View File

@@ -7,9 +7,11 @@ use ruff_annotate_snippets::{
use ruff_source_file::{LineIndex, OneIndexed, SourceCode};
use ruff_text_size::{TextRange, TextSize};
use crate::diagnostic::stylesheet::{fmt_styled, DiagnosticStylesheet};
use crate::{
files::File,
source::{line_index, source_text, SourceText},
system::SystemPath,
Db,
};
@@ -47,6 +49,7 @@ impl<'a> DisplayDiagnostic<'a> {
} else {
AnnotateRenderer::plain()
};
DisplayDiagnostic {
config,
resolver,
@@ -58,31 +61,64 @@ impl<'a> DisplayDiagnostic<'a> {
impl std::fmt::Display for DisplayDiagnostic<'_> {
fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
if matches!(self.config.format, DiagnosticFormat::Concise) {
match self.diag.severity() {
Severity::Info => f.write_str("info")?,
Severity::Warning => f.write_str("warning")?,
Severity::Error => f.write_str("error")?,
Severity::Fatal => f.write_str("fatal")?,
}
let stylesheet = if self.config.color {
DiagnosticStylesheet::styled()
} else {
DiagnosticStylesheet::plain()
};
if matches!(self.config.format, DiagnosticFormat::Concise) {
let (severity, severity_style) = match self.diag.severity() {
Severity::Info => ("info", stylesheet.info),
Severity::Warning => ("warning", stylesheet.warning),
Severity::Error => ("error", stylesheet.error),
Severity::Fatal => ("fatal", stylesheet.error),
};
write!(
f,
"{severity}[{id}]",
severity = fmt_styled(severity, severity_style),
id = fmt_styled(self.diag.id(), stylesheet.emphasis)
)?;
write!(f, "[{rule}]", rule = self.diag.id())?;
if let Some(span) = self.diag.primary_span() {
write!(f, " {path}", path = self.resolver.path(span.file()))?;
write!(
f,
" {path}",
path = fmt_styled(self.resolver.path(span.file()), stylesheet.emphasis)
)?;
if let Some(range) = span.range() {
let input = self.resolver.input(span.file());
let start = input.as_source_code().line_column(range.start());
write!(f, ":{line}:{col}", line = start.line, col = start.column)?;
write!(
f,
":{line}:{col}",
line = fmt_styled(start.line, stylesheet.emphasis),
col = fmt_styled(start.column, stylesheet.emphasis),
)?;
}
write!(f, ":")?;
}
return writeln!(f, " {}", self.diag.concise_message());
return writeln!(f, " {message}", message = self.diag.concise_message());
}
let mut renderer = self.annotate_renderer.clone();
renderer = renderer
.error(stylesheet.error)
.warning(stylesheet.warning)
.info(stylesheet.info)
.note(stylesheet.note)
.help(stylesheet.help)
.line_no(stylesheet.line_no)
.emphasis(stylesheet.emphasis)
.none(stylesheet.none);
let resolved = Resolved::new(&self.resolver, self.diag);
let renderable = resolved.to_renderable(self.config.context);
for diag in renderable.diagnostics.iter() {
writeln!(f, "{}", self.annotate_renderer.render(diag.to_annotate()))?;
writeln!(f, "{}", renderer.render(diag.to_annotate()))?;
}
writeln!(f)
}
@@ -340,7 +376,7 @@ struct Renderable<'r> {
// (At time of writing, 2025-03-13, we currently render the diagnostic
// ID into the main message of the parent diagnostic. We don't use this
// specific field to do that though.)
#[allow(dead_code)]
#[expect(dead_code)]
id: &'r str,
diagnostics: Vec<RenderableDiagnostic<'r>>,
}
@@ -588,7 +624,7 @@ impl<'r> RenderableAnnotation<'r> {
/// For example, at time of writing (2025-03-07), the plan is (roughly) for
/// Ruff to grow its own interner of file paths so that a `Span` can store an
/// interned ID instead of a (roughly) `Arc<Path>`. This interner is planned
/// to be entirely separate from the Salsa interner used by Red Knot, and so,
/// to be entirely separate from the Salsa interner used by ty, and so,
/// callers will need to pass in a different "resolver" for turning `Span`s
/// into actual file paths/contents. The infrastructure for this isn't fully in
/// place, but this type serves to demarcate the intended abstraction boundary.
@@ -604,7 +640,10 @@ impl<'a> FileResolver<'a> {
/// Returns the path associated with the file given.
fn path(&self, file: File) -> &'a str {
file.path(self.db).as_str()
relativize_path(
self.db.system().current_directory(),
file.path(self.db).as_str(),
)
}
/// Returns the input contents associated with the file given.
@@ -676,6 +715,14 @@ fn context_after(source: &SourceCode<'_, '_>, len: usize, start: OneIndexed) ->
line
}
/// Convert an absolute path to be relative to the current working directory.
fn relativize_path<'p>(cwd: &SystemPath, path: &'p str) -> &'p str {
if let Ok(path) = SystemPath::new(path).strip_prefix(cwd) {
return path.as_str();
}
path
}
#[cfg(test)]
mod tests {
@@ -757,7 +804,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:5:1
--> animals:5:1
|
3 | canary
4 | dog
@@ -781,7 +828,7 @@ watermelon
env.render(&diag),
@r"
warning: lint:test-diagnostic: main diagnostic message
--> /animals:5:1
--> animals:5:1
|
3 | canary
4 | dog
@@ -801,7 +848,7 @@ watermelon
env.render(&diag),
@r"
info: lint:test-diagnostic: main diagnostic message
--> /animals:5:1
--> animals:5:1
|
3 | canary
4 | dog
@@ -828,7 +875,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:1:1
--> animals:1:1
|
1 | aardvark
| ^
@@ -847,7 +894,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:1:1
--> animals:1:1
|
1 | aardvark
| ^ primary annotation message
@@ -868,7 +915,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /non-ascii:5:1
--> non-ascii:5:1
|
3 | ΔΔΔΔΔΔΔΔΔΔΔΔ
4 | ββββββββββββ
@@ -887,7 +934,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /non-ascii:2:2
--> non-ascii:2:2
|
1 | ☃☃☃☃☃☃☃☃☃☃☃☃
2 | 💩💩💩💩💩💩💩💩💩💩💩💩
@@ -911,7 +958,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:5:1
--> animals:5:1
|
4 | dog
5 | elephant
@@ -928,7 +975,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:5:1
--> animals:5:1
|
5 | elephant
| ^^^^^^^^
@@ -943,7 +990,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:1:1
--> animals:1:1
|
1 | aardvark
| ^^^^^^^^
@@ -960,7 +1007,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:11:1
--> animals:11:1
|
9 | inchworm
10 | jackrabbit
@@ -977,7 +1024,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:5:1
--> animals:5:1
|
1 | aardvark
2 | beetle
@@ -1010,14 +1057,14 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:1:1
--> animals:1:1
|
1 | aardvark
| ^^^^^^^^
2 | beetle
3 | canary
|
::: /animals:11:1
::: animals:11:1
|
9 | inchworm
10 | jackrabbit
@@ -1054,7 +1101,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:1:1
--> animals:1:1
|
1 | aardvark
| ^^^^^^^^
@@ -1079,7 +1126,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:1:1
--> animals:1:1
|
1 | aardvark
| ^^^^^^^^
@@ -1107,13 +1154,13 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:1:1
--> animals:1:1
|
1 | aardvark
| ^^^^^^^^
2 | beetle
|
::: /animals:5:1
::: animals:5:1
|
4 | dog
5 | elephant
@@ -1135,7 +1182,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:1:1
--> animals:1:1
|
1 | aardvark
| ^^^^^^^^
@@ -1160,7 +1207,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:1:1
--> animals:1:1
|
1 | aardvark
| ^^^^^^^^
@@ -1191,7 +1238,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:1:1
--> animals:1:1
|
1 | aardvark
| ^^^^^^^^
@@ -1199,7 +1246,7 @@ watermelon
3 | canary
4 | dog
|
::: /animals:9:1
::: animals:9:1
|
6 | finch
7 | gorilla
@@ -1229,7 +1276,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /spacey-animals:8:1
--> spacey-animals:8:1
|
7 | dog
8 | elephant
@@ -1246,7 +1293,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /spacey-animals:12:1
--> spacey-animals:12:1
|
11 | gorilla
12 | hippopotamus
@@ -1264,7 +1311,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /spacey-animals:13:1
--> spacey-animals:13:1
|
11 | gorilla
12 | hippopotamus
@@ -1304,12 +1351,12 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /spacey-animals:3:1
--> spacey-animals:3:1
|
3 | beetle
| ^^^^^^
|
::: /spacey-animals:5:1
::: spacey-animals:5:1
|
5 | canary
| ^^^^^^
@@ -1333,7 +1380,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:3:1
--> animals:3:1
|
1 | aardvark
2 | beetle
@@ -1342,7 +1389,7 @@ watermelon
4 | dog
5 | elephant
|
::: /fruits:3:1
::: fruits:3:1
|
1 | apple
2 | banana
@@ -1370,7 +1417,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:3:1
--> animals:3:1
|
1 | aardvark
2 | beetle
@@ -1407,7 +1454,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:3:1
--> animals:3:1
|
1 | aardvark
2 | beetle
@@ -1435,7 +1482,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:3:1
--> animals:3:1
|
1 | aardvark
2 | beetle
@@ -1445,7 +1492,7 @@ watermelon
5 | elephant
|
warning: sub-diagnostic message
--> /fruits:3:1
--> fruits:3:1
|
1 | apple
2 | banana
@@ -1471,7 +1518,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:3:1
--> animals:3:1
|
1 | aardvark
2 | beetle
@@ -1481,7 +1528,7 @@ watermelon
5 | elephant
|
warning: sub-diagnostic message
--> /fruits:3:1
--> fruits:3:1
|
1 | apple
2 | banana
@@ -1491,7 +1538,7 @@ watermelon
5 | orange
|
warning: sub-diagnostic message
--> /animals:11:1
--> animals:11:1
|
9 | inchworm
10 | jackrabbit
@@ -1510,7 +1557,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:3:1
--> animals:3:1
|
1 | aardvark
2 | beetle
@@ -1520,7 +1567,7 @@ watermelon
5 | elephant
|
warning: sub-diagnostic message
--> /animals:11:1
--> animals:11:1
|
9 | inchworm
10 | jackrabbit
@@ -1528,7 +1575,7 @@ watermelon
| ^^^^^^^^
|
warning: sub-diagnostic message
--> /fruits:3:1
--> fruits:3:1
|
1 | apple
2 | banana
@@ -1558,7 +1605,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:3:1
--> animals:3:1
|
1 | aardvark
2 | beetle
@@ -1568,7 +1615,7 @@ watermelon
5 | elephant
|
warning: sub-diagnostic message
--> /animals:3:1
--> animals:3:1
|
1 | aardvark
2 | beetle
@@ -1594,7 +1641,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:5:1
--> animals:5:1
|
3 | canary
4 | dog
@@ -1617,7 +1664,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:5:1
--> animals:5:1
|
3 | canary
4 | dog
@@ -1637,7 +1684,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:5:1
--> animals:5:1
|
3 | canary
4 | dog
@@ -1657,7 +1704,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:5:4
--> animals:5:4
|
3 | canary
4 | dog
@@ -1679,7 +1726,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:5:4
--> animals:5:4
|
3 | canary
4 | dog
@@ -1711,7 +1758,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:4:1
--> animals:4:1
|
2 | beetle
3 | canary
@@ -1740,7 +1787,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:4:1
--> animals:4:1
|
2 | beetle
3 | canary
@@ -1771,7 +1818,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:5:1
--> animals:5:1
|
3 | canary
4 | dog
@@ -1806,7 +1853,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:5:1
--> animals:5:1
|
3 | canary
4 | dog
@@ -1834,7 +1881,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:5:1
--> animals:5:1
|
3 | canary
4 | dog
@@ -1866,7 +1913,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:5:3
--> animals:5:3
|
3 | canary
4 | dog
@@ -1888,7 +1935,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:5:3
--> animals:5:3
|
3 | canary
4 | dog
@@ -1921,7 +1968,7 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:8:1
--> animals:8:1
|
6 | finch
7 | gorilla
@@ -1930,7 +1977,7 @@ watermelon
9 | inchworm
10 | jackrabbit
|
::: /animals:1:1
::: animals:1:1
|
1 | aardvark
| -------- secondary
@@ -1961,27 +2008,27 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:5:1
--> animals:5:1
|
5 | elephant
| ^^^^^^^^ primary 5
|
::: /animals:9:1
::: animals:9:1
|
9 | inchworm
| ^^^^^^^^ primary 9
|
::: /animals:1:1
::: animals:1:1
|
1 | aardvark
| -------- secondary 1
|
::: /animals:3:1
::: animals:3:1
|
3 | canary
| ------ secondary 3
|
::: /animals:7:1
::: animals:7:1
|
7 | gorilla
| ------- secondary 7
@@ -2005,14 +2052,14 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /fruits:1:1
--> fruits:1:1
|
1 | apple
| ^^^^^ primary
2 | banana
3 | cantelope
|
::: /animals:1:1
::: animals:1:1
|
1 | aardvark
| -------- secondary
@@ -2040,32 +2087,32 @@ watermelon
env.render(&diag),
@r"
error: lint:test-diagnostic: main diagnostic message
--> /animals:11:1
--> animals:11:1
|
11 | kangaroo
| ^^^^^^^^ primary animals 11
|
::: /animals:1:1
::: animals:1:1
|
1 | aardvark
| -------- secondary animals 1
|
::: /animals:3:1
::: animals:3:1
|
3 | canary
| ------ secondary animals 3
|
::: /animals:7:1
::: animals:7:1
|
7 | gorilla
| ------- secondary animals 7
|
::: /fruits:10:1
::: fruits:10:1
|
10 | watermelon
| ^^^^^^^^^^ primary fruits 10
|
::: /fruits:2:1
::: fruits:2:1
|
2 | banana
| ------ secondary fruits 2

View File

@@ -0,0 +1,80 @@
use anstyle::{AnsiColor, Effects, Style};
use std::fmt::Formatter;
pub(super) const fn fmt_styled<'a, T>(
content: T,
style: anstyle::Style,
) -> impl std::fmt::Display + 'a
where
T: std::fmt::Display + 'a,
{
struct FmtStyled<T> {
content: T,
style: anstyle::Style,
}
impl<T> std::fmt::Display for FmtStyled<T>
where
T: std::fmt::Display,
{
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
write!(
f,
"{style_start}{content}{style_end}",
style_start = self.style.render(),
content = self.content,
style_end = self.style.render_reset()
)
}
}
FmtStyled { content, style }
}
#[derive(Clone, Debug)]
pub struct DiagnosticStylesheet {
pub(crate) error: Style,
pub(crate) warning: Style,
pub(crate) info: Style,
pub(crate) note: Style,
pub(crate) help: Style,
pub(crate) line_no: Style,
pub(crate) emphasis: Style,
pub(crate) none: Style,
}
impl Default for DiagnosticStylesheet {
fn default() -> Self {
Self::plain()
}
}
impl DiagnosticStylesheet {
/// Default terminal styling
pub fn styled() -> Self {
let bright_blue = AnsiColor::BrightBlue.on_default();
Self {
error: AnsiColor::BrightRed.on_default().effects(Effects::BOLD),
warning: AnsiColor::Yellow.on_default().effects(Effects::BOLD),
info: bright_blue.effects(Effects::BOLD),
note: AnsiColor::BrightGreen.on_default().effects(Effects::BOLD),
help: AnsiColor::BrightCyan.on_default().effects(Effects::BOLD),
line_no: bright_blue.effects(Effects::BOLD),
emphasis: Style::new().effects(Effects::BOLD),
none: Style::new(),
}
}
pub fn plain() -> Self {
Self {
error: Style::new(),
warning: Style::new(),
info: Style::new(),
note: Style::new(),
help: Style::new(),
line_no: Style::new(),
emphasis: Style::new(),
none: Style::new(),
}
}
}

View File

@@ -94,7 +94,9 @@ impl Files {
.root(db, path)
.map_or(Durability::default(), |root| root.durability(db));
let builder = File::builder(FilePath::System(absolute)).durability(durability);
let builder = File::builder(FilePath::System(absolute))
.durability(durability)
.path_durability(Durability::HIGH);
let builder = match metadata {
Ok(metadata) if metadata.file_type().is_file() => builder
@@ -159,9 +161,11 @@ impl Files {
tracing::trace!("Adding virtual file {}", path);
let virtual_file = VirtualFile(
File::builder(FilePath::SystemVirtual(path.to_path_buf()))
.path_durability(Durability::HIGH)
.status(FileStatus::Exists)
.revision(FileRevision::zero())
.permissions(None)
.permissions_durability(Durability::HIGH)
.new(db),
);
self.inner
@@ -272,7 +276,7 @@ impl std::panic::RefUnwindSafe for Files {}
/// A file that's either stored on the host system's file system or in the vendored file system.
#[salsa::input]
pub struct File {
/// The path of the file.
/// The path of the file (immutable).
#[return_ref]
pub path: FilePath,

View File

@@ -1,11 +1,10 @@
use std::hash::BuildHasherDefault;
use ruff_python_ast::PythonVersion;
use rustc_hash::FxHasher;
use crate::files::Files;
use crate::system::System;
use crate::vendored::VendoredFileSystem;
use ruff_python_ast::PythonVersion;
use rustc_hash::FxHasher;
use std::hash::BuildHasherDefault;
use std::num::NonZeroUsize;
pub mod diagnostic;
pub mod display;
@@ -37,9 +36,32 @@ pub trait Upcast<T: ?Sized> {
fn upcast_mut(&mut self) -> &mut T;
}
/// Returns the maximum number of tasks that ty is allowed
/// to process in parallel.
///
/// Returns [`std::thread::available_parallelism`], unless the environment
/// variable `TY_MAX_PARALLELISM` or `RAYON_NUM_THREADS` is set. `TY_MAX_PARALLELISM` takes
/// precedence over `RAYON_NUM_THREADS`.
///
/// Falls back to `1` if `available_parallelism` is not available.
///
/// Setting `TY_MAX_PARALLELISM` to `2` only restricts the number of threads that ty spawns
/// to process work in parallel. For example, to index a directory or checking the files of a project.
/// ty can still spawn more threads for other tasks, e.g. to wait for a Ctrl+C signal or
/// watching the files for changes.
pub fn max_parallelism() -> NonZeroUsize {
std::env::var("TY_MAX_PARALLELISM")
.or_else(|_| std::env::var("RAYON_NUM_THREADS"))
.ok()
.and_then(|s| s.parse().ok())
.unwrap_or_else(|| {
std::thread::available_parallelism().unwrap_or_else(|_| NonZeroUsize::new(1).unwrap())
})
}
#[cfg(test)]
mod tests {
use std::sync::Arc;
use std::sync::{Arc, Mutex};
use crate::files::Files;
use crate::system::TestSystem;
@@ -47,6 +69,8 @@ mod tests {
use crate::vendored::VendoredFileSystem;
use crate::Db;
type Events = Arc<Mutex<Vec<salsa::Event>>>;
/// Database that can be used for testing.
///
/// Uses an in memory filesystem and it stubs out the vendored files by default.
@@ -57,36 +81,37 @@ mod tests {
files: Files,
system: TestSystem,
vendored: VendoredFileSystem,
events: Arc<std::sync::Mutex<Vec<salsa::Event>>>,
events: Events,
}
impl TestDb {
pub(crate) fn new() -> Self {
let events = Events::default();
Self {
storage: salsa::Storage::default(),
storage: salsa::Storage::new(Some(Box::new({
let events = events.clone();
move |event| {
tracing::trace!("event: {:?}", event);
let mut events = events.lock().unwrap();
events.push(event);
}
}))),
system: TestSystem::default(),
vendored: VendoredFileSystem::default(),
events: std::sync::Arc::default(),
events,
files: Files::default(),
}
}
/// Empties the internal store of salsa events that have been emitted,
/// and returns them as a `Vec` (equivalent to [`std::mem::take`]).
///
/// ## Panics
/// If there are pending database snapshots.
pub(crate) fn take_salsa_events(&mut self) -> Vec<salsa::Event> {
let inner = Arc::get_mut(&mut self.events)
.expect("expected no pending salsa database snapshots.");
let mut events = self.events.lock().unwrap();
std::mem::take(inner.get_mut().unwrap())
std::mem::take(&mut *events)
}
/// Clears the emitted salsa events.
///
/// ## Panics
/// If there are pending database snapshots.
pub(crate) fn clear_salsa_events(&mut self) {
self.take_salsa_events();
}
@@ -126,12 +151,5 @@ mod tests {
}
#[salsa::db]
impl salsa::Database for TestDb {
fn salsa_event(&self, event: &dyn Fn() -> salsa::Event) {
let event = event();
tracing::trace!("event: {:?}", event);
let mut events = self.events.lock().unwrap();
events.push(event);
}
}
impl salsa::Database for TestDb {}
}

View File

@@ -1,12 +1,29 @@
use std::backtrace::BacktraceStatus;
use std::cell::Cell;
use std::panic::Location;
use std::sync::OnceLock;
#[derive(Default, Debug)]
#[derive(Debug)]
pub struct PanicError {
pub location: Option<String>,
pub payload: Option<String>,
pub payload: Payload,
pub backtrace: Option<std::backtrace::Backtrace>,
pub salsa_backtrace: Option<salsa::Backtrace>,
}
#[derive(Debug)]
pub struct Payload(Box<dyn std::any::Any + Send>);
impl Payload {
pub fn as_str(&self) -> Option<&str> {
if let Some(s) = self.0.downcast_ref::<String>() {
Some(s)
} else if let Some(s) = self.0.downcast_ref::<&str>() {
Some(s)
} else {
None
}
}
}
impl std::fmt::Display for PanicError {
@@ -15,20 +32,39 @@ impl std::fmt::Display for PanicError {
if let Some(location) = &self.location {
write!(f, " {location}")?;
}
if let Some(payload) = &self.payload {
if let Some(payload) = self.payload.as_str() {
write!(f, ":\n{payload}")?;
}
if let Some(backtrace) = &self.backtrace {
writeln!(f, "\nBacktrace: {backtrace}")?;
match backtrace.status() {
BacktraceStatus::Disabled => {
writeln!(
f,
"\nrun with `RUST_BACKTRACE=1` environment variable to display a backtrace"
)?;
}
BacktraceStatus::Captured => {
writeln!(f, "\nBacktrace: {backtrace}")?;
}
_ => {}
}
}
Ok(())
}
}
#[derive(Default)]
struct CapturedPanicInfo {
backtrace: Option<std::backtrace::Backtrace>,
location: Option<String>,
salsa_backtrace: Option<salsa::Backtrace>,
}
thread_local! {
static CAPTURE_PANIC_INFO: Cell<bool> = const { Cell::new(false) };
static OUR_HOOK_RAN: Cell<bool> = const { Cell::new(false) };
static LAST_PANIC: Cell<Option<PanicError>> = const { Cell::new(None) };
static LAST_BACKTRACE: Cell<CapturedPanicInfo> = const {
Cell::new(CapturedPanicInfo { backtrace: None, location: None, salsa_backtrace: None })
};
}
fn install_hook() {
@@ -36,24 +72,18 @@ fn install_hook() {
ONCE.get_or_init(|| {
let prev = std::panic::take_hook();
std::panic::set_hook(Box::new(move |info| {
OUR_HOOK_RAN.with(|cell| cell.set(true));
let should_capture = CAPTURE_PANIC_INFO.with(Cell::get);
if !should_capture {
return (*prev)(info);
}
let payload = if let Some(s) = info.payload().downcast_ref::<&str>() {
Some(s.to_string())
} else {
info.payload().downcast_ref::<String>().cloned()
};
let location = info.location().map(Location::to_string);
let backtrace = std::backtrace::Backtrace::force_capture();
LAST_PANIC.with(|cell| {
cell.set(Some(PanicError {
payload,
location,
backtrace: Some(backtrace),
}));
let backtrace = Some(std::backtrace::Backtrace::capture());
LAST_BACKTRACE.set(CapturedPanicInfo {
backtrace,
location,
salsa_backtrace: salsa::Backtrace::capture(),
});
}));
});
@@ -70,7 +100,7 @@ fn install_hook() {
/// stderr).
///
/// We assume that there is nothing else running in this process that needs to install a competing
/// panic hook. We are careful to install our custom hook only once, and we do not ever restore
/// panic hook. We are careful to install our custom hook only once, and we do not ever restore
/// the previous hook (since you can always retain the previous hook's behavior by not calling this
/// wrapper).
pub fn catch_unwind<F, R>(f: F) -> Result<R, PanicError>
@@ -78,15 +108,83 @@ where
F: FnOnce() -> R + std::panic::UnwindSafe,
{
install_hook();
OUR_HOOK_RAN.with(|cell| cell.set(false));
let prev_should_capture = CAPTURE_PANIC_INFO.with(|cell| cell.replace(true));
let result = std::panic::catch_unwind(f).map_err(|_| {
let our_hook_ran = OUR_HOOK_RAN.with(Cell::get);
if !our_hook_ran {
panic!("detected a competing panic hook");
let prev_should_capture = CAPTURE_PANIC_INFO.replace(true);
let result = std::panic::catch_unwind(f).map_err(|payload| {
// Try to get the backtrace and location from our custom panic hook.
// The custom panic hook only runs once when `panic!` is called (or similar). It doesn't
// run when the panic is propagated with `std::panic::resume_unwind`. The panic hook
// is also not called when the panic is raised with `std::panic::resum_unwind` as is the
// case for salsa unwinds (see the ignored test below).
// Because of that, always take the payload from `catch_unwind` because it may have been transformed
// by an inner `std::panic::catch_unwind` handlers and only use the information
// from the custom handler to enrich the error with the backtrace and location.
let CapturedPanicInfo {
location,
backtrace,
salsa_backtrace,
} = LAST_BACKTRACE.with(Cell::take);
PanicError {
location,
payload: Payload(payload),
backtrace,
salsa_backtrace,
}
LAST_PANIC.with(Cell::take).unwrap_or_default()
});
CAPTURE_PANIC_INFO.with(|cell| cell.set(prev_should_capture));
CAPTURE_PANIC_INFO.set(prev_should_capture);
result
}
#[cfg(test)]
mod tests {
use salsa::{Database, Durability};
#[test]
#[ignore = "super::catch_unwind installs a custom panic handler, which could effect test isolation"]
fn no_backtrace_for_salsa_cancelled() {
#[salsa::input]
struct Input {
value: u32,
}
#[salsa::tracked]
fn test_query(db: &dyn Database, input: Input) -> u32 {
loop {
// This should throw a cancelled error
let _ = input.value(db);
}
}
let db = salsa::DatabaseImpl::new();
let input = Input::new(&db, 42);
let result = std::thread::scope(move |scope| {
{
let mut db = db.clone();
scope.spawn(move || {
// This will cancel the other thread by throwing a `salsa::Cancelled` error.
db.synthetic_write(Durability::MEDIUM);
});
}
{
scope.spawn(move || {
super::catch_unwind(|| {
test_query(&db, input);
})
})
}
.join()
.unwrap()
});
match result {
Ok(_) => panic!("Expected query to panic"),
Err(err) => {
// Panics triggered with `resume_unwind` have no backtrace.
assert!(err.backtrace.is_none());
}
}
}
}

View File

@@ -1,19 +1,19 @@
use filetime::FileTime;
use ruff_notebook::{Notebook, NotebookError};
use rustc_hash::FxHashSet;
use std::panic::RefUnwindSafe;
use std::sync::Arc;
use std::{any::Any, path::PathBuf};
use crate::system::{
CaseSensitivity, DirectoryEntry, FileType, GlobError, GlobErrorKind, Metadata, Result, System,
SystemPath, SystemPathBuf, SystemVirtualPath, WritableSystem,
};
use super::walk_directory::{
self, DirectoryWalker, WalkDirectoryBuilder, WalkDirectoryConfiguration,
WalkDirectoryVisitorBuilder, WalkState,
};
use crate::max_parallelism;
use crate::system::{
CaseSensitivity, DirectoryEntry, FileType, GlobError, GlobErrorKind, Metadata, Result, System,
SystemPath, SystemPathBuf, SystemVirtualPath, WritableSystem,
};
use filetime::FileTime;
use ruff_notebook::{Notebook, NotebookError};
use rustc_hash::FxHashSet;
use std::num::NonZeroUsize;
use std::panic::RefUnwindSafe;
use std::sync::Arc;
use std::{any::Any, path::PathBuf};
/// A system implementation that uses the OS file system.
#[derive(Debug, Clone)]
@@ -50,7 +50,6 @@ impl OsSystem {
Self {
// Spreading `..Default` because it isn't possible to feature gate the initializer of a single field.
#[allow(clippy::needless_update)]
inner: Arc::new(OsSystemInner {
cwd: cwd.to_path_buf(),
case_sensitivity,
@@ -427,11 +426,7 @@ impl DirectoryWalker for OsDirectoryWalker {
builder.add(additional_path.as_std_path());
}
builder.threads(
std::thread::available_parallelism()
.map_or(1, std::num::NonZeroUsize::get)
.min(12),
);
builder.threads(max_parallelism().min(NonZeroUsize::new(12).unwrap()).get());
builder.build_parallel().run(|| {
let mut visitor = visitor_builder.build();

View File

@@ -35,7 +35,7 @@ impl WalkDirectoryBuilder {
/// Each additional path is traversed recursively.
/// This should be preferred over building multiple
/// walkers since it enables reusing resources.
#[allow(clippy::should_implement_trait)]
#[expect(clippy::should_implement_trait)]
pub fn add(mut self, path: impl AsRef<SystemPath>) -> Self {
self.paths.push(path.as_ref().to_path_buf());
self

View File

@@ -107,7 +107,7 @@ fn query_name<Q>(_query: &Q) -> &'static str {
.unwrap_or(full_qualified_query_name)
}
/// Sets up logging for the current thread. It captures all `red_knot` and `ruff` events.
/// Sets up logging for the current thread. It captures all `ty` and `ruff` events.
///
/// Useful for capturing the tracing output in a failing test.
///
@@ -128,7 +128,7 @@ pub fn setup_logging() -> LoggingGuard {
/// # Examples
/// ```
/// use ruff_db::testing::setup_logging_with_filter;
/// let _logging = setup_logging_with_filter("red_knot_module_resolver::resolver");
/// let _logging = setup_logging_with_filter("ty_module_resolver::resolver");
/// ```
///
/// # Filter
@@ -148,11 +148,7 @@ impl LoggingBuilder {
pub fn new() -> Self {
Self {
filter: EnvFilter::default()
.add_directive(
"red_knot=trace"
.parse()
.expect("Hardcoded directive to be valid"),
)
.add_directive("ty=trace".parse().expect("Hardcoded directive to be valid"))
.add_directive(
"ruff=trace"
.parse()

View File

@@ -172,7 +172,7 @@ impl Default for VendoredFileSystem {
/// that users of the `VendoredFileSystem` could realistically need.
/// For debugging purposes, however, we want to have all information
/// available.
#[allow(unused)]
#[expect(unused)]
#[derive(Debug)]
struct ZipFileDebugInfo {
crc32_hash: u32,

View File

@@ -11,7 +11,7 @@ repository = { workspace = true }
license = { workspace = true }
[dependencies]
red_knot_project = { workspace = true, features = ["schemars"] }
ty_project = { workspace = true, features = ["schemars"] }
ruff = { workspace = true }
ruff_diagnostics = { workspace = true }
ruff_formatter = { workspace = true }

View File

@@ -63,7 +63,6 @@ fn find_pyproject_config(
}
/// Find files that ruff would check so we can format them. Adapted from `ruff`.
#[allow(clippy::type_complexity)]
fn ruff_check_paths<'a>(
pyproject_config: &'a PyprojectConfig,
cli: &FormatArguments,
@@ -135,12 +134,12 @@ impl Statistics {
}
/// We currently prefer the similarity index, but i'd like to keep this around
#[allow(clippy::cast_precision_loss, unused)]
#[expect(clippy::cast_precision_loss, unused)]
pub(crate) fn jaccard_index(&self) -> f32 {
self.intersection as f32 / (self.black_input + self.ruff_output + self.intersection) as f32
}
#[allow(clippy::cast_precision_loss)]
#[expect(clippy::cast_precision_loss)]
pub(crate) fn similarity_index(&self) -> f32 {
self.intersection as f32 / (self.black_input + self.intersection) as f32
}
@@ -177,7 +176,7 @@ pub(crate) enum Format {
Full,
}
#[allow(clippy::struct_excessive_bools)]
#[expect(clippy::struct_excessive_bools)]
#[derive(clap::Args)]
pub(crate) struct Args {
/// Like `ruff check`'s files. See `--multi-project` if you want to format an ecosystem
@@ -222,7 +221,7 @@ pub(crate) struct Args {
#[arg(long)]
pub(crate) files_with_errors: Option<u32>,
#[clap(flatten)]
#[allow(clippy::struct_field_names)]
#[expect(clippy::struct_field_names)]
pub(crate) log_level_args: LogLevelArgs,
}

View File

@@ -2,7 +2,7 @@
use anyhow::Result;
use crate::{generate_cli_help, generate_docs, generate_json_schema, generate_knot_schema};
use crate::{generate_cli_help, generate_docs, generate_json_schema, generate_ty_schema};
pub(crate) const REGENERATE_ALL_COMMAND: &str = "cargo dev generate-all";
@@ -33,7 +33,7 @@ impl Mode {
pub(crate) fn main(args: &Args) -> Result<()> {
generate_json_schema::main(&generate_json_schema::Args { mode: args.mode })?;
generate_knot_schema::main(&generate_knot_schema::Args { mode: args.mode })?;
generate_ty_schema::main(&generate_ty_schema::Args { mode: args.mode })?;
generate_cli_help::main(&generate_cli_help::Args { mode: args.mode })?;
generate_docs::main(&generate_docs::Args {
dry_run: args.mode.is_dry_run(),

View File

@@ -34,7 +34,6 @@ fn generate_table(table_out: &mut String, rules: impl IntoIterator<Item = Rule>,
RuleGroup::Deprecated => {
format!("<span title='Rule has been deprecated'>{WARNING_SYMBOL}</span>")
}
#[allow(deprecated)]
RuleGroup::Preview => {
format!("<span title='Rule is in preview'>{PREVIEW_SYMBOL}</span>")
}
@@ -78,7 +77,7 @@ fn generate_table(table_out: &mut String, rules: impl IntoIterator<Item = Rule>,
se = "</span>";
}
#[allow(clippy::or_fun_call)]
#[expect(clippy::or_fun_call)]
let _ = write!(
table_out,
"| {ss}{0}{1}{se} {{ #{0}{1} }} | {ss}{2}{se} | {ss}{3}{se} | {ss}{4}{se} |",

View File

@@ -9,11 +9,11 @@ use schemars::schema_for;
use crate::generate_all::{Mode, REGENERATE_ALL_COMMAND};
use crate::ROOT_DIR;
use red_knot_project::metadata::options::Options;
use ty_project::metadata::options::Options;
#[derive(clap::Args)]
pub(crate) struct Args {
/// Write the generated table to stdout (rather than to `knot.schema.json`).
/// Write the generated table to stdout (rather than to `ty.schema.json`).
#[arg(long, default_value_t, value_enum)]
pub(crate) mode: Mode,
}
@@ -21,7 +21,7 @@ pub(crate) struct Args {
pub(crate) fn main(args: &Args) -> Result<()> {
let schema = schema_for!(Options);
let schema_string = serde_json::to_string_pretty(&schema).unwrap();
let filename = "knot.schema.json";
let filename = "ty.schema.json";
let schema_path = PathBuf::from(ROOT_DIR).join(filename);
match args.mode {
@@ -62,7 +62,7 @@ mod tests {
#[test]
fn test_generate_json_schema() -> Result<()> {
let mode = if env::var("KNOT_UPDATE_SCHEMA").as_deref() == Ok("1") {
let mode = if env::var("TY_UPDATE_SCHEMA").as_deref() == Ok("1") {
Mode::Write
} else {
Mode::Check

View File

@@ -13,9 +13,9 @@ mod generate_all;
mod generate_cli_help;
mod generate_docs;
mod generate_json_schema;
mod generate_knot_schema;
mod generate_options;
mod generate_rules_table;
mod generate_ty_schema;
mod print_ast;
mod print_cst;
mod print_tokens;
@@ -34,14 +34,14 @@ struct Args {
}
#[derive(Subcommand)]
#[allow(clippy::large_enum_variant)]
#[expect(clippy::large_enum_variant)]
enum Command {
/// Run all code and documentation generation steps.
GenerateAll(generate_all::Args),
/// Generate JSON schema for the TOML configuration file.
GenerateJSONSchema(generate_json_schema::Args),
/// Generate JSON schema for the Red Knot TOML configuration file.
GenerateKnotSchema(generate_knot_schema::Args),
/// Generate JSON schema for the ty TOML configuration file.
GenerateTySchema(generate_ty_schema::Args),
/// Generate a Markdown-compatible table of supported lint rules.
GenerateRulesTable,
/// Generate a Markdown-compatible listing of configuration options.
@@ -82,11 +82,11 @@ fn main() -> Result<ExitCode> {
command,
global_options,
} = Args::parse();
#[allow(clippy::print_stdout)]
#[expect(clippy::print_stdout)]
match command {
Command::GenerateAll(args) => generate_all::main(&args)?,
Command::GenerateJSONSchema(args) => generate_json_schema::main(&args)?,
Command::GenerateKnotSchema(args) => generate_knot_schema::main(&args)?,
Command::GenerateTySchema(args) => generate_ty_schema::main(&args)?,
Command::GenerateRulesTable => println!("{}", generate_rules_table::generate()),
Command::GenerateOptions => println!("{}", generate_options::generate()),
Command::GenerateCliHelp(args) => generate_cli_help::main(&args)?,

View File

@@ -519,7 +519,7 @@ impl TextWidth {
let char_width = match c {
'\t' => indent_width.value(),
'\n' => return TextWidth::Multiline,
#[allow(clippy::cast_possible_truncation)]
#[expect(clippy::cast_possible_truncation)]
c => c.width().unwrap_or(0) as u32,
};
width += char_width;

View File

@@ -280,7 +280,7 @@ impl Format<IrFormatContext<'_>> for &[FormatElement] {
| FormatElement::SourceCodeSlice { .. }) => {
fn write_escaped(element: &FormatElement, f: &mut Formatter<IrFormatContext>) {
let (text, text_width) = match element {
#[allow(clippy::cast_possible_truncation)]
#[expect(clippy::cast_possible_truncation)]
FormatElement::Token { text } => {
(*text, TextWidth::Width(Width::new(text.len() as u32)))
}

View File

@@ -379,7 +379,7 @@ impl PartialEq for LabelId {
}
impl LabelId {
#[allow(clippy::needless_pass_by_value)]
#[expect(clippy::needless_pass_by_value)]
pub fn of<T: LabelDefinition>(label: T) -> Self {
Self {
value: label.value(),

View File

@@ -925,7 +925,7 @@ pub struct FormatState<Context> {
group_id_builder: UniqueGroupIdBuilder,
}
#[allow(clippy::missing_fields_in_debug)]
#[expect(clippy::missing_fields_in_debug)]
impl<Context> std::fmt::Debug for FormatState<Context>
where
Context: std::fmt::Debug,

View File

@@ -331,7 +331,7 @@ impl<'a> Printer<'a> {
FormatElement::Tag(StartVerbatim(kind)) => {
if let VerbatimKind::Verbatim { length } = kind {
// SAFETY: Ruff only supports formatting files <= 4GB
#[allow(clippy::cast_possible_truncation)]
#[expect(clippy::cast_possible_truncation)]
self.state.verbatim_markers.push(TextRange::at(
TextSize::from(self.state.buffer.len() as u32),
*length,
@@ -464,7 +464,7 @@ impl<'a> Printer<'a> {
self.push_marker();
match text {
#[allow(clippy::cast_possible_truncation)]
#[expect(clippy::cast_possible_truncation)]
Text::Token(token) => {
self.state.buffer.push_str(token);
self.state.line_width += token.len() as u32;
@@ -831,7 +831,7 @@ impl<'a> Printer<'a> {
} else {
self.state.buffer.push(char);
#[allow(clippy::cast_possible_truncation)]
#[expect(clippy::cast_possible_truncation)]
let char_width = if char == '\t' {
self.options.indent_width.value()
} else {
@@ -1480,7 +1480,7 @@ impl<'a, 'print> FitsMeasurer<'a, 'print> {
u32::from(indent.level()) * self.options().indent_width() + u32::from(indent.align());
match text {
#[allow(clippy::cast_possible_truncation)]
#[expect(clippy::cast_possible_truncation)]
Text::Token(token) => {
self.state.line_width += token.len() as u32;
}
@@ -1511,7 +1511,7 @@ impl<'a, 'print> FitsMeasurer<'a, 'print> {
}
}
// SAFETY: A u32 is sufficient to format files <= 4GB
#[allow(clippy::cast_possible_truncation)]
#[expect(clippy::cast_possible_truncation)]
c => c.width().unwrap_or(0) as u32,
};
self.state.line_width += char_width;

View File

@@ -10,13 +10,13 @@ authors.workspace = true
license.workspace = true
[dependencies]
red_knot_python_semantic = { workspace = true }
ruff_cache = { workspace = true }
ruff_db = { workspace = true, features = ["os", "serde"] }
ruff_linter = { workspace = true }
ruff_macros = { workspace = true }
ruff_python_ast = { workspace = true }
ruff_python_parser = { workspace = true }
ty_python_semantic = { workspace = true }
anyhow = { workspace = true }
clap = { workspace = true, optional = true }

View File

@@ -1,8 +1,8 @@
use red_knot_python_semantic::ModuleName;
use ruff_python_ast::visitor::source_order::{
walk_expr, walk_module, walk_stmt, SourceOrderVisitor,
};
use ruff_python_ast::{self as ast, Expr, Mod, Stmt};
use ty_python_semantic::ModuleName;
/// Collect all imports for a given Python file.
#[derive(Default, Debug)]

View File

@@ -2,15 +2,16 @@ use anyhow::Result;
use std::sync::Arc;
use zip::CompressionMethod;
use red_knot_python_semantic::lint::{LintRegistry, RuleSelection};
use red_knot_python_semantic::{
default_lint_registry, Db, Program, ProgramSettings, PythonPlatform, SearchPathSettings,
};
use ruff_db::files::{File, Files};
use ruff_db::system::{OsSystem, System, SystemPathBuf};
use ruff_db::vendored::{VendoredFileSystem, VendoredFileSystemBuilder};
use ruff_db::{Db as SourceDb, Upcast};
use ruff_python_ast::PythonVersion;
use ty_python_semantic::lint::{LintRegistry, RuleSelection};
use ty_python_semantic::{
default_lint_registry, Db, Program, ProgramSettings, PythonPath, PythonPlatform,
SearchPathSettings,
};
static EMPTY_VENDORED: std::sync::LazyLock<VendoredFileSystem> = std::sync::LazyLock::new(|| {
let mut builder = VendoredFileSystemBuilder::new(CompressionMethod::Stored);
@@ -32,8 +33,12 @@ impl ModuleDb {
pub fn from_src_roots(
src_roots: Vec<SystemPathBuf>,
python_version: PythonVersion,
venv_path: Option<SystemPathBuf>,
) -> Result<Self> {
let search_paths = SearchPathSettings::new(src_roots);
let mut search_paths = SearchPathSettings::new(src_roots);
if let Some(venv_path) = venv_path {
search_paths.python_path = PythonPath::from_cli_flag(venv_path);
}
let db = Self::default();
Program::from_settings(
@@ -93,6 +98,4 @@ impl Db for ModuleDb {
}
#[salsa::db]
impl salsa::Database for ModuleDb {
fn salsa_event(&self, _event: &dyn Fn() -> salsa::Event) {}
}
impl salsa::Database for ModuleDb {}

View File

@@ -1,5 +1,5 @@
use red_knot_python_semantic::resolve_module;
use ruff_db::files::FilePath;
use ty_python_semantic::resolve_module;
use crate::collector::CollectedImport;
use crate::ModuleDb;

View File

@@ -22,7 +22,7 @@ impl<I: Idx, T> IndexSlice<I, T> {
pub const fn from_raw(raw: &[T]) -> &Self {
let ptr: *const [T] = raw;
#[allow(unsafe_code)]
#[expect(unsafe_code)]
// SAFETY: `IndexSlice` is `repr(transparent)` over a normal slice
unsafe {
&*(ptr as *const Self)
@@ -33,7 +33,7 @@ impl<I: Idx, T> IndexSlice<I, T> {
pub fn from_raw_mut(raw: &mut [T]) -> &mut Self {
let ptr: *mut [T] = raw;
#[allow(unsafe_code)]
#[expect(unsafe_code)]
// SAFETY: `IndexSlice` is `repr(transparent)` over a normal slice
unsafe {
&mut *(ptr as *mut Self)
@@ -209,5 +209,5 @@ impl<I: Idx, T> Default for &mut IndexSlice<I, T> {
// Whether `IndexSlice` is `Send` depends only on the data,
// not the phantom data.
#[allow(unsafe_code)]
#[expect(unsafe_code)]
unsafe impl<I: Idx, T> Send for IndexSlice<I, T> where T: Send {}

View File

@@ -179,16 +179,16 @@ impl<I: Idx, T, const N: usize> From<[T; N]> for IndexVec<I, T> {
// Whether `IndexVec` is `Send` depends only on the data,
// not the phantom data.
#[allow(unsafe_code)]
#[expect(unsafe_code)]
unsafe impl<I: Idx, T> Send for IndexVec<I, T> where T: Send {}
#[allow(unsafe_code)]
#[expect(unsafe_code)]
#[cfg(feature = "salsa")]
unsafe impl<I, T> salsa::Update for IndexVec<I, T>
where
T: salsa::Update,
{
#[allow(unsafe_code)]
#[expect(unsafe_code)]
unsafe fn maybe_update(old_pointer: *mut Self, new_value: Self) -> bool {
let old_vec: &mut IndexVec<I, T> = unsafe { &mut *old_pointer };
salsa::Update::maybe_update(&mut old_vec.raw, new_value.raw)

View File

@@ -1,6 +1,6 @@
[package]
name = "ruff_linter"
version = "0.11.7"
version = "0.11.8"
publish = false
authors = { workspace = true }
edition = { workspace = true }
@@ -76,6 +76,7 @@ insta = { workspace = true, features = ["filters", "json", "redactions"] }
test-case = { workspace = true }
# Disable colored output in tests
colored = { workspace = true, features = ["no-color"] }
tempfile = { workspace = true }
[features]
default = []

View File

@@ -6,3 +6,12 @@ except ModuleNotFoundError:
from airflow.datasets import Dataset as Asset
Asset
try:
from airflow.sdk import Asset
except ModuleNotFoundError:
from airflow import datasets
Asset = datasets.Dataset
asset = Asset()

View File

@@ -1,481 +0,0 @@
from __future__ import annotations
from airflow.api.auth.backend import basic_auth, kerberos_auth
from airflow.api.auth.backend.basic_auth import auth_current_user
from airflow.auth.managers.fab.api.auth.backend import (
kerberos_auth as backend_kerberos_auth,
)
from airflow.auth.managers.fab.fab_auth_manager import FabAuthManager
from airflow.auth.managers.fab.security_manager import override as fab_override
from airflow.config_templates.default_celery import DEFAULT_CELERY_CONFIG
from airflow.executors.celery_executor import CeleryExecutor, app
from airflow.executors.celery_kubernetes_executor import CeleryKubernetesExecutor
from airflow.executors.dask_executor import DaskExecutor
from airflow.executors.kubernetes_executor_types import (
ALL_NAMESPACES,
POD_EXECUTOR_DONE_KEY,
)
from airflow.hooks.dbapi import ConnectorProtocol, DbApiHook
from airflow.hooks.dbapi_hook import DbApiHook as DbApiHook2
from airflow.hooks.docker_hook import DockerHook
from airflow.hooks.druid_hook import DruidDbApiHook, DruidHook
from airflow.hooks.filesystem import FSHook
from airflow.hooks.hive_hooks import (
HIVE_QUEUE_PRIORITIES,
HiveCliHook,
HiveMetastoreHook,
HiveServer2Hook,
)
from airflow.hooks.http_hook import HttpHook
from airflow.hooks.jdbc_hook import JdbcHook, jaydebeapi
from airflow.hooks.mssql_hook import MsSqlHook
from airflow.hooks.mysql_hook import MySqlHook
from airflow.hooks.oracle_hook import OracleHook
from airflow.hooks.package_index import PackageIndexHook
from airflow.hooks.pig_hook import PigCliHook
from airflow.hooks.postgres_hook import PostgresHook
from airflow.hooks.presto_hook import PrestoHook
from airflow.hooks.S3_hook import S3Hook, provide_bucket_name
from airflow.hooks.samba_hook import SambaHook
from airflow.hooks.slack_hook import SlackHook
from airflow.hooks.sqlite_hook import SqliteHook
from airflow.hooks.subprocess import SubprocessHook, SubprocessResult, working_directory
from airflow.hooks.webhdfs_hook import WebHDFSHook
from airflow.hooks.zendesk_hook import ZendeskHook
from airflow.kubernetes.k8s_model import K8SModel, append_to_pod
from airflow.kubernetes.kube_client import (
_disable_verify_ssl,
_enable_tcp_keepalive,
get_kube_client,
)
from airflow.kubernetes.kubernetes_helper_functions import (
add_pod_suffix,
annotations_for_logging_task_metadata,
annotations_to_key,
create_pod_id,
get_logs_task_metadata,
rand_str,
)
from airflow.kubernetes.pod import Port, Resources
from airflow.kubernetes.pod_generator import (
PodDefaults,
PodGenerator,
PodGeneratorDeprecated,
datetime_to_label_safe_datestring,
extend_object_field,
label_safe_datestring_to_datetime,
make_safe_label_value,
merge_objects,
)
from airflow.kubernetes.pod_generator import (
add_pod_suffix as add_pod_suffix2,
)
from airflow.kubernetes.pod_generator import (
rand_str as rand_str2,
)
from airflow.kubernetes.pod_generator_deprecated import (
PodDefaults as PodDefaults3,
)
from airflow.kubernetes.pod_generator_deprecated import (
PodGenerator as PodGenerator2,
)
from airflow.kubernetes.pod_generator_deprecated import (
make_safe_label_value as make_safe_label_value2,
)
from airflow.kubernetes.pod_launcher import PodLauncher, PodStatus
from airflow.kubernetes.pod_launcher_deprecated import (
PodDefaults as PodDefaults2,
)
from airflow.kubernetes.pod_launcher_deprecated import (
PodLauncher as PodLauncher2,
)
from airflow.kubernetes.pod_launcher_deprecated import (
PodStatus as PodStatus2,
)
from airflow.kubernetes.pod_launcher_deprecated import (
get_kube_client as get_kube_client2,
)
from airflow.kubernetes.pod_runtime_info_env import PodRuntimeInfoEnv
from airflow.kubernetes.secret import K8SModel2, Secret
from airflow.kubernetes.volume import Volume
from airflow.kubernetes.volume_mount import VolumeMount
from airflow.macros.hive import closest_ds_partition, max_partition
from airflow.operators.bash import BashOperator
from airflow.operators.bash_operator import BashOperator as LegacyBashOperator
from airflow.operators.check_operator import (
CheckOperator,
IntervalCheckOperator,
SQLCheckOperator,
SQLIntervalCheckOperator,
SQLThresholdCheckOperator,
SQLValueCheckOperator,
ThresholdCheckOperator,
ValueCheckOperator,
)
from airflow.operators.datetime import BranchDateTimeOperator, target_times_as_dates
from airflow.operators.docker_operator import DockerOperator
from airflow.operators.druid_check_operator import DruidCheckOperator
from airflow.operators.dummy import DummyOperator, EmptyOperator
from airflow.operators.email import EmailOperator
from airflow.operators.email_operator import EmailOperator
from airflow.operators.gcs_to_s3 import GCSToS3Operator
from airflow.operators.google_api_to_s3_transfer import (
GoogleApiToS3Operator,
GoogleApiToS3Transfer,
)
from airflow.operators.hive_operator import HiveOperator
from airflow.operators.hive_stats_operator import HiveStatsCollectionOperator
from airflow.operators.hive_to_druid import HiveToDruidOperator, HiveToDruidTransfer
from airflow.operators.hive_to_mysql import HiveToMySqlOperator, HiveToMySqlTransfer
from airflow.operators.hive_to_samba_operator import HiveToSambaOperator
from airflow.operators.http_operator import SimpleHttpOperator
from airflow.operators.jdbc_operator import JdbcOperator
from airflow.operators.mssql_operator import MsSqlOperator
from airflow.operators.mssql_to_hive import MsSqlToHiveOperator, MsSqlToHiveTransfer
from airflow.operators.mysql_operator import MySqlOperator
from airflow.operators.mysql_to_hive import MySqlToHiveOperator, MySqlToHiveTransfer
from airflow.operators.oracle_operator import OracleOperator
from airflow.operators.papermill_operator import PapermillOperator
from airflow.operators.pig_operator import PigOperator
from airflow.operators.postgres_operator import Mapping, PostgresOperator
from airflow.operators.presto_check_operator import (
CheckOperator as SQLCheckOperator2,
)
from airflow.operators.presto_check_operator import (
IntervalCheckOperator as SQLIntervalCheckOperator2,
)
from airflow.operators.presto_check_operator import (
PrestoCheckOperator,
PrestoIntervalCheckOperator,
PrestoValueCheckOperator,
)
from airflow.operators.presto_check_operator import (
ValueCheckOperator as SQLValueCheckOperator2,
)
from airflow.operators.presto_to_mysql import (
PrestoToMySqlOperator,
PrestoToMySqlTransfer,
)
from airflow.operators.python import (
BranchPythonOperator,
PythonOperator,
PythonVirtualenvOperator,
ShortCircuitOperator,
)
from airflow.operators.redshift_to_s3_operator import (
RedshiftToS3Operator,
RedshiftToS3Transfer,
)
from airflow.operators.s3_file_transform_operator import S3FileTransformOperator
from airflow.operators.s3_to_hive_operator import S3ToHiveOperator, S3ToHiveTransfer
from airflow.operators.s3_to_redshift_operator import (
S3ToRedshiftOperator,
S3ToRedshiftTransfer,
)
from airflow.operators.slack_operator import SlackAPIOperator, SlackAPIPostOperator
from airflow.operators.sql import (
BaseSQLOperator,
BranchSQLOperator,
SQLTableCheckOperator,
_convert_to_float_if_possible,
parse_boolean,
)
from airflow.operators.sql import (
SQLCheckOperator as SQLCheckOperator3,
)
from airflow.operators.sql import (
SQLColumnCheckOperator as SQLColumnCheckOperator2,
)
from airflow.operators.sql import (
SQLIntervalCheckOperator as SQLIntervalCheckOperator3,
)
from airflow.operators.sql import (
SQLThresholdCheckOperator as SQLThresholdCheckOperator2,
)
from airflow.operators.sql import (
SQLValueCheckOperator as SQLValueCheckOperator3,
)
from airflow.operators.sqlite_operator import SqliteOperator
from airflow.operators.trigger_dagrun import TriggerDagRunOperator
from airflow.operators.weekday import BranchDayOfWeekOperator
from airflow.sensors import filesystem
from airflow.sensors.date_time import DateTimeSensor, DateTimeSensorAsync
from airflow.sensors.date_time_sensor import DateTimeSensor
from airflow.sensors.external_task import (
ExternalTaskMarker,
ExternalTaskSensor,
ExternalTaskSensorLink,
)
from airflow.sensors.filesystem import FileSensor
from airflow.sensors.hive_partition_sensor import HivePartitionSensor
from airflow.sensors.http_sensor import HttpSensor
from airflow.sensors.metastore_partition_sensor import MetastorePartitionSensor
from airflow.sensors.named_hive_partition_sensor import NamedHivePartitionSensor
from airflow.sensors.sql import SqlSensor
from airflow.sensors.sql_sensor import SqlSensor2
from airflow.sensors.time_delta import TimeDeltaSensor, TimeDeltaSensorAsync, WaitSensor
from airflow.sensors.time_sensor import TimeSensor, TimeSensorAsync
from airflow.sensors.web_hdfs_sensor import WebHdfsSensor
from airflow.sensors.weekday import DayOfWeekSensor
from airflow.triggers.external_task import DagStateTrigger, WorkflowTrigger
from airflow.triggers.file import FileTrigger
from airflow.triggers.temporal import DateTimeTrigger, TimeDeltaTrigger
from airflow.www.security import FabAirflowSecurityManagerOverride
# apache-airflow-providers-amazon
provide_bucket_name()
GCSToS3Operator()
GoogleApiToS3Operator()
GoogleApiToS3Transfer()
RedshiftToS3Operator()
RedshiftToS3Transfer()
S3FileTransformOperator()
S3Hook()
SSQLTableCheckOperator3KeySensor()
S3ToRedshiftOperator()
S3ToRedshiftTransfer()
# apache-airflow-providers-celery
DEFAULT_CELERY_CONFIG
app
CeleryExecutor()
CeleryKubernetesExecutor()
# apache-airflow-providers-common-sql
_convert_to_float_if_possible()
parse_boolean()
BaseSQLOperator()
BashOperator()
LegacyBashOperator()
BranchSQLOperator()
CheckOperator()
ConnectorProtocol()
DbApiHook()
DbApiHook2()
IntervalCheckOperator()
PrestoCheckOperator()
PrestoIntervalCheckOperator()
PrestoValueCheckOperator()
SQLCheckOperator()
SQLCheckOperator2()
SQLCheckOperator3()
SQLColumnCheckOperator2()
SQLIntervalCheckOperator()
SQLIntervalCheckOperator2()
SQLIntervalCheckOperator3()
SQLTableCheckOperator()
SQLThresholdCheckOperator()
SQLThresholdCheckOperator2()
SQLValueCheckOperator()
SQLValueCheckOperator2()
SQLValueCheckOperator3()
SqlSensor()
SqlSensor2()
ThresholdCheckOperator()
ValueCheckOperator()
# apache-airflow-providers-daskexecutor
DaskExecutor()
# apache-airflow-providers-docker
DockerHook()
DockerOperator()
# apache-airflow-providers-apache-druid
DruidDbApiHook()
DruidHook()
DruidCheckOperator()
# apache-airflow-providers-apache-hdfs
WebHDFSHook()
WebHdfsSensor()
# apache-airflow-providers-apache-hive
HIVE_QUEUE_PRIORITIES
closest_ds_partition()
max_partition()
HiveCliHook()
HiveMetastoreHook()
HiveOperator()
HivePartitionSensor()
HiveServer2Hook()
HiveStatsCollectionOperator()
HiveToDruidOperator()
HiveToDruidTransfer()
HiveToSambaOperator()
S3ToHiveOperator()
S3ToHiveTransfer()
MetastorePartitionSensor()
NamedHivePartitionSensor()
# apache-airflow-providers-http
HttpHook()
HttpSensor()
SimpleHttpOperator()
# apache-airflow-providers-jdbc
jaydebeapi
JdbcHook()
JdbcOperator()
# apache-airflow-providers-fab
basic_auth.CLIENT_AUTH
basic_auth.init_app
basic_auth.auth_current_user
basic_auth.requires_authentication
kerberos_auth.log
kerberos_auth.CLIENT_AUTH
kerberos_auth.find_user
kerberos_auth.init_app
kerberos_auth.requires_authentication
auth_current_user
backend_kerberos_auth
fab_override
FabAuthManager()
FabAirflowSecurityManagerOverride()
# check whether attribute access
basic_auth.auth_current_user
# apache-airflow-providers-cncf-kubernetes
ALL_NAMESPACES
POD_EXECUTOR_DONE_KEY
_disable_verify_ssl()
_enable_tcp_keepalive()
append_to_pod()
annotations_for_logging_task_metadata()
annotations_to_key()
create_pod_id()
datetime_to_label_safe_datestring()
extend_object_field()
get_logs_task_metadata()
label_safe_datestring_to_datetime()
merge_objects()
Port()
Resources()
PodRuntimeInfoEnv()
PodGeneratorDeprecated()
Volume()
VolumeMount()
Secret()
add_pod_suffix()
add_pod_suffix2()
get_kube_client()
get_kube_client2()
make_safe_label_value()
make_safe_label_value2()
rand_str()
rand_str2()
K8SModel()
K8SModel2()
PodLauncher()
PodLauncher2()
PodStatus()
PodStatus2()
PodDefaults()
PodDefaults2()
PodDefaults3()
PodGenerator()
PodGenerator2()
# apache-airflow-providers-microsoft-mssql
MsSqlHook()
MsSqlOperator()
MsSqlToHiveOperator()
MsSqlToHiveTransfer()
# apache-airflow-providers-mysql
HiveToMySqlOperator()
HiveToMySqlTransfer()
MySqlHook()
MySqlOperator()
MySqlToHiveOperator()
MySqlToHiveTransfer()
PrestoToMySqlOperator()
PrestoToMySqlTransfer()
# apache-airflow-providers-oracle
OracleHook()
OracleOperator()
# apache-airflow-providers-papermill
PapermillOperator()
# apache-airflow-providers-apache-pig
PigCliHook()
PigOperator()
# apache-airflow-providers-postgres
Mapping
PostgresHook()
PostgresOperator()
# apache-airflow-providers-presto
PrestoHook()
# apache-airflow-providers-samba
SambaHook()
# apache-airflow-providers-slack
SlackHook()
SlackAPIOperator()
SlackAPIPostOperator()
# apache-airflow-providers-sqlite
SqliteHook()
SqliteOperator()
# apache-airflow-providers-zendesk
ZendeskHook()
# apache-airflow-providers-smtp
EmailOperator()
# apache-airflow-providers-standard
filesystem.FileSensor()
FileSensor()
TriggerDagRunOperator()
ExternalTaskMarker()
ExternalTaskSensor()
BranchDateTimeOperator()
BranchDayOfWeekOperator()
BranchPythonOperator()
DateTimeSensor()
DateTimeSensorAsync()
TimeSensor()
TimeDeltaSensor()
DayOfWeekSensor()
DummyOperator()
EmptyOperator()
ExternalTaskMarker()
ExternalTaskSensor()
ExternalTaskSensorLink()
FileSensor()
FileTrigger()
FSHook()
PackageIndexHook()
SubprocessHook()
ShortCircuitOperator()
TimeDeltaSensor()
TimeSensor()
TriggerDagRunOperator()
WorkflowTrigger()
PythonOperator()
PythonVirtualenvOperator()
DagStateTrigger()
FileTrigger()
DateTimeTrigger()
TimeDeltaTrigger()
SubprocessResult()
SubprocessHook()
TimeDeltaSensor()
TimeDeltaSensorAsync()
WaitSensor()
TimeSensor()
TimeSensorAsync()
BranchDateTimeOperator()
working_directory()
target_times_as_dates()

View File

@@ -0,0 +1,39 @@
from __future__ import annotations
from airflow.hooks.S3_hook import (
S3Hook,
provide_bucket_name,
)
from airflow.operators.gcs_to_s3 import GCSToS3Operator
from airflow.operators.google_api_to_s3_transfer import (
GoogleApiToS3Operator,
GoogleApiToS3Transfer,
)
from airflow.operators.redshift_to_s3_operator import (
RedshiftToS3Operator,
RedshiftToS3Transfer,
)
from airflow.operators.s3_file_transform_operator import S3FileTransformOperator
from airflow.operators.s3_to_redshift_operator import (
S3ToRedshiftOperator,
S3ToRedshiftTransfer,
)
from airflow.sensors.s3_key_sensor import S3KeySensor
S3Hook()
provide_bucket_name()
GCSToS3Operator()
GoogleApiToS3Operator()
GoogleApiToS3Transfer()
RedshiftToS3Operator()
RedshiftToS3Transfer()
S3FileTransformOperator()
S3ToRedshiftOperator()
S3ToRedshiftTransfer()
S3KeySensor()

View File

@@ -0,0 +1,12 @@
from __future__ import annotations
from airflow.config_templates.default_celery import DEFAULT_CELERY_CONFIG
from airflow.executors.celery_executor import (
CeleryExecutor,
app,
)
DEFAULT_CELERY_CONFIG
app
CeleryExecutor()

View File

@@ -0,0 +1,129 @@
from __future__ import annotations
from airflow.hooks.dbapi import (
ConnectorProtocol,
DbApiHook,
)
from airflow.hooks.dbapi_hook import DbApiHook
from airflow.operators.check_operator import SQLCheckOperator
ConnectorProtocol()
DbApiHook()
SQLCheckOperator()
from airflow.operators.check_operator import CheckOperator
from airflow.operators.sql import SQLCheckOperator
SQLCheckOperator()
CheckOperator()
from airflow.operators.druid_check_operator import CheckOperator
CheckOperator()
from airflow.operators.presto_check_operator import CheckOperator
CheckOperator()
from airflow.operators.check_operator import (
IntervalCheckOperator,
SQLIntervalCheckOperator,
)
from airflow.operators.druid_check_operator import DruidCheckOperator
from airflow.operators.presto_check_operator import PrestoCheckOperator
DruidCheckOperator()
PrestoCheckOperator()
IntervalCheckOperator()
SQLIntervalCheckOperator()
from airflow.operators.presto_check_operator import (
IntervalCheckOperator,
PrestoIntervalCheckOperator,
)
from airflow.operators.sql import SQLIntervalCheckOperator
IntervalCheckOperator()
SQLIntervalCheckOperator()
PrestoIntervalCheckOperator()
from airflow.operators.check_operator import (
SQLThresholdCheckOperator,
ThresholdCheckOperator,
)
SQLThresholdCheckOperator()
ThresholdCheckOperator()
from airflow.operators.sql import SQLThresholdCheckOperator
SQLThresholdCheckOperator()
from airflow.operators.check_operator import (
SQLValueCheckOperator,
ValueCheckOperator,
)
SQLValueCheckOperator()
ValueCheckOperator()
from airflow.operators.presto_check_operator import (
PrestoValueCheckOperator,
ValueCheckOperator,
)
from airflow.operators.sql import SQLValueCheckOperator
SQLValueCheckOperator()
ValueCheckOperator()
PrestoValueCheckOperator()
from airflow.operators.sql import (
BaseSQLOperator,
BranchSQLOperator,
SQLColumnCheckOperator,
SQLTablecheckOperator,
_convert_to_float_if_possible,
parse_boolean,
)
BaseSQLOperator()
BranchSQLOperator()
SQLTablecheckOperator()
SQLColumnCheckOperator()
_convert_to_float_if_possible()
parse_boolean()
from airflow.sensors.sql import SqlSensor
SqlSensor()
from airflow.sensors.sql_sensor import SqlSensor
SqlSensor()
from airflow.operators.jdbc_operator import JdbcOperator
from airflow.operators.mssql_operator import MsSqlOperator
from airflow.operators.mysql_operator import MySqlOperator
from airflow.operators.oracle_operator import OracleOperator
from airflow.operators.postgres_operator import PostgresOperator
from airflow.operators.sqlite_operator import SqliteOperator
JdbcOperator()
MsSqlOperator()
MySqlOperator()
OracleOperator()
PostgresOperator()
SqliteOperator()

View File

@@ -0,0 +1,5 @@
from __future__ import annotations
from airflow.executors.dask_executor import DaskExecutor
DaskExecutor()

View File

@@ -0,0 +1,7 @@
from __future__ import annotations
from airflow.hooks.docker_hook import DockerHook
from airflow.operators.docker_operator import DockerOperator
DockerHook()
DockerOperator()

View File

@@ -0,0 +1,16 @@
from __future__ import annotations
from airflow.hooks.druid_hook import (
DruidDbApiHook,
DruidHook,
)
from airflow.operators.hive_to_druid import (
HiveToDruidOperator,
HiveToDruidTransfer,
)
DruidDbApiHook()
DruidHook()
HiveToDruidOperator()
HiveToDruidTransfer()

View File

@@ -0,0 +1,55 @@
from __future__ import annotations
from airflow.api.auth.backend.basic_auth import (
CLIENT_AUTH,
auth_current_user,
init_app,
requires_authentication,
)
CLIENT_AUTH
init_app()
auth_current_user()
requires_authentication()
from airflow.api.auth.backend.kerberos_auth import (
CLIENT_AUTH,
find_user,
init_app,
log,
requires_authentication,
)
log()
CLIENT_AUTH
find_user()
init_app()
requires_authentication()
from airflow.auth.managers.fab.api.auth.backend.kerberos_auth import (
CLIENT_AUTH,
find_user,
init_app,
log,
requires_authentication,
)
log()
CLIENT_AUTH
find_user()
init_app()
requires_authentication()
from airflow.auth.managers.fab.fab_auth_manager import FabAuthManager
from airflow.auth.managers.fab.security_manager.override import (
MAX_NUM_DATABASE_USER_SESSIONS,
FabAirflowSecurityManagerOverride,
)
FabAuthManager()
MAX_NUM_DATABASE_USER_SESSIONS
FabAirflowSecurityManagerOverride()
from airflow.www.security import FabAirflowSecurityManagerOverride
FabAirflowSecurityManagerOverride()

View File

@@ -0,0 +1,7 @@
from __future__ import annotations
from airflow.hooks.webhdfs_hook import WebHDFSHook
from airflow.sensors.web_hdfs_sensor import WebHdfsSensor
WebHDFSHook()
WebHdfsSensor()

View File

@@ -0,0 +1,66 @@
from __future__ import annotations
from airflow.hooks.hive_hooks import (
HIVE_QUEUE_PRIORITIES,
HiveCliHook,
HiveMetastoreHook,
HiveServer2Hook,
)
from airflow.macros.hive import (
closest_ds_partition,
max_partition,
)
from airflow.operators.hive_operator import HiveOperator
from airflow.operators.hive_stats_operator import HiveStatsCollectionOperator
from airflow.operators.hive_to_mysql import (
HiveToMySqlOperator,
HiveToMySqlTransfer,
)
from airflow.operators.hive_to_samba_operator import HiveToSambaOperator
from airflow.operators.mssql_to_hive import (
MsSqlToHiveOperator,
MsSqlToHiveTransfer,
)
from airflow.operators.mysql_to_hive import (
MySqlToHiveOperator,
MySqlToHiveTransfer,
)
from airflow.operators.s3_to_hive_operator import (
S3ToHiveOperator,
S3ToHiveTransfer,
)
from airflow.sensors.hive_partition_sensor import HivePartitionSensor
from airflow.sensors.metastore_partition_sensor import MetastorePartitionSensor
from airflow.sensors.named_hive_partition_sensor import NamedHivePartitionSensor
closest_ds_partition()
max_partition()
HiveCliHook()
HiveMetastoreHook()
HiveServer2Hook()
HIVE_QUEUE_PRIORITIES
HiveOperator()
HiveStatsCollectionOperator()
HiveToMySqlOperator()
HiveToMySqlTransfer()
HiveToSambaOperator()
MsSqlToHiveOperator()
MsSqlToHiveTransfer()
MySqlToHiveOperator()
MySqlToHiveTransfer()
S3ToHiveOperator()
S3ToHiveTransfer()
HivePartitionSensor()
MetastorePartitionSensor()
NamedHivePartitionSensor()

View File

@@ -0,0 +1,9 @@
from __future__ import annotations
from airflow.hooks.http_hook import HttpHook
from airflow.operators.http_operator import SimpleHttpOperator
from airflow.sensors.http_sensor import HttpSensor
HttpHook()
SimpleHttpOperator()
HttpSensor()

Some files were not shown because too many files have changed in this diff Show More