Compare commits

..

400 Commits

Author SHA1 Message Date
konsti
e87032d57d Explain check_docs_formatted.py error message (#6125)
## Summary

This is an error message only change to lead an implementor of a new
rule that has an unformatted or invalid bad example to the
right code.

## Test Plan

n/a
2023-07-27 15:53:52 +02:00
Charlie Marsh
0c2abf8316 Avoid walking past root when resolving imports (#6126)
## Summary

Noticed in #5954: we walk _past_ the root rather than stopping _at_ the
root when attempting to traverse along the parent path. It's effectively
an off-by-one bug.
2023-07-27 13:38:51 +00:00
konsti
6e85e0010f Fix windows test warnings (#6124)
See
https://github.com/astral-sh/ruff/actions/runs/5679922286/job/15392998698.
These didn't fail CI because we run clippy on linux only.
2023-07-27 14:03:53 +02:00
Micha Reiser
2ea0a0b28a Respect indent when measuring with MeasureMode::AllLines (#6120) 2023-07-27 13:34:36 +02:00
konsti
10abc8fd92 Unbreak main (#6123)
This fixes main breaking due to two merges.
2023-07-27 11:32:55 +00:00
konsti
61195bc7b1 Don't format trailing comma for lambda arguments (#5946)
**Summary** lambda arguments don't have parentheses, so they shouldn't
get a magic trailing comma either. This fixes some unstable formatting

**Test Plan** Added a regression test.

89 (from previously 145) instances of unstable formatting remaining.

```
$ cargo run --bin ruff_dev --release -- format-dev --stability-check --error-file formatter-ecosystem-errors.txt --multi-project target/checkouts > formatter-ecosystem-progress.txt
$ rg "Unstable formatting" target/formatter-ecosystem-errors.txt | wc -l
89
```

Closes #5892
2023-07-27 12:32:10 +02:00
Micha Reiser
9040554be5 Merge pull request #6119 from astral-sh/preserve-history 2023-07-27 12:12:57 +02:00
Micha Reiser
06ba1cf0ad Move RustPython Files 2023-07-27 11:46:18 +02:00
Micha Reiser
7a63a8f45a Merge branch 'main' of ../parser-move into preserve-history 2023-07-27 11:39:22 +02:00
Micha Reiser
c1bc0dc83d Delete RustPython files 2023-07-27 11:39:14 +02:00
Micha Reiser
90573975f3 Delete more unused files 2023-07-27 11:32:12 +02:00
Micha Reiser
802df97d55 Fix clippy issues 2023-07-27 11:31:10 +02:00
Micha Reiser
6f4ee32807 Move files to common directory 2023-07-27 11:27:37 +02:00
Zanie Blue
7ec979d022 Fix TypeAliasName to store name instead of load (#42)
Copies the fix in https://github.com/RustPython/Parser/pull/99
2023-07-26 17:53:34 -05:00
Micha Reiser
fab9cc5294 Remove parser ast re-export (#41) 2023-07-26 14:54:29 +02:00
Micha Reiser
ff7bab589e Prepare for moving the parser into the Ruff monorepo (#40) 2023-07-26 12:28:43 +02:00
Micha Reiser
593b46be5e perf: Cursor based lexer (#38) 2023-07-26 07:50:45 +02:00
David Szotten
13196fc500 fix the ranges of constants inside f-strings (#33) 2023-07-25 19:17:06 +02:00
Dhruv Manilawala
5ef4ccd632 Add line magic stmt and expr AST nodes (#31)
This PR introduces two new nodes for the parser to recognize the
`MagicCommand` tokens:

* `StmtLineMagic` for statement position i.e., magic commands on their
own line:
	```python
	%matplotlib inline
	!pwd
	```

* `ExprLineMagic` for expression position i.e., magic commands in an
assignment statement:
	```python
	# Only `?` and `!` are valid in this position
	dir = !pwd
	```

Both nodes are identical in their structure as in it contains the magic
kind and the command value as `String`.
2023-07-24 21:20:31 +05:30
Dhruv Manilawala
e363fb860e Lex Jupyter Magic in assignment value position (#30)
Emit `MagicCommand` token when it is the assignment value[^1] i.e., on
the right side of an assignment statement.

Examples:
```python
pwd = !pwd
foo = %timeit a = b
bar = %timeit a % 3
baz = %matplotlib \
        inline"
```

[^1]: Only `%` and `!` are valid in that position, other magic kinds are
not valid
2023-07-24 17:44:03 +05:30
Chris Pryer
4888d800fb Fix BoolOp python.org link (#37)
Noticed https://docs.python.org/3/library/ast.html#ast.boolop wasn't
working.
2023-07-22 14:10:59 -04:00
konsti
4d03b9b5b2 Fix empty lambda args range (#35)
Previously, empty lambda arguments (e.g. `lambda: 1`) would get the
range of the entire expression, which leads to incorrect comment
placement. Now empty lambda arguments get an empty range between the
`lambda` and the `:` tokens.
2023-07-21 17:41:19 +02:00
Zanie Blue
fb365c642b Implement Ranged for TypeParam (#32)
Required for https://github.com/astral-sh/ruff/pull/5927
2023-07-20 20:03:17 -04:00
konsti
db04fd4157 Refactor if statement, introduce nodes for elif and else (#22)
The old if layout couldn't differentiate between an else block with a
single if statement and an elif statement. Additionally we getting rid
of the recursion in favor of a single if struct with a vec of elif/else
children. This is accompanied by a big refactoring in ruff which removes
a bunch of TODOs and false negatives.
2023-07-18 13:14:40 +02:00
konstin
ff3e8ead36 Revert "snapshot"
This reverts commit 1bbec2b967.
2023-07-18 11:09:43 +02:00
konstin
1bbec2b967 snapshot 2023-07-18 11:08:14 +02:00
Dhruv Manilawala
510be51cfa Fix Jupyter Magic test snapshot (#29)
Sorry for the churn, I should've rebased and merged the original PR to
avoid this.
2023-07-17 23:06:53 -05:00
Dhruv Manilawala
3b4c8fffe5 Lex Jupyter line magic with Mode::Jupyter (#23)
Lex Jupyter line magic with `Mode::Jupyter`

This PR adds a new token `MagicCommand`[^1] which the lexer will
recognize when in `Mode::Jupyter`. The rules for the lexer is as
follows:
1. Given that we are at the start of line, skip the indentation and look
for [characters that represent the start of a magic
command](635815e8f1/IPython/core/inputtransformer2.py (L335-L346)),
determine the magic kind and capture all the characters following it as
the command string.
2. If the command extends multiple lines, the lexer will skip the line
continuation character (`\`) but only if it's followed by a newline
(`\n` or `\r`). The reason to skip this only in case of newline is
because they can occur in the command string which we should not skip:

	```rust
    //        Skip this backslash
    //        v
    //   !pwd \
    //      && ls -a | sed 's/^/\\    /'
    //                          ^^
    //                          Don't skip these backslashes
	```

3. The parser, when in `Mode::Jupyter`, will filter these tokens before
the parsing begins. There is a small caveat when the magic command is
indented. In the following example, when the parser filters out magic
command, it'll throw an indentation error:

	```python
	for i in range(5):
		!ls

	# What the parser will see
	for i in range(5):
	
	```

[^1]: I would prefer to have some other name as this not only represent
a line magic (`%`) but also shell command (`!`), help command (`?`) and
others. In original implementation, it's named as ["IPython
Syntax"](635815e8f1/IPython/core/inputtransformer2.py (L332))
2023-07-18 09:24:24 +05:30
Zanie
126652b684 Fix decorator ranges
Incorrectly merged LALRPOP file
2023-07-17 14:49:14 -05:00
Zanie
57e8712d76 Bump expected size of Stmt to 168 bytes 2023-07-17 14:49:14 -05:00
Zanie
78c6ede1c9 Format 2023-07-17 14:49:14 -05:00
Zanie Blue
a843a00f6b Add parsing of type alias statements i.e. the type keyword (#97)
Extends #95
Closes #82

Adds parsing of new `type` soft keyword for defining type aliases.

Supports type alias statements as defined in PEP 695 e.g.

```python
type IntOrStr = int | str

type ListOrSet[T] = list[T] | set[T]

type AnimalOrVegetable = Animal | "Vegetable"

type RecursiveList[T] = T | list[RecursiveList[T]]
```

All type parameter kinds are supported as in #95.

Builds on soft keyword abstractions introduced in https://github.com/RustPython/RustPython/pull/4519
2023-07-17 14:49:14 -05:00
Zanie Blue
f846f1ea42 Parse type parameters in function definitions (#96)
* Parse type parameters in function definitions
* Add test for combined items
2023-07-17 14:49:14 -05:00
Zanie
1d4b7a395f Consolidate tests and add coverage for trailing comma 2023-07-17 14:49:14 -05:00
Zanie
ce3ce0734b Add bound to test case test_parse_class_with_all_possible_generic_types 2023-07-17 14:49:14 -05:00
Zanie
b0e119f049 Add test for tuple bounds 2023-07-17 14:49:14 -05:00
Zanie
1f5e707829 Remove test for empty generic class Foo[]: ...
Not valid syntax
2023-07-17 14:49:14 -05:00
Zanie
ed7acfe477 Parse type parameters in class definitions 2023-07-17 14:49:14 -05:00
Zanie
c31b58eb39 Move type_param stubs into LALRPOP definition 2023-07-17 14:49:14 -05:00
Zanie
c0a3a20c63 Bump size assertion for Stmt from 136 to 160 bytes 2023-07-17 14:49:14 -05:00
Zanie
05ae26b935 Regenerate code with latest ASDL 2023-07-17 14:49:14 -05:00
David Szotten
b996b21ffc tuple constants are for optimisations, not source (#28)
my reading of https://docs.python.org/3/library/ast.html#ast.unparse and
https://discuss.python.org/t/ast-constant-value-tuple-s-and-frozenset-s/22578
is that tuple constants cannot come from parsing python source, they are
only for optimised bytecode

see also https://github.com/astral-sh/ruff/pull/5812
2023-07-17 15:46:37 -04:00
konsti
2d1f69cbb9 Remove asdl (#21)
This removes the ASDL code generation in favor of handwriting the AST. 

The motivations for moving away from the ASDL are:

* CPython compatibility is no longer a goal
* The ASDL grammar isn't as expressive as we would like
* The codegen scripts have a high complexity which makes extensions time
consuming
* We don't make heavy use of code generation (compared to e.g.
RustPython that generates Pyo3 bindings, a fold implementation etc).

We may want to revisit a grammar based code generation in the future,
e.g. by using [ungrammar](https://github.com/rust-analyzer/ungrammar)
2023-07-05 14:25:26 +02:00
konsti
0f2e295f3a impl Ranged for TextRange (#20)
This adds the missing implementation of `Ranged` for `TextRange` itself

```rust
impl Ranged for TextRange {
    fn range(&self) -> TextRange {
        *self
    }
}
```

This allows e.g. using `has_comments` with arbitrary ranges instead of
just a node.

It also adds .venv to the .gitignore
2023-07-02 10:11:06 +02:00
konstin
c174bbf1f2 impl<T> Ranged for &T where T: Ranged (#16)
In the example below, `arg` is `&Expr`, so `&Ranged`, but `entries()`
want a `T: Ranged`. This adds the missing bridge impl.

```rust
        let all_args = format_with(|f| {
            f.join_comma_separated()
                .entries(
                    // We have the parentheses from the call so the arguments never need any
                    args.iter()
                        .map(|arg| (arg, arg.format().with_options(Parenthesize::Never))),
                )
                .nodes(keywords.iter())
                .finish()
        });
```
2023-06-26 13:55:35 +02:00
Micha Reiser
8078663b6c Remove Range type parameter from AST nodes (#15) 2023-06-23 21:18:55 +01:00
Micha Reiser
f60e204b73 Fix range of keyword identifier (#14) 2023-06-22 10:00:25 +02:00
Micha Reiser
08ebbe40d7 Fix ArgWithDefault TextRange (#13) 2023-06-20 18:21:09 +02:00
Charlie Marsh
ed3b4eb72b Add TextRange to Identifier (#8)
## Summary

This PR adds `TextRange` to `Identifier`. Right now, the AST only
includes ranges for identifiers in certain cases (`Expr::Name`,
`Keyword`, etc.), namely when the identifier comprises an entire AST
node. In Ruff, we do additional ad-hoc lexing to extract identifiers
from source code.

One frequent example: given a function `def f(): ...`, we lex to find
the range of `f`, for use in diagnostics.

Another: `except ValueError as e`, for which the AST doesn't include a
range for `e`.

Note that, as an optimization, we avoid storing the `TextRange` for
`Expr::Name`, since it's already included.
2023-06-20 11:19:27 -04:00
Charlie Marsh
f0d200c8a1 Remove fold, unparse, and location features (#9) 2023-06-19 17:26:17 -04:00
Charlie Marsh
8d74eee750 Make malachite-bigint an optional dependency for rustpython-format (#12) 2023-06-19 16:31:25 -04:00
Charlie Marsh
21aa0b8d84 Optimize validate_arguments (#10) 2023-06-19 15:32:58 -04:00
Charlie Marsh
9cb00518e5 Update snapshot tests 2023-06-19 13:16:07 -04:00
Micha Reiser
6f65c5cba7 Add Decorator node (#7) 2023-06-19 12:53:05 -04:00
Micha Reiser
8a415fa61e Include argument parentheses in range (#5) 2023-06-19 12:51:18 -04:00
Micha Reiser
5054cbe84f Merge branch 'RustPython:main' into main 2023-06-19 12:38:24 -04:00
Jeong, YunWon
69d27d924c Rename unconventional nodes (#74) 2023-06-17 01:54:00 +09:00
Jeong, YunWon
5270020423 rustpython_ast python package (#79) 2023-06-17 01:32:27 +09:00
yt2b
edcfcb4a74 Fix bool format (#91)
* Fix format_bool

* Add test_format_bool
2023-06-15 14:56:03 +09:00
Jeong, YunWon
40a603208f Add Pylyzer
done by https://github.com/mtshiba/pylyzer/pull/37
2023-06-11 21:56:59 +09:00
Jeong, YunWon
b2f95e2848 Fix LinearLocator \r handling (#80) 2023-06-02 22:35:53 +09:00
Steve Shi
a2e3209c42 Replace num-bigint with malachite-bigint (#18)
Co-authored-by: Jeong YunWon <jeong@youknowone.org>
2023-06-02 17:06:18 +09:00
Jeong, YunWon
5e9e8a7589 Linear Locator (#46) 2023-06-01 13:53:31 +09:00
Jeong, YunWon
fdec727f80 New Arguments and Arg/ArgWithDefault AST representation (#59) 2023-06-01 01:15:23 +09:00
Jeong, YunWon
3fbf4f6804 Parse for expr and stmt variants + identifier, constant (#78) 2023-05-31 20:03:46 +09:00
Micha Reiser
fe25708d89 Merge pull request #76 from astral-sh/match-case-end-location 2023-05-31 09:48:42 +02:00
Micha Reiser
342cd19f50 Add safety comment 2023-05-31 09:37:48 +02:00
Micha Reiser
4a2c4aad0b Align MatchCase end location 2023-05-30 17:31:51 +02:00
Jeong, YunWon
ae3a477c97 Add parser deps to rustpython_ast_pyo3 (#75) 2023-05-29 16:48:05 +09:00
Jeong, YunWon
531aeb3511 Cmpop::as_str (#72)
* clean up pyo3 generation

* Cmpop::as_str
2023-05-29 01:53:54 +09:00
Jeong, YunWon
4de0cb1827 Parse Trait (#71) 2023-05-28 21:03:27 +09:00
Micha Reiser
5493c9f4e3 Avoid removing elements from the beginning of a vec (#69) 2023-05-25 19:44:49 +09:00
Jeong, YunWon
d23611db65 Merge pull request #65 from youknowone/refactor-pyo3
to_pyo3_ast to return &'py + Separate rustpython_ast_pyo3
2023-05-23 03:03:01 +09:00
Jeong YunWon
7d384d88d0 Separate rustpython_ast_pyo3 2023-05-23 02:50:12 +09:00
Jeong YunWon
b81273e9bc to_pyo3_ast to return &'py 2023-05-23 00:12:22 +09:00
Micha Reiser
335780aeea Merge pull request #2 from astral-sh/include-decorators-in-class-and-func-range 2023-05-22 13:46:38 +02:00
Jeong, YunWon
e1f02fced7 Fix Vec::to_pyo3_ast (#63) 2023-05-22 15:15:05 +09:00
Jeong, YunWon
d4084fb17a impl From<T> for ast::Ast (#62) 2023-05-21 22:44:07 +09:00
Micha Reiser
41a0ef8740 Add test 2023-05-19 10:14:46 +02:00
Micha Reiser
25cc1da319 Include decorators in Class and FunctionDef range 2023-05-19 08:18:43 +02:00
Micha Reiser
33a3c407a9 Merge pull request #60 from astral-sh/reduce-copy 2023-05-19 08:16:28 +02:00
Micha Reiser
fac0c25343 Update python.rs 2023-05-19 08:08:35 +02:00
Micha Reiser
4ff779c298 Update parser/src/python.lalrpop
Co-authored-by: Jeong, YunWon <69878+youknowone@users.noreply.github.com>
2023-05-18 22:32:05 +02:00
Micha Reiser
726884b287 Avoid allocating vecs for each statement 2023-05-18 21:56:17 +02:00
Micha Reiser
851f23668f Use named parameters to reducue copying 2023-05-18 21:41:06 +02:00
Jeong, YunWon
3654cf0bdf Add Ast as top level enum (#58) 2023-05-19 01:53:39 +09:00
Jeong, YunWon
6c5c311bab Fix range field order (#56)
* Skip validate_arguments when empty

* Fix `range` field order

* Fix unused variable
2023-05-18 21:44:01 +09:00
Jeong, YunWon
531e41ae2c Upgrade bitflags to remove clippy wanrings (#54)
* Upgrade bitflags to remove clippy wanrings

* Fix python lint warnings
2023-05-18 01:50:00 +09:00
Jeong, YunWon
b48834fe2d More flexible map_user and fold for new constructor nodes (#53)
* make fold.rs file

* Split user_map steps

* Fold for new constructor nodes
2023-05-18 00:16:04 +09:00
Jeong, YunWon
205ee80033 README (#47) 2023-05-17 18:01:00 +09:00
Jeong, YunWon
fc301ab1b0 Fix build (#45)
* remove dedent

* Revert parser-pyo3 crate
2023-05-16 23:54:49 +09:00
Jeong, YunWon
e820928f11 Add experimental pyo3-wrapper feature (#41)
* Fix pyo3 unit type value error

* Add experimental pyo3-wrapper feature

* location support
2023-05-16 23:45:31 +09:00
Jeong, YunWon
ff17f6e178 Add utilities to enum (#44)
* Add utilities to enum

* Fix unexpected pyo3 dependency propagation
2023-05-16 23:29:49 +09:00
Jeong YunWon
3bdf8a940a Hash for ConversionFlag 2023-05-16 22:56:17 +09:00
Jeong, YunWon
9d47d3d212 specialize ConversionFlag (#42)
* specialize ConversionFlag

* Change value of ConversionFlag to i8 and None to -1

* is_* methods to ConversionFlag
2023-05-16 22:52:50 +09:00
Jeong, YunWon
611dcc2e9b rustpython_ast + pyo3 (#25) 2023-05-16 18:06:54 +09:00
Jeong, YunWon
53de75efc3 Add all node classes to ast module (#40) 2023-05-16 16:55:26 +09:00
Jeong, YunWon
0c7d16b61a Add is_* methods to Tok (#39) 2023-05-16 16:08:25 +09:00
Jeong, YunWon
735c06d5f4 Fix full-lexer feature (#38) 2023-05-16 15:45:03 +09:00
Jeong, YunWon
02f13abf50 Add Node trait for node type information (#31) 2023-05-16 15:41:30 +09:00
Charlie Marsh
10dda125ff Always emit non-logical newlines for 'empty' lines (#27) 2023-05-15 14:13:05 -04:00
Jeong, YunWon
27e3873dc2 Add full-lexer feature (#36) 2023-05-16 02:21:34 +09:00
Micha Reiser
dd4cc25227 Reduce copying elements when parsing (#35) 2023-05-15 23:20:44 +09:00
Jeong, YunWon
b78001c953 Generic types to generic (#30) 2023-05-15 20:22:42 +09:00
Jeong, YunWon
718354673e Move range to node (#23)
* black + clippy

* Fix module generation
2023-05-15 16:20:22 +09:00
Micha Reiser
192379cede Move range from Attributed to Nodes (#22)
* Move `range` from `Attributed` to `Node`s

* No Attributed + custom for Range PoC

* Generate all located variants, generate enum implementations

* Implement `Copy` on simple enums

* Move `Suite` to `ranged` and `located`

* Update tests
---------

Co-authored-by: Jeong YunWon <jeong@youknowone.org>
2023-05-15 15:08:12 +09:00
Micha Reiser
a983f4383f Add format and cformat modules from RustPython (#24)
* Add `format` and `cformat` modules from `RustPython`

* Introduce `rustpython-format` crate

* Remove unused dependencies
2023-05-12 18:27:05 +09:00
Jeong, YunWon
947fb53d0b Field accessor and utilities (#20)
* Apply is-macro to Constant and ast nodes
2023-05-11 19:44:20 +09:00
Jeong, YunWon
2baad9ead6 Merge pull request #19 from RustPython/identifier
A few more Identifier utilities
2023-05-11 16:12:46 +09:00
Jeong YunWon
2af9805662 A few more Identifier utilities 2023-05-11 16:05:11 +09:00
Jeong, YunWon
5b2af304a2 Merge pull request #17 from youknowone/ruff
ruff integration support
2023-05-11 04:42:52 +09:00
Jeong YunWon
cbe4e8c5f3 Make parser location optional 2023-05-11 04:40:10 +09:00
Jeong YunWon
aabc96dde9 ruff integration support 2023-05-11 04:08:57 +09:00
Jeong YunWon
99e108dd53 *Escape::with_preferred_quote 2023-05-10 21:51:01 +09:00
Jeong YunWon
d6b6df5d1c ast::Int::to_usize 2023-05-10 21:31:02 +09:00
Jeong YunWon
822cac5aa0 parse_expression{=>_starts}_at 2023-05-10 21:25:58 +09:00
Jeong YunWon
aa101e4f26 fix ImportDots 2023-05-10 20:31:04 +09:00
Jeong, YunWon
cbc4cb286b Merge pull request #16 from youknowone/ast-int
Give identifier and int ast types
2023-05-10 20:19:01 +09:00
Jeong YunWon
6fa3d0f90a Fine-tune int types 2023-05-10 19:33:39 +09:00
Jeong YunWon
455bcc01a0 Give identifier and int ast types 2023-05-10 19:33:39 +09:00
Jeong, YunWon
d495cd9129 Merge pull request #15 from youknowone/spellchecker
Setup spell checker
2023-05-10 18:11:40 +09:00
Jeong YunWon
d8822d1091 spell check ast/asdl_rs.py 2023-05-10 18:05:39 +09:00
Jeong YunWon
75f6ce1ae5 Setup spell checker 2023-05-10 17:57:15 +09:00
Jeong, YunWon
41e9e7280a Merge pull request #12 from youknowone/visitor
Visitor
2023-05-10 17:52:25 +09:00
Jeong YunWon
17c8abcec1 asdl_rs.py to multiple file output 2023-05-10 17:41:01 +09:00
Jeong YunWon
e000b1c304 Remove redundant types 2023-05-10 17:15:01 +09:00
Jeong YunWon
243ca16b34 Fix visitor to fit in new structure 2023-05-10 17:14:44 +09:00
Joshua
4de9580b92 Generate a visitor trait to ast_gen.rs 2023-05-10 17:14:34 +09:00
Jeong, YunWon
5cf85f0b9d Merge pull request #13 from youknowone/locator
Python location transform and related refactoring
2023-05-10 17:04:30 +09:00
Jeong YunWon
4dc030ba9d Vendor SourceLocation from ruff 2023-05-10 17:00:12 +09:00
Jeong YunWon
1d366d52ab Let located only for python located stuff 2023-05-10 14:35:38 +09:00
Jeong YunWon
a3d9d8cb14 numerous refactoring
- Split parser core and compiler core. Fix #14
- AST int type to `u32`
- Updated asdl_rs.py and update_asdl.sh fix #6
- Use `ruff_python_ast::SourceLocation` for Python source location. Deleted our own Location.
- Renamed ast::Located to ast::Attributed to distinguish terms for TextSize and SourceLocation
- `ast::<Node>`s for TextSize located ast. `ast::located::<Node>` for Python source located ast.
- And also strictly renaming `located` to refer only python location related interfaces.
- `SourceLocator` to convert locations.
- New `source-code` features of to disable python locations when unnecessary.
- Also including fully merging https://github.com/astral-sh/RustPython/pull/4 closes #9
2023-05-10 14:35:38 +09:00
Jeong YunWon
09a6afdd04 Adapt SourceLocation 2023-05-09 20:34:48 +09:00
Jeong YunWon
a14e43e03a Separate byteoffset ast and located ast 2023-05-09 00:21:52 +09:00
Jeong YunWon
f47dfca4e3 Rename compiler Location to TextSize 2023-05-08 03:38:10 +09:00
Micha Reiser
58c35ab458 Replace row/column based Location with byte-offsets. 2023-05-08 03:38:10 +09:00
Jeong, YunWon
7b8844bd3e Merge pull request #11 from youknowone/refactor-asdl
Refactor ast to hold data as seperated type
2023-05-07 19:29:02 +09:00
Jeong YunWon
6d7358090b Refactor ast to hold data as seperated type 2023-05-07 19:20:47 +09:00
Jeong YunWon
9f1a538eba gitattribute 2023-05-07 17:20:52 +09:00
Jeong, YunWon
48920a034e Merge pull request #10 from youknowone/remove-compile-error
Remove CompileError
2023-05-06 17:20:44 +09:00
Jeong YunWon
13d6e275ef Remove CompileError 2023-05-06 17:15:18 +09:00
Jeong, YunWon
96eb80f5cf Merge pull request #8 from youknowone/python-lint
Add python lint
2023-05-06 14:42:15 +09:00
Jeong YunWon
a73bac9ed1 Add python lint 2023-05-06 14:34:31 +09:00
Jeong, YunWon
6b60f85cc4 Merge pull request #7 from youknowone/lalrpop
Embed generated parser + update lalrpop
2023-05-06 13:57:15 +09:00
Jeong YunWon
39b2dbe04d Update lalrpop to 0.20.0 2023-05-06 13:48:20 +09:00
Jeong YunWon
e1f70100ac Update parser/build.rs to embed python.rs 2023-05-06 05:28:30 +09:00
Jeong, YunWon
d66d935879 Merge pull request #5 from youknowone/ci
Add CI
2023-05-06 03:42:28 +09:00
Jeong YunWon
e28f333f23 Add CI 2023-05-06 03:39:20 +09:00
Jeong YunWon
0adcdd995d Remove unused workspace dependencies 2023-05-06 00:44:34 +09:00
Jeong YunWon
e6eb49ffb0 Set up workspace - Forked from
git@github.com:RustPython/RustPython.git ff5076b12c075b3e87c0ac2971e390b4a209d14f
2023-05-05 23:49:50 +09:00
Jeong YunWon
80109b1fe0 rustpython-literal 2023-05-05 21:49:12 +09:00
Jeong YunWon
bd64603950 Refactor common::bytes::repr using common::escape 2023-05-03 16:08:48 +09:00
Jeong YunWon
5b0e92d725 Refactor common::str::repr using common::escape 2023-05-03 13:47:04 +09:00
Micha Reiser
ae9d3c3193 Add Located::start, Located::end and impl Deref 2023-04-26 10:24:34 -06:00
Micha Reiser
3873414b30 Use Located::new over struct initializer 2023-04-25 18:10:13 -06:00
Jeong YunWon
a9f4b59f40 module objects' type as PyModule 2023-04-20 20:48:57 +09:00
Jeong YunWon
6d6155413b Update lalrpop 2023-03-25 22:18:02 +09:00
Jeong YunWon
f9b5469642 Update cspell for compiler 2023-03-16 22:39:09 +09:00
Charlie Marsh
ef38eb6b1a Include Derive feature with optional Serde dependency 2023-03-12 14:46:10 -04:00
Jeong YunWon
0ea53825db Merge pull request #4608 from coolreader18/bag-deser
Rework frozen modules and directly deserialize to CodeObject<Literal>
2023-03-10 05:19:23 +09:00
Noa
9d6ae774f8 Rework frozen modules and directly deserialize to CodeObject<Literal> 2023-03-06 13:45:33 -06:00
Charlie Marsh
55fc0e83f3 Treat match and case as soft keywords in lambda assignments (#4623) 2023-03-04 12:42:05 -05:00
Charlie Marsh
713dd2b91e Add optional serde dependency 2023-03-02 15:38:40 -05:00
Noa
969ea23d67 Address review comments 2023-03-01 21:11:59 -06:00
Noa
b80bbec8e6 Custom marshal enc/decoding impl 2023-03-01 20:47:21 -06:00
DimitrisJim
8f425e9ce2 Use insta to verify values. 2023-02-28 20:00:47 +09:00
Jeong YunWon
d7e2e7361e Use git version of unicode_names2 to avoid alias search failure 2023-02-28 20:00:47 +09:00
Charlie Marsh
c37e0c7f03 Use proper locations for sub-expressions in constant matches 2023-02-26 23:10:38 -05:00
Charlie Marsh
0d7b94817d Allow type variable tuple for *args 2023-02-23 08:39:59 -05:00
Jeong YunWon
f43e5b72e2 Merge pull request #4552 from charliermarsh/charlie/loc
Limit match range to end of last statement
2023-02-23 02:11:42 +09:00
Jeong YunWon
a19f294b0c Merge pull request #4543 from youknowone/flatten-parser
Flatten parser interface
2023-02-23 02:09:46 +09:00
Charlie Marsh
7ebef61c47 Limit match range to end of last statement 2023-02-22 11:25:50 -05:00
Charlie Marsh
b61f4d7b69 Allow trailing commas in MappingPattern 2023-02-22 10:02:41 -05:00
Jeong YunWon
e26369a34e use super::* from tests submodules 2023-02-22 21:01:39 +09:00
Jeong YunWon
97a08ee77b remove #[macro_use] 2023-02-22 20:41:27 +09:00
Jeong YunWon
cb8c6fb78d Flatten rustpython_parser interface 2023-02-22 20:32:31 +09:00
Jeong YunWon
8580e4ebb5 make_tokenizer -> lex to integrate terms
we don't distinguish scanner or tokenizer from lexer
2023-02-22 20:28:15 +09:00
Jeong YunWon
39fc23cf92 relocate feature-independent use 2023-02-22 20:28:03 +09:00
Jeong YunWon
66e3080173 Fix ModeParseError message 2023-02-22 20:28:03 +09:00
Jeong YunWon
1511b6631b Break down rustpython_parser::error module
because it doesn't share any common errors but specific error for each sub module
2023-02-22 20:28:01 +09:00
Charlie Marsh
2a8aa6f308 Always wrap in SoftKeywordTransformer 2023-02-21 19:18:42 -05:00
Charlie Marsh
dc628cab8f Expose SoftKeywordTransformer on public API 2023-02-21 19:00:32 -05:00
Jim Fasarakis-Hilliard
c137bc9d77 Merge pull request #4519 from charliermarsh/charlie/match
Add support for match statements to parser
2023-02-21 19:43:28 +02:00
Jeong YunWon
60180fd54c Merge pull request #4531 from charliermarsh/charlie/exception-groups
Implement except* syntax
2023-02-21 13:20:18 +09:00
Charlie Marsh
c7ed645cc6 Implement except* syntax 2023-02-21 12:19:54 +09:00
Charlie Marsh
8caa28f0f8 Update Python.asdl from CPython 3.11.1 2023-02-21 12:19:51 +09:00
Charlie Marsh
8aa3bc93f3 Allow starred expressions in subscripts 2023-02-20 17:59:35 -05:00
Charlie Marsh
f39ffef370 Update compiler/parser/src/soft_keywords.rs
Co-authored-by: Jim Fasarakis-Hilliard <d.f.hilliard@gmail.com>
2023-02-20 15:03:39 -05:00
Jeong YunWon
e093d2bee6 clean up soft-keyword transform 2023-02-20 15:03:39 -05:00
Charlie Marsh
ca5b474d45 Use muiltipeek 2023-02-20 15:03:39 -05:00
Charlie Marsh
2b43d45bd5 Add support for match statements to parser 2023-02-20 15:03:39 -05:00
John Pham
4bdc2d47c1 Make common::repr throw error instead of panic (#4520) 2023-02-19 22:09:54 +09:00
Jeong YunWon
09b82e41d5 Merge pull request #4490 from DimitrisJim/function_parser
Add tests, some comments, to function.rs.
2023-02-13 20:00:35 +09:00
Jeong YunWon
cc6d8a1c58 Merge pull request #4492 from DimitrisJim/doc_parser_uno
Document parser crate.
2023-02-13 17:25:14 +09:00
Dimitris Fasarakis Hilliard
07918f0a9a Document parser crate. 2023-02-12 17:58:19 +02:00
Dimitris Fasarakis Hilliard
a0786ea872 Add tests, some comments, to function.rs. 2023-02-11 23:07:57 +02:00
Dimitris Fasarakis Hilliard
4713b2b3ab Refactor: Join string and string_parser. 2023-02-11 18:05:06 +02:00
Jeong YunWon
e7f14ab9b8 Add test_generator_expression_argument 2023-02-11 05:20:39 +09:00
Charlie Marsh
56c73cc63d Use entire range for generators-as-arguments 2023-02-10 10:39:40 -05:00
Dimitris Fasarakis Hilliard
659f4dd8bf Document lexer. 2023-02-07 21:43:57 +02:00
Dimitris Fasarakis Hilliard
bd158089e0 Move NewLineHandler inline, don't check each character twice. 2023-02-07 20:58:53 +02:00
Jeong YunWon
a73bee7aae use workspace dependencies 2023-02-06 15:30:38 +09:00
Dimitris Fasarakis Hilliard
1468fe46ab Hint that the unwrap should always succeed. 2023-02-01 12:17:33 +02:00
Dimitris Fasarakis Hilliard
c9364718b4 Eat for comma. 2023-01-31 12:26:05 +02:00
Dimitris Fasarakis Hilliard
38cf933bcb Add initial capacities, use u32s for indents/spaces. 2023-01-31 12:26:05 +02:00
Dimitris Fasarakis Hilliard
838990ae15 Don't call is_emoji_presentation for each invocation of consume_normal 2023-01-31 12:26:05 +02:00
Dimitris Fasarakis Hilliard
aa0290bbfc Match on ascii start/continuation characters before calling functions. 2023-01-31 12:26:05 +02:00
Aarni Koskela
f74e44d1e8 Bump phf to 0.11 series
string_cache is still using phf_shared 0.10.0 though
2023-01-25 19:58:43 +02:00
Jeong YunWon
d9df131720 Merge pull request #4449 from harupy/fix-dict-spread-in-dict
Fix AST generated from a dict literal containing dict unpacking
2023-01-22 20:44:26 +09:00
harupy
f2ffe12f8c Fix comment 2023-01-22 00:06:52 +09:00
Anders Kaseorg
6dba8430be Fix end location for elif blocks
Since we parse an `elif:` block as an `If` node, its location should
include its `orelse` node like it would for an `if:` block.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2023-01-17 22:39:44 -05:00
harupy
b26365b215 Remove useless String::from
Signed-off-by: harupy <hkawamura0130@gmail.com>
2023-01-16 21:27:57 +09:00
harupy
2d019930e9 Rename test 2023-01-15 23:36:07 +09:00
harupy
d5fc7c4c87 Improve test 2023-01-15 16:53:13 +09:00
harupy
393869c47c Add Option to Dict.keys field 2023-01-15 16:43:13 +09:00
harupy
4edd2bf78a Remove commented-out code 2023-01-15 16:10:51 +09:00
harupy
d3c4551629 Fix unparse 2023-01-15 13:11:55 +09:00
harupy
581f6e176c Fix dict spreading in dict literal 2023-01-15 13:01:59 +09:00
Jim Fasarakis-Hilliard
ac4d3c076c Merge pull request #4443 from bluetech/non-logical-newline-token-fixup
Fixup parse_tokens after "Add NonLogicalNewline token"
2023-01-14 12:49:04 +02:00
Ran Benita
e5fe037e38 Fixup parse_tokens after "Add NonLogicalNewline token"
I only updated `parse()` to ignore `NonLogicalNewline`, didn't notice
it's also needed in `parse_tokens()`.
2023-01-14 11:52:33 +02:00
Noa
872b9d4765 Switch from 64-bit instruction enum to out-of-line arg values 2023-01-12 23:05:17 -06:00
Ran Benita
674eeec29c Add NonLogicalNewline token
This token is completely ignored by the parser, but it's useful for
other users of the lexer, such as the Ruff linter. For example, the
token is helpful for a "trailing comma" lint.

The same idea exists in Python's `tokenize` module - there is a NEWLINE
token (logical newline), and a NL token (non-logical newline).

Fixes #4385.
2023-01-12 16:47:12 +02:00
Martin Fischer
4f1e7c6291 Fix docs.rs build for rustpython-parser
docs.rs failed to build the documentation of the recently released
rustpython-parser 0.2.0 because the build.rs script couldn't write the
parser.rs file because docs.rs builds the documentation in a sandbox
with a read-only filesystem.

This commit fixes this by writing the parser.rs file to the cargo output
directory instead, as recommended by the docs.rs documentation.[1]

Fixes #4436.

[1]: https://docs.rs/about/builds#read-only-directories
2023-01-11 09:58:10 +01:00
Noa
884a7bdb15 Bump all crate versions to 0.2.0 2023-01-11 00:14:28 -06:00
Jeong YunWon
7885344bcf first cspell dict 2023-01-09 19:57:23 +09:00
Jeong YunWon
e8fef39861 Merge pull request #4429 from youknowone/fix-format
Fix nightly clippy warnings
2023-01-07 21:38:41 +09:00
Jeong YunWon
509cf7ed0d Fix nightly clippy warnings 2023-01-07 21:07:10 +09:00
harupy
a4a5366504 Include comment text in token 2023-01-06 23:29:20 +09:00
harupy
2dfd053bed Implement Default for Location 2023-01-05 22:48:47 +09:00
harupy
7f552e4594 Address comments 2023-01-05 18:24:54 +09:00
harupy
9efa872023 Use try_from 2023-01-05 01:18:30 +09:00
harupy
fd8468c5eb Simplify string check 2023-01-04 23:57:50 +09:00
Jeong YunWon
958c7e33ad Merge pull request #4417 from harupy/add-with-offset-methods
Add `with_col_offset` and `with_row_offset` to `Location` for conveniece
2023-01-04 17:07:31 +09:00
Jeong YunWon
0f311cd5e5 Merge pull request #4413 from harupy/more-generic-window-impl
Update `CharWindow` in `compiler/parser/src/lexer.rs` to allow slicing
2023-01-04 17:06:06 +09:00
harupy
0365752bf3 Use Self 2023-01-04 13:19:23 +09:00
harupy
6d140426c1 Add with_col_offset and with_row_offset to Location 2023-01-04 13:16:03 +09:00
harupy
33a62789f7 Address comment 2023-01-04 00:03:18 +09:00
harupy
84dff79ddc Remove incorrect EmptyExpression in parse_formatted_value 2023-01-03 23:55:50 +09:00
harupy
300710f7db Improve CharWindow 2023-01-03 17:27:35 +09:00
harupy
515dceb07b Remove repetitive to_string in parse_escaped_char 2023-01-03 14:48:00 +09:00
Jeong YunWon
2858c315bf Merge pull request #4409 from harupy/improve-error-conversion-in-string-parser
Improve error conversion in `string_parsers.rs`
2023-01-03 14:39:01 +09:00
harupy
3b0fd61b3b Fix clippy error 2023-01-03 12:51:31 +09:00
harupy
9030679193 Improve error conversion in string_parsers.rs 2023-01-03 12:46:10 +09:00
Jeong YunWon
71ba4226c1 Merge pull request #4405 from harupy/use-drain
Use `drain` to simplify `compiler/parser/src/string_parser.rs`
2023-01-03 12:41:23 +09:00
Jeong YunWon
3d03439618 Merge pull request #4399 from branai/shell-continuing-fix
Fix IndentationError works differently with cpython in interective shell
2023-01-03 04:32:03 +09:00
harupy
39f410c909 Use drain 2023-01-03 01:23:44 +09:00
Jim Fasarakis-Hilliard
4222b13e6c Merge pull request #4404 from harupy/merge-match-arms
Merge match arms in `StringParser.parse_formatted_value`
2023-01-02 18:18:40 +02:00
Jim Fasarakis-Hilliard
d691527ead Merge pull request #4402 from harupy/remove-unreachable-if
Remove unreachable code in `compiler/parser/src/string_parser.rs`
2023-01-02 17:05:45 +02:00
harupy
40e2150f7e Merge match arms in parse_formatted_value 2023-01-02 23:16:51 +09:00
harupy
92bf96608c Fix match 2023-01-02 22:54:48 +09:00
harupy
fac6a857f6 Simplify code using match 2023-01-02 22:26:09 +09:00
harupy
68586f8e3c Remove unreachable code in compiler/parser/src/string_parser.rs 2023-01-02 20:48:40 +09:00
Jeong YunWon
f8787a9377 Move (c)format basic implementations to rustpython-common 2023-01-02 20:21:36 +09:00
Bijan Naimi
530a20cc96 forgot to add formatted errors.rs 2023-01-01 17:28:49 -08:00
Bijan Naimi
6cb57b2075 changed the shell logic for handling indents 2023-01-01 15:41:51 -08:00
Dimitris Fasarakis Hilliard
92b2574d52 Move tests for with into parser. 2023-01-01 21:36:07 +02:00
Jim Fasarakis-Hilliard
95fb938bd6 Merge pull request #4389 from harupy/4384-follow-up
Follow-up for #4384
2023-01-01 14:53:54 +02:00
harupy
9683314264 Remove unreachable code 2023-01-01 17:43:25 +09:00
harupy
d9bbeeb9b3 Fix NamedExpr location 2022-12-31 23:32:08 +09:00
harupy
68116d5c11 Move tests 2022-12-31 12:15:33 +09:00
Jeong YunWon
e164a41723 Merge pull request #4373 from andersk/named
Allow named expression in subscript and set comprehension
2022-12-31 10:58:09 +09:00
Jeong YunWon
aa32a73c5b Merge pull request #4379 from harupy/refactor-FStringParser
Refactor `FStringParser`
2022-12-31 10:56:52 +09:00
Jeong YunWon
9d7d629cef Merge pull request #4384 from harupy/parse-formatted-value
Fix the location of `FormattedValue`
2022-12-31 10:52:44 +09:00
harupy
439298e735 Fix FormattedValue location 2022-12-30 21:39:29 +09:00
anilbey
16173bf581 Update compiler/parser/src/error.rs
Co-authored-by: fanninpm <fanninpm@miamioh.edu>
2022-12-29 22:56:34 +01:00
Anil Tuncel
7e7b2eadee format using cargo fmt 2022-12-29 22:15:44 +01:00
Anil Tuncel
80116c768d arg name to be written upon duplicate kwargs error #4381 2022-12-29 22:06:41 +01:00
Nick Liu
200390c1ab format code 2022-12-29 22:49:26 +08:00
Nick Liu
24d2ab8b0a use is_none 2022-12-29 22:49:26 +08:00
Nick Liu
41f21a7b5d add arg_name in duplicate argument error msg 2022-12-29 22:49:26 +08:00
Nick Liu
63e4a36e27 added check: named arguments must follow bare star 2022-12-29 22:49:26 +08:00
Nick Liu
15ab44384c added lex error: DuplicateArguments 2022-12-29 22:49:26 +08:00
Jim Fasarakis-Hilliard
5380b85579 Merge pull request #4367 from andersk/star-order
Prohibit starred arguments after double-starred arguments
2022-12-29 16:27:16 +02:00
Harutaka Kawamura
b707f53f23 Update compiler/parser/src/fstring.rs
Co-authored-by: Zgarbul Andrey <zgarbul.andrey@gmail.com>
2022-12-29 08:10:33 +09:00
Anders Kaseorg
6439867f78 Allow named expression in set comprehension: {a := b for c in d}
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-12-28 09:57:32 -08:00
Anders Kaseorg
1904d095f9 Allow named expression in subscript: a[b := c]
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-12-28 09:57:32 -08:00
harupy
7e8f683808 Rename 2022-12-29 01:01:41 +09:00
harupy
37b1894834 Clean up FStringParser 2022-12-29 00:08:59 +09:00
Jim Fasarakis-Hilliard
201d08583a Merge pull request #4377 from andersk/duplicate-from
Remove duplicate declaration of "from" token
2022-12-28 11:19:54 +02:00
Anders Kaseorg
107b2e11ae Remove duplicate declaration of "from" token
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-12-27 18:56:24 -08:00
Anders Kaseorg
661b210391 Prohibit starred arguments after double-starred arguments
CPython prohibits ‘f(**kwargs, *args)’; we should too.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-12-27 12:49:50 -08:00
harupy
9aed9143fe Refactor parse_formatted_value 2022-12-28 00:23:25 +09:00
harupy
c21d0d9283 Fix make_tokenizer_located 2022-12-27 21:54:18 +09:00
Jeong YunWon
313fd7d28c Merge pull request #4359 from yt2b/check_bom
Add BOM check
2022-12-26 16:03:51 +09:00
Jeong YunWon
3aa0096212 Merge pull request #4358 from harupy/fix-slice-location
Fix `Slice` location
2022-12-26 16:03:09 +09:00
Jeong YunWon
53dec88029 Merge pull request #4356 from andersk/with-tuple-named
Fix parsing of tuple with named expression as context manager
2022-12-26 16:02:17 +09:00
yt2b
bd0c15d34e Fix comment 2022-12-26 09:30:12 +09:00
yt2b
ce0be73841 Add BOM check 2022-12-25 11:15:29 +09:00
harupy
b2ac4f60f1 Fix slice location 2022-12-25 09:37:07 +09:00
Anders Kaseorg
1fdfa5fe1b Fix parsing of tuple with named expression as context manager
Because the ‘with’ item grammar disallows named expressions, CPython
parses ‘with (a := 0, b := 1):’ as a tuple rather than two ‘with’
items.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-12-24 13:15:53 -08:00
harupy
c0f390ebc6 Fix IfExp location 2022-12-25 00:33:02 +09:00
Anders Kaseorg
c387b5d523 Simplify parenthesized context manager parsing with LALRPOP conditions
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-12-24 23:13:10 +09:00
harupy
ea95e1a715 Fix the location of BinOp 2022-12-21 22:05:05 +09:00
Jim Fasarakis-Hilliard
c16e08d59b Merge pull request #4340 from harupy/fix-locations-of-parethesized-expressions
Fix the start and end locations of `Tuple`
2022-12-18 15:17:40 +02:00
harupy
ff00460ff4 Fix locations of parethesized expressions 2022-12-18 20:53:30 +09:00
Anders Kaseorg
f4672e4256 Remove unnecessary boxing of ASDL product children
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-12-14 02:30:02 -08:00
Dimitris Fasarakis Hilliard
6e89b3ab1a Fix end location in with statements. 2022-12-13 13:28:17 +02:00
Jim Fasarakis-Hilliard
6a174dae45 Merge pull request #4327 from harupy/fix-end-location-body
Fix end location of compound statements
2022-12-13 12:30:21 +02:00
Charlie Marsh
8936ab2f8d Set ExprContext::Store on parenthesized with expressions 2022-12-12 09:09:15 -05:00
harupy
165b979733 Refactor 2022-12-12 22:36:34 +09:00
harupy
06c92bb899 Update snapshot 2022-12-12 22:26:03 +09:00
harupy
1a657731dd Format 2022-12-12 22:18:26 +09:00
harupy
d8cfc7e84f Resolve conflict 2022-12-12 22:16:46 +09:00
harupy
de2e88656e Address comments
Signed-off-by: harupy <hkawamura0130@gmail.com>
2022-12-12 22:14:05 +09:00
Anders Kaseorg
052dee72b8 Parse Python 3.9+ parenthesized context managers
Since the upstream grammar for this is not LR(1), we abuse LALRPOP
macros and the Into/TryInto traits to build a cover grammar that
converts to either tuples or `with` items after additional validation.
It’s annoying and ugly, but something like this is basically our only
option short of switching to a more powerful parser algorithm.

Fixes #4145.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-12-12 00:47:33 -08:00
Anders Kaseorg
751f9d304f Split and simplify some LALRPOP rules
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-12-11 22:02:08 -08:00
harupy
d6f9dd0763 Use method chaining 2022-12-12 10:24:00 +09:00
harupy
5fc8c4d0b1 Fix other compound statements 2022-12-12 01:10:42 +09:00
harupy
1b7a272b77 Fix end location of nodes containing body 2022-12-11 12:35:28 +09:00
harupy
3abdc87076 Refactor 2022-12-10 22:01:42 +09:00
harupy
99b02be35a Fix 2022-12-10 18:45:36 +09:00
harupy
f99167d4ed Fix plain string 2022-12-10 18:09:26 +09:00
harupy
816e1e711c Fix the end location of an implicitly-concatenated string 2022-12-10 17:49:57 +09:00
Jeong YunWon
e2b28d07c8 Merge pull request #4310 from youknowone/fix-clippy
Fix nightly clippy warnings
2022-12-05 12:57:01 +09:00
Jeong YunWon
28785784b2 Fix nightly clippy warnings 2022-12-05 12:18:16 +09:00
harupy
6f6b7b2312 add tests 2022-12-04 05:59:51 +09:00
harupy
93cb05ebda Fix location 2022-12-04 05:45:15 +09:00
yt2b
e8200ab674 use bool.then 2022-11-24 09:23:20 +09:00
yt2b
492f09298f restore if expression 2022-11-23 11:30:54 +09:00
yt2b
35eea0b8ec Refactor lexer functions 2022-11-22 21:10:19 +09:00
yt2b
1eeddb521e Add test 2022-11-21 22:36:05 +09:00
Charlie Marsh
96a50810a6 Use match 2022-11-20 23:41:52 -05:00
Charlie Marsh
51b7dbb89c Use rustc-hash 2022-11-20 15:30:19 -05:00
Charlie Marsh
28a8c3a062 Implement some minor performance optimizations 2022-11-20 13:33:20 -05:00
Bongjun Jang
4e5626dfd5 Refactor lexer struct (#4257) 2022-11-19 21:43:34 +09:00
Jeong YunWon
f979d8dbc3 Apply let-else statements 2022-11-10 15:39:31 +09:00
Jim Fasarakis-Hilliard
cc084b4fec Merge pull request #4266 from charliermarsh/charlie/comments
Implement Tok::Comment
2022-11-07 23:20:15 +02:00
Charlie Marsh
bbeec36fdb Set comparator start location to beginning of comparison 2022-11-07 12:24:14 -05:00
Charlie Marsh
b6c230f3ca Implement Tok::Comment 2022-11-07 10:33:55 -05:00
dvermd
a5b59f3c9d improve col_offset in new line and lalr 2022-10-26 21:30:51 +02:00
dvermd
d5a208ca9d improve fstring parser
part of: #1671
2022-10-26 21:15:24 +02:00
dvermd
fa41a1e2f6 Fix ast types' _fields and use 0-based column 2022-10-27 03:09:38 +09:00
Charlie Marsh
952d70b9d1 Add expression context parsing 2022-10-17 15:20:33 -04:00
Charlie Marsh
02953b9fe6 Remove parse_program_tokens 2022-10-17 12:04:30 -04:00
Charlie Marsh
8adc74fe26 Expose a method to parse AST from tokens directly 2022-10-17 09:39:48 -04:00
Jeong YunWon
48c0cb5599 Merge pull request #4218 from charliermarsh/charlie/clone
Make AST nodes Clone-able
2022-10-17 13:47:21 +09:00
Jeong YunWon
2e33a3d0e9 Merge pull request #4223 from youknowone/nightly-clippy
Fix nightly clippy warnings
2022-10-17 13:28:53 +09:00
Charlie Marsh
1cc342e4ed Add end locations to all nodes (#4192) 2022-10-17 13:18:30 +09:00
Charlie Marsh
519718e65d Start simple string at quote mark 2022-10-16 11:25:46 -04:00
Charlie Marsh
5da5490b19 Make AST nodes Clone-able 2022-10-16 11:01:17 -04:00
Jeong YunWon
518cf728c3 Fix nightly clippy warnings 2022-10-16 02:38:50 +09:00
Charlie Marsh
3397737a76 Start string location at kind or quote prefix 2022-10-15 11:03:50 -04:00
dvermd
6211a3a3a8 Refactor fstrings (#4188) 2022-10-14 12:16:34 +09:00
fanninpm
e64d7d196d Merge pull request #4186 from andersk/arithmetic
Spell “arithmetic” correctly
2022-09-30 09:29:36 -04:00
Anders Kaseorg
f8b45e48e1 Spell “arithmetic” correctly
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-09-28 17:58:30 -07:00
Jeong YunWon
f99e4789ed temporary fix of parser build 2022-09-21 16:49:35 +09:00
Jeong YunWon
86b847204e better parser build experience 2022-09-21 16:47:06 +09:00
Charlie Marsh
26b529f9dc Add PartialOrd to Location 2022-09-04 16:31:43 -04:00
Jeong YunWon
b21ed24025 Merge pull request #4116 from charliermarsh/charlie/f-string
Avoid creating unused JoinedStr in FStringParser
2022-08-23 19:30:36 +09:00
Jeong YunWon
a414677892 remove Mode from codegen root 2022-08-23 05:15:27 +09:00
Jeong YunWon
bfe4795b6c parser::Mode from compile::Mode 2022-08-23 05:08:24 +09:00
Jeong YunWon
40690b9761 use thiserror 2022-08-23 05:08:24 +09:00
Jeong YunWon
1fd898c14c new_syntax_error working without full compiler
Fix #4100
2022-08-23 05:08:24 +09:00
Jeong YunWon
42b95a9a95 Move out CompileError to core as generic form 2022-08-23 01:30:00 +09:00
Jeong YunWon
e8230efe1a Integrate CompileError to compiler-core::BaseError 2022-08-22 23:18:41 +09:00
Jeong YunWon
7fcc18daea integrate CodegenError to compiler-core::Error 2022-08-22 18:43:03 +09:00
Jeong YunWon
2b7bf79d29 Integrate ParseError to compiler-core::Error 2022-08-22 16:28:08 +09:00
Jeong YunWon
904fc477f1 integrate ast::Location into compilre-core::Location 2022-08-22 08:42:20 +09:00
Jeong YunWon
bfac0355dc Share location between compiler crates 2022-08-22 08:42:20 +09:00
Jeong YunWon
c16e650071 rustpython-bytecode -> rustpython-compiler-core 2022-08-22 08:42:20 +09:00
Jeong YunWon
acde8bb625 sort Cargo.toml dependencies 2022-08-22 08:42:20 +09:00
Jeong YunWon
7f99404618 clean up cargo.toml sort 2022-08-22 08:42:20 +09:00
Jeong YunWon
3742f9117b Add source_path to ParseError 2022-08-22 08:42:20 +09:00
Jeong YunWon
a66902406f Refactor Mode and partial parser/codegen for eval/exec 2022-08-22 08:42:20 +09:00
Charles Marsh
51b6571ee1 Fix f-string regression 2022-08-21 19:20:23 -04:00
Charles Marsh
2345bc895d Avoid creating unused JoinedStr in FStringParser 2022-08-21 16:59:36 -04:00
Jeong YunWon
53c48bf6b9 reorganize compiler crates 2022-08-22 04:42:42 +09:00
Jeong YunWon
ffacac05bb Clean up imports 2022-08-22 03:42:29 +09:00
Jeong YunWon
3bae0823f7 replace try_parse!() macro to closure 2022-08-12 08:35:34 +09:00
Jeong Yunwon
29644a30d7 Add compile::Mode::BlockExpr 2022-05-27 10:14:13 +09:00
Jeong Yunwon
07712eda58 Upgrade libraries 2022-04-18 00:09:22 +09:00
Noa
76728bb69d Run cargo upgrade 2021-10-30 19:42:35 -05:00
Noa
721c2709c8 Migrate to 2021 edition 2021-10-21 11:46:24 -05:00
Noa
7865cace06 Add unparse feature to ast 2021-10-17 21:55:27 -05:00
Noah
a9330765fa Add an ast optimizer 2021-04-05 09:46:30 -05:00
Noah
9941eef853 Upgrade dependencies 2021-03-25 08:06:56 -05:00
Noah
e494175cd2 Update the compiler to use the new asdl ast types 2021-01-14 12:37:37 -06:00
Noah
dd7560c494 Implement asdl in the ast and parser 2021-01-14 12:37:37 -06:00
Noah
6603b4a124 Use ahash/phf in parser 2021-01-08 17:09:09 -06:00
Noah
dab7e20a33 Use ahash in the compiler 2021-01-08 17:00:23 -06:00
Noah
13f3ec5b2b Update other stuff to use the root of rustpython_bytecode 2020-12-19 14:59:07 -06:00
Noah
452cd308d5 Optimize the size of Instruction 2020-12-14 14:25:19 -06:00
Noah
3c469eaa26 compiler/porcelain wrapper 2020-11-07 15:43:23 -06:00
Noah
0c5bf8e470 Split the ast from the parser, remove compiler dep on parser 2020-11-07 15:43:23 -06:00
Noah
3f52fdd50a Use disassembly for snapshot testing 2020-10-19 23:55:56 -05:00
Noah
f0c0897f23 Update compiler to output codeobj.constants, use insta for snapshot testing 2020-10-19 23:55:55 -05:00
Jeong YunWon
a35d4dc22a clean up PyComplex 2020-08-09 21:46:16 +09:00
Noah
5fe1bd3900 Update itertools 2020-07-26 15:21:39 -05:00
Noah
3737af157f Release 0.1.2
rustpython@0.1.2
rustpython-bytecode@0.1.2
rustpython-compiler@0.1.2
rustpython-derive@0.1.2
rustpython-parser@0.1.2
rustpython-vm@0.1.2
rustpython_freeze@0.1.2
rustpython_wasm@0.1.2

Generated by cargo-workspaces
2020-06-23 18:47:08 -05:00
Chris West (Faux)
ed01cd63ee upgrade low-risk deps 2019-10-10 21:07:26 +01:00
Seo Sanghyeon
1dd57971a9 Optimize BuildMap bytecode emission 2019-10-03 00:03:08 +09:00
Noah
d1ae5a7448 Bump crate versions 2019-09-25 11:57:38 -05:00
coolreader18
e89d6febea Make peephole optimizer a stream processor 2019-08-03 22:02:29 -05:00
Windel Bouwman
4dcad05aa0 Extend symtable module. 2019-07-19 22:05:35 +02:00
Windel Bouwman
b0d270b881 Set all versions to 0.1.0 2019-07-07 13:04:12 +02:00
Windel Bouwman
e3ccef2504 Change authors to team name. 2019-07-01 21:14:07 +02:00
Windel Bouwman
dcdd964e0c Change underscore into hyphen 2019-07-01 21:05:29 +02:00
Windel Bouwman
107442cc06 Make bytecode crate independent of parser crate. 2019-06-30 11:42:36 +02:00
Windel Bouwman
10957879db Move bytecode into own crate. 2019-06-30 11:01:40 +02:00
coolreader18
f0148f46d0 Split off bytecode compilation into a separate crate 2019-06-12 21:43:43 -05:00
6933 changed files with 259773 additions and 459990 deletions

View File

@@ -1,3 +1,37 @@
[alias]
dev = "run --package ruff_dev --bin ruff_dev"
benchmark = "bench -p ruff_benchmark --bench linter --bench formatter --"
[target.'cfg(all())']
rustflags = [
# CLIPPY LINT SETTINGS
# This is a workaround to configure lints for the entire workspace, pending the ability to configure this via TOML.
# See: `https://github.com/rust-lang/cargo/issues/5034`
# `https://github.com/EmbarkStudios/rust-ecosystem/issues/22#issuecomment-947011395`
"-Dunsafe_code",
"-Wclippy::pedantic",
# Allowed pedantic lints
"-Wclippy::char_lit_as_u8",
"-Aclippy::collapsible_else_if",
"-Aclippy::collapsible_if",
"-Aclippy::implicit_hasher",
"-Aclippy::match_same_arms",
"-Aclippy::missing_errors_doc",
"-Aclippy::missing_panics_doc",
"-Aclippy::module_name_repetitions",
"-Aclippy::must_use_candidate",
"-Aclippy::similar_names",
"-Aclippy::too_many_lines",
# Disallowed restriction lints
"-Wclippy::print_stdout",
"-Wclippy::print_stderr",
"-Wclippy::dbg_macro",
"-Wclippy::empty_drop",
"-Wclippy::empty_structs_with_brackets",
"-Wclippy::exit",
"-Wclippy::get_unwrap",
"-Wclippy::rc_buffer",
"-Wclippy::rc_mutex",
"-Wclippy::rest_pat_in_fully_bound_structs",
"-Wunreachable_pub"
]

View File

@@ -1,8 +0,0 @@
[profile.ci]
# Print out output for failing tests as soon as they fail, and also at the end
# of the run (for easy scrollability).
failure-output = "immediate-final"
# Do not cancel the test run on the first failure.
fail-fast = false
status-level = "skip"

View File

@@ -10,7 +10,7 @@ indent_style = space
insert_final_newline = true
indent_size = 2
[*.{rs,py,pyi}]
[*.{rs,py}]
indent_size = 4
[*.snap]

7
.gitattributes vendored
View File

@@ -1,10 +1,7 @@
* text=auto eol=lf
crates/ruff_linter/resources/test/fixtures/isort/line_ending_crlf.py text eol=crlf
crates/ruff_linter/resources/test/fixtures/pycodestyle/W605_1.py text eol=crlf
crates/ruff_python_formatter/resources/test/fixtures/ruff/docstring_code_examples_crlf.py text eol=crlf
crates/ruff_python_formatter/tests/snapshots/format@docstring_code_examples_crlf.py.snap text eol=crlf
crates/ruff/resources/test/fixtures/isort/line_ending_crlf.py text eol=crlf
crates/ruff/resources/test/fixtures/pycodestyle/W605_1.py text eol=crlf
ruff.schema.json linguist-generated=true text=auto eol=lf
*.md.snap linguist-language=Markdown

2
.github/CODEOWNERS vendored
View File

@@ -6,4 +6,4 @@
# - Order is important. The last matching pattern has the most precedence.
# Jupyter
/crates/ruff_linter/src/jupyter/ @dhruvmanila
/crates/ruff/src/jupyter/ @dhruvmanila

View File

@@ -4,18 +4,8 @@ updates:
directory: "/"
schedule:
interval: "weekly"
labels: ["internal"]
groups:
actions:
patterns:
- "*"
ignore:
# The latest versions of these are not compatible with our release workflow
- dependency-name: "actions/upload-artifact"
- dependency-name: "actions/download-artifact"
- package-ecosystem: "cargo"
directory: "/"
schedule:
interval: "weekly"
labels: ["internal"]
day: "monday"
time: "12:00"
timezone: "America/New_York"
commit-message:
prefix: "ci(deps)"

24
.github/release.yml vendored Normal file
View File

@@ -0,0 +1,24 @@
# https://docs.github.com/en/repositories/releasing-projects-on-github/automatically-generated-release-notes#configuring-automatically-generated-release-notes
changelog:
exclude:
labels:
- internal
- documentation
categories:
- title: Breaking Changes
labels:
- breaking
- title: Rules
labels:
- rule
- autofix
- title: Settings
labels:
- configuration
- cli
- title: Bug Fixes
labels:
- bug
- title: Other Changes
labels:
- "*"

136
.github/workflows/benchmark.yaml vendored Normal file
View File

@@ -0,0 +1,136 @@
name: Benchmark
on:
pull_request:
paths:
- 'Cargo.toml'
- 'Cargo.lock'
- 'rust-toolchain'
- 'crates/**'
- '!crates/ruff_dev'
- '!crates/ruff_shrinking'
workflow_dispatch:
concurrency:
group: ${{ github.workflow }}-${{ github.ref_name }}-${{ github.event.pull_request.number || github.sha }}
cancel-in-progress: true
jobs:
run-benchmark:
if: github.event_name == 'pull_request'
name: "Run | ${{ matrix.os }}"
strategy:
matrix:
os: [ ubuntu-latest, windows-latest ]
runs-on: ${{ matrix.os }}
steps:
- name: "PR - Checkout Branch"
uses: actions/checkout@v3
with:
ref: ${{ github.event.pull_request.head.sha }}
- name: "PR - Install Rust toolchain"
run: rustup show
- uses: Swatinem/rust-cache@v2
- name: "PR - Build benchmarks"
run: cargo bench -p ruff_benchmark --no-run
- name: "PR - Run benchmarks"
run: cargo benchmark --save-baseline=pr
- name: "Main - Checkout Branch"
uses: actions/checkout@v3
with:
clean: false
ref: main
- name: "Main - Install Rust toolchain"
run: rustup show
- name: "Main - Build benchmarks"
run: cargo bench -p ruff_benchmark --no-run
- name: "Main - Run benchmarks"
run: cargo benchmark --save-baseline=main
- name: "Upload benchmark results"
uses: actions/upload-artifact@v3
with:
name: benchmark-results-${{ matrix.os }}
path: ./target/criterion
# Cleanup
- name: Remove Criterion Artifact
uses: JesseTG/rm@v1.0.3
with:
path: ./target/criterion
benchmark-compare:
if: github.event_name == 'pull_request'
runs-on: ubuntu-latest
name: Compare
needs:
- run-benchmark
steps:
- name: "Install Rust toolchain"
run: rustup show
- name: "Install critcmp"
uses: taiki-e/install-action@v2
with:
tool: critcmp
- name: "Linux | Download PR benchmark results"
uses: actions/download-artifact@v3
with:
name: benchmark-results-ubuntu-latest
path: ./target/criterion
- name: "Linux | Compare benchmark results"
shell: bash
run: |
echo "### Benchmark" >> summary.md
echo "#### Linux" >> summary.md
echo "\`\`\`" >> summary.md
critcmp main pr >> summary.md
echo "\`\`\`" >> summary.md
echo "" >> summary.md
- name: "Linux | Cleanup benchmark results"
run: rm -rf ./target/criterion
- name: "Windows | Download PR benchmark results"
uses: actions/download-artifact@v3
with:
name: benchmark-results-windows-latest
path: ./target/criterion
- name: "Windows | Compare benchmark results"
shell: bash
run: |
echo "#### Windows" >> summary.md
echo "\`\`\`" >> summary.md
critcmp main pr >> summary.md
echo "\`\`\`" >> summary.md
echo "" >> summary.md
echo ${{ github.event.pull_request.number }} > pr-number
cat summary.md > $GITHUB_STEP_SUMMARY
- uses: actions/upload-artifact@v3
name: Upload PR Number
with:
name: pr-number
path: pr-number
- uses: actions/upload-artifact@v3
name: Upload Summary
with:
name: summary
path: summary.md

View File

@@ -2,7 +2,7 @@ name: CI
on:
push:
branches: [main]
branches: [ main ]
pull_request:
workflow_dispatch:
@@ -16,26 +16,21 @@ env:
CARGO_TERM_COLOR: always
RUSTUP_MAX_RETRIES: 10
PACKAGE_NAME: ruff
PYTHON_VERSION: "3.11"
PYTHON_VERSION: "3.11" # to build abi3 wheels
jobs:
determine_changes:
name: "Determine changes"
runs-on: ubuntu-latest
outputs:
# Flag that is raised when any code that affects linter is changed
linter: ${{ steps.changed.outputs.linter_any_changed }}
# Flag that is raised when any code that affects formatter is changed
formatter: ${{ steps.changed.outputs.formatter_any_changed }}
# Flag that is raised when any code is changed
# This is superset of the linter and formatter
code: ${{ steps.changed.outputs.code_any_changed }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v3
with:
fetch-depth: 0
- uses: tj-actions/changed-files@v42
- uses: tj-actions/changed-files@v37
id: changed
with:
files_yaml: |
@@ -47,9 +42,6 @@ jobs:
- "!crates/ruff_formatter/**"
- "!crates/ruff_dev/**"
- "!crates/ruff_shrinking/**"
- scripts/*
- python/**
- .github/workflows/ci.yaml
formatter:
- Cargo.toml
@@ -58,27 +50,13 @@ jobs:
- crates/ruff_formatter/**
- crates/ruff_python_trivia/**
- crates/ruff_python_ast/**
- crates/ruff_source_file/**
- crates/ruff_python_index/**
- crates/ruff_text_size/**
- crates/ruff_python_parser/**
- crates/ruff_dev/**
- scripts/*
- python/**
- .github/workflows/ci.yaml
code:
- "**/*"
- "!**/*.md"
- "!docs/**"
- "!assets/**"
cargo-fmt:
name: "cargo fmt"
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v3
- name: "Install Rust toolchain"
run: rustup component add rustfmt
- run: cargo fmt --all --check
@@ -86,90 +64,81 @@ jobs:
cargo-clippy:
name: "cargo clippy"
runs-on: ubuntu-latest
needs: determine_changes
if: ${{ needs.determine_changes.outputs.code == 'true' || github.ref == 'refs/heads/main' }}
timeout-minutes: 20
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v3
- name: "Install Rust toolchain"
run: |
rustup component add clippy
rustup target add wasm32-unknown-unknown
- uses: Swatinem/rust-cache@v2
- name: "Clippy"
run: cargo clippy --workspace --all-targets --all-features --locked -- -D warnings
run: cargo clippy --workspace --all-targets --all-features -- -D warnings
- name: "Clippy (wasm)"
run: cargo clippy -p ruff_wasm --target wasm32-unknown-unknown --all-features --locked -- -D warnings
run: cargo clippy -p ruff_wasm --target wasm32-unknown-unknown --all-features -- -D warnings
cargo-test-linux:
name: "cargo test (linux)"
runs-on: ubuntu-latest
needs: determine_changes
if: ${{ needs.determine_changes.outputs.code == 'true' || github.ref == 'refs/heads/main' }}
timeout-minutes: 20
cargo-test:
strategy:
matrix:
os: [ ubuntu-latest, windows-latest ]
runs-on: ${{ matrix.os }}
name: "cargo test | ${{ matrix.os }}"
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v3
- name: "Install Rust toolchain"
run: rustup show
- name: "Install mold"
uses: rui314/setup-mold@v1
- name: "Install cargo nextest"
uses: taiki-e/install-action@v2
with:
tool: cargo-nextest
- name: "Install cargo insta"
uses: taiki-e/install-action@v2
with:
tool: cargo-insta
- run: pip install black[d]==23.1.0
- uses: Swatinem/rust-cache@v2
- name: "Run tests"
- name: "Run tests (Ubuntu)"
if: ${{ matrix.os == 'ubuntu-latest' }}
run: cargo insta test --all --all-features --unreferenced reject
- name: "Run tests (Windows)"
if: ${{ matrix.os == 'windows-latest' }}
shell: bash
env:
NEXTEST_PROFILE: "ci"
run: cargo insta test --all-features --unreferenced reject --test-runner nextest
# We can't reject unreferenced snapshots on windows because flake8_executable can't run on windows
run: cargo insta test --all --all-features
- run: cargo test --package ruff_cli --test black_compatibility_test -- --ignored
# TODO: Skipped as it's currently broken. The resource were moved from the
# ruff_cli to ruff crate, but this test was not updated.
if: false
# Check for broken links in the documentation.
- run: cargo doc --all --no-deps
env:
# Setting RUSTDOCFLAGS because `cargo doc --check` isn't yet implemented (https://github.com/rust-lang/cargo/issues/10025).
RUSTDOCFLAGS: "-D warnings"
- uses: actions/upload-artifact@v3
if: ${{ matrix.os == 'ubuntu-latest' }}
with:
name: ruff
path: target/debug/ruff
cargo-test-windows:
name: "cargo test (windows)"
runs-on: windows-latest
needs: determine_changes
if: ${{ needs.determine_changes.outputs.code == 'true' || github.ref == 'refs/heads/main' }}
timeout-minutes: 20
cargo-fuzz:
runs-on: ubuntu-latest
name: "cargo fuzz"
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v3
- name: "Install Rust toolchain"
run: rustup show
- name: "Install cargo nextest"
- uses: Swatinem/rust-cache@v2
with:
workspaces: "fuzz -> target"
- name: "Install cargo-fuzz"
uses: taiki-e/install-action@v2
with:
tool: cargo-nextest
- uses: Swatinem/rust-cache@v2
- name: "Run tests"
shell: bash
run: |
cargo nextest run --all-features --profile ci
cargo test --all-features --doc
tool: cargo-fuzz@0.11
- run: cargo fuzz build -s none
cargo-test-wasm:
name: "cargo test (wasm)"
runs-on: ubuntu-latest
needs: determine_changes
if: ${{ needs.determine_changes.outputs.code == 'true' || github.ref == 'refs/heads/main' }}
timeout-minutes: 10
name: "cargo test (wasm)"
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v3
- name: "Install Rust toolchain"
run: rustup target add wasm32-unknown-unknown
- uses: actions/setup-node@v4
- uses: actions/setup-node@v3
with:
node-version: 18
cache: "npm"
@@ -181,33 +150,11 @@ jobs:
cd crates/ruff_wasm
wasm-pack test --node
cargo-fuzz:
name: "cargo fuzz"
runs-on: ubuntu-latest
needs: determine_changes
if: ${{ needs.determine_changes.outputs.code == 'true' || github.ref == 'refs/heads/main' }}
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
- name: "Install Rust toolchain"
run: rustup show
- uses: Swatinem/rust-cache@v2
with:
workspaces: "fuzz -> target"
- name: "Install cargo-fuzz"
uses: taiki-e/install-action@v2
with:
tool: cargo-fuzz@0.11.2
- run: cargo fuzz build -s none
scripts:
name: "test scripts"
runs-on: ubuntu-latest
needs: determine_changes
if: ${{ needs.determine_changes.outputs.code == 'true' || github.ref == 'refs/heads/main' }}
timeout-minutes: 5
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v3
- name: "Install Rust toolchain"
run: rustup component add rustfmt
- uses: Swatinem/rust-cache@v2
@@ -224,104 +171,38 @@ jobs:
name: "ecosystem"
runs-on: ubuntu-latest
needs:
- cargo-test-linux
- cargo-test
- determine_changes
# Only runs on pull requests, since that is the only we way we can find the base version for comparison.
# Ecosystem check needs linter and/or formatter changes.
if: github.event_name == 'pull_request' && ${{
needs.determine_changes.outputs.code == 'true'
}}
timeout-minutes: 20
if: github.event_name == 'pull_request' && needs.determine_changes.outputs.linter == 'true'
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
- uses: actions/download-artifact@v3
name: Download comparison Ruff binary
name: Download Ruff binary
id: ruff-target
with:
name: ruff
path: target/debug
- uses: dawidd6/action-download-artifact@v3
name: Download baseline Ruff binary
- uses: dawidd6/action-download-artifact@v2
name: Download base results
with:
name: ruff
branch: ${{ github.event.pull_request.base.ref }}
check_artifacts: true
- name: Install ruff-ecosystem
run: |
pip install ./python/ruff-ecosystem
- name: Run `ruff check` stable ecosystem check
if: ${{ needs.determine_changes.outputs.linter == 'true' }}
- name: Run ecosystem check
run: |
# Make executable, since artifact download doesn't preserve this
chmod +x ./ruff ${{ steps.ruff-target.outputs.download-path }}/ruff
chmod +x ruff ${{ steps.ruff-target.outputs.download-path }}/ruff
# Set pipefail to avoid hiding errors with tee
set -eo pipefail
scripts/check_ecosystem.py ruff ${{ steps.ruff-target.outputs.download-path }}/ruff | tee ecosystem-result
cat ecosystem-result > $GITHUB_STEP_SUMMARY
ruff-ecosystem check ./ruff ${{ steps.ruff-target.outputs.download-path }}/ruff --cache ./checkouts --output-format markdown | tee ecosystem-result-check-stable
cat ecosystem-result-check-stable > $GITHUB_STEP_SUMMARY
echo "### Linter (stable)" > ecosystem-result
cat ecosystem-result-check-stable >> ecosystem-result
echo "" >> ecosystem-result
- name: Run `ruff check` preview ecosystem check
if: ${{ needs.determine_changes.outputs.linter == 'true' }}
run: |
# Make executable, since artifact download doesn't preserve this
chmod +x ./ruff ${{ steps.ruff-target.outputs.download-path }}/ruff
# Set pipefail to avoid hiding errors with tee
set -eo pipefail
ruff-ecosystem check ./ruff ${{ steps.ruff-target.outputs.download-path }}/ruff --cache ./checkouts --output-format markdown --force-preview | tee ecosystem-result-check-preview
cat ecosystem-result-check-preview > $GITHUB_STEP_SUMMARY
echo "### Linter (preview)" >> ecosystem-result
cat ecosystem-result-check-preview >> ecosystem-result
echo "" >> ecosystem-result
- name: Run `ruff format` stable ecosystem check
if: ${{ needs.determine_changes.outputs.formatter == 'true' }}
run: |
# Make executable, since artifact download doesn't preserve this
chmod +x ./ruff ${{ steps.ruff-target.outputs.download-path }}/ruff
# Set pipefail to avoid hiding errors with tee
set -eo pipefail
ruff-ecosystem format ./ruff ${{ steps.ruff-target.outputs.download-path }}/ruff --cache ./checkouts --output-format markdown | tee ecosystem-result-format-stable
cat ecosystem-result-format-stable > $GITHUB_STEP_SUMMARY
echo "### Formatter (stable)" >> ecosystem-result
cat ecosystem-result-format-stable >> ecosystem-result
echo "" >> ecosystem-result
- name: Run `ruff format` preview ecosystem check
if: ${{ needs.determine_changes.outputs.formatter == 'true' }}
run: |
# Make executable, since artifact download doesn't preserve this
chmod +x ./ruff ${{ steps.ruff-target.outputs.download-path }}/ruff
# Set pipefail to avoid hiding errors with tee
set -eo pipefail
ruff-ecosystem format ./ruff ${{ steps.ruff-target.outputs.download-path }}/ruff --cache ./checkouts --output-format markdown --force-preview | tee ecosystem-result-format-preview
cat ecosystem-result-format-preview > $GITHUB_STEP_SUMMARY
echo "### Formatter (preview)" >> ecosystem-result
cat ecosystem-result-format-preview >> ecosystem-result
echo "" >> ecosystem-result
- name: Export pull request number
run: |
echo ${{ github.event.number }} > pr-number
- uses: actions/upload-artifact@v3
@@ -339,27 +220,24 @@ jobs:
cargo-udeps:
name: "cargo udeps"
runs-on: ubuntu-latest
needs: determine_changes
if: ${{ needs.determine_changes.outputs.code == 'true' || github.ref == 'refs/heads/main' }}
timeout-minutes: 20
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v3
- name: "Install nightly Rust toolchain"
# Only pinned to make caching work, update freely
run: rustup toolchain install nightly-2023-10-15
run: rustup toolchain install nightly-2023-06-08
- uses: Swatinem/rust-cache@v2
- name: "Install cargo-udeps"
uses: taiki-e/install-action@cargo-udeps
- name: "Run cargo-udeps"
run: cargo +nightly-2023-10-15 udeps
run: cargo +nightly-2023-06-08 udeps
python-package:
name: "python package"
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
architecture: x64
@@ -381,10 +259,9 @@ jobs:
pre-commit:
name: "pre-commit"
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: "Install Rust toolchain"
@@ -393,7 +270,7 @@ jobs:
- name: "Install pre-commit"
run: pip install pre-commit
- name: "Cache pre-commit"
uses: actions/cache@v4
uses: actions/cache@v3
with:
path: ~/.cache/pre-commit
key: pre-commit-${{ hashFiles('.pre-commit-config.yaml') }}
@@ -410,15 +287,14 @@ jobs:
docs:
name: "mkdocs"
runs-on: ubuntu-latest
timeout-minutes: 10
env:
MKDOCS_INSIDERS_SSH_KEY_EXISTS: ${{ secrets.MKDOCS_INSIDERS_SSH_KEY != '' }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
- name: "Add SSH key"
if: ${{ env.MKDOCS_INSIDERS_SSH_KEY_EXISTS == 'true' }}
uses: webfactory/ssh-agent@v0.9.0
uses: webfactory/ssh-agent@v0.8.0
with:
ssh-private-key: ${{ secrets.MKDOCS_INSIDERS_SSH_KEY }}
- name: "Install Rust toolchain"
@@ -441,94 +317,25 @@ jobs:
run: mkdocs build --strict -f mkdocs.insiders.yml
- name: "Build docs"
if: ${{ env.MKDOCS_INSIDERS_SSH_KEY_EXISTS != 'true' }}
run: mkdocs build --strict -f mkdocs.public.yml
run: mkdocs build --strict -f mkdocs.generated.yml
check-formatter-instability-and-black-similarity:
name: "formatter instabilities and black similarity"
check-formatter-ecosystem:
name: "Formatter ecosystem and progress checks"
runs-on: ubuntu-latest
needs: determine_changes
if: needs.determine_changes.outputs.formatter == 'true' || github.ref == 'refs/heads/main'
timeout-minutes: 10
if: needs.determine_changes.outputs.formatter == 'true'
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v3
- name: "Install Rust toolchain"
run: rustup show
- name: "Cache rust"
uses: Swatinem/rust-cache@v2
- name: "Formatter progress"
run: scripts/formatter_ecosystem_checks.sh
run: scripts/formatter_progress.sh
- name: "Github step summary"
run: cat target/progress_projects_stats.txt > $GITHUB_STEP_SUMMARY
- name: "Remove checkouts from cache"
run: rm -r target/progress_projects
check-ruff-lsp:
name: "test ruff-lsp"
runs-on: ubuntu-latest
timeout-minutes: 5
needs:
- cargo-test-linux
- determine_changes
if: ${{ needs.determine_changes.outputs.code == 'true' || github.ref == 'refs/heads/main' }}
steps:
- uses: extractions/setup-just@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- uses: actions/checkout@v4
name: "Download ruff-lsp source"
with:
repository: "astral-sh/ruff-lsp"
- uses: actions/setup-python@v5
with:
python-version: ${{ env.PYTHON_VERSION }}
- uses: actions/download-artifact@v3
name: Download development ruff binary
id: ruff-target
with:
name: ruff
path: target/debug
- name: Install ruff-lsp dependencies
run: |
just install
- name: Run ruff-lsp tests
run: |
# Setup development binary
pip uninstall --yes ruff
chmod +x ${{ steps.ruff-target.outputs.download-path }}/ruff
export PATH=${{ steps.ruff-target.outputs.download-path }}:$PATH
ruff version
just test
benchmarks:
runs-on: ubuntu-latest
needs: determine_changes
if: ${{ needs.determine_changes.outputs.code == 'true' || github.ref == 'refs/heads/main' }}
timeout-minutes: 20
steps:
- name: "Checkout Branch"
uses: actions/checkout@v4
- name: "Install Rust toolchain"
run: rustup show
- name: "Install codspeed"
uses: taiki-e/install-action@v2
with:
tool: cargo-codspeed
- uses: Swatinem/rust-cache@v2
- name: "Build benchmarks"
run: cargo codspeed build --features codspeed -p ruff_benchmark
- name: "Run benchmarks"
uses: CodSpeedHQ/action@v2
with:
run: cargo codspeed run
token: ${{ secrets.CODSPEED_TOKEN }}
run: grep "similarity index" target/progress_projects_report.txt | sort > $GITHUB_STEP_SUMMARY
# CPython is not black formatted, so we run only the stability check
- name: "Clone CPython 3.10"
run: git clone --branch 3.10 --depth 1 https://github.com/python/cpython.git crates/ruff/resources/test/cpython
- name: "Check CPython stability"
run: cargo run --bin ruff_dev -- format-dev --stability-check crates/ruff/resources/test/cpython

View File

@@ -2,13 +2,8 @@ name: mkdocs
on:
workflow_dispatch:
inputs:
ref:
description: "The commit SHA, tag, or branch to publish. Uses the default branch if not specified."
default: ""
type: string
release:
types: [published]
types: [ published ]
jobs:
mkdocs:
@@ -17,13 +12,11 @@ jobs:
CF_API_TOKEN_EXISTS: ${{ secrets.CF_API_TOKEN != '' }}
MKDOCS_INSIDERS_SSH_KEY_EXISTS: ${{ secrets.MKDOCS_INSIDERS_SSH_KEY != '' }}
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.ref }}
- uses: actions/setup-python@v5
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
- name: "Add SSH key"
if: ${{ env.MKDOCS_INSIDERS_SSH_KEY_EXISTS == 'true' }}
uses: webfactory/ssh-agent@v0.9.0
uses: webfactory/ssh-agent@v0.8.0
with:
ssh-private-key: ${{ secrets.MKDOCS_INSIDERS_SSH_KEY }}
- name: "Install Rust toolchain"
@@ -44,12 +37,11 @@ jobs:
run: mkdocs build --strict -f mkdocs.insiders.yml
- name: "Build docs"
if: ${{ env.MKDOCS_INSIDERS_SSH_KEY_EXISTS != 'true' }}
run: mkdocs build --strict -f mkdocs.public.yml
run: mkdocs build --strict -f mkdocs.generated.yml
- name: "Deploy to Cloudflare Pages"
if: ${{ env.CF_API_TOKEN_EXISTS == 'true' }}
uses: cloudflare/wrangler-action@v3.4.1
uses: cloudflare/wrangler-action@2.0.0
with:
apiToken: ${{ secrets.CF_API_TOKEN }}
accountId: ${{ secrets.CF_ACCOUNT_ID }}
# `github.head_ref` is only set during pull requests and for manual runs or tags we use `main` to deploy to production
command: pages deploy site --project-name=astral-docs --branch ${{ github.head_ref || 'main' }} --commit-hash ${GITHUB_SHA}
command: pages publish site --project-name=ruff-docs --branch ${GITHUB_HEAD_REF} --commit-hash ${GITHUB_SHA}

247
.github/workflows/flake8-to-ruff.yaml vendored Normal file
View File

@@ -0,0 +1,247 @@
name: "[flake8-to-ruff] Release"
on: workflow_dispatch
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
env:
PACKAGE_NAME: flake8-to-ruff
CRATE_NAME: flake8_to_ruff
PYTHON_VERSION: "3.11"
CARGO_INCREMENTAL: 0
CARGO_NET_RETRY: 10
CARGO_TERM_COLOR: always
RUSTUP_MAX_RETRIES: 10
jobs:
macos-x86_64:
runs-on: macos-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
architecture: x64
- name: "Install Rust toolchain"
run: rustup show
- name: "Build wheels - x86_64"
uses: PyO3/maturin-action@v1
with:
target: x86_64
args: --release --out dist --sdist -m ./${{ env.CRATE_NAME }}/Cargo.toml
- name: "Install built wheel - x86_64"
run: |
pip install dist/${{ env.CRATE_NAME }}-*.whl --force-reinstall
- name: "Upload wheels"
uses: actions/upload-artifact@v3
with:
name: wheels
path: dist
macos-universal:
runs-on: macos-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
architecture: x64
- name: "Install Rust toolchain"
run: rustup show
- name: "Build wheels - universal2"
uses: PyO3/maturin-action@v1
with:
args: --release --target universal2-apple-darwin --out dist -m ./${{ env.CRATE_NAME }}/Cargo.toml
- name: "Install built wheel - universal2"
run: |
pip install dist/${{ env.CRATE_NAME }}-*universal2.whl --force-reinstall
- name: "Upload wheels"
uses: actions/upload-artifact@v3
with:
name: wheels
path: dist
windows:
runs-on: windows-latest
strategy:
matrix:
target: [ x64, x86 ]
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
architecture: ${{ matrix.target }}
- name: "Install Rust toolchain"
run: rustup show
- name: "Build wheels"
uses: PyO3/maturin-action@v1
with:
target: ${{ matrix.target }}
args: --release --out dist -m ./${{ env.CRATE_NAME }}/Cargo.toml
- name: "Install built wheel"
shell: bash
run: |
python -m pip install dist/${{ env.CRATE_NAME }}-*.whl --force-reinstall
- name: "Upload wheels"
uses: actions/upload-artifact@v3
with:
name: wheels
path: dist
linux:
runs-on: ubuntu-latest
strategy:
matrix:
target: [ x86_64, i686 ]
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
architecture: x64
- name: "Build wheels"
uses: PyO3/maturin-action@v1
with:
target: ${{ matrix.target }}
manylinux: auto
args: --release --out dist -m ./${{ env.CRATE_NAME }}/Cargo.toml
- name: "Install built wheel"
if: matrix.target == 'x86_64'
run: |
pip install dist/${{ env.CRATE_NAME }}-*.whl --force-reinstall
- name: "Upload wheels"
uses: actions/upload-artifact@v3
with:
name: wheels
path: dist
linux-cross:
runs-on: ubuntu-latest
strategy:
matrix:
target: [ aarch64, armv7, s390x, ppc64le, ppc64 ]
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: "Build wheels"
uses: PyO3/maturin-action@v1
with:
target: ${{ matrix.target }}
manylinux: auto
args: --no-default-features --release --out dist -m ./${{ env.CRATE_NAME }}/Cargo.toml
- uses: uraimo/run-on-arch-action@v2
if: matrix.target != 'ppc64'
name: Install built wheel
with:
arch: ${{ matrix.target }}
distro: ubuntu20.04
githubToken: ${{ github.token }}
install: |
apt-get update
apt-get install -y --no-install-recommends python3 python3-pip
pip3 install -U pip
run: |
pip3 install ${{ env.PACKAGE_NAME }} --no-index --find-links dist/ --force-reinstall
- name: "Upload wheels"
uses: actions/upload-artifact@v3
with:
name: wheels
path: dist
musllinux:
runs-on: ubuntu-latest
strategy:
matrix:
target:
- x86_64-unknown-linux-musl
- i686-unknown-linux-musl
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
architecture: x64
- name: "Build wheels"
uses: PyO3/maturin-action@v1
with:
target: ${{ matrix.target }}
manylinux: musllinux_1_2
args: --release --out dist -m ./${{ env.CRATE_NAME }}/Cargo.toml
- name: "Install built wheel"
if: matrix.target == 'x86_64-unknown-linux-musl'
uses: addnab/docker-run-action@v3
with:
image: alpine:latest
options: -v ${{ github.workspace }}:/io -w /io
run: |
apk add py3-pip
pip3 install ${{ env.PACKAGE_NAME }} --no-index --find-links /io/dist/ --force-reinstall
- name: "Upload wheels"
uses: actions/upload-artifact@v3
with:
name: wheels
path: dist
musllinux-cross:
runs-on: ubuntu-latest
strategy:
matrix:
platform:
- target: aarch64-unknown-linux-musl
arch: aarch64
- target: armv7-unknown-linux-musleabihf
arch: armv7
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: "Build wheels"
uses: PyO3/maturin-action@v1
with:
target: ${{ matrix.platform.target }}
manylinux: musllinux_1_2
args: --release --out dist -m ./${{ env.CRATE_NAME }}/Cargo.toml
- uses: uraimo/run-on-arch-action@v2
name: Install built wheel
with:
arch: ${{ matrix.platform.arch }}
distro: alpine_latest
githubToken: ${{ github.token }}
install: |
apk add py3-pip
run: |
pip3 install ${{ env.PACKAGE_NAME }} --no-index --find-links dist/ --force-reinstall
- name: "Upload wheels"
uses: actions/upload-artifact@v3
with:
name: wheels
path: dist
release:
name: Release
runs-on: ubuntu-latest
needs:
- macos-universal
- macos-x86_64
- windows
- linux
- linux-cross
- musllinux
- musllinux-cross
steps:
- uses: actions/download-artifact@v3
with:
name: wheels
- uses: actions/setup-python@v4
- name: "Publish to PyPi"
env:
TWINE_USERNAME: __token__
TWINE_PASSWORD: ${{ secrets.FLAKE8_TO_RUFF_TOKEN }}
run: |
pip install --upgrade twine
twine upload --skip-existing *

View File

@@ -3,7 +3,7 @@ name: "[Playground] Release"
on:
workflow_dispatch:
release:
types: [published]
types: [ published ]
env:
CARGO_INCREMENTAL: 0
@@ -17,10 +17,10 @@ jobs:
env:
CF_API_TOKEN_EXISTS: ${{ secrets.CF_API_TOKEN != '' }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v3
- name: "Install Rust toolchain"
run: rustup target add wasm32-unknown-unknown
- uses: actions/setup-node@v4
- uses: actions/setup-node@v3
with:
node-version: 18
cache: "npm"
@@ -40,9 +40,8 @@ jobs:
working-directory: playground
- name: "Deploy to Cloudflare Pages"
if: ${{ env.CF_API_TOKEN_EXISTS == 'true' }}
uses: cloudflare/wrangler-action@v3.4.1
uses: cloudflare/wrangler-action@2.0.0
with:
apiToken: ${{ secrets.CF_API_TOKEN }}
accountId: ${{ secrets.CF_ACCOUNT_ID }}
# `github.head_ref` is only set during pull requests and for manual runs or tags we use `main` to deploy to production
command: pages deploy playground/dist --project-name=ruff-playground --branch ${{ github.head_ref || 'main' }} --commit-hash ${GITHUB_SHA}
command: pages publish playground/dist --project-name=ruff --branch ${GITHUB_HEAD_REF} --commit-hash ${GITHUB_SHA}

View File

@@ -1,9 +1,9 @@
name: Ecosystem check comment
name: PR Check Comment
on:
workflow_run:
workflows: [CI]
types: [completed]
workflows: [ CI, Benchmark ]
types: [ completed ]
workflow_dispatch:
inputs:
workflow_run_id:
@@ -17,14 +17,14 @@ jobs:
comment:
runs-on: ubuntu-latest
steps:
- uses: dawidd6/action-download-artifact@v3
name: Download pull request number
- uses: dawidd6/action-download-artifact@v2
name: Download PR Number
with:
name: pr-number
run_id: ${{ github.event.workflow_run.id || github.event.inputs.workflow_run_id }}
if_no_artifact_found: ignore
- name: Parse pull request number
- name: Extract PR Number
id: pr-number
run: |
if [[ -f pr-number ]]
@@ -32,8 +32,8 @@ jobs:
echo "pr-number=$(<pr-number)" >> $GITHUB_OUTPUT
fi
- uses: dawidd6/action-download-artifact@v3
name: "Download ecosystem results"
- uses: dawidd6/action-download-artifact@v2
name: "Download Ecosystem Result"
id: download-ecosystem-result
if: steps.pr-number.outputs.pr-number
with:
@@ -41,39 +41,44 @@ jobs:
workflow: ci.yaml
pr: ${{ steps.pr-number.outputs.pr-number }}
path: pr/ecosystem
workflow_conclusion: completed
if_no_artifact_found: ignore
- name: Generate comment content
- uses: dawidd6/action-download-artifact@v2
name: "Download Benchmark Result"
id: download-benchmark-result
if: steps.pr-number.outputs.pr-number
with:
name: summary
workflow: benchmark.yaml
pr: ${{ steps.pr-number.outputs.pr-number }}
path: pr/benchmark
if_no_artifact_found: ignore
- name: Generate Comment
id: generate-comment
if: steps.download-ecosystem-result.outputs.found_artifact == 'true'
if: steps.download-ecosystem-result.outputs.found_artifact == 'true' || steps.download-benchmark-result.outputs.found_artifact == 'true'
run: |
# Note this identifier is used to find the comment to update on
# subsequent runs
echo '<!-- generated-comment ecosystem -->' >> comment.txt
echo '## `ruff-ecosystem` results' >> comment.txt
cat pr/ecosystem/ecosystem-result >> comment.txt
echo "" >> comment.txt
echo 'comment<<EOF' >> $GITHUB_OUTPUT
cat comment.txt >> $GITHUB_OUTPUT
echo '## PR Check Results' >> $GITHUB_OUTPUT
if [[ -f pr/ecosystem/ecosystem-result ]]
then
echo "### Ecosystem" >> $GITHUB_OUTPUT
cat pr/ecosystem/ecosystem-result >> $GITHUB_OUTPUT
echo "" >> $GITHUB_OUTPUT
fi
if [[ -f pr/benchmark/summary.md ]]
then
cat pr/benchmark/summary.md >> $GITHUB_OUTPUT
fi
echo 'EOF' >> $GITHUB_OUTPUT
- name: Find existing comment
uses: peter-evans/find-comment@v3
if: steps.generate-comment.outcome == 'success'
id: find-comment
with:
issue-number: ${{ steps.pr-number.outputs.pr-number }}
comment-author: "github-actions[bot]"
body-includes: "<!-- generated-comment ecosystem -->"
- name: Create or update comment
if: steps.find-comment.outcome == 'success'
uses: peter-evans/create-or-update-comment@v4
if: steps.generate-comment.outputs.comment
uses: thollander/actions-comment-pull-request@v2
with:
comment-id: ${{ steps.find-comment.outputs.comment-id }}
issue-number: ${{ steps.pr-number.outputs.pr-number }}
body-path: comment.txt
edit-mode: replace
pr_number: ${{ steps.pr-number.outputs.pr-number }}
message: ${{ steps.generate-comment.outputs.comment }}
comment_tag: PR Check Results

View File

@@ -7,15 +7,12 @@ on:
description: "The version to tag, without the leading 'v'. If omitted, will initiate a dry run (no uploads)."
type: string
sha:
description: "The full sha of the commit to be released. If omitted, the latest commit on the default branch will be used."
default: ""
description: "Optionally, the full sha of the commit to be released"
type: string
pull_request:
paths:
# When we change pyproject.toml, we want to ensure that the maturin builds still work
- pyproject.toml
# And when we change this workflow itself...
- .github/workflows/release.yaml
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
@@ -33,10 +30,8 @@ jobs:
sdist:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.sha }}
- uses: actions/setup-python@v5
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: "Prep README.md"
@@ -48,6 +43,7 @@ jobs:
args: --out dist
- name: "Test sdist"
run: |
rustup default $(cat rust-toolchain)
pip install dist/${{ env.PACKAGE_NAME }}-*.tar.gz --force-reinstall
ruff --help
python -m ruff --help
@@ -60,10 +56,8 @@ jobs:
macos-x86_64:
runs-on: macos-latest
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.sha }}
- uses: actions/setup-python@v5
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
architecture: x64
@@ -73,7 +67,7 @@ jobs:
uses: PyO3/maturin-action@v1
with:
target: x86_64
args: --release --locked --out dist
args: --release --out dist
- name: "Test wheel - x86_64"
run: |
pip install dist/${{ env.PACKAGE_NAME }}-*.whl --force-reinstall
@@ -86,7 +80,7 @@ jobs:
path: dist
- name: "Archive binary"
run: |
ARCHIVE_FILE=ruff-${{ inputs.tag }}-x86_64-apple-darwin.tar.gz
ARCHIVE_FILE=ruff-x86_64-apple-darwin.tar.gz
tar czvf $ARCHIVE_FILE -C target/x86_64-apple-darwin/release ruff
shasum -a 256 $ARCHIVE_FILE > $ARCHIVE_FILE.sha256
- name: "Upload binary"
@@ -100,10 +94,8 @@ jobs:
macos-universal:
runs-on: macos-latest
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.sha }}
- uses: actions/setup-python@v5
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
architecture: x64
@@ -112,7 +104,7 @@ jobs:
- name: "Build wheels - universal2"
uses: PyO3/maturin-action@v1
with:
args: --release --locked --target universal2-apple-darwin --out dist
args: --release --target universal2-apple-darwin --out dist
- name: "Test wheel - universal2"
run: |
pip install dist/${{ env.PACKAGE_NAME }}-*universal2.whl --force-reinstall
@@ -125,7 +117,7 @@ jobs:
path: dist
- name: "Archive binary"
run: |
ARCHIVE_FILE=ruff-${{ inputs.tag }}-aarch64-apple-darwin.tar.gz
ARCHIVE_FILE=ruff-aarch64-apple-darwin.tar.gz
tar czvf $ARCHIVE_FILE -C target/aarch64-apple-darwin/release ruff
shasum -a 256 $ARCHIVE_FILE > $ARCHIVE_FILE.sha256
- name: "Upload binary"
@@ -148,10 +140,8 @@ jobs:
- target: aarch64-pc-windows-msvc
arch: x64
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.sha }}
- uses: actions/setup-python@v5
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
architecture: ${{ matrix.platform.arch }}
@@ -161,7 +151,7 @@ jobs:
uses: PyO3/maturin-action@v1
with:
target: ${{ matrix.platform.target }}
args: --release --locked --out dist
args: --release --out dist
- name: "Test wheel"
if: ${{ !startsWith(matrix.platform.target, 'aarch64') }}
shell: bash
@@ -177,7 +167,7 @@ jobs:
- name: "Archive binary"
shell: bash
run: |
ARCHIVE_FILE=ruff-${{ inputs.tag }}-${{ matrix.platform.target }}.zip
ARCHIVE_FILE=ruff-${{ matrix.platform.target }}.zip
7z a $ARCHIVE_FILE ./target/${{ matrix.platform.target }}/release/ruff.exe
sha256sum $ARCHIVE_FILE > $ARCHIVE_FILE.sha256
- name: "Upload binary"
@@ -196,10 +186,8 @@ jobs:
- x86_64-unknown-linux-gnu
- i686-unknown-linux-gnu
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.sha }}
- uses: actions/setup-python@v5
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
architecture: x64
@@ -210,7 +198,7 @@ jobs:
with:
target: ${{ matrix.target }}
manylinux: auto
args: --release --locked --out dist
args: --release --out dist
- name: "Test wheel"
if: ${{ startsWith(matrix.target, 'x86_64') }}
run: |
@@ -224,7 +212,7 @@ jobs:
path: dist
- name: "Archive binary"
run: |
ARCHIVE_FILE=ruff-${{ inputs.tag }}-${{ matrix.target }}.tar.gz
ARCHIVE_FILE=ruff-${{ matrix.target }}.tar.gz
tar czvf $ARCHIVE_FILE -C target/${{ matrix.target }}/release ruff
shasum -a 256 $ARCHIVE_FILE > $ARCHIVE_FILE.sha256
- name: "Upload binary"
@@ -255,10 +243,8 @@ jobs:
arch: ppc64
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.sha }}
- uses: actions/setup-python@v5
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: "Prep README.md"
@@ -269,7 +255,7 @@ jobs:
target: ${{ matrix.platform.target }}
manylinux: auto
docker-options: ${{ matrix.platform.maturin_docker_options }}
args: --release --locked --out dist
args: --release --out dist
- uses: uraimo/run-on-arch-action@v2
if: matrix.platform.arch != 'ppc64'
name: Test wheel
@@ -291,7 +277,7 @@ jobs:
path: dist
- name: "Archive binary"
run: |
ARCHIVE_FILE=ruff-${{ inputs.tag }}-${{ matrix.platform.target }}.tar.gz
ARCHIVE_FILE=ruff-${{ matrix.platform.target }}.tar.gz
tar czvf $ARCHIVE_FILE -C target/${{ matrix.platform.target }}/release ruff
shasum -a 256 $ARCHIVE_FILE > $ARCHIVE_FILE.sha256
- name: "Upload binary"
@@ -310,10 +296,8 @@ jobs:
- x86_64-unknown-linux-musl
- i686-unknown-linux-musl
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.sha }}
- uses: actions/setup-python@v5
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
architecture: x64
@@ -324,7 +308,7 @@ jobs:
with:
target: ${{ matrix.target }}
manylinux: musllinux_1_2
args: --release --locked --out dist
args: --release --out dist
- name: "Test wheel"
if: matrix.target == 'x86_64-unknown-linux-musl'
uses: addnab/docker-run-action@v3
@@ -332,10 +316,10 @@ jobs:
image: alpine:latest
options: -v ${{ github.workspace }}:/io -w /io
run: |
apk add python3
python -m venv .venv
.venv/bin/pip3 install ${{ env.PACKAGE_NAME }} --no-index --find-links dist/ --force-reinstall
.venv/bin/ruff check --help
apk add py3-pip
pip3 install ${{ env.PACKAGE_NAME }} --no-index --find-links /io/dist/ --force-reinstall
ruff --help
python -m ruff --help
- name: "Upload wheels"
uses: actions/upload-artifact@v3
with:
@@ -343,7 +327,7 @@ jobs:
path: dist
- name: "Archive binary"
run: |
ARCHIVE_FILE=ruff-${{ inputs.tag }}-${{ matrix.target }}.tar.gz
ARCHIVE_FILE=ruff-${{ matrix.target }}.tar.gz
tar czvf $ARCHIVE_FILE -C target/${{ matrix.target }}/release ruff
shasum -a 256 $ARCHIVE_FILE > $ARCHIVE_FILE.sha256
- name: "Upload binary"
@@ -366,10 +350,8 @@ jobs:
arch: armv7
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.sha }}
- uses: actions/setup-python@v5
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: "Prep README.md"
@@ -379,7 +361,7 @@ jobs:
with:
target: ${{ matrix.platform.target }}
manylinux: musllinux_1_2
args: --release --locked --out dist
args: --release --out dist
docker-options: ${{ matrix.platform.maturin_docker_options }}
- uses: uraimo/run-on-arch-action@v2
name: Test wheel
@@ -388,11 +370,10 @@ jobs:
distro: alpine_latest
githubToken: ${{ github.token }}
install: |
apk add python3
apk add py3-pip
run: |
python -m venv .venv
.venv/bin/pip3 install ${{ env.PACKAGE_NAME }} --no-index --find-links dist/ --force-reinstall
.venv/bin/ruff check --help
pip3 install ${{ env.PACKAGE_NAME }} --no-index --find-links dist/ --force-reinstall
ruff check --help
- name: "Upload wheels"
uses: actions/upload-artifact@v3
with:
@@ -400,7 +381,7 @@ jobs:
path: dist
- name: "Archive binary"
run: |
ARCHIVE_FILE=ruff-${{ inputs.tag }}-${{ matrix.platform.target }}.tar.gz
ARCHIVE_FILE=ruff-${{ matrix.platform.target }}.tar.gz
tar czvf $ARCHIVE_FILE -C target/${{ matrix.platform.target }}/release ruff
shasum -a 256 $ARCHIVE_FILE > $ARCHIVE_FILE.sha256
- name: "Upload binary"
@@ -417,22 +398,9 @@ jobs:
# If you don't set an input tag, it's a dry run (no uploads).
if: ${{ inputs.tag }}
steps:
- uses: actions/checkout@v4
with:
ref: main # We checkout the main branch to check for the commit
- name: Check main branch
if: ${{ inputs.sha }}
run: |
# Fetch the main branch since a shallow checkout is used by default
git fetch origin main --unshallow
if ! git branch --contains ${{ inputs.sha }} | grep -E '(^|\s)main$'; then
echo "The specified sha is not on the main branch" >&2
exit 1
fi
- uses: actions/checkout@v3
- name: Check tag consistency
run: |
# Switch to the commit we want to release
git checkout ${{ inputs.sha }}
version=$(grep "version = " pyproject.toml | sed -e 's/version = "\(.*\)"/\1/g')
if [ "${{ inputs.tag }}" != "${version}" ]; then
echo "The input tag does not match the version from pyproject.toml:" >&2
@@ -442,6 +410,18 @@ jobs:
else
echo "Releasing ${version}"
fi
- name: Check SHA consistency
if: ${{ inputs.sha }}
run: |
git_sha=$(git rev-parse HEAD)
if [ "${{ inputs.sha }}" != "${git_sha}" ]; then
echo "The specified sha does not match the git checkout" >&2
echo "${{ inputs.sha }}" >&2
echo "${git_sha}" >&2
exit 1
else
echo "Releasing ${git_sha}"
fi
upload-release:
name: Upload to PyPI
@@ -484,9 +464,7 @@ jobs:
# For git tag
contents: write
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.sha }}
- uses: actions/checkout@v3
- name: git tag
run: |
git config user.email "hey@astral.sh"
@@ -517,62 +495,6 @@ jobs:
files: binaries/*
tag_name: v${{ inputs.tag }}
docker-publish:
# This action doesn't need to wait on any other task, it's easy to re-tag if something failed and we're validating
# the tag here also
name: Push Docker image ghcr.io/astral-sh/ruff
runs-on: ubuntu-latest
environment:
name: release
permissions:
# For the docker push
packages: write
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.sha }}
- uses: docker/setup-buildx-action@v3
- uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v5
with:
images: ghcr.io/astral-sh/ruff
- name: Check tag consistency
# Unlike validate-tag we don't check if the commit is on the main branch, but it seems good enough since we can
# change docker tags
if: ${{ inputs.tag }}
run: |
version=$(grep "version = " pyproject.toml | sed -e 's/version = "\(.*\)"/\1/g')
if [ "${{ inputs.tag }}" != "${version}" ]; then
echo "The input tag does not match the version from pyproject.toml:" >&2
echo "${{ inputs.tag }}" >&2
echo "${version}" >&2
exit 1
else
echo "Releasing ${version}"
fi
- name: "Build and push Docker image"
uses: docker/build-push-action@v5
with:
context: .
platforms: linux/amd64,linux/arm64
# Reuse the builder
cache-from: type=gha
cache-to: type=gha,mode=max
push: ${{ inputs.tag != '' }}
tags: ghcr.io/astral-sh/ruff:latest,ghcr.io/astral-sh/ruff:${{ inputs.tag || 'dry-run' }}
labels: ${{ steps.meta.outputs.labels }}
# After the release has been published, we update downstream repositories
# This is separate because if this fails the release is still fine, we just need to do some manual workflow triggers
update-dependents:
@@ -581,7 +503,7 @@ jobs:
needs: publish-release
steps:
- name: "Update pre-commit mirror"
uses: actions/github-script@v7
uses: actions/github-script@v6
with:
github-token: ${{ secrets.RUFF_PRE_COMMIT_PAT }}
script: |

8
.gitignore vendored
View File

@@ -1,5 +1,5 @@
# Benchmarking cpython (CONTRIBUTING.md)
crates/ruff_linter/resources/test/cpython
crates/ruff/resources/test/cpython
# generate_mkdocs.py
mkdocs.generated.yml
# check_ecosystem.py
@@ -208,9 +208,3 @@ cython_debug/
# VIM
.*.sw?
.sw?
# Custom re-inclusions for the resolver test cases
!crates/ruff_python_resolver/resources/test/airflow/venv/
!crates/ruff_python_resolver/resources/test/airflow/venv/lib
!crates/ruff_python_resolver/resources/test/airflow/venv/lib/python3.11/site-packages/_watchdog_fsevents.cpython-311-darwin.so
!crates/ruff_python_resolver/resources/test/airflow/venv/lib/python3.11/site-packages/orjson/orjson.cpython-311-darwin.so

View File

@@ -13,8 +13,3 @@ MD041: false
# MD013/line-length
MD013: false
# MD024/no-duplicate-heading
MD024:
# Allow when nested under different parents e.g. CHANGELOG.md
allow_different_nesting: true

View File

@@ -2,9 +2,9 @@ fail_fast: true
exclude: |
(?x)^(
crates/ruff_linter/resources/.*|
crates/ruff_linter/src/rules/.*/snapshots/.*|
crates/ruff/resources/.*|
crates/ruff/src/rules/.*/snapshots/.*|
crates/ruff_cli/resources/.*|
crates/ruff_python_formatter/resources/.*|
crates/ruff_python_formatter/tests/snapshots/.*|
crates/ruff_python_resolver/resources/.*|
@@ -13,35 +13,26 @@ exclude: |
repos:
- repo: https://github.com/abravalheri/validate-pyproject
rev: v0.15
rev: v0.12.1
hooks:
- id: validate-pyproject
- repo: https://github.com/executablebooks/mdformat
rev: 0.7.17
rev: 0.7.16
hooks:
- id: mdformat
additional_dependencies:
- mdformat-mkdocs
- mdformat-admon
exclude: |
(?x)^(
docs/formatter/black\.md
| docs/\w+\.md
)$
- mdformat-black
- black==23.1.0 # Must be the latest version of Black
- repo: https://github.com/igorshubovych/markdownlint-cli
rev: v0.37.0
rev: v0.33.0
hooks:
- id: markdownlint-fix
exclude: |
(?x)^(
docs/formatter/black\.md
| docs/\w+\.md
)$
- repo: https://github.com/crate-ci/typos
rev: v1.16.22
rev: v1.14.12
hooks:
- id: typos
@@ -51,29 +42,25 @@ repos:
name: cargo fmt
entry: cargo fmt --
language: system
types: [rust]
types: [ rust ]
pass_filenames: false # This makes it a lot faster
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.1.4
hooks:
- id: ruff-format
- id: ruff
args: [--fix, --exit-non-zero-on-fix]
types_or: [python, pyi]
name: ruff
entry: cargo run --bin ruff -- check --no-cache --force-exclude --fix --exit-non-zero-on-fix
language: system
types_or: [ python, pyi ]
require_serial: true
exclude: |
(?x)^(
crates/ruff_linter/resources/.*|
crates/ruff/resources/.*|
crates/ruff_python_formatter/resources/.*
)$
# Prettier
- repo: https://github.com/pre-commit/mirrors-prettier
rev: v3.0.3
# Black
- repo: https://github.com/psf/black
rev: 23.1.0
hooks:
- id: prettier
types: [yaml]
- id: black
ci:
skip: [cargo-fmt, dev-generate-all]
skip: [ cargo-fmt, dev-generate-all ]

View File

@@ -1,97 +1,5 @@
# Breaking Changes
## 0.1.9
### `site-packages` is now excluded by default ([#5513](https://github.com/astral-sh/ruff/pull/5513))
Ruff maintains a list of default exclusions, which now consists of the following patterns:
- `.bzr`
- `.direnv`
- `.eggs`
- `.git-rewrite`
- `.git`
- `.hg`
- `.ipynb_checkpoints`
- `.mypy_cache`
- `.nox`
- `.pants.d`
- `.pyenv`
- `.pytest_cache`
- `.pytype`
- `.ruff_cache`
- `.svn`
- `.tox`
- `.venv`
- `.vscode`
- `__pypackages__`
- `_build`
- `buck-out`
- `build`
- `dist`
- `node_modules`
- `site-packages`
- `venv`
Previously, the `site-packages` directory was not excluded by default. While `site-packages` tends
to be excluded anyway by virtue of the `.venv` exclusion, this may not be the case when using Ruff
from VS Code outside a virtual environment.
## 0.1.0
### The deprecated `format` setting has been removed
Ruff previously used the `format` setting, `--format` CLI option, and `RUFF_FORMAT` environment variable to
configure the output format of the CLI. This usage was deprecated in `v0.0.291` — the `format` setting is now used
to control Ruff's code formatting. As of this release:
- The `format` setting cannot be used to configure the output format, use `output-format` instead
- The `RUFF_FORMAT` environment variable is ignored, use `RUFF_OUTPUT_FORMAT` instead
- The `--format` option has been removed from `ruff check`, use `--output-format` instead
### Unsafe fixes are not applied by default ([#7769](https://github.com/astral-sh/ruff/pull/7769))
Ruff labels fixes as "safe" and "unsafe". The meaning and intent of your code will be retained when applying safe
fixes, but the meaning could be changed when applying unsafe fixes. Previously, unsafe fixes were always displayed
and applied when fixing was enabled. Now, unsafe fixes are hidden by default and not applied. The `--unsafe-fixes`
flag or `unsafe-fixes` configuration option can be used to enable unsafe fixes.
See the [docs](https://docs.astral.sh/ruff/configuration/#fix-safety) for details.
### Remove formatter-conflicting rules from the default rule set ([#7900](https://github.com/astral-sh/ruff/pull/7900))
Previously, Ruff enabled all implemented rules in Pycodestyle (`E`) by default. Ruff now only includes the
Pycodestyle prefixes `E4`, `E7`, and `E9` to exclude rules that conflict with automatic formatters. Consequently,
the stable rule set no longer includes `line-too-long` (`E501`) and `mixed-spaces-and-tabs` (`E101`). Other
excluded Pycodestyle rules include whitespace enforcement in `E1` and `E2`; these rules are currently in preview, and are already omitted by default.
This change only affects those using Ruff under its default rule set. Users that include `E` in their `select` will experience no change in behavior.
## 0.0.288
### Remove support for emoji identifiers ([#7212](https://github.com/astral-sh/ruff/pull/7212))
Previously, Ruff supported the non-standard compliant emoji identifiers e.g. `📦 = 1`.
We decided to remove this non-standard language extension, and Ruff now reports syntax errors for emoji identifiers in your code, the same as CPython.
### Improved GitLab fingerprints ([#7203](https://github.com/astral-sh/ruff/pull/7203))
GitLab uses fingerprints to identify new, existing, or fixed violations. Previously, Ruff included the violation's position in the fingerprint. Using the location has the downside that changing any code before the violation causes the fingerprint to change, resulting in GitLab reporting one fixed and one new violation even though it is a pre-existing violation.
Ruff now uses a more stable location-agnostic fingerprint to minimize that existing violations incorrectly get marked as fixed and re-reported as new violations.
Expect GitLab to report each pre-existing violation in your project as fixed and a new violation in your Ruff upgrade PR.
## 0.0.283 / 0.284
### The target Python version now defaults to 3.8 instead of 3.10 ([#6397](https://github.com/astral-sh/ruff/pull/6397))
Previously, when a target Python version was not specified, Ruff would use a default of Python 3.10. However, it is safer to default to an _older_ Python version to avoid assuming the availability of new features. We now default to the oldest supported Python version which is currently Python 3.8.
(We still support Python 3.7 but since [it has reached EOL](https://devguide.python.org/versions/#unsupported-versions) we've decided not to make it the default here.)
Note this change was announced in 0.0.283 but not active until 0.0.284.
## 0.0.277
### `.ipynb_checkpoints`, `.pyenv`, `.pytest_cache`, and `.vscode` are now excluded by default ([#5513](https://github.com/astral-sh/ruff/pull/5513))
@@ -366,4 +274,4 @@ default.
`pyproject.toml` files are now resolved hierarchically, such that for each Python file, we find
the first `pyproject.toml` file in its path, and use that to determine its lint settings.
See the [documentation](https://docs.astral.sh/ruff/configuration/#python-file-discovery) for more.
See the [documentation](https://beta.ruff.rs/docs/configuration/#python-file-discovery) for more.

File diff suppressed because it is too large Load Diff

View File

@@ -72,7 +72,7 @@ representative at an online or offline event.
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported to the community leaders responsible for enforcement at
<charlie.r.marsh@gmail.com>.
charlie.r.marsh@gmail.com.
All complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the

View File

@@ -26,10 +26,6 @@ Welcome! We're happy to have you here. Thank you in advance for your contributio
- [`cargo dev`](#cargo-dev)
- [Subsystems](#subsystems)
- [Compilation Pipeline](#compilation-pipeline)
- [Import Categorization](#import-categorization)
- [Project root](#project-root)
- [Package root](#package-root)
- [Import categorization](#import-categorization-1)
## The Basics
@@ -67,35 +63,18 @@ You'll also need [Insta](https://insta.rs/docs/) to update snapshot tests:
cargo install cargo-insta
```
And you'll need pre-commit to run some validation checks:
and pre-commit to run some validation checks:
```shell
pipx install pre-commit # or `pip install pre-commit` if you have a virtualenv
```
You can optionally install pre-commit hooks to automatically run the validation checks
when making a commit:
```shell
pre-commit install
```
We recommend [nextest](https://nexte.st/) to run Ruff's test suite (via `cargo nextest run`),
though it's not strictly necessary:
```shell
cargo install cargo-nextest --locked
```
Throughout this guide, any usages of `cargo test` can be replaced with `cargo nextest run`,
if you choose to install `nextest`.
### Development
After cloning the repository, run Ruff locally from the repository root with:
```shell
cargo run -p ruff -- check /path/to/file.py --no-cache
cargo run -p ruff_cli -- check /path/to/file.py --no-cache
```
Prior to opening a pull request, ensure that your code has been auto-formatted,
@@ -126,15 +105,15 @@ Ruff is structured as a monorepo with a [flat crate structure](https://matklad.g
such that all crates are contained in a flat `crates` directory.
The vast majority of the code, including all lint rules, lives in the `ruff` crate (located at
`crates/ruff_linter`). As a contributor, that's the crate that'll be most relevant to you.
`crates/ruff`). As a contributor, that's the crate that'll be most relevant to you.
At the time of writing, the repository includes the following crates:
At time of writing, the repository includes the following crates:
- `crates/ruff_linter`: library crate containing all lint rules and the core logic for running them.
- `crates/ruff`: library crate containing all lint rules and the core logic for running them.
If you're working on a rule, this is the crate for you.
- `crates/ruff_benchmark`: binary crate for running micro-benchmarks.
- `crates/ruff_cache`: library crate for caching lint results.
- `crates/ruff`: binary crate containing Ruff's command-line interface.
- `crates/ruff_cli`: binary crate containing Ruff's command-line interface.
- `crates/ruff_dev`: binary crate containing utilities used in the development of Ruff itself (e.g.,
`cargo dev generate-all`), see the [`cargo dev`](#cargo-dev) section below.
- `crates/ruff_diagnostics`: library crate for the rule-independent abstractions in the lint
@@ -143,9 +122,9 @@ At the time of writing, the repository includes the following crates:
intermediate representation. The backend for `ruff_python_formatter`.
- `crates/ruff_index`: library crate inspired by `rustc_index`.
- `crates/ruff_macros`: proc macro crate containing macros used by Ruff.
- `crates/ruff_notebook`: library crate for parsing and manipulating Jupyter notebooks.
- `crates/ruff_python_ast`: library crate containing Python-specific AST types and utilities.
- `crates/ruff_python_codegen`: library crate containing utilities for generating Python source code.
- `crates/ruff_python_codegen`: library crate containing utilities for generating Python source code.
- `crates/ruff_python_formatter`: library crate implementing the Python formatter. Emits an
intermediate representation for each node, which `ruff_formatter` prints based on the configured
line length.
@@ -167,7 +146,7 @@ At a high level, the steps involved in adding a new lint rule are as follows:
1. Determine a name for the new rule as per our [rule naming convention](#rule-naming-convention)
(e.g., `AssertFalse`, as in, "allow `assert False`").
1. Create a file for your rule (e.g., `crates/ruff_linter/src/rules/flake8_bugbear/rules/assert_false.rs`).
1. Create a file for your rule (e.g., `crates/ruff/src/rules/flake8_bugbear/rules/assert_false.rs`).
1. In that file, define a violation struct (e.g., `pub struct AssertFalse`). You can grep for
`#[violation]` to see examples.
@@ -176,22 +155,21 @@ At a high level, the steps involved in adding a new lint rule are as follows:
(e.g., `pub(crate) fn assert_false`) based on whatever inputs are required for the rule (e.g.,
an `ast::StmtAssert` node).
1. Define the logic for invoking the diagnostic in `crates/ruff_linter/src/checkers/ast/analyze` (for
AST-based rules), `crates/ruff_linter/src/checkers/tokens.rs` (for token-based rules),
`crates/ruff_linter/src/checkers/physical_lines.rs` (for text-based rules),
`crates/ruff_linter/src/checkers/filesystem.rs` (for filesystem-based rules), etc. For AST-based rules,
1. Define the logic for invoking the diagnostic in `crates/ruff/src/checkers/ast/analyze` (for
AST-based rules), `crates/ruff/src/checkers/tokens.rs` (for token-based rules),
`crates/ruff/src/checkers/physical_lines.rs` (for text-based rules),
`crates/ruff/src/checkers/filesystem.rs` (for filesystem-based rules), etc. For AST-based rules,
you'll likely want to modify `analyze/statement.rs` (if your rule is based on analyzing
statements, like imports) or `analyze/expression.rs` (if your rule is based on analyzing
expressions, like function calls).
1. Map the violation struct to a rule code in `crates/ruff_linter/src/codes.rs` (e.g., `B011`). New rules
should be added in `RuleGroup::Preview`.
1. Map the violation struct to a rule code in `crates/ruff/src/codes.rs` (e.g., `B011`).
1. Add proper [testing](#rule-testing-fixtures-and-snapshots) for your rule.
1. Update the generated files (documentation and generated code).
To trigger the violation, you'll likely want to augment the logic in `crates/ruff_linter/src/checkers/ast.rs`
To trigger the violation, you'll likely want to augment the logic in `crates/ruff/src/checkers/ast.rs`
to call your new function at the appropriate time and with the appropriate inputs. The `Checker`
defined therein is a Python AST visitor, which iterates over the AST, building up a semantic model,
and calling out to lint rule analyzer functions as it goes.
@@ -219,7 +197,7 @@ As such, rule names should...
For example, `AssertFalse` guards against `assert False` statements.
- _Not_ contain instructions on how to fix the violation, which instead belong in the rule
documentation and the `fix_title`.
documentation and the `autofix_title`.
- _Not_ contain a redundant prefix, like `Disallow` or `Banned`, which are already implied by the
convention.
@@ -236,7 +214,7 @@ Ruff's output for each fixture, which you can then commit alongside your changes
Once you've completed the code for the rule itself, you can define tests with the following steps:
1. Add a Python file to `crates/ruff_linter/resources/test/fixtures/[linter]` that contains the code you
1. Add a Python file to `crates/ruff/resources/test/fixtures/[linter]` that contains the code you
want to test. The file name should match the rule name (e.g., `E402.py`), and it should include
examples of both violations and non-violations.
@@ -245,16 +223,16 @@ Once you've completed the code for the rule itself, you can define tests with th
For example, if you're adding a new rule named `E402`, you would run:
```shell
cargo run -p ruff -- check crates/ruff_linter/resources/test/fixtures/pycodestyle/E402.py --no-cache --preview --select E402
cargo run -p ruff_cli -- check crates/ruff/resources/test/fixtures/pycodestyle/E402.py --no-cache --select E402
```
**Note:** Only a subset of rules are enabled by default. When testing a new rule, ensure that
you activate it by adding `--select ${rule_code}` to the command.
1. Add the test to the relevant `crates/ruff_linter/src/rules/[linter]/mod.rs` file. If you're contributing
1. Add the test to the relevant `crates/ruff/src/rules/[linter]/mod.rs` file. If you're contributing
a rule to a pre-existing set, you should be able to find a similar example to pattern-match
against. If you're adding a new linter, you'll need to create a new `mod.rs` file (see,
e.g., `crates/ruff_linter/src/rules/flake8_bugbear/mod.rs`)
e.g., `crates/ruff/src/rules/flake8_bugbear/mod.rs`)
1. Run `cargo test`. Your test will fail, but you'll be prompted to follow-up
with `cargo insta review`. Run `cargo insta review`, review and accept the generated snapshot,
@@ -266,24 +244,25 @@ Once you've completed the code for the rule itself, you can define tests with th
Ruff's user-facing settings live in a few different places.
First, the command-line options are defined via the `Args` struct in `crates/ruff/src/args.rs`.
First, the command-line options are defined via the `Cli` struct in `crates/ruff/src/cli.rs`.
Second, the `pyproject.toml` options are defined in `crates/ruff_workspace/src/options.rs` (via the
`Options` struct), `crates/ruff_workspace/src/configuration.rs` (via the `Configuration` struct),
and `crates/ruff_workspace/src/settings.rs` (via the `Settings` struct), which then includes
the `LinterSettings` struct as a field.
These represent, respectively: the schema used to parse the `pyproject.toml` file; an internal,
intermediate representation; and the final, internal representation used to power Ruff.
Second, the `pyproject.toml` options are defined in `crates/ruff/src/settings/options.rs` (via the
`Options` struct), `crates/ruff/src/settings/configuration.rs` (via the `Configuration` struct), and
`crates/ruff/src/settings/mod.rs` (via the `Settings` struct). These represent, respectively: the
schema used to parse the `pyproject.toml` file; an internal, intermediate representation; and the
final, internal representation used to power Ruff.
To add a new configuration option, you'll likely want to modify these latter few files (along with
`arg.rs`, if appropriate). If you want to pattern-match against an existing example, grep for
`cli.rs`, if appropriate). If you want to pattern-match against an existing example, grep for
`dummy_variable_rgx`, which defines a regular expression to match against acceptable unused
variables (e.g., `_`).
Note that plugin-specific configuration options are defined in their own modules (e.g.,
`Settings` in `crates/ruff_linter/src/flake8_unused_arguments/settings.rs` coupled with
`Flake8UnusedArgumentsOptions` in `crates/ruff_workspace/src/options.rs`).
`crates/ruff/src/flake8_unused_arguments/settings.rs`).
You may also want to add the new configuration option to the `flake8-to-ruff` tool, which is
responsible for converting `flake8` configuration files to Ruff's TOML format. This logic
lives in `crates/ruff/src/flake8_to_ruff/converter.rs`.
Finally, regenerate the documentation and generated code with `cargo dev generate-all`.
@@ -309,7 +288,7 @@ To preview any changes to the documentation locally:
```shell
# For contributors.
mkdocs serve -f mkdocs.public.yml
mkdocs serve -f mkdocs.generated.yml
# For members of the Astral org, which has access to MkDocs Insiders via sponsorship.
mkdocs serve -f mkdocs.insiders.yml
@@ -329,115 +308,93 @@ even patch releases may contain [non-backwards-compatible changes](https://semve
### Creating a new release
We use an experimental in-house tool for managing releases.
1. Install `rooster`: `pip install git+https://github.com/zanieb/rooster@main`
1. Run `rooster release`; this command will:
- Generate a changelog entry in `CHANGELOG.md`
- Update versions in `pyproject.toml` and `Cargo.toml`
- Update references to versions in the `README.md` and documentation
1. The changelog should then be editorialized for consistency
- Often labels will be missing from pull requests they will need to be manually organized into the proper section
- Changes should be edited to be user-facing descriptions, avoiding internal details
1. Highlight any breaking changes in `BREAKING_CHANGES.md`
1. Run `cargo check`. This should update the lock file with new versions.
1. Create a pull request with the changelog and version updates
1. Update the version with `rg 0.0.269 --files-with-matches | xargs sed -i 's/0.0.269/0.0.270/g'`
1. Update `BREAKING_CHANGES.md`
1. Create a PR with the version and `BREAKING_CHANGES.md` updated
1. Merge the PR
1. Run the [release workflow](https://github.com/astral-sh/ruff/actions/workflows/release.yaml) with:
- The new version number (without starting `v`)
- The commit hash of the merged release pull request on `main`
1. Run the release workflow with the version number (without starting `v`) as input. Make sure
main has your merged PR as last commit
1. The release workflow will do the following:
1. Build all the assets. If this fails (even though we tested in step 4), we haven't tagged or
uploaded anything, you can restart after pushing a fix.
1. Upload to PyPI.
1. Create and push the Git tag (as extracted from `pyproject.toml`). We create the Git tag only
after building the wheels and uploading to PyPI, since we can't delete or modify the tag ([#4468](https://github.com/astral-sh/ruff/issues/4468)).
after building the wheels and uploading to PyPI, since we can't delete or modify the tag ([#4468](https://github.com/charliermarsh/ruff/issues/4468)).
1. Attach artifacts to draft GitHub release
1. Trigger downstream repositories. This can fail non-catastrophically, as we can run any
downstream jobs manually if needed.
1. Publish the GitHub release
1. Open the draft release in the GitHub release section
1. Copy the changelog for the release into the GitHub release
- See previous releases for formatting of section headers
1. Generate the contributor list with `rooster contributors` and add to the release notes
1. If needed, [update the schemastore](https://github.com/astral-sh/ruff/blob/main/scripts/update_schemastore.py).
1. One can determine if an update is needed when
`git diff old-version-tag new-version-tag -- ruff.schema.json` returns a non-empty diff.
1. Once run successfully, you should follow the link in the output to create a PR.
1. Create release notes in GitHub UI and promote from draft.
1. If needed, [update the schemastore](https://github.com/charliermarsh/ruff/blob/main/scripts/update_schemastore.py)
1. If needed, update the `ruff-lsp` and `ruff-vscode` repositories.
## Ecosystem CI
GitHub Actions will run your changes against a number of real-world projects from GitHub and
report on any linter or formatter differences. You can also run those checks locally via:
report on any diagnostic differences. You can also run those checks locally via:
```shell
pip install -e ./python/ruff-ecosystem
ruff-ecosystem check ruff "./target/debug/ruff"
ruff-ecosystem format ruff "./target/debug/ruff"
python scripts/check_ecosystem.py path/to/your/ruff path/to/older/ruff
```
See the [ruff-ecosystem package](https://github.com/astral-sh/ruff/tree/main/python/ruff-ecosystem) for more details.
You can also run the Ecosystem CI check in a Docker container across a larger set of projects by
downloading the [`known-github-tomls.json`](https://github.com/akx/ruff-usage-aggregate/blob/master/data/known-github-tomls.jsonl)
as `github_search.jsonl` and following the instructions in [scripts/Dockerfile.ecosystem](https://github.com/astral-sh/ruff/blob/main/scripts/Dockerfile.ecosystem).
Note that this check will take a while to run.
## Benchmarking and Profiling
We have several ways of benchmarking and profiling Ruff:
- Our main performance benchmark comparing Ruff with other tools on the CPython codebase
- Microbenchmarks which run the linter or the formatter on individual files. These run on pull requests.
- Microbenchmarks which the linter or the formatter on individual files. There run on pull requests.
- Profiling the linter on either the microbenchmarks or entire projects
> \[!NOTE\]
> When running benchmarks, ensure that your CPU is otherwise idle (e.g., close any background
> applications, like web browsers). You may also want to switch your CPU to a "performance"
> mode, if it exists, especially when benchmarking short-lived processes.
### CPython Benchmark
First, clone [CPython](https://github.com/python/cpython). It's a large and diverse Python codebase,
which makes it a good target for benchmarking.
```shell
git clone --branch 3.10 https://github.com/python/cpython.git crates/ruff_linter/resources/test/cpython
git clone --branch 3.10 https://github.com/python/cpython.git crates/ruff/resources/test/cpython
```
To benchmark the release build:
```shell
cargo build --release && hyperfine --warmup 10 \
"./target/release/ruff ./crates/ruff_linter/resources/test/cpython/ --no-cache -e" \
"./target/release/ruff ./crates/ruff_linter/resources/test/cpython/ -e"
"./target/release/ruff ./crates/ruff/resources/test/cpython/ --no-cache -e" \
"./target/release/ruff ./crates/ruff/resources/test/cpython/ -e"
Benchmark 1: ./target/release/ruff ./crates/ruff_linter/resources/test/cpython/ --no-cache
Benchmark 1: ./target/release/ruff ./crates/ruff/resources/test/cpython/ --no-cache
Time (mean ± σ): 293.8 ms ± 3.2 ms [User: 2384.6 ms, System: 90.3 ms]
Range (min … max): 289.9 ms … 301.6 ms 10 runs
Benchmark 2: ./target/release/ruff ./crates/ruff_linter/resources/test/cpython/
Benchmark 2: ./target/release/ruff ./crates/ruff/resources/test/cpython/
Time (mean ± σ): 48.0 ms ± 3.1 ms [User: 65.2 ms, System: 124.7 ms]
Range (min … max): 45.0 ms … 66.7 ms 62 runs
Summary
'./target/release/ruff ./crates/ruff_linter/resources/test/cpython/' ran
6.12 ± 0.41 times faster than './target/release/ruff ./crates/ruff_linter/resources/test/cpython/ --no-cache'
'./target/release/ruff ./crates/ruff/resources/test/cpython/' ran
6.12 ± 0.41 times faster than './target/release/ruff ./crates/ruff/resources/test/cpython/ --no-cache'
```
To benchmark against the ecosystem's existing tools:
```shell
hyperfine --ignore-failure --warmup 5 \
"./target/release/ruff ./crates/ruff_linter/resources/test/cpython/ --no-cache" \
"pyflakes crates/ruff_linter/resources/test/cpython" \
"./target/release/ruff ./crates/ruff/resources/test/cpython/ --no-cache" \
"pyflakes crates/ruff/resources/test/cpython" \
"autoflake --recursive --expand-star-imports --remove-all-unused-imports --remove-unused-variables --remove-duplicate-keys resources/test/cpython" \
"pycodestyle crates/ruff_linter/resources/test/cpython" \
"flake8 crates/ruff_linter/resources/test/cpython"
"pycodestyle crates/ruff/resources/test/cpython" \
"flake8 crates/ruff/resources/test/cpython"
Benchmark 1: ./target/release/ruff ./crates/ruff_linter/resources/test/cpython/ --no-cache
Benchmark 1: ./target/release/ruff ./crates/ruff/resources/test/cpython/ --no-cache
Time (mean ± σ): 294.3 ms ± 3.3 ms [User: 2467.5 ms, System: 89.6 ms]
Range (min … max): 291.1 ms … 302.8 ms 10 runs
Warning: Ignoring non-zero exit code.
Benchmark 2: pyflakes crates/ruff_linter/resources/test/cpython
Benchmark 2: pyflakes crates/ruff/resources/test/cpython
Time (mean ± σ): 15.786 s ± 0.143 s [User: 15.560 s, System: 0.214 s]
Range (min … max): 15.640 s … 16.157 s 10 runs
@@ -447,31 +404,31 @@ Benchmark 3: autoflake --recursive --expand-star-imports --remove-all-unused-imp
Time (mean ± σ): 6.175 s ± 0.169 s [User: 54.102 s, System: 1.057 s]
Range (min … max): 5.950 s … 6.391 s 10 runs
Benchmark 4: pycodestyle crates/ruff_linter/resources/test/cpython
Benchmark 4: pycodestyle crates/ruff/resources/test/cpython
Time (mean ± σ): 46.921 s ± 0.508 s [User: 46.699 s, System: 0.202 s]
Range (min … max): 46.171 s … 47.863 s 10 runs
Warning: Ignoring non-zero exit code.
Benchmark 5: flake8 crates/ruff_linter/resources/test/cpython
Benchmark 5: flake8 crates/ruff/resources/test/cpython
Time (mean ± σ): 12.260 s ± 0.321 s [User: 102.934 s, System: 1.230 s]
Range (min … max): 11.848 s … 12.933 s 10 runs
Warning: Ignoring non-zero exit code.
Summary
'./target/release/ruff ./crates/ruff_linter/resources/test/cpython/ --no-cache' ran
'./target/release/ruff ./crates/ruff/resources/test/cpython/ --no-cache' ran
20.98 ± 0.62 times faster than 'autoflake --recursive --expand-star-imports --remove-all-unused-imports --remove-unused-variables --remove-duplicate-keys resources/test/cpython'
41.66 ± 1.18 times faster than 'flake8 crates/ruff_linter/resources/test/cpython'
53.64 ± 0.77 times faster than 'pyflakes crates/ruff_linter/resources/test/cpython'
159.43 ± 2.48 times faster than 'pycodestyle crates/ruff_linter/resources/test/cpython'
41.66 ± 1.18 times faster than 'flake8 crates/ruff/resources/test/cpython'
53.64 ± 0.77 times faster than 'pyflakes crates/ruff/resources/test/cpython'
159.43 ± 2.48 times faster than 'pycodestyle crates/ruff/resources/test/cpython'
```
To benchmark a subset of rules, e.g. `LineTooLong` and `DocLineTooLong`:
```shell
cargo build --release && hyperfine --warmup 10 \
"./target/release/ruff ./crates/ruff_linter/resources/test/cpython/ --no-cache -e --select W505,E501"
"./target/release/ruff ./crates/ruff/resources/test/cpython/ --no-cache -e --select W505,E501"
```
You can run `poetry install` from `./scripts/benchmarks` to create a working environment for the
@@ -504,10 +461,10 @@ rm Lib/test/bad_coding.py \
Lib/test/test_typing.py
```
Then, from `crates/ruff_linter/resources/test/cpython`, run: `time pylint -j 0 -E $(git ls-files '*.py')`. This
Then, from `crates/ruff/resources/test/cpython`, run: `time pylint -j 0 -E $(git ls-files '*.py')`. This
will execute Pylint with maximum parallelism and only report errors.
To benchmark Pyupgrade, run the following from `crates/ruff_linter/resources/test/cpython`:
To benchmark Pyupgrade, run the following from `crates/ruff/resources/test/cpython`:
```shell
hyperfine --ignore-failure --warmup 5 --prepare "git reset --hard HEAD" \
@@ -537,10 +494,10 @@ if the benchmark improved/regressed compared to that baseline.
```shell
# Run once on your "baseline" code
cargo bench -p ruff_benchmark -- --save-baseline=main
cargo benchmark --save-baseline=main
# Then iterate with
cargo bench -p ruff_benchmark -- --baseline=main
cargo benchmark --baseline=main
```
#### PR Summary
@@ -550,10 +507,10 @@ This is useful to illustrate the improvements of a PR.
```shell
# On main
cargo bench -p ruff_benchmark -- --save-baseline=main
cargo benchmark --save-baseline=main
# After applying your changes
cargo bench -p ruff_benchmark -- --save-baseline=pr
cargo benchmark --save-baseline=pr
critcmp main pr
```
@@ -566,10 +523,10 @@ cargo install critcmp
#### Tips
- Use `cargo bench -p ruff_benchmark <filter>` to only run specific benchmarks. For example: `cargo benchmark lexer`
to only run the lexer benchmarks.
- Use `cargo bench -p ruff_benchmark -- --quiet` for a more cleaned up output (without statistical relevance)
- Use `cargo bench -p ruff_benchmark -- --quick` to get faster results (more prone to noise)
- Use `cargo benchmark <filter>` to only run specific benchmarks. For example: `cargo benchmark linter/pydantic`
to only run the pydantic tests.
- Use `cargo benchmark --quiet` for a more cleaned up output (without statistical relevance)
- Use `cargo benchmark --quick` to get faster results (more prone to noise)
### Profiling Projects
@@ -580,10 +537,10 @@ examples.
#### Linux
Install `perf` and build `ruff_benchmark` with the `profiling` profile and then run it with perf
Install `perf` and build `ruff_benchmark` with the `release-debug` profile and then run it with perf
```shell
cargo bench -p ruff_benchmark --no-run --profile=profiling && perf record --call-graph dwarf -F 9999 cargo bench -p ruff_benchmark --profile=profiling -- --profile-time=1
cargo bench -p ruff_benchmark --no-run --profile=release-debug && perf record --call-graph dwarf -F 9999 cargo bench -p ruff_benchmark --profile=release-debug -- --profile-time=1
```
You can also use the `ruff_dev` launcher to run `ruff check` multiple times on a repository to
@@ -591,8 +548,8 @@ gather enough samples for a good flamegraph (change the 999, the sample rate, an
of checks, to your liking)
```shell
cargo build --bin ruff_dev --profile=profiling
perf record -g -F 999 target/profiling/ruff_dev repeat --repeat 30 --exit-zero --no-cache path/to/cpython > /dev/null
cargo build --bin ruff_dev --profile=release-debug
perf record -g -F 999 target/release-debug/ruff_dev repeat --repeat 30 --exit-zero --no-cache path/to/cpython > /dev/null
```
Then convert the recorded profile
@@ -608,7 +565,7 @@ An alternative is to convert the perf data to `flamegraph.svg` using
[flamegraph](https://github.com/flamegraph-rs/flamegraph) (`cargo install flamegraph`):
```shell
flamegraph --perfdata perf.data --no-inline
flamegraph --perfdata perf.data
```
#### Mac
@@ -622,7 +579,7 @@ cargo install cargo-instruments
Then run the profiler with
```shell
cargo instruments -t time --bench linter --profile profiling -p ruff_benchmark -- --profile-time=1
cargo instruments -t time --bench linter --profile release-debug -p ruff_benchmark -- --profile-time=1
```
- `-t`: Specifies what to profile. Useful options are `time` to profile the wall time and `alloc`
@@ -637,7 +594,7 @@ Otherwise, follow the instructions from the linux section.
utils with it:
- `cargo dev print-ast <file>`: Print the AST of a python file using the
[RustPython parser](https://github.com/astral-sh/ruff/tree/main/crates/ruff_python_parser) that is
[RustPython parser](https://github.com/astral-sh/RustPython-Parser/tree/main/parser) that is
mainly used in Ruff. For `if True: pass # comment`, you can see the syntax tree, the byte offsets
for start and stop of each node and also how the `:` token, the comment and whitespace are not
represented anymore:
@@ -755,8 +712,8 @@ Module {
- `cargo dev generate-cli-help`, `cargo dev generate-docs` and `cargo dev generate-json-schema`:
Update just `docs/configuration.md`, `docs/rules` and `ruff.schema.json` respectively.
- `cargo dev generate-options`: Generate a markdown-compatible table of all `pyproject.toml`
options. Used for <https://docs.astral.sh/ruff/settings/>.
- `cargo dev generate-rules-table`: Generate a markdown-compatible table of all rules. Used for <https://docs.astral.sh/ruff/rules/>.
options. Used for <https://beta.ruff.rs/docs/settings/>
- `cargo dev generate-rules-table`: Generate a markdown-compatible table of all rules. Used for <https://beta.ruff.rs/docs/rules/>
- `cargo dev round-trip <python file or jupyter notebook>`: Read a Python file or Jupyter Notebook,
parse it, serialize the parsed representation and write it back. Used to check how good our
representation is so that fixes don't rewrite irrelevant parts of a file.
@@ -814,7 +771,7 @@ To understand Ruff's import categorization system, we first need to define two c
- "Package root": The top-most directory defining the Python package that includes a given Python
file. To find the package root for a given Python file, traverse up its parent directories until
you reach a parent directory that doesn't contain an `__init__.py` file (and isn't marked as
a [namespace package](https://docs.astral.sh/ruff/settings/#namespace-packages)); take the directory
a [namespace package](https://beta.ruff.rs/docs/settings/#namespace-packages)); take the directory
just before that, i.e., the first directory in the package.
For example, given:
@@ -903,7 +860,7 @@ There are three ways in which an import can be categorized as "first-party":
package (e.g., `from foo import bar` or `import foo.bar`), they'll be classified as first-party
automatically. This check is as simple as comparing the first segment of the current file's
module path to the first segment of the import.
1. **Source roots**: Ruff supports a `[src](https://docs.astral.sh/ruff/settings/#src)` setting, which
1. **Source roots**: Ruff supports a `[src](https://beta.ruff.rs/docs/settings/#src)` setting, which
sets the directories to scan when identifying first-party imports. The algorithm is
straightforward: given an import, like `import foo`, iterate over the directories enumerated in
the `src` setting and, for each directory, check for the existence of a subdirectory `foo` or a
@@ -913,5 +870,5 @@ By default, `src` is set to the project root. In the above example, we'd want to
`src = ["./src"]` to ensure that we locate `./my_project/src/foo` and thus categorize `import foo`
as first-party in `baz.py`. In practice, for this limited example, setting `src = ["./src"]` is
unnecessary, as all imports within `./my_project/src/foo` would be categorized as first-party via
the same-package heuristic; but if your project contains multiple packages, you'll want to set `src`
the same-package heuristic; but your project contains multiple packages, you'll want to set `src`
explicitly.

1933
Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -4,160 +4,55 @@ resolver = "2"
[workspace.package]
edition = "2021"
rust-version = "1.71"
homepage = "https://docs.astral.sh/ruff"
documentation = "https://docs.astral.sh/ruff"
rust-version = "1.70"
homepage = "https://beta.ruff.rs/docs"
documentation = "https://beta.ruff.rs/docs"
repository = "https://github.com/astral-sh/ruff"
authors = ["Charlie Marsh <charlie.r.marsh@gmail.com>"]
license = "MIT"
[workspace.dependencies]
aho-corasick = { version = "1.1.2" }
annotate-snippets = { version = "0.9.2", features = ["color"] }
anyhow = { version = "1.0.79" }
argfile = { version = "0.1.6" }
assert_cmd = { version = "2.0.13" }
bincode = { version = "1.3.3" }
bitflags = { version = "2.4.1" }
bstr = { version = "1.9.0" }
cachedir = { version = "0.3.1" }
chrono = { version = "0.4.34", default-features = false, features = ["clock"] }
clap = { version = "4.4.18", features = ["derive"] }
clap_complete_command = { version = "0.5.1" }
clearscreen = { version = "2.0.0" }
codspeed-criterion-compat = { version = "2.3.3", default-features = false }
colored = { version = "2.1.0" }
configparser = { version = "3.0.3" }
console_error_panic_hook = { version = "0.1.7" }
console_log = { version = "1.0.0" }
countme = { version ="3.0.1"}
criterion = { version = "0.5.1", default-features = false }
dirs = { version = "5.0.0" }
drop_bomb = { version = "0.1.5" }
env_logger = { version ="0.10.1"}
fern = { version = "0.6.1" }
filetime = { version = "0.2.23" }
fs-err = { version ="2.11.0"}
anyhow = { version = "1.0.69" }
bitflags = { version = "2.3.1" }
chrono = { version = "0.4.23", default-features = false, features = ["clock"] }
clap = { version = "4.1.8", features = ["derive"] }
colored = { version = "2.0.0" }
filetime = { version = "0.2.20" }
glob = { version = "0.3.1" }
globset = { version = "0.4.14" }
hexf-parse = { version ="0.2.1"}
ignore = { version = "0.4.22" }
imara-diff ={ version = "0.1.5"}
imperative = { version = "1.0.4" }
indicatif ={ version = "0.17.8"}
indoc ={ version = "2.0.4"}
insta = { version = "1.34.0", feature = ["filters", "glob"] }
insta-cmd = { version = "0.4.0" }
is-macro = { version = "0.3.5" }
is-wsl = { version = "0.4.0" }
itertools = { version = "0.12.1" }
js-sys = { version = "0.3.67" }
lalrpop-util = { version = "0.20.0", default-features = false }
lexical-parse-float = { version = "0.8.0", features = ["format"] }
libcst = { version = "1.1.0", default-features = false }
globset = { version = "0.4.10" }
ignore = { version = "0.4.20" }
insta = { version = "1.31.0", feature = ["filters", "glob"] }
is-macro = { version = "0.2.2" }
itertools = { version = "0.10.5" }
log = { version = "0.4.17" }
memchr = { version = "2.7.1" }
mimalloc = { version ="0.1.39"}
natord = { version = "1.0.9" }
notify = { version = "6.1.1" }
once_cell = { version = "1.19.0" }
path-absolutize = { version = "3.1.1" }
pathdiff = { version = "0.2.1" }
pep440_rs = { version = "0.4.0", features = ["serde"] }
pretty_assertions = "1.3.0"
proc-macro2 = { version = "1.0.78" }
pyproject-toml = { version = "0.9.0" }
quick-junit = { version = "0.3.5" }
memchr = "2.5.0"
num-bigint = { version = "0.4.3" }
num-traits = { version = "0.2.15" }
once_cell = { version = "1.17.1" }
path-absolutize = { version = "3.0.14" }
proc-macro2 = { version = "1.0.51" }
quote = { version = "1.0.23" }
rand = { version = "0.8.5" }
rayon = { version = "1.8.1" }
regex = { version = "1.10.2" }
result-like = { version = "0.5.0" }
regex = { version = "1.7.1" }
rustc-hash = { version = "1.1.0" }
schemars = { version = "0.8.16" }
seahash = { version ="4.1.0"}
semver = { version = "1.0.21" }
serde = { version = "1.0.196", features = ["derive"] }
serde-wasm-bindgen = { version = "0.6.3" }
serde_json = { version = "1.0.113" }
serde_test = { version = "1.0.152" }
serde_with = { version = "3.6.0", default-features = false, features = ["macros"] }
schemars = { version = "0.8.12" }
serde = { version = "1.0.152", features = ["derive"] }
serde_json = { version = "1.0.93" }
shellexpand = { version = "3.0.0" }
shlex = { version ="1.3.0"}
similar = { version = "2.4.0", features = ["inline"] }
smallvec = { version = "1.13.1" }
static_assertions = "1.1.0"
strum = { version = "0.25.0", features = ["strum_macros"] }
strum_macros = { version = "0.25.3" }
syn = { version = "2.0.40" }
tempfile = { version ="3.9.0"}
test-case = { version = "3.3.1" }
thiserror = { version = "1.0.57" }
tikv-jemallocator = { version ="0.5.0"}
toml = { version = "0.8.9" }
tracing = { version = "0.1.40" }
tracing-indicatif = { version = "0.3.6" }
tracing-subscriber = { version = "0.3.18", features = ["env-filter"] }
typed-arena = { version = "2.0.2" }
unic-ucd-category = { version ="0.9"}
unicode-ident = { version = "1.0.12" }
unicode-width = { version = "0.1.11" }
unicode_names2 = { version = "1.2.1" }
ureq = { version = "2.9.1" }
url = { version = "2.5.0" }
uuid = { version = "1.6.1", features = ["v4", "fast-rng", "macro-diagnostics", "js"] }
walkdir = { version = "2.3.2" }
wasm-bindgen = { version = "0.2.84" }
wasm-bindgen-test = { version = "0.3.40" }
wild = { version = "2" }
similar = { version = "2.2.1", features = ["inline"] }
smallvec = { version = "1.10.0" }
strum = { version = "0.24.1", features = ["strum_macros"] }
strum_macros = { version = "0.24.3" }
syn = { version = "2.0.15" }
test-case = { version = "3.0.0" }
thiserror = { version = "1.0.43" }
toml = { version = "0.7.2" }
wsl = { version = "0.1.0" }
[workspace.lints.rust]
unsafe_code = "warn"
unreachable_pub = "warn"
[workspace.lints.clippy]
pedantic = { level = "warn", priority = -2 }
# Allowed pedantic lints
char_lit_as_u8 = "allow"
collapsible_else_if = "allow"
collapsible_if = "allow"
implicit_hasher = "allow"
match_same_arms = "allow"
missing_errors_doc = "allow"
missing_panics_doc = "allow"
module_name_repetitions = "allow"
must_use_candidate = "allow"
similar_names = "allow"
too_many_lines = "allow"
# To allow `#[allow(clippy::all)]` in `crates/ruff_python_parser/src/python.rs`.
needless_raw_string_hashes = "allow"
# Disallowed restriction lints
print_stdout = "warn"
print_stderr = "warn"
dbg_macro = "warn"
empty_drop = "warn"
empty_structs_with_brackets = "warn"
exit = "warn"
get_unwrap = "warn"
rc_buffer = "warn"
rc_mutex = "warn"
rest_pat_in_fully_bound_structs = "warn"
# v1.0.1
libcst = { git = "https://github.com/Instagram/LibCST.git", rev = "3cacca1a1029f05707e50703b49fe3dd860aa839", default-features = false }
[profile.release]
# Note that we set these explicitly, and these values
# were chosen based on a trade-off between compile times
# and runtime performance[1].
#
# [1]: https://github.com/astral-sh/ruff/pull/9031
lto = "thin"
codegen-units = 16
# Some crates don't change as much but benefit more from
# more expensive optimization passes, so we selectively
# decrease codegen-units in some cases.
[profile.release.package.ruff_python_parser]
codegen-units = 1
[profile.release.package.ruff_python_ast]
lto = "fat"
codegen-units = 1
[profile.dev.package.insta]
@@ -171,8 +66,8 @@ opt-level = 3
[profile.dev.package.ruff_python_parser]
opt-level = 1
# Use the `--profile profiling` flag to show symbols in release mode.
# e.g. `cargo build --profile profiling`
[profile.profiling]
# Use the `--profile release-debug` flag to show symbols in release mode.
# e.g. `cargo build --profile release-debug`
[profile.release-debug]
inherits = "release"
debug = 1

View File

@@ -1,38 +0,0 @@
FROM --platform=$BUILDPLATFORM ubuntu as build
ENV HOME="/root"
WORKDIR $HOME
RUN apt update && apt install -y build-essential curl python3-venv
# Setup zig as cross compiling linker
RUN python3 -m venv $HOME/.venv
RUN .venv/bin/pip install cargo-zigbuild
ENV PATH="$HOME/.venv/bin:$PATH"
# Install rust
ARG TARGETPLATFORM
RUN case "$TARGETPLATFORM" in \
"linux/arm64") echo "aarch64-unknown-linux-musl" > rust_target.txt ;; \
"linux/amd64") echo "x86_64-unknown-linux-musl" > rust_target.txt ;; \
*) exit 1 ;; \
esac
# Update rustup whenever we bump the rust version
COPY rust-toolchain.toml rust-toolchain.toml
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --target $(cat rust_target.txt) --profile minimal --default-toolchain none
ENV PATH="$HOME/.cargo/bin:$PATH"
# Installs the correct toolchain version from rust-toolchain.toml and then the musl target
RUN rustup target add $(cat rust_target.txt)
# Build
COPY crates crates
COPY Cargo.toml Cargo.toml
COPY Cargo.lock Cargo.lock
RUN cargo zigbuild --bin ruff --target $(cat rust_target.txt) --release
RUN cp target/$(cat rust_target.txt)/release/ruff /ruff
# TODO: Optimize binary size, with a version that also works when cross compiling
# RUN strip --strip-all /ruff
FROM scratch
COPY --from=build /ruff /ruff
WORKDIR /io
ENTRYPOINT ["/ruff"]

72
LICENSE
View File

@@ -1194,27 +1194,7 @@ are:
- flake8-self, licensed as follows:
"""
MIT License
Copyright (c) 2023 Korijn van Golen
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
Freely Distributable
"""
- flake8-django, licensed under the GPL license.
@@ -1244,56 +1224,6 @@ are:
SOFTWARE.
"""
- flake8-logging, licensed as follows:
"""
MIT License
Copyright (c) 2023 Adam Johnson
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
"""
- flake8-trio, licensed as follows:
"""
MIT License
Copyright (c) 2022 Zac Hatfield-Dodds
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
"""
- Pyright, licensed as follows:
"""
MIT License

161
README.md
View File

@@ -8,9 +8,9 @@
[![image](https://img.shields.io/pypi/pyversions/ruff.svg)](https://pypi.python.org/pypi/ruff)
[![Actions status](https://github.com/astral-sh/ruff/workflows/CI/badge.svg)](https://github.com/astral-sh/ruff/actions)
[**Discord**](https://discord.gg/c9MhzV8aU5) | [**Docs**](https://docs.astral.sh/ruff/) | [**Playground**](https://play.ruff.rs/)
[**Discord**](https://discord.gg/c9MhzV8aU5) | [**Docs**](https://beta.ruff.rs/docs/) | [**Playground**](https://play.ruff.rs/)
An extremely fast Python linter and code formatter, written in Rust.
An extremely fast Python linter, written in Rust.
<p align="center">
<picture align="center">
@@ -24,27 +24,28 @@ An extremely fast Python linter and code formatter, written in Rust.
<i>Linting the CPython codebase from scratch.</i>
</p>
- ⚡️ 10-100x faster than existing linters (like Flake8) and formatters (like Black)
- ⚡️ 10-100x faster than existing linters
- 🐍 Installable via `pip`
- 🛠️ `pyproject.toml` support
- 🤝 Python 3.12 compatibility
- ⚖️ Drop-in parity with [Flake8](https://docs.astral.sh/ruff/faq/#how-does-ruff-compare-to-flake8), isort, and Black
- 🤝 Python 3.11 compatibility
- 📦 Built-in caching, to avoid re-analyzing unchanged files
- 🔧 Fix support, for automatic error correction (e.g., automatically remove unused imports)
- 📏 Over [700 built-in rules](https://docs.astral.sh/ruff/rules/), with native re-implementations
of popular Flake8 plugins, like flake8-bugbear
- ⌨️ First-party [editor integrations](https://docs.astral.sh/ruff/integrations/) for
- 🔧 Autofix support, for automatic error correction (e.g., automatically remove unused imports)
- 📏 Over [500 built-in rules](https://beta.ruff.rs/docs/rules/)
- ⚖️ [Near-parity](https://beta.ruff.rs/docs/faq/#how-does-ruff-compare-to-flake8) with the
built-in Flake8 rule set
- 🔌 Native re-implementations of dozens of Flake8 plugins, like flake8-bugbear
- ⌨️ First-party [editor integrations](https://beta.ruff.rs/docs/editor-integrations/) for
[VS Code](https://github.com/astral-sh/ruff-vscode) and [more](https://github.com/astral-sh/ruff-lsp)
- 🌎 Monorepo-friendly, with [hierarchical and cascading configuration](https://docs.astral.sh/ruff/configuration/#pyprojecttoml-discovery)
- 🌎 Monorepo-friendly, with [hierarchical and cascading configuration](https://beta.ruff.rs/docs/configuration/#pyprojecttoml-discovery)
Ruff aims to be orders of magnitude faster than alternative tools while integrating more
functionality behind a single, common interface.
Ruff can be used to replace [Flake8](https://pypi.org/project/flake8/) (plus dozens of plugins),
[Black](https://github.com/psf/black), [isort](https://pypi.org/project/isort/),
[pydocstyle](https://pypi.org/project/pydocstyle/), [pyupgrade](https://pypi.org/project/pyupgrade/),
[autoflake](https://pypi.org/project/autoflake/), and more, all while executing tens or hundreds of
times faster than any individual tool.
[isort](https://pypi.org/project/isort/), [pydocstyle](https://pypi.org/project/pydocstyle/),
[yesqa](https://github.com/asottile/yesqa), [eradicate](https://pypi.org/project/eradicate/),
[pyupgrade](https://pypi.org/project/pyupgrade/), and [autoflake](https://pypi.org/project/autoflake/),
all while executing tens or hundreds of times faster than any individual tool.
Ruff is extremely actively developed and used in major open-source projects like:
@@ -54,7 +55,7 @@ Ruff is extremely actively developed and used in major open-source projects like
- [Pandas](https://github.com/pandas-dev/pandas)
- [SciPy](https://github.com/scipy/scipy)
...and [many more](#whos-using-ruff).
...and many more.
Ruff is backed by [Astral](https://astral.sh). Read the [launch post](https://astral.sh/blog/announcing-astral-the-company-behind-ruff),
or the original [project announcement](https://notes.crmarsh.com/python-tooling-could-be-much-much-faster).
@@ -97,7 +98,7 @@ developer of [Zulip](https://github.com/zulip/zulip):
## Table of Contents
For more, see the [documentation](https://docs.astral.sh/ruff/).
For more, see the [documentation](https://beta.ruff.rs/docs/).
1. [Getting Started](#getting-started)
1. [Configuration](#configuration)
@@ -110,7 +111,7 @@ For more, see the [documentation](https://docs.astral.sh/ruff/).
## Getting Started
For more, see the [documentation](https://docs.astral.sh/ruff/).
For more, see the [documentation](https://beta.ruff.rs/docs/).
### Installation
@@ -121,42 +122,27 @@ pip install ruff
```
You can also install Ruff via [Homebrew](https://formulae.brew.sh/formula/ruff), [Conda](https://anaconda.org/conda-forge/ruff),
and with [a variety of other package managers](https://docs.astral.sh/ruff/installation/).
and with [a variety of other package managers](https://beta.ruff.rs/docs/installation/).
### Usage
To run Ruff as a linter, try any of the following:
To run Ruff, try any of the following:
```shell
ruff check . # Lint all files in the current directory (and any subdirectories).
ruff check path/to/code/ # Lint all files in `/path/to/code` (and any subdirectories).
ruff check path/to/code/*.py # Lint all `.py` files in `/path/to/code`.
ruff check path/to/code/to/file.py # Lint `file.py`.
ruff check @arguments.txt # Lint using an input file, treating its contents as newline-delimited command-line arguments.
ruff check . # Lint all files in the current directory (and any subdirectories)
ruff check path/to/code/ # Lint all files in `/path/to/code` (and any subdirectories)
ruff check path/to/code/*.py # Lint all `.py` files in `/path/to/code`
ruff check path/to/code/to/file.py # Lint `file.py`
```
Or, to run Ruff as a formatter:
```shell
ruff format . # Format all files in the current directory (and any subdirectories).
ruff format path/to/code/ # Format all files in `/path/to/code` (and any subdirectories).
ruff format path/to/code/*.py # Format all `.py` files in `/path/to/code`.
ruff format path/to/code/to/file.py # Format `file.py`.
ruff format @arguments.txt # Format using an input file, treating its contents as newline-delimited command-line arguments.
```
Ruff can also be used as a [pre-commit](https://pre-commit.com/) hook via [`ruff-pre-commit`](https://github.com/astral-sh/ruff-pre-commit):
Ruff can also be used as a [pre-commit](https://pre-commit.com) hook:
```yaml
- repo: https://github.com/astral-sh/ruff-pre-commit
# Ruff version.
rev: v0.2.1
rev: v0.0.280
hooks:
# Run the linter.
- id: ruff
args: [ --fix ]
# Run the formatter.
- id: ruff-format
```
Ruff can also be used as a [VS Code extension](https://github.com/astral-sh/ruff-vscode) or
@@ -179,13 +165,21 @@ jobs:
### Configuration
Ruff can be configured through a `pyproject.toml`, `ruff.toml`, or `.ruff.toml` file (see:
[_Configuration_](https://docs.astral.sh/ruff/configuration/), or [_Settings_](https://docs.astral.sh/ruff/settings/)
[_Configuration_](https://beta.ruff.rs/docs/configuration/), or [_Settings_](https://beta.ruff.rs/docs/settings/)
for a complete list of all configuration options).
If left unspecified, Ruff's default configuration is equivalent to:
If left unspecified, the default configuration is equivalent to:
```toml
[tool.ruff]
# Enable pycodestyle (`E`) and Pyflakes (`F`) codes by default.
select = ["E", "F"]
ignore = []
# Allow autofix for all enabled rules (when `--fix`) is provided.
fixable = ["A", "B", "C", "D", "E", "F", "G", "I", "N", "Q", "S", "T", "W", "ANN", "ARG", "BLE", "COM", "DJ", "DTZ", "EM", "ERA", "EXE", "FBT", "ICN", "INP", "ISC", "NPY", "PD", "PGH", "PIE", "PL", "PT", "PTH", "PYI", "RET", "RSE", "RUF", "SIM", "SLF", "TCH", "TID", "TRY", "UP", "YTT"]
unfixable = []
# Exclude a variety of commonly ignored directories.
exclude = [
".bzr",
@@ -194,81 +188,58 @@ exclude = [
".git",
".git-rewrite",
".hg",
".ipynb_checkpoints",
".mypy_cache",
".nox",
".pants.d",
".pyenv",
".pytest_cache",
".pytype",
".ruff_cache",
".svn",
".tox",
".venv",
".vscode",
"__pypackages__",
"_build",
"buck-out",
"build",
"dist",
"node_modules",
"site-packages",
"venv",
]
# Same as Black.
line-length = 88
indent-width = 4
# Assume Python 3.8
target-version = "py38"
[tool.ruff.lint]
# Enable Pyflakes (`F`) and a subset of the pycodestyle (`E`) codes by default.
select = ["E4", "E7", "E9", "F"]
ignore = []
# Allow fix for all enabled rules (when `--fix`) is provided.
fixable = ["ALL"]
unfixable = []
# Allow unused variables when underscore-prefixed.
dummy-variable-rgx = "^(_+|(_+[a-zA-Z0-9_]*[a-zA-Z0-9]+?))$"
[tool.ruff.format]
# Like Black, use double quotes for strings.
quote-style = "double"
# Assume Python 3.10.
target-version = "py310"
# Like Black, indent with spaces, rather than tabs.
indent-style = "space"
# Like Black, respect magic trailing commas.
skip-magic-trailing-comma = false
# Like Black, automatically detect the appropriate line ending.
line-ending = "auto"
[tool.ruff.mccabe]
# Unlike Flake8, default to a complexity level of 10.
max-complexity = 10
```
Some configuration options can be provided via the command-line, such as those related to
rule enablement and disablement, file discovery, and logging level:
rule enablement and disablement, file discovery, logging level, and more:
```shell
ruff check path/to/code/ --select F401 --select F403 --quiet
```
See `ruff help` for more on Ruff's top-level commands, or `ruff help check` and `ruff help format`
for more on the linting and formatting commands, respectively.
See `ruff help` for more on Ruff's top-level commands, or `ruff help check` for more on the
linting command.
## Rules
<!-- Begin section: Rules -->
**Ruff supports over 700 lint rules**, many of which are inspired by popular tools like Flake8,
**Ruff supports over 500 lint rules**, many of which are inspired by popular tools like Flake8,
isort, pyupgrade, and others. Regardless of the rule's origin, Ruff re-implements every rule in
Rust as a first-party feature.
By default, Ruff enables Flake8's `F` rules, along with a subset of the `E` rules, omitting any
stylistic rules that overlap with the use of a formatter, like `ruff format` or
By default, Ruff enables Flake8's `E` and `F` rules. Ruff supports all rules from the `F` category,
and a [subset](https://beta.ruff.rs/docs/rules/#error-e) of the `E` category, omitting those
stylistic rules made obsolete by the use of an autoformatter, like
[Black](https://github.com/psf/black).
If you're just getting started with Ruff, **the default rule set is a great place to start**: it
@@ -303,7 +274,6 @@ quality tools, including:
- [flake8-gettext](https://pypi.org/project/flake8-gettext/)
- [flake8-implicit-str-concat](https://pypi.org/project/flake8-implicit-str-concat/)
- [flake8-import-conventions](https://github.com/joaopalmeiro/flake8-import-conventions)
- [flake8-logging](https://pypi.org/project/flake8-logging/)
- [flake8-logging-format](https://pypi.org/project/flake8-logging-format/)
- [flake8-no-pep420](https://pypi.org/project/flake8-no-pep420)
- [flake8-pie](https://pypi.org/project/flake8-pie/)
@@ -319,7 +289,6 @@ quality tools, including:
- [flake8-super](https://pypi.org/project/flake8-super/)
- [flake8-tidy-imports](https://pypi.org/project/flake8-tidy-imports/)
- [flake8-todos](https://pypi.org/project/flake8-todos/)
- [flake8-trio](https://pypi.org/project/flake8-trio/)
- [flake8-type-checking](https://pypi.org/project/flake8-type-checking/)
- [flake8-use-pathlib](https://pypi.org/project/flake8-use-pathlib/)
- [flynt](https://pypi.org/project/flynt/) ([#2102](https://github.com/astral-sh/ruff/issues/2102))
@@ -334,12 +303,12 @@ quality tools, including:
- [tryceratops](https://pypi.org/project/tryceratops/)
- [yesqa](https://pypi.org/project/yesqa/)
For a complete enumeration of the supported rules, see [_Rules_](https://docs.astral.sh/ruff/rules/).
For a complete enumeration of the supported rules, see [_Rules_](https://beta.ruff.rs/docs/rules/).
## Contributing
Contributions are welcome and highly appreciated. To get started, check out the
[**contributing guidelines**](https://docs.astral.sh/ruff/contributing/).
[**contributing guidelines**](https://beta.ruff.rs/docs/contributing/).
You can also join us on [**Discord**](https://discord.gg/c9MhzV8aU5).
@@ -361,7 +330,7 @@ In some cases, Ruff includes a "direct" Rust port of the corresponding tool.
We're grateful to the maintainers of these tools for their work, and for all
the value they've provided to the Python community.
Ruff's formatter is built on a fork of Rome's [`rome_formatter`](https://github.com/rome/tools/tree/main/crates/rome_formatter),
Ruff's autoformatter is built on a fork of Rome's [`rome_formatter`](https://github.com/rome/tools/tree/main/crates/rome_formatter),
and again draws on both API and implementation details from [Rome](https://github.com/rome/tools),
[Prettier](https://github.com/prettier/prettier), and [Black](https://github.com/psf/black).
@@ -382,11 +351,10 @@ Ruff is used by a number of major open-source projects and companies, including:
- Anthropic ([Python SDK](https://github.com/anthropics/anthropic-sdk-python))
- [Apache Airflow](https://github.com/apache/airflow)
- AstraZeneca ([Magnus](https://github.com/AstraZeneca/magnus-core))
- [Babel](https://github.com/python-babel/babel)
- Benchling ([Refac](https://github.com/benchling/refac))
- [Babel](https://github.com/python-babel/babel)
- [Bokeh](https://github.com/bokeh/bokeh)
- [Cryptography (PyCA)](https://github.com/pyca/cryptography)
- CERN ([Indico](https://getindico.io/))
- [DVC](https://github.com/iterative/dvc)
- [Dagger](https://github.com/dagger/dagger)
- [Dagster](https://github.com/dagster-io/dagster)
@@ -395,17 +363,15 @@ Ruff is used by a number of major open-source projects and companies, including:
- [Gradio](https://github.com/gradio-app/gradio)
- [Great Expectations](https://github.com/great-expectations/great_expectations)
- [HTTPX](https://github.com/encode/httpx)
- [Hatch](https://github.com/pypa/hatch)
- [Home Assistant](https://github.com/home-assistant/core)
- Hugging Face ([Transformers](https://github.com/huggingface/transformers),
[Datasets](https://github.com/huggingface/datasets),
[Diffusers](https://github.com/huggingface/diffusers))
- [Hatch](https://github.com/pypa/hatch)
- [Home Assistant](https://github.com/home-assistant/core)
- ING Bank ([popmon](https://github.com/ing-bank/popmon), [probatus](https://github.com/ing-bank/probatus))
- [Ibis](https://github.com/ibis-project/ibis)
- [ivy](https://github.com/unifyai/ivy)
- [Jupyter](https://github.com/jupyter-server/jupyter_server)
- [LangChain](https://github.com/hwchase17/langchain)
- [Litestar](https://litestar.dev/)
- [LlamaIndex](https://github.com/jerryjliu/llama_index)
- Matrix ([Synapse](https://github.com/matrix-org/synapse))
- [MegaLinter](https://github.com/oxsecurity/megalinter)
@@ -418,37 +384,28 @@ Ruff is used by a number of major open-source projects and companies, including:
- [Mypy](https://github.com/python/mypy)
- Netflix ([Dispatch](https://github.com/Netflix/dispatch))
- [Neon](https://github.com/neondatabase/neon)
- [NoneBot](https://github.com/nonebot/nonebot2)
- [NumPyro](https://github.com/pyro-ppl/numpyro)
- [ONNX](https://github.com/onnx/onnx)
- [OpenBB](https://github.com/OpenBB-finance/OpenBBTerminal)
- [PDM](https://github.com/pdm-project/pdm)
- [PaddlePaddle](https://github.com/PaddlePaddle/Paddle)
- [Pandas](https://github.com/pandas-dev/pandas)
- [Pillow](https://github.com/python-pillow/Pillow)
- [Poetry](https://github.com/python-poetry/poetry)
- [Polars](https://github.com/pola-rs/polars)
- [PostHog](https://github.com/PostHog/posthog)
- Prefect ([Python SDK](https://github.com/PrefectHQ/prefect), [Marvin](https://github.com/PrefectHQ/marvin))
- [PyInstaller](https://github.com/pyinstaller/pyinstaller)
- [PyMC](https://github.com/pymc-devs/pymc/)
- [PyMC-Marketing](https://github.com/pymc-labs/pymc-marketing)
- [pytest](https://github.com/pytest-dev/pytest)
- [PyTorch](https://github.com/pytorch/pytorch)
- [Pydantic](https://github.com/pydantic/pydantic)
- [Pylint](https://github.com/PyCQA/pylint)
- [PyVista](https://github.com/pyvista/pyvista)
- [Reflex](https://github.com/reflex-dev/reflex)
- [River](https://github.com/online-ml/river)
- [Rippling](https://rippling.com)
- [Robyn](https://github.com/sansyrox/robyn)
- [Saleor](https://github.com/saleor/saleor)
- Scale AI ([Launch SDK](https://github.com/scaleapi/launch-python-client))
- [SciPy](https://github.com/scipy/scipy)
- Snowflake ([SnowCLI](https://github.com/Snowflake-Labs/snowcli))
- [Saleor](https://github.com/saleor/saleor)
- [SciPy](https://github.com/scipy/scipy)
- [Sphinx](https://github.com/sphinx-doc/sphinx)
- [Stable Baselines3](https://github.com/DLR-RM/stable-baselines3)
- [Starlette](https://github.com/encode/starlette)
- [Litestar](https://litestar.dev/)
- [The Algorithms](https://github.com/TheAlgorithms/Python)
- [Vega-Altair](https://github.com/altair-viz/altair)
- WordPress ([Openverse](https://github.com/WordPress/openverse))
@@ -464,7 +421,7 @@ Ruff is used by a number of major open-source projects and companies, including:
### Show Your Support
If you're using Ruff, consider adding the Ruff badge to your project's `README.md`:
If you're using Ruff, consider adding the Ruff badge to project's `README.md`:
```md
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
@@ -490,6 +447,6 @@ MIT
<div align="center">
<a target="_blank" href="https://astral.sh" style="background:none">
<img src="https://raw.githubusercontent.com/astral-sh/ruff/main/assets/svg/Astral.svg" alt="Made by Astral">
<img src="https://raw.githubusercontent.com/astral-sh/ruff/main/assets/svg/Astral.svg">
</a>
</div>

View File

@@ -1,6 +1,5 @@
[files]
# https://github.com/crate-ci/typos/issues/868
extend-exclude = ["**/resources/**/*", "**/snapshots/**/*"]
extend-exclude = ["resources", "snapshots"]
[default.extend-words]
hel = "hel"

View File

@@ -1,8 +0,0 @@
{
"label": "code style",
"message": "Ruff",
"logoSvg": "<svg width=\"510\" height=\"622\" viewBox=\"0 0 510 622\" fill=\"none\" xmlns=\"http://www.w3.org/2000/svg\"><path fill-rule=\"evenodd\" clip-rule=\"evenodd\" d=\"M206.701 0C200.964 0 196.314 4.64131 196.314 10.3667V41.4667C196.314 47.192 191.663 51.8333 185.927 51.8333H156.843C151.107 51.8333 146.456 56.4746 146.456 62.2V145.133C146.456 150.859 141.806 155.5 136.069 155.5H106.986C101.249 155.5 96.5988 160.141 96.5988 165.867V222.883C96.5988 228.609 91.9484 233.25 86.2118 233.25H57.1283C51.3917 233.25 46.7413 237.891 46.7413 243.617V300.633C46.7413 306.359 42.0909 311 36.3544 311H10.387C4.6504 311 0 315.641 0 321.367V352.467C0 358.192 4.6504 362.833 10.387 362.833H145.418C151.154 362.833 155.804 367.475 155.804 373.2V430.217C155.804 435.942 151.154 440.583 145.418 440.583H116.334C110.597 440.583 105.947 445.225 105.947 450.95V507.967C105.947 513.692 101.297 518.333 95.5601 518.333H66.4766C60.74 518.333 56.0896 522.975 56.0896 528.7V611.633C56.0896 617.359 60.74 622 66.4766 622H149.572C155.309 622 159.959 617.359 159.959 611.633V570.167H201.507C207.244 570.167 211.894 565.525 211.894 559.8V528.7C211.894 522.975 216.544 518.333 222.281 518.333H251.365C257.101 518.333 261.752 513.692 261.752 507.967V476.867C261.752 471.141 266.402 466.5 272.138 466.5H301.222C306.959 466.5 311.609 461.859 311.609 456.133V425.033C311.609 419.308 316.259 414.667 321.996 414.667H351.079C356.816 414.667 361.466 410.025 361.466 404.3V373.2C361.466 367.475 366.117 362.833 371.853 362.833H400.937C406.673 362.833 411.324 358.192 411.324 352.467V321.367C411.324 315.641 415.974 311 421.711 311H450.794C456.531 311 461.181 306.359 461.181 300.633V217.7C461.181 211.975 456.531 207.333 450.794 207.333H420.672C414.936 207.333 410.285 202.692 410.285 196.967V165.867C410.285 160.141 414.936 155.5 420.672 155.5H449.756C455.492 155.5 460.143 150.859 460.143 145.133V114.033C460.143 108.308 464.793 103.667 470.53 103.667H499.613C505.35 103.667 510 99.0253 510 93.3V10.3667C510 4.64132 505.35 0 499.613 0H206.701ZM168.269 440.583C162.532 440.583 157.882 445.225 157.882 450.95V507.967C157.882 513.692 153.231 518.333 147.495 518.333H118.411C112.675 518.333 108.024 522.975 108.024 528.7V559.8C108.024 565.525 112.675 570.167 118.411 570.167H159.959V528.7C159.959 522.975 164.61 518.333 170.346 518.333H199.43C205.166 518.333 209.817 513.692 209.817 507.967V476.867C209.817 471.141 214.467 466.5 220.204 466.5H249.287C255.024 466.5 259.674 461.859 259.674 456.133V425.033C259.674 419.308 264.325 414.667 270.061 414.667H299.145C304.881 414.667 309.532 410.025 309.532 404.3V373.2C309.532 367.475 314.182 362.833 319.919 362.833H349.002C354.739 362.833 359.389 358.192 359.389 352.467V321.367C359.389 315.641 364.039 311 369.776 311H398.859C404.596 311 409.246 306.359 409.246 300.633V269.533C409.246 263.808 404.596 259.167 398.859 259.167H318.88C313.143 259.167 308.493 254.525 308.493 248.8V217.7C308.493 211.975 313.143 207.333 318.88 207.333H347.963C353.7 207.333 358.35 202.692 358.35 196.967V165.867C358.35 160.141 363.001 155.5 368.737 155.5H397.821C403.557 155.5 408.208 150.859 408.208 145.133V114.033C408.208 108.308 412.858 103.667 418.595 103.667H447.678C453.415 103.667 458.065 99.0253 458.065 93.3V62.2C458.065 56.4746 453.415 51.8333 447.678 51.8333H208.778C203.041 51.8333 198.391 56.4746 198.391 62.2V145.133C198.391 150.859 193.741 155.5 188.004 155.5H158.921C153.184 155.5 148.534 160.141 148.534 165.867V222.883C148.534 228.609 143.883 233.25 138.147 233.25H109.063C103.327 233.25 98.6762 237.891 98.6762 243.617V300.633C98.6762 306.359 103.327 311 109.063 311H197.352C203.089 311 207.739 315.641 207.739 321.367V430.217C207.739 435.942 203.089 440.583 197.352 440.583H168.269Z\" fill=\"#D7FF64\"/></svg>",
"logoWidth": 10,
"labelColor": "grey",
"color": "#261230"
}

View File

@@ -0,0 +1,29 @@
[package]
name = "flake8-to-ruff"
version = "0.0.280"
description = """
Convert Flake8 configuration files to Ruff configuration files.
"""
authors = { workspace = true }
edition = { workspace = true }
rust-version = { workspace = true }
homepage = { workspace = true }
documentation = { workspace = true }
repository = { workspace = true }
license = { workspace = true }
[dependencies]
ruff = { path = "../ruff", default-features = false }
anyhow = { workspace = true }
clap = { workspace = true }
colored = { workspace = true }
configparser = { version = "3.0.2" }
once_cell = { workspace = true }
regex = { workspace = true }
rustc-hash = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
strum = { workspace = true }
strum_macros = { workspace = true }
toml = { workspace = true }

View File

@@ -0,0 +1,99 @@
# flake8-to-ruff
Convert existing Flake8 configuration files (`setup.cfg`, `tox.ini`, or `.flake8`) for use with
[Ruff](https://github.com/astral-sh/ruff).
Generates a Ruff-compatible `pyproject.toml` section.
## Installation and Usage
### Installation
Available as [`flake8-to-ruff`](https://pypi.org/project/flake8-to-ruff/) on PyPI:
```shell
pip install flake8-to-ruff
```
### Usage
To run `flake8-to-ruff`:
```shell
flake8-to-ruff path/to/setup.cfg
flake8-to-ruff path/to/tox.ini
flake8-to-ruff path/to/.flake8
```
`flake8-to-ruff` will print the relevant `pyproject.toml` sections to standard output, like so:
```toml
[tool.ruff]
exclude = [
'.svn',
'CVS',
'.bzr',
'.hg',
'.git',
'__pycache__',
'.tox',
'.idea',
'.mypy_cache',
'.venv',
'node_modules',
'_state_machine.py',
'test_fstring.py',
'bad_coding2.py',
'badsyntax_*.py',
]
select = [
'A',
'E',
'F',
'Q',
]
ignore = []
[tool.ruff.flake8-quotes]
inline-quotes = 'single'
[tool.ruff.pep8-naming]
ignore-names = [
'foo',
'bar',
]
```
### Plugins
`flake8-to-ruff` will attempt to infer any activated plugins based on the settings provided in your
configuration file.
For example, if your `.flake8` file includes a `docstring-convention` property, `flake8-to-ruff`
will enable the appropriate [`flake8-docstrings`](https://pypi.org/project/flake8-docstrings/)
checks.
Alternatively, you can manually specify plugins on the command-line:
```shell
flake8-to-ruff path/to/.flake8 --plugin flake8-builtins --plugin flake8-quotes
```
## Limitations
1. Ruff only supports a subset of the Flake configuration options. `flake8-to-ruff` will warn on and
ignore unsupported options in the `.flake8` file (or equivalent). (Similarly, Ruff has a few
configuration options that don't exist in Flake8.)
1. Ruff will omit any rule codes that are unimplemented or unsupported by Ruff, including rule
codes from unsupported plugins. (See the
[documentation](https://beta.ruff.rs/docs/faq/#how-does-ruff-compare-to-flake8) for the complete
list of supported plugins.)
## License
MIT
## Contributing
Contributions are welcome and hugely appreciated. To get started, check out the
[contributing guidelines](https://github.com/astral-sh/ruff/blob/main/CONTRIBUTING.md).

View File

@@ -0,0 +1,65 @@
[build-system]
requires = [
# The minimum setuptools version is specific to the PEP 517 backend,
# and may be stricter than the version required in `setup.cfg`
"setuptools>=40.6.0,!=60.9.0",
"wheel",
# Must be kept in sync with the `install_requirements` in `setup.cfg`
"cffi>=1.12; platform_python_implementation != 'PyPy'",
"setuptools-rust>=0.11.4",
]
build-backend = "setuptools.build_meta"
[tool.black]
line-length = 79
target-version = ["py36"]
[tool.pytest.ini_options]
addopts = "-r s --capture=no --strict-markers --benchmark-disable"
markers = [
"skip_fips: this test is not executed in FIPS mode",
"supported: parametrized test requiring only_if and skip_message",
]
[tool.mypy]
show_error_codes = true
check_untyped_defs = true
no_implicit_reexport = true
warn_redundant_casts = true
warn_unused_ignores = true
warn_unused_configs = true
strict_equality = true
[[tool.mypy.overrides]]
module = [
"pretend"
]
ignore_missing_imports = true
[tool.coverage.run]
branch = true
relative_files = true
source = [
"cryptography",
"tests/",
]
[tool.coverage.paths]
source = [
"src/cryptography",
"*.tox/*/lib*/python*/site-packages/cryptography",
"*.tox\\*\\Lib\\site-packages\\cryptography",
"*.tox/pypy/site-packages/cryptography",
]
tests =[
"tests/",
"*tests\\",
]
[tool.coverage.report]
exclude_lines = [
"@abc.abstractmethod",
"@abc.abstractproperty",
"@typing.overload",
"if typing.TYPE_CHECKING",
]

View File

@@ -0,0 +1,91 @@
[metadata]
name = cryptography
version = attr: cryptography.__version__
description = cryptography is a package which provides cryptographic recipes and primitives to Python developers.
long_description = file: README.rst
long_description_content_type = text/x-rst
license = BSD-3-Clause OR Apache-2.0
url = https://github.com/pyca/cryptography
author = The Python Cryptographic Authority and individual contributors
author_email = cryptography-dev@python.org
project_urls =
Documentation=https://cryptography.io/
Source=https://github.com/pyca/cryptography/
Issues=https://github.com/pyca/cryptography/issues
Changelog=https://cryptography.io/en/latest/changelog/
classifiers =
Development Status :: 5 - Production/Stable
Intended Audience :: Developers
License :: OSI Approved :: Apache Software License
License :: OSI Approved :: BSD License
Natural Language :: English
Operating System :: MacOS :: MacOS X
Operating System :: POSIX
Operating System :: POSIX :: BSD
Operating System :: POSIX :: Linux
Operating System :: Microsoft :: Windows
Programming Language :: Python
Programming Language :: Python :: 3
Programming Language :: Python :: 3 :: Only
Programming Language :: Python :: 3.6
Programming Language :: Python :: 3.7
Programming Language :: Python :: 3.8
Programming Language :: Python :: 3.9
Programming Language :: Python :: 3.10
Programming Language :: Python :: 3.11
Programming Language :: Python :: Implementation :: CPython
Programming Language :: Python :: Implementation :: PyPy
Topic :: Security :: Cryptography
[options]
python_requires = >=3.6
include_package_data = True
zip_safe = False
package_dir =
=src
packages = find:
# `install_requires` must be kept in sync with `pyproject.toml`
install_requires =
cffi >=1.12
[options.packages.find]
where = src
exclude =
_cffi_src
_cffi_src.*
[options.extras_require]
test =
pytest>=6.2.0
pytest-benchmark
pytest-cov
pytest-subtests
pytest-xdist
pretend
iso8601
pytz
hypothesis>=1.11.4,!=3.79.2
docs =
sphinx >= 1.6.5,!=1.8.0,!=3.1.0,!=3.1.1,!=5.2.0,!=5.2.0.post0
sphinx_rtd_theme
docstest =
pyenchant >= 1.6.11
twine >= 1.12.0
sphinxcontrib-spelling >= 4.0.1
sdist =
setuptools_rust >= 0.11.4
pep8test =
black
flake8
flake8-import-order
pep8-naming
# This extra is for OpenSSH private keys that use bcrypt KDF
# Versions: v3.1.3 - ignore_few_rounds, v3.1.5 - abi3
ssh =
bcrypt >= 3.1.5
[flake8]
ignore = E203,E211,W503,W504,N818
exclude = .tox,*.egg,.git,_build,.hypothesis
select = E,W,F,N,I
application-import-names = cryptography,cryptography_vectors,tests

View File

@@ -0,0 +1,19 @@
[flake8]
# Ignore style and complexity
# E: style errors
# W: style warnings
# C: complexity
# D: docstring warnings (unused pydocstyle extension)
# F841: local variable assigned but never used
ignore = E, C, W, D, F841
builtins = c, get_config
exclude =
.cache,
.github,
docs,
jupyterhub/alembic*,
onbuild,
scripts,
share,
tools,
setup.py

View File

@@ -0,0 +1,43 @@
[flake8]
# Exclude the grpc generated code
exclude = ./manim/grpc/gen/*
max-complexity = 15
max-line-length = 88
statistics = True
# Prevents some flake8-rst-docstrings errors
rst-roles = attr,class,func,meth,mod,obj,ref,doc,exc
rst-directives = manim, SEEALSO, seealso
docstring-convention=numpy
select = A,A00,B,B9,C4,C90,D,E,F,F,PT,RST,SIM,W
# General Compatibility
extend-ignore = E203, W503, D202, D212, D213, D404
# Misc
F401, F403, F405, F841, E501, E731, E402, F811, F821,
# Plug-in: flake8-builtins
A001, A002, A003,
# Plug-in: flake8-bugbear
B006, B007, B008, B009, B010, B903, B950,
# Plug-in: flake8-simplify
SIM105, SIM106, SIM119,
# Plug-in: flake8-comprehensions
C901
# Plug-in: flake8-pytest-style
PT001, PT004, PT006, PT011, PT018, PT022, PT023,
# Plug-in: flake8-docstrings
D100, D101, D102, D103, D104, D105, D106, D107,
D200, D202, D204, D205, D209,
D301,
D400, D401, D402, D403, D405, D406, D407, D409, D411, D412, D414,
# Plug-in: flake8-rst-docstrings
RST201, RST203, RST210, RST212, RST213, RST215,
RST301, RST303,

View File

@@ -0,0 +1,36 @@
[flake8]
min_python_version = 3.7.0
max-line-length = 88
ban-relative-imports = true
# flake8-use-fstring: https://github.com/MichaelKim0407/flake8-use-fstring#--percent-greedy-and---format-greedy
format-greedy = 1
inline-quotes = double
enable-extensions = TC, TC1
type-checking-strict = true
eradicate-whitelist-extend = ^-.*;
extend-ignore =
# E203: Whitespace before ':' (pycqa/pycodestyle#373)
E203,
# SIM106: Handle error-cases first
SIM106,
# ANN101: Missing type annotation for self in method
ANN101,
# ANN102: Missing type annotation for cls in classmethod
ANN102,
# PIE781: assign-and-return
PIE781,
# PIE798 no-unnecessary-class: Consider using a module for namespacing instead
PIE798,
per-file-ignores =
# TC002: Move third-party import '...' into a type-checking block
__init__.py:TC002,
# ANN201: Missing return type annotation for public function
tests/test_*:ANN201
tests/**/test_*:ANN201
extend-exclude =
# Frozen and not subject to change in this repo:
get-poetry.py,
install-poetry.py,
# External to the project's coding standards:
tests/fixtures/*,
tests/**/fixtures/*,

View File

@@ -0,0 +1,19 @@
[flake8]
max-line-length=120
docstring-convention=all
import-order-style=pycharm
application_import_names=bot,tests
exclude=.cache,.venv,.git,constants.py
extend-ignore=
B311,W503,E226,S311,T000,E731
# Missing Docstrings
D100,D104,D105,D107,
# Docstring Whitespace
D203,D212,D214,D215,
# Docstring Quotes
D301,D302,
# Docstring Content
D400,D401,D402,D404,D405,D406,D407,D408,D409,D410,D411,D412,D413,D414,D416,D417
# Type Annotations
ANN002,ANN003,ANN101,ANN102,ANN204,ANN206,ANN401
per-file-ignores=tests/*:D,ANN

View File

@@ -0,0 +1,6 @@
[flake8]
ignore = E203, E501, W503
per-file-ignores =
requests/__init__.py:E402, F401
requests/compat.py:E402, F401
tests/compat.py:F401

View File

@@ -0,0 +1,34 @@
[project]
name = "flake8-to-ruff"
keywords = ["automation", "flake8", "pycodestyle", "pyflakes", "pylint", "clippy"]
classifiers = [
"Development Status :: 3 - Alpha",
"Environment :: Console",
"Intended Audience :: Developers",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
"Programming Language :: Python",
"Programming Language :: Python :: 3.7",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3 :: Only",
"Topic :: Software Development :: Libraries :: Python Modules",
"Topic :: Software Development :: Quality Assurance",
]
author = "Charlie Marsh"
author_email = "charlie.r.marsh@gmail.com"
description = "Convert existing Flake8 configuration to Ruff."
requires-python = ">=3.7"
[project.urls]
repository = "https://github.com/astral-sh/ruff#subdirectory=crates/flake8_to_ruff"
[build-system]
requires = ["maturin>=1.0,<2.0"]
build-backend = "maturin"
[tool.maturin]
bindings = "bin"
strip = true

View File

@@ -0,0 +1,68 @@
//! Utility to generate Ruff's `pyproject.toml` section from a Flake8 INI file.
use std::path::PathBuf;
use anyhow::Result;
use clap::Parser;
use configparser::ini::Ini;
use ruff::flake8_to_ruff::{self, ExternalConfig};
use ruff::logging::{set_up_logging, LogLevel};
#[derive(Parser)]
#[command(
about = "Convert existing Flake8 configuration to Ruff.",
long_about = None
)]
struct Args {
/// Path to the Flake8 configuration file (e.g., `setup.cfg`, `tox.ini`, or
/// `.flake8`).
#[arg(required = true)]
file: PathBuf,
/// Optional path to a `pyproject.toml` file, used to ensure compatibility
/// with Black.
#[arg(long)]
pyproject: Option<PathBuf>,
/// List of plugins to enable.
#[arg(long, value_delimiter = ',')]
plugin: Option<Vec<flake8_to_ruff::Plugin>>,
}
fn main() -> Result<()> {
set_up_logging(&LogLevel::Default)?;
let args = Args::parse();
// Read the INI file.
let mut ini = Ini::new_cs();
ini.set_multiline(true);
let config = ini.load(args.file).map_err(|msg| anyhow::anyhow!(msg))?;
// Read the pyproject.toml file.
let pyproject = args.pyproject.map(flake8_to_ruff::parse).transpose()?;
let external_config = pyproject
.as_ref()
.and_then(|pyproject| pyproject.tool.as_ref())
.map(|tool| ExternalConfig {
black: tool.black.as_ref(),
isort: tool.isort.as_ref(),
..Default::default()
})
.unwrap_or_default();
let external_config = ExternalConfig {
project: pyproject
.as_ref()
.and_then(|pyproject| pyproject.project.as_ref()),
..external_config
};
// Create Ruff's pyproject.toml section.
let pyproject = flake8_to_ruff::convert(&config, &external_config, args.plugin)?;
#[allow(clippy::print_stdout)]
{
println!("{}", toml::to_string_pretty(&pyproject)?);
}
Ok(())
}

View File

@@ -1,6 +1,6 @@
[package]
name = "ruff"
version = "0.2.1"
version = "0.0.280"
publish = false
authors = { workspace = true }
edition = { workspace = true }
@@ -9,68 +9,88 @@ homepage = { workspace = true }
documentation = { workspace = true }
repository = { workspace = true }
license = { workspace = true }
readme = "../../README.md"
default-run = "ruff"
readme = "README.md"
[lib]
name = "ruff"
[dependencies]
ruff_cache = { path = "../ruff_cache" }
ruff_diagnostics = { path = "../ruff_diagnostics" }
ruff_linter = { path = "../ruff_linter", features = ["clap"] }
ruff_diagnostics = { path = "../ruff_diagnostics", features = ["serde"] }
ruff_index = { path = "../ruff_index" }
ruff_macros = { path = "../ruff_macros" }
ruff_notebook = { path = "../ruff_notebook" }
ruff_python_ast = { path = "../ruff_python_ast" }
ruff_python_formatter = { path = "../ruff_python_formatter" }
ruff_source_file = { path = "../ruff_source_file" }
ruff_python_ast = { path = "../ruff_python_ast", features = ["serde"] }
ruff_python_codegen = { path = "../ruff_python_codegen" }
ruff_python_index = { path = "../ruff_python_index" }
ruff_python_literal = { path = "../ruff_python_literal" }
ruff_python_semantic = { path = "../ruff_python_semantic" }
ruff_python_stdlib = { path = "../ruff_python_stdlib" }
ruff_python_trivia = { path = "../ruff_python_trivia" }
ruff_python_parser = { path = "../ruff_python_parser" }
ruff_source_file = { path = "../ruff_source_file", features = ["serde"] }
ruff_text_size = { path = "../ruff_text_size" }
ruff_workspace = { path = "../ruff_workspace" }
annotate-snippets = { version = "0.9.1", features = ["color"] }
anyhow = { workspace = true }
argfile = { workspace = true }
bincode = { workspace = true }
bitflags = { workspace = true }
cachedir = { workspace = true }
chrono = { workspace = true }
clap = { workspace = true, features = ["derive", "env", "wrap_help"] }
clap_complete_command = { workspace = true }
clearscreen = { workspace = true }
clap = { workspace = true, features = ["derive", "string"], optional = true }
colored = { workspace = true }
filetime = { workspace = true }
dirs = { version = "5.0.0" }
fern = { version = "0.6.1" }
glob = { workspace = true }
globset = { workspace = true }
ignore = { workspace = true }
imperative = { version = "1.0.4" }
is-macro = { workspace = true }
itertools = { workspace = true }
libcst = { workspace = true }
log = { workspace = true }
notify = { workspace = true }
path-absolutize = { workspace = true, features = ["once_cell_cache"] }
rayon = { workspace = true }
memchr = { workspace = true }
natord = { version = "1.0.9" }
num-bigint = { workspace = true }
num-traits = { workspace = true }
once_cell = { workspace = true }
path-absolutize = { workspace = true, features = [
"once_cell_cache",
"use_unix_paths_on_wasm",
] }
pathdiff = { version = "0.2.1" }
pep440_rs = { version = "0.3.1", features = ["serde"] }
pyproject-toml = { version = "0.6.0" }
quick-junit = { version = "0.3.2" }
regex = { workspace = true }
result-like = { version = "0.4.6" }
rustc-hash = { workspace = true }
schemars = { workspace = true, optional = true }
semver = { version = "1.0.16" }
serde = { workspace = true }
serde_json = { workspace = true }
serde_with = { version = "3.0.0" }
similar = { workspace = true }
shellexpand = { workspace = true }
strum = { workspace = true, features = [] }
tempfile = { workspace = true }
thiserror = { workspace = true }
smallvec = { workspace = true }
strum = { workspace = true }
strum_macros = { workspace = true }
thiserror = { version = "1.0.43" }
toml = { workspace = true }
tracing = { workspace = true, features = ["log"] }
walkdir = { workspace = true }
wild = { workspace = true }
typed-arena = { version = "2.0.2" }
unicode-width = { version = "0.1.10" }
unicode_names2 = { version = "0.6.0", git = "https://github.com/youknowone/unicode_names2.git", rev = "4ce16aa85cbcdd9cc830410f1a72ef9a235f2fde" }
wsl = { version = "0.1.0" }
[dev-dependencies]
# Enable test rules during development
ruff_linter = { path = "../ruff_linter", features = ["clap", "test-rules"] }
assert_cmd = { workspace = true }
# Avoid writing colored snapshots when running tests from the terminal
colored = { workspace = true, features = ["no-color"]}
insta = { workspace = true, features = ["filters", "json"] }
insta-cmd = { workspace = true }
tempfile = { workspace = true }
insta = { workspace = true }
pretty_assertions = "1.3.0"
test-case = { workspace = true }
# Disable colored output in tests
colored = { workspace = true, features = ["no-color"] }
tempfile = "3.6.0"
[target.'cfg(target_os = "windows")'.dependencies]
mimalloc = { workspace = true }
[target.'cfg(all(not(target_os = "windows"), not(target_os = "openbsd"), any(target_arch = "x86_64", target_arch = "aarch64", target_arch = "powerpc64")))'.dependencies]
tikv-jemallocator = { workspace = true }
[lints]
workspace = true
[features]
default = []
schemars = ["dep:schemars"]
# Enables the UnreachableCode rule
unreachable-code = []

View File

@@ -1,80 +0,0 @@
use std::{fs, path::Path, process::Command};
fn main() {
// The workspace root directory is not available without walking up the tree
// https://github.com/rust-lang/cargo/issues/3946
let workspace_root = Path::new(&std::env::var("CARGO_MANIFEST_DIR").unwrap())
.join("..")
.join("..");
commit_info(&workspace_root);
#[allow(clippy::disallowed_methods)]
let target = std::env::var("TARGET").unwrap();
println!("cargo:rustc-env=RUST_HOST_TARGET={target}");
}
fn commit_info(workspace_root: &Path) {
// If not in a git repository, do not attempt to retrieve commit information
let git_dir = workspace_root.join(".git");
if !git_dir.exists() {
return;
}
let git_head_path = git_dir.join("HEAD");
println!(
"cargo:rerun-if-changed={}",
git_head_path.as_path().display()
);
let git_head_contents = fs::read_to_string(git_head_path);
if let Ok(git_head_contents) = git_head_contents {
// The contents are either a commit or a reference in the following formats
// - "<commit>" when the head is detached
// - "ref <ref>" when working on a branch
// If a commit, checking if the HEAD file has changed is sufficient
// If a ref, we need to add the head file for that ref to rebuild on commit
let mut git_ref_parts = git_head_contents.split_whitespace();
git_ref_parts.next();
if let Some(git_ref) = git_ref_parts.next() {
let git_ref_path = git_dir.join(git_ref);
println!(
"cargo:rerun-if-changed={}",
git_ref_path.as_path().display()
);
}
}
let output = match Command::new("git")
.arg("log")
.arg("-1")
.arg("--date=short")
.arg("--abbrev=9")
.arg("--format=%H %h %cd %(describe)")
.output()
{
Ok(output) if output.status.success() => output,
_ => return,
};
let stdout = String::from_utf8(output.stdout).unwrap();
let mut parts = stdout.split_whitespace();
let mut next = || parts.next().unwrap();
println!("cargo:rustc-env=RUFF_COMMIT_HASH={}", next());
println!("cargo:rustc-env=RUFF_COMMIT_SHORT_HASH={}", next());
println!("cargo:rustc-env=RUFF_COMMIT_DATE={}", next());
// Describe can fail for some commits
// https://git-scm.com/docs/pretty-formats#Documentation/pretty-formats.txt-emdescribeoptionsem
if let Some(describe) = parts.next() {
let mut describe_parts = describe.split('-');
println!(
"cargo:rustc-env=RUFF_LAST_TAG={}",
describe_parts.next().unwrap()
);
// If this is the tagged commit, this component will be missing
println!(
"cargo:rustc-env=RUFF_LAST_TAG_DISTANCE={}",
describe_parts.next().unwrap_or("0")
);
}
}

View File

@@ -0,0 +1,3 @@
# fixtures
Fixture files used for snapshot testing.

View File

@@ -0,0 +1,41 @@
def func():
for i in range(5):
print(i)
def func():
for i in range(20):
print(i)
else:
return 0
def func():
for i in range(10):
if i == 5:
return 1
return 0
def func():
for i in range(111):
if i == 5:
return 1
else:
return 0
return 2
def func():
for i in range(12):
continue
def func():
for i in range(1110):
if True:
continue
def func():
for i in range(13):
break
def func():
for i in range(1110):
if True:
break

View File

@@ -0,0 +1,131 @@
def func(status):
match status:
case _:
return 0
return "unreachable"
def func(status):
match status:
case 1:
return 1
return 0
def func(status):
match status:
case 1:
return 1
case _:
return 0
def func(status):
match status:
case 1 | 2 | 3:
return 5
return 6
def func(status):
match status:
case 1 | 2 | 3:
return 5
case _:
return 10
return 0
def func(status):
match status:
case 0:
return 0
case 1:
return 1
case 1:
return "1 again"
case _:
return 3
def func(status):
i = 0
match status, i:
case _, _:
return 0
def func(status):
i = 0
match status, i:
case _, 0:
return 0
case _, 2:
return 0
def func(point):
match point:
case (0, 0):
print("Origin")
case _:
raise ValueError("oops")
def func(point):
match point:
case (0, 0):
print("Origin")
case (0, y):
print(f"Y={y}")
case (x, 0):
print(f"X={x}")
case (x, y):
print(f"X={x}, Y={y}")
case _:
raise ValueError("Not a point")
def where_is(point):
class Point:
x: int
y: int
match point:
case Point(x=0, y=0):
print("Origin")
case Point(x=0, y=y):
print(f"Y={y}")
case Point(x=x, y=0):
print(f"X={x}")
case Point():
print("Somewhere else")
case _:
print("Not a point")
def func(points):
match points:
case []:
print("No points")
case [Point(0, 0)]:
print("The origin")
case [Point(x, y)]:
print(f"Single point {x}, {y}")
case [Point(0, y1), Point(0, y2)]:
print(f"Two on the Y axis at {y1}, {y2}")
case _:
print("Something else")
def func(point):
match point:
case Point(x, y) if x == y:
print(f"Y=X at {x}")
case Point(x, y):
print(f"Not on the diagonal")
def func():
from enum import Enum
class Color(Enum):
RED = 'red'
GREEN = 'green'
BLUE = 'blue'
color = Color(input("Enter your choice of 'red', 'blue' or 'green': "))
match color:
case Color.RED:
print("I see red!")
case Color.GREEN:
print("Grass is green")
case Color.BLUE:
print("I'm feeling the blues :(")

View File

@@ -0,0 +1,21 @@
#import os
# from foo import junk
#a = 3
a = 4
#foo(1, 2, 3)
def foo(x, y, z):
content = 1 # print('hello')
print(x, y, z)
# This is a real comment.
# # This is a (nested) comment.
#return True
return False
#import os # noqa: ERA001
class A():
pass
# b = c

View File

@@ -152,15 +152,3 @@ def f(a: Union[str, bytes, Any]) -> None: ...
def f(a: Optional[Any]) -> None: ...
def f(a: Annotated[Any, ...]) -> None: ...
def f(a: "Union[str, bytes, Any]") -> None: ...
class Foo:
@decorator()
def __init__(self: "Foo", foo: int):
...
# Regression test for: https://github.com/astral-sh/ruff/issues/7711
class Class:
def __init__(self):
print(f"{self.attr=}")

View File

@@ -0,0 +1,31 @@
import os
import subprocess
import time
async def foo():
open("foo")
async def foo():
time.sleep(1)
async def foo():
subprocess.run("foo")
async def foo():
subprocess.call("foo")
async def foo():
subprocess.foo(0)
async def foo():
os.wait4(10)
async def foo():
os.wait(12)

View File

@@ -20,4 +20,3 @@ os.chmod(keyfile, stat.S_IRWXO | stat.S_IRWXG | stat.S_IRWXU) # Error
os.chmod("~/hidden_exec", stat.S_IXGRP) # Error
os.chmod("~/hidden_exec", stat.S_IXOTH) # OK
os.chmod("/etc/passwd", stat.S_IWOTH) # Error
os.chmod("/etc/passwd", 0o100000000) # Error

View File

@@ -8,7 +8,6 @@ def func(address):
# Error
"0.0.0.0"
'0.0.0.0'
f"0.0.0.0"
# Error

View File

@@ -0,0 +1,16 @@
# ok
with open("/abc/tmp", "w") as f:
f.write("def")
with open("/tmp/abc", "w") as f:
f.write("def")
with open("/var/tmp/123", "w") as f:
f.write("def")
with open("/dev/shm/unit/test", "w") as f:
f.write("def")
# not ok by config
with open("/foo/bar", "w") as f:
f.write("def")

View File

@@ -0,0 +1,31 @@
import json
import yaml
from yaml import CSafeLoader
from yaml import SafeLoader
from yaml import SafeLoader as NewSafeLoader
def test_yaml_load():
ystr = yaml.dump({"a": 1, "b": 2, "c": 3})
y = yaml.load(ystr)
yaml.dump(y)
try:
y = yaml.load(ystr, Loader=yaml.CSafeLoader)
except AttributeError:
# CSafeLoader only exists if you build yaml with LibYAML
y = yaml.load(ystr, Loader=yaml.SafeLoader)
def test_json_load():
# no issue should be found
j = json.load("{}")
yaml.load("{}", Loader=yaml.Loader)
# no issue should be found
yaml.load("{}", SafeLoader)
yaml.load("{}", yaml.SafeLoader)
yaml.load("{}", CSafeLoader)
yaml.load("{}", yaml.CSafeLoader)
yaml.load("{}", NewSafeLoader)

Some files were not shown because too many files have changed in this diff Show More