Compare commits

...

400 Commits

Author SHA1 Message Date
konsti
e87032d57d Explain check_docs_formatted.py error message (#6125)
## Summary

This is an error message only change to lead an implementor of a new
rule that has an unformatted or invalid bad example to the
right code.

## Test Plan

n/a
2023-07-27 15:53:52 +02:00
Charlie Marsh
0c2abf8316 Avoid walking past root when resolving imports (#6126)
## Summary

Noticed in #5954: we walk _past_ the root rather than stopping _at_ the
root when attempting to traverse along the parent path. It's effectively
an off-by-one bug.
2023-07-27 13:38:51 +00:00
konsti
6e85e0010f Fix windows test warnings (#6124)
See
https://github.com/astral-sh/ruff/actions/runs/5679922286/job/15392998698.
These didn't fail CI because we run clippy on linux only.
2023-07-27 14:03:53 +02:00
Micha Reiser
2ea0a0b28a Respect indent when measuring with MeasureMode::AllLines (#6120) 2023-07-27 13:34:36 +02:00
konsti
10abc8fd92 Unbreak main (#6123)
This fixes main breaking due to two merges.
2023-07-27 11:32:55 +00:00
konsti
61195bc7b1 Don't format trailing comma for lambda arguments (#5946)
**Summary** lambda arguments don't have parentheses, so they shouldn't
get a magic trailing comma either. This fixes some unstable formatting

**Test Plan** Added a regression test.

89 (from previously 145) instances of unstable formatting remaining.

```
$ cargo run --bin ruff_dev --release -- format-dev --stability-check --error-file formatter-ecosystem-errors.txt --multi-project target/checkouts > formatter-ecosystem-progress.txt
$ rg "Unstable formatting" target/formatter-ecosystem-errors.txt | wc -l
89
```

Closes #5892
2023-07-27 12:32:10 +02:00
Micha Reiser
9040554be5 Merge pull request #6119 from astral-sh/preserve-history 2023-07-27 12:12:57 +02:00
Micha Reiser
06ba1cf0ad Move RustPython Files 2023-07-27 11:46:18 +02:00
Micha Reiser
7a63a8f45a Merge branch 'main' of ../parser-move into preserve-history 2023-07-27 11:39:22 +02:00
Micha Reiser
c1bc0dc83d Delete RustPython files 2023-07-27 11:39:14 +02:00
Micha Reiser
90573975f3 Delete more unused files 2023-07-27 11:32:12 +02:00
Micha Reiser
802df97d55 Fix clippy issues 2023-07-27 11:31:10 +02:00
Micha Reiser
6f4ee32807 Move files to common directory 2023-07-27 11:27:37 +02:00
Zanie Blue
7ec979d022 Fix TypeAliasName to store name instead of load (#42)
Copies the fix in https://github.com/RustPython/Parser/pull/99
2023-07-26 17:53:34 -05:00
Micha Reiser
fab9cc5294 Remove parser ast re-export (#41) 2023-07-26 14:54:29 +02:00
Micha Reiser
ff7bab589e Prepare for moving the parser into the Ruff monorepo (#40) 2023-07-26 12:28:43 +02:00
Micha Reiser
593b46be5e perf: Cursor based lexer (#38) 2023-07-26 07:50:45 +02:00
David Szotten
13196fc500 fix the ranges of constants inside f-strings (#33) 2023-07-25 19:17:06 +02:00
Dhruv Manilawala
5ef4ccd632 Add line magic stmt and expr AST nodes (#31)
This PR introduces two new nodes for the parser to recognize the
`MagicCommand` tokens:

* `StmtLineMagic` for statement position i.e., magic commands on their
own line:
	```python
	%matplotlib inline
	!pwd
	```

* `ExprLineMagic` for expression position i.e., magic commands in an
assignment statement:
	```python
	# Only `?` and `!` are valid in this position
	dir = !pwd
	```

Both nodes are identical in their structure as in it contains the magic
kind and the command value as `String`.
2023-07-24 21:20:31 +05:30
Dhruv Manilawala
e363fb860e Lex Jupyter Magic in assignment value position (#30)
Emit `MagicCommand` token when it is the assignment value[^1] i.e., on
the right side of an assignment statement.

Examples:
```python
pwd = !pwd
foo = %timeit a = b
bar = %timeit a % 3
baz = %matplotlib \
        inline"
```

[^1]: Only `%` and `!` are valid in that position, other magic kinds are
not valid
2023-07-24 17:44:03 +05:30
Chris Pryer
4888d800fb Fix BoolOp python.org link (#37)
Noticed https://docs.python.org/3/library/ast.html#ast.boolop wasn't
working.
2023-07-22 14:10:59 -04:00
konsti
4d03b9b5b2 Fix empty lambda args range (#35)
Previously, empty lambda arguments (e.g. `lambda: 1`) would get the
range of the entire expression, which leads to incorrect comment
placement. Now empty lambda arguments get an empty range between the
`lambda` and the `:` tokens.
2023-07-21 17:41:19 +02:00
Zanie Blue
fb365c642b Implement Ranged for TypeParam (#32)
Required for https://github.com/astral-sh/ruff/pull/5927
2023-07-20 20:03:17 -04:00
konsti
db04fd4157 Refactor if statement, introduce nodes for elif and else (#22)
The old if layout couldn't differentiate between an else block with a
single if statement and an elif statement. Additionally we getting rid
of the recursion in favor of a single if struct with a vec of elif/else
children. This is accompanied by a big refactoring in ruff which removes
a bunch of TODOs and false negatives.
2023-07-18 13:14:40 +02:00
konstin
ff3e8ead36 Revert "snapshot"
This reverts commit 1bbec2b967.
2023-07-18 11:09:43 +02:00
konstin
1bbec2b967 snapshot 2023-07-18 11:08:14 +02:00
Dhruv Manilawala
510be51cfa Fix Jupyter Magic test snapshot (#29)
Sorry for the churn, I should've rebased and merged the original PR to
avoid this.
2023-07-17 23:06:53 -05:00
Dhruv Manilawala
3b4c8fffe5 Lex Jupyter line magic with Mode::Jupyter (#23)
Lex Jupyter line magic with `Mode::Jupyter`

This PR adds a new token `MagicCommand`[^1] which the lexer will
recognize when in `Mode::Jupyter`. The rules for the lexer is as
follows:
1. Given that we are at the start of line, skip the indentation and look
for [characters that represent the start of a magic
command](635815e8f1/IPython/core/inputtransformer2.py (L335-L346)),
determine the magic kind and capture all the characters following it as
the command string.
2. If the command extends multiple lines, the lexer will skip the line
continuation character (`\`) but only if it's followed by a newline
(`\n` or `\r`). The reason to skip this only in case of newline is
because they can occur in the command string which we should not skip:

	```rust
    //        Skip this backslash
    //        v
    //   !pwd \
    //      && ls -a | sed 's/^/\\    /'
    //                          ^^
    //                          Don't skip these backslashes
	```

3. The parser, when in `Mode::Jupyter`, will filter these tokens before
the parsing begins. There is a small caveat when the magic command is
indented. In the following example, when the parser filters out magic
command, it'll throw an indentation error:

	```python
	for i in range(5):
		!ls

	# What the parser will see
	for i in range(5):
	
	```

[^1]: I would prefer to have some other name as this not only represent
a line magic (`%`) but also shell command (`!`), help command (`?`) and
others. In original implementation, it's named as ["IPython
Syntax"](635815e8f1/IPython/core/inputtransformer2.py (L332))
2023-07-18 09:24:24 +05:30
Zanie
126652b684 Fix decorator ranges
Incorrectly merged LALRPOP file
2023-07-17 14:49:14 -05:00
Zanie
57e8712d76 Bump expected size of Stmt to 168 bytes 2023-07-17 14:49:14 -05:00
Zanie
78c6ede1c9 Format 2023-07-17 14:49:14 -05:00
Zanie Blue
a843a00f6b Add parsing of type alias statements i.e. the type keyword (#97)
Extends #95
Closes #82

Adds parsing of new `type` soft keyword for defining type aliases.

Supports type alias statements as defined in PEP 695 e.g.

```python
type IntOrStr = int | str

type ListOrSet[T] = list[T] | set[T]

type AnimalOrVegetable = Animal | "Vegetable"

type RecursiveList[T] = T | list[RecursiveList[T]]
```

All type parameter kinds are supported as in #95.

Builds on soft keyword abstractions introduced in https://github.com/RustPython/RustPython/pull/4519
2023-07-17 14:49:14 -05:00
Zanie Blue
f846f1ea42 Parse type parameters in function definitions (#96)
* Parse type parameters in function definitions
* Add test for combined items
2023-07-17 14:49:14 -05:00
Zanie
1d4b7a395f Consolidate tests and add coverage for trailing comma 2023-07-17 14:49:14 -05:00
Zanie
ce3ce0734b Add bound to test case test_parse_class_with_all_possible_generic_types 2023-07-17 14:49:14 -05:00
Zanie
b0e119f049 Add test for tuple bounds 2023-07-17 14:49:14 -05:00
Zanie
1f5e707829 Remove test for empty generic class Foo[]: ...
Not valid syntax
2023-07-17 14:49:14 -05:00
Zanie
ed7acfe477 Parse type parameters in class definitions 2023-07-17 14:49:14 -05:00
Zanie
c31b58eb39 Move type_param stubs into LALRPOP definition 2023-07-17 14:49:14 -05:00
Zanie
c0a3a20c63 Bump size assertion for Stmt from 136 to 160 bytes 2023-07-17 14:49:14 -05:00
Zanie
05ae26b935 Regenerate code with latest ASDL 2023-07-17 14:49:14 -05:00
David Szotten
b996b21ffc tuple constants are for optimisations, not source (#28)
my reading of https://docs.python.org/3/library/ast.html#ast.unparse and
https://discuss.python.org/t/ast-constant-value-tuple-s-and-frozenset-s/22578
is that tuple constants cannot come from parsing python source, they are
only for optimised bytecode

see also https://github.com/astral-sh/ruff/pull/5812
2023-07-17 15:46:37 -04:00
konsti
2d1f69cbb9 Remove asdl (#21)
This removes the ASDL code generation in favor of handwriting the AST. 

The motivations for moving away from the ASDL are:

* CPython compatibility is no longer a goal
* The ASDL grammar isn't as expressive as we would like
* The codegen scripts have a high complexity which makes extensions time
consuming
* We don't make heavy use of code generation (compared to e.g.
RustPython that generates Pyo3 bindings, a fold implementation etc).

We may want to revisit a grammar based code generation in the future,
e.g. by using [ungrammar](https://github.com/rust-analyzer/ungrammar)
2023-07-05 14:25:26 +02:00
konsti
0f2e295f3a impl Ranged for TextRange (#20)
This adds the missing implementation of `Ranged` for `TextRange` itself

```rust
impl Ranged for TextRange {
    fn range(&self) -> TextRange {
        *self
    }
}
```

This allows e.g. using `has_comments` with arbitrary ranges instead of
just a node.

It also adds .venv to the .gitignore
2023-07-02 10:11:06 +02:00
konstin
c174bbf1f2 impl<T> Ranged for &T where T: Ranged (#16)
In the example below, `arg` is `&Expr`, so `&Ranged`, but `entries()`
want a `T: Ranged`. This adds the missing bridge impl.

```rust
        let all_args = format_with(|f| {
            f.join_comma_separated()
                .entries(
                    // We have the parentheses from the call so the arguments never need any
                    args.iter()
                        .map(|arg| (arg, arg.format().with_options(Parenthesize::Never))),
                )
                .nodes(keywords.iter())
                .finish()
        });
```
2023-06-26 13:55:35 +02:00
Micha Reiser
8078663b6c Remove Range type parameter from AST nodes (#15) 2023-06-23 21:18:55 +01:00
Micha Reiser
f60e204b73 Fix range of keyword identifier (#14) 2023-06-22 10:00:25 +02:00
Micha Reiser
08ebbe40d7 Fix ArgWithDefault TextRange (#13) 2023-06-20 18:21:09 +02:00
Charlie Marsh
ed3b4eb72b Add TextRange to Identifier (#8)
## Summary

This PR adds `TextRange` to `Identifier`. Right now, the AST only
includes ranges for identifiers in certain cases (`Expr::Name`,
`Keyword`, etc.), namely when the identifier comprises an entire AST
node. In Ruff, we do additional ad-hoc lexing to extract identifiers
from source code.

One frequent example: given a function `def f(): ...`, we lex to find
the range of `f`, for use in diagnostics.

Another: `except ValueError as e`, for which the AST doesn't include a
range for `e`.

Note that, as an optimization, we avoid storing the `TextRange` for
`Expr::Name`, since it's already included.
2023-06-20 11:19:27 -04:00
Charlie Marsh
f0d200c8a1 Remove fold, unparse, and location features (#9) 2023-06-19 17:26:17 -04:00
Charlie Marsh
8d74eee750 Make malachite-bigint an optional dependency for rustpython-format (#12) 2023-06-19 16:31:25 -04:00
Charlie Marsh
21aa0b8d84 Optimize validate_arguments (#10) 2023-06-19 15:32:58 -04:00
Charlie Marsh
9cb00518e5 Update snapshot tests 2023-06-19 13:16:07 -04:00
Micha Reiser
6f65c5cba7 Add Decorator node (#7) 2023-06-19 12:53:05 -04:00
Micha Reiser
8a415fa61e Include argument parentheses in range (#5) 2023-06-19 12:51:18 -04:00
Micha Reiser
5054cbe84f Merge branch 'RustPython:main' into main 2023-06-19 12:38:24 -04:00
Jeong, YunWon
69d27d924c Rename unconventional nodes (#74) 2023-06-17 01:54:00 +09:00
Jeong, YunWon
5270020423 rustpython_ast python package (#79) 2023-06-17 01:32:27 +09:00
yt2b
edcfcb4a74 Fix bool format (#91)
* Fix format_bool

* Add test_format_bool
2023-06-15 14:56:03 +09:00
Jeong, YunWon
40a603208f Add Pylyzer
done by https://github.com/mtshiba/pylyzer/pull/37
2023-06-11 21:56:59 +09:00
Jeong, YunWon
b2f95e2848 Fix LinearLocator \r handling (#80) 2023-06-02 22:35:53 +09:00
Steve Shi
a2e3209c42 Replace num-bigint with malachite-bigint (#18)
Co-authored-by: Jeong YunWon <jeong@youknowone.org>
2023-06-02 17:06:18 +09:00
Jeong, YunWon
5e9e8a7589 Linear Locator (#46) 2023-06-01 13:53:31 +09:00
Jeong, YunWon
fdec727f80 New Arguments and Arg/ArgWithDefault AST representation (#59) 2023-06-01 01:15:23 +09:00
Jeong, YunWon
3fbf4f6804 Parse for expr and stmt variants + identifier, constant (#78) 2023-05-31 20:03:46 +09:00
Micha Reiser
fe25708d89 Merge pull request #76 from astral-sh/match-case-end-location 2023-05-31 09:48:42 +02:00
Micha Reiser
342cd19f50 Add safety comment 2023-05-31 09:37:48 +02:00
Micha Reiser
4a2c4aad0b Align MatchCase end location 2023-05-30 17:31:51 +02:00
Jeong, YunWon
ae3a477c97 Add parser deps to rustpython_ast_pyo3 (#75) 2023-05-29 16:48:05 +09:00
Jeong, YunWon
531aeb3511 Cmpop::as_str (#72)
* clean up pyo3 generation

* Cmpop::as_str
2023-05-29 01:53:54 +09:00
Jeong, YunWon
4de0cb1827 Parse Trait (#71) 2023-05-28 21:03:27 +09:00
Micha Reiser
5493c9f4e3 Avoid removing elements from the beginning of a vec (#69) 2023-05-25 19:44:49 +09:00
Jeong, YunWon
d23611db65 Merge pull request #65 from youknowone/refactor-pyo3
to_pyo3_ast to return &'py + Separate rustpython_ast_pyo3
2023-05-23 03:03:01 +09:00
Jeong YunWon
7d384d88d0 Separate rustpython_ast_pyo3 2023-05-23 02:50:12 +09:00
Jeong YunWon
b81273e9bc to_pyo3_ast to return &'py 2023-05-23 00:12:22 +09:00
Micha Reiser
335780aeea Merge pull request #2 from astral-sh/include-decorators-in-class-and-func-range 2023-05-22 13:46:38 +02:00
Jeong, YunWon
e1f02fced7 Fix Vec::to_pyo3_ast (#63) 2023-05-22 15:15:05 +09:00
Jeong, YunWon
d4084fb17a impl From<T> for ast::Ast (#62) 2023-05-21 22:44:07 +09:00
Micha Reiser
41a0ef8740 Add test 2023-05-19 10:14:46 +02:00
Micha Reiser
25cc1da319 Include decorators in Class and FunctionDef range 2023-05-19 08:18:43 +02:00
Micha Reiser
33a3c407a9 Merge pull request #60 from astral-sh/reduce-copy 2023-05-19 08:16:28 +02:00
Micha Reiser
fac0c25343 Update python.rs 2023-05-19 08:08:35 +02:00
Micha Reiser
4ff779c298 Update parser/src/python.lalrpop
Co-authored-by: Jeong, YunWon <69878+youknowone@users.noreply.github.com>
2023-05-18 22:32:05 +02:00
Micha Reiser
726884b287 Avoid allocating vecs for each statement 2023-05-18 21:56:17 +02:00
Micha Reiser
851f23668f Use named parameters to reducue copying 2023-05-18 21:41:06 +02:00
Jeong, YunWon
3654cf0bdf Add Ast as top level enum (#58) 2023-05-19 01:53:39 +09:00
Jeong, YunWon
6c5c311bab Fix range field order (#56)
* Skip validate_arguments when empty

* Fix `range` field order

* Fix unused variable
2023-05-18 21:44:01 +09:00
Jeong, YunWon
531e41ae2c Upgrade bitflags to remove clippy wanrings (#54)
* Upgrade bitflags to remove clippy wanrings

* Fix python lint warnings
2023-05-18 01:50:00 +09:00
Jeong, YunWon
b48834fe2d More flexible map_user and fold for new constructor nodes (#53)
* make fold.rs file

* Split user_map steps

* Fold for new constructor nodes
2023-05-18 00:16:04 +09:00
Jeong, YunWon
205ee80033 README (#47) 2023-05-17 18:01:00 +09:00
Jeong, YunWon
fc301ab1b0 Fix build (#45)
* remove dedent

* Revert parser-pyo3 crate
2023-05-16 23:54:49 +09:00
Jeong, YunWon
e820928f11 Add experimental pyo3-wrapper feature (#41)
* Fix pyo3 unit type value error

* Add experimental pyo3-wrapper feature

* location support
2023-05-16 23:45:31 +09:00
Jeong, YunWon
ff17f6e178 Add utilities to enum (#44)
* Add utilities to enum

* Fix unexpected pyo3 dependency propagation
2023-05-16 23:29:49 +09:00
Jeong YunWon
3bdf8a940a Hash for ConversionFlag 2023-05-16 22:56:17 +09:00
Jeong, YunWon
9d47d3d212 specialize ConversionFlag (#42)
* specialize ConversionFlag

* Change value of ConversionFlag to i8 and None to -1

* is_* methods to ConversionFlag
2023-05-16 22:52:50 +09:00
Jeong, YunWon
611dcc2e9b rustpython_ast + pyo3 (#25) 2023-05-16 18:06:54 +09:00
Jeong, YunWon
53de75efc3 Add all node classes to ast module (#40) 2023-05-16 16:55:26 +09:00
Jeong, YunWon
0c7d16b61a Add is_* methods to Tok (#39) 2023-05-16 16:08:25 +09:00
Jeong, YunWon
735c06d5f4 Fix full-lexer feature (#38) 2023-05-16 15:45:03 +09:00
Jeong, YunWon
02f13abf50 Add Node trait for node type information (#31) 2023-05-16 15:41:30 +09:00
Charlie Marsh
10dda125ff Always emit non-logical newlines for 'empty' lines (#27) 2023-05-15 14:13:05 -04:00
Jeong, YunWon
27e3873dc2 Add full-lexer feature (#36) 2023-05-16 02:21:34 +09:00
Micha Reiser
dd4cc25227 Reduce copying elements when parsing (#35) 2023-05-15 23:20:44 +09:00
Jeong, YunWon
b78001c953 Generic types to generic (#30) 2023-05-15 20:22:42 +09:00
Jeong, YunWon
718354673e Move range to node (#23)
* black + clippy

* Fix module generation
2023-05-15 16:20:22 +09:00
Micha Reiser
192379cede Move range from Attributed to Nodes (#22)
* Move `range` from `Attributed` to `Node`s

* No Attributed + custom for Range PoC

* Generate all located variants, generate enum implementations

* Implement `Copy` on simple enums

* Move `Suite` to `ranged` and `located`

* Update tests
---------

Co-authored-by: Jeong YunWon <jeong@youknowone.org>
2023-05-15 15:08:12 +09:00
Micha Reiser
a983f4383f Add format and cformat modules from RustPython (#24)
* Add `format` and `cformat` modules from `RustPython`

* Introduce `rustpython-format` crate

* Remove unused dependencies
2023-05-12 18:27:05 +09:00
Jeong, YunWon
947fb53d0b Field accessor and utilities (#20)
* Apply is-macro to Constant and ast nodes
2023-05-11 19:44:20 +09:00
Jeong, YunWon
2baad9ead6 Merge pull request #19 from RustPython/identifier
A few more Identifier utilities
2023-05-11 16:12:46 +09:00
Jeong YunWon
2af9805662 A few more Identifier utilities 2023-05-11 16:05:11 +09:00
Jeong, YunWon
5b2af304a2 Merge pull request #17 from youknowone/ruff
ruff integration support
2023-05-11 04:42:52 +09:00
Jeong YunWon
cbe4e8c5f3 Make parser location optional 2023-05-11 04:40:10 +09:00
Jeong YunWon
aabc96dde9 ruff integration support 2023-05-11 04:08:57 +09:00
Jeong YunWon
99e108dd53 *Escape::with_preferred_quote 2023-05-10 21:51:01 +09:00
Jeong YunWon
d6b6df5d1c ast::Int::to_usize 2023-05-10 21:31:02 +09:00
Jeong YunWon
822cac5aa0 parse_expression{=>_starts}_at 2023-05-10 21:25:58 +09:00
Jeong YunWon
aa101e4f26 fix ImportDots 2023-05-10 20:31:04 +09:00
Jeong, YunWon
cbc4cb286b Merge pull request #16 from youknowone/ast-int
Give identifier and int ast types
2023-05-10 20:19:01 +09:00
Jeong YunWon
6fa3d0f90a Fine-tune int types 2023-05-10 19:33:39 +09:00
Jeong YunWon
455bcc01a0 Give identifier and int ast types 2023-05-10 19:33:39 +09:00
Jeong, YunWon
d495cd9129 Merge pull request #15 from youknowone/spellchecker
Setup spell checker
2023-05-10 18:11:40 +09:00
Jeong YunWon
d8822d1091 spell check ast/asdl_rs.py 2023-05-10 18:05:39 +09:00
Jeong YunWon
75f6ce1ae5 Setup spell checker 2023-05-10 17:57:15 +09:00
Jeong, YunWon
41e9e7280a Merge pull request #12 from youknowone/visitor
Visitor
2023-05-10 17:52:25 +09:00
Jeong YunWon
17c8abcec1 asdl_rs.py to multiple file output 2023-05-10 17:41:01 +09:00
Jeong YunWon
e000b1c304 Remove redundant types 2023-05-10 17:15:01 +09:00
Jeong YunWon
243ca16b34 Fix visitor to fit in new structure 2023-05-10 17:14:44 +09:00
Joshua
4de9580b92 Generate a visitor trait to ast_gen.rs 2023-05-10 17:14:34 +09:00
Jeong, YunWon
5cf85f0b9d Merge pull request #13 from youknowone/locator
Python location transform and related refactoring
2023-05-10 17:04:30 +09:00
Jeong YunWon
4dc030ba9d Vendor SourceLocation from ruff 2023-05-10 17:00:12 +09:00
Jeong YunWon
1d366d52ab Let located only for python located stuff 2023-05-10 14:35:38 +09:00
Jeong YunWon
a3d9d8cb14 numerous refactoring
- Split parser core and compiler core. Fix #14
- AST int type to `u32`
- Updated asdl_rs.py and update_asdl.sh fix #6
- Use `ruff_python_ast::SourceLocation` for Python source location. Deleted our own Location.
- Renamed ast::Located to ast::Attributed to distinguish terms for TextSize and SourceLocation
- `ast::<Node>`s for TextSize located ast. `ast::located::<Node>` for Python source located ast.
- And also strictly renaming `located` to refer only python location related interfaces.
- `SourceLocator` to convert locations.
- New `source-code` features of to disable python locations when unnecessary.
- Also including fully merging https://github.com/astral-sh/RustPython/pull/4 closes #9
2023-05-10 14:35:38 +09:00
Jeong YunWon
09a6afdd04 Adapt SourceLocation 2023-05-09 20:34:48 +09:00
Jeong YunWon
a14e43e03a Separate byteoffset ast and located ast 2023-05-09 00:21:52 +09:00
Jeong YunWon
f47dfca4e3 Rename compiler Location to TextSize 2023-05-08 03:38:10 +09:00
Micha Reiser
58c35ab458 Replace row/column based Location with byte-offsets. 2023-05-08 03:38:10 +09:00
Jeong, YunWon
7b8844bd3e Merge pull request #11 from youknowone/refactor-asdl
Refactor ast to hold data as seperated type
2023-05-07 19:29:02 +09:00
Jeong YunWon
6d7358090b Refactor ast to hold data as seperated type 2023-05-07 19:20:47 +09:00
Jeong YunWon
9f1a538eba gitattribute 2023-05-07 17:20:52 +09:00
Jeong, YunWon
48920a034e Merge pull request #10 from youknowone/remove-compile-error
Remove CompileError
2023-05-06 17:20:44 +09:00
Jeong YunWon
13d6e275ef Remove CompileError 2023-05-06 17:15:18 +09:00
Jeong, YunWon
96eb80f5cf Merge pull request #8 from youknowone/python-lint
Add python lint
2023-05-06 14:42:15 +09:00
Jeong YunWon
a73bac9ed1 Add python lint 2023-05-06 14:34:31 +09:00
Jeong, YunWon
6b60f85cc4 Merge pull request #7 from youknowone/lalrpop
Embed generated parser + update lalrpop
2023-05-06 13:57:15 +09:00
Jeong YunWon
39b2dbe04d Update lalrpop to 0.20.0 2023-05-06 13:48:20 +09:00
Jeong YunWon
e1f70100ac Update parser/build.rs to embed python.rs 2023-05-06 05:28:30 +09:00
Jeong, YunWon
d66d935879 Merge pull request #5 from youknowone/ci
Add CI
2023-05-06 03:42:28 +09:00
Jeong YunWon
e28f333f23 Add CI 2023-05-06 03:39:20 +09:00
Jeong YunWon
0adcdd995d Remove unused workspace dependencies 2023-05-06 00:44:34 +09:00
Jeong YunWon
e6eb49ffb0 Set up workspace - Forked from
git@github.com:RustPython/RustPython.git ff5076b12c075b3e87c0ac2971e390b4a209d14f
2023-05-05 23:49:50 +09:00
Jeong YunWon
80109b1fe0 rustpython-literal 2023-05-05 21:49:12 +09:00
Jeong YunWon
bd64603950 Refactor common::bytes::repr using common::escape 2023-05-03 16:08:48 +09:00
Jeong YunWon
5b0e92d725 Refactor common::str::repr using common::escape 2023-05-03 13:47:04 +09:00
Micha Reiser
ae9d3c3193 Add Located::start, Located::end and impl Deref 2023-04-26 10:24:34 -06:00
Micha Reiser
3873414b30 Use Located::new over struct initializer 2023-04-25 18:10:13 -06:00
Jeong YunWon
a9f4b59f40 module objects' type as PyModule 2023-04-20 20:48:57 +09:00
Jeong YunWon
6d6155413b Update lalrpop 2023-03-25 22:18:02 +09:00
Jeong YunWon
f9b5469642 Update cspell for compiler 2023-03-16 22:39:09 +09:00
Charlie Marsh
ef38eb6b1a Include Derive feature with optional Serde dependency 2023-03-12 14:46:10 -04:00
Jeong YunWon
0ea53825db Merge pull request #4608 from coolreader18/bag-deser
Rework frozen modules and directly deserialize to CodeObject<Literal>
2023-03-10 05:19:23 +09:00
Noa
9d6ae774f8 Rework frozen modules and directly deserialize to CodeObject<Literal> 2023-03-06 13:45:33 -06:00
Charlie Marsh
55fc0e83f3 Treat match and case as soft keywords in lambda assignments (#4623) 2023-03-04 12:42:05 -05:00
Charlie Marsh
713dd2b91e Add optional serde dependency 2023-03-02 15:38:40 -05:00
Noa
969ea23d67 Address review comments 2023-03-01 21:11:59 -06:00
Noa
b80bbec8e6 Custom marshal enc/decoding impl 2023-03-01 20:47:21 -06:00
DimitrisJim
8f425e9ce2 Use insta to verify values. 2023-02-28 20:00:47 +09:00
Jeong YunWon
d7e2e7361e Use git version of unicode_names2 to avoid alias search failure 2023-02-28 20:00:47 +09:00
Charlie Marsh
c37e0c7f03 Use proper locations for sub-expressions in constant matches 2023-02-26 23:10:38 -05:00
Charlie Marsh
0d7b94817d Allow type variable tuple for *args 2023-02-23 08:39:59 -05:00
Jeong YunWon
f43e5b72e2 Merge pull request #4552 from charliermarsh/charlie/loc
Limit match range to end of last statement
2023-02-23 02:11:42 +09:00
Jeong YunWon
a19f294b0c Merge pull request #4543 from youknowone/flatten-parser
Flatten parser interface
2023-02-23 02:09:46 +09:00
Charlie Marsh
7ebef61c47 Limit match range to end of last statement 2023-02-22 11:25:50 -05:00
Charlie Marsh
b61f4d7b69 Allow trailing commas in MappingPattern 2023-02-22 10:02:41 -05:00
Jeong YunWon
e26369a34e use super::* from tests submodules 2023-02-22 21:01:39 +09:00
Jeong YunWon
97a08ee77b remove #[macro_use] 2023-02-22 20:41:27 +09:00
Jeong YunWon
cb8c6fb78d Flatten rustpython_parser interface 2023-02-22 20:32:31 +09:00
Jeong YunWon
8580e4ebb5 make_tokenizer -> lex to integrate terms
we don't distinguish scanner or tokenizer from lexer
2023-02-22 20:28:15 +09:00
Jeong YunWon
39fc23cf92 relocate feature-independent use 2023-02-22 20:28:03 +09:00
Jeong YunWon
66e3080173 Fix ModeParseError message 2023-02-22 20:28:03 +09:00
Jeong YunWon
1511b6631b Break down rustpython_parser::error module
because it doesn't share any common errors but specific error for each sub module
2023-02-22 20:28:01 +09:00
Charlie Marsh
2a8aa6f308 Always wrap in SoftKeywordTransformer 2023-02-21 19:18:42 -05:00
Charlie Marsh
dc628cab8f Expose SoftKeywordTransformer on public API 2023-02-21 19:00:32 -05:00
Jim Fasarakis-Hilliard
c137bc9d77 Merge pull request #4519 from charliermarsh/charlie/match
Add support for match statements to parser
2023-02-21 19:43:28 +02:00
Jeong YunWon
60180fd54c Merge pull request #4531 from charliermarsh/charlie/exception-groups
Implement except* syntax
2023-02-21 13:20:18 +09:00
Charlie Marsh
c7ed645cc6 Implement except* syntax 2023-02-21 12:19:54 +09:00
Charlie Marsh
8caa28f0f8 Update Python.asdl from CPython 3.11.1 2023-02-21 12:19:51 +09:00
Charlie Marsh
8aa3bc93f3 Allow starred expressions in subscripts 2023-02-20 17:59:35 -05:00
Charlie Marsh
f39ffef370 Update compiler/parser/src/soft_keywords.rs
Co-authored-by: Jim Fasarakis-Hilliard <d.f.hilliard@gmail.com>
2023-02-20 15:03:39 -05:00
Jeong YunWon
e093d2bee6 clean up soft-keyword transform 2023-02-20 15:03:39 -05:00
Charlie Marsh
ca5b474d45 Use muiltipeek 2023-02-20 15:03:39 -05:00
Charlie Marsh
2b43d45bd5 Add support for match statements to parser 2023-02-20 15:03:39 -05:00
John Pham
4bdc2d47c1 Make common::repr throw error instead of panic (#4520) 2023-02-19 22:09:54 +09:00
Jeong YunWon
09b82e41d5 Merge pull request #4490 from DimitrisJim/function_parser
Add tests, some comments, to function.rs.
2023-02-13 20:00:35 +09:00
Jeong YunWon
cc6d8a1c58 Merge pull request #4492 from DimitrisJim/doc_parser_uno
Document parser crate.
2023-02-13 17:25:14 +09:00
Dimitris Fasarakis Hilliard
07918f0a9a Document parser crate. 2023-02-12 17:58:19 +02:00
Dimitris Fasarakis Hilliard
a0786ea872 Add tests, some comments, to function.rs. 2023-02-11 23:07:57 +02:00
Dimitris Fasarakis Hilliard
4713b2b3ab Refactor: Join string and string_parser. 2023-02-11 18:05:06 +02:00
Jeong YunWon
e7f14ab9b8 Add test_generator_expression_argument 2023-02-11 05:20:39 +09:00
Charlie Marsh
56c73cc63d Use entire range for generators-as-arguments 2023-02-10 10:39:40 -05:00
Dimitris Fasarakis Hilliard
659f4dd8bf Document lexer. 2023-02-07 21:43:57 +02:00
Dimitris Fasarakis Hilliard
bd158089e0 Move NewLineHandler inline, don't check each character twice. 2023-02-07 20:58:53 +02:00
Jeong YunWon
a73bee7aae use workspace dependencies 2023-02-06 15:30:38 +09:00
Dimitris Fasarakis Hilliard
1468fe46ab Hint that the unwrap should always succeed. 2023-02-01 12:17:33 +02:00
Dimitris Fasarakis Hilliard
c9364718b4 Eat for comma. 2023-01-31 12:26:05 +02:00
Dimitris Fasarakis Hilliard
38cf933bcb Add initial capacities, use u32s for indents/spaces. 2023-01-31 12:26:05 +02:00
Dimitris Fasarakis Hilliard
838990ae15 Don't call is_emoji_presentation for each invocation of consume_normal 2023-01-31 12:26:05 +02:00
Dimitris Fasarakis Hilliard
aa0290bbfc Match on ascii start/continuation characters before calling functions. 2023-01-31 12:26:05 +02:00
Aarni Koskela
f74e44d1e8 Bump phf to 0.11 series
string_cache is still using phf_shared 0.10.0 though
2023-01-25 19:58:43 +02:00
Jeong YunWon
d9df131720 Merge pull request #4449 from harupy/fix-dict-spread-in-dict
Fix AST generated from a dict literal containing dict unpacking
2023-01-22 20:44:26 +09:00
harupy
f2ffe12f8c Fix comment 2023-01-22 00:06:52 +09:00
Anders Kaseorg
6dba8430be Fix end location for elif blocks
Since we parse an `elif:` block as an `If` node, its location should
include its `orelse` node like it would for an `if:` block.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2023-01-17 22:39:44 -05:00
harupy
b26365b215 Remove useless String::from
Signed-off-by: harupy <hkawamura0130@gmail.com>
2023-01-16 21:27:57 +09:00
harupy
2d019930e9 Rename test 2023-01-15 23:36:07 +09:00
harupy
d5fc7c4c87 Improve test 2023-01-15 16:53:13 +09:00
harupy
393869c47c Add Option to Dict.keys field 2023-01-15 16:43:13 +09:00
harupy
4edd2bf78a Remove commented-out code 2023-01-15 16:10:51 +09:00
harupy
d3c4551629 Fix unparse 2023-01-15 13:11:55 +09:00
harupy
581f6e176c Fix dict spreading in dict literal 2023-01-15 13:01:59 +09:00
Jim Fasarakis-Hilliard
ac4d3c076c Merge pull request #4443 from bluetech/non-logical-newline-token-fixup
Fixup parse_tokens after "Add NonLogicalNewline token"
2023-01-14 12:49:04 +02:00
Ran Benita
e5fe037e38 Fixup parse_tokens after "Add NonLogicalNewline token"
I only updated `parse()` to ignore `NonLogicalNewline`, didn't notice
it's also needed in `parse_tokens()`.
2023-01-14 11:52:33 +02:00
Noa
872b9d4765 Switch from 64-bit instruction enum to out-of-line arg values 2023-01-12 23:05:17 -06:00
Ran Benita
674eeec29c Add NonLogicalNewline token
This token is completely ignored by the parser, but it's useful for
other users of the lexer, such as the Ruff linter. For example, the
token is helpful for a "trailing comma" lint.

The same idea exists in Python's `tokenize` module - there is a NEWLINE
token (logical newline), and a NL token (non-logical newline).

Fixes #4385.
2023-01-12 16:47:12 +02:00
Martin Fischer
4f1e7c6291 Fix docs.rs build for rustpython-parser
docs.rs failed to build the documentation of the recently released
rustpython-parser 0.2.0 because the build.rs script couldn't write the
parser.rs file because docs.rs builds the documentation in a sandbox
with a read-only filesystem.

This commit fixes this by writing the parser.rs file to the cargo output
directory instead, as recommended by the docs.rs documentation.[1]

Fixes #4436.

[1]: https://docs.rs/about/builds#read-only-directories
2023-01-11 09:58:10 +01:00
Noa
884a7bdb15 Bump all crate versions to 0.2.0 2023-01-11 00:14:28 -06:00
Jeong YunWon
7885344bcf first cspell dict 2023-01-09 19:57:23 +09:00
Jeong YunWon
e8fef39861 Merge pull request #4429 from youknowone/fix-format
Fix nightly clippy warnings
2023-01-07 21:38:41 +09:00
Jeong YunWon
509cf7ed0d Fix nightly clippy warnings 2023-01-07 21:07:10 +09:00
harupy
a4a5366504 Include comment text in token 2023-01-06 23:29:20 +09:00
harupy
2dfd053bed Implement Default for Location 2023-01-05 22:48:47 +09:00
harupy
7f552e4594 Address comments 2023-01-05 18:24:54 +09:00
harupy
9efa872023 Use try_from 2023-01-05 01:18:30 +09:00
harupy
fd8468c5eb Simplify string check 2023-01-04 23:57:50 +09:00
Jeong YunWon
958c7e33ad Merge pull request #4417 from harupy/add-with-offset-methods
Add `with_col_offset` and `with_row_offset` to `Location` for conveniece
2023-01-04 17:07:31 +09:00
Jeong YunWon
0f311cd5e5 Merge pull request #4413 from harupy/more-generic-window-impl
Update `CharWindow` in `compiler/parser/src/lexer.rs` to allow slicing
2023-01-04 17:06:06 +09:00
harupy
0365752bf3 Use Self 2023-01-04 13:19:23 +09:00
harupy
6d140426c1 Add with_col_offset and with_row_offset to Location 2023-01-04 13:16:03 +09:00
harupy
33a62789f7 Address comment 2023-01-04 00:03:18 +09:00
harupy
84dff79ddc Remove incorrect EmptyExpression in parse_formatted_value 2023-01-03 23:55:50 +09:00
harupy
300710f7db Improve CharWindow 2023-01-03 17:27:35 +09:00
harupy
515dceb07b Remove repetitive to_string in parse_escaped_char 2023-01-03 14:48:00 +09:00
Jeong YunWon
2858c315bf Merge pull request #4409 from harupy/improve-error-conversion-in-string-parser
Improve error conversion in `string_parsers.rs`
2023-01-03 14:39:01 +09:00
harupy
3b0fd61b3b Fix clippy error 2023-01-03 12:51:31 +09:00
harupy
9030679193 Improve error conversion in string_parsers.rs 2023-01-03 12:46:10 +09:00
Jeong YunWon
71ba4226c1 Merge pull request #4405 from harupy/use-drain
Use `drain` to simplify `compiler/parser/src/string_parser.rs`
2023-01-03 12:41:23 +09:00
Jeong YunWon
3d03439618 Merge pull request #4399 from branai/shell-continuing-fix
Fix IndentationError works differently with cpython in interective shell
2023-01-03 04:32:03 +09:00
harupy
39f410c909 Use drain 2023-01-03 01:23:44 +09:00
Jim Fasarakis-Hilliard
4222b13e6c Merge pull request #4404 from harupy/merge-match-arms
Merge match arms in `StringParser.parse_formatted_value`
2023-01-02 18:18:40 +02:00
Jim Fasarakis-Hilliard
d691527ead Merge pull request #4402 from harupy/remove-unreachable-if
Remove unreachable code in `compiler/parser/src/string_parser.rs`
2023-01-02 17:05:45 +02:00
harupy
40e2150f7e Merge match arms in parse_formatted_value 2023-01-02 23:16:51 +09:00
harupy
92bf96608c Fix match 2023-01-02 22:54:48 +09:00
harupy
fac6a857f6 Simplify code using match 2023-01-02 22:26:09 +09:00
harupy
68586f8e3c Remove unreachable code in compiler/parser/src/string_parser.rs 2023-01-02 20:48:40 +09:00
Jeong YunWon
f8787a9377 Move (c)format basic implementations to rustpython-common 2023-01-02 20:21:36 +09:00
Bijan Naimi
530a20cc96 forgot to add formatted errors.rs 2023-01-01 17:28:49 -08:00
Bijan Naimi
6cb57b2075 changed the shell logic for handling indents 2023-01-01 15:41:51 -08:00
Dimitris Fasarakis Hilliard
92b2574d52 Move tests for with into parser. 2023-01-01 21:36:07 +02:00
Jim Fasarakis-Hilliard
95fb938bd6 Merge pull request #4389 from harupy/4384-follow-up
Follow-up for #4384
2023-01-01 14:53:54 +02:00
harupy
9683314264 Remove unreachable code 2023-01-01 17:43:25 +09:00
harupy
d9bbeeb9b3 Fix NamedExpr location 2022-12-31 23:32:08 +09:00
harupy
68116d5c11 Move tests 2022-12-31 12:15:33 +09:00
Jeong YunWon
e164a41723 Merge pull request #4373 from andersk/named
Allow named expression in subscript and set comprehension
2022-12-31 10:58:09 +09:00
Jeong YunWon
aa32a73c5b Merge pull request #4379 from harupy/refactor-FStringParser
Refactor `FStringParser`
2022-12-31 10:56:52 +09:00
Jeong YunWon
9d7d629cef Merge pull request #4384 from harupy/parse-formatted-value
Fix the location of `FormattedValue`
2022-12-31 10:52:44 +09:00
harupy
439298e735 Fix FormattedValue location 2022-12-30 21:39:29 +09:00
anilbey
16173bf581 Update compiler/parser/src/error.rs
Co-authored-by: fanninpm <fanninpm@miamioh.edu>
2022-12-29 22:56:34 +01:00
Anil Tuncel
7e7b2eadee format using cargo fmt 2022-12-29 22:15:44 +01:00
Anil Tuncel
80116c768d arg name to be written upon duplicate kwargs error #4381 2022-12-29 22:06:41 +01:00
Nick Liu
200390c1ab format code 2022-12-29 22:49:26 +08:00
Nick Liu
24d2ab8b0a use is_none 2022-12-29 22:49:26 +08:00
Nick Liu
41f21a7b5d add arg_name in duplicate argument error msg 2022-12-29 22:49:26 +08:00
Nick Liu
63e4a36e27 added check: named arguments must follow bare star 2022-12-29 22:49:26 +08:00
Nick Liu
15ab44384c added lex error: DuplicateArguments 2022-12-29 22:49:26 +08:00
Jim Fasarakis-Hilliard
5380b85579 Merge pull request #4367 from andersk/star-order
Prohibit starred arguments after double-starred arguments
2022-12-29 16:27:16 +02:00
Harutaka Kawamura
b707f53f23 Update compiler/parser/src/fstring.rs
Co-authored-by: Zgarbul Andrey <zgarbul.andrey@gmail.com>
2022-12-29 08:10:33 +09:00
Anders Kaseorg
6439867f78 Allow named expression in set comprehension: {a := b for c in d}
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-12-28 09:57:32 -08:00
Anders Kaseorg
1904d095f9 Allow named expression in subscript: a[b := c]
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-12-28 09:57:32 -08:00
harupy
7e8f683808 Rename 2022-12-29 01:01:41 +09:00
harupy
37b1894834 Clean up FStringParser 2022-12-29 00:08:59 +09:00
Jim Fasarakis-Hilliard
201d08583a Merge pull request #4377 from andersk/duplicate-from
Remove duplicate declaration of "from" token
2022-12-28 11:19:54 +02:00
Anders Kaseorg
107b2e11ae Remove duplicate declaration of "from" token
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-12-27 18:56:24 -08:00
Anders Kaseorg
661b210391 Prohibit starred arguments after double-starred arguments
CPython prohibits ‘f(**kwargs, *args)’; we should too.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-12-27 12:49:50 -08:00
harupy
9aed9143fe Refactor parse_formatted_value 2022-12-28 00:23:25 +09:00
harupy
c21d0d9283 Fix make_tokenizer_located 2022-12-27 21:54:18 +09:00
Jeong YunWon
313fd7d28c Merge pull request #4359 from yt2b/check_bom
Add BOM check
2022-12-26 16:03:51 +09:00
Jeong YunWon
3aa0096212 Merge pull request #4358 from harupy/fix-slice-location
Fix `Slice` location
2022-12-26 16:03:09 +09:00
Jeong YunWon
53dec88029 Merge pull request #4356 from andersk/with-tuple-named
Fix parsing of tuple with named expression as context manager
2022-12-26 16:02:17 +09:00
yt2b
bd0c15d34e Fix comment 2022-12-26 09:30:12 +09:00
yt2b
ce0be73841 Add BOM check 2022-12-25 11:15:29 +09:00
harupy
b2ac4f60f1 Fix slice location 2022-12-25 09:37:07 +09:00
Anders Kaseorg
1fdfa5fe1b Fix parsing of tuple with named expression as context manager
Because the ‘with’ item grammar disallows named expressions, CPython
parses ‘with (a := 0, b := 1):’ as a tuple rather than two ‘with’
items.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-12-24 13:15:53 -08:00
harupy
c0f390ebc6 Fix IfExp location 2022-12-25 00:33:02 +09:00
Anders Kaseorg
c387b5d523 Simplify parenthesized context manager parsing with LALRPOP conditions
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-12-24 23:13:10 +09:00
harupy
ea95e1a715 Fix the location of BinOp 2022-12-21 22:05:05 +09:00
Jim Fasarakis-Hilliard
c16e08d59b Merge pull request #4340 from harupy/fix-locations-of-parethesized-expressions
Fix the start and end locations of `Tuple`
2022-12-18 15:17:40 +02:00
harupy
ff00460ff4 Fix locations of parethesized expressions 2022-12-18 20:53:30 +09:00
Anders Kaseorg
f4672e4256 Remove unnecessary boxing of ASDL product children
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-12-14 02:30:02 -08:00
Dimitris Fasarakis Hilliard
6e89b3ab1a Fix end location in with statements. 2022-12-13 13:28:17 +02:00
Jim Fasarakis-Hilliard
6a174dae45 Merge pull request #4327 from harupy/fix-end-location-body
Fix end location of compound statements
2022-12-13 12:30:21 +02:00
Charlie Marsh
8936ab2f8d Set ExprContext::Store on parenthesized with expressions 2022-12-12 09:09:15 -05:00
harupy
165b979733 Refactor 2022-12-12 22:36:34 +09:00
harupy
06c92bb899 Update snapshot 2022-12-12 22:26:03 +09:00
harupy
1a657731dd Format 2022-12-12 22:18:26 +09:00
harupy
d8cfc7e84f Resolve conflict 2022-12-12 22:16:46 +09:00
harupy
de2e88656e Address comments
Signed-off-by: harupy <hkawamura0130@gmail.com>
2022-12-12 22:14:05 +09:00
Anders Kaseorg
052dee72b8 Parse Python 3.9+ parenthesized context managers
Since the upstream grammar for this is not LR(1), we abuse LALRPOP
macros and the Into/TryInto traits to build a cover grammar that
converts to either tuples or `with` items after additional validation.
It’s annoying and ugly, but something like this is basically our only
option short of switching to a more powerful parser algorithm.

Fixes #4145.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-12-12 00:47:33 -08:00
Anders Kaseorg
751f9d304f Split and simplify some LALRPOP rules
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-12-11 22:02:08 -08:00
harupy
d6f9dd0763 Use method chaining 2022-12-12 10:24:00 +09:00
harupy
5fc8c4d0b1 Fix other compound statements 2022-12-12 01:10:42 +09:00
harupy
1b7a272b77 Fix end location of nodes containing body 2022-12-11 12:35:28 +09:00
harupy
3abdc87076 Refactor 2022-12-10 22:01:42 +09:00
harupy
99b02be35a Fix 2022-12-10 18:45:36 +09:00
harupy
f99167d4ed Fix plain string 2022-12-10 18:09:26 +09:00
harupy
816e1e711c Fix the end location of an implicitly-concatenated string 2022-12-10 17:49:57 +09:00
Jeong YunWon
e2b28d07c8 Merge pull request #4310 from youknowone/fix-clippy
Fix nightly clippy warnings
2022-12-05 12:57:01 +09:00
Jeong YunWon
28785784b2 Fix nightly clippy warnings 2022-12-05 12:18:16 +09:00
harupy
6f6b7b2312 add tests 2022-12-04 05:59:51 +09:00
harupy
93cb05ebda Fix location 2022-12-04 05:45:15 +09:00
yt2b
e8200ab674 use bool.then 2022-11-24 09:23:20 +09:00
yt2b
492f09298f restore if expression 2022-11-23 11:30:54 +09:00
yt2b
35eea0b8ec Refactor lexer functions 2022-11-22 21:10:19 +09:00
yt2b
1eeddb521e Add test 2022-11-21 22:36:05 +09:00
Charlie Marsh
96a50810a6 Use match 2022-11-20 23:41:52 -05:00
Charlie Marsh
51b7dbb89c Use rustc-hash 2022-11-20 15:30:19 -05:00
Charlie Marsh
28a8c3a062 Implement some minor performance optimizations 2022-11-20 13:33:20 -05:00
Bongjun Jang
4e5626dfd5 Refactor lexer struct (#4257) 2022-11-19 21:43:34 +09:00
Jeong YunWon
f979d8dbc3 Apply let-else statements 2022-11-10 15:39:31 +09:00
Jim Fasarakis-Hilliard
cc084b4fec Merge pull request #4266 from charliermarsh/charlie/comments
Implement Tok::Comment
2022-11-07 23:20:15 +02:00
Charlie Marsh
bbeec36fdb Set comparator start location to beginning of comparison 2022-11-07 12:24:14 -05:00
Charlie Marsh
b6c230f3ca Implement Tok::Comment 2022-11-07 10:33:55 -05:00
dvermd
a5b59f3c9d improve col_offset in new line and lalr 2022-10-26 21:30:51 +02:00
dvermd
d5a208ca9d improve fstring parser
part of: #1671
2022-10-26 21:15:24 +02:00
dvermd
fa41a1e2f6 Fix ast types' _fields and use 0-based column 2022-10-27 03:09:38 +09:00
Charlie Marsh
952d70b9d1 Add expression context parsing 2022-10-17 15:20:33 -04:00
Charlie Marsh
02953b9fe6 Remove parse_program_tokens 2022-10-17 12:04:30 -04:00
Charlie Marsh
8adc74fe26 Expose a method to parse AST from tokens directly 2022-10-17 09:39:48 -04:00
Jeong YunWon
48c0cb5599 Merge pull request #4218 from charliermarsh/charlie/clone
Make AST nodes Clone-able
2022-10-17 13:47:21 +09:00
Jeong YunWon
2e33a3d0e9 Merge pull request #4223 from youknowone/nightly-clippy
Fix nightly clippy warnings
2022-10-17 13:28:53 +09:00
Charlie Marsh
1cc342e4ed Add end locations to all nodes (#4192) 2022-10-17 13:18:30 +09:00
Charlie Marsh
519718e65d Start simple string at quote mark 2022-10-16 11:25:46 -04:00
Charlie Marsh
5da5490b19 Make AST nodes Clone-able 2022-10-16 11:01:17 -04:00
Jeong YunWon
518cf728c3 Fix nightly clippy warnings 2022-10-16 02:38:50 +09:00
Charlie Marsh
3397737a76 Start string location at kind or quote prefix 2022-10-15 11:03:50 -04:00
dvermd
6211a3a3a8 Refactor fstrings (#4188) 2022-10-14 12:16:34 +09:00
fanninpm
e64d7d196d Merge pull request #4186 from andersk/arithmetic
Spell “arithmetic” correctly
2022-09-30 09:29:36 -04:00
Anders Kaseorg
f8b45e48e1 Spell “arithmetic” correctly
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-09-28 17:58:30 -07:00
Jeong YunWon
f99e4789ed temporary fix of parser build 2022-09-21 16:49:35 +09:00
Jeong YunWon
86b847204e better parser build experience 2022-09-21 16:47:06 +09:00
Charlie Marsh
26b529f9dc Add PartialOrd to Location 2022-09-04 16:31:43 -04:00
Jeong YunWon
b21ed24025 Merge pull request #4116 from charliermarsh/charlie/f-string
Avoid creating unused JoinedStr in FStringParser
2022-08-23 19:30:36 +09:00
Jeong YunWon
a414677892 remove Mode from codegen root 2022-08-23 05:15:27 +09:00
Jeong YunWon
bfe4795b6c parser::Mode from compile::Mode 2022-08-23 05:08:24 +09:00
Jeong YunWon
40690b9761 use thiserror 2022-08-23 05:08:24 +09:00
Jeong YunWon
1fd898c14c new_syntax_error working without full compiler
Fix #4100
2022-08-23 05:08:24 +09:00
Jeong YunWon
42b95a9a95 Move out CompileError to core as generic form 2022-08-23 01:30:00 +09:00
Jeong YunWon
e8230efe1a Integrate CompileError to compiler-core::BaseError 2022-08-22 23:18:41 +09:00
Jeong YunWon
7fcc18daea integrate CodegenError to compiler-core::Error 2022-08-22 18:43:03 +09:00
Jeong YunWon
2b7bf79d29 Integrate ParseError to compiler-core::Error 2022-08-22 16:28:08 +09:00
Jeong YunWon
904fc477f1 integrate ast::Location into compilre-core::Location 2022-08-22 08:42:20 +09:00
Jeong YunWon
bfac0355dc Share location between compiler crates 2022-08-22 08:42:20 +09:00
Jeong YunWon
c16e650071 rustpython-bytecode -> rustpython-compiler-core 2022-08-22 08:42:20 +09:00
Jeong YunWon
acde8bb625 sort Cargo.toml dependencies 2022-08-22 08:42:20 +09:00
Jeong YunWon
7f99404618 clean up cargo.toml sort 2022-08-22 08:42:20 +09:00
Jeong YunWon
3742f9117b Add source_path to ParseError 2022-08-22 08:42:20 +09:00
Jeong YunWon
a66902406f Refactor Mode and partial parser/codegen for eval/exec 2022-08-22 08:42:20 +09:00
Charles Marsh
51b6571ee1 Fix f-string regression 2022-08-21 19:20:23 -04:00
Charles Marsh
2345bc895d Avoid creating unused JoinedStr in FStringParser 2022-08-21 16:59:36 -04:00
Jeong YunWon
53c48bf6b9 reorganize compiler crates 2022-08-22 04:42:42 +09:00
Jeong YunWon
ffacac05bb Clean up imports 2022-08-22 03:42:29 +09:00
Jeong YunWon
3bae0823f7 replace try_parse!() macro to closure 2022-08-12 08:35:34 +09:00
Jeong Yunwon
29644a30d7 Add compile::Mode::BlockExpr 2022-05-27 10:14:13 +09:00
Jeong Yunwon
07712eda58 Upgrade libraries 2022-04-18 00:09:22 +09:00
Noa
76728bb69d Run cargo upgrade 2021-10-30 19:42:35 -05:00
Noa
721c2709c8 Migrate to 2021 edition 2021-10-21 11:46:24 -05:00
Noa
7865cace06 Add unparse feature to ast 2021-10-17 21:55:27 -05:00
Noah
a9330765fa Add an ast optimizer 2021-04-05 09:46:30 -05:00
Noah
9941eef853 Upgrade dependencies 2021-03-25 08:06:56 -05:00
Noah
e494175cd2 Update the compiler to use the new asdl ast types 2021-01-14 12:37:37 -06:00
Noah
dd7560c494 Implement asdl in the ast and parser 2021-01-14 12:37:37 -06:00
Noah
6603b4a124 Use ahash/phf in parser 2021-01-08 17:09:09 -06:00
Noah
dab7e20a33 Use ahash in the compiler 2021-01-08 17:00:23 -06:00
Noah
13f3ec5b2b Update other stuff to use the root of rustpython_bytecode 2020-12-19 14:59:07 -06:00
Noah
452cd308d5 Optimize the size of Instruction 2020-12-14 14:25:19 -06:00
Noah
3c469eaa26 compiler/porcelain wrapper 2020-11-07 15:43:23 -06:00
Noah
0c5bf8e470 Split the ast from the parser, remove compiler dep on parser 2020-11-07 15:43:23 -06:00
Noah
3f52fdd50a Use disassembly for snapshot testing 2020-10-19 23:55:56 -05:00
Noah
f0c0897f23 Update compiler to output codeobj.constants, use insta for snapshot testing 2020-10-19 23:55:55 -05:00
Jeong YunWon
a35d4dc22a clean up PyComplex 2020-08-09 21:46:16 +09:00
Noah
5fe1bd3900 Update itertools 2020-07-26 15:21:39 -05:00
Noah
3737af157f Release 0.1.2
rustpython@0.1.2
rustpython-bytecode@0.1.2
rustpython-compiler@0.1.2
rustpython-derive@0.1.2
rustpython-parser@0.1.2
rustpython-vm@0.1.2
rustpython_freeze@0.1.2
rustpython_wasm@0.1.2

Generated by cargo-workspaces
2020-06-23 18:47:08 -05:00
Chris West (Faux)
ed01cd63ee upgrade low-risk deps 2019-10-10 21:07:26 +01:00
Seo Sanghyeon
1dd57971a9 Optimize BuildMap bytecode emission 2019-10-03 00:03:08 +09:00
Noah
d1ae5a7448 Bump crate versions 2019-09-25 11:57:38 -05:00
coolreader18
e89d6febea Make peephole optimizer a stream processor 2019-08-03 22:02:29 -05:00
Windel Bouwman
4dcad05aa0 Extend symtable module. 2019-07-19 22:05:35 +02:00
Windel Bouwman
b0d270b881 Set all versions to 0.1.0 2019-07-07 13:04:12 +02:00
Windel Bouwman
e3ccef2504 Change authors to team name. 2019-07-01 21:14:07 +02:00
Windel Bouwman
dcdd964e0c Change underscore into hyphen 2019-07-01 21:05:29 +02:00
Windel Bouwman
107442cc06 Make bytecode crate independent of parser crate. 2019-06-30 11:42:36 +02:00
Windel Bouwman
10957879db Move bytecode into own crate. 2019-06-30 11:01:40 +02:00
coolreader18
f0148f46d0 Split off bytecode compilation into a separate crate 2019-06-12 21:43:43 -05:00
128 changed files with 379 additions and 227 deletions

View File

@@ -21,7 +21,9 @@ use path_absolutize::path_dedot;
#[cfg(unix)]
use tempfile::TempDir;
#[cfg(unix)]
use ruff_cli::args::Args;
#[cfg(unix)]
use ruff_cli::run;
const BIN_NAME: &str = "ruff";

View File

@@ -116,7 +116,6 @@ impl<'a> Printer<'a> {
self.print_str("\n");
}
self.state.pending_space = false;
self.state.pending_indent = args.indention();
}
}
@@ -744,7 +743,6 @@ struct PrinterState<'a> {
source_markers: Vec<SourceMarker>,
source_position: TextSize,
pending_indent: Indention,
pending_space: bool,
measured_group_fits: bool,
generated_line: usize,
generated_column: usize,
@@ -1009,12 +1007,12 @@ impl<'a, 'print> FitsMeasurer<'a, 'print> {
let args = self.stack.top();
match element {
FormatElement::Space => return Ok(self.fits_text(" ")),
FormatElement::Space => return Ok(self.fits_text(" ", args)),
FormatElement::Line(line_mode) => {
match args.mode() {
PrintMode::Flat => match line_mode {
LineMode::SoftOrSpace => return Ok(self.fits_text(" ")),
LineMode::SoftOrSpace => return Ok(self.fits_text(" ", args)),
LineMode::Soft => {}
LineMode::Hard | LineMode::Empty => {
return Ok(if self.must_be_flat {
@@ -1036,17 +1034,18 @@ impl<'a, 'print> FitsMeasurer<'a, 'print> {
MeasureMode::AllLines => {
// Continue measuring on the next line
self.state.line_width = 0;
self.state.pending_indent = args.indention();
}
}
}
}
}
FormatElement::StaticText { text } => return Ok(self.fits_text(text)),
FormatElement::DynamicText { text, .. } => return Ok(self.fits_text(text)),
FormatElement::StaticText { text } => return Ok(self.fits_text(text, args)),
FormatElement::DynamicText { text, .. } => return Ok(self.fits_text(text, args)),
FormatElement::SourceCodeSlice { slice, .. } => {
let text = slice.text(self.printer.source_code);
return Ok(self.fits_text(text));
return Ok(self.fits_text(text, args));
}
FormatElement::LineSuffixBoundary => {
if self.state.has_line_suffix {
@@ -1255,7 +1254,7 @@ impl<'a, 'print> FitsMeasurer<'a, 'print> {
Ok(Fits::Maybe)
}
fn fits_text(&mut self, text: &str) -> Fits {
fn fits_text(&mut self, text: &str, args: PrintElementArgs) -> Fits {
let indent = std::mem::take(&mut self.state.pending_indent);
self.state.line_width += indent.level() as usize * self.options().indent_width() as usize
+ indent.align() as usize;
@@ -1264,10 +1263,17 @@ impl<'a, 'print> FitsMeasurer<'a, 'print> {
let char_width = match c {
'\t' => self.options().tab_width as usize,
'\n' => {
return if self.must_be_flat {
Fits::No
if self.must_be_flat {
return Fits::No;
} else {
Fits::Yes
match args.measure_mode() {
MeasureMode::FirstLine => return Fits::Yes,
MeasureMode::AllLines => {
self.state.line_width = 0;
self.state.pending_indent = args.indention();
continue;
}
}
};
}
c => c.width().unwrap_or(0),

View File

@@ -66,3 +66,12 @@ a = (
# formatting
(lambda:(#
),)
# lambda arguments don't have parentheses, so we never add a magic trailing comma ...
def f(
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa: bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb = lambda x: y,
):
pass
# ...but we do preserve a trailing comma after the arguments
a = lambda b,: 0

View File

@@ -20,3 +20,10 @@
"seventh entry",
"eighth entry",
)
# Regression test: Respect setting in Arguments formatting
def f(a): pass
def g(a,): pass
x1 = lambda y: 1
x2 = lambda y,: 1

View File

@@ -75,7 +75,7 @@ impl FormatNodeRule<ExprAttribute> for FormatExprAttribute {
impl NeedsParentheses for ExprAttribute {
fn needs_parentheses(
&self,
parent: AnyNodeRef,
_parent: AnyNodeRef,
context: &PyFormatContext,
) -> OptionalParentheses {
// Checks if there are any own line comments in an attribute chain (a.b.c).
@@ -88,7 +88,7 @@ impl NeedsParentheses for ExprAttribute {
{
OptionalParentheses::Always
} else {
self.value.needs_parentheses(parent, context)
self.value.needs_parentheses(self.into(), context)
}
}
}

View File

@@ -119,10 +119,10 @@ impl FormatNodeRule<ExprCall> for FormatExprCall {
impl NeedsParentheses for ExprCall {
fn needs_parentheses(
&self,
parent: AnyNodeRef,
_parent: AnyNodeRef,
context: &PyFormatContext,
) -> OptionalParentheses {
self.func.needs_parentheses(parent, context)
self.func.needs_parentheses(self.into(), context)
}
}

View File

@@ -5,7 +5,7 @@ use ruff_text_size::{TextRange, TextSize};
use ruff_formatter::{format_args, write, FormatRuleWithOptions};
use ruff_python_ast::node::{AnyNodeRef, AstNode};
use ruff_python_trivia::{first_non_trivia_token, SimpleToken, SimpleTokenKind, SimpleTokenizer};
use ruff_python_trivia::{SimpleToken, SimpleTokenKind, SimpleTokenizer};
use crate::comments::{
dangling_comments, leading_comments, leading_node_comments, trailing_comments,
@@ -160,34 +160,31 @@ impl FormatNodeRule<Arguments> for FormatArguments {
joiner.finish()?;
write!(f, [if_group_breaks(&text(","))])?;
// Functions use the regular magic trailing comma logic, lambdas may or may not have
// a trailing comma but it's just preserved without any magic.
// ```python
// # Add magic trailing comma if its expands
// def f(a): pass
// # Expands if magic trailing comma setting is respect, otherwise remove the comma
// def g(a,): pass
// # Never expands
// x1 = lambda y: 1
// # Never expands, the comma is always preserved
// x2 = lambda y,: 1
// ```
if self.parentheses == ArgumentsParentheses::Never {
// For lambdas (no parentheses), preserve the trailing comma. It doesn't
// behave like a magic trailing comma, it's just preserved
if has_trailing_comma(item, last_node, f.context().source()) {
write!(f, [text(",")])?;
}
} else {
write!(f, [if_group_breaks(&text(","))])?;
// Expand the group if the source has a trailing *magic* comma.
if let Some(last_node) = last_node {
let ends_with_pos_only_argument_separator = !posonlyargs.is_empty()
&& args.is_empty()
&& vararg.is_none()
&& kwonlyargs.is_empty()
&& kwarg.is_none();
let maybe_comma_token = if ends_with_pos_only_argument_separator {
// `def a(b, c, /): ... `
let mut tokens =
SimpleTokenizer::starts_at(last_node.end(), f.context().source())
.skip_trivia();
let comma = tokens.next();
assert!(matches!(comma, Some(SimpleToken { kind: SimpleTokenKind::Comma, .. })), "The last positional only argument must be separated by a `,` from the positional only arguments separator `/` but found '{comma:?}'.");
let slash = tokens.next();
assert!(matches!(slash, Some(SimpleToken { kind: SimpleTokenKind::Slash, .. })), "The positional argument separator must be present for a function that has positional only arguments but found '{slash:?}'.");
tokens.next()
} else {
first_non_trivia_token(last_node.end(), f.context().source())
};
if maybe_comma_token.map_or(false, |token| token.kind() == SimpleTokenKind::Comma) {
if f.options().magic_trailing_comma().is_respect()
&& has_trailing_comma(item, last_node, f.context().source())
{
// Make the magic trailing comma expand the group
write!(f, [hard_line_break()])?;
}
}
@@ -591,3 +588,33 @@ pub(crate) enum ArgumentSeparatorCommentLocation {
StarLeading,
StarTrailing,
}
fn has_trailing_comma(arguments: &Arguments, last_node: Option<AnyNodeRef>, source: &str) -> bool {
// No nodes, no trailing comma
let Some(last_node) = last_node else {
return false;
};
let ends_with_pos_only_argument_separator = !arguments.posonlyargs.is_empty()
&& arguments.args.is_empty()
&& arguments.vararg.is_none()
&& arguments.kwonlyargs.is_empty()
&& arguments.kwarg.is_none();
let mut tokens = SimpleTokenizer::starts_at(last_node.end(), source).skip_trivia();
// `def a(b, c, /): ... `
// The slash lacks its own node
if ends_with_pos_only_argument_separator {
let comma = tokens.next();
assert!(matches!(comma, Some(SimpleToken { kind: SimpleTokenKind::Comma, .. })), "The last positional only argument must be separated by a `,` from the positional only arguments separator `/` but found '{comma:?}'.");
let slash = tokens.next();
assert!(matches!(slash, Some(SimpleToken { kind: SimpleTokenKind::Slash, .. })), "The positional argument separator must be present for a function that has positional only arguments but found '{slash:?}'.");
}
tokens
.next()
.expect("There must be a token after the argument list")
.kind()
== SimpleTokenKind::Comma
}

View File

@@ -203,18 +203,41 @@ class C:
print(i)
xxxxxxxxxxxxxxxx = Yyyy2YyyyyYyyyyy(
push_manager=context.request.resource_manager,
@@ -120,9 +120,7 @@
key7: value7,
key8: value8,
key9: value9,
@@ -110,19 +110,20 @@
value, is_going_to_be="too long to fit in a single line", srsly=True
), "Not what we expected"
- assert {
- key1: value1,
- key2: value2,
- key3: value3,
- key4: value4,
- key5: value5,
- key6: value6,
- key7: value7,
- key8: value8,
- key9: value9,
- } == expected, (
- "Not what we expected and the message is too long to fit in one line"
- )
+ } == expected, "Not what we expected and the message is too long to fit in one line"
+ assert (
+ {
+ key1: value1,
+ key2: value2,
+ key3: value3,
+ key4: value4,
+ key5: value5,
+ key6: value6,
+ key7: value7,
+ key8: value8,
+ key9: value9,
+ }
+ == expected
+ ), "Not what we expected and the message is too long to fit in one line"
assert expected(
value, is_going_to_be="too long to fit in a single line", srsly=True
@@ -153,7 +151,8 @@
@@ -153,7 +154,8 @@
" because it's too long"
)
@@ -224,7 +247,7 @@ class C:
%3d 0 LOAD_FAST 1 (x)
2 LOAD_CONST 1 (1)
4 COMPARE_OP 2 (==)
@@ -161,8 +160,8 @@
@@ -161,8 +163,8 @@
8 STORE_ATTR 0 (x)
10 LOAD_CONST 0 (None)
12 RETURN_VALUE
@@ -352,17 +375,20 @@ class C:
value, is_going_to_be="too long to fit in a single line", srsly=True
), "Not what we expected"
assert {
key1: value1,
key2: value2,
key3: value3,
key4: value4,
key5: value5,
key6: value6,
key7: value7,
key8: value8,
key9: value9,
} == expected, "Not what we expected and the message is too long to fit in one line"
assert (
{
key1: value1,
key2: value2,
key3: value3,
key4: value4,
key5: value5,
key6: value6,
key7: value7,
key8: value8,
key9: value9,
}
== expected
), "Not what we expected and the message is too long to fit in one line"
assert expected(
value, is_going_to_be="too long to fit in a single line", srsly=True

View File

@@ -203,18 +203,41 @@ class C:
print(i)
xxxxxxxxxxxxxxxx = Yyyy2YyyyyYyyyyy(
push_manager=context.request.resource_manager,
@@ -120,9 +120,7 @@
key7: value7,
key8: value8,
key9: value9,
@@ -110,19 +110,20 @@
value, is_going_to_be="too long to fit in a single line", srsly=True
), "Not what we expected"
- assert {
- key1: value1,
- key2: value2,
- key3: value3,
- key4: value4,
- key5: value5,
- key6: value6,
- key7: value7,
- key8: value8,
- key9: value9,
- } == expected, (
- "Not what we expected and the message is too long to fit in one line"
- )
+ } == expected, "Not what we expected and the message is too long to fit in one line"
+ assert (
+ {
+ key1: value1,
+ key2: value2,
+ key3: value3,
+ key4: value4,
+ key5: value5,
+ key6: value6,
+ key7: value7,
+ key8: value8,
+ key9: value9,
+ }
+ == expected
+ ), "Not what we expected and the message is too long to fit in one line"
assert expected(
value, is_going_to_be="too long to fit in a single line", srsly=True
@@ -153,7 +151,8 @@
@@ -153,7 +154,8 @@
" because it's too long"
)
@@ -224,7 +247,7 @@ class C:
%3d 0 LOAD_FAST 1 (x)
2 LOAD_CONST 1 (1)
4 COMPARE_OP 2 (==)
@@ -161,8 +160,8 @@
@@ -161,8 +163,8 @@
8 STORE_ATTR 0 (x)
10 LOAD_CONST 0 (None)
12 RETURN_VALUE
@@ -352,17 +375,20 @@ class C:
value, is_going_to_be="too long to fit in a single line", srsly=True
), "Not what we expected"
assert {
key1: value1,
key2: value2,
key3: value3,
key4: value4,
key5: value5,
key6: value6,
key7: value7,
key8: value8,
key9: value9,
} == expected, "Not what we expected and the message is too long to fit in one line"
assert (
{
key1: value1,
key2: value2,
key3: value3,
key4: value4,
key5: value5,
key6: value6,
key7: value7,
key8: value8,
key9: value9,
}
== expected
), "Not what we expected and the message is too long to fit in one line"
assert expected(
value, is_going_to_be="too long to fit in a single line", srsly=True

View File

@@ -72,6 +72,15 @@ a = (
# formatting
(lambda:(#
),)
# lambda arguments don't have parentheses, so we never add a magic trailing comma ...
def f(
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa: bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb = lambda x: y,
):
pass
# ...but we do preserve a trailing comma after the arguments
a = lambda b,: 0
```
## Output
@@ -143,6 +152,17 @@ a = (
lambda: ( #
),
)
# lambda arguments don't have parentheses, so we never add a magic trailing comma ...
def f(
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa: bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb = lambda x: y,
):
pass
# ...but we do preserve a trailing comma after the arguments
a = lambda b,: 0
```

View File

@@ -26,6 +26,13 @@ input_file: crates/ruff_python_formatter/resources/test/fixtures/ruff/skip_magic
"seventh entry",
"eighth entry",
)
# Regression test: Respect setting in Arguments formatting
def f(a): pass
def g(a,): pass
x1 = lambda y: 1
x2 = lambda y,: 1
```
## Outputs
@@ -56,6 +63,21 @@ magic-trailing-comma = Respect
"seventh entry",
"eighth entry",
)
# Regression test: Respect setting in Arguments formatting
def f(a):
pass
def g(
a,
):
pass
x1 = lambda y: 1
x2 = lambda y,: 1
```
@@ -82,6 +104,19 @@ magic-trailing-comma = Ignore
"seventh entry",
"eighth entry",
)
# Regression test: Respect setting in Arguments formatting
def f(a):
pass
def g(a):
pass
x1 = lambda y: 1
x2 = lambda y,: 1
```

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/context.rs
source: ruff_python_parser/src/context.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/context.rs
source: ruff_python_parser/src/context.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/context.rs
source: ruff_python_parser/src/context.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/context.rs
source: ruff_python_parser/src/context.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/context.rs
source: ruff_python_parser/src/context.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/context.rs
source: ruff_python_parser/src/context.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/context.rs
source: ruff_python_parser/src/context.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/context.rs
source: ruff_python_parser/src/context.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/context.rs
source: ruff_python_parser/src/context.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/context.rs
source: ruff_python_parser/src/context.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/context.rs
source: ruff_python_parser/src/context.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/context.rs
source: ruff_python_parser/src/context.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/context.rs
source: ruff_python_parser/src/context.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/context.rs
source: ruff_python_parser/src/context.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/context.rs
source: ruff_python_parser/src/context.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/context.rs
source: ruff_python_parser/src/context.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/context.rs
source: ruff_python_parser/src/context.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/context.rs
source: ruff_python_parser/src/context.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/function.rs
source: ruff_python_parser/src/function.rs
expression: parse_ast
---
Ok(

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/function.rs
source: ruff_python_parser/src/function.rs
expression: parse_ast
---
Ok(

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/function.rs
source: ruff_python_parser/src/function.rs
expression: parse_ast
---
Ok(

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/function.rs
source: ruff_python_parser/src/function.rs
expression: parse_ast
---
Ok(

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/function.rs
source: ruff_python_parser/src/function.rs
expression: parse_ast
---
Ok(

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/function.rs
source: ruff_python_parser/src/function.rs
expression: parse_ast
---
Ok(

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/function.rs
source: ruff_python_parser/src/function.rs
expression: parse_ast
---
Ok(

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/function.rs
source: ruff_python_parser/src/function.rs
expression: parse_ast
---
Ok(

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/function.rs
source: ruff_python_parser/src/function.rs
expression: parse_ast
---
Ok(

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/function.rs
source: ruff_python_parser/src/function.rs
expression: parse_ast
---
Ok(

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/function.rs
source: ruff_python_parser/src/function.rs
expression: parse_ast
---
Ok(

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/function.rs
source: ruff_python_parser/src/function.rs
expression: parse_ast
---
Ok(

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/function.rs
source: ruff_python_parser/src/function.rs
expression: parse_ast
---
Ok(

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/function.rs
source: ruff_python_parser/src/function.rs
expression: parse_ast
---
Ok(

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/function.rs
source: ruff_python_parser/src/function.rs
expression: parse_ast
---
Ok(

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/function.rs
source: ruff_python_parser/src/function.rs
expression: parse_ast
---
Ok(

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/function.rs
source: ruff_python_parser/src/function.rs
expression: parse_ast
---
Ok(

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: parse_ast
---
Dict(

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: parse_ast
---
Call(

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: parse_ast
---
Module(

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: "ast::Suite::parse(source, \"<test>\").unwrap()"
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: "ast::Suite::parse(source, \"<test>\").unwrap()"
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: "ast::Suite::parse(source, \"<test>\").unwrap()"
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: parse_ast
---
BoolOp(

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: parse_ast
---
BoolOp(

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: "ast::Suite::parse(source, \"<test>\").unwrap()"
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: "ast::Suite::parse(source, \"<test>\").unwrap()"
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: parse_ast
---
DictComp(

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: parse_ast
---
ListComp(

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: parse_ast
---
[]

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: "ast::Suite::parse(source, \"<test>\").unwrap()"
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: parse_ast
---
GeneratorExp(

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: parse_ast
---
GeneratorExp(

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: parse_ast
---
ListComp(

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: parse_ast
---
GeneratorExp(

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: "ast::Suite::parse(source, \"<test>\").unwrap()"
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: parse_ast
---
Subscript(

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: "ast::Suite::parse(source, \"<test>\").unwrap()"
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/parser.rs
source: ruff_python_parser/src/parser.rs
expression: "ast::Suite::parse(source, \"<test>\").unwrap()"
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/string.rs
source: ruff_python_parser/src/string.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/string.rs
source: ruff_python_parser/src/string.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/string.rs
source: ruff_python_parser/src/string.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/string.rs
source: ruff_python_parser/src/string.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/string.rs
source: ruff_python_parser/src/string.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/string.rs
source: ruff_python_parser/src/string.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/string.rs
source: ruff_python_parser/src/string.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/string.rs
source: ruff_python_parser/src/string.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/string.rs
source: ruff_python_parser/src/string.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/string.rs
source: ruff_python_parser/src/string.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/string.rs
source: ruff_python_parser/src/string.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/string.rs
source: ruff_python_parser/src/string.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/string.rs
source: ruff_python_parser/src/string.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/string.rs
source: ruff_python_parser/src/string.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/string.rs
source: ruff_python_parser/src/string.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/string.rs
source: ruff_python_parser/src/string.rs
expression: parse_ast
---
[

View File

@@ -1,5 +1,5 @@
---
source: crates/ruff_python_parser/src/string.rs
source: ruff_python_parser/src/string.rs
expression: parse_ast
---
[

Some files were not shown because too many files have changed in this diff Show More