konsti
e87032d57d
Explain check_docs_formatted.py error message ( #6125 )
...
## Summary
This is an error message only change to lead an implementor of a new
rule that has an unformatted or invalid bad example to the
right code.
## Test Plan
n/a
2023-07-27 15:53:52 +02:00
Charlie Marsh
0c2abf8316
Avoid walking past root when resolving imports ( #6126 )
...
## Summary
Noticed in #5954 : we walk _past_ the root rather than stopping _at_ the
root when attempting to traverse along the parent path. It's effectively
an off-by-one bug.
2023-07-27 13:38:51 +00:00
konsti
6e85e0010f
Fix windows test warnings ( #6124 )
...
See
https://github.com/astral-sh/ruff/actions/runs/5679922286/job/15392998698 .
These didn't fail CI because we run clippy on linux only.
2023-07-27 14:03:53 +02:00
Micha Reiser
2ea0a0b28a
Respect indent when measuring with MeasureMode::AllLines ( #6120 )
2023-07-27 13:34:36 +02:00
konsti
10abc8fd92
Unbreak main ( #6123 )
...
This fixes main breaking due to two merges.
2023-07-27 11:32:55 +00:00
konsti
61195bc7b1
Don't format trailing comma for lambda arguments ( #5946 )
...
**Summary** lambda arguments don't have parentheses, so they shouldn't
get a magic trailing comma either. This fixes some unstable formatting
**Test Plan** Added a regression test.
89 (from previously 145) instances of unstable formatting remaining.
```
$ cargo run --bin ruff_dev --release -- format-dev --stability-check --error-file formatter-ecosystem-errors.txt --multi-project target/checkouts > formatter-ecosystem-progress.txt
$ rg "Unstable formatting" target/formatter-ecosystem-errors.txt | wc -l
89
```
Closes #5892
2023-07-27 12:32:10 +02:00
Micha Reiser
9040554be5
Merge pull request #6119 from astral-sh/preserve-history
2023-07-27 12:12:57 +02:00
Micha Reiser
06ba1cf0ad
Move RustPython Files
2023-07-27 11:46:18 +02:00
Micha Reiser
7a63a8f45a
Merge branch 'main' of ../parser-move into preserve-history
2023-07-27 11:39:22 +02:00
Micha Reiser
c1bc0dc83d
Delete RustPython files
2023-07-27 11:39:14 +02:00
Micha Reiser
90573975f3
Delete more unused files
2023-07-27 11:32:12 +02:00
Micha Reiser
802df97d55
Fix clippy issues
2023-07-27 11:31:10 +02:00
Micha Reiser
6f4ee32807
Move files to common directory
2023-07-27 11:27:37 +02:00
Zanie Blue
7ec979d022
Fix TypeAliasName to store name instead of load ( #42 )
...
Copies the fix in https://github.com/RustPython/Parser/pull/99
2023-07-26 17:53:34 -05:00
Micha Reiser
fab9cc5294
Remove parser ast re-export ( #41 )
2023-07-26 14:54:29 +02:00
Micha Reiser
ff7bab589e
Prepare for moving the parser into the Ruff monorepo ( #40 )
2023-07-26 12:28:43 +02:00
Micha Reiser
593b46be5e
perf: Cursor based lexer ( #38 )
2023-07-26 07:50:45 +02:00
David Szotten
13196fc500
fix the ranges of constants inside f-strings ( #33 )
2023-07-25 19:17:06 +02:00
Dhruv Manilawala
5ef4ccd632
Add line magic stmt and expr AST nodes ( #31 )
...
This PR introduces two new nodes for the parser to recognize the
`MagicCommand` tokens:
* `StmtLineMagic` for statement position i.e., magic commands on their
own line:
```python
%matplotlib inline
!pwd
```
* `ExprLineMagic` for expression position i.e., magic commands in an
assignment statement:
```python
# Only `?` and `!` are valid in this position
dir = !pwd
```
Both nodes are identical in their structure as in it contains the magic
kind and the command value as `String`.
2023-07-24 21:20:31 +05:30
Dhruv Manilawala
e363fb860e
Lex Jupyter Magic in assignment value position ( #30 )
...
Emit `MagicCommand` token when it is the assignment value[^1] i.e., on
the right side of an assignment statement.
Examples:
```python
pwd = !pwd
foo = %timeit a = b
bar = %timeit a % 3
baz = %matplotlib \
inline"
```
[^1]: Only `%` and `!` are valid in that position, other magic kinds are
not valid
2023-07-24 17:44:03 +05:30
Chris Pryer
4888d800fb
Fix BoolOp python.org link ( #37 )
...
Noticed https://docs.python.org/3/library/ast.html#ast.boolop wasn't
working.
2023-07-22 14:10:59 -04:00
konsti
4d03b9b5b2
Fix empty lambda args range ( #35 )
...
Previously, empty lambda arguments (e.g. `lambda: 1`) would get the
range of the entire expression, which leads to incorrect comment
placement. Now empty lambda arguments get an empty range between the
`lambda` and the `:` tokens.
2023-07-21 17:41:19 +02:00
Zanie Blue
fb365c642b
Implement Ranged for TypeParam ( #32 )
...
Required for https://github.com/astral-sh/ruff/pull/5927
2023-07-20 20:03:17 -04:00
konsti
db04fd4157
Refactor if statement, introduce nodes for elif and else ( #22 )
...
The old if layout couldn't differentiate between an else block with a
single if statement and an elif statement. Additionally we getting rid
of the recursion in favor of a single if struct with a vec of elif/else
children. This is accompanied by a big refactoring in ruff which removes
a bunch of TODOs and false negatives.
2023-07-18 13:14:40 +02:00
konstin
ff3e8ead36
Revert "snapshot"
...
This reverts commit 1bbec2b967 .
2023-07-18 11:09:43 +02:00
konstin
1bbec2b967
snapshot
2023-07-18 11:08:14 +02:00
Dhruv Manilawala
510be51cfa
Fix Jupyter Magic test snapshot ( #29 )
...
Sorry for the churn, I should've rebased and merged the original PR to
avoid this.
2023-07-17 23:06:53 -05:00
Dhruv Manilawala
3b4c8fffe5
Lex Jupyter line magic with Mode::Jupyter ( #23 )
...
Lex Jupyter line magic with `Mode::Jupyter`
This PR adds a new token `MagicCommand`[^1] which the lexer will
recognize when in `Mode::Jupyter`. The rules for the lexer is as
follows:
1. Given that we are at the start of line, skip the indentation and look
for [characters that represent the start of a magic
command](635815e8f1/IPython/core/inputtransformer2.py (L335-L346) ),
determine the magic kind and capture all the characters following it as
the command string.
2. If the command extends multiple lines, the lexer will skip the line
continuation character (`\`) but only if it's followed by a newline
(`\n` or `\r`). The reason to skip this only in case of newline is
because they can occur in the command string which we should not skip:
```rust
// Skip this backslash
// v
// !pwd \
// && ls -a | sed 's/^/\\ /'
// ^^
// Don't skip these backslashes
```
3. The parser, when in `Mode::Jupyter`, will filter these tokens before
the parsing begins. There is a small caveat when the magic command is
indented. In the following example, when the parser filters out magic
command, it'll throw an indentation error:
```python
for i in range(5):
!ls
# What the parser will see
for i in range(5):
```
[^1]: I would prefer to have some other name as this not only represent
a line magic (`%`) but also shell command (`!`), help command (`?`) and
others. In original implementation, it's named as ["IPython
Syntax"](635815e8f1/IPython/core/inputtransformer2.py (L332) )
2023-07-18 09:24:24 +05:30
Zanie
126652b684
Fix decorator ranges
...
Incorrectly merged LALRPOP file
2023-07-17 14:49:14 -05:00
Zanie
57e8712d76
Bump expected size of Stmt to 168 bytes
2023-07-17 14:49:14 -05:00
Zanie
78c6ede1c9
Format
2023-07-17 14:49:14 -05:00
Zanie Blue
a843a00f6b
Add parsing of type alias statements i.e. the type keyword ( #97 )
...
Extends #95
Closes #82
Adds parsing of new `type` soft keyword for defining type aliases.
Supports type alias statements as defined in PEP 695 e.g.
```python
type IntOrStr = int | str
type ListOrSet[T] = list[T] | set[T]
type AnimalOrVegetable = Animal | "Vegetable"
type RecursiveList[T] = T | list[RecursiveList[T]]
```
All type parameter kinds are supported as in #95 .
Builds on soft keyword abstractions introduced in https://github.com/RustPython/RustPython/pull/4519
2023-07-17 14:49:14 -05:00
Zanie Blue
f846f1ea42
Parse type parameters in function definitions ( #96 )
...
* Parse type parameters in function definitions
* Add test for combined items
2023-07-17 14:49:14 -05:00
Zanie
1d4b7a395f
Consolidate tests and add coverage for trailing comma
2023-07-17 14:49:14 -05:00
Zanie
ce3ce0734b
Add bound to test case test_parse_class_with_all_possible_generic_types
2023-07-17 14:49:14 -05:00
Zanie
b0e119f049
Add test for tuple bounds
2023-07-17 14:49:14 -05:00
Zanie
1f5e707829
Remove test for empty generic class Foo[]: ...
...
Not valid syntax
2023-07-17 14:49:14 -05:00
Zanie
ed7acfe477
Parse type parameters in class definitions
2023-07-17 14:49:14 -05:00
Zanie
c31b58eb39
Move type_param stubs into LALRPOP definition
2023-07-17 14:49:14 -05:00
Zanie
c0a3a20c63
Bump size assertion for Stmt from 136 to 160 bytes
2023-07-17 14:49:14 -05:00
Zanie
05ae26b935
Regenerate code with latest ASDL
2023-07-17 14:49:14 -05:00
David Szotten
b996b21ffc
tuple constants are for optimisations, not source ( #28 )
...
my reading of https://docs.python.org/3/library/ast.html#ast.unparse and
https://discuss.python.org/t/ast-constant-value-tuple-s-and-frozenset-s/22578
is that tuple constants cannot come from parsing python source, they are
only for optimised bytecode
see also https://github.com/astral-sh/ruff/pull/5812
2023-07-17 15:46:37 -04:00
konsti
2d1f69cbb9
Remove asdl ( #21 )
...
This removes the ASDL code generation in favor of handwriting the AST.
The motivations for moving away from the ASDL are:
* CPython compatibility is no longer a goal
* The ASDL grammar isn't as expressive as we would like
* The codegen scripts have a high complexity which makes extensions time
consuming
* We don't make heavy use of code generation (compared to e.g.
RustPython that generates Pyo3 bindings, a fold implementation etc).
We may want to revisit a grammar based code generation in the future,
e.g. by using [ungrammar](https://github.com/rust-analyzer/ungrammar )
2023-07-05 14:25:26 +02:00
konsti
0f2e295f3a
impl Ranged for TextRange ( #20 )
...
This adds the missing implementation of `Ranged` for `TextRange` itself
```rust
impl Ranged for TextRange {
fn range(&self) -> TextRange {
*self
}
}
```
This allows e.g. using `has_comments` with arbitrary ranges instead of
just a node.
It also adds .venv to the .gitignore
2023-07-02 10:11:06 +02:00
konstin
c174bbf1f2
impl<T> Ranged for &T where T: Ranged ( #16 )
...
In the example below, `arg` is `&Expr`, so `&Ranged`, but `entries()`
want a `T: Ranged`. This adds the missing bridge impl.
```rust
let all_args = format_with(|f| {
f.join_comma_separated()
.entries(
// We have the parentheses from the call so the arguments never need any
args.iter()
.map(|arg| (arg, arg.format().with_options(Parenthesize::Never))),
)
.nodes(keywords.iter())
.finish()
});
```
2023-06-26 13:55:35 +02:00
Micha Reiser
8078663b6c
Remove Range type parameter from AST nodes ( #15 )
2023-06-23 21:18:55 +01:00
Micha Reiser
f60e204b73
Fix range of keyword identifier ( #14 )
2023-06-22 10:00:25 +02:00
Micha Reiser
08ebbe40d7
Fix ArgWithDefault TextRange ( #13 )
2023-06-20 18:21:09 +02:00
Charlie Marsh
ed3b4eb72b
Add TextRange to Identifier ( #8 )
...
## Summary
This PR adds `TextRange` to `Identifier`. Right now, the AST only
includes ranges for identifiers in certain cases (`Expr::Name`,
`Keyword`, etc.), namely when the identifier comprises an entire AST
node. In Ruff, we do additional ad-hoc lexing to extract identifiers
from source code.
One frequent example: given a function `def f(): ...`, we lex to find
the range of `f`, for use in diagnostics.
Another: `except ValueError as e`, for which the AST doesn't include a
range for `e`.
Note that, as an optimization, we avoid storing the `TextRange` for
`Expr::Name`, since it's already included.
2023-06-20 11:19:27 -04:00
Charlie Marsh
f0d200c8a1
Remove fold, unparse, and location features ( #9 )
2023-06-19 17:26:17 -04:00
Charlie Marsh
8d74eee750
Make malachite-bigint an optional dependency for rustpython-format ( #12 )
2023-06-19 16:31:25 -04:00
Charlie Marsh
21aa0b8d84
Optimize validate_arguments ( #10 )
2023-06-19 15:32:58 -04:00
Charlie Marsh
9cb00518e5
Update snapshot tests
2023-06-19 13:16:07 -04:00
Micha Reiser
6f65c5cba7
Add Decorator node ( #7 )
2023-06-19 12:53:05 -04:00
Micha Reiser
8a415fa61e
Include argument parentheses in range ( #5 )
2023-06-19 12:51:18 -04:00
Micha Reiser
5054cbe84f
Merge branch 'RustPython:main' into main
2023-06-19 12:38:24 -04:00
Jeong, YunWon
69d27d924c
Rename unconventional nodes ( #74 )
2023-06-17 01:54:00 +09:00
Jeong, YunWon
5270020423
rustpython_ast python package ( #79 )
2023-06-17 01:32:27 +09:00
yt2b
edcfcb4a74
Fix bool format ( #91 )
...
* Fix format_bool
* Add test_format_bool
2023-06-15 14:56:03 +09:00
Jeong, YunWon
40a603208f
Add Pylyzer
...
done by https://github.com/mtshiba/pylyzer/pull/37
2023-06-11 21:56:59 +09:00
Jeong, YunWon
b2f95e2848
Fix LinearLocator \r handling ( #80 )
2023-06-02 22:35:53 +09:00
Steve Shi
a2e3209c42
Replace num-bigint with malachite-bigint ( #18 )
...
Co-authored-by: Jeong YunWon <jeong@youknowone.org >
2023-06-02 17:06:18 +09:00
Jeong, YunWon
5e9e8a7589
Linear Locator ( #46 )
2023-06-01 13:53:31 +09:00
Jeong, YunWon
fdec727f80
New Arguments and Arg/ArgWithDefault AST representation ( #59 )
2023-06-01 01:15:23 +09:00
Jeong, YunWon
3fbf4f6804
Parse for expr and stmt variants + identifier, constant ( #78 )
2023-05-31 20:03:46 +09:00
Micha Reiser
fe25708d89
Merge pull request #76 from astral-sh/match-case-end-location
2023-05-31 09:48:42 +02:00
Micha Reiser
342cd19f50
Add safety comment
2023-05-31 09:37:48 +02:00
Micha Reiser
4a2c4aad0b
Align MatchCase end location
2023-05-30 17:31:51 +02:00
Jeong, YunWon
ae3a477c97
Add parser deps to rustpython_ast_pyo3 ( #75 )
2023-05-29 16:48:05 +09:00
Jeong, YunWon
531aeb3511
Cmpop::as_str ( #72 )
...
* clean up pyo3 generation
* Cmpop::as_str
2023-05-29 01:53:54 +09:00
Jeong, YunWon
4de0cb1827
Parse Trait ( #71 )
2023-05-28 21:03:27 +09:00
Micha Reiser
5493c9f4e3
Avoid removing elements from the beginning of a vec ( #69 )
2023-05-25 19:44:49 +09:00
Jeong, YunWon
d23611db65
Merge pull request #65 from youknowone/refactor-pyo3
...
to_pyo3_ast to return &'py + Separate rustpython_ast_pyo3
2023-05-23 03:03:01 +09:00
Jeong YunWon
7d384d88d0
Separate rustpython_ast_pyo3
2023-05-23 02:50:12 +09:00
Jeong YunWon
b81273e9bc
to_pyo3_ast to return &'py
2023-05-23 00:12:22 +09:00
Micha Reiser
335780aeea
Merge pull request #2 from astral-sh/include-decorators-in-class-and-func-range
2023-05-22 13:46:38 +02:00
Jeong, YunWon
e1f02fced7
Fix Vec::to_pyo3_ast ( #63 )
2023-05-22 15:15:05 +09:00
Jeong, YunWon
d4084fb17a
impl From<T> for ast::Ast ( #62 )
2023-05-21 22:44:07 +09:00
Micha Reiser
41a0ef8740
Add test
2023-05-19 10:14:46 +02:00
Micha Reiser
25cc1da319
Include decorators in Class and FunctionDef range
2023-05-19 08:18:43 +02:00
Micha Reiser
33a3c407a9
Merge pull request #60 from astral-sh/reduce-copy
2023-05-19 08:16:28 +02:00
Micha Reiser
fac0c25343
Update python.rs
2023-05-19 08:08:35 +02:00
Micha Reiser
4ff779c298
Update parser/src/python.lalrpop
...
Co-authored-by: Jeong, YunWon <69878+youknowone@users.noreply.github.com >
2023-05-18 22:32:05 +02:00
Micha Reiser
726884b287
Avoid allocating vecs for each statement
2023-05-18 21:56:17 +02:00
Micha Reiser
851f23668f
Use named parameters to reducue copying
2023-05-18 21:41:06 +02:00
Jeong, YunWon
3654cf0bdf
Add Ast as top level enum ( #58 )
2023-05-19 01:53:39 +09:00
Jeong, YunWon
6c5c311bab
Fix range field order ( #56 )
...
* Skip validate_arguments when empty
* Fix `range` field order
* Fix unused variable
2023-05-18 21:44:01 +09:00
Jeong, YunWon
531e41ae2c
Upgrade bitflags to remove clippy wanrings ( #54 )
...
* Upgrade bitflags to remove clippy wanrings
* Fix python lint warnings
2023-05-18 01:50:00 +09:00
Jeong, YunWon
b48834fe2d
More flexible map_user and fold for new constructor nodes ( #53 )
...
* make fold.rs file
* Split user_map steps
* Fold for new constructor nodes
2023-05-18 00:16:04 +09:00
Jeong, YunWon
205ee80033
README ( #47 )
2023-05-17 18:01:00 +09:00
Jeong, YunWon
fc301ab1b0
Fix build ( #45 )
...
* remove dedent
* Revert parser-pyo3 crate
2023-05-16 23:54:49 +09:00
Jeong, YunWon
e820928f11
Add experimental pyo3-wrapper feature ( #41 )
...
* Fix pyo3 unit type value error
* Add experimental pyo3-wrapper feature
* location support
2023-05-16 23:45:31 +09:00
Jeong, YunWon
ff17f6e178
Add utilities to enum ( #44 )
...
* Add utilities to enum
* Fix unexpected pyo3 dependency propagation
2023-05-16 23:29:49 +09:00
Jeong YunWon
3bdf8a940a
Hash for ConversionFlag
2023-05-16 22:56:17 +09:00
Jeong, YunWon
9d47d3d212
specialize ConversionFlag ( #42 )
...
* specialize ConversionFlag
* Change value of ConversionFlag to i8 and None to -1
* is_* methods to ConversionFlag
2023-05-16 22:52:50 +09:00
Jeong, YunWon
611dcc2e9b
rustpython_ast + pyo3 ( #25 )
2023-05-16 18:06:54 +09:00
Jeong, YunWon
53de75efc3
Add all node classes to ast module ( #40 )
2023-05-16 16:55:26 +09:00
Jeong, YunWon
0c7d16b61a
Add is_* methods to Tok ( #39 )
2023-05-16 16:08:25 +09:00
Jeong, YunWon
735c06d5f4
Fix full-lexer feature ( #38 )
2023-05-16 15:45:03 +09:00
Jeong, YunWon
02f13abf50
Add Node trait for node type information ( #31 )
2023-05-16 15:41:30 +09:00
Charlie Marsh
10dda125ff
Always emit non-logical newlines for 'empty' lines ( #27 )
2023-05-15 14:13:05 -04:00
Jeong, YunWon
27e3873dc2
Add full-lexer feature ( #36 )
2023-05-16 02:21:34 +09:00
Micha Reiser
dd4cc25227
Reduce copying elements when parsing ( #35 )
2023-05-15 23:20:44 +09:00
Jeong, YunWon
b78001c953
Generic types to generic ( #30 )
2023-05-15 20:22:42 +09:00
Jeong, YunWon
718354673e
Move range to node ( #23 )
...
* black + clippy
* Fix module generation
2023-05-15 16:20:22 +09:00
Micha Reiser
192379cede
Move range from Attributed to Nodes ( #22 )
...
* Move `range` from `Attributed` to `Node`s
* No Attributed + custom for Range PoC
* Generate all located variants, generate enum implementations
* Implement `Copy` on simple enums
* Move `Suite` to `ranged` and `located`
* Update tests
---------
Co-authored-by: Jeong YunWon <jeong@youknowone.org >
2023-05-15 15:08:12 +09:00
Micha Reiser
a983f4383f
Add format and cformat modules from RustPython ( #24 )
...
* Add `format` and `cformat` modules from `RustPython`
* Introduce `rustpython-format` crate
* Remove unused dependencies
2023-05-12 18:27:05 +09:00
Jeong, YunWon
947fb53d0b
Field accessor and utilities ( #20 )
...
* Apply is-macro to Constant and ast nodes
2023-05-11 19:44:20 +09:00
Jeong, YunWon
2baad9ead6
Merge pull request #19 from RustPython/identifier
...
A few more Identifier utilities
2023-05-11 16:12:46 +09:00
Jeong YunWon
2af9805662
A few more Identifier utilities
2023-05-11 16:05:11 +09:00
Jeong, YunWon
5b2af304a2
Merge pull request #17 from youknowone/ruff
...
ruff integration support
2023-05-11 04:42:52 +09:00
Jeong YunWon
cbe4e8c5f3
Make parser location optional
2023-05-11 04:40:10 +09:00
Jeong YunWon
aabc96dde9
ruff integration support
2023-05-11 04:08:57 +09:00
Jeong YunWon
99e108dd53
*Escape::with_preferred_quote
2023-05-10 21:51:01 +09:00
Jeong YunWon
d6b6df5d1c
ast::Int::to_usize
2023-05-10 21:31:02 +09:00
Jeong YunWon
822cac5aa0
parse_expression{=>_starts}_at
2023-05-10 21:25:58 +09:00
Jeong YunWon
aa101e4f26
fix ImportDots
2023-05-10 20:31:04 +09:00
Jeong, YunWon
cbc4cb286b
Merge pull request #16 from youknowone/ast-int
...
Give identifier and int ast types
2023-05-10 20:19:01 +09:00
Jeong YunWon
6fa3d0f90a
Fine-tune int types
2023-05-10 19:33:39 +09:00
Jeong YunWon
455bcc01a0
Give identifier and int ast types
2023-05-10 19:33:39 +09:00
Jeong, YunWon
d495cd9129
Merge pull request #15 from youknowone/spellchecker
...
Setup spell checker
2023-05-10 18:11:40 +09:00
Jeong YunWon
d8822d1091
spell check ast/asdl_rs.py
2023-05-10 18:05:39 +09:00
Jeong YunWon
75f6ce1ae5
Setup spell checker
2023-05-10 17:57:15 +09:00
Jeong, YunWon
41e9e7280a
Merge pull request #12 from youknowone/visitor
...
Visitor
2023-05-10 17:52:25 +09:00
Jeong YunWon
17c8abcec1
asdl_rs.py to multiple file output
2023-05-10 17:41:01 +09:00
Jeong YunWon
e000b1c304
Remove redundant types
2023-05-10 17:15:01 +09:00
Jeong YunWon
243ca16b34
Fix visitor to fit in new structure
2023-05-10 17:14:44 +09:00
Joshua
4de9580b92
Generate a visitor trait to ast_gen.rs
2023-05-10 17:14:34 +09:00
Jeong, YunWon
5cf85f0b9d
Merge pull request #13 from youknowone/locator
...
Python location transform and related refactoring
2023-05-10 17:04:30 +09:00
Jeong YunWon
4dc030ba9d
Vendor SourceLocation from ruff
2023-05-10 17:00:12 +09:00
Jeong YunWon
1d366d52ab
Let located only for python located stuff
2023-05-10 14:35:38 +09:00
Jeong YunWon
a3d9d8cb14
numerous refactoring
...
- Split parser core and compiler core. Fix #14
- AST int type to `u32`
- Updated asdl_rs.py and update_asdl.sh fix #6
- Use `ruff_python_ast::SourceLocation` for Python source location. Deleted our own Location.
- Renamed ast::Located to ast::Attributed to distinguish terms for TextSize and SourceLocation
- `ast::<Node>`s for TextSize located ast. `ast::located::<Node>` for Python source located ast.
- And also strictly renaming `located` to refer only python location related interfaces.
- `SourceLocator` to convert locations.
- New `source-code` features of to disable python locations when unnecessary.
- Also including fully merging https://github.com/astral-sh/RustPython/pull/4 closes #9
2023-05-10 14:35:38 +09:00
Jeong YunWon
09a6afdd04
Adapt SourceLocation
2023-05-09 20:34:48 +09:00
Jeong YunWon
a14e43e03a
Separate byteoffset ast and located ast
2023-05-09 00:21:52 +09:00
Jeong YunWon
f47dfca4e3
Rename compiler Location to TextSize
2023-05-08 03:38:10 +09:00
Micha Reiser
58c35ab458
Replace row/column based Location with byte-offsets.
2023-05-08 03:38:10 +09:00
Jeong, YunWon
7b8844bd3e
Merge pull request #11 from youknowone/refactor-asdl
...
Refactor ast to hold data as seperated type
2023-05-07 19:29:02 +09:00
Jeong YunWon
6d7358090b
Refactor ast to hold data as seperated type
2023-05-07 19:20:47 +09:00
Jeong YunWon
9f1a538eba
gitattribute
2023-05-07 17:20:52 +09:00
Jeong, YunWon
48920a034e
Merge pull request #10 from youknowone/remove-compile-error
...
Remove CompileError
2023-05-06 17:20:44 +09:00
Jeong YunWon
13d6e275ef
Remove CompileError
2023-05-06 17:15:18 +09:00
Jeong, YunWon
96eb80f5cf
Merge pull request #8 from youknowone/python-lint
...
Add python lint
2023-05-06 14:42:15 +09:00
Jeong YunWon
a73bac9ed1
Add python lint
2023-05-06 14:34:31 +09:00
Jeong, YunWon
6b60f85cc4
Merge pull request #7 from youknowone/lalrpop
...
Embed generated parser + update lalrpop
2023-05-06 13:57:15 +09:00
Jeong YunWon
39b2dbe04d
Update lalrpop to 0.20.0
2023-05-06 13:48:20 +09:00
Jeong YunWon
e1f70100ac
Update parser/build.rs to embed python.rs
2023-05-06 05:28:30 +09:00
Jeong, YunWon
d66d935879
Merge pull request #5 from youknowone/ci
...
Add CI
2023-05-06 03:42:28 +09:00
Jeong YunWon
e28f333f23
Add CI
2023-05-06 03:39:20 +09:00
Jeong YunWon
0adcdd995d
Remove unused workspace dependencies
2023-05-06 00:44:34 +09:00
Jeong YunWon
e6eb49ffb0
Set up workspace - Forked from
...
git@github.com:RustPython/RustPython.git ff5076b12c075b3e87c0ac2971e390b4a209d14f
2023-05-05 23:49:50 +09:00
Jeong YunWon
80109b1fe0
rustpython-literal
2023-05-05 21:49:12 +09:00
Jeong YunWon
bd64603950
Refactor common::bytes::repr using common::escape
2023-05-03 16:08:48 +09:00
Jeong YunWon
5b0e92d725
Refactor common::str::repr using common::escape
2023-05-03 13:47:04 +09:00
Micha Reiser
ae9d3c3193
Add Located::start, Located::end and impl Deref
2023-04-26 10:24:34 -06:00
Micha Reiser
3873414b30
Use Located::new over struct initializer
2023-04-25 18:10:13 -06:00
Jeong YunWon
a9f4b59f40
module objects' type as PyModule
2023-04-20 20:48:57 +09:00
Jeong YunWon
6d6155413b
Update lalrpop
2023-03-25 22:18:02 +09:00
Jeong YunWon
f9b5469642
Update cspell for compiler
2023-03-16 22:39:09 +09:00
Charlie Marsh
ef38eb6b1a
Include Derive feature with optional Serde dependency
2023-03-12 14:46:10 -04:00
Jeong YunWon
0ea53825db
Merge pull request #4608 from coolreader18/bag-deser
...
Rework frozen modules and directly deserialize to CodeObject<Literal>
2023-03-10 05:19:23 +09:00
Noa
9d6ae774f8
Rework frozen modules and directly deserialize to CodeObject<Literal>
2023-03-06 13:45:33 -06:00
Charlie Marsh
55fc0e83f3
Treat match and case as soft keywords in lambda assignments ( #4623 )
2023-03-04 12:42:05 -05:00
Charlie Marsh
713dd2b91e
Add optional serde dependency
2023-03-02 15:38:40 -05:00
Noa
969ea23d67
Address review comments
2023-03-01 21:11:59 -06:00
Noa
b80bbec8e6
Custom marshal enc/decoding impl
2023-03-01 20:47:21 -06:00
DimitrisJim
8f425e9ce2
Use insta to verify values.
2023-02-28 20:00:47 +09:00
Jeong YunWon
d7e2e7361e
Use git version of unicode_names2 to avoid alias search failure
2023-02-28 20:00:47 +09:00
Charlie Marsh
c37e0c7f03
Use proper locations for sub-expressions in constant matches
2023-02-26 23:10:38 -05:00
Charlie Marsh
0d7b94817d
Allow type variable tuple for *args
2023-02-23 08:39:59 -05:00
Jeong YunWon
f43e5b72e2
Merge pull request #4552 from charliermarsh/charlie/loc
...
Limit match range to end of last statement
2023-02-23 02:11:42 +09:00
Jeong YunWon
a19f294b0c
Merge pull request #4543 from youknowone/flatten-parser
...
Flatten parser interface
2023-02-23 02:09:46 +09:00
Charlie Marsh
7ebef61c47
Limit match range to end of last statement
2023-02-22 11:25:50 -05:00
Charlie Marsh
b61f4d7b69
Allow trailing commas in MappingPattern
2023-02-22 10:02:41 -05:00
Jeong YunWon
e26369a34e
use super::* from tests submodules
2023-02-22 21:01:39 +09:00
Jeong YunWon
97a08ee77b
remove #[macro_use]
2023-02-22 20:41:27 +09:00
Jeong YunWon
cb8c6fb78d
Flatten rustpython_parser interface
2023-02-22 20:32:31 +09:00
Jeong YunWon
8580e4ebb5
make_tokenizer -> lex to integrate terms
...
we don't distinguish scanner or tokenizer from lexer
2023-02-22 20:28:15 +09:00
Jeong YunWon
39fc23cf92
relocate feature-independent use
2023-02-22 20:28:03 +09:00
Jeong YunWon
66e3080173
Fix ModeParseError message
2023-02-22 20:28:03 +09:00
Jeong YunWon
1511b6631b
Break down rustpython_parser::error module
...
because it doesn't share any common errors but specific error for each sub module
2023-02-22 20:28:01 +09:00
Charlie Marsh
2a8aa6f308
Always wrap in SoftKeywordTransformer
2023-02-21 19:18:42 -05:00
Charlie Marsh
dc628cab8f
Expose SoftKeywordTransformer on public API
2023-02-21 19:00:32 -05:00
Jim Fasarakis-Hilliard
c137bc9d77
Merge pull request #4519 from charliermarsh/charlie/match
...
Add support for match statements to parser
2023-02-21 19:43:28 +02:00
Jeong YunWon
60180fd54c
Merge pull request #4531 from charliermarsh/charlie/exception-groups
...
Implement except* syntax
2023-02-21 13:20:18 +09:00
Charlie Marsh
c7ed645cc6
Implement except* syntax
2023-02-21 12:19:54 +09:00
Charlie Marsh
8caa28f0f8
Update Python.asdl from CPython 3.11.1
2023-02-21 12:19:51 +09:00
Charlie Marsh
8aa3bc93f3
Allow starred expressions in subscripts
2023-02-20 17:59:35 -05:00
Charlie Marsh
f39ffef370
Update compiler/parser/src/soft_keywords.rs
...
Co-authored-by: Jim Fasarakis-Hilliard <d.f.hilliard@gmail.com >
2023-02-20 15:03:39 -05:00
Jeong YunWon
e093d2bee6
clean up soft-keyword transform
2023-02-20 15:03:39 -05:00
Charlie Marsh
ca5b474d45
Use muiltipeek
2023-02-20 15:03:39 -05:00
Charlie Marsh
2b43d45bd5
Add support for match statements to parser
2023-02-20 15:03:39 -05:00
John Pham
4bdc2d47c1
Make common::repr throw error instead of panic ( #4520 )
2023-02-19 22:09:54 +09:00
Jeong YunWon
09b82e41d5
Merge pull request #4490 from DimitrisJim/function_parser
...
Add tests, some comments, to function.rs.
2023-02-13 20:00:35 +09:00
Jeong YunWon
cc6d8a1c58
Merge pull request #4492 from DimitrisJim/doc_parser_uno
...
Document parser crate.
2023-02-13 17:25:14 +09:00
Dimitris Fasarakis Hilliard
07918f0a9a
Document parser crate.
2023-02-12 17:58:19 +02:00
Dimitris Fasarakis Hilliard
a0786ea872
Add tests, some comments, to function.rs.
2023-02-11 23:07:57 +02:00
Dimitris Fasarakis Hilliard
4713b2b3ab
Refactor: Join string and string_parser.
2023-02-11 18:05:06 +02:00
Jeong YunWon
e7f14ab9b8
Add test_generator_expression_argument
2023-02-11 05:20:39 +09:00
Charlie Marsh
56c73cc63d
Use entire range for generators-as-arguments
2023-02-10 10:39:40 -05:00
Dimitris Fasarakis Hilliard
659f4dd8bf
Document lexer.
2023-02-07 21:43:57 +02:00
Dimitris Fasarakis Hilliard
bd158089e0
Move NewLineHandler inline, don't check each character twice.
2023-02-07 20:58:53 +02:00
Jeong YunWon
a73bee7aae
use workspace dependencies
2023-02-06 15:30:38 +09:00
Dimitris Fasarakis Hilliard
1468fe46ab
Hint that the unwrap should always succeed.
2023-02-01 12:17:33 +02:00
Dimitris Fasarakis Hilliard
c9364718b4
Eat for comma.
2023-01-31 12:26:05 +02:00
Dimitris Fasarakis Hilliard
38cf933bcb
Add initial capacities, use u32s for indents/spaces.
2023-01-31 12:26:05 +02:00
Dimitris Fasarakis Hilliard
838990ae15
Don't call is_emoji_presentation for each invocation of consume_normal
2023-01-31 12:26:05 +02:00
Dimitris Fasarakis Hilliard
aa0290bbfc
Match on ascii start/continuation characters before calling functions.
2023-01-31 12:26:05 +02:00
Aarni Koskela
f74e44d1e8
Bump phf to 0.11 series
...
string_cache is still using phf_shared 0.10.0 though
2023-01-25 19:58:43 +02:00
Jeong YunWon
d9df131720
Merge pull request #4449 from harupy/fix-dict-spread-in-dict
...
Fix AST generated from a dict literal containing dict unpacking
2023-01-22 20:44:26 +09:00
harupy
f2ffe12f8c
Fix comment
2023-01-22 00:06:52 +09:00
Anders Kaseorg
6dba8430be
Fix end location for elif blocks
...
Since we parse an `elif:` block as an `If` node, its location should
include its `orelse` node like it would for an `if:` block.
Signed-off-by: Anders Kaseorg <andersk@mit.edu >
2023-01-17 22:39:44 -05:00
harupy
b26365b215
Remove useless String::from
...
Signed-off-by: harupy <hkawamura0130@gmail.com >
2023-01-16 21:27:57 +09:00
harupy
2d019930e9
Rename test
2023-01-15 23:36:07 +09:00
harupy
d5fc7c4c87
Improve test
2023-01-15 16:53:13 +09:00
harupy
393869c47c
Add Option to Dict.keys field
2023-01-15 16:43:13 +09:00
harupy
4edd2bf78a
Remove commented-out code
2023-01-15 16:10:51 +09:00
harupy
d3c4551629
Fix unparse
2023-01-15 13:11:55 +09:00
harupy
581f6e176c
Fix dict spreading in dict literal
2023-01-15 13:01:59 +09:00
Jim Fasarakis-Hilliard
ac4d3c076c
Merge pull request #4443 from bluetech/non-logical-newline-token-fixup
...
Fixup parse_tokens after "Add NonLogicalNewline token"
2023-01-14 12:49:04 +02:00
Ran Benita
e5fe037e38
Fixup parse_tokens after "Add NonLogicalNewline token"
...
I only updated `parse()` to ignore `NonLogicalNewline`, didn't notice
it's also needed in `parse_tokens()`.
2023-01-14 11:52:33 +02:00
Noa
872b9d4765
Switch from 64-bit instruction enum to out-of-line arg values
2023-01-12 23:05:17 -06:00
Ran Benita
674eeec29c
Add NonLogicalNewline token
...
This token is completely ignored by the parser, but it's useful for
other users of the lexer, such as the Ruff linter. For example, the
token is helpful for a "trailing comma" lint.
The same idea exists in Python's `tokenize` module - there is a NEWLINE
token (logical newline), and a NL token (non-logical newline).
Fixes #4385 .
2023-01-12 16:47:12 +02:00
Martin Fischer
4f1e7c6291
Fix docs.rs build for rustpython-parser
...
docs.rs failed to build the documentation of the recently released
rustpython-parser 0.2.0 because the build.rs script couldn't write the
parser.rs file because docs.rs builds the documentation in a sandbox
with a read-only filesystem.
This commit fixes this by writing the parser.rs file to the cargo output
directory instead, as recommended by the docs.rs documentation.[1]
Fixes #4436 .
[1]: https://docs.rs/about/builds#read-only-directories
2023-01-11 09:58:10 +01:00
Noa
884a7bdb15
Bump all crate versions to 0.2.0
2023-01-11 00:14:28 -06:00
Jeong YunWon
7885344bcf
first cspell dict
2023-01-09 19:57:23 +09:00
Jeong YunWon
e8fef39861
Merge pull request #4429 from youknowone/fix-format
...
Fix nightly clippy warnings
2023-01-07 21:38:41 +09:00
Jeong YunWon
509cf7ed0d
Fix nightly clippy warnings
2023-01-07 21:07:10 +09:00
harupy
a4a5366504
Include comment text in token
2023-01-06 23:29:20 +09:00
harupy
2dfd053bed
Implement Default for Location
2023-01-05 22:48:47 +09:00
harupy
7f552e4594
Address comments
2023-01-05 18:24:54 +09:00
harupy
9efa872023
Use try_from
2023-01-05 01:18:30 +09:00
harupy
fd8468c5eb
Simplify string check
2023-01-04 23:57:50 +09:00
Jeong YunWon
958c7e33ad
Merge pull request #4417 from harupy/add-with-offset-methods
...
Add `with_col_offset` and `with_row_offset` to `Location` for conveniece
2023-01-04 17:07:31 +09:00
Jeong YunWon
0f311cd5e5
Merge pull request #4413 from harupy/more-generic-window-impl
...
Update `CharWindow` in `compiler/parser/src/lexer.rs` to allow slicing
2023-01-04 17:06:06 +09:00
harupy
0365752bf3
Use Self
2023-01-04 13:19:23 +09:00
harupy
6d140426c1
Add with_col_offset and with_row_offset to Location
2023-01-04 13:16:03 +09:00
harupy
33a62789f7
Address comment
2023-01-04 00:03:18 +09:00
harupy
84dff79ddc
Remove incorrect EmptyExpression in parse_formatted_value
2023-01-03 23:55:50 +09:00
harupy
300710f7db
Improve CharWindow
2023-01-03 17:27:35 +09:00
harupy
515dceb07b
Remove repetitive to_string in parse_escaped_char
2023-01-03 14:48:00 +09:00
Jeong YunWon
2858c315bf
Merge pull request #4409 from harupy/improve-error-conversion-in-string-parser
...
Improve error conversion in `string_parsers.rs`
2023-01-03 14:39:01 +09:00
harupy
3b0fd61b3b
Fix clippy error
2023-01-03 12:51:31 +09:00
harupy
9030679193
Improve error conversion in string_parsers.rs
2023-01-03 12:46:10 +09:00
Jeong YunWon
71ba4226c1
Merge pull request #4405 from harupy/use-drain
...
Use `drain` to simplify `compiler/parser/src/string_parser.rs`
2023-01-03 12:41:23 +09:00
Jeong YunWon
3d03439618
Merge pull request #4399 from branai/shell-continuing-fix
...
Fix IndentationError works differently with cpython in interective shell
2023-01-03 04:32:03 +09:00
harupy
39f410c909
Use drain
2023-01-03 01:23:44 +09:00
Jim Fasarakis-Hilliard
4222b13e6c
Merge pull request #4404 from harupy/merge-match-arms
...
Merge match arms in `StringParser.parse_formatted_value`
2023-01-02 18:18:40 +02:00
Jim Fasarakis-Hilliard
d691527ead
Merge pull request #4402 from harupy/remove-unreachable-if
...
Remove unreachable code in `compiler/parser/src/string_parser.rs`
2023-01-02 17:05:45 +02:00
harupy
40e2150f7e
Merge match arms in parse_formatted_value
2023-01-02 23:16:51 +09:00
harupy
92bf96608c
Fix match
2023-01-02 22:54:48 +09:00
harupy
fac6a857f6
Simplify code using match
2023-01-02 22:26:09 +09:00
harupy
68586f8e3c
Remove unreachable code in compiler/parser/src/string_parser.rs
2023-01-02 20:48:40 +09:00
Jeong YunWon
f8787a9377
Move (c)format basic implementations to rustpython-common
2023-01-02 20:21:36 +09:00
Bijan Naimi
530a20cc96
forgot to add formatted errors.rs
2023-01-01 17:28:49 -08:00
Bijan Naimi
6cb57b2075
changed the shell logic for handling indents
2023-01-01 15:41:51 -08:00
Dimitris Fasarakis Hilliard
92b2574d52
Move tests for with into parser.
2023-01-01 21:36:07 +02:00
Jim Fasarakis-Hilliard
95fb938bd6
Merge pull request #4389 from harupy/4384-follow-up
...
Follow-up for #4384
2023-01-01 14:53:54 +02:00
harupy
9683314264
Remove unreachable code
2023-01-01 17:43:25 +09:00
harupy
d9bbeeb9b3
Fix NamedExpr location
2022-12-31 23:32:08 +09:00
harupy
68116d5c11
Move tests
2022-12-31 12:15:33 +09:00
Jeong YunWon
e164a41723
Merge pull request #4373 from andersk/named
...
Allow named expression in subscript and set comprehension
2022-12-31 10:58:09 +09:00
Jeong YunWon
aa32a73c5b
Merge pull request #4379 from harupy/refactor-FStringParser
...
Refactor `FStringParser`
2022-12-31 10:56:52 +09:00
Jeong YunWon
9d7d629cef
Merge pull request #4384 from harupy/parse-formatted-value
...
Fix the location of `FormattedValue`
2022-12-31 10:52:44 +09:00
harupy
439298e735
Fix FormattedValue location
2022-12-30 21:39:29 +09:00
anilbey
16173bf581
Update compiler/parser/src/error.rs
...
Co-authored-by: fanninpm <fanninpm@miamioh.edu >
2022-12-29 22:56:34 +01:00
Anil Tuncel
7e7b2eadee
format using cargo fmt
2022-12-29 22:15:44 +01:00
Anil Tuncel
80116c768d
arg name to be written upon duplicate kwargs error #4381
2022-12-29 22:06:41 +01:00
Nick Liu
200390c1ab
format code
2022-12-29 22:49:26 +08:00
Nick Liu
24d2ab8b0a
use is_none
2022-12-29 22:49:26 +08:00
Nick Liu
41f21a7b5d
add arg_name in duplicate argument error msg
2022-12-29 22:49:26 +08:00
Nick Liu
63e4a36e27
added check: named arguments must follow bare star
2022-12-29 22:49:26 +08:00
Nick Liu
15ab44384c
added lex error: DuplicateArguments
2022-12-29 22:49:26 +08:00
Jim Fasarakis-Hilliard
5380b85579
Merge pull request #4367 from andersk/star-order
...
Prohibit starred arguments after double-starred arguments
2022-12-29 16:27:16 +02:00
Harutaka Kawamura
b707f53f23
Update compiler/parser/src/fstring.rs
...
Co-authored-by: Zgarbul Andrey <zgarbul.andrey@gmail.com >
2022-12-29 08:10:33 +09:00
Anders Kaseorg
6439867f78
Allow named expression in set comprehension: {a := b for c in d}
...
Signed-off-by: Anders Kaseorg <andersk@mit.edu >
2022-12-28 09:57:32 -08:00
Anders Kaseorg
1904d095f9
Allow named expression in subscript: a[b := c]
...
Signed-off-by: Anders Kaseorg <andersk@mit.edu >
2022-12-28 09:57:32 -08:00
harupy
7e8f683808
Rename
2022-12-29 01:01:41 +09:00
harupy
37b1894834
Clean up FStringParser
2022-12-29 00:08:59 +09:00
Jim Fasarakis-Hilliard
201d08583a
Merge pull request #4377 from andersk/duplicate-from
...
Remove duplicate declaration of "from" token
2022-12-28 11:19:54 +02:00
Anders Kaseorg
107b2e11ae
Remove duplicate declaration of "from" token
...
Signed-off-by: Anders Kaseorg <andersk@mit.edu >
2022-12-27 18:56:24 -08:00
Anders Kaseorg
661b210391
Prohibit starred arguments after double-starred arguments
...
CPython prohibits ‘f(**kwargs, *args)’; we should too.
Signed-off-by: Anders Kaseorg <andersk@mit.edu >
2022-12-27 12:49:50 -08:00
harupy
9aed9143fe
Refactor parse_formatted_value
2022-12-28 00:23:25 +09:00
harupy
c21d0d9283
Fix make_tokenizer_located
2022-12-27 21:54:18 +09:00
Jeong YunWon
313fd7d28c
Merge pull request #4359 from yt2b/check_bom
...
Add BOM check
2022-12-26 16:03:51 +09:00
Jeong YunWon
3aa0096212
Merge pull request #4358 from harupy/fix-slice-location
...
Fix `Slice` location
2022-12-26 16:03:09 +09:00
Jeong YunWon
53dec88029
Merge pull request #4356 from andersk/with-tuple-named
...
Fix parsing of tuple with named expression as context manager
2022-12-26 16:02:17 +09:00
yt2b
bd0c15d34e
Fix comment
2022-12-26 09:30:12 +09:00
yt2b
ce0be73841
Add BOM check
2022-12-25 11:15:29 +09:00
harupy
b2ac4f60f1
Fix slice location
2022-12-25 09:37:07 +09:00
Anders Kaseorg
1fdfa5fe1b
Fix parsing of tuple with named expression as context manager
...
Because the ‘with’ item grammar disallows named expressions, CPython
parses ‘with (a := 0, b := 1):’ as a tuple rather than two ‘with’
items.
Signed-off-by: Anders Kaseorg <andersk@mit.edu >
2022-12-24 13:15:53 -08:00
harupy
c0f390ebc6
Fix IfExp location
2022-12-25 00:33:02 +09:00
Anders Kaseorg
c387b5d523
Simplify parenthesized context manager parsing with LALRPOP conditions
...
Signed-off-by: Anders Kaseorg <andersk@mit.edu >
2022-12-24 23:13:10 +09:00
harupy
ea95e1a715
Fix the location of BinOp
2022-12-21 22:05:05 +09:00
Jim Fasarakis-Hilliard
c16e08d59b
Merge pull request #4340 from harupy/fix-locations-of-parethesized-expressions
...
Fix the start and end locations of `Tuple`
2022-12-18 15:17:40 +02:00
harupy
ff00460ff4
Fix locations of parethesized expressions
2022-12-18 20:53:30 +09:00
Anders Kaseorg
f4672e4256
Remove unnecessary boxing of ASDL product children
...
Signed-off-by: Anders Kaseorg <andersk@mit.edu >
2022-12-14 02:30:02 -08:00
Dimitris Fasarakis Hilliard
6e89b3ab1a
Fix end location in with statements.
2022-12-13 13:28:17 +02:00
Jim Fasarakis-Hilliard
6a174dae45
Merge pull request #4327 from harupy/fix-end-location-body
...
Fix end location of compound statements
2022-12-13 12:30:21 +02:00
Charlie Marsh
8936ab2f8d
Set ExprContext::Store on parenthesized with expressions
2022-12-12 09:09:15 -05:00
harupy
165b979733
Refactor
2022-12-12 22:36:34 +09:00
harupy
06c92bb899
Update snapshot
2022-12-12 22:26:03 +09:00
harupy
1a657731dd
Format
2022-12-12 22:18:26 +09:00
harupy
d8cfc7e84f
Resolve conflict
2022-12-12 22:16:46 +09:00
harupy
de2e88656e
Address comments
...
Signed-off-by: harupy <hkawamura0130@gmail.com >
2022-12-12 22:14:05 +09:00
Anders Kaseorg
052dee72b8
Parse Python 3.9+ parenthesized context managers
...
Since the upstream grammar for this is not LR(1), we abuse LALRPOP
macros and the Into/TryInto traits to build a cover grammar that
converts to either tuples or `with` items after additional validation.
It’s annoying and ugly, but something like this is basically our only
option short of switching to a more powerful parser algorithm.
Fixes #4145 .
Signed-off-by: Anders Kaseorg <andersk@mit.edu >
2022-12-12 00:47:33 -08:00
Anders Kaseorg
751f9d304f
Split and simplify some LALRPOP rules
...
Signed-off-by: Anders Kaseorg <andersk@mit.edu >
2022-12-11 22:02:08 -08:00
harupy
d6f9dd0763
Use method chaining
2022-12-12 10:24:00 +09:00
harupy
5fc8c4d0b1
Fix other compound statements
2022-12-12 01:10:42 +09:00
harupy
1b7a272b77
Fix end location of nodes containing body
2022-12-11 12:35:28 +09:00
harupy
3abdc87076
Refactor
2022-12-10 22:01:42 +09:00
harupy
99b02be35a
Fix
2022-12-10 18:45:36 +09:00
harupy
f99167d4ed
Fix plain string
2022-12-10 18:09:26 +09:00
harupy
816e1e711c
Fix the end location of an implicitly-concatenated string
2022-12-10 17:49:57 +09:00
Jeong YunWon
e2b28d07c8
Merge pull request #4310 from youknowone/fix-clippy
...
Fix nightly clippy warnings
2022-12-05 12:57:01 +09:00
Jeong YunWon
28785784b2
Fix nightly clippy warnings
2022-12-05 12:18:16 +09:00
harupy
6f6b7b2312
add tests
2022-12-04 05:59:51 +09:00
harupy
93cb05ebda
Fix location
2022-12-04 05:45:15 +09:00
yt2b
e8200ab674
use bool.then
2022-11-24 09:23:20 +09:00
yt2b
492f09298f
restore if expression
2022-11-23 11:30:54 +09:00
yt2b
35eea0b8ec
Refactor lexer functions
2022-11-22 21:10:19 +09:00
yt2b
1eeddb521e
Add test
2022-11-21 22:36:05 +09:00
Charlie Marsh
96a50810a6
Use match
2022-11-20 23:41:52 -05:00
Charlie Marsh
51b7dbb89c
Use rustc-hash
2022-11-20 15:30:19 -05:00
Charlie Marsh
28a8c3a062
Implement some minor performance optimizations
2022-11-20 13:33:20 -05:00
Bongjun Jang
4e5626dfd5
Refactor lexer struct ( #4257 )
2022-11-19 21:43:34 +09:00
Jeong YunWon
f979d8dbc3
Apply let-else statements
2022-11-10 15:39:31 +09:00
Jim Fasarakis-Hilliard
cc084b4fec
Merge pull request #4266 from charliermarsh/charlie/comments
...
Implement Tok::Comment
2022-11-07 23:20:15 +02:00
Charlie Marsh
bbeec36fdb
Set comparator start location to beginning of comparison
2022-11-07 12:24:14 -05:00
Charlie Marsh
b6c230f3ca
Implement Tok::Comment
2022-11-07 10:33:55 -05:00
dvermd
a5b59f3c9d
improve col_offset in new line and lalr
2022-10-26 21:30:51 +02:00
dvermd
d5a208ca9d
improve fstring parser
...
part of: #1671
2022-10-26 21:15:24 +02:00
dvermd
fa41a1e2f6
Fix ast types' _fields and use 0-based column
2022-10-27 03:09:38 +09:00
Charlie Marsh
952d70b9d1
Add expression context parsing
2022-10-17 15:20:33 -04:00
Charlie Marsh
02953b9fe6
Remove parse_program_tokens
2022-10-17 12:04:30 -04:00
Charlie Marsh
8adc74fe26
Expose a method to parse AST from tokens directly
2022-10-17 09:39:48 -04:00
Jeong YunWon
48c0cb5599
Merge pull request #4218 from charliermarsh/charlie/clone
...
Make AST nodes Clone-able
2022-10-17 13:47:21 +09:00
Jeong YunWon
2e33a3d0e9
Merge pull request #4223 from youknowone/nightly-clippy
...
Fix nightly clippy warnings
2022-10-17 13:28:53 +09:00
Charlie Marsh
1cc342e4ed
Add end locations to all nodes ( #4192 )
2022-10-17 13:18:30 +09:00
Charlie Marsh
519718e65d
Start simple string at quote mark
2022-10-16 11:25:46 -04:00
Charlie Marsh
5da5490b19
Make AST nodes Clone-able
2022-10-16 11:01:17 -04:00
Jeong YunWon
518cf728c3
Fix nightly clippy warnings
2022-10-16 02:38:50 +09:00
Charlie Marsh
3397737a76
Start string location at kind or quote prefix
2022-10-15 11:03:50 -04:00
dvermd
6211a3a3a8
Refactor fstrings ( #4188 )
2022-10-14 12:16:34 +09:00
fanninpm
e64d7d196d
Merge pull request #4186 from andersk/arithmetic
...
Spell “arithmetic” correctly
2022-09-30 09:29:36 -04:00
Anders Kaseorg
f8b45e48e1
Spell “arithmetic” correctly
...
Signed-off-by: Anders Kaseorg <andersk@mit.edu >
2022-09-28 17:58:30 -07:00
Jeong YunWon
f99e4789ed
temporary fix of parser build
2022-09-21 16:49:35 +09:00
Jeong YunWon
86b847204e
better parser build experience
2022-09-21 16:47:06 +09:00
Charlie Marsh
26b529f9dc
Add PartialOrd to Location
2022-09-04 16:31:43 -04:00
Jeong YunWon
b21ed24025
Merge pull request #4116 from charliermarsh/charlie/f-string
...
Avoid creating unused JoinedStr in FStringParser
2022-08-23 19:30:36 +09:00
Jeong YunWon
a414677892
remove Mode from codegen root
2022-08-23 05:15:27 +09:00
Jeong YunWon
bfe4795b6c
parser::Mode from compile::Mode
2022-08-23 05:08:24 +09:00
Jeong YunWon
40690b9761
use thiserror
2022-08-23 05:08:24 +09:00
Jeong YunWon
1fd898c14c
new_syntax_error working without full compiler
...
Fix #4100
2022-08-23 05:08:24 +09:00
Jeong YunWon
42b95a9a95
Move out CompileError to core as generic form
2022-08-23 01:30:00 +09:00
Jeong YunWon
e8230efe1a
Integrate CompileError to compiler-core::BaseError
2022-08-22 23:18:41 +09:00
Jeong YunWon
7fcc18daea
integrate CodegenError to compiler-core::Error
2022-08-22 18:43:03 +09:00
Jeong YunWon
2b7bf79d29
Integrate ParseError to compiler-core::Error
2022-08-22 16:28:08 +09:00
Jeong YunWon
904fc477f1
integrate ast::Location into compilre-core::Location
2022-08-22 08:42:20 +09:00
Jeong YunWon
bfac0355dc
Share location between compiler crates
2022-08-22 08:42:20 +09:00
Jeong YunWon
c16e650071
rustpython-bytecode -> rustpython-compiler-core
2022-08-22 08:42:20 +09:00
Jeong YunWon
acde8bb625
sort Cargo.toml dependencies
2022-08-22 08:42:20 +09:00
Jeong YunWon
7f99404618
clean up cargo.toml sort
2022-08-22 08:42:20 +09:00
Jeong YunWon
3742f9117b
Add source_path to ParseError
2022-08-22 08:42:20 +09:00
Jeong YunWon
a66902406f
Refactor Mode and partial parser/codegen for eval/exec
2022-08-22 08:42:20 +09:00
Charles Marsh
51b6571ee1
Fix f-string regression
2022-08-21 19:20:23 -04:00
Charles Marsh
2345bc895d
Avoid creating unused JoinedStr in FStringParser
2022-08-21 16:59:36 -04:00
Jeong YunWon
53c48bf6b9
reorganize compiler crates
2022-08-22 04:42:42 +09:00
Jeong YunWon
ffacac05bb
Clean up imports
2022-08-22 03:42:29 +09:00
Jeong YunWon
3bae0823f7
replace try_parse!() macro to closure
2022-08-12 08:35:34 +09:00
Jeong Yunwon
29644a30d7
Add compile::Mode::BlockExpr
2022-05-27 10:14:13 +09:00
Jeong Yunwon
07712eda58
Upgrade libraries
2022-04-18 00:09:22 +09:00
Noa
76728bb69d
Run cargo upgrade
2021-10-30 19:42:35 -05:00
Noa
721c2709c8
Migrate to 2021 edition
2021-10-21 11:46:24 -05:00
Noa
7865cace06
Add unparse feature to ast
2021-10-17 21:55:27 -05:00
Noah
a9330765fa
Add an ast optimizer
2021-04-05 09:46:30 -05:00
Noah
9941eef853
Upgrade dependencies
2021-03-25 08:06:56 -05:00
Noah
e494175cd2
Update the compiler to use the new asdl ast types
2021-01-14 12:37:37 -06:00
Noah
dd7560c494
Implement asdl in the ast and parser
2021-01-14 12:37:37 -06:00
Noah
6603b4a124
Use ahash/phf in parser
2021-01-08 17:09:09 -06:00
Noah
dab7e20a33
Use ahash in the compiler
2021-01-08 17:00:23 -06:00
Noah
13f3ec5b2b
Update other stuff to use the root of rustpython_bytecode
2020-12-19 14:59:07 -06:00
Noah
452cd308d5
Optimize the size of Instruction
2020-12-14 14:25:19 -06:00
Noah
3c469eaa26
compiler/porcelain wrapper
2020-11-07 15:43:23 -06:00
Noah
0c5bf8e470
Split the ast from the parser, remove compiler dep on parser
2020-11-07 15:43:23 -06:00
Noah
3f52fdd50a
Use disassembly for snapshot testing
2020-10-19 23:55:56 -05:00
Noah
f0c0897f23
Update compiler to output codeobj.constants, use insta for snapshot testing
2020-10-19 23:55:55 -05:00
Jeong YunWon
a35d4dc22a
clean up PyComplex
2020-08-09 21:46:16 +09:00
Noah
5fe1bd3900
Update itertools
2020-07-26 15:21:39 -05:00
Noah
3737af157f
Release 0.1.2
...
rustpython@0.1.2
rustpython-bytecode@0.1.2
rustpython-compiler@0.1.2
rustpython-derive@0.1.2
rustpython-parser@0.1.2
rustpython-vm@0.1.2
rustpython_freeze@0.1.2
rustpython_wasm@0.1.2
Generated by cargo-workspaces
2020-06-23 18:47:08 -05:00
Chris West (Faux)
ed01cd63ee
upgrade low-risk deps
2019-10-10 21:07:26 +01:00
Seo Sanghyeon
1dd57971a9
Optimize BuildMap bytecode emission
2019-10-03 00:03:08 +09:00
Noah
d1ae5a7448
Bump crate versions
2019-09-25 11:57:38 -05:00
coolreader18
e89d6febea
Make peephole optimizer a stream processor
2019-08-03 22:02:29 -05:00
Windel Bouwman
4dcad05aa0
Extend symtable module.
2019-07-19 22:05:35 +02:00
Windel Bouwman
b0d270b881
Set all versions to 0.1.0
2019-07-07 13:04:12 +02:00
Windel Bouwman
e3ccef2504
Change authors to team name.
2019-07-01 21:14:07 +02:00
Windel Bouwman
dcdd964e0c
Change underscore into hyphen
2019-07-01 21:05:29 +02:00
Windel Bouwman
107442cc06
Make bytecode crate independent of parser crate.
2019-06-30 11:42:36 +02:00
Windel Bouwman
10957879db
Move bytecode into own crate.
2019-06-30 11:01:40 +02:00
coolreader18
f0148f46d0
Split off bytecode compilation into a separate crate
2019-06-12 21:43:43 -05:00