[ty] AST garbage collection (#18482)
## Summary Garbage collect ASTs once we are done checking a given file. Queries with a cross-file dependency on the AST will reparse the file on demand. This reduces ty's peak memory usage by ~20-30%. The primary change of this PR is adding a `node_index` field to every AST node, that is assigned by the parser. `ParsedModule` can use this to create a flat index of AST nodes any time the file is parsed (or reparsed). This allows `AstNodeRef` to simply index into the current instance of the `ParsedModule`, instead of storing a pointer directly. The indices are somewhat hackily (using an atomic integer) assigned by the `parsed_module` query instead of by the parser directly. Assigning the indices in source-order in the (recursive) parser turns out to be difficult, and collecting the nodes during semantic indexing is impossible as `SemanticIndex` does not hold onto a specific `ParsedModuleRef`, which the pointers in the flat AST are tied to. This means that we have to do an extra AST traversal to assign and collect the nodes into a flat index, but the small performance impact (~3% on cold runs) seems worth it for the memory savings. Part of https://github.com/astral-sh/ty/issues/214.
This commit is contained in:
@@ -7,7 +7,11 @@ pub struct FormatModExpression;
|
||||
|
||||
impl FormatNodeRule<ModExpression> for FormatModExpression {
|
||||
fn fmt_fields(&self, item: &ModExpression, f: &mut PyFormatter) -> FormatResult<()> {
|
||||
let ModExpression { body, range: _ } = item;
|
||||
let ModExpression {
|
||||
body,
|
||||
range: _,
|
||||
node_index: _,
|
||||
} = item;
|
||||
body.format().fmt(f)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -11,7 +11,11 @@ pub struct FormatModModule;
|
||||
|
||||
impl FormatNodeRule<ModModule> for FormatModModule {
|
||||
fn fmt_fields(&self, item: &ModModule, f: &mut PyFormatter) -> FormatResult<()> {
|
||||
let ModModule { range, body } = item;
|
||||
let ModModule {
|
||||
range,
|
||||
body,
|
||||
node_index: _,
|
||||
} = item;
|
||||
|
||||
if body.is_empty() {
|
||||
// Only preserve an empty line if the source contains an empty line too.
|
||||
|
||||
Reference in New Issue
Block a user