dfd6ed0524
## Summary
This PR adds the possibility to write mdtests that specify external
dependencies in a `project` section of TOML blocks. For example, here is
a test that makes sure that we understand Pydantic's dataclass-transform
setup:
````markdown
```toml
[environment]
python-version = "3.12"
python-platform = "linux"
[project]
dependencies = ["pydantic==2.12.2"]
```
```py
from pydantic import BaseModel
class User(BaseModel):
id: int
name: str
user = User(id=1, name="Alice")
reveal_type(user.id) # revealed: int
reveal_type(user.name) # revealed: str
# error: [missing-argument] "No argument provided for required parameter
`name`"
invalid_user = User(id=2)
```
````
## How?
Using the `python-version` and the `dependencies` fields from the
Markdown section, we generate a `pyproject.toml` file, write it to a
temporary directory, and use `uv sync` to install the dependencies into
a virtual environment. We then copy the Python source files from that
venv's `site-packages` folder to a corresponding directory structure in
the in-memory filesystem. Finally, we configure the search paths
accordingly, and run the mdtest as usual.
I fully understand that there are valid concerns here:
* Doesn't this require network access? (yes, it does)
* Is this fast enough? (`uv` caching makes this almost unnoticeable,
actually)
* Is this deterministic? ~~(probably not, package resolution can depend
on the platform you're on)~~ (yes, hopefully)
For this reason, this first version is opt-in, locally. ~~We don't even
run these tests in CI (even though they worked fine in a previous
iteration of this PR).~~ You need to set `MDTEST_EXTERNAL=1`, or use the
new `-e/--enable-external` command line option of the `mdtest.py`
runner. For example:
```bash
# Skip mdtests with external dependencies (default):
uv run crates/ty_python_semantic/mdtest.py
# Run all mdtests, including those with external dependencies:
uv run crates/ty_python_semantic/mdtest.py -e
# Only run the `pydantic` tests. Use `-e` to make sure it is not skipped:
uv run crates/ty_python_semantic/mdtest.py -e pydantic
```
## Why?
I believe that this can be a useful addition to our testing strategy,
which lies somewhere between ecosystem tests and normal mdtests.
Ecosystem tests cover much more code, but they have the disadvantage
that we only see second- or third-order effects via diagnostic diffs. If
we unexpectedly gain or lose type coverage somewhere, we might not even
notice (assuming the gradual guarantee holds, and ecosystem code is
mostly correct). Another disadvantage of ecosystem checks is that they
only test checked-in code that is usually correct. However, we also want
to test what happens on wrong code, like the code that is momentarily
written in an editor, before fixing it. On the other end of the spectrum
we have normal mdtests, which have the disadvantage that they do not
reflect the reality of complex real-world code. We experience this
whenever we're surprised by an ecosystem report on a PR.
That said, these tests should not be seen as a replacement for either of
these things. For example, we should still strive to write detailed
self-contained mdtests for user-reported issues. But we might use this
new layer for regression tests, or simply as a debugging tool. It can
also serve as a tool to document our support for popular third-party
libraries.
## Test Plan
* I've been locally using this for a couple of weeks now.
* `uv run crates/ty_python_semantic/mdtest.py -e`
1051 lines
41 KiB
YAML
1051 lines
41 KiB
YAML
name: CI
|
|
|
|
permissions: {}
|
|
|
|
on:
|
|
push:
|
|
branches: [main]
|
|
pull_request:
|
|
workflow_dispatch:
|
|
|
|
concurrency:
|
|
group: ${{ github.workflow }}-${{ github.ref_name }}-${{ github.event.pull_request.number || github.sha }}
|
|
cancel-in-progress: true
|
|
|
|
defaults:
|
|
run:
|
|
shell: bash
|
|
|
|
env:
|
|
CARGO_INCREMENTAL: 0
|
|
CARGO_NET_RETRY: 10
|
|
CARGO_TERM_COLOR: always
|
|
RUSTUP_MAX_RETRIES: 10
|
|
PACKAGE_NAME: ruff
|
|
PYTHON_VERSION: "3.14"
|
|
NEXTEST_PROFILE: ci
|
|
# Enable mdtests that require external dependencies
|
|
MDTEST_EXTERNAL: "1"
|
|
|
|
jobs:
|
|
determine_changes:
|
|
name: "Determine changes"
|
|
runs-on: ubuntu-latest
|
|
outputs:
|
|
# Flag that is raised when any code that affects parser is changed
|
|
parser: ${{ steps.check_parser.outputs.changed }}
|
|
# Flag that is raised when any code that affects linter is changed
|
|
linter: ${{ steps.check_linter.outputs.changed }}
|
|
# Flag that is raised when any code that affects formatter is changed
|
|
formatter: ${{ steps.check_formatter.outputs.changed }}
|
|
# Flag that is raised when any code is changed
|
|
# This is superset of the linter and formatter
|
|
code: ${{ steps.check_code.outputs.changed }}
|
|
# Flag that is raised when any code that affects the fuzzer is changed
|
|
fuzz: ${{ steps.check_fuzzer.outputs.changed }}
|
|
# Flag that is set to "true" when code related to ty changes.
|
|
ty: ${{ steps.check_ty.outputs.changed }}
|
|
# Flag that is set to "true" when code related to the py-fuzzer folder changes.
|
|
py-fuzzer: ${{ steps.check_py_fuzzer.outputs.changed }}
|
|
# Flag that is set to "true" when code related to the playground changes.
|
|
playground: ${{ steps.check_playground.outputs.changed }}
|
|
steps:
|
|
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
|
with:
|
|
fetch-depth: 0
|
|
persist-credentials: false
|
|
|
|
- name: Determine merge base
|
|
id: merge_base
|
|
env:
|
|
BASE_REF: ${{ github.event.pull_request.base.ref || 'main' }}
|
|
run: |
|
|
sha=$(git merge-base HEAD "origin/${BASE_REF}")
|
|
echo "sha=${sha}" >> "$GITHUB_OUTPUT"
|
|
|
|
- name: Check if the parser code changed
|
|
id: check_parser
|
|
env:
|
|
MERGE_BASE: ${{ steps.merge_base.outputs.sha }}
|
|
run: |
|
|
if git diff --quiet "${MERGE_BASE}...HEAD" -- \
|
|
':Cargo.toml' \
|
|
':Cargo.lock' \
|
|
':crates/ruff_python_trivia/**' \
|
|
':crates/ruff_source_file/**' \
|
|
':crates/ruff_text_size/**' \
|
|
':crates/ruff_python_ast/**' \
|
|
':crates/ruff_python_parser/**' \
|
|
':.github/workflows/ci.yaml' \
|
|
; then
|
|
echo "changed=false" >> "$GITHUB_OUTPUT"
|
|
else
|
|
echo "changed=true" >> "$GITHUB_OUTPUT"
|
|
fi
|
|
|
|
- name: Check if the linter code changed
|
|
id: check_linter
|
|
env:
|
|
MERGE_BASE: ${{ steps.merge_base.outputs.sha }}
|
|
run: |
|
|
if git diff --quiet "${MERGE_BASE}...HEAD" -- ':Cargo.toml' \
|
|
':Cargo.lock' \
|
|
':crates/**' \
|
|
':!crates/ty*/**' \
|
|
':!crates/ruff_python_formatter/**' \
|
|
':!crates/ruff_formatter/**' \
|
|
':!crates/ruff_dev/**' \
|
|
':scripts/*' \
|
|
':python/**' \
|
|
':.github/workflows/ci.yaml' \
|
|
; then
|
|
echo "changed=false" >> "$GITHUB_OUTPUT"
|
|
else
|
|
echo "changed=true" >> "$GITHUB_OUTPUT"
|
|
fi
|
|
|
|
- name: Check if the formatter code changed
|
|
id: check_formatter
|
|
env:
|
|
MERGE_BASE: ${{ steps.merge_base.outputs.sha }}
|
|
run: |
|
|
if git diff --quiet "${MERGE_BASE}...HEAD" -- ':Cargo.toml' \
|
|
':Cargo.lock' \
|
|
':crates/ruff_python_formatter/**' \
|
|
':crates/ruff_formatter/**' \
|
|
':crates/ruff_python_trivia/**' \
|
|
':crates/ruff_python_ast/**' \
|
|
':crates/ruff_source_file/**' \
|
|
':crates/ruff_python_index/**' \
|
|
':crates/ruff_python_index/**' \
|
|
':crates/ruff_text_size/**' \
|
|
':crates/ruff_python_parser/**' \
|
|
':scripts/*' \
|
|
':python/**' \
|
|
':.github/workflows/ci.yaml' \
|
|
; then
|
|
echo "changed=false" >> "$GITHUB_OUTPUT"
|
|
else
|
|
echo "changed=true" >> "$GITHUB_OUTPUT"
|
|
fi
|
|
|
|
- name: Check if the fuzzer code changed
|
|
id: check_fuzzer
|
|
env:
|
|
MERGE_BASE: ${{ steps.merge_base.outputs.sha }}
|
|
run: |
|
|
if git diff --quiet "${MERGE_BASE}...HEAD" -- ':Cargo.toml' \
|
|
':Cargo.lock' \
|
|
':fuzz/fuzz_targets/**' \
|
|
':.github/workflows/ci.yaml' \
|
|
; then
|
|
echo "changed=false" >> "$GITHUB_OUTPUT"
|
|
else
|
|
echo "changed=true" >> "$GITHUB_OUTPUT"
|
|
fi
|
|
|
|
- name: Check if the py-fuzzer code changed
|
|
id: check_py_fuzzer
|
|
env:
|
|
MERGE_BASE: ${{ steps.merge_base.outputs.sha }}
|
|
run: |
|
|
if git diff --quiet "${MERGE_BASE}...HEAD" -- 'python/py-fuzzer/**' \
|
|
; then
|
|
echo "changed=false" >> "$GITHUB_OUTPUT"
|
|
else
|
|
echo "changed=true" >> "$GITHUB_OUTPUT"
|
|
fi
|
|
|
|
- name: Check if there was any code related change
|
|
id: check_code
|
|
env:
|
|
MERGE_BASE: ${{ steps.merge_base.outputs.sha }}
|
|
run: |
|
|
# NOTE: Do not exclude all Markdown files here, but rather use
|
|
# specific exclude patterns like 'docs/**'), because tests for
|
|
# 'ty' are written in Markdown.
|
|
if git diff --quiet "${MERGE_BASE}...HEAD" -- \
|
|
':!docs/**' \
|
|
':!assets/**' \
|
|
; then
|
|
echo "changed=false" >> "$GITHUB_OUTPUT"
|
|
else
|
|
echo "changed=true" >> "$GITHUB_OUTPUT"
|
|
fi
|
|
|
|
- name: Check if there was any playground related change
|
|
id: check_playground
|
|
env:
|
|
MERGE_BASE: ${{ steps.merge_base.outputs.sha }}
|
|
run: |
|
|
if git diff --quiet "${MERGE_BASE}...HEAD" -- \
|
|
':playground/**' \
|
|
; then
|
|
echo "changed=false" >> "$GITHUB_OUTPUT"
|
|
else
|
|
echo "changed=true" >> "$GITHUB_OUTPUT"
|
|
fi
|
|
|
|
- name: Check if the ty code changed
|
|
id: check_ty
|
|
env:
|
|
MERGE_BASE: ${{ steps.merge_base.outputs.sha }}
|
|
run: |
|
|
if git diff --quiet "${MERGE_BASE}...HEAD" -- \
|
|
':Cargo.toml' \
|
|
':Cargo.lock' \
|
|
':crates/ty*/**' \
|
|
':crates/ruff_db/**' \
|
|
':crates/ruff_annotate_snippets/**' \
|
|
':crates/ruff_python_ast/**' \
|
|
':crates/ruff_python_parser/**' \
|
|
':crates/ruff_python_trivia/**' \
|
|
':crates/ruff_source_file/**' \
|
|
':crates/ruff_text_size/**' \
|
|
':crates/ruff_benchmark/**' \
|
|
':.github/workflows/ci.yaml' \
|
|
; then
|
|
echo "changed=false" >> "$GITHUB_OUTPUT"
|
|
else
|
|
echo "changed=true" >> "$GITHUB_OUTPUT"
|
|
fi
|
|
|
|
cargo-fmt:
|
|
name: "cargo fmt"
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 10
|
|
steps:
|
|
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
|
with:
|
|
persist-credentials: false
|
|
- name: "Install Rust toolchain"
|
|
run: rustup component add rustfmt
|
|
- run: cargo fmt --all --check
|
|
|
|
cargo-clippy:
|
|
name: "cargo clippy"
|
|
runs-on: ubuntu-latest
|
|
needs: determine_changes
|
|
if: ${{ needs.determine_changes.outputs.code == 'true' || github.ref == 'refs/heads/main' }}
|
|
timeout-minutes: 20
|
|
steps:
|
|
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
|
with:
|
|
persist-credentials: false
|
|
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
|
|
with:
|
|
save-if: ${{ github.ref == 'refs/heads/main' }}
|
|
- name: "Install Rust toolchain"
|
|
run: |
|
|
rustup component add clippy
|
|
rustup target add wasm32-unknown-unknown
|
|
- name: "Clippy"
|
|
run: cargo clippy --workspace --all-targets --all-features --locked -- -D warnings
|
|
- name: "Clippy (wasm)"
|
|
run: cargo clippy -p ruff_wasm -p ty_wasm --target wasm32-unknown-unknown --all-features --locked -- -D warnings
|
|
|
|
cargo-test-linux:
|
|
name: "cargo test (linux)"
|
|
runs-on: ${{ github.repository == 'astral-sh/ruff' && 'depot-ubuntu-22.04-16' || 'ubuntu-latest' }}
|
|
needs: determine_changes
|
|
if: ${{ !contains(github.event.pull_request.labels.*.name, 'no-test') && (needs.determine_changes.outputs.code == 'true' || github.ref == 'refs/heads/main') }}
|
|
timeout-minutes: 20
|
|
steps:
|
|
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
|
with:
|
|
persist-credentials: false
|
|
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
|
|
with:
|
|
shared-key: ruff-linux-debug
|
|
save-if: ${{ github.ref == 'refs/heads/main' }}
|
|
- name: "Install Rust toolchain"
|
|
run: rustup show
|
|
- name: "Install mold"
|
|
uses: rui314/setup-mold@725a8794d15fc7563f59595bd9556495c0564878 # v1
|
|
- name: "Install cargo nextest"
|
|
uses: taiki-e/install-action@3575e532701a5fc614b0c842e4119af4cc5fd16d # v2.62.60
|
|
with:
|
|
tool: cargo-nextest
|
|
- name: "Install cargo insta"
|
|
uses: taiki-e/install-action@3575e532701a5fc614b0c842e4119af4cc5fd16d # v2.62.60
|
|
with:
|
|
tool: cargo-insta
|
|
- name: "Install uv"
|
|
uses: astral-sh/setup-uv@1e862dfacbd1d6d858c55d9b792c756523627244 # v7.1.4
|
|
with:
|
|
enable-cache: "true"
|
|
- name: ty mdtests (GitHub annotations)
|
|
if: ${{ needs.determine_changes.outputs.ty == 'true' }}
|
|
env:
|
|
NO_COLOR: 1
|
|
MDTEST_GITHUB_ANNOTATIONS_FORMAT: 1
|
|
# Ignore errors if this step fails; we want to continue to later steps in the workflow anyway.
|
|
# This step is just to get nice GitHub annotations on the PR diff in the files-changed tab.
|
|
run: cargo test -p ty_python_semantic --test mdtest || true
|
|
- name: "Run tests"
|
|
run: cargo insta test --all-features --unreferenced reject --test-runner nextest
|
|
- name: Dogfood ty on py-fuzzer
|
|
run: uv run --project=./python/py-fuzzer cargo run -p ty check --project=./python/py-fuzzer
|
|
- name: Dogfood ty on the scripts directory
|
|
run: uv run --project=./scripts cargo run -p ty check --project=./scripts
|
|
- name: Dogfood ty on ty_benchmark
|
|
run: uv run --project=./scripts/ty_benchmark cargo run -p ty check --project=./scripts/ty_benchmark
|
|
# Check for broken links in the documentation.
|
|
- run: cargo doc --all --no-deps
|
|
env:
|
|
RUSTDOCFLAGS: "-D warnings"
|
|
# Use --document-private-items so that all our doc comments are kept in
|
|
# sync, not just public items. Eventually we should do this for all
|
|
# crates; for now add crates here as they are warning-clean to prevent
|
|
# regression.
|
|
- run: cargo doc --no-deps -p ty_python_semantic -p ty -p ty_test -p ruff_db --document-private-items
|
|
env:
|
|
# Setting RUSTDOCFLAGS because `cargo doc --check` isn't yet implemented (https://github.com/rust-lang/cargo/issues/10025).
|
|
RUSTDOCFLAGS: "-D warnings"
|
|
|
|
cargo-test-linux-release:
|
|
name: "cargo test (linux, release)"
|
|
# release builds timeout on GitHub runners, so this job is just skipped on forks in the `if` check
|
|
runs-on: depot-ubuntu-22.04-16
|
|
needs: determine_changes
|
|
if: |
|
|
github.repository == 'astral-sh/ruff' &&
|
|
!contains(github.event.pull_request.labels.*.name, 'no-test') &&
|
|
(needs.determine_changes.outputs.code == 'true' || github.ref == 'refs/heads/main')
|
|
timeout-minutes: 20
|
|
steps:
|
|
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
|
with:
|
|
persist-credentials: false
|
|
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
|
|
with:
|
|
save-if: ${{ github.ref == 'refs/heads/main' }}
|
|
- name: "Install Rust toolchain"
|
|
run: rustup show
|
|
- name: "Install mold"
|
|
uses: rui314/setup-mold@725a8794d15fc7563f59595bd9556495c0564878 # v1
|
|
- name: "Install cargo nextest"
|
|
uses: taiki-e/install-action@3575e532701a5fc614b0c842e4119af4cc5fd16d # v2.62.60
|
|
with:
|
|
tool: cargo-nextest
|
|
- name: "Install uv"
|
|
uses: astral-sh/setup-uv@1e862dfacbd1d6d858c55d9b792c756523627244 # v7.1.4
|
|
with:
|
|
enable-cache: "true"
|
|
- name: "Run tests"
|
|
run: cargo nextest run --cargo-profile profiling --all-features
|
|
- name: "Run doctests"
|
|
run: cargo test --doc --profile profiling --all-features
|
|
|
|
cargo-test-other:
|
|
strategy:
|
|
matrix:
|
|
platform:
|
|
- ${{ github.repository == 'astral-sh/ruff' && 'depot-windows-2022-16' || 'windows-latest' }}
|
|
- macos-latest
|
|
name: "cargo test (${{ matrix.platform }})"
|
|
runs-on: ${{ matrix.platform }}
|
|
needs: determine_changes
|
|
if: ${{ !contains(github.event.pull_request.labels.*.name, 'no-test') && (needs.determine_changes.outputs.code == 'true' || github.ref == 'refs/heads/main') }}
|
|
timeout-minutes: 20
|
|
steps:
|
|
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
|
with:
|
|
persist-credentials: false
|
|
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
|
|
with:
|
|
save-if: ${{ github.ref == 'refs/heads/main' }}
|
|
- name: "Install Rust toolchain"
|
|
run: rustup show
|
|
- name: "Install cargo nextest"
|
|
uses: taiki-e/install-action@3575e532701a5fc614b0c842e4119af4cc5fd16d # v2.62.60
|
|
with:
|
|
tool: cargo-nextest
|
|
- name: "Install uv"
|
|
uses: astral-sh/setup-uv@1e862dfacbd1d6d858c55d9b792c756523627244 # v7.1.4
|
|
with:
|
|
enable-cache: "true"
|
|
- name: "Run tests"
|
|
run: |
|
|
cargo nextest run --all-features --profile ci
|
|
cargo test --all-features --doc
|
|
|
|
cargo-test-wasm:
|
|
name: "cargo test (wasm)"
|
|
runs-on: ubuntu-latest
|
|
needs: determine_changes
|
|
if: ${{ !contains(github.event.pull_request.labels.*.name, 'no-test') && (needs.determine_changes.outputs.code == 'true' || github.ref == 'refs/heads/main') }}
|
|
timeout-minutes: 10
|
|
steps:
|
|
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
|
with:
|
|
persist-credentials: false
|
|
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
|
|
with:
|
|
save-if: ${{ github.ref == 'refs/heads/main' }}
|
|
- name: "Install Rust toolchain"
|
|
run: rustup target add wasm32-unknown-unknown
|
|
- uses: actions/setup-node@2028fbc5c25fe9cf00d9f06a71cc4710d4507903 # v6.0.0
|
|
with:
|
|
node-version: 22
|
|
cache: "npm"
|
|
cache-dependency-path: playground/package-lock.json
|
|
- uses: jetli/wasm-pack-action@0d096b08b4e5a7de8c28de67e11e945404e9eefa # v0.4.0
|
|
with:
|
|
version: v0.13.1
|
|
- name: "Test ruff_wasm"
|
|
run: |
|
|
cd crates/ruff_wasm
|
|
wasm-pack test --node
|
|
- name: "Test ty_wasm"
|
|
run: |
|
|
cd crates/ty_wasm
|
|
wasm-pack test --node
|
|
|
|
cargo-build-msrv:
|
|
name: "cargo build (msrv)"
|
|
runs-on: ${{ github.repository == 'astral-sh/ruff' && 'depot-ubuntu-latest-8' || 'ubuntu-latest' }}
|
|
needs: determine_changes
|
|
if: ${{ !contains(github.event.pull_request.labels.*.name, 'no-test') && (needs.determine_changes.outputs.code == 'true' || github.ref == 'refs/heads/main') }}
|
|
timeout-minutes: 20
|
|
steps:
|
|
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
|
with:
|
|
persist-credentials: false
|
|
- uses: SebRollen/toml-action@b1b3628f55fc3a28208d4203ada8b737e9687876 # v1.2.0
|
|
id: msrv
|
|
with:
|
|
file: "Cargo.toml"
|
|
field: "workspace.package.rust-version"
|
|
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
|
|
with:
|
|
save-if: ${{ github.ref == 'refs/heads/main' }}
|
|
- name: "Install Rust toolchain"
|
|
env:
|
|
MSRV: ${{ steps.msrv.outputs.value }}
|
|
run: rustup default "${MSRV}"
|
|
- name: "Install mold"
|
|
uses: rui314/setup-mold@725a8794d15fc7563f59595bd9556495c0564878 # v1
|
|
- name: "Build tests"
|
|
env:
|
|
MSRV: ${{ steps.msrv.outputs.value }}
|
|
run: cargo "+${MSRV}" test --no-run --all-features
|
|
|
|
cargo-fuzz-build:
|
|
name: "cargo fuzz build"
|
|
runs-on: ubuntu-latest
|
|
needs: determine_changes
|
|
if: ${{ github.ref == 'refs/heads/main' || needs.determine_changes.outputs.fuzz == 'true' || needs.determine_changes.outputs.code == 'true' }}
|
|
timeout-minutes: 10
|
|
steps:
|
|
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
|
with:
|
|
persist-credentials: false
|
|
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
|
|
with:
|
|
workspaces: "fuzz -> target"
|
|
save-if: ${{ github.ref == 'refs/heads/main' }}
|
|
- name: "Install Rust toolchain"
|
|
run: rustup show
|
|
- name: "Install mold"
|
|
uses: rui314/setup-mold@725a8794d15fc7563f59595bd9556495c0564878 # v1
|
|
- name: "Install cargo-binstall"
|
|
uses: cargo-bins/cargo-binstall@3fc81674af4165a753833a94cae9f91d8849049f # v1.16.2
|
|
- name: "Install cargo-fuzz"
|
|
# Download the latest version from quick install and not the github releases because github releases only has MUSL targets.
|
|
run: cargo binstall cargo-fuzz --force --disable-strategies crate-meta-data --no-confirm
|
|
- run: cargo fuzz build -s none
|
|
|
|
fuzz-parser:
|
|
name: "fuzz parser"
|
|
runs-on: ubuntu-latest
|
|
needs: determine_changes
|
|
if: ${{ !contains(github.event.pull_request.labels.*.name, 'no-test') && (needs.determine_changes.outputs.parser == 'true' || needs.determine_changes.outputs.py-fuzzer == 'true') }}
|
|
timeout-minutes: 20
|
|
env:
|
|
FORCE_COLOR: 1
|
|
steps:
|
|
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
|
with:
|
|
persist-credentials: false
|
|
- uses: astral-sh/setup-uv@1e862dfacbd1d6d858c55d9b792c756523627244 # v7.1.4
|
|
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
|
|
with:
|
|
shared-key: ruff-linux-debug
|
|
save-if: false
|
|
- name: "Install Rust toolchain"
|
|
run: rustup show
|
|
- name: Build Ruff binary
|
|
run: cargo build --bin ruff
|
|
- name: Fuzz
|
|
run: |
|
|
(
|
|
uv run \
|
|
--python="${PYTHON_VERSION}" \
|
|
--project=./python/py-fuzzer \
|
|
--locked \
|
|
fuzz \
|
|
--test-executable=target/debug/ruff \
|
|
--bin=ruff \
|
|
0-500
|
|
)
|
|
|
|
scripts:
|
|
name: "test scripts"
|
|
runs-on: ubuntu-latest
|
|
needs: determine_changes
|
|
if: ${{ !contains(github.event.pull_request.labels.*.name, 'no-test') && (needs.determine_changes.outputs.code == 'true' || github.ref == 'refs/heads/main') }}
|
|
timeout-minutes: 5
|
|
steps:
|
|
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
|
with:
|
|
persist-credentials: false
|
|
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
|
|
with:
|
|
save-if: ${{ github.ref == 'refs/heads/main' }}
|
|
- uses: astral-sh/setup-uv@1e862dfacbd1d6d858c55d9b792c756523627244 # v7.1.4
|
|
- name: "Install Rust toolchain"
|
|
run: rustup component add rustfmt
|
|
# Run all code generation scripts, and verify that the current output is
|
|
# already checked into git.
|
|
- run: python crates/ruff_python_ast/generate.py
|
|
- run: python crates/ruff_python_formatter/generate.py
|
|
- run: test -z "$(git status --porcelain)"
|
|
# Verify that adding a plugin or rule produces clean code.
|
|
- run: ./scripts/add_rule.py --name DoTheThing --prefix F --code 999 --linter pyflakes
|
|
- run: cargo check
|
|
- run: |
|
|
./scripts/add_plugin.py test --url https://pypi.org/project/-test/0.1.0/ --prefix TST
|
|
./scripts/add_rule.py --name FirstRule --prefix TST --code 001 --linter test
|
|
- run: cargo check
|
|
# Lint/format/type-check py-fuzzer
|
|
# (dogfooding with ty is done in a separate job)
|
|
- run: uv run --directory=./python/py-fuzzer mypy
|
|
- run: uv run --directory=./python/py-fuzzer ruff format --check
|
|
- run: uv run --directory=./python/py-fuzzer ruff check
|
|
|
|
ecosystem:
|
|
name: "ecosystem"
|
|
runs-on: ${{ github.repository == 'astral-sh/ruff' && 'depot-ubuntu-latest-8' || 'ubuntu-latest' }}
|
|
needs: determine_changes
|
|
# Only runs on pull requests, since that is the only we way we can find the base version for comparison.
|
|
# Ecosystem check needs linter and/or formatter changes.
|
|
if: ${{ !contains(github.event.pull_request.labels.*.name, 'no-test') && github.event_name == 'pull_request' && needs.determine_changes.outputs.code == 'true' }}
|
|
timeout-minutes: 20
|
|
steps:
|
|
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
|
with:
|
|
ref: ${{ github.event.pull_request.base.ref }}
|
|
persist-credentials: false
|
|
|
|
- uses: astral-sh/setup-uv@1e862dfacbd1d6d858c55d9b792c756523627244 # v7.1.4
|
|
with:
|
|
python-version: ${{ env.PYTHON_VERSION }}
|
|
activate-environment: true
|
|
|
|
- name: "Install Rust toolchain"
|
|
run: rustup show
|
|
|
|
- name: "Install mold"
|
|
uses: rui314/setup-mold@725a8794d15fc7563f59595bd9556495c0564878 # v1
|
|
|
|
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
|
|
with:
|
|
shared-key: ruff-linux-debug
|
|
save-if: false
|
|
|
|
- name: Build baseline version
|
|
run: |
|
|
cargo build --bin ruff
|
|
mv target/debug/ruff target/debug/ruff-baseline
|
|
|
|
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
|
with:
|
|
persist-credentials: false
|
|
clean: false
|
|
|
|
- name: Build comparison version
|
|
run: cargo build --bin ruff
|
|
|
|
- name: Install ruff-ecosystem
|
|
run: |
|
|
uv pip install ./python/ruff-ecosystem
|
|
|
|
- name: Run `ruff check` stable ecosystem check
|
|
if: ${{ needs.determine_changes.outputs.linter == 'true' }}
|
|
run: |
|
|
# Set pipefail to avoid hiding errors with tee
|
|
set -eo pipefail
|
|
|
|
ruff-ecosystem check ./target/debug/ruff-baseline ./target/debug/ruff --cache ./checkouts --output-format markdown | tee ecosystem-result-check-stable
|
|
|
|
cat ecosystem-result-check-stable > "$GITHUB_STEP_SUMMARY"
|
|
echo "### Linter (stable)" > ecosystem-result
|
|
cat ecosystem-result-check-stable >> ecosystem-result
|
|
echo "" >> ecosystem-result
|
|
|
|
- name: Run `ruff check` preview ecosystem check
|
|
if: ${{ needs.determine_changes.outputs.linter == 'true' }}
|
|
run: |
|
|
# Set pipefail to avoid hiding errors with tee
|
|
set -eo pipefail
|
|
|
|
ruff-ecosystem check ./target/debug/ruff-baseline ./target/debug/ruff --cache ./checkouts --output-format markdown --force-preview | tee ecosystem-result-check-preview
|
|
|
|
cat ecosystem-result-check-preview > "$GITHUB_STEP_SUMMARY"
|
|
echo "### Linter (preview)" >> ecosystem-result
|
|
cat ecosystem-result-check-preview >> ecosystem-result
|
|
echo "" >> ecosystem-result
|
|
|
|
- name: Run `ruff format` stable ecosystem check
|
|
if: ${{ needs.determine_changes.outputs.formatter == 'true' }}
|
|
run: |
|
|
# Set pipefail to avoid hiding errors with tee
|
|
set -eo pipefail
|
|
|
|
ruff-ecosystem format ./target/debug/ruff-baseline ./target/debug/ruff --cache ./checkouts --output-format markdown | tee ecosystem-result-format-stable
|
|
|
|
cat ecosystem-result-format-stable > "$GITHUB_STEP_SUMMARY"
|
|
echo "### Formatter (stable)" >> ecosystem-result
|
|
cat ecosystem-result-format-stable >> ecosystem-result
|
|
echo "" >> ecosystem-result
|
|
|
|
- name: Run `ruff format` preview ecosystem check
|
|
if: ${{ needs.determine_changes.outputs.formatter == 'true' }}
|
|
run: |
|
|
# Set pipefail to avoid hiding errors with tee
|
|
set -eo pipefail
|
|
|
|
ruff-ecosystem format ./target/debug/ruff-baseline ./target/debug/ruff --cache ./checkouts --output-format markdown --force-preview | tee ecosystem-result-format-preview
|
|
|
|
cat ecosystem-result-format-preview > "$GITHUB_STEP_SUMMARY"
|
|
echo "### Formatter (preview)" >> ecosystem-result
|
|
cat ecosystem-result-format-preview >> ecosystem-result
|
|
echo "" >> ecosystem-result
|
|
|
|
# NOTE: astral-sh-bot uses this artifact to post comments on PRs.
|
|
# Make sure to update the bot if you rename the artifact.
|
|
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
|
name: Upload Results
|
|
with:
|
|
name: ecosystem-result
|
|
path: ecosystem-result
|
|
|
|
fuzz-ty:
|
|
name: "Fuzz for new ty panics"
|
|
runs-on: ${{ github.repository == 'astral-sh/ruff' && 'depot-ubuntu-22.04-16' || 'ubuntu-latest' }}
|
|
needs:
|
|
- determine_changes
|
|
# Only runs on pull requests, since that is the only we way we can find the base version for comparison.
|
|
if: ${{ !contains(github.event.pull_request.labels.*.name, 'no-test') && github.event_name == 'pull_request' && (needs.determine_changes.outputs.ty == 'true' || needs.determine_changes.outputs.py-fuzzer == 'true') }}
|
|
timeout-minutes: ${{ github.repository == 'astral-sh/ruff' && 10 || 20 }}
|
|
steps:
|
|
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
|
with:
|
|
fetch-depth: 0
|
|
persist-credentials: false
|
|
- uses: astral-sh/setup-uv@1e862dfacbd1d6d858c55d9b792c756523627244 # v7.1.4
|
|
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
|
|
with:
|
|
save-if: ${{ github.ref == 'refs/heads/main' }}
|
|
- name: "Install Rust toolchain"
|
|
run: rustup show
|
|
- name: "Install mold"
|
|
uses: rui314/setup-mold@725a8794d15fc7563f59595bd9556495c0564878 # v1
|
|
- name: Fuzz
|
|
env:
|
|
FORCE_COLOR: 1
|
|
run: |
|
|
echo "new commit"
|
|
git rev-list --format=%s --max-count=1 "$GITHUB_SHA"
|
|
cargo build --profile=profiling --bin=ty
|
|
mv target/profiling/ty ty-new
|
|
|
|
MERGE_BASE="$(git merge-base "$GITHUB_SHA" "origin/$GITHUB_BASE_REF")"
|
|
git checkout -b old_commit "$MERGE_BASE"
|
|
echo "old commit (merge base)"
|
|
git rev-list --format=%s --max-count=1 old_commit
|
|
cargo build --profile=profiling --bin=ty
|
|
mv target/profiling/ty ty-old
|
|
|
|
(
|
|
uv run \
|
|
--python="${PYTHON_VERSION}" \
|
|
--project=./python/py-fuzzer \
|
|
--locked \
|
|
fuzz \
|
|
--test-executable=ty-new \
|
|
--baseline-executable=ty-old \
|
|
--only-new-bugs \
|
|
--bin=ty \
|
|
0-1000
|
|
)
|
|
|
|
cargo-shear:
|
|
name: "cargo shear"
|
|
runs-on: ubuntu-latest
|
|
needs: determine_changes
|
|
if: ${{ needs.determine_changes.outputs.code == 'true' || github.ref == 'refs/heads/main' }}
|
|
steps:
|
|
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
|
with:
|
|
persist-credentials: false
|
|
- uses: cargo-bins/cargo-binstall@3fc81674af4165a753833a94cae9f91d8849049f # v1.16.2
|
|
- run: cargo binstall --no-confirm cargo-shear
|
|
- run: cargo shear
|
|
|
|
ty-completion-evaluation:
|
|
name: "ty completion evaluation"
|
|
runs-on: ${{ github.repository == 'astral-sh/ruff' && 'depot-ubuntu-22.04-16' || 'ubuntu-latest' }}
|
|
needs: determine_changes
|
|
if: ${{ needs.determine_changes.outputs.ty == 'true' || github.ref == 'refs/heads/main' }}
|
|
steps:
|
|
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
|
with:
|
|
persist-credentials: false
|
|
- uses: astral-sh/setup-uv@1e862dfacbd1d6d858c55d9b792c756523627244 # v7.1.4
|
|
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
|
|
with:
|
|
save-if: ${{ github.ref == 'refs/heads/main' }}
|
|
- name: "Install Rust toolchain"
|
|
run: rustup show
|
|
- name: "Install mold"
|
|
uses: rui314/setup-mold@725a8794d15fc7563f59595bd9556495c0564878 # v1
|
|
- name: "Run ty completion evaluation"
|
|
run: cargo run --profile profiling --package ty_completion_eval -- all --threshold 0.4 --tasks /tmp/completion-evaluation-tasks.csv
|
|
- name: "Ensure there are no changes"
|
|
run: diff ./crates/ty_completion_eval/completion-evaluation-tasks.csv /tmp/completion-evaluation-tasks.csv
|
|
|
|
python-package:
|
|
name: "python package"
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 20
|
|
if: ${{ !contains(github.event.pull_request.labels.*.name, 'no-test') }}
|
|
steps:
|
|
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
|
with:
|
|
persist-credentials: false
|
|
- uses: actions/setup-python@83679a892e2d95755f2dac6acb0bfd1e9ac5d548 # v6.1.0
|
|
with:
|
|
python-version: ${{ env.PYTHON_VERSION }}
|
|
architecture: x64
|
|
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
|
|
with:
|
|
save-if: ${{ github.ref == 'refs/heads/main' }}
|
|
- name: "Prep README.md"
|
|
run: python scripts/transform_readme.py --target pypi
|
|
- name: "Build wheels"
|
|
uses: PyO3/maturin-action@86b9d133d34bc1b40018696f782949dac11bd380 # v1.49.4
|
|
with:
|
|
args: --out dist
|
|
- name: "Test wheel"
|
|
run: |
|
|
pip install --force-reinstall --find-links dist "${PACKAGE_NAME}"
|
|
ruff --help
|
|
python -m ruff --help
|
|
- name: "Remove wheels from cache"
|
|
run: rm -rf target/wheels
|
|
|
|
pre-commit:
|
|
name: "pre-commit"
|
|
runs-on: ${{ github.repository == 'astral-sh/ruff' && 'depot-ubuntu-22.04-16' || 'ubuntu-latest' }}
|
|
timeout-minutes: 10
|
|
steps:
|
|
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
|
with:
|
|
persist-credentials: false
|
|
- uses: astral-sh/setup-uv@1e862dfacbd1d6d858c55d9b792c756523627244 # v7.1.4
|
|
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
|
|
with:
|
|
save-if: ${{ github.ref == 'refs/heads/main' }}
|
|
- uses: actions/setup-node@2028fbc5c25fe9cf00d9f06a71cc4710d4507903 # v6.0.0
|
|
with:
|
|
node-version: 22
|
|
- name: "Cache pre-commit"
|
|
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
|
|
with:
|
|
path: ~/.cache/pre-commit
|
|
key: pre-commit-${{ hashFiles('.pre-commit-config.yaml') }}
|
|
- name: "Run pre-commit"
|
|
run: |
|
|
echo '```console' > "$GITHUB_STEP_SUMMARY"
|
|
# Enable color output for pre-commit and remove it for the summary
|
|
# Use --hook-stage=manual to enable slower pre-commit hooks that are skipped by default
|
|
SKIP=cargo-fmt,clippy,dev-generate-all uvx --python="${PYTHON_VERSION}" pre-commit run --all-files --show-diff-on-failure --color=always --hook-stage=manual | \
|
|
tee >(sed -E 's/\x1B\[([0-9]{1,2}(;[0-9]{1,2})*)?[mGK]//g' >> "$GITHUB_STEP_SUMMARY") >&1
|
|
exit_code="${PIPESTATUS[0]}"
|
|
echo '```' >> "$GITHUB_STEP_SUMMARY"
|
|
exit "$exit_code"
|
|
|
|
docs:
|
|
name: "mkdocs"
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 10
|
|
steps:
|
|
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
|
with:
|
|
persist-credentials: false
|
|
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
|
|
with:
|
|
save-if: ${{ github.ref == 'refs/heads/main' }}
|
|
- name: "Install Rust toolchain"
|
|
run: rustup show
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@1e862dfacbd1d6d858c55d9b792c756523627244 # v7.1.4
|
|
with:
|
|
python-version: 3.13
|
|
activate-environment: true
|
|
- name: "Install dependencies"
|
|
run: uv pip install -r docs/requirements.txt
|
|
- name: "Update README File"
|
|
run: python scripts/transform_readme.py --target mkdocs
|
|
- name: "Generate docs"
|
|
run: python scripts/generate_mkdocs.py
|
|
- name: "Check docs formatting"
|
|
run: python scripts/check_docs_formatted.py
|
|
- name: "Build docs"
|
|
run: mkdocs build --strict -f mkdocs.yml
|
|
|
|
check-formatter-instability-and-black-similarity:
|
|
name: "formatter instabilities and black similarity"
|
|
runs-on: ubuntu-latest
|
|
needs: determine_changes
|
|
if: ${{ !contains(github.event.pull_request.labels.*.name, 'no-test') && (needs.determine_changes.outputs.formatter == 'true' || github.ref == 'refs/heads/main') }}
|
|
timeout-minutes: 10
|
|
steps:
|
|
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
|
with:
|
|
persist-credentials: false
|
|
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
|
|
with:
|
|
save-if: ${{ github.ref == 'refs/heads/main' }}
|
|
- name: "Install Rust toolchain"
|
|
run: rustup show
|
|
- name: "Run checks"
|
|
run: scripts/formatter_ecosystem_checks.sh
|
|
- name: "Github step summary"
|
|
run: cat target/formatter-ecosystem/stats.txt > "$GITHUB_STEP_SUMMARY"
|
|
- name: "Remove checkouts from cache"
|
|
run: rm -r target/formatter-ecosystem
|
|
|
|
check-ruff-lsp:
|
|
name: "test ruff-lsp"
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 5
|
|
needs: determine_changes
|
|
if: ${{ !contains(github.event.pull_request.labels.*.name, 'no-test') && (needs.determine_changes.outputs.code == 'true' || github.ref == 'refs/heads/main') }}
|
|
steps:
|
|
- uses: extractions/setup-just@e33e0265a09d6d736e2ee1e0eb685ef1de4669ff # v3.0.0
|
|
env:
|
|
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
|
|
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
|
name: "Checkout ruff source"
|
|
with:
|
|
persist-credentials: false
|
|
|
|
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
|
|
with:
|
|
shared-key: ruff-linux-debug
|
|
save-if: false
|
|
|
|
- name: "Install Rust toolchain"
|
|
run: rustup show
|
|
|
|
- name: Build Ruff binary
|
|
run: cargo build -p ruff --bin ruff
|
|
|
|
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
|
name: "Checkout ruff-lsp source"
|
|
with:
|
|
persist-credentials: false
|
|
repository: "astral-sh/ruff-lsp"
|
|
path: ruff-lsp
|
|
|
|
- uses: actions/setup-python@83679a892e2d95755f2dac6acb0bfd1e9ac5d548 # v6.1.0
|
|
with:
|
|
# installation fails on 3.13 and newer
|
|
python-version: "3.12"
|
|
|
|
- name: Install ruff-lsp dependencies
|
|
run: |
|
|
cd ruff-lsp
|
|
just install
|
|
|
|
- name: Run ruff-lsp tests
|
|
run: |
|
|
# Setup development binary
|
|
pip uninstall --yes ruff
|
|
export PATH="${PWD}/target/debug:${PATH}"
|
|
ruff version
|
|
|
|
cd ruff-lsp
|
|
just test
|
|
|
|
check-playground:
|
|
name: "check playground"
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 5
|
|
needs:
|
|
- determine_changes
|
|
if: ${{ (needs.determine_changes.outputs.playground == 'true') }}
|
|
steps:
|
|
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
|
with:
|
|
persist-credentials: false
|
|
- name: "Install Rust toolchain"
|
|
run: rustup target add wasm32-unknown-unknown
|
|
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
|
|
with:
|
|
save-if: ${{ github.ref == 'refs/heads/main' }}
|
|
- uses: actions/setup-node@2028fbc5c25fe9cf00d9f06a71cc4710d4507903 # v6.0.0
|
|
with:
|
|
node-version: 22
|
|
cache: "npm"
|
|
cache-dependency-path: playground/package-lock.json
|
|
- uses: jetli/wasm-bindgen-action@20b33e20595891ab1a0ed73145d8a21fc96e7c29 # v0.2.0
|
|
- name: "Install Node dependencies"
|
|
run: npm ci --ignore-scripts
|
|
working-directory: playground
|
|
- name: "Build playgrounds"
|
|
run: npm run dev:wasm
|
|
working-directory: playground
|
|
- name: "Run TypeScript checks"
|
|
run: npm run check
|
|
working-directory: playground
|
|
- name: "Check formatting"
|
|
run: npm run fmt:check
|
|
working-directory: playground
|
|
|
|
benchmarks-instrumented-ruff:
|
|
name: "benchmarks instrumented (ruff)"
|
|
runs-on: ubuntu-24.04
|
|
needs: determine_changes
|
|
if: |
|
|
github.repository == 'astral-sh/ruff' &&
|
|
(
|
|
github.ref == 'refs/heads/main' ||
|
|
needs.determine_changes.outputs.formatter == 'true' ||
|
|
needs.determine_changes.outputs.linter == 'true'
|
|
)
|
|
timeout-minutes: 20
|
|
permissions:
|
|
contents: read # required for actions/checkout
|
|
id-token: write # required for OIDC authentication with CodSpeed
|
|
steps:
|
|
- name: "Checkout Branch"
|
|
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
|
with:
|
|
persist-credentials: false
|
|
|
|
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
|
|
with:
|
|
save-if: ${{ github.ref == 'refs/heads/main' }}
|
|
- uses: astral-sh/setup-uv@1e862dfacbd1d6d858c55d9b792c756523627244 # v7.1.4
|
|
|
|
- name: "Install Rust toolchain"
|
|
run: rustup show
|
|
|
|
- name: "Install codspeed"
|
|
uses: taiki-e/install-action@3575e532701a5fc614b0c842e4119af4cc5fd16d # v2.62.60
|
|
with:
|
|
tool: cargo-codspeed
|
|
|
|
- name: "Build benchmarks"
|
|
run: cargo codspeed build --features "codspeed,instrumented" --profile profiling --no-default-features -p ruff_benchmark --bench formatter --bench lexer --bench linter --bench parser
|
|
|
|
- name: "Run benchmarks"
|
|
uses: CodSpeedHQ/action@346a2d8a8d9d38909abd0bc3d23f773110f076ad # v4.4.1
|
|
with:
|
|
mode: simulation
|
|
run: cargo codspeed run
|
|
|
|
benchmarks-instrumented-ty:
|
|
name: "benchmarks instrumented (ty)"
|
|
runs-on: ubuntu-24.04
|
|
needs: determine_changes
|
|
if: |
|
|
github.repository == 'astral-sh/ruff' &&
|
|
(
|
|
github.ref == 'refs/heads/main' ||
|
|
needs.determine_changes.outputs.ty == 'true'
|
|
)
|
|
timeout-minutes: 20
|
|
permissions:
|
|
contents: read # required for actions/checkout
|
|
id-token: write # required for OIDC authentication with CodSpeed
|
|
steps:
|
|
- name: "Checkout Branch"
|
|
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
|
with:
|
|
persist-credentials: false
|
|
|
|
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
|
|
with:
|
|
save-if: ${{ github.ref == 'refs/heads/main' }}
|
|
- uses: astral-sh/setup-uv@1e862dfacbd1d6d858c55d9b792c756523627244 # v7.1.4
|
|
|
|
- name: "Install Rust toolchain"
|
|
run: rustup show
|
|
|
|
- name: "Install codspeed"
|
|
uses: taiki-e/install-action@3575e532701a5fc614b0c842e4119af4cc5fd16d # v2.62.60
|
|
with:
|
|
tool: cargo-codspeed
|
|
|
|
- name: "Build benchmarks"
|
|
run: cargo codspeed build --features "codspeed,instrumented" --profile profiling --no-default-features -p ruff_benchmark --bench ty
|
|
|
|
- name: "Run benchmarks"
|
|
uses: CodSpeedHQ/action@346a2d8a8d9d38909abd0bc3d23f773110f076ad # v4.4.1
|
|
with:
|
|
mode: simulation
|
|
run: cargo codspeed run
|
|
|
|
benchmarks-walltime:
|
|
name: "benchmarks walltime (${{ matrix.benchmarks }})"
|
|
runs-on: codspeed-macro
|
|
needs: determine_changes
|
|
if: ${{ github.repository == 'astral-sh/ruff' && !contains(github.event.pull_request.labels.*.name, 'no-test') && (needs.determine_changes.outputs.ty == 'true' || github.ref == 'refs/heads/main') }}
|
|
timeout-minutes: 20
|
|
permissions:
|
|
contents: read # required for actions/checkout
|
|
id-token: write # required for OIDC authentication with CodSpeed
|
|
strategy:
|
|
matrix:
|
|
benchmarks:
|
|
- "medium|multithreaded"
|
|
- "small|large"
|
|
steps:
|
|
- name: "Checkout Branch"
|
|
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
|
with:
|
|
persist-credentials: false
|
|
|
|
- uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2.8.2
|
|
with:
|
|
save-if: ${{ github.ref == 'refs/heads/main' }}
|
|
- uses: astral-sh/setup-uv@1e862dfacbd1d6d858c55d9b792c756523627244 # v7.1.4
|
|
|
|
- name: "Install Rust toolchain"
|
|
run: rustup show
|
|
|
|
- name: "Install codspeed"
|
|
uses: taiki-e/install-action@3575e532701a5fc614b0c842e4119af4cc5fd16d # v2.62.60
|
|
with:
|
|
tool: cargo-codspeed
|
|
|
|
- name: "Build benchmarks"
|
|
run: cargo codspeed build --features "codspeed,walltime" --profile profiling --no-default-features -p ruff_benchmark
|
|
|
|
- name: "Run benchmarks"
|
|
uses: CodSpeedHQ/action@346a2d8a8d9d38909abd0bc3d23f773110f076ad # v4.4.1
|
|
env:
|
|
# enabling walltime flamegraphs adds ~6 minutes to the CI time, and they don't
|
|
# appear to provide much useful insight for our walltime benchmarks right now
|
|
# (see https://github.com/astral-sh/ruff/pull/20419)
|
|
CODSPEED_PERF_ENABLED: false
|
|
with:
|
|
mode: walltime
|
|
run: cargo codspeed run --bench ty_walltime "${{ matrix.benchmarks }}"
|