Files
ruff/crates/ruff_python_formatter/src/builders.rs
Dhruv Manilawala 72bf1c2880 Preview minimal f-string formatting (#9642)
## Summary

_This is preview only feature and is available using the `--preview`
command-line flag._

With the implementation of [PEP 701] in Python 3.12, f-strings can now
be broken into multiple lines, can contain comments, and can re-use the
same quote character. Currently, no other Python formatter formats the
f-strings so there's some discussion which needs to happen in defining
the style used for f-string formatting. Relevant discussion:
https://github.com/astral-sh/ruff/discussions/9785

The goal for this PR is to add minimal support for f-string formatting.
This would be to format expression within the replacement field without
introducing any major style changes.

### Newlines

The heuristics for adding newline is similar to that of
[Prettier](https://prettier.io/docs/en/next/rationale.html#template-literals)
where the formatter would only split an expression in the replacement
field across multiple lines if there was already a line break within the
replacement field.

In other words, the formatter would not add any newlines unless they
were already present i.e., they were added by the user. This makes
breaking any expression inside an f-string optional and in control of
the user. For example,

```python
# We wouldn't break this
aaaaaaaaaaa = f"asaaaaaaaaaaaaaaaa { aaaaaaaaaaaa + bbbbbbbbbbbb + ccccccccccccccc } cccccccccc"

# But, we would break the following as there's already a newline
aaaaaaaaaaa = f"asaaaaaaaaaaaaaaaa {
	aaaaaaaaaaaa + bbbbbbbbbbbb + ccccccccccccccc } cccccccccc"
```


If there are comments in any of the replacement field of the f-string,
then it will always be a multi-line f-string in which case the formatter
would prefer to break expressions i.e., introduce newlines. For example,

```python
x = f"{ # comment
    a }"
```

### Quotes

The logic for formatting quotes remains unchanged. The existing logic is
used to determine the necessary quote char and is used accordingly.

Now, if the expression inside an f-string is itself a string like, then
we need to make sure to preserve the existing quote and not change it to
the preferred quote unless it's 3.12. For example,

```python
f"outer {'inner'} outer"

# For pre 3.12, preserve the single quote
f"outer {'inner'} outer"

# While for 3.12 and later, the quotes can be changed
f"outer {"inner"} outer"
```

But, for triple-quoted strings, we can re-use the same quote char unless
the inner string is itself a triple-quoted string.

```python
f"""outer {"inner"} outer"""  # valid
f"""outer {'''inner'''} outer"""  # preserve the single quote char for the inner string
```

### Debug expressions

If debug expressions are present in the replacement field of a f-string,
then the whitespace needs to be preserved as they will be rendered as it
is (for example, `f"{ x = }"`. If there are any nested f-strings, then
the whitespace in them needs to be preserved as well which means that
we'll stop formatting the f-string as soon as we encounter a debug
expression.

```python
f"outer {   x =  !s  :.3f}"
#                  ^^
#                  We can remove these whitespaces
```

Now, the whitespace doesn't need to be preserved around conversion spec
and format specifiers, so we'll format them as usual but we won't be
formatting any nested f-string within the format specifier.

### Miscellaneous

- The
[`hug_parens_with_braces_and_square_brackets`](https://github.com/astral-sh/ruff/issues/8279)
preview style isn't implemented w.r.t. the f-string curly braces.
- The
[indentation](https://github.com/astral-sh/ruff/discussions/9785#discussioncomment-8470590)
is always relative to the f-string containing statement

## Test Plan

* Add new test cases
* Review existing snapshot changes
* Review the ecosystem changes

[PEP 701]: https://peps.python.org/pep-0701/
2024-02-16 20:28:11 +05:30

244 lines
7.6 KiB
Rust

use ruff_formatter::{write, Argument, Arguments};
use ruff_text_size::{Ranged, TextRange, TextSize};
use crate::context::{FStringState, NodeLevel, WithNodeLevel};
use crate::other::commas::has_magic_trailing_comma;
use crate::prelude::*;
/// Adds parentheses and indents `content` if it doesn't fit on a line.
pub(crate) fn parenthesize_if_expands<'ast, T>(content: &T) -> ParenthesizeIfExpands<'_, 'ast>
where
T: Format<PyFormatContext<'ast>>,
{
ParenthesizeIfExpands {
inner: Argument::new(content),
indent: true,
}
}
pub(crate) struct ParenthesizeIfExpands<'a, 'ast> {
inner: Argument<'a, PyFormatContext<'ast>>,
indent: bool,
}
impl ParenthesizeIfExpands<'_, '_> {
pub(crate) fn with_indent(mut self, indent: bool) -> Self {
self.indent = indent;
self
}
}
impl<'ast> Format<PyFormatContext<'ast>> for ParenthesizeIfExpands<'_, 'ast> {
fn fmt(&self, f: &mut Formatter<PyFormatContext<'ast>>) -> FormatResult<()> {
{
let mut f = WithNodeLevel::new(NodeLevel::ParenthesizedExpression, f);
write!(
f,
[group(&format_with(|f| {
if_group_breaks(&token("(")).fmt(f)?;
if self.indent {
soft_block_indent(&Arguments::from(&self.inner)).fmt(f)?;
} else {
Arguments::from(&self.inner).fmt(f)?;
};
if_group_breaks(&token(")")).fmt(f)
}))]
)
}
}
}
/// Provides Python specific extensions to [`Formatter`].
pub(crate) trait PyFormatterExtensions<'ast, 'buf> {
/// A builder that separates each element by a `,` and a [`soft_line_break_or_space`].
/// It emits a trailing `,` that is only shown if the enclosing group expands. It forces the enclosing
/// group to expand if the last item has a trailing `comma` and the magical comma option is enabled.
fn join_comma_separated<'fmt>(
&'fmt mut self,
sequence_end: TextSize,
) -> JoinCommaSeparatedBuilder<'fmt, 'ast, 'buf>;
}
impl<'buf, 'ast> PyFormatterExtensions<'ast, 'buf> for PyFormatter<'ast, 'buf> {
fn join_comma_separated<'fmt>(
&'fmt mut self,
sequence_end: TextSize,
) -> JoinCommaSeparatedBuilder<'fmt, 'ast, 'buf> {
JoinCommaSeparatedBuilder::new(self, sequence_end)
}
}
#[derive(Copy, Clone, Debug)]
enum Entries {
/// No previous entry
None,
/// One previous ending at the given position.
One(TextSize),
/// More than one entry, the last one ending at the specific position.
MoreThanOne(TextSize),
}
impl Entries {
fn position(self) -> Option<TextSize> {
match self {
Entries::None => None,
Entries::One(position) | Entries::MoreThanOne(position) => Some(position),
}
}
const fn is_one_or_more(self) -> bool {
!matches!(self, Entries::None)
}
const fn is_more_than_one(self) -> bool {
matches!(self, Entries::MoreThanOne(_))
}
const fn next(self, end_position: TextSize) -> Self {
match self {
Entries::None => Entries::One(end_position),
Entries::One(_) | Entries::MoreThanOne(_) => Entries::MoreThanOne(end_position),
}
}
}
#[derive(Debug, Copy, Clone, PartialEq, Eq, Default)]
pub(crate) enum TrailingComma {
/// Add a trailing comma if the group breaks and there's more than one element (or if the last
/// element has a trailing comma and the magical trailing comma option is enabled).
#[default]
MoreThanOne,
/// Add a trailing comma if the group breaks (or if the last element has a trailing comma and
/// the magical trailing comma option is enabled).
OneOrMore,
}
pub(crate) struct JoinCommaSeparatedBuilder<'fmt, 'ast, 'buf> {
result: FormatResult<()>,
fmt: &'fmt mut PyFormatter<'ast, 'buf>,
entries: Entries,
sequence_end: TextSize,
trailing_comma: TrailingComma,
}
impl<'fmt, 'ast, 'buf> JoinCommaSeparatedBuilder<'fmt, 'ast, 'buf> {
fn new(f: &'fmt mut PyFormatter<'ast, 'buf>, sequence_end: TextSize) -> Self {
Self {
fmt: f,
result: Ok(()),
entries: Entries::None,
sequence_end,
trailing_comma: TrailingComma::default(),
}
}
/// Set the trailing comma behavior for the builder. Trailing commas will only be inserted if
/// the group breaks, and will _always_ be inserted if the last element has a trailing comma
/// (and the magical trailing comma option is enabled). However, this setting dictates whether
/// trailing commas are inserted for single element groups.
pub(crate) fn with_trailing_comma(mut self, trailing_comma: TrailingComma) -> Self {
self.trailing_comma = trailing_comma;
self
}
pub(crate) fn entry<T>(
&mut self,
node: &T,
content: &dyn Format<PyFormatContext<'ast>>,
) -> &mut Self
where
T: Ranged,
{
self.entry_with_line_separator(node, content, soft_line_break_or_space())
}
pub(crate) fn entry_with_line_separator<N, Separator>(
&mut self,
node: &N,
content: &dyn Format<PyFormatContext<'ast>>,
separator: Separator,
) -> &mut Self
where
N: Ranged,
Separator: Format<PyFormatContext<'ast>>,
{
self.result = self.result.and_then(|()| {
if self.entries.is_one_or_more() {
write!(self.fmt, [token(","), separator])?;
}
self.entries = self.entries.next(node.end());
content.fmt(self.fmt)
});
self
}
#[allow(unused)]
pub(crate) fn entries<T, I, F>(&mut self, entries: I) -> &mut Self
where
T: Ranged,
F: Format<PyFormatContext<'ast>>,
I: IntoIterator<Item = (T, F)>,
{
for (node, content) in entries {
self.entry(&node, &content);
}
self
}
pub(crate) fn nodes<'a, T, I>(&mut self, entries: I) -> &mut Self
where
T: Ranged + AsFormat<PyFormatContext<'ast>> + 'a,
I: IntoIterator<Item = &'a T>,
{
for node in entries {
self.entry(node, &node.format());
}
self
}
pub(crate) fn finish(&mut self) -> FormatResult<()> {
self.result.and_then(|()| {
// If the formatter is inside an f-string expression element, and the layout
// is flat, then we don't need to add a trailing comma.
if let FStringState::InsideExpressionElement(context) =
self.fmt.context().f_string_state()
{
if context.layout().is_flat() {
return Ok(());
}
}
if let Some(last_end) = self.entries.position() {
let magic_trailing_comma = has_magic_trailing_comma(
TextRange::new(last_end, self.sequence_end),
self.fmt.options(),
self.fmt.context(),
);
// If there is a single entry, only keep the magic trailing comma, don't add it if
// it wasn't there -- unless the trailing comma behavior is set to one-or-more.
if magic_trailing_comma
|| self.trailing_comma == TrailingComma::OneOrMore
|| self.entries.is_more_than_one()
{
if_group_breaks(&token(",")).fmt(self.fmt)?;
}
if magic_trailing_comma {
expand_parent().fmt(self.fmt)?;
}
}
Ok(())
})
}
}