interpolated strings: align f-string and t-string diagnostics with CPython - #8601

…dd__

`str.__add__` and `Template.__add__` built their TypeError with
`PyType::name()`, which drops the module, where CPython formats `tp_name`:

    >>> t"a" + "b"
    TypeError: can only concatenate Template (not 'str') to Template

CPython 3.14 reports the qualified name and quotes the operand with double
quotes, as `Objects/unicodeobject.c` and `Objects/templateobject.c` do:

    TypeError: can only concatenate string.templatelib.Template
               (not "str") to string.templatelib.Template

`Template.__add__` also spelled its own name literally rather than taking it
from `PyClassDef::TP_NAME`, so the two halves of the message could drift apart.

Known gap: `slot_name()` still does not qualify a class defined in a module, so
`"a" + collections.OrderedDict()` names `OrderedDict` where CPython names
`collections.OrderedDict`. This affects both methods.

Unblocks test_template_concatenation in test_tstring.

Assisted-by: Claude Code:claude-opus-5

…ython

RustPython re-derives CPython's syntax diagnostics by scanning the source rather
than by translating ruff's parse errors, and the scanner shared by f-strings and
t-strings misread several replacement field states:

- A field that ran off the end of the literal (`f'{'`) was reported as an
  expression that failed to start, where CPython asks for the closing brace.
- A field whose expression cannot start (`f'{;'`) and one whose expression runs
  into a stray character (`f'{a;'`) both fell through to the missing brace
  message instead of CPython's two distinct ones.
- A conversion or format spec following `=` was never validated, and a format
  spec following a valid conversion was skipped entirely.
- Format specs were scanned with the `{{` escape rule that only applies to
  literal text, so nested replacement fields inside them went unchecked.
- Brackets opened inside a field were never matched, so `f'{a[4)}'` and
  `f'{3)+(4}'` fell back to a bare "invalid syntax".
- Comments inside a field were not recognised, so a `#` that swallows the
  closing brace was reported as an unterminated literal, and a comment in a
  multi-line field hid the expression behind it.
- Unterminated literals were named "string" whatever their prefix, and an
  interpolated literal left with a replacement field open reported the missing
  quote rather than the brace.

A field is only "never closed" while the tokenizer is still reading its
expression. Past the field's own `:` it is emitting literal text again, so
`f'{a:>5` runs out of input as an ordinary unterminated literal while `f'{a`
reports the brace. The field's own `:` is also not the one in a slice, a display
or a lambda, and an unclosed bracket inside the expression is what CPython names
rather than the field around it, so this walk tracks the whole delimiter stack.

Two messages were worded from the wrong branch of CPython's tokenizer. The hint
"perhaps you escaped the end quote?" is raised only where lexer.c handles a
literal without an interpolation prefix; its `%c-string` branch has just the
triple-quoted and plain forms, so pairing the hint with a prefix produced
`unterminated f-string literal (...); perhaps you escaped the end quote?`, which
CPython never emits. And a bracket mismatch inside a field dropped the
` on line %d` clause that lexer.c adds whenever `parenlinenostack[level]` differs
from the current `lineno`; the general bracket scanner in this file already
computed that suffix, so `f"""{a[\n4)}"""` now names line 1 as CPython does.

The rules these now follow are the ones CPython spells out in
Grammar/python.gram (`invalid_fstring_replacement_field` and its t-string twin)
and in Parser/lexer/lexer.c, which likewise parameterises the literal's prefix
rather than duplicating the messages.

Only a field's expression is code. A format spec is text, so a `(` there opens
nothing and a `#` selects the alternate form instead of starting a comment;
inside the expression a `#` does start one. Scanning the whole field for either
therefore reported diagnostics against valid literals, and because these
scanners only run once the source has already failed to parse, that let a good
literal take the blame for an error further down the file. Expression-level
checks now stop at the field's top-level separator and skip comments, and the
literal is walked field by field rather than byte by byte so that a `{` inside a
comment no longer opens one. A pre-existing instance of the same bug in the
line-continuation check is fixed along with them.

Ruff reports a mixed literal concatenation only as a bytes/non-bytes mix, so
`t"x" b"y"` arrived as a bytes error where CPython names the t-string. CPython's
`invalid_string_tstring_concat` is an `invalid_` rule, reached only on the error
pass, and `strings` tries `(fstring|string)+` ahead of it: that alternative
consumes the concatenation's leading run of non-t-string literals, so a mix among
those raises from `_PyPegen_concatenate_strings` on the first pass and sets
`error_indicator`, which suppresses the t-string rule. The t-string message
therefore wins for `t"x" b"y"` and the bytes message keeps precedence for
`"a" b"b" t"c"`. Both are now settled here rather than left to whichever scanner
runs next, which had been answering the second case with an unrelated
"Is this intended to be part of the string?".

A leading doubled `=` or `!` was read as a marker with an empty expression before it,
so `f"{==a}"` reported `valid expression required before '='` where CPython has no
expression to report at all. The same lookahead now guards that branch, and a
doubled `=` or `!` counts as an expression that cannot start.

A top-level `=` or `!` was read as a debug or conversion marker without checking
whether it belonged to a longer operator, so `f"{a==b}"` and `f"{a!=b}"` were also
read as fields ending early. CPython tokenises `==`, `!=`, `<=` and the rest as
single tokens before it ever considers the debug marker, so the separator scan now
skips a `=` that follows one of `= ! < > + - * / % & | ^ @ :` or precedes another
`=`, and a `!` that precedes one.

An expression cannot end on an operator that still wants an operand, and CPython
points at that operator rather than at the brace. `f"{a==}"`, `f"{a and}"` and
`f"{a.b.}"` fell through to a plain `invalid syntax`; they now carry the same
message and column CPython gives, reusing the scan that already handled `;` and
`$` for exactly this shape. `is not` and `not in` are single operators, so the
first word is what gets pointed at, and `...` is consumed in whole triples so
that `f"{....}"` blames the fourth dot rather than the first. The region has to
be walked forwards: a comment's terminating newline is whitespace, so trimming
backwards from the end would step into the comment body and read a triple-quoted
field whose comment ends in `+` as an expression ending in `+`.

That check runs only once the field is known to close, which is where a stray
character and a dangling operator part ways. A stray character is a finished
token, so the parser rejects it on the lookahead in
`annotated_rhs !('='|'!'|':'|'}')` and `f"{a;"` gets the separator message even
with no closing brace. A dangling operator instead makes the parser ask for one
more token, and producing it runs into the literal's closing quote, where
lexer.c answers from its `INSIDE_FSTRING(tok)` branch with
`%c-string: expecting '}'` before any `invalid_` rule is reached. So `f"{a and"`
is a missing brace while `f"{a and}"` names the operator.

A leading `.` was read as a character that cannot start an expression, so the
Ellipsis literal `f"{...}"`, the float `f"{.5}"` and its signed form `f"{-.5}"`
were reported as broken whenever the file failed to parse somewhere else.

`pegen` joins the cpython spelling dictionary, for the `_PyPegen_*` names these
comments cite.

Known gaps, none covered by either test file. Needing the longest valid expression
prefix, which a character scanner cannot compute: a starred tuple (`f"{*a,}"`) and
a dict-unpacking display are read as unable to start an expression, and
`f"{a===b}"`, `f"{a b}"` and `f"{a,,}"` fall through to a bare `invalid syntax`.
The caret still differs from CPython's on `expecting a valid expression after '{'`,
`expecting '}'`, `expecting '}', or format specs` and `unterminated ... literal`,
and on the messages CPython points at a whole token for, such as
`f"{lambda x: x}"` and `f"{x! r}"`. A quote inside a format spec is still treated
as a string delimiter.

Two more are reachable only through a t-string concatenation that ruff does not
report as a bytes mix, so `mixed_tstring_literal_error` never runs: three or more
literals with no bytes literal among them (`t"a" t"b" "c"`, `"a" "b" t"c"`) answer
with `invalid syntax. Is this intended to be part of the string?`, and so does a
tokenizer-level nesting overflow where CPython has
`too many nested f-strings or t-strings`. Two-literal mixes are correct because
ruff's own error carries the wording.

Two pre-existing bugs outside this change are worth naming, since it touches
their neighbourhood. `unterminated triple-quoted ... literal` reports
`detected at line 2` for a one-line source where CPython reports line 1, for every
prefix including none, so it is in the shared line arithmetic rather than the
interpolated path. And `str.__add__`'s message is unreachable for an operand that
defines `__radd__`: `"a" + 1` falls through to the generic
`unsupported operand type(s)` from the binop dispatch, where CPython has
`can only concatenate str (not "int") to str`.

Unblocks test_syntax_errors and test_literal_concatenation in test_tstring, and
test_comments,
test_conversions, test_invalid_syntax_error_message, test_mismatched_braces,
test_mismatched_parens, test_parens_in_expressions and
test_syntax_error_after_debug in test_fstring.

Assisted-by: Claude Code:claude-opus-5