Skip to content

ast: fix identifier interning - #8361

Merged
youknowone merged 2 commits into
RustPython:mainfrom
widehyo1:fix-ast-identifier-interning
Jul 25, 2026
Merged

ast: fix identifier interning#8361
youknowone merged 2 commits into
RustPython:mainfrom
widehyo1:fix-ast-identifier-interning

Conversation

@widehyo1

@widehyo1 widehyo1 commented Jul 25, 2026 •
edited by coderabbitai Bot
Loading

Copy link
Copy Markdown
Contributor

Summary

Ensure that identifier fields produced while converting parsed AST nodes use
interned Python strings, matching CPython behavior. In particular, this fixes
Name.id and function definition name fields.

This restores identity-sensitive behavior used by copy.replace() and removes
the three test_ast expected-failure markers that now pass. Add regression
coverage for expression, synchronous-function, and async-function identifiers.

Testing

  • uv tool run prek run --all-files
  • cargo fmt --check
  • cargo run --release Lib/test/test_ast/test_ast.py
    • 199 tests passed
    • 7 skipped
    • 21 expected failures
  • cargo run -- extra_tests/snippets/stdlib_ast.py
  • cargo clippy -p rustpython-stdlib --all-targets
  • the configured workspace test suite

AI assistance: Codex:gpt-5.6-sol

Summary by CodeRabbit

  • Bug Fixes

    • Fixed AST identifier and function name conversion so id/name values match the expected representation and preserve correct identity semantics.
    • Improved AST node copy/replace behavior to retain the original identifier and context while dropping unrelated dynamically added attributes.

  • Tests

    • Added regression coverage for identifier interning, function and async function name handling, and AST copy/replace semantics to align with CPython behavior.

@widehyo1
Fix AST identifier interning
2b3e622 
Ensure parsed name and function identifier fields use interned strings,
matching CPython behavior.

Assisted-by: Codex:gpt-5.6-sol
@coderabbitai

coderabbitai Bot commented Jul 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro Plus

Run ID: 9fbc7acd-f429-4fd4-be13-c1d915ae7aff

📥 Commits

Reviewing files that changed from the base of the PR and between 2b3e622 and dd94804.

📒 Files selected for processing (1)
  • extra_tests/snippets/stdlib_ast.py
📝 Walkthrough

Walkthrough

AST expression and function-definition name conversion now uses AST-aware serialization, with regression tests covering interned identifier identity and copy.replace behavior.

Changes

AST identifier serialization

Layer / File(s) Summary
AST-aware identifier serialization and regression coverage
crates/vm/src/stdlib/_ast/expression.rs, crates/vm/src/stdlib/_ast/statement.rs, extra_tests/snippets/stdlib_ast.py		 ExprName.id and StmtFunctionDef.name now use ast_to_object; tests verify interned identities and preservation of id/ctx during copy.replace.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is concise and accurately summarizes the main change: fixing AST identifier interning behavior.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

📦 Library Dependencies

The following Lib/ modules were modified. Here are their dependencies:

[x] lib: cpython/Lib/ast.py
[x] lib: cpython/Lib/_ast_unparse.py
[x] test: cpython/Lib/test/test_unparse.py
[x] test: cpython/Lib/test/test_type_comments.py

dependencies:

  • ast

dependent tests: (149 tests)

  • ast: test_ast test_builtin test_compile test_compiler_codegen test_dis test_fstring test_future_stmt test_peepholer test_peg_generator test_site test_ssl test_type_comments test_ucn test_unparse
    • annotationlib: test_annotationlib test_functools test_grammar test_inspect test_reprlib test_type_annotations test_type_params test_typing
      • dataclasses: test__colorize test_copy test_ctypes test_enum test_genericalias test_patma test_pprint test_pydoc test_regrtest test_zoneinfo
      • inspect: test_abc test_argparse test_asyncgen test_buffer test_clinic test_code test_collections test_coroutines test_decimal test_generators test_monitoring test_ntpath test_operator test_posixpath test_signal test_sqlite3 test_traceback test_turtle test_types test_unittest test_yield_from test_zipimport test_zipimport_support
    • dbm.dumb: test_dbm_dumb
    • inspect:
      • bdb: test_bdb test_pdb
      • cmd: test_cmd
      • importlib.metadata: test_importlib
      • pkgutil: test_pkgutil test_pyrepl test_runpy
      • rlcompleter: test_pyrepl test_rlcompleter
      • trace: test_trace
      • xmlrpc.server: test_docxmlrpc test_xmlrpc
    • pyclbr: test_pyclbr
    • traceback: test_asyncio test_code_module test_contextlib test_contextlib_async test_dictcomps test_exceptions test_http_cookiejar test_importlib test_iter test_listcomps test_pyexpat test_setcomps test_socket test_subprocess test_sys test_threadedtempfile test_threading test_unittest test_with
      • concurrent.futures.process: test_compileall test_concurrent_futures
      • http.cookiejar: test_urllib2
      • logging: test_asyncio test_hashlib test_logging test_support test_urllib2net
      • multiprocessing: test_asyncio test_concurrent_futures test_fcntl test_memoryview test_multiprocessing_main_handling test_re
      • py_compile: test_cmd_line_script test_importlib test_modulefinder test_py_compile
      • socketserver: test_imaplib test_socketserver test_wsgiref
      • threading: test_android test_asyncio test_bytes test_bz2 test_concurrent_futures test_context test_ctypes test_email test_external_inspection test_fork1 test_frame test_ftplib test_gc test_httplib test_httpservers test_importlib test_io test_ioctl test_itertools test_largefile test_linecache test_opcache test_pathlib test_poll test_poplib test_pyrepl test_queue test_robotparser test_sched test_smtplib test_super test_syslog test_termios test_threading_local test_time test_urllib2_localnet test_weakref test_winreg test_zstd
      • timeit: test_timeit

Legend:

  • [+] path exists in CPython
  • [x] up-to-date, [ ] outdated

coderabbitai[bot]
coderabbitai Bot reviewed Jul 25, 2026
View reviewed changes

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@extra_tests/snippets/stdlib_ast.py`:
- Line 43: Move the copy import from its current position after executable
top-level statements into the module’s existing top-level import block,
preserving the surrounding import ordering and leaving executable code
unchanged.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro Plus

Run ID: e7382c2a-71e2-4390-b543-ea7a719d828a

📥 Commits

Reviewing files that changed from the base of the PR and between 28454cc and 2b3e622.

⛔ Files ignored due to path filters (1)
  • Lib/test/test_ast/test_ast.py is excluded by !Lib/**
📒 Files selected for processing (3)
  • crates/vm/src/stdlib/_ast/expression.rs
  • crates/vm/src/stdlib/_ast/statement.rs
  • extra_tests/snippets/stdlib_ast.py

Comment thread extra_tests/snippets/stdlib_ast.py Outdated
@youknowone youknowone added the z-ca-2026 Tag to track Contribution Academy 2026 label Jul 25, 2026
youknowone
youknowone approved these changes Jul 25, 2026
View reviewed changes

@youknowone youknowone left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@youknowone
youknowone merged commit 4a80f2e into RustPython:main Jul 25, 2026
27 checks passed
@widehyo1
widehyo1 deleted the fix-ast-identifier-interning branch July 26, 2026 04:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@coderabbitai coderabbitai[bot] coderabbitai[bot] left review comments
@youknowone youknowone youknowone approved these changes

Assignees

No one assigned

Labels

z-ca-2026 Tag to track Contribution Academy 2026

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@widehyo1 @youknowone