<!-- If you're new to Python and you're not sure whether what you're experiencing is a bug, the CPython issue tracker is not the right place to seek help. Consider the following options instead: - reading the Python tutorial: https://docs.python.org/3/tutorial/ - posting in the "Users" category on discuss.python.org: https://discuss.python.org/c/users/7 - emailing the Python-list mailing list: https://mail.python.org/mailman/listinfo/python-list - searching our issue tracker (https://github.com/python/cpython/issues) to see if your problem has already been reported --> # Bug report the document exception type for the `tokenize.*` functions is `TokenError` -- however it appears it now raises `SyntaxError`: https://docs.python.org/3.12/library/tokenize.html#tokenize.TokenError > Raised when either a docstring or expression that may be split over several lines is not completed anywhere in the file, for example: ```python import io import tokenize list(tokenize.generate_tokens(io.StringIO('"""').readline)) ``` ```console $ python3.11 t.py Traceback (most recent call last): File "/home/asottile/workspace/switch-microcontroller/t.py", line 4, in <module> list(tokenize.generate_tokens(io.StringIO('"""').readline)) File "/usr/lib/python3.11/tokenize.py", line 465, in _tokenize raise TokenError("EOF in multi-line string", strstart) tokenize.TokenError: ('EOF in multi-line string', (1, 0)) ``` and ```console # python3.13 t.py Traceback (most recent call last): File "/y/pyupgrade/t.py", line 4, in <module> list(tokenize.generate_tokens(io.StringIO('"""').readline)) File "/usr/lib/python3.13/tokenize.py", line 526, in _generate_tokens_from_c_tokenizer for info in it: File "<string>", line 1 """ ^ SyntaxError: unterminated triple-quoted string literal (detected at line 1) ``` # Your environment <!-- Include as many relevant details as possible about the environment you experienced the bug in --> - CPython versions tested on: c7bf74bacd - Operating system and architecture: ubuntu 22.04 x86_64 <!-- You can freely edit this text. Remove any lines you believe are unnecessary. --> <!-- gh-linked-prs --> ### Linked PRs * gh-105399 * gh-105439 * gh-105466 * gh-105472 <!-- /gh-linked-prs -->