Skip to content

[3.13] gh-69371: Fix pydoc for modules whose path contains undecodable bytes (GH-155239) - #156610

Open
miss-islington wants to merge 1 commit into
python:3.13from
miss-islington:backport-c8ca336-3.13
Open

[3.13] gh-69371: Fix pydoc for modules whose path contains undecodable bytes (GH-155239)#156610
miss-islington wants to merge 1 commit into
python:3.13from
miss-islington:backport-c8ca336-3.13

Conversation

@miss-islington

@miss-islington miss-islington commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

"python -m pydoc -w" and the pydoc HTTP server failed with
UnicodeEncodeError. The file URL is now created with
urllib.request.pathname2url(), which percent-encodes the path using the
filesystem encoding, and characters unencodable in the generated UTF-8
page are escaped with backslashes.
(cherry picked from commit c8ca336)

Co-authored-by: Serhiy Storchaka storchaka@gmail.com
Co-authored-by: Claude Opus 5 (1M context) noreply@anthropic.com

… bytes (pythonGH-155239)

"python -m pydoc -w" and the pydoc HTTP server failed with
UnicodeEncodeError.  The file URL is now created with
urllib.request.pathname2url(), which percent-encodes the path using the
filesystem encoding, and characters unencodable in the generated UTF-8
page are escaped with backslashes.
(cherry picked from commit c8ca336)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
@hugovk

hugovk commented Aug 30, 2026

Copy link
Copy Markdown
Member

Windows is failing with UnicodeEncodeError:

======================================================================
ERROR: test_html_doc_undecodable_path (test.test_pydoc.test_pydoc.PydocDocTest.test_html_doc_undecodable_path)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\a\cpython\cpython\Lib\test\test_pydoc\test_pydoc.py", line 1022, in test_html_doc_undecodable_path
    doc = pydoc.HTMLDoc().docmodule(mod)
  File "D:\a\cpython\cpython\Lib\pydoc.py", line 849, in docmodule
    url = urllib.request.pathname2url(path)
  File "D:\a\cpython\cpython\Lib\nturl2path.py", line 68, in pathname2url
    tail = urllib.parse.quote(comp[1])
  File "D:\a\cpython\cpython\Lib\urllib\parse.py", line 923, in quote
    string = string.encode(encoding, errors)
UnicodeEncodeError: 'utf-8' codec can't encode character '\udd00' in position 65: surrogates not allowed

@StanFromIreland

Copy link
Copy Markdown
Member

CC @serhiy-storchaka (see Hugo's comment above).

@serhiy-storchaka

Copy link
Copy Markdown
Member

This is a little complicated. pathname2url() was fixed in #85168 only on Posix. Then #125866 deprecated nturl2path and stopped using nturl2path.pathname2url() in favor of general implementation in 3.14. This is why the test was passed on 3.14+.

Applying the similar fix on nturl2path.pathname2url() and backporting it will fix the test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting review type-bug An unexpected behavior, bug, or error

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants