This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: importlib.resources.path raises RuntimeError when FileNotFoundError is raise in context manager
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.9, Python 3.8
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: FFY00, Nils Kattenbeck, brett.cannon, jaraco, miguendes
Priority: normal Keywords:

Created on 2021-05-14 19:29 by Nils Kattenbeck, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (7)
msg393686 - (view) Author: Nils Kattenbeck (Nils Kattenbeck) * Date: 2021-05-14 19:29
When a FileNotFoundError is raised inside while the importlib.resources.path context manager is active a RuntimeError is raised.
Looking at the (3.8) code it seems that FileNotFound exceptions are handled specially from all other exceptions which may lead to this behaviour. While the code in 3.9 changed significantly the same behaviour can be observed.

Files:
.
└── my_package
    ├── data.txt (empty)
    ├── __init__.py (empty)
    └── test.py

Content of test.py:
import importlib.resources
def main():
    with importlib.resources.path('my_package', 'data.txt') as p:
        raise FileNotFoundError()
if __name__ == '__main__':
    main()

Exact error message:
RuntimeError: generator didn't stop after throw()
msg393708 - (view) Author: Miguel Brito (miguendes) * Date: 2021-05-15 09:01
I can reproduce this. From what I can see the issue is that `importlib.resources.path` intentionally suppresses `FileNotFoundError` errors.

https://github.com/python/cpython/blob/main/Lib/importlib/resources.py#L138

Based on `importlib.resources.path` docstring I think this is to avoid raising an exception if the file is deleted before the context manager exits.

On contextlib.py, since type is not None, it will try to throw the exception but since it's supressed nothing will happen and the code will reach the end by raising the RuntimeError.

https://github.com/python/cpython/blob/main/Lib/contextlib.py#L151

If I'm not mistaken, this is the test that verifies that behaviour: https://github.com/python/cpython/blob/main/Lib/test/test_importlib/test_path.py#L51


I'm not a core dev but it looks like it's an intentional behaviour.

Maybe the docs should be more clear about this. It's not obvious to me when I read the docs.
msg393813 - (view) Author: Filipe Laíns (FFY00) * (Python triager) Date: 2021-05-17 14:51
This was fixed in https://github.com/python/cpython/pull/22915. It can be closed now.
msg393815 - (view) Author: Nils Kattenbeck (Nils Kattenbeck) * Date: 2021-05-17 15:05
Yes I understand that the function handles this specially to not raise an exception if the file is not found in the package (even though the intention behind this is not clear to me). However if a user causes a FileNotFoundException itself inside of the context manager everything breaks (e.g. does something erroneous with the path, calls subprocess.run with a non existing binary etc).
msg393816 - (view) Author: Filipe Laíns (FFY00) * (Python triager) Date: 2021-05-17 15:21
The supress(FileNotFoundError) context manager now is only active when fetching the file, not during importlib.resources.path. _path_from_resource_path is not longer a generator, so supress(FileNotFoundError) is promptly closed after the return, preventing its side effects from leaking outside the function.

Your reproducible now works as intended:

Traceback (most recent call last):
  File "/home/anubis/git/cpython/test.py", line 9, in <module>
    main()
  File "/home/anubis/git/cpython/test.py", line 6, in main
    raise FileNotFoundError()
FileNotFoundError
msg394127 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2021-05-21 16:49
Thanks for tracking this down, Filipe. Agreed it sounds like it's fixed in Python 3.10. It's unlikely the fix will be backported to Python 3.9. Instead, if this behavior affects your usage, consider using the `importlib_resources` backport, which also includes the fix. And please follow-up if there are issues not addressed by this approach.
msg394184 - (view) Author: Nils Kattenbeck (Nils Kattenbeck) * Date: 2021-05-22 14:08
Thanks for looking into it. Yes I can confirm that `importlib_resources` has the expected behaviour - I did not download Python 3.10 as the code seems to be the same.
History
Date User Action Args
2022-04-11 14:59:45adminsetgithub: 88303
2021-05-22 14:08:24Nils Kattenbecksetmessages: + msg394184
2021-05-21 16:49:08jaracosetstatus: open -> closed
resolution: out of date
messages: + msg394127

stage: resolved
2021-05-17 15:21:58FFY00setmessages: + msg393816
2021-05-17 15:05:40Nils Kattenbecksetmessages: + msg393815
2021-05-17 14:51:44FFY00setnosy: + FFY00
messages: + msg393813
2021-05-15 09:01:30miguendessetnosy: + miguendes
messages: + msg393708
2021-05-14 19:29:35Nils Kattenbecksettitle: importlib.resources.path raises RuntimeError import FileNotFoundError is raise in context manager -> importlib.resources.path raises RuntimeError when FileNotFoundError is raise in context manager
2021-05-14 19:29:20Nils Kattenbeckcreate