When a FileNotFoundError is raised inside while the importlib.resources.path context manager is active a RuntimeError is raised.
Looking at the (3.8) code it seems that FileNotFound exceptions are handled specially from all other exceptions which may lead to this behaviour. While the code in 3.9 changed significantly the same behaviour can be observed.

Files:
.
└── my_package
    ├── data.txt (empty)
    ├── __init__.py (empty)
    └── test.py

Content of test.py:
import importlib.resources
def main():
    with importlib.resources.path('my_package', 'data.txt') as p:
        raise FileNotFoundError()
if __name__ == '__main__':
    main()

Exact error message:
RuntimeError: generator didn't stop after throw()