After discovering my initial approach to PEP-402 style imports wasn't very robust, I implemented a better way of going about things but it later turned out that the recursive approach which importlib.__import__() employs is not well suited for virtual packages. Because of that, I ended up integrating P.J. Eby's (who is the author of PEP 402) iterative _gcd_import(). After a few minor changes it passes all unittests for importlib, but some details of virtual package imports still need to be worked out in the abstract before they can be implemented. In particular, it's not entirely clear if it's better to let virtual packages hang around should import of a module fail or should they be removed.
Sunday, August 14, 2011
Tuesday, August 9, 2011
PEP 402
I realised I didn't say much about PEP 402 last time round so here's some more details:
PEP 402 aims to address the long standing problem with storing contents of a Python package in several directories. A previous proposal to fix this issue was PEP 382 which proposed extensions to the existing *.pth mechanism available on the top-level python path. PEP 402, however, describes a simplified solution where the requirement for directories with packages to contain an __init__.py file is lifted in some cases:
I spent the last few days working on a proof-of-concept implementation. The changes outlined in the PEP are quite small but identifying the correct places to inject new code was quite tricky. I had to read the importlib code in much more detail to identify the places where to make the changes but eventually I got the code to support simple use cases. I created a separate repository for this part of the project: https://bitbucket.org/jergosh/pep-402
Here's the commit with the initial implementation: https://bitbucket.org/jergosh/pep-402/changeset/2c60dc2d17f1
PEP 402 aims to address the long standing problem with storing contents of a Python package in several directories. A previous proposal to fix this issue was PEP 382 which proposed extensions to the existing *.pth mechanism available on the top-level python path. PEP 402, however, describes a simplified solution where the requirement for directories with packages to contain an __init__.py file is lifted in some cases:
- when importing submodules from a directory which contains Python modules. E. g. if there's a directory Foo containing Bar.py present on sys.path, one can import Foo.Bar, just as if Foo also contained __init__.py but not import Foo alone.
- when importing *submodules* from a directory with the same name as an already imported package, e. g. a standard library package.
I spent the last few days working on a proof-of-concept implementation. The changes outlined in the PEP are quite small but identifying the correct places to inject new code was quite tricky. I had to read the importlib code in much more detail to identify the places where to make the changes but eventually I got the code to support simple use cases. I created a separate repository for this part of the project: https://bitbucket.org/jergosh/pep-402
Here's the commit with the initial implementation: https://bitbucket.org/jergosh/pep-402/changeset/2c60dc2d17f1
Subscribe to:
Posts (Atom)