import from anywhere

Update (20140102): This solution is not perfect either. As discussed here, each time imp.load_source is run on a module, the module-level code in that module will be re-executed, which could cause problems in more complicated cases. Thanks to alb1 for the info.

---

Problem
You have a project structure with several submodules and in a source you want to import something from another submodule by using relative import. Example:

project/
    main.py
    sub1/
        __init__.py
        helper.py
    sub2/
        __init__.py
        utils.py

As long as you launch main.py only, there are no big problems. However, I like adding simple tests to the end of modules after the ‘if __name__ == "__main__":‘ line. Say utils.py wants to import something from helper.py. How to do it properly so that I can still execute utils.py as a stand-alone script? Unfortunately, this is a real pain in Python :(

Solution
I think I managed to find a simple and elegant solution for this problem. Let’s see utils.py:

def load_src(name, fpath):
    import os, imp
    return imp.load_source(name, os.path.join(os.path.dirname(__file__), fpath))

load_src("helper", "../sub1/helper.py")
import helper
from helper import hi    # if you need just this, "import helper" is not required

print helper.PI
print hi()

Provide two parameters to load_src. The first one is a name. You will be able to import the module using this name. The second parameter is the relative path of the source file you want to import. Note that we can only load files this way, not directories!

I prefer this way because you need to import the desired module explicitly (using the import keyword). From the Zen of Python we know that “explicit is better than implicit”.

Another way would be this:

helper = load_src("helper", "../sub1/helper.py")
print helper.PI

Here helper is imported implicitly, so after loading it you can refer to its content (e.g. helper.PI). I prefer the first version; that makes the source code more readable.

Update: Version 2
Here is an updated version that supports file names given either in absolute or relative path.

def load_src(name, fpath):
    import os, imp
    p = fpath if os.path.isabs(fpath) \
        else os.path.join(os.path.dirname(__file__), fpath)
    return imp.load_source(name, p)

Alternatives
You could also modify the sys.path list to include the parent of “sub1/“. Then you could import helper like this: “from sub1 import helper“. However, modifying sys.path is considered by many as a bad solution.

Relative import in Python

The following post is based on nosklo’s answer who explained relative imports very nicely in this thread.

Imagine the following project structure:

main.py
setup.py
app/ ->
    __init__.py
    package_a/ ->
       __init__.py
       module_a.py
    package_b/ ->
       __init__.py
       module_b.py

Problem: how to use module_b.py from module_a.py?

Solution:

”

1. you run: python main.py
2. main.py does: import app.package_a.module_a
3. module_a.py does: import app.package_b.module_b

Alternatively, 2 or 3 could use: from app.package_a import module_a

That will work as long as you have app in your PYTHONPATH. main.py could be anywhere then.”