import from anywhere

Update (20140102): This solution is not perfect either. As discussed here, each time imp.load_source is run on a module, the module-level code in that module will be re-executed, which could cause problems in more complicated cases. Thanks to alb1 for the info.

---

Problem
You have a project structure with several submodules and in a source you want to import something from another submodule by using relative import. Example:

project/
    main.py
    sub1/
        __init__.py
        helper.py
    sub2/
        __init__.py
        utils.py

As long as you launch main.py only, there are no big problems. However, I like adding simple tests to the end of modules after the ‘if __name__ == "__main__":‘ line. Say utils.py wants to import something from helper.py. How to do it properly so that I can still execute utils.py as a stand-alone script? Unfortunately, this is a real pain in Python :(

Solution
I think I managed to find a simple and elegant solution for this problem. Let’s see utils.py:

def load_src(name, fpath):
    import os, imp
    return imp.load_source(name, os.path.join(os.path.dirname(__file__), fpath))

load_src("helper", "../sub1/helper.py")
import helper
from helper import hi    # if you need just this, "import helper" is not required

print helper.PI
print hi()

Provide two parameters to load_src. The first one is a name. You will be able to import the module using this name. The second parameter is the relative path of the source file you want to import. Note that we can only load files this way, not directories!

I prefer this way because you need to import the desired module explicitly (using the import keyword). From the Zen of Python we know that “explicit is better than implicit”.

Another way would be this:

helper = load_src("helper", "../sub1/helper.py")
print helper.PI

Here helper is imported implicitly, so after loading it you can refer to its content (e.g. helper.PI). I prefer the first version; that makes the source code more readable.

Update: Version 2
Here is an updated version that supports file names given either in absolute or relative path.

def load_src(name, fpath):
    import os, imp
    p = fpath if os.path.isabs(fpath) \
        else os.path.join(os.path.dirname(__file__), fpath)
    return imp.load_source(name, p)

Alternatives
You could also modify the sys.path list to include the parent of “sub1/“. Then you could import helper like this: “from sub1 import helper“. However, modifying sys.path is considered by many as a bad solution.