Archive

Posts Tagged ‘context manager’

ChDir: a context manager for switching working directories

December 15, 2013 3 comments

Problem
In your program you want to change the working directory temporarily, do some job there, then switch back to the original directory. Say you want to download some images to /tmp. When done, you want to get back to the original location correctly, even if an exception was raised at the temp. location.

Naïve way
Let’s see the following example. We have a script, say at /home/jabba/python/fetcher.py . We want to download some images to /tmp, then work with them. After the download we want to create a subfolder “process” in the same directory where the script fetcher.py is located. We want to collect some extra info about the downloaded images and we want to store these pieces of information in the “process” folder.

import os

def download(li, folder):
    try:
        backup = os.getcwd()
        os.chdir(folder)
        for img in li:
            # download img somehow
        os.chdir(backup)
    except:
        # problem with download, handle it

def main():
    # step 1: download images to /tmp
    li = ["http://...1.jpg", "http://...2.jpg", "http://...3.jpg"]
    download(li, "/tmp")
    # step 2: create a "process" dir. HERE (where the script was launched)
    os.mkdir("process")
    # ...do some extra work...

There is a problem with the download method. If an image cannot be downloaded correctly and an exception occurs, we return from the method. However, os.chdir(backup) is not executed and we remain in the /tmp folder! In main() in step 2 the process directory will be created in /tmp and not in the folder where we wanted it to be.

Well, you can always add a finally block to the exception handler and place os.chdir(backup) there, but it’s easy to forget. Is there an easier solution?

Solution
Yes, there is an easier solution. Use a context manager.

The previous example with a context manager:

import os

def download(li, folder):
    with ChDir(folder):
        for img in li:
            # download img somehow

def main():
    # step 1: download images to /tmp
    li = ["http://...1.jpg", "http://...2.jpg", "http://...3.jpg"]
    download(li, "/tmp")
    # step 2: create a "process" dir. HERE (where the script was launched)
    os.mkdir("process")
    # ...do some extra work...

And now the source code of ChDir:

import os

class ChDir(object):
    """
    Step into a directory temporarily.
    """
    def __init__(self, path):
        self.old_dir = os.getcwd()
        self.new_dir = path

    def __enter__(self):
        os.chdir(self.new_dir)

    def __exit__(self, *args):
        os.chdir(self.old_dir)

Since ChDir is a context manager, you use it in a with block. At the beginning of the block you enter the given folder. When you leave the with block (even if you leave because of an exception), you are put back to the folder where you were before entering the with block.

Update
Following this discussion thread @reddit, someone suggested using the PyFilesytem library. I think PyFilesytem is a very good solution but it may be too much for a short script. It’s like shooting a sparrow with a cannon :) For a simple script ChDir is good enough for me. For a serious application, check out PyFilesytem.

How to ignore an exception — the elegant way

September 7, 2013 1 comment

This idea was presented by Raymond Hettinger at PyCon US 2013. He is talking about it at 43:30: http://www.youtube.com/watch?v=OSGv2VnC0go.

Problem
You want to ignore an exception.

Solution 1: the classical way
Say you want to delete a file but it’s not sure it exists.

try:
    os.unlink('somefile.txt')
except OSError:
    pass

That is, if the exception occurs, we do nothing.

Solution 2: the elegant way

with ignored(OSError):
    os.unlink('somefile.txt')

Its source code in Python 2.7:

from contextlib import contextmanager

@contextmanager
def ignored(*exceptions):
    try:
        yield
    except exceptions:
        pass

This is part of Python 3.4, thus in Python 3.4 all you need is this line:

from contextlib import ignored

See the docs here.

Autoflush

September 27, 2010 Leave a comment

Printing to the standard output is buffered. What to do if you want to see the output immediately?

import sys
import os

# reopen stdout file descriptor with write mode
# and 0 as the buffer size (unbuffered)
sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0)

print "unbuffered text"

Credits

This tip is from http://www.gossamer-threads.com/lists/python/python/658167.

Update (20130206)
The solution above switches buffered mode off, but you can’t switch it back on because you lose the original sys.stdout file descriptor. I have a more sophisticated solution, available here (autoflush.py) as part of my jabbapylib library.

Usage #1:

autoflush(True)
# text that you want to print in unbuffered mode comes here
autoflush(False)    # back to normal

Usage #2:

# using a context manager
with AutoFlush():
    # unbuffered text comes here
    sys.stdout.write(...)

# here you are back to normal

Let’s not forget the simplest and most trivial solution either:

sys.stdout.write(...)
sys.stdout.flush()    # flush out immediately