Archive

Archive for June, 2014

built-in Python modules you should know

June 24, 2014 Leave a comment

Here is an interesting thread about built-in modules that you should know: http://redd.it/28yo37 . If you wonder how to improve your Python knowledge, check ’em out.

In the thread some useful 3rd party modules are also listed.

Categories: python Tags:

call a function by knowing its name as a string

June 24, 2014 Leave a comment

Problem
I want to call a function but its name is stored in a string. How to do that?

Solution
I wanted to create a menu where the user can select which entry to call. It looks like this:

(1)[r] radio
(2)[ctd] create temp. directory
--------
[m] menu
[q] <

In Python I stored it like this:

menu = OrderedDict()
menu[(1, 'r')] = ('radio', 'apps.radio.radio_player')
menu[(2, 'ctd')] = ('create temp. directory', 'apps.temp_folder.create_temp_folder')

That is, if the user selects “1“, then apps.radio.radio_player() must be called.

Here is the method that can call the appropriate function:

import importlib

def start_app(val):
    """
    Call a function by name (string).
    
    Tip from here: http://stackoverflow.com/questions/3061 .
    """ 
    _, to_call = val
    function_string = to_call # ex.: 'apps.radio.radio_player'
    mod_name, func_name = function_string.rsplit('.', 1)
    mod = importlib.import_module(mod_name)
    func = getattr(mod, func_name)
    func()

The tip is from here.

extract all links from a file

June 17, 2014 Leave a comment

Problem
You want to extract all links (URLs) from a text file.

Solution

def extract_urls(fname):
    with open(fname) as f:
        return re.findall('http[s]?://(?:[a-zA-Z]|[0-9]|[$-_@.&+]|[!*\(\),]|(?:%[0-9a-fA-F][0-9a-fA-F]))+', f.read())
Categories: python Tags: , ,

Is a file binary?

June 17, 2014 Leave a comment

Problem
I want to process all text files in a folder recursively. (Actually, I want to extract all URLs from them). However, their extensions are not necessarily .txt. How to separate text files from binary files?

Solution
In this thread I found a solution. Here is my slightly modified version:

def is_binary(fname):
    """
    Return true if the given filename is binary.

    found at http://stackoverflow.com/questions/898669
    """
    CHUNKSIZE = 1024
    with open(fname, 'rb') as f:
        while True:
            chunk = f.read(CHUNKSIZE)
            if '\0' in chunk: # found null byte
                return True
            if len(chunk) < CHUNKSIZE:
                break # done

    return False

If it finds a '\0' character, then the file is considered to be binary. Note that it will also classify UTF-16-encoded text files as “binary”.

Categories: python Tags: ,

virtualenvwrapper (Part 2)

Here I describe how I use virtualenvwrapper.

I keep my virtual environments in the default location (~/.virtualenvs folder). My Python projects are kept in Dropbox. If a project (call it “stuff” for instance) has a virtual env., then I add the suffix “_project” to the projects’s folder name (resulting “stuff_project” in this example). This way I can see it immediately (without entering the folder) that it has an attached venv.

In the stuff_project folder create a requirements.txt file that contains all the modules that you installed in your venv. This way you can re-create your venv on another machine and continue to work on yout project.

Let’s see a concrete example. Say I want to create a new Django project:

$ cd $HOME/Dropbox/python/webapps
$ mkdir stuff_project
# ^ every project file will be in this folder
$ cd stuff_project
$ django-admin.py startproject stuff
# ^ create the Django project
$ mkvirtualenv stuff_project
# ^ venv is created AND activated
$ pip install django
$ pip install django-extensions
# ^ useful extensions
$ pip freeze --local >requirements.txt
# ^ packages installed in the venv are listed in this file

Since I put the project folder in Dropbox, it will be synced on all my machines. What to do if I want to work on this project on another machine of mine (let’s call it “machine #2”)? The virtual environment will be missing. But there is no need to worry. We have a requirements.txt file and from this we can populate a newly created venv. Install virtualenvwrapper on machine #2 and create an empty venv for the project:

# actions to perform on machine #2:
$ cd $HOME/Dropbox/python/webapps/stuff_project
$ mkvirtualenv stuff_project
# ^ the venv is created and activated
$ pip install -r requirements.txt
# ^ packages are installed in the venv

When you are done with the project, deactivate the venv with the “deactivate” command. Since it’s too long, I put an alias on it:

alias off='deactivate'

If you install new packages in the venv, don’t forget to update the requirements.txt file ( pip freeze >requirements.txt )!

virtualenvwrapper (Part 1)

I use virtualenv but so far it’s been enough for me. However, the time has come to step on to the next level: virtualenvwrapper. Virtualenvwrapper is just a wrapper around virtualenv, but it makes working with virtualenv so much easier.

Problem
I keep all my Python projects in Dropbox. Every project is located in a dedicated folder. If I use virtualenv, inside a project folder I need to create a subfolder for the the virtual environment (typically called “venv”). However, this folder can be really huge. In the case of a Flask project it can be up to 50-60 MB. Keeping it in Dropbox is a luxury. It would be great to move the virtual environments outside of Dropbox. My first idea was to put a symbolic link on the venv folder from the project folder but Dropbox doesn’t support symlinks :(

Solution
Let’s use virtualenvwrapper! Virtualenvwrapper was created with this idea: collect virtual environments in a separated folder, outside of your projects. In addition, it’s very easy to switch between virtual environments.

Here are some great resources on virtualenvwrapper:

For more info, refer to Bob’s blog (last link above), where he gives a very nice summary about the usage of virtualenvwrapper.

Categories: python Tags: ,