Archive

Archive for March, 2012

ipython notebook

March 31, 2012 Leave a comment

IPython notebook can be installed the following way on Ubuntu:

sudo pip install ipython -U
sudo pip install tornado
sudo apt-get install libzmq-dev
sudo apt-get install python2.7-dev
sudo pip install pyzmq

Execution:

ipython notebook

Update (20120418)
To learn more about IPython, check out this monstre tutorial (more than 3 hours long :))

Advertisements

Import a file from another directory of the project

March 31, 2012 1 comment

Imagine the project structure on the right side. The project is called “Sortings”, thus everything is under this directory. It’s a good idea to put your project in a package, thus all the source files are in the “sortings” package. Here we have a “config.py” file that is used by almost all the source files in the project. We also have a subpackage called “sorts”, and in this subpackage there is a script called “shell.py”. Say “shell.py” wants to include “config.py” from the parent directory. It can be done this way:

# shell.py
from sortings import config

Now you see why we put everything under the named package “sortings”.

It works well from the IDE, great. However, if you go to the command-line and you want to launch “shell.py”, you’ll get a surprise:

$ ./shell.py
Traceback (most recent call last):
  File "./shell.py", line 3, in
    from sortings import config
ImportError: No module named sortings

Hmmm… Let’s see the PYTHONPATH:

$ echo $PYTHONPATH
:/home/jabba/python/lib/jabbapylib:/home/jabba/python/lib

He is right, the “Sortings” directory is not in the PYTHONPATH, thus it cannot find the module “sortings”.

Let’s add the required directory then. Open “shell.py” and add these lines to the beginning. It is important that these lines should be executed first.

import sys
sys.path.append('../..')

from sortings import config
...

Now it works… as long as you launch “shell.py” from the current directory. Step up to the parent directory and try to launch it again:

$ ./shell.py    # works
$ cd ..
$ sorts/shell.py
Traceback (most recent call last):
  File "sorts/shell.py", line 6, in
    from sortings import config
ImportError: No module named sortings

Damn, what’s wrong again? Well, we appended a relative path, not an absolute path. Let’s change the beginning of “shell.py” again:

import os
import sys
sys.path.append(os.path.abspath(os.path.dirname(__file__) + '/' + '../..'))

from sortings import config
...

Super! Now you can call “shell.py” from everywhere, even from crontab.

Another way
There is another way too. Add the absolute path of the directory “Sortings” to PYTHONPATH. If you want it permanent, add the following to your “.bashrc” file:

export PYTHONPATH=$PYTHONPATH:/abs/path/to/Sortings

Note that this is not the recommended way though.

Calling a Python script from crontab

March 31, 2012 6 comments

Problem
You want to call a Python script of yours from crontab but it doesn’t really want to work :(

Solution
First, redirect everything to a file, thus you can check the error messages. For testing, let’s call the script at every minute. Edit your crontab settings with “crontab -e“.

* * * * * python /absolute/path/to/script.py >>/tmp/out.txt 2>&1

Don’t redirect to /var/log/cronrun (as many guides suggest), because if you have no rights to modify it, you won’t see anything… Choose a file where you can write to.

Then start checking the output of the log file:

touch /tmp/out.txt
tail -f /tmp/out.txt

tail -f” monitors the file constantly and prints every change.

Bare in mind that crontab doesn’t read your startup settings from .bashrc for instance, thus if you added something to PYTHONPATH, it won’t be visible! You have to repeat your PYTHONPATH settings in crontab too. “Each directory you need will have to be listed, because you can’t append to it since the previous value isn’t available.” (tip from here)

PYTHONPATH=/dir1:/dir2
* * * * * python /absolute/path/to/script.py >>/tmp/out.txt 2>&1

If it works, customize the execution time of the script by changing the “* * * * *” part.

Troubleshooting
You may also have problems with imports. Say your script wants to import another module from a different folder. In this case you can have a look at this post: Import a file from another directory of the project.

Related posts

Categories: python Tags: ,

uploading an image to imgur

March 30, 2012 Leave a comment

Problem
You want to upload an image to imgur.com anonymously, from your Python script of course.

Solution

#!/usr/bin/env python

"""
Upload an image to imgur anonymously.

# from jabbapylib.imgur import imgur
"""

from jabbapylib import config as cfg
import pycurl
import cStringIO
import untangle


def upload_from_computer(fpath):
    """
    Upload an image from the local machine.
    The return value is an XML string.
    
    Beware! fpath must be normal string, not unicode!
    With unicode it'll drop an error.
    """
    response = cStringIO.StringIO()
    c = pycurl.Curl()

    values = [("key", cfg.IMGUR_KEY),
              ("image", (c.FORM_FILE, fpath))]

    c.setopt(c.URL, "http://api.imgur.com/2/upload.xml")
    c.setopt(c.HTTPPOST, values)
    c.setopt(c.WRITEFUNCTION, response.write)
    c.perform()
    c.close()

    return response.getvalue()


def upload_from_web(url):
    """
    Upload an image from the web.
    The return value is an XML string.
    """
    response = cStringIO.StringIO()
    c = pycurl.Curl()

    values = [("key", cfg.IMGUR_KEY),
              ("image", url)]

    c.setopt(c.URL, "http://api.imgur.com/2/upload.xml")
    c.setopt(c.HTTPPOST, values)
    c.setopt(c.WRITEFUNCTION, response.write)
    c.perform()
    c.close()

    return response.getvalue()


def process(xml):
    """
    Process the returned XML string.
    """
    o = untangle.parse(xml)
    url = o.upload.links.original.cdata
    delete_page = o.upload.links.delete_page.cdata

    print '# url:        ', url
    print '# delete page:', delete_page
    
##########################
## some simple wrappers ##
##########################

def upload_local_img(fpath):
    """
    Upload a local image.
    The return value is a tuple: (imgur_url, imgur_delete_url)
    """
    xml = upload_from_computer(fpath)
    o = untangle.parse(xml)
    url = o.upload.links.original.cdata
    delete_page = o.upload.links.delete_page.cdata
    return (url, delete_page)

def upload_web_img(url):
    """
    Upload a web image.
    The return value is a tuple: (imgur_url, imgur_delete_url)
    """
    xml = upload_from_web(url)
    o = untangle.parse(xml)
    url = o.upload.links.original.cdata
    delete_page = o.upload.links.delete_page.cdata
    return (url, delete_page)

#############################################################################

if __name__ == "__main__":
#    img = '/tmp/test.jpg'
#    xml = upload_from_computer(img)
#    process(xml)
    #
#    url = 'http://...'
#    xml = upload_from_web(url)
#    process(xml)
    #
#    print upload_local_img('/tmp/test.jpg')
    #
#    print upload_web_img('http://...')
    pass

You can find the latest version in my jabbapylib library.

submitting a link to reddit and adding a comment

March 30, 2012 Leave a comment

Problem
You want to send a large number of links to reddit and you want to get it done with a script. How to send links and how to add comments to them?

Solution
First, install the reddit api:

sudo pip install reddit -U

Then you can use my RedBot to perform the task. It is part of my jabbapylib library.

#!/usr/bin/env python

"""
Simple API for:
* posting links to reddit
* commenting on a post

This requires that you have a reddit account. Put your username and
password in the following files:
* ~/reddit_username.txt
* ~/reddit_password.txt

To use RedBot, just make a subclass of it and give it a cool name:

class HyperBot(RedBot):
    def __init__(self):
        super(HyperBot, self).__init__()
        self.name = 'HyperBot'
        
Now you are ready to flood reddit :)

# from jabbapylib.reddit import red
"""

import reddit
from jabbapylib.filesystem import fs
from jabbapylib.platform import platform

USERNAME_TXT = '{home}/reddit_username.txt'.format(home=platform.get_home_dir())
PASSWORD_TXT = '{home}/reddit_password.txt'.format(home=platform.get_home_dir())
#
USERNAME = fs.read_first_line(USERNAME_TXT)
PASSWORD = fs.read_first_line(PASSWORD_TXT)


class RedBot(object):
    def __init__(self):
        self.name = 'RedBot'
        self.username = USERNAME
        self.password = PASSWORD
        #
        self.r = reddit.Reddit(user_agent=self.name)
        self.r.login(username=self.username, password=self.password)
        self.last_post = None   # Submission object
        self.permalink = None   # URL of the last post
        
    def submit_link(self, url, subreddit, title):
        """
        The return value (res) is a Submission object or None.
        URL of the newly created post: res.permalink
        """
        try:
            self.last_post = self.r.submit(subreddit, title, url=url)
            self.permalink = self.last_post.permalink
            print '# url to send: {url}'.format(url=url)
            print '# submitted to: {pl}'.format(pl=self.permalink)
            return self.last_post
        except:
            print >>sys.stderr, "Warning: couldn't submit {url}".format(url=url)
            return None
        
    def add_comment(self, comment):
        if self.last_post:
            self.last_post.add_comment(comment)
            print '# comment added'

#############################################################################

if __name__ == "__main__":
# here is how to use it:
#    url = '...'
#    subreddit = '...'
#    title = "..."
#    comment = '...'
#    r = RedBot()
#    r.submit_link(url, subreddit, title)
#    r.add_comment(comment)
    pass

You can find the current version here.

Thanks to Bryce Boe, the maintainer of the reddit api, who kindly answered my questions.

Links
Python Reddit API Wrapper’s documentation

Categories: python Tags: , ,

Catch a specific IOError

March 30, 2012 Leave a comment

Problem
If you want to open a non-existing file, you get the following error message: “IOError: [Errno 2] No such file or directory:...“. How to catch this specific IOError? For instance you want to notify the user that the file is missing instead of just saying “an I/O error occurred”.

Solution
Example:

def read_data_file(self):
    try:
        with open(self.data_file) as f:
           return json.load(f)
    except IOError, e:
        if e.errno == errno.ENOENT:
            print "Error: the given file doesn't exist."
            sys.exit(1)

Errno 2 == errno.ENOENT.

This tip is from here.

‘ascii’ codec can’t encode character: ordinal not in range(128)

March 29, 2012 6 comments

Problem

UnicodeEncodeError: 'ascii' codec can't encode character u'\xe1' in position 1: ordinal not in range(128)

Solution

def encode(text):
    """
    For printing unicode characters to the console.
    """
    return text.encode('utf-8')

Or:

reload(sys)
sys.setdefaultencoding("latin-1")

a = u'\xe1'
print str(a) # no exception

This tip is from here.

Categories: python Tags: , ,