Archive
ipython notebook
IPython notebook can be installed the following way on Ubuntu:
sudo pip install ipython -U sudo pip install tornado sudo apt-get install libzmq-dev sudo apt-get install python2.7-dev sudo pip install pyzmq
Execution:
ipython notebook
Update (20120418)
To learn more about IPython, check out this monstre tutorial (more than 3 hours long :))
Import a file from another directory of the project
Imagine the project structure on the right side. The project is called “Sortings”, thus everything is under this directory. It’s a good idea to put your project in a package, thus all the source files are in the “sortings” package. Here we have a “config.py” file that is used by almost all the source files in the project. We also have a subpackage called “sorts”, and in this subpackage there is a script called “shell.py”. Say “shell.py” wants to include “config.py” from the parent directory. It can be done this way:
# shell.py from sortings import config
Now you see why we put everything under the named package “sortings”.
It works well from the IDE, great. However, if you go to the command-line and you want to launch “shell.py”, you’ll get a surprise:
$ ./shell.py
Traceback (most recent call last):
File "./shell.py", line 3, in
from sortings import config
ImportError: No module named sortings
Hmmm… Let’s see the PYTHONPATH:
$ echo $PYTHONPATH :/home/jabba/python/lib/jabbapylib:/home/jabba/python/lib
He is right, the “Sortings” directory is not in the PYTHONPATH, thus it cannot find the module “sortings”.
Let’s add the required directory then. Open “shell.py” and add these lines to the beginning. It is important that these lines should be executed first.
import sys
sys.path.append('../..')
from sortings import config
...
Now it works… as long as you launch “shell.py” from the current directory. Step up to the parent directory and try to launch it again:
$ ./shell.py # works
$ cd ..
$ sorts/shell.py
Traceback (most recent call last):
File "sorts/shell.py", line 6, in
from sortings import config
ImportError: No module named sortings
Damn, what’s wrong again? Well, we appended a relative path, not an absolute path. Let’s change the beginning of “shell.py” again:
import os import sys sys.path.append(os.path.abspath(os.path.dirname(__file__) + '/' + '../..')) from sortings import config ...
Super! Now you can call “shell.py” from everywhere, even from crontab.
Another way
There is another way too. Add the absolute path of the directory “Sortings” to PYTHONPATH. If you want it permanent, add the following to your “.bashrc” file:
export PYTHONPATH=$PYTHONPATH:/abs/path/to/Sortings
Note that this is not the recommended way though.
Calling a Python script from crontab
Problem
You want to call a Python script of yours from crontab but it doesn’t really want to work :(
Solution
First, redirect everything to a file, thus you can check the error messages. For testing, let’s call the script at every minute. Edit your crontab settings with “crontab -e“.
* * * * * python /absolute/path/to/script.py >>/tmp/out.txt 2>&1
Don’t redirect to /var/log/cronrun (as many guides suggest), because if you have no rights to modify it, you won’t see anything… Choose a file where you can write to.
Then start checking the output of the log file:
touch /tmp/out.txt tail -f /tmp/out.txt
“tail -f” monitors the file constantly and prints every change.
Bare in mind that crontab doesn’t read your startup settings from .bashrc for instance, thus if you added something to PYTHONPATH, it won’t be visible! You have to repeat your PYTHONPATH settings in crontab too. “Each directory you need will have to be listed, because you can’t append to it since the previous value isn’t available.” (tip from here)
PYTHONPATH=/dir1:/dir2 * * * * * python /absolute/path/to/script.py >>/tmp/out.txt 2>&1
If it works, customize the execution time of the script by changing the “* * * * *” part.
Troubleshooting
You may also have problems with imports. Say your script wants to import another module from a different folder. In this case you can have a look at this post: Import a file from another directory of the project.
Related posts
uploading an image to imgur
Problem
You want to upload an image to imgur.com anonymously, from your Python script of course.
Solution
#!/usr/bin/env python
"""
Upload an image to imgur anonymously.
# from jabbapylib.imgur import imgur
"""
from jabbapylib import config as cfg
import pycurl
import cStringIO
import untangle
def upload_from_computer(fpath):
"""
Upload an image from the local machine.
The return value is an XML string.
Beware! fpath must be normal string, not unicode!
With unicode it'll drop an error.
"""
response = cStringIO.StringIO()
c = pycurl.Curl()
values = [("key", cfg.IMGUR_KEY),
("image", (c.FORM_FILE, fpath))]
c.setopt(c.URL, "http://api.imgur.com/2/upload.xml")
c.setopt(c.HTTPPOST, values)
c.setopt(c.WRITEFUNCTION, response.write)
c.perform()
c.close()
return response.getvalue()
def upload_from_web(url):
"""
Upload an image from the web.
The return value is an XML string.
"""
response = cStringIO.StringIO()
c = pycurl.Curl()
values = [("key", cfg.IMGUR_KEY),
("image", url)]
c.setopt(c.URL, "http://api.imgur.com/2/upload.xml")
c.setopt(c.HTTPPOST, values)
c.setopt(c.WRITEFUNCTION, response.write)
c.perform()
c.close()
return response.getvalue()
def process(xml):
"""
Process the returned XML string.
"""
o = untangle.parse(xml)
url = o.upload.links.original.cdata
delete_page = o.upload.links.delete_page.cdata
print '# url: ', url
print '# delete page:', delete_page
##########################
## some simple wrappers ##
##########################
def upload_local_img(fpath):
"""
Upload a local image.
The return value is a tuple: (imgur_url, imgur_delete_url)
"""
xml = upload_from_computer(fpath)
o = untangle.parse(xml)
url = o.upload.links.original.cdata
delete_page = o.upload.links.delete_page.cdata
return (url, delete_page)
def upload_web_img(url):
"""
Upload a web image.
The return value is a tuple: (imgur_url, imgur_delete_url)
"""
xml = upload_from_web(url)
o = untangle.parse(xml)
url = o.upload.links.original.cdata
delete_page = o.upload.links.delete_page.cdata
return (url, delete_page)
#############################################################################
if __name__ == "__main__":
# img = '/tmp/test.jpg'
# xml = upload_from_computer(img)
# process(xml)
#
# url = 'http://...'
# xml = upload_from_web(url)
# process(xml)
#
# print upload_local_img('/tmp/test.jpg')
#
# print upload_web_img('http://...')
pass
You can find the latest version in my jabbapylib library.
submitting a link to reddit and adding a comment
Problem
You want to send a large number of links to reddit and you want to get it done with a script. How to send links and how to add comments to them?
Solution
First, install the reddit api:
sudo pip install reddit -U
Then you can use my RedBot to perform the task. It is part of my jabbapylib library.
#!/usr/bin/env python
"""
Simple API for:
* posting links to reddit
* commenting on a post
This requires that you have a reddit account. Put your username and
password in the following files:
* ~/reddit_username.txt
* ~/reddit_password.txt
To use RedBot, just make a subclass of it and give it a cool name:
class HyperBot(RedBot):
def __init__(self):
super(HyperBot, self).__init__()
self.name = 'HyperBot'
Now you are ready to flood reddit :)
# from jabbapylib.reddit import red
"""
import reddit
from jabbapylib.filesystem import fs
from jabbapylib.platform import platform
USERNAME_TXT = '{home}/reddit_username.txt'.format(home=platform.get_home_dir())
PASSWORD_TXT = '{home}/reddit_password.txt'.format(home=platform.get_home_dir())
#
USERNAME = fs.read_first_line(USERNAME_TXT)
PASSWORD = fs.read_first_line(PASSWORD_TXT)
class RedBot(object):
def __init__(self):
self.name = 'RedBot'
self.username = USERNAME
self.password = PASSWORD
#
self.r = reddit.Reddit(user_agent=self.name)
self.r.login(username=self.username, password=self.password)
self.last_post = None # Submission object
self.permalink = None # URL of the last post
def submit_link(self, url, subreddit, title):
"""
The return value (res) is a Submission object or None.
URL of the newly created post: res.permalink
"""
try:
self.last_post = self.r.submit(subreddit, title, url=url)
self.permalink = self.last_post.permalink
print '# url to send: {url}'.format(url=url)
print '# submitted to: {pl}'.format(pl=self.permalink)
return self.last_post
except:
print >>sys.stderr, "Warning: couldn't submit {url}".format(url=url)
return None
def add_comment(self, comment):
if self.last_post:
self.last_post.add_comment(comment)
print '# comment added'
#############################################################################
if __name__ == "__main__":
# here is how to use it:
# url = '...'
# subreddit = '...'
# title = "..."
# comment = '...'
# r = RedBot()
# r.submit_link(url, subreddit, title)
# r.add_comment(comment)
pass
You can find the current version here.
Thanks to Bryce Boe, the maintainer of the reddit api, who kindly answered my questions.
Catch a specific IOError
Problem
If you want to open a non-existing file, you get the following error message: “IOError: [Errno 2] No such file or directory:...“. How to catch this specific IOError? For instance you want to notify the user that the file is missing instead of just saying “an I/O error occurred”.
Solution
Example:
def read_data_file(self):
try:
with open(self.data_file) as f:
return json.load(f)
except IOError, e:
if e.errno == errno.ENOENT:
print "Error: the given file doesn't exist."
sys.exit(1)
Errno 2 == errno.ENOENT.
This tip is from here.
‘ascii’ codec can’t encode character: ordinal not in range(128)
Problem
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe1' in position 1: ordinal not in range(128)
Solution
def encode(text):
"""
For printing unicode characters to the console.
"""
return text.encode('utf-8')
Or:
reload(sys)
sys.setdefaultencoding("latin-1")
a = u'\xe1'
print str(a) # no exception
This tip is from here.
