Archive

Archive for the ‘python’ Category

create a virtual environment easily

August 30, 2015 Leave a comment

Problem
I prefer to put my larger projects in a virtual environment (there are people who put every project in a virt. env. …). I keep my projects in Dropbox (thus they are available on all my machines), but the virt. env.’s are kept outside Dropbox since they can grow quite big (and they are easily reproducible).

For creating virt. env.’s, I use virtualenvwrapper, which (by default) puts virt. env.’s in the folder ~/.virtualenvs. Say I have a project in Dropbox, and I want to create a virt. env. for it. How to do it easily?

Solution
First, you need to know if your project is written in Python 2 or Python 3. Then, you need to use the mkvirtualenv command but I always need to look up its syntax. Solution: in the root folder of my project I want a script that will create a virt. env. for the project. Here is the script:

#!/usr/bin/env bash

# mk_venv.sh

# which Python version to use in the created virt. env. (2 or 3)
PYTHON_VER=2

a=`pwd`
f=${a##*/}
source `which virtualenvwrapper.sh` && mkvirtualenv -p `which python${PYTHON_VER}` $f

Just set PYTHON_VER and launch the script. It will figure out the name of the current folder and create a virt. env. with this name. For instance, if you have your project in ~/projects/stuff, then the virt. env. will be created in the folder ~/.virtualenvs/stuff.

Related links

Categories: bash, python Tags: ,

[flask] validate a URL

August 23, 2015 Leave a comment

Problem
In a Flask application I wanted to verify if a user-given URL is valid.

Solution
I found a simple validator package for that called validators (see it on GitHub).

Sample usage:

$ pip install validators
$ python
>>> import validators
>>> url = "http://index.hu"
>>> validators.url(url)
True
>>> url = "http://index.h/"
>>> validators.url(url)
ValidationFailure(func=url, args={'value': 'http://index.h/', 'require_tld': True})

The ValidationFailure class implements the __bool__ method, so you can easily check if validation failed:

if not validators.url(url):
    flash("Error: you must provide a valid URL!")
Categories: flask, python Tags: ,

[flask] show render time on a page

August 22, 2015 Leave a comment

Problem
For debugging purposes, I wanted to see the rendering time of each page.

Solution
I found a solution here. Here are the links:

Note that Flask-DebugToolbar is very useful but it really slowed down my application. I even switched it off and I only use it when I want to debug something. When the issue is solved I switch it off again.

working with zip files

August 21, 2015 Leave a comment

Problem

In a project of mine I had to deal with folders, where a folder can contain several thousands of small text files. I kept this project on Dropbox, so I could use it on all my machines. However, Dropbox is quite slow when trying to synchronize several thousand files. So I decided to put files in a folder into a zip file.

So the question is: how to deal with zip files? How to do basic operations with them: create zip, delete from zip, list zip, add to zip, move to zip, extract from zip, etc.

Solution

In this project of mine I used the external zip command as well as the zipfile package from the stdlib. Let’s see both of them.

Manipulating zip files from the command-line
Let’s see some examples. Compress every .json file in the current directory except the desc.json file:

zip -9 files.zip *.json -x desc.json

The switch “-9” gives the best compression, files.zip is the output, and “-x” is short for “--exclude“. From Python you can call it as an external command with os.system() for instance.

The previous example creates a zip file and leaves the original files. Now let’s move files into a zip file (and delete the original files when they were added successfully to the archive):

zip -9 -m files.zip *.json -x desc.json

Delete a file from an archive:

zip -d files.zip desc.json

It will delete desc.json from the zip file.

List the content of a zip file:

zipinfo files.zip

Add a file to the archive:

zip -g files.zip new.json

Where “-g” means: grow.

Extract just one file from a zip file:

unzip files.zip this.json

It will extract this.json from the archive.

Read the content of a zip file in Python
OK, say we have a zip file that contains some files. How to get the filenames? How to read them? I found some nice examples here.

List the file names in a zip file:

import zipfile

zfile = zipfile.ZipFile("files.zip", "r")

for name in zfile.namelist():
    print(name)

Read files in a zip file:

import zipfile

zfile = zipfile.ZipFile("files.zip", "r")

for name in zfile.namelist():
    data = zfile.read(name)
    print(data)

Links

Categories: bash, python Tags: ,

trending Python repositories on GitHub

August 9, 2015 Leave a comment

https://github.com/trending?l=python

Find what repositories the GitHub community is most excited about today.

Categories: python Tags: ,

Reading (writing) unicode text from (to) files

August 6, 2015 Leave a comment

Problem
You want to write some special characters to a file (e.g. f.write("voilá")) but you get immediately some unicode error in your face.

Solution
Instead of messing with the encode, decode methods, use the codecs module.

import codecs

# read
with codecs.open(fname, "r", "utf-8") as f:
    text = f.read()

# write
with codecs.open(tmp, "w", "utf-8") as to:
    to.write(text)

As can be seen, its usage is very similar to the well-known open function.

This tip is from here.

Categories: python Tags: , ,

screenshot.py

July 26, 2015 Leave a comment

I made a simple wrapper script called screenshot.py that calls phantomjs and convert to do the real job. The advantage of screenshot.py is its very simple usage.

Usage

screenshot.py -full http://reddit.com full.jpg

Screenshot of the entire page (can be very high).

screenshot.py -window http://reddit.com window.jpg

Screenshot of the area that you see in the browser.

screenshot.py -thumb http://reddit.com thumb.jpg

Thumbnail of the area that you see in the browser.

Links

Follow

Get every new post delivered to your Inbox.

Join 97 other followers