[flask] Jinja2: don’t print empty lines

Problem
When using Flask (or Django), I don’t care much about the generated HTMLs. They may be ugly, who cares. However, there is one thing that bothers me. When I write this for instance in a template:

<div>
    {% if True %}
        yay
    {% endif %}
</div>

the generated output looks like this:

<div>

        yay

</div>

See? Jinja2 litters the output with empty lines. How to get rid of them?

Solution
The official documentation talks about this here. It says you need to enable both trim_blocks and lstrip_blocks. In Flask, you can do that like this:

...
app = Flask(__name__)
app.jinja_env.trim_blocks = True
app.jinja_env.lstrip_blocks = True
...

Done.

UTC now → timestamp → UTC time

Problem
Say you have an application and you want to store the date/time of an event. Later you want to see this date/time.

Solution
Take the current UTC date/time and convert it to a timestamp, which is an integer. Store this value. Then read it and convert it back to UTC date/time.

>>> import datetime
>>>
>>> utcnow = datetime.datetime.utcnow()
>>> utcnow
datetime.datetime(2015, 1, 25, 18, 10, 41, 803198)
>>> ts = int(utcnow.timestamp())
>>> ts
1422205841
>>> datetime.datetime.fromtimestamp(ts)
datetime.datetime(2015, 1, 25, 18, 10, 41)
>>>

[mongodb] get a random document from a collection

Problem
From a MongoDB collection, you want to get a random document.

Solution

import random

def get_random_doc():
    # coll refers to your collection
    count = coll.count()
    return coll.find()[random.randrange(count)]

Pymongo documentation on cursors: here.

opening gzipped JSON files

Problem
I have a project where the input JSON file is almost 7 MB. I keep this project in my Dropbox folder, so that 7 MB text file seems to be a waste. Any way to reduce its size?

Solution
I zipped it up with gzip: “gzip -9 input.json“. This command produced a 1.3 MB “input.json.gz” file and deleted the original. Good. But how to open it in Python?

Normal way (without gzip):

import json

with open("input.json") as f:
    d = json.load(f)

Compressed way (with gzip):

import json
import gzip

with gzip.open("input.json.gz", "rb") as f:
    d = json.loads(f.read().decode("ascii"))

I didn’t notice any performance penalty. The application that first reads this json file starts as fast as before.

[flask] generate a secret key

Problem
To implement CSRF protection, Flask-WTF needs the application to configure an encryption key. This key is used to generate encrypted tokens that are used to verify the authenticity of requests with form data.

It looks like this:

app = Flask(__name__)
app.config['SECRET_KEY'] = '<the super secret key comes here>'

What secret key to use? How to generate this secret key?

Solution #1
In the official Quickstart the following method is suggested:

>>> import os
>>> os.urandom(24)
'\xfd{H\xe5<\x95\xf9\xe3\x96.5\xd1\x01O<!\xd5\xa2\xa0\x9fR"\xa1\xa8'

Just take that thing and copy/paste it into your code and you’re done.

Solution #2
In Django, when you create a new project, you get a settings file that contains a freshly generated 50 characters long secret key. Why not reuse this part from Django? The relevant section was easy to locate in the source code of Django:

import random
random = random.SystemRandom()

def get_random_string(length=12,
                      allowed_chars='abcdefghijklmnopqrstuvwxyz'
                                    'ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'):
    """
    Returns a securely generated random string.

    The default length of 12 with the a-z, A-Z, 0-9 character set returns
    a 71-bit value. log_2((26+26+10)^12) =~ 71 bits.

    Taken from the django.utils.crypto module.
    """
    return ''.join(random.choice(allowed_chars) for i in range(length))

def get_secret_key():
    """
    Create a random secret key.

    Taken from the Django project.
    """
    chars = 'abcdefghijklmnopqrstuvwxyz0123456789!@#$%^&*(-_=+)'
    return get_random_string(50, chars)

Its usage is very simple: just call the “get_secret_key()” function and copy/paste the output into your code.

pyvenv: create virtual environmets for Python 3.4+

For creating virtual environmets, I’ve used virtualenvwrapper so far. However, Python 3.4 contains the command pyvenv that does the same thing. Since it also installs pip in the virt. env., it can replace virtualenvwrapper.

I like to store my virtual environments in a dedicated folder, separated from the project directory. virtualenvwrapper, by default, stores the virt. env.’s in the ~/.virtualenvs folder. Since I got used to this folder, I will continue to keep my virt. env.’s in this folder.

pyvenv
Say we have our project folder here: ~/python/webapps/flasky_project. Create a virt. env. for this the following way:

pyvenv ~/.virtualenvs/flasky_project

It will create a Python 3 virt. env.

virtualenv / virtualenvwrapper
For the sake of completeness, I also write here how to create virt. env.’s with virtualenv and virtualenvwrapper:

# blog post: http://goo.gl/oEdtT3

# virtualenvwrapper for Python 3 or Python 2
mkvirtualenv -p `which python3` myenv3
mkvirtualenv -p `which python2` myenv2

# virtualenv for Python 3 or Python 2
virtualenv -p python3 myproject3
virtualenv -p python2 myproject2

# When the env. is created, activate it
# and launch the command python within.
# Verify if it's the correct version.

TL; DR
If you use Python 3.4+ and you need a virt. env., use the command “pyvenv“.