Getting started with Django on a DigitalOcean VPS

“Django is a high-level Python framework for developing web applications rapidly. DigitalOcean’s Django One-Click app quickly deploys a preconfigured development environment to your VPS employing Django, Nginx, Gunicorn, and Postgres.”

More info here.

An SQLite example

Here you will find a very basic program that stores data in SQLite. It is a simple example. If I need to write a program that needs sqlite, it can serve as a basis and I won’t need to write it from scratch.

Problem
Visit the page /r/earthporn, extract the image URLs and save the URLs of images stored on imgur in an sqlite table.

Solution
earthporn.py:

#!/usr/bin/env python
# encoding: utf-8

import requests
import database as db    # see below

subreddit = "earthporn"

def main():
    db.init()    # Important! It will connect to the DB.

    r = requests.get('http://www.reddit.com/r/{sr}/.json'.format(sr=subreddit))
    d = r.json()
    children = d["data"]["children"]
    for e in children:
        url = e["data"]["url"]
        if "imgur.com" in url:
#            print url
            db.add_image(url, subreddit)

####################

if __name__ == "__main__":
    main()

database.py:

#!/usr/bin/env python

"""
Sqlite database handler.
"""

import os
import sqlite3
import atexit
import random
import termcolor

PATH = os.path.dirname(os.path.abspath(__file__))
SCHEMA = """
CREATE TABLE "images" (
    "url" TEXT PRIMARY KEY NOT NULL,
    "subreddit" TEXT,
    "insert_date" DEFAULT CURRENT_TIMESTAMP
)
"""
SQLITE_DB = PATH + '/images.sqlite'

conn = None

def get_random_image():
    query = "SELECT url FROM images"
    cursor = conn.cursor()
    cursor.execute(query)
    result = cursor.fetchall()
    return random.choice(result)[0]

def add_image(url, subreddit=None):
    try:
        if subreddit:
            query = "INSERT INTO images (url, subreddit) VALUES (?, ?)"
            conn.execute(query, (url, subreddit))
        else:
            query = "INSERT INTO images (url, subreddit) VALUES (?, NULL)"
            conn.execute(query, (url,))
        #
        print termcolor.colored("# {url} added to DB".format(url=url), "cyan")
    except sqlite3.IntegrityError:
        print termcolor.colored("# the image {0} is already in the DB...".format(url), "red")

def get_all_images():
    query = "SELECT url FROM images"
    cursor = conn.cursor()
    cursor.execute(query)
    result = cursor.fetchall()
    return result

def create_db():
    """
    Create the DB if not exists.
    """
    global conn
    conn = sqlite3.connect(SQLITE_DB)
    conn.executescript(SCHEMA)

def init(commit=True):
    """
    Initialize the DB.
    """
    global conn
    if commit:
        atexit.register(commit_and_close)
    else:
        atexit.register(close)

    if not os.path.exists(SQLITE_DB):
        create_db()
    if not conn:
        conn = sqlite3.connect(SQLITE_DB)

def commit():
    """
    Commit.
    """
    if conn:
        conn.commit()

def close():
    """
    Close.
    """
    if conn:
        conn.close()

def commit_and_close():
    """
    Commit and close DB connection.

    As I noticed, commit() must be called, otherwise changes
    are not committed automatically when the program terminates.
    """
    if conn:
        conn.commit()
        conn.close()

####################

if __name__ == "__main__":
    init()

The module database.py is responsible for handling the database. When you need to work with the database, just include this file and call the appropriate function(s).

Get an email notification when a site is up

Problem
Today I wanted to submit a paper to a conference but the site that manages submissions (EasyChair) is down for maintenance. I got this message:

EasyChair is Being Moved to a New Server

Quite unexpectedly, EasyChair had to be moved to a new server.

We are moving the system itself and restoring all data. We are sorry for any inconveniences caused.

The move is to be completed during Friday, May the 9th. We cannot give precise time at the moment. We are working hard to make it up and running as soon as possible.

I don’t want to check this site manually. How could I get notified when the site is up?

Solution
I wrote a simple script that regularly checks the site. When it’s up, it sends me an email notification and quits.

Here I reused some functions from my jabbapylib library. You can easily include those functions to make it standalone.

#!/usr/bin/env python
# encoding: utf-8

from __future__ import (absolute_import, division,
                        print_function, unicode_literals)

from time import sleep

import schedule

from jabbapylib.mail.gmail_send import mail_html
from jabbapylib.web.web import get_page, html_to_text
import sys

URL = 'http://www.easychair.org/'
USERNAME = "..."
PASSWORD = "..."
sender = {
    'gmail_user': USERNAME,
    'gmail_name': "PyChecker",
    'gmail_pwd': PASSWORD,
}
STOP = False

def is_down(url):
    """
    Check if the site is down.

    Return True if it's down. Otherwise, return False.
    """
    html = get_page(url)
    text = " ".join(html_to_text(html).split())
    return "We are sorry for any inconveniences caused." in text

def job():
    global STOP

    print()
    if is_down(URL):
        print("The site is still down.")
    else:
        print("The site is up.")
        mail_html(sender, "TO@gmail.com",
            "EasyChair is up!",
            "Send your paper: <a href='https://www.easychair.org'>EasyChair</a>.")
        print("E-mail notification was sent.")
        STOP = True

def main():
    schedule.every(5).minutes.do(job)

    print("Running...")

    while not STOP:
        schedule.run_pending()
        sleep(10)
        sys.stdout.write('.'); sys.stdout.flush()

####################

if __name__ == "__main__":
    main()