a command line progress bar for your loops

Problem
When I was working on Project Euler, there were several problems that I solved with a brute force approach and thus the runtime was several hours. To see some changes and to know approximately when it finishes, I added a simple counter to the main loop that showed the progress in percentage.

Is there a simpler way to add a progress bar to a loop?

Solution
The project tqdm addresses this problem. Reddit discussion is here.

Usage example:

import time
from tqdm import tqdm

def main():
    for i in tqdm(range(10000)):
        time.sleep(.005)

Notes:

• you can use xrange too: tqdm(xrange(10000))
• you can write trange: trange(10000)

On the github page of the project you will find an animated gif too.

bash-like functionalities in command-line Python script

Problem
You have an interactive Python script that reads input from the command line. You want to add bash-like functionalities to it like moving the cursor with the arrows, jump to the front with Home, jump to the end with End, browse previous commands with the up arrow, etc.

Solution
You won’t believe what is needed for this:

import readline

Yes, that’s it. Just import it and you are good to go.

Minimal example:

import readline

while True:
    inp = raw_input("> ")
    print "You entered", inp

Alternative
Before discovering the readline module, I used to start my scripts with “rlwrap“:

rlwrap my_script.py

It does the trick too. I put the line above in a script called “my_script.sh” and I launched this latter one. However, “import readline” is simpler.

docopt: a pythonic command line arguments parser that kicks your ass

Docopt is a parser for command line arguments. There are other alternatives like optparse and argparse but docopt is the king. Why? Because it cannot be any simpler :) All you have to do is write an interface description to your script that is well-known from man pages and from the help of other command line applications.

Example (taken from the docopt site):

Naval Fate.

Usage:
  naval_fate.py ship new <name>...
  naval_fate.py ship <name> move <x> <y> [--speed=<kn>]
  naval_fate.py ship shoot <x> <y>
  naval_fate.py mine (set|remove) <x> <y> [--moored|--drifting]
  naval_fate.py -h | --help
  naval_fate.py --version

Options:
  -h --help     Show this screen.
  --version     Show version.
  --speed=<kn>  Speed in knots [default: 10].
  --moored      Moored (anchored) mine.
  --drifting    Drifting mine.

Then you simply pass this string to docopt. When you execute the script, the parameters will be parsed by docopt according to the textual description above. Say you launch this script with the following parameters:

./naval_fate.py ship Guardian move 10 50 --speed=20

Docopt will parse it and return the following dictionary:

{"--drifting": false,
 "--help": false,
 "--moored": false,
 "--speed": "20",
 "--version": false,
 "<name>": ["Guardian"],
 "<x>": "10",
 "<y>": "50",
 "mine": false,
 "move": true,
 "new": false,
 "remove": false,
 "set": false,
 "ship": true,
 "shoot": false}

That’s all. You can try docopt online here.

If you got interested, watch the presentation of its author (you will find it on the top of the home page of the project).

It’s available on GitHub, where you can also check out some more examples.

Usage: either install it via pip, or just simply add the file docopt.py next to your script.

Countdown in command line

I have a console application that runs in an eternal loop. It does something then it sleeps for 5 minutes. How to do this pause in a nice way?

(1) time.sleep(300)

The first idea is to put it to sleep for 5 minutes. It does the job but when I check the output of the script, I have no idea when it continues. So it’s not very informative.

(2) print a dot each second

Another way is to print a dot each second. If you use sys.stdout.write('.'), then don’t forget to set the output unbuffered. It’s a bit better but still not perfect.

(3) countdown

The nicest way may be a countdown that goes from 300 down to 0. Now, you don’t want to print each number in a new line, do you? How to keep the countdown in the same line?

#!/usr/bin/python2.7 -u
# -u switch: unbuffered output

def wait(sec):
    while sec > 0:
        sys.stdout.write(str(sec) + '     \r')
        sec -= 1
        time.sleep(1)

The trick is the ‘\r‘, which means “carriage return”. That is: print the number, clear with some spaces, then jump back to the beginning of the line.

Nice, but… How to get rid of the cursor? You can switch it off/on in Linux easily. Here is my complete solution:

#!/usr/bin/python2.7 -u

import os
import sys
from time import sleep

def wait(sec):
    while sec > 0:
        sys.stdout.write(str(sec) + '     \r')
        sec -= 1
        sleep(1)
        
def main():
    os.system('setterm -cursor off')
    try:
        while True:
            # an external command that I call regularly
            os.system('./proxies.py')
            wait(300)
    except KeyboardInterrupt:
        print
    finally:
        os.system('setterm -cursor on')

############################################ 

if __name__ == "__main__":
    main()

The finally part makes sure that the cursor will be switched back on, even if the script is interrupted with CTRL+C for instance.

The same idea in a more elegant way:

class CursorOff(object):
    def __enter__(self):
        os.system('setterm -cursor off')
        
    def __exit__(self, *args):
        os.system('setterm -cursor on')

def wait(sec):
    while sec > 0:
        sys.stdout.write(str(sec) + '     \r')
        sec -= 1
        try:
            sleep(1)
        except KeyboardInterrupt:
            print
            return

with CursorOff():
    wait(3)

I put it in a module that you can find in my jabbapylib library, here.

Terminate a script after X seconds

Problem
In Qt, there is a class called QTimer. With QTimer you can, for instance, terminate your application in X seconds. Example:

#!/usr/bin/env python

import sys

from PySide.QtCore import *
from PySide.QtGui import *

SEC = 1000    # 1 sec. is 1000 msec.

def main():
    app = QApplication(sys.argv)
    form = QDialog()
    form.show()
    # suicide in 3 seconds:
    QTimer.singleShot(3 * SEC, app.quit)
    app.exec_()

if __name__ == "__main__":
   main()

Question: how to have the same effect in a command-line application?

Solution
I came up with the following solution:

#!/usr/bin/env python

import sys
from time import sleep
import signal

class MyTimer(object):
    """
    Similar to Qt's QTimer. Call a function in a specified time.

    Time is given in sec. Usage:
    mt = MyTimer()
    mt.singleShot(<sec>, <function_name>)

    After setting it, you can still disable it:
    mt.disable()

    If you call it several times, any previously scheduled alarm
    will be canceled (only one alarm can be scheduled at any time).
    """
    def singleShot(self, sec, func):
        self.f = func
        signal.signal(signal.SIGALRM, self.handler)
        signal.alarm(sec)

    def handler(self, *args):
        self.f()

    def disable(self):
        signal.alarm(0)

def main():
    while True:
        print '.',
        sleep(0.5)
        
if __name__ == "__main__":
    mt = MyTimer()
    mt.singleShot(3, sys.exit)
    main()

As can be seen, the main function has an eternal loop. However, this program will terminate in 3 seconds. I’m imitating QTimer’s singleshot. The differences: (1) you must create a MyTimer object, and (2) time is given in seconds, not in milliseconds. You can also write it in one line if you want: MyTimer().singleShot(3, sys.exit).

It is written in a general form, so instead of sys.exit, you can also call a different function.

get screen resolution

Problem
You need the current screen resolution.

Solution #1 (command-line)

xrandr | grep '*'

Sample output:

   1920x1080      50.0*    51.0     52.0

_{This tip is from here.}

Solution #2 (Python)
Let’s see the previous solution in Python:

from jabbapylib.process import process

def get_screen_resolution():
    """
    Screen resolution (as a tuple).
    """
    result = [x for x in process.get_simple_cmd_output('xrandr').split('\n') if '*' in x][0]
    result = tuple([int(x) for x in result.split()[0].split('x')])
    return result

Sample output:

(1920, 1080)

process.get_simple_cmd_output is part of my jabbapylib library. get_simple_cmd_output() simply executes an external command and returns its output as a string.

pyp – Python Power at the Prompt (The Pyed Piper)

“Pyp is a Linux command-line text manipulation tool similar to awk or sed, but which uses standard python string and list methods as well as custom functions evolved to generate fast results in an intense production environment. Pyed Pyper was developed at Sony Pictures Imageworks to facilitate the construction of complex image manipulation “one-liner” commands during visual effects work on Alice in Wonderland, Green Lantern, and the upcoming The Amazing Spiderman.” (source)

Installable via jabbatron.

This post is a reminder for me. As soon as I go through its documentation, I will write some notes about it.