Signal processing (subscribe, emit, etc.)

If you had used a GUI framework like Qt, then you must have met signals. Signals provide an easy way to handle events in your program. For instance, when a button widget is created, you don’t need to code right there what should be executed when the button is clicked. Instead, the button emits the clicked signal and whatever callback is subscribed to that signal is executed to perform the desired action.

You can also add this functionality to simple scripts that have no GUI interfaces at all. Let’s see two solutions:

(1) blinker
“Blinker provides fast & simple object-to-object and broadcast signaling for Python objects.”

The Flask project too uses Blinker for signal processing.

Example #1:

from blinker import signal

>>> started = signal('round-started')
>>> def each(round):
...     print "Round %s!" % round
...
>>> started.connect(each)

>>> def round_two(round):
...     print "This is round two."
...
>>> started.connect(round_two, sender=2)

>>> for round in range(1, 4):
...     started.send(round)
...
# Round 1!
# Round 2!
# This is round two.
# Round 3!

How to read it? First, we create a global signal called “started“. The line “started.connect(each)” means: if the “started” signal is emitted (i.e. if this event happens), then call the “each” function.

Notice the “round” parameter of the “each” function: when a signal is emitted, it can transmit an object, i.e. at the place of signal emission you can send an arbitrary object with the signal.

The line “started.connect(round_two, sender=2)” means: if the “started” signal is emitted, then call the “round_two” function ONLY IF the object “2” is sent with the signal.

Then there is a loop from 1 to 3. In the loop we emit the “started” signal and the numbers are sent together with the signal. When the signal is emitted with “1“, “each” is called. When the signal is emitted with “2“, first “each” is called, then “round_two” is also executed since the signal holds the object “2” (the functions are called in the order of registration). Finally “each” is executed again with the number “3“.

Example #2 (taken from here):

class One:
    def __init__(self):
        self.two = Two()
        self.two.some_signal.connect(self.callback)

    def callback(self, data):    # notice the data parameter
        print 'Called'

class Two:
    some_signal = signal('some_signal')

    def process(self):
        # Do something
        self.some_signal.send()

one = One()
one.two.process()

“In code above, the Two object doesn’t even know if there’s some other object interested in its internal state changes. However, it does notify about them by emitting a signal that might be used by other objects (in this case a One object) to perform some specific action.” (by jcollado @SO)

(2) smokesignal
The project smokesignal is lighter than blinker.

Example #3:

from time import sleep
import smokesignal

@smokesignal.on('debug')
def verbose(val):
    print "#", val

# smokesignal.on('debug', verbose)
# smokesignal.on('debug', verbose, max_calls=5)    ## respond max. 5 times to the signal
# smokesignal.once('debug', verbose)    ## max_calls=1 this time

def main():
    for i in range(100):
        if i and i%10==0:
            smokesignal.emit('debug', i)
        sleep(.1)

It’s very similar to blinker. First we do the registration: if the “debug” signal is emitted, then execute the function “verbose“. In the loop if “i” is 10, 20, etc., then emit the signal “debug” and attach the value of “i” to the signal.

Smokesignal is just one file, thus it’s very easy to add to a project. However, it has a disadvantage:

“What would be great is if you could decorate instance methods… However, that doesn’t work because there is no knowledge of the class instance at the time the callback is registered to respond to signals.” (source)

They have a workaround but I find it ugly.

As seen in Example #2, with blinker you can register instance methods to be called when a signal is emitted.

Conclusion
If you want a lightweight solution without instance methods, use smokesignal. If you also want to call instance methods when an event happens, use blinker.

What’s New In Python 3.0

What’s New In Python 3.0 by Guido @docs.python.org

static variables

Problem
In C, you can use static variables in a function. These variables are initialized once and then they keep their values between function calls. How to do the same in Python?

Solution
Here is an example:

def name_generator():
    if not hasattr(name_generator, "cnt"):
        name_generator.cnt = 0    # it doesn't exist yet, so initialize it
    #
    result = "name-{cnt}".format(cnt=name_generator.cnt)
    name_generator.cnt += 1
    return result


for _ in range(3):
    print name_generator()
# name-0
# name-1
# name-2

Variables and functions of an object

Problem
You are using a library and a function returns an object. What kind of object is it? What variables does it have? What functions does it have? You would like to investigate this object you received.

Solution
I came up with the following methods:

from pprint import pprint


def print_variables_of(obj):
    """variables of an object"""
    pprint (vars(obj))


def print_callables_of(obj):
    """callables (functions too) of an object"""
    li = []
    for name in dir(obj):
        attr = getattr(obj, name)
        if hasattr(attr, '__call__'):
            li.append(name)
    pprint(li)

The first one prints the variables of an object, while the second one prints the callables (functions fall in this category) of the object.

Posts on Python and Big Data

See Neal Caren’s tutorials and posts on Python and Big Data here: http://nealcaren.web.unc.edu/big-data/.

How to make a python, command-line program autocomplete arbitrary things

This entry is based on this SO post.

Problem
You have an interactive command-line Python script and you want to add autocompletion to it when hitting the TAB key.

Solution
Here is a working example (taken from here):

import readline

addrs = ['angela@domain.com', 'michael@domain.com', 'david@test.com']

def completer(text, state):
    options = [x for x in addrs if x.startswith(text)]
    try:
        return options[state]
    except IndexError:
        return None

readline.set_completer(completer)
readline.parse_and_bind("tab: complete")

while True:
    inp = raw_input("> ")
    print "You entered", inp

bash-like functionalities in command-line Python script

Problem
You have an interactive Python script that reads input from the command line. You want to add bash-like functionalities to it like moving the cursor with the arrows, jump to the front with Home, jump to the end with End, browse previous commands with the up arrow, etc.

Solution
You won’t believe what is needed for this:

import readline

Yes, that’s it. Just import it and you are good to go.

Minimal example:

import readline

while True:
    inp = raw_input("> ")
    print "You entered", inp

Alternative
Before discovering the readline module, I used to start my scripts with “rlwrap“:

rlwrap my_script.py

It does the trick too. I put the line above in a script called “my_script.sh” and I launched this latter one. However, “import readline” is simpler.