angle between two lines

December 10, 2019 Leave a comment

Problem
You have two lines. What is the angle between them?

Or, you have 3 points, say A, B and C. If you go from A to B (vector 1), then from B to C (vector 2), then what is the angle at point B between the two vectors?

Solution
There is a nice blog post about it here: Find the Angle between three points from 2D using python.

Here is a Python code that is based on the one that you can find in the aforementioned blog post:

import math
from typing import NamedTuple

class Point(NamedTuple):
    x: int
    y: int

def angle(a: Point, b: Point, c: Point) -> float:
    ang = math.degrees(math.atan2(c[1]-b[1], c[0]-b[0]) - math.atan2(a[1]-b[1], a[0]-b[0]))
    return ang + 360 if ang < 0 else ang
Categories: python Tags: , , , , ,

monitoring key presses in a console application in a thread

November 16, 2019 Leave a comment

Problem
I wrote a console application that shows a table and updates the screen every second. Later, I wanted to add a table sorting functionality. For instance, if you press ‘b’, then the table is sorted by the 2nd column, pressing ‘c’ would sort the table by the 3rd column, etc.

I found some keyboard monitoring libraries but they were all blocking, i.e. you had to call a function which was waiting for a key press. If you didn’t press any key, this function was just waiting.

However, in my program I had an infinite loop that was doing the following steps: (1) clear the screen, (2) draw the table, (3) repeat. If I add anywhere the keyboard monitoring, the loop gets blocked somewhere.

Solution
I asked this question on reddit (see here), and /u/Viddog4 suggested that I should use a thread. Of course! I have the main loop that draws the table, and I have a thread in the background that monitors the keyboard.

Here is a simplified code that demonstrates the idea:

#!/usr/bin/env python3

"""
pip3 install pynput xlib
"""

import threading
from time import sleep

from pynput.keyboard import Key, Listener


class myThread(threading.Thread):
    def __init__(self, _id, name):
        super().__init__()
        self.daemon = True    # daemon threads are killed as soon as the main program exits
        self._id = _id
        self.name = name

    def on_press(self, key):
        print('{0} pressed'.format(key))

    def on_release(self, key):
        print('{0} release'.format(key))
        if key == Key.esc:
            # Stop listener
            return False

    def run(self):
        with Listener(on_press=self.on_press, on_release=self.on_release) as listener:
            listener.join()


def main():
    thread1 = myThread(1, "thread_1")
    thread1.start()
    # main loop:
    while True:
        print(".", flush=True)
        try:
            sleep(1)
        except KeyboardInterrupt:
            break

##########

if __name__ == "__main__":
    main()

You can stop the thread with Esc. You can terminate the whole program with Ctrl+C. The thread is registered as a daemon thread, which means that if the main program exits (e.g. you press Ctrl+C), then daemon threads are automatically stopped.

Links

Categories: python Tags: , , ,

simple keylogger

November 16, 2019 Leave a comment

I was working on a console application and I wanted to add the functionality to listen to keyboard presses in an infinite loop. I used the pynput library and tried this basic code that I found on the project’s web site:

# pip3 install pynput xlib

from pynput.keyboard import Key, Listener

def on_press(key):
    print('{0} pressed'.format(key))

def on_release(key):
    print('{0} release'.format(key))
    if key == Key.esc:
        # Stop listener
        return False

# Collect events until released
with Listener(on_press=on_press, on_release=on_release) as listener:
    listener.join()

It worked well. I let it run and switched to another window when I noticed that the script was still monitoring what keys I press, though I was in another window! So it monitors the keyboard globally. And, under Linux, I didn’t even have to start it with sudo.

So, if you need a simple keylogger, here it is :) You don’t need to add much to the code above to have a working keylogger.

Categories: python Tags: , ,

flush the stdin

November 16, 2019 Leave a comment

Problem
I wrote a terminal application that was reading key presses from the keyboard using the pynput library. When the program terminated, it printed on the screen the keys that I pressed while the program was running. How to get rid of this side effect?

Solution
First I tried sys.stdin.flush() but it didn’t help. However, the following worked:

import sys
import termios

termios.tcflush(sys.stdin, termios.TCIOFLUSH)

Calling this line before quitting successfully flushed the standard input.

Categories: python Tags: , , ,

Getting started with PyQt5

November 9, 2019 Leave a comment

I made a GitHub repository where I collected some simple PyQt5 examples: https://github.com/jabbalaci/GUI-Dev-with-Python . I provide some examples written manually and you can also find an example that was made with Qt Designer.

Categories: python Tags: , ,

[flask] show external images with an app. running on localhost

November 3, 2019 Leave a comment

Problem
I have a Flask app. that I run on my local machine. I wrote it years ago and it worked well at that time. Now, I tried to run it again but the external images didn’t appear, they were broken. These are images that are located somewhere else, I just link them. When I copied their URL and pasted in the location bar of the browser, they appeared. What da?

Solution
It seems Firefox is blocking localhost, so when I visited http://127.0.0.1:5000, some of its elements got blocked.

As a workaround, I added the following line in my /etc/hosts file:

127.0.0.1   jabba-hq

Then, when I visted http://jabba-hq:5000, the images appeared normally.

Categories: flask, python Tags: ,

moving from pipenv to poetry

November 2, 2019 Leave a comment

In the past 1-1.5 years I’ve used pipenv, which worked very well for me but it was annoyingly slow. Creating a virtual environment took a minute sometimes, and installing the dependencies often took several minutes. I’ve heard lots of good things about poetry, so I decided to give it a try. After reading its documentation, I could do everything with it that I needed, so in the future I’ll use poetry for managing my virtual environments.

Getting started
The home page of poetry is here.

The installation is a bit speacial: https://poetry.eustace.io/docs/. After the installation, open a new terminal and issue the command “poetry“. If it starts, then the installation was successful :)

The installer puts poetry in the folder ~/.poetry .

By default, new virtual environments are put in the folder ~/.cache/pypoetry/virtualenvs/. However, I prefer collecting the virtual environments in ~/.virtualenvs. For this, issue the following command:

$ poetry config settings.virtualenvs.path $HOME/.virtualenvs

Actually, it’s a good idea to put this value in an environment variable too. Add this line to the end of your ~/.bashrc file:

export WORKON_HOME=$HOME/.virtualenvs

The config info is stored in the file ~/.config/pypoetry/config.toml .

Update poetry

poetry self:update

Use case #1: Create a new poetry project from scratch

$ mkdir demo
$ cd demo
$ poetry new [--src] .

It will create the basic structure of a new project (scaffolding). If you want to store the source files in the directory “src/“, then add the option “--src“. The dot at the end means the current directory.

Create / activate / deactivate the virtual environment

poetry shell

If the virt. env. doesn’t exist yet, poetry will create it. For this to work, you need to have the file pyproject.toml .
If the virt. env. exists, then “poetry shell” will activate it. Actually, it opens a subshell.
To deactivate the virt. env., simply close the subshell (with Ctrl-D for instance).

Install / remove a package

poetry add requests [--dev]

It’ll install the given dependency (requests in the example) and add it to your pyproject.toml file. With the option “--dev“, the package will be installed as a development dependency.

poetry remove requests

Remove the package and if it installed some sub-dependencies, then those packages are also removed.

Help
Simply launch “poetry” to see all the options. To get help about an option, e.g. about “install”, then use this:

poetry help install

Show installed packages

$ poetry show [--no-dev]
$ poetry show --tree [--no-dev]

Show the list of installed packages. With “--tree“, show the result in a tree layout. With “--no-dev“, development packages will be hidden.

Version control
Upload both pyproject.toml and poetry.lock in your version control system.

Where is the directory of the virtual environment?
To figure out the directory of the virt. env., use the command

poetry show -v

and the result will be in the first line. If you need just this information, then here is how to extract it:

venv_folder=`(poetry show -v | head -1) 2>/dev/null | cut -d" " -f3`

Which packages are outdated?

$ poetry show -o
$ poetry show --outdated    # same thing

It will show which packages have newer versions.

Update a package / all packages

$ poetry update requests

Update the given package. However, it takes into account the SemVer rules, defined in pyproject.toml . Read more about versioning here: https://poetry.eustace.io/docs/versions/.

poetry update

Update all packages. Again, it follows the SemVer rules.

Edit your pyproject.toml
If you edit the .toml file, you can verify it with

poetry check

Use case #2: Convert an existing project to a poetry project
All right. Say you have a project and you want to convert it to a poetry project. For example it was managed by pipenv but you want to switch to poetry. Here are the steps:

  • enter the project directory
  • poetry init -n” will create a pyproject.toml file without asking any embarrassing questions. Review this file if you want. For instance, set the proper Python version.
  • poetry shell” — create a virt. env. for the project
  • poetry add pkg1 pkg2” — install the given packages (copy them from your Pipfile, for instance)
  • poetry add pkg1 pkg2 --dev” — install the development packages

Use case #3: Checkout a poetry project
Let’s say you download a project from GitHub and it uses poetry, i.e. it has a pyproject.toml file and possibly a poetry.lock file too. How to create a virt. env. for it? How to install its packages? Aaaargghhhh!

Relax and issue this simple command inside the project folder:

poetry install [--no-dev]

By default, it also installs the development packages. Use “--no-dev” if you don’t need the dev. packages.

Windows support
Poetry works under Windows too. The installation is the same. I had to install curl for Windows, but then I could download the installer without any problem.

The installer adds the folder %USERPROFILE%\.poetry\bin to your PATH.

The virtual environments are created here: %USERPROFILE%\AppData\Local\pypoetry\Cache\virtualenvs\ .

Unfortunately, when I activated a virt. env., the prompt didn’t change.

Troubleshooting

Under Ubuntu 18.04 I had to use some tricks to force poetry to use Python 3. Find the details here. In short: edit the file ~/.poetry/bin/poetry and update its first line to #!/usr/bin/env python3. If you update poetry, it’s very likely you’ll have to change this file again.

Links

Categories: python Tags: , ,

Python Developers Survey 2018 Results

November 2, 2019 Leave a comment

I just filled out the Python Developers Survey 2019, and at the end there was a link to the summary of the last year’s survey: Python Developers Survey 2018 Results. It can be an interesting read.

Categories: python Tags: , ,

force requests to use IPv4

June 28, 2019 Leave a comment

Problem
I have a script that periodically calls the API of a server to fetch some data. It worked well under Manjaro and Ubuntu. However, after a system update, the script stopped working on Ubuntu.

Diagnosis
It turned out that requests.get() couldn’t connect to the server. I tried to ping the host and under Ubuntu ping resolved an IPv6 address and it was unreachable. You can force ping to use IPv4 with the “-4” switch (ex.: “ping example.com -4“). Under Manjaro ping resolved an IPv4 address by default and the Python script worked well. Under Ubuntu, however, requests.get() wanted to use IPv6 and for some reason the given host was not reachable through that protocol.

Solution
In my Python code I used the following patch to force the usage of IPv4. requests relies on a lower level library, urllib3, thus the urllib3 part had to be patched:

import socket
import requests.packages.urllib3.util.connection as urllib3_cn

def allowed_gai_family():
    family = socket.AF_INET    # force IPv4
    return family

urllib3_cn.allowed_gai_family = allowed_gai_family

It solved the issue under Ubuntu. This tip is from here.

Categories: python Tags: , , , , ,

APScheduler3 examples

June 26, 2019 Leave a comment

A long time ago (in 2013) I wrote a post about APScheduler, see here. Since then a new version of APScheduler has come out, so it was time to update that old post.

So, the examples below were tested with APScheduler 3.6. The source code of the examples can be found on GitHub too (here).

#!/usr/bin/env python3

import os
from apscheduler.schedulers.blocking import BlockingScheduler
import mylogging as log

sched = BlockingScheduler()


@sched.scheduled_job('interval', minutes=60)
def once_per_hour():
    """
    If you lauch this script at time T, then this function will be called
    at T+60 minutes for the first time.
    Ex.: if you lauch the script at 13h07, then this function will be called at 14h07
    for the first time.
    """
    log.info('calling once_per_hour')


@sched.scheduled_job('interval', minutes=2)
def once_per_hour():
    """
    Call it every 2 minutes.
    """
    log.info('2 minutes passed')


@sched.scheduled_job('cron', hour='*/4', minute=2)
def four_hours():
    """
    Run this function every four hour + 2 minutes.
    Ex.: it's called at 00h02, 04h02, 08h02, etc.
    """
    log.info('calling four_hours')


@sched.scheduled_job('cron', day='*', hour=0, minute=5)
def daily_backup():
    """
    Run it once a day at 5 minutes after midnight.

    !!! If it takes time, then don't do the work here because the work
    here will block the calling of the other functions! If it takes time, then
    simply launch the work in the background. Here the slow work is collected in
    a batch file and the batch file is launched in the background.
    """
    log.info('calling daily_backup')
    os.system("./daily_backup.sh &")


@sched.scheduled_job('cron', day='*', hour=0)
def midnight():
    """
    Call it at midnight.
    """
    log.info('calling midnight')


@sched.scheduled_job('cron', day='*', hour=18)
@sched.scheduled_job('cron', day='*', hour=19)
@sched.scheduled_job('cron', day='*', hour=20)
def triple_check():
    """
    Call this function every day at 18h00, 19h00 and 20h00.
    """
    log.info('calling triple_check')


@sched.scheduled_job('cron', day_of_week='wed', hour=19, minute=0)
@sched.scheduled_job('cron', day_of_week='sun', hour=19, minute=0)
def mini_backup():
    """
    Call this function on Wednesday at 19h00 and
    on Sunday at 19h00.
    """
    log.info('calling mini_backup')


def main():
    log.info('the scheduler is running...')
    sched.start()

##############################################################################

if __name__ == "__main__":
    main()

Find the complete source code on GitHub (here).

Categories: python Tags: , , ,