Archive
force requests to use IPv4
Problem
I have a script that periodically calls the API of a server to fetch some data. It worked well under Manjaro and Ubuntu. However, after a system update, the script stopped working on Ubuntu.
Diagnosis
It turned out that requests.get() couldn’t connect to the server. I tried to ping the host and under Ubuntu ping resolved an IPv6 address and it was unreachable. You can force ping to use IPv4 with the “-4” switch (ex.: “ping example.com -4“). Under Manjaro ping resolved an IPv4 address by default and the Python script worked well. Under Ubuntu, however, requests.get() wanted to use IPv6 and for some reason the given host was not reachable through that protocol.
Solution
In my Python code I used the following patch to force the usage of IPv4. requests relies on a lower level library, urllib3, thus the urllib3 part had to be patched:
import socket
import requests.packages.urllib3.util.connection as urllib3_cn
def allowed_gai_family():
family = socket.AF_INET # force IPv4
return family
urllib3_cn.allowed_gai_family = allowed_gai_family
It solved the issue under Ubuntu. This tip is from here.
APScheduler3 examples
A long time ago (in 2013) I wrote a post about APScheduler, see here. Since then a new version of APScheduler has come out, so it was time to update that old post.
So, the examples below were tested with APScheduler 3.6. The source code of the examples can be found on GitHub too (here).
#!/usr/bin/env python3
import os
from apscheduler.schedulers.blocking import BlockingScheduler
import mylogging as log
sched = BlockingScheduler()
@sched.scheduled_job('interval', minutes=60)
def once_per_hour():
"""
If you lauch this script at time T, then this function will be called
at T+60 minutes for the first time.
Ex.: if you lauch the script at 13h07, then this function will be called at 14h07
for the first time.
"""
log.info('calling once_per_hour')
@sched.scheduled_job('interval', minutes=2)
def once_per_hour():
"""
Call it every 2 minutes.
"""
log.info('2 minutes passed')
@sched.scheduled_job('cron', hour='*/4', minute=2)
def four_hours():
"""
Run this function every four hour + 2 minutes.
Ex.: it's called at 00h02, 04h02, 08h02, etc.
"""
log.info('calling four_hours')
@sched.scheduled_job('cron', day='*', hour=0, minute=5)
def daily_backup():
"""
Run it once a day at 5 minutes after midnight.
!!! If it takes time, then don't do the work here because the work
here will block the calling of the other functions! If it takes time, then
simply launch the work in the background. Here the slow work is collected in
a batch file and the batch file is launched in the background.
"""
log.info('calling daily_backup')
os.system("./daily_backup.sh &")
@sched.scheduled_job('cron', day='*', hour=0)
def midnight():
"""
Call it at midnight.
"""
log.info('calling midnight')
@sched.scheduled_job('cron', day='*', hour=18)
@sched.scheduled_job('cron', day='*', hour=19)
@sched.scheduled_job('cron', day='*', hour=20)
def triple_check():
"""
Call this function every day at 18h00, 19h00 and 20h00.
"""
log.info('calling triple_check')
@sched.scheduled_job('cron', day_of_week='wed', hour=19, minute=0)
@sched.scheduled_job('cron', day_of_week='sun', hour=19, minute=0)
def mini_backup():
"""
Call this function on Wednesday at 19h00 and
on Sunday at 19h00.
"""
log.info('calling mini_backup')
def main():
log.info('the scheduler is running...')
sched.start()
##############################################################################
if __name__ == "__main__":
main()
Find the complete source code on GitHub (here).
How to send a notification to your Android phone from a Python script
Problem
I have a long-running script and when something important happens, I’d like to send a notification to my Android phone. This way I shouldn’t be close to my machine. How to do it?
Solution
I found a very simple solution for this that can be set up in 3 minutes. This service can be found at https://notify.run/ . First, install the necessary package:
$ pip install notify-run
Then, register a channel:
$ notify-run register
It’ll print a QR code to the screen. Scan it with your phone and it brings you to a page where you can accept the notifications to your phone. Then, you can send notifications with the following Python snippet:
from notify_run import Notify
notify = Notify()
notify.send('sent from my Python script')
As can be seen, you don’t need to specify the name of the channel. It’s because it’s stored in the file ~/.config/notify-run and the notify_run module picks it up automatically.
You can also send a message to the channel from the command line, thus you can integrate it in any application:
curl https://notify.run/<channel_id> -d "message goes here"
Links
pdfmanip
Today I wrote a simple PDF manipulation CLI tool. You can find it here: https://github.com/jabbalaci/pdfmanip .
How old are you in days?
Problem
You want to calculate how old you are in days.
Solution
Let’s use a popular 3rd party date/time library for this purpose called pendulum.
As an example, let’s take Arnold Schwarzenegger, who was born on July 30, 1947. So let’s answer the following question: how old is Schwarzenegger today?
>>> import pendulum
>>>
>>> born = pendulum.parse("1947-07-30")
>>> born
DateTime(1947, 7, 30, 0, 0, 0, tzinfo=Timezone('UTC'))
>>> today = pendulum.now()
>>> today
DateTime(2018, 8, 14, 21, 32, 33, 248489, tzinfo=Timezone('Europe/Budapest'))
>>>
>>> diff = today - born
>>> diff
<Period [1947-07-30T00:00:00+00:00 -> 2018-08-14T21:32:33.248489+02:00]>
>>>
>>> diff.in_years()
71
>>> diff.in_words()
'71 years 2 weeks 1 day 19 hours 32 minutes 33 seconds'
>>> diff.in_days()
25948
table2csv
Problem
I wanted to extract a table from an HTML. I wanted to import it to Excel, thus I wanted it in CSV format for instance.
Solution
table2csv can do exactly this. Visit the project’s page on GitHub for examples.
Note that I could only make it work under Python 2.7.
