Archive

Posts Tagged ‘reddit’

An SQLite example

Here you will find a very basic program that stores data in SQLite. It is a simple example. If I need to write a program that needs sqlite, it can serve as a basis and I won’t need to write it from scratch.

Problem
Visit the page /r/earthporn, extract the image URLs and save the URLs of images stored on imgur in an sqlite table.

Solution
earthporn.py:

#!/usr/bin/env python
# encoding: utf-8

import requests
import database as db    # see below

subreddit = "earthporn"

def main():
    db.init()    # Important! It will connect to the DB.

    r = requests.get('http://www.reddit.com/r/{sr}/.json'.format(sr=subreddit))
    d = r.json()
    children = d["data"]["children"]
    for e in children:
        url = e["data"]["url"]
        if "imgur.com" in url:
#            print url
            db.add_image(url, subreddit)

####################

if __name__ == "__main__":
    main()

database.py:

#!/usr/bin/env python

"""
Sqlite database handler.
"""

import os
import sqlite3
import atexit
import random
import termcolor

PATH = os.path.dirname(os.path.abspath(__file__))
SCHEMA = """
CREATE TABLE "images" (
    "url" TEXT PRIMARY KEY NOT NULL,
    "subreddit" TEXT,
    "insert_date" DEFAULT CURRENT_TIMESTAMP
)
"""
SQLITE_DB = PATH + '/images.sqlite'

conn = None

def get_random_image():
    query = "SELECT url FROM images"
    cursor = conn.cursor()
    cursor.execute(query)
    result = cursor.fetchall()
    return random.choice(result)[0]

def add_image(url, subreddit=None):
    try:
        if subreddit:
            query = "INSERT INTO images (url, subreddit) VALUES (?, ?)"
            conn.execute(query, (url, subreddit))
        else:
            query = "INSERT INTO images (url, subreddit) VALUES (?, NULL)"
            conn.execute(query, (url,))
        #
        print termcolor.colored("# {url} added to DB".format(url=url), "cyan")
    except sqlite3.IntegrityError:
        print termcolor.colored("# the image {0} is already in the DB...".format(url), "red")

def get_all_images():
    query = "SELECT url FROM images"
    cursor = conn.cursor()
    cursor.execute(query)
    result = cursor.fetchall()
    return result

def create_db():
    """
    Create the DB if not exists.
    """
    global conn
    conn = sqlite3.connect(SQLITE_DB)
    conn.executescript(SCHEMA)

def init(commit=True):
    """
    Initialize the DB.
    """
    global conn
    if commit:
        atexit.register(commit_and_close)
    else:
        atexit.register(close)

    if not os.path.exists(SQLITE_DB):
        create_db()
    if not conn:
        conn = sqlite3.connect(SQLITE_DB)

def commit():
    """
    Commit.
    """
    if conn:
        conn.commit()

def close():
    """
    Close.
    """
    if conn:
        conn.close()

def commit_and_close():
    """
    Commit and close DB connection.

    As I noticed, commit() must be called, otherwise changes
    are not committed automatically when the program terminates.
    """
    if conn:
        conn.commit()
        conn.close()

####################

if __name__ == "__main__":
    init()

The module database.py is responsible for handling the database. When you need to work with the database, just include this file and call the appropriate function(s).

Categories: python Tags: , ,

imgur album downloader

April 5, 2014 Leave a comment

This little script can download all the images in an imgur gallery.

Categories: python Tags: , ,

Improving the sidebar of /r/python

March 21, 2014 Leave a comment

In the sidebar of /r/python, there was a very unpythonic infinite loop for a long time:

while 1:
    # do something

Today I sent a message to the moderators and they changed it:

while

Update: I got a message from them.

Categories: python Tags: , , ,

A course on Udemy about Python GUI programming with Qt

December 4, 2012 Leave a comment
Categories: python Tags: , , , ,

submitting a link to reddit and adding a comment

March 30, 2012 Leave a comment

Problem
You want to send a large number of links to reddit and you want to get it done with a script. How to send links and how to add comments to them?

Solution
First, install the reddit api:

sudo pip install reddit -U

Then you can use my RedBot to perform the task. It is part of my jabbapylib library.

#!/usr/bin/env python

"""
Simple API for:
* posting links to reddit
* commenting on a post

This requires that you have a reddit account. Put your username and
password in the following files:
* ~/reddit_username.txt
* ~/reddit_password.txt

To use RedBot, just make a subclass of it and give it a cool name:

class HyperBot(RedBot):
    def __init__(self):
        super(HyperBot, self).__init__()
        self.name = 'HyperBot'
        
Now you are ready to flood reddit :)

# from jabbapylib.reddit import red
"""

import reddit
from jabbapylib.filesystem import fs
from jabbapylib.platform import platform

USERNAME_TXT = '{home}/reddit_username.txt'.format(home=platform.get_home_dir())
PASSWORD_TXT = '{home}/reddit_password.txt'.format(home=platform.get_home_dir())
#
USERNAME = fs.read_first_line(USERNAME_TXT)
PASSWORD = fs.read_first_line(PASSWORD_TXT)


class RedBot(object):
    def __init__(self):
        self.name = 'RedBot'
        self.username = USERNAME
        self.password = PASSWORD
        #
        self.r = reddit.Reddit(user_agent=self.name)
        self.r.login(username=self.username, password=self.password)
        self.last_post = None   # Submission object
        self.permalink = None   # URL of the last post
        
    def submit_link(self, url, subreddit, title):
        """
        The return value (res) is a Submission object or None.
        URL of the newly created post: res.permalink
        """
        try:
            self.last_post = self.r.submit(subreddit, title, url=url)
            self.permalink = self.last_post.permalink
            print '# url to send: {url}'.format(url=url)
            print '# submitted to: {pl}'.format(pl=self.permalink)
            return self.last_post
        except:
            print >>sys.stderr, "Warning: couldn't submit {url}".format(url=url)
            return None
        
    def add_comment(self, comment):
        if self.last_post:
            self.last_post.add_comment(comment)
            print '# comment added'

#############################################################################

if __name__ == "__main__":
# here is how to use it:
#    url = '...'
#    subreddit = '...'
#    title = "..."
#    comment = '...'
#    r = RedBot()
#    r.submit_link(url, subreddit, title)
#    r.add_comment(comment)
    pass

You can find the current version here.

Thanks to Bryce Boe, the maintainer of the reddit api, who kindly answered my questions.

Links
Python Reddit API Wrapper’s documentation

Categories: python Tags: , ,

Upload an image to imgur.com from Python

November 6, 2011 Leave a comment

If you are familiar with reddit, you must have noticed that most images are hosted on imgur. I would like to upload several images from my computer and I want to collect their URLs on imgur. Let’s see how to do that.

Imgur has an API, this is what we’ll use. Anonymous upload is fine for my needs. For this you need to register and you get an API key. Under the examples there is a very simple Python code. When you execute it, pycurl prints the server’s XML response to the standard output. How to store that in a variable? From that XML we want to extract some data.

Here is an extended version of the uploader script:

#!/usr/bin/env python

import pycurl
import cStringIO
import untangle    # XML parser

def upload_from_computer(image):
    response = cStringIO.StringIO()   # XML response is stored here
    
    c = pycurl.Curl()
    
    values = [
              ("key", your_api_key),
              ("image", (c.FORM_FILE, image))]
    # OR:     ("image", "http://example.com/example.jpg")]
    # OR:     ("image", "YOUR_BASE64_ENCODED_IMAGE_DATA")]
    
    c.setopt(c.URL, "http://api.imgur.com/2/upload.xml")
    c.setopt(c.HTTPPOST, values)
    c.setopt(c.WRITEFUNCTION, response.write)   # put the server's output in here
    c.perform()
    c.close()
    
    return response.getvalue()

def process(xml):
    o = untangle.parse(xml)
    url = o.upload.links.original.cdata
    delete_page = o.upload.links.delete_page.cdata
    
    print 'url:        ', url
    print 'delete page:', delete_page

#############################################################################

if __name__ == "__main__":
    img = '/tmp/something.jpg'
    xml = upload_from_computer(img)
    process(xml)

The tip for storing the XML output in a variable is from here. Untangle is a lightweight XML parser; more info here.

Send a post to reddit from Python

October 30, 2011 1 comment

Problem
How to send a post to reddit.com from a Python script? Motivation: when you send a post, you have to wait 8 minutes before you could send the next one. Imagine you have 10 posts to submit. It’d be nice to launch a script at night which would send everything till next morning.

Submit a post
Now I only show how to send one post. Batch processing is left as a future project.

The official Reddit API is here. There is a wrapper for it called reddit_api, which greatly simplifies its usage.

Install reddit_api:

sudo pip install reddit

Submit a post:

#!/usr/bin/env python

import reddit

subreddit = '...' # name of the subreddit where to send the post
url = '...'       # what you want to send
title = '...'     # title of your post

# change user_agent if you want:
r = reddit.Reddit(user_agent="my_cool_application")
# your username and password on reddit:
r.login(user="...", password="...")

# the output is a JSON text that contains the link to your post:
print r.submit(subreddit, url, title)

Submit a comment (update, 20111107)
Let’s see how to add a comment to a post. First, we need the URL of a post.

Example: http://www.reddit.com/r/thewalkingdead/comments/lkycy/that_look_on_the_kids_face/. Here, the last part of the URL is just garbage, the following URL is equivalent with it: http://www.reddit.com/r/thewalkingdead/comments/lkycy. The unique ID of the post is the last part: “lkycy”. Thus, this image can be accessed via this URL too: http://www.reddit.com/lkycy.

Now, let’s log in to reddit, fetch the post by its ID and add a comment.

def get_reddit_id(url):
    result = re.search('/comments/(.*?)/', url)
    return result.group(1)

def add_comment(r, reddit_url):
    reddit_id = get_reddit_id(reddit_url)
    post = r.get_submission_by_id(reddit_id)
    comment = "first"   # just to make reddit happy ;)
    post.add_comment(comment)
    print '# comment added:', comment

def main():
    r = reddit.Reddit(user_agent="my_cool_application")
    r.login(user="...", password="...")
    reddit_url = ...
    add_comment(r, reddit_url)
Follow

Get every new post delivered to your Inbox.

Join 72 other followers