From a MongoDB collection, you want to get a random document.
import random def get_random_doc(): # coll refers to your collection count = coll.count() return coll.find()[random.randrange(count)]
Pymongo documentation on cursors: here.
I have a project where the input JSON file is almost 7 MB. I keep this project in my Dropbox folder, so that 7 MB text file seems to be a waste. Any way to reduce its size?
I zipped it up with gzip: “
gzip -9 input.json“. This command produced a 1.3 MB “input.json.gz” file and deleted the original. Good. But how to open it in Python?
Normal way (without gzip):
import json with open("input.json") as f: d = json.load(f)
Compressed way (with gzip):
import json import gzip with gzip.open("input.json.gz", "rb") as f: d = json.loads(f.read().decode("ascii"))
I didn’t notice any performance penalty. The application that first reads this json file starts as fast as before.
For creating virtual environmets, I’ve used
virtualenvwrapper so far. However, Python 3.4 contains the command
pyvenv that does the same thing. Since it also installs
pip in the virt. env., it can replace
I like to store my virtual environments in a dedicated folder, separated from the project directory. virtualenvwrapper, by default, stores the virt. env.’s in the
~/.virtualenvs folder. Since I got used to this folder, I will continue to keep my virt. env.’s in this folder.
Say we have our project folder here:
~/python/webapps/flasky_project. Create a virt. env. for this the following way:
It will create a Python 3 virt. env.
virtualenv / virtualenvwrapper
For the sake of completeness, I also write here how to create virt. env.’s with virtualenv and virtualenvwrapper:
# blog post: http://goo.gl/oEdtT3 # virtualenvwrapper for Python 3 or Python 2 mkvirtualenv -p `which python3` myenv3 mkvirtualenv -p `which python2` myenv2 # virtualenv for Python 3 or Python 2 virtualenv -p python3 myproject3 virtualenv -p python2 myproject2 # When the env. is created, activate it # and launch the command python within. # Verify if it's the correct version.
If you use Python 3.4+ and you need a virt. env., use the command “
The WordPress.com stats helper monkeys prepared a 2014 annual report for this blog.
Here's an excerpt:
The Louvre Museum has 8.5 million visitors per year. This blog was viewed about 190,000 times in 2014. If it were an exhibit at the Louvre Museum, it would take about 8 days for that many people to see it.
You have an XML file and you want to convert it to dict or JSON.
Well, if you have a dict, you can convert it to JSON with “
json.dump()“, so the real question is: how to convert an XML file to a dictionary?
There is an excellent library for this purpose called xmltodict. Its usage is very simple:
import xmltodict # It doesn't work with Python 3! Read on for the solution! def convert(xml_file, xml_attribs=True): with open(xml_file) as f: d = xmltodict.parse(f, xml_attribs=xml_attribs) return d
This worked well under Python 2.7 but I got an error under Python 3. I checked the project’s documentation and it claimed to be Python 3 compatible. What the hell?
The error message was this:
Traceback (most recent call last): File "/home/jabba/Dropbox/python/lib/jabbapylib2/apps/xmltodict.py", line 247, in parse parser.ParseFile(xml_input) TypeError: read() did not return a bytes object (type=str) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "./xml2json.py", line 27, in <module> print(convert(sys.argv)) File "./xml2json.py", line 17, in convert d = xmltodict.parse(f, xml_attribs=xml_attribs) File "/home/jabba/Dropbox/python/lib/jabbapylib2/apps/xmltodict.py", line 249, in parse parser.Parse(xml_input, True) TypeError: '_io.TextIOWrapper' does not support the buffer interface
I even filed an issue ticket :)
After some debugging I found a hint here: you need to open the XML file in binary mode!
XML to dict (Python 2 & 3)
So the correct version that works with Python 3 too is this:
import xmltodict def convert(xml_file, xml_attribs=True): with open(xml_file, "rb") as f: # notice the "rb" mode d = xmltodict.parse(f, xml_attribs=xml_attribs) return d
XML to JSON (Python 2 & 3)
If you want JSON output:
import json import xmltodict def convert(xml_file, xml_attribs=True): with open(xml_file, "rb") as f: # notice the "rb" mode d = xmltodict.parse(f, xml_attribs=xml_attribs) return json.dumps(d, indent=4)